2024-04-23T14:12:28.869 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T14:12:28.869 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --name rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576 --verbose --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/orig.config.yaml 2024-04-23T14:12:28.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T14:12:29.007 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576 branch: wip-rishabh-testing-20240416.193735-3 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: null first_in_suite: false job_id: '7669576' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735-3 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735-3 sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpnB0LGa4cS+W36Aj/as5sETZJ8Aj1oZaMwoiqsqf7kljO7nt46i/ouz7jwY3Pr3lLkNOX2bweR2pmXsxAiJoE= smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBByGZNsW/Qlm58n8wjoKYEesFXpNRdCXRv8uJXOaipMKNEMW9KHWFUXl4pdf8XasVMXR7jLR//U/s+yzg9lWBNI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_14:04:26 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T14:12:29.007 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa; will attempt to use it 2024-04-23T14:12:29.008 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks 2024-04-23T14:12:29.008 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T14:12:29.009 INFO:teuthology.task.internal:Checking packages... 2024-04-23T14:12:29.031 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd44fddd910b5e94da0f2357af4be37d7bd495ae0' 2024-04-23T14:12:29.031 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T14:12:29.031 INFO:teuthology.packaging:ref: None 2024-04-23T14:12:29.032 INFO:teuthology.packaging:tag: None 2024-04-23T14:12:29.032 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735-3 2024-04-23T14:12:29.032 INFO:teuthology.packaging:sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:12:29.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735-3 2024-04-23T14:12:29.170 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3126.gd44fddd9 2024-04-23T14:12:29.172 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T14:12:29.179 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T14:12:29.179 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T14:12:29.192 INFO:teuthology.task.internal:Saving configuration 2024-04-23T14:12:29.208 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T14:12:29.215 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T14:12:29.240 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:38.763255', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpnB0LGa4cS+W36Aj/as5sETZJ8Aj1oZaMwoiqsqf7kljO7nt46i/ouz7jwY3Pr3lLkNOX2bweR2pmXsxAiJoE='} 2024-04-23T14:12:29.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:38.761876', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBByGZNsW/Qlm58n8wjoKYEesFXpNRdCXRv8uJXOaipMKNEMW9KHWFUXl4pdf8XasVMXR7jLR//U/s+yzg9lWBNI='} 2024-04-23T14:12:29.258 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T14:12:29.264 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-23T14:12:29.264 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-23T14:12:29.265 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T14:12:29.333 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa34843e160>, signals=[15]) 2024-04-23T14:12:29.333 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T14:12:29.340 INFO:teuthology.task.internal:Opening connections... 2024-04-23T14:12:29.340 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:12:29.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:12:29.419 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:12:29.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:12:29.492 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T14:12:29.498 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-04-23T14:12:29.517 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-04-23T14:12:29.517 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-04-23T14:12:29.575 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-04-23T14:12:29.575 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-04-23T14:12:29.575 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T14:12:29.576 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T14:12:29.577 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-04-23T14:12:29.605 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-04-23T14:12:29.622 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-04-23T14:12:29.622 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T14:12:29.679 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-04-23T14:12:29.680 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-04-23T14:12:29.680 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T14:12:29.680 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-04-23T14:12:29.680 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T14:12:29.680 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T14:12:29.680 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T14:12:29.680 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-04-23T14:12:29.704 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T14:12:29.714 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T14:12:29.719 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T14:12:29.719 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-04-23T14:12:29.722 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-04-23T14:12:29.737 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T14:12:29.743 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T14:12:29.743 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-04-23T14:12:29.779 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-04-23T14:12:30.009 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T14:12:30.061 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T14:12:30.075 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-23T14:12:30.075 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-23T14:12:30.076 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T14:12:30.076 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T14:12:30.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:12:30.092 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:12:30.093 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-23T14:12:30.093 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-04-23T14:12:30.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:12:30.094 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:12:30.094 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-23T14:12:30.094 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-04-23T14:12:30.148 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-437.el9.x86_64 2024-04-23T14:12:30.149 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:12:30.149 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-23T14:12:30.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:12:30.153 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-437.el9.x86_64 2024-04-23T14:12:30.153 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:12:30.153 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-23T14:12:30.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:12:30.299 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum upgrade -y linux-firmware 2024-04-23T14:12:30.300 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum upgrade -y linux-firmware 2024-04-23T14:12:33.391 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-04-23T14:12:35.150 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-23T14:12:35.218 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-04-23T14:12:46.209 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 851 kB/s | 6.2 MB 00:07 2024-04-23T14:12:47.852 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-23T14:12:49.523 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-23T14:12:50.926 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-04-23T14:12:54.437 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-23T14:12:55.219 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-23T14:12:56.363 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-23T14:12:56.364 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-04-23T14:12:56.364 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-04-23T14:12:56.496 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-23T14:12:56.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:12:56.735 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-23T14:12:56.735 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-23T14:12:56.738 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi076.front.sepia.ceph.com to complete... 2024-04-23T14:12:57.312 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 16 kB 00:00 2024-04-23T14:12:58.263 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-23T14:13:03.446 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-23T14:13:05.456 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-04-23T14:13:05.457 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-04-23T14:13:05.458 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-04-23T14:13:05.568 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-23T14:13:05.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:13:05.827 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-23T14:13:05.828 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-23T14:13:05.830 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi059.front.sepia.ceph.com to complete... 2024-04-23T14:16:34.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23 14:16:34 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-23T14:16:34.653 INFO:teuthology.orchestra.run.smithi059.stderr:FINISHED --2024-04-23 14:16:34-- 2024-04-23T14:16:34.653 INFO:teuthology.orchestra.run.smithi059.stderr:Total wall clock time: 3m 29s 2024-04-23T14:16:34.653 INFO:teuthology.orchestra.run.smithi059.stderr:Downloaded: 1 files, 341M in 3m 28s (1.64 MB/s) 2024-04-23T14:16:34.657 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-23T14:16:34.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-23T14:16:36.855 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying... ######################################## 2024-04-23T14:16:36.992 INFO:teuthology.orchestra.run.smithi059.stdout:Preparing... ######################################## 2024-04-23T14:16:37.119 INFO:teuthology.orchestra.run.smithi059.stdout:Updating / installing... 2024-04-23T14:16:45.410 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-23T14:17:23.115 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-23T14:17:23.116 DEBUG:teuthology.orchestra.run.smithi059:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-23T14:17:23.206 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-23T14:17:23.207 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-23T14:17:23.208 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-23T14:17:23.209 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-23T14:17:23.210 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-23T14:17:23.211 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-23T14:17:23.212 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-23T14:17:23.213 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-23T14:17:23.214 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-23T14:17:23.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-23T14:17:23.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-23T14:17:23.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-23T14:17:23.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-23T14:17:23.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-23T14:17:23.215 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-23T14:17:23.216 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-23T14:17:23.217 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-23T14:17:23.218 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-23T14:17:23.219 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-23T14:17:23.220 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-23T14:17:23.221 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-23T14:17:23.222 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-23T14:17:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-23T14:17:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-23T14:17:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-23T14:17:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-23T14:17:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-23T14:17:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-23T14:17:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-23T14:17:23.224 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-23T14:17:23.225 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-23T14:17:23.226 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-23T14:17:23.227 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-23T14:17:23.228 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-23T14:17:23.229 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-23T14:17:23.230 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-23T14:17:23.231 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-23T14:17:23.232 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-23T14:17:23.233 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-23T14:17:23.234 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-23T14:17:23.235 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-23T14:17:23.236 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-23T14:17:23.237 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-23T14:17:23.238 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-23T14:17:23.239 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-23T14:17:23.240 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-23T14:17:23.241 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-23T14:17:23.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-23T14:17:23.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-23T14:17:23.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-23T14:17:23.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-23T14:17:23.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-23T14:17:23.242 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-23T14:17:23.243 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-23T14:17:23.244 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-23T14:17:23.245 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-23T14:17:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-23T14:17:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-23T14:17:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-23T14:17:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-23T14:17:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-23T14:17:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-23T14:17:23.246 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-23T14:17:23.247 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-23T14:17:23.248 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-23T14:17:23.249 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-23T14:17:23.250 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-23T14:17:23.251 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-23T14:17:23.252 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-23T14:17:23.253 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-23T14:17:23.254 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-23T14:17:23.255 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-23T14:17:23.256 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-23T14:17:23.257 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-23T14:17:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-23T14:17:23.259 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-23T14:17:23.260 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-23T14:17:23.261 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-23T14:17:23.262 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-23T14:17:23.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-23T14:17:23.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-23T14:17:23.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-23T14:17:23.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-23T14:17:23.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-23T14:17:23.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-23T14:17:23.263 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-23T14:17:23.264 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-23T14:17:23.265 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-23T14:17:23.266 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-23T14:17:23.267 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-23T14:17:23.268 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-23T14:17:23.269 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-23T14:17:23.270 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-23T14:17:23.271 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-23T14:17:23.272 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-23T14:17:23.273 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-23T14:17:23.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-23T14:17:23.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-23T14:17:23.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-23T14:17:23.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-23T14:17:23.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-23T14:17:23.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-23T14:17:23.274 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-23T14:17:23.275 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-23T14:17:23.276 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-23T14:17:23.277 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-23T14:17:23.278 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-23T14:17:23.279 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-23T14:17:23.280 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-23T14:17:23.281 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-23T14:17:23.282 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-23T14:17:23.283 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-23T14:17:23.284 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-23T14:17:23.285 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-23T14:17:23.286 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-23T14:17:23.287 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-23T14:17:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-23T14:17:23.289 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-23T14:17:23.290 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-23T14:17:23.291 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-23T14:17:23.292 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-23T14:17:23.293 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-23T14:17:23.294 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-23T14:17:23.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-23T14:17:23.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-23T14:17:23.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-23T14:17:23.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-23T14:17:23.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-23T14:17:23.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-23T14:17:23.295 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-23T14:17:23.296 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-23T14:17:23.297 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-23T14:17:23.298 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-23T14:17:23.299 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-23T14:17:23.300 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-23T14:17:23.301 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-23T14:17:23.302 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-23T14:17:23.303 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-23T14:17:23.304 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-23T14:17:23.305 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-23T14:17:23.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-23T14:17:23.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-23T14:17:23.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-23T14:17:23.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-23T14:17:23.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-23T14:17:23.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-23T14:17:23.306 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-23T14:17:23.307 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-23T14:17:23.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-23T14:17:23.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-23T14:17:23.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-23T14:17:23.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-23T14:17:23.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-23T14:17:23.308 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-23T14:17:23.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-23T14:17:23.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-23T14:17:23.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-23T14:17:23.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-23T14:17:23.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-23T14:17:23.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-23T14:17:23.309 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-23T14:17:23.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-23T14:17:23.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-23T14:17:23.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-23T14:17:23.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-23T14:17:23.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-23T14:17:23.310 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-23T14:17:23.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-23T14:17:23.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-23T14:17:23.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-23T14:17:23.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-23T14:17:23.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-23T14:17:23.311 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-23T14:17:23.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-23T14:17:23.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-23T14:17:23.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-23T14:17:23.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-23T14:17:23.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-23T14:17:23.312 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-23T14:17:23.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-23T14:17:23.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-23T14:17:23.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-23T14:17:23.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-23T14:17:23.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-23T14:17:23.313 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-23T14:17:23.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-23T14:17:23.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-23T14:17:23.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-23T14:17:23.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-23T14:17:23.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-23T14:17:23.314 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-23T14:17:23.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-23T14:17:23.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-23T14:17:23.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-23T14:17:23.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-23T14:17:23.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-23T14:17:23.315 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-23T14:17:23.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-23T14:17:23.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-23T14:17:23.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-23T14:17:23.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-23T14:17:23.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-23T14:17:23.316 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-23T14:17:23.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-23T14:17:23.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-23T14:17:23.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-23T14:17:23.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-23T14:17:23.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-23T14:17:23.317 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-23T14:17:23.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-23T14:17:23.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-23T14:17:23.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-23T14:17:23.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-23T14:17:23.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-23T14:17:23.318 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-23T14:17:23.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-23T14:17:23.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-23T14:17:23.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-23T14:17:23.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-23T14:17:23.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-23T14:17:23.319 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-23T14:17:23.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-23T14:17:23.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-23T14:17:23.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-23T14:17:23.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-23T14:17:23.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-23T14:17:23.320 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-23T14:17:23.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-23T14:17:23.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-23T14:17:23.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-23T14:17:23.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-23T14:17:23.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-23T14:17:23.321 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-23T14:17:23.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-23T14:17:23.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-23T14:17:23.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-23T14:17:23.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-23T14:17:23.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-23T14:17:23.322 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-23T14:17:23.323 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-23T14:17:23.324 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-23T14:17:23.325 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-23T14:17:23.326 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-23T14:17:23.327 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-23T14:17:23.328 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-23T14:17:23.329 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-23T14:17:23.330 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-23T14:17:23.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-23T14:17:23.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-23T14:17:23.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-23T14:17:23.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-23T14:17:23.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-23T14:17:23.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-23T14:17:23.331 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-23T14:17:23.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-23T14:17:23.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-23T14:17:23.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-23T14:17:23.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-23T14:17:23.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-23T14:17:23.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-23T14:17:23.332 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-23T14:17:23.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-23T14:17:23.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-23T14:17:23.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-23T14:17:23.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-23T14:17:23.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-23T14:17:23.333 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-23T14:17:23.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-23T14:17:23.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-23T14:17:23.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-23T14:17:23.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-23T14:17:23.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-23T14:17:23.334 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-23T14:17:23.335 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-23T14:17:23.336 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-23T14:17:23.337 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-23T14:17:23.338 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-23T14:17:23.339 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-23T14:17:23.340 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-23T14:17:23.341 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-23T14:17:23.342 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-23T14:17:23.343 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-23T14:17:23.344 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-23T14:17:23.345 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-23T14:17:23.346 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-23T14:17:23.347 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-23T14:17:23.348 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-23T14:17:23.349 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-23T14:17:23.350 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-23T14:17:23.351 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-23T14:17:23.352 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-23T14:17:23.353 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-23T14:17:23.354 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-23T14:17:23.355 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-23T14:17:23.356 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-23T14:17:23.357 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-23T14:17:23.358 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-23T14:17:23.359 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-23T14:17:23.360 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-23T14:17:23.361 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-23T14:17:23.362 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-23T14:17:23.363 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-23T14:17:23.364 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-23T14:17:23.365 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-23T14:17:23.366 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-23T14:17:23.367 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-23T14:17:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-23T14:17:23.369 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-23T14:17:23.370 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-23T14:17:23.371 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-23T14:17:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-23T14:17:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-23T14:17:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-23T14:17:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-23T14:17:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-23T14:17:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-23T14:17:23.372 INFO:teuthology.orchestra.run.smithi059.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-23T14:17:23.376 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:23.376 DEBUG:teuthology.orchestra.run.smithi059:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-23T14:17:23.402 INFO:teuthology.orchestra.run.smithi059.stdout:postinstall scriptlet (using /bin/sh): 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout:for file in vmlinuz System.map config; do 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: fi 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout:done 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:17:23.403 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-23T14:17:23.404 INFO:teuthology.orchestra.run.smithi059.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-23T14:17:23.404 INFO:teuthology.orchestra.run.smithi059.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:23.404 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-23T14:17:23.404 INFO:teuthology.orchestra.run.smithi059.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-23T14:17:23.404 INFO:teuthology.orchestra.run.smithi059.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-23T14:17:23.404 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:23.404 INFO:teuthology.orchestra.run.smithi059.stdout:fi 2024-04-23T14:17:23.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2-tools 2024-04-23T14:17:23.483 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2-tools 2024-04-23T14:17:23.483 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2024-04-23T14:17:23.483 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.06 2024-04-23T14:17:23.483 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 68.el9 2024-04-23T14:17:23.483 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Group : Unspecified 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 8264254 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : builder@centos.org 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Support tools for GRUB. 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:17:23.484 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-23T14:17:23.485 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-23T14:17:23.485 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-23T14:17:23.485 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2024-04-23T14:17:23.485 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:17:23.485 INFO:teuthology.orchestra.run.smithi059.stdout:This subpackage provides tools for support of all platforms. 2024-04-23T14:17:23.486 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-23T14:17:23.486 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:23.486 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-23T14:17:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2024-04-23T14:17:25.520 INFO:teuthology.orchestra.run.smithi059.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-23T14:17:25.555 INFO:teuthology.orchestra.run.smithi059.stderr:done 2024-04-23T14:17:25.557 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-04-23T14:17:25.573 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.LNYlzxbESC 2024-04-23T14:17:25.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LNYlzxbESC 2024-04-23T14:17:25.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.LNYlzxbESC 2024-04-23T14:17:25.782 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.LNYlzxbESC is 6KB 2024-04-23T14:17:25.831 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.LNYlzxbESC 2024-04-23T14:17:25.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo /bin/ls /boot/loader/entries || true 2024-04-23T14:17:25.908 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-23T14:17:25.908 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-23T14:17:25.908 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-23T14:17:25.908 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-23T14:17:25.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-23T14:17:26.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2024-04-23T14:17:56.059 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-23T14:17:56.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-23T14:17:56.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:18:14.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-04-23T14:18:23.431 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-23T14:18:23.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:01.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23 14:19:01 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-23T14:19:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:FINISHED --2024-04-23 14:19:01-- 2024-04-23T14:19:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:Total wall clock time: 6m 5s 2024-04-23T14:19:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:Downloaded: 1 files, 341M in 6m 3s (962 KB/s) 2024-04-23T14:19:01.717 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-23T14:19:01.717 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-23T14:19:03.853 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying... ######################################## 2024-04-23T14:19:04.063 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing... ######################################## 2024-04-23T14:19:04.168 INFO:teuthology.orchestra.run.smithi076.stdout:Updating / installing... 2024-04-23T14:19:12.224 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-23T14:19:23.435 DEBUG:teuthology.orchestra.remote:timed out 2024-04-23T14:19:35.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-23T14:19:35.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:19:35.869 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-04-23T14:19:36.493 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-23T14:19:36.493 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-23T14:19:36.494 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-04-23T14:19:36.550 INFO:teuthology.orchestra.run.smithi059.stdout:6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:36.551 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:36.551 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-23T14:19:36.551 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-23T14:19:37.551 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-23T14:19:37.552 DEBUG:teuthology.orchestra.run.smithi059:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T14:19:37.680 INFO:teuthology.orchestra.run.smithi059.stdout:ttyS1 2024-04-23T14:19:37.743 DEBUG:teuthology.parallel:result is None 2024-04-23T14:19:49.858 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-23T14:19:49.858 DEBUG:teuthology.orchestra.run.smithi076:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-23T14:19:49.944 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-23T14:19:49.945 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-23T14:19:49.946 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-23T14:19:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-23T14:19:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-23T14:19:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-23T14:19:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-23T14:19:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-23T14:19:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-23T14:19:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-23T14:19:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-23T14:19:49.955 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-23T14:19:49.956 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-23T14:19:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-23T14:19:49.958 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-23T14:19:49.959 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-23T14:19:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-23T14:19:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-23T14:19:49.962 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-23T14:19:49.963 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-23T14:19:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-23T14:19:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-23T14:19:49.966 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-23T14:19:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-23T14:19:49.968 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-23T14:19:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-23T14:19:49.970 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-23T14:19:49.971 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-23T14:19:49.972 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-23T14:19:49.973 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-23T14:19:49.974 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-23T14:19:49.975 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-23T14:19:49.976 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-23T14:19:49.977 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-23T14:19:49.978 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-23T14:19:49.979 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-23T14:19:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-23T14:19:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-23T14:19:49.982 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-23T14:19:49.983 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-23T14:19:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-23T14:19:49.985 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-23T14:19:49.986 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-23T14:19:49.987 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-23T14:19:49.988 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-23T14:19:49.989 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-23T14:19:49.990 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-23T14:19:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-23T14:19:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-23T14:19:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-23T14:19:49.994 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-23T14:19:49.995 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-23T14:19:49.996 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-23T14:19:49.997 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-23T14:19:49.998 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-23T14:19:49.999 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-23T14:19:50.000 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-23T14:19:50.001 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-23T14:19:50.002 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-23T14:19:50.003 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-23T14:19:50.004 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-23T14:19:50.005 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-23T14:19:50.006 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-23T14:19:50.007 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-23T14:19:50.008 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-23T14:19:50.009 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-23T14:19:50.010 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-23T14:19:50.011 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-23T14:19:50.012 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-23T14:19:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-23T14:19:50.014 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-23T14:19:50.015 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-23T14:19:50.016 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-23T14:19:50.017 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-23T14:19:50.018 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-23T14:19:50.019 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-23T14:19:50.020 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-23T14:19:50.021 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-23T14:19:50.022 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-23T14:19:50.023 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-23T14:19:50.024 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-23T14:19:50.025 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-23T14:19:50.026 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-23T14:19:50.027 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-23T14:19:50.028 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-23T14:19:50.029 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-23T14:19:50.030 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-23T14:19:50.031 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-23T14:19:50.032 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-23T14:19:50.033 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-23T14:19:50.034 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-23T14:19:50.035 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-23T14:19:50.036 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-23T14:19:50.037 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-23T14:19:50.038 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-23T14:19:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-23T14:19:50.040 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-23T14:19:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-23T14:19:50.042 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-23T14:19:50.043 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-23T14:19:50.044 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-23T14:19:50.045 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-23T14:19:50.046 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-23T14:19:50.047 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-23T14:19:50.048 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-23T14:19:50.049 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-23T14:19:50.050 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-23T14:19:50.051 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-23T14:19:50.052 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-23T14:19:50.053 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-23T14:19:50.054 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-23T14:19:50.055 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-23T14:19:50.056 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-23T14:19:50.057 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-23T14:19:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-23T14:19:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-23T14:19:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-23T14:19:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-23T14:19:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-23T14:19:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-23T14:19:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-23T14:19:50.065 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-23T14:19:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-23T14:19:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-23T14:19:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-23T14:19:50.069 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-23T14:19:50.070 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-23T14:19:50.071 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-23T14:19:50.072 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-23T14:19:50.073 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-23T14:19:50.074 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-23T14:19:50.075 INFO:teuthology.orchestra.run.smithi076.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-23T14:19:50.079 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:50.079 DEBUG:teuthology.orchestra.run.smithi076:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout:postinstall scriptlet (using /bin/sh): 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout:for file in vmlinuz System.map config; do 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout: fi 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout:done 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-23T14:19:50.104 INFO:teuthology.orchestra.run.smithi076.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:50.105 INFO:teuthology.orchestra.run.smithi076.stdout:fi 2024-04-23T14:19:50.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 68.el9 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2024-04-23T14:19:50.181 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8264254 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-23T14:19:50.182 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2024-04-23T14:19:50.183 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:19:50.183 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2024-04-23T14:19:50.183 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-23T14:19:50.184 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:50.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-23T14:19:50.776 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2024-04-23T14:19:52.084 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-23T14:19:52.113 INFO:teuthology.orchestra.run.smithi076.stderr:done 2024-04-23T14:19:52.115 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-04-23T14:19:52.130 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.ZpF8rVeAiK 2024-04-23T14:19:52.130 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZpF8rVeAiK 2024-04-23T14:19:52.206 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.ZpF8rVeAiK 2024-04-23T14:19:52.346 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.ZpF8rVeAiK is 6KB 2024-04-23T14:19:52.395 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.ZpF8rVeAiK 2024-04-23T14:19:52.409 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2024-04-23T14:19:52.470 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-23T14:19:52.471 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-23T14:19:52.471 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-23T14:19:52.471 INFO:teuthology.orchestra.run.smithi076.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-23T14:19:52.472 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-23T14:19:52.616 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2024-04-23T14:20:22.620 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-23T14:20:22.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-23T14:20:22.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:20:41.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-04-23T14:20:50.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-23T14:20:50.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:20:53.186 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-04-23T14:21:05.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-23T14:21:05.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:22:05.191 DEBUG:teuthology.orchestra.remote:timed out 2024-04-23T14:22:20.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-23T14:22:20.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:22:20.638 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-04-23T14:22:21.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-23T14:22:21.256 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-23T14:22:21.256 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-04-23T14:22:21.307 INFO:teuthology.orchestra.run.smithi076.stdout:6.9.0-rc3-g806ec36b2112 2024-04-23T14:22:21.307 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-23T14:22:21.307 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-23T14:22:21.307 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-23T14:22:22.308 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-23T14:22:22.309 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T14:22:22.432 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2024-04-23T14:22:22.490 DEBUG:teuthology.parallel:result is None 2024-04-23T14:22:22.490 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T14:22:22.500 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T14:22:22.500 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T14:22:22.509 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T14:22:22.567 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T14:22:22.579 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T14:22:22.596 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T14:22:22.597 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T14:22:22.602 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T14:22:22.674 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T14:22:22.687 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T14:22:22.687 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T14:22:22.695 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T14:22:22.788 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:22:22.789 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:22:22.820 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:22:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:22:22.826 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T14:22:22.837 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T14:22:22.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T14:22:22.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T14:22:22.929 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T14:22:22.943 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T14:22:22.943 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T14:22:22.951 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T14:22:23.007 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:22:23.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:22:23.209 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:22:23.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:22:23.389 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:22:23.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T14:22:23.478 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:22:23.567 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:22:23.672 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:22:23.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:22:23.841 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:22:23.841 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T14:22:23.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-04-23T14:22:23.924 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-04-23T14:22:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T14:22:24.057 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T14:22:24.474 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T14:22:24.486 INFO:teuthology.task.internal:Starting timer... 2024-04-23T14:22:24.486 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T14:22:24.506 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T14:22:24.521 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-23T14:22:24.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-04-23T14:22:24.671 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-04-23T14:22:24.677 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-04-23T14:22:24.809 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-04-23T14:22:24.813 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T14:22:24.813 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-04-23T14:22:24.870 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-04-23T14:22:24.870 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-04-23T14:22:24.921 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-04-23T14:22:24.921 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi076.front.sepia.ceph.com': 'permissive'} 2024-04-23T14:22:24.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T14:22:25.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:22:25.009 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T14:22:25.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:22:25.086 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T14:22:25.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-04-23T14:22:25.155 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-04-23T14:22:25.235 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T14:22:25.248 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T14:22:25.266 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T14:22:25.267 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi076.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T14:27:47.937 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi076.front.sepia.ceph.com')] 2024-04-23T14:27:47.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-23T14:27:47.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:27:48.057 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-04-23T14:27:48.215 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-04-23T14:27:48.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-23T14:27:48.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:27:48.349 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-04-23T14:27:48.488 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-23T14:27:48.488 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T14:27:48.507 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T14:27:48.508 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T14:27:48.508 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:27:48.519 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T14:27:48.519 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:27:48.612 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T14:27:48.618 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T14:27:48.654 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T14:27:48.663 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T14:27:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-04-23T14:27:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-04-23T14:27:48.757 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-04-23T14:27:48.772 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-04-23T14:27:48.793 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T14:27:48.816 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T14:27:48.834 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T14:27:48.860 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T14:27:48.971 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-04-23T14:27:48.978 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T14:27:48.978 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-04-23T14:27:48.978 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:48.978 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:48.978 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:48.978 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-04-23T14:27:49.005 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T14:27:49.005 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-04-23T14:27:49.005 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:49.005 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:49.005 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:49.005 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:27:49.007 INFO:teuthology.run_tasks:Running task install... 2024-04-23T14:27:49.019 DEBUG:teuthology.task.install:project ceph 2024-04-23T14:27:49.019 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}} 2024-04-23T14:27:49.020 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'} 2024-04-23T14:27:49.020 INFO:teuthology.task.install:Using flavor: default 2024-04-23T14:27:49.026 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T14:27:49.026 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T14:27:49.026 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'tag': None, 'wait_for_package': False} 2024-04-23T14:27:49.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:27:49.030 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'tag': None, 'wait_for_package': False} 2024-04-23T14:27:49.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:27:49.345 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/ 2024-04-23T14:27:49.345 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3126.gd44fddd9 2024-04-23T14:27:49.348 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/ 2024-04-23T14:27:49.349 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3126.gd44fddd9 2024-04-23T14:27:49.468 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T14:27:49.469 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:27:49.469 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T14:27:49.560 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T14:27:49.560 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d44fddd910b5e94da0f2357af4be37d7bd495ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T14:27:49.607 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T14:27:49.608 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:27:49.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T14:27:49.690 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T14:27:49.702 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T14:27:49.702 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d44fddd910b5e94da0f2357af4be37d7bd495ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T14:27:49.803 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T14:27:49.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T14:27:49.882 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-04-23T14:27:49.886 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-04-23T14:27:49.957 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T14:27:50.039 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-04-23T14:27:50.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-04-23T14:27:50.205 INFO:teuthology.orchestra.run.smithi076.stdout:48 files removed 2024-04-23T14:27:50.244 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T14:27:50.379 INFO:teuthology.orchestra.run.smithi059.stdout:48 files removed 2024-04-23T14:27:50.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T14:27:51.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 161 kB/s | 83 kB 00:00 2024-04-23T14:27:51.401 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 157 kB/s | 83 kB 00:00 2024-04-23T14:27:51.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-23T14:27:52.093 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 20 kB/s | 12 kB 00:00 2024-04-23T14:27:52.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 3.1 kB/s | 1.9 kB 00:00 2024-04-23T14:27:52.877 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-23T14:27:56.108 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-04-23T14:27:58.142 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-23T14:28:01.922 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 882 kB/s | 8.0 MB 00:09 2024-04-23T14:28:03.959 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-23T14:28:12.018 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 641 kB/s | 6.2 MB 00:09 2024-04-23T14:28:13.948 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-23T14:28:15.795 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-23T14:28:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 757 kB/s | 6.2 MB 00:08 2024-04-23T14:28:18.094 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-23T14:28:19.854 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-23T14:28:21.878 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-23T14:28:24.357 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:28:24.358 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:28:24.379 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T14:28:24.380 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T14:28:24.382 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T14:28:24.383 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T14:28:24.386 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T14:28:24.389 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T14:28:24.390 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T14:28:24.466 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-04-23T14:28:24.487 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-04-23T14:28:24.487 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-04-23T14:28:24.487 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-04-23T14:28:24.487 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-04-23T14:28:24.487 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 6.6 k 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 891 k 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 145 k 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 1.5 M 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 142 k 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 3.5 M 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 7.4 M 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 49 k 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 11 M 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 49 M 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 273 k 2024-04-23T14:28:24.488 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 762 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 225 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 27 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 18 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 32 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 746 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 127 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T14:28:24.489 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 161 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 320 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 297 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 100 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 85 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 171 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.4 M 2024-04-23T14:28:24.490 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:28:24.491 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T14:28:24.491 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T14:28:24.491 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-04-23T14:28:24.491 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T14:28:24.491 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 21 M 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 34 k 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 2.4 M 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 249 k 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 4.7 M 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 17 M 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 16 k 2024-04-23T14:28:24.492 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 25 k 2024-04-23T14:28:24.493 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T14:28:24.493 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T14:28:24.493 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T14:28:24.493 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T14:28:24.493 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T14:28:24.493 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 163 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 500 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:28:24.494 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T14:28:24.495 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 45 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 135 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T14:28:24.496 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T14:28:24.497 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T14:28:24.498 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T14:28:24.499 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-04-23T14:28:24.500 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T14:28:24.501 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:28:24.501 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-04-23T14:28:24.501 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-04-23T14:28:24.501 INFO:teuthology.orchestra.run.smithi059.stdout:Install 148 Packages 2024-04-23T14:28:24.501 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 4 Packages 2024-04-23T14:28:24.501 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:28:24.502 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 222 M 2024-04-23T14:28:24.502 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-04-23T14:28:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-04-23T14:28:26.138 INFO:teuthology.orchestra.run.smithi059.stdout:(1/152): ceph-19.0.0-3126.gd44fddd9.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-23T14:28:27.116 INFO:teuthology.orchestra.run.smithi059.stdout:(2/152): ceph-fuse-19.0.0-3126.gd44fddd9.el9.x8 913 kB/s | 891 kB 00:00 2024-04-23T14:28:27.314 INFO:teuthology.orchestra.run.smithi059.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 738 kB/s | 145 kB 00:00 2024-04-23T14:28:28.113 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:28:28.115 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:28:28.135 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T14:28:28.136 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T14:28:28.137 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T14:28:28.140 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T14:28:28.141 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T14:28:28.145 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T14:28:28.146 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T14:28:28.214 INFO:teuthology.orchestra.run.smithi059.stdout:(4/152): ceph-mds-19.0.0-3126.gd44fddd9.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-04-23T14:28:28.217 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 6.6 k 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 891 k 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 145 k 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 1.5 M 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 142 k 2024-04-23T14:28:28.239 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 3.5 M 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 7.4 M 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 49 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 11 M 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 49 M 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 273 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 762 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 225 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 27 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 18 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 32 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 746 k 2024-04-23T14:28:28.240 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 127 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 161 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 320 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 297 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 100 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 85 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 171 k 2024-04-23T14:28:28.241 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.4 M 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 21 M 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 34 k 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 2.4 M 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 249 k 2024-04-23T14:28:28.242 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 4.7 M 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 17 M 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 16 k 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 25 k 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T14:28:28.243 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 163 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 500 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T14:28:28.244 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 45 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 135 k 2024-04-23T14:28:28.245 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T14:28:28.246 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T14:28:28.247 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T14:28:28.248 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T14:28:28.249 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:28:28.250 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-04-23T14:28:28.251 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-23T14:28:28.251 INFO:teuthology.orchestra.run.smithi076.stdout:Install 148 Packages 2024-04-23T14:28:28.251 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 4 Packages 2024-04-23T14:28:28.251 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:28:28.251 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 222 M 2024-04-23T14:28:28.251 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-04-23T14:28:28.664 INFO:teuthology.orchestra.run.smithi059.stdout:(5/152): ceph-mgr-19.0.0-3126.gd44fddd9.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-23T14:28:29.240 INFO:teuthology.orchestra.run.smithi076.stdout:(1/152): ceph-19.0.0-3126.gd44fddd9.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-23T14:28:30.218 INFO:teuthology.orchestra.run.smithi059.stdout:(6/152): ceph-base-19.0.0-3126.gd44fddd9.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-23T14:28:30.531 INFO:teuthology.orchestra.run.smithi059.stdout:(7/152): ceph-mon-19.0.0-3126.gd44fddd9.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-04-23T14:28:31.515 INFO:teuthology.orchestra.run.smithi076.stdout:(2/152): ceph-fuse-19.0.0-3126.gd44fddd9.el9.x8 392 kB/s | 891 kB 00:02 2024-04-23T14:28:31.840 INFO:teuthology.orchestra.run.smithi076.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 447 kB/s | 145 kB 00:00 2024-04-23T14:28:32.474 INFO:teuthology.orchestra.run.smithi076.stdout:(4/152): ceph-base-19.0.0-3126.gd44fddd9.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-23T14:28:33.309 INFO:teuthology.orchestra.run.smithi076.stdout:(5/152): ceph-common-19.0.0-3126.gd44fddd9.el9. 4.6 MB/s | 21 MB 00:04 2024-04-23T14:28:33.374 INFO:teuthology.orchestra.run.smithi076.stdout:(6/152): ceph-mgr-19.0.0-3126.gd44fddd9.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-23T14:28:33.867 INFO:teuthology.orchestra.run.smithi076.stdout:(7/152): ceph-mon-19.0.0-3126.gd44fddd9.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-04-23T14:28:33.914 INFO:teuthology.orchestra.run.smithi059.stdout:(8/152): ceph-common-19.0.0-3126.gd44fddd9.el9. 2.6 MB/s | 21 MB 00:08 2024-04-23T14:28:34.114 INFO:teuthology.orchestra.run.smithi059.stdout:(9/152): ceph-radosgw-19.0.0-3126.gd44fddd9.el9 3.0 MB/s | 11 MB 00:03 2024-04-23T14:28:34.144 INFO:teuthology.orchestra.run.smithi059.stdout:(10/152): ceph-selinux-19.0.0-3126.gd44fddd9.el 112 kB/s | 25 kB 00:00 2024-04-23T14:28:34.274 INFO:teuthology.orchestra.run.smithi059.stdout:(11/152): cephfs-mirror-19.0.0-3126.gd44fddd9.e 1.7 MB/s | 225 kB 00:00 2024-04-23T14:28:34.349 INFO:teuthology.orchestra.run.smithi059.stdout:(12/152): cephfs-shell-19.0.0-3126.gd44fddd9.el 364 kB/s | 27 kB 00:00 2024-04-23T14:28:34.424 INFO:teuthology.orchestra.run.smithi059.stdout:(13/152): libcephfs-devel-19.0.0-3126.gd44fddd9 428 kB/s | 32 kB 00:00 2024-04-23T14:28:34.591 INFO:teuthology.orchestra.run.smithi059.stdout:(14/152): libcephfs2-19.0.0-3126.gd44fddd9.el9. 4.4 MB/s | 746 kB 00:00 2024-04-23T14:28:34.650 INFO:teuthology.orchestra.run.smithi076.stdout:(8/152): ceph-radosgw-19.0.0-3126.gd44fddd9.el9 14 MB/s | 11 MB 00:00 2024-04-23T14:28:34.666 INFO:teuthology.orchestra.run.smithi059.stdout:(15/152): libcephsqlite-19.0.0-3126.gd44fddd9.e 2.2 MB/s | 163 kB 00:00 2024-04-23T14:28:34.733 INFO:teuthology.orchestra.run.smithi059.stdout:(16/152): librados-devel-19.0.0-3126.gd44fddd9. 1.9 MB/s | 127 kB 00:00 2024-04-23T14:28:34.759 INFO:teuthology.orchestra.run.smithi076.stdout:(9/152): ceph-mds-19.0.0-3126.gd44fddd9.el9.x86 828 kB/s | 2.4 MB 00:02 2024-04-23T14:28:34.793 INFO:teuthology.orchestra.run.smithi076.stdout:(10/152): ceph-selinux-19.0.0-3126.gd44fddd9.el 178 kB/s | 25 kB 00:00 2024-04-23T14:28:34.841 INFO:teuthology.orchestra.run.smithi059.stdout:(17/152): libradosstriper1-19.0.0-3126.gd44fddd 4.6 MB/s | 500 kB 00:00 2024-04-23T14:28:34.859 INFO:teuthology.orchestra.run.smithi076.stdout:(11/152): cephfs-mirror-19.0.0-3126.gd44fddd9.e 3.3 MB/s | 225 kB 00:00 2024-04-23T14:28:34.918 INFO:teuthology.orchestra.run.smithi076.stdout:(12/152): cephfs-shell-19.0.0-3126.gd44fddd9.el 469 kB/s | 27 kB 00:00 2024-04-23T14:28:34.985 INFO:teuthology.orchestra.run.smithi076.stdout:(13/152): libcephfs-devel-19.0.0-3126.gd44fddd9 482 kB/s | 32 kB 00:00 2024-04-23T14:28:35.086 INFO:teuthology.orchestra.run.smithi076.stdout:(14/152): libcephfs2-19.0.0-3126.gd44fddd9.el9. 7.4 MB/s | 746 kB 00:00 2024-04-23T14:28:35.153 INFO:teuthology.orchestra.run.smithi076.stdout:(15/152): libcephsqlite-19.0.0-3126.gd44fddd9.e 2.4 MB/s | 163 kB 00:00 2024-04-23T14:28:35.219 INFO:teuthology.orchestra.run.smithi076.stdout:(16/152): librados-devel-19.0.0-3126.gd44fddd9. 1.9 MB/s | 127 kB 00:00 2024-04-23T14:28:35.320 INFO:teuthology.orchestra.run.smithi076.stdout:(17/152): libradosstriper1-19.0.0-3126.gd44fddd 4.9 MB/s | 500 kB 00:00 2024-04-23T14:28:35.825 INFO:teuthology.orchestra.run.smithi059.stdout:(18/152): librgw2-19.0.0-3126.gd44fddd9.el9.x86 5.5 MB/s | 5.4 MB 00:00 2024-04-23T14:28:35.837 INFO:teuthology.orchestra.run.smithi076.stdout:(18/152): librgw2-19.0.0-3126.gd44fddd9.el9.x86 10 MB/s | 5.4 MB 00:00 2024-04-23T14:28:35.892 INFO:teuthology.orchestra.run.smithi059.stdout:(19/152): python3-ceph-argparse-19.0.0-3126.gd4 689 kB/s | 45 kB 00:00 2024-04-23T14:28:35.929 INFO:teuthology.orchestra.run.smithi076.stdout:(19/152): python3-ceph-argparse-19.0.0-3126.gd4 497 kB/s | 45 kB 00:00 2024-04-23T14:28:35.994 INFO:teuthology.orchestra.run.smithi059.stdout:(20/152): python3-ceph-common-19.0.0-3126.gd44f 1.3 MB/s | 135 kB 00:00 2024-04-23T14:28:35.998 INFO:teuthology.orchestra.run.smithi076.stdout:(20/152): python3-ceph-common-19.0.0-3126.gd44f 2.0 MB/s | 135 kB 00:00 2024-04-23T14:28:36.059 INFO:teuthology.orchestra.run.smithi059.stdout:(21/152): python3-cephfs-19.0.0-3126.gd44fddd9. 2.4 MB/s | 161 kB 00:00 2024-04-23T14:28:36.071 INFO:teuthology.orchestra.run.smithi076.stdout:(21/152): python3-cephfs-19.0.0-3126.gd44fddd9. 2.1 MB/s | 161 kB 00:00 2024-04-23T14:28:36.173 INFO:teuthology.orchestra.run.smithi076.stdout:(22/152): python3-rados-19.0.0-3126.gd44fddd9.e 3.2 MB/s | 320 kB 00:00 2024-04-23T14:28:36.201 INFO:teuthology.orchestra.run.smithi059.stdout:(22/152): python3-rados-19.0.0-3126.gd44fddd9.e 2.2 MB/s | 320 kB 00:00 2024-04-23T14:28:36.280 INFO:teuthology.orchestra.run.smithi076.stdout:(23/152): python3-rbd-19.0.0-3126.gd44fddd9.el9 2.7 MB/s | 297 kB 00:00 2024-04-23T14:28:36.293 INFO:teuthology.orchestra.run.smithi059.stdout:(23/152): python3-rbd-19.0.0-3126.gd44fddd9.el9 3.2 MB/s | 297 kB 00:00 2024-04-23T14:28:36.347 INFO:teuthology.orchestra.run.smithi076.stdout:(24/152): python3-rgw-19.0.0-3126.gd44fddd9.el9 1.5 MB/s | 100 kB 00:00 2024-04-23T14:28:36.360 INFO:teuthology.orchestra.run.smithi059.stdout:(24/152): python3-rgw-19.0.0-3126.gd44fddd9.el9 1.5 MB/s | 100 kB 00:00 2024-04-23T14:28:36.414 INFO:teuthology.orchestra.run.smithi076.stdout:(25/152): rbd-fuse-19.0.0-3126.gd44fddd9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-23T14:28:36.435 INFO:teuthology.orchestra.run.smithi059.stdout:(25/152): rbd-fuse-19.0.0-3126.gd44fddd9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-23T14:28:36.739 INFO:teuthology.orchestra.run.smithi076.stdout:(26/152): rbd-mirror-19.0.0-3126.gd44fddd9.el9. 9.5 MB/s | 3.1 MB 00:00 2024-04-23T14:28:36.806 INFO:teuthology.orchestra.run.smithi076.stdout:(27/152): rbd-nbd-19.0.0-3126.gd44fddd9.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-23T14:28:36.882 INFO:teuthology.orchestra.run.smithi076.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3126.g 460 kB/s | 34 kB 00:00 2024-04-23T14:28:36.982 INFO:teuthology.orchestra.run.smithi076.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3126.gd44fddd 1.4 MB/s | 142 kB 00:00 2024-04-23T14:28:37.153 INFO:teuthology.orchestra.run.smithi059.stdout:(26/152): rbd-mirror-19.0.0-3126.gd44fddd9.el9. 4.3 MB/s | 3.1 MB 00:00 2024-04-23T14:28:37.285 INFO:teuthology.orchestra.run.smithi059.stdout:(27/152): rbd-nbd-19.0.0-3126.gd44fddd9.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-23T14:28:37.329 INFO:teuthology.orchestra.run.smithi076.stdout:(30/152): ceph-osd-19.0.0-3126.gd44fddd9.el9.x8 4.3 MB/s | 17 MB 00:03 2024-04-23T14:28:37.386 INFO:teuthology.orchestra.run.smithi059.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3126.g 344 kB/s | 34 kB 00:00 2024-04-23T14:28:37.466 INFO:teuthology.orchestra.run.smithi076.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3126.gd44fd 7.3 MB/s | 3.5 MB 00:00 2024-04-23T14:28:37.470 INFO:teuthology.orchestra.run.smithi059.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3126.gd44fddd 1.7 MB/s | 142 kB 00:00 2024-04-23T14:28:37.566 INFO:teuthology.orchestra.run.smithi076.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3126.gd4 2.5 MB/s | 249 kB 00:00 2024-04-23T14:28:37.642 INFO:teuthology.orchestra.run.smithi076.stdout:(33/152): ceph-mgr-rook-19.0.0-3126.gd44fddd9.e 663 kB/s | 49 kB 00:00 2024-04-23T14:28:37.717 INFO:teuthology.orchestra.run.smithi076.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3126.gd 221 kB/s | 16 kB 00:00 2024-04-23T14:28:37.819 INFO:teuthology.orchestra.run.smithi076.stdout:(35/152): ceph-volume-19.0.0-3126.gd44fddd9.el9 2.6 MB/s | 273 kB 00:00 2024-04-23T14:28:37.987 INFO:teuthology.orchestra.run.smithi076.stdout:(36/152): cephadm-19.0.0-3126.gd44fddd9.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-23T14:28:38.070 INFO:teuthology.orchestra.run.smithi076.stdout:(37/152): cephfs-top-19.0.0-3126.gd44fddd9.el9. 221 kB/s | 18 kB 00:00 2024-04-23T14:28:38.187 INFO:teuthology.orchestra.run.smithi076.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 694 kB/s | 80 kB 00:00 2024-04-23T14:28:38.245 INFO:teuthology.orchestra.run.smithi059.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3126.gd44fd 4.6 MB/s | 3.5 MB 00:00 2024-04-23T14:28:38.296 INFO:teuthology.orchestra.run.smithi076.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-23T14:28:38.346 INFO:teuthology.orchestra.run.smithi076.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-23T14:28:38.397 INFO:teuthology.orchestra.run.smithi076.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-23T14:28:38.564 INFO:teuthology.orchestra.run.smithi076.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-04-23T14:28:38.622 INFO:teuthology.orchestra.run.smithi076.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 5.5 MB/s | 1.2 MB 00:00 2024-04-23T14:28:38.706 INFO:teuthology.orchestra.run.smithi076.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-23T14:28:38.857 INFO:teuthology.orchestra.run.smithi076.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 365 kB/s | 106 kB 00:00 2024-04-23T14:28:38.882 INFO:teuthology.orchestra.run.smithi076.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 715 kB/s | 125 kB 00:00 2024-04-23T14:28:38.932 INFO:teuthology.orchestra.run.smithi076.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-23T14:28:39.233 INFO:teuthology.orchestra.run.smithi076.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-04-23T14:28:39.333 INFO:teuthology.orchestra.run.smithi076.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-23T14:28:39.375 INFO:teuthology.orchestra.run.smithi076.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-04-23T14:28:39.400 INFO:teuthology.orchestra.run.smithi076.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-04-23T14:28:39.442 INFO:teuthology.orchestra.run.smithi076.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-23T14:28:39.468 INFO:teuthology.orchestra.run.smithi076.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 492 kB/s | 33 kB 00:00 2024-04-23T14:28:39.526 INFO:teuthology.orchestra.run.smithi076.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-04-23T14:28:39.585 INFO:teuthology.orchestra.run.smithi076.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-04-23T14:28:39.652 INFO:teuthology.orchestra.run.smithi076.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-23T14:28:39.711 INFO:teuthology.orchestra.run.smithi076.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 787 kB/s | 45 kB 00:00 2024-04-23T14:28:39.795 INFO:teuthology.orchestra.run.smithi076.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-23T14:28:39.836 INFO:teuthology.orchestra.run.smithi076.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-23T14:28:39.878 INFO:teuthology.orchestra.run.smithi076.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 582 kB 00:00 2024-04-23T14:28:39.904 INFO:teuthology.orchestra.run.smithi076.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 382 kB/s | 25 kB 00:00 2024-04-23T14:28:39.929 INFO:teuthology.orchestra.run.smithi076.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 958 kB/s | 47 kB 00:00 2024-04-23T14:28:39.963 INFO:teuthology.orchestra.run.smithi076.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-23T14:28:39.997 INFO:teuthology.orchestra.run.smithi076.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-23T14:28:40.020 INFO:teuthology.orchestra.run.smithi059.stdout:(31/152): ceph-osd-19.0.0-3126.gd44fddd9.el9.x8 1.7 MB/s | 17 MB 00:09 2024-04-23T14:28:40.021 INFO:teuthology.orchestra.run.smithi076.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 609 kB/s | 35 kB 00:00 2024-04-23T14:28:40.162 INFO:teuthology.orchestra.run.smithi059.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-04-23T14:28:40.254 INFO:teuthology.orchestra.run.smithi059.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3126.gd4 1.0 MB/s | 249 kB 00:00 2024-04-23T14:28:40.274 INFO:teuthology.orchestra.run.smithi059.stdout:(34/152): ceph-mgr-rook-19.0.0-3126.gd44fddd9.e 459 kB/s | 49 kB 00:00 2024-04-23T14:28:40.329 INFO:teuthology.orchestra.run.smithi059.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3126.gd 221 kB/s | 16 kB 00:00 2024-04-23T14:28:40.364 INFO:teuthology.orchestra.run.smithi076.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-23T14:28:40.405 INFO:teuthology.orchestra.run.smithi059.stdout:(36/152): ceph-volume-19.0.0-3126.gd44fddd9.el9 2.0 MB/s | 273 kB 00:00 2024-04-23T14:28:40.464 INFO:teuthology.orchestra.run.smithi076.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-23T14:28:40.488 INFO:teuthology.orchestra.run.smithi059.stdout:(37/152): cephfs-top-19.0.0-3126.gd44fddd9.el9. 220 kB/s | 18 kB 00:00 2024-04-23T14:28:40.540 INFO:teuthology.orchestra.run.smithi076.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-23T14:28:40.623 INFO:teuthology.orchestra.run.smithi076.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-23T14:28:40.657 INFO:teuthology.orchestra.run.smithi076.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-23T14:28:40.722 INFO:teuthology.orchestra.run.smithi059.stdout:(38/152): cephadm-19.0.0-3126.gd44fddd9.el9.noa 1.9 MB/s | 762 kB 00:00 2024-04-23T14:28:40.757 INFO:teuthology.orchestra.run.smithi076.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-23T14:28:40.817 INFO:teuthology.orchestra.run.smithi076.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-23T14:28:40.881 INFO:teuthology.orchestra.run.smithi059.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 205 kB/s | 80 kB 00:00 2024-04-23T14:28:40.941 INFO:teuthology.orchestra.run.smithi076.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-23T14:28:41.000 INFO:teuthology.orchestra.run.smithi076.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.2 MB/s | 244 kB 00:00 2024-04-23T14:28:41.081 INFO:teuthology.orchestra.run.smithi059.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 979 kB/s | 195 kB 00:00 2024-04-23T14:28:41.083 INFO:teuthology.orchestra.run.smithi076.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-23T14:28:41.134 INFO:teuthology.orchestra.run.smithi076.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 521 kB/s | 26 kB 00:00 2024-04-23T14:28:41.251 INFO:teuthology.orchestra.run.smithi076.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-23T14:28:41.273 INFO:teuthology.orchestra.run.smithi059.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-23T14:28:41.276 INFO:teuthology.orchestra.run.smithi076.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 136 kB/s | 19 kB 00:00 2024-04-23T14:28:41.310 INFO:teuthology.orchestra.run.smithi076.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-23T14:28:41.335 INFO:teuthology.orchestra.run.smithi076.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 601 kB/s | 35 kB 00:00 2024-04-23T14:28:41.410 INFO:teuthology.orchestra.run.smithi076.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-23T14:28:41.486 INFO:teuthology.orchestra.run.smithi076.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-23T14:28:41.540 INFO:teuthology.orchestra.run.smithi059.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 983 kB/s | 803 kB 00:00 2024-04-23T14:28:41.603 INFO:teuthology.orchestra.run.smithi076.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-23T14:28:41.628 INFO:teuthology.orchestra.run.smithi076.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 545 kB/s | 77 kB 00:00 2024-04-23T14:28:41.654 INFO:teuthology.orchestra.run.smithi076.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-23T14:28:41.657 INFO:teuthology.orchestra.run.smithi059.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 920 kB/s | 106 kB 00:00 2024-04-23T14:28:41.688 INFO:teuthology.orchestra.run.smithi076.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-23T14:28:41.721 INFO:teuthology.orchestra.run.smithi076.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-23T14:28:41.755 INFO:teuthology.orchestra.run.smithi076.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-23T14:28:41.780 INFO:teuthology.orchestra.run.smithi076.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 883 kB/s | 51 kB 00:00 2024-04-23T14:28:41.805 INFO:teuthology.orchestra.run.smithi076.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-23T14:28:41.807 INFO:teuthology.orchestra.run.smithi059.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-04-23T14:28:41.856 INFO:teuthology.orchestra.run.smithi076.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 847 kB/s | 42 kB 00:00 2024-04-23T14:28:41.874 INFO:teuthology.orchestra.run.smithi059.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 625 kB/s | 135 kB 00:00 2024-04-23T14:28:41.906 INFO:teuthology.orchestra.run.smithi076.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 897 kB/s | 44 kB 00:00 2024-04-23T14:28:41.926 INFO:teuthology.orchestra.run.smithi059.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-23T14:28:41.965 INFO:teuthology.orchestra.run.smithi076.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2024-04-23T14:28:41.991 INFO:teuthology.orchestra.run.smithi059.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-23T14:28:42.041 INFO:teuthology.orchestra.run.smithi076.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 480 kB/s | 36 kB 00:00 2024-04-23T14:28:42.050 INFO:teuthology.orchestra.run.smithi059.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-23T14:28:42.091 INFO:teuthology.orchestra.run.smithi076.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 967 kB/s | 47 kB 00:00 2024-04-23T14:28:42.158 INFO:teuthology.orchestra.run.smithi076.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-23T14:28:42.167 INFO:teuthology.orchestra.run.smithi059.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 919 kB/s | 107 kB 00:00 2024-04-23T14:28:42.208 INFO:teuthology.orchestra.run.smithi076.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T14:28:42.259 INFO:teuthology.orchestra.run.smithi076.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-23T14:28:42.301 INFO:teuthology.orchestra.run.smithi059.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 162 kB/s | 21 kB 00:00 2024-04-23T14:28:42.309 INFO:teuthology.orchestra.run.smithi076.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-23T14:28:42.351 INFO:teuthology.orchestra.run.smithi059.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-23T14:28:42.418 INFO:teuthology.orchestra.run.smithi059.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-23T14:28:42.460 INFO:teuthology.orchestra.run.smithi059.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-04-23T14:28:42.536 INFO:teuthology.orchestra.run.smithi076.stdout:(100/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-23T14:28:42.560 INFO:teuthology.orchestra.run.smithi076.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 179 kB/s | 45 kB 00:00 2024-04-23T14:28:42.568 INFO:teuthology.orchestra.run.smithi059.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 169 kB/s | 18 kB 00:00 2024-04-23T14:28:42.577 INFO:teuthology.orchestra.run.smithi076.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 546 kB/s | 22 kB 00:00 2024-04-23T14:28:42.644 INFO:teuthology.orchestra.run.smithi076.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-23T14:28:42.677 INFO:teuthology.orchestra.run.smithi059.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-23T14:28:42.769 INFO:teuthology.orchestra.run.smithi059.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-23T14:28:42.828 INFO:teuthology.orchestra.run.smithi076.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-23T14:28:42.895 INFO:teuthology.orchestra.run.smithi076.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 295 kB/s | 19 kB 00:00 2024-04-23T14:28:42.993 INFO:teuthology.orchestra.run.smithi076.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-23T14:28:43.062 INFO:teuthology.orchestra.run.smithi076.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 382 kB/s | 26 kB 00:00 2024-04-23T14:28:43.129 INFO:teuthology.orchestra.run.smithi076.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 737 kB/s | 49 kB 00:00 2024-04-23T14:28:43.204 INFO:teuthology.orchestra.run.smithi076.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 909 kB/s | 67 kB 00:00 2024-04-23T14:28:43.371 INFO:teuthology.orchestra.run.smithi076.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 911 kB/s | 151 kB 00:00 2024-04-23T14:28:43.403 INFO:teuthology.orchestra.run.smithi059.stdout:(57/152): ceph-test-19.0.0-3126.gd44fddd9.el9.x 5.3 MB/s | 49 MB 00:09 2024-04-23T14:28:43.428 INFO:teuthology.orchestra.run.smithi059.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 214 kB/s | 160 kB 00:00 2024-04-23T14:28:43.453 INFO:teuthology.orchestra.run.smithi059.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-04-23T14:28:43.603 INFO:teuthology.orchestra.run.smithi059.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 170 kB/s | 25 kB 00:00 2024-04-23T14:28:43.846 INFO:teuthology.orchestra.run.smithi059.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 197 kB/s | 47 kB 00:00 2024-04-23T14:28:43.880 INFO:teuthology.orchestra.run.smithi059.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.4 MB/s | 662 kB 00:00 2024-04-23T14:28:43.880 INFO:teuthology.orchestra.run.smithi076.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-04-23T14:28:43.996 INFO:teuthology.orchestra.run.smithi059.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-23T14:28:44.029 INFO:teuthology.orchestra.run.smithi059.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-23T14:28:44.030 INFO:teuthology.orchestra.run.smithi076.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-04-23T14:28:44.098 INFO:teuthology.orchestra.run.smithi076.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 661 kB/s | 43 kB 00:00 2024-04-23T14:28:44.113 INFO:teuthology.orchestra.run.smithi059.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 303 kB/s | 35 kB 00:00 2024-04-23T14:28:44.164 INFO:teuthology.orchestra.run.smithi076.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 489 kB/s | 32 kB 00:00 2024-04-23T14:28:44.198 INFO:teuthology.orchestra.run.smithi076.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-23T14:28:44.223 INFO:teuthology.orchestra.run.smithi076.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-04-23T14:28:44.282 INFO:teuthology.orchestra.run.smithi076.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-23T14:28:44.324 INFO:teuthology.orchestra.run.smithi076.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-23T14:28:44.400 INFO:teuthology.orchestra.run.smithi076.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 474 kB/s | 35 kB 00:00 2024-04-23T14:28:44.497 INFO:teuthology.orchestra.run.smithi059.stdout:(66/152): libtool-2.4.6-45.el9.x86_64.rpm 545 kB/s | 582 kB 00:01 2024-04-23T14:28:44.500 INFO:teuthology.orchestra.run.smithi076.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-04-23T14:28:44.533 INFO:teuthology.orchestra.run.smithi076.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-23T14:28:45.792 INFO:teuthology.orchestra.run.smithi076.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 115 kB/s | 144 kB 00:01 2024-04-23T14:28:46.059 INFO:teuthology.orchestra.run.smithi076.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 40 kB/s | 11 kB 00:00 2024-04-23T14:28:46.251 INFO:teuthology.orchestra.run.smithi076.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 93 kB/s | 18 kB 00:00 2024-04-23T14:28:46.326 INFO:teuthology.orchestra.run.smithi076.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.1 MB/s | 2.0 MB 00:01 2024-04-23T14:28:46.410 INFO:teuthology.orchestra.run.smithi076.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-23T14:28:46.435 INFO:teuthology.orchestra.run.smithi076.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 126 kB/s | 23 kB 00:00 2024-04-23T14:28:46.494 INFO:teuthology.orchestra.run.smithi076.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-04-23T14:28:46.594 INFO:teuthology.orchestra.run.smithi076.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 464 kB/s | 46 kB 00:00 2024-04-23T14:28:46.745 INFO:teuthology.orchestra.run.smithi076.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 528 kB/s | 79 kB 00:00 2024-04-23T14:28:46.828 INFO:teuthology.orchestra.run.smithi076.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 698 kB/s | 58 kB 00:00 2024-04-23T14:28:47.037 INFO:teuthology.orchestra.run.smithi076.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-23T14:28:47.079 INFO:teuthology.orchestra.run.smithi076.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 402 kB/s | 16 kB 00:00 2024-04-23T14:28:47.180 INFO:teuthology.orchestra.run.smithi076.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 906 kB/s | 90 kB 00:00 2024-04-23T14:28:47.238 INFO:teuthology.orchestra.run.smithi076.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 491 kB/s | 28 kB 00:00 2024-04-23T14:28:47.264 INFO:teuthology.orchestra.run.smithi059.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 588 kB/s | 1.6 MB 00:02 2024-04-23T14:28:47.305 INFO:teuthology.orchestra.run.smithi076.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 467 kB/s | 31 kB 00:00 2024-04-23T14:28:47.439 INFO:teuthology.orchestra.run.smithi076.stdout:(137/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-23T14:28:47.523 INFO:teuthology.orchestra.run.smithi076.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 713 kB/s | 59 kB 00:00 2024-04-23T14:28:47.581 INFO:teuthology.orchestra.run.smithi076.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 623 kB/s | 36 kB 00:00 2024-04-23T14:28:47.623 INFO:teuthology.orchestra.run.smithi076.stdout:(140/152): python3-kubernetes-26.1.0-2.el9.noar 880 kB/s | 1.0 MB 00:01 2024-04-23T14:28:47.665 INFO:teuthology.orchestra.run.smithi076.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 615 kB/s | 51 kB 00:00 2024-04-23T14:28:47.749 INFO:teuthology.orchestra.run.smithi076.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-23T14:28:47.816 INFO:teuthology.orchestra.run.smithi076.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-23T14:28:47.858 INFO:teuthology.orchestra.run.smithi076.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 487 kB/s | 20 kB 00:00 2024-04-23T14:28:48.025 INFO:teuthology.orchestra.run.smithi076.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.5 MB/s | 427 kB 00:00 2024-04-23T14:28:48.050 INFO:teuthology.orchestra.run.smithi076.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-23T14:28:48.364 INFO:teuthology.orchestra.run.smithi059.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 829 kB/s | 911 kB 00:01 2024-04-23T14:28:48.548 INFO:teuthology.orchestra.run.smithi059.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 731 kB/s | 133 kB 00:00 2024-04-23T14:28:48.659 INFO:teuthology.orchestra.run.smithi076.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 2.5 MB/s | 1.6 MB 00:00 2024-04-23T14:28:49.435 INFO:teuthology.orchestra.run.smithi076.stdout:(148/152): librbd1-19.0.0-3126.gd44fddd9.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-23T14:28:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-04-23T14:28:49.727 INFO:teuthology.orchestra.run.smithi076.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-23T14:28:50.306 INFO:teuthology.orchestra.run.smithi059.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 773 kB/s | 4.7 MB 00:06 2024-04-23T14:28:50.440 INFO:teuthology.orchestra.run.smithi059.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-23T14:28:50.548 INFO:teuthology.orchestra.run.smithi059.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 813 kB/s | 87 kB 00:00 2024-04-23T14:28:50.961 INFO:teuthology.orchestra.run.smithi076.stdout:(151/152): librados2-19.0.0-3126.gd44fddd9.el9. 1.2 MB/s | 3.4 MB 00:02 2024-04-23T14:28:51.232 INFO:teuthology.orchestra.run.smithi059.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.7 MB/s | 6.0 MB 00:00 2024-04-23T14:28:51.342 INFO:teuthology.orchestra.run.smithi059.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-23T14:28:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-23T14:28:51.566 INFO:teuthology.orchestra.run.smithi059.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 238 kB/s | 26 kB 00:00 2024-04-23T14:28:51.683 INFO:teuthology.orchestra.run.smithi059.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 165 kB/s | 19 kB 00:00 2024-04-23T14:28:51.791 INFO:teuthology.orchestra.run.smithi059.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-23T14:28:51.900 INFO:teuthology.orchestra.run.smithi059.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 323 kB/s | 35 kB 00:00 2024-04-23T14:28:52.384 INFO:teuthology.orchestra.run.smithi059.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-23T14:28:52.776 INFO:teuthology.orchestra.run.smithi059.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 641 kB/s | 250 kB 00:00 2024-04-23T14:28:52.902 INFO:teuthology.orchestra.run.smithi059.stdout:(82/152): postgresql-server-13.14-1.el9.x86_64. 1.3 MB/s | 5.8 MB 00:04 2024-04-23T14:28:52.969 INFO:teuthology.orchestra.run.smithi059.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-04-23T14:28:53.010 INFO:teuthology.orchestra.run.smithi059.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 717 kB/s | 77 kB 00:00 2024-04-23T14:28:53.077 INFO:teuthology.orchestra.run.smithi059.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 391 kB/s | 42 kB 00:00 2024-04-23T14:28:53.136 INFO:teuthology.orchestra.run.smithi059.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-23T14:28:53.186 INFO:teuthology.orchestra.run.smithi059.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-23T14:28:53.245 INFO:teuthology.orchestra.run.smithi059.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-23T14:28:53.295 INFO:teuthology.orchestra.run.smithi059.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 472 kB/s | 51 kB 00:00 2024-04-23T14:28:53.381 INFO:teuthology.orchestra.run.smithi059.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 396 kB/s | 54 kB 00:00 2024-04-23T14:28:53.496 INFO:teuthology.orchestra.run.smithi059.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 366 kB/s | 42 kB 00:00 2024-04-23T14:28:53.621 INFO:teuthology.orchestra.run.smithi059.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 354 kB/s | 44 kB 00:00 2024-04-23T14:28:53.747 INFO:teuthology.orchestra.run.smithi059.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-23T14:28:53.863 INFO:teuthology.orchestra.run.smithi059.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 311 kB/s | 36 kB 00:00 2024-04-23T14:28:53.972 INFO:teuthology.orchestra.run.smithi059.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 440 kB/s | 47 kB 00:00 2024-04-23T14:28:54.172 INFO:teuthology.orchestra.run.smithi059.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-23T14:28:54.348 INFO:teuthology.orchestra.run.smithi059.stdout:(97/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 443 kB/s | 4.4 MB 00:10 2024-04-23T14:28:54.373 INFO:teuthology.orchestra.run.smithi059.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 290 kB/s | 58 kB 00:00 2024-04-23T14:28:54.465 INFO:teuthology.orchestra.run.smithi059.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 808 kB/s | 94 kB 00:00 2024-04-23T14:28:54.490 INFO:teuthology.orchestra.run.smithi059.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 547 kB/s | 64 kB 00:00 2024-04-23T14:28:54.582 INFO:teuthology.orchestra.run.smithi059.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-04-23T14:28:54.607 INFO:teuthology.orchestra.run.smithi059.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 316 kB/s | 45 kB 00:00 2024-04-23T14:28:54.741 INFO:teuthology.orchestra.run.smithi059.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-04-23T14:28:55.000 INFO:teuthology.orchestra.run.smithi059.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.5 MB/s | 16 MB 00:01 2024-04-23T14:28:55.159 INFO:teuthology.orchestra.run.smithi059.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 123 kB/s | 19 kB 00:00 2024-04-23T14:28:55.300 INFO:teuthology.orchestra.run.smithi059.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 553 kB/s | 309 kB 00:00 2024-04-23T14:28:55.384 INFO:teuthology.orchestra.run.smithi059.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 315 kB/s | 26 kB 00:00 2024-04-23T14:28:55.425 INFO:teuthology.orchestra.run.smithi059.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 671 kB/s | 548 kB 00:00 2024-04-23T14:28:55.542 INFO:teuthology.orchestra.run.smithi059.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 308 kB/s | 49 kB 00:00 2024-04-23T14:28:55.659 INFO:teuthology.orchestra.run.smithi059.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-04-23T14:28:55.685 INFO:teuthology.orchestra.run.smithi059.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 261 kB/s | 67 kB 00:00 2024-04-23T14:28:55.710 INFO:teuthology.orchestra.run.smithi059.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 907 kB/s | 151 kB 00:00 2024-04-23T14:28:55.777 INFO:teuthology.orchestra.run.smithi059.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-04-23T14:28:55.885 INFO:teuthology.orchestra.run.smithi059.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-04-23T14:28:55.944 INFO:teuthology.orchestra.run.smithi059.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 186 kB/s | 43 kB 00:00 2024-04-23T14:28:56.011 INFO:teuthology.orchestra.run.smithi059.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-04-23T14:28:56.079 INFO:teuthology.orchestra.run.smithi059.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-04-23T14:28:56.112 INFO:teuthology.orchestra.run.smithi059.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-23T14:28:56.153 INFO:teuthology.orchestra.run.smithi059.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-23T14:28:56.229 INFO:teuthology.orchestra.run.smithi059.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 304 kB/s | 35 kB 00:00 2024-04-23T14:28:56.262 INFO:teuthology.orchestra.run.smithi059.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-23T14:28:56.340 INFO:teuthology.orchestra.run.smithi059.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-04-23T14:28:56.382 INFO:teuthology.orchestra.run.smithi059.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-23T14:28:56.432 INFO:teuthology.orchestra.run.smithi059.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-23T14:28:56.457 INFO:teuthology.orchestra.run.smithi059.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-04-23T14:28:56.524 INFO:teuthology.orchestra.run.smithi059.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 254 kB/s | 23 kB 00:00 2024-04-23T14:28:56.549 INFO:teuthology.orchestra.run.smithi059.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 213 kB/s | 19 kB 00:00 2024-04-23T14:28:56.625 INFO:teuthology.orchestra.run.smithi059.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-04-23T14:28:56.650 INFO:teuthology.orchestra.run.smithi059.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 158 kB/s | 20 kB 00:00 2024-04-23T14:28:56.708 INFO:teuthology.orchestra.run.smithi059.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-04-23T14:28:56.734 INFO:teuthology.orchestra.run.smithi059.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 429 kB/s | 46 kB 00:00 2024-04-23T14:28:56.792 INFO:teuthology.orchestra.run.smithi059.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 692 kB/s | 58 kB 00:00 2024-04-23T14:28:56.826 INFO:teuthology.orchestra.run.smithi059.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 450 kB/s | 79 kB 00:00 2024-04-23T14:28:56.859 INFO:teuthology.orchestra.run.smithi059.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-23T14:28:56.885 INFO:teuthology.orchestra.run.smithi059.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-23T14:28:56.960 INFO:teuthology.orchestra.run.smithi059.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-04-23T14:28:56.985 INFO:teuthology.orchestra.run.smithi059.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 568 kB/s | 90 kB 00:00 2024-04-23T14:28:57.010 INFO:teuthology.orchestra.run.smithi059.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-04-23T14:28:57.077 INFO:teuthology.orchestra.run.smithi059.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 648 kB/s | 59 kB 00:00 2024-04-23T14:28:57.103 INFO:teuthology.orchestra.run.smithi059.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 392 kB/s | 36 kB 00:00 2024-04-23T14:28:57.119 INFO:teuthology.orchestra.run.smithi059.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-23T14:28:57.245 INFO:teuthology.orchestra.run.smithi059.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 307 kB/s | 51 kB 00:00 2024-04-23T14:28:57.278 INFO:teuthology.orchestra.run.smithi059.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-23T14:28:57.379 INFO:teuthology.orchestra.run.smithi059.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-23T14:28:57.404 INFO:teuthology.orchestra.run.smithi059.stdout:(145/152): python3-websocket-client-1.2.3-2.el9 316 kB/s | 90 kB 00:00 2024-04-23T14:28:57.521 INFO:teuthology.orchestra.run.smithi059.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 83 kB/s | 20 kB 00:00 2024-04-23T14:28:57.663 INFO:teuthology.orchestra.run.smithi059.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 677 kB/s | 192 kB 00:00 2024-04-23T14:28:58.230 INFO:teuthology.orchestra.run.smithi059.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-04-23T14:28:58.405 INFO:teuthology.orchestra.run.smithi059.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 1.3 MB 00:00 2024-04-23T14:28:58.714 INFO:teuthology.orchestra.run.smithi059.stdout:(150/152): librados2-19.0.0-3126.gd44fddd9.el9. 2.8 MB/s | 3.4 MB 00:01 2024-04-23T14:28:58.797 INFO:teuthology.orchestra.run.smithi059.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 2.4 MB 00:00 2024-04-23T14:29:00.006 INFO:teuthology.orchestra.run.smithi059.stdout:(152/152): librbd1-19.0.0-3126.gd44fddd9.el9.x8 1.3 MB/s | 3.1 MB 00:02 2024-04-23T14:29:00.012 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-04-23T14:29:00.012 INFO:teuthology.orchestra.run.smithi059.stdout:Total 6.2 MB/s | 222 MB 00:35 2024-04-23T14:29:01.402 INFO:teuthology.orchestra.run.smithi076.stdout:(152/152): ceph-test-19.0.0-3126.gd44fddd9.el9. 1.8 MB/s | 49 MB 00:26 2024-04-23T14:29:01.410 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-04-23T14:29:01.410 INFO:teuthology.orchestra.run.smithi076.stdout:Total 6.7 MB/s | 222 MB 00:33 2024-04-23T14:29:03.016 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-04-23T14:29:03.290 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-04-23T14:29:03.290 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-04-23T14:29:04.279 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-04-23T14:29:04.522 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-04-23T14:29:04.522 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-04-23T14:29:06.119 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-04-23T14:29:06.119 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-04-23T14:29:07.183 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-04-23T14:29:07.183 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-04-23T14:29:12.767 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-04-23T14:29:12.866 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T14:29:12.992 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T14:29:13.114 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T14:29:13.177 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T14:29:13.214 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T14:29:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T14:29:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-04-23T14:29:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T14:29:13.679 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T14:29:13.784 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T14:29:13.817 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T14:29:13.834 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T14:29:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T14:29:13.928 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:29:14.026 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T14:29:14.105 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:29:14.137 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:29:14.306 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:29:14.354 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:29:14.466 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T14:29:14.577 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:29:14.700 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T14:29:14.767 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T14:29:14.787 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:29:14.792 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:29:14.844 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:29:14.955 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:29:14.984 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:29:15.045 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:29:15.083 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:29:15.086 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:29:15.127 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:29:15.198 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:29:15.422 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T14:29:15.499 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T14:29:15.503 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:29:15.524 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:29:15.584 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:29:15.721 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:29:15.750 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:29:15.791 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T14:29:15.898 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:29:15.923 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T14:29:15.928 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:29:15.969 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T14:29:16.007 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:29:16.036 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T14:29:16.143 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T14:29:16.198 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T14:29:16.262 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T14:29:16.275 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:29:16.312 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T14:29:16.345 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:29:16.351 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T14:29:16.474 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T14:29:16.543 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T14:29:16.548 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T14:29:16.586 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T14:29:16.644 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T14:29:16.684 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T14:29:16.736 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T14:29:16.766 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T14:29:16.807 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 31/156 2024-04-23T14:29:16.811 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T14:29:16.865 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 32/156 2024-04-23T14:29:16.914 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 33/156 2024-04-23T14:29:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T14:29:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T14:29:17.039 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T14:29:17.042 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T14:29:17.096 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T14:29:17.111 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T14:29:17.136 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T14:29:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T14:29:17.256 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T14:29:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T14:29:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T14:29:17.415 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T14:29:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T14:29:17.487 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T14:29:17.537 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T14:29:17.574 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 31/156 2024-04-23T14:29:17.627 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 32/156 2024-04-23T14:29:17.665 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 33/156 2024-04-23T14:29:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T14:29:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T14:29:17.932 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T14:29:18.111 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T14:29:18.180 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T14:29:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T14:29:18.329 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T14:29:18.369 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T14:29:18.421 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T14:29:18.464 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 43/156 2024-04-23T14:29:18.526 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T14:29:18.854 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T14:29:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T14:29:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T14:29:18.979 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T14:29:19.011 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T14:29:19.023 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T14:29:19.069 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T14:29:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 43/156 2024-04-23T14:29:19.135 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T14:29:19.168 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T14:29:19.181 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T14:29:19.221 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T14:29:19.294 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T14:29:19.336 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T14:29:19.402 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T14:29:19.424 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T14:29:19.466 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T14:29:19.492 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T14:29:19.562 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T14:29:19.654 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T14:29:19.678 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T14:29:19.701 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T14:29:19.733 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T14:29:19.741 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T14:29:19.771 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T14:29:19.808 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T14:29:19.831 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T14:29:19.848 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T14:29:19.911 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T14:29:19.974 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T14:29:19.976 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T14:29:20.020 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T14:29:20.068 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T14:29:20.102 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T14:29:20.135 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T14:29:20.178 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T14:29:20.231 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T14:29:20.268 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T14:29:20.324 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T14:29:20.339 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T14:29:20.453 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T14:29:20.497 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T14:29:20.530 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T14:29:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T14:29:20.582 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T14:29:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T14:29:20.625 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T14:29:20.650 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T14:29:20.801 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T14:29:20.970 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T14:29:20.991 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T14:29:21.051 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T14:29:21.051 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T14:29:21.089 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T14:29:21.111 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T14:29:21.136 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T14:29:21.164 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T14:29:21.203 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T14:29:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T14:29:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T14:29:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T14:29:21.313 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T14:29:21.346 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T14:29:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T14:29:21.406 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T14:29:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T14:29:21.485 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T14:29:21.529 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T14:29:21.532 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T14:29:21.595 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T14:29:21.638 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T14:29:21.705 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T14:29:21.711 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T14:29:21.772 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T14:29:21.814 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T14:29:21.874 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T14:29:21.884 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T14:29:21.943 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T14:29:22.030 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T14:29:22.094 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T14:29:22.958 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T14:29:23.009 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T14:29:23.023 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T14:29:23.042 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T14:29:23.086 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T14:29:23.120 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T14:29:23.168 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T14:29:23.245 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T14:29:23.245 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T14:29:23.314 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T14:29:23.356 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T14:29:23.410 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T14:29:23.413 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T14:29:23.454 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T14:29:23.657 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T14:29:23.689 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T14:29:23.718 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T14:29:23.752 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T14:29:23.788 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T14:29:23.860 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T14:29:24.671 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T14:29:24.708 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T14:29:24.826 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T14:29:24.843 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T14:29:26.402 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T14:29:26.504 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T14:29:26.536 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T14:29:26.544 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T14:29:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T14:29:26.638 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T14:29:26.704 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T14:29:26.750 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T14:29:26.971 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T14:29:27.017 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:29:27.156 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:29:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T14:29:27.191 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 100/156 2024-04-23T14:29:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:29:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T14:29:27.322 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:29:27.356 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 100/156 2024-04-23T14:29:27.415 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T14:29:27.828 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T14:29:27.887 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:29:27.989 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:29:28.004 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T14:29:28.014 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 104/156 2024-04-23T14:29:28.044 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:29:28.145 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:29:28.171 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 104/156 2024-04-23T14:29:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:29:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:29:30.973 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:29:31.017 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:29:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:29:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:29:31.740 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:29:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:29:31.791 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T14:29:31.791 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T14:29:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:31.796 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:29:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:29:31.981 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:29:31.982 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T14:29:31.982 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T14:29:31.982 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:32.030 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:29:45.159 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:29:45.159 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-04-23T14:29:45.159 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-04-23T14:29:45.159 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-04-23T14:29:45.159 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-04-23T14:29:45.160 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-04-23T14:29:45.160 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-04-23T14:29:45.160 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-04-23T14:29:45.160 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:45.381 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:29:45.429 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:29:45.429 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:45.429 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T14:29:45.429 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-04-23T14:29:45.692 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:45.943 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:29:45.991 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:29:45.991 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:45.992 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T14:29:45.992 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:47.034 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:29:47.112 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:29:47.112 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:47.112 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T14:29:47.112 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:29:47.112 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:29:47.112 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:47.384 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:29:47.453 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:29:47.453 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:47.453 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T14:29:47.454 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:29:47.454 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:29:47.454 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:47.695 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:29:47.779 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:29:47.780 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:47.780 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T14:29:47.780 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:29:47.780 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:29:47.780 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:47.943 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:29:48.013 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:29:48.013 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:48.013 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T14:29:48.013 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:29:48.013 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:29:48.013 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:48.045 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:29:48.118 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:29:48.118 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:48.118 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T14:29:48.118 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:29:48.118 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:29:48.119 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:48.152 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T14:29:48.216 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T14:29:48.292 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T14:29:48.557 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T14:29:48.616 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T14:29:48.655 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:29:48.700 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T14:29:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:29:48.727 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:48.727 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T14:29:48.727 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:29:48.727 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:29:48.727 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:48.802 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:29:48.873 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T14:29:48.939 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T14:29:48.970 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:29:49.014 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T14:29:49.023 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T14:29:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T14:29:49.098 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T14:29:49.268 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T14:29:49.336 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T14:29:49.429 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T14:29:49.547 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:29:49.737 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:29:49.754 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:29:49.793 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:29:49.802 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T14:29:49.846 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 124/156 2024-04-23T14:29:49.849 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T14:29:49.890 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T14:29:49.947 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 125/156 2024-04-23T14:29:49.979 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:29:50.593 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:29:50.628 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:29:50.666 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 124/156 2024-04-23T14:29:50.763 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 125/156 2024-04-23T14:29:50.795 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:29:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:29:51.331 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:29:52.134 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:29:52.197 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:29:52.431 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:29:52.465 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:29:52.613 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:29:52.789 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 129/156 2024-04-23T14:29:52.818 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:29:52.914 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:29:52.914 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:52.914 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T14:29:52.914 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:29:52.915 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:29:52.915 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:52.985 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:29:53.024 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:29:53.187 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 132/156 2024-04-23T14:29:53.267 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 133/156 2024-04-23T14:29:53.366 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:29:53.397 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:29:53.551 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:29:53.738 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 129/156 2024-04-23T14:29:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:29:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:29:53.852 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:53.852 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T14:29:53.852 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:29:53.852 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:29:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:53.935 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:29:53.974 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:29:54.145 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 132/156 2024-04-23T14:29:54.226 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 133/156 2024-04-23T14:29:54.324 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T14:29:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:29:54.452 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:29:54.452 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:54.453 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T14:29:54.453 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:29:54.453 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:29:54.453 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:29:54.592 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:29:54.592 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:54.592 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T14:29:54.592 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:29:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:29:54.768 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:54.769 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T14:29:54.769 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:29:54.769 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:29:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:29:55.275 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:29:55.275 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:55.275 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T14:29:55.275 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:29:55.275 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:29:55.276 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:29:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T14:29:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:29:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:29:55.467 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:55.467 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T14:29:55.467 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:29:55.467 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:29:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:29:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:29:55.595 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:55.595 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T14:29:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:29:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:29:55.786 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:55.786 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T14:29:55.786 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:29:55.787 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:29:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:29:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:29:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:29:56.243 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:29:56.243 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T14:29:56.243 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:29:56.243 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:29:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:00.315 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 139/156 2024-04-23T14:30:00.371 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 140/156 2024-04-23T14:30:00.442 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 141/156 2024-04-23T14:30:00.484 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T14:30:00.531 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T14:30:00.836 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 144/156 2024-04-23T14:30:00.895 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T14:30:00.923 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 146/156 2024-04-23T14:30:01.588 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 139/156 2024-04-23T14:30:01.655 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 140/156 2024-04-23T14:30:01.749 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 141/156 2024-04-23T14:30:01.853 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:30:01.906 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:30:01.916 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T14:30:01.961 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:30:01.983 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T14:30:02.329 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 144/156 2024-04-23T14:30:02.424 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T14:30:02.451 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 146/156 2024-04-23T14:30:03.281 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 148/156 2024-04-23T14:30:03.392 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T14:30:03.469 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T14:30:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:30:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:30:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:30:03.649 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T14:30:03.715 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T14:30:03.715 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:30:03.841 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:30:03.888 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:30:03.889 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:30:04.020 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:30:04.150 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:30:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:30:04.445 INFO:teuthology.orchestra.run.smithi076.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-23T14:30:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:05.067 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 148/156 2024-04-23T14:30:05.164 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T14:30:05.241 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T14:30:05.418 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T14:30:05.497 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T14:30:05.497 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:30:05.632 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:30:05.678 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:30:05.678 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:30:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:30:05.962 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:30:06.248 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:30:06.248 INFO:teuthology.orchestra.run.smithi059.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-23T14:30:06.248 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:10.327 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 1/156 2024-04-23T14:30:10.327 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2/156 2024-04-23T14:30:10.327 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 3/156 2024-04-23T14:30:10.327 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 4/156 2024-04-23T14:30:10.327 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 5/156 2024-04-23T14:30:10.327 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 6/156 2024-04-23T14:30:10.327 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 7/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 11/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 12/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 14/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 15/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 16/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 17/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 18/156 2024-04-23T14:30:10.328 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 19/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 20/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 21/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 22/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 23/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 24/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 25/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 26/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 27/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 28/156 2024-04-23T14:30:10.329 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 29/156 2024-04-23T14:30:10.342 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 30/156 2024-04-23T14:30:10.342 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 31/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 32/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 33/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 34/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 35/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 36/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 37/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 38/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 39/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T14:30:10.343 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T14:30:10.344 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T14:30:10.345 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T14:30:10.346 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T14:30:10.347 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T14:30:10.348 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:30:10.349 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T14:30:10.350 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T14:30:10.351 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 149/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 151/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:30:10.352 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T14:30:12.143 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 1/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 3/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 4/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 5/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 6/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 7/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 11/156 2024-04-23T14:30:12.144 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 12/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 14/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 15/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 16/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 17/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 18/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 19/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 20/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 21/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 22/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 23/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 24/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 25/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 26/156 2024-04-23T14:30:12.145 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 27/156 2024-04-23T14:30:12.146 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 28/156 2024-04-23T14:30:12.146 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 29/156 2024-04-23T14:30:12.157 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 30/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 31/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 32/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 33/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 34/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 35/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 36/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 37/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 38/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 39/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T14:30:12.158 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T14:30:12.159 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T14:30:12.160 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T14:30:12.161 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T14:30:12.162 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T14:30:12.163 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T14:30:12.164 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T14:30:12.165 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 149/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 151/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:30:12.166 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T14:30:12.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.691 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.692 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T14:30:12.693 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.694 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T14:30:12.695 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T14:30:12.696 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T14:30:12.697 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T14:30:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T14:30:12.699 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T14:30:12.700 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T14:30:12.701 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T14:30:12.701 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T14:30:12.701 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T14:30:12.701 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T14:30:12.701 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T14:30:12.701 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T14:30:12.701 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T14:30:12.702 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T14:30:12.702 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T14:30:12.702 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T14:30:12.702 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T14:30:12.702 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T14:30:12.702 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T14:30:12.703 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T14:30:12.704 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:12.704 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-04-23T14:30:13.328 DEBUG:teuthology.parallel:result is None 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-04-23T14:30:14.750 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.751 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.752 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T14:30:14.754 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T14:30:14.755 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T14:30:14.756 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T14:30:14.757 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T14:30:14.758 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.759 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T14:30:14.760 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T14:30:14.761 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:14.762 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-04-23T14:30:15.488 DEBUG:teuthology.parallel:result is None 2024-04-23T14:30:15.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:30:15.790 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T14:30:15.850 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-3126.gd44fddd9.el9 2024-04-23T14:30:15.850 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3126.gd44fddd9.el9 2024-04-23T14:30:15.850 INFO:teuthology.task.install:The correct ceph version 19.0.0-3126.gd44fddd9 is installed. 2024-04-23T14:30:15.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:30:15.990 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T14:30:16.045 INFO:teuthology.orchestra.run.smithi076.stdout:19.0.0-3126.gd44fddd9.el9 2024-04-23T14:30:16.045 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3126.gd44fddd9.el9 2024-04-23T14:30:16.045 INFO:teuthology.task.install:The correct ceph version 19.0.0-3126.gd44fddd9 is installed. 2024-04-23T14:30:16.047 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T14:30:16.047 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:16.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T14:30:16.123 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:16.123 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T14:30:16.203 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T14:30:16.203 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:16.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T14:30:16.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T14:30:16.353 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:16.353 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T14:30:16.419 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T14:30:16.525 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T14:30:16.526 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:16.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T14:30:16.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T14:30:16.709 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:16.709 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T14:30:16.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T14:30:16.883 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T14:30:16.883 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:16.883 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T14:30:16.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T14:30:17.063 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:17.063 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T14:30:17.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T14:30:17.235 INFO:teuthology.run_tasks:Running task ceph... 2024-04-23T14:30:17.333 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-23T14:30:17.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /var/log/ceph 2024-04-23T14:30:17.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2024-04-23T14:30:17.404 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-23T14:30:17.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T14:30:17.409 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T14:30:17.469 INFO:tasks.ceph:Creating extra log directories... 2024-04-23T14:30:17.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T14:30:17.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T14:30:17.543 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-23T14:30:17.544 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-23T14:30:17.544 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-23T14:30:17.544 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:17.544 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-23T14:30:17.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-23T14:30:17.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-23T14:30:17.847 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:17.847 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-23T14:30:17.889 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-23T14:30:18.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-23T14:30:18.134 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-23T14:30:18.134 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-23T14:30:18.135 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576', 'branch': 'wip-rishabh-testing-20240416.193735-3', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7669576', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735-3'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735-3', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735-3', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi059.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpnB0LGa4cS+W36Aj/as5sETZJ8Aj1oZaMwoiqsqf7kljO7nt46i/ouz7jwY3Pr3lLkNOX2bweR2pmXsxAiJoE=', 'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBByGZNsW/Qlm58n8wjoKYEesFXpNRdCXRv8uJXOaipMKNEMW9KHWFUXl4pdf8XasVMXR7jLR//U/s+yzg9lWBNI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_14:04:26', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-23T14:30:18.135 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T14:30:18.141 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-23T14:30:18.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T14:30:18.227 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T14:30:18.294 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:18.295 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T14:30:18.337 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T14:30:18.337 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-04-23T14:30:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T14:30:18.418 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:18.418 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-23T14:30:18.418 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:18.418 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:18.418 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-23 14:30:06.728556215 +0000 2024-04-23T14:30:18.419 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-23 14:27:18.411941759 +0000 2024-04-23T14:30:18.419 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-23 14:27:18.411941759 +0000 2024-04-23T14:30:18.419 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-23 14:27:18.411941759 +0000 2024-04-23T14:30:18.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T14:30:18.522 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-23T14:30:18.522 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-23T14:30:18.522 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000383821 s, 1.3 MB/s 2024-04-23T14:30:18.525 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T14:30:18.568 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-04-23T14:30:18.652 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T14:30:18.652 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:18.652 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-23T14:30:18.652 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:18.652 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:18.652 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-23 14:30:06.728556215 +0000 2024-04-23T14:30:18.653 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-23 14:27:19.030933262 +0000 2024-04-23T14:30:18.653 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-23 14:27:19.030933262 +0000 2024-04-23T14:30:18.653 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-23 14:27:19.030933262 +0000 2024-04-23T14:30:18.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T14:30:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-23T14:30:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-23T14:30:18.757 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000320021 s, 1.6 MB/s 2024-04-23T14:30:18.760 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T14:30:18.801 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-23 14:30:06.729556202 +0000 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-23 14:27:19.643924846 +0000 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-23 14:27:19.643924846 +0000 2024-04-23T14:30:18.882 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-23 14:27:19.643924846 +0000 2024-04-23T14:30:18.883 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T14:30:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-23T14:30:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-23T14:30:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.00031957 s, 1.6 MB/s 2024-04-23T14:30:18.986 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T14:30:19.027 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-04-23T14:30:19.111 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T14:30:19.111 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:19.112 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-23T14:30:19.112 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:19.112 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:19.112 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-04-23 14:30:06.730556188 +0000 2024-04-23T14:30:19.112 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-04-23 14:27:20.309915703 +0000 2024-04-23T14:30:19.112 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-04-23 14:27:20.309915703 +0000 2024-04-23T14:30:19.112 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-04-23 14:27:20.309915703 +0000 2024-04-23T14:30:19.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T14:30:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-04-23T14:30:19.213 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-04-23T14:30:19.214 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000308065 s, 1.7 MB/s 2024-04-23T14:30:19.217 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T14:30:19.259 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:19.259 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:19.259 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T14:30:19.301 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T14:30:19.301 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-23 14:30:04.949722302 +0000 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-23 14:27:10.186999482 +0000 2024-04-23T14:30:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-23 14:27:10.186999482 +0000 2024-04-23T14:30:19.383 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-23 14:27:10.186999482 +0000 2024-04-23T14:30:19.383 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T14:30:19.479 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-23T14:30:19.479 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-23T14:30:19.480 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00032741 s, 1.6 MB/s 2024-04-23T14:30:19.484 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T14:30:19.522 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-04-23T14:30:19.600 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T14:30:19.600 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-23T14:30:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-23 14:30:04.950722273 +0000 2024-04-23T14:30:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-23 14:27:10.775982242 +0000 2024-04-23T14:30:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-23 14:27:10.775982242 +0000 2024-04-23T14:30:19.601 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-23 14:27:10.775982242 +0000 2024-04-23T14:30:19.601 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T14:30:19.696 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-23T14:30:19.696 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-23T14:30:19.696 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000290132 s, 1.8 MB/s 2024-04-23T14:30:19.699 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T14:30:19.741 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-04-23T14:30:19.820 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T14:30:19.820 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:19.820 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-23T14:30:19.820 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-23 14:30:04.951722243 +0000 2024-04-23T14:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-23 14:27:11.409963686 +0000 2024-04-23T14:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-23 14:27:11.409963686 +0000 2024-04-23T14:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-23 14:27:11.409963686 +0000 2024-04-23T14:30:19.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T14:30:19.925 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-23T14:30:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-23T14:30:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000316354 s, 1.6 MB/s 2024-04-23T14:30:19.928 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T14:30:19.970 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-04-23T14:30:20.052 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T14:30:20.052 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:30:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-23T14:30:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:30:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:30:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-23 14:30:04.951722243 +0000 2024-04-23T14:30:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-23 14:27:12.038945276 +0000 2024-04-23T14:30:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-23 14:27:12.038945276 +0000 2024-04-23T14:30:20.053 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-23 14:27:12.038945276 +0000 2024-04-23T14:30:20.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T14:30:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-23T14:30:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-23T14:30:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000250496 s, 2.0 MB/s 2024-04-23T14:30:20.161 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T14:30:20.201 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:20.201 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-23T14:30:20.201 INFO:tasks.ceph:Generating config... 2024-04-23T14:30:20.203 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-23T14:30:20.203 INFO:tasks.ceph:[client] debug client = 20 2024-04-23T14:30:20.203 INFO:tasks.ceph:[client] debug ms = 1 2024-04-23T14:30:20.203 INFO:tasks.ceph:[client] pid file = 2024-04-23T14:30:20.203 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-23T14:30:20.203 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-23T14:30:20.203 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-23T14:30:20.203 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-23T14:30:20.203 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-23T14:30:20.203 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-23T14:30:20.203 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-23T14:30:20.203 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-23T14:30:20.203 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-23T14:30:20.204 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-23T14:30:20.204 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-23T14:30:20.204 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-23T14:30:20.204 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-23T14:30:20.204 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-23T14:30:20.204 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-23T14:30:20.205 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-23T14:30:20.205 INFO:tasks.ceph:Setting up mon.a... 2024-04-23T14:30:20.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-23T14:30:20.342 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.keyring 2024-04-23T14:30:20.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-23T14:30:20.449 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T14:30:20.510 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.59'), ('mon.b', '172.21.15.76'), ('mon.c', '[v2:172.21.15.76:3301,v1:172.21.15.76:6790]')] 2024-04-23T14:30:20.510 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-23T14:30:20.511 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:20.511 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T14:30:20.550 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.59 --add b 172.21.15.76 --addv c '[v2:172.21.15.76:3301,v1:172.21.15.76:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: generated fsid b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = quincy 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 0 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:fsid b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:last_changed 2024-04-23T14:30:20.682521+0000 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-04-23T14:30:20.682521+0000 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:min_mon_release 17 (quincy) 2024-04-23T14:30:20.682 INFO:teuthology.orchestra.run.smithi059.stdout:election_strategy: 1 2024-04-23T14:30:20.683 INFO:teuthology.orchestra.run.smithi059.stdout:0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-04-23T14:30:20.683 INFO:teuthology.orchestra.run.smithi059.stdout:1: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2024-04-23T14:30:20.683 INFO:teuthology.orchestra.run.smithi059.stdout:2: [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] mon.c 2024-04-23T14:30:20.683 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-23T14:30:20.686 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T14:30:20.727 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b8f41672-e126-44ae-8657-b688ecdd9da9... 2024-04-23T14:30:20.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:30:20.772 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-23T14:30:20.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.881 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.882 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.883 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-23T14:30:20.884 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-23T14:30:20.885 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:30:20.886 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-23T14:30:20.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.888 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:30:20.889 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2024-04-23T14:30:20.890 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:30:20.891 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:30:20.891 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:30:20.907 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-04-23T14:30:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-04-23T14:30:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:30:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-04-23T14:30:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:30:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:30:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-04-23T14:30:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:30:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-04-23T14:30:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:30:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:30:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-04-23T14:30:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.911 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:30:20.911 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.911 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-04-23T14:30:20.911 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-04-23T14:30:20.911 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-04-23T14:30:20.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-04-23T14:30:20.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.914 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:30:20.914 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.914 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-04-23T14:30:20.914 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: bdev async discard = True 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: bdev enable discard = True 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore allocator = bitmap 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-04-23T14:30:20.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:30:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.917 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:30:20.918 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-04-23T14:30:20.919 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-04-23T14:30:20.919 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:30:20.919 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.919 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:30:20.920 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:30:20.920 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:30:20.920 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.920 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:30:20.920 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:30:20.920 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:30:20.920 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: mon down mkfs grace = 300 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:30:20.921 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-04-23T14:30:20.922 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:30:20.923 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:30:20.937 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-23T14:30:20.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-23T14:30:21.048 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-23T14:30:21.048 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:21.048 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-23T14:30:21.087 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:21.087 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-23T14:30:21.169 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:30:21.170 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:21.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T14:30:21.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T14:30:21.307 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:21.307 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:21.394 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:30:21.394 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:21.395 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-23T14:30:21.395 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-23T14:30:21.517 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:21.517 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:21.615 INFO:tasks.ceph:Setting up mon nodes... 2024-04-23T14:30:21.615 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-23T14:30:21.615 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-23T14:30:21.737 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-23T14:30:21.744 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T14:30:21.935 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-23T14:30:21.943 INFO:tasks.ceph:Setting up mds nodes... 2024-04-23T14:30:21.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-23T14:30:22.076 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-23T14:30:22.083 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-23T14:30:22.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-23T14:30:22.284 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-23T14:30:22.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-23T14:30:22.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-23T14:30:22.514 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-23T14:30:22.521 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-23T14:30:22.593 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-23T14:30:22.728 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-23T14:30:22.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-23T14:30:22.810 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-23T14:30:22.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T14:30:22.911 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-23T14:30:22.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-23T14:30:23.086 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-23T14:30:23.127 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-23T14:30:23.241 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-23T14:30:23.280 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-23T14:30:23.382 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-23T14:30:23.419 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-23T14:30:23.421 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-23T14:30:23.422 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-23T14:30:23.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:23.512 INFO:tasks.ceph:role: osd.0 2024-04-23T14:30:23.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:30:23.512 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T14:30:23.600 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:23.601 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:24.013 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-23T14:30:24.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-23T14:30:24.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-23T14:30:24.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-23T14:30:24.257 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-23T14:30:24.348 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:24.348 INFO:tasks.ceph:role: osd.1 2024-04-23T14:30:24.348 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:30:24.349 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T14:30:24.419 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:24.419 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:24.419 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:24.419 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:24.419 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:24.419 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:24.419 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:24.420 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:24.420 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:24.420 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:24.774 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-23T14:30:24.779 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-23T14:30:24.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-23T14:30:24.895 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-23T14:30:24.967 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-23T14:30:25.034 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:25.034 INFO:tasks.ceph:role: osd.2 2024-04-23T14:30:25.034 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:30:25.034 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:25.104 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:25.483 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-23T14:30:25.488 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-23T14:30:25.488 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-23T14:30:25.606 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-23T14:30:25.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-23T14:30:25.745 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:25.745 INFO:tasks.ceph:role: osd.3 2024-04-23T14:30:25.745 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:30:25.745 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-23T14:30:25.819 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:25.820 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:26.229 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-04-23T14:30:26.236 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-04-23T14:30:26.236 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-23T14:30:26.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-23T14:30:26.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:26.662+0000 7f941031f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-23T14:30:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:26.663+0000 7f941031f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-23T14:30:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:26.663+0000 7f941031f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T14:30:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:26.663+0000 7f941031f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T14:30:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:26.663+0000 7f941031f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-23T14:30:26.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:26.667+0000 7f941031f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-23T14:30:29.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-23T14:30:29.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:29.876+0000 7fb652823640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-23T14:30:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:29.877+0000 7fb652823640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-23T14:30:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:29.877+0000 7fb652823640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T14:30:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:29.877+0000 7fb652823640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T14:30:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:29.877+0000 7fb652823640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-23T14:30:29.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:29.879+0000 7fb652823640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-23T14:30:33.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-23T14:30:33.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:33.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:33.328+0000 7efda2e5d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-23T14:30:33.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:33.329+0000 7efda2e5d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-23T14:30:33.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:33.329+0000 7efda2e5d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T14:30:33.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:33.329+0000 7efda2e5d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T14:30:33.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:33.329+0000 7efda2e5d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-23T14:30:33.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:33.332+0000 7efda2e5d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-23T14:30:36.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-23T14:30:36.394 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:36.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:36.527+0000 7ffa0e417640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-23T14:30:36.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:36.527+0000 7ffa0e417640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-23T14:30:36.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:36.527+0000 7ffa0e417640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T14:30:36.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:36.527+0000 7ffa0e417640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T14:30:36.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:36.527+0000 7ffa0e417640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-23T14:30:36.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:30:36.530+0000 7ffa0e417640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-23T14:30:39.488 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-23T14:30:39.581 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-23T14:30:39.662 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:39.662 INFO:tasks.ceph:role: osd.4 2024-04-23T14:30:39.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:30:39.662 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T14:30:39.752 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:39.753 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:39.753 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:39.753 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:39.753 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:39.753 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:39.753 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:39.753 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:39.754 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:39.754 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:40.160 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-23T14:30:40.165 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-23T14:30:40.165 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-23T14:30:40.300 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-23T14:30:40.378 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-23T14:30:40.450 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:40.450 INFO:tasks.ceph:role: osd.5 2024-04-23T14:30:40.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:30:40.450 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:40.540 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:40.892 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-23T14:30:40.898 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-23T14:30:40.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-23T14:30:41.006 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-23T14:30:41.074 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-23T14:30:41.146 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:41.146 INFO:tasks.ceph:role: osd.6 2024-04-23T14:30:41.147 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:30:41.147 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T14:30:41.219 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:41.219 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:41.220 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:41.220 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:41.220 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:41.600 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-23T14:30:41.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-23T14:30:41.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-23T14:30:41.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-23T14:30:41.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-23T14:30:41.863 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-23T14:30:41.863 INFO:tasks.ceph:role: osd.7 2024-04-23T14:30:41.864 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:30:41.864 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-23T14:30:41.944 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T14:30:41.945 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T14:30:42.358 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-23T14:30:42.365 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-23T14:30:42.365 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-23T14:30:42.513 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-23T14:30:42.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:42.735+0000 7f6fc58f0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-23T14:30:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:42.736+0000 7f6fc58f0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-23T14:30:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:42.736+0000 7f6fc58f0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T14:30:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:42.736+0000 7f6fc58f0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T14:30:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:42.736+0000 7f6fc58f0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-23T14:30:42.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:42.740+0000 7f6fc58f0640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-23T14:30:45.943 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-23T14:30:46.033 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:46.155+0000 7f048792c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-23T14:30:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:46.156+0000 7f048792c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-23T14:30:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:46.156+0000 7f048792c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T14:30:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:46.156+0000 7f048792c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T14:30:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:46.156+0000 7f048792c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-23T14:30:46.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:46.159+0000 7f048792c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-23T14:30:48.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:30:48.145 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:30:49.369 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-23T14:30:49.455 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:49.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:49.587+0000 7f4ded555640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-23T14:30:49.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:49.588+0000 7f4ded555640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-23T14:30:49.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:49.588+0000 7f4ded555640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T14:30:49.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:49.588+0000 7f4ded555640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T14:30:49.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:49.588+0000 7f4ded555640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-23T14:30:49.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:49.590+0000 7f4ded555640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-23T14:30:52.557 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-23T14:30:52.641 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:30:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:52.769+0000 7fb2ad957640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-23T14:30:52.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:52.770+0000 7fb2ad957640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-23T14:30:52.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:52.770+0000 7fb2ad957640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T14:30:52.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:52.770+0000 7fb2ad957640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T14:30:52.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:52.770+0000 7fb2ad957640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-23T14:30:52.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:30:52.772+0000 7fb2ad957640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-23T14:30:55.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-23T14:30:55.821 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-23T14:30:55.821 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:55.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-23T14:30:55.915 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:55.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-23T14:30:56.003 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:56.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-23T14:30:56.091 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:56.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-23T14:30:56.180 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:56.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-23T14:30:56.260 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:56.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-23T14:30:56.348 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:56.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-23T14:30:56.436 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:56.436 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-23T14:30:56.526 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:56.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-23T14:30:56.608 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:56.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-23T14:30:56.691 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:56.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-23T14:30:56.773 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:56.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-23T14:30:56.856 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:56.856 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-23T14:30:56.939 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:56.939 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-23T14:30:57.022 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:57.022 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-23T14:30:57.084 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:30:57.084 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-23T14:30:57.139 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:57.139 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-23T14:30:57.196 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:30:57.196 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-23T14:30:57.247 INFO:tasks.ceph:Adding keys to all mons... 2024-04-23T14:30:57.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T14:30:57.255 DEBUG:teuthology.orchestra.run.smithi076:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-23T14:30:57.327 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.y] 2024-04-23T14:30:57.327 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ== 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A== 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw== 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.0] 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow== 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.1] 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ== 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.2] 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ== 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.3] 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g== 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.x] 2024-04-23T14:30:57.328 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA== 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA== 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg== 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.4] 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA== 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.5] 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA== 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.6] 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA== 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.7] 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ== 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-04-23T14:30:57.329 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:30:57.330 INFO:teuthology.orchestra.run.smithi076.stdout:[client.2] 2024-04-23T14:30:57.330 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg== 2024-04-23T14:30:57.330 INFO:teuthology.orchestra.run.smithi076.stdout:[client.1] 2024-04-23T14:30:57.330 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q== 2024-04-23T14:30:57.330 INFO:teuthology.orchestra.run.smithi076.stdout:[client.3] 2024-04-23T14:30:57.330 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg== 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.y] 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ== 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A== 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw== 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.0] 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow== 2024-04-23T14:30:57.335 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.1] 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ== 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.2] 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ== 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.3] 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g== 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.x] 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA== 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA== 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg== 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.4] 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA== 2024-04-23T14:30:57.336 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.5] 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA== 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.6] 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA== 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.7] 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ== 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout:[client.2] 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg== 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout:[client.1] 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q== 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout:[client.3] 2024-04-23T14:30:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg== 2024-04-23T14:30:57.339 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:30:57.387 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:30:57.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:57.530 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:57.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:57.673 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:57.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:57.813 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:57.950 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:57.957 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.098 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.231 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.238 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:30:58.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T14:30:58.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:58.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:58.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:58.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-23T14:30:58.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.813 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:58.951 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:59.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:59.090 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:59.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:59.226 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T14:30:59.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.367 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.512 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.657 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T14:30:59.941 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-23T14:30:59.941 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-23T14:31:00.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T14:31:00.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-23T14:31:00.274 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-23T14:31:00.363 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T14:31:00.513 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-23T14:31:00.580 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-23T14:31:00.660 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-23T14:31:00.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-23T14:31:00.878 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:00.885 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-23T14:31:00.936 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-23T14:31:00.936 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-23T14:31:00.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-23T14:31:00.943 INFO:tasks.ceph.mon.a:Started 2024-04-23T14:31:00.943 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-23T14:31:00.944 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-23T14:31:00.948 INFO:tasks.ceph.mon.b:Started 2024-04-23T14:31:00.948 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-23T14:31:00.948 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-23T14:31:00.953 INFO:tasks.ceph.mon.c:Started 2024-04-23T14:31:00.953 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-23T14:31:00.953 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-23T14:31:00.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-23T14:31:00.960 INFO:tasks.ceph.mgr.y:Started 2024-04-23T14:31:00.960 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-23T14:31:00.960 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-23T14:31:00.964 INFO:tasks.ceph.mgr.x:Started 2024-04-23T14:31:00.964 INFO:tasks.ceph:Setting crush tunables to default 2024-04-23T14:31:00.964 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd crush tunables default 2024-04-23T14:31:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.143+0000 7f7520a00640 1 Processor -- start 2024-04-23T14:31:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.143+0000 7f7520a00640 1 -- start start 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7520a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 0x7f751c0d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7520a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f751c0cff50 con 0x7f751c160180 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7520a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f751c0d01c0 con 0x7f751c15cb20 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7520a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f751c15cf00 con 0x7f751c163770 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7511400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 0x7f751c0d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7511400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 0x7f751c0d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:52032/0 (socket says 172.21.15.59:52032) 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7511400640 1 -- 172.21.15.59:0/1301835361 learned_addr learned my addr 172.21.15.59:0/1301835361 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.145+0000 7f7519e00640 1 --1- 172.21.15.59:0/1301835361 >> v1:172.21.15.76:6789/0 conn(0x7f751c15cb20 0x7f751c0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:34990/0 (socket says 172.21.15.59:34990) 2024-04-23T14:31:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.146+0000 7f7511400640 1 -- 172.21.15.59:0/1301835361 >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 msgr2=0x7f751c0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-23T14:31:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.146+0000 7f7511400640 1 -- 172.21.15.59:0/1301835361 >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 msgr2=0x7f751c0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-23T14:31:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.146+0000 7f7511400640 1 --2- 172.21.15.59:0/1301835361 >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 0x7f751c0d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T14:31:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.146+0000 7f7511400640 1 --2- 172.21.15.59:0/1301835361 >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 0x7f751c0d1240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:31:01.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.156+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1255143892 0 0) 0x7f751c0cff50 con 0x7f751c160180 2024-04-23T14:31:01.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.156+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7500003520 con 0x7f751c160180 2024-04-23T14:31:01.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705873068 0 0) 0x7f751c0d01c0 con 0x7f751c15cb20 2024-04-23T14:31:01.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f751c0cff50 con 0x7f751c15cb20 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 <== mon.1 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2861948125 0 0) 0x7f751c0cff50 con 0x7f751c15cb20 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 msgr2=0x7f751c0d1240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 --2- 172.21.15.59:0/1301835361 >> v2:172.21.15.76:3300/0 conn(0x7f751c163770 0x7f751c0d1240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 >> v1:172.21.15.59:6789/0 conn(0x7f751c160180 legacy=0x7f751c0d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c15f6a0 con 0x7f751c15cb20 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 <== mon.1 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f750c0019f0 con 0x7f751c15cb20 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 >> v1:172.21.15.76:6789/0 conn(0x7f751c15cb20 legacy=0x7f751c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7500003b90 0x7f7500003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500004800 0x7f7500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f750000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f751c15cf00 con 0x7f7500005290 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f750000c1e0 con 0x7f7500004800 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.157+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f750000c3e0 con 0x7f7500003b90 2024-04-23T14:31:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.158+0000 7f7519e00640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7500003b90 0x7f7500003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.158+0000 7f751a800640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f750000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.158+0000 7f751a800640 1 -- 172.21.15.59:0/1301835361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7500003b90 msgr2=0x7f7500003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.158+0000 7f751a800640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7500003b90 0x7f7500003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.158+0000 7f751a800640 1 -- 172.21.15.59:0/1301835361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500004800 msgr2=0x7f7500004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.158+0000 7f751a800640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500004800 0x7f7500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.158+0000 7f751a800640 1 -- 172.21.15.59:0/1301835361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c0cff50 con 0x7f7500005290 2024-04-23T14:31:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.159+0000 7f751a800640 1 -- 172.21.15.59:0/1301835361 reap_dead start 2024-04-23T14:31:01.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.160+0000 7f751a800640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f750000b9c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7514001900 tx=0x7f7500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a065d217f7f37302 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.161+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514001520 con 0x7f7500005290 2024-04-23T14:31:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.161+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751402ad60 con 0x7f7500005290 2024-04-23T14:31:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.161+0000 7f7519400640 1 -- 172.21.15.59:0/1301835361 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514005310 con 0x7f7500005290 2024-04-23T14:31:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.162+0000 7f7520a00640 1 -- 172.21.15.59:0/1301835361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 msgr2=0x7f750000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.162+0000 7f7520a00640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f750000b9c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7514001900 tx=0x7f7500005830 comp rx=0 tx=0).stop 2024-04-23T14:31:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.162+0000 7f7520a00640 1 -- 172.21.15.59:0/1301835361 shutdown_connections 2024-04-23T14:31:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.162+0000 7f7520a00640 1 --2- 172.21.15.59:0/1301835361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f750000b9c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.162+0000 7f7520a00640 1 -- 172.21.15.59:0/1301835361 >> 172.21.15.59:0/1301835361 conn(0x7f751c06cc70 msgr2=0x7f751c06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.162+0000 7f7520a00640 1 -- 172.21.15.59:0/1301835361 shutdown_connections 2024-04-23T14:31:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.162+0000 7f7520a00640 1 -- 172.21.15.59:0/1301835361 wait complete. 2024-04-23T14:31:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.164+0000 7f7520a00640 1 Processor -- start 2024-04-23T14:31:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.165+0000 7f7520a00640 1 -- start start 2024-04-23T14:31:01.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7520a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f751c0c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:01.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7520a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0cb900 0x7f751c0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7520a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c0cd910 0x7f751c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7520a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f751c0cf9c0 con 0x7f7500005290 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7520a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f751c0cf840 con 0x7f751c0cd910 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7520a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f751c0cfb40 con 0x7f751c0cb900 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7519e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f751c0c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7511400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0cb900 0x7f751c0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7519e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f751c0c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34332/0 (socket says 172.21.15.59:34332) 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f7519e00640 1 -- 172.21.15.59:0/1797957433 learned_addr learned my addr 172.21.15.59:0/1797957433 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.166+0000 7f751a800640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c0cd910 0x7f751c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7511400640 1 -- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c0cd910 msgr2=0x7f751c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7511400640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c0cd910 0x7f751c0cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7511400640 1 -- 172.21.15.59:0/1797957433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 msgr2=0x7f751c0c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7511400640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f751c0c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7511400640 1 -- 172.21.15.59:0/1797957433 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c0ce3e0 con 0x7f751c0cb900 2024-04-23T14:31:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f751a800640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c0cd910 0x7f751c0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7511400640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0cb900 0x7f751c0cd3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f750400e600 tx=0x7f750400eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7512a00640 1 -- 172.21.15.59:0/1797957433 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504017070 con 0x7f751c0cb900 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7512a00640 1 -- 172.21.15.59:0/1797957433 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7504013610 con 0x7f751c0cb900 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7512a00640 1 -- 172.21.15.59:0/1797957433 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750400f040 con 0x7f751c0cb900 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.167+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c0ce6c0 con 0x7f751c0cb900 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.168+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c0c86e0 con 0x7f751c0cb900 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.168+0000 7f7512a00640 1 -- 172.21.15.59:0/1797957433 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f750401a070 con 0x7f751c0cb900 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.168+0000 7f7512a00640 1 -- 172.21.15.59:0/1797957433 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f750400c700 con 0x7f751c0cb900 2024-04-23T14:31:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.169+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751c15f010 con 0x7f751c0cb900 2024-04-23T14:31:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.172+0000 7f7512a00640 1 -- 172.21.15.59:0/1797957433 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f751c15f010 con 0x7f751c0cb900 2024-04-23T14:31:01.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.246+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f751c068a80 con 0x7f751c0cb900 2024-04-23T14:31:01.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.253+0000 7f7512a00640 1 -- 172.21.15.59:0/1797957433 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f7504013b40 con 0x7f751c0cb900 2024-04-23T14:31:01.254 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2024-04-23T14:31:01.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.254+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0cb900 msgr2=0x7f751c0cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.254+0000 7f7520a00640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0cb900 0x7f751c0cd3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f750400e600 tx=0x7f750400eac0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.255+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 shutdown_connections 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.255+0000 7f7520a00640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0cb900 0x7f751c0cd3d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.255+0000 7f7520a00640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c0cd910 0x7f751c0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.255+0000 7f7520a00640 1 --2- 172.21.15.59:0/1797957433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7500005290 0x7f751c0c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.255+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 >> 172.21.15.59:0/1797957433 conn(0x7f751c06cc70 msgr2=0x7f751c0d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.255+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 shutdown_connections 2024-04-23T14:31:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:01.255+0000 7f7520a00640 1 -- 172.21.15.59:0/1797957433 wait complete. 2024-04-23T14:31:01.274 INFO:tasks.ceph:check_enable_crimson: False 2024-04-23T14:31:01.274 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-23T14:31:01.274 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:31:01.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-23T14:31:01.322 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:01.321+0000 7f998b211040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:31:01.350 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:31:01.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-23T14:31:01.465 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:31:01.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-23T14:31:01.467 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:01.468+0000 7f77d7d41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:31:01.509 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:01.508+0000 7f998b211040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:01.535 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:31:01.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-23T14:31:01.580 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:01.577+0000 7f998b211040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:31:01.650 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:31:01.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-23T14:31:01.662 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:01.662+0000 7f77d7d41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:01.725 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:31:01.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-23T14:31:01.730 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:01.730+0000 7f77d7d41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:31:01.795 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:31:01.795 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-23T14:31:01.884 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:01.883+0000 7f998b211040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:31:01.911 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:31:01.911 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-23T14:31:01.940 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:01.939+0000 7f998b211040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:31:02.032 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new c32b555b-b29b-44b0-b863-8a04bc4adf5b 0 2024-04-23T14:31:02.033 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:02.032+0000 7f77d7d41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:31:02.089 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:02.089+0000 7f77d7d41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:31:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.215+0000 7fada6600640 1 Processor -- start 2024-04-23T14:31:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.216+0000 7fada6600640 1 -- start start 2024-04-23T14:31:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.217+0000 7fada6600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fada006c0e0 0x7fada00cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.217+0000 7fada6600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada00ce390 con 0x7fada00cdc40 2024-04-23T14:31:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.217+0000 7fada6600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada00ce670 con 0x7fada015e920 2024-04-23T14:31:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.217+0000 7fada6600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fada00b0bb0 con 0x7fada006c0e0 2024-04-23T14:31:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.217+0000 7fad9fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fada006c0e0 0x7fada00cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.217+0000 7fad9fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fada006c0e0 0x7fada00cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46348/0 (socket says 172.21.15.76:46348) 2024-04-23T14:31:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.217+0000 7fad9fe00640 1 -- 172.21.15.76:0/86163776 learned_addr learned my addr 172.21.15.76:0/86163776 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.218+0000 7fad9fe00640 1 -- 172.21.15.76:0/86163776 >> v1:172.21.15.76:6789/0 conn(0x7fada015e920 legacy=0x7fada006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.218+0000 7fad9fe00640 1 -- 172.21.15.76:0/86163776 >> v1:172.21.15.59:6789/0 conn(0x7fada00cdc40 legacy=0x7fada0168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.218+0000 7fad9fe00640 1 -- 172.21.15.76:0/86163776 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada00ce950 con 0x7fada006c0e0 2024-04-23T14:31:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.219+0000 7fad9fe00640 1 --2- 172.21.15.76:0/86163776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada006c0e0 0x7fada00cd5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009870 tx=0x7fad8c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27035cdeff7fa0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.219+0000 7fad9f400640 1 -- 172.21.15.76:0/86163776 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c037070 con 0x7fada006c0e0 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.219+0000 7fad9f400640 1 -- 172.21.15.76:0/86163776 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c032440 con 0x7fada006c0e0 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.219+0000 7fad9f400640 1 -- 172.21.15.76:0/86163776 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c032810 con 0x7fada006c0e0 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.220+0000 7fada6600640 1 -- 172.21.15.76:0/86163776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada006c0e0 msgr2=0x7fada00cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.220+0000 7fada6600640 1 --2- 172.21.15.76:0/86163776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada006c0e0 0x7fada00cd5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009870 tx=0x7fad8c02a180 comp rx=0 tx=0).stop 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.220+0000 7fada6600640 1 -- 172.21.15.76:0/86163776 shutdown_connections 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.220+0000 7fada6600640 1 --2- 172.21.15.76:0/86163776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada006c0e0 0x7fada00cd5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.220+0000 7fada6600640 1 -- 172.21.15.76:0/86163776 >> 172.21.15.76:0/86163776 conn(0x7fada00b2f10 msgr2=0x7fada00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.220+0000 7fada6600640 1 -- 172.21.15.76:0/86163776 shutdown_connections 2024-04-23T14:31:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.220+0000 7fada6600640 1 -- 172.21.15.76:0/86163776 wait complete. 2024-04-23T14:31:02.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.221+0000 7fada6600640 1 Processor -- start 2024-04-23T14:31:02.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.222+0000 7fada6600640 1 -- start start 2024-04-23T14:31:02.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.222+0000 7fada6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada006c0e0 0x7fada00b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.222+0000 7fada6600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fada00cdc40 0x7fada00ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.222+0000 7fada6600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada015e920 0x7fada00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fada6600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fada00ccff0 con 0x7fada006c0e0 2024-04-23T14:31:02.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fada6600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fada00cec30 con 0x7fada015e920 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fada6600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fada00cd170 con 0x7fada00cdc40 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fada00cdc40 0x7fada00ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fada5c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada015e920 0x7fada00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fada00cdc40 0x7fada00ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:40778/0 (socket says 172.21.15.76:40778) 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 -- 172.21.15.76:0/4135136049 learned_addr learned my addr 172.21.15.76:0/4135136049 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fada5200640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada006c0e0 0x7fada00b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 -- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada015e920 msgr2=0x7fada00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada015e920 0x7fada00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 -- 172.21.15.76:0/4135136049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada006c0e0 msgr2=0x7fada00b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada006c0e0 0x7fada00b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fad9fe00640 1 -- 172.21.15.76:0/4135136049 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada00c94e0 con 0x7fada00cdc40 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.223+0000 7fada5c00640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada015e920 0x7fada00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fad9fe00640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fada00cdc40 0x7fada00ca870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009870 tx=0x7fad8c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:02.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fad9cc00640 1 -- 172.21.15.76:0/4135136049 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c041020 con 0x7fada00cdc40 2024-04-23T14:31:02.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fad9cc00640 1 -- 172.21.15.76:0/4135136049 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c030970 con 0x7fada00cdc40 2024-04-23T14:31:02.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fad9cc00640 1 -- 172.21.15.76:0/4135136049 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c037040 con 0x7fada00cdc40 2024-04-23T14:31:02.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada00c97c0 con 0x7fada00cdc40 2024-04-23T14:31:02.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fada015fc70 con 0x7fada00cdc40 2024-04-23T14:31:02.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fad9cc00640 1 -- 172.21.15.76:0/4135136049 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fad8c044020 con 0x7fada00cdc40 2024-04-23T14:31:02.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.224+0000 7fad9cc00640 1 -- 172.21.15.76:0/4135136049 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fad8c046400 con 0x7fada00cdc40 2024-04-23T14:31:02.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.225+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada004fd90 con 0x7fada00cdc40 2024-04-23T14:31:02.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.227+0000 7fad9cc00640 1 -- 172.21.15.76:0/4135136049 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fad8c04b030 con 0x7fada00cdc40 2024-04-23T14:31:02.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.305+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c32b555b-b29b-44b0-b863-8a04bc4adf5b", "id": 0} v 0) -- 0x7fada0162040 con 0x7fada00cdc40 2024-04-23T14:31:02.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.313+0000 7fad9cc00640 1 -- 172.21.15.76:0/4135136049 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c32b555b-b29b-44b0-b863-8a04bc4adf5b", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fad8c03d940 con 0x7fada00cdc40 2024-04-23T14:31:02.313 INFO:teuthology.orchestra.run.smithi076.stdout:0 2024-04-23T14:31:02.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.314+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fada00cdc40 msgr2=0x7fada00ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.314+0000 7fada6600640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fada00cdc40 0x7fada00ca870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009870 tx=0x7fad8c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.314+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 shutdown_connections 2024-04-23T14:31:02.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.314+0000 7fada6600640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fada00cdc40 0x7fada00ca870 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.314+0000 7fada6600640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fada015e920 0x7fada00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.314+0000 7fada6600640 1 --2- 172.21.15.76:0/4135136049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada006c0e0 0x7fada00b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.314+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 >> 172.21.15.76:0/4135136049 conn(0x7fada00b2f10 msgr2=0x7fada00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:02.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.315+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 shutdown_connections 2024-04-23T14:31:02.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.315+0000 7fada6600640 1 -- 172.21.15.76:0/4135136049 wait complete. 2024-04-23T14:31:02.334 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 722b82c6-69c6-4a02-ae94-6aad5483b240 1 2024-04-23T14:31:02.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.514+0000 7fe088600640 1 Processor -- start 2024-04-23T14:31:02.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.514+0000 7fe088600640 1 -- start start 2024-04-23T14:31:02.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.516+0000 7fe088600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0800d1950 0x7fe080169e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.516+0000 7fe088600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0800cff50 con 0x7fe08015cb20 2024-04-23T14:31:02.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.516+0000 7fe088600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0800d01c0 con 0x7fe080163770 2024-04-23T14:31:02.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.516+0000 7fe088600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe08015cf00 con 0x7fe0800d1950 2024-04-23T14:31:02.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.516+0000 7fe086600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0800d1950 0x7fe080169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.516+0000 7fe086600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0800d1950 0x7fe080169e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46368/0 (socket says 172.21.15.76:46368) 2024-04-23T14:31:02.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.516+0000 7fe086600640 1 -- 172.21.15.76:0/1792564393 learned_addr learned my addr 172.21.15.76:0/1792564393 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:02.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.517+0000 7fe086600640 1 -- 172.21.15.76:0/1792564393 >> v1:172.21.15.76:6789/0 conn(0x7fe080163770 legacy=0x7fe0800d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.517+0000 7fe086600640 1 -- 172.21.15.76:0/1792564393 >> v1:172.21.15.59:6789/0 conn(0x7fe08015cb20 legacy=0x7fe0800cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:02.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.517+0000 7fe086600640 1 -- 172.21.15.76:0/1792564393 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08015f6a0 con 0x7fe0800d1950 2024-04-23T14:31:02.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.518+0000 7fe086600640 1 --2- 172.21.15.76:0/1792564393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0800d1950 0x7fe080169e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe078009e50 tx=0x7fe07802a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2150fe3a161f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:02.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.518+0000 7fe07fe00640 1 -- 172.21.15.76:0/1792564393 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078036070 con 0x7fe0800d1950 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.518+0000 7fe07fe00640 1 -- 172.21.15.76:0/1792564393 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078031600 con 0x7fe0800d1950 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.518+0000 7fe07fe00640 1 -- 172.21.15.76:0/1792564393 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780319d0 con 0x7fe0800d1950 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.518+0000 7fe088600640 1 -- 172.21.15.76:0/1792564393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0800d1950 msgr2=0x7fe080169e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.518+0000 7fe088600640 1 --2- 172.21.15.76:0/1792564393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0800d1950 0x7fe080169e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe078009e50 tx=0x7fe07802a6d0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.519+0000 7fe088600640 1 -- 172.21.15.76:0/1792564393 shutdown_connections 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.519+0000 7fe088600640 1 --2- 172.21.15.76:0/1792564393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0800d1950 0x7fe080169e10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.519+0000 7fe088600640 1 -- 172.21.15.76:0/1792564393 >> 172.21.15.76:0/1792564393 conn(0x7fe08006cc70 msgr2=0x7fe08006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.519+0000 7fe088600640 1 -- 172.21.15.76:0/1792564393 shutdown_connections 2024-04-23T14:31:02.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.519+0000 7fe088600640 1 -- 172.21.15.76:0/1792564393 wait complete. 2024-04-23T14:31:02.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.520+0000 7fe088600640 1 Processor -- start 2024-04-23T14:31:02.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.521+0000 7fe088600640 1 -- start start 2024-04-23T14:31:02.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe088600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0800d1950 0x7fe0800cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe088600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe08015cb20 0x7fe0800ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe088600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe080163770 0x7fe0800caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe088600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe080160300 con 0x7fe0800d1950 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe088600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe080160180 con 0x7fe080163770 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe088600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe080160480 con 0x7fe08015cb20 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe080163770 0x7fe0800caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe080163770 0x7fe0800caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46370/0 (socket says 172.21.15.76:46370) 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 -- 172.21.15.76:0/894320903 learned_addr learned my addr 172.21.15.76:0/894320903 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe085200640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe08015cb20 0x7fe0800ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe085c00640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0800d1950 0x7fe0800cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 -- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe08015cb20 msgr2=0x7fe0800ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe08015cb20 0x7fe0800ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 -- 172.21.15.76:0/894320903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0800d1950 msgr2=0x7fe0800cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0800d1950 0x7fe0800cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.522+0000 7fe086600640 1 -- 172.21.15.76:0/894320903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08014b240 con 0x7fe080163770 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.523+0000 7fe086600640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe080163770 0x7fe0800caa70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe078009e50 tx=0x7fe078004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:02.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.523+0000 7fe07d600640 1 -- 172.21.15.76:0/894320903 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078036050 con 0x7fe080163770 2024-04-23T14:31:02.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.523+0000 7fe07d600640 1 -- 172.21.15.76:0/894320903 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078042070 con 0x7fe080163770 2024-04-23T14:31:02.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.523+0000 7fe07d600640 1 -- 172.21.15.76:0/894320903 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078009510 con 0x7fe080163770 2024-04-23T14:31:02.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.523+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08014a1a0 con 0x7fe080163770 2024-04-23T14:31:02.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.523+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08014a760 con 0x7fe080163770 2024-04-23T14:31:02.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.524+0000 7fe07d600640 1 -- 172.21.15.76:0/894320903 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe078007bd0 con 0x7fe080163770 2024-04-23T14:31:02.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.524+0000 7fe07d600640 1 -- 172.21.15.76:0/894320903 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fe07803d800 con 0x7fe080163770 2024-04-23T14:31:02.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.524+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe080068a80 con 0x7fe080163770 2024-04-23T14:31:02.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.530+0000 7fe07d600640 1 -- 172.21.15.76:0/894320903 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe078047460 con 0x7fe080163770 2024-04-23T14:31:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.608+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "722b82c6-69c6-4a02-ae94-6aad5483b240", "id": 1} v 0) -- 0x7fe0800d29c0 con 0x7fe080163770 2024-04-23T14:31:02.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.616+0000 7fe07d600640 1 -- 172.21.15.76:0/894320903 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "722b82c6-69c6-4a02-ae94-6aad5483b240", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fe078047640 con 0x7fe080163770 2024-04-23T14:31:02.616 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe080163770 msgr2=0x7fe0800caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe080163770 0x7fe0800caa70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe078009e50 tx=0x7fe078004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 shutdown_connections 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe08015cb20 0x7fe0800ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe080163770 0x7fe0800caa70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 --2- 172.21.15.76:0/894320903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0800d1950 0x7fe0800cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 >> 172.21.15.76:0/894320903 conn(0x7fe08006cc70 msgr2=0x7fe0800b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 shutdown_connections 2024-04-23T14:31:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.618+0000 7fe088600640 1 -- 172.21.15.76:0/894320903 wait complete. 2024-04-23T14:31:02.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6 2 2024-04-23T14:31:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.804+0000 7f0bf1400640 1 Processor -- start 2024-04-23T14:31:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.804+0000 7f0bf1400640 1 -- start start 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.805+0000 7f0bf1400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bec145e10 0x7f0bec06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.805+0000 7f0bf1400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.805+0000 7f0bf1400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bec0cfc20 con 0x7f0bec071d70 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.805+0000 7f0bf1400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0bec0cff00 con 0x7f0bec145e10 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.805+0000 7f0bf1400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0bec0d01e0 con 0x7f0bec0d0cc0 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.805+0000 7f0bea800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bec145e10 0x7f0bec06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0be3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0bea800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bec145e10 0x7f0bec06f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46376/0 (socket says 172.21.15.76:46376) 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0bea800640 1 -- 172.21.15.76:0/2082987309 learned_addr learned my addr 172.21.15.76:0/2082987309 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:02.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0bea800640 1 -- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 msgr2=0x7f0bec0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:02.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0bea800640 1 --2- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec0cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0bea800640 1 -- 172.21.15.76:0/2082987309 >> v1:172.21.15.59:6789/0 conn(0x7f0bec071d70 legacy=0x7f0bec15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:02.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0bea800640 1 -- 172.21.15.76:0/2082987309 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec0d04c0 con 0x7f0bec145e10 2024-04-23T14:31:02.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.806+0000 7f0be3e00640 1 --2- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.807+0000 7f0bea800640 1 --2- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec06f130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0098a0 tx=0x7f0bdc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2fb57505a82004 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.807+0000 7f0be9e00640 1 -- 172.21.15.76:0/2082987309 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc037070 con 0x7f0bec145e10 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.807+0000 7f0be9e00640 1 -- 172.21.15.76:0/2082987309 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bdc032440 con 0x7f0bec145e10 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.807+0000 7f0be9e00640 1 -- 172.21.15.76:0/2082987309 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc032810 con 0x7f0bec145e10 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 -- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 msgr2=0x7f0bec06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 --2- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec06f130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0098a0 tx=0x7f0bdc02a250 comp rx=0 tx=0).stop 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 -- 172.21.15.76:0/2082987309 shutdown_connections 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 --2- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 --2- 172.21.15.76:0/2082987309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec06f130 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 -- 172.21.15.76:0/2082987309 >> 172.21.15.76:0/2082987309 conn(0x7f0bec067d90 msgr2=0x7f0bec06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:02.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 -- 172.21.15.76:0/2082987309 shutdown_connections 2024-04-23T14:31:02.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.808+0000 7f0bf1400640 1 -- 172.21.15.76:0/2082987309 wait complete. 2024-04-23T14:31:02.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.810+0000 7f0bf1400640 1 Processor -- start 2024-04-23T14:31:02.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.810+0000 7f0bf1400640 1 -- start start 2024-04-23T14:31:02.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0bf1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bec071d70 0x7f0bec15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0bf1400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec1e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0bf1400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec1e8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0bf1400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec0cfa00 con 0x7f0bec071d70 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0bf1400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec0cf880 con 0x7f0bec145e10 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0bf1400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0bec1461f0 con 0x7f0bec0d0cc0 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0beb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec1e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0be3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec1e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0beb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec1e8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46380/0 (socket says 172.21.15.76:46380) 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0be3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec1e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:40800/0 (socket says 172.21.15.76:40800) 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0beb200640 1 -- 172.21.15.76:0/2553422335 learned_addr learned my addr 172.21.15.76:0/2553422335 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.811+0000 7f0bea800640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bec071d70 0x7f0bec15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:02.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0beb200640 1 -- 172.21.15.76:0/2553422335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 msgr2=0x7f0bec1e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0beb200640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec1e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0beb200640 1 -- 172.21.15.76:0/2553422335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bec071d70 msgr2=0x7f0bec15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0beb200640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bec071d70 0x7f0bec15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0beb200640 1 -- 172.21.15.76:0/2553422335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec0b3530 con 0x7f0bec145e10 2024-04-23T14:31:02.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0beb200640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec1e8590 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0be400c780 tx=0x7f0be400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0be2a00640 1 -- 172.21.15.76:0/2553422335 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4004b60 con 0x7f0bec145e10 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0be2a00640 1 -- 172.21.15.76:0/2553422335 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be4004d00 con 0x7f0bec145e10 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0be2a00640 1 -- 172.21.15.76:0/2553422335 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be400aa90 con 0x7f0bec145e10 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bec0b3810 con 0x7f0bec145e10 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.812+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bec0b3dd0 con 0x7f0bec145e10 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.813+0000 7f0be2a00640 1 -- 172.21.15.76:0/2553422335 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0be4006b50 con 0x7f0bec145e10 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.813+0000 7f0be2a00640 1 -- 172.21.15.76:0/2553422335 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0be4005450 con 0x7f0bec145e10 2024-04-23T14:31:02.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.813+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bec068a80 con 0x7f0bec145e10 2024-04-23T14:31:02.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.818+0000 7f0be2a00640 1 -- 172.21.15.76:0/2553422335 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0be4020070 con 0x7f0bec145e10 2024-04-23T14:31:02.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.896+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6", "id": 2} v 0) -- 0x7f0bec06f2f0 con 0x7f0bec145e10 2024-04-23T14:31:02.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.903+0000 7f0be2a00640 1 -- 172.21.15.76:0/2553422335 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0be4005710 con 0x7f0bec145e10 2024-04-23T14:31:02.903 INFO:teuthology.orchestra.run.smithi076.stdout:2 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 msgr2=0x7f0bec1e8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec1e8590 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0be400c780 tx=0x7f0be400cc40 comp rx=0 tx=0).stop 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 shutdown_connections 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bec0d0cc0 0x7f0bec1e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bec145e10 0x7f0bec1e8590 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 --2- 172.21.15.76:0/2553422335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bec071d70 0x7f0bec15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 >> 172.21.15.76:0/2553422335 conn(0x7f0bec067d90 msgr2=0x7f0bec15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 shutdown_connections 2024-04-23T14:31:02.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:02.905+0000 7f0bf1400640 1 -- 172.21.15.76:0/2553422335 wait complete. 2024-04-23T14:31:02.922 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new ee2b3514-932e-467d-9511-732911d272d4 3 2024-04-23T14:31:02.948 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:02.947+0000 7f998b211040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:31:03.056 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:03.055+0000 7f998b211040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:03.074 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:03.074+0000 7f77d7d41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:31:03.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.104+0000 7f75f2600640 1 Processor -- start 2024-04-23T14:31:03.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.104+0000 7f75f2600640 1 -- start start 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.105+0000 7f75f2600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75ec163770 0x7f75ec0d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.105+0000 7f75f2600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec160180 0x7f75ec0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.105+0000 7f75f2600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75ec0cff50 con 0x7f75ec15cb20 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.105+0000 7f75f2600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f75ec15cf00 con 0x7f75ec163770 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.105+0000 7f75f2600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec0d01c0 con 0x7f75ec160180 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec160180 0x7f75ec0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec160180 0x7f75ec0d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:40816/0 (socket says 172.21.15.76:40816) 2024-04-23T14:31:03.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75ebe00640 1 -- 172.21.15.76:0/381119242 learned_addr learned my addr 172.21.15.76:0/381119242 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:03.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75e3e00640 1 --2- 172.21.15.76:0/381119242 >> v2:172.21.15.59:3300/0 conn(0x7f75ec163770 0x7f75ec0d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75ebe00640 1 -- 172.21.15.76:0/381119242 >> v2:172.21.15.59:3300/0 conn(0x7f75ec163770 msgr2=0x7f75ec0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:03.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75ebe00640 1 --2- 172.21.15.76:0/381119242 >> v2:172.21.15.59:3300/0 conn(0x7f75ec163770 0x7f75ec0d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75ebe00640 1 -- 172.21.15.76:0/381119242 >> v1:172.21.15.59:6789/0 conn(0x7f75ec15cb20 legacy=0x7f75ec0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.106+0000 7f75ebe00640 1 -- 172.21.15.76:0/381119242 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec15f6a0 con 0x7f75ec160180 2024-04-23T14:31:03.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.108+0000 7f75ebe00640 1 --2- 172.21.15.76:0/381119242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec160180 0x7f75ec0d1780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009e50 tx=0x7f75e402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef36ba4a27297ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:03.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.108+0000 7f75eaa00640 1 -- 172.21.15.76:0/381119242 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4036070 con 0x7f75ec160180 2024-04-23T14:31:03.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.108+0000 7f75eaa00640 1 -- 172.21.15.76:0/381119242 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e4031600 con 0x7f75ec160180 2024-04-23T14:31:03.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.108+0000 7f75eaa00640 1 -- 172.21.15.76:0/381119242 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e40319d0 con 0x7f75ec160180 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 -- 172.21.15.76:0/381119242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec160180 msgr2=0x7f75ec0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 --2- 172.21.15.76:0/381119242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec160180 0x7f75ec0d1780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009e50 tx=0x7f75e402a680 comp rx=0 tx=0).stop 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 -- 172.21.15.76:0/381119242 shutdown_connections 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 --2- 172.21.15.76:0/381119242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec160180 0x7f75ec0d1780 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 --2- 172.21.15.76:0/381119242 >> v2:172.21.15.59:3300/0 conn(0x7f75ec163770 0x7f75ec0d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 -- 172.21.15.76:0/381119242 >> 172.21.15.76:0/381119242 conn(0x7f75ec06cc70 msgr2=0x7f75ec06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 -- 172.21.15.76:0/381119242 shutdown_connections 2024-04-23T14:31:03.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.109+0000 7f75f2600640 1 -- 172.21.15.76:0/381119242 wait complete. 2024-04-23T14:31:03.110 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:03.108+0000 7f998b211040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:31:03.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.111+0000 7f75f2600640 1 Processor -- start 2024-04-23T14:31:03.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.111+0000 7f75f2600640 1 -- start start 2024-04-23T14:31:03.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75f2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15cb20 0x7f75ec14ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75f2600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75ec160180 0x7f75ec1e9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75f2600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec163770 0x7f75ec159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75f2600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec0d2d60 con 0x7f75ec15cb20 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75f2600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec0d2be0 con 0x7f75ec160180 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75f2600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec0d2ee0 con 0x7f75ec163770 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75e3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75ec160180 0x7f75ec1e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec163770 0x7f75ec159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75e3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75ec160180 0x7f75ec1e9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46390/0 (socket says 172.21.15.76:46390) 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75e3e00640 1 -- 172.21.15.76:0/289751811 learned_addr learned my addr 172.21.15.76:0/289751811 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.112+0000 7f75eb400640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15cb20 0x7f75ec14ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3e00640 1 -- 172.21.15.76:0/289751811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec163770 msgr2=0x7f75ec159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3e00640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec163770 0x7f75ec159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3e00640 1 -- 172.21.15.76:0/289751811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15cb20 msgr2=0x7f75ec14ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3e00640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15cb20 0x7f75ec14ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3e00640 1 -- 172.21.15.76:0/289751811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec1eadd0 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3e00640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75ec160180 0x7f75ec1e9270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f75d400e8a0 tx=0x7f75d400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3400640 1 -- 172.21.15.76:0/289751811 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4017070 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3400640 1 -- 172.21.15.76:0/289751811 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d400c940 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75e3400640 1 -- 172.21.15.76:0/289751811 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d400cd30 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec1e9d30 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.113+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec1ea2f0 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.114+0000 7f75e3400640 1 -- 172.21.15.76:0/289751811 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f75d401a070 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.114+0000 7f75e3400640 1 -- 172.21.15.76:0/289751811 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f75d4006020 con 0x7f75ec160180 2024-04-23T14:31:03.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.114+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec068a80 con 0x7f75ec160180 2024-04-23T14:31:03.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.117+0000 7f75e3400640 1 -- 172.21.15.76:0/289751811 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f75d4006480 con 0x7f75ec160180 2024-04-23T14:31:03.189 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:03.189+0000 7f77d7d41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.197+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ee2b3514-932e-467d-9511-732911d272d4", "id": 3} v 0) -- 0x7f75ec0d29c0 con 0x7f75ec160180 2024-04-23T14:31:03.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.203+0000 7f75e3400640 1 -- 172.21.15.76:0/289751811 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ee2b3514-932e-467d-9511-732911d272d4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f75d4006700 con 0x7f75ec160180 2024-04-23T14:31:03.203 INFO:teuthology.orchestra.run.smithi076.stdout:3 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75ec160180 msgr2=0x7f75ec1e9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75ec160180 0x7f75ec1e9270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f75d400e8a0 tx=0x7f75d400ed60 comp rx=0 tx=0).stop 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 shutdown_connections 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75ec163770 0x7f75ec159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75ec160180 0x7f75ec1e9270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 --2- 172.21.15.76:0/289751811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15cb20 0x7f75ec14ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 >> 172.21.15.76:0/289751811 conn(0x7f75ec06cc70 msgr2=0x7f75ec15d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 shutdown_connections 2024-04-23T14:31:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.205+0000 7f75f2600640 1 -- 172.21.15.76:0/289751811 wait complete. 2024-04-23T14:31:03.220 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 5796b331-54b0-4a70-a36b-80cb3c7ad4bb 4 2024-04-23T14:31:03.246 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:03.246+0000 7f77d7d41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:31:03.309 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:03.308+0000 7f998b211040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:31:03.368 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:03.366+0000 7f998b211040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:31:03.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.391+0000 7f7183200640 1 Processor -- start 2024-04-23T14:31:03.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.391+0000 7f7183200640 1 -- start start 2024-04-23T14:31:03.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f7183200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71840d15b0 0x7f71840cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f7183200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71840b4f20 0x7f71840cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f7183200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71840b5da0 con 0x7f718415ed20 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f7183200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f718415f100 con 0x7f71840b4f20 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f7183200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f71840d0a20 con 0x7f71840d15b0 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f717be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71840d15b0 0x7f71840cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f717be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71840d15b0 0x7f71840cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:40840/0 (socket says 172.21.15.76:40840) 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f717be00640 1 -- 172.21.15.76:0/2328047944 learned_addr learned my addr 172.21.15.76:0/2328047944 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f717be00640 1 -- 172.21.15.76:0/2328047944 >> v2:172.21.15.59:3300/0 conn(0x7f71840b4f20 msgr2=0x7f71840cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f717be00640 1 --2- 172.21.15.76:0/2328047944 >> v2:172.21.15.59:3300/0 conn(0x7f71840b4f20 0x7f71840cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f717be00640 1 -- 172.21.15.76:0/2328047944 >> v1:172.21.15.59:6789/0 conn(0x7f718415ed20 legacy=0x7f71840d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:03.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.393+0000 7f717be00640 1 -- 172.21.15.76:0/2328047944 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71840d2ed0 con 0x7f71840d15b0 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.394+0000 7f717be00640 1 --2- 172.21.15.76:0/2328047944 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71840d15b0 0x7f71840cf280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f716c009870 tx=0x7f716c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16daa3c07d3403e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7181400640 1 -- 172.21.15.76:0/2328047944 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c037070 con 0x7f71840d15b0 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7181400640 1 -- 172.21.15.76:0/2328047944 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c02ae10 con 0x7f71840d15b0 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7181400640 1 -- 172.21.15.76:0/2328047944 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c032660 con 0x7f71840d15b0 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7183200640 1 -- 172.21.15.76:0/2328047944 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71840d15b0 msgr2=0x7f71840cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7183200640 1 --2- 172.21.15.76:0/2328047944 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71840d15b0 0x7f71840cf280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f716c009870 tx=0x7f716c02a100 comp rx=0 tx=0).stop 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7183200640 1 -- 172.21.15.76:0/2328047944 shutdown_connections 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7183200640 1 --2- 172.21.15.76:0/2328047944 >> v2:172.21.15.59:3300/0 conn(0x7f71840b4f20 0x7f71840cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7183200640 1 --2- 172.21.15.76:0/2328047944 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71840d15b0 0x7f71840cf280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.395+0000 7f7183200640 1 -- 172.21.15.76:0/2328047944 >> 172.21.15.76:0/2328047944 conn(0x7f718408bf90 msgr2=0x7f71840d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.396+0000 7f7183200640 1 -- 172.21.15.76:0/2328047944 shutdown_connections 2024-04-23T14:31:03.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.396+0000 7f7183200640 1 -- 172.21.15.76:0/2328047944 wait complete. 2024-04-23T14:31:03.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.397+0000 7f7183200640 1 Processor -- start 2024-04-23T14:31:03.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.397+0000 7f7183200640 1 -- start start 2024-04-23T14:31:03.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f7183200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71840b4f20 0x7f718415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f7183200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71840d15b0 0x7f71840c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f7183200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f718415ed20 0x7f71840c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f7183200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71840b3600 con 0x7f71840b4f20 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f7183200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71840d3490 con 0x7f71840d15b0 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f7183200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f71840b3780 con 0x7f718415ed20 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f717be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71840d15b0 0x7f71840c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f7182800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f718415ed20 0x7f71840c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f717be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71840d15b0 0x7f71840c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46394/0 (socket says 172.21.15.76:46394) 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.398+0000 7f717be00640 1 -- 172.21.15.76:0/1777087438 learned_addr learned my addr 172.21.15.76:0/1777087438 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f7181e00640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71840b4f20 0x7f718415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f7182800640 1 -- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71840d15b0 msgr2=0x7f71840c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f7182800640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71840d15b0 0x7f71840c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f7182800640 1 -- 172.21.15.76:0/1777087438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71840b4f20 msgr2=0x7f718415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f7182800640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71840b4f20 0x7f718415bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f7182800640 1 -- 172.21.15.76:0/1777087438 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71840c98b0 con 0x7f718415ed20 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f717be00640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71840d15b0 0x7f71840c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f7182800640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f718415ed20 0x7f71840c9140 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f717c00bdf0 tx=0x7f717c00bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f717a000640 1 -- 172.21.15.76:0/1777087438 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c019070 con 0x7f718415ed20 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f717a000640 1 -- 172.21.15.76:0/1777087438 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717c014440 con 0x7f718415ed20 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.399+0000 7f717a000640 1 -- 172.21.15.76:0/1777087438 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c014830 con 0x7f718415ed20 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.400+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71840c9b90 con 0x7f718415ed20 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.400+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71840ca150 con 0x7f718415ed20 2024-04-23T14:31:03.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.400+0000 7f717a000640 1 -- 172.21.15.76:0/1777087438 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f717c01c070 con 0x7f718415ed20 2024-04-23T14:31:03.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.400+0000 7f717a000640 1 -- 172.21.15.76:0/1777087438 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f717c01e440 con 0x7f718415ed20 2024-04-23T14:31:03.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.401+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718404fd90 con 0x7f718415ed20 2024-04-23T14:31:03.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.403+0000 7f717a000640 1 -- 172.21.15.76:0/1777087438 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f717c01ea80 con 0x7f718415ed20 2024-04-23T14:31:03.431 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:03.430+0000 7f998b211040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:31:03.463 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:03.463+0000 7f77d7d41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:31:03.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.482+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5796b331-54b0-4a70-a36b-80cb3c7ad4bb", "id": 4} v 0) -- 0x7f718415d0e0 con 0x7f718415ed20 2024-04-23T14:31:03.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.488+0000 7f717a000640 1 -- 172.21.15.76:0/1777087438 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5796b331-54b0-4a70-a36b-80cb3c7ad4bb", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f717c01ec60 con 0x7f718415ed20 2024-04-23T14:31:03.489 INFO:teuthology.orchestra.run.smithi076.stdout:4 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.489+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f718415ed20 msgr2=0x7f71840c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.489+0000 7f7183200640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f718415ed20 0x7f71840c9140 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f717c00bdf0 tx=0x7f717c00bee0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.490+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 shutdown_connections 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.490+0000 7f7183200640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f718415ed20 0x7f71840c9140 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.490+0000 7f7183200640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71840d15b0 0x7f71840c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.490+0000 7f7183200640 1 --2- 172.21.15.76:0/1777087438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71840b4f20 0x7f718415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.490+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 >> 172.21.15.76:0/1777087438 conn(0x7f718408bf90 msgr2=0x7f71840cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.490+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 shutdown_connections 2024-04-23T14:31:03.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.490+0000 7f7183200640 1 -- 172.21.15.76:0/1777087438 wait complete. 2024-04-23T14:31:03.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 4b57258f-8745-4ffc-9413-86fb13026f2f 5 2024-04-23T14:31:03.524 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:03.523+0000 7f77d7d41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:31:03.568 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:03.567+0000 7f998b211040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:03.590 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:03.589+0000 7f77d7d41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:31:03.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.697+0000 7f4b13e00640 1 Processor -- start 2024-04-23T14:31:03.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.698+0000 7f4b13e00640 1 -- start start 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b13e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4b140d15b0 0x7f4b140cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140b4f20 0x7f4b140cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b13e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b140b5da0 con 0x7f4b1415ed20 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b13e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1415f100 con 0x7f4b140d15b0 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b13e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4b140d0a20 con 0x7f4b140b4f20 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b12a00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4b1415ed20 0x7f4b140d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57388/0 (socket says 172.21.15.76:57388) 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b13400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140b4f20 0x7f4b140cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.699+0000 7f4b12a00640 1 -- 172.21.15.76:0/2759050193 learned_addr learned my addr 172.21.15.76:0/2759050193 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.700+0000 7f4b0a000640 1 --2- 172.21.15.76:0/2759050193 >> v2:172.21.15.59:3300/0 conn(0x7f4b140d15b0 0x7f4b140cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.700+0000 7f4b13400640 1 -- 172.21.15.76:0/2759050193 >> v2:172.21.15.59:3300/0 conn(0x7f4b140d15b0 msgr2=0x7f4b140cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.700+0000 7f4b12000640 1 -- 172.21.15.76:0/2759050193 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944970451 0 0) 0x7f4b140b5da0 con 0x7f4b1415ed20 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.700+0000 7f4b13400640 1 --2- 172.21.15.76:0/2759050193 >> v2:172.21.15.59:3300/0 conn(0x7f4b140d15b0 0x7f4b140cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.700+0000 7f4b13400640 1 -- 172.21.15.76:0/2759050193 >> v1:172.21.15.76:6789/0 conn(0x7f4b1415ed20 legacy=0x7f4b140d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.700+0000 7f4b13400640 1 -- 172.21.15.76:0/2759050193 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b140d2ed0 con 0x7f4b140b4f20 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.700+0000 7f4b0a000640 1 --2- 172.21.15.76:0/2759050193 >> v2:172.21.15.59:3300/0 conn(0x7f4b140d15b0 0x7f4b140cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.701+0000 7f4b13400640 1 --2- 172.21.15.76:0/2759050193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140b4f20 0x7f4b140cf7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097a0 tx=0x7f4b0c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9813fa522f08fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.701+0000 7f4b12000640 1 -- 172.21.15.76:0/2759050193 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c037070 con 0x7f4b140b4f20 2024-04-23T14:31:03.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.701+0000 7f4b12000640 1 -- 172.21.15.76:0/2759050193 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c02adb0 con 0x7f4b140b4f20 2024-04-23T14:31:03.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.701+0000 7f4b12000640 1 -- 172.21.15.76:0/2759050193 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c032660 con 0x7f4b140b4f20 2024-04-23T14:31:03.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 -- 172.21.15.76:0/2759050193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140b4f20 msgr2=0x7f4b140cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 --2- 172.21.15.76:0/2759050193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140b4f20 0x7f4b140cf7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097a0 tx=0x7f4b0c02a060 comp rx=0 tx=0).stop 2024-04-23T14:31:03.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 -- 172.21.15.76:0/2759050193 shutdown_connections 2024-04-23T14:31:03.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 --2- 172.21.15.76:0/2759050193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140b4f20 0x7f4b140cf7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 --2- 172.21.15.76:0/2759050193 >> v2:172.21.15.59:3300/0 conn(0x7f4b140d15b0 0x7f4b140cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 -- 172.21.15.76:0/2759050193 >> 172.21.15.76:0/2759050193 conn(0x7f4b1408bf90 msgr2=0x7f4b140d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:03.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 -- 172.21.15.76:0/2759050193 shutdown_connections 2024-04-23T14:31:03.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.702+0000 7f4b13e00640 1 -- 172.21.15.76:0/2759050193 wait complete. 2024-04-23T14:31:03.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.703+0000 7f4b13e00640 1 Processor -- start 2024-04-23T14:31:03.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.704+0000 7f4b13e00640 1 -- start start 2024-04-23T14:31:03.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.704+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b140b4f20 0x7f4b141520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.704+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140d15b0 0x7f4b1414edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b1415ed20 0x7f4b140cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b140b3600 con 0x7f4b140b4f20 2024-04-23T14:31:03.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4b140d3490 con 0x7f4b1415ed20 2024-04-23T14:31:03.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4b140b3780 con 0x7f4b140d15b0 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b1415ed20 0x7f4b140cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b1415ed20 0x7f4b140cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46400/0 (socket says 172.21.15.76:46400) 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 -- 172.21.15.76:0/3270237418 learned_addr learned my addr 172.21.15.76:0/3270237418 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b0a000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140d15b0 0x7f4b1414edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b12a00640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b140b4f20 0x7f4b141520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 -- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140d15b0 msgr2=0x7f4b1414edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140d15b0 0x7f4b1414edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 -- 172.21.15.76:0/3270237418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b140b4f20 msgr2=0x7f4b141520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b140b4f20 0x7f4b141520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b13400640 1 -- 172.21.15.76:0/3270237418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b141525e0 con 0x7f4b1415ed20 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b0a000640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140d15b0 0x7f4b1414edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.705+0000 7f4b12a00640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b140b4f20 0x7f4b141520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b13400640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b1415ed20 0x7f4b140cb690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c035040 tx=0x7f4b0c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b0b400640 1 -- 172.21.15.76:0/3270237418 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c041070 con 0x7f4b1415ed20 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b0b400640 1 -- 172.21.15.76:0/3270237418 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c032e10 con 0x7f4b1415ed20 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b0b400640 1 -- 172.21.15.76:0/3270237418 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c037040 con 0x7f4b1415ed20 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1414f2f0 con 0x7f4b1415ed20 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b141f82f0 con 0x7f4b1415ed20 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b0b400640 1 -- 172.21.15.76:0/3270237418 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b0c044070 con 0x7f4b1415ed20 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.706+0000 7f4b0b400640 1 -- 172.21.15.76:0/3270237418 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f4b0c046ca0 con 0x7f4b1415ed20 2024-04-23T14:31:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.707+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1404fd90 con 0x7f4b1415ed20 2024-04-23T14:31:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.710+0000 7f4b0b400640 1 -- 172.21.15.76:0/3270237418 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4b141525e0 con 0x7f4b1415ed20 2024-04-23T14:31:03.726 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:03.725+0000 7f77d7d41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:31:03.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.788+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4b57258f-8745-4ffc-9413-86fb13026f2f", "id": 5} v 0) -- 0x7f4b140b6d60 con 0x7f4b1415ed20 2024-04-23T14:31:03.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.795+0000 7f4b0b400640 1 -- 172.21.15.76:0/3270237418 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4b57258f-8745-4ffc-9413-86fb13026f2f", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f4b0c046720 con 0x7f4b1415ed20 2024-04-23T14:31:03.796 INFO:teuthology.orchestra.run.smithi076.stdout:5 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.796+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b1415ed20 msgr2=0x7f4b140cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.796+0000 7f4b13e00640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b1415ed20 0x7f4b140cb690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c035040 tx=0x7f4b0c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.797+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 shutdown_connections 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.797+0000 7f4b13e00640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b140d15b0 0x7f4b1414edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.797+0000 7f4b13e00640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b1415ed20 0x7f4b140cb690 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.797+0000 7f4b13e00640 1 --2- 172.21.15.76:0/3270237418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b140b4f20 0x7f4b141520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.797+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 >> 172.21.15.76:0/3270237418 conn(0x7f4b1408bf90 msgr2=0x7f4b1415df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.797+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 shutdown_connections 2024-04-23T14:31:03.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.797+0000 7f4b13e00640 1 -- 172.21.15.76:0/3270237418 wait complete. 2024-04-23T14:31:03.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 9869c195-a732-4e0e-b45d-9d5784308783 6 2024-04-23T14:31:03.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.991+0000 7f3b54c00640 1 Processor -- start 2024-04-23T14:31:03.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.992+0000 7f3b54c00640 1 -- start start 2024-04-23T14:31:03.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b54c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b5015ed20 0x7f3b500d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3600 0x7f3b500d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:03.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b54c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b500b5da0 con 0x7f3b500b3f10 2024-04-23T14:31:03.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b54c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5015f100 con 0x7f3b5015ed20 2024-04-23T14:31:03.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b54c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3b500d0a20 con 0x7f3b500b3600 2024-04-23T14:31:03.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b47e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3600 0x7f3b500d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b4f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b5015ed20 0x7f3b500d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:03.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b47e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3600 0x7f3b500d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:40880/0 (socket says 172.21.15.76:40880) 2024-04-23T14:31:03.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.993+0000 7f3b47e00640 1 -- 172.21.15.76:0/1853126108 learned_addr learned my addr 172.21.15.76:0/1853126108 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:03.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.994+0000 7f3b47e00640 1 -- 172.21.15.76:0/1853126108 >> v2:172.21.15.76:3300/0 conn(0x7f3b5015ed20 msgr2=0x7f3b500d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:03.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.994+0000 7f3b47e00640 1 --2- 172.21.15.76:0/1853126108 >> v2:172.21.15.76:3300/0 conn(0x7f3b5015ed20 0x7f3b500d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.994+0000 7f3b47e00640 1 -- 172.21.15.76:0/1853126108 >> v1:172.21.15.76:6789/0 conn(0x7f3b500b3f10 legacy=0x7f3b500b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.994+0000 7f3b47e00640 1 -- 172.21.15.76:0/1853126108 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b500d2ed0 con 0x7f3b500b3600 2024-04-23T14:31:03.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.994+0000 7f3b4f400640 1 --2- 172.21.15.76:0/1853126108 >> v2:172.21.15.76:3300/0 conn(0x7f3b5015ed20 0x7f3b500d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:03.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.995+0000 7f3b47e00640 1 --2- 172.21.15.76:0/1853126108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3600 0x7f3b500d13e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009870 tx=0x7f3b3802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ada25fbe4a853d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.995+0000 7f3b4ea00640 1 -- 172.21.15.76:0/1853126108 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38037070 con 0x7f3b500b3600 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.995+0000 7f3b4ea00640 1 -- 172.21.15.76:0/1853126108 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38032440 con 0x7f3b500b3600 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.995+0000 7f3b4ea00640 1 -- 172.21.15.76:0/1853126108 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38032810 con 0x7f3b500b3600 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 -- 172.21.15.76:0/1853126108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3600 msgr2=0x7f3b500d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 --2- 172.21.15.76:0/1853126108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3600 0x7f3b500d13e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009870 tx=0x7f3b3802a130 comp rx=0 tx=0).stop 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 -- 172.21.15.76:0/1853126108 shutdown_connections 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 --2- 172.21.15.76:0/1853126108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3600 0x7f3b500d13e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 --2- 172.21.15.76:0/1853126108 >> v2:172.21.15.76:3300/0 conn(0x7f3b5015ed20 0x7f3b500d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 -- 172.21.15.76:0/1853126108 >> 172.21.15.76:0/1853126108 conn(0x7f3b5008bf90 msgr2=0x7f3b500d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:03.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 -- 172.21.15.76:0/1853126108 shutdown_connections 2024-04-23T14:31:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.996+0000 7f3b54c00640 1 -- 172.21.15.76:0/1853126108 wait complete. 2024-04-23T14:31:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.998+0000 7f3b54c00640 1 Processor -- start 2024-04-23T14:31:03.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.998+0000 7f3b54c00640 1 -- start start 2024-04-23T14:31:04.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b500b3600 0x7f3b50155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3f10 0x7f3b500ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b5015ed20 0x7f3b500b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b54c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b500d0290 con 0x7f3b5015ed20 2024-04-23T14:31:04.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b54c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b500d3490 con 0x7f3b500b3600 2024-04-23T14:31:04.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b54c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3b500d0410 con 0x7f3b500b3f10 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b500b3600 0x7f3b50155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b500b3600 0x7f3b50155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46424/0 (socket says 172.21.15.76:46424) 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b47e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3f10 0x7f3b500ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 -- 172.21.15.76:0/4163802679 learned_addr learned my addr 172.21.15.76:0/4163802679 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4fe00640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b5015ed20 0x7f3b500b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 -- 172.21.15.76:0/4163802679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3f10 msgr2=0x7f3b500ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3f10 0x7f3b500ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 -- 172.21.15.76:0/4163802679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b5015ed20 msgr2=0x7f3b500b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b5015ed20 0x7f3b500b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:03.999+0000 7f3b4f400640 1 -- 172.21.15.76:0/4163802679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b500ce5c0 con 0x7f3b500b3600 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b4f400640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b500b3600 0x7f3b50155be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e960 tx=0x7f3b4000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b47400640 1 -- 172.21.15.76:0/4163802679 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40017070 con 0x7f3b500b3600 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b47400640 1 -- 172.21.15.76:0/4163802679 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4000ca00 con 0x7f3b500b3600 2024-04-23T14:31:04.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b501522c0 con 0x7f3b500b3600 2024-04-23T14:31:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b47400640 1 -- 172.21.15.76:0/4163802679 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4000cdd0 con 0x7f3b500b3600 2024-04-23T14:31:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b50152880 con 0x7f3b500b3600 2024-04-23T14:31:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b47400640 1 -- 172.21.15.76:0/4163802679 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b4001a070 con 0x7f3b500b3600 2024-04-23T14:31:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.000+0000 7f3b47400640 1 -- 172.21.15.76:0/4163802679 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f3b40009400 con 0x7f3b500b3600 2024-04-23T14:31:04.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.001+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b5004fd90 con 0x7f3b500b3600 2024-04-23T14:31:04.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.003+0000 7f3b47400640 1 -- 172.21.15.76:0/4163802679 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3b400064d0 con 0x7f3b500b3600 2024-04-23T14:31:04.045 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:04.044+0000 7f998b211040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:31:04.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.082+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9869c195-a732-4e0e-b45d-9d5784308783", "id": 6} v 0) -- 0x7f3b500d19d0 con 0x7f3b500b3600 2024-04-23T14:31:04.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.089+0000 7f3b47400640 1 -- 172.21.15.76:0/4163802679 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9869c195-a732-4e0e-b45d-9d5784308783", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3b40006780 con 0x7f3b500b3600 2024-04-23T14:31:04.090 INFO:teuthology.orchestra.run.smithi076.stdout:6 2024-04-23T14:31:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b500b3600 msgr2=0x7f3b50155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b500b3600 0x7f3b50155be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e960 tx=0x7f3b4000ee20 comp rx=0 tx=0).stop 2024-04-23T14:31:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 shutdown_connections 2024-04-23T14:31:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b500b3f10 0x7f3b500ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b5015ed20 0x7f3b500b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 --2- 172.21.15.76:0/4163802679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b500b3600 0x7f3b50155be0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 >> 172.21.15.76:0/4163802679 conn(0x7f3b5008bf90 msgr2=0x7f3b500b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 shutdown_connections 2024-04-23T14:31:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.090+0000 7f3b54c00640 1 -- 172.21.15.76:0/4163802679 wait complete. 2024-04-23T14:31:04.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 82e9249f-85be-4a8a-890e-4f6c83e70085 7 2024-04-23T14:31:04.209 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:04.208+0000 7f77d7d41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:31:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.278+0000 7f10fcc00640 1 Processor -- start 2024-04-23T14:31:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.278+0000 7f10fcc00640 1 -- start start 2024-04-23T14:31:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10fcc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f10f8161c50 0x7f10f80d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10fcc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10fcc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10f80d22e0 con 0x7f10f80cfc50 2024-04-23T14:31:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10fcc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f10f80d2ba0 con 0x7f10f8161c50 2024-04-23T14:31:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10fcc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10f80d2550 con 0x7f10f80d0de0 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10f7400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f10f8161c50 0x7f10f80d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10f7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10f7400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f10f8161c50 0x7f10f80d5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46438/0 (socket says 172.21.15.76:46438) 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.280+0000 7f10f7400640 1 -- 172.21.15.76:0/2197783064 learned_addr learned my addr 172.21.15.76:0/2197783064 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.281+0000 7f10f7400640 1 -- 172.21.15.76:0/2197783064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 msgr2=0x7f10f80d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.281+0000 7f10f7400640 1 --2- 172.21.15.76:0/2197783064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.281+0000 7f10f7400640 1 -- 172.21.15.76:0/2197783064 >> v1:172.21.15.59:6789/0 conn(0x7f10f80cfc50 legacy=0x7f10f80d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.281+0000 7f10f7400640 1 -- 172.21.15.76:0/2197783064 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f80d2830 con 0x7f10f8161c50 2024-04-23T14:31:04.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.282+0000 7f10f7400640 1 --2- 172.21.15.76:0/2197783064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80d5240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009870 tx=0x7f10e802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89f4cba7a4fbedf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:04.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.282+0000 7f10f6a00640 1 -- 172.21.15.76:0/2197783064 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8037070 con 0x7f10f8161c50 2024-04-23T14:31:04.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.282+0000 7f10f6a00640 1 -- 172.21.15.76:0/2197783064 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e8032440 con 0x7f10f8161c50 2024-04-23T14:31:04.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.282+0000 7f10f6a00640 1 -- 172.21.15.76:0/2197783064 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8032810 con 0x7f10f8161c50 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 -- 172.21.15.76:0/2197783064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 msgr2=0x7f10f80d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 --2- 172.21.15.76:0/2197783064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80d5240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009870 tx=0x7f10e802a210 comp rx=0 tx=0).stop 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 -- 172.21.15.76:0/2197783064 shutdown_connections 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 --2- 172.21.15.76:0/2197783064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 --2- 172.21.15.76:0/2197783064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80d5240 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 -- 172.21.15.76:0/2197783064 >> 172.21.15.76:0/2197783064 conn(0x7f10f808bf90 msgr2=0x7f10f80d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 -- 172.21.15.76:0/2197783064 shutdown_connections 2024-04-23T14:31:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.283+0000 7f10fcc00640 1 -- 172.21.15.76:0/2197783064 wait complete. 2024-04-23T14:31:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.285+0000 7f10fcc00640 1 Processor -- start 2024-04-23T14:31:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.285+0000 7f10fcc00640 1 -- start start 2024-04-23T14:31:04.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10fcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f80cfc50 0x7f10f814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10fcc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10fcc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10fcc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10f80d0090 con 0x7f10f80cfc50 2024-04-23T14:31:04.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10fcc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10f80d0460 con 0x7f10f8161c50 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10fcc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10f80d0210 con 0x7f10f80d0de0 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10efe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10f7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10f7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46442/0 (socket says 172.21.15.76:46442) 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10f7e00640 1 -- 172.21.15.76:0/4030502090 learned_addr learned my addr 172.21.15.76:0/4030502090 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10f7400640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f80cfc50 0x7f10f814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.286+0000 7f10f7e00640 1 -- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 msgr2=0x7f10f80ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10f7e00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10f7e00640 1 -- 172.21.15.76:0/4030502090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f80cfc50 msgr2=0x7f10f814b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10f7e00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f80cfc50 0x7f10f814b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10f7e00640 1 -- 172.21.15.76:0/4030502090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f80cd780 con 0x7f10f8161c50 2024-04-23T14:31:04.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10efe00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:04.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10f7e00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80b7220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009dd0 tx=0x7f10f000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:04.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10ef400640 1 -- 172.21.15.76:0/4030502090 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0018020 con 0x7f10f8161c50 2024-04-23T14:31:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10ef400640 1 -- 172.21.15.76:0/4030502090 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f0014480 con 0x7f10f8161c50 2024-04-23T14:31:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f80cda60 con 0x7f10f8161c50 2024-04-23T14:31:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10ef400640 1 -- 172.21.15.76:0/4030502090 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0014870 con 0x7f10f8161c50 2024-04-23T14:31:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.287+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f80ce020 con 0x7f10f8161c50 2024-04-23T14:31:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.288+0000 7f10ef400640 1 -- 172.21.15.76:0/4030502090 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f10f001b020 con 0x7f10f8161c50 2024-04-23T14:31:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.288+0000 7f10ef400640 1 -- 172.21.15.76:0/4030502090 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f10f0014da0 con 0x7f10f8161c50 2024-04-23T14:31:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.288+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f804fd90 con 0x7f10f8161c50 2024-04-23T14:31:04.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.291+0000 7f10ef400640 1 -- 172.21.15.76:0/4030502090 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f10f0022020 con 0x7f10f8161c50 2024-04-23T14:31:04.354 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:04.352+0000 7f998b211040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:31:04.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.370+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "82e9249f-85be-4a8a-890e-4f6c83e70085", "id": 7} v 0) -- 0x7f10f80b5e80 con 0x7f10f8161c50 2024-04-23T14:31:04.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.377+0000 7f10ef400640 1 -- 172.21.15.76:0/4030502090 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "82e9249f-85be-4a8a-890e-4f6c83e70085", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f10f0010040 con 0x7f10f8161c50 2024-04-23T14:31:04.378 INFO:teuthology.orchestra.run.smithi076.stdout:7 2024-04-23T14:31:04.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 msgr2=0x7f10f80b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80b7220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009dd0 tx=0x7f10f000c490 comp rx=0 tx=0).stop 2024-04-23T14:31:04.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 shutdown_connections 2024-04-23T14:31:04.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10f80d0de0 0x7f10f80ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10f8161c50 0x7f10f80b7220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 --2- 172.21.15.76:0/4030502090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10f80cfc50 0x7f10f814b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 >> 172.21.15.76:0/4030502090 conn(0x7f10f808bf90 msgr2=0x7f10f80b5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:04.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 shutdown_connections 2024-04-23T14:31:04.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:04.379+0000 7f10fcc00640 1 -- 172.21.15.76:0/4030502090 wait complete. 2024-04-23T14:31:04.397 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-23T14:31:04.397 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-23T14:31:04.404 INFO:tasks.ceph.osd.0:Started 2024-04-23T14:31:04.405 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-23T14:31:04.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-23T14:31:04.414 INFO:tasks.ceph.osd.1:Started 2024-04-23T14:31:04.414 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-23T14:31:04.415 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-23T14:31:04.420 INFO:tasks.ceph.osd.2:Started 2024-04-23T14:31:04.420 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-23T14:31:04.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-23T14:31:04.421 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:04.416+0000 7f998b211040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:31:04.425 INFO:tasks.ceph.osd.3:Started 2024-04-23T14:31:04.425 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-23T14:31:04.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-23T14:31:04.432 INFO:tasks.ceph.osd.4:Started 2024-04-23T14:31:04.432 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-23T14:31:04.432 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-23T14:31:04.439 INFO:tasks.ceph.osd.5:Started 2024-04-23T14:31:04.439 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-23T14:31:04.439 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-23T14:31:04.444 INFO:tasks.ceph.osd.6:Started 2024-04-23T14:31:04.445 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-23T14:31:04.445 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-23T14:31:04.451 INFO:tasks.ceph.osd.7:Started 2024-04-23T14:31:04.451 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:04.529 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:04.528+0000 7f77d7d41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:31:04.600 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:04.600+0000 7f77d7d41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:31:04.616 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:04.616+0000 7f998b211040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:31:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.661+0000 7f638f400640 1 Processor -- start 2024-04-23T14:31:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.661+0000 7f638f400640 1 -- start start 2024-04-23T14:31:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.662+0000 7f638f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f638815c5e0 0x7f63880af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.662+0000 7f638f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63880d08b0 0x7f63881624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.662+0000 7f638f400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f638815f0f0 con 0x7f63880af920 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.662+0000 7f638f400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f63880afd80 con 0x7f638815c5e0 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.662+0000 7f638f400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f638815fa20 con 0x7f63880d08b0 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f638815c5e0 0x7f63880af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f638815c5e0 0x7f63880af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34358/0 (socket says 172.21.15.59:34358) 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638ca00640 1 -- 172.21.15.59:0/3081178706 learned_addr learned my addr 172.21.15.59:0/3081178706 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638d400640 1 --2- 172.21.15.59:0/3081178706 >> v2:172.21.15.76:3300/0 conn(0x7f63880d08b0 0x7f63881624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638ca00640 1 -- 172.21.15.59:0/3081178706 >> v2:172.21.15.76:3300/0 conn(0x7f63880d08b0 msgr2=0x7f63881624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638ca00640 1 --2- 172.21.15.59:0/3081178706 >> v2:172.21.15.76:3300/0 conn(0x7f63880d08b0 0x7f63881624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638ca00640 1 -- 172.21.15.59:0/3081178706 >> v1:172.21.15.59:6789/0 conn(0x7f63880af920 legacy=0x7f63880d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.663+0000 7f638ca00640 1 -- 172.21.15.59:0/3081178706 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638815f3d0 con 0x7f638815c5e0 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.664+0000 7f638ca00640 1 --2- 172.21.15.59:0/3081178706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880af3e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f638806c860 tx=0x7f637c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8baf35f7097f8f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.664+0000 7f6387400640 1 -- 172.21.15.59:0/3081178706 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c037070 con 0x7f638815c5e0 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.664+0000 7f6387400640 1 -- 172.21.15.59:0/3081178706 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c032440 con 0x7f638815c5e0 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.664+0000 7f6387400640 1 -- 172.21.15.59:0/3081178706 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c032810 con 0x7f638815c5e0 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 -- 172.21.15.59:0/3081178706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 msgr2=0x7f63880af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 --2- 172.21.15.59:0/3081178706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880af3e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f638806c860 tx=0x7f637c02a230 comp rx=0 tx=0).stop 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 -- 172.21.15.59:0/3081178706 shutdown_connections 2024-04-23T14:31:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 --2- 172.21.15.59:0/3081178706 >> v2:172.21.15.76:3300/0 conn(0x7f63880d08b0 0x7f63881624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 --2- 172.21.15.59:0/3081178706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880af3e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 -- 172.21.15.59:0/3081178706 >> 172.21.15.59:0/3081178706 conn(0x7f6388067ea0 msgr2=0x7f638806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 -- 172.21.15.59:0/3081178706 shutdown_connections 2024-04-23T14:31:04.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.665+0000 7f638f400640 1 -- 172.21.15.59:0/3081178706 wait complete. 2024-04-23T14:31:04.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.667+0000 7f638f400640 1 Processor -- start 2024-04-23T14:31:04.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.667+0000 7f638f400640 1 -- start start 2024-04-23T14:31:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63880af920 0x7f638815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638f400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63880d08b0 0x7f63880b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638f400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f638806e600 con 0x7f638815c5e0 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638f400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f638806e480 con 0x7f63880af920 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638f400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6388162a10 con 0x7f63880d08b0 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63880af920 0x7f638815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63880af920 0x7f638815e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:52116/0 (socket says 172.21.15.59:52116) 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f638ca00640 1 -- 172.21.15.59:0/1188687455 learned_addr learned my addr 172.21.15.59:0/1188687455 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.668+0000 7f6387e00640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63880d08b0 0x7f63880b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638ca00640 1 -- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63880d08b0 msgr2=0x7f63880b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638ca00640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63880d08b0 0x7f63880b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638ca00640 1 -- 172.21.15.59:0/1188687455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 msgr2=0x7f63880b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638ca00640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638ca00640 1 -- 172.21.15.59:0/1188687455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638815d5f0 con 0x7f63880af920 2024-04-23T14:31:04.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638d400640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638ca00640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63880af920 0x7f638815e6f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f637c02a270 tx=0x7f637c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f6384c00640 1 -- 172.21.15.59:0/1188687455 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c041070 con 0x7f63880af920 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f6384c00640 1 -- 172.21.15.59:0/1188687455 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c03c870 con 0x7f63880af920 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f6384c00640 1 -- 172.21.15.59:0/1188687455 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c037040 con 0x7f63880af920 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.669+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638815d8d0 con 0x7f63880af920 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.670+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6388151f10 con 0x7f63880af920 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.670+0000 7f6384c00640 1 -- 172.21.15.59:0/1188687455 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f637c044070 con 0x7f63880af920 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.670+0000 7f6384c00640 1 -- 172.21.15.59:0/1188687455 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f637c046450 con 0x7f63880af920 2024-04-23T14:31:04.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.671+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6388068a80 con 0x7f63880af920 2024-04-23T14:31:04.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.673+0000 7f6384c00640 1 -- 172.21.15.59:0/1188687455 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f637c03cc60 con 0x7f63880af920 2024-04-23T14:31:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.738+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f638806e260 con 0x7f63880af920 2024-04-23T14:31:04.739 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:04.738+0000 7f77d7d41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:31:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.739+0000 7f6384c00640 1 -- 172.21.15.59:0/1188687455 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f637c050430 con 0x7f63880af920 2024-04-23T14:31:04.740 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:04.740 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":10,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:04.373113+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:04.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.740+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63880af920 msgr2=0x7f638815e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.740+0000 7f638f400640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63880af920 0x7f638815e6f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f637c02a270 tx=0x7f637c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.741+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 shutdown_connections 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.741+0000 7f638f400640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63880d08b0 0x7f63880b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.741+0000 7f638f400640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f638815c5e0 0x7f63880b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.741+0000 7f638f400640 1 --2- 172.21.15.59:0/1188687455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63880af920 0x7f638815e6f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.741+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 >> 172.21.15.59:0/1188687455 conn(0x7f6388067ea0 msgr2=0x7f63880b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.741+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 shutdown_connections 2024-04-23T14:31:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:04.741+0000 7f638f400640 1 -- 172.21.15.59:0/1188687455 wait complete. 2024-04-23T14:31:04.757 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-23T14:31:04.757 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-23T14:31:04.981 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:04.980+0000 7f998b211040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:31:05.039 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:05.038+0000 7f998b211040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:31:05.093 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:05.092+0000 7f77d7d41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:31:05.113 INFO:tasks.ceph.osd.6.smithi076.stderr:2024-04-23T14:31:05.112+0000 7ff885dcb640 -1 Falling back to public interface 2024-04-23T14:31:05.122 INFO:tasks.ceph.osd.5.smithi076.stderr:2024-04-23T14:31:05.121+0000 7fc1a362d640 -1 Falling back to public interface 2024-04-23T14:31:05.122 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-23T14:31:05.121+0000 7f4c5edcb640 -1 Falling back to public interface 2024-04-23T14:31:05.135 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-23T14:31:05.134+0000 7fa3bbedd640 -1 Falling back to public interface 2024-04-23T14:31:05.142 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-23T14:31:05.141+0000 7f494fe19640 -1 Falling back to public interface 2024-04-23T14:31:05.142 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-23T14:31:05.142+0000 7f7cdadd6640 -1 Falling back to public interface 2024-04-23T14:31:05.149 INFO:tasks.ceph.osd.4.smithi076.stderr:2024-04-23T14:31:05.149+0000 7f601fe0f640 -1 Falling back to public interface 2024-04-23T14:31:05.154 INFO:tasks.ceph.osd.7.smithi076.stderr:2024-04-23T14:31:05.154+0000 7f8396357640 -1 Falling back to public interface 2024-04-23T14:31:05.156 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:05.155+0000 7f77d7d41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:31:05.232 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:05.231+0000 7f998b211040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:31:05.340 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:05.339+0000 7f77d7d41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:31:05.406 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:05.406+0000 7f998b211040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:31:05.468 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:05.467+0000 7f998b211040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:31:05.509 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:05.509+0000 7f77d7d41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:31:05.552 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:05.551+0000 7f998b211040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:05.565 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:05.568 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:05.567+0000 7f77d7d41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:31:05.608 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:05.607+0000 7f998b211040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:31:05.649 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:05.648+0000 7f77d7d41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:31:05.703 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:05.702+0000 7f77d7d41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:31:05.736 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.735+0000 7fea74600640 1 Processor -- start 2024-04-23T14:31:05.736 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.735+0000 7fea74600640 1 -- start start 2024-04-23T14:31:05.738 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.736+0000 7fea74600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea6c15e980 0x7fea6c0d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:05.738 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.736+0000 7fea74600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea6c0ba120 0x7fea6c0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:05.738 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.736+0000 7fea74600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c1624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:05.738 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.736+0000 7fea74600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fea6c0d20c0 con 0x7fea6c15e980 2024-04-23T14:31:05.738 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.736+0000 7fea74600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fea6c0d1580 con 0x7fea6c0ba120 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.736+0000 7fea74600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c0d1820 con 0x7fea6c0cf9a0 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea6c15e980 0x7fea6c0d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea6c15e980 0x7fea6c0d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34370/0 (socket says 172.21.15.59:34370) 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 -- 172.21.15.59:0/2513708375 learned_addr learned my addr 172.21.15.59:0/2513708375 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71200640 1 --2- 172.21.15.59:0/2513708375 >> v2:172.21.15.76:3300/0 conn(0x7fea6c0ba120 0x7fea6c0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea72600640 1 --2- 172.21.15.59:0/2513708375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c1624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 -- 172.21.15.59:0/2513708375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 msgr2=0x7fea6c1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 --2- 172.21.15.59:0/2513708375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c1624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 -- 172.21.15.59:0/2513708375 >> v2:172.21.15.76:3300/0 conn(0x7fea6c0ba120 msgr2=0x7fea6c0cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 --2- 172.21.15.59:0/2513708375 >> v2:172.21.15.76:3300/0 conn(0x7fea6c0ba120 0x7fea6c0cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71c00640 1 -- 172.21.15.59:0/2513708375 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c0d1b00 con 0x7fea6c15e980 2024-04-23T14:31:05.739 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.737+0000 7fea71200640 1 --2- 172.21.15.59:0/2513708375 >> v2:172.21.15.76:3300/0 conn(0x7fea6c0ba120 0x7fea6c0cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:05.740 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.738+0000 7fea71c00640 1 --2- 172.21.15.59:0/2513708375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c15e980 0x7fea6c0d1040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea6c08d400 tx=0x7fea6002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1b046f9f165ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea6be00640 1 -- 172.21.15.59:0/2513708375 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60037070 con 0x7fea6c15e980 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea6be00640 1 -- 172.21.15.59:0/2513708375 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea60032440 con 0x7fea6c15e980 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea6be00640 1 -- 172.21.15.59:0/2513708375 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60032810 con 0x7fea6c15e980 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea74600640 1 -- 172.21.15.59:0/2513708375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c15e980 msgr2=0x7fea6c0d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea74600640 1 --2- 172.21.15.59:0/2513708375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c15e980 0x7fea6c0d1040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea6c08d400 tx=0x7fea6002a140 comp rx=0 tx=0).stop 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea74600640 1 -- 172.21.15.59:0/2513708375 shutdown_connections 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea74600640 1 --2- 172.21.15.59:0/2513708375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c1624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea74600640 1 --2- 172.21.15.59:0/2513708375 >> v2:172.21.15.76:3300/0 conn(0x7fea6c0ba120 0x7fea6c0cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea74600640 1 --2- 172.21.15.59:0/2513708375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c15e980 0x7fea6c0d1040 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.739+0000 7fea74600640 1 -- 172.21.15.59:0/2513708375 >> 172.21.15.59:0/2513708375 conn(0x7fea6c06ccd0 msgr2=0x7fea6c06d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.740+0000 7fea74600640 1 -- 172.21.15.59:0/2513708375 shutdown_connections 2024-04-23T14:31:05.741 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.740+0000 7fea74600640 1 -- 172.21.15.59:0/2513708375 wait complete. 2024-04-23T14:31:05.742 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.741+0000 7fea74600640 1 Processor -- start 2024-04-23T14:31:05.742 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.741+0000 7fea74600640 1 -- start start 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.742+0000 7fea74600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c0ba120 0x7fea6c1617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.742+0000 7fea74600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.742+0000 7fea74600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea6c15e980 0x7fea6c15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.742+0000 7fea74600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c15ed60 con 0x7fea6c0ba120 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.742+0000 7fea74600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c0d2c90 con 0x7fea6c15e980 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.742+0000 7fea74600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c162a10 con 0x7fea6c0cf9a0 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.742+0000 7fea71c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c0ba120 0x7fea6c1617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c0ba120 0x7fea6c1617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34374/0 (socket says 172.21.15.59:34374) 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 -- 172.21.15.59:0/86275566 learned_addr learned my addr 172.21.15.59:0/86275566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea72600640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea6c15e980 0x7fea6c15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:05.744 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71200640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:05.745 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 -- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 msgr2=0x7fea6c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:05.745 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.745 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 -- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea6c15e980 msgr2=0x7fea6c15cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:05.745 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea6c15e980 0x7fea6c15cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.745 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 -- 172.21.15.59:0/86275566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c0ce660 con 0x7fea6c0ba120 2024-04-23T14:31:05.746 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.743+0000 7fea71c00640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c0ba120 0x7fea6c1617a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c00 tx=0x7fea600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:05.746 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.744+0000 7fea69600640 1 -- 172.21.15.59:0/86275566 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60041020 con 0x7fea6c0ba120 2024-04-23T14:31:05.746 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.744+0000 7fea69600640 1 -- 172.21.15.59:0/86275566 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6003d620 con 0x7fea6c0ba120 2024-04-23T14:31:05.746 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.744+0000 7fea69600640 1 -- 172.21.15.59:0/86275566 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60037040 con 0x7fea6c0ba120 2024-04-23T14:31:05.747 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.744+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c0cddc0 con 0x7fea6c0ba120 2024-04-23T14:31:05.747 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.744+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c0ce380 con 0x7fea6c0ba120 2024-04-23T14:31:05.747 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.744+0000 7fea69600640 1 -- 172.21.15.59:0/86275566 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea60044020 con 0x7fea6c0ba120 2024-04-23T14:31:05.747 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.744+0000 7fea69600640 1 -- 172.21.15.59:0/86275566 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fea60046bd0 con 0x7fea6c0ba120 2024-04-23T14:31:05.747 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.745+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6c1e8260 con 0x7fea6c0ba120 2024-04-23T14:31:05.748 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.748+0000 7fea69600640 1 -- 172.21.15.59:0/86275566 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fea6c1e8260 con 0x7fea6c0ba120 2024-04-23T14:31:05.816 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.815+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea6c04fd90 con 0x7fea6c0ba120 2024-04-23T14:31:05.817 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.816+0000 7fea69600640 1 -- 172.21.15.59:0/86275566 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fea60046e20 con 0x7fea6c0ba120 2024-04-23T14:31:05.817 INFO:teuthology.misc.health.smithi059.stdout: 2024-04-23T14:31:05.817 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":10,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:04.373113+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:05.818 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c0ba120 msgr2=0x7fea6c1617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:05.818 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c0ba120 0x7fea6c1617a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c00 tx=0x7fea600046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.818 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 shutdown_connections 2024-04-23T14:31:05.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea6c0cf9a0 0x7fea6c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea6c15e980 0x7fea6c15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 --2- 172.21.15.59:0/86275566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea6c0ba120 0x7fea6c1617a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:05.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 >> 172.21.15.59:0/86275566 conn(0x7fea6c06ccd0 msgr2=0x7fea6c1e7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:05.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 shutdown_connections 2024-04-23T14:31:05.819 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:05.817+0000 7fea74600640 1 -- 172.21.15.59:0/86275566 wait complete. 2024-04-23T14:31:05.831 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-23T14:31:06.179 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:06.178+0000 7f998b211040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:31:06.249 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:06.248+0000 7f77d7d41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:31:06.316 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:06.315+0000 7f998b211040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:31:06.392 INFO:tasks.ceph.mgr.x.smithi076.stderr:2024-04-23T14:31:06.391+0000 7f77d7d41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:31:07.366 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:07.365+0000 7f992b400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-23T14:31:07.368 INFO:tasks.ceph.mgr.y.smithi059.stderr:2024-04-23T14:31:07.368+0000 7f992b400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-23T14:31:08.575 INFO:tasks.ceph.osd.7.smithi076.stderr:2024-04-23T14:31:08.574+0000 7f8396357640 -1 osd.7 0 log_to_monitors true 2024-04-23T14:31:08.580 INFO:tasks.ceph.osd.6.smithi076.stderr:2024-04-23T14:31:08.580+0000 7ff885dcb640 -1 osd.6 0 log_to_monitors true 2024-04-23T14:31:08.714 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-23T14:31:08.714+0000 7fa3bbedd640 -1 osd.1 0 log_to_monitors true 2024-04-23T14:31:08.742 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-23T14:31:08.742+0000 7f4c5edcb640 -1 osd.3 0 log_to_monitors true 2024-04-23T14:31:08.755 INFO:tasks.ceph.osd.5.smithi076.stderr:2024-04-23T14:31:08.754+0000 7fc1a362d640 -1 osd.5 0 log_to_monitors true 2024-04-23T14:31:08.810 INFO:tasks.ceph.osd.4.smithi076.stderr:2024-04-23T14:31:08.809+0000 7f601fe0f640 -1 osd.4 0 log_to_monitors true 2024-04-23T14:31:09.004 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-23T14:31:09.003+0000 7f494fe19640 -1 osd.2 0 log_to_monitors true 2024-04-23T14:31:09.006 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-23T14:31:09.005+0000 7f7cdadd6640 -1 osd.0 0 log_to_monitors true 2024-04-23T14:31:12.637 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-23T14:31:12.636+0000 7f494a600640 -1 osd.2 0 waiting for initial osdmap 2024-04-23T14:31:12.638 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:12.645 INFO:tasks.ceph.osd.2.smithi059.stderr:2024-04-23T14:31:12.644+0000 7f4940600640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.653 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-23T14:31:12.652+0000 7f4c59c00640 -1 osd.3 0 waiting for initial osdmap 2024-04-23T14:31:12.672 INFO:tasks.ceph.osd.3.smithi059.stderr:2024-04-23T14:31:12.672+0000 7f4c4fc00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.688 INFO:tasks.ceph.osd.7.smithi076.stderr:2024-04-23T14:31:12.687+0000 7f8391c00640 -1 osd.7 0 waiting for initial osdmap 2024-04-23T14:31:12.697 INFO:tasks.ceph.osd.7.smithi076.stderr:2024-04-23T14:31:12.695+0000 7f8387200640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.698 INFO:tasks.ceph.osd.6.smithi076.stderr:2024-04-23T14:31:12.698+0000 7ff880c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-23T14:31:12.713 INFO:tasks.ceph.osd.6.smithi076.stderr:2024-04-23T14:31:12.712+0000 7ff876c00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.738 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-23T14:31:12.737+0000 7f7cd6600640 -1 osd.0 0 waiting for initial osdmap 2024-04-23T14:31:12.741 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-23T14:31:12.741+0000 7fa3b7800640 -1 osd.1 0 waiting for initial osdmap 2024-04-23T14:31:12.746 INFO:tasks.ceph.osd.0.smithi059.stderr:2024-04-23T14:31:12.745+0000 7f7ccbc00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.747 INFO:tasks.ceph.osd.1.smithi059.stderr:2024-04-23T14:31:12.746+0000 7fa3ace00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.753 INFO:tasks.ceph.osd.4.smithi076.stderr:2024-04-23T14:31:12.753+0000 7f601a800640 -1 osd.4 0 waiting for initial osdmap 2024-04-23T14:31:12.760 INFO:tasks.ceph.osd.4.smithi076.stderr:2024-04-23T14:31:12.760+0000 7f6010800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.794 INFO:tasks.ceph.osd.5.smithi076.stderr:2024-04-23T14:31:12.793+0000 7fc19da00640 -1 osd.5 0 waiting for initial osdmap 2024-04-23T14:31:12.799 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.798+0000 7f0526400640 1 Processor -- start 2024-04-23T14:31:12.799 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.798+0000 7f0526400640 1 -- start start 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.799+0000 7f0526400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f05200b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.799+0000 7f0526400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05200b4980 0x7f0520163380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.799+0000 7f0526400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05201678b0 con 0x7f05200b3b10 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.799+0000 7f0526400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f05200b3f50 con 0x7f05200b4980 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.799+0000 7f0526400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0520166af0 con 0x7f0520149000 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05200b4980 0x7f0520163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05200b4980 0x7f0520163380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51894/0 (socket says 172.21.15.59:51894) 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051fe00640 1 -- 172.21.15.59:0/3731806658 learned_addr learned my addr 172.21.15.59:0/3731806658 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:12.801 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051f400640 1 --2- 172.21.15.59:0/3731806658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f05200b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:12.802 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051fe00640 1 -- 172.21.15.59:0/3731806658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 msgr2=0x7f05200b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:12.802 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051fe00640 1 --2- 172.21.15.59:0/3731806658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f05200b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:12.802 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051fe00640 1 -- 172.21.15.59:0/3731806658 >> v1:172.21.15.59:6789/0 conn(0x7f05200b3b10 legacy=0x7f05200b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:12.802 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.800+0000 7f051fe00640 1 -- 172.21.15.59:0/3731806658 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0520166dd0 con 0x7f05200b4980 2024-04-23T14:31:12.802 INFO:tasks.ceph.osd.5.smithi076.stderr:2024-04-23T14:31:12.800+0000 7fc193a00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:31:12.803 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.801+0000 7f051fe00640 1 --2- 172.21.15.59:0/3731806658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b4980 0x7f0520163380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0518009e50 tx=0x7f051802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6eb5e03f1594ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.801+0000 7f051e000640 1 -- 172.21.15.59:0/3731806658 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518036070 con 0x7f05200b4980 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.801+0000 7f051e000640 1 -- 172.21.15.59:0/3731806658 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0518031600 con 0x7f05200b4980 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.801+0000 7f051e000640 1 -- 172.21.15.59:0/3731806658 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05180319d0 con 0x7f05200b4980 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 -- 172.21.15.59:0/3731806658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b4980 msgr2=0x7f0520163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 --2- 172.21.15.59:0/3731806658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b4980 0x7f0520163380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0518009e50 tx=0x7f051802a6c0 comp rx=0 tx=0).stop 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 -- 172.21.15.59:0/3731806658 shutdown_connections 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 --2- 172.21.15.59:0/3731806658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b4980 0x7f0520163380 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 --2- 172.21.15.59:0/3731806658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f05200b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 -- 172.21.15.59:0/3731806658 >> 172.21.15.59:0/3731806658 conn(0x7f052006b9e0 msgr2=0x7f052006bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 -- 172.21.15.59:0/3731806658 shutdown_connections 2024-04-23T14:31:12.804 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.802+0000 7f0526400640 1 -- 172.21.15.59:0/3731806658 wait complete. 2024-04-23T14:31:12.806 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.803+0000 7f0526400640 1 Processor -- start 2024-04-23T14:31:12.806 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.804+0000 7f0526400640 1 -- start start 2024-04-23T14:31:12.806 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.804+0000 7f0526400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b3b10 0x7f052013c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:12.806 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.804+0000 7f0526400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05200b4980 0x7f052015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:12.806 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.804+0000 7f0526400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f052015da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.804+0000 7f0526400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f05201493e0 con 0x7f05200b3b10 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.804+0000 7f0526400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0520167390 con 0x7f05200b4980 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.804+0000 7f0526400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f05201638c0 con 0x7f0520149000 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b3b10 0x7f052013c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b3b10 0x7f052013c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51898/0 (socket says 172.21.15.59:51898) 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 -- 172.21.15.59:0/210399760 learned_addr learned my addr 172.21.15.59:0/210399760 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051fe00640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f052015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051ea00640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05200b4980 0x7f052015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 -- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 msgr2=0x7f052015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f052015da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 -- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05200b4980 msgr2=0x7f052015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05200b4980 0x7f052015c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:12.807 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 -- 172.21.15.59:0/210399760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052015e4e0 con 0x7f05200b3b10 2024-04-23T14:31:12.808 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051ea00640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05200b4980 0x7f052015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:12.808 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.805+0000 7f051f400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b3b10 0x7f052013c570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f051000d5f0 tx=0x7f051000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:12.809 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.806+0000 7f0517400640 1 -- 172.21.15.59:0/210399760 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100125f0 con 0x7f05200b3b10 2024-04-23T14:31:12.809 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.806+0000 7f0517400640 1 -- 172.21.15.59:0/210399760 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0510019070 con 0x7f05200b3b10 2024-04-23T14:31:12.809 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.806+0000 7f0517400640 1 -- 172.21.15.59:0/210399760 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05100129d0 con 0x7f05200b3b10 2024-04-23T14:31:12.809 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.806+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052015e7c0 con 0x7f05200b3b10 2024-04-23T14:31:12.809 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.806+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052015ce40 con 0x7f05200b3b10 2024-04-23T14:31:12.809 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.807+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052006c9f0 con 0x7f05200b3b10 2024-04-23T14:31:12.809 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.808+0000 7f0517400640 1 -- 172.21.15.59:0/210399760 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f0510012b70 con 0x7f05200b3b10 2024-04-23T14:31:12.810 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.808+0000 7f0517400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f050006c890 0x7f050006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:12.810 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.809+0000 7f0517400640 1 -- 172.21.15.59:0/210399760 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f051008b0d0 con 0x7f05200b3b10 2024-04-23T14:31:12.810 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.809+0000 7f051ea00640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f050006c890 0x7f050006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:12.810 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.809+0000 7f051ea00640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f050006c890 0x7f050006ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05201fc130 tx=0x7f050c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:12.815 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:12.814+0000 7f0517400640 1 -- 172.21.15.59:0/210399760 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f051005a2d0 con 0x7f05200b3b10 2024-04-23T14:31:13.016 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.015+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f052004fd90 con 0x7f05200b3b10 2024-04-23T14:31:13.017 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.017+0000 7f0517400640 1 -- 172.21.15.59:0/210399760 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f0510059bd0 con 0x7f05200b3b10 2024-04-23T14:31:13.018 INFO:teuthology.misc.health.smithi059.stdout: 2024-04-23T14:31:13.018 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":12,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:10.368502+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.019+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f050006c890 msgr2=0x7f050006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.019+0000 7f0526400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f050006c890 0x7f050006ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05201fc130 tx=0x7f050c009040 comp rx=0 tx=0).stop 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.019+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b3b10 msgr2=0x7f052013c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.019+0000 7f0526400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b3b10 0x7f052013c570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f051000d5f0 tx=0x7f051000dab0 comp rx=0 tx=0).stop 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.019+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 shutdown_connections 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.019+0000 7f0526400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f050006c890 0x7f050006ed50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.020+0000 7f0526400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0520149000 0x7f052015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.020+0000 7f0526400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05200b4980 0x7f052015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.020+0000 7f0526400640 1 --2- 172.21.15.59:0/210399760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05200b3b10 0x7f052013c570 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.020+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 >> 172.21.15.59:0/210399760 conn(0x7f052006b9e0 msgr2=0x7f0520162b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.020+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 shutdown_connections 2024-04-23T14:31:13.021 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:13.020+0000 7f0526400640 1 -- 172.21.15.59:0/210399760 wait complete. 2024-04-23T14:31:13.034 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-23T14:31:18.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:18.318 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:19.841 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:20.034 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.033+0000 7f6899a00640 1 Processor -- start 2024-04-23T14:31:20.034 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.033+0000 7f6899a00640 1 -- start start 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6899a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6894156b80 0x7f689415d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6899a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f68940b3200 0x7f68940aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6899a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68940b3a10 con 0x7f68940af520 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6899a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f689415ed90 con 0x7f68940b3200 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6899a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f68940b3cf0 con 0x7f6894156b80 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f688be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f68940b3200 0x7f68940aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6899000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f68940af520 0x7f68941601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47876/0 (socket says 172.21.15.59:47876) 2024-04-23T14:31:20.036 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6899000640 1 -- 172.21.15.59:0/2652606728 learned_addr learned my addr 172.21.15.59:0/2652606728 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:20.037 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.035+0000 7f6893e00640 1 --2- 172.21.15.59:0/2652606728 >> v2:172.21.15.76:3300/0 conn(0x7f6894156b80 0x7f689415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.037 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.036+0000 7f6893400640 1 -- 172.21.15.59:0/2652606728 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654538985 0 0) 0x7f68940b3a10 con 0x7f68940af520 2024-04-23T14:31:20.037 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.036+0000 7f6893400640 1 -- 172.21.15.59:0/2652606728 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6878003520 con 0x7f68940af520 2024-04-23T14:31:20.037 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.036+0000 7f688be00640 1 -- 172.21.15.59:0/2652606728 >> v2:172.21.15.76:3300/0 conn(0x7f6894156b80 msgr2=0x7f689415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:20.037 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.036+0000 7f688be00640 1 --2- 172.21.15.59:0/2652606728 >> v2:172.21.15.76:3300/0 conn(0x7f6894156b80 0x7f689415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.037 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.036+0000 7f688be00640 1 -- 172.21.15.59:0/2652606728 >> v1:172.21.15.59:6789/0 conn(0x7f68940af520 legacy=0x7f68941601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.038 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.036+0000 7f688be00640 1 -- 172.21.15.59:0/2652606728 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68940b3fd0 con 0x7f68940b3200 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.037+0000 7f688be00640 1 --2- 172.21.15.59:0/2652606728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68940b3200 0x7f68940aefe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68940716d0 tx=0x7f687c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe14b4600a4d5c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.037+0000 7f6893400640 1 -- 172.21.15.59:0/2652606728 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c037070 con 0x7f68940b3200 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.037+0000 7f6893400640 1 -- 172.21.15.59:0/2652606728 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687c032440 con 0x7f68940b3200 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.037+0000 7f6893400640 1 -- 172.21.15.59:0/2652606728 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c032810 con 0x7f68940b3200 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 -- 172.21.15.59:0/2652606728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68940b3200 msgr2=0x7f68940aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 --2- 172.21.15.59:0/2652606728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68940b3200 0x7f68940aefe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68940716d0 tx=0x7f687c02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 -- 172.21.15.59:0/2652606728 shutdown_connections 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 --2- 172.21.15.59:0/2652606728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68940b3200 0x7f68940aefe0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 --2- 172.21.15.59:0/2652606728 >> v2:172.21.15.76:3300/0 conn(0x7f6894156b80 0x7f689415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.039 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 -- 172.21.15.59:0/2652606728 >> 172.21.15.59:0/2652606728 conn(0x7f68940b1f00 msgr2=0x7f68940b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:20.040 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 -- 172.21.15.59:0/2652606728 shutdown_connections 2024-04-23T14:31:20.040 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.038+0000 7f6899a00640 1 -- 172.21.15.59:0/2652606728 wait complete. 2024-04-23T14:31:20.041 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.040+0000 7f6899a00640 1 Processor -- start 2024-04-23T14:31:20.041 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.040+0000 7f6899a00640 1 -- start start 2024-04-23T14:31:20.043 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68940af520 0x7f689409c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.043 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68940b3200 0x7f68940cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.043 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6894156b80 0x7f68940cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.043 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f68940af990 con 0x7f6894156b80 2024-04-23T14:31:20.043 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f68940b4590 con 0x7f68940b3200 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f68940b36b0 con 0x7f68940af520 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6894156b80 0x7f68940cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6894156b80 0x7f68940cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59520/0 (socket says 172.21.15.59:59520) 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.041+0000 7f6899000640 1 -- 172.21.15.59:0/755826012 learned_addr learned my addr 172.21.15.59:0/755826012 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f688be00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68940b3200 0x7f68940cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6893e00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68940af520 0x7f689409c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6899000640 1 -- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68940af520 msgr2=0x7f689409c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6899000640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68940af520 0x7f689409c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6899000640 1 -- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68940b3200 msgr2=0x7f68940cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6899000640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68940b3200 0x7f68940cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6899000640 1 -- 172.21.15.59:0/755826012 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689409ce60 con 0x7f6894156b80 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6899000640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6894156b80 0x7f68940cfd80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c460 tx=0x7f688c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6890c00640 1 -- 172.21.15.59:0/755826012 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c017070 con 0x7f6894156b80 2024-04-23T14:31:20.044 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6890c00640 1 -- 172.21.15.59:0/755826012 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c005ce0 con 0x7f6894156b80 2024-04-23T14:31:20.045 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6890c00640 1 -- 172.21.15.59:0/755826012 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c0072b0 con 0x7f6894156b80 2024-04-23T14:31:20.045 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.042+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689409d140 con 0x7f6894156b80 2024-04-23T14:31:20.045 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.043+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6894151c90 con 0x7f6894156b80 2024-04-23T14:31:20.045 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.044+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689404fd90 con 0x7f6894156b80 2024-04-23T14:31:20.045 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.045+0000 7f6890c00640 1 -- 172.21.15.59:0/755826012 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f688c01a070 con 0x7f6894156b80 2024-04-23T14:31:20.047 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.046+0000 7f6890c00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f687006c920 0x7f687006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.047 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.046+0000 7f6890c00640 1 -- 172.21.15.59:0/755826012 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f688c08abc0 con 0x7f6894156b80 2024-04-23T14:31:20.047 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.046+0000 7f6893e00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f687006c920 0x7f687006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.047 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.047+0000 7f6893e00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f687006c920 0x7f687006ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68840073f0 tx=0x7f688400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.052 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.051+0000 7f6890c00640 1 -- 172.21.15.59:0/755826012 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f688c0592a0 con 0x7f6894156b80 2024-04-23T14:31:20.265 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.264+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68940cd5c0 con 0x7f6894156b80 2024-04-23T14:31:20.268 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.267+0000 7f6890c00640 1 -- 172.21.15.59:0/755826012 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12824 (secure 0 0 0) 0x7f688c058ba0 con 0x7f6894156b80 2024-04-23T14:31:20.268 INFO:teuthology.misc.health.smithi059.stdout: 2024-04-23T14:31:20.268 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":16,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:16.697460+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:20.270 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f687006c920 msgr2=0x7f687006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f687006c920 0x7f687006ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68840073f0 tx=0x7f688400b040 comp rx=0 tx=0).stop 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6894156b80 msgr2=0x7f68940cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6894156b80 0x7f68940cfd80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c460 tx=0x7f688c00c920 comp rx=0 tx=0).stop 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 shutdown_connections 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f687006c920 0x7f687006ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6894156b80 0x7f68940cfd80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68940b3200 0x7f68940cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 --2- 172.21.15.59:0/755826012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68940af520 0x7f689409c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.269+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 >> 172.21.15.59:0/755826012 conn(0x7f68940b1f00 msgr2=0x7f689406f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.270+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 shutdown_connections 2024-04-23T14:31:20.271 INFO:teuthology.misc.health.smithi059.stderr:2024-04-23T14:31:20.270+0000 7f6899a00640 1 -- 172.21.15.59:0/755826012 wait complete. 2024-04-23T14:31:20.286 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-23T14:31:20.286 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-23T14:31:20.287 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:31:20.287 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:31:20.294 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:31:20.294 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:31:20.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:31:20.301 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:31:20.301 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:31:20.301 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:31:20.309 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:31:20.309 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:31:20.309 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:31:20.317 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:31:20.317 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-23T14:31:20.317 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:20.442 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:31:20.443 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:31:20.465 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:31:20.473 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:31:20.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.500+0000 7f5fc3000640 1 Processor -- start 2024-04-23T14:31:20.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.500+0000 7f5fc3000640 1 -- start start 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fc3000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fbc0d0370 0x7f5fbc0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fc3000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fbc15e200 0x7f5fbc15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fc3000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fbc0cd9f0 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fc3000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5fbc09b6f0 con 0x7f5fbc0d0370 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fc3000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5fbc0cd790 con 0x7f5fbc15e200 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fbbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5fbc0d0b70 0x7f5fbc0d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47892/0 (socket says 172.21.15.59:47892) 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fbbe00640 1 -- 172.21.15.59:0/2733495138 learned_addr learned my addr 172.21.15.59:0/2733495138 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fbb400640 1 --2- 172.21.15.59:0/2733495138 >> v2:172.21.15.59:3300/0 conn(0x7f5fbc0d0370 0x7f5fbc0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fc1000640 1 --2- 172.21.15.59:0/2733495138 >> v2:172.21.15.76:3300/0 conn(0x7f5fbc15e200 0x7f5fbc15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fbaa00640 1 -- 172.21.15.59:0/2733495138 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277990416 0 0) 0x7f5fbc0cd9f0 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.502+0000 7f5fbaa00640 1 -- 172.21.15.59:0/2733495138 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fa0003520 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.503+0000 7f5fbb400640 1 -- 172.21.15.59:0/2733495138 >> v2:172.21.15.76:3300/0 conn(0x7f5fbc15e200 msgr2=0x7f5fbc15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.503+0000 7f5fbb400640 1 --2- 172.21.15.59:0/2733495138 >> v2:172.21.15.76:3300/0 conn(0x7f5fbc15e200 0x7f5fbc15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.503+0000 7f5fbb400640 1 -- 172.21.15.59:0/2733495138 >> v1:172.21.15.59:6789/0 conn(0x7f5fbc0d0b70 legacy=0x7f5fbc0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.503+0000 7f5fbb400640 1 -- 172.21.15.59:0/2733495138 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc0cdcb0 con 0x7f5fbc0d0370 2024-04-23T14:31:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.504+0000 7f5fbb400640 1 --2- 172.21.15.59:0/2733495138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0370 0x7f5fbc0cd250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097a0 tx=0x7f5fa402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8439f595e1aad2a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.504+0000 7f5fbaa00640 1 -- 172.21.15.59:0/2733495138 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4037070 con 0x7f5fbc0d0370 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.504+0000 7f5fbaa00640 1 -- 172.21.15.59:0/2733495138 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa402ae50 con 0x7f5fbc0d0370 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.504+0000 7f5fbaa00640 1 -- 172.21.15.59:0/2733495138 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4032660 con 0x7f5fbc0d0370 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 -- 172.21.15.59:0/2733495138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0370 msgr2=0x7f5fbc0cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 --2- 172.21.15.59:0/2733495138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0370 0x7f5fbc0cd250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097a0 tx=0x7f5fa402a140 comp rx=0 tx=0).stop 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 -- 172.21.15.59:0/2733495138 shutdown_connections 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 --2- 172.21.15.59:0/2733495138 >> v2:172.21.15.76:3300/0 conn(0x7f5fbc15e200 0x7f5fbc15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 --2- 172.21.15.59:0/2733495138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0370 0x7f5fbc0cd250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 -- 172.21.15.59:0/2733495138 >> 172.21.15.59:0/2733495138 conn(0x7f5fbc0b4080 msgr2=0x7f5fbc0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 -- 172.21.15.59:0/2733495138 shutdown_connections 2024-04-23T14:31:20.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.505+0000 7f5fc3000640 1 -- 172.21.15.59:0/2733495138 wait complete. 2024-04-23T14:31:20.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.506+0000 7f5fc3000640 1 Processor -- start 2024-04-23T14:31:20.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.507+0000 7f5fc3000640 1 -- start start 2024-04-23T14:31:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.507+0000 7f5fc3000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5fbc0d0370 0x7f5fbc155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.507+0000 7f5fc3000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0b70 0x7f5fbc1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fc3000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fbc15e200 0x7f5fbc15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fc3000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc15eb80 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fc3000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc0ce270 con 0x7f5fbc15e200 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fc3000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5fbc15ed00 con 0x7f5fbc0d0370 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0b70 0x7f5fbc1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0b70 0x7f5fbc1f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59554/0 (socket says 172.21.15.59:59554) 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 -- 172.21.15.59:0/429992482 learned_addr learned my addr 172.21.15.59:0/429992482 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbbe00640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5fbc0d0370 0x7f5fbc155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fc1000640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fbc15e200 0x7f5fbc15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 -- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5fbc0d0370 msgr2=0x7f5fbc155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5fbc0d0370 0x7f5fbc155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 -- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fbc15e200 msgr2=0x7f5fbc15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fbc15e200 0x7f5fbc15aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.508+0000 7f5fbb400640 1 -- 172.21.15.59:0/429992482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc1f5950 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.509+0000 7f5fbb400640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0b70 0x7f5fbc1f4f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4035040 tx=0x7f5fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.509+0000 7f5fb3e00640 1 -- 172.21.15.59:0/429992482 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4041070 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.509+0000 7f5fb3e00640 1 -- 172.21.15.59:0/429992482 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa403c460 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.509+0000 7f5fb3e00640 1 -- 172.21.15.59:0/429992482 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4037040 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.509+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fbc1f5c30 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.509+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fbc0b6b00 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.510+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fbc068a80 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.511+0000 7f5fb3e00640 1 -- 172.21.15.59:0/429992482 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fa403a070 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.512+0000 7f5fb3e00640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f9806c920 0x7f5f9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.512+0000 7f5fb3e00640 1 -- 172.21.15.59:0/429992482 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5fa40ab920 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.512+0000 7f5fbbe00640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f9806c920 0x7f5f9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.513+0000 7f5fbbe00640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f9806c920 0x7f5f9806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005c00 tx=0x7f5fac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.517+0000 7f5fb3e00640 1 -- 172.21.15.59:0/429992482 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fa4079a40 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.723+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fbc06d6d0 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.726+0000 7f5fb3e00640 1 -- 172.21.15.59:0/429992482 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12824 (secure 0 0 0) 0x7f5fa407dd40 con 0x7f5fbc0d0b70 2024-04-23T14:31:20.728 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:20.728 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:16.697460+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:20.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f9806c920 msgr2=0x7f5f9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f9806c920 0x7f5f9806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005c00 tx=0x7f5fac009040 comp rx=0 tx=0).stop 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0b70 msgr2=0x7f5fbc1f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0b70 0x7f5fbc1f4f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4035040 tx=0x7f5fa40046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 shutdown_connections 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f9806c920 0x7f5f9806ede0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fbc15e200 0x7f5fbc15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fbc0d0b70 0x7f5fbc1f4f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 --2- 172.21.15.59:0/429992482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5fbc0d0370 0x7f5fbc155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.729+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 >> 172.21.15.59:0/429992482 conn(0x7f5fbc0b4080 msgr2=0x7f5fbc06d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.730+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 shutdown_connections 2024-04-23T14:31:20.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.730+0000 7f5fc3000640 1 -- 172.21.15.59:0/429992482 wait complete. 2024-04-23T14:31:20.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:31:20.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:31:20.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.952+0000 7f0233200640 1 Processor -- start 2024-04-23T14:31:20.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.952+0000 7f0233200640 1 -- start start 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f0233200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02341567e0 0x7f0234156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f0233200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f023415d810 0x7f02340d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f0233200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023415cab0 con 0x7f02340d0b40 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f0233200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f02340d0f90 con 0x7f023415d810 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f0233200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0234160e40 con 0x7f02341567e0 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f022be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f023415d810 0x7f02340d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f022be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f023415d810 0x7f02340d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59570/0 (socket says 172.21.15.59:59570) 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f022be00640 1 -- 172.21.15.59:0/1246541692 learned_addr learned my addr 172.21.15.59:0/1246541692 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:20.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.954+0000 7f0231e00640 1 --2- 172.21.15.59:0/1246541692 >> v2:172.21.15.76:3300/0 conn(0x7f02341567e0 0x7f0234156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.955+0000 7f022be00640 1 -- 172.21.15.59:0/1246541692 >> v2:172.21.15.76:3300/0 conn(0x7f02341567e0 msgr2=0x7f0234156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:20.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.955+0000 7f022be00640 1 --2- 172.21.15.59:0/1246541692 >> v2:172.21.15.76:3300/0 conn(0x7f02341567e0 0x7f0234156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.955+0000 7f022be00640 1 -- 172.21.15.59:0/1246541692 >> v1:172.21.15.76:6789/0 conn(0x7f02340d0b40 legacy=0x7f0234160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:20.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.955+0000 7f022be00640 1 -- 172.21.15.59:0/1246541692 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023415cd30 con 0x7f023415d810 2024-04-23T14:31:20.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.956+0000 7f022be00640 1 --2- 172.21.15.59:0/1246541692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f02340d0600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02340b6360 tx=0x7f021c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cd95c51911d054 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.956+0000 7f0231400640 1 -- 172.21.15.59:0/1246541692 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c037070 con 0x7f023415d810 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.956+0000 7f0231400640 1 -- 172.21.15.59:0/1246541692 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021c032440 con 0x7f023415d810 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.956+0000 7f0231400640 1 -- 172.21.15.59:0/1246541692 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c032810 con 0x7f023415d810 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.956+0000 7f0233200640 1 -- 172.21.15.59:0/1246541692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 msgr2=0x7f02340d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.956+0000 7f0233200640 1 --2- 172.21.15.59:0/1246541692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f02340d0600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02340b6360 tx=0x7f021c02a230 comp rx=0 tx=0).stop 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.957+0000 7f0233200640 1 -- 172.21.15.59:0/1246541692 shutdown_connections 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.957+0000 7f0233200640 1 --2- 172.21.15.59:0/1246541692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f02340d0600 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.957+0000 7f0233200640 1 --2- 172.21.15.59:0/1246541692 >> v2:172.21.15.76:3300/0 conn(0x7f02341567e0 0x7f0234156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.957+0000 7f0233200640 1 -- 172.21.15.59:0/1246541692 >> 172.21.15.59:0/1246541692 conn(0x7f023406bdf0 msgr2=0x7f023406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.957+0000 7f0233200640 1 -- 172.21.15.59:0/1246541692 shutdown_connections 2024-04-23T14:31:20.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.957+0000 7f0233200640 1 -- 172.21.15.59:0/1246541692 wait complete. 2024-04-23T14:31:20.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.958+0000 7f0233200640 1 Processor -- start 2024-04-23T14:31:20.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.959+0000 7f0233200640 1 -- start start 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0233200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02340d0b40 0x7f02340733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0233200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02341567e0 0x7f02341f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0233200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f023406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0233200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0234160820 con 0x7f023415d810 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0233200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f023415d2f0 con 0x7f02340d0b40 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0233200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f02341609a0 con 0x7f02341567e0 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f023406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f023406ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59580/0 (socket says 172.21.15.59:59580) 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 -- 172.21.15.59:0/2651556829 learned_addr learned my addr 172.21.15.59:0/2651556829 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f022be00640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02341567e0 0x7f02341f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0231e00640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02340d0b40 0x7f02340733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 -- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02341567e0 msgr2=0x7f02341f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02341567e0 0x7f02341f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 -- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02340d0b40 msgr2=0x7f02340733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02340d0b40 0x7f02340733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.960+0000 7f0232800640 1 -- 172.21.15.59:0/2651556829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023409dfe0 con 0x7f023415d810 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.961+0000 7f0232800640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f023406ee30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f022c00b9a0 tx=0x7f022c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.961+0000 7f022a000640 1 -- 172.21.15.59:0/2651556829 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c019070 con 0x7f023415d810 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.961+0000 7f022a000640 1 -- 172.21.15.59:0/2651556829 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c00cba0 con 0x7f023415d810 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.961+0000 7f022a000640 1 -- 172.21.15.59:0/2651556829 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c014440 con 0x7f023415d810 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.961+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0234157d40 con 0x7f023415d810 2024-04-23T14:31:20.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.961+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0234073900 con 0x7f023415d810 2024-04-23T14:31:20.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.962+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023404fd90 con 0x7f023415d810 2024-04-23T14:31:20.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.963+0000 7f022a000640 1 -- 172.21.15.59:0/2651556829 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f022c01c070 con 0x7f023415d810 2024-04-23T14:31:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.965+0000 7f022a000640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f021006c8f0 0x7f021006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.965+0000 7f022a000640 1 -- 172.21.15.59:0/2651556829 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f022c08b430 con 0x7f023415d810 2024-04-23T14:31:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.965+0000 7f0231e00640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f021006c8f0 0x7f021006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.966+0000 7f0231e00640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f021006c8f0 0x7f021006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0224006fd0 tx=0x7f0224008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:20.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:20.970+0000 7f022a000640 1 -- 172.21.15.59:0/2651556829 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f022c059ba0 con 0x7f023415d810 2024-04-23T14:31:21.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.174+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f023409e2c0 con 0x7f023415d810 2024-04-23T14:31:21.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.175+0000 7f022a000640 1 -- 172.21.15.59:0/2651556829 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f022c0594a0 con 0x7f023415d810 2024-04-23T14:31:21.175 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:31:21.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f021006c8f0 msgr2=0x7f021006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f021006c8f0 0x7f021006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0224006fd0 tx=0x7f0224008040 comp rx=0 tx=0).stop 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 msgr2=0x7f023406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f023406ee30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f022c00b9a0 tx=0x7f022c00be60 comp rx=0 tx=0).stop 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 shutdown_connections 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f021006c8f0 0x7f021006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02341567e0 0x7f02341f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f023415d810 0x7f023406ee30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 --2- 172.21.15.59:0/2651556829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02340d0b40 0x7f02340733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.177+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 >> 172.21.15.59:0/2651556829 conn(0x7f023406bdf0 msgr2=0x7f023415b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.178+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 shutdown_connections 2024-04-23T14:31:21.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.178+0000 7f0233200640 1 -- 172.21.15.59:0/2651556829 wait complete. 2024-04-23T14:31:21.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:31:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.399+0000 7fd80a400640 1 Processor -- start 2024-04-23T14:31:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.399+0000 7fd80a400640 1 -- start start 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.400+0000 7fd80a400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd80415c3d0 0x7fd8040cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.400+0000 7fd80a400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8040cd590 0x7fd8040b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd80a400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd804160210 con 0x7fd8040b4690 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd80a400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd8041604f0 con 0x7fd8040cd590 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd80a400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd8041607d0 con 0x7fd80415c3d0 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd803e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8040cd590 0x7fd8040b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd803e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8040cd590 0x7fd8040b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59592/0 (socket says 172.21.15.59:59592) 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd803e00640 1 -- 172.21.15.59:0/578552440 learned_addr learned my addr 172.21.15.59:0/578552440 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:21.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd809000640 1 --2- 172.21.15.59:0/578552440 >> v2:172.21.15.76:3300/0 conn(0x7fd80415c3d0 0x7fd8040cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd803e00640 1 -- 172.21.15.59:0/578552440 >> v2:172.21.15.76:3300/0 conn(0x7fd80415c3d0 msgr2=0x7fd8040cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd803e00640 1 --2- 172.21.15.59:0/578552440 >> v2:172.21.15.76:3300/0 conn(0x7fd80415c3d0 0x7fd8040cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.401+0000 7fd803e00640 1 -- 172.21.15.59:0/578552440 >> v1:172.21.15.76:6789/0 conn(0x7fd8040b4690 legacy=0x7fd80406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:21.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.402+0000 7fd803e00640 1 -- 172.21.15.59:0/578552440 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd804160ab0 con 0x7fd8040cd590 2024-04-23T14:31:21.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.403+0000 7fd803e00640 1 --2- 172.21.15.59:0/578552440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040cd590 0x7fd8040b4150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd80406d9b0 tx=0x7fd7f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e8aaf058bc9053 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:21.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.403+0000 7fd803400640 1 -- 172.21.15.59:0/578552440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0037070 con 0x7fd8040cd590 2024-04-23T14:31:21.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.403+0000 7fd803400640 1 -- 172.21.15.59:0/578552440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f0032440 con 0x7fd8040cd590 2024-04-23T14:31:21.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.403+0000 7fd803400640 1 -- 172.21.15.59:0/578552440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0032810 con 0x7fd8040cd590 2024-04-23T14:31:21.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 -- 172.21.15.59:0/578552440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040cd590 msgr2=0x7fd8040b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 --2- 172.21.15.59:0/578552440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040cd590 0x7fd8040b4150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd80406d9b0 tx=0x7fd7f002a270 comp rx=0 tx=0).stop 2024-04-23T14:31:21.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 -- 172.21.15.59:0/578552440 shutdown_connections 2024-04-23T14:31:21.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 --2- 172.21.15.59:0/578552440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040cd590 0x7fd8040b4150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 --2- 172.21.15.59:0/578552440 >> v2:172.21.15.76:3300/0 conn(0x7fd80415c3d0 0x7fd8040cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 -- 172.21.15.59:0/578552440 >> 172.21.15.59:0/578552440 conn(0x7fd8040b21a0 msgr2=0x7fd8040b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:21.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 -- 172.21.15.59:0/578552440 shutdown_connections 2024-04-23T14:31:21.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.404+0000 7fd80a400640 1 -- 172.21.15.59:0/578552440 wait complete. 2024-04-23T14:31:21.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.406+0000 7fd80a400640 1 Processor -- start 2024-04-23T14:31:21.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.406+0000 7fd80a400640 1 -- start start 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd80a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040b4690 0x7fd8040ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd80a400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8040cd590 0x7fd8040ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd80a400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80415c3d0 0x7fd8041f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd80a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd80415c7b0 con 0x7fd8040b4690 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd80a400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8040b4ac0 con 0x7fd80415c3d0 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd80a400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd80415c930 con 0x7fd8040cd590 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd809000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040b4690 0x7fd8040ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd809000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040b4690 0x7fd8040ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59600/0 (socket says 172.21.15.59:59600) 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd809000640 1 -- 172.21.15.59:0/69738957 learned_addr learned my addr 172.21.15.59:0/69738957 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.407+0000 7fd809a00640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80415c3d0 0x7fd8041f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd809000640 1 -- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8040cd590 msgr2=0x7fd8040ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd809000640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8040cd590 0x7fd8040ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd809000640 1 -- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80415c3d0 msgr2=0x7fd8041f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd809000640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80415c3d0 0x7fd8041f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd809000640 1 -- 172.21.15.59:0/69738957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8040ad8a0 con 0x7fd8040b4690 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd809a00640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80415c3d0 0x7fd8041f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd809000640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040b4690 0x7fd8040ad2c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800d5f0 tx=0x7fd7f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd800c00640 1 -- 172.21.15.59:0/69738957 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80125f0 con 0x7fd8040b4690 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd800c00640 1 -- 172.21.15.59:0/69738957 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f8019070 con 0x7fd8040b4690 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd800c00640 1 -- 172.21.15.59:0/69738957 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80129d0 con 0x7fd8040b4690 2024-04-23T14:31:21.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8040adb80 con 0x7fd8040b4690 2024-04-23T14:31:21.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.408+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8040ae140 con 0x7fd8040b4690 2024-04-23T14:31:21.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.410+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80404fd90 con 0x7fd8040b4690 2024-04-23T14:31:21.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.411+0000 7fd800c00640 1 -- 172.21.15.59:0/69738957 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7f8012b70 con 0x7fd8040b4690 2024-04-23T14:31:21.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.412+0000 7fd800c00640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7e406c8f0 0x7fd7e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.412+0000 7fd800c00640 1 -- 172.21.15.59:0/69738957 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd7f808bc60 con 0x7fd8040b4690 2024-04-23T14:31:21.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.412+0000 7fd803e00640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7e406c8f0 0x7fd7e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.413+0000 7fd803e00640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7e406c8f0 0x7fd7e406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0034000 tx=0x7fd7f002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:21.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.417+0000 7fd800c00640 1 -- 172.21.15.59:0/69738957 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7f805a3d0 con 0x7fd8040b4690 2024-04-23T14:31:21.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.661+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8040cfca0 con 0x7fd8040b4690 2024-04-23T14:31:21.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.662+0000 7fd800c00640 1 -- 172.21.15.59:0/69738957 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7fd7f8059cd0 con 0x7fd8040b4690 2024-04-23T14:31:21.663 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:21.663 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T14:31:21.663 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2024-04-23T14:31:21.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7e406c8f0 msgr2=0x7fd7e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7e406c8f0 0x7fd7e406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0034000 tx=0x7fd7f002c000 comp rx=0 tx=0).stop 2024-04-23T14:31:21.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040b4690 msgr2=0x7fd8040ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040b4690 0x7fd8040ad2c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800d5f0 tx=0x7fd7f800dab0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 shutdown_connections 2024-04-23T14:31:21.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7e406c8f0 0x7fd7e406edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8040cd590 0x7fd8040ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80415c3d0 0x7fd8041f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 --2- 172.21.15.59:0/69738957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8040b4690 0x7fd8040ad2c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 >> 172.21.15.59:0/69738957 conn(0x7fd8040b21a0 msgr2=0x7fd80406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 shutdown_connections 2024-04-23T14:31:21.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.665+0000 7fd80a400640 1 -- 172.21.15.59:0/69738957 wait complete. 2024-04-23T14:31:21.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:21.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.883+0000 7ff101200640 1 Processor -- start 2024-04-23T14:31:21.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.883+0000 7ff101200640 1 -- start start 2024-04-23T14:31:21.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.884+0000 7ff101200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0fc0cd760 0x7ff0fc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff101200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc160210 con 0x7ff0fc0b4690 2024-04-23T14:31:21.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff101200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc1604f0 con 0x7ff0fc15c3d0 2024-04-23T14:31:21.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff101200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff0fc1607d0 con 0x7ff0fc0cd760 2024-04-23T14:31:21.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff0fbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff0fc0b4690 0x7ff0fc0cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47902/0 (socket says 172.21.15.59:47902) 2024-04-23T14:31:21.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff0fbe00640 1 -- 172.21.15.59:0/1882767026 learned_addr learned my addr 172.21.15.59:0/1882767026 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:21.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/1882767026 >> v2:172.21.15.76:3300/0 conn(0x7ff0fc0cd760 0x7ff0fc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301501845 0 0) 0x7ff0fc160210 con 0x7ff0fc0b4690 2024-04-23T14:31:21.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.885+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0e0003520 con 0x7ff0fc0b4690 2024-04-23T14:31:21.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1052828747 0 0) 0x7ff0e0003520 con 0x7ff0fc0b4690 2024-04-23T14:31:21.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 >> v2:172.21.15.76:3300/0 conn(0x7ff0fc0cd760 msgr2=0x7ff0fc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:21.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 --2- 172.21.15.59:0/1882767026 >> v2:172.21.15.76:3300/0 conn(0x7ff0fc0cd760 0x7ff0fc0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 >> v1:172.21.15.76:6789/0 conn(0x7ff0fc15c3d0 legacy=0x7ff0fc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc160ab0 con 0x7ff0fc0b4690 2024-04-23T14:31:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7ff0f40019f0 con 0x7ff0fc0b4690 2024-04-23T14:31:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 >> v1:172.21.15.59:6789/0 conn(0x7ff0fc0b4690 legacy=0x7ff0fc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e00039b0 0x7ff0e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff0e00051c0 0x7ff0e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc1607d0 con 0x7ff0e0004800 2024-04-23T14:31:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e000c060 con 0x7ff0e00039b0 2024-04-23T14:31:21.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e000c260 con 0x7ff0e00051c0 2024-04-23T14:31:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.886+0000 7ff0fb400640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e00039b0 0x7ff0e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.887+0000 7ff0f3e00640 1 -- 172.21.15.59:0/1882767026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff0e00051c0 msgr2=0x7ff0e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.887+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff0e00051c0 0x7ff0e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.887+0000 7ff0f3e00640 1 -- 172.21.15.59:0/1882767026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e00039b0 msgr2=0x7ff0e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:21.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.887+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0e00039b0 0x7ff0e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.887+0000 7ff0f3e00640 1 -- 172.21.15.59:0/1882767026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e0003520 con 0x7ff0e0004800 2024-04-23T14:31:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.887+0000 7ff0fbe00640 1 -- 172.21.15.59:0/1882767026 reap_dead start 2024-04-23T14:31:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.888+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009ac0 tx=0x7ff0e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36e319fc1444c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.888+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4036070 con 0x7ff0e0004800 2024-04-23T14:31:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.888+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e4031400 con 0x7ff0e0004800 2024-04-23T14:31:21.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.888+0000 7ff0faa00640 1 -- 172.21.15.59:0/1882767026 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4009690 con 0x7ff0e0004800 2024-04-23T14:31:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.889+0000 7ff101200640 1 -- 172.21.15.59:0/1882767026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 msgr2=0x7ff0e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.889+0000 7ff101200640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009ac0 tx=0x7ff0e0005830 comp rx=0 tx=0).stop 2024-04-23T14:31:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.889+0000 7ff101200640 1 -- 172.21.15.59:0/1882767026 shutdown_connections 2024-04-23T14:31:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.889+0000 7ff101200640 1 --2- 172.21.15.59:0/1882767026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0e0004c80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.889+0000 7ff101200640 1 -- 172.21.15.59:0/1882767026 >> 172.21.15.59:0/1882767026 conn(0x7ff0fc0b21a0 msgr2=0x7ff0fc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.889+0000 7ff101200640 1 -- 172.21.15.59:0/1882767026 shutdown_connections 2024-04-23T14:31:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.889+0000 7ff101200640 1 -- 172.21.15.59:0/1882767026 wait complete. 2024-04-23T14:31:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.890+0000 7ff101200640 1 Processor -- start 2024-04-23T14:31:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.891+0000 7ff101200640 1 -- start start 2024-04-23T14:31:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.891+0000 7ff101200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0fc0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.891+0000 7ff101200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0c60f0 0x7ff0fc15f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.891+0000 7ff101200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff0fc06ee50 0x7ff0fc06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.891+0000 7ff101200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc15c7b0 con 0x7ff0e0004800 2024-04-23T14:31:21.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.891+0000 7ff101200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc0b4ae0 con 0x7ff0fc0c60f0 2024-04-23T14:31:21.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.891+0000 7ff101200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc15c930 con 0x7ff0fc06ee50 2024-04-23T14:31:21.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0fc0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0fc0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59612/0 (socket says 172.21.15.59:59612) 2024-04-23T14:31:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 -- 172.21.15.59:0/2120116029 learned_addr learned my addr 172.21.15.59:0/2120116029 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0c60f0 0x7ff0fc15f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 -- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff0fc06ee50 msgr2=0x7ff0fc06f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff0fc06ee50 0x7ff0fc06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 -- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0c60f0 msgr2=0x7ff0fc15f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0c60f0 0x7ff0fc15f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 -- 172.21.15.59:0/2120116029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc0c9950 con 0x7ff0e0004800 2024-04-23T14:31:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0c60f0 0x7ff0fc15f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.892+0000 7ff0fb400640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0fc0c9340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00e580 tx=0x7ff0ec00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:21.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.893+0000 7ff0f3400640 1 -- 172.21.15.59:0/2120116029 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0052e0 con 0x7ff0e0004800 2024-04-23T14:31:21.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.893+0000 7ff0f3400640 1 -- 172.21.15.59:0/2120116029 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec018070 con 0x7ff0e0004800 2024-04-23T14:31:21.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.893+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc0c9c30 con 0x7ff0e0004800 2024-04-23T14:31:21.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.893+0000 7ff0f3400640 1 -- 172.21.15.59:0/2120116029 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec003810 con 0x7ff0e0004800 2024-04-23T14:31:21.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.893+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc15cbe0 con 0x7ff0e0004800 2024-04-23T14:31:21.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.894+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc04fd90 con 0x7ff0e0004800 2024-04-23T14:31:21.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.895+0000 7ff0f3400640 1 -- 172.21.15.59:0/2120116029 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0ec0039b0 con 0x7ff0e0004800 2024-04-23T14:31:21.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.896+0000 7ff0f3400640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff0d806c920 0x7ff0d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:21.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.896+0000 7ff0f3400640 1 -- 172.21.15.59:0/2120116029 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff0ec08b490 con 0x7ff0e0004800 2024-04-23T14:31:21.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.896+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff0d806c920 0x7ff0d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:21.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.897+0000 7ff0f3e00640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff0d806c920 0x7ff0d806ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4033000 tx=0x7ff0e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:21.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:21.901+0000 7ff0f3400640 1 -- 172.21.15.59:0/2120116029 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0ec059b80 con 0x7ff0e0004800 2024-04-23T14:31:22.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.105+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0fc06e400 con 0x7ff0e0004800 2024-04-23T14:31:22.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.107+0000 7ff0f3400640 1 -- 172.21.15.59:0/2120116029 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12824 (secure 0 0 0) 0x7ff0ec059480 con 0x7ff0e0004800 2024-04-23T14:31:22.108 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:22.108 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:16.697460+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:22.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff0d806c920 msgr2=0x7ff0d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff0d806c920 0x7ff0d806ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4033000 tx=0x7ff0e4008030 comp rx=0 tx=0).stop 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 msgr2=0x7ff0fc0c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0fc0c9340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00e580 tx=0x7ff0ec00ea40 comp rx=0 tx=0).stop 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 shutdown_connections 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff0d806c920 0x7ff0d806ede0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff0fc06ee50 0x7ff0fc06f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0c60f0 0x7ff0fc15f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 --2- 172.21.15.59:0/2120116029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0e0004800 0x7ff0fc0c9340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.109+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 >> 172.21.15.59:0/2120116029 conn(0x7ff0fc0b21a0 msgr2=0x7ff0fc0b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:22.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.110+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 shutdown_connections 2024-04-23T14:31:22.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.110+0000 7ff101200640 1 -- 172.21.15.59:0/2120116029 wait complete. 2024-04-23T14:31:22.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:31:22.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:31:22.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.332+0000 7f9742000640 1 Processor -- start 2024-04-23T14:31:22.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.333+0000 7f9742000640 1 -- start start 2024-04-23T14:31:22.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9742000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f973c15ed20 0x7f973c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9742000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9742000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f973c0b5da0 con 0x7f973c0b3f10 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9742000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f973c15f100 con 0x7f973c15ed20 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9742000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f973c0d0a20 con 0x7f973c0b3600 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9740c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f973c15ed20 0x7f973c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9740c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f973c15ed20 0x7f973c0d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59622/0 (socket says 172.21.15.59:59622) 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.334+0000 7f9740c00640 1 -- 172.21.15.59:0/1883399077 learned_addr learned my addr 172.21.15.59:0/1883399077 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.335+0000 7f973be00640 1 --2- 172.21.15.59:0/1883399077 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.335+0000 7f9740c00640 1 -- 172.21.15.59:0/1883399077 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 msgr2=0x7f973c0d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.335+0000 7f9740c00640 1 --2- 172.21.15.59:0/1883399077 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.335+0000 7f9740c00640 1 -- 172.21.15.59:0/1883399077 >> v1:172.21.15.76:6789/0 conn(0x7f973c0b3f10 legacy=0x7f973c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.335+0000 7f9740c00640 1 -- 172.21.15.59:0/1883399077 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c0d2ed0 con 0x7f973c15ed20 2024-04-23T14:31:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.335+0000 7f973be00640 1 --2- 172.21.15.59:0/1883399077 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.336+0000 7f9740c00640 1 --2- 172.21.15.59:0/1883399077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0d0ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9730009870 tx=0x7f973002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93d6d003a8c32f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.336+0000 7f973b400640 1 -- 172.21.15.59:0/1883399077 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730037070 con 0x7f973c15ed20 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.336+0000 7f973b400640 1 -- 172.21.15.59:0/1883399077 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973002ae10 con 0x7f973c15ed20 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.336+0000 7f973b400640 1 -- 172.21.15.59:0/1883399077 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730032660 con 0x7f973c15ed20 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 -- 172.21.15.59:0/1883399077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 msgr2=0x7f973c0d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 --2- 172.21.15.59:0/1883399077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0d0ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9730009870 tx=0x7f973002a100 comp rx=0 tx=0).stop 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 -- 172.21.15.59:0/1883399077 shutdown_connections 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 --2- 172.21.15.59:0/1883399077 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 --2- 172.21.15.59:0/1883399077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0d0ea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 -- 172.21.15.59:0/1883399077 >> 172.21.15.59:0/1883399077 conn(0x7f973c08bf90 msgr2=0x7f973c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 -- 172.21.15.59:0/1883399077 shutdown_connections 2024-04-23T14:31:22.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.337+0000 7f9742000640 1 -- 172.21.15.59:0/1883399077 wait complete. 2024-04-23T14:31:22.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.339+0000 7f9742000640 1 Processor -- start 2024-04-23T14:31:22.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.339+0000 7f9742000640 1 -- start start 2024-04-23T14:31:22.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9742000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9742000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973c0b3f10 0x7f973c15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9742000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9742000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f973c0d0290 con 0x7f973c15ed20 2024-04-23T14:31:22.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9742000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f973c0d3490 con 0x7f973c0b3f10 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9742000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f973c0d0410 con 0x7f973c0b3600 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9741600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9741600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59624/0 (socket says 172.21.15.59:59624) 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9741600640 1 -- 172.21.15.59:0/2825992 learned_addr learned my addr 172.21.15.59:0/2825992 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9740c00640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f973be00640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973c0b3f10 0x7f973c15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9741600640 1 -- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 msgr2=0x7f973c0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9741600640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9741600640 1 -- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973c0b3f10 msgr2=0x7f973c15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.340+0000 7f9741600640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973c0b3f10 0x7f973c15acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.341+0000 7f9741600640 1 -- 172.21.15.59:0/2825992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c0b6a40 con 0x7f973c15ed20 2024-04-23T14:31:22.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.341+0000 7f9741600640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0ce000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f973400c780 tx=0x7f973400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:22.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.341+0000 7f9738c00640 1 -- 172.21.15.59:0/2825992 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734004b60 con 0x7f973c15ed20 2024-04-23T14:31:22.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.341+0000 7f9738c00640 1 -- 172.21.15.59:0/2825992 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9734004d00 con 0x7f973c15ed20 2024-04-23T14:31:22.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.341+0000 7f9738c00640 1 -- 172.21.15.59:0/2825992 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973400aa90 con 0x7f973c15ed20 2024-04-23T14:31:22.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.341+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c0b6d20 con 0x7f973c15ed20 2024-04-23T14:31:22.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.341+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c0b85c0 con 0x7f973c15ed20 2024-04-23T14:31:22.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.343+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973c04fd90 con 0x7f973c15ed20 2024-04-23T14:31:22.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.344+0000 7f9738c00640 1 -- 172.21.15.59:0/2825992 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f973400ac30 con 0x7f973c15ed20 2024-04-23T14:31:22.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.344+0000 7f9738c00640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.345+0000 7f9738c00640 1 -- 172.21.15.59:0/2825992 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f973408bb30 con 0x7f973c15ed20 2024-04-23T14:31:22.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.345+0000 7f9740c00640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.346+0000 7f9740c00640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9730034000 tx=0x7f973002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:22.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.349+0000 7f9738c00640 1 -- 172.21.15.59:0/2825992 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9734020070 con 0x7f973c15ed20 2024-04-23T14:31:22.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.550+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f973c0b2640 con 0x7f973c15ed20 2024-04-23T14:31:22.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.551+0000 7f9738c00640 1 -- 172.21.15.59:0/2825992 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f973405a210 con 0x7f973c15ed20 2024-04-23T14:31:22.552 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.553+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 msgr2=0x7f971c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.553+0000 7f9742000640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9730034000 tx=0x7f973002e000 comp rx=0 tx=0).stop 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.553+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 msgr2=0x7f973c0ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.553+0000 7f9742000640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0ce000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f973400c780 tx=0x7f973400cc40 comp rx=0 tx=0).stop 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 shutdown_connections 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f973c15ed20 0x7f973c0ce000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973c0b3f10 0x7f973c15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 --2- 172.21.15.59:0/2825992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f973c0b3600 0x7f973c0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 >> 172.21.15.59:0/2825992 conn(0x7f973c08bf90 msgr2=0x7f973c0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:22.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 shutdown_connections 2024-04-23T14:31:22.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.554+0000 7f9742000640 1 -- 172.21.15.59:0/2825992 wait complete. 2024-04-23T14:31:22.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:31:22.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.774+0000 7f2acbe00640 1 Processor -- start 2024-04-23T14:31:22.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.774+0000 7f2acbe00640 1 -- start start 2024-04-23T14:31:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2acbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0cc7e0 0x7f2acc167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2acbe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2acc160210 con 0x7f2acc0cd760 2024-04-23T14:31:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2acbe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2acc1604f0 con 0x7f2acc15c3d0 2024-04-23T14:31:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2acbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc1607d0 con 0x7f2acc0cc7e0 2024-04-23T14:31:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2ac2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2acc0cd760 0x7f2acc0b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47910/0 (socket says 172.21.15.59:47910) 2024-04-23T14:31:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2ac2000640 1 -- 172.21.15.59:0/1574075708 learned_addr learned my addr 172.21.15.59:0/1574075708 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2acb400640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0cc7e0 0x7f2acc167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.776+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2217481239 0 0) 0x7f2acc160210 con 0x7f2acc0cd760 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab0003520 con 0x7f2acc0cd760 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 947723405 0 0) 0x7f2ab0003520 con 0x7f2acc0cd760 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0cc7e0 msgr2=0x7f2acc167400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0cc7e0 0x7f2acc167400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 >> v1:172.21.15.76:6789/0 conn(0x7f2acc15c3d0 legacy=0x7f2acc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc160ab0 con 0x7f2acc0cd760 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2ab4001670 con 0x7f2acc0cd760 2024-04-23T14:31:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 >> v1:172.21.15.59:6789/0 conn(0x7f2acc0cd760 legacy=0x7f2acc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab00039e0 0x7f2ab0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0004800 0x7f2ab0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0cc7e0 0x7f2ab00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc1607d0 con 0x7f2ab0004800 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab000baa0 con 0x7f2ab00039e0 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab000bca0 con 0x7f2acc0cc7e0 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2ac2000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0004800 0x7f2ab0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.777+0000 7f2acaa00640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab00039e0 0x7f2ab0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.778+0000 7f2ac2000640 1 -- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0cc7e0 msgr2=0x7f2ab00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.778+0000 7f2ac2000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0cc7e0 0x7f2ab00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.778+0000 7f2ac2000640 1 -- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab00039e0 msgr2=0x7f2ab0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.778+0000 7f2ac2000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab00039e0 0x7f2ab0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.778+0000 7f2ac2000640 1 -- 172.21.15.59:0/1574075708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab0003520 con 0x7f2ab0004800 2024-04-23T14:31:22.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.779+0000 7f2ac2000640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0004800 0x7f2ab0004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c210 tx=0x7f2ab0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e33c32e97d88dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.779+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4007d10 con 0x7f2ab0004800 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.779+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab4005ad0 con 0x7f2ab0004800 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.779+0000 7f2aca000640 1 -- 172.21.15.59:0/1574075708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4007070 con 0x7f2ab0004800 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.780+0000 7f2acbe00640 1 -- 172.21.15.59:0/1574075708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0004800 msgr2=0x7f2ab0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.780+0000 7f2acbe00640 1 --2- 172.21.15.59:0/1574075708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0004800 0x7f2ab0004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c210 tx=0x7f2ab0005830 comp rx=0 tx=0).stop 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.780+0000 7f2acb400640 1 -- 172.21.15.59:0/1574075708 reap_dead start 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.780+0000 7f2acbe00640 1 -- 172.21.15.59:0/1574075708 shutdown_connections 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.780+0000 7f2acbe00640 1 -- 172.21.15.59:0/1574075708 >> 172.21.15.59:0/1574075708 conn(0x7f2acc0b21a0 msgr2=0x7f2acc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.780+0000 7f2acbe00640 1 -- 172.21.15.59:0/1574075708 shutdown_connections 2024-04-23T14:31:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.780+0000 7f2acbe00640 1 -- 172.21.15.59:0/1574075708 wait complete. 2024-04-23T14:31:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.782+0000 7f2acbe00640 1 Processor -- start 2024-04-23T14:31:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.782+0000 7f2acbe00640 1 -- start start 2024-04-23T14:31:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acbe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004800 0x7f2acc1f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0cfad0 0x7f2acc0c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0ae0b0 0x7f2acc0726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc15c930 con 0x7f2acc0cfad0 2024-04-23T14:31:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acbe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc15c7b0 con 0x7f2ab0004800 2024-04-23T14:31:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc0b4860 con 0x7f2acc0ae0b0 2024-04-23T14:31:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0cfad0 0x7f2acc0c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0cfad0 0x7f2acc0c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59654/0 (socket says 172.21.15.59:59654) 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 -- 172.21.15.59:0/1134646425 learned_addr learned my addr 172.21.15.59:0/1134646425 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acaa00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004800 0x7f2acc1f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2acb400640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0ae0b0 0x7f2acc0726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 -- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0ae0b0 msgr2=0x7f2acc0726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0ae0b0 0x7f2acc0726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 -- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004800 msgr2=0x7f2acc1f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004800 0x7f2acc1f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.783+0000 7f2ac2000640 1 -- 172.21.15.59:0/1134646425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc1f85d0 con 0x7f2acc0cfad0 2024-04-23T14:31:22.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.784+0000 7f2ac2000640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0cfad0 0x7f2acc0c63f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c210 tx=0x7f2acc1603e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:22.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.784+0000 7f2ac3400640 1 -- 172.21.15.59:0/1134646425 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4007d10 con 0x7f2acc0cfad0 2024-04-23T14:31:22.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.784+0000 7f2ac3400640 1 -- 172.21.15.59:0/1134646425 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab4042070 con 0x7f2acc0cfad0 2024-04-23T14:31:22.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.784+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc1f88b0 con 0x7f2acc0cfad0 2024-04-23T14:31:22.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.784+0000 7f2ac3400640 1 -- 172.21.15.59:0/1134646425 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4001670 con 0x7f2acc0cfad0 2024-04-23T14:31:22.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.784+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc0c9680 con 0x7f2acc0cfad0 2024-04-23T14:31:22.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.786+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc04fd90 con 0x7f2acc0cfad0 2024-04-23T14:31:22.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.787+0000 7f2ac3400640 1 -- 172.21.15.59:0/1134646425 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ab40072b0 con 0x7f2acc0cfad0 2024-04-23T14:31:22.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.787+0000 7f2ac3400640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aa806c920 0x7f2aa806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:22.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.788+0000 7f2ac3400640 1 -- 172.21.15.59:0/1134646425 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2ab40ab870 con 0x7f2acc0cfad0 2024-04-23T14:31:22.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.788+0000 7f2acaa00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aa806c920 0x7f2aa806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:22.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.788+0000 7f2acaa00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aa806c920 0x7f2aa806ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2abc007fd0 tx=0x7f2abc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:22.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.792+0000 7f2ac3400640 1 -- 172.21.15.59:0/1134646425 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ab407a080 con 0x7f2acc0cfad0 2024-04-23T14:31:22.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:22.999+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2acc06e400 con 0x7f2acc0cfad0 2024-04-23T14:31:23.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.000+0000 7f2ac3400640 1 -- 172.21.15.59:0/1134646425 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f2ab4005aa0 con 0x7f2acc0cfad0 2024-04-23T14:31:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:23.001 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:31:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:31:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:31:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:31:23.001 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:31:23.001 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.002+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aa806c920 msgr2=0x7f2aa806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.002+0000 7f2acbe00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aa806c920 0x7f2aa806ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2abc007fd0 tx=0x7f2abc009040 comp rx=0 tx=0).stop 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.002+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0cfad0 msgr2=0x7f2acc0c63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.002+0000 7f2acbe00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0cfad0 0x7f2acc0c63f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400c210 tx=0x7f2acc1603e0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 shutdown_connections 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aa806c920 0x7f2aa806ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc0ae0b0 0x7f2acc0726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0cfad0 0x7f2acc0c63f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 --2- 172.21.15.59:0/1134646425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004800 0x7f2acc1f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 >> 172.21.15.59:0/1134646425 conn(0x7f2acc0b21a0 msgr2=0x7f2acc0d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 shutdown_connections 2024-04-23T14:31:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.003+0000 7f2acbe00640 1 -- 172.21.15.59:0/1134646425 wait complete. 2024-04-23T14:31:23.024 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:31:23.024 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:31:23.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.240+0000 7f272c000640 1 Processor -- start 2024-04-23T14:31:23.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.241+0000 7f272c000640 1 -- start start 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.242+0000 7f272c000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f27240b3930 0x7f272406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.242+0000 7f272c000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f272409b820 0x7f272409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.242+0000 7f272c000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27240d0ac0 con 0x7f272409af40 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.242+0000 7f272c000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f272409b390 con 0x7f272409b820 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.242+0000 7f272c000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f27240d16d0 con 0x7f27240b3930 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f2728c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f272409b820 0x7f272409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f272a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f272409af40 0x7f272415e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47912/0 (socket says 172.21.15.59:47912) 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f272a000640 1 -- 172.21.15.59:0/677415793 learned_addr learned my addr 172.21.15.59:0/677415793 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:23.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f2729600640 1 --2- 172.21.15.59:0/677415793 >> v2:172.21.15.76:3300/0 conn(0x7f27240b3930 0x7f272406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f2723e00640 1 -- 172.21.15.59:0/677415793 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291823713 0 0) 0x7f27240d0ac0 con 0x7f272409af40 2024-04-23T14:31:23.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f2723e00640 1 -- 172.21.15.59:0/677415793 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f270c003520 con 0x7f272409af40 2024-04-23T14:31:23.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f2728c00640 1 -- 172.21.15.59:0/677415793 >> v2:172.21.15.76:3300/0 conn(0x7f27240b3930 msgr2=0x7f272406ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:23.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f2728c00640 1 --2- 172.21.15.59:0/677415793 >> v2:172.21.15.76:3300/0 conn(0x7f27240b3930 0x7f272406ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.243+0000 7f2728c00640 1 -- 172.21.15.59:0/677415793 >> v1:172.21.15.59:6789/0 conn(0x7f272409af40 legacy=0x7f272415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.244+0000 7f2728c00640 1 -- 172.21.15.59:0/677415793 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27240d0da0 con 0x7f272409b820 2024-04-23T14:31:23.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.245+0000 7f2728c00640 1 --2- 172.21.15.59:0/677415793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272409b820 0x7f272409aa00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27240731b0 tx=0x7f271002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120b0739dbe318ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:23.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.245+0000 7f2723e00640 1 -- 172.21.15.59:0/677415793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710037070 con 0x7f272409b820 2024-04-23T14:31:23.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.245+0000 7f2723e00640 1 -- 172.21.15.59:0/677415793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271002add0 con 0x7f272409b820 2024-04-23T14:31:23.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.245+0000 7f2723e00640 1 -- 172.21.15.59:0/677415793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710032640 con 0x7f272409b820 2024-04-23T14:31:23.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 -- 172.21.15.59:0/677415793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272409b820 msgr2=0x7f272409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 --2- 172.21.15.59:0/677415793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272409b820 0x7f272409aa00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27240731b0 tx=0x7f271002a080 comp rx=0 tx=0).stop 2024-04-23T14:31:23.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 -- 172.21.15.59:0/677415793 shutdown_connections 2024-04-23T14:31:23.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 --2- 172.21.15.59:0/677415793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272409b820 0x7f272409aa00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 --2- 172.21.15.59:0/677415793 >> v2:172.21.15.76:3300/0 conn(0x7f27240b3930 0x7f272406ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 -- 172.21.15.59:0/677415793 >> 172.21.15.59:0/677415793 conn(0x7f27240b4080 msgr2=0x7f27240b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:23.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 -- 172.21.15.59:0/677415793 shutdown_connections 2024-04-23T14:31:23.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.246+0000 7f272c000640 1 -- 172.21.15.59:0/677415793 wait complete. 2024-04-23T14:31:23.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.248+0000 7f272c000640 1 Processor -- start 2024-04-23T14:31:23.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.248+0000 7f272c000640 1 -- start start 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272c000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f272409af40 0x7f27240c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272c000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272409b820 0x7f27240cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27240b3930 0x7f27240b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272c000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f272406d230 con 0x7f27240b3930 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272c000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f27240d1360 con 0x7f272409b820 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272c000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f27240b3d10 con 0x7f272409af40 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27240b3930 0x7f27240b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27240b3930 0x7f27240b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59676/0 (socket says 172.21.15.59:59676) 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.249+0000 7f272a000640 1 -- 172.21.15.59:0/3713688654 learned_addr learned my addr 172.21.15.59:0/3713688654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:23.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f2728c00640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272409b820 0x7f27240cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f2729600640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f272409af40 0x7f27240c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f272a000640 1 -- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f272409af40 msgr2=0x7f27240c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f272a000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f272409af40 0x7f27240c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f272a000640 1 -- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272409b820 msgr2=0x7f27240cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f272a000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272409b820 0x7f27240cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f272a000640 1 -- 172.21.15.59:0/3713688654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27240c9ba0 con 0x7f27240b3930 2024-04-23T14:31:23.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f272a000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27240b3930 0x7f27240b1eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f271c00cb40 tx=0x7f271c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:23.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.250+0000 7f2721600640 1 -- 172.21.15.59:0/3713688654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c017070 con 0x7f27240b3930 2024-04-23T14:31:23.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.251+0000 7f2721600640 1 -- 172.21.15.59:0/3713688654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c00aca0 con 0x7f27240b3930 2024-04-23T14:31:23.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.251+0000 7f2721600640 1 -- 172.21.15.59:0/3713688654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c012620 con 0x7f27240b3930 2024-04-23T14:31:23.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.251+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27240c9e40 con 0x7f27240b3930 2024-04-23T14:31:23.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.251+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27240ca400 con 0x7f27240b3930 2024-04-23T14:31:23.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.252+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2724068a80 con 0x7f27240b3930 2024-04-23T14:31:23.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.253+0000 7f2721600640 1 -- 172.21.15.59:0/3713688654 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f271c01a070 con 0x7f27240b3930 2024-04-23T14:31:23.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.254+0000 7f2721600640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f270406c8f0 0x7f270406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.254+0000 7f2721600640 1 -- 172.21.15.59:0/3713688654 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f271c08b620 con 0x7f27240b3930 2024-04-23T14:31:23.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.254+0000 7f2729600640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f270406c8f0 0x7f270406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.255+0000 7f2729600640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f270406c8f0 0x7f270406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2718009970 tx=0x7f2718008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:23.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.258+0000 7f2721600640 1 -- 172.21.15.59:0/3713688654 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f271c059e30 con 0x7f27240b3930 2024-04-23T14:31:23.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.463+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f272409ab80 con 0x7f27240b3930 2024-04-23T14:31:23.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.757+0000 7f2721600640 1 -- 172.21.15.59:0/3713688654 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f271c059730 con 0x7f27240b3930 2024-04-23T14:31:23.759 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:31:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.761+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f270406c8f0 msgr2=0x7f270406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.761+0000 7f272c000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f270406c8f0 0x7f270406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2718009970 tx=0x7f2718008040 comp rx=0 tx=0).stop 2024-04-23T14:31:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.761+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27240b3930 msgr2=0x7f27240b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.761+0000 7f272c000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27240b3930 0x7f27240b1eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f271c00cb40 tx=0x7f271c00a060 comp rx=0 tx=0).stop 2024-04-23T14:31:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.762+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 shutdown_connections 2024-04-23T14:31:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.762+0000 7f272c000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f270406c8f0 0x7f270406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.762+0000 7f272c000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27240b3930 0x7f27240b1eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.762+0000 7f272c000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f272409b820 0x7f27240cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.762+0000 7f272c000640 1 --2- 172.21.15.59:0/3713688654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f272409af40 0x7f27240c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.762+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 >> 172.21.15.59:0/3713688654 conn(0x7f27240b4080 msgr2=0x7f27240b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.762+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 shutdown_connections 2024-04-23T14:31:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.763+0000 7f272c000640 1 -- 172.21.15.59:0/3713688654 wait complete. 2024-04-23T14:31:23.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:31:23.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.989+0000 7f0b0fe00640 1 Processor -- start 2024-04-23T14:31:23.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.990+0000 7f0b0fe00640 1 -- start start 2024-04-23T14:31:23.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.991+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100d0ec0 0x7f0b100d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.991+0000 7f0b0fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b100cff10 0x7f0b100d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:23.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.991+0000 7f0b0fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b100d17e0 con 0x7f0b100d0330 2024-04-23T14:31:23.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.991+0000 7f0b0fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100b3b60 con 0x7f0b100cff10 2024-04-23T14:31:23.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.991+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1015fd00 con 0x7f0b100d0ec0 2024-04-23T14:31:23.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.992+0000 7f0b0f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0b100d0330 0x7f0b1015eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47922/0 (socket says 172.21.15.59:47922) 2024-04-23T14:31:23.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.992+0000 7f0b0f400640 1 -- 172.21.15.59:0/1576466863 learned_addr learned my addr 172.21.15.59:0/1576466863 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:23.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.992+0000 7f0b0ea00640 1 --2- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100d0ec0 0x7f0b100d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.992+0000 7f0b07e00640 1 --2- 172.21.15.59:0/1576466863 >> v2:172.21.15.76:3300/0 conn(0x7f0b100cff10 0x7f0b100d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.992+0000 7f0b0e000640 1 -- 172.21.15.59:0/1576466863 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2319090806 0 0) 0x7f0b100d17e0 con 0x7f0b100d0330 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.992+0000 7f0b0e000640 1 -- 172.21.15.59:0/1576466863 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af4003520 con 0x7f0b100d0330 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.993+0000 7f0b0e000640 1 -- 172.21.15.59:0/1576466863 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 351984448 0 0) 0x7f0af4003520 con 0x7f0b100d0330 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.993+0000 7f0b07e00640 1 -- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100d0ec0 msgr2=0x7f0b100d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.993+0000 7f0b07e00640 1 --2- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100d0ec0 0x7f0b100d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.993+0000 7f0b07e00640 1 -- 172.21.15.59:0/1576466863 >> v1:172.21.15.59:6789/0 conn(0x7f0b100d0330 legacy=0x7f0b1015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.993+0000 7f0b07e00640 1 -- 172.21.15.59:0/1576466863 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1015fa20 con 0x7f0b100cff10 2024-04-23T14:31:23.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.994+0000 7f0b07e00640 1 --2- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100cff10 0x7f0b100d21a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097a0 tx=0x7f0af802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f95e554edaefe7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:23.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.994+0000 7f0b0e000640 1 -- 172.21.15.59:0/1576466863 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8037070 con 0x7f0b100cff10 2024-04-23T14:31:23.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.994+0000 7f0b0e000640 1 -- 172.21.15.59:0/1576466863 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af802ac30 con 0x7f0b100cff10 2024-04-23T14:31:23.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.994+0000 7f0b0e000640 1 -- 172.21.15.59:0/1576466863 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080019f0 con 0x7f0b100cff10 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 -- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100cff10 msgr2=0x7f0b100d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100cff10 0x7f0b100d21a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0af80097a0 tx=0x7f0af802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 -- 172.21.15.59:0/1576466863 shutdown_connections 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100cff10 0x7f0b100d21a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/1576466863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100d0ec0 0x7f0b100d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 -- 172.21.15.59:0/1576466863 >> 172.21.15.59:0/1576466863 conn(0x7f0b1008bf90 msgr2=0x7f0b100d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 -- 172.21.15.59:0/1576466863 shutdown_connections 2024-04-23T14:31:23.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.995+0000 7f0b0fe00640 1 -- 172.21.15.59:0/1576466863 wait complete. 2024-04-23T14:31:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.998+0000 7f0b0fe00640 1 Processor -- start 2024-04-23T14:31:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.998+0000 7f0b0fe00640 1 -- start start 2024-04-23T14:31:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.999+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100cff10 0x7f0b1009f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.999+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100d0330 0x7f0b100ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.999+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b100d0ec0 0x7f0b100a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:23.999+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100cf9f0 con 0x7f0b100d0ec0 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100cf710 con 0x7f0b100d0330 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100d26e0 con 0x7f0b100cff10 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b100d0ec0 0x7f0b100a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b100d0ec0 0x7f0b100a0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59684/0 (socket says 172.21.15.59:59684) 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 -- 172.21.15.59:0/873605325 learned_addr learned my addr 172.21.15.59:0/873605325 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0ea00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100cff10 0x7f0b1009f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b07e00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100d0330 0x7f0b100ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 -- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100cff10 msgr2=0x7f0b1009f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100cff10 0x7f0b1009f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 -- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100d0330 msgr2=0x7f0b100ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100d0330 0x7f0b100ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.000+0000 7f0b0f400640 1 -- 172.21.15.59:0/873605325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1009f720 con 0x7f0b100d0ec0 2024-04-23T14:31:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.001+0000 7f0b0f400640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b100d0ec0 0x7f0b100a0f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800cc90 tx=0x7f0b0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.001+0000 7f0b06a00640 1 -- 172.21.15.59:0/873605325 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08017070 con 0x7f0b100d0ec0 2024-04-23T14:31:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.001+0000 7f0b06a00640 1 -- 172.21.15.59:0/873605325 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b08012400 con 0x7f0b100d0ec0 2024-04-23T14:31:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.001+0000 7f0b06a00640 1 -- 172.21.15.59:0/873605325 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080127b0 con 0x7f0b100d0ec0 2024-04-23T14:31:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.001+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1009fa00 con 0x7f0b100d0ec0 2024-04-23T14:31:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.001+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1009ffc0 con 0x7f0b100d0ec0 2024-04-23T14:31:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.002+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1004fd90 con 0x7f0b100d0ec0 2024-04-23T14:31:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.004+0000 7f0b06a00640 1 -- 172.21.15.59:0/873605325 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b0801a070 con 0x7f0b100d0ec0 2024-04-23T14:31:24.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.005+0000 7f0b06a00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.005+0000 7f0b06a00640 1 -- 172.21.15.59:0/873605325 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f0b0808bc50 con 0x7f0b100d0ec0 2024-04-23T14:31:24.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.005+0000 7f0b0ea00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.006+0000 7f0b0ea00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0aec06c8f0 0x7f0aec06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b100cad30 tx=0x7f0b00009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:24.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.010+0000 7f0b06a00640 1 -- 172.21.15.59:0/873605325 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b08059c80 con 0x7f0b100d0ec0 2024-04-23T14:31:24.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.215+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0b100d1f00 con 0x7f0b100d0ec0 2024-04-23T14:31:24.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.764+0000 7f0b06a00640 1 -- 172.21.15.59:0/873605325 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f0b0805df80 con 0x7f0b100d0ec0 2024-04-23T14:31:24.765 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:31:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0aec06c8f0 msgr2=0x7f0aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0aec06c8f0 0x7f0aec06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b100cad30 tx=0x7f0b00009190 comp rx=0 tx=0).stop 2024-04-23T14:31:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b100d0ec0 msgr2=0x7f0b100a0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:24.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b100d0ec0 0x7f0b100a0f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800cc90 tx=0x7f0b0800a430 comp rx=0 tx=0).stop 2024-04-23T14:31:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 shutdown_connections 2024-04-23T14:31:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b100d0ec0 0x7f0b100a0f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b100d0330 0x7f0b100ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 --2- 172.21.15.59:0/873605325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b100cff10 0x7f0b1009f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.768+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 >> 172.21.15.59:0/873605325 conn(0x7f0b1008bf90 msgr2=0x7f0b1015f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:24.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.769+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 shutdown_connections 2024-04-23T14:31:24.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.769+0000 7f0b0fe00640 1 -- 172.21.15.59:0/873605325 wait complete. 2024-04-23T14:31:24.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:31:24.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.990+0000 7fe80ee00640 1 Processor -- start 2024-04-23T14:31:24.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.990+0000 7fe80ee00640 1 -- start start 2024-04-23T14:31:24.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.992+0000 7fe80ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8081567e0 0x7fe808156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.992+0000 7fe80ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80815cab0 con 0x7fe8080d0d10 2024-04-23T14:31:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.992+0000 7fe80ee00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe808160e40 con 0x7fe80815d810 2024-04-23T14:31:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.992+0000 7fe80ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe80815cd30 con 0x7fe8081567e0 2024-04-23T14:31:24.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.992+0000 7fe80e400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe8080d0d10 0x7fe808160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47924/0 (socket says 172.21.15.59:47924) 2024-04-23T14:31:24.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.992+0000 7fe80e400640 1 -- 172.21.15.59:0/5363799 learned_addr learned my addr 172.21.15.59:0/5363799 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.992+0000 7fe80da00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8081567e0 0x7fe808156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1126400827 0 0) 0x7fe800001490 con 0x7fe8080d0d10 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7f0003520 con 0x7fe8080d0d10 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2677685670 0 0) 0x7fe808160e40 con 0x7fe80815d810 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe800001490 con 0x7fe80815d810 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 836464716 0 0) 0x7fe7f0003520 con 0x7fe8080d0d10 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8081567e0 msgr2=0x7fe808156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8081567e0 0x7fe808156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 >> v1:172.21.15.76:6789/0 conn(0x7fe80815d810 legacy=0x7fe8080d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80815d010 con 0x7fe8080d0d10 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fe800001bc0 con 0x7fe8080d0d10 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 >> v1:172.21.15.59:6789/0 conn(0x7fe8080d0d10 legacy=0x7fe808160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7f0003b70 0x7fe7f0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0004800 0x7fe7f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f00051c0 0x7fe7f000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe80815cd30 con 0x7fe7f0003b70 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00103f0 con 0x7fe7f00051c0 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.993+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f00105f0 con 0x7fe7f0004800 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80da00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7f0003b70 0x7fe7f0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80d000640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0004800 0x7fe7f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80e400640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f00051c0 0x7fe7f000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80da00640 1 -- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0004800 msgr2=0x7fe7f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80da00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0004800 0x7fe7f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80da00640 1 -- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f00051c0 msgr2=0x7fe7f000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80da00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f00051c0 0x7fe7f000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.994+0000 7fe80da00640 1 -- 172.21.15.59:0/5363799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0003520 con 0x7fe7f0003b70 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.996+0000 7fe80da00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7f0003b70 0x7fe7f0003f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8080741a0 tx=0x7fe7f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8767a5272d7b08b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.996+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc036070 con 0x7fe7f0003b70 2024-04-23T14:31:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.996+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7fc031400 con 0x7fe7f0003b70 2024-04-23T14:31:24.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.996+0000 7fe807e00640 1 -- 172.21.15.59:0/5363799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc009650 con 0x7fe7f0003b70 2024-04-23T14:31:24.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.997+0000 7fe80ee00640 1 -- 172.21.15.59:0/5363799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7f0003b70 msgr2=0x7fe7f0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:24.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.997+0000 7fe80ee00640 1 --2- 172.21.15.59:0/5363799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7f0003b70 0x7fe7f0003f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe8080741a0 tx=0x7fe7f0005830 comp rx=0 tx=0).stop 2024-04-23T14:31:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.997+0000 7fe80e400640 1 -- 172.21.15.59:0/5363799 reap_dead start 2024-04-23T14:31:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.997+0000 7fe80ee00640 1 -- 172.21.15.59:0/5363799 shutdown_connections 2024-04-23T14:31:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.997+0000 7fe80ee00640 1 -- 172.21.15.59:0/5363799 >> 172.21.15.59:0/5363799 conn(0x7fe80806bdf0 msgr2=0x7fe80806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.997+0000 7fe80ee00640 1 -- 172.21.15.59:0/5363799 shutdown_connections 2024-04-23T14:31:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.997+0000 7fe80ee00640 1 -- 172.21.15.59:0/5363799 wait complete. 2024-04-23T14:31:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.999+0000 7fe80ee00640 1 Processor -- start 2024-04-23T14:31:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:24.999+0000 7fe80ee00640 1 -- start start 2024-04-23T14:31:25.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0003b70 0x7fe80806f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:25.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8080999c0 0x7fe8080afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:25.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8080b0a90 0x7fe8080b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:25.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe808156d40 con 0x7fe8080b0a90 2024-04-23T14:31:25.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe808156bc0 con 0x7fe8080999c0 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe808156ec0 con 0x7fe7f0003b70 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8080b0a90 0x7fe8080b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8080b0a90 0x7fe8080b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59706/0 (socket says 172.21.15.59:59706) 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80e400640 1 -- 172.21.15.59:0/2729744848 learned_addr learned my addr 172.21.15.59:0/2729744848 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80da00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0003b70 0x7fe80806f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.001+0000 7fe80d000640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8080999c0 0x7fe8080afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80e400640 1 -- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0003b70 msgr2=0x7fe80806f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80e400640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0003b70 0x7fe80806f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80e400640 1 -- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8080999c0 msgr2=0x7fe8080afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80e400640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8080999c0 0x7fe8080afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80e400640 1 -- 172.21.15.59:0/2729744848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80814cba0 con 0x7fe8080b0a90 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80da00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0003b70 0x7fe80806f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80d000640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8080999c0 0x7fe8080afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80e400640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8080b0a90 0x7fe8080b0f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8080b4cb0 tx=0x7fe80000a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:25.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe805600640 1 -- 172.21.15.59:0/2729744848 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80000adb0 con 0x7fe8080b0a90 2024-04-23T14:31:25.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe805600640 1 -- 172.21.15.59:0/2729744848 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe800019070 con 0x7fe8080b0a90 2024-04-23T14:31:25.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe805600640 1 -- 172.21.15.59:0/2729744848 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800013610 con 0x7fe8080b0a90 2024-04-23T14:31:25.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.002+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8081511e0 con 0x7fe8080b0a90 2024-04-23T14:31:25.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.003+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe808150f00 con 0x7fe8080b0a90 2024-04-23T14:31:25.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.004+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80804fd90 con 0x7fe8080b0a90 2024-04-23T14:31:25.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.005+0000 7fe805600640 1 -- 172.21.15.59:0/2729744848 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8000137b0 con 0x7fe8080b0a90 2024-04-23T14:31:25.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.006+0000 7fe805600640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe7e806c920 0x7fe7e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:25.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.006+0000 7fe805600640 1 -- 172.21.15.59:0/2729744848 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fe80008c380 con 0x7fe8080b0a90 2024-04-23T14:31:25.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.006+0000 7fe80da00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe7e806c920 0x7fe7e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:25.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.007+0000 7fe80da00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe7e806c920 0x7fe7e806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc033000 tx=0x7fe7fc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:25.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.013+0000 7fe805600640 1 -- 172.21.15.59:0/2729744848 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe80815cab0 con 0x7fe8080b0a90 2024-04-23T14:31:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.256+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe8080b5150 con 0x7fe8080b0a90 2024-04-23T14:31:25.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.811+0000 7fe805600640 1 -- 172.21.15.59:0/2729744848 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fe80005a7a0 con 0x7fe8080b0a90 2024-04-23T14:31:25.813 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T14:31:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.824+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe7e806c920 msgr2=0x7fe7e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.824+0000 7fe80ee00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe7e806c920 0x7fe7e806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc033000 tx=0x7fe7fc008030 comp rx=0 tx=0).stop 2024-04-23T14:31:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.824+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8080b0a90 msgr2=0x7fe8080b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.824+0000 7fe80ee00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8080b0a90 0x7fe8080b0f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8080b4cb0 tx=0x7fe80000a7b0 comp rx=0 tx=0).stop 2024-04-23T14:31:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.825+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 shutdown_connections 2024-04-23T14:31:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.825+0000 7fe80ee00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe7e806c920 0x7fe7e806ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.825+0000 7fe80ee00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8080b0a90 0x7fe8080b0f10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.825+0000 7fe80ee00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8080999c0 0x7fe8080afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.825+0000 7fe80ee00640 1 --2- 172.21.15.59:0/2729744848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7f0003b70 0x7fe80806f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.825+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 >> 172.21.15.59:0/2729744848 conn(0x7fe80806bdf0 msgr2=0x7fe80815a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.825+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 shutdown_connections 2024-04-23T14:31:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:25.826+0000 7fe80ee00640 1 -- 172.21.15.59:0/2729744848 wait complete. 2024-04-23T14:31:25.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:26.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.028+0000 7f5890400640 1 Processor -- start 2024-04-23T14:31:26.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.028+0000 7f5890400640 1 -- start start 2024-04-23T14:31:26.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f5890400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f588815c5e0 0x7f58880af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f5890400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58880af920 0x7f58880d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f5890400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588815f0f0 con 0x7f58880d06e0 2024-04-23T14:31:26.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f5890400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f58880afd80 con 0x7f588815c5e0 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f5890400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f588815fa20 con 0x7f58880af920 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f588815c5e0 0x7f58880af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f588815c5e0 0x7f58880af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59716/0 (socket says 172.21.15.59:59716) 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588da00640 1 -- 172.21.15.59:0/3310450210 learned_addr learned my addr 172.21.15.59:0/3310450210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588d000640 1 --2- 172.21.15.59:0/3310450210 >> v2:172.21.15.76:3300/0 conn(0x7f58880af920 0x7f58880d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588da00640 1 -- 172.21.15.59:0/3310450210 >> v2:172.21.15.76:3300/0 conn(0x7f58880af920 msgr2=0x7f58880d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588da00640 1 --2- 172.21.15.59:0/3310450210 >> v2:172.21.15.76:3300/0 conn(0x7f58880af920 0x7f58880d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588da00640 1 -- 172.21.15.59:0/3310450210 >> v1:172.21.15.76:6789/0 conn(0x7f58880d06e0 legacy=0x7f5888071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.030+0000 7f588da00640 1 -- 172.21.15.59:0/3310450210 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588815f3d0 con 0x7f588815c5e0 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.031+0000 7f588da00640 1 --2- 172.21.15.59:0/3310450210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588815c5e0 0x7f58880af3e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f58880b3a30 tx=0x7f587c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6987bcc876d8dff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5887e00640 1 -- 172.21.15.59:0/3310450210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c037070 con 0x7f588815c5e0 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5887e00640 1 -- 172.21.15.59:0/3310450210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f587c032440 con 0x7f588815c5e0 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5887e00640 1 -- 172.21.15.59:0/3310450210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c032810 con 0x7f588815c5e0 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5890400640 1 -- 172.21.15.59:0/3310450210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588815c5e0 msgr2=0x7f58880af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5890400640 1 --2- 172.21.15.59:0/3310450210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588815c5e0 0x7f58880af3e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f58880b3a30 tx=0x7f587c02a230 comp rx=0 tx=0).stop 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5890400640 1 -- 172.21.15.59:0/3310450210 shutdown_connections 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5890400640 1 --2- 172.21.15.59:0/3310450210 >> v2:172.21.15.76:3300/0 conn(0x7f58880af920 0x7f58880d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5890400640 1 --2- 172.21.15.59:0/3310450210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f588815c5e0 0x7f58880af3e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5890400640 1 -- 172.21.15.59:0/3310450210 >> 172.21.15.59:0/3310450210 conn(0x7f5888067ea0 msgr2=0x7f588806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.032+0000 7f5890400640 1 -- 172.21.15.59:0/3310450210 shutdown_connections 2024-04-23T14:31:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.033+0000 7f5890400640 1 -- 172.21.15.59:0/3310450210 wait complete. 2024-04-23T14:31:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.034+0000 7f5890400640 1 Processor -- start 2024-04-23T14:31:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.034+0000 7f5890400640 1 -- start start 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.035+0000 7f5890400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58880af920 0x7f5888156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.035+0000 7f5890400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58880d06e0 0x7f58881608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.035+0000 7f5890400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f588815c5e0 0x7f5888155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.035+0000 7f5890400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58880d0cf0 con 0x7f58880d06e0 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.035+0000 7f5890400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58880d0b70 con 0x7f588815c5e0 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.035+0000 7f5890400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f588806e480 con 0x7f58880af920 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.035+0000 7f588d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58880d06e0 0x7f58881608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58880d06e0 0x7f58881608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59722/0 (socket says 172.21.15.59:59722) 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 -- 172.21.15.59:0/2106094973 learned_addr learned my addr 172.21.15.59:0/2106094973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588e400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f588815c5e0 0x7f5888155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 -- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58880af920 msgr2=0x7f5888156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58880af920 0x7f5888156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 -- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f588815c5e0 msgr2=0x7f5888155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f588815c5e0 0x7f5888155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 -- 172.21.15.59:0/2106094973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5888160e00 con 0x7f58880d06e0 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588e400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f588815c5e0 0x7f5888155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f588d000640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58880d06e0 0x7f58881608c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f587400d800 tx=0x7f587400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.036+0000 7f5885600640 1 -- 172.21.15.59:0/2106094973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587400b6c0 con 0x7f58880d06e0 2024-04-23T14:31:26.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.037+0000 7f5885600640 1 -- 172.21.15.59:0/2106094973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5874019070 con 0x7f58880d06e0 2024-04-23T14:31:26.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.037+0000 7f5885600640 1 -- 172.21.15.59:0/2106094973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587400ba80 con 0x7f58880d06e0 2024-04-23T14:31:26.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.037+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58881610b0 con 0x7f58880d06e0 2024-04-23T14:31:26.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.037+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5888161670 con 0x7f58880d06e0 2024-04-23T14:31:26.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.038+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5888068a80 con 0x7f58880d06e0 2024-04-23T14:31:26.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.039+0000 7f5885600640 1 -- 172.21.15.59:0/2106094973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f587400bc20 con 0x7f58880d06e0 2024-04-23T14:31:26.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.040+0000 7f5885600640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f586806c8f0 0x7f586806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.040+0000 7f5885600640 1 -- 172.21.15.59:0/2106094973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f587408c0e0 con 0x7f58880d06e0 2024-04-23T14:31:26.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.040+0000 7f588da00640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f586806c8f0 0x7f586806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.041+0000 7f588da00640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f586806c8f0 0x7f586806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f587c034000 tx=0x7f587c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.045+0000 7f5885600640 1 -- 172.21.15.59:0/2106094973 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f587401c360 con 0x7f58880d06e0 2024-04-23T14:31:26.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.242+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f588806e260 con 0x7f58880d06e0 2024-04-23T14:31:26.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.245+0000 7f5885600640 1 -- 172.21.15.59:0/2106094973 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16558 (secure 0 0 0) 0x7f587405a470 con 0x7f58880d06e0 2024-04-23T14:31:26.246 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:26.246 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:25.258503+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.247+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f586806c8f0 msgr2=0x7f586806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.247+0000 7f5890400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f586806c8f0 0x7f586806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f587c034000 tx=0x7f587c02c000 comp rx=0 tx=0).stop 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.247+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58880d06e0 msgr2=0x7f58881608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.247+0000 7f5890400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58880d06e0 0x7f58881608c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f587400d800 tx=0x7f587400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 shutdown_connections 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f586806c8f0 0x7f586806edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f588815c5e0 0x7f5888155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58880d06e0 0x7f58881608c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 --2- 172.21.15.59:0/2106094973 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58880af920 0x7f5888156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 >> 172.21.15.59:0/2106094973 conn(0x7f5888067ea0 msgr2=0x7f588806f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 shutdown_connections 2024-04-23T14:31:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.248+0000 7f5890400640 1 -- 172.21.15.59:0/2106094973 wait complete. 2024-04-23T14:31:26.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:26.472 INFO:tasks.ceph.mds.d.smithi076.stderr:2024-04-23T14:31:26.471+0000 7f3f7d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:31:26.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.471+0000 7f7c73200640 1 Processor -- start 2024-04-23T14:31:26.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.472+0000 7f7c73200640 1 -- start start 2024-04-23T14:31:26.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c73200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7c740cd050 0x7f7c7406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c73200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c7415ece0 0x7f7c740cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c73200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7c740b3080 0x7f7c740cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c73200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7c740ced20 con 0x7f7c740cd050 2024-04-23T14:31:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c73200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7c7415fcd0 con 0x7f7c740b3080 2024-04-23T14:31:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c73200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7c740cd970 con 0x7f7c7415ece0 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c71e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7c740cd050 0x7f7c7406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c71e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7c740cd050 0x7f7c7406f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59734/0 (socket says 172.21.15.59:59734) 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.473+0000 7f7c71e00640 1 -- 172.21.15.59:0/3340042957 learned_addr learned my addr 172.21.15.59:0/3340042957 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.474+0000 7f7c71400640 1 --2- 172.21.15.59:0/3340042957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c7415ece0 0x7f7c740cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.474+0000 7f7c72800640 1 --2- 172.21.15.59:0/3340042957 >> v2:172.21.15.76:3300/0 conn(0x7f7c740b3080 0x7f7c740cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.474+0000 7f7c71e00640 1 -- 172.21.15.59:0/3340042957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c7415ece0 msgr2=0x7f7c740cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.474+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3340042957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c7415ece0 0x7f7c740cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.474+0000 7f7c71e00640 1 -- 172.21.15.59:0/3340042957 >> v2:172.21.15.76:3300/0 conn(0x7f7c740b3080 msgr2=0x7f7c740cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.474+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3340042957 >> v2:172.21.15.76:3300/0 conn(0x7f7c740b3080 0x7f7c740cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.474+0000 7f7c71e00640 1 -- 172.21.15.59:0/3340042957 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7415f160 con 0x7f7c740cd050 2024-04-23T14:31:26.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.475+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3340042957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c7406f620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c7415c8b0 tx=0x7f7c6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edb006c3c8668a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.475+0000 7f7c70a00640 1 -- 172.21.15.59:0/3340042957 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64037070 con 0x7f7c740cd050 2024-04-23T14:31:26.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.475+0000 7f7c70a00640 1 -- 172.21.15.59:0/3340042957 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c64032440 con 0x7f7c740cd050 2024-04-23T14:31:26.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.475+0000 7f7c70a00640 1 -- 172.21.15.59:0/3340042957 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64032810 con 0x7f7c740cd050 2024-04-23T14:31:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 -- 172.21.15.59:0/3340042957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 msgr2=0x7f7c7406f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 --2- 172.21.15.59:0/3340042957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c7406f620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c7415c8b0 tx=0x7f7c6402a140 comp rx=0 tx=0).stop 2024-04-23T14:31:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 -- 172.21.15.59:0/3340042957 shutdown_connections 2024-04-23T14:31:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 --2- 172.21.15.59:0/3340042957 >> v2:172.21.15.76:3300/0 conn(0x7f7c740b3080 0x7f7c740cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 --2- 172.21.15.59:0/3340042957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c7415ece0 0x7f7c740cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 --2- 172.21.15.59:0/3340042957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c7406f620 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 -- 172.21.15.59:0/3340042957 >> 172.21.15.59:0/3340042957 conn(0x7f7c740b21a0 msgr2=0x7f7c740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 -- 172.21.15.59:0/3340042957 shutdown_connections 2024-04-23T14:31:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.476+0000 7f7c73200640 1 -- 172.21.15.59:0/3340042957 wait complete. 2024-04-23T14:31:26.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.478+0000 7f7c73200640 1 Processor -- start 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.478+0000 7f7c73200640 1 -- start start 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c73200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c740b3080 0x7f7c740b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c73200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c741f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c73200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c7415ece0 0x7f7c740c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c73200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7415f9f0 con 0x7f7c740cd050 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c73200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7415f710 con 0x7f7c7415ece0 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c73200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7c740cef50 con 0x7f7c740b3080 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c741f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c741f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59746/0 (socket says 172.21.15.59:59746) 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.479+0000 7f7c71400640 1 -- 172.21.15.59:0/1668003839 learned_addr learned my addr 172.21.15.59:0/1668003839 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71e00640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c740b3080 0x7f7c740b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c72800640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c7415ece0 0x7f7c740c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71400640 1 -- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c740b3080 msgr2=0x7f7c740b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71400640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c740b3080 0x7f7c740b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71400640 1 -- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c7415ece0 msgr2=0x7f7c740c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71400640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c7415ece0 0x7f7c740c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71400640 1 -- 172.21.15.59:0/1668003839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c74157a30 con 0x7f7c740cd050 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71e00640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c740b3080 0x7f7c740b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c72800640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c7415ece0 0x7f7c740c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c71400640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c741f4930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000d800 tx=0x7f7c6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c6a000640 1 -- 172.21.15.59:0/1668003839 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000b6c0 con 0x7f7c740cd050 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c6a000640 1 -- 172.21.15.59:0/1668003839 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c60019070 con 0x7f7c740cd050 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c6a000640 1 -- 172.21.15.59:0/1668003839 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000ba80 con 0x7f7c740cd050 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.480+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c740c9be0 con 0x7f7c740cd050 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.481+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c740b0830 con 0x7f7c740cd050 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.482+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7404fd90 con 0x7f7c740cd050 2024-04-23T14:31:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.483+0000 7f7c6a000640 1 -- 172.21.15.59:0/1668003839 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c6000bc20 con 0x7f7c740cd050 2024-04-23T14:31:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.484+0000 7f7c6a000640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c5406c8f0 0x7f7c5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.484+0000 7f7c6a000640 1 -- 172.21.15.59:0/1668003839 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f7c6008bfe0 con 0x7f7c740cd050 2024-04-23T14:31:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.484+0000 7f7c71e00640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c5406c8f0 0x7f7c5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.485+0000 7f7c71e00640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c5406c8f0 0x7f7c5406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c64034000 tx=0x7f7c6402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.490+0000 7f7c6a000640 1 -- 172.21.15.59:0/1668003839 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c6005a3b0 con 0x7f7c740cd050 2024-04-23T14:31:26.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.697+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c740b1b60 con 0x7f7c740cd050 2024-04-23T14:31:26.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.700+0000 7f7c6a000640 1 -- 172.21.15.59:0/1668003839 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16558 (secure 0 0 0) 0x7f7c60059cb0 con 0x7f7c740cd050 2024-04-23T14:31:26.702 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:26.702 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:25.258503+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c5406c8f0 msgr2=0x7f7c5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c5406c8f0 0x7f7c5406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c64034000 tx=0x7f7c6402e000 comp rx=0 tx=0).stop 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 msgr2=0x7f7c741f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c741f4930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000d800 tx=0x7f7c6000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 shutdown_connections 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c5406c8f0 0x7f7c5406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c7415ece0 0x7f7c740c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c740cd050 0x7f7c741f4930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 --2- 172.21.15.59:0/1668003839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c740b3080 0x7f7c740b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 >> 172.21.15.59:0/1668003839 conn(0x7f7c740b21a0 msgr2=0x7f7c7406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:26.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 shutdown_connections 2024-04-23T14:31:26.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.703+0000 7f7c73200640 1 -- 172.21.15.59:0/1668003839 wait complete. 2024-04-23T14:31:26.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:31:26.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.938+0000 7fc6f5e00640 1 Processor -- start 2024-04-23T14:31:26.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.938+0000 7fc6f5e00640 1 -- start start 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.939+0000 7fc6f5e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6f01567e0 0x7fc6f0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.939+0000 7fc6f5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc6f00d0d10 0x7fc6f0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.939+0000 7fc6f5e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f015cab0 con 0x7fc6f015d810 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.939+0000 7fc6f5e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f0160e40 con 0x7fc6f00d0d10 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.939+0000 7fc6f5e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f015cd30 con 0x7fc6f01567e0 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6efe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc6f015d810 0x7fc6f00d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47936/0 (socket says 172.21.15.59:47936) 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6efe00640 1 -- 172.21.15.59:0/1071140047 learned_addr learned my addr 172.21.15.59:0/1071140047 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6f5400640 1 --2- 172.21.15.59:0/1071140047 >> v2:172.21.15.59:3300/0 conn(0x7fc6f00d0d10 0x7fc6f0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6f4a00640 1 --2- 172.21.15.59:0/1071140047 >> v2:172.21.15.76:3300/0 conn(0x7fc6f01567e0 0x7fc6f0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6f5400640 1 -- 172.21.15.59:0/1071140047 >> v2:172.21.15.76:3300/0 conn(0x7fc6f01567e0 msgr2=0x7fc6f0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:26.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6f5400640 1 --2- 172.21.15.59:0/1071140047 >> v2:172.21.15.76:3300/0 conn(0x7fc6f01567e0 0x7fc6f0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6f5400640 1 -- 172.21.15.59:0/1071140047 >> v1:172.21.15.59:6789/0 conn(0x7fc6f015d810 legacy=0x7fc6f00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6ef400640 1 -- 172.21.15.59:0/1071140047 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938071812 0 0) 0x7fc6e0001110 con 0x7fc6f015d810 2024-04-23T14:31:26.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.940+0000 7fc6f5400640 1 -- 172.21.15.59:0/1071140047 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f015d010 con 0x7fc6f00d0d10 2024-04-23T14:31:26.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.942+0000 7fc6f5400640 1 --2- 172.21.15.59:0/1071140047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f00d0d10 0x7fc6f0160110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00b6360 tx=0x7fc6e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db9193a4f83bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.942+0000 7fc6ef400640 1 -- 172.21.15.59:0/1071140047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8007740 con 0x7fc6f00d0d10 2024-04-23T14:31:26.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.942+0000 7fc6ef400640 1 -- 172.21.15.59:0/1071140047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e80078e0 con 0x7fc6f00d0d10 2024-04-23T14:31:26.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.942+0000 7fc6ef400640 1 -- 172.21.15.59:0/1071140047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8007cb0 con 0x7fc6f00d0d10 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 -- 172.21.15.59:0/1071140047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f00d0d10 msgr2=0x7fc6f0160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/1071140047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f00d0d10 0x7fc6f0160110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00b6360 tx=0x7fc6e802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 -- 172.21.15.59:0/1071140047 shutdown_connections 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/1071140047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f00d0d10 0x7fc6f0160110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/1071140047 >> v2:172.21.15.76:3300/0 conn(0x7fc6f01567e0 0x7fc6f0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 -- 172.21.15.59:0/1071140047 >> 172.21.15.59:0/1071140047 conn(0x7fc6f006bdf0 msgr2=0x7fc6f006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 -- 172.21.15.59:0/1071140047 shutdown_connections 2024-04-23T14:31:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.943+0000 7fc6f5e00640 1 -- 172.21.15.59:0/1071140047 wait complete. 2024-04-23T14:31:26.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.945+0000 7fc6f5e00640 1 Processor -- start 2024-04-23T14:31:26.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.945+0000 7fc6f5e00640 1 -- start start 2024-04-23T14:31:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.946+0000 7fc6f5e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6f00d0d10 0x7fc6f0146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.946+0000 7fc6f5e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6f01567e0 0x7fc6f015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.946+0000 7fc6f5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f015d810 0x7fc6f015b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.946+0000 7fc6f5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01607d0 con 0x7fc6f015d810 2024-04-23T14:31:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.946+0000 7fc6f5e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f0160650 con 0x7fc6f01567e0 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.946+0000 7fc6f5e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f0160950 con 0x7fc6f00d0d10 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f015d810 0x7fc6f015b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f015d810 0x7fc6f015b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59768/0 (socket says 172.21.15.59:59768) 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 -- 172.21.15.59:0/3283481313 learned_addr learned my addr 172.21.15.59:0/3283481313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6efe00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6f01567e0 0x7fc6f015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f4a00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6f00d0d10 0x7fc6f0146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 -- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6f00d0d10 msgr2=0x7fc6f0146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6f00d0d10 0x7fc6f0146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 -- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6f01567e0 msgr2=0x7fc6f015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6f01567e0 0x7fc6f015b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:26.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 -- 172.21.15.59:0/3283481313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f009a520 con 0x7fc6f015d810 2024-04-23T14:31:26.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.947+0000 7fc6f5400640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f015d810 0x7fc6f015b770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6e802ac90 tx=0x7fc6e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.948+0000 7fc6ecc00640 1 -- 172.21.15.59:0/3283481313 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e803f070 con 0x7fc6f015d810 2024-04-23T14:31:26.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.948+0000 7fc6ecc00640 1 -- 172.21.15.59:0/3283481313 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e8031bc0 con 0x7fc6f015d810 2024-04-23T14:31:26.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.948+0000 7fc6ecc00640 1 -- 172.21.15.59:0/3283481313 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8044460 con 0x7fc6f015d810 2024-04-23T14:31:26.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.948+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f009db40 con 0x7fc6f015d810 2024-04-23T14:31:26.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.948+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f009e100 con 0x7fc6f015d810 2024-04-23T14:31:26.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.949+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f004fd90 con 0x7fc6f015d810 2024-04-23T14:31:26.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.950+0000 7fc6ecc00640 1 -- 172.21.15.59:0/3283481313 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6e8044600 con 0x7fc6f015d810 2024-04-23T14:31:26.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.951+0000 7fc6ecc00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc6d406c940 0x7fc6d406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:26.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.951+0000 7fc6ecc00640 1 -- 172.21.15.59:0/3283481313 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc6e80ad300 con 0x7fc6f015d810 2024-04-23T14:31:26.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.951+0000 7fc6f4a00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc6d406c940 0x7fc6d406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:26.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.952+0000 7fc6f4a00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc6d406c940 0x7fc6d406ee00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4006fd0 tx=0x7fc6e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:26.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:26.956+0000 7fc6ecc00640 1 -- 172.21.15.59:0/3283481313 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc6e807b7a0 con 0x7fc6f015d810 2024-04-23T14:31:27.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:27.206+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc6f015fa60 con 0x7fc6f015d810 2024-04-23T14:31:28.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.128+0000 7fc6ecc00640 1 -- 172.21.15.59:0/3283481313 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fc6e807b0a0 con 0x7fc6f015d810 2024-04-23T14:31:28.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.132+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc6d406c940 msgr2=0x7fc6d406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:28.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.132+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc6d406c940 0x7fc6d406ee00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4006fd0 tx=0x7fc6e4008040 comp rx=0 tx=0).stop 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.132+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f015d810 msgr2=0x7fc6f015b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.132+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f015d810 0x7fc6f015b770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6e802ac90 tx=0x7fc6e8004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 shutdown_connections 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc6d406c940 0x7fc6d406ee00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6f015d810 0x7fc6f015b770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6f01567e0 0x7fc6f015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 --2- 172.21.15.59:0/3283481313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6f00d0d10 0x7fc6f0146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 >> 172.21.15.59:0/3283481313 conn(0x7fc6f006bdf0 msgr2=0x7fc6f015a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:28.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 shutdown_connections 2024-04-23T14:31:28.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.133+0000 7fc6f5e00640 1 -- 172.21.15.59:0/3283481313 wait complete. 2024-04-23T14:31:28.160 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:31:28.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:31:28.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.380+0000 7fefce200640 1 Processor -- start 2024-04-23T14:31:28.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.380+0000 7fefce200640 1 -- start start 2024-04-23T14:31:28.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.382+0000 7fefce200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 0x7fefc806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.382+0000 7fefce200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc80cfc20 con 0x7fefc80d0cc0 2024-04-23T14:31:28.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.382+0000 7fefce200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc80cff00 con 0x7fefc8145e10 2024-04-23T14:31:28.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.382+0000 7fefce200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefc80d01e0 con 0x7fefc8072480 2024-04-23T14:31:28.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.382+0000 7fefbfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fefc80d0cc0 0x7fefc8071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47952/0 (socket says 172.21.15.59:47952) 2024-04-23T14:31:28.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.382+0000 7fefbfe00640 1 -- 172.21.15.59:0/3103361462 learned_addr learned my addr 172.21.15.59:0/3103361462 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:28.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.382+0000 7fefc7e00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 0x7fefc806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750174643 0 0) 0x7fefc80cfc20 con 0x7fefc80d0cc0 2024-04-23T14:31:28.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefac003520 con 0x7fefc80d0cc0 2024-04-23T14:31:28.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3026919911 0 0) 0x7fefac003520 con 0x7fefc80d0cc0 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 msgr2=0x7fefc806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 0x7fefc806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 >> v1:172.21.15.76:6789/0 conn(0x7fefc8145e10 legacy=0x7fefc80cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc80d04c0 con 0x7fefc80d0cc0 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fefb0001670 con 0x7fefc80d0cc0 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 >> v1:172.21.15.59:6789/0 conn(0x7fefc80d0cc0 legacy=0x7fefc8071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 0x7fefac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac003f60 0x7fefac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefac004d40 0x7fefac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefc80d01e0 con 0x7fefac004d40 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefac00baa0 con 0x7fefac003f60 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefac00bca0 con 0x7fefc8072480 2024-04-23T14:31:28.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.383+0000 7fefc7e00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefac004d40 0x7fefac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.384+0000 7fefbfe00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac003f60 0x7fefac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.384+0000 7fefc7400640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 0x7fefac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.384+0000 7fefc7e00640 1 -- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 msgr2=0x7fefac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.384+0000 7fefc7e00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc8072480 0x7fefac0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.384+0000 7fefc7e00640 1 -- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac003f60 msgr2=0x7fefac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.384+0000 7fefc7e00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac003f60 0x7fefac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.384+0000 7fefc7e00640 1 -- 172.21.15.59:0/3103361462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac003520 con 0x7fefac004d40 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.385+0000 7fefc7e00640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefac004d40 0x7fefac0051c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009fc0 tx=0x7fefac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f3ae9fdece1d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.385+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0009c10 con 0x7fefac004d40 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.385+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0036070 con 0x7fefac004d40 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.385+0000 7fefc6a00640 1 -- 172.21.15.59:0/3103361462 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0031880 con 0x7fefac004d40 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.386+0000 7fefce200640 1 -- 172.21.15.59:0/3103361462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefac004d40 msgr2=0x7fefac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.386+0000 7fefce200640 1 --2- 172.21.15.59:0/3103361462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefac004d40 0x7fefac0051c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009fc0 tx=0x7fefac005830 comp rx=0 tx=0).stop 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.386+0000 7fefc7e00640 1 -- 172.21.15.59:0/3103361462 reap_dead start 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.386+0000 7fefce200640 1 -- 172.21.15.59:0/3103361462 shutdown_connections 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.386+0000 7fefce200640 1 -- 172.21.15.59:0/3103361462 >> 172.21.15.59:0/3103361462 conn(0x7fefc8067d90 msgr2=0x7fefc806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.386+0000 7fefce200640 1 -- 172.21.15.59:0/3103361462 shutdown_connections 2024-04-23T14:31:28.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.386+0000 7fefce200640 1 -- 172.21.15.59:0/3103361462 wait complete. 2024-04-23T14:31:28.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.388+0000 7fefce200640 1 Processor -- start 2024-04-23T14:31:28.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.388+0000 7fefce200640 1 -- start start 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.389+0000 7fefce200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac004d40 0x7fefc81f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.389+0000 7fefce200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc814cac0 0x7fefc814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.389+0000 7fefce200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc815edf0 0x7fefc815f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.389+0000 7fefce200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81461f0 con 0x7fefc814cac0 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.389+0000 7fefce200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefc806f130 con 0x7fefac004d40 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.389+0000 7fefce200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefc81f45d0 con 0x7fefc815edf0 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.389+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc814cac0 0x7fefc814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc814cac0 0x7fefc814cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59798/0 (socket says 172.21.15.59:59798) 2024-04-23T14:31:28.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 -- 172.21.15.59:0/1241803131 learned_addr learned my addr 172.21.15.59:0/1241803131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefc7400640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac004d40 0x7fefc81f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefc7e00640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc815edf0 0x7fefc815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 -- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc815edf0 msgr2=0x7fefc815f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc815edf0 0x7fefc815f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 -- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac004d40 msgr2=0x7fefc81f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac004d40 0x7fefc81f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 -- 172.21.15.59:0/1241803131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc814d540 con 0x7fefc814cac0 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefc7400640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac004d40 0x7fefc81f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefc7e00640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc815edf0 0x7fefc815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbfe00640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc814cac0 0x7fefc814cf40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fefb000e7d0 tx=0x7fefb000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.390+0000 7fefbf400640 1 -- 172.21.15.59:0/1241803131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb000c870 con 0x7fefc814cac0 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.391+0000 7fefbf400640 1 -- 172.21.15.59:0/1241803131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb0018070 con 0x7fefc814cac0 2024-04-23T14:31:28.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.391+0000 7fefbf400640 1 -- 172.21.15.59:0/1241803131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb000cc50 con 0x7fefc814cac0 2024-04-23T14:31:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.391+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc814d820 con 0x7fefc814cac0 2024-04-23T14:31:28.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.391+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc814dde0 con 0x7fefc814cac0 2024-04-23T14:31:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.392+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc8068a80 con 0x7fefc814cac0 2024-04-23T14:31:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.393+0000 7fefbf400640 1 -- 172.21.15.59:0/1241803131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefb001d460 con 0x7fefc814cac0 2024-04-23T14:31:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.394+0000 7fefbf400640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefa406c8f0 0x7fefa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.394+0000 7fefbf400640 1 -- 172.21.15.59:0/1241803131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefb008ba80 con 0x7fefc814cac0 2024-04-23T14:31:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.394+0000 7fefc7400640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefa406c8f0 0x7fefa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:28.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.395+0000 7fefc7400640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefa406c8f0 0x7fefa406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefb80016b0 tx=0x7fefb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.398+0000 7fefbf400640 1 -- 172.21.15.59:0/1241803131 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefb008de20 con 0x7fefc814cac0 2024-04-23T14:31:28.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:28.646+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fefc80d1190 con 0x7fefc814cac0 2024-04-23T14:31:29.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.139+0000 7fefbf400640 1 -- 172.21.15.59:0/1241803131 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fefb0059e10 con 0x7fefc814cac0 2024-04-23T14:31:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.142+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefa406c8f0 msgr2=0x7fefa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.142+0000 7fefce200640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefa406c8f0 0x7fefa406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefb80016b0 tx=0x7fefb8009040 comp rx=0 tx=0).stop 2024-04-23T14:31:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.142+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc814cac0 msgr2=0x7fefc814cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.142+0000 7fefce200640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc814cac0 0x7fefc814cf40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fefb000e7d0 tx=0x7fefb000ec90 comp rx=0 tx=0).stop 2024-04-23T14:31:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 shutdown_connections 2024-04-23T14:31:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefa406c8f0 0x7fefa406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc815edf0 0x7fefc815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc814cac0 0x7fefc814cf40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 --2- 172.21.15.59:0/1241803131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefac004d40 0x7fefc81f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 >> 172.21.15.59:0/1241803131 conn(0x7fefc8067d90 msgr2=0x7fefc8144240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 shutdown_connections 2024-04-23T14:31:29.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.143+0000 7fefce200640 1 -- 172.21.15.59:0/1241803131 wait complete. 2024-04-23T14:31:29.168 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:31:29.168 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:31:29.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.381+0000 7f229cc00640 1 Processor -- start 2024-04-23T14:31:29.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.381+0000 7f229cc00640 1 -- start start 2024-04-23T14:31:29.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f229cc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2298072d60 0x7f22980d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f229cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22980cf3c0 con 0x7f229806f6c0 2024-04-23T14:31:29.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f229cc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22980cf6a0 con 0x7f229815cb60 2024-04-23T14:31:29.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f229cc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f229806fb20 con 0x7f2298072d60 2024-04-23T14:31:29.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f2295400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f229806f6c0 0x7f2298072650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47958/0 (socket says 172.21.15.59:47958) 2024-04-23T14:31:29.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f2295400640 1 -- 172.21.15.59:0/3114847180 learned_addr learned my addr 172.21.15.59:0/3114847180 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:29.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f2296800640 1 --2- 172.21.15.59:0/3114847180 >> v2:172.21.15.76:3300/0 conn(0x7f2298072d60 0x7f22980d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2353301779 0 0) 0x7f22980cf3c0 con 0x7f229806f6c0 2024-04-23T14:31:29.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.383+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f227c003520 con 0x7f229806f6c0 2024-04-23T14:31:29.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2667720286 0 0) 0x7f227c003520 con 0x7f229806f6c0 2024-04-23T14:31:29.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 >> v2:172.21.15.76:3300/0 conn(0x7f2298072d60 msgr2=0x7f22980d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 --2- 172.21.15.59:0/3114847180 >> v2:172.21.15.76:3300/0 conn(0x7f2298072d60 0x7f22980d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 >> v1:172.21.15.76:6789/0 conn(0x7f229815cb60 legacy=0x7f22980ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22980cf980 con 0x7f229806f6c0 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2280001670 con 0x7f229806f6c0 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 >> v1:172.21.15.59:6789/0 conn(0x7f229806f6c0 legacy=0x7f2298072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c003a00 0x7f227c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f227c004800 0x7f227c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f229806fb20 con 0x7f227c004800 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f227c00c0c0 con 0x7f227c0051c0 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f227c00c2c0 con 0x7f227c003a00 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.384+0000 7f2295400640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f227c004800 0x7f227c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2296800640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2295e00640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c003a00 0x7f227c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2295400640 1 -- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c003a00 msgr2=0x7f227c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2295400640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c003a00 0x7f227c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2295400640 1 -- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f227c0051c0 msgr2=0x7f227c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2295400640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2295400640 1 -- 172.21.15.59:0/3114847180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c003520 con 0x7f227c004800 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2296800640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.385+0000 7f2296800640 1 -- 172.21.15.59:0/3114847180 reap_dead start 2024-04-23T14:31:29.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.386+0000 7f2295400640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f227c004800 0x7f227c004c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f228000c0e0 tx=0x7f227c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0f2c808a2c0179 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:29.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.386+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280007c20 con 0x7f227c004800 2024-04-23T14:31:29.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.386+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280005af0 con 0x7f227c004800 2024-04-23T14:31:29.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.386+0000 7f2294a00640 1 -- 172.21.15.59:0/3114847180 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280007070 con 0x7f227c004800 2024-04-23T14:31:29.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.387+0000 7f229cc00640 1 -- 172.21.15.59:0/3114847180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f227c004800 msgr2=0x7f227c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.387+0000 7f229cc00640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f227c004800 0x7f227c004c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f228000c0e0 tx=0x7f227c005830 comp rx=0 tx=0).stop 2024-04-23T14:31:29.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.387+0000 7f229cc00640 1 -- 172.21.15.59:0/3114847180 shutdown_connections 2024-04-23T14:31:29.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.387+0000 7f229cc00640 1 --2- 172.21.15.59:0/3114847180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f227c004800 0x7f227c004c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.387+0000 7f229cc00640 1 -- 172.21.15.59:0/3114847180 >> 172.21.15.59:0/3114847180 conn(0x7f22980d0c70 msgr2=0x7f22980d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:29.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.387+0000 7f229cc00640 1 -- 172.21.15.59:0/3114847180 shutdown_connections 2024-04-23T14:31:29.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.387+0000 7f229cc00640 1 -- 172.21.15.59:0/3114847180 wait complete. 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.389+0000 7f229cc00640 1 Processor -- start 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.389+0000 7f229cc00640 1 -- start start 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f229cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c004800 0x7f229814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f229cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f229815e9a0 0x7f229814b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f229cc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22981f1560 0x7f22981f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f229cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22980d05e0 con 0x7f229815e9a0 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f229cc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22980d0460 con 0x7f22981f1560 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f229cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22980d0760 con 0x7f227c004800 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f2295400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f229815e9a0 0x7f229814b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f2295400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f229815e9a0 0x7f229814b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59822/0 (socket says 172.21.15.59:59822) 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f2295400640 1 -- 172.21.15.59:0/2024015156 learned_addr learned my addr 172.21.15.59:0/2024015156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f2295e00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c004800 0x7f229814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f2296800640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22981f1560 0x7f22981f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f2295400640 1 -- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c004800 msgr2=0x7f229814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.390+0000 7f2295400640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c004800 0x7f229814ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f2295400640 1 -- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22981f1560 msgr2=0x7f22981f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f2295400640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22981f1560 0x7f22981f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f2295400640 1 -- 172.21.15.59:0/2024015156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229815f160 con 0x7f229815e9a0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f2295400640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f229815e9a0 0x7f229814b220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f228002ad20 tx=0x7f22980cf590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f228e000640 1 -- 172.21.15.59:0/2024015156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280001670 con 0x7f229815e9a0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f228e000640 1 -- 172.21.15.59:0/2024015156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2280042070 con 0x7f229815e9a0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f228e000640 1 -- 172.21.15.59:0/2024015156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800072b0 con 0x7f229815e9a0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229815f440 con 0x7f229815e9a0 2024-04-23T14:31:29.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.391+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2298149130 con 0x7f229815e9a0 2024-04-23T14:31:29.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.392+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2298148f20 con 0x7f229815e9a0 2024-04-23T14:31:29.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.393+0000 7f228e000640 1 -- 172.21.15.59:0/2024015156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2280007450 con 0x7f229815e9a0 2024-04-23T14:31:29.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.394+0000 7f228e000640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f227406c920 0x7f227406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.394+0000 7f228e000640 1 -- 172.21.15.59:0/2024015156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22800abbe0 con 0x7f229815e9a0 2024-04-23T14:31:29.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.395+0000 7f2295e00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f227406c920 0x7f227406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.395+0000 7f2295e00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f227406c920 0x7f227406ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2288000f80 tx=0x7f2288009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:29.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.399+0000 7f228e000640 1 -- 172.21.15.59:0/2024015156 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2280079f80 con 0x7f229815e9a0 2024-04-23T14:31:29.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.650+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2298069190 con 0x7f229815e9a0 2024-04-23T14:31:29.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.651+0000 7f228e000640 1 -- 172.21.15.59:0/2024015156 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f2280079880 con 0x7f229815e9a0 2024-04-23T14:31:29.653 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:29.653 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:31:29.653 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:31:29.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f227406c920 msgr2=0x7f227406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f227406c920 0x7f227406ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2288000f80 tx=0x7f2288009040 comp rx=0 tx=0).stop 2024-04-23T14:31:29.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f229815e9a0 msgr2=0x7f229814b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f229815e9a0 0x7f229814b220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f228002ad20 tx=0x7f22980cf590 comp rx=0 tx=0).stop 2024-04-23T14:31:29.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 shutdown_connections 2024-04-23T14:31:29.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f227406c920 0x7f227406ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22981f1560 0x7f22981f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f229815e9a0 0x7f229814b220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 --2- 172.21.15.59:0/2024015156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f227c004800 0x7f229814ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 >> 172.21.15.59:0/2024015156 conn(0x7f22980d0c70 msgr2=0x7f22980b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:29.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 shutdown_connections 2024-04-23T14:31:29.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.654+0000 7f229cc00640 1 -- 172.21.15.59:0/2024015156 wait complete. 2024-04-23T14:31:29.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:29.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.879+0000 7faf00a00640 1 Processor -- start 2024-04-23T14:31:29.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.880+0000 7faf00a00640 1 -- start start 2024-04-23T14:31:29.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.881+0000 7faf00a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faefc0cd760 0x7faefc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.881+0000 7faf00a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 0x7faefc0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.881+0000 7faf00a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faefc160210 con 0x7faefc15c3d0 2024-04-23T14:31:29.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.881+0000 7faf00a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faefc1604f0 con 0x7faefc0cd760 2024-04-23T14:31:29.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.881+0000 7faf00a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faefc1607d0 con 0x7faefc0b4690 2024-04-23T14:31:29.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.881+0000 7faefa800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faefc15c3d0 0x7faefc0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47964/0 (socket says 172.21.15.59:47964) 2024-04-23T14:31:29.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.881+0000 7faefa800640 1 -- 172.21.15.59:0/1519081816 learned_addr learned my addr 172.21.15.59:0/1519081816 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:29.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef3e00640 1 --2- 172.21.15.59:0/1519081816 >> v2:172.21.15.76:3300/0 conn(0x7faefc0cd760 0x7faefc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4072976900 0 0) 0x7faefc160210 con 0x7faefc15c3d0 2024-04-23T14:31:29.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faefb200640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 0x7faefc0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faee0003520 con 0x7faefc15c3d0 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 733307582 0 0) 0x7faee0003520 con 0x7faefc15c3d0 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 msgr2=0x7faefc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 0x7faefc0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 >> v2:172.21.15.76:3300/0 conn(0x7faefc0cd760 msgr2=0x7faefc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 --2- 172.21.15.59:0/1519081816 >> v2:172.21.15.76:3300/0 conn(0x7faefc0cd760 0x7faefc0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc160ab0 con 0x7faefc15c3d0 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7faeec001670 con 0x7faefc15c3d0 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 >> v1:172.21.15.59:6789/0 conn(0x7faefc15c3d0 legacy=0x7faefc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faee0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 0x7faee0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faee0004d40 0x7faee00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faefc1604f0 con 0x7faee00039b0 2024-04-23T14:31:29.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faefc1607d0 con 0x7faee0004d40 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.882+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faee000b9c0 con 0x7faefc0b4690 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefa800640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faee0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faef3e00640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 0x7faee0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefa800640 1 -- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 msgr2=0x7faee0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefa800640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc0b4690 0x7faee0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefa800640 1 -- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faee0004d40 msgr2=0x7faee00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefb200640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faee0004d40 0x7faee00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefa800640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faee0004d40 0x7faee00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefa800640 1 -- 172.21.15.59:0/1519081816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee0003520 con 0x7faee00039b0 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.883+0000 7faefb200640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faee0004d40 0x7faee00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.884+0000 7faefa800640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faee0003dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faeec007fd0 tx=0x7faee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58683959e37b034c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.884+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec007d50 con 0x7faee00039b0 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.884+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec005ad0 con 0x7faee00039b0 2024-04-23T14:31:29.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.884+0000 7faef9e00640 1 -- 172.21.15.59:0/1519081816 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec007070 con 0x7faee00039b0 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.885+0000 7faf00a00640 1 -- 172.21.15.59:0/1519081816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 msgr2=0x7faee0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.885+0000 7faf00a00640 1 --2- 172.21.15.59:0/1519081816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faee0003dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faeec007fd0 tx=0x7faee0005830 comp rx=0 tx=0).stop 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.885+0000 7faefb200640 1 -- 172.21.15.59:0/1519081816 reap_dead start 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.885+0000 7faf00a00640 1 -- 172.21.15.59:0/1519081816 shutdown_connections 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.885+0000 7faf00a00640 1 -- 172.21.15.59:0/1519081816 >> 172.21.15.59:0/1519081816 conn(0x7faefc0b21a0 msgr2=0x7faefc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.885+0000 7faf00a00640 1 -- 172.21.15.59:0/1519081816 shutdown_connections 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.885+0000 7faf00a00640 1 -- 172.21.15.59:0/1519081816 wait complete. 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.887+0000 7faf00a00640 1 Processor -- start 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.887+0000 7faf00a00640 1 -- start start 2024-04-23T14:31:29.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faf00a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faefc15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faf00a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faefc1f4160 0x7faefc1f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faf00a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc15cc20 0x7faefc15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faf00a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faefc15c7b0 con 0x7faee00039b0 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faf00a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faefc0b4b00 con 0x7faefc1f4160 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faf00a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faefc15c930 con 0x7faefc15cc20 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faefa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faefc15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faefa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faefc15f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59850/0 (socket says 172.21.15.59:59850) 2024-04-23T14:31:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.888+0000 7faefa800640 1 -- 172.21.15.59:0/1206195326 learned_addr learned my addr 172.21.15.59:0/1206195326 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:29.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faef3e00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faefc1f4160 0x7faefc1f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faefa800640 1 -- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc15cc20 msgr2=0x7faefc15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faefa800640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc15cc20 0x7faefc15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faefa800640 1 -- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faefc1f4160 msgr2=0x7faefc1f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faefa800640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faefc1f4160 0x7faefc1f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faefa800640 1 -- 172.21.15.59:0/1206195326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc15fb20 con 0x7faee00039b0 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faef3e00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faefc1f4160 0x7faefc1f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faefa800640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faefc15f5e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faeec02ae20 tx=0x7faefc160c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faef2a00640 1 -- 172.21.15.59:0/1206195326 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec042020 con 0x7faee00039b0 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faef2a00640 1 -- 172.21.15.59:0/1206195326 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec004bf0 con 0x7faee00039b0 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faef2a00640 1 -- 172.21.15.59:0/1206195326 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec001cb0 con 0x7faee00039b0 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.889+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc1f4cc0 con 0x7faee00039b0 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.890+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc071db0 con 0x7faee00039b0 2024-04-23T14:31:29.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.891+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc071ba0 con 0x7faee00039b0 2024-04-23T14:31:29.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.892+0000 7faef2a00640 1 -- 172.21.15.59:0/1206195326 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faeec045020 con 0x7faee00039b0 2024-04-23T14:31:29.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.893+0000 7faef2a00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faed806c920 0x7faed806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:29.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.893+0000 7faef2a00640 1 -- 172.21.15.59:0/1206195326 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faeec0abeb0 con 0x7faee00039b0 2024-04-23T14:31:29.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.893+0000 7faef3e00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faed806c920 0x7faed806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:29.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.894+0000 7faef3e00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faed806c920 0x7faed806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faefc09e380 tx=0x7faee4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:29.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:29.897+0000 7faef2a00640 1 -- 172.21.15.59:0/1206195326 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faeec04c070 con 0x7faee00039b0 2024-04-23T14:31:30.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.105+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faefc04fd90 con 0x7faee00039b0 2024-04-23T14:31:30.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.108+0000 7faef2a00640 1 -- 172.21.15.59:0/1206195326 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7faeec07a240 con 0x7faee00039b0 2024-04-23T14:31:30.109 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:30.110 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:30.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.110+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faed806c920 msgr2=0x7faed806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.110+0000 7faf00a00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faed806c920 0x7faed806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faefc09e380 tx=0x7faee4008040 comp rx=0 tx=0).stop 2024-04-23T14:31:30.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.110+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 msgr2=0x7faefc15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.110+0000 7faf00a00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faefc15f5e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faeec02ae20 tx=0x7faefc160c80 comp rx=0 tx=0).stop 2024-04-23T14:31:30.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 shutdown_connections 2024-04-23T14:31:30.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faed806c920 0x7faed806ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faefc15cc20 0x7faefc15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faefc1f4160 0x7faefc1f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 --2- 172.21.15.59:0/1206195326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faee00039b0 0x7faefc15f5e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 >> 172.21.15.59:0/1206195326 conn(0x7faefc0b21a0 msgr2=0x7faefc0b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:30.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 shutdown_connections 2024-04-23T14:31:30.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.111+0000 7faf00a00640 1 -- 172.21.15.59:0/1206195326 wait complete. 2024-04-23T14:31:30.131 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:31:30.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:31:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.338+0000 7f3acac00640 1 Processor -- start 2024-04-23T14:31:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.338+0000 7f3acac00640 1 -- start start 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.339+0000 7f3acac00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3ac406f620 0x7f3ac415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.339+0000 7f3acac00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3ac40b3080 0x7f3ac40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.339+0000 7f3acac00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ac415f430 con 0x7f3ac40cd050 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.339+0000 7f3acac00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3ac40ced20 con 0x7f3ac406f620 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.339+0000 7f3acac00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3ac415fcd0 con 0x7f3ac40b3080 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3ac8e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3ac406f620 0x7f3ac415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3ac8e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3ac406f620 0x7f3ac415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33082/0 (socket says 172.21.15.59:33082) 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3ac8e00640 1 -- 172.21.15.59:0/3832702155 learned_addr learned my addr 172.21.15.59:0/3832702155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3aca200640 1 --2- 172.21.15.59:0/3832702155 >> v2:172.21.15.76:3300/0 conn(0x7f3ac40b3080 0x7f3ac40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3ac8e00640 1 -- 172.21.15.59:0/3832702155 >> v2:172.21.15.76:3300/0 conn(0x7f3ac40b3080 msgr2=0x7f3ac40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:30.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3ac8e00640 1 --2- 172.21.15.59:0/3832702155 >> v2:172.21.15.76:3300/0 conn(0x7f3ac40b3080 0x7f3ac40cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3ac8e00640 1 -- 172.21.15.59:0/3832702155 >> v1:172.21.15.76:6789/0 conn(0x7f3ac40cd050 legacy=0x7f3ac40cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:30.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.340+0000 7f3ac8e00640 1 -- 172.21.15.59:0/3832702155 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac415f710 con 0x7f3ac406f620 2024-04-23T14:31:30.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.341+0000 7f3ac8e00640 1 --2- 172.21.15.59:0/3832702155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac406f620 0x7f3ac415ece0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cc960 tx=0x7f3ab002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772a2efe883f938b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:30.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.342+0000 7f3ac3e00640 1 -- 172.21.15.59:0/3832702155 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0037070 con 0x7f3ac406f620 2024-04-23T14:31:30.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.342+0000 7f3ac3e00640 1 -- 172.21.15.59:0/3832702155 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab0032440 con 0x7f3ac406f620 2024-04-23T14:31:30.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.342+0000 7f3ac3e00640 1 -- 172.21.15.59:0/3832702155 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0032810 con 0x7f3ac406f620 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 -- 172.21.15.59:0/3832702155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac406f620 msgr2=0x7f3ac415ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 --2- 172.21.15.59:0/3832702155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac406f620 0x7f3ac415ece0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cc960 tx=0x7f3ab002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 -- 172.21.15.59:0/3832702155 shutdown_connections 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 --2- 172.21.15.59:0/3832702155 >> v2:172.21.15.76:3300/0 conn(0x7f3ac40b3080 0x7f3ac40cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 --2- 172.21.15.59:0/3832702155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac406f620 0x7f3ac415ece0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 -- 172.21.15.59:0/3832702155 >> 172.21.15.59:0/3832702155 conn(0x7f3ac40b21a0 msgr2=0x7f3ac40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 -- 172.21.15.59:0/3832702155 shutdown_connections 2024-04-23T14:31:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.343+0000 7f3acac00640 1 -- 172.21.15.59:0/3832702155 wait complete. 2024-04-23T14:31:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.344+0000 7f3acac00640 1 Processor -- start 2024-04-23T14:31:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.345+0000 7f3acac00640 1 -- start start 2024-04-23T14:31:30.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3acac00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ac406f620 0x7f3ac4156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3acac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ac40b3080 0x7f3ac40c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3acac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac40cd050 0x7f3ac409d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3acac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac40cef50 con 0x7f3ac40cd050 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3acac00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac406faa0 con 0x7f3ac406f620 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3acac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac4159110 con 0x7f3ac40b3080 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3aca200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac40cd050 0x7f3ac409d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3aca200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac40cd050 0x7f3ac409d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33086/0 (socket says 172.21.15.59:33086) 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3aca200640 1 -- 172.21.15.59:0/1420626157 learned_addr learned my addr 172.21.15.59:0/1420626157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3ac9800640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ac406f620 0x7f3ac4156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.346+0000 7f3ac8e00640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ac40b3080 0x7f3ac40c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3aca200640 1 -- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ac40b3080 msgr2=0x7f3ac40c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3aca200640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ac40b3080 0x7f3ac40c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3aca200640 1 -- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ac406f620 msgr2=0x7f3ac4156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3aca200640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ac406f620 0x7f3ac4156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3aca200640 1 -- 172.21.15.59:0/1420626157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac40ca2f0 con 0x7f3ac40cd050 2024-04-23T14:31:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3aca200640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac40cd050 0x7f3ac409d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3abc007e40 tx=0x7f3abc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3ac1600640 1 -- 172.21.15.59:0/1420626157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc014620 con 0x7f3ac40cd050 2024-04-23T14:31:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3ac1600640 1 -- 172.21.15.59:0/1420626157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc019070 con 0x7f3ac40cd050 2024-04-23T14:31:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3ac1600640 1 -- 172.21.15.59:0/1420626157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc014a20 con 0x7f3ac40cd050 2024-04-23T14:31:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac409df20 con 0x7f3ac40cd050 2024-04-23T14:31:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.347+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac4157160 con 0x7f3ac40cd050 2024-04-23T14:31:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.348+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac404fd90 con 0x7f3ac40cd050 2024-04-23T14:31:30.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.349+0000 7f3ac1600640 1 -- 172.21.15.59:0/1420626157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3abc014bc0 con 0x7f3ac40cd050 2024-04-23T14:31:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.350+0000 7f3ac1600640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3aa406c8f0 0x7f3aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.350+0000 7f3ac1600640 1 -- 172.21.15.59:0/1420626157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3abc08c500 con 0x7f3ac40cd050 2024-04-23T14:31:30.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.351+0000 7f3ac9800640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3aa406c8f0 0x7f3aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.351+0000 7f3ac9800640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3aa406c8f0 0x7f3aa406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006fd0 tx=0x7f3ab8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:30.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.354+0000 7f3ac1600640 1 -- 172.21.15.59:0/1420626157 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ac40ca2f0 con 0x7f3ac40cd050 2024-04-23T14:31:30.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.556+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ac409e0f0 con 0x7f3aa406c8f0 2024-04-23T14:31:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.566+0000 7f3ac1600640 1 -- 172.21.15.59:0/1420626157 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f3ac409e0f0 con 0x7f3aa406c8f0 2024-04-23T14:31:30.571 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:30.572 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3aa406c8f0 msgr2=0x7f3aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3aa406c8f0 0x7f3aa406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006fd0 tx=0x7f3ab8008040 comp rx=0 tx=0).stop 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac40cd050 msgr2=0x7f3ac409d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac40cd050 0x7f3ac409d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3abc007e40 tx=0x7f3abc00e4b0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 shutdown_connections 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3aa406c8f0 0x7f3aa406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ac40b3080 0x7f3ac40c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ac40cd050 0x7f3ac409d9e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 --2- 172.21.15.59:0/1420626157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ac406f620 0x7f3ac4156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 >> 172.21.15.59:0/1420626157 conn(0x7f3ac40b21a0 msgr2=0x7f3ac40b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:30.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 shutdown_connections 2024-04-23T14:31:30.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.572+0000 7f3acac00640 1 -- 172.21.15.59:0/1420626157 wait complete. 2024-04-23T14:31:30.593 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-23T14:31:30.345260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001810"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227768+0000","last_change":"2024-04-23T14:31:25.849551+0000","last_active":"2024-04-23T14:31:27.227768+0000","last_peered":"2024-04-23T14:31:27.227768+0000","last_clean":"2024-04-23T14:31:27.227768+0000","last_became_active":"2024-04-23T14:31:25.848756+0000","last_became_peered":"2024-04-23T14:31:25.848756+0000","last_unstale":"2024-04-23T14:31:27.227768+0000","last_undegraded":"2024-04-23T14:31:27.227768+0000","last_fullsized":"2024-04-23T14:31:27.227768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833869+0000","last_change":"2024-04-23T14:31:25.819653+0000","last_active":"2024-04-23T14:31:26.833869+0000","last_peered":"2024-04-23T14:31:26.833869+0000","last_clean":"2024-04-23T14:31:26.833869+0000","last_became_active":"2024-04-23T14:31:25.819296+0000","last_became_peered":"2024-04-23T14:31:25.819296+0000","last_unstale":"2024-04-23T14:31:26.833869+0000","last_undegraded":"2024-04-23T14:31:26.833869+0000","last_fullsized":"2024-04-23T14:31:26.833869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225689+0000","last_change":"2024-04-23T14:31:25.818736+0000","last_active":"2024-04-23T14:31:27.225689+0000","last_peered":"2024-04-23T14:31:27.225689+0000","last_clean":"2024-04-23T14:31:27.225689+0000","last_became_active":"2024-04-23T14:31:25.816583+0000","last_became_peered":"2024-04-23T14:31:25.816583+0000","last_unstale":"2024-04-23T14:31:27.225689+0000","last_undegraded":"2024-04-23T14:31:27.225689+0000","last_fullsized":"2024-04-23T14:31:27.225689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229281+0000","last_change":"2024-04-23T14:31:25.836663+0000","last_active":"2024-04-23T14:31:27.229281+0000","last_peered":"2024-04-23T14:31:27.229281+0000","last_clean":"2024-04-23T14:31:27.229281+0000","last_became_active":"2024-04-23T14:31:25.836357+0000","last_became_peered":"2024-04-23T14:31:25.836357+0000","last_unstale":"2024-04-23T14:31:27.229281+0000","last_undegraded":"2024-04-23T14:31:27.229281+0000","last_fullsized":"2024-04-23T14:31:27.229281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227740+0000","last_change":"2024-04-23T14:31:25.830442+0000","last_active":"2024-04-23T14:31:27.227740+0000","last_peered":"2024-04-23T14:31:27.227740+0000","last_clean":"2024-04-23T14:31:27.227740+0000","last_became_active":"2024-04-23T14:31:25.830032+0000","last_became_peered":"2024-04-23T14:31:25.830032+0000","last_unstale":"2024-04-23T14:31:27.227740+0000","last_undegraded":"2024-04-23T14:31:27.227740+0000","last_fullsized":"2024-04-23T14:31:27.227740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228994+0000","last_change":"2024-04-23T14:31:25.835030+0000","last_active":"2024-04-23T14:31:27.228994+0000","last_peered":"2024-04-23T14:31:27.228994+0000","last_clean":"2024-04-23T14:31:27.228994+0000","last_became_active":"2024-04-23T14:31:25.834764+0000","last_became_peered":"2024-04-23T14:31:25.834764+0000","last_unstale":"2024-04-23T14:31:27.228994+0000","last_undegraded":"2024-04-23T14:31:27.228994+0000","last_fullsized":"2024-04-23T14:31:27.228994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226818+0000","last_change":"2024-04-23T14:31:25.833373+0000","last_active":"2024-04-23T14:31:27.226818+0000","last_peered":"2024-04-23T14:31:27.226818+0000","last_clean":"2024-04-23T14:31:27.226818+0000","last_became_active":"2024-04-23T14:31:25.831555+0000","last_became_peered":"2024-04-23T14:31:25.831555+0000","last_unstale":"2024-04-23T14:31:27.226818+0000","last_undegraded":"2024-04-23T14:31:27.226818+0000","last_fullsized":"2024-04-23T14:31:27.226818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224117+0000","last_change":"2024-04-23T14:31:25.828675+0000","last_active":"2024-04-23T14:31:27.224117+0000","last_peered":"2024-04-23T14:31:27.224117+0000","last_clean":"2024-04-23T14:31:27.224117+0000","last_became_active":"2024-04-23T14:31:25.828381+0000","last_became_peered":"2024-04-23T14:31:25.828381+0000","last_unstale":"2024-04-23T14:31:27.224117+0000","last_undegraded":"2024-04-23T14:31:27.224117+0000","last_fullsized":"2024-04-23T14:31:27.224117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228299+0000","last_change":"2024-04-23T14:31:25.839705+0000","last_active":"2024-04-23T14:31:27.228299+0000","last_peered":"2024-04-23T14:31:27.228299+0000","last_clean":"2024-04-23T14:31:27.228299+0000","last_became_active":"2024-04-23T14:31:25.839432+0000","last_became_peered":"2024-04-23T14:31:25.839432+0000","last_unstale":"2024-04-23T14:31:27.228299+0000","last_undegraded":"2024-04-23T14:31:27.228299+0000","last_fullsized":"2024-04-23T14:31:27.228299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821382+0000","last_change":"2024-04-23T14:31:25.833382+0000","last_active":"2024-04-23T14:31:26.821382+0000","last_peered":"2024-04-23T14:31:26.821382+0000","last_clean":"2024-04-23T14:31:26.821382+0000","last_became_active":"2024-04-23T14:31:25.833010+0000","last_became_peered":"2024-04-23T14:31:25.833010+0000","last_unstale":"2024-04-23T14:31:26.821382+0000","last_undegraded":"2024-04-23T14:31:26.821382+0000","last_fullsized":"2024-04-23T14:31:26.821382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820889+0000","last_change":"2024-04-23T14:31:25.828614+0000","last_active":"2024-04-23T14:31:26.820889+0000","last_peered":"2024-04-23T14:31:26.820889+0000","last_clean":"2024-04-23T14:31:26.820889+0000","last_became_active":"2024-04-23T14:31:25.828342+0000","last_became_peered":"2024-04-23T14:31:25.828342+0000","last_unstale":"2024-04-23T14:31:26.820889+0000","last_undegraded":"2024-04-23T14:31:26.820889+0000","last_fullsized":"2024-04-23T14:31:26.820889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821732+0000","last_change":"2024-04-23T14:31:25.825396+0000","last_active":"2024-04-23T14:31:26.821732+0000","last_peered":"2024-04-23T14:31:26.821732+0000","last_clean":"2024-04-23T14:31:26.821732+0000","last_became_active":"2024-04-23T14:31:25.822425+0000","last_became_peered":"2024-04-23T14:31:25.822425+0000","last_unstale":"2024-04-23T14:31:26.821732+0000","last_undegraded":"2024-04-23T14:31:26.821732+0000","last_fullsized":"2024-04-23T14:31:26.821732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:05.846065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223667+0000","last_change":"2024-04-23T14:31:25.823390+0000","last_active":"2024-04-23T14:31:27.223667+0000","last_peered":"2024-04-23T14:31:27.223667+0000","last_clean":"2024-04-23T14:31:27.223667+0000","last_became_active":"2024-04-23T14:31:25.823054+0000","last_became_peered":"2024-04-23T14:31:25.823054+0000","last_unstale":"2024-04-23T14:31:27.223667+0000","last_undegraded":"2024-04-23T14:31:27.223667+0000","last_fullsized":"2024-04-23T14:31:27.223667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225120+0000","last_change":"2024-04-23T14:31:25.818198+0000","last_active":"2024-04-23T14:31:27.225120+0000","last_peered":"2024-04-23T14:31:27.225120+0000","last_clean":"2024-04-23T14:31:27.225120+0000","last_became_active":"2024-04-23T14:31:25.817873+0000","last_became_peered":"2024-04-23T14:31:25.817873+0000","last_unstale":"2024-04-23T14:31:27.225120+0000","last_undegraded":"2024-04-23T14:31:27.225120+0000","last_fullsized":"2024-04-23T14:31:27.225120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:08:54.362020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833481+0000","last_change":"2024-04-23T14:31:25.829886+0000","last_active":"2024-04-23T14:31:26.833481+0000","last_peered":"2024-04-23T14:31:26.833481+0000","last_clean":"2024-04-23T14:31:26.833481+0000","last_became_active":"2024-04-23T14:31:25.829086+0000","last_became_peered":"2024-04-23T14:31:25.829086+0000","last_unstale":"2024-04-23T14:31:26.833481+0000","last_undegraded":"2024-04-23T14:31:26.833481+0000","last_fullsized":"2024-04-23T14:31:26.833481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228453+0000","last_change":"2024-04-23T14:31:25.818775+0000","last_active":"2024-04-23T14:31:27.228453+0000","last_peered":"2024-04-23T14:31:27.228453+0000","last_clean":"2024-04-23T14:31:27.228453+0000","last_became_active":"2024-04-23T14:31:25.818318+0000","last_became_peered":"2024-04-23T14:31:25.818318+0000","last_unstale":"2024-04-23T14:31:27.228453+0000","last_undegraded":"2024-04-23T14:31:27.228453+0000","last_fullsized":"2024-04-23T14:31:27.228453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230731+0000","last_change":"2024-04-23T14:31:25.850111+0000","last_active":"2024-04-23T14:31:27.230731+0000","last_peered":"2024-04-23T14:31:27.230731+0000","last_clean":"2024-04-23T14:31:27.230731+0000","last_became_active":"2024-04-23T14:31:25.848968+0000","last_became_peered":"2024-04-23T14:31:25.848968+0000","last_unstale":"2024-04-23T14:31:27.230731+0000","last_undegraded":"2024-04-23T14:31:27.230731+0000","last_fullsized":"2024-04-23T14:31:27.230731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.815057+0000","last_change":"2024-04-23T14:31:25.813711+0000","last_active":"2024-04-23T14:31:26.815057+0000","last_peered":"2024-04-23T14:31:26.815057+0000","last_clean":"2024-04-23T14:31:26.815057+0000","last_became_active":"2024-04-23T14:31:25.813377+0000","last_became_peered":"2024-04-23T14:31:25.813377+0000","last_unstale":"2024-04-23T14:31:26.815057+0000","last_undegraded":"2024-04-23T14:31:26.815057+0000","last_fullsized":"2024-04-23T14:31:26.815057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230045+0000","last_change":"2024-04-23T14:31:25.842518+0000","last_active":"2024-04-23T14:31:27.230045+0000","last_peered":"2024-04-23T14:31:27.230045+0000","last_clean":"2024-04-23T14:31:27.230045+0000","last_became_active":"2024-04-23T14:31:25.842265+0000","last_became_peered":"2024-04-23T14:31:25.842265+0000","last_unstale":"2024-04-23T14:31:27.230045+0000","last_undegraded":"2024-04-23T14:31:27.230045+0000","last_fullsized":"2024-04-23T14:31:27.230045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:10:49.364737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820245+0000","last_change":"2024-04-23T14:31:25.824601+0000","last_active":"2024-04-23T14:31:26.820245+0000","last_peered":"2024-04-23T14:31:26.820245+0000","last_clean":"2024-04-23T14:31:26.820245+0000","last_became_active":"2024-04-23T14:31:25.824326+0000","last_became_peered":"2024-04-23T14:31:25.824326+0000","last_unstale":"2024-04-23T14:31:26.820245+0000","last_undegraded":"2024-04-23T14:31:26.820245+0000","last_fullsized":"2024-04-23T14:31:26.820245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225202+0000","last_change":"2024-04-23T14:31:25.824382+0000","last_active":"2024-04-23T14:31:27.225202+0000","last_peered":"2024-04-23T14:31:27.225202+0000","last_clean":"2024-04-23T14:31:27.225202+0000","last_became_active":"2024-04-23T14:31:25.823991+0000","last_became_peered":"2024-04-23T14:31:25.823991+0000","last_unstale":"2024-04-23T14:31:27.225202+0000","last_undegraded":"2024-04-23T14:31:27.225202+0000","last_fullsized":"2024-04-23T14:31:27.225202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229105+0000","last_change":"2024-04-23T14:31:25.842389+0000","last_active":"2024-04-23T14:31:27.229105+0000","last_peered":"2024-04-23T14:31:27.229105+0000","last_clean":"2024-04-23T14:31:27.229105+0000","last_became_active":"2024-04-23T14:31:25.841584+0000","last_became_peered":"2024-04-23T14:31:25.841584+0000","last_unstale":"2024-04-23T14:31:27.229105+0000","last_undegraded":"2024-04-23T14:31:27.229105+0000","last_fullsized":"2024-04-23T14:31:27.229105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223827+0000","last_change":"2024-04-23T14:31:25.830515+0000","last_active":"2024-04-23T14:31:27.223827+0000","last_peered":"2024-04-23T14:31:27.223827+0000","last_clean":"2024-04-23T14:31:27.223827+0000","last_became_active":"2024-04-23T14:31:25.830209+0000","last_became_peered":"2024-04-23T14:31:25.830209+0000","last_unstale":"2024-04-23T14:31:27.223827+0000","last_undegraded":"2024-04-23T14:31:27.223827+0000","last_fullsized":"2024-04-23T14:31:27.223827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228540+0000","last_change":"2024-04-23T14:31:25.824411+0000","last_active":"2024-04-23T14:31:27.228540+0000","last_peered":"2024-04-23T14:31:27.228540+0000","last_clean":"2024-04-23T14:31:27.228540+0000","last_became_active":"2024-04-23T14:31:25.824052+0000","last_became_peered":"2024-04-23T14:31:25.824052+0000","last_unstale":"2024-04-23T14:31:27.228540+0000","last_undegraded":"2024-04-23T14:31:27.228540+0000","last_fullsized":"2024-04-23T14:31:27.228540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224322+0000","last_change":"2024-04-23T14:31:25.830939+0000","last_active":"2024-04-23T14:31:27.224322+0000","last_peered":"2024-04-23T14:31:27.224322+0000","last_clean":"2024-04-23T14:31:27.224322+0000","last_became_active":"2024-04-23T14:31:25.830632+0000","last_became_peered":"2024-04-23T14:31:25.830632+0000","last_unstale":"2024-04-23T14:31:27.224322+0000","last_undegraded":"2024-04-23T14:31:27.224322+0000","last_fullsized":"2024-04-23T14:31:27.224322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821455+0000","last_change":"2024-04-23T14:31:25.832634+0000","last_active":"2024-04-23T14:31:26.821455+0000","last_peered":"2024-04-23T14:31:26.821455+0000","last_clean":"2024-04-23T14:31:26.821455+0000","last_became_active":"2024-04-23T14:31:25.832329+0000","last_became_peered":"2024-04-23T14:31:25.832329+0000","last_unstale":"2024-04-23T14:31:26.821455+0000","last_undegraded":"2024-04-23T14:31:26.821455+0000","last_fullsized":"2024-04-23T14:31:26.821455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833312+0000","last_change":"2024-04-23T14:31:25.822589+0000","last_active":"2024-04-23T14:31:26.833312+0000","last_peered":"2024-04-23T14:31:26.833312+0000","last_clean":"2024-04-23T14:31:26.833312+0000","last_became_active":"2024-04-23T14:31:25.822061+0000","last_became_peered":"2024-04-23T14:31:25.822061+0000","last_unstale":"2024-04-23T14:31:26.833312+0000","last_undegraded":"2024-04-23T14:31:26.833312+0000","last_fullsized":"2024-04-23T14:31:26.833312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223980+0000","last_change":"2024-04-23T14:31:25.832096+0000","last_active":"2024-04-23T14:31:27.223980+0000","last_peered":"2024-04-23T14:31:27.223980+0000","last_clean":"2024-04-23T14:31:27.223980+0000","last_became_active":"2024-04-23T14:31:25.829333+0000","last_became_peered":"2024-04-23T14:31:25.829333+0000","last_unstale":"2024-04-23T14:31:27.223980+0000","last_undegraded":"2024-04-23T14:31:27.223980+0000","last_fullsized":"2024-04-23T14:31:27.223980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225363+0000","last_change":"2024-04-23T14:31:25.826410+0000","last_active":"2024-04-23T14:31:27.225363+0000","last_peered":"2024-04-23T14:31:27.225363+0000","last_clean":"2024-04-23T14:31:27.225363+0000","last_became_active":"2024-04-23T14:31:25.826088+0000","last_became_peered":"2024-04-23T14:31:25.826088+0000","last_unstale":"2024-04-23T14:31:27.225363+0000","last_undegraded":"2024-04-23T14:31:27.225363+0000","last_fullsized":"2024-04-23T14:31:27.225363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229439+0000","last_change":"2024-04-23T14:31:25.832202+0000","last_active":"2024-04-23T14:31:27.229439+0000","last_peered":"2024-04-23T14:31:27.229439+0000","last_clean":"2024-04-23T14:31:27.229439+0000","last_became_active":"2024-04-23T14:31:25.831719+0000","last_became_peered":"2024-04-23T14:31:25.831719+0000","last_unstale":"2024-04-23T14:31:27.229439+0000","last_undegraded":"2024-04-23T14:31:27.229439+0000","last_fullsized":"2024-04-23T14:31:27.229439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814413+0000","last_change":"2024-04-23T14:31:25.827667+0000","last_active":"2024-04-23T14:31:26.814413+0000","last_peered":"2024-04-23T14:31:26.814413+0000","last_clean":"2024-04-23T14:31:26.814413+0000","last_became_active":"2024-04-23T14:31:25.827302+0000","last_became_peered":"2024-04-23T14:31:25.827302+0000","last_unstale":"2024-04-23T14:31:26.814413+0000","last_undegraded":"2024-04-23T14:31:26.814413+0000","last_fullsized":"2024-04-23T14:31:26.814413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.819912+0000","last_change":"2024-04-23T14:31:25.817837+0000","last_active":"2024-04-23T14:31:26.819912+0000","last_peered":"2024-04-23T14:31:26.819912+0000","last_clean":"2024-04-23T14:31:26.819912+0000","last_became_active":"2024-04-23T14:31:25.817156+0000","last_became_peered":"2024-04-23T14:31:25.817156+0000","last_unstale":"2024-04-23T14:31:26.819912+0000","last_undegraded":"2024-04-23T14:31:26.819912+0000","last_fullsized":"2024-04-23T14:31:26.819912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820171+0000","last_change":"2024-04-23T14:31:25.821505+0000","last_active":"2024-04-23T14:31:26.820171+0000","last_peered":"2024-04-23T14:31:26.820171+0000","last_clean":"2024-04-23T14:31:26.820171+0000","last_became_active":"2024-04-23T14:31:25.821186+0000","last_became_peered":"2024-04-23T14:31:25.821186+0000","last_unstale":"2024-04-23T14:31:26.820171+0000","last_undegraded":"2024-04-23T14:31:26.820171+0000","last_fullsized":"2024-04-23T14:31:26.820171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:30.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224053+0000","last_change":"2024-04-23T14:31:24.810696+0000","last_active":"2024-04-23T14:31:27.224053+0000","last_peered":"2024-04-23T14:31:27.224053+0000","last_clean":"2024-04-23T14:31:27.224053+0000","last_became_active":"2024-04-23T14:31:24.810349+0000","last_became_peered":"2024-04-23T14:31:24.810349+0000","last_unstale":"2024-04-23T14:31:27.224053+0000","last_undegraded":"2024-04-23T14:31:27.224053+0000","last_fullsized":"2024-04-23T14:31:27.224053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225550+0000","last_change":"2024-04-23T14:31:25.824408+0000","last_active":"2024-04-23T14:31:27.225550+0000","last_peered":"2024-04-23T14:31:27.225550+0000","last_clean":"2024-04-23T14:31:27.225550+0000","last_became_active":"2024-04-23T14:31:25.822620+0000","last_became_peered":"2024-04-23T14:31:25.822620+0000","last_unstale":"2024-04-23T14:31:27.225550+0000","last_undegraded":"2024-04-23T14:31:27.225550+0000","last_fullsized":"2024-04-23T14:31:27.225550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820803+0000","last_change":"2024-04-23T14:31:24.801642+0000","last_active":"2024-04-23T14:31:26.820803+0000","last_peered":"2024-04-23T14:31:26.820803+0000","last_clean":"2024-04-23T14:31:26.820803+0000","last_became_active":"2024-04-23T14:31:24.801244+0000","last_became_peered":"2024-04-23T14:31:24.801244+0000","last_unstale":"2024-04-23T14:31:26.820803+0000","last_undegraded":"2024-04-23T14:31:26.820803+0000","last_fullsized":"2024-04-23T14:31:26.820803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223904+0000","last_change":"2024-04-23T14:31:24.796017+0000","last_active":"2024-04-23T14:31:27.223904+0000","last_peered":"2024-04-23T14:31:27.223904+0000","last_clean":"2024-04-23T14:31:27.223904+0000","last_became_active":"2024-04-23T14:31:24.795734+0000","last_became_peered":"2024-04-23T14:31:24.795734+0000","last_unstale":"2024-04-23T14:31:27.223904+0000","last_undegraded":"2024-04-23T14:31:27.223904+0000","last_fullsized":"2024-04-23T14:31:27.223904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223975+0000","last_change":"2024-04-23T14:31:25.819835+0000","last_active":"2024-04-23T14:31:27.223975+0000","last_peered":"2024-04-23T14:31:27.223975+0000","last_clean":"2024-04-23T14:31:27.223975+0000","last_became_active":"2024-04-23T14:31:25.819532+0000","last_became_peered":"2024-04-23T14:31:25.819532+0000","last_unstale":"2024-04-23T14:31:27.223975+0000","last_undegraded":"2024-04-23T14:31:27.223975+0000","last_fullsized":"2024-04-23T14:31:27.223975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228610+0000","last_change":"2024-04-23T14:31:24.802566+0000","last_active":"2024-04-23T14:31:27.228610+0000","last_peered":"2024-04-23T14:31:27.228610+0000","last_clean":"2024-04-23T14:31:27.228610+0000","last_became_active":"2024-04-23T14:31:24.802299+0000","last_became_peered":"2024-04-23T14:31:24.802299+0000","last_unstale":"2024-04-23T14:31:27.228610+0000","last_undegraded":"2024-04-23T14:31:27.228610+0000","last_fullsized":"2024-04-23T14:31:27.228610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833252+0000","last_change":"2024-04-23T14:31:25.822370+0000","last_active":"2024-04-23T14:31:26.833252+0000","last_peered":"2024-04-23T14:31:26.833252+0000","last_clean":"2024-04-23T14:31:26.833252+0000","last_became_active":"2024-04-23T14:31:25.822039+0000","last_became_peered":"2024-04-23T14:31:25.822039+0000","last_unstale":"2024-04-23T14:31:26.833252+0000","last_undegraded":"2024-04-23T14:31:26.833252+0000","last_fullsized":"2024-04-23T14:31:26.833252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821576+0000","last_change":"2024-04-23T14:31:24.806864+0000","last_active":"2024-04-23T14:31:26.821576+0000","last_peered":"2024-04-23T14:31:26.821576+0000","last_clean":"2024-04-23T14:31:26.821576+0000","last_became_active":"2024-04-23T14:31:24.806597+0000","last_became_peered":"2024-04-23T14:31:24.806597+0000","last_unstale":"2024-04-23T14:31:26.821576+0000","last_undegraded":"2024-04-23T14:31:26.821576+0000","last_fullsized":"2024-04-23T14:31:26.821576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230291+0000","last_change":"2024-04-23T14:31:25.842918+0000","last_active":"2024-04-23T14:31:27.230291+0000","last_peered":"2024-04-23T14:31:27.230291+0000","last_clean":"2024-04-23T14:31:27.230291+0000","last_became_active":"2024-04-23T14:31:25.842581+0000","last_became_peered":"2024-04-23T14:31:25.842581+0000","last_unstale":"2024-04-23T14:31:27.230291+0000","last_undegraded":"2024-04-23T14:31:27.230291+0000","last_fullsized":"2024-04-23T14:31:27.230291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229964+0000","last_change":"2024-04-23T14:31:24.805711+0000","last_active":"2024-04-23T14:31:27.229964+0000","last_peered":"2024-04-23T14:31:27.229964+0000","last_clean":"2024-04-23T14:31:27.229964+0000","last_became_active":"2024-04-23T14:31:24.805351+0000","last_became_peered":"2024-04-23T14:31:24.805351+0000","last_unstale":"2024-04-23T14:31:27.229964+0000","last_undegraded":"2024-04-23T14:31:27.229964+0000","last_fullsized":"2024-04-23T14:31:27.229964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225997+0000","last_change":"2024-04-23T14:31:25.837626+0000","last_active":"2024-04-23T14:31:27.225997+0000","last_peered":"2024-04-23T14:31:27.225997+0000","last_clean":"2024-04-23T14:31:27.225997+0000","last_became_active":"2024-04-23T14:31:25.836840+0000","last_became_peered":"2024-04-23T14:31:25.836840+0000","last_unstale":"2024-04-23T14:31:27.225997+0000","last_undegraded":"2024-04-23T14:31:27.225997+0000","last_fullsized":"2024-04-23T14:31:27.225997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833398+0000","last_change":"2024-04-23T14:31:24.802741+0000","last_active":"2024-04-23T14:31:26.833398+0000","last_peered":"2024-04-23T14:31:26.833398+0000","last_clean":"2024-04-23T14:31:26.833398+0000","last_became_active":"2024-04-23T14:31:24.802462+0000","last_became_peered":"2024-04-23T14:31:24.802462+0000","last_unstale":"2024-04-23T14:31:26.833398+0000","last_undegraded":"2024-04-23T14:31:26.833398+0000","last_fullsized":"2024-04-23T14:31:26.833398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821929+0000","last_change":"2024-04-23T14:31:25.840470+0000","last_active":"2024-04-23T14:31:26.821929+0000","last_peered":"2024-04-23T14:31:26.821929+0000","last_clean":"2024-04-23T14:31:26.821929+0000","last_became_active":"2024-04-23T14:31:25.840160+0000","last_became_peered":"2024-04-23T14:31:25.840160+0000","last_unstale":"2024-04-23T14:31:26.821929+0000","last_undegraded":"2024-04-23T14:31:26.821929+0000","last_fullsized":"2024-04-23T14:31:26.821929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820475+0000","last_change":"2024-04-23T14:31:24.798634+0000","last_active":"2024-04-23T14:31:26.820475+0000","last_peered":"2024-04-23T14:31:26.820475+0000","last_clean":"2024-04-23T14:31:26.820475+0000","last_became_active":"2024-04-23T14:31:24.798307+0000","last_became_peered":"2024-04-23T14:31:24.798307+0000","last_unstale":"2024-04-23T14:31:26.820475+0000","last_undegraded":"2024-04-23T14:31:26.820475+0000","last_fullsized":"2024-04-23T14:31:26.820475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227416+0000","last_change":"2024-04-23T14:31:25.830485+0000","last_active":"2024-04-23T14:31:27.227416+0000","last_peered":"2024-04-23T14:31:27.227416+0000","last_clean":"2024-04-23T14:31:27.227416+0000","last_became_active":"2024-04-23T14:31:25.830001+0000","last_became_peered":"2024-04-23T14:31:25.830001+0000","last_unstale":"2024-04-23T14:31:27.227416+0000","last_undegraded":"2024-04-23T14:31:27.227416+0000","last_fullsized":"2024-04-23T14:31:27.227416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225604+0000","last_change":"2024-04-23T14:31:24.800143+0000","last_active":"2024-04-23T14:31:27.225604+0000","last_peered":"2024-04-23T14:31:27.225604+0000","last_clean":"2024-04-23T14:31:27.225604+0000","last_became_active":"2024-04-23T14:31:24.799828+0000","last_became_peered":"2024-04-23T14:31:24.799828+0000","last_unstale":"2024-04-23T14:31:27.225604+0000","last_undegraded":"2024-04-23T14:31:27.225604+0000","last_fullsized":"2024-04-23T14:31:27.225604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225544+0000","last_change":"2024-04-23T14:31:25.839070+0000","last_active":"2024-04-23T14:31:27.225544+0000","last_peered":"2024-04-23T14:31:27.225544+0000","last_clean":"2024-04-23T14:31:27.225544+0000","last_became_active":"2024-04-23T14:31:25.838724+0000","last_became_peered":"2024-04-23T14:31:25.838724+0000","last_unstale":"2024-04-23T14:31:27.225544+0000","last_undegraded":"2024-04-23T14:31:27.225544+0000","last_fullsized":"2024-04-23T14:31:27.225544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229381+0000","last_change":"2024-04-23T14:31:24.805607+0000","last_active":"2024-04-23T14:31:27.229381+0000","last_peered":"2024-04-23T14:31:27.229381+0000","last_clean":"2024-04-23T14:31:27.229381+0000","last_became_active":"2024-04-23T14:31:24.805268+0000","last_became_peered":"2024-04-23T14:31:24.805268+0000","last_unstale":"2024-04-23T14:31:27.229381+0000","last_undegraded":"2024-04-23T14:31:27.229381+0000","last_fullsized":"2024-04-23T14:31:27.229381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228113+0000","last_change":"2024-04-23T14:31:25.835264+0000","last_active":"2024-04-23T14:31:27.228113+0000","last_peered":"2024-04-23T14:31:27.228113+0000","last_clean":"2024-04-23T14:31:27.228113+0000","last_became_active":"2024-04-23T14:31:25.834845+0000","last_became_peered":"2024-04-23T14:31:25.834845+0000","last_unstale":"2024-04-23T14:31:27.228113+0000","last_undegraded":"2024-04-23T14:31:27.228113+0000","last_fullsized":"2024-04-23T14:31:27.228113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820500+0000","last_change":"2024-04-23T14:31:24.803064+0000","last_active":"2024-04-23T14:31:26.820500+0000","last_peered":"2024-04-23T14:31:26.820500+0000","last_clean":"2024-04-23T14:31:26.820500+0000","last_became_active":"2024-04-23T14:31:24.801163+0000","last_became_peered":"2024-04-23T14:31:24.801163+0000","last_unstale":"2024-04-23T14:31:26.820500+0000","last_undegraded":"2024-04-23T14:31:26.820500+0000","last_fullsized":"2024-04-23T14:31:26.820500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820307+0000","last_change":"2024-04-23T14:31:25.818141+0000","last_active":"2024-04-23T14:31:26.820307+0000","last_peered":"2024-04-23T14:31:26.820307+0000","last_clean":"2024-04-23T14:31:26.820307+0000","last_became_active":"2024-04-23T14:31:25.817558+0000","last_became_peered":"2024-04-23T14:31:25.817558+0000","last_unstale":"2024-04-23T14:31:26.820307+0000","last_undegraded":"2024-04-23T14:31:26.820307+0000","last_fullsized":"2024-04-23T14:31:26.820307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814267+0000","last_change":"2024-04-23T14:31:24.792560+0000","last_active":"2024-04-23T14:31:26.814267+0000","last_peered":"2024-04-23T14:31:26.814267+0000","last_clean":"2024-04-23T14:31:26.814267+0000","last_became_active":"2024-04-23T14:31:24.792055+0000","last_became_peered":"2024-04-23T14:31:24.792055+0000","last_unstale":"2024-04-23T14:31:26.814267+0000","last_undegraded":"2024-04-23T14:31:26.814267+0000","last_fullsized":"2024-04-23T14:31:26.814267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226832+0000","last_change":"2024-04-23T14:31:25.833895+0000","last_active":"2024-04-23T14:31:27.226832+0000","last_peered":"2024-04-23T14:31:27.226832+0000","last_clean":"2024-04-23T14:31:27.226832+0000","last_became_active":"2024-04-23T14:31:25.833570+0000","last_became_peered":"2024-04-23T14:31:25.833570+0000","last_unstale":"2024-04-23T14:31:27.226832+0000","last_undegraded":"2024-04-23T14:31:27.226832+0000","last_fullsized":"2024-04-23T14:31:27.226832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820916+0000","last_change":"2024-04-23T14:31:24.794838+0000","last_active":"2024-04-23T14:31:26.820916+0000","last_peered":"2024-04-23T14:31:26.820916+0000","last_clean":"2024-04-23T14:31:26.820916+0000","last_became_active":"2024-04-23T14:31:24.794545+0000","last_became_peered":"2024-04-23T14:31:24.794545+0000","last_unstale":"2024-04-23T14:31:26.820916+0000","last_undegraded":"2024-04-23T14:31:26.820916+0000","last_fullsized":"2024-04-23T14:31:26.820916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:47.137471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228074+0000","last_change":"2024-04-23T14:31:25.850256+0000","last_active":"2024-04-23T14:31:27.228074+0000","last_peered":"2024-04-23T14:31:27.228074+0000","last_clean":"2024-04-23T14:31:27.228074+0000","last_became_active":"2024-04-23T14:31:25.849503+0000","last_became_peered":"2024-04-23T14:31:25.849503+0000","last_unstale":"2024-04-23T14:31:27.228074+0000","last_undegraded":"2024-04-23T14:31:27.228074+0000","last_fullsized":"2024-04-23T14:31:27.228074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814819+0000","last_change":"2024-04-23T14:31:24.798518+0000","last_active":"2024-04-23T14:31:26.814819+0000","last_peered":"2024-04-23T14:31:26.814819+0000","last_clean":"2024-04-23T14:31:26.814819+0000","last_became_active":"2024-04-23T14:31:24.798170+0000","last_became_peered":"2024-04-23T14:31:24.798170+0000","last_unstale":"2024-04-23T14:31:26.814819+0000","last_undegraded":"2024-04-23T14:31:26.814819+0000","last_fullsized":"2024-04-23T14:31:26.814819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228158+0000","last_change":"2024-04-23T14:31:25.849745+0000","last_active":"2024-04-23T14:31:27.228158+0000","last_peered":"2024-04-23T14:31:27.228158+0000","last_clean":"2024-04-23T14:31:27.228158+0000","last_became_active":"2024-04-23T14:31:25.849311+0000","last_became_peered":"2024-04-23T14:31:25.849311+0000","last_unstale":"2024-04-23T14:31:27.228158+0000","last_undegraded":"2024-04-23T14:31:27.228158+0000","last_fullsized":"2024-04-23T14:31:27.228158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833547+0000","last_change":"2024-04-23T14:31:24.801624+0000","last_active":"2024-04-23T14:31:26.833547+0000","last_peered":"2024-04-23T14:31:26.833547+0000","last_clean":"2024-04-23T14:31:26.833547+0000","last_became_active":"2024-04-23T14:31:24.800838+0000","last_became_peered":"2024-04-23T14:31:24.800838+0000","last_unstale":"2024-04-23T14:31:26.833547+0000","last_undegraded":"2024-04-23T14:31:26.833547+0000","last_fullsized":"2024-04-23T14:31:26.833547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229691+0000","last_change":"2024-04-23T14:31:25.836837+0000","last_active":"2024-04-23T14:31:27.229691+0000","last_peered":"2024-04-23T14:31:27.229691+0000","last_clean":"2024-04-23T14:31:27.229691+0000","last_became_active":"2024-04-23T14:31:25.836473+0000","last_became_peered":"2024-04-23T14:31:25.836473+0000","last_unstale":"2024-04-23T14:31:27.229691+0000","last_undegraded":"2024-04-23T14:31:27.229691+0000","last_fullsized":"2024-04-23T14:31:27.229691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814648+0000","last_change":"2024-04-23T14:31:24.801079+0000","last_active":"2024-04-23T14:31:26.814648+0000","last_peered":"2024-04-23T14:31:26.814648+0000","last_clean":"2024-04-23T14:31:26.814648+0000","last_became_active":"2024-04-23T14:31:24.800766+0000","last_became_peered":"2024-04-23T14:31:24.800766+0000","last_unstale":"2024-04-23T14:31:26.814648+0000","last_undegraded":"2024-04-23T14:31:26.814648+0000","last_fullsized":"2024-04-23T14:31:26.814648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229022+0000","last_change":"2024-04-23T14:31:25.842503+0000","last_active":"2024-04-23T14:31:27.229022+0000","last_peered":"2024-04-23T14:31:27.229022+0000","last_clean":"2024-04-23T14:31:27.229022+0000","last_became_active":"2024-04-23T14:31:25.842174+0000","last_became_peered":"2024-04-23T14:31:25.842174+0000","last_unstale":"2024-04-23T14:31:27.229022+0000","last_undegraded":"2024-04-23T14:31:27.229022+0000","last_fullsized":"2024-04-23T14:31:27.229022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:24:27.716211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226647+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:31:27.226647+0000","last_peered":"2024-04-23T14:31:27.226647+0000","last_clean":"2024-04-23T14:31:27.226647+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:31:27.226647+0000","last_undegraded":"2024-04-23T14:31:27.226647+0000","last_fullsized":"2024-04-23T14:31:27.226647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228944+0000","last_change":"2024-04-23T14:31:24.803500+0000","last_active":"2024-04-23T14:31:27.228944+0000","last_peered":"2024-04-23T14:31:27.228944+0000","last_clean":"2024-04-23T14:31:27.228944+0000","last_became_active":"2024-04-23T14:31:24.803188+0000","last_became_peered":"2024-04-23T14:31:24.803188+0000","last_unstale":"2024-04-23T14:31:27.228944+0000","last_undegraded":"2024-04-23T14:31:27.228944+0000","last_fullsized":"2024-04-23T14:31:27.228944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224272+0000","last_change":"2024-04-23T14:31:25.842991+0000","last_active":"2024-04-23T14:31:27.224272+0000","last_peered":"2024-04-23T14:31:27.224272+0000","last_clean":"2024-04-23T14:31:27.224272+0000","last_became_active":"2024-04-23T14:31:25.842681+0000","last_became_peered":"2024-04-23T14:31:25.842681+0000","last_unstale":"2024-04-23T14:31:27.224272+0000","last_undegraded":"2024-04-23T14:31:27.224272+0000","last_fullsized":"2024-04-23T14:31:27.224272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226837+0000","last_change":"2024-04-23T14:31:24.815103+0000","last_active":"2024-04-23T14:31:27.226837+0000","last_peered":"2024-04-23T14:31:27.226837+0000","last_clean":"2024-04-23T14:31:27.226837+0000","last_became_active":"2024-04-23T14:31:24.814780+0000","last_became_peered":"2024-04-23T14:31:24.814780+0000","last_unstale":"2024-04-23T14:31:27.226837+0000","last_undegraded":"2024-04-23T14:31:27.226837+0000","last_fullsized":"2024-04-23T14:31:27.226837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225528+0000","last_change":"2024-04-23T14:31:25.828041+0000","last_active":"2024-04-23T14:31:27.225528+0000","last_peered":"2024-04-23T14:31:27.225528+0000","last_clean":"2024-04-23T14:31:27.225528+0000","last_became_active":"2024-04-23T14:31:25.827517+0000","last_became_peered":"2024-04-23T14:31:25.827517+0000","last_unstale":"2024-04-23T14:31:27.225528+0000","last_undegraded":"2024-04-23T14:31:27.225528+0000","last_fullsized":"2024-04-23T14:31:27.225528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814388+0000","last_change":"2024-04-23T14:31:24.800057+0000","last_active":"2024-04-23T14:31:26.814388+0000","last_peered":"2024-04-23T14:31:26.814388+0000","last_clean":"2024-04-23T14:31:26.814388+0000","last_became_active":"2024-04-23T14:31:24.799710+0000","last_became_peered":"2024-04-23T14:31:24.799710+0000","last_unstale":"2024-04-23T14:31:26.814388+0000","last_undegraded":"2024-04-23T14:31:26.814388+0000","last_fullsized":"2024-04-23T14:31:26.814388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833395+0000","last_change":"2024-04-23T14:31:25.820222+0000","last_active":"2024-04-23T14:31:26.833395+0000","last_peered":"2024-04-23T14:31:26.833395+0000","last_clean":"2024-04-23T14:31:26.833395+0000","last_became_active":"2024-04-23T14:31:25.818514+0000","last_became_peered":"2024-04-23T14:31:25.818514+0000","last_unstale":"2024-04-23T14:31:26.833395+0000","last_undegraded":"2024-04-23T14:31:26.833395+0000","last_fullsized":"2024-04-23T14:31:26.833395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229585+0000","last_change":"2024-04-23T14:31:24.802434+0000","last_active":"2024-04-23T14:31:27.229585+0000","last_peered":"2024-04-23T14:31:27.229585+0000","last_clean":"2024-04-23T14:31:27.229585+0000","last_became_active":"2024-04-23T14:31:24.800504+0000","last_became_peered":"2024-04-23T14:31:24.800504+0000","last_unstale":"2024-04-23T14:31:27.229585+0000","last_undegraded":"2024-04-23T14:31:27.229585+0000","last_fullsized":"2024-04-23T14:31:27.229585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.822417+0000","last_change":"2024-04-23T14:31:25.824755+0000","last_active":"2024-04-23T14:31:26.822417+0000","last_peered":"2024-04-23T14:31:26.822417+0000","last_clean":"2024-04-23T14:31:26.822417+0000","last_became_active":"2024-04-23T14:31:25.820209+0000","last_became_peered":"2024-04-23T14:31:25.820209+0000","last_unstale":"2024-04-23T14:31:26.822417+0000","last_undegraded":"2024-04-23T14:31:26.822417+0000","last_fullsized":"2024-04-23T14:31:26.822417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833340+0000","last_change":"2024-04-23T14:31:24.795154+0000","last_active":"2024-04-23T14:31:26.833340+0000","last_peered":"2024-04-23T14:31:26.833340+0000","last_clean":"2024-04-23T14:31:26.833340+0000","last_became_active":"2024-04-23T14:31:24.794851+0000","last_became_peered":"2024-04-23T14:31:24.794851+0000","last_unstale":"2024-04-23T14:31:26.833340+0000","last_undegraded":"2024-04-23T14:31:26.833340+0000","last_fullsized":"2024-04-23T14:31:26.833340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230608+0000","last_change":"2024-04-23T14:31:25.836961+0000","last_active":"2024-04-23T14:31:27.230608+0000","last_peered":"2024-04-23T14:31:27.230608+0000","last_clean":"2024-04-23T14:31:27.230608+0000","last_became_active":"2024-04-23T14:31:25.836649+0000","last_became_peered":"2024-04-23T14:31:25.836649+0000","last_unstale":"2024-04-23T14:31:27.230608+0000","last_undegraded":"2024-04-23T14:31:27.230608+0000","last_fullsized":"2024-04-23T14:31:27.230608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229915+0000","last_change":"2024-04-23T14:31:24.817767+0000","last_active":"2024-04-23T14:31:27.229915+0000","last_peered":"2024-04-23T14:31:27.229915+0000","last_clean":"2024-04-23T14:31:27.229915+0000","last_became_active":"2024-04-23T14:31:24.817440+0000","last_became_peered":"2024-04-23T14:31:24.817440+0000","last_unstale":"2024-04-23T14:31:27.229915+0000","last_undegraded":"2024-04-23T14:31:27.229915+0000","last_fullsized":"2024-04-23T14:31:27.229915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833407+0000","last_change":"2024-04-23T14:31:25.821773+0000","last_active":"2024-04-23T14:31:26.833407+0000","last_peered":"2024-04-23T14:31:26.833407+0000","last_clean":"2024-04-23T14:31:26.833407+0000","last_became_active":"2024-04-23T14:31:25.820550+0000","last_became_peered":"2024-04-23T14:31:25.820550+0000","last_unstale":"2024-04-23T14:31:26.833407+0000","last_undegraded":"2024-04-23T14:31:26.833407+0000","last_fullsized":"2024-04-23T14:31:26.833407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.834047+0000","last_change":"2024-04-23T14:31:24.798112+0000","last_active":"2024-04-23T14:31:26.834047+0000","last_peered":"2024-04-23T14:31:26.834047+0000","last_clean":"2024-04-23T14:31:26.834047+0000","last_became_active":"2024-04-23T14:31:24.797826+0000","last_became_peered":"2024-04-23T14:31:24.797826+0000","last_unstale":"2024-04-23T14:31:26.834047+0000","last_undegraded":"2024-04-23T14:31:26.834047+0000","last_fullsized":"2024-04-23T14:31:26.834047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225465+0000","last_change":"2024-04-23T14:31:25.835731+0000","last_active":"2024-04-23T14:31:27.225465+0000","last_peered":"2024-04-23T14:31:27.225465+0000","last_clean":"2024-04-23T14:31:27.225465+0000","last_became_active":"2024-04-23T14:31:25.835493+0000","last_became_peered":"2024-04-23T14:31:25.835493+0000","last_unstale":"2024-04-23T14:31:27.225465+0000","last_undegraded":"2024-04-23T14:31:27.225465+0000","last_fullsized":"2024-04-23T14:31:27.225465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821845+0000","last_change":"2024-04-23T14:31:24.813578+0000","last_active":"2024-04-23T14:31:26.821845+0000","last_peered":"2024-04-23T14:31:26.821845+0000","last_clean":"2024-04-23T14:31:26.821845+0000","last_became_active":"2024-04-23T14:31:24.812481+0000","last_became_peered":"2024-04-23T14:31:24.812481+0000","last_unstale":"2024-04-23T14:31:26.821845+0000","last_undegraded":"2024-04-23T14:31:26.821845+0000","last_fullsized":"2024-04-23T14:31:26.821845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225467+0000","last_change":"2024-04-23T14:31:25.808683+0000","last_active":"2024-04-23T14:31:27.225467+0000","last_peered":"2024-04-23T14:31:27.225467+0000","last_clean":"2024-04-23T14:31:27.225467+0000","last_became_active":"2024-04-23T14:31:25.808301+0000","last_became_peered":"2024-04-23T14:31:25.808301+0000","last_unstale":"2024-04-23T14:31:27.225467+0000","last_undegraded":"2024-04-23T14:31:27.225467+0000","last_fullsized":"2024-04-23T14:31:27.225467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226336+0000","last_change":"2024-04-23T14:31:24.817508+0000","last_active":"2024-04-23T14:31:27.226336+0000","last_peered":"2024-04-23T14:31:27.226336+0000","last_clean":"2024-04-23T14:31:27.226336+0000","last_became_active":"2024-04-23T14:31:24.817260+0000","last_became_peered":"2024-04-23T14:31:24.817260+0000","last_unstale":"2024-04-23T14:31:27.226336+0000","last_undegraded":"2024-04-23T14:31:27.226336+0000","last_fullsized":"2024-04-23T14:31:27.226336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821504+0000","last_change":"2024-04-23T14:31:25.827518+0000","last_active":"2024-04-23T14:31:26.821504+0000","last_peered":"2024-04-23T14:31:26.821504+0000","last_clean":"2024-04-23T14:31:26.821504+0000","last_became_active":"2024-04-23T14:31:25.822569+0000","last_became_peered":"2024-04-23T14:31:25.822569+0000","last_unstale":"2024-04-23T14:31:26.821504+0000","last_undegraded":"2024-04-23T14:31:26.821504+0000","last_fullsized":"2024-04-23T14:31:26.821504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:58.606933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833567+0000","last_change":"2024-04-23T14:31:24.807224+0000","last_active":"2024-04-23T14:31:26.833567+0000","last_peered":"2024-04-23T14:31:26.833567+0000","last_clean":"2024-04-23T14:31:26.833567+0000","last_became_active":"2024-04-23T14:31:24.806958+0000","last_became_peered":"2024-04-23T14:31:24.806958+0000","last_unstale":"2024-04-23T14:31:26.833567+0000","last_undegraded":"2024-04-23T14:31:26.833567+0000","last_fullsized":"2024-04-23T14:31:26.833567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224355+0000","last_change":"2024-04-23T14:31:25.821318+0000","last_active":"2024-04-23T14:31:27.224355+0000","last_peered":"2024-04-23T14:31:27.224355+0000","last_clean":"2024-04-23T14:31:27.224355+0000","last_became_active":"2024-04-23T14:31:25.820930+0000","last_became_peered":"2024-04-23T14:31:25.820930+0000","last_unstale":"2024-04-23T14:31:27.224355+0000","last_undegraded":"2024-04-23T14:31:27.224355+0000","last_fullsized":"2024-04-23T14:31:27.224355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833571+0000","last_change":"2024-04-23T14:31:24.799281+0000","last_active":"2024-04-23T14:31:26.833571+0000","last_peered":"2024-04-23T14:31:26.833571+0000","last_clean":"2024-04-23T14:31:26.833571+0000","last_became_active":"2024-04-23T14:31:24.798804+0000","last_became_peered":"2024-04-23T14:31:24.798804+0000","last_unstale":"2024-04-23T14:31:26.833571+0000","last_undegraded":"2024-04-23T14:31:26.833571+0000","last_fullsized":"2024-04-23T14:31:26.833571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224405+0000","last_change":"2024-04-23T14:31:25.821631+0000","last_active":"2024-04-23T14:31:27.224405+0000","last_peered":"2024-04-23T14:31:27.224405+0000","last_clean":"2024-04-23T14:31:27.224405+0000","last_became_active":"2024-04-23T14:31:25.820439+0000","last_became_peered":"2024-04-23T14:31:25.820439+0000","last_unstale":"2024-04-23T14:31:27.224405+0000","last_undegraded":"2024-04-23T14:31:27.224405+0000","last_fullsized":"2024-04-23T14:31:27.224405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228114+0000","last_change":"2024-04-23T14:31:24.814559+0000","last_active":"2024-04-23T14:31:27.228114+0000","last_peered":"2024-04-23T14:31:27.228114+0000","last_clean":"2024-04-23T14:31:27.228114+0000","last_became_active":"2024-04-23T14:31:24.814274+0000","last_became_peered":"2024-04-23T14:31:24.814274+0000","last_unstale":"2024-04-23T14:31:27.228114+0000","last_undegraded":"2024-04-23T14:31:27.228114+0000","last_fullsized":"2024-04-23T14:31:27.228114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821011+0000","last_change":"2024-04-23T14:31:25.829871+0000","last_active":"2024-04-23T14:31:26.821011+0000","last_peered":"2024-04-23T14:31:26.821011+0000","last_clean":"2024-04-23T14:31:26.821011+0000","last_became_active":"2024-04-23T14:31:25.828620+0000","last_became_peered":"2024-04-23T14:31:25.828620+0000","last_unstale":"2024-04-23T14:31:26.821011+0000","last_undegraded":"2024-04-23T14:31:26.821011+0000","last_fullsized":"2024-04-23T14:31:26.821011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230152+0000","last_change":"2024-04-23T14:31:25.846861+0000","last_active":"2024-04-23T14:31:27.230152+0000","last_peered":"2024-04-23T14:31:27.230152+0000","last_clean":"2024-04-23T14:31:27.230152+0000","last_became_active":"2024-04-23T14:31:25.846267+0000","last_became_peered":"2024-04-23T14:31:25.846267+0000","last_unstale":"2024-04-23T14:31:27.230152+0000","last_undegraded":"2024-04-23T14:31:27.230152+0000","last_fullsized":"2024-04-23T14:31:27.230152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227598+0000","last_change":"2024-04-23T14:31:24.816507+0000","last_active":"2024-04-23T14:31:27.227598+0000","last_peered":"2024-04-23T14:31:27.227598+0000","last_clean":"2024-04-23T14:31:27.227598+0000","last_became_active":"2024-04-23T14:31:24.816159+0000","last_became_peered":"2024-04-23T14:31:24.816159+0000","last_unstale":"2024-04-23T14:31:27.227598+0000","last_undegraded":"2024-04-23T14:31:27.227598+0000","last_fullsized":"2024-04-23T14:31:27.227598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229590+0000","last_change":"2024-04-23T14:31:25.846016+0000","last_active":"2024-04-23T14:31:27.229590+0000","last_peered":"2024-04-23T14:31:27.229590+0000","last_clean":"2024-04-23T14:31:27.229590+0000","last_became_active":"2024-04-23T14:31:25.845682+0000","last_became_peered":"2024-04-23T14:31:25.845682+0000","last_unstale":"2024-04-23T14:31:27.229590+0000","last_undegraded":"2024-04-23T14:31:27.229590+0000","last_fullsized":"2024-04-23T14:31:27.229590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229170+0000","last_change":"2024-04-23T14:31:24.819529+0000","last_active":"2024-04-23T14:31:27.229170+0000","last_peered":"2024-04-23T14:31:27.229170+0000","last_clean":"2024-04-23T14:31:27.229170+0000","last_became_active":"2024-04-23T14:31:24.819162+0000","last_became_peered":"2024-04-23T14:31:24.819162+0000","last_unstale":"2024-04-23T14:31:27.229170+0000","last_undegraded":"2024-04-23T14:31:27.229170+0000","last_fullsized":"2024-04-23T14:31:27.229170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225173+0000","last_change":"2024-04-23T14:31:25.837928+0000","last_active":"2024-04-23T14:31:27.225173+0000","last_peered":"2024-04-23T14:31:27.225173+0000","last_clean":"2024-04-23T14:31:27.225173+0000","last_became_active":"2024-04-23T14:31:25.837281+0000","last_became_peered":"2024-04-23T14:31:25.837281+0000","last_unstale":"2024-04-23T14:31:27.225173+0000","last_undegraded":"2024-04-23T14:31:27.225173+0000","last_fullsized":"2024-04-23T14:31:27.225173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:59:15.348815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224998+0000","last_change":"2024-04-23T14:31:24.793060+0000","last_active":"2024-04-23T14:31:27.224998+0000","last_peered":"2024-04-23T14:31:27.224998+0000","last_clean":"2024-04-23T14:31:27.224998+0000","last_became_active":"2024-04-23T14:31:24.792763+0000","last_became_peered":"2024-04-23T14:31:24.792763+0000","last_unstale":"2024-04-23T14:31:27.224998+0000","last_undegraded":"2024-04-23T14:31:27.224998+0000","last_fullsized":"2024-04-23T14:31:27.224998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224786+0000","last_change":"2024-04-23T14:31:25.824548+0000","last_active":"2024-04-23T14:31:27.224786+0000","last_peered":"2024-04-23T14:31:27.224786+0000","last_clean":"2024-04-23T14:31:27.224786+0000","last_became_active":"2024-04-23T14:31:25.824207+0000","last_became_peered":"2024-04-23T14:31:25.824207+0000","last_unstale":"2024-04-23T14:31:27.224786+0000","last_undegraded":"2024-04-23T14:31:27.224786+0000","last_fullsized":"2024-04-23T14:31:27.224786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225623+0000","last_change":"2024-04-23T14:31:24.810861+0000","last_active":"2024-04-23T14:31:27.225623+0000","last_peered":"2024-04-23T14:31:27.225623+0000","last_clean":"2024-04-23T14:31:27.225623+0000","last_became_active":"2024-04-23T14:31:24.810466+0000","last_became_peered":"2024-04-23T14:31:24.810466+0000","last_unstale":"2024-04-23T14:31:27.225623+0000","last_undegraded":"2024-04-23T14:31:27.225623+0000","last_fullsized":"2024-04-23T14:31:27.225623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:31:30.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:31:30.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.846+0000 7fa041800640 1 Processor -- start 2024-04-23T14:31:30.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.846+0000 7fa041800640 1 -- start start 2024-04-23T14:31:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.847+0000 7fa041800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 0x7fa03c1676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.847+0000 7fa041800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa03c0d0c70 con 0x7fa03c15c700 2024-04-23T14:31:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.847+0000 7fa041800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa03c0d0f50 con 0x7fa03c0d0600 2024-04-23T14:31:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.848+0000 7fa041800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa03c0b46b0 con 0x7fa03c06c0e0 2024-04-23T14:31:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.848+0000 7fa03be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa03c15c700 0x7fa03c160d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36434/0 (socket says 172.21.15.59:36434) 2024-04-23T14:31:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.848+0000 7fa03be00640 1 -- 172.21.15.59:0/1249876038 learned_addr learned my addr 172.21.15.59:0/1249876038 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.848+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229776721 0 0) 0x7fa03c0d0c70 con 0x7fa03c15c700 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.848+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa020003520 con 0x7fa03c15c700 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.848+0000 7fa040e00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 0x7fa03c1676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 315758843 0 0) 0x7fa020003520 con 0x7fa03c15c700 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 msgr2=0x7fa03c1676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 0x7fa03c1676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 >> v1:172.21.15.76:6789/0 conn(0x7fa03c0d0600 legacy=0x7fa03c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c0b4340 con 0x7fa03c15c700 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa02c001a10 con 0x7fa03c15c700 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 >> v1:172.21.15.59:6789/0 conn(0x7fa03c15c700 legacy=0x7fa03c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 0x7fa0200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa020004010 0x7fa020004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa020004d40 0x7fa0200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c0b46b0 con 0x7fa020004d40 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa02000bb80 con 0x7fa020004010 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa02000bd80 con 0x7fa03c06c0e0 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa040e00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa020004d40 0x7fa0200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03b400640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa020004010 0x7fa020004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.849+0000 7fa03be00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 0x7fa0200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.850+0000 7fa040e00640 1 -- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 msgr2=0x7fa0200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:30.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.850+0000 7fa040e00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa03c06c0e0 0x7fa0200039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.850+0000 7fa040e00640 1 -- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa020004010 msgr2=0x7fa020004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.850+0000 7fa040e00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa020004010 0x7fa020004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.850+0000 7fa040e00640 1 -- 172.21.15.59:0/1249876038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa020003520 con 0x7fa020004d40 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.851+0000 7fa040e00640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa020004d40 0x7fa0200051c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa03c072300 tx=0x7fa020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56416e906ab97c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.851+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034030070 con 0x7fa020004d40 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.851+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03402ad70 con 0x7fa020004d40 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.851+0000 7fa03aa00640 1 -- 172.21.15.59:0/1249876038 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400b600 con 0x7fa020004d40 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.852+0000 7fa041800640 1 -- 172.21.15.59:0/1249876038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa020004d40 msgr2=0x7fa0200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.852+0000 7fa041800640 1 --2- 172.21.15.59:0/1249876038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa020004d40 0x7fa0200051c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa03c072300 tx=0x7fa020005830 comp rx=0 tx=0).stop 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.852+0000 7fa040e00640 1 -- 172.21.15.59:0/1249876038 reap_dead start 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.852+0000 7fa041800640 1 -- 172.21.15.59:0/1249876038 shutdown_connections 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.852+0000 7fa041800640 1 -- 172.21.15.59:0/1249876038 >> 172.21.15.59:0/1249876038 conn(0x7fa03c09aa00 msgr2=0x7fa03c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.852+0000 7fa041800640 1 -- 172.21.15.59:0/1249876038 shutdown_connections 2024-04-23T14:31:30.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.852+0000 7fa041800640 1 -- 172.21.15.59:0/1249876038 wait complete. 2024-04-23T14:31:30.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.854+0000 7fa041800640 1 Processor -- start 2024-04-23T14:31:30.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.854+0000 7fa041800640 1 -- start start 2024-04-23T14:31:30.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa041800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa020004d40 0x7fa03c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa041800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa03c0c6460 0x7fa03c0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa041800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03c0cc980 0x7fa03c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa041800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c0b5cd0 con 0x7fa03c0cc980 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa041800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa03c0b5b50 con 0x7fa03c0c6460 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa041800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa03c0b5e50 con 0x7fa020004d40 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa040e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03c0cc980 0x7fa03c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa040e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03c0cc980 0x7fa03c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33110/0 (socket says 172.21.15.59:33110) 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.855+0000 7fa040e00640 1 -- 172.21.15.59:0/3036941571 learned_addr learned my addr 172.21.15.59:0/3036941571 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa040e00640 1 -- 172.21.15.59:0/3036941571 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa020004d40 msgr2=0x7fa03c15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa040e00640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa020004d40 0x7fa03c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa040e00640 1 -- 172.21.15.59:0/3036941571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa03c0c6460 msgr2=0x7fa03c0c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa040e00640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa03c0c6460 0x7fa03c0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa040e00640 1 -- 172.21.15.59:0/3036941571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03c154d20 con 0x7fa03c0cc980 2024-04-23T14:31:30.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa040e00640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03c0cc980 0x7fa03c0cce00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa034035d10 tx=0x7fa034004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:30.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa033e00640 1 -- 172.21.15.59:0/3036941571 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400b600 con 0x7fa03c0cc980 2024-04-23T14:31:30.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa033e00640 1 -- 172.21.15.59:0/3036941571 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03402ab70 con 0x7fa03c0cc980 2024-04-23T14:31:30.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa033e00640 1 -- 172.21.15.59:0/3036941571 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034004020 con 0x7fa03c0cc980 2024-04-23T14:31:30.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03c1f7ba0 con 0x7fa03c0cc980 2024-04-23T14:31:30.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.856+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03c1f8160 con 0x7fa03c0cc980 2024-04-23T14:31:30.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.857+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa03c04fd90 con 0x7fa03c0cc980 2024-04-23T14:31:30.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.859+0000 7fa033e00640 1 -- 172.21.15.59:0/3036941571 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa034047030 con 0x7fa03c0cc980 2024-04-23T14:31:30.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.860+0000 7fa033e00640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa01806c8f0 0x7fa01806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:30.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.860+0000 7fa033e00640 1 -- 172.21.15.59:0/3036941571 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa0340abd10 con 0x7fa03c0cc980 2024-04-23T14:31:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.860+0000 7fa03be00640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa01806c8f0 0x7fa01806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.861+0000 7fa03be00640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa01806c8f0 0x7fa01806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa02c007c40 tx=0x7fa02c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:30.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:30.865+0000 7fa033e00640 1 -- 172.21.15.59:0/3036941571 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0340b1020 con 0x7fa03c0cc980 2024-04-23T14:31:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.059+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa03c0b7260 con 0x7fa01806c8f0 2024-04-23T14:31:31.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.069+0000 7fa033e00640 1 -- 172.21.15.59:0/3036941571 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fa03c0b7260 con 0x7fa01806c8f0 2024-04-23T14:31:31.076 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:31.078 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:31:31.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa01806c8f0 msgr2=0x7fa01806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa01806c8f0 0x7fa01806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa02c007c40 tx=0x7fa02c009040 comp rx=0 tx=0).stop 2024-04-23T14:31:31.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03c0cc980 msgr2=0x7fa03c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03c0cc980 0x7fa03c0cce00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa034035d10 tx=0x7fa034004730 comp rx=0 tx=0).stop 2024-04-23T14:31:31.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 shutdown_connections 2024-04-23T14:31:31.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa01806c8f0 0x7fa01806edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03c0cc980 0x7fa03c0cce00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa03c0c6460 0x7fa03c0c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 --2- 172.21.15.59:0/3036941571 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa020004d40 0x7fa03c15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 >> 172.21.15.59:0/3036941571 conn(0x7fa03c09aa00 msgr2=0x7fa03c0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:31.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 shutdown_connections 2024-04-23T14:31:31.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.075+0000 7fa041800640 1 -- 172.21.15.59:0/3036941571 wait complete. 2024-04-23T14:31:31.094 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-23T14:31:30.345260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001810"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227768+0000","last_change":"2024-04-23T14:31:25.849551+0000","last_active":"2024-04-23T14:31:27.227768+0000","last_peered":"2024-04-23T14:31:27.227768+0000","last_clean":"2024-04-23T14:31:27.227768+0000","last_became_active":"2024-04-23T14:31:25.848756+0000","last_became_peered":"2024-04-23T14:31:25.848756+0000","last_unstale":"2024-04-23T14:31:27.227768+0000","last_undegraded":"2024-04-23T14:31:27.227768+0000","last_fullsized":"2024-04-23T14:31:27.227768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833869+0000","last_change":"2024-04-23T14:31:25.819653+0000","last_active":"2024-04-23T14:31:26.833869+0000","last_peered":"2024-04-23T14:31:26.833869+0000","last_clean":"2024-04-23T14:31:26.833869+0000","last_became_active":"2024-04-23T14:31:25.819296+0000","last_became_peered":"2024-04-23T14:31:25.819296+0000","last_unstale":"2024-04-23T14:31:26.833869+0000","last_undegraded":"2024-04-23T14:31:26.833869+0000","last_fullsized":"2024-04-23T14:31:26.833869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225689+0000","last_change":"2024-04-23T14:31:25.818736+0000","last_active":"2024-04-23T14:31:27.225689+0000","last_peered":"2024-04-23T14:31:27.225689+0000","last_clean":"2024-04-23T14:31:27.225689+0000","last_became_active":"2024-04-23T14:31:25.816583+0000","last_became_peered":"2024-04-23T14:31:25.816583+0000","last_unstale":"2024-04-23T14:31:27.225689+0000","last_undegraded":"2024-04-23T14:31:27.225689+0000","last_fullsized":"2024-04-23T14:31:27.225689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229281+0000","last_change":"2024-04-23T14:31:25.836663+0000","last_active":"2024-04-23T14:31:27.229281+0000","last_peered":"2024-04-23T14:31:27.229281+0000","last_clean":"2024-04-23T14:31:27.229281+0000","last_became_active":"2024-04-23T14:31:25.836357+0000","last_became_peered":"2024-04-23T14:31:25.836357+0000","last_unstale":"2024-04-23T14:31:27.229281+0000","last_undegraded":"2024-04-23T14:31:27.229281+0000","last_fullsized":"2024-04-23T14:31:27.229281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227740+0000","last_change":"2024-04-23T14:31:25.830442+0000","last_active":"2024-04-23T14:31:27.227740+0000","last_peered":"2024-04-23T14:31:27.227740+0000","last_clean":"2024-04-23T14:31:27.227740+0000","last_became_active":"2024-04-23T14:31:25.830032+0000","last_became_peered":"2024-04-23T14:31:25.830032+0000","last_unstale":"2024-04-23T14:31:27.227740+0000","last_undegraded":"2024-04-23T14:31:27.227740+0000","last_fullsized":"2024-04-23T14:31:27.227740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228994+0000","last_change":"2024-04-23T14:31:25.835030+0000","last_active":"2024-04-23T14:31:27.228994+0000","last_peered":"2024-04-23T14:31:27.228994+0000","last_clean":"2024-04-23T14:31:27.228994+0000","last_became_active":"2024-04-23T14:31:25.834764+0000","last_became_peered":"2024-04-23T14:31:25.834764+0000","last_unstale":"2024-04-23T14:31:27.228994+0000","last_undegraded":"2024-04-23T14:31:27.228994+0000","last_fullsized":"2024-04-23T14:31:27.228994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226818+0000","last_change":"2024-04-23T14:31:25.833373+0000","last_active":"2024-04-23T14:31:27.226818+0000","last_peered":"2024-04-23T14:31:27.226818+0000","last_clean":"2024-04-23T14:31:27.226818+0000","last_became_active":"2024-04-23T14:31:25.831555+0000","last_became_peered":"2024-04-23T14:31:25.831555+0000","last_unstale":"2024-04-23T14:31:27.226818+0000","last_undegraded":"2024-04-23T14:31:27.226818+0000","last_fullsized":"2024-04-23T14:31:27.226818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224117+0000","last_change":"2024-04-23T14:31:25.828675+0000","last_active":"2024-04-23T14:31:27.224117+0000","last_peered":"2024-04-23T14:31:27.224117+0000","last_clean":"2024-04-23T14:31:27.224117+0000","last_became_active":"2024-04-23T14:31:25.828381+0000","last_became_peered":"2024-04-23T14:31:25.828381+0000","last_unstale":"2024-04-23T14:31:27.224117+0000","last_undegraded":"2024-04-23T14:31:27.224117+0000","last_fullsized":"2024-04-23T14:31:27.224117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228299+0000","last_change":"2024-04-23T14:31:25.839705+0000","last_active":"2024-04-23T14:31:27.228299+0000","last_peered":"2024-04-23T14:31:27.228299+0000","last_clean":"2024-04-23T14:31:27.228299+0000","last_became_active":"2024-04-23T14:31:25.839432+0000","last_became_peered":"2024-04-23T14:31:25.839432+0000","last_unstale":"2024-04-23T14:31:27.228299+0000","last_undegraded":"2024-04-23T14:31:27.228299+0000","last_fullsized":"2024-04-23T14:31:27.228299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821382+0000","last_change":"2024-04-23T14:31:25.833382+0000","last_active":"2024-04-23T14:31:26.821382+0000","last_peered":"2024-04-23T14:31:26.821382+0000","last_clean":"2024-04-23T14:31:26.821382+0000","last_became_active":"2024-04-23T14:31:25.833010+0000","last_became_peered":"2024-04-23T14:31:25.833010+0000","last_unstale":"2024-04-23T14:31:26.821382+0000","last_undegraded":"2024-04-23T14:31:26.821382+0000","last_fullsized":"2024-04-23T14:31:26.821382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820889+0000","last_change":"2024-04-23T14:31:25.828614+0000","last_active":"2024-04-23T14:31:26.820889+0000","last_peered":"2024-04-23T14:31:26.820889+0000","last_clean":"2024-04-23T14:31:26.820889+0000","last_became_active":"2024-04-23T14:31:25.828342+0000","last_became_peered":"2024-04-23T14:31:25.828342+0000","last_unstale":"2024-04-23T14:31:26.820889+0000","last_undegraded":"2024-04-23T14:31:26.820889+0000","last_fullsized":"2024-04-23T14:31:26.820889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821732+0000","last_change":"2024-04-23T14:31:25.825396+0000","last_active":"2024-04-23T14:31:26.821732+0000","last_peered":"2024-04-23T14:31:26.821732+0000","last_clean":"2024-04-23T14:31:26.821732+0000","last_became_active":"2024-04-23T14:31:25.822425+0000","last_became_peered":"2024-04-23T14:31:25.822425+0000","last_unstale":"2024-04-23T14:31:26.821732+0000","last_undegraded":"2024-04-23T14:31:26.821732+0000","last_fullsized":"2024-04-23T14:31:26.821732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:05.846065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223667+0000","last_change":"2024-04-23T14:31:25.823390+0000","last_active":"2024-04-23T14:31:27.223667+0000","last_peered":"2024-04-23T14:31:27.223667+0000","last_clean":"2024-04-23T14:31:27.223667+0000","last_became_active":"2024-04-23T14:31:25.823054+0000","last_became_peered":"2024-04-23T14:31:25.823054+0000","last_unstale":"2024-04-23T14:31:27.223667+0000","last_undegraded":"2024-04-23T14:31:27.223667+0000","last_fullsized":"2024-04-23T14:31:27.223667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225120+0000","last_change":"2024-04-23T14:31:25.818198+0000","last_active":"2024-04-23T14:31:27.225120+0000","last_peered":"2024-04-23T14:31:27.225120+0000","last_clean":"2024-04-23T14:31:27.225120+0000","last_became_active":"2024-04-23T14:31:25.817873+0000","last_became_peered":"2024-04-23T14:31:25.817873+0000","last_unstale":"2024-04-23T14:31:27.225120+0000","last_undegraded":"2024-04-23T14:31:27.225120+0000","last_fullsized":"2024-04-23T14:31:27.225120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:08:54.362020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833481+0000","last_change":"2024-04-23T14:31:25.829886+0000","last_active":"2024-04-23T14:31:26.833481+0000","last_peered":"2024-04-23T14:31:26.833481+0000","last_clean":"2024-04-23T14:31:26.833481+0000","last_became_active":"2024-04-23T14:31:25.829086+0000","last_became_peered":"2024-04-23T14:31:25.829086+0000","last_unstale":"2024-04-23T14:31:26.833481+0000","last_undegraded":"2024-04-23T14:31:26.833481+0000","last_fullsized":"2024-04-23T14:31:26.833481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228453+0000","last_change":"2024-04-23T14:31:25.818775+0000","last_active":"2024-04-23T14:31:27.228453+0000","last_peered":"2024-04-23T14:31:27.228453+0000","last_clean":"2024-04-23T14:31:27.228453+0000","last_became_active":"2024-04-23T14:31:25.818318+0000","last_became_peered":"2024-04-23T14:31:25.818318+0000","last_unstale":"2024-04-23T14:31:27.228453+0000","last_undegraded":"2024-04-23T14:31:27.228453+0000","last_fullsized":"2024-04-23T14:31:27.228453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230731+0000","last_change":"2024-04-23T14:31:25.850111+0000","last_active":"2024-04-23T14:31:27.230731+0000","last_peered":"2024-04-23T14:31:27.230731+0000","last_clean":"2024-04-23T14:31:27.230731+0000","last_became_active":"2024-04-23T14:31:25.848968+0000","last_became_peered":"2024-04-23T14:31:25.848968+0000","last_unstale":"2024-04-23T14:31:27.230731+0000","last_undegraded":"2024-04-23T14:31:27.230731+0000","last_fullsized":"2024-04-23T14:31:27.230731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.815057+0000","last_change":"2024-04-23T14:31:25.813711+0000","last_active":"2024-04-23T14:31:26.815057+0000","last_peered":"2024-04-23T14:31:26.815057+0000","last_clean":"2024-04-23T14:31:26.815057+0000","last_became_active":"2024-04-23T14:31:25.813377+0000","last_became_peered":"2024-04-23T14:31:25.813377+0000","last_unstale":"2024-04-23T14:31:26.815057+0000","last_undegraded":"2024-04-23T14:31:26.815057+0000","last_fullsized":"2024-04-23T14:31:26.815057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230045+0000","last_change":"2024-04-23T14:31:25.842518+0000","last_active":"2024-04-23T14:31:27.230045+0000","last_peered":"2024-04-23T14:31:27.230045+0000","last_clean":"2024-04-23T14:31:27.230045+0000","last_became_active":"2024-04-23T14:31:25.842265+0000","last_became_peered":"2024-04-23T14:31:25.842265+0000","last_unstale":"2024-04-23T14:31:27.230045+0000","last_undegraded":"2024-04-23T14:31:27.230045+0000","last_fullsized":"2024-04-23T14:31:27.230045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:10:49.364737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820245+0000","last_change":"2024-04-23T14:31:25.824601+0000","last_active":"2024-04-23T14:31:26.820245+0000","last_peered":"2024-04-23T14:31:26.820245+0000","last_clean":"2024-04-23T14:31:26.820245+0000","last_became_active":"2024-04-23T14:31:25.824326+0000","last_became_peered":"2024-04-23T14:31:25.824326+0000","last_unstale":"2024-04-23T14:31:26.820245+0000","last_undegraded":"2024-04-23T14:31:26.820245+0000","last_fullsized":"2024-04-23T14:31:26.820245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225202+0000","last_change":"2024-04-23T14:31:25.824382+0000","last_active":"2024-04-23T14:31:27.225202+0000","last_peered":"2024-04-23T14:31:27.225202+0000","last_clean":"2024-04-23T14:31:27.225202+0000","last_became_active":"2024-04-23T14:31:25.823991+0000","last_became_peered":"2024-04-23T14:31:25.823991+0000","last_unstale":"2024-04-23T14:31:27.225202+0000","last_undegraded":"2024-04-23T14:31:27.225202+0000","last_fullsized":"2024-04-23T14:31:27.225202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229105+0000","last_change":"2024-04-23T14:31:25.842389+0000","last_active":"2024-04-23T14:31:27.229105+0000","last_peered":"2024-04-23T14:31:27.229105+0000","last_clean":"2024-04-23T14:31:27.229105+0000","last_became_active":"2024-04-23T14:31:25.841584+0000","last_became_peered":"2024-04-23T14:31:25.841584+0000","last_unstale":"2024-04-23T14:31:27.229105+0000","last_undegraded":"2024-04-23T14:31:27.229105+0000","last_fullsized":"2024-04-23T14:31:27.229105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223827+0000","last_change":"2024-04-23T14:31:25.830515+0000","last_active":"2024-04-23T14:31:27.223827+0000","last_peered":"2024-04-23T14:31:27.223827+0000","last_clean":"2024-04-23T14:31:27.223827+0000","last_became_active":"2024-04-23T14:31:25.830209+0000","last_became_peered":"2024-04-23T14:31:25.830209+0000","last_unstale":"2024-04-23T14:31:27.223827+0000","last_undegraded":"2024-04-23T14:31:27.223827+0000","last_fullsized":"2024-04-23T14:31:27.223827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228540+0000","last_change":"2024-04-23T14:31:25.824411+0000","last_active":"2024-04-23T14:31:27.228540+0000","last_peered":"2024-04-23T14:31:27.228540+0000","last_clean":"2024-04-23T14:31:27.228540+0000","last_became_active":"2024-04-23T14:31:25.824052+0000","last_became_peered":"2024-04-23T14:31:25.824052+0000","last_unstale":"2024-04-23T14:31:27.228540+0000","last_undegraded":"2024-04-23T14:31:27.228540+0000","last_fullsized":"2024-04-23T14:31:27.228540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224322+0000","last_change":"2024-04-23T14:31:25.830939+0000","last_active":"2024-04-23T14:31:27.224322+0000","last_peered":"2024-04-23T14:31:27.224322+0000","last_clean":"2024-04-23T14:31:27.224322+0000","last_became_active":"2024-04-23T14:31:25.830632+0000","last_became_peered":"2024-04-23T14:31:25.830632+0000","last_unstale":"2024-04-23T14:31:27.224322+0000","last_undegraded":"2024-04-23T14:31:27.224322+0000","last_fullsized":"2024-04-23T14:31:27.224322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821455+0000","last_change":"2024-04-23T14:31:25.832634+0000","last_active":"2024-04-23T14:31:26.821455+0000","last_peered":"2024-04-23T14:31:26.821455+0000","last_clean":"2024-04-23T14:31:26.821455+0000","last_became_active":"2024-04-23T14:31:25.832329+0000","last_became_peered":"2024-04-23T14:31:25.832329+0000","last_unstale":"2024-04-23T14:31:26.821455+0000","last_undegraded":"2024-04-23T14:31:26.821455+0000","last_fullsized":"2024-04-23T14:31:26.821455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833312+0000","last_change":"2024-04-23T14:31:25.822589+0000","last_active":"2024-04-23T14:31:26.833312+0000","last_peered":"2024-04-23T14:31:26.833312+0000","last_clean":"2024-04-23T14:31:26.833312+0000","last_became_active":"2024-04-23T14:31:25.822061+0000","last_became_peered":"2024-04-23T14:31:25.822061+0000","last_unstale":"2024-04-23T14:31:26.833312+0000","last_undegraded":"2024-04-23T14:31:26.833312+0000","last_fullsized":"2024-04-23T14:31:26.833312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223980+0000","last_change":"2024-04-23T14:31:25.832096+0000","last_active":"2024-04-23T14:31:27.223980+0000","last_peered":"2024-04-23T14:31:27.223980+0000","last_clean":"2024-04-23T14:31:27.223980+0000","last_became_active":"2024-04-23T14:31:25.829333+0000","last_became_peered":"2024-04-23T14:31:25.829333+0000","last_unstale":"2024-04-23T14:31:27.223980+0000","last_undegraded":"2024-04-23T14:31:27.223980+0000","last_fullsized":"2024-04-23T14:31:27.223980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225363+0000","last_change":"2024-04-23T14:31:25.826410+0000","last_active":"2024-04-23T14:31:27.225363+0000","last_peered":"2024-04-23T14:31:27.225363+0000","last_clean":"2024-04-23T14:31:27.225363+0000","last_became_active":"2024-04-23T14:31:25.826088+0000","last_became_peered":"2024-04-23T14:31:25.826088+0000","last_unstale":"2024-04-23T14:31:27.225363+0000","last_undegraded":"2024-04-23T14:31:27.225363+0000","last_fullsized":"2024-04-23T14:31:27.225363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229439+0000","last_change":"2024-04-23T14:31:25.832202+0000","last_active":"2024-04-23T14:31:27.229439+0000","last_peered":"2024-04-23T14:31:27.229439+0000","last_clean":"2024-04-23T14:31:27.229439+0000","last_became_active":"2024-04-23T14:31:25.831719+0000","last_became_peered":"2024-04-23T14:31:25.831719+0000","last_unstale":"2024-04-23T14:31:27.229439+0000","last_undegraded":"2024-04-23T14:31:27.229439+0000","last_fullsized":"2024-04-23T14:31:27.229439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814413+0000","last_change":"2024-04-23T14:31:25.827667+0000","last_active":"2024-04-23T14:31:26.814413+0000","last_peered":"2024-04-23T14:31:26.814413+0000","last_clean":"2024-04-23T14:31:26.814413+0000","last_became_active":"2024-04-23T14:31:25.827302+0000","last_became_peered":"2024-04-23T14:31:25.827302+0000","last_unstale":"2024-04-23T14:31:26.814413+0000","last_undegraded":"2024-04-23T14:31:26.814413+0000","last_fullsized":"2024-04-23T14:31:26.814413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.819912+0000","last_change":"2024-04-23T14:31:25.817837+0000","last_active":"2024-04-23T14:31:26.819912+0000","last_peered":"2024-04-23T14:31:26.819912+0000","last_clean":"2024-04-23T14:31:26.819912+0000","last_became_active":"2024-04-23T14:31:25.817156+0000","last_became_peered":"2024-04-23T14:31:25.817156+0000","last_unstale":"2024-04-23T14:31:26.819912+0000","last_undegraded":"2024-04-23T14:31:26.819912+0000","last_fullsized":"2024-04-23T14:31:26.819912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820171+0000","last_change":"2024-04-23T14:31:25.821505+0000","last_active":"2024-04-23T14:31:26.820171+0000","last_peered":"2024-04-23T14:31:26.820171+0000","last_clean":"2024-04-23T14:31:26.820171+0000","last_became_active":"2024-04-23T14:31:25.821186+0000","last_became_peered":"2024-04-23T14:31:25.821186+0000","last_unstale":"2024-04-23T14:31:26.820171+0000","last_undegraded":"2024-04-23T14:31:26.820171+0000","last_fullsized":"2024-04-23T14:31:26.820171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:30.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224053+0000","last_change":"2024-04-23T14:31:24.810696+0000","last_active":"2024-04-23T14:31:27.224053+0000","last_peered":"2024-04-23T14:31:27.224053+0000","last_clean":"2024-04-23T14:31:27.224053+0000","last_became_active":"2024-04-23T14:31:24.810349+0000","last_became_peered":"2024-04-23T14:31:24.810349+0000","last_unstale":"2024-04-23T14:31:27.224053+0000","last_undegraded":"2024-04-23T14:31:27.224053+0000","last_fullsized":"2024-04-23T14:31:27.224053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225550+0000","last_change":"2024-04-23T14:31:25.824408+0000","last_active":"2024-04-23T14:31:27.225550+0000","last_peered":"2024-04-23T14:31:27.225550+0000","last_clean":"2024-04-23T14:31:27.225550+0000","last_became_active":"2024-04-23T14:31:25.822620+0000","last_became_peered":"2024-04-23T14:31:25.822620+0000","last_unstale":"2024-04-23T14:31:27.225550+0000","last_undegraded":"2024-04-23T14:31:27.225550+0000","last_fullsized":"2024-04-23T14:31:27.225550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820803+0000","last_change":"2024-04-23T14:31:24.801642+0000","last_active":"2024-04-23T14:31:26.820803+0000","last_peered":"2024-04-23T14:31:26.820803+0000","last_clean":"2024-04-23T14:31:26.820803+0000","last_became_active":"2024-04-23T14:31:24.801244+0000","last_became_peered":"2024-04-23T14:31:24.801244+0000","last_unstale":"2024-04-23T14:31:26.820803+0000","last_undegraded":"2024-04-23T14:31:26.820803+0000","last_fullsized":"2024-04-23T14:31:26.820803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223904+0000","last_change":"2024-04-23T14:31:24.796017+0000","last_active":"2024-04-23T14:31:27.223904+0000","last_peered":"2024-04-23T14:31:27.223904+0000","last_clean":"2024-04-23T14:31:27.223904+0000","last_became_active":"2024-04-23T14:31:24.795734+0000","last_became_peered":"2024-04-23T14:31:24.795734+0000","last_unstale":"2024-04-23T14:31:27.223904+0000","last_undegraded":"2024-04-23T14:31:27.223904+0000","last_fullsized":"2024-04-23T14:31:27.223904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223975+0000","last_change":"2024-04-23T14:31:25.819835+0000","last_active":"2024-04-23T14:31:27.223975+0000","last_peered":"2024-04-23T14:31:27.223975+0000","last_clean":"2024-04-23T14:31:27.223975+0000","last_became_active":"2024-04-23T14:31:25.819532+0000","last_became_peered":"2024-04-23T14:31:25.819532+0000","last_unstale":"2024-04-23T14:31:27.223975+0000","last_undegraded":"2024-04-23T14:31:27.223975+0000","last_fullsized":"2024-04-23T14:31:27.223975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228610+0000","last_change":"2024-04-23T14:31:24.802566+0000","last_active":"2024-04-23T14:31:27.228610+0000","last_peered":"2024-04-23T14:31:27.228610+0000","last_clean":"2024-04-23T14:31:27.228610+0000","last_became_active":"2024-04-23T14:31:24.802299+0000","last_became_peered":"2024-04-23T14:31:24.802299+0000","last_unstale":"2024-04-23T14:31:27.228610+0000","last_undegraded":"2024-04-23T14:31:27.228610+0000","last_fullsized":"2024-04-23T14:31:27.228610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833252+0000","last_change":"2024-04-23T14:31:25.822370+0000","last_active":"2024-04-23T14:31:26.833252+0000","last_peered":"2024-04-23T14:31:26.833252+0000","last_clean":"2024-04-23T14:31:26.833252+0000","last_became_active":"2024-04-23T14:31:25.822039+0000","last_became_peered":"2024-04-23T14:31:25.822039+0000","last_unstale":"2024-04-23T14:31:26.833252+0000","last_undegraded":"2024-04-23T14:31:26.833252+0000","last_fullsized":"2024-04-23T14:31:26.833252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821576+0000","last_change":"2024-04-23T14:31:24.806864+0000","last_active":"2024-04-23T14:31:26.821576+0000","last_peered":"2024-04-23T14:31:26.821576+0000","last_clean":"2024-04-23T14:31:26.821576+0000","last_became_active":"2024-04-23T14:31:24.806597+0000","last_became_peered":"2024-04-23T14:31:24.806597+0000","last_unstale":"2024-04-23T14:31:26.821576+0000","last_undegraded":"2024-04-23T14:31:26.821576+0000","last_fullsized":"2024-04-23T14:31:26.821576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230291+0000","last_change":"2024-04-23T14:31:25.842918+0000","last_active":"2024-04-23T14:31:27.230291+0000","last_peered":"2024-04-23T14:31:27.230291+0000","last_clean":"2024-04-23T14:31:27.230291+0000","last_became_active":"2024-04-23T14:31:25.842581+0000","last_became_peered":"2024-04-23T14:31:25.842581+0000","last_unstale":"2024-04-23T14:31:27.230291+0000","last_undegraded":"2024-04-23T14:31:27.230291+0000","last_fullsized":"2024-04-23T14:31:27.230291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229964+0000","last_change":"2024-04-23T14:31:24.805711+0000","last_active":"2024-04-23T14:31:27.229964+0000","last_peered":"2024-04-23T14:31:27.229964+0000","last_clean":"2024-04-23T14:31:27.229964+0000","last_became_active":"2024-04-23T14:31:24.805351+0000","last_became_peered":"2024-04-23T14:31:24.805351+0000","last_unstale":"2024-04-23T14:31:27.229964+0000","last_undegraded":"2024-04-23T14:31:27.229964+0000","last_fullsized":"2024-04-23T14:31:27.229964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225997+0000","last_change":"2024-04-23T14:31:25.837626+0000","last_active":"2024-04-23T14:31:27.225997+0000","last_peered":"2024-04-23T14:31:27.225997+0000","last_clean":"2024-04-23T14:31:27.225997+0000","last_became_active":"2024-04-23T14:31:25.836840+0000","last_became_peered":"2024-04-23T14:31:25.836840+0000","last_unstale":"2024-04-23T14:31:27.225997+0000","last_undegraded":"2024-04-23T14:31:27.225997+0000","last_fullsized":"2024-04-23T14:31:27.225997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833398+0000","last_change":"2024-04-23T14:31:24.802741+0000","last_active":"2024-04-23T14:31:26.833398+0000","last_peered":"2024-04-23T14:31:26.833398+0000","last_clean":"2024-04-23T14:31:26.833398+0000","last_became_active":"2024-04-23T14:31:24.802462+0000","last_became_peered":"2024-04-23T14:31:24.802462+0000","last_unstale":"2024-04-23T14:31:26.833398+0000","last_undegraded":"2024-04-23T14:31:26.833398+0000","last_fullsized":"2024-04-23T14:31:26.833398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821929+0000","last_change":"2024-04-23T14:31:25.840470+0000","last_active":"2024-04-23T14:31:26.821929+0000","last_peered":"2024-04-23T14:31:26.821929+0000","last_clean":"2024-04-23T14:31:26.821929+0000","last_became_active":"2024-04-23T14:31:25.840160+0000","last_became_peered":"2024-04-23T14:31:25.840160+0000","last_unstale":"2024-04-23T14:31:26.821929+0000","last_undegraded":"2024-04-23T14:31:26.821929+0000","last_fullsized":"2024-04-23T14:31:26.821929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820475+0000","last_change":"2024-04-23T14:31:24.798634+0000","last_active":"2024-04-23T14:31:26.820475+0000","last_peered":"2024-04-23T14:31:26.820475+0000","last_clean":"2024-04-23T14:31:26.820475+0000","last_became_active":"2024-04-23T14:31:24.798307+0000","last_became_peered":"2024-04-23T14:31:24.798307+0000","last_unstale":"2024-04-23T14:31:26.820475+0000","last_undegraded":"2024-04-23T14:31:26.820475+0000","last_fullsized":"2024-04-23T14:31:26.820475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227416+0000","last_change":"2024-04-23T14:31:25.830485+0000","last_active":"2024-04-23T14:31:27.227416+0000","last_peered":"2024-04-23T14:31:27.227416+0000","last_clean":"2024-04-23T14:31:27.227416+0000","last_became_active":"2024-04-23T14:31:25.830001+0000","last_became_peered":"2024-04-23T14:31:25.830001+0000","last_unstale":"2024-04-23T14:31:27.227416+0000","last_undegraded":"2024-04-23T14:31:27.227416+0000","last_fullsized":"2024-04-23T14:31:27.227416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225604+0000","last_change":"2024-04-23T14:31:24.800143+0000","last_active":"2024-04-23T14:31:27.225604+0000","last_peered":"2024-04-23T14:31:27.225604+0000","last_clean":"2024-04-23T14:31:27.225604+0000","last_became_active":"2024-04-23T14:31:24.799828+0000","last_became_peered":"2024-04-23T14:31:24.799828+0000","last_unstale":"2024-04-23T14:31:27.225604+0000","last_undegraded":"2024-04-23T14:31:27.225604+0000","last_fullsized":"2024-04-23T14:31:27.225604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225544+0000","last_change":"2024-04-23T14:31:25.839070+0000","last_active":"2024-04-23T14:31:27.225544+0000","last_peered":"2024-04-23T14:31:27.225544+0000","last_clean":"2024-04-23T14:31:27.225544+0000","last_became_active":"2024-04-23T14:31:25.838724+0000","last_became_peered":"2024-04-23T14:31:25.838724+0000","last_unstale":"2024-04-23T14:31:27.225544+0000","last_undegraded":"2024-04-23T14:31:27.225544+0000","last_fullsized":"2024-04-23T14:31:27.225544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229381+0000","last_change":"2024-04-23T14:31:24.805607+0000","last_active":"2024-04-23T14:31:27.229381+0000","last_peered":"2024-04-23T14:31:27.229381+0000","last_clean":"2024-04-23T14:31:27.229381+0000","last_became_active":"2024-04-23T14:31:24.805268+0000","last_became_peered":"2024-04-23T14:31:24.805268+0000","last_unstale":"2024-04-23T14:31:27.229381+0000","last_undegraded":"2024-04-23T14:31:27.229381+0000","last_fullsized":"2024-04-23T14:31:27.229381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228113+0000","last_change":"2024-04-23T14:31:25.835264+0000","last_active":"2024-04-23T14:31:27.228113+0000","last_peered":"2024-04-23T14:31:27.228113+0000","last_clean":"2024-04-23T14:31:27.228113+0000","last_became_active":"2024-04-23T14:31:25.834845+0000","last_became_peered":"2024-04-23T14:31:25.834845+0000","last_unstale":"2024-04-23T14:31:27.228113+0000","last_undegraded":"2024-04-23T14:31:27.228113+0000","last_fullsized":"2024-04-23T14:31:27.228113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820500+0000","last_change":"2024-04-23T14:31:24.803064+0000","last_active":"2024-04-23T14:31:26.820500+0000","last_peered":"2024-04-23T14:31:26.820500+0000","last_clean":"2024-04-23T14:31:26.820500+0000","last_became_active":"2024-04-23T14:31:24.801163+0000","last_became_peered":"2024-04-23T14:31:24.801163+0000","last_unstale":"2024-04-23T14:31:26.820500+0000","last_undegraded":"2024-04-23T14:31:26.820500+0000","last_fullsized":"2024-04-23T14:31:26.820500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820307+0000","last_change":"2024-04-23T14:31:25.818141+0000","last_active":"2024-04-23T14:31:26.820307+0000","last_peered":"2024-04-23T14:31:26.820307+0000","last_clean":"2024-04-23T14:31:26.820307+0000","last_became_active":"2024-04-23T14:31:25.817558+0000","last_became_peered":"2024-04-23T14:31:25.817558+0000","last_unstale":"2024-04-23T14:31:26.820307+0000","last_undegraded":"2024-04-23T14:31:26.820307+0000","last_fullsized":"2024-04-23T14:31:26.820307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814267+0000","last_change":"2024-04-23T14:31:24.792560+0000","last_active":"2024-04-23T14:31:26.814267+0000","last_peered":"2024-04-23T14:31:26.814267+0000","last_clean":"2024-04-23T14:31:26.814267+0000","last_became_active":"2024-04-23T14:31:24.792055+0000","last_became_peered":"2024-04-23T14:31:24.792055+0000","last_unstale":"2024-04-23T14:31:26.814267+0000","last_undegraded":"2024-04-23T14:31:26.814267+0000","last_fullsized":"2024-04-23T14:31:26.814267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226832+0000","last_change":"2024-04-23T14:31:25.833895+0000","last_active":"2024-04-23T14:31:27.226832+0000","last_peered":"2024-04-23T14:31:27.226832+0000","last_clean":"2024-04-23T14:31:27.226832+0000","last_became_active":"2024-04-23T14:31:25.833570+0000","last_became_peered":"2024-04-23T14:31:25.833570+0000","last_unstale":"2024-04-23T14:31:27.226832+0000","last_undegraded":"2024-04-23T14:31:27.226832+0000","last_fullsized":"2024-04-23T14:31:27.226832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820916+0000","last_change":"2024-04-23T14:31:24.794838+0000","last_active":"2024-04-23T14:31:26.820916+0000","last_peered":"2024-04-23T14:31:26.820916+0000","last_clean":"2024-04-23T14:31:26.820916+0000","last_became_active":"2024-04-23T14:31:24.794545+0000","last_became_peered":"2024-04-23T14:31:24.794545+0000","last_unstale":"2024-04-23T14:31:26.820916+0000","last_undegraded":"2024-04-23T14:31:26.820916+0000","last_fullsized":"2024-04-23T14:31:26.820916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:47.137471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228074+0000","last_change":"2024-04-23T14:31:25.850256+0000","last_active":"2024-04-23T14:31:27.228074+0000","last_peered":"2024-04-23T14:31:27.228074+0000","last_clean":"2024-04-23T14:31:27.228074+0000","last_became_active":"2024-04-23T14:31:25.849503+0000","last_became_peered":"2024-04-23T14:31:25.849503+0000","last_unstale":"2024-04-23T14:31:27.228074+0000","last_undegraded":"2024-04-23T14:31:27.228074+0000","last_fullsized":"2024-04-23T14:31:27.228074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814819+0000","last_change":"2024-04-23T14:31:24.798518+0000","last_active":"2024-04-23T14:31:26.814819+0000","last_peered":"2024-04-23T14:31:26.814819+0000","last_clean":"2024-04-23T14:31:26.814819+0000","last_became_active":"2024-04-23T14:31:24.798170+0000","last_became_peered":"2024-04-23T14:31:24.798170+0000","last_unstale":"2024-04-23T14:31:26.814819+0000","last_undegraded":"2024-04-23T14:31:26.814819+0000","last_fullsized":"2024-04-23T14:31:26.814819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228158+0000","last_change":"2024-04-23T14:31:25.849745+0000","last_active":"2024-04-23T14:31:27.228158+0000","last_peered":"2024-04-23T14:31:27.228158+0000","last_clean":"2024-04-23T14:31:27.228158+0000","last_became_active":"2024-04-23T14:31:25.849311+0000","last_became_peered":"2024-04-23T14:31:25.849311+0000","last_unstale":"2024-04-23T14:31:27.228158+0000","last_undegraded":"2024-04-23T14:31:27.228158+0000","last_fullsized":"2024-04-23T14:31:27.228158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833547+0000","last_change":"2024-04-23T14:31:24.801624+0000","last_active":"2024-04-23T14:31:26.833547+0000","last_peered":"2024-04-23T14:31:26.833547+0000","last_clean":"2024-04-23T14:31:26.833547+0000","last_became_active":"2024-04-23T14:31:24.800838+0000","last_became_peered":"2024-04-23T14:31:24.800838+0000","last_unstale":"2024-04-23T14:31:26.833547+0000","last_undegraded":"2024-04-23T14:31:26.833547+0000","last_fullsized":"2024-04-23T14:31:26.833547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229691+0000","last_change":"2024-04-23T14:31:25.836837+0000","last_active":"2024-04-23T14:31:27.229691+0000","last_peered":"2024-04-23T14:31:27.229691+0000","last_clean":"2024-04-23T14:31:27.229691+0000","last_became_active":"2024-04-23T14:31:25.836473+0000","last_became_peered":"2024-04-23T14:31:25.836473+0000","last_unstale":"2024-04-23T14:31:27.229691+0000","last_undegraded":"2024-04-23T14:31:27.229691+0000","last_fullsized":"2024-04-23T14:31:27.229691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814648+0000","last_change":"2024-04-23T14:31:24.801079+0000","last_active":"2024-04-23T14:31:26.814648+0000","last_peered":"2024-04-23T14:31:26.814648+0000","last_clean":"2024-04-23T14:31:26.814648+0000","last_became_active":"2024-04-23T14:31:24.800766+0000","last_became_peered":"2024-04-23T14:31:24.800766+0000","last_unstale":"2024-04-23T14:31:26.814648+0000","last_undegraded":"2024-04-23T14:31:26.814648+0000","last_fullsized":"2024-04-23T14:31:26.814648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229022+0000","last_change":"2024-04-23T14:31:25.842503+0000","last_active":"2024-04-23T14:31:27.229022+0000","last_peered":"2024-04-23T14:31:27.229022+0000","last_clean":"2024-04-23T14:31:27.229022+0000","last_became_active":"2024-04-23T14:31:25.842174+0000","last_became_peered":"2024-04-23T14:31:25.842174+0000","last_unstale":"2024-04-23T14:31:27.229022+0000","last_undegraded":"2024-04-23T14:31:27.229022+0000","last_fullsized":"2024-04-23T14:31:27.229022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:24:27.716211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226647+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:31:27.226647+0000","last_peered":"2024-04-23T14:31:27.226647+0000","last_clean":"2024-04-23T14:31:27.226647+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:31:27.226647+0000","last_undegraded":"2024-04-23T14:31:27.226647+0000","last_fullsized":"2024-04-23T14:31:27.226647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228944+0000","last_change":"2024-04-23T14:31:24.803500+0000","last_active":"2024-04-23T14:31:27.228944+0000","last_peered":"2024-04-23T14:31:27.228944+0000","last_clean":"2024-04-23T14:31:27.228944+0000","last_became_active":"2024-04-23T14:31:24.803188+0000","last_became_peered":"2024-04-23T14:31:24.803188+0000","last_unstale":"2024-04-23T14:31:27.228944+0000","last_undegraded":"2024-04-23T14:31:27.228944+0000","last_fullsized":"2024-04-23T14:31:27.228944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224272+0000","last_change":"2024-04-23T14:31:25.842991+0000","last_active":"2024-04-23T14:31:27.224272+0000","last_peered":"2024-04-23T14:31:27.224272+0000","last_clean":"2024-04-23T14:31:27.224272+0000","last_became_active":"2024-04-23T14:31:25.842681+0000","last_became_peered":"2024-04-23T14:31:25.842681+0000","last_unstale":"2024-04-23T14:31:27.224272+0000","last_undegraded":"2024-04-23T14:31:27.224272+0000","last_fullsized":"2024-04-23T14:31:27.224272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226837+0000","last_change":"2024-04-23T14:31:24.815103+0000","last_active":"2024-04-23T14:31:27.226837+0000","last_peered":"2024-04-23T14:31:27.226837+0000","last_clean":"2024-04-23T14:31:27.226837+0000","last_became_active":"2024-04-23T14:31:24.814780+0000","last_became_peered":"2024-04-23T14:31:24.814780+0000","last_unstale":"2024-04-23T14:31:27.226837+0000","last_undegraded":"2024-04-23T14:31:27.226837+0000","last_fullsized":"2024-04-23T14:31:27.226837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225528+0000","last_change":"2024-04-23T14:31:25.828041+0000","last_active":"2024-04-23T14:31:27.225528+0000","last_peered":"2024-04-23T14:31:27.225528+0000","last_clean":"2024-04-23T14:31:27.225528+0000","last_became_active":"2024-04-23T14:31:25.827517+0000","last_became_peered":"2024-04-23T14:31:25.827517+0000","last_unstale":"2024-04-23T14:31:27.225528+0000","last_undegraded":"2024-04-23T14:31:27.225528+0000","last_fullsized":"2024-04-23T14:31:27.225528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814388+0000","last_change":"2024-04-23T14:31:24.800057+0000","last_active":"2024-04-23T14:31:26.814388+0000","last_peered":"2024-04-23T14:31:26.814388+0000","last_clean":"2024-04-23T14:31:26.814388+0000","last_became_active":"2024-04-23T14:31:24.799710+0000","last_became_peered":"2024-04-23T14:31:24.799710+0000","last_unstale":"2024-04-23T14:31:26.814388+0000","last_undegraded":"2024-04-23T14:31:26.814388+0000","last_fullsized":"2024-04-23T14:31:26.814388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833395+0000","last_change":"2024-04-23T14:31:25.820222+0000","last_active":"2024-04-23T14:31:26.833395+0000","last_peered":"2024-04-23T14:31:26.833395+0000","last_clean":"2024-04-23T14:31:26.833395+0000","last_became_active":"2024-04-23T14:31:25.818514+0000","last_became_peered":"2024-04-23T14:31:25.818514+0000","last_unstale":"2024-04-23T14:31:26.833395+0000","last_undegraded":"2024-04-23T14:31:26.833395+0000","last_fullsized":"2024-04-23T14:31:26.833395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229585+0000","last_change":"2024-04-23T14:31:24.802434+0000","last_active":"2024-04-23T14:31:27.229585+0000","last_peered":"2024-04-23T14:31:27.229585+0000","last_clean":"2024-04-23T14:31:27.229585+0000","last_became_active":"2024-04-23T14:31:24.800504+0000","last_became_peered":"2024-04-23T14:31:24.800504+0000","last_unstale":"2024-04-23T14:31:27.229585+0000","last_undegraded":"2024-04-23T14:31:27.229585+0000","last_fullsized":"2024-04-23T14:31:27.229585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.822417+0000","last_change":"2024-04-23T14:31:25.824755+0000","last_active":"2024-04-23T14:31:26.822417+0000","last_peered":"2024-04-23T14:31:26.822417+0000","last_clean":"2024-04-23T14:31:26.822417+0000","last_became_active":"2024-04-23T14:31:25.820209+0000","last_became_peered":"2024-04-23T14:31:25.820209+0000","last_unstale":"2024-04-23T14:31:26.822417+0000","last_undegraded":"2024-04-23T14:31:26.822417+0000","last_fullsized":"2024-04-23T14:31:26.822417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833340+0000","last_change":"2024-04-23T14:31:24.795154+0000","last_active":"2024-04-23T14:31:26.833340+0000","last_peered":"2024-04-23T14:31:26.833340+0000","last_clean":"2024-04-23T14:31:26.833340+0000","last_became_active":"2024-04-23T14:31:24.794851+0000","last_became_peered":"2024-04-23T14:31:24.794851+0000","last_unstale":"2024-04-23T14:31:26.833340+0000","last_undegraded":"2024-04-23T14:31:26.833340+0000","last_fullsized":"2024-04-23T14:31:26.833340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230608+0000","last_change":"2024-04-23T14:31:25.836961+0000","last_active":"2024-04-23T14:31:27.230608+0000","last_peered":"2024-04-23T14:31:27.230608+0000","last_clean":"2024-04-23T14:31:27.230608+0000","last_became_active":"2024-04-23T14:31:25.836649+0000","last_became_peered":"2024-04-23T14:31:25.836649+0000","last_unstale":"2024-04-23T14:31:27.230608+0000","last_undegraded":"2024-04-23T14:31:27.230608+0000","last_fullsized":"2024-04-23T14:31:27.230608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229915+0000","last_change":"2024-04-23T14:31:24.817767+0000","last_active":"2024-04-23T14:31:27.229915+0000","last_peered":"2024-04-23T14:31:27.229915+0000","last_clean":"2024-04-23T14:31:27.229915+0000","last_became_active":"2024-04-23T14:31:24.817440+0000","last_became_peered":"2024-04-23T14:31:24.817440+0000","last_unstale":"2024-04-23T14:31:27.229915+0000","last_undegraded":"2024-04-23T14:31:27.229915+0000","last_fullsized":"2024-04-23T14:31:27.229915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833407+0000","last_change":"2024-04-23T14:31:25.821773+0000","last_active":"2024-04-23T14:31:26.833407+0000","last_peered":"2024-04-23T14:31:26.833407+0000","last_clean":"2024-04-23T14:31:26.833407+0000","last_became_active":"2024-04-23T14:31:25.820550+0000","last_became_peered":"2024-04-23T14:31:25.820550+0000","last_unstale":"2024-04-23T14:31:26.833407+0000","last_undegraded":"2024-04-23T14:31:26.833407+0000","last_fullsized":"2024-04-23T14:31:26.833407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.834047+0000","last_change":"2024-04-23T14:31:24.798112+0000","last_active":"2024-04-23T14:31:26.834047+0000","last_peered":"2024-04-23T14:31:26.834047+0000","last_clean":"2024-04-23T14:31:26.834047+0000","last_became_active":"2024-04-23T14:31:24.797826+0000","last_became_peered":"2024-04-23T14:31:24.797826+0000","last_unstale":"2024-04-23T14:31:26.834047+0000","last_undegraded":"2024-04-23T14:31:26.834047+0000","last_fullsized":"2024-04-23T14:31:26.834047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225465+0000","last_change":"2024-04-23T14:31:25.835731+0000","last_active":"2024-04-23T14:31:27.225465+0000","last_peered":"2024-04-23T14:31:27.225465+0000","last_clean":"2024-04-23T14:31:27.225465+0000","last_became_active":"2024-04-23T14:31:25.835493+0000","last_became_peered":"2024-04-23T14:31:25.835493+0000","last_unstale":"2024-04-23T14:31:27.225465+0000","last_undegraded":"2024-04-23T14:31:27.225465+0000","last_fullsized":"2024-04-23T14:31:27.225465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821845+0000","last_change":"2024-04-23T14:31:24.813578+0000","last_active":"2024-04-23T14:31:26.821845+0000","last_peered":"2024-04-23T14:31:26.821845+0000","last_clean":"2024-04-23T14:31:26.821845+0000","last_became_active":"2024-04-23T14:31:24.812481+0000","last_became_peered":"2024-04-23T14:31:24.812481+0000","last_unstale":"2024-04-23T14:31:26.821845+0000","last_undegraded":"2024-04-23T14:31:26.821845+0000","last_fullsized":"2024-04-23T14:31:26.821845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225467+0000","last_change":"2024-04-23T14:31:25.808683+0000","last_active":"2024-04-23T14:31:27.225467+0000","last_peered":"2024-04-23T14:31:27.225467+0000","last_clean":"2024-04-23T14:31:27.225467+0000","last_became_active":"2024-04-23T14:31:25.808301+0000","last_became_peered":"2024-04-23T14:31:25.808301+0000","last_unstale":"2024-04-23T14:31:27.225467+0000","last_undegraded":"2024-04-23T14:31:27.225467+0000","last_fullsized":"2024-04-23T14:31:27.225467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226336+0000","last_change":"2024-04-23T14:31:24.817508+0000","last_active":"2024-04-23T14:31:27.226336+0000","last_peered":"2024-04-23T14:31:27.226336+0000","last_clean":"2024-04-23T14:31:27.226336+0000","last_became_active":"2024-04-23T14:31:24.817260+0000","last_became_peered":"2024-04-23T14:31:24.817260+0000","last_unstale":"2024-04-23T14:31:27.226336+0000","last_undegraded":"2024-04-23T14:31:27.226336+0000","last_fullsized":"2024-04-23T14:31:27.226336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821504+0000","last_change":"2024-04-23T14:31:25.827518+0000","last_active":"2024-04-23T14:31:26.821504+0000","last_peered":"2024-04-23T14:31:26.821504+0000","last_clean":"2024-04-23T14:31:26.821504+0000","last_became_active":"2024-04-23T14:31:25.822569+0000","last_became_peered":"2024-04-23T14:31:25.822569+0000","last_unstale":"2024-04-23T14:31:26.821504+0000","last_undegraded":"2024-04-23T14:31:26.821504+0000","last_fullsized":"2024-04-23T14:31:26.821504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:58.606933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833567+0000","last_change":"2024-04-23T14:31:24.807224+0000","last_active":"2024-04-23T14:31:26.833567+0000","last_peered":"2024-04-23T14:31:26.833567+0000","last_clean":"2024-04-23T14:31:26.833567+0000","last_became_active":"2024-04-23T14:31:24.806958+0000","last_became_peered":"2024-04-23T14:31:24.806958+0000","last_unstale":"2024-04-23T14:31:26.833567+0000","last_undegraded":"2024-04-23T14:31:26.833567+0000","last_fullsized":"2024-04-23T14:31:26.833567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224355+0000","last_change":"2024-04-23T14:31:25.821318+0000","last_active":"2024-04-23T14:31:27.224355+0000","last_peered":"2024-04-23T14:31:27.224355+0000","last_clean":"2024-04-23T14:31:27.224355+0000","last_became_active":"2024-04-23T14:31:25.820930+0000","last_became_peered":"2024-04-23T14:31:25.820930+0000","last_unstale":"2024-04-23T14:31:27.224355+0000","last_undegraded":"2024-04-23T14:31:27.224355+0000","last_fullsized":"2024-04-23T14:31:27.224355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833571+0000","last_change":"2024-04-23T14:31:24.799281+0000","last_active":"2024-04-23T14:31:26.833571+0000","last_peered":"2024-04-23T14:31:26.833571+0000","last_clean":"2024-04-23T14:31:26.833571+0000","last_became_active":"2024-04-23T14:31:24.798804+0000","last_became_peered":"2024-04-23T14:31:24.798804+0000","last_unstale":"2024-04-23T14:31:26.833571+0000","last_undegraded":"2024-04-23T14:31:26.833571+0000","last_fullsized":"2024-04-23T14:31:26.833571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224405+0000","last_change":"2024-04-23T14:31:25.821631+0000","last_active":"2024-04-23T14:31:27.224405+0000","last_peered":"2024-04-23T14:31:27.224405+0000","last_clean":"2024-04-23T14:31:27.224405+0000","last_became_active":"2024-04-23T14:31:25.820439+0000","last_became_peered":"2024-04-23T14:31:25.820439+0000","last_unstale":"2024-04-23T14:31:27.224405+0000","last_undegraded":"2024-04-23T14:31:27.224405+0000","last_fullsized":"2024-04-23T14:31:27.224405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228114+0000","last_change":"2024-04-23T14:31:24.814559+0000","last_active":"2024-04-23T14:31:27.228114+0000","last_peered":"2024-04-23T14:31:27.228114+0000","last_clean":"2024-04-23T14:31:27.228114+0000","last_became_active":"2024-04-23T14:31:24.814274+0000","last_became_peered":"2024-04-23T14:31:24.814274+0000","last_unstale":"2024-04-23T14:31:27.228114+0000","last_undegraded":"2024-04-23T14:31:27.228114+0000","last_fullsized":"2024-04-23T14:31:27.228114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821011+0000","last_change":"2024-04-23T14:31:25.829871+0000","last_active":"2024-04-23T14:31:26.821011+0000","last_peered":"2024-04-23T14:31:26.821011+0000","last_clean":"2024-04-23T14:31:26.821011+0000","last_became_active":"2024-04-23T14:31:25.828620+0000","last_became_peered":"2024-04-23T14:31:25.828620+0000","last_unstale":"2024-04-23T14:31:26.821011+0000","last_undegraded":"2024-04-23T14:31:26.821011+0000","last_fullsized":"2024-04-23T14:31:26.821011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230152+0000","last_change":"2024-04-23T14:31:25.846861+0000","last_active":"2024-04-23T14:31:27.230152+0000","last_peered":"2024-04-23T14:31:27.230152+0000","last_clean":"2024-04-23T14:31:27.230152+0000","last_became_active":"2024-04-23T14:31:25.846267+0000","last_became_peered":"2024-04-23T14:31:25.846267+0000","last_unstale":"2024-04-23T14:31:27.230152+0000","last_undegraded":"2024-04-23T14:31:27.230152+0000","last_fullsized":"2024-04-23T14:31:27.230152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227598+0000","last_change":"2024-04-23T14:31:24.816507+0000","last_active":"2024-04-23T14:31:27.227598+0000","last_peered":"2024-04-23T14:31:27.227598+0000","last_clean":"2024-04-23T14:31:27.227598+0000","last_became_active":"2024-04-23T14:31:24.816159+0000","last_became_peered":"2024-04-23T14:31:24.816159+0000","last_unstale":"2024-04-23T14:31:27.227598+0000","last_undegraded":"2024-04-23T14:31:27.227598+0000","last_fullsized":"2024-04-23T14:31:27.227598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229590+0000","last_change":"2024-04-23T14:31:25.846016+0000","last_active":"2024-04-23T14:31:27.229590+0000","last_peered":"2024-04-23T14:31:27.229590+0000","last_clean":"2024-04-23T14:31:27.229590+0000","last_became_active":"2024-04-23T14:31:25.845682+0000","last_became_peered":"2024-04-23T14:31:25.845682+0000","last_unstale":"2024-04-23T14:31:27.229590+0000","last_undegraded":"2024-04-23T14:31:27.229590+0000","last_fullsized":"2024-04-23T14:31:27.229590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229170+0000","last_change":"2024-04-23T14:31:24.819529+0000","last_active":"2024-04-23T14:31:27.229170+0000","last_peered":"2024-04-23T14:31:27.229170+0000","last_clean":"2024-04-23T14:31:27.229170+0000","last_became_active":"2024-04-23T14:31:24.819162+0000","last_became_peered":"2024-04-23T14:31:24.819162+0000","last_unstale":"2024-04-23T14:31:27.229170+0000","last_undegraded":"2024-04-23T14:31:27.229170+0000","last_fullsized":"2024-04-23T14:31:27.229170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225173+0000","last_change":"2024-04-23T14:31:25.837928+0000","last_active":"2024-04-23T14:31:27.225173+0000","last_peered":"2024-04-23T14:31:27.225173+0000","last_clean":"2024-04-23T14:31:27.225173+0000","last_became_active":"2024-04-23T14:31:25.837281+0000","last_became_peered":"2024-04-23T14:31:25.837281+0000","last_unstale":"2024-04-23T14:31:27.225173+0000","last_undegraded":"2024-04-23T14:31:27.225173+0000","last_fullsized":"2024-04-23T14:31:27.225173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:59:15.348815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224998+0000","last_change":"2024-04-23T14:31:24.793060+0000","last_active":"2024-04-23T14:31:27.224998+0000","last_peered":"2024-04-23T14:31:27.224998+0000","last_clean":"2024-04-23T14:31:27.224998+0000","last_became_active":"2024-04-23T14:31:24.792763+0000","last_became_peered":"2024-04-23T14:31:24.792763+0000","last_unstale":"2024-04-23T14:31:27.224998+0000","last_undegraded":"2024-04-23T14:31:27.224998+0000","last_fullsized":"2024-04-23T14:31:27.224998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224786+0000","last_change":"2024-04-23T14:31:25.824548+0000","last_active":"2024-04-23T14:31:27.224786+0000","last_peered":"2024-04-23T14:31:27.224786+0000","last_clean":"2024-04-23T14:31:27.224786+0000","last_became_active":"2024-04-23T14:31:25.824207+0000","last_became_peered":"2024-04-23T14:31:25.824207+0000","last_unstale":"2024-04-23T14:31:27.224786+0000","last_undegraded":"2024-04-23T14:31:27.224786+0000","last_fullsized":"2024-04-23T14:31:27.224786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225623+0000","last_change":"2024-04-23T14:31:24.810861+0000","last_active":"2024-04-23T14:31:27.225623+0000","last_peered":"2024-04-23T14:31:27.225623+0000","last_clean":"2024-04-23T14:31:27.225623+0000","last_became_active":"2024-04-23T14:31:24.810466+0000","last_became_peered":"2024-04-23T14:31:24.810466+0000","last_unstale":"2024-04-23T14:31:27.225623+0000","last_undegraded":"2024-04-23T14:31:27.225623+0000","last_fullsized":"2024-04-23T14:31:27.225623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:31:31.098 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:31:31.098 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T14:31:31.098 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-23T14:31:31.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-23T14:31:31.099 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-23T14:31:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.350+0000 7fb2cbe00640 1 Processor -- start 2024-04-23T14:31:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.350+0000 7fb2cbe00640 1 -- start start 2024-04-23T14:31:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.352+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc0cd050 0x7fb2cc06f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.352+0000 7fb2cbe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc15f160 con 0x7fb2cc15ece0 2024-04-23T14:31:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.352+0000 7fb2cbe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc15f430 con 0x7fb2cc0b3080 2024-04-23T14:31:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.352+0000 7fb2cbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc0ced20 con 0x7fb2cc0cd050 2024-04-23T14:31:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.352+0000 7fb2c3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb2cc15ece0 0x7fb2cc0cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36448/0 (socket says 172.21.15.59:36448) 2024-04-23T14:31:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.352+0000 7fb2c3e00640 1 -- 172.21.15.59:0/2922985367 learned_addr learned my addr 172.21.15.59:0/2922985367 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.352+0000 7fb2caa00640 1 --2- 172.21.15.59:0/2922985367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc0cd050 0x7fb2cc06f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.353+0000 7fb2ca000640 1 -- 172.21.15.59:0/2922985367 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1019752455 0 0) 0x7fb2cc15f160 con 0x7fb2cc15ece0 2024-04-23T14:31:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.353+0000 7fb2ca000640 1 -- 172.21.15.59:0/2922985367 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2b0003520 con 0x7fb2cc15ece0 2024-04-23T14:31:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.353+0000 7fb2caa00640 1 -- 172.21.15.59:0/2922985367 >> v1:172.21.15.76:6789/0 conn(0x7fb2cc0b3080 legacy=0x7fb2cc0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.353+0000 7fb2ca000640 1 -- 172.21.15.59:0/2922985367 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2451591530 0 0) 0x7fb2cc15f430 con 0x7fb2cc0b3080 2024-04-23T14:31:31.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.353+0000 7fb2caa00640 1 -- 172.21.15.59:0/2922985367 >> v1:172.21.15.59:6789/0 conn(0x7fb2cc15ece0 legacy=0x7fb2cc0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.353+0000 7fb2caa00640 1 -- 172.21.15.59:0/2922985367 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc15f710 con 0x7fb2cc0cd050 2024-04-23T14:31:31.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.355+0000 7fb2caa00640 1 --2- 172.21.15.59:0/2922985367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc0cd050 0x7fb2cc06f620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0097d0 tx=0x7fb2bc02a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a3f637f26740d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:31.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.355+0000 7fb2ca000640 1 -- 172.21.15.59:0/2922985367 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc037070 con 0x7fb2cc0cd050 2024-04-23T14:31:31.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.355+0000 7fb2ca000640 1 -- 172.21.15.59:0/2922985367 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc02ab70 con 0x7fb2cc0cd050 2024-04-23T14:31:31.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.355+0000 7fb2ca000640 1 -- 172.21.15.59:0/2922985367 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4001670 con 0x7fb2cc0cd050 2024-04-23T14:31:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.356+0000 7fb2cbe00640 1 -- 172.21.15.59:0/2922985367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc0cd050 msgr2=0x7fb2cc06f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.356+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/2922985367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc0cd050 0x7fb2cc06f620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0097d0 tx=0x7fb2bc02a0e0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.356+0000 7fb2cbe00640 1 -- 172.21.15.59:0/2922985367 shutdown_connections 2024-04-23T14:31:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.356+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/2922985367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc0cd050 0x7fb2cc06f620 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.356+0000 7fb2cbe00640 1 -- 172.21.15.59:0/2922985367 >> 172.21.15.59:0/2922985367 conn(0x7fb2cc0b21a0 msgr2=0x7fb2cc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.356+0000 7fb2cbe00640 1 -- 172.21.15.59:0/2922985367 shutdown_connections 2024-04-23T14:31:31.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.356+0000 7fb2cbe00640 1 -- 172.21.15.59:0/2922985367 wait complete. 2024-04-23T14:31:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.358+0000 7fb2cbe00640 1 Processor -- start 2024-04-23T14:31:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.358+0000 7fb2cbe00640 1 -- start start 2024-04-23T14:31:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.359+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2cc0b3080 0x7fb2cc157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.359+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2cc0cd050 0x7fb2cc0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.359+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc15ece0 0x7fb2cc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2cbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc0cef50 con 0x7fb2cc0cd050 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2cbe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc15fcd0 con 0x7fb2cc0b3080 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2cbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc159110 con 0x7fb2cc15ece0 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2cc0cd050 0x7fb2cc0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2cc0cd050 0x7fb2cc0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33120/0 (socket says 172.21.15.59:33120) 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 -- 172.21.15.59:0/1437359362 learned_addr learned my addr 172.21.15.59:0/1437359362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2caa00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2cc0b3080 0x7fb2cc157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2cb400640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc15ece0 0x7fb2cc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 -- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc15ece0 msgr2=0x7fb2cc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc15ece0 0x7fb2cc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 -- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2cc0b3080 msgr2=0x7fb2cc157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2cc0b3080 0x7fb2cc157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2c3e00640 1 -- 172.21.15.59:0/1437359362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc0cbb50 con 0x7fb2cc0cd050 2024-04-23T14:31:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.360+0000 7fb2caa00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2cc0b3080 0x7fb2cc157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.361+0000 7fb2c3e00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2cc0cd050 0x7fb2cc0cb300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c7d0 tx=0x7fb2b400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.361+0000 7fb2c2a00640 1 -- 172.21.15.59:0/1437359362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4018070 con 0x7fb2cc0cd050 2024-04-23T14:31:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.361+0000 7fb2c2a00640 1 -- 172.21.15.59:0/1437359362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b40074e0 con 0x7fb2cc0cd050 2024-04-23T14:31:31.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.361+0000 7fb2c2a00640 1 -- 172.21.15.59:0/1437359362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4004a20 con 0x7fb2cc0cd050 2024-04-23T14:31:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.361+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc0cbe30 con 0x7fb2cc0cd050 2024-04-23T14:31:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.361+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc0cc3f0 con 0x7fb2cc0cd050 2024-04-23T14:31:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.362+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc04fd90 con 0x7fb2cc0cd050 2024-04-23T14:31:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.363+0000 7fb2c2a00640 1 -- 172.21.15.59:0/1437359362 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2b401b070 con 0x7fb2cc0cd050 2024-04-23T14:31:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.364+0000 7fb2c2a00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb2a806c940 0x7fb2a806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.365+0000 7fb2c2a00640 1 -- 172.21.15.59:0/1437359362 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2b408b210 con 0x7fb2cc0cd050 2024-04-23T14:31:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.365+0000 7fb2caa00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb2a806c940 0x7fb2a806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.365+0000 7fb2caa00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb2a806c940 0x7fb2a806ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc034000 tx=0x7fb2bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:31.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.369+0000 7fb2c2a00640 1 -- 172.21.15.59:0/1437359362 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2b40596a0 con 0x7fb2cc0cd050 2024-04-23T14:31:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.624+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb2cc0cd5d0 con 0x7fb2cc0cd050 2024-04-23T14:31:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.629+0000 7fb2c2a00640 1 -- 172.21.15.59:0/1437359362 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174134 (secure 0 0 0) 0x7fb2b4058fa0 con 0x7fb2cc0cd050 2024-04-23T14:31:31.632 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:31.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb2a806c940 msgr2=0x7fb2a806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb2a806c940 0x7fb2a806ee00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc034000 tx=0x7fb2bc02e000 comp rx=0 tx=0).stop 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2cc0cd050 msgr2=0x7fb2cc0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2cc0cd050 0x7fb2cc0cb300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c7d0 tx=0x7fb2b400cc90 comp rx=0 tx=0).stop 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 shutdown_connections 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb2a806c940 0x7fb2a806ee00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2cc15ece0 0x7fb2cc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2cc0cd050 0x7fb2cc0cb300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 --2- 172.21.15.59:0/1437359362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2cc0b3080 0x7fb2cc157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 >> 172.21.15.59:0/1437359362 conn(0x7fb2cc0b21a0 msgr2=0x7fb2cc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 shutdown_connections 2024-04-23T14:31:31.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.633+0000 7fb2cbe00640 1 -- 172.21.15.59:0/1437359362 wait complete. 2024-04-23T14:31:31.653 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3176066673},{"type":"v1","addr":"172.21.15.59:6833","nonce":3176066673}]},"active_addr":"172.21.15.59:6833/3176066673","active_change":"2024-04-23T14:31:06.319565+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":100923761}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":1599165205}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":54900843}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3965495262}]}]} 2024-04-23T14:31:31.659 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-23T14:31:31.659 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-23T14:31:31.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:31.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.870+0000 7fd21f400640 1 Processor -- start 2024-04-23T14:31:31.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.870+0000 7fd21f400640 1 -- start start 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.871+0000 7fd21f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd21815e750 0x7fd2180ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.871+0000 7fd21f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd21815eb30 0x7fd2180cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.871+0000 7fd21f400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2180cfa20 0x7fd2180d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.871+0000 7fd21f400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd2180b8fc0 con 0x7fd21815eb30 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.871+0000 7fd21f400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd2180b92a0 con 0x7fd21815e750 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.871+0000 7fd21f400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2180b9580 con 0x7fd2180cfa20 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd21815eb30 0x7fd2180cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd21815eb30 0x7fd2180cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33136/0 (socket says 172.21.15.59:33136) 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 -- 172.21.15.59:0/806025340 learned_addr learned my addr 172.21.15.59:0/806025340 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:31.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd21ca00640 1 --2- 172.21.15.59:0/806025340 >> v2:172.21.15.76:3300/0 conn(0x7fd21815e750 0x7fd2180ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd21d400640 1 --2- 172.21.15.59:0/806025340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2180cfa20 0x7fd2180d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 -- 172.21.15.59:0/806025340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2180cfa20 msgr2=0x7fd2180d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:31.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 --2- 172.21.15.59:0/806025340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2180cfa20 0x7fd2180d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 -- 172.21.15.59:0/806025340 >> v2:172.21.15.76:3300/0 conn(0x7fd21815e750 msgr2=0x7fd2180ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:31.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 --2- 172.21.15.59:0/806025340 >> v2:172.21.15.76:3300/0 conn(0x7fd21815e750 0x7fd2180ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.872+0000 7fd217e00640 1 -- 172.21.15.59:0/806025340 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2180b9860 con 0x7fd21815eb30 2024-04-23T14:31:31.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.873+0000 7fd217e00640 1 --2- 172.21.15.59:0/806025340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd2180cf4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2180d11f0 tx=0x7fd20002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2607a591e7f03c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:31.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.874+0000 7fd217400640 1 -- 172.21.15.59:0/806025340 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200037070 con 0x7fd21815eb30 2024-04-23T14:31:31.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.874+0000 7fd217400640 1 -- 172.21.15.59:0/806025340 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd200032440 con 0x7fd21815eb30 2024-04-23T14:31:31.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.874+0000 7fd217400640 1 -- 172.21.15.59:0/806025340 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200032810 con 0x7fd21815eb30 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.874+0000 7fd21f400640 1 -- 172.21.15.59:0/806025340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 msgr2=0x7fd2180cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.874+0000 7fd21f400640 1 --2- 172.21.15.59:0/806025340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd2180cf4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2180d11f0 tx=0x7fd20002a1e0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.875+0000 7fd21f400640 1 -- 172.21.15.59:0/806025340 shutdown_connections 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.875+0000 7fd21f400640 1 --2- 172.21.15.59:0/806025340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2180cfa20 0x7fd2180d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.875+0000 7fd21f400640 1 --2- 172.21.15.59:0/806025340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd2180cf4e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.875+0000 7fd21f400640 1 --2- 172.21.15.59:0/806025340 >> v2:172.21.15.76:3300/0 conn(0x7fd21815e750 0x7fd2180ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.875+0000 7fd21f400640 1 -- 172.21.15.59:0/806025340 >> 172.21.15.59:0/806025340 conn(0x7fd218067b50 msgr2=0x7fd218067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:31.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.875+0000 7fd21f400640 1 -- 172.21.15.59:0/806025340 shutdown_connections 2024-04-23T14:31:31.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.875+0000 7fd21f400640 1 -- 172.21.15.59:0/806025340 wait complete. 2024-04-23T14:31:31.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.876+0000 7fd21f400640 1 Processor -- start 2024-04-23T14:31:31.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.877+0000 7fd21f400640 1 -- start start 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.877+0000 7fd21f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2180cfa20 0x7fd21815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21f400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd21815e750 0x7fd218152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd218161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21f400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180d31b0 con 0x7fd21815eb30 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21f400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180ba6e0 con 0x7fd2180cfa20 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21f400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2180d3330 con 0x7fd21815e750 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd218161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd218161770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33152/0 (socket says 172.21.15.59:33152) 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 -- 172.21.15.59:0/3089850913 learned_addr learned my addr 172.21.15.59:0/3089850913 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21ca00640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2180cfa20 0x7fd21815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd217e00640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd21815e750 0x7fd218152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 -- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd21815e750 msgr2=0x7fd218152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd21815e750 0x7fd218152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 -- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2180cfa20 msgr2=0x7fd21815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2180cfa20 0x7fd21815fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd21d400640 1 -- 172.21.15.59:0/3089850913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2181601a0 con 0x7fd21815eb30 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.878+0000 7fd217e00640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd21815e750 0x7fd218152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.879+0000 7fd21d400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd218161770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd210007e40 tx=0x7fd21000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.879+0000 7fd214c00640 1 -- 172.21.15.59:0/3089850913 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210014620 con 0x7fd21815eb30 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.879+0000 7fd214c00640 1 -- 172.21.15.59:0/3089850913 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd210019070 con 0x7fd21815eb30 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.879+0000 7fd214c00640 1 -- 172.21.15.59:0/3089850913 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210014a20 con 0x7fd21815eb30 2024-04-23T14:31:31.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.879+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd218160480 con 0x7fd21815eb30 2024-04-23T14:31:31.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.879+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd218160a40 con 0x7fd21815eb30 2024-04-23T14:31:31.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.880+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd218068a80 con 0x7fd21815eb30 2024-04-23T14:31:31.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.881+0000 7fd214c00640 1 -- 172.21.15.59:0/3089850913 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd210014bc0 con 0x7fd21815eb30 2024-04-23T14:31:31.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.882+0000 7fd214c00640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f406c940 0x7fd1f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:31.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.882+0000 7fd214c00640 1 -- 172.21.15.59:0/3089850913 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd21008c3f0 con 0x7fd21815eb30 2024-04-23T14:31:31.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.883+0000 7fd21ca00640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f406c940 0x7fd1f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:31.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.883+0000 7fd21ca00640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f406c940 0x7fd1f406ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd208009a50 tx=0x7fd208008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:31.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:31.887+0000 7fd214c00640 1 -- 172.21.15.59:0/3089850913 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd21005a890 con 0x7fd21815eb30 2024-04-23T14:31:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.097+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2180b7f80 con 0x7fd21815eb30 2024-04-23T14:31:32.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.101+0000 7fd214c00640 1 -- 172.21.15.59:0/3089850913 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fd21005a190 con 0x7fd21815eb30 2024-04-23T14:31:32.102 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:32.103 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:32.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f406c940 msgr2=0x7fd1f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f406c940 0x7fd1f406ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd208009a50 tx=0x7fd208008040 comp rx=0 tx=0).stop 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 msgr2=0x7fd218161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd218161770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd210007e40 tx=0x7fd21000e4b0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 shutdown_connections 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f406c940 0x7fd1f406ee00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd21815e750 0x7fd218152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21815eb30 0x7fd218161770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 --2- 172.21.15.59:0/3089850913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2180cfa20 0x7fd21815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 >> 172.21.15.59:0/3089850913 conn(0x7fd218067b50 msgr2=0x7fd21815cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.103+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 shutdown_connections 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.104+0000 7fd21f400640 1 -- 172.21.15.59:0/3089850913 wait complete. 2024-04-23T14:31:32.125 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-23T14:31:32.125 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:32.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.331+0000 7f0c06000640 1 Processor -- start 2024-04-23T14:31:32.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.331+0000 7f0c06000640 1 -- start start 2024-04-23T14:31:32.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.332+0000 7f0c06000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c000d14a0 0x7f0c000b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.332+0000 7f0c06000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c000cf030 con 0x7f0c0015e7c0 2024-04-23T14:31:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.333+0000 7f0c06000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c000b6a40 con 0x7f0c000b5ff0 2024-04-23T14:31:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.333+0000 7f0c06000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000b7000 con 0x7f0c000d14a0 2024-04-23T14:31:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.333+0000 7f0c04c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0c0015e7c0 0x7f0c000d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36460/0 (socket says 172.21.15.59:36460) 2024-04-23T14:31:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.333+0000 7f0c04c00640 1 -- 172.21.15.59:0/1143281547 learned_addr learned my addr 172.21.15.59:0/1143281547 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.333+0000 7f0c05600640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c000d14a0 0x7f0c000b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.333+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2090696318 0 0) 0x7f0bf0001110 con 0x7f0c0015e7c0 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.333+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be4003520 con 0x7f0c0015e7c0 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1666200679 0 0) 0x7f0be4003520 con 0x7f0c0015e7c0 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c000d14a0 msgr2=0x7f0c000b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c000d14a0 0x7f0c000b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 >> v1:172.21.15.76:6789/0 conn(0x7f0c000b5ff0 legacy=0x7f0c000d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c000b6d20 con 0x7f0c0015e7c0 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f0bf0001840 con 0x7f0c0015e7c0 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 >> v1:172.21.15.59:6789/0 conn(0x7f0c0015e7c0 legacy=0x7f0c000d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0be4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c000d14a0 0x7f0be4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0be4004d40 0x7f0be40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000b7000 con 0x7f0be40039e0 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0be400bbd0 con 0x7f0be4004d40 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0be400bdd0 con 0x7f0c000d14a0 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0c04c00640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0be4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0c05600640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0be4004d40 0x7f0be40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0c04c00640 1 -- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c000d14a0 msgr2=0x7f0be4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0c04c00640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c000d14a0 0x7f0be4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0c04c00640 1 -- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0be4004d40 msgr2=0x7f0be40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0c04c00640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0be4004d40 0x7f0be40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.334+0000 7f0c04c00640 1 -- 172.21.15.59:0/1143281547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be4003520 con 0x7f0be40039e0 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.335+0000 7f0c04c00640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0be4003e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0007fd0 tx=0x7f0be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9c67e5fa62807f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0007e30 con 0x7f0be40039e0 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf0005af0 con 0x7f0be40039e0 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0bff400640 1 -- 172.21.15.59:0/1143281547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0007070 con 0x7f0be40039e0 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0c06000640 1 -- 172.21.15.59:0/1143281547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 msgr2=0x7f0be4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0c06000640 1 --2- 172.21.15.59:0/1143281547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0be4003e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0007fd0 tx=0x7f0be4005830 comp rx=0 tx=0).stop 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0c05600640 1 -- 172.21.15.59:0/1143281547 reap_dead start 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0c06000640 1 -- 172.21.15.59:0/1143281547 shutdown_connections 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.336+0000 7f0c06000640 1 -- 172.21.15.59:0/1143281547 >> 172.21.15.59:0/1143281547 conn(0x7f0c0009d920 msgr2=0x7f0c0008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.337+0000 7f0c06000640 1 -- 172.21.15.59:0/1143281547 shutdown_connections 2024-04-23T14:31:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.337+0000 7f0c06000640 1 -- 172.21.15.59:0/1143281547 wait complete. 2024-04-23T14:31:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.338+0000 7f0c06000640 1 Processor -- start 2024-04-23T14:31:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.338+0000 7f0c06000640 1 -- start start 2024-04-23T14:31:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.339+0000 7f0c06000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0c0014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.339+0000 7f0c06000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c001e8040 0x7f0c001e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.339+0000 7f0c06000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c000b1ab0 0x7f0c000b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.339+0000 7f0c06000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000d2800 con 0x7f0be40039e0 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.339+0000 7f0c06000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000cf620 con 0x7f0c000b1ab0 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.339+0000 7f0c06000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0c000d1a80 con 0x7f0c001e8040 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0c0014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0c0014eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33180/0 (socket says 172.21.15.59:33180) 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 -- 172.21.15.59:0/4124800231 learned_addr learned my addr 172.21.15.59:0/4124800231 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c05600640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c000b1ab0 0x7f0c000b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 -- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c001e8040 msgr2=0x7f0c001e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0bffe00640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c001e8040 0x7f0c001e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c001e8040 0x7f0c001e84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 -- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c000b1ab0 msgr2=0x7f0c000b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c000b1ab0 0x7f0c000b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c04c00640 1 -- 172.21.15.59:0/4124800231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c000b28e0 con 0x7f0be40039e0 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.340+0000 7f0c05600640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c000b1ab0 0x7f0c000b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.341+0000 7f0c04c00640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0c0014eaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bf002af20 tx=0x7f0bf0001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.341+0000 7f0bfcc00640 1 -- 172.21.15.59:0/4124800231 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0007270 con 0x7f0be40039e0 2024-04-23T14:31:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.341+0000 7f0bfcc00640 1 -- 172.21.15.59:0/4124800231 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf0007410 con 0x7f0be40039e0 2024-04-23T14:31:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.341+0000 7f0bfcc00640 1 -- 172.21.15.59:0/4124800231 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf003c430 con 0x7f0be40039e0 2024-04-23T14:31:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.341+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001492c0 con 0x7f0be40039e0 2024-04-23T14:31:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.341+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c000b2470 con 0x7f0be40039e0 2024-04-23T14:31:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.342+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c000d2540 con 0x7f0be40039e0 2024-04-23T14:31:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.343+0000 7f0bfcc00640 1 -- 172.21.15.59:0/4124800231 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bf003c5d0 con 0x7f0be40039e0 2024-04-23T14:31:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.344+0000 7f0bfcc00640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.344+0000 7f0bfcc00640 1 -- 172.21.15.59:0/4124800231 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0bf00ace90 con 0x7f0be40039e0 2024-04-23T14:31:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.344+0000 7f0bffe00640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.345+0000 7f0bffe00640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c0014f850 tx=0x7f0be8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.348+0000 7f0bfcc00640 1 -- 172.21.15.59:0/4124800231 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c000d2540 con 0x7f0be40039e0 2024-04-23T14:31:32.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.556+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c0004fd90 con 0x7f0be40039e0 2024-04-23T14:31:32.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.559+0000 7f0bfcc00640 1 -- 172.21.15.59:0/4124800231 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f0bf0040030 con 0x7f0be40039e0 2024-04-23T14:31:32.561 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:32.561 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:32.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bdc06c8f0 msgr2=0x7f0bdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c0014f850 tx=0x7f0be8009040 comp rx=0 tx=0).stop 2024-04-23T14:31:32.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 msgr2=0x7f0c0014eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0c0014eaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bf002af20 tx=0x7f0bf0001b40 comp rx=0 tx=0).stop 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 shutdown_connections 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bdc06c8f0 0x7f0bdc06edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c001e8040 0x7f0c001e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c000b1ab0 0x7f0c000b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 --2- 172.21.15.59:0/4124800231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be40039e0 0x7f0c0014eaf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 >> 172.21.15.59:0/4124800231 conn(0x7f0c0009d920 msgr2=0x7f0c0008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 shutdown_connections 2024-04-23T14:31:32.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.562+0000 7f0c06000640 1 -- 172.21.15.59:0/4124800231 wait complete. 2024-04-23T14:31:32.583 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-23T14:31:32.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-23T14:31:32.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-23T14:31:32.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-23T14:31:32.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-23T14:31:32.586 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-23T14:31:32.586 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-23T14:31:32.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-23T14:31:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.894+0000 7f8b2aa00640 1 Processor -- start 2024-04-23T14:31:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.894+0000 7f8b2aa00640 1 -- start start 2024-04-23T14:31:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.897+0000 7f8b2aa00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0e3040 0x7f8b1c03c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.898+0000 7f8b2aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0e78d0 0x7f8b1c053d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.898+0000 7f8b2aa00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b1c058980 con 0x7f8b1c0e3420 2024-04-23T14:31:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.898+0000 7f7475e00640 1 Processor -- start 2024-04-23T14:31:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.898+0000 7f8b28c00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8b1c0e3420 0x7f8b1c058130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43442/0 (socket says 172.21.15.59:43442) 2024-04-23T14:31:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f8b2aa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1c006860 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f8b2aa00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1c058c60 con 0x7f8b1c0e3040 2024-04-23T14:31:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.898+0000 7f8b29600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0e3040 0x7f8b1c03c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f8b28c00640 1 -- 172.21.15.59:0/3927223726 learned_addr learned my addr 172.21.15.59:0/3927223726 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f8cf7e00640 1 Processor -- start 2024-04-23T14:31:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f8cf7e00640 1 -- start start 2024-04-23T14:31:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f8b2a000640 1 --2- 172.21.15.59:0/3927223726 >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0e78d0 0x7f8b1c053d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.898+0000 7f7475e00640 1 -- start start 2024-04-23T14:31:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f0b43600640 1 Processor -- start 2024-04-23T14:31:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.899+0000 7f8b23e00640 1 -- 172.21.15.59:0/3927223726 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1618712195 0 0) 0x7f8b1c058980 con 0x7f8b1c0e3420 2024-04-23T14:31:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.900+0000 7f8b23e00640 1 -- 172.21.15.59:0/3927223726 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b0c003520 con 0x7f8b1c0e3420 2024-04-23T14:31:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.900+0000 7f8b29600640 1 -- 172.21.15.59:0/3927223726 >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0e78d0 msgr2=0x7f8b1c053d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.900+0000 7f8b29600640 1 --2- 172.21.15.59:0/3927223726 >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0e78d0 0x7f8b1c053d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.900+0000 7f8b29600640 1 -- 172.21.15.59:0/3927223726 >> v1:172.21.15.76:6789/0 conn(0x7f8b1c0e3420 legacy=0x7f8b1c058130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.900+0000 7f8b29600640 1 -- 172.21.15.59:0/3927223726 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c006210 con 0x7f8b1c0e3040 2024-04-23T14:31:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.900+0000 7f0b43600640 1 -- start start 2024-04-23T14:31:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.901+0000 7f8cf7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8cf80d4bc0 0x7f8cf80b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.901+0000 7f8b29600640 1 --2- 172.21.15.59:0/3927223726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3040 0x7f8b1c03c180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b24065520 tx=0x7f8b240a5c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e851664ecee8927 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8cf7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8cf80d4bc0 0x7f8cf80b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8cf7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8cf80d4bc0 0x7f8cf80b7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33200/0 (socket says 172.21.15.59:33200) 2024-04-23T14:31:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8cf6000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8cf8162640 0x7f8cf80d44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36476/0 (socket says 172.21.15.59:36476) 2024-04-23T14:31:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f0b43600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b3c0b34a0 0x7f0b3c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8cf7400640 1 -- 172.21.15.59:0/1300933488 learned_addr learned my addr 172.21.15.59:0/1300933488 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8b23e00640 1 -- 172.21.15.59:0/3927223726 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240b3070 con 0x7f8b1c0e3040 2024-04-23T14:31:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8b23e00640 1 -- 172.21.15.59:0/3927223726 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240a6aa0 con 0x7f8b1c0e3040 2024-04-23T14:31:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8b23e00640 1 -- 172.21.15.59:0/3927223726 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240ae440 con 0x7f8b1c0e3040 2024-04-23T14:31:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f0b43600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b3c15b860 con 0x7f0b3c06c4f0 2024-04-23T14:31:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f0b43600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b3c15bb40 con 0x7f0b3c15b0c0 2024-04-23T14:31:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f0b43600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3c06fcc0 con 0x7f0b3c0b34a0 2024-04-23T14:31:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f0b40c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0b3c06c4f0 0x7f0b3c15eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36490/0 (socket says 172.21.15.59:36490) 2024-04-23T14:31:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f0b40c00640 1 -- 172.21.15.59:0/331193354 learned_addr learned my addr 172.21.15.59:0/331193354 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:32.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f0b3be00640 1 --2- 172.21.15.59:0/331193354 >> v2:172.21.15.76:3300/0 conn(0x7f0b3c0b34a0 0x7f0b3c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3927223726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3040 msgr2=0x7f8b1c03c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f8b2aa00640 1 --2- 172.21.15.59:0/3927223726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3040 0x7f8b1c03c180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b24065520 tx=0x7f8b240a5c30 comp rx=0 tx=0).stop 2024-04-23T14:31:32.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.902+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf8162c20 con 0x7f8cf8162640 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf8162f00 con 0x7f8cf815ea50 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf80cf260 con 0x7f8cf80d4bc0 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f8cf7400640 1 -- 172.21.15.59:0/1300933488 >> v1:172.21.15.76:6789/0 conn(0x7f8cf815ea50 legacy=0x7f8cf80b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f8cf7400640 1 -- 172.21.15.59:0/1300933488 >> v1:172.21.15.59:6789/0 conn(0x7f8cf8162640 legacy=0x7f8cf80d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f8cf7400640 1 -- 172.21.15.59:0/1300933488 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf81631e0 con 0x7f8cf80d4bc0 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.903+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169267488 0 0) 0x7f0b3c15bb40 con 0x7f0b3c15b0c0 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b24003440 con 0x7f0b3c15b0c0 2024-04-23T14:31:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930161145 0 0) 0x7f0b3c15b860 con 0x7f0b3c06c4f0 2024-04-23T14:31:32.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b24003970 con 0x7f0b3c06c4f0 2024-04-23T14:31:32.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3927223726 shutdown_connections 2024-04-23T14:31:32.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f8b2aa00640 1 --2- 172.21.15.59:0/3927223726 >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0e78d0 0x7f8b1c053d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f8b2aa00640 1 --2- 172.21.15.59:0/3927223726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3040 0x7f8b1c03c180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3927223726 >> 172.21.15.59:0/3927223726 conn(0x7f8b1c03aa40 msgr2=0x7f8b1c03ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3927223726 shutdown_connections 2024-04-23T14:31:32.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3241414420 0 0) 0x7f0b24003970 con 0x7f0b3c06c4f0 2024-04-23T14:31:32.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f7475e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f74680e3300 0x7f74680572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f7475e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468053750 0x7f7468053bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 >> v2:172.21.15.76:3300/0 conn(0x7f0b3c0b34a0 msgr2=0x7f0b3c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 --2- 172.21.15.59:0/331193354 >> v2:172.21.15.76:3300/0 conn(0x7f0b3c0b34a0 0x7f0b3c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 >> v1:172.21.15.76:6789/0 conn(0x7f0b3c15b0c0 legacy=0x7f0b3c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c15be20 con 0x7f0b3c06c4f0 2024-04-23T14:31:32.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.904+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f0b280015a0 con 0x7f0b3c06c4f0 2024-04-23T14:31:32.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f7475e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74680e36e0 con 0x7f7468054110 2024-04-23T14:31:32.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f7475e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7468054560 con 0x7f74680e3300 2024-04-23T14:31:32.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f7475e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7468057b40 con 0x7f7468053750 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 >> v1:172.21.15.59:6789/0 conn(0x7f0b3c06c4f0 legacy=0x7f0b3c15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3b400640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24003ea0 0x7f0b24008850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3b400640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b24008d90 0x7f0b2400b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3b400640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b2400b860 0x7f0b2400dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2400e5b0 con 0x7f0b24008d90 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2400e760 con 0x7f0b24003ea0 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2400e960 con 0x7f0b2400b860 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f8cf7400640 1 --2- 172.21.15.59:0/1300933488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 0x7f8cf80b7410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009e50 tx=0x7f8cf002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d01285c79ff6cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f746fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468053750 0x7f7468053bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f8cf5600640 1 -- 172.21.15.59:0/1300933488 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0036070 con 0x7f8cf80d4bc0 2024-04-23T14:31:32.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f0b3be00640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b24008d90 0x7f0b2400b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f0b40c00640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24003ea0 0x7f0b24008850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f8cf5600640 1 -- 172.21.15.59:0/1300933488 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf0031600 con 0x7f8cf80d4bc0 2024-04-23T14:31:32.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f8cf5600640 1 -- 172.21.15.59:0/1300933488 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0036070 con 0x7f8cf80d4bc0 2024-04-23T14:31:32.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f0b3be00640 1 -- 172.21.15.59:0/331193354 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b2400b860 msgr2=0x7f0b2400dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f0b3be00640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b2400b860 0x7f0b2400dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f7475400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7468054110 0x7f74680061f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43484/0 (socket says 172.21.15.59:43484) 2024-04-23T14:31:32.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f0b3be00640 1 -- 172.21.15.59:0/331193354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24003ea0 msgr2=0x7f0b24008850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f0b3be00640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24003ea0 0x7f0b24008850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f0b3be00640 1 -- 172.21.15.59:0/331193354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b24003970 con 0x7f0b24008d90 2024-04-23T14:31:32.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f7475400640 1 -- 172.21.15.59:0/2817309333 learned_addr learned my addr 172.21.15.59:0/2817309333 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:32.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f0b41600640 1 -- 172.21.15.59:0/331193354 reap_dead start 2024-04-23T14:31:32.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.906+0000 7f7474a00640 1 --2- 172.21.15.59:0/2817309333 >> v2:172.21.15.59:3300/0 conn(0x7f74680e3300 0x7f74680572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.907+0000 7f746f400640 1 -- 172.21.15.59:0/2817309333 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811127942 0 0) 0x7f74680e36e0 con 0x7f7468054110 2024-04-23T14:31:32.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.907+0000 7f746f400640 1 -- 172.21.15.59:0/2817309333 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7458003520 con 0x7f7468054110 2024-04-23T14:31:32.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.907+0000 7f7474a00640 1 -- 172.21.15.59:0/2817309333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468053750 msgr2=0x7f7468053bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.907+0000 7f7474a00640 1 --2- 172.21.15.59:0/2817309333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468053750 0x7f7468053bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.907+0000 7f7474a00640 1 -- 172.21.15.59:0/2817309333 >> v1:172.21.15.76:6789/0 conn(0x7f7468054110 legacy=0x7f74680061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.907+0000 7f7474a00640 1 -- 172.21.15.59:0/2817309333 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7468057e20 con 0x7f74680e3300 2024-04-23T14:31:32.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.905+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3927223726 wait complete. 2024-04-23T14:31:32.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.908+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 msgr2=0x7f8cf80b7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.908+0000 7f8cf7e00640 1 --2- 172.21.15.59:0/1300933488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 0x7f8cf80b7410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009e50 tx=0x7f8cf002a6d0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.908+0000 7f0b3be00640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b24008d90 0x7f0b2400b180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000ba30 tx=0x7f0b24003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fd0f9e4827b41e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.908+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30030070 con 0x7f0b24008d90 2024-04-23T14:31:32.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.908+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3002add0 con 0x7f0b24008d90 2024-04-23T14:31:32.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.908+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 shutdown_connections 2024-04-23T14:31:32.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.908+0000 7f8cf7e00640 1 --2- 172.21.15.59:0/1300933488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 0x7f8cf80b7410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.909+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 >> 172.21.15.59:0/1300933488 conn(0x7f8cf809d950 msgr2=0x7f8cf808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.909+0000 7f0b3b400640 1 -- 172.21.15.59:0/331193354 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30030070 con 0x7f0b24008d90 2024-04-23T14:31:32.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.909+0000 7f4607000640 1 Processor -- start 2024-04-23T14:31:32.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.909+0000 7f0b43600640 1 -- 172.21.15.59:0/331193354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b24008d90 msgr2=0x7f0b2400b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.909+0000 7f0b43600640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b24008d90 0x7f0b2400b180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000ba30 tx=0x7f0b24003730 comp rx=0 tx=0).stop 2024-04-23T14:31:32.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f0b43600640 1 -- 172.21.15.59:0/331193354 shutdown_connections 2024-04-23T14:31:32.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f7474a00640 1 --2- 172.21.15.59:0/2817309333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f74680572f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74640097a0 tx=0x7f746402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7426cb3d6d5ff24 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f0b43600640 1 --2- 172.21.15.59:0/331193354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b24008d90 0x7f0b2400b180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f0b43600640 1 -- 172.21.15.59:0/331193354 >> 172.21.15.59:0/331193354 conn(0x7f0b3c06b9d0 msgr2=0x7f0b3c06bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f8b2aa00640 1 Processor -- start 2024-04-23T14:31:32.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 shutdown_connections 2024-04-23T14:31:32.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f506c200640 1 Processor -- start 2024-04-23T14:31:32.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.909+0000 7f4607000640 1 -- start start 2024-04-23T14:31:32.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f0b43600640 1 -- 172.21.15.59:0/331193354 shutdown_connections 2024-04-23T14:31:32.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1300933488 wait complete. 2024-04-23T14:31:32.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f506c200640 1 -- start start 2024-04-23T14:31:32.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.911+0000 7f746f400640 1 -- 172.21.15.59:0/2817309333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464037070 con 0x7f74680e3300 2024-04-23T14:31:32.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f0b43600640 1 -- 172.21.15.59:0/331193354 wait complete. 2024-04-23T14:31:32.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.910+0000 7f8b2aa00640 1 -- start start 2024-04-23T14:31:32.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.911+0000 7f4607000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f460015e9c0 0x7f46000d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.911+0000 7f746f400640 1 -- 172.21.15.59:0/2817309333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746402ae40 con 0x7f74680e3300 2024-04-23T14:31:32.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.911+0000 7f746f400640 1 -- 172.21.15.59:0/2817309333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464032660 con 0x7f74680e3300 2024-04-23T14:31:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f4607000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f46000d2d70 0x7f46000d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506c200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5064071d90 0x7f50640cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506c200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5064072880 0x7f506406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506c200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f506406e040 con 0x7f50640b3430 2024-04-23T14:31:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506c200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f506406ddf0 con 0x7f5064071d90 2024-04-23T14:31:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506c200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f506406e320 con 0x7f5064072880 2024-04-23T14:31:32.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f4607000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f46000d10f0 0x7f46000d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f460015e9c0 0x7f46000d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506a200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5064072880 0x7f506406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506a200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5064072880 0x7f506406ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39460/0 (socket says 172.21.15.59:39460) 2024-04-23T14:31:32.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.912+0000 7f506a200640 1 -- 172.21.15.59:0/1102803547 learned_addr learned my addr 172.21.15.59:0/1102803547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0e3040 0x7f8b1c0529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f0b43600640 1 Processor -- start 2024-04-23T14:31:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f506a200640 1 -- 172.21.15.59:0/1102803547 >> v2:172.21.15.59:3300/0 conn(0x7f5064071d90 msgr2=0x7f50640cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f506a200640 1 --2- 172.21.15.59:0/1102803547 >> v2:172.21.15.59:3300/0 conn(0x7f5064071d90 0x7f50640cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f506a200640 1 -- 172.21.15.59:0/1102803547 >> v1:172.21.15.59:6789/0 conn(0x7f50640b3430 legacy=0x7f5064072170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:31:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3420 0x7f8b1c0e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0e78d0 0x7f8b1c0e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f7475e00640 1 -- 172.21.15.59:0/2817309333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 msgr2=0x7f74680572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f7475e00640 1 --2- 172.21.15.59:0/2817309333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f74680572f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74640097a0 tx=0x7f746402a0f0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f0b43600640 1 -- start start 2024-04-23T14:31:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f506a200640 1 -- 172.21.15.59:0/1102803547 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506406e600 con 0x7f5064072880 2024-04-23T14:31:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b2aa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c054450 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b2aa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0542d0 con 0x7f8b1c0e3420 2024-04-23T14:31:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b2aa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c0545d0 con 0x7f8b1c0e3040 2024-04-23T14:31:32.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b2a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0e78d0 0x7f8b1c0e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b2a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0e78d0 0x7f8b1c0e5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33270/0 (socket says 172.21.15.59:33270) 2024-04-23T14:31:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b2a000640 1 -- 172.21.15.59:0/3182847914 learned_addr learned my addr 172.21.15.59:0/3182847914 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f438b800640 1 Processor -- start 2024-04-23T14:31:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b29600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0e3040 0x7f8b1c0529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8b28c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3420 0x7f8b1c0e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f8cf7e00640 1 Processor -- start 2024-04-23T14:31:32.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f8cf7e00640 1 -- start start 2024-04-23T14:31:32.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8b2a000640 1 -- 172.21.15.59:0/3182847914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0e3040 msgr2=0x7f8b1c0529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8b2a000640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0e3040 0x7f8b1c0529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8b2a000640 1 -- 172.21.15.59:0/3182847914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3420 msgr2=0x7f8b1c0e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8b2a000640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c0e3420 0x7f8b1c0e59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8b2a000640 1 -- 172.21.15.59:0/3182847914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c0d73d0 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f460015e9c0 0x7f46000d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51262/0 (socket says 172.21.15.59:51262) 2024-04-23T14:31:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f45ffe00640 1 -- 172.21.15.59:0/1194099345 learned_addr learned my addr 172.21.15.59:0/1194099345 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.913+0000 7f45ff400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f46000d2d70 0x7f46000d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.914+0000 7f4605000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f46000d10f0 0x7f46000d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f46000d3710 con 0x7f46000d2d70 2024-04-23T14:31:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f46000cf3c0 con 0x7f46000d10f0 2024-04-23T14:31:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f46000cf640 con 0x7f460015e9c0 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f7475e00640 1 -- 172.21.15.59:0/2817309333 shutdown_connections 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f7475e00640 1 --2- 172.21.15.59:0/2817309333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468053750 0x7f7468053bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f7475e00640 1 --2- 172.21.15.59:0/2817309333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f74680572f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f7475e00640 1 -- 172.21.15.59:0/2817309333 >> 172.21.15.59:0/2817309333 conn(0x7f7468058ef0 msgr2=0x7f7468059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 0x7f8cf80ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf815ea50 0x7f8cf80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cf8162640 0x7f8cf81596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8cf7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf815ee30 con 0x7f8cf80d4bc0 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8cf7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf80cf6f0 con 0x7f8cf815ea50 2024-04-23T14:31:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f8cf7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf80ce6f0 con 0x7f8cf8162640 2024-04-23T14:31:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f506a200640 1 --2- 172.21.15.59:0/1102803547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5064072880 0x7f506406ee00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5064075670 tx=0x7f505c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f89964bddae82a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8cf6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 0x7f8cf80ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8b2a000640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0e78d0 0x7f8b1c0e5ef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d600 tx=0x7f8b1000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8cf6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 0x7f8cf80ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33278/0 (socket says 172.21.15.59:33278) 2024-04-23T14:31:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8cf6a00640 1 -- 172.21.15.59:0/1286220451 learned_addr learned my addr 172.21.15.59:0/1286220451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f7475e00640 1 -- 172.21.15.59:0/2817309333 shutdown_connections 2024-04-23T14:31:32.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f5063e00640 1 -- 172.21.15.59:0/1102803547 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c037070 con 0x7f5064072880 2024-04-23T14:31:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100125f0 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b10019070 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100129f0 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f5063e00640 1 -- 172.21.15.59:0/1102803547 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c032440 con 0x7f5064072880 2024-04-23T14:31:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8cf7400640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cf8162640 0x7f8cf81596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8cf6000640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf815ea50 0x7f8cf80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f0b43600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24008d90 0x7f0b3c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f0b43600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3c14f1a0 0x7f0b3c150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f0b43600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b3c150640 0x7f0b3c14a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f8cf7400640 1 -- 172.21.15.59:0/1286220451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf815ea50 msgr2=0x7f8cf80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f8cf7400640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cf815ea50 0x7f8cf80ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f8cf7400640 1 -- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 msgr2=0x7f8cf80ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f8cf7400640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cf80d4bc0 0x7f8cf80ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f8cf7400640 1 -- 172.21.15.59:0/1286220451 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf80caee0 con 0x7f8cf8162640 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.915+0000 7f438b800640 1 -- start start 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f438b800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f438415e9c0 0x7f43840d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f438b800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d2d70 0x7f43840d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f438b800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43840cf3c0 con 0x7f43840d10f0 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f438b800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f43840d3710 con 0x7f438415e9c0 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f438b800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f43840cf640 con 0x7f43840d2d70 2024-04-23T14:31:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f4383e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d2d70 0x7f43840d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f4605000640 1 -- 172.21.15.59:0/1194099345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f460015e9c0 msgr2=0x7f46000d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f4605000640 1 --2- 172.21.15.59:0/1194099345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f460015e9c0 0x7f46000d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f4605000640 1 -- 172.21.15.59:0/1194099345 >> v2:172.21.15.59:3300/0 conn(0x7f46000d2d70 msgr2=0x7f46000d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f4605000640 1 --2- 172.21.15.59:0/1194099345 >> v2:172.21.15.59:3300/0 conn(0x7f46000d2d70 0x7f46000d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f4605000640 1 -- 172.21.15.59:0/1194099345 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46000cf900 con 0x7f46000d10f0 2024-04-23T14:31:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f45ff400640 1 --2- 172.21.15.59:0/1194099345 >> v2:172.21.15.59:3300/0 conn(0x7f46000d2d70 0x7f46000d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f5063e00640 1 -- 172.21.15.59:0/1102803547 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c032810 con 0x7f5064072880 2024-04-23T14:31:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f0b43600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c15b4d0 con 0x7f0b3c14f1a0 2024-04-23T14:31:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f4383e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d2d70 0x7f43840d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51296/0 (socket says 172.21.15.59:51296) 2024-04-23T14:31:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f0b43600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c15e880 con 0x7f0b24008d90 2024-04-23T14:31:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f4383e00640 1 -- 172.21.15.59:0/2456168831 learned_addr learned my addr 172.21.15.59:0/2456168831 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f506c200640 1 -- 172.21.15.59:0/1102803547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5064072880 msgr2=0x7f506406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f506c200640 1 --2- 172.21.15.59:0/1102803547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5064072880 0x7f506406ee00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5064075670 tx=0x7f505c02a230 comp rx=0 tx=0).stop 2024-04-23T14:31:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f0b43600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3c15ab80 con 0x7f0b3c150640 2024-04-23T14:31:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f0b40c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24008d90 0x7f0b3c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f506c200640 1 -- 172.21.15.59:0/1102803547 shutdown_connections 2024-04-23T14:31:32.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f506c200640 1 --2- 172.21.15.59:0/1102803547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5064072880 0x7f506406ee00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f506c200640 1 --2- 172.21.15.59:0/1102803547 >> v2:172.21.15.59:3300/0 conn(0x7f5064071d90 0x7f50640cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f506c200640 1 -- 172.21.15.59:0/1102803547 >> 172.21.15.59:0/1102803547 conn(0x7f50640b2130 msgr2=0x7f50640b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c050270 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c050830 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f506c200640 1 -- 172.21.15.59:0/1102803547 shutdown_connections 2024-04-23T14:31:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7fa959e00640 1 Processor -- start 2024-04-23T14:31:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.916+0000 7fa959e00640 1 -- start start 2024-04-23T14:31:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f0b40c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24008d90 0x7f0b3c15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39504/0 (socket says 172.21.15.59:39504) 2024-04-23T14:31:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f0b40c00640 1 -- 172.21.15.59:0/338391448 learned_addr learned my addr 172.21.15.59:0/338391448 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f7475e00640 1 -- 172.21.15.59:0/2817309333 wait complete. 2024-04-23T14:31:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f8cf7400640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cf8162640 0x7f8cf81596e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0030000 tx=0x7f8cf0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.917+0000 7f4388e00640 1 --2- 172.21.15.59:0/2456168831 >> v2:172.21.15.59:3300/0 conn(0x7f438415e9c0 0x7f43840d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.918+0000 7f506c200640 1 -- 172.21.15.59:0/1102803547 wait complete. 2024-04-23T14:31:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f0b3be00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3c14f1a0 0x7f0b3c150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7fa959e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95406e0b0 0x7fa95406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7fa959e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9540b6690 con 0x7fa95415a740 2024-04-23T14:31:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7fa959e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9540cf790 con 0x7fa9540b5880 2024-04-23T14:31:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7fa959e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9540cfa30 con 0x7fa95406e0b0 2024-04-23T14:31:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f4605000640 1 --2- 172.21.15.59:0/1194099345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000d1580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f46000cb810 tx=0x7f45f802a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69665ea4da073d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f4383e00640 1 -- 172.21.15.59:0/2456168831 >> v2:172.21.15.59:3300/0 conn(0x7f438415e9c0 msgr2=0x7f43840d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f4383e00640 1 --2- 172.21.15.59:0/2456168831 >> v2:172.21.15.59:3300/0 conn(0x7f438415e9c0 0x7f43840d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f0b41600640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b3c150640 0x7f0b3c14a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7fa958a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa95415a740 0x7fa9540d07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36494/0 (socket says 172.21.15.59:36494) 2024-04-23T14:31:32.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f4383400640 1 -- 172.21.15.59:0/2456168831 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 978574902 0 0) 0x7f437c001490 con 0x7f43840d10f0 2024-04-23T14:31:32.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f4383e00640 1 -- 172.21.15.59:0/2456168831 >> v1:172.21.15.76:6789/0 conn(0x7f43840d10f0 legacy=0x7f43840d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f4383e00640 1 -- 172.21.15.59:0/2456168831 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43840cf900 con 0x7f43840d2d70 2024-04-23T14:31:32.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b00001030 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f0b40c00640 1 -- 172.21.15.59:0/338391448 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b3c150640 msgr2=0x7f0b3c14a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f0b40c00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b3c150640 0x7f0b3c14a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f0b40c00640 1 -- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3c14f1a0 msgr2=0x7f0b3c150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f0b40c00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3c14f1a0 0x7f0b3c150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f0b40c00640 1 -- 172.21.15.59:0/338391448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c14e400 con 0x7f0b24008d90 2024-04-23T14:31:32.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f4388e00640 1 --2- 172.21.15.59:0/2456168831 >> v2:172.21.15.59:3300/0 conn(0x7f438415e9c0 0x7f43840d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7fa958a00640 1 -- 172.21.15.59:0/4040805750 learned_addr learned my addr 172.21.15.59:0/4040805750 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7fa953e00640 1 --2- 172.21.15.59:0/4040805750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95406e0b0 0x7fa95406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0041070 con 0x7f8cf8162640 2024-04-23T14:31:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf00079b0 con 0x7f8cf8162640 2024-04-23T14:31:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.919+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0036070 con 0x7f8cf8162640 2024-04-23T14:31:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f506c200640 1 Processor -- start 2024-04-23T14:31:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f45fea00640 1 -- 172.21.15.59:0/1194099345 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8007740 con 0x7f46000d10f0 2024-04-23T14:31:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7fa953400640 1 -- 172.21.15.59:0/4040805750 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137636451 0 0) 0x7fa9540b6690 con 0x7fa95415a740 2024-04-23T14:31:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f45fea00640 1 -- 172.21.15.59:0/1194099345 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f80078e0 con 0x7f46000d10f0 2024-04-23T14:31:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7f45fea00640 1 -- 172.21.15.59:0/1194099345 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8007cb0 con 0x7f46000d10f0 2024-04-23T14:31:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7fa953400640 1 -- 172.21.15.59:0/4040805750 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa938003520 con 0x7fa95415a740 2024-04-23T14:31:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.920+0000 7fa953e00640 1 -- 172.21.15.59:0/4040805750 >> v1:172.21.15.76:6789/0 conn(0x7fa9540b5880 legacy=0x7fa95415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.921+0000 7fa953400640 1 -- 172.21.15.59:0/4040805750 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629585770 0 0) 0x7fa938003520 con 0x7fa95415a740 2024-04-23T14:31:32.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.921+0000 7fa953e00640 1 -- 172.21.15.59:0/4040805750 >> v1:172.21.15.59:6789/0 conn(0x7fa95415a740 legacy=0x7fa9540d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.921+0000 7f506c200640 1 -- start start 2024-04-23T14:31:32.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.921+0000 7f0b40c00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24008d90 0x7f0b3c15d2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e760 tx=0x7f0b2800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.921+0000 7fa953e00640 1 -- 172.21.15.59:0/4040805750 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9540cfd10 con 0x7fa95406e0b0 2024-04-23T14:31:32.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.921+0000 7f4383e00640 1 --2- 172.21.15.59:0/2456168831 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d2d70 0x7f43840d31d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4370009850 tx=0x7f437002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd2bb1080a43485 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 msgr2=0x7f46000d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 --2- 172.21.15.59:0/1194099345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000d1580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f46000cb810 tx=0x7f45f802a540 comp rx=0 tx=0).stop 2024-04-23T14:31:32.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 shutdown_connections 2024-04-23T14:31:32.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 --2- 172.21.15.59:0/1194099345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000d1580 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 --2- 172.21.15.59:0/1194099345 >> v2:172.21.15.59:3300/0 conn(0x7f46000d2d70 0x7f46000d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 --2- 172.21.15.59:0/1194099345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f460015e9c0 0x7f46000d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f7475e00640 1 Processor -- start 2024-04-23T14:31:32.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 >> 172.21.15.59:0/1194099345 conn(0x7f460008bf90 msgr2=0x7f46000d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 shutdown_connections 2024-04-23T14:31:32.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f7475e00640 1 -- start start 2024-04-23T14:31:32.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f506c200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5064071d90 0x7f506415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f506c200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5064072880 0x7f506415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f506c200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50640b3430 0x7f506415f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f506c200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50640b3a20 con 0x7f5064072880 2024-04-23T14:31:32.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f506c200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50640b38a0 con 0x7f50640b3430 2024-04-23T14:31:32.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f506c200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5064166430 con 0x7f5064071d90 2024-04-23T14:31:32.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7fa953e00640 1 --2- 172.21.15.59:0/4040805750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95406e0b0 0x7fa95406ef90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9400097d0 tx=0x7fa94002b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca52eb236a4e82c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b10012b90 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf815fb50 con 0x7f8cf8162640 2024-04-23T14:31:32.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f5069800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5064071d90 0x7f506415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f5069800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5064071d90 0x7f506415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51338/0 (socket says 172.21.15.59:51338) 2024-04-23T14:31:32.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f5069800640 1 -- 172.21.15.59:0/390722857 learned_addr learned my addr 172.21.15.59:0/390722857 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf8160110 con 0x7f8cf8162640 2024-04-23T14:31:32.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800c660 con 0x7f0b24008d90 2024-04-23T14:31:32.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b28018070 con 0x7f0b24008d90 2024-04-23T14:31:32.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800ca60 con 0x7f0b24008d90 2024-04-23T14:31:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f5068e00640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5064072880 0x7f506415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7468053750 0x7f746804f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.923+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3c150b10 con 0x7f0b24008d90 2024-04-23T14:31:32.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3c14e9a0 con 0x7f0b24008d90 2024-04-23T14:31:32.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468054110 0x7f74680518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f7468051df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7468057830 con 0x7f74680e3300 2024-04-23T14:31:32.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74680583e0 con 0x7f7468053750 2024-04-23T14:31:32.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f746803d130 con 0x7f7468054110 2024-04-23T14:31:32.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cf81604f0 con 0x7f8cf8162640 2024-04-23T14:31:32.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f4383400640 1 -- 172.21.15.59:0/2456168831 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370004030 con 0x7f43840d2d70 2024-04-23T14:31:32.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f4383400640 1 -- 172.21.15.59:0/2456168831 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43700041d0 con 0x7f43840d2d70 2024-04-23T14:31:32.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f7468051df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f7468051df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33306/0 (socket says 172.21.15.59:33306) 2024-04-23T14:31:32.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f4383400640 1 -- 172.21.15.59:0/2456168831 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370003610 con 0x7f43840d2d70 2024-04-23T14:31:32.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f506a200640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50640b3430 0x7f506415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.924+0000 7f7475400640 1 -- 172.21.15.59:0/38060272 learned_addr learned my addr 172.21.15.59:0/38060272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.922+0000 7f4607000640 1 -- 172.21.15.59:0/1194099345 wait complete. 2024-04-23T14:31:32.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f7474a00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7468053750 0x7f746804f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b3c068a80 con 0x7f0b24008d90 2024-04-23T14:31:32.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f7475400640 1 -- 172.21.15.59:0/38060272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468054110 msgr2=0x7f74680518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f7475400640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7468054110 0x7f74680518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f7475400640 1 -- 172.21.15.59:0/38060272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7468053750 msgr2=0x7f746804f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7fa953400640 1 -- 172.21.15.59:0/4040805750 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940038070 con 0x7fa95406e0b0 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7fa953400640 1 -- 172.21.15.59:0/4040805750 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94002be40 con 0x7fa95406e0b0 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7fa953400640 1 -- 172.21.15.59:0/4040805750 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c005050 con 0x7fa95406e0b0 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f5068e00640 1 -- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5064071d90 msgr2=0x7f506415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f5068e00640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5064071d90 0x7f506415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f5068e00640 1 -- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50640b3430 msgr2=0x7f506415f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f5068e00640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50640b3430 0x7f506415f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f5068e00640 1 -- 172.21.15.59:0/390722857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506415d0c0 con 0x7f5064072880 2024-04-23T14:31:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f438b800640 1 -- 172.21.15.59:0/2456168831 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d2d70 msgr2=0x7f43840d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f438b800640 1 --2- 172.21.15.59:0/2456168831 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d2d70 0x7f43840d31d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4370009850 tx=0x7f437002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f4607000640 1 Processor -- start 2024-04-23T14:31:32.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f438b800640 1 -- 172.21.15.59:0/2456168831 shutdown_connections 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f438b800640 1 --2- 172.21.15.59:0/2456168831 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d2d70 0x7f43840d31d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f438b800640 1 --2- 172.21.15.59:0/2456168831 >> v2:172.21.15.59:3300/0 conn(0x7f438415e9c0 0x7f43840d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f438b800640 1 -- 172.21.15.59:0/2456168831 >> 172.21.15.59:0/2456168831 conn(0x7f438408bf90 msgr2=0x7f43840d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f5069800640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5064071d90 0x7f506415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f8b21600640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b0406c9c0 0x7f8b0406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8b1008bd50 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f8b21600640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f8b04074be0 0x7f8b04077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b040776b0 con 0x7f8b04074be0 2024-04-23T14:31:32.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8b1008e3b0 con 0x7f8b1c0e78d0 2024-04-23T14:31:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f7475400640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7468053750 0x7f746804f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.925+0000 7f7475400640 1 -- 172.21.15.59:0/38060272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7468052470 con 0x7f74680e3300 2024-04-23T14:31:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f7474a00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7468053750 0x7f746804f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:32.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f5068e00640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5064072880 0x7f506415ba80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f505000b3e0 tx=0x7f505000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7fa959e00640 1 -- 172.21.15.59:0/4040805750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95406e0b0 msgr2=0x7fa95406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7fa959e00640 1 --2- 172.21.15.59:0/4040805750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95406e0b0 0x7fa95406ef90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9400097d0 tx=0x7fa94002b3b0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7fa959e00640 1 -- 172.21.15.59:0/4040805750 shutdown_connections 2024-04-23T14:31:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7fa959e00640 1 --2- 172.21.15.59:0/4040805750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95406e0b0 0x7fa95406ef90 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9400097d0 tx=0x7fa94002b3b0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050015030 con 0x7f5064072880 2024-04-23T14:31:32.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7fa959e00640 1 -- 172.21.15.59:0/4040805750 >> 172.21.15.59:0/4040805750 conn(0x7fa954067e70 msgr2=0x7fa95406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:32.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f4607000640 1 -- start start 2024-04-23T14:31:32.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5050004020 con 0x7f5064072880 2024-04-23T14:31:32.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050004450 con 0x7f5064072880 2024-04-23T14:31:32.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f7475400640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f7468051df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f74700635c0 tx=0x7f7470069560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506415d3a0 con 0x7f5064072880 2024-04-23T14:31:32.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747006b9d0 con 0x7f74680e3300 2024-04-23T14:31:32.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7468052750 con 0x7f74680e3300 2024-04-23T14:31:32.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747006bb70 con 0x7f74680e3300 2024-04-23T14:31:32.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746804d590 con 0x7f74680e3300 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747006d110 con 0x7f74680e3300 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f4607000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f4607000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46000d2d70 0x7f46001f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f4607000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f460015e9c0 0x7f46001f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f4607000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46000d1ac0 con 0x7f460015e9c0 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f4607000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f460015eda0 con 0x7f46000d10f0 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f4607000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f460006cb90 con 0x7f46000d2d70 2024-04-23T14:31:32.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f744c001030 con 0x7f74680e3300 2024-04-23T14:31:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7fa959e00640 1 -- 172.21.15.59:0/4040805750 shutdown_connections 2024-04-23T14:31:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f45ff400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46000d2d70 0x7f46001f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7fa959e00640 1 -- 172.21.15.59:0/4040805750 wait complete. 2024-04-23T14:31:32.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.926+0000 7f8b29600640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b0406c9c0 0x7f8b0406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.927+0000 7f8b28c00640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f8b04074be0 0x7f8b04077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f8b29600640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b0406c9c0 0x7f8b0406ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0069e0 tx=0x7f8b240749e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f8b28c00640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f8b04074be0 0x7f8b04077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f438b800640 1 -- 172.21.15.59:0/2456168831 shutdown_connections 2024-04-23T14:31:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39538/0 (socket says 172.21.15.59:39538) 2024-04-23T14:31:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f45ffe00640 1 -- 172.21.15.59:0/2243663369 learned_addr learned my addr 172.21.15.59:0/2243663369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.930+0000 7f4605000640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f460015e9c0 0x7f46001f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506415bfc0 con 0x7f5064072880 2024-04-23T14:31:32.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.928+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cf0044070 con 0x7f8cf8162640 2024-04-23T14:31:32.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f8ceea00640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8cd806c940 0x7f8cd806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f8cf6a00640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8cd806c940 0x7f8cd806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.930+0000 7f8cf6a00640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8cd806c940 0x7f8cd806ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8001770 tx=0x7f8ce8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5064068a80 con 0x7f5064072880 2024-04-23T14:31:32.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8cf00789d0 con 0x7f8cf8162640 2024-04-23T14:31:32.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b2800cc00 con 0x7f0b24008d90 2024-04-23T14:31:32.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.930+0000 7f0b38c00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b1c06c9c0 0x7f0b1c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.930+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0b2808c2f0 con 0x7f0b24008d90 2024-04-23T14:31:32.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f0b38c00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f0b1c074be0 0x7f0b1c077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b2800fe50 con 0x7f0b1c074be0 2024-04-23T14:31:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0b2808c6e0 con 0x7f0b24008d90 2024-04-23T14:31:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f8ceea00640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f8cd8075770 0x7f8cd8077c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f0b3be00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b1c06c9c0 0x7f0b1c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f0b41600640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f0b1c074be0 0x7f0b1c077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cd80782e0 con 0x7f8cd8075770 2024-04-23T14:31:32.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8cf0077360 con 0x7f8cf8162640 2024-04-23T14:31:32.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.929+0000 7f438b800640 1 -- 172.21.15.59:0/2456168831 wait complete. 2024-04-23T14:31:32.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f438b800640 1 Processor -- start 2024-04-23T14:31:32.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f747006d2b0 con 0x7f74680e3300 2024-04-23T14:31:32.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f438b800640 1 -- start start 2024-04-23T14:31:32.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.933+0000 7fa959e00640 1 Processor -- start 2024-04-23T14:31:32.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f45ffe00640 1 -- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46000d2d70 msgr2=0x7f46001f8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f45ffe00640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46000d2d70 0x7f46001f8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f45ffe00640 1 -- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f460015e9c0 msgr2=0x7f46001f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f45ffe00640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f460015e9c0 0x7f46001f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f45ffe00640 1 -- 172.21.15.59:0/2243663369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4600160c10 con 0x7f46000d10f0 2024-04-23T14:31:32.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.932+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5050018070 con 0x7f5064072880 2024-04-23T14:31:32.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.933+0000 7f0b3be00640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b1c06c9c0 0x7f0b1c06ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000ba00 tx=0x7f0b30030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.933+0000 7fa959e00640 1 -- start start 2024-04-23T14:31:32.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.933+0000 7f45ffe00640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000b8840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e7d0 tx=0x7f45f000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.933+0000 7f746cc00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f745006c8f0 0x7f745006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.931+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b040776b0 con 0x7f8b04074be0 2024-04-23T14:31:32.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.933+0000 7f8cf6000640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f8cd8075770 0x7f8cd8077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.933+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0017070 con 0x7f46000d10f0 2024-04-23T14:31:32.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7470107830 con 0x7f74680e3300 2024-04-23T14:31:32.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f000c940 con 0x7f46000d10f0 2024-04-23T14:31:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f000cd30 con 0x7f46000d10f0 2024-04-23T14:31:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4600160ef0 con 0x7f46000d10f0 2024-04-23T14:31:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f438b800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d10f0 0x7f43840cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f0b41600640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f0b1c074be0 0x7f0b1c077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f5061600640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f504406c8f0 0x7f504406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f7474a00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f745006c8f0 0x7f745006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f746cc00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7f7450074b70 0x7f7450076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f747006fe80 con 0x7f7450074b70 2024-04-23T14:31:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7470107c20 con 0x7f74680e3300 2024-04-23T14:31:32.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f438b800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f438415e9c0 0x7f43840b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f746fe00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7f7450074b70 0x7f7450076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f505008a940 con 0x7f5064072880 2024-04-23T14:31:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f5061600640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7f5044074a90 0x7f5044076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5044077560 con 0x7f5044074a90 2024-04-23T14:31:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f505008ac40 con 0x7f5064072880 2024-04-23T14:31:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f5069800640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f504406c8f0 0x7f504406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f8cf6000640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f8cd8075770 0x7f8cd8077c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f7474a00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f745006c8f0 0x7f745006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7464034000 tx=0x7f746402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f438b800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43840b9fe0 0x7f4384160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f4388e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d10f0 0x7f43840cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f438b800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f438406cb90 con 0x7f43840b9fe0 2024-04-23T14:31:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f438b800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f438415eda0 con 0x7f438415e9c0 2024-04-23T14:31:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f4388e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d10f0 0x7f43840cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51374/0 (socket says 172.21.15.59:51374) 2024-04-23T14:31:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f4388e00640 1 -- 172.21.15.59:0/1242701955 learned_addr learned my addr 172.21.15.59:0/1242701955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f4389800640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43840b9fe0 0x7f4384160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f4383e00640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f438415e9c0 0x7f43840b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f438406cd10 con 0x7f43840d10f0 2024-04-23T14:31:32.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7f506a200640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7f5044074a90 0x7f5044076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.934+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001614b0 con 0x7f46000d10f0 2024-04-23T14:31:32.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.936+0000 7f4389800640 1 -- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d10f0 msgr2=0x7f43840cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.936+0000 7f4389800640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43840d10f0 0x7f43840cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.936+0000 7f4389800640 1 -- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f438415e9c0 msgr2=0x7f43840b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.936+0000 7f4389800640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f438415e9c0 0x7f43840b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.936+0000 7f4389800640 1 -- 172.21.15.59:0/1242701955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43840ba540 con 0x7f43840b9fe0 2024-04-23T14:31:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7fa959e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9540b5880 0x7fa9541fb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7fa959e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95415a740 0x7fa95415e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7fa959e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95415eea0 0x7fa954152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7fa959e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540d0f00 con 0x7fa95415eea0 2024-04-23T14:31:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7fa959e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540d02d0 con 0x7fa9540b5880 2024-04-23T14:31:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.935+0000 7fa959e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9540d1080 con 0x7fa95415a740 2024-04-23T14:31:32.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.936+0000 7f506a200640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7f5044074a90 0x7f5044076eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.936+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cd80782e0 con 0x7f8cd8075770 2024-04-23T14:31:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f5069800640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f504406c8f0 0x7f504406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f505800c530 tx=0x7f505800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f4389800640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43840b9fe0 0x7f4384160200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f437c00c700 tx=0x7f437c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c017070 con 0x7f43840b9fe0 2024-04-23T14:31:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7fa958a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9540b5880 0x7fa9541fb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7fa958a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9540b5880 0x7fa9541fb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39552/0 (socket says 172.21.15.59:39552) 2024-04-23T14:31:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7fa958a00640 1 -- 172.21.15.59:0/691947275 learned_addr learned my addr 172.21.15.59:0/691947275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7fa953e00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95415a740 0x7fa95415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f437c003dc0 con 0x7f43840b9fe0 2024-04-23T14:31:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0052b0 con 0x7f43840b9fe0 2024-04-23T14:31:32.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438415f440 con 0x7f43840b9fe0 2024-04-23T14:31:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46000b8d80 con 0x7f46000d10f0 2024-04-23T14:31:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.938+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b2800fe50 con 0x7f0b1c074be0 2024-04-23T14:31:32.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.937+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438415fa00 con 0x7f43840b9fe0 2024-04-23T14:31:32.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.938+0000 7fa959400640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95415eea0 0x7fa954152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.938+0000 7fa958a00640 1 -- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95415a740 msgr2=0x7fa95415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.938+0000 7fa958a00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa95415a740 0x7fa95415e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.938+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4384068a80 con 0x7f43840b9fe0 2024-04-23T14:31:32.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.939+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5044077560 con 0x7f5044074a90 2024-04-23T14:31:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.938+0000 7f746fe00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7f7450074b70 0x7f7450076f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.939+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f747006fe80 con 0x7f7450074b70 2024-04-23T14:31:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.941+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f437c01a070 con 0x7f43840b9fe0 2024-04-23T14:31:32.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.942+0000 7f4380c00640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f436406c9c0 0x7f436406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.939+0000 7fa958a00640 1 -- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95415eea0 msgr2=0x7fa954152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.939+0000 7fa958a00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95415eea0 0x7fa954152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.939+0000 7fa958a00640 1 -- 172.21.15.59:0/691947275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9542024c0 con 0x7fa9540b5880 2024-04-23T14:31:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.939+0000 7fa959400640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95415eea0 0x7fa954152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.942+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f437c08b0a0 con 0x7f43840b9fe0 2024-04-23T14:31:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.942+0000 7f4380c00640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f4364074bc0 0x7f4364076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.941+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45f001a070 con 0x7f46000d10f0 2024-04-23T14:31:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.942+0000 7f45f7e00640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f45e006c8f0 0x7f45e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.942+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f45f008c0e0 con 0x7f46000d10f0 2024-04-23T14:31:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.943+0000 7f45f7e00640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7f45e0074a90 0x7f45e0076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.943+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45f0001d30 con 0x7f45e0074a90 2024-04-23T14:31:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.943+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f45f008c490 con 0x7f46000d10f0 2024-04-23T14:31:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.943+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4364077690 con 0x7f4364074bc0 2024-04-23T14:31:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.943+0000 7fa958a00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9540b5880 0x7fa9541fb730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa94c001a60 tx=0x7fa94c001f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.943+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f437c092050 con 0x7f43840b9fe0 2024-04-23T14:31:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.943+0000 7f4388e00640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f436406c9c0 0x7f436406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.944+0000 7f4605000640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7f45e0074a90 0x7f45e0076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.944+0000 7f4383e00640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f4364074bc0 0x7f4364076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.944+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c018070 con 0x7fa9540b5880 2024-04-23T14:31:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.944+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c00d6c0 con 0x7fa9540b5880 2024-04-23T14:31:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.944+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c018070 con 0x7fa9540b5880 2024-04-23T14:31:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.945+0000 7f4605000640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7f45e0074a90 0x7f45e0076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.945+0000 7f45ff400640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f45e006c8f0 0x7f45e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.945+0000 7f4383e00640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f4364074bc0 0x7f4364076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.945+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95415de80 con 0x7fa9540b5880 2024-04-23T14:31:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.946+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95415e440 con 0x7fa9540b5880 2024-04-23T14:31:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.946+0000 7f45ff400640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f45e006c8f0 0x7f45e006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009500 tx=0x7f45ec009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.947+0000 7f4388e00640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f436406c9c0 0x7f436406ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4378009500 tx=0x7f4378009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.947+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa95404fd90 con 0x7fa9540b5880 2024-04-23T14:31:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.949+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f0001d30 con 0x7f45e0074a90 2024-04-23T14:31:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.949+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4364077690 con 0x7f4364074bc0 2024-04-23T14:31:33.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.950+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa94c01b070 con 0x7fa9540b5880 2024-04-23T14:31:33.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.951+0000 7fa950c00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa93406caf0 0x7fa93406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.952+0000 7fa953e00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa93406caf0 0x7fa93406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.953+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa94c056d80 con 0x7fa9540b5880 2024-04-23T14:31:33.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.953+0000 7fa953e00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa93406caf0 0x7fa93406efb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa940035000 tx=0x7fa94002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.954+0000 7fa950c00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7fa934075970 0x7fa934077e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.954+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9340784e0 con 0x7fa934075970 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.954+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa94c055710 con 0x7fa9540b5880 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.955+0000 7fa959400640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7fa934075970 0x7fa934077e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.956+0000 7fa959400640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7fa934075970 0x7fa934077e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.958+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9340784e0 con 0x7fa934075970 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.973+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b00002dd0 con 0x7f8b04074be0 2024-04-23T14:31:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.973+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8cf80d5360 con 0x7f8cd8075770 2024-04-23T14:31:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.973+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f744c002dd0 con 0x7f7450074b70 2024-04-23T14:31:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.974+0000 7f746cc00640 1 -- 172.21.15.59:0/38060272 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f744c002dd0 con 0x7f7450074b70 2024-04-23T14:31:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.974+0000 7f8ceea00640 1 -- 172.21.15.59:0/1286220451 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8cf80d5360 con 0x7f8cd8075770 2024-04-23T14:31:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.975+0000 7f8b21600640 1 -- 172.21.15.59:0/3182847914 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8b00002dd0 con 0x7f8b04074be0 2024-04-23T14:31:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.975+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b3c0ccf00 con 0x7f0b1c074be0 2024-04-23T14:31:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.975+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f8cd8075770 msgr2=0x7f8cd8077c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.975+0000 7f8cf7e00640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f8cd8075770 0x7f8cd8077c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.975+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f8b04074be0 msgr2=0x7f8b04077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7f7450074b70 msgr2=0x7f7450076f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7f7450074b70 0x7f7450076f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.975+0000 7f8b2aa00640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f8b04074be0 0x7f8b04077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b0406c9c0 msgr2=0x7f8b0406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8b2aa00640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b0406c9c0 0x7f8b0406ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0069e0 tx=0x7f8b240749e0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0e78d0 msgr2=0x7f8b1c0e5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8b2aa00640 1 --2- 172.21.15.59:0/3182847914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0e78d0 0x7f8b1c0e5ef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d600 tx=0x7f8b1000dac0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8b2a000640 1 -- 172.21.15.59:0/3182847914 reap_dead start 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f745006c8f0 msgr2=0x7f745006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f745006c8f0 0x7f745006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7464034000 tx=0x7f746402e000 comp rx=0 tx=0).stop 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 msgr2=0x7f7468051df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 --2- 172.21.15.59:0/38060272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74680e3300 0x7f7468051df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f74700635c0 tx=0x7f7470069560 comp rx=0 tx=0).stop 2024-04-23T14:31:33.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475400640 1 -- 172.21.15.59:0/38060272 reap_dead start 2024-04-23T14:31:33.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f0b38c00640 1 -- 172.21.15.59:0/338391448 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b3c0ccf00 con 0x7f0b1c074be0 2024-04-23T14:31:33.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8cd806c940 msgr2=0x7f8cd806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 shutdown_connections 2024-04-23T14:31:33.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 >> 172.21.15.59:0/38060272 conn(0x7f7468058ef0 msgr2=0x7f746803d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.975+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5064165910 con 0x7f5044074a90 2024-04-23T14:31:33.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8cf7e00640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8cd806c940 0x7f8cd806ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8001770 tx=0x7f8ce8009040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cf8162640 msgr2=0x7f8cf81596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.976+0000 7f8cf7e00640 1 --2- 172.21.15.59:0/1286220451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cf8162640 0x7f8cf81596e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0030000 tx=0x7f8cf0004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:33.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f0b1c074be0 msgr2=0x7f0b1c077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f0b43600640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f0b1c074be0 0x7f0b1c077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b1c06c9c0 msgr2=0x7f0b1c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f0b43600640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b1c06c9c0 0x7f0b1c06ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000ba00 tx=0x7f0b30030000 comp rx=0 tx=0).stop 2024-04-23T14:31:33.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24008d90 msgr2=0x7f0b3c15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f0b43600640 1 --2- 172.21.15.59:0/338391448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b24008d90 0x7f0b3c15d2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e760 tx=0x7f0b2800ec20 comp rx=0 tx=0).stop 2024-04-23T14:31:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f0b41600640 1 -- 172.21.15.59:0/338391448 reap_dead start 2024-04-23T14:31:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.977+0000 7f5061600640 1 -- 172.21.15.59:0/390722857 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5064165910 con 0x7f5044074a90 2024-04-23T14:31:33.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 shutdown_connections 2024-04-23T14:31:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 >> 172.21.15.59:0/3182847914 conn(0x7f8b1c03aa40 msgr2=0x7f8b1c0385d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 shutdown_connections 2024-04-23T14:31:33.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f8b2aa00640 1 -- 172.21.15.59:0/3182847914 wait complete. 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7f5044074a90 msgr2=0x7f5044076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7f5044074a90 0x7f5044076eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f504406c8f0 msgr2=0x7f504406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f504406c8f0 0x7f504406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f505800c530 tx=0x7f505800b040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5064072880 msgr2=0x7f506415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 --2- 172.21.15.59:0/390722857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5064072880 0x7f506415ba80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f505000b3e0 tx=0x7f505000b8a0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506a200640 1 -- 172.21.15.59:0/390722857 reap_dead start 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 shutdown_connections 2024-04-23T14:31:33.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f7475e00640 1 -- 172.21.15.59:0/38060272 wait complete. 2024-04-23T14:31:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 shutdown_connections 2024-04-23T14:31:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.978+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 >> 172.21.15.59:0/390722857 conn(0x7f50640b2130 msgr2=0x7f50640b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f8cf7400640 1 -- 172.21.15.59:0/1286220451 reap_dead start 2024-04-23T14:31:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 shutdown_connections 2024-04-23T14:31:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 >> 172.21.15.59:0/1286220451 conn(0x7f8cf809d950 msgr2=0x7f8cf815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 shutdown_connections 2024-04-23T14:31:33.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 >> 172.21.15.59:0/338391448 conn(0x7f0b3c06b9d0 msgr2=0x7f0b3c0b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 shutdown_connections 2024-04-23T14:31:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 shutdown_connections 2024-04-23T14:31:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f8cf7e00640 1 -- 172.21.15.59:0/1286220451 wait complete. 2024-04-23T14:31:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f0b43600640 1 -- 172.21.15.59:0/338391448 wait complete. 2024-04-23T14:31:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 shutdown_connections 2024-04-23T14:31:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.979+0000 7f506c200640 1 -- 172.21.15.59:0/390722857 wait complete. 2024-04-23T14:31:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.981+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f43840d2630 con 0x7f4364074bc0 2024-04-23T14:31:33.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.983+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9540d15d0 con 0x7fa934075970 2024-04-23T14:31:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.984+0000 7f4380c00640 1 -- 172.21.15.59:0/1242701955 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f43840d2630 con 0x7f4364074bc0 2024-04-23T14:31:33.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.984+0000 7fa950c00640 1 -- 172.21.15.59:0/691947275 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9540d15d0 con 0x7fa934075970 2024-04-23T14:31:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f4364074bc0 msgr2=0x7f4364076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4600068a80 con 0x7f45e0074a90 2024-04-23T14:31:33.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7f438b800640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f4364074bc0 0x7f4364076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7fa934075970 msgr2=0x7fa934077e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7fa959e00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7fa934075970 0x7fa934077e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa93406caf0 msgr2=0x7fa93406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7fa959e00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa93406caf0 0x7fa93406efb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa940035000 tx=0x7fa94002f000 comp rx=0 tx=0).stop 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9540b5880 msgr2=0x7fa9541fb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7fa959e00640 1 --2- 172.21.15.59:0/691947275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9540b5880 0x7fa9541fb730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa94c001a60 tx=0x7fa94c001f20 comp rx=0 tx=0).stop 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.985+0000 7fa959400640 1 -- 172.21.15.59:0/691947275 reap_dead start 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 shutdown_connections 2024-04-23T14:31:33.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 >> 172.21.15.59:0/691947275 conn(0x7fa954067e70 msgr2=0x7fa954159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7f45f7e00640 1 -- 172.21.15.59:0/2243663369 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4600068a80 con 0x7f45e0074a90 2024-04-23T14:31:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f436406c9c0 msgr2=0x7f436406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7f438b800640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f436406c9c0 0x7f436406ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4378009500 tx=0x7f4378009110 comp rx=0 tx=0).stop 2024-04-23T14:31:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43840b9fe0 msgr2=0x7f4384160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7f438b800640 1 --2- 172.21.15.59:0/1242701955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43840b9fe0 0x7f4384160200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f437c00c700 tx=0x7f437c00cbc0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 shutdown_connections 2024-04-23T14:31:33.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.987+0000 7fa959e00640 1 -- 172.21.15.59:0/691947275 wait complete. 2024-04-23T14:31:33.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.986+0000 7f4389800640 1 -- 172.21.15.59:0/1242701955 reap_dead start 2024-04-23T14:31:33.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 shutdown_connections 2024-04-23T14:31:33.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 >> 172.21.15.59:0/1242701955 conn(0x7f438408bf90 msgr2=0x7f43840b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7f45e0074a90 msgr2=0x7f45e0076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f4607000640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7f45e0074a90 0x7f45e0076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f45e006c8f0 msgr2=0x7f45e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f4607000640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f45e006c8f0 0x7f45e006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009500 tx=0x7f45ec009110 comp rx=0 tx=0).stop 2024-04-23T14:31:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 msgr2=0x7f46000b8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.989+0000 7f4607000640 1 --2- 172.21.15.59:0/2243663369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46000d10f0 0x7f46000b8840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e7d0 tx=0x7f45f000ec90 comp rx=0 tx=0).stop 2024-04-23T14:31:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.990+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 shutdown_connections 2024-04-23T14:31:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.990+0000 7f4605000640 1 -- 172.21.15.59:0/2243663369 reap_dead start 2024-04-23T14:31:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.990+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 shutdown_connections 2024-04-23T14:31:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.990+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 >> 172.21.15.59:0/2243663369 conn(0x7f460008bf90 msgr2=0x7f46000b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.990+0000 7f438b800640 1 -- 172.21.15.59:0/1242701955 wait complete. 2024-04-23T14:31:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.990+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 shutdown_connections 2024-04-23T14:31:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:32.990+0000 7f4607000640 1 -- 172.21.15.59:0/2243663369 wait complete. 2024-04-23T14:31:33.022 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T14:31:33.024 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:33.024 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T14:31:33.025 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:33.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T14:31:33.025 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T14:31:33.027 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T14:31:33.028 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:33.028 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T14:31:33.029 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:33.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T14:31:33.030 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T14:31:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.347+0000 7f1797200640 1 Processor -- start 2024-04-23T14:31:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.347+0000 7f1797200640 1 -- start start 2024-04-23T14:31:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.349+0000 7f1797200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17900d0f80 0x7f17900cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.349+0000 7f1797200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f179015fd30 con 0x7f179015f880 2024-04-23T14:31:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.349+0000 7f1797200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17900cf930 con 0x7f179015ea40 2024-04-23T14:31:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.349+0000 7f1797200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f17900cfc10 con 0x7f17900d0f80 2024-04-23T14:31:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.351+0000 7f1796800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f179015f880 0x7f17900d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36506/0 (socket says 172.21.15.59:36506) 2024-04-23T14:31:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.351+0000 7f1795e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17900d0f80 0x7f17900cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.352+0000 7f1795e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17900d0f80 0x7f17900cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33326/0 (socket says 172.21.15.59:33326) 2024-04-23T14:31:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.352+0000 7f1795e00640 1 -- 172.21.15.59:0/752669671 learned_addr learned my addr 172.21.15.59:0/752669671 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.352+0000 7f1795e00640 1 -- 172.21.15.59:0/752669671 >> v1:172.21.15.76:6789/0 conn(0x7f179015ea40 legacy=0x7f17900d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.351+0000 7f1795400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f179015ea40 0x7f17900d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43502/0 (socket says 172.21.15.59:43502) 2024-04-23T14:31:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.353+0000 7f1795e00640 1 -- 172.21.15.59:0/752669671 >> v1:172.21.15.59:6789/0 conn(0x7f179015f880 legacy=0x7f17900d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.353+0000 7f1795e00640 1 -- 172.21.15.59:0/752669671 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17900cfef0 con 0x7f17900d0f80 2024-04-23T14:31:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.355+0000 7f1796800640 1 --1- 172.21.15.59:0/752669671 >> v1:172.21.15.59:6789/0 conn(0x7f179015f880 0x7f17900d2310 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-23T14:31:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.355+0000 7f1795e00640 1 --2- 172.21.15.59:0/752669671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17900d0f80 0x7f17900cf250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1780009870 tx=0x7f178002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e0f9153c806706 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.355+0000 7fefa5600640 1 Processor -- start 2024-04-23T14:31:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.356+0000 7f1794a00640 1 -- 172.21.15.59:0/752669671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780037070 con 0x7f17900d0f80 2024-04-23T14:31:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.356+0000 7f1794a00640 1 -- 172.21.15.59:0/752669671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780032440 con 0x7f17900d0f80 2024-04-23T14:31:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.356+0000 7f1794a00640 1 -- 172.21.15.59:0/752669671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780032810 con 0x7f17900d0f80 2024-04-23T14:31:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.358+0000 7f1797200640 1 -- 172.21.15.59:0/752669671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17900d0f80 msgr2=0x7f17900cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.356+0000 7fefa5600640 1 -- start start 2024-04-23T14:31:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.358+0000 7fefa5600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fef98006db0 0x7fef980ede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.358+0000 7fefa5600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef980e36e0 con 0x7fef980061f0 2024-04-23T14:31:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.358+0000 7fefa5600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef98057b40 con 0x7fef980e3300 2024-04-23T14:31:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.358+0000 7fefa5600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fef98057e20 con 0x7fef98006db0 2024-04-23T14:31:33.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.358+0000 7f1797200640 1 --2- 172.21.15.59:0/752669671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17900d0f80 0x7f17900cf250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1780009870 tx=0x7f178002a1c0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.359+0000 7fef9f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fef98006db0 0x7fef980ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.359+0000 7fef9e000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fef980061f0 0x7fef980066a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36514/0 (socket says 172.21.15.59:36514) 2024-04-23T14:31:33.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.359+0000 7fef9e000640 1 -- 172.21.15.59:0/2312295265 learned_addr learned my addr 172.21.15.59:0/2312295265 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:33.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.360+0000 7f1797200640 1 -- 172.21.15.59:0/752669671 shutdown_connections 2024-04-23T14:31:33.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.360+0000 7f1797200640 1 --2- 172.21.15.59:0/752669671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17900d0f80 0x7f17900cf250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.360+0000 7f1797200640 1 -- 172.21.15.59:0/752669671 >> 172.21.15.59:0/752669671 conn(0x7f179008bf90 msgr2=0x7f17900d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.360+0000 7fe4aa600640 1 Processor -- start 2024-04-23T14:31:33.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.360+0000 7fef9d600640 1 -- 172.21.15.59:0/2312295265 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 458660489 0 0) 0x7fef980e36e0 con 0x7fef980061f0 2024-04-23T14:31:33.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.360+0000 7fef9d600640 1 -- 172.21.15.59:0/2312295265 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef84003520 con 0x7fef980061f0 2024-04-23T14:31:33.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.361+0000 7fe4aa600640 1 -- start start 2024-04-23T14:31:33.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.361+0000 7f1797200640 1 -- 172.21.15.59:0/752669671 shutdown_connections 2024-04-23T14:31:33.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.361+0000 7fef9f400640 1 -- 172.21.15.59:0/2312295265 >> v1:172.21.15.76:6789/0 conn(0x7fef980e3300 legacy=0x7fef980572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.361+0000 7fef9f400640 1 -- 172.21.15.59:0/2312295265 >> v1:172.21.15.59:6789/0 conn(0x7fef980061f0 legacy=0x7fef980066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.361+0000 7fef9f400640 1 -- 172.21.15.59:0/2312295265 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef98058100 con 0x7fef98006db0 2024-04-23T14:31:33.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.361+0000 7f1797200640 1 -- 172.21.15.59:0/752669671 wait complete. 2024-04-23T14:31:33.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7fe4aa600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe4a40d02f0 0x7fe4a415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7fe4aa600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a4163180 con 0x7fe4a415ee30 2024-04-23T14:31:33.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7fe4aa600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a40d3130 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7fe4aa600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a415f5f0 con 0x7fe4a40d02f0 2024-04-23T14:31:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7f1797200640 1 Processor -- start 2024-04-23T14:31:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7fef9f400640 1 --2- 172.21.15.59:0/2312295265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef980ede10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009720 tx=0x7fef8c02a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287e24db41536523 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7fe4a3e00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe4a40d2bc0 0x7fe4a40d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43520/0 (socket says 172.21.15.59:43520) 2024-04-23T14:31:33.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.363+0000 7fe4a3e00640 1 -- 172.21.15.59:0/3544738328 learned_addr learned my addr 172.21.15.59:0/3544738328 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.364+0000 7fe4a2a00640 1 --2- 172.21.15.59:0/3544738328 >> v2:172.21.15.76:3300/0 conn(0x7fe4a40d02f0 0x7fe4a415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.364+0000 7f1797200640 1 -- start start 2024-04-23T14:31:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.364+0000 7fef9d600640 1 -- 172.21.15.59:0/2312295265 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c037070 con 0x7fef98006db0 2024-04-23T14:31:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.364+0000 7fef9d600640 1 -- 172.21.15.59:0/2312295265 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c02ad40 con 0x7fef98006db0 2024-04-23T14:31:33.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.364+0000 7fef9d600640 1 -- 172.21.15.59:0/2312295265 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c032660 con 0x7fef98006db0 2024-04-23T14:31:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.364+0000 7fe4a2000640 1 -- 172.21.15.59:0/3544738328 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283853392 0 0) 0x7fe4a40d3130 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.364+0000 7fe4a2000640 1 -- 172.21.15.59:0/3544738328 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe488003520 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.365+0000 7fefa5600640 1 -- 172.21.15.59:0/2312295265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 msgr2=0x7fef980ede10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.365+0000 7fefa5600640 1 --2- 172.21.15.59:0/2312295265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef980ede10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009720 tx=0x7fef8c02a030 comp rx=0 tx=0).stop 2024-04-23T14:31:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.365+0000 7f1797200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17900d0f80 0x7f17900b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.365+0000 7f1797200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f179015ea40 0x7f17900b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.365+0000 7f1797200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f179015f880 0x7f17900b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1797200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17900d2a20 con 0x7f179015ea40 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1797200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17900d04b0 con 0x7f17900d0f80 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1797200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f17900d2ba0 con 0x7f179015f880 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1795400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f179015ea40 0x7f17900b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1795400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f179015ea40 0x7f17900b7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33330/0 (socket says 172.21.15.59:33330) 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1795400640 1 -- 172.21.15.59:0/3664318869 learned_addr learned my addr 172.21.15.59:0/3664318869 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.365+0000 7fe4a2a00640 1 -- 172.21.15.59:0/3544738328 >> v1:172.21.15.76:6789/0 conn(0x7fe4a40d2bc0 legacy=0x7fe4a40d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.365+0000 7fe4a2000640 1 -- 172.21.15.59:0/3544738328 <== mon.1 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1414589958 0 0) 0x7fe488003520 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7fe4a2a00640 1 -- 172.21.15.59:0/3544738328 >> v1:172.21.15.59:6789/0 conn(0x7fe4a415ee30 legacy=0x7fe4a4162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1795e00640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17900d0f80 0x7f17900b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.366+0000 7f1796800640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f179015f880 0x7f17900b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7f1795400640 1 -- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f179015f880 msgr2=0x7f17900b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7fefa5600640 1 -- 172.21.15.59:0/2312295265 shutdown_connections 2024-04-23T14:31:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7fefa5600640 1 --2- 172.21.15.59:0/2312295265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef980ede10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7fefa5600640 1 -- 172.21.15.59:0/2312295265 >> 172.21.15.59:0/2312295265 conn(0x7fef98058ef0 msgr2=0x7fef98059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7f1795400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f179015f880 0x7f17900b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7f1795400640 1 -- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17900d0f80 msgr2=0x7f17900b9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7f1795400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17900d0f80 0x7f17900b9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7f1795400640 1 -- 172.21.15.59:0/3664318869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17900c8240 con 0x7f179015ea40 2024-04-23T14:31:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7f1795400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f179015ea40 0x7f17900b7e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f178800c4b0 tx=0x7f178800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.367+0000 7fe4a2a00640 1 -- 172.21.15.59:0/3544738328 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a40d3410 con 0x7fe4a40d02f0 2024-04-23T14:31:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7fefa5600640 1 -- 172.21.15.59:0/2312295265 shutdown_connections 2024-04-23T14:31:33.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7f1786000640 1 -- 172.21.15.59:0/3664318869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788017070 con 0x7f179015ea40 2024-04-23T14:31:33.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7fefa5600640 1 -- 172.21.15.59:0/2312295265 wait complete. 2024-04-23T14:31:33.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7f1786000640 1 -- 172.21.15.59:0/3664318869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1788007480 con 0x7f179015ea40 2024-04-23T14:31:33.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7f1786000640 1 -- 172.21.15.59:0/3664318869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788017070 con 0x7f179015ea40 2024-04-23T14:31:33.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7f1797200640 1 -- 172.21.15.59:0/3664318869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17900c8520 con 0x7f179015ea40 2024-04-23T14:31:33.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.368+0000 7f1797200640 1 -- 172.21.15.59:0/3664318869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17900c9000 con 0x7f179015ea40 2024-04-23T14:31:33.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.369+0000 7fe4a2a00640 1 --2- 172.21.15.59:0/3544738328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a40d02f0 0x7fe4a415fa80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe490009720 tx=0x7fe49002a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46d83a55208d6ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.370+0000 7fe4a2000640 1 -- 172.21.15.59:0/3544738328 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490037070 con 0x7fe4a40d02f0 2024-04-23T14:31:33.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.370+0000 7fefa5600640 1 Processor -- start 2024-04-23T14:31:33.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.371+0000 7f0ddd600640 1 Processor -- start 2024-04-23T14:31:33.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.371+0000 7fe4a2000640 1 -- 172.21.15.59:0/3544738328 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49002ac60 con 0x7fe4a40d02f0 2024-04-23T14:31:33.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.371+0000 7fe4a2000640 1 -- 172.21.15.59:0/3544738328 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0019f0 con 0x7fe4a40d02f0 2024-04-23T14:31:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.371+0000 7fd027600640 1 Processor -- start 2024-04-23T14:31:33.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.371+0000 7f0ddd600640 1 -- start start 2024-04-23T14:31:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.372+0000 7fe4aa600640 1 -- 172.21.15.59:0/3544738328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a40d02f0 msgr2=0x7fe4a415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.372+0000 7fe4aa600640 1 --2- 172.21.15.59:0/3544738328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a40d02f0 0x7fe4a415fa80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe490009720 tx=0x7fe49002a150 comp rx=0 tx=0).stop 2024-04-23T14:31:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.372+0000 7fe4aa600640 1 -- 172.21.15.59:0/3544738328 shutdown_connections 2024-04-23T14:31:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.372+0000 7fe4aa600640 1 --2- 172.21.15.59:0/3544738328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a40d02f0 0x7fe4a415fa80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.372+0000 7fe4aa600640 1 -- 172.21.15.59:0/3544738328 >> 172.21.15.59:0/3544738328 conn(0x7fe4a406cc70 msgr2=0x7fe4a406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.372+0000 7fefa5600640 1 -- start start 2024-04-23T14:31:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.372+0000 7fd027600640 1 -- start start 2024-04-23T14:31:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f0ddd600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd80d0f80 0x7f0dd80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f0ddd600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd80cf930 con 0x7f0dd815f880 2024-04-23T14:31:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f0ddd600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd80cfc10 con 0x7f0dd815ea40 2024-04-23T14:31:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f0ddd600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd80cfef0 con 0x7f0dd80d0f80 2024-04-23T14:31:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fe4aa600640 1 -- 172.21.15.59:0/3544738328 shutdown_connections 2024-04-23T14:31:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fe4aa600640 1 -- 172.21.15.59:0/3544738328 wait complete. 2024-04-23T14:31:33.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f3dad000640 1 Processor -- start 2024-04-23T14:31:33.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f3dad000640 1 -- start start 2024-04-23T14:31:33.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fd027600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd02006c4f0 0x7fd02015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fd027600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 0x7fd02015ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fd027600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd02015b860 con 0x7fd0200b34a0 2024-04-23T14:31:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fd027600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd02006fcc0 con 0x7fd02006c4f0 2024-04-23T14:31:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fd027600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd02015bb40 con 0x7fd0200d08e0 2024-04-23T14:31:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fd01fe00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd0200b34a0 0x7fd0200d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43534/0 (socket says 172.21.15.59:43534) 2024-04-23T14:31:33.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fd01fe00640 1 -- 172.21.15.59:0/1430774028 learned_addr learned my addr 172.21.15.59:0/1430774028 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fefa5600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fef980061f0 0x7fef9817ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fefa5600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef9803a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fefa5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980e3300 0x7fef98052190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fefa5600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef98058be0 con 0x7fef980e3300 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fefa5600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fef9803d130 con 0x7fef98006db0 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7fefa5600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fef980e5a80 con 0x7fef980061f0 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fef9f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980e3300 0x7fef98052190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fef9e000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef9803a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fef9e000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef9803a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39574/0 (socket says 172.21.15.59:39574) 2024-04-23T14:31:33.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.374+0000 7fef9e000640 1 -- 172.21.15.59:0/304728430 learned_addr learned my addr 172.21.15.59:0/304728430 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fef9e000640 1 -- 172.21.15.59:0/304728430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fef980061f0 msgr2=0x7fef9817ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fef9e000640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fef980061f0 0x7fef9817ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fef9e000640 1 -- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980e3300 msgr2=0x7fef98052190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fef9e000640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980e3300 0x7fef98052190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fef9e000640 1 -- 172.21.15.59:0/304728430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9803ab80 con 0x7fef98006db0 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fd025600640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 0x7fd02015ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065043049 0 0) 0x7fd02015b860 con 0x7fd0200b34a0 2024-04-23T14:31:33.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd008003520 con 0x7fd0200b34a0 2024-04-23T14:31:33.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7fef9e000640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef9803a230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefa00635c0 tx=0x7fefa006df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f3dad000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da80b3950 0x7f3da8071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f3dad000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3da806f850 0x7f3da80722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f3dad000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da808b460 con 0x7f3da808b760 2024-04-23T14:31:33.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f3dad000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3da80d0e80 con 0x7f3da806f850 2024-04-23T14:31:33.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f3dad000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3da8072bc0 con 0x7f3da80b3950 2024-04-23T14:31:33.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7fd024c00640 1 --2- 172.21.15.59:0/1430774028 >> v2:172.21.15.76:3300/0 conn(0x7fd02006c4f0 0x7fd02015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7fef96a00640 1 -- 172.21.15.59:0/304728430 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa008f070 con 0x7fef98006db0 2024-04-23T14:31:33.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9803ae60 con 0x7fef98006db0 2024-04-23T14:31:33.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7f3da5400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3da806f850 0x7f3da80722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7fef96a00640 1 -- 172.21.15.59:0/304728430 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa006bbe0 con 0x7fef98006db0 2024-04-23T14:31:33.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7fef96a00640 1 -- 172.21.15.59:0/304728430 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa006be10 con 0x7fef98006db0 2024-04-23T14:31:33.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7f3da5400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3da806f850 0x7f3da80722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39590/0 (socket says 172.21.15.59:39590) 2024-04-23T14:31:33.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7f3da5400640 1 -- 172.21.15.59:0/1730719933 learned_addr learned my addr 172.21.15.59:0/1730719933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7f3da5e00640 1 --2- 172.21.15.59:0/1730719933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da80b3950 0x7f3da8071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9803b420 con 0x7fef98006db0 2024-04-23T14:31:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f1786000640 1 -- 172.21.15.59:0/3664318869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f178801a070 con 0x7f179015ea40 2024-04-23T14:31:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.373+0000 7f1797200640 1 -- 172.21.15.59:0/3664318869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179004fd90 con 0x7f179015ea40 2024-04-23T14:31:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.376+0000 7f1786000640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f177406caf0 0x7f177406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fe4aa600640 1 Processor -- start 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fe4aa600640 1 -- start start 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 <== mon.1 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2951537460 0 0) 0x7fd008003520 con 0x7fd0200b34a0 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 msgr2=0x7fd02015ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 0x7fd02015ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 >> v2:172.21.15.76:3300/0 conn(0x7fd02006c4f0 msgr2=0x7fd02015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 --2- 172.21.15.59:0/1430774028 >> v2:172.21.15.76:3300/0 conn(0x7fd02006c4f0 0x7fd02015eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02015be20 con 0x7fd0200b34a0 2024-04-23T14:31:33.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 <== mon.1 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fd00c001670 con 0x7fd0200b34a0 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 >> v1:172.21.15.76:6789/0 conn(0x7fd0200b34a0 legacy=0x7fd0200d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080039b0 0x7fd008003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd008004800 0x7fd008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 0x7fd0080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd02015bb40 con 0x7fd008004800 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd00800b9c0 con 0x7fd0080039b0 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd00800bb70 con 0x7fd0200d08e0 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7f3da5e00640 1 -- 172.21.15.59:0/1730719933 >> v2:172.21.15.76:3300/0 conn(0x7f3da806f850 msgr2=0x7f3da80722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7f3da5e00640 1 --2- 172.21.15.59:0/1730719933 >> v2:172.21.15.76:3300/0 conn(0x7f3da806f850 0x7f3da80722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7f3da5e00640 1 -- 172.21.15.59:0/1730719933 >> v1:172.21.15.59:6789/0 conn(0x7f3da808b760 legacy=0x7f3da806e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7f3da5e00640 1 -- 172.21.15.59:0/1730719933 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da80b3eb0 con 0x7f3da80b3950 2024-04-23T14:31:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f0ddcc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0dd815f880 0x7f0dd80d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36534/0 (socket says 172.21.15.59:36534) 2024-04-23T14:31:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.377+0000 7f0ddcc00640 1 -- 172.21.15.59:0/1470529153 learned_addr learned my addr 172.21.15.59:0/1470529153 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f0dd7400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0dd815ea40 0x7f0dd80d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43524/0 (socket says 172.21.15.59:43524) 2024-04-23T14:31:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.375+0000 7f0dd7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd80d0f80 0x7f0dd80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.378+0000 7fd025600640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 0x7fd0080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.378+0000 7f1786000640 1 -- 172.21.15.59:0/3664318869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f17880565c0 con 0x7f179015ea40 2024-04-23T14:31:33.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.378+0000 7fef9f400640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980e3300 0x7fef98052190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:33.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fe4aa600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a40d02f0 0x7fe4a4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fe4aa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a40d2bc0 0x7fe4a4071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fe4aa600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a415ee30 0x7fe4a40b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fe4a3400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a40d02f0 0x7fe4a4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fe4aa600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a415f210 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fe4aa600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a40d39d0 con 0x7fe4a415ee30 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fe4aa600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a415f390 con 0x7fe4a40d02f0 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1443708352 0 0) 0x7f0dd80cfc10 con 0x7f0dd815ea40 2024-04-23T14:31:33.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dbc003520 con 0x7f0dd815ea40 2024-04-23T14:31:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.378+0000 7fcfe6400640 1 Processor -- start 2024-04-23T14:31:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.378+0000 7fcfe6400640 1 -- start start 2024-04-23T14:31:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fe4a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a40d2bc0 0x7fe4a4071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fd025600640 1 -- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080039b0 msgr2=0x7fd008003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fd025600640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080039b0 0x7fd008003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fd025600640 1 -- 172.21.15.59:0/1430774028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd008004800 msgr2=0x7fd008004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fd025600640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd008004800 0x7fd008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.379+0000 7fd025600640 1 -- 172.21.15.59:0/1430774028 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02015be20 con 0x7fd0200d08e0 2024-04-23T14:31:33.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fe4a2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a40d2bc0 0x7fe4a4071990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33344/0 (socket says 172.21.15.59:33344) 2024-04-23T14:31:33.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fe4a2a00640 1 -- 172.21.15.59:0/865655142 learned_addr learned my addr 172.21.15.59:0/865655142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 <== mon.1 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3700465804 0 0) 0x7f0dbc003520 con 0x7f0dd815ea40 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd80d0f80 msgr2=0x7f0dd80cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd80d0f80 0x7f0dd80cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 >> v1:172.21.15.59:6789/0 conn(0x7f0dd815f880 legacy=0x7f0dd80d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd80d01d0 con 0x7f0dd815ea40 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 <== mon.1 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f0dc0001650 con 0x7f0dd815ea40 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 >> v1:172.21.15.76:6789/0 conn(0x7f0dd815ea40 legacy=0x7f0dd80d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fcfe6400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcfe00724a0 0x7fcfe0167880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd80cfef0 con 0x7f0dbc0039b0 2024-04-23T14:31:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc00c0c0 con 0x7f0dbc0051c0 2024-04-23T14:31:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fcfe6400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe015ad70 con 0x7fcfe015a910 2024-04-23T14:31:33.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fcfe6400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe00b3e90 con 0x7fcfe006fe80 2024-04-23T14:31:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fcfe6400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe00b4aa0 con 0x7fcfe00724a0 2024-04-23T14:31:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0dbc00c2c0 con 0x7f0dbc004800 2024-04-23T14:31:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7f1795e00640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f177406caf0 0x7f177406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7fd025600640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 0x7fd0080051c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd018009f90 tx=0x7fd008005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08df9d58fe630b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7fcfdfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcfe00724a0 0x7fcfe0167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7fcfdfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcfe00724a0 0x7fcfe0167880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39624/0 (socket says 172.21.15.59:39624) 2024-04-23T14:31:33.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7fcfdfe00640 1 -- 172.21.15.59:0/1952607723 learned_addr learned my addr 172.21.15.59:0/1952607723 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.380+0000 7fef96a00640 1 -- 172.21.15.59:0/304728430 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefa006d110 con 0x7fef98006db0 2024-04-23T14:31:33.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef98005710 con 0x7fef98006db0 2024-04-23T14:31:33.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fef96a00640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef8006c8f0 0x7fef8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fef96a00640 1 -- 172.21.15.59:0/304728430 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefa0106570 con 0x7fef98006db0 2024-04-23T14:31:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7fe4a2a00640 1 -- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a40d02f0 msgr2=0x7fe4a4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fe4a3e00640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a415ee30 0x7fe4a40b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018009c10 con 0x7fd0200d08e0 2024-04-23T14:31:33.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7f0dd7e00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.381+0000 7f0ddcc00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7f0dd7400640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd018036070 con 0x7fd0200d08e0 2024-04-23T14:31:33.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fd01f400640 1 -- 172.21.15.59:0/1430774028 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01802d070 con 0x7fd0200d08e0 2024-04-23T14:31:33.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fef9ea00640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef8006c8f0 0x7fef8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fe4a2a00640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a40d02f0 0x7fe4a4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fe4a2a00640 1 -- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a415ee30 msgr2=0x7fe4a40b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7f1795e00640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f177406caf0 0x7f177406efb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1780034000 tx=0x7f178002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fe4a2a00640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a415ee30 0x7fe4a40b98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7fe4a2a00640 1 -- 172.21.15.59:0/865655142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a4072390 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7fe4a3e00640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a415ee30 0x7fe4a40b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:33.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7fe4a2a00640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a40d2bc0 0x7fe4a4071990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe490009840 tx=0x7fe4900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7f0dd7e00640 1 -- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dbc004800 msgr2=0x7f0dbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7fef9ea00640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef8006c8f0 0x7fef8006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef90000f60 tx=0x7fef90007990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7fd027600640 1 -- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 msgr2=0x7fd0080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7fd027600640 1 --2- 172.21.15.59:0/1430774028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0200d08e0 0x7fd0080051c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd018009f90 tx=0x7fd008005830 comp rx=0 tx=0).stop 2024-04-23T14:31:33.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7fd025600640 1 -- 172.21.15.59:0/1430774028 reap_dead start 2024-04-23T14:31:33.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7fcfdfe00640 1 -- 172.21.15.59:0/1952607723 >> v1:172.21.15.76:6789/0 conn(0x7fcfe006fe80 legacy=0x7fcfe015b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7fcfdfe00640 1 -- 172.21.15.59:0/1952607723 >> v1:172.21.15.59:6789/0 conn(0x7fcfe015a910 legacy=0x7fcfe0071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7fcfdfe00640 1 -- 172.21.15.59:0/1952607723 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe00b4170 con 0x7fcfe00724a0 2024-04-23T14:31:33.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7f0dd7e00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7f0dd7e00640 1 -- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dbc0051c0 msgr2=0x7f0dbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7f0dd7e00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7f0dd7e00640 1 -- 172.21.15.59:0/1470529153 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc003520 con 0x7f0dbc0039b0 2024-04-23T14:31:33.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.382+0000 7f3da5e00640 1 --2- 172.21.15.59:0/1730719933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da80b3950 0x7f3da8071d90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3d940097a0 tx=0x7f3d9402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa9e4de653ac34a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7f3da4a00640 1 -- 172.21.15.59:0/1730719933 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94037070 con 0x7f3da80b3950 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7f3da4a00640 1 -- 172.21.15.59:0/1730719933 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d94032440 con 0x7f3da80b3950 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.383+0000 7f3da4a00640 1 -- 172.21.15.59:0/1730719933 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94032810 con 0x7f3da80b3950 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 -- 172.21.15.59:0/1730719933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da80b3950 msgr2=0x7f3da8071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 --2- 172.21.15.59:0/1730719933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da80b3950 0x7f3da8071d90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3d940097a0 tx=0x7f3d9402a1c0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 -- 172.21.15.59:0/1730719933 shutdown_connections 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 --2- 172.21.15.59:0/1730719933 >> v2:172.21.15.76:3300/0 conn(0x7f3da806f850 0x7f3da80722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 --2- 172.21.15.59:0/1730719933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da80b3950 0x7f3da8071d90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 -- 172.21.15.59:0/1730719933 >> 172.21.15.59:0/1730719933 conn(0x7f3da806b9d0 msgr2=0x7f3da806bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 -- 172.21.15.59:0/1730719933 shutdown_connections 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f3dad000640 1 -- 172.21.15.59:0/1730719933 wait complete. 2024-04-23T14:31:33.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f0dd7e00640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8073bd0 tx=0x7f0dbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08b1fa03969cd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f995a400640 1 Processor -- start 2024-04-23T14:31:33.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.384+0000 7f995a400640 1 -- start start 2024-04-23T14:31:33.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fd027600640 1 -- 172.21.15.59:0/1430774028 shutdown_connections 2024-04-23T14:31:33.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fd027600640 1 -- 172.21.15.59:0/1430774028 >> 172.21.15.59:0/1430774028 conn(0x7fd02006b9d0 msgr2=0x7fd02006bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0036070 con 0x7f0dbc0039b0 2024-04-23T14:31:33.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fe49b400640 1 -- 172.21.15.59:0/865655142 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490037040 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fd027600640 1 -- 172.21.15.59:0/1430774028 shutdown_connections 2024-04-23T14:31:33.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fe49b400640 1 -- 172.21.15.59:0/865655142 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe490004230 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fe49b400640 1 -- 172.21.15.59:0/865655142 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490003610 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd0031600 con 0x7f0dbc0039b0 2024-04-23T14:31:33.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fd027600640 1 -- 172.21.15.59:0/1430774028 wait complete. 2024-04-23T14:31:33.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7f0dd6a00640 1 -- 172.21.15.59:0/1470529153 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0036070 con 0x7f0dbc0039b0 2024-04-23T14:31:33.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7f1786000640 1 -- 172.21.15.59:0/3664318869 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f178808d030 con 0x7f179015ea40 2024-04-23T14:31:33.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.385+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a4072670 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7f995a400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f99540b3950 0x7f9954071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7f995a400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f995416b810 0x7f99540722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7f995a400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99540b3eb0 con 0x7f995406f850 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7f995a400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f99540d0e80 con 0x7f995416b810 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7f995a400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9954072bc0 con 0x7f99540b3950 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a4072c30 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7f0ddd600640 1 -- 172.21.15.59:0/1470529153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dbc0039b0 msgr2=0x7f0dbc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7f0ddd600640 1 --2- 172.21.15.59:0/1470529153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8073bd0 tx=0x7f0dbc005830 comp rx=0 tx=0).stop 2024-04-23T14:31:33.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.387+0000 7f3dad000640 1 Processor -- start 2024-04-23T14:31:33.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.387+0000 7f0ddcc00640 1 -- 172.21.15.59:0/1470529153 reap_dead start 2024-04-23T14:31:33.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.387+0000 7f0ddd600640 1 -- 172.21.15.59:0/1470529153 shutdown_connections 2024-04-23T14:31:33.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.387+0000 7f0ddd600640 1 -- 172.21.15.59:0/1470529153 >> 172.21.15.59:0/1470529153 conn(0x7f0dd808bf90 msgr2=0x7f0dd80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.387+0000 7f0ddd600640 1 -- 172.21.15.59:0/1470529153 shutdown_connections 2024-04-23T14:31:33.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.387+0000 7f3dad000640 1 -- start start 2024-04-23T14:31:33.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.386+0000 7fcfdfe00640 1 --2- 172.21.15.59:0/1952607723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 0x7fcfe0167880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009e50 tx=0x7fcfd802a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4a505e809df036 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.387+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a4068a80 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f9953400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f99540b3950 0x7f9954071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f9952a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f995416b810 0x7f99540722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f9952a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f995416b810 0x7f99540722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33360/0 (socket says 172.21.15.59:33360) 2024-04-23T14:31:33.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f9952a00640 1 -- 172.21.15.59:0/876786740 learned_addr learned my addr 172.21.15.59:0/876786740 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7fd027600640 1 Processor -- start 2024-04-23T14:31:33.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f0ddd600640 1 -- 172.21.15.59:0/1470529153 wait complete. 2024-04-23T14:31:33.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f9952a00640 1 -- 172.21.15.59:0/876786740 >> v2:172.21.15.76:3300/0 conn(0x7f99540b3950 msgr2=0x7f9954071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f9952a00640 1 --2- 172.21.15.59:0/876786740 >> v2:172.21.15.76:3300/0 conn(0x7f99540b3950 0x7f9954071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7f9952a00640 1 -- 172.21.15.59:0/876786740 >> v1:172.21.15.59:6789/0 conn(0x7f995406f850 legacy=0x7f995406e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:33.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.388+0000 7fd027600640 1 -- start start 2024-04-23T14:31:33.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3dad000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da806f850 0x7f3da817c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3dad000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da808b760 0x7f3da8179990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3dad000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da817a4f0 0x7f3da81637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3dad000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3da816ba80 con 0x7f3da808b760 2024-04-23T14:31:33.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3dad000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3da808b210 con 0x7f3da806f850 2024-04-23T14:31:33.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3dad000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80b4470 con 0x7f3da817a4f0 2024-04-23T14:31:33.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f9952a00640 1 -- 172.21.15.59:0/876786740 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99540b4190 con 0x7f995416b810 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3da5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da808b760 0x7f3da8179990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3da5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da808b760 0x7f3da8179990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33366/0 (socket says 172.21.15.59:33366) 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3da5400640 1 -- 172.21.15.59:0/54453334 learned_addr learned my addr 172.21.15.59:0/54453334 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.389+0000 7f3da6800640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da817a4f0 0x7f3da81637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5400640 1 -- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da817a4f0 msgr2=0x7f3da81637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5e00640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da806f850 0x7f3da817c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5400640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da817a4f0 0x7f3da81637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5400640 1 -- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da806f850 msgr2=0x7f3da817c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5400640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da806f850 0x7f3da817c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5400640 1 -- 172.21.15.59:0/54453334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da817d080 con 0x7f3da808b760 2024-04-23T14:31:33.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5e00640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da806f850 0x7f3da817c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:33.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7fd027600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0200d08e0 0x7fd0200aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7fe49b400640 1 -- 172.21.15.59:0/865655142 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe49003a070 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7f3da5400640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da808b760 0x7f3da8179990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009fd0 tx=0x7f3d9800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.390+0000 7fd027600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0200abb30 0x7fd02014c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd027600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd02014ce90 0x7fd0200ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7f0ddd600640 1 Processor -- start 2024-04-23T14:31:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd027600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd02015f2e0 con 0x7fd0200abb30 2024-04-23T14:31:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd01fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0200abb30 0x7fd02014c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd01fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0200abb30 0x7fd02014c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33374/0 (socket says 172.21.15.59:33374) 2024-04-23T14:31:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd01fe00640 1 -- 172.21.15.59:0/3002616020 learned_addr learned my addr 172.21.15.59:0/3002616020 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd02015e880 con 0x7fd0200d08e0 2024-04-23T14:31:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd02014d320 con 0x7fd02014ce90 2024-04-23T14:31:33.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7fd025600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd02014ce90 0x7fd0200ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7f9952a00640 1 --2- 172.21.15.59:0/876786740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995416b810 0x7f99540722d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f99540715b0 tx=0x7f994002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4716a721244d76a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.391+0000 7f0ddd600640 1 -- start start 2024-04-23T14:31:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7f0ddd600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dd815ab20 0x7f0dd80cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7fd025600640 1 -- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0200d08e0 msgr2=0x7fd0200aa4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:31:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7fd025600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0200d08e0 0x7fd0200aa4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7fd025600640 1 -- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0200abb30 msgr2=0x7fd02014c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7fd025600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0200abb30 0x7fd02014c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7fd025600640 1 -- 172.21.15.59:0/3002616020 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0200aaa90 con 0x7fd02014ce90 2024-04-23T14:31:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7fe49b400640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe48406c940 0x7fe48406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7f0ddd600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd815af00 0x7f0dd80b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7f0ddd600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd80ce460 0x7f0dd80b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7f0ddd600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd80cf250 con 0x7f0dd815ab20 2024-04-23T14:31:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7f0ddd600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd815fce0 con 0x7f0dd80ce460 2024-04-23T14:31:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.392+0000 7f0ddd600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd80cf3d0 con 0x7f0dd815af00 2024-04-23T14:31:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7fe49b400640 1 -- 172.21.15.59:0/865655142 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe4900ac370 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7fd025600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd02014ce90 0x7fd0200ae5e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd018032000 tx=0x7fd018009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dd815ab20 0x7f0dd80cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7fe4a3400640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe48406c940 0x7fe48406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dd815ab20 0x7f0dd80cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33388/0 (socket says 172.21.15.59:33388) 2024-04-23T14:31:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 -- 172.21.15.59:0/2444042065 learned_addr learned my addr 172.21.15.59:0/2444042065 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0ddcc00640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd80ce460 0x7f0dd80b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7400640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd815af00 0x7f0dd80b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7fd01cc00640 1 -- 172.21.15.59:0/3002616020 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018009c10 con 0x7fd02014ce90 2024-04-23T14:31:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 -- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd815af00 msgr2=0x7f0dd80b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0200aad70 con 0x7fd02014ce90 2024-04-23T14:31:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.394+0000 7fd01cc00640 1 -- 172.21.15.59:0/3002616020 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd018045430 con 0x7fd02014ce90 2024-04-23T14:31:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.394+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0200ab330 con 0x7fd02014ce90 2024-04-23T14:31:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.394+0000 7fd01cc00640 1 -- 172.21.15.59:0/3002616020 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0180457e0 con 0x7fd02014ce90 2024-04-23T14:31:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd815af00 0x7f0dd80b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 -- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd80ce460 msgr2=0x7f0dd80b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd80ce460 0x7f0dd80b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 -- 172.21.15.59:0/2444042065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd80cc6b0 con 0x7f0dd815ab20 2024-04-23T14:31:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.393+0000 7f0dd7e00640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dd815ab20 0x7f0dd80cdf20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009fd0 tx=0x7f0dd0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.394+0000 7fe4a3400640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe48406c940 0x7fe48406ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe494000f60 tx=0x7fe494009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.396+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0200abf10 con 0x7fd02014ce90 2024-04-23T14:31:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.396+0000 7fd01cc00640 1 -- 172.21.15.59:0/3002616020 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd018005d10 con 0x7fd02014ce90 2024-04-23T14:31:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f9952000640 1 -- 172.21.15.59:0/876786740 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037070 con 0x7f995416b810 2024-04-23T14:31:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f3d9e000640 1 -- 172.21.15.59:0/54453334 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98017070 con 0x7f3da808b760 2024-04-23T14:31:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f3d9e000640 1 -- 172.21.15.59:0/54453334 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9800c940 con 0x7f3da808b760 2024-04-23T14:31:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f3d9e000640 1 -- 172.21.15.59:0/54453334 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800cd10 con 0x7f3da808b760 2024-04-23T14:31:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7fd01cc00640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd00006c8f0 0x7fd00006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7fd01cc00640 1 -- 172.21.15.59:0/3002616020 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd0180ace60 con 0x7fd02014ce90 2024-04-23T14:31:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f0dcfe00640 1 -- 172.21.15.59:0/2444042065 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0040020 con 0x7f0dd815ab20 2024-04-23T14:31:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f0dcfe00640 1 -- 172.21.15.59:0/2444042065 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd0030750 con 0x7f0dd815ab20 2024-04-23T14:31:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f0dcfe00640 1 -- 172.21.15.59:0/2444042065 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0036040 con 0x7f0dd815ab20 2024-04-23T14:31:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.398+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd80cc950 con 0x7f0dd815ab20 2024-04-23T14:31:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.398+0000 7fe49b400640 1 -- 172.21.15.59:0/865655142 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe490041020 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.398+0000 7f9952000640 1 -- 172.21.15.59:0/876786740 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9940032440 con 0x7f995416b810 2024-04-23T14:31:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.398+0000 7f9952000640 1 -- 172.21.15.59:0/876786740 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037070 con 0x7f995416b810 2024-04-23T14:31:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.397+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da817d360 con 0x7f3da808b760 2024-04-23T14:31:33.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.398+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da8162fd0 con 0x7f3da808b760 2024-04-23T14:31:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.398+0000 7fd024c00640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd00006c8f0 0x7fd00006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.398+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd80ccf10 con 0x7f0dd815ab20 2024-04-23T14:31:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.399+0000 7f995a400640 1 -- 172.21.15.59:0/876786740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995416b810 msgr2=0x7f99540722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.399+0000 7f995a400640 1 --2- 172.21.15.59:0/876786740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995416b810 0x7f99540722d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f99540715b0 tx=0x7f994002a290 comp rx=0 tx=0).stop 2024-04-23T14:31:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.399+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd804fd90 con 0x7f0dd815ab20 2024-04-23T14:31:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.399+0000 7fd024c00640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd00006c8f0 0x7fd00006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd0140047b0 tx=0x7fd014008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.400+0000 7f995a400640 1 -- 172.21.15.59:0/876786740 shutdown_connections 2024-04-23T14:31:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.400+0000 7f995a400640 1 --2- 172.21.15.59:0/876786740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995416b810 0x7f99540722d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.400+0000 7f995a400640 1 --2- 172.21.15.59:0/876786740 >> v2:172.21.15.76:3300/0 conn(0x7f99540b3950 0x7f9954071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.400+0000 7f995a400640 1 -- 172.21.15.59:0/876786740 >> 172.21.15.59:0/876786740 conn(0x7f995406b9d0 msgr2=0x7f995406bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.400+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da8068a80 con 0x7f3da808b760 2024-04-23T14:31:33.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.401+0000 7f995a400640 1 -- 172.21.15.59:0/876786740 shutdown_connections 2024-04-23T14:31:33.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.401+0000 7f995a400640 1 -- 172.21.15.59:0/876786740 wait complete. 2024-04-23T14:31:33.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.401+0000 7f0dcfe00640 1 -- 172.21.15.59:0/2444042065 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dd0005d10 con 0x7f0dd815ab20 2024-04-23T14:31:33.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7f995a400640 1 Processor -- start 2024-04-23T14:31:33.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7fcfde000640 1 -- 172.21.15.59:0/1952607723 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8036070 con 0x7fcfe00724a0 2024-04-23T14:31:33.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7fcfde000640 1 -- 172.21.15.59:0/1952607723 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd8031600 con 0x7fcfe00724a0 2024-04-23T14:31:33.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7fcfde000640 1 -- 172.21.15.59:0/1952607723 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80319f0 con 0x7fcfe00724a0 2024-04-23T14:31:33.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7fcfe6400640 1 -- 172.21.15.59:0/1952607723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 msgr2=0x7fcfe0167880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7fcfe6400640 1 --2- 172.21.15.59:0/1952607723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 0x7fcfe0167880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009e50 tx=0x7fcfd802a7c0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7f995a400640 1 -- start start 2024-04-23T14:31:33.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7fcfe6400640 1 -- 172.21.15.59:0/1952607723 shutdown_connections 2024-04-23T14:31:33.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7fcfe6400640 1 --2- 172.21.15.59:0/1952607723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 0x7fcfe0167880 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7fcfe6400640 1 -- 172.21.15.59:0/1952607723 >> 172.21.15.59:0/1952607723 conn(0x7fcfe00b2130 msgr2=0x7fcfe00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7fcfe6400640 1 -- 172.21.15.59:0/1952607723 shutdown_connections 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7fcfe6400640 1 -- 172.21.15.59:0/1952607723 wait complete. 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7f995a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995406f850 0x7f995415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7f995a400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f995416b810 0x7f99540b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7f995a400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99540b2650 0x7f995415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7f995a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f995408b210 con 0x7f995406f850 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7f995a400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f99540b3430 con 0x7f99540b2650 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7f995a400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f995408b760 con 0x7f995416b810 2024-04-23T14:31:33.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7fef96a00640 1 -- 172.21.15.59:0/304728430 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefa00d49a0 con 0x7fef98006db0 2024-04-23T14:31:33.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7f0dcfe00640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0db406c970 0x7f0db406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7f0dcfe00640 1 -- 172.21.15.59:0/2444042065 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0dd00acdf0 con 0x7f0dd815ab20 2024-04-23T14:31:33.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.403+0000 7f0dd7400640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0db406c970 0x7f0db406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.404+0000 7f0dd7400640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0db406c970 0x7f0db406ee30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80b9cc0 tx=0x7f0dc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.406+0000 7f9952a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f995416b810 0x7f99540b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.406+0000 7f9952a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f995416b810 0x7f99540b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51498/0 (socket says 172.21.15.59:51498) 2024-04-23T14:31:33.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.406+0000 7f9952a00640 1 -- 172.21.15.59:0/1900084027 learned_addr learned my addr 172.21.15.59:0/1900084027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.406+0000 7f9953e00640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99540b2650 0x7f995415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.406+0000 7f9953400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995406f850 0x7f995415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7fcfe6400640 1 Processor -- start 2024-04-23T14:31:33.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7f9953e00640 1 -- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f995416b810 msgr2=0x7f99540b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7f9953e00640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f995416b810 0x7f99540b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7f9953e00640 1 -- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995406f850 msgr2=0x7f995415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7f9953e00640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995406f850 0x7f995415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7f9953e00640 1 -- 172.21.15.59:0/1900084027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99540b2c20 con 0x7f99540b2650 2024-04-23T14:31:33.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7f9953400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995406f850 0x7f995415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:31:33.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.405+0000 7f3d9e000640 1 -- 172.21.15.59:0/54453334 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d9801a070 con 0x7f3da808b760 2024-04-23T14:31:33.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7f9953e00640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99540b2650 0x7f995415b420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f994400c3c0 tx=0x7f994400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7f994b400640 1 -- 172.21.15.59:0/1900084027 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99440052d0 con 0x7f99540b2650 2024-04-23T14:31:33.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.407+0000 7fcfe6400640 1 -- start start 2024-04-23T14:31:33.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7f994b400640 1 -- 172.21.15.59:0/1900084027 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9944005470 con 0x7f99540b2650 2024-04-23T14:31:33.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7f994b400640 1 -- 172.21.15.59:0/1900084027 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944007220 con 0x7f99540b2650 2024-04-23T14:31:33.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7f3d9e000640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8806c940 0x7f3d8806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7fcfe6400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfe006fe80 0x7fcfe014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7fcfe6400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 0x7fcfe0155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7fcfe6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfe015a910 0x7fcfe015a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7fcfe6400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00b3b80 con 0x7fcfe015a910 2024-04-23T14:31:33.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7fcfe6400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00b4730 con 0x7fcfe00724a0 2024-04-23T14:31:33.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.408+0000 7fcfe6400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe00b3550 con 0x7fcfe006fe80 2024-04-23T14:31:33.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.409+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9954158b10 con 0x7f99540b2650 2024-04-23T14:31:33.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.409+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9954159160 con 0x7f99540b2650 2024-04-23T14:31:33.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.409+0000 7f3d9e000640 1 -- 172.21.15.59:0/54453334 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d9808bcb0 con 0x7f3da808b760 2024-04-23T14:31:33.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfe015a910 0x7fcfe015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfe015a910 0x7fcfe015a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33402/0 (socket says 172.21.15.59:33402) 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 -- 172.21.15.59:0/2842936269 learned_addr learned my addr 172.21.15.59:0/2842936269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdf400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfe006fe80 0x7fcfe014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 -- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfe006fe80 msgr2=0x7fcfe014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfe006fe80 0x7fcfe014ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 -- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 msgr2=0x7fcfe0155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 0x7fcfe0155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.410+0000 7fcfdfe00640 1 -- 172.21.15.59:0/2842936269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe014f020 con 0x7fcfe015a910 2024-04-23T14:31:33.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.411+0000 7f3da5e00640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8806c940 0x7f3d8806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.411+0000 7fcfdf400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfe006fe80 0x7fcfe014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:33.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.412+0000 7fcfdfe00640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfe015a910 0x7fcfe015a150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802a800 tx=0x7fcfd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.412+0000 7f994b400640 1 -- 172.21.15.59:0/1900084027 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99440073c0 con 0x7f99540b2650 2024-04-23T14:31:33.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.413+0000 7f994b400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f993406c8f0 0x7f993406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.413+0000 7f994b400640 1 -- 172.21.15.59:0/1900084027 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f994408adf0 con 0x7f99540b2650 2024-04-23T14:31:33.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.413+0000 7f3da5e00640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8806c940 0x7f3d8806ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3d94034000 tx=0x7f3d9402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.414+0000 7f0dcfe00640 1 -- 172.21.15.59:0/2444042065 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0dd004a070 con 0x7f0dd815ab20 2024-04-23T14:31:33.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.414+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9954068a80 con 0x7f99540b2650 2024-04-23T14:31:33.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.414+0000 7f9953400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f993406c8f0 0x7f993406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.415+0000 7f9953400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f993406c8f0 0x7f993406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e740 tx=0x7f994c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.416+0000 7fd01cc00640 1 -- 172.21.15.59:0/3002616020 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd01807b2a0 con 0x7fd02014ce90 2024-04-23T14:31:33.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.417+0000 7fcfd7400640 1 -- 172.21.15.59:0/2842936269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd803f030 con 0x7fcfe015a910 2024-04-23T14:31:33.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.417+0000 7fcfd7400640 1 -- 172.21.15.59:0/2842936269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd8031400 con 0x7fcfe015a910 2024-04-23T14:31:33.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.417+0000 7fcfd7400640 1 -- 172.21.15.59:0/2842936269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8036040 con 0x7fcfe015a910 2024-04-23T14:31:33.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.418+0000 7f3d9e000640 1 -- 172.21.15.59:0/54453334 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d9805a250 con 0x7f3da808b760 2024-04-23T14:31:33.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.418+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe014f300 con 0x7fcfe015a910 2024-04-23T14:31:33.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.419+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe014f8c0 con 0x7fcfe015a910 2024-04-23T14:31:33.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.421+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe0068a80 con 0x7fcfe015a910 2024-04-23T14:31:33.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.422+0000 7fcfd7400640 1 -- 172.21.15.59:0/2842936269 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfd8042070 con 0x7fcfe015a910 2024-04-23T14:31:33.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.423+0000 7fcfd7400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcfc006c940 0x7fcfc006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:33.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.424+0000 7fcfd7400640 1 -- 172.21.15.59:0/2842936269 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcfd80aba10 con 0x7fcfe015a910 2024-04-23T14:31:33.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.425+0000 7fcfdf400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcfc006c940 0x7fcfc006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:33.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.426+0000 7fcfdf400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcfc006c940 0x7fcfc006ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0007fd0 tx=0x7fcfd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:33.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.430+0000 7f994b400640 1 -- 172.21.15.59:0/1900084027 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9944058bd0 con 0x7f99540b2650 2024-04-23T14:31:33.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.433+0000 7fcfd7400640 1 -- 172.21.15.59:0/2842936269 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfd8079eb0 con 0x7fcfe015a910 2024-04-23T14:31:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.841+0000 7f1797200640 1 -- 172.21.15.59:0/3664318869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f17900ce950 con 0x7f179015ea40 2024-04-23T14:31:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.842+0000 7f1786000640 1 -- 172.21.15.59:0/3664318869 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1788058c20 con 0x7f179015ea40 2024-04-23T14:31:33.846 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-23T14:31:33.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.846+0000 7f177b400640 1 -- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f177406caf0 msgr2=0x7f177406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.846+0000 7f177b400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f177406caf0 0x7f177406efb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1780034000 tx=0x7f178002e000 comp rx=0 tx=0).stop 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.846+0000 7f177b400640 1 -- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f179015ea40 msgr2=0x7f17900b7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.846+0000 7f177b400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f179015ea40 0x7f17900b7e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f178800c4b0 tx=0x7f178800c970 comp rx=0 tx=0).stop 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.847+0000 7f177b400640 1 -- 172.21.15.59:0/3664318869 shutdown_connections 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.847+0000 7f177b400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f177406caf0 0x7f177406efb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.847+0000 7f177b400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f179015f880 0x7f17900b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.847+0000 7f177b400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f179015ea40 0x7f17900b7e60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.847+0000 7f177b400640 1 --2- 172.21.15.59:0/3664318869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17900d0f80 0x7f17900b9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.847+0000 7f177b400640 1 -- 172.21.15.59:0/3664318869 >> 172.21.15.59:0/3664318869 conn(0x7f179008bf90 msgr2=0x7f1790160280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.848+0000 7f177b400640 1 -- 172.21.15.59:0/3664318869 shutdown_connections 2024-04-23T14:31:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.848+0000 7f177b400640 1 -- 172.21.15.59:0/3664318869 wait complete. 2024-04-23T14:31:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.852+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe4a40d29a0 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.853+0000 7fe49b400640 1 -- 172.21.15.59:0/865655142 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe49007a140 con 0x7fe4a40d2bc0 2024-04-23T14:31:33.857 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.857+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fef98005fd0 con 0x7fef98006db0 2024-04-23T14:31:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.857+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe48406c940 msgr2=0x7fe48406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.857+0000 7fe4aa600640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe48406c940 0x7fe48406ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe494000f60 tx=0x7fe494009040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.857+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a40d2bc0 msgr2=0x7fe4a4071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.857+0000 7fe4aa600640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a40d2bc0 0x7fe4a4071990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe490009840 tx=0x7fe4900046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.858+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 shutdown_connections 2024-04-23T14:31:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.858+0000 7fe4aa600640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe48406c940 0x7fe48406ee00 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe494000f60 tx=0x7fe494009040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.858+0000 7fef96a00640 1 -- 172.21.15.59:0/304728430 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefa00d42a0 con 0x7fef98006db0 2024-04-23T14:31:33.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.858+0000 7fe4aa600640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a415ee30 0x7fe4a40b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.858+0000 7fe4aa600640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a40d2bc0 0x7fe4a4071990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.858+0000 7fe4aa600640 1 --2- 172.21.15.59:0/865655142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a40d02f0 0x7fe4a4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.861 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.858+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 >> 172.21.15.59:0/865655142 conn(0x7fe4a406cc70 msgr2=0x7fe4a415d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.862+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 shutdown_connections 2024-04-23T14:31:33.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.862+0000 7fe4aa600640 1 -- 172.21.15.59:0/865655142 wait complete. 2024-04-23T14:31:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.863+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef8006c8f0 msgr2=0x7fef8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.863+0000 7fefa5600640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef8006c8f0 0x7fef8006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef90000f60 tx=0x7fef90007990 comp rx=0 tx=0).stop 2024-04-23T14:31:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 msgr2=0x7fef9803a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef9803a230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefa00635c0 tx=0x7fefa006df90 comp rx=0 tx=0).stop 2024-04-23T14:31:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 shutdown_connections 2024-04-23T14:31:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef8006c8f0 0x7fef8006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980e3300 0x7fef98052190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef98006db0 0x7fef9803a230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 --2- 172.21.15.59:0/304728430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fef980061f0 0x7fef9817ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.864+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 >> 172.21.15.59:0/304728430 conn(0x7fef98058ef0 msgr2=0x7fef98007200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.867+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 shutdown_connections 2024-04-23T14:31:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.867+0000 7fefa5600640 1 -- 172.21.15.59:0/304728430 wait complete. 2024-04-23T14:31:33.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-23T14:31:33.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.873+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd020068a80 con 0x7fd02014ce90 2024-04-23T14:31:33.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.873+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0dd80d1ec0 con 0x7f0dd815ab20 2024-04-23T14:31:33.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.874+0000 7f0dcfe00640 1 -- 172.21.15.59:0/2444042065 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0dd007b170 con 0x7f0dd815ab20 2024-04-23T14:31:33.876 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-23T14:31:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.874+0000 7fd01cc00640 1 -- 172.21.15.59:0/3002616020 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd01807aba0 con 0x7fd02014ce90 2024-04-23T14:31:33.876 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.875+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3da80b1ea0 con 0x7f3da808b760 2024-04-23T14:31:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.876+0000 7f3d9e000640 1 -- 172.21.15.59:0/54453334 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d98059b50 con 0x7f3da808b760 2024-04-23T14:31:33.878 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-23T14:31:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0db406c970 msgr2=0x7f0db406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0db406c970 0x7f0db406ee30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80b9cc0 tx=0x7f0dc0009040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dd815ab20 msgr2=0x7f0dd80cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dd815ab20 0x7f0dd80cdf20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009fd0 tx=0x7f0dd0009750 comp rx=0 tx=0).stop 2024-04-23T14:31:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 shutdown_connections 2024-04-23T14:31:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0db406c970 0x7f0db406ee30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0dd815af00 0x7f0dd80b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dd80ce460 0x7f0dd80b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 --2- 172.21.15.59:0/2444042065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dd815ab20 0x7f0dd80cdf20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.877+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 >> 172.21.15.59:0/2444042065 conn(0x7f0dd808bf90 msgr2=0x7f0dd80b9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd00006c8f0 msgr2=0x7fd00006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd00006c8f0 0x7fd00006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd0140047b0 tx=0x7fd014008040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd02014ce90 msgr2=0x7fd0200ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd02014ce90 0x7fd0200ae5e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd018032000 tx=0x7fd018009750 comp rx=0 tx=0).stop 2024-04-23T14:31:33.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 shutdown_connections 2024-04-23T14:31:33.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 shutdown_connections 2024-04-23T14:31:33.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7f0ddd600640 1 -- 172.21.15.59:0/2444042065 wait complete. 2024-04-23T14:31:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd00006c8f0 0x7fd00006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd02014ce90 0x7fd0200ae5e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0200abb30 0x7fd02014c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 --2- 172.21.15.59:0/3002616020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0200d08e0 0x7fd0200aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.878+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 >> 172.21.15.59:0/3002616020 conn(0x7fd02006b9d0 msgr2=0x7fd02006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.879+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 shutdown_connections 2024-04-23T14:31:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.879+0000 7fd027600640 1 -- 172.21.15.59:0/3002616020 wait complete. 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8806c940 msgr2=0x7f3d8806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8806c940 0x7f3d8806ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3d94034000 tx=0x7f3d9402e000 comp rx=0 tx=0).stop 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da808b760 msgr2=0x7f3da8179990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da808b760 0x7f3da8179990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009fd0 tx=0x7f3d9800ed40 comp rx=0 tx=0).stop 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 shutdown_connections 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8806c940 0x7f3d8806ee00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da817a4f0 0x7f3da81637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da808b760 0x7f3da8179990 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009fd0 tx=0x7f3d9800ed40 comp rx=0 tx=0).stop 2024-04-23T14:31:33.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 --2- 172.21.15.59:0/54453334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da806f850 0x7f3da817c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.880+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 >> 172.21.15.59:0/54453334 conn(0x7f3da806b9d0 msgr2=0x7f3da816bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.882+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 shutdown_connections 2024-04-23T14:31:33.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.882+0000 7f3dad000640 1 -- 172.21.15.59:0/54453334 wait complete. 2024-04-23T14:31:33.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.883+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcfe006f0a0 con 0x7fcfe015a910 2024-04-23T14:31:33.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.884+0000 7fcfd7400640 1 -- 172.21.15.59:0/2842936269 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcfd80797b0 con 0x7fcfe015a910 2024-04-23T14:31:33.886 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:33.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.889+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f995415b180 con 0x7f99540b2650 2024-04-23T14:31:33.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.889+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcfc006c940 msgr2=0x7fcfc006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.889+0000 7fcfe6400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcfc006c940 0x7fcfc006ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0007fd0 tx=0x7fcfd0009040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-04-23T14:31:33.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfe015a910 msgr2=0x7fcfe015a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfe015a910 0x7fcfe015a150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802a800 tx=0x7fcfd8004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 shutdown_connections 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcfc006c940 0x7fcfc006ee00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfe015a910 0x7fcfe015a150 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfe00724a0 0x7fcfe0155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 --2- 172.21.15.59:0/2842936269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfe006fe80 0x7fcfe014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 >> 172.21.15.59:0/2842936269 conn(0x7fcfe00b2130 msgr2=0x7fcfe015e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.890+0000 7f994b400640 1 -- 172.21.15.59:0/1900084027 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f994405ced0 con 0x7f99540b2650 2024-04-23T14:31:33.892 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2024-04-23T14:31:33.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.891+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 shutdown_connections 2024-04-23T14:31:33.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.891+0000 7fcfe6400640 1 -- 172.21.15.59:0/2842936269 wait complete. 2024-04-23T14:31:33.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-23T14:31:33.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f993406c8f0 msgr2=0x7f993406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f993406c8f0 0x7f993406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e740 tx=0x7f994c00b040 comp rx=0 tx=0).stop 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99540b2650 msgr2=0x7f995415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99540b2650 0x7f995415b420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f994400c3c0 tx=0x7f994400c880 comp rx=0 tx=0).stop 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 shutdown_connections 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f993406c8f0 0x7f993406edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f995416b810 0x7f99540b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99540b2650 0x7f995415b420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 --2- 172.21.15.59:0/1900084027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f995406f850 0x7f995415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.894+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 >> 172.21.15.59:0/1900084027 conn(0x7f995406b9d0 msgr2=0x7f995415ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.895+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 shutdown_connections 2024-04-23T14:31:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:33.895+0000 7f995a400640 1 -- 172.21.15.59:0/1900084027 wait complete. 2024-04-23T14:31:33.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-23T14:31:33.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-23T14:31:33.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-23T14:31:33.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-23T14:31:33.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-23T14:31:34.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-23T14:31:34.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-23T14:31:34.895 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-23T14:31:34.902 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-23T14:31:34.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-23T14:31:34.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-23T14:31:34.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-23T14:31:34.917 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-23T14:31:35.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.170+0000 7faad7a00640 1 Processor -- start 2024-04-23T14:31:35.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.170+0000 7faad7a00640 1 -- start start 2024-04-23T14:31:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.176+0000 7faad7a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac80e3300 0x7faac80572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.176+0000 7faad7a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faac8053750 0x7faac8053bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.177+0000 7faad7a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac80e36e0 con 0x7faac8054110 2024-04-23T14:31:35.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.177+0000 7faad7a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faac8054560 con 0x7faac8053750 2024-04-23T14:31:35.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.177+0000 7faad7a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faac8057b40 con 0x7faac80e3300 2024-04-23T14:31:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.180+0000 7faad5000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac80e3300 0x7faac80572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.180+0000 7faad5a00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faac8054110 0x7faac80061f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43560/0 (socket says 172.21.15.59:43560) 2024-04-23T14:31:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.180+0000 7faad5a00640 1 -- 172.21.15.59:0/2646740943 learned_addr learned my addr 172.21.15.59:0/2646740943 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.180+0000 7faacfe00640 1 --2- 172.21.15.59:0/2646740943 >> v2:172.21.15.76:3300/0 conn(0x7faac8053750 0x7faac8053bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.181+0000 7faad5000640 1 -- 172.21.15.59:0/2646740943 >> v2:172.21.15.76:3300/0 conn(0x7faac8053750 msgr2=0x7faac8053bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.181+0000 7faad5000640 1 --2- 172.21.15.59:0/2646740943 >> v2:172.21.15.76:3300/0 conn(0x7faac8053750 0x7faac8053bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.181+0000 7faad5000640 1 -- 172.21.15.59:0/2646740943 >> v1:172.21.15.76:6789/0 conn(0x7faac8054110 legacy=0x7faac80061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.181+0000 7faad5000640 1 -- 172.21.15.59:0/2646740943 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac8057e20 con 0x7faac80e3300 2024-04-23T14:31:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.183+0000 7faad5000640 1 --2- 172.21.15.59:0/2646740943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac80e3300 0x7faac80572f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faabc0097a0 tx=0x7faabc02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8beb36edcf423b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.184+0000 7faacf400640 1 -- 172.21.15.59:0/2646740943 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc037070 con 0x7faac80e3300 2024-04-23T14:31:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.184+0000 7faacf400640 1 -- 172.21.15.59:0/2646740943 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faabc032440 con 0x7faac80e3300 2024-04-23T14:31:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.185+0000 7faad7a00640 1 -- 172.21.15.59:0/2646740943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac80e3300 msgr2=0x7faac80572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.186+0000 7faacf400640 1 -- 172.21.15.59:0/2646740943 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc037070 con 0x7faac80e3300 2024-04-23T14:31:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.186+0000 7faad7a00640 1 --2- 172.21.15.59:0/2646740943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac80e3300 0x7faac80572f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faabc0097a0 tx=0x7faabc02a130 comp rx=0 tx=0).stop 2024-04-23T14:31:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.186+0000 7faad7a00640 1 -- 172.21.15.59:0/2646740943 shutdown_connections 2024-04-23T14:31:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.186+0000 7faad7a00640 1 --2- 172.21.15.59:0/2646740943 >> v2:172.21.15.76:3300/0 conn(0x7faac8053750 0x7faac8053bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.186+0000 7faad7a00640 1 --2- 172.21.15.59:0/2646740943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac80e3300 0x7faac80572f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.186+0000 7faad7a00640 1 -- 172.21.15.59:0/2646740943 >> 172.21.15.59:0/2646740943 conn(0x7faac8058ef0 msgr2=0x7faac8059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.188+0000 7faad7a00640 1 -- 172.21.15.59:0/2646740943 shutdown_connections 2024-04-23T14:31:35.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.189+0000 7faad7a00640 1 -- 172.21.15.59:0/2646740943 wait complete. 2024-04-23T14:31:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.191+0000 7faad7a00640 1 Processor -- start 2024-04-23T14:31:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.196+0000 7faad7a00640 1 -- start start 2024-04-23T14:31:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.197+0000 7faad7a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac8053750 0x7faac817ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.197+0000 7faad7a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac8054110 0x7faac80528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.197+0000 7faad7a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac80e3300 0x7faac80d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.197+0000 7faad7a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faac8057830 con 0x7faac80e3300 2024-04-23T14:31:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.198+0000 7faad7a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faac80583e0 con 0x7faac8054110 2024-04-23T14:31:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.198+0000 7faad7a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faac803d130 con 0x7faac8053750 2024-04-23T14:31:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.199+0000 7faad5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac80e3300 0x7faac80d94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.199+0000 7faad5000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac8053750 0x7faac817ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.199+0000 7faacfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac8054110 0x7faac80528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.199+0000 7faad5000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac8053750 0x7faac817ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51520/0 (socket says 172.21.15.59:51520) 2024-04-23T14:31:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.199+0000 7faad5000640 1 -- 172.21.15.59:0/1609513084 learned_addr learned my addr 172.21.15.59:0/1609513084 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.200+0000 7faad5000640 1 -- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac8054110 msgr2=0x7faac80528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.200+0000 7faad5000640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac8054110 0x7faac80528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.200+0000 7faad5000640 1 -- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac80e3300 msgr2=0x7faac80d94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.200+0000 7faad5000640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac80e3300 0x7faac80d94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.200+0000 7faad5000640 1 -- 172.21.15.59:0/1609513084 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac8051ca0 con 0x7faac8053750 2024-04-23T14:31:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.200+0000 7faad5a00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac80e3300 0x7faac80d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.201+0000 7faacfe00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac8054110 0x7faac80528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.202+0000 7faad5000640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac8053750 0x7faac817ce60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faabc000c00 tx=0x7faabc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.202+0000 7faaccc00640 1 -- 172.21.15.59:0/1609513084 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc037050 con 0x7faac8053750 2024-04-23T14:31:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.202+0000 7faaccc00640 1 -- 172.21.15.59:0/1609513084 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faabc043070 con 0x7faac8053750 2024-04-23T14:31:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.202+0000 7faaccc00640 1 -- 172.21.15.59:0/1609513084 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc0327f0 con 0x7faac8053750 2024-04-23T14:31:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.204+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac80e3d60 con 0x7faac8053750 2024-04-23T14:31:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.204+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faac80e4320 con 0x7faac8053750 2024-04-23T14:31:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.207+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac8005710 con 0x7faac8053750 2024-04-23T14:31:35.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.207+0000 7faaccc00640 1 -- 172.21.15.59:0/1609513084 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faabc032990 con 0x7faac8053750 2024-04-23T14:31:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.209+0000 7faaccc00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faab006c8f0 0x7faab006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.209+0000 7faaccc00640 1 -- 172.21.15.59:0/1609513084 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faabc0ac800 con 0x7faac8053750 2024-04-23T14:31:35.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.209+0000 7faacfe00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faab006c8f0 0x7faab006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.210+0000 7faacfe00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faab006c8f0 0x7faab006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faac80dfc30 tx=0x7faac4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.218+0000 7fc975200640 1 Processor -- start 2024-04-23T14:31:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.218+0000 7f915d200640 1 Processor -- start 2024-04-23T14:31:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.218+0000 7fc975200640 1 -- start start 2024-04-23T14:31:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.218+0000 7f915d200640 1 -- start start 2024-04-23T14:31:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.218+0000 7fae41600640 1 Processor -- start 2024-04-23T14:31:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.219+0000 7f915d200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f91580d1840 0x7f91581698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7f915d200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91580d2560 con 0x7f91580d05f0 2024-04-23T14:31:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7f915d200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91580d2840 con 0x7f915815e890 2024-04-23T14:31:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7f915d200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f91580d2310 con 0x7f91580d1840 2024-04-23T14:31:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7fc975200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc96803c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7fc975200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc9680e78d0 0x7fc968053d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7fc975200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc968058980 con 0x7fc9680e3040 2024-04-23T14:31:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7fc975200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc968006860 con 0x7fc9680e78d0 2024-04-23T14:31:35.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7fc975200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc968058c60 con 0x7fc9680e3420 2024-04-23T14:31:35.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.219+0000 7fae41600640 1 -- start start 2024-04-23T14:31:35.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7f9155400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f91580d05f0 0x7f91580d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36570/0 (socket says 172.21.15.59:36570) 2024-04-23T14:31:35.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.220+0000 7f9155400640 1 -- 172.21.15.59:0/964362029 learned_addr learned my addr 172.21.15.59:0/964362029 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:35.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7fc96fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc9680e78d0 0x7fc968053d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7f9156800640 1 --2- 172.21.15.59:0/964362029 >> v2:172.21.15.76:3300/0 conn(0x7f91580d1840 0x7f91581698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7fc96f400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc9680e3040 0x7fc968058130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43576/0 (socket says 172.21.15.59:43576) 2024-04-23T14:31:35.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7fc96f400640 1 -- 172.21.15.59:0/765159952 learned_addr learned my addr 172.21.15.59:0/765159952 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:35.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7fc96ea00640 1 --2- 172.21.15.59:0/765159952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc96803c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3490633153 0 0) 0x7f91580d2560 con 0x7f91580d05f0 2024-04-23T14:31:35.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f913c003520 con 0x7f91580d05f0 2024-04-23T14:31:35.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.221+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524749197 0 0) 0x7f91580d2840 con 0x7f915815e890 2024-04-23T14:31:35.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91580d2560 con 0x7f915815e890 2024-04-23T14:31:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7fc96e000640 1 -- 172.21.15.59:0/765159952 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733192824 0 0) 0x7fc968058980 con 0x7fc9680e3040 2024-04-23T14:31:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7fc96e000640 1 -- 172.21.15.59:0/765159952 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc958003520 con 0x7fc9680e3040 2024-04-23T14:31:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7fc96ea00640 1 -- 172.21.15.59:0/765159952 >> v2:172.21.15.59:3300/0 conn(0x7fc9680e78d0 msgr2=0x7fc968053d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7fc96ea00640 1 --2- 172.21.15.59:0/765159952 >> v2:172.21.15.59:3300/0 conn(0x7fc9680e78d0 0x7fc968053d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7fc96ea00640 1 -- 172.21.15.59:0/765159952 >> v1:172.21.15.76:6789/0 conn(0x7fc9680e3040 legacy=0x7fc968058130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7fc96ea00640 1 -- 172.21.15.59:0/765159952 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc968006210 con 0x7fc9680e3420 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1411417229 0 0) 0x7f913c003520 con 0x7f91580d05f0 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 >> v2:172.21.15.76:3300/0 conn(0x7f91580d1840 msgr2=0x7f91581698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 --2- 172.21.15.59:0/964362029 >> v2:172.21.15.76:3300/0 conn(0x7f91580d1840 0x7f91581698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 >> v1:172.21.15.76:6789/0 conn(0x7f915815e890 legacy=0x7f91580cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91580d2b20 con 0x7f91580d05f0 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9144001670 con 0x7f91580d05f0 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 >> v1:172.21.15.59:6789/0 conn(0x7f91580d05f0 legacy=0x7f91580d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c003b90 0x7f913c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f913c004800 0x7f913c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913c0051c0 0x7f913c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f91580d2310 con 0x7f913c004800 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f913c00c090 con 0x7f913c0051c0 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.222+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f913c00c290 con 0x7f913c003b90 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.223+0000 7fae41600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae340589d0 0x7fae34007670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.223+0000 7fae41600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae340e79e0 con 0x7fae34053c50 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.223+0000 7fae41600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae340065d0 con 0x7fae340e7600 2024-04-23T14:31:35.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.223+0000 7fae41600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fae34007bb0 con 0x7fae340589d0 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.223+0000 7f9155400640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f913c004800 0x7f913c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.223+0000 7f9156800640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913c0051c0 0x7f913c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7f9155400640 1 -- 172.21.15.59:0/964362029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c003b90 msgr2=0x7f913c003f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7f9155400640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c003b90 0x7f913c003f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7f9155400640 1 -- 172.21.15.59:0/964362029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913c0051c0 msgr2=0x7f913c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7f9155400640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f913c0051c0 0x7f913c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7f9155400640 1 -- 172.21.15.59:0/964362029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91580d2b20 con 0x7f913c004800 2024-04-23T14:31:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7f9156800640 1 -- 172.21.15.59:0/964362029 reap_dead start 2024-04-23T14:31:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7fae40c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae340589d0 0x7fae34007670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7fae3b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fae34053c50 0x7fae3403bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36580/0 (socket says 172.21.15.59:36580) 2024-04-23T14:31:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7fae40c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae340589d0 0x7fae34007670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39746/0 (socket says 172.21.15.59:39746) 2024-04-23T14:31:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7fae40c00640 1 -- 172.21.15.59:0/438454083 learned_addr learned my addr 172.21.15.59:0/438454083 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7fae3be00640 1 --1- 172.21.15.59:0/438454083 >> v1:172.21.15.76:6789/0 conn(0x7fae340e7600 0x7fae34053540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43590/0 (socket says 172.21.15.59:43590) 2024-04-23T14:31:35.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7fc96ea00640 1 --2- 172.21.15.59:0/765159952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc96803c180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc9680069e0 tx=0x7fc95c02a090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6952a259be6fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7faaccc00640 1 -- 172.21.15.59:0/1609513084 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faabc07ac30 con 0x7faac8053750 2024-04-23T14:31:35.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.225+0000 7f9155400640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f913c004800 0x7f913c004c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91580ba770 tx=0x7f913c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685e8cc68d638f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.224+0000 7fc96e000640 1 -- 172.21.15.59:0/765159952 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c037070 con 0x7fc9680e3420 2024-04-23T14:31:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.225+0000 7fc96e000640 1 -- 172.21.15.59:0/765159952 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c02ada0 con 0x7fc9680e3420 2024-04-23T14:31:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.225+0000 7fc96e000640 1 -- 172.21.15.59:0/765159952 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c032640 con 0x7fc9680e3420 2024-04-23T14:31:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.225+0000 7fae40c00640 1 -- 172.21.15.59:0/438454083 >> v1:172.21.15.76:6789/0 conn(0x7fae340e7600 legacy=0x7fae34053540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.225+0000 7fae40c00640 1 -- 172.21.15.59:0/438454083 >> v1:172.21.15.59:6789/0 conn(0x7fae34053c50 legacy=0x7fae3403bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.225+0000 7fae40c00640 1 -- 172.21.15.59:0/438454083 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae340068b0 con 0x7fae340589d0 2024-04-23T14:31:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 -- 172.21.15.59:0/765159952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 msgr2=0x7fc96803c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 --2- 172.21.15.59:0/765159952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc96803c180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc9680069e0 tx=0x7fc95c02a090 comp rx=0 tx=0).stop 2024-04-23T14:31:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 -- 172.21.15.59:0/765159952 shutdown_connections 2024-04-23T14:31:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 --2- 172.21.15.59:0/765159952 >> v2:172.21.15.59:3300/0 conn(0x7fc9680e78d0 0x7fc968053d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 --2- 172.21.15.59:0/765159952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc96803c180 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 -- 172.21.15.59:0/765159952 >> 172.21.15.59:0/765159952 conn(0x7fc96803aa40 msgr2=0x7fc96803ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 -- 172.21.15.59:0/765159952 shutdown_connections 2024-04-23T14:31:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7fc975200640 1 -- 172.21.15.59:0/765159952 wait complete. 2024-04-23T14:31:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.227+0000 7fae40c00640 1 --2- 172.21.15.59:0/438454083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae340589d0 0x7fae34007670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae3c075a50 tx=0x7fae3c0a5d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=554ebd6973b11b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144007d90 con 0x7f913c004800 2024-04-23T14:31:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9144005ad0 con 0x7f913c004800 2024-04-23T14:31:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.226+0000 7f9154a00640 1 -- 172.21.15.59:0/964362029 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144007070 con 0x7f913c004800 2024-04-23T14:31:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.227+0000 7f8119600640 1 Processor -- start 2024-04-23T14:31:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.227+0000 7fae3aa00640 1 -- 172.21.15.59:0/438454083 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0b3070 con 0x7fae340589d0 2024-04-23T14:31:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.227+0000 7fae3aa00640 1 -- 172.21.15.59:0/438454083 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c0a6bf0 con 0x7fae340589d0 2024-04-23T14:31:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.227+0000 7fae3aa00640 1 -- 172.21.15.59:0/438454083 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0a6e60 con 0x7fae340589d0 2024-04-23T14:31:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.227+0000 7f8119600640 1 -- start start 2024-04-23T14:31:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.228+0000 7f915d200640 1 -- 172.21.15.59:0/964362029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f913c004800 msgr2=0x7f913c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.228+0000 7f915d200640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f913c004800 0x7f913c004c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91580ba770 tx=0x7f913c005830 comp rx=0 tx=0).stop 2024-04-23T14:31:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.228+0000 7f915d200640 1 -- 172.21.15.59:0/964362029 shutdown_connections 2024-04-23T14:31:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.228+0000 7f915d200640 1 --2- 172.21.15.59:0/964362029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f913c004800 0x7f913c004c80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.228+0000 7f915d200640 1 -- 172.21.15.59:0/964362029 >> 172.21.15.59:0/964362029 conn(0x7f915808bf90 msgr2=0x7f91580d4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fae41600640 1 -- 172.21.15.59:0/438454083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae340589d0 msgr2=0x7fae34007670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fae41600640 1 --2- 172.21.15.59:0/438454083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae340589d0 0x7fae34007670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae3c075a50 tx=0x7fae3c0a5d80 comp rx=0 tx=0).stop 2024-04-23T14:31:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7f915d200640 1 -- 172.21.15.59:0/964362029 shutdown_connections 2024-04-23T14:31:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fc975200640 1 Processor -- start 2024-04-23T14:31:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7f915d200640 1 -- 172.21.15.59:0/964362029 wait complete. 2024-04-23T14:31:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fae41600640 1 -- 172.21.15.59:0/438454083 shutdown_connections 2024-04-23T14:31:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fae41600640 1 --2- 172.21.15.59:0/438454083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae340589d0 0x7fae34007670 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fae41600640 1 -- 172.21.15.59:0/438454083 >> 172.21.15.59:0/438454083 conn(0x7fae34058ef0 msgr2=0x7fae34059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fae41600640 1 -- 172.21.15.59:0/438454083 shutdown_connections 2024-04-23T14:31:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7fc975200640 1 -- start start 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7f8119600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f811415ee70 0x7f811406b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7f8119600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f811406bf10 0x7f81140cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7f8119600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f811415f490 con 0x7f811415ea90 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7f8119600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f811415f270 con 0x7f811406bf10 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.229+0000 7f8119600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f811406c380 con 0x7f811415ee70 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7f8112a00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f811415ea90 0x7f81140b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43598/0 (socket says 172.21.15.59:43598) 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7f8112a00640 1 -- 172.21.15.59:0/3558481670 learned_addr learned my addr 172.21.15.59:0/3558481670 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7f8112000640 1 --2- 172.21.15.59:0/3558481670 >> v2:172.21.15.76:3300/0 conn(0x7f811415ee70 0x7f811406b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7f8113400640 1 --2- 172.21.15.59:0/3558481670 >> v2:172.21.15.59:3300/0 conn(0x7f811406bf10 0x7f81140cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7f915d200640 1 Processor -- start 2024-04-23T14:31:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7fc975200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9680e3040 0x7fc96804f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7fc975200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc9680e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7fc975200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9680e78d0 0x7fc9680e0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7fc96f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9680e3040 0x7fc96804f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7fc975200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc968054450 con 0x7fc9680e78d0 2024-04-23T14:31:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.231+0000 7fc96ea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc9680e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9680e3040 0x7fc96804f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39772/0 (socket says 172.21.15.59:39772) 2024-04-23T14:31:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96f400640 1 -- 172.21.15.59:0/2660360670 learned_addr learned my addr 172.21.15.59:0/2660360670 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96fe00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9680e78d0 0x7fc9680e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc975200640 1 -- 172.21.15.59:0/2660360670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9680542d0 con 0x7fc9680e3040 2024-04-23T14:31:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc975200640 1 -- 172.21.15.59:0/2660360670 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc9680545d0 con 0x7fc9680e3420 2024-04-23T14:31:35.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fa26ee00640 1 Processor -- start 2024-04-23T14:31:35.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96fe00640 1 -- 172.21.15.59:0/2660360670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 msgr2=0x7fc9680e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96fe00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc9680e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96fe00640 1 -- 172.21.15.59:0/2660360670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9680e3040 msgr2=0x7fc96804f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96fe00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9680e3040 0x7fc96804f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fc96fe00640 1 -- 172.21.15.59:0/2660360670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96804fd90 con 0x7fc9680e78d0 2024-04-23T14:31:35.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.230+0000 7fae41600640 1 -- 172.21.15.59:0/438454083 wait complete. 2024-04-23T14:31:35.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.232+0000 7fae41600640 1 Processor -- start 2024-04-23T14:31:35.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7f628f400640 1 Processor -- start 2024-04-23T14:31:35.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7fae41600640 1 -- start start 2024-04-23T14:31:35.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7f628f400640 1 -- start start 2024-04-23T14:31:35.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7f915d200640 1 -- start start 2024-04-23T14:31:35.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7fc96fe00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9680e78d0 0x7fc9680e0500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc970050b40 tx=0x7fc970069560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7f8112000640 1 -- 172.21.15.59:0/3558481670 >> v2:172.21.15.59:3300/0 conn(0x7f811406bf10 msgr2=0x7f81140cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7f8112000640 1 --2- 172.21.15.59:0/3558481670 >> v2:172.21.15.59:3300/0 conn(0x7f811406bf10 0x7f81140cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7f8112000640 1 -- 172.21.15.59:0/3558481670 >> v1:172.21.15.76:6789/0 conn(0x7f811415ea90 legacy=0x7f81140b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7f8112000640 1 -- 172.21.15.59:0/3558481670 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811415f770 con 0x7f811415ee70 2024-04-23T14:31:35.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.233+0000 7fa26ee00640 1 -- start start 2024-04-23T14:31:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.234+0000 7fc963400640 1 -- 172.21.15.59:0/2660360670 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970099070 con 0x7fc9680e78d0 2024-04-23T14:31:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.234+0000 7fc963400640 1 -- 172.21.15.59:0/2660360670 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9700751f0 con 0x7fc9680e78d0 2024-04-23T14:31:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.234+0000 7fc963400640 1 -- 172.21.15.59:0/2660360670 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9700755e0 con 0x7fc9680e78d0 2024-04-23T14:31:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.234+0000 7fc975200640 1 -- 172.21.15.59:0/2660360670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9680e4250 con 0x7fc9680e78d0 2024-04-23T14:31:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.234+0000 7fc975200640 1 -- 172.21.15.59:0/2660360670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9680e4810 con 0x7fc9680e78d0 2024-04-23T14:31:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.235+0000 7fa26ee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa268071d90 0x7fa2680cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.235+0000 7fa26ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa26806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.235+0000 7fa26ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa26806e040 con 0x7fa2680b3430 2024-04-23T14:31:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.235+0000 7fa26ee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa26806ddf0 con 0x7fa268071d90 2024-04-23T14:31:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.235+0000 7fa26ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa26806e320 con 0x7fa268072880 2024-04-23T14:31:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.235+0000 7fae41600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae34053c50 0x7fae340dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.235+0000 7f8112000640 1 --2- 172.21.15.59:0/3558481670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 0x7f811406b9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81140b4190 tx=0x7f810002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e8c9461c4aa4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fae41600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae340589d0 0x7fae341855f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fa267400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa2680b3430 0x7fa268072170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36586/0 (socket says 172.21.15.59:36586) 2024-04-23T14:31:35.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fa267400640 1 -- 172.21.15.59:0/1481485072 learned_addr learned my addr 172.21.15.59:0/1481485072 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:35.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fa267e00640 1 --2- 172.21.15.59:0/1481485072 >> v2:172.21.15.59:3300/0 conn(0x7fa268071d90 0x7fa2680cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fae3be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae34053c50 0x7fae340dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fae3be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae34053c50 0x7fae340dfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39788/0 (socket says 172.21.15.59:39788) 2024-04-23T14:31:35.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fae3be00640 1 -- 172.21.15.59:0/1927017656 learned_addr learned my addr 172.21.15.59:0/1927017656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fa26ce00640 1 --2- 172.21.15.59:0/1481485072 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa26806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7fae41600640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae340e7600 0x7fae340509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7feff4800640 1 Processor -- start 2024-04-23T14:31:35.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae34007150 con 0x7fae340589d0 2024-04-23T14:31:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f915d200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c004800 0x7f915815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae34006e70 con 0x7fae34053c50 2024-04-23T14:31:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae340061f0 con 0x7fae340e7600 2024-04-23T14:31:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f915d200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91580a0750 0x7f91580a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7f8111600640 1 -- 172.21.15.59:0/3558481670 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100037070 con 0x7f811415ee70 2024-04-23T14:31:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7f8111600640 1 -- 172.21.15.59:0/3558481670 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8100032440 con 0x7f811415ee70 2024-04-23T14:31:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.236+0000 7f8111600640 1 -- 172.21.15.59:0/3558481670 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100032810 con 0x7f811415ee70 2024-04-23T14:31:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f8119600640 1 -- 172.21.15.59:0/3558481670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 msgr2=0x7f811406b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f8119600640 1 --2- 172.21.15.59:0/3558481670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 0x7f811406b9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81140b4190 tx=0x7f810002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f915d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91580c98c0 0x7f91580c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7fae3b400640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae340589d0 0x7fae341855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f628f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f628806ddf0 0x7f628806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f628f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6288072680 0x7f6288072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f628f400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62880cd880 con 0x7f62880cf410 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f628f400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6288071ec0 con 0x7f628806ddf0 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f628f400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f62880cdb20 con 0x7f6288072680 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f6287e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f628806ddf0 0x7f628806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7f6287e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f628806ddf0 0x7f628806e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33472/0 (socket says 172.21.15.59:33472) 2024-04-23T14:31:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f6287e00640 1 -- 172.21.15.59:0/598206585 learned_addr learned my addr 172.21.15.59:0/598206585 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f9155400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91580a0750 0x7f91580a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f9155400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91580a0750 0x7f91580a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39798/0 (socket says 172.21.15.59:39798) 2024-04-23T14:31:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f6287e00640 1 -- 172.21.15.59:0/598206585 >> v2:172.21.15.76:3300/0 conn(0x7f6288072680 msgr2=0x7f6288072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f6287e00640 1 --2- 172.21.15.59:0/598206585 >> v2:172.21.15.76:3300/0 conn(0x7f6288072680 0x7f6288072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f6287e00640 1 -- 172.21.15.59:0/598206585 >> v1:172.21.15.59:6789/0 conn(0x7f62880cf410 legacy=0x7f628815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7fae40c00640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae340e7600 0x7fae340509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f9156800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91580c98c0 0x7f91580c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f9155e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c004800 0x7f915815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f9155e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c004800 0x7f915815de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51576/0 (socket says 172.21.15.59:51576) 2024-04-23T14:31:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7f6287e00640 1 -- 172.21.15.59:0/598206585 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6288072160 con 0x7f628806ddf0 2024-04-23T14:31:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f9156800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91580c98c0 0x7f91580c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33482/0 (socket says 172.21.15.59:33482) 2024-04-23T14:31:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f915815ec70 con 0x7f91580c98c0 2024-04-23T14:31:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91580d30e0 con 0x7f91580a0750 2024-04-23T14:31:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f915815edf0 con 0x7f913c004800 2024-04-23T14:31:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f9155400640 1 -- 172.21.15.59:0/2365601515 learned_addr learned my addr 172.21.15.59:0/2365601515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f8119600640 1 -- 172.21.15.59:0/3558481670 shutdown_connections 2024-04-23T14:31:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f8119600640 1 --2- 172.21.15.59:0/3558481670 >> v2:172.21.15.59:3300/0 conn(0x7f811406bf10 0x7f81140cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f8119600640 1 --2- 172.21.15.59:0/3558481670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 0x7f811406b9d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7f8119600640 1 -- 172.21.15.59:0/3558481670 >> 172.21.15.59:0/3558481670 conn(0x7f811409aa00 msgr2=0x7f81140b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f8119600640 1 -- 172.21.15.59:0/3558481670 shutdown_connections 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7fa267e00640 1 -- 172.21.15.59:0/1481485072 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 msgr2=0x7fa26806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7fa267e00640 1 --2- 172.21.15.59:0/1481485072 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa26806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7fa267e00640 1 -- 172.21.15.59:0/1481485072 >> v1:172.21.15.59:6789/0 conn(0x7fa2680b3430 legacy=0x7fa268072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.238+0000 7fa267e00640 1 -- 172.21.15.59:0/1481485072 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26806e600 con 0x7fa268071d90 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7fa26ce00640 1 --2- 172.21.15.59:0/1481485072 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa26806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7fae3be00640 1 -- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae340e7600 msgr2=0x7fae340509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7fae3be00640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae340e7600 0x7fae340509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7fae3be00640 1 -- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae340589d0 msgr2=0x7fae341855f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7fc975200640 1 -- 172.21.15.59:0/2660360670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94c005290 con 0x7fc9680e78d0 2024-04-23T14:31:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7fc963400640 1 -- 172.21.15.59:0/2660360670 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc97009c070 con 0x7fc9680e78d0 2024-04-23T14:31:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.237+0000 7feff4800640 1 -- start start 2024-04-23T14:31:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7feff4800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefec0d1800 0x7fefec0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7feff4800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefec0cf3c0 con 0x7fefec15e9c0 2024-04-23T14:31:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7feff4800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefec0cf640 con 0x7fefec0d3480 2024-04-23T14:31:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.239+0000 7feff4800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fefec0cf900 con 0x7fefec0d1800 2024-04-23T14:31:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f9155400640 1 -- 172.21.15.59:0/2365601515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c004800 msgr2=0x7f915815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f9155400640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c004800 0x7f915815de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f9155400640 1 -- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91580c98c0 msgr2=0x7f91580c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f9155400640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91580c98c0 0x7f91580c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f9155400640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91580ca5f0 con 0x7f91580a0750 2024-04-23T14:31:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7fae3be00640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae340589d0 0x7fae341855f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7fa267e00640 1 --2- 172.21.15.59:0/1481485072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268071d90 0x7fa2680cd1b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa2540097a0 tx=0x7fa25402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2bd3bd46059e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7fae3be00640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae34051400 con 0x7fae34053c50 2024-04-23T14:31:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7fae3b400640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae340589d0 0x7fae341855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f8119600640 1 -- 172.21.15.59:0/3558481670 wait complete. 2024-04-23T14:31:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7feff1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fefec15e9c0 0x7fefec0d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36590/0 (socket says 172.21.15.59:36590) 2024-04-23T14:31:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7feff1e00640 1 -- 172.21.15.59:0/2170566793 learned_addr learned my addr 172.21.15.59:0/2170566793 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7feff2800640 1 --2- 172.21.15.59:0/2170566793 >> v2:172.21.15.59:3300/0 conn(0x7fefec0d1800 0x7fefec0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.240+0000 7f9155400640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91580a0750 0x7f91580a0bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f914400c400 tx=0x7f9144001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7f6287e00640 1 --2- 172.21.15.59:0/598206585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f628806ddf0 0x7f628806e250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6288075670 tx=0x7f6274032890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ea5c40a0f54f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7fae3be00640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae34053c50 0x7fae340dfd00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c500 tx=0x7fae3000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7f914e000640 1 -- 172.21.15.59:0/2365601515 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144038040 con 0x7f91580a0750 2024-04-23T14:31:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915809eaa0 con 0x7f91580a0750 2024-04-23T14:31:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7f914e000640 1 -- 172.21.15.59:0/2365601515 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9144042070 con 0x7f91580a0750 2024-04-23T14:31:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7f914e000640 1 -- 172.21.15.59:0/2365601515 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91440049e0 con 0x7f91580a0750 2024-04-23T14:31:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915809f060 con 0x7f91580a0750 2024-04-23T14:31:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7feff0a00640 1 -- 172.21.15.59:0/2170566793 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787598273 0 0) 0x7fefec0cf640 con 0x7fefec0d3480 2024-04-23T14:31:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.241+0000 7feff0a00640 1 -- 172.21.15.59:0/2170566793 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefd4003520 con 0x7fefec0d3480 2024-04-23T14:31:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.242+0000 7fae2fe00640 1 -- 172.21.15.59:0/1927017656 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30018070 con 0x7fae34053c50 2024-04-23T14:31:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.242+0000 7fc963400640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc95006c9c0 0x7fc95006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.242+0000 7fae2fe00640 1 -- 172.21.15.59:0/1927017656 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae30007500 con 0x7fae34053c50 2024-04-23T14:31:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.242+0000 7fae2fe00640 1 -- 172.21.15.59:0/1927017656 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae300049e0 con 0x7fae34053c50 2024-04-23T14:31:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.243+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3404c9f0 con 0x7fae34053c50 2024-04-23T14:31:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.243+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3404cfb0 con 0x7fae34053c50 2024-04-23T14:31:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.243+0000 7f8119600640 1 Processor -- start 2024-04-23T14:31:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.243+0000 7fc963400640 1 -- 172.21.15.59:0/2660360670 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc9701068d0 con 0x7fc9680e78d0 2024-04-23T14:31:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.243+0000 7fc96f400640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc95006c9c0 0x7fc95006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.243+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9158069190 con 0x7f91580a0750 2024-04-23T14:31:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.242+0000 7feff2800640 1 -- 172.21.15.59:0/2170566793 >> v1:172.21.15.76:6789/0 conn(0x7fefec0d3480 legacy=0x7fefec0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.242+0000 7feff2800640 1 -- 172.21.15.59:0/2170566793 >> v1:172.21.15.59:6789/0 conn(0x7fefec15e9c0 legacy=0x7fefec0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.242+0000 7feff2800640 1 -- 172.21.15.59:0/2170566793 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec0cfbe0 con 0x7fefec0d1800 2024-04-23T14:31:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.243+0000 7feff2800640 1 --2- 172.21.15.59:0/2170566793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec0d1800 0x7fefec0d03e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009d90 tx=0x7fefe802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975f7f9c1fea1de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.244+0000 7f8119600640 1 -- start start 2024-04-23T14:31:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.244+0000 7fc96f400640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc95006c9c0 0x7fc95006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc964007680 tx=0x7fc964007c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.245+0000 7f8119600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406bf10 0x7f8114202a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.245+0000 7f8119600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f811415ea90 0x7f811415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.245+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae34005710 con 0x7fae34053c50 2024-04-23T14:31:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8119600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 0x7f81140b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8119600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f811415fd30 con 0x7f811406bf10 2024-04-23T14:31:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8119600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f81141600a0 con 0x7f811415ee70 2024-04-23T14:31:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8119600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f81140b2de0 con 0x7f811415ea90 2024-04-23T14:31:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.245+0000 7f914e000640 1 -- 172.21.15.59:0/2365601515 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9144004b80 con 0x7f91580a0750 2024-04-23T14:31:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8112a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406bf10 0x7f8114202a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8112a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406bf10 0x7f8114202a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33504/0 (socket says 172.21.15.59:33504) 2024-04-23T14:31:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8112a00640 1 -- 172.21.15.59:0/3213135160 learned_addr learned my addr 172.21.15.59:0/3213135160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8112000640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f811415ea90 0x7f811415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f914e000640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f913806c8f0 0x7f913806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f8113400640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 0x7f81140b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.246+0000 7f914e000640 1 -- 172.21.15.59:0/2365601515 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91440ab8d0 con 0x7f91580a0750 2024-04-23T14:31:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7f9155e00640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f913806c8f0 0x7f913806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7fae2fe00640 1 -- 172.21.15.59:0/1927017656 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae3001b070 con 0x7fae34053c50 2024-04-23T14:31:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7f8112000640 1 -- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 msgr2=0x7f81140b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7f8112000640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 0x7f81140b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7f8112000640 1 -- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406bf10 msgr2=0x7f8114202a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7f8112000640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406bf10 0x7f8114202a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7f8112000640 1 -- 172.21.15.59:0/3213135160 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81142055a0 con 0x7f811415ea90 2024-04-23T14:31:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.247+0000 7f9155e00640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f913806c8f0 0x7f913806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91580d0eb0 tx=0x7f9148009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.248+0000 7f8112000640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f811415ea90 0x7f811415d2c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8100031000 tx=0x7f81000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.248+0000 7fae2fe00640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae1c06c970 0x7fae1c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.248+0000 7fae2fe00640 1 -- 172.21.15.59:0/1927017656 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae3008ad10 con 0x7fae34053c50 2024-04-23T14:31:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.248+0000 7fae3b400640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae1c06c970 0x7fae1c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.248+0000 7f810aa00640 1 -- 172.21.15.59:0/3213135160 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100037050 con 0x7f811415ea90 2024-04-23T14:31:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.248+0000 7f810aa00640 1 -- 172.21.15.59:0/3213135160 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8100043070 con 0x7f811415ea90 2024-04-23T14:31:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.249+0000 7f810aa00640 1 -- 172.21.15.59:0/3213135160 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100030b10 con 0x7f811415ea90 2024-04-23T14:31:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.249+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8114205880 con 0x7f811415ea90 2024-04-23T14:31:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.249+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8114205e40 con 0x7f811415ea90 2024-04-23T14:31:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.249+0000 7fa266a00640 1 -- 172.21.15.59:0/1481485072 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254037070 con 0x7fa268071d90 2024-04-23T14:31:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.249+0000 7fa266a00640 1 -- 172.21.15.59:0/1481485072 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25402ad10 con 0x7fa268071d90 2024-04-23T14:31:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7fae3b400640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae1c06c970 0x7fae1c06ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae340e47f0 tx=0x7fae28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7f6287400640 1 -- 172.21.15.59:0/598206585 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627403f070 con 0x7f628806ddf0 2024-04-23T14:31:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7f6287400640 1 -- 172.21.15.59:0/598206585 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6274036050 con 0x7f628806ddf0 2024-04-23T14:31:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7fc963400640 1 -- 172.21.15.59:0/2660360670 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc970091070 con 0x7fc9680e78d0 2024-04-23T14:31:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7f6287400640 1 -- 172.21.15.59:0/598206585 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627400f850 con 0x7f628806ddf0 2024-04-23T14:31:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7fa26ee00640 1 -- 172.21.15.59:0/1481485072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268071d90 msgr2=0x7fa2680cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7fa266a00640 1 -- 172.21.15.59:0/1481485072 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254037070 con 0x7fa268071d90 2024-04-23T14:31:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7fa26ee00640 1 --2- 172.21.15.59:0/1481485072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268071d90 0x7fa2680cd1b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa2540097a0 tx=0x7fa25402a080 comp rx=0 tx=0).stop 2024-04-23T14:31:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7fa26ee00640 1 -- 172.21.15.59:0/1481485072 shutdown_connections 2024-04-23T14:31:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8114068a80 con 0x7f811415ea90 2024-04-23T14:31:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7fa26ee00640 1 --2- 172.21.15.59:0/1481485072 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa26806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7fa26ee00640 1 --2- 172.21.15.59:0/1481485072 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa268071d90 0x7fa2680cd1b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7fa26ee00640 1 -- 172.21.15.59:0/1481485072 >> 172.21.15.59:0/1481485072 conn(0x7fa2680b2130 msgr2=0x7fa2680b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7f628f400640 1 -- 172.21.15.59:0/598206585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f628806ddf0 msgr2=0x7f628806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7f628f400640 1 --2- 172.21.15.59:0/598206585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f628806ddf0 0x7f628806e250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6288075670 tx=0x7f6274032890 comp rx=0 tx=0).stop 2024-04-23T14:31:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.250+0000 7feff0a00640 1 -- 172.21.15.59:0/2170566793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8007740 con 0x7fefec0d1800 2024-04-23T14:31:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7feff0a00640 1 -- 172.21.15.59:0/2170566793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe80078e0 con 0x7fefec0d1800 2024-04-23T14:31:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7feff0a00640 1 -- 172.21.15.59:0/2170566793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8007c90 con 0x7fefec0d1800 2024-04-23T14:31:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7fa26ee00640 1 -- 172.21.15.59:0/1481485072 shutdown_connections 2024-04-23T14:31:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.252+0000 7feff4800640 1 -- 172.21.15.59:0/2170566793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec0d1800 msgr2=0x7fefec0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.252+0000 7feff4800640 1 --2- 172.21.15.59:0/2170566793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec0d1800 0x7fefec0d03e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009d90 tx=0x7fefe802a660 comp rx=0 tx=0).stop 2024-04-23T14:31:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.251+0000 7fa26ee00640 1 -- 172.21.15.59:0/1481485072 wait complete. 2024-04-23T14:31:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7f628f400640 1 -- 172.21.15.59:0/598206585 shutdown_connections 2024-04-23T14:31:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7f628f400640 1 --2- 172.21.15.59:0/598206585 >> v2:172.21.15.76:3300/0 conn(0x7f6288072680 0x7f6288072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7f628f400640 1 --2- 172.21.15.59:0/598206585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f628806ddf0 0x7f628806e250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7f628f400640 1 -- 172.21.15.59:0/598206585 >> 172.21.15.59:0/598206585 conn(0x7f62880b2130 msgr2=0x7f62880b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7f628f400640 1 -- 172.21.15.59:0/598206585 shutdown_connections 2024-04-23T14:31:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7feff4800640 1 -- 172.21.15.59:0/2170566793 shutdown_connections 2024-04-23T14:31:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7feff4800640 1 --2- 172.21.15.59:0/2170566793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec0d1800 0x7fefec0d03e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7feff4800640 1 -- 172.21.15.59:0/2170566793 >> 172.21.15.59:0/2170566793 conn(0x7fefec08bf90 msgr2=0x7fefec0d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7f628f400640 1 -- 172.21.15.59:0/598206585 wait complete. 2024-04-23T14:31:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7f810aa00640 1 -- 172.21.15.59:0/3213135160 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8100030cb0 con 0x7f811415ea90 2024-04-23T14:31:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7fa26ee00640 1 Processor -- start 2024-04-23T14:31:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7feff4800640 1 -- 172.21.15.59:0/2170566793 shutdown_connections 2024-04-23T14:31:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.253+0000 7feff4800640 1 -- 172.21.15.59:0/2170566793 wait complete. 2024-04-23T14:31:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7f810aa00640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f80f406c9c0 0x7f80f406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7f810aa00640 1 -- 172.21.15.59:0/3213135160 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f81000ad250 con 0x7f811415ea90 2024-04-23T14:31:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.254+0000 7fa26ee00640 1 -- start start 2024-04-23T14:31:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7f628f400640 1 Processor -- start 2024-04-23T14:31:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7fa26ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa268071d90 0x7fa26815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7fa26ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa268159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7fa26ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2680b3430 0x7fa2681f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7fa26ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2680b3a20 con 0x7fa2680b3430 2024-04-23T14:31:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7fa26ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2680b38a0 con 0x7fa268071d90 2024-04-23T14:31:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.255+0000 7fa26ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa268166430 con 0x7fa268072880 2024-04-23T14:31:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.256+0000 7f628f400640 1 -- start start 2024-04-23T14:31:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.256+0000 7f8112a00640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f80f406c9c0 0x7f80f406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.256+0000 7fa26ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2680b3430 0x7fa2681f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.256+0000 7fa26ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2680b3430 0x7fa2681f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33508/0 (socket says 172.21.15.59:33508) 2024-04-23T14:31:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.256+0000 7fa26ce00640 1 -- 172.21.15.59:0/3960031691 learned_addr learned my addr 172.21.15.59:0/3960031691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.256+0000 7fa267400640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa268159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.256+0000 7feff4800640 1 Processor -- start 2024-04-23T14:31:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7fa26ce00640 1 -- 172.21.15.59:0/3960031691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 msgr2=0x7fa268159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7fa26ce00640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa268159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7fa26ce00640 1 -- 172.21.15.59:0/3960031691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa268071d90 msgr2=0x7fa26815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7fa26ce00640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa268071d90 0x7fa26815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7fa26ce00640 1 -- 172.21.15.59:0/3960031691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26815a850 con 0x7fa2680b3430 2024-04-23T14:31:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7feff4800640 1 -- start start 2024-04-23T14:31:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7f8112a00640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f80f406c9c0 0x7f80f406ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8104004e40 tx=0x7f8104007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.257+0000 7fa26ce00640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2680b3430 0x7fa2681f8af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa260009fa0 tx=0x7fa26000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7fa25fe00640 1 -- 172.21.15.59:0/3960031691 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26001a070 con 0x7fa2680b3430 2024-04-23T14:31:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7fa25fe00640 1 -- 172.21.15.59:0/3960031691 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa260003bb0 con 0x7fa2680b3430 2024-04-23T14:31:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7fa25fe00640 1 -- 172.21.15.59:0/3960031691 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260004030 con 0x7fa2680b3430 2024-04-23T14:31:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7fa26ee00640 1 -- 172.21.15.59:0/3960031691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26815d770 con 0x7fa2680b3430 2024-04-23T14:31:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7feff4800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec0d1800 0x7fefec0b9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7fa26ee00640 1 -- 172.21.15.59:0/3960031691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa268163000 con 0x7fa2680b3430 2024-04-23T14:31:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7feff4800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefec0d3480 0x7fefec0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7feff4800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec15e9c0 0x7fefec0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7f628f400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f628806ddf0 0x7f628815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7f628f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288072680 0x7f6288151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7f628f400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62880cf410 0x7f6288151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7f628f400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f628815a820 con 0x7f6288072680 2024-04-23T14:31:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7f628f400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f628815a6a0 con 0x7f628806ddf0 2024-04-23T14:31:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.258+0000 7f628f400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f62880cf7f0 con 0x7f62880cf410 2024-04-23T14:31:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.259+0000 7f6287e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288072680 0x7f6288151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.259+0000 7feff2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec15e9c0 0x7fefec0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.259+0000 7f6287e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288072680 0x7f6288151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33540/0 (socket says 172.21.15.59:33540) 2024-04-23T14:31:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.259+0000 7f6287e00640 1 -- 172.21.15.59:0/3955798689 learned_addr learned my addr 172.21.15.59:0/3955798689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.259+0000 7f628ca00640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f628806ddf0 0x7f628815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7feff2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec15e9c0 0x7fefec0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33524/0 (socket says 172.21.15.59:33524) 2024-04-23T14:31:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7f628d400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62880cf410 0x7f6288151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7feff1e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec0d1800 0x7fefec0b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7feff1e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec0d1800 0x7fefec0b9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:39826/0 (socket says 172.21.15.59:39826) 2024-04-23T14:31:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7feff2800640 1 -- 172.21.15.59:0/1415076732 learned_addr learned my addr 172.21.15.59:0/1415076732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7f6287e00640 1 -- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62880cf410 msgr2=0x7f6288151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7f6287e00640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62880cf410 0x7f6288151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7f6287e00640 1 -- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f628806ddf0 msgr2=0x7f628815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefec06cd10 con 0x7fefec15e9c0 2024-04-23T14:31:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7f6287e00640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f628806ddf0 0x7f628815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7f6287e00640 1 -- 172.21.15.59:0/3955798689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6288157e30 con 0x7f6288072680 2024-04-23T14:31:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7f628d400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62880cf410 0x7f6288151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefec06cb90 con 0x7fefec0d1800 2024-04-23T14:31:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23c005290 con 0x7fa2680b3430 2024-04-23T14:31:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.260+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefec06ce90 con 0x7fefec0d3480 2024-04-23T14:31:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.261+0000 7feff2800640 1 -- 172.21.15.59:0/1415076732 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefec0d3480 msgr2=0x7fefec0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.261+0000 7feff2800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefec0d3480 0x7fefec0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.261+0000 7feff2800640 1 -- 172.21.15.59:0/1415076732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec0d1800 msgr2=0x7fefec0b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.261+0000 7feff2800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec0d1800 0x7fefec0b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.261+0000 7f6287e00640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288072680 0x7f6288151840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6274013240 tx=0x7f62740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.261+0000 7feff2800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec0b9b90 con 0x7fefec15e9c0 2024-04-23T14:31:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7f6284c00640 1 -- 172.21.15.59:0/3955798689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627403f050 con 0x7f6288072680 2024-04-23T14:31:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7feff2800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec15e9c0 0x7fefec0cd3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fefe802ac50 tx=0x7fefe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7f6284c00640 1 -- 172.21.15.59:0/3955798689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627404b070 con 0x7f6288072680 2024-04-23T14:31:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62880b5020 con 0x7f6288072680 2024-04-23T14:31:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7f914e000640 1 -- 172.21.15.59:0/2365601515 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9144001630 con 0x7f91580a0750 2024-04-23T14:31:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6288152e40 con 0x7f6288072680 2024-04-23T14:31:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7fefe6000640 1 -- 172.21.15.59:0/1415076732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8040070 con 0x7fefec15e9c0 2024-04-23T14:31:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.262+0000 7f6284c00640 1 -- 172.21.15.59:0/3955798689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627403ab50 con 0x7f6288072680 2024-04-23T14:31:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.263+0000 7fefe6000640 1 -- 172.21.15.59:0/1415076732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe8031bc0 con 0x7fefec15e9c0 2024-04-23T14:31:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.263+0000 7fefe6000640 1 -- 172.21.15.59:0/1415076732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8009450 con 0x7fefec15e9c0 2024-04-23T14:31:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.263+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec0b9e70 con 0x7fefec15e9c0 2024-04-23T14:31:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.263+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec158c50 con 0x7fefec15e9c0 2024-04-23T14:31:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.264+0000 7fa25fe00640 1 -- 172.21.15.59:0/3960031691 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa26001d070 con 0x7fa2680b3430 2024-04-23T14:31:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.265+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6264005290 con 0x7f6288072680 2024-04-23T14:31:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.265+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefec0ba330 con 0x7fefec15e9c0 2024-04-23T14:31:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.265+0000 7f6284c00640 1 -- 172.21.15.59:0/3955798689 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f627403acf0 con 0x7f6288072680 2024-04-23T14:31:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.266+0000 7f6284c00640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f626806c9c0 0x7f626806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.267+0000 7fefe6000640 1 -- 172.21.15.59:0/1415076732 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefe8039040 con 0x7fefec15e9c0 2024-04-23T14:31:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.267+0000 7f6284c00640 1 -- 172.21.15.59:0/3955798689 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62740b4e30 con 0x7f6288072680 2024-04-23T14:31:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.267+0000 7f628ca00640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f626806c9c0 0x7f626806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.268+0000 7fefe6000640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefd006c8f0 0x7fefd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.268+0000 7fefe6000640 1 -- 172.21.15.59:0/1415076732 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefe80ad360 con 0x7fefec15e9c0 2024-04-23T14:31:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.268+0000 7f628ca00640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f626806c9c0 0x7f626806ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f627c006fd0 tx=0x7f627c009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.265+0000 7fa25fe00640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c940 0x7fa24806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.265+0000 7fa25fe00640 1 -- 172.21.15.59:0/3960031691 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa26008bf70 con 0x7fa2680b3430 2024-04-23T14:31:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.266+0000 7fa267e00640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c940 0x7fa24806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.268+0000 7fa267e00640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c940 0x7fa24806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa254034000 tx=0x7fa25402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.268+0000 7f810aa00640 1 -- 172.21.15.59:0/3213135160 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f810007b010 con 0x7f811415ea90 2024-04-23T14:31:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.271+0000 7feff1e00640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefd006c8f0 0x7fefd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.272+0000 7feff1e00640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefd006c8f0 0x7fefd006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fefdc005880 tx=0x7fefdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.275+0000 7fae2fe00640 1 -- 172.21.15.59:0/1927017656 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae30058c60 con 0x7fae34053c50 2024-04-23T14:31:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.276+0000 7fa25fe00640 1 -- 172.21.15.59:0/3960031691 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa260010070 con 0x7fa2680b3430 2024-04-23T14:31:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.292+0000 7f6284c00640 1 -- 172.21.15.59:0/3955798689 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62740832d0 con 0x7f6288072680 2024-04-23T14:31:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.297+0000 7fefe6000640 1 -- 172.21.15.59:0/1415076732 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefe807b780 con 0x7fefec15e9c0 2024-04-23T14:31:35.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.655+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7faac804f3a0 con 0x7faac8053750 2024-04-23T14:31:35.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.656+0000 7faaccc00640 1 -- 172.21.15.59:0/1609513084 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faabc07a530 con 0x7faac8053750 2024-04-23T14:31:35.660 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:35.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.660+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faab006c8f0 msgr2=0x7faab006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.660+0000 7faad7a00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faab006c8f0 0x7faab006edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faac80dfc30 tx=0x7faac4008040 comp rx=0 tx=0).stop 2024-04-23T14:31:35.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.660+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac8053750 msgr2=0x7faac817ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.660+0000 7faad7a00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac8053750 0x7faac817ce60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faabc000c00 tx=0x7faabc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.661+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 shutdown_connections 2024-04-23T14:31:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.661+0000 7faad7a00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faab006c8f0 0x7faab006edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.661+0000 7faad7a00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac80e3300 0x7faac80d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.661+0000 7faad7a00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac8054110 0x7faac80528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.661+0000 7faad7a00640 1 --2- 172.21.15.59:0/1609513084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac8053750 0x7faac817ce60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.661+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 >> 172.21.15.59:0/1609513084 conn(0x7faac8058ef0 msgr2=0x7faac8037b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.662+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 shutdown_connections 2024-04-23T14:31:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.662+0000 7faad7a00640 1 -- 172.21.15.59:0/1609513084 wait complete. 2024-04-23T14:31:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.663+0000 7fc975200640 1 -- 172.21.15.59:0/2660360670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc94c005580 con 0x7fc9680e78d0 2024-04-23T14:31:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.664+0000 7fc963400640 1 -- 172.21.15.59:0/2660360670 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc9700d4690 con 0x7fc9680e78d0 2024-04-23T14:31:35.667 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.668+0000 7fc960c00640 1 -- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc95006c9c0 msgr2=0x7fc95006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.668+0000 7fc960c00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc95006c9c0 0x7fc95006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc964007680 tx=0x7fc964007c30 comp rx=0 tx=0).stop 2024-04-23T14:31:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.668+0000 7fc960c00640 1 -- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9680e78d0 msgr2=0x7fc9680e0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.668+0000 7fc960c00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9680e78d0 0x7fc9680e0500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc970050b40 tx=0x7fc970069560 comp rx=0 tx=0).stop 2024-04-23T14:31:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.669+0000 7fc960c00640 1 -- 172.21.15.59:0/2660360670 shutdown_connections 2024-04-23T14:31:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.669+0000 7fc960c00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc95006c9c0 0x7fc95006ee80 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc964007680 tx=0x7fc964007c30 comp rx=0 tx=0).stop 2024-04-23T14:31:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.669+0000 7fc960c00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9680e3420 0x7fc9680e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.669+0000 7fc960c00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9680e78d0 0x7fc9680e0500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.669+0000 7fc960c00640 1 --2- 172.21.15.59:0/2660360670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9680e3040 0x7fc96804f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.670+0000 7fc960c00640 1 -- 172.21.15.59:0/2660360670 >> 172.21.15.59:0/2660360670 conn(0x7fc96803aa40 msgr2=0x7fc968056ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.671+0000 7fc960c00640 1 -- 172.21.15.59:0/2660360670 shutdown_connections 2024-04-23T14:31:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.671+0000 7fc960c00640 1 -- 172.21.15.59:0/2660360670 wait complete. 2024-04-23T14:31:35.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-23T14:31:35.692 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-23T14:31:35.699 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.702+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9158074d30 con 0x7f91580a0750 2024-04-23T14:31:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.702+0000 7f914e000640 1 -- 172.21.15.59:0/2365601515 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f91440796a0 con 0x7f91580a0750 2024-04-23T14:31:35.704 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f913806c8f0 msgr2=0x7f913806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f913806c8f0 0x7f913806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91580d0eb0 tx=0x7f9148009040 comp rx=0 tx=0).stop 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91580a0750 msgr2=0x7f91580a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91580a0750 0x7f91580a0bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f914400c400 tx=0x7f9144001970 comp rx=0 tx=0).stop 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 shutdown_connections 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f913806c8f0 0x7f913806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91580c98c0 0x7f91580c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91580a0750 0x7f91580a0bd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 --2- 172.21.15.59:0/2365601515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f913c004800 0x7f915815de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 >> 172.21.15.59:0/2365601515 conn(0x7f915808bf90 msgr2=0x7f915815f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 shutdown_connections 2024-04-23T14:31:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.705+0000 7f915d200640 1 -- 172.21.15.59:0/2365601515 wait complete. 2024-04-23T14:31:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.713+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6264005580 con 0x7f6288072680 2024-04-23T14:31:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.713+0000 7f6284c00640 1 -- 172.21.15.59:0/3955798689 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6274082bd0 con 0x7f6288072680 2024-04-23T14:31:35.714 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f626806c9c0 msgr2=0x7f626806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f626806c9c0 0x7f626806ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f627c006fd0 tx=0x7f627c009060 comp rx=0 tx=0).stop 2024-04-23T14:31:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288072680 msgr2=0x7f6288151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288072680 0x7f6288151840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6274013240 tx=0x7f62740046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 shutdown_connections 2024-04-23T14:31:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f626806c9c0 0x7f626806ee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62880cf410 0x7f6288151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6288072680 0x7f6288151840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 --2- 172.21.15.59:0/3955798689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f628806ddf0 0x7f628815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 >> 172.21.15.59:0/3955798689 conn(0x7f62880b2130 msgr2=0x7f62880b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 shutdown_connections 2024-04-23T14:31:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.716+0000 7f628f400640 1 -- 172.21.15.59:0/3955798689 wait complete. 2024-04-23T14:31:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.722+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f81140cedc0 con 0x7f811415ea90 2024-04-23T14:31:35.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-23T14:31:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.723+0000 7f810aa00640 1 -- 172.21.15.59:0/3213135160 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f810007f310 con 0x7f811415ea90 2024-04-23T14:31:35.724 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:35.724 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f80f406c9c0 msgr2=0x7f80f406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f80f406c9c0 0x7f80f406ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8104004e40 tx=0x7f8104007f10 comp rx=0 tx=0).stop 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f811415ea90 msgr2=0x7f811415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f811415ea90 0x7f811415d2c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8100031000 tx=0x7f81000046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 shutdown_connections 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f80f406c9c0 0x7f80f406ee80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f811415ea90 0x7f811415d2c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f811415ee70 0x7f81140b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 --2- 172.21.15.59:0/3213135160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811406bf10 0x7f8114202a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.725+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 >> 172.21.15.59:0/3213135160 conn(0x7f811409aa00 msgr2=0x7f81140cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.726+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 shutdown_connections 2024-04-23T14:31:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.726+0000 7f8119600640 1 -- 172.21.15.59:0/3213135160 wait complete. 2024-04-23T14:31:35.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.731+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fae340e5b80 con 0x7fae34053c50 2024-04-23T14:31:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.731+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa23c005580 con 0x7fa2680b3430 2024-04-23T14:31:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.732+0000 7fa25fe00640 1 -- 172.21.15.59:0/3960031691 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa26005a2f0 con 0x7fa2680b3430 2024-04-23T14:31:35.733 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.732+0000 7fae2fe00640 1 -- 172.21.15.59:0/1927017656 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae3005cf60 con 0x7fae34053c50 2024-04-23T14:31:35.733 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2024-04-23T14:31:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.734+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c940 msgr2=0x7fa24806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.734+0000 7fa25d600640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c940 0x7fa24806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa254034000 tx=0x7fa25402c000 comp rx=0 tx=0).stop 2024-04-23T14:31:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.734+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2680b3430 msgr2=0x7fa2681f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.734+0000 7fa25d600640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2680b3430 0x7fa2681f8af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa260009fa0 tx=0x7fa26000d560 comp rx=0 tx=0).stop 2024-04-23T14:31:35.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-23T14:31:35.737 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 shutdown_connections 2024-04-23T14:31:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c940 0x7fa24806ee00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa268072880 0x7fa268159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2680b3430 0x7fa2681f8af0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 --2- 172.21.15.59:0/3960031691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa268071d90 0x7fa26815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 >> 172.21.15.59:0/3960031691 conn(0x7fa2680b2130 msgr2=0x7fa2680b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 shutdown_connections 2024-04-23T14:31:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.735+0000 7fa25d600640 1 -- 172.21.15.59:0/3960031691 wait complete. 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae1c06c970 msgr2=0x7fae1c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae1c06c970 0x7fae1c06ee30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae340e47f0 tx=0x7fae28009040 comp rx=0 tx=0).stop 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae34053c50 msgr2=0x7fae340dfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae34053c50 0x7fae340dfd00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c500 tx=0x7fae3000c9c0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 shutdown_connections 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae1c06c970 0x7fae1c06ee30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae340e7600 0x7fae340509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae340589d0 0x7fae341855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 --2- 172.21.15.59:0/1927017656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae34053c50 0x7fae340dfd00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 >> 172.21.15.59:0/1927017656 conn(0x7fae34058ef0 msgr2=0x7fae340e60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 shutdown_connections 2024-04-23T14:31:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.736+0000 7fae41600640 1 -- 172.21.15.59:0/1927017656 wait complete. 2024-04-23T14:31:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.738+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fefec068a80 con 0x7fefec15e9c0 2024-04-23T14:31:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.738+0000 7fefe6000640 1 -- 172.21.15.59:0/1415076732 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefe807b080 con 0x7fefec15e9c0 2024-04-23T14:31:35.740 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.741+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefd006c8f0 msgr2=0x7fefd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.741+0000 7feff4800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefd006c8f0 0x7fefd006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fefdc005880 tx=0x7fefdc009040 comp rx=0 tx=0).stop 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.741+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec15e9c0 msgr2=0x7fefec0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.741+0000 7feff4800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec15e9c0 0x7fefec0cd3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fefe802ac50 tx=0x7fefe8004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 shutdown_connections 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefd006c8f0 0x7fefd006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefec0d3480 0x7fefec0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefec15e9c0 0x7fefec0cd3b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 --2- 172.21.15.59:0/1415076732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefec0d1800 0x7fefec0b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 >> 172.21.15.59:0/1415076732 conn(0x7fefec08bf90 msgr2=0x7fefec0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 shutdown_connections 2024-04-23T14:31:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.742+0000 7feff4800640 1 -- 172.21.15.59:0/1415076732 wait complete. 2024-04-23T14:31:35.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-23T14:31:35.748 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-23T14:31:35.755 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-23T14:31:35.761 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-23T14:31:35.764 DEBUG:teuthology.parallel:result is None 2024-04-23T14:31:35.764 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T14:31:35.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:31:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.990+0000 7f9797000640 1 Processor -- start 2024-04-23T14:31:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.990+0000 7f9797000640 1 -- start start 2024-04-23T14:31:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.992+0000 7f9797000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 0x7f97900af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.992+0000 7f9797000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f979015f0f0 con 0x7f97900af920 2024-04-23T14:31:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.992+0000 7f9797000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f979015f3d0 con 0x7f97900d08b0 2024-04-23T14:31:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.992+0000 7f9797000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f97900afd80 con 0x7f979015c5e0 2024-04-23T14:31:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.992+0000 7f978f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f97900af920 0x7f97900d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36592/0 (socket says 172.21.15.59:36592) 2024-04-23T14:31:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.992+0000 7f978f400640 1 -- 172.21.15.59:0/4252257590 learned_addr learned my addr 172.21.15.59:0/4252257590 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978fe00640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 0x7f97900af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 237815764 0 0) 0x7f979015f0f0 con 0x7f97900af920 2024-04-23T14:31:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9774003520 con 0x7f97900af920 2024-04-23T14:31:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3740671686 0 0) 0x7f9774003520 con 0x7f97900af920 2024-04-23T14:31:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 msgr2=0x7f97900af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 0x7f97900af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 >> v1:172.21.15.76:6789/0 conn(0x7f97900d08b0 legacy=0x7f97901624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979015f6b0 con 0x7f97900af920 2024-04-23T14:31:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f977c001670 con 0x7f97900af920 2024-04-23T14:31:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 >> v1:172.21.15.59:6789/0 conn(0x7f97900af920 legacy=0x7f97900d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97740039e0 0x7f9774003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9774004800 0x7f9774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 0x7f97740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f977400ba70 con 0x7f9774004800 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f977400bbf0 con 0x7f97740039e0 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.993+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f977400bdf0 con 0x7f979015c5e0 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978f400640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9774004800 0x7f9774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978fe00640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97740039e0 0x7f9774003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f9795000640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 0x7f97740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978f400640 1 -- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 msgr2=0x7f97740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978f400640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f979015c5e0 0x7f97740051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978f400640 1 -- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97740039e0 msgr2=0x7f9774003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978f400640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97740039e0 0x7f9774003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978f400640 1 -- 172.21.15.59:0/4252257590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9774003520 con 0x7f9774004800 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.994+0000 7f978fe00640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97740039e0 0x7f9774003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.995+0000 7f978f400640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9774004800 0x7f9774004c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c210 tx=0x7f9774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9fe4dbe4581bed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.995+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c007d10 con 0x7f9774004800 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.995+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977c005ad0 con 0x7f9774004800 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.995+0000 7f978ea00640 1 -- 172.21.15.59:0/4252257590 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c007070 con 0x7f9774004800 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.996+0000 7f9797000640 1 -- 172.21.15.59:0/4252257590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9774004800 msgr2=0x7f9774004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.996+0000 7f9797000640 1 --2- 172.21.15.59:0/4252257590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9774004800 0x7f9774004c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c210 tx=0x7f9774005830 comp rx=0 tx=0).stop 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.996+0000 7f9795000640 1 -- 172.21.15.59:0/4252257590 reap_dead start 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.996+0000 7f9797000640 1 -- 172.21.15.59:0/4252257590 shutdown_connections 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.996+0000 7f9797000640 1 -- 172.21.15.59:0/4252257590 >> 172.21.15.59:0/4252257590 conn(0x7f9790067ea0 msgr2=0x7f979006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.996+0000 7f9797000640 1 -- 172.21.15.59:0/4252257590 shutdown_connections 2024-04-23T14:31:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.996+0000 7f9797000640 1 -- 172.21.15.59:0/4252257590 wait complete. 2024-04-23T14:31:36.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.998+0000 7f9797000640 1 Processor -- start 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.998+0000 7f9797000640 1 -- start start 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9797000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9774004800 0x7f979014bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9797000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9790158720 0x7f97901590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9797000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97901595f0 0x7f97900b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9797000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97900af3e0 con 0x7f97901595f0 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9797000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f979015fa20 con 0x7f9790158720 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9797000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f97900af560 con 0x7f9774004800 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97901595f0 0x7f97900b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97901595f0 0x7f97900b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33554/0 (socket says 172.21.15.59:33554) 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 -- 172.21.15.59:0/555211174 learned_addr learned my addr 172.21.15.59:0/555211174 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f978f400640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9790158720 0x7f97901590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f978fe00640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9774004800 0x7f979014bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 -- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9774004800 msgr2=0x7f979014bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9774004800 0x7f979014bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 -- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9790158720 msgr2=0x7f97901590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9790158720 0x7f97901590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f9795000640 1 -- 172.21.15.59:0/555211174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97900b1e40 con 0x7f97901595f0 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f978fe00640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9774004800 0x7f979014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:35.999+0000 7f978f400640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9790158720 0x7f97901590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.000+0000 7f9795000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97901595f0 0x7f97900b0d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9790072e60 tx=0x7f978800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.000+0000 7f9787e00640 1 -- 172.21.15.59:0/555211174 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788018070 con 0x7f97901595f0 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.000+0000 7f9787e00640 1 -- 172.21.15.59:0/555211174 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9788003b70 con 0x7f97901595f0 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.000+0000 7f9787e00640 1 -- 172.21.15.59:0/555211174 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788005070 con 0x7f97901595f0 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.000+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979015b190 con 0x7f97901595f0 2024-04-23T14:31:36.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.000+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97900b23e0 con 0x7f97901595f0 2024-04-23T14:31:36.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.001+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9790068a80 con 0x7f97901595f0 2024-04-23T14:31:36.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.002+0000 7f9787e00640 1 -- 172.21.15.59:0/555211174 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f978801b070 con 0x7f97901595f0 2024-04-23T14:31:36.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.003+0000 7f9787e00640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f977006c8f0 0x7f977006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.003+0000 7f9787e00640 1 -- 172.21.15.59:0/555211174 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f978808b2e0 con 0x7f97901595f0 2024-04-23T14:31:36.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.003+0000 7f978fe00640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f977006c8f0 0x7f977006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.004+0000 7f978fe00640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f977006c8f0 0x7f977006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9780009aa0 tx=0x7f9780008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.007+0000 7f9787e00640 1 -- 172.21.15.59:0/555211174 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9788059670 con 0x7f97901595f0 2024-04-23T14:31:36.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.203+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97900b25b0 con 0x7f977006c8f0 2024-04-23T14:31:36.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.221+0000 7f9787e00640 1 -- 172.21.15.59:0/555211174 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f97900b25b0 con 0x7f977006c8f0 2024-04-23T14:31:36.225 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:36.227 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:31:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f977006c8f0 msgr2=0x7f977006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f977006c8f0 0x7f977006edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9780009aa0 tx=0x7f9780008040 comp rx=0 tx=0).stop 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97901595f0 msgr2=0x7f97900b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97901595f0 0x7f97900b0d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9790072e60 tx=0x7f978800dc00 comp rx=0 tx=0).stop 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 shutdown_connections 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f977006c8f0 0x7f977006edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97901595f0 0x7f97900b0d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9790158720 0x7f97901590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 --2- 172.21.15.59:0/555211174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9774004800 0x7f979014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 >> 172.21.15.59:0/555211174 conn(0x7f9790067ea0 msgr2=0x7f979006f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 shutdown_connections 2024-04-23T14:31:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.226+0000 7f9797000640 1 -- 172.21.15.59:0/555211174 wait complete. 2024-04-23T14:31:36.245 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-23T14:31:34.346170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.556306"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227768+0000","last_change":"2024-04-23T14:31:25.849551+0000","last_active":"2024-04-23T14:31:27.227768+0000","last_peered":"2024-04-23T14:31:27.227768+0000","last_clean":"2024-04-23T14:31:27.227768+0000","last_became_active":"2024-04-23T14:31:25.848756+0000","last_became_peered":"2024-04-23T14:31:25.848756+0000","last_unstale":"2024-04-23T14:31:27.227768+0000","last_undegraded":"2024-04-23T14:31:27.227768+0000","last_fullsized":"2024-04-23T14:31:27.227768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833869+0000","last_change":"2024-04-23T14:31:25.819653+0000","last_active":"2024-04-23T14:31:26.833869+0000","last_peered":"2024-04-23T14:31:26.833869+0000","last_clean":"2024-04-23T14:31:26.833869+0000","last_became_active":"2024-04-23T14:31:25.819296+0000","last_became_peered":"2024-04-23T14:31:25.819296+0000","last_unstale":"2024-04-23T14:31:26.833869+0000","last_undegraded":"2024-04-23T14:31:26.833869+0000","last_fullsized":"2024-04-23T14:31:26.833869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225689+0000","last_change":"2024-04-23T14:31:25.818736+0000","last_active":"2024-04-23T14:31:27.225689+0000","last_peered":"2024-04-23T14:31:27.225689+0000","last_clean":"2024-04-23T14:31:27.225689+0000","last_became_active":"2024-04-23T14:31:25.816583+0000","last_became_peered":"2024-04-23T14:31:25.816583+0000","last_unstale":"2024-04-23T14:31:27.225689+0000","last_undegraded":"2024-04-23T14:31:27.225689+0000","last_fullsized":"2024-04-23T14:31:27.225689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229281+0000","last_change":"2024-04-23T14:31:25.836663+0000","last_active":"2024-04-23T14:31:27.229281+0000","last_peered":"2024-04-23T14:31:27.229281+0000","last_clean":"2024-04-23T14:31:27.229281+0000","last_became_active":"2024-04-23T14:31:25.836357+0000","last_became_peered":"2024-04-23T14:31:25.836357+0000","last_unstale":"2024-04-23T14:31:27.229281+0000","last_undegraded":"2024-04-23T14:31:27.229281+0000","last_fullsized":"2024-04-23T14:31:27.229281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227740+0000","last_change":"2024-04-23T14:31:25.830442+0000","last_active":"2024-04-23T14:31:27.227740+0000","last_peered":"2024-04-23T14:31:27.227740+0000","last_clean":"2024-04-23T14:31:27.227740+0000","last_became_active":"2024-04-23T14:31:25.830032+0000","last_became_peered":"2024-04-23T14:31:25.830032+0000","last_unstale":"2024-04-23T14:31:27.227740+0000","last_undegraded":"2024-04-23T14:31:27.227740+0000","last_fullsized":"2024-04-23T14:31:27.227740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228994+0000","last_change":"2024-04-23T14:31:25.835030+0000","last_active":"2024-04-23T14:31:27.228994+0000","last_peered":"2024-04-23T14:31:27.228994+0000","last_clean":"2024-04-23T14:31:27.228994+0000","last_became_active":"2024-04-23T14:31:25.834764+0000","last_became_peered":"2024-04-23T14:31:25.834764+0000","last_unstale":"2024-04-23T14:31:27.228994+0000","last_undegraded":"2024-04-23T14:31:27.228994+0000","last_fullsized":"2024-04-23T14:31:27.228994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226818+0000","last_change":"2024-04-23T14:31:25.833373+0000","last_active":"2024-04-23T14:31:27.226818+0000","last_peered":"2024-04-23T14:31:27.226818+0000","last_clean":"2024-04-23T14:31:27.226818+0000","last_became_active":"2024-04-23T14:31:25.831555+0000","last_became_peered":"2024-04-23T14:31:25.831555+0000","last_unstale":"2024-04-23T14:31:27.226818+0000","last_undegraded":"2024-04-23T14:31:27.226818+0000","last_fullsized":"2024-04-23T14:31:27.226818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224117+0000","last_change":"2024-04-23T14:31:25.828675+0000","last_active":"2024-04-23T14:31:27.224117+0000","last_peered":"2024-04-23T14:31:27.224117+0000","last_clean":"2024-04-23T14:31:27.224117+0000","last_became_active":"2024-04-23T14:31:25.828381+0000","last_became_peered":"2024-04-23T14:31:25.828381+0000","last_unstale":"2024-04-23T14:31:27.224117+0000","last_undegraded":"2024-04-23T14:31:27.224117+0000","last_fullsized":"2024-04-23T14:31:27.224117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228299+0000","last_change":"2024-04-23T14:31:25.839705+0000","last_active":"2024-04-23T14:31:27.228299+0000","last_peered":"2024-04-23T14:31:27.228299+0000","last_clean":"2024-04-23T14:31:27.228299+0000","last_became_active":"2024-04-23T14:31:25.839432+0000","last_became_peered":"2024-04-23T14:31:25.839432+0000","last_unstale":"2024-04-23T14:31:27.228299+0000","last_undegraded":"2024-04-23T14:31:27.228299+0000","last_fullsized":"2024-04-23T14:31:27.228299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821382+0000","last_change":"2024-04-23T14:31:25.833382+0000","last_active":"2024-04-23T14:31:26.821382+0000","last_peered":"2024-04-23T14:31:26.821382+0000","last_clean":"2024-04-23T14:31:26.821382+0000","last_became_active":"2024-04-23T14:31:25.833010+0000","last_became_peered":"2024-04-23T14:31:25.833010+0000","last_unstale":"2024-04-23T14:31:26.821382+0000","last_undegraded":"2024-04-23T14:31:26.821382+0000","last_fullsized":"2024-04-23T14:31:26.821382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820889+0000","last_change":"2024-04-23T14:31:25.828614+0000","last_active":"2024-04-23T14:31:26.820889+0000","last_peered":"2024-04-23T14:31:26.820889+0000","last_clean":"2024-04-23T14:31:26.820889+0000","last_became_active":"2024-04-23T14:31:25.828342+0000","last_became_peered":"2024-04-23T14:31:25.828342+0000","last_unstale":"2024-04-23T14:31:26.820889+0000","last_undegraded":"2024-04-23T14:31:26.820889+0000","last_fullsized":"2024-04-23T14:31:26.820889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821732+0000","last_change":"2024-04-23T14:31:25.825396+0000","last_active":"2024-04-23T14:31:26.821732+0000","last_peered":"2024-04-23T14:31:26.821732+0000","last_clean":"2024-04-23T14:31:26.821732+0000","last_became_active":"2024-04-23T14:31:25.822425+0000","last_became_peered":"2024-04-23T14:31:25.822425+0000","last_unstale":"2024-04-23T14:31:26.821732+0000","last_undegraded":"2024-04-23T14:31:26.821732+0000","last_fullsized":"2024-04-23T14:31:26.821732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:05.846065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223667+0000","last_change":"2024-04-23T14:31:25.823390+0000","last_active":"2024-04-23T14:31:27.223667+0000","last_peered":"2024-04-23T14:31:27.223667+0000","last_clean":"2024-04-23T14:31:27.223667+0000","last_became_active":"2024-04-23T14:31:25.823054+0000","last_became_peered":"2024-04-23T14:31:25.823054+0000","last_unstale":"2024-04-23T14:31:27.223667+0000","last_undegraded":"2024-04-23T14:31:27.223667+0000","last_fullsized":"2024-04-23T14:31:27.223667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225120+0000","last_change":"2024-04-23T14:31:25.818198+0000","last_active":"2024-04-23T14:31:27.225120+0000","last_peered":"2024-04-23T14:31:27.225120+0000","last_clean":"2024-04-23T14:31:27.225120+0000","last_became_active":"2024-04-23T14:31:25.817873+0000","last_became_peered":"2024-04-23T14:31:25.817873+0000","last_unstale":"2024-04-23T14:31:27.225120+0000","last_undegraded":"2024-04-23T14:31:27.225120+0000","last_fullsized":"2024-04-23T14:31:27.225120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:08:54.362020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833481+0000","last_change":"2024-04-23T14:31:25.829886+0000","last_active":"2024-04-23T14:31:26.833481+0000","last_peered":"2024-04-23T14:31:26.833481+0000","last_clean":"2024-04-23T14:31:26.833481+0000","last_became_active":"2024-04-23T14:31:25.829086+0000","last_became_peered":"2024-04-23T14:31:25.829086+0000","last_unstale":"2024-04-23T14:31:26.833481+0000","last_undegraded":"2024-04-23T14:31:26.833481+0000","last_fullsized":"2024-04-23T14:31:26.833481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228453+0000","last_change":"2024-04-23T14:31:25.818775+0000","last_active":"2024-04-23T14:31:27.228453+0000","last_peered":"2024-04-23T14:31:27.228453+0000","last_clean":"2024-04-23T14:31:27.228453+0000","last_became_active":"2024-04-23T14:31:25.818318+0000","last_became_peered":"2024-04-23T14:31:25.818318+0000","last_unstale":"2024-04-23T14:31:27.228453+0000","last_undegraded":"2024-04-23T14:31:27.228453+0000","last_fullsized":"2024-04-23T14:31:27.228453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230731+0000","last_change":"2024-04-23T14:31:25.850111+0000","last_active":"2024-04-23T14:31:27.230731+0000","last_peered":"2024-04-23T14:31:27.230731+0000","last_clean":"2024-04-23T14:31:27.230731+0000","last_became_active":"2024-04-23T14:31:25.848968+0000","last_became_peered":"2024-04-23T14:31:25.848968+0000","last_unstale":"2024-04-23T14:31:27.230731+0000","last_undegraded":"2024-04-23T14:31:27.230731+0000","last_fullsized":"2024-04-23T14:31:27.230731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.815057+0000","last_change":"2024-04-23T14:31:25.813711+0000","last_active":"2024-04-23T14:31:26.815057+0000","last_peered":"2024-04-23T14:31:26.815057+0000","last_clean":"2024-04-23T14:31:26.815057+0000","last_became_active":"2024-04-23T14:31:25.813377+0000","last_became_peered":"2024-04-23T14:31:25.813377+0000","last_unstale":"2024-04-23T14:31:26.815057+0000","last_undegraded":"2024-04-23T14:31:26.815057+0000","last_fullsized":"2024-04-23T14:31:26.815057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230045+0000","last_change":"2024-04-23T14:31:25.842518+0000","last_active":"2024-04-23T14:31:27.230045+0000","last_peered":"2024-04-23T14:31:27.230045+0000","last_clean":"2024-04-23T14:31:27.230045+0000","last_became_active":"2024-04-23T14:31:25.842265+0000","last_became_peered":"2024-04-23T14:31:25.842265+0000","last_unstale":"2024-04-23T14:31:27.230045+0000","last_undegraded":"2024-04-23T14:31:27.230045+0000","last_fullsized":"2024-04-23T14:31:27.230045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:10:49.364737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820245+0000","last_change":"2024-04-23T14:31:25.824601+0000","last_active":"2024-04-23T14:31:26.820245+0000","last_peered":"2024-04-23T14:31:26.820245+0000","last_clean":"2024-04-23T14:31:26.820245+0000","last_became_active":"2024-04-23T14:31:25.824326+0000","last_became_peered":"2024-04-23T14:31:25.824326+0000","last_unstale":"2024-04-23T14:31:26.820245+0000","last_undegraded":"2024-04-23T14:31:26.820245+0000","last_fullsized":"2024-04-23T14:31:26.820245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225202+0000","last_change":"2024-04-23T14:31:25.824382+0000","last_active":"2024-04-23T14:31:27.225202+0000","last_peered":"2024-04-23T14:31:27.225202+0000","last_clean":"2024-04-23T14:31:27.225202+0000","last_became_active":"2024-04-23T14:31:25.823991+0000","last_became_peered":"2024-04-23T14:31:25.823991+0000","last_unstale":"2024-04-23T14:31:27.225202+0000","last_undegraded":"2024-04-23T14:31:27.225202+0000","last_fullsized":"2024-04-23T14:31:27.225202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229105+0000","last_change":"2024-04-23T14:31:25.842389+0000","last_active":"2024-04-23T14:31:27.229105+0000","last_peered":"2024-04-23T14:31:27.229105+0000","last_clean":"2024-04-23T14:31:27.229105+0000","last_became_active":"2024-04-23T14:31:25.841584+0000","last_became_peered":"2024-04-23T14:31:25.841584+0000","last_unstale":"2024-04-23T14:31:27.229105+0000","last_undegraded":"2024-04-23T14:31:27.229105+0000","last_fullsized":"2024-04-23T14:31:27.229105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223827+0000","last_change":"2024-04-23T14:31:25.830515+0000","last_active":"2024-04-23T14:31:27.223827+0000","last_peered":"2024-04-23T14:31:27.223827+0000","last_clean":"2024-04-23T14:31:27.223827+0000","last_became_active":"2024-04-23T14:31:25.830209+0000","last_became_peered":"2024-04-23T14:31:25.830209+0000","last_unstale":"2024-04-23T14:31:27.223827+0000","last_undegraded":"2024-04-23T14:31:27.223827+0000","last_fullsized":"2024-04-23T14:31:27.223827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228540+0000","last_change":"2024-04-23T14:31:25.824411+0000","last_active":"2024-04-23T14:31:27.228540+0000","last_peered":"2024-04-23T14:31:27.228540+0000","last_clean":"2024-04-23T14:31:27.228540+0000","last_became_active":"2024-04-23T14:31:25.824052+0000","last_became_peered":"2024-04-23T14:31:25.824052+0000","last_unstale":"2024-04-23T14:31:27.228540+0000","last_undegraded":"2024-04-23T14:31:27.228540+0000","last_fullsized":"2024-04-23T14:31:27.228540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224322+0000","last_change":"2024-04-23T14:31:25.830939+0000","last_active":"2024-04-23T14:31:27.224322+0000","last_peered":"2024-04-23T14:31:27.224322+0000","last_clean":"2024-04-23T14:31:27.224322+0000","last_became_active":"2024-04-23T14:31:25.830632+0000","last_became_peered":"2024-04-23T14:31:25.830632+0000","last_unstale":"2024-04-23T14:31:27.224322+0000","last_undegraded":"2024-04-23T14:31:27.224322+0000","last_fullsized":"2024-04-23T14:31:27.224322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821455+0000","last_change":"2024-04-23T14:31:25.832634+0000","last_active":"2024-04-23T14:31:26.821455+0000","last_peered":"2024-04-23T14:31:26.821455+0000","last_clean":"2024-04-23T14:31:26.821455+0000","last_became_active":"2024-04-23T14:31:25.832329+0000","last_became_peered":"2024-04-23T14:31:25.832329+0000","last_unstale":"2024-04-23T14:31:26.821455+0000","last_undegraded":"2024-04-23T14:31:26.821455+0000","last_fullsized":"2024-04-23T14:31:26.821455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833312+0000","last_change":"2024-04-23T14:31:25.822589+0000","last_active":"2024-04-23T14:31:26.833312+0000","last_peered":"2024-04-23T14:31:26.833312+0000","last_clean":"2024-04-23T14:31:26.833312+0000","last_became_active":"2024-04-23T14:31:25.822061+0000","last_became_peered":"2024-04-23T14:31:25.822061+0000","last_unstale":"2024-04-23T14:31:26.833312+0000","last_undegraded":"2024-04-23T14:31:26.833312+0000","last_fullsized":"2024-04-23T14:31:26.833312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223980+0000","last_change":"2024-04-23T14:31:25.832096+0000","last_active":"2024-04-23T14:31:27.223980+0000","last_peered":"2024-04-23T14:31:27.223980+0000","last_clean":"2024-04-23T14:31:27.223980+0000","last_became_active":"2024-04-23T14:31:25.829333+0000","last_became_peered":"2024-04-23T14:31:25.829333+0000","last_unstale":"2024-04-23T14:31:27.223980+0000","last_undegraded":"2024-04-23T14:31:27.223980+0000","last_fullsized":"2024-04-23T14:31:27.223980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225363+0000","last_change":"2024-04-23T14:31:25.826410+0000","last_active":"2024-04-23T14:31:27.225363+0000","last_peered":"2024-04-23T14:31:27.225363+0000","last_clean":"2024-04-23T14:31:27.225363+0000","last_became_active":"2024-04-23T14:31:25.826088+0000","last_became_peered":"2024-04-23T14:31:25.826088+0000","last_unstale":"2024-04-23T14:31:27.225363+0000","last_undegraded":"2024-04-23T14:31:27.225363+0000","last_fullsized":"2024-04-23T14:31:27.225363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229439+0000","last_change":"2024-04-23T14:31:25.832202+0000","last_active":"2024-04-23T14:31:27.229439+0000","last_peered":"2024-04-23T14:31:27.229439+0000","last_clean":"2024-04-23T14:31:27.229439+0000","last_became_active":"2024-04-23T14:31:25.831719+0000","last_became_peered":"2024-04-23T14:31:25.831719+0000","last_unstale":"2024-04-23T14:31:27.229439+0000","last_undegraded":"2024-04-23T14:31:27.229439+0000","last_fullsized":"2024-04-23T14:31:27.229439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814413+0000","last_change":"2024-04-23T14:31:25.827667+0000","last_active":"2024-04-23T14:31:26.814413+0000","last_peered":"2024-04-23T14:31:26.814413+0000","last_clean":"2024-04-23T14:31:26.814413+0000","last_became_active":"2024-04-23T14:31:25.827302+0000","last_became_peered":"2024-04-23T14:31:25.827302+0000","last_unstale":"2024-04-23T14:31:26.814413+0000","last_undegraded":"2024-04-23T14:31:26.814413+0000","last_fullsized":"2024-04-23T14:31:26.814413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.819912+0000","last_change":"2024-04-23T14:31:25.817837+0000","last_active":"2024-04-23T14:31:26.819912+0000","last_peered":"2024-04-23T14:31:26.819912+0000","last_clean":"2024-04-23T14:31:26.819912+0000","last_became_active":"2024-04-23T14:31:25.817156+0000","last_became_peered":"2024-04-23T14:31:25.817156+0000","last_unstale":"2024-04-23T14:31:26.819912+0000","last_undegraded":"2024-04-23T14:31:26.819912+0000","last_fullsized":"2024-04-23T14:31:26.819912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820171+0000","last_change":"2024-04-23T14:31:25.821505+0000","last_active":"2024-04-23T14:31:26.820171+0000","last_peered":"2024-04-23T14:31:26.820171+0000","last_clean":"2024-04-23T14:31:26.820171+0000","last_became_active":"2024-04-23T14:31:25.821186+0000","last_became_peered":"2024-04-23T14:31:25.821186+0000","last_unstale":"2024-04-23T14:31:26.820171+0000","last_undegraded":"2024-04-23T14:31:26.820171+0000","last_fullsized":"2024-04-23T14:31:26.820171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:30.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224053+0000","last_change":"2024-04-23T14:31:24.810696+0000","last_active":"2024-04-23T14:31:27.224053+0000","last_peered":"2024-04-23T14:31:27.224053+0000","last_clean":"2024-04-23T14:31:27.224053+0000","last_became_active":"2024-04-23T14:31:24.810349+0000","last_became_peered":"2024-04-23T14:31:24.810349+0000","last_unstale":"2024-04-23T14:31:27.224053+0000","last_undegraded":"2024-04-23T14:31:27.224053+0000","last_fullsized":"2024-04-23T14:31:27.224053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225550+0000","last_change":"2024-04-23T14:31:25.824408+0000","last_active":"2024-04-23T14:31:27.225550+0000","last_peered":"2024-04-23T14:31:27.225550+0000","last_clean":"2024-04-23T14:31:27.225550+0000","last_became_active":"2024-04-23T14:31:25.822620+0000","last_became_peered":"2024-04-23T14:31:25.822620+0000","last_unstale":"2024-04-23T14:31:27.225550+0000","last_undegraded":"2024-04-23T14:31:27.225550+0000","last_fullsized":"2024-04-23T14:31:27.225550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820803+0000","last_change":"2024-04-23T14:31:24.801642+0000","last_active":"2024-04-23T14:31:26.820803+0000","last_peered":"2024-04-23T14:31:26.820803+0000","last_clean":"2024-04-23T14:31:26.820803+0000","last_became_active":"2024-04-23T14:31:24.801244+0000","last_became_peered":"2024-04-23T14:31:24.801244+0000","last_unstale":"2024-04-23T14:31:26.820803+0000","last_undegraded":"2024-04-23T14:31:26.820803+0000","last_fullsized":"2024-04-23T14:31:26.820803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223904+0000","last_change":"2024-04-23T14:31:24.796017+0000","last_active":"2024-04-23T14:31:27.223904+0000","last_peered":"2024-04-23T14:31:27.223904+0000","last_clean":"2024-04-23T14:31:27.223904+0000","last_became_active":"2024-04-23T14:31:24.795734+0000","last_became_peered":"2024-04-23T14:31:24.795734+0000","last_unstale":"2024-04-23T14:31:27.223904+0000","last_undegraded":"2024-04-23T14:31:27.223904+0000","last_fullsized":"2024-04-23T14:31:27.223904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223975+0000","last_change":"2024-04-23T14:31:25.819835+0000","last_active":"2024-04-23T14:31:27.223975+0000","last_peered":"2024-04-23T14:31:27.223975+0000","last_clean":"2024-04-23T14:31:27.223975+0000","last_became_active":"2024-04-23T14:31:25.819532+0000","last_became_peered":"2024-04-23T14:31:25.819532+0000","last_unstale":"2024-04-23T14:31:27.223975+0000","last_undegraded":"2024-04-23T14:31:27.223975+0000","last_fullsized":"2024-04-23T14:31:27.223975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228610+0000","last_change":"2024-04-23T14:31:24.802566+0000","last_active":"2024-04-23T14:31:27.228610+0000","last_peered":"2024-04-23T14:31:27.228610+0000","last_clean":"2024-04-23T14:31:27.228610+0000","last_became_active":"2024-04-23T14:31:24.802299+0000","last_became_peered":"2024-04-23T14:31:24.802299+0000","last_unstale":"2024-04-23T14:31:27.228610+0000","last_undegraded":"2024-04-23T14:31:27.228610+0000","last_fullsized":"2024-04-23T14:31:27.228610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833252+0000","last_change":"2024-04-23T14:31:25.822370+0000","last_active":"2024-04-23T14:31:26.833252+0000","last_peered":"2024-04-23T14:31:26.833252+0000","last_clean":"2024-04-23T14:31:26.833252+0000","last_became_active":"2024-04-23T14:31:25.822039+0000","last_became_peered":"2024-04-23T14:31:25.822039+0000","last_unstale":"2024-04-23T14:31:26.833252+0000","last_undegraded":"2024-04-23T14:31:26.833252+0000","last_fullsized":"2024-04-23T14:31:26.833252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821576+0000","last_change":"2024-04-23T14:31:24.806864+0000","last_active":"2024-04-23T14:31:26.821576+0000","last_peered":"2024-04-23T14:31:26.821576+0000","last_clean":"2024-04-23T14:31:26.821576+0000","last_became_active":"2024-04-23T14:31:24.806597+0000","last_became_peered":"2024-04-23T14:31:24.806597+0000","last_unstale":"2024-04-23T14:31:26.821576+0000","last_undegraded":"2024-04-23T14:31:26.821576+0000","last_fullsized":"2024-04-23T14:31:26.821576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230291+0000","last_change":"2024-04-23T14:31:25.842918+0000","last_active":"2024-04-23T14:31:27.230291+0000","last_peered":"2024-04-23T14:31:27.230291+0000","last_clean":"2024-04-23T14:31:27.230291+0000","last_became_active":"2024-04-23T14:31:25.842581+0000","last_became_peered":"2024-04-23T14:31:25.842581+0000","last_unstale":"2024-04-23T14:31:27.230291+0000","last_undegraded":"2024-04-23T14:31:27.230291+0000","last_fullsized":"2024-04-23T14:31:27.230291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229964+0000","last_change":"2024-04-23T14:31:24.805711+0000","last_active":"2024-04-23T14:31:27.229964+0000","last_peered":"2024-04-23T14:31:27.229964+0000","last_clean":"2024-04-23T14:31:27.229964+0000","last_became_active":"2024-04-23T14:31:24.805351+0000","last_became_peered":"2024-04-23T14:31:24.805351+0000","last_unstale":"2024-04-23T14:31:27.229964+0000","last_undegraded":"2024-04-23T14:31:27.229964+0000","last_fullsized":"2024-04-23T14:31:27.229964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225997+0000","last_change":"2024-04-23T14:31:25.837626+0000","last_active":"2024-04-23T14:31:27.225997+0000","last_peered":"2024-04-23T14:31:27.225997+0000","last_clean":"2024-04-23T14:31:27.225997+0000","last_became_active":"2024-04-23T14:31:25.836840+0000","last_became_peered":"2024-04-23T14:31:25.836840+0000","last_unstale":"2024-04-23T14:31:27.225997+0000","last_undegraded":"2024-04-23T14:31:27.225997+0000","last_fullsized":"2024-04-23T14:31:27.225997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833398+0000","last_change":"2024-04-23T14:31:24.802741+0000","last_active":"2024-04-23T14:31:26.833398+0000","last_peered":"2024-04-23T14:31:26.833398+0000","last_clean":"2024-04-23T14:31:26.833398+0000","last_became_active":"2024-04-23T14:31:24.802462+0000","last_became_peered":"2024-04-23T14:31:24.802462+0000","last_unstale":"2024-04-23T14:31:26.833398+0000","last_undegraded":"2024-04-23T14:31:26.833398+0000","last_fullsized":"2024-04-23T14:31:26.833398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821929+0000","last_change":"2024-04-23T14:31:25.840470+0000","last_active":"2024-04-23T14:31:26.821929+0000","last_peered":"2024-04-23T14:31:26.821929+0000","last_clean":"2024-04-23T14:31:26.821929+0000","last_became_active":"2024-04-23T14:31:25.840160+0000","last_became_peered":"2024-04-23T14:31:25.840160+0000","last_unstale":"2024-04-23T14:31:26.821929+0000","last_undegraded":"2024-04-23T14:31:26.821929+0000","last_fullsized":"2024-04-23T14:31:26.821929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820475+0000","last_change":"2024-04-23T14:31:24.798634+0000","last_active":"2024-04-23T14:31:26.820475+0000","last_peered":"2024-04-23T14:31:26.820475+0000","last_clean":"2024-04-23T14:31:26.820475+0000","last_became_active":"2024-04-23T14:31:24.798307+0000","last_became_peered":"2024-04-23T14:31:24.798307+0000","last_unstale":"2024-04-23T14:31:26.820475+0000","last_undegraded":"2024-04-23T14:31:26.820475+0000","last_fullsized":"2024-04-23T14:31:26.820475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227416+0000","last_change":"2024-04-23T14:31:25.830485+0000","last_active":"2024-04-23T14:31:27.227416+0000","last_peered":"2024-04-23T14:31:27.227416+0000","last_clean":"2024-04-23T14:31:27.227416+0000","last_became_active":"2024-04-23T14:31:25.830001+0000","last_became_peered":"2024-04-23T14:31:25.830001+0000","last_unstale":"2024-04-23T14:31:27.227416+0000","last_undegraded":"2024-04-23T14:31:27.227416+0000","last_fullsized":"2024-04-23T14:31:27.227416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225604+0000","last_change":"2024-04-23T14:31:24.800143+0000","last_active":"2024-04-23T14:31:27.225604+0000","last_peered":"2024-04-23T14:31:27.225604+0000","last_clean":"2024-04-23T14:31:27.225604+0000","last_became_active":"2024-04-23T14:31:24.799828+0000","last_became_peered":"2024-04-23T14:31:24.799828+0000","last_unstale":"2024-04-23T14:31:27.225604+0000","last_undegraded":"2024-04-23T14:31:27.225604+0000","last_fullsized":"2024-04-23T14:31:27.225604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225544+0000","last_change":"2024-04-23T14:31:25.839070+0000","last_active":"2024-04-23T14:31:27.225544+0000","last_peered":"2024-04-23T14:31:27.225544+0000","last_clean":"2024-04-23T14:31:27.225544+0000","last_became_active":"2024-04-23T14:31:25.838724+0000","last_became_peered":"2024-04-23T14:31:25.838724+0000","last_unstale":"2024-04-23T14:31:27.225544+0000","last_undegraded":"2024-04-23T14:31:27.225544+0000","last_fullsized":"2024-04-23T14:31:27.225544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229381+0000","last_change":"2024-04-23T14:31:24.805607+0000","last_active":"2024-04-23T14:31:27.229381+0000","last_peered":"2024-04-23T14:31:27.229381+0000","last_clean":"2024-04-23T14:31:27.229381+0000","last_became_active":"2024-04-23T14:31:24.805268+0000","last_became_peered":"2024-04-23T14:31:24.805268+0000","last_unstale":"2024-04-23T14:31:27.229381+0000","last_undegraded":"2024-04-23T14:31:27.229381+0000","last_fullsized":"2024-04-23T14:31:27.229381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228113+0000","last_change":"2024-04-23T14:31:25.835264+0000","last_active":"2024-04-23T14:31:27.228113+0000","last_peered":"2024-04-23T14:31:27.228113+0000","last_clean":"2024-04-23T14:31:27.228113+0000","last_became_active":"2024-04-23T14:31:25.834845+0000","last_became_peered":"2024-04-23T14:31:25.834845+0000","last_unstale":"2024-04-23T14:31:27.228113+0000","last_undegraded":"2024-04-23T14:31:27.228113+0000","last_fullsized":"2024-04-23T14:31:27.228113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820500+0000","last_change":"2024-04-23T14:31:24.803064+0000","last_active":"2024-04-23T14:31:26.820500+0000","last_peered":"2024-04-23T14:31:26.820500+0000","last_clean":"2024-04-23T14:31:26.820500+0000","last_became_active":"2024-04-23T14:31:24.801163+0000","last_became_peered":"2024-04-23T14:31:24.801163+0000","last_unstale":"2024-04-23T14:31:26.820500+0000","last_undegraded":"2024-04-23T14:31:26.820500+0000","last_fullsized":"2024-04-23T14:31:26.820500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820307+0000","last_change":"2024-04-23T14:31:25.818141+0000","last_active":"2024-04-23T14:31:26.820307+0000","last_peered":"2024-04-23T14:31:26.820307+0000","last_clean":"2024-04-23T14:31:26.820307+0000","last_became_active":"2024-04-23T14:31:25.817558+0000","last_became_peered":"2024-04-23T14:31:25.817558+0000","last_unstale":"2024-04-23T14:31:26.820307+0000","last_undegraded":"2024-04-23T14:31:26.820307+0000","last_fullsized":"2024-04-23T14:31:26.820307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814267+0000","last_change":"2024-04-23T14:31:24.792560+0000","last_active":"2024-04-23T14:31:26.814267+0000","last_peered":"2024-04-23T14:31:26.814267+0000","last_clean":"2024-04-23T14:31:26.814267+0000","last_became_active":"2024-04-23T14:31:24.792055+0000","last_became_peered":"2024-04-23T14:31:24.792055+0000","last_unstale":"2024-04-23T14:31:26.814267+0000","last_undegraded":"2024-04-23T14:31:26.814267+0000","last_fullsized":"2024-04-23T14:31:26.814267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226832+0000","last_change":"2024-04-23T14:31:25.833895+0000","last_active":"2024-04-23T14:31:27.226832+0000","last_peered":"2024-04-23T14:31:27.226832+0000","last_clean":"2024-04-23T14:31:27.226832+0000","last_became_active":"2024-04-23T14:31:25.833570+0000","last_became_peered":"2024-04-23T14:31:25.833570+0000","last_unstale":"2024-04-23T14:31:27.226832+0000","last_undegraded":"2024-04-23T14:31:27.226832+0000","last_fullsized":"2024-04-23T14:31:27.226832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820916+0000","last_change":"2024-04-23T14:31:24.794838+0000","last_active":"2024-04-23T14:31:26.820916+0000","last_peered":"2024-04-23T14:31:26.820916+0000","last_clean":"2024-04-23T14:31:26.820916+0000","last_became_active":"2024-04-23T14:31:24.794545+0000","last_became_peered":"2024-04-23T14:31:24.794545+0000","last_unstale":"2024-04-23T14:31:26.820916+0000","last_undegraded":"2024-04-23T14:31:26.820916+0000","last_fullsized":"2024-04-23T14:31:26.820916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:47.137471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228074+0000","last_change":"2024-04-23T14:31:25.850256+0000","last_active":"2024-04-23T14:31:27.228074+0000","last_peered":"2024-04-23T14:31:27.228074+0000","last_clean":"2024-04-23T14:31:27.228074+0000","last_became_active":"2024-04-23T14:31:25.849503+0000","last_became_peered":"2024-04-23T14:31:25.849503+0000","last_unstale":"2024-04-23T14:31:27.228074+0000","last_undegraded":"2024-04-23T14:31:27.228074+0000","last_fullsized":"2024-04-23T14:31:27.228074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814819+0000","last_change":"2024-04-23T14:31:24.798518+0000","last_active":"2024-04-23T14:31:26.814819+0000","last_peered":"2024-04-23T14:31:26.814819+0000","last_clean":"2024-04-23T14:31:26.814819+0000","last_became_active":"2024-04-23T14:31:24.798170+0000","last_became_peered":"2024-04-23T14:31:24.798170+0000","last_unstale":"2024-04-23T14:31:26.814819+0000","last_undegraded":"2024-04-23T14:31:26.814819+0000","last_fullsized":"2024-04-23T14:31:26.814819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228158+0000","last_change":"2024-04-23T14:31:25.849745+0000","last_active":"2024-04-23T14:31:27.228158+0000","last_peered":"2024-04-23T14:31:27.228158+0000","last_clean":"2024-04-23T14:31:27.228158+0000","last_became_active":"2024-04-23T14:31:25.849311+0000","last_became_peered":"2024-04-23T14:31:25.849311+0000","last_unstale":"2024-04-23T14:31:27.228158+0000","last_undegraded":"2024-04-23T14:31:27.228158+0000","last_fullsized":"2024-04-23T14:31:27.228158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833547+0000","last_change":"2024-04-23T14:31:24.801624+0000","last_active":"2024-04-23T14:31:26.833547+0000","last_peered":"2024-04-23T14:31:26.833547+0000","last_clean":"2024-04-23T14:31:26.833547+0000","last_became_active":"2024-04-23T14:31:24.800838+0000","last_became_peered":"2024-04-23T14:31:24.800838+0000","last_unstale":"2024-04-23T14:31:26.833547+0000","last_undegraded":"2024-04-23T14:31:26.833547+0000","last_fullsized":"2024-04-23T14:31:26.833547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229691+0000","last_change":"2024-04-23T14:31:25.836837+0000","last_active":"2024-04-23T14:31:27.229691+0000","last_peered":"2024-04-23T14:31:27.229691+0000","last_clean":"2024-04-23T14:31:27.229691+0000","last_became_active":"2024-04-23T14:31:25.836473+0000","last_became_peered":"2024-04-23T14:31:25.836473+0000","last_unstale":"2024-04-23T14:31:27.229691+0000","last_undegraded":"2024-04-23T14:31:27.229691+0000","last_fullsized":"2024-04-23T14:31:27.229691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814648+0000","last_change":"2024-04-23T14:31:24.801079+0000","last_active":"2024-04-23T14:31:26.814648+0000","last_peered":"2024-04-23T14:31:26.814648+0000","last_clean":"2024-04-23T14:31:26.814648+0000","last_became_active":"2024-04-23T14:31:24.800766+0000","last_became_peered":"2024-04-23T14:31:24.800766+0000","last_unstale":"2024-04-23T14:31:26.814648+0000","last_undegraded":"2024-04-23T14:31:26.814648+0000","last_fullsized":"2024-04-23T14:31:26.814648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229022+0000","last_change":"2024-04-23T14:31:25.842503+0000","last_active":"2024-04-23T14:31:27.229022+0000","last_peered":"2024-04-23T14:31:27.229022+0000","last_clean":"2024-04-23T14:31:27.229022+0000","last_became_active":"2024-04-23T14:31:25.842174+0000","last_became_peered":"2024-04-23T14:31:25.842174+0000","last_unstale":"2024-04-23T14:31:27.229022+0000","last_undegraded":"2024-04-23T14:31:27.229022+0000","last_fullsized":"2024-04-23T14:31:27.229022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:24:27.716211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226647+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:31:27.226647+0000","last_peered":"2024-04-23T14:31:27.226647+0000","last_clean":"2024-04-23T14:31:27.226647+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:31:27.226647+0000","last_undegraded":"2024-04-23T14:31:27.226647+0000","last_fullsized":"2024-04-23T14:31:27.226647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228944+0000","last_change":"2024-04-23T14:31:24.803500+0000","last_active":"2024-04-23T14:31:27.228944+0000","last_peered":"2024-04-23T14:31:27.228944+0000","last_clean":"2024-04-23T14:31:27.228944+0000","last_became_active":"2024-04-23T14:31:24.803188+0000","last_became_peered":"2024-04-23T14:31:24.803188+0000","last_unstale":"2024-04-23T14:31:27.228944+0000","last_undegraded":"2024-04-23T14:31:27.228944+0000","last_fullsized":"2024-04-23T14:31:27.228944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224272+0000","last_change":"2024-04-23T14:31:25.842991+0000","last_active":"2024-04-23T14:31:27.224272+0000","last_peered":"2024-04-23T14:31:27.224272+0000","last_clean":"2024-04-23T14:31:27.224272+0000","last_became_active":"2024-04-23T14:31:25.842681+0000","last_became_peered":"2024-04-23T14:31:25.842681+0000","last_unstale":"2024-04-23T14:31:27.224272+0000","last_undegraded":"2024-04-23T14:31:27.224272+0000","last_fullsized":"2024-04-23T14:31:27.224272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226837+0000","last_change":"2024-04-23T14:31:24.815103+0000","last_active":"2024-04-23T14:31:27.226837+0000","last_peered":"2024-04-23T14:31:27.226837+0000","last_clean":"2024-04-23T14:31:27.226837+0000","last_became_active":"2024-04-23T14:31:24.814780+0000","last_became_peered":"2024-04-23T14:31:24.814780+0000","last_unstale":"2024-04-23T14:31:27.226837+0000","last_undegraded":"2024-04-23T14:31:27.226837+0000","last_fullsized":"2024-04-23T14:31:27.226837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225528+0000","last_change":"2024-04-23T14:31:25.828041+0000","last_active":"2024-04-23T14:31:27.225528+0000","last_peered":"2024-04-23T14:31:27.225528+0000","last_clean":"2024-04-23T14:31:27.225528+0000","last_became_active":"2024-04-23T14:31:25.827517+0000","last_became_peered":"2024-04-23T14:31:25.827517+0000","last_unstale":"2024-04-23T14:31:27.225528+0000","last_undegraded":"2024-04-23T14:31:27.225528+0000","last_fullsized":"2024-04-23T14:31:27.225528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814388+0000","last_change":"2024-04-23T14:31:24.800057+0000","last_active":"2024-04-23T14:31:26.814388+0000","last_peered":"2024-04-23T14:31:26.814388+0000","last_clean":"2024-04-23T14:31:26.814388+0000","last_became_active":"2024-04-23T14:31:24.799710+0000","last_became_peered":"2024-04-23T14:31:24.799710+0000","last_unstale":"2024-04-23T14:31:26.814388+0000","last_undegraded":"2024-04-23T14:31:26.814388+0000","last_fullsized":"2024-04-23T14:31:26.814388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833395+0000","last_change":"2024-04-23T14:31:25.820222+0000","last_active":"2024-04-23T14:31:26.833395+0000","last_peered":"2024-04-23T14:31:26.833395+0000","last_clean":"2024-04-23T14:31:26.833395+0000","last_became_active":"2024-04-23T14:31:25.818514+0000","last_became_peered":"2024-04-23T14:31:25.818514+0000","last_unstale":"2024-04-23T14:31:26.833395+0000","last_undegraded":"2024-04-23T14:31:26.833395+0000","last_fullsized":"2024-04-23T14:31:26.833395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229585+0000","last_change":"2024-04-23T14:31:24.802434+0000","last_active":"2024-04-23T14:31:27.229585+0000","last_peered":"2024-04-23T14:31:27.229585+0000","last_clean":"2024-04-23T14:31:27.229585+0000","last_became_active":"2024-04-23T14:31:24.800504+0000","last_became_peered":"2024-04-23T14:31:24.800504+0000","last_unstale":"2024-04-23T14:31:27.229585+0000","last_undegraded":"2024-04-23T14:31:27.229585+0000","last_fullsized":"2024-04-23T14:31:27.229585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.822417+0000","last_change":"2024-04-23T14:31:25.824755+0000","last_active":"2024-04-23T14:31:26.822417+0000","last_peered":"2024-04-23T14:31:26.822417+0000","last_clean":"2024-04-23T14:31:26.822417+0000","last_became_active":"2024-04-23T14:31:25.820209+0000","last_became_peered":"2024-04-23T14:31:25.820209+0000","last_unstale":"2024-04-23T14:31:26.822417+0000","last_undegraded":"2024-04-23T14:31:26.822417+0000","last_fullsized":"2024-04-23T14:31:26.822417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833340+0000","last_change":"2024-04-23T14:31:24.795154+0000","last_active":"2024-04-23T14:31:26.833340+0000","last_peered":"2024-04-23T14:31:26.833340+0000","last_clean":"2024-04-23T14:31:26.833340+0000","last_became_active":"2024-04-23T14:31:24.794851+0000","last_became_peered":"2024-04-23T14:31:24.794851+0000","last_unstale":"2024-04-23T14:31:26.833340+0000","last_undegraded":"2024-04-23T14:31:26.833340+0000","last_fullsized":"2024-04-23T14:31:26.833340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230608+0000","last_change":"2024-04-23T14:31:25.836961+0000","last_active":"2024-04-23T14:31:27.230608+0000","last_peered":"2024-04-23T14:31:27.230608+0000","last_clean":"2024-04-23T14:31:27.230608+0000","last_became_active":"2024-04-23T14:31:25.836649+0000","last_became_peered":"2024-04-23T14:31:25.836649+0000","last_unstale":"2024-04-23T14:31:27.230608+0000","last_undegraded":"2024-04-23T14:31:27.230608+0000","last_fullsized":"2024-04-23T14:31:27.230608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229915+0000","last_change":"2024-04-23T14:31:24.817767+0000","last_active":"2024-04-23T14:31:27.229915+0000","last_peered":"2024-04-23T14:31:27.229915+0000","last_clean":"2024-04-23T14:31:27.229915+0000","last_became_active":"2024-04-23T14:31:24.817440+0000","last_became_peered":"2024-04-23T14:31:24.817440+0000","last_unstale":"2024-04-23T14:31:27.229915+0000","last_undegraded":"2024-04-23T14:31:27.229915+0000","last_fullsized":"2024-04-23T14:31:27.229915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833407+0000","last_change":"2024-04-23T14:31:25.821773+0000","last_active":"2024-04-23T14:31:26.833407+0000","last_peered":"2024-04-23T14:31:26.833407+0000","last_clean":"2024-04-23T14:31:26.833407+0000","last_became_active":"2024-04-23T14:31:25.820550+0000","last_became_peered":"2024-04-23T14:31:25.820550+0000","last_unstale":"2024-04-23T14:31:26.833407+0000","last_undegraded":"2024-04-23T14:31:26.833407+0000","last_fullsized":"2024-04-23T14:31:26.833407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.834047+0000","last_change":"2024-04-23T14:31:24.798112+0000","last_active":"2024-04-23T14:31:26.834047+0000","last_peered":"2024-04-23T14:31:26.834047+0000","last_clean":"2024-04-23T14:31:26.834047+0000","last_became_active":"2024-04-23T14:31:24.797826+0000","last_became_peered":"2024-04-23T14:31:24.797826+0000","last_unstale":"2024-04-23T14:31:26.834047+0000","last_undegraded":"2024-04-23T14:31:26.834047+0000","last_fullsized":"2024-04-23T14:31:26.834047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225465+0000","last_change":"2024-04-23T14:31:25.835731+0000","last_active":"2024-04-23T14:31:27.225465+0000","last_peered":"2024-04-23T14:31:27.225465+0000","last_clean":"2024-04-23T14:31:27.225465+0000","last_became_active":"2024-04-23T14:31:25.835493+0000","last_became_peered":"2024-04-23T14:31:25.835493+0000","last_unstale":"2024-04-23T14:31:27.225465+0000","last_undegraded":"2024-04-23T14:31:27.225465+0000","last_fullsized":"2024-04-23T14:31:27.225465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821845+0000","last_change":"2024-04-23T14:31:24.813578+0000","last_active":"2024-04-23T14:31:26.821845+0000","last_peered":"2024-04-23T14:31:26.821845+0000","last_clean":"2024-04-23T14:31:26.821845+0000","last_became_active":"2024-04-23T14:31:24.812481+0000","last_became_peered":"2024-04-23T14:31:24.812481+0000","last_unstale":"2024-04-23T14:31:26.821845+0000","last_undegraded":"2024-04-23T14:31:26.821845+0000","last_fullsized":"2024-04-23T14:31:26.821845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225467+0000","last_change":"2024-04-23T14:31:25.808683+0000","last_active":"2024-04-23T14:31:27.225467+0000","last_peered":"2024-04-23T14:31:27.225467+0000","last_clean":"2024-04-23T14:31:27.225467+0000","last_became_active":"2024-04-23T14:31:25.808301+0000","last_became_peered":"2024-04-23T14:31:25.808301+0000","last_unstale":"2024-04-23T14:31:27.225467+0000","last_undegraded":"2024-04-23T14:31:27.225467+0000","last_fullsized":"2024-04-23T14:31:27.225467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226336+0000","last_change":"2024-04-23T14:31:24.817508+0000","last_active":"2024-04-23T14:31:27.226336+0000","last_peered":"2024-04-23T14:31:27.226336+0000","last_clean":"2024-04-23T14:31:27.226336+0000","last_became_active":"2024-04-23T14:31:24.817260+0000","last_became_peered":"2024-04-23T14:31:24.817260+0000","last_unstale":"2024-04-23T14:31:27.226336+0000","last_undegraded":"2024-04-23T14:31:27.226336+0000","last_fullsized":"2024-04-23T14:31:27.226336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821504+0000","last_change":"2024-04-23T14:31:25.827518+0000","last_active":"2024-04-23T14:31:26.821504+0000","last_peered":"2024-04-23T14:31:26.821504+0000","last_clean":"2024-04-23T14:31:26.821504+0000","last_became_active":"2024-04-23T14:31:25.822569+0000","last_became_peered":"2024-04-23T14:31:25.822569+0000","last_unstale":"2024-04-23T14:31:26.821504+0000","last_undegraded":"2024-04-23T14:31:26.821504+0000","last_fullsized":"2024-04-23T14:31:26.821504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:58.606933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833567+0000","last_change":"2024-04-23T14:31:24.807224+0000","last_active":"2024-04-23T14:31:26.833567+0000","last_peered":"2024-04-23T14:31:26.833567+0000","last_clean":"2024-04-23T14:31:26.833567+0000","last_became_active":"2024-04-23T14:31:24.806958+0000","last_became_peered":"2024-04-23T14:31:24.806958+0000","last_unstale":"2024-04-23T14:31:26.833567+0000","last_undegraded":"2024-04-23T14:31:26.833567+0000","last_fullsized":"2024-04-23T14:31:26.833567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224355+0000","last_change":"2024-04-23T14:31:25.821318+0000","last_active":"2024-04-23T14:31:27.224355+0000","last_peered":"2024-04-23T14:31:27.224355+0000","last_clean":"2024-04-23T14:31:27.224355+0000","last_became_active":"2024-04-23T14:31:25.820930+0000","last_became_peered":"2024-04-23T14:31:25.820930+0000","last_unstale":"2024-04-23T14:31:27.224355+0000","last_undegraded":"2024-04-23T14:31:27.224355+0000","last_fullsized":"2024-04-23T14:31:27.224355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833571+0000","last_change":"2024-04-23T14:31:24.799281+0000","last_active":"2024-04-23T14:31:26.833571+0000","last_peered":"2024-04-23T14:31:26.833571+0000","last_clean":"2024-04-23T14:31:26.833571+0000","last_became_active":"2024-04-23T14:31:24.798804+0000","last_became_peered":"2024-04-23T14:31:24.798804+0000","last_unstale":"2024-04-23T14:31:26.833571+0000","last_undegraded":"2024-04-23T14:31:26.833571+0000","last_fullsized":"2024-04-23T14:31:26.833571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224405+0000","last_change":"2024-04-23T14:31:25.821631+0000","last_active":"2024-04-23T14:31:27.224405+0000","last_peered":"2024-04-23T14:31:27.224405+0000","last_clean":"2024-04-23T14:31:27.224405+0000","last_became_active":"2024-04-23T14:31:25.820439+0000","last_became_peered":"2024-04-23T14:31:25.820439+0000","last_unstale":"2024-04-23T14:31:27.224405+0000","last_undegraded":"2024-04-23T14:31:27.224405+0000","last_fullsized":"2024-04-23T14:31:27.224405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228114+0000","last_change":"2024-04-23T14:31:24.814559+0000","last_active":"2024-04-23T14:31:27.228114+0000","last_peered":"2024-04-23T14:31:27.228114+0000","last_clean":"2024-04-23T14:31:27.228114+0000","last_became_active":"2024-04-23T14:31:24.814274+0000","last_became_peered":"2024-04-23T14:31:24.814274+0000","last_unstale":"2024-04-23T14:31:27.228114+0000","last_undegraded":"2024-04-23T14:31:27.228114+0000","last_fullsized":"2024-04-23T14:31:27.228114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821011+0000","last_change":"2024-04-23T14:31:25.829871+0000","last_active":"2024-04-23T14:31:26.821011+0000","last_peered":"2024-04-23T14:31:26.821011+0000","last_clean":"2024-04-23T14:31:26.821011+0000","last_became_active":"2024-04-23T14:31:25.828620+0000","last_became_peered":"2024-04-23T14:31:25.828620+0000","last_unstale":"2024-04-23T14:31:26.821011+0000","last_undegraded":"2024-04-23T14:31:26.821011+0000","last_fullsized":"2024-04-23T14:31:26.821011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230152+0000","last_change":"2024-04-23T14:31:25.846861+0000","last_active":"2024-04-23T14:31:27.230152+0000","last_peered":"2024-04-23T14:31:27.230152+0000","last_clean":"2024-04-23T14:31:27.230152+0000","last_became_active":"2024-04-23T14:31:25.846267+0000","last_became_peered":"2024-04-23T14:31:25.846267+0000","last_unstale":"2024-04-23T14:31:27.230152+0000","last_undegraded":"2024-04-23T14:31:27.230152+0000","last_fullsized":"2024-04-23T14:31:27.230152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227598+0000","last_change":"2024-04-23T14:31:24.816507+0000","last_active":"2024-04-23T14:31:27.227598+0000","last_peered":"2024-04-23T14:31:27.227598+0000","last_clean":"2024-04-23T14:31:27.227598+0000","last_became_active":"2024-04-23T14:31:24.816159+0000","last_became_peered":"2024-04-23T14:31:24.816159+0000","last_unstale":"2024-04-23T14:31:27.227598+0000","last_undegraded":"2024-04-23T14:31:27.227598+0000","last_fullsized":"2024-04-23T14:31:27.227598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229590+0000","last_change":"2024-04-23T14:31:25.846016+0000","last_active":"2024-04-23T14:31:27.229590+0000","last_peered":"2024-04-23T14:31:27.229590+0000","last_clean":"2024-04-23T14:31:27.229590+0000","last_became_active":"2024-04-23T14:31:25.845682+0000","last_became_peered":"2024-04-23T14:31:25.845682+0000","last_unstale":"2024-04-23T14:31:27.229590+0000","last_undegraded":"2024-04-23T14:31:27.229590+0000","last_fullsized":"2024-04-23T14:31:27.229590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229170+0000","last_change":"2024-04-23T14:31:24.819529+0000","last_active":"2024-04-23T14:31:27.229170+0000","last_peered":"2024-04-23T14:31:27.229170+0000","last_clean":"2024-04-23T14:31:27.229170+0000","last_became_active":"2024-04-23T14:31:24.819162+0000","last_became_peered":"2024-04-23T14:31:24.819162+0000","last_unstale":"2024-04-23T14:31:27.229170+0000","last_undegraded":"2024-04-23T14:31:27.229170+0000","last_fullsized":"2024-04-23T14:31:27.229170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225173+0000","last_change":"2024-04-23T14:31:25.837928+0000","last_active":"2024-04-23T14:31:27.225173+0000","last_peered":"2024-04-23T14:31:27.225173+0000","last_clean":"2024-04-23T14:31:27.225173+0000","last_became_active":"2024-04-23T14:31:25.837281+0000","last_became_peered":"2024-04-23T14:31:25.837281+0000","last_unstale":"2024-04-23T14:31:27.225173+0000","last_undegraded":"2024-04-23T14:31:27.225173+0000","last_fullsized":"2024-04-23T14:31:27.225173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:59:15.348815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224998+0000","last_change":"2024-04-23T14:31:24.793060+0000","last_active":"2024-04-23T14:31:27.224998+0000","last_peered":"2024-04-23T14:31:27.224998+0000","last_clean":"2024-04-23T14:31:27.224998+0000","last_became_active":"2024-04-23T14:31:24.792763+0000","last_became_peered":"2024-04-23T14:31:24.792763+0000","last_unstale":"2024-04-23T14:31:27.224998+0000","last_undegraded":"2024-04-23T14:31:27.224998+0000","last_fullsized":"2024-04-23T14:31:27.224998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224786+0000","last_change":"2024-04-23T14:31:25.824548+0000","last_active":"2024-04-23T14:31:27.224786+0000","last_peered":"2024-04-23T14:31:27.224786+0000","last_clean":"2024-04-23T14:31:27.224786+0000","last_became_active":"2024-04-23T14:31:25.824207+0000","last_became_peered":"2024-04-23T14:31:25.824207+0000","last_unstale":"2024-04-23T14:31:27.224786+0000","last_undegraded":"2024-04-23T14:31:27.224786+0000","last_fullsized":"2024-04-23T14:31:27.224786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225623+0000","last_change":"2024-04-23T14:31:24.810861+0000","last_active":"2024-04-23T14:31:27.225623+0000","last_peered":"2024-04-23T14:31:27.225623+0000","last_clean":"2024-04-23T14:31:27.225623+0000","last_became_active":"2024-04-23T14:31:24.810466+0000","last_became_peered":"2024-04-23T14:31:24.810466+0000","last_unstale":"2024-04-23T14:31:27.225623+0000","last_undegraded":"2024-04-23T14:31:27.225623+0000","last_fullsized":"2024-04-23T14:31:27.225623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:31:36.249 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:31:36.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.495+0000 7f665f200640 1 Processor -- start 2024-04-23T14:31:36.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.495+0000 7f665f200640 1 -- start start 2024-04-23T14:31:36.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.496+0000 7f665f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f666015b450 0x7f66600b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.496+0000 7f665f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666006f3f0 0x7f66600cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.496+0000 7f665f200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66600b3870 con 0x7f66600b3080 2024-04-23T14:31:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.496+0000 7f665f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f66600b41a0 con 0x7f666015b450 2024-04-23T14:31:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.496+0000 7f665f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f66600b3b50 con 0x7f666006f3f0 2024-04-23T14:31:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f666015b450 0x7f66600b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f666015b450 0x7f66600b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33558/0 (socket says 172.21.15.59:33558) 2024-04-23T14:31:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665de00640 1 -- 172.21.15.59:0/3417942110 learned_addr learned my addr 172.21.15.59:0/3417942110 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665d400640 1 --2- 172.21.15.59:0/3417942110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666006f3f0 0x7f66600cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665de00640 1 -- 172.21.15.59:0/3417942110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666006f3f0 msgr2=0x7f66600cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:36.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665de00640 1 --2- 172.21.15.59:0/3417942110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666006f3f0 0x7f66600cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665de00640 1 -- 172.21.15.59:0/3417942110 >> v1:172.21.15.76:6789/0 conn(0x7f66600b3080 legacy=0x7f66600ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:36.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.497+0000 7f665de00640 1 -- 172.21.15.59:0/3417942110 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66600b3e30 con 0x7f666015b450 2024-04-23T14:31:36.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.498+0000 7f665de00640 1 --2- 172.21.15.59:0/3417942110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f666015b450 0x7f66600b4630 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6650009870 tx=0x7f665002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3fb6851ad38e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665ca00640 1 -- 172.21.15.59:0/3417942110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650037070 con 0x7f666015b450 2024-04-23T14:31:36.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665ca00640 1 -- 172.21.15.59:0/3417942110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650032440 con 0x7f666015b450 2024-04-23T14:31:36.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665ca00640 1 -- 172.21.15.59:0/3417942110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650032810 con 0x7f666015b450 2024-04-23T14:31:36.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 -- 172.21.15.59:0/3417942110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f666015b450 msgr2=0x7f66600b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 --2- 172.21.15.59:0/3417942110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f666015b450 0x7f66600b4630 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6650009870 tx=0x7f665002a270 comp rx=0 tx=0).stop 2024-04-23T14:31:36.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 -- 172.21.15.59:0/3417942110 shutdown_connections 2024-04-23T14:31:36.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 --2- 172.21.15.59:0/3417942110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666006f3f0 0x7f66600cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 --2- 172.21.15.59:0/3417942110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f666015b450 0x7f66600b4630 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 -- 172.21.15.59:0/3417942110 >> 172.21.15.59:0/3417942110 conn(0x7f66600b21a0 msgr2=0x7f66600b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:36.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 -- 172.21.15.59:0/3417942110 shutdown_connections 2024-04-23T14:31:36.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.499+0000 7f665f200640 1 -- 172.21.15.59:0/3417942110 wait complete. 2024-04-23T14:31:36.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.501+0000 7f665f200640 1 Processor -- start 2024-04-23T14:31:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.501+0000 7f665f200640 1 -- start start 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f666006f3f0 0x7f66600cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66600b3080 0x7f66600c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666015b450 0x7f66600c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66600b4ff0 con 0x7f66600b3080 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f666015b830 con 0x7f666006f3f0 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f66600cd590 con 0x7f666015b450 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66600b3080 0x7f66600c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66600b3080 0x7f66600c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33562/0 (socket says 172.21.15.59:33562) 2024-04-23T14:31:36.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.502+0000 7f665d400640 1 -- 172.21.15.59:0/3681793035 learned_addr learned my addr 172.21.15.59:0/3681793035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665de00640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f666006f3f0 0x7f66600cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665d400640 1 -- 172.21.15.59:0/3681793035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666015b450 msgr2=0x7f66600c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665d400640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666015b450 0x7f66600c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665d400640 1 -- 172.21.15.59:0/3681793035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f666006f3f0 msgr2=0x7f66600cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665d400640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f666006f3f0 0x7f66600cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665d400640 1 -- 172.21.15.59:0/3681793035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6660072730 con 0x7f66600b3080 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665d400640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66600b3080 0x7f66600c5a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f664c00d800 tx=0x7f664c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f6656000640 1 -- 172.21.15.59:0/3681793035 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c00b6c0 con 0x7f66600b3080 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f6656000640 1 -- 172.21.15.59:0/3681793035 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c019070 con 0x7f66600b3080 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f6656000640 1 -- 172.21.15.59:0/3681793035 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c00ba80 con 0x7f66600b3080 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66600c4fa0 con 0x7f66600b3080 2024-04-23T14:31:36.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.503+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66600c5560 con 0x7f66600b3080 2024-04-23T14:31:36.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.505+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660072a70 con 0x7f66600b3080 2024-04-23T14:31:36.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.506+0000 7f6656000640 1 -- 172.21.15.59:0/3681793035 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f664c00bc20 con 0x7f66600b3080 2024-04-23T14:31:36.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.506+0000 7f6656000640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f664006c8f0 0x7f664006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.506+0000 7f6656000640 1 -- 172.21.15.59:0/3681793035 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f664c08c040 con 0x7f66600b3080 2024-04-23T14:31:36.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.507+0000 7f665de00640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f664006c8f0 0x7f664006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.507+0000 7f665de00640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f664006c8f0 0x7f664006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6650034000 tx=0x7f665002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.511+0000 7f6656000640 1 -- 172.21.15.59:0/3681793035 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f664c01c360 con 0x7f66600b3080 2024-04-23T14:31:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.701+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f666004fd90 con 0x7f664006c8f0 2024-04-23T14:31:36.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.712+0000 7f6656000640 1 -- 172.21.15.59:0/3681793035 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f666004fd90 con 0x7f664006c8f0 2024-04-23T14:31:36.717 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:36.718 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f664006c8f0 msgr2=0x7f664006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f664006c8f0 0x7f664006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6650034000 tx=0x7f665002e000 comp rx=0 tx=0).stop 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66600b3080 msgr2=0x7f66600c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66600b3080 0x7f66600c5a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f664c00d800 tx=0x7f664c00dcc0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 shutdown_connections 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f664006c8f0 0x7f664006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f666015b450 0x7f66600c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66600b3080 0x7f66600c5a80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 --2- 172.21.15.59:0/3681793035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f666006f3f0 0x7f66600cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 >> 172.21.15.59:0/3681793035 conn(0x7f66600b21a0 msgr2=0x7f66600d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 shutdown_connections 2024-04-23T14:31:36.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.718+0000 7f665f200640 1 -- 172.21.15.59:0/3681793035 wait complete. 2024-04-23T14:31:36.736 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-23T14:31:36.347204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003226"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227768+0000","last_change":"2024-04-23T14:31:25.849551+0000","last_active":"2024-04-23T14:31:27.227768+0000","last_peered":"2024-04-23T14:31:27.227768+0000","last_clean":"2024-04-23T14:31:27.227768+0000","last_became_active":"2024-04-23T14:31:25.848756+0000","last_became_peered":"2024-04-23T14:31:25.848756+0000","last_unstale":"2024-04-23T14:31:27.227768+0000","last_undegraded":"2024-04-23T14:31:27.227768+0000","last_fullsized":"2024-04-23T14:31:27.227768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833869+0000","last_change":"2024-04-23T14:31:25.819653+0000","last_active":"2024-04-23T14:31:26.833869+0000","last_peered":"2024-04-23T14:31:26.833869+0000","last_clean":"2024-04-23T14:31:26.833869+0000","last_became_active":"2024-04-23T14:31:25.819296+0000","last_became_peered":"2024-04-23T14:31:25.819296+0000","last_unstale":"2024-04-23T14:31:26.833869+0000","last_undegraded":"2024-04-23T14:31:26.833869+0000","last_fullsized":"2024-04-23T14:31:26.833869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225689+0000","last_change":"2024-04-23T14:31:25.818736+0000","last_active":"2024-04-23T14:31:27.225689+0000","last_peered":"2024-04-23T14:31:27.225689+0000","last_clean":"2024-04-23T14:31:27.225689+0000","last_became_active":"2024-04-23T14:31:25.816583+0000","last_became_peered":"2024-04-23T14:31:25.816583+0000","last_unstale":"2024-04-23T14:31:27.225689+0000","last_undegraded":"2024-04-23T14:31:27.225689+0000","last_fullsized":"2024-04-23T14:31:27.225689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229281+0000","last_change":"2024-04-23T14:31:25.836663+0000","last_active":"2024-04-23T14:31:27.229281+0000","last_peered":"2024-04-23T14:31:27.229281+0000","last_clean":"2024-04-23T14:31:27.229281+0000","last_became_active":"2024-04-23T14:31:25.836357+0000","last_became_peered":"2024-04-23T14:31:25.836357+0000","last_unstale":"2024-04-23T14:31:27.229281+0000","last_undegraded":"2024-04-23T14:31:27.229281+0000","last_fullsized":"2024-04-23T14:31:27.229281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227740+0000","last_change":"2024-04-23T14:31:25.830442+0000","last_active":"2024-04-23T14:31:27.227740+0000","last_peered":"2024-04-23T14:31:27.227740+0000","last_clean":"2024-04-23T14:31:27.227740+0000","last_became_active":"2024-04-23T14:31:25.830032+0000","last_became_peered":"2024-04-23T14:31:25.830032+0000","last_unstale":"2024-04-23T14:31:27.227740+0000","last_undegraded":"2024-04-23T14:31:27.227740+0000","last_fullsized":"2024-04-23T14:31:27.227740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228994+0000","last_change":"2024-04-23T14:31:25.835030+0000","last_active":"2024-04-23T14:31:27.228994+0000","last_peered":"2024-04-23T14:31:27.228994+0000","last_clean":"2024-04-23T14:31:27.228994+0000","last_became_active":"2024-04-23T14:31:25.834764+0000","last_became_peered":"2024-04-23T14:31:25.834764+0000","last_unstale":"2024-04-23T14:31:27.228994+0000","last_undegraded":"2024-04-23T14:31:27.228994+0000","last_fullsized":"2024-04-23T14:31:27.228994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226818+0000","last_change":"2024-04-23T14:31:25.833373+0000","last_active":"2024-04-23T14:31:27.226818+0000","last_peered":"2024-04-23T14:31:27.226818+0000","last_clean":"2024-04-23T14:31:27.226818+0000","last_became_active":"2024-04-23T14:31:25.831555+0000","last_became_peered":"2024-04-23T14:31:25.831555+0000","last_unstale":"2024-04-23T14:31:27.226818+0000","last_undegraded":"2024-04-23T14:31:27.226818+0000","last_fullsized":"2024-04-23T14:31:27.226818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224117+0000","last_change":"2024-04-23T14:31:25.828675+0000","last_active":"2024-04-23T14:31:27.224117+0000","last_peered":"2024-04-23T14:31:27.224117+0000","last_clean":"2024-04-23T14:31:27.224117+0000","last_became_active":"2024-04-23T14:31:25.828381+0000","last_became_peered":"2024-04-23T14:31:25.828381+0000","last_unstale":"2024-04-23T14:31:27.224117+0000","last_undegraded":"2024-04-23T14:31:27.224117+0000","last_fullsized":"2024-04-23T14:31:27.224117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228299+0000","last_change":"2024-04-23T14:31:25.839705+0000","last_active":"2024-04-23T14:31:27.228299+0000","last_peered":"2024-04-23T14:31:27.228299+0000","last_clean":"2024-04-23T14:31:27.228299+0000","last_became_active":"2024-04-23T14:31:25.839432+0000","last_became_peered":"2024-04-23T14:31:25.839432+0000","last_unstale":"2024-04-23T14:31:27.228299+0000","last_undegraded":"2024-04-23T14:31:27.228299+0000","last_fullsized":"2024-04-23T14:31:27.228299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821382+0000","last_change":"2024-04-23T14:31:25.833382+0000","last_active":"2024-04-23T14:31:26.821382+0000","last_peered":"2024-04-23T14:31:26.821382+0000","last_clean":"2024-04-23T14:31:26.821382+0000","last_became_active":"2024-04-23T14:31:25.833010+0000","last_became_peered":"2024-04-23T14:31:25.833010+0000","last_unstale":"2024-04-23T14:31:26.821382+0000","last_undegraded":"2024-04-23T14:31:26.821382+0000","last_fullsized":"2024-04-23T14:31:26.821382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820889+0000","last_change":"2024-04-23T14:31:25.828614+0000","last_active":"2024-04-23T14:31:26.820889+0000","last_peered":"2024-04-23T14:31:26.820889+0000","last_clean":"2024-04-23T14:31:26.820889+0000","last_became_active":"2024-04-23T14:31:25.828342+0000","last_became_peered":"2024-04-23T14:31:25.828342+0000","last_unstale":"2024-04-23T14:31:26.820889+0000","last_undegraded":"2024-04-23T14:31:26.820889+0000","last_fullsized":"2024-04-23T14:31:26.820889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821732+0000","last_change":"2024-04-23T14:31:25.825396+0000","last_active":"2024-04-23T14:31:26.821732+0000","last_peered":"2024-04-23T14:31:26.821732+0000","last_clean":"2024-04-23T14:31:26.821732+0000","last_became_active":"2024-04-23T14:31:25.822425+0000","last_became_peered":"2024-04-23T14:31:25.822425+0000","last_unstale":"2024-04-23T14:31:26.821732+0000","last_undegraded":"2024-04-23T14:31:26.821732+0000","last_fullsized":"2024-04-23T14:31:26.821732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:05.846065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223667+0000","last_change":"2024-04-23T14:31:25.823390+0000","last_active":"2024-04-23T14:31:27.223667+0000","last_peered":"2024-04-23T14:31:27.223667+0000","last_clean":"2024-04-23T14:31:27.223667+0000","last_became_active":"2024-04-23T14:31:25.823054+0000","last_became_peered":"2024-04-23T14:31:25.823054+0000","last_unstale":"2024-04-23T14:31:27.223667+0000","last_undegraded":"2024-04-23T14:31:27.223667+0000","last_fullsized":"2024-04-23T14:31:27.223667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225120+0000","last_change":"2024-04-23T14:31:25.818198+0000","last_active":"2024-04-23T14:31:27.225120+0000","last_peered":"2024-04-23T14:31:27.225120+0000","last_clean":"2024-04-23T14:31:27.225120+0000","last_became_active":"2024-04-23T14:31:25.817873+0000","last_became_peered":"2024-04-23T14:31:25.817873+0000","last_unstale":"2024-04-23T14:31:27.225120+0000","last_undegraded":"2024-04-23T14:31:27.225120+0000","last_fullsized":"2024-04-23T14:31:27.225120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:08:54.362020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833481+0000","last_change":"2024-04-23T14:31:25.829886+0000","last_active":"2024-04-23T14:31:26.833481+0000","last_peered":"2024-04-23T14:31:26.833481+0000","last_clean":"2024-04-23T14:31:26.833481+0000","last_became_active":"2024-04-23T14:31:25.829086+0000","last_became_peered":"2024-04-23T14:31:25.829086+0000","last_unstale":"2024-04-23T14:31:26.833481+0000","last_undegraded":"2024-04-23T14:31:26.833481+0000","last_fullsized":"2024-04-23T14:31:26.833481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228453+0000","last_change":"2024-04-23T14:31:25.818775+0000","last_active":"2024-04-23T14:31:27.228453+0000","last_peered":"2024-04-23T14:31:27.228453+0000","last_clean":"2024-04-23T14:31:27.228453+0000","last_became_active":"2024-04-23T14:31:25.818318+0000","last_became_peered":"2024-04-23T14:31:25.818318+0000","last_unstale":"2024-04-23T14:31:27.228453+0000","last_undegraded":"2024-04-23T14:31:27.228453+0000","last_fullsized":"2024-04-23T14:31:27.228453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230731+0000","last_change":"2024-04-23T14:31:25.850111+0000","last_active":"2024-04-23T14:31:27.230731+0000","last_peered":"2024-04-23T14:31:27.230731+0000","last_clean":"2024-04-23T14:31:27.230731+0000","last_became_active":"2024-04-23T14:31:25.848968+0000","last_became_peered":"2024-04-23T14:31:25.848968+0000","last_unstale":"2024-04-23T14:31:27.230731+0000","last_undegraded":"2024-04-23T14:31:27.230731+0000","last_fullsized":"2024-04-23T14:31:27.230731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.418617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.815057+0000","last_change":"2024-04-23T14:31:25.813711+0000","last_active":"2024-04-23T14:31:26.815057+0000","last_peered":"2024-04-23T14:31:26.815057+0000","last_clean":"2024-04-23T14:31:26.815057+0000","last_became_active":"2024-04-23T14:31:25.813377+0000","last_became_peered":"2024-04-23T14:31:25.813377+0000","last_unstale":"2024-04-23T14:31:26.815057+0000","last_undegraded":"2024-04-23T14:31:26.815057+0000","last_fullsized":"2024-04-23T14:31:26.815057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230045+0000","last_change":"2024-04-23T14:31:25.842518+0000","last_active":"2024-04-23T14:31:27.230045+0000","last_peered":"2024-04-23T14:31:27.230045+0000","last_clean":"2024-04-23T14:31:27.230045+0000","last_became_active":"2024-04-23T14:31:25.842265+0000","last_became_peered":"2024-04-23T14:31:25.842265+0000","last_unstale":"2024-04-23T14:31:27.230045+0000","last_undegraded":"2024-04-23T14:31:27.230045+0000","last_fullsized":"2024-04-23T14:31:27.230045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:10:49.364737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820245+0000","last_change":"2024-04-23T14:31:25.824601+0000","last_active":"2024-04-23T14:31:26.820245+0000","last_peered":"2024-04-23T14:31:26.820245+0000","last_clean":"2024-04-23T14:31:26.820245+0000","last_became_active":"2024-04-23T14:31:25.824326+0000","last_became_peered":"2024-04-23T14:31:25.824326+0000","last_unstale":"2024-04-23T14:31:26.820245+0000","last_undegraded":"2024-04-23T14:31:26.820245+0000","last_fullsized":"2024-04-23T14:31:26.820245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225202+0000","last_change":"2024-04-23T14:31:25.824382+0000","last_active":"2024-04-23T14:31:27.225202+0000","last_peered":"2024-04-23T14:31:27.225202+0000","last_clean":"2024-04-23T14:31:27.225202+0000","last_became_active":"2024-04-23T14:31:25.823991+0000","last_became_peered":"2024-04-23T14:31:25.823991+0000","last_unstale":"2024-04-23T14:31:27.225202+0000","last_undegraded":"2024-04-23T14:31:27.225202+0000","last_fullsized":"2024-04-23T14:31:27.225202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229105+0000","last_change":"2024-04-23T14:31:25.842389+0000","last_active":"2024-04-23T14:31:27.229105+0000","last_peered":"2024-04-23T14:31:27.229105+0000","last_clean":"2024-04-23T14:31:27.229105+0000","last_became_active":"2024-04-23T14:31:25.841584+0000","last_became_peered":"2024-04-23T14:31:25.841584+0000","last_unstale":"2024-04-23T14:31:27.229105+0000","last_undegraded":"2024-04-23T14:31:27.229105+0000","last_fullsized":"2024-04-23T14:31:27.229105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223827+0000","last_change":"2024-04-23T14:31:25.830515+0000","last_active":"2024-04-23T14:31:27.223827+0000","last_peered":"2024-04-23T14:31:27.223827+0000","last_clean":"2024-04-23T14:31:27.223827+0000","last_became_active":"2024-04-23T14:31:25.830209+0000","last_became_peered":"2024-04-23T14:31:25.830209+0000","last_unstale":"2024-04-23T14:31:27.223827+0000","last_undegraded":"2024-04-23T14:31:27.223827+0000","last_fullsized":"2024-04-23T14:31:27.223827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228540+0000","last_change":"2024-04-23T14:31:25.824411+0000","last_active":"2024-04-23T14:31:27.228540+0000","last_peered":"2024-04-23T14:31:27.228540+0000","last_clean":"2024-04-23T14:31:27.228540+0000","last_became_active":"2024-04-23T14:31:25.824052+0000","last_became_peered":"2024-04-23T14:31:25.824052+0000","last_unstale":"2024-04-23T14:31:27.228540+0000","last_undegraded":"2024-04-23T14:31:27.228540+0000","last_fullsized":"2024-04-23T14:31:27.228540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224322+0000","last_change":"2024-04-23T14:31:25.830939+0000","last_active":"2024-04-23T14:31:27.224322+0000","last_peered":"2024-04-23T14:31:27.224322+0000","last_clean":"2024-04-23T14:31:27.224322+0000","last_became_active":"2024-04-23T14:31:25.830632+0000","last_became_peered":"2024-04-23T14:31:25.830632+0000","last_unstale":"2024-04-23T14:31:27.224322+0000","last_undegraded":"2024-04-23T14:31:27.224322+0000","last_fullsized":"2024-04-23T14:31:27.224322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821455+0000","last_change":"2024-04-23T14:31:25.832634+0000","last_active":"2024-04-23T14:31:26.821455+0000","last_peered":"2024-04-23T14:31:26.821455+0000","last_clean":"2024-04-23T14:31:26.821455+0000","last_became_active":"2024-04-23T14:31:25.832329+0000","last_became_peered":"2024-04-23T14:31:25.832329+0000","last_unstale":"2024-04-23T14:31:26.821455+0000","last_undegraded":"2024-04-23T14:31:26.821455+0000","last_fullsized":"2024-04-23T14:31:26.821455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833312+0000","last_change":"2024-04-23T14:31:25.822589+0000","last_active":"2024-04-23T14:31:26.833312+0000","last_peered":"2024-04-23T14:31:26.833312+0000","last_clean":"2024-04-23T14:31:26.833312+0000","last_became_active":"2024-04-23T14:31:25.822061+0000","last_became_peered":"2024-04-23T14:31:25.822061+0000","last_unstale":"2024-04-23T14:31:26.833312+0000","last_undegraded":"2024-04-23T14:31:26.833312+0000","last_fullsized":"2024-04-23T14:31:26.833312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223980+0000","last_change":"2024-04-23T14:31:25.832096+0000","last_active":"2024-04-23T14:31:27.223980+0000","last_peered":"2024-04-23T14:31:27.223980+0000","last_clean":"2024-04-23T14:31:27.223980+0000","last_became_active":"2024-04-23T14:31:25.829333+0000","last_became_peered":"2024-04-23T14:31:25.829333+0000","last_unstale":"2024-04-23T14:31:27.223980+0000","last_undegraded":"2024-04-23T14:31:27.223980+0000","last_fullsized":"2024-04-23T14:31:27.223980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225363+0000","last_change":"2024-04-23T14:31:25.826410+0000","last_active":"2024-04-23T14:31:27.225363+0000","last_peered":"2024-04-23T14:31:27.225363+0000","last_clean":"2024-04-23T14:31:27.225363+0000","last_became_active":"2024-04-23T14:31:25.826088+0000","last_became_peered":"2024-04-23T14:31:25.826088+0000","last_unstale":"2024-04-23T14:31:27.225363+0000","last_undegraded":"2024-04-23T14:31:27.225363+0000","last_fullsized":"2024-04-23T14:31:27.225363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:38:19.636274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229439+0000","last_change":"2024-04-23T14:31:25.832202+0000","last_active":"2024-04-23T14:31:27.229439+0000","last_peered":"2024-04-23T14:31:27.229439+0000","last_clean":"2024-04-23T14:31:27.229439+0000","last_became_active":"2024-04-23T14:31:25.831719+0000","last_became_peered":"2024-04-23T14:31:25.831719+0000","last_unstale":"2024-04-23T14:31:27.229439+0000","last_undegraded":"2024-04-23T14:31:27.229439+0000","last_fullsized":"2024-04-23T14:31:27.229439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814413+0000","last_change":"2024-04-23T14:31:25.827667+0000","last_active":"2024-04-23T14:31:26.814413+0000","last_peered":"2024-04-23T14:31:26.814413+0000","last_clean":"2024-04-23T14:31:26.814413+0000","last_became_active":"2024-04-23T14:31:25.827302+0000","last_became_peered":"2024-04-23T14:31:25.827302+0000","last_unstale":"2024-04-23T14:31:26.814413+0000","last_undegraded":"2024-04-23T14:31:26.814413+0000","last_fullsized":"2024-04-23T14:31:26.814413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.819912+0000","last_change":"2024-04-23T14:31:25.817837+0000","last_active":"2024-04-23T14:31:26.819912+0000","last_peered":"2024-04-23T14:31:26.819912+0000","last_clean":"2024-04-23T14:31:26.819912+0000","last_became_active":"2024-04-23T14:31:25.817156+0000","last_became_peered":"2024-04-23T14:31:25.817156+0000","last_unstale":"2024-04-23T14:31:26.819912+0000","last_undegraded":"2024-04-23T14:31:26.819912+0000","last_fullsized":"2024-04-23T14:31:26.819912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:17.572256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820171+0000","last_change":"2024-04-23T14:31:25.821505+0000","last_active":"2024-04-23T14:31:26.820171+0000","last_peered":"2024-04-23T14:31:26.820171+0000","last_clean":"2024-04-23T14:31:26.820171+0000","last_became_active":"2024-04-23T14:31:25.821186+0000","last_became_peered":"2024-04-23T14:31:25.821186+0000","last_unstale":"2024-04-23T14:31:26.820171+0000","last_undegraded":"2024-04-23T14:31:26.820171+0000","last_fullsized":"2024-04-23T14:31:26.820171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:30.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224053+0000","last_change":"2024-04-23T14:31:24.810696+0000","last_active":"2024-04-23T14:31:27.224053+0000","last_peered":"2024-04-23T14:31:27.224053+0000","last_clean":"2024-04-23T14:31:27.224053+0000","last_became_active":"2024-04-23T14:31:24.810349+0000","last_became_peered":"2024-04-23T14:31:24.810349+0000","last_unstale":"2024-04-23T14:31:27.224053+0000","last_undegraded":"2024-04-23T14:31:27.224053+0000","last_fullsized":"2024-04-23T14:31:27.224053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225550+0000","last_change":"2024-04-23T14:31:25.824408+0000","last_active":"2024-04-23T14:31:27.225550+0000","last_peered":"2024-04-23T14:31:27.225550+0000","last_clean":"2024-04-23T14:31:27.225550+0000","last_became_active":"2024-04-23T14:31:25.822620+0000","last_became_peered":"2024-04-23T14:31:25.822620+0000","last_unstale":"2024-04-23T14:31:27.225550+0000","last_undegraded":"2024-04-23T14:31:27.225550+0000","last_fullsized":"2024-04-23T14:31:27.225550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820803+0000","last_change":"2024-04-23T14:31:24.801642+0000","last_active":"2024-04-23T14:31:26.820803+0000","last_peered":"2024-04-23T14:31:26.820803+0000","last_clean":"2024-04-23T14:31:26.820803+0000","last_became_active":"2024-04-23T14:31:24.801244+0000","last_became_peered":"2024-04-23T14:31:24.801244+0000","last_unstale":"2024-04-23T14:31:26.820803+0000","last_undegraded":"2024-04-23T14:31:26.820803+0000","last_fullsized":"2024-04-23T14:31:26.820803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223904+0000","last_change":"2024-04-23T14:31:24.796017+0000","last_active":"2024-04-23T14:31:27.223904+0000","last_peered":"2024-04-23T14:31:27.223904+0000","last_clean":"2024-04-23T14:31:27.223904+0000","last_became_active":"2024-04-23T14:31:24.795734+0000","last_became_peered":"2024-04-23T14:31:24.795734+0000","last_unstale":"2024-04-23T14:31:27.223904+0000","last_undegraded":"2024-04-23T14:31:27.223904+0000","last_fullsized":"2024-04-23T14:31:27.223904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.223975+0000","last_change":"2024-04-23T14:31:25.819835+0000","last_active":"2024-04-23T14:31:27.223975+0000","last_peered":"2024-04-23T14:31:27.223975+0000","last_clean":"2024-04-23T14:31:27.223975+0000","last_became_active":"2024-04-23T14:31:25.819532+0000","last_became_peered":"2024-04-23T14:31:25.819532+0000","last_unstale":"2024-04-23T14:31:27.223975+0000","last_undegraded":"2024-04-23T14:31:27.223975+0000","last_fullsized":"2024-04-23T14:31:27.223975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228610+0000","last_change":"2024-04-23T14:31:24.802566+0000","last_active":"2024-04-23T14:31:27.228610+0000","last_peered":"2024-04-23T14:31:27.228610+0000","last_clean":"2024-04-23T14:31:27.228610+0000","last_became_active":"2024-04-23T14:31:24.802299+0000","last_became_peered":"2024-04-23T14:31:24.802299+0000","last_unstale":"2024-04-23T14:31:27.228610+0000","last_undegraded":"2024-04-23T14:31:27.228610+0000","last_fullsized":"2024-04-23T14:31:27.228610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833252+0000","last_change":"2024-04-23T14:31:25.822370+0000","last_active":"2024-04-23T14:31:26.833252+0000","last_peered":"2024-04-23T14:31:26.833252+0000","last_clean":"2024-04-23T14:31:26.833252+0000","last_became_active":"2024-04-23T14:31:25.822039+0000","last_became_peered":"2024-04-23T14:31:25.822039+0000","last_unstale":"2024-04-23T14:31:26.833252+0000","last_undegraded":"2024-04-23T14:31:26.833252+0000","last_fullsized":"2024-04-23T14:31:26.833252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821576+0000","last_change":"2024-04-23T14:31:24.806864+0000","last_active":"2024-04-23T14:31:26.821576+0000","last_peered":"2024-04-23T14:31:26.821576+0000","last_clean":"2024-04-23T14:31:26.821576+0000","last_became_active":"2024-04-23T14:31:24.806597+0000","last_became_peered":"2024-04-23T14:31:24.806597+0000","last_unstale":"2024-04-23T14:31:26.821576+0000","last_undegraded":"2024-04-23T14:31:26.821576+0000","last_fullsized":"2024-04-23T14:31:26.821576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230291+0000","last_change":"2024-04-23T14:31:25.842918+0000","last_active":"2024-04-23T14:31:27.230291+0000","last_peered":"2024-04-23T14:31:27.230291+0000","last_clean":"2024-04-23T14:31:27.230291+0000","last_became_active":"2024-04-23T14:31:25.842581+0000","last_became_peered":"2024-04-23T14:31:25.842581+0000","last_unstale":"2024-04-23T14:31:27.230291+0000","last_undegraded":"2024-04-23T14:31:27.230291+0000","last_fullsized":"2024-04-23T14:31:27.230291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229964+0000","last_change":"2024-04-23T14:31:24.805711+0000","last_active":"2024-04-23T14:31:27.229964+0000","last_peered":"2024-04-23T14:31:27.229964+0000","last_clean":"2024-04-23T14:31:27.229964+0000","last_became_active":"2024-04-23T14:31:24.805351+0000","last_became_peered":"2024-04-23T14:31:24.805351+0000","last_unstale":"2024-04-23T14:31:27.229964+0000","last_undegraded":"2024-04-23T14:31:27.229964+0000","last_fullsized":"2024-04-23T14:31:27.229964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225997+0000","last_change":"2024-04-23T14:31:25.837626+0000","last_active":"2024-04-23T14:31:27.225997+0000","last_peered":"2024-04-23T14:31:27.225997+0000","last_clean":"2024-04-23T14:31:27.225997+0000","last_became_active":"2024-04-23T14:31:25.836840+0000","last_became_peered":"2024-04-23T14:31:25.836840+0000","last_unstale":"2024-04-23T14:31:27.225997+0000","last_undegraded":"2024-04-23T14:31:27.225997+0000","last_fullsized":"2024-04-23T14:31:27.225997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:35.528943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833398+0000","last_change":"2024-04-23T14:31:24.802741+0000","last_active":"2024-04-23T14:31:26.833398+0000","last_peered":"2024-04-23T14:31:26.833398+0000","last_clean":"2024-04-23T14:31:26.833398+0000","last_became_active":"2024-04-23T14:31:24.802462+0000","last_became_peered":"2024-04-23T14:31:24.802462+0000","last_unstale":"2024-04-23T14:31:26.833398+0000","last_undegraded":"2024-04-23T14:31:26.833398+0000","last_fullsized":"2024-04-23T14:31:26.833398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821929+0000","last_change":"2024-04-23T14:31:25.840470+0000","last_active":"2024-04-23T14:31:26.821929+0000","last_peered":"2024-04-23T14:31:26.821929+0000","last_clean":"2024-04-23T14:31:26.821929+0000","last_became_active":"2024-04-23T14:31:25.840160+0000","last_became_peered":"2024-04-23T14:31:25.840160+0000","last_unstale":"2024-04-23T14:31:26.821929+0000","last_undegraded":"2024-04-23T14:31:26.821929+0000","last_fullsized":"2024-04-23T14:31:26.821929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:32.378417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820475+0000","last_change":"2024-04-23T14:31:24.798634+0000","last_active":"2024-04-23T14:31:26.820475+0000","last_peered":"2024-04-23T14:31:26.820475+0000","last_clean":"2024-04-23T14:31:26.820475+0000","last_became_active":"2024-04-23T14:31:24.798307+0000","last_became_peered":"2024-04-23T14:31:24.798307+0000","last_unstale":"2024-04-23T14:31:26.820475+0000","last_undegraded":"2024-04-23T14:31:26.820475+0000","last_fullsized":"2024-04-23T14:31:26.820475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227416+0000","last_change":"2024-04-23T14:31:25.830485+0000","last_active":"2024-04-23T14:31:27.227416+0000","last_peered":"2024-04-23T14:31:27.227416+0000","last_clean":"2024-04-23T14:31:27.227416+0000","last_became_active":"2024-04-23T14:31:25.830001+0000","last_became_peered":"2024-04-23T14:31:25.830001+0000","last_unstale":"2024-04-23T14:31:27.227416+0000","last_undegraded":"2024-04-23T14:31:27.227416+0000","last_fullsized":"2024-04-23T14:31:27.227416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225604+0000","last_change":"2024-04-23T14:31:24.800143+0000","last_active":"2024-04-23T14:31:27.225604+0000","last_peered":"2024-04-23T14:31:27.225604+0000","last_clean":"2024-04-23T14:31:27.225604+0000","last_became_active":"2024-04-23T14:31:24.799828+0000","last_became_peered":"2024-04-23T14:31:24.799828+0000","last_unstale":"2024-04-23T14:31:27.225604+0000","last_undegraded":"2024-04-23T14:31:27.225604+0000","last_fullsized":"2024-04-23T14:31:27.225604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225544+0000","last_change":"2024-04-23T14:31:25.839070+0000","last_active":"2024-04-23T14:31:27.225544+0000","last_peered":"2024-04-23T14:31:27.225544+0000","last_clean":"2024-04-23T14:31:27.225544+0000","last_became_active":"2024-04-23T14:31:25.838724+0000","last_became_peered":"2024-04-23T14:31:25.838724+0000","last_unstale":"2024-04-23T14:31:27.225544+0000","last_undegraded":"2024-04-23T14:31:27.225544+0000","last_fullsized":"2024-04-23T14:31:27.225544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229381+0000","last_change":"2024-04-23T14:31:24.805607+0000","last_active":"2024-04-23T14:31:27.229381+0000","last_peered":"2024-04-23T14:31:27.229381+0000","last_clean":"2024-04-23T14:31:27.229381+0000","last_became_active":"2024-04-23T14:31:24.805268+0000","last_became_peered":"2024-04-23T14:31:24.805268+0000","last_unstale":"2024-04-23T14:31:27.229381+0000","last_undegraded":"2024-04-23T14:31:27.229381+0000","last_fullsized":"2024-04-23T14:31:27.229381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228113+0000","last_change":"2024-04-23T14:31:25.835264+0000","last_active":"2024-04-23T14:31:27.228113+0000","last_peered":"2024-04-23T14:31:27.228113+0000","last_clean":"2024-04-23T14:31:27.228113+0000","last_became_active":"2024-04-23T14:31:25.834845+0000","last_became_peered":"2024-04-23T14:31:25.834845+0000","last_unstale":"2024-04-23T14:31:27.228113+0000","last_undegraded":"2024-04-23T14:31:27.228113+0000","last_fullsized":"2024-04-23T14:31:27.228113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:16.926073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820500+0000","last_change":"2024-04-23T14:31:24.803064+0000","last_active":"2024-04-23T14:31:26.820500+0000","last_peered":"2024-04-23T14:31:26.820500+0000","last_clean":"2024-04-23T14:31:26.820500+0000","last_became_active":"2024-04-23T14:31:24.801163+0000","last_became_peered":"2024-04-23T14:31:24.801163+0000","last_unstale":"2024-04-23T14:31:26.820500+0000","last_undegraded":"2024-04-23T14:31:26.820500+0000","last_fullsized":"2024-04-23T14:31:26.820500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820307+0000","last_change":"2024-04-23T14:31:25.818141+0000","last_active":"2024-04-23T14:31:26.820307+0000","last_peered":"2024-04-23T14:31:26.820307+0000","last_clean":"2024-04-23T14:31:26.820307+0000","last_became_active":"2024-04-23T14:31:25.817558+0000","last_became_peered":"2024-04-23T14:31:25.817558+0000","last_unstale":"2024-04-23T14:31:26.820307+0000","last_undegraded":"2024-04-23T14:31:26.820307+0000","last_fullsized":"2024-04-23T14:31:26.820307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814267+0000","last_change":"2024-04-23T14:31:24.792560+0000","last_active":"2024-04-23T14:31:26.814267+0000","last_peered":"2024-04-23T14:31:26.814267+0000","last_clean":"2024-04-23T14:31:26.814267+0000","last_became_active":"2024-04-23T14:31:24.792055+0000","last_became_peered":"2024-04-23T14:31:24.792055+0000","last_unstale":"2024-04-23T14:31:26.814267+0000","last_undegraded":"2024-04-23T14:31:26.814267+0000","last_fullsized":"2024-04-23T14:31:26.814267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226832+0000","last_change":"2024-04-23T14:31:25.833895+0000","last_active":"2024-04-23T14:31:27.226832+0000","last_peered":"2024-04-23T14:31:27.226832+0000","last_clean":"2024-04-23T14:31:27.226832+0000","last_became_active":"2024-04-23T14:31:25.833570+0000","last_became_peered":"2024-04-23T14:31:25.833570+0000","last_unstale":"2024-04-23T14:31:27.226832+0000","last_undegraded":"2024-04-23T14:31:27.226832+0000","last_fullsized":"2024-04-23T14:31:27.226832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.820916+0000","last_change":"2024-04-23T14:31:24.794838+0000","last_active":"2024-04-23T14:31:26.820916+0000","last_peered":"2024-04-23T14:31:26.820916+0000","last_clean":"2024-04-23T14:31:26.820916+0000","last_became_active":"2024-04-23T14:31:24.794545+0000","last_became_peered":"2024-04-23T14:31:24.794545+0000","last_unstale":"2024-04-23T14:31:26.820916+0000","last_undegraded":"2024-04-23T14:31:26.820916+0000","last_fullsized":"2024-04-23T14:31:26.820916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:47.137471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228074+0000","last_change":"2024-04-23T14:31:25.850256+0000","last_active":"2024-04-23T14:31:27.228074+0000","last_peered":"2024-04-23T14:31:27.228074+0000","last_clean":"2024-04-23T14:31:27.228074+0000","last_became_active":"2024-04-23T14:31:25.849503+0000","last_became_peered":"2024-04-23T14:31:25.849503+0000","last_unstale":"2024-04-23T14:31:27.228074+0000","last_undegraded":"2024-04-23T14:31:27.228074+0000","last_fullsized":"2024-04-23T14:31:27.228074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814819+0000","last_change":"2024-04-23T14:31:24.798518+0000","last_active":"2024-04-23T14:31:26.814819+0000","last_peered":"2024-04-23T14:31:26.814819+0000","last_clean":"2024-04-23T14:31:26.814819+0000","last_became_active":"2024-04-23T14:31:24.798170+0000","last_became_peered":"2024-04-23T14:31:24.798170+0000","last_unstale":"2024-04-23T14:31:26.814819+0000","last_undegraded":"2024-04-23T14:31:26.814819+0000","last_fullsized":"2024-04-23T14:31:26.814819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:32.212080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228158+0000","last_change":"2024-04-23T14:31:25.849745+0000","last_active":"2024-04-23T14:31:27.228158+0000","last_peered":"2024-04-23T14:31:27.228158+0000","last_clean":"2024-04-23T14:31:27.228158+0000","last_became_active":"2024-04-23T14:31:25.849311+0000","last_became_peered":"2024-04-23T14:31:25.849311+0000","last_unstale":"2024-04-23T14:31:27.228158+0000","last_undegraded":"2024-04-23T14:31:27.228158+0000","last_fullsized":"2024-04-23T14:31:27.228158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833547+0000","last_change":"2024-04-23T14:31:24.801624+0000","last_active":"2024-04-23T14:31:26.833547+0000","last_peered":"2024-04-23T14:31:26.833547+0000","last_clean":"2024-04-23T14:31:26.833547+0000","last_became_active":"2024-04-23T14:31:24.800838+0000","last_became_peered":"2024-04-23T14:31:24.800838+0000","last_unstale":"2024-04-23T14:31:26.833547+0000","last_undegraded":"2024-04-23T14:31:26.833547+0000","last_fullsized":"2024-04-23T14:31:26.833547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229691+0000","last_change":"2024-04-23T14:31:25.836837+0000","last_active":"2024-04-23T14:31:27.229691+0000","last_peered":"2024-04-23T14:31:27.229691+0000","last_clean":"2024-04-23T14:31:27.229691+0000","last_became_active":"2024-04-23T14:31:25.836473+0000","last_became_peered":"2024-04-23T14:31:25.836473+0000","last_unstale":"2024-04-23T14:31:27.229691+0000","last_undegraded":"2024-04-23T14:31:27.229691+0000","last_fullsized":"2024-04-23T14:31:27.229691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:17.503601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814648+0000","last_change":"2024-04-23T14:31:24.801079+0000","last_active":"2024-04-23T14:31:26.814648+0000","last_peered":"2024-04-23T14:31:26.814648+0000","last_clean":"2024-04-23T14:31:26.814648+0000","last_became_active":"2024-04-23T14:31:24.800766+0000","last_became_peered":"2024-04-23T14:31:24.800766+0000","last_unstale":"2024-04-23T14:31:26.814648+0000","last_undegraded":"2024-04-23T14:31:26.814648+0000","last_fullsized":"2024-04-23T14:31:26.814648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229022+0000","last_change":"2024-04-23T14:31:25.842503+0000","last_active":"2024-04-23T14:31:27.229022+0000","last_peered":"2024-04-23T14:31:27.229022+0000","last_clean":"2024-04-23T14:31:27.229022+0000","last_became_active":"2024-04-23T14:31:25.842174+0000","last_became_peered":"2024-04-23T14:31:25.842174+0000","last_unstale":"2024-04-23T14:31:27.229022+0000","last_undegraded":"2024-04-23T14:31:27.229022+0000","last_fullsized":"2024-04-23T14:31:27.229022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:24:27.716211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226647+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:31:27.226647+0000","last_peered":"2024-04-23T14:31:27.226647+0000","last_clean":"2024-04-23T14:31:27.226647+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:31:27.226647+0000","last_undegraded":"2024-04-23T14:31:27.226647+0000","last_fullsized":"2024-04-23T14:31:27.226647+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228944+0000","last_change":"2024-04-23T14:31:24.803500+0000","last_active":"2024-04-23T14:31:27.228944+0000","last_peered":"2024-04-23T14:31:27.228944+0000","last_clean":"2024-04-23T14:31:27.228944+0000","last_became_active":"2024-04-23T14:31:24.803188+0000","last_became_peered":"2024-04-23T14:31:24.803188+0000","last_unstale":"2024-04-23T14:31:27.228944+0000","last_undegraded":"2024-04-23T14:31:27.228944+0000","last_fullsized":"2024-04-23T14:31:27.228944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:40.010846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224272+0000","last_change":"2024-04-23T14:31:25.842991+0000","last_active":"2024-04-23T14:31:27.224272+0000","last_peered":"2024-04-23T14:31:27.224272+0000","last_clean":"2024-04-23T14:31:27.224272+0000","last_became_active":"2024-04-23T14:31:25.842681+0000","last_became_peered":"2024-04-23T14:31:25.842681+0000","last_unstale":"2024-04-23T14:31:27.224272+0000","last_undegraded":"2024-04-23T14:31:27.224272+0000","last_fullsized":"2024-04-23T14:31:27.224272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226837+0000","last_change":"2024-04-23T14:31:24.815103+0000","last_active":"2024-04-23T14:31:27.226837+0000","last_peered":"2024-04-23T14:31:27.226837+0000","last_clean":"2024-04-23T14:31:27.226837+0000","last_became_active":"2024-04-23T14:31:24.814780+0000","last_became_peered":"2024-04-23T14:31:24.814780+0000","last_unstale":"2024-04-23T14:31:27.226837+0000","last_undegraded":"2024-04-23T14:31:27.226837+0000","last_fullsized":"2024-04-23T14:31:27.226837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225528+0000","last_change":"2024-04-23T14:31:25.828041+0000","last_active":"2024-04-23T14:31:27.225528+0000","last_peered":"2024-04-23T14:31:27.225528+0000","last_clean":"2024-04-23T14:31:27.225528+0000","last_became_active":"2024-04-23T14:31:25.827517+0000","last_became_peered":"2024-04-23T14:31:25.827517+0000","last_unstale":"2024-04-23T14:31:27.225528+0000","last_undegraded":"2024-04-23T14:31:27.225528+0000","last_fullsized":"2024-04-23T14:31:27.225528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:54.674411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.814388+0000","last_change":"2024-04-23T14:31:24.800057+0000","last_active":"2024-04-23T14:31:26.814388+0000","last_peered":"2024-04-23T14:31:26.814388+0000","last_clean":"2024-04-23T14:31:26.814388+0000","last_became_active":"2024-04-23T14:31:24.799710+0000","last_became_peered":"2024-04-23T14:31:24.799710+0000","last_unstale":"2024-04-23T14:31:26.814388+0000","last_undegraded":"2024-04-23T14:31:26.814388+0000","last_fullsized":"2024-04-23T14:31:26.814388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833395+0000","last_change":"2024-04-23T14:31:25.820222+0000","last_active":"2024-04-23T14:31:26.833395+0000","last_peered":"2024-04-23T14:31:26.833395+0000","last_clean":"2024-04-23T14:31:26.833395+0000","last_became_active":"2024-04-23T14:31:25.818514+0000","last_became_peered":"2024-04-23T14:31:25.818514+0000","last_unstale":"2024-04-23T14:31:26.833395+0000","last_undegraded":"2024-04-23T14:31:26.833395+0000","last_fullsized":"2024-04-23T14:31:26.833395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:55:48.815407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229585+0000","last_change":"2024-04-23T14:31:24.802434+0000","last_active":"2024-04-23T14:31:27.229585+0000","last_peered":"2024-04-23T14:31:27.229585+0000","last_clean":"2024-04-23T14:31:27.229585+0000","last_became_active":"2024-04-23T14:31:24.800504+0000","last_became_peered":"2024-04-23T14:31:24.800504+0000","last_unstale":"2024-04-23T14:31:27.229585+0000","last_undegraded":"2024-04-23T14:31:27.229585+0000","last_fullsized":"2024-04-23T14:31:27.229585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.822417+0000","last_change":"2024-04-23T14:31:25.824755+0000","last_active":"2024-04-23T14:31:26.822417+0000","last_peered":"2024-04-23T14:31:26.822417+0000","last_clean":"2024-04-23T14:31:26.822417+0000","last_became_active":"2024-04-23T14:31:25.820209+0000","last_became_peered":"2024-04-23T14:31:25.820209+0000","last_unstale":"2024-04-23T14:31:26.822417+0000","last_undegraded":"2024-04-23T14:31:26.822417+0000","last_fullsized":"2024-04-23T14:31:26.822417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:41.728479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833340+0000","last_change":"2024-04-23T14:31:24.795154+0000","last_active":"2024-04-23T14:31:26.833340+0000","last_peered":"2024-04-23T14:31:26.833340+0000","last_clean":"2024-04-23T14:31:26.833340+0000","last_became_active":"2024-04-23T14:31:24.794851+0000","last_became_peered":"2024-04-23T14:31:24.794851+0000","last_unstale":"2024-04-23T14:31:26.833340+0000","last_undegraded":"2024-04-23T14:31:26.833340+0000","last_fullsized":"2024-04-23T14:31:26.833340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230608+0000","last_change":"2024-04-23T14:31:25.836961+0000","last_active":"2024-04-23T14:31:27.230608+0000","last_peered":"2024-04-23T14:31:27.230608+0000","last_clean":"2024-04-23T14:31:27.230608+0000","last_became_active":"2024-04-23T14:31:25.836649+0000","last_became_peered":"2024-04-23T14:31:25.836649+0000","last_unstale":"2024-04-23T14:31:27.230608+0000","last_undegraded":"2024-04-23T14:31:27.230608+0000","last_fullsized":"2024-04-23T14:31:27.230608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.896865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229915+0000","last_change":"2024-04-23T14:31:24.817767+0000","last_active":"2024-04-23T14:31:27.229915+0000","last_peered":"2024-04-23T14:31:27.229915+0000","last_clean":"2024-04-23T14:31:27.229915+0000","last_became_active":"2024-04-23T14:31:24.817440+0000","last_became_peered":"2024-04-23T14:31:24.817440+0000","last_unstale":"2024-04-23T14:31:27.229915+0000","last_undegraded":"2024-04-23T14:31:27.229915+0000","last_fullsized":"2024-04-23T14:31:27.229915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833407+0000","last_change":"2024-04-23T14:31:25.821773+0000","last_active":"2024-04-23T14:31:26.833407+0000","last_peered":"2024-04-23T14:31:26.833407+0000","last_clean":"2024-04-23T14:31:26.833407+0000","last_became_active":"2024-04-23T14:31:25.820550+0000","last_became_peered":"2024-04-23T14:31:25.820550+0000","last_unstale":"2024-04-23T14:31:26.833407+0000","last_undegraded":"2024-04-23T14:31:26.833407+0000","last_fullsized":"2024-04-23T14:31:26.833407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:24.757671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.834047+0000","last_change":"2024-04-23T14:31:24.798112+0000","last_active":"2024-04-23T14:31:26.834047+0000","last_peered":"2024-04-23T14:31:26.834047+0000","last_clean":"2024-04-23T14:31:26.834047+0000","last_became_active":"2024-04-23T14:31:24.797826+0000","last_became_peered":"2024-04-23T14:31:24.797826+0000","last_unstale":"2024-04-23T14:31:26.834047+0000","last_undegraded":"2024-04-23T14:31:26.834047+0000","last_fullsized":"2024-04-23T14:31:26.834047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225465+0000","last_change":"2024-04-23T14:31:25.835731+0000","last_active":"2024-04-23T14:31:27.225465+0000","last_peered":"2024-04-23T14:31:27.225465+0000","last_clean":"2024-04-23T14:31:27.225465+0000","last_became_active":"2024-04-23T14:31:25.835493+0000","last_became_peered":"2024-04-23T14:31:25.835493+0000","last_unstale":"2024-04-23T14:31:27.225465+0000","last_undegraded":"2024-04-23T14:31:27.225465+0000","last_fullsized":"2024-04-23T14:31:27.225465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821845+0000","last_change":"2024-04-23T14:31:24.813578+0000","last_active":"2024-04-23T14:31:26.821845+0000","last_peered":"2024-04-23T14:31:26.821845+0000","last_clean":"2024-04-23T14:31:26.821845+0000","last_became_active":"2024-04-23T14:31:24.812481+0000","last_became_peered":"2024-04-23T14:31:24.812481+0000","last_unstale":"2024-04-23T14:31:26.821845+0000","last_undegraded":"2024-04-23T14:31:26.821845+0000","last_fullsized":"2024-04-23T14:31:26.821845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225467+0000","last_change":"2024-04-23T14:31:25.808683+0000","last_active":"2024-04-23T14:31:27.225467+0000","last_peered":"2024-04-23T14:31:27.225467+0000","last_clean":"2024-04-23T14:31:27.225467+0000","last_became_active":"2024-04-23T14:31:25.808301+0000","last_became_peered":"2024-04-23T14:31:25.808301+0000","last_unstale":"2024-04-23T14:31:27.225467+0000","last_undegraded":"2024-04-23T14:31:27.225467+0000","last_fullsized":"2024-04-23T14:31:27.225467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:32.954515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.226336+0000","last_change":"2024-04-23T14:31:24.817508+0000","last_active":"2024-04-23T14:31:27.226336+0000","last_peered":"2024-04-23T14:31:27.226336+0000","last_clean":"2024-04-23T14:31:27.226336+0000","last_became_active":"2024-04-23T14:31:24.817260+0000","last_became_peered":"2024-04-23T14:31:24.817260+0000","last_unstale":"2024-04-23T14:31:27.226336+0000","last_undegraded":"2024-04-23T14:31:27.226336+0000","last_fullsized":"2024-04-23T14:31:27.226336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:13:29.074550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821504+0000","last_change":"2024-04-23T14:31:25.827518+0000","last_active":"2024-04-23T14:31:26.821504+0000","last_peered":"2024-04-23T14:31:26.821504+0000","last_clean":"2024-04-23T14:31:26.821504+0000","last_became_active":"2024-04-23T14:31:25.822569+0000","last_became_peered":"2024-04-23T14:31:25.822569+0000","last_unstale":"2024-04-23T14:31:26.821504+0000","last_undegraded":"2024-04-23T14:31:26.821504+0000","last_fullsized":"2024-04-23T14:31:26.821504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:58.606933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833567+0000","last_change":"2024-04-23T14:31:24.807224+0000","last_active":"2024-04-23T14:31:26.833567+0000","last_peered":"2024-04-23T14:31:26.833567+0000","last_clean":"2024-04-23T14:31:26.833567+0000","last_became_active":"2024-04-23T14:31:24.806958+0000","last_became_peered":"2024-04-23T14:31:24.806958+0000","last_unstale":"2024-04-23T14:31:26.833567+0000","last_undegraded":"2024-04-23T14:31:26.833567+0000","last_fullsized":"2024-04-23T14:31:26.833567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:24.724387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224355+0000","last_change":"2024-04-23T14:31:25.821318+0000","last_active":"2024-04-23T14:31:27.224355+0000","last_peered":"2024-04-23T14:31:27.224355+0000","last_clean":"2024-04-23T14:31:27.224355+0000","last_became_active":"2024-04-23T14:31:25.820930+0000","last_became_peered":"2024-04-23T14:31:25.820930+0000","last_unstale":"2024-04-23T14:31:27.224355+0000","last_undegraded":"2024-04-23T14:31:27.224355+0000","last_fullsized":"2024-04-23T14:31:27.224355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:52:41.167953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.833571+0000","last_change":"2024-04-23T14:31:24.799281+0000","last_active":"2024-04-23T14:31:26.833571+0000","last_peered":"2024-04-23T14:31:26.833571+0000","last_clean":"2024-04-23T14:31:26.833571+0000","last_became_active":"2024-04-23T14:31:24.798804+0000","last_became_peered":"2024-04-23T14:31:24.798804+0000","last_unstale":"2024-04-23T14:31:26.833571+0000","last_undegraded":"2024-04-23T14:31:26.833571+0000","last_fullsized":"2024-04-23T14:31:26.833571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:40.714099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224405+0000","last_change":"2024-04-23T14:31:25.821631+0000","last_active":"2024-04-23T14:31:27.224405+0000","last_peered":"2024-04-23T14:31:27.224405+0000","last_clean":"2024-04-23T14:31:27.224405+0000","last_became_active":"2024-04-23T14:31:25.820439+0000","last_became_peered":"2024-04-23T14:31:25.820439+0000","last_unstale":"2024-04-23T14:31:27.224405+0000","last_undegraded":"2024-04-23T14:31:27.224405+0000","last_fullsized":"2024-04-23T14:31:27.224405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:41.038179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.228114+0000","last_change":"2024-04-23T14:31:24.814559+0000","last_active":"2024-04-23T14:31:27.228114+0000","last_peered":"2024-04-23T14:31:27.228114+0000","last_clean":"2024-04-23T14:31:27.228114+0000","last_became_active":"2024-04-23T14:31:24.814274+0000","last_became_peered":"2024-04-23T14:31:24.814274+0000","last_unstale":"2024-04-23T14:31:27.228114+0000","last_undegraded":"2024-04-23T14:31:27.228114+0000","last_fullsized":"2024-04-23T14:31:27.228114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:20:04.831686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:26.821011+0000","last_change":"2024-04-23T14:31:25.829871+0000","last_active":"2024-04-23T14:31:26.821011+0000","last_peered":"2024-04-23T14:31:26.821011+0000","last_clean":"2024-04-23T14:31:26.821011+0000","last_became_active":"2024-04-23T14:31:25.828620+0000","last_became_peered":"2024-04-23T14:31:25.828620+0000","last_unstale":"2024-04-23T14:31:26.821011+0000","last_undegraded":"2024-04-23T14:31:26.821011+0000","last_fullsized":"2024-04-23T14:31:26.821011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:39.048069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.230152+0000","last_change":"2024-04-23T14:31:25.846861+0000","last_active":"2024-04-23T14:31:27.230152+0000","last_peered":"2024-04-23T14:31:27.230152+0000","last_clean":"2024-04-23T14:31:27.230152+0000","last_became_active":"2024-04-23T14:31:25.846267+0000","last_became_peered":"2024-04-23T14:31:25.846267+0000","last_unstale":"2024-04-23T14:31:27.230152+0000","last_undegraded":"2024-04-23T14:31:27.230152+0000","last_fullsized":"2024-04-23T14:31:27.230152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:37:01.405552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.227598+0000","last_change":"2024-04-23T14:31:24.816507+0000","last_active":"2024-04-23T14:31:27.227598+0000","last_peered":"2024-04-23T14:31:27.227598+0000","last_clean":"2024-04-23T14:31:27.227598+0000","last_became_active":"2024-04-23T14:31:24.816159+0000","last_became_peered":"2024-04-23T14:31:24.816159+0000","last_unstale":"2024-04-23T14:31:27.227598+0000","last_undegraded":"2024-04-23T14:31:27.227598+0000","last_fullsized":"2024-04-23T14:31:27.227598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:12:57.592554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229590+0000","last_change":"2024-04-23T14:31:25.846016+0000","last_active":"2024-04-23T14:31:27.229590+0000","last_peered":"2024-04-23T14:31:27.229590+0000","last_clean":"2024-04-23T14:31:27.229590+0000","last_became_active":"2024-04-23T14:31:25.845682+0000","last_became_peered":"2024-04-23T14:31:25.845682+0000","last_unstale":"2024-04-23T14:31:27.229590+0000","last_undegraded":"2024-04-23T14:31:27.229590+0000","last_fullsized":"2024-04-23T14:31:27.229590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:51:42.434243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.229170+0000","last_change":"2024-04-23T14:31:24.819529+0000","last_active":"2024-04-23T14:31:27.229170+0000","last_peered":"2024-04-23T14:31:27.229170+0000","last_clean":"2024-04-23T14:31:27.229170+0000","last_became_active":"2024-04-23T14:31:24.819162+0000","last_became_peered":"2024-04-23T14:31:24.819162+0000","last_unstale":"2024-04-23T14:31:27.229170+0000","last_undegraded":"2024-04-23T14:31:27.229170+0000","last_fullsized":"2024-04-23T14:31:27.229170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:41:31.364038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225173+0000","last_change":"2024-04-23T14:31:25.837928+0000","last_active":"2024-04-23T14:31:27.225173+0000","last_peered":"2024-04-23T14:31:27.225173+0000","last_clean":"2024-04-23T14:31:27.225173+0000","last_became_active":"2024-04-23T14:31:25.837281+0000","last_became_peered":"2024-04-23T14:31:25.837281+0000","last_unstale":"2024-04-23T14:31:27.225173+0000","last_undegraded":"2024-04-23T14:31:27.225173+0000","last_fullsized":"2024-04-23T14:31:27.225173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:59:15.348815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224998+0000","last_change":"2024-04-23T14:31:24.793060+0000","last_active":"2024-04-23T14:31:27.224998+0000","last_peered":"2024-04-23T14:31:27.224998+0000","last_clean":"2024-04-23T14:31:27.224998+0000","last_became_active":"2024-04-23T14:31:24.792763+0000","last_became_peered":"2024-04-23T14:31:24.792763+0000","last_unstale":"2024-04-23T14:31:27.224998+0000","last_undegraded":"2024-04-23T14:31:27.224998+0000","last_fullsized":"2024-04-23T14:31:27.224998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:07.209554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.224786+0000","last_change":"2024-04-23T14:31:25.824548+0000","last_active":"2024-04-23T14:31:27.224786+0000","last_peered":"2024-04-23T14:31:27.224786+0000","last_clean":"2024-04-23T14:31:27.224786+0000","last_became_active":"2024-04-23T14:31:25.824207+0000","last_became_peered":"2024-04-23T14:31:25.824207+0000","last_unstale":"2024-04-23T14:31:27.224786+0000","last_undegraded":"2024-04-23T14:31:27.224786+0000","last_fullsized":"2024-04-23T14:31:27.224786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:24.758587+0000","last_clean_scrub_stamp":"2024-04-23T14:31:24.758587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:07.556813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-23T14:31:27.225623+0000","last_change":"2024-04-23T14:31:24.810861+0000","last_active":"2024-04-23T14:31:27.225623+0000","last_peered":"2024-04-23T14:31:27.225623+0000","last_clean":"2024-04-23T14:31:27.225623+0000","last_became_active":"2024-04-23T14:31:24.810466+0000","last_became_peered":"2024-04-23T14:31:24.810466+0000","last_unstale":"2024-04-23T14:31:27.225623+0000","last_undegraded":"2024-04-23T14:31:27.225623+0000","last_fullsized":"2024-04-23T14:31:27.225623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:23.744208+0000","last_clean_scrub_stamp":"2024-04-23T14:31:23.744208+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:34.534346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:31:36.740 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T14:31:36.740 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T14:31:36.740 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-23T14:31:36.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:31:36.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.971+0000 7fc26e400640 1 Processor -- start 2024-04-23T14:31:36.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.971+0000 7fc26e400640 1 -- start start 2024-04-23T14:31:36.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26e400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2680cd050 0x7fc26806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc2680b3080 0x7fc2680cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26e400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc26815f160 con 0x7fc26815ece0 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26e400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc2680ced20 con 0x7fc2680b3080 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26e400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc26815fcd0 con 0x7fc2680cd050 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc2680b3080 0x7fc2680cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc2680b3080 0x7fc2680cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33568/0 (socket says 172.21.15.59:33568) 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26da00640 1 -- 172.21.15.59:0/3859553933 learned_addr learned my addr 172.21.15.59:0/3859553933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.973+0000 7fc26d000640 1 --2- 172.21.15.59:0/3859553933 >> v2:172.21.15.76:3300/0 conn(0x7fc2680cd050 0x7fc26806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.974+0000 7fc26da00640 1 -- 172.21.15.59:0/3859553933 >> v2:172.21.15.76:3300/0 conn(0x7fc2680cd050 msgr2=0x7fc26806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.974+0000 7fc26da00640 1 --2- 172.21.15.59:0/3859553933 >> v2:172.21.15.76:3300/0 conn(0x7fc2680cd050 0x7fc26806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.974+0000 7fc26da00640 1 -- 172.21.15.59:0/3859553933 >> v1:172.21.15.76:6789/0 conn(0x7fc26815ece0 legacy=0x7fc2680cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:36.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.974+0000 7fc26da00640 1 -- 172.21.15.59:0/3859553933 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26815f430 con 0x7fc2680b3080 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.975+0000 7fc26da00640 1 --2- 172.21.15.59:0/3859553933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2680b3080 0x7fc2680cf470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc26806e080 tx=0x7fc26002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58af24ddc27fa3bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.975+0000 7fc267400640 1 -- 172.21.15.59:0/3859553933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260036070 con 0x7fc2680b3080 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.975+0000 7fc267400640 1 -- 172.21.15.59:0/3859553933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc260031600 con 0x7fc2680b3080 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.975+0000 7fc267400640 1 -- 172.21.15.59:0/3859553933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2600319d0 con 0x7fc2680b3080 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 -- 172.21.15.59:0/3859553933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2680b3080 msgr2=0x7fc2680cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 --2- 172.21.15.59:0/3859553933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2680b3080 0x7fc2680cf470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc26806e080 tx=0x7fc26002a660 comp rx=0 tx=0).stop 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 -- 172.21.15.59:0/3859553933 shutdown_connections 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 --2- 172.21.15.59:0/3859553933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2680b3080 0x7fc2680cf470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 --2- 172.21.15.59:0/3859553933 >> v2:172.21.15.76:3300/0 conn(0x7fc2680cd050 0x7fc26806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 -- 172.21.15.59:0/3859553933 >> 172.21.15.59:0/3859553933 conn(0x7fc2680b21a0 msgr2=0x7fc2680b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 -- 172.21.15.59:0/3859553933 shutdown_connections 2024-04-23T14:31:36.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.976+0000 7fc26e400640 1 -- 172.21.15.59:0/3859553933 wait complete. 2024-04-23T14:31:36.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.977+0000 7fc26e400640 1 Processor -- start 2024-04-23T14:31:36.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.977+0000 7fc26e400640 1 -- start start 2024-04-23T14:31:36.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.978+0000 7fc26e400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2680b3080 0x7fc2680c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.978+0000 7fc26e400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2680cd050 0x7fc2680c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.978+0000 7fc26e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc26815ece0 0x7fc268157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.978+0000 7fc26e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2680cef50 con 0x7fc26815ece0 2024-04-23T14:31:36.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.978+0000 7fc26e400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc26815f9f0 con 0x7fc2680cd050 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.978+0000 7fc26e400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc268159110 con 0x7fc2680b3080 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc26815ece0 0x7fc268157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc26815ece0 0x7fc268157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33580/0 (socket says 172.21.15.59:33580) 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 -- 172.21.15.59:0/4260366039 learned_addr learned my addr 172.21.15.59:0/4260366039 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc267e00640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2680cd050 0x7fc2680c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26d000640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2680b3080 0x7fc2680c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 -- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2680b3080 msgr2=0x7fc2680c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2680b3080 0x7fc2680c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 -- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2680cd050 msgr2=0x7fc2680c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2680cd050 0x7fc2680c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc26da00640 1 -- 172.21.15.59:0/4260366039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2681579c0 con 0x7fc26815ece0 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.979+0000 7fc267e00640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2680cd050 0x7fc2680c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.980+0000 7fc26da00640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc26815ece0 0x7fc268157480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc260034040 tx=0x7fc260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.980+0000 7fc264c00640 1 -- 172.21.15.59:0/4260366039 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260036050 con 0x7fc26815ece0 2024-04-23T14:31:36.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.980+0000 7fc264c00640 1 -- 172.21.15.59:0/4260366039 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc260042070 con 0x7fc26815ece0 2024-04-23T14:31:36.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.980+0000 7fc264c00640 1 -- 172.21.15.59:0/4260366039 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260009510 con 0x7fc26815ece0 2024-04-23T14:31:36.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.980+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2680c9980 con 0x7fc26815ece0 2024-04-23T14:31:36.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.980+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2680c9f40 con 0x7fc26815ece0 2024-04-23T14:31:36.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.981+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26804fd90 con 0x7fc26815ece0 2024-04-23T14:31:36.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.982+0000 7fc264c00640 1 -- 172.21.15.59:0/4260366039 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc260007bd0 con 0x7fc26815ece0 2024-04-23T14:31:36.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.983+0000 7fc264c00640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc24806c8f0 0x7fc24806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:36.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.983+0000 7fc264c00640 1 -- 172.21.15.59:0/4260366039 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc2600aca80 con 0x7fc26815ece0 2024-04-23T14:31:36.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.983+0000 7fc26d000640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc24806c8f0 0x7fc24806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:36.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.984+0000 7fc26d000640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc24806c8f0 0x7fc24806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc25c006fd0 tx=0x7fc25c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:36.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:36.987+0000 7fc264c00640 1 -- 172.21.15.59:0/4260366039 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2600aee20 con 0x7fc26815ece0 2024-04-23T14:31:37.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.231+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc26806de10 con 0x7fc26815ece0 2024-04-23T14:31:37.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.234+0000 7fc264c00640 1 -- 172.21.15.59:0/4260366039 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc260045360 con 0x7fc26815ece0 2024-04-23T14:31:37.235 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:37.236 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T14:31:37.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc24806c8f0 msgr2=0x7fc24806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc24806c8f0 0x7fc24806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc25c006fd0 tx=0x7fc25c008040 comp rx=0 tx=0).stop 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc26815ece0 msgr2=0x7fc268157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc26815ece0 0x7fc268157480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc260034040 tx=0x7fc260004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 shutdown_connections 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc24806c8f0 0x7fc24806edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc26815ece0 0x7fc268157480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2680cd050 0x7fc2680c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 --2- 172.21.15.59:0/4260366039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2680b3080 0x7fc2680c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.237+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 >> 172.21.15.59:0/4260366039 conn(0x7fc2680b21a0 msgr2=0x7fc2680ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.238+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 shutdown_connections 2024-04-23T14:31:37.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.238+0000 7fc26e400640 1 -- 172.21.15.59:0/4260366039 wait complete. 2024-04-23T14:31:37.256 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-23T14:31:37.257 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.471+0000 7f222a200640 1 Processor -- start 2024-04-23T14:31:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.471+0000 7f222a200640 1 -- start start 2024-04-23T14:31:37.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.472+0000 7f222a200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f22240cd050 0x7f222406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.472+0000 7f222a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240b3080 0x7f22240cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.472+0000 7f222a200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222415f160 con 0x7f222415ece0 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.472+0000 7f222a200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f22240ced20 con 0x7f22240cd050 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.472+0000 7f222a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f222415fcd0 con 0x7f22240b3080 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2223e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f222415ece0 0x7f22240cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36606/0 (socket says 172.21.15.59:36606) 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2223e00640 1 -- 172.21.15.59:0/3024530392 learned_addr learned my addr 172.21.15.59:0/3024530392 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2228e00640 1 --2- 172.21.15.59:0/3024530392 >> v2:172.21.15.59:3300/0 conn(0x7f22240cd050 0x7f222406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2229800640 1 --2- 172.21.15.59:0/3024530392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240b3080 0x7f22240cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2223400640 1 -- 172.21.15.59:0/3024530392 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3267135726 0 0) 0x7f222415f160 con 0x7f222415ece0 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2223400640 1 -- 172.21.15.59:0/3024530392 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2208003520 con 0x7f222415ece0 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2228e00640 1 -- 172.21.15.59:0/3024530392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240b3080 msgr2=0x7f22240cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2228e00640 1 --2- 172.21.15.59:0/3024530392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240b3080 0x7f22240cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2228e00640 1 -- 172.21.15.59:0/3024530392 >> v1:172.21.15.59:6789/0 conn(0x7f222415ece0 legacy=0x7f22240cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2228e00640 1 -- 172.21.15.59:0/3024530392 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222415f430 con 0x7f22240cd050 2024-04-23T14:31:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.473+0000 7f2229800640 1 --2- 172.21.15.59:0/3024530392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240b3080 0x7f22240cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.474+0000 7f2228e00640 1 --2- 172.21.15.59:0/3024530392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240cd050 0x7f222406f620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2214009d90 tx=0x7f221402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60b55f7ec489034 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f2223400640 1 -- 172.21.15.59:0/3024530392 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214031600 con 0x7f22240cd050 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f2223400640 1 -- 172.21.15.59:0/3024530392 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22140317a0 con 0x7f22240cd050 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f2223400640 1 -- 172.21.15.59:0/3024530392 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214031b70 con 0x7f22240cd050 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f222a200640 1 -- 172.21.15.59:0/3024530392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240cd050 msgr2=0x7f222406f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f222a200640 1 --2- 172.21.15.59:0/3024530392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240cd050 0x7f222406f620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2214009d90 tx=0x7f221402a5d0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f222a200640 1 -- 172.21.15.59:0/3024530392 shutdown_connections 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f222a200640 1 --2- 172.21.15.59:0/3024530392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240b3080 0x7f22240cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f222a200640 1 --2- 172.21.15.59:0/3024530392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240cd050 0x7f222406f620 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f222a200640 1 -- 172.21.15.59:0/3024530392 >> 172.21.15.59:0/3024530392 conn(0x7f22240b21a0 msgr2=0x7f22240b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.475+0000 7f222a200640 1 -- 172.21.15.59:0/3024530392 shutdown_connections 2024-04-23T14:31:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.476+0000 7f222a200640 1 -- 172.21.15.59:0/3024530392 wait complete. 2024-04-23T14:31:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.477+0000 7f222a200640 1 Processor -- start 2024-04-23T14:31:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.477+0000 7f222a200640 1 -- start start 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.478+0000 7f222a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240b3080 0x7f222415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.478+0000 7f222a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240cd050 0x7f2224159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.478+0000 7f222a200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f222415ece0 0x7f22241598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.478+0000 7f222a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22240cef50 con 0x7f22240b3080 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.478+0000 7f222a200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f222415f9f0 con 0x7f222415ece0 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.478+0000 7f222a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2224159110 con 0x7f22240cd050 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240b3080 0x7f222415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240b3080 0x7f222415af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33610/0 (socket says 172.21.15.59:33610) 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 -- 172.21.15.59:0/148206826 learned_addr learned my addr 172.21.15.59:0/148206826 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2223e00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240cd050 0x7f2224159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 -- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240cd050 msgr2=0x7f2224159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240cd050 0x7f2224159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 -- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f222415ece0 msgr2=0x7f22241598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2229800640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f222415ece0 0x7f22241598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f222415ece0 0x7f22241598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2228e00640 1 -- 172.21.15.59:0/148206826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2224159ee0 con 0x7f22240b3080 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.479+0000 7f2223e00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240cd050 0x7f2224159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.480+0000 7f2228e00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240b3080 0x7f222415af80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2214009eb0 tx=0x7f2214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.480+0000 7f2220c00640 1 -- 172.21.15.59:0/148206826 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214040070 con 0x7f22240b3080 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.480+0000 7f2220c00640 1 -- 172.21.15.59:0/148206826 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2214030be0 con 0x7f22240b3080 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.480+0000 7f2220c00640 1 -- 172.21.15.59:0/148206826 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214009450 con 0x7f22240b3080 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.480+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222415a1c0 con 0x7f22240b3080 2024-04-23T14:31:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.480+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222415a780 con 0x7f22240b3080 2024-04-23T14:31:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.481+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222404fd90 con 0x7f22240b3080 2024-04-23T14:31:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.482+0000 7f2220c00640 1 -- 172.21.15.59:0/148206826 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2214039040 con 0x7f22240b3080 2024-04-23T14:31:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.483+0000 7f2220c00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.483+0000 7f2220c00640 1 -- 172.21.15.59:0/148206826 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22140ad1e0 con 0x7f22240b3080 2024-04-23T14:31:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.483+0000 7f2223e00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.484+0000 7f2223e00640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22240c7210 tx=0x7f2210009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:37.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.488+0000 7f2220c00640 1 -- 172.21.15.59:0/148206826 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f221407afb0 con 0x7f22240b3080 2024-04-23T14:31:37.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.692+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f222409e140 con 0x7f22240b3080 2024-04-23T14:31:37.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.695+0000 7f2220c00640 1 -- 172.21.15.59:0/148206826 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f221407f2b0 con 0x7f22240b3080 2024-04-23T14:31:37.696 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:37.696 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:37.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220406c8f0 msgr2=0x7f220406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22240c7210 tx=0x7f2210009040 comp rx=0 tx=0).stop 2024-04-23T14:31:37.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240b3080 msgr2=0x7f222415af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240b3080 0x7f222415af80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2214009eb0 tx=0x7f2214004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 shutdown_connections 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22240cd050 0x7f2224159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f222415ece0 0x7f22241598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 --2- 172.21.15.59:0/148206826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22240b3080 0x7f222415af80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 >> 172.21.15.59:0/148206826 conn(0x7f22240b21a0 msgr2=0x7f222406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 shutdown_connections 2024-04-23T14:31:37.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.697+0000 7f222a200640 1 -- 172.21.15.59:0/148206826 wait complete. 2024-04-23T14:31:37.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:31:37.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:31:37.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.937+0000 7fee8d200640 1 Processor -- start 2024-04-23T14:31:37.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.938+0000 7fee8d200640 1 -- start start 2024-04-23T14:31:37.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.939+0000 7fee8d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee880b4f20 0x7fee880cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.939+0000 7fee8d200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee880b5da0 con 0x7fee880d15b0 2024-04-23T14:31:37.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.939+0000 7fee8d200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee880d0a20 con 0x7fee8815ed20 2024-04-23T14:31:37.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.939+0000 7fee8d200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fee8815f100 con 0x7fee880b4f20 2024-04-23T14:31:37.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.939+0000 7fee87e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee880b4f20 0x7fee880cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.939+0000 7fee86a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fee880d15b0 0x7fee880cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36614/0 (socket says 172.21.15.59:36614) 2024-04-23T14:31:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.939+0000 7fee86a00640 1 -- 172.21.15.59:0/2513817498 learned_addr learned my addr 172.21.15.59:0/2513817498 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.940+0000 7fee87e00640 1 -- 172.21.15.59:0/2513817498 >> v1:172.21.15.76:6789/0 conn(0x7fee8815ed20 legacy=0x7fee880d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.940+0000 7fee87e00640 1 -- 172.21.15.59:0/2513817498 >> v1:172.21.15.59:6789/0 conn(0x7fee880d15b0 legacy=0x7fee880cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.940+0000 7fee87e00640 1 -- 172.21.15.59:0/2513817498 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee880d2ed0 con 0x7fee880b4f20 2024-04-23T14:31:37.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.940+0000 7fee86000640 1 -- 172.21.15.59:0/2513817498 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4131296717 0 0) 0x7fee880b5da0 con 0x7fee880d15b0 2024-04-23T14:31:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.941+0000 7fee87e00640 1 --2- 172.21.15.59:0/2513817498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880cf990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee80009d90 tx=0x7fee8002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d106ad20568913cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.941+0000 7fee86000640 1 -- 172.21.15.59:0/2513817498 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80031600 con 0x7fee880b4f20 2024-04-23T14:31:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.941+0000 7fee86000640 1 -- 172.21.15.59:0/2513817498 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee800317a0 con 0x7fee880b4f20 2024-04-23T14:31:37.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.941+0000 7fee86000640 1 -- 172.21.15.59:0/2513817498 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80031b70 con 0x7fee880b4f20 2024-04-23T14:31:37.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.942+0000 7fee8d200640 1 -- 172.21.15.59:0/2513817498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 msgr2=0x7fee880cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.942+0000 7fee8d200640 1 --2- 172.21.15.59:0/2513817498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880cf990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee80009d90 tx=0x7fee8002a5d0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.942+0000 7fee8d200640 1 -- 172.21.15.59:0/2513817498 shutdown_connections 2024-04-23T14:31:37.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.942+0000 7fee8d200640 1 --2- 172.21.15.59:0/2513817498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880cf990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.942+0000 7fee8d200640 1 -- 172.21.15.59:0/2513817498 >> 172.21.15.59:0/2513817498 conn(0x7fee8808bf90 msgr2=0x7fee880d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:37.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.942+0000 7fee8d200640 1 -- 172.21.15.59:0/2513817498 shutdown_connections 2024-04-23T14:31:37.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.942+0000 7fee8d200640 1 -- 172.21.15.59:0/2513817498 wait complete. 2024-04-23T14:31:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.944+0000 7fee8d200640 1 Processor -- start 2024-04-23T14:31:37.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.944+0000 7fee8d200640 1 -- start start 2024-04-23T14:31:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee8d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee8d200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee880d15b0 0x7fee881f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee8d200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee8815ed20 0x7fee880b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee8d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee880b3600 con 0x7fee880b4f20 2024-04-23T14:31:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee8d200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee880d3490 con 0x7fee8815ed20 2024-04-23T14:31:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee8d200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee880b3780 con 0x7fee880d15b0 2024-04-23T14:31:37.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee87400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee87400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33634/0 (socket says 172.21.15.59:33634) 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.945+0000 7fee87400640 1 -- 172.21.15.59:0/2537729674 learned_addr learned my addr 172.21.15.59:0/2537729674 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee87400640 1 -- 172.21.15.59:0/2537729674 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee880d15b0 msgr2=0x7fee881f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee87400640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee880d15b0 0x7fee881f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee87400640 1 -- 172.21.15.59:0/2537729674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee8815ed20 msgr2=0x7fee880b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee87e00640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee8815ed20 0x7fee880b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee87400640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee8815ed20 0x7fee880b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee87400640 1 -- 172.21.15.59:0/2537729674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8815a410 con 0x7fee880b4f20 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee87400640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880b2400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c430 tx=0x7fee7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee7f400640 1 -- 172.21.15.59:0/2537729674 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78017070 con 0x7fee880b4f20 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee7f400640 1 -- 172.21.15.59:0/2537729674 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee780052d0 con 0x7fee880b4f20 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee7f400640 1 -- 172.21.15.59:0/2537729674 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78003830 con 0x7fee880b4f20 2024-04-23T14:31:37.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8815a6f0 con 0x7fee880b4f20 2024-04-23T14:31:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.946+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8815acb0 con 0x7fee880b4f20 2024-04-23T14:31:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.948+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee8804fd90 con 0x7fee880b4f20 2024-04-23T14:31:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.949+0000 7fee7f400640 1 -- 172.21.15.59:0/2537729674 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee7801a070 con 0x7fee880b4f20 2024-04-23T14:31:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.950+0000 7fee7f400640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee6806c8f0 0x7fee6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.950+0000 7fee7f400640 1 -- 172.21.15.59:0/2537729674 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee7808aed0 con 0x7fee880b4f20 2024-04-23T14:31:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.950+0000 7fee86a00640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee6806c8f0 0x7fee6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.951+0000 7fee86a00640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee6806c8f0 0x7fee6806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fee74007fd0 tx=0x7fee74007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:37.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:37.954+0000 7fee7f400640 1 -- 172.21.15.59:0/2537729674 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee78059300 con 0x7fee880b4f20 2024-04-23T14:31:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.159+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee880d4f00 con 0x7fee880b4f20 2024-04-23T14:31:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.161+0000 7fee7f400640 1 -- 172.21.15.59:0/2537729674 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fee78058c00 con 0x7fee880b4f20 2024-04-23T14:31:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:31:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee6806c8f0 msgr2=0x7fee6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee6806c8f0 0x7fee6806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fee74007fd0 tx=0x7fee74007900 comp rx=0 tx=0).stop 2024-04-23T14:31:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 msgr2=0x7fee880b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880b2400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c430 tx=0x7fee7800c8f0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 shutdown_connections 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee6806c8f0 0x7fee6806edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee880d15b0 0x7fee881f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee8815ed20 0x7fee880b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 --2- 172.21.15.59:0/2537729674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee880b4f20 0x7fee880b2400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 >> 172.21.15.59:0/2537729674 conn(0x7fee8808bf90 msgr2=0x7fee8815d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 shutdown_connections 2024-04-23T14:31:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.163+0000 7fee8d200640 1 -- 172.21.15.59:0/2537729674 wait complete. 2024-04-23T14:31:38.184 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:31:38.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.392+0000 7f48a7200640 1 Processor -- start 2024-04-23T14:31:38.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.392+0000 7f48a7200640 1 -- start start 2024-04-23T14:31:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.394+0000 7f48a7200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f48a80d15b0 0x7f48a80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.394+0000 7f48a7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 0x7f48a80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.394+0000 7f48a7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a80b5da0 con 0x7f48a815ed20 2024-04-23T14:31:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.394+0000 7f48a7200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f48a815f100 con 0x7f48a80d15b0 2024-04-23T14:31:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.394+0000 7f48a7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f48a80d0a20 con 0x7f48a80b4f20 2024-04-23T14:31:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.394+0000 7f48a5e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f48a815ed20 0x7f48a80d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36618/0 (socket says 172.21.15.59:36618) 2024-04-23T14:31:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.394+0000 7f48a5e00640 1 -- 172.21.15.59:0/3280809649 learned_addr learned my addr 172.21.15.59:0/3280809649 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:38.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f489d400640 1 --2- 172.21.15.59:0/3280809649 >> v2:172.21.15.76:3300/0 conn(0x7f48a80d15b0 0x7f48a80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a6800640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 0x7f48a80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079713425 0 0) 0x7f48a80b5da0 con 0x7f48a815ed20 2024-04-23T14:31:38.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f488c003520 con 0x7f48a815ed20 2024-04-23T14:31:38.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3150148726 0 0) 0x7f488c003520 con 0x7f48a815ed20 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 msgr2=0x7f48a80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 0x7f48a80cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 >> v2:172.21.15.76:3300/0 conn(0x7f48a80d15b0 msgr2=0x7f48a80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 --2- 172.21.15.59:0/3280809649 >> v2:172.21.15.76:3300/0 conn(0x7f48a80d15b0 0x7f48a80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a80d2ed0 con 0x7f48a815ed20 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f48980019f0 con 0x7f48a815ed20 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 >> v1:172.21.15.59:6789/0 conn(0x7f48a815ed20 legacy=0x7f48a80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 0x7f488c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c003f30 0x7f488c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488c004d40 0x7f488c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48a80d0a20 con 0x7f488c004d40 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f488c00ba20 con 0x7f488c003f30 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.395+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f488c00bc20 con 0x7f48a80b4f20 2024-04-23T14:31:38.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f48a6800640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488c004d40 0x7f488c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f489d400640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c003f30 0x7f488c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f48a5e00640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 0x7f488c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f48a6800640 1 -- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 msgr2=0x7f488c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:38.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f48a6800640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80b4f20 0x7f488c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f48a6800640 1 -- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c003f30 msgr2=0x7f488c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:38.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f48a6800640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c003f30 0x7f488c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.396+0000 7f48a6800640 1 -- 172.21.15.59:0/3280809649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c003520 con 0x7f488c004d40 2024-04-23T14:31:38.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.397+0000 7f48a6800640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488c004d40 0x7f488c0051c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009a90 tx=0x7f488c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea08814200c5218 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:38.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.397+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0036070 con 0x7f488c004d40 2024-04-23T14:31:38.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a0031400 con 0x7f488c004d40 2024-04-23T14:31:38.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a5400640 1 -- 172.21.15.59:0/3280809649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0009690 con 0x7f488c004d40 2024-04-23T14:31:38.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a7200640 1 -- 172.21.15.59:0/3280809649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488c004d40 msgr2=0x7f488c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a7200640 1 --2- 172.21.15.59:0/3280809649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488c004d40 0x7f488c0051c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009a90 tx=0x7f488c005830 comp rx=0 tx=0).stop 2024-04-23T14:31:38.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a6800640 1 -- 172.21.15.59:0/3280809649 reap_dead start 2024-04-23T14:31:38.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a7200640 1 -- 172.21.15.59:0/3280809649 shutdown_connections 2024-04-23T14:31:38.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a7200640 1 -- 172.21.15.59:0/3280809649 >> 172.21.15.59:0/3280809649 conn(0x7f48a808bf90 msgr2=0x7f48a80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:38.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a7200640 1 -- 172.21.15.59:0/3280809649 shutdown_connections 2024-04-23T14:31:38.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.398+0000 7f48a7200640 1 -- 172.21.15.59:0/3280809649 wait complete. 2024-04-23T14:31:38.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.400+0000 7f48a7200640 1 Processor -- start 2024-04-23T14:31:38.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.400+0000 7f48a7200640 1 -- start start 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.401+0000 7f48a7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c004d40 0x7f48a80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.401+0000 7f48a7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80c9870 0x7f48a814d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.401+0000 7f48a7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a80c9cf0 0x7f48a814db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.401+0000 7f48a7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48a80b3600 con 0x7f48a80c9cf0 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.401+0000 7f48a7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f48a80d3490 con 0x7f488c004d40 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.401+0000 7f48a7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f48a80b3780 con 0x7f48a80c9870 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a80c9cf0 0x7f48a814db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a80c9cf0 0x7f48a814db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33648/0 (socket says 172.21.15.59:33648) 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 -- 172.21.15.59:0/2823392031 learned_addr learned my addr 172.21.15.59:0/2823392031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f489d400640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80c9870 0x7f48a814d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a5e00640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c004d40 0x7f48a80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 -- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80c9870 msgr2=0x7f48a814d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80c9870 0x7f48a814d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 -- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c004d40 msgr2=0x7f48a80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c004d40 0x7f48a80ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 -- 172.21.15.59:0/2823392031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a80ce6f0 con 0x7f48a80c9cf0 2024-04-23T14:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.402+0000 7f48a6800640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a80c9cf0 0x7f48a814db20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009b80 tx=0x7f48a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.403+0000 7f489ea00640 1 -- 172.21.15.59:0/2823392031 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0036040 con 0x7f48a80c9cf0 2024-04-23T14:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.403+0000 7f489ea00640 1 -- 172.21.15.59:0/2823392031 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a0043070 con 0x7f48a80c9cf0 2024-04-23T14:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.403+0000 7f489ea00640 1 -- 172.21.15.59:0/2823392031 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a003c690 con 0x7f48a80c9cf0 2024-04-23T14:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.403+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a80cba30 con 0x7f48a80c9cf0 2024-04-23T14:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.403+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a80b8750 con 0x7f48a80c9cf0 2024-04-23T14:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.404+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a804fd90 con 0x7f48a80c9cf0 2024-04-23T14:31:38.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.405+0000 7f489ea00640 1 -- 172.21.15.59:0/2823392031 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48a003c830 con 0x7f48a80c9cf0 2024-04-23T14:31:38.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.406+0000 7f489ea00640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f488406c8f0 0x7f488406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.406+0000 7f489ea00640 1 -- 172.21.15.59:0/2823392031 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48a00abd60 con 0x7f48a80c9cf0 2024-04-23T14:31:38.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.406+0000 7f48a5e00640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f488406c8f0 0x7f488406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.407+0000 7f48a5e00640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f488406c8f0 0x7f488406edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4898001330 tx=0x7f4898009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:38.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.411+0000 7f489ea00640 1 -- 172.21.15.59:0/2823392031 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48a004c020 con 0x7f48a80c9cf0 2024-04-23T14:31:38.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.610+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48a80b5710 con 0x7f48a80c9cf0 2024-04-23T14:31:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.611+0000 7f489ea00640 1 -- 172.21.15.59:0/2823392031 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f48a007a190 con 0x7f48a80c9cf0 2024-04-23T14:31:38.612 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f488406c8f0 msgr2=0x7f488406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f488406c8f0 0x7f488406edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4898001330 tx=0x7f4898009040 comp rx=0 tx=0).stop 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a80c9cf0 msgr2=0x7f48a814db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a80c9cf0 0x7f48a814db20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009b80 tx=0x7f48a0004730 comp rx=0 tx=0).stop 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 shutdown_connections 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f488406c8f0 0x7f488406edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a80c9870 0x7f48a814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a80c9cf0 0x7f48a814db20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 --2- 172.21.15.59:0/2823392031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488c004d40 0x7f48a80ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.614+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 >> 172.21.15.59:0/2823392031 conn(0x7f48a808bf90 msgr2=0x7f48a806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.615+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 shutdown_connections 2024-04-23T14:31:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.615+0000 7f48a7200640 1 -- 172.21.15.59:0/2823392031 wait complete. 2024-04-23T14:31:38.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:31:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.841+0000 7f1643200640 1 Processor -- start 2024-04-23T14:31:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.842+0000 7f1643200640 1 -- start start 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.843+0000 7f1643200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f164415ed20 0x7f16440d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.843+0000 7f1643200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f16440b3600 0x7f16440d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.843+0000 7f1643200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16440b5da0 con 0x7f16440b3f10 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.843+0000 7f1643200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f164415f100 con 0x7f16440b3600 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.843+0000 7f1643200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f16440d0a20 con 0x7f164415ed20 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.843+0000 7f1641400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f16440b3600 0x7f16440d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.844+0000 7f1641400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f16440b3600 0x7f16440d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33654/0 (socket says 172.21.15.59:33654) 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.844+0000 7f1641400640 1 -- 172.21.15.59:0/2076290419 learned_addr learned my addr 172.21.15.59:0/2076290419 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.844+0000 7f1641e00640 1 --2- 172.21.15.59:0/2076290419 >> v2:172.21.15.76:3300/0 conn(0x7f164415ed20 0x7f16440d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.844+0000 7f1641400640 1 -- 172.21.15.59:0/2076290419 >> v2:172.21.15.76:3300/0 conn(0x7f164415ed20 msgr2=0x7f16440d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.844+0000 7f1641400640 1 --2- 172.21.15.59:0/2076290419 >> v2:172.21.15.76:3300/0 conn(0x7f164415ed20 0x7f16440d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.844+0000 7f1641400640 1 -- 172.21.15.59:0/2076290419 >> v1:172.21.15.76:6789/0 conn(0x7f16440b3f10 legacy=0x7f16440b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.844+0000 7f1641400640 1 -- 172.21.15.59:0/2076290419 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16440d2ed0 con 0x7f16440b3600 2024-04-23T14:31:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.845+0000 7f1641400640 1 --2- 172.21.15.59:0/2076290419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16440b3600 0x7f16440d13e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f164408e2a0 tx=0x7f163002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c92e11acbd9bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.845+0000 7f1640a00640 1 -- 172.21.15.59:0/2076290419 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630037070 con 0x7f16440b3600 2024-04-23T14:31:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.845+0000 7f1640a00640 1 -- 172.21.15.59:0/2076290419 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163002ae20 con 0x7f16440b3600 2024-04-23T14:31:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.845+0000 7f1640a00640 1 -- 172.21.15.59:0/2076290419 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630032660 con 0x7f16440b3600 2024-04-23T14:31:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 -- 172.21.15.59:0/2076290419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16440b3600 msgr2=0x7f16440d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 --2- 172.21.15.59:0/2076290419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16440b3600 0x7f16440d13e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f164408e2a0 tx=0x7f163002a110 comp rx=0 tx=0).stop 2024-04-23T14:31:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 -- 172.21.15.59:0/2076290419 shutdown_connections 2024-04-23T14:31:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 --2- 172.21.15.59:0/2076290419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16440b3600 0x7f16440d13e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 --2- 172.21.15.59:0/2076290419 >> v2:172.21.15.76:3300/0 conn(0x7f164415ed20 0x7f16440d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 -- 172.21.15.59:0/2076290419 >> 172.21.15.59:0/2076290419 conn(0x7f164408bf90 msgr2=0x7f16440d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 -- 172.21.15.59:0/2076290419 shutdown_connections 2024-04-23T14:31:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.846+0000 7f1643200640 1 -- 172.21.15.59:0/2076290419 wait complete. 2024-04-23T14:31:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.848+0000 7f1643200640 1 Processor -- start 2024-04-23T14:31:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.848+0000 7f1643200640 1 -- start start 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1643200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16440b3600 0x7f16441ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1643200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16440b3f10 0x7f16440b9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1643200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f164415ed20 0x7f16440b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1643200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16440d0290 con 0x7f164415ed20 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1643200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f16440d3490 con 0x7f16440b3f10 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1643200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f16440d0410 con 0x7f16440b3600 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1642800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f164415ed20 0x7f16440b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1642800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f164415ed20 0x7f16440b9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33666/0 (socket says 172.21.15.59:33666) 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1642800640 1 -- 172.21.15.59:0/1518908739 learned_addr learned my addr 172.21.15.59:0/1518908739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1641e00640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16440b3600 0x7f16441ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1641400640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16440b3f10 0x7f16440b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1642800640 1 -- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16440b3600 msgr2=0x7f16441ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.849+0000 7f1642800640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16440b3600 0x7f16441ee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f1642800640 1 -- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16440b3f10 msgr2=0x7f16440b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f1642800640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16440b3f10 0x7f16440b9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f1642800640 1 -- 172.21.15.59:0/1518908739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16441f4710 con 0x7f164415ed20 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f1641400640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16440b3f10 0x7f16440b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f1642800640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f164415ed20 0x7f16440b9b60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c780 tx=0x7f163c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f163a000640 1 -- 172.21.15.59:0/1518908739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c004b60 con 0x7f164415ed20 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f163a000640 1 -- 172.21.15.59:0/1518908739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c004d00 con 0x7f164415ed20 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f163a000640 1 -- 172.21.15.59:0/1518908739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c00aa90 con 0x7f164415ed20 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16441f49d0 con 0x7f164415ed20 2024-04-23T14:31:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.850+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16441f4f90 con 0x7f164415ed20 2024-04-23T14:31:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.851+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164404fd90 con 0x7f164415ed20 2024-04-23T14:31:38.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.852+0000 7f163a000640 1 -- 172.21.15.59:0/1518908739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f163c00ac30 con 0x7f164415ed20 2024-04-23T14:31:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.853+0000 7f163a000640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f162406c8f0 0x7f162406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.853+0000 7f163a000640 1 -- 172.21.15.59:0/1518908739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f163c08be60 con 0x7f164415ed20 2024-04-23T14:31:38.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.854+0000 7f1641e00640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f162406c8f0 0x7f162406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:38.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.854+0000 7f1641e00640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f162406c8f0 0x7f162406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1634006fd0 tx=0x7f1634008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:38.858+0000 7f163a000640 1 -- 172.21.15.59:0/1518908739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f163c020070 con 0x7f164415ed20 2024-04-23T14:31:39.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.054+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f16440d1360 con 0x7f164415ed20 2024-04-23T14:31:39.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.055+0000 7f163a000640 1 -- 172.21.15.59:0/1518908739 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f163c05a280 con 0x7f164415ed20 2024-04-23T14:31:39.056 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:31:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f162406c8f0 msgr2=0x7f162406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f162406c8f0 0x7f162406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1634006fd0 tx=0x7f1634008040 comp rx=0 tx=0).stop 2024-04-23T14:31:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f164415ed20 msgr2=0x7f16440b9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f164415ed20 0x7f16440b9b60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c780 tx=0x7f163c00cc40 comp rx=0 tx=0).stop 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 shutdown_connections 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f162406c8f0 0x7f162406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f164415ed20 0x7f16440b9b60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16440b3f10 0x7f16440b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 --2- 172.21.15.59:0/1518908739 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16440b3600 0x7f16441ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 >> 172.21.15.59:0/1518908739 conn(0x7f164408bf90 msgr2=0x7f16440b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 shutdown_connections 2024-04-23T14:31:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.057+0000 7f1643200640 1 -- 172.21.15.59:0/1518908739 wait complete. 2024-04-23T14:31:39.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:31:39.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.286+0000 7f2d2fc00640 1 Processor -- start 2024-04-23T14:31:39.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.286+0000 7f2d2fc00640 1 -- start start 2024-04-23T14:31:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.288+0000 7f2d2fc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d281454e0 0x7f2d28145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.288+0000 7f2d2fc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d280cd020 0x7f2d280b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.288+0000 7f2d2fc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d280cd520 con 0x7f2d280b3e40 2024-04-23T14:31:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.288+0000 7f2d2fc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2d280cddc0 con 0x7f2d281454e0 2024-04-23T14:31:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.288+0000 7f2d2fc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2d280cd800 con 0x7f2d280cd020 2024-04-23T14:31:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.288+0000 7f2d2d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d281454e0 0x7f2d28145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d2d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d281454e0 0x7f2d28145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33682/0 (socket says 172.21.15.59:33682) 2024-04-23T14:31:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d2d200640 1 -- 172.21.15.59:0/2900348459 learned_addr learned my addr 172.21.15.59:0/2900348459 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d2dc00640 1 --1- 172.21.15.59:0/2900348459 >> v1:172.21.15.59:6789/0 conn(0x7f2d280b3e40 0x7f2d28160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36630/0 (socket says 172.21.15.59:36630) 2024-04-23T14:31:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d27e00640 1 --2- 172.21.15.59:0/2900348459 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d280cd020 0x7f2d280b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d2d200640 1 -- 172.21.15.59:0/2900348459 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d280cd020 msgr2=0x7f2d280b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d2d200640 1 --2- 172.21.15.59:0/2900348459 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d280cd020 0x7f2d280b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d2d200640 1 -- 172.21.15.59:0/2900348459 >> v1:172.21.15.59:6789/0 conn(0x7f2d280b3e40 legacy=0x7f2d28160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d2d200640 1 -- 172.21.15.59:0/2900348459 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d280cdae0 con 0x7f2d281454e0 2024-04-23T14:31:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.289+0000 7f2d27400640 1 -- 172.21.15.59:0/2900348459 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678900208 0 0) 0x7f2d280cd520 con 0x7f2d280b3e40 2024-04-23T14:31:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.291+0000 7f2d2d200640 1 --2- 172.21.15.59:0/2900348459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d281454e0 0x7f2d28145bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009870 tx=0x7f2d1c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44384ff589330a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.291+0000 7f2d27400640 1 -- 172.21.15.59:0/2900348459 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c037070 con 0x7f2d281454e0 2024-04-23T14:31:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.291+0000 7f2d27400640 1 -- 172.21.15.59:0/2900348459 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1c032440 con 0x7f2d281454e0 2024-04-23T14:31:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.291+0000 7f2d27400640 1 -- 172.21.15.59:0/2900348459 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c032810 con 0x7f2d281454e0 2024-04-23T14:31:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2900348459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d281454e0 msgr2=0x7f2d28145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2900348459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d281454e0 0x7f2d28145bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009870 tx=0x7f2d1c02a1c0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2900348459 shutdown_connections 2024-04-23T14:31:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2900348459 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d280cd020 0x7f2d280b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2900348459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d281454e0 0x7f2d28145bb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2900348459 >> 172.21.15.59:0/2900348459 conn(0x7f2d28067ea0 msgr2=0x7f2d2806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2900348459 shutdown_connections 2024-04-23T14:31:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.292+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2900348459 wait complete. 2024-04-23T14:31:39.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.294+0000 7f2d2fc00640 1 Processor -- start 2024-04-23T14:31:39.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.294+0000 7f2d2fc00640 1 -- start start 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2fc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d280b3e40 0x7f2d28140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2fc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d280cd020 0x7f2d2806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2fc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d281454e0 0x7f2d2806f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2fc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28146270 con 0x7f2d280b3e40 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2fc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281460f0 con 0x7f2d280cd020 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2fc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2d28161020 con 0x7f2d281454e0 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d280b3e40 0x7f2d28140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d280b3e40 0x7f2d28140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33694/0 (socket says 172.21.15.59:33694) 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.295+0000 7f2d2d200640 1 -- 172.21.15.59:0/2806613357 learned_addr learned my addr 172.21.15.59:0/2806613357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2dc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d281454e0 0x7f2d2806f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2d200640 1 -- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d281454e0 msgr2=0x7f2d2806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2d200640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d281454e0 0x7f2d2806f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2d200640 1 -- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d280cd020 msgr2=0x7f2d2806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d27e00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d280cd020 0x7f2d2806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2d200640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d280cd020 0x7f2d2806f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2d200640 1 -- 172.21.15.59:0/2806613357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d28143d00 con 0x7f2d280b3e40 2024-04-23T14:31:39.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2dc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d281454e0 0x7f2d2806f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:39.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2d200640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d280b3e40 0x7f2d28140da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c035040 tx=0x7f2d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d24c00640 1 -- 172.21.15.59:0/2806613357 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c037040 con 0x7f2d280b3e40 2024-04-23T14:31:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d24c00640 1 -- 172.21.15.59:0/2806613357 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1c004230 con 0x7f2d280b3e40 2024-04-23T14:31:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d28143fe0 con 0x7f2d280b3e40 2024-04-23T14:31:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.296+0000 7f2d24c00640 1 -- 172.21.15.59:0/2806613357 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c003610 con 0x7f2d280b3e40 2024-04-23T14:31:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.297+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281445a0 con 0x7f2d280b3e40 2024-04-23T14:31:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.298+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d28068a80 con 0x7f2d280b3e40 2024-04-23T14:31:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.299+0000 7f2d24c00640 1 -- 172.21.15.59:0/2806613357 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d1c02ad10 con 0x7f2d280b3e40 2024-04-23T14:31:39.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.300+0000 7f2d24c00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d0806c8f0 0x7f2d0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.300+0000 7f2d24c00640 1 -- 172.21.15.59:0/2806613357 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d1c0ac720 con 0x7f2d280b3e40 2024-04-23T14:31:39.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.300+0000 7f2d27e00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d0806c8f0 0x7f2d0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.301+0000 7f2d27e00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d0806c8f0 0x7f2d0806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d2806fe30 tx=0x7f2d14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.304+0000 7f2d24c00640 1 -- 172.21.15.59:0/2806613357 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d1c07aab0 con 0x7f2d280b3e40 2024-04-23T14:31:39.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.551+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d28160030 con 0x7f2d280b3e40 2024-04-23T14:31:39.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.553+0000 7f2d24c00640 1 -- 172.21.15.59:0/2806613357 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f2d1c07a3b0 con 0x7f2d280b3e40 2024-04-23T14:31:39.554 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:39.554 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:31:39.554 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.555+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d0806c8f0 msgr2=0x7f2d0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.555+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d0806c8f0 0x7f2d0806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d2806fe30 tx=0x7f2d14008040 comp rx=0 tx=0).stop 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.555+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d280b3e40 msgr2=0x7f2d28140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.555+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d280b3e40 0x7f2d28140da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c035040 tx=0x7f2d1c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.555+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 shutdown_connections 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.555+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d0806c8f0 0x7f2d0806edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.556+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d281454e0 0x7f2d2806f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.556+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d280cd020 0x7f2d2806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.556+0000 7f2d2fc00640 1 --2- 172.21.15.59:0/2806613357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d280b3e40 0x7f2d28140da0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.556+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 >> 172.21.15.59:0/2806613357 conn(0x7f2d28067ea0 msgr2=0x7f2d280b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.556+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 shutdown_connections 2024-04-23T14:31:39.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.556+0000 7f2d2fc00640 1 -- 172.21.15.59:0/2806613357 wait complete. 2024-04-23T14:31:39.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.784+0000 7f8a63200640 1 Processor -- start 2024-04-23T14:31:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.785+0000 7f8a63200640 1 -- start start 2024-04-23T14:31:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a63200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a6415c3d0 0x7f8a640cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a63200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a640cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a63200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a64160210 con 0x7f8a640cd590 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a63200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8a641604f0 con 0x7f8a6415c3d0 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a63200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8a641607d0 con 0x7f8a640b4860 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a61e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a6415c3d0 0x7f8a640cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a61e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a6415c3d0 0x7f8a640cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33708/0 (socket says 172.21.15.59:33708) 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.786+0000 7f8a61e00640 1 -- 172.21.15.59:0/843075633 learned_addr learned my addr 172.21.15.59:0/843075633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.787+0000 7f8a62800640 1 --2- 172.21.15.59:0/843075633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a640cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.787+0000 7f8a61e00640 1 -- 172.21.15.59:0/843075633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 msgr2=0x7f8a640cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.787+0000 7f8a61e00640 1 --2- 172.21.15.59:0/843075633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a640cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.787+0000 7f8a61e00640 1 -- 172.21.15.59:0/843075633 >> v1:172.21.15.76:6789/0 conn(0x7f8a640cd590 legacy=0x7f8a640b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:39.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.787+0000 7f8a61e00640 1 -- 172.21.15.59:0/843075633 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a64160ab0 con 0x7f8a6415c3d0 2024-04-23T14:31:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.788+0000 7f8a61e00640 1 --2- 172.21.15.59:0/843075633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a640cd050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009e50 tx=0x7f8a5402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc1505c3a031010 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.788+0000 7f8a61400640 1 -- 172.21.15.59:0/843075633 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54036070 con 0x7f8a6415c3d0 2024-04-23T14:31:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.788+0000 7f8a61400640 1 -- 172.21.15.59:0/843075633 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a54031600 con 0x7f8a6415c3d0 2024-04-23T14:31:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.788+0000 7f8a61400640 1 -- 172.21.15.59:0/843075633 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a540319d0 con 0x7f8a6415c3d0 2024-04-23T14:31:39.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 -- 172.21.15.59:0/843075633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 msgr2=0x7f8a640cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 --2- 172.21.15.59:0/843075633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a640cd050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009e50 tx=0x7f8a5402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 -- 172.21.15.59:0/843075633 shutdown_connections 2024-04-23T14:31:39.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 --2- 172.21.15.59:0/843075633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a640cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 --2- 172.21.15.59:0/843075633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a640cd050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 -- 172.21.15.59:0/843075633 >> 172.21.15.59:0/843075633 conn(0x7f8a640b21a0 msgr2=0x7f8a640b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:39.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 -- 172.21.15.59:0/843075633 shutdown_connections 2024-04-23T14:31:39.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.789+0000 7f8a63200640 1 -- 172.21.15.59:0/843075633 wait complete. 2024-04-23T14:31:39.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.791+0000 7f8a63200640 1 Processor -- start 2024-04-23T14:31:39.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.791+0000 7f8a63200640 1 -- start start 2024-04-23T14:31:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.792+0000 7f8a63200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a6415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.792+0000 7f8a63200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cd590 0x7f8a6415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.792+0000 7f8a63200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a6415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.792+0000 7f8a63200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6415c930 con 0x7f8a6415c3d0 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.792+0000 7f8a63200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6415c7b0 con 0x7f8a640cd590 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.792+0000 7f8a63200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8a640cbd10 con 0x7f8a640b4860 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a6415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a6415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33722/0 (socket says 172.21.15.59:33722) 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 -- 172.21.15.59:0/3713583509 learned_addr learned my addr 172.21.15.59:0/3713583509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a61e00640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a6415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a59400640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cd590 0x7f8a6415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 -- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 msgr2=0x7f8a6415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a6415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 -- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cd590 msgr2=0x7f8a6415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cd590 0x7f8a6415f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 -- 172.21.15.59:0/3713583509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a641f58b0 con 0x7f8a6415c3d0 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a59400640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cd590 0x7f8a6415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.793+0000 7f8a62800640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a6415f800 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d800 tx=0x7f8a5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.794+0000 7f8a5aa00640 1 -- 172.21.15.59:0/3713583509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00b6c0 con 0x7f8a6415c3d0 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.794+0000 7f8a5aa00640 1 -- 172.21.15.59:0/3713583509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c019070 con 0x7f8a6415c3d0 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.794+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6415b700 con 0x7f8a6415c3d0 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.794+0000 7f8a5aa00640 1 -- 172.21.15.59:0/3713583509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00bac0 con 0x7f8a6415c3d0 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.794+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641f83c0 con 0x7f8a6415c3d0 2024-04-23T14:31:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.795+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6404fd90 con 0x7f8a6415c3d0 2024-04-23T14:31:39.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.796+0000 7f8a5aa00640 1 -- 172.21.15.59:0/3713583509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a5c00bc60 con 0x7f8a6415c3d0 2024-04-23T14:31:39.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.797+0000 7f8a5aa00640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:39.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.797+0000 7f8a5aa00640 1 -- 172.21.15.59:0/3713583509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a5c08c060 con 0x7f8a6415c3d0 2024-04-23T14:31:39.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.797+0000 7f8a61e00640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:39.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.798+0000 7f8a61e00640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8a4006c8f0 0x7f8a4006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a54033000 tx=0x7f8a54009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:39.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:39.801+0000 7f8a5aa00640 1 -- 172.21.15.59:0/3713583509 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a5c05a3f0 con 0x7f8a6415c3d0 2024-04-23T14:31:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.007+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a6406e400 con 0x7f8a6415c3d0 2024-04-23T14:31:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.010+0000 7f8a5aa00640 1 -- 172.21.15.59:0/3713583509 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f8a5c059cf0 con 0x7f8a6415c3d0 2024-04-23T14:31:40.011 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:40.011 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.012+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8a4006c8f0 msgr2=0x7f8a4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.012+0000 7f8a63200640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8a4006c8f0 0x7f8a4006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a54033000 tx=0x7f8a54009000 comp rx=0 tx=0).stop 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.012+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 msgr2=0x7f8a6415f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.012+0000 7f8a63200640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a6415f800 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d800 tx=0x7f8a5c00dcc0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 shutdown_connections 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6415c3d0 0x7f8a6415f800 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cd590 0x7f8a6415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 --2- 172.21.15.59:0/3713583509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a640b4860 0x7f8a6415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 >> 172.21.15.59:0/3713583509 conn(0x7f8a640b21a0 msgr2=0x7f8a640b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 shutdown_connections 2024-04-23T14:31:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.013+0000 7f8a63200640 1 -- 172.21.15.59:0/3713583509 wait complete. 2024-04-23T14:31:40.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:31:40.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:31:40.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.251+0000 7f76e7000640 1 Processor -- start 2024-04-23T14:31:40.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.251+0000 7f76e7000640 1 -- start start 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.253+0000 7f76e7000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f76e00b08f0 0x7f76e00c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.253+0000 7f76e7000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f76e00ce850 0x7f76e015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.253+0000 7f76e7000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e00b0cd0 con 0x7f76e00b3a50 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.253+0000 7f76e7000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f76e00cdcb0 con 0x7f76e00b08f0 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.253+0000 7f76e7000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f76e0070da0 con 0x7f76e00ce850 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e5c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f76e00b08f0 0x7f76e00c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e5c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f76e00b08f0 0x7f76e00c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47748/0 (socket says 172.21.15.59:47748) 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e5c00640 1 -- 172.21.15.59:0/2944037533 learned_addr learned my addr 172.21.15.59:0/2944037533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:40.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e6600640 1 --2- 172.21.15.59:0/2944037533 >> v2:172.21.15.76:3300/0 conn(0x7f76e00ce850 0x7f76e015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e5c00640 1 -- 172.21.15.59:0/2944037533 >> v2:172.21.15.76:3300/0 conn(0x7f76e00ce850 msgr2=0x7f76e015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e5c00640 1 --2- 172.21.15.59:0/2944037533 >> v2:172.21.15.76:3300/0 conn(0x7f76e00ce850 0x7f76e015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e5c00640 1 -- 172.21.15.59:0/2944037533 >> v1:172.21.15.59:6789/0 conn(0x7f76e00b3a50 legacy=0x7f76e00ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.254+0000 7f76e5c00640 1 -- 172.21.15.59:0/2944037533 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e00707e0 con 0x7f76e00b08f0 2024-04-23T14:31:40.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.255+0000 7f76e5c00640 1 --2- 172.21.15.59:0/2944037533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b08f0 0x7f76e00c61a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76e00b74d0 tx=0x7f76d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a89fe7cc93b1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:40.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.255+0000 7f76dfe00640 1 -- 172.21.15.59:0/2944037533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4037070 con 0x7f76e00b08f0 2024-04-23T14:31:40.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76dfe00640 1 -- 172.21.15.59:0/2944037533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d402ae10 con 0x7f76e00b08f0 2024-04-23T14:31:40.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76dfe00640 1 -- 172.21.15.59:0/2944037533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4032660 con 0x7f76e00b08f0 2024-04-23T14:31:40.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 -- 172.21.15.59:0/2944037533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b08f0 msgr2=0x7f76e00c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 --2- 172.21.15.59:0/2944037533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b08f0 0x7f76e00c61a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76e00b74d0 tx=0x7f76d402a180 comp rx=0 tx=0).stop 2024-04-23T14:31:40.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 -- 172.21.15.59:0/2944037533 shutdown_connections 2024-04-23T14:31:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 --2- 172.21.15.59:0/2944037533 >> v2:172.21.15.76:3300/0 conn(0x7f76e00ce850 0x7f76e015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 --2- 172.21.15.59:0/2944037533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b08f0 0x7f76e00c61a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 -- 172.21.15.59:0/2944037533 >> 172.21.15.59:0/2944037533 conn(0x7f76e00b2f30 msgr2=0x7f76e00b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 -- 172.21.15.59:0/2944037533 shutdown_connections 2024-04-23T14:31:40.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.256+0000 7f76e7000640 1 -- 172.21.15.59:0/2944037533 wait complete. 2024-04-23T14:31:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.258+0000 7f76e7000640 1 Processor -- start 2024-04-23T14:31:40.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.258+0000 7f76e7000640 1 -- start start 2024-04-23T14:31:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e7000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00b08f0 0x7f76e0161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e7000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b3a50 0x7f76e01547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e7000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f76e00ce850 0x7f76e00ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e7000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76e0070420 con 0x7f76e00b3a50 2024-04-23T14:31:40.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e7000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f76e00d1000 con 0x7f76e00b08f0 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e7000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f76e00705a0 con 0x7f76e00ce850 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b3a50 0x7f76e01547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b3a50 0x7f76e01547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47760/0 (socket says 172.21.15.59:47760) 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.259+0000 7f76e5200640 1 -- 172.21.15.59:0/3552613301 learned_addr learned my addr 172.21.15.59:0/3552613301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e5c00640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00b08f0 0x7f76e0161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e5200640 1 -- 172.21.15.59:0/3552613301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f76e00ce850 msgr2=0x7f76e00ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e5200640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f76e00ce850 0x7f76e00ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e5200640 1 -- 172.21.15.59:0/3552613301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00b08f0 msgr2=0x7f76e0161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e5200640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00b08f0 0x7f76e0161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e5200640 1 -- 172.21.15.59:0/3552613301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e00c9560 con 0x7f76e00b3a50 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e5200640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b3a50 0x7f76e01547d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c780 tx=0x7f76cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76dd600640 1 -- 172.21.15.59:0/3552613301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc017070 con 0x7f76e00b3a50 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76dd600640 1 -- 172.21.15.59:0/3552613301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc005290 con 0x7f76e00b3a50 2024-04-23T14:31:40.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76dd600640 1 -- 172.21.15.59:0/3552613301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00a880 con 0x7f76e00b3a50 2024-04-23T14:31:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.260+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e00c9840 con 0x7f76e00b3a50 2024-04-23T14:31:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.261+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e00c9e00 con 0x7f76e00b3a50 2024-04-23T14:31:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.262+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e004fd90 con 0x7f76e00b3a50 2024-04-23T14:31:40.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.263+0000 7f76dd600640 1 -- 172.21.15.59:0/3552613301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76cc01a070 con 0x7f76e00b3a50 2024-04-23T14:31:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.263+0000 7f76dd600640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f76c006c940 0x7f76c006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.264+0000 7f76dd600640 1 -- 172.21.15.59:0/3552613301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f76cc08b0c0 con 0x7f76e00b3a50 2024-04-23T14:31:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.264+0000 7f76e5c00640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f76c006c940 0x7f76c006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.264+0000 7f76e5c00640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f76c006c940 0x7f76c006ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f76d4034000 tx=0x7f76d402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:40.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.268+0000 7f76dd600640 1 -- 172.21.15.59:0/3552613301 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76cc0594d0 con 0x7f76e00b3a50 2024-04-23T14:31:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.472+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76e00cc3f0 con 0x7f76e00b3a50 2024-04-23T14:31:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.473+0000 7f76dd600640 1 -- 172.21.15.59:0/3552613301 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f76cc058dd0 con 0x7f76e00b3a50 2024-04-23T14:31:40.473 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:31:40.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f76c006c940 msgr2=0x7f76c006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f76c006c940 0x7f76c006ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f76d4034000 tx=0x7f76d402e000 comp rx=0 tx=0).stop 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b3a50 msgr2=0x7f76e01547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b3a50 0x7f76e01547d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c780 tx=0x7f76cc00cc40 comp rx=0 tx=0).stop 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 shutdown_connections 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f76c006c940 0x7f76c006ee00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f76e00ce850 0x7f76e00ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76e00b3a50 0x7f76e01547d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 --2- 172.21.15.59:0/3552613301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00b08f0 0x7f76e0161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 >> 172.21.15.59:0/3552613301 conn(0x7f76e00b2f30 msgr2=0x7f76e00cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:40.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.475+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 shutdown_connections 2024-04-23T14:31:40.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.476+0000 7f76e7000640 1 -- 172.21.15.59:0/3552613301 wait complete. 2024-04-23T14:31:40.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:31:40.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.699+0000 7fe505800640 1 Processor -- start 2024-04-23T14:31:40.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.699+0000 7fe505800640 1 -- start start 2024-04-23T14:31:40.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe505800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5001567e0 0x7fe500156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe505800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5000d0d10 0x7fe500160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe505800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe50015cab0 con 0x7fe50015d810 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe505800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe500160e40 con 0x7fe5000d0d10 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe505800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe50015cd30 con 0x7fe5001567e0 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe504e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5000d0d10 0x7fe500160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe504e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5000d0d10 0x7fe500160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47770/0 (socket says 172.21.15.59:47770) 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.701+0000 7fe504e00640 1 -- 172.21.15.59:0/3950674982 learned_addr learned my addr 172.21.15.59:0/3950674982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.702+0000 7fe4ffe00640 1 --2- 172.21.15.59:0/3950674982 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5001567e0 0x7fe500156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.702+0000 7fe504e00640 1 -- 172.21.15.59:0/3950674982 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5001567e0 msgr2=0x7fe500156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.702+0000 7fe504e00640 1 --2- 172.21.15.59:0/3950674982 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5001567e0 0x7fe500156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.702+0000 7fe504e00640 1 -- 172.21.15.59:0/3950674982 >> v1:172.21.15.76:6789/0 conn(0x7fe50015d810 legacy=0x7fe5000d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:40.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.702+0000 7fe504e00640 1 -- 172.21.15.59:0/3950674982 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50015d010 con 0x7fe5000d0d10 2024-04-23T14:31:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.703+0000 7fe504e00640 1 --2- 172.21.15.59:0/3950674982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe500160110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009e50 tx=0x7fe4f802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6735230406f90a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.703+0000 7fe4fea00640 1 -- 172.21.15.59:0/3950674982 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8036070 con 0x7fe5000d0d10 2024-04-23T14:31:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.703+0000 7fe4fea00640 1 -- 172.21.15.59:0/3950674982 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f8031600 con 0x7fe5000d0d10 2024-04-23T14:31:40.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.703+0000 7fe4fea00640 1 -- 172.21.15.59:0/3950674982 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f80319d0 con 0x7fe5000d0d10 2024-04-23T14:31:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 -- 172.21.15.59:0/3950674982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 msgr2=0x7fe500160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 --2- 172.21.15.59:0/3950674982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe500160110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009e50 tx=0x7fe4f802a6c0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 -- 172.21.15.59:0/3950674982 shutdown_connections 2024-04-23T14:31:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 --2- 172.21.15.59:0/3950674982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe500160110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 --2- 172.21.15.59:0/3950674982 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5001567e0 0x7fe500156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 -- 172.21.15.59:0/3950674982 >> 172.21.15.59:0/3950674982 conn(0x7fe50006bdf0 msgr2=0x7fe50006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 -- 172.21.15.59:0/3950674982 shutdown_connections 2024-04-23T14:31:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.704+0000 7fe505800640 1 -- 172.21.15.59:0/3950674982 wait complete. 2024-04-23T14:31:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.707+0000 7fe505800640 1 Processor -- start 2024-04-23T14:31:40.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.707+0000 7fe505800640 1 -- start start 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.708+0000 7fe505800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe5001f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.708+0000 7fe505800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5001567e0 0x7fe50014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.708+0000 7fe505800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe50015d810 0x7fe500153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.708+0000 7fe505800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001607d0 con 0x7fe5000d0d10 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.708+0000 7fe505800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe500160650 con 0x7fe5001567e0 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.708+0000 7fe505800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe500160950 con 0x7fe50015d810 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe5001f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe5001f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47780/0 (socket says 172.21.15.59:47780) 2024-04-23T14:31:40.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 -- 172.21.15.59:0/3326958046 learned_addr learned my addr 172.21.15.59:0/3326958046 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ff400640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5001567e0 0x7fe50014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe504e00640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe50015d810 0x7fe500153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 -- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe50015d810 msgr2=0x7fe500153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe50015d810 0x7fe500153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 -- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5001567e0 msgr2=0x7fe50014e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5001567e0 0x7fe50014e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 -- 172.21.15.59:0/3326958046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe500072ba0 con 0x7fe5000d0d10 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.709+0000 7fe4ffe00640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe5001f89a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000d800 tx=0x7fe4f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.710+0000 7fe4f7e00640 1 -- 172.21.15.59:0/3326958046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000b6c0 con 0x7fe5000d0d10 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.710+0000 7fe4f7e00640 1 -- 172.21.15.59:0/3326958046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f0019070 con 0x7fe5000d0d10 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.710+0000 7fe4f7e00640 1 -- 172.21.15.59:0/3326958046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000ba80 con 0x7fe5000d0d10 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.710+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe500072e80 con 0x7fe5000d0d10 2024-04-23T14:31:40.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.710+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe500073440 con 0x7fe5000d0d10 2024-04-23T14:31:40.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.711+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50004fd90 con 0x7fe5000d0d10 2024-04-23T14:31:40.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.712+0000 7fe4f7e00640 1 -- 172.21.15.59:0/3326958046 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4f000bc20 con 0x7fe5000d0d10 2024-04-23T14:31:40.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.713+0000 7fe4f7e00640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:40.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.713+0000 7fe4f7e00640 1 -- 172.21.15.59:0/3326958046 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe4f008c0e0 con 0x7fe5000d0d10 2024-04-23T14:31:40.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.713+0000 7fe4ff400640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:40.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.714+0000 7fe4ff400640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe5000d0c90 tx=0x7fe4ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:40.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.717+0000 7fe4f7e00640 1 -- 172.21.15.59:0/3326958046 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4f001c360 con 0x7fe5000d0d10 2024-04-23T14:31:40.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.923+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5000b5150 con 0x7fe5000d0d10 2024-04-23T14:31:40.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.924+0000 7fe4f7e00640 1 -- 172.21.15.59:0/3326958046 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fe4f005a470 con 0x7fe5000d0d10 2024-04-23T14:31:40.925 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:31:40.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe4e006c8f0 msgr2=0x7fe4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe5000d0c90 tx=0x7fe4ec009040 comp rx=0 tx=0).stop 2024-04-23T14:31:40.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 msgr2=0x7fe5001f89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:40.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe5001f89a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000d800 tx=0x7fe4f000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 shutdown_connections 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe50015d810 0x7fe500153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5001567e0 0x7fe50014e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 --2- 172.21.15.59:0/3326958046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5000d0d10 0x7fe5001f89a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 >> 172.21.15.59:0/3326958046 conn(0x7fe50006bdf0 msgr2=0x7fe50015ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 shutdown_connections 2024-04-23T14:31:40.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:40.927+0000 7fe505800640 1 -- 172.21.15.59:0/3326958046 wait complete. 2024-04-23T14:31:40.949 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:31:41.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.164+0000 7fda7b200640 1 Processor -- start 2024-04-23T14:31:41.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.164+0000 7fda7b200640 1 -- start start 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.166+0000 7fda7b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda7c0d15b0 0x7fda7c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.166+0000 7fda7b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda7c0b5da0 con 0x7fda7c0b4f20 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.166+0000 7fda7b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda7c0d0a20 con 0x7fda7c15ed20 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.166+0000 7fda7b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c15f100 con 0x7fda7c0d15b0 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.166+0000 7fda7a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fda7c0b4f20 0x7fda7c0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47602/0 (socket says 172.21.15.59:47602) 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.166+0000 7fda7a800640 1 -- 172.21.15.59:0/3208582219 learned_addr learned my addr 172.21.15.59:0/3208582219 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda71400640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda7c0d15b0 0x7fda7c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515846030 0 0) 0x7fda7c0b5da0 con 0x7fda7c0b4f20 2024-04-23T14:31:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda60003520 con 0x7fda7c0b4f20 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 256017408 0 0) 0x7fda60003520 con 0x7fda7c0b4f20 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda7c0d15b0 msgr2=0x7fda7c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda7c0d15b0 0x7fda7c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 >> v1:172.21.15.76:6789/0 conn(0x7fda7c15ed20 legacy=0x7fda7c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c0d2ed0 con 0x7fda7c0b4f20 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.167+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fda74001670 con 0x7fda7c0b4f20 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 >> v1:172.21.15.59:6789/0 conn(0x7fda7c0b4f20 legacy=0x7fda7c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79400640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda600039e0 0x7fda60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79400640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda60004800 0x7fda60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79400640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda7c0d15b0 0x7fda60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda6000bac0 con 0x7fda600039e0 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda6000bc70 con 0x7fda60004800 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda6000be70 con 0x7fda7c0d15b0 2024-04-23T14:31:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79e00640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda600039e0 0x7fda60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda71400640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda60004800 0x7fda60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79e00640 1 -- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda7c0d15b0 msgr2=0x7fda60005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79e00640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda7c0d15b0 0x7fda60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79e00640 1 -- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda60004800 msgr2=0x7fda60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79e00640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda60004800 0x7fda60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.168+0000 7fda79e00640 1 -- 172.21.15.59:0/3208582219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda60003520 con 0x7fda600039e0 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.170+0000 7fda79e00640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda600039e0 0x7fda60003dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fda6c001f60 tx=0x7fda60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85aa10b2a9f40b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.170+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c001b30 con 0x7fda600039e0 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.170+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c031620 con 0x7fda600039e0 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.170+0000 7fda79400640 1 -- 172.21.15.59:0/3208582219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c031a10 con 0x7fda600039e0 2024-04-23T14:31:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.170+0000 7fda7b200640 1 -- 172.21.15.59:0/3208582219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda600039e0 msgr2=0x7fda60003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.170+0000 7fda7b200640 1 --2- 172.21.15.59:0/3208582219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda600039e0 0x7fda60003dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fda6c001f60 tx=0x7fda60005830 comp rx=0 tx=0).stop 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.170+0000 7fda7a800640 1 -- 172.21.15.59:0/3208582219 reap_dead start 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.171+0000 7fda7b200640 1 -- 172.21.15.59:0/3208582219 shutdown_connections 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.171+0000 7fda7b200640 1 -- 172.21.15.59:0/3208582219 >> 172.21.15.59:0/3208582219 conn(0x7fda7c08bf90 msgr2=0x7fda7c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.171+0000 7fda7b200640 1 -- 172.21.15.59:0/3208582219 shutdown_connections 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.171+0000 7fda7b200640 1 -- 172.21.15.59:0/3208582219 wait complete. 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.172+0000 7fda7b200640 1 Processor -- start 2024-04-23T14:31:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.173+0000 7fda7b200640 1 -- start start 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.173+0000 7fda7b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda600039e0 0x7fda7c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.173+0000 7fda7b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda7c160540 0x7fda7c0b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.173+0000 7fda7b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c1609c0 0x7fda7c0d4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda7b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0b3600 con 0x7fda7c160540 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda7b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0d3490 con 0x7fda7c1609c0 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda7b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c0b3780 con 0x7fda600039e0 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda7c160540 0x7fda7c0b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda7c160540 0x7fda7c0b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47794/0 (socket says 172.21.15.59:47794) 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 -- 172.21.15.59:0/1714282845 learned_addr learned my addr 172.21.15.59:0/1714282845 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:41.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda79e00640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda600039e0 0x7fda7c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda7a800640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c1609c0 0x7fda7c0d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 -- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda600039e0 msgr2=0x7fda7c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda600039e0 0x7fda7c157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 -- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c1609c0 msgr2=0x7fda7c0d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c1609c0 0x7fda7c0d4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.174+0000 7fda71400640 1 -- 172.21.15.59:0/1714282845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c0b1c10 con 0x7fda7c160540 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.175+0000 7fda71400640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda7c160540 0x7fda7c0b1190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda64011600 tx=0x7fda64011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.175+0000 7fda72a00640 1 -- 172.21.15.59:0/1714282845 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6400f920 con 0x7fda7c160540 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.175+0000 7fda72a00640 1 -- 172.21.15.59:0/1714282845 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6400fac0 con 0x7fda7c160540 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.175+0000 7fda72a00640 1 -- 172.21.15.59:0/1714282845 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6400fe70 con 0x7fda7c160540 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.175+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c0b1ef0 con 0x7fda7c160540 2024-04-23T14:31:41.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.175+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c0cb730 con 0x7fda7c160540 2024-04-23T14:31:41.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.177+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c04fd90 con 0x7fda7c160540 2024-04-23T14:31:41.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.177+0000 7fda72a00640 1 -- 172.21.15.59:0/1714282845 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda64015460 con 0x7fda7c160540 2024-04-23T14:31:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.178+0000 7fda72a00640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.179+0000 7fda72a00640 1 -- 172.21.15.59:0/1714282845 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda6408c520 con 0x7fda7c160540 2024-04-23T14:31:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.179+0000 7fda79e00640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.179+0000 7fda79e00640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda5806c8f0 0x7fda5806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda6c034000 tx=0x7fda6c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.183+0000 7fda72a00640 1 -- 172.21.15.59:0/1714282845 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda64019030 con 0x7fda7c160540 2024-04-23T14:31:41.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.391+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fda7c15ce80 con 0x7fda7c160540 2024-04-23T14:31:41.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.392+0000 7fda72a00640 1 -- 172.21.15.59:0/1714282845 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fda6405a8b0 con 0x7fda7c160540 2024-04-23T14:31:41.393 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.394+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda5806c8f0 msgr2=0x7fda5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.394+0000 7fda7b200640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda5806c8f0 0x7fda5806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda6c034000 tx=0x7fda6c02f000 comp rx=0 tx=0).stop 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.394+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda7c160540 msgr2=0x7fda7c0b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.394+0000 7fda7b200640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda7c160540 0x7fda7c0b1190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda64011600 tx=0x7fda64011ac0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 shutdown_connections 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c1609c0 0x7fda7c0d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda7c160540 0x7fda7c0b1190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 --2- 172.21.15.59:0/1714282845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda600039e0 0x7fda7c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 >> 172.21.15.59:0/1714282845 conn(0x7fda7c08bf90 msgr2=0x7fda7c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 shutdown_connections 2024-04-23T14:31:41.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.395+0000 7fda7b200640 1 -- 172.21.15.59:0/1714282845 wait complete. 2024-04-23T14:31:41.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:31:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.629+0000 7f5994c00640 1 Processor -- start 2024-04-23T14:31:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.629+0000 7f5994c00640 1 -- start start 2024-04-23T14:31:41.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f5994c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59901454e0 0x7f5990145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f5994c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59900cd020 0x7f59900b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f5994c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59900b3e40 0x7f5990160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f5994c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f59900cddc0 con 0x7f59900b3e40 2024-04-23T14:31:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f5994c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f59900cd520 con 0x7f59900cd020 2024-04-23T14:31:41.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f5994c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f59900cd800 con 0x7f59901454e0 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f598e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59900b3e40 0x7f5990160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f598e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59900b3e40 0x7f5990160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47804/0 (socket says 172.21.15.59:47804) 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f598e800640 1 -- 172.21.15.59:0/4127991479 learned_addr learned my addr 172.21.15.59:0/4127991479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f598d400640 1 --2- 172.21.15.59:0/4127991479 >> v2:172.21.15.76:3300/0 conn(0x7f59900cd020 0x7f59900b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.631+0000 7f598de00640 1 --2- 172.21.15.59:0/4127991479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59901454e0 0x7f5990145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.632+0000 7f598e800640 1 -- 172.21.15.59:0/4127991479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59901454e0 msgr2=0x7f5990145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.632+0000 7f598e800640 1 --2- 172.21.15.59:0/4127991479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59901454e0 0x7f5990145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.632+0000 7f598e800640 1 -- 172.21.15.59:0/4127991479 >> v2:172.21.15.76:3300/0 conn(0x7f59900cd020 msgr2=0x7f59900b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.632+0000 7f598e800640 1 --2- 172.21.15.59:0/4127991479 >> v2:172.21.15.76:3300/0 conn(0x7f59900cd020 0x7f59900b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.632+0000 7f598e800640 1 -- 172.21.15.59:0/4127991479 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59900cdae0 con 0x7f59900b3e40 2024-04-23T14:31:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.633+0000 7f598e800640 1 --2- 172.21.15.59:0/4127991479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f5990160910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59900701e0 tx=0x7f598802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197cfddb54090fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.633+0000 7f598ca00640 1 -- 172.21.15.59:0/4127991479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988036070 con 0x7f59900b3e40 2024-04-23T14:31:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.633+0000 7f598ca00640 1 -- 172.21.15.59:0/4127991479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5988031600 con 0x7f59900b3e40 2024-04-23T14:31:41.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.633+0000 7f598ca00640 1 -- 172.21.15.59:0/4127991479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59880319d0 con 0x7f59900b3e40 2024-04-23T14:31:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 -- 172.21.15.59:0/4127991479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 msgr2=0x7f5990160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 --2- 172.21.15.59:0/4127991479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f5990160910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59900701e0 tx=0x7f598802a610 comp rx=0 tx=0).stop 2024-04-23T14:31:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 -- 172.21.15.59:0/4127991479 shutdown_connections 2024-04-23T14:31:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 --2- 172.21.15.59:0/4127991479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f5990160910 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 --2- 172.21.15.59:0/4127991479 >> v2:172.21.15.76:3300/0 conn(0x7f59900cd020 0x7f59900b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 --2- 172.21.15.59:0/4127991479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59901454e0 0x7f5990145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 -- 172.21.15.59:0/4127991479 >> 172.21.15.59:0/4127991479 conn(0x7f5990067ea0 msgr2=0x7f599006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 -- 172.21.15.59:0/4127991479 shutdown_connections 2024-04-23T14:31:41.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.634+0000 7f5994c00640 1 -- 172.21.15.59:0/4127991479 wait complete. 2024-04-23T14:31:41.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.636+0000 7f5994c00640 1 Processor -- start 2024-04-23T14:31:41.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.636+0000 7f5994c00640 1 -- start start 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f5994c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f599015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f5994c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59900cd020 0x7f59901e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f5994c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59901454e0 0x7f599015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f5994c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5990146270 con 0x7f59900b3e40 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f5994c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59901460f0 con 0x7f59901454e0 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f5994c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5990160e50 con 0x7f59900cd020 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f598de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f599015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f598de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f599015bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47814/0 (socket says 172.21.15.59:47814) 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.637+0000 7f598de00640 1 -- 172.21.15.59:0/470104547 learned_addr learned my addr 172.21.15.59:0/470104547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598d400640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59900cd020 0x7f59901e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598de00640 1 -- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59900cd020 msgr2=0x7f59901e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598de00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59900cd020 0x7f59901e6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598de00640 1 -- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59901454e0 msgr2=0x7f599015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598de00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59901454e0 0x7f599015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598de00640 1 -- 172.21.15.59:0/470104547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599015c240 con 0x7f59900b3e40 2024-04-23T14:31:41.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598d400640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59900cd020 0x7f59901e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:41.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f598de00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f599015bd00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f598000e8a0 tx=0x7f598000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:41.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.638+0000 7f5986000640 1 -- 172.21.15.59:0/470104547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980017070 con 0x7f59900b3e40 2024-04-23T14:31:41.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.639+0000 7f5986000640 1 -- 172.21.15.59:0/470104547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598000c8c0 con 0x7f59900b3e40 2024-04-23T14:31:41.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.639+0000 7f5986000640 1 -- 172.21.15.59:0/470104547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598000ccb0 con 0x7f59900b3e40 2024-04-23T14:31:41.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.639+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59901400a0 con 0x7f59900b3e40 2024-04-23T14:31:41.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.639+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599006df10 con 0x7f59900b3e40 2024-04-23T14:31:41.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.640+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5990068a80 con 0x7f59900b3e40 2024-04-23T14:31:41.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.641+0000 7f5986000640 1 -- 172.21.15.59:0/470104547 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f598001a070 con 0x7f59900b3e40 2024-04-23T14:31:41.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.642+0000 7f5986000640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597006c920 0x7f597006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:41.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.642+0000 7f5986000640 1 -- 172.21.15.59:0/470104547 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f598008bc90 con 0x7f59900b3e40 2024-04-23T14:31:41.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.642+0000 7f598d400640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597006c920 0x7f597006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:41.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.643+0000 7f598d400640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597006c920 0x7f597006ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59900b3150 tx=0x7f597c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:41.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.647+0000 7f5986000640 1 -- 172.21.15.59:0/470104547 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f598005a0b0 con 0x7f59900b3e40 2024-04-23T14:31:41.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.897+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5990160030 con 0x7f59900b3e40 2024-04-23T14:31:41.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.898+0000 7f5986000640 1 -- 172.21.15.59:0/470104547 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f59800599b0 con 0x7f59900b3e40 2024-04-23T14:31:41.899 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:41.899 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:31:41.899 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:31:41.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.900+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597006c920 msgr2=0x7f597006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.900+0000 7f5994c00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597006c920 0x7f597006ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59900b3150 tx=0x7f597c008040 comp rx=0 tx=0).stop 2024-04-23T14:31:41.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.900+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 msgr2=0x7f599015bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:41.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.900+0000 7f5994c00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f599015bd00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f598000e8a0 tx=0x7f598000ed60 comp rx=0 tx=0).stop 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.900+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 shutdown_connections 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.901+0000 7f5994c00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597006c920 0x7f597006ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.901+0000 7f5994c00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59900cd020 0x7f59901e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.901+0000 7f5994c00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59901454e0 0x7f599015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.901+0000 7f5994c00640 1 --2- 172.21.15.59:0/470104547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59900b3e40 0x7f599015bd00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.901+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 >> 172.21.15.59:0/470104547 conn(0x7f5990067ea0 msgr2=0x7f59900d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.901+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 shutdown_connections 2024-04-23T14:31:41.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:41.901+0000 7f5994c00640 1 -- 172.21.15.59:0/470104547 wait complete. 2024-04-23T14:31:41.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:31:42.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.128+0000 7fb6b1a00640 1 Processor -- start 2024-04-23T14:31:42.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.128+0000 7fb6b1a00640 1 -- start start 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6b1a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6ac160960 0x7fb6ac163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6b1a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac1644b0 0x7fb6ac0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6b1a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6ac160d40 con 0x7fb6ac1668e0 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6b1a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb6ac164270 con 0x7fb6ac160960 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6b1a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac166de0 con 0x7fb6ac1644b0 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6aaa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6ac160960 0x7fb6ac163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6aaa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6ac160960 0x7fb6ac163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47818/0 (socket says 172.21.15.59:47818) 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6aaa00640 1 -- 172.21.15.59:0/100102474 learned_addr learned my addr 172.21.15.59:0/100102474 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6ab400640 1 --1- 172.21.15.59:0/100102474 >> v1:172.21.15.59:6789/0 conn(0x7fb6ac1668e0 0x7fb6ac071de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47606/0 (socket says 172.21.15.59:47606) 2024-04-23T14:31:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.130+0000 7fb6aa000640 1 --2- 172.21.15.59:0/100102474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac1644b0 0x7fb6ac0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.131+0000 7fb6aaa00640 1 -- 172.21.15.59:0/100102474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac1644b0 msgr2=0x7fb6ac0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.131+0000 7fb6aaa00640 1 --2- 172.21.15.59:0/100102474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac1644b0 0x7fb6ac0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.131+0000 7fb6aaa00640 1 -- 172.21.15.59:0/100102474 >> v1:172.21.15.59:6789/0 conn(0x7fb6ac1668e0 legacy=0x7fb6ac071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.131+0000 7fb6aaa00640 1 -- 172.21.15.59:0/100102474 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac1670c0 con 0x7fb6ac160960 2024-04-23T14:31:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.131+0000 7fb6a9600640 1 -- 172.21.15.59:0/100102474 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222935214 0 0) 0x7fb6ac160d40 con 0x7fb6ac1668e0 2024-04-23T14:31:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.132+0000 7fb6aaa00640 1 --2- 172.21.15.59:0/100102474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac160960 0x7fb6ac163d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009870 tx=0x7fb69c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bae892fb0bcce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.132+0000 7fb6a9600640 1 -- 172.21.15.59:0/100102474 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c037070 con 0x7fb6ac160960 2024-04-23T14:31:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.132+0000 7fb6a9600640 1 -- 172.21.15.59:0/100102474 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69c032440 con 0x7fb6ac160960 2024-04-23T14:31:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.132+0000 7fb6a9600640 1 -- 172.21.15.59:0/100102474 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c032810 con 0x7fb6ac160960 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 -- 172.21.15.59:0/100102474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac160960 msgr2=0x7fb6ac163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/100102474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac160960 0x7fb6ac163d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009870 tx=0x7fb69c02a270 comp rx=0 tx=0).stop 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 -- 172.21.15.59:0/100102474 shutdown_connections 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/100102474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac1644b0 0x7fb6ac0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/100102474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac160960 0x7fb6ac163d30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 -- 172.21.15.59:0/100102474 >> 172.21.15.59:0/100102474 conn(0x7fb6ac067ea0 msgr2=0x7fb6ac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 -- 172.21.15.59:0/100102474 shutdown_connections 2024-04-23T14:31:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.133+0000 7fb6b1a00640 1 -- 172.21.15.59:0/100102474 wait complete. 2024-04-23T14:31:42.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.135+0000 7fb6b1a00640 1 Processor -- start 2024-04-23T14:31:42.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.136+0000 7fb6b1a00640 1 -- start start 2024-04-23T14:31:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6b1a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac160960 0x7fb6ac14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6b1a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6ac1644b0 0x7fb6ac162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6b1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac1668e0 0x7fb6ac1f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6b1a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac0b4380 con 0x7fb6ac1668e0 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6b1a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac167680 con 0x7fb6ac1644b0 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6b1a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac0b4500 con 0x7fb6ac160960 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac1668e0 0x7fb6ac1f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac1668e0 0x7fb6ac1f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47830/0 (socket says 172.21.15.59:47830) 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6ab400640 1 -- 172.21.15.59:0/1209541645 learned_addr learned my addr 172.21.15.59:0/1209541645 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.137+0000 7fb6aaa00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac160960 0x7fb6ac14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6aa000640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6ac1644b0 0x7fb6ac162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6ab400640 1 -- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac160960 msgr2=0x7fb6ac14e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6ab400640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac160960 0x7fb6ac14e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6ab400640 1 -- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6ac1644b0 msgr2=0x7fb6ac162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6ab400640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6ac1644b0 0x7fb6ac162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6ab400640 1 -- 172.21.15.59:0/1209541645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac0cfba0 con 0x7fb6ac1668e0 2024-04-23T14:31:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6ab400640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac1668e0 0x7fb6ac1f8c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400c410 tx=0x7fb6a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6a2a00640 1 -- 172.21.15.59:0/1209541645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40074e0 con 0x7fb6ac1668e0 2024-04-23T14:31:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6a2a00640 1 -- 172.21.15.59:0/1209541645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a40047e0 con 0x7fb6ac1668e0 2024-04-23T14:31:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6a2a00640 1 -- 172.21.15.59:0/1209541645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4004bb0 con 0x7fb6ac1668e0 2024-04-23T14:31:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.138+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac1f9690 con 0x7fb6ac1668e0 2024-04-23T14:31:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.139+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac1f9c50 con 0x7fb6ac1668e0 2024-04-23T14:31:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.140+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ac1f9e50 con 0x7fb6ac1668e0 2024-04-23T14:31:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.141+0000 7fb6a2a00640 1 -- 172.21.15.59:0/1209541645 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6a4011070 con 0x7fb6ac1668e0 2024-04-23T14:31:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.142+0000 7fb6a2a00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb68806c8f0 0x7fb68806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.142+0000 7fb6a2a00640 1 -- 172.21.15.59:0/1209541645 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6a408b650 con 0x7fb6ac1668e0 2024-04-23T14:31:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.143+0000 7fb6aaa00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb68806c8f0 0x7fb68806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.143+0000 7fb6aaa00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb68806c8f0 0x7fb68806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb69c034000 tx=0x7fb69c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:42.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.147+0000 7fb6a2a00640 1 -- 172.21.15.59:0/1209541645 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6ac0cfba0 con 0x7fb6ac1668e0 2024-04-23T14:31:42.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.354+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6ac06e260 con 0x7fb6ac1668e0 2024-04-23T14:31:42.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.356+0000 7fb6a2a00640 1 -- 172.21.15.59:0/1209541645 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fb6a401a2b0 con 0x7fb6ac1668e0 2024-04-23T14:31:42.357 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:42.357 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb68806c8f0 msgr2=0x7fb68806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb68806c8f0 0x7fb68806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb69c034000 tx=0x7fb69c02e000 comp rx=0 tx=0).stop 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac1668e0 msgr2=0x7fb6ac1f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac1668e0 0x7fb6ac1f8c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400c410 tx=0x7fb6a400c8d0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 shutdown_connections 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb68806c8f0 0x7fb68806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6ac1668e0 0x7fb6ac1f8c70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6ac1644b0 0x7fb6ac162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 --2- 172.21.15.59:0/1209541645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6ac160960 0x7fb6ac14e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 >> 172.21.15.59:0/1209541645 conn(0x7fb6ac067ea0 msgr2=0x7fb6ac0b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:42.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 shutdown_connections 2024-04-23T14:31:42.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.359+0000 7fb6b1a00640 1 -- 172.21.15.59:0/1209541645 wait complete. 2024-04-23T14:31:42.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:31:42.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.589+0000 7fabcdc00640 1 Processor -- start 2024-04-23T14:31:42.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.589+0000 7fabcdc00640 1 -- start start 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabcdc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fabc80b5ff0 0x7fabc80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabcdc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80cf030 con 0x7fabc80d0d90 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabcdc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80b6a40 con 0x7fabc815e7c0 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabcdc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fabc80b7000 con 0x7fabc80b5ff0 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabbfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fabc80b5ff0 0x7fabc80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabbfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fabc80b5ff0 0x7fabc80b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47838/0 (socket says 172.21.15.59:47838) 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabbfe00640 1 -- 172.21.15.59:0/2473190159 learned_addr learned my addr 172.21.15.59:0/2473190159 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:42.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.591+0000 7fabcd200640 1 --1- 172.21.15.59:0/2473190159 >> v1:172.21.15.59:6789/0 conn(0x7fabc80d0d90 0x7fabc80cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47618/0 (socket says 172.21.15.59:47618) 2024-04-23T14:31:42.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.592+0000 7fabbfe00640 1 -- 172.21.15.59:0/2473190159 >> v1:172.21.15.76:6789/0 conn(0x7fabc815e7c0 legacy=0x7fabc80d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:42.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.592+0000 7fabbfe00640 1 -- 172.21.15.59:0/2473190159 >> v1:172.21.15.59:6789/0 conn(0x7fabc80d0d90 legacy=0x7fabc80cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.592+0000 7fabbfe00640 1 -- 172.21.15.59:0/2473190159 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80b6d20 con 0x7fabc80b5ff0 2024-04-23T14:31:42.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.592+0000 7fabc7400640 1 -- 172.21.15.59:0/2473190159 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131101228 0 0) 0x7fabc0001490 con 0x7fabc80d0d90 2024-04-23T14:31:42.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.593+0000 7fabbfe00640 1 --2- 172.21.15.59:0/2473190159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc80b5ff0 0x7fabc80b6450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009870 tx=0x7fabb002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e04ea30d14377ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:42.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.593+0000 7fabc7400640 1 -- 172.21.15.59:0/2473190159 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0037070 con 0x7fabc80b5ff0 2024-04-23T14:31:42.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.593+0000 7fabc7400640 1 -- 172.21.15.59:0/2473190159 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb0032440 con 0x7fabc80b5ff0 2024-04-23T14:31:42.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.593+0000 7fabc7400640 1 -- 172.21.15.59:0/2473190159 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0032810 con 0x7fabc80b5ff0 2024-04-23T14:31:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.594+0000 7fabcdc00640 1 -- 172.21.15.59:0/2473190159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc80b5ff0 msgr2=0x7fabc80b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.594+0000 7fabcdc00640 1 --2- 172.21.15.59:0/2473190159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc80b5ff0 0x7fabc80b6450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009870 tx=0x7fabb002a2a0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.594+0000 7fabcdc00640 1 -- 172.21.15.59:0/2473190159 shutdown_connections 2024-04-23T14:31:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.594+0000 7fabcdc00640 1 --2- 172.21.15.59:0/2473190159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc80b5ff0 0x7fabc80b6450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.594+0000 7fabcdc00640 1 -- 172.21.15.59:0/2473190159 >> 172.21.15.59:0/2473190159 conn(0x7fabc809d920 msgr2=0x7fabc808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.594+0000 7fabcdc00640 1 -- 172.21.15.59:0/2473190159 shutdown_connections 2024-04-23T14:31:42.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.594+0000 7fabcdc00640 1 -- 172.21.15.59:0/2473190159 wait complete. 2024-04-23T14:31:42.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.596+0000 7fabcdc00640 1 Processor -- start 2024-04-23T14:31:42.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.596+0000 7fabcdc00640 1 -- start start 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcdc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc80b5ff0 0x7fabc80ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcdc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc80d0d90 0x7fabc80cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc815e7c0 0x7fabc815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcdc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80d2800 con 0x7fabc815e7c0 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcdc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80cf620 con 0x7fabc80d0d90 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcdc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fabc80d1a80 con 0x7fabc80b5ff0 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc815e7c0 0x7fabc815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc815e7c0 0x7fabc815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47848/0 (socket says 172.21.15.59:47848) 2024-04-23T14:31:42.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabcd200640 1 -- 172.21.15.59:0/874344805 learned_addr learned my addr 172.21.15.59:0/874344805 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:42.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabbfe00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc80d0d90 0x7fabc80cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.597+0000 7fabc7e00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc80b5ff0 0x7fabc80ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabcd200640 1 -- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc80b5ff0 msgr2=0x7fabc80ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabcd200640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc80b5ff0 0x7fabc80ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabcd200640 1 -- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc80d0d90 msgr2=0x7fabc80cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabcd200640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc80d0d90 0x7fabc80cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabcd200640 1 -- 172.21.15.59:0/874344805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80cb690 con 0x7fabc815e7c0 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabcd200640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc815e7c0 0x7fabc815b5d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fabc000c7f0 tx=0x7fabc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabc4c00640 1 -- 172.21.15.59:0/874344805 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0018070 con 0x7fabc815e7c0 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabc4c00640 1 -- 172.21.15.59:0/874344805 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc000a5e0 con 0x7fabc815e7c0 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabc4c00640 1 -- 172.21.15.59:0/874344805 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc000aa10 con 0x7fabc815e7c0 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.598+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc80cb970 con 0x7fabc815e7c0 2024-04-23T14:31:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.599+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc80cbf30 con 0x7fabc815e7c0 2024-04-23T14:31:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.600+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc804fd90 con 0x7fabc815e7c0 2024-04-23T14:31:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.601+0000 7fabc4c00640 1 -- 172.21.15.59:0/874344805 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabc000abb0 con 0x7fabc815e7c0 2024-04-23T14:31:42.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.602+0000 7fabc4c00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:42.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.602+0000 7fabc4c00640 1 -- 172.21.15.59:0/874344805 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fabc008b280 con 0x7fabc815e7c0 2024-04-23T14:31:42.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.602+0000 7fabc7e00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:42.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.603+0000 7fabc7e00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabb8004ea0 tx=0x7fabb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:42.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.607+0000 7fabc4c00640 1 -- 172.21.15.59:0/874344805 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fabc80cf030 con 0x7fabc815e7c0 2024-04-23T14:31:42.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.857+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabc80d1eb0 con 0x7fabc815e7c0 2024-04-23T14:31:42.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.859+0000 7fabc4c00640 1 -- 172.21.15.59:0/874344805 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7fabc0059610 con 0x7fabc815e7c0 2024-04-23T14:31:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:31:42.860 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:31:42.860 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.861+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 msgr2=0x7faba406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.861+0000 7fabcdc00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabb8004ea0 tx=0x7fabb8009040 comp rx=0 tx=0).stop 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.861+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc815e7c0 msgr2=0x7fabc815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.861+0000 7fabcdc00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc815e7c0 0x7fabc815b5d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fabc000c7f0 tx=0x7fabc000ccb0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 shutdown_connections 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabc815e7c0 0x7fabc815b5d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc80d0d90 0x7fabc80cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 --2- 172.21.15.59:0/874344805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc80b5ff0 0x7fabc80ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 >> 172.21.15.59:0/874344805 conn(0x7fabc809d920 msgr2=0x7fabc80b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 shutdown_connections 2024-04-23T14:31:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:42.862+0000 7fabcdc00640 1 -- 172.21.15.59:0/874344805 wait complete. 2024-04-23T14:31:43.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:31:25.258431+0000', 'modified': '2024-04-23T14:31:28.648219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/592246573', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 592246573}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 592246573}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-04-23T14:31:43.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:31:43.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-23T14:31:43.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.503+0000 7ff665000640 1 Processor -- start 2024-04-23T14:31:43.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.503+0000 7ff665000640 1 -- start start 2024-04-23T14:31:43.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.504+0000 7ff665000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff6600d0cd0 0x7ff6601674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.504+0000 7ff665000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6600b4cf0 con 0x7ff6600cd8b0 2024-04-23T14:31:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.504+0000 7ff665000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6600ccdd0 con 0x7ff66015c670 2024-04-23T14:31:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.504+0000 7ff665000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff66015ca50 con 0x7ff6600d0cd0 2024-04-23T14:31:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.505+0000 7ff65ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff6600cd8b0 0x7ff6600d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47632/0 (socket says 172.21.15.59:47632) 2024-04-23T14:31:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.505+0000 7ff65ea00640 1 -- 172.21.15.59:0/1226444248 learned_addr learned my addr 172.21.15.59:0/1226444248 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:43.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.505+0000 7ff65fe00640 1 --2- 172.21.15.59:0/1226444248 >> v2:172.21.15.76:3300/0 conn(0x7ff6600d0cd0 0x7ff6601674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.505+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454827410 0 0) 0x7ff6600b4cf0 con 0x7ff6600cd8b0 2024-04-23T14:31:43.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.505+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff644003520 con 0x7ff6600cd8b0 2024-04-23T14:31:43.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432503302 0 0) 0x7ff644003520 con 0x7ff6600cd8b0 2024-04-23T14:31:43.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 >> v2:172.21.15.76:3300/0 conn(0x7ff6600d0cd0 msgr2=0x7ff6601674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 --2- 172.21.15.59:0/1226444248 >> v2:172.21.15.76:3300/0 conn(0x7ff6600d0cd0 0x7ff6601674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 >> v1:172.21.15.76:6789/0 conn(0x7ff66015c670 legacy=0x7ff6600b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6600cd0b0 con 0x7ff6600cd8b0 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7ff64c001670 con 0x7ff6600cd8b0 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 >> v1:172.21.15.59:6789/0 conn(0x7ff6600cd8b0 legacy=0x7ff6600d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6440039b0 0x7ff644003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff644004800 0x7ff644004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6440051c0 0x7ff64400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff64400c030 con 0x7ff6440051c0 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff64400c1e0 con 0x7ff6440039b0 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff64400c3e0 con 0x7ff644004800 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65fe00640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6440051c0 0x7ff64400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65f400640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6440039b0 0x7ff644003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.506+0000 7ff65ea00640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff644004800 0x7ff644004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.507+0000 7ff65fe00640 1 -- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff644004800 msgr2=0x7ff644004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.507+0000 7ff65fe00640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff644004800 0x7ff644004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.507+0000 7ff65fe00640 1 -- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6440039b0 msgr2=0x7ff644003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.507+0000 7ff65fe00640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6440039b0 0x7ff644003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.507+0000 7ff65fe00640 1 -- 172.21.15.59:0/1226444248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff644003520 con 0x7ff6440051c0 2024-04-23T14:31:43.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.507+0000 7ff65fe00640 1 -- 172.21.15.59:0/1226444248 reap_dead start 2024-04-23T14:31:43.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.508+0000 7ff65fe00640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6440051c0 0x7ff64400b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff658009fc0 tx=0x7ff644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a51747eaa50323 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.508+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658035020 con 0x7ff6440051c0 2024-04-23T14:31:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.508+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff658003bd0 con 0x7ff6440051c0 2024-04-23T14:31:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.508+0000 7ff65e000640 1 -- 172.21.15.59:0/1226444248 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658009c10 con 0x7ff6440051c0 2024-04-23T14:31:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.508+0000 7ff665000640 1 -- 172.21.15.59:0/1226444248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6440051c0 msgr2=0x7ff64400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.508+0000 7ff665000640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6440051c0 0x7ff64400b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff658009fc0 tx=0x7ff644005830 comp rx=0 tx=0).stop 2024-04-23T14:31:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.509+0000 7ff665000640 1 -- 172.21.15.59:0/1226444248 shutdown_connections 2024-04-23T14:31:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.509+0000 7ff665000640 1 --2- 172.21.15.59:0/1226444248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6440051c0 0x7ff64400b9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.509+0000 7ff665000640 1 -- 172.21.15.59:0/1226444248 >> 172.21.15.59:0/1226444248 conn(0x7ff66006bdf0 msgr2=0x7ff66006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.509+0000 7ff665000640 1 -- 172.21.15.59:0/1226444248 shutdown_connections 2024-04-23T14:31:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.509+0000 7ff665000640 1 -- 172.21.15.59:0/1226444248 wait complete. 2024-04-23T14:31:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.510+0000 7ff665000640 1 Processor -- start 2024-04-23T14:31:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.511+0000 7ff665000640 1 -- start start 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.511+0000 7ff665000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff6440051c0 0x7ff66014c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.511+0000 7ff665000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66009cd70 0x7ff6601f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.511+0000 7ff665000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6601f4ab0 0x7ff6601f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.511+0000 7ff665000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600b3e60 con 0x7ff6601f4ab0 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.511+0000 7ff665000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600ccb80 con 0x7ff66009cd70 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.511+0000 7ff665000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600b3fe0 con 0x7ff6440051c0 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6601f4ab0 0x7ff6601f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6601f4ab0 0x7ff6601f7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47862/0 (socket says 172.21.15.59:47862) 2024-04-23T14:31:43.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 -- 172.21.15.59:0/3025117086 learned_addr learned my addr 172.21.15.59:0/3025117086 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65f400640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff6440051c0 0x7ff66014c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65ea00640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66009cd70 0x7ff6601f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 -- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff6440051c0 msgr2=0x7ff66014c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff6440051c0 0x7ff66014c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 -- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66009cd70 msgr2=0x7ff6601f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66009cd70 0x7ff6601f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.512+0000 7ff65fe00640 1 -- 172.21.15.59:0/3025117086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66009da70 con 0x7ff6601f4ab0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.513+0000 7ff65fe00640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6601f4ab0 0x7ff6601f7b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e230 tx=0x7ff658009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.513+0000 7ff657400640 1 -- 172.21.15.59:0/3025117086 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658035030 con 0x7ff6601f4ab0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.513+0000 7ff657400640 1 -- 172.21.15.59:0/3025117086 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff658043070 con 0x7ff6601f4ab0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.513+0000 7ff657400640 1 -- 172.21.15.59:0/3025117086 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65802d070 con 0x7ff6601f4ab0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.513+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6601f4f30 con 0x7ff6601f4ab0 2024-04-23T14:31:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.513+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66009b1d0 con 0x7ff6601f4ab0 2024-04-23T14:31:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.515+0000 7ff657400640 1 -- 172.21.15.59:0/3025117086 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff658005d10 con 0x7ff6601f4ab0 2024-04-23T14:31:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.516+0000 7ff657400640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64006c8f0 0x7ff64006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.516+0000 7ff657400640 1 -- 172.21.15.59:0/3025117086 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6580aca70 con 0x7ff6601f4ab0 2024-04-23T14:31:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.516+0000 7ff65f400640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64006c8f0 0x7ff64006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.517+0000 7ff66940b740 1 Processor -- start 2024-04-23T14:31:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.517+0000 7ff65f400640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64006c8f0 0x7ff64006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff650005490 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.517+0000 7ff66940b740 1 -- start start 2024-04-23T14:31:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff66940b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef14fa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef1529e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152a360 0x561ef14ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff66940b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561ef1464b10 con 0x561ef1547980 2024-04-23T14:31:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ef140b2b0 con 0x561ef152aaa0 2024-04-23T14:31:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x561ef1460ad0 con 0x561ef152a360 2024-04-23T14:31:43.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff65f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef14fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff65f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef14fa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47872/0 (socket says 172.21.15.59:47872) 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.518+0000 7ff65f400640 1 -- 172.21.15.59:0/1248578046 learned_addr learned my addr 172.21.15.59:0/1248578046 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff65f400640 1 -- 172.21.15.59:0/1248578046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152a360 msgr2=0x561ef14ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff65f400640 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152a360 0x561ef14ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff65f400640 1 -- 172.21.15.59:0/1248578046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 msgr2=0x561ef1529e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff65ea00640 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef1529e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff65f400640 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef1529e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff65f400640 1 -- 172.21.15.59:0/1248578046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef11772d0 con 0x561ef1547980 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff65f400640 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef14fa310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff650001580 tx=0x7ff650013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a5875b0a201efe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff64be00640 1 -- 172.21.15.59:0/1248578046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650027070 con 0x561ef1547980 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff64be00640 1 -- 172.21.15.59:0/1248578046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff650014800 con 0x561ef1547980 2024-04-23T14:31:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.519+0000 7ff64be00640 1 -- 172.21.15.59:0/1248578046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650014bb0 con 0x561ef1547980 2024-04-23T14:31:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 -- 172.21.15.59:0/1248578046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 msgr2=0x561ef14fa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef14fa310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff650001580 tx=0x7ff650013000 comp rx=0 tx=0).stop 2024-04-23T14:31:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 -- 172.21.15.59:0/1248578046 shutdown_connections 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152a360 0x561ef14ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef1529e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 --2- 172.21.15.59:0/1248578046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef14fa310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 -- 172.21.15.59:0/1248578046 >> 172.21.15.59:0/1248578046 conn(0x561ef14fc580 msgr2=0x561ef1544e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 -- 172.21.15.59:0/1248578046 shutdown_connections 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.520+0000 7ff66940b740 1 -- 172.21.15.59:0/1248578046 wait complete. 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.521+0000 7ff66940b740 1 -- start start 2024-04-23T14:31:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.521+0000 7ff66940b740 1 Processor -- start 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.522+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef15438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.522+0000 7ff66940b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef1545220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.522+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef1546720 0x561ef1524e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.522+0000 7ff66940b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561ef14fe730 con 0x561ef1547980 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.522+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ef141e590 con 0x561ef152aaa0 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.522+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x561ef14fdb20 con 0x561ef1546720 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.522+0000 7ff66940b740 10 client.0 _ll_register_callbacks cb 0x561ef14013a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef1545220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef1545220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47884/0 (socket says 172.21.15.59:47884) 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 -- 172.21.15.59:0/3149626335 learned_addr learned my addr 172.21.15.59:0/3149626335 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65f400640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef15438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65fe00640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef1546720 0x561ef1524e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 -- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef1546720 msgr2=0x561ef1524e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef1546720 0x561ef1524e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 -- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 msgr2=0x561ef15438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef15438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 -- 172.21.15.59:0/3149626335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef12416b0 con 0x561ef1547980 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65f400640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef15438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff65ea00640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef1545220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0112d0 tx=0x7ff64c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa215cfe4047d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.523+0000 7ff648c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff648c00640 1 -- 172.21.15.59:0/3149626335 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c010040 con 0x561ef1547980 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff648c00640 1 -- 172.21.15.59:0/3149626335 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff64c014780 con 0x561ef1547980 2024-04-23T14:31:43.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff650014800 con 0x561ef1547980 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff648c00640 1 -- 172.21.15.59:0/3149626335 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7ff64c022050 con 0x561ef1547980 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 10 client.4629 fetch_fsmap learned FSMap version 7 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561ef153ca80 con 0x561ef1547980 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff648c00640 1 -- 172.21.15.59:0/3149626335 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff650014800 con 0x561ef1547980 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff648c00640 1 -- 172.21.15.59:0/3149626335 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7ff64c025020 con 0x561ef1547980 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 10 client.4629 fetch_fsmap finished waiting for FSMap version 7 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 10 client.4629 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 20 client.4629 populate_metadata read hostname 'smithi059' 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef152c4b0 0x561ef1558ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 4 client.4629 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.524+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 --> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ef1472530 con 0x561ef152c4b0 2024-04-23T14:31:43.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.525+0000 7ff65f400640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef152c4b0 0x561ef1558ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.526+0000 7ff65f400640 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef152c4b0 0x561ef1558ee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad5d7c3cf3a183f server_cookie=91e1732209180f02 in_seq=0 out_seq=0 2024-04-23T14:31:43.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.526+0000 7ff648c00640 10 client.4629 ms_handle_connect on v2:172.21.15.76:6834/592246573 2024-04-23T14:31:43.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff648c00640 1 -- 172.21.15.59:0/3149626335 <== mds.0 v2:172.21.15.76:6834/592246573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ef1472530 con 0x561ef152c4b0 2024-04-23T14:31:43.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff648c00640 10 client.4629 handle_command_reply: tid=0 2024-04-23T14:31:43.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 client.4629 shutdown 2024-04-23T14:31:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 msgr2=0x561ef1545220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef1545220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0112d0 tx=0x7ff64c0017a0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 shutdown_connections 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef152c4b0 0x561ef1558ee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef1546720 0x561ef1524e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef1545220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 --2- 172.21.15.59:0/3149626335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef152aaa0 0x561ef15438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff648c00640 0 client.4629 ms_handle_reset on v2:172.21.15.76:6834/592246573 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 >> 172.21.15.59:0/3149626335 conn(0x561ef152a360 msgr2=0x561ef1527350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 shutdown_connections 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 1 -- 172.21.15.59:0/3149626335 wait complete. 2024-04-23T14:31:43.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.529+0000 7ff66940b740 20 client.4629 trim_cache size 0 max 16384 2024-04-23T14:31:43.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.549+0000 7ff66940b740 1 Processor -- start 2024-04-23T14:31:43.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.549+0000 7ff66940b740 1 -- start start 2024-04-23T14:31:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef14fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff66940b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff66940b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6602001e0 con 0x561ef1547980 2024-04-23T14:31:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601e6d90 con 0x561ef1546720 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff6601e73e0 con 0x561ef152aaa0 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47886/0 (socket says 172.21.15.59:47886) 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.550+0000 7ff65fe00640 1 -- 172.21.15.59:0/3541618795 learned_addr learned my addr 172.21.15.59:0/3541618795 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65f400640 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef14fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65ea00640 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65fe00640 1 -- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 msgr2=0x561ef14fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65fe00640 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef14fb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65fe00640 1 -- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1546720 msgr2=0x561ef152d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65fe00640 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65fe00640 1 -- 172.21.15.59:0/3541618795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef1526120 con 0x561ef1547980 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65ea00640 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff65fe00640 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef152d640 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e200 tx=0x7ff65807a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414695a27cfab207 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.551+0000 7ff64b400640 1 -- 172.21.15.59:0/3541618795 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658083400 con 0x561ef1547980 2024-04-23T14:31:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff64b400640 1 -- 172.21.15.59:0/3541618795 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6580788c0 con 0x561ef1547980 2024-04-23T14:31:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff64b400640 1 -- 172.21.15.59:0/3541618795 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658009c10 con 0x561ef1547980 2024-04-23T14:31:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 -- 172.21.15.59:0/3541618795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 msgr2=0x561ef152d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef152d640 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e200 tx=0x7ff65807a240 comp rx=0 tx=0).stop 2024-04-23T14:31:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 -- 172.21.15.59:0/3541618795 shutdown_connections 2024-04-23T14:31:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 --2- 172.21.15.59:0/3541618795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef14fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 -- 172.21.15.59:0/3541618795 >> 172.21.15.59:0/3541618795 conn(0x561ef152a360 msgr2=0x561ef1545ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 -- 172.21.15.59:0/3541618795 shutdown_connections 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.552+0000 7ff66940b740 1 -- 172.21.15.59:0/3541618795 wait complete. 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.553+0000 7ff66940b740 1 -- start start 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.553+0000 7ff66940b740 1 Processor -- start 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.554+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef15482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.554+0000 7ff66940b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.554+0000 7ff66940b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.554+0000 7ff66940b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561ef153d640 con 0x561ef1546720 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.554+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ef152ccb0 con 0x561ef1547980 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.554+0000 7ff66940b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x561ef152c8d0 con 0x561ef152aaa0 2024-04-23T14:31:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.554+0000 7ff66940b740 10 client.0 _ll_register_callbacks cb 0x7ff64c001d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:31:43.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47888/0 (socket says 172.21.15.59:47888) 2024-04-23T14:31:43.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 -- 172.21.15.59:0/856251302 learned_addr learned my addr 172.21.15.59:0/856251302 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65f400640 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef15482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65fe00640 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 -- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 msgr2=0x561ef15482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef15482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 -- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1547980 msgr2=0x561ef152d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.555+0000 7ff65ea00640 1 -- 172.21.15.59:0/856251302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef1548db0 con 0x561ef1546720 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff65ea00640 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1546720 0x561ef152d100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0112d0 tx=0x7ff6600b4ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c9a5c37e204d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff648c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff648c00640 1 -- 172.21.15.59:0/856251302 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c02a020 con 0x561ef1546720 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff648c00640 1 -- 172.21.15.59:0/856251302 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff64c011c20 con 0x561ef1546720 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6580788c0 con 0x561ef1546720 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff648c00640 1 -- 172.21.15.59:0/856251302 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7ff64c034050 con 0x561ef1546720 2024-04-23T14:31:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff66940b740 10 client.4635 fetch_fsmap learned FSMap version 7 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.556+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561ef15bf7d0 con 0x561ef1546720 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff648c00640 1 -- 172.21.15.59:0/856251302 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6580788c0 con 0x561ef1546720 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff648c00640 1 -- 172.21.15.59:0/856251302 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7ff64c037020 con 0x561ef1546720 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff66940b740 10 client.4635 fetch_fsmap finished waiting for FSMap version 7 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff66940b740 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff66940b740 20 client.4635 populate_metadata read hostname 'smithi059' 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff66940b740 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef1567aa0 0x561ef1567e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff66940b740 4 client.4635 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 --> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] -- command(tid 0: {"prefix": "status"}) -- 0x561ef146c3a0 con 0x561ef1567aa0 2024-04-23T14:31:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.557+0000 7ff65f400640 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef1567aa0 0x561ef1567e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.558+0000 7ff65f400640 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef1567aa0 0x561ef1567e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1144f03f481d22 server_cookie=6f80b874199a6826 in_seq=0 out_seq=0 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.558+0000 7ff648c00640 10 client.4635 ms_handle_connect on v2:172.21.15.76:6834/592246573 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff648c00640 1 -- 172.21.15.59:0/856251302 <== mds.0 v2:172.21.15.76:6834/592246573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x561ef146c3a0 con 0x561ef1567aa0 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff648c00640 10 client.4635 handle_command_reply: tid=0 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 client.4635 shutdown 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1546720 msgr2=0x561ef152d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1546720 0x561ef152d100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0112d0 tx=0x7ff6600b4ec0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 shutdown_connections 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:6834/592246573,v1:172.21.15.76:6835/592246573] conn(0x561ef1567aa0 0x561ef1567e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ef1547980 0x561ef152d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561ef1546720 0x561ef152d100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 --2- 172.21.15.59:0/856251302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561ef152aaa0 0x561ef15482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 >> 172.21.15.59:0/856251302 conn(0x561ef152a360 msgr2=0x561ef1543e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff648c00640 0 client.4635 ms_handle_reset on v2:172.21.15.76:6834/592246573 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 shutdown_connections 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.559+0000 7ff66940b740 1 -- 172.21.15.59:0/856251302 wait complete. 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff66940b740 20 client.4635 trim_cache size 0 max 16384 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4279, 2024-04-23T14:31:43.563 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 17.746774350999999, 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 7, 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 20, 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 20, 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 23.082654711 2024-04-23T14:31:43.564 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64006c8f0 msgr2=0x7ff64006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64006c8f0 0x7ff64006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff650005490 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6601f4ab0 msgr2=0x7ff6601f7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6601f4ab0 0x7ff6601f7b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e230 tx=0x7ff658009750 comp rx=0 tx=0).stop 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 shutdown_connections 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64006c8f0 0x7ff64006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6601f4ab0 0x7ff6601f7b30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66009cd70 0x7ff6601f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 --2- 172.21.15.59:0/3025117086 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff6440051c0 0x7ff66014c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 >> 172.21.15.59:0/3025117086 conn(0x7ff66006bdf0 msgr2=0x7ff660160930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.560+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 shutdown_connections 2024-04-23T14:31:43.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:43.561+0000 7ff665000640 1 -- 172.21.15.59:0/3025117086 wait complete. 2024-04-23T14:31:43.579 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T14:31:43.589 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T14:31:43.589 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T14:31:43.589 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T14:31:43.589 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-23T14:31:43.618 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-23T14:31:43.618 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-23T14:31:43.618 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-23T14:31:43.618 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:31:43.618 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:43.618 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-23T14:31:43.618 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:31:43.619 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-23T14:31:43.619 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-23T14:31:43.619 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:31:43.619 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-23T14:31:43.619 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-23T14:31:43.619 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:31:43.619 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-23T14:31:43.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:43.619 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:31:43.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:43.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-04-23T14:31:43.771 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2024-04-23T14:31:43.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:43.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:43.775 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-04-23T14:31:43.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:43.835 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-04-23T14:31:43.925 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2024-04-23T14:31:43.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:31:43.929 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-23T14:31:43.929 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:31:44.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.149+0000 7f554be00640 1 Processor -- start 2024-04-23T14:31:44.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.150+0000 7f554be00640 1 -- start start 2024-04-23T14:31:44.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.151+0000 7f554be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f554c0cd760 0x7f554c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.151+0000 7f554be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c160210 con 0x7f554c0b4690 2024-04-23T14:31:44.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.151+0000 7f554be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c1604f0 con 0x7f554c15c3d0 2024-04-23T14:31:44.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.151+0000 7f554be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f554c1607d0 con 0x7f554c0cd760 2024-04-23T14:31:44.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.151+0000 7f554b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f554c0b4690 0x7f554c0cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47638/0 (socket says 172.21.15.59:47638) 2024-04-23T14:31:44.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.151+0000 7f554b400640 1 -- 172.21.15.59:0/331261073 learned_addr learned my addr 172.21.15.59:0/331261073 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:44.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f554a000640 1 --2- 172.21.15.59:0/331261073 >> v2:172.21.15.76:3300/0 conn(0x7f554c0cd760 0x7f554c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079743080 0 0) 0x7f554c160210 con 0x7f554c0b4690 2024-04-23T14:31:44.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5530003520 con 0x7f554c0b4690 2024-04-23T14:31:44.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3717832342 0 0) 0x7f5530003520 con 0x7f554c0b4690 2024-04-23T14:31:44.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 >> v2:172.21.15.76:3300/0 conn(0x7f554c0cd760 msgr2=0x7f554c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 --2- 172.21.15.59:0/331261073 >> v2:172.21.15.76:3300/0 conn(0x7f554c0cd760 0x7f554c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 >> v1:172.21.15.76:6789/0 conn(0x7f554c15c3d0 legacy=0x7f554c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c160ab0 con 0x7f554c0b4690 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f55440019f0 con 0x7f554c0b4690 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 >> v1:172.21.15.59:6789/0 conn(0x7f554c0b4690 legacy=0x7f554c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55300039b0 0x7f5530003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f5530004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55300051c0 0x7f553000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1607d0 con 0x7f5530004800 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f553000c060 con 0x7f55300051c0 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.152+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f553000c260 con 0x7f55300039b0 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554a000640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f5530004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554a000640 1 -- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55300039b0 msgr2=0x7f5530003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:44.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554a000640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55300039b0 0x7f5530003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554a000640 1 -- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55300051c0 msgr2=0x7f553000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:44.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554b400640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55300051c0 0x7f553000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554a000640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55300051c0 0x7f553000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554a000640 1 -- 172.21.15.59:0/331261073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5530003520 con 0x7f5530004800 2024-04-23T14:31:44.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554b400640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55300051c0 0x7f553000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:44.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.153+0000 7f554b400640 1 -- 172.21.15.59:0/331261073 reap_dead start 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.154+0000 7f554a000640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f5530004c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5534009ac0 tx=0x7f5530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6069375db5f7e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.154+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534036070 con 0x7f5530004800 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.154+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5534031400 con 0x7f5530004800 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.154+0000 7f5549600640 1 -- 172.21.15.59:0/331261073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534009690 con 0x7f5530004800 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.155+0000 7f554be00640 1 -- 172.21.15.59:0/331261073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 msgr2=0x7f5530004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.155+0000 7f554be00640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f5530004c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5534009ac0 tx=0x7f5530005830 comp rx=0 tx=0).stop 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.155+0000 7f554be00640 1 -- 172.21.15.59:0/331261073 shutdown_connections 2024-04-23T14:31:44.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.155+0000 7f554be00640 1 --2- 172.21.15.59:0/331261073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f5530004c80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.155+0000 7f554be00640 1 -- 172.21.15.59:0/331261073 >> 172.21.15.59:0/331261073 conn(0x7f554c0b21a0 msgr2=0x7f554c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.155+0000 7f554be00640 1 -- 172.21.15.59:0/331261073 shutdown_connections 2024-04-23T14:31:44.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.155+0000 7f554be00640 1 -- 172.21.15.59:0/331261073 wait complete. 2024-04-23T14:31:44.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 Processor -- start 2024-04-23T14:31:44.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 -- start start 2024-04-23T14:31:44.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f554c1e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0c7db0 0x7f554c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f554c15f4a0 0x7f554c0c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f554c15c7b0 con 0x7f5530004800 2024-04-23T14:31:44.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0b4ae0 con 0x7f554c0c7db0 2024-04-23T14:31:44.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f554c15c930 con 0x7f554c15f4a0 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f554c1e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f554c1e5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47904/0 (socket says 172.21.15.59:47904) 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.158+0000 7f554aa00640 1 -- 172.21.15.59:0/512785167 learned_addr learned my addr 172.21.15.59:0/512785167 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554a000640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0c7db0 0x7f554c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554b400640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f554c15f4a0 0x7f554c0c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554aa00640 1 -- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f554c15f4a0 msgr2=0x7f554c0c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554aa00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f554c15f4a0 0x7f554c0c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554aa00640 1 -- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0c7db0 msgr2=0x7f554c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554aa00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0c7db0 0x7f554c15aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554aa00640 1 -- 172.21.15.59:0/512785167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c15db80 con 0x7f5530004800 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554b400640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f554c15f4a0 0x7f554c0c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554aa00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f554c1e5ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f553c00c530 tx=0x7f553c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f5542a00640 1 -- 172.21.15.59:0/512785167 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0052d0 con 0x7f5530004800 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f5542a00640 1 -- 172.21.15.59:0/512785167 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c005470 con 0x7f5530004800 2024-04-23T14:31:44.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f5542a00640 1 -- 172.21.15.59:0/512785167 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c012870 con 0x7f5530004800 2024-04-23T14:31:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.159+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c15de20 con 0x7f5530004800 2024-04-23T14:31:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.160+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c0c8570 con 0x7f5530004800 2024-04-23T14:31:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.161+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c04fd90 con 0x7f5530004800 2024-04-23T14:31:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.163+0000 7f5542a00640 1 -- 172.21.15.59:0/512785167 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f553c012a10 con 0x7f5530004800 2024-04-23T14:31:44.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.163+0000 7f5542a00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f552806c940 0x7f552806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:44.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.163+0000 7f5542a00640 1 -- 172.21.15.59:0/512785167 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f553c08b7d0 con 0x7f5530004800 2024-04-23T14:31:44.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.163+0000 7f554a000640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f552806c940 0x7f552806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:44.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.164+0000 7f554a000640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f552806c940 0x7f552806ee00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5534033000 tx=0x7f5534008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:44.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.167+0000 7f5542a00640 1 -- 172.21.15.59:0/512785167 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f553c059c70 con 0x7f5530004800 2024-04-23T14:31:44.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.415+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f554c06e400 con 0x7f5530004800 2024-04-23T14:31:44.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.416+0000 7f5542a00640 1 -- 172.21.15.59:0/512785167 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f553c059570 con 0x7f5530004800 2024-04-23T14:31:44.416 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:31:44.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f552806c940 msgr2=0x7f552806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f552806c940 0x7f552806ee00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5534033000 tx=0x7f5534008030 comp rx=0 tx=0).stop 2024-04-23T14:31:44.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 msgr2=0x7f554c1e5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:44.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f554c1e5ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f553c00c530 tx=0x7f553c00c9f0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 shutdown_connections 2024-04-23T14:31:44.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f552806c940 0x7f552806ee00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f554c15f4a0 0x7f554c0c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0c7db0 0x7f554c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 --2- 172.21.15.59:0/512785167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5530004800 0x7f554c1e5ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:44.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 >> 172.21.15.59:0/512785167 conn(0x7f554c0b21a0 msgr2=0x7f554c0b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:44.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 shutdown_connections 2024-04-23T14:31:44.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44.419+0000 7f554be00640 1 -- 172.21.15.59:0/512785167 wait complete. 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:31:44.439 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:31:44.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:44.440 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:31:44.490 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:44.490 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:44.490 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:44.490 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42444sec preferred_lft 42444sec 2024-04-23T14:31:44.491 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:31:44.492 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:44.492 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:44.492 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:44.492 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:31:44.493 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T14:31:44.493 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:44.493 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:44.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-04-23T14:31:44.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-04-23T14:31:44.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-04-23T14:31:44.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T14:31:44.493 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:44.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:44.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:44.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:44.979 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:31:45.048 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:31:45.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:45.052 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:31:45.110 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:31:45.111 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:31:45.111 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:31:45.111 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:31:45.111 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:31:45.112 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:45.112 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:45.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:31:45.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:31:45.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:31:45.112 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:45.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:45.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:45.900 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:31:45.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:45.957 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-04-23T14:31:46.012 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:46.012 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:46.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:46.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T14:31:46.012 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:46.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:46.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:46.208 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T14:31:46.208 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:46.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:46.516 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:46.516 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:46.516 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:31:46.516 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T14:31:46.516 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:46.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:46.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:46.728 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:31:46.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:31:46.728 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:46.773 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:31:46.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:31:46.773 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:46.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:31:46.937 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:31:46.991 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:31:46.992 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:31:46.993 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:31:46.993 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:31:46.993 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:46.993 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:46.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:46.994 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:31:47.041 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:31:47.041 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:31:47.089 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:31:47.166 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:47.217 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:31:47.217 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:31:47.217 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:31:47.217 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:31:47.217 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:31:47.217 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:47.218 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:47.219 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:47.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:47.220 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:31:47.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:31:47.284+0000 7fd72a435480 -1 init, newargv = 0x55ae91508020 newargc=15 2024-04-23T14:31:47.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[41461]: starting ceph client 2024-04-23T14:31:47.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[41461]: starting fuse 2024-04-23T14:31:48.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:31:48.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:48.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:31:48.463 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:48.558 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:31:48.558 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:31:48.558 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:31:48.558 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:31:48.558 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:48.559 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:48.560 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:48.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:48.561 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:31:48.662 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:31:48.662 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:31:48.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:31:48.662 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:31:48.662 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:31:48.662 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:31:48.662 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:31:48.662 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:31:48.663 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:31:48.664 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:31:48.664 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:48.664 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:31:48.664 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:31:48.836 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.41461.asok 2024-04-23T14:31:48.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:48.854 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41461.asok 2024-04-23T14:31:48.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:48.854 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41461.asok status 2024-04-23T14:31:49.090 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:31:49.090 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:31:49.090 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:31:49.090 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:31:49.090 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:31:49.090 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "41461", 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4650, 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 4650 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:31:49.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 419687339 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 419687339 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.4650 192.168.0.1:0/419687339", 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/419687339", 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 7, 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 20, 2024-04-23T14:31:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:31:49.093 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:31:49.093 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:31:49.093 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:31:49.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:31:49.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.315+0000 7f226b200640 1 Processor -- start 2024-04-23T14:31:49.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.316+0000 7f226b200640 1 -- start start 2024-04-23T14:31:49.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.317+0000 7f226b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f226c0b5ff0 0x7f226c0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.317+0000 7f226b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f226c0cf030 con 0x7f226c15e7c0 2024-04-23T14:31:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.317+0000 7f226b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f226c0b6a40 con 0x7f226c0d0d90 2024-04-23T14:31:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.317+0000 7f226b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f226c0b7000 con 0x7f226c0b5ff0 2024-04-23T14:31:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.318+0000 7f2269e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f226c15e7c0 0x7f226c0d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47644/0 (socket says 172.21.15.59:47644) 2024-04-23T14:31:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.318+0000 7f2269e00640 1 -- 172.21.15.59:0/1141729005 learned_addr learned my addr 172.21.15.59:0/1141729005 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.318+0000 7f2269400640 1 --2- 172.21.15.59:0/1141729005 >> v2:172.21.15.76:3300/0 conn(0x7f226c0b5ff0 0x7f226c0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.318+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419087349 0 0) 0x7f225c001110 con 0x7f226c15e7c0 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.318+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2250003520 con 0x7f226c15e7c0 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1976471287 0 0) 0x7f2250003520 con 0x7f226c15e7c0 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 >> v2:172.21.15.76:3300/0 conn(0x7f226c0b5ff0 msgr2=0x7f226c0b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 --2- 172.21.15.59:0/1141729005 >> v2:172.21.15.76:3300/0 conn(0x7f226c0b5ff0 0x7f226c0b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 >> v1:172.21.15.76:6789/0 conn(0x7f226c0d0d90 legacy=0x7f226c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c0b6d20 con 0x7f226c15e7c0 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f225c001840 con 0x7f226c15e7c0 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 >> v1:172.21.15.59:6789/0 conn(0x7f226c15e7c0 legacy=0x7f226c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250003a00 0x7f2250003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2250004800 0x7f2250004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22500051c0 0x7f225000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0b7000 con 0x7f2250004800 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f225000c0c0 con 0x7f22500051c0 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f225000c2c0 con 0x7f2250003a00 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f2269400640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2250004800 0x7f2250004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.319+0000 7f226a800640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22500051c0 0x7f225000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.320+0000 7f2269400640 1 -- 172.21.15.59:0/1141729005 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250003a00 msgr2=0x7f2250003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.320+0000 7f2269400640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250003a00 0x7f2250003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.320+0000 7f2269400640 1 -- 172.21.15.59:0/1141729005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22500051c0 msgr2=0x7f225000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.320+0000 7f2269400640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22500051c0 0x7f225000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.320+0000 7f2269400640 1 -- 172.21.15.59:0/1141729005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2250003520 con 0x7f2250004800 2024-04-23T14:31:49.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.320+0000 7f226a800640 1 -- 172.21.15.59:0/1141729005 reap_dead start 2024-04-23T14:31:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.321+0000 7f2269400640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2250004800 0x7f2250004c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ac0 tx=0x7f2250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c110e0caf1ebf08f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.321+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258036070 con 0x7f2250004800 2024-04-23T14:31:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.321+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2258031400 con 0x7f2250004800 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.321+0000 7f2268a00640 1 -- 172.21.15.59:0/1141729005 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258009690 con 0x7f2250004800 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.321+0000 7f226b200640 1 -- 172.21.15.59:0/1141729005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2250004800 msgr2=0x7f2250004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.321+0000 7f226b200640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2250004800 0x7f2250004c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ac0 tx=0x7f2250005830 comp rx=0 tx=0).stop 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.322+0000 7f226b200640 1 -- 172.21.15.59:0/1141729005 shutdown_connections 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.322+0000 7f226b200640 1 --2- 172.21.15.59:0/1141729005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2250004800 0x7f2250004c80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.322+0000 7f226b200640 1 -- 172.21.15.59:0/1141729005 >> 172.21.15.59:0/1141729005 conn(0x7f226c09d920 msgr2=0x7f226c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.322+0000 7f226b200640 1 -- 172.21.15.59:0/1141729005 shutdown_connections 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.322+0000 7f226b200640 1 -- 172.21.15.59:0/1141729005 wait complete. 2024-04-23T14:31:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.323+0000 7f226b200640 1 Processor -- start 2024-04-23T14:31:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.324+0000 7f226b200640 1 -- start start 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.324+0000 7f226b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250004800 0x7f226c0b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.324+0000 7f226b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c1f8750 0x7f226c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.324+0000 7f226b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c1f8bb0 0x7f226c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.324+0000 7f226b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0d1a80 con 0x7f226c1f8bb0 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.324+0000 7f226b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0cf620 con 0x7f226c1f8750 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.324+0000 7f226b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f226c0d1c00 con 0x7f2250004800 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c1f8bb0 0x7f226c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c1f8bb0 0x7f226c159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47914/0 (socket says 172.21.15.59:47914) 2024-04-23T14:31:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 -- 172.21.15.59:0/223263550 learned_addr learned my addr 172.21.15.59:0/223263550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f2269e00640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250004800 0x7f226c0b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f2269400640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c1f8750 0x7f226c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 -- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250004800 msgr2=0x7f226c0b9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250004800 0x7f226c0b9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 -- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c1f8750 msgr2=0x7f226c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c1f8750 0x7f226c159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.325+0000 7f226a800640 1 -- 172.21.15.59:0/223263550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c0cc380 con 0x7f226c1f8bb0 2024-04-23T14:31:49.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.326+0000 7f226a800640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c1f8bb0 0x7f226c159c60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f226c0a24f0 tx=0x7f226400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:49.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.326+0000 7f2262000640 1 -- 172.21.15.59:0/223263550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264018070 con 0x7f226c1f8bb0 2024-04-23T14:31:49.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.326+0000 7f2262000640 1 -- 172.21.15.59:0/223263550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2264003b70 con 0x7f226c1f8bb0 2024-04-23T14:31:49.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.326+0000 7f2262000640 1 -- 172.21.15.59:0/223263550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264005070 con 0x7f226c1f8bb0 2024-04-23T14:31:49.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.326+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c0b3730 con 0x7f226c1f8bb0 2024-04-23T14:31:49.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.326+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c0cc620 con 0x7f226c1f8bb0 2024-04-23T14:31:49.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.327+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226c04fd90 con 0x7f226c1f8bb0 2024-04-23T14:31:49.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.328+0000 7f2262000640 1 -- 172.21.15.59:0/223263550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f226401b070 con 0x7f226c1f8bb0 2024-04-23T14:31:49.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.329+0000 7f2262000640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224c06c920 0x7f224c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:49.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.329+0000 7f2262000640 1 -- 172.21.15.59:0/223263550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f226408b280 con 0x7f226c1f8bb0 2024-04-23T14:31:49.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.329+0000 7f2269e00640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224c06c920 0x7f224c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:49.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.330+0000 7f2269e00640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224c06c920 0x7f224c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f225c000fd0 tx=0x7f225c0016e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.334+0000 7f2262000640 1 -- 172.21.15.59:0/223263550 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22640015b0 con 0x7f226c1f8bb0 2024-04-23T14:31:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.585+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f226c0d1eb0 con 0x7f226c1f8bb0 2024-04-23T14:31:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.586+0000 7f2262000640 1 -- 172.21.15.59:0/223263550 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f2264059610 con 0x7f226c1f8bb0 2024-04-23T14:31:49.586 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224c06c920 msgr2=0x7f224c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224c06c920 0x7f224c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f225c000fd0 tx=0x7f225c0016e0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c1f8bb0 msgr2=0x7f226c159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c1f8bb0 0x7f226c159c60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f226c0a24f0 tx=0x7f226400dc00 comp rx=0 tx=0).stop 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 shutdown_connections 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224c06c920 0x7f224c06ede0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c1f8bb0 0x7f226c159c60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c1f8750 0x7f226c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 --2- 172.21.15.59:0/223263550 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2250004800 0x7f226c0b9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 >> 172.21.15.59:0/223263550 conn(0x7f226c09d920 msgr2=0x7f226c06cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:49.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 shutdown_connections 2024-04-23T14:31:49.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49.589+0000 7f226b200640 1 -- 172.21.15.59:0/223263550 wait complete. 2024-04-23T14:31:49.607 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:31:49.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:31:49.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:31:49.607 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-23T14:31:49.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-23T14:31:49.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:31:49.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:31:49.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:31:49.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-23T14:31:49.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:49.608 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:31:49.660 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:31:49.661 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42439sec preferred_lft 42439sec 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:49.662 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:31:49.663 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:49.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:49.663 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:31:49.745 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:31:49.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:49.750 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:31:49.804 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:31:49.804 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:31:49.804 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:31:49.804 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:31:49.804 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:31:49.805 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:49.805 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:49.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:31:49.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:31:49.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:31:49.806 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:50.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:50.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:50.030 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:31:50.080 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:31:50.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:50.082 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-04-23T14:31:50.137 INFO:teuthology.orchestra.run.smithi059.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-23T14:31:50.138 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:50.138 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:50.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:31:50.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-23T14:31:50.138 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:50.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:50.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:31:50.410 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 6e:0e:8d:f5:36:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:31:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:31:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::6c0e:8dff:fef5:367f/64 scope link 2024-04-23T14:31:50.411 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:50.414 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-23T14:31:50.494 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:50.494 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:50.494 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 6e:0e:8d:f5:36:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::6c0e:8dff:fef5:367f/64 scope link 2024-04-23T14:31:50.495 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:50.498 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-23T14:31:50.499 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:50.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:50.781 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:50.782 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:31:50.782 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.1 up 2024-04-23T14:31:50.782 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.1 master ceph-brx 2024-04-23T14:31:50.782 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:31:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:50.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:50.983 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-23T14:31:50.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:31:50.983 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-23T14:31:51.027 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-23T14:31:51.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:31:51.027 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-23T14:31:51.126 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:31:51.216 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:51.275 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:31:51.275 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:31:51.275 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:31:51.275 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:31:51.275 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:51.276 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:51.277 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:31:51.278 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:31:51.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:51.279 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:31:51.334 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:31:51.334 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-23T14:31:51.334 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-23T14:31:51.339 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:31:51.408 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:51.455 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:31:51.456 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:31:51.457 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:31:51.458 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:31:51.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:51.459 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:31:51.500 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:31:51.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi059.stderr:2024-04-23T14:31:51.516+0000 7f349b084480 -1 init, newargv = 0x560f55ea2420 newargc=15 2024-04-23T14:31:51.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi059.stderr:ceph-fuse[42043]: starting ceph client 2024-04-23T14:31:51.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi059.stderr:ceph-fuse[42043]: starting fuse 2024-04-23T14:31:52.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:31:52.701 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:52.760 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:31:52.761 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:31:52.761 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:31:52.761 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:31:52.761 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:31:52.761 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:31:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:31:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:31:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:31:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:31:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:31:52.762 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:31:52.763 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:52.763 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:31:52.763 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:31:52.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:31:52.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:31:52.763 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:52.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:52.764 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:31:52.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:52.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:52.764 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:52.764 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:31:52.765 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:31:52.765 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:31:52.765 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:31:52.765 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:31:52.765 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:31:52.765 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:31:52.765 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:52.766 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:31:52.766 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:31:52.766 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:31:52.766 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:31:52.766 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-23T14:31:52.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:52.768 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:31:52.828 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:31:52.828 INFO:teuthology.orchestra.run.smithi059.stdout:43 2024-04-23T14:31:52.828 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-23T14:31:52.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:31:52.828 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:31:52.828 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:31:52.828 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:31:52.828 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:31:52.828 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:31:52.829 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:31:52.830 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:31:52.830 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:31:52.830 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:31:52.830 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.2")) 2024-04-23T14:31:52.830 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:31:53.001 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.2.42043.asok 2024-04-23T14:31:53.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:53.021 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.42043.asok 2024-04-23T14:31:53.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:53.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.42043.asok status 2024-04-23T14:31:53.260 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "2", 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "42043", 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4665, 2024-04-23T14:31:53.261 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 4665 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1030502152 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:31:53.262 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1030502152 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.4665 192.168.0.2:0/1030502152", 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.2:0/1030502152", 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 7, 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 20, 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:31:53.263 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:31:53.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:31:53.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.488+0000 7f22c0c00640 1 Processor -- start 2024-04-23T14:31:53.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.488+0000 7f22c0c00640 1 -- start start 2024-04-23T14:31:53.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.489+0000 7f22c0c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f22bc15c3d0 0x7f22bc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 0x7f22bc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22c0c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22bc160210 con 0x7f22bc0b4690 2024-04-23T14:31:53.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22c0c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f22bc1604f0 con 0x7f22bc15c3d0 2024-04-23T14:31:53.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22c0c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc1607d0 con 0x7f22bc0cd590 2024-04-23T14:31:53.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22bbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f22bc0b4690 0x7f22bc06b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55916/0 (socket says 172.21.15.59:55916) 2024-04-23T14:31:53.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22bbe00640 1 -- 172.21.15.59:0/86147020 learned_addr learned my addr 172.21.15.59:0/86147020 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893094586 0 0) 0x7f22bc160210 con 0x7f22bc0b4690 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22baa00640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 0x7f22bc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22bb400640 1 --2- 172.21.15.59:0/86147020 >> v2:172.21.15.76:3300/0 conn(0x7f22bc15c3d0 0x7f22bc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.490+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22a0003520 con 0x7f22bc0b4690 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1231794913 0 0) 0x7f22a0003520 con 0x7f22bc0b4690 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 msgr2=0x7f22bc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 0x7f22bc0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 >> v2:172.21.15.76:3300/0 conn(0x7f22bc15c3d0 msgr2=0x7f22bc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 --2- 172.21.15.59:0/86147020 >> v2:172.21.15.76:3300/0 conn(0x7f22bc15c3d0 0x7f22bc0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc160ab0 con 0x7f22bc0b4690 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f22b40019f0 con 0x7f22bc0b4690 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 >> v1:172.21.15.59:6789/0 conn(0x7f22bc0b4690 legacy=0x7f22bc06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22a00039b0 0x7f22a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 0x7f22a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0004d40 0x7f22a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1604f0 con 0x7f22a0004d40 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1607d0 con 0x7f22a00039b0 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22a000ba70 con 0x7f22bc0cd590 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22bb400640 1 --2- 172.21.15.59:0/86147020 >> v2:172.21.15.76:3300/0 conn(0x7f22bc15c3d0 0x7f22bc0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.491+0000 7f22bbe00640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0004d40 0x7f22a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.492+0000 7f22baa00640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 0x7f22a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.492+0000 7f22bbe00640 1 -- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 msgr2=0x7f22a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:53.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.492+0000 7f22bb400640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22a00039b0 0x7f22a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.492+0000 7f22bbe00640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc0cd590 0x7f22a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.492+0000 7f22bbe00640 1 -- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22a00039b0 msgr2=0x7f22a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:53.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.492+0000 7f22bbe00640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22a00039b0 0x7f22a0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.492+0000 7f22bbe00640 1 -- 172.21.15.59:0/86147020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a0003520 con 0x7f22a0004d40 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.493+0000 7f22bbe00640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0004d40 0x7f22a00051c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22b400a7a0 tx=0x7f22a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a5b5e826120a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.493+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b400a3f0 con 0x7f22a0004d40 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.493+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b402f070 con 0x7f22a0004d40 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.493+0000 7f22ba000640 1 -- 172.21.15.59:0/86147020 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b402bc60 con 0x7f22a0004d40 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.494+0000 7f22c0c00640 1 -- 172.21.15.59:0/86147020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0004d40 msgr2=0x7f22a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.494+0000 7f22c0c00640 1 --2- 172.21.15.59:0/86147020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0004d40 0x7f22a00051c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22b400a7a0 tx=0x7f22a0005830 comp rx=0 tx=0).stop 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.494+0000 7f22bbe00640 1 -- 172.21.15.59:0/86147020 reap_dead start 2024-04-23T14:31:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.494+0000 7f22c0c00640 1 -- 172.21.15.59:0/86147020 shutdown_connections 2024-04-23T14:31:53.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.494+0000 7f22c0c00640 1 -- 172.21.15.59:0/86147020 >> 172.21.15.59:0/86147020 conn(0x7f22bc0b21a0 msgr2=0x7f22bc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:53.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.494+0000 7f22c0c00640 1 -- 172.21.15.59:0/86147020 shutdown_connections 2024-04-23T14:31:53.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.495+0000 7f22c0c00640 1 -- 172.21.15.59:0/86147020 wait complete. 2024-04-23T14:31:53.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.496+0000 7f22c0c00640 1 Processor -- start 2024-04-23T14:31:53.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.496+0000 7f22c0c00640 1 -- start start 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.497+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc1f77a0 0x7f22bc0997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.497+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22bc0c5820 0x7f22bc0ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.497+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22bc0ae5f0 0x7f22bc0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.497+0000 7f22c0c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc15c7b0 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.497+0000 7f22c0c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc0b4b00 con 0x7f22bc0c5820 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.497+0000 7f22c0c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc15c930 con 0x7f22bc1f77a0 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22bc0ae5f0 0x7f22bc0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22bc0ae5f0 0x7f22bc0c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52728/0 (socket says 172.21.15.59:52728) 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 -- 172.21.15.59:0/2630168097 learned_addr learned my addr 172.21.15.59:0/2630168097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bb400640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc1f77a0 0x7f22bc0997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22baa00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22bc0c5820 0x7f22bc0ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 -- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc1f77a0 msgr2=0x7f22bc0997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc1f77a0 0x7f22bc0997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 -- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22bc0c5820 msgr2=0x7f22bc0ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22bc0c5820 0x7f22bc0ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 -- 172.21.15.59:0/2630168097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc0ce790 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.498+0000 7f22bbe00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22bc0ae5f0 0x7f22bc0c7280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22b400a7a0 tx=0x7f22bc160c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.499+0000 7f22b3400640 1 -- 172.21.15.59:0/2630168097 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b4045070 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.499+0000 7f22b3400640 1 -- 172.21.15.59:0/2630168097 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b40019e0 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.499+0000 7f22b3400640 1 -- 172.21.15.59:0/2630168097 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b400a3f0 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.499+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc0cea70 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.499+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc0cf030 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.500+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc04fd90 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.501+0000 7f22b3400640 1 -- 172.21.15.59:0/2630168097 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22b4048070 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.502+0000 7f22b3400640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f229c06c920 0x7f229c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:53.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.503+0000 7f22b3400640 1 -- 172.21.15.59:0/2630168097 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22b40abc80 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.503+0000 7f22bb400640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f229c06c920 0x7f229c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.503+0000 7f22bb400640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f229c06c920 0x7f229c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22ac006fd0 tx=0x7f22ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:53.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.507+0000 7f22b3400640 1 -- 172.21.15.59:0/2630168097 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22b407a0a0 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.756+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f22bc06e400 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.757+0000 7f22b3400640 1 -- 172.21.15.59:0/2630168097 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f22b40799a0 con 0x7f22bc0ae5f0 2024-04-23T14:31:53.757 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.759+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f229c06c920 msgr2=0x7f229c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.759+0000 7f22c0c00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f229c06c920 0x7f229c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22ac006fd0 tx=0x7f22ac008040 comp rx=0 tx=0).stop 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.759+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22bc0ae5f0 msgr2=0x7f22bc0c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.759+0000 7f22c0c00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22bc0ae5f0 0x7f22bc0c7280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22b400a7a0 tx=0x7f22bc160c80 comp rx=0 tx=0).stop 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 shutdown_connections 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f229c06c920 0x7f229c06ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22bc0ae5f0 0x7f22bc0c7280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22bc0c5820 0x7f22bc0ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 --2- 172.21.15.59:0/2630168097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22bc1f77a0 0x7f22bc0997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:53.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 >> 172.21.15.59:0/2630168097 conn(0x7f22bc0b21a0 msgr2=0x7f22bc0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:53.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 shutdown_connections 2024-04-23T14:31:53.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:53.760+0000 7f22c0c00640 1 -- 172.21.15.59:0/2630168097 wait complete. 2024-04-23T14:31:53.778 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:31:53.779 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-23T14:31:53.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:53.779 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-04-23T14:31:53.836 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:53.836 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:53.836 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:53.836 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:53.836 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:53.836 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42592sec preferred_lft 42592sec 2024-04-23T14:31:53.837 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-04-23T14:31:53.838 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:53.838 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T14:31:53.839 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:53.839 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:53.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2024-04-23T14:31:53.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2024-04-23T14:31:53.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2024-04-23T14:31:53.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T14:31:53.839 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:54.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:54.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:54.359 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:31:54.432 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-04-23T14:31:54.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:54.435 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-04-23T14:31:54.496 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-04-23T14:31:54.496 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:31:54.496 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:31:54.496 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:31:54.496 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:31:54.497 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:54.497 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:54.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:31:54.498 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:31:54.498 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:31:54.498 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:55.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:55.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:55.319 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-04-23T14:31:55.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:55.374 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-04-23T14:31:55.430 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:55.430 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:55.430 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:55.430 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-23T14:31:55.430 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:55.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:55.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:55.607 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-23T14:31:55.607 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:55.608 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:55.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-23T14:31:55.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T14:31:55.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-23T14:31:55.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-23T14:31:55.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-23T14:31:55.608 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:55.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:55.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:55.915 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:55.916 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:55.916 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-04-23T14:31:55.916 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T14:31:55.916 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:56.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:56.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:56.116 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-23T14:31:56.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:31:56.116 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-23T14:31:56.161 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-23T14:31:56.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:31:56.162 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-23T14:31:56.252 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-04-23T14:31:56.331 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:56.380 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-04-23T14:31:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-04-23T14:31:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-04-23T14:31:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-04-23T14:31:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-04-23T14:31:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-04-23T14:31:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-04-23T14:31:56.382 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-04-23T14:31:56.383 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:56.384 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:56.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:56.385 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-04-23T14:31:56.435 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:31:56.435 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-23T14:31:56.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-04-23T14:31:56.507 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:56.551 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-04-23T14:31:56.552 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:56.553 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:56.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:56.555 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-04-23T14:31:56.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi076.stderr:2024-04-23T14:31:56.641+0000 7fed0711d480 -1 init, newargv = 0x55b3e45fb440 newargc=15 2024-04-23T14:31:56.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi076.stderr:ceph-fuse[38830]: starting ceph client 2024-04-23T14:31:56.653 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi076.stderr:ceph-fuse[38830]: starting fuse 2024-04-23T14:31:57.745 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-04-23T14:31:57.839 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:31:57.893 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-04-23T14:31:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-04-23T14:31:57.895 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:31:57.896 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:31:57.896 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:31:57.896 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:31:57.896 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-04-23T14:31:57.896 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:57.896 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:57.896 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-23T14:31:57.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:57.897 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-04-23T14:31:57.946 INFO:teuthology.orchestra.run.smithi076.stdout:42 2024-04-23T14:31:57.946 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:31:57.946 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:31:57.946 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-04-23T14:31:57.946 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-04-23T14:31:57.946 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-04-23T14:31:57.946 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-04-23T14:31:57.946 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:31:57.946 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-04-23T14:31:57.947 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-04-23T14:31:57.948 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:31:57.948 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:31:57.948 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.1")) 2024-04-23T14:31:57.948 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-04-23T14:31:58.112 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.1.38830.asok 2024-04-23T14:31:58.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:58.128 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38830.asok 2024-04-23T14:31:58.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:58.129 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38830.asok status 2024-04-23T14:31:58.355 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "1", 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "38830", 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-04-23T14:31:58.356 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4612, 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4612 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 818857543 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-23T14:31:58.357 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 818857543 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4612 192.168.0.1:0/818857543", 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/818857543", 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 7, 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 20, 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-04-23T14:31:58.358 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-23T14:31:58.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:31:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.594+0000 7f1e1d000640 1 Processor -- start 2024-04-23T14:31:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.594+0000 7f1e1d000640 1 -- start start 2024-04-23T14:31:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e1d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e180b5ff0 0x7f1e180b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e1d000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e180d0d90 0x7f1e180d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e1d000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e180cf030 con 0x7f1e1815e7c0 2024-04-23T14:31:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e1d000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1e180b7000 con 0x7f1e180b5ff0 2024-04-23T14:31:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e1d000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1e180d1720 con 0x7f1e180d0d90 2024-04-23T14:31:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e16a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e180b5ff0 0x7f1e180b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e16a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e180b5ff0 0x7f1e180b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52744/0 (socket says 172.21.15.59:52744) 2024-04-23T14:31:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.595+0000 7f1e16a00640 1 -- 172.21.15.59:0/1119322066 learned_addr learned my addr 172.21.15.59:0/1119322066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.596+0000 7f1e16a00640 1 -- 172.21.15.59:0/1119322066 >> v2:172.21.15.76:3300/0 conn(0x7f1e180d0d90 msgr2=0x7f1e180d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.596+0000 7f1e16a00640 1 --2- 172.21.15.59:0/1119322066 >> v2:172.21.15.76:3300/0 conn(0x7f1e180d0d90 0x7f1e180d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.596+0000 7f1e16a00640 1 -- 172.21.15.59:0/1119322066 >> v1:172.21.15.76:6789/0 conn(0x7f1e1815e7c0 legacy=0x7f1e180d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:31:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.596+0000 7f1e16a00640 1 -- 172.21.15.59:0/1119322066 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e180b6a40 con 0x7f1e180b5ff0 2024-04-23T14:31:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.597+0000 7f1e16a00640 1 --2- 172.21.15.59:0/1119322066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180b5ff0 0x7f1e180b6450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e1808da50 tx=0x7f1e0802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492f64bae84020b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.597+0000 7f1e16000640 1 -- 172.21.15.59:0/1119322066 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08036070 con 0x7f1e180b5ff0 2024-04-23T14:31:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.597+0000 7f1e16000640 1 -- 172.21.15.59:0/1119322066 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e08031600 con 0x7f1e180b5ff0 2024-04-23T14:31:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.597+0000 7f1e16000640 1 -- 172.21.15.59:0/1119322066 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080319d0 con 0x7f1e180b5ff0 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 -- 172.21.15.59:0/1119322066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180b5ff0 msgr2=0x7f1e180b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 --2- 172.21.15.59:0/1119322066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180b5ff0 0x7f1e180b6450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e1808da50 tx=0x7f1e0802a6f0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 -- 172.21.15.59:0/1119322066 shutdown_connections 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 --2- 172.21.15.59:0/1119322066 >> v2:172.21.15.76:3300/0 conn(0x7f1e180d0d90 0x7f1e180d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 --2- 172.21.15.59:0/1119322066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180b5ff0 0x7f1e180b6450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 -- 172.21.15.59:0/1119322066 >> 172.21.15.59:0/1119322066 conn(0x7f1e1809d920 msgr2=0x7f1e1808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 -- 172.21.15.59:0/1119322066 shutdown_connections 2024-04-23T14:31:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.598+0000 7f1e1d000640 1 -- 172.21.15.59:0/1119322066 wait complete. 2024-04-23T14:31:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.600+0000 7f1e1d000640 1 Processor -- start 2024-04-23T14:31:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.600+0000 7f1e1d000640 1 -- start start 2024-04-23T14:31:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.601+0000 7f1e1d000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e180b5ff0 0x7f1e180cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.601+0000 7f1e1d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180d0d90 0x7f1e18151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.601+0000 7f1e1d000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1815e7c0 0x7f1e180cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.601+0000 7f1e1d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180cf2b0 con 0x7f1e180d0d90 2024-04-23T14:31:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.601+0000 7f1e1d000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180cf620 con 0x7f1e180b5ff0 2024-04-23T14:31:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.601+0000 7f1e1d000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1e180d2800 con 0x7f1e1815e7c0 2024-04-23T14:31:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180d0d90 0x7f1e18151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180d0d90 0x7f1e18151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52750/0 (socket says 172.21.15.59:52750) 2024-04-23T14:31:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 -- 172.21.15.59:0/3739769197 learned_addr learned my addr 172.21.15.59:0/3739769197 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:31:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e17400640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e180b5ff0 0x7f1e180cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e17e00640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1815e7c0 0x7f1e180cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 -- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1815e7c0 msgr2=0x7f1e180cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1815e7c0 0x7f1e180cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 -- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e180b5ff0 msgr2=0x7f1e180cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e180b5ff0 0x7f1e180cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e16a00640 1 -- 172.21.15.59:0/3739769197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e180cb9b0 con 0x7f1e180d0d90 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.602+0000 7f1e17400640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e180b5ff0 0x7f1e180cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.603+0000 7f1e16a00640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180d0d90 0x7f1e18151130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009f40 tx=0x7f1e08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.603+0000 7f1e0f400640 1 -- 172.21.15.59:0/3739769197 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08041020 con 0x7f1e180d0d90 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.603+0000 7f1e0f400640 1 -- 172.21.15.59:0/3739769197 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e08037040 con 0x7f1e180d0d90 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.603+0000 7f1e0f400640 1 -- 172.21.15.59:0/3739769197 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08036070 con 0x7f1e180d0d90 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.603+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e180cbcf0 con 0x7f1e180d0d90 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.603+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e180cc530 con 0x7f1e180d0d90 2024-04-23T14:31:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.604+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e180ccfd0 con 0x7f1e180d0d90 2024-04-23T14:31:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.605+0000 7f1e0f400640 1 -- 172.21.15.59:0/3739769197 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e08044020 con 0x7f1e180d0d90 2024-04-23T14:31:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.606+0000 7f1e0f400640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1df406c940 0x7f1df406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:31:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.606+0000 7f1e0f400640 1 -- 172.21.15.59:0/3739769197 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e080ac730 con 0x7f1e180d0d90 2024-04-23T14:31:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.607+0000 7f1e17400640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1df406c940 0x7f1df406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:31:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.607+0000 7f1e17400640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1df406c940 0x7f1df406ee00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006fd0 tx=0x7f1e00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:31:58.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.611+0000 7f1e0f400640 1 -- 172.21.15.59:0/3739769197 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e0804b030 con 0x7f1e180d0d90 2024-04-23T14:31:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.856+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1e1804fd90 con 0x7f1e180d0d90 2024-04-23T14:31:58.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.857+0000 7f1e0f400640 1 -- 172.21.15.59:0/3739769197 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f1e180cbcf0 con 0x7f1e180d0d90 2024-04-23T14:31:58.857 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1df406c940 msgr2=0x7f1df406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1df406c940 0x7f1df406ee00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006fd0 tx=0x7f1e00008040 comp rx=0 tx=0).stop 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180d0d90 msgr2=0x7f1e18151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180d0d90 0x7f1e18151130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009f40 tx=0x7f1e08004a40 comp rx=0 tx=0).stop 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 shutdown_connections 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1df406c940 0x7f1df406ee00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1815e7c0 0x7f1e180cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e180d0d90 0x7f1e18151130 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 --2- 172.21.15.59:0/3739769197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e180b5ff0 0x7f1e180cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.859+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 >> 172.21.15.59:0/3739769197 conn(0x7f1e1809d920 msgr2=0x7f1e180b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.860+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 shutdown_connections 2024-04-23T14:31:58.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:31:58.860+0000 7f1e1d000640 1 -- 172.21.15.59:0/3739769197 wait complete. 2024-04-23T14:31:58.879 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:31:58.879 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:31:58.879 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-04-23T14:31:58.879 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-23T14:31:58.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-23T14:31:58.880 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:31:58.880 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:31:58.880 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:31:58.880 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-23T14:31:58.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:58.880 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:58.937 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42587sec preferred_lft 42587sec 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:31:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 56:90:ea:0a:70:32 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::3437:82ff:fef3:8146/64 scope link 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 56:90:ea:0a:70:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5490:eaff:fe0a:7032/64 scope link 2024-04-23T14:31:58.939 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:58.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:58.941 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:31:59.023 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-04-23T14:31:59.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:59.027 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-04-23T14:31:59.082 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-04-23T14:31:59.082 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:31:59.082 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:31:59.082 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:31:59.082 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:31:59.082 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:59.082 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:59.083 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:31:59.083 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:31:59.083 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:31:59.083 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:59.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:59.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:59.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:59.295 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-04-23T14:31:59.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-23T14:31:59.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:59.345 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-04-23T14:31:59.398 INFO:teuthology.orchestra.run.smithi076.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-23T14:31:59.399 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:59.399 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:59.399 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:31:59.400 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-23T14:31:59.400 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:59.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:59.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:31:59.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:59.578 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-23T14:31:59.651 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:59.651 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:59.651 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:59.651 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.651 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:59.651 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.652 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:31:59.652 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 7a:69:05:43:6c:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:31:59.652 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:31:59.652 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.652 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::7869:5ff:fe43:6c8d/64 scope link 2024-04-23T14:31:59.652 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:31:59.655 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 7a:69:05:43:6c:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::7869:5ff:fe43:6c8d/64 scope link 2024-04-23T14:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:31:59.739 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-23T14:31:59.739 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:31:59.740 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:31:59.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-23T14:31:59.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-23T14:31:59.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-23T14:31:59.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-23T14:31:59.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-23T14:31:59.740 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:31:59.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:31:59.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:31:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:00.008 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:32:00.008 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-23T14:32:00.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.1 up 2024-04-23T14:32:00.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.1 master ceph-brx 2024-04-23T14:32:00.008 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-23T14:32:00.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:32:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:32:00.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:32:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:00.207 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-23T14:32:00.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:00.208 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:00.249 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-23T14:32:00.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:32:00.250 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:00.342 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-04-23T14:32:00.420 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:00.469 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-04-23T14:32:00.469 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-04-23T14:32:00.469 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-04-23T14:32:00.470 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:00.471 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:00.472 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:00.472 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:00.472 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:00.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:00.473 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-04-23T14:32:00.521 INFO:teuthology.orchestra.run.smithi076.stdout:42 2024-04-23T14:32:00.521 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-23T14:32:00.521 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-23T14:32:00.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-04-23T14:32:00.593 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-04-23T14:32:00.633 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-04-23T14:32:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-4 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-5 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-6 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-7 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:00.635 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:32:00.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:00.636 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-04-23T14:32:00.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi076.stderr:2024-04-23T14:32:00.699+0000 7fb535f1e480 -1 init, newargv = 0x55b82e08e5f0 newargc=15 2024-04-23T14:32:00.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi076.stderr:ceph-fuse[39353]: starting ceph client 2024-04-23T14:32:00.710 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi076.stderr:ceph-fuse[39353]: starting fuse 2024-04-23T14:32:00.723 INFO:teuthology.orchestra.run.smithi076.stdout:42 2024-04-23T14:32:00.723 INFO:teuthology.orchestra.run.smithi076.stdout:43 2024-04-23T14:32:00.723 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-23T14:32:00.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:32:00.723 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-04-23T14:32:00.723 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-04-23T14:32:00.723 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-04-23T14:32:00.723 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-04-23T14:32:00.723 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:32:00.723 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-04-23T14:32:00.724 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-04-23T14:32:00.725 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:32:00.725 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-23T14:32:00.725 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.3")) 2024-04-23T14:32:00.725 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-04-23T14:32:00.907 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.3.39353.asok 2024-04-23T14:32:00.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-23T14:32:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:32:00.923 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39353.asok 2024-04-23T14:32:00.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:00.924 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39353.asok status 2024-04-23T14:32:01.153 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-23T14:32:01.153 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-04-23T14:32:01.153 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:32:01.153 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "3", 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "39353", 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4475, 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4475 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:32:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2006933447 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.2:0", 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2006933447 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4475 192.168.0.2:0/2006933447", 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.2:0/2006933447", 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 7, 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 20, 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:32:01.155 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-04-23T14:32:01.156 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-04-23T14:32:01.156 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-23T14:32:01.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.168 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:32:01.168 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:01.231 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:32:01.231 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:01.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:01.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.327 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:32:01.327 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:32:01.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:01.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:01.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.486 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-04-23T14:32:01.486 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:01.541 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-04-23T14:32:01.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:01.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.541 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:01.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.623 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-04-23T14:32:01.623 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:01.677 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-04-23T14:32:01.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:01.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:01.677 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:01.759 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-23T14:32:01.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.995+0000 7f5653200640 1 Processor -- start 2024-04-23T14:32:01.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.995+0000 7f5653200640 1 -- start start 2024-04-23T14:32:01.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.996+0000 7f5653200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f56540cd760 0x7f56540b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.996+0000 7f5653200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 0x7f56540cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.996+0000 7f5653200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5654160210 con 0x7f565415c3d0 2024-04-23T14:32:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.996+0000 7f5653200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f56541604f0 con 0x7f56540cd760 2024-04-23T14:32:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.997+0000 7f5653200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f56541607d0 con 0x7f56540b4690 2024-04-23T14:32:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.997+0000 7f5651e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f565415c3d0 0x7f56540cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35162/0 (socket says 172.21.15.59:35162) 2024-04-23T14:32:01.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.997+0000 7f5651e00640 1 -- 172.21.15.59:0/1885365657 learned_addr learned my addr 172.21.15.59:0/1885365657 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.997+0000 7f5651400640 1 --2- 172.21.15.59:0/1885365657 >> v2:172.21.15.76:3300/0 conn(0x7f56540cd760 0x7f56540b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.997+0000 7f5652800640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 0x7f56540cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.997+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1319886693 0 0) 0x7f5654160210 con 0x7f565415c3d0 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.997+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5638003520 con 0x7f565415c3d0 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1796052797 0 0) 0x7f5638003520 con 0x7f565415c3d0 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 msgr2=0x7f56540cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 0x7f56540cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 >> v2:172.21.15.76:3300/0 conn(0x7f56540cd760 msgr2=0x7f56540b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 --2- 172.21.15.59:0/1885365657 >> v2:172.21.15.76:3300/0 conn(0x7f56540cd760 0x7f56540b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:01.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5654160ab0 con 0x7f565415c3d0 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f5644001670 con 0x7f565415c3d0 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 >> v1:172.21.15.59:6789/0 conn(0x7f565415c3d0 legacy=0x7f56540cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 0x7f56380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5638003f30 0x7f5638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638004d40 0x7f56380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56541604f0 con 0x7f5638003f30 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f56541607d0 con 0x7f5638004d40 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f563800b9c0 con 0x7f56540b4690 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.998+0000 7f5651400640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5638003f30 0x7f5638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5652800640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638004d40 0x7f56380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5651e00640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 0x7f56380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5651400640 1 -- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 msgr2=0x7f56380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5651400640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56540b4690 0x7f56380039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5651400640 1 -- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638004d40 msgr2=0x7f56380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5651400640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638004d40 0x7f56380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5651400640 1 -- 172.21.15.59:0/1885365657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5638003520 con 0x7f5638003f30 2024-04-23T14:32:02.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:01.999+0000 7f5652800640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638004d40 0x7f56380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.000+0000 7f5651400640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5638003f30 0x7f5638004800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5640009ac0 tx=0x7f5638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f48916a8d70ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.000+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640036070 con 0x7f5638003f30 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.000+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5640031400 con 0x7f5638003f30 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.000+0000 7f5650a00640 1 -- 172.21.15.59:0/1885365657 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640009690 con 0x7f5638003f30 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.001+0000 7f5653200640 1 -- 172.21.15.59:0/1885365657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5638003f30 msgr2=0x7f5638004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.001+0000 7f5653200640 1 --2- 172.21.15.59:0/1885365657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5638003f30 0x7f5638004800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5640009ac0 tx=0x7f5638005830 comp rx=0 tx=0).stop 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.001+0000 7f5652800640 1 -- 172.21.15.59:0/1885365657 reap_dead start 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.001+0000 7f5653200640 1 -- 172.21.15.59:0/1885365657 shutdown_connections 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.001+0000 7f5653200640 1 -- 172.21.15.59:0/1885365657 >> 172.21.15.59:0/1885365657 conn(0x7f56540b21a0 msgr2=0x7f56540b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.001+0000 7f5653200640 1 -- 172.21.15.59:0/1885365657 shutdown_connections 2024-04-23T14:32:02.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.001+0000 7f5653200640 1 -- 172.21.15.59:0/1885365657 wait complete. 2024-04-23T14:32:02.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.003+0000 7f5653200640 1 Processor -- start 2024-04-23T14:32:02.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.003+0000 7f5653200640 1 -- start start 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5653200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638003f30 0x7f565415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5653200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5654157780 0x7f5654157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5653200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56540ae290 0x7f56540ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5653200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f565415c7b0 con 0x7f56540ae290 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5653200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f56540b4b00 con 0x7f5638003f30 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5653200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f565415c930 con 0x7f5654157780 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5652800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56540ae290 0x7f56540ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5652800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56540ae290 0x7f56540ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37748/0 (socket says 172.21.15.59:37748) 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.004+0000 7f5652800640 1 -- 172.21.15.59:0/813586360 learned_addr learned my addr 172.21.15.59:0/813586360 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5651e00640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638003f30 0x7f565415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5651400640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5654157780 0x7f5654157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5652800640 1 -- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5654157780 msgr2=0x7f5654157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5652800640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5654157780 0x7f5654157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5652800640 1 -- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638003f30 msgr2=0x7f565415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5652800640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638003f30 0x7f565415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5652800640 1 -- 172.21.15.59:0/813586360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5654157150 con 0x7f56540ae290 2024-04-23T14:32:02.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5651400640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5654157780 0x7f5654157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:02.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f5652800640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56540ae290 0x7f56540ae740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f565406ffe0 tx=0x7f564c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.005+0000 7f564a000640 1 -- 172.21.15.59:0/813586360 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c01c070 con 0x7f56540ae290 2024-04-23T14:32:02.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.006+0000 7f564a000640 1 -- 172.21.15.59:0/813586360 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f564c017400 con 0x7f56540ae290 2024-04-23T14:32:02.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.006+0000 7f564a000640 1 -- 172.21.15.59:0/813586360 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c0177f0 con 0x7f56540ae290 2024-04-23T14:32:02.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.006+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56541560b0 con 0x7f56540ae290 2024-04-23T14:32:02.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.006+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5654156670 con 0x7f56540ae290 2024-04-23T14:32:02.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.007+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f565404fd90 con 0x7f56540ae290 2024-04-23T14:32:02.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.008+0000 7f564a000640 1 -- 172.21.15.59:0/813586360 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f564c01f070 con 0x7f56540ae290 2024-04-23T14:32:02.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.009+0000 7f564a000640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f563406c920 0x7f563406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.009+0000 7f564a000640 1 -- 172.21.15.59:0/813586360 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f564c08baf0 con 0x7f56540ae290 2024-04-23T14:32:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.009+0000 7f5651e00640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f563406c920 0x7f563406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.010+0000 7f5651e00640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f563406c920 0x7f563406ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56540b2c90 tx=0x7f5644009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.014+0000 7f564a000640 1 -- 172.21.15.59:0/813586360 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f564c012070 con 0x7f56540ae290 2024-04-23T14:32:02.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.220+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f565406e400 con 0x7f56540ae290 2024-04-23T14:32:02.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.223+0000 7f564a000640 1 -- 172.21.15.59:0/813586360 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f564c059f10 con 0x7f56540ae290 2024-04-23T14:32:02.223 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:02.223 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.225+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f563406c920 msgr2=0x7f563406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f563406c920 0x7f563406ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56540b2c90 tx=0x7f5644009040 comp rx=0 tx=0).stop 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56540ae290 msgr2=0x7f56540ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56540ae290 0x7f56540ae740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f565406ffe0 tx=0x7f564c00f4a0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 shutdown_connections 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f563406c920 0x7f563406ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5654157780 0x7f5654157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56540ae290 0x7f56540ae740 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 --2- 172.21.15.59:0/813586360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5638003f30 0x7f565415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 >> 172.21.15.59:0/813586360 conn(0x7f56540b21a0 msgr2=0x7f56540cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 shutdown_connections 2024-04-23T14:32:02.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.226+0000 7f5653200640 1 -- 172.21.15.59:0/813586360 wait complete. 2024-04-23T14:32:02.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:02.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:02.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.467+0000 7f1f42400640 1 Processor -- start 2024-04-23T14:32:02.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.467+0000 7f1f42400640 1 -- start start 2024-04-23T14:32:02.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.468+0000 7f1f42400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 0x7f1f3c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.468+0000 7f1f42400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f3c15d810 0x7f1f3c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.468+0000 7f1f42400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f3c15cab0 con 0x7f1f3c1567e0 2024-04-23T14:32:02.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.468+0000 7f1f42400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1f3c160e40 con 0x7f1f3c15d810 2024-04-23T14:32:02.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.468+0000 7f1f42400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c15cd30 con 0x7f1f3c0d0d10 2024-04-23T14:32:02.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.469+0000 7f1f41000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1f3c1567e0 0x7f1f3c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35176/0 (socket says 172.21.15.59:35176) 2024-04-23T14:32:02.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.469+0000 7f1f41000640 1 -- 172.21.15.59:0/521355467 learned_addr learned my addr 172.21.15.59:0/521355467 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.469+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224186128 0 0) 0x7f1f2c001110 con 0x7f1f3c1567e0 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.469+0000 7f1f41a00640 1 --2- 172.21.15.59:0/521355467 >> v2:172.21.15.76:3300/0 conn(0x7f1f3c15d810 0x7f1f3c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.469+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f20003520 con 0x7f1f3c1567e0 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.469+0000 7f1f3be00640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 0x7f1f3c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1231867709 0 0) 0x7f1f20003520 con 0x7f1f3c1567e0 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 msgr2=0x7f1f3c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 0x7f1f3c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 >> v2:172.21.15.76:3300/0 conn(0x7f1f3c15d810 msgr2=0x7f1f3c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 --2- 172.21.15.59:0/521355467 >> v2:172.21.15.76:3300/0 conn(0x7f1f3c15d810 0x7f1f3c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c15d010 con 0x7f1f3c1567e0 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1f2c001840 con 0x7f1f3c1567e0 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 >> v1:172.21.15.59:6789/0 conn(0x7f1f3c1567e0 legacy=0x7f1f3c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f200039b0 0x7f1f20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 0x7f1f20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f20004d40 0x7f1f200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c160e40 con 0x7f1f200039b0 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c15cd30 con 0x7f1f20004d40 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2000baf0 con 0x7f1f3c0d0d10 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f41000640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f200039b0 0x7f1f20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.470+0000 7f1f3be00640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 0x7f1f20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.471+0000 7f1f41a00640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f20004d40 0x7f1f200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.471+0000 7f1f41000640 1 -- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 msgr2=0x7f1f20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.471+0000 7f1f41000640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c0d0d10 0x7f1f20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.471+0000 7f1f41000640 1 -- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f20004d40 msgr2=0x7f1f200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.471+0000 7f1f41000640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f20004d40 0x7f1f200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.471+0000 7f1f41000640 1 -- 172.21.15.59:0/521355467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f20003520 con 0x7f1f200039b0 2024-04-23T14:32:02.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.471+0000 7f1f41a00640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f20004d40 0x7f1f200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.472+0000 7f1f41000640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f200039b0 0x7f1f20003dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c2f0 tx=0x7f1f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a293105ebfe5cac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.473+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c035070 con 0x7f1f200039b0 2024-04-23T14:32:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.473+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2c0058d0 con 0x7f1f200039b0 2024-04-23T14:32:02.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.473+0000 7f1f3b400640 1 -- 172.21.15.59:0/521355467 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c007e30 con 0x7f1f200039b0 2024-04-23T14:32:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.474+0000 7f1f42400640 1 -- 172.21.15.59:0/521355467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f200039b0 msgr2=0x7f1f20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.474+0000 7f1f42400640 1 --2- 172.21.15.59:0/521355467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f200039b0 0x7f1f20003dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c2f0 tx=0x7f1f20005830 comp rx=0 tx=0).stop 2024-04-23T14:32:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.474+0000 7f1f41a00640 1 -- 172.21.15.59:0/521355467 reap_dead start 2024-04-23T14:32:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.474+0000 7f1f42400640 1 -- 172.21.15.59:0/521355467 shutdown_connections 2024-04-23T14:32:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.474+0000 7f1f42400640 1 -- 172.21.15.59:0/521355467 >> 172.21.15.59:0/521355467 conn(0x7f1f3c06bdf0 msgr2=0x7f1f3c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.474+0000 7f1f42400640 1 -- 172.21.15.59:0/521355467 shutdown_connections 2024-04-23T14:32:02.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.474+0000 7f1f42400640 1 -- 172.21.15.59:0/521355467 wait complete. 2024-04-23T14:32:02.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.476+0000 7f1f42400640 1 Processor -- start 2024-04-23T14:32:02.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.476+0000 7f1f42400640 1 -- start start 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f42400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200039b0 0x7f1f3c13f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f42400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c1556a0 0x7f1f3c0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f42400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f3c0afdb0 0x7f1f3c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f42400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c156d40 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f42400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c156bc0 con 0x7f1f200039b0 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f42400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c156ec0 con 0x7f1f3c1556a0 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f41a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f3c0afdb0 0x7f1f3c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f41a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f3c0afdb0 0x7f1f3c1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37766/0 (socket says 172.21.15.59:37766) 2024-04-23T14:32:02.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.477+0000 7f1f41a00640 1 -- 172.21.15.59:0/3885430513 learned_addr learned my addr 172.21.15.59:0/3885430513 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:02.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f3be00640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c1556a0 0x7f1f3c0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f41a00640 1 -- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c1556a0 msgr2=0x7f1f3c0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f41a00640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c1556a0 0x7f1f3c0b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f41a00640 1 -- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200039b0 msgr2=0x7f1f3c13f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f41000640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200039b0 0x7f1f3c13f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f41a00640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200039b0 0x7f1f3c13f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f41a00640 1 -- 172.21.15.59:0/3885430513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c0b19b0 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f41a00640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f3c0afdb0 0x7f1f3c1529c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c15a6a0 tx=0x7f1f34009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f38c00640 1 -- 172.21.15.59:0/3885430513 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34019070 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f38c00640 1 -- 172.21.15.59:0/3885430513 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f34007ce0 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f38c00640 1 -- 172.21.15.59:0/3885430513 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34014640 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.478+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c0b1c90 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.479+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c0b0230 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.480+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3c155a80 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.481+0000 7f1f38c00640 1 -- 172.21.15.59:0/3885430513 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f3401c070 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.482+0000 7f1f38c00640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f1806c890 0x7f1f1806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.482+0000 7f1f38c00640 1 -- 172.21.15.59:0/3885430513 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f3408b680 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.482+0000 7f1f41000640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f1806c890 0x7f1f1806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.483+0000 7f1f41000640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f1806c890 0x7f1f1806ed50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c032000 tx=0x7f1f2c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.487+0000 7f1f38c00640 1 -- 172.21.15.59:0/3885430513 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f34059ab0 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.689+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f3c04fd90 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.690+0000 7f1f38c00640 1 -- 172.21.15.59:0/3885430513 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1f340593b0 con 0x7f1f3c0afdb0 2024-04-23T14:32:02.691 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.692+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f1806c890 msgr2=0x7f1f1806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.692+0000 7f1f42400640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f1806c890 0x7f1f1806ed50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c032000 tx=0x7f1f2c009030 comp rx=0 tx=0).stop 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.692+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f3c0afdb0 msgr2=0x7f1f3c1529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.692+0000 7f1f42400640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f3c0afdb0 0x7f1f3c1529c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c15a6a0 tx=0x7f1f34009d40 comp rx=0 tx=0).stop 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 shutdown_connections 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f1806c890 0x7f1f1806ed50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f3c1556a0 0x7f1f3c0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f3c0afdb0 0x7f1f3c1529c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 --2- 172.21.15.59:0/3885430513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200039b0 0x7f1f3c13f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 >> 172.21.15.59:0/3885430513 conn(0x7f1f3c06bdf0 msgr2=0x7f1f3c15dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 shutdown_connections 2024-04-23T14:32:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.693+0000 7f1f42400640 1 -- 172.21.15.59:0/3885430513 wait complete. 2024-04-23T14:32:02.714 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.925+0000 7f8019e00640 1 Processor -- start 2024-04-23T14:32:02.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.926+0000 7f8019e00640 1 -- start start 2024-04-23T14:32:02.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.927+0000 7f8019e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8014071c70 0x7f8014163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.927+0000 7f8019e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8014163320 con 0x7f801415d300 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.927+0000 7f8019e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80140720d0 con 0x7f80140d0600 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.927+0000 7f8019e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8014163650 con 0x7f8014071c70 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.928+0000 7f8018a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f801415d300 0x7f8014167180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35180/0 (socket says 172.21.15.59:35180) 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.928+0000 7f8018a00640 1 -- 172.21.15.59:0/3251418736 learned_addr learned my addr 172.21.15.59:0/3251418736 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.928+0000 7f8013e00640 1 --2- 172.21.15.59:0/3251418736 >> v2:172.21.15.59:3300/0 conn(0x7f8014071c70 0x7f8014163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.928+0000 7f8013400640 1 -- 172.21.15.59:0/3251418736 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 953338604 0 0) 0x7f8014163320 con 0x7f801415d300 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.928+0000 7f8013e00640 1 -- 172.21.15.59:0/3251418736 >> v1:172.21.15.76:6789/0 conn(0x7f80140d0600 legacy=0x7f801406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.928+0000 7f8013e00640 1 -- 172.21.15.59:0/3251418736 >> v1:172.21.15.59:6789/0 conn(0x7f801415d300 legacy=0x7f8014167180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.928+0000 7f8013e00640 1 -- 172.21.15.59:0/3251418736 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8014072380 con 0x7f8014071c70 2024-04-23T14:32:02.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.929+0000 7f8013e00640 1 --2- 172.21.15.59:0/3251418736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014071c70 0x7f8014163ae0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80040097a0 tx=0x7f800402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cdd6a28058a159 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8013400640 1 -- 172.21.15.59:0/3251418736 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004037070 con 0x7f8014071c70 2024-04-23T14:32:02.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8013400640 1 -- 172.21.15.59:0/3251418736 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8004032440 con 0x7f8014071c70 2024-04-23T14:32:02.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8013400640 1 -- 172.21.15.59:0/3251418736 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004032810 con 0x7f8014071c70 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8019e00640 1 -- 172.21.15.59:0/3251418736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014071c70 msgr2=0x7f8014163ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8019e00640 1 --2- 172.21.15.59:0/3251418736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014071c70 0x7f8014163ae0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80040097a0 tx=0x7f800402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8019e00640 1 -- 172.21.15.59:0/3251418736 shutdown_connections 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8019e00640 1 --2- 172.21.15.59:0/3251418736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8014071c70 0x7f8014163ae0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.930+0000 7f8019e00640 1 -- 172.21.15.59:0/3251418736 >> 172.21.15.59:0/3251418736 conn(0x7f801406bdf0 msgr2=0x7f801406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.931+0000 7f8019e00640 1 -- 172.21.15.59:0/3251418736 shutdown_connections 2024-04-23T14:32:02.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.931+0000 7f8019e00640 1 -- 172.21.15.59:0/3251418736 wait complete. 2024-04-23T14:32:02.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.932+0000 7f8019e00640 1 Processor -- start 2024-04-23T14:32:02.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.932+0000 7f8019e00640 1 -- start start 2024-04-23T14:32:02.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.933+0000 7f8019e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8014071c70 0x7f80140b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.933+0000 7f8019e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80140d0600 0x7f801409c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.933+0000 7f8019e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801415d300 0x7f801409cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.933+0000 7f8019e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8014164020 con 0x7f80140d0600 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.933+0000 7f8019e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8014072940 con 0x7f801415d300 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.933+0000 7f8019e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f801406e8f0 con 0x7f8014071c70 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80140d0600 0x7f801409c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80140d0600 0x7f801409c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37780/0 (socket says 172.21.15.59:37780) 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 -- 172.21.15.59:0/3256793238 learned_addr learned my addr 172.21.15.59:0/3256793238 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8018a00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8014071c70 0x7f80140b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8019400640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801415d300 0x7f801409cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 -- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8014071c70 msgr2=0x7f80140b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8014071c70 0x7f80140b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 -- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801415d300 msgr2=0x7f801409cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801415d300 0x7f801409cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.934+0000 7f8013e00640 1 -- 172.21.15.59:0/3256793238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801409d3c0 con 0x7f80140d0600 2024-04-23T14:32:02.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.935+0000 7f8013e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80140d0600 0x7f801409c7e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80040098c0 tx=0x7f80040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.935+0000 7f8010c00640 1 -- 172.21.15.59:0/3256793238 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004041070 con 0x7f80140d0600 2024-04-23T14:32:02.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.935+0000 7f8010c00640 1 -- 172.21.15.59:0/3256793238 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800403c870 con 0x7f80140d0600 2024-04-23T14:32:02.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.935+0000 7f8010c00640 1 -- 172.21.15.59:0/3256793238 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004037040 con 0x7f80140d0600 2024-04-23T14:32:02.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.935+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801409d6a0 con 0x7f80140d0600 2024-04-23T14:32:02.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.935+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80140b0940 con 0x7f80140d0600 2024-04-23T14:32:02.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.936+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801404fd90 con 0x7f80140d0600 2024-04-23T14:32:02.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.937+0000 7f8010c00640 1 -- 172.21.15.59:0/3256793238 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8004044070 con 0x7f80140d0600 2024-04-23T14:32:02.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.939+0000 7f8010c00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:02.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.939+0000 7f8010c00640 1 -- 172.21.15.59:0/3256793238 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80040ac690 con 0x7f80140d0600 2024-04-23T14:32:02.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.939+0000 7f8018a00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:02.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.940+0000 7f8018a00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ff806c8f0 0x7f7ff806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8008000f60 tx=0x7f8008007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:02.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:02.943+0000 7f8010c00640 1 -- 172.21.15.59:0/3256793238 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f800407abe0 con 0x7f80140d0600 2024-04-23T14:32:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.148+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f801415c9e0 con 0x7f80140d0600 2024-04-23T14:32:03.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.149+0000 7f8010c00640 1 -- 172.21.15.59:0/3256793238 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f800407a4e0 con 0x7f80140d0600 2024-04-23T14:32:03.151 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ff806c8f0 msgr2=0x7f7ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ff806c8f0 0x7f7ff806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8008000f60 tx=0x7f8008007c00 comp rx=0 tx=0).stop 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80140d0600 msgr2=0x7f801409c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80140d0600 0x7f801409c7e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80040098c0 tx=0x7f80040046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 shutdown_connections 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801415d300 0x7f801409cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.152+0000 7f8019e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80140d0600 0x7f801409c7e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.153+0000 7f8019e00640 1 --2- 172.21.15.59:0/3256793238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8014071c70 0x7f80140b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.153+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 >> 172.21.15.59:0/3256793238 conn(0x7f801406bdf0 msgr2=0x7f80140ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:03.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.153+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 shutdown_connections 2024-04-23T14:32:03.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.153+0000 7f8019e00640 1 -- 172.21.15.59:0/3256793238 wait complete. 2024-04-23T14:32:03.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:03.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.389+0000 7fd6e3200640 1 Processor -- start 2024-04-23T14:32:03.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.389+0000 7fd6e3200640 1 -- start start 2024-04-23T14:32:03.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.391+0000 7fd6e3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd6e40d0d10 0x7fd6e4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.391+0000 7fd6e3200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e415cab0 con 0x7fd6e41567e0 2024-04-23T14:32:03.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.391+0000 7fd6e3200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e4160e40 con 0x7fd6e415d810 2024-04-23T14:32:03.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.391+0000 7fd6e3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e415cd30 con 0x7fd6e40d0d10 2024-04-23T14:32:03.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.391+0000 7fd6e1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd6e41567e0 0x7fd6e40d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35196/0 (socket says 172.21.15.59:35196) 2024-04-23T14:32:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.391+0000 7fd6e1e00640 1 -- 172.21.15.59:0/1598421578 learned_addr learned my addr 172.21.15.59:0/1598421578 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.391+0000 7fd6e1400640 1 --2- 172.21.15.59:0/1598421578 >> v2:172.21.15.59:3300/0 conn(0x7fd6e40d0d10 0x7fd6e4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.392+0000 7fd6e1400640 1 -- 172.21.15.59:0/1598421578 >> v1:172.21.15.76:6789/0 conn(0x7fd6e415d810 legacy=0x7fd6e4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.392+0000 7fd6e1400640 1 -- 172.21.15.59:0/1598421578 >> v1:172.21.15.59:6789/0 conn(0x7fd6e41567e0 legacy=0x7fd6e40d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.392+0000 7fd6e1400640 1 -- 172.21.15.59:0/1598421578 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e415d010 con 0x7fd6e40d0d10 2024-04-23T14:32:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.393+0000 7fd6e1400640 1 --2- 172.21.15.59:0/1598421578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4156bc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00097a0 tx=0x7fd6d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7fd6cb67e20e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.393+0000 7fd6e0a00640 1 -- 172.21.15.59:0/1598421578 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0037070 con 0x7fd6e40d0d10 2024-04-23T14:32:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.393+0000 7fd6e0a00640 1 -- 172.21.15.59:0/1598421578 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d0032440 con 0x7fd6e40d0d10 2024-04-23T14:32:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.393+0000 7fd6e0a00640 1 -- 172.21.15.59:0/1598421578 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0032810 con 0x7fd6e40d0d10 2024-04-23T14:32:03.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.394+0000 7fd6e3200640 1 -- 172.21.15.59:0/1598421578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 msgr2=0x7fd6e4156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.394+0000 7fd6e3200640 1 --2- 172.21.15.59:0/1598421578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4156bc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00097a0 tx=0x7fd6d002a230 comp rx=0 tx=0).stop 2024-04-23T14:32:03.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.394+0000 7fd6e3200640 1 -- 172.21.15.59:0/1598421578 shutdown_connections 2024-04-23T14:32:03.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.394+0000 7fd6e3200640 1 --2- 172.21.15.59:0/1598421578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4156bc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.394+0000 7fd6e3200640 1 -- 172.21.15.59:0/1598421578 >> 172.21.15.59:0/1598421578 conn(0x7fd6e406bdf0 msgr2=0x7fd6e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:03.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.394+0000 7fd6e3200640 1 -- 172.21.15.59:0/1598421578 shutdown_connections 2024-04-23T14:32:03.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.394+0000 7fd6e3200640 1 -- 172.21.15.59:0/1598421578 wait complete. 2024-04-23T14:32:03.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.396+0000 7fd6e3200640 1 Processor -- start 2024-04-23T14:32:03.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.396+0000 7fd6e3200640 1 -- start start 2024-04-23T14:32:03.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e41567e0 0x7fd6e41e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e415d810 0x7fd6e4159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41609a0 con 0x7fd6e40d0d10 2024-04-23T14:32:03.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4160820 con 0x7fd6e415d810 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e4160b20 con 0x7fd6e41567e0 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37790/0 (socket says 172.21.15.59:37790) 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.397+0000 7fd6e1e00640 1 -- 172.21.15.59:0/245035625 learned_addr learned my addr 172.21.15.59:0/245035625 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e2800640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e415d810 0x7fd6e4159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e1400640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e41567e0 0x7fd6e41e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e1e00640 1 -- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e41567e0 msgr2=0x7fd6e41e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e1e00640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e41567e0 0x7fd6e41e5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e1e00640 1 -- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e415d810 msgr2=0x7fd6e4159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e1e00640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e415d810 0x7fd6e4159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e1e00640 1 -- 172.21.15.59:0/245035625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e409c1c0 con 0x7fd6e40d0d10 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e1e00640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4146aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c480 tx=0x7fd6d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6da000640 1 -- 172.21.15.59:0/245035625 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4017070 con 0x7fd6e40d0d10 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6da000640 1 -- 172.21.15.59:0/245035625 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d4005ce0 con 0x7fd6e40d0d10 2024-04-23T14:32:03.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6da000640 1 -- 172.21.15.59:0/245035625 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40072f0 con 0x7fd6e40d0d10 2024-04-23T14:32:03.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.398+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e409c4a0 con 0x7fd6e40d0d10 2024-04-23T14:32:03.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.399+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e409ca60 con 0x7fd6e40d0d10 2024-04-23T14:32:03.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.400+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e404fd90 con 0x7fd6e40d0d10 2024-04-23T14:32:03.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.401+0000 7fd6da000640 1 -- 172.21.15.59:0/245035625 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6d401a070 con 0x7fd6e40d0d10 2024-04-23T14:32:03.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.402+0000 7fd6da000640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd6c406c8f0 0x7fd6c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.402+0000 7fd6da000640 1 -- 172.21.15.59:0/245035625 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6d408ade0 con 0x7fd6e40d0d10 2024-04-23T14:32:03.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.402+0000 7fd6e1400640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd6c406c8f0 0x7fd6c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.403+0000 7fd6e1400640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd6c406c8f0 0x7fd6c406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0034000 tx=0x7fd6d002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:03.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.406+0000 7fd6da000640 1 -- 172.21.15.59:0/245035625 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6e415cab0 con 0x7fd6e40d0d10 2024-04-23T14:32:03.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.606+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6e40ad820 con 0x7fd6e40d0d10 2024-04-23T14:32:03.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.607+0000 7fd6da000640 1 -- 172.21.15.59:0/245035625 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd6d4058c50 con 0x7fd6e40d0d10 2024-04-23T14:32:03.608 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.609+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd6c406c8f0 msgr2=0x7fd6c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.609+0000 7fd6e3200640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd6c406c8f0 0x7fd6c406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0034000 tx=0x7fd6d002e000 comp rx=0 tx=0).stop 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.609+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 msgr2=0x7fd6e4146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.609+0000 7fd6e3200640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4146aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c480 tx=0x7fd6d400c940 comp rx=0 tx=0).stop 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 shutdown_connections 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd6c406c8f0 0x7fd6c406edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e41567e0 0x7fd6e41e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e415d810 0x7fd6e4159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 --2- 172.21.15.59:0/245035625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6e40d0d10 0x7fd6e4146aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 >> 172.21.15.59:0/245035625 conn(0x7fd6e406bdf0 msgr2=0x7fd6e415ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:03.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 shutdown_connections 2024-04-23T14:32:03.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.610+0000 7fd6e3200640 1 -- 172.21.15.59:0/245035625 wait complete. 2024-04-23T14:32:03.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:03.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.835+0000 7f44aba00640 1 Processor -- start 2024-04-23T14:32:03.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.835+0000 7f44aba00640 1 -- start start 2024-04-23T14:32:03.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.837+0000 7f44aba00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44a40b4550 0x7f44a4163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.837+0000 7f44aba00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a4160d40 con 0x7f44a4160960 2024-04-23T14:32:03.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.837+0000 7f44aba00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a4166de0 con 0x7f44a41644b0 2024-04-23T14:32:03.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.837+0000 7f44aba00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f44a4164270 con 0x7f44a40b4550 2024-04-23T14:32:03.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.838+0000 7f44a9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44a40b4550 0x7f44a4163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.838+0000 7f44a9000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f44a4160960 0x7f44a40b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35208/0 (socket says 172.21.15.59:35208) 2024-04-23T14:32:03.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.838+0000 7f44a9000640 1 -- 172.21.15.59:0/3980405007 learned_addr learned my addr 172.21.15.59:0/3980405007 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:03.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.838+0000 7f44a9a00640 1 -- 172.21.15.59:0/3980405007 >> v1:172.21.15.76:6789/0 conn(0x7f44a41644b0 legacy=0x7f44a406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.838+0000 7f44a9a00640 1 -- 172.21.15.59:0/3980405007 >> v1:172.21.15.59:6789/0 conn(0x7f44a4160960 legacy=0x7f44a40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.838+0000 7f44a9a00640 1 -- 172.21.15.59:0/3980405007 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a41670c0 con 0x7f44a40b4550 2024-04-23T14:32:03.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.839+0000 7f44a9a00640 1 --2- 172.21.15.59:0/3980405007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a40b4550 0x7f44a4163d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f449c009d90 tx=0x7f449c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ecf0109f579880 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:03.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.840+0000 7f44a3400640 1 -- 172.21.15.59:0/3980405007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c007740 con 0x7f44a40b4550 2024-04-23T14:32:03.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.840+0000 7f44a3400640 1 -- 172.21.15.59:0/3980405007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449c0078e0 con 0x7f44a40b4550 2024-04-23T14:32:03.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.840+0000 7f44a3400640 1 -- 172.21.15.59:0/3980405007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c007cb0 con 0x7f44a40b4550 2024-04-23T14:32:03.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.840+0000 7f44aba00640 1 -- 172.21.15.59:0/3980405007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a40b4550 msgr2=0x7f44a4163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.840+0000 7f44aba00640 1 --2- 172.21.15.59:0/3980405007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a40b4550 0x7f44a4163d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f449c009d90 tx=0x7f449c02a6f0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.841+0000 7f44aba00640 1 -- 172.21.15.59:0/3980405007 shutdown_connections 2024-04-23T14:32:03.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.841+0000 7f44aba00640 1 --2- 172.21.15.59:0/3980405007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a40b4550 0x7f44a4163d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.841+0000 7f44aba00640 1 -- 172.21.15.59:0/3980405007 >> 172.21.15.59:0/3980405007 conn(0x7f44a4067ea0 msgr2=0x7f44a406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:03.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.841+0000 7f44aba00640 1 -- 172.21.15.59:0/3980405007 shutdown_connections 2024-04-23T14:32:03.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.841+0000 7f44aba00640 1 -- 172.21.15.59:0/3980405007 wait complete. 2024-04-23T14:32:03.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.842+0000 7f44aba00640 1 Processor -- start 2024-04-23T14:32:03.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.843+0000 7f44aba00640 1 -- start start 2024-04-23T14:32:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.843+0000 7f44aba00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a40b4550 0x7f44a415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.843+0000 7f44aba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4160960 0x7f44a40d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.843+0000 7f44aba00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a41644b0 0x7f44a40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44aba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41668e0 con 0x7f44a4160960 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44aba00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4167680 con 0x7f44a41644b0 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44aba00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f44a4166a60 con 0x7f44a40b4550 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4160960 0x7f44a40d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4160960 0x7f44a40d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37802/0 (socket says 172.21.15.59:37802) 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 -- 172.21.15.59:0/1484498184 learned_addr learned my addr 172.21.15.59:0/1484498184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a9000640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a40b4550 0x7f44a415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a9a00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a41644b0 0x7f44a40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 -- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a40b4550 msgr2=0x7f44a415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a40b4550 0x7f44a415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 -- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a41644b0 msgr2=0x7f44a40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a41644b0 0x7f44a40cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a3e00640 1 -- 172.21.15.59:0/1484498184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a40d0ea0 con 0x7f44a4160960 2024-04-23T14:32:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.844+0000 7f44a9a00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a41644b0 0x7f44a40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.845+0000 7f44a3e00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4160960 0x7f44a40d0960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f449000c400 tx=0x7f449000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.845+0000 7f44a0c00640 1 -- 172.21.15.59:0/1484498184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490017070 con 0x7f44a4160960 2024-04-23T14:32:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.845+0000 7f44a0c00640 1 -- 172.21.15.59:0/1484498184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44900052d0 con 0x7f44a4160960 2024-04-23T14:32:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.845+0000 7f44a0c00640 1 -- 172.21.15.59:0/1484498184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490003830 con 0x7f44a4160960 2024-04-23T14:32:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.845+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a40cf8c0 con 0x7f44a4160960 2024-04-23T14:32:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.845+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a40cfe80 con 0x7f44a4160960 2024-04-23T14:32:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.846+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a4068a80 con 0x7f44a4160960 2024-04-23T14:32:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.847+0000 7f44a0c00640 1 -- 172.21.15.59:0/1484498184 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f449001a070 con 0x7f44a4160960 2024-04-23T14:32:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.848+0000 7f44a0c00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f448406c920 0x7f448406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.848+0000 7f44a0c00640 1 -- 172.21.15.59:0/1484498184 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f449008ae40 con 0x7f44a4160960 2024-04-23T14:32:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.849+0000 7f44a9000640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f448406c920 0x7f448406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:03.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.849+0000 7f44a9000640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f448406c920 0x7f448406ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4498000f60 tx=0x7f4498009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:03.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:03.853+0000 7f44a0c00640 1 -- 172.21.15.59:0/1484498184 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4490059260 con 0x7f44a4160960 2024-04-23T14:32:04.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.056+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44a40cca30 con 0x7f44a4160960 2024-04-23T14:32:04.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.058+0000 7f44a0c00640 1 -- 172.21.15.59:0/1484498184 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f4490058b60 con 0x7f44a4160960 2024-04-23T14:32:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:04.059 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:04.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f448406c920 msgr2=0x7f448406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f448406c920 0x7f448406ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4498000f60 tx=0x7f4498009040 comp rx=0 tx=0).stop 2024-04-23T14:32:04.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4160960 msgr2=0x7f44a40d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4160960 0x7f44a40d0960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f449000c400 tx=0x7f449000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 shutdown_connections 2024-04-23T14:32:04.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f448406c920 0x7f448406ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a41644b0 0x7f44a40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4160960 0x7f44a40d0960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 --2- 172.21.15.59:0/1484498184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a40b4550 0x7f44a415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 >> 172.21.15.59:0/1484498184 conn(0x7f44a4067ea0 msgr2=0x7f44a4072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:04.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 shutdown_connections 2024-04-23T14:32:04.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.061+0000 7f44aba00640 1 -- 172.21.15.59:0/1484498184 wait complete. 2024-04-23T14:32:04.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:04.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:04.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.281+0000 7f99ca400640 1 Processor -- start 2024-04-23T14:32:04.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.281+0000 7f99ca400640 1 -- start start 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99ca400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f99c4146030 0x7f99c415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99ca400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f99c4160980 0x7f99c40b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99ca400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c4146410 con 0x7f99c40b06a0 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99ca400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f99c40b10f0 con 0x7f99c4160980 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99ca400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f99c40b13d0 con 0x7f99c4146030 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99c3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f99c40b06a0 0x7f99c4160270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35212/0 (socket says 172.21.15.59:35212) 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99c3e00640 1 -- 172.21.15.59:0/618054672 learned_addr learned my addr 172.21.15.59:0/618054672 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99c9a00640 1 --2- 172.21.15.59:0/618054672 >> v2:172.21.15.59:3300/0 conn(0x7f99c4160980 0x7f99c40b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.283+0000 7f99c9000640 1 --2- 172.21.15.59:0/618054672 >> v2:172.21.15.76:3300/0 conn(0x7f99c4146030 0x7f99c415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.284+0000 7f99c3400640 1 -- 172.21.15.59:0/618054672 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80406889 0 0) 0x7f99c4146410 con 0x7f99c40b06a0 2024-04-23T14:32:04.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.284+0000 7f99c3400640 1 -- 172.21.15.59:0/618054672 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99ac003520 con 0x7f99c40b06a0 2024-04-23T14:32:04.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.284+0000 7f99c9a00640 1 -- 172.21.15.59:0/618054672 >> v2:172.21.15.76:3300/0 conn(0x7f99c4146030 msgr2=0x7f99c415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:04.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.284+0000 7f99c9a00640 1 --2- 172.21.15.59:0/618054672 >> v2:172.21.15.76:3300/0 conn(0x7f99c4146030 0x7f99c415ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.284+0000 7f99c9a00640 1 -- 172.21.15.59:0/618054672 >> v1:172.21.15.59:6789/0 conn(0x7f99c40b06a0 legacy=0x7f99c4160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.284+0000 7f99c9a00640 1 -- 172.21.15.59:0/618054672 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c40b16b0 con 0x7f99c4160980 2024-04-23T14:32:04.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.285+0000 7f99c9a00640 1 --2- 172.21.15.59:0/618054672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c40b0ae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99c40d23b0 tx=0x7f99bc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af9348fd1f3596 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:04.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.286+0000 7f99c3400640 1 -- 172.21.15.59:0/618054672 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007740 con 0x7f99c4160980 2024-04-23T14:32:04.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.286+0000 7f99c3400640 1 -- 172.21.15.59:0/618054672 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc0078e0 con 0x7f99c4160980 2024-04-23T14:32:04.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.286+0000 7f99c3400640 1 -- 172.21.15.59:0/618054672 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007cb0 con 0x7f99c4160980 2024-04-23T14:32:04.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.286+0000 7f99ca400640 1 -- 172.21.15.59:0/618054672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 msgr2=0x7f99c40b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.286+0000 7f99ca400640 1 --2- 172.21.15.59:0/618054672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c40b0ae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f99c40d23b0 tx=0x7f99bc02a680 comp rx=0 tx=0).stop 2024-04-23T14:32:04.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.287+0000 7f99ca400640 1 -- 172.21.15.59:0/618054672 shutdown_connections 2024-04-23T14:32:04.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.287+0000 7f99ca400640 1 --2- 172.21.15.59:0/618054672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c40b0ae0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.287+0000 7f99ca400640 1 --2- 172.21.15.59:0/618054672 >> v2:172.21.15.76:3300/0 conn(0x7f99c4146030 0x7f99c415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.287+0000 7f99ca400640 1 -- 172.21.15.59:0/618054672 >> 172.21.15.59:0/618054672 conn(0x7f99c40b6910 msgr2=0x7f99c40b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:04.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.287+0000 7f99ca400640 1 -- 172.21.15.59:0/618054672 shutdown_connections 2024-04-23T14:32:04.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.287+0000 7f99ca400640 1 -- 172.21.15.59:0/618054672 wait complete. 2024-04-23T14:32:04.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.288+0000 7f99ca400640 1 Processor -- start 2024-04-23T14:32:04.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.288+0000 7f99ca400640 1 -- start start 2024-04-23T14:32:04.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.289+0000 7f99ca400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99c40b06a0 0x7f99c40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.289+0000 7f99ca400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99c4146030 0x7f99c409dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.289+0000 7f99ca400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c409e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.289+0000 7f99ca400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4160fc0 con 0x7f99c4160980 2024-04-23T14:32:04.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.289+0000 7f99ca400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f99c40b1c70 con 0x7f99c4146030 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.289+0000 7f99ca400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f99c4161140 con 0x7f99c40b06a0 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c409e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c409e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37816/0 (socket says 172.21.15.59:37816) 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 -- 172.21.15.59:0/508149928 learned_addr learned my addr 172.21.15.59:0/508149928 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c3e00640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99c4146030 0x7f99c409dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9000640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99c40b06a0 0x7f99c40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 -- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99c40b06a0 msgr2=0x7f99c40cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99c40b06a0 0x7f99c40cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 -- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99c4146030 msgr2=0x7f99c409dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99c4146030 0x7f99c409dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.290+0000 7f99c9a00640 1 -- 172.21.15.59:0/508149928 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c40cd760 con 0x7f99c4160980 2024-04-23T14:32:04.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.291+0000 7f99c9a00640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c409e0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99bc02ac70 tx=0x7f99bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:04.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.291+0000 7f99c0c00640 1 -- 172.21.15.59:0/508149928 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc03f070 con 0x7f99c4160980 2024-04-23T14:32:04.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.291+0000 7f99c0c00640 1 -- 172.21.15.59:0/508149928 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc031bc0 con 0x7f99c4160980 2024-04-23T14:32:04.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.291+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c409cf60 con 0x7f99c4160980 2024-04-23T14:32:04.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.291+0000 7f99c0c00640 1 -- 172.21.15.59:0/508149928 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc009450 con 0x7f99c4160980 2024-04-23T14:32:04.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.291+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c409c6c0 con 0x7f99c4160980 2024-04-23T14:32:04.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.292+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c404fd90 con 0x7f99c4160980 2024-04-23T14:32:04.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.293+0000 7f99c0c00640 1 -- 172.21.15.59:0/508149928 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99bc03b410 con 0x7f99c4160980 2024-04-23T14:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.294+0000 7f99c0c00640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99a406c8f0 0x7f99a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.294+0000 7f99c0c00640 1 -- 172.21.15.59:0/508149928 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99bc0ad270 con 0x7f99c4160980 2024-04-23T14:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.295+0000 7f99c9000640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99a406c8f0 0x7f99a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.295+0000 7f99c9000640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99a406c8f0 0x7f99a406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99c4070ac0 tx=0x7f99b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.299+0000 7f99c0c00640 1 -- 172.21.15.59:0/508149928 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99c40cd760 con 0x7f99c4160980 2024-04-23T14:32:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.506+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99c4142ff0 con 0x7f99c4160980 2024-04-23T14:32:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.507+0000 7f99c0c00640 1 -- 172.21.15.59:0/508149928 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f99bc07b040 con 0x7f99c4160980 2024-04-23T14:32:04.508 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99a406c8f0 msgr2=0x7f99a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99a406c8f0 0x7f99a406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99c4070ac0 tx=0x7f99b8008040 comp rx=0 tx=0).stop 2024-04-23T14:32:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 msgr2=0x7f99c409e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c409e0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99bc02ac70 tx=0x7f99bc004a40 comp rx=0 tx=0).stop 2024-04-23T14:32:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 shutdown_connections 2024-04-23T14:32:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99a406c8f0 0x7f99a406edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99c4160980 0x7f99c409e0e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99c4146030 0x7f99c409dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 --2- 172.21.15.59:0/508149928 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99c40b06a0 0x7f99c40cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 >> 172.21.15.59:0/508149928 conn(0x7f99c40b6910 msgr2=0x7f99c40b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 shutdown_connections 2024-04-23T14:32:04.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.509+0000 7f99ca400640 1 -- 172.21.15.59:0/508149928 wait complete. 2024-04-23T14:32:04.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:04.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.717+0000 7f1fc0a00640 1 Processor -- start 2024-04-23T14:32:04.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.717+0000 7f1fc0a00640 1 -- start start 2024-04-23T14:32:04.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.718+0000 7f1fc0a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1fbc0d16c0 0x7f1fbc0d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.718+0000 7f1fc0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 0x7f1fbc0b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.718+0000 7f1fc0a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fbc15eee0 con 0x7f1fbc15eb00 2024-04-23T14:32:04.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.718+0000 7f1fc0a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1fbc0d3350 con 0x7f1fbc0d16c0 2024-04-23T14:32:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.718+0000 7f1fc0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc0d3630 con 0x7f1fbc0b78b0 2024-04-23T14:32:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.719+0000 7f1fb9e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1fbc15eb00 0x7f1fbc0d2890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35214/0 (socket says 172.21.15.59:35214) 2024-04-23T14:32:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.719+0000 7f1fb9e00640 1 -- 172.21.15.59:0/93944903 learned_addr learned my addr 172.21.15.59:0/93944903 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.719+0000 7f1fb9400640 1 --2- 172.21.15.59:0/93944903 >> v2:172.21.15.76:3300/0 conn(0x7f1fbc0d16c0 0x7f1fbc0d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.719+0000 7f1fba800640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 0x7f1fbc0b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.719+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 915935052 0 0) 0x7f1fbc15eee0 con 0x7f1fbc15eb00 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.719+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa0003520 con 0x7f1fbc15eb00 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.719+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2409819865 0 0) 0x7f1fa0003520 con 0x7f1fbc15eb00 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 msgr2=0x7f1fbc0b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 0x7f1fbc0b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 >> v2:172.21.15.76:3300/0 conn(0x7f1fbc0d16c0 msgr2=0x7f1fbc0d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 --2- 172.21.15.59:0/93944903 >> v2:172.21.15.76:3300/0 conn(0x7f1fbc0d16c0 0x7f1fbc0d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc0d3910 con 0x7f1fbc15eb00 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1fac001690 con 0x7f1fbc15eb00 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 >> v1:172.21.15.59:6789/0 conn(0x7f1fbc15eb00 legacy=0x7f1fbc0d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fa00039b0 0x7f1fa0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 0x7f1fa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc0d3350 con 0x7f1fa0004d40 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc0d3630 con 0x7f1fa00039b0 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa000b9c0 con 0x7f1fbc0b78b0 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fba800640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb9e00640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fa00039b0 0x7f1fa0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.720+0000 7f1fb9400640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 0x7f1fa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.721+0000 7f1fba800640 1 -- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 msgr2=0x7f1fa0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.721+0000 7f1fba800640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 0x7f1fa0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.721+0000 7f1fba800640 1 -- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fa00039b0 msgr2=0x7f1fa0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.721+0000 7f1fba800640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fa00039b0 0x7f1fa0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.721+0000 7f1fba800640 1 -- 172.21.15.59:0/93944903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa0003520 con 0x7f1fa0004d40 2024-04-23T14:32:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.721+0000 7f1fb9400640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc0b78b0 0x7f1fa0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:04.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.722+0000 7f1fba800640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fa00051c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4007390 tx=0x7f1fa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16eb91a52bc824b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:04.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.722+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4009be0 con 0x7f1fa0004d40 2024-04-23T14:32:04.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.722+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb4036600 con 0x7f1fa0004d40 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.722+0000 7f1fb8a00640 1 -- 172.21.15.59:0/93944903 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40369d0 con 0x7f1fa0004d40 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.723+0000 7f1fc0a00640 1 -- 172.21.15.59:0/93944903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 msgr2=0x7f1fa00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.723+0000 7f1fc0a00640 1 --2- 172.21.15.59:0/93944903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fa00051c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4007390 tx=0x7f1fa0005830 comp rx=0 tx=0).stop 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.723+0000 7f1fba800640 1 -- 172.21.15.59:0/93944903 reap_dead start 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.723+0000 7f1fc0a00640 1 -- 172.21.15.59:0/93944903 shutdown_connections 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.723+0000 7f1fc0a00640 1 -- 172.21.15.59:0/93944903 >> 172.21.15.59:0/93944903 conn(0x7f1fbc067b50 msgr2=0x7f1fbc067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.723+0000 7f1fc0a00640 1 -- 172.21.15.59:0/93944903 shutdown_connections 2024-04-23T14:32:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.723+0000 7f1fc0a00640 1 -- 172.21.15.59:0/93944903 wait complete. 2024-04-23T14:32:04.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.725+0000 7f1fc0a00640 1 Processor -- start 2024-04-23T14:32:04.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.725+0000 7f1fc0a00640 1 -- start start 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fc0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fbc15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fc0a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fbc1f5430 0x7f1fbc1612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fc0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc1f58b0 0x7f1fbc1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fc0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc0ba650 con 0x7f1fa0004d40 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fc0a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc0d43f0 con 0x7f1fbc1f5430 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fc0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc0d2fa0 con 0x7f1fbc1f58b0 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fb9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fbc15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fb9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fbc15cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37836/0 (socket says 172.21.15.59:37836) 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.726+0000 7f1fb9e00640 1 -- 172.21.15.59:0/260750984 learned_addr learned my addr 172.21.15.59:0/260750984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fba800640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc1f58b0 0x7f1fbc1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fb9400640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fbc1f5430 0x7f1fbc1612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fb9e00640 1 -- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc1f58b0 msgr2=0x7f1fbc1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fb9e00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc1f58b0 0x7f1fbc1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fb9e00640 1 -- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fbc1f5430 msgr2=0x7f1fbc1612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fb9e00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fbc1f5430 0x7f1fbc1612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fb9e00640 1 -- 172.21.15.59:0/260750984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc1f51d0 con 0x7f1fa0004d40 2024-04-23T14:32:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.727+0000 7f1fb9e00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fbc15cca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1fac007340 tx=0x7f1fac007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.728+0000 7f1fb2000640 1 -- 172.21.15.59:0/260750984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00f9c0 con 0x7f1fa0004d40 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.728+0000 7f1fb2000640 1 -- 172.21.15.59:0/260750984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac018070 con 0x7f1fa0004d40 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.728+0000 7f1fb2000640 1 -- 172.21.15.59:0/260750984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00fdc0 con 0x7f1fa0004d40 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.728+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc0ac780 con 0x7f1fa0004d40 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.728+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc0acd40 con 0x7f1fa0004d40 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.729+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc068a80 con 0x7f1fa0004d40 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.730+0000 7f1fb2000640 1 -- 172.21.15.59:0/260750984 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fac01d460 con 0x7f1fa0004d40 2024-04-23T14:32:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.731+0000 7f1fb2000640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f9c06c920 0x7f1f9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.731+0000 7f1fb2000640 1 -- 172.21.15.59:0/260750984 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1fac08bc80 con 0x7f1fa0004d40 2024-04-23T14:32:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.731+0000 7f1fb9400640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f9c06c920 0x7f1f9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:04.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.732+0000 7f1fb9400640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f9c06c920 0x7f1f9c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc1f77c0 tx=0x7f1fa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:04.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.736+0000 7f1fb2000640 1 -- 172.21.15.59:0/260750984 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fac05a010 con 0x7f1fa0004d40 2024-04-23T14:32:04.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.948+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1fbc0d52c0 con 0x7f1fa0004d40 2024-04-23T14:32:04.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.949+0000 7f1fb2000640 1 -- 172.21.15.59:0/260750984 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f1fac059910 con 0x7f1fa0004d40 2024-04-23T14:32:04.950 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f9c06c920 msgr2=0x7f1f9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f9c06c920 0x7f1f9c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc1f77c0 tx=0x7f1fa8008040 comp rx=0 tx=0).stop 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 msgr2=0x7f1fbc15cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fbc15cca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1fac007340 tx=0x7f1fac007800 comp rx=0 tx=0).stop 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 shutdown_connections 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f9c06c920 0x7f1f9c06ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fbc1f58b0 0x7f1fbc1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fbc1f5430 0x7f1fbc1612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 --2- 172.21.15.59:0/260750984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0004d40 0x7f1fbc15cca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 >> 172.21.15.59:0/260750984 conn(0x7f1fbc067b50 msgr2=0x7f1fbc0cb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.953+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 shutdown_connections 2024-04-23T14:32:04.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:04.954+0000 7f1fc0a00640 1 -- 172.21.15.59:0/260750984 wait complete. 2024-04-23T14:32:04.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.175+0000 7f9cfa800640 1 Processor -- start 2024-04-23T14:32:05.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.176+0000 7f9cfa800640 1 -- start start 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.177+0000 7f9cfa800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 0x7f9cf40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.177+0000 7f9cfa800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf40b5da0 con 0x7f9cf40b4f20 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.177+0000 7f9cfa800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cf40d0a20 con 0x7f9cf415ed20 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.177+0000 7f9cfa800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf415f100 con 0x7f9cf40d15b0 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.177+0000 7f9cf9e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9cf40b4f20 0x7f9cf40cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35218/0 (socket says 172.21.15.59:35218) 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.177+0000 7f9cf9e00640 1 -- 172.21.15.59:0/3336269012 learned_addr learned my addr 172.21.15.59:0/3336269012 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf8a00640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 0x7f9cf40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436669874 0 0) 0x7f9cf40b5da0 con 0x7f9cf40b4f20 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cdc003520 con 0x7f9cf40b4f20 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2261421693 0 0) 0x7f9cdc003520 con 0x7f9cf40b4f20 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 msgr2=0x7f9cf40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 0x7f9cf40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 >> v1:172.21.15.76:6789/0 conn(0x7f9cf415ed20 legacy=0x7f9cf40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf40d2ed0 con 0x7f9cf40b4f20 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9cec0019f0 con 0x7f9cf40b4f20 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 >> v1:172.21.15.59:6789/0 conn(0x7f9cf40b4f20 legacy=0x7f9cf40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cdc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.178+0000 7f9cf3e00640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 0x7f9cdc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf3e00640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cdc004d40 0x7f9cdc0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00bba0 con 0x7f9cdc0039e0 2024-04-23T14:32:05.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc00bd50 con 0x7f9cdc004d40 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9cdc00bf50 con 0x7f9cf40d15b0 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf9400640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cdc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf8a00640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 0x7f9cdc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf9400640 1 -- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 msgr2=0x7f9cdc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf9400640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40d15b0 0x7f9cdc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf9400640 1 -- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cdc004d40 msgr2=0x7f9cdc0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf9400640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cdc004d40 0x7f9cdc0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.179+0000 7f9cf9400640 1 -- 172.21.15.59:0/3336269012 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cdc003520 con 0x7f9cdc0039e0 2024-04-23T14:32:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.180+0000 7f9cf9400640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cdc003e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8001750 tx=0x7f9cdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb963c9bfaaab57 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8001370 con 0x7f9cdc0039e0 2024-04-23T14:32:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce80052d0 con 0x7f9cdc0039e0 2024-04-23T14:32:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cf3e00640 1 -- 172.21.15.59:0/3336269012 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce802abe0 con 0x7f9cdc0039e0 2024-04-23T14:32:05.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cfa800640 1 -- 172.21.15.59:0/3336269012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 msgr2=0x7f9cdc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cfa800640 1 --2- 172.21.15.59:0/3336269012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cdc003e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8001750 tx=0x7f9cdc005830 comp rx=0 tx=0).stop 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cf9e00640 1 -- 172.21.15.59:0/3336269012 reap_dead start 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cfa800640 1 -- 172.21.15.59:0/3336269012 shutdown_connections 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cfa800640 1 -- 172.21.15.59:0/3336269012 >> 172.21.15.59:0/3336269012 conn(0x7f9cf408bf90 msgr2=0x7f9cf40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cfa800640 1 -- 172.21.15.59:0/3336269012 shutdown_connections 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.181+0000 7f9cfa800640 1 -- 172.21.15.59:0/3336269012 wait complete. 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.183+0000 7f9cfa800640 1 Processor -- start 2024-04-23T14:32:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.183+0000 7f9cfa800640 1 -- start start 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cf40b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cfa800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40c96c0 0x7f9cf4158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cfa800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf406cc40 0x7f9cf406d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cfa800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf40b3600 con 0x7f9cdc0039e0 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cfa800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf40d3490 con 0x7f9cf406cc40 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cfa800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf40b3780 con 0x7f9cf40c96c0 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cf9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cf40b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.184+0000 7f9cf9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cf40b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37844/0 (socket says 172.21.15.59:37844) 2024-04-23T14:32:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9400640 1 -- 172.21.15.59:0/875068125 learned_addr learned my addr 172.21.15.59:0/875068125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9400640 1 -- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40c96c0 msgr2=0x7f9cf4158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9e00640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf406cc40 0x7f9cf406d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf8a00640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40c96c0 0x7f9cf4158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9400640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40c96c0 0x7f9cf4158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9400640 1 -- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf406cc40 msgr2=0x7f9cf406d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9400640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf406cc40 0x7f9cf406d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9400640 1 -- 172.21.15.59:0/875068125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf40b8ec0 con 0x7f9cdc0039e0 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9e00640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf406cc40 0x7f9cf406d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.185+0000 7f9cf9400640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cf40b87e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8001870 tx=0x7f9ce8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.186+0000 7f9cf1600640 1 -- 172.21.15.59:0/875068125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8040020 con 0x7f9cdc0039e0 2024-04-23T14:32:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.186+0000 7f9cf1600640 1 -- 172.21.15.59:0/875068125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce8007680 con 0x7f9cdc0039e0 2024-04-23T14:32:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.186+0000 7f9cf1600640 1 -- 172.21.15.59:0/875068125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8007a30 con 0x7f9cdc0039e0 2024-04-23T14:32:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.186+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf40b91a0 con 0x7f9cdc0039e0 2024-04-23T14:32:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.186+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf40b9760 con 0x7f9cdc0039e0 2024-04-23T14:32:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.187+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf4073590 con 0x7f9cdc0039e0 2024-04-23T14:32:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.188+0000 7f9cf1600640 1 -- 172.21.15.59:0/875068125 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ce8043020 con 0x7f9cdc0039e0 2024-04-23T14:32:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.189+0000 7f9cf1600640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9cd406c8f0 0x7f9cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.189+0000 7f9cf1600640 1 -- 172.21.15.59:0/875068125 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9ce80abba0 con 0x7f9cdc0039e0 2024-04-23T14:32:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.189+0000 7f9cf8a00640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9cd406c8f0 0x7f9cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.190+0000 7f9cf8a00640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9cd406c8f0 0x7f9cd406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9cf406e410 tx=0x7f9ce0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.193+0000 7f9cf1600640 1 -- 172.21.15.59:0/875068125 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9cf4073590 con 0x7f9cdc0039e0 2024-04-23T14:32:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.393+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9cf404fd90 con 0x7f9cdc0039e0 2024-04-23T14:32:05.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.393+0000 7f9cf1600640 1 -- 172.21.15.59:0/875068125 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f9ce804a030 con 0x7f9cdc0039e0 2024-04-23T14:32:05.394 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9cd406c8f0 msgr2=0x7f9cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9cd406c8f0 0x7f9cd406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9cf406e410 tx=0x7f9ce0008040 comp rx=0 tx=0).stop 2024-04-23T14:32:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 msgr2=0x7f9cf40b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cf40b87e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8001870 tx=0x7f9ce8001120 comp rx=0 tx=0).stop 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 shutdown_connections 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9cd406c8f0 0x7f9cd406edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9cf40c96c0 0x7f9cf4158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf406cc40 0x7f9cf406d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 --2- 172.21.15.59:0/875068125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cdc0039e0 0x7f9cf40b87e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 >> 172.21.15.59:0/875068125 conn(0x7f9cf408bf90 msgr2=0x7f9cf40b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 shutdown_connections 2024-04-23T14:32:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.396+0000 7f9cfa800640 1 -- 172.21.15.59:0/875068125 wait complete. 2024-04-23T14:32:05.415 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:05.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.605+0000 7fb791800640 1 Processor -- start 2024-04-23T14:32:05.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.605+0000 7fb791800640 1 -- start start 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.606+0000 7fb791800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb78c0d0600 0x7fb78c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.606+0000 7fb791800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c0d0c70 con 0x7fb78c15c700 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.606+0000 7fb791800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c0d0f50 con 0x7fb78c06bf10 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.606+0000 7fb791800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb78c06c360 con 0x7fb78c0d0600 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.607+0000 7fb78b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb78c0d0600 0x7fb78c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.607+0000 7fb78be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb78c15c700 0x7fb78c160d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35226/0 (socket says 172.21.15.59:35226) 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.607+0000 7fb78be00640 1 -- 172.21.15.59:0/4132176141 learned_addr learned my addr 172.21.15.59:0/4132176141 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.607+0000 7fb78b400640 1 -- 172.21.15.59:0/4132176141 >> v1:172.21.15.76:6789/0 conn(0x7fb78c06bf10 legacy=0x7fb78c0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.607+0000 7fb78b400640 1 -- 172.21.15.59:0/4132176141 >> v1:172.21.15.59:6789/0 conn(0x7fb78c15c700 legacy=0x7fb78c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.607+0000 7fb78b400640 1 -- 172.21.15.59:0/4132176141 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c0b4340 con 0x7fb78c0d0600 2024-04-23T14:32:05.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.608+0000 7fb78b400640 1 --2- 172.21.15.59:0/4132176141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c06b9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7780097a0 tx=0x7fb77802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6878a171ed7db45c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.608+0000 7fb78aa00640 1 -- 172.21.15.59:0/4132176141 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778037070 con 0x7fb78c0d0600 2024-04-23T14:32:05.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.608+0000 7fb78aa00640 1 -- 172.21.15.59:0/4132176141 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb778032440 con 0x7fb78c0d0600 2024-04-23T14:32:05.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.608+0000 7fb78aa00640 1 -- 172.21.15.59:0/4132176141 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778032810 con 0x7fb78c0d0600 2024-04-23T14:32:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.609+0000 7fb791800640 1 -- 172.21.15.59:0/4132176141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 msgr2=0x7fb78c06b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.609+0000 7fb791800640 1 --2- 172.21.15.59:0/4132176141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c06b9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7780097a0 tx=0x7fb77802a1e0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.609+0000 7fb791800640 1 -- 172.21.15.59:0/4132176141 shutdown_connections 2024-04-23T14:32:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.609+0000 7fb791800640 1 --2- 172.21.15.59:0/4132176141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c06b9d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.609+0000 7fb791800640 1 -- 172.21.15.59:0/4132176141 >> 172.21.15.59:0/4132176141 conn(0x7fb78c09aa00 msgr2=0x7fb78c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.609+0000 7fb791800640 1 -- 172.21.15.59:0/4132176141 shutdown_connections 2024-04-23T14:32:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.609+0000 7fb791800640 1 -- 172.21.15.59:0/4132176141 wait complete. 2024-04-23T14:32:05.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.611+0000 7fb791800640 1 Processor -- start 2024-04-23T14:32:05.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.611+0000 7fb791800640 1 -- start start 2024-04-23T14:32:05.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb791800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c06bf10 0x7fb78c09ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb791800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c09bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb791800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb78c15c700 0x7fb78c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb791800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c0b5250 con 0x7fb78c0d0600 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb791800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c0b46b0 con 0x7fb78c06bf10 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb791800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c0b53d0 con 0x7fb78c15c700 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c09bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c09bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37856/0 (socket says 172.21.15.59:37856) 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 -- 172.21.15.59:0/638337146 learned_addr learned my addr 172.21.15.59:0/638337146 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 -- 172.21.15.59:0/638337146 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb78c15c700 msgr2=0x7fb78c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb78c15c700 0x7fb78c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 -- 172.21.15.59:0/638337146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c06bf10 msgr2=0x7fb78c09ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c06bf10 0x7fb78c09ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.612+0000 7fb78b400640 1 -- 172.21.15.59:0/638337146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c09c6d0 con 0x7fb78c0d0600 2024-04-23T14:32:05.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.613+0000 7fb78b400640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c09bf50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb778000c00 tx=0x7fb7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.613+0000 7fb783e00640 1 -- 172.21.15.59:0/638337146 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778037040 con 0x7fb78c0d0600 2024-04-23T14:32:05.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.613+0000 7fb783e00640 1 -- 172.21.15.59:0/638337146 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb778042070 con 0x7fb78c0d0600 2024-04-23T14:32:05.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.613+0000 7fb783e00640 1 -- 172.21.15.59:0/638337146 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778030a70 con 0x7fb78c0d0600 2024-04-23T14:32:05.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.613+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c09c9b0 con 0x7fb78c0d0600 2024-04-23T14:32:05.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.613+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c0c5df0 con 0x7fb78c0d0600 2024-04-23T14:32:05.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.614+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78c0c5ff0 con 0x7fb78c0d0600 2024-04-23T14:32:05.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.615+0000 7fb783e00640 1 -- 172.21.15.59:0/638337146 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb778030c10 con 0x7fb78c0d0600 2024-04-23T14:32:05.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.616+0000 7fb783e00640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:05.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.616+0000 7fb783e00640 1 -- 172.21.15.59:0/638337146 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7780ac5e0 con 0x7fb78c0d0600 2024-04-23T14:32:05.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.616+0000 7fb78be00640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:05.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.617+0000 7fb78be00640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb76c06c8f0 0x7fb76c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f60 tx=0x7fb77c007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:05.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.625+0000 7fb783e00640 1 -- 172.21.15.59:0/638337146 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb78c0c5ff0 con 0x7fb78c0d0600 2024-04-23T14:32:05.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.838+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb78c04fd90 con 0x7fb78c0d0600 2024-04-23T14:32:05.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.842+0000 7fb783e00640 1 -- 172.21.15.59:0/638337146 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fb778045360 con 0x7fb78c0d0600 2024-04-23T14:32:05.843 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:05.843 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:31:26.807044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.844+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb76c06c8f0 msgr2=0x7fb76c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.844+0000 7fb791800640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb76c06c8f0 0x7fb76c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f60 tx=0x7fb77c007a80 comp rx=0 tx=0).stop 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.844+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 msgr2=0x7fb78c09bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.844+0000 7fb791800640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c09bf50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb778000c00 tx=0x7fb7780046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 shutdown_connections 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb76c06c8f0 0x7fb76c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb78c15c700 0x7fb78c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb78c0d0600 0x7fb78c09bf50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 --2- 172.21.15.59:0/638337146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c06bf10 0x7fb78c09ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 >> 172.21.15.59:0/638337146 conn(0x7fb78c09aa00 msgr2=0x7fb78c15b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 shutdown_connections 2024-04-23T14:32:05.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:05.845+0000 7fb791800640 1 -- 172.21.15.59:0/638337146 wait complete. 2024-04-23T14:32:05.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:05.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:06.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.076+0000 7f965e000640 1 Processor -- start 2024-04-23T14:32:06.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.076+0000 7f965e000640 1 -- start start 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f965e000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f965815c430 0x7f96580cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f965e000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f965815c810 0x7f96580b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f965e000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96580ce4b0 con 0x7f96580b2c10 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f965e000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f965815f470 con 0x7f965815c810 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f965e000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f965815f750 con 0x7f965815c430 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f9657e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f96580b2c10 0x7f965806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35230/0 (socket says 172.21.15.59:35230) 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f9657e00640 1 -- 172.21.15.59:0/2099552752 learned_addr learned my addr 172.21.15.59:0/2099552752 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:06.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f9656a00640 1 --2- 172.21.15.59:0/2099552752 >> v2:172.21.15.59:3300/0 conn(0x7f965815c810 0x7f96580b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.078+0000 7f9657400640 1 --2- 172.21.15.59:0/2099552752 >> v2:172.21.15.76:3300/0 conn(0x7f965815c430 0x7f96580cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.079+0000 7f9656000640 1 -- 172.21.15.59:0/2099552752 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 697139317 0 0) 0x7f96580ce4b0 con 0x7f96580b2c10 2024-04-23T14:32:06.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.079+0000 7f9656000640 1 -- 172.21.15.59:0/2099552752 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f963c003520 con 0x7f96580b2c10 2024-04-23T14:32:06.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.079+0000 7f9656a00640 1 -- 172.21.15.59:0/2099552752 >> v2:172.21.15.76:3300/0 conn(0x7f965815c430 msgr2=0x7f96580cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:06.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.079+0000 7f9656a00640 1 --2- 172.21.15.59:0/2099552752 >> v2:172.21.15.76:3300/0 conn(0x7f965815c430 0x7f96580cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.079+0000 7f9656a00640 1 -- 172.21.15.59:0/2099552752 >> v1:172.21.15.59:6789/0 conn(0x7f96580b2c10 legacy=0x7f965806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.079+0000 7f9656a00640 1 -- 172.21.15.59:0/2099552752 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965815fa30 con 0x7f965815c810 2024-04-23T14:32:06.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.080+0000 7f9656a00640 1 --2- 172.21.15.59:0/2099552752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c810 0x7f96580b26d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96580d2600 tx=0x7f964402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2c65a8faddaf23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.080+0000 7f9656000640 1 -- 172.21.15.59:0/2099552752 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644037070 con 0x7f965815c810 2024-04-23T14:32:06.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.080+0000 7f9656000640 1 -- 172.21.15.59:0/2099552752 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9644032440 con 0x7f965815c810 2024-04-23T14:32:06.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.080+0000 7f9656000640 1 -- 172.21.15.59:0/2099552752 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644032810 con 0x7f965815c810 2024-04-23T14:32:06.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 -- 172.21.15.59:0/2099552752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c810 msgr2=0x7f96580b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 --2- 172.21.15.59:0/2099552752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c810 0x7f96580b26d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f96580d2600 tx=0x7f964402a1a0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 -- 172.21.15.59:0/2099552752 shutdown_connections 2024-04-23T14:32:06.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 --2- 172.21.15.59:0/2099552752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c810 0x7f96580b26d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 --2- 172.21.15.59:0/2099552752 >> v2:172.21.15.76:3300/0 conn(0x7f965815c430 0x7f96580cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 -- 172.21.15.59:0/2099552752 >> 172.21.15.59:0/2099552752 conn(0x7f9658073430 msgr2=0x7f965809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 -- 172.21.15.59:0/2099552752 shutdown_connections 2024-04-23T14:32:06.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.081+0000 7f965e000640 1 -- 172.21.15.59:0/2099552752 wait complete. 2024-04-23T14:32:06.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.083+0000 7f965e000640 1 Processor -- start 2024-04-23T14:32:06.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.083+0000 7f965e000640 1 -- start start 2024-04-23T14:32:06.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f965e000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96580b2c10 0x7f9658160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f965e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c430 0x7f96581628f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f965e000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f965815c810 0x7f9658162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f965e000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f965815f200 con 0x7f965815c430 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f965e000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f965815f080 con 0x7f96580b2c10 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f965e000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f965806c0e0 con 0x7f965815c810 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f9656a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c430 0x7f96581628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f9656a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c430 0x7f96581628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37866/0 (socket says 172.21.15.59:37866) 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.084+0000 7f9656a00640 1 -- 172.21.15.59:0/277493651 learned_addr learned my addr 172.21.15.59:0/277493651 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f9657400640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96580b2c10 0x7f9658160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f9656a00640 1 -- 172.21.15.59:0/277493651 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f965815c810 msgr2=0x7f9658162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f9656a00640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f965815c810 0x7f9658162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f9656a00640 1 -- 172.21.15.59:0/277493651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96580b2c10 msgr2=0x7f9658160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f9656a00640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96580b2c10 0x7f9658160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f9656a00640 1 -- 172.21.15.59:0/277493651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96580af190 con 0x7f965815c430 2024-04-23T14:32:06.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f9656a00640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c430 0x7f96581628f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9644000c00 tx=0x7f96440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f964f400640 1 -- 172.21.15.59:0/277493651 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644041020 con 0x7f965815c430 2024-04-23T14:32:06.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f964f400640 1 -- 172.21.15.59:0/277493651 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964403d700 con 0x7f965815c430 2024-04-23T14:32:06.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96580af470 con 0x7f965815c430 2024-04-23T14:32:06.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.085+0000 7f964f400640 1 -- 172.21.15.59:0/277493651 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644037040 con 0x7f965815c430 2024-04-23T14:32:06.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.086+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96580afa30 con 0x7f965815c430 2024-04-23T14:32:06.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.087+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9658069190 con 0x7f965815c430 2024-04-23T14:32:06.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.088+0000 7f964f400640 1 -- 172.21.15.59:0/277493651 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9644044020 con 0x7f965815c430 2024-04-23T14:32:06.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.088+0000 7f964f400640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f963806c8f0 0x7f963806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.089+0000 7f964f400640 1 -- 172.21.15.59:0/277493651 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96440ac360 con 0x7f965815c430 2024-04-23T14:32:06.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.089+0000 7f9657400640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f963806c8f0 0x7f963806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.090+0000 7f9657400640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f963806c8f0 0x7f963806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9648009a10 tx=0x7f9648008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.093+0000 7f964f400640 1 -- 172.21.15.59:0/277493651 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f964404b030 con 0x7f965815c430 2024-04-23T14:32:06.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.292+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96580b7040 con 0x7f965815c430 2024-04-23T14:32:06.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.293+0000 7f964f400640 1 -- 172.21.15.59:0/277493651 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f964407a700 con 0x7f965815c430 2024-04-23T14:32:06.294 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f963806c8f0 msgr2=0x7f963806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f963806c8f0 0x7f963806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9648009a10 tx=0x7f9648008040 comp rx=0 tx=0).stop 2024-04-23T14:32:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c430 msgr2=0x7f96581628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c430 0x7f96581628f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9644000c00 tx=0x7f96440046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 shutdown_connections 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f963806c8f0 0x7f963806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f965815c810 0x7f9658162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f965815c430 0x7f96581628f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 --2- 172.21.15.59:0/277493651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96580b2c10 0x7f9658160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.295+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 >> 172.21.15.59:0/277493651 conn(0x7f9658073430 msgr2=0x7f96580cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.296+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 shutdown_connections 2024-04-23T14:32:06.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.296+0000 7f965e000640 1 -- 172.21.15.59:0/277493651 wait complete. 2024-04-23T14:32:06.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:06.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.518+0000 7f817ce00640 1 Processor -- start 2024-04-23T14:32:06.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.518+0000 7f817ce00640 1 -- start start 2024-04-23T14:32:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.519+0000 7f817ce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d21c0 0x7f81780d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.519+0000 7f817ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f81780d1500 0x7f81780cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.519+0000 7f817ce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81780d37e0 con 0x7f817815e770 2024-04-23T14:32:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.519+0000 7f817ce00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f81780d3350 con 0x7f81780d1500 2024-04-23T14:32:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.519+0000 7f817ce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f81780d1950 con 0x7f81780d21c0 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.519+0000 7f8177400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f817815e770 0x7f817815f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35238/0 (socket says 172.21.15.59:35238) 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8177400640 1 -- 172.21.15.59:0/1214783166 learned_addr learned my addr 172.21.15.59:0/1214783166 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8177e00640 1 --2- 172.21.15.59:0/1214783166 >> v2:172.21.15.59:3300/0 conn(0x7f81780d1500 0x7f81780cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8176a00640 1 --2- 172.21.15.59:0/1214783166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d21c0 0x7f81780d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8176000640 1 -- 172.21.15.59:0/1214783166 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909292668 0 0) 0x7f81780d37e0 con 0x7f817815e770 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8177e00640 1 -- 172.21.15.59:0/1214783166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d21c0 msgr2=0x7f81780d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8177e00640 1 --2- 172.21.15.59:0/1214783166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d21c0 0x7f81780d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8177e00640 1 -- 172.21.15.59:0/1214783166 >> v1:172.21.15.59:6789/0 conn(0x7f817815e770 legacy=0x7f817815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.520+0000 7f8177e00640 1 -- 172.21.15.59:0/1214783166 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81780d2740 con 0x7f81780d1500 2024-04-23T14:32:06.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.521+0000 7f8177e00640 1 --2- 172.21.15.59:0/1214783166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81780d1500 0x7f81780cf3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f81600097a0 tx=0x7f816002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8786ff45438a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.521+0000 7f8176000640 1 -- 172.21.15.59:0/1214783166 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160037070 con 0x7f81780d1500 2024-04-23T14:32:06.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.521+0000 7f8176000640 1 -- 172.21.15.59:0/1214783166 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8160032440 con 0x7f81780d1500 2024-04-23T14:32:06.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.521+0000 7f8176000640 1 -- 172.21.15.59:0/1214783166 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160032810 con 0x7f81780d1500 2024-04-23T14:32:06.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 -- 172.21.15.59:0/1214783166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81780d1500 msgr2=0x7f81780cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 --2- 172.21.15.59:0/1214783166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81780d1500 0x7f81780cf3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f81600097a0 tx=0x7f816002a180 comp rx=0 tx=0).stop 2024-04-23T14:32:06.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 -- 172.21.15.59:0/1214783166 shutdown_connections 2024-04-23T14:32:06.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 --2- 172.21.15.59:0/1214783166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81780d1500 0x7f81780cf3e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 --2- 172.21.15.59:0/1214783166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d21c0 0x7f81780d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 -- 172.21.15.59:0/1214783166 >> 172.21.15.59:0/1214783166 conn(0x7f817808bf90 msgr2=0x7f81780d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 -- 172.21.15.59:0/1214783166 shutdown_connections 2024-04-23T14:32:06.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.522+0000 7f817ce00640 1 -- 172.21.15.59:0/1214783166 wait complete. 2024-04-23T14:32:06.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.524+0000 7f817ce00640 1 Processor -- start 2024-04-23T14:32:06.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.524+0000 7f817ce00640 1 -- start start 2024-04-23T14:32:06.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.525+0000 7f817ce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d1500 0x7f81780ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.525+0000 7f817ce00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81780d21c0 0x7f81780cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.525+0000 7f817ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f817815e770 0x7f81781f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.525+0000 7f817ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81780d2fe0 con 0x7f817815e770 2024-04-23T14:32:06.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.525+0000 7f817ce00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f81780d2d00 con 0x7f81780d21c0 2024-04-23T14:32:06.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.525+0000 7f817ce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f817815eb50 con 0x7f81780d1500 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f817815e770 0x7f81781f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f817815e770 0x7f81781f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37878/0 (socket says 172.21.15.59:37878) 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 -- 172.21.15.59:0/4217782489 learned_addr learned my addr 172.21.15.59:0/4217782489 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 -- 172.21.15.59:0/4217782489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d1500 msgr2=0x7f81780ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d1500 0x7f81780ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 -- 172.21.15.59:0/4217782489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81780d21c0 msgr2=0x7f81780cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81780d21c0 0x7f81780cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 -- 172.21.15.59:0/4217782489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81780cbd20 con 0x7f817815e770 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.526+0000 7f8177e00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f817815e770 0x7f81781f3b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8160035040 tx=0x7f81600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.527+0000 7f816f400640 1 -- 172.21.15.59:0/4217782489 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160037040 con 0x7f817815e770 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.527+0000 7f816f400640 1 -- 172.21.15.59:0/4217782489 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8160042070 con 0x7f817815e770 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.527+0000 7f816f400640 1 -- 172.21.15.59:0/4217782489 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160030b50 con 0x7f817815e770 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.527+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81780b1370 con 0x7f817815e770 2024-04-23T14:32:06.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.527+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81780b1930 con 0x7f817815e770 2024-04-23T14:32:06.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.528+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817804fd90 con 0x7f817815e770 2024-04-23T14:32:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.529+0000 7f816f400640 1 -- 172.21.15.59:0/4217782489 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8160030cf0 con 0x7f817815e770 2024-04-23T14:32:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.530+0000 7f816f400640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f815406c940 0x7f815406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.530+0000 7f816f400640 1 -- 172.21.15.59:0/4217782489 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f81600ac820 con 0x7f817815e770 2024-04-23T14:32:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.530+0000 7f8177400640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f815406c940 0x7f815406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.531+0000 7f8177400640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f815406c940 0x7f815406ee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8170007fd0 tx=0x7f8170007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.535+0000 7f816f400640 1 -- 172.21.15.59:0/4217782489 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8160032e20 con 0x7f817815e770 2024-04-23T14:32:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.736+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81780ba570 con 0x7f817815e770 2024-04-23T14:32:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.737+0000 7f816f400640 1 -- 172.21.15.59:0/4217782489 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f816007abb0 con 0x7f817815e770 2024-04-23T14:32:06.738 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.739+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f815406c940 msgr2=0x7f815406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.739+0000 7f817ce00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f815406c940 0x7f815406ee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8170007fd0 tx=0x7f8170007ce0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.739+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f817815e770 msgr2=0x7f81781f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.739+0000 7f817ce00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f817815e770 0x7f81781f3b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8160035040 tx=0x7f81600046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 shutdown_connections 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f815406c940 0x7f815406ee00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f817815e770 0x7f81781f3b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81780d21c0 0x7f81780cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 --2- 172.21.15.59:0/4217782489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81780d1500 0x7f81780ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 >> 172.21.15.59:0/4217782489 conn(0x7f817808bf90 msgr2=0x7f817806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 shutdown_connections 2024-04-23T14:32:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.740+0000 7f817ce00640 1 -- 172.21.15.59:0/4217782489 wait complete. 2024-04-23T14:32:06.760 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:06.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.978+0000 7f0a22400640 1 Processor -- start 2024-04-23T14:32:06.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.979+0000 7f0a22400640 1 -- start start 2024-04-23T14:32:06.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.980+0000 7f0a22400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a1c0d0600 0x7f0a1c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.980+0000 7f0a22400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.980+0000 7f0a22400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a1c163240 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.980+0000 7f0a22400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0a1c072a20 con 0x7f0a1c0d0600 2024-04-23T14:32:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.980+0000 7f0a22400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1c071e10 con 0x7f0a1c06dcb0 2024-04-23T14:32:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a21000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a1c0d0600 0x7f0a1c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a21a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0a1c0d09e0 0x7f0a1c06e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35242/0 (socket says 172.21.15.59:35242) 2024-04-23T14:32:06.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a21a00640 1 -- 172.21.15.59:0/3872553119 learned_addr learned my addr 172.21.15.59:0/3872553119 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a1be00640 1 --2- 172.21.15.59:0/3872553119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a21000640 1 -- 172.21.15.59:0/3872553119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 msgr2=0x7f0a1c162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a21000640 1 --2- 172.21.15.59:0/3872553119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a21000640 1 -- 172.21.15.59:0/3872553119 >> v1:172.21.15.59:6789/0 conn(0x7f0a1c0d09e0 legacy=0x7f0a1c06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a21000640 1 -- 172.21.15.59:0/3872553119 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c0720f0 con 0x7f0a1c0d0600 2024-04-23T14:32:06.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.981+0000 7f0a1b400640 1 -- 172.21.15.59:0/3872553119 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4137461410 0 0) 0x7f0a14001490 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.983+0000 7f0a21000640 1 --2- 172.21.15.59:0/3872553119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d0600 0x7f0a1c0ad170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a100097a0 tx=0x7f0a1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34f44c0051b7854 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.983+0000 7f0a1b400640 1 -- 172.21.15.59:0/3872553119 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10037070 con 0x7f0a1c0d0600 2024-04-23T14:32:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.983+0000 7f0a1b400640 1 -- 172.21.15.59:0/3872553119 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a10032440 con 0x7f0a1c0d0600 2024-04-23T14:32:06.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.983+0000 7f0a1b400640 1 -- 172.21.15.59:0/3872553119 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10032810 con 0x7f0a1c0d0600 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.983+0000 7f0a22400640 1 -- 172.21.15.59:0/3872553119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d0600 msgr2=0x7f0a1c0ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.983+0000 7f0a22400640 1 --2- 172.21.15.59:0/3872553119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d0600 0x7f0a1c0ad170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a100097a0 tx=0x7f0a1002a180 comp rx=0 tx=0).stop 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.984+0000 7f0a22400640 1 -- 172.21.15.59:0/3872553119 shutdown_connections 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.984+0000 7f0a22400640 1 --2- 172.21.15.59:0/3872553119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.984+0000 7f0a22400640 1 --2- 172.21.15.59:0/3872553119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d0600 0x7f0a1c0ad170 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.984+0000 7f0a22400640 1 -- 172.21.15.59:0/3872553119 >> 172.21.15.59:0/3872553119 conn(0x7f0a1c06bdf0 msgr2=0x7f0a1c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.984+0000 7f0a22400640 1 -- 172.21.15.59:0/3872553119 shutdown_connections 2024-04-23T14:32:06.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.984+0000 7f0a22400640 1 -- 172.21.15.59:0/3872553119 wait complete. 2024-04-23T14:32:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.985+0000 7f0a22400640 1 Processor -- start 2024-04-23T14:32:06.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.986+0000 7f0a22400640 1 -- start start 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.986+0000 7f0a22400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a22400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a1c0d0600 0x7f0a1c160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a22400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d09e0 0x7f0a1c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a22400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15eaf0 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a22400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0726b0 con 0x7f0a1c0d0600 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a22400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1c0d0e70 con 0x7f0a1c06dcb0 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d09e0 0x7f0a1c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d09e0 0x7f0a1c1608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37890/0 (socket says 172.21.15.59:37890) 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 -- 172.21.15.59:0/2603048129 learned_addr learned my addr 172.21.15.59:0/2603048129 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a1be00640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a1c0d0600 0x7f0a1c160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21000640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 -- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 msgr2=0x7f0a1c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c0cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 -- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a1c0d0600 msgr2=0x7f0a1c160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a1c0d0600 0x7f0a1c160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21a00640 1 -- 172.21.15.59:0/2603048129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c0caf40 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.987+0000 7f0a21000640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.988+0000 7f0a21a00640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d09e0 0x7f0a1c1608b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c700 tx=0x7f0a1400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.988+0000 7f0a18c00640 1 -- 172.21.15.59:0/2603048129 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14018070 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.988+0000 7f0a18c00640 1 -- 172.21.15.59:0/2603048129 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1400ce10 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.988+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c0cb220 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.988+0000 7f0a18c00640 1 -- 172.21.15.59:0/2603048129 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400a860 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.988+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c0cb7e0 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.989+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1c04fd90 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.990+0000 7f0a18c00640 1 -- 172.21.15.59:0/2603048129 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a1401b070 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.991+0000 7f0a18c00640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f09fc06c8f0 0x7f09fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:06.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.991+0000 7f0a18c00640 1 -- 172.21.15.59:0/2603048129 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f0a1408b1e0 con 0x7f0a1c0d09e0 2024-04-23T14:32:06.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.991+0000 7f0a21000640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f09fc06c8f0 0x7f09fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:06.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.992+0000 7f0a21000640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f09fc06c8f0 0x7f09fc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a10034000 tx=0x7f0a1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:06.996+0000 7f0a18c00640 1 -- 172.21.15.59:0/2603048129 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a1c163240 con 0x7f0a1c0d09e0 2024-04-23T14:32:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.204+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a1c0cff50 con 0x7f0a1c0d09e0 2024-04-23T14:32:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.205+0000 7f0a18c00640 1 -- 172.21.15.59:0/2603048129 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f0a14059580 con 0x7f0a1c0d09e0 2024-04-23T14:32:07.206 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.207+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f09fc06c8f0 msgr2=0x7f09fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.207+0000 7f0a22400640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f09fc06c8f0 0x7f09fc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a10034000 tx=0x7f0a1002e000 comp rx=0 tx=0).stop 2024-04-23T14:32:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.207+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d09e0 msgr2=0x7f0a1c1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.207+0000 7f0a22400640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d09e0 0x7f0a1c1608b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c700 tx=0x7f0a1400cbc0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 shutdown_connections 2024-04-23T14:32:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f09fc06c8f0 0x7f09fc06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a1c0d09e0 0x7f0a1c1608b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a1c0d0600 0x7f0a1c160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 --2- 172.21.15.59:0/2603048129 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a1c06dcb0 0x7f0a1c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 >> 172.21.15.59:0/2603048129 conn(0x7f0a1c06bdf0 msgr2=0x7f0a1c159230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 shutdown_connections 2024-04-23T14:32:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.208+0000 7f0a22400640 1 -- 172.21.15.59:0/2603048129 wait complete. 2024-04-23T14:32:07.229 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:07.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.422+0000 7f57a9e00640 1 Processor -- start 2024-04-23T14:32:07.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.422+0000 7f57a9e00640 1 -- start start 2024-04-23T14:32:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a415c3d0 0x7f57a40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a9e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57a40cd590 0x7f57a40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a9e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a4160210 con 0x7f57a40b4690 2024-04-23T14:32:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a9e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f57a40cd9f0 con 0x7f57a40cd590 2024-04-23T14:32:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f57a41604f0 con 0x7f57a415c3d0 2024-04-23T14:32:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a415c3d0 0x7f57a40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a415c3d0 0x7f57a40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:33472/0 (socket says 172.21.15.59:33472) 2024-04-23T14:32:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.424+0000 7f57a8a00640 1 -- 172.21.15.59:0/2580940628 learned_addr learned my addr 172.21.15.59:0/2580940628 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.425+0000 7f57a3e00640 1 --2- 172.21.15.59:0/2580940628 >> v2:172.21.15.76:3300/0 conn(0x7f57a40cd590 0x7f57a40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.425+0000 7f57a8a00640 1 -- 172.21.15.59:0/2580940628 >> v2:172.21.15.76:3300/0 conn(0x7f57a40cd590 msgr2=0x7f57a40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.425+0000 7f57a8a00640 1 --2- 172.21.15.59:0/2580940628 >> v2:172.21.15.76:3300/0 conn(0x7f57a40cd590 0x7f57a40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.425+0000 7f57a8a00640 1 -- 172.21.15.59:0/2580940628 >> v1:172.21.15.76:6789/0 conn(0x7f57a40b4690 legacy=0x7f57a406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.425+0000 7f57a8a00640 1 -- 172.21.15.59:0/2580940628 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a41607d0 con 0x7f57a415c3d0 2024-04-23T14:32:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.427+0000 7f57a8a00640 1 --2- 172.21.15.59:0/2580940628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a415c3d0 0x7f57a40cd050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5798009870 tx=0x7f579802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f00ba34f5b709f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.427+0000 7f57a3400640 1 -- 172.21.15.59:0/2580940628 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798037070 con 0x7f57a415c3d0 2024-04-23T14:32:07.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.427+0000 7f57a3400640 1 -- 172.21.15.59:0/2580940628 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798032440 con 0x7f57a415c3d0 2024-04-23T14:32:07.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.427+0000 7f57a3400640 1 -- 172.21.15.59:0/2580940628 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798032810 con 0x7f57a415c3d0 2024-04-23T14:32:07.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 -- 172.21.15.59:0/2580940628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a415c3d0 msgr2=0x7f57a40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 --2- 172.21.15.59:0/2580940628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a415c3d0 0x7f57a40cd050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5798009870 tx=0x7f579802a270 comp rx=0 tx=0).stop 2024-04-23T14:32:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 -- 172.21.15.59:0/2580940628 shutdown_connections 2024-04-23T14:32:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 --2- 172.21.15.59:0/2580940628 >> v2:172.21.15.76:3300/0 conn(0x7f57a40cd590 0x7f57a40b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 --2- 172.21.15.59:0/2580940628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a415c3d0 0x7f57a40cd050 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 -- 172.21.15.59:0/2580940628 >> 172.21.15.59:0/2580940628 conn(0x7f57a40b21a0 msgr2=0x7f57a40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 -- 172.21.15.59:0/2580940628 shutdown_connections 2024-04-23T14:32:07.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.428+0000 7f57a9e00640 1 -- 172.21.15.59:0/2580940628 wait complete. 2024-04-23T14:32:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.429+0000 7f57a9e00640 1 Processor -- start 2024-04-23T14:32:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.430+0000 7f57a9e00640 1 -- start start 2024-04-23T14:32:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a40b4690 0x7f57a415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a40cd590 0x7f57a415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a415c3d0 0x7f57a415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40b4ae0 con 0x7f57a40cd590 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4160d90 con 0x7f57a415c3d0 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f57a415c7b0 con 0x7f57a40b4690 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a40cd590 0x7f57a415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a40cd590 0x7f57a415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37892/0 (socket says 172.21.15.59:37892) 2024-04-23T14:32:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 -- 172.21.15.59:0/3262833899 learned_addr learned my addr 172.21.15.59:0/3262833899 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a8a00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a40b4690 0x7f57a415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9400640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a415c3d0 0x7f57a415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 -- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a40b4690 msgr2=0x7f57a415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a40b4690 0x7f57a415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 -- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a415c3d0 msgr2=0x7f57a415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a415c3d0 0x7f57a415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a3e00640 1 -- 172.21.15.59:0/3262833899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a41f5b60 con 0x7f57a40cd590 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.431+0000 7f57a9400640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a415c3d0 0x7f57a415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.432+0000 7f57a3e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a40cd590 0x7f57a415a630 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f579000e9b0 tx=0x7f579000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.432+0000 7f57a0c00640 1 -- 172.21.15.59:0/3262833899 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790017070 con 0x7f57a40cd590 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.432+0000 7f57a0c00640 1 -- 172.21.15.59:0/3262833899 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579000c9a0 con 0x7f57a40cd590 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.432+0000 7f57a0c00640 1 -- 172.21.15.59:0/3262833899 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579000cd90 con 0x7f57a40cd590 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.432+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a41f5e40 con 0x7f57a40cd590 2024-04-23T14:32:07.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.432+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a41f6400 con 0x7f57a40cd590 2024-04-23T14:32:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.434+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a404fd90 con 0x7f57a40cd590 2024-04-23T14:32:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.434+0000 7f57a0c00640 1 -- 172.21.15.59:0/3262833899 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f579001a070 con 0x7f57a40cd590 2024-04-23T14:32:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.435+0000 7f57a0c00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.435+0000 7f57a0c00640 1 -- 172.21.15.59:0/3262833899 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f579008bfe0 con 0x7f57a40cd590 2024-04-23T14:32:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.436+0000 7f57a8a00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.436+0000 7f57a8a00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5798034000 tx=0x7f579802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.440+0000 7f57a0c00640 1 -- 172.21.15.59:0/3262833899 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f579005a310 con 0x7f57a40cd590 2024-04-23T14:32:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.662+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57a406e400 con 0x7f57a40cd590 2024-04-23T14:32:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.665+0000 7f57a0c00640 1 -- 172.21.15.59:0/3262833899 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7f5790059c10 con 0x7f57a40cd590 2024-04-23T14:32:07.666 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:07.666 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 msgr2=0x7f578406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5798034000 tx=0x7f579802e000 comp rx=0 tx=0).stop 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a40cd590 msgr2=0x7f57a415a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a40cd590 0x7f57a415a630 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f579000e9b0 tx=0x7f579000ee70 comp rx=0 tx=0).stop 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 shutdown_connections 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a415c3d0 0x7f57a415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a40cd590 0x7f57a415a630 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 --2- 172.21.15.59:0/3262833899 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a40b4690 0x7f57a415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 >> 172.21.15.59:0/3262833899 conn(0x7f57a40b21a0 msgr2=0x7f57a40b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:07.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 shutdown_connections 2024-04-23T14:32:07.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.668+0000 7f57a9e00640 1 -- 172.21.15.59:0/3262833899 wait complete. 2024-04-23T14:32:07.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:07.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:07.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.896+0000 7f1ced400640 1 Processor -- start 2024-04-23T14:32:07.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.896+0000 7f1ced400640 1 -- start start 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ced400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ce815ed20 0x7f1ce80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ced400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ce80b3600 0x7f1ce80d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ced400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce80b5da0 con 0x7f1ce80b3f10 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ced400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce815f100 con 0x7f1ce80b3600 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ced400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce80d0a20 con 0x7f1ce815ed20 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ce7400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ce80b3600 0x7f1ce80d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ceca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1ce80b3f10 0x7f1ce80b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35248/0 (socket says 172.21.15.59:35248) 2024-04-23T14:32:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ceca00640 1 -- 172.21.15.59:0/3694127082 learned_addr learned my addr 172.21.15.59:0/3694127082 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.898+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/3694127082 >> v2:172.21.15.76:3300/0 conn(0x7f1ce815ed20 0x7f1ce80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.899+0000 7f1ce7400640 1 -- 172.21.15.59:0/3694127082 >> v2:172.21.15.76:3300/0 conn(0x7f1ce815ed20 msgr2=0x7f1ce80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.899+0000 7f1ce7400640 1 --2- 172.21.15.59:0/3694127082 >> v2:172.21.15.76:3300/0 conn(0x7f1ce815ed20 0x7f1ce80d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.899+0000 7f1ce7400640 1 -- 172.21.15.59:0/3694127082 >> v1:172.21.15.59:6789/0 conn(0x7f1ce80b3f10 legacy=0x7f1ce80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.899+0000 7f1ce7400640 1 -- 172.21.15.59:0/3694127082 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce80d2ed0 con 0x7f1ce80b3600 2024-04-23T14:32:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.899+0000 7f1ce6a00640 1 -- 172.21.15.59:0/3694127082 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2052597310 0 0) 0x7f1ce80b5da0 con 0x7f1ce80b3f10 2024-04-23T14:32:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.900+0000 7f1ce7400640 1 --2- 172.21.15.59:0/3694127082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80b3600 0x7f1ce80d13e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ce808e2a0 tx=0x7f1cd402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2882b31e97710f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.900+0000 7f1ce6a00640 1 -- 172.21.15.59:0/3694127082 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4037070 con 0x7f1ce80b3600 2024-04-23T14:32:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.900+0000 7f1ce6a00640 1 -- 172.21.15.59:0/3694127082 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd402add0 con 0x7f1ce80b3600 2024-04-23T14:32:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.900+0000 7f1ce6a00640 1 -- 172.21.15.59:0/3694127082 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4032660 con 0x7f1ce80b3600 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 -- 172.21.15.59:0/3694127082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80b3600 msgr2=0x7f1ce80d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 --2- 172.21.15.59:0/3694127082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80b3600 0x7f1ce80d13e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ce808e2a0 tx=0x7f1cd402a080 comp rx=0 tx=0).stop 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 -- 172.21.15.59:0/3694127082 shutdown_connections 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 --2- 172.21.15.59:0/3694127082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce80b3600 0x7f1ce80d13e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 --2- 172.21.15.59:0/3694127082 >> v2:172.21.15.76:3300/0 conn(0x7f1ce815ed20 0x7f1ce80d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 -- 172.21.15.59:0/3694127082 >> 172.21.15.59:0/3694127082 conn(0x7f1ce808bf90 msgr2=0x7f1ce80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 -- 172.21.15.59:0/3694127082 shutdown_connections 2024-04-23T14:32:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.901+0000 7f1ced400640 1 -- 172.21.15.59:0/3694127082 wait complete. 2024-04-23T14:32:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.903+0000 7f1ced400640 1 Processor -- start 2024-04-23T14:32:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.903+0000 7f1ced400640 1 -- start start 2024-04-23T14:32:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ced400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ce80b3600 0x7f1ce81f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ced400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce80b3f10 0x7f1ce80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ced400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce815ed20 0x7f1ce80cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ced400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80d0290 con 0x7f1ce815ed20 2024-04-23T14:32:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ced400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80d3490 con 0x7f1ce80b3f10 2024-04-23T14:32:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ced400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce80d0410 con 0x7f1ce80b3600 2024-04-23T14:32:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ceca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce815ed20 0x7f1ce80cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ceca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce815ed20 0x7f1ce80cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37908/0 (socket says 172.21.15.59:37908) 2024-04-23T14:32:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.904+0000 7f1ceca00640 1 -- 172.21.15.59:0/3629092965 learned_addr learned my addr 172.21.15.59:0/3629092965 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ce7400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce80b3f10 0x7f1ce80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ce80b3600 0x7f1ce81f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ceca00640 1 -- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ce80b3600 msgr2=0x7f1ce81f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ceca00640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ce80b3600 0x7f1ce81f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ceca00640 1 -- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce80b3f10 msgr2=0x7f1ce80cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ceca00640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce80b3f10 0x7f1ce80cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ceca00640 1 -- 172.21.15.59:0/3629092965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce81f8b80 con 0x7f1ce815ed20 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ce7400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce80b3f10 0x7f1ce80cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1ceca00640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce815ed20 0x7f1ce80cbbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000c480 tx=0x7f1ce000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3629092965 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0018070 con 0x7f1ce815ed20 2024-04-23T14:32:07.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3629092965 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce0007500 con 0x7f1ce815ed20 2024-04-23T14:32:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.905+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3629092965 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0004a20 con 0x7f1ce815ed20 2024-04-23T14:32:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.906+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce81f82e0 con 0x7f1ce815ed20 2024-04-23T14:32:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.906+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce81f88a0 con 0x7f1ce815ed20 2024-04-23T14:32:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.907+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce804fd90 con 0x7f1ce815ed20 2024-04-23T14:32:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.908+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3629092965 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ce001b070 con 0x7f1ce815ed20 2024-04-23T14:32:07.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.909+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cc806c8f0 0x7f1cc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:07.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.909+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3629092965 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1ce008aef0 con 0x7f1ce815ed20 2024-04-23T14:32:07.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.909+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cc806c8f0 0x7f1cc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.910+0000 7f1ce7e00640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cc806c8f0 0x7f1cc806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009e90 tx=0x7f1cd800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:07.915+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3629092965 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ce0059290 con 0x7f1ce815ed20 2024-04-23T14:32:08.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.122+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ce815da80 con 0x7f1ce815ed20 2024-04-23T14:32:08.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.123+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3629092965 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1ce0058b90 con 0x7f1ce815ed20 2024-04-23T14:32:08.124 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:08.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cc806c8f0 msgr2=0x7f1cc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cc806c8f0 0x7f1cc806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009e90 tx=0x7f1cd800b040 comp rx=0 tx=0).stop 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce815ed20 msgr2=0x7f1ce80cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce815ed20 0x7f1ce80cbbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000c480 tx=0x7f1ce000c940 comp rx=0 tx=0).stop 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 shutdown_connections 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cc806c8f0 0x7f1cc806edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce815ed20 0x7f1ce80cbbd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce80b3f10 0x7f1ce80cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 --2- 172.21.15.59:0/3629092965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ce80b3600 0x7f1ce81f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.126+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 >> 172.21.15.59:0/3629092965 conn(0x7f1ce808bf90 msgr2=0x7f1ce806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.127+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 shutdown_connections 2024-04-23T14:32:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.127+0000 7f1ced400640 1 -- 172.21.15.59:0/3629092965 wait complete. 2024-04-23T14:32:08.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:08.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.366+0000 7f4335400640 1 Processor -- start 2024-04-23T14:32:08.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.366+0000 7f4335400640 1 -- start start 2024-04-23T14:32:08.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.367+0000 7f4335400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 0x7f43300b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.367+0000 7f4335400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300b3d90 con 0x7f43300d0500 2024-04-23T14:32:08.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.367+0000 7f4335400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433015cb00 con 0x7f433015c720 2024-04-23T14:32:08.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.367+0000 7f4335400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f43300d1430 con 0x7f43300d09b0 2024-04-23T14:32:08.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.368+0000 7f432f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f43300d0500 0x7f43300b3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35260/0 (socket says 172.21.15.59:35260) 2024-04-23T14:32:08.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.368+0000 7f432f400640 1 -- 172.21.15.59:0/2297309165 learned_addr learned my addr 172.21.15.59:0/2297309165 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.368+0000 7f4334a00640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 0x7f43300b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.368+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3499233626 0 0) 0x7f431c001110 con 0x7f43300d0500 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.368+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4314003520 con 0x7f43300d0500 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735282265 0 0) 0x7f4314003520 con 0x7f43300d0500 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 msgr2=0x7f43300b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 0x7f43300b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 >> v1:172.21.15.76:6789/0 conn(0x7f433015c720 legacy=0x7f43300cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433009ac50 con 0x7f43300d0500 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f431c001840 con 0x7f43300d0500 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 >> v1:172.21.15.59:6789/0 conn(0x7f43300d0500 legacy=0x7f43300b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43140039e0 0x7f4314003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4314004800 0x7f4314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 0x7f43140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43300d1430 con 0x7f4314004800 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f431400baa0 con 0x7f43140039e0 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f431400bca0 con 0x7f43300d09b0 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.369+0000 7f432f400640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4314004800 0x7f4314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.370+0000 7f4334a00640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 0x7f43140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.370+0000 7f432fe00640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43140039e0 0x7f4314003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.370+0000 7f432f400640 1 -- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 msgr2=0x7f43140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.370+0000 7f432f400640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43300d09b0 0x7f43140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.370+0000 7f432f400640 1 -- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43140039e0 msgr2=0x7f4314003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.370+0000 7f432f400640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43140039e0 0x7f4314003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.370+0000 7f432f400640 1 -- 172.21.15.59:0/2297309165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4314003520 con 0x7f4314004800 2024-04-23T14:32:08.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.371+0000 7f432f400640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4314004800 0x7f4314004c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f431c001eb0 tx=0x7f4314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5786392efeb77f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.371+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c036070 con 0x7f4314004800 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.371+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431c005af0 con 0x7f4314004800 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.371+0000 7f432ea00640 1 -- 172.21.15.59:0/2297309165 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c007e30 con 0x7f4314004800 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.372+0000 7f4335400640 1 -- 172.21.15.59:0/2297309165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4314004800 msgr2=0x7f4314004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.372+0000 7f4335400640 1 --2- 172.21.15.59:0/2297309165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4314004800 0x7f4314004c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f431c001eb0 tx=0x7f4314005830 comp rx=0 tx=0).stop 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.372+0000 7f4334a00640 1 -- 172.21.15.59:0/2297309165 reap_dead start 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.372+0000 7f4335400640 1 -- 172.21.15.59:0/2297309165 shutdown_connections 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.372+0000 7f4335400640 1 -- 172.21.15.59:0/2297309165 >> 172.21.15.59:0/2297309165 conn(0x7f43300b3f70 msgr2=0x7f43300b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.372+0000 7f4335400640 1 -- 172.21.15.59:0/2297309165 shutdown_connections 2024-04-23T14:32:08.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.372+0000 7f4335400640 1 -- 172.21.15.59:0/2297309165 wait complete. 2024-04-23T14:32:08.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.374+0000 7f4335400640 1 Processor -- start 2024-04-23T14:32:08.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.374+0000 7f4335400640 1 -- start start 2024-04-23T14:32:08.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f4335400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4314004800 0x7f4330159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f4335400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43300cc160 0x7f433015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f4335400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f433015ac30 0x7f43301f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f4335400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f433009b210 con 0x7f43300cc160 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f4335400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f433009aa00 con 0x7f4314004800 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f4335400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f433009b4f0 con 0x7f433015ac30 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f432f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43300cc160 0x7f433015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f432f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43300cc160 0x7f433015a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37916/0 (socket says 172.21.15.59:37916) 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f432f400640 1 -- 172.21.15.59:0/575087140 learned_addr learned my addr 172.21.15.59:0/575087140 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.375+0000 7f432fe00640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4314004800 0x7f4330159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f4334a00640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f433015ac30 0x7f43301f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f432f400640 1 -- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f433015ac30 msgr2=0x7f43301f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f432f400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f433015ac30 0x7f43301f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f432f400640 1 -- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4314004800 msgr2=0x7f4330159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f432f400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4314004800 0x7f4330159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f432f400640 1 -- 172.21.15.59:0/575087140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43301f5460 con 0x7f43300cc160 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f4334a00640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f433015ac30 0x7f43301f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:08.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f432f400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43300cc160 0x7f433015a6f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f431c007fd0 tx=0x7f431c001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f4327e00640 1 -- 172.21.15.59:0/575087140 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c036070 con 0x7f43300cc160 2024-04-23T14:32:08.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f4327e00640 1 -- 172.21.15.59:0/575087140 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431c037040 con 0x7f43300cc160 2024-04-23T14:32:08.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f4327e00640 1 -- 172.21.15.59:0/575087140 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c007490 con 0x7f43300cc160 2024-04-23T14:32:08.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.376+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43300cb680 con 0x7f43300cc160 2024-04-23T14:32:08.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.377+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43300cbc40 con 0x7f43300cc160 2024-04-23T14:32:08.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.378+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433015b040 con 0x7f43300cc160 2024-04-23T14:32:08.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.379+0000 7f4327e00640 1 -- 172.21.15.59:0/575087140 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431c047450 con 0x7f43300cc160 2024-04-23T14:32:08.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.380+0000 7f4327e00640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f431006c8f0 0x7f431006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.380+0000 7f4327e00640 1 -- 172.21.15.59:0/575087140 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f431c0ac1f0 con 0x7f43300cc160 2024-04-23T14:32:08.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.380+0000 7f432fe00640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f431006c8f0 0x7f431006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.381+0000 7f432fe00640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f431006c8f0 0x7f431006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4320001670 tx=0x7f4320009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.384+0000 7f4327e00640 1 -- 172.21.15.59:0/575087140 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f433015b040 con 0x7f43300cc160 2024-04-23T14:32:08.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.587+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f433004fd90 con 0x7f43300cc160 2024-04-23T14:32:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.588+0000 7f4327e00640 1 -- 172.21.15.59:0/575087140 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f43300b3d90 con 0x7f43300cc160 2024-04-23T14:32:08.589 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.590+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f431006c8f0 msgr2=0x7f431006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.590+0000 7f4335400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f431006c8f0 0x7f431006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4320001670 tx=0x7f4320009040 comp rx=0 tx=0).stop 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.590+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43300cc160 msgr2=0x7f433015a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.590+0000 7f4335400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43300cc160 0x7f433015a6f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f431c007fd0 tx=0x7f431c001b40 comp rx=0 tx=0).stop 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 shutdown_connections 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f431006c8f0 0x7f431006edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f433015ac30 0x7f43301f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43300cc160 0x7f433015a6f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 --2- 172.21.15.59:0/575087140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4314004800 0x7f4330159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 >> 172.21.15.59:0/575087140 conn(0x7f43300b3f70 msgr2=0x7f433015b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 shutdown_connections 2024-04-23T14:32:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.591+0000 7f4335400640 1 -- 172.21.15.59:0/575087140 wait complete. 2024-04-23T14:32:08.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:08.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.829+0000 7fee57e00640 1 Processor -- start 2024-04-23T14:32:08.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.830+0000 7fee57e00640 1 -- start start 2024-04-23T14:32:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.831+0000 7fee57e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee5015e790 0x7fee500aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.831+0000 7fee57e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fee5006f6f0 0x7fee50162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.831+0000 7fee57e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee50163370 con 0x7fee50071d70 2024-04-23T14:32:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.831+0000 7fee57e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fee500aef40 con 0x7fee5015e790 2024-04-23T14:32:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.831+0000 7fee57e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fee500afd80 con 0x7fee5006f6f0 2024-04-23T14:32:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.832+0000 7fee55400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee5015e790 0x7fee500aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.832+0000 7fee55400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee5015e790 0x7fee500aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37918/0 (socket says 172.21.15.59:37918) 2024-04-23T14:32:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.832+0000 7fee55400640 1 -- 172.21.15.59:0/2969286398 learned_addr learned my addr 172.21.15.59:0/2969286398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.832+0000 7fee55400640 1 -- 172.21.15.59:0/2969286398 >> v2:172.21.15.76:3300/0 conn(0x7fee5006f6f0 msgr2=0x7fee50162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.832+0000 7fee55400640 1 --2- 172.21.15.59:0/2969286398 >> v2:172.21.15.76:3300/0 conn(0x7fee5006f6f0 0x7fee50162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.832+0000 7fee55400640 1 -- 172.21.15.59:0/2969286398 >> v1:172.21.15.76:6789/0 conn(0x7fee50071d70 legacy=0x7fee500b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.832+0000 7fee55400640 1 -- 172.21.15.59:0/2969286398 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee500adc80 con 0x7fee5015e790 2024-04-23T14:32:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.833+0000 7fee55400640 1 --2- 172.21.15.59:0/2969286398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5015e790 0x7fee500aea00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee500b5b20 tx=0x7fee4402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b4592ae5c32ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.834+0000 7fee4fe00640 1 -- 172.21.15.59:0/2969286398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44037070 con 0x7fee5015e790 2024-04-23T14:32:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.834+0000 7fee4fe00640 1 -- 172.21.15.59:0/2969286398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee44032440 con 0x7fee5015e790 2024-04-23T14:32:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.834+0000 7fee4fe00640 1 -- 172.21.15.59:0/2969286398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44032810 con 0x7fee5015e790 2024-04-23T14:32:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.834+0000 7fee57e00640 1 -- 172.21.15.59:0/2969286398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5015e790 msgr2=0x7fee500aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.834+0000 7fee57e00640 1 --2- 172.21.15.59:0/2969286398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5015e790 0x7fee500aea00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee500b5b20 tx=0x7fee4402a230 comp rx=0 tx=0).stop 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.835+0000 7fee57e00640 1 -- 172.21.15.59:0/2969286398 shutdown_connections 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.835+0000 7fee57e00640 1 --2- 172.21.15.59:0/2969286398 >> v2:172.21.15.76:3300/0 conn(0x7fee5006f6f0 0x7fee50162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.835+0000 7fee57e00640 1 --2- 172.21.15.59:0/2969286398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5015e790 0x7fee500aea00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.835+0000 7fee57e00640 1 -- 172.21.15.59:0/2969286398 >> 172.21.15.59:0/2969286398 conn(0x7fee50067d90 msgr2=0x7fee5006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.835+0000 7fee57e00640 1 -- 172.21.15.59:0/2969286398 shutdown_connections 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.835+0000 7fee57e00640 1 -- 172.21.15.59:0/2969286398 wait complete. 2024-04-23T14:32:08.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.836+0000 7fee57e00640 1 Processor -- start 2024-04-23T14:32:08.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.837+0000 7fee57e00640 1 -- start start 2024-04-23T14:32:08.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee57e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5006f6f0 0x7fee500cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee57e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee50071d70 0x7fee5014cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee57e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee5015e790 0x7fee500cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee57e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee500ae4e0 con 0x7fee5006f6f0 2024-04-23T14:32:08.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee57e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee500ae200 con 0x7fee50071d70 2024-04-23T14:32:08.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee57e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee5006fbb0 con 0x7fee5015e790 2024-04-23T14:32:08.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5006f6f0 0x7fee500cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5006f6f0 0x7fee500cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37934/0 (socket says 172.21.15.59:37934) 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 -- 172.21.15.59:0/393641704 learned_addr learned my addr 172.21.15.59:0/393641704 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee54a00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee50071d70 0x7fee5014cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 -- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee5015e790 msgr2=0x7fee500cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee5015e790 0x7fee500cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 -- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee50071d70 msgr2=0x7fee5014cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee50071d70 0x7fee5014cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee55400640 1 -- 172.21.15.59:0/393641704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee500cf7b0 con 0x7fee5006f6f0 2024-04-23T14:32:08.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.838+0000 7fee54a00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee50071d70 0x7fee5014cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:08.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.839+0000 7fee55400640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5006f6f0 0x7fee500cde60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee44000c00 tx=0x7fee440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.839+0000 7fee4d600640 1 -- 172.21.15.59:0/393641704 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44041070 con 0x7fee5006f6f0 2024-04-23T14:32:08.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.839+0000 7fee4d600640 1 -- 172.21.15.59:0/393641704 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4403c870 con 0x7fee5006f6f0 2024-04-23T14:32:08.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.839+0000 7fee4d600640 1 -- 172.21.15.59:0/393641704 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44037040 con 0x7fee5006f6f0 2024-04-23T14:32:08.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.839+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee500ce490 con 0x7fee5006f6f0 2024-04-23T14:32:08.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.839+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5015f870 con 0x7fee5006f6f0 2024-04-23T14:32:08.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.840+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee50068a80 con 0x7fee5006f6f0 2024-04-23T14:32:08.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.841+0000 7fee4d600640 1 -- 172.21.15.59:0/393641704 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee44044070 con 0x7fee5006f6f0 2024-04-23T14:32:08.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.842+0000 7fee4d600640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3006c8f0 0x7fee3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:08.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.843+0000 7fee4d600640 1 -- 172.21.15.59:0/393641704 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fee440ac8f0 con 0x7fee5006f6f0 2024-04-23T14:32:08.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.843+0000 7fee54a00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3006c8f0 0x7fee3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:08.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.843+0000 7fee54a00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3006c8f0 0x7fee3006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee5015a5e0 tx=0x7fee3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:08.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:08.847+0000 7fee4d600640 1 -- 172.21.15.59:0/393641704 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee500cf7b0 con 0x7fee5006f6f0 2024-04-23T14:32:09.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.050+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fee50162190 con 0x7fee5006f6f0 2024-04-23T14:32:09.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.052+0000 7fee4d600640 1 -- 172.21.15.59:0/393641704 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fee50163370 con 0x7fee5006f6f0 2024-04-23T14:32:09.053 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.054+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3006c8f0 msgr2=0x7fee3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.054+0000 7fee57e00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3006c8f0 0x7fee3006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee5015a5e0 tx=0x7fee3c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.054+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5006f6f0 msgr2=0x7fee500cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.054+0000 7fee57e00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5006f6f0 0x7fee500cde60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee44000c00 tx=0x7fee440046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 shutdown_connections 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3006c8f0 0x7fee3006edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee5015e790 0x7fee500cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee50071d70 0x7fee5014cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 --2- 172.21.15.59:0/393641704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5006f6f0 0x7fee500cde60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 >> 172.21.15.59:0/393641704 conn(0x7fee50067d90 msgr2=0x7fee500b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 shutdown_connections 2024-04-23T14:32:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.055+0000 7fee57e00640 1 -- 172.21.15.59:0/393641704 wait complete. 2024-04-23T14:32:09.084 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-23T14:32:09.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:09.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.297+0000 7f0888e00640 1 Processor -- start 2024-04-23T14:32:09.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.297+0000 7f0888e00640 1 -- start start 2024-04-23T14:32:09.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.298+0000 7f0888e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 0x7f08840ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0888e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08840d0b80 0x7f08840cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0888e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f088409aaa0 con 0x7f088409bad0 2024-04-23T14:32:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0888e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f088409b640 con 0x7f08840d0b80 2024-04-23T14:32:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0888e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f088415ca50 con 0x7f088415c670 2024-04-23T14:32:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0882a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f088409bad0 0x7f08840d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35270/0 (socket says 172.21.15.59:35270) 2024-04-23T14:32:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0882a00640 1 -- 172.21.15.59:0/2948877897 learned_addr learned my addr 172.21.15.59:0/2948877897 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0883400640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 0x7f08840ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2327086531 0 0) 0x7f088409aaa0 con 0x7f088409bad0 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0883e00640 1 --2- 172.21.15.59:0/2948877897 >> v2:172.21.15.76:3300/0 conn(0x7f08840d0b80 0x7f08840cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.299+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0868003520 con 0x7f088409bad0 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3726946905 0 0) 0x7f0868003520 con 0x7f088409bad0 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 msgr2=0x7f08840ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 0x7f08840ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 >> v2:172.21.15.76:3300/0 conn(0x7f08840d0b80 msgr2=0x7f08840cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 --2- 172.21.15.59:0/2948877897 >> v2:172.21.15.76:3300/0 conn(0x7f08840d0b80 0x7f08840cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088409ad10 con 0x7f088409bad0 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f086c001670 con 0x7f088409bad0 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0883e00640 1 --2- 172.21.15.59:0/2948877897 >> v2:172.21.15.76:3300/0 conn(0x7f08840d0b80 0x7f08840cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 >> v1:172.21.15.59:6789/0 conn(0x7f088409bad0 legacy=0x7f08840d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 0x7f08680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0868003f30 0x7f0868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0868004d40 0x7f08680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f088409b640 con 0x7f0868003f30 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f088415ca50 con 0x7f0868004d40 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f086800ba10 con 0x7f088415c670 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.300+0000 7f0882a00640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0868003f30 0x7f0868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.301+0000 7f0883400640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 0x7f08680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.301+0000 7f0883e00640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0868004d40 0x7f08680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.301+0000 7f0882a00640 1 -- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 msgr2=0x7f08680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:09.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.301+0000 7f0882a00640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f088415c670 0x7f08680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.301+0000 7f0882a00640 1 -- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0868004d40 msgr2=0x7f08680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:09.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.301+0000 7f0882a00640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0868004d40 0x7f08680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.301+0000 7f0882a00640 1 -- 172.21.15.59:0/2948877897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0868003520 con 0x7f0868003f30 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.302+0000 7f0882a00640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0868003f30 0x7f0868004800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c210 tx=0x7f0868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9a1ba51eb3276a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.302+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c007d50 con 0x7f0868003f30 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.302+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f086c005ad0 con 0x7f0868003f30 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.302+0000 7f0882000640 1 -- 172.21.15.59:0/2948877897 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c007070 con 0x7f0868003f30 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.303+0000 7f0888e00640 1 -- 172.21.15.59:0/2948877897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0868003f30 msgr2=0x7f0868004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.303+0000 7f0888e00640 1 --2- 172.21.15.59:0/2948877897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0868003f30 0x7f0868004800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c210 tx=0x7f0868005830 comp rx=0 tx=0).stop 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.303+0000 7f0883e00640 1 -- 172.21.15.59:0/2948877897 reap_dead start 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.303+0000 7f0888e00640 1 -- 172.21.15.59:0/2948877897 shutdown_connections 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.303+0000 7f0888e00640 1 -- 172.21.15.59:0/2948877897 >> 172.21.15.59:0/2948877897 conn(0x7f08840b3ee0 msgr2=0x7f08840b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:09.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.303+0000 7f0888e00640 1 -- 172.21.15.59:0/2948877897 shutdown_connections 2024-04-23T14:32:09.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.303+0000 7f0888e00640 1 -- 172.21.15.59:0/2948877897 wait complete. 2024-04-23T14:32:09.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.305+0000 7f0888e00640 1 Processor -- start 2024-04-23T14:32:09.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.305+0000 7f0888e00640 1 -- start start 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.306+0000 7f0888e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0868003f30 0x7f08841568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.306+0000 7f0888e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08840b5db0 0x7f08840c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.306+0000 7f0888e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08840c9a00 0x7f088406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.306+0000 7f0888e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08840d1190 con 0x7f08840b5db0 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.306+0000 7f0888e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f088409b2d0 con 0x7f08840c9a00 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.306+0000 7f0888e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f08840d1310 con 0x7f0868003f30 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08840b5db0 0x7f08840c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08840b5db0 0x7f08840c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37964/0 (socket says 172.21.15.59:37964) 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 -- 172.21.15.59:0/3122603830 learned_addr learned my addr 172.21.15.59:0/3122603830 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0883400640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0868003f30 0x7f08841568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0883e00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08840c9a00 0x7f088406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 -- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0868003f30 msgr2=0x7f08841568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0868003f30 0x7f08841568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 -- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08840c9a00 msgr2=0x7f088406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08840c9a00 0x7f088406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.307+0000 7f0882a00640 1 -- 172.21.15.59:0/3122603830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08840c9e80 con 0x7f08840b5db0 2024-04-23T14:32:09.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.308+0000 7f0882a00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08840b5db0 0x7f08840c94c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f086c034040 tx=0x7f086c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.308+0000 7f087b400640 1 -- 172.21.15.59:0/3122603830 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c042070 con 0x7f08840b5db0 2024-04-23T14:32:09.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.308+0000 7f087b400640 1 -- 172.21.15.59:0/3122603830 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f086c007480 con 0x7f08840b5db0 2024-04-23T14:32:09.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.308+0000 7f087b400640 1 -- 172.21.15.59:0/3122603830 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c001c90 con 0x7f08840b5db0 2024-04-23T14:32:09.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.308+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08840ca160 con 0x7f08840b5db0 2024-04-23T14:32:09.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.308+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0884159520 con 0x7f08840b5db0 2024-04-23T14:32:09.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.309+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088404fd90 con 0x7f08840b5db0 2024-04-23T14:32:09.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.310+0000 7f087b400640 1 -- 172.21.15.59:0/3122603830 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f086c045070 con 0x7f08840b5db0 2024-04-23T14:32:09.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.311+0000 7f087b400640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f086006c8f0 0x7f086006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.311+0000 7f087b400640 1 -- 172.21.15.59:0/3122603830 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f086c0ac770 con 0x7f08840b5db0 2024-04-23T14:32:09.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.311+0000 7f0883400640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f086006c8f0 0x7f086006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.312+0000 7f0883400640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f086006c8f0 0x7f086006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0874009930 tx=0x7f0874008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.316+0000 7f087b400640 1 -- 172.21.15.59:0/3122603830 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f086c04c030 con 0x7f08840b5db0 2024-04-23T14:32:09.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.524+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f088415be90 con 0x7f08840b5db0 2024-04-23T14:32:09.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.526+0000 7f087b400640 1 -- 172.21.15.59:0/3122603830 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7f086c07aa90 con 0x7f08840b5db0 2024-04-23T14:32:09.527 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:09.528 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f086006c8f0 msgr2=0x7f086006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f086006c8f0 0x7f086006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0874009930 tx=0x7f0874008040 comp rx=0 tx=0).stop 2024-04-23T14:32:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08840b5db0 msgr2=0x7f08840c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08840b5db0 0x7f08840c94c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f086c034040 tx=0x7f086c001a40 comp rx=0 tx=0).stop 2024-04-23T14:32:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 shutdown_connections 2024-04-23T14:32:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f086006c8f0 0x7f086006edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08840c9a00 0x7f088406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08840b5db0 0x7f08840c94c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 --2- 172.21.15.59:0/3122603830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0868003f30 0x7f08841568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 >> 172.21.15.59:0/3122603830 conn(0x7f08840b3ee0 msgr2=0x7f08840afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:09.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 shutdown_connections 2024-04-23T14:32:09.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.529+0000 7f0888e00640 1 -- 172.21.15.59:0/3122603830 wait complete. 2024-04-23T14:32:09.552 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:09.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:09.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.763+0000 7fc3bdc00640 1 Processor -- start 2024-04-23T14:32:09.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.763+0000 7fc3bdc00640 1 -- start start 2024-04-23T14:32:09.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3bdc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3b81567e0 0x7fc3b8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3bdc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3b815d810 0x7fc3b80d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3bdc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b815cab0 con 0x7fc3b80d0b40 2024-04-23T14:32:09.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3bdc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b80d0f90 con 0x7fc3b81567e0 2024-04-23T14:32:09.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3bdc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b8160e40 con 0x7fc3b815d810 2024-04-23T14:32:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3bd200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc3b80d0b40 0x7fc3b8160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35280/0 (socket says 172.21.15.59:35280) 2024-04-23T14:32:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3bd200640 1 -- 172.21.15.59:0/100529397 learned_addr learned my addr 172.21.15.59:0/100529397 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3b7e00640 1 --2- 172.21.15.59:0/100529397 >> v2:172.21.15.59:3300/0 conn(0x7fc3b81567e0 0x7fc3b8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.765+0000 7fc3afe00640 1 --2- 172.21.15.59:0/100529397 >> v2:172.21.15.76:3300/0 conn(0x7fc3b815d810 0x7fc3b80d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.766+0000 7fc3b7400640 1 -- 172.21.15.59:0/100529397 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 566718439 0 0) 0x7fc3b0001490 con 0x7fc3b80d0b40 2024-04-23T14:32:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.766+0000 7fc3b7400640 1 -- 172.21.15.59:0/100529397 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc39c003520 con 0x7fc3b80d0b40 2024-04-23T14:32:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.766+0000 7fc3b7e00640 1 -- 172.21.15.59:0/100529397 >> v2:172.21.15.76:3300/0 conn(0x7fc3b815d810 msgr2=0x7fc3b80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.766+0000 7fc3b7e00640 1 --2- 172.21.15.59:0/100529397 >> v2:172.21.15.76:3300/0 conn(0x7fc3b815d810 0x7fc3b80d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.766+0000 7fc3b7e00640 1 -- 172.21.15.59:0/100529397 >> v1:172.21.15.59:6789/0 conn(0x7fc3b80d0b40 legacy=0x7fc3b8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.766+0000 7fc3b7e00640 1 -- 172.21.15.59:0/100529397 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b815cd30 con 0x7fc3b81567e0 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.767+0000 7fc3b7e00640 1 --2- 172.21.15.59:0/100529397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b81567e0 0x7fc3b8156bc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8074140 tx=0x7fc3a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8079d97f7db0f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.767+0000 7fc3b7400640 1 -- 172.21.15.59:0/100529397 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8037070 con 0x7fc3b81567e0 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.767+0000 7fc3b7400640 1 -- 172.21.15.59:0/100529397 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a8032440 con 0x7fc3b81567e0 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.767+0000 7fc3b7400640 1 -- 172.21.15.59:0/100529397 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8032810 con 0x7fc3b81567e0 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 -- 172.21.15.59:0/100529397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b81567e0 msgr2=0x7fc3b8156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/100529397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b81567e0 0x7fc3b8156bc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8074140 tx=0x7fc3a802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 -- 172.21.15.59:0/100529397 shutdown_connections 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/100529397 >> v2:172.21.15.76:3300/0 conn(0x7fc3b815d810 0x7fc3b80d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/100529397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b81567e0 0x7fc3b8156bc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 -- 172.21.15.59:0/100529397 >> 172.21.15.59:0/100529397 conn(0x7fc3b806bdf0 msgr2=0x7fc3b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 -- 172.21.15.59:0/100529397 shutdown_connections 2024-04-23T14:32:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.768+0000 7fc3bdc00640 1 -- 172.21.15.59:0/100529397 wait complete. 2024-04-23T14:32:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.770+0000 7fc3bdc00640 1 Processor -- start 2024-04-23T14:32:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.770+0000 7fc3bdc00640 1 -- start start 2024-04-23T14:32:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3bdc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b80d0b40 0x7fc3b8150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3bdc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3b81567e0 0x7fc3b8158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3bdc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3b815d810 0x7fc3b8159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3bdc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8160820 con 0x7fc3b80d0b40 2024-04-23T14:32:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3bdc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b815d2f0 con 0x7fc3b815d810 2024-04-23T14:32:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3bdc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b81609a0 con 0x7fc3b81567e0 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3b7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b80d0b40 0x7fc3b8150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3b7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b80d0b40 0x7fc3b8150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37982/0 (socket says 172.21.15.59:37982) 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.771+0000 7fc3b7e00640 1 -- 172.21.15.59:0/413234120 learned_addr learned my addr 172.21.15.59:0/413234120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.772+0000 7fc3b7e00640 1 -- 172.21.15.59:0/413234120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3b81567e0 msgr2=0x7fc3b8158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.772+0000 7fc3b7e00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3b81567e0 0x7fc3b8158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.772+0000 7fc3b7e00640 1 -- 172.21.15.59:0/413234120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3b815d810 msgr2=0x7fc3b8159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.772+0000 7fc3b7e00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3b815d810 0x7fc3b8159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.772+0000 7fc3b7e00640 1 -- 172.21.15.59:0/413234120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b814fe20 con 0x7fc3b80d0b40 2024-04-23T14:32:09.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.772+0000 7fc3b7e00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b80d0b40 0x7fc3b8150be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000c00 tx=0x7fc3a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.773+0000 7fc3b4c00640 1 -- 172.21.15.59:0/413234120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8041020 con 0x7fc3b80d0b40 2024-04-23T14:32:09.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.773+0000 7fc3b4c00640 1 -- 172.21.15.59:0/413234120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a803d6e0 con 0x7fc3b80d0b40 2024-04-23T14:32:09.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.773+0000 7fc3b4c00640 1 -- 172.21.15.59:0/413234120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8037040 con 0x7fc3b80d0b40 2024-04-23T14:32:09.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.773+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b8150100 con 0x7fc3b80d0b40 2024-04-23T14:32:09.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.773+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b815a010 con 0x7fc3b80d0b40 2024-04-23T14:32:09.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.774+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b804fd90 con 0x7fc3b80d0b40 2024-04-23T14:32:09.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.775+0000 7fc3b4c00640 1 -- 172.21.15.59:0/413234120 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3a8044020 con 0x7fc3b80d0b40 2024-04-23T14:32:09.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.776+0000 7fc3b4c00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc39406c8f0 0x7fc39406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:09.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.776+0000 7fc3b4c00640 1 -- 172.21.15.59:0/413234120 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc3a80ac420 con 0x7fc3b80d0b40 2024-04-23T14:32:09.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.776+0000 7fc3afe00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc39406c8f0 0x7fc39406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:09.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.777+0000 7fc3afe00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc39406c8f0 0x7fc39406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80727c0 tx=0x7fc3a0009320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:09.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.780+0000 7fc3b4c00640 1 -- 172.21.15.59:0/413234120 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3a804b030 con 0x7fc3b80d0b40 2024-04-23T14:32:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.981+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc3b80ad870 con 0x7fc3b80d0b40 2024-04-23T14:32:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.982+0000 7fc3b4c00640 1 -- 172.21.15.59:0/413234120 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fc3a807a740 con 0x7fc3b80d0b40 2024-04-23T14:32:09.986 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc39406c8f0 msgr2=0x7fc39406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc39406c8f0 0x7fc39406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80727c0 tx=0x7fc3a0009320 comp rx=0 tx=0).stop 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b80d0b40 msgr2=0x7fc3b8150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b80d0b40 0x7fc3b8150be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000c00 tx=0x7fc3a80046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 shutdown_connections 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc39406c8f0 0x7fc39406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3b81567e0 0x7fc3b8158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3b815d810 0x7fc3b8159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 --2- 172.21.15.59:0/413234120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3b80d0b40 0x7fc3b8150be0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.985+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 >> 172.21.15.59:0/413234120 conn(0x7fc3b806bdf0 msgr2=0x7fc3b8099540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.986+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 shutdown_connections 2024-04-23T14:32:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:09.986+0000 7fc3bdc00640 1 -- 172.21.15.59:0/413234120 wait complete. 2024-04-23T14:32:10.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:10.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.222+0000 7f1ea9e00640 1 Processor -- start 2024-04-23T14:32:10.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.222+0000 7f1ea9e00640 1 -- start start 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.223+0000 7f1ea9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ea40d0600 0x7f1ea40ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea406dcb0 0x7f1ea4162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea9e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ea40d09e0 0x7f1ea406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea9e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea4072a20 con 0x7f1ea40d0600 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea9e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea4163240 con 0x7f1ea40d09e0 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea4071e10 con 0x7f1ea406dcb0 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea8a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ea40d0600 0x7f1ea40ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea8a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ea40d0600 0x7f1ea40ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56906/0 (socket says 172.21.15.59:56906) 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea8a00640 1 -- 172.21.15.59:0/1844072437 learned_addr learned my addr 172.21.15.59:0/1844072437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea3e00640 1 --2- 172.21.15.59:0/1844072437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea406dcb0 0x7f1ea4162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.224+0000 7f1ea9400640 1 --2- 172.21.15.59:0/1844072437 >> v2:172.21.15.76:3300/0 conn(0x7f1ea40d09e0 0x7f1ea406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.225+0000 7f1ea8a00640 1 -- 172.21.15.59:0/1844072437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea406dcb0 msgr2=0x7f1ea4162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.225+0000 7f1ea8a00640 1 --2- 172.21.15.59:0/1844072437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea406dcb0 0x7f1ea4162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.225+0000 7f1ea8a00640 1 -- 172.21.15.59:0/1844072437 >> v2:172.21.15.76:3300/0 conn(0x7f1ea40d09e0 msgr2=0x7f1ea406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.225+0000 7f1ea8a00640 1 --2- 172.21.15.59:0/1844072437 >> v2:172.21.15.76:3300/0 conn(0x7f1ea40d09e0 0x7f1ea406e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.225+0000 7f1ea8a00640 1 -- 172.21.15.59:0/1844072437 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea40720f0 con 0x7f1ea40d0600 2024-04-23T14:32:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.226+0000 7f1ea8a00640 1 --2- 172.21.15.59:0/1844072437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d0600 0x7f1ea40ad170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4071c00 tx=0x7f1e9402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410a35eb068daa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.226+0000 7f1ea3400640 1 -- 172.21.15.59:0/1844072437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94036070 con 0x7f1ea40d0600 2024-04-23T14:32:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.226+0000 7f1ea3400640 1 -- 172.21.15.59:0/1844072437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e94031600 con 0x7f1ea40d0600 2024-04-23T14:32:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.226+0000 7f1ea3400640 1 -- 172.21.15.59:0/1844072437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e940319d0 con 0x7f1ea40d0600 2024-04-23T14:32:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 -- 172.21.15.59:0/1844072437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d0600 msgr2=0x7f1ea40ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/1844072437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d0600 0x7f1ea40ad170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4071c00 tx=0x7f1e9402a6a0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 -- 172.21.15.59:0/1844072437 shutdown_connections 2024-04-23T14:32:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/1844072437 >> v2:172.21.15.76:3300/0 conn(0x7f1ea40d09e0 0x7f1ea406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/1844072437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea406dcb0 0x7f1ea4162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/1844072437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d0600 0x7f1ea40ad170 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 -- 172.21.15.59:0/1844072437 >> 172.21.15.59:0/1844072437 conn(0x7f1ea406bdf0 msgr2=0x7f1ea406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:10.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 -- 172.21.15.59:0/1844072437 shutdown_connections 2024-04-23T14:32:10.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.227+0000 7f1ea9e00640 1 -- 172.21.15.59:0/1844072437 wait complete. 2024-04-23T14:32:10.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.229+0000 7f1ea9e00640 1 Processor -- start 2024-04-23T14:32:10.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.229+0000 7f1ea9e00640 1 -- start start 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea406dcb0 0x7f1ea41496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea40d0600 0x7f1ea415a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d09e0 0x7f1ea415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea415eaf0 con 0x7f1ea40d09e0 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea40726b0 con 0x7f1ea406dcb0 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea40d0e70 con 0x7f1ea40d0600 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d09e0 0x7f1ea415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d09e0 0x7f1ea415af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56914/0 (socket says 172.21.15.59:56914) 2024-04-23T14:32:10.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.230+0000 7f1ea9400640 1 -- 172.21.15.59:0/2860213291 learned_addr learned my addr 172.21.15.59:0/2860213291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:10.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea3e00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea40d0600 0x7f1ea415a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea8a00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea406dcb0 0x7f1ea41496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea9400640 1 -- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea40d0600 msgr2=0x7f1ea415a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea9400640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea40d0600 0x7f1ea415a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea9400640 1 -- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea406dcb0 msgr2=0x7f1ea41496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea9400640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea406dcb0 0x7f1ea41496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea9400640 1 -- 172.21.15.59:0/2860213291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea40cb740 con 0x7f1ea40d09e0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea9400640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d09e0 0x7f1ea415af10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d600 tx=0x7f1e9c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea0c00640 1 -- 172.21.15.59:0/2860213291 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0125f0 con 0x7f1ea40d09e0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea0c00640 1 -- 172.21.15.59:0/2860213291 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e9c019070 con 0x7f1ea40d09e0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea0c00640 1 -- 172.21.15.59:0/2860213291 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0129f0 con 0x7f1ea40d09e0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.231+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea415b540 con 0x7f1ea40d09e0 2024-04-23T14:32:10.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.232+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea415bb00 con 0x7f1ea40d09e0 2024-04-23T14:32:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.233+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea404fd90 con 0x7f1ea40d09e0 2024-04-23T14:32:10.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.234+0000 7f1ea0c00640 1 -- 172.21.15.59:0/2860213291 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e9c012b90 con 0x7f1ea40d09e0 2024-04-23T14:32:10.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.234+0000 7f1ea0c00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e8406c8f0 0x7f1e8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.235+0000 7f1ea0c00640 1 -- 172.21.15.59:0/2860213291 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1e9c08bfe0 con 0x7f1ea40d09e0 2024-04-23T14:32:10.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.235+0000 7f1ea8a00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e8406c8f0 0x7f1e8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.236+0000 7f1ea8a00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e8406c8f0 0x7f1e8406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e94033000 tx=0x7f1e94009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.239+0000 7f1ea0c00640 1 -- 172.21.15.59:0/2860213291 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e9c05a320 con 0x7f1ea40d09e0 2024-04-23T14:32:10.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.444+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ea40ced10 con 0x7f1ea40d09e0 2024-04-23T14:32:10.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.445+0000 7f1ea0c00640 1 -- 172.21.15.59:0/2860213291 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f1e9c059c20 con 0x7f1ea40d09e0 2024-04-23T14:32:10.446 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.447+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e8406c8f0 msgr2=0x7f1e8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.447+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e8406c8f0 0x7f1e8406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e94033000 tx=0x7f1e94009000 comp rx=0 tx=0).stop 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.447+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d09e0 msgr2=0x7f1ea415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.447+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d09e0 0x7f1ea415af10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d600 tx=0x7f1e9c00dac0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.447+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 shutdown_connections 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.448+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e8406c8f0 0x7f1e8406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.448+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ea40d0600 0x7f1ea415a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.448+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ea40d09e0 0x7f1ea415af10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.448+0000 7f1ea9e00640 1 --2- 172.21.15.59:0/2860213291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea406dcb0 0x7f1ea41496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.448+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 >> 172.21.15.59:0/2860213291 conn(0x7f1ea406bdf0 msgr2=0x7f1ea40ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.448+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 shutdown_connections 2024-04-23T14:32:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.448+0000 7f1ea9e00640 1 -- 172.21.15.59:0/2860213291 wait complete. 2024-04-23T14:32:10.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:10.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.680+0000 7f56fa000640 1 Processor -- start 2024-04-23T14:32:10.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.680+0000 7f56fa000640 1 -- start start 2024-04-23T14:32:10.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.681+0000 7f56fa000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 0x7f56f40aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56fa000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f56f406f6f0 0x7f56f4162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56fa000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f4163370 con 0x7f56f4071d70 2024-04-23T14:32:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56fa000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f56f40aef40 con 0x7f56f406f6f0 2024-04-23T14:32:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56fa000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f56f40afd80 con 0x7f56f415e790 2024-04-23T14:32:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56f3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f56f4071d70 0x7f56f40b3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57622/0 (socket says 172.21.15.59:57622) 2024-04-23T14:32:10.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56f3e00640 1 -- 172.21.15.59:0/2873754403 learned_addr learned my addr 172.21.15.59:0/2873754403 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56ebe00640 1 --2- 172.21.15.59:0/2873754403 >> v2:172.21.15.76:3300/0 conn(0x7f56f406f6f0 0x7f56f4162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.682+0000 7f56f3400640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 0x7f56f40aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581220094 0 0) 0x7f56ec001490 con 0x7f56f4071d70 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56d8003520 con 0x7f56f4071d70 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747793109 0 0) 0x7f56d8003520 con 0x7f56f4071d70 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 msgr2=0x7f56f40aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 0x7f56f40aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 >> v2:172.21.15.76:3300/0 conn(0x7f56f406f6f0 msgr2=0x7f56f4162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 --2- 172.21.15.59:0/2873754403 >> v2:172.21.15.76:3300/0 conn(0x7f56f406f6f0 0x7f56f4162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f40adc80 con 0x7f56f4071d70 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f56ec001bc0 con 0x7f56f4071d70 2024-04-23T14:32:10.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 >> v1:172.21.15.59:6789/0 conn(0x7f56f4071d70 legacy=0x7f56f40b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56d80039b0 0x7f56d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 0x7f56d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56d8004d40 0x7f56d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40aef40 con 0x7f56d8004d40 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f56d800b9c0 con 0x7f56d80039b0 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f56d800bb40 con 0x7f56f415e790 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.683+0000 7f56f3e00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56d8004d40 0x7f56d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56ebe00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 0x7f56d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56f3400640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56d80039b0 0x7f56d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56f3e00640 1 -- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 msgr2=0x7f56d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56f3e00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56f415e790 0x7f56d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56f3e00640 1 -- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56d80039b0 msgr2=0x7f56d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56f3e00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56d80039b0 0x7f56d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56f3e00640 1 -- 172.21.15.59:0/2873754403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d8003520 con 0x7f56d8004d40 2024-04-23T14:32:10.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.684+0000 7f56f3400640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56d80039b0 0x7f56d8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:10.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.685+0000 7f56f3e00640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56d8004d40 0x7f56d80051c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00aa70 tx=0x7f56d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd79f3756cdbc65 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.685+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00a690 con 0x7f56d8004d40 2024-04-23T14:32:10.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.685+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec02f070 con 0x7f56d8004d40 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.685+0000 7f56f2a00640 1 -- 172.21.15.59:0/2873754403 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec034470 con 0x7f56d8004d40 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.686+0000 7f56fa000640 1 -- 172.21.15.59:0/2873754403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56d8004d40 msgr2=0x7f56d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.686+0000 7f56fa000640 1 --2- 172.21.15.59:0/2873754403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56d8004d40 0x7f56d80051c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00aa70 tx=0x7f56d8005830 comp rx=0 tx=0).stop 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.686+0000 7f56f3e00640 1 -- 172.21.15.59:0/2873754403 reap_dead start 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.686+0000 7f56fa000640 1 -- 172.21.15.59:0/2873754403 shutdown_connections 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.686+0000 7f56fa000640 1 -- 172.21.15.59:0/2873754403 >> 172.21.15.59:0/2873754403 conn(0x7f56f4067d90 msgr2=0x7f56f406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.686+0000 7f56fa000640 1 -- 172.21.15.59:0/2873754403 shutdown_connections 2024-04-23T14:32:10.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.686+0000 7f56fa000640 1 -- 172.21.15.59:0/2873754403 wait complete. 2024-04-23T14:32:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.688+0000 7f56fa000640 1 Processor -- start 2024-04-23T14:32:10.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.688+0000 7f56fa000640 1 -- start start 2024-04-23T14:32:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56fa000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56d8004d40 0x7f56f41f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56fa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f414ef60 0x7f56f41f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56fa000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56f41560b0 0x7f56f4157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56fa000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40ae4e0 con 0x7f56f414ef60 2024-04-23T14:32:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56fa000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f56f40ae200 con 0x7f56f41560b0 2024-04-23T14:32:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56fa000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f56f406fbb0 con 0x7f56d8004d40 2024-04-23T14:32:10.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f414ef60 0x7f56f41f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56ebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f414ef60 0x7f56f41f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56924/0 (socket says 172.21.15.59:56924) 2024-04-23T14:32:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.689+0000 7f56ebe00640 1 -- 172.21.15.59:0/593890507 learned_addr learned my addr 172.21.15.59:0/593890507 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56f3400640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56d8004d40 0x7f56f41f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56f3e00640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56f41560b0 0x7f56f4157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56ebe00640 1 -- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56d8004d40 msgr2=0x7f56f41f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56ebe00640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56d8004d40 0x7f56f41f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56ebe00640 1 -- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56f41560b0 msgr2=0x7f56f4157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56ebe00640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56f41560b0 0x7f56f4157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56ebe00640 1 -- 172.21.15.59:0/593890507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f4156850 con 0x7f56f414ef60 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56ebe00640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f414ef60 0x7f56f41f7730 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56dc011700 tx=0x7f56dc011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.690+0000 7f56eb400640 1 -- 172.21.15.59:0/593890507 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00f960 con 0x7f56f414ef60 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.691+0000 7f56eb400640 1 -- 172.21.15.59:0/593890507 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56dc00fb00 con 0x7f56f414ef60 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.691+0000 7f56eb400640 1 -- 172.21.15.59:0/593890507 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc015460 con 0x7f56f414ef60 2024-04-23T14:32:10.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.691+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f41518e0 con 0x7f56f414ef60 2024-04-23T14:32:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.691+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f4151ea0 con 0x7f56f414ef60 2024-04-23T14:32:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.692+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f4068a80 con 0x7f56f414ef60 2024-04-23T14:32:10.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.693+0000 7f56eb400640 1 -- 172.21.15.59:0/593890507 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56dc015600 con 0x7f56f414ef60 2024-04-23T14:32:10.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.694+0000 7f56eb400640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f56d006c970 0x7f56d006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:10.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.694+0000 7f56eb400640 1 -- 172.21.15.59:0/593890507 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f56dc08c510 con 0x7f56f414ef60 2024-04-23T14:32:10.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.694+0000 7f56f3400640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f56d006c970 0x7f56d006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:10.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.695+0000 7f56f3400640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f56d006c970 0x7f56d006ee30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009aa0 tx=0x7f56e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:10.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.698+0000 7f56eb400640 1 -- 172.21.15.59:0/593890507 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56dc019030 con 0x7f56f414ef60 2024-04-23T14:32:10.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.904+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f56f4162190 con 0x7f56f414ef60 2024-04-23T14:32:10.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.905+0000 7f56eb400640 1 -- 172.21.15.59:0/593890507 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f56dc05a930 con 0x7f56f414ef60 2024-04-23T14:32:10.906 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:10.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f56d006c970 msgr2=0x7f56d006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f56d006c970 0x7f56d006ee30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009aa0 tx=0x7f56e4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f414ef60 msgr2=0x7f56f41f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f414ef60 0x7f56f41f7730 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56dc011700 tx=0x7f56dc011bc0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 shutdown_connections 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f56d006c970 0x7f56d006ee30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56f41560b0 0x7f56f4157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56f414ef60 0x7f56f41f7730 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 --2- 172.21.15.59:0/593890507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56d8004d40 0x7f56f41f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 >> 172.21.15.59:0/593890507 conn(0x7f56f4067d90 msgr2=0x7f56f40b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 shutdown_connections 2024-04-23T14:32:10.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:10.908+0000 7f56fa000640 1 -- 172.21.15.59:0/593890507 wait complete. 2024-04-23T14:32:10.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-23T14:32:11.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.140+0000 7fe8d6e00640 1 Processor -- start 2024-04-23T14:32:11.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.140+0000 7fe8d6e00640 1 -- start start 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.141+0000 7fe8d6e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe8d0162df0 0x7fe8d006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.141+0000 7fe8d6e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8d00b4130 0x7fe8d00c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.141+0000 7fe8d6e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d006ed40 con 0x7fe8d00cfb90 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8d6e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d006eaf0 con 0x7fe8d00b4130 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8d6e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d00b3590 con 0x7fe8d0162df0 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8d6400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8d00b4130 0x7fe8d00c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8d5000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe8d00cfb90 0x7fe8d00b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57632/0 (socket says 172.21.15.59:57632) 2024-04-23T14:32:11.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8d5000640 1 -- 172.21.15.59:0/3345368971 learned_addr learned my addr 172.21.15.59:0/3345368971 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8d5a00640 1 --2- 172.21.15.59:0/3345368971 >> v2:172.21.15.76:3300/0 conn(0x7fe8d0162df0 0x7fe8d006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8cfe00640 1 -- 172.21.15.59:0/3345368971 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16981534 0 0) 0x7fe8d006ed40 con 0x7fe8d00cfb90 2024-04-23T14:32:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.142+0000 7fe8d6400640 1 -- 172.21.15.59:0/3345368971 >> v2:172.21.15.76:3300/0 conn(0x7fe8d0162df0 msgr2=0x7fe8d006f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.143+0000 7fe8d6400640 1 --2- 172.21.15.59:0/3345368971 >> v2:172.21.15.76:3300/0 conn(0x7fe8d0162df0 0x7fe8d006f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.143+0000 7fe8d6400640 1 -- 172.21.15.59:0/3345368971 >> v1:172.21.15.59:6789/0 conn(0x7fe8d00cfb90 legacy=0x7fe8d00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.143+0000 7fe8d6400640 1 -- 172.21.15.59:0/3345368971 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d00b3220 con 0x7fe8d00b4130 2024-04-23T14:32:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.144+0000 7fe8d6400640 1 --2- 172.21.15.59:0/3345368971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00b4130 0x7fe8d00c31a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00cd890 tx=0x7fe8c802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31f134dade0086b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.144+0000 7fe8cfe00640 1 -- 172.21.15.59:0/3345368971 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8007740 con 0x7fe8d00b4130 2024-04-23T14:32:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.144+0000 7fe8cfe00640 1 -- 172.21.15.59:0/3345368971 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c80078e0 con 0x7fe8d00b4130 2024-04-23T14:32:11.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.144+0000 7fe8cfe00640 1 -- 172.21.15.59:0/3345368971 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8007cb0 con 0x7fe8d00b4130 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 -- 172.21.15.59:0/3345368971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00b4130 msgr2=0x7fe8d00c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/3345368971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00b4130 0x7fe8d00c31a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00cd890 tx=0x7fe8c802a680 comp rx=0 tx=0).stop 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 -- 172.21.15.59:0/3345368971 shutdown_connections 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/3345368971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00b4130 0x7fe8d00c31a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/3345368971 >> v2:172.21.15.76:3300/0 conn(0x7fe8d0162df0 0x7fe8d006f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 -- 172.21.15.59:0/3345368971 >> 172.21.15.59:0/3345368971 conn(0x7fe8d00b21a0 msgr2=0x7fe8d00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 -- 172.21.15.59:0/3345368971 shutdown_connections 2024-04-23T14:32:11.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.145+0000 7fe8d6e00640 1 -- 172.21.15.59:0/3345368971 wait complete. 2024-04-23T14:32:11.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.147+0000 7fe8d6e00640 1 Processor -- start 2024-04-23T14:32:11.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.147+0000 7fe8d6e00640 1 -- start start 2024-04-23T14:32:11.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d6e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8d00b4130 0x7fe8d015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00cfb90 0x7fe8d00c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d6e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d0162df0 0x7fe8d00c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d00c3860 con 0x7fe8d00cfb90 2024-04-23T14:32:11.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d6e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d00c36e0 con 0x7fe8d0162df0 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d6e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d00c39e0 con 0x7fe8d00b4130 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00cfb90 0x7fe8d00c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00cfb90 0x7fe8d00c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56940/0 (socket says 172.21.15.59:56940) 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.148+0000 7fe8d5000640 1 -- 172.21.15.59:0/1894216356 learned_addr learned my addr 172.21.15.59:0/1894216356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d5a00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8d00b4130 0x7fe8d015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d5000640 1 -- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8d00b4130 msgr2=0x7fe8d015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d6400640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d0162df0 0x7fe8d00c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d5000640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8d00b4130 0x7fe8d015f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d5000640 1 -- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d0162df0 msgr2=0x7fe8d00c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d5000640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d0162df0 0x7fe8d00c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d5000640 1 -- 172.21.15.59:0/1894216356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d015fc20 con 0x7fe8d00cfb90 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d5000640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00cfb90 0x7fe8d00c5000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00c4d0 tx=0x7fe8bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8cd600640 1 -- 172.21.15.59:0/1894216356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc018070 con 0x7fe8d00cfb90 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8cd600640 1 -- 172.21.15.59:0/1894216356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8bc007500 con 0x7fe8d00cfb90 2024-04-23T14:32:11.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8cd600640 1 -- 172.21.15.59:0/1894216356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc004a20 con 0x7fe8d00cfb90 2024-04-23T14:32:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.149+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d01f9320 con 0x7fe8d00cfb90 2024-04-23T14:32:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.150+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d0072840 con 0x7fe8d00cfb90 2024-04-23T14:32:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.151+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d004fd90 con 0x7fe8d00cfb90 2024-04-23T14:32:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.152+0000 7fe8cd600640 1 -- 172.21.15.59:0/1894216356 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8bc004bc0 con 0x7fe8d00cfb90 2024-04-23T14:32:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.153+0000 7fe8cd600640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe8b006c8f0 0x7fe8b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.153+0000 7fe8cd600640 1 -- 172.21.15.59:0/1894216356 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe8bc08af10 con 0x7fe8d00cfb90 2024-04-23T14:32:11.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.153+0000 7fe8d5a00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe8b006c8f0 0x7fe8b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.154+0000 7fe8d5a00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe8b006c8f0 0x7fe8b006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006fd0 tx=0x7fe8c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:11.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.158+0000 7fe8cd600640 1 -- 172.21.15.59:0/1894216356 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8bc059220 con 0x7fe8d00cfb90 2024-04-23T14:32:11.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.437+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fe8d00c5540 con 0x7fe8d00cfb90 2024-04-23T14:32:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.543+0000 7fe8cd600640 1 -- 172.21.15.59:0/1894216356 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v61) ==== 145+0+0 (secure 0 0 0) 0x7fe8bc058b20 con 0x7fe8d00cfb90 2024-04-23T14:32:11.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe8b006c8f0 msgr2=0x7fe8b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe8b006c8f0 0x7fe8b006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4006fd0 tx=0x7fe8c4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:11.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00cfb90 msgr2=0x7fe8d00c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00cfb90 0x7fe8d00c5000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00c4d0 tx=0x7fe8bc00c990 comp rx=0 tx=0).stop 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 shutdown_connections 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe8b006c8f0 0x7fe8b006edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d0162df0 0x7fe8d00c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d00cfb90 0x7fe8d00c5000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 --2- 172.21.15.59:0/1894216356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8d00b4130 0x7fe8d015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.547+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 >> 172.21.15.59:0/1894216356 conn(0x7fe8d00b21a0 msgr2=0x7fe8d00b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.548+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 shutdown_connections 2024-04-23T14:32:11.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.548+0000 7fe8d6e00640 1 -- 172.21.15.59:0/1894216356 wait complete. 2024-04-23T14:32:11.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:32:11.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.790+0000 7effd9200640 1 Processor -- start 2024-04-23T14:32:11.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.791+0000 7effd9200640 1 -- start start 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effd9200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7effd4155a80 0x7effd406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effd9200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7effd40af2a0 0x7effd40d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effd9200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd415ef20 con 0x7effd40d0600 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effd9200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7effd415f1c0 con 0x7effd40af2a0 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effd9200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7effd415f4a0 con 0x7effd4155a80 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effcbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7effd40d0600 0x7effd40b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57644/0 (socket says 172.21.15.59:57644) 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effcbe00640 1 -- 172.21.15.59:0/3258823032 learned_addr learned my addr 172.21.15.59:0/3258823032 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:11.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effd3e00640 1 --2- 172.21.15.59:0/3258823032 >> v2:172.21.15.59:3300/0 conn(0x7effd40af2a0 0x7effd40d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.792+0000 7effd3400640 1 --2- 172.21.15.59:0/3258823032 >> v2:172.21.15.76:3300/0 conn(0x7effd4155a80 0x7effd406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.793+0000 7effd2a00640 1 -- 172.21.15.59:0/3258823032 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1772689899 0 0) 0x7effd415ef20 con 0x7effd40d0600 2024-04-23T14:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.793+0000 7effd2a00640 1 -- 172.21.15.59:0/3258823032 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effb8005730 con 0x7effd40d0600 2024-04-23T14:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.793+0000 7effd3e00640 1 -- 172.21.15.59:0/3258823032 >> v2:172.21.15.76:3300/0 conn(0x7effd4155a80 msgr2=0x7effd406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.793+0000 7effd3e00640 1 --2- 172.21.15.59:0/3258823032 >> v2:172.21.15.76:3300/0 conn(0x7effd4155a80 0x7effd406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.793+0000 7effd3e00640 1 -- 172.21.15.59:0/3258823032 >> v1:172.21.15.59:6789/0 conn(0x7effd40d0600 legacy=0x7effd40b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.793+0000 7effd3e00640 1 -- 172.21.15.59:0/3258823032 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd415f780 con 0x7effd40af2a0 2024-04-23T14:32:11.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.793+0000 7effd3400640 1 --2- 172.21.15.59:0/3258823032 >> v2:172.21.15.76:3300/0 conn(0x7effd4155a80 0x7effd406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.794+0000 7effd3e00640 1 --2- 172.21.15.59:0/3258823032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40af2a0 0x7effd40d0a90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7effd406dfb0 tx=0x7effcc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bb495260aadbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.794+0000 7effd2a00640 1 -- 172.21.15.59:0/3258823032 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc007740 con 0x7effd40af2a0 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.794+0000 7effd2a00640 1 -- 172.21.15.59:0/3258823032 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0078e0 con 0x7effd40af2a0 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.794+0000 7effd2a00640 1 -- 172.21.15.59:0/3258823032 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc007cb0 con 0x7effd40af2a0 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 -- 172.21.15.59:0/3258823032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40af2a0 msgr2=0x7effd40d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 --2- 172.21.15.59:0/3258823032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40af2a0 0x7effd40d0a90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7effd406dfb0 tx=0x7effcc02a680 comp rx=0 tx=0).stop 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 -- 172.21.15.59:0/3258823032 shutdown_connections 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 --2- 172.21.15.59:0/3258823032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40af2a0 0x7effd40d0a90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 --2- 172.21.15.59:0/3258823032 >> v2:172.21.15.76:3300/0 conn(0x7effd4155a80 0x7effd406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 -- 172.21.15.59:0/3258823032 >> 172.21.15.59:0/3258823032 conn(0x7effd406bdf0 msgr2=0x7effd406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 -- 172.21.15.59:0/3258823032 shutdown_connections 2024-04-23T14:32:11.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.795+0000 7effd9200640 1 -- 172.21.15.59:0/3258823032 wait complete. 2024-04-23T14:32:11.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.797+0000 7effd9200640 1 Processor -- start 2024-04-23T14:32:11.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.797+0000 7effd9200640 1 -- start start 2024-04-23T14:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd9200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd40af2a0 0x7effd40b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40d0600 0x7effd40b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd9200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effd4155a80 0x7effd41f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7effd406ff00 con 0x7effd40d0600 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd9200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7effd415fd40 con 0x7effd40af2a0 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd9200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7effd406ef80 con 0x7effd4155a80 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effcbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40d0600 0x7effd40b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effcbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40d0600 0x7effd40b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56962/0 (socket says 172.21.15.59:56962) 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effcbe00640 1 -- 172.21.15.59:0/3301376315 learned_addr learned my addr 172.21.15.59:0/3301376315 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effcbe00640 1 -- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effd4155a80 msgr2=0x7effd41f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd3400640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd40af2a0 0x7effd40b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effd3e00640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effd4155a80 0x7effd41f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.798+0000 7effcbe00640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effd4155a80 0x7effd41f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effcbe00640 1 -- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd40af2a0 msgr2=0x7effd40b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effcbe00640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd40af2a0 0x7effd40b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effcbe00640 1 -- 172.21.15.59:0/3301376315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd40b2200 con 0x7effd40d0600 2024-04-23T14:32:11.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effcbe00640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40d0600 0x7effd40b1a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c430 tx=0x7effbc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effcb400640 1 -- 172.21.15.59:0/3301376315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc017070 con 0x7effd40d0600 2024-04-23T14:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effcb400640 1 -- 172.21.15.59:0/3301376315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effbc005ce0 con 0x7effd40d0600 2024-04-23T14:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effcb400640 1 -- 172.21.15.59:0/3301376315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc0072f0 con 0x7effd40d0600 2024-04-23T14:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd40b24e0 con 0x7effd40d0600 2024-04-23T14:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.799+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd40b2aa0 con 0x7effd40d0600 2024-04-23T14:32:11.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.800+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd404fd90 con 0x7effd40d0600 2024-04-23T14:32:11.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.802+0000 7effcb400640 1 -- 172.21.15.59:0/3301376315 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effbc01a070 con 0x7effd40d0600 2024-04-23T14:32:11.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.802+0000 7effcb400640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effb006c920 0x7effb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:11.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.803+0000 7effcb400640 1 -- 172.21.15.59:0/3301376315 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7effbc08b100 con 0x7effd40d0600 2024-04-23T14:32:11.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.803+0000 7effd3400640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effb006c920 0x7effb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:11.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.804+0000 7effd3400640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effb006c920 0x7effb006ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7effc40096d0 tx=0x7effc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:11.808+0000 7effcb400640 1 -- 172.21.15.59:0/3301376315 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effbc059410 con 0x7effd40d0600 2024-04-23T14:32:12.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.007+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7effd406eba0 con 0x7effd40d0600 2024-04-23T14:32:12.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.014+0000 7effcb400640 1 -- 172.21.15.59:0/3301376315 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7effbc058d10 con 0x7effd40d0600 2024-04-23T14:32:12.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effb006c920 msgr2=0x7effb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effb006c920 0x7effb006ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7effc40096d0 tx=0x7effc4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:12.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40d0600 msgr2=0x7effd40b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40d0600 0x7effd40b1a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c430 tx=0x7effbc00c8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 shutdown_connections 2024-04-23T14:32:12.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effb006c920 0x7effb006ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effd4155a80 0x7effd41f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effd40d0600 0x7effd40b1a70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 --2- 172.21.15.59:0/3301376315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd40af2a0 0x7effd40b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 >> 172.21.15.59:0/3301376315 conn(0x7effd406bdf0 msgr2=0x7effd4072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:12.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 shutdown_connections 2024-04-23T14:32:12.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.017+0000 7effd9200640 1 -- 172.21.15.59:0/3301376315 wait complete. 2024-04-23T14:32:12.037 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:32:12.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:32:12.154 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:32:12.276 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:32:12.276 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:32:12.276 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:12.337 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:32:12.337 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:12.394 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:12.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.596+0000 7f7e37e00640 1 Processor -- start 2024-04-23T14:32:12.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.597+0000 7f7e37e00640 1 -- start start 2024-04-23T14:32:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.598+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 0x7f7e380cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.598+0000 7f7e37e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e380b3870 con 0x7f7e3815b450 2024-04-23T14:32:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.598+0000 7f7e37e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e380b3b50 con 0x7f7e380b3080 2024-04-23T14:32:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.598+0000 7f7e37e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7e380b41a0 con 0x7f7e3806f3f0 2024-04-23T14:32:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.598+0000 7f7e36a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7e3815b450 0x7f7e380b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57654/0 (socket says 172.21.15.59:57654) 2024-04-23T14:32:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.598+0000 7f7e36a00640 1 -- 172.21.15.59:0/4093993696 learned_addr learned my addr 172.21.15.59:0/4093993696 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3071421699 0 0) 0x7f7e380b3870 con 0x7f7e3815b450 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e36000640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 0x7f7e380cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e18005730 con 0x7f7e3815b450 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3282672541 0 0) 0x7f7e18005730 con 0x7f7e3815b450 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 msgr2=0x7f7e380cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 0x7f7e380cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 >> v1:172.21.15.76:6789/0 conn(0x7f7e380b3080 legacy=0x7f7e380ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e380b3e30 con 0x7f7e3815b450 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f7e28001670 con 0x7f7e3815b450 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 >> v1:172.21.15.59:6789/0 conn(0x7f7e3815b450 legacy=0x7f7e380b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e18005bf0 0x7f7e18006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 0x7f7e18006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18006f50 0x7f7e180073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380b41a0 con 0x7f7e18005bf0 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1800dcb0 con 0x7f7e18006f50 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.599+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1800deb0 con 0x7f7e3806f3f0 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.600+0000 7f7e36a00640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e18005bf0 0x7f7e18006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.600+0000 7f7e36000640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 0x7f7e18006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.600+0000 7f7e36a00640 1 -- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 msgr2=0x7f7e18006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.600+0000 7f7e36a00640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3806f3f0 0x7f7e18006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.600+0000 7f7e36a00640 1 -- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18006f50 msgr2=0x7f7e180073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.600+0000 7f7e36a00640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18006f50 0x7f7e180073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.600+0000 7f7e36a00640 1 -- 172.21.15.59:0/4093993696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e18005730 con 0x7f7e18005bf0 2024-04-23T14:32:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.601+0000 7f7e36a00640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e18005bf0 0x7f7e18006010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c0e0 tx=0x7f7e18007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe0c2c07aa4a5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.601+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28007c40 con 0x7f7e18005bf0 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.601+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e28005af0 con 0x7f7e18005bf0 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.601+0000 7f7e35600640 1 -- 172.21.15.59:0/4093993696 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28007070 con 0x7f7e18005bf0 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.602+0000 7f7e37e00640 1 -- 172.21.15.59:0/4093993696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e18005bf0 msgr2=0x7f7e18006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.602+0000 7f7e37e00640 1 --2- 172.21.15.59:0/4093993696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e18005bf0 0x7f7e18006010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c0e0 tx=0x7f7e18007a40 comp rx=0 tx=0).stop 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.602+0000 7f7e37400640 1 -- 172.21.15.59:0/4093993696 reap_dead start 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.602+0000 7f7e37e00640 1 -- 172.21.15.59:0/4093993696 shutdown_connections 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.602+0000 7f7e37e00640 1 -- 172.21.15.59:0/4093993696 >> 172.21.15.59:0/4093993696 conn(0x7f7e380b21a0 msgr2=0x7f7e380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.602+0000 7f7e37e00640 1 -- 172.21.15.59:0/4093993696 shutdown_connections 2024-04-23T14:32:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.602+0000 7f7e37e00640 1 -- 172.21.15.59:0/4093993696 wait complete. 2024-04-23T14:32:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.604+0000 7f7e37e00640 1 Processor -- start 2024-04-23T14:32:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.604+0000 7f7e37e00640 1 -- start start 2024-04-23T14:32:12.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.605+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18005bf0 0x7f7e38097ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.605+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e3815bb80 0x7f7e380cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.605+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3815e1d0 0x7f7e38154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.605+0000 7f7e37e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380ad770 con 0x7f7e3815bb80 2024-04-23T14:32:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.605+0000 7f7e37e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380b4ff0 con 0x7f7e18005bf0 2024-04-23T14:32:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.605+0000 7f7e37e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7e380ad8f0 con 0x7f7e3815e1d0 2024-04-23T14:32:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.605+0000 7f7e36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e3815bb80 0x7f7e380cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e3815bb80 0x7f7e380cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56976/0 (socket says 172.21.15.59:56976) 2024-04-23T14:32:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 -- 172.21.15.59:0/3143269567 learned_addr learned my addr 172.21.15.59:0/3143269567 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36a00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18005bf0 0x7f7e38097ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e37400640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3815e1d0 0x7f7e38154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 -- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3815e1d0 msgr2=0x7f7e38154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3815e1d0 0x7f7e38154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 -- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18005bf0 msgr2=0x7f7e38097ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18005bf0 0x7f7e38097ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 -- 172.21.15.59:0/3143269567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3815d730 con 0x7f7e3815bb80 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.606+0000 7f7e36000640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e3815bb80 0x7f7e380cb900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000f3b0 tx=0x7f7e2000f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.607+0000 7f7e2f400640 1 -- 172.21.15.59:0/3143269567 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2001c070 con 0x7f7e3815bb80 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.607+0000 7f7e2f400640 1 -- 172.21.15.59:0/3143269567 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e20017620 con 0x7f7e3815bb80 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.607+0000 7f7e2f400640 1 -- 172.21.15.59:0/3143269567 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20017a10 con 0x7f7e3815bb80 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.607+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3815c000 con 0x7f7e3815bb80 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.607+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3815d9d0 con 0x7f7e3815bb80 2024-04-23T14:32:12.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.608+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3804fd90 con 0x7f7e3815bb80 2024-04-23T14:32:12.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.609+0000 7f7e2f400640 1 -- 172.21.15.59:0/3143269567 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e20005d10 con 0x7f7e3815bb80 2024-04-23T14:32:12.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.610+0000 7f7e2f400640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e1406c8f0 0x7f7e1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:12.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.610+0000 7f7e2f400640 1 -- 172.21.15.59:0/3143269567 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7e2008c180 con 0x7f7e3815bb80 2024-04-23T14:32:12.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.610+0000 7f7e36a00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e1406c8f0 0x7f7e1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:12.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.611+0000 7f7e36a00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e1406c8f0 0x7f7e1406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e28032000 tx=0x7f7e28009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:12.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.614+0000 7f7e2f400640 1 -- 172.21.15.59:0/3143269567 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e20012070 con 0x7f7e3815bb80 2024-04-23T14:32:12.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.822+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e381595a0 con 0x7f7e3815bb80 2024-04-23T14:32:12.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.823+0000 7f7e2f400640 1 -- 172.21.15.59:0/3143269567 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7f7e2005a5e0 con 0x7f7e3815bb80 2024-04-23T14:32:12.825 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:12.825 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e1406c8f0 msgr2=0x7f7e1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e1406c8f0 0x7f7e1406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e28032000 tx=0x7f7e28009030 comp rx=0 tx=0).stop 2024-04-23T14:32:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e3815bb80 msgr2=0x7f7e380cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e3815bb80 0x7f7e380cb900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000f3b0 tx=0x7f7e2000f870 comp rx=0 tx=0).stop 2024-04-23T14:32:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 shutdown_connections 2024-04-23T14:32:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e1406c8f0 0x7f7e1406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e3815e1d0 0x7f7e38154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e3815bb80 0x7f7e380cb900 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 --2- 172.21.15.59:0/3143269567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e18005bf0 0x7f7e38097ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 >> 172.21.15.59:0/3143269567 conn(0x7f7e380b21a0 msgr2=0x7f7e380cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 shutdown_connections 2024-04-23T14:32:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:12.826+0000 7f7e37e00640 1 -- 172.21.15.59:0/3143269567 wait complete. 2024-04-23T14:32:12.845 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:12.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.052+0000 7fc339000640 1 Processor -- start 2024-04-23T14:32:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.052+0000 7fc339000640 1 -- start start 2024-04-23T14:32:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.054+0000 7fc339000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 0x7fc3340d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.054+0000 7fc339000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc33415cfb0 con 0x7fc33409aa00 2024-04-23T14:32:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.054+0000 7fc339000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc33415d250 con 0x7fc3340cd120 2024-04-23T14:32:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.054+0000 7fc339000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc33415dd30 con 0x7fc33409b750 2024-04-23T14:32:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.054+0000 7fc331e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc33409aa00 0x7fc33409ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57670/0 (socket says 172.21.15.59:57670) 2024-04-23T14:32:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.054+0000 7fc331e00640 1 -- 172.21.15.59:0/3624886534 learned_addr learned my addr 172.21.15.59:0/3624886534 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.055+0000 7fc331400640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 0x7fc3340d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.055+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2138582373 0 0) 0x7fc33415cfb0 con 0x7fc33409aa00 2024-04-23T14:32:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.055+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc318003520 con 0x7fc33409aa00 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.055+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2505443646 0 0) 0x7fc318003520 con 0x7fc33409aa00 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.055+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 msgr2=0x7fc3340d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.055+0000 7fc330a00640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 0x7fc3340d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.055+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 >> v1:172.21.15.76:6789/0 conn(0x7fc3340cd120 legacy=0x7fc3340cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33415d530 con 0x7fc33409aa00 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fc324001670 con 0x7fc33409aa00 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 >> v1:172.21.15.59:6789/0 conn(0x7fc33409aa00 legacy=0x7fc33409ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 0x7fc3180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc318003fb0 0x7fc318004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc33415dd30 con 0x7fc318004d40 2024-04-23T14:32:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc31800bbb0 con 0x7fc318003fb0 2024-04-23T14:32:13.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc31800bdb0 con 0x7fc33409b750 2024-04-23T14:32:13.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc332800640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc331400640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc318003fb0 0x7fc318004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.056+0000 7fc331e00640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 0x7fc3180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.057+0000 7fc332800640 1 -- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 msgr2=0x7fc3180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.057+0000 7fc332800640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc33409b750 0x7fc3180039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.057+0000 7fc332800640 1 -- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc318003fb0 msgr2=0x7fc318004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.057+0000 7fc332800640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc318003fb0 0x7fc318004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.057+0000 7fc332800640 1 -- 172.21.15.59:0/3624886534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc318003520 con 0x7fc318004d40 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.058+0000 7fc332800640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3180051c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc32c001e50 tx=0x7fc318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89629ed12dc933e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.058+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c001a70 con 0x7fc318004d40 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.058+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc32c003970 con 0x7fc318004d40 2024-04-23T14:32:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.058+0000 7fc330a00640 1 -- 172.21.15.59:0/3624886534 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c003d60 con 0x7fc318004d40 2024-04-23T14:32:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.058+0000 7fc339000640 1 -- 172.21.15.59:0/3624886534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 msgr2=0x7fc3180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.058+0000 7fc339000640 1 --2- 172.21.15.59:0/3624886534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3180051c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc32c001e50 tx=0x7fc318005830 comp rx=0 tx=0).stop 2024-04-23T14:32:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.058+0000 7fc332800640 1 -- 172.21.15.59:0/3624886534 reap_dead start 2024-04-23T14:32:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.059+0000 7fc339000640 1 -- 172.21.15.59:0/3624886534 shutdown_connections 2024-04-23T14:32:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.059+0000 7fc339000640 1 -- 172.21.15.59:0/3624886534 >> 172.21.15.59:0/3624886534 conn(0x7fc3340b4090 msgr2=0x7fc3340b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.059+0000 7fc339000640 1 -- 172.21.15.59:0/3624886534 shutdown_connections 2024-04-23T14:32:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.059+0000 7fc339000640 1 -- 172.21.15.59:0/3624886534 wait complete. 2024-04-23T14:32:13.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.060+0000 7fc339000640 1 Processor -- start 2024-04-23T14:32:13.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.061+0000 7fc339000640 1 -- start start 2024-04-23T14:32:13.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.061+0000 7fc339000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3340b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc339000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc334154ad0 0x7fc33415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc339000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc33415f470 0x7fc3341597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc339000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc33415d8a0 con 0x7fc318004d40 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc339000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3340d09c0 con 0x7fc33415f470 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc339000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc3340d0170 con 0x7fc334154ad0 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc331e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3340b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc331e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3340b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56994/0 (socket says 172.21.15.59:56994) 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc331e00640 1 -- 172.21.15.59:0/331292341 learned_addr learned my addr 172.21.15.59:0/331292341 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc331400640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc334154ad0 0x7fc33415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.062+0000 7fc332800640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc33415f470 0x7fc3341597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc331e00640 1 -- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc334154ad0 msgr2=0x7fc33415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc331e00640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc334154ad0 0x7fc33415ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc331e00640 1 -- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc33415f470 msgr2=0x7fc3341597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc331e00640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc33415f470 0x7fc3341597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc331e00640 1 -- 172.21.15.59:0/331292341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3340af390 con 0x7fc318004d40 2024-04-23T14:32:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc332800640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc33415f470 0x7fc3341597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc331e00640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3340b0150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc32400d810 tx=0x7fc32400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc32a000640 1 -- 172.21.15.59:0/331292341 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324019070 con 0x7fc318004d40 2024-04-23T14:32:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc32a000640 1 -- 172.21.15.59:0/331292341 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3240074e0 con 0x7fc318004d40 2024-04-23T14:32:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc32a000640 1 -- 172.21.15.59:0/331292341 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324004a00 con 0x7fc318004d40 2024-04-23T14:32:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.063+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3340af670 con 0x7fc318004d40 2024-04-23T14:32:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.064+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc334154f50 con 0x7fc318004d40 2024-04-23T14:32:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.065+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc334068a80 con 0x7fc318004d40 2024-04-23T14:32:13.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.066+0000 7fc32a000640 1 -- 172.21.15.59:0/331292341 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc324004ba0 con 0x7fc318004d40 2024-04-23T14:32:13.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.066+0000 7fc32a000640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc31406c920 0x7fc31406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.067+0000 7fc32a000640 1 -- 172.21.15.59:0/331292341 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc32408b7b0 con 0x7fc318004d40 2024-04-23T14:32:13.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.067+0000 7fc331400640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc31406c920 0x7fc31406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.067+0000 7fc331400640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc31406c920 0x7fc31406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc33415a6a0 tx=0x7fc320008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.071+0000 7fc32a000640 1 -- 172.21.15.59:0/331292341 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc324059a30 con 0x7fc318004d40 2024-04-23T14:32:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.277+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc33409b530 con 0x7fc318004d40 2024-04-23T14:32:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.278+0000 7fc32a000640 1 -- 172.21.15.59:0/331292341 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fc324059330 con 0x7fc318004d40 2024-04-23T14:32:13.279 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc31406c920 msgr2=0x7fc31406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc31406c920 0x7fc31406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc33415a6a0 tx=0x7fc320008040 comp rx=0 tx=0).stop 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 msgr2=0x7fc3340b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3340b0150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc32400d810 tx=0x7fc32400dcd0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 shutdown_connections 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc31406c920 0x7fc31406ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc334154ad0 0x7fc33415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc33415f470 0x7fc3341597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.280+0000 7fc339000640 1 --2- 172.21.15.59:0/331292341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc318004d40 0x7fc3340b0150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.281+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 >> 172.21.15.59:0/331292341 conn(0x7fc3340b4090 msgr2=0x7fc3340b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.281+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 shutdown_connections 2024-04-23T14:32:13.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.281+0000 7fc339000640 1 -- 172.21.15.59:0/331292341 wait complete. 2024-04-23T14:32:13.302 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.509+0000 7fc1a6200640 1 Processor -- start 2024-04-23T14:32:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.509+0000 7fc1a6200640 1 -- start start 2024-04-23T14:32:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.511+0000 7fc1a6200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc1a015ed20 0x7fc1a00d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.511+0000 7fc1a6200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a00b3600 0x7fc1a00d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.511+0000 7fc1a6200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a00b5da0 con 0x7fc1a00b3f10 2024-04-23T14:32:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.511+0000 7fc1a6200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a015f100 con 0x7fc1a015ed20 2024-04-23T14:32:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.511+0000 7fc1a6200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a00d0a20 con 0x7fc1a00b3600 2024-04-23T14:32:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.511+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a00b3600 0x7fc1a00d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.512+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a00b3600 0x7fc1a00d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:39790/0 (socket says 172.21.15.59:39790) 2024-04-23T14:32:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.512+0000 7fc19fe00640 1 -- 172.21.15.59:0/3188296339 learned_addr learned my addr 172.21.15.59:0/3188296339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.512+0000 7fc1a4e00640 1 --2- 172.21.15.59:0/3188296339 >> v2:172.21.15.76:3300/0 conn(0x7fc1a015ed20 0x7fc1a00d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.512+0000 7fc19fe00640 1 -- 172.21.15.59:0/3188296339 >> v2:172.21.15.76:3300/0 conn(0x7fc1a015ed20 msgr2=0x7fc1a00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.512+0000 7fc19fe00640 1 --2- 172.21.15.59:0/3188296339 >> v2:172.21.15.76:3300/0 conn(0x7fc1a015ed20 0x7fc1a00d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.512+0000 7fc19fe00640 1 -- 172.21.15.59:0/3188296339 >> v1:172.21.15.76:6789/0 conn(0x7fc1a00b3f10 legacy=0x7fc1a00b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.512+0000 7fc19fe00640 1 -- 172.21.15.59:0/3188296339 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00d2ed0 con 0x7fc1a00b3600 2024-04-23T14:32:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.514+0000 7fc19fe00640 1 --2- 172.21.15.59:0/3188296339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a00b3600 0x7fc1a00d13e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009870 tx=0x7fc18c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31ac75279c05f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.515+0000 7fc19f400640 1 -- 172.21.15.59:0/3188296339 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c037070 con 0x7fc1a00b3600 2024-04-23T14:32:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.515+0000 7fc19f400640 1 -- 172.21.15.59:0/3188296339 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c032440 con 0x7fc1a00b3600 2024-04-23T14:32:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.515+0000 7fc19f400640 1 -- 172.21.15.59:0/3188296339 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c032810 con 0x7fc1a00b3600 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.515+0000 7fc1a6200640 1 -- 172.21.15.59:0/3188296339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a00b3600 msgr2=0x7fc1a00d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.515+0000 7fc1a6200640 1 --2- 172.21.15.59:0/3188296339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a00b3600 0x7fc1a00d13e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009870 tx=0x7fc18c02a130 comp rx=0 tx=0).stop 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.516+0000 7fc1a6200640 1 -- 172.21.15.59:0/3188296339 shutdown_connections 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.516+0000 7fc1a6200640 1 --2- 172.21.15.59:0/3188296339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a00b3600 0x7fc1a00d13e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.516+0000 7fc1a6200640 1 --2- 172.21.15.59:0/3188296339 >> v2:172.21.15.76:3300/0 conn(0x7fc1a015ed20 0x7fc1a00d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.516+0000 7fc1a6200640 1 -- 172.21.15.59:0/3188296339 >> 172.21.15.59:0/3188296339 conn(0x7fc1a008bf90 msgr2=0x7fc1a00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.516+0000 7fc1a6200640 1 -- 172.21.15.59:0/3188296339 shutdown_connections 2024-04-23T14:32:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.516+0000 7fc1a6200640 1 -- 172.21.15.59:0/3188296339 wait complete. 2024-04-23T14:32:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.518+0000 7fc1a6200640 1 Processor -- start 2024-04-23T14:32:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.519+0000 7fc1a6200640 1 -- start start 2024-04-23T14:32:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc1a6200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00b3600 0x7fc1a0154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc1a6200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a00b3f10 0x7fc1a00a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc1a6200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015ed20 0x7fc1a00c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc1a6200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00d0290 con 0x7fc1a00b3f10 2024-04-23T14:32:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc1a6200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00d3490 con 0x7fc1a00b3600 2024-04-23T14:32:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc1a6200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a00d0410 con 0x7fc1a015ed20 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a00b3f10 0x7fc1a00a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a00b3f10 0x7fc1a00a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57010/0 (socket says 172.21.15.59:57010) 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.520+0000 7fc19fe00640 1 -- 172.21.15.59:0/893516453 learned_addr learned my addr 172.21.15.59:0/893516453 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc1a4e00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00b3600 0x7fc1a0154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc1a5800640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015ed20 0x7fc1a00c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19fe00640 1 -- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015ed20 msgr2=0x7fc1a00c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19fe00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015ed20 0x7fc1a00c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19fe00640 1 -- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00b3600 msgr2=0x7fc1a0154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19fe00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00b3600 0x7fc1a0154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19fe00640 1 -- 172.21.15.59:0/893516453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00cb810 con 0x7fc1a00b3f10 2024-04-23T14:32:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc1a4e00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00b3600 0x7fc1a0154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19fe00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a00b3f10 0x7fc1a00a12e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19cc00640 1 -- 172.21.15.59:0/893516453 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c041020 con 0x7fc1a00b3f10 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.521+0000 7fc19cc00640 1 -- 172.21.15.59:0/893516453 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c030640 con 0x7fc1a00b3f10 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.522+0000 7fc19cc00640 1 -- 172.21.15.59:0/893516453 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c037040 con 0x7fc1a00b3f10 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.522+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a00a1d30 con 0x7fc1a00b3f10 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.522+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a0154e70 con 0x7fc1a00b3f10 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.523+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a004fd90 con 0x7fc1a00b3f10 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.524+0000 7fc19cc00640 1 -- 172.21.15.59:0/893516453 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc18c044020 con 0x7fc1a00b3f10 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.525+0000 7fc19cc00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc18006c8f0 0x7fc18006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.525+0000 7fc19cc00640 1 -- 172.21.15.59:0/893516453 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc18c0ac470 con 0x7fc1a00b3f10 2024-04-23T14:32:13.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.525+0000 7fc1a4e00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc18006c8f0 0x7fc18006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.526+0000 7fc1a4e00640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc18006c8f0 0x7fc18006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc194009a20 tx=0x7fc194008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.529+0000 7fc19cc00640 1 -- 172.21.15.59:0/893516453 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc18c04b030 con 0x7fc1a00b3f10 2024-04-23T14:32:13.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.727+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1a00d28f0 con 0x7fc1a00b3f10 2024-04-23T14:32:13.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.729+0000 7fc19cc00640 1 -- 172.21.15.59:0/893516453 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fc18c07a790 con 0x7fc1a00b3f10 2024-04-23T14:32:13.729 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.731+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc18006c8f0 msgr2=0x7fc18006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.731+0000 7fc1a6200640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc18006c8f0 0x7fc18006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc194009a20 tx=0x7fc194008040 comp rx=0 tx=0).stop 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.731+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a00b3f10 msgr2=0x7fc1a00a12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.731+0000 7fc1a6200640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a00b3f10 0x7fc1a00a12e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 shutdown_connections 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc18006c8f0 0x7fc18006edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015ed20 0x7fc1a00c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a00b3f10 0x7fc1a00a12e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 --2- 172.21.15.59:0/893516453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00b3600 0x7fc1a0154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 >> 172.21.15.59:0/893516453 conn(0x7fc1a008bf90 msgr2=0x7fc1a00b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 shutdown_connections 2024-04-23T14:32:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.732+0000 7fc1a6200640 1 -- 172.21.15.59:0/893516453 wait complete. 2024-04-23T14:32:13.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:13.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.966+0000 7f5349e00640 1 Processor -- start 2024-04-23T14:32:13.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.966+0000 7f5349e00640 1 -- start start 2024-04-23T14:32:13.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.967+0000 7f5349e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53441567e0 0x7f5344156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.967+0000 7f5349e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f534415d810 0x7f53440d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.967+0000 7f5349e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f534415cab0 con 0x7f53440d0b40 2024-04-23T14:32:13.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.967+0000 7f5349e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f53440d0f70 con 0x7f53441567e0 2024-04-23T14:32:13.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.967+0000 7f5349e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5344160e40 con 0x7f534415d810 2024-04-23T14:32:13.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.968+0000 7f533be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f534415d810 0x7f53440d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.968+0000 7f533be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f534415d810 0x7f53440d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:39802/0 (socket says 172.21.15.59:39802) 2024-04-23T14:32:13.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.969+0000 7f533be00640 1 -- 172.21.15.59:0/4076890429 learned_addr learned my addr 172.21.15.59:0/4076890429 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:13.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.969+0000 7f5348a00640 1 --2- 172.21.15.59:0/4076890429 >> v2:172.21.15.76:3300/0 conn(0x7f53441567e0 0x7f5344156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.969+0000 7f5343e00640 1 -- 172.21.15.59:0/4076890429 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3772918476 0 0) 0x7f533c001490 con 0x7f53440d0b40 2024-04-23T14:32:13.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.969+0000 7f5343e00640 1 -- 172.21.15.59:0/4076890429 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5328003520 con 0x7f53440d0b40 2024-04-23T14:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.970+0000 7f533be00640 1 -- 172.21.15.59:0/4076890429 >> v2:172.21.15.76:3300/0 conn(0x7f53441567e0 msgr2=0x7f5344156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.970+0000 7f533be00640 1 --2- 172.21.15.59:0/4076890429 >> v2:172.21.15.76:3300/0 conn(0x7f53441567e0 0x7f5344156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.970+0000 7f533be00640 1 -- 172.21.15.59:0/4076890429 >> v1:172.21.15.76:6789/0 conn(0x7f53440d0b40 legacy=0x7f5344160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.970+0000 7f533be00640 1 -- 172.21.15.59:0/4076890429 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534415cd30 con 0x7f534415d810 2024-04-23T14:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.971+0000 7f533be00640 1 --2- 172.21.15.59:0/4076890429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f534415d810 0x7f53440d0600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f532c009870 tx=0x7f532c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427f66076a0a5dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.971+0000 7f5343e00640 1 -- 172.21.15.59:0/4076890429 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c037070 con 0x7f534415d810 2024-04-23T14:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.971+0000 7f5343e00640 1 -- 172.21.15.59:0/4076890429 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532c032440 con 0x7f534415d810 2024-04-23T14:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.971+0000 7f5343e00640 1 -- 172.21.15.59:0/4076890429 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c032810 con 0x7f534415d810 2024-04-23T14:32:13.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.972+0000 7f5349e00640 1 -- 172.21.15.59:0/4076890429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f534415d810 msgr2=0x7f53440d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.972+0000 7f5349e00640 1 --2- 172.21.15.59:0/4076890429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f534415d810 0x7f53440d0600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f532c009870 tx=0x7f532c02a270 comp rx=0 tx=0).stop 2024-04-23T14:32:13.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.972+0000 7f5349e00640 1 -- 172.21.15.59:0/4076890429 shutdown_connections 2024-04-23T14:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.972+0000 7f5349e00640 1 --2- 172.21.15.59:0/4076890429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f534415d810 0x7f53440d0600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.972+0000 7f5349e00640 1 --2- 172.21.15.59:0/4076890429 >> v2:172.21.15.76:3300/0 conn(0x7f53441567e0 0x7f5344156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.972+0000 7f5349e00640 1 -- 172.21.15.59:0/4076890429 >> 172.21.15.59:0/4076890429 conn(0x7f534406bdf0 msgr2=0x7f534406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.972+0000 7f5349e00640 1 -- 172.21.15.59:0/4076890429 shutdown_connections 2024-04-23T14:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.973+0000 7f5349e00640 1 -- 172.21.15.59:0/4076890429 wait complete. 2024-04-23T14:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.974+0000 7f5349e00640 1 Processor -- start 2024-04-23T14:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.974+0000 7f5349e00640 1 -- start start 2024-04-23T14:32:13.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.975+0000 7f5349e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53440d0b40 0x7f53441f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.975+0000 7f5349e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53441567e0 0x7f5344151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.975+0000 7f5349e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534415d810 0x7f5344152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.975+0000 7f5349e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5344160820 con 0x7f534415d810 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.975+0000 7f5349e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f534415d2f0 con 0x7f53440d0b40 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.975+0000 7f5349e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f53441609a0 con 0x7f53441567e0 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534415d810 0x7f5344152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534415d810 0x7f5344152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57020/0 (socket says 172.21.15.59:57020) 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 -- 172.21.15.59:0/147547333 learned_addr learned my addr 172.21.15.59:0/147547333 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f533be00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53441567e0 0x7f5344151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5348a00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53440d0b40 0x7f53441f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 -- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53441567e0 msgr2=0x7f5344151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53441567e0 0x7f5344151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 -- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53440d0b40 msgr2=0x7f53441f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53440d0b40 0x7f53441f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.976+0000 7f5349400640 1 -- 172.21.15.59:0/147547333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53441545e0 con 0x7f534415d810 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.977+0000 7f5349400640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534415d810 0x7f5344152030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f533c00c700 tx=0x7f533c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.977+0000 7f5341600640 1 -- 172.21.15.59:0/147547333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c003b70 con 0x7f534415d810 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.977+0000 7f5341600640 1 -- 172.21.15.59:0/147547333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533c003d30 con 0x7f534415d810 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.977+0000 7f5341600640 1 -- 172.21.15.59:0/147547333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c00aa90 con 0x7f534415d810 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.977+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53441f5300 con 0x7f534415d810 2024-04-23T14:32:13.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.977+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53441f58c0 con 0x7f534415d810 2024-04-23T14:32:13.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.978+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534404fd90 con 0x7f534415d810 2024-04-23T14:32:13.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.979+0000 7f5341600640 1 -- 172.21.15.59:0/147547333 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f533c00ac30 con 0x7f534415d810 2024-04-23T14:32:13.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.980+0000 7f5341600640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f532006c920 0x7f532006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:13.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.980+0000 7f5341600640 1 -- 172.21.15.59:0/147547333 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f533c08ba70 con 0x7f534415d810 2024-04-23T14:32:13.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.981+0000 7f5348a00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f532006c920 0x7f532006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:13.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.981+0000 7f5348a00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f532006c920 0x7f532006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5334009ad0 tx=0x7f5334008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:13.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:13.985+0000 7f5341600640 1 -- 172.21.15.59:0/147547333 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f534415cab0 con 0x7f534415d810 2024-04-23T14:32:14.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.187+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f53440b5150 con 0x7f534415d810 2024-04-23T14:32:14.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.188+0000 7f5341600640 1 -- 172.21.15.59:0/147547333 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f533c059e10 con 0x7f534415d810 2024-04-23T14:32:14.189 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:14.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f532006c920 msgr2=0x7f532006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f532006c920 0x7f532006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5334009ad0 tx=0x7f5334008040 comp rx=0 tx=0).stop 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534415d810 msgr2=0x7f5344152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534415d810 0x7f5344152030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f533c00c700 tx=0x7f533c00cbc0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 shutdown_connections 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f532006c920 0x7f532006ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53441567e0 0x7f5344151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f534415d810 0x7f5344152030 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 --2- 172.21.15.59:0/147547333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53440d0b40 0x7f53441f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 >> 172.21.15.59:0/147547333 conn(0x7f534406bdf0 msgr2=0x7f534406dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.191+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 shutdown_connections 2024-04-23T14:32:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.192+0000 7f5349e00640 1 -- 172.21.15.59:0/147547333 wait complete. 2024-04-23T14:32:14.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:14.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.424+0000 7f4fdbe00640 1 Processor -- start 2024-04-23T14:32:14.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.425+0000 7f4fdbe00640 1 -- start start 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.426+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc15e930 0x7f4fdc06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.426+0000 7f4fdbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4fdc0cee90 0x7f4fdc0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.426+0000 7f4fdbe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4fdc0b25a0 0x7f4fdc162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fdbe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4fdc0d1090 con 0x7f4fdc0cee90 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fdbe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4fdc0cd780 con 0x7f4fdc0b25a0 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fdbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc06c920 con 0x7f4fdc15e930 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fda000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4fdc0cee90 0x7f4fdc0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fda000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4fdc0cee90 0x7f4fdc0b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57030/0 (socket says 172.21.15.59:57030) 2024-04-23T14:32:14.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fda000640 1 -- 172.21.15.59:0/3704070442 learned_addr learned my addr 172.21.15.59:0/3704070442 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fdaa00640 1 --2- 172.21.15.59:0/3704070442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc15e930 0x7f4fdc06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fdb400640 1 --2- 172.21.15.59:0/3704070442 >> v2:172.21.15.76:3300/0 conn(0x7f4fdc0b25a0 0x7f4fdc162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.427+0000 7f4fda000640 1 -- 172.21.15.59:0/3704070442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc15e930 msgr2=0x7f4fdc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.428+0000 7f4fda000640 1 --2- 172.21.15.59:0/3704070442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc15e930 0x7f4fdc06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.428+0000 7f4fda000640 1 -- 172.21.15.59:0/3704070442 >> v2:172.21.15.76:3300/0 conn(0x7f4fdc0b25a0 msgr2=0x7f4fdc162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.428+0000 7f4fda000640 1 --2- 172.21.15.59:0/3704070442 >> v2:172.21.15.76:3300/0 conn(0x7f4fdc0b25a0 0x7f4fdc162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.428+0000 7f4fda000640 1 -- 172.21.15.59:0/3704070442 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc15ed10 con 0x7f4fdc0cee90 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.428+0000 7f4fdb400640 1 --2- 172.21.15.59:0/3704070442 >> v2:172.21.15.76:3300/0 conn(0x7f4fdc0b25a0 0x7f4fdc162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.429+0000 7f4fda000640 1 --2- 172.21.15.59:0/3704070442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc0cee90 0x7f4fdc0b2060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0cfb20 tx=0x7f4fc802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cba317c1525629c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.429+0000 7f4fd9600640 1 -- 172.21.15.59:0/3704070442 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8037070 con 0x7f4fdc0cee90 2024-04-23T14:32:14.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.429+0000 7f4fd9600640 1 -- 172.21.15.59:0/3704070442 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc8032440 con 0x7f4fdc0cee90 2024-04-23T14:32:14.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.429+0000 7f4fd9600640 1 -- 172.21.15.59:0/3704070442 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8032810 con 0x7f4fdc0cee90 2024-04-23T14:32:14.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 -- 172.21.15.59:0/3704070442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc0cee90 msgr2=0x7f4fdc0b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/3704070442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc0cee90 0x7f4fdc0b2060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0cfb20 tx=0x7f4fc802a210 comp rx=0 tx=0).stop 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 -- 172.21.15.59:0/3704070442 shutdown_connections 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/3704070442 >> v2:172.21.15.76:3300/0 conn(0x7f4fdc0b25a0 0x7f4fdc162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/3704070442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc0cee90 0x7f4fdc0b2060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/3704070442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc15e930 0x7f4fdc06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 -- 172.21.15.59:0/3704070442 >> 172.21.15.59:0/3704070442 conn(0x7f4fdc0d12c0 msgr2=0x7f4fdc0d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 -- 172.21.15.59:0/3704070442 shutdown_connections 2024-04-23T14:32:14.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.430+0000 7f4fdbe00640 1 -- 172.21.15.59:0/3704070442 wait complete. 2024-04-23T14:32:14.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.432+0000 7f4fdbe00640 1 Processor -- start 2024-04-23T14:32:14.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.432+0000 7f4fdbe00640 1 -- start start 2024-04-23T14:32:14.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc0b25a0 0x7f4fdc1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fdc0cee90 0x7f4fdc0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc15e930 0x7f4fdc15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc0d0db0 con 0x7f4fdc15e930 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdbe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc0d0ad0 con 0x7f4fdc0cee90 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc0cb910 con 0x7f4fdc0b25a0 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc15e930 0x7f4fdc15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc15e930 0x7f4fdc15cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57044/0 (socket says 172.21.15.59:57044) 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.433+0000 7f4fdb400640 1 -- 172.21.15.59:0/930598376 learned_addr learned my addr 172.21.15.59:0/930598376 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdaa00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc0b25a0 0x7f4fdc1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdb400640 1 -- 172.21.15.59:0/930598376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc0b25a0 msgr2=0x7f4fdc1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdb400640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc0b25a0 0x7f4fdc1f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdb400640 1 -- 172.21.15.59:0/930598376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fdc0cee90 msgr2=0x7f4fdc0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdb400640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fdc0cee90 0x7f4fdc0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdb400640 1 -- 172.21.15.59:0/930598376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc1f4ac0 con 0x7f4fdc15e930 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdb400640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc15e930 0x7f4fdc15cd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c7f0 tx=0x7f4fd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fd2a00640 1 -- 172.21.15.59:0/930598376 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4015070 con 0x7f4fdc15e930 2024-04-23T14:32:14.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fd2a00640 1 -- 172.21.15.59:0/930598376 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd40077c0 con 0x7f4fdc15e930 2024-04-23T14:32:14.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fd2a00640 1 -- 172.21.15.59:0/930598376 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4007bb0 con 0x7f4fdc15e930 2024-04-23T14:32:14.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.434+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc160710 con 0x7f4fdc15e930 2024-04-23T14:32:14.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.435+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc160cd0 con 0x7f4fdc15e930 2024-04-23T14:32:14.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.436+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc04fd90 con 0x7f4fdc15e930 2024-04-23T14:32:14.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.437+0000 7f4fd2a00640 1 -- 172.21.15.59:0/930598376 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fd4018070 con 0x7f4fdc15e930 2024-04-23T14:32:14.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.437+0000 7f4fd2a00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.438+0000 7f4fd2a00640 1 -- 172.21.15.59:0/930598376 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4fd408bc90 con 0x7f4fdc15e930 2024-04-23T14:32:14.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.438+0000 7f4fdaa00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.439+0000 7f4fdaa00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009e90 tx=0x7f4fcc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.442+0000 7f4fd2a00640 1 -- 172.21.15.59:0/930598376 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4fdc1f4ac0 con 0x7f4fdc15e930 2024-04-23T14:32:14.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.670+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4fdc0b7240 con 0x7f4fdc15e930 2024-04-23T14:32:14.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.671+0000 7f4fd2a00640 1 -- 172.21.15.59:0/930598376 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f4fd405a0f0 con 0x7f4fdc15e930 2024-04-23T14:32:14.672 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.673+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fbc06c8f0 msgr2=0x7f4fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.673+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009e90 tx=0x7f4fcc00b040 comp rx=0 tx=0).stop 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.673+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc15e930 msgr2=0x7f4fdc15cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.673+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc15e930 0x7f4fdc15cd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c7f0 tx=0x7f4fd400ccb0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 shutdown_connections 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fdc15e930 0x7f4fdc15cd60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fdc0cee90 0x7f4fdc0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 --2- 172.21.15.59:0/930598376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fdc0b25a0 0x7f4fdc1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 >> 172.21.15.59:0/930598376 conn(0x7f4fdc0d12c0 msgr2=0x7f4fdc06efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:14.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 shutdown_connections 2024-04-23T14:32:14.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.674+0000 7f4fdbe00640 1 -- 172.21.15.59:0/930598376 wait complete. 2024-04-23T14:32:14.695 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:32:14.696 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:14.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:14.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.964+0000 7fb6c5600640 1 Processor -- start 2024-04-23T14:32:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.965+0000 7fb6c5600640 1 -- start start 2024-04-23T14:32:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.966+0000 7fb6c5600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6c006f620 0x7fb6c015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.966+0000 7fb6c5600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb6c00b3080 0x7fb6c00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.966+0000 7fb6c5600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c015f430 con 0x7fb6c00cd050 2024-04-23T14:32:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.966+0000 7fb6c5600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c00ced20 con 0x7fb6c006f620 2024-04-23T14:32:14.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.966+0000 7fb6c5600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c015fcd0 con 0x7fb6c00b3080 2024-04-23T14:32:14.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6bfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb6c00cd050 0x7fb6c00cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57686/0 (socket says 172.21.15.59:57686) 2024-04-23T14:32:14.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6bfe00640 1 -- 172.21.15.59:0/2244592043 learned_addr learned my addr 172.21.15.59:0/2244592043 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:14.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6b7e00640 1 --2- 172.21.15.59:0/2244592043 >> v2:172.21.15.59:3300/0 conn(0x7fb6c006f620 0x7fb6c015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6c4c00640 1 --2- 172.21.15.59:0/2244592043 >> v2:172.21.15.76:3300/0 conn(0x7fb6c00b3080 0x7fb6c00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6bf400640 1 -- 172.21.15.59:0/2244592043 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312921985 0 0) 0x7fb6c015f430 con 0x7fb6c00cd050 2024-04-23T14:32:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6bf400640 1 -- 172.21.15.59:0/2244592043 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6a4003520 con 0x7fb6c00cd050 2024-04-23T14:32:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6b7e00640 1 -- 172.21.15.59:0/2244592043 >> v2:172.21.15.76:3300/0 conn(0x7fb6c00b3080 msgr2=0x7fb6c00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6b7e00640 1 --2- 172.21.15.59:0/2244592043 >> v2:172.21.15.76:3300/0 conn(0x7fb6c00b3080 0x7fb6c00cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6b7e00640 1 -- 172.21.15.59:0/2244592043 >> v1:172.21.15.59:6789/0 conn(0x7fb6c00cd050 legacy=0x7fb6c00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.967+0000 7fb6b7e00640 1 -- 172.21.15.59:0/2244592043 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c015f710 con 0x7fb6c006f620 2024-04-23T14:32:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.968+0000 7fb6b7e00640 1 --2- 172.21.15.59:0/2244592043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c006f620 0x7fb6c015ece0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00cc960 tx=0x7fb6a802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62ecfda9de8b351 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.969+0000 7fb6bf400640 1 -- 172.21.15.59:0/2244592043 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8037070 con 0x7fb6c006f620 2024-04-23T14:32:14.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.969+0000 7fb6bf400640 1 -- 172.21.15.59:0/2244592043 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a8032440 con 0x7fb6c006f620 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.969+0000 7fb6bf400640 1 -- 172.21.15.59:0/2244592043 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a802ae60 con 0x7fb6c006f620 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.969+0000 7fb6c5600640 1 -- 172.21.15.59:0/2244592043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c006f620 msgr2=0x7fb6c015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.969+0000 7fb6c5600640 1 --2- 172.21.15.59:0/2244592043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c006f620 0x7fb6c015ece0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00cc960 tx=0x7fb6a802a110 comp rx=0 tx=0).stop 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.970+0000 7fb6c5600640 1 -- 172.21.15.59:0/2244592043 shutdown_connections 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.970+0000 7fb6c5600640 1 --2- 172.21.15.59:0/2244592043 >> v2:172.21.15.76:3300/0 conn(0x7fb6c00b3080 0x7fb6c00cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.970+0000 7fb6c5600640 1 --2- 172.21.15.59:0/2244592043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c006f620 0x7fb6c015ece0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.970+0000 7fb6c5600640 1 -- 172.21.15.59:0/2244592043 >> 172.21.15.59:0/2244592043 conn(0x7fb6c00b21a0 msgr2=0x7fb6c00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.970+0000 7fb6c5600640 1 -- 172.21.15.59:0/2244592043 shutdown_connections 2024-04-23T14:32:14.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.970+0000 7fb6c5600640 1 -- 172.21.15.59:0/2244592043 wait complete. 2024-04-23T14:32:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.971+0000 7fb6c5600640 1 Processor -- start 2024-04-23T14:32:14.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.972+0000 7fb6c5600640 1 -- start start 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c5600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6c006f620 0x7fb6c00b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c5600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6c00b3080 0x7fb6c009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c00cd050 0x7fb6c00cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c5600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c00cef50 con 0x7fb6c00cd050 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c5600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c006faa0 con 0x7fb6c006f620 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c5600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c0159110 con 0x7fb6c00b3080 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c00cd050 0x7fb6c00cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c00cd050 0x7fb6c00cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57068/0 (socket says 172.21.15.59:57068) 2024-04-23T14:32:14.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 -- 172.21.15.59:0/12900661 learned_addr learned my addr 172.21.15.59:0/12900661 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 -- 172.21.15.59:0/12900661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6c00b3080 msgr2=0x7fb6c009da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6c00b3080 0x7fb6c009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 -- 172.21.15.59:0/12900661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6c006f620 msgr2=0x7fb6c00b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6c006f620 0x7fb6c00b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:14.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.973+0000 7fb6c4c00640 1 -- 172.21.15.59:0/12900661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c00c3fc0 con 0x7fb6c00cd050 2024-04-23T14:32:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.974+0000 7fb6c4c00640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c00cd050 0x7fb6c00cfb30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8007e40 tx=0x7fb6b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.974+0000 7fb6bcc00640 1 -- 172.21.15.59:0/12900661 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8014620 con 0x7fb6c00cd050 2024-04-23T14:32:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.974+0000 7fb6bcc00640 1 -- 172.21.15.59:0/12900661 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b8019070 con 0x7fb6c00cd050 2024-04-23T14:32:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.974+0000 7fb6bcc00640 1 -- 172.21.15.59:0/12900661 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8014a20 con 0x7fb6c00cd050 2024-04-23T14:32:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.974+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c00c42a0 con 0x7fb6c00cd050 2024-04-23T14:32:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.974+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c00c4860 con 0x7fb6c00cd050 2024-04-23T14:32:14.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.975+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c004fd90 con 0x7fb6c00cd050 2024-04-23T14:32:14.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.976+0000 7fb6bcc00640 1 -- 172.21.15.59:0/12900661 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6b8014bc0 con 0x7fb6c00cd050 2024-04-23T14:32:14.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.977+0000 7fb6bcc00640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb69c06c8f0 0x7fb69c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:14.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.977+0000 7fb6bcc00640 1 -- 172.21.15.59:0/12900661 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb6b808c5d0 con 0x7fb6c00cd050 2024-04-23T14:32:14.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.977+0000 7fb6bfe00640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb69c06c8f0 0x7fb69c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:14.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.978+0000 7fb6bfe00640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb69c06c8f0 0x7fb69c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0000f60 tx=0x7fb6b00078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:14.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:14.982+0000 7fb6bcc00640 1 -- 172.21.15.59:0/12900661 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6b805a8e0 con 0x7fb6c00cd050 2024-04-23T14:32:15.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.190+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6c00b0ef0 con 0x7fb6c00cd050 2024-04-23T14:32:15.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.193+0000 7fb6bcc00640 1 -- 172.21.15.59:0/12900661 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fb6b805a1e0 con 0x7fb6c00cd050 2024-04-23T14:32:15.195 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:15.195 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb69c06c8f0 msgr2=0x7fb69c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb69c06c8f0 0x7fb69c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0000f60 tx=0x7fb6b00078f0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c00cd050 msgr2=0x7fb6c00cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c00cd050 0x7fb6c00cfb30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8007e40 tx=0x7fb6b800e4b0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 shutdown_connections 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb69c06c8f0 0x7fb69c06edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6c00b3080 0x7fb6c009da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6c00cd050 0x7fb6c00cfb30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 --2- 172.21.15.59:0/12900661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6c006f620 0x7fb6c00b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.196+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 >> 172.21.15.59:0/12900661 conn(0x7fb6c00b21a0 msgr2=0x7fb6c00b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.197+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 shutdown_connections 2024-04-23T14:32:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.197+0000 7fb6c5600640 1 -- 172.21.15.59:0/12900661 wait complete. 2024-04-23T14:32:15.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:15.218 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.419+0000 7f1bc6400640 1 Processor -- start 2024-04-23T14:32:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.419+0000 7f1bc6400640 1 -- start start 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc6400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1bc00d0d10 0x7f1bc0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc6400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1bc015d810 0x7f1bc0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc6400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc015cab0 con 0x7f1bc01567e0 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc6400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc0160e40 con 0x7f1bc015d810 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc6400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc015cd30 con 0x7f1bc00d0d10 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc5a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1bc015d810 0x7f1bc0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc5a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1bc015d810 0x7f1bc0160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57074/0 (socket says 172.21.15.59:57074) 2024-04-23T14:32:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.421+0000 7f1bc5a00640 1 -- 172.21.15.59:0/4412298 learned_addr learned my addr 172.21.15.59:0/4412298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.422+0000 7f1bc5a00640 1 -- 172.21.15.59:0/4412298 >> v2:172.21.15.76:3300/0 conn(0x7f1bc00d0d10 msgr2=0x7f1bc0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.422+0000 7f1bc5a00640 1 --2- 172.21.15.59:0/4412298 >> v2:172.21.15.76:3300/0 conn(0x7f1bc00d0d10 0x7f1bc0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.422+0000 7f1bc5a00640 1 -- 172.21.15.59:0/4412298 >> v1:172.21.15.76:6789/0 conn(0x7f1bc01567e0 legacy=0x7f1bc00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.422+0000 7f1bc5a00640 1 -- 172.21.15.59:0/4412298 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc015d010 con 0x7f1bc015d810 2024-04-23T14:32:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.423+0000 7f1bc5a00640 1 --2- 172.21.15.59:0/4412298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc015d810 0x7f1bc0160110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0074140 tx=0x7f1bb802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1852a77d53f756 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.423+0000 7f1bbf400640 1 -- 172.21.15.59:0/4412298 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8036070 con 0x7f1bc015d810 2024-04-23T14:32:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.423+0000 7f1bbf400640 1 -- 172.21.15.59:0/4412298 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb8031600 con 0x7f1bc015d810 2024-04-23T14:32:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.423+0000 7f1bbf400640 1 -- 172.21.15.59:0/4412298 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80319d0 con 0x7f1bc015d810 2024-04-23T14:32:15.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 -- 172.21.15.59:0/4412298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc015d810 msgr2=0x7f1bc0160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 --2- 172.21.15.59:0/4412298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc015d810 0x7f1bc0160110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0074140 tx=0x7f1bb802a710 comp rx=0 tx=0).stop 2024-04-23T14:32:15.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 -- 172.21.15.59:0/4412298 shutdown_connections 2024-04-23T14:32:15.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 --2- 172.21.15.59:0/4412298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc015d810 0x7f1bc0160110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 --2- 172.21.15.59:0/4412298 >> v2:172.21.15.76:3300/0 conn(0x7f1bc00d0d10 0x7f1bc0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 -- 172.21.15.59:0/4412298 >> 172.21.15.59:0/4412298 conn(0x7f1bc006bdf0 msgr2=0x7f1bc006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 -- 172.21.15.59:0/4412298 shutdown_connections 2024-04-23T14:32:15.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.424+0000 7f1bc6400640 1 -- 172.21.15.59:0/4412298 wait complete. 2024-04-23T14:32:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.426+0000 7f1bc6400640 1 Processor -- start 2024-04-23T14:32:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.426+0000 7f1bc6400640 1 -- start start 2024-04-23T14:32:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.427+0000 7f1bc6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc00d0d10 0x7f1bc0143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.427+0000 7f1bc6400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc01567e0 0x7f1bc01f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.427+0000 7f1bc6400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc015d810 0x7f1bc0158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.427+0000 7f1bc6400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc01607d0 con 0x7f1bc00d0d10 2024-04-23T14:32:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.427+0000 7f1bc6400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc0160650 con 0x7f1bc01567e0 2024-04-23T14:32:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.427+0000 7f1bc6400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc0160950 con 0x7f1bc015d810 2024-04-23T14:32:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc00d0d10 0x7f1bc0143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc00d0d10 0x7f1bc0143dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57080/0 (socket says 172.21.15.59:57080) 2024-04-23T14:32:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 -- 172.21.15.59:0/110515107 learned_addr learned my addr 172.21.15.59:0/110515107 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5a00640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc015d810 0x7f1bc0158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc01567e0 0x7f1bc01f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 -- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc015d810 msgr2=0x7f1bc0158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc015d810 0x7f1bc0158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 -- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc01567e0 msgr2=0x7f1bc01f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc01567e0 0x7f1bc01f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 -- 172.21.15.59:0/110515107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc0153b30 con 0x7f1bc00d0d10 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc01567e0 0x7f1bc01f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.428+0000 7f1bc5000640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc00d0d10 0x7f1bc0143dd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400c410 tx=0x7f1bb400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.429+0000 7f1bbcc00640 1 -- 172.21.15.59:0/110515107 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4017070 con 0x7f1bc00d0d10 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.429+0000 7f1bbcc00640 1 -- 172.21.15.59:0/110515107 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb40035f0 con 0x7f1bc00d0d10 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.429+0000 7f1bbcc00640 1 -- 172.21.15.59:0/110515107 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4003a20 con 0x7f1bc00d0d10 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.429+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc0153e10 con 0x7f1bc00d0d10 2024-04-23T14:32:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.429+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc01543d0 con 0x7f1bc00d0d10 2024-04-23T14:32:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.430+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc004fd90 con 0x7f1bc00d0d10 2024-04-23T14:32:15.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.431+0000 7f1bbcc00640 1 -- 172.21.15.59:0/110515107 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bb401a070 con 0x7f1bc00d0d10 2024-04-23T14:32:15.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.432+0000 7f1bbcc00640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba006c940 0x7f1ba006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.432+0000 7f1bbcc00640 1 -- 172.21.15.59:0/110515107 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1bb408b200 con 0x7f1bc00d0d10 2024-04-23T14:32:15.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.432+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba006c940 0x7f1ba006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.433+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba006c940 0x7f1ba006ee00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0156540 tx=0x7f1bac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.437+0000 7f1bbcc00640 1 -- 172.21.15.59:0/110515107 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bb4059500 con 0x7f1bc00d0d10 2024-04-23T14:32:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.644+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1bc00b5150 con 0x7f1bc00d0d10 2024-04-23T14:32:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.645+0000 7f1bbcc00640 1 -- 172.21.15.59:0/110515107 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1bb4058e00 con 0x7f1bc00d0d10 2024-04-23T14:32:15.645 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.647+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba006c940 msgr2=0x7f1ba006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.647+0000 7f1bc6400640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba006c940 0x7f1ba006ee00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0156540 tx=0x7f1bac008040 comp rx=0 tx=0).stop 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.647+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc00d0d10 msgr2=0x7f1bc0143dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.647+0000 7f1bc6400640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc00d0d10 0x7f1bc0143dd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400c410 tx=0x7f1bb400c8d0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 shutdown_connections 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba006c940 0x7f1ba006ee00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc015d810 0x7f1bc0158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc01567e0 0x7f1bc01f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 --2- 172.21.15.59:0/110515107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc00d0d10 0x7f1bc0143dd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 >> 172.21.15.59:0/110515107 conn(0x7f1bc006bdf0 msgr2=0x7f1bc0071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 shutdown_connections 2024-04-23T14:32:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.648+0000 7f1bc6400640 1 -- 172.21.15.59:0/110515107 wait complete. 2024-04-23T14:32:15.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:15.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.875+0000 7fdc01000640 1 Processor -- start 2024-04-23T14:32:15.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.876+0000 7fdc01000640 1 -- start start 2024-04-23T14:32:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.877+0000 7fdc01000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc15ed20 0x7fdbfc0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.877+0000 7fdc01000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdbfc0b3600 0x7fdbfc0d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.877+0000 7fdc01000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbfc0b5da0 con 0x7fdbfc0b3f10 2024-04-23T14:32:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.877+0000 7fdc01000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdbfc15f100 con 0x7fdbfc0b3600 2024-04-23T14:32:15.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.877+0000 7fdc01000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d0a20 con 0x7fdbfc15ed20 2024-04-23T14:32:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbf2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdbfc0b3600 0x7fdbfc0d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbf2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdbfc0b3600 0x7fdbfc0d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57094/0 (socket says 172.21.15.59:57094) 2024-04-23T14:32:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbf2a00640 1 -- 172.21.15.59:0/1097201360 learned_addr learned my addr 172.21.15.59:0/1097201360 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbfb400640 1 --2- 172.21.15.59:0/1097201360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc15ed20 0x7fdbfc0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbf2a00640 1 -- 172.21.15.59:0/1097201360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc15ed20 msgr2=0x7fdbfc0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:15.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbf2a00640 1 --2- 172.21.15.59:0/1097201360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc15ed20 0x7fdbfc0d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbf2a00640 1 -- 172.21.15.59:0/1097201360 >> v1:172.21.15.76:6789/0 conn(0x7fdbfc0b3f10 legacy=0x7fdbfc0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.878+0000 7fdbf2a00640 1 -- 172.21.15.59:0/1097201360 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc0d2ed0 con 0x7fdbfc0b3600 2024-04-23T14:32:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.879+0000 7fdbf2a00640 1 --2- 172.21.15.59:0/1097201360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc0d13e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009870 tx=0x7fdbe402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf798a48aae877c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdbfaa00640 1 -- 172.21.15.59:0/1097201360 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4037070 con 0x7fdbfc0b3600 2024-04-23T14:32:15.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdbfaa00640 1 -- 172.21.15.59:0/1097201360 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe402ae10 con 0x7fdbfc0b3600 2024-04-23T14:32:15.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdbfaa00640 1 -- 172.21.15.59:0/1097201360 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4032660 con 0x7fdbfc0b3600 2024-04-23T14:32:15.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdc01000640 1 -- 172.21.15.59:0/1097201360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 msgr2=0x7fdbfc0d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdc01000640 1 --2- 172.21.15.59:0/1097201360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc0d13e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009870 tx=0x7fdbe402a100 comp rx=0 tx=0).stop 2024-04-23T14:32:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdc01000640 1 -- 172.21.15.59:0/1097201360 shutdown_connections 2024-04-23T14:32:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdc01000640 1 --2- 172.21.15.59:0/1097201360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc0d13e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdc01000640 1 --2- 172.21.15.59:0/1097201360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc15ed20 0x7fdbfc0d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdc01000640 1 -- 172.21.15.59:0/1097201360 >> 172.21.15.59:0/1097201360 conn(0x7fdbfc08bf90 msgr2=0x7fdbfc0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.880+0000 7fdc01000640 1 -- 172.21.15.59:0/1097201360 shutdown_connections 2024-04-23T14:32:15.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.881+0000 7fdc01000640 1 -- 172.21.15.59:0/1097201360 wait complete. 2024-04-23T14:32:15.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.882+0000 7fdc01000640 1 Processor -- start 2024-04-23T14:32:15.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.882+0000 7fdc01000640 1 -- start start 2024-04-23T14:32:15.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.883+0000 7fdc01000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.883+0000 7fdc01000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc0b3f10 0x7fdbfc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.883+0000 7fdc01000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc15ed20 0x7fdbfc0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.883+0000 7fdc01000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d0290 con 0x7fdbfc0b3600 2024-04-23T14:32:15.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.883+0000 7fdc01000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d3490 con 0x7fdbfc15ed20 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.883+0000 7fdc01000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d0410 con 0x7fdbfc0b3f10 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57098/0 (socket says 172.21.15.59:57098) 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 -- 172.21.15.59:0/3868796610 learned_addr learned my addr 172.21.15.59:0/3868796610 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 -- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc0b3f10 msgr2=0x7fdbfc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbf2a00640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc0b3f10 0x7fdbfc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc0b3f10 0x7fdbfc0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 -- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc15ed20 msgr2=0x7fdbfc0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc15ed20 0x7fdbfc0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc15ed20 0x7fdbfc0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 -- 172.21.15.59:0/3868796610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc0ce700 con 0x7fdbfc0b3600 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfbe00640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc15ed20 0x7fdbfc0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:15.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.884+0000 7fdbfb400640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc1529d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00ccd0 tx=0x7fdbec00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.885+0000 7fdbf3e00640 1 -- 172.21.15.59:0/3868796610 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec003990 con 0x7fdbfc0b3600 2024-04-23T14:32:15.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.885+0000 7fdbf3e00640 1 -- 172.21.15.59:0/3868796610 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec003b30 con 0x7fdbfc0b3600 2024-04-23T14:32:15.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.885+0000 7fdbf3e00640 1 -- 172.21.15.59:0/3868796610 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec004020 con 0x7fdbfc0b3600 2024-04-23T14:32:15.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.885+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbfc1f7cf0 con 0x7fdbfc0b3600 2024-04-23T14:32:15.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.885+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbfc1f82b0 con 0x7fdbfc0b3600 2024-04-23T14:32:15.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.886+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbfc04fd90 con 0x7fdbfc0b3600 2024-04-23T14:32:15.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.887+0000 7fdbf3e00640 1 -- 172.21.15.59:0/3868796610 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbec0041c0 con 0x7fdbfc0b3600 2024-04-23T14:32:15.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.888+0000 7fdbf3e00640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdbd806c8f0 0x7fdbd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:15.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.888+0000 7fdbf3e00640 1 -- 172.21.15.59:0/3868796610 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdbec08c0f0 con 0x7fdbfc0b3600 2024-04-23T14:32:15.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.888+0000 7fdbf2a00640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdbd806c8f0 0x7fdbd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:15.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.889+0000 7fdbf2a00640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdbd806c8f0 0x7fdbd806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4034000 tx=0x7fdbe402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:15.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:15.892+0000 7fdbf3e00640 1 -- 172.21.15.59:0/3868796610 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbec014030 con 0x7fdbfc0b3600 2024-04-23T14:32:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.096+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbfc0b51f0 con 0x7fdbfc0b3600 2024-04-23T14:32:16.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.097+0000 7fdbf3e00640 1 -- 172.21.15.59:0/3868796610 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fdbec05a490 con 0x7fdbfc0b3600 2024-04-23T14:32:16.098 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:16.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdbd806c8f0 msgr2=0x7fdbd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdbd806c8f0 0x7fdbd806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4034000 tx=0x7fdbe402e000 comp rx=0 tx=0).stop 2024-04-23T14:32:16.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 msgr2=0x7fdbfc1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc1529d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00ccd0 tx=0x7fdbec00a5c0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 shutdown_connections 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdbd806c8f0 0x7fdbd806edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbfc0b3f10 0x7fdbfc0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbfc15ed20 0x7fdbfc0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 --2- 172.21.15.59:0/3868796610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbfc0b3600 0x7fdbfc1529d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.099+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 >> 172.21.15.59:0/3868796610 conn(0x7fdbfc08bf90 msgr2=0x7fdbfc06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.100+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 shutdown_connections 2024-04-23T14:32:16.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.100+0000 7fdc01000640 1 -- 172.21.15.59:0/3868796610 wait complete. 2024-04-23T14:32:16.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:16.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.335+0000 7f2192a00640 1 Processor -- start 2024-04-23T14:32:16.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.335+0000 7f2192a00640 1 -- start start 2024-04-23T14:32:16.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.336+0000 7f2192a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c145e10 0x7f218c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.336+0000 7f2192a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f218c0d0cc0 0x7f218c0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.336+0000 7f2192a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c0cfc20 con 0x7f218c071d70 2024-04-23T14:32:16.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.336+0000 7f2192a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f218c0cff00 con 0x7f218c0d0cc0 2024-04-23T14:32:16.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.336+0000 7f2192a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f218c0d01e0 con 0x7f218c145e10 2024-04-23T14:32:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f218c0d0cc0 0x7f218c0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f2190a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f218c071d70 0x7f218c15cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57702/0 (socket says 172.21.15.59:57702) 2024-04-23T14:32:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f2190a00640 1 -- 172.21.15.59:0/3672876540 learned_addr learned my addr 172.21.15.59:0/3672876540 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218be00640 1 --2- 172.21.15.59:0/3672876540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c145e10 0x7f218c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218b400640 1 -- 172.21.15.59:0/3672876540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c145e10 msgr2=0x7f218c06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218aa00640 1 -- 172.21.15.59:0/3672876540 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100197325 0 0) 0x7f218c0cfc20 con 0x7f218c071d70 2024-04-23T14:32:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218b400640 1 --2- 172.21.15.59:0/3672876540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c145e10 0x7f218c06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218b400640 1 -- 172.21.15.59:0/3672876540 >> v1:172.21.15.59:6789/0 conn(0x7f218c071d70 legacy=0x7f218c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218b400640 1 -- 172.21.15.59:0/3672876540 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c0d04c0 con 0x7f218c0d0cc0 2024-04-23T14:32:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.337+0000 7f218be00640 1 --2- 172.21.15.59:0/3672876540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c145e10 0x7f218c06f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.338+0000 7f218b400640 1 --2- 172.21.15.59:0/3672876540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0cf340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f21740097d0 tx=0x7f217402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789883fb67a19d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.338+0000 7f218aa00640 1 -- 172.21.15.59:0/3672876540 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174037070 con 0x7f218c0d0cc0 2024-04-23T14:32:16.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.339+0000 7f218aa00640 1 -- 172.21.15.59:0/3672876540 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2174032440 con 0x7f218c0d0cc0 2024-04-23T14:32:16.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.339+0000 7f218aa00640 1 -- 172.21.15.59:0/3672876540 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174032810 con 0x7f218c0d0cc0 2024-04-23T14:32:16.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.339+0000 7f2192a00640 1 -- 172.21.15.59:0/3672876540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 msgr2=0x7f218c0cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.339+0000 7f2192a00640 1 --2- 172.21.15.59:0/3672876540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0cf340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f21740097d0 tx=0x7f217402a1c0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.339+0000 7f2192a00640 1 -- 172.21.15.59:0/3672876540 shutdown_connections 2024-04-23T14:32:16.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.339+0000 7f2192a00640 1 --2- 172.21.15.59:0/3672876540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0cf340 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.339+0000 7f2192a00640 1 --2- 172.21.15.59:0/3672876540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c145e10 0x7f218c06f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.340+0000 7f2192a00640 1 -- 172.21.15.59:0/3672876540 >> 172.21.15.59:0/3672876540 conn(0x7f218c067d90 msgr2=0x7f218c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.340+0000 7f2192a00640 1 -- 172.21.15.59:0/3672876540 shutdown_connections 2024-04-23T14:32:16.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.340+0000 7f2192a00640 1 -- 172.21.15.59:0/3672876540 wait complete. 2024-04-23T14:32:16.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.341+0000 7f2192a00640 1 Processor -- start 2024-04-23T14:32:16.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.341+0000 7f2192a00640 1 -- start start 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.342+0000 7f2192a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c071d70 0x7f218c140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.342+0000 7f2192a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.342+0000 7f2192a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218c145e10 0x7f218c1e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.342+0000 7f2192a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f218c0cfa00 con 0x7f218c0d0cc0 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.342+0000 7f2192a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f218c0cf880 con 0x7f218c145e10 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.342+0000 7f2192a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f218c1461f0 con 0x7f218c071d70 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57122/0 (socket says 172.21.15.59:57122) 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 -- 172.21.15.59:0/262987030 learned_addr learned my addr 172.21.15.59:0/262987030 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:16.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218be00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c071d70 0x7f218c140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 -- 172.21.15.59:0/262987030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c071d70 msgr2=0x7f218c140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c071d70 0x7f218c140320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 -- 172.21.15.59:0/262987030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218c145e10 msgr2=0x7f218c1e6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218c145e10 0x7f218c1e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.343+0000 7f218b400640 1 -- 172.21.15.59:0/262987030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c1e5d50 con 0x7f218c0d0cc0 2024-04-23T14:32:16.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.344+0000 7f218b400640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0b1020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21740097d0 tx=0x7f21740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.344+0000 7f2183e00640 1 -- 172.21.15.59:0/262987030 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174037050 con 0x7f218c0d0cc0 2024-04-23T14:32:16.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.344+0000 7f2183e00640 1 -- 172.21.15.59:0/262987030 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2174038040 con 0x7f218c0d0cc0 2024-04-23T14:32:16.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.344+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218c1e6030 con 0x7f218c0d0cc0 2024-04-23T14:32:16.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.344+0000 7f2183e00640 1 -- 172.21.15.59:0/262987030 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174004490 con 0x7f218c0d0cc0 2024-04-23T14:32:16.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.344+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218c1e65f0 con 0x7f218c0d0cc0 2024-04-23T14:32:16.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.345+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218c144690 con 0x7f218c0d0cc0 2024-04-23T14:32:16.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.346+0000 7f2183e00640 1 -- 172.21.15.59:0/262987030 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2174030980 con 0x7f218c0d0cc0 2024-04-23T14:32:16.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.347+0000 7f2183e00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f216806c920 0x7f216806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.347+0000 7f2183e00640 1 -- 172.21.15.59:0/262987030 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f21740ac800 con 0x7f218c0d0cc0 2024-04-23T14:32:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.348+0000 7f218be00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f216806c920 0x7f216806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.349+0000 7f218be00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f216806c920 0x7f216806ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f217c008f80 tx=0x7f217c0084d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.352+0000 7f2183e00640 1 -- 172.21.15.59:0/262987030 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f218c144690 con 0x7f218c0d0cc0 2024-04-23T14:32:16.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.557+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f218c143ef0 con 0x7f218c0d0cc0 2024-04-23T14:32:16.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.558+0000 7f2183e00640 1 -- 172.21.15.59:0/262987030 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f218c143ef0 con 0x7f218c0d0cc0 2024-04-23T14:32:16.559 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f216806c920 msgr2=0x7f216806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f216806c920 0x7f216806ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f217c008f80 tx=0x7f217c0084d0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 msgr2=0x7f218c0b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0b1020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21740097d0 tx=0x7f21740046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 shutdown_connections 2024-04-23T14:32:16.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f216806c920 0x7f216806ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f218c145e10 0x7f218c1e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f218c0d0cc0 0x7f218c0b1020 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 --2- 172.21.15.59:0/262987030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f218c071d70 0x7f218c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 >> 172.21.15.59:0/262987030 conn(0x7f218c067d90 msgr2=0x7f218c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 shutdown_connections 2024-04-23T14:32:16.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.561+0000 7f2192a00640 1 -- 172.21.15.59:0/262987030 wait complete. 2024-04-23T14:32:16.581 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:16.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.786+0000 7f4cfda00640 1 Processor -- start 2024-04-23T14:32:16.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.786+0000 7f4cfda00640 1 -- start start 2024-04-23T14:32:16.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.787+0000 7f4cfda00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf80d05c0 0x7f4cf81416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.787+0000 7f4cfda00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4cf815f060 0x7f4cf80d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.787+0000 7f4cfda00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf80d0ee0 con 0x7f4cf815c250 2024-04-23T14:32:16.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.788+0000 7f4cfda00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf8141c20 con 0x7f4cf815f060 2024-04-23T14:32:16.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.788+0000 7f4cfda00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf815bdc0 con 0x7f4cf80d05c0 2024-04-23T14:32:16.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.788+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf80d05c0 0x7f4cf81416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.788+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf80d05c0 0x7f4cf81416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:39894/0 (socket says 172.21.15.59:39894) 2024-04-23T14:32:16.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.788+0000 7f4cf7e00640 1 -- 172.21.15.59:0/3848636965 learned_addr learned my addr 172.21.15.59:0/3848636965 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:16.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.788+0000 7f4cf7400640 1 --2- 172.21.15.59:0/3848636965 >> v2:172.21.15.76:3300/0 conn(0x7f4cf815f060 0x7f4cf80d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.789+0000 7f4cf6a00640 1 -- 172.21.15.59:0/3848636965 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2296512819 0 0) 0x7f4cf80d0ee0 con 0x7f4cf815c250 2024-04-23T14:32:16.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.789+0000 7f4cf6a00640 1 -- 172.21.15.59:0/3848636965 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cdc005730 con 0x7f4cf815c250 2024-04-23T14:32:16.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.789+0000 7f4cf7e00640 1 -- 172.21.15.59:0/3848636965 >> v2:172.21.15.76:3300/0 conn(0x7f4cf815f060 msgr2=0x7f4cf80d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:16.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.789+0000 7f4cf7e00640 1 --2- 172.21.15.59:0/3848636965 >> v2:172.21.15.76:3300/0 conn(0x7f4cf815f060 0x7f4cf80d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.789+0000 7f4cf7e00640 1 -- 172.21.15.59:0/3848636965 >> v1:172.21.15.76:6789/0 conn(0x7f4cf815c250 legacy=0x7f4cf81555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.789+0000 7f4cf7e00640 1 -- 172.21.15.59:0/3848636965 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf815ba70 con 0x7f4cf80d05c0 2024-04-23T14:32:16.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.789+0000 7f4cf7400640 1 --2- 172.21.15.59:0/3848636965 >> v2:172.21.15.76:3300/0 conn(0x7f4cf815f060 0x7f4cf80d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:16.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.791+0000 7f4cf7e00640 1 --2- 172.21.15.59:0/3848636965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf80d05c0 0x7f4cf81416e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009870 tx=0x7f4ce802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=375ecb46836c91f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.791+0000 7f4cf6a00640 1 -- 172.21.15.59:0/3848636965 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8037070 con 0x7f4cf80d05c0 2024-04-23T14:32:16.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.791+0000 7f4cf6a00640 1 -- 172.21.15.59:0/3848636965 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce8032440 con 0x7f4cf80d05c0 2024-04-23T14:32:16.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.791+0000 7f4cf6a00640 1 -- 172.21.15.59:0/3848636965 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80327f0 con 0x7f4cf80d05c0 2024-04-23T14:32:16.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 -- 172.21.15.59:0/3848636965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf80d05c0 msgr2=0x7f4cf81416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 --2- 172.21.15.59:0/3848636965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf80d05c0 0x7f4cf81416e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009870 tx=0x7f4ce802a250 comp rx=0 tx=0).stop 2024-04-23T14:32:16.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 -- 172.21.15.59:0/3848636965 shutdown_connections 2024-04-23T14:32:16.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 --2- 172.21.15.59:0/3848636965 >> v2:172.21.15.76:3300/0 conn(0x7f4cf815f060 0x7f4cf80d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 --2- 172.21.15.59:0/3848636965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf80d05c0 0x7f4cf81416e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 -- 172.21.15.59:0/3848636965 >> 172.21.15.59:0/3848636965 conn(0x7f4cf806bdf0 msgr2=0x7f4cf806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:16.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 -- 172.21.15.59:0/3848636965 shutdown_connections 2024-04-23T14:32:16.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.792+0000 7f4cfda00640 1 -- 172.21.15.59:0/3848636965 wait complete. 2024-04-23T14:32:16.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.794+0000 7f4cfda00640 1 Processor -- start 2024-04-23T14:32:16.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.794+0000 7f4cfda00640 1 -- start start 2024-04-23T14:32:16.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cfda00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf80d05c0 0x7f4cf81ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cfda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cf815c250 0x7f4cf80cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cfda00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf815f060 0x7f4cf80cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cfda00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf815fdc0 con 0x7f4cf815c250 2024-04-23T14:32:16.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cfda00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf815fae0 con 0x7f4cf80d05c0 2024-04-23T14:32:16.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cfda00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf815f4c0 con 0x7f4cf815f060 2024-04-23T14:32:16.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cf7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cf815c250 0x7f4cf80cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cf7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cf815c250 0x7f4cf80cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57124/0 (socket says 172.21.15.59:57124) 2024-04-23T14:32:16.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.795+0000 7f4cf7400640 1 -- 172.21.15.59:0/2608324661 learned_addr learned my addr 172.21.15.59:0/2608324661 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:16.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cf7e00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf80d05c0 0x7f4cf81ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cfd000640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf815f060 0x7f4cf80cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cf7400640 1 -- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf815f060 msgr2=0x7f4cf80cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cf7400640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf815f060 0x7f4cf80cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cf7400640 1 -- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf80d05c0 msgr2=0x7f4cf81ee520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:16.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cf7400640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf80d05c0 0x7f4cf81ee520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cf7400640 1 -- 172.21.15.59:0/2608324661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf80ca720 con 0x7f4cf815c250 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cfd000640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf815f060 0x7f4cf80cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.796+0000 7f4cf7400640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cf815c250 0x7f4cf80cb4e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400e960 tx=0x7f4ce400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.797+0000 7f4cefe00640 1 -- 172.21.15.59:0/2608324661 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4017070 con 0x7f4cf815c250 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.797+0000 7f4cefe00640 1 -- 172.21.15.59:0/2608324661 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce400ca00 con 0x7f4cf815c250 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.797+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf80caa00 con 0x7f4cf815c250 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.797+0000 7f4cefe00640 1 -- 172.21.15.59:0/2608324661 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400cdd0 con 0x7f4cf815c250 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.797+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf80cafc0 con 0x7f4cf815c250 2024-04-23T14:32:16.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.798+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf804fd90 con 0x7f4cf815c250 2024-04-23T14:32:16.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.799+0000 7f4cefe00640 1 -- 172.21.15.59:0/2608324661 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ce401a070 con 0x7f4cf815c250 2024-04-23T14:32:16.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.800+0000 7f4cefe00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cd806c8f0 0x7f4cd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:16.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.800+0000 7f4cefe00640 1 -- 172.21.15.59:0/2608324661 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4ce408bdb0 con 0x7f4cf815c250 2024-04-23T14:32:16.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.801+0000 7f4cf7e00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cd806c8f0 0x7f4cd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:16.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.801+0000 7f4cf7e00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cd806c8f0 0x7f4cd806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:16.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:16.805+0000 7f4cefe00640 1 -- 172.21.15.59:0/2608324661 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ce405a1f0 con 0x7f4cf815c250 2024-04-23T14:32:17.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.020+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4cf80cd1c0 con 0x7f4cf815c250 2024-04-23T14:32:17.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.021+0000 7f4cefe00640 1 -- 172.21.15.59:0/2608324661 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f4ce4059af0 con 0x7f4cf815c250 2024-04-23T14:32:17.022 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:17.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cd806c8f0 msgr2=0x7f4cd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cd806c8f0 0x7f4cd806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).stop 2024-04-23T14:32:17.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cf815c250 msgr2=0x7f4cf80cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cf815c250 0x7f4cf80cb4e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400e960 tx=0x7f4ce400ee20 comp rx=0 tx=0).stop 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 shutdown_connections 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cd806c8f0 0x7f4cd806edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cf815f060 0x7f4cf80cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cf815c250 0x7f4cf80cb4e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 --2- 172.21.15.59:0/2608324661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf80d05c0 0x7f4cf81ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 >> 172.21.15.59:0/2608324661 conn(0x7f4cf806bdf0 msgr2=0x7f4cf815a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 shutdown_connections 2024-04-23T14:32:17.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.024+0000 7f4cfda00640 1 -- 172.21.15.59:0/2608324661 wait complete. 2024-04-23T14:32:17.043 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:32:17.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:17.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:32:17.132 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:17.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:32:17.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:17.133 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:32:17.199 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:32:17.199 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:17.256 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:32:17.256 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:17.302 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:17.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.522+0000 7fb98b200640 1 Processor -- start 2024-04-23T14:32:17.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.523+0000 7fb98b200640 1 -- start start 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.524+0000 7fb98b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb98c15c3d0 0x7fb98c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.524+0000 7fb98b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0cd590 0x7fb98c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.524+0000 7fb98b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb98c0b4690 0x7fb98c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.524+0000 7fb98b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c160210 con 0x7fb98c15c3d0 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.524+0000 7fb98b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c1604f0 con 0x7fb98c0b4690 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.524+0000 7fb98b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c1607d0 con 0x7fb98c0cd590 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb98c15c3d0 0x7fb98c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb98c15c3d0 0x7fb98c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57132/0 (socket says 172.21.15.59:57132) 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 -- 172.21.15.59:0/1871168863 learned_addr learned my addr 172.21.15.59:0/1871168863 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989400640 1 --2- 172.21.15.59:0/1871168863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0cd590 0x7fb98c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb98a800640 1 --2- 172.21.15.59:0/1871168863 >> v2:172.21.15.76:3300/0 conn(0x7fb98c0b4690 0x7fb98c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 -- 172.21.15.59:0/1871168863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0cd590 msgr2=0x7fb98c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 --2- 172.21.15.59:0/1871168863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0cd590 0x7fb98c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 -- 172.21.15.59:0/1871168863 >> v2:172.21.15.76:3300/0 conn(0x7fb98c0b4690 msgr2=0x7fb98c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 --2- 172.21.15.59:0/1871168863 >> v2:172.21.15.76:3300/0 conn(0x7fb98c0b4690 0x7fb98c06b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.525+0000 7fb989e00640 1 -- 172.21.15.59:0/1871168863 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c160ab0 con 0x7fb98c15c3d0 2024-04-23T14:32:17.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.526+0000 7fb989e00640 1 --2- 172.21.15.59:0/1871168863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15c3d0 0x7fb98c0cd050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb98c161170 tx=0x7fb97c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477f128b3139cd1d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb988a00640 1 -- 172.21.15.59:0/1871168863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c037070 con 0x7fb98c15c3d0 2024-04-23T14:32:17.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb988a00640 1 -- 172.21.15.59:0/1871168863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c032440 con 0x7fb98c15c3d0 2024-04-23T14:32:17.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb988a00640 1 -- 172.21.15.59:0/1871168863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c032810 con 0x7fb98c15c3d0 2024-04-23T14:32:17.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb98b200640 1 -- 172.21.15.59:0/1871168863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15c3d0 msgr2=0x7fb98c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb98b200640 1 --2- 172.21.15.59:0/1871168863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15c3d0 0x7fb98c0cd050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb98c161170 tx=0x7fb97c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb98b200640 1 -- 172.21.15.59:0/1871168863 shutdown_connections 2024-04-23T14:32:17.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb98b200640 1 --2- 172.21.15.59:0/1871168863 >> v2:172.21.15.76:3300/0 conn(0x7fb98c0b4690 0x7fb98c06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb98b200640 1 --2- 172.21.15.59:0/1871168863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0cd590 0x7fb98c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb98b200640 1 --2- 172.21.15.59:0/1871168863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c15c3d0 0x7fb98c0cd050 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.527+0000 7fb98b200640 1 -- 172.21.15.59:0/1871168863 >> 172.21.15.59:0/1871168863 conn(0x7fb98c0b21a0 msgr2=0x7fb98c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.528+0000 7fb98b200640 1 -- 172.21.15.59:0/1871168863 shutdown_connections 2024-04-23T14:32:17.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.528+0000 7fb98b200640 1 -- 172.21.15.59:0/1871168863 wait complete. 2024-04-23T14:32:17.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.529+0000 7fb98b200640 1 Processor -- start 2024-04-23T14:32:17.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.530+0000 7fb98b200640 1 -- start start 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.530+0000 7fb98b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0b4690 0x7fb98c15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb98b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0cd590 0x7fb98c1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb98b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c15c3d0 0x7fb98c15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb98b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c15c7b0 con 0x7fb98c0cd590 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb98b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0b4b00 con 0x7fb98c15c3d0 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb98b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c15c930 con 0x7fb98c0b4690 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0cd590 0x7fb98c1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0cd590 0x7fb98c1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57144/0 (socket says 172.21.15.59:57144) 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 -- 172.21.15.59:0/3657205586 learned_addr learned my addr 172.21.15.59:0/3657205586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:17.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989e00640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0b4690 0x7fb98c15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb98a800640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c15c3d0 0x7fb98c15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 -- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0b4690 msgr2=0x7fb98c15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0b4690 0x7fb98c15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 -- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c15c3d0 msgr2=0x7fb98c15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c15c3d0 0x7fb98c15aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.531+0000 7fb989400640 1 -- 172.21.15.59:0/3657205586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c15d4a0 con 0x7fb98c0cd590 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.532+0000 7fb989400640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0cd590 0x7fb98c1f4c60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d800 tx=0x7fb97800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.532+0000 7fb982000640 1 -- 172.21.15.59:0/3657205586 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97800b6c0 con 0x7fb98c0cd590 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.532+0000 7fb982000640 1 -- 172.21.15.59:0/3657205586 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb978019070 con 0x7fb98c0cd590 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.532+0000 7fb982000640 1 -- 172.21.15.59:0/3657205586 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97800ba80 con 0x7fb98c0cd590 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.532+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c15d780 con 0x7fb98c0cd590 2024-04-23T14:32:17.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.532+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c15dd40 con 0x7fb98c0cd590 2024-04-23T14:32:17.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.533+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c04fd90 con 0x7fb98c0cd590 2024-04-23T14:32:17.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.534+0000 7fb982000640 1 -- 172.21.15.59:0/3657205586 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb97800bc20 con 0x7fb98c0cd590 2024-04-23T14:32:17.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.535+0000 7fb982000640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96c06c8f0 0x7fb96c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.535+0000 7fb982000640 1 -- 172.21.15.59:0/3657205586 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb97808bfe0 con 0x7fb98c0cd590 2024-04-23T14:32:17.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.536+0000 7fb989e00640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96c06c8f0 0x7fb96c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.537+0000 7fb989e00640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96c06c8f0 0x7fb96c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb97c034000 tx=0x7fb97c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.540+0000 7fb982000640 1 -- 172.21.15.59:0/3657205586 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb97805a380 con 0x7fb98c0cd590 2024-04-23T14:32:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.749+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb98c06e400 con 0x7fb98c0cd590 2024-04-23T14:32:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.752+0000 7fb982000640 1 -- 172.21.15.59:0/3657205586 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fb978059c80 con 0x7fb98c0cd590 2024-04-23T14:32:17.753 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:17.754 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96c06c8f0 msgr2=0x7fb96c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96c06c8f0 0x7fb96c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb97c034000 tx=0x7fb97c02e000 comp rx=0 tx=0).stop 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0cd590 msgr2=0x7fb98c1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0cd590 0x7fb98c1f4c60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d800 tx=0x7fb97800dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 shutdown_connections 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96c06c8f0 0x7fb96c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c15c3d0 0x7fb98c15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c0cd590 0x7fb98c1f4c60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 --2- 172.21.15.59:0/3657205586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb98c0b4690 0x7fb98c15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.755+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 >> 172.21.15.59:0/3657205586 conn(0x7fb98c0b21a0 msgr2=0x7fb98c0b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.756+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 shutdown_connections 2024-04-23T14:32:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.756+0000 7fb98b200640 1 -- 172.21.15.59:0/3657205586 wait complete. 2024-04-23T14:32:17.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:17.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:17.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.991+0000 7fb593e00640 1 Processor -- start 2024-04-23T14:32:17.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.991+0000 7fb593e00640 1 -- start start 2024-04-23T14:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.993+0000 7fb593e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 0x7fb594156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.993+0000 7fb593e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59415cab0 con 0x7fb5941567e0 2024-04-23T14:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.993+0000 7fb593e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb594160e40 con 0x7fb59415d810 2024-04-23T14:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.993+0000 7fb593e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb59415cd30 con 0x7fb5940d0d10 2024-04-23T14:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.993+0000 7fb592a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb5941567e0 0x7fb5940d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57718/0 (socket says 172.21.15.59:57718) 2024-04-23T14:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.993+0000 7fb592a00640 1 -- 172.21.15.59:0/2573780865 learned_addr learned my addr 172.21.15.59:0/2573780865 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb592000640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 0x7fb594156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139115016 0 0) 0x7fb584001490 con 0x7fb5941567e0 2024-04-23T14:32:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb578003520 con 0x7fb5941567e0 2024-04-23T14:32:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3894081603 0 0) 0x7fb578003520 con 0x7fb5941567e0 2024-04-23T14:32:17.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 msgr2=0x7fb594156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 0x7fb594156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 >> v1:172.21.15.76:6789/0 conn(0x7fb59415d810 legacy=0x7fb594160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59415d010 con 0x7fb5941567e0 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fb584001bc0 con 0x7fb5941567e0 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 >> v1:172.21.15.59:6789/0 conn(0x7fb5941567e0 legacy=0x7fb5940d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5780039e0 0x7fb578003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 0x7fb578004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.994+0000 7fb591600640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb578004d40 0x7fb5780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb59415cd30 con 0x7fb578004d40 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb57800baa0 con 0x7fb5780039e0 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb57800bca0 con 0x7fb5940d0d10 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb593400640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb578004d40 0x7fb5780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb592a00640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5780039e0 0x7fb578003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb592000640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 0x7fb578004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:17.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb593400640 1 -- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 msgr2=0x7fb578004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb593400640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940d0d10 0x7fb578004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb593400640 1 -- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5780039e0 msgr2=0x7fb578003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:17.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb593400640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5780039e0 0x7fb578003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:17.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.995+0000 7fb593400640 1 -- 172.21.15.59:0/2573780865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb578003520 con 0x7fb578004d40 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.996+0000 7fb593400640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb578004d40 0x7fb5780051c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb59415dc20 tx=0x7fb578005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f30fa1fb42825e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c001370 con 0x7fb578004d40 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c0052d0 con 0x7fb578004d40 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb591600640 1 -- 172.21.15.59:0/2573780865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c02ab50 con 0x7fb578004d40 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb593e00640 1 -- 172.21.15.59:0/2573780865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb578004d40 msgr2=0x7fb5780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb593e00640 1 --2- 172.21.15.59:0/2573780865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb578004d40 0x7fb5780051c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb59415dc20 tx=0x7fb578005830 comp rx=0 tx=0).stop 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb593400640 1 -- 172.21.15.59:0/2573780865 reap_dead start 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb593e00640 1 -- 172.21.15.59:0/2573780865 shutdown_connections 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.997+0000 7fb593e00640 1 -- 172.21.15.59:0/2573780865 >> 172.21.15.59:0/2573780865 conn(0x7fb59406bdf0 msgr2=0x7fb59406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:17.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.998+0000 7fb593e00640 1 -- 172.21.15.59:0/2573780865 shutdown_connections 2024-04-23T14:32:18.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.998+0000 7fb593e00640 1 -- 172.21.15.59:0/2573780865 wait complete. 2024-04-23T14:32:18.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:17.999+0000 7fb593e00640 1 Processor -- start 2024-04-23T14:32:18.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.000+0000 7fb593e00640 1 -- start start 2024-04-23T14:32:18.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.000+0000 7fb593e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940999e0 0x7fb5940b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.000+0000 7fb593e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5941e6ba0 0x7fb5941f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.000+0000 7fb593e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5940b18b0 0x7fb5940b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb594156d40 con 0x7fb5940b18b0 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb594156bc0 con 0x7fb5941e6ba0 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb594156ec0 con 0x7fb5940999e0 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5940b18b0 0x7fb5940b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5940b18b0 0x7fb5940b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57160/0 (socket says 172.21.15.59:57160) 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 -- 172.21.15.59:0/384961814 learned_addr learned my addr 172.21.15.59:0/384961814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb592a00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940999e0 0x7fb5940b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb592000640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5941e6ba0 0x7fb5941f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 -- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940999e0 msgr2=0x7fb5940b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940999e0 0x7fb5940b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 -- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5941e6ba0 msgr2=0x7fb5941f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5941e6ba0 0x7fb5941f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.001+0000 7fb593400640 1 -- 172.21.15.59:0/384961814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5941f5310 con 0x7fb5940b18b0 2024-04-23T14:32:18.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.002+0000 7fb593400640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5940b18b0 0x7fb5940b1d30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000fb0 tx=0x7fb58c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.002+0000 7fb58aa00640 1 -- 172.21.15.59:0/384961814 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c007a30 con 0x7fb5940b18b0 2024-04-23T14:32:18.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.002+0000 7fb58aa00640 1 -- 172.21.15.59:0/384961814 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c040070 con 0x7fb5940b18b0 2024-04-23T14:32:18.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.002+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5941e7280 con 0x7fb5940b18b0 2024-04-23T14:32:18.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.002+0000 7fb58aa00640 1 -- 172.21.15.59:0/384961814 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c005480 con 0x7fb5940b18b0 2024-04-23T14:32:18.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.002+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5941f55b0 con 0x7fb5940b18b0 2024-04-23T14:32:18.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.003+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5940b4030 con 0x7fb5940b18b0 2024-04-23T14:32:18.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.004+0000 7fb58aa00640 1 -- 172.21.15.59:0/384961814 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb58c03a070 con 0x7fb5940b18b0 2024-04-23T14:32:18.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.006+0000 7fb58aa00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb57406c8f0 0x7fb57406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.006+0000 7fb58aa00640 1 -- 172.21.15.59:0/384961814 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb58c0ab0f0 con 0x7fb5940b18b0 2024-04-23T14:32:18.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.006+0000 7fb592a00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb57406c8f0 0x7fb57406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.007+0000 7fb592a00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb57406c8f0 0x7fb57406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5840013d0 tx=0x7fb584009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.010+0000 7fb58aa00640 1 -- 172.21.15.59:0/384961814 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5940b4030 con 0x7fb5940b18b0 2024-04-23T14:32:18.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.214+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb594156600 con 0x7fb5940b18b0 2024-04-23T14:32:18.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.215+0000 7fb58aa00640 1 -- 172.21.15.59:0/384961814 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fb594156600 con 0x7fb5940b18b0 2024-04-23T14:32:18.216 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.217+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb57406c8f0 msgr2=0x7fb57406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.217+0000 7fb593e00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb57406c8f0 0x7fb57406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5840013d0 tx=0x7fb584009040 comp rx=0 tx=0).stop 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.217+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5940b18b0 msgr2=0x7fb5940b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.217+0000 7fb593e00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5940b18b0 0x7fb5940b1d30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000fb0 tx=0x7fb58c001120 comp rx=0 tx=0).stop 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 shutdown_connections 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb57406c8f0 0x7fb57406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5940b18b0 0x7fb5940b1d30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5941e6ba0 0x7fb5941f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 --2- 172.21.15.59:0/384961814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5940999e0 0x7fb5940b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 >> 172.21.15.59:0/384961814 conn(0x7fb59406bdf0 msgr2=0x7fb59415dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:18.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 shutdown_connections 2024-04-23T14:32:18.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.218+0000 7fb593e00640 1 -- 172.21.15.59:0/384961814 wait complete. 2024-04-23T14:32:18.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:18.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.454+0000 7fa1ddc00640 1 Processor -- start 2024-04-23T14:32:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.455+0000 7fa1ddc00640 1 -- start start 2024-04-23T14:32:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.456+0000 7fa1ddc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1d815eb60 0x7fa1d80d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.456+0000 7fa1ddc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80d2d60 0x7fa1d80b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.456+0000 7fa1ddc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d80b87f0 con 0x7fa1d80b3490 2024-04-23T14:32:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.456+0000 7fa1ddc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d80ba7a0 con 0x7fa1d815eb60 2024-04-23T14:32:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.456+0000 7fa1ddc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d80ba1e0 con 0x7fa1d80d2d60 2024-04-23T14:32:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.456+0000 7fa1d7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa1d80b3490 0x7fa1d80b6280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57730/0 (socket says 172.21.15.59:57730) 2024-04-23T14:32:18.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.456+0000 7fa1d7400640 1 -- 172.21.15.59:0/3184979680 learned_addr learned my addr 172.21.15.59:0/3184979680 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/3184979680 >> v2:172.21.15.76:3300/0 conn(0x7fa1d815eb60 0x7fa1d80d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1cfe00640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80d2d60 0x7fa1d80b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145001985 0 0) 0x7fa1d80b87f0 con 0x7fa1d80b3490 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1bc003520 con 0x7fa1d80b3490 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2842414535 0 0) 0x7fa1bc003520 con 0x7fa1d80b3490 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80d2d60 msgr2=0x7fa1d80b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6000640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80d2d60 0x7fa1d80b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 >> v2:172.21.15.76:3300/0 conn(0x7fa1d815eb60 msgr2=0x7fa1d80d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.457+0000 7fa1d6000640 1 --2- 172.21.15.59:0/3184979680 >> v2:172.21.15.76:3300/0 conn(0x7fa1d815eb60 0x7fa1d80d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d80ba4c0 con 0x7fa1d80b3490 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa1d00019f0 con 0x7fa1d80b3490 2024-04-23T14:32:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 >> v1:172.21.15.59:6789/0 conn(0x7fa1d80b3490 legacy=0x7fa1d80b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1bc004800 0x7fa1bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1bc0051c0 0x7fa1bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d80ba7a0 con 0x7fa1bc0039b0 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d80ba1e0 con 0x7fa1bc004800 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa1bc0045a0 con 0x7fa1bc0051c0 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.458+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1cfe00640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1bc004800 0x7fa1bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1d7400640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1bc0051c0 0x7fa1bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1d6a00640 1 -- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1bc0051c0 msgr2=0x7fa1bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1bc0051c0 0x7fa1bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1d6a00640 1 -- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1bc004800 msgr2=0x7fa1bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1bc004800 0x7fa1bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1d6a00640 1 -- 172.21.15.59:0/3184979680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc003520 con 0x7fa1bc0039b0 2024-04-23T14:32:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.459+0000 7fa1d7400640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1bc0051c0 0x7fa1bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.460+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1bc003dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009af0 tx=0x7fa1bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae1745883b74515 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.460+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8036070 con 0x7fa1bc0039b0 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.460+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c8031400 con 0x7fa1bc0039b0 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.460+0000 7fa1d6000640 1 -- 172.21.15.59:0/3184979680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8009690 con 0x7fa1bc0039b0 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.461+0000 7fa1ddc00640 1 -- 172.21.15.59:0/3184979680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 msgr2=0x7fa1bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.461+0000 7fa1ddc00640 1 --2- 172.21.15.59:0/3184979680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1bc003dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009af0 tx=0x7fa1bc005830 comp rx=0 tx=0).stop 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.461+0000 7fa1d7400640 1 -- 172.21.15.59:0/3184979680 reap_dead start 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.461+0000 7fa1ddc00640 1 -- 172.21.15.59:0/3184979680 shutdown_connections 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.461+0000 7fa1ddc00640 1 -- 172.21.15.59:0/3184979680 >> 172.21.15.59:0/3184979680 conn(0x7fa1d8067b50 msgr2=0x7fa1d8067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.461+0000 7fa1ddc00640 1 -- 172.21.15.59:0/3184979680 shutdown_connections 2024-04-23T14:32:18.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.461+0000 7fa1ddc00640 1 -- 172.21.15.59:0/3184979680 wait complete. 2024-04-23T14:32:18.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.463+0000 7fa1ddc00640 1 Processor -- start 2024-04-23T14:32:18.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.463+0000 7fa1ddc00640 1 -- start start 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1ddc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1d8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1ddc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d80c8330 0x7fa1d815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1ddc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80c87b0 0x7fa1d8160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1ddc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d80b3c80 con 0x7fa1bc0039b0 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1ddc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d80b8d70 con 0x7fa1d80c8330 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1ddc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d80b6990 con 0x7fa1d80c87b0 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1d8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1d8158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57166/0 (socket says 172.21.15.59:57166) 2024-04-23T14:32:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 -- 172.21.15.59:0/528691689 learned_addr learned my addr 172.21.15.59:0/528691689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 -- 172.21.15.59:0/528691689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80c87b0 msgr2=0x7fa1d8160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80c87b0 0x7fa1d8160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 -- 172.21.15.59:0/528691689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d80c8330 msgr2=0x7fa1d815fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d80c8330 0x7fa1d815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.464+0000 7fa1d6a00640 1 -- 172.21.15.59:0/528691689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d8159400 con 0x7fa1bc0039b0 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.465+0000 7fa1d6a00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1d8158ec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1c802ac10 tx=0x7fa1c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.465+0000 7fa1cea00640 1 -- 172.21.15.59:0/528691689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8041020 con 0x7fa1bc0039b0 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.465+0000 7fa1cea00640 1 -- 172.21.15.59:0/528691689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c802ade0 con 0x7fa1bc0039b0 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.465+0000 7fa1cea00640 1 -- 172.21.15.59:0/528691689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8036040 con 0x7fa1bc0039b0 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.465+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d80cd000 con 0x7fa1bc0039b0 2024-04-23T14:32:18.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.465+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d80cd5c0 con 0x7fa1bc0039b0 2024-04-23T14:32:18.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.466+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d8068a80 con 0x7fa1bc0039b0 2024-04-23T14:32:18.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.467+0000 7fa1cea00640 1 -- 172.21.15.59:0/528691689 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1c8044020 con 0x7fa1bc0039b0 2024-04-23T14:32:18.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.469+0000 7fa1cea00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa1b406c8f0 0x7fa1b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.469+0000 7fa1cea00640 1 -- 172.21.15.59:0/528691689 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa1c80ac180 con 0x7fa1bc0039b0 2024-04-23T14:32:18.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.469+0000 7fa1cfe00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa1b406c8f0 0x7fa1b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.470+0000 7fa1cfe00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa1b406c8f0 0x7fa1b406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1d81f7670 tx=0x7fa1c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.473+0000 7fa1cea00640 1 -- 172.21.15.59:0/528691689 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1c804b030 con 0x7fa1bc0039b0 2024-04-23T14:32:18.567 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:18.574 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.684+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1d80cfcc0 con 0x7fa1bc0039b0 2024-04-23T14:32:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.685+0000 7fa1cea00640 1 -- 172.21.15.59:0/528691689 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fa1c807a490 con 0x7fa1bc0039b0 2024-04-23T14:32:18.686 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa1b406c8f0 msgr2=0x7fa1b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa1b406c8f0 0x7fa1b406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1d81f7670 tx=0x7fa1c0008040 comp rx=0 tx=0).stop 2024-04-23T14:32:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 msgr2=0x7fa1d8158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1d8158ec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1c802ac10 tx=0x7fa1c8004730 comp rx=0 tx=0).stop 2024-04-23T14:32:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 shutdown_connections 2024-04-23T14:32:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa1b406c8f0 0x7fa1b406edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1d80c87b0 0x7fa1d8160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d80c8330 0x7fa1d815fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 --2- 172.21.15.59:0/528691689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1bc0039b0 0x7fa1d8158ec0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 >> 172.21.15.59:0/528691689 conn(0x7fa1d8067b50 msgr2=0x7fa1d80d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 shutdown_connections 2024-04-23T14:32:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.688+0000 7fa1ddc00640 1 -- 172.21.15.59:0/528691689 wait complete. 2024-04-23T14:32:18.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:18.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.969+0000 7fbfe6600640 1 Processor -- start 2024-04-23T14:32:18.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.969+0000 7fbfe6600640 1 -- start start 2024-04-23T14:32:18.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfe6600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbfe015ed20 0x7fbfe00d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfe6600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfe00b3600 0x7fbfe00d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfe6600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfe00b5da0 con 0x7fbfe00b3f10 2024-04-23T14:32:18.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfe6600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe015f100 con 0x7fbfe00b3600 2024-04-23T14:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfe6600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe00d0a20 con 0x7fbfe015ed20 2024-04-23T14:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfdfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfe00b3600 0x7fbfe00d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfdfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfe00b3600 0x7fbfe00d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57170/0 (socket says 172.21.15.59:57170) 2024-04-23T14:32:18.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.971+0000 7fbfdfe00640 1 -- 172.21.15.59:0/3355382269 learned_addr learned my addr 172.21.15.59:0/3355382269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.972+0000 7fbfe5200640 1 --2- 172.21.15.59:0/3355382269 >> v2:172.21.15.76:3300/0 conn(0x7fbfe015ed20 0x7fbfe00d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.972+0000 7fbfdfe00640 1 -- 172.21.15.59:0/3355382269 >> v2:172.21.15.76:3300/0 conn(0x7fbfe015ed20 msgr2=0x7fbfe00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.972+0000 7fbfdfe00640 1 --2- 172.21.15.59:0/3355382269 >> v2:172.21.15.76:3300/0 conn(0x7fbfe015ed20 0x7fbfe00d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.972+0000 7fbfdfe00640 1 -- 172.21.15.59:0/3355382269 >> v1:172.21.15.76:6789/0 conn(0x7fbfe00b3f10 legacy=0x7fbfe00b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.972+0000 7fbfdfe00640 1 -- 172.21.15.59:0/3355382269 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe00d2ed0 con 0x7fbfe00b3600 2024-04-23T14:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.972+0000 7fbfe5200640 1 --2- 172.21.15.59:0/3355382269 >> v2:172.21.15.76:3300/0 conn(0x7fbfe015ed20 0x7fbfe00d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.973+0000 7fbfdfe00640 1 --2- 172.21.15.59:0/3355382269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe00b3600 0x7fbfe00d13e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbfe008e2a0 tx=0x7fbfcc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895e58c1fcb34687 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.973+0000 7fbfdf400640 1 -- 172.21.15.59:0/3355382269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc037070 con 0x7fbfe00b3600 2024-04-23T14:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.973+0000 7fbfdf400640 1 -- 172.21.15.59:0/3355382269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfcc02ae20 con 0x7fbfe00b3600 2024-04-23T14:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.973+0000 7fbfdf400640 1 -- 172.21.15.59:0/3355382269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc032660 con 0x7fbfe00b3600 2024-04-23T14:32:18.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 -- 172.21.15.59:0/3355382269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe00b3600 msgr2=0x7fbfe00d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 --2- 172.21.15.59:0/3355382269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe00b3600 0x7fbfe00d13e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbfe008e2a0 tx=0x7fbfcc02a110 comp rx=0 tx=0).stop 2024-04-23T14:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 -- 172.21.15.59:0/3355382269 shutdown_connections 2024-04-23T14:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 --2- 172.21.15.59:0/3355382269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe00b3600 0x7fbfe00d13e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 --2- 172.21.15.59:0/3355382269 >> v2:172.21.15.76:3300/0 conn(0x7fbfe015ed20 0x7fbfe00d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 -- 172.21.15.59:0/3355382269 >> 172.21.15.59:0/3355382269 conn(0x7fbfe008bf90 msgr2=0x7fbfe00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 -- 172.21.15.59:0/3355382269 shutdown_connections 2024-04-23T14:32:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.974+0000 7fbfe6600640 1 -- 172.21.15.59:0/3355382269 wait complete. 2024-04-23T14:32:18.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.976+0000 7fbfe6600640 1 Processor -- start 2024-04-23T14:32:18.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.976+0000 7fbfe6600640 1 -- start start 2024-04-23T14:32:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.977+0000 7fbfe6600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe00b3600 0x7fbfe01f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.977+0000 7fbfe6600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfe00b3f10 0x7fbfe00b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.977+0000 7fbfe6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe015ed20 0x7fbfe00cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.977+0000 7fbfe6600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe00d0290 con 0x7fbfe015ed20 2024-04-23T14:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.977+0000 7fbfe6600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe00d3490 con 0x7fbfe00b3600 2024-04-23T14:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.977+0000 7fbfe6600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe00d0410 con 0x7fbfe00b3f10 2024-04-23T14:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe015ed20 0x7fbfe00cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe015ed20 0x7fbfe00cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57182/0 (socket says 172.21.15.59:57182) 2024-04-23T14:32:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 -- 172.21.15.59:0/2027301899 learned_addr learned my addr 172.21.15.59:0/2027301899 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5200640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe00b3600 0x7fbfe01f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 -- 172.21.15.59:0/2027301899 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfe00b3f10 msgr2=0x7fbfe00b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfe00b3f10 0x7fbfe00b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 -- 172.21.15.59:0/2027301899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe00b3600 msgr2=0x7fbfe01f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe00b3600 0x7fbfe01f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 -- 172.21.15.59:0/2027301899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe00b8cd0 con 0x7fbfe015ed20 2024-04-23T14:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.978+0000 7fbfe5c00640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe015ed20 0x7fbfe00cd6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c780 tx=0x7fbfd800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.979+0000 7fbfdcc00640 1 -- 172.21.15.59:0/2027301899 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8017070 con 0x7fbfe015ed20 2024-04-23T14:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.979+0000 7fbfdcc00640 1 -- 172.21.15.59:0/2027301899 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd8003da0 con 0x7fbfe015ed20 2024-04-23T14:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.979+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe00b8fb0 con 0x7fbfe015ed20 2024-04-23T14:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.979+0000 7fbfdcc00640 1 -- 172.21.15.59:0/2027301899 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80052b0 con 0x7fbfe015ed20 2024-04-23T14:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.979+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe00b9570 con 0x7fbfe015ed20 2024-04-23T14:32:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.980+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe004fd90 con 0x7fbfe015ed20 2024-04-23T14:32:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.981+0000 7fbfdcc00640 1 -- 172.21.15.59:0/2027301899 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfd801a070 con 0x7fbfe015ed20 2024-04-23T14:32:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.982+0000 7fbfdcc00640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfc006c8f0 0x7fbfc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.982+0000 7fbfdcc00640 1 -- 172.21.15.59:0/2027301899 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbfd808b210 con 0x7fbfe015ed20 2024-04-23T14:32:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.982+0000 7fbfe5200640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfc006c8f0 0x7fbfc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.983+0000 7fbfe5200640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfc006c8f0 0x7fbfc006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400b840 tx=0x7fbfd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:18.987+0000 7fbfdcc00640 1 -- 172.21.15.59:0/2027301899 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfd80595c0 con 0x7fbfe015ed20 2024-04-23T14:32:19.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.187+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbfe015c1d0 con 0x7fbfe015ed20 2024-04-23T14:32:19.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.189+0000 7fbfdcc00640 1 -- 172.21.15.59:0/2027301899 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fbfd8058ec0 con 0x7fbfe015ed20 2024-04-23T14:32:19.190 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:19.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfc006c8f0 msgr2=0x7fbfc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfc006c8f0 0x7fbfc006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400b840 tx=0x7fbfd4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:19.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe015ed20 msgr2=0x7fbfe00cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe015ed20 0x7fbfe00cd6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c780 tx=0x7fbfd800cc40 comp rx=0 tx=0).stop 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 shutdown_connections 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfc006c8f0 0x7fbfc006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfe00b3f10 0x7fbfe00b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfe015ed20 0x7fbfe00cd6a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 --2- 172.21.15.59:0/2027301899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe00b3600 0x7fbfe01f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.191+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 >> 172.21.15.59:0/2027301899 conn(0x7fbfe008bf90 msgr2=0x7fbfe00b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.192+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 shutdown_connections 2024-04-23T14:32:19.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.192+0000 7fbfe6600640 1 -- 172.21.15.59:0/2027301899 wait complete. 2024-04-23T14:32:19.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:19.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.412+0000 7f2b3f000640 1 Processor -- start 2024-04-23T14:32:19.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.412+0000 7f2b3f000640 1 -- start start 2024-04-23T14:32:19.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.413+0000 7f2b3f000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2b380d0f80 0x7f2b380cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.413+0000 7f2b3f000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b3815f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.413+0000 7f2b3f000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b380d1360 0x7f2b38162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.413+0000 7f2b3f000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2b3815fdc0 con 0x7f2b380d1360 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3f000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2b380cf930 con 0x7f2b380d0f80 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3f000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b380cfc10 con 0x7f2b3815ea40 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b380d1360 0x7f2b38162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b380d1360 0x7f2b38162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57194/0 (socket says 172.21.15.59:57194) 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 -- 172.21.15.59:0/3230983345 learned_addr learned my addr 172.21.15.59:0/3230983345 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3dc00640 1 --2- 172.21.15.59:0/3230983345 >> v2:172.21.15.76:3300/0 conn(0x7f2b380d0f80 0x7f2b380cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3d200640 1 --2- 172.21.15.59:0/3230983345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b3815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 -- 172.21.15.59:0/3230983345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 msgr2=0x7f2b3815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 --2- 172.21.15.59:0/3230983345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b3815f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 -- 172.21.15.59:0/3230983345 >> v2:172.21.15.76:3300/0 conn(0x7f2b380d0f80 msgr2=0x7f2b380cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 --2- 172.21.15.59:0/3230983345 >> v2:172.21.15.76:3300/0 conn(0x7f2b380d0f80 0x7f2b380cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3e600640 1 -- 172.21.15.59:0/3230983345 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b380cfef0 con 0x7f2b380d1360 2024-04-23T14:32:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.414+0000 7f2b3d200640 1 --2- 172.21.15.59:0/3230983345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b3815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.416+0000 7f2b3e600640 1 --2- 172.21.15.59:0/3230983345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b38162370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b380d1e90 tx=0x7f2b3002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd57f47f5dafed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.416+0000 7f2b37e00640 1 -- 172.21.15.59:0/3230983345 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30036070 con 0x7f2b380d1360 2024-04-23T14:32:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.416+0000 7f2b37e00640 1 -- 172.21.15.59:0/3230983345 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b30031600 con 0x7f2b380d1360 2024-04-23T14:32:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.416+0000 7f2b37e00640 1 -- 172.21.15.59:0/3230983345 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b300319d0 con 0x7f2b380d1360 2024-04-23T14:32:19.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 -- 172.21.15.59:0/3230983345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 msgr2=0x7f2b38162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 --2- 172.21.15.59:0/3230983345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b38162370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b380d1e90 tx=0x7f2b3002a610 comp rx=0 tx=0).stop 2024-04-23T14:32:19.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 -- 172.21.15.59:0/3230983345 shutdown_connections 2024-04-23T14:32:19.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 --2- 172.21.15.59:0/3230983345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b38162370 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 --2- 172.21.15.59:0/3230983345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b3815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 --2- 172.21.15.59:0/3230983345 >> v2:172.21.15.76:3300/0 conn(0x7f2b380d0f80 0x7f2b380cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 -- 172.21.15.59:0/3230983345 >> 172.21.15.59:0/3230983345 conn(0x7f2b3808bf90 msgr2=0x7f2b380d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:19.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 -- 172.21.15.59:0/3230983345 shutdown_connections 2024-04-23T14:32:19.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.417+0000 7f2b3f000640 1 -- 172.21.15.59:0/3230983345 wait complete. 2024-04-23T14:32:19.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.419+0000 7f2b3f000640 1 Processor -- start 2024-04-23T14:32:19.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.419+0000 7f2b3f000640 1 -- start start 2024-04-23T14:32:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3f000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b380d0f80 0x7f2b3815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b3815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3f000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b380b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b380d17e0 con 0x7f2b380d1360 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3f000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b380d04b0 con 0x7f2b380d0f80 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3f000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b381628b0 con 0x7f2b3815ea40 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b3815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b3815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57208/0 (socket says 172.21.15.59:57208) 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3d200640 1 -- 172.21.15.59:0/249696398 learned_addr learned my addr 172.21.15.59:0/249696398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3dc00640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b380d0f80 0x7f2b3815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.420+0000 7f2b3e600640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b380b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b3d200640 1 -- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 msgr2=0x7f2b380b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b3d200640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b380b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b3d200640 1 -- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b380d0f80 msgr2=0x7f2b3815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b3d200640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b380d0f80 0x7f2b3815ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b3d200640 1 -- 172.21.15.59:0/249696398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b380ce690 con 0x7f2b380d1360 2024-04-23T14:32:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b3d200640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b3815a040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c7c0 tx=0x7f2b2400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b35600640 1 -- 172.21.15.59:0/249696398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24015070 con 0x7f2b380d1360 2024-04-23T14:32:19.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b35600640 1 -- 172.21.15.59:0/249696398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b24007820 con 0x7f2b380d1360 2024-04-23T14:32:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b35600640 1 -- 172.21.15.59:0/249696398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24007bf0 con 0x7f2b380d1360 2024-04-23T14:32:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.421+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b380b55f0 con 0x7f2b380d1360 2024-04-23T14:32:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.422+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b380b5bb0 con 0x7f2b380d1360 2024-04-23T14:32:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.423+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3804fd90 con 0x7f2b380d1360 2024-04-23T14:32:19.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.424+0000 7f2b35600640 1 -- 172.21.15.59:0/249696398 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b24018070 con 0x7f2b380d1360 2024-04-23T14:32:19.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.425+0000 7f2b35600640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1806c8f0 0x7f2b1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.425+0000 7f2b35600640 1 -- 172.21.15.59:0/249696398 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2b2408baa0 con 0x7f2b380d1360 2024-04-23T14:32:19.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.425+0000 7f2b3dc00640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1806c8f0 0x7f2b1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.425+0000 7f2b3dc00640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1806c8f0 0x7f2b1806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c006fd0 tx=0x7f2b2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.429+0000 7f2b35600640 1 -- 172.21.15.59:0/249696398 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b24059db0 con 0x7f2b380d1360 2024-04-23T14:32:19.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.650+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2b380d1ec0 con 0x7f2b380d1360 2024-04-23T14:32:19.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.651+0000 7f2b35600640 1 -- 172.21.15.59:0/249696398 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f2b240596b0 con 0x7f2b380d1360 2024-04-23T14:32:19.652 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.654+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1806c8f0 msgr2=0x7f2b1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.654+0000 7f2b3f000640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1806c8f0 0x7f2b1806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c006fd0 tx=0x7f2b2c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.654+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 msgr2=0x7f2b3815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.654+0000 7f2b3f000640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b3815a040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c7c0 tx=0x7f2b2400cc80 comp rx=0 tx=0).stop 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 shutdown_connections 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1806c8f0 0x7f2b1806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b3815ea40 0x7f2b380b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b380d1360 0x7f2b3815a040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 --2- 172.21.15.59:0/249696398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b380d0f80 0x7f2b3815ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 >> 172.21.15.59:0/249696398 conn(0x7f2b3808bf90 msgr2=0x7f2b3815d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 shutdown_connections 2024-04-23T14:32:19.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.655+0000 7f2b3f000640 1 -- 172.21.15.59:0/249696398 wait complete. 2024-04-23T14:32:19.675 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:32:19.675 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:19.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:19.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.933+0000 7fc1a4a00640 1 Processor -- start 2024-04-23T14:32:19.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.933+0000 7fc1a4a00640 1 -- start start 2024-04-23T14:32:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.934+0000 7fc1a4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1a009b410 0x7fc1a00af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.934+0000 7fc1a4a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a00b35f0 con 0x7fc1a015c9d0 2024-04-23T14:32:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.934+0000 7fc1a4a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a00b3890 con 0x7fc1a00d09e0 2024-04-23T14:32:19.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.934+0000 7fc1a4a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a00cd2d0 con 0x7fc1a009b410 2024-04-23T14:32:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.935+0000 7fc19e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc1a015c9d0 0x7fc1a00b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57734/0 (socket says 172.21.15.59:57734) 2024-04-23T14:32:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.935+0000 7fc19e800640 1 -- 172.21.15.59:0/458381712 learned_addr learned my addr 172.21.15.59:0/458381712 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.935+0000 7fc19de00640 1 --2- 172.21.15.59:0/458381712 >> v2:172.21.15.59:3300/0 conn(0x7fc1a009b410 0x7fc1a00af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.935+0000 7fc19de00640 1 -- 172.21.15.59:0/458381712 >> v1:172.21.15.76:6789/0 conn(0x7fc1a00d09e0 legacy=0x7fc1a00d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.935+0000 7fc19de00640 1 -- 172.21.15.59:0/458381712 >> v1:172.21.15.59:6789/0 conn(0x7fc1a015c9d0 legacy=0x7fc1a00b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.935+0000 7fc19de00640 1 -- 172.21.15.59:0/458381712 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00b3b70 con 0x7fc1a009b410 2024-04-23T14:32:19.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.936+0000 7fc19de00640 1 --2- 172.21.15.59:0/458381712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a00af090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00cd450 tx=0x7fc18802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eed3fcf783d5a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.937+0000 7fc19d400640 1 -- 172.21.15.59:0/458381712 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188037070 con 0x7fc1a009b410 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.937+0000 7fc19d400640 1 -- 172.21.15.59:0/458381712 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc188032440 con 0x7fc1a009b410 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.937+0000 7fc19d400640 1 -- 172.21.15.59:0/458381712 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188032810 con 0x7fc1a009b410 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.937+0000 7fc1a4a00640 1 -- 172.21.15.59:0/458381712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 msgr2=0x7fc1a00af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.937+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/458381712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a00af090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00cd450 tx=0x7fc18802a1c0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.938+0000 7fc1a4a00640 1 -- 172.21.15.59:0/458381712 shutdown_connections 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.938+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/458381712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a00af090 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.938+0000 7fc1a4a00640 1 -- 172.21.15.59:0/458381712 >> 172.21.15.59:0/458381712 conn(0x7fc1a00b3ef0 msgr2=0x7fc1a00b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.938+0000 7fc1a4a00640 1 -- 172.21.15.59:0/458381712 shutdown_connections 2024-04-23T14:32:19.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.938+0000 7fc1a4a00640 1 -- 172.21.15.59:0/458381712 wait complete. 2024-04-23T14:32:19.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.939+0000 7fc1a4a00640 1 Processor -- start 2024-04-23T14:32:19.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.940+0000 7fc1a4a00640 1 -- start start 2024-04-23T14:32:19.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.940+0000 7fc1a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.940+0000 7fc1a4a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00d09e0 0x7fc1a00c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.940+0000 7fc1a4a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015c9d0 0x7fc1a00c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.940+0000 7fc1a4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a015cdb0 con 0x7fc1a009b410 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.940+0000 7fc1a4a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00ccef0 con 0x7fc1a00d09e0 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc1a4a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a015cf30 con 0x7fc1a015c9d0 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a015f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57236/0 (socket says 172.21.15.59:57236) 2024-04-23T14:32:19.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 -- 172.21.15.59:0/2971307836 learned_addr learned my addr 172.21.15.59:0/2971307836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19f200640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015c9d0 0x7fc1a00c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19de00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00d09e0 0x7fc1a00c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 -- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015c9d0 msgr2=0x7fc1a00c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015c9d0 0x7fc1a00c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 -- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00d09e0 msgr2=0x7fc1a00c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00d09e0 0x7fc1a00c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.941+0000 7fc19e800640 1 -- 172.21.15.59:0/2971307836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00c9c20 con 0x7fc1a009b410 2024-04-23T14:32:19.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.942+0000 7fc19e800640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a015f3d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc19000caa0 tx=0x7fc19000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.942+0000 7fc196a00640 1 -- 172.21.15.59:0/2971307836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190017070 con 0x7fc1a009b410 2024-04-23T14:32:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.942+0000 7fc196a00640 1 -- 172.21.15.59:0/2971307836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19000ac00 con 0x7fc1a009b410 2024-04-23T14:32:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.942+0000 7fc196a00640 1 -- 172.21.15.59:0/2971307836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190012400 con 0x7fc1a009b410 2024-04-23T14:32:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.942+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a006d670 con 0x7fc1a009b410 2024-04-23T14:32:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.942+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a006dc30 con 0x7fc1a009b410 2024-04-23T14:32:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.943+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a004fd90 con 0x7fc1a009b410 2024-04-23T14:32:19.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.945+0000 7fc196a00640 1 -- 172.21.15.59:0/2971307836 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc19001a070 con 0x7fc1a009b410 2024-04-23T14:32:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.946+0000 7fc196a00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.946+0000 7fc196a00640 1 -- 172.21.15.59:0/2971307836 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc19008b920 con 0x7fc1a009b410 2024-04-23T14:32:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.947+0000 7fc19de00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:19.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.947+0000 7fc19de00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc17c06c8f0 0x7fc17c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc188034000 tx=0x7fc18802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:19.951+0000 7fc196a00640 1 -- 172.21.15.59:0/2971307836 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc190059cc0 con 0x7fc1a009b410 2024-04-23T14:32:20.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.154+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1a00b3330 con 0x7fc1a009b410 2024-04-23T14:32:20.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.155+0000 7fc196a00640 1 -- 172.21.15.59:0/2971307836 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fc1900595c0 con 0x7fc1a009b410 2024-04-23T14:32:20.157 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:20.157 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc17c06c8f0 msgr2=0x7fc17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc17c06c8f0 0x7fc17c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc188034000 tx=0x7fc18802e000 comp rx=0 tx=0).stop 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 msgr2=0x7fc1a015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a015f3d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc19000caa0 tx=0x7fc19000cf60 comp rx=0 tx=0).stop 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 shutdown_connections 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc17c06c8f0 0x7fc17c06edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1a015c9d0 0x7fc1a00c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1a00d09e0 0x7fc1a00c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 --2- 172.21.15.59:0/2971307836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1a009b410 0x7fc1a015f3d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.158+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 >> 172.21.15.59:0/2971307836 conn(0x7fc1a00b3ef0 msgr2=0x7fc1a00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:20.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.159+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 shutdown_connections 2024-04-23T14:32:20.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.159+0000 7fc1a4a00640 1 -- 172.21.15.59:0/2971307836 wait complete. 2024-04-23T14:32:20.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:20.179 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:20.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.384+0000 7f2b84800640 1 Processor -- start 2024-04-23T14:32:20.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.384+0000 7f2b84800640 1 -- start start 2024-04-23T14:32:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b84800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2b7c163770 0x7f2b7c0d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b84800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b7c160180 0x7f2b7c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b84800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c0cff50 con 0x7f2b7c15cb20 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b84800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c15cf00 con 0x7f2b7c160180 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b84800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c0d01c0 con 0x7f2b7c163770 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b81e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2b7c15cb20 0x7f2b7c0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46554/0 (socket says 172.21.15.59:46554) 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b81e00640 1 -- 172.21.15.59:0/2791037160 learned_addr learned my addr 172.21.15.59:0/2791037160 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b82800640 1 --2- 172.21.15.59:0/2791037160 >> v2:172.21.15.59:3300/0 conn(0x7f2b7c160180 0x7f2b7c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.386+0000 7f2b81400640 1 --2- 172.21.15.59:0/2791037160 >> v2:172.21.15.76:3300/0 conn(0x7f2b7c163770 0x7f2b7c0d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.387+0000 7f2b82800640 1 -- 172.21.15.59:0/2791037160 >> v2:172.21.15.76:3300/0 conn(0x7f2b7c163770 msgr2=0x7f2b7c0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.387+0000 7f2b82800640 1 --2- 172.21.15.59:0/2791037160 >> v2:172.21.15.76:3300/0 conn(0x7f2b7c163770 0x7f2b7c0d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.387+0000 7f2b82800640 1 -- 172.21.15.59:0/2791037160 >> v1:172.21.15.59:6789/0 conn(0x7f2b7c15cb20 legacy=0x7f2b7c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.387+0000 7f2b82800640 1 -- 172.21.15.59:0/2791037160 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c15f6a0 con 0x7f2b7c160180 2024-04-23T14:32:20.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.388+0000 7f2b82800640 1 --2- 172.21.15.59:0/2791037160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c160180 0x7f2b7c0d1780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c06dc70 tx=0x7f2b7802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636ea263db15897f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:20.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.388+0000 7f2b80a00640 1 -- 172.21.15.59:0/2791037160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78007740 con 0x7f2b7c160180 2024-04-23T14:32:20.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.388+0000 7f2b80a00640 1 -- 172.21.15.59:0/2791037160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b780078e0 con 0x7f2b7c160180 2024-04-23T14:32:20.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.388+0000 7f2b80a00640 1 -- 172.21.15.59:0/2791037160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78007cb0 con 0x7f2b7c160180 2024-04-23T14:32:20.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 -- 172.21.15.59:0/2791037160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c160180 msgr2=0x7f2b7c0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 --2- 172.21.15.59:0/2791037160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c160180 0x7f2b7c0d1780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c06dc70 tx=0x7f2b7802a5f0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 -- 172.21.15.59:0/2791037160 shutdown_connections 2024-04-23T14:32:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 --2- 172.21.15.59:0/2791037160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c160180 0x7f2b7c0d1780 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 --2- 172.21.15.59:0/2791037160 >> v2:172.21.15.76:3300/0 conn(0x7f2b7c163770 0x7f2b7c0d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 -- 172.21.15.59:0/2791037160 >> 172.21.15.59:0/2791037160 conn(0x7f2b7c06cc70 msgr2=0x7f2b7c06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 -- 172.21.15.59:0/2791037160 shutdown_connections 2024-04-23T14:32:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.389+0000 7f2b84800640 1 -- 172.21.15.59:0/2791037160 wait complete. 2024-04-23T14:32:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.391+0000 7f2b84800640 1 Processor -- start 2024-04-23T14:32:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.392+0000 7f2b84800640 1 -- start start 2024-04-23T14:32:20.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.393+0000 7f2b84800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15cb20 0x7f2b7c14fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.393+0000 7f2b84800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c160180 0x7f2b7c0b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.393+0000 7f2b84800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c163770 0x7f2b7c0b9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.393+0000 7f2b84800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0d2d60 con 0x7f2b7c15cb20 2024-04-23T14:32:20.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.393+0000 7f2b84800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0d2be0 con 0x7f2b7c163770 2024-04-23T14:32:20.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.393+0000 7f2b84800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c0d2ee0 con 0x7f2b7c160180 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15cb20 0x7f2b7c14fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15cb20 0x7f2b7c14fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46524/0 (socket says 172.21.15.59:46524) 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 -- 172.21.15.59:0/2589931148 learned_addr learned my addr 172.21.15.59:0/2589931148 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b82800640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c163770 0x7f2b7c0b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81400640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c160180 0x7f2b7c0b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 -- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c160180 msgr2=0x7f2b7c0b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c160180 0x7f2b7c0b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 -- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c163770 msgr2=0x7f2b7c0b9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c163770 0x7f2b7c0b9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.394+0000 7f2b81e00640 1 -- 172.21.15.59:0/2589931148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0c9b10 con 0x7f2b7c15cb20 2024-04-23T14:32:20.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.395+0000 7f2b81e00640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15cb20 0x7f2b7c14fb60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000c400 tx=0x7f2b7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:20.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.395+0000 7f2b76000640 1 -- 172.21.15.59:0/2589931148 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70017070 con 0x7f2b7c15cb20 2024-04-23T14:32:20.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.395+0000 7f2b76000640 1 -- 172.21.15.59:0/2589931148 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b70005ce0 con 0x7f2b7c15cb20 2024-04-23T14:32:20.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.395+0000 7f2b76000640 1 -- 172.21.15.59:0/2589931148 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700072f0 con 0x7f2b7c15cb20 2024-04-23T14:32:20.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.395+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c0c9df0 con 0x7f2b7c15cb20 2024-04-23T14:32:20.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.395+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c14dab0 con 0x7f2b7c15cb20 2024-04-23T14:32:20.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.396+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c068a80 con 0x7f2b7c15cb20 2024-04-23T14:32:20.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.397+0000 7f2b76000640 1 -- 172.21.15.59:0/2589931148 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b7001a070 con 0x7f2b7c15cb20 2024-04-23T14:32:20.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.398+0000 7f2b76000640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c8f0 0x7f2b6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.399+0000 7f2b76000640 1 -- 172.21.15.59:0/2589931148 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2b7008aee0 con 0x7f2b7c15cb20 2024-04-23T14:32:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.399+0000 7f2b81400640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c8f0 0x7f2b6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.399+0000 7f2b81400640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c8f0 0x7f2b6006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c006fd0 tx=0x7f2b6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:20.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.403+0000 7f2b76000640 1 -- 172.21.15.59:0/2589931148 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b70058c30 con 0x7f2b7c15cb20 2024-04-23T14:32:20.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.610+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b7c14dc80 con 0x7f2b7c15cb20 2024-04-23T14:32:20.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.611+0000 7f2b76000640 1 -- 172.21.15.59:0/2589931148 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f2b7005cf30 con 0x7f2b7c15cb20 2024-04-23T14:32:20.612 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:20.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c8f0 msgr2=0x7f2b6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c8f0 0x7f2b6006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c006fd0 tx=0x7f2b6c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15cb20 msgr2=0x7f2b7c14fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15cb20 0x7f2b7c14fb60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000c400 tx=0x7f2b7000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 shutdown_connections 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c8f0 0x7f2b6006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c160180 0x7f2b7c0b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c163770 0x7f2b7c0b9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 --2- 172.21.15.59:0/2589931148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15cb20 0x7f2b7c14fb60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 >> 172.21.15.59:0/2589931148 conn(0x7f2b7c06cc70 msgr2=0x7f2b7c161d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.614+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 shutdown_connections 2024-04-23T14:32:20.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.615+0000 7f2b84800640 1 -- 172.21.15.59:0/2589931148 wait complete. 2024-04-23T14:32:20.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:20.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.849+0000 7f1351000640 1 Processor -- start 2024-04-23T14:32:20.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.850+0000 7f1351000640 1 -- start start 2024-04-23T14:32:20.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.851+0000 7f1351000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 0x7f134c160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.851+0000 7f1351000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c15cab0 con 0x7f134c1567e0 2024-04-23T14:32:20.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.851+0000 7f1351000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f134c160e40 con 0x7f134c0d0d10 2024-04-23T14:32:20.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.851+0000 7f1351000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f134c15cd30 con 0x7f134c15d810 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.852+0000 7f134b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f134c1567e0 0x7f134c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46556/0 (socket says 172.21.15.59:46556) 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.852+0000 7f134b400640 1 -- 172.21.15.59:0/1362542419 learned_addr learned my addr 172.21.15.59:0/1362542419 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.852+0000 7f134be00640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 0x7f134c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.852+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164612846 0 0) 0x7f133c001490 con 0x7f134c1567e0 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.852+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1330003520 con 0x7f134c1567e0 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051847367 0 0) 0x7f1330003520 con 0x7f134c1567e0 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 msgr2=0x7f134c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 0x7f134c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 >> v1:172.21.15.76:6789/0 conn(0x7f134c0d0d10 legacy=0x7f134c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c15d010 con 0x7f134c1567e0 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f133c001bc0 con 0x7f134c1567e0 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 >> v1:172.21.15.59:6789/0 conn(0x7f134c1567e0 legacy=0x7f134c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f1330003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 0x7f1330004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1330004d40 0x7f13300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f134c15cd30 con 0x7f13300039e0 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f133000baa0 con 0x7f1330004d40 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f133000bca0 con 0x7f134c15d810 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134b400640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f1330003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.853+0000 7f134aa00640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 0x7f1330004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.854+0000 7f134b400640 1 -- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 msgr2=0x7f1330004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.854+0000 7f134b400640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c15d810 0x7f1330004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.854+0000 7f134b400640 1 -- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1330004d40 msgr2=0x7f13300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:20.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.854+0000 7f134be00640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1330004d40 0x7f13300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.854+0000 7f134b400640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1330004d40 0x7f13300051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.854+0000 7f134b400640 1 -- 172.21.15.59:0/1362542419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1330003520 con 0x7f13300039e0 2024-04-23T14:32:20.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.855+0000 7f134b400640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f1330003e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f133c00a920 tx=0x7f1330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e430b1f4612708bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:20.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.855+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c00a570 con 0x7f13300039e0 2024-04-23T14:32:20.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.855+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c02f070 con 0x7f13300039e0 2024-04-23T14:32:20.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.855+0000 7f134a000640 1 -- 172.21.15.59:0/1362542419 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c02be10 con 0x7f13300039e0 2024-04-23T14:32:20.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.855+0000 7f1351000640 1 -- 172.21.15.59:0/1362542419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 msgr2=0x7f1330003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.855+0000 7f1351000640 1 --2- 172.21.15.59:0/1362542419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f1330003e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f133c00a920 tx=0x7f1330005830 comp rx=0 tx=0).stop 2024-04-23T14:32:20.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.855+0000 7f134be00640 1 -- 172.21.15.59:0/1362542419 reap_dead start 2024-04-23T14:32:20.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.856+0000 7f1351000640 1 -- 172.21.15.59:0/1362542419 shutdown_connections 2024-04-23T14:32:20.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.856+0000 7f1351000640 1 -- 172.21.15.59:0/1362542419 >> 172.21.15.59:0/1362542419 conn(0x7f134c06bdf0 msgr2=0x7f134c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:20.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.856+0000 7f1351000640 1 -- 172.21.15.59:0/1362542419 shutdown_connections 2024-04-23T14:32:20.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.856+0000 7f1351000640 1 -- 172.21.15.59:0/1362542419 wait complete. 2024-04-23T14:32:20.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.857+0000 7f1351000640 1 Processor -- start 2024-04-23T14:32:20.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.858+0000 7f1351000640 1 -- start start 2024-04-23T14:32:20.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.858+0000 7f1351000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f134c14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f1351000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c0b0720 0x7f134c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f1351000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f134c0b0b80 0x7f134c0b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f1351000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f134c1609a0 con 0x7f13300039e0 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f1351000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f134c160820 con 0x7f134c0b0b80 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f1351000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f134c160b20 con 0x7f134c0b0720 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f134c14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f134c14c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46538/0 (socket says 172.21.15.59:46538) 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 -- 172.21.15.59:0/1578117779 learned_addr learned my addr 172.21.15.59:0/1578117779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134aa00640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c0b0720 0x7f134c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134be00640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f134c0b0b80 0x7f134c0b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 -- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c0b0720 msgr2=0x7f134c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c0b0720 0x7f134c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 -- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f134c0b0b80 msgr2=0x7f134c0b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f134c0b0b80 0x7f134c0b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:20.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.859+0000 7f134b400640 1 -- 172.21.15.59:0/1578117779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c146930 con 0x7f13300039e0 2024-04-23T14:32:20.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.860+0000 7f134b400640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f134c14c260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f133c02b5e0 tx=0x7f134c15d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:20.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.860+0000 7f1343400640 1 -- 172.21.15.59:0/1578117779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c001250 con 0x7f13300039e0 2024-04-23T14:32:20.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.860+0000 7f1343400640 1 -- 172.21.15.59:0/1578117779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133c02bb40 con 0x7f13300039e0 2024-04-23T14:32:20.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.860+0000 7f1343400640 1 -- 172.21.15.59:0/1578117779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c034470 con 0x7f13300039e0 2024-04-23T14:32:20.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.860+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c146c10 con 0x7f13300039e0 2024-04-23T14:32:20.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.860+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c1471d0 con 0x7f13300039e0 2024-04-23T14:32:20.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.861+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c04fd90 con 0x7f13300039e0 2024-04-23T14:32:20.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.862+0000 7f1343400640 1 -- 172.21.15.59:0/1578117779 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f133c034610 con 0x7f13300039e0 2024-04-23T14:32:20.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.863+0000 7f1343400640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f132c06ca90 0x7f132c06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:20.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.863+0000 7f1343400640 1 -- 172.21.15.59:0/1578117779 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f133c0ac270 con 0x7f13300039e0 2024-04-23T14:32:20.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.864+0000 7f134aa00640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f132c06ca90 0x7f132c06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:20.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.864+0000 7f134aa00640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f132c06ca90 0x7f132c06ef50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1338000c00 tx=0x7f1338009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:20.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:20.868+0000 7f1343400640 1 -- 172.21.15.59:0/1578117779 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f134c15cab0 con 0x7f13300039e0 2024-04-23T14:32:21.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.073+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f134c1554f0 con 0x7f13300039e0 2024-04-23T14:32:21.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.074+0000 7f1343400640 1 -- 172.21.15.59:0/1578117779 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f133c07a4f0 con 0x7f13300039e0 2024-04-23T14:32:21.075 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:21.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f132c06ca90 msgr2=0x7f132c06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f132c06ca90 0x7f132c06ef50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1338000c00 tx=0x7f1338009040 comp rx=0 tx=0).stop 2024-04-23T14:32:21.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 msgr2=0x7f134c14c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f134c14c260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f133c02b5e0 tx=0x7f134c15d1e0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 shutdown_connections 2024-04-23T14:32:21.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f132c06ca90 0x7f132c06ef50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f134c0b0720 0x7f134c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f134c0b0b80 0x7f134c0b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 --2- 172.21.15.59:0/1578117779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13300039e0 0x7f134c14c260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 >> 172.21.15.59:0/1578117779 conn(0x7f134c06bdf0 msgr2=0x7f134c15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:21.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 shutdown_connections 2024-04-23T14:32:21.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.078+0000 7f1351000640 1 -- 172.21.15.59:0/1578117779 wait complete. 2024-04-23T14:32:21.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:21.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.314+0000 7f342c200640 1 Processor -- start 2024-04-23T14:32:21.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.315+0000 7f342c200640 1 -- start start 2024-04-23T14:32:21.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f342c200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f342415e790 0x7f34240aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f342c200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f3424162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f342c200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3424163370 con 0x7f3424071d70 2024-04-23T14:32:21.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f342c200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f34240aef40 con 0x7f342415e790 2024-04-23T14:32:21.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f342c200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f34240afd80 con 0x7f342406f6f0 2024-04-23T14:32:21.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f3429800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f342415e790 0x7f34240aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f3429800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f342415e790 0x7f34240aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46548/0 (socket says 172.21.15.59:46548) 2024-04-23T14:32:21.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f3429800640 1 -- 172.21.15.59:0/3132040907 learned_addr learned my addr 172.21.15.59:0/3132040907 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:21.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.317+0000 7f3428e00640 1 --2- 172.21.15.59:0/3132040907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f3424162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.318+0000 7f3429800640 1 -- 172.21.15.59:0/3132040907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 msgr2=0x7f3424162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.318+0000 7f3429800640 1 --2- 172.21.15.59:0/3132040907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f3424162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.318+0000 7f3429800640 1 -- 172.21.15.59:0/3132040907 >> v1:172.21.15.76:6789/0 conn(0x7f3424071d70 legacy=0x7f34240b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.318+0000 7f3429800640 1 -- 172.21.15.59:0/3132040907 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34240adc80 con 0x7f342415e790 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.319+0000 7f3429800640 1 --2- 172.21.15.59:0/3132040907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f34240aea00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3418009870 tx=0x7f341802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefcc67deea7a1b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.319+0000 7f3423e00640 1 -- 172.21.15.59:0/3132040907 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418037070 con 0x7f342415e790 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.319+0000 7f3423e00640 1 -- 172.21.15.59:0/3132040907 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3418032440 con 0x7f342415e790 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.319+0000 7f3423e00640 1 -- 172.21.15.59:0/3132040907 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418032810 con 0x7f342415e790 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 -- 172.21.15.59:0/3132040907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 msgr2=0x7f34240aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 --2- 172.21.15.59:0/3132040907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f34240aea00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3418009870 tx=0x7f341802a210 comp rx=0 tx=0).stop 2024-04-23T14:32:21.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 -- 172.21.15.59:0/3132040907 shutdown_connections 2024-04-23T14:32:21.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 --2- 172.21.15.59:0/3132040907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f3424162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 --2- 172.21.15.59:0/3132040907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f34240aea00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 -- 172.21.15.59:0/3132040907 >> 172.21.15.59:0/3132040907 conn(0x7f3424067d90 msgr2=0x7f342406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:21.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 -- 172.21.15.59:0/3132040907 shutdown_connections 2024-04-23T14:32:21.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.320+0000 7f342c200640 1 -- 172.21.15.59:0/3132040907 wait complete. 2024-04-23T14:32:21.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.322+0000 7f342c200640 1 Processor -- start 2024-04-23T14:32:21.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.322+0000 7f342c200640 1 -- start start 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342c200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f34240cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342c200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3424071d70 0x7f34241f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342c200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f342415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342c200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34240ae4e0 con 0x7f342415e790 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342c200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f34240ae200 con 0x7f3424071d70 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342c200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f342406fbb0 con 0x7f342406f6f0 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f342415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.323+0000 7f342a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f342415fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46550/0 (socket says 172.21.15.59:46550) 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f342a200640 1 -- 172.21.15.59:0/227001655 learned_addr learned my addr 172.21.15.59:0/227001655 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f3428e00640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3424071d70 0x7f34241f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f3429800640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f34240cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f342a200640 1 -- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 msgr2=0x7f34240cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f342a200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f34240cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f342a200640 1 -- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3424071d70 msgr2=0x7f34241f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f342a200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3424071d70 0x7f34241f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f342a200640 1 -- 172.21.15.59:0/227001655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3424149760 con 0x7f342415e790 2024-04-23T14:32:21.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.324+0000 7f3428e00640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3424071d70 0x7f34241f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:21.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.325+0000 7f342a200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f342415fed0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c7b0 tx=0x7f341c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.325+0000 7f3421600640 1 -- 172.21.15.59:0/227001655 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c004b60 con 0x7f342415e790 2024-04-23T14:32:21.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.325+0000 7f3421600640 1 -- 172.21.15.59:0/227001655 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c004d00 con 0x7f342415e790 2024-04-23T14:32:21.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.325+0000 7f3421600640 1 -- 172.21.15.59:0/227001655 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c00aa30 con 0x7f342415e790 2024-04-23T14:32:21.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.325+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34240ce4c0 con 0x7f342415e790 2024-04-23T14:32:21.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.325+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34240cea80 con 0x7f342415e790 2024-04-23T14:32:21.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.326+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342406bf00 con 0x7f342415e790 2024-04-23T14:32:21.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.327+0000 7f3421600640 1 -- 172.21.15.59:0/227001655 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f341c00abd0 con 0x7f342415e790 2024-04-23T14:32:21.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.328+0000 7f3421600640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f340406c8f0 0x7f340406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.329+0000 7f3421600640 1 -- 172.21.15.59:0/227001655 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f341c08be00 con 0x7f342415e790 2024-04-23T14:32:21.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.329+0000 7f3429800640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f340406c8f0 0x7f340406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.330+0000 7f3429800640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f340406c8f0 0x7f340406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3418034000 tx=0x7f341802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.333+0000 7f3421600640 1 -- 172.21.15.59:0/227001655 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f342406bf00 con 0x7f342415e790 2024-04-23T14:32:21.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.537+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3424068a80 con 0x7f342415e790 2024-04-23T14:32:21.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.538+0000 7f3421600640 1 -- 172.21.15.59:0/227001655 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f341c020070 con 0x7f342415e790 2024-04-23T14:32:21.539 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:21.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f340406c8f0 msgr2=0x7f340406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f340406c8f0 0x7f340406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3418034000 tx=0x7f341802e000 comp rx=0 tx=0).stop 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 msgr2=0x7f342415fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f342415fed0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c7b0 tx=0x7f341c00cc70 comp rx=0 tx=0).stop 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 shutdown_connections 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f340406c8f0 0x7f340406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f342415e790 0x7f342415fed0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3424071d70 0x7f34241f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 --2- 172.21.15.59:0/227001655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f342406f6f0 0x7f34240cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.540+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 >> 172.21.15.59:0/227001655 conn(0x7f3424067d90 msgr2=0x7f34240721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.541+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 shutdown_connections 2024-04-23T14:32:21.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.541+0000 7f342c200640 1 -- 172.21.15.59:0/227001655 wait complete. 2024-04-23T14:32:21.561 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:21.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.766+0000 7f7dc6e00640 1 Processor -- start 2024-04-23T14:32:21.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.766+0000 7f7dc6e00640 1 -- start start 2024-04-23T14:32:21.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.768+0000 7f7dc6e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7dc00cf7b0 0x7f7dc01663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.768+0000 7f7dc6e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc015c180 con 0x7f7dc006ef00 2024-04-23T14:32:21.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.768+0000 7f7dc6e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc015c460 con 0x7f7dc015bc00 2024-04-23T14:32:21.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.768+0000 7f7dc6e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc00b2f50 con 0x7f7dc00cf7b0 2024-04-23T14:32:21.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.768+0000 7f7dbfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7dc006ef00 0x7f7dc006f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46568/0 (socket says 172.21.15.59:46568) 2024-04-23T14:32:21.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.768+0000 7f7dbfe00640 1 -- 172.21.15.59:0/2542354652 learned_addr learned my addr 172.21.15.59:0/2542354652 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:21.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.768+0000 7f7dc4e00640 1 --2- 172.21.15.59:0/2542354652 >> v2:172.21.15.59:3300/0 conn(0x7f7dc00cf7b0 0x7f7dc01663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.769+0000 7f7dbf400640 1 -- 172.21.15.59:0/2542354652 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1116599708 0 0) 0x7f7dc015c180 con 0x7f7dc006ef00 2024-04-23T14:32:21.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.769+0000 7f7dbf400640 1 -- 172.21.15.59:0/2542354652 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7da4003520 con 0x7f7dc006ef00 2024-04-23T14:32:21.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.769+0000 7f7dc4e00640 1 -- 172.21.15.59:0/2542354652 >> v1:172.21.15.76:6789/0 conn(0x7f7dc015bc00 legacy=0x7f7dc015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:21.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.769+0000 7f7dc4e00640 1 -- 172.21.15.59:0/2542354652 >> v1:172.21.15.59:6789/0 conn(0x7f7dc006ef00 legacy=0x7f7dc006f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.769+0000 7f7dc4e00640 1 -- 172.21.15.59:0/2542354652 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc015c740 con 0x7f7dc00cf7b0 2024-04-23T14:32:21.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.770+0000 7f7dc4e00640 1 --2- 172.21.15.59:0/2542354652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc00cf7b0 0x7f7dc01663c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009d90 tx=0x7f7db802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c7300e072bc361 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dbf400640 1 -- 172.21.15.59:0/2542354652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8007740 con 0x7f7dc00cf7b0 2024-04-23T14:32:21.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dbf400640 1 -- 172.21.15.59:0/2542354652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db80078e0 con 0x7f7dc00cf7b0 2024-04-23T14:32:21.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dbf400640 1 -- 172.21.15.59:0/2542354652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8007cb0 con 0x7f7dc00cf7b0 2024-04-23T14:32:21.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dc6e00640 1 -- 172.21.15.59:0/2542354652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc00cf7b0 msgr2=0x7f7dc01663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/2542354652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc00cf7b0 0x7f7dc01663c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009d90 tx=0x7f7db802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dc6e00640 1 -- 172.21.15.59:0/2542354652 shutdown_connections 2024-04-23T14:32:21.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/2542354652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc00cf7b0 0x7f7dc01663c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.771+0000 7f7dc6e00640 1 -- 172.21.15.59:0/2542354652 >> 172.21.15.59:0/2542354652 conn(0x7f7dc00b2130 msgr2=0x7f7dc00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:21.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.772+0000 7f7dc6e00640 1 -- 172.21.15.59:0/2542354652 shutdown_connections 2024-04-23T14:32:21.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.772+0000 7f7dc6e00640 1 -- 172.21.15.59:0/2542354652 wait complete. 2024-04-23T14:32:21.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.773+0000 7f7dc6e00640 1 Processor -- start 2024-04-23T14:32:21.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.774+0000 7f7dc6e00640 1 -- start start 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.774+0000 7f7dc6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc006ef00 0x7f7dc0142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.774+0000 7f7dc6e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7dc00cf7b0 0x7f7dc013fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.774+0000 7f7dc6e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc015bc00 0x7f7dc00b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dc6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0155ff0 con 0x7f7dc006ef00 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dc6e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc00ce210 con 0x7f7dc015bc00 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dc6e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc0156170 con 0x7f7dc00cf7b0 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc006ef00 0x7f7dc0142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc006ef00 0x7f7dc0142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46564/0 (socket says 172.21.15.59:46564) 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 -- 172.21.15.59:0/3488419134 learned_addr learned my addr 172.21.15.59:0/3488419134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dc4e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc015bc00 0x7f7dc00b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7db7e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7dc00cf7b0 0x7f7dc013fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 -- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7dc00cf7b0 msgr2=0x7f7dc013fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7dc00cf7b0 0x7f7dc013fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 -- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc015bc00 msgr2=0x7f7dc00b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc015bc00 0x7f7dc00b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dbfe00640 1 -- 172.21.15.59:0/3488419134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc00b3990 con 0x7f7dc006ef00 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7dc4e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc015bc00 0x7f7dc00b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.775+0000 7f7db7e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7dc00cf7b0 0x7f7dc013fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.776+0000 7f7dbfe00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc006ef00 0x7f7dc0142db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7db000c430 tx=0x7f7db000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.776+0000 7f7dbcc00640 1 -- 172.21.15.59:0/3488419134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0017070 con 0x7f7dc006ef00 2024-04-23T14:32:21.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.776+0000 7f7dbcc00640 1 -- 172.21.15.59:0/3488419134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db0005ce0 con 0x7f7dc006ef00 2024-04-23T14:32:21.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.776+0000 7f7dbcc00640 1 -- 172.21.15.59:0/3488419134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00072f0 con 0x7f7dc006ef00 2024-04-23T14:32:21.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.776+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc00b3c70 con 0x7f7dc006ef00 2024-04-23T14:32:21.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.776+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc00b4230 con 0x7f7dc006ef00 2024-04-23T14:32:21.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.777+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc0068a80 con 0x7f7dc006ef00 2024-04-23T14:32:21.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.779+0000 7f7dbcc00640 1 -- 172.21.15.59:0/3488419134 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7db001a070 con 0x7f7dc006ef00 2024-04-23T14:32:21.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.779+0000 7f7dbcc00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:21.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.780+0000 7f7dbcc00640 1 -- 172.21.15.59:0/3488419134 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7db008b130 con 0x7f7dc006ef00 2024-04-23T14:32:21.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.780+0000 7f7db7e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:21.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.780+0000 7f7db7e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00b58c0 tx=0x7f7da8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:21.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:21.784+0000 7f7dbcc00640 1 -- 172.21.15.59:0/3488419134 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7db0059440 con 0x7f7dc006ef00 2024-04-23T14:32:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.005+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7dc00b27e0 con 0x7f7dc006ef00 2024-04-23T14:32:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.006+0000 7f7dbcc00640 1 -- 172.21.15.59:0/3488419134 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f7db0058d40 con 0x7f7dc006ef00 2024-04-23T14:32:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d9c06c8f0 msgr2=0x7f7d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00b58c0 tx=0x7f7da8008040 comp rx=0 tx=0).stop 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc006ef00 msgr2=0x7f7dc0142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc006ef00 0x7f7dc0142db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7db000c430 tx=0x7f7db000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 shutdown_connections 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d9c06c8f0 0x7f7d9c06edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7dc00cf7b0 0x7f7dc013fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc015bc00 0x7f7dc00b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 --2- 172.21.15.59:0/3488419134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc006ef00 0x7f7dc0142db0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.009+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 >> 172.21.15.59:0/3488419134 conn(0x7f7dc00b2130 msgr2=0x7f7dc006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.010+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 shutdown_connections 2024-04-23T14:32:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.010+0000 7f7dc6e00640 1 -- 172.21.15.59:0/3488419134 wait complete. 2024-04-23T14:32:22.029 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-04-23T14:32:22.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:22.029 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-23T14:32:22.123 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:22.123 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:32:22.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:22.123 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:32:22.187 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:32:22.187 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:22.247 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:32:22.248 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:22.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.512+0000 7fa091e00640 1 Processor -- start 2024-04-23T14:32:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.513+0000 7fa091e00640 1 -- start start 2024-04-23T14:32:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.514+0000 7fa091e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 0x7fa08c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.514+0000 7fa091e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa08c0cd520 con 0x7fa08c1454e0 2024-04-23T14:32:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.514+0000 7fa091e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa08c0cd800 con 0x7fa08c0cd020 2024-04-23T14:32:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.514+0000 7fa091e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c0cddc0 con 0x7fa08c160910 2024-04-23T14:32:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.514+0000 7fa08aa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa08c1454e0 0x7fa08c145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46570/0 (socket says 172.21.15.59:46570) 2024-04-23T14:32:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.514+0000 7fa08aa00640 1 -- 172.21.15.59:0/1469153032 learned_addr learned my addr 172.21.15.59:0/1469153032 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:22.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08b400640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 0x7fa08c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945340383 0 0) 0x7fa08c0cd520 con 0x7fa08c1454e0 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa070003520 con 0x7fa08c1454e0 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 522111375 0 0) 0x7fa070003520 con 0x7fa08c1454e0 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 msgr2=0x7fa08c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 0x7fa08c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 >> v1:172.21.15.76:6789/0 conn(0x7fa08c0cd020 legacy=0x7fa08c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c0cdae0 con 0x7fa08c1454e0 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa07c001670 con 0x7fa08c1454e0 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 >> v1:172.21.15.59:6789/0 conn(0x7fa08c1454e0 legacy=0x7fa08c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0700039e0 0x7fa070003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.515+0000 7fa08a000640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 0x7fa070004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08a000640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa070004d40 0x7fa0700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0cddc0 con 0x7fa0700039e0 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa07000bc40 con 0x7fa070004d40 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa07000be40 con 0x7fa08c160910 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08aa00640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0700039e0 0x7fa070003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa082000640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 0x7fa070004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08b400640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa070004d40 0x7fa0700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08aa00640 1 -- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 msgr2=0x7fa070004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08aa00640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c160910 0x7fa070004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08aa00640 1 -- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa070004d40 msgr2=0x7fa0700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08aa00640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa070004d40 0x7fa0700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.516+0000 7fa08aa00640 1 -- 172.21.15.59:0/1469153032 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa070003520 con 0x7fa0700039e0 2024-04-23T14:32:22.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.517+0000 7fa08aa00640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0700039e0 0x7fa070003e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00c160 tx=0x7fa070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71668450bd596efb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.518+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c007c60 con 0x7fa0700039e0 2024-04-23T14:32:22.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.518+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c005af0 con 0x7fa0700039e0 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.518+0000 7fa08a000640 1 -- 172.21.15.59:0/1469153032 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c007070 con 0x7fa0700039e0 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.519+0000 7fa091e00640 1 -- 172.21.15.59:0/1469153032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0700039e0 msgr2=0x7fa070003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.519+0000 7fa091e00640 1 --2- 172.21.15.59:0/1469153032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0700039e0 0x7fa070003e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00c160 tx=0x7fa070005830 comp rx=0 tx=0).stop 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.519+0000 7fa08b400640 1 -- 172.21.15.59:0/1469153032 reap_dead start 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.519+0000 7fa091e00640 1 -- 172.21.15.59:0/1469153032 shutdown_connections 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.519+0000 7fa091e00640 1 -- 172.21.15.59:0/1469153032 >> 172.21.15.59:0/1469153032 conn(0x7fa08c067ea0 msgr2=0x7fa08c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.519+0000 7fa091e00640 1 -- 172.21.15.59:0/1469153032 shutdown_connections 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.519+0000 7fa091e00640 1 -- 172.21.15.59:0/1469153032 wait complete. 2024-04-23T14:32:22.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.521+0000 7fa091e00640 1 Processor -- start 2024-04-23T14:32:22.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.521+0000 7fa091e00640 1 -- start start 2024-04-23T14:32:22.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa091e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0700039e0 0x7fa08c144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa091e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa08c15b7d0 0x7fa08c06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa091e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c1f9140 0x7fa08c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa091e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0b4010 con 0x7fa08c15b7d0 2024-04-23T14:32:22.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa091e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c1462c0 con 0x7fa0700039e0 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa091e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c15c450 con 0x7fa08c1f9140 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa082000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa08c15b7d0 0x7fa08c06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa082000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa08c15b7d0 0x7fa08c06e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46574/0 (socket says 172.21.15.59:46574) 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.522+0000 7fa082000640 1 -- 172.21.15.59:0/3407969196 learned_addr learned my addr 172.21.15.59:0/3407969196 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa082000640 1 -- 172.21.15.59:0/3407969196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c1f9140 msgr2=0x7fa08c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa082000640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c1f9140 0x7fa08c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa082000640 1 -- 172.21.15.59:0/3407969196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0700039e0 msgr2=0x7fa08c144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa082000640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0700039e0 0x7fa08c144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa082000640 1 -- 172.21.15.59:0/3407969196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c142c70 con 0x7fa08c15b7d0 2024-04-23T14:32:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa082000640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa08c15b7d0 0x7fa08c06e9b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa07400e500 tx=0x7fa07400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa083e00640 1 -- 172.21.15.59:0/3407969196 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0740053a0 con 0x7fa08c15b7d0 2024-04-23T14:32:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa083e00640 1 -- 172.21.15.59:0/3407969196 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa074018070 con 0x7fa08c15b7d0 2024-04-23T14:32:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa083e00640 1 -- 172.21.15.59:0/3407969196 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074003810 con 0x7fa08c15b7d0 2024-04-23T14:32:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c142f50 con 0x7fa08c15b7d0 2024-04-23T14:32:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.523+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c143510 con 0x7fa08c15b7d0 2024-04-23T14:32:22.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.525+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c0d12a0 con 0x7fa08c15b7d0 2024-04-23T14:32:22.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.526+0000 7fa083e00640 1 -- 172.21.15.59:0/3407969196 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0740039b0 con 0x7fa08c15b7d0 2024-04-23T14:32:22.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.527+0000 7fa083e00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa06806c920 0x7fa06806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.527+0000 7fa083e00640 1 -- 172.21.15.59:0/3407969196 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa07408b8d0 con 0x7fa08c15b7d0 2024-04-23T14:32:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.527+0000 7fa08aa00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa06806c920 0x7fa06806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.528+0000 7fa08aa00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa06806c920 0x7fa06806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa07c001050 tx=0x7fa07c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.531+0000 7fa083e00640 1 -- 172.21.15.59:0/3407969196 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa08c0d12a0 con 0x7fa08c15b7d0 2024-04-23T14:32:22.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.736+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa08c145bb0 con 0x7fa08c15b7d0 2024-04-23T14:32:22.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.738+0000 7fa083e00640 1 -- 172.21.15.59:0/3407969196 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fa08c145bb0 con 0x7fa08c15b7d0 2024-04-23T14:32:22.739 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:22.740 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa06806c920 msgr2=0x7fa06806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa06806c920 0x7fa06806ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa07c001050 tx=0x7fa07c007000 comp rx=0 tx=0).stop 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa08c15b7d0 msgr2=0x7fa08c06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa08c15b7d0 0x7fa08c06e9b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa07400e500 tx=0x7fa07400e9c0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 shutdown_connections 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa06806c920 0x7fa06806ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa08c1f9140 0x7fa08c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa08c15b7d0 0x7fa08c06e9b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 --2- 172.21.15.59:0/3407969196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0700039e0 0x7fa08c144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 >> 172.21.15.59:0/3407969196 conn(0x7fa08c067ea0 msgr2=0x7fa08c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 shutdown_connections 2024-04-23T14:32:22.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.741+0000 7fa091e00640 1 -- 172.21.15.59:0/3407969196 wait complete. 2024-04-23T14:32:22.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:22.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:22.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.980+0000 7fe263e00640 1 Processor -- start 2024-04-23T14:32:22.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.980+0000 7fe263e00640 1 -- start start 2024-04-23T14:32:22.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe263e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2640d0d10 0x7fe264156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe263e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe26415d810 0x7fe264160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe263e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26415cab0 con 0x7fe2641567e0 2024-04-23T14:32:22.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe263e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe264160e40 con 0x7fe2640d0d10 2024-04-23T14:32:22.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe263e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe26415cd30 con 0x7fe26415d810 2024-04-23T14:32:22.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe262000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2640d0d10 0x7fe264156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe262000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2640d0d10 0x7fe264156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46584/0 (socket says 172.21.15.59:46584) 2024-04-23T14:32:22.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.982+0000 7fe262000640 1 -- 172.21.15.59:0/2846708119 learned_addr learned my addr 172.21.15.59:0/2846708119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:22.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.983+0000 7fe263400640 1 --2- 172.21.15.59:0/2846708119 >> v2:172.21.15.76:3300/0 conn(0x7fe26415d810 0x7fe264160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.983+0000 7fe262000640 1 -- 172.21.15.59:0/2846708119 >> v2:172.21.15.76:3300/0 conn(0x7fe26415d810 msgr2=0x7fe264160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:22.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.983+0000 7fe262000640 1 --2- 172.21.15.59:0/2846708119 >> v2:172.21.15.76:3300/0 conn(0x7fe26415d810 0x7fe264160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.983+0000 7fe262000640 1 -- 172.21.15.59:0/2846708119 >> v1:172.21.15.59:6789/0 conn(0x7fe2641567e0 legacy=0x7fe2640d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.983+0000 7fe262000640 1 -- 172.21.15.59:0/2846708119 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26415d010 con 0x7fe2640d0d10 2024-04-23T14:32:22.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.983+0000 7fe261600640 1 -- 172.21.15.59:0/2846708119 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604129239 0 0) 0x7fe254001110 con 0x7fe2641567e0 2024-04-23T14:32:22.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.983+0000 7fe263400640 1 --2- 172.21.15.59:0/2846708119 >> v2:172.21.15.76:3300/0 conn(0x7fe26415d810 0x7fe264160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:22.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.984+0000 7fe262000640 1 --2- 172.21.15.59:0/2846708119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe264156bc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2640b4cb0 tx=0x7fe25002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb4f212e23073ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.984+0000 7fe261600640 1 -- 172.21.15.59:0/2846708119 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250037070 con 0x7fe2640d0d10 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.984+0000 7fe261600640 1 -- 172.21.15.59:0/2846708119 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe250032440 con 0x7fe2640d0d10 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.984+0000 7fe261600640 1 -- 172.21.15.59:0/2846708119 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250032810 con 0x7fe2640d0d10 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 -- 172.21.15.59:0/2846708119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 msgr2=0x7fe264156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 --2- 172.21.15.59:0/2846708119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe264156bc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2640b4cb0 tx=0x7fe25002a270 comp rx=0 tx=0).stop 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 -- 172.21.15.59:0/2846708119 shutdown_connections 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 --2- 172.21.15.59:0/2846708119 >> v2:172.21.15.76:3300/0 conn(0x7fe26415d810 0x7fe264160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 --2- 172.21.15.59:0/2846708119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe264156bc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 -- 172.21.15.59:0/2846708119 >> 172.21.15.59:0/2846708119 conn(0x7fe26406bdf0 msgr2=0x7fe26406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 -- 172.21.15.59:0/2846708119 shutdown_connections 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.985+0000 7fe263e00640 1 -- 172.21.15.59:0/2846708119 wait complete. 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.987+0000 7fe263e00640 1 Processor -- start 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.987+0000 7fe263e00640 1 -- start start 2024-04-23T14:32:22.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe263e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe2641e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe263e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2641567e0 0x7fe2641595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe263e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe26415d810 0x7fe2641f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe263e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641607d0 con 0x7fe2640d0d10 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe263e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe264160650 con 0x7fe2641567e0 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe263e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe264160950 con 0x7fe26415d810 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe262a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe2641e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe262a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe2641e62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46592/0 (socket says 172.21.15.59:46592) 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe262a00640 1 -- 172.21.15.59:0/4023820598 learned_addr learned my addr 172.21.15.59:0/4023820598 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe262a00640 1 -- 172.21.15.59:0/4023820598 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe26415d810 msgr2=0x7fe2641f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.988+0000 7fe263400640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe26415d810 0x7fe2641f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe262a00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe26415d810 0x7fe2641f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe262a00640 1 -- 172.21.15.59:0/4023820598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2641567e0 msgr2=0x7fe2641595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe262a00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2641567e0 0x7fe2641595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:22.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe262a00640 1 -- 172.21.15.59:0/4023820598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2640b35e0 con 0x7fe2640d0d10 2024-04-23T14:32:22.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe262a00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe2641e62f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c780 tx=0x7fe25400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe25b400640 1 -- 172.21.15.59:0/4023820598 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254017070 con 0x7fe2640d0d10 2024-04-23T14:32:22.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe25b400640 1 -- 172.21.15.59:0/4023820598 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2540052b0 con 0x7fe2640d0d10 2024-04-23T14:32:22.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe25b400640 1 -- 172.21.15.59:0/4023820598 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25400a880 con 0x7fe2640d0d10 2024-04-23T14:32:22.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26406ef10 con 0x7fe2640d0d10 2024-04-23T14:32:22.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.989+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe264159af0 con 0x7fe2640d0d10 2024-04-23T14:32:22.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.990+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26404fd90 con 0x7fe2640d0d10 2024-04-23T14:32:22.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.991+0000 7fe25b400640 1 -- 172.21.15.59:0/4023820598 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe25401a070 con 0x7fe2640d0d10 2024-04-23T14:32:22.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.992+0000 7fe25b400640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe24406c8f0 0x7fe24406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:22.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.993+0000 7fe25b400640 1 -- 172.21.15.59:0/4023820598 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe25408aea0 con 0x7fe2640d0d10 2024-04-23T14:32:22.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.993+0000 7fe262000640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe24406c8f0 0x7fe24406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:22.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.994+0000 7fe262000640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe24406c8f0 0x7fe24406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe250034000 tx=0x7fe25002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:22.997+0000 7fe25b400640 1 -- 172.21.15.59:0/4023820598 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe26415cab0 con 0x7fe2640d0d10 2024-04-23T14:32:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.200+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe2640b5150 con 0x7fe2640d0d10 2024-04-23T14:32:23.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.201+0000 7fe25b400640 1 -- 172.21.15.59:0/4023820598 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fe254058bf0 con 0x7fe2640d0d10 2024-04-23T14:32:23.202 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.203+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe24406c8f0 msgr2=0x7fe24406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.203+0000 7fe263e00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe24406c8f0 0x7fe24406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe250034000 tx=0x7fe25002c000 comp rx=0 tx=0).stop 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.203+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 msgr2=0x7fe2641e62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.203+0000 7fe263e00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe2641e62f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c780 tx=0x7fe25400cc40 comp rx=0 tx=0).stop 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 shutdown_connections 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe24406c8f0 0x7fe24406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe26415d810 0x7fe2641f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2641567e0 0x7fe2641595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 --2- 172.21.15.59:0/4023820598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2640d0d10 0x7fe2641e62f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 >> 172.21.15.59:0/4023820598 conn(0x7fe26406bdf0 msgr2=0x7fe264072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 shutdown_connections 2024-04-23T14:32:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.204+0000 7fe263e00640 1 -- 172.21.15.59:0/4023820598 wait complete. 2024-04-23T14:32:23.224 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:23.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.432+0000 7fc839600640 1 Processor -- start 2024-04-23T14:32:23.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.432+0000 7fc839600640 1 -- start start 2024-04-23T14:32:23.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.433+0000 7fc839600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc83415c790 0x7fc8340cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.433+0000 7fc839600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340cdec0 0x7fc8340b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.433+0000 7fc839600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83409b3c0 con 0x7fc8340b1890 2024-04-23T14:32:23.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.433+0000 7fc839600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc8340b1d70 con 0x7fc83415c790 2024-04-23T14:32:23.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.433+0000 7fc839600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc83409b660 con 0x7fc8340cdec0 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.434+0000 7fc833e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc83415c790 0x7fc8340cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.434+0000 7fc838c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340cdec0 0x7fc8340b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.434+0000 7fc833e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc83415c790 0x7fc8340cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:45844/0 (socket says 172.21.15.59:45844) 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.434+0000 7fc833e00640 1 -- 172.21.15.59:0/334553110 learned_addr learned my addr 172.21.15.59:0/334553110 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.435+0000 7fc832a00640 1 -- 172.21.15.59:0/334553110 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790937566 0 0) 0x7fc83409b3c0 con 0x7fc8340b1890 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.435+0000 7fc832a00640 1 -- 172.21.15.59:0/334553110 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc818003520 con 0x7fc8340b1890 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.435+0000 7fc838c00640 1 -- 172.21.15.59:0/334553110 >> v2:172.21.15.76:3300/0 conn(0x7fc83415c790 msgr2=0x7fc8340cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:23.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.435+0000 7fc838c00640 1 --2- 172.21.15.59:0/334553110 >> v2:172.21.15.76:3300/0 conn(0x7fc83415c790 0x7fc8340cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.435+0000 7fc838c00640 1 -- 172.21.15.59:0/334553110 >> v1:172.21.15.76:6789/0 conn(0x7fc8340b1890 legacy=0x7fc8340d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.435+0000 7fc838c00640 1 -- 172.21.15.59:0/334553110 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83409b940 con 0x7fc8340cdec0 2024-04-23T14:32:23.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.435+0000 7fc833e00640 1 --2- 172.21.15.59:0/334553110 >> v2:172.21.15.76:3300/0 conn(0x7fc83415c790 0x7fc8340cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:23.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.436+0000 7fc838c00640 1 --2- 172.21.15.59:0/334553110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340cdec0 0x7fc8340b2200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097a0 tx=0x7fc82c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5457c9e74deef0a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.437+0000 7fc832a00640 1 -- 172.21.15.59:0/334553110 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c037070 con 0x7fc8340cdec0 2024-04-23T14:32:23.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.437+0000 7fc832a00640 1 -- 172.21.15.59:0/334553110 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c032440 con 0x7fc8340cdec0 2024-04-23T14:32:23.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.437+0000 7fc832a00640 1 -- 172.21.15.59:0/334553110 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c032810 con 0x7fc8340cdec0 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 -- 172.21.15.59:0/334553110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340cdec0 msgr2=0x7fc8340b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 --2- 172.21.15.59:0/334553110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340cdec0 0x7fc8340b2200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097a0 tx=0x7fc82c02a180 comp rx=0 tx=0).stop 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 -- 172.21.15.59:0/334553110 shutdown_connections 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 --2- 172.21.15.59:0/334553110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340cdec0 0x7fc8340b2200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 --2- 172.21.15.59:0/334553110 >> v2:172.21.15.76:3300/0 conn(0x7fc83415c790 0x7fc8340cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 -- 172.21.15.59:0/334553110 >> 172.21.15.59:0/334553110 conn(0x7fc8340b3f00 msgr2=0x7fc8340b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 -- 172.21.15.59:0/334553110 shutdown_connections 2024-04-23T14:32:23.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.438+0000 7fc839600640 1 -- 172.21.15.59:0/334553110 wait complete. 2024-04-23T14:32:23.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.440+0000 7fc839600640 1 Processor -- start 2024-04-23T14:32:23.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.440+0000 7fc839600640 1 -- start start 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc839600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340b1890 0x7fc8340c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc839600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340cdec0 0x7fc83415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc839600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc83415c790 0x7fc834155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc839600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8340d0c30 con 0x7fc83415c790 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc839600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc83415cb70 con 0x7fc8340cdec0 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc839600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc8340d0db0 con 0x7fc8340b1890 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc838c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc83415c790 0x7fc834155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc838c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc83415c790 0x7fc834155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46600/0 (socket says 172.21.15.59:46600) 2024-04-23T14:32:23.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc838c00640 1 -- 172.21.15.59:0/1553936877 learned_addr learned my addr 172.21.15.59:0/1553936877 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc833e00640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340b1890 0x7fc8340c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.442+0000 7fc833400640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340cdec0 0x7fc83415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc838c00640 1 -- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340b1890 msgr2=0x7fc8340c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc838c00640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340b1890 0x7fc8340c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc838c00640 1 -- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340cdec0 msgr2=0x7fc83415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc838c00640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340cdec0 0x7fc83415d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc838c00640 1 -- 172.21.15.59:0/1553936877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83406ce70 con 0x7fc83415c790 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc833400640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340cdec0 0x7fc83415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc838c00640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc83415c790 0x7fc834155650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc82c035040 tx=0x7fc82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc82be00640 1 -- 172.21.15.59:0/1553936877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c041020 con 0x7fc83415c790 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc82be00640 1 -- 172.21.15.59:0/1553936877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c03d640 con 0x7fc83415c790 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.443+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83406d150 con 0x7fc83415c790 2024-04-23T14:32:23.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.444+0000 7fc82be00640 1 -- 172.21.15.59:0/1553936877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c037040 con 0x7fc83415c790 2024-04-23T14:32:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.444+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83406d710 con 0x7fc83415c790 2024-04-23T14:32:23.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.445+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83404fd90 con 0x7fc83415c790 2024-04-23T14:32:23.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.446+0000 7fc82be00640 1 -- 172.21.15.59:0/1553936877 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc82c044020 con 0x7fc83415c790 2024-04-23T14:32:23.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.447+0000 7fc82be00640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.447+0000 7fc82be00640 1 -- 172.21.15.59:0/1553936877 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc82c0ac2e0 con 0x7fc83415c790 2024-04-23T14:32:23.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.447+0000 7fc833e00640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.448+0000 7fc833e00640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc81006c8f0 0x7fc81006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc824009ce0 tx=0x7fc82400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.453+0000 7fc82be00640 1 -- 172.21.15.59:0/1553936877 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc82c04b030 con 0x7fc83415c790 2024-04-23T14:32:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.658+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc834068c90 con 0x7fc83415c790 2024-04-23T14:32:23.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.659+0000 7fc82be00640 1 -- 172.21.15.59:0/1553936877 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fc82c07a6a0 con 0x7fc83415c790 2024-04-23T14:32:23.660 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc81006c8f0 msgr2=0x7fc81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc81006c8f0 0x7fc81006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc824009ce0 tx=0x7fc82400b040 comp rx=0 tx=0).stop 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc83415c790 msgr2=0x7fc834155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc83415c790 0x7fc834155650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc82c035040 tx=0x7fc82c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 shutdown_connections 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc81006c8f0 0x7fc81006edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc83415c790 0x7fc834155650 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340cdec0 0x7fc83415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 --2- 172.21.15.59:0/1553936877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8340b1890 0x7fc8340c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.661+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 >> 172.21.15.59:0/1553936877 conn(0x7fc8340b3f00 msgr2=0x7fc8340d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.662+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 shutdown_connections 2024-04-23T14:32:23.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.662+0000 7fc839600640 1 -- 172.21.15.59:0/1553936877 wait complete. 2024-04-23T14:32:23.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.897+0000 7f8ec9c00640 1 Processor -- start 2024-04-23T14:32:23.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.897+0000 7f8ec9c00640 1 -- start start 2024-04-23T14:32:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.899+0000 7f8ec9c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ec4145e10 0x7f8ec406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.899+0000 7f8ec9c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec40cfc20 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.899+0000 7f8ec9c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec40cff00 con 0x7f8ec4071d70 2024-04-23T14:32:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.899+0000 7f8ec9c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec40d01e0 con 0x7f8ec4145e10 2024-04-23T14:32:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.899+0000 7f8ec2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ec4145e10 0x7f8ec406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.899+0000 7f8ec2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8ec40d0cc0 0x7f8ec40cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46576/0 (socket says 172.21.15.59:46576) 2024-04-23T14:32:23.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.899+0000 7f8ec2000640 1 -- 172.21.15.59:0/329566978 learned_addr learned my addr 172.21.15.59:0/329566978 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:23.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.900+0000 7f8ec1600640 1 -- 172.21.15.59:0/329566978 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160361799 0 0) 0x7f8ec40cfc20 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.900+0000 7f8ec1600640 1 -- 172.21.15.59:0/329566978 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea8003520 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.900+0000 7f8ec2a00640 1 -- 172.21.15.59:0/329566978 >> v1:172.21.15.76:6789/0 conn(0x7f8ec4071d70 legacy=0x7f8ec415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:23.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.900+0000 7f8ec2a00640 1 -- 172.21.15.59:0/329566978 >> v1:172.21.15.59:6789/0 conn(0x7f8ec40d0cc0 legacy=0x7f8ec40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.900+0000 7f8ec2a00640 1 -- 172.21.15.59:0/329566978 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec40d04c0 con 0x7f8ec4145e10 2024-04-23T14:32:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.901+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/329566978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec4145e10 0x7f8ec406f130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40097d0 tx=0x7f8eb402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe2164860c447f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.902+0000 7f8ec1600640 1 -- 172.21.15.59:0/329566978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4037070 con 0x7f8ec4145e10 2024-04-23T14:32:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.902+0000 7f8ec1600640 1 -- 172.21.15.59:0/329566978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4032440 con 0x7f8ec4145e10 2024-04-23T14:32:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.902+0000 7f8ec1600640 1 -- 172.21.15.59:0/329566978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4032810 con 0x7f8ec4145e10 2024-04-23T14:32:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.902+0000 7f8ec9c00640 1 -- 172.21.15.59:0/329566978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec4145e10 msgr2=0x7f8ec406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.902+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/329566978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec4145e10 0x7f8ec406f130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40097d0 tx=0x7f8eb402a210 comp rx=0 tx=0).stop 2024-04-23T14:32:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.903+0000 7f8ec9c00640 1 -- 172.21.15.59:0/329566978 shutdown_connections 2024-04-23T14:32:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.903+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/329566978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec4145e10 0x7f8ec406f130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.903+0000 7f8ec9c00640 1 -- 172.21.15.59:0/329566978 >> 172.21.15.59:0/329566978 conn(0x7f8ec4067d90 msgr2=0x7f8ec406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.903+0000 7f8ec9c00640 1 -- 172.21.15.59:0/329566978 shutdown_connections 2024-04-23T14:32:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.903+0000 7f8ec9c00640 1 -- 172.21.15.59:0/329566978 wait complete. 2024-04-23T14:32:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.904+0000 7f8ec9c00640 1 Processor -- start 2024-04-23T14:32:23.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.904+0000 7f8ec9c00640 1 -- start start 2024-04-23T14:32:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.905+0000 7f8ec9c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec41e7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.905+0000 7f8ec9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec41e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.905+0000 7f8ec9c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.905+0000 7f8ec9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec41461f0 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.905+0000 7f8ec9c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4072200 con 0x7f8ec4145e10 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.905+0000 7f8ec9c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec41e5fa0 con 0x7f8ec4071d70 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec41e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec41e7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46606/0 (socket says 172.21.15.59:46606) 2024-04-23T14:32:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 -- 172.21.15.59:0/1805879369 learned_addr learned my addr 172.21.15.59:0/1805879369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec41e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec3400640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 -- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 msgr2=0x7f8ec41e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec41e7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 -- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 msgr2=0x7f8ec41e5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2000640 1 -- 172.21.15.59:0/1805879369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec41e6280 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.906+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec41e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.907+0000 7f8ec2000640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec41e7d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000c430 tx=0x7f8eb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.907+0000 7f8ebaa00640 1 -- 172.21.15.59:0/1805879369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0017070 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.907+0000 7f8ebaa00640 1 -- 172.21.15.59:0/1805879369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb0005ce0 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.907+0000 7f8ebaa00640 1 -- 172.21.15.59:0/1805879369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00072f0 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.907+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec41e6560 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.907+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec41e82d0 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.908+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec4068a80 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.909+0000 7f8ebaa00640 1 -- 172.21.15.59:0/1805879369 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eb001a070 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.910+0000 7f8ebaa00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea406c8f0 0x7f8ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.910+0000 7f8ebaa00640 1 -- 172.21.15.59:0/1805879369 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8eb008b130 con 0x7f8ec40d0cc0 2024-04-23T14:32:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.911+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea406c8f0 0x7f8ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.911+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea406c8f0 0x7f8ea406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4034000 tx=0x7f8eb402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:23.914+0000 7f8ebaa00640 1 -- 172.21.15.59:0/1805879369 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8eb0059440 con 0x7f8ec40d0cc0 2024-04-23T14:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.120+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ec406e600 con 0x7f8ec40d0cc0 2024-04-23T14:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.121+0000 7f8ebaa00640 1 -- 172.21.15.59:0/1805879369 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f8eb0058d40 con 0x7f8ec40d0cc0 2024-04-23T14:32:24.122 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea406c8f0 msgr2=0x7f8ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea406c8f0 0x7f8ea406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4034000 tx=0x7f8eb402e000 comp rx=0 tx=0).stop 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 msgr2=0x7f8ec41e7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec41e7d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000c430 tx=0x7f8eb000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 shutdown_connections 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea406c8f0 0x7f8ea406edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec41e7d90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 --2- 172.21.15.59:0/1805879369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec41e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 >> 172.21.15.59:0/1805879369 conn(0x7f8ec4067d90 msgr2=0x7f8ec40a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.123+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 shutdown_connections 2024-04-23T14:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.124+0000 7f8ec9c00640 1 -- 172.21.15.59:0/1805879369 wait complete. 2024-04-23T14:32:24.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:24.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.356+0000 7f3279000640 1 Processor -- start 2024-04-23T14:32:24.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.356+0000 7f3279000640 1 -- start start 2024-04-23T14:32:24.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.358+0000 7f3279000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f327415e950 0x7f32740d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.358+0000 7f3279000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32740d2070 con 0x7f32740d1c50 2024-04-23T14:32:24.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.358+0000 7f3279000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32740b91e0 con 0x7f32740ba470 2024-04-23T14:32:24.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.358+0000 7f3279000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f32740b94c0 con 0x7f327415e950 2024-04-23T14:32:24.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.358+0000 7f3272800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f32740d1c50 0x7f32740b87e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46586/0 (socket says 172.21.15.59:46586) 2024-04-23T14:32:24.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.358+0000 7f3272800640 1 -- 172.21.15.59:0/2147163687 learned_addr learned my addr 172.21.15.59:0/2147163687 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:24.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.358+0000 7f3271e00640 1 --2- 172.21.15.59:0/2147163687 >> v2:172.21.15.76:3300/0 conn(0x7f327415e950 0x7f32740d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 820374060 0 0) 0x7f32740d2070 con 0x7f32740d1c50 2024-04-23T14:32:24.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3258003520 con 0x7f32740d1c50 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3726544706 0 0) 0x7f3258003520 con 0x7f32740d1c50 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 >> v2:172.21.15.76:3300/0 conn(0x7f327415e950 msgr2=0x7f32740d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 --2- 172.21.15.59:0/2147163687 >> v2:172.21.15.76:3300/0 conn(0x7f327415e950 0x7f32740d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 >> v1:172.21.15.76:6789/0 conn(0x7f32740ba470 legacy=0x7f32740d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32740b97a0 con 0x7f32740d1c50 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f3264001670 con 0x7f32740d1c50 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 >> v1:172.21.15.59:6789/0 conn(0x7f32740d1c50 legacy=0x7f32740b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32580039b0 0x7f3258003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3258004800 0x7f3258004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580051c0 0x7f325800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32740b94c0 con 0x7f3258004800 2024-04-23T14:32:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f325800c060 con 0x7f32580051c0 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.359+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f325800c260 con 0x7f32580039b0 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3271400640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3258004800 0x7f3258004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3272800640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580051c0 0x7f325800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3271e00640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32580039b0 0x7f3258003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3271400640 1 -- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32580039b0 msgr2=0x7f3258003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3271400640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32580039b0 0x7f3258003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3271400640 1 -- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580051c0 msgr2=0x7f325800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3271400640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580051c0 0x7f325800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3271400640 1 -- 172.21.15.59:0/2147163687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3258003520 con 0x7f3258004800 2024-04-23T14:32:24.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.360+0000 7f3272800640 1 -- 172.21.15.59:0/2147163687 reap_dead start 2024-04-23T14:32:24.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.361+0000 7f3271400640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3258004800 0x7f3258004c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f325c001750 tx=0x7f3258005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d203b91e9f1f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.362+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c001370 con 0x7f3258004800 2024-04-23T14:32:24.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.362+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c0052d0 con 0x7f3258004800 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.362+0000 7f3270a00640 1 -- 172.21.15.59:0/2147163687 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c02abc0 con 0x7f3258004800 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.362+0000 7f3279000640 1 -- 172.21.15.59:0/2147163687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3258004800 msgr2=0x7f3258004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.362+0000 7f3279000640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3258004800 0x7f3258004c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f325c001750 tx=0x7f3258005830 comp rx=0 tx=0).stop 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.363+0000 7f3279000640 1 -- 172.21.15.59:0/2147163687 shutdown_connections 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.363+0000 7f3279000640 1 --2- 172.21.15.59:0/2147163687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3258004800 0x7f3258004c80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.363+0000 7f3279000640 1 -- 172.21.15.59:0/2147163687 >> 172.21.15.59:0/2147163687 conn(0x7f3274067b80 msgr2=0x7f3274067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.363+0000 7f3279000640 1 -- 172.21.15.59:0/2147163687 shutdown_connections 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.363+0000 7f3279000640 1 -- 172.21.15.59:0/2147163687 wait complete. 2024-04-23T14:32:24.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.364+0000 7f3279000640 1 Processor -- start 2024-04-23T14:32:24.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.365+0000 7f3279000640 1 -- start start 2024-04-23T14:32:24.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3279000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3258004800 0x7f32741e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3279000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32740cd5d0 0x7f32740ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3279000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3274160890 0x7f3274160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3279000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f327415ed30 con 0x7f32740cd5d0 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3279000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32740d27a0 con 0x7f3258004800 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3279000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32740d1710 con 0x7f3274160890 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32740cd5d0 0x7f32740ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32740cd5d0 0x7f32740ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46616/0 (socket says 172.21.15.59:46616) 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 -- 172.21.15.59:0/3150455643 learned_addr learned my addr 172.21.15.59:0/3150455643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3272800640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3274160890 0x7f3274160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271e00640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3258004800 0x7f32741e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 -- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3274160890 msgr2=0x7f3274160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3274160890 0x7f3274160d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 -- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3258004800 msgr2=0x7f32741e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3258004800 0x7f32741e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271400640 1 -- 172.21.15.59:0/3150455643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32741f5050 con 0x7f32740cd5d0 2024-04-23T14:32:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.366+0000 7f3271e00640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3258004800 0x7f32741e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:24.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.367+0000 7f3271400640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32740cd5d0 0x7f32740ca250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f325c001870 tx=0x7f325c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.367+0000 7f326a000640 1 -- 172.21.15.59:0/3150455643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c03f030 con 0x7f32740cd5d0 2024-04-23T14:32:24.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.367+0000 7f326a000640 1 -- 172.21.15.59:0/3150455643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c005480 con 0x7f32740cd5d0 2024-04-23T14:32:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.367+0000 7f326a000640 1 -- 172.21.15.59:0/3150455643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c037040 con 0x7f32740cd5d0 2024-04-23T14:32:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.367+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32741f5330 con 0x7f32740cd5d0 2024-04-23T14:32:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.367+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32741f58f0 con 0x7f32740cd5d0 2024-04-23T14:32:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.368+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32740b87e0 con 0x7f32740cd5d0 2024-04-23T14:32:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.369+0000 7f326a000640 1 -- 172.21.15.59:0/3150455643 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f325c042070 con 0x7f32740cd5d0 2024-04-23T14:32:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.370+0000 7f326a000640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.371+0000 7f326a000640 1 -- 172.21.15.59:0/3150455643 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f325c007640 con 0x7f32740cd5d0 2024-04-23T14:32:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.371+0000 7f3271e00640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.371+0000 7f3271e00640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f326c007460 tx=0x7f326c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.375+0000 7f326a000640 1 -- 172.21.15.59:0/3150455643 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32740b87e0 con 0x7f32740cd5d0 2024-04-23T14:32:24.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.582+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3274068a80 con 0x7f32740cd5d0 2024-04-23T14:32:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.583+0000 7f326a000640 1 -- 172.21.15.59:0/3150455643 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f325c0796a0 con 0x7f32740cd5d0 2024-04-23T14:32:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f325006c8f0 msgr2=0x7f325006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f326c007460 tx=0x7f326c00b040 comp rx=0 tx=0).stop 2024-04-23T14:32:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32740cd5d0 msgr2=0x7f32740ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32740cd5d0 0x7f32740ca250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f325c001870 tx=0x7f325c001120 comp rx=0 tx=0).stop 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 shutdown_connections 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3274160890 0x7f3274160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32740cd5d0 0x7f32740ca250 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 --2- 172.21.15.59:0/3150455643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3258004800 0x7f32741e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 >> 172.21.15.59:0/3150455643 conn(0x7f3274067b80 msgr2=0x7f32740d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 shutdown_connections 2024-04-23T14:32:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.586+0000 7f3279000640 1 -- 172.21.15.59:0/3150455643 wait complete. 2024-04-23T14:32:24.605 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:32:24.606 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:24.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:24.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.863+0000 7f0e8b600640 1 Processor -- start 2024-04-23T14:32:24.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.863+0000 7f0e8b600640 1 -- start start 2024-04-23T14:32:24.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.865+0000 7f0e8b600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0e84160960 0x7f0e84163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.865+0000 7f0e8b600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e840b4550 0x7f0e8406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.865+0000 7f0e8b600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e84160d40 con 0x7f0e841644b0 2024-04-23T14:32:24.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.865+0000 7f0e8b600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0e84164270 con 0x7f0e84160960 2024-04-23T14:32:24.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.865+0000 7f0e8b600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0e84166de0 con 0x7f0e840b4550 2024-04-23T14:32:24.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.866+0000 7f0e88c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0e84160960 0x7f0e84163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.866+0000 7f0e89600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e840b4550 0x7f0e8406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.866+0000 7f0e88c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0e84160960 0x7f0e84163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:45888/0 (socket says 172.21.15.59:45888) 2024-04-23T14:32:24.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.866+0000 7f0e88c00640 1 -- 172.21.15.59:0/90079528 learned_addr learned my addr 172.21.15.59:0/90079528 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:24.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.867+0000 7f0e89600640 1 -- 172.21.15.59:0/90079528 >> v2:172.21.15.76:3300/0 conn(0x7f0e84160960 msgr2=0x7f0e84163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:24.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.867+0000 7f0e89600640 1 --2- 172.21.15.59:0/90079528 >> v2:172.21.15.76:3300/0 conn(0x7f0e84160960 0x7f0e84163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.867+0000 7f0e89600640 1 -- 172.21.15.59:0/90079528 >> v1:172.21.15.76:6789/0 conn(0x7f0e841644b0 legacy=0x7f0e840b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.867+0000 7f0e89600640 1 -- 172.21.15.59:0/90079528 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e841670c0 con 0x7f0e840b4550 2024-04-23T14:32:24.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.867+0000 7f0e88c00640 1 --2- 172.21.15.59:0/90079528 >> v2:172.21.15.76:3300/0 conn(0x7f0e84160960 0x7f0e84163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:24.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.868+0000 7f0e89600640 1 --2- 172.21.15.59:0/90079528 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e840b4550 0x7f0e8406e8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009d90 tx=0x7f0e7c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78db96674dacac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e83400640 1 -- 172.21.15.59:0/90079528 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c007740 con 0x7f0e840b4550 2024-04-23T14:32:24.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e83400640 1 -- 172.21.15.59:0/90079528 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7c0078e0 con 0x7f0e840b4550 2024-04-23T14:32:24.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e83400640 1 -- 172.21.15.59:0/90079528 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c007cb0 con 0x7f0e840b4550 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e8b600640 1 -- 172.21.15.59:0/90079528 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e840b4550 msgr2=0x7f0e8406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e8b600640 1 --2- 172.21.15.59:0/90079528 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e840b4550 0x7f0e8406e8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009d90 tx=0x7f0e7c02a660 comp rx=0 tx=0).stop 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e8b600640 1 -- 172.21.15.59:0/90079528 shutdown_connections 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e8b600640 1 --2- 172.21.15.59:0/90079528 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e840b4550 0x7f0e8406e8d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.869+0000 7f0e8b600640 1 --2- 172.21.15.59:0/90079528 >> v2:172.21.15.76:3300/0 conn(0x7f0e84160960 0x7f0e84163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.870+0000 7f0e8b600640 1 -- 172.21.15.59:0/90079528 >> 172.21.15.59:0/90079528 conn(0x7f0e84067ea0 msgr2=0x7f0e8406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.870+0000 7f0e8b600640 1 -- 172.21.15.59:0/90079528 shutdown_connections 2024-04-23T14:32:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.870+0000 7f0e8b600640 1 -- 172.21.15.59:0/90079528 wait complete. 2024-04-23T14:32:24.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.872+0000 7f0e8b600640 1 Processor -- start 2024-04-23T14:32:24.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.872+0000 7f0e8b600640 1 -- start start 2024-04-23T14:32:24.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e8b600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840b4550 0x7f0e84158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e8b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84160960 0x7f0e840d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e8b600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e841644b0 0x7f0e8415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e8b600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841668e0 con 0x7f0e84160960 2024-04-23T14:32:24.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e8b600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84167680 con 0x7f0e840b4550 2024-04-23T14:32:24.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e8b600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0e84166a60 con 0x7f0e841644b0 2024-04-23T14:32:24.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84160960 0x7f0e840d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84160960 0x7f0e840d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46624/0 (socket says 172.21.15.59:46624) 2024-04-23T14:32:24.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e83e00640 1 -- 172.21.15.59:0/2671436561 learned_addr learned my addr 172.21.15.59:0/2671436561 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:24.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e89600640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e841644b0 0x7f0e8415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e83e00640 1 -- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e841644b0 msgr2=0x7f0e8415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e83e00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e841644b0 0x7f0e8415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.873+0000 7f0e88c00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840b4550 0x7f0e84158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e83e00640 1 -- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840b4550 msgr2=0x7f0e84158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e83e00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840b4550 0x7f0e84158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e83e00640 1 -- 172.21.15.59:0/2671436561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e841ec670 con 0x7f0e84160960 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e88c00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840b4550 0x7f0e84158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e83e00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84160960 0x7f0e840d0650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c400 tx=0x7f0e6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e80c00640 1 -- 172.21.15.59:0/2671436561 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c017070 con 0x7f0e84160960 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e80c00640 1 -- 172.21.15.59:0/2671436561 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0052d0 con 0x7f0e84160960 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e80c00640 1 -- 172.21.15.59:0/2671436561 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c003830 con 0x7f0e84160960 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e841ec950 con 0x7f0e84160960 2024-04-23T14:32:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.874+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e841ecf10 con 0x7f0e84160960 2024-04-23T14:32:24.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.875+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e84068a80 con 0x7f0e84160960 2024-04-23T14:32:24.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.877+0000 7f0e80c00640 1 -- 172.21.15.59:0/2671436561 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e6c01a070 con 0x7f0e84160960 2024-04-23T14:32:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.878+0000 7f0e80c00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c8f0 0x7f0e6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.878+0000 7f0e80c00640 1 -- 172.21.15.59:0/2671436561 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0e6c08af10 con 0x7f0e84160960 2024-04-23T14:32:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.879+0000 7f0e88c00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c8f0 0x7f0e6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.879+0000 7f0e88c00640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c8f0 0x7f0e6006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009630 tx=0x7f0e74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:24.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:24.883+0000 7f0e80c00640 1 -- 172.21.15.59:0/2671436561 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e6c0592c0 con 0x7f0e84160960 2024-04-23T14:32:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.085+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e8406bde0 con 0x7f0e84160960 2024-04-23T14:32:25.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.086+0000 7f0e80c00640 1 -- 172.21.15.59:0/2671436561 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7f0e6c058bc0 con 0x7f0e84160960 2024-04-23T14:32:25.088 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:25.088 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:25.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c8f0 msgr2=0x7f0e6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c8f0 0x7f0e6006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009630 tx=0x7f0e74008040 comp rx=0 tx=0).stop 2024-04-23T14:32:25.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84160960 msgr2=0x7f0e840d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84160960 0x7f0e840d0650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c400 tx=0x7f0e6c00c8c0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 shutdown_connections 2024-04-23T14:32:25.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c8f0 0x7f0e6006edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e841644b0 0x7f0e8415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e84160960 0x7f0e840d0650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 --2- 172.21.15.59:0/2671436561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840b4550 0x7f0e84158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 >> 172.21.15.59:0/2671436561 conn(0x7f0e84067ea0 msgr2=0x7f0e840b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 shutdown_connections 2024-04-23T14:32:25.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.089+0000 7f0e8b600640 1 -- 172.21.15.59:0/2671436561 wait complete. 2024-04-23T14:32:25.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:25.108 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.309+0000 7f6677e00640 1 Processor -- start 2024-04-23T14:32:25.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.309+0000 7f6677e00640 1 -- start start 2024-04-23T14:32:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.310+0000 7f6677e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6678155a80 0x7f667806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.310+0000 7f6677e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 0x7f66780d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.310+0000 7f6677e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667815ef20 con 0x7f66780b43a0 2024-04-23T14:32:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.310+0000 7f6677e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f66780d0fa0 con 0x7f6678155a80 2024-04-23T14:32:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.310+0000 7f6677e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f667815f1c0 con 0x7f66780d0600 2024-04-23T14:32:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.311+0000 7f6677400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f66780b43a0 0x7f66780b4830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46592/0 (socket says 172.21.15.59:46592) 2024-04-23T14:32:25.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.311+0000 7f6677400640 1 -- 172.21.15.59:0/1577655800 learned_addr learned my addr 172.21.15.59:0/1577655800 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.311+0000 7f6676a00640 1 --2- 172.21.15.59:0/1577655800 >> v2:172.21.15.76:3300/0 conn(0x7f6678155a80 0x7f667806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.311+0000 7f666fe00640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 0x7f66780d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.311+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701833373 0 0) 0x7f667815ef20 con 0x7f66780b43a0 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.311+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f665c003520 con 0x7f66780b43a0 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 801905700 0 0) 0x7f665c003520 con 0x7f66780b43a0 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 msgr2=0x7f66780d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 0x7f66780d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 >> v2:172.21.15.76:3300/0 conn(0x7f6678155a80 msgr2=0x7f667806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 --2- 172.21.15.59:0/1577655800 >> v2:172.21.15.76:3300/0 conn(0x7f6678155a80 0x7f667806f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667815f4a0 con 0x7f66780b43a0 2024-04-23T14:32:25.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f66700019f0 con 0x7f66780b43a0 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 >> v1:172.21.15.59:6789/0 conn(0x7f66780b43a0 legacy=0x7f66780b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 0x7f665c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c003f30 0x7f665c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f665c004d40 0x7f665c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f667815f1c0 con 0x7f665c004d40 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f665c00dbd0 con 0x7f665c003f30 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f665c00dd50 con 0x7f66780d0600 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6677400640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f665c004d40 0x7f665c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.312+0000 7f6676a00640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 0x7f665c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.313+0000 7f666fe00640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c003f30 0x7f665c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.313+0000 7f6677400640 1 -- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 msgr2=0x7f665c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.313+0000 7f6677400640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780d0600 0x7f665c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.313+0000 7f6677400640 1 -- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c003f30 msgr2=0x7f665c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.313+0000 7f6677400640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c003f30 0x7f665c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.313+0000 7f6677400640 1 -- 172.21.15.59:0/1577655800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c003520 con 0x7f665c004d40 2024-04-23T14:32:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.313+0000 7f666fe00640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c003f30 0x7f665c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:25.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6677400640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f665c004d40 0x7f665c0051c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f667000a8a0 tx=0x7f665c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68449d81c0707f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667000a4c0 con 0x7f665c004d40 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667002f070 con 0x7f665c004d40 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6676000640 1 -- 172.21.15.59:0/1577655800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667002bd50 con 0x7f665c004d40 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6677e00640 1 -- 172.21.15.59:0/1577655800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f665c004d40 msgr2=0x7f665c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6677e00640 1 --2- 172.21.15.59:0/1577655800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f665c004d40 0x7f665c0051c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f667000a8a0 tx=0x7f665c005830 comp rx=0 tx=0).stop 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6677400640 1 -- 172.21.15.59:0/1577655800 reap_dead start 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.314+0000 7f6677e00640 1 -- 172.21.15.59:0/1577655800 shutdown_connections 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.315+0000 7f6677e00640 1 -- 172.21.15.59:0/1577655800 >> 172.21.15.59:0/1577655800 conn(0x7f667806bdf0 msgr2=0x7f667806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.315+0000 7f6677e00640 1 -- 172.21.15.59:0/1577655800 shutdown_connections 2024-04-23T14:32:25.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.315+0000 7f6677e00640 1 -- 172.21.15.59:0/1577655800 wait complete. 2024-04-23T14:32:25.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.316+0000 7f6677e00640 1 Processor -- start 2024-04-23T14:32:25.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.316+0000 7f6677e00640 1 -- start start 2024-04-23T14:32:25.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.317+0000 7f6677e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c004d40 0x7f66780cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.317+0000 7f6677e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66781f3d10 0x7f66780b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.317+0000 7f6677e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780b17c0 0x7f667815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.317+0000 7f6677e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f667815fd40 con 0x7f66781f3d10 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.317+0000 7f6677e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f667815fa60 con 0x7f665c004d40 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.317+0000 7f6677e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f667806ff00 con 0x7f66780b17c0 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66781f3d10 0x7f66780b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66781f3d10 0x7f66780b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46644/0 (socket says 172.21.15.59:46644) 2024-04-23T14:32:25.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 -- 172.21.15.59:0/1059543517 learned_addr learned my addr 172.21.15.59:0/1059543517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:25.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f6676a00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c004d40 0x7f66780cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f6677400640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780b17c0 0x7f667815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 -- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780b17c0 msgr2=0x7f667815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780b17c0 0x7f667815ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 -- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c004d40 msgr2=0x7f66780cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c004d40 0x7f66780cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f666fe00640 1 -- 172.21.15.59:0/1059543517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66780cf720 con 0x7f66781f3d10 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.318+0000 7f6676a00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c004d40 0x7f66780cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.319+0000 7f666fe00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66781f3d10 0x7f66780b1280 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6660010890 tx=0x7f6660010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.319+0000 7f666ea00640 1 -- 172.21.15.59:0/1059543517 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666000f040 con 0x7f66781f3d10 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.319+0000 7f666ea00640 1 -- 172.21.15.59:0/1059543517 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666000eb00 con 0x7f66781f3d10 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.319+0000 7f666ea00640 1 -- 172.21.15.59:0/1059543517 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666001c400 con 0x7f66781f3d10 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.319+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667815c740 con 0x7f66781f3d10 2024-04-23T14:32:25.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.319+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66780ce9e0 con 0x7f66781f3d10 2024-04-23T14:32:25.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.320+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667804fd90 con 0x7f66781f3d10 2024-04-23T14:32:25.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.321+0000 7f666ea00640 1 -- 172.21.15.59:0/1059543517 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f666001c5a0 con 0x7f66781f3d10 2024-04-23T14:32:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.322+0000 7f666ea00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f665406c8f0 0x7f665406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.323+0000 7f666ea00640 1 -- 172.21.15.59:0/1059543517 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f666008c2a0 con 0x7f66781f3d10 2024-04-23T14:32:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.323+0000 7f6676a00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f665406c8f0 0x7f665406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.323+0000 7f6676a00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f665406c8f0 0x7f665406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6668009a80 tx=0x7f6668008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.327+0000 7f666ea00640 1 -- 172.21.15.59:0/1059543517 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6660018030 con 0x7f66781f3d10 2024-04-23T14:32:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.527+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66780b1ba0 con 0x7f66781f3d10 2024-04-23T14:32:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.528+0000 7f666ea00640 1 -- 172.21.15.59:0/1059543517 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f6660059f60 con 0x7f66781f3d10 2024-04-23T14:32:25.529 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.530+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f665406c8f0 msgr2=0x7f665406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.530+0000 7f6677e00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f665406c8f0 0x7f665406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6668009a80 tx=0x7f6668008040 comp rx=0 tx=0).stop 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.530+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66781f3d10 msgr2=0x7f66780b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.530+0000 7f6677e00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66781f3d10 0x7f66780b1280 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6660010890 tx=0x7f6660010d50 comp rx=0 tx=0).stop 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.530+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 shutdown_connections 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.530+0000 7f6677e00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f665406c8f0 0x7f665406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.530+0000 7f6677e00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f66780b17c0 0x7f667815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.531+0000 7f6677e00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66781f3d10 0x7f66780b1280 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.531+0000 7f6677e00640 1 --2- 172.21.15.59:0/1059543517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f665c004d40 0x7f66780cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.531+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 >> 172.21.15.59:0/1059543517 conn(0x7f667806bdf0 msgr2=0x7f66780ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.531+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 shutdown_connections 2024-04-23T14:32:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.531+0000 7f6677e00640 1 -- 172.21.15.59:0/1059543517 wait complete. 2024-04-23T14:32:25.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:25.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.761+0000 7fdb7aa00640 1 Processor -- start 2024-04-23T14:32:25.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.761+0000 7fdb7aa00640 1 -- start start 2024-04-23T14:32:25.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.762+0000 7fdb7aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb741567e0 0x7fdb74156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.762+0000 7fdb7aa00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb7415d810 0x7fdb740d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.762+0000 7fdb7aa00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7415cab0 con 0x7fdb740d0b40 2024-04-23T14:32:25.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.762+0000 7fdb7aa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdb740d0f90 con 0x7fdb741567e0 2024-04-23T14:32:25.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.762+0000 7fdb7aa00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdb74160e40 con 0x7fdb7415d810 2024-04-23T14:32:25.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.762+0000 7fdb79600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb741567e0 0x7fdb74156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb79600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb741567e0 0x7fdb74156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46646/0 (socket says 172.21.15.59:46646) 2024-04-23T14:32:25.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb79600640 1 -- 172.21.15.59:0/3425874778 learned_addr learned my addr 172.21.15.59:0/3425874778 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:25.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb78c00640 1 --2- 172.21.15.59:0/3425874778 >> v2:172.21.15.76:3300/0 conn(0x7fdb7415d810 0x7fdb740d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb79600640 1 -- 172.21.15.59:0/3425874778 >> v2:172.21.15.76:3300/0 conn(0x7fdb7415d810 msgr2=0x7fdb740d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb79600640 1 --2- 172.21.15.59:0/3425874778 >> v2:172.21.15.76:3300/0 conn(0x7fdb7415d810 0x7fdb740d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb79600640 1 -- 172.21.15.59:0/3425874778 >> v1:172.21.15.76:6789/0 conn(0x7fdb740d0b40 legacy=0x7fdb74160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb79600640 1 -- 172.21.15.59:0/3425874778 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7415cd30 con 0x7fdb741567e0 2024-04-23T14:32:25.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.763+0000 7fdb78c00640 1 --2- 172.21.15.59:0/3425874778 >> v2:172.21.15.76:3300/0 conn(0x7fdb7415d810 0x7fdb740d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.764+0000 7fdb79600640 1 --2- 172.21.15.59:0/3425874778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb741567e0 0x7fdb74156bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdb74074140 tx=0x7fdb6802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35e458c56da696a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb73e00640 1 -- 172.21.15.59:0/3425874778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68037070 con 0x7fdb741567e0 2024-04-23T14:32:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb73e00640 1 -- 172.21.15.59:0/3425874778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb68032440 con 0x7fdb741567e0 2024-04-23T14:32:25.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb73e00640 1 -- 172.21.15.59:0/3425874778 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68032810 con 0x7fdb741567e0 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb7aa00640 1 -- 172.21.15.59:0/3425874778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb741567e0 msgr2=0x7fdb74156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/3425874778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb741567e0 0x7fdb74156bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdb74074140 tx=0x7fdb6802a230 comp rx=0 tx=0).stop 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb7aa00640 1 -- 172.21.15.59:0/3425874778 shutdown_connections 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/3425874778 >> v2:172.21.15.76:3300/0 conn(0x7fdb7415d810 0x7fdb740d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/3425874778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb741567e0 0x7fdb74156bc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb7aa00640 1 -- 172.21.15.59:0/3425874778 >> 172.21.15.59:0/3425874778 conn(0x7fdb7406bdf0 msgr2=0x7fdb7406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.765+0000 7fdb7aa00640 1 -- 172.21.15.59:0/3425874778 shutdown_connections 2024-04-23T14:32:25.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.766+0000 7fdb7aa00640 1 -- 172.21.15.59:0/3425874778 wait complete. 2024-04-23T14:32:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.767+0000 7fdb7aa00640 1 Processor -- start 2024-04-23T14:32:25.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.767+0000 7fdb7aa00640 1 -- start start 2024-04-23T14:32:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.768+0000 7fdb7aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb740d0b40 0x7fdb7414dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.768+0000 7fdb7aa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb741567e0 0x7fdb74151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.768+0000 7fdb7aa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb7415d810 0x7fdb7414c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.768+0000 7fdb7aa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74160820 con 0x7fdb740d0b40 2024-04-23T14:32:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.768+0000 7fdb7aa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7415d2f0 con 0x7fdb741567e0 2024-04-23T14:32:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.768+0000 7fdb7aa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdb741609a0 con 0x7fdb7415d810 2024-04-23T14:32:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb740d0b40 0x7fdb7414dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb740d0b40 0x7fdb7414dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46656/0 (socket says 172.21.15.59:46656) 2024-04-23T14:32:25.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 -- 172.21.15.59:0/2499052414 learned_addr learned my addr 172.21.15.59:0/2499052414 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 -- 172.21.15.59:0/2499052414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb7415d810 msgr2=0x7fdb7414c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb7415d810 0x7fdb7414c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 -- 172.21.15.59:0/2499052414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb741567e0 msgr2=0x7fdb74151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb741567e0 0x7fdb74151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 -- 172.21.15.59:0/2499052414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7414cd40 con 0x7fdb740d0b40 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.769+0000 7fdb79600640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb740d0b40 0x7fdb7414dde0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdb68000c00 tx=0x7fdb680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.770+0000 7fdb71600640 1 -- 172.21.15.59:0/2499052414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68041070 con 0x7fdb740d0b40 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.770+0000 7fdb71600640 1 -- 172.21.15.59:0/2499052414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6803c890 con 0x7fdb740d0b40 2024-04-23T14:32:25.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.770+0000 7fdb71600640 1 -- 172.21.15.59:0/2499052414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68037040 con 0x7fdb740d0b40 2024-04-23T14:32:25.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.770+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7414d020 con 0x7fdb740d0b40 2024-04-23T14:32:25.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.770+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7414d5e0 con 0x7fdb740d0b40 2024-04-23T14:32:25.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.771+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7404fd90 con 0x7fdb740d0b40 2024-04-23T14:32:25.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.772+0000 7fdb71600640 1 -- 172.21.15.59:0/2499052414 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb68044070 con 0x7fdb740d0b40 2024-04-23T14:32:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.773+0000 7fdb71600640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb5406c8f0 0x7fdb5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.773+0000 7fdb71600640 1 -- 172.21.15.59:0/2499052414 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdb680ac7b0 con 0x7fdb740d0b40 2024-04-23T14:32:25.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.773+0000 7fdb78c00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb5406c8f0 0x7fdb5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:25.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.774+0000 7fdb78c00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb5406c8f0 0x7fdb5406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb741e8a40 tx=0x7fdb60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:25.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.778+0000 7fdb71600640 1 -- 172.21.15.59:0/2499052414 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb7415cab0 con 0x7fdb740d0b40 2024-04-23T14:32:25.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.972+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb740b5150 con 0x7fdb740d0b40 2024-04-23T14:32:25.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.973+0000 7fdb71600640 1 -- 172.21.15.59:0/2499052414 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fdb6807ab60 con 0x7fdb740d0b40 2024-04-23T14:32:25.974 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.976+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb5406c8f0 msgr2=0x7fdb5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.976+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb5406c8f0 0x7fdb5406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb741e8a40 tx=0x7fdb60008040 comp rx=0 tx=0).stop 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.976+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb740d0b40 msgr2=0x7fdb7414dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.976+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb740d0b40 0x7fdb7414dde0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdb68000c00 tx=0x7fdb680046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 shutdown_connections 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb5406c8f0 0x7fdb5406edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb7415d810 0x7fdb7414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb741567e0 0x7fdb74151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 --2- 172.21.15.59:0/2499052414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb740d0b40 0x7fdb7414dde0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 >> 172.21.15.59:0/2499052414 conn(0x7fdb7406bdf0 msgr2=0x7fdb74155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 shutdown_connections 2024-04-23T14:32:25.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:25.977+0000 7fdb7aa00640 1 -- 172.21.15.59:0/2499052414 wait complete. 2024-04-23T14:32:25.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.195+0000 7f7520e00640 1 Processor -- start 2024-04-23T14:32:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.195+0000 7f7520e00640 1 -- start start 2024-04-23T14:32:26.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.196+0000 7f7520e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c15ed20 0x7f751c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.196+0000 7f7520e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f751c0b3600 0x7f751c0d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.196+0000 7f7520e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f751c0b3f10 0x7f751c0b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.196+0000 7f7520e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f751c15f100 con 0x7f751c0b3f10 2024-04-23T14:32:26.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.197+0000 7f7520e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f751c0b5da0 con 0x7f751c0b3600 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.197+0000 7f7520e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f751c0d0a20 con 0x7f751c15ed20 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.197+0000 7f751be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f751c0b3f10 0x7f751c0b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.197+0000 7f751be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f751c0b3f10 0x7f751c0b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46672/0 (socket says 172.21.15.59:46672) 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.197+0000 7f751be00640 1 -- 172.21.15.59:0/734724868 learned_addr learned my addr 172.21.15.59:0/734724868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.197+0000 7f751aa00640 1 --2- 172.21.15.59:0/734724868 >> v2:172.21.15.76:3300/0 conn(0x7f751c0b3600 0x7f751c0d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.197+0000 7f751b400640 1 --2- 172.21.15.59:0/734724868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c15ed20 0x7f751c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.198+0000 7f751be00640 1 -- 172.21.15.59:0/734724868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c15ed20 msgr2=0x7f751c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.198+0000 7f751be00640 1 --2- 172.21.15.59:0/734724868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c15ed20 0x7f751c0d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.198+0000 7f751be00640 1 -- 172.21.15.59:0/734724868 >> v2:172.21.15.76:3300/0 conn(0x7f751c0b3600 msgr2=0x7f751c0d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:26.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.198+0000 7f751be00640 1 --2- 172.21.15.59:0/734724868 >> v2:172.21.15.76:3300/0 conn(0x7f751c0b3600 0x7f751c0d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.198+0000 7f751be00640 1 -- 172.21.15.59:0/734724868 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c0d2ed0 con 0x7f751c0b3f10 2024-04-23T14:32:26.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.199+0000 7f751be00640 1 --2- 172.21.15.59:0/734724868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c0b43a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f751c06c640 tx=0x7f751402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c99109717fb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:26.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.199+0000 7f751a000640 1 -- 172.21.15.59:0/734724868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514036070 con 0x7f751c0b3f10 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.199+0000 7f751a000640 1 -- 172.21.15.59:0/734724868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7514031600 con 0x7f751c0b3f10 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.199+0000 7f751a000640 1 -- 172.21.15.59:0/734724868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75140319d0 con 0x7f751c0b3f10 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 -- 172.21.15.59:0/734724868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 msgr2=0x7f751c0b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 --2- 172.21.15.59:0/734724868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c0b43a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f751c06c640 tx=0x7f751402a570 comp rx=0 tx=0).stop 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 -- 172.21.15.59:0/734724868 shutdown_connections 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 --2- 172.21.15.59:0/734724868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c0b43a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 --2- 172.21.15.59:0/734724868 >> v2:172.21.15.76:3300/0 conn(0x7f751c0b3600 0x7f751c0d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 --2- 172.21.15.59:0/734724868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c15ed20 0x7f751c0d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 -- 172.21.15.59:0/734724868 >> 172.21.15.59:0/734724868 conn(0x7f751c08bf90 msgr2=0x7f751c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 -- 172.21.15.59:0/734724868 shutdown_connections 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.200+0000 7f7520e00640 1 -- 172.21.15.59:0/734724868 wait complete. 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.203+0000 7f7520e00640 1 Processor -- start 2024-04-23T14:32:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.203+0000 7f7520e00640 1 -- start start 2024-04-23T14:32:26.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.204+0000 7f7520e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0b3600 0x7f751c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.204+0000 7f7520e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.204+0000 7f7520e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c15ed20 0x7f751c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.204+0000 7f7520e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f751c0b48e0 con 0x7f751c0b3f10 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.204+0000 7f7520e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f751c0d3490 con 0x7f751c15ed20 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.204+0000 7f7520e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f751c0b4a60 con 0x7f751c0b3600 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c149880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46684/0 (socket says 172.21.15.59:46684) 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 -- 172.21.15.59:0/1786312662 learned_addr learned my addr 172.21.15.59:0/1786312662 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751b400640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0b3600 0x7f751c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751be00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c15ed20 0x7f751c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 -- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0b3600 msgr2=0x7f751c0cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0b3600 0x7f751c0cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 -- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c15ed20 msgr2=0x7f751c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c15ed20 0x7f751c0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751aa00640 1 -- 172.21.15.59:0/1786312662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c0cb690 con 0x7f751c0b3f10 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.205+0000 7f751be00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c15ed20 0x7f751c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.206+0000 7f751aa00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c149880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d800 tx=0x7f750c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.206+0000 7f7513400640 1 -- 172.21.15.59:0/1786312662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c018020 con 0x7f751c0b3f10 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.206+0000 7f7513400640 1 -- 172.21.15.59:0/1786312662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c0096a0 con 0x7f751c0b3f10 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.206+0000 7f7513400640 1 -- 172.21.15.59:0/1786312662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c009a70 con 0x7f751c0b3f10 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.206+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c0cb970 con 0x7f751c0b3f10 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.206+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c0cbf30 con 0x7f751c0b3f10 2024-04-23T14:32:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.208+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751c04fd90 con 0x7f751c0b3f10 2024-04-23T14:32:26.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.209+0000 7f7513400640 1 -- 172.21.15.59:0/1786312662 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f750c01b020 con 0x7f751c0b3f10 2024-04-23T14:32:26.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.210+0000 7f7513400640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74f806c8f0 0x7f74f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.210+0000 7f7513400640 1 -- 172.21.15.59:0/1786312662 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f750c08bf00 con 0x7f751c0b3f10 2024-04-23T14:32:26.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.210+0000 7f751b400640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74f806c8f0 0x7f74f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.211+0000 7f751b400640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74f806c8f0 0x7f74f806edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7504006fd0 tx=0x7f7504008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.215+0000 7f7513400640 1 -- 172.21.15.59:0/1786312662 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f750c022030 con 0x7f751c0b3f10 2024-04-23T14:32:26.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.418+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f751c15d8c0 con 0x7f751c0b3f10 2024-04-23T14:32:26.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.419+0000 7f7513400640 1 -- 172.21.15.59:0/1786312662 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f750c05a210 con 0x7f751c0b3f10 2024-04-23T14:32:26.420 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74f806c8f0 msgr2=0x7f74f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74f806c8f0 0x7f74f806edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7504006fd0 tx=0x7f7504008040 comp rx=0 tx=0).stop 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 msgr2=0x7f751c149880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c149880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d800 tx=0x7f750c00dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 shutdown_connections 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74f806c8f0 0x7f74f806edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f751c15ed20 0x7f751c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f751c0b3f10 0x7f751c149880 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 --2- 172.21.15.59:0/1786312662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f751c0b3600 0x7f751c0cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 >> 172.21.15.59:0/1786312662 conn(0x7f751c08bf90 msgr2=0x7f751c0cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:26.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 shutdown_connections 2024-04-23T14:32:26.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.422+0000 7f7520e00640 1 -- 172.21.15.59:0/1786312662 wait complete. 2024-04-23T14:32:26.443 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.646+0000 7f9c20a00640 1 Processor -- start 2024-04-23T14:32:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.646+0000 7f9c20a00640 1 -- start start 2024-04-23T14:32:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.647+0000 7f9c20a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c1c0cf4e0 0x7f9c1c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.647+0000 7f9c20a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9c1c15fc20 0x7f9c1c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.647+0000 7f9c20a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c1c0cfe00 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.647+0000 7f9c20a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9c1c1601c0 con 0x7f9c1c15fc20 2024-04-23T14:32:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.647+0000 7f9c20a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9c1c0ce8d0 con 0x7f9c1c0cf4e0 2024-04-23T14:32:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.648+0000 7f9c19400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9c1c0d1f80 0x7f9c1c15bc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46596/0 (socket says 172.21.15.59:46596) 2024-04-23T14:32:26.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.648+0000 7f9c19400640 1 -- 172.21.15.59:0/2560588147 learned_addr learned my addr 172.21.15.59:0/2560588147 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:26.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.648+0000 7f9c19e00640 1 --2- 172.21.15.59:0/2560588147 >> v2:172.21.15.76:3300/0 conn(0x7f9c1c0cf4e0 0x7f9c1c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.648+0000 7f9c1a800640 1 --2- 172.21.15.59:0/2560588147 >> v2:172.21.15.59:3300/0 conn(0x7f9c1c15fc20 0x7f9c1c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.649+0000 7f9c19e00640 1 -- 172.21.15.59:0/2560588147 >> v2:172.21.15.59:3300/0 conn(0x7f9c1c15fc20 msgr2=0x7f9c1c15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:26.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.649+0000 7f9c19e00640 1 --2- 172.21.15.59:0/2560588147 >> v2:172.21.15.59:3300/0 conn(0x7f9c1c15fc20 0x7f9c1c15c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.649+0000 7f9c19e00640 1 -- 172.21.15.59:0/2560588147 >> v1:172.21.15.59:6789/0 conn(0x7f9c1c0d1f80 legacy=0x7f9c1c15bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.649+0000 7f9c19e00640 1 -- 172.21.15.59:0/2560588147 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c0cdfa0 con 0x7f9c1c0cf4e0 2024-04-23T14:32:26.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.649+0000 7f9c1a800640 1 --2- 172.21.15.59:0/2560588147 >> v2:172.21.15.59:3300/0 conn(0x7f9c1c15fc20 0x7f9c1c15c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:26.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.651+0000 7f9c19e00640 1 --2- 172.21.15.59:0/2560588147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c0cf8c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c072fd0 tx=0x7f9c0c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa567bf31e7141cf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.652+0000 7f9c18a00640 1 -- 172.21.15.59:0/2560588147 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c037070 con 0x7f9c1c0cf4e0 2024-04-23T14:32:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.652+0000 7f9c18a00640 1 -- 172.21.15.59:0/2560588147 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0c032440 con 0x7f9c1c0cf4e0 2024-04-23T14:32:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.652+0000 7f9c18a00640 1 -- 172.21.15.59:0/2560588147 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c032810 con 0x7f9c1c0cf4e0 2024-04-23T14:32:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 -- 172.21.15.59:0/2560588147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 msgr2=0x7f9c1c0cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 --2- 172.21.15.59:0/2560588147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c0cf8c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c072fd0 tx=0x7f9c0c02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 -- 172.21.15.59:0/2560588147 shutdown_connections 2024-04-23T14:32:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 --2- 172.21.15.59:0/2560588147 >> v2:172.21.15.59:3300/0 conn(0x7f9c1c15fc20 0x7f9c1c15c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 --2- 172.21.15.59:0/2560588147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c0cf8c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 -- 172.21.15.59:0/2560588147 >> 172.21.15.59:0/2560588147 conn(0x7f9c1c0d1460 msgr2=0x7f9c1c0d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 -- 172.21.15.59:0/2560588147 shutdown_connections 2024-04-23T14:32:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.653+0000 7f9c20a00640 1 -- 172.21.15.59:0/2560588147 wait complete. 2024-04-23T14:32:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.655+0000 7f9c20a00640 1 Processor -- start 2024-04-23T14:32:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.655+0000 7f9c20a00640 1 -- start start 2024-04-23T14:32:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c20a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c1f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c20a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c1c0d1f80 0x7f9c1c15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c20a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c1c15fc20 0x7f9c1c0b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c20a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c0cd060 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c20a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c0ce560 con 0x7f9c1c0cf4e0 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c20a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1c0ced60 con 0x7f9c1c15fc20 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c19400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c1c0d1f80 0x7f9c1c15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c19400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c1c0d1f80 0x7f9c1c15e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46712/0 (socket says 172.21.15.59:46712) 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.656+0000 7f9c19400640 1 -- 172.21.15.59:0/506863910 learned_addr learned my addr 172.21.15.59:0/506863910 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19e00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c1f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19400640 1 -- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c1c15fc20 msgr2=0x7f9c1c0b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19400640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c1c15fc20 0x7f9c1c0b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19400640 1 -- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 msgr2=0x7f9c1c1f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19400640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c1f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19400640 1 -- 172.21.15.59:0/506863910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c0b6aa0 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19e00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c1f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c19400640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c1c0d1f80 0x7f9c1c15e220 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c380 tx=0x7f9c0800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c12000640 1 -- 172.21.15.59:0/506863910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08017070 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c12000640 1 -- 172.21.15.59:0/506863910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08005ce0 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c12000640 1 -- 172.21.15.59:0/506863910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080072f0 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c0b6d80 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.657+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c0b5300 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.659+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c069190 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.660+0000 7f9c12000640 1 -- 172.21.15.59:0/506863910 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c0801a070 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.661+0000 7f9c12000640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bfc06c8f0 0x7f9bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.661+0000 7f9c12000640 1 -- 172.21.15.59:0/506863910 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9c0808af30 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.661+0000 7f9c19e00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bfc06c8f0 0x7f9bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.662+0000 7f9c19e00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bfc06c8f0 0x7f9bfc06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c034000 tx=0x7f9c0c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:26.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.666+0000 7f9c12000640 1 -- 172.21.15.59:0/506863910 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c080592d0 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.884+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c1c0b73e0 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.885+0000 7f9c12000640 1 -- 172.21.15.59:0/506863910 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f9c08058bd0 con 0x7f9c1c0d1f80 2024-04-23T14:32:26.886 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:26.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bfc06c8f0 msgr2=0x7f9bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bfc06c8f0 0x7f9bfc06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c034000 tx=0x7f9c0c02e000 comp rx=0 tx=0).stop 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c1c0d1f80 msgr2=0x7f9c1c15e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c1c0d1f80 0x7f9c1c15e220 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c380 tx=0x7f9c0800c840 comp rx=0 tx=0).stop 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 shutdown_connections 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bfc06c8f0 0x7f9bfc06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c1c15fc20 0x7f9c1c0b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c1c0d1f80 0x7f9c1c15e220 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 --2- 172.21.15.59:0/506863910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c1c0cf4e0 0x7f9c1c1f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.887+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 >> 172.21.15.59:0/506863910 conn(0x7f9c1c0d1460 msgr2=0x7f9c1c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.888+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 shutdown_connections 2024-04-23T14:32:26.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:26.888+0000 7f9c20a00640 1 -- 172.21.15.59:0/506863910 wait complete. 2024-04-23T14:32:26.908 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:32:26.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:26.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-23T14:32:26.995 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:26.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:32:26.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:26.996 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:32:27.050 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:32:27.050 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:27.108 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:32:27.108 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:27.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:27.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.371+0000 7fbcf6c00640 1 Processor -- start 2024-04-23T14:32:27.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.372+0000 7fbcf6c00640 1 -- start start 2024-04-23T14:32:27.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.373+0000 7fbcf6c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.373+0000 7fbcf6c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.373+0000 7fbcf6c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf00cd520 con 0x7fbcf00b3e40 2024-04-23T14:32:27.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.373+0000 7fbcf6c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf00cddc0 con 0x7fbcf01454e0 2024-04-23T14:32:27.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.373+0000 7fbcf6c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf00cd800 con 0x7fbcf00cd020 2024-04-23T14:32:27.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcefe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcefe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46714/0 (socket says 172.21.15.59:46714) 2024-04-23T14:32:27.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcefe00640 1 -- 172.21.15.59:0/328277028 learned_addr learned my addr 172.21.15.59:0/328277028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:27.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcefe00640 1 -- 172.21.15.59:0/328277028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 msgr2=0x7fbcf00b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:27.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcef400640 1 --2- 172.21.15.59:0/328277028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcefe00640 1 --2- 172.21.15.59:0/328277028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcefe00640 1 -- 172.21.15.59:0/328277028 >> v1:172.21.15.76:6789/0 conn(0x7fbcf00b3e40 legacy=0x7fbcf0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:27.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcefe00640 1 -- 172.21.15.59:0/328277028 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf00cdae0 con 0x7fbcf01454e0 2024-04-23T14:32:27.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.374+0000 7fbcef400640 1 --2- 172.21.15.59:0/328277028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:27.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.375+0000 7fbcefe00640 1 --2- 172.21.15.59:0/328277028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf01454e0 0x7fbcf0145bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009870 tx=0x7fbce002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e477aa69499c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:27.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.376+0000 7fbceea00640 1 -- 172.21.15.59:0/328277028 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0037070 con 0x7fbcf01454e0 2024-04-23T14:32:27.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.376+0000 7fbceea00640 1 -- 172.21.15.59:0/328277028 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce0032440 con 0x7fbcf01454e0 2024-04-23T14:32:27.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.376+0000 7fbceea00640 1 -- 172.21.15.59:0/328277028 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0032810 con 0x7fbcf01454e0 2024-04-23T14:32:27.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.376+0000 7fbcf6c00640 1 -- 172.21.15.59:0/328277028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf01454e0 msgr2=0x7fbcf0145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.376+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/328277028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf01454e0 0x7fbcf0145bb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009870 tx=0x7fbce002a180 comp rx=0 tx=0).stop 2024-04-23T14:32:27.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.377+0000 7fbcf6c00640 1 -- 172.21.15.59:0/328277028 shutdown_connections 2024-04-23T14:32:27.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.377+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/328277028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf00b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.377+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/328277028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf01454e0 0x7fbcf0145bb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.377+0000 7fbcf6c00640 1 -- 172.21.15.59:0/328277028 >> 172.21.15.59:0/328277028 conn(0x7fbcf0067ea0 msgr2=0x7fbcf006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:27.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.377+0000 7fbcf6c00640 1 -- 172.21.15.59:0/328277028 shutdown_connections 2024-04-23T14:32:27.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.377+0000 7fbcf6c00640 1 -- 172.21.15.59:0/328277028 wait complete. 2024-04-23T14:32:27.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.378+0000 7fbcf6c00640 1 Processor -- start 2024-04-23T14:32:27.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.379+0000 7fbcf6c00640 1 -- start start 2024-04-23T14:32:27.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.379+0000 7fbcf6c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf015e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.379+0000 7fbcf6c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf01f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.379+0000 7fbcf6c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcf01454e0 0x7fbcf0143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.379+0000 7fbcf6c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0146270 con 0x7fbcf00b3e40 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.379+0000 7fbcf6c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01460f0 con 0x7fbcf01454e0 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcf6c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf0161020 con 0x7fbcf00cd020 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf015e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf015e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46718/0 (socket says 172.21.15.59:46718) 2024-04-23T14:32:27.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 -- 172.21.15.59:0/2051093258 learned_addr learned my addr 172.21.15.59:0/2051093258 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:27.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 -- 172.21.15.59:0/2051093258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 msgr2=0x7fbcf01f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf01f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 -- 172.21.15.59:0/2051093258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcf01454e0 msgr2=0x7fbcf0143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcf01454e0 0x7fbcf0143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.380+0000 7fbcefe00640 1 -- 172.21.15.59:0/2051093258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf015e740 con 0x7fbcf00b3e40 2024-04-23T14:32:27.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.381+0000 7fbcefe00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf015e0d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:27.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.381+0000 7fbce7e00640 1 -- 172.21.15.59:0/2051093258 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0041020 con 0x7fbcf00b3e40 2024-04-23T14:32:27.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.381+0000 7fbce7e00640 1 -- 172.21.15.59:0/2051093258 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce0030800 con 0x7fbcf00b3e40 2024-04-23T14:32:27.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.381+0000 7fbce7e00640 1 -- 172.21.15.59:0/2051093258 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0037040 con 0x7fbcf00b3e40 2024-04-23T14:32:27.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.381+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf01f4e30 con 0x7fbcf00b3e40 2024-04-23T14:32:27.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.381+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf01f53f0 con 0x7fbcf00b3e40 2024-04-23T14:32:27.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.382+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf0068a80 con 0x7fbcf00b3e40 2024-04-23T14:32:27.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.384+0000 7fbce7e00640 1 -- 172.21.15.59:0/2051093258 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbce0044020 con 0x7fbcf00b3e40 2024-04-23T14:32:27.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.385+0000 7fbce7e00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c920 0x7fbcd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.385+0000 7fbce7e00640 1 -- 172.21.15.59:0/2051093258 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbce00ac460 con 0x7fbcf00b3e40 2024-04-23T14:32:27.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.385+0000 7fbcef400640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c920 0x7fbcd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.386+0000 7fbcef400640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c920 0x7fbcd006ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006fd0 tx=0x7fbcdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.389+0000 7fbce7e00640 1 -- 172.21.15.59:0/2051093258 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbce004b030 con 0x7fbcf00b3e40 2024-04-23T14:32:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.590+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcf0160030 con 0x7fbcf00b3e40 2024-04-23T14:32:27.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.592+0000 7fbce7e00640 1 -- 172.21.15.59:0/2051093258 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fbce007a800 con 0x7fbcf00b3e40 2024-04-23T14:32:27.594 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:27.594 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:27.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c920 msgr2=0x7fbcd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c920 0x7fbcd006ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006fd0 tx=0x7fbcdc008040 comp rx=0 tx=0).stop 2024-04-23T14:32:27.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 msgr2=0x7fbcf015e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf015e0d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce00046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 shutdown_connections 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c920 0x7fbcd006ede0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcf00cd020 0x7fbcf01f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcf01454e0 0x7fbcf0143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 --2- 172.21.15.59:0/2051093258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcf00b3e40 0x7fbcf015e0d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.595+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 >> 172.21.15.59:0/2051093258 conn(0x7fbcf0067ea0 msgr2=0x7fbcf00b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.596+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 shutdown_connections 2024-04-23T14:32:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.596+0000 7fbcf6c00640 1 -- 172.21.15.59:0/2051093258 wait complete. 2024-04-23T14:32:27.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:27.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:27.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.825+0000 7fbbcb200640 1 Processor -- start 2024-04-23T14:32:27.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.826+0000 7fbbcb200640 1 -- start start 2024-04-23T14:32:27.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.827+0000 7fbbcb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbbcc15ed20 0x7fbbcc0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.827+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.827+0000 7fbbcb200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc0b5da0 con 0x7fbbcc0b3600 2024-04-23T14:32:27.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.827+0000 7fbbcb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbbcc15f100 con 0x7fbbcc15ed20 2024-04-23T14:32:27.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.827+0000 7fbbcb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc0d07d0 con 0x7fbbcc0d13e0 2024-04-23T14:32:27.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbc9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbbcc15ed20 0x7fbbcc0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbc9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbbcc15ed20 0x7fbbcc0d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46732/0 (socket says 172.21.15.59:46732) 2024-04-23T14:32:27.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbc9e00640 1 -- 172.21.15.59:0/2339415275 learned_addr learned my addr 172.21.15.59:0/2339415275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:27.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbc9e00640 1 -- 172.21.15.59:0/2339415275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 msgr2=0x7fbbcc0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:27.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbca800640 1 --2- 172.21.15.59:0/2339415275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbc9e00640 1 --2- 172.21.15.59:0/2339415275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc0d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbc9e00640 1 -- 172.21.15.59:0/2339415275 >> v1:172.21.15.76:6789/0 conn(0x7fbbcc0b3600 legacy=0x7fbbcc0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:27.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.828+0000 7fbbc9e00640 1 -- 172.21.15.59:0/2339415275 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc0d0a20 con 0x7fbbcc15ed20 2024-04-23T14:32:27.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.829+0000 7fbbc9e00640 1 --2- 172.21.15.59:0/2339415275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc15ed20 0x7fbbcc0d0ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009870 tx=0x7fbbbc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00575a3ecca5346 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:27.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.829+0000 7fbbc8a00640 1 -- 172.21.15.59:0/2339415275 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc037070 con 0x7fbbcc15ed20 2024-04-23T14:32:27.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbc8a00640 1 -- 172.21.15.59:0/2339415275 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbbc02ae10 con 0x7fbbcc15ed20 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbc8a00640 1 -- 172.21.15.59:0/2339415275 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc032660 con 0x7fbbcc15ed20 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 -- 172.21.15.59:0/2339415275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc15ed20 msgr2=0x7fbbcc0d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 --2- 172.21.15.59:0/2339415275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc15ed20 0x7fbbcc0d0ea0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009870 tx=0x7fbbbc02a100 comp rx=0 tx=0).stop 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 -- 172.21.15.59:0/2339415275 shutdown_connections 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 --2- 172.21.15.59:0/2339415275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 --2- 172.21.15.59:0/2339415275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc15ed20 0x7fbbcc0d0ea0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 -- 172.21.15.59:0/2339415275 >> 172.21.15.59:0/2339415275 conn(0x7fbbcc08bf90 msgr2=0x7fbbcc0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 -- 172.21.15.59:0/2339415275 shutdown_connections 2024-04-23T14:32:27.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.830+0000 7fbbcb200640 1 -- 172.21.15.59:0/2339415275 wait complete. 2024-04-23T14:32:27.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.832+0000 7fbbcb200640 1 Processor -- start 2024-04-23T14:32:27.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.832+0000 7fbbcb200640 1 -- start start 2024-04-23T14:32:27.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbcc0b3600 0x7fbbcc153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc1529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc0b9780 0x7fbbcc0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbcb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0d3490 con 0x7fbbcc0b9780 2024-04-23T14:32:27.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbcb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0d31b0 con 0x7fbbcc0b3600 2024-04-23T14:32:27.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbcb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc0b4620 con 0x7fbbcc0d13e0 2024-04-23T14:32:27.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbca800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc0b9780 0x7fbbcc0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbca800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc0b9780 0x7fbbcc0ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46740/0 (socket says 172.21.15.59:46740) 2024-04-23T14:32:27.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.833+0000 7fbbca800640 1 -- 172.21.15.59:0/173356322 learned_addr learned my addr 172.21.15.59:0/173356322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:27.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbc9e00640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbcc0b3600 0x7fbbcc153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbca800640 1 -- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 msgr2=0x7fbbcc1529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbc9400640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc1529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbca800640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc1529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbca800640 1 -- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbcc0b3600 msgr2=0x7fbbcc153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbca800640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbcc0b3600 0x7fbbcc153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbca800640 1 -- 172.21.15.59:0/173356322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc0b61c0 con 0x7fbbcc0b9780 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbc9400640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc1529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.834+0000 7fbbca800640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc0b9780 0x7fbbcc0ca2c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400cc30 tx=0x7fbbc400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.835+0000 7fbbc2000640 1 -- 172.21.15.59:0/173356322 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400ce40 con 0x7fbbcc0b9780 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.835+0000 7fbbc2000640 1 -- 172.21.15.59:0/173356322 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc4003990 con 0x7fbbcc0b9780 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.835+0000 7fbbc2000640 1 -- 172.21.15.59:0/173356322 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4003d40 con 0x7fbbcc0b9780 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.835+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc0b64a0 con 0x7fbbcc0b9780 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.835+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc0b6a60 con 0x7fbbcc0b9780 2024-04-23T14:32:27.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.836+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbcc04fd90 con 0x7fbbcc0b9780 2024-04-23T14:32:27.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.837+0000 7fbbc2000640 1 -- 172.21.15.59:0/173356322 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbc4003ee0 con 0x7fbbcc0b9780 2024-04-23T14:32:27.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.839+0000 7fbbc2000640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbbac06c8f0 0x7fbbac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:27.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.839+0000 7fbbc2000640 1 -- 172.21.15.59:0/173356322 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbbc408c050 con 0x7fbbcc0b9780 2024-04-23T14:32:27.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.839+0000 7fbbc9e00640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbbac06c8f0 0x7fbbac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:27.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.840+0000 7fbbc9e00640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbbac06c8f0 0x7fbbac06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc034000 tx=0x7fbbbc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:27.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:27.843+0000 7fbbc2000640 1 -- 172.21.15.59:0/173356322 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbbc4014030 con 0x7fbbcc0b9780 2024-04-23T14:32:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.043+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbbcc152ef0 con 0x7fbbcc0b9780 2024-04-23T14:32:28.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.044+0000 7fbbc2000640 1 -- 172.21.15.59:0/173356322 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fbbc405a400 con 0x7fbbcc0b9780 2024-04-23T14:32:28.045 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:28.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.046+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbbac06c8f0 msgr2=0x7fbbac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.046+0000 7fbbcb200640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbbac06c8f0 0x7fbbac06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc034000 tx=0x7fbbbc02e000 comp rx=0 tx=0).stop 2024-04-23T14:32:28.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.046+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc0b9780 msgr2=0x7fbbcc0ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.046+0000 7fbbcb200640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc0b9780 0x7fbbcc0ca2c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400cc30 tx=0x7fbbc400a5c0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.046+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 shutdown_connections 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.047+0000 7fbbcb200640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbbac06c8f0 0x7fbbac06edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.047+0000 7fbbcb200640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbbcc0d13e0 0x7fbbcc1529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.047+0000 7fbbcb200640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbcc0b9780 0x7fbbcc0ca2c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.047+0000 7fbbcb200640 1 --2- 172.21.15.59:0/173356322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbcc0b3600 0x7fbbcc153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.047+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 >> 172.21.15.59:0/173356322 conn(0x7fbbcc08bf90 msgr2=0x7fbbcc0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.047+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 shutdown_connections 2024-04-23T14:32:28.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.047+0000 7fbbcb200640 1 -- 172.21.15.59:0/173356322 wait complete. 2024-04-23T14:32:28.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.276+0000 7f653ca00640 1 Processor -- start 2024-04-23T14:32:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.277+0000 7f653ca00640 1 -- start start 2024-04-23T14:32:28.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.278+0000 7f653ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f653806fa50 0x7f653806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f653ca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653815bf60 con 0x7f65380d0350 2024-04-23T14:32:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f653ca00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653815c240 con 0x7f65380b56f0 2024-04-23T14:32:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f653ca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f653815c520 con 0x7f653806fa50 2024-04-23T14:32:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f6535400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f653806fa50 0x7f653806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f6536800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f65380d0350 0x7f65380b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46606/0 (socket says 172.21.15.59:46606) 2024-04-23T14:32:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f6536800640 1 -- 172.21.15.59:0/4154159354 learned_addr learned my addr 172.21.15.59:0/4154159354 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:28.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f6535400640 1 -- 172.21.15.59:0/4154159354 >> v1:172.21.15.76:6789/0 conn(0x7f65380b56f0 legacy=0x7f653806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f6535400640 1 -- 172.21.15.59:0/4154159354 >> v1:172.21.15.59:6789/0 conn(0x7f65380d0350 legacy=0x7f65380b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f6535400640 1 -- 172.21.15.59:0/4154159354 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653815c800 con 0x7f653806fa50 2024-04-23T14:32:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.279+0000 7f6534a00640 1 -- 172.21.15.59:0/4154159354 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832644964 0 0) 0x7f653815bf60 con 0x7f65380d0350 2024-04-23T14:32:28.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.281+0000 7f6535400640 1 --2- 172.21.15.59:0/4154159354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f653806fa50 0x7f653806ddd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6520009d90 tx=0x7f652002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484557c95ff8cd7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.281+0000 7f6534a00640 1 -- 172.21.15.59:0/4154159354 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520007740 con 0x7f653806fa50 2024-04-23T14:32:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.281+0000 7f6534a00640 1 -- 172.21.15.59:0/4154159354 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65200078e0 con 0x7f653806fa50 2024-04-23T14:32:28.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.281+0000 7f6534a00640 1 -- 172.21.15.59:0/4154159354 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520007cb0 con 0x7f653806fa50 2024-04-23T14:32:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.282+0000 7f653ca00640 1 -- 172.21.15.59:0/4154159354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f653806fa50 msgr2=0x7f653806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.282+0000 7f653ca00640 1 --2- 172.21.15.59:0/4154159354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f653806fa50 0x7f653806ddd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6520009d90 tx=0x7f652002a6f0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.282+0000 7f653ca00640 1 -- 172.21.15.59:0/4154159354 shutdown_connections 2024-04-23T14:32:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.282+0000 7f653ca00640 1 --2- 172.21.15.59:0/4154159354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f653806fa50 0x7f653806ddd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.282+0000 7f653ca00640 1 -- 172.21.15.59:0/4154159354 >> 172.21.15.59:0/4154159354 conn(0x7f6538067d90 msgr2=0x7f653806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.282+0000 7f653ca00640 1 -- 172.21.15.59:0/4154159354 shutdown_connections 2024-04-23T14:32:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.282+0000 7f653ca00640 1 -- 172.21.15.59:0/4154159354 wait complete. 2024-04-23T14:32:28.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.284+0000 7f653ca00640 1 Processor -- start 2024-04-23T14:32:28.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.284+0000 7f653ca00640 1 -- start start 2024-04-23T14:32:28.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f653ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f653806fa50 0x7f65380af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f653ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65380b56f0 0x7f65380ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f653ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380d0350 0x7f65380a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f653ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65380d07e0 con 0x7f65380d0350 2024-04-23T14:32:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f653ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f653815bc10 con 0x7f65380b56f0 2024-04-23T14:32:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f653ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f65380d0960 con 0x7f653806fa50 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f6536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380d0350 0x7f65380a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f6536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380d0350 0x7f65380a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46756/0 (socket says 172.21.15.59:46756) 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.285+0000 7f6536800640 1 -- 172.21.15.59:0/3998324927 learned_addr learned my addr 172.21.15.59:0/3998324927 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6535400640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65380b56f0 0x7f65380ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6536800640 1 -- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f653806fa50 msgr2=0x7f65380af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6535e00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f653806fa50 0x7f65380af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6536800640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f653806fa50 0x7f65380af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6536800640 1 -- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65380b56f0 msgr2=0x7f65380ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6536800640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65380b56f0 0x7f65380ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6536800640 1 -- 172.21.15.59:0/3998324927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65380cec30 con 0x7f65380d0350 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6535400640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65380b56f0 0x7f65380ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6535e00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f653806fa50 0x7f65380af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:28.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f6536800640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380d0350 0x7f65380a9460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f653000c440 tx=0x7f653000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f652e000640 1 -- 172.21.15.59:0/3998324927 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300074e0 con 0x7f65380d0350 2024-04-23T14:32:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f652e000640 1 -- 172.21.15.59:0/3998324927 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65300047e0 con 0x7f65380d0350 2024-04-23T14:32:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f652e000640 1 -- 172.21.15.59:0/3998324927 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530004bd0 con 0x7f65380d0350 2024-04-23T14:32:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.286+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65380cef10 con 0x7f65380d0350 2024-04-23T14:32:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.287+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65380cf4d0 con 0x7f65380d0350 2024-04-23T14:32:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.288+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6538068a80 con 0x7f65380d0350 2024-04-23T14:32:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.289+0000 7f652e000640 1 -- 172.21.15.59:0/3998324927 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6530011070 con 0x7f65380d0350 2024-04-23T14:32:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.290+0000 7f652e000640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.290+0000 7f652e000640 1 -- 172.21.15.59:0/3998324927 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f653008b810 con 0x7f65380d0350 2024-04-23T14:32:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.290+0000 7f6535e00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.291+0000 7f6535e00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651406c8f0 0x7f651406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65381546d0 tx=0x7f6528009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.295+0000 7f652e000640 1 -- 172.21.15.59:0/3998324927 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6530059b20 con 0x7f65380d0350 2024-04-23T14:32:28.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.497+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f653806ef20 con 0x7f65380d0350 2024-04-23T14:32:28.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.498+0000 7f652e000640 1 -- 172.21.15.59:0/3998324927 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f6530059420 con 0x7f65380d0350 2024-04-23T14:32:28.500 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:28.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651406c8f0 msgr2=0x7f651406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651406c8f0 0x7f651406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65381546d0 tx=0x7f6528009040 comp rx=0 tx=0).stop 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380d0350 msgr2=0x7f65380a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380d0350 0x7f65380a9460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f653000c440 tx=0x7f653000c900 comp rx=0 tx=0).stop 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 shutdown_connections 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651406c8f0 0x7f651406edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380d0350 0x7f65380a9460 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65380b56f0 0x7f65380ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 --2- 172.21.15.59:0/3998324927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f653806fa50 0x7f65380af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.501+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 >> 172.21.15.59:0/3998324927 conn(0x7f6538067d90 msgr2=0x7f653806f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.502+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 shutdown_connections 2024-04-23T14:32:28.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.502+0000 7f653ca00640 1 -- 172.21.15.59:0/3998324927 wait complete. 2024-04-23T14:32:28.523 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:28.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.728+0000 7f51cac00640 1 Processor -- start 2024-04-23T14:32:28.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.729+0000 7f51cac00640 1 -- start start 2024-04-23T14:32:28.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.730+0000 7f51cac00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f51c415e790 0x7f51c40aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.730+0000 7f51cac00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f51c406f6f0 0x7f51c4162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.730+0000 7f51cac00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c4163370 con 0x7f51c4071d70 2024-04-23T14:32:28.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.730+0000 7f51cac00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f51c40aef40 con 0x7f51c406f6f0 2024-04-23T14:32:28.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.730+0000 7f51cac00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f51c40afd80 con 0x7f51c415e790 2024-04-23T14:32:28.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f51c406f6f0 0x7f51c4162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f51c406f6f0 0x7f51c4162e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46760/0 (socket says 172.21.15.59:46760) 2024-04-23T14:32:28.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3400640 1 -- 172.21.15.59:0/3935603753 learned_addr learned my addr 172.21.15.59:0/3935603753 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:28.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3e00640 1 --2- 172.21.15.59:0/3935603753 >> v2:172.21.15.76:3300/0 conn(0x7f51c415e790 0x7f51c40aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3400640 1 -- 172.21.15.59:0/3935603753 >> v2:172.21.15.76:3300/0 conn(0x7f51c415e790 msgr2=0x7f51c40aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:28.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3400640 1 --2- 172.21.15.59:0/3935603753 >> v2:172.21.15.76:3300/0 conn(0x7f51c415e790 0x7f51c40aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3400640 1 -- 172.21.15.59:0/3935603753 >> v1:172.21.15.76:6789/0 conn(0x7f51c4071d70 legacy=0x7f51c40b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:28.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.731+0000 7f51c3400640 1 -- 172.21.15.59:0/3935603753 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c40adc80 con 0x7f51c406f6f0 2024-04-23T14:32:28.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.733+0000 7f51c3400640 1 --2- 172.21.15.59:0/3935603753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c406f6f0 0x7f51c4162e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f51c4070010 tx=0x7f51ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b152142e42ae1a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.733+0000 7f51c2a00640 1 -- 172.21.15.59:0/3935603753 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac037070 con 0x7f51c406f6f0 2024-04-23T14:32:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.733+0000 7f51c2a00640 1 -- 172.21.15.59:0/3935603753 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ac032440 con 0x7f51c406f6f0 2024-04-23T14:32:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.733+0000 7f51c2a00640 1 -- 172.21.15.59:0/3935603753 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac032810 con 0x7f51c406f6f0 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 -- 172.21.15.59:0/3935603753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c406f6f0 msgr2=0x7f51c4162e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 --2- 172.21.15.59:0/3935603753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c406f6f0 0x7f51c4162e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f51c4070010 tx=0x7f51ac02a230 comp rx=0 tx=0).stop 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 -- 172.21.15.59:0/3935603753 shutdown_connections 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 --2- 172.21.15.59:0/3935603753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c406f6f0 0x7f51c4162e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 --2- 172.21.15.59:0/3935603753 >> v2:172.21.15.76:3300/0 conn(0x7f51c415e790 0x7f51c40aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 -- 172.21.15.59:0/3935603753 >> 172.21.15.59:0/3935603753 conn(0x7f51c4067d90 msgr2=0x7f51c406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 -- 172.21.15.59:0/3935603753 shutdown_connections 2024-04-23T14:32:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.734+0000 7f51cac00640 1 -- 172.21.15.59:0/3935603753 wait complete. 2024-04-23T14:32:28.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.736+0000 7f51cac00640 1 Processor -- start 2024-04-23T14:32:28.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.736+0000 7f51cac00640 1 -- start start 2024-04-23T14:32:28.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51cac00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c406f6f0 0x7f51c4154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51cac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c4071d70 0x7f51c415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51cac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f51c415e790 0x7f51c414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51cac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51c40ae4e0 con 0x7f51c4071d70 2024-04-23T14:32:28.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51cac00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f51c40ae200 con 0x7f51c406f6f0 2024-04-23T14:32:28.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51cac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f51c406fbb0 con 0x7f51c415e790 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c4071d70 0x7f51c415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c4071d70 0x7f51c415c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46772/0 (socket says 172.21.15.59:46772) 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 -- 172.21.15.59:0/3282020125 learned_addr learned my addr 172.21.15.59:0/3282020125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3e00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c406f6f0 0x7f51c4154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 -- 172.21.15.59:0/3282020125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f51c415e790 msgr2=0x7f51c414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f51c415e790 0x7f51c414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 -- 172.21.15.59:0/3282020125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c406f6f0 msgr2=0x7f51c4154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c406f6f0 0x7f51c4154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.738+0000 7f51c3400640 1 -- 172.21.15.59:0/3282020125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c415ccd0 con 0x7f51c4071d70 2024-04-23T14:32:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.739+0000 7f51c3400640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c4071d70 0x7f51c415c790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f51ac000c00 tx=0x7f51ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.739+0000 7f51bbe00640 1 -- 172.21.15.59:0/3282020125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac041020 con 0x7f51c4071d70 2024-04-23T14:32:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.739+0000 7f51bbe00640 1 -- 172.21.15.59:0/3282020125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ac030970 con 0x7f51c4071d70 2024-04-23T14:32:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.739+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c415cf70 con 0x7f51c4071d70 2024-04-23T14:32:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.739+0000 7f51bbe00640 1 -- 172.21.15.59:0/3282020125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac037040 con 0x7f51c4071d70 2024-04-23T14:32:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.739+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c415d530 con 0x7f51c4071d70 2024-04-23T14:32:28.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.741+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c4068a80 con 0x7f51c4071d70 2024-04-23T14:32:28.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.742+0000 7f51bbe00640 1 -- 172.21.15.59:0/3282020125 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51ac044020 con 0x7f51c4071d70 2024-04-23T14:32:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.743+0000 7f51bbe00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f51a006c8f0 0x7f51a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.743+0000 7f51bbe00640 1 -- 172.21.15.59:0/3282020125 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f51ac0ac6e0 con 0x7f51c4071d70 2024-04-23T14:32:28.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.743+0000 7f51c3e00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f51a006c8f0 0x7f51a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:28.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.744+0000 7f51c3e00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f51a006c8f0 0x7f51a006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51c40d0510 tx=0x7f51b4008460 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.747+0000 7f51bbe00640 1 -- 172.21.15.59:0/3282020125 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51ac04b030 con 0x7f51c4071d70 2024-04-23T14:32:28.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.954+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51c406e430 con 0x7f51c4071d70 2024-04-23T14:32:28.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.955+0000 7f51bbe00640 1 -- 172.21.15.59:0/3282020125 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f51c4163370 con 0x7f51c4071d70 2024-04-23T14:32:28.956 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:28.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f51a006c8f0 msgr2=0x7f51a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f51a006c8f0 0x7f51a006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51c40d0510 tx=0x7f51b4008460 comp rx=0 tx=0).stop 2024-04-23T14:32:28.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c4071d70 msgr2=0x7f51c415c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c4071d70 0x7f51c415c790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f51ac000c00 tx=0x7f51ac0046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 shutdown_connections 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f51a006c8f0 0x7f51a006edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f51c415e790 0x7f51c414e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51c4071d70 0x7f51c415c790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 --2- 172.21.15.59:0/3282020125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c406f6f0 0x7f51c4154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.957+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 >> 172.21.15.59:0/3282020125 conn(0x7f51c4067d90 msgr2=0x7f51c40b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.958+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 shutdown_connections 2024-04-23T14:32:28.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:28.958+0000 7f51cac00640 1 -- 172.21.15.59:0/3282020125 wait complete. 2024-04-23T14:32:28.977 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:29.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.176+0000 7f14ba400640 1 Processor -- start 2024-04-23T14:32:29.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.176+0000 7f14ba400640 1 -- start start 2024-04-23T14:32:29.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.178+0000 7f14ba400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14b415ea40 0x7f14b40cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.178+0000 7f14ba400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415f880 0x7f14b40d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.178+0000 7f14ba400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b415fd30 con 0x7f14b40d0f80 2024-04-23T14:32:29.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.178+0000 7f14ba400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f14b40cf930 con 0x7f14b415ea40 2024-04-23T14:32:29.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.178+0000 7f14ba400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f14b40cfc10 con 0x7f14b415f880 2024-04-23T14:32:29.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.179+0000 7f14b9000640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f14b40d0f80 0x7f14b40d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:39360/0 (socket says 172.21.15.59:39360) 2024-04-23T14:32:29.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.179+0000 7f14b9000640 1 -- 172.21.15.59:0/4251099903 learned_addr learned my addr 172.21.15.59:0/4251099903 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:29.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.179+0000 7f14b9a00640 1 --2- 172.21.15.59:0/4251099903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415f880 0x7f14b40d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.179+0000 7f14b3e00640 1 --2- 172.21.15.59:0/4251099903 >> v2:172.21.15.76:3300/0 conn(0x7f14b415ea40 0x7f14b40cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.179+0000 7f14b3400640 1 -- 172.21.15.59:0/4251099903 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1353890624 0 0) 0x7f14b415fd30 con 0x7f14b40d0f80 2024-04-23T14:32:29.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.180+0000 7f14b9a00640 1 -- 172.21.15.59:0/4251099903 >> v2:172.21.15.76:3300/0 conn(0x7f14b415ea40 msgr2=0x7f14b40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:29.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.180+0000 7f14b9a00640 1 --2- 172.21.15.59:0/4251099903 >> v2:172.21.15.76:3300/0 conn(0x7f14b415ea40 0x7f14b40cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.180+0000 7f14b9a00640 1 -- 172.21.15.59:0/4251099903 >> v1:172.21.15.76:6789/0 conn(0x7f14b40d0f80 legacy=0x7f14b40d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.180+0000 7f14b9a00640 1 -- 172.21.15.59:0/4251099903 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b40cfef0 con 0x7f14b415f880 2024-04-23T14:32:29.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.181+0000 7f14b9a00640 1 --2- 172.21.15.59:0/4251099903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415f880 0x7f14b40d2310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=939b93fe47b66605 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:29.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.181+0000 7f14b3400640 1 -- 172.21.15.59:0/4251099903 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac031600 con 0x7f14b415f880 2024-04-23T14:32:29.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.181+0000 7f14b3400640 1 -- 172.21.15.59:0/4251099903 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac0317a0 con 0x7f14b415f880 2024-04-23T14:32:29.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.181+0000 7f14b3400640 1 -- 172.21.15.59:0/4251099903 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac031b70 con 0x7f14b415f880 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 -- 172.21.15.59:0/4251099903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415f880 msgr2=0x7f14b40d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 --2- 172.21.15.59:0/4251099903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415f880 0x7f14b40d2310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac02a5d0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 -- 172.21.15.59:0/4251099903 shutdown_connections 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 --2- 172.21.15.59:0/4251099903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415f880 0x7f14b40d2310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 --2- 172.21.15.59:0/4251099903 >> v2:172.21.15.76:3300/0 conn(0x7f14b415ea40 0x7f14b40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 -- 172.21.15.59:0/4251099903 >> 172.21.15.59:0/4251099903 conn(0x7f14b408bf90 msgr2=0x7f14b40d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 -- 172.21.15.59:0/4251099903 shutdown_connections 2024-04-23T14:32:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.182+0000 7f14ba400640 1 -- 172.21.15.59:0/4251099903 wait complete. 2024-04-23T14:32:29.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.184+0000 7f14ba400640 1 Processor -- start 2024-04-23T14:32:29.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.184+0000 7f14ba400640 1 -- start start 2024-04-23T14:32:29.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14ba400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b40d0f80 0x7f14b40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14ba400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415ea40 0x7f14b40cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14ba400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b415f880 0x7f14b41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14ba400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14b40d2850 con 0x7f14b415f880 2024-04-23T14:32:29.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14ba400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f14b40d04b0 con 0x7f14b40d0f80 2024-04-23T14:32:29.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14ba400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f14b40d29d0 con 0x7f14b415ea40 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14b9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b415f880 0x7f14b41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14b9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b415f880 0x7f14b41543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46782/0 (socket says 172.21.15.59:46782) 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.185+0000 7f14b9a00640 1 -- 172.21.15.59:0/3484879639 learned_addr learned my addr 172.21.15.59:0/3484879639 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b9000640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b40d0f80 0x7f14b40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b3e00640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415ea40 0x7f14b40cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b9a00640 1 -- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415ea40 msgr2=0x7f14b40cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b9a00640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415ea40 0x7f14b40cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b9a00640 1 -- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b40d0f80 msgr2=0x7f14b40b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b9a00640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b40d0f80 0x7f14b40b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b9a00640 1 -- 172.21.15.59:0/3484879639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b41548f0 con 0x7f14b415f880 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b3e00640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415ea40 0x7f14b40cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:29.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.186+0000 7f14b9a00640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b415f880 0x7f14b41543b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:29.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.187+0000 7f14b0c00640 1 -- 172.21.15.59:0/3484879639 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac03f070 con 0x7f14b415f880 2024-04-23T14:32:29.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.187+0000 7f14b0c00640 1 -- 172.21.15.59:0/3484879639 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac030be0 con 0x7f14b415f880 2024-04-23T14:32:29.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.187+0000 7f14b0c00640 1 -- 172.21.15.59:0/3484879639 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac009450 con 0x7f14b415f880 2024-04-23T14:32:29.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.187+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b40b9b70 con 0x7f14b415f880 2024-04-23T14:32:29.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.187+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b40ba130 con 0x7f14b415f880 2024-04-23T14:32:29.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.188+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b40752f0 con 0x7f14b415f880 2024-04-23T14:32:29.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.189+0000 7f14b0c00640 1 -- 172.21.15.59:0/3484879639 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14ac003990 con 0x7f14b415f880 2024-04-23T14:32:29.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.190+0000 7f14b0c00640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f149406c8f0 0x7f149406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.190+0000 7f14b0c00640 1 -- 172.21.15.59:0/3484879639 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f14ac0acc20 con 0x7f14b415f880 2024-04-23T14:32:29.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.190+0000 7f14b9000640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f149406c8f0 0x7f149406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.191+0000 7f14b9000640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f149406c8f0 0x7f149406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f14a8000f60 tx=0x7f14a8007b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:29.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.195+0000 7f14b0c00640 1 -- 172.21.15.59:0/3484879639 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14ac07aeb0 con 0x7f14b415f880 2024-04-23T14:32:29.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.402+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f14b404fd90 con 0x7f14b415f880 2024-04-23T14:32:29.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.403+0000 7f14b0c00640 1 -- 172.21.15.59:0/3484879639 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f14ac07a7b0 con 0x7f14b415f880 2024-04-23T14:32:29.404 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f149406c8f0 msgr2=0x7f149406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f149406c8f0 0x7f149406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f14a8000f60 tx=0x7f14a8007b90 comp rx=0 tx=0).stop 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b415f880 msgr2=0x7f14b41543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b415f880 0x7f14b41543b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac004a40 comp rx=0 tx=0).stop 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 shutdown_connections 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f149406c8f0 0x7f149406edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b415ea40 0x7f14b40cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b415f880 0x7f14b41543b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 --2- 172.21.15.59:0/3484879639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b40d0f80 0x7f14b40b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 >> 172.21.15.59:0/3484879639 conn(0x7f14b408bf90 msgr2=0x7f14b40d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:29.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 shutdown_connections 2024-04-23T14:32:29.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.406+0000 7f14ba400640 1 -- 172.21.15.59:0/3484879639 wait complete. 2024-04-23T14:32:29.424 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:32:29.424 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:32:29.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:29.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.699+0000 7f9e81a00640 1 Processor -- start 2024-04-23T14:32:29.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.699+0000 7f9e81a00640 1 -- start start 2024-04-23T14:32:29.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.700+0000 7f9e81a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c15b450 0x7f9e7c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.700+0000 7f9e81a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e7c06f3f0 0x7f9e7c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.700+0000 7f9e81a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e7c0b3080 0x7f9e7c0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.700+0000 7f9e81a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7c0b41a0 con 0x7f9e7c06f3f0 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.700+0000 7f9e81a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7c0b3870 con 0x7f9e7c0b3080 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.700+0000 7f9e81a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c0b3b50 con 0x7f9e7c15b450 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e7c06f3f0 0x7f9e7c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e7c06f3f0 0x7f9e7c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46794/0 (socket says 172.21.15.59:46794) 2024-04-23T14:32:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 -- 172.21.15.59:0/3512869851 learned_addr learned my addr 172.21.15.59:0/3512869851 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e7be00640 1 --2- 172.21.15.59:0/3512869851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c15b450 0x7f9e7c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e81000640 1 --2- 172.21.15.59:0/3512869851 >> v2:172.21.15.76:3300/0 conn(0x7f9e7c0b3080 0x7f9e7c0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 -- 172.21.15.59:0/3512869851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c15b450 msgr2=0x7f9e7c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 --2- 172.21.15.59:0/3512869851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c15b450 0x7f9e7c0b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 -- 172.21.15.59:0/3512869851 >> v2:172.21.15.76:3300/0 conn(0x7f9e7c0b3080 msgr2=0x7f9e7c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 --2- 172.21.15.59:0/3512869851 >> v2:172.21.15.76:3300/0 conn(0x7f9e7c0b3080 0x7f9e7c0cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.701+0000 7f9e73400640 1 -- 172.21.15.59:0/3512869851 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c0b3e30 con 0x7f9e7c06f3f0 2024-04-23T14:32:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.702+0000 7f9e73400640 1 --2- 172.21.15.59:0/3512869851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c0cd050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c06ee90 tx=0x7f9e6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa057e4295365951 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:29.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e7b400640 1 -- 172.21.15.59:0/3512869851 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64037070 con 0x7f9e7c06f3f0 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e7b400640 1 -- 172.21.15.59:0/3512869851 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e64032440 con 0x7f9e7c06f3f0 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e7b400640 1 -- 172.21.15.59:0/3512869851 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64032810 con 0x7f9e7c06f3f0 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e81a00640 1 -- 172.21.15.59:0/3512869851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c06f3f0 msgr2=0x7f9e7c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e81a00640 1 --2- 172.21.15.59:0/3512869851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c0cd050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c06ee90 tx=0x7f9e6402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e81a00640 1 -- 172.21.15.59:0/3512869851 shutdown_connections 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e81a00640 1 --2- 172.21.15.59:0/3512869851 >> v2:172.21.15.76:3300/0 conn(0x7f9e7c0b3080 0x7f9e7c0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e81a00640 1 --2- 172.21.15.59:0/3512869851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c0cd050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e81a00640 1 --2- 172.21.15.59:0/3512869851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c15b450 0x7f9e7c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.703+0000 7f9e81a00640 1 -- 172.21.15.59:0/3512869851 >> 172.21.15.59:0/3512869851 conn(0x7f9e7c0b21a0 msgr2=0x7f9e7c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.704+0000 7f9e81a00640 1 -- 172.21.15.59:0/3512869851 shutdown_connections 2024-04-23T14:32:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.704+0000 7f9e81a00640 1 -- 172.21.15.59:0/3512869851 wait complete. 2024-04-23T14:32:29.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.706+0000 7f9e81a00640 1 Processor -- start 2024-04-23T14:32:29.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.706+0000 7f9e81a00640 1 -- start start 2024-04-23T14:32:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.707+0000 7f9e81a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.707+0000 7f9e81a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c0b3080 0x7f9e7c0c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.707+0000 7f9e81a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c15b450 0x7f9e7c14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.707+0000 7f9e81a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c0b4ff0 con 0x7f9e7c15b450 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.707+0000 7f9e81a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c15b830 con 0x7f9e7c06f3f0 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.707+0000 7f9e81a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c0ad060 con 0x7f9e7c0b3080 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c15b450 0x7f9e7c14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c15b450 0x7f9e7c14fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46798/0 (socket says 172.21.15.59:46798) 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 -- 172.21.15.59:0/2850263566 learned_addr learned my addr 172.21.15.59:0/2850263566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e7be00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e73400640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c0b3080 0x7f9e7c0c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 -- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c0b3080 msgr2=0x7f9e7c0c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c0b3080 0x7f9e7c0c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 -- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e7c06f3f0 msgr2=0x7f9e7c152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e81000640 1 -- 172.21.15.59:0/2850263566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c0967d0 con 0x7f9e7c15b450 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.708+0000 7f9e7be00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.709+0000 7f9e81000640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c15b450 0x7f9e7c14fb30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400cd70 tx=0x7f9e7400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:29.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.709+0000 7f9e78c00640 1 -- 172.21.15.59:0/2850263566 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74003b90 con 0x7f9e7c15b450 2024-04-23T14:32:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.709+0000 7f9e78c00640 1 -- 172.21.15.59:0/2850263566 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e74003d30 con 0x7f9e7c15b450 2024-04-23T14:32:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.709+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c153160 con 0x7f9e7c15b450 2024-04-23T14:32:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.709+0000 7f9e78c00640 1 -- 172.21.15.59:0/2850263566 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74004240 con 0x7f9e7c15b450 2024-04-23T14:32:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.709+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c152b10 con 0x7f9e7c15b450 2024-04-23T14:32:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.710+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c04fd90 con 0x7f9e7c15b450 2024-04-23T14:32:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.711+0000 7f9e78c00640 1 -- 172.21.15.59:0/2850263566 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e740043e0 con 0x7f9e7c15b450 2024-04-23T14:32:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.712+0000 7f9e78c00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9e5806c8f0 0x7f9e5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.712+0000 7f9e78c00640 1 -- 172.21.15.59:0/2850263566 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9e7408c6b0 con 0x7f9e7c15b450 2024-04-23T14:32:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.713+0000 7f9e7be00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9e5806c8f0 0x7f9e5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.713+0000 7f9e7be00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9e5806c8f0 0x7f9e5806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c006fd0 tx=0x7f9e6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:29.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.717+0000 7f9e78c00640 1 -- 172.21.15.59:0/2850263566 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e7405aa50 con 0x7f9e7c15b450 2024-04-23T14:32:29.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.924+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e7c0d0730 con 0x7f9e7c15b450 2024-04-23T14:32:29.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.927+0000 7f9e78c00640 1 -- 172.21.15.59:0/2850263566 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7f9e7405a350 con 0x7f9e7c15b450 2024-04-23T14:32:29.929 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:29.929 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.930+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9e5806c8f0 msgr2=0x7f9e5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.930+0000 7f9e81a00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9e5806c8f0 0x7f9e5806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c006fd0 tx=0x7f9e6c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.930+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c15b450 msgr2=0x7f9e7c14fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.930+0000 7f9e81a00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c15b450 0x7f9e7c14fb30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400cd70 tx=0x7f9e7400a660 comp rx=0 tx=0).stop 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 shutdown_connections 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9e5806c8f0 0x7f9e5806edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e7c0b3080 0x7f9e7c0c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7c15b450 0x7f9e7c14fb30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 --2- 172.21.15.59:0/2850263566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e7c06f3f0 0x7f9e7c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 >> 172.21.15.59:0/2850263566 conn(0x7f9e7c0b21a0 msgr2=0x7f9e7c0cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 shutdown_connections 2024-04-23T14:32:29.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:29.931+0000 7f9e81a00640 1 -- 172.21.15.59:0/2850263566 wait complete. 2024-04-23T14:32:29.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:29.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:30.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.153+0000 7f2388a00640 1 Processor -- start 2024-04-23T14:32:30.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.154+0000 7f2388a00640 1 -- start start 2024-04-23T14:32:30.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.155+0000 7f2388a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f23840b81b0 0x7f23840d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.155+0000 7f2388a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f23840d1960 0x7f238409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.155+0000 7f2388a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23840cce30 con 0x7f238409af40 2024-04-23T14:32:30.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.155+0000 7f2388a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f238409b3b0 con 0x7f23840b81b0 2024-04-23T14:32:30.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.155+0000 7f2388a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f23840b8590 con 0x7f23840d1960 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.156+0000 7f2382800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f238409af40 0x7f238406ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47478/0 (socket says 172.21.15.59:47478) 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.156+0000 7f2382800640 1 -- 172.21.15.59:0/2044773191 learned_addr learned my addr 172.21.15.59:0/2044773191 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.156+0000 7f237be00640 1 --2- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f23840d1960 0x7f238409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.156+0000 7f2381e00640 1 --2- 172.21.15.59:0/2044773191 >> v2:172.21.15.76:3300/0 conn(0x7f23840b81b0 0x7f23840d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.156+0000 7f2381400640 1 -- 172.21.15.59:0/2044773191 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3807052836 0 0) 0x7f23840cce30 con 0x7f238409af40 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.156+0000 7f2381400640 1 -- 172.21.15.59:0/2044773191 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2368003520 con 0x7f238409af40 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.157+0000 7f2381e00640 1 -- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f23840d1960 msgr2=0x7f238409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.157+0000 7f2381e00640 1 --2- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f23840d1960 0x7f238409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.157+0000 7f2381e00640 1 -- 172.21.15.59:0/2044773191 >> v1:172.21.15.59:6789/0 conn(0x7f238409af40 legacy=0x7f238406ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.157+0000 7f2381e00640 1 -- 172.21.15.59:0/2044773191 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23840cd110 con 0x7f23840b81b0 2024-04-23T14:32:30.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.157+0000 7f237be00640 1 --2- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f23840d1960 0x7f238409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.158+0000 7f2381e00640 1 --2- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840b81b0 0x7f23840d1420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f23740097a0 tx=0x7f237402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=259374e75172a99d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.159+0000 7f2381400640 1 -- 172.21.15.59:0/2044773191 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374037070 con 0x7f23840b81b0 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.159+0000 7f2381400640 1 -- 172.21.15.59:0/2044773191 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2374032440 con 0x7f23840b81b0 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.159+0000 7f2381400640 1 -- 172.21.15.59:0/2044773191 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374032810 con 0x7f23840b81b0 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 -- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840b81b0 msgr2=0x7f23840d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 --2- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840b81b0 0x7f23840d1420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f23740097a0 tx=0x7f237402a180 comp rx=0 tx=0).stop 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 -- 172.21.15.59:0/2044773191 shutdown_connections 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 --2- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f23840d1960 0x7f238409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 --2- 172.21.15.59:0/2044773191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840b81b0 0x7f23840d1420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 -- 172.21.15.59:0/2044773191 >> 172.21.15.59:0/2044773191 conn(0x7f23840b4130 msgr2=0x7f23840b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:30.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 -- 172.21.15.59:0/2044773191 shutdown_connections 2024-04-23T14:32:30.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.160+0000 7f2388a00640 1 -- 172.21.15.59:0/2044773191 wait complete. 2024-04-23T14:32:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.163+0000 7f2388a00640 1 Processor -- start 2024-04-23T14:32:30.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.163+0000 7f2388a00640 1 -- start start 2024-04-23T14:32:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.164+0000 7f2388a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f238409af40 0x7f2384153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.164+0000 7f2388a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23840b81b0 0x7f23840c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.164+0000 7f2388a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840d1960 0x7f23840c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.164+0000 7f2388a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23840b4f10 con 0x7f23840b81b0 2024-04-23T14:32:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.164+0000 7f2388a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23840cd6d0 con 0x7f23840d1960 2024-04-23T14:32:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.164+0000 7f2388a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f23840b5490 con 0x7f238409af40 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.164+0000 7f237be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23840b81b0 0x7f23840c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23840b81b0 0x7f23840c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41192/0 (socket says 172.21.15.59:41192) 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 -- 172.21.15.59:0/2626043305 learned_addr learned my addr 172.21.15.59:0/2626043305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f2381e00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f238409af40 0x7f2384153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f2382800640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840d1960 0x7f23840c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 -- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f238409af40 msgr2=0x7f2384153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f238409af40 0x7f2384153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 -- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840d1960 msgr2=0x7f23840c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840d1960 0x7f23840c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 -- 172.21.15.59:0/2626043305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238415f3f0 con 0x7f23840b81b0 2024-04-23T14:32:30.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.165+0000 7f237be00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23840b81b0 0x7f23840c9bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f236c00e990 tx=0x7f236c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.166+0000 7f237a000640 1 -- 172.21.15.59:0/2626043305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c017070 con 0x7f23840b81b0 2024-04-23T14:32:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.166+0000 7f237a000640 1 -- 172.21.15.59:0/2626043305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c00c9e0 con 0x7f23840b81b0 2024-04-23T14:32:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.166+0000 7f237a000640 1 -- 172.21.15.59:0/2626043305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c00cdb0 con 0x7f23840b81b0 2024-04-23T14:32:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.166+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238415f6d0 con 0x7f23840b81b0 2024-04-23T14:32:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.166+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238415fc90 con 0x7f23840b81b0 2024-04-23T14:32:30.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.167+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238406c500 con 0x7f23840b81b0 2024-04-23T14:32:30.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.168+0000 7f237a000640 1 -- 172.21.15.59:0/2626043305 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f236c01a070 con 0x7f23840b81b0 2024-04-23T14:32:30.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.169+0000 7f237a000640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f236006c8f0 0x7f236006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.169+0000 7f237a000640 1 -- 172.21.15.59:0/2626043305 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f236c08bdf0 con 0x7f23840b81b0 2024-04-23T14:32:30.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.169+0000 7f2381e00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f236006c8f0 0x7f236006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.170+0000 7f2381e00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f236006c8f0 0x7f236006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2374034000 tx=0x7f237402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.174+0000 7f237a000640 1 -- 172.21.15.59:0/2626043305 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f238406c500 con 0x7f23840b81b0 2024-04-23T14:32:30.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.381+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23840b77c0 con 0x7f23840b81b0 2024-04-23T14:32:30.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.382+0000 7f237a000640 1 -- 172.21.15.59:0/2626043305 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f236c059ab0 con 0x7f23840b81b0 2024-04-23T14:32:30.383 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f236006c8f0 msgr2=0x7f236006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f236006c8f0 0x7f236006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2374034000 tx=0x7f237402e000 comp rx=0 tx=0).stop 2024-04-23T14:32:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23840b81b0 msgr2=0x7f23840c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23840b81b0 0x7f23840c9bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f236c00e990 tx=0x7f236c00ee50 comp rx=0 tx=0).stop 2024-04-23T14:32:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 shutdown_connections 2024-04-23T14:32:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f236006c8f0 0x7f236006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840d1960 0x7f23840c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23840b81b0 0x7f23840c9bd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 --2- 172.21.15.59:0/2626043305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f238409af40 0x7f2384153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 >> 172.21.15.59:0/2626043305 conn(0x7f23840b4130 msgr2=0x7f2384070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:30.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 shutdown_connections 2024-04-23T14:32:30.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.385+0000 7f2388a00640 1 -- 172.21.15.59:0/2626043305 wait complete. 2024-04-23T14:32:30.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:30.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.621+0000 7fc8f0400640 1 Processor -- start 2024-04-23T14:32:30.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.622+0000 7fc8f0400640 1 -- start start 2024-04-23T14:32:30.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.623+0000 7fc8f0400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e80d12b0 0x7fc8e80b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.623+0000 7fc8f0400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e8163320 con 0x7fc8e80d0ed0 2024-04-23T14:32:30.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.623+0000 7fc8f0400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e80b2770 con 0x7fc8e815ecc0 2024-04-23T14:32:30.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.623+0000 7fc8f0400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e80b2dc0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.623+0000 7fc8eda00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc8e80d0ed0 0x7fc8e815e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47488/0 (socket says 172.21.15.59:47488) 2024-04-23T14:32:30.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.623+0000 7fc8eda00640 1 -- 172.21.15.59:0/3331342270 learned_addr learned my addr 172.21.15.59:0/3331342270 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:30.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.623+0000 7fc8ee400640 1 --2- 172.21.15.59:0/3331342270 >> v2:172.21.15.59:3300/0 conn(0x7fc8e80d12b0 0x7fc8e80b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.624+0000 7fc8e7e00640 1 -- 172.21.15.59:0/3331342270 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087811378 0 0) 0x7fc8e8163320 con 0x7fc8e80d0ed0 2024-04-23T14:32:30.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.624+0000 7fc8ee400640 1 -- 172.21.15.59:0/3331342270 >> v1:172.21.15.76:6789/0 conn(0x7fc8e815ecc0 legacy=0x7fc8e8162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:30.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.624+0000 7fc8ee400640 1 -- 172.21.15.59:0/3331342270 >> v1:172.21.15.59:6789/0 conn(0x7fc8e80d0ed0 legacy=0x7fc8e815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.624+0000 7fc8ee400640 1 -- 172.21.15.59:0/3331342270 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e80b2a50 con 0x7fc8e80d12b0 2024-04-23T14:32:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.625+0000 7fc8ee400640 1 --2- 172.21.15.59:0/3331342270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e80b3250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009d90 tx=0x7fc8e002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cd8bea5007419b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.625+0000 7fc8e7e00640 1 -- 172.21.15.59:0/3331342270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0007740 con 0x7fc8e80d12b0 2024-04-23T14:32:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.625+0000 7fc8e7e00640 1 -- 172.21.15.59:0/3331342270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e00078e0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.625+0000 7fc8e7e00640 1 -- 172.21.15.59:0/3331342270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0007cb0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.626+0000 7fc8f0400640 1 -- 172.21.15.59:0/3331342270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 msgr2=0x7fc8e80b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.626+0000 7fc8f0400640 1 --2- 172.21.15.59:0/3331342270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e80b3250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009d90 tx=0x7fc8e002a730 comp rx=0 tx=0).stop 2024-04-23T14:32:30.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.626+0000 7fc8f0400640 1 -- 172.21.15.59:0/3331342270 shutdown_connections 2024-04-23T14:32:30.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.626+0000 7fc8f0400640 1 --2- 172.21.15.59:0/3331342270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e80b3250 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.626+0000 7fc8f0400640 1 -- 172.21.15.59:0/3331342270 >> 172.21.15.59:0/3331342270 conn(0x7fc8e80d2680 msgr2=0x7fc8e80d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:30.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.626+0000 7fc8f0400640 1 -- 172.21.15.59:0/3331342270 shutdown_connections 2024-04-23T14:32:30.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.626+0000 7fc8f0400640 1 -- 172.21.15.59:0/3331342270 wait complete. 2024-04-23T14:32:30.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.628+0000 7fc8f0400640 1 Processor -- start 2024-04-23T14:32:30.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.628+0000 7fc8f0400640 1 -- start start 2024-04-23T14:32:30.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8f0400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e80d0ed0 0x7fc8e8155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8f0400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e81f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8f0400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e815ecc0 0x7fc8e815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8f0400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e806de90 con 0x7fc8e80d12b0 2024-04-23T14:32:30.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8f0400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8162fc0 con 0x7fc8e815ecc0 2024-04-23T14:32:30.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8f0400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e806e010 con 0x7fc8e80d0ed0 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8ed000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e81f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8ed000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e81f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41222/0 (socket says 172.21.15.59:41222) 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8ed000640 1 -- 172.21.15.59:0/4291676087 learned_addr learned my addr 172.21.15.59:0/4291676087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.629+0000 7fc8eda00640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e80d0ed0 0x7fc8e8155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ee400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e815ecc0 0x7fc8e815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ed000640 1 -- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e80d0ed0 msgr2=0x7fc8e8155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ed000640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e80d0ed0 0x7fc8e8155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ed000640 1 -- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e815ecc0 msgr2=0x7fc8e815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ed000640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e815ecc0 0x7fc8e815fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ed000640 1 -- 172.21.15.59:0/4291676087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e80c6510 con 0x7fc8e80d12b0 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8eda00640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e80d0ed0 0x7fc8e8155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ee400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e815ecc0 0x7fc8e815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8ed000640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e81f4870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800b3f0 tx=0x7fc8d800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8e5600640 1 -- 172.21.15.59:0/4291676087 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8015020 con 0x7fc8e80d12b0 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8e5600640 1 -- 172.21.15.59:0/4291676087 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d8011830 con 0x7fc8e80d12b0 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8e5600640 1 -- 172.21.15.59:0/4291676087 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8011c20 con 0x7fc8e80d12b0 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e80c67f0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.630+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e80c6db0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.632+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e8069190 con 0x7fc8e80d12b0 2024-04-23T14:32:30.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.633+0000 7fc8e5600640 1 -- 172.21.15.59:0/4291676087 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8d8018020 con 0x7fc8e80d12b0 2024-04-23T14:32:30.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.634+0000 7fc8e5600640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8cc06c8f0 0x7fc8cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:30.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.634+0000 7fc8e5600640 1 -- 172.21.15.59:0/4291676087 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc8d808b8b0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.634+0000 7fc8eda00640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8cc06c8f0 0x7fc8cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:30.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.635+0000 7fc8eda00640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8cc06c8f0 0x7fc8cc06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000f60 tx=0x7fc8dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:30.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.638+0000 7fc8e5600640 1 -- 172.21.15.59:0/4291676087 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8d801f030 con 0x7fc8e80d12b0 2024-04-23T14:32:30.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.844+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8e806cde0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.845+0000 7fc8e5600640 1 -- 172.21.15.59:0/4291676087 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fc8d80172c0 con 0x7fc8e80d12b0 2024-04-23T14:32:30.846 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:30.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8cc06c8f0 msgr2=0x7fc8cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8cc06c8f0 0x7fc8cc06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000f60 tx=0x7fc8dc009040 comp rx=0 tx=0).stop 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 msgr2=0x7fc8e81f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e81f4870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800b3f0 tx=0x7fc8d800b8b0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 shutdown_connections 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8cc06c8f0 0x7fc8cc06edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e815ecc0 0x7fc8e815fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e80d12b0 0x7fc8e81f4870 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 --2- 172.21.15.59:0/4291676087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e80d0ed0 0x7fc8e8155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 >> 172.21.15.59:0/4291676087 conn(0x7fc8e80d2680 msgr2=0x7fc8e80d18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 shutdown_connections 2024-04-23T14:32:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:30.848+0000 7fc8f0400640 1 -- 172.21.15.59:0/4291676087 wait complete. 2024-04-23T14:32:30.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:31.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.085+0000 7f2771200640 1 Processor -- start 2024-04-23T14:32:31.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.085+0000 7f2771200640 1 -- start start 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.086+0000 7f2771200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f276c155a80 0x7f276c06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.086+0000 7f2771200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f276c0d0600 0x7f276c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.086+0000 7f2771200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276c15ef20 con 0x7f276c0b43a0 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.086+0000 7f2771200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f276c15f1c0 con 0x7f276c155a80 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.086+0000 7f2771200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f276c15f4a0 con 0x7f276c0d0600 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.087+0000 7f276b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f276c155a80 0x7f276c06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.087+0000 7f276b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f276c155a80 0x7f276c06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41226/0 (socket says 172.21.15.59:41226) 2024-04-23T14:32:31.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.087+0000 7f276b400640 1 -- 172.21.15.59:0/4184211394 learned_addr learned my addr 172.21.15.59:0/4184211394 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.087+0000 7f276aa00640 1 --2- 172.21.15.59:0/4184211394 >> v2:172.21.15.76:3300/0 conn(0x7f276c0d0600 0x7f276c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.087+0000 7f276b400640 1 -- 172.21.15.59:0/4184211394 >> v2:172.21.15.76:3300/0 conn(0x7f276c0d0600 msgr2=0x7f276c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.087+0000 7f276b400640 1 --2- 172.21.15.59:0/4184211394 >> v2:172.21.15.76:3300/0 conn(0x7f276c0d0600 0x7f276c0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.088+0000 7f276b400640 1 -- 172.21.15.59:0/4184211394 >> v1:172.21.15.76:6789/0 conn(0x7f276c0b43a0 legacy=0x7f276c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.088+0000 7f276b400640 1 -- 172.21.15.59:0/4184211394 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c15f780 con 0x7f276c155a80 2024-04-23T14:32:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.089+0000 7f276b400640 1 --2- 172.21.15.59:0/4184211394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c06f9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f276c0cfcf0 tx=0x7f275c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2f3b5151033f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.089+0000 7f276a000640 1 -- 172.21.15.59:0/4184211394 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c036070 con 0x7f276c155a80 2024-04-23T14:32:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.089+0000 7f276a000640 1 -- 172.21.15.59:0/4184211394 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c031600 con 0x7f276c155a80 2024-04-23T14:32:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.089+0000 7f276a000640 1 -- 172.21.15.59:0/4184211394 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c0319d0 con 0x7f276c155a80 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 -- 172.21.15.59:0/4184211394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 msgr2=0x7f276c06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 --2- 172.21.15.59:0/4184211394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c06f9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f276c0cfcf0 tx=0x7f275c02a6f0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 -- 172.21.15.59:0/4184211394 shutdown_connections 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 --2- 172.21.15.59:0/4184211394 >> v2:172.21.15.76:3300/0 conn(0x7f276c0d0600 0x7f276c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 --2- 172.21.15.59:0/4184211394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c06f9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 -- 172.21.15.59:0/4184211394 >> 172.21.15.59:0/4184211394 conn(0x7f276c06bdf0 msgr2=0x7f276c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 -- 172.21.15.59:0/4184211394 shutdown_connections 2024-04-23T14:32:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.090+0000 7f2771200640 1 -- 172.21.15.59:0/4184211394 wait complete. 2024-04-23T14:32:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.092+0000 7f2771200640 1 Processor -- start 2024-04-23T14:32:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.092+0000 7f2771200640 1 -- start start 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f2771200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f276c0b43a0 0x7f276c0b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f2771200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c0d0600 0x7f276c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f2771200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f2771200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f276c06ff00 con 0x7f276c155a80 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f2771200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f276c15fd40 con 0x7f276c0d0600 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f2771200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f276c06ef80 con 0x7f276c0b43a0 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f276be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f276be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c0cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41242/0 (socket says 172.21.15.59:41242) 2024-04-23T14:32:31.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f276be00640 1 -- 172.21.15.59:0/1863508215 learned_addr learned my addr 172.21.15.59:0/1863508215 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:31.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f276b400640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f276c0b43a0 0x7f276c0b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.093+0000 7f276aa00640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c0d0600 0x7f276c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f276be00640 1 -- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f276c0b43a0 msgr2=0x7f276c0b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f276be00640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f276c0b43a0 0x7f276c0b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f276be00640 1 -- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c0d0600 msgr2=0x7f276c0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f276be00640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c0d0600 0x7f276c0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f276be00640 1 -- 172.21.15.59:0/1863508215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c15d760 con 0x7f276c155a80 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f276be00640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c0cdf60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f276400c440 tx=0x7f276400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f2763400640 1 -- 172.21.15.59:0/1863508215 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764011790 con 0x7f276c155a80 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f2763400640 1 -- 172.21.15.59:0/1863508215 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2764017070 con 0x7f276c155a80 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f2763400640 1 -- 172.21.15.59:0/1863508215 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764011b70 con 0x7f276c155a80 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c15d480 con 0x7f276c155a80 2024-04-23T14:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.094+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c0b00f0 con 0x7f276c155a80 2024-04-23T14:32:31.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.096+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c04fd90 con 0x7f276c155a80 2024-04-23T14:32:31.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.097+0000 7f2763400640 1 -- 172.21.15.59:0/1863508215 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2764011d10 con 0x7f276c155a80 2024-04-23T14:32:31.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.098+0000 7f2763400640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274c06c8f0 0x7f274c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.098+0000 7f2763400640 1 -- 172.21.15.59:0/1863508215 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f276408b990 con 0x7f276c155a80 2024-04-23T14:32:31.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.098+0000 7f276b400640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274c06c8f0 0x7f274c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.099+0000 7f276b400640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274c06c8f0 0x7f274c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f275c033000 tx=0x7f275c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:31.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.102+0000 7f2763400640 1 -- 172.21.15.59:0/1863508215 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2764059d30 con 0x7f276c155a80 2024-04-23T14:32:31.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.307+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f276c06eba0 con 0x7f276c155a80 2024-04-23T14:32:31.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.308+0000 7f2763400640 1 -- 172.21.15.59:0/1863508215 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f2764059630 con 0x7f276c155a80 2024-04-23T14:32:31.309 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:31.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274c06c8f0 msgr2=0x7f274c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274c06c8f0 0x7f274c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f275c033000 tx=0x7f275c009000 comp rx=0 tx=0).stop 2024-04-23T14:32:31.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 msgr2=0x7f276c0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c0cdf60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f276400c440 tx=0x7f276400c900 comp rx=0 tx=0).stop 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 shutdown_connections 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274c06c8f0 0x7f274c06edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c155a80 0x7f276c0cdf60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c0d0600 0x7f276c0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 --2- 172.21.15.59:0/1863508215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f276c0b43a0 0x7f276c0b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 >> 172.21.15.59:0/1863508215 conn(0x7f276c06bdf0 msgr2=0x7f276c0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 shutdown_connections 2024-04-23T14:32:31.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.311+0000 7f2771200640 1 -- 172.21.15.59:0/1863508215 wait complete. 2024-04-23T14:32:31.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:32:31.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.541+0000 7fb1a1000640 1 Processor -- start 2024-04-23T14:32:31.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.541+0000 7fb1a1000640 1 -- start start 2024-04-23T14:32:31.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.542+0000 7fb1a1000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb19c15ea40 0x7fb19c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.542+0000 7fb1a1000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.542+0000 7fb1a1000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb19c15fd30 con 0x7fb19c0d0f80 2024-04-23T14:32:31.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.542+0000 7fb1a1000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb19c0cf930 con 0x7fb19c15ea40 2024-04-23T14:32:31.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.542+0000 7fb1a1000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb19c0cfc10 con 0x7fb19c15f880 2024-04-23T14:32:31.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.543+0000 7fb19be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.543+0000 7fb19b400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb19c0d0f80 0x7fb19c0d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:57988/0 (socket says 172.21.15.59:57988) 2024-04-23T14:32:31.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.543+0000 7fb19b400640 1 -- 172.21.15.59:0/2681643946 learned_addr learned my addr 172.21.15.59:0/2681643946 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:31.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.543+0000 7fb19aa00640 1 --2- 172.21.15.59:0/2681643946 >> v2:172.21.15.76:3300/0 conn(0x7fb19c15ea40 0x7fb19c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.544+0000 7fb19a000640 1 -- 172.21.15.59:0/2681643946 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 527033517 0 0) 0x7fb19c15fd30 con 0x7fb19c0d0f80 2024-04-23T14:32:31.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.544+0000 7fb19a000640 1 -- 172.21.15.59:0/2681643946 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb180003520 con 0x7fb19c0d0f80 2024-04-23T14:32:31.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.544+0000 7fb19be00640 1 -- 172.21.15.59:0/2681643946 >> v2:172.21.15.76:3300/0 conn(0x7fb19c15ea40 msgr2=0x7fb19c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:31.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.544+0000 7fb19be00640 1 --2- 172.21.15.59:0/2681643946 >> v2:172.21.15.76:3300/0 conn(0x7fb19c15ea40 0x7fb19c0cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.544+0000 7fb19be00640 1 -- 172.21.15.59:0/2681643946 >> v1:172.21.15.76:6789/0 conn(0x7fb19c0d0f80 legacy=0x7fb19c0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.544+0000 7fb19be00640 1 -- 172.21.15.59:0/2681643946 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c0cfef0 con 0x7fb19c15f880 2024-04-23T14:32:31.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.544+0000 7fb19aa00640 1 --2- 172.21.15.59:0/2681643946 >> v2:172.21.15.76:3300/0 conn(0x7fb19c15ea40 0x7fb19c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:31.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.546+0000 7fb19be00640 1 --2- 172.21.15.59:0/2681643946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c0d2310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb194009d90 tx=0x7fb19402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312a71166e23e9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:31.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.546+0000 7fb19a000640 1 -- 172.21.15.59:0/2681643946 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194031600 con 0x7fb19c15f880 2024-04-23T14:32:31.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.546+0000 7fb19a000640 1 -- 172.21.15.59:0/2681643946 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1940317a0 con 0x7fb19c15f880 2024-04-23T14:32:31.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.546+0000 7fb19a000640 1 -- 172.21.15.59:0/2681643946 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194031b70 con 0x7fb19c15f880 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 -- 172.21.15.59:0/2681643946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 msgr2=0x7fb19c0d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 --2- 172.21.15.59:0/2681643946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c0d2310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb194009d90 tx=0x7fb19402a5d0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 -- 172.21.15.59:0/2681643946 shutdown_connections 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 --2- 172.21.15.59:0/2681643946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c0d2310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 --2- 172.21.15.59:0/2681643946 >> v2:172.21.15.76:3300/0 conn(0x7fb19c15ea40 0x7fb19c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 -- 172.21.15.59:0/2681643946 >> 172.21.15.59:0/2681643946 conn(0x7fb19c08bf90 msgr2=0x7fb19c0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 -- 172.21.15.59:0/2681643946 shutdown_connections 2024-04-23T14:32:31.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.547+0000 7fb1a1000640 1 -- 172.21.15.59:0/2681643946 wait complete. 2024-04-23T14:32:31.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.549+0000 7fb1a1000640 1 Processor -- start 2024-04-23T14:32:31.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.550+0000 7fb1a1000640 1 -- start start 2024-04-23T14:32:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.551+0000 7fb1a1000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c0d0f80 0x7fb19c0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.551+0000 7fb1a1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb19c15ea40 0x7fb19c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.551+0000 7fb1a1000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c14f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.551+0000 7fb1a1000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c0d2850 con 0x7fb19c15ea40 2024-04-23T14:32:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.551+0000 7fb1a1000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c0d04b0 con 0x7fb19c0d0f80 2024-04-23T14:32:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.551+0000 7fb1a1000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb19c0d29d0 con 0x7fb19c15f880 2024-04-23T14:32:31.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb19c15ea40 0x7fb19c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb19c15ea40 0x7fb19c0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41244/0 (socket says 172.21.15.59:41244) 2024-04-23T14:32:31.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 -- 172.21.15.59:0/1208763407 learned_addr learned my addr 172.21.15.59:0/1208763407 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:31.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19b400640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c0d0f80 0x7fb19c0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19be00640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c14f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 -- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 msgr2=0x7fb19c14f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c14f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 -- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c0d0f80 msgr2=0x7fb19c0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c0d0f80 0x7fb19c0b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.552+0000 7fb19aa00640 1 -- 172.21.15.59:0/1208763407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c1f7cd0 con 0x7fb19c15ea40 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.553+0000 7fb19aa00640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb19c15ea40 0x7fb19c0b1d50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb18800e980 tx=0x7fb18800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.553+0000 7fb193400640 1 -- 172.21.15.59:0/1208763407 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188017070 con 0x7fb19c15ea40 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.553+0000 7fb193400640 1 -- 172.21.15.59:0/1208763407 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18800ca00 con 0x7fb19c15ea40 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.553+0000 7fb193400640 1 -- 172.21.15.59:0/1208763407 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18800cdb0 con 0x7fb19c15ea40 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.553+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19c1f7fb0 con 0x7fb19c15ea40 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.553+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19c1f8570 con 0x7fb19c15ea40 2024-04-23T14:32:31.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.555+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19c04fd90 con 0x7fb19c15ea40 2024-04-23T14:32:31.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.556+0000 7fb193400640 1 -- 172.21.15.59:0/1208763407 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb18801a070 con 0x7fb19c15ea40 2024-04-23T14:32:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.556+0000 7fb193400640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb17c06c920 0x7fb17c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:31.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.557+0000 7fb193400640 1 -- 172.21.15.59:0/1208763407 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb18808bdc0 con 0x7fb19c15ea40 2024-04-23T14:32:31.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.557+0000 7fb19b400640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb17c06c920 0x7fb17c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:31.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.558+0000 7fb19b400640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb17c06c920 0x7fb17c06ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005880 tx=0x7fb18c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:31.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.562+0000 7fb193400640 1 -- 172.21.15.59:0/1208763407 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb188059b10 con 0x7fb19c15ea40 2024-04-23T14:32:31.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.782+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb19c06ead0 con 0x7fb19c15ea40 2024-04-23T14:32:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.783+0000 7fb193400640 1 -- 172.21.15.59:0/1208763407 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fb18805de10 con 0x7fb19c15ea40 2024-04-23T14:32:31.784 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:32:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb17c06c920 msgr2=0x7fb17c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb17c06c920 0x7fb17c06ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005880 tx=0x7fb18c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:31.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb19c15ea40 msgr2=0x7fb19c0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb19c15ea40 0x7fb19c0b1d50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb18800e980 tx=0x7fb18800ee40 comp rx=0 tx=0).stop 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 shutdown_connections 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb17c06c920 0x7fb17c06ede0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb19c15f880 0x7fb19c14f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb19c15ea40 0x7fb19c0b1d50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 --2- 172.21.15.59:0/1208763407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb19c0d0f80 0x7fb19c0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 >> 172.21.15.59:0/1208763407 conn(0x7fb19c08bf90 msgr2=0x7fb19c161470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 shutdown_connections 2024-04-23T14:32:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:31.786+0000 7fb1a1000640 1 -- 172.21.15.59:0/1208763407 wait complete. 2024-04-23T14:32:31.807 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-04-23T14:32:31.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:31.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-23T14:32:31.891 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:31.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:32:31.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:32:31.891 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:32:31.942 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:32:31.942 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:32.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.161+0000 7fce41800640 1 Processor -- start 2024-04-23T14:32:32.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.162+0000 7fce41800640 1 -- start start 2024-04-23T14:32:32.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.163+0000 7fce41800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.163+0000 7fce41800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fce3c0d28b0 0x7fce3c15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.163+0000 7fce41800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce3c0d17e0 con 0x7fce3c0cff10 2024-04-23T14:32:32.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.163+0000 7fce41800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fce3c0b3b60 con 0x7fce3c0d28b0 2024-04-23T14:32:32.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.163+0000 7fce41800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c15fd00 con 0x7fce3c0d0ec0 2024-04-23T14:32:32.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce40e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fce3c0d28b0 0x7fce3c15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce40e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fce3c0d28b0 0x7fce3c15eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41250/0 (socket says 172.21.15.59:41250) 2024-04-23T14:32:32.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce40e00640 1 -- 172.21.15.59:0/1867787762 learned_addr learned my addr 172.21.15.59:0/1867787762 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:32.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce3be00640 1 --2- 172.21.15.59:0/1867787762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce40e00640 1 -- 172.21.15.59:0/1867787762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 msgr2=0x7fce3c0d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce40e00640 1 --2- 172.21.15.59:0/1867787762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce40e00640 1 -- 172.21.15.59:0/1867787762 >> v1:172.21.15.76:6789/0 conn(0x7fce3c0cff10 legacy=0x7fce3c0d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.164+0000 7fce40e00640 1 -- 172.21.15.59:0/1867787762 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c15fa20 con 0x7fce3c0d28b0 2024-04-23T14:32:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.165+0000 7fce40e00640 1 --2- 172.21.15.59:0/1867787762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c15eb10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce34009e50 tx=0x7fce3402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d91ce0188df9d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce3aa00640 1 -- 172.21.15.59:0/1867787762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34036070 con 0x7fce3c0d28b0 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce3aa00640 1 -- 172.21.15.59:0/1867787762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce34031600 con 0x7fce3c0d28b0 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce3aa00640 1 -- 172.21.15.59:0/1867787762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340319d0 con 0x7fce3c0d28b0 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce41800640 1 -- 172.21.15.59:0/1867787762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 msgr2=0x7fce3c15eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce41800640 1 --2- 172.21.15.59:0/1867787762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c15eb10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce34009e50 tx=0x7fce3402a750 comp rx=0 tx=0).stop 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce41800640 1 -- 172.21.15.59:0/1867787762 shutdown_connections 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce41800640 1 --2- 172.21.15.59:0/1867787762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c15eb10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce41800640 1 --2- 172.21.15.59:0/1867787762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.166+0000 7fce41800640 1 -- 172.21.15.59:0/1867787762 >> 172.21.15.59:0/1867787762 conn(0x7fce3c08bf90 msgr2=0x7fce3c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.167+0000 7fce41800640 1 -- 172.21.15.59:0/1867787762 shutdown_connections 2024-04-23T14:32:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.167+0000 7fce41800640 1 -- 172.21.15.59:0/1867787762 wait complete. 2024-04-23T14:32:32.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.168+0000 7fce41800640 1 Processor -- start 2024-04-23T14:32:32.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.168+0000 7fce41800640 1 -- start start 2024-04-23T14:32:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.169+0000 7fce41800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c0cff10 0x7fce3c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.169+0000 7fce41800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.169+0000 7fce41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c14f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.169+0000 7fce41800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0cf9f0 con 0x7fce3c0d28b0 2024-04-23T14:32:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.169+0000 7fce41800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0cf710 con 0x7fce3c0cff10 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce41800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c0d0330 con 0x7fce3c0d0ec0 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c14f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c14f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41252/0 (socket says 172.21.15.59:41252) 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 -- 172.21.15.59:0/3025187680 learned_addr learned my addr 172.21.15.59:0/3025187680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce3be00640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c0cff10 0x7fce3c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce3b400640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 -- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 msgr2=0x7fce3c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 -- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c0cff10 msgr2=0x7fce3c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c0cff10 0x7fce3c0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.170+0000 7fce40e00640 1 -- 172.21.15.59:0/3025187680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c153000 con 0x7fce3c0d28b0 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.171+0000 7fce40e00640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c14f250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fce34009e50 tx=0x7fce34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.171+0000 7fce33e00640 1 -- 172.21.15.59:0/3025187680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34041020 con 0x7fce3c0d28b0 2024-04-23T14:32:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.171+0000 7fce33e00640 1 -- 172.21.15.59:0/3025187680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce340077a0 con 0x7fce3c0d28b0 2024-04-23T14:32:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.171+0000 7fce33e00640 1 -- 172.21.15.59:0/3025187680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34036040 con 0x7fce3c0d28b0 2024-04-23T14:32:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.171+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c0c9680 con 0x7fce3c0d28b0 2024-04-23T14:32:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.171+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c14d460 con 0x7fce3c0d28b0 2024-04-23T14:32:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.172+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3c04fd90 con 0x7fce3c0d28b0 2024-04-23T14:32:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.173+0000 7fce33e00640 1 -- 172.21.15.59:0/3025187680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce34044020 con 0x7fce3c0d28b0 2024-04-23T14:32:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.174+0000 7fce33e00640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fce1806c8f0 0x7fce1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.174+0000 7fce33e00640 1 -- 172.21.15.59:0/3025187680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fce340ac930 con 0x7fce3c0d28b0 2024-04-23T14:32:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.175+0000 7fce3be00640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fce1806c8f0 0x7fce1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.175+0000 7fce3be00640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fce1806c8f0 0x7fce1806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006fd0 tx=0x7fce2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.179+0000 7fce33e00640 1 -- 172.21.15.59:0/3025187680 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce3404b030 con 0x7fce3c0d28b0 2024-04-23T14:32:32.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.423+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce3c0cebb0 con 0x7fce3c0d28b0 2024-04-23T14:32:32.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.425+0000 7fce33e00640 1 -- 172.21.15.59:0/3025187680 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7fce3407ac40 con 0x7fce3c0d28b0 2024-04-23T14:32:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:32.426 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fce1806c8f0 msgr2=0x7fce1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fce1806c8f0 0x7fce1806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006fd0 tx=0x7fce2c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 msgr2=0x7fce3c14f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c14f250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fce34009e50 tx=0x7fce34004a40 comp rx=0 tx=0).stop 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 shutdown_connections 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fce1806c8f0 0x7fce1806edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce3c0d0ec0 0x7fce3c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce3c0d28b0 0x7fce3c14f250 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 --2- 172.21.15.59:0/3025187680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c0cff10 0x7fce3c0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.427+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 >> 172.21.15.59:0/3025187680 conn(0x7fce3c08bf90 msgr2=0x7fce3c1614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.428+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 shutdown_connections 2024-04-23T14:32:32.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.428+0000 7fce41800640 1 -- 172.21.15.59:0/3025187680 wait complete. 2024-04-23T14:32:32.449 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.663+0000 7fcadfe00640 1 Processor -- start 2024-04-23T14:32:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.664+0000 7fcadfe00640 1 -- start start 2024-04-23T14:32:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcae015c3d0 0x7fcae00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadfe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae0160210 con 0x7fcae00cd590 2024-04-23T14:32:32.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadfe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcae01604f0 con 0x7fcae015c3d0 2024-04-23T14:32:32.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcae01607d0 con 0x7fcae00b4860 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadea00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcae015c3d0 0x7fcae00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadf400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadea00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcae015c3d0 0x7fcae00cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:53272/0 (socket says 172.21.15.59:53272) 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.665+0000 7fcadea00640 1 -- 172.21.15.59:0/1305344179 learned_addr learned my addr 172.21.15.59:0/1305344179 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.666+0000 7fcadf400640 1 -- 172.21.15.59:0/1305344179 >> v2:172.21.15.76:3300/0 conn(0x7fcae015c3d0 msgr2=0x7fcae00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.666+0000 7fcadf400640 1 --2- 172.21.15.59:0/1305344179 >> v2:172.21.15.76:3300/0 conn(0x7fcae015c3d0 0x7fcae00cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.666+0000 7fcadf400640 1 -- 172.21.15.59:0/1305344179 >> v1:172.21.15.76:6789/0 conn(0x7fcae00cd590 legacy=0x7fcae00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.666+0000 7fcadf400640 1 -- 172.21.15.59:0/1305344179 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae0160ab0 con 0x7fcae00b4860 2024-04-23T14:32:32.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.668+0000 7fcadf400640 1 --2- 172.21.15.59:0/1305344179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cb7d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcad80097a0 tx=0x7fcad802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0eb0cf2ab953c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadd600640 1 -- 172.21.15.59:0/1305344179 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8037070 con 0x7fcae00b4860 2024-04-23T14:32:32.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadd600640 1 -- 172.21.15.59:0/1305344179 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad8032440 con 0x7fcae00b4860 2024-04-23T14:32:32.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadd600640 1 -- 172.21.15.59:0/1305344179 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8032810 con 0x7fcae00b4860 2024-04-23T14:32:32.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadfe00640 1 -- 172.21.15.59:0/1305344179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 msgr2=0x7fcae00cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadfe00640 1 --2- 172.21.15.59:0/1305344179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cb7d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcad80097a0 tx=0x7fcad802a180 comp rx=0 tx=0).stop 2024-04-23T14:32:32.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadfe00640 1 -- 172.21.15.59:0/1305344179 shutdown_connections 2024-04-23T14:32:32.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadfe00640 1 --2- 172.21.15.59:0/1305344179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cb7d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadfe00640 1 --2- 172.21.15.59:0/1305344179 >> v2:172.21.15.76:3300/0 conn(0x7fcae015c3d0 0x7fcae00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.669+0000 7fcadfe00640 1 -- 172.21.15.59:0/1305344179 >> 172.21.15.59:0/1305344179 conn(0x7fcae00b21a0 msgr2=0x7fcae00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.670+0000 7fcadfe00640 1 -- 172.21.15.59:0/1305344179 shutdown_connections 2024-04-23T14:32:32.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.670+0000 7fcadfe00640 1 -- 172.21.15.59:0/1305344179 wait complete. 2024-04-23T14:32:32.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.671+0000 7fcadfe00640 1 Processor -- start 2024-04-23T14:32:32.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.672+0000 7fcadfe00640 1 -- start start 2024-04-23T14:32:32.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.672+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae00cd590 0x7fcae015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcadfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae015c3d0 0x7fcae015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcadfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcae015c930 con 0x7fcae00cd590 2024-04-23T14:32:32.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcadfe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcae015c7b0 con 0x7fcae015c3d0 2024-04-23T14:32:32.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcadfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00cbd10 con 0x7fcae00b4860 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcade000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae00cd590 0x7fcae015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcade000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae00cd590 0x7fcae015ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41262/0 (socket says 172.21.15.59:41262) 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcade000640 1 -- 172.21.15.59:0/3562986312 learned_addr learned my addr 172.21.15.59:0/3562986312 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcadf400640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae015c3d0 0x7fcae015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.673+0000 7fcadea00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcade000640 1 -- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 msgr2=0x7fcae00cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcade000640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcade000640 1 -- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae015c3d0 msgr2=0x7fcae015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcade000640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae015c3d0 0x7fcae015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcade000640 1 -- 172.21.15.59:0/3562986312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae00ce830 con 0x7fcae00cd590 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcade000640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae00cd590 0x7fcae015ae90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcac800c400 tx=0x7fcac800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcad6a00640 1 -- 172.21.15.59:0/3562986312 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac80052d0 con 0x7fcae00cd590 2024-04-23T14:32:32.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.674+0000 7fcad6a00640 1 -- 172.21.15.59:0/3562986312 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac8005490 con 0x7fcae00cd590 2024-04-23T14:32:32.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.675+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae01e6c10 con 0x7fcae00cd590 2024-04-23T14:32:32.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.675+0000 7fcad6a00640 1 -- 172.21.15.59:0/3562986312 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8007260 con 0x7fcae00cd590 2024-04-23T14:32:32.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.675+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae00cedb0 con 0x7fcae00cd590 2024-04-23T14:32:32.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.676+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae004fd90 con 0x7fcae00cd590 2024-04-23T14:32:32.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.677+0000 7fcad6a00640 1 -- 172.21.15.59:0/3562986312 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcac8007400 con 0x7fcae00cd590 2024-04-23T14:32:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.678+0000 7fcad6a00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcabc06c940 0x7fcabc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:32.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.678+0000 7fcad6a00640 1 -- 172.21.15.59:0/3562986312 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fcac808afe0 con 0x7fcae00cd590 2024-04-23T14:32:32.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.678+0000 7fcadea00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcabc06c940 0x7fcabc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:32.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.679+0000 7fcadea00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcabc06c940 0x7fcabc06ee00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcad0011340 tx=0x7fcad000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.682+0000 7fcad6a00640 1 -- 172.21.15.59:0/3562986312 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcac8058e20 con 0x7fcae00cd590 2024-04-23T14:32:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.891+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcae0073b00 con 0x7fcae00cd590 2024-04-23T14:32:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.894+0000 7fcad6a00640 1 -- 172.21.15.59:0/3562986312 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fcac805d120 con 0x7fcae00cd590 2024-04-23T14:32:32.895 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:32.896 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.897+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcabc06c940 msgr2=0x7fcabc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.897+0000 7fcadfe00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcabc06c940 0x7fcabc06ee00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcad0011340 tx=0x7fcad000b040 comp rx=0 tx=0).stop 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.897+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae00cd590 msgr2=0x7fcae015ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.897+0000 7fcadfe00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae00cd590 0x7fcae015ae90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcac800c400 tx=0x7fcac800c8c0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.897+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 shutdown_connections 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.897+0000 7fcadfe00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcabc06c940 0x7fcabc06ee00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.897+0000 7fcadfe00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae015c3d0 0x7fcae015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.898+0000 7fcadfe00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae00cd590 0x7fcae015ae90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.898+0000 7fcadfe00640 1 --2- 172.21.15.59:0/3562986312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00b4860 0x7fcae00cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.898+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 >> 172.21.15.59:0/3562986312 conn(0x7fcae00b21a0 msgr2=0x7fcae00cc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.898+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 shutdown_connections 2024-04-23T14:32:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:32.898+0000 7fcadfe00640 1 -- 172.21.15.59:0/3562986312 wait complete. 2024-04-23T14:32:32.918 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:31:23.479186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:31:24.232329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:32.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.128+0000 7f35cfe00640 1 Processor -- start 2024-04-23T14:32:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.128+0000 7f35cfe00640 1 -- start start 2024-04-23T14:32:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.129+0000 7f35cfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35d00d15b0 0x7f35d00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.129+0000 7f35cfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f35d00b4f20 0x7f35d00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.129+0000 7f35cfe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d00b5da0 con 0x7f35d015ed20 2024-04-23T14:32:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.129+0000 7f35cfe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f35d015f100 con 0x7f35d00d15b0 2024-04-23T14:32:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.129+0000 7f35cfe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f35d00d0a20 con 0x7f35d00b4f20 2024-04-23T14:32:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35ce000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35d00d15b0 0x7f35d00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35ce000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35d00d15b0 0x7f35d00cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41274/0 (socket says 172.21.15.59:41274) 2024-04-23T14:32:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35ce000640 1 -- 172.21.15.59:0/1098516553 learned_addr learned my addr 172.21.15.59:0/1098516553 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:33.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35cf400640 1 --2- 172.21.15.59:0/1098516553 >> v2:172.21.15.76:3300/0 conn(0x7f35d00b4f20 0x7f35d00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35ce000640 1 -- 172.21.15.59:0/1098516553 >> v2:172.21.15.76:3300/0 conn(0x7f35d00b4f20 msgr2=0x7f35d00cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:33.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35ce000640 1 --2- 172.21.15.59:0/1098516553 >> v2:172.21.15.76:3300/0 conn(0x7f35d00b4f20 0x7f35d00cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35ce000640 1 -- 172.21.15.59:0/1098516553 >> v1:172.21.15.76:6789/0 conn(0x7f35d015ed20 legacy=0x7f35d00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:33.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.130+0000 7f35ce000640 1 -- 172.21.15.59:0/1098516553 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d00d2ed0 con 0x7f35d00d15b0 2024-04-23T14:32:33.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.132+0000 7f35ce000640 1 --2- 172.21.15.59:0/1098516553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00cf280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f35d008dad0 tx=0x7f35b802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f08220a81c68e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.132+0000 7f35cd600640 1 -- 172.21.15.59:0/1098516553 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8037070 con 0x7f35d00d15b0 2024-04-23T14:32:33.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.132+0000 7f35cd600640 1 -- 172.21.15.59:0/1098516553 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b802ae10 con 0x7f35d00d15b0 2024-04-23T14:32:33.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.132+0000 7f35cd600640 1 -- 172.21.15.59:0/1098516553 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8032660 con 0x7f35d00d15b0 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 -- 172.21.15.59:0/1098516553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 msgr2=0x7f35d00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 --2- 172.21.15.59:0/1098516553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00cf280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f35d008dad0 tx=0x7f35b802a100 comp rx=0 tx=0).stop 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 -- 172.21.15.59:0/1098516553 shutdown_connections 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 --2- 172.21.15.59:0/1098516553 >> v2:172.21.15.76:3300/0 conn(0x7f35d00b4f20 0x7f35d00cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 --2- 172.21.15.59:0/1098516553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00cf280 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 -- 172.21.15.59:0/1098516553 >> 172.21.15.59:0/1098516553 conn(0x7f35d008bf90 msgr2=0x7f35d00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 -- 172.21.15.59:0/1098516553 shutdown_connections 2024-04-23T14:32:33.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.133+0000 7f35cfe00640 1 -- 172.21.15.59:0/1098516553 wait complete. 2024-04-23T14:32:33.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.135+0000 7f35cfe00640 1 Processor -- start 2024-04-23T14:32:33.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.135+0000 7f35cfe00640 1 -- start start 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00b4f20 0x7f35d00c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d015ed20 0x7f35d00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35d00b3600 con 0x7f35d00d15b0 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cfe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35d00d3490 con 0x7f35d00b4f20 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f35d00b3780 con 0x7f35d015ed20 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35ce000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35ce000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41288/0 (socket says 172.21.15.59:41288) 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35ce000640 1 -- 172.21.15.59:0/307676661 learned_addr learned my addr 172.21.15.59:0/307676661 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:33.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cea00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00b4f20 0x7f35d00c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.136+0000 7f35cf400640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d015ed20 0x7f35d00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35ce000640 1 -- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d015ed20 msgr2=0x7f35d00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35ce000640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d015ed20 0x7f35d00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35ce000640 1 -- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00b4f20 msgr2=0x7f35d00c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35ce000640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00b4f20 0x7f35d00c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35ce000640 1 -- 172.21.15.59:0/307676661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d00b8340 con 0x7f35d00d15b0 2024-04-23T14:32:33.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35ce000640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00b7a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000c00 tx=0x7f35b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35c6a00640 1 -- 172.21.15.59:0/307676661 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8041070 con 0x7f35d00d15b0 2024-04-23T14:32:33.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35c6a00640 1 -- 172.21.15.59:0/307676661 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b803c670 con 0x7f35d00d15b0 2024-04-23T14:32:33.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.137+0000 7f35c6a00640 1 -- 172.21.15.59:0/307676661 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8037040 con 0x7f35d00d15b0 2024-04-23T14:32:33.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.138+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d00b8620 con 0x7f35d00d15b0 2024-04-23T14:32:33.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.138+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d00c9420 con 0x7f35d00d15b0 2024-04-23T14:32:33.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.139+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d004fd90 con 0x7f35d00d15b0 2024-04-23T14:32:33.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.140+0000 7f35c6a00640 1 -- 172.21.15.59:0/307676661 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35b8044070 con 0x7f35d00d15b0 2024-04-23T14:32:33.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.141+0000 7f35c6a00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35ac06c8f0 0x7f35ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.141+0000 7f35c6a00640 1 -- 172.21.15.59:0/307676661 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f35b80ac290 con 0x7f35d00d15b0 2024-04-23T14:32:33.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.141+0000 7f35cea00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35ac06c8f0 0x7f35ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.142+0000 7f35cea00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35ac06c8f0 0x7f35ac06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35c0004ea0 tx=0x7f35c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.146+0000 7f35c6a00640 1 -- 172.21.15.59:0/307676661 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35b807a5a0 con 0x7f35d00d15b0 2024-04-23T14:32:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.349+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35d006ba80 con 0x7f35d00d15b0 2024-04-23T14:32:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.350+0000 7f35c6a00640 1 -- 172.21.15.59:0/307676661 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f35b8079ea0 con 0x7f35d00d15b0 2024-04-23T14:32:33.351 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35ac06c8f0 msgr2=0x7f35ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35ac06c8f0 0x7f35ac06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35c0004ea0 tx=0x7f35c0009040 comp rx=0 tx=0).stop 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 msgr2=0x7f35d00b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00b7a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000c00 tx=0x7f35b80046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 shutdown_connections 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35ac06c8f0 0x7f35ac06edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d015ed20 0x7f35d00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00d15b0 0x7f35d00b7a40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 --2- 172.21.15.59:0/307676661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00b4f20 0x7f35d00c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.352+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 >> 172.21.15.59:0/307676661 conn(0x7f35d008bf90 msgr2=0x7f35d00cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:33.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.353+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 shutdown_connections 2024-04-23T14:32:33.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.353+0000 7f35cfe00640 1 -- 172.21.15.59:0/307676661 wait complete. 2024-04-23T14:32:33.373 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:32:33.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.583+0000 7f2c06a00640 1 Processor -- start 2024-04-23T14:32:33.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.584+0000 7f2c06a00640 1 -- start start 2024-04-23T14:32:33.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.585+0000 7f2c06a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c0015d300 0x7f2c00071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.585+0000 7f2c06a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c00163320 con 0x7f2c000d0600 2024-04-23T14:32:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.585+0000 7f2c06a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c00072380 con 0x7f2c00167180 2024-04-23T14:32:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.585+0000 7f2c06a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2c00163650 con 0x7f2c0015d300 2024-04-23T14:32:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.586+0000 7f2c05600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c0015d300 0x7f2c00071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.586+0000 7f2c05600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c0015d300 0x7f2c00071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41300/0 (socket says 172.21.15.59:41300) 2024-04-23T14:32:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.586+0000 7f2c05600640 1 -- 172.21.15.59:0/1996208976 learned_addr learned my addr 172.21.15.59:0/1996208976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.586+0000 7f2c05600640 1 -- 172.21.15.59:0/1996208976 >> v1:172.21.15.76:6789/0 conn(0x7f2c00167180 legacy=0x7f2c00163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:33.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.586+0000 7f2c05600640 1 -- 172.21.15.59:0/1996208976 >> v1:172.21.15.59:6789/0 conn(0x7f2c000d0600 legacy=0x7f2c0006e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.586+0000 7f2c05600640 1 -- 172.21.15.59:0/1996208976 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c00072660 con 0x7f2c0015d300 2024-04-23T14:32:33.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.586+0000 7f2bffe00640 1 -- 172.21.15.59:0/1996208976 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035724240 0 0) 0x7f2c00163320 con 0x7f2c000d0600 2024-04-23T14:32:33.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.587+0000 7f2c05600640 1 --2- 172.21.15.59:0/1996208976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c0015d300 0x7f2c00071c70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009870 tx=0x7f2bf402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee530a85626721b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.588+0000 7f2bffe00640 1 -- 172.21.15.59:0/1996208976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4037070 con 0x7f2c0015d300 2024-04-23T14:32:33.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.588+0000 7f2bffe00640 1 -- 172.21.15.59:0/1996208976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf4032440 con 0x7f2c0015d300 2024-04-23T14:32:33.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.588+0000 7f2bffe00640 1 -- 172.21.15.59:0/1996208976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4032810 con 0x7f2c0015d300 2024-04-23T14:32:33.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.588+0000 7f2c06a00640 1 -- 172.21.15.59:0/1996208976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c0015d300 msgr2=0x7f2c00071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.588+0000 7f2c06a00640 1 --2- 172.21.15.59:0/1996208976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c0015d300 0x7f2c00071c70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009870 tx=0x7f2bf402a2a0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.589+0000 7f2c06a00640 1 -- 172.21.15.59:0/1996208976 shutdown_connections 2024-04-23T14:32:33.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.589+0000 7f2c06a00640 1 --2- 172.21.15.59:0/1996208976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c0015d300 0x7f2c00071c70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.589+0000 7f2c06a00640 1 -- 172.21.15.59:0/1996208976 >> 172.21.15.59:0/1996208976 conn(0x7f2c0006bdf0 msgr2=0x7f2c0006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:33.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.589+0000 7f2c06a00640 1 -- 172.21.15.59:0/1996208976 shutdown_connections 2024-04-23T14:32:33.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.589+0000 7f2c06a00640 1 -- 172.21.15.59:0/1996208976 wait complete. 2024-04-23T14:32:33.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.590+0000 7f2c06a00640 1 Processor -- start 2024-04-23T14:32:33.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.591+0000 7f2c06a00640 1 -- start start 2024-04-23T14:32:33.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c000d0600 0x7f2c0014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c0015d300 0x7f2c000b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c00167180 0x7f2c0015e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c06a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0006e8f0 con 0x7f2c000d0600 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c06a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001675e0 con 0x7f2c0015d300 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c06a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0006ea70 con 0x7f2c00167180 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c000d0600 0x7f2c0014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c000d0600 0x7f2c0014b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41304/0 (socket says 172.21.15.59:41304) 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 -- 172.21.15.59:0/2953564690 learned_addr learned my addr 172.21.15.59:0/2953564690 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c04c00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c0015d300 0x7f2c000b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 -- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c00167180 msgr2=0x7f2c0015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c00167180 0x7f2c0015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 -- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c0015d300 msgr2=0x7f2c000b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c0015d300 0x7f2c000b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c05600640 1 -- 172.21.15.59:0/2953564690 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0015dbe0 con 0x7f2c000d0600 2024-04-23T14:32:33.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.592+0000 7f2c04c00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c0015d300 0x7f2c000b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:33.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.593+0000 7f2c05600640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c000d0600 0x7f2c0014b500 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009870 tx=0x7f2bf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.593+0000 7f2bfd600640 1 -- 172.21.15.59:0/2953564690 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4041070 con 0x7f2c000d0600 2024-04-23T14:32:33.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.593+0000 7f2bfd600640 1 -- 172.21.15.59:0/2953564690 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf403c870 con 0x7f2c000d0600 2024-04-23T14:32:33.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.593+0000 7f2bfd600640 1 -- 172.21.15.59:0/2953564690 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4037040 con 0x7f2c000d0600 2024-04-23T14:32:33.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.593+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c000b34c0 con 0x7f2c000d0600 2024-04-23T14:32:33.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.593+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c000b3a80 con 0x7f2c000d0600 2024-04-23T14:32:33.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.594+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c000b32d0 con 0x7f2c000d0600 2024-04-23T14:32:33.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.595+0000 7f2bfd600640 1 -- 172.21.15.59:0/2953564690 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bf4044070 con 0x7f2c000d0600 2024-04-23T14:32:33.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.596+0000 7f2bfd600640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2be006c8f0 0x7f2be006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:33.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.596+0000 7f2bfd600640 1 -- 172.21.15.59:0/2953564690 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2bf40ac8f0 con 0x7f2c000d0600 2024-04-23T14:32:33.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.597+0000 7f2c04c00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2be006c8f0 0x7f2be006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:33.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.597+0000 7f2c04c00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2be006c8f0 0x7f2be006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c00152440 tx=0x7f2bec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:33.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.601+0000 7f2bfd600640 1 -- 172.21.15.59:0/2953564690 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bf407adb0 con 0x7f2c000d0600 2024-04-23T14:32:33.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.804+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c000cff50 con 0x7f2c000d0600 2024-04-23T14:32:33.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.805+0000 7f2bfd600640 1 -- 172.21.15.59:0/2953564690 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f2bf407a6b0 con 0x7f2c000d0600 2024-04-23T14:32:33.806 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:32:33.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2be006c8f0 msgr2=0x7f2be006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2be006c8f0 0x7f2be006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c00152440 tx=0x7f2bec008040 comp rx=0 tx=0).stop 2024-04-23T14:32:33.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c000d0600 msgr2=0x7f2c0014b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:33.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c000d0600 0x7f2c0014b500 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009870 tx=0x7f2bf40046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 shutdown_connections 2024-04-23T14:32:33.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2be006c8f0 0x7f2be006edb0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c00152440 tx=0x7f2bec008040 comp rx=0 tx=0).stop 2024-04-23T14:32:33.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c00167180 0x7f2c0015e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c0015d300 0x7f2c000b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 --2- 172.21.15.59:0/2953564690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c000d0600 0x7f2c0014b500 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:33.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 >> 172.21.15.59:0/2953564690 conn(0x7f2c0006bdf0 msgr2=0x7f2c0006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:33.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 shutdown_connections 2024-04-23T14:32:33.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:33.808+0000 7f2c06a00640 1 -- 172.21.15.59:0/2953564690 wait complete. 2024-04-23T14:32:33.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:32:34.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.039+0000 7fba93400640 1 Processor -- start 2024-04-23T14:32:34.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.039+0000 7fba93400640 1 -- start start 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.041+0000 7fba93400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba8c0b56f0 0x7fba8c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.041+0000 7fba93400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c0d0a60 0x7fba8c06f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.041+0000 7fba93400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8c15bf60 con 0x7fba8c06f340 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.041+0000 7fba93400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fba8c15c240 con 0x7fba8c0b56f0 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.041+0000 7fba93400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c15c520 con 0x7fba8c0d0a60 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba90a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba8c0b56f0 0x7fba8c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba90a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba8c0b56f0 0x7fba8c06ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41318/0 (socket says 172.21.15.59:41318) 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba90a00640 1 -- 172.21.15.59:0/4072900507 learned_addr learned my addr 172.21.15.59:0/4072900507 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba8be00640 1 --1- 172.21.15.59:0/4072900507 >> v1:172.21.15.59:6789/0 conn(0x7fba8c06f340 0x7fba8c0d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47506/0 (socket says 172.21.15.59:47506) 2024-04-23T14:32:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba91400640 1 --2- 172.21.15.59:0/4072900507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c0d0a60 0x7fba8c06f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba90a00640 1 -- 172.21.15.59:0/4072900507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c0d0a60 msgr2=0x7fba8c06f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba90a00640 1 --2- 172.21.15.59:0/4072900507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c0d0a60 0x7fba8c06f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.042+0000 7fba90a00640 1 -- 172.21.15.59:0/4072900507 >> v1:172.21.15.59:6789/0 conn(0x7fba8c06f340 legacy=0x7fba8c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.043+0000 7fba90a00640 1 -- 172.21.15.59:0/4072900507 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c15c800 con 0x7fba8c0b56f0 2024-04-23T14:32:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.043+0000 7fba8b400640 1 -- 172.21.15.59:0/4072900507 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654639790 0 0) 0x7fba8c15bf60 con 0x7fba8c06f340 2024-04-23T14:32:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.044+0000 7fba90a00640 1 --2- 172.21.15.59:0/4072900507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c06ddd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fba80009870 tx=0x7fba8002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88142d0ba8e43ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.044+0000 7fba8b400640 1 -- 172.21.15.59:0/4072900507 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80037070 con 0x7fba8c0b56f0 2024-04-23T14:32:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.044+0000 7fba8b400640 1 -- 172.21.15.59:0/4072900507 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba80032440 con 0x7fba8c0b56f0 2024-04-23T14:32:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.044+0000 7fba8b400640 1 -- 172.21.15.59:0/4072900507 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80032810 con 0x7fba8c0b56f0 2024-04-23T14:32:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 -- 172.21.15.59:0/4072900507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 msgr2=0x7fba8c06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 --2- 172.21.15.59:0/4072900507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c06ddd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fba80009870 tx=0x7fba8002a250 comp rx=0 tx=0).stop 2024-04-23T14:32:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 -- 172.21.15.59:0/4072900507 shutdown_connections 2024-04-23T14:32:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 --2- 172.21.15.59:0/4072900507 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c0d0a60 0x7fba8c06f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 --2- 172.21.15.59:0/4072900507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c06ddd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 -- 172.21.15.59:0/4072900507 >> 172.21.15.59:0/4072900507 conn(0x7fba8c067d90 msgr2=0x7fba8c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 -- 172.21.15.59:0/4072900507 shutdown_connections 2024-04-23T14:32:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.045+0000 7fba93400640 1 -- 172.21.15.59:0/4072900507 wait complete. 2024-04-23T14:32:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.047+0000 7fba93400640 1 Processor -- start 2024-04-23T14:32:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.047+0000 7fba93400640 1 -- start start 2024-04-23T14:32:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba93400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c06f340 0x7fba8c144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba93400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba93400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba8c0d0a60 0x7fba8c14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba93400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c0b3710 con 0x7fba8c0b56f0 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba93400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c15bc10 con 0x7fba8c0d0a60 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba93400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c0b3890 con 0x7fba8c06f340 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba8be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.048+0000 7fba8be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c15cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41320/0 (socket says 172.21.15.59:41320) 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba8be00640 1 -- 172.21.15.59:0/1225809296 learned_addr learned my addr 172.21.15.59:0/1225809296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba90a00640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c06f340 0x7fba8c144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba91400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba8c0d0a60 0x7fba8c14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba8be00640 1 -- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c06f340 msgr2=0x7fba8c144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba8be00640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c06f340 0x7fba8c144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba8be00640 1 -- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba8c0d0a60 msgr2=0x7fba8c14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba8be00640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba8c0d0a60 0x7fba8c14d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba8be00640 1 -- 172.21.15.59:0/1225809296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c15dc50 con 0x7fba8c0b56f0 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba91400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba8c0d0a60 0x7fba8c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.049+0000 7fba8be00640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c15cd40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c4d0 tx=0x7fba7800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.050+0000 7fba88c00640 1 -- 172.21.15.59:0/1225809296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78018070 con 0x7fba8c0b56f0 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.050+0000 7fba88c00640 1 -- 172.21.15.59:0/1225809296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba78007500 con 0x7fba8c0b56f0 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.050+0000 7fba88c00640 1 -- 172.21.15.59:0/1225809296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78004a20 con 0x7fba8c0b56f0 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.050+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c15d690 con 0x7fba8c0b56f0 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.050+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c14d1f0 con 0x7fba8c0b56f0 2024-04-23T14:32:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.051+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c068a80 con 0x7fba8c0b56f0 2024-04-23T14:32:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.052+0000 7fba88c00640 1 -- 172.21.15.59:0/1225809296 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba7801b070 con 0x7fba8c0b56f0 2024-04-23T14:32:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.053+0000 7fba88c00640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba6c06c920 0x7fba6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.053+0000 7fba88c00640 1 -- 172.21.15.59:0/1225809296 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fba7808af30 con 0x7fba8c0b56f0 2024-04-23T14:32:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.053+0000 7fba90a00640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba6c06c920 0x7fba6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.054+0000 7fba90a00640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba6c06c920 0x7fba6c06ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.058+0000 7fba88c00640 1 -- 172.21.15.59:0/1225809296 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba78059240 con 0x7fba8c0b56f0 2024-04-23T14:32:34.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.269+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba8c06cd70 con 0x7fba8c0b56f0 2024-04-23T14:32:34.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.270+0000 7fba88c00640 1 -- 172.21.15.59:0/1225809296 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fba78058b40 con 0x7fba8c0b56f0 2024-04-23T14:32:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:32:34.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba6c06c920 msgr2=0x7fba6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba6c06c920 0x7fba6c06ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).stop 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 msgr2=0x7fba8c15cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c15cd40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c4d0 tx=0x7fba7800c990 comp rx=0 tx=0).stop 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 shutdown_connections 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba6c06c920 0x7fba6c06ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba8c0d0a60 0x7fba8c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba8c0b56f0 0x7fba8c15cd40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 --2- 172.21.15.59:0/1225809296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba8c06f340 0x7fba8c144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 >> 172.21.15.59:0/1225809296 conn(0x7fba8c067d90 msgr2=0x7fba8c158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 shutdown_connections 2024-04-23T14:32:34.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.273+0000 7fba93400640 1 -- 172.21.15.59:0/1225809296 wait complete. 2024-04-23T14:32:34.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:34.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.509+0000 7fd6a0e00640 1 Processor -- start 2024-04-23T14:32:34.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.509+0000 7fd6a0e00640 1 -- start start 2024-04-23T14:32:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.511+0000 7fd6a0e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd69c15c3d0 0x7fd69c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.511+0000 7fd6a0e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c0b4860 0x7fd69c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.511+0000 7fd6a0e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd69c160210 con 0x7fd69c0cd590 2024-04-23T14:32:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.511+0000 7fd6a0e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd69c1604f0 con 0x7fd69c15c3d0 2024-04-23T14:32:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.511+0000 7fd6a0e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd69c1607d0 con 0x7fd69c0b4860 2024-04-23T14:32:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.511+0000 7fd69aa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd69c0cd590 0x7fd69c0b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47514/0 (socket says 172.21.15.59:47514) 2024-04-23T14:32:34.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.511+0000 7fd69aa00640 1 -- 172.21.15.59:0/2827986963 learned_addr learned my addr 172.21.15.59:0/2827986963 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:34.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69b400640 1 --2- 172.21.15.59:0/2827986963 >> v2:172.21.15.76:3300/0 conn(0x7fd69c15c3d0 0x7fd69c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69be00640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c0b4860 0x7fd69c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673910824 0 0) 0x7fd69c160210 con 0x7fd69c0cd590 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd680003520 con 0x7fd69c0cd590 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1685228980 0 0) 0x7fd680003520 con 0x7fd69c0cd590 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c0b4860 msgr2=0x7fd69c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c0b4860 0x7fd69c0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 >> v2:172.21.15.76:3300/0 conn(0x7fd69c15c3d0 msgr2=0x7fd69c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 --2- 172.21.15.59:0/2827986963 >> v2:172.21.15.76:3300/0 conn(0x7fd69c15c3d0 0x7fd69c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c160ab0 con 0x7fd69c0cd590 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fd688001670 con 0x7fd69c0cd590 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 >> v1:172.21.15.59:6789/0 conn(0x7fd69c0cd590 legacy=0x7fd69c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c0b4860 0x7fd6800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.512+0000 7fd69a000640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd680003f80 0x7fd680004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69a000640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd6800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1604f0 con 0x7fd680004d40 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1607d0 con 0x7fd680003f80 2024-04-23T14:32:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd68000b9c0 con 0x7fd69c0b4860 2024-04-23T14:32:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69be00640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd6800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69aa00640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd680003f80 0x7fd680004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69be00640 1 -- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c0b4860 msgr2=0x7fd6800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69be00640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c0b4860 0x7fd6800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69be00640 1 -- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd680003f80 msgr2=0x7fd680004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69be00640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd680003f80 0x7fd680004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.513+0000 7fd69be00640 1 -- 172.21.15.59:0/2827986963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd680003520 con 0x7fd680004d40 2024-04-23T14:32:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.514+0000 7fd69be00640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd6800051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e170 tx=0x7fd680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1680aec25aa2f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694034020 con 0x7fd680004d40 2024-04-23T14:32:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd694030430 con 0x7fd680004d40 2024-04-23T14:32:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd69a000640 1 -- 172.21.15.59:0/2827986963 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694009c10 con 0x7fd680004d40 2024-04-23T14:32:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd6a0e00640 1 -- 172.21.15.59:0/2827986963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 msgr2=0x7fd6800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd6a0e00640 1 --2- 172.21.15.59:0/2827986963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd6800051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e170 tx=0x7fd680005830 comp rx=0 tx=0).stop 2024-04-23T14:32:34.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd69be00640 1 -- 172.21.15.59:0/2827986963 reap_dead start 2024-04-23T14:32:34.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd6a0e00640 1 -- 172.21.15.59:0/2827986963 shutdown_connections 2024-04-23T14:32:34.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.515+0000 7fd6a0e00640 1 -- 172.21.15.59:0/2827986963 >> 172.21.15.59:0/2827986963 conn(0x7fd69c0b21a0 msgr2=0x7fd69c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:34.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.516+0000 7fd6a0e00640 1 -- 172.21.15.59:0/2827986963 shutdown_connections 2024-04-23T14:32:34.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.516+0000 7fd6a0e00640 1 -- 172.21.15.59:0/2827986963 wait complete. 2024-04-23T14:32:34.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.517+0000 7fd6a0e00640 1 Processor -- start 2024-04-23T14:32:34.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.518+0000 7fd6a0e00640 1 -- start start 2024-04-23T14:32:34.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.518+0000 7fd6a0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd69c1f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd6a0e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c15f200 0x7fd69c15f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd6a0e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd69c15cab0 0x7fd69c15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd6a0e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c15c930 con 0x7fd680004d40 2024-04-23T14:32:34.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd6a0e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c15c7b0 con 0x7fd69c15cab0 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd6a0e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd69c0cb7d0 con 0x7fd69c15f200 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd69c1f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd69c1f4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41340/0 (socket says 172.21.15.59:41340) 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 -- 172.21.15.59:0/3929375105 learned_addr learned my addr 172.21.15.59:0/3929375105 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69aa00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c15f200 0x7fd69c15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 -- 172.21.15.59:0/3929375105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c15f200 msgr2=0x7fd69c15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c15f200 0x7fd69c15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 -- 172.21.15.59:0/3929375105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd69c15cab0 msgr2=0x7fd69c15cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd69c15cab0 0x7fd69c15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.519+0000 7fd69b400640 1 -- 172.21.15.59:0/3929375105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c1f5100 con 0x7fd680004d40 2024-04-23T14:32:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.520+0000 7fd69b400640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd69c1f4770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e7f0 tx=0x7fd68c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.520+0000 7fd693400640 1 -- 172.21.15.59:0/3929375105 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c017070 con 0x7fd680004d40 2024-04-23T14:32:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.520+0000 7fd693400640 1 -- 172.21.15.59:0/3929375105 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c00c8c0 con 0x7fd680004d40 2024-04-23T14:32:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.520+0000 7fd693400640 1 -- 172.21.15.59:0/3929375105 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00ccb0 con 0x7fd680004d40 2024-04-23T14:32:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.520+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69c1f53e0 con 0x7fd680004d40 2024-04-23T14:32:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.520+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69c1f4cb0 con 0x7fd680004d40 2024-04-23T14:32:34.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.522+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd69c04fd90 con 0x7fd680004d40 2024-04-23T14:32:34.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.523+0000 7fd693400640 1 -- 172.21.15.59:0/3929375105 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd68c01a070 con 0x7fd680004d40 2024-04-23T14:32:34.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.523+0000 7fd693400640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd67c06c8f0 0x7fd67c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:34.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.524+0000 7fd693400640 1 -- 172.21.15.59:0/3929375105 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd68c08bcf0 con 0x7fd680004d40 2024-04-23T14:32:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.524+0000 7fd69aa00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd67c06c8f0 0x7fd67c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.524+0000 7fd69aa00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd67c06c8f0 0x7fd67c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd688001eb0 tx=0x7fd688009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:34.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.528+0000 7fd693400640 1 -- 172.21.15.59:0/3929375105 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd68c090070 con 0x7fd680004d40 2024-04-23T14:32:34.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.778+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd69c06e400 con 0x7fd680004d40 2024-04-23T14:32:34.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.779+0000 7fd693400640 1 -- 172.21.15.59:0/3929375105 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7fd68c05a130 con 0x7fd680004d40 2024-04-23T14:32:34.780 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:34.780 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:34.780 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:32:34.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd67c06c8f0 msgr2=0x7fd67c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd67c06c8f0 0x7fd67c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd688001eb0 tx=0x7fd688009040 comp rx=0 tx=0).stop 2024-04-23T14:32:34.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 msgr2=0x7fd69c1f4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:34.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd69c1f4770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e7f0 tx=0x7fd68c00ecb0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 shutdown_connections 2024-04-23T14:32:34.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd67c06c8f0 0x7fd67c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd69c15f200 0x7fd69c15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd69c15cab0 0x7fd69c15cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 --2- 172.21.15.59:0/3929375105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd680004d40 0x7fd69c1f4770 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:34.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.782+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 >> 172.21.15.59:0/3929375105 conn(0x7fd69c0b21a0 msgr2=0x7fd69c0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:34.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.783+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 shutdown_connections 2024-04-23T14:32:34.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:34.783+0000 7fd6a0e00640 1 -- 172.21.15.59:0/3929375105 wait complete. 2024-04-23T14:32:34.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:35.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.011+0000 7fa9ea400640 1 Processor -- start 2024-04-23T14:32:35.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.012+0000 7fa9ea400640 1 -- start start 2024-04-23T14:32:35.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.013+0000 7fa9ea400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa9e415c8a0 0x7fa9e40ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.013+0000 7fa9ea400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e415cc80 con 0x7fa9e40d12d0 2024-04-23T14:32:35.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.013+0000 7fa9ea400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e40cd440 con 0x7fa9e409b410 2024-04-23T14:32:35.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.013+0000 7fa9ea400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e40cd720 con 0x7fa9e415c8a0 2024-04-23T14:32:35.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.014+0000 7fa9e3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa9e40d12d0 0x7fa9e40b1dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47526/0 (socket says 172.21.15.59:47526) 2024-04-23T14:32:35.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.014+0000 7fa9e3e00640 1 -- 172.21.15.59:0/2594996585 learned_addr learned my addr 172.21.15.59:0/2594996585 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:35.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.014+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2594996585 >> v2:172.21.15.76:3300/0 conn(0x7fa9e415c8a0 0x7fa9e40ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.014+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 390943999 0 0) 0x7fa9e415cc80 con 0x7fa9e40d12d0 2024-04-23T14:32:35.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.014+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9c8003520 con 0x7fa9e40d12d0 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3074855515 0 0) 0x7fa9c8003520 con 0x7fa9e40d12d0 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 >> v2:172.21.15.76:3300/0 conn(0x7fa9e415c8a0 msgr2=0x7fa9e40ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 --2- 172.21.15.59:0/2594996585 >> v2:172.21.15.76:3300/0 conn(0x7fa9e415c8a0 0x7fa9e40ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 >> v1:172.21.15.76:6789/0 conn(0x7fa9e409b410 legacy=0x7fa9e40d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e40cda00 con 0x7fa9e40d12d0 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa9dc0019f0 con 0x7fa9e40d12d0 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 >> v1:172.21.15.59:6789/0 conn(0x7fa9e40d12d0 legacy=0x7fa9e40b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c8004800 0x7fa9c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c8005210 0x7fa9c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40cd720 con 0x7fa9c8003a00 2024-04-23T14:32:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c800c160 con 0x7fa9c8005210 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c800c360 con 0x7fa9c8004800 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.015+0000 7fa9e3e00640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c8005210 0x7fa9c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.016+0000 7fa9e3400640 1 -- 172.21.15.59:0/2594996585 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c8004800 msgr2=0x7fa9c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.016+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c8004800 0x7fa9c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.016+0000 7fa9e3400640 1 -- 172.21.15.59:0/2594996585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c8005210 msgr2=0x7fa9c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.016+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c8005210 0x7fa9c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.016+0000 7fa9e3400640 1 -- 172.21.15.59:0/2594996585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c8003520 con 0x7fa9c8003a00 2024-04-23T14:32:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.016+0000 7fa9e3e00640 1 -- 172.21.15.59:0/2594996585 reap_dead start 2024-04-23T14:32:35.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.017+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9c8003e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009a90 tx=0x7fa9c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511cb7a5a4482023 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.017+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4036070 con 0x7fa9c8003a00 2024-04-23T14:32:35.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.017+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d4031400 con 0x7fa9c8003a00 2024-04-23T14:32:35.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.017+0000 7fa9e2a00640 1 -- 172.21.15.59:0/2594996585 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4009690 con 0x7fa9c8003a00 2024-04-23T14:32:35.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.018+0000 7fa9ea400640 1 -- 172.21.15.59:0/2594996585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 msgr2=0x7fa9c8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.018+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9c8003e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009a90 tx=0x7fa9c8005830 comp rx=0 tx=0).stop 2024-04-23T14:32:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.018+0000 7fa9ea400640 1 -- 172.21.15.59:0/2594996585 shutdown_connections 2024-04-23T14:32:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.018+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2594996585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9c8003e20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.018+0000 7fa9ea400640 1 -- 172.21.15.59:0/2594996585 >> 172.21.15.59:0/2594996585 conn(0x7fa9e40b40f0 msgr2=0x7fa9e40b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.018+0000 7fa9ea400640 1 -- 172.21.15.59:0/2594996585 shutdown_connections 2024-04-23T14:32:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.018+0000 7fa9ea400640 1 -- 172.21.15.59:0/2594996585 wait complete. 2024-04-23T14:32:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.020+0000 7fa9ea400640 1 Processor -- start 2024-04-23T14:32:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.020+0000 7fa9ea400640 1 -- start start 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9ea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9e415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9ea400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406d160 0x7fa9e406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9ea400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e415ea10 0x7fa9e415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9ea400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40d0380 con 0x7fa9c8003a00 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9ea400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40cd110 con 0x7fa9e406d160 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9ea400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40d0500 con 0x7fa9e415ea10 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9e415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9e415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41362/0 (socket says 172.21.15.59:41362) 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.021+0000 7fa9e3400640 1 -- 172.21.15.59:0/2561358049 learned_addr learned my addr 172.21.15.59:0/2561358049 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9e3400640 1 -- 172.21.15.59:0/2561358049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e415ea10 msgr2=0x7fa9e415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9dbe00640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406d160 0x7fa9e406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e415ea10 0x7fa9e415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9e3400640 1 -- 172.21.15.59:0/2561358049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406d160 msgr2=0x7fa9e406d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406d160 0x7fa9e406d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9e3400640 1 -- 172.21.15.59:0/2561358049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e415cfa0 con 0x7fa9c8003a00 2024-04-23T14:32:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9e3400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9e415b970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009bb0 tx=0x7fa9d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.022+0000 7fa9db400640 1 -- 172.21.15.59:0/2561358049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d403f020 con 0x7fa9c8003a00 2024-04-23T14:32:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.023+0000 7fa9db400640 1 -- 172.21.15.59:0/2561358049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d4037040 con 0x7fa9c8003a00 2024-04-23T14:32:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.023+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e415d280 con 0x7fa9c8003a00 2024-04-23T14:32:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.023+0000 7fa9db400640 1 -- 172.21.15.59:0/2561358049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4036070 con 0x7fa9c8003a00 2024-04-23T14:32:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.023+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e415f3d0 con 0x7fa9c8003a00 2024-04-23T14:32:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.024+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e4068a80 con 0x7fa9c8003a00 2024-04-23T14:32:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.025+0000 7fa9db400640 1 -- 172.21.15.59:0/2561358049 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9d4042020 con 0x7fa9c8003a00 2024-04-23T14:32:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.026+0000 7fa9db400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9c006c8f0 0x7fa9c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.026+0000 7fa9db400640 1 -- 172.21.15.59:0/2561358049 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa9d40ab930 con 0x7fa9c8003a00 2024-04-23T14:32:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.027+0000 7fa9dbe00640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9c006c8f0 0x7fa9c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.027+0000 7fa9dbe00640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9c006c8f0 0x7fa9c006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40ccb10 tx=0x7fa9cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.031+0000 7fa9db400640 1 -- 172.21.15.59:0/2561358049 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9d4049070 con 0x7fa9c8003a00 2024-04-23T14:32:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.231+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9e415fea0 con 0x7fa9c8003a00 2024-04-23T14:32:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.233+0000 7fa9db400640 1 -- 172.21.15.59:0/2561358049 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fa9d4079cd0 con 0x7fa9c8003a00 2024-04-23T14:32:35.235 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:35.235 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9c006c8f0 msgr2=0x7fa9c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9c006c8f0 0x7fa9c006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40ccb10 tx=0x7fa9cc009040 comp rx=0 tx=0).stop 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 msgr2=0x7fa9e415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9e415b970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009bb0 tx=0x7fa9d4004730 comp rx=0 tx=0).stop 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 shutdown_connections 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9c006c8f0 0x7fa9c006edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e415ea10 0x7fa9e415ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406d160 0x7fa9e406d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 --2- 172.21.15.59:0/2561358049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c8003a00 0x7fa9e415b970 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 >> 172.21.15.59:0/2561358049 conn(0x7fa9e40b40f0 msgr2=0x7fa9e40cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 shutdown_connections 2024-04-23T14:32:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.236+0000 7fa9ea400640 1 -- 172.21.15.59:0/2561358049 wait complete. 2024-04-23T14:32:35.255 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:32:35.256 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.457+0000 7f12a3c00640 1 Processor -- start 2024-04-23T14:32:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.458+0000 7f12a3c00640 1 -- start start 2024-04-23T14:32:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.459+0000 7f12a3c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f129c0cd020 0x7f129c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.459+0000 7f12a3c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129c0cd520 con 0x7f129c0b3e40 2024-04-23T14:32:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.459+0000 7f12a3c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129c0cd800 con 0x7f129c1454e0 2024-04-23T14:32:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.459+0000 7f12a3c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f129c0cddc0 con 0x7f129c0cd020 2024-04-23T14:32:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f12a1c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f129c0b3e40 0x7f129c160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47536/0 (socket says 172.21.15.59:47536) 2024-04-23T14:32:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f12a1c00640 1 -- 172.21.15.59:0/1364466325 learned_addr learned my addr 172.21.15.59:0/1364466325 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129be00640 1 --2- 172.21.15.59:0/1364466325 >> v2:172.21.15.76:3300/0 conn(0x7f129c0cd020 0x7f129c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368492088 0 0) 0x7f129c0cd520 con 0x7f129c0b3e40 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1280003520 con 0x7f129c0b3e40 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2241795663 0 0) 0x7f1280003520 con 0x7f129c0b3e40 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 >> v2:172.21.15.76:3300/0 conn(0x7f129c0cd020 msgr2=0x7f129c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 --2- 172.21.15.59:0/1364466325 >> v2:172.21.15.76:3300/0 conn(0x7f129c0cd020 0x7f129c0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 >> v1:172.21.15.76:6789/0 conn(0x7f129c1454e0 legacy=0x7f129c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c0cdae0 con 0x7f129c0b3e40 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.460+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f12940019f0 con 0x7f129c0b3e40 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 >> v1:172.21.15.59:6789/0 conn(0x7f129c0b3e40 legacy=0x7f129c160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129b400640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12800039b0 0x7f1280003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129b400640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1280004800 0x7f1280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129b400640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12800051c0 0x7f128000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0cddc0 con 0x7f12800051c0 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f128000c160 con 0x7f1280004800 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f128000c360 con 0x7f12800039b0 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1c00640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12800051c0 0x7f128000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f129be00640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1280004800 0x7f1280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1200640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12800039b0 0x7f1280003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1c00640 1 -- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12800039b0 msgr2=0x7f1280003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1c00640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12800039b0 0x7f1280003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1c00640 1 -- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1280004800 msgr2=0x7f1280004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1c00640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1280004800 0x7f1280004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1c00640 1 -- 172.21.15.59:0/1364466325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1280003520 con 0x7f12800051c0 2024-04-23T14:32:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.461+0000 7f12a1200640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12800039b0 0x7f1280003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.462+0000 7f12a1c00640 1 -- 172.21.15.59:0/1364466325 reap_dead start 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.463+0000 7f12a1c00640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12800051c0 0x7f128000b9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f129400a840 tx=0x7f1280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46940a0627ed46ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.463+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129400a460 con 0x7f12800051c0 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.463+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129402f070 con 0x7f12800051c0 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.463+0000 7f129b400640 1 -- 172.21.15.59:0/1364466325 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129402bcb0 con 0x7f12800051c0 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.463+0000 7f12a3c00640 1 -- 172.21.15.59:0/1364466325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12800051c0 msgr2=0x7f128000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.463+0000 7f12a3c00640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12800051c0 0x7f128000b9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f129400a840 tx=0x7f1280005830 comp rx=0 tx=0).stop 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.464+0000 7f12a3c00640 1 -- 172.21.15.59:0/1364466325 shutdown_connections 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.464+0000 7f12a3c00640 1 --2- 172.21.15.59:0/1364466325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12800051c0 0x7f128000b9c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.464+0000 7f12a3c00640 1 -- 172.21.15.59:0/1364466325 >> 172.21.15.59:0/1364466325 conn(0x7f129c067ea0 msgr2=0x7f129c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.464+0000 7f12a3c00640 1 -- 172.21.15.59:0/1364466325 shutdown_connections 2024-04-23T14:32:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.464+0000 7f12a3c00640 1 -- 172.21.15.59:0/1364466325 wait complete. 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.466+0000 7f12a3c00640 1 Processor -- start 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.466+0000 7f12a3c00640 1 -- start start 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f12a3c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12800051c0 0x7f129c15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f12a3c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f129c13fef0 0x7f129c13d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f12a3c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f129c15f760 0x7f129c13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f12a3c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b3a80 con 0x7f129c13fef0 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f12a3c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b3900 con 0x7f12800051c0 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f12a3c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f129c0b3c00 con 0x7f129c15f760 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f129be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f129c13fef0 0x7f129c13d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f129be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f129c13fef0 0x7f129c13d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41374/0 (socket says 172.21.15.59:41374) 2024-04-23T14:32:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f129be00640 1 -- 172.21.15.59:0/747047959 learned_addr learned my addr 172.21.15.59:0/747047959 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f12a1200640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12800051c0 0x7f129c15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f129be00640 1 -- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f129c15f760 msgr2=0x7f129c13bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f129be00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f129c15f760 0x7f129c13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f129be00640 1 -- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12800051c0 msgr2=0x7f129c15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.467+0000 7f129be00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12800051c0 0x7f129c15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f129be00640 1 -- 172.21.15.59:0/747047959 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c13ee50 con 0x7f129c13fef0 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f12a1200640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12800051c0 0x7f129c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f129be00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f129c13fef0 0x7f129c13d150 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1288011540 tx=0x7f1288011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f1298c00640 1 -- 172.21.15.59:0/747047959 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288015930 con 0x7f129c13fef0 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f1298c00640 1 -- 172.21.15.59:0/747047959 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1288015ad0 con 0x7f129c13fef0 2024-04-23T14:32:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f1298c00640 1 -- 172.21.15.59:0/747047959 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128800f460 con 0x7f129c13fef0 2024-04-23T14:32:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c13f130 con 0x7f129c13fef0 2024-04-23T14:32:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.468+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c13f6f0 con 0x7f129c13fef0 2024-04-23T14:32:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.469+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c068a80 con 0x7f129c13fef0 2024-04-23T14:32:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.470+0000 7f1298c00640 1 -- 172.21.15.59:0/747047959 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f128800f600 con 0x7f129c13fef0 2024-04-23T14:32:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.471+0000 7f1298c00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f127c06c920 0x7f127c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.471+0000 7f1298c00640 1 -- 172.21.15.59:0/747047959 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f128808c420 con 0x7f129c13fef0 2024-04-23T14:32:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.472+0000 7f12a1200640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f127c06c920 0x7f127c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.473+0000 7f12a1200640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f127c06c920 0x7f127c06ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f129c140680 tx=0x7f128c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.476+0000 7f1298c00640 1 -- 172.21.15.59:0/747047959 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1288019030 con 0x7f129c13fef0 2024-04-23T14:32:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.724+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f129c0b36e0 con 0x7f129c13fef0 2024-04-23T14:32:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.725+0000 7f1298c00640 1 -- 172.21.15.59:0/747047959 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f128805a7c0 con 0x7f129c13fef0 2024-04-23T14:32:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:35.726 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f127c06c920 msgr2=0x7f127c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f127c06c920 0x7f127c06ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f129c140680 tx=0x7f128c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f129c13fef0 msgr2=0x7f129c13d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f129c13fef0 0x7f129c13d150 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1288011540 tx=0x7f1288011a00 comp rx=0 tx=0).stop 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 shutdown_connections 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f127c06c920 0x7f127c06ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f129c15f760 0x7f129c13bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f129c13fef0 0x7f129c13d150 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 --2- 172.21.15.59:0/747047959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12800051c0 0x7f129c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 >> 172.21.15.59:0/747047959 conn(0x7f129c067ea0 msgr2=0x7f129c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 shutdown_connections 2024-04-23T14:32:35.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.728+0000 7f12a3c00640 1 -- 172.21.15.59:0/747047959 wait complete. 2024-04-23T14:32:35.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.964+0000 7f7041800640 1 Processor -- start 2024-04-23T14:32:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.964+0000 7f7041800640 1 -- start start 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7041800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f703c06f350 0x7f703c0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7041800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c06b9d0 0x7f703c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7041800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c15b270 con 0x7f703c15ae90 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7041800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f703c0b3080 con 0x7f703c06f350 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7041800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f703c0b4930 con 0x7f703c06b9d0 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7033e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f703c06f350 0x7f703c0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7033e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f703c06f350 0x7f703c0cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41386/0 (socket says 172.21.15.59:41386) 2024-04-23T14:32:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7033e00640 1 -- 172.21.15.59:0/3937417292 learned_addr learned my addr 172.21.15.59:0/3937417292 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7040e00640 1 --2- 172.21.15.59:0/3937417292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c06b9d0 0x7f703c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.966+0000 7f7033e00640 1 -- 172.21.15.59:0/3937417292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c06b9d0 msgr2=0x7f703c06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.967+0000 7f7033e00640 1 --2- 172.21.15.59:0/3937417292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c06b9d0 0x7f703c06be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.967+0000 7f7033e00640 1 -- 172.21.15.59:0/3937417292 >> v1:172.21.15.76:6789/0 conn(0x7f703c15ae90 legacy=0x7f703c071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.967+0000 7f7033e00640 1 -- 172.21.15.59:0/3937417292 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c0b4c10 con 0x7f703c06f350 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.967+0000 7f7040e00640 1 --2- 172.21.15.59:0/3937417292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c06b9d0 0x7f703c06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.968+0000 7f7033e00640 1 --2- 172.21.15.59:0/3937417292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c0cdd80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7024009870 tx=0x7f702402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fa67be0979a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.968+0000 7f703b400640 1 -- 172.21.15.59:0/3937417292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024037070 con 0x7f703c06f350 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.968+0000 7f703b400640 1 -- 172.21.15.59:0/3937417292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7024032440 con 0x7f703c06f350 2024-04-23T14:32:35.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.968+0000 7f703b400640 1 -- 172.21.15.59:0/3937417292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024032810 con 0x7f703c06f350 2024-04-23T14:32:35.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 -- 172.21.15.59:0/3937417292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 msgr2=0x7f703c0cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 --2- 172.21.15.59:0/3937417292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c0cdd80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7024009870 tx=0x7f702402a210 comp rx=0 tx=0).stop 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 -- 172.21.15.59:0/3937417292 shutdown_connections 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 --2- 172.21.15.59:0/3937417292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c06b9d0 0x7f703c06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 --2- 172.21.15.59:0/3937417292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c0cdd80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 -- 172.21.15.59:0/3937417292 >> 172.21.15.59:0/3937417292 conn(0x7f703c0b21a0 msgr2=0x7f703c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 -- 172.21.15.59:0/3937417292 shutdown_connections 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.969+0000 7f7041800640 1 -- 172.21.15.59:0/3937417292 wait complete. 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.971+0000 7f7041800640 1 Processor -- start 2024-04-23T14:32:35.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.971+0000 7f7041800640 1 -- start start 2024-04-23T14:32:35.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7041800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f703c06b9d0 0x7f703c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7041800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7041800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c15ae90 0x7f703c06e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7041800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0722f0 con 0x7f703c06f350 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7041800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0b32d0 con 0x7f703c06b9d0 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7041800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f703c0b45c0 con 0x7f703c15ae90 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7033e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7033e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41396/0 (socket says 172.21.15.59:41396) 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7033e00640 1 -- 172.21.15.59:0/2681601423 learned_addr learned my addr 172.21.15.59:0/2681601423 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f703be00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f703c06b9d0 0x7f703c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7033e00640 1 -- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c15ae90 msgr2=0x7f703c06e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.972+0000 7f7040e00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c15ae90 0x7f703c06e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7033e00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c15ae90 0x7f703c06e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7033e00640 1 -- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f703c06b9d0 msgr2=0x7f703c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:35.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7033e00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f703c06b9d0 0x7f703c0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:35.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7033e00640 1 -- 172.21.15.59:0/2681601423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c154770 con 0x7f703c06f350 2024-04-23T14:32:35.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7040e00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c15ae90 0x7f703c06e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:35.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7033e00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c154230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7024009870 tx=0x7f70240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7038c00640 1 -- 172.21.15.59:0/2681601423 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024037040 con 0x7f703c06f350 2024-04-23T14:32:35.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7038c00640 1 -- 172.21.15.59:0/2681601423 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7024042070 con 0x7f703c06f350 2024-04-23T14:32:35.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7038c00640 1 -- 172.21.15.59:0/2681601423 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024030bb0 con 0x7f703c06f350 2024-04-23T14:32:35.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c095670 con 0x7f703c06f350 2024-04-23T14:32:35.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.973+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c095c30 con 0x7f703c06f350 2024-04-23T14:32:35.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.975+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c04fd90 con 0x7f703c06f350 2024-04-23T14:32:35.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.976+0000 7f7038c00640 1 -- 172.21.15.59:0/2681601423 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7024047460 con 0x7f703c06f350 2024-04-23T14:32:35.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.977+0000 7f7038c00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:35.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.977+0000 7f7038c00640 1 -- 172.21.15.59:0/2681601423 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f70240ac830 con 0x7f703c06f350 2024-04-23T14:32:35.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.977+0000 7f703be00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:35.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.978+0000 7f703be00640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f701806c8f0 0x7f701806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f702c006fd0 tx=0x7f702c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:35.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:35.982+0000 7f7038c00640 1 -- 172.21.15.59:0/2681601423 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f703c154770 con 0x7f703c06f350 2024-04-23T14:32:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.228+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f703c0af760 con 0x7f703c06f350 2024-04-23T14:32:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.229+0000 7f7038c00640 1 -- 172.21.15.59:0/2681601423 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f7024045360 con 0x7f703c06f350 2024-04-23T14:32:36.230 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:36.231 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/1797959969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1797959969},{"type":"v1","addr":"172.21.15.59:6836","nonce":1797959969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2131694643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2131694643},{"type":"v1","addr":"172.21.15.59:6837","nonce":2131694643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2913234590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2913234590},{"type":"v1","addr":"172.21.15.76:6833","nonce":2913234590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:31:25.258431+0000","modified":"2024-04-23T14:31:28.648219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/592246573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":592246573},{"type":"v1","addr":"172.21.15.76:6835","nonce":592246573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-23T14:32:36.231 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f701806c8f0 msgr2=0x7f701806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f701806c8f0 0x7f701806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f702c006fd0 tx=0x7f702c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 msgr2=0x7f703c154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c154230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7024009870 tx=0x7f70240046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 shutdown_connections 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f703c15ae90 0x7f703c06e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f703c06f350 0x7f703c154230 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 --2- 172.21.15.59:0/2681601423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f703c06b9d0 0x7f703c0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.232+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 >> 172.21.15.59:0/2681601423 conn(0x7f703c0b21a0 msgr2=0x7f703c0cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.233+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 shutdown_connections 2024-04-23T14:32:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.233+0000 7f7041800640 1 -- 172.21.15.59:0/2681601423 wait complete. 2024-04-23T14:32:36.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:36.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.462+0000 7fae76c00640 1 Processor -- start 2024-04-23T14:32:36.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.462+0000 7fae76c00640 1 -- start start 2024-04-23T14:32:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae700cd050 0x7fae7006f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae7015ece0 0x7fae700cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae700b3080 0x7fae700cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fae700ced20 con 0x7fae700b3080 2024-04-23T14:32:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fae7015fcd0 con 0x7fae7015ece0 2024-04-23T14:32:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae700cd970 con 0x7fae700cd050 2024-04-23T14:32:36.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae700b3080 0x7fae700cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae700b3080 0x7fae700cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41410/0 (socket says 172.21.15.59:41410) 2024-04-23T14:32:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.464+0000 7fae76200640 1 -- 172.21.15.59:0/2725858423 learned_addr learned my addr 172.21.15.59:0/2725858423 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.465+0000 7fae76200640 1 -- 172.21.15.59:0/2725858423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae700cd050 msgr2=0x7fae7006f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.465+0000 7fae76200640 1 --2- 172.21.15.59:0/2725858423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae700cd050 0x7fae7006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.465+0000 7fae76200640 1 -- 172.21.15.59:0/2725858423 >> v2:172.21.15.76:3300/0 conn(0x7fae7015ece0 msgr2=0x7fae700cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.465+0000 7fae76200640 1 --2- 172.21.15.59:0/2725858423 >> v2:172.21.15.76:3300/0 conn(0x7fae7015ece0 0x7fae700cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.465+0000 7fae76200640 1 -- 172.21.15.59:0/2725858423 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7015f160 con 0x7fae700b3080 2024-04-23T14:32:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.466+0000 7fae76200640 1 --2- 172.21.15.59:0/2725858423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae700cf470 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae700b2cc0 tx=0x7fae6802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15316210657521a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.466+0000 7fae6fe00640 1 -- 172.21.15.59:0/2725858423 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68036070 con 0x7fae700b3080 2024-04-23T14:32:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.466+0000 7fae6fe00640 1 -- 172.21.15.59:0/2725858423 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae68031600 con 0x7fae700b3080 2024-04-23T14:32:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.466+0000 7fae6fe00640 1 -- 172.21.15.59:0/2725858423 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae680319d0 con 0x7fae700b3080 2024-04-23T14:32:36.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 -- 172.21.15.59:0/2725858423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 msgr2=0x7fae700cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 --2- 172.21.15.59:0/2725858423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae700cf470 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae700b2cc0 tx=0x7fae6802a610 comp rx=0 tx=0).stop 2024-04-23T14:32:36.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 -- 172.21.15.59:0/2725858423 shutdown_connections 2024-04-23T14:32:36.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 --2- 172.21.15.59:0/2725858423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae700cf470 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 --2- 172.21.15.59:0/2725858423 >> v2:172.21.15.76:3300/0 conn(0x7fae7015ece0 0x7fae700cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 --2- 172.21.15.59:0/2725858423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae700cd050 0x7fae7006f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 -- 172.21.15.59:0/2725858423 >> 172.21.15.59:0/2725858423 conn(0x7fae700b21a0 msgr2=0x7fae700b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 -- 172.21.15.59:0/2725858423 shutdown_connections 2024-04-23T14:32:36.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.467+0000 7fae76c00640 1 -- 172.21.15.59:0/2725858423 wait complete. 2024-04-23T14:32:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.469+0000 7fae76c00640 1 Processor -- start 2024-04-23T14:32:36.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.469+0000 7fae76c00640 1 -- start start 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae76c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae7009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae76c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae700cd050 0x7fae7015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae76c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae7015ece0 0x7fae7015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae76c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae7015f9f0 con 0x7fae700b3080 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae76c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae7015f710 con 0x7fae700cd050 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae76c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae700cef50 con 0x7fae7015ece0 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae75800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae7009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae75800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae7009da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41416/0 (socket says 172.21.15.59:41416) 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.470+0000 7fae75800640 1 -- 172.21.15.59:0/2032676786 learned_addr learned my addr 172.21.15.59:0/2032676786 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae75800640 1 -- 172.21.15.59:0/2032676786 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae7015ece0 msgr2=0x7fae7015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae75800640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae7015ece0 0x7fae7015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae75800640 1 -- 172.21.15.59:0/2032676786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae700cd050 msgr2=0x7fae7015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae75800640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae700cd050 0x7fae7015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae75800640 1 -- 172.21.15.59:0/2032676786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7009df40 con 0x7fae700b3080 2024-04-23T14:32:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae75800640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae7009da00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b4a0 tx=0x7fae6400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae6d600640 1 -- 172.21.15.59:0/2032676786 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64014070 con 0x7fae700b3080 2024-04-23T14:32:36.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.471+0000 7fae6d600640 1 -- 172.21.15.59:0/2032676786 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64010610 con 0x7fae700b3080 2024-04-23T14:32:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.472+0000 7fae6d600640 1 -- 172.21.15.59:0/2032676786 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64010a00 con 0x7fae700b3080 2024-04-23T14:32:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.472+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae701f47d0 con 0x7fae700b3080 2024-04-23T14:32:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.472+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701f4d90 con 0x7fae700b3080 2024-04-23T14:32:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.473+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7004fd90 con 0x7fae700b3080 2024-04-23T14:32:36.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.474+0000 7fae6d600640 1 -- 172.21.15.59:0/2032676786 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae64017070 con 0x7fae700b3080 2024-04-23T14:32:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.475+0000 7fae6d600640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae5006c8f0 0x7fae5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.475+0000 7fae6d600640 1 -- 172.21.15.59:0/2032676786 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fae6408b6c0 con 0x7fae700b3080 2024-04-23T14:32:36.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.475+0000 7fae74e00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae5006c8f0 0x7fae5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.476+0000 7fae74e00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae5006c8f0 0x7fae5006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae700c8e10 tx=0x7fae5c006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.480+0000 7fae6d600640 1 -- 172.21.15.59:0/2032676786 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae640599d0 con 0x7fae700b3080 2024-04-23T14:32:36.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.686+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae700aee50 con 0x7fae700b3080 2024-04-23T14:32:36.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.689+0000 7fae6d600640 1 -- 172.21.15.59:0/2032676786 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16794 (secure 0 0 0) 0x7fae640592d0 con 0x7fae700b3080 2024-04-23T14:32:36.691 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:36.691 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:07.506381+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:31:23.479186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:31:24.232329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:36.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae5006c8f0 msgr2=0x7fae5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae5006c8f0 0x7fae5006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae700c8e10 tx=0x7fae5c006c90 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 msgr2=0x7fae7009da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae7009da00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b4a0 tx=0x7fae6400b960 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 shutdown_connections 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae5006c8f0 0x7fae5006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae7015ece0 0x7fae7015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae700cd050 0x7fae7015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 --2- 172.21.15.59:0/2032676786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae700b3080 0x7fae7009da00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.692+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 >> 172.21.15.59:0/2032676786 conn(0x7fae700b21a0 msgr2=0x7fae700ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.693+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 shutdown_connections 2024-04-23T14:32:36.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.693+0000 7fae76c00640 1 -- 172.21.15.59:0/2032676786 wait complete. 2024-04-23T14:32:36.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:32:36.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.914+0000 7f5ca7200640 1 Processor -- start 2024-04-23T14:32:36.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.915+0000 7f5ca7200640 1 -- start start 2024-04-23T14:32:36.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.916+0000 7f5ca7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca80d0f80 0x7f5ca80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.916+0000 7f5ca7200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ca815f880 0x7f5ca80d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.916+0000 7f5ca7200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca80cf930 con 0x7f5ca815ea40 2024-04-23T14:32:36.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.916+0000 7f5ca7200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca80cfc10 con 0x7f5ca815f880 2024-04-23T14:32:36.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.916+0000 7f5ca7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca80cfef0 con 0x7f5ca80d0f80 2024-04-23T14:32:36.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ca815f880 0x7f5ca80d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ca815f880 0x7f5ca80d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41420/0 (socket says 172.21.15.59:41420) 2024-04-23T14:32:36.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca6800640 1 -- 172.21.15.59:0/750865336 learned_addr learned my addr 172.21.15.59:0/750865336 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:36.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca5e00640 1 --2- 172.21.15.59:0/750865336 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca80d0f80 0x7f5ca80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca6800640 1 -- 172.21.15.59:0/750865336 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca80d0f80 msgr2=0x7f5ca80cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:36.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca6800640 1 --2- 172.21.15.59:0/750865336 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca80d0f80 0x7f5ca80cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca6800640 1 -- 172.21.15.59:0/750865336 >> v1:172.21.15.76:6789/0 conn(0x7f5ca815ea40 legacy=0x7f5ca80d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:36.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.917+0000 7f5ca6800640 1 -- 172.21.15.59:0/750865336 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca80d01d0 con 0x7f5ca815f880 2024-04-23T14:32:36.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.918+0000 7f5ca6800640 1 --2- 172.21.15.59:0/750865336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80d2310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009e50 tx=0x7f5ca002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e90f404e2c66fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca4a00640 1 -- 172.21.15.59:0/750865336 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0036070 con 0x7f5ca815f880 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca4a00640 1 -- 172.21.15.59:0/750865336 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca0031600 con 0x7f5ca815f880 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca4a00640 1 -- 172.21.15.59:0/750865336 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca00319d0 con 0x7f5ca815f880 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca7200640 1 -- 172.21.15.59:0/750865336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 msgr2=0x7f5ca80d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca7200640 1 --2- 172.21.15.59:0/750865336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80d2310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009e50 tx=0x7f5ca002a630 comp rx=0 tx=0).stop 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca7200640 1 -- 172.21.15.59:0/750865336 shutdown_connections 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca7200640 1 --2- 172.21.15.59:0/750865336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80d2310 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca7200640 1 --2- 172.21.15.59:0/750865336 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca80d0f80 0x7f5ca80cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.919+0000 7f5ca7200640 1 -- 172.21.15.59:0/750865336 >> 172.21.15.59:0/750865336 conn(0x7f5ca808bf90 msgr2=0x7f5ca80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:36.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.920+0000 7f5ca7200640 1 -- 172.21.15.59:0/750865336 shutdown_connections 2024-04-23T14:32:36.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.920+0000 7f5ca7200640 1 -- 172.21.15.59:0/750865336 wait complete. 2024-04-23T14:32:36.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.921+0000 7f5ca7200640 1 Processor -- start 2024-04-23T14:32:36.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.921+0000 7f5ca7200640 1 -- start start 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.922+0000 7f5ca7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ca80d0f80 0x7f5ca80c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.922+0000 7f5ca7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca815ea40 0x7f5ca80b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.922+0000 7f5ca7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.922+0000 7f5ca7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca80d2850 con 0x7f5ca815f880 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.922+0000 7f5ca7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca815fce0 con 0x7f5ca80d0f80 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.922+0000 7f5ca7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca80d29d0 con 0x7f5ca815ea40 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41434/0 (socket says 172.21.15.59:41434) 2024-04-23T14:32:36.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 -- 172.21.15.59:0/1336059618 learned_addr learned my addr 172.21.15.59:0/1336059618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca5e00640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ca80d0f80 0x7f5ca80c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 -- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca815ea40 msgr2=0x7f5ca80b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca5400640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca815ea40 0x7f5ca80b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca815ea40 0x7f5ca80b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 -- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ca80d0f80 msgr2=0x7f5ca80c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ca80d0f80 0x7f5ca80c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca6800640 1 -- 172.21.15.59:0/1336059618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca80c9700 con 0x7f5ca815f880 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca5e00640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ca80d0f80 0x7f5ca80c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.923+0000 7f5ca5400640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca815ea40 0x7f5ca80b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.924+0000 7f5ca6800640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80b1bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0034040 tx=0x7f5ca0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.924+0000 7f5c9e000640 1 -- 172.21.15.59:0/1336059618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0036050 con 0x7f5ca815f880 2024-04-23T14:32:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.924+0000 7f5c9e000640 1 -- 172.21.15.59:0/1336059618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca0042070 con 0x7f5ca815f880 2024-04-23T14:32:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.924+0000 7f5c9e000640 1 -- 172.21.15.59:0/1336059618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0009510 con 0x7f5ca815f880 2024-04-23T14:32:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.924+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca80c99e0 con 0x7f5ca815f880 2024-04-23T14:32:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.924+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca80c9fa0 con 0x7f5ca815f880 2024-04-23T14:32:36.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.925+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca804fd90 con 0x7f5ca815f880 2024-04-23T14:32:36.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.926+0000 7f5c9e000640 1 -- 172.21.15.59:0/1336059618 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ca0007bd0 con 0x7f5ca815f880 2024-04-23T14:32:36.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.927+0000 7f5c9e000640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c8806c8f0 0x7f5c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:36.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.927+0000 7f5c9e000640 1 -- 172.21.15.59:0/1336059618 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5ca00acaf0 con 0x7f5ca815f880 2024-04-23T14:32:36.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.927+0000 7f5ca5e00640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c8806c8f0 0x7f5c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:36.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.928+0000 7f5ca5e00640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c8806c8f0 0x7f5c8806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006fd0 tx=0x7f5c98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:36.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:36.932+0000 7f5c9e000640 1 -- 172.21.15.59:0/1336059618 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ca007ae00 con 0x7f5ca815f880 2024-04-23T14:32:37.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.178+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5ca80d2570 con 0x7f5ca815f880 2024-04-23T14:32:37.180 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:32:37.179+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:32:37.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.183+0000 7f5c9e000640 1 -- 172.21.15.59:0/1336059618 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f5ca007a700 con 0x7f5ca815f880 2024-04-23T14:32:37.184 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:32:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c8806c8f0 msgr2=0x7f5c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c8806c8f0 0x7f5c8806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006fd0 tx=0x7f5c98008040 comp rx=0 tx=0).stop 2024-04-23T14:32:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 msgr2=0x7f5ca80b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80b1bc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0034040 tx=0x7f5ca0004a40 comp rx=0 tx=0).stop 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 shutdown_connections 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c8806c8f0 0x7f5c8806edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ca815ea40 0x7f5ca80b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ca815f880 0x7f5ca80b1bc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 --2- 172.21.15.59:0/1336059618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ca80d0f80 0x7f5ca80c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.187+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 >> 172.21.15.59:0/1336059618 conn(0x7f5ca808bf90 msgr2=0x7f5ca8161920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.188+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 shutdown_connections 2024-04-23T14:32:37.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.188+0000 7f5ca7200640 1 -- 172.21.15.59:0/1336059618 wait complete. 2024-04-23T14:32:37.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:32:37.216 INFO:tasks.ceph.mds.d.smithi076.stderr: -4437> 2024-04-23T14:31:26.471+0000 7f3f7d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:32:37.300 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:32:37.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.410+0000 7fd321c00640 1 Processor -- start 2024-04-23T14:32:37.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.410+0000 7fd321c00640 1 -- start start 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.411+0000 7fd321c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd31c0d0ce0 0x7fd31c0af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.411+0000 7fd321c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c0af5c0 0x7fd31c071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.411+0000 7fd321c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c160120 con 0x7fd31c15c410 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd321c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c160400 con 0x7fd31c0d0ce0 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd321c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c1606e0 con 0x7fd31c0af5c0 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd31b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd31c0d0ce0 0x7fd31c0af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd31b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd31c0d0ce0 0x7fd31c0af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41442/0 (socket says 172.21.15.59:41442) 2024-04-23T14:32:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd31b400640 1 -- 172.21.15.59:0/26451445 learned_addr learned my addr 172.21.15.59:0/26451445 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd321200640 1 --2- 172.21.15.59:0/26451445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c0af5c0 0x7fd31c071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd31b400640 1 -- 172.21.15.59:0/26451445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c0af5c0 msgr2=0x7fd31c071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd31b400640 1 --2- 172.21.15.59:0/26451445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c0af5c0 0x7fd31c071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.412+0000 7fd31b400640 1 -- 172.21.15.59:0/26451445 >> v1:172.21.15.76:6789/0 conn(0x7fd31c15c410 legacy=0x7fd31c0d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.413+0000 7fd31b400640 1 -- 172.21.15.59:0/26451445 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c1609c0 con 0x7fd31c0d0ce0 2024-04-23T14:32:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.413+0000 7fd321200640 1 --2- 172.21.15.59:0/26451445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c0af5c0 0x7fd31c071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.414+0000 7fd31b400640 1 --2- 172.21.15.59:0/26451445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c0af080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd308009870 tx=0x7fd30802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe25e500db5eae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.414+0000 7fd31aa00640 1 -- 172.21.15.59:0/26451445 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308037070 con 0x7fd31c0d0ce0 2024-04-23T14:32:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.414+0000 7fd31aa00640 1 -- 172.21.15.59:0/26451445 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd308032440 con 0x7fd31c0d0ce0 2024-04-23T14:32:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.414+0000 7fd31aa00640 1 -- 172.21.15.59:0/26451445 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308032810 con 0x7fd31c0d0ce0 2024-04-23T14:32:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 -- 172.21.15.59:0/26451445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0d0ce0 msgr2=0x7fd31c0af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 --2- 172.21.15.59:0/26451445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c0af080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd308009870 tx=0x7fd30802a210 comp rx=0 tx=0).stop 2024-04-23T14:32:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 -- 172.21.15.59:0/26451445 shutdown_connections 2024-04-23T14:32:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 --2- 172.21.15.59:0/26451445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c0af5c0 0x7fd31c071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 --2- 172.21.15.59:0/26451445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c0af080 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 -- 172.21.15.59:0/26451445 >> 172.21.15.59:0/26451445 conn(0x7fd31c06bdf0 msgr2=0x7fd31c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 -- 172.21.15.59:0/26451445 shutdown_connections 2024-04-23T14:32:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.415+0000 7fd321c00640 1 -- 172.21.15.59:0/26451445 wait complete. 2024-04-23T14:32:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.417+0000 7fd321c00640 1 Processor -- start 2024-04-23T14:32:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.417+0000 7fd321c00640 1 -- start start 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd321c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0af5c0 0x7fd31c06f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd321c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c1f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd321c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c15c410 0x7fd31c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd321c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0afb80 con 0x7fd31c0af5c0 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd321c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0afa00 con 0x7fd31c0d0ce0 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd321c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c0721b0 con 0x7fd31c15c410 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd31be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0af5c0 0x7fd31c06f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd31be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0af5c0 0x7fd31c06f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41456/0 (socket says 172.21.15.59:41456) 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.418+0000 7fd31be00640 1 -- 172.21.15.59:0/2240415339 learned_addr learned my addr 172.21.15.59:0/2240415339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31b400640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c1f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31be00640 1 -- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c15c410 msgr2=0x7fd31c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31be00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c15c410 0x7fd31c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31be00640 1 -- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31c0d0ce0 msgr2=0x7fd31c1f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31be00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c1f4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31be00640 1 -- 172.21.15.59:0/2240415339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c06e760 con 0x7fd31c0af5c0 2024-04-23T14:32:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31b400640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c1f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd31be00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0af5c0 0x7fd31c06f4e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00ba50 tx=0x7fd30c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd313e00640 1 -- 172.21.15.59:0/2240415339 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c019070 con 0x7fd31c0af5c0 2024-04-23T14:32:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd313e00640 1 -- 172.21.15.59:0/2240415339 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c00cc20 con 0x7fd31c0af5c0 2024-04-23T14:32:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd313e00640 1 -- 172.21.15.59:0/2240415339 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c014680 con 0x7fd31c0af5c0 2024-04-23T14:32:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.419+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c0b2940 con 0x7fd31c0af5c0 2024-04-23T14:32:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.420+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c06ece0 con 0x7fd31c0af5c0 2024-04-23T14:32:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.421+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c04fd90 con 0x7fd31c0af5c0 2024-04-23T14:32:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.422+0000 7fd313e00640 1 -- 172.21.15.59:0/2240415339 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd30c01c070 con 0x7fd31c0af5c0 2024-04-23T14:32:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.422+0000 7fd313e00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2fc06c920 0x7fd2fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.423+0000 7fd313e00640 1 -- 172.21.15.59:0/2240415339 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fd30c08b990 con 0x7fd31c0af5c0 2024-04-23T14:32:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.423+0000 7fd31b400640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2fc06c920 0x7fd2fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:37.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.424+0000 7fd31b400640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2fc06c920 0x7fd2fc06ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd308034000 tx=0x7fd30802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:37.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.427+0000 7fd313e00640 1 -- 172.21.15.59:0/2240415339 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd30c059600 con 0x7fd31c0af5c0 2024-04-23T14:32:37.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:37.672+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd31c15b1f0 con 0x7fd31c0af5c0 2024-04-23T14:32:38.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.187+0000 7fd313e00640 1 -- 172.21.15.59:0/2240415339 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fd30c05d900 con 0x7fd31c0af5c0 2024-04-23T14:32:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2fc06c920 msgr2=0x7fd2fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2fc06c920 0x7fd2fc06ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd308034000 tx=0x7fd30802e000 comp rx=0 tx=0).stop 2024-04-23T14:32:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0af5c0 msgr2=0x7fd31c06f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0af5c0 0x7fd31c06f4e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00ba50 tx=0x7fd30c00bf10 comp rx=0 tx=0).stop 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 shutdown_connections 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2fc06c920 0x7fd2fc06ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd31c15c410 0x7fd31c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31c0d0ce0 0x7fd31c1f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 --2- 172.21.15.59:0/2240415339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd31c0af5c0 0x7fd31c06f4e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.190+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 >> 172.21.15.59:0/2240415339 conn(0x7fd31c06bdf0 msgr2=0x7fd31c156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.191+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 shutdown_connections 2024-04-23T14:32:38.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.191+0000 7fd321c00640 1 -- 172.21.15.59:0/2240415339 wait complete. 2024-04-23T14:32:38.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.429+0000 7f9049000640 1 Processor -- start 2024-04-23T14:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.429+0000 7f9049000640 1 -- start start 2024-04-23T14:32:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f9049000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904415e8e0 0x7f90440cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f9049000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90440d4210 0x7f90440d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f9049000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90440cf070 con 0x7f904406cbb0 2024-04-23T14:32:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f9049000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f90440d4bd0 con 0x7f90440d4210 2024-04-23T14:32:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f9049000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f90440cf2f0 con 0x7f904415e8e0 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f903aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90440d4210 0x7f90440d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f9043e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f904406cbb0 0x7f904406d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47552/0 (socket says 172.21.15.59:47552) 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.430+0000 7f9043e00640 1 -- 172.21.15.59:0/3158529355 learned_addr learned my addr 172.21.15.59:0/3158529355 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.431+0000 7f9043400640 1 --2- 172.21.15.59:0/3158529355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904415e8e0 0x7f90440cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.431+0000 7f903aa00640 1 -- 172.21.15.59:0/3158529355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904415e8e0 msgr2=0x7f90440cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.431+0000 7f903aa00640 1 --2- 172.21.15.59:0/3158529355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904415e8e0 0x7f90440cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.431+0000 7f903aa00640 1 -- 172.21.15.59:0/3158529355 >> v1:172.21.15.59:6789/0 conn(0x7f904406cbb0 legacy=0x7f904406d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.431+0000 7f903aa00640 1 -- 172.21.15.59:0/3158529355 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90440d1c60 con 0x7f90440d4210 2024-04-23T14:32:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.432+0000 7f903aa00640 1 --2- 172.21.15.59:0/3158529355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440d4690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f902c009790 tx=0x7f902c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1985bed75151cca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.432+0000 7f9042a00640 1 -- 172.21.15.59:0/3158529355 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c037070 con 0x7f90440d4210 2024-04-23T14:32:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.432+0000 7f9042a00640 1 -- 172.21.15.59:0/3158529355 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c032440 con 0x7f90440d4210 2024-04-23T14:32:38.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.432+0000 7f9042a00640 1 -- 172.21.15.59:0/3158529355 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c032810 con 0x7f90440d4210 2024-04-23T14:32:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 -- 172.21.15.59:0/3158529355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 msgr2=0x7f90440d4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 --2- 172.21.15.59:0/3158529355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440d4690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f902c009790 tx=0x7f902c02a180 comp rx=0 tx=0).stop 2024-04-23T14:32:38.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 -- 172.21.15.59:0/3158529355 shutdown_connections 2024-04-23T14:32:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 --2- 172.21.15.59:0/3158529355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440d4690 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 --2- 172.21.15.59:0/3158529355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904415e8e0 0x7f90440cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 -- 172.21.15.59:0/3158529355 >> 172.21.15.59:0/3158529355 conn(0x7f904408e060 msgr2=0x7f904409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 -- 172.21.15.59:0/3158529355 shutdown_connections 2024-04-23T14:32:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.433+0000 7f9049000640 1 -- 172.21.15.59:0/3158529355 wait complete. 2024-04-23T14:32:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.435+0000 7f9049000640 1 Processor -- start 2024-04-23T14:32:38.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.435+0000 7f9049000640 1 -- start start 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f9049000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904406cbb0 0x7f90440cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f9049000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f9049000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904415e8e0 0x7f90440b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f9049000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f904415ecc0 con 0x7f90440d4210 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f9049000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f90440cf890 con 0x7f904415e8e0 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f9049000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f90440d02f0 con 0x7f904406cbb0 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f903aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f903aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41472/0 (socket says 172.21.15.59:41472) 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.436+0000 7f903aa00640 1 -- 172.21.15.59:0/3608694914 learned_addr learned my addr 172.21.15.59:0/3608694914 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f9043e00640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904415e8e0 0x7f90440b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f903aa00640 1 -- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904406cbb0 msgr2=0x7f90440cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f903aa00640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904406cbb0 0x7f90440cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f903aa00640 1 -- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904415e8e0 msgr2=0x7f90440b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f903aa00640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904415e8e0 0x7f90440b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f903aa00640 1 -- 172.21.15.59:0/3608694914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90440cc8f0 con 0x7f90440d4210 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f9043e00640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904415e8e0 0x7f90440b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:38.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f903aa00640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440b1e80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f902c0098b0 tx=0x7f902c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.437+0000 7f903be00640 1 -- 172.21.15.59:0/3608694914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c037040 con 0x7f90440d4210 2024-04-23T14:32:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.438+0000 7f903be00640 1 -- 172.21.15.59:0/3608694914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c005b90 con 0x7f90440d4210 2024-04-23T14:32:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.438+0000 7f903be00640 1 -- 172.21.15.59:0/3608694914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c030770 con 0x7f90440d4210 2024-04-23T14:32:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.438+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90440ccbd0 con 0x7f90440d4210 2024-04-23T14:32:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.438+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9044161ff0 con 0x7f90440d4210 2024-04-23T14:32:38.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.439+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904404fd90 con 0x7f90440d4210 2024-04-23T14:32:38.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.440+0000 7f903be00640 1 -- 172.21.15.59:0/3608694914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f902c030910 con 0x7f90440d4210 2024-04-23T14:32:38.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.441+0000 7f903be00640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f902006c8f0 0x7f902006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:38.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.441+0000 7f903be00640 1 -- 172.21.15.59:0/3608694914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f902c0ac650 con 0x7f90440d4210 2024-04-23T14:32:38.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.441+0000 7f9043400640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f902006c8f0 0x7f902006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:38.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.442+0000 7f9043400640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f902006c8f0 0x7f902006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90340070b0 tx=0x7f9034007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:38.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.445+0000 7f903be00640 1 -- 172.21.15.59:0/3608694914 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f902c032e20 con 0x7f90440d4210 2024-04-23T14:32:38.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:38.649+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f90440cea00 con 0x7f90440d4210 2024-04-23T14:32:39.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.197+0000 7f903be00640 1 -- 172.21.15.59:0/3608694914 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f902c07a9a0 con 0x7f90440d4210 2024-04-23T14:32:39.199 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:32:39.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.202+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f902006c8f0 msgr2=0x7f902006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.202+0000 7f9049000640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f902006c8f0 0x7f902006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90340070b0 tx=0x7f9034007040 comp rx=0 tx=0).stop 2024-04-23T14:32:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.202+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 msgr2=0x7f90440b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.202+0000 7f9049000640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440b1e80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f902c0098b0 tx=0x7f902c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.203+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 shutdown_connections 2024-04-23T14:32:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.203+0000 7f9049000640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f902006c8f0 0x7f902006edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.203+0000 7f9049000640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904415e8e0 0x7f90440b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.203+0000 7f9049000640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90440d4210 0x7f90440b1e80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.203+0000 7f9049000640 1 --2- 172.21.15.59:0/3608694914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f904406cbb0 0x7f90440cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.203+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 >> 172.21.15.59:0/3608694914 conn(0x7f904408e060 msgr2=0x7f90440b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.204+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 shutdown_connections 2024-04-23T14:32:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.204+0000 7f9049000640 1 -- 172.21.15.59:0/3608694914 wait complete. 2024-04-23T14:32:39.229 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:32:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.446+0000 7f8a1ea00640 1 Processor -- start 2024-04-23T14:32:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.446+0000 7f8a1ea00640 1 -- start start 2024-04-23T14:32:39.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.449+0000 7f8a1ea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a18161e70 0x7f8a180ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ea00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a180cf4f0 0x7f8a180cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a180d2c10 0x7f8a180d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ea00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8a180cfe90 con 0x7f8a180d2c10 2024-04-23T14:32:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ea00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8a180b9b00 con 0x7f8a180cf4f0 2024-04-23T14:32:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ea00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8a18162250 con 0x7f8a18161e70 2024-04-23T14:32:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a180d2c10 0x7f8a180d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a180d2c10 0x7f8a180d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41482/0 (socket says 172.21.15.59:41482) 2024-04-23T14:32:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a1ca00640 1 -- 172.21.15.59:0/3634537877 learned_addr learned my addr 172.21.15.59:0/3634537877 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a17400640 1 --2- 172.21.15.59:0/3634537877 >> v2:172.21.15.76:3300/0 conn(0x7f8a180cf4f0 0x7f8a180cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.450+0000 7f8a17e00640 1 --2- 172.21.15.59:0/3634537877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a18161e70 0x7f8a180ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.451+0000 7f8a1ca00640 1 -- 172.21.15.59:0/3634537877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a18161e70 msgr2=0x7f8a180ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.451+0000 7f8a1ca00640 1 --2- 172.21.15.59:0/3634537877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a18161e70 0x7f8a180ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.451+0000 7f8a1ca00640 1 -- 172.21.15.59:0/3634537877 >> v2:172.21.15.76:3300/0 conn(0x7f8a180cf4f0 msgr2=0x7f8a180cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:39.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.451+0000 7f8a1ca00640 1 --2- 172.21.15.59:0/3634537877 >> v2:172.21.15.76:3300/0 conn(0x7f8a180cf4f0 0x7f8a180cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.451+0000 7f8a1ca00640 1 -- 172.21.15.59:0/3634537877 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a180b8ef0 con 0x7f8a180d2c10 2024-04-23T14:32:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.452+0000 7f8a1ca00640 1 --2- 172.21.15.59:0/3634537877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a180d30a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8a1806dd00 tx=0x7f8a1002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bb139ab71f79fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.453+0000 7f8a16a00640 1 -- 172.21.15.59:0/3634537877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10036070 con 0x7f8a180d2c10 2024-04-23T14:32:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.453+0000 7f8a16a00640 1 -- 172.21.15.59:0/3634537877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a10031600 con 0x7f8a180d2c10 2024-04-23T14:32:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.453+0000 7f8a16a00640 1 -- 172.21.15.59:0/3634537877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a100319d0 con 0x7f8a180d2c10 2024-04-23T14:32:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 -- 172.21.15.59:0/3634537877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 msgr2=0x7f8a180d30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/3634537877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a180d30a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8a1806dd00 tx=0x7f8a1002a6d0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 -- 172.21.15.59:0/3634537877 shutdown_connections 2024-04-23T14:32:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/3634537877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a180d30a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/3634537877 >> v2:172.21.15.76:3300/0 conn(0x7f8a180cf4f0 0x7f8a180cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/3634537877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a18161e70 0x7f8a180ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 -- 172.21.15.59:0/3634537877 >> 172.21.15.59:0/3634537877 conn(0x7f8a18067b50 msgr2=0x7f8a18067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:39.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 -- 172.21.15.59:0/3634537877 shutdown_connections 2024-04-23T14:32:39.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.454+0000 7f8a1ea00640 1 -- 172.21.15.59:0/3634537877 wait complete. 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.456+0000 7f8a1ea00640 1 Processor -- start 2024-04-23T14:32:39.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.456+0000 7f8a1ea00640 1 -- start start 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.457+0000 7f8a1ea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a180cf4f0 0x7f8a181f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.457+0000 7f8a1ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a18156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.457+0000 7f8a1ea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a18161e70 0x7f8a180c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.457+0000 7f8a1ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a180b9790 con 0x7f8a180d2c10 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.457+0000 7f8a1ea00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a180b94b0 con 0x7f8a18161e70 2024-04-23T14:32:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.457+0000 7f8a1ea00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8a180ba680 con 0x7f8a180cf4f0 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a18156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a18156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41498/0 (socket says 172.21.15.59:41498) 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 -- 172.21.15.59:0/1582786927 learned_addr learned my addr 172.21.15.59:0/1582786927 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a1ca00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a18161e70 0x7f8a180c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 -- 172.21.15.59:0/1582786927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a180cf4f0 msgr2=0x7f8a181f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a180cf4f0 0x7f8a181f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 -- 172.21.15.59:0/1582786927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a18161e70 msgr2=0x7f8a180c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a18161e70 0x7f8a180c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 -- 172.21.15.59:0/1582786927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a180cba20 con 0x7f8a180d2c10 2024-04-23T14:32:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.458+0000 7f8a17400640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a18156180 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400d5f0 tx=0x7f8a0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.459+0000 7f8a0fe00640 1 -- 172.21.15.59:0/1582786927 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040125f0 con 0x7f8a180d2c10 2024-04-23T14:32:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.459+0000 7f8a0fe00640 1 -- 172.21.15.59:0/1582786927 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04019070 con 0x7f8a180d2c10 2024-04-23T14:32:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.459+0000 7f8a0fe00640 1 -- 172.21.15.59:0/1582786927 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040129d0 con 0x7f8a180d2c10 2024-04-23T14:32:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.459+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a180cbd00 con 0x7f8a180d2c10 2024-04-23T14:32:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.459+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a180cc2c0 con 0x7f8a180d2c10 2024-04-23T14:32:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.460+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a18068a80 con 0x7f8a180d2c10 2024-04-23T14:32:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.461+0000 7f8a0fe00640 1 -- 172.21.15.59:0/1582786927 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a04012b70 con 0x7f8a180d2c10 2024-04-23T14:32:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.462+0000 7f8a0fe00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f89f806c8f0 0x7f89f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.462+0000 7f8a0fe00640 1 -- 172.21.15.59:0/1582786927 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f8a0408be80 con 0x7f8a180d2c10 2024-04-23T14:32:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.462+0000 7f8a17e00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f89f806c8f0 0x7f89f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.463+0000 7f8a17e00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f89f806c8f0 0x7f89f806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a181f87f0 tx=0x7f8a08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.467+0000 7f8a0fe00640 1 -- 172.21.15.59:0/1582786927 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a0405a380 con 0x7f8a180d2c10 2024-04-23T14:32:39.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:39.672+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a180b8650 con 0x7f8a180d2c10 2024-04-23T14:32:40.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.201+0000 7f8a0fe00640 1 -- 172.21.15.59:0/1582786927 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f8a04059c80 con 0x7f8a180d2c10 2024-04-23T14:32:40.202 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:32:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.206+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f89f806c8f0 msgr2=0x7f89f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.206+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f89f806c8f0 0x7f89f806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a181f87f0 tx=0x7f8a08008040 comp rx=0 tx=0).stop 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.206+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 msgr2=0x7f8a18156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.206+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a18156180 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400d5f0 tx=0x7f8a0400dab0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.207+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 shutdown_connections 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.207+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f89f806c8f0 0x7f89f806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.207+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a18161e70 0x7f8a180c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.207+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a180d2c10 0x7f8a18156180 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.207+0000 7f8a1ea00640 1 --2- 172.21.15.59:0/1582786927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a180cf4f0 0x7f8a181f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.207+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 >> 172.21.15.59:0/1582786927 conn(0x7f8a18067b50 msgr2=0x7f8a180d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:40.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.209+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 shutdown_connections 2024-04-23T14:32:40.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.209+0000 7f8a1ea00640 1 -- 172.21.15.59:0/1582786927 wait complete. 2024-04-23T14:32:40.238 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:32:40.238 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:32:40.238 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:32:40.238 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:40.238 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:32:40.238 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:32:40.239 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:32:40.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:40.239 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:32:40.239 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:32:40.239 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:32:40.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:40.239 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:32:40.240 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:32:40.240 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:32:40.240 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:32:40.259 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:32:40.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:32:40.260 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:32:40.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:32:40.261 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:32:40.261 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:32:40.265 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:32:40.265 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:32:40.303 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:32:40.304 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:40.309 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:32:40.309 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:40.312 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:32:40.313 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:40.319 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:32:40.319 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:40.319 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:32:40.457 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:32:40.458 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:32:40.464 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:32:40.465 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:32:40.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.500+0000 7f5f00c00640 1 Processor -- start 2024-04-23T14:32:40.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.500+0000 7f5f00c00640 1 -- start start 2024-04-23T14:32:40.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5f00c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5efc161e60 0x7f5efc0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5f00c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5efc1632e0 con 0x7f5efc0b44c0 2024-04-23T14:32:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5f00c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5efc0adcd0 con 0x7f5efc162de0 2024-04-23T14:32:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5f00c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5efc0aefd0 con 0x7f5efc161e60 2024-04-23T14:32:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5ef9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5efc161e60 0x7f5efc0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5ef9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5efc161e60 0x7f5efc0aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49932/0 (socket says 172.21.15.59:49932) 2024-04-23T14:32:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5ef9e00640 1 -- 172.21.15.59:0/923879270 learned_addr learned my addr 172.21.15.59:0/923879270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:40.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.502+0000 7f5ef9e00640 1 -- 172.21.15.59:0/923879270 >> v1:172.21.15.76:6789/0 conn(0x7f5efc162de0 legacy=0x7f5efc0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.503+0000 7f5ef9e00640 1 -- 172.21.15.59:0/923879270 >> v1:172.21.15.59:6789/0 conn(0x7f5efc0b44c0 legacy=0x7f5efc06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.503+0000 7f5ef9e00640 1 -- 172.21.15.59:0/923879270 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc0adfb0 con 0x7f5efc161e60 2024-04-23T14:32:40.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.504+0000 7f5ef9e00640 1 --2- 172.21.15.59:0/923879270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc161e60 0x7f5efc0aea90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009870 tx=0x7f5eec02a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85168326fddc6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.504+0000 7f5ef9400640 1 -- 172.21.15.59:0/923879270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec037070 con 0x7f5efc161e60 2024-04-23T14:32:40.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.504+0000 7f5ef9400640 1 -- 172.21.15.59:0/923879270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec032440 con 0x7f5efc161e60 2024-04-23T14:32:40.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.504+0000 7f5ef9400640 1 -- 172.21.15.59:0/923879270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec032810 con 0x7f5efc161e60 2024-04-23T14:32:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.504+0000 7f5f00c00640 1 -- 172.21.15.59:0/923879270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc161e60 msgr2=0x7f5efc0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.504+0000 7f5f00c00640 1 --2- 172.21.15.59:0/923879270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc161e60 0x7f5efc0aea90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009870 tx=0x7f5eec02a280 comp rx=0 tx=0).stop 2024-04-23T14:32:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.505+0000 7f5f00c00640 1 -- 172.21.15.59:0/923879270 shutdown_connections 2024-04-23T14:32:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.505+0000 7f5f00c00640 1 --2- 172.21.15.59:0/923879270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc161e60 0x7f5efc0aea90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.505+0000 7f5f00c00640 1 -- 172.21.15.59:0/923879270 >> 172.21.15.59:0/923879270 conn(0x7f5efc0b2130 msgr2=0x7f5efc0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.505+0000 7f5f00c00640 1 -- 172.21.15.59:0/923879270 shutdown_connections 2024-04-23T14:32:40.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.505+0000 7f5f00c00640 1 -- 172.21.15.59:0/923879270 wait complete. 2024-04-23T14:32:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.507+0000 7f5f00c00640 1 Processor -- start 2024-04-23T14:32:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.507+0000 7f5f00c00640 1 -- start start 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5f00c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc0b44c0 0x7f5efc14aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5f00c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5efc161e60 0x7f5efc1602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5f00c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc162de0 0x7f5efc1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5f00c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0af480 con 0x7f5efc162de0 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5f00c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0ae570 con 0x7f5efc0b44c0 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5f00c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5efc0af600 con 0x7f5efc161e60 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5efa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc162de0 0x7f5efc1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5efa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc162de0 0x7f5efc1607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49948/0 (socket says 172.21.15.59:49948) 2024-04-23T14:32:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5efa800640 1 -- 172.21.15.59:0/394424385 learned_addr learned my addr 172.21.15.59:0/394424385 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5ef9e00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc0b44c0 0x7f5efc14aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.508+0000 7f5ef1400640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5efc161e60 0x7f5efc1602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5efa800640 1 -- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5efc161e60 msgr2=0x7f5efc1602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5efa800640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5efc161e60 0x7f5efc1602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5efa800640 1 -- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc0b44c0 msgr2=0x7f5efc14aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:40.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5efa800640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc0b44c0 0x7f5efc14aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:40.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5efa800640 1 -- 172.21.15.59:0/394424385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc14afa0 con 0x7f5efc162de0 2024-04-23T14:32:40.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5ef9e00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc0b44c0 0x7f5efc14aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:40.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5efa800640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc162de0 0x7f5efc1607e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400c760 tx=0x7f5ef400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5ef2a00640 1 -- 172.21.15.59:0/394424385 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4003b70 con 0x7f5efc162de0 2024-04-23T14:32:40.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5ef2a00640 1 -- 172.21.15.59:0/394424385 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef4003d10 con 0x7f5efc162de0 2024-04-23T14:32:40.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5ef2a00640 1 -- 172.21.15.59:0/394424385 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400aa90 con 0x7f5efc162de0 2024-04-23T14:32:40.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc1e9350 con 0x7f5efc162de0 2024-04-23T14:32:40.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.509+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc1e8590 con 0x7f5efc162de0 2024-04-23T14:32:40.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.511+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5efc068a80 con 0x7f5efc162de0 2024-04-23T14:32:40.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.512+0000 7f5ef2a00640 1 -- 172.21.15.59:0/394424385 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ef400ac30 con 0x7f5efc162de0 2024-04-23T14:32:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.513+0000 7f5ef2a00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ed806c8f0 0x7f5ed806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.514+0000 7f5ef2a00640 1 -- 172.21.15.59:0/394424385 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f5ef408b600 con 0x7f5efc162de0 2024-04-23T14:32:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.514+0000 7f5ef9e00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ed806c8f0 0x7f5ed806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:40.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.515+0000 7f5ef9e00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ed806c8f0 0x7f5ed806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5eec034000 tx=0x7f5eec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:40.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.518+0000 7f5ef2a00640 1 -- 172.21.15.59:0/394424385 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ef40596d0 con 0x7f5efc162de0 2024-04-23T14:32:40.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:40.736+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5efc0b0c30 con 0x7f5efc162de0 2024-04-23T14:32:41.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.234+0000 7f5ef2a00640 1 -- 172.21.15.59:0/394424385 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f5ef408b020 con 0x7f5efc162de0 2024-04-23T14:32:41.235 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ed806c8f0 msgr2=0x7f5ed806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ed806c8f0 0x7f5ed806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5eec034000 tx=0x7f5eec02e000 comp rx=0 tx=0).stop 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc162de0 msgr2=0x7f5efc1607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc162de0 0x7f5efc1607e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400c760 tx=0x7f5ef400cc20 comp rx=0 tx=0).stop 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 shutdown_connections 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ed806c8f0 0x7f5ed806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5efc161e60 0x7f5efc1602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5efc162de0 0x7f5efc1607e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 --2- 172.21.15.59:0/394424385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5efc0b44c0 0x7f5efc14aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 >> 172.21.15.59:0/394424385 conn(0x7f5efc0b2130 msgr2=0x7f5efc0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 shutdown_connections 2024-04-23T14:32:41.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.238+0000 7f5f00c00640 1 -- 172.21.15.59:0/394424385 wait complete. 2024-04-23T14:32:41.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:32:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.450+0000 7f74b3400640 1 Processor -- start 2024-04-23T14:32:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.450+0000 7f74b3400640 1 -- start start 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b3400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0cd020 0x7f74ac0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f74ac0b3e40 0x7f74ac160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b3400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74ac0cd520 con 0x7f74ac1454e0 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b3400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f74ac0cddc0 con 0x7f74ac0b3e40 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b3400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f74ac1462c0 con 0x7f74ac0cd020 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b0a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f74ac1454e0 0x7f74ac145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49912/0 (socket says 172.21.15.59:49912) 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b0a00640 1 -- 172.21.15.59:0/2210365988 learned_addr learned my addr 172.21.15.59:0/2210365988 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.452+0000 7f74b1400640 1 --2- 172.21.15.59:0/2210365988 >> v2:172.21.15.59:3300/0 conn(0x7f74ac0b3e40 0x7f74ac160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.453+0000 7f74abe00640 1 --2- 172.21.15.59:0/2210365988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0cd020 0x7f74ac0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.453+0000 7f74ab400640 1 -- 172.21.15.59:0/2210365988 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993665710 0 0) 0x7f74ac0cd520 con 0x7f74ac1454e0 2024-04-23T14:32:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.453+0000 7f74ab400640 1 -- 172.21.15.59:0/2210365988 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7490003520 con 0x7f74ac1454e0 2024-04-23T14:32:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.453+0000 7f74b1400640 1 -- 172.21.15.59:0/2210365988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0cd020 msgr2=0x7f74ac0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.453+0000 7f74b1400640 1 --2- 172.21.15.59:0/2210365988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0cd020 0x7f74ac0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.453+0000 7f74b1400640 1 -- 172.21.15.59:0/2210365988 >> v1:172.21.15.59:6789/0 conn(0x7f74ac1454e0 legacy=0x7f74ac145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.453+0000 7f74b1400640 1 -- 172.21.15.59:0/2210365988 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac0cd800 con 0x7f74ac0b3e40 2024-04-23T14:32:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.454+0000 7f74b1400640 1 --2- 172.21.15.59:0/2210365988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac0b3e40 0x7f74ac160910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f74a40097a0 tx=0x7f74a402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fa294947740929 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.454+0000 7f74ab400640 1 -- 172.21.15.59:0/2210365988 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4037070 con 0x7f74ac0b3e40 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.454+0000 7f74ab400640 1 -- 172.21.15.59:0/2210365988 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a402ae40 con 0x7f74ac0b3e40 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.454+0000 7f74ab400640 1 -- 172.21.15.59:0/2210365988 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4032640 con 0x7f74ac0b3e40 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 -- 172.21.15.59:0/2210365988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac0b3e40 msgr2=0x7f74ac160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 --2- 172.21.15.59:0/2210365988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac0b3e40 0x7f74ac160910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f74a40097a0 tx=0x7f74a402a0f0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 -- 172.21.15.59:0/2210365988 shutdown_connections 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 --2- 172.21.15.59:0/2210365988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac0b3e40 0x7f74ac160910 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 --2- 172.21.15.59:0/2210365988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0cd020 0x7f74ac0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 -- 172.21.15.59:0/2210365988 >> 172.21.15.59:0/2210365988 conn(0x7f74ac067ea0 msgr2=0x7f74ac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 -- 172.21.15.59:0/2210365988 shutdown_connections 2024-04-23T14:32:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.455+0000 7f74b3400640 1 -- 172.21.15.59:0/2210365988 wait complete. 2024-04-23T14:32:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.456+0000 7f74b3400640 1 Processor -- start 2024-04-23T14:32:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.457+0000 7f74b3400640 1 -- start start 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b3400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0b3e40 0x7f74ac1e6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b3400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74ac0cd020 0x7f74ac1f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac1454e0 0x7f74ac1f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b3400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac160e50 con 0x7f74ac1454e0 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b3400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1603f0 con 0x7f74ac0cd020 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b3400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f74ac160fd0 con 0x7f74ac0b3e40 2024-04-23T14:32:41.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac1454e0 0x7f74ac1f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac1454e0 0x7f74ac1f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49968/0 (socket says 172.21.15.59:49968) 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 -- 172.21.15.59:0/2716281164 learned_addr learned my addr 172.21.15.59:0/2716281164 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b0a00640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0b3e40 0x7f74ac1e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 -- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0b3e40 msgr2=0x7f74ac1e6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74abe00640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74ac0cd020 0x7f74ac1f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0b3e40 0x7f74ac1e6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 -- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74ac0cd020 msgr2=0x7f74ac1f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74ac0cd020 0x7f74ac1f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b1400640 1 -- 172.21.15.59:0/2716281164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac15e180 con 0x7f74ac1454e0 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74b0a00640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0b3e40 0x7f74ac1e6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.458+0000 7f74abe00640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74ac0cd020 0x7f74ac1f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.459+0000 7f74b1400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac1454e0 0x7f74ac1f4a70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f74a4035040 tx=0x7f74a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.459+0000 7f74a8c00640 1 -- 172.21.15.59:0/2716281164 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4041070 con 0x7f74ac1454e0 2024-04-23T14:32:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.459+0000 7f74a8c00640 1 -- 172.21.15.59:0/2716281164 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a403c670 con 0x7f74ac1454e0 2024-04-23T14:32:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.459+0000 7f74a8c00640 1 -- 172.21.15.59:0/2716281164 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4037040 con 0x7f74ac1454e0 2024-04-23T14:32:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.459+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac15e460 con 0x7f74ac1454e0 2024-04-23T14:32:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.459+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac15ea20 con 0x7f74ac1454e0 2024-04-23T14:32:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.460+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac15eeb0 con 0x7f74ac1454e0 2024-04-23T14:32:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.461+0000 7f74a8c00640 1 -- 172.21.15.59:0/2716281164 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74a403a070 con 0x7f74ac1454e0 2024-04-23T14:32:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.463+0000 7f74a8c00640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f748806c8f0 0x7f748806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.463+0000 7f74a8c00640 1 -- 172.21.15.59:0/2716281164 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f74a40ab980 con 0x7f74ac1454e0 2024-04-23T14:32:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.463+0000 7f74b0a00640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f748806c8f0 0x7f748806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.464+0000 7f74b0a00640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f748806c8f0 0x7f748806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7494005880 tx=0x7f7494009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.468+0000 7f74a8c00640 1 -- 172.21.15.59:0/2716281164 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74ac15eeb0 con 0x7f74ac1454e0 2024-04-23T14:32:41.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.753+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f74ac068a80 con 0x7f74ac1454e0 2024-04-23T14:32:41.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.755+0000 7f74a8c00640 1 -- 172.21.15.59:0/2716281164 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v5) ==== 80+0+6645 (secure 0 0 0) 0x7f74a407a0f0 con 0x7f74ac1454e0 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A==", 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.756 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA==", 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.757 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw==", 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg==", 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow==", 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-23T14:32:41.760 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ==", 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ==", 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.761 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g==", 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA==", 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA==", 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.763 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA==", 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ==", 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-23T14:32:41.765 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ==", 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q==", 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.766 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg==", 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:32:41.767 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg==", 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmEteHAhAAwfUHmvRmEMcPIphwikq6sw==", 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.769 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-mds", 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAlxidmDAg5CBAAWuW2lNa5n/fd3HUi7qUvfQ==", 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-mds" 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-mgr", 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAlxidmdRI5CBAAWz+7fGd0GigMue/9EccJsA==", 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-osd", 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAlxidmrxw5CBAADcpJ1prueNdQKmSicuY9kA==", 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-osd" 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rbd", 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAlxidm8SU5CBAAE81guMtbGH59lznq8jXEiQ==", 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.771 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAlxidmFi85CBAAr9/EUiId0JZ4dGwEaL7RAg==", 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.773 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rgw", 2024-04-23T14:32:41.773 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAlxidmpzg5CBAAl8cdx0/YhMaXQNHTwaX3qQ==", 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA==", 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:32:41.774 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ==", 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:32:41.775 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:32:41.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f748806c8f0 msgr2=0x7f748806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f748806c8f0 0x7f748806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7494005880 tx=0x7f7494009040 comp rx=0 tx=0).stop 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac1454e0 msgr2=0x7f74ac1f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac1454e0 0x7f74ac1f4a70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f74a4035040 tx=0x7f74a40046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 shutdown_connections 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f748806c8f0 0x7f748806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74ac1454e0 0x7f74ac1f4a70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74ac0cd020 0x7f74ac1f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 --2- 172.21.15.59:0/2716281164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74ac0b3e40 0x7f74ac1e6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.757+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 >> 172.21.15.59:0/2716281164 conn(0x7f74ac067ea0 msgr2=0x7f74ac0aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.758+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 shutdown_connections 2024-04-23T14:32:41.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.758+0000 7f74b3400640 1 -- 172.21.15.59:0/2716281164 wait complete. 2024-04-23T14:32:41.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-23T14:32:41.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.985+0000 7fcc88a00640 1 Processor -- start 2024-04-23T14:32:41.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.985+0000 7fcc88a00640 1 -- start start 2024-04-23T14:32:41.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.987+0000 7fcc88a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc840af4b0 0x7fcc8415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.987+0000 7fcc88a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc840af890 0x7fcc84144a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.987+0000 7fcc88a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc8415f3f0 con 0x7fcc840b3010 2024-04-23T14:32:41.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.987+0000 7fcc88a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcc84144f60 con 0x7fcc840af4b0 2024-04-23T14:32:41.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.987+0000 7fcc88a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8415f6d0 con 0x7fcc840af890 2024-04-23T14:32:41.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.987+0000 7fcc83e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcc840b3010 0x7fcc840d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49924/0 (socket says 172.21.15.59:49924) 2024-04-23T14:32:41.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.987+0000 7fcc83e00640 1 -- 172.21.15.59:0/3959409645 learned_addr learned my addr 172.21.15.59:0/3959409645 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.988+0000 7fcc83400640 1 --2- 172.21.15.59:0/3959409645 >> v2:172.21.15.76:3300/0 conn(0x7fcc840af4b0 0x7fcc8415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.988+0000 7fcc7be00640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc840af890 0x7fcc84144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.988+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 480028161 0 0) 0x7fcc8415f3f0 con 0x7fcc840b3010 2024-04-23T14:32:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.988+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc68003520 con 0x7fcc840b3010 2024-04-23T14:32:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.988+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1304287276 0 0) 0x7fcc68003520 con 0x7fcc840b3010 2024-04-23T14:32:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc840af890 msgr2=0x7fcc84144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc840af890 0x7fcc84144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 >> v2:172.21.15.76:3300/0 conn(0x7fcc840af4b0 msgr2=0x7fcc8415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 --2- 172.21.15.59:0/3959409645 >> v2:172.21.15.76:3300/0 conn(0x7fcc840af4b0 0x7fcc8415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8415f9b0 con 0x7fcc840b3010 2024-04-23T14:32:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fcc7c0019f0 con 0x7fcc840b3010 2024-04-23T14:32:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 >> v1:172.21.15.59:6789/0 conn(0x7fcc840b3010 legacy=0x7fcc840d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc680039b0 0x7fcc68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc68004800 0x7fcc68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc68005180 0x7fcc6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84144f60 con 0x7fcc680039b0 2024-04-23T14:32:41.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8415f6d0 con 0x7fcc68005180 2024-04-23T14:32:41.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcc68004600 con 0x7fcc68004800 2024-04-23T14:32:41.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc83400640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc680039b0 0x7fcc68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.989+0000 7fcc83e00640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc68005180 0x7fcc6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:41.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.990+0000 7fcc83400640 1 -- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc68004800 msgr2=0x7fcc68004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.990+0000 7fcc83400640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc68004800 0x7fcc68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.990+0000 7fcc83400640 1 -- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc68005180 msgr2=0x7fcc6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.990+0000 7fcc83400640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc68005180 0x7fcc6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.990+0000 7fcc83400640 1 -- 172.21.15.59:0/3959409645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc68003520 con 0x7fcc680039b0 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.991+0000 7fcc83400640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc680039b0 0x7fcc68003dd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009ac0 tx=0x7fcc68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873bd494b515c6af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.991+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74036070 con 0x7fcc680039b0 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.991+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc74031400 con 0x7fcc680039b0 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.991+0000 7fcc82a00640 1 -- 172.21.15.59:0/3959409645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74009690 con 0x7fcc680039b0 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.992+0000 7fcc88a00640 1 -- 172.21.15.59:0/3959409645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc680039b0 msgr2=0x7fcc68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.992+0000 7fcc88a00640 1 --2- 172.21.15.59:0/3959409645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc680039b0 0x7fcc68003dd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009ac0 tx=0x7fcc68005830 comp rx=0 tx=0).stop 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.992+0000 7fcc83e00640 1 -- 172.21.15.59:0/3959409645 reap_dead start 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.992+0000 7fcc88a00640 1 -- 172.21.15.59:0/3959409645 shutdown_connections 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.992+0000 7fcc88a00640 1 -- 172.21.15.59:0/3959409645 >> 172.21.15.59:0/3959409645 conn(0x7fcc8406bdf0 msgr2=0x7fcc8406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:41.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.992+0000 7fcc88a00640 1 -- 172.21.15.59:0/3959409645 shutdown_connections 2024-04-23T14:32:41.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.992+0000 7fcc88a00640 1 -- 172.21.15.59:0/3959409645 wait complete. 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.994+0000 7fcc88a00640 1 Processor -- start 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.994+0000 7fcc88a00640 1 -- start start 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc88a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc680039b0 0x7fcc8406db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc88a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc8415b090 0x7fcc841e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc88a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc841f6f30 0x7fcc841f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc88a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc840d0e50 con 0x7fcc8415b090 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc88a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc840d0cd0 con 0x7fcc841f6f30 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc88a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcc840b34c0 con 0x7fcc680039b0 2024-04-23T14:32:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc7be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc8415b090 0x7fcc841e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc7be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc8415b090 0x7fcc841e8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49984/0 (socket says 172.21.15.59:49984) 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.995+0000 7fcc7be00640 1 -- 172.21.15.59:0/955809034 learned_addr learned my addr 172.21.15.59:0/955809034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc83e00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc841f6f30 0x7fcc841f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7be00640 1 -- 172.21.15.59:0/955809034 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc680039b0 msgr2=0x7fcc8406db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7be00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc680039b0 0x7fcc8406db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7be00640 1 -- 172.21.15.59:0/955809034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc841f6f30 msgr2=0x7fcc841f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7be00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc841f6f30 0x7fcc841f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7be00640 1 -- 172.21.15.59:0/955809034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8406e0c0 con 0x7fcc8415b090 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7be00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc8415b090 0x7fcc841e8d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00e7f0 tx=0x7fcc6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7b400640 1 -- 172.21.15.59:0/955809034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x7fcc8415b090 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7b400640 1 -- 172.21.15.59:0/955809034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c00c940 con 0x7fcc8415b090 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc7b400640 1 -- 172.21.15.59:0/955809034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00cd10 con 0x7fcc8415b090 2024-04-23T14:32:42.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.996+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8415b5e0 con 0x7fcc8415b090 2024-04-23T14:32:42.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.997+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc84159780 con 0x7fcc8415b090 2024-04-23T14:32:42.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.998+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8404fd90 con 0x7fcc8415b090 2024-04-23T14:32:42.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:41.999+0000 7fcc7b400640 1 -- 172.21.15.59:0/955809034 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc6c01a070 con 0x7fcc8415b090 2024-04-23T14:32:42.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.000+0000 7fcc7b400640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc6006c8f0 0x7fcc6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.000+0000 7fcc7b400640 1 -- 172.21.15.59:0/955809034 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fcc6c08b6c0 con 0x7fcc8415b090 2024-04-23T14:32:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.000+0000 7fcc83400640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc6006c8f0 0x7fcc6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.001+0000 7fcc83400640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc6006c8f0 0x7fcc6006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc74033000 tx=0x7fcc7402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.004+0000 7fcc7b400640 1 -- 172.21.15.59:0/955809034 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc6c059e30 con 0x7fcc8415b090 2024-04-23T14:32:42.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.296+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fcc840cedb0 con 0x7fcc8415b090 2024-04-23T14:32:42.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.301+0000 7fcc7b400640 1 -- 172.21.15.59:0/955809034 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fcc6c059730 con 0x7fcc8415b090 2024-04-23T14:32:42.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc6006c8f0 msgr2=0x7fcc6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc6006c8f0 0x7fcc6006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc74033000 tx=0x7fcc7402e000 comp rx=0 tx=0).stop 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc8415b090 msgr2=0x7fcc841e8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc8415b090 0x7fcc841e8d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00e7f0 tx=0x7fcc6c00ecb0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 shutdown_connections 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc6006c8f0 0x7fcc6006edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc841f6f30 0x7fcc841f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc8415b090 0x7fcc841e8d00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 --2- 172.21.15.59:0/955809034 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc680039b0 0x7fcc8406db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.303+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 >> 172.21.15.59:0/955809034 conn(0x7fcc8406bdf0 msgr2=0x7fcc84074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.304+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 shutdown_connections 2024-04-23T14:32:42.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.304+0000 7fcc88a00640 1 -- 172.21.15.59:0/955809034 wait complete. 2024-04-23T14:32:42.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-23T14:32:42.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.537+0000 7f325d200640 1 Processor -- start 2024-04-23T14:32:42.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.537+0000 7f325d200640 1 -- start start 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f325d200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f325815c670 0x7f32580cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f325d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32580d0cd0 0x7f32580b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f325d200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32580b4cf0 con 0x7f32580b4230 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f325d200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f325815ca50 con 0x7f32580d0cd0 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f325d200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f32580ccb80 con 0x7f325815c670 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f3257e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32580d0cd0 0x7f32580b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f3257e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32580d0cd0 0x7f32580b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49996/0 (socket says 172.21.15.59:49996) 2024-04-23T14:32:42.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.539+0000 7f3257e00640 1 -- 172.21.15.59:0/1651627634 learned_addr learned my addr 172.21.15.59:0/1651627634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.540+0000 7f3257e00640 1 -- 172.21.15.59:0/1651627634 >> v2:172.21.15.76:3300/0 conn(0x7f325815c670 msgr2=0x7f32580cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.540+0000 7f3257400640 1 --2- 172.21.15.59:0/1651627634 >> v2:172.21.15.76:3300/0 conn(0x7f325815c670 0x7f32580cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.540+0000 7f3257e00640 1 --2- 172.21.15.59:0/1651627634 >> v2:172.21.15.76:3300/0 conn(0x7f325815c670 0x7f32580cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.540+0000 7f3257e00640 1 -- 172.21.15.59:0/1651627634 >> v1:172.21.15.76:6789/0 conn(0x7f32580b4230 legacy=0x7f32580d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.540+0000 7f3257e00640 1 -- 172.21.15.59:0/1651627634 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32580ccdd0 con 0x7f32580d0cd0 2024-04-23T14:32:42.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.541+0000 7f3257e00640 1 --2- 172.21.15.59:0/1651627634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580d0cd0 0x7f32580b4690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f32580d1270 tx=0x7f325002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c22f8166711954b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.541+0000 7f3256000640 1 -- 172.21.15.59:0/1651627634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250036070 con 0x7f32580d0cd0 2024-04-23T14:32:42.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.541+0000 7f3256000640 1 -- 172.21.15.59:0/1651627634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3250031600 con 0x7f32580d0cd0 2024-04-23T14:32:42.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.541+0000 7f3256000640 1 -- 172.21.15.59:0/1651627634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32500319d0 con 0x7f32580d0cd0 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 -- 172.21.15.59:0/1651627634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580d0cd0 msgr2=0x7f32580b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 --2- 172.21.15.59:0/1651627634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580d0cd0 0x7f32580b4690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f32580d1270 tx=0x7f325002a6f0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 -- 172.21.15.59:0/1651627634 shutdown_connections 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 --2- 172.21.15.59:0/1651627634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580d0cd0 0x7f32580b4690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 --2- 172.21.15.59:0/1651627634 >> v2:172.21.15.76:3300/0 conn(0x7f325815c670 0x7f32580cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 -- 172.21.15.59:0/1651627634 >> 172.21.15.59:0/1651627634 conn(0x7f325806bdf0 msgr2=0x7f325806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 -- 172.21.15.59:0/1651627634 shutdown_connections 2024-04-23T14:32:42.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.542+0000 7f325d200640 1 -- 172.21.15.59:0/1651627634 wait complete. 2024-04-23T14:32:42.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.544+0000 7f325d200640 1 Processor -- start 2024-04-23T14:32:42.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.544+0000 7f325d200640 1 -- start start 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f325d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580b4230 0x7f3258148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f325d200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580d0cd0 0x7f325809c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f325d200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f325815c670 0x7f325815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f325d200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32580b3e60 con 0x7f32580b4230 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f325d200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32580cd390 con 0x7f32580d0cd0 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f325d200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32580b3fe0 con 0x7f325815c670 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f3257400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580b4230 0x7f3258148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f3257400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580b4230 0x7f3258148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50008/0 (socket says 172.21.15.59:50008) 2024-04-23T14:32:42.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f3257400640 1 -- 172.21.15.59:0/479172055 learned_addr learned my addr 172.21.15.59:0/479172055 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:42.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.545+0000 7f3256a00640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580d0cd0 0x7f325809c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f3257400640 1 -- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f325815c670 msgr2=0x7f325815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f3257400640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f325815c670 0x7f325815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f3257400640 1 -- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580d0cd0 msgr2=0x7f325809c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f3257400640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580d0cd0 0x7f325809c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f3257400640 1 -- 172.21.15.59:0/479172055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32581f2a70 con 0x7f32580b4230 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f3256a00640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580d0cd0 0x7f325809c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f3257400640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580b4230 0x7f3258148070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f324800c820 tx=0x7f324800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f324f400640 1 -- 172.21.15.59:0/479172055 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248015070 con 0x7f32580b4230 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f324f400640 1 -- 172.21.15.59:0/479172055 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3248004020 con 0x7f32580b4230 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3258148650 con 0x7f32580b4230 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.546+0000 7f324f400640 1 -- 172.21.15.59:0/479172055 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32480043f0 con 0x7f32580b4230 2024-04-23T14:32:42.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.547+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3258148c10 con 0x7f32580b4230 2024-04-23T14:32:42.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.548+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325804fd90 con 0x7f32580b4230 2024-04-23T14:32:42.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.549+0000 7f324f400640 1 -- 172.21.15.59:0/479172055 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32480077b0 con 0x7f32580b4230 2024-04-23T14:32:42.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.550+0000 7f324f400640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f323806c8f0 0x7f323806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:42.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.550+0000 7f324f400640 1 -- 172.21.15.59:0/479172055 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f324808b5e0 con 0x7f32580b4230 2024-04-23T14:32:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.550+0000 7f3256a00640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f323806c8f0 0x7f323806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:42.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.551+0000 7f3256a00640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f323806c8f0 0x7f323806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f325815f630 tx=0x7f3244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:42.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.554+0000 7f324f400640 1 -- 172.21.15.59:0/479172055 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3248059cc0 con 0x7f32580b4230 2024-04-23T14:32:42.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.825+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f325815d470 con 0x7f32580b4230 2024-04-23T14:32:42.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.830+0000 7f324f400640 1 -- 172.21.15.59:0/479172055 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f32480595c0 con 0x7f32580b4230 2024-04-23T14:32:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f323806c8f0 msgr2=0x7f323806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f323806c8f0 0x7f323806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f325815f630 tx=0x7f3244009040 comp rx=0 tx=0).stop 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580b4230 msgr2=0x7f3258148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580b4230 0x7f3258148070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f324800c820 tx=0x7f324800cce0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 shutdown_connections 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f323806c8f0 0x7f323806edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f325815c670 0x7f325815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32580d0cd0 0x7f325809c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 --2- 172.21.15.59:0/479172055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32580b4230 0x7f3258148070 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 >> 172.21.15.59:0/479172055 conn(0x7f325806bdf0 msgr2=0x7f32581557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.832+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 shutdown_connections 2024-04-23T14:32:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:42.833+0000 7f325d200640 1 -- 172.21.15.59:0/479172055 wait complete. 2024-04-23T14:32:42.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-23T14:32:43.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.033+0000 7f9c8a200640 1 Processor -- start 2024-04-23T14:32:43.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.033+0000 7f9c8a200640 1 -- start start 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.034+0000 7f9c8a200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9c840af4b0 0x7f9c8415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c8a200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c840af890 0x7f9c84144a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c8a200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8415f3f0 con 0x7f9c840b3010 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c8a200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9c84144f60 con 0x7f9c840af4b0 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c8a200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8415f6d0 con 0x7f9c840af890 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c89800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9c840b3010 0x7f9c840d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49930/0 (socket says 172.21.15.59:49930) 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c89800640 1 -- 172.21.15.59:0/1915412172 learned_addr learned my addr 172.21.15.59:0/1915412172 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c88e00640 1 --2- 172.21.15.59:0/1915412172 >> v2:172.21.15.59:3300/0 conn(0x7f9c840af4b0 0x7f9c8415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c83e00640 1 --2- 172.21.15.59:0/1915412172 >> v2:172.21.15.76:3300/0 conn(0x7f9c840af890 0x7f9c84144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c83400640 1 -- 172.21.15.59:0/1915412172 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352026405 0 0) 0x7f9c8415f3f0 con 0x7f9c840b3010 2024-04-23T14:32:43.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.035+0000 7f9c83400640 1 -- 172.21.15.59:0/1915412172 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c6c003520 con 0x7f9c840b3010 2024-04-23T14:32:43.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.036+0000 7f9c88e00640 1 -- 172.21.15.59:0/1915412172 >> v2:172.21.15.76:3300/0 conn(0x7f9c840af890 msgr2=0x7f9c84144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.036+0000 7f9c88e00640 1 --2- 172.21.15.59:0/1915412172 >> v2:172.21.15.76:3300/0 conn(0x7f9c840af890 0x7f9c84144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.036+0000 7f9c88e00640 1 -- 172.21.15.59:0/1915412172 >> v1:172.21.15.59:6789/0 conn(0x7f9c840b3010 legacy=0x7f9c840d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.036+0000 7f9c88e00640 1 -- 172.21.15.59:0/1915412172 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8415f9b0 con 0x7f9c840af4b0 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.037+0000 7f9c88e00640 1 --2- 172.21.15.59:0/1915412172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c8415ed50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c840cdc00 tx=0x7f9c7802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753f33323b79649c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.037+0000 7f9c83400640 1 -- 172.21.15.59:0/1915412172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78037070 con 0x7f9c840af4b0 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.037+0000 7f9c83400640 1 -- 172.21.15.59:0/1915412172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c78032440 con 0x7f9c840af4b0 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.037+0000 7f9c83400640 1 -- 172.21.15.59:0/1915412172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780327f0 con 0x7f9c840af4b0 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.037+0000 7f9c8a200640 1 -- 172.21.15.59:0/1915412172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 msgr2=0x7f9c8415ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.037+0000 7f9c8a200640 1 --2- 172.21.15.59:0/1915412172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c8415ed50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c840cdc00 tx=0x7f9c7802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.038+0000 7f9c8a200640 1 -- 172.21.15.59:0/1915412172 shutdown_connections 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.038+0000 7f9c8a200640 1 --2- 172.21.15.59:0/1915412172 >> v2:172.21.15.76:3300/0 conn(0x7f9c840af890 0x7f9c84144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.038+0000 7f9c8a200640 1 --2- 172.21.15.59:0/1915412172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c8415ed50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.038+0000 7f9c8a200640 1 -- 172.21.15.59:0/1915412172 >> 172.21.15.59:0/1915412172 conn(0x7f9c8406bdf0 msgr2=0x7f9c8406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.038+0000 7f9c8a200640 1 -- 172.21.15.59:0/1915412172 shutdown_connections 2024-04-23T14:32:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.038+0000 7f9c8a200640 1 -- 172.21.15.59:0/1915412172 wait complete. 2024-04-23T14:32:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.039+0000 7f9c8a200640 1 Processor -- start 2024-04-23T14:32:43.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.040+0000 7f9c8a200640 1 -- start start 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.040+0000 7f9c8a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c84158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c8a200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c840b3010 0x7f9c8415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c8a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c8415fea0 0x7f9c84160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c8a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c840d0e50 con 0x7f9c840af4b0 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c8a200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9c840d0cd0 con 0x7f9c840b3010 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c8a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840b34c0 con 0x7f9c8415fea0 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c84158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c84158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50028/0 (socket says 172.21.15.59:50028) 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 -- 172.21.15.59:0/2348343028 learned_addr learned my addr 172.21.15.59:0/2348343028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c83e00640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c840b3010 0x7f9c8415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 -- 172.21.15.59:0/2348343028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c8415fea0 msgr2=0x7f9c84160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c8415fea0 0x7f9c84160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 -- 172.21.15.59:0/2348343028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c840b3010 msgr2=0x7f9c8415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c840b3010 0x7f9c8415b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.041+0000 7f9c88e00640 1 -- 172.21.15.59:0/2348343028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c84159240 con 0x7f9c840af4b0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.042+0000 7f9c88e00640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c84158d00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000c00 tx=0x7f9c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.042+0000 7f9c80c00640 1 -- 172.21.15.59:0/2348343028 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78037040 con 0x7f9c840af4b0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.042+0000 7f9c80c00640 1 -- 172.21.15.59:0/2348343028 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c780304c0 con 0x7f9c840af4b0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.042+0000 7f9c80c00640 1 -- 172.21.15.59:0/2348343028 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78030890 con 0x7f9c840af4b0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.042+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8406ecb0 con 0x7f9c840af4b0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.042+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8406f270 con 0x7f9c840af4b0 2024-04-23T14:32:43.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.043+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8404fd90 con 0x7f9c840af4b0 2024-04-23T14:32:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.044+0000 7f9c80c00640 1 -- 172.21.15.59:0/2348343028 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c78044020 con 0x7f9c840af4b0 2024-04-23T14:32:43.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.045+0000 7f9c80c00640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9c6406c8f0 0x7f9c6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.045+0000 7f9c80c00640 1 -- 172.21.15.59:0/2348343028 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9c780abad0 con 0x7f9c840af4b0 2024-04-23T14:32:43.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.046+0000 7f9c83e00640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9c6406c8f0 0x7f9c6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.046+0000 7f9c83e00640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9c6406c8f0 0x7f9c6406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c840cdb80 tx=0x7f9c70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.050+0000 7f9c80c00640 1 -- 172.21.15.59:0/2348343028 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c84159240 con 0x7f9c840af4b0 2024-04-23T14:32:43.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.315+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9c840cedb0 con 0x7f9c840af4b0 2024-04-23T14:32:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.319+0000 7f9c80c00640 1 -- 172.21.15.59:0/2348343028 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f9c7807a250 con 0x7f9c840af4b0 2024-04-23T14:32:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9c6406c8f0 msgr2=0x7f9c6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9c6406c8f0 0x7f9c6406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c840cdb80 tx=0x7f9c70008040 comp rx=0 tx=0).stop 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 msgr2=0x7f9c84158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c84158d00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000c00 tx=0x7f9c780046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 shutdown_connections 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9c6406c8f0 0x7f9c6406edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c8415fea0 0x7f9c84160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c840b3010 0x7f9c8415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 --2- 172.21.15.59:0/2348343028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c840af4b0 0x7f9c84158d00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 >> 172.21.15.59:0/2348343028 conn(0x7f9c8406bdf0 msgr2=0x7f9c840afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.322+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 shutdown_connections 2024-04-23T14:32:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.323+0000 7f9c8a200640 1 -- 172.21.15.59:0/2348343028 wait complete. 2024-04-23T14:32:43.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-23T14:32:43.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.539+0000 7fded9000640 1 Processor -- start 2024-04-23T14:32:43.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.540+0000 7fded9000640 1 -- start start 2024-04-23T14:32:43.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.541+0000 7fded9000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fded415c670 0x7fded40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.541+0000 7fded9000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded40b4cf0 con 0x7fded40d0cd0 2024-04-23T14:32:43.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.541+0000 7fded9000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded40ccdd0 con 0x7fded40b4230 2024-04-23T14:32:43.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.541+0000 7fded9000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fded415ca50 con 0x7fded415c670 2024-04-23T14:32:43.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fded40d0cd0 0x7fded406ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49944/0 (socket says 172.21.15.59:49944) 2024-04-23T14:32:43.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded3e00640 1 -- 172.21.15.59:0/3106466368 learned_addr learned my addr 172.21.15.59:0/3106466368 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded3400640 1 --2- 172.21.15.59:0/3106466368 >> v2:172.21.15.76:3300/0 conn(0x7fded415c670 0x7fded40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3904564190 0 0) 0x7fded40b4cf0 con 0x7fded40d0cd0 2024-04-23T14:32:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdeb8003520 con 0x7fded40d0cd0 2024-04-23T14:32:43.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 289706268 0 0) 0x7fdeb8003520 con 0x7fded40d0cd0 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 >> v2:172.21.15.76:3300/0 conn(0x7fded415c670 msgr2=0x7fded40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 --2- 172.21.15.59:0/3106466368 >> v2:172.21.15.76:3300/0 conn(0x7fded415c670 0x7fded40cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 >> v1:172.21.15.76:6789/0 conn(0x7fded40b4230 legacy=0x7fded40d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded40cd0b0 con 0x7fded40d0cd0 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fdecc0019f0 con 0x7fded40d0cd0 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.542+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 >> v1:172.21.15.59:6789/0 conn(0x7fded40d0cd0 legacy=0x7fded406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2000640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb80039b0 0x7fdeb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2000640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb8004800 0x7fdeb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2000640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb80051c0 0x7fdeb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb800c0e0 con 0x7fdeb8004800 2024-04-23T14:32:43.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb800c290 con 0x7fdeb80039b0 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb800c490 con 0x7fdeb80051c0 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2a00640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb8004800 0x7fdeb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2a00640 1 -- 172.21.15.59:0/3106466368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb80051c0 msgr2=0x7fdeb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2a00640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb80051c0 0x7fdeb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2a00640 1 -- 172.21.15.59:0/3106466368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb80039b0 msgr2=0x7fdeb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2a00640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb80039b0 0x7fdeb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded2a00640 1 -- 172.21.15.59:0/3106466368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb8003520 con 0x7fdeb8004800 2024-04-23T14:32:43.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.543+0000 7fded3e00640 1 -- 172.21.15.59:0/3106466368 reap_dead start 2024-04-23T14:32:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.544+0000 7fded2a00640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb8004800 0x7fdeb8004c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdec0001720 tx=0x7fdeb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2fe181d23f81c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.544+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0001370 con 0x7fdeb8004800 2024-04-23T14:32:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec00052d0 con 0x7fdeb8004800 2024-04-23T14:32:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded2000640 1 -- 172.21.15.59:0/3106466368 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec002ab90 con 0x7fdeb8004800 2024-04-23T14:32:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded9000640 1 -- 172.21.15.59:0/3106466368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb8004800 msgr2=0x7fdeb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded9000640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb8004800 0x7fdeb8004c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdec0001720 tx=0x7fdeb8005830 comp rx=0 tx=0).stop 2024-04-23T14:32:43.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded9000640 1 -- 172.21.15.59:0/3106466368 shutdown_connections 2024-04-23T14:32:43.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded9000640 1 --2- 172.21.15.59:0/3106466368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb8004800 0x7fdeb8004c80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded9000640 1 -- 172.21.15.59:0/3106466368 >> 172.21.15.59:0/3106466368 conn(0x7fded406bdf0 msgr2=0x7fded406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded9000640 1 -- 172.21.15.59:0/3106466368 shutdown_connections 2024-04-23T14:32:43.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.545+0000 7fded9000640 1 -- 172.21.15.59:0/3106466368 wait complete. 2024-04-23T14:32:43.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.547+0000 7fded9000640 1 Processor -- start 2024-04-23T14:32:43.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.547+0000 7fded9000640 1 -- start start 2024-04-23T14:32:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded9000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb8004800 0x7fded41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fded415cb30 0x7fded4152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded9000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fded415d970 0x7fded415ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded9000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fded40b3e60 con 0x7fded415cb30 2024-04-23T14:32:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded9000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fded40ccb80 con 0x7fded415d970 2024-04-23T14:32:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded9000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fded40b3fe0 con 0x7fdeb8004800 2024-04-23T14:32:43.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fded415cb30 0x7fded4152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fded415cb30 0x7fded4152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50054/0 (socket says 172.21.15.59:50054) 2024-04-23T14:32:43.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.548+0000 7fded2a00640 1 -- 172.21.15.59:0/648969886 learned_addr learned my addr 172.21.15.59:0/648969886 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fded2a00640 1 -- 172.21.15.59:0/648969886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb8004800 msgr2=0x7fded41465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fded2a00640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb8004800 0x7fded41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fded2a00640 1 -- 172.21.15.59:0/648969886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fded415d970 msgr2=0x7fded415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fded2a00640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fded415d970 0x7fded415ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fded2a00640 1 -- 172.21.15.59:0/648969886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded4153440 con 0x7fded415cb30 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fded2a00640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fded415cb30 0x7fded4152ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fdecb400640 1 -- 172.21.15.59:0/648969886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec003f030 con 0x7fded415cb30 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fdecb400640 1 -- 172.21.15.59:0/648969886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec0005480 con 0x7fded415cb30 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fdecb400640 1 -- 172.21.15.59:0/648969886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0037040 con 0x7fded415cb30 2024-04-23T14:32:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.549+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded4153720 con 0x7fded415cb30 2024-04-23T14:32:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.550+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded4153ce0 con 0x7fded415cb30 2024-04-23T14:32:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.551+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded404fd90 con 0x7fded415cb30 2024-04-23T14:32:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.552+0000 7fdecb400640 1 -- 172.21.15.59:0/648969886 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdec0042070 con 0x7fded415cb30 2024-04-23T14:32:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.552+0000 7fdecb400640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.553+0000 7fdecb400640 1 -- 172.21.15.59:0/648969886 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdec00aae90 con 0x7fded415cb30 2024-04-23T14:32:43.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.553+0000 7fded3400640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.553+0000 7fded3400640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdeb406c8f0 0x7fdeb406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fded415d920 tx=0x7fdec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:43.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.557+0000 7fdecb400640 1 -- 172.21.15.59:0/648969886 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdec00794e0 con 0x7fded415cb30 2024-04-23T14:32:43.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.847+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fded40ceab0 con 0x7fded415cb30 2024-04-23T14:32:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.852+0000 7fdecb400640 1 -- 172.21.15.59:0/648969886 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fdec0078de0 con 0x7fded415cb30 2024-04-23T14:32:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdeb406c8f0 msgr2=0x7fdeb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdeb406c8f0 0x7fdeb406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fded415d920 tx=0x7fdec4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fded415cb30 msgr2=0x7fded4152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fded415cb30 0x7fded4152ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0001120 comp rx=0 tx=0).stop 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 shutdown_connections 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fded415d970 0x7fded415ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fded415cb30 0x7fded4152ad0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 --2- 172.21.15.59:0/648969886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb8004800 0x7fded41465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.855+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 >> 172.21.15.59:0/648969886 conn(0x7fded406bdf0 msgr2=0x7fded41600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.856+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 shutdown_connections 2024-04-23T14:32:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:43.856+0000 7fded9000640 1 -- 172.21.15.59:0/648969886 wait complete. 2024-04-23T14:32:43.877 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-23T14:32:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.091+0000 7fab0e800640 1 Processor -- start 2024-04-23T14:32:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.091+0000 7fab0e800640 1 -- start start 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab0815c670 0x7fab080cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0e800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab080b4230 0x7fab080b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0e800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab080b4cf0 con 0x7fab080d05c0 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0e800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fab0815ca50 con 0x7fab0815c670 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0e800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fab080ccb80 con 0x7fab080b4230 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fab080d05c0 0x7fab08160160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49952/0 (socket says 172.21.15.59:49952) 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0de00640 1 -- 172.21.15.59:0/1469357438 learned_addr learned my addr 172.21.15.59:0/1469357438 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0d400640 1 --2- 172.21.15.59:0/1469357438 >> v2:172.21.15.59:3300/0 conn(0x7fab0815c670 0x7fab080cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.093+0000 7fab0ca00640 1 --2- 172.21.15.59:0/1469357438 >> v2:172.21.15.76:3300/0 conn(0x7fab080b4230 0x7fab080b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.094+0000 7fab0d400640 1 -- 172.21.15.59:0/1469357438 >> v2:172.21.15.76:3300/0 conn(0x7fab080b4230 msgr2=0x7fab080b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:44.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.094+0000 7fab0d400640 1 --2- 172.21.15.59:0/1469357438 >> v2:172.21.15.76:3300/0 conn(0x7fab080b4230 0x7fab080b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.094+0000 7fab0d400640 1 -- 172.21.15.59:0/1469357438 >> v1:172.21.15.59:6789/0 conn(0x7fab080d05c0 legacy=0x7fab08160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.094+0000 7fab0d400640 1 -- 172.21.15.59:0/1469357438 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab080ccdd0 con 0x7fab0815c670 2024-04-23T14:32:44.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.095+0000 7fab0d400640 1 --2- 172.21.15.59:0/1469357438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0815c670 0x7fab080cd8b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab080ce7e0 tx=0x7faafc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21a4be050704237 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.095+0000 7fab07e00640 1 -- 172.21.15.59:0/1469357438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc037070 con 0x7fab0815c670 2024-04-23T14:32:44.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.095+0000 7fab07e00640 1 -- 172.21.15.59:0/1469357438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc032440 con 0x7fab0815c670 2024-04-23T14:32:44.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.095+0000 7fab07e00640 1 -- 172.21.15.59:0/1469357438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc032810 con 0x7fab0815c670 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 -- 172.21.15.59:0/1469357438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0815c670 msgr2=0x7fab080cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 --2- 172.21.15.59:0/1469357438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0815c670 0x7fab080cd8b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab080ce7e0 tx=0x7faafc02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 -- 172.21.15.59:0/1469357438 shutdown_connections 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 --2- 172.21.15.59:0/1469357438 >> v2:172.21.15.76:3300/0 conn(0x7fab080b4230 0x7fab080b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 --2- 172.21.15.59:0/1469357438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab0815c670 0x7fab080cd8b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 -- 172.21.15.59:0/1469357438 >> 172.21.15.59:0/1469357438 conn(0x7fab0806bdf0 msgr2=0x7fab0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 -- 172.21.15.59:0/1469357438 shutdown_connections 2024-04-23T14:32:44.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.096+0000 7fab0e800640 1 -- 172.21.15.59:0/1469357438 wait complete. 2024-04-23T14:32:44.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.098+0000 7fab0e800640 1 Processor -- start 2024-04-23T14:32:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.098+0000 7fab0e800640 1 -- start start 2024-04-23T14:32:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080b4230 0x7fab081e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0e800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fab080d05c0 0x7fab081e7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0e800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0815c670 0x7fab0814a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab080b3e60 con 0x7fab080b4230 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0e800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fab080cd390 con 0x7fab0815c670 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0e800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fab080b3fe0 con 0x7fab080d05c0 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080b4230 0x7fab081e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080b4230 0x7fab081e9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50078/0 (socket says 172.21.15.59:50078) 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.099+0000 7fab0d400640 1 -- 172.21.15.59:0/1200895753 learned_addr learned my addr 172.21.15.59:0/1200895753 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.100+0000 7fab0d400640 1 -- 172.21.15.59:0/1200895753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fab080d05c0 msgr2=0x7fab081e7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.100+0000 7fab0d400640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fab080d05c0 0x7fab081e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.100+0000 7fab0d400640 1 -- 172.21.15.59:0/1200895753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0815c670 msgr2=0x7fab0814a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.100+0000 7fab0d400640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0815c670 0x7fab0814a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.100+0000 7fab0d400640 1 -- 172.21.15.59:0/1200895753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab081e7aa0 con 0x7fab080b4230 2024-04-23T14:32:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.100+0000 7fab0d400640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080b4230 0x7fab081e9d50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faafc000c00 tx=0x7faafc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.100+0000 7fab05600640 1 -- 172.21.15.59:0/1200895753 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc037040 con 0x7fab080b4230 2024-04-23T14:32:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.101+0000 7fab05600640 1 -- 172.21.15.59:0/1200895753 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc042070 con 0x7fab080b4230 2024-04-23T14:32:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.101+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0814adb0 con 0x7fab080b4230 2024-04-23T14:32:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.101+0000 7fab05600640 1 -- 172.21.15.59:0/1200895753 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc030bb0 con 0x7fab080b4230 2024-04-23T14:32:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.101+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab081e8000 con 0x7fab080b4230 2024-04-23T14:32:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.102+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0804fd90 con 0x7fab080b4230 2024-04-23T14:32:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.103+0000 7fab05600640 1 -- 172.21.15.59:0/1200895753 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faafc047460 con 0x7fab080b4230 2024-04-23T14:32:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.104+0000 7fab05600640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faae806c8f0 0x7faae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.104+0000 7fab05600640 1 -- 172.21.15.59:0/1200895753 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7faafc0ac4a0 con 0x7fab080b4230 2024-04-23T14:32:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.104+0000 7fab0ca00640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faae806c8f0 0x7faae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.105+0000 7fab0ca00640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faae806c8f0 0x7faae806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab081e92e0 tx=0x7faaf4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.109+0000 7fab05600640 1 -- 172.21.15.59:0/1200895753 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab081e7aa0 con 0x7fab080b4230 2024-04-23T14:32:44.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.398+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fab080cc810 con 0x7fab080b4230 2024-04-23T14:32:44.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.403+0000 7fab05600640 1 -- 172.21.15.59:0/1200895753 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v10) ==== 97+0+0 (secure 0 0 0) 0x7faafc032e20 con 0x7fab080b4230 2024-04-23T14:32:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faae806c8f0 msgr2=0x7faae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faae806c8f0 0x7faae806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab081e92e0 tx=0x7faaf4008040 comp rx=0 tx=0).stop 2024-04-23T14:32:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080b4230 msgr2=0x7fab081e9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080b4230 0x7fab081e9d50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faafc000c00 tx=0x7faafc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 shutdown_connections 2024-04-23T14:32:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faae806c8f0 0x7faae806edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fab080d05c0 0x7fab081e7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab0815c670 0x7fab0814a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 --2- 172.21.15.59:0/1200895753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab080b4230 0x7fab081e9d50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 >> 172.21.15.59:0/1200895753 conn(0x7fab0806bdf0 msgr2=0x7fab0815a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 shutdown_connections 2024-04-23T14:32:44.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.406+0000 7fab0e800640 1 -- 172.21.15.59:0/1200895753 wait complete. 2024-04-23T14:32:44.427 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-23T14:32:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.633+0000 7f4b0d800640 1 Processor -- start 2024-04-23T14:32:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.633+0000 7f4b0d800640 1 -- start start 2024-04-23T14:32:44.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.634+0000 7f4b0d800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4b0815c670 0x7f4b080cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.634+0000 7f4b0d800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b080b4230 0x7f4b080b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.634+0000 7f4b0d800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b080d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.634+0000 7f4b0d800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0815ca50 con 0x7f4b0815c670 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.634+0000 7f4b0d800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4b080ccb80 con 0x7f4b080b4230 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.634+0000 7f4b0d800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4b080b4cf0 con 0x7f4b080d05c0 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4b0815c670 0x7f4b080cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4b0815c670 0x7f4b080cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50080/0 (socket says 172.21.15.59:50080) 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 -- 172.21.15.59:0/1301513623 learned_addr learned my addr 172.21.15.59:0/1301513623 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4affe00640 1 --2- 172.21.15.59:0/1301513623 >> v2:172.21.15.76:3300/0 conn(0x7f4b080b4230 0x7f4b080b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b0ce00640 1 --2- 172.21.15.59:0/1301513623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b080d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 -- 172.21.15.59:0/1301513623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 msgr2=0x7f4b080d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 --2- 172.21.15.59:0/1301513623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b080d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 -- 172.21.15.59:0/1301513623 >> v2:172.21.15.76:3300/0 conn(0x7f4b080b4230 msgr2=0x7f4b080b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 --2- 172.21.15.59:0/1301513623 >> v2:172.21.15.76:3300/0 conn(0x7f4b080b4230 0x7f4b080b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b07e00640 1 -- 172.21.15.59:0/1301513623 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080ccdd0 con 0x7f4b0815c670 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4affe00640 1 --2- 172.21.15.59:0/1301513623 >> v2:172.21.15.76:3300/0 conn(0x7f4b080b4230 0x7f4b080b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:44.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.635+0000 7f4b0ce00640 1 --2- 172.21.15.59:0/1301513623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b080d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.637+0000 7f4b07e00640 1 --2- 172.21.15.59:0/1301513623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b080cd8b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ce7e0 tx=0x7f4af802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67dd98f06b8cee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.637+0000 7f4b07400640 1 -- 172.21.15.59:0/1301513623 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8037070 con 0x7f4b0815c670 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.637+0000 7f4b07400640 1 -- 172.21.15.59:0/1301513623 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af8032440 con 0x7f4b0815c670 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.637+0000 7f4b07400640 1 -- 172.21.15.59:0/1301513623 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8032810 con 0x7f4b0815c670 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 -- 172.21.15.59:0/1301513623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 msgr2=0x7f4b080cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 --2- 172.21.15.59:0/1301513623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b080cd8b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ce7e0 tx=0x7f4af802a1e0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 -- 172.21.15.59:0/1301513623 shutdown_connections 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 --2- 172.21.15.59:0/1301513623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b080d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 --2- 172.21.15.59:0/1301513623 >> v2:172.21.15.76:3300/0 conn(0x7f4b080b4230 0x7f4b080b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 --2- 172.21.15.59:0/1301513623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b080cd8b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 -- 172.21.15.59:0/1301513623 >> 172.21.15.59:0/1301513623 conn(0x7f4b0806bdf0 msgr2=0x7f4b0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 -- 172.21.15.59:0/1301513623 shutdown_connections 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.638+0000 7f4b0d800640 1 -- 172.21.15.59:0/1301513623 wait complete. 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.640+0000 7f4b0d800640 1 Processor -- start 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.640+0000 7f4b0d800640 1 -- start start 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0d800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080b4230 0x7f4b0814a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0d800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b08146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b081e6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0d800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080b3e60 con 0x7f4b0815c670 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0d800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080cd390 con 0x7f4b080b4230 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0d800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4b080b3fe0 con 0x7f4b080d05c0 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b081e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b081e6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50090/0 (socket says 172.21.15.59:50090) 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b0ce00640 1 -- 172.21.15.59:0/3490242510 learned_addr learned my addr 172.21.15.59:0/3490242510 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4b07e00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080b4230 0x7f4b0814a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.641+0000 7f4affe00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b08146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0ce00640 1 -- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 msgr2=0x7f4b08146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0ce00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b08146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0ce00640 1 -- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080b4230 msgr2=0x7f4b0814a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0ce00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080b4230 0x7f4b0814a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0ce00640 1 -- 172.21.15.59:0/3490242510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0814a760 con 0x7f4b0815c670 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b07e00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080b4230 0x7f4b0814a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0ce00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b081e6390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000cf20 tx=0x7f4b0000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b04c00640 1 -- 172.21.15.59:0/3490242510 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0001a070 con 0x7f4b0815c670 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b04c00640 1 -- 172.21.15.59:0/3490242510 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0000d840 con 0x7f4b0815c670 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b04c00640 1 -- 172.21.15.59:0/3490242510 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000dc30 con 0x7f4b0815c670 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b08147470 con 0x7f4b0815c670 2024-04-23T14:32:44.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.642+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b08147d70 con 0x7f4b0815c670 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.644+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0806e550 con 0x7f4b0815c670 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.645+0000 7f4b04c00640 1 -- 172.21.15.59:0/3490242510 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b0001d070 con 0x7f4b0815c670 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.646+0000 7f4b04c00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae406c940 0x7f4ae406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:44.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.646+0000 7f4b04c00640 1 -- 172.21.15.59:0/3490242510 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f4b0008bfe0 con 0x7f4b0815c670 2024-04-23T14:32:44.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.647+0000 7f4b07e00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae406c940 0x7f4ae406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:44.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.647+0000 7f4b07e00640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae406c940 0x7f4ae406ee00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4af8034000 tx=0x7f4af802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.651+0000 7f4b04c00640 1 -- 172.21.15.59:0/3490242510 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b0806e550 con 0x7f4b0815c670 2024-04-23T14:32:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.916+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f4b080ceab0 con 0x7f4b0815c670 2024-04-23T14:32:44.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.920+0000 7f4b04c00640 1 -- 172.21.15.59:0/3490242510 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f4b0814a760 con 0x7f4b0815c670 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae406c940 msgr2=0x7f4ae406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae406c940 0x7f4ae406ee00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4af8034000 tx=0x7f4af802e000 comp rx=0 tx=0).stop 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 msgr2=0x7f4b081e6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b081e6390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000cf20 tx=0x7f4b0000a7d0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 shutdown_connections 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae406c940 0x7f4ae406ee00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b080d05c0 0x7f4b08146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0815c670 0x7f4b081e6390 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.922+0000 7f4b0d800640 1 --2- 172.21.15.59:0/3490242510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080b4230 0x7f4b0814a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.923+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 >> 172.21.15.59:0/3490242510 conn(0x7f4b0806bdf0 msgr2=0x7f4b08072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.923+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 shutdown_connections 2024-04-23T14:32:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:44.923+0000 7f4b0d800640 1 -- 172.21.15.59:0/3490242510 wait complete. 2024-04-23T14:32:44.942 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.146+0000 7f302ae00640 1 Processor -- start 2024-04-23T14:32:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.146+0000 7f302ae00640 1 -- start start 2024-04-23T14:32:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.147+0000 7f302ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f30240ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f302ae00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30240b32c0 0x7f30240d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f302ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30240d1b30 0x7f302409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f302ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f302406dac0 con 0x7f30240d1b30 2024-04-23T14:32:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f302ae00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f30240d0b10 con 0x7f30240b32c0 2024-04-23T14:32:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f302ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30240d0df0 con 0x7f30240b2d30 2024-04-23T14:32:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f3028e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30240d1b30 0x7f302409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f3028e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30240d1b30 0x7f302409aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50100/0 (socket says 172.21.15.59:50100) 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f3028e00640 1 -- 172.21.15.59:0/450643112 learned_addr learned my addr 172.21.15.59:0/450643112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f3023400640 1 --2- 172.21.15.59:0/450643112 >> v2:172.21.15.76:3300/0 conn(0x7f30240b32c0 0x7f30240d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f3028e00640 1 -- 172.21.15.59:0/450643112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 msgr2=0x7f30240ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f3023e00640 1 --2- 172.21.15.59:0/450643112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f30240ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.148+0000 7f3028e00640 1 --2- 172.21.15.59:0/450643112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f30240ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.149+0000 7f3028e00640 1 -- 172.21.15.59:0/450643112 >> v2:172.21.15.76:3300/0 conn(0x7f30240b32c0 msgr2=0x7f30240d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.149+0000 7f3028e00640 1 --2- 172.21.15.59:0/450643112 >> v2:172.21.15.76:3300/0 conn(0x7f30240b32c0 0x7f30240d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.149+0000 7f3028e00640 1 -- 172.21.15.59:0/450643112 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240d10d0 con 0x7f30240d1b30 2024-04-23T14:32:45.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.149+0000 7f3023e00640 1 --2- 172.21.15.59:0/450643112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f30240ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.150+0000 7f3028e00640 1 --2- 172.21.15.59:0/450643112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302409aa00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3024073690 tx=0x7f301c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e060283810df5a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.150+0000 7f3022a00640 1 -- 172.21.15.59:0/450643112 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c036070 con 0x7f30240d1b30 2024-04-23T14:32:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.150+0000 7f3022a00640 1 -- 172.21.15.59:0/450643112 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c031600 con 0x7f30240d1b30 2024-04-23T14:32:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.150+0000 7f3022a00640 1 -- 172.21.15.59:0/450643112 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0319d0 con 0x7f30240d1b30 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 -- 172.21.15.59:0/450643112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 msgr2=0x7f302409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 --2- 172.21.15.59:0/450643112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302409aa00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3024073690 tx=0x7f301c02a6a0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 -- 172.21.15.59:0/450643112 shutdown_connections 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 --2- 172.21.15.59:0/450643112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302409aa00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 --2- 172.21.15.59:0/450643112 >> v2:172.21.15.76:3300/0 conn(0x7f30240b32c0 0x7f30240d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 --2- 172.21.15.59:0/450643112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f30240ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 -- 172.21.15.59:0/450643112 >> 172.21.15.59:0/450643112 conn(0x7f30240b4120 msgr2=0x7f30240b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 -- 172.21.15.59:0/450643112 shutdown_connections 2024-04-23T14:32:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.151+0000 7f302ae00640 1 -- 172.21.15.59:0/450643112 wait complete. 2024-04-23T14:32:45.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.153+0000 7f302ae00640 1 Processor -- start 2024-04-23T14:32:45.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.153+0000 7f302ae00640 1 -- start start 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f302ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f302409f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f302ae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30240b32c0 0x7f3024161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f302ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f302ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f302406d880 con 0x7f30240d1b30 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f302ae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f302406d700 con 0x7f30240b32c0 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f302ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30240d1fc0 con 0x7f30240b2d30 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f3028e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f3028e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302415fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50108/0 (socket says 172.21.15.59:50108) 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f3028e00640 1 -- 172.21.15.59:0/2800534532 learned_addr learned my addr 172.21.15.59:0/2800534532 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.154+0000 7f3023400640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30240b32c0 0x7f3024161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3023e00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f302409f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3028e00640 1 -- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 msgr2=0x7f302409f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3028e00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f302409f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3028e00640 1 -- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30240b32c0 msgr2=0x7f3024161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3028e00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30240b32c0 0x7f3024161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3028e00640 1 -- 172.21.15.59:0/2800534532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3024160bb0 con 0x7f30240d1b30 2024-04-23T14:32:45.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3023e00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f302409f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:45.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f3028e00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302415fcf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f301c034040 tx=0x7f301c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f301be00640 1 -- 172.21.15.59:0/2800534532 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c041020 con 0x7f30240d1b30 2024-04-23T14:32:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f301be00640 1 -- 172.21.15.59:0/2800534532 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c0077a0 con 0x7f30240d1b30 2024-04-23T14:32:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f301be00640 1 -- 172.21.15.59:0/2800534532 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c036040 con 0x7f30240d1b30 2024-04-23T14:32:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.155+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3024160e90 con 0x7f30240d1b30 2024-04-23T14:32:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.156+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3024161450 con 0x7f30240d1b30 2024-04-23T14:32:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.157+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3024069190 con 0x7f30240d1b30 2024-04-23T14:32:45.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.158+0000 7f301be00640 1 -- 172.21.15.59:0/2800534532 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f301c044020 con 0x7f30240d1b30 2024-04-23T14:32:45.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.159+0000 7f301be00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f300406c8f0 0x7f300406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.159+0000 7f301be00640 1 -- 172.21.15.59:0/2800534532 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f301c0ac520 con 0x7f30240d1b30 2024-04-23T14:32:45.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.160+0000 7f3023e00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f300406c8f0 0x7f300406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.160+0000 7f3023e00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f300406c8f0 0x7f300406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30140088b0 tx=0x7f3014008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.164+0000 7f301be00640 1 -- 172.21.15.59:0/2800534532 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f301c04b030 con 0x7f30240d1b30 2024-04-23T14:32:45.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.370+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f302406f880 con 0x7f30240d1b30 2024-04-23T14:32:45.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.371+0000 7f301be00640 1 -- 172.21.15.59:0/2800534532 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12831 (secure 0 0 0) 0x7f301c07ac00 con 0x7f30240d1b30 2024-04-23T14:32:45.373 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:45.373 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:41.248257+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f300406c8f0 msgr2=0x7f300406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f300406c8f0 0x7f300406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30140088b0 tx=0x7f3014008040 comp rx=0 tx=0).stop 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 msgr2=0x7f302415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302415fcf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f301c034040 tx=0x7f301c004a40 comp rx=0 tx=0).stop 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 shutdown_connections 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f300406c8f0 0x7f300406edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30240d1b30 0x7f302415fcf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30240b32c0 0x7f3024161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.374+0000 7f302ae00640 1 --2- 172.21.15.59:0/2800534532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30240b2d30 0x7f302409f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.375+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 >> 172.21.15.59:0/2800534532 conn(0x7f30240b4120 msgr2=0x7f30240af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.375+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 shutdown_connections 2024-04-23T14:32:45.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.375+0000 7f302ae00640 1 -- 172.21.15.59:0/2800534532 wait complete. 2024-04-23T14:32:45.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:32:45.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:32:45.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.599+0000 7fc529600640 1 Processor -- start 2024-04-23T14:32:45.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.599+0000 7fc529600640 1 -- start start 2024-04-23T14:32:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc529600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc52415e870 0x7fc5240cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc529600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d16b0 0x7fc5240b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc529600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5240d23f0 con 0x7fc5240d02a0 2024-04-23T14:32:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc529600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc5240d21a0 con 0x7fc52415e870 2024-04-23T14:32:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc529600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc5240d3000 con 0x7fc5240d16b0 2024-04-23T14:32:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc523e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc52415e870 0x7fc5240cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc523400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc5240d02a0 0x7fc5240d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49960/0 (socket says 172.21.15.59:49960) 2024-04-23T14:32:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.601+0000 7fc523400640 1 -- 172.21.15.59:0/2965352245 learned_addr learned my addr 172.21.15.59:0/2965352245 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.602+0000 7fc528c00640 1 --2- 172.21.15.59:0/2965352245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d16b0 0x7fc5240b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.602+0000 7fc523e00640 1 -- 172.21.15.59:0/2965352245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d16b0 msgr2=0x7fc5240b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.602+0000 7fc523e00640 1 --2- 172.21.15.59:0/2965352245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d16b0 0x7fc5240b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.602+0000 7fc523e00640 1 -- 172.21.15.59:0/2965352245 >> v1:172.21.15.59:6789/0 conn(0x7fc5240d02a0 legacy=0x7fc5240d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.602+0000 7fc523e00640 1 -- 172.21.15.59:0/2965352245 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5240d26d0 con 0x7fc52415e870 2024-04-23T14:32:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.602+0000 7fc528c00640 1 --2- 172.21.15.59:0/2965352245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d16b0 0x7fc5240b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.603+0000 7fc523e00640 1 --2- 172.21.15.59:0/2965352245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc52415e870 0x7fc5240cfd60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc5140097a0 tx=0x7fc51402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b53422220ea3a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.603+0000 7fc522a00640 1 -- 172.21.15.59:0/2965352245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514037070 con 0x7fc52415e870 2024-04-23T14:32:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.603+0000 7fc522a00640 1 -- 172.21.15.59:0/2965352245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514032440 con 0x7fc52415e870 2024-04-23T14:32:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.603+0000 7fc522a00640 1 -- 172.21.15.59:0/2965352245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514032810 con 0x7fc52415e870 2024-04-23T14:32:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 -- 172.21.15.59:0/2965352245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc52415e870 msgr2=0x7fc5240cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 --2- 172.21.15.59:0/2965352245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc52415e870 0x7fc5240cfd60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc5140097a0 tx=0x7fc51402a180 comp rx=0 tx=0).stop 2024-04-23T14:32:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 -- 172.21.15.59:0/2965352245 shutdown_connections 2024-04-23T14:32:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 --2- 172.21.15.59:0/2965352245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d16b0 0x7fc5240b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 --2- 172.21.15.59:0/2965352245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc52415e870 0x7fc5240cfd60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 -- 172.21.15.59:0/2965352245 >> 172.21.15.59:0/2965352245 conn(0x7fc52408bf90 msgr2=0x7fc5240d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 -- 172.21.15.59:0/2965352245 shutdown_connections 2024-04-23T14:32:45.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.604+0000 7fc529600640 1 -- 172.21.15.59:0/2965352245 wait complete. 2024-04-23T14:32:45.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.606+0000 7fc529600640 1 Processor -- start 2024-04-23T14:32:45.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.606+0000 7fc529600640 1 -- start start 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc529600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d02a0 0x7fc52415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc529600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5240d16b0 0x7fc5240b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc529600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc52415e870 0x7fc5240c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc529600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc52415ec50 con 0x7fc5240d16b0 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc529600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc5240d2c90 con 0x7fc52415e870 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc529600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc5240d3490 con 0x7fc5240d02a0 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc523400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5240d16b0 0x7fc5240b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc523400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5240d16b0 0x7fc5240b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50122/0 (socket says 172.21.15.59:50122) 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.607+0000 7fc523400640 1 -- 172.21.15.59:0/1036308661 learned_addr learned my addr 172.21.15.59:0/1036308661 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc523e00640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d02a0 0x7fc52415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc528c00640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc52415e870 0x7fc5240c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc523400640 1 -- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d02a0 msgr2=0x7fc52415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc523400640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d02a0 0x7fc52415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc523400640 1 -- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc52415e870 msgr2=0x7fc5240c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc523400640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc52415e870 0x7fc5240c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc523400640 1 -- 172.21.15.59:0/1036308661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52415c2b0 con 0x7fc5240d16b0 2024-04-23T14:32:45.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc523400640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5240d16b0 0x7fc5240b82d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc51000c430 tx=0x7fc51000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.608+0000 7fc51be00640 1 -- 172.21.15.59:0/1036308661 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510017070 con 0x7fc5240d16b0 2024-04-23T14:32:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.609+0000 7fc51be00640 1 -- 172.21.15.59:0/1036308661 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc510005ce0 con 0x7fc5240d16b0 2024-04-23T14:32:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.609+0000 7fc51be00640 1 -- 172.21.15.59:0/1036308661 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5100072f0 con 0x7fc5240d16b0 2024-04-23T14:32:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.609+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5240cc420 con 0x7fc5240d16b0 2024-04-23T14:32:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.609+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5240cc9e0 con 0x7fc5240d16b0 2024-04-23T14:32:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.610+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52404fd90 con 0x7fc5240d16b0 2024-04-23T14:32:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.611+0000 7fc51be00640 1 -- 172.21.15.59:0/1036308661 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc51001a070 con 0x7fc5240d16b0 2024-04-23T14:32:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.613+0000 7fc51be00640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc50406c920 0x7fc50406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.613+0000 7fc51be00640 1 -- 172.21.15.59:0/1036308661 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fc51008ab90 con 0x7fc5240d16b0 2024-04-23T14:32:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.613+0000 7fc523e00640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc50406c920 0x7fc50406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.614+0000 7fc523e00640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc50406c920 0x7fc50406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc514034000 tx=0x7fc51402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.617+0000 7fc51be00640 1 -- 172.21.15.59:0/1036308661 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5100593a0 con 0x7fc5240d16b0 2024-04-23T14:32:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.818+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5240d1c80 con 0x7fc5240d16b0 2024-04-23T14:32:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.819+0000 7fc51be00640 1 -- 172.21.15.59:0/1036308661 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fc510058ca0 con 0x7fc5240d16b0 2024-04-23T14:32:45.820 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.821+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc50406c920 msgr2=0x7fc50406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.821+0000 7fc529600640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc50406c920 0x7fc50406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc514034000 tx=0x7fc51402e000 comp rx=0 tx=0).stop 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.821+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5240d16b0 msgr2=0x7fc5240b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.821+0000 7fc529600640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5240d16b0 0x7fc5240b82d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc51000c430 tx=0x7fc51000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 shutdown_connections 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc50406c920 0x7fc50406ede0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc52415e870 0x7fc5240c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5240d16b0 0x7fc5240b82d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 --2- 172.21.15.59:0/1036308661 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5240d02a0 0x7fc52415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 >> 172.21.15.59:0/1036308661 conn(0x7fc52408bf90 msgr2=0x7fc52415f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 shutdown_connections 2024-04-23T14:32:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:45.822+0000 7fc529600640 1 -- 172.21.15.59:0/1036308661 wait complete. 2024-04-23T14:32:45.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:32:46.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.044+0000 7f1109a00640 1 Processor -- start 2024-04-23T14:32:46.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.045+0000 7f1109a00640 1 -- start start 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.046+0000 7f1109a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f110415e7c0 0x7f11040d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.046+0000 7f1109a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040d2630 0x7f11040b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.046+0000 7f1109a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11040cf030 con 0x7f11040b6530 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.046+0000 7f1109a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f11040b7000 con 0x7f110415e7c0 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.046+0000 7f1109a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f11040cf620 con 0x7f11040d2630 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f110415e7c0 0x7f11040d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f110415e7c0 0x7f11040d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50136/0 (socket says 172.21.15.59:50136) 2024-04-23T14:32:46.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103e00640 1 -- 172.21.15.59:0/4051345658 learned_addr learned my addr 172.21.15.59:0/4051345658 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:46.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103400640 1 --2- 172.21.15.59:0/4051345658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040d2630 0x7f11040b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103e00640 1 -- 172.21.15.59:0/4051345658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040d2630 msgr2=0x7f11040b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103e00640 1 --2- 172.21.15.59:0/4051345658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040d2630 0x7f11040b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103e00640 1 -- 172.21.15.59:0/4051345658 >> v1:172.21.15.76:6789/0 conn(0x7f11040b6530 legacy=0x7f11040d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.047+0000 7f1103e00640 1 -- 172.21.15.59:0/4051345658 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11040b6a40 con 0x7f110415e7c0 2024-04-23T14:32:46.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.049+0000 7f1103e00640 1 --2- 172.21.15.59:0/4051345658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040d20f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009e50 tx=0x7f10f402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7944621f0da0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.049+0000 7f1102a00640 1 -- 172.21.15.59:0/4051345658 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4036070 con 0x7f110415e7c0 2024-04-23T14:32:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.049+0000 7f1102a00640 1 -- 172.21.15.59:0/4051345658 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f4031600 con 0x7f110415e7c0 2024-04-23T14:32:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.049+0000 7f1102a00640 1 -- 172.21.15.59:0/4051345658 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40319d0 con 0x7f110415e7c0 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 -- 172.21.15.59:0/4051345658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 msgr2=0x7f11040d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 --2- 172.21.15.59:0/4051345658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040d20f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009e50 tx=0x7f10f402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 -- 172.21.15.59:0/4051345658 shutdown_connections 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 --2- 172.21.15.59:0/4051345658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040d2630 0x7f11040b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 --2- 172.21.15.59:0/4051345658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040d20f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 -- 172.21.15.59:0/4051345658 >> 172.21.15.59:0/4051345658 conn(0x7f110409d920 msgr2=0x7f110408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 -- 172.21.15.59:0/4051345658 shutdown_connections 2024-04-23T14:32:46.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.050+0000 7f1109a00640 1 -- 172.21.15.59:0/4051345658 wait complete. 2024-04-23T14:32:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.052+0000 7f1109a00640 1 Processor -- start 2024-04-23T14:32:46.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.052+0000 7f1109a00640 1 -- start start 2024-04-23T14:32:46.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040b6530 0x7f11040cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11040d2630 0x7f11040b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11040d1a80 con 0x7f110415e7c0 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11040cf2b0 con 0x7f11040d2630 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f11040d1c00 con 0x7f11040b6530 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50138/0 (socket says 172.21.15.59:50138) 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 -- 172.21.15.59:0/893665053 learned_addr learned my addr 172.21.15.59:0/893665053 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1103e00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040b6530 0x7f11040cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1103400640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11040d2630 0x7f11040b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 -- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040b6530 msgr2=0x7f11040cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040b6530 0x7f11040cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 -- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11040d2630 msgr2=0x7f11040b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11040d2630 0x7f11040b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1109000640 1 -- 172.21.15.59:0/893665053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11040b1c20 con 0x7f110415e7c0 2024-04-23T14:32:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.054+0000 7f1103400640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11040d2630 0x7f11040b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.055+0000 7f1109000640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040b2cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c3e0 tx=0x7f10fc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.055+0000 7f10fbe00640 1 -- 172.21.15.59:0/893665053 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc011790 con 0x7f110415e7c0 2024-04-23T14:32:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.055+0000 7f10fbe00640 1 -- 172.21.15.59:0/893665053 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10fc017070 con 0x7f110415e7c0 2024-04-23T14:32:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.055+0000 7f10fbe00640 1 -- 172.21.15.59:0/893665053 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc011bb0 con 0x7f110415e7c0 2024-04-23T14:32:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.055+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11040b1f00 con 0x7f110415e7c0 2024-04-23T14:32:46.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.055+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11040b24c0 con 0x7f110415e7c0 2024-04-23T14:32:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.056+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110404fd90 con 0x7f110415e7c0 2024-04-23T14:32:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.057+0000 7f10fbe00640 1 -- 172.21.15.59:0/893665053 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10fc01c460 con 0x7f110415e7c0 2024-04-23T14:32:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.058+0000 7f10fbe00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10e406c920 0x7f10e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.058+0000 7f10fbe00640 1 -- 172.21.15.59:0/893665053 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f10fc08b6c0 con 0x7f110415e7c0 2024-04-23T14:32:46.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.058+0000 7f1103e00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10e406c920 0x7f10e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.059+0000 7f1103e00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10e406c920 0x7f10e406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f10f4033000 tx=0x7f10f4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.063+0000 7f10fbe00640 1 -- 172.21.15.59:0/893665053 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10fc059d10 con 0x7f110415e7c0 2024-04-23T14:32:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.269+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f11040d1eb0 con 0x7f110415e7c0 2024-04-23T14:32:46.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.270+0000 7f10fbe00640 1 -- 172.21.15.59:0/893665053 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f10fc059610 con 0x7f110415e7c0 2024-04-23T14:32:46.275 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:46.275 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:32:46.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:32:46.275 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:32:46.275 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:32:46.275 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:32:46.275 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:32:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10e406c920 msgr2=0x7f10e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10e406c920 0x7f10e406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f10f4033000 tx=0x7f10f4009000 comp rx=0 tx=0).stop 2024-04-23T14:32:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 msgr2=0x7f11040b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040b2cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c3e0 tx=0x7f10fc00c8a0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 shutdown_connections 2024-04-23T14:32:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10e406c920 0x7f10e406ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f110415e7c0 0x7f11040b2cc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11040d2630 0x7f11040b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 --2- 172.21.15.59:0/893665053 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11040b6530 0x7f11040cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.273+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 >> 172.21.15.59:0/893665053 conn(0x7f110409d920 msgr2=0x7f110415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.274+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 shutdown_connections 2024-04-23T14:32:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.274+0000 7f1109a00640 1 -- 172.21.15.59:0/893665053 wait complete. 2024-04-23T14:32:46.293 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:32:46.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:32:46.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.514+0000 7f65f9200640 1 Processor -- start 2024-04-23T14:32:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.515+0000 7f65f9200640 1 -- start start 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.516+0000 7f65f9200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 0x7f65f4071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f9200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f40724c0 con 0x7f65f406c500 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f9200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f406dce0 con 0x7f65f41673c0 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f9200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f65f406dfc0 con 0x7f65f4162a90 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f65f406c500 0x7f65f406e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49966/0 (socket says 172.21.15.59:49966) 2024-04-23T14:32:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f3e00640 1 -- 172.21.15.59:0/2941777942 learned_addr learned my addr 172.21.15.59:0/2941777942 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f3400640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 0x7f65f4071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123573022 0 0) 0x7f65f40724c0 con 0x7f65f406c500 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.517+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65d4003520 con 0x7f65f406c500 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2815396599 0 0) 0x7f65d4003520 con 0x7f65f406c500 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 msgr2=0x7f65f4071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 0x7f65f4071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 >> v1:172.21.15.76:6789/0 conn(0x7f65f41673c0 legacy=0x7f65f406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f406e2a0 con 0x7f65f406c500 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f65ec001670 con 0x7f65f406c500 2024-04-23T14:32:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 >> v1:172.21.15.59:6789/0 conn(0x7f65f406c500 legacy=0x7f65f406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 0x7f65d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65d4003f60 0x7f65d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65f406dfc0 con 0x7f65d4004d40 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65d400baa0 con 0x7f65d4003f60 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f65d400bca0 con 0x7f65f4162a90 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.518+0000 7f65f3e00640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f3400640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 0x7f65d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f2a00640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65d4003f60 0x7f65d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f3e00640 1 -- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 msgr2=0x7f65d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f3e00640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 0x7f65d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f3e00640 1 -- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65d4003f60 msgr2=0x7f65d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f3e00640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65d4003f60 0x7f65d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f3e00640 1 -- 172.21.15.59:0/2941777942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d4003520 con 0x7f65d4004d40 2024-04-23T14:32:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f3400640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f4162a90 0x7f65d40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.519+0000 7f65f2a00640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65d4003f60 0x7f65d4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.520+0000 7f65f3e00640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65d40051c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00c240 tx=0x7f65d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ba4ae4042e1703 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.520+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec007cf0 con 0x7f65d4004d40 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.520+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec0058d0 con 0x7f65d4004d40 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.520+0000 7f65f2000640 1 -- 172.21.15.59:0/2941777942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec005d60 con 0x7f65d4004d40 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.521+0000 7f65f9200640 1 -- 172.21.15.59:0/2941777942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 msgr2=0x7f65d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.521+0000 7f65f9200640 1 --2- 172.21.15.59:0/2941777942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65d40051c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00c240 tx=0x7f65d4005830 comp rx=0 tx=0).stop 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.521+0000 7f65f3e00640 1 -- 172.21.15.59:0/2941777942 reap_dead start 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.521+0000 7f65f9200640 1 -- 172.21.15.59:0/2941777942 shutdown_connections 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.521+0000 7f65f9200640 1 -- 172.21.15.59:0/2941777942 >> 172.21.15.59:0/2941777942 conn(0x7f65f40b2200 msgr2=0x7f65f40b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.521+0000 7f65f9200640 1 -- 172.21.15.59:0/2941777942 shutdown_connections 2024-04-23T14:32:46.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.521+0000 7f65f9200640 1 -- 172.21.15.59:0/2941777942 wait complete. 2024-04-23T14:32:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.523+0000 7f65f9200640 1 Processor -- start 2024-04-23T14:32:46.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.523+0000 7f65f9200640 1 -- start start 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65f41fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f9200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65f4161980 0x7f65f415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f9200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f415ec10 0x7f65f409c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4162ff0 con 0x7f65d4004d40 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f9200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4162e70 con 0x7f65f4161980 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f9200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f65f4071be0 con 0x7f65f415ec10 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65f41fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65f41fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50150/0 (socket says 172.21.15.59:50150) 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.524+0000 7f65f3400640 1 -- 172.21.15.59:0/241972580 learned_addr learned my addr 172.21.15.59:0/241972580 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f2a00640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65f4161980 0x7f65f415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f3400640 1 -- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f415ec10 msgr2=0x7f65f409c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f3400640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f415ec10 0x7f65f409c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f3400640 1 -- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65f4161980 msgr2=0x7f65f415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f3400640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65f4161980 0x7f65f415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f3400640 1 -- 172.21.15.59:0/241972580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41fd3a0 con 0x7f65d4004d40 2024-04-23T14:32:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f2a00640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65f4161980 0x7f65f415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:46.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f3400640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65f41fce60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f65e40108b0 tx=0x7f65e4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65eb400640 1 -- 172.21.15.59:0/241972580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400f040 con 0x7f65d4004d40 2024-04-23T14:32:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65eb400640 1 -- 172.21.15.59:0/241972580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e400ea80 con 0x7f65d4004d40 2024-04-23T14:32:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f415d8a0 con 0x7f65d4004d40 2024-04-23T14:32:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.525+0000 7f65eb400640 1 -- 172.21.15.59:0/241972580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400ee30 con 0x7f65d4004d40 2024-04-23T14:32:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.526+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f409b8d0 con 0x7f65d4004d40 2024-04-23T14:32:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.527+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f404fd90 con 0x7f65d4004d40 2024-04-23T14:32:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.528+0000 7f65eb400640 1 -- 172.21.15.59:0/241972580 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65e401c400 con 0x7f65d4004d40 2024-04-23T14:32:46.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.529+0000 7f65eb400640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65d006c970 0x7f65d006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.529+0000 7f65eb400640 1 -- 172.21.15.59:0/241972580 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f65e408be30 con 0x7f65d4004d40 2024-04-23T14:32:46.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.529+0000 7f65f2a00640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65d006c970 0x7f65d006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.530+0000 7f65f2a00640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65d006c970 0x7f65d006ee30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f65f40cfa50 tx=0x7f65dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:46.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.542+0000 7f65eb400640 1 -- 172.21.15.59:0/241972580 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65f41fd3a0 con 0x7f65d4004d40 2024-04-23T14:32:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:46.745+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f65f40b5330 con 0x7f65d4004d40 2024-04-23T14:32:47.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.337+0000 7f65eb400640 1 -- 172.21.15.59:0/241972580 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f65e4018030 con 0x7f65d4004d40 2024-04-23T14:32:47.338 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:32:47.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65d006c970 msgr2=0x7f65d006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65d006c970 0x7f65d006ee30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f65f40cfa50 tx=0x7f65dc009040 comp rx=0 tx=0).stop 2024-04-23T14:32:47.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 msgr2=0x7f65f41fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65f41fce60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f65e40108b0 tx=0x7f65e4010d70 comp rx=0 tx=0).stop 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 shutdown_connections 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65d006c970 0x7f65d006ee30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65f415ec10 0x7f65f409c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65f4161980 0x7f65f415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 --2- 172.21.15.59:0/241972580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65d4004d40 0x7f65f41fce60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.341+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 >> 172.21.15.59:0/241972580 conn(0x7f65f40b2200 msgr2=0x7f65f40afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.342+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 shutdown_connections 2024-04-23T14:32:47.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.342+0000 7f65f9200640 1 -- 172.21.15.59:0/241972580 wait complete. 2024-04-23T14:32:47.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:32:47.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.576+0000 7f2226600640 1 Processor -- start 2024-04-23T14:32:47.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.576+0000 7f2226600640 1 -- start start 2024-04-23T14:32:47.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.577+0000 7f2226600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 0x7f22200c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.577+0000 7f2226600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222009baa0 con 0x7f222015ca40 2024-04-23T14:32:47.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.577+0000 7f2226600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22200d02a0 con 0x7f22200d1340 2024-04-23T14:32:47.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.577+0000 7f2226600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f222015ce20 con 0x7f222009b110 2024-04-23T14:32:47.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.578+0000 7f221f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f222015ca40 0x7f22200cdd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49968/0 (socket says 172.21.15.59:49968) 2024-04-23T14:32:47.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.578+0000 7f221f400640 1 -- 172.21.15.59:0/1237469805 learned_addr learned my addr 172.21.15.59:0/1237469805 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.578+0000 7f221fe00640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 0x7f22200c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.578+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800054055 0 0) 0x7f222009baa0 con 0x7f222015ca40 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.578+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2204003520 con 0x7f222015ca40 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 993401551 0 0) 0x7f2204003520 con 0x7f222015ca40 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 msgr2=0x7f22200c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 0x7f22200c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 >> v1:172.21.15.76:6789/0 conn(0x7f22200d1340 legacy=0x7f222009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22200d0580 con 0x7f222015ca40 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2210001670 con 0x7f222015ca40 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 >> v1:172.21.15.59:6789/0 conn(0x7f222015ca40 legacy=0x7f22200cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22040039e0 0x7f2204003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2204004800 0x7f2204004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 0x7f220400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f222015ce20 con 0x7f22040039e0 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2204005550 con 0x7f2204004800 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f220400bf00 con 0x7f222009b110 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.579+0000 7f221f400640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22040039e0 0x7f2204003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.580+0000 7f221ea00640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2204004800 0x7f2204004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.580+0000 7f221fe00640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 0x7f220400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.580+0000 7f221f400640 1 -- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 msgr2=0x7f220400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:47.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.580+0000 7f221f400640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f222009b110 0x7f220400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.580+0000 7f221f400640 1 -- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2204004800 msgr2=0x7f2204004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:47.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.580+0000 7f221f400640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2204004800 0x7f2204004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.580+0000 7f221f400640 1 -- 172.21.15.59:0/1237469805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2204003520 con 0x7f22040039e0 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.581+0000 7f221f400640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22040039e0 0x7f2204003e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f221000c210 tx=0x7f2204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc86808a92c1466 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.581+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210007d30 con 0x7f22040039e0 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.581+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2210005ad0 con 0x7f22040039e0 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.581+0000 7f221e000640 1 -- 172.21.15.59:0/1237469805 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210007070 con 0x7f22040039e0 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.582+0000 7f2226600640 1 -- 172.21.15.59:0/1237469805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22040039e0 msgr2=0x7f2204003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.582+0000 7f2226600640 1 --2- 172.21.15.59:0/1237469805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22040039e0 0x7f2204003e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f221000c210 tx=0x7f2204005830 comp rx=0 tx=0).stop 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.582+0000 7f221fe00640 1 -- 172.21.15.59:0/1237469805 reap_dead start 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.582+0000 7f2226600640 1 -- 172.21.15.59:0/1237469805 shutdown_connections 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.582+0000 7f2226600640 1 -- 172.21.15.59:0/1237469805 >> 172.21.15.59:0/1237469805 conn(0x7f22200b4120 msgr2=0x7f22200b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.582+0000 7f2226600640 1 -- 172.21.15.59:0/1237469805 shutdown_connections 2024-04-23T14:32:47.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.582+0000 7f2226600640 1 -- 172.21.15.59:0/1237469805 wait complete. 2024-04-23T14:32:47.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.584+0000 7f2226600640 1 Processor -- start 2024-04-23T14:32:47.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.584+0000 7f2226600640 1 -- start start 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f2226600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22040039e0 0x7f22201f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f2226600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22200b5780 0x7f222015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f2226600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22200b5be0 0x7f22200cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f2226600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22200d0b40 con 0x7f22200b5780 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f2226600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22200d0eb0 con 0x7f22040039e0 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f2226600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22200ce460 con 0x7f22200b5be0 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f221ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22200b5780 0x7f222015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f221ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22200b5780 0x7f222015dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50166/0 (socket says 172.21.15.59:50166) 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.585+0000 7f221ea00640 1 -- 172.21.15.59:0/4108570109 learned_addr learned my addr 172.21.15.59:0/4108570109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221f400640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22040039e0 0x7f22201f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221fe00640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22200b5be0 0x7f22200cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221ea00640 1 -- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22200b5be0 msgr2=0x7f22200cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221ea00640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22200b5be0 0x7f22200cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221ea00640 1 -- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22040039e0 msgr2=0x7f22201f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:47.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221ea00640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22040039e0 0x7f22201f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:47.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221ea00640 1 -- 172.21.15.59:0/4108570109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222015ea00 con 0x7f22200b5780 2024-04-23T14:32:47.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221f400640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22040039e0 0x7f22201f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:47.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f221ea00640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22200b5780 0x7f222015dff0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f220c00e550 tx=0x7f220c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:47.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f2217400640 1 -- 172.21.15.59:0/4108570109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c0052e0 con 0x7f22200b5780 2024-04-23T14:32:47.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f2217400640 1 -- 172.21.15.59:0/4108570109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220c018070 con 0x7f22200b5780 2024-04-23T14:32:47.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222015ece0 con 0x7f22200b5780 2024-04-23T14:32:47.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.586+0000 7f2217400640 1 -- 172.21.15.59:0/4108570109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c003810 con 0x7f22200b5780 2024-04-23T14:32:47.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.587+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201f85b0 con 0x7f22200b5780 2024-04-23T14:32:47.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.588+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22200cff30 con 0x7f22200b5780 2024-04-23T14:32:47.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.589+0000 7f2217400640 1 -- 172.21.15.59:0/4108570109 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f220c0039b0 con 0x7f22200b5780 2024-04-23T14:32:47.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.590+0000 7f2217400640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220006c8f0 0x7f220006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:47.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.591+0000 7f2217400640 1 -- 172.21.15.59:0/4108570109 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f220c08b690 con 0x7f22200b5780 2024-04-23T14:32:47.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.591+0000 7f221f400640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220006c8f0 0x7f220006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:47.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.592+0000 7f221f400640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220006c8f0 0x7f220006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2210033000 tx=0x7f221002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:47.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.596+0000 7f2217400640 1 -- 172.21.15.59:0/4108570109 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22200cff30 con 0x7f22200b5780 2024-04-23T14:32:47.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:47.792+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2220069190 con 0x7f22200b5780 2024-04-23T14:32:48.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.343+0000 7f2217400640 1 -- 172.21.15.59:0/4108570109 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f220c059c00 con 0x7f22200b5780 2024-04-23T14:32:48.345 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:32:48.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220006c8f0 msgr2=0x7f220006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220006c8f0 0x7f220006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2210033000 tx=0x7f221002e000 comp rx=0 tx=0).stop 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22200b5780 msgr2=0x7f222015dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22200b5780 0x7f222015dff0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f220c00e550 tx=0x7f220c00ea10 comp rx=0 tx=0).stop 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 shutdown_connections 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f220006c8f0 0x7f220006edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22200b5be0 0x7f22200cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22200b5780 0x7f222015dff0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 --2- 172.21.15.59:0/4108570109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22040039e0 0x7f22201f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 >> 172.21.15.59:0/4108570109 conn(0x7f22200b4120 msgr2=0x7f2220070080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.347+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 shutdown_connections 2024-04-23T14:32:48.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.348+0000 7f2226600640 1 -- 172.21.15.59:0/4108570109 wait complete. 2024-04-23T14:32:48.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:32:48.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.566+0000 7f0041c00640 1 Processor -- start 2024-04-23T14:32:48.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.566+0000 7f0041c00640 1 -- start start 2024-04-23T14:32:48.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.568+0000 7f0041c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f003c15d810 0x7f003c160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.568+0000 7f0041c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003c15cab0 con 0x7f003c0d0d10 2024-04-23T14:32:48.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.568+0000 7f0041c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003c160e40 con 0x7f003c1567e0 2024-04-23T14:32:48.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.568+0000 7f0041c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f003c15cd30 con 0x7f003c15d810 2024-04-23T14:32:48.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.568+0000 7f0033400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f003c0d0d10 0x7f003c160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49970/0 (socket says 172.21.15.59:49970) 2024-04-23T14:32:48.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.568+0000 7f0033400640 1 -- 172.21.15.59:0/2039314210 learned_addr learned my addr 172.21.15.59:0/2039314210 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013854480 0 0) 0x7f0024001110 con 0x7f003c0d0d10 2024-04-23T14:32:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0020003520 con 0x7f003c0d0d10 2024-04-23T14:32:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f0041200640 1 --2- 172.21.15.59:0/2039314210 >> v2:172.21.15.76:3300/0 conn(0x7f003c15d810 0x7f003c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3048601491 0 0) 0x7f0020003520 con 0x7f003c0d0d10 2024-04-23T14:32:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 >> v2:172.21.15.76:3300/0 conn(0x7f003c15d810 msgr2=0x7f003c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 --2- 172.21.15.59:0/2039314210 >> v2:172.21.15.76:3300/0 conn(0x7f003c15d810 0x7f003c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 >> v1:172.21.15.76:6789/0 conn(0x7f003c1567e0 legacy=0x7f003c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c15d010 con 0x7f003c0d0d10 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f0024001840 con 0x7f003c0d0d10 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 >> v1:172.21.15.59:6789/0 conn(0x7f003c0d0d10 legacy=0x7f003c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00200039b0 0x7f0020003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0020004800 0x7f0020004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200051c0 0x7f002000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f003c15cd30 con 0x7f00200051c0 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f002000c110 con 0x7f00200039b0 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f002000c310 con 0x7f0020004800 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.569+0000 7f0041200640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200051c0 0x7f002000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f003be00640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00200039b0 0x7f0020003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f0033400640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0020004800 0x7f0020004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f0041200640 1 -- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0020004800 msgr2=0x7f0020004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f0041200640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0020004800 0x7f0020004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f0041200640 1 -- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00200039b0 msgr2=0x7f0020003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f0041200640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00200039b0 0x7f0020003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f0041200640 1 -- 172.21.15.59:0/2039314210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0020003520 con 0x7f00200051c0 2024-04-23T14:32:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.570+0000 7f0033400640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0020004800 0x7f0020004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.571+0000 7f0041200640 1 -- 172.21.15.59:0/2039314210 reap_dead start 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.571+0000 7f0041200640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200051c0 0x7f002000b9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f003400ba00 tx=0x7f0020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86981fa4c058c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.571+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034030070 con 0x7f00200051c0 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.571+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003402ade0 con 0x7f00200051c0 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.571+0000 7f003b400640 1 -- 172.21.15.59:0/2039314210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003400b600 con 0x7f00200051c0 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.572+0000 7f0041c00640 1 -- 172.21.15.59:0/2039314210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200051c0 msgr2=0x7f002000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.572+0000 7f0041c00640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200051c0 0x7f002000b9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f003400ba00 tx=0x7f0020005830 comp rx=0 tx=0).stop 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.572+0000 7f0041c00640 1 -- 172.21.15.59:0/2039314210 shutdown_connections 2024-04-23T14:32:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.572+0000 7f0041c00640 1 --2- 172.21.15.59:0/2039314210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200051c0 0x7f002000b9c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.572+0000 7f0041c00640 1 -- 172.21.15.59:0/2039314210 >> 172.21.15.59:0/2039314210 conn(0x7f003c06bdf0 msgr2=0x7f003c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:48.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.572+0000 7f0041c00640 1 -- 172.21.15.59:0/2039314210 shutdown_connections 2024-04-23T14:32:48.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.572+0000 7f0041c00640 1 -- 172.21.15.59:0/2039314210 wait complete. 2024-04-23T14:32:48.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.574+0000 7f0041c00640 1 Processor -- start 2024-04-23T14:32:48.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.574+0000 7f0041c00640 1 -- start start 2024-04-23T14:32:48.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.575+0000 7f0041c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00200051c0 0x7f003c1f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.575+0000 7f0041c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f003c150570 0x7f003c1509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.575+0000 7f0041c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f003c06e630 0x7f003c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.575+0000 7f0041c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f003c156d40 con 0x7f003c150570 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.575+0000 7f0041c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f003c156bc0 con 0x7f003c06e630 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.575+0000 7f0041c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f003c156ec0 con 0x7f00200051c0 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f003c150570 0x7f003c1509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f003c150570 0x7f003c1509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50176/0 (socket says 172.21.15.59:50176) 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 -- 172.21.15.59:0/35735652 learned_addr learned my addr 172.21.15.59:0/35735652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f003be00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00200051c0 0x7f003c1f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0041200640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f003c06e630 0x7f003c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 -- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00200051c0 msgr2=0x7f003c1f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00200051c0 0x7f003c1f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 -- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f003c06e630 msgr2=0x7f003c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f003c06e630 0x7f003c06eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.576+0000 7f0033400640 1 -- 172.21.15.59:0/35735652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c159080 con 0x7f003c150570 2024-04-23T14:32:48.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.577+0000 7f0033400640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f003c150570 0x7f003c1509f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f002400d7a0 tx=0x7f002400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:48.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.577+0000 7f0038c00640 1 -- 172.21.15.59:0/35735652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024019070 con 0x7f003c150570 2024-04-23T14:32:48.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.577+0000 7f0038c00640 1 -- 172.21.15.59:0/35735652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00240074e0 con 0x7f003c150570 2024-04-23T14:32:48.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.577+0000 7f0038c00640 1 -- 172.21.15.59:0/35735652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00240049e0 con 0x7f003c150570 2024-04-23T14:32:48.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.577+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c159360 con 0x7f003c150570 2024-04-23T14:32:48.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.577+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c159920 con 0x7f003c150570 2024-04-23T14:32:48.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.578+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c04fd90 con 0x7f003c150570 2024-04-23T14:32:48.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.579+0000 7f0038c00640 1 -- 172.21.15.59:0/35735652 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0024004b80 con 0x7f003c150570 2024-04-23T14:32:48.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.581+0000 7f0038c00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f001806c920 0x7f001806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:48.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.581+0000 7f0038c00640 1 -- 172.21.15.59:0/35735652 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f002408b4f0 con 0x7f003c150570 2024-04-23T14:32:48.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.581+0000 7f003be00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f001806c920 0x7f001806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:48.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.582+0000 7f003be00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f001806c920 0x7f001806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f002c007fd0 tx=0x7f002c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:48.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.586+0000 7f0038c00640 1 -- 172.21.15.59:0/35735652 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f003c15cab0 con 0x7f003c150570 2024-04-23T14:32:48.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:48.702 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:32:48.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:48.828+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f003c0b5150 con 0x7f003c150570 2024-04-23T14:32:49.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.388+0000 7f0038c00640 1 -- 172.21.15.59:0/35735652 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f00240598e0 con 0x7f003c150570 2024-04-23T14:32:49.391 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 4 and data pool 5 2024-04-23T14:32:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.399+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f001806c920 msgr2=0x7f001806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.399+0000 7f0041c00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f001806c920 0x7f001806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f002c007fd0 tx=0x7f002c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.399+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f003c150570 msgr2=0x7f003c1509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.399+0000 7f0041c00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f003c150570 0x7f003c1509f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f002400d7a0 tx=0x7f002400dc60 comp rx=0 tx=0).stop 2024-04-23T14:32:49.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.400+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 shutdown_connections 2024-04-23T14:32:49.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.401+0000 7f0041c00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f001806c920 0x7f001806ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.401+0000 7f0041c00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f003c06e630 0x7f003c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.401+0000 7f0041c00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f003c150570 0x7f003c1509f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.401+0000 7f0041c00640 1 --2- 172.21.15.59:0/35735652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00200051c0 0x7f003c1f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.401+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 >> 172.21.15.59:0/35735652 conn(0x7f003c06bdf0 msgr2=0x7f003c0728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:49.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.401+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 shutdown_connections 2024-04-23T14:32:49.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.402+0000 7f0041c00640 1 -- 172.21.15.59:0/35735652 wait complete. 2024-04-23T14:32:49.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:49.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.655+0000 7f166b200640 1 Processor -- start 2024-04-23T14:32:49.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.655+0000 7f166b200640 1 -- start start 2024-04-23T14:32:49.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.657+0000 7f166b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c15e7c0 0x7f166c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:49.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.657+0000 7f166b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c0cf030 con 0x7f166c0d2630 2024-04-23T14:32:49.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.657+0000 7f166b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c0b6a40 con 0x7f166c0d0d90 2024-04-23T14:32:49.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.657+0000 7f166b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f166c0b7000 con 0x7f166c15e7c0 2024-04-23T14:32:49.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.657+0000 7f1661400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f166c0d2630 0x7f166c0b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49982/0 (socket says 172.21.15.59:49982) 2024-04-23T14:32:49.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.657+0000 7f1661400640 1 -- 172.21.15.59:0/1911157251 learned_addr learned my addr 172.21.15.59:0/1911157251 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:49.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.657+0000 7f1669e00640 1 --2- 172.21.15.59:0/1911157251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c15e7c0 0x7f166c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:49.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.658+0000 7f1669400640 1 -- 172.21.15.59:0/1911157251 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420809445 0 0) 0x7f1654001110 con 0x7f166c0d2630 2024-04-23T14:32:49.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.658+0000 7f1669400640 1 -- 172.21.15.59:0/1911157251 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1650003520 con 0x7f166c0d2630 2024-04-23T14:32:49.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.658+0000 7f1669e00640 1 -- 172.21.15.59:0/1911157251 >> v1:172.21.15.76:6789/0 conn(0x7f166c0d0d90 legacy=0x7f166c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.658+0000 7f1669400640 1 -- 172.21.15.59:0/1911157251 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073439029 0 0) 0x7f166c0b6a40 con 0x7f166c0d0d90 2024-04-23T14:32:49.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.658+0000 7f1669e00640 1 -- 172.21.15.59:0/1911157251 >> v1:172.21.15.59:6789/0 conn(0x7f166c0d2630 legacy=0x7f166c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.658+0000 7f1669e00640 1 -- 172.21.15.59:0/1911157251 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0b6d20 con 0x7f166c15e7c0 2024-04-23T14:32:49.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.659+0000 7f1669e00640 1 --2- 172.21.15.59:0/1911157251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c15e7c0 0x7f166c0d20f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f165c0097d0 tx=0x7f165c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0438985e2e8c943 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:49.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.660+0000 7f1669400640 1 -- 172.21.15.59:0/1911157251 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c037070 con 0x7f166c15e7c0 2024-04-23T14:32:49.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.660+0000 7f1669400640 1 -- 172.21.15.59:0/1911157251 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c032440 con 0x7f166c15e7c0 2024-04-23T14:32:49.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.660+0000 7f1669400640 1 -- 172.21.15.59:0/1911157251 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c032810 con 0x7f166c15e7c0 2024-04-23T14:32:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.661+0000 7f166b200640 1 -- 172.21.15.59:0/1911157251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c15e7c0 msgr2=0x7f166c0d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.661+0000 7f166b200640 1 --2- 172.21.15.59:0/1911157251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c15e7c0 0x7f166c0d20f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f165c0097d0 tx=0x7f165c02a210 comp rx=0 tx=0).stop 2024-04-23T14:32:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.661+0000 7f166b200640 1 -- 172.21.15.59:0/1911157251 shutdown_connections 2024-04-23T14:32:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.661+0000 7f166b200640 1 --2- 172.21.15.59:0/1911157251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c15e7c0 0x7f166c0d20f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.661+0000 7f166b200640 1 -- 172.21.15.59:0/1911157251 >> 172.21.15.59:0/1911157251 conn(0x7f166c09d920 msgr2=0x7f166c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.661+0000 7f166b200640 1 -- 172.21.15.59:0/1911157251 shutdown_connections 2024-04-23T14:32:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.661+0000 7f166b200640 1 -- 172.21.15.59:0/1911157251 wait complete. 2024-04-23T14:32:49.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.662+0000 7f166b200640 1 Processor -- start 2024-04-23T14:32:49.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.663+0000 7f166b200640 1 -- start start 2024-04-23T14:32:49.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c0d0d90 0x7f166c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:49.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f166c0d2630 0x7f166c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:49.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f166c15e7c0 0x7f166c0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:49.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0b6700 con 0x7f166c15e7c0 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0cf620 con 0x7f166c0d2630 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f166c0d1a80 con 0x7f166c0d0d90 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f166c15e7c0 0x7f166c0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f166c15e7c0 0x7f166c0cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50186/0 (socket says 172.21.15.59:50186) 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166a800640 1 -- 172.21.15.59:0/1876035396 learned_addr learned my addr 172.21.15.59:0/1876035396 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f1661400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f166c0d2630 0x7f166c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f1669e00640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c0d0d90 0x7f166c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.664+0000 7f166a800640 1 -- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c0d0d90 msgr2=0x7f166c1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f166a800640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c0d0d90 0x7f166c1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f166a800640 1 -- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f166c0d2630 msgr2=0x7f166c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f166a800640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f166c0d2630 0x7f166c0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f166a800640 1 -- 172.21.15.59:0/1876035396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0ce400 con 0x7f166c15e7c0 2024-04-23T14:32:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f1661400640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f166c0d2630 0x7f166c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f166a800640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f166c15e7c0 0x7f166c0cc5d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f166400c760 tx=0x7f166400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f1662a00640 1 -- 172.21.15.59:0/1876035396 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664003da0 con 0x7f166c15e7c0 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f1662a00640 1 -- 172.21.15.59:0/1876035396 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1664005070 con 0x7f166c15e7c0 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f1662a00640 1 -- 172.21.15.59:0/1876035396 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664005460 con 0x7f166c15e7c0 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c0ca770 con 0x7f166c15e7c0 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.665+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c0cad30 con 0x7f166c15e7c0 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.666+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c0d20f0 con 0x7f166c15e7c0 2024-04-23T14:32:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.667+0000 7f1662a00640 1 -- 172.21.15.59:0/1876035396 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1664005600 con 0x7f166c15e7c0 2024-04-23T14:32:49.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.668+0000 7f1662a00640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f164806c8f0 0x7f164806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:49.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.669+0000 7f1662a00640 1 -- 172.21.15.59:0/1876035396 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f166408b940 con 0x7f166c15e7c0 2024-04-23T14:32:49.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.669+0000 7f1669e00640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f164806c8f0 0x7f164806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:49.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.670+0000 7f1669e00640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f164806c8f0 0x7f164806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:49.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.676+0000 7f1662a00640 1 -- 172.21.15.59:0/1876035396 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f166c0d20f0 con 0x7f166c15e7c0 2024-04-23T14:32:49.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.880+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f166c04fd90 con 0x7f166c15e7c0 2024-04-23T14:32:49.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.882+0000 7f1662a00640 1 -- 172.21.15.59:0/1876035396 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16622 (secure 0 0 0) 0x7f1664059710 con 0x7f166c15e7c0 2024-04-23T14:32:49.884 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:49.884 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":32,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:48.830044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:49.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f164806c8f0 msgr2=0x7f164806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f164806c8f0 0x7f164806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).stop 2024-04-23T14:32:49.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f166c15e7c0 msgr2=0x7f166c0cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:49.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f166c15e7c0 0x7f166c0cc5d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f166400c760 tx=0x7f166400cc20 comp rx=0 tx=0).stop 2024-04-23T14:32:49.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 shutdown_connections 2024-04-23T14:32:49.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f164806c8f0 0x7f164806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f166c15e7c0 0x7f166c0cc5d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f166c0d2630 0x7f166c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 --2- 172.21.15.59:0/1876035396 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f166c0d0d90 0x7f166c1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:49.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 >> 172.21.15.59:0/1876035396 conn(0x7f166c09d920 msgr2=0x7f166c15cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:49.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 shutdown_connections 2024-04-23T14:32:49.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:49.885+0000 7f166b200640 1 -- 172.21.15.59:0/1876035396 wait complete. 2024-04-23T14:32:49.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.111+0000 7f84f6000640 1 Processor -- start 2024-04-23T14:32:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.112+0000 7f84f6000640 1 -- start start 2024-04-23T14:32:50.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.113+0000 7f84f6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f84f00d0b70 0x7f84f00d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.113+0000 7f84f6000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f00cd250 0x7f84f00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.113+0000 7f84f6000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f84f015e200 0x7f84f015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.113+0000 7f84f6000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f84f009b6f0 con 0x7f84f00d0b70 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.113+0000 7f84f6000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f84f00d1490 con 0x7f84f015e200 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.113+0000 7f84f6000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f84f00cd9f0 con 0x7f84f00cd250 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f84f00d0b70 0x7f84f00d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f84f00d0b70 0x7f84f00d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45500/0 (socket says 172.21.15.59:45500) 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 -- 172.21.15.59:0/890815007 learned_addr learned my addr 172.21.15.59:0/890815007 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84eea00640 1 --2- 172.21.15.59:0/890815007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f00cd250 0x7f84f00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84efe00640 1 --2- 172.21.15.59:0/890815007 >> v2:172.21.15.76:3300/0 conn(0x7f84f015e200 0x7f84f015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 -- 172.21.15.59:0/890815007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f00cd250 msgr2=0x7f84f00d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 --2- 172.21.15.59:0/890815007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f00cd250 0x7f84f00d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 -- 172.21.15.59:0/890815007 >> v2:172.21.15.76:3300/0 conn(0x7f84f015e200 msgr2=0x7f84f015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 --2- 172.21.15.59:0/890815007 >> v2:172.21.15.76:3300/0 conn(0x7f84f015e200 0x7f84f015e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84ef400640 1 -- 172.21.15.59:0/890815007 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f00cdcb0 con 0x7f84f00d0b70 2024-04-23T14:32:50.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.114+0000 7f84efe00640 1 --2- 172.21.15.59:0/890815007 >> v2:172.21.15.76:3300/0 conn(0x7f84f015e200 0x7f84f015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:50.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.115+0000 7f84ef400640 1 --2- 172.21.15.59:0/890815007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f00d0370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f84f00cd000 tx=0x7f84e002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1469a419f5b8296 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.116+0000 7f84ee000640 1 -- 172.21.15.59:0/890815007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0036070 con 0x7f84f00d0b70 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.116+0000 7f84ee000640 1 -- 172.21.15.59:0/890815007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0031600 con 0x7f84f00d0b70 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.116+0000 7f84ee000640 1 -- 172.21.15.59:0/890815007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00319d0 con 0x7f84f00d0b70 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.116+0000 7f84f6000640 1 -- 172.21.15.59:0/890815007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 msgr2=0x7f84f00d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.116+0000 7f84f6000640 1 --2- 172.21.15.59:0/890815007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f00d0370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f84f00cd000 tx=0x7f84e002a610 comp rx=0 tx=0).stop 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.116+0000 7f84f6000640 1 -- 172.21.15.59:0/890815007 shutdown_connections 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.117+0000 7f84f6000640 1 --2- 172.21.15.59:0/890815007 >> v2:172.21.15.76:3300/0 conn(0x7f84f015e200 0x7f84f015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.117+0000 7f84f6000640 1 --2- 172.21.15.59:0/890815007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f00cd250 0x7f84f00d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.117+0000 7f84f6000640 1 --2- 172.21.15.59:0/890815007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f00d0370 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.117+0000 7f84f6000640 1 -- 172.21.15.59:0/890815007 >> 172.21.15.59:0/890815007 conn(0x7f84f00b4080 msgr2=0x7f84f00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.117+0000 7f84f6000640 1 -- 172.21.15.59:0/890815007 shutdown_connections 2024-04-23T14:32:50.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.117+0000 7f84f6000640 1 -- 172.21.15.59:0/890815007 wait complete. 2024-04-23T14:32:50.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.119+0000 7f84f6000640 1 Processor -- start 2024-04-23T14:32:50.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.119+0000 7f84f6000640 1 -- start start 2024-04-23T14:32:50.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84f6000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84f00cd250 0x7f84f009fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84f6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f01f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84f6000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f015e200 0x7f84f006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84f6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84f00cd690 con 0x7f84f00d0b70 2024-04-23T14:32:50.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84f6000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84f00ce270 con 0x7f84f00cd250 2024-04-23T14:32:50.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84f6000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f84f015ebd0 con 0x7f84f015e200 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84eea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f01f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84eea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f01f4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45508/0 (socket says 172.21.15.59:45508) 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84eea00640 1 -- 172.21.15.59:0/3869617940 learned_addr learned my addr 172.21.15.59:0/3869617940 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84ef400640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84f00cd250 0x7f84f009fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.120+0000 7f84efe00640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f015e200 0x7f84f006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84eea00640 1 -- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f015e200 msgr2=0x7f84f006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84eea00640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f015e200 0x7f84f006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84eea00640 1 -- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84f00cd250 msgr2=0x7f84f009fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84eea00640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84f00cd250 0x7f84f009fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84eea00640 1 -- 172.21.15.59:0/3869617940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f01f5090 con 0x7f84f00d0b70 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84efe00640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f015e200 0x7f84f006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84eea00640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f01f4b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f84d800d800 tx=0x7f84d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84e7400640 1 -- 172.21.15.59:0/3869617940 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8018020 con 0x7f84f00d0b70 2024-04-23T14:32:50.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84e7400640 1 -- 172.21.15.59:0/3869617940 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d80096a0 con 0x7f84f00d0b70 2024-04-23T14:32:50.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84e7400640 1 -- 172.21.15.59:0/3869617940 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8009a70 con 0x7f84f00d0b70 2024-04-23T14:32:50.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f01f5330 con 0x7f84f00d0b70 2024-04-23T14:32:50.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.121+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01f58f0 con 0x7f84f00d0b70 2024-04-23T14:32:50.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.123+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f0068a80 con 0x7f84f00d0b70 2024-04-23T14:32:50.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.124+0000 7f84e7400640 1 -- 172.21.15.59:0/3869617940 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84d801b020 con 0x7f84f00d0b70 2024-04-23T14:32:50.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.124+0000 7f84e7400640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84cc06c8f0 0x7f84cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.125+0000 7f84e7400640 1 -- 172.21.15.59:0/3869617940 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f84d808bcc0 con 0x7f84f00d0b70 2024-04-23T14:32:50.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.125+0000 7f84ef400640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84cc06c8f0 0x7f84cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.126+0000 7f84ef400640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84cc06c8f0 0x7f84cc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84e0033000 tx=0x7f84e0009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.129+0000 7f84e7400640 1 -- 172.21.15.59:0/3869617940 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84d8092080 con 0x7f84f00d0b70 2024-04-23T14:32:50.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.334+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84f00d0950 con 0x7f84f00d0b70 2024-04-23T14:32:50.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.337+0000 7f84e7400640 1 -- 172.21.15.59:0/3869617940 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16622 (secure 0 0 0) 0x7f84d8022030 con 0x7f84f00d0b70 2024-04-23T14:32:50.339 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:50.339 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":32,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:48.830044+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:50.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.339+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84cc06c8f0 msgr2=0x7f84cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.339+0000 7f84f6000640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84cc06c8f0 0x7f84cc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f84e0033000 tx=0x7f84e0009000 comp rx=0 tx=0).stop 2024-04-23T14:32:50.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 msgr2=0x7f84f01f4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f01f4b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f84d800d800 tx=0x7f84d800dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 shutdown_connections 2024-04-23T14:32:50.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84cc06c8f0 0x7f84cc06edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84f015e200 0x7f84f006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f00d0b70 0x7f84f01f4b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 --2- 172.21.15.59:0/3869617940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84f00cd250 0x7f84f009fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 >> 172.21.15.59:0/3869617940 conn(0x7f84f00b4080 msgr2=0x7f84f015d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:50.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 shutdown_connections 2024-04-23T14:32:50.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.340+0000 7f84f6000640 1 -- 172.21.15.59:0/3869617940 wait complete. 2024-04-23T14:32:50.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:32:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.556+0000 7fae8ca00640 1 Processor -- start 2024-04-23T14:32:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.556+0000 7fae8ca00640 1 -- start start 2024-04-23T14:32:50.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.558+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae880ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.558+0000 7fae8ca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8815ef20 con 0x7fae880d0600 2024-04-23T14:32:50.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.558+0000 7fae8ca00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae880d0f20 con 0x7fae88155a80 2024-04-23T14:32:50.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.558+0000 7fae8ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae8815f1c0 con 0x7fae880d0a90 2024-04-23T14:32:50.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.558+0000 7fae86a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fae880d0600 0x7fae880ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35150/0 (socket says 172.21.15.59:35150) 2024-04-23T14:32:50.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.558+0000 7fae86a00640 1 -- 172.21.15.59:0/4188291524 learned_addr learned my addr 172.21.15.59:0/4188291524 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:50.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.558+0000 7fae87e00640 1 --2- 172.21.15.59:0/4188291524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae880ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.559+0000 7fae86000640 1 -- 172.21.15.59:0/4188291524 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606562780 0 0) 0x7fae8815ef20 con 0x7fae880d0600 2024-04-23T14:32:50.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.559+0000 7fae86000640 1 -- 172.21.15.59:0/4188291524 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae6c003520 con 0x7fae880d0600 2024-04-23T14:32:50.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.559+0000 7fae87e00640 1 -- 172.21.15.59:0/4188291524 >> v1:172.21.15.76:6789/0 conn(0x7fae88155a80 legacy=0x7fae8806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.559+0000 7fae87e00640 1 -- 172.21.15.59:0/4188291524 >> v1:172.21.15.59:6789/0 conn(0x7fae880d0600 legacy=0x7fae880ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.559+0000 7fae87e00640 1 -- 172.21.15.59:0/4188291524 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8815f4a0 con 0x7fae880d0a90 2024-04-23T14:32:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.560+0000 7fae87e00640 1 --2- 172.21.15.59:0/4188291524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae880ae9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fae80009dc0 tx=0x7fae8002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475172414b03998a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.560+0000 7fae86000640 1 -- 172.21.15.59:0/4188291524 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80007740 con 0x7fae880d0a90 2024-04-23T14:32:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.560+0000 7fae86000640 1 -- 172.21.15.59:0/4188291524 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae800078e0 con 0x7fae880d0a90 2024-04-23T14:32:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.560+0000 7fae86000640 1 -- 172.21.15.59:0/4188291524 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80007c90 con 0x7fae880d0a90 2024-04-23T14:32:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.561+0000 7fae8ca00640 1 -- 172.21.15.59:0/4188291524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 msgr2=0x7fae880ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.561+0000 7fae8ca00640 1 --2- 172.21.15.59:0/4188291524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae880ae9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fae80009dc0 tx=0x7fae8002a6d0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.561+0000 7fae8ca00640 1 -- 172.21.15.59:0/4188291524 shutdown_connections 2024-04-23T14:32:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.561+0000 7fae8ca00640 1 --2- 172.21.15.59:0/4188291524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae880ae9a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.561+0000 7fae8ca00640 1 -- 172.21.15.59:0/4188291524 >> 172.21.15.59:0/4188291524 conn(0x7fae8806bdf0 msgr2=0x7fae8806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.561+0000 7fae8ca00640 1 -- 172.21.15.59:0/4188291524 shutdown_connections 2024-04-23T14:32:50.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.561+0000 7fae8ca00640 1 -- 172.21.15.59:0/4188291524 wait complete. 2024-04-23T14:32:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.563+0000 7fae8ca00640 1 Processor -- start 2024-04-23T14:32:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.564+0000 7fae8ca00640 1 -- start start 2024-04-23T14:32:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880d0600 0x7fae881509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae88153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae8ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae88155a80 0x7fae8815a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae8ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae8815fd40 con 0x7fae88155a80 2024-04-23T14:32:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae8ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae8815fa60 con 0x7fae880d0600 2024-04-23T14:32:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae8ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae8806ef80 con 0x7fae880d0a90 2024-04-23T14:32:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae87e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae88155a80 0x7fae8815a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae87e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae88155a80 0x7fae8815a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45520/0 (socket says 172.21.15.59:45520) 2024-04-23T14:32:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae87e00640 1 -- 172.21.15.59:0/2993386397 learned_addr learned my addr 172.21.15.59:0/2993386397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae86a00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae88153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.565+0000 7fae87400640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880d0600 0x7fae881509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae87e00640 1 -- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 msgr2=0x7fae88153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae87e00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae88153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae87e00640 1 -- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880d0600 msgr2=0x7fae881509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae87e00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880d0600 0x7fae881509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae87e00640 1 -- 172.21.15.59:0/2993386397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae881f8be0 con 0x7fae88155a80 2024-04-23T14:32:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae86a00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae88153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae87e00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae88155a80 0x7fae8815a850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae80009ee0 tx=0x7fae80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.566+0000 7fae7f400640 1 -- 172.21.15.59:0/2993386397 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80009450 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.567+0000 7fae7f400640 1 -- 172.21.15.59:0/2993386397 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae80004240 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.567+0000 7fae7f400640 1 -- 172.21.15.59:0/2993386397 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80004610 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.567+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881f7e20 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.567+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881f83e0 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.568+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8804fd90 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.569+0000 7fae7f400640 1 -- 172.21.15.59:0/2993386397 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae80039040 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.570+0000 7fae7f400640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae6806c920 0x7fae6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.570+0000 7fae7f400640 1 -- 172.21.15.59:0/2993386397 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fae800acaa0 con 0x7fae88155a80 2024-04-23T14:32:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.570+0000 7fae87400640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae6806c920 0x7fae6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.571+0000 7fae87400640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae6806c920 0x7fae6806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae78001780 tx=0x7fae78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.575+0000 7fae7f400640 1 -- 172.21.15.59:0/2993386397 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae80040030 con 0x7fae88155a80 2024-04-23T14:32:50.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:50.824+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fae8806eba0 con 0x7fae88155a80 2024-04-23T14:32:51.238 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:32:51.237+0000 7fe562400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:32:51.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.414+0000 7fae7f400640 1 -- 172.21.15.59:0/2993386397 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fae8007a870 con 0x7fae88155a80 2024-04-23T14:32:51.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae6806c920 msgr2=0x7fae6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae6806c920 0x7fae6806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae78001780 tx=0x7fae78009040 comp rx=0 tx=0).stop 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae88155a80 msgr2=0x7fae8815a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae88155a80 0x7fae8815a850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae80009ee0 tx=0x7fae80004a40 comp rx=0 tx=0).stop 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 shutdown_connections 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae6806c920 0x7fae6806ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae880d0a90 0x7fae88153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae88155a80 0x7fae8815a850 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 --2- 172.21.15.59:0/2993386397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880d0600 0x7fae881509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.417+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 >> 172.21.15.59:0/2993386397 conn(0x7fae8806bdf0 msgr2=0x7fae880af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.418+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 shutdown_connections 2024-04-23T14:32:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.418+0000 7fae8ca00640 1 -- 172.21.15.59:0/2993386397 wait complete. 2024-04-23T14:32:51.444 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:32:51.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:32:51.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.663+0000 7f3193200640 1 Processor -- start 2024-04-23T14:32:51.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.663+0000 7f3193200640 1 -- start start 2024-04-23T14:32:51.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.664+0000 7f3193200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c15ed20 0x7f318c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:51.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.664+0000 7f3193200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f318c0b3600 0x7f318c0d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:51.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.664+0000 7f3193200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318c0b5da0 con 0x7f318c0b3f10 2024-04-23T14:32:51.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.664+0000 7f3193200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f318c15f100 con 0x7f318c0b3600 2024-04-23T14:32:51.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.664+0000 7f3193200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f318c0d0a20 con 0x7f318c15ed20 2024-04-23T14:32:51.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.665+0000 7f3191e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c15ed20 0x7f318c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:51.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.665+0000 7f3191e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c15ed20 0x7f318c0d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:38608/0 (socket says 172.21.15.59:38608) 2024-04-23T14:32:51.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.665+0000 7f3191e00640 1 -- 172.21.15.59:0/620995560 learned_addr learned my addr 172.21.15.59:0/620995560 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:51.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.666+0000 7f3191e00640 1 -- 172.21.15.59:0/620995560 >> v2:172.21.15.76:3300/0 conn(0x7f318c0b3600 msgr2=0x7f318c0d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:51.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.666+0000 7f3191e00640 1 --2- 172.21.15.59:0/620995560 >> v2:172.21.15.76:3300/0 conn(0x7f318c0b3600 0x7f318c0d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.666+0000 7f3191e00640 1 -- 172.21.15.59:0/620995560 >> v1:172.21.15.76:6789/0 conn(0x7f318c0b3f10 legacy=0x7f318c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:51.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.666+0000 7f3191e00640 1 -- 172.21.15.59:0/620995560 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c0d2ed0 con 0x7f318c15ed20 2024-04-23T14:32:51.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.667+0000 7f3191e00640 1 --2- 172.21.15.59:0/620995560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c15ed20 0x7f318c0d0ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3184009e50 tx=0x7f318402a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a32ff296b50bd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:51.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.667+0000 7f3190a00640 1 -- 172.21.15.59:0/620995560 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184036070 con 0x7f318c15ed20 2024-04-23T14:32:51.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.667+0000 7f3190a00640 1 -- 172.21.15.59:0/620995560 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3184031600 con 0x7f318c15ed20 2024-04-23T14:32:51.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.667+0000 7f3190a00640 1 -- 172.21.15.59:0/620995560 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31840319d0 con 0x7f318c15ed20 2024-04-23T14:32:51.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 -- 172.21.15.59:0/620995560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c15ed20 msgr2=0x7f318c0d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:51.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 --2- 172.21.15.59:0/620995560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c15ed20 0x7f318c0d0ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3184009e50 tx=0x7f318402a5b0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 -- 172.21.15.59:0/620995560 shutdown_connections 2024-04-23T14:32:51.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 --2- 172.21.15.59:0/620995560 >> v2:172.21.15.76:3300/0 conn(0x7f318c0b3600 0x7f318c0d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 --2- 172.21.15.59:0/620995560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c15ed20 0x7f318c0d0ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 -- 172.21.15.59:0/620995560 >> 172.21.15.59:0/620995560 conn(0x7f318c08bf90 msgr2=0x7f318c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:51.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 -- 172.21.15.59:0/620995560 shutdown_connections 2024-04-23T14:32:51.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.668+0000 7f3193200640 1 -- 172.21.15.59:0/620995560 wait complete. 2024-04-23T14:32:51.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.670+0000 7f3193200640 1 Processor -- start 2024-04-23T14:32:51.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.671+0000 7f3193200640 1 -- start start 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3193200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f318c0b3600 0x7f318c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3193200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c0b3f10 0x7f318c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3193200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318c15ed20 0x7f318c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3193200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0d0290 con 0x7f318c0b3600 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3193200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0d3490 con 0x7f318c15ed20 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3193200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f318c0d0410 con 0x7f318c0b3f10 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3191e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f318c0b3600 0x7f318c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:51.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3191e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f318c0b3600 0x7f318c15ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45534/0 (socket says 172.21.15.59:45534) 2024-04-23T14:32:51.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.672+0000 7f3191e00640 1 -- 172.21.15.59:0/1325020140 learned_addr learned my addr 172.21.15.59:0/1325020140 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3191400640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c0b3f10 0x7f318c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3192800640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318c15ed20 0x7f318c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3191e00640 1 -- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c0b3f10 msgr2=0x7f318c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3191e00640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c0b3f10 0x7f318c0b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3191e00640 1 -- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318c15ed20 msgr2=0x7f318c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3191e00640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318c15ed20 0x7f318c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3191e00640 1 -- 172.21.15.59:0/1325020140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c0b7160 con 0x7f318c0b3600 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.673+0000 7f3191e00640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f318c0b3600 0x7f318c15ae50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f31840093e0 tx=0x7f3184004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:51.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.674+0000 7f3182000640 1 -- 172.21.15.59:0/1325020140 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184041020 con 0x7f318c0b3600 2024-04-23T14:32:51.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.674+0000 7f3182000640 1 -- 172.21.15.59:0/1325020140 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3184037040 con 0x7f318c0b3600 2024-04-23T14:32:51.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.674+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c0b7440 con 0x7f318c0b3600 2024-04-23T14:32:51.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.674+0000 7f3182000640 1 -- 172.21.15.59:0/1325020140 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184036070 con 0x7f318c0b3600 2024-04-23T14:32:51.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.674+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c0cce70 con 0x7f318c0b3600 2024-04-23T14:32:51.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.675+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c04fd90 con 0x7f318c0b3600 2024-04-23T14:32:51.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.676+0000 7f3182000640 1 -- 172.21.15.59:0/1325020140 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3184044020 con 0x7f318c0b3600 2024-04-23T14:32:51.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.677+0000 7f3182000640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f317006c8f0 0x7f317006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:51.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.678+0000 7f3182000640 1 -- 172.21.15.59:0/1325020140 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f31840ac800 con 0x7f318c0b3600 2024-04-23T14:32:51.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.678+0000 7f3191400640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f317006c8f0 0x7f317006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:51.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.679+0000 7f3191400640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f317006c8f0 0x7f317006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f318c0b4eb0 tx=0x7f317c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:51.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.682+0000 7f3182000640 1 -- 172.21.15.59:0/1325020140 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f318404b030 con 0x7f318c0b3600 2024-04-23T14:32:51.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:51.930+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f318c0b2a60 con 0x7f318c0b3600 2024-04-23T14:32:52.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.425+0000 7f3182000640 1 -- 172.21.15.59:0/1325020140 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f318407acc0 con 0x7f318c0b3600 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.428+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f317006c8f0 msgr2=0x7f317006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.428+0000 7f3193200640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f317006c8f0 0x7f317006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f318c0b4eb0 tx=0x7f317c008040 comp rx=0 tx=0).stop 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.428+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f318c0b3600 msgr2=0x7f318c15ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.428+0000 7f3193200640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f318c0b3600 0x7f318c15ae50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f31840093e0 tx=0x7f3184004a40 comp rx=0 tx=0).stop 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 shutdown_connections 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f317006c8f0 0x7f317006edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f318c0b3f10 0x7f318c0b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318c15ed20 0x7f318c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 --2- 172.21.15.59:0/1325020140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f318c0b3600 0x7f318c15ae50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 >> 172.21.15.59:0/1325020140 conn(0x7f318c08bf90 msgr2=0x7f318c0d4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 shutdown_connections 2024-04-23T14:32:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.429+0000 7f3193200640 1 -- 172.21.15.59:0/1325020140 wait complete. 2024-04-23T14:32:52.452 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:32:52.452 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:32:52.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.666+0000 7f653de00640 1 Processor -- start 2024-04-23T14:32:52.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.666+0000 7f653de00640 1 -- start start 2024-04-23T14:32:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.667+0000 7f653de00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f65380cd760 0x7f65380b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f653de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65380b4690 0x7f65380cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f653de00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6538160210 con 0x7f653815c3d0 2024-04-23T14:32:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f653de00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f65381604f0 con 0x7f65380b4690 2024-04-23T14:32:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f653de00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f65381607d0 con 0x7f65380cd760 2024-04-23T14:32:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f653d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65380b4690 0x7f65380cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f653d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65380b4690 0x7f65380cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45546/0 (socket says 172.21.15.59:45546) 2024-04-23T14:32:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f653d400640 1 -- 172.21.15.59:0/4163749303 learned_addr learned my addr 172.21.15.59:0/4163749303 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.668+0000 7f6537e00640 1 --2- 172.21.15.59:0/4163749303 >> v2:172.21.15.76:3300/0 conn(0x7f65380cd760 0x7f65380b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.669+0000 7f653d400640 1 -- 172.21.15.59:0/4163749303 >> v2:172.21.15.76:3300/0 conn(0x7f65380cd760 msgr2=0x7f65380b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.669+0000 7f653d400640 1 --2- 172.21.15.59:0/4163749303 >> v2:172.21.15.76:3300/0 conn(0x7f65380cd760 0x7f65380b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.669+0000 7f653d400640 1 -- 172.21.15.59:0/4163749303 >> v1:172.21.15.76:6789/0 conn(0x7f653815c3d0 legacy=0x7f65380cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.669+0000 7f653d400640 1 -- 172.21.15.59:0/4163749303 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6538160ab0 con 0x7f65380b4690 2024-04-23T14:32:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.670+0000 7f653d400640 1 --2- 172.21.15.59:0/4163749303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380b4690 0x7f65380cb7d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f653806e620 tx=0x7f653002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0335f7d18ef5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f6537400640 1 -- 172.21.15.59:0/4163749303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530036070 con 0x7f65380b4690 2024-04-23T14:32:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f6537400640 1 -- 172.21.15.59:0/4163749303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6530031600 con 0x7f65380b4690 2024-04-23T14:32:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f6537400640 1 -- 172.21.15.59:0/4163749303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300319d0 con 0x7f65380b4690 2024-04-23T14:32:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f653de00640 1 -- 172.21.15.59:0/4163749303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380b4690 msgr2=0x7f65380cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f653de00640 1 --2- 172.21.15.59:0/4163749303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380b4690 0x7f65380cb7d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f653806e620 tx=0x7f653002a730 comp rx=0 tx=0).stop 2024-04-23T14:32:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f653de00640 1 -- 172.21.15.59:0/4163749303 shutdown_connections 2024-04-23T14:32:52.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f653de00640 1 --2- 172.21.15.59:0/4163749303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380b4690 0x7f65380cb7d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f653de00640 1 --2- 172.21.15.59:0/4163749303 >> v2:172.21.15.76:3300/0 conn(0x7f65380cd760 0x7f65380b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.671+0000 7f653de00640 1 -- 172.21.15.59:0/4163749303 >> 172.21.15.59:0/4163749303 conn(0x7f65380b21a0 msgr2=0x7f65380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:52.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.672+0000 7f653de00640 1 -- 172.21.15.59:0/4163749303 shutdown_connections 2024-04-23T14:32:52.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.672+0000 7f653de00640 1 -- 172.21.15.59:0/4163749303 wait complete. 2024-04-23T14:32:52.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.673+0000 7f653de00640 1 Processor -- start 2024-04-23T14:32:52.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.673+0000 7f653de00640 1 -- start start 2024-04-23T14:32:52.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.674+0000 7f653de00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65380b4690 0x7f65380ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.674+0000 7f653de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380cd760 0x7f65380c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.674+0000 7f653de00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653815c3d0 0x7f65380ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.674+0000 7f653de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f653815c7b0 con 0x7f65380cd760 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.674+0000 7f653de00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65380b4ae0 con 0x7f653815c3d0 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.674+0000 7f653de00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f653815c930 con 0x7f65380b4690 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380cd760 0x7f65380c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380cd760 0x7f65380c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45552/0 (socket says 172.21.15.59:45552) 2024-04-23T14:32:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 -- 172.21.15.59:0/2933127796 learned_addr learned my addr 172.21.15.59:0/2933127796 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f653d400640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653815c3d0 0x7f65380ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f653ca00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65380b4690 0x7f65380ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 -- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65380b4690 msgr2=0x7f65380ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65380b4690 0x7f65380ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 -- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653815c3d0 msgr2=0x7f65380ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653815c3d0 0x7f65380ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f6537e00640 1 -- 172.21.15.59:0/2933127796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65380ae1c0 con 0x7f65380cd760 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.675+0000 7f653d400640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653815c3d0 0x7f65380ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.676+0000 7f6537e00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380cd760 0x7f65380c5540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f652400d800 tx=0x7f652400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.676+0000 7f6534c00640 1 -- 172.21.15.59:0/2933127796 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652400b6c0 con 0x7f65380cd760 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.676+0000 7f6534c00640 1 -- 172.21.15.59:0/2933127796 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6524019070 con 0x7f65380cd760 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.676+0000 7f6534c00640 1 -- 172.21.15.59:0/2933127796 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652400ba80 con 0x7f65380cd760 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.676+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65380ae500 con 0x7f65380cd760 2024-04-23T14:32:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.676+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65380aed40 con 0x7f65380cd760 2024-04-23T14:32:52.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.677+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653804fd90 con 0x7f65380cd760 2024-04-23T14:32:52.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.679+0000 7f6534c00640 1 -- 172.21.15.59:0/2933127796 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f652400bc20 con 0x7f65380cd760 2024-04-23T14:32:52.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.679+0000 7f6534c00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651806c8f0 0x7f651806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:52.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.680+0000 7f6534c00640 1 -- 172.21.15.59:0/2933127796 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f652408c0e0 con 0x7f65380cd760 2024-04-23T14:32:52.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.680+0000 7f653ca00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651806c8f0 0x7f651806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.680+0000 7f653ca00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651806c8f0 0x7f651806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f652c007fd0 tx=0x7f652c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.684+0000 7f6534c00640 1 -- 172.21.15.59:0/2933127796 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65380ae500 con 0x7f65380cd760 2024-04-23T14:32:52.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.926+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f653806e400 con 0x7f65380cd760 2024-04-23T14:32:52.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.927+0000 7f6534c00640 1 -- 172.21.15.59:0/2933127796 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f652401c360 con 0x7f65380cd760 2024-04-23T14:32:52.928 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:52.928 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:32:52.929 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:32:52.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651806c8f0 msgr2=0x7f651806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651806c8f0 0x7f651806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f652c007fd0 tx=0x7f652c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:52.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380cd760 msgr2=0x7f65380c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:52.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380cd760 0x7f65380c5540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f652400d800 tx=0x7f652400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 shutdown_connections 2024-04-23T14:32:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f651806c8f0 0x7f651806edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f653815c3d0 0x7f65380ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65380cd760 0x7f65380c5540 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 --2- 172.21.15.59:0/2933127796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65380b4690 0x7f65380ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 >> 172.21.15.59:0/2933127796 conn(0x7f65380b21a0 msgr2=0x7f65380cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 shutdown_connections 2024-04-23T14:32:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:52.930+0000 7f653de00640 1 -- 172.21.15.59:0/2933127796 wait complete. 2024-04-23T14:32:52.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:32:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.152+0000 7f0e89200640 1 Processor -- start 2024-04-23T14:32:53.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.153+0000 7f0e89200640 1 -- start start 2024-04-23T14:32:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.154+0000 7f0e89200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.154+0000 7f0e89200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0e840d0d90 0x7f0e840d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.154+0000 7f0e89200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e840cf030 con 0x7f0e840d2630 2024-04-23T14:32:53.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.154+0000 7f0e89200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0e840b7000 con 0x7f0e840d0d90 2024-04-23T14:32:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.154+0000 7f0e89200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0e840cf620 con 0x7f0e8415e7c0 2024-04-23T14:32:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.155+0000 7f0e83400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.155+0000 7f0e83400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:38628/0 (socket says 172.21.15.59:38628) 2024-04-23T14:32:53.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.155+0000 7f0e82a00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0e840d2630 0x7f0e840b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:60802/0 (socket says 172.21.15.59:60802) 2024-04-23T14:32:53.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.155+0000 7f0e83400640 1 -- 172.21.15.59:0/3460180378 learned_addr learned my addr 172.21.15.59:0/3460180378 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:53.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.155+0000 7f0e83e00640 1 --2- 172.21.15.59:0/3460180378 >> v2:172.21.15.76:3300/0 conn(0x7f0e840d0d90 0x7f0e840d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.156+0000 7f0e82000640 1 -- 172.21.15.59:0/3460180378 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103647182 0 0) 0x7f0e6c001110 con 0x7f0e840d2630 2024-04-23T14:32:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.156+0000 7f0e82000640 1 -- 172.21.15.59:0/3460180378 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e68003520 con 0x7f0e840d2630 2024-04-23T14:32:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.156+0000 7f0e83e00640 1 -- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 msgr2=0x7f0e840d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.156+0000 7f0e83e00640 1 --2- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.156+0000 7f0e83e00640 1 -- 172.21.15.59:0/3460180378 >> v1:172.21.15.76:6789/0 conn(0x7f0e840d2630 legacy=0x7f0e840b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.156+0000 7f0e83e00640 1 -- 172.21.15.59:0/3460180378 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e840b6a40 con 0x7f0e840d0d90 2024-04-23T14:32:53.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.156+0000 7f0e83400640 1 --2- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:32:53.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.158+0000 7f0e83e00640 1 --2- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d0d90 0x7f0e840d1220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0097a0 tx=0x7f0e7c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf00f99b563c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:53.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.158+0000 7f0e82000640 1 -- 172.21.15.59:0/3460180378 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c037070 con 0x7f0e840d0d90 2024-04-23T14:32:53.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.158+0000 7f0e82000640 1 -- 172.21.15.59:0/3460180378 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7c032440 con 0x7f0e840d0d90 2024-04-23T14:32:53.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.158+0000 7f0e82000640 1 -- 172.21.15.59:0/3460180378 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c032810 con 0x7f0e840d0d90 2024-04-23T14:32:53.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 -- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d0d90 msgr2=0x7f0e840d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 --2- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d0d90 0x7f0e840d1220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0097a0 tx=0x7f0e7c02a180 comp rx=0 tx=0).stop 2024-04-23T14:32:53.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 -- 172.21.15.59:0/3460180378 shutdown_connections 2024-04-23T14:32:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 --2- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d0d90 0x7f0e840d1220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 --2- 172.21.15.59:0/3460180378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 -- 172.21.15.59:0/3460180378 >> 172.21.15.59:0/3460180378 conn(0x7f0e8409d920 msgr2=0x7f0e8408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 -- 172.21.15.59:0/3460180378 shutdown_connections 2024-04-23T14:32:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.159+0000 7f0e89200640 1 -- 172.21.15.59:0/3460180378 wait complete. 2024-04-23T14:32:53.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.161+0000 7f0e89200640 1 Processor -- start 2024-04-23T14:32:53.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.161+0000 7f0e89200640 1 -- start start 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.162+0000 7f0e89200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e840d0d90 0x7f0e840cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.162+0000 7f0e89200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d2630 0x7f0e840b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.162+0000 7f0e89200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.162+0000 7f0e89200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e840b6700 con 0x7f0e840d0d90 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.162+0000 7f0e89200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0e840cf2b0 con 0x7f0e840d2630 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.162+0000 7f0e89200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0e840d1a80 con 0x7f0e8415e7c0 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.162+0000 7f0e83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e840d0d90 0x7f0e840cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e840d0d90 0x7f0e840cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45554/0 (socket says 172.21.15.59:45554) 2024-04-23T14:32:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 -- 172.21.15.59:0/2307290415 learned_addr learned my addr 172.21.15.59:0/2307290415 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e82a00640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d2630 0x7f0e840b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 -- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 msgr2=0x7f0e840b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 -- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d2630 msgr2=0x7f0e840b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d2630 0x7f0e840b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 -- 172.21.15.59:0/2307290415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e840cae80 con 0x7f0e840d0d90 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e82a00640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d2630 0x7f0e840b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.163+0000 7f0e83400640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e840d0d90 0x7f0e840cdf50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009e80 tx=0x7f0e7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.164+0000 7f0e7b400640 1 -- 172.21.15.59:0/2307290415 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400cde0 con 0x7f0e840d0d90 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.164+0000 7f0e7b400640 1 -- 172.21.15.59:0/2307290415 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e74019070 con 0x7f0e840d0d90 2024-04-23T14:32:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.164+0000 7f0e7b400640 1 -- 172.21.15.59:0/2307290415 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74014680 con 0x7f0e840d0d90 2024-04-23T14:32:53.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.164+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e840ca8c0 con 0x7f0e840d0d90 2024-04-23T14:32:53.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.164+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e840cb690 con 0x7f0e840d0d90 2024-04-23T14:32:53.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.165+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8404fd90 con 0x7f0e840d0d90 2024-04-23T14:32:53.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.166+0000 7f0e7b400640 1 -- 172.21.15.59:0/2307290415 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e74005d10 con 0x7f0e840d0d90 2024-04-23T14:32:53.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.167+0000 7f0e7b400640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.167+0000 7f0e7b400640 1 -- 172.21.15.59:0/2307290415 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0e7408bff0 con 0x7f0e840d0d90 2024-04-23T14:32:53.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.167+0000 7f0e82a00640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.168+0000 7f0e82a00640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c920 0x7f0e6006ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e841f84e0 tx=0x7f0e6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:53.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.172+0000 7f0e7b400640 1 -- 172.21.15.59:0/2307290415 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e74010040 con 0x7f0e840d0d90 2024-04-23T14:32:53.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.380+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e840b64e0 con 0x7f0e840d0d90 2024-04-23T14:32:53.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.382+0000 7f0e7b400640 1 -- 172.21.15.59:0/2307290415 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f0e7405a3a0 con 0x7f0e840d0d90 2024-04-23T14:32:53.384 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:53.384 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:50.389436+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:32:53.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c920 msgr2=0x7f0e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c920 0x7f0e6006ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e841f84e0 tx=0x7f0e6c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:53.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e840d0d90 msgr2=0x7f0e840cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e840d0d90 0x7f0e840cdf50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009e80 tx=0x7f0e7400c490 comp rx=0 tx=0).stop 2024-04-23T14:32:53.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 shutdown_connections 2024-04-23T14:32:53.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e8415e7c0 0x7f0e840b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e840d2630 0x7f0e840b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 --2- 172.21.15.59:0/2307290415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e840d0d90 0x7f0e840cdf50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 >> 172.21.15.59:0/2307290415 conn(0x7f0e8409d920 msgr2=0x7f0e8406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:53.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 shutdown_connections 2024-04-23T14:32:53.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.386+0000 7f0e89200640 1 -- 172.21.15.59:0/2307290415 wait complete. 2024-04-23T14:32:53.408 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:32:53.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.610+0000 7f1f9c200640 1 Processor -- start 2024-04-23T14:32:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.611+0000 7f1f9c200640 1 -- start start 2024-04-23T14:32:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.612+0000 7f1f9c200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f940cd020 0x7f1f940b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.613+0000 7f1f9c200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f940cd520 con 0x7f1f941454e0 2024-04-23T14:32:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.613+0000 7f1f9c200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f940cd800 con 0x7f1f940b3e40 2024-04-23T14:32:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.613+0000 7f1f9c200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f940cddc0 con 0x7f1f940cd020 2024-04-23T14:32:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.613+0000 7f1f99800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1f941454e0 0x7f1f94145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35164/0 (socket says 172.21.15.59:35164) 2024-04-23T14:32:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.613+0000 7f1f99800640 1 -- 172.21.15.59:0/286541565 learned_addr learned my addr 172.21.15.59:0/286541565 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.613+0000 7f1f98e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f940cd020 0x7f1f940b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1619129709 0 0) 0x7f1f940cd520 con 0x7f1f941454e0 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f7c003520 con 0x7f1f941454e0 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 58097693 0 0) 0x7f1f7c003520 con 0x7f1f941454e0 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f940cd020 msgr2=0x7f1f940b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f940cd020 0x7f1f940b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 >> v1:172.21.15.76:6789/0 conn(0x7f1f940b3e40 legacy=0x7f1f94160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f940cdae0 con 0x7f1f941454e0 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1f88001670 con 0x7f1f941454e0 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 >> v1:172.21.15.59:6789/0 conn(0x7f1f941454e0 legacy=0x7f1f94145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c0039e0 0x7f1f7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f940cd020 0x7f1f7c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f940cddc0 con 0x7f1f7c004800 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c00bb50 con 0x7f1f7c0039e0 2024-04-23T14:32:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.614+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7c00bd50 con 0x7f1f940cd020 2024-04-23T14:32:53.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f98e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f98e00640 1 -- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f940cd020 msgr2=0x7f1f7c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f98e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f940cd020 0x7f1f7c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f98e00640 1 -- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c0039e0 msgr2=0x7f1f7c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:53.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f99800640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c0039e0 0x7f1f7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f98e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c0039e0 0x7f1f7c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f98e00640 1 -- 172.21.15.59:0/286541565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c003520 con 0x7f1f7c004800 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.615+0000 7f1f99800640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c0039e0 0x7f1f7c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.616+0000 7f1f98e00640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009ac0 tx=0x7f1f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba0cf45e68f7979 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80036070 con 0x7f1f7c004800 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f80031400 con 0x7f1f7c004800 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f93e00640 1 -- 172.21.15.59:0/286541565 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80009690 con 0x7f1f7c004800 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f9c200640 1 -- 172.21.15.59:0/286541565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f7c004800 msgr2=0x7f1f7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f9c200640 1 --2- 172.21.15.59:0/286541565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009ac0 tx=0x7f1f7c005830 comp rx=0 tx=0).stop 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f9a200640 1 -- 172.21.15.59:0/286541565 reap_dead start 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f9c200640 1 -- 172.21.15.59:0/286541565 shutdown_connections 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f9c200640 1 -- 172.21.15.59:0/286541565 >> 172.21.15.59:0/286541565 conn(0x7f1f94067ea0 msgr2=0x7f1f9406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f9c200640 1 -- 172.21.15.59:0/286541565 shutdown_connections 2024-04-23T14:32:53.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.617+0000 7f1f9c200640 1 -- 172.21.15.59:0/286541565 wait complete. 2024-04-23T14:32:53.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.619+0000 7f1f9c200640 1 Processor -- start 2024-04-23T14:32:53.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.619+0000 7f1f9c200640 1 -- start start 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.620+0000 7f1f9c200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c004800 0x7f1f940aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.620+0000 7f1f9c200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f9415d670 0x7f1f9406dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.620+0000 7f1f9c200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9415daf0 0x7f1f9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.620+0000 7f1f9c200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94161020 con 0x7f1f9415daf0 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.620+0000 7f1f9c200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941462c0 con 0x7f1f7c004800 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.620+0000 7f1f9c200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f941611a0 con 0x7f1f9415d670 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9415daf0 0x7f1f9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9415daf0 0x7f1f9406e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45576/0 (socket says 172.21.15.59:45576) 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 -- 172.21.15.59:0/3567264861 learned_addr learned my addr 172.21.15.59:0/3567264861 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f99800640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c004800 0x7f1f940aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f98e00640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f9415d670 0x7f1f9406dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 -- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f9415d670 msgr2=0x7f1f9406dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f9415d670 0x7f1f9406dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 -- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c004800 msgr2=0x7f1f940aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c004800 0x7f1f940aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.621+0000 7f1f9a200640 1 -- 172.21.15.59:0/3567264861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9415f680 con 0x7f1f9415daf0 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.622+0000 7f1f9a200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9415daf0 0x7f1f9406e320 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c004780 tx=0x7f1f8c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.622+0000 7f1f91600640 1 -- 172.21.15.59:0/3567264861 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c017070 con 0x7f1f9415daf0 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.622+0000 7f1f91600640 1 -- 172.21.15.59:0/3567264861 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c0070d0 con 0x7f1f9415daf0 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.622+0000 7f1f91600640 1 -- 172.21.15.59:0/3567264861 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c007500 con 0x7f1f9415daf0 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.622+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9415f960 con 0x7f1f9415daf0 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.622+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9415c620 con 0x7f1f9415daf0 2024-04-23T14:32:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.623+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f94068340 con 0x7f1f9415daf0 2024-04-23T14:32:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.624+0000 7f1f91600640 1 -- 172.21.15.59:0/3567264861 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f8c01a070 con 0x7f1f9415daf0 2024-04-23T14:32:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.625+0000 7f1f91600640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f7406c970 0x7f1f7406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.625+0000 7f1f91600640 1 -- 172.21.15.59:0/3567264861 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1f8c08acc0 con 0x7f1f9415daf0 2024-04-23T14:32:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.626+0000 7f1f99800640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f7406c970 0x7f1f7406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:53.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.626+0000 7f1f99800640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f7406c970 0x7f1f7406ee30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f88000ff0 tx=0x7f1f88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:53.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.630+0000 7f1f91600640 1 -- 172.21.15.59:0/3567264861 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f94068340 con 0x7f1f9415daf0 2024-04-23T14:32:53.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.828+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f94068a80 con 0x7f1f7406c970 2024-04-23T14:32:53.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.844+0000 7f1f91600640 1 -- 172.21.15.59:0/3567264861 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251346 (secure 0 0 0) 0x7f1f94068a80 con 0x7f1f7406c970 2024-04-23T14:32:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:53.849 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:32:53.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.848+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f7406c970 msgr2=0x7f1f7406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.848+0000 7f1f9c200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f7406c970 0x7f1f7406ee30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f88000ff0 tx=0x7f1f88009040 comp rx=0 tx=0).stop 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.848+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9415daf0 msgr2=0x7f1f9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.848+0000 7f1f9c200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9415daf0 0x7f1f9406e320 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c004780 tx=0x7f1f8c00c980 comp rx=0 tx=0).stop 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 shutdown_connections 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f7406c970 0x7f1f7406ee30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f9415d670 0x7f1f9406dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9415daf0 0x7f1f9406e320 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 --2- 172.21.15.59:0/3567264861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f7c004800 0x7f1f940aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 >> 172.21.15.59:0/3567264861 conn(0x7f1f94067ea0 msgr2=0x7f1f94071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 shutdown_connections 2024-04-23T14:32:53.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:53.849+0000 7f1f9c200640 1 -- 172.21.15.59:0/3567264861 wait complete. 2024-04-23T14:32:53.867 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-23T14:32:52.370386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224208,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750512,"statfs":{"total":773094113280,"available":772864524288,"internally_reserved":0,"allocated":3973120,"data_stored":2249000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.029950"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408276+0000","last_change":"2024-04-23T14:32:49.396796+0000","last_active":"2024-04-23T14:32:49.408276+0000","last_peered":"2024-04-23T14:32:49.408276+0000","last_clean":"2024-04-23T14:32:49.408276+0000","last_became_active":"2024-04-23T14:32:49.396505+0000","last_became_peered":"2024-04-23T14:32:49.396505+0000","last_unstale":"2024-04-23T14:32:49.408276+0000","last_undegraded":"2024-04-23T14:32:49.408276+0000","last_fullsized":"2024-04-23T14:32:49.408276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407907+0000","last_change":"2024-04-23T14:32:49.396404+0000","last_active":"2024-04-23T14:32:49.407907+0000","last_peered":"2024-04-23T14:32:49.407907+0000","last_clean":"2024-04-23T14:32:49.407907+0000","last_became_active":"2024-04-23T14:32:49.396103+0000","last_became_peered":"2024-04-23T14:32:49.396103+0000","last_unstale":"2024-04-23T14:32:49.407907+0000","last_undegraded":"2024-04-23T14:32:49.407907+0000","last_fullsized":"2024-04-23T14:32:49.407907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.382675+0000","last_change":"2024-04-23T14:32:49.382675+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.382675+0000","last_undegraded":"2024-04-23T14:32:49.382675+0000","last_fullsized":"2024-04-23T14:32:49.382675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418842+0000","last_change":"2024-04-23T14:32:49.412161+0000","last_active":"2024-04-23T14:32:49.418842+0000","last_peered":"2024-04-23T14:32:49.418842+0000","last_clean":"2024-04-23T14:32:49.418842+0000","last_became_active":"2024-04-23T14:32:49.411609+0000","last_became_peered":"2024-04-23T14:32:49.411609+0000","last_unstale":"2024-04-23T14:32:49.418842+0000","last_undegraded":"2024-04-23T14:32:49.418842+0000","last_fullsized":"2024-04-23T14:32:49.418842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411075+0000","last_change":"2024-04-23T14:32:49.411075+0000","last_active":"2024-04-23T14:32:49.411075+0000","last_peered":"2024-04-23T14:32:49.411075+0000","last_clean":"2024-04-23T14:32:49.411075+0000","last_became_active":"2024-04-23T14:32:49.410648+0000","last_became_peered":"2024-04-23T14:32:49.410648+0000","last_unstale":"2024-04-23T14:32:49.411075+0000","last_undegraded":"2024-04-23T14:32:49.411075+0000","last_fullsized":"2024-04-23T14:32:49.411075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:19.233384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412676+0000","last_change":"2024-04-23T14:32:49.396761+0000","last_active":"2024-04-23T14:32:49.412676+0000","last_peered":"2024-04-23T14:32:49.412676+0000","last_clean":"2024-04-23T14:32:49.412676+0000","last_became_active":"2024-04-23T14:32:49.396598+0000","last_became_peered":"2024-04-23T14:32:49.396598+0000","last_unstale":"2024-04-23T14:32:49.412676+0000","last_undegraded":"2024-04-23T14:32:49.412676+0000","last_fullsized":"2024-04-23T14:32:49.412676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407019+0000","last_change":"2024-04-23T14:32:49.398865+0000","last_active":"2024-04-23T14:32:49.407019+0000","last_peered":"2024-04-23T14:32:49.407019+0000","last_clean":"2024-04-23T14:32:49.407019+0000","last_became_active":"2024-04-23T14:32:49.398581+0000","last_became_peered":"2024-04-23T14:32:49.398581+0000","last_unstale":"2024-04-23T14:32:49.407019+0000","last_undegraded":"2024-04-23T14:32:49.407019+0000","last_fullsized":"2024-04-23T14:32:49.407019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409232+0000","last_change":"2024-04-23T14:32:49.409232+0000","last_active":"2024-04-23T14:32:49.409232+0000","last_peered":"2024-04-23T14:32:49.409232+0000","last_clean":"2024-04-23T14:32:49.409232+0000","last_became_active":"2024-04-23T14:32:49.408162+0000","last_became_peered":"2024-04-23T14:32:49.408162+0000","last_unstale":"2024-04-23T14:32:49.409232+0000","last_undegraded":"2024-04-23T14:32:49.409232+0000","last_fullsized":"2024-04-23T14:32:49.409232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.424075+0000","last_change":"2024-04-23T14:32:49.424075+0000","last_active":"2024-04-23T14:32:49.424075+0000","last_peered":"2024-04-23T14:32:49.424075+0000","last_clean":"2024-04-23T14:32:49.424075+0000","last_became_active":"2024-04-23T14:32:49.423714+0000","last_became_peered":"2024-04-23T14:32:49.423714+0000","last_unstale":"2024-04-23T14:32:49.424075+0000","last_undegraded":"2024-04-23T14:32:49.424075+0000","last_fullsized":"2024-04-23T14:32:49.424075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421421+0000","last_change":"2024-04-23T14:32:49.419419+0000","last_active":"2024-04-23T14:32:49.421421+0000","last_peered":"2024-04-23T14:32:49.421421+0000","last_clean":"2024-04-23T14:32:49.421421+0000","last_became_active":"2024-04-23T14:32:49.416916+0000","last_became_peered":"2024-04-23T14:32:49.416916+0000","last_unstale":"2024-04-23T14:32:49.421421+0000","last_undegraded":"2024-04-23T14:32:49.421421+0000","last_fullsized":"2024-04-23T14:32:49.421421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389894+0000","last_change":"2024-04-23T14:32:49.389894+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389894+0000","last_undegraded":"2024-04-23T14:32:49.389894+0000","last_fullsized":"2024-04-23T14:32:49.389894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414777+0000","last_change":"2024-04-23T14:32:49.414777+0000","last_active":"2024-04-23T14:32:49.414777+0000","last_peered":"2024-04-23T14:32:49.414777+0000","last_clean":"2024-04-23T14:32:49.414777+0000","last_became_active":"2024-04-23T14:32:49.414208+0000","last_became_peered":"2024-04-23T14:32:49.414208+0000","last_unstale":"2024-04-23T14:32:49.414777+0000","last_undegraded":"2024-04-23T14:32:49.414777+0000","last_fullsized":"2024-04-23T14:32:49.414777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389314+0000","last_change":"2024-04-23T14:32:49.389314+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389314+0000","last_undegraded":"2024-04-23T14:32:49.389314+0000","last_fullsized":"2024-04-23T14:32:49.389314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409664+0000","last_change":"2024-04-23T14:32:49.409664+0000","last_active":"2024-04-23T14:32:49.409664+0000","last_peered":"2024-04-23T14:32:49.409664+0000","last_clean":"2024-04-23T14:32:49.409664+0000","last_became_active":"2024-04-23T14:32:49.409369+0000","last_became_peered":"2024-04-23T14:32:49.409369+0000","last_unstale":"2024-04-23T14:32:49.409664+0000","last_undegraded":"2024-04-23T14:32:49.409664+0000","last_fullsized":"2024-04-23T14:32:49.409664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:53.618947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404125+0000","last_change":"2024-04-23T14:32:49.395881+0000","last_active":"2024-04-23T14:32:49.404125+0000","last_peered":"2024-04-23T14:32:49.404125+0000","last_clean":"2024-04-23T14:32:49.404125+0000","last_became_active":"2024-04-23T14:32:49.395640+0000","last_became_peered":"2024-04-23T14:32:49.395640+0000","last_unstale":"2024-04-23T14:32:49.404125+0000","last_undegraded":"2024-04-23T14:32:49.404125+0000","last_fullsized":"2024-04-23T14:32:49.404125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410041+0000","last_change":"2024-04-23T14:32:49.410041+0000","last_active":"2024-04-23T14:32:49.410041+0000","last_peered":"2024-04-23T14:32:49.410041+0000","last_clean":"2024-04-23T14:32:49.410041+0000","last_became_active":"2024-04-23T14:32:49.409759+0000","last_became_peered":"2024-04-23T14:32:49.409759+0000","last_unstale":"2024-04-23T14:32:49.410041+0000","last_undegraded":"2024-04-23T14:32:49.410041+0000","last_fullsized":"2024-04-23T14:32:49.410041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408902+0000","last_change":"2024-04-23T14:32:49.405810+0000","last_active":"2024-04-23T14:32:49.408902+0000","last_peered":"2024-04-23T14:32:49.408902+0000","last_clean":"2024-04-23T14:32:49.408902+0000","last_became_active":"2024-04-23T14:32:49.405283+0000","last_became_peered":"2024-04-23T14:32:49.405283+0000","last_unstale":"2024-04-23T14:32:49.408902+0000","last_undegraded":"2024-04-23T14:32:49.408902+0000","last_fullsized":"2024-04-23T14:32:49.408902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.379405+0000","last_change":"2024-04-23T14:32:49.379405+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.379405+0000","last_undegraded":"2024-04-23T14:32:49.379405+0000","last_fullsized":"2024-04-23T14:32:49.379405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.378840+0000","last_change":"2024-04-23T14:32:49.378840+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.378840+0000","last_undegraded":"2024-04-23T14:32:49.378840+0000","last_fullsized":"2024-04-23T14:32:49.378840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408152+0000","last_change":"2024-04-23T14:32:49.408152+0000","last_active":"2024-04-23T14:32:49.408152+0000","last_peered":"2024-04-23T14:32:49.408152+0000","last_clean":"2024-04-23T14:32:49.408152+0000","last_became_active":"2024-04-23T14:32:49.407758+0000","last_became_peered":"2024-04-23T14:32:49.407758+0000","last_unstale":"2024-04-23T14:32:49.408152+0000","last_undegraded":"2024-04-23T14:32:49.408152+0000","last_fullsized":"2024-04-23T14:32:49.408152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:42.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416792+0000","last_change":"2024-04-23T14:32:49.413362+0000","last_active":"2024-04-23T14:32:49.416792+0000","last_peered":"2024-04-23T14:32:49.416792+0000","last_clean":"2024-04-23T14:32:49.416792+0000","last_became_active":"2024-04-23T14:32:49.413082+0000","last_became_peered":"2024-04-23T14:32:49.413082+0000","last_unstale":"2024-04-23T14:32:49.416792+0000","last_undegraded":"2024-04-23T14:32:49.416792+0000","last_fullsized":"2024-04-23T14:32:49.416792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:57.535200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417468+0000","last_change":"2024-04-23T14:32:49.414460+0000","last_active":"2024-04-23T14:32:49.417468+0000","last_peered":"2024-04-23T14:32:49.417468+0000","last_clean":"2024-04-23T14:32:49.417468+0000","last_became_active":"2024-04-23T14:32:49.413658+0000","last_became_peered":"2024-04-23T14:32:49.413658+0000","last_unstale":"2024-04-23T14:32:49.417468+0000","last_undegraded":"2024-04-23T14:32:49.417468+0000","last_fullsized":"2024-04-23T14:32:49.417468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:32.704139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412931+0000","last_change":"2024-04-23T14:32:49.404657+0000","last_active":"2024-04-23T14:32:49.412931+0000","last_peered":"2024-04-23T14:32:49.412931+0000","last_clean":"2024-04-23T14:32:49.412931+0000","last_became_active":"2024-04-23T14:32:49.404322+0000","last_became_peered":"2024-04-23T14:32:49.404322+0000","last_unstale":"2024-04-23T14:32:49.412931+0000","last_undegraded":"2024-04-23T14:32:49.412931+0000","last_fullsized":"2024-04-23T14:32:49.412931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416596+0000","last_change":"2024-04-23T14:32:49.408981+0000","last_active":"2024-04-23T14:32:49.416596+0000","last_peered":"2024-04-23T14:32:49.416596+0000","last_clean":"2024-04-23T14:32:49.416596+0000","last_became_active":"2024-04-23T14:32:49.408732+0000","last_became_peered":"2024-04-23T14:32:49.408732+0000","last_unstale":"2024-04-23T14:32:49.416596+0000","last_undegraded":"2024-04-23T14:32:49.416596+0000","last_fullsized":"2024-04-23T14:32:49.416596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419806+0000","last_change":"2024-04-23T14:32:49.411802+0000","last_active":"2024-04-23T14:32:49.419806+0000","last_peered":"2024-04-23T14:32:49.419806+0000","last_clean":"2024-04-23T14:32:49.419806+0000","last_became_active":"2024-04-23T14:32:49.411472+0000","last_became_peered":"2024-04-23T14:32:49.411472+0000","last_unstale":"2024-04-23T14:32:49.419806+0000","last_undegraded":"2024-04-23T14:32:49.419806+0000","last_fullsized":"2024-04-23T14:32:49.419806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408321+0000","last_change":"2024-04-23T14:32:49.408321+0000","last_active":"2024-04-23T14:32:49.408321+0000","last_peered":"2024-04-23T14:32:49.408321+0000","last_clean":"2024-04-23T14:32:49.408321+0000","last_became_active":"2024-04-23T14:32:49.407885+0000","last_became_peered":"2024-04-23T14:32:49.407885+0000","last_unstale":"2024-04-23T14:32:49.408321+0000","last_undegraded":"2024-04-23T14:32:49.408321+0000","last_fullsized":"2024-04-23T14:32:49.408321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420428+0000","last_change":"2024-04-23T14:32:49.418183+0000","last_active":"2024-04-23T14:32:49.420428+0000","last_peered":"2024-04-23T14:32:49.420428+0000","last_clean":"2024-04-23T14:32:49.420428+0000","last_became_active":"2024-04-23T14:32:49.417903+0000","last_became_peered":"2024-04-23T14:32:49.417903+0000","last_unstale":"2024-04-23T14:32:49.420428+0000","last_undegraded":"2024-04-23T14:32:49.420428+0000","last_fullsized":"2024-04-23T14:32:49.420428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404038+0000","last_change":"2024-04-23T14:32:49.396858+0000","last_active":"2024-04-23T14:32:49.404038+0000","last_peered":"2024-04-23T14:32:49.404038+0000","last_clean":"2024-04-23T14:32:49.404038+0000","last_became_active":"2024-04-23T14:32:49.396565+0000","last_became_peered":"2024-04-23T14:32:49.396565+0000","last_unstale":"2024-04-23T14:32:49.404038+0000","last_undegraded":"2024-04-23T14:32:49.404038+0000","last_fullsized":"2024-04-23T14:32:49.404038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419860+0000","last_change":"2024-04-23T14:32:49.419860+0000","last_active":"2024-04-23T14:32:49.419860+0000","last_peered":"2024-04-23T14:32:49.419860+0000","last_clean":"2024-04-23T14:32:49.419860+0000","last_became_active":"2024-04-23T14:32:49.419578+0000","last_became_peered":"2024-04-23T14:32:49.419578+0000","last_unstale":"2024-04-23T14:32:49.419860+0000","last_undegraded":"2024-04-23T14:32:49.419860+0000","last_fullsized":"2024-04-23T14:32:49.419860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407177+0000","last_change":"2024-04-23T14:32:49.403367+0000","last_active":"2024-04-23T14:32:49.407177+0000","last_peered":"2024-04-23T14:32:49.407177+0000","last_clean":"2024-04-23T14:32:49.407177+0000","last_became_active":"2024-04-23T14:32:49.403043+0000","last_became_peered":"2024-04-23T14:32:49.403043+0000","last_unstale":"2024-04-23T14:32:49.407177+0000","last_undegraded":"2024-04-23T14:32:49.407177+0000","last_fullsized":"2024-04-23T14:32:49.407177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413518+0000","last_change":"2024-04-23T14:32:49.407909+0000","last_active":"2024-04-23T14:32:49.413518+0000","last_peered":"2024-04-23T14:32:49.413518+0000","last_clean":"2024-04-23T14:32:49.413518+0000","last_became_active":"2024-04-23T14:32:49.407653+0000","last_became_peered":"2024-04-23T14:32:49.407653+0000","last_unstale":"2024-04-23T14:32:49.413518+0000","last_undegraded":"2024-04-23T14:32:49.413518+0000","last_fullsized":"2024-04-23T14:32:49.413518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412291+0000","last_change":"2024-04-23T14:32:49.412291+0000","last_active":"2024-04-23T14:32:49.412291+0000","last_peered":"2024-04-23T14:32:49.412291+0000","last_clean":"2024-04-23T14:32:49.412291+0000","last_became_active":"2024-04-23T14:32:49.411966+0000","last_became_peered":"2024-04-23T14:32:49.411966+0000","last_unstale":"2024-04-23T14:32:49.412291+0000","last_undegraded":"2024-04-23T14:32:49.412291+0000","last_fullsized":"2024-04-23T14:32:49.412291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401707+0000","last_change":"2024-04-23T14:32:49.398365+0000","last_active":"2024-04-23T14:32:49.401707+0000","last_peered":"2024-04-23T14:32:49.401707+0000","last_clean":"2024-04-23T14:32:49.401707+0000","last_became_active":"2024-04-23T14:32:49.398036+0000","last_became_peered":"2024-04-23T14:32:49.398036+0000","last_unstale":"2024-04-23T14:32:49.401707+0000","last_undegraded":"2024-04-23T14:32:49.401707+0000","last_fullsized":"2024-04-23T14:32:49.401707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.380750+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.380399+0000","last_became_peered":"2024-04-23T14:32:48.380399+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:31.689650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414566+0000","last_change":"2024-04-23T14:32:49.411206+0000","last_active":"2024-04-23T14:32:49.414566+0000","last_peered":"2024-04-23T14:32:49.414566+0000","last_clean":"2024-04-23T14:32:49.414566+0000","last_became_active":"2024-04-23T14:32:49.410469+0000","last_became_peered":"2024-04-23T14:32:49.410469+0000","last_unstale":"2024-04-23T14:32:49.414566+0000","last_undegraded":"2024-04-23T14:32:49.414566+0000","last_fullsized":"2024-04-23T14:32:49.414566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395347+0000","last_change":"2024-04-23T14:32:48.378555+0000","last_active":"2024-04-23T14:32:49.395347+0000","last_peered":"2024-04-23T14:32:49.395347+0000","last_clean":"2024-04-23T14:32:49.395347+0000","last_became_active":"2024-04-23T14:32:48.377594+0000","last_became_peered":"2024-04-23T14:32:48.377594+0000","last_unstale":"2024-04-23T14:32:49.395347+0000","last_undegraded":"2024-04-23T14:32:49.395347+0000","last_fullsized":"2024-04-23T14:32:49.395347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418168+0000","last_change":"2024-04-23T14:32:49.418168+0000","last_active":"2024-04-23T14:32:49.418168+0000","last_peered":"2024-04-23T14:32:49.418168+0000","last_clean":"2024-04-23T14:32:49.418168+0000","last_became_active":"2024-04-23T14:32:49.417542+0000","last_became_peered":"2024-04-23T14:32:49.417542+0000","last_unstale":"2024-04-23T14:32:49.418168+0000","last_undegraded":"2024-04-23T14:32:49.418168+0000","last_fullsized":"2024-04-23T14:32:49.418168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381457+0000","last_change":"2024-04-23T14:32:48.383157+0000","last_active":"2024-04-23T14:32:49.381457+0000","last_peered":"2024-04-23T14:32:49.381457+0000","last_clean":"2024-04-23T14:32:49.381457+0000","last_became_active":"2024-04-23T14:32:48.382568+0000","last_became_peered":"2024-04-23T14:32:48.382568+0000","last_unstale":"2024-04-23T14:32:49.381457+0000","last_undegraded":"2024-04-23T14:32:49.381457+0000","last_fullsized":"2024-04-23T14:32:49.381457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:35.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420450+0000","last_change":"2024-04-23T14:32:49.418053+0000","last_active":"2024-04-23T14:32:49.420450+0000","last_peered":"2024-04-23T14:32:49.420450+0000","last_clean":"2024-04-23T14:32:49.420450+0000","last_became_active":"2024-04-23T14:32:49.417733+0000","last_became_peered":"2024-04-23T14:32:49.417733+0000","last_unstale":"2024-04-23T14:32:49.420450+0000","last_undegraded":"2024-04-23T14:32:49.420450+0000","last_fullsized":"2024-04-23T14:32:49.420450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391055+0000","last_change":"2024-04-23T14:32:48.374028+0000","last_active":"2024-04-23T14:32:49.391055+0000","last_peered":"2024-04-23T14:32:49.391055+0000","last_clean":"2024-04-23T14:32:49.391055+0000","last_became_active":"2024-04-23T14:32:48.372871+0000","last_became_peered":"2024-04-23T14:32:48.372871+0000","last_unstale":"2024-04-23T14:32:49.391055+0000","last_undegraded":"2024-04-23T14:32:49.391055+0000","last_fullsized":"2024-04-23T14:32:49.391055+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:14.297285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410412+0000","last_change":"2024-04-23T14:32:49.410412+0000","last_active":"2024-04-23T14:32:49.410412+0000","last_peered":"2024-04-23T14:32:49.410412+0000","last_clean":"2024-04-23T14:32:49.410412+0000","last_became_active":"2024-04-23T14:32:49.409645+0000","last_became_peered":"2024-04-23T14:32:49.409645+0000","last_unstale":"2024-04-23T14:32:49.410412+0000","last_undegraded":"2024-04-23T14:32:49.410412+0000","last_fullsized":"2024-04-23T14:32:49.410412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401440+0000","last_change":"2024-04-23T14:32:48.381407+0000","last_active":"2024-04-23T14:32:49.401440+0000","last_peered":"2024-04-23T14:32:49.401440+0000","last_clean":"2024-04-23T14:32:49.401440+0000","last_became_active":"2024-04-23T14:32:48.381122+0000","last_became_peered":"2024-04-23T14:32:48.381122+0000","last_unstale":"2024-04-23T14:32:49.401440+0000","last_undegraded":"2024-04-23T14:32:49.401440+0000","last_fullsized":"2024-04-23T14:32:49.401440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:16:40.112260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408197+0000","last_change":"2024-04-23T14:32:49.400326+0000","last_active":"2024-04-23T14:32:49.408197+0000","last_peered":"2024-04-23T14:32:49.408197+0000","last_clean":"2024-04-23T14:32:49.408197+0000","last_became_active":"2024-04-23T14:32:49.399716+0000","last_became_peered":"2024-04-23T14:32:49.399716+0000","last_unstale":"2024-04-23T14:32:49.408197+0000","last_undegraded":"2024-04-23T14:32:49.408197+0000","last_fullsized":"2024-04-23T14:32:49.408197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401159+0000","last_change":"2024-04-23T14:32:48.380347+0000","last_active":"2024-04-23T14:32:49.401159+0000","last_peered":"2024-04-23T14:32:49.401159+0000","last_clean":"2024-04-23T14:32:49.401159+0000","last_became_active":"2024-04-23T14:32:48.379799+0000","last_became_peered":"2024-04-23T14:32:48.379799+0000","last_unstale":"2024-04-23T14:32:49.401159+0000","last_undegraded":"2024-04-23T14:32:49.401159+0000","last_fullsized":"2024-04-23T14:32:49.401159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:13.096562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421993+0000","last_change":"2024-04-23T14:32:49.415576+0000","last_active":"2024-04-23T14:32:49.421993+0000","last_peered":"2024-04-23T14:32:49.421993+0000","last_clean":"2024-04-23T14:32:49.421993+0000","last_became_active":"2024-04-23T14:32:49.415107+0000","last_became_peered":"2024-04-23T14:32:49.415107+0000","last_unstale":"2024-04-23T14:32:49.421993+0000","last_undegraded":"2024-04-23T14:32:49.421993+0000","last_fullsized":"2024-04-23T14:32:49.421993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:21.969136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375434+0000","last_change":"2024-04-23T14:32:48.378186+0000","last_active":"2024-04-23T14:32:49.375434+0000","last_peered":"2024-04-23T14:32:49.375434+0000","last_clean":"2024-04-23T14:32:49.375434+0000","last_became_active":"2024-04-23T14:32:48.377889+0000","last_became_peered":"2024-04-23T14:32:48.377889+0000","last_unstale":"2024-04-23T14:32:49.375434+0000","last_undegraded":"2024-04-23T14:32:49.375434+0000","last_fullsized":"2024-04-23T14:32:49.375434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420363+0000","last_change":"2024-04-23T14:32:49.418062+0000","last_active":"2024-04-23T14:32:49.420363+0000","last_peered":"2024-04-23T14:32:49.420363+0000","last_clean":"2024-04-23T14:32:49.420363+0000","last_became_active":"2024-04-23T14:32:49.417335+0000","last_became_peered":"2024-04-23T14:32:49.417335+0000","last_unstale":"2024-04-23T14:32:49.420363+0000","last_undegraded":"2024-04-23T14:32:49.420363+0000","last_fullsized":"2024-04-23T14:32:49.420363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401048+0000","last_change":"2024-04-23T14:32:48.376375+0000","last_active":"2024-04-23T14:32:49.401048+0000","last_peered":"2024-04-23T14:32:49.401048+0000","last_clean":"2024-04-23T14:32:49.401048+0000","last_became_active":"2024-04-23T14:32:48.375807+0000","last_became_peered":"2024-04-23T14:32:48.375807+0000","last_unstale":"2024-04-23T14:32:49.401048+0000","last_undegraded":"2024-04-23T14:32:49.401048+0000","last_fullsized":"2024-04-23T14:32:49.401048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:21.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414650+0000","last_change":"2024-04-23T14:32:49.414650+0000","last_active":"2024-04-23T14:32:49.414650+0000","last_peered":"2024-04-23T14:32:49.414650+0000","last_clean":"2024-04-23T14:32:49.414650+0000","last_became_active":"2024-04-23T14:32:49.414352+0000","last_became_peered":"2024-04-23T14:32:49.414352+0000","last_unstale":"2024-04-23T14:32:49.414650+0000","last_undegraded":"2024-04-23T14:32:49.414650+0000","last_fullsized":"2024-04-23T14:32:49.414650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380697+0000","last_change":"2024-04-23T14:32:48.380295+0000","last_active":"2024-04-23T14:32:49.380697+0000","last_peered":"2024-04-23T14:32:49.380697+0000","last_clean":"2024-04-23T14:32:49.380697+0000","last_became_active":"2024-04-23T14:32:48.379968+0000","last_became_peered":"2024-04-23T14:32:48.379968+0000","last_unstale":"2024-04-23T14:32:49.380697+0000","last_undegraded":"2024-04-23T14:32:49.380697+0000","last_fullsized":"2024-04-23T14:32:49.380697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:28.545681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408527+0000","last_change":"2024-04-23T14:32:49.408527+0000","last_active":"2024-04-23T14:32:49.408527+0000","last_peered":"2024-04-23T14:32:49.408527+0000","last_clean":"2024-04-23T14:32:49.408527+0000","last_became_active":"2024-04-23T14:32:49.408214+0000","last_became_peered":"2024-04-23T14:32:49.408214+0000","last_unstale":"2024-04-23T14:32:49.408527+0000","last_undegraded":"2024-04-23T14:32:49.408527+0000","last_fullsized":"2024-04-23T14:32:49.408527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395886+0000","last_change":"2024-04-23T14:32:48.369769+0000","last_active":"2024-04-23T14:32:49.395886+0000","last_peered":"2024-04-23T14:32:49.395886+0000","last_clean":"2024-04-23T14:32:49.395886+0000","last_became_active":"2024-04-23T14:32:48.369112+0000","last_became_peered":"2024-04-23T14:32:48.369112+0000","last_unstale":"2024-04-23T14:32:49.395886+0000","last_undegraded":"2024-04-23T14:32:49.395886+0000","last_fullsized":"2024-04-23T14:32:49.395886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:14.196347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409092+0000","last_change":"2024-04-23T14:32:49.409092+0000","last_active":"2024-04-23T14:32:49.409092+0000","last_peered":"2024-04-23T14:32:49.409092+0000","last_clean":"2024-04-23T14:32:49.409092+0000","last_became_active":"2024-04-23T14:32:49.408769+0000","last_became_peered":"2024-04-23T14:32:49.408769+0000","last_unstale":"2024-04-23T14:32:49.409092+0000","last_undegraded":"2024-04-23T14:32:49.409092+0000","last_fullsized":"2024-04-23T14:32:49.409092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:07.688839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.383330+0000","last_change":"2024-04-23T14:32:48.379746+0000","last_active":"2024-04-23T14:32:49.383330+0000","last_peered":"2024-04-23T14:32:49.383330+0000","last_clean":"2024-04-23T14:32:49.383330+0000","last_became_active":"2024-04-23T14:32:48.378760+0000","last_became_peered":"2024-04-23T14:32:48.378760+0000","last_unstale":"2024-04-23T14:32:49.383330+0000","last_undegraded":"2024-04-23T14:32:49.383330+0000","last_fullsized":"2024-04-23T14:32:49.383330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383029+0000","last_change":"2024-04-23T14:32:49.383029+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383029+0000","last_undegraded":"2024-04-23T14:32:49.383029+0000","last_fullsized":"2024-04-23T14:32:49.383029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.378297+0000","last_change":"2024-04-23T14:32:48.379776+0000","last_active":"2024-04-23T14:32:49.378297+0000","last_peered":"2024-04-23T14:32:49.378297+0000","last_clean":"2024-04-23T14:32:49.378297+0000","last_became_active":"2024-04-23T14:32:48.379485+0000","last_became_peered":"2024-04-23T14:32:48.379485+0000","last_unstale":"2024-04-23T14:32:49.378297+0000","last_undegraded":"2024-04-23T14:32:49.378297+0000","last_fullsized":"2024-04-23T14:32:49.378297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:00.386998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418531+0000","last_change":"2024-04-23T14:32:49.409577+0000","last_active":"2024-04-23T14:32:49.418531+0000","last_peered":"2024-04-23T14:32:49.418531+0000","last_clean":"2024-04-23T14:32:49.418531+0000","last_became_active":"2024-04-23T14:32:49.409264+0000","last_became_peered":"2024-04-23T14:32:49.409264+0000","last_unstale":"2024-04-23T14:32:49.418531+0000","last_undegraded":"2024-04-23T14:32:49.418531+0000","last_fullsized":"2024-04-23T14:32:49.418531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401594+0000","last_change":"2024-04-23T14:32:48.376245+0000","last_active":"2024-04-23T14:32:49.401594+0000","last_peered":"2024-04-23T14:32:49.401594+0000","last_clean":"2024-04-23T14:32:49.401594+0000","last_became_active":"2024-04-23T14:32:48.375882+0000","last_became_peered":"2024-04-23T14:32:48.375882+0000","last_unstale":"2024-04-23T14:32:49.401594+0000","last_undegraded":"2024-04-23T14:32:49.401594+0000","last_fullsized":"2024-04-23T14:32:49.401594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:27.386064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.403473+0000","last_change":"2024-04-23T14:32:49.403473+0000","last_active":"2024-04-23T14:32:49.403473+0000","last_peered":"2024-04-23T14:32:49.403473+0000","last_clean":"2024-04-23T14:32:49.403473+0000","last_became_active":"2024-04-23T14:32:49.399541+0000","last_became_peered":"2024-04-23T14:32:49.399541+0000","last_unstale":"2024-04-23T14:32:49.403473+0000","last_undegraded":"2024-04-23T14:32:49.403473+0000","last_fullsized":"2024-04-23T14:32:49.403473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397792+0000","last_change":"2024-04-23T14:32:48.381791+0000","last_active":"2024-04-23T14:32:49.397792+0000","last_peered":"2024-04-23T14:32:49.397792+0000","last_clean":"2024-04-23T14:32:49.397792+0000","last_became_active":"2024-04-23T14:32:48.381531+0000","last_became_peered":"2024-04-23T14:32:48.381531+0000","last_unstale":"2024-04-23T14:32:49.397792+0000","last_undegraded":"2024-04-23T14:32:49.397792+0000","last_fullsized":"2024-04-23T14:32:49.397792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412273+0000","last_change":"2024-04-23T14:32:49.412273+0000","last_active":"2024-04-23T14:32:49.412273+0000","last_peered":"2024-04-23T14:32:49.412273+0000","last_clean":"2024-04-23T14:32:49.412273+0000","last_became_active":"2024-04-23T14:32:49.411939+0000","last_became_peered":"2024-04-23T14:32:49.411939+0000","last_unstale":"2024-04-23T14:32:49.412273+0000","last_undegraded":"2024-04-23T14:32:49.412273+0000","last_fullsized":"2024-04-23T14:32:49.412273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375257+0000","last_change":"2024-04-23T14:32:48.383719+0000","last_active":"2024-04-23T14:32:49.375257+0000","last_peered":"2024-04-23T14:32:49.375257+0000","last_clean":"2024-04-23T14:32:49.375257+0000","last_became_active":"2024-04-23T14:32:48.382885+0000","last_became_peered":"2024-04-23T14:32:48.382885+0000","last_unstale":"2024-04-23T14:32:49.375257+0000","last_undegraded":"2024-04-23T14:32:49.375257+0000","last_fullsized":"2024-04-23T14:32:49.375257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414486+0000","last_change":"2024-04-23T14:32:49.406690+0000","last_active":"2024-04-23T14:32:49.414486+0000","last_peered":"2024-04-23T14:32:49.414486+0000","last_clean":"2024-04-23T14:32:49.414486+0000","last_became_active":"2024-04-23T14:32:49.406431+0000","last_became_peered":"2024-04-23T14:32:49.406431+0000","last_unstale":"2024-04-23T14:32:49.414486+0000","last_undegraded":"2024-04-23T14:32:49.414486+0000","last_fullsized":"2024-04-23T14:32:49.414486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375967+0000","last_change":"2024-04-23T14:32:48.375851+0000","last_active":"2024-04-23T14:32:49.375967+0000","last_peered":"2024-04-23T14:32:49.375967+0000","last_clean":"2024-04-23T14:32:49.375967+0000","last_became_active":"2024-04-23T14:32:48.375572+0000","last_became_peered":"2024-04-23T14:32:48.375572+0000","last_unstale":"2024-04-23T14:32:49.375967+0000","last_undegraded":"2024-04-23T14:32:49.375967+0000","last_fullsized":"2024-04-23T14:32:49.375967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417434+0000","last_change":"2024-04-23T14:32:49.409809+0000","last_active":"2024-04-23T14:32:49.417434+0000","last_peered":"2024-04-23T14:32:49.417434+0000","last_clean":"2024-04-23T14:32:49.417434+0000","last_became_active":"2024-04-23T14:32:49.409554+0000","last_became_peered":"2024-04-23T14:32:49.409554+0000","last_unstale":"2024-04-23T14:32:49.417434+0000","last_undegraded":"2024-04-23T14:32:49.417434+0000","last_fullsized":"2024-04-23T14:32:49.417434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:39.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400370+0000","last_change":"2024-04-23T14:32:48.383194+0000","last_active":"2024-04-23T14:32:49.400370+0000","last_peered":"2024-04-23T14:32:49.400370+0000","last_clean":"2024-04-23T14:32:49.400370+0000","last_became_active":"2024-04-23T14:32:48.382867+0000","last_became_peered":"2024-04-23T14:32:48.382867+0000","last_unstale":"2024-04-23T14:32:49.400370+0000","last_undegraded":"2024-04-23T14:32:49.400370+0000","last_fullsized":"2024-04-23T14:32:49.400370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413412+0000","last_change":"2024-04-23T14:32:49.413412+0000","last_active":"2024-04-23T14:32:49.413412+0000","last_peered":"2024-04-23T14:32:49.413412+0000","last_clean":"2024-04-23T14:32:49.413412+0000","last_became_active":"2024-04-23T14:32:49.413066+0000","last_became_peered":"2024-04-23T14:32:49.413066+0000","last_unstale":"2024-04-23T14:32:49.413412+0000","last_undegraded":"2024-04-23T14:32:49.413412+0000","last_fullsized":"2024-04-23T14:32:49.413412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401743+0000","last_change":"2024-04-23T14:32:48.373256+0000","last_active":"2024-04-23T14:32:49.401743+0000","last_peered":"2024-04-23T14:32:49.401743+0000","last_clean":"2024-04-23T14:32:49.401743+0000","last_became_active":"2024-04-23T14:32:48.372720+0000","last_became_peered":"2024-04-23T14:32:48.372720+0000","last_unstale":"2024-04-23T14:32:49.401743+0000","last_undegraded":"2024-04-23T14:32:49.401743+0000","last_fullsized":"2024-04-23T14:32:49.401743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:04:15.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418820+0000","last_change":"2024-04-23T14:32:49.411786+0000","last_active":"2024-04-23T14:32:49.418820+0000","last_peered":"2024-04-23T14:32:49.418820+0000","last_clean":"2024-04-23T14:32:49.418820+0000","last_became_active":"2024-04-23T14:32:49.411468+0000","last_became_peered":"2024-04-23T14:32:49.411468+0000","last_unstale":"2024-04-23T14:32:49.418820+0000","last_undegraded":"2024-04-23T14:32:49.418820+0000","last_fullsized":"2024-04-23T14:32:49.418820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401645+0000","last_change":"2024-04-23T14:32:48.374462+0000","last_active":"2024-04-23T14:32:49.401645+0000","last_peered":"2024-04-23T14:32:49.401645+0000","last_clean":"2024-04-23T14:32:49.401645+0000","last_became_active":"2024-04-23T14:32:48.372060+0000","last_became_peered":"2024-04-23T14:32:48.372060+0000","last_unstale":"2024-04-23T14:32:49.401645+0000","last_undegraded":"2024-04-23T14:32:49.401645+0000","last_fullsized":"2024-04-23T14:32:49.401645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:40.111344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.406897+0000","last_change":"2024-04-23T14:32:49.406897+0000","last_active":"2024-04-23T14:32:49.406897+0000","last_peered":"2024-04-23T14:32:49.406897+0000","last_clean":"2024-04-23T14:32:49.406897+0000","last_became_active":"2024-04-23T14:32:49.406604+0000","last_became_peered":"2024-04-23T14:32:49.406604+0000","last_unstale":"2024-04-23T14:32:49.406897+0000","last_undegraded":"2024-04-23T14:32:49.406897+0000","last_fullsized":"2024-04-23T14:32:49.406897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:32.281665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401493+0000","last_change":"2024-04-23T14:32:48.382650+0000","last_active":"2024-04-23T14:32:49.401493+0000","last_peered":"2024-04-23T14:32:49.401493+0000","last_clean":"2024-04-23T14:32:49.401493+0000","last_became_active":"2024-04-23T14:32:48.382391+0000","last_became_peered":"2024-04-23T14:32:48.382391+0000","last_unstale":"2024-04-23T14:32:49.401493+0000","last_undegraded":"2024-04-23T14:32:49.401493+0000","last_fullsized":"2024-04-23T14:32:49.401493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:51.353237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410725+0000","last_change":"2024-04-23T14:32:49.410725+0000","last_active":"2024-04-23T14:32:49.410725+0000","last_peered":"2024-04-23T14:32:49.410725+0000","last_clean":"2024-04-23T14:32:49.410725+0000","last_became_active":"2024-04-23T14:32:49.410335+0000","last_became_peered":"2024-04-23T14:32:49.410335+0000","last_unstale":"2024-04-23T14:32:49.410725+0000","last_undegraded":"2024-04-23T14:32:49.410725+0000","last_fullsized":"2024-04-23T14:32:49.410725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:23:29.638232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396023+0000","last_change":"2024-04-23T14:32:48.372373+0000","last_active":"2024-04-23T14:32:49.396023+0000","last_peered":"2024-04-23T14:32:49.396023+0000","last_clean":"2024-04-23T14:32:49.396023+0000","last_became_active":"2024-04-23T14:32:48.372040+0000","last_became_peered":"2024-04-23T14:32:48.372040+0000","last_unstale":"2024-04-23T14:32:49.396023+0000","last_undegraded":"2024-04-23T14:32:49.396023+0000","last_fullsized":"2024-04-23T14:32:49.396023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:24.434665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413110+0000","last_change":"2024-04-23T14:32:49.403841+0000","last_active":"2024-04-23T14:32:49.413110+0000","last_peered":"2024-04-23T14:32:49.413110+0000","last_clean":"2024-04-23T14:32:49.413110+0000","last_became_active":"2024-04-23T14:32:49.403486+0000","last_became_peered":"2024-04-23T14:32:49.403486+0000","last_unstale":"2024-04-23T14:32:49.413110+0000","last_undegraded":"2024-04-23T14:32:49.413110+0000","last_fullsized":"2024-04-23T14:32:49.413110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:48.649791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.381995+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.381595+0000","last_became_peered":"2024-04-23T14:32:48.381595+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:56.520711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413093+0000","last_change":"2024-04-23T14:32:49.405667+0000","last_active":"2024-04-23T14:32:49.413093+0000","last_peered":"2024-04-23T14:32:49.413093+0000","last_clean":"2024-04-23T14:32:49.413093+0000","last_became_active":"2024-04-23T14:32:49.403620+0000","last_became_peered":"2024-04-23T14:32:49.403620+0000","last_unstale":"2024-04-23T14:32:49.413093+0000","last_undegraded":"2024-04-23T14:32:49.413093+0000","last_fullsized":"2024-04-23T14:32:49.413093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:25.894354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401019+0000","last_change":"2024-04-23T14:32:48.375422+0000","last_active":"2024-04-23T14:32:49.401019+0000","last_peered":"2024-04-23T14:32:49.401019+0000","last_clean":"2024-04-23T14:32:49.401019+0000","last_became_active":"2024-04-23T14:32:48.375126+0000","last_became_peered":"2024-04-23T14:32:48.375126+0000","last_unstale":"2024-04-23T14:32:49.401019+0000","last_undegraded":"2024-04-23T14:32:49.401019+0000","last_fullsized":"2024-04-23T14:32:49.401019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:18.218895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407040+0000","last_change":"2024-04-23T14:32:49.399201+0000","last_active":"2024-04-23T14:32:49.407040+0000","last_peered":"2024-04-23T14:32:49.407040+0000","last_clean":"2024-04-23T14:32:49.407040+0000","last_became_active":"2024-04-23T14:32:49.398682+0000","last_became_peered":"2024-04-23T14:32:49.398682+0000","last_unstale":"2024-04-23T14:32:49.407040+0000","last_undegraded":"2024-04-23T14:32:49.407040+0000","last_fullsized":"2024-04-23T14:32:49.407040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381590+0000","last_change":"2024-04-23T14:32:48.378313+0000","last_active":"2024-04-23T14:32:49.381590+0000","last_peered":"2024-04-23T14:32:49.381590+0000","last_clean":"2024-04-23T14:32:49.381590+0000","last_became_active":"2024-04-23T14:32:48.377976+0000","last_became_peered":"2024-04-23T14:32:48.377976+0000","last_unstale":"2024-04-23T14:32:49.381590+0000","last_undegraded":"2024-04-23T14:32:49.381590+0000","last_fullsized":"2024-04-23T14:32:49.381590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408098+0000","last_change":"2024-04-23T14:32:49.408098+0000","last_active":"2024-04-23T14:32:49.408098+0000","last_peered":"2024-04-23T14:32:49.408098+0000","last_clean":"2024-04-23T14:32:49.408098+0000","last_became_active":"2024-04-23T14:32:49.407785+0000","last_became_peered":"2024-04-23T14:32:49.407785+0000","last_unstale":"2024-04-23T14:32:49.408098+0000","last_undegraded":"2024-04-23T14:32:49.408098+0000","last_fullsized":"2024-04-23T14:32:49.408098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:12:49.629971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400540+0000","last_change":"2024-04-23T14:32:48.386689+0000","last_active":"2024-04-23T14:32:49.400540+0000","last_peered":"2024-04-23T14:32:49.400540+0000","last_clean":"2024-04-23T14:32:49.400540+0000","last_became_active":"2024-04-23T14:32:48.386272+0000","last_became_peered":"2024-04-23T14:32:48.386272+0000","last_unstale":"2024-04-23T14:32:49.400540+0000","last_undegraded":"2024-04-23T14:32:49.400540+0000","last_fullsized":"2024-04-23T14:32:49.400540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411036+0000","last_change":"2024-04-23T14:32:49.407292+0000","last_active":"2024-04-23T14:32:49.411036+0000","last_peered":"2024-04-23T14:32:49.411036+0000","last_clean":"2024-04-23T14:32:49.411036+0000","last_became_active":"2024-04-23T14:32:49.406967+0000","last_became_peered":"2024-04-23T14:32:49.406967+0000","last_unstale":"2024-04-23T14:32:49.411036+0000","last_undegraded":"2024-04-23T14:32:49.411036+0000","last_fullsized":"2024-04-23T14:32:49.411036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396038+0000","last_change":"2024-04-23T14:32:48.372538+0000","last_active":"2024-04-23T14:32:49.396038+0000","last_peered":"2024-04-23T14:32:49.396038+0000","last_clean":"2024-04-23T14:32:49.396038+0000","last_became_active":"2024-04-23T14:32:48.372276+0000","last_became_peered":"2024-04-23T14:32:48.372276+0000","last_unstale":"2024-04-23T14:32:49.396038+0000","last_undegraded":"2024-04-23T14:32:49.396038+0000","last_fullsized":"2024-04-23T14:32:49.396038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:57.601612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383866+0000","last_change":"2024-04-23T14:32:49.383866+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383866+0000","last_undegraded":"2024-04-23T14:32:49.383866+0000","last_fullsized":"2024-04-23T14:32:49.383866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397243+0000","last_change":"2024-04-23T14:32:48.373983+0000","last_active":"2024-04-23T14:32:49.397243+0000","last_peered":"2024-04-23T14:32:49.397243+0000","last_clean":"2024-04-23T14:32:49.397243+0000","last_became_active":"2024-04-23T14:32:48.373688+0000","last_became_peered":"2024-04-23T14:32:48.373688+0000","last_unstale":"2024-04-23T14:32:49.397243+0000","last_undegraded":"2024-04-23T14:32:49.397243+0000","last_fullsized":"2024-04-23T14:32:49.397243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411462+0000","last_change":"2024-04-23T14:32:49.398323+0000","last_active":"2024-04-23T14:32:49.411462+0000","last_peered":"2024-04-23T14:32:49.411462+0000","last_clean":"2024-04-23T14:32:49.411462+0000","last_became_active":"2024-04-23T14:32:49.398039+0000","last_became_peered":"2024-04-23T14:32:49.398039+0000","last_unstale":"2024-04-23T14:32:49.411462+0000","last_undegraded":"2024-04-23T14:32:49.411462+0000","last_fullsized":"2024-04-23T14:32:49.411462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396203+0000","last_change":"2024-04-23T14:32:48.384396+0000","last_active":"2024-04-23T14:32:49.396203+0000","last_peered":"2024-04-23T14:32:49.396203+0000","last_clean":"2024-04-23T14:32:49.396203+0000","last_became_active":"2024-04-23T14:32:48.383739+0000","last_became_peered":"2024-04-23T14:32:48.383739+0000","last_unstale":"2024-04-23T14:32:49.396203+0000","last_undegraded":"2024-04-23T14:32:49.396203+0000","last_fullsized":"2024-04-23T14:32:49.396203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401815+0000","last_change":"2024-04-23T14:32:49.396368+0000","last_active":"2024-04-23T14:32:49.401815+0000","last_peered":"2024-04-23T14:32:49.401815+0000","last_clean":"2024-04-23T14:32:49.401815+0000","last_became_active":"2024-04-23T14:32:49.395031+0000","last_became_peered":"2024-04-23T14:32:49.395031+0000","last_unstale":"2024-04-23T14:32:49.401815+0000","last_undegraded":"2024-04-23T14:32:49.401815+0000","last_fullsized":"2024-04-23T14:32:49.401815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397942+0000","last_change":"2024-04-23T14:32:48.387008+0000","last_active":"2024-04-23T14:32:49.397942+0000","last_peered":"2024-04-23T14:32:49.397942+0000","last_clean":"2024-04-23T14:32:49.397942+0000","last_became_active":"2024-04-23T14:32:48.386260+0000","last_became_peered":"2024-04-23T14:32:48.386260+0000","last_unstale":"2024-04-23T14:32:49.397942+0000","last_undegraded":"2024-04-23T14:32:49.397942+0000","last_fullsized":"2024-04-23T14:32:49.397942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.415060+0000","last_change":"2024-04-23T14:32:49.415060+0000","last_active":"2024-04-23T14:32:49.415060+0000","last_peered":"2024-04-23T14:32:49.415060+0000","last_clean":"2024-04-23T14:32:49.415060+0000","last_became_active":"2024-04-23T14:32:49.414707+0000","last_became_peered":"2024-04-23T14:32:49.414707+0000","last_unstale":"2024-04-23T14:32:49.415060+0000","last_undegraded":"2024-04-23T14:32:49.415060+0000","last_fullsized":"2024-04-23T14:32:49.415060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:02.059103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401039+0000","last_change":"2024-04-23T14:32:48.380600+0000","last_active":"2024-04-23T14:32:49.401039+0000","last_peered":"2024-04-23T14:32:49.401039+0000","last_clean":"2024-04-23T14:32:49.401039+0000","last_became_active":"2024-04-23T14:32:48.379900+0000","last_became_peered":"2024-04-23T14:32:48.379900+0000","last_unstale":"2024-04-23T14:32:49.401039+0000","last_undegraded":"2024-04-23T14:32:49.401039+0000","last_fullsized":"2024-04-23T14:32:49.401039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:31.916332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.425051+0000","last_change":"2024-04-23T14:32:49.417179+0000","last_active":"2024-04-23T14:32:49.425051+0000","last_peered":"2024-04-23T14:32:49.425051+0000","last_clean":"2024-04-23T14:32:49.425051+0000","last_became_active":"2024-04-23T14:32:49.414238+0000","last_became_peered":"2024-04-23T14:32:49.414238+0000","last_unstale":"2024-04-23T14:32:49.425051+0000","last_undegraded":"2024-04-23T14:32:49.425051+0000","last_fullsized":"2024-04-23T14:32:49.425051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375409+0000","last_change":"2024-04-23T14:32:48.382557+0000","last_active":"2024-04-23T14:32:49.375409+0000","last_peered":"2024-04-23T14:32:49.375409+0000","last_clean":"2024-04-23T14:32:49.375409+0000","last_became_active":"2024-04-23T14:32:48.381989+0000","last_became_peered":"2024-04-23T14:32:48.381989+0000","last_unstale":"2024-04-23T14:32:49.375409+0000","last_undegraded":"2024-04-23T14:32:49.375409+0000","last_fullsized":"2024-04-23T14:32:49.375409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411232+0000","last_change":"2024-04-23T14:32:49.406874+0000","last_active":"2024-04-23T14:32:49.411232+0000","last_peered":"2024-04-23T14:32:49.411232+0000","last_clean":"2024-04-23T14:32:49.411232+0000","last_became_active":"2024-04-23T14:32:49.406634+0000","last_became_peered":"2024-04-23T14:32:49.406634+0000","last_unstale":"2024-04-23T14:32:49.411232+0000","last_undegraded":"2024-04-23T14:32:49.411232+0000","last_fullsized":"2024-04-23T14:32:49.411232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380918+0000","last_change":"2024-04-23T14:32:48.381294+0000","last_active":"2024-04-23T14:32:49.380918+0000","last_peered":"2024-04-23T14:32:49.380918+0000","last_clean":"2024-04-23T14:32:49.380918+0000","last_became_active":"2024-04-23T14:32:48.380990+0000","last_became_peered":"2024-04-23T14:32:48.380990+0000","last_unstale":"2024-04-23T14:32:49.380918+0000","last_undegraded":"2024-04-23T14:32:49.380918+0000","last_fullsized":"2024-04-23T14:32:49.380918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391358+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:32:49.391358+0000","last_peered":"2024-04-23T14:32:49.391358+0000","last_clean":"2024-04-23T14:32:49.391358+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:32:49.391358+0000","last_undegraded":"2024-04-23T14:32:49.391358+0000","last_fullsized":"2024-04-23T14:32:49.391358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:53.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.121+0000 7fd6abe00640 1 Processor -- start 2024-04-23T14:32:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.122+0000 7fd6abe00640 1 -- start start 2024-04-23T14:32:54.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.123+0000 7fd6abe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd6ac0d0600 0x7fd6ac06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.123+0000 7fd6abe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac0d0c70 con 0x7fd6ac15c700 2024-04-23T14:32:54.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6abe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac0d0f50 con 0x7fd6ac06bf10 2024-04-23T14:32:54.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6abe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ac06c360 con 0x7fd6ac0d0600 2024-04-23T14:32:54.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6aaa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd6ac15c700 0x7fd6ac160d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35166/0 (socket says 172.21.15.59:35166) 2024-04-23T14:32:54.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6aaa00640 1 -- 172.21.15.59:0/405545633 learned_addr learned my addr 172.21.15.59:0/405545633 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:54.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6a3e00640 1 --2- 172.21.15.59:0/405545633 >> v2:172.21.15.59:3300/0 conn(0x7fd6ac0d0600 0x7fd6ac06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6aa000640 1 -- 172.21.15.59:0/405545633 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942308307 0 0) 0x7fd6ac0d0c70 con 0x7fd6ac15c700 2024-04-23T14:32:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6aa000640 1 -- 172.21.15.59:0/405545633 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd690003520 con 0x7fd6ac15c700 2024-04-23T14:32:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6a3e00640 1 -- 172.21.15.59:0/405545633 >> v1:172.21.15.76:6789/0 conn(0x7fd6ac06bf10 legacy=0x7fd6ac0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6a3e00640 1 -- 172.21.15.59:0/405545633 >> v1:172.21.15.59:6789/0 conn(0x7fd6ac15c700 legacy=0x7fd6ac160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.124+0000 7fd6a3e00640 1 -- 172.21.15.59:0/405545633 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac0b4340 con 0x7fd6ac0d0600 2024-04-23T14:32:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.125+0000 7fd6a3e00640 1 --2- 172.21.15.59:0/405545633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac0d0600 0x7fd6ac06b9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd6940097a0 tx=0x7fd69402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5df16699b24d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6aa000640 1 -- 172.21.15.59:0/405545633 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694037070 con 0x7fd6ac0d0600 2024-04-23T14:32:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6aa000640 1 -- 172.21.15.59:0/405545633 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd694032440 con 0x7fd6ac0d0600 2024-04-23T14:32:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6aa000640 1 -- 172.21.15.59:0/405545633 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6940327f0 con 0x7fd6ac0d0600 2024-04-23T14:32:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6abe00640 1 -- 172.21.15.59:0/405545633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac0d0600 msgr2=0x7fd6ac06b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6abe00640 1 --2- 172.21.15.59:0/405545633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac0d0600 0x7fd6ac06b9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd6940097a0 tx=0x7fd69402a230 comp rx=0 tx=0).stop 2024-04-23T14:32:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6abe00640 1 -- 172.21.15.59:0/405545633 shutdown_connections 2024-04-23T14:32:54.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6abe00640 1 --2- 172.21.15.59:0/405545633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac0d0600 0x7fd6ac06b9d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.126+0000 7fd6abe00640 1 -- 172.21.15.59:0/405545633 >> 172.21.15.59:0/405545633 conn(0x7fd6ac09aa00 msgr2=0x7fd6ac0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:54.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.127+0000 7fd6abe00640 1 -- 172.21.15.59:0/405545633 shutdown_connections 2024-04-23T14:32:54.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.127+0000 7fd6abe00640 1 -- 172.21.15.59:0/405545633 wait complete. 2024-04-23T14:32:54.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.129+0000 7fd6abe00640 1 Processor -- start 2024-04-23T14:32:54.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.129+0000 7fd6abe00640 1 -- start start 2024-04-23T14:32:54.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.130+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6ac06bf10 0x7fd6ac15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.130+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6ac0d0600 0x7fd6ac0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.130+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac15c700 0x7fd6ac0c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.130+0000 7fd6abe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac0b5250 con 0x7fd6ac15c700 2024-04-23T14:32:54.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.130+0000 7fd6abe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac0b46b0 con 0x7fd6ac0d0600 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.130+0000 7fd6abe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ac0b53d0 con 0x7fd6ac06bf10 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac15c700 0x7fd6ac0c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac15c700 0x7fd6ac0c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45596/0 (socket says 172.21.15.59:45596) 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 -- 172.21.15.59:0/2526606986 learned_addr learned my addr 172.21.15.59:0/2526606986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6a3e00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6ac0d0600 0x7fd6ac0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6aaa00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6ac06bf10 0x7fd6ac15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 -- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6ac06bf10 msgr2=0x7fd6ac15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6ac06bf10 0x7fd6ac15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 -- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6ac0d0600 msgr2=0x7fd6ac0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6ac0d0600 0x7fd6ac0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.131+0000 7fd6ab400640 1 -- 172.21.15.59:0/2526606986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac1e6320 con 0x7fd6ac15c700 2024-04-23T14:32:54.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.132+0000 7fd6ab400640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac15c700 0x7fd6ac0c6b60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c710 tx=0x7fd6a400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.132+0000 7fd6a2a00640 1 -- 172.21.15.59:0/2526606986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4016020 con 0x7fd6ac15c700 2024-04-23T14:32:54.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.132+0000 7fd6a2a00640 1 -- 172.21.15.59:0/2526606986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a4006d50 con 0x7fd6ac15c700 2024-04-23T14:32:54.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.132+0000 7fd6a2a00640 1 -- 172.21.15.59:0/2526606986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4003bb0 con 0x7fd6ac15c700 2024-04-23T14:32:54.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.132+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac1e6600 con 0x7fd6ac15c700 2024-04-23T14:32:54.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.132+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac1e6bc0 con 0x7fd6ac15c700 2024-04-23T14:32:54.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.134+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac04fd90 con 0x7fd6ac15c700 2024-04-23T14:32:54.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.135+0000 7fd6a2a00640 1 -- 172.21.15.59:0/2526606986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6a400a850 con 0x7fd6ac15c700 2024-04-23T14:32:54.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.136+0000 7fd6a2a00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd68806c920 0x7fd68806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.136+0000 7fd6a2a00640 1 -- 172.21.15.59:0/2526606986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd6a408b810 con 0x7fd6ac15c700 2024-04-23T14:32:54.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.136+0000 7fd6aaa00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd68806c920 0x7fd68806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.137+0000 7fd6aaa00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd68806c920 0x7fd68806ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0011e0 tx=0x7fd69c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.142+0000 7fd6a2a00640 1 -- 172.21.15.59:0/2526606986 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6a4020020 con 0x7fd6ac15c700 2024-04-23T14:32:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.334+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6ac0b7260 con 0x7fd68806c920 2024-04-23T14:32:54.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.353+0000 7fd6a2a00640 1 -- 172.21.15.59:0/2526606986 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251346 (secure 0 0 0) 0x7fd6ac0b7260 con 0x7fd68806c920 2024-04-23T14:32:54.358 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:54.359 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd68806c920 msgr2=0x7fd68806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd68806c920 0x7fd68806ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0011e0 tx=0x7fd69c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac15c700 msgr2=0x7fd6ac0c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac15c700 0x7fd6ac0c6b60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c710 tx=0x7fd6a400cbd0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 shutdown_connections 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd68806c920 0x7fd68806ede0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6ac15c700 0x7fd6ac0c6b60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6ac0d0600 0x7fd6ac0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 --2- 172.21.15.59:0/2526606986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6ac06bf10 0x7fd6ac15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.359+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 >> 172.21.15.59:0/2526606986 conn(0x7fd6ac09aa00 msgr2=0x7fd6ac0ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.360+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 shutdown_connections 2024-04-23T14:32:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.360+0000 7fd6abe00640 1 -- 172.21.15.59:0/2526606986 wait complete. 2024-04-23T14:32:54.379 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-23T14:32:52.370386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224208,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750512,"statfs":{"total":773094113280,"available":772864524288,"internally_reserved":0,"allocated":3973120,"data_stored":2249000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.029950"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408276+0000","last_change":"2024-04-23T14:32:49.396796+0000","last_active":"2024-04-23T14:32:49.408276+0000","last_peered":"2024-04-23T14:32:49.408276+0000","last_clean":"2024-04-23T14:32:49.408276+0000","last_became_active":"2024-04-23T14:32:49.396505+0000","last_became_peered":"2024-04-23T14:32:49.396505+0000","last_unstale":"2024-04-23T14:32:49.408276+0000","last_undegraded":"2024-04-23T14:32:49.408276+0000","last_fullsized":"2024-04-23T14:32:49.408276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407907+0000","last_change":"2024-04-23T14:32:49.396404+0000","last_active":"2024-04-23T14:32:49.407907+0000","last_peered":"2024-04-23T14:32:49.407907+0000","last_clean":"2024-04-23T14:32:49.407907+0000","last_became_active":"2024-04-23T14:32:49.396103+0000","last_became_peered":"2024-04-23T14:32:49.396103+0000","last_unstale":"2024-04-23T14:32:49.407907+0000","last_undegraded":"2024-04-23T14:32:49.407907+0000","last_fullsized":"2024-04-23T14:32:49.407907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.382675+0000","last_change":"2024-04-23T14:32:49.382675+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.382675+0000","last_undegraded":"2024-04-23T14:32:49.382675+0000","last_fullsized":"2024-04-23T14:32:49.382675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418842+0000","last_change":"2024-04-23T14:32:49.412161+0000","last_active":"2024-04-23T14:32:49.418842+0000","last_peered":"2024-04-23T14:32:49.418842+0000","last_clean":"2024-04-23T14:32:49.418842+0000","last_became_active":"2024-04-23T14:32:49.411609+0000","last_became_peered":"2024-04-23T14:32:49.411609+0000","last_unstale":"2024-04-23T14:32:49.418842+0000","last_undegraded":"2024-04-23T14:32:49.418842+0000","last_fullsized":"2024-04-23T14:32:49.418842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411075+0000","last_change":"2024-04-23T14:32:49.411075+0000","last_active":"2024-04-23T14:32:49.411075+0000","last_peered":"2024-04-23T14:32:49.411075+0000","last_clean":"2024-04-23T14:32:49.411075+0000","last_became_active":"2024-04-23T14:32:49.410648+0000","last_became_peered":"2024-04-23T14:32:49.410648+0000","last_unstale":"2024-04-23T14:32:49.411075+0000","last_undegraded":"2024-04-23T14:32:49.411075+0000","last_fullsized":"2024-04-23T14:32:49.411075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:19.233384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412676+0000","last_change":"2024-04-23T14:32:49.396761+0000","last_active":"2024-04-23T14:32:49.412676+0000","last_peered":"2024-04-23T14:32:49.412676+0000","last_clean":"2024-04-23T14:32:49.412676+0000","last_became_active":"2024-04-23T14:32:49.396598+0000","last_became_peered":"2024-04-23T14:32:49.396598+0000","last_unstale":"2024-04-23T14:32:49.412676+0000","last_undegraded":"2024-04-23T14:32:49.412676+0000","last_fullsized":"2024-04-23T14:32:49.412676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407019+0000","last_change":"2024-04-23T14:32:49.398865+0000","last_active":"2024-04-23T14:32:49.407019+0000","last_peered":"2024-04-23T14:32:49.407019+0000","last_clean":"2024-04-23T14:32:49.407019+0000","last_became_active":"2024-04-23T14:32:49.398581+0000","last_became_peered":"2024-04-23T14:32:49.398581+0000","last_unstale":"2024-04-23T14:32:49.407019+0000","last_undegraded":"2024-04-23T14:32:49.407019+0000","last_fullsized":"2024-04-23T14:32:49.407019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409232+0000","last_change":"2024-04-23T14:32:49.409232+0000","last_active":"2024-04-23T14:32:49.409232+0000","last_peered":"2024-04-23T14:32:49.409232+0000","last_clean":"2024-04-23T14:32:49.409232+0000","last_became_active":"2024-04-23T14:32:49.408162+0000","last_became_peered":"2024-04-23T14:32:49.408162+0000","last_unstale":"2024-04-23T14:32:49.409232+0000","last_undegraded":"2024-04-23T14:32:49.409232+0000","last_fullsized":"2024-04-23T14:32:49.409232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.424075+0000","last_change":"2024-04-23T14:32:49.424075+0000","last_active":"2024-04-23T14:32:49.424075+0000","last_peered":"2024-04-23T14:32:49.424075+0000","last_clean":"2024-04-23T14:32:49.424075+0000","last_became_active":"2024-04-23T14:32:49.423714+0000","last_became_peered":"2024-04-23T14:32:49.423714+0000","last_unstale":"2024-04-23T14:32:49.424075+0000","last_undegraded":"2024-04-23T14:32:49.424075+0000","last_fullsized":"2024-04-23T14:32:49.424075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421421+0000","last_change":"2024-04-23T14:32:49.419419+0000","last_active":"2024-04-23T14:32:49.421421+0000","last_peered":"2024-04-23T14:32:49.421421+0000","last_clean":"2024-04-23T14:32:49.421421+0000","last_became_active":"2024-04-23T14:32:49.416916+0000","last_became_peered":"2024-04-23T14:32:49.416916+0000","last_unstale":"2024-04-23T14:32:49.421421+0000","last_undegraded":"2024-04-23T14:32:49.421421+0000","last_fullsized":"2024-04-23T14:32:49.421421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389894+0000","last_change":"2024-04-23T14:32:49.389894+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389894+0000","last_undegraded":"2024-04-23T14:32:49.389894+0000","last_fullsized":"2024-04-23T14:32:49.389894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414777+0000","last_change":"2024-04-23T14:32:49.414777+0000","last_active":"2024-04-23T14:32:49.414777+0000","last_peered":"2024-04-23T14:32:49.414777+0000","last_clean":"2024-04-23T14:32:49.414777+0000","last_became_active":"2024-04-23T14:32:49.414208+0000","last_became_peered":"2024-04-23T14:32:49.414208+0000","last_unstale":"2024-04-23T14:32:49.414777+0000","last_undegraded":"2024-04-23T14:32:49.414777+0000","last_fullsized":"2024-04-23T14:32:49.414777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389314+0000","last_change":"2024-04-23T14:32:49.389314+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389314+0000","last_undegraded":"2024-04-23T14:32:49.389314+0000","last_fullsized":"2024-04-23T14:32:49.389314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409664+0000","last_change":"2024-04-23T14:32:49.409664+0000","last_active":"2024-04-23T14:32:49.409664+0000","last_peered":"2024-04-23T14:32:49.409664+0000","last_clean":"2024-04-23T14:32:49.409664+0000","last_became_active":"2024-04-23T14:32:49.409369+0000","last_became_peered":"2024-04-23T14:32:49.409369+0000","last_unstale":"2024-04-23T14:32:49.409664+0000","last_undegraded":"2024-04-23T14:32:49.409664+0000","last_fullsized":"2024-04-23T14:32:49.409664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:53.618947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404125+0000","last_change":"2024-04-23T14:32:49.395881+0000","last_active":"2024-04-23T14:32:49.404125+0000","last_peered":"2024-04-23T14:32:49.404125+0000","last_clean":"2024-04-23T14:32:49.404125+0000","last_became_active":"2024-04-23T14:32:49.395640+0000","last_became_peered":"2024-04-23T14:32:49.395640+0000","last_unstale":"2024-04-23T14:32:49.404125+0000","last_undegraded":"2024-04-23T14:32:49.404125+0000","last_fullsized":"2024-04-23T14:32:49.404125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410041+0000","last_change":"2024-04-23T14:32:49.410041+0000","last_active":"2024-04-23T14:32:49.410041+0000","last_peered":"2024-04-23T14:32:49.410041+0000","last_clean":"2024-04-23T14:32:49.410041+0000","last_became_active":"2024-04-23T14:32:49.409759+0000","last_became_peered":"2024-04-23T14:32:49.409759+0000","last_unstale":"2024-04-23T14:32:49.410041+0000","last_undegraded":"2024-04-23T14:32:49.410041+0000","last_fullsized":"2024-04-23T14:32:49.410041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408902+0000","last_change":"2024-04-23T14:32:49.405810+0000","last_active":"2024-04-23T14:32:49.408902+0000","last_peered":"2024-04-23T14:32:49.408902+0000","last_clean":"2024-04-23T14:32:49.408902+0000","last_became_active":"2024-04-23T14:32:49.405283+0000","last_became_peered":"2024-04-23T14:32:49.405283+0000","last_unstale":"2024-04-23T14:32:49.408902+0000","last_undegraded":"2024-04-23T14:32:49.408902+0000","last_fullsized":"2024-04-23T14:32:49.408902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.379405+0000","last_change":"2024-04-23T14:32:49.379405+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.379405+0000","last_undegraded":"2024-04-23T14:32:49.379405+0000","last_fullsized":"2024-04-23T14:32:49.379405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.378840+0000","last_change":"2024-04-23T14:32:49.378840+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.378840+0000","last_undegraded":"2024-04-23T14:32:49.378840+0000","last_fullsized":"2024-04-23T14:32:49.378840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408152+0000","last_change":"2024-04-23T14:32:49.408152+0000","last_active":"2024-04-23T14:32:49.408152+0000","last_peered":"2024-04-23T14:32:49.408152+0000","last_clean":"2024-04-23T14:32:49.408152+0000","last_became_active":"2024-04-23T14:32:49.407758+0000","last_became_peered":"2024-04-23T14:32:49.407758+0000","last_unstale":"2024-04-23T14:32:49.408152+0000","last_undegraded":"2024-04-23T14:32:49.408152+0000","last_fullsized":"2024-04-23T14:32:49.408152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:42.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416792+0000","last_change":"2024-04-23T14:32:49.413362+0000","last_active":"2024-04-23T14:32:49.416792+0000","last_peered":"2024-04-23T14:32:49.416792+0000","last_clean":"2024-04-23T14:32:49.416792+0000","last_became_active":"2024-04-23T14:32:49.413082+0000","last_became_peered":"2024-04-23T14:32:49.413082+0000","last_unstale":"2024-04-23T14:32:49.416792+0000","last_undegraded":"2024-04-23T14:32:49.416792+0000","last_fullsized":"2024-04-23T14:32:49.416792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:57.535200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417468+0000","last_change":"2024-04-23T14:32:49.414460+0000","last_active":"2024-04-23T14:32:49.417468+0000","last_peered":"2024-04-23T14:32:49.417468+0000","last_clean":"2024-04-23T14:32:49.417468+0000","last_became_active":"2024-04-23T14:32:49.413658+0000","last_became_peered":"2024-04-23T14:32:49.413658+0000","last_unstale":"2024-04-23T14:32:49.417468+0000","last_undegraded":"2024-04-23T14:32:49.417468+0000","last_fullsized":"2024-04-23T14:32:49.417468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:32.704139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412931+0000","last_change":"2024-04-23T14:32:49.404657+0000","last_active":"2024-04-23T14:32:49.412931+0000","last_peered":"2024-04-23T14:32:49.412931+0000","last_clean":"2024-04-23T14:32:49.412931+0000","last_became_active":"2024-04-23T14:32:49.404322+0000","last_became_peered":"2024-04-23T14:32:49.404322+0000","last_unstale":"2024-04-23T14:32:49.412931+0000","last_undegraded":"2024-04-23T14:32:49.412931+0000","last_fullsized":"2024-04-23T14:32:49.412931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416596+0000","last_change":"2024-04-23T14:32:49.408981+0000","last_active":"2024-04-23T14:32:49.416596+0000","last_peered":"2024-04-23T14:32:49.416596+0000","last_clean":"2024-04-23T14:32:49.416596+0000","last_became_active":"2024-04-23T14:32:49.408732+0000","last_became_peered":"2024-04-23T14:32:49.408732+0000","last_unstale":"2024-04-23T14:32:49.416596+0000","last_undegraded":"2024-04-23T14:32:49.416596+0000","last_fullsized":"2024-04-23T14:32:49.416596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419806+0000","last_change":"2024-04-23T14:32:49.411802+0000","last_active":"2024-04-23T14:32:49.419806+0000","last_peered":"2024-04-23T14:32:49.419806+0000","last_clean":"2024-04-23T14:32:49.419806+0000","last_became_active":"2024-04-23T14:32:49.411472+0000","last_became_peered":"2024-04-23T14:32:49.411472+0000","last_unstale":"2024-04-23T14:32:49.419806+0000","last_undegraded":"2024-04-23T14:32:49.419806+0000","last_fullsized":"2024-04-23T14:32:49.419806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408321+0000","last_change":"2024-04-23T14:32:49.408321+0000","last_active":"2024-04-23T14:32:49.408321+0000","last_peered":"2024-04-23T14:32:49.408321+0000","last_clean":"2024-04-23T14:32:49.408321+0000","last_became_active":"2024-04-23T14:32:49.407885+0000","last_became_peered":"2024-04-23T14:32:49.407885+0000","last_unstale":"2024-04-23T14:32:49.408321+0000","last_undegraded":"2024-04-23T14:32:49.408321+0000","last_fullsized":"2024-04-23T14:32:49.408321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420428+0000","last_change":"2024-04-23T14:32:49.418183+0000","last_active":"2024-04-23T14:32:49.420428+0000","last_peered":"2024-04-23T14:32:49.420428+0000","last_clean":"2024-04-23T14:32:49.420428+0000","last_became_active":"2024-04-23T14:32:49.417903+0000","last_became_peered":"2024-04-23T14:32:49.417903+0000","last_unstale":"2024-04-23T14:32:49.420428+0000","last_undegraded":"2024-04-23T14:32:49.420428+0000","last_fullsized":"2024-04-23T14:32:49.420428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404038+0000","last_change":"2024-04-23T14:32:49.396858+0000","last_active":"2024-04-23T14:32:49.404038+0000","last_peered":"2024-04-23T14:32:49.404038+0000","last_clean":"2024-04-23T14:32:49.404038+0000","last_became_active":"2024-04-23T14:32:49.396565+0000","last_became_peered":"2024-04-23T14:32:49.396565+0000","last_unstale":"2024-04-23T14:32:49.404038+0000","last_undegraded":"2024-04-23T14:32:49.404038+0000","last_fullsized":"2024-04-23T14:32:49.404038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419860+0000","last_change":"2024-04-23T14:32:49.419860+0000","last_active":"2024-04-23T14:32:49.419860+0000","last_peered":"2024-04-23T14:32:49.419860+0000","last_clean":"2024-04-23T14:32:49.419860+0000","last_became_active":"2024-04-23T14:32:49.419578+0000","last_became_peered":"2024-04-23T14:32:49.419578+0000","last_unstale":"2024-04-23T14:32:49.419860+0000","last_undegraded":"2024-04-23T14:32:49.419860+0000","last_fullsized":"2024-04-23T14:32:49.419860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407177+0000","last_change":"2024-04-23T14:32:49.403367+0000","last_active":"2024-04-23T14:32:49.407177+0000","last_peered":"2024-04-23T14:32:49.407177+0000","last_clean":"2024-04-23T14:32:49.407177+0000","last_became_active":"2024-04-23T14:32:49.403043+0000","last_became_peered":"2024-04-23T14:32:49.403043+0000","last_unstale":"2024-04-23T14:32:49.407177+0000","last_undegraded":"2024-04-23T14:32:49.407177+0000","last_fullsized":"2024-04-23T14:32:49.407177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413518+0000","last_change":"2024-04-23T14:32:49.407909+0000","last_active":"2024-04-23T14:32:49.413518+0000","last_peered":"2024-04-23T14:32:49.413518+0000","last_clean":"2024-04-23T14:32:49.413518+0000","last_became_active":"2024-04-23T14:32:49.407653+0000","last_became_peered":"2024-04-23T14:32:49.407653+0000","last_unstale":"2024-04-23T14:32:49.413518+0000","last_undegraded":"2024-04-23T14:32:49.413518+0000","last_fullsized":"2024-04-23T14:32:49.413518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412291+0000","last_change":"2024-04-23T14:32:49.412291+0000","last_active":"2024-04-23T14:32:49.412291+0000","last_peered":"2024-04-23T14:32:49.412291+0000","last_clean":"2024-04-23T14:32:49.412291+0000","last_became_active":"2024-04-23T14:32:49.411966+0000","last_became_peered":"2024-04-23T14:32:49.411966+0000","last_unstale":"2024-04-23T14:32:49.412291+0000","last_undegraded":"2024-04-23T14:32:49.412291+0000","last_fullsized":"2024-04-23T14:32:49.412291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401707+0000","last_change":"2024-04-23T14:32:49.398365+0000","last_active":"2024-04-23T14:32:49.401707+0000","last_peered":"2024-04-23T14:32:49.401707+0000","last_clean":"2024-04-23T14:32:49.401707+0000","last_became_active":"2024-04-23T14:32:49.398036+0000","last_became_peered":"2024-04-23T14:32:49.398036+0000","last_unstale":"2024-04-23T14:32:49.401707+0000","last_undegraded":"2024-04-23T14:32:49.401707+0000","last_fullsized":"2024-04-23T14:32:49.401707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.380750+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.380399+0000","last_became_peered":"2024-04-23T14:32:48.380399+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:31.689650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414566+0000","last_change":"2024-04-23T14:32:49.411206+0000","last_active":"2024-04-23T14:32:49.414566+0000","last_peered":"2024-04-23T14:32:49.414566+0000","last_clean":"2024-04-23T14:32:49.414566+0000","last_became_active":"2024-04-23T14:32:49.410469+0000","last_became_peered":"2024-04-23T14:32:49.410469+0000","last_unstale":"2024-04-23T14:32:49.414566+0000","last_undegraded":"2024-04-23T14:32:49.414566+0000","last_fullsized":"2024-04-23T14:32:49.414566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395347+0000","last_change":"2024-04-23T14:32:48.378555+0000","last_active":"2024-04-23T14:32:49.395347+0000","last_peered":"2024-04-23T14:32:49.395347+0000","last_clean":"2024-04-23T14:32:49.395347+0000","last_became_active":"2024-04-23T14:32:48.377594+0000","last_became_peered":"2024-04-23T14:32:48.377594+0000","last_unstale":"2024-04-23T14:32:49.395347+0000","last_undegraded":"2024-04-23T14:32:49.395347+0000","last_fullsized":"2024-04-23T14:32:49.395347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418168+0000","last_change":"2024-04-23T14:32:49.418168+0000","last_active":"2024-04-23T14:32:49.418168+0000","last_peered":"2024-04-23T14:32:49.418168+0000","last_clean":"2024-04-23T14:32:49.418168+0000","last_became_active":"2024-04-23T14:32:49.417542+0000","last_became_peered":"2024-04-23T14:32:49.417542+0000","last_unstale":"2024-04-23T14:32:49.418168+0000","last_undegraded":"2024-04-23T14:32:49.418168+0000","last_fullsized":"2024-04-23T14:32:49.418168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381457+0000","last_change":"2024-04-23T14:32:48.383157+0000","last_active":"2024-04-23T14:32:49.381457+0000","last_peered":"2024-04-23T14:32:49.381457+0000","last_clean":"2024-04-23T14:32:49.381457+0000","last_became_active":"2024-04-23T14:32:48.382568+0000","last_became_peered":"2024-04-23T14:32:48.382568+0000","last_unstale":"2024-04-23T14:32:49.381457+0000","last_undegraded":"2024-04-23T14:32:49.381457+0000","last_fullsized":"2024-04-23T14:32:49.381457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:35.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420450+0000","last_change":"2024-04-23T14:32:49.418053+0000","last_active":"2024-04-23T14:32:49.420450+0000","last_peered":"2024-04-23T14:32:49.420450+0000","last_clean":"2024-04-23T14:32:49.420450+0000","last_became_active":"2024-04-23T14:32:49.417733+0000","last_became_peered":"2024-04-23T14:32:49.417733+0000","last_unstale":"2024-04-23T14:32:49.420450+0000","last_undegraded":"2024-04-23T14:32:49.420450+0000","last_fullsized":"2024-04-23T14:32:49.420450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391055+0000","last_change":"2024-04-23T14:32:48.374028+0000","last_active":"2024-04-23T14:32:49.391055+0000","last_peered":"2024-04-23T14:32:49.391055+0000","last_clean":"2024-04-23T14:32:49.391055+0000","last_became_active":"2024-04-23T14:32:48.372871+0000","last_became_peered":"2024-04-23T14:32:48.372871+0000","last_unstale":"2024-04-23T14:32:49.391055+0000","last_undegraded":"2024-04-23T14:32:49.391055+0000","last_fullsized":"2024-04-23T14:32:49.391055+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:14.297285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410412+0000","last_change":"2024-04-23T14:32:49.410412+0000","last_active":"2024-04-23T14:32:49.410412+0000","last_peered":"2024-04-23T14:32:49.410412+0000","last_clean":"2024-04-23T14:32:49.410412+0000","last_became_active":"2024-04-23T14:32:49.409645+0000","last_became_peered":"2024-04-23T14:32:49.409645+0000","last_unstale":"2024-04-23T14:32:49.410412+0000","last_undegraded":"2024-04-23T14:32:49.410412+0000","last_fullsized":"2024-04-23T14:32:49.410412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401440+0000","last_change":"2024-04-23T14:32:48.381407+0000","last_active":"2024-04-23T14:32:49.401440+0000","last_peered":"2024-04-23T14:32:49.401440+0000","last_clean":"2024-04-23T14:32:49.401440+0000","last_became_active":"2024-04-23T14:32:48.381122+0000","last_became_peered":"2024-04-23T14:32:48.381122+0000","last_unstale":"2024-04-23T14:32:49.401440+0000","last_undegraded":"2024-04-23T14:32:49.401440+0000","last_fullsized":"2024-04-23T14:32:49.401440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:16:40.112260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408197+0000","last_change":"2024-04-23T14:32:49.400326+0000","last_active":"2024-04-23T14:32:49.408197+0000","last_peered":"2024-04-23T14:32:49.408197+0000","last_clean":"2024-04-23T14:32:49.408197+0000","last_became_active":"2024-04-23T14:32:49.399716+0000","last_became_peered":"2024-04-23T14:32:49.399716+0000","last_unstale":"2024-04-23T14:32:49.408197+0000","last_undegraded":"2024-04-23T14:32:49.408197+0000","last_fullsized":"2024-04-23T14:32:49.408197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401159+0000","last_change":"2024-04-23T14:32:48.380347+0000","last_active":"2024-04-23T14:32:49.401159+0000","last_peered":"2024-04-23T14:32:49.401159+0000","last_clean":"2024-04-23T14:32:49.401159+0000","last_became_active":"2024-04-23T14:32:48.379799+0000","last_became_peered":"2024-04-23T14:32:48.379799+0000","last_unstale":"2024-04-23T14:32:49.401159+0000","last_undegraded":"2024-04-23T14:32:49.401159+0000","last_fullsized":"2024-04-23T14:32:49.401159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:13.096562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421993+0000","last_change":"2024-04-23T14:32:49.415576+0000","last_active":"2024-04-23T14:32:49.421993+0000","last_peered":"2024-04-23T14:32:49.421993+0000","last_clean":"2024-04-23T14:32:49.421993+0000","last_became_active":"2024-04-23T14:32:49.415107+0000","last_became_peered":"2024-04-23T14:32:49.415107+0000","last_unstale":"2024-04-23T14:32:49.421993+0000","last_undegraded":"2024-04-23T14:32:49.421993+0000","last_fullsized":"2024-04-23T14:32:49.421993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:21.969136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375434+0000","last_change":"2024-04-23T14:32:48.378186+0000","last_active":"2024-04-23T14:32:49.375434+0000","last_peered":"2024-04-23T14:32:49.375434+0000","last_clean":"2024-04-23T14:32:49.375434+0000","last_became_active":"2024-04-23T14:32:48.377889+0000","last_became_peered":"2024-04-23T14:32:48.377889+0000","last_unstale":"2024-04-23T14:32:49.375434+0000","last_undegraded":"2024-04-23T14:32:49.375434+0000","last_fullsized":"2024-04-23T14:32:49.375434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420363+0000","last_change":"2024-04-23T14:32:49.418062+0000","last_active":"2024-04-23T14:32:49.420363+0000","last_peered":"2024-04-23T14:32:49.420363+0000","last_clean":"2024-04-23T14:32:49.420363+0000","last_became_active":"2024-04-23T14:32:49.417335+0000","last_became_peered":"2024-04-23T14:32:49.417335+0000","last_unstale":"2024-04-23T14:32:49.420363+0000","last_undegraded":"2024-04-23T14:32:49.420363+0000","last_fullsized":"2024-04-23T14:32:49.420363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401048+0000","last_change":"2024-04-23T14:32:48.376375+0000","last_active":"2024-04-23T14:32:49.401048+0000","last_peered":"2024-04-23T14:32:49.401048+0000","last_clean":"2024-04-23T14:32:49.401048+0000","last_became_active":"2024-04-23T14:32:48.375807+0000","last_became_peered":"2024-04-23T14:32:48.375807+0000","last_unstale":"2024-04-23T14:32:49.401048+0000","last_undegraded":"2024-04-23T14:32:49.401048+0000","last_fullsized":"2024-04-23T14:32:49.401048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:21.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414650+0000","last_change":"2024-04-23T14:32:49.414650+0000","last_active":"2024-04-23T14:32:49.414650+0000","last_peered":"2024-04-23T14:32:49.414650+0000","last_clean":"2024-04-23T14:32:49.414650+0000","last_became_active":"2024-04-23T14:32:49.414352+0000","last_became_peered":"2024-04-23T14:32:49.414352+0000","last_unstale":"2024-04-23T14:32:49.414650+0000","last_undegraded":"2024-04-23T14:32:49.414650+0000","last_fullsized":"2024-04-23T14:32:49.414650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380697+0000","last_change":"2024-04-23T14:32:48.380295+0000","last_active":"2024-04-23T14:32:49.380697+0000","last_peered":"2024-04-23T14:32:49.380697+0000","last_clean":"2024-04-23T14:32:49.380697+0000","last_became_active":"2024-04-23T14:32:48.379968+0000","last_became_peered":"2024-04-23T14:32:48.379968+0000","last_unstale":"2024-04-23T14:32:49.380697+0000","last_undegraded":"2024-04-23T14:32:49.380697+0000","last_fullsized":"2024-04-23T14:32:49.380697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:28.545681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408527+0000","last_change":"2024-04-23T14:32:49.408527+0000","last_active":"2024-04-23T14:32:49.408527+0000","last_peered":"2024-04-23T14:32:49.408527+0000","last_clean":"2024-04-23T14:32:49.408527+0000","last_became_active":"2024-04-23T14:32:49.408214+0000","last_became_peered":"2024-04-23T14:32:49.408214+0000","last_unstale":"2024-04-23T14:32:49.408527+0000","last_undegraded":"2024-04-23T14:32:49.408527+0000","last_fullsized":"2024-04-23T14:32:49.408527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395886+0000","last_change":"2024-04-23T14:32:48.369769+0000","last_active":"2024-04-23T14:32:49.395886+0000","last_peered":"2024-04-23T14:32:49.395886+0000","last_clean":"2024-04-23T14:32:49.395886+0000","last_became_active":"2024-04-23T14:32:48.369112+0000","last_became_peered":"2024-04-23T14:32:48.369112+0000","last_unstale":"2024-04-23T14:32:49.395886+0000","last_undegraded":"2024-04-23T14:32:49.395886+0000","last_fullsized":"2024-04-23T14:32:49.395886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:14.196347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409092+0000","last_change":"2024-04-23T14:32:49.409092+0000","last_active":"2024-04-23T14:32:49.409092+0000","last_peered":"2024-04-23T14:32:49.409092+0000","last_clean":"2024-04-23T14:32:49.409092+0000","last_became_active":"2024-04-23T14:32:49.408769+0000","last_became_peered":"2024-04-23T14:32:49.408769+0000","last_unstale":"2024-04-23T14:32:49.409092+0000","last_undegraded":"2024-04-23T14:32:49.409092+0000","last_fullsized":"2024-04-23T14:32:49.409092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:07.688839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.383330+0000","last_change":"2024-04-23T14:32:48.379746+0000","last_active":"2024-04-23T14:32:49.383330+0000","last_peered":"2024-04-23T14:32:49.383330+0000","last_clean":"2024-04-23T14:32:49.383330+0000","last_became_active":"2024-04-23T14:32:48.378760+0000","last_became_peered":"2024-04-23T14:32:48.378760+0000","last_unstale":"2024-04-23T14:32:49.383330+0000","last_undegraded":"2024-04-23T14:32:49.383330+0000","last_fullsized":"2024-04-23T14:32:49.383330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383029+0000","last_change":"2024-04-23T14:32:49.383029+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383029+0000","last_undegraded":"2024-04-23T14:32:49.383029+0000","last_fullsized":"2024-04-23T14:32:49.383029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.378297+0000","last_change":"2024-04-23T14:32:48.379776+0000","last_active":"2024-04-23T14:32:49.378297+0000","last_peered":"2024-04-23T14:32:49.378297+0000","last_clean":"2024-04-23T14:32:49.378297+0000","last_became_active":"2024-04-23T14:32:48.379485+0000","last_became_peered":"2024-04-23T14:32:48.379485+0000","last_unstale":"2024-04-23T14:32:49.378297+0000","last_undegraded":"2024-04-23T14:32:49.378297+0000","last_fullsized":"2024-04-23T14:32:49.378297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:00.386998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418531+0000","last_change":"2024-04-23T14:32:49.409577+0000","last_active":"2024-04-23T14:32:49.418531+0000","last_peered":"2024-04-23T14:32:49.418531+0000","last_clean":"2024-04-23T14:32:49.418531+0000","last_became_active":"2024-04-23T14:32:49.409264+0000","last_became_peered":"2024-04-23T14:32:49.409264+0000","last_unstale":"2024-04-23T14:32:49.418531+0000","last_undegraded":"2024-04-23T14:32:49.418531+0000","last_fullsized":"2024-04-23T14:32:49.418531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401594+0000","last_change":"2024-04-23T14:32:48.376245+0000","last_active":"2024-04-23T14:32:49.401594+0000","last_peered":"2024-04-23T14:32:49.401594+0000","last_clean":"2024-04-23T14:32:49.401594+0000","last_became_active":"2024-04-23T14:32:48.375882+0000","last_became_peered":"2024-04-23T14:32:48.375882+0000","last_unstale":"2024-04-23T14:32:49.401594+0000","last_undegraded":"2024-04-23T14:32:49.401594+0000","last_fullsized":"2024-04-23T14:32:49.401594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:27.386064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.403473+0000","last_change":"2024-04-23T14:32:49.403473+0000","last_active":"2024-04-23T14:32:49.403473+0000","last_peered":"2024-04-23T14:32:49.403473+0000","last_clean":"2024-04-23T14:32:49.403473+0000","last_became_active":"2024-04-23T14:32:49.399541+0000","last_became_peered":"2024-04-23T14:32:49.399541+0000","last_unstale":"2024-04-23T14:32:49.403473+0000","last_undegraded":"2024-04-23T14:32:49.403473+0000","last_fullsized":"2024-04-23T14:32:49.403473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397792+0000","last_change":"2024-04-23T14:32:48.381791+0000","last_active":"2024-04-23T14:32:49.397792+0000","last_peered":"2024-04-23T14:32:49.397792+0000","last_clean":"2024-04-23T14:32:49.397792+0000","last_became_active":"2024-04-23T14:32:48.381531+0000","last_became_peered":"2024-04-23T14:32:48.381531+0000","last_unstale":"2024-04-23T14:32:49.397792+0000","last_undegraded":"2024-04-23T14:32:49.397792+0000","last_fullsized":"2024-04-23T14:32:49.397792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412273+0000","last_change":"2024-04-23T14:32:49.412273+0000","last_active":"2024-04-23T14:32:49.412273+0000","last_peered":"2024-04-23T14:32:49.412273+0000","last_clean":"2024-04-23T14:32:49.412273+0000","last_became_active":"2024-04-23T14:32:49.411939+0000","last_became_peered":"2024-04-23T14:32:49.411939+0000","last_unstale":"2024-04-23T14:32:49.412273+0000","last_undegraded":"2024-04-23T14:32:49.412273+0000","last_fullsized":"2024-04-23T14:32:49.412273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375257+0000","last_change":"2024-04-23T14:32:48.383719+0000","last_active":"2024-04-23T14:32:49.375257+0000","last_peered":"2024-04-23T14:32:49.375257+0000","last_clean":"2024-04-23T14:32:49.375257+0000","last_became_active":"2024-04-23T14:32:48.382885+0000","last_became_peered":"2024-04-23T14:32:48.382885+0000","last_unstale":"2024-04-23T14:32:49.375257+0000","last_undegraded":"2024-04-23T14:32:49.375257+0000","last_fullsized":"2024-04-23T14:32:49.375257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414486+0000","last_change":"2024-04-23T14:32:49.406690+0000","last_active":"2024-04-23T14:32:49.414486+0000","last_peered":"2024-04-23T14:32:49.414486+0000","last_clean":"2024-04-23T14:32:49.414486+0000","last_became_active":"2024-04-23T14:32:49.406431+0000","last_became_peered":"2024-04-23T14:32:49.406431+0000","last_unstale":"2024-04-23T14:32:49.414486+0000","last_undegraded":"2024-04-23T14:32:49.414486+0000","last_fullsized":"2024-04-23T14:32:49.414486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375967+0000","last_change":"2024-04-23T14:32:48.375851+0000","last_active":"2024-04-23T14:32:49.375967+0000","last_peered":"2024-04-23T14:32:49.375967+0000","last_clean":"2024-04-23T14:32:49.375967+0000","last_became_active":"2024-04-23T14:32:48.375572+0000","last_became_peered":"2024-04-23T14:32:48.375572+0000","last_unstale":"2024-04-23T14:32:49.375967+0000","last_undegraded":"2024-04-23T14:32:49.375967+0000","last_fullsized":"2024-04-23T14:32:49.375967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417434+0000","last_change":"2024-04-23T14:32:49.409809+0000","last_active":"2024-04-23T14:32:49.417434+0000","last_peered":"2024-04-23T14:32:49.417434+0000","last_clean":"2024-04-23T14:32:49.417434+0000","last_became_active":"2024-04-23T14:32:49.409554+0000","last_became_peered":"2024-04-23T14:32:49.409554+0000","last_unstale":"2024-04-23T14:32:49.417434+0000","last_undegraded":"2024-04-23T14:32:49.417434+0000","last_fullsized":"2024-04-23T14:32:49.417434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:39.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400370+0000","last_change":"2024-04-23T14:32:48.383194+0000","last_active":"2024-04-23T14:32:49.400370+0000","last_peered":"2024-04-23T14:32:49.400370+0000","last_clean":"2024-04-23T14:32:49.400370+0000","last_became_active":"2024-04-23T14:32:48.382867+0000","last_became_peered":"2024-04-23T14:32:48.382867+0000","last_unstale":"2024-04-23T14:32:49.400370+0000","last_undegraded":"2024-04-23T14:32:49.400370+0000","last_fullsized":"2024-04-23T14:32:49.400370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413412+0000","last_change":"2024-04-23T14:32:49.413412+0000","last_active":"2024-04-23T14:32:49.413412+0000","last_peered":"2024-04-23T14:32:49.413412+0000","last_clean":"2024-04-23T14:32:49.413412+0000","last_became_active":"2024-04-23T14:32:49.413066+0000","last_became_peered":"2024-04-23T14:32:49.413066+0000","last_unstale":"2024-04-23T14:32:49.413412+0000","last_undegraded":"2024-04-23T14:32:49.413412+0000","last_fullsized":"2024-04-23T14:32:49.413412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401743+0000","last_change":"2024-04-23T14:32:48.373256+0000","last_active":"2024-04-23T14:32:49.401743+0000","last_peered":"2024-04-23T14:32:49.401743+0000","last_clean":"2024-04-23T14:32:49.401743+0000","last_became_active":"2024-04-23T14:32:48.372720+0000","last_became_peered":"2024-04-23T14:32:48.372720+0000","last_unstale":"2024-04-23T14:32:49.401743+0000","last_undegraded":"2024-04-23T14:32:49.401743+0000","last_fullsized":"2024-04-23T14:32:49.401743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:04:15.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418820+0000","last_change":"2024-04-23T14:32:49.411786+0000","last_active":"2024-04-23T14:32:49.418820+0000","last_peered":"2024-04-23T14:32:49.418820+0000","last_clean":"2024-04-23T14:32:49.418820+0000","last_became_active":"2024-04-23T14:32:49.411468+0000","last_became_peered":"2024-04-23T14:32:49.411468+0000","last_unstale":"2024-04-23T14:32:49.418820+0000","last_undegraded":"2024-04-23T14:32:49.418820+0000","last_fullsized":"2024-04-23T14:32:49.418820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401645+0000","last_change":"2024-04-23T14:32:48.374462+0000","last_active":"2024-04-23T14:32:49.401645+0000","last_peered":"2024-04-23T14:32:49.401645+0000","last_clean":"2024-04-23T14:32:49.401645+0000","last_became_active":"2024-04-23T14:32:48.372060+0000","last_became_peered":"2024-04-23T14:32:48.372060+0000","last_unstale":"2024-04-23T14:32:49.401645+0000","last_undegraded":"2024-04-23T14:32:49.401645+0000","last_fullsized":"2024-04-23T14:32:49.401645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:40.111344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.406897+0000","last_change":"2024-04-23T14:32:49.406897+0000","last_active":"2024-04-23T14:32:49.406897+0000","last_peered":"2024-04-23T14:32:49.406897+0000","last_clean":"2024-04-23T14:32:49.406897+0000","last_became_active":"2024-04-23T14:32:49.406604+0000","last_became_peered":"2024-04-23T14:32:49.406604+0000","last_unstale":"2024-04-23T14:32:49.406897+0000","last_undegraded":"2024-04-23T14:32:49.406897+0000","last_fullsized":"2024-04-23T14:32:49.406897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:32.281665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401493+0000","last_change":"2024-04-23T14:32:48.382650+0000","last_active":"2024-04-23T14:32:49.401493+0000","last_peered":"2024-04-23T14:32:49.401493+0000","last_clean":"2024-04-23T14:32:49.401493+0000","last_became_active":"2024-04-23T14:32:48.382391+0000","last_became_peered":"2024-04-23T14:32:48.382391+0000","last_unstale":"2024-04-23T14:32:49.401493+0000","last_undegraded":"2024-04-23T14:32:49.401493+0000","last_fullsized":"2024-04-23T14:32:49.401493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:51.353237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410725+0000","last_change":"2024-04-23T14:32:49.410725+0000","last_active":"2024-04-23T14:32:49.410725+0000","last_peered":"2024-04-23T14:32:49.410725+0000","last_clean":"2024-04-23T14:32:49.410725+0000","last_became_active":"2024-04-23T14:32:49.410335+0000","last_became_peered":"2024-04-23T14:32:49.410335+0000","last_unstale":"2024-04-23T14:32:49.410725+0000","last_undegraded":"2024-04-23T14:32:49.410725+0000","last_fullsized":"2024-04-23T14:32:49.410725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:23:29.638232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396023+0000","last_change":"2024-04-23T14:32:48.372373+0000","last_active":"2024-04-23T14:32:49.396023+0000","last_peered":"2024-04-23T14:32:49.396023+0000","last_clean":"2024-04-23T14:32:49.396023+0000","last_became_active":"2024-04-23T14:32:48.372040+0000","last_became_peered":"2024-04-23T14:32:48.372040+0000","last_unstale":"2024-04-23T14:32:49.396023+0000","last_undegraded":"2024-04-23T14:32:49.396023+0000","last_fullsized":"2024-04-23T14:32:49.396023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:24.434665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413110+0000","last_change":"2024-04-23T14:32:49.403841+0000","last_active":"2024-04-23T14:32:49.413110+0000","last_peered":"2024-04-23T14:32:49.413110+0000","last_clean":"2024-04-23T14:32:49.413110+0000","last_became_active":"2024-04-23T14:32:49.403486+0000","last_became_peered":"2024-04-23T14:32:49.403486+0000","last_unstale":"2024-04-23T14:32:49.413110+0000","last_undegraded":"2024-04-23T14:32:49.413110+0000","last_fullsized":"2024-04-23T14:32:49.413110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:48.649791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.381995+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.381595+0000","last_became_peered":"2024-04-23T14:32:48.381595+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:56.520711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413093+0000","last_change":"2024-04-23T14:32:49.405667+0000","last_active":"2024-04-23T14:32:49.413093+0000","last_peered":"2024-04-23T14:32:49.413093+0000","last_clean":"2024-04-23T14:32:49.413093+0000","last_became_active":"2024-04-23T14:32:49.403620+0000","last_became_peered":"2024-04-23T14:32:49.403620+0000","last_unstale":"2024-04-23T14:32:49.413093+0000","last_undegraded":"2024-04-23T14:32:49.413093+0000","last_fullsized":"2024-04-23T14:32:49.413093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:25.894354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401019+0000","last_change":"2024-04-23T14:32:48.375422+0000","last_active":"2024-04-23T14:32:49.401019+0000","last_peered":"2024-04-23T14:32:49.401019+0000","last_clean":"2024-04-23T14:32:49.401019+0000","last_became_active":"2024-04-23T14:32:48.375126+0000","last_became_peered":"2024-04-23T14:32:48.375126+0000","last_unstale":"2024-04-23T14:32:49.401019+0000","last_undegraded":"2024-04-23T14:32:49.401019+0000","last_fullsized":"2024-04-23T14:32:49.401019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:18.218895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407040+0000","last_change":"2024-04-23T14:32:49.399201+0000","last_active":"2024-04-23T14:32:49.407040+0000","last_peered":"2024-04-23T14:32:49.407040+0000","last_clean":"2024-04-23T14:32:49.407040+0000","last_became_active":"2024-04-23T14:32:49.398682+0000","last_became_peered":"2024-04-23T14:32:49.398682+0000","last_unstale":"2024-04-23T14:32:49.407040+0000","last_undegraded":"2024-04-23T14:32:49.407040+0000","last_fullsized":"2024-04-23T14:32:49.407040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381590+0000","last_change":"2024-04-23T14:32:48.378313+0000","last_active":"2024-04-23T14:32:49.381590+0000","last_peered":"2024-04-23T14:32:49.381590+0000","last_clean":"2024-04-23T14:32:49.381590+0000","last_became_active":"2024-04-23T14:32:48.377976+0000","last_became_peered":"2024-04-23T14:32:48.377976+0000","last_unstale":"2024-04-23T14:32:49.381590+0000","last_undegraded":"2024-04-23T14:32:49.381590+0000","last_fullsized":"2024-04-23T14:32:49.381590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408098+0000","last_change":"2024-04-23T14:32:49.408098+0000","last_active":"2024-04-23T14:32:49.408098+0000","last_peered":"2024-04-23T14:32:49.408098+0000","last_clean":"2024-04-23T14:32:49.408098+0000","last_became_active":"2024-04-23T14:32:49.407785+0000","last_became_peered":"2024-04-23T14:32:49.407785+0000","last_unstale":"2024-04-23T14:32:49.408098+0000","last_undegraded":"2024-04-23T14:32:49.408098+0000","last_fullsized":"2024-04-23T14:32:49.408098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:12:49.629971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400540+0000","last_change":"2024-04-23T14:32:48.386689+0000","last_active":"2024-04-23T14:32:49.400540+0000","last_peered":"2024-04-23T14:32:49.400540+0000","last_clean":"2024-04-23T14:32:49.400540+0000","last_became_active":"2024-04-23T14:32:48.386272+0000","last_became_peered":"2024-04-23T14:32:48.386272+0000","last_unstale":"2024-04-23T14:32:49.400540+0000","last_undegraded":"2024-04-23T14:32:49.400540+0000","last_fullsized":"2024-04-23T14:32:49.400540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411036+0000","last_change":"2024-04-23T14:32:49.407292+0000","last_active":"2024-04-23T14:32:49.411036+0000","last_peered":"2024-04-23T14:32:49.411036+0000","last_clean":"2024-04-23T14:32:49.411036+0000","last_became_active":"2024-04-23T14:32:49.406967+0000","last_became_peered":"2024-04-23T14:32:49.406967+0000","last_unstale":"2024-04-23T14:32:49.411036+0000","last_undegraded":"2024-04-23T14:32:49.411036+0000","last_fullsized":"2024-04-23T14:32:49.411036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396038+0000","last_change":"2024-04-23T14:32:48.372538+0000","last_active":"2024-04-23T14:32:49.396038+0000","last_peered":"2024-04-23T14:32:49.396038+0000","last_clean":"2024-04-23T14:32:49.396038+0000","last_became_active":"2024-04-23T14:32:48.372276+0000","last_became_peered":"2024-04-23T14:32:48.372276+0000","last_unstale":"2024-04-23T14:32:49.396038+0000","last_undegraded":"2024-04-23T14:32:49.396038+0000","last_fullsized":"2024-04-23T14:32:49.396038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:57.601612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383866+0000","last_change":"2024-04-23T14:32:49.383866+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383866+0000","last_undegraded":"2024-04-23T14:32:49.383866+0000","last_fullsized":"2024-04-23T14:32:49.383866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397243+0000","last_change":"2024-04-23T14:32:48.373983+0000","last_active":"2024-04-23T14:32:49.397243+0000","last_peered":"2024-04-23T14:32:49.397243+0000","last_clean":"2024-04-23T14:32:49.397243+0000","last_became_active":"2024-04-23T14:32:48.373688+0000","last_became_peered":"2024-04-23T14:32:48.373688+0000","last_unstale":"2024-04-23T14:32:49.397243+0000","last_undegraded":"2024-04-23T14:32:49.397243+0000","last_fullsized":"2024-04-23T14:32:49.397243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411462+0000","last_change":"2024-04-23T14:32:49.398323+0000","last_active":"2024-04-23T14:32:49.411462+0000","last_peered":"2024-04-23T14:32:49.411462+0000","last_clean":"2024-04-23T14:32:49.411462+0000","last_became_active":"2024-04-23T14:32:49.398039+0000","last_became_peered":"2024-04-23T14:32:49.398039+0000","last_unstale":"2024-04-23T14:32:49.411462+0000","last_undegraded":"2024-04-23T14:32:49.411462+0000","last_fullsized":"2024-04-23T14:32:49.411462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396203+0000","last_change":"2024-04-23T14:32:48.384396+0000","last_active":"2024-04-23T14:32:49.396203+0000","last_peered":"2024-04-23T14:32:49.396203+0000","last_clean":"2024-04-23T14:32:49.396203+0000","last_became_active":"2024-04-23T14:32:48.383739+0000","last_became_peered":"2024-04-23T14:32:48.383739+0000","last_unstale":"2024-04-23T14:32:49.396203+0000","last_undegraded":"2024-04-23T14:32:49.396203+0000","last_fullsized":"2024-04-23T14:32:49.396203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401815+0000","last_change":"2024-04-23T14:32:49.396368+0000","last_active":"2024-04-23T14:32:49.401815+0000","last_peered":"2024-04-23T14:32:49.401815+0000","last_clean":"2024-04-23T14:32:49.401815+0000","last_became_active":"2024-04-23T14:32:49.395031+0000","last_became_peered":"2024-04-23T14:32:49.395031+0000","last_unstale":"2024-04-23T14:32:49.401815+0000","last_undegraded":"2024-04-23T14:32:49.401815+0000","last_fullsized":"2024-04-23T14:32:49.401815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397942+0000","last_change":"2024-04-23T14:32:48.387008+0000","last_active":"2024-04-23T14:32:49.397942+0000","last_peered":"2024-04-23T14:32:49.397942+0000","last_clean":"2024-04-23T14:32:49.397942+0000","last_became_active":"2024-04-23T14:32:48.386260+0000","last_became_peered":"2024-04-23T14:32:48.386260+0000","last_unstale":"2024-04-23T14:32:49.397942+0000","last_undegraded":"2024-04-23T14:32:49.397942+0000","last_fullsized":"2024-04-23T14:32:49.397942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.415060+0000","last_change":"2024-04-23T14:32:49.415060+0000","last_active":"2024-04-23T14:32:49.415060+0000","last_peered":"2024-04-23T14:32:49.415060+0000","last_clean":"2024-04-23T14:32:49.415060+0000","last_became_active":"2024-04-23T14:32:49.414707+0000","last_became_peered":"2024-04-23T14:32:49.414707+0000","last_unstale":"2024-04-23T14:32:49.415060+0000","last_undegraded":"2024-04-23T14:32:49.415060+0000","last_fullsized":"2024-04-23T14:32:49.415060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:02.059103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401039+0000","last_change":"2024-04-23T14:32:48.380600+0000","last_active":"2024-04-23T14:32:49.401039+0000","last_peered":"2024-04-23T14:32:49.401039+0000","last_clean":"2024-04-23T14:32:49.401039+0000","last_became_active":"2024-04-23T14:32:48.379900+0000","last_became_peered":"2024-04-23T14:32:48.379900+0000","last_unstale":"2024-04-23T14:32:49.401039+0000","last_undegraded":"2024-04-23T14:32:49.401039+0000","last_fullsized":"2024-04-23T14:32:49.401039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:31.916332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.425051+0000","last_change":"2024-04-23T14:32:49.417179+0000","last_active":"2024-04-23T14:32:49.425051+0000","last_peered":"2024-04-23T14:32:49.425051+0000","last_clean":"2024-04-23T14:32:49.425051+0000","last_became_active":"2024-04-23T14:32:49.414238+0000","last_became_peered":"2024-04-23T14:32:49.414238+0000","last_unstale":"2024-04-23T14:32:49.425051+0000","last_undegraded":"2024-04-23T14:32:49.425051+0000","last_fullsized":"2024-04-23T14:32:49.425051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375409+0000","last_change":"2024-04-23T14:32:48.382557+0000","last_active":"2024-04-23T14:32:49.375409+0000","last_peered":"2024-04-23T14:32:49.375409+0000","last_clean":"2024-04-23T14:32:49.375409+0000","last_became_active":"2024-04-23T14:32:48.381989+0000","last_became_peered":"2024-04-23T14:32:48.381989+0000","last_unstale":"2024-04-23T14:32:49.375409+0000","last_undegraded":"2024-04-23T14:32:49.375409+0000","last_fullsized":"2024-04-23T14:32:49.375409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411232+0000","last_change":"2024-04-23T14:32:49.406874+0000","last_active":"2024-04-23T14:32:49.411232+0000","last_peered":"2024-04-23T14:32:49.411232+0000","last_clean":"2024-04-23T14:32:49.411232+0000","last_became_active":"2024-04-23T14:32:49.406634+0000","last_became_peered":"2024-04-23T14:32:49.406634+0000","last_unstale":"2024-04-23T14:32:49.411232+0000","last_undegraded":"2024-04-23T14:32:49.411232+0000","last_fullsized":"2024-04-23T14:32:49.411232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380918+0000","last_change":"2024-04-23T14:32:48.381294+0000","last_active":"2024-04-23T14:32:49.380918+0000","last_peered":"2024-04-23T14:32:49.380918+0000","last_clean":"2024-04-23T14:32:49.380918+0000","last_became_active":"2024-04-23T14:32:48.380990+0000","last_became_peered":"2024-04-23T14:32:48.380990+0000","last_unstale":"2024-04-23T14:32:49.380918+0000","last_undegraded":"2024-04-23T14:32:49.380918+0000","last_fullsized":"2024-04-23T14:32:49.380918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391358+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:32:49.391358+0000","last_peered":"2024-04-23T14:32:49.391358+0000","last_clean":"2024-04-23T14:32:49.391358+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:32:49.391358+0000","last_undegraded":"2024-04-23T14:32:49.391358+0000","last_fullsized":"2024-04-23T14:32:49.391358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:54.383 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.630+0000 7f7602800640 1 Processor -- start 2024-04-23T14:32:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.630+0000 7f7602800640 1 -- start start 2024-04-23T14:32:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.632+0000 7f7602800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f75fc160910 0x7f75fc160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.632+0000 7f7602800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc0cd520 con 0x7f75fc1454e0 2024-04-23T14:32:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.632+0000 7f7602800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc0cd800 con 0x7f75fc0cd020 2024-04-23T14:32:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.632+0000 7f7602800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc0cddc0 con 0x7f75fc160910 2024-04-23T14:32:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.632+0000 7f75fb400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f75fc1454e0 0x7f75fc145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35174/0 (socket says 172.21.15.59:35174) 2024-04-23T14:32:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.632+0000 7f75fb400640 1 -- 172.21.15.59:0/2666956483 learned_addr learned my addr 172.21.15.59:0/2666956483 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fbe00640 1 --2- 172.21.15.59:0/2666956483 >> v2:172.21.15.76:3300/0 conn(0x7f75fc160910 0x7f75fc160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244284561 0 0) 0x7f75fc0cd520 con 0x7f75fc1454e0 2024-04-23T14:32:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75e0003520 con 0x7f75fc1454e0 2024-04-23T14:32:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3123636536 0 0) 0x7f75e0003520 con 0x7f75fc1454e0 2024-04-23T14:32:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 >> v2:172.21.15.76:3300/0 conn(0x7f75fc160910 msgr2=0x7f75fc160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 --2- 172.21.15.59:0/2666956483 >> v2:172.21.15.76:3300/0 conn(0x7f75fc160910 0x7f75fc160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 >> v1:172.21.15.76:6789/0 conn(0x7f75fc0cd020 legacy=0x7f75fc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc0cdae0 con 0x7f75fc1454e0 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f75ec001670 con 0x7f75fc1454e0 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 >> v1:172.21.15.59:6789/0 conn(0x7f75fc1454e0 legacy=0x7f75fc145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75e00039b0 0x7f75e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e00051c0 0x7f75e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0cddc0 con 0x7f75e00039b0 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75e000c160 con 0x7f75e00051c0 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.633+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f75e000c360 con 0x7f75e0004800 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fb400640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75e00039b0 0x7f75e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75faa00640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fbe00640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e00051c0 0x7f75e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fb400640 1 -- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e0004800 msgr2=0x7f75e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fb400640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fb400640 1 -- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e00051c0 msgr2=0x7f75e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fb400640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75e00051c0 0x7f75e000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fb400640 1 -- 172.21.15.59:0/2666956483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e0003520 con 0x7f75e00039b0 2024-04-23T14:32:54.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.634+0000 7f75fbe00640 1 -- 172.21.15.59:0/2666956483 reap_dead start 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.635+0000 7f75fb400640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75e00039b0 0x7f75e0003dd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c240 tx=0x7f75e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d517b656bb35e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.635+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec007cf0 con 0x7f75e00039b0 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.635+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec005af0 con 0x7f75e00039b0 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.635+0000 7f75fa000640 1 -- 172.21.15.59:0/2666956483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec007070 con 0x7f75e00039b0 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.636+0000 7f7602800640 1 -- 172.21.15.59:0/2666956483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75e00039b0 msgr2=0x7f75e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.636+0000 7f7602800640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75e00039b0 0x7f75e0003dd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c240 tx=0x7f75e0005830 comp rx=0 tx=0).stop 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.636+0000 7f7602800640 1 -- 172.21.15.59:0/2666956483 shutdown_connections 2024-04-23T14:32:54.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.636+0000 7f7602800640 1 --2- 172.21.15.59:0/2666956483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75e00039b0 0x7f75e0003dd0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.636+0000 7f7602800640 1 -- 172.21.15.59:0/2666956483 >> 172.21.15.59:0/2666956483 conn(0x7f75fc067ea0 msgr2=0x7f75fc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:54.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.636+0000 7f7602800640 1 -- 172.21.15.59:0/2666956483 shutdown_connections 2024-04-23T14:32:54.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.636+0000 7f7602800640 1 -- 172.21.15.59:0/2666956483 wait complete. 2024-04-23T14:32:54.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.638+0000 7f7602800640 1 Processor -- start 2024-04-23T14:32:54.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.638+0000 7f7602800640 1 -- start start 2024-04-23T14:32:54.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.639+0000 7f7602800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e00039b0 0x7f75fc1f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.639+0000 7f7602800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75fc13cbb0 0x7f75fc13d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.639+0000 7f7602800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75fc0b3900 0x7f75fc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.639+0000 7f7602800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc160f00 con 0x7f75fc0b3900 2024-04-23T14:32:54.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.639+0000 7f7602800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc160d80 con 0x7f75fc13cbb0 2024-04-23T14:32:54.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.639+0000 7f7602800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc161080 con 0x7f75e00039b0 2024-04-23T14:32:54.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.639+0000 7f75fbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75fc0b3900 0x7f75fc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75fc0b3900 0x7f75fc0b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45606/0 (socket says 172.21.15.59:45606) 2024-04-23T14:32:54.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 -- 172.21.15.59:0/752761085 learned_addr learned my addr 172.21.15.59:0/752761085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:54.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75faa00640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75fc13cbb0 0x7f75fc13d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fb400640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e00039b0 0x7f75fc1f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 -- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e00039b0 msgr2=0x7f75fc1f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e00039b0 0x7f75fc1f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 -- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75fc13cbb0 msgr2=0x7f75fc13d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75fc13cbb0 0x7f75fc13d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 -- 172.21.15.59:0/752761085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc13bfe0 con 0x7f75fc0b3900 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75faa00640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75fc13cbb0 0x7f75fc13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.640+0000 7f75fbe00640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75fc0b3900 0x7f75fc0b3d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0b3680 tx=0x7f75f400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.641+0000 7f75f3400640 1 -- 172.21.15.59:0/752761085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4016420 con 0x7f75fc0b3900 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.641+0000 7f75f3400640 1 -- 172.21.15.59:0/752761085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f401b070 con 0x7f75fc0b3900 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.641+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc14fa00 con 0x7f75fc0b3900 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.641+0000 7f75f3400640 1 -- 172.21.15.59:0/752761085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4016820 con 0x7f75fc0b3900 2024-04-23T14:32:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.641+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc14ffc0 con 0x7f75fc0b3900 2024-04-23T14:32:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.642+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc1527d0 con 0x7f75fc0b3900 2024-04-23T14:32:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.643+0000 7f75f3400640 1 -- 172.21.15.59:0/752761085 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75f40169c0 con 0x7f75fc0b3900 2024-04-23T14:32:54.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.644+0000 7f75f3400640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f75dc06c920 0x7f75dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:54.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.644+0000 7f75f3400640 1 -- 172.21.15.59:0/752761085 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f75f408cab0 con 0x7f75fc0b3900 2024-04-23T14:32:54.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.644+0000 7f75fb400640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f75dc06c920 0x7f75dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:54.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.645+0000 7f75fb400640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f75dc06c920 0x7f75dc06ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75ec032000 tx=0x7f75ec009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:54.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.648+0000 7f75f3400640 1 -- 172.21.15.59:0/752761085 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75fc1527d0 con 0x7f75fc0b3900 2024-04-23T14:32:54.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.848+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75fc068a80 con 0x7f75dc06c920 2024-04-23T14:32:54.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.861+0000 7f75f3400640 1 -- 172.21.15.59:0/752761085 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251346 (secure 0 0 0) 0x7f75fc068a80 con 0x7f75dc06c920 2024-04-23T14:32:54.865 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:54.866 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:32:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.865+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f75dc06c920 msgr2=0x7f75dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.865+0000 7f7602800640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f75dc06c920 0x7f75dc06ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75ec032000 tx=0x7f75ec009030 comp rx=0 tx=0).stop 2024-04-23T14:32:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.865+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75fc0b3900 msgr2=0x7f75fc0b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.865+0000 7f7602800640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75fc0b3900 0x7f75fc0b3d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0b3680 tx=0x7f75f400f520 comp rx=0 tx=0).stop 2024-04-23T14:32:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 shutdown_connections 2024-04-23T14:32:54.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f75dc06c920 0x7f75dc06ede0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75fc0b3900 0x7f75fc0b3d90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75fc13cbb0 0x7f75fc13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 --2- 172.21.15.59:0/752761085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75e00039b0 0x7f75fc1f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 >> 172.21.15.59:0/752761085 conn(0x7f75fc067ea0 msgr2=0x7f75fc0b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 shutdown_connections 2024-04-23T14:32:54.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:54.866+0000 7f7602800640 1 -- 172.21.15.59:0/752761085 wait complete. 2024-04-23T14:32:54.885 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-23T14:32:54.371142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224208,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750512,"statfs":{"total":773094113280,"available":772864524288,"internally_reserved":0,"allocated":3973120,"data_stored":2249000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.023568"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408276+0000","last_change":"2024-04-23T14:32:49.396796+0000","last_active":"2024-04-23T14:32:49.408276+0000","last_peered":"2024-04-23T14:32:49.408276+0000","last_clean":"2024-04-23T14:32:49.408276+0000","last_became_active":"2024-04-23T14:32:49.396505+0000","last_became_peered":"2024-04-23T14:32:49.396505+0000","last_unstale":"2024-04-23T14:32:49.408276+0000","last_undegraded":"2024-04-23T14:32:49.408276+0000","last_fullsized":"2024-04-23T14:32:49.408276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407907+0000","last_change":"2024-04-23T14:32:49.396404+0000","last_active":"2024-04-23T14:32:49.407907+0000","last_peered":"2024-04-23T14:32:49.407907+0000","last_clean":"2024-04-23T14:32:49.407907+0000","last_became_active":"2024-04-23T14:32:49.396103+0000","last_became_peered":"2024-04-23T14:32:49.396103+0000","last_unstale":"2024-04-23T14:32:49.407907+0000","last_undegraded":"2024-04-23T14:32:49.407907+0000","last_fullsized":"2024-04-23T14:32:49.407907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.382675+0000","last_change":"2024-04-23T14:32:49.382675+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.382675+0000","last_undegraded":"2024-04-23T14:32:49.382675+0000","last_fullsized":"2024-04-23T14:32:49.382675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418842+0000","last_change":"2024-04-23T14:32:49.412161+0000","last_active":"2024-04-23T14:32:49.418842+0000","last_peered":"2024-04-23T14:32:49.418842+0000","last_clean":"2024-04-23T14:32:49.418842+0000","last_became_active":"2024-04-23T14:32:49.411609+0000","last_became_peered":"2024-04-23T14:32:49.411609+0000","last_unstale":"2024-04-23T14:32:49.418842+0000","last_undegraded":"2024-04-23T14:32:49.418842+0000","last_fullsized":"2024-04-23T14:32:49.418842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411075+0000","last_change":"2024-04-23T14:32:49.411075+0000","last_active":"2024-04-23T14:32:49.411075+0000","last_peered":"2024-04-23T14:32:49.411075+0000","last_clean":"2024-04-23T14:32:49.411075+0000","last_became_active":"2024-04-23T14:32:49.410648+0000","last_became_peered":"2024-04-23T14:32:49.410648+0000","last_unstale":"2024-04-23T14:32:49.411075+0000","last_undegraded":"2024-04-23T14:32:49.411075+0000","last_fullsized":"2024-04-23T14:32:49.411075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:19.233384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412676+0000","last_change":"2024-04-23T14:32:49.396761+0000","last_active":"2024-04-23T14:32:49.412676+0000","last_peered":"2024-04-23T14:32:49.412676+0000","last_clean":"2024-04-23T14:32:49.412676+0000","last_became_active":"2024-04-23T14:32:49.396598+0000","last_became_peered":"2024-04-23T14:32:49.396598+0000","last_unstale":"2024-04-23T14:32:49.412676+0000","last_undegraded":"2024-04-23T14:32:49.412676+0000","last_fullsized":"2024-04-23T14:32:49.412676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407019+0000","last_change":"2024-04-23T14:32:49.398865+0000","last_active":"2024-04-23T14:32:49.407019+0000","last_peered":"2024-04-23T14:32:49.407019+0000","last_clean":"2024-04-23T14:32:49.407019+0000","last_became_active":"2024-04-23T14:32:49.398581+0000","last_became_peered":"2024-04-23T14:32:49.398581+0000","last_unstale":"2024-04-23T14:32:49.407019+0000","last_undegraded":"2024-04-23T14:32:49.407019+0000","last_fullsized":"2024-04-23T14:32:49.407019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409232+0000","last_change":"2024-04-23T14:32:49.409232+0000","last_active":"2024-04-23T14:32:49.409232+0000","last_peered":"2024-04-23T14:32:49.409232+0000","last_clean":"2024-04-23T14:32:49.409232+0000","last_became_active":"2024-04-23T14:32:49.408162+0000","last_became_peered":"2024-04-23T14:32:49.408162+0000","last_unstale":"2024-04-23T14:32:49.409232+0000","last_undegraded":"2024-04-23T14:32:49.409232+0000","last_fullsized":"2024-04-23T14:32:49.409232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.424075+0000","last_change":"2024-04-23T14:32:49.424075+0000","last_active":"2024-04-23T14:32:49.424075+0000","last_peered":"2024-04-23T14:32:49.424075+0000","last_clean":"2024-04-23T14:32:49.424075+0000","last_became_active":"2024-04-23T14:32:49.423714+0000","last_became_peered":"2024-04-23T14:32:49.423714+0000","last_unstale":"2024-04-23T14:32:49.424075+0000","last_undegraded":"2024-04-23T14:32:49.424075+0000","last_fullsized":"2024-04-23T14:32:49.424075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421421+0000","last_change":"2024-04-23T14:32:49.419419+0000","last_active":"2024-04-23T14:32:49.421421+0000","last_peered":"2024-04-23T14:32:49.421421+0000","last_clean":"2024-04-23T14:32:49.421421+0000","last_became_active":"2024-04-23T14:32:49.416916+0000","last_became_peered":"2024-04-23T14:32:49.416916+0000","last_unstale":"2024-04-23T14:32:49.421421+0000","last_undegraded":"2024-04-23T14:32:49.421421+0000","last_fullsized":"2024-04-23T14:32:49.421421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389894+0000","last_change":"2024-04-23T14:32:49.389894+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389894+0000","last_undegraded":"2024-04-23T14:32:49.389894+0000","last_fullsized":"2024-04-23T14:32:49.389894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414777+0000","last_change":"2024-04-23T14:32:49.414777+0000","last_active":"2024-04-23T14:32:49.414777+0000","last_peered":"2024-04-23T14:32:49.414777+0000","last_clean":"2024-04-23T14:32:49.414777+0000","last_became_active":"2024-04-23T14:32:49.414208+0000","last_became_peered":"2024-04-23T14:32:49.414208+0000","last_unstale":"2024-04-23T14:32:49.414777+0000","last_undegraded":"2024-04-23T14:32:49.414777+0000","last_fullsized":"2024-04-23T14:32:49.414777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389314+0000","last_change":"2024-04-23T14:32:49.389314+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389314+0000","last_undegraded":"2024-04-23T14:32:49.389314+0000","last_fullsized":"2024-04-23T14:32:49.389314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409664+0000","last_change":"2024-04-23T14:32:49.409664+0000","last_active":"2024-04-23T14:32:49.409664+0000","last_peered":"2024-04-23T14:32:49.409664+0000","last_clean":"2024-04-23T14:32:49.409664+0000","last_became_active":"2024-04-23T14:32:49.409369+0000","last_became_peered":"2024-04-23T14:32:49.409369+0000","last_unstale":"2024-04-23T14:32:49.409664+0000","last_undegraded":"2024-04-23T14:32:49.409664+0000","last_fullsized":"2024-04-23T14:32:49.409664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:53.618947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404125+0000","last_change":"2024-04-23T14:32:49.395881+0000","last_active":"2024-04-23T14:32:49.404125+0000","last_peered":"2024-04-23T14:32:49.404125+0000","last_clean":"2024-04-23T14:32:49.404125+0000","last_became_active":"2024-04-23T14:32:49.395640+0000","last_became_peered":"2024-04-23T14:32:49.395640+0000","last_unstale":"2024-04-23T14:32:49.404125+0000","last_undegraded":"2024-04-23T14:32:49.404125+0000","last_fullsized":"2024-04-23T14:32:49.404125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410041+0000","last_change":"2024-04-23T14:32:49.410041+0000","last_active":"2024-04-23T14:32:49.410041+0000","last_peered":"2024-04-23T14:32:49.410041+0000","last_clean":"2024-04-23T14:32:49.410041+0000","last_became_active":"2024-04-23T14:32:49.409759+0000","last_became_peered":"2024-04-23T14:32:49.409759+0000","last_unstale":"2024-04-23T14:32:49.410041+0000","last_undegraded":"2024-04-23T14:32:49.410041+0000","last_fullsized":"2024-04-23T14:32:49.410041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408902+0000","last_change":"2024-04-23T14:32:49.405810+0000","last_active":"2024-04-23T14:32:49.408902+0000","last_peered":"2024-04-23T14:32:49.408902+0000","last_clean":"2024-04-23T14:32:49.408902+0000","last_became_active":"2024-04-23T14:32:49.405283+0000","last_became_peered":"2024-04-23T14:32:49.405283+0000","last_unstale":"2024-04-23T14:32:49.408902+0000","last_undegraded":"2024-04-23T14:32:49.408902+0000","last_fullsized":"2024-04-23T14:32:49.408902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.379405+0000","last_change":"2024-04-23T14:32:49.379405+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.379405+0000","last_undegraded":"2024-04-23T14:32:49.379405+0000","last_fullsized":"2024-04-23T14:32:49.379405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.378840+0000","last_change":"2024-04-23T14:32:49.378840+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.378840+0000","last_undegraded":"2024-04-23T14:32:49.378840+0000","last_fullsized":"2024-04-23T14:32:49.378840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408152+0000","last_change":"2024-04-23T14:32:49.408152+0000","last_active":"2024-04-23T14:32:49.408152+0000","last_peered":"2024-04-23T14:32:49.408152+0000","last_clean":"2024-04-23T14:32:49.408152+0000","last_became_active":"2024-04-23T14:32:49.407758+0000","last_became_peered":"2024-04-23T14:32:49.407758+0000","last_unstale":"2024-04-23T14:32:49.408152+0000","last_undegraded":"2024-04-23T14:32:49.408152+0000","last_fullsized":"2024-04-23T14:32:49.408152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:42.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416792+0000","last_change":"2024-04-23T14:32:49.413362+0000","last_active":"2024-04-23T14:32:49.416792+0000","last_peered":"2024-04-23T14:32:49.416792+0000","last_clean":"2024-04-23T14:32:49.416792+0000","last_became_active":"2024-04-23T14:32:49.413082+0000","last_became_peered":"2024-04-23T14:32:49.413082+0000","last_unstale":"2024-04-23T14:32:49.416792+0000","last_undegraded":"2024-04-23T14:32:49.416792+0000","last_fullsized":"2024-04-23T14:32:49.416792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:57.535200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417468+0000","last_change":"2024-04-23T14:32:49.414460+0000","last_active":"2024-04-23T14:32:49.417468+0000","last_peered":"2024-04-23T14:32:49.417468+0000","last_clean":"2024-04-23T14:32:49.417468+0000","last_became_active":"2024-04-23T14:32:49.413658+0000","last_became_peered":"2024-04-23T14:32:49.413658+0000","last_unstale":"2024-04-23T14:32:49.417468+0000","last_undegraded":"2024-04-23T14:32:49.417468+0000","last_fullsized":"2024-04-23T14:32:49.417468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:32.704139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412931+0000","last_change":"2024-04-23T14:32:49.404657+0000","last_active":"2024-04-23T14:32:49.412931+0000","last_peered":"2024-04-23T14:32:49.412931+0000","last_clean":"2024-04-23T14:32:49.412931+0000","last_became_active":"2024-04-23T14:32:49.404322+0000","last_became_peered":"2024-04-23T14:32:49.404322+0000","last_unstale":"2024-04-23T14:32:49.412931+0000","last_undegraded":"2024-04-23T14:32:49.412931+0000","last_fullsized":"2024-04-23T14:32:49.412931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416596+0000","last_change":"2024-04-23T14:32:49.408981+0000","last_active":"2024-04-23T14:32:49.416596+0000","last_peered":"2024-04-23T14:32:49.416596+0000","last_clean":"2024-04-23T14:32:49.416596+0000","last_became_active":"2024-04-23T14:32:49.408732+0000","last_became_peered":"2024-04-23T14:32:49.408732+0000","last_unstale":"2024-04-23T14:32:49.416596+0000","last_undegraded":"2024-04-23T14:32:49.416596+0000","last_fullsized":"2024-04-23T14:32:49.416596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419806+0000","last_change":"2024-04-23T14:32:49.411802+0000","last_active":"2024-04-23T14:32:49.419806+0000","last_peered":"2024-04-23T14:32:49.419806+0000","last_clean":"2024-04-23T14:32:49.419806+0000","last_became_active":"2024-04-23T14:32:49.411472+0000","last_became_peered":"2024-04-23T14:32:49.411472+0000","last_unstale":"2024-04-23T14:32:49.419806+0000","last_undegraded":"2024-04-23T14:32:49.419806+0000","last_fullsized":"2024-04-23T14:32:49.419806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408321+0000","last_change":"2024-04-23T14:32:49.408321+0000","last_active":"2024-04-23T14:32:49.408321+0000","last_peered":"2024-04-23T14:32:49.408321+0000","last_clean":"2024-04-23T14:32:49.408321+0000","last_became_active":"2024-04-23T14:32:49.407885+0000","last_became_peered":"2024-04-23T14:32:49.407885+0000","last_unstale":"2024-04-23T14:32:49.408321+0000","last_undegraded":"2024-04-23T14:32:49.408321+0000","last_fullsized":"2024-04-23T14:32:49.408321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420428+0000","last_change":"2024-04-23T14:32:49.418183+0000","last_active":"2024-04-23T14:32:49.420428+0000","last_peered":"2024-04-23T14:32:49.420428+0000","last_clean":"2024-04-23T14:32:49.420428+0000","last_became_active":"2024-04-23T14:32:49.417903+0000","last_became_peered":"2024-04-23T14:32:49.417903+0000","last_unstale":"2024-04-23T14:32:49.420428+0000","last_undegraded":"2024-04-23T14:32:49.420428+0000","last_fullsized":"2024-04-23T14:32:49.420428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404038+0000","last_change":"2024-04-23T14:32:49.396858+0000","last_active":"2024-04-23T14:32:49.404038+0000","last_peered":"2024-04-23T14:32:49.404038+0000","last_clean":"2024-04-23T14:32:49.404038+0000","last_became_active":"2024-04-23T14:32:49.396565+0000","last_became_peered":"2024-04-23T14:32:49.396565+0000","last_unstale":"2024-04-23T14:32:49.404038+0000","last_undegraded":"2024-04-23T14:32:49.404038+0000","last_fullsized":"2024-04-23T14:32:49.404038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419860+0000","last_change":"2024-04-23T14:32:49.419860+0000","last_active":"2024-04-23T14:32:49.419860+0000","last_peered":"2024-04-23T14:32:49.419860+0000","last_clean":"2024-04-23T14:32:49.419860+0000","last_became_active":"2024-04-23T14:32:49.419578+0000","last_became_peered":"2024-04-23T14:32:49.419578+0000","last_unstale":"2024-04-23T14:32:49.419860+0000","last_undegraded":"2024-04-23T14:32:49.419860+0000","last_fullsized":"2024-04-23T14:32:49.419860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407177+0000","last_change":"2024-04-23T14:32:49.403367+0000","last_active":"2024-04-23T14:32:49.407177+0000","last_peered":"2024-04-23T14:32:49.407177+0000","last_clean":"2024-04-23T14:32:49.407177+0000","last_became_active":"2024-04-23T14:32:49.403043+0000","last_became_peered":"2024-04-23T14:32:49.403043+0000","last_unstale":"2024-04-23T14:32:49.407177+0000","last_undegraded":"2024-04-23T14:32:49.407177+0000","last_fullsized":"2024-04-23T14:32:49.407177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413518+0000","last_change":"2024-04-23T14:32:49.407909+0000","last_active":"2024-04-23T14:32:49.413518+0000","last_peered":"2024-04-23T14:32:49.413518+0000","last_clean":"2024-04-23T14:32:49.413518+0000","last_became_active":"2024-04-23T14:32:49.407653+0000","last_became_peered":"2024-04-23T14:32:49.407653+0000","last_unstale":"2024-04-23T14:32:49.413518+0000","last_undegraded":"2024-04-23T14:32:49.413518+0000","last_fullsized":"2024-04-23T14:32:49.413518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412291+0000","last_change":"2024-04-23T14:32:49.412291+0000","last_active":"2024-04-23T14:32:49.412291+0000","last_peered":"2024-04-23T14:32:49.412291+0000","last_clean":"2024-04-23T14:32:49.412291+0000","last_became_active":"2024-04-23T14:32:49.411966+0000","last_became_peered":"2024-04-23T14:32:49.411966+0000","last_unstale":"2024-04-23T14:32:49.412291+0000","last_undegraded":"2024-04-23T14:32:49.412291+0000","last_fullsized":"2024-04-23T14:32:49.412291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401707+0000","last_change":"2024-04-23T14:32:49.398365+0000","last_active":"2024-04-23T14:32:49.401707+0000","last_peered":"2024-04-23T14:32:49.401707+0000","last_clean":"2024-04-23T14:32:49.401707+0000","last_became_active":"2024-04-23T14:32:49.398036+0000","last_became_peered":"2024-04-23T14:32:49.398036+0000","last_unstale":"2024-04-23T14:32:49.401707+0000","last_undegraded":"2024-04-23T14:32:49.401707+0000","last_fullsized":"2024-04-23T14:32:49.401707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.380750+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.380399+0000","last_became_peered":"2024-04-23T14:32:48.380399+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:31.689650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414566+0000","last_change":"2024-04-23T14:32:49.411206+0000","last_active":"2024-04-23T14:32:49.414566+0000","last_peered":"2024-04-23T14:32:49.414566+0000","last_clean":"2024-04-23T14:32:49.414566+0000","last_became_active":"2024-04-23T14:32:49.410469+0000","last_became_peered":"2024-04-23T14:32:49.410469+0000","last_unstale":"2024-04-23T14:32:49.414566+0000","last_undegraded":"2024-04-23T14:32:49.414566+0000","last_fullsized":"2024-04-23T14:32:49.414566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395347+0000","last_change":"2024-04-23T14:32:48.378555+0000","last_active":"2024-04-23T14:32:49.395347+0000","last_peered":"2024-04-23T14:32:49.395347+0000","last_clean":"2024-04-23T14:32:49.395347+0000","last_became_active":"2024-04-23T14:32:48.377594+0000","last_became_peered":"2024-04-23T14:32:48.377594+0000","last_unstale":"2024-04-23T14:32:49.395347+0000","last_undegraded":"2024-04-23T14:32:49.395347+0000","last_fullsized":"2024-04-23T14:32:49.395347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418168+0000","last_change":"2024-04-23T14:32:49.418168+0000","last_active":"2024-04-23T14:32:49.418168+0000","last_peered":"2024-04-23T14:32:49.418168+0000","last_clean":"2024-04-23T14:32:49.418168+0000","last_became_active":"2024-04-23T14:32:49.417542+0000","last_became_peered":"2024-04-23T14:32:49.417542+0000","last_unstale":"2024-04-23T14:32:49.418168+0000","last_undegraded":"2024-04-23T14:32:49.418168+0000","last_fullsized":"2024-04-23T14:32:49.418168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381457+0000","last_change":"2024-04-23T14:32:48.383157+0000","last_active":"2024-04-23T14:32:49.381457+0000","last_peered":"2024-04-23T14:32:49.381457+0000","last_clean":"2024-04-23T14:32:49.381457+0000","last_became_active":"2024-04-23T14:32:48.382568+0000","last_became_peered":"2024-04-23T14:32:48.382568+0000","last_unstale":"2024-04-23T14:32:49.381457+0000","last_undegraded":"2024-04-23T14:32:49.381457+0000","last_fullsized":"2024-04-23T14:32:49.381457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:35.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420450+0000","last_change":"2024-04-23T14:32:49.418053+0000","last_active":"2024-04-23T14:32:49.420450+0000","last_peered":"2024-04-23T14:32:49.420450+0000","last_clean":"2024-04-23T14:32:49.420450+0000","last_became_active":"2024-04-23T14:32:49.417733+0000","last_became_peered":"2024-04-23T14:32:49.417733+0000","last_unstale":"2024-04-23T14:32:49.420450+0000","last_undegraded":"2024-04-23T14:32:49.420450+0000","last_fullsized":"2024-04-23T14:32:49.420450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391055+0000","last_change":"2024-04-23T14:32:48.374028+0000","last_active":"2024-04-23T14:32:49.391055+0000","last_peered":"2024-04-23T14:32:49.391055+0000","last_clean":"2024-04-23T14:32:49.391055+0000","last_became_active":"2024-04-23T14:32:48.372871+0000","last_became_peered":"2024-04-23T14:32:48.372871+0000","last_unstale":"2024-04-23T14:32:49.391055+0000","last_undegraded":"2024-04-23T14:32:49.391055+0000","last_fullsized":"2024-04-23T14:32:49.391055+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:14.297285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410412+0000","last_change":"2024-04-23T14:32:49.410412+0000","last_active":"2024-04-23T14:32:49.410412+0000","last_peered":"2024-04-23T14:32:49.410412+0000","last_clean":"2024-04-23T14:32:49.410412+0000","last_became_active":"2024-04-23T14:32:49.409645+0000","last_became_peered":"2024-04-23T14:32:49.409645+0000","last_unstale":"2024-04-23T14:32:49.410412+0000","last_undegraded":"2024-04-23T14:32:49.410412+0000","last_fullsized":"2024-04-23T14:32:49.410412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401440+0000","last_change":"2024-04-23T14:32:48.381407+0000","last_active":"2024-04-23T14:32:49.401440+0000","last_peered":"2024-04-23T14:32:49.401440+0000","last_clean":"2024-04-23T14:32:49.401440+0000","last_became_active":"2024-04-23T14:32:48.381122+0000","last_became_peered":"2024-04-23T14:32:48.381122+0000","last_unstale":"2024-04-23T14:32:49.401440+0000","last_undegraded":"2024-04-23T14:32:49.401440+0000","last_fullsized":"2024-04-23T14:32:49.401440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:16:40.112260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408197+0000","last_change":"2024-04-23T14:32:49.400326+0000","last_active":"2024-04-23T14:32:49.408197+0000","last_peered":"2024-04-23T14:32:49.408197+0000","last_clean":"2024-04-23T14:32:49.408197+0000","last_became_active":"2024-04-23T14:32:49.399716+0000","last_became_peered":"2024-04-23T14:32:49.399716+0000","last_unstale":"2024-04-23T14:32:49.408197+0000","last_undegraded":"2024-04-23T14:32:49.408197+0000","last_fullsized":"2024-04-23T14:32:49.408197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401159+0000","last_change":"2024-04-23T14:32:48.380347+0000","last_active":"2024-04-23T14:32:49.401159+0000","last_peered":"2024-04-23T14:32:49.401159+0000","last_clean":"2024-04-23T14:32:49.401159+0000","last_became_active":"2024-04-23T14:32:48.379799+0000","last_became_peered":"2024-04-23T14:32:48.379799+0000","last_unstale":"2024-04-23T14:32:49.401159+0000","last_undegraded":"2024-04-23T14:32:49.401159+0000","last_fullsized":"2024-04-23T14:32:49.401159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:13.096562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421993+0000","last_change":"2024-04-23T14:32:49.415576+0000","last_active":"2024-04-23T14:32:49.421993+0000","last_peered":"2024-04-23T14:32:49.421993+0000","last_clean":"2024-04-23T14:32:49.421993+0000","last_became_active":"2024-04-23T14:32:49.415107+0000","last_became_peered":"2024-04-23T14:32:49.415107+0000","last_unstale":"2024-04-23T14:32:49.421993+0000","last_undegraded":"2024-04-23T14:32:49.421993+0000","last_fullsized":"2024-04-23T14:32:49.421993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:21.969136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375434+0000","last_change":"2024-04-23T14:32:48.378186+0000","last_active":"2024-04-23T14:32:49.375434+0000","last_peered":"2024-04-23T14:32:49.375434+0000","last_clean":"2024-04-23T14:32:49.375434+0000","last_became_active":"2024-04-23T14:32:48.377889+0000","last_became_peered":"2024-04-23T14:32:48.377889+0000","last_unstale":"2024-04-23T14:32:49.375434+0000","last_undegraded":"2024-04-23T14:32:49.375434+0000","last_fullsized":"2024-04-23T14:32:49.375434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420363+0000","last_change":"2024-04-23T14:32:49.418062+0000","last_active":"2024-04-23T14:32:49.420363+0000","last_peered":"2024-04-23T14:32:49.420363+0000","last_clean":"2024-04-23T14:32:49.420363+0000","last_became_active":"2024-04-23T14:32:49.417335+0000","last_became_peered":"2024-04-23T14:32:49.417335+0000","last_unstale":"2024-04-23T14:32:49.420363+0000","last_undegraded":"2024-04-23T14:32:49.420363+0000","last_fullsized":"2024-04-23T14:32:49.420363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401048+0000","last_change":"2024-04-23T14:32:48.376375+0000","last_active":"2024-04-23T14:32:49.401048+0000","last_peered":"2024-04-23T14:32:49.401048+0000","last_clean":"2024-04-23T14:32:49.401048+0000","last_became_active":"2024-04-23T14:32:48.375807+0000","last_became_peered":"2024-04-23T14:32:48.375807+0000","last_unstale":"2024-04-23T14:32:49.401048+0000","last_undegraded":"2024-04-23T14:32:49.401048+0000","last_fullsized":"2024-04-23T14:32:49.401048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:21.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414650+0000","last_change":"2024-04-23T14:32:49.414650+0000","last_active":"2024-04-23T14:32:49.414650+0000","last_peered":"2024-04-23T14:32:49.414650+0000","last_clean":"2024-04-23T14:32:49.414650+0000","last_became_active":"2024-04-23T14:32:49.414352+0000","last_became_peered":"2024-04-23T14:32:49.414352+0000","last_unstale":"2024-04-23T14:32:49.414650+0000","last_undegraded":"2024-04-23T14:32:49.414650+0000","last_fullsized":"2024-04-23T14:32:49.414650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380697+0000","last_change":"2024-04-23T14:32:48.380295+0000","last_active":"2024-04-23T14:32:49.380697+0000","last_peered":"2024-04-23T14:32:49.380697+0000","last_clean":"2024-04-23T14:32:49.380697+0000","last_became_active":"2024-04-23T14:32:48.379968+0000","last_became_peered":"2024-04-23T14:32:48.379968+0000","last_unstale":"2024-04-23T14:32:49.380697+0000","last_undegraded":"2024-04-23T14:32:49.380697+0000","last_fullsized":"2024-04-23T14:32:49.380697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:28.545681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408527+0000","last_change":"2024-04-23T14:32:49.408527+0000","last_active":"2024-04-23T14:32:49.408527+0000","last_peered":"2024-04-23T14:32:49.408527+0000","last_clean":"2024-04-23T14:32:49.408527+0000","last_became_active":"2024-04-23T14:32:49.408214+0000","last_became_peered":"2024-04-23T14:32:49.408214+0000","last_unstale":"2024-04-23T14:32:49.408527+0000","last_undegraded":"2024-04-23T14:32:49.408527+0000","last_fullsized":"2024-04-23T14:32:49.408527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395886+0000","last_change":"2024-04-23T14:32:48.369769+0000","last_active":"2024-04-23T14:32:49.395886+0000","last_peered":"2024-04-23T14:32:49.395886+0000","last_clean":"2024-04-23T14:32:49.395886+0000","last_became_active":"2024-04-23T14:32:48.369112+0000","last_became_peered":"2024-04-23T14:32:48.369112+0000","last_unstale":"2024-04-23T14:32:49.395886+0000","last_undegraded":"2024-04-23T14:32:49.395886+0000","last_fullsized":"2024-04-23T14:32:49.395886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:14.196347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409092+0000","last_change":"2024-04-23T14:32:49.409092+0000","last_active":"2024-04-23T14:32:49.409092+0000","last_peered":"2024-04-23T14:32:49.409092+0000","last_clean":"2024-04-23T14:32:49.409092+0000","last_became_active":"2024-04-23T14:32:49.408769+0000","last_became_peered":"2024-04-23T14:32:49.408769+0000","last_unstale":"2024-04-23T14:32:49.409092+0000","last_undegraded":"2024-04-23T14:32:49.409092+0000","last_fullsized":"2024-04-23T14:32:49.409092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:07.688839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.383330+0000","last_change":"2024-04-23T14:32:48.379746+0000","last_active":"2024-04-23T14:32:49.383330+0000","last_peered":"2024-04-23T14:32:49.383330+0000","last_clean":"2024-04-23T14:32:49.383330+0000","last_became_active":"2024-04-23T14:32:48.378760+0000","last_became_peered":"2024-04-23T14:32:48.378760+0000","last_unstale":"2024-04-23T14:32:49.383330+0000","last_undegraded":"2024-04-23T14:32:49.383330+0000","last_fullsized":"2024-04-23T14:32:49.383330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383029+0000","last_change":"2024-04-23T14:32:49.383029+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383029+0000","last_undegraded":"2024-04-23T14:32:49.383029+0000","last_fullsized":"2024-04-23T14:32:49.383029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.378297+0000","last_change":"2024-04-23T14:32:48.379776+0000","last_active":"2024-04-23T14:32:49.378297+0000","last_peered":"2024-04-23T14:32:49.378297+0000","last_clean":"2024-04-23T14:32:49.378297+0000","last_became_active":"2024-04-23T14:32:48.379485+0000","last_became_peered":"2024-04-23T14:32:48.379485+0000","last_unstale":"2024-04-23T14:32:49.378297+0000","last_undegraded":"2024-04-23T14:32:49.378297+0000","last_fullsized":"2024-04-23T14:32:49.378297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:00.386998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418531+0000","last_change":"2024-04-23T14:32:49.409577+0000","last_active":"2024-04-23T14:32:49.418531+0000","last_peered":"2024-04-23T14:32:49.418531+0000","last_clean":"2024-04-23T14:32:49.418531+0000","last_became_active":"2024-04-23T14:32:49.409264+0000","last_became_peered":"2024-04-23T14:32:49.409264+0000","last_unstale":"2024-04-23T14:32:49.418531+0000","last_undegraded":"2024-04-23T14:32:49.418531+0000","last_fullsized":"2024-04-23T14:32:49.418531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401594+0000","last_change":"2024-04-23T14:32:48.376245+0000","last_active":"2024-04-23T14:32:49.401594+0000","last_peered":"2024-04-23T14:32:49.401594+0000","last_clean":"2024-04-23T14:32:49.401594+0000","last_became_active":"2024-04-23T14:32:48.375882+0000","last_became_peered":"2024-04-23T14:32:48.375882+0000","last_unstale":"2024-04-23T14:32:49.401594+0000","last_undegraded":"2024-04-23T14:32:49.401594+0000","last_fullsized":"2024-04-23T14:32:49.401594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:27.386064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.403473+0000","last_change":"2024-04-23T14:32:49.403473+0000","last_active":"2024-04-23T14:32:49.403473+0000","last_peered":"2024-04-23T14:32:49.403473+0000","last_clean":"2024-04-23T14:32:49.403473+0000","last_became_active":"2024-04-23T14:32:49.399541+0000","last_became_peered":"2024-04-23T14:32:49.399541+0000","last_unstale":"2024-04-23T14:32:49.403473+0000","last_undegraded":"2024-04-23T14:32:49.403473+0000","last_fullsized":"2024-04-23T14:32:49.403473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397792+0000","last_change":"2024-04-23T14:32:48.381791+0000","last_active":"2024-04-23T14:32:49.397792+0000","last_peered":"2024-04-23T14:32:49.397792+0000","last_clean":"2024-04-23T14:32:49.397792+0000","last_became_active":"2024-04-23T14:32:48.381531+0000","last_became_peered":"2024-04-23T14:32:48.381531+0000","last_unstale":"2024-04-23T14:32:49.397792+0000","last_undegraded":"2024-04-23T14:32:49.397792+0000","last_fullsized":"2024-04-23T14:32:49.397792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412273+0000","last_change":"2024-04-23T14:32:49.412273+0000","last_active":"2024-04-23T14:32:49.412273+0000","last_peered":"2024-04-23T14:32:49.412273+0000","last_clean":"2024-04-23T14:32:49.412273+0000","last_became_active":"2024-04-23T14:32:49.411939+0000","last_became_peered":"2024-04-23T14:32:49.411939+0000","last_unstale":"2024-04-23T14:32:49.412273+0000","last_undegraded":"2024-04-23T14:32:49.412273+0000","last_fullsized":"2024-04-23T14:32:49.412273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375257+0000","last_change":"2024-04-23T14:32:48.383719+0000","last_active":"2024-04-23T14:32:49.375257+0000","last_peered":"2024-04-23T14:32:49.375257+0000","last_clean":"2024-04-23T14:32:49.375257+0000","last_became_active":"2024-04-23T14:32:48.382885+0000","last_became_peered":"2024-04-23T14:32:48.382885+0000","last_unstale":"2024-04-23T14:32:49.375257+0000","last_undegraded":"2024-04-23T14:32:49.375257+0000","last_fullsized":"2024-04-23T14:32:49.375257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414486+0000","last_change":"2024-04-23T14:32:49.406690+0000","last_active":"2024-04-23T14:32:49.414486+0000","last_peered":"2024-04-23T14:32:49.414486+0000","last_clean":"2024-04-23T14:32:49.414486+0000","last_became_active":"2024-04-23T14:32:49.406431+0000","last_became_peered":"2024-04-23T14:32:49.406431+0000","last_unstale":"2024-04-23T14:32:49.414486+0000","last_undegraded":"2024-04-23T14:32:49.414486+0000","last_fullsized":"2024-04-23T14:32:49.414486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375967+0000","last_change":"2024-04-23T14:32:48.375851+0000","last_active":"2024-04-23T14:32:49.375967+0000","last_peered":"2024-04-23T14:32:49.375967+0000","last_clean":"2024-04-23T14:32:49.375967+0000","last_became_active":"2024-04-23T14:32:48.375572+0000","last_became_peered":"2024-04-23T14:32:48.375572+0000","last_unstale":"2024-04-23T14:32:49.375967+0000","last_undegraded":"2024-04-23T14:32:49.375967+0000","last_fullsized":"2024-04-23T14:32:49.375967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417434+0000","last_change":"2024-04-23T14:32:49.409809+0000","last_active":"2024-04-23T14:32:49.417434+0000","last_peered":"2024-04-23T14:32:49.417434+0000","last_clean":"2024-04-23T14:32:49.417434+0000","last_became_active":"2024-04-23T14:32:49.409554+0000","last_became_peered":"2024-04-23T14:32:49.409554+0000","last_unstale":"2024-04-23T14:32:49.417434+0000","last_undegraded":"2024-04-23T14:32:49.417434+0000","last_fullsized":"2024-04-23T14:32:49.417434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:39.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400370+0000","last_change":"2024-04-23T14:32:48.383194+0000","last_active":"2024-04-23T14:32:49.400370+0000","last_peered":"2024-04-23T14:32:49.400370+0000","last_clean":"2024-04-23T14:32:49.400370+0000","last_became_active":"2024-04-23T14:32:48.382867+0000","last_became_peered":"2024-04-23T14:32:48.382867+0000","last_unstale":"2024-04-23T14:32:49.400370+0000","last_undegraded":"2024-04-23T14:32:49.400370+0000","last_fullsized":"2024-04-23T14:32:49.400370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413412+0000","last_change":"2024-04-23T14:32:49.413412+0000","last_active":"2024-04-23T14:32:49.413412+0000","last_peered":"2024-04-23T14:32:49.413412+0000","last_clean":"2024-04-23T14:32:49.413412+0000","last_became_active":"2024-04-23T14:32:49.413066+0000","last_became_peered":"2024-04-23T14:32:49.413066+0000","last_unstale":"2024-04-23T14:32:49.413412+0000","last_undegraded":"2024-04-23T14:32:49.413412+0000","last_fullsized":"2024-04-23T14:32:49.413412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401743+0000","last_change":"2024-04-23T14:32:48.373256+0000","last_active":"2024-04-23T14:32:49.401743+0000","last_peered":"2024-04-23T14:32:49.401743+0000","last_clean":"2024-04-23T14:32:49.401743+0000","last_became_active":"2024-04-23T14:32:48.372720+0000","last_became_peered":"2024-04-23T14:32:48.372720+0000","last_unstale":"2024-04-23T14:32:49.401743+0000","last_undegraded":"2024-04-23T14:32:49.401743+0000","last_fullsized":"2024-04-23T14:32:49.401743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:04:15.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418820+0000","last_change":"2024-04-23T14:32:49.411786+0000","last_active":"2024-04-23T14:32:49.418820+0000","last_peered":"2024-04-23T14:32:49.418820+0000","last_clean":"2024-04-23T14:32:49.418820+0000","last_became_active":"2024-04-23T14:32:49.411468+0000","last_became_peered":"2024-04-23T14:32:49.411468+0000","last_unstale":"2024-04-23T14:32:49.418820+0000","last_undegraded":"2024-04-23T14:32:49.418820+0000","last_fullsized":"2024-04-23T14:32:49.418820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401645+0000","last_change":"2024-04-23T14:32:48.374462+0000","last_active":"2024-04-23T14:32:49.401645+0000","last_peered":"2024-04-23T14:32:49.401645+0000","last_clean":"2024-04-23T14:32:49.401645+0000","last_became_active":"2024-04-23T14:32:48.372060+0000","last_became_peered":"2024-04-23T14:32:48.372060+0000","last_unstale":"2024-04-23T14:32:49.401645+0000","last_undegraded":"2024-04-23T14:32:49.401645+0000","last_fullsized":"2024-04-23T14:32:49.401645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:40.111344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.406897+0000","last_change":"2024-04-23T14:32:49.406897+0000","last_active":"2024-04-23T14:32:49.406897+0000","last_peered":"2024-04-23T14:32:49.406897+0000","last_clean":"2024-04-23T14:32:49.406897+0000","last_became_active":"2024-04-23T14:32:49.406604+0000","last_became_peered":"2024-04-23T14:32:49.406604+0000","last_unstale":"2024-04-23T14:32:49.406897+0000","last_undegraded":"2024-04-23T14:32:49.406897+0000","last_fullsized":"2024-04-23T14:32:49.406897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:32.281665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401493+0000","last_change":"2024-04-23T14:32:48.382650+0000","last_active":"2024-04-23T14:32:49.401493+0000","last_peered":"2024-04-23T14:32:49.401493+0000","last_clean":"2024-04-23T14:32:49.401493+0000","last_became_active":"2024-04-23T14:32:48.382391+0000","last_became_peered":"2024-04-23T14:32:48.382391+0000","last_unstale":"2024-04-23T14:32:49.401493+0000","last_undegraded":"2024-04-23T14:32:49.401493+0000","last_fullsized":"2024-04-23T14:32:49.401493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:51.353237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410725+0000","last_change":"2024-04-23T14:32:49.410725+0000","last_active":"2024-04-23T14:32:49.410725+0000","last_peered":"2024-04-23T14:32:49.410725+0000","last_clean":"2024-04-23T14:32:49.410725+0000","last_became_active":"2024-04-23T14:32:49.410335+0000","last_became_peered":"2024-04-23T14:32:49.410335+0000","last_unstale":"2024-04-23T14:32:49.410725+0000","last_undegraded":"2024-04-23T14:32:49.410725+0000","last_fullsized":"2024-04-23T14:32:49.410725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:23:29.638232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396023+0000","last_change":"2024-04-23T14:32:48.372373+0000","last_active":"2024-04-23T14:32:49.396023+0000","last_peered":"2024-04-23T14:32:49.396023+0000","last_clean":"2024-04-23T14:32:49.396023+0000","last_became_active":"2024-04-23T14:32:48.372040+0000","last_became_peered":"2024-04-23T14:32:48.372040+0000","last_unstale":"2024-04-23T14:32:49.396023+0000","last_undegraded":"2024-04-23T14:32:49.396023+0000","last_fullsized":"2024-04-23T14:32:49.396023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:24.434665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413110+0000","last_change":"2024-04-23T14:32:49.403841+0000","last_active":"2024-04-23T14:32:49.413110+0000","last_peered":"2024-04-23T14:32:49.413110+0000","last_clean":"2024-04-23T14:32:49.413110+0000","last_became_active":"2024-04-23T14:32:49.403486+0000","last_became_peered":"2024-04-23T14:32:49.403486+0000","last_unstale":"2024-04-23T14:32:49.413110+0000","last_undegraded":"2024-04-23T14:32:49.413110+0000","last_fullsized":"2024-04-23T14:32:49.413110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:48.649791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.381995+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.381595+0000","last_became_peered":"2024-04-23T14:32:48.381595+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:56.520711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413093+0000","last_change":"2024-04-23T14:32:49.405667+0000","last_active":"2024-04-23T14:32:49.413093+0000","last_peered":"2024-04-23T14:32:49.413093+0000","last_clean":"2024-04-23T14:32:49.413093+0000","last_became_active":"2024-04-23T14:32:49.403620+0000","last_became_peered":"2024-04-23T14:32:49.403620+0000","last_unstale":"2024-04-23T14:32:49.413093+0000","last_undegraded":"2024-04-23T14:32:49.413093+0000","last_fullsized":"2024-04-23T14:32:49.413093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:25.894354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401019+0000","last_change":"2024-04-23T14:32:48.375422+0000","last_active":"2024-04-23T14:32:49.401019+0000","last_peered":"2024-04-23T14:32:49.401019+0000","last_clean":"2024-04-23T14:32:49.401019+0000","last_became_active":"2024-04-23T14:32:48.375126+0000","last_became_peered":"2024-04-23T14:32:48.375126+0000","last_unstale":"2024-04-23T14:32:49.401019+0000","last_undegraded":"2024-04-23T14:32:49.401019+0000","last_fullsized":"2024-04-23T14:32:49.401019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:18.218895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407040+0000","last_change":"2024-04-23T14:32:49.399201+0000","last_active":"2024-04-23T14:32:49.407040+0000","last_peered":"2024-04-23T14:32:49.407040+0000","last_clean":"2024-04-23T14:32:49.407040+0000","last_became_active":"2024-04-23T14:32:49.398682+0000","last_became_peered":"2024-04-23T14:32:49.398682+0000","last_unstale":"2024-04-23T14:32:49.407040+0000","last_undegraded":"2024-04-23T14:32:49.407040+0000","last_fullsized":"2024-04-23T14:32:49.407040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381590+0000","last_change":"2024-04-23T14:32:48.378313+0000","last_active":"2024-04-23T14:32:49.381590+0000","last_peered":"2024-04-23T14:32:49.381590+0000","last_clean":"2024-04-23T14:32:49.381590+0000","last_became_active":"2024-04-23T14:32:48.377976+0000","last_became_peered":"2024-04-23T14:32:48.377976+0000","last_unstale":"2024-04-23T14:32:49.381590+0000","last_undegraded":"2024-04-23T14:32:49.381590+0000","last_fullsized":"2024-04-23T14:32:49.381590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408098+0000","last_change":"2024-04-23T14:32:49.408098+0000","last_active":"2024-04-23T14:32:49.408098+0000","last_peered":"2024-04-23T14:32:49.408098+0000","last_clean":"2024-04-23T14:32:49.408098+0000","last_became_active":"2024-04-23T14:32:49.407785+0000","last_became_peered":"2024-04-23T14:32:49.407785+0000","last_unstale":"2024-04-23T14:32:49.408098+0000","last_undegraded":"2024-04-23T14:32:49.408098+0000","last_fullsized":"2024-04-23T14:32:49.408098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:12:49.629971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400540+0000","last_change":"2024-04-23T14:32:48.386689+0000","last_active":"2024-04-23T14:32:49.400540+0000","last_peered":"2024-04-23T14:32:49.400540+0000","last_clean":"2024-04-23T14:32:49.400540+0000","last_became_active":"2024-04-23T14:32:48.386272+0000","last_became_peered":"2024-04-23T14:32:48.386272+0000","last_unstale":"2024-04-23T14:32:49.400540+0000","last_undegraded":"2024-04-23T14:32:49.400540+0000","last_fullsized":"2024-04-23T14:32:49.400540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411036+0000","last_change":"2024-04-23T14:32:49.407292+0000","last_active":"2024-04-23T14:32:49.411036+0000","last_peered":"2024-04-23T14:32:49.411036+0000","last_clean":"2024-04-23T14:32:49.411036+0000","last_became_active":"2024-04-23T14:32:49.406967+0000","last_became_peered":"2024-04-23T14:32:49.406967+0000","last_unstale":"2024-04-23T14:32:49.411036+0000","last_undegraded":"2024-04-23T14:32:49.411036+0000","last_fullsized":"2024-04-23T14:32:49.411036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396038+0000","last_change":"2024-04-23T14:32:48.372538+0000","last_active":"2024-04-23T14:32:49.396038+0000","last_peered":"2024-04-23T14:32:49.396038+0000","last_clean":"2024-04-23T14:32:49.396038+0000","last_became_active":"2024-04-23T14:32:48.372276+0000","last_became_peered":"2024-04-23T14:32:48.372276+0000","last_unstale":"2024-04-23T14:32:49.396038+0000","last_undegraded":"2024-04-23T14:32:49.396038+0000","last_fullsized":"2024-04-23T14:32:49.396038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:57.601612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383866+0000","last_change":"2024-04-23T14:32:49.383866+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383866+0000","last_undegraded":"2024-04-23T14:32:49.383866+0000","last_fullsized":"2024-04-23T14:32:49.383866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397243+0000","last_change":"2024-04-23T14:32:48.373983+0000","last_active":"2024-04-23T14:32:49.397243+0000","last_peered":"2024-04-23T14:32:49.397243+0000","last_clean":"2024-04-23T14:32:49.397243+0000","last_became_active":"2024-04-23T14:32:48.373688+0000","last_became_peered":"2024-04-23T14:32:48.373688+0000","last_unstale":"2024-04-23T14:32:49.397243+0000","last_undegraded":"2024-04-23T14:32:49.397243+0000","last_fullsized":"2024-04-23T14:32:49.397243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411462+0000","last_change":"2024-04-23T14:32:49.398323+0000","last_active":"2024-04-23T14:32:49.411462+0000","last_peered":"2024-04-23T14:32:49.411462+0000","last_clean":"2024-04-23T14:32:49.411462+0000","last_became_active":"2024-04-23T14:32:49.398039+0000","last_became_peered":"2024-04-23T14:32:49.398039+0000","last_unstale":"2024-04-23T14:32:49.411462+0000","last_undegraded":"2024-04-23T14:32:49.411462+0000","last_fullsized":"2024-04-23T14:32:49.411462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396203+0000","last_change":"2024-04-23T14:32:48.384396+0000","last_active":"2024-04-23T14:32:49.396203+0000","last_peered":"2024-04-23T14:32:49.396203+0000","last_clean":"2024-04-23T14:32:49.396203+0000","last_became_active":"2024-04-23T14:32:48.383739+0000","last_became_peered":"2024-04-23T14:32:48.383739+0000","last_unstale":"2024-04-23T14:32:49.396203+0000","last_undegraded":"2024-04-23T14:32:49.396203+0000","last_fullsized":"2024-04-23T14:32:49.396203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401815+0000","last_change":"2024-04-23T14:32:49.396368+0000","last_active":"2024-04-23T14:32:49.401815+0000","last_peered":"2024-04-23T14:32:49.401815+0000","last_clean":"2024-04-23T14:32:49.401815+0000","last_became_active":"2024-04-23T14:32:49.395031+0000","last_became_peered":"2024-04-23T14:32:49.395031+0000","last_unstale":"2024-04-23T14:32:49.401815+0000","last_undegraded":"2024-04-23T14:32:49.401815+0000","last_fullsized":"2024-04-23T14:32:49.401815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397942+0000","last_change":"2024-04-23T14:32:48.387008+0000","last_active":"2024-04-23T14:32:49.397942+0000","last_peered":"2024-04-23T14:32:49.397942+0000","last_clean":"2024-04-23T14:32:49.397942+0000","last_became_active":"2024-04-23T14:32:48.386260+0000","last_became_peered":"2024-04-23T14:32:48.386260+0000","last_unstale":"2024-04-23T14:32:49.397942+0000","last_undegraded":"2024-04-23T14:32:49.397942+0000","last_fullsized":"2024-04-23T14:32:49.397942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.415060+0000","last_change":"2024-04-23T14:32:49.415060+0000","last_active":"2024-04-23T14:32:49.415060+0000","last_peered":"2024-04-23T14:32:49.415060+0000","last_clean":"2024-04-23T14:32:49.415060+0000","last_became_active":"2024-04-23T14:32:49.414707+0000","last_became_peered":"2024-04-23T14:32:49.414707+0000","last_unstale":"2024-04-23T14:32:49.415060+0000","last_undegraded":"2024-04-23T14:32:49.415060+0000","last_fullsized":"2024-04-23T14:32:49.415060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:02.059103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401039+0000","last_change":"2024-04-23T14:32:48.380600+0000","last_active":"2024-04-23T14:32:49.401039+0000","last_peered":"2024-04-23T14:32:49.401039+0000","last_clean":"2024-04-23T14:32:49.401039+0000","last_became_active":"2024-04-23T14:32:48.379900+0000","last_became_peered":"2024-04-23T14:32:48.379900+0000","last_unstale":"2024-04-23T14:32:49.401039+0000","last_undegraded":"2024-04-23T14:32:49.401039+0000","last_fullsized":"2024-04-23T14:32:49.401039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:31.916332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.425051+0000","last_change":"2024-04-23T14:32:49.417179+0000","last_active":"2024-04-23T14:32:49.425051+0000","last_peered":"2024-04-23T14:32:49.425051+0000","last_clean":"2024-04-23T14:32:49.425051+0000","last_became_active":"2024-04-23T14:32:49.414238+0000","last_became_peered":"2024-04-23T14:32:49.414238+0000","last_unstale":"2024-04-23T14:32:49.425051+0000","last_undegraded":"2024-04-23T14:32:49.425051+0000","last_fullsized":"2024-04-23T14:32:49.425051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375409+0000","last_change":"2024-04-23T14:32:48.382557+0000","last_active":"2024-04-23T14:32:49.375409+0000","last_peered":"2024-04-23T14:32:49.375409+0000","last_clean":"2024-04-23T14:32:49.375409+0000","last_became_active":"2024-04-23T14:32:48.381989+0000","last_became_peered":"2024-04-23T14:32:48.381989+0000","last_unstale":"2024-04-23T14:32:49.375409+0000","last_undegraded":"2024-04-23T14:32:49.375409+0000","last_fullsized":"2024-04-23T14:32:49.375409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411232+0000","last_change":"2024-04-23T14:32:49.406874+0000","last_active":"2024-04-23T14:32:49.411232+0000","last_peered":"2024-04-23T14:32:49.411232+0000","last_clean":"2024-04-23T14:32:49.411232+0000","last_became_active":"2024-04-23T14:32:49.406634+0000","last_became_peered":"2024-04-23T14:32:49.406634+0000","last_unstale":"2024-04-23T14:32:49.411232+0000","last_undegraded":"2024-04-23T14:32:49.411232+0000","last_fullsized":"2024-04-23T14:32:49.411232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380918+0000","last_change":"2024-04-23T14:32:48.381294+0000","last_active":"2024-04-23T14:32:49.380918+0000","last_peered":"2024-04-23T14:32:49.380918+0000","last_clean":"2024-04-23T14:32:49.380918+0000","last_became_active":"2024-04-23T14:32:48.380990+0000","last_became_peered":"2024-04-23T14:32:48.380990+0000","last_unstale":"2024-04-23T14:32:49.380918+0000","last_undegraded":"2024-04-23T14:32:49.380918+0000","last_fullsized":"2024-04-23T14:32:49.380918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391358+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:32:49.391358+0000","last_peered":"2024-04-23T14:32:49.391358+0000","last_clean":"2024-04-23T14:32:49.391358+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:32:49.391358+0000","last_undegraded":"2024-04-23T14:32:49.391358+0000","last_fullsized":"2024-04-23T14:32:49.391358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-04-23T14:32:49.382675+0000', 'last_change': '2024-04-23T14:32:49.382675+0000', 'last_active': '2024-04-23T14:32:48.338071+0000', 'last_peered': '2024-04-23T14:32:48.338071+0000', 'last_clean': '2024-04-23T14:32:48.338071+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:32:49.382675+0000', 'last_undegraded': '2024-04-23T14:32:49.382675+0000', 'last_fullsized': '2024-04-23T14:32:49.382675+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_clean_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-04-23T14:32:49.389894+0000', 'last_change': '2024-04-23T14:32:49.389894+0000', 'last_active': '2024-04-23T14:32:48.338071+0000', 'last_peered': '2024-04-23T14:32:48.338071+0000', 'last_clean': '2024-04-23T14:32:48.338071+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:32:49.389894+0000', 'last_undegraded': '2024-04-23T14:32:49.389894+0000', 'last_fullsized': '2024-04-23T14:32:49.389894+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_clean_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': ['7', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-04-23T14:32:49.389314+0000', 'last_change': '2024-04-23T14:32:49.389314+0000', 'last_active': '2024-04-23T14:32:48.338071+0000', 'last_peered': '2024-04-23T14:32:48.338071+0000', 'last_clean': '2024-04-23T14:32:48.338071+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:32:49.389314+0000', 'last_undegraded': '2024-04-23T14:32:49.389314+0000', 'last_fullsized': '2024-04-23T14:32:49.389314+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_clean_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-04-23T14:32:49.379405+0000', 'last_change': '2024-04-23T14:32:49.379405+0000', 'last_active': '2024-04-23T14:32:48.338071+0000', 'last_peered': '2024-04-23T14:32:48.338071+0000', 'last_clean': '2024-04-23T14:32:48.338071+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:32:49.379405+0000', 'last_undegraded': '2024-04-23T14:32:49.379405+0000', 'last_fullsized': '2024-04-23T14:32:49.379405+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_clean_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': ['3', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:32:54.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-23T14:32:54.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-04-23T14:32:49.378840+0000', 'last_change': '2024-04-23T14:32:49.378840+0000', 'last_active': '2024-04-23T14:32:48.338071+0000', 'last_peered': '2024-04-23T14:32:48.338071+0000', 'last_clean': '2024-04-23T14:32:48.338071+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:32:49.378840+0000', 'last_undegraded': '2024-04-23T14:32:49.378840+0000', 'last_fullsized': '2024-04-23T14:32:49.378840+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_clean_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': ['3', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:32:54.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-23T14:32:54.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-04-23T14:32:49.383029+0000', 'last_change': '2024-04-23T14:32:49.383029+0000', 'last_active': '2024-04-23T14:32:48.338071+0000', 'last_peered': '2024-04-23T14:32:48.338071+0000', 'last_clean': '2024-04-23T14:32:48.338071+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:32:49.383029+0000', 'last_undegraded': '2024-04-23T14:32:49.383029+0000', 'last_fullsized': '2024-04-23T14:32:49.383029+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_clean_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:32:54.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-23T14:32:54.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 32, 'state': 'creating+activating', 'last_fresh': '2024-04-23T14:32:49.383866+0000', 'last_change': '2024-04-23T14:32:49.383866+0000', 'last_active': '2024-04-23T14:32:48.338071+0000', 'last_peered': '2024-04-23T14:32:48.338071+0000', 'last_clean': '2024-04-23T14:32:48.338071+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:32:49.383866+0000', 'last_undegraded': '2024-04-23T14:32:49.383866+0000', 'last_fullsized': '2024-04-23T14:32:49.383866+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'last_clean_scrub_stamp': '2024-04-23T14:32:48.338071+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': ['0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-23T14:32:54.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-23T14:32:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.136+0000 7f3a5f000640 1 Processor -- start 2024-04-23T14:32:55.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.136+0000 7f3a5f000640 1 -- start start 2024-04-23T14:32:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.137+0000 7f3a5f000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a580cd760 0x7f3a580b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.137+0000 7f3a5f000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a580b4690 0x7f3a580cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.137+0000 7f3a5f000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a58160210 con 0x7f3a5815c3d0 2024-04-23T14:32:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.137+0000 7f3a5f000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3a581604f0 con 0x7f3a580b4690 2024-04-23T14:32:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.137+0000 7f3a5f000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3a581607d0 con 0x7f3a580cd760 2024-04-23T14:32:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.138+0000 7f3a5e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a580b4690 0x7f3a580cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.138+0000 7f3a5e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a580b4690 0x7f3a580cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45622/0 (socket says 172.21.15.59:45622) 2024-04-23T14:32:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.138+0000 7f3a5e600640 1 -- 172.21.15.59:0/1791711008 learned_addr learned my addr 172.21.15.59:0/1791711008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.138+0000 7f3a5d200640 1 --2- 172.21.15.59:0/1791711008 >> v2:172.21.15.76:3300/0 conn(0x7f3a580cd760 0x7f3a580b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.139+0000 7f3a5e600640 1 -- 172.21.15.59:0/1791711008 >> v2:172.21.15.76:3300/0 conn(0x7f3a580cd760 msgr2=0x7f3a580b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.139+0000 7f3a5e600640 1 --2- 172.21.15.59:0/1791711008 >> v2:172.21.15.76:3300/0 conn(0x7f3a580cd760 0x7f3a580b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.139+0000 7f3a5e600640 1 -- 172.21.15.59:0/1791711008 >> v1:172.21.15.76:6789/0 conn(0x7f3a5815c3d0 legacy=0x7f3a580cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.139+0000 7f3a5e600640 1 -- 172.21.15.59:0/1791711008 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a58160ab0 con 0x7f3a580b4690 2024-04-23T14:32:55.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.140+0000 7f3a5e600640 1 --2- 172.21.15.59:0/1791711008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580b4690 0x7f3a580cb7d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3a5806e620 tx=0x7f3a5002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9712ded5d37e525e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:55.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.140+0000 7f3a57e00640 1 -- 172.21.15.59:0/1791711008 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50036070 con 0x7f3a580b4690 2024-04-23T14:32:55.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.140+0000 7f3a57e00640 1 -- 172.21.15.59:0/1791711008 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a50031600 con 0x7f3a580b4690 2024-04-23T14:32:55.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.140+0000 7f3a57e00640 1 -- 172.21.15.59:0/1791711008 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500319d0 con 0x7f3a580b4690 2024-04-23T14:32:55.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 -- 172.21.15.59:0/1791711008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580b4690 msgr2=0x7f3a580cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 --2- 172.21.15.59:0/1791711008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580b4690 0x7f3a580cb7d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3a5806e620 tx=0x7f3a5002a730 comp rx=0 tx=0).stop 2024-04-23T14:32:55.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 -- 172.21.15.59:0/1791711008 shutdown_connections 2024-04-23T14:32:55.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 --2- 172.21.15.59:0/1791711008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580b4690 0x7f3a580cb7d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 --2- 172.21.15.59:0/1791711008 >> v2:172.21.15.76:3300/0 conn(0x7f3a580cd760 0x7f3a580b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 -- 172.21.15.59:0/1791711008 >> 172.21.15.59:0/1791711008 conn(0x7f3a580b21a0 msgr2=0x7f3a580b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:55.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 -- 172.21.15.59:0/1791711008 shutdown_connections 2024-04-23T14:32:55.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.141+0000 7f3a5f000640 1 -- 172.21.15.59:0/1791711008 wait complete. 2024-04-23T14:32:55.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.143+0000 7f3a5f000640 1 Processor -- start 2024-04-23T14:32:55.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.143+0000 7f3a5f000640 1 -- start start 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5f000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a580b4690 0x7f3a5815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580cd760 0x7f3a581e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5f000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5815c3d0 0x7f3a581e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5815c7b0 con 0x7f3a580cd760 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5f000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580b4ae0 con 0x7f3a580b4690 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5f000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5815c930 con 0x7f3a5815c3d0 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580cd760 0x7f3a581e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580cd760 0x7f3a581e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45624/0 (socket says 172.21.15.59:45624) 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.144+0000 7f3a5d200640 1 -- 172.21.15.59:0/3815068283 learned_addr learned my addr 172.21.15.59:0/3815068283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5dc00640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a580b4690 0x7f3a5815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5d200640 1 -- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5815c3d0 msgr2=0x7f3a581e7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5d200640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5815c3d0 0x7f3a581e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5d200640 1 -- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a580b4690 msgr2=0x7f3a5815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5d200640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a580b4690 0x7f3a5815cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5d200640 1 -- 172.21.15.59:0/3815068283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a580ce000 con 0x7f3a580cd760 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5dc00640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a580b4690 0x7f3a5815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:55.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a5d200640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580cd760 0x7f3a581e6e30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400d800 tx=0x7f3a4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:55.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.145+0000 7f3a55600640 1 -- 172.21.15.59:0/3815068283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400b6c0 con 0x7f3a580cd760 2024-04-23T14:32:55.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.146+0000 7f3a55600640 1 -- 172.21.15.59:0/3815068283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a44019070 con 0x7f3a580cd760 2024-04-23T14:32:55.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.146+0000 7f3a55600640 1 -- 172.21.15.59:0/3815068283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400ba80 con 0x7f3a580cd760 2024-04-23T14:32:55.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.146+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a580ce2e0 con 0x7f3a580cd760 2024-04-23T14:32:55.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.146+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5815cff0 con 0x7f3a580cd760 2024-04-23T14:32:55.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.147+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5804fd90 con 0x7f3a580cd760 2024-04-23T14:32:55.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.148+0000 7f3a55600640 1 -- 172.21.15.59:0/3815068283 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a4400bc20 con 0x7f3a580cd760 2024-04-23T14:32:55.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.149+0000 7f3a55600640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c8f0 0x7f3a3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.149+0000 7f3a55600640 1 -- 172.21.15.59:0/3815068283 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3a4408bff0 con 0x7f3a580cd760 2024-04-23T14:32:55.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.149+0000 7f3a5dc00640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c8f0 0x7f3a3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.150+0000 7f3a5dc00640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c8f0 0x7f3a3806edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0015e0 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:55.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.153+0000 7f3a55600640 1 -- 172.21.15.59:0/3815068283 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a4405a450 con 0x7f3a580cd760 2024-04-23T14:32:55.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.430+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3a581591c0 con 0x7f3a580cd760 2024-04-23T14:32:55.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.431+0000 7f3a55600640 1 -- 172.21.15.59:0/3815068283 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1102 (secure 0 0 0) 0x7f3a44059d50 con 0x7f3a580cd760 2024-04-23T14:32:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:55.432 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":114,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1713882673,"num_in_osds":8,"osd_in_since":1713882664,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":90},{"state_name":"creating+activating","count":7}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":590584,"bytes_used":229588992,"bytes_avail":772864524288,"bytes_total":773094113280,"inactive_pgs_ratio":0.072164945304393768,"write_bytes_sec":1699,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5205}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-23T14:32:32.365520+0000","services":{}},"progress_events":{}} 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.433+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c8f0 msgr2=0x7f3a3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.433+0000 7f3a5f000640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c8f0 0x7f3a3806edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0015e0 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.433+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580cd760 msgr2=0x7f3a581e6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.433+0000 7f3a5f000640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580cd760 0x7f3a581e6e30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400d800 tx=0x7f3a4400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 shutdown_connections 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c8f0 0x7f3a3806edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5815c3d0 0x7f3a581e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a580cd760 0x7f3a581e6e30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 --2- 172.21.15.59:0/3815068283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a580b4690 0x7f3a5815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 >> 172.21.15.59:0/3815068283 conn(0x7f3a580b21a0 msgr2=0x7f3a5806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:55.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 shutdown_connections 2024-04-23T14:32:55.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.434+0000 7f3a5f000640 1 -- 172.21.15.59:0/3815068283 wait complete. 2024-04-23T14:32:55.454 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-23T14:32:55.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:55.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.658+0000 7f6057200640 1 Processor -- start 2024-04-23T14:32:55.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.658+0000 7f6057200640 1 -- start start 2024-04-23T14:32:55.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.659+0000 7f6057200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f605015c3d0 0x7f60500cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6057200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6050160210 con 0x7f60500cd590 2024-04-23T14:32:55.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6057200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60501604f0 con 0x7f60500b4860 2024-04-23T14:32:55.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6057200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f60501607d0 con 0x7f605015c3d0 2024-04-23T14:32:55.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6055e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f605015c3d0 0x7f60500cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6055400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f60500cd590 0x7f60500b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35188/0 (socket says 172.21.15.59:35188) 2024-04-23T14:32:55.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6055400640 1 -- 172.21.15.59:0/2576447788 learned_addr learned my addr 172.21.15.59:0/2576447788 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:55.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6055e00640 1 -- 172.21.15.59:0/2576447788 >> v1:172.21.15.76:6789/0 conn(0x7f60500b4860 legacy=0x7f60500cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:55.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6055e00640 1 -- 172.21.15.59:0/2576447788 >> v1:172.21.15.59:6789/0 conn(0x7f60500cd590 legacy=0x7f60500b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.660+0000 7f6055e00640 1 -- 172.21.15.59:0/2576447788 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6050160ab0 con 0x7f605015c3d0 2024-04-23T14:32:55.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.662+0000 7f6055e00640 1 --2- 172.21.15.59:0/2576447788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f605015c3d0 0x7f60500cd050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60440097a0 tx=0x7f604402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0701bf4dec43ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:55.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.662+0000 7f6054a00640 1 -- 172.21.15.59:0/2576447788 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044037070 con 0x7f605015c3d0 2024-04-23T14:32:55.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.662+0000 7f6054a00640 1 -- 172.21.15.59:0/2576447788 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6044032440 con 0x7f605015c3d0 2024-04-23T14:32:55.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.662+0000 7f6054a00640 1 -- 172.21.15.59:0/2576447788 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044032810 con 0x7f605015c3d0 2024-04-23T14:32:55.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.662+0000 7f6057200640 1 -- 172.21.15.59:0/2576447788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f605015c3d0 msgr2=0x7f60500cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.662+0000 7f6057200640 1 --2- 172.21.15.59:0/2576447788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f605015c3d0 0x7f60500cd050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60440097a0 tx=0x7f604402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.663+0000 7f6057200640 1 -- 172.21.15.59:0/2576447788 shutdown_connections 2024-04-23T14:32:55.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.663+0000 7f6057200640 1 --2- 172.21.15.59:0/2576447788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f605015c3d0 0x7f60500cd050 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.663+0000 7f6057200640 1 -- 172.21.15.59:0/2576447788 >> 172.21.15.59:0/2576447788 conn(0x7f60500b21a0 msgr2=0x7f60500b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:55.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.663+0000 7f6057200640 1 -- 172.21.15.59:0/2576447788 shutdown_connections 2024-04-23T14:32:55.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.663+0000 7f6057200640 1 -- 172.21.15.59:0/2576447788 wait complete. 2024-04-23T14:32:55.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.664+0000 7f6057200640 1 Processor -- start 2024-04-23T14:32:55.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.664+0000 7f6057200640 1 -- start start 2024-04-23T14:32:55.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.665+0000 7f6057200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60500b4860 0x7f60500c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.665+0000 7f6057200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500cd590 0x7f60501e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.665+0000 7f6057200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f605015c3d0 0x7f605015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.665+0000 7f6057200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f605015c930 con 0x7f60500b4860 2024-04-23T14:32:55.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.665+0000 7f6057200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f605015c7b0 con 0x7f60500cd590 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.665+0000 7f6057200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f60500cbee0 con 0x7f605015c3d0 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60500b4860 0x7f60500c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60500b4860 0x7f60500c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45636/0 (socket says 172.21.15.59:45636) 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 -- 172.21.15.59:0/3037330965 learned_addr learned my addr 172.21.15.59:0/3037330965 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6056800640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f605015c3d0 0x7f605015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055400640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500cd590 0x7f60501e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 -- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f605015c3d0 msgr2=0x7f605015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f605015c3d0 0x7f605015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 -- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500cd590 msgr2=0x7f60501e60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500cd590 0x7f60501e60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.666+0000 7f6055e00640 1 -- 172.21.15.59:0/3037330965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605015d6e0 con 0x7f60500b4860 2024-04-23T14:32:55.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.667+0000 7f6055e00640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60500b4860 0x7f60500c77e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f60440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:55.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.667+0000 7f604a000640 1 -- 172.21.15.59:0/3037330965 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044037040 con 0x7f60500b4860 2024-04-23T14:32:55.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.667+0000 7f604a000640 1 -- 172.21.15.59:0/3037330965 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6044042070 con 0x7f60500b4860 2024-04-23T14:32:55.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.667+0000 7f604a000640 1 -- 172.21.15.59:0/3037330965 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044030a70 con 0x7f60500b4860 2024-04-23T14:32:55.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.667+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605015d9c0 con 0x7f60500b4860 2024-04-23T14:32:55.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.667+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605015df80 con 0x7f60500b4860 2024-04-23T14:32:55.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.668+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605004fd90 con 0x7f60500b4860 2024-04-23T14:32:55.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.669+0000 7f604a000640 1 -- 172.21.15.59:0/3037330965 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6044030c10 con 0x7f60500b4860 2024-04-23T14:32:55.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.670+0000 7f604a000640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f603406c8f0 0x7f603406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:55.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.670+0000 7f604a000640 1 -- 172.21.15.59:0/3037330965 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f60440ac640 con 0x7f60500b4860 2024-04-23T14:32:55.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.670+0000 7f6055400640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f603406c8f0 0x7f603406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:55.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.671+0000 7f6055400640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f603406c8f0 0x7f603406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f605006f9c0 tx=0x7f6040009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:55.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.675+0000 7f604a000640 1 -- 172.21.15.59:0/3037330965 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6044045360 con 0x7f60500b4860 2024-04-23T14:32:55.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.879+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f605006e400 con 0x7f603406c8f0 2024-04-23T14:32:55.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.887+0000 7f604a000640 1 -- 172.21.15.59:0/3037330965 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251346 (secure 0 0 0) 0x7f605006e400 con 0x7f603406c8f0 2024-04-23T14:32:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:55.893 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:32:55.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f603406c8f0 msgr2=0x7f603406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f603406c8f0 0x7f603406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f605006f9c0 tx=0x7f6040009040 comp rx=0 tx=0).stop 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60500b4860 msgr2=0x7f60500c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60500b4860 0x7f60500c77e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f60440046c0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 shutdown_connections 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f603406c8f0 0x7f603406edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f605015c3d0 0x7f605015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60500cd590 0x7f60501e60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 --2- 172.21.15.59:0/3037330965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60500b4860 0x7f60500c77e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.893+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 >> 172.21.15.59:0/3037330965 conn(0x7f60500b21a0 msgr2=0x7f60500cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.894+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 shutdown_connections 2024-04-23T14:32:55.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:55.894+0000 7f6057200640 1 -- 172.21.15.59:0/3037330965 wait complete. 2024-04-23T14:32:55.914 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-23T14:32:54.371142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590584,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224208,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750512,"statfs":{"total":773094113280,"available":772864524288,"internally_reserved":0,"allocated":3973120,"data_stored":2249000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.023568"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408276+0000","last_change":"2024-04-23T14:32:49.396796+0000","last_active":"2024-04-23T14:32:49.408276+0000","last_peered":"2024-04-23T14:32:49.408276+0000","last_clean":"2024-04-23T14:32:49.408276+0000","last_became_active":"2024-04-23T14:32:49.396505+0000","last_became_peered":"2024-04-23T14:32:49.396505+0000","last_unstale":"2024-04-23T14:32:49.408276+0000","last_undegraded":"2024-04-23T14:32:49.408276+0000","last_fullsized":"2024-04-23T14:32:49.408276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407907+0000","last_change":"2024-04-23T14:32:49.396404+0000","last_active":"2024-04-23T14:32:49.407907+0000","last_peered":"2024-04-23T14:32:49.407907+0000","last_clean":"2024-04-23T14:32:49.407907+0000","last_became_active":"2024-04-23T14:32:49.396103+0000","last_became_peered":"2024-04-23T14:32:49.396103+0000","last_unstale":"2024-04-23T14:32:49.407907+0000","last_undegraded":"2024-04-23T14:32:49.407907+0000","last_fullsized":"2024-04-23T14:32:49.407907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.382675+0000","last_change":"2024-04-23T14:32:49.382675+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.382675+0000","last_undegraded":"2024-04-23T14:32:49.382675+0000","last_fullsized":"2024-04-23T14:32:49.382675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418842+0000","last_change":"2024-04-23T14:32:49.412161+0000","last_active":"2024-04-23T14:32:49.418842+0000","last_peered":"2024-04-23T14:32:49.418842+0000","last_clean":"2024-04-23T14:32:49.418842+0000","last_became_active":"2024-04-23T14:32:49.411609+0000","last_became_peered":"2024-04-23T14:32:49.411609+0000","last_unstale":"2024-04-23T14:32:49.418842+0000","last_undegraded":"2024-04-23T14:32:49.418842+0000","last_fullsized":"2024-04-23T14:32:49.418842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411075+0000","last_change":"2024-04-23T14:32:49.411075+0000","last_active":"2024-04-23T14:32:49.411075+0000","last_peered":"2024-04-23T14:32:49.411075+0000","last_clean":"2024-04-23T14:32:49.411075+0000","last_became_active":"2024-04-23T14:32:49.410648+0000","last_became_peered":"2024-04-23T14:32:49.410648+0000","last_unstale":"2024-04-23T14:32:49.411075+0000","last_undegraded":"2024-04-23T14:32:49.411075+0000","last_fullsized":"2024-04-23T14:32:49.411075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:19.233384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412676+0000","last_change":"2024-04-23T14:32:49.396761+0000","last_active":"2024-04-23T14:32:49.412676+0000","last_peered":"2024-04-23T14:32:49.412676+0000","last_clean":"2024-04-23T14:32:49.412676+0000","last_became_active":"2024-04-23T14:32:49.396598+0000","last_became_peered":"2024-04-23T14:32:49.396598+0000","last_unstale":"2024-04-23T14:32:49.412676+0000","last_undegraded":"2024-04-23T14:32:49.412676+0000","last_fullsized":"2024-04-23T14:32:49.412676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407019+0000","last_change":"2024-04-23T14:32:49.398865+0000","last_active":"2024-04-23T14:32:49.407019+0000","last_peered":"2024-04-23T14:32:49.407019+0000","last_clean":"2024-04-23T14:32:49.407019+0000","last_became_active":"2024-04-23T14:32:49.398581+0000","last_became_peered":"2024-04-23T14:32:49.398581+0000","last_unstale":"2024-04-23T14:32:49.407019+0000","last_undegraded":"2024-04-23T14:32:49.407019+0000","last_fullsized":"2024-04-23T14:32:49.407019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409232+0000","last_change":"2024-04-23T14:32:49.409232+0000","last_active":"2024-04-23T14:32:49.409232+0000","last_peered":"2024-04-23T14:32:49.409232+0000","last_clean":"2024-04-23T14:32:49.409232+0000","last_became_active":"2024-04-23T14:32:49.408162+0000","last_became_peered":"2024-04-23T14:32:49.408162+0000","last_unstale":"2024-04-23T14:32:49.409232+0000","last_undegraded":"2024-04-23T14:32:49.409232+0000","last_fullsized":"2024-04-23T14:32:49.409232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.424075+0000","last_change":"2024-04-23T14:32:49.424075+0000","last_active":"2024-04-23T14:32:49.424075+0000","last_peered":"2024-04-23T14:32:49.424075+0000","last_clean":"2024-04-23T14:32:49.424075+0000","last_became_active":"2024-04-23T14:32:49.423714+0000","last_became_peered":"2024-04-23T14:32:49.423714+0000","last_unstale":"2024-04-23T14:32:49.424075+0000","last_undegraded":"2024-04-23T14:32:49.424075+0000","last_fullsized":"2024-04-23T14:32:49.424075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421421+0000","last_change":"2024-04-23T14:32:49.419419+0000","last_active":"2024-04-23T14:32:49.421421+0000","last_peered":"2024-04-23T14:32:49.421421+0000","last_clean":"2024-04-23T14:32:49.421421+0000","last_became_active":"2024-04-23T14:32:49.416916+0000","last_became_peered":"2024-04-23T14:32:49.416916+0000","last_unstale":"2024-04-23T14:32:49.421421+0000","last_undegraded":"2024-04-23T14:32:49.421421+0000","last_fullsized":"2024-04-23T14:32:49.421421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389894+0000","last_change":"2024-04-23T14:32:49.389894+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389894+0000","last_undegraded":"2024-04-23T14:32:49.389894+0000","last_fullsized":"2024-04-23T14:32:49.389894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414777+0000","last_change":"2024-04-23T14:32:49.414777+0000","last_active":"2024-04-23T14:32:49.414777+0000","last_peered":"2024-04-23T14:32:49.414777+0000","last_clean":"2024-04-23T14:32:49.414777+0000","last_became_active":"2024-04-23T14:32:49.414208+0000","last_became_peered":"2024-04-23T14:32:49.414208+0000","last_unstale":"2024-04-23T14:32:49.414777+0000","last_undegraded":"2024-04-23T14:32:49.414777+0000","last_fullsized":"2024-04-23T14:32:49.414777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.389314+0000","last_change":"2024-04-23T14:32:49.389314+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.389314+0000","last_undegraded":"2024-04-23T14:32:49.389314+0000","last_fullsized":"2024-04-23T14:32:49.389314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409664+0000","last_change":"2024-04-23T14:32:49.409664+0000","last_active":"2024-04-23T14:32:49.409664+0000","last_peered":"2024-04-23T14:32:49.409664+0000","last_clean":"2024-04-23T14:32:49.409664+0000","last_became_active":"2024-04-23T14:32:49.409369+0000","last_became_peered":"2024-04-23T14:32:49.409369+0000","last_unstale":"2024-04-23T14:32:49.409664+0000","last_undegraded":"2024-04-23T14:32:49.409664+0000","last_fullsized":"2024-04-23T14:32:49.409664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:53.618947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404125+0000","last_change":"2024-04-23T14:32:49.395881+0000","last_active":"2024-04-23T14:32:49.404125+0000","last_peered":"2024-04-23T14:32:49.404125+0000","last_clean":"2024-04-23T14:32:49.404125+0000","last_became_active":"2024-04-23T14:32:49.395640+0000","last_became_peered":"2024-04-23T14:32:49.395640+0000","last_unstale":"2024-04-23T14:32:49.404125+0000","last_undegraded":"2024-04-23T14:32:49.404125+0000","last_fullsized":"2024-04-23T14:32:49.404125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410041+0000","last_change":"2024-04-23T14:32:49.410041+0000","last_active":"2024-04-23T14:32:49.410041+0000","last_peered":"2024-04-23T14:32:49.410041+0000","last_clean":"2024-04-23T14:32:49.410041+0000","last_became_active":"2024-04-23T14:32:49.409759+0000","last_became_peered":"2024-04-23T14:32:49.409759+0000","last_unstale":"2024-04-23T14:32:49.410041+0000","last_undegraded":"2024-04-23T14:32:49.410041+0000","last_fullsized":"2024-04-23T14:32:49.410041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408902+0000","last_change":"2024-04-23T14:32:49.405810+0000","last_active":"2024-04-23T14:32:49.408902+0000","last_peered":"2024-04-23T14:32:49.408902+0000","last_clean":"2024-04-23T14:32:49.408902+0000","last_became_active":"2024-04-23T14:32:49.405283+0000","last_became_peered":"2024-04-23T14:32:49.405283+0000","last_unstale":"2024-04-23T14:32:49.408902+0000","last_undegraded":"2024-04-23T14:32:49.408902+0000","last_fullsized":"2024-04-23T14:32:49.408902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.379405+0000","last_change":"2024-04-23T14:32:49.379405+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.379405+0000","last_undegraded":"2024-04-23T14:32:49.379405+0000","last_fullsized":"2024-04-23T14:32:49.379405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.378840+0000","last_change":"2024-04-23T14:32:49.378840+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.378840+0000","last_undegraded":"2024-04-23T14:32:49.378840+0000","last_fullsized":"2024-04-23T14:32:49.378840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408152+0000","last_change":"2024-04-23T14:32:49.408152+0000","last_active":"2024-04-23T14:32:49.408152+0000","last_peered":"2024-04-23T14:32:49.408152+0000","last_clean":"2024-04-23T14:32:49.408152+0000","last_became_active":"2024-04-23T14:32:49.407758+0000","last_became_peered":"2024-04-23T14:32:49.407758+0000","last_unstale":"2024-04-23T14:32:49.408152+0000","last_undegraded":"2024-04-23T14:32:49.408152+0000","last_fullsized":"2024-04-23T14:32:49.408152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:42.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416792+0000","last_change":"2024-04-23T14:32:49.413362+0000","last_active":"2024-04-23T14:32:49.416792+0000","last_peered":"2024-04-23T14:32:49.416792+0000","last_clean":"2024-04-23T14:32:49.416792+0000","last_became_active":"2024-04-23T14:32:49.413082+0000","last_became_peered":"2024-04-23T14:32:49.413082+0000","last_unstale":"2024-04-23T14:32:49.416792+0000","last_undegraded":"2024-04-23T14:32:49.416792+0000","last_fullsized":"2024-04-23T14:32:49.416792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:57.535200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417468+0000","last_change":"2024-04-23T14:32:49.414460+0000","last_active":"2024-04-23T14:32:49.417468+0000","last_peered":"2024-04-23T14:32:49.417468+0000","last_clean":"2024-04-23T14:32:49.417468+0000","last_became_active":"2024-04-23T14:32:49.413658+0000","last_became_peered":"2024-04-23T14:32:49.413658+0000","last_unstale":"2024-04-23T14:32:49.417468+0000","last_undegraded":"2024-04-23T14:32:49.417468+0000","last_fullsized":"2024-04-23T14:32:49.417468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:32.704139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412931+0000","last_change":"2024-04-23T14:32:49.404657+0000","last_active":"2024-04-23T14:32:49.412931+0000","last_peered":"2024-04-23T14:32:49.412931+0000","last_clean":"2024-04-23T14:32:49.412931+0000","last_became_active":"2024-04-23T14:32:49.404322+0000","last_became_peered":"2024-04-23T14:32:49.404322+0000","last_unstale":"2024-04-23T14:32:49.412931+0000","last_undegraded":"2024-04-23T14:32:49.412931+0000","last_fullsized":"2024-04-23T14:32:49.412931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.416596+0000","last_change":"2024-04-23T14:32:49.408981+0000","last_active":"2024-04-23T14:32:49.416596+0000","last_peered":"2024-04-23T14:32:49.416596+0000","last_clean":"2024-04-23T14:32:49.416596+0000","last_became_active":"2024-04-23T14:32:49.408732+0000","last_became_peered":"2024-04-23T14:32:49.408732+0000","last_unstale":"2024-04-23T14:32:49.416596+0000","last_undegraded":"2024-04-23T14:32:49.416596+0000","last_fullsized":"2024-04-23T14:32:49.416596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419806+0000","last_change":"2024-04-23T14:32:49.411802+0000","last_active":"2024-04-23T14:32:49.419806+0000","last_peered":"2024-04-23T14:32:49.419806+0000","last_clean":"2024-04-23T14:32:49.419806+0000","last_became_active":"2024-04-23T14:32:49.411472+0000","last_became_peered":"2024-04-23T14:32:49.411472+0000","last_unstale":"2024-04-23T14:32:49.419806+0000","last_undegraded":"2024-04-23T14:32:49.419806+0000","last_fullsized":"2024-04-23T14:32:49.419806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408321+0000","last_change":"2024-04-23T14:32:49.408321+0000","last_active":"2024-04-23T14:32:49.408321+0000","last_peered":"2024-04-23T14:32:49.408321+0000","last_clean":"2024-04-23T14:32:49.408321+0000","last_became_active":"2024-04-23T14:32:49.407885+0000","last_became_peered":"2024-04-23T14:32:49.407885+0000","last_unstale":"2024-04-23T14:32:49.408321+0000","last_undegraded":"2024-04-23T14:32:49.408321+0000","last_fullsized":"2024-04-23T14:32:49.408321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420428+0000","last_change":"2024-04-23T14:32:49.418183+0000","last_active":"2024-04-23T14:32:49.420428+0000","last_peered":"2024-04-23T14:32:49.420428+0000","last_clean":"2024-04-23T14:32:49.420428+0000","last_became_active":"2024-04-23T14:32:49.417903+0000","last_became_peered":"2024-04-23T14:32:49.417903+0000","last_unstale":"2024-04-23T14:32:49.420428+0000","last_undegraded":"2024-04-23T14:32:49.420428+0000","last_fullsized":"2024-04-23T14:32:49.420428+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:12.318765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.404038+0000","last_change":"2024-04-23T14:32:49.396858+0000","last_active":"2024-04-23T14:32:49.404038+0000","last_peered":"2024-04-23T14:32:49.404038+0000","last_clean":"2024-04-23T14:32:49.404038+0000","last_became_active":"2024-04-23T14:32:49.396565+0000","last_became_peered":"2024-04-23T14:32:49.396565+0000","last_unstale":"2024-04-23T14:32:49.404038+0000","last_undegraded":"2024-04-23T14:32:49.404038+0000","last_fullsized":"2024-04-23T14:32:49.404038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.419860+0000","last_change":"2024-04-23T14:32:49.419860+0000","last_active":"2024-04-23T14:32:49.419860+0000","last_peered":"2024-04-23T14:32:49.419860+0000","last_clean":"2024-04-23T14:32:49.419860+0000","last_became_active":"2024-04-23T14:32:49.419578+0000","last_became_peered":"2024-04-23T14:32:49.419578+0000","last_unstale":"2024-04-23T14:32:49.419860+0000","last_undegraded":"2024-04-23T14:32:49.419860+0000","last_fullsized":"2024-04-23T14:32:49.419860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407177+0000","last_change":"2024-04-23T14:32:49.403367+0000","last_active":"2024-04-23T14:32:49.407177+0000","last_peered":"2024-04-23T14:32:49.407177+0000","last_clean":"2024-04-23T14:32:49.407177+0000","last_became_active":"2024-04-23T14:32:49.403043+0000","last_became_peered":"2024-04-23T14:32:49.403043+0000","last_unstale":"2024-04-23T14:32:49.407177+0000","last_undegraded":"2024-04-23T14:32:49.407177+0000","last_fullsized":"2024-04-23T14:32:49.407177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413518+0000","last_change":"2024-04-23T14:32:49.407909+0000","last_active":"2024-04-23T14:32:49.413518+0000","last_peered":"2024-04-23T14:32:49.413518+0000","last_clean":"2024-04-23T14:32:49.413518+0000","last_became_active":"2024-04-23T14:32:49.407653+0000","last_became_peered":"2024-04-23T14:32:49.407653+0000","last_unstale":"2024-04-23T14:32:49.413518+0000","last_undegraded":"2024-04-23T14:32:49.413518+0000","last_fullsized":"2024-04-23T14:32:49.413518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:07.378302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412291+0000","last_change":"2024-04-23T14:32:49.412291+0000","last_active":"2024-04-23T14:32:49.412291+0000","last_peered":"2024-04-23T14:32:49.412291+0000","last_clean":"2024-04-23T14:32:49.412291+0000","last_became_active":"2024-04-23T14:32:49.411966+0000","last_became_peered":"2024-04-23T14:32:49.411966+0000","last_unstale":"2024-04-23T14:32:49.412291+0000","last_undegraded":"2024-04-23T14:32:49.412291+0000","last_fullsized":"2024-04-23T14:32:49.412291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401707+0000","last_change":"2024-04-23T14:32:49.398365+0000","last_active":"2024-04-23T14:32:49.401707+0000","last_peered":"2024-04-23T14:32:49.401707+0000","last_clean":"2024-04-23T14:32:49.401707+0000","last_became_active":"2024-04-23T14:32:49.398036+0000","last_became_peered":"2024-04-23T14:32:49.398036+0000","last_unstale":"2024-04-23T14:32:49.401707+0000","last_undegraded":"2024-04-23T14:32:49.401707+0000","last_fullsized":"2024-04-23T14:32:49.401707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.380750+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.380399+0000","last_became_peered":"2024-04-23T14:32:48.380399+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:31.689650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414566+0000","last_change":"2024-04-23T14:32:49.411206+0000","last_active":"2024-04-23T14:32:49.414566+0000","last_peered":"2024-04-23T14:32:49.414566+0000","last_clean":"2024-04-23T14:32:49.414566+0000","last_became_active":"2024-04-23T14:32:49.410469+0000","last_became_peered":"2024-04-23T14:32:49.410469+0000","last_unstale":"2024-04-23T14:32:49.414566+0000","last_undegraded":"2024-04-23T14:32:49.414566+0000","last_fullsized":"2024-04-23T14:32:49.414566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395347+0000","last_change":"2024-04-23T14:32:48.378555+0000","last_active":"2024-04-23T14:32:49.395347+0000","last_peered":"2024-04-23T14:32:49.395347+0000","last_clean":"2024-04-23T14:32:49.395347+0000","last_became_active":"2024-04-23T14:32:48.377594+0000","last_became_peered":"2024-04-23T14:32:48.377594+0000","last_unstale":"2024-04-23T14:32:49.395347+0000","last_undegraded":"2024-04-23T14:32:49.395347+0000","last_fullsized":"2024-04-23T14:32:49.395347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418168+0000","last_change":"2024-04-23T14:32:49.418168+0000","last_active":"2024-04-23T14:32:49.418168+0000","last_peered":"2024-04-23T14:32:49.418168+0000","last_clean":"2024-04-23T14:32:49.418168+0000","last_became_active":"2024-04-23T14:32:49.417542+0000","last_became_peered":"2024-04-23T14:32:49.417542+0000","last_unstale":"2024-04-23T14:32:49.418168+0000","last_undegraded":"2024-04-23T14:32:49.418168+0000","last_fullsized":"2024-04-23T14:32:49.418168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381457+0000","last_change":"2024-04-23T14:32:48.383157+0000","last_active":"2024-04-23T14:32:49.381457+0000","last_peered":"2024-04-23T14:32:49.381457+0000","last_clean":"2024-04-23T14:32:49.381457+0000","last_became_active":"2024-04-23T14:32:48.382568+0000","last_became_peered":"2024-04-23T14:32:48.382568+0000","last_unstale":"2024-04-23T14:32:49.381457+0000","last_undegraded":"2024-04-23T14:32:49.381457+0000","last_fullsized":"2024-04-23T14:32:49.381457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:35.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420450+0000","last_change":"2024-04-23T14:32:49.418053+0000","last_active":"2024-04-23T14:32:49.420450+0000","last_peered":"2024-04-23T14:32:49.420450+0000","last_clean":"2024-04-23T14:32:49.420450+0000","last_became_active":"2024-04-23T14:32:49.417733+0000","last_became_peered":"2024-04-23T14:32:49.417733+0000","last_unstale":"2024-04-23T14:32:49.420450+0000","last_undegraded":"2024-04-23T14:32:49.420450+0000","last_fullsized":"2024-04-23T14:32:49.420450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391055+0000","last_change":"2024-04-23T14:32:48.374028+0000","last_active":"2024-04-23T14:32:49.391055+0000","last_peered":"2024-04-23T14:32:49.391055+0000","last_clean":"2024-04-23T14:32:49.391055+0000","last_became_active":"2024-04-23T14:32:48.372871+0000","last_became_peered":"2024-04-23T14:32:48.372871+0000","last_unstale":"2024-04-23T14:32:49.391055+0000","last_undegraded":"2024-04-23T14:32:49.391055+0000","last_fullsized":"2024-04-23T14:32:49.391055+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:14.297285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410412+0000","last_change":"2024-04-23T14:32:49.410412+0000","last_active":"2024-04-23T14:32:49.410412+0000","last_peered":"2024-04-23T14:32:49.410412+0000","last_clean":"2024-04-23T14:32:49.410412+0000","last_became_active":"2024-04-23T14:32:49.409645+0000","last_became_peered":"2024-04-23T14:32:49.409645+0000","last_unstale":"2024-04-23T14:32:49.410412+0000","last_undegraded":"2024-04-23T14:32:49.410412+0000","last_fullsized":"2024-04-23T14:32:49.410412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401440+0000","last_change":"2024-04-23T14:32:48.381407+0000","last_active":"2024-04-23T14:32:49.401440+0000","last_peered":"2024-04-23T14:32:49.401440+0000","last_clean":"2024-04-23T14:32:49.401440+0000","last_became_active":"2024-04-23T14:32:48.381122+0000","last_became_peered":"2024-04-23T14:32:48.381122+0000","last_unstale":"2024-04-23T14:32:49.401440+0000","last_undegraded":"2024-04-23T14:32:49.401440+0000","last_fullsized":"2024-04-23T14:32:49.401440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:16:40.112260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408197+0000","last_change":"2024-04-23T14:32:49.400326+0000","last_active":"2024-04-23T14:32:49.408197+0000","last_peered":"2024-04-23T14:32:49.408197+0000","last_clean":"2024-04-23T14:32:49.408197+0000","last_became_active":"2024-04-23T14:32:49.399716+0000","last_became_peered":"2024-04-23T14:32:49.399716+0000","last_unstale":"2024-04-23T14:32:49.408197+0000","last_undegraded":"2024-04-23T14:32:49.408197+0000","last_fullsized":"2024-04-23T14:32:49.408197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401159+0000","last_change":"2024-04-23T14:32:48.380347+0000","last_active":"2024-04-23T14:32:49.401159+0000","last_peered":"2024-04-23T14:32:49.401159+0000","last_clean":"2024-04-23T14:32:49.401159+0000","last_became_active":"2024-04-23T14:32:48.379799+0000","last_became_peered":"2024-04-23T14:32:48.379799+0000","last_unstale":"2024-04-23T14:32:49.401159+0000","last_undegraded":"2024-04-23T14:32:49.401159+0000","last_fullsized":"2024-04-23T14:32:49.401159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:13.096562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.421993+0000","last_change":"2024-04-23T14:32:49.415576+0000","last_active":"2024-04-23T14:32:49.421993+0000","last_peered":"2024-04-23T14:32:49.421993+0000","last_clean":"2024-04-23T14:32:49.421993+0000","last_became_active":"2024-04-23T14:32:49.415107+0000","last_became_peered":"2024-04-23T14:32:49.415107+0000","last_unstale":"2024-04-23T14:32:49.421993+0000","last_undegraded":"2024-04-23T14:32:49.421993+0000","last_fullsized":"2024-04-23T14:32:49.421993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:21.969136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375434+0000","last_change":"2024-04-23T14:32:48.378186+0000","last_active":"2024-04-23T14:32:49.375434+0000","last_peered":"2024-04-23T14:32:49.375434+0000","last_clean":"2024-04-23T14:32:49.375434+0000","last_became_active":"2024-04-23T14:32:48.377889+0000","last_became_peered":"2024-04-23T14:32:48.377889+0000","last_unstale":"2024-04-23T14:32:49.375434+0000","last_undegraded":"2024-04-23T14:32:49.375434+0000","last_fullsized":"2024-04-23T14:32:49.375434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.420363+0000","last_change":"2024-04-23T14:32:49.418062+0000","last_active":"2024-04-23T14:32:49.420363+0000","last_peered":"2024-04-23T14:32:49.420363+0000","last_clean":"2024-04-23T14:32:49.420363+0000","last_became_active":"2024-04-23T14:32:49.417335+0000","last_became_peered":"2024-04-23T14:32:49.417335+0000","last_unstale":"2024-04-23T14:32:49.420363+0000","last_undegraded":"2024-04-23T14:32:49.420363+0000","last_fullsized":"2024-04-23T14:32:49.420363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401048+0000","last_change":"2024-04-23T14:32:48.376375+0000","last_active":"2024-04-23T14:32:49.401048+0000","last_peered":"2024-04-23T14:32:49.401048+0000","last_clean":"2024-04-23T14:32:49.401048+0000","last_became_active":"2024-04-23T14:32:48.375807+0000","last_became_peered":"2024-04-23T14:32:48.375807+0000","last_unstale":"2024-04-23T14:32:49.401048+0000","last_undegraded":"2024-04-23T14:32:49.401048+0000","last_fullsized":"2024-04-23T14:32:49.401048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:21.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414650+0000","last_change":"2024-04-23T14:32:49.414650+0000","last_active":"2024-04-23T14:32:49.414650+0000","last_peered":"2024-04-23T14:32:49.414650+0000","last_clean":"2024-04-23T14:32:49.414650+0000","last_became_active":"2024-04-23T14:32:49.414352+0000","last_became_peered":"2024-04-23T14:32:49.414352+0000","last_unstale":"2024-04-23T14:32:49.414650+0000","last_undegraded":"2024-04-23T14:32:49.414650+0000","last_fullsized":"2024-04-23T14:32:49.414650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380697+0000","last_change":"2024-04-23T14:32:48.380295+0000","last_active":"2024-04-23T14:32:49.380697+0000","last_peered":"2024-04-23T14:32:49.380697+0000","last_clean":"2024-04-23T14:32:49.380697+0000","last_became_active":"2024-04-23T14:32:48.379968+0000","last_became_peered":"2024-04-23T14:32:48.379968+0000","last_unstale":"2024-04-23T14:32:49.380697+0000","last_undegraded":"2024-04-23T14:32:49.380697+0000","last_fullsized":"2024-04-23T14:32:49.380697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:28.545681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408527+0000","last_change":"2024-04-23T14:32:49.408527+0000","last_active":"2024-04-23T14:32:49.408527+0000","last_peered":"2024-04-23T14:32:49.408527+0000","last_clean":"2024-04-23T14:32:49.408527+0000","last_became_active":"2024-04-23T14:32:49.408214+0000","last_became_peered":"2024-04-23T14:32:49.408214+0000","last_unstale":"2024-04-23T14:32:49.408527+0000","last_undegraded":"2024-04-23T14:32:49.408527+0000","last_fullsized":"2024-04-23T14:32:49.408527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.395886+0000","last_change":"2024-04-23T14:32:48.369769+0000","last_active":"2024-04-23T14:32:49.395886+0000","last_peered":"2024-04-23T14:32:49.395886+0000","last_clean":"2024-04-23T14:32:49.395886+0000","last_became_active":"2024-04-23T14:32:48.369112+0000","last_became_peered":"2024-04-23T14:32:48.369112+0000","last_unstale":"2024-04-23T14:32:49.395886+0000","last_undegraded":"2024-04-23T14:32:49.395886+0000","last_fullsized":"2024-04-23T14:32:49.395886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:14.196347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.409092+0000","last_change":"2024-04-23T14:32:49.409092+0000","last_active":"2024-04-23T14:32:49.409092+0000","last_peered":"2024-04-23T14:32:49.409092+0000","last_clean":"2024-04-23T14:32:49.409092+0000","last_became_active":"2024-04-23T14:32:49.408769+0000","last_became_peered":"2024-04-23T14:32:49.408769+0000","last_unstale":"2024-04-23T14:32:49.409092+0000","last_undegraded":"2024-04-23T14:32:49.409092+0000","last_fullsized":"2024-04-23T14:32:49.409092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:07.688839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.383330+0000","last_change":"2024-04-23T14:32:48.379746+0000","last_active":"2024-04-23T14:32:49.383330+0000","last_peered":"2024-04-23T14:32:49.383330+0000","last_clean":"2024-04-23T14:32:49.383330+0000","last_became_active":"2024-04-23T14:32:48.378760+0000","last_became_peered":"2024-04-23T14:32:48.378760+0000","last_unstale":"2024-04-23T14:32:49.383330+0000","last_undegraded":"2024-04-23T14:32:49.383330+0000","last_fullsized":"2024-04-23T14:32:49.383330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383029+0000","last_change":"2024-04-23T14:32:49.383029+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383029+0000","last_undegraded":"2024-04-23T14:32:49.383029+0000","last_fullsized":"2024-04-23T14:32:49.383029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.378297+0000","last_change":"2024-04-23T14:32:48.379776+0000","last_active":"2024-04-23T14:32:49.378297+0000","last_peered":"2024-04-23T14:32:49.378297+0000","last_clean":"2024-04-23T14:32:49.378297+0000","last_became_active":"2024-04-23T14:32:48.379485+0000","last_became_peered":"2024-04-23T14:32:48.379485+0000","last_unstale":"2024-04-23T14:32:49.378297+0000","last_undegraded":"2024-04-23T14:32:49.378297+0000","last_fullsized":"2024-04-23T14:32:49.378297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:00.386998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418531+0000","last_change":"2024-04-23T14:32:49.409577+0000","last_active":"2024-04-23T14:32:49.418531+0000","last_peered":"2024-04-23T14:32:49.418531+0000","last_clean":"2024-04-23T14:32:49.418531+0000","last_became_active":"2024-04-23T14:32:49.409264+0000","last_became_peered":"2024-04-23T14:32:49.409264+0000","last_unstale":"2024-04-23T14:32:49.418531+0000","last_undegraded":"2024-04-23T14:32:49.418531+0000","last_fullsized":"2024-04-23T14:32:49.418531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401594+0000","last_change":"2024-04-23T14:32:48.376245+0000","last_active":"2024-04-23T14:32:49.401594+0000","last_peered":"2024-04-23T14:32:49.401594+0000","last_clean":"2024-04-23T14:32:49.401594+0000","last_became_active":"2024-04-23T14:32:48.375882+0000","last_became_peered":"2024-04-23T14:32:48.375882+0000","last_unstale":"2024-04-23T14:32:49.401594+0000","last_undegraded":"2024-04-23T14:32:49.401594+0000","last_fullsized":"2024-04-23T14:32:49.401594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:27.386064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.403473+0000","last_change":"2024-04-23T14:32:49.403473+0000","last_active":"2024-04-23T14:32:49.403473+0000","last_peered":"2024-04-23T14:32:49.403473+0000","last_clean":"2024-04-23T14:32:49.403473+0000","last_became_active":"2024-04-23T14:32:49.399541+0000","last_became_peered":"2024-04-23T14:32:49.399541+0000","last_unstale":"2024-04-23T14:32:49.403473+0000","last_undegraded":"2024-04-23T14:32:49.403473+0000","last_fullsized":"2024-04-23T14:32:49.403473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:13.686318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397792+0000","last_change":"2024-04-23T14:32:48.381791+0000","last_active":"2024-04-23T14:32:49.397792+0000","last_peered":"2024-04-23T14:32:49.397792+0000","last_clean":"2024-04-23T14:32:49.397792+0000","last_became_active":"2024-04-23T14:32:48.381531+0000","last_became_peered":"2024-04-23T14:32:48.381531+0000","last_unstale":"2024-04-23T14:32:49.397792+0000","last_undegraded":"2024-04-23T14:32:49.397792+0000","last_fullsized":"2024-04-23T14:32:49.397792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.412273+0000","last_change":"2024-04-23T14:32:49.412273+0000","last_active":"2024-04-23T14:32:49.412273+0000","last_peered":"2024-04-23T14:32:49.412273+0000","last_clean":"2024-04-23T14:32:49.412273+0000","last_became_active":"2024-04-23T14:32:49.411939+0000","last_became_peered":"2024-04-23T14:32:49.411939+0000","last_unstale":"2024-04-23T14:32:49.412273+0000","last_undegraded":"2024-04-23T14:32:49.412273+0000","last_fullsized":"2024-04-23T14:32:49.412273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375257+0000","last_change":"2024-04-23T14:32:48.383719+0000","last_active":"2024-04-23T14:32:49.375257+0000","last_peered":"2024-04-23T14:32:49.375257+0000","last_clean":"2024-04-23T14:32:49.375257+0000","last_became_active":"2024-04-23T14:32:48.382885+0000","last_became_peered":"2024-04-23T14:32:48.382885+0000","last_unstale":"2024-04-23T14:32:49.375257+0000","last_undegraded":"2024-04-23T14:32:49.375257+0000","last_fullsized":"2024-04-23T14:32:49.375257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.414486+0000","last_change":"2024-04-23T14:32:49.406690+0000","last_active":"2024-04-23T14:32:49.414486+0000","last_peered":"2024-04-23T14:32:49.414486+0000","last_clean":"2024-04-23T14:32:49.414486+0000","last_became_active":"2024-04-23T14:32:49.406431+0000","last_became_peered":"2024-04-23T14:32:49.406431+0000","last_unstale":"2024-04-23T14:32:49.414486+0000","last_undegraded":"2024-04-23T14:32:49.414486+0000","last_fullsized":"2024-04-23T14:32:49.414486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:58.616101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375967+0000","last_change":"2024-04-23T14:32:48.375851+0000","last_active":"2024-04-23T14:32:49.375967+0000","last_peered":"2024-04-23T14:32:49.375967+0000","last_clean":"2024-04-23T14:32:49.375967+0000","last_became_active":"2024-04-23T14:32:48.375572+0000","last_became_peered":"2024-04-23T14:32:48.375572+0000","last_unstale":"2024-04-23T14:32:49.375967+0000","last_undegraded":"2024-04-23T14:32:49.375967+0000","last_fullsized":"2024-04-23T14:32:49.375967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.417434+0000","last_change":"2024-04-23T14:32:49.409809+0000","last_active":"2024-04-23T14:32:49.417434+0000","last_peered":"2024-04-23T14:32:49.417434+0000","last_clean":"2024-04-23T14:32:49.417434+0000","last_became_active":"2024-04-23T14:32:49.409554+0000","last_became_peered":"2024-04-23T14:32:49.409554+0000","last_unstale":"2024-04-23T14:32:49.417434+0000","last_undegraded":"2024-04-23T14:32:49.417434+0000","last_fullsized":"2024-04-23T14:32:49.417434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:39.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400370+0000","last_change":"2024-04-23T14:32:48.383194+0000","last_active":"2024-04-23T14:32:49.400370+0000","last_peered":"2024-04-23T14:32:49.400370+0000","last_clean":"2024-04-23T14:32:49.400370+0000","last_became_active":"2024-04-23T14:32:48.382867+0000","last_became_peered":"2024-04-23T14:32:48.382867+0000","last_unstale":"2024-04-23T14:32:49.400370+0000","last_undegraded":"2024-04-23T14:32:49.400370+0000","last_fullsized":"2024-04-23T14:32:49.400370+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413412+0000","last_change":"2024-04-23T14:32:49.413412+0000","last_active":"2024-04-23T14:32:49.413412+0000","last_peered":"2024-04-23T14:32:49.413412+0000","last_clean":"2024-04-23T14:32:49.413412+0000","last_became_active":"2024-04-23T14:32:49.413066+0000","last_became_peered":"2024-04-23T14:32:49.413066+0000","last_unstale":"2024-04-23T14:32:49.413412+0000","last_undegraded":"2024-04-23T14:32:49.413412+0000","last_fullsized":"2024-04-23T14:32:49.413412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:21:29.871890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401743+0000","last_change":"2024-04-23T14:32:48.373256+0000","last_active":"2024-04-23T14:32:49.401743+0000","last_peered":"2024-04-23T14:32:49.401743+0000","last_clean":"2024-04-23T14:32:49.401743+0000","last_became_active":"2024-04-23T14:32:48.372720+0000","last_became_peered":"2024-04-23T14:32:48.372720+0000","last_unstale":"2024-04-23T14:32:49.401743+0000","last_undegraded":"2024-04-23T14:32:49.401743+0000","last_fullsized":"2024-04-23T14:32:49.401743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:04:15.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.418820+0000","last_change":"2024-04-23T14:32:49.411786+0000","last_active":"2024-04-23T14:32:49.418820+0000","last_peered":"2024-04-23T14:32:49.418820+0000","last_clean":"2024-04-23T14:32:49.418820+0000","last_became_active":"2024-04-23T14:32:49.411468+0000","last_became_peered":"2024-04-23T14:32:49.411468+0000","last_unstale":"2024-04-23T14:32:49.418820+0000","last_undegraded":"2024-04-23T14:32:49.418820+0000","last_fullsized":"2024-04-23T14:32:49.418820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401645+0000","last_change":"2024-04-23T14:32:48.374462+0000","last_active":"2024-04-23T14:32:49.401645+0000","last_peered":"2024-04-23T14:32:49.401645+0000","last_clean":"2024-04-23T14:32:49.401645+0000","last_became_active":"2024-04-23T14:32:48.372060+0000","last_became_peered":"2024-04-23T14:32:48.372060+0000","last_unstale":"2024-04-23T14:32:49.401645+0000","last_undegraded":"2024-04-23T14:32:49.401645+0000","last_fullsized":"2024-04-23T14:32:49.401645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:40.111344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.406897+0000","last_change":"2024-04-23T14:32:49.406897+0000","last_active":"2024-04-23T14:32:49.406897+0000","last_peered":"2024-04-23T14:32:49.406897+0000","last_clean":"2024-04-23T14:32:49.406897+0000","last_became_active":"2024-04-23T14:32:49.406604+0000","last_became_peered":"2024-04-23T14:32:49.406604+0000","last_unstale":"2024-04-23T14:32:49.406897+0000","last_undegraded":"2024-04-23T14:32:49.406897+0000","last_fullsized":"2024-04-23T14:32:49.406897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:32.281665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401493+0000","last_change":"2024-04-23T14:32:48.382650+0000","last_active":"2024-04-23T14:32:49.401493+0000","last_peered":"2024-04-23T14:32:49.401493+0000","last_clean":"2024-04-23T14:32:49.401493+0000","last_became_active":"2024-04-23T14:32:48.382391+0000","last_became_peered":"2024-04-23T14:32:48.382391+0000","last_unstale":"2024-04-23T14:32:49.401493+0000","last_undegraded":"2024-04-23T14:32:49.401493+0000","last_fullsized":"2024-04-23T14:32:49.401493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:51.353237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.410725+0000","last_change":"2024-04-23T14:32:49.410725+0000","last_active":"2024-04-23T14:32:49.410725+0000","last_peered":"2024-04-23T14:32:49.410725+0000","last_clean":"2024-04-23T14:32:49.410725+0000","last_became_active":"2024-04-23T14:32:49.410335+0000","last_became_peered":"2024-04-23T14:32:49.410335+0000","last_unstale":"2024-04-23T14:32:49.410725+0000","last_undegraded":"2024-04-23T14:32:49.410725+0000","last_fullsized":"2024-04-23T14:32:49.410725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:23:29.638232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396023+0000","last_change":"2024-04-23T14:32:48.372373+0000","last_active":"2024-04-23T14:32:49.396023+0000","last_peered":"2024-04-23T14:32:49.396023+0000","last_clean":"2024-04-23T14:32:49.396023+0000","last_became_active":"2024-04-23T14:32:48.372040+0000","last_became_peered":"2024-04-23T14:32:48.372040+0000","last_unstale":"2024-04-23T14:32:49.396023+0000","last_undegraded":"2024-04-23T14:32:49.396023+0000","last_fullsized":"2024-04-23T14:32:49.396023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:24.434665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413110+0000","last_change":"2024-04-23T14:32:49.403841+0000","last_active":"2024-04-23T14:32:49.413110+0000","last_peered":"2024-04-23T14:32:49.413110+0000","last_clean":"2024-04-23T14:32:49.413110+0000","last_became_active":"2024-04-23T14:32:49.403486+0000","last_became_peered":"2024-04-23T14:32:49.403486+0000","last_unstale":"2024-04-23T14:32:49.413110+0000","last_undegraded":"2024-04-23T14:32:49.413110+0000","last_fullsized":"2024-04-23T14:32:49.413110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:48.649791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380488+0000","last_change":"2024-04-23T14:32:48.381995+0000","last_active":"2024-04-23T14:32:49.380488+0000","last_peered":"2024-04-23T14:32:49.380488+0000","last_clean":"2024-04-23T14:32:49.380488+0000","last_became_active":"2024-04-23T14:32:48.381595+0000","last_became_peered":"2024-04-23T14:32:48.381595+0000","last_unstale":"2024-04-23T14:32:49.380488+0000","last_undegraded":"2024-04-23T14:32:49.380488+0000","last_fullsized":"2024-04-23T14:32:49.380488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:56.520711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.413093+0000","last_change":"2024-04-23T14:32:49.405667+0000","last_active":"2024-04-23T14:32:49.413093+0000","last_peered":"2024-04-23T14:32:49.413093+0000","last_clean":"2024-04-23T14:32:49.413093+0000","last_became_active":"2024-04-23T14:32:49.403620+0000","last_became_peered":"2024-04-23T14:32:49.403620+0000","last_unstale":"2024-04-23T14:32:49.413093+0000","last_undegraded":"2024-04-23T14:32:49.413093+0000","last_fullsized":"2024-04-23T14:32:49.413093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:25.894354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401019+0000","last_change":"2024-04-23T14:32:48.375422+0000","last_active":"2024-04-23T14:32:49.401019+0000","last_peered":"2024-04-23T14:32:49.401019+0000","last_clean":"2024-04-23T14:32:49.401019+0000","last_became_active":"2024-04-23T14:32:48.375126+0000","last_became_peered":"2024-04-23T14:32:48.375126+0000","last_unstale":"2024-04-23T14:32:49.401019+0000","last_undegraded":"2024-04-23T14:32:49.401019+0000","last_fullsized":"2024-04-23T14:32:49.401019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:18.218895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.407040+0000","last_change":"2024-04-23T14:32:49.399201+0000","last_active":"2024-04-23T14:32:49.407040+0000","last_peered":"2024-04-23T14:32:49.407040+0000","last_clean":"2024-04-23T14:32:49.407040+0000","last_became_active":"2024-04-23T14:32:49.398682+0000","last_became_peered":"2024-04-23T14:32:49.398682+0000","last_unstale":"2024-04-23T14:32:49.407040+0000","last_undegraded":"2024-04-23T14:32:49.407040+0000","last_fullsized":"2024-04-23T14:32:49.407040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.381590+0000","last_change":"2024-04-23T14:32:48.378313+0000","last_active":"2024-04-23T14:32:49.381590+0000","last_peered":"2024-04-23T14:32:49.381590+0000","last_clean":"2024-04-23T14:32:49.381590+0000","last_became_active":"2024-04-23T14:32:48.377976+0000","last_became_peered":"2024-04-23T14:32:48.377976+0000","last_unstale":"2024-04-23T14:32:49.381590+0000","last_undegraded":"2024-04-23T14:32:49.381590+0000","last_fullsized":"2024-04-23T14:32:49.381590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.408098+0000","last_change":"2024-04-23T14:32:49.408098+0000","last_active":"2024-04-23T14:32:49.408098+0000","last_peered":"2024-04-23T14:32:49.408098+0000","last_clean":"2024-04-23T14:32:49.408098+0000","last_became_active":"2024-04-23T14:32:49.407785+0000","last_became_peered":"2024-04-23T14:32:49.407785+0000","last_unstale":"2024-04-23T14:32:49.408098+0000","last_undegraded":"2024-04-23T14:32:49.408098+0000","last_fullsized":"2024-04-23T14:32:49.408098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:12:49.629971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.400540+0000","last_change":"2024-04-23T14:32:48.386689+0000","last_active":"2024-04-23T14:32:49.400540+0000","last_peered":"2024-04-23T14:32:49.400540+0000","last_clean":"2024-04-23T14:32:49.400540+0000","last_became_active":"2024-04-23T14:32:48.386272+0000","last_became_peered":"2024-04-23T14:32:48.386272+0000","last_unstale":"2024-04-23T14:32:49.400540+0000","last_undegraded":"2024-04-23T14:32:49.400540+0000","last_fullsized":"2024-04-23T14:32:49.400540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411036+0000","last_change":"2024-04-23T14:32:49.407292+0000","last_active":"2024-04-23T14:32:49.411036+0000","last_peered":"2024-04-23T14:32:49.411036+0000","last_clean":"2024-04-23T14:32:49.411036+0000","last_became_active":"2024-04-23T14:32:49.406967+0000","last_became_peered":"2024-04-23T14:32:49.406967+0000","last_unstale":"2024-04-23T14:32:49.411036+0000","last_undegraded":"2024-04-23T14:32:49.411036+0000","last_fullsized":"2024-04-23T14:32:49.411036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396038+0000","last_change":"2024-04-23T14:32:48.372538+0000","last_active":"2024-04-23T14:32:49.396038+0000","last_peered":"2024-04-23T14:32:49.396038+0000","last_clean":"2024-04-23T14:32:49.396038+0000","last_became_active":"2024-04-23T14:32:48.372276+0000","last_became_peered":"2024-04-23T14:32:48.372276+0000","last_unstale":"2024-04-23T14:32:49.396038+0000","last_undegraded":"2024-04-23T14:32:49.396038+0000","last_fullsized":"2024-04-23T14:32:49.396038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:57.601612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-23T14:32:49.383866+0000","last_change":"2024-04-23T14:32:49.383866+0000","last_active":"2024-04-23T14:32:48.338071+0000","last_peered":"2024-04-23T14:32:48.338071+0000","last_clean":"2024-04-23T14:32:48.338071+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:32:49.383866+0000","last_undegraded":"2024-04-23T14:32:49.383866+0000","last_fullsized":"2024-04-23T14:32:49.383866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397243+0000","last_change":"2024-04-23T14:32:48.373983+0000","last_active":"2024-04-23T14:32:49.397243+0000","last_peered":"2024-04-23T14:32:49.397243+0000","last_clean":"2024-04-23T14:32:49.397243+0000","last_became_active":"2024-04-23T14:32:48.373688+0000","last_became_peered":"2024-04-23T14:32:48.373688+0000","last_unstale":"2024-04-23T14:32:49.397243+0000","last_undegraded":"2024-04-23T14:32:49.397243+0000","last_fullsized":"2024-04-23T14:32:49.397243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411462+0000","last_change":"2024-04-23T14:32:49.398323+0000","last_active":"2024-04-23T14:32:49.411462+0000","last_peered":"2024-04-23T14:32:49.411462+0000","last_clean":"2024-04-23T14:32:49.411462+0000","last_became_active":"2024-04-23T14:32:49.398039+0000","last_became_peered":"2024-04-23T14:32:49.398039+0000","last_unstale":"2024-04-23T14:32:49.411462+0000","last_undegraded":"2024-04-23T14:32:49.411462+0000","last_fullsized":"2024-04-23T14:32:49.411462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:25.449154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.396203+0000","last_change":"2024-04-23T14:32:48.384396+0000","last_active":"2024-04-23T14:32:49.396203+0000","last_peered":"2024-04-23T14:32:49.396203+0000","last_clean":"2024-04-23T14:32:49.396203+0000","last_became_active":"2024-04-23T14:32:48.383739+0000","last_became_peered":"2024-04-23T14:32:48.383739+0000","last_unstale":"2024-04-23T14:32:49.396203+0000","last_undegraded":"2024-04-23T14:32:49.396203+0000","last_fullsized":"2024-04-23T14:32:49.396203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401815+0000","last_change":"2024-04-23T14:32:49.396368+0000","last_active":"2024-04-23T14:32:49.401815+0000","last_peered":"2024-04-23T14:32:49.401815+0000","last_clean":"2024-04-23T14:32:49.401815+0000","last_became_active":"2024-04-23T14:32:49.395031+0000","last_became_peered":"2024-04-23T14:32:49.395031+0000","last_unstale":"2024-04-23T14:32:49.401815+0000","last_undegraded":"2024-04-23T14:32:49.401815+0000","last_fullsized":"2024-04-23T14:32:49.401815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.397942+0000","last_change":"2024-04-23T14:32:48.387008+0000","last_active":"2024-04-23T14:32:49.397942+0000","last_peered":"2024-04-23T14:32:49.397942+0000","last_clean":"2024-04-23T14:32:49.397942+0000","last_became_active":"2024-04-23T14:32:48.386260+0000","last_became_peered":"2024-04-23T14:32:48.386260+0000","last_unstale":"2024-04-23T14:32:49.397942+0000","last_undegraded":"2024-04-23T14:32:49.397942+0000","last_fullsized":"2024-04-23T14:32:49.397942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.415060+0000","last_change":"2024-04-23T14:32:49.415060+0000","last_active":"2024-04-23T14:32:49.415060+0000","last_peered":"2024-04-23T14:32:49.415060+0000","last_clean":"2024-04-23T14:32:49.415060+0000","last_became_active":"2024-04-23T14:32:49.414707+0000","last_became_peered":"2024-04-23T14:32:49.414707+0000","last_unstale":"2024-04-23T14:32:49.415060+0000","last_undegraded":"2024-04-23T14:32:49.415060+0000","last_fullsized":"2024-04-23T14:32:49.415060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:02.059103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.401039+0000","last_change":"2024-04-23T14:32:48.380600+0000","last_active":"2024-04-23T14:32:49.401039+0000","last_peered":"2024-04-23T14:32:49.401039+0000","last_clean":"2024-04-23T14:32:49.401039+0000","last_became_active":"2024-04-23T14:32:48.379900+0000","last_became_peered":"2024-04-23T14:32:48.379900+0000","last_unstale":"2024-04-23T14:32:49.401039+0000","last_undegraded":"2024-04-23T14:32:49.401039+0000","last_fullsized":"2024-04-23T14:32:49.401039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:31.916332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.425051+0000","last_change":"2024-04-23T14:32:49.417179+0000","last_active":"2024-04-23T14:32:49.425051+0000","last_peered":"2024-04-23T14:32:49.425051+0000","last_clean":"2024-04-23T14:32:49.425051+0000","last_became_active":"2024-04-23T14:32:49.414238+0000","last_became_peered":"2024-04-23T14:32:49.414238+0000","last_unstale":"2024-04-23T14:32:49.425051+0000","last_undegraded":"2024-04-23T14:32:49.425051+0000","last_fullsized":"2024-04-23T14:32:49.425051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:15.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.375409+0000","last_change":"2024-04-23T14:32:48.382557+0000","last_active":"2024-04-23T14:32:49.375409+0000","last_peered":"2024-04-23T14:32:49.375409+0000","last_clean":"2024-04-23T14:32:49.375409+0000","last_became_active":"2024-04-23T14:32:48.381989+0000","last_became_peered":"2024-04-23T14:32:48.381989+0000","last_unstale":"2024-04-23T14:32:49.375409+0000","last_undegraded":"2024-04-23T14:32:49.375409+0000","last_fullsized":"2024-04-23T14:32:49.375409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.411232+0000","last_change":"2024-04-23T14:32:49.406874+0000","last_active":"2024-04-23T14:32:49.411232+0000","last_peered":"2024-04-23T14:32:49.411232+0000","last_clean":"2024-04-23T14:32:49.411232+0000","last_became_active":"2024-04-23T14:32:49.406634+0000","last_became_peered":"2024-04-23T14:32:49.406634+0000","last_unstale":"2024-04-23T14:32:49.411232+0000","last_undegraded":"2024-04-23T14:32:49.411232+0000","last_fullsized":"2024-04-23T14:32:49.411232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.380918+0000","last_change":"2024-04-23T14:32:48.381294+0000","last_active":"2024-04-23T14:32:49.380918+0000","last_peered":"2024-04-23T14:32:49.380918+0000","last_clean":"2024-04-23T14:32:49.380918+0000","last_became_active":"2024-04-23T14:32:48.380990+0000","last_became_peered":"2024-04-23T14:32:48.380990+0000","last_unstale":"2024-04-23T14:32:49.380918+0000","last_undegraded":"2024-04-23T14:32:49.380918+0000","last_fullsized":"2024-04-23T14:32:49.380918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-23T14:32:49.391358+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:32:49.391358+0000","last_peered":"2024-04-23T14:32:49.391358+0000","last_clean":"2024-04-23T14:32:49.391358+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:32:49.391358+0000","last_undegraded":"2024-04-23T14:32:49.391358+0000","last_fullsized":"2024-04-23T14:32:49.391358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":216,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:58.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:32:59.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.139+0000 7f8199000640 1 Processor -- start 2024-04-23T14:32:59.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.139+0000 7f8199000640 1 -- start start 2024-04-23T14:32:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.140+0000 7f8199000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f819415c3d0 0x7f81940cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.140+0000 7f8199000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81940b4860 0x7f81940cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.140+0000 7f8199000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8194160210 con 0x7f81940cd590 2024-04-23T14:32:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.140+0000 7f8199000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f81941604f0 con 0x7f819415c3d0 2024-04-23T14:32:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.140+0000 7f8199000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f81941607d0 con 0x7f81940b4860 2024-04-23T14:32:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8192a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f81940cd590 0x7f81940b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35190/0 (socket says 172.21.15.59:35190) 2024-04-23T14:32:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8192a00640 1 -- 172.21.15.59:0/4145067794 learned_addr learned my addr 172.21.15.59:0/4145067794 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8193400640 1 --2- 172.21.15.59:0/4145067794 >> v2:172.21.15.59:3300/0 conn(0x7f819415c3d0 0x7f81940cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8192000640 1 -- 172.21.15.59:0/4145067794 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2636061736 0 0) 0x7f8194160210 con 0x7f81940cd590 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8192000640 1 -- 172.21.15.59:0/4145067794 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8178003520 con 0x7f81940cd590 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8193e00640 1 --2- 172.21.15.59:0/4145067794 >> v2:172.21.15.76:3300/0 conn(0x7f81940b4860 0x7f81940cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8193400640 1 -- 172.21.15.59:0/4145067794 >> v2:172.21.15.76:3300/0 conn(0x7f81940b4860 msgr2=0x7f81940cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8193400640 1 --2- 172.21.15.59:0/4145067794 >> v2:172.21.15.76:3300/0 conn(0x7f81940b4860 0x7f81940cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8193400640 1 -- 172.21.15.59:0/4145067794 >> v1:172.21.15.59:6789/0 conn(0x7f81940cd590 legacy=0x7f81940b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8193400640 1 -- 172.21.15.59:0/4145067794 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8194160ab0 con 0x7f819415c3d0 2024-04-23T14:32:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.141+0000 7f8193e00640 1 --2- 172.21.15.59:0/4145067794 >> v2:172.21.15.76:3300/0 conn(0x7f81940b4860 0x7f81940cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.143+0000 7f8193400640 1 --2- 172.21.15.59:0/4145067794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f81940cd050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81940b2db0 tx=0x7f818402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f639c85f3e4098cf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.143+0000 7f8192000640 1 -- 172.21.15.59:0/4145067794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184037070 con 0x7f819415c3d0 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.143+0000 7f8192000640 1 -- 172.21.15.59:0/4145067794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8184032440 con 0x7f819415c3d0 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.143+0000 7f8192000640 1 -- 172.21.15.59:0/4145067794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184032810 con 0x7f819415c3d0 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.143+0000 7f8199000640 1 -- 172.21.15.59:0/4145067794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 msgr2=0x7f81940cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.143+0000 7f8199000640 1 --2- 172.21.15.59:0/4145067794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f81940cd050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81940b2db0 tx=0x7f818402a1a0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.144+0000 7f8199000640 1 -- 172.21.15.59:0/4145067794 shutdown_connections 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.144+0000 7f8199000640 1 --2- 172.21.15.59:0/4145067794 >> v2:172.21.15.76:3300/0 conn(0x7f81940b4860 0x7f81940cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.144+0000 7f8199000640 1 --2- 172.21.15.59:0/4145067794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f81940cd050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.144+0000 7f8199000640 1 -- 172.21.15.59:0/4145067794 >> 172.21.15.59:0/4145067794 conn(0x7f81940b21a0 msgr2=0x7f81940b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.144+0000 7f8199000640 1 -- 172.21.15.59:0/4145067794 shutdown_connections 2024-04-23T14:32:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.144+0000 7f8199000640 1 -- 172.21.15.59:0/4145067794 wait complete. 2024-04-23T14:32:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.145+0000 7f8199000640 1 Processor -- start 2024-04-23T14:32:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.146+0000 7f8199000640 1 -- start start 2024-04-23T14:32:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8199000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81940b4860 0x7f81940c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8199000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81940cd590 0x7f81941e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8199000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f8194155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8199000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f819415c930 con 0x7f819415c3d0 2024-04-23T14:32:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8199000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f819415c7b0 con 0x7f81940cd590 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8199000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f81940cbd10 con 0x7f81940b4860 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f8194155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f8194155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45660/0 (socket says 172.21.15.59:45660) 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 -- 172.21.15.59:0/1586581648 learned_addr learned my addr 172.21.15.59:0/1586581648 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8192a00640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81940cd590 0x7f81941e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193400640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81940b4860 0x7f81940c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 -- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81940b4860 msgr2=0x7f81940c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81940b4860 0x7f81940c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 -- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81940cd590 msgr2=0x7f81941e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81940cd590 0x7f81941e8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.147+0000 7f8193e00640 1 -- 172.21.15.59:0/1586581648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819415d950 con 0x7f819415c3d0 2024-04-23T14:32:59.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.148+0000 7f8193e00640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f8194155fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f818c00cef0 tx=0x7f818c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.148+0000 7f818b400640 1 -- 172.21.15.59:0/1586581648 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c01a070 con 0x7f819415c3d0 2024-04-23T14:32:59.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.148+0000 7f818b400640 1 -- 172.21.15.59:0/1586581648 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818c00d860 con 0x7f819415c3d0 2024-04-23T14:32:59.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.148+0000 7f818b400640 1 -- 172.21.15.59:0/1586581648 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c00dc50 con 0x7f819415c3d0 2024-04-23T14:32:59.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.148+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819415dc30 con 0x7f819415c3d0 2024-04-23T14:32:59.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.148+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819415e1f0 con 0x7f819415c3d0 2024-04-23T14:32:59.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.150+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819404fd90 con 0x7f819415c3d0 2024-04-23T14:32:59.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.151+0000 7f818b400640 1 -- 172.21.15.59:0/1586581648 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f818c01d070 con 0x7f819415c3d0 2024-04-23T14:32:59.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.152+0000 7f818b400640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.152+0000 7f818b400640 1 -- 172.21.15.59:0/1586581648 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f818c08c430 con 0x7f819415c3d0 2024-04-23T14:32:59.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.152+0000 7f8193400640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.153+0000 7f8193400640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f817006c8f0 0x7f817006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8184034000 tx=0x7f818402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.157+0000 7f818b400640 1 -- 172.21.15.59:0/1586581648 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f818c05a910 con 0x7f819415c3d0 2024-04-23T14:32:59.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.363+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81940d0cb0 con 0x7f817006c8f0 2024-04-23T14:32:59.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.382+0000 7f818b400640 1 -- 172.21.15.59:0/1586581648 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f81940d0cb0 con 0x7f817006c8f0 2024-04-23T14:32:59.386 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:32:59.387 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f817006c8f0 msgr2=0x7f817006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f817006c8f0 0x7f817006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8184034000 tx=0x7f818402e000 comp rx=0 tx=0).stop 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 msgr2=0x7f8194155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f8194155fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f818c00cef0 tx=0x7f818c00a710 comp rx=0 tx=0).stop 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 shutdown_connections 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f819415c3d0 0x7f8194155fe0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81940cd590 0x7f81941e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 --2- 172.21.15.59:0/1586581648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81940b4860 0x7f81940c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.387+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 >> 172.21.15.59:0/1586581648 conn(0x7f81940b21a0 msgr2=0x7f81940d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.388+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 shutdown_connections 2024-04-23T14:32:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.388+0000 7f8199000640 1 -- 172.21.15.59:0/1586581648 wait complete. 2024-04-23T14:32:59.406 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-23T14:32:58.372869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224508,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750212,"statfs":{"total":773094113280,"available":772864217088,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.994403"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406882+0000","last_change":"2024-04-23T14:32:49.396796+0000","last_active":"2024-04-23T14:32:50.406882+0000","last_peered":"2024-04-23T14:32:50.406882+0000","last_clean":"2024-04-23T14:32:50.406882+0000","last_became_active":"2024-04-23T14:32:49.396505+0000","last_became_peered":"2024-04-23T14:32:49.396505+0000","last_unstale":"2024-04-23T14:32:50.406882+0000","last_undegraded":"2024-04-23T14:32:50.406882+0000","last_fullsized":"2024-04-23T14:32:50.406882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545219+0000","last_change":"2024-04-23T14:32:49.396404+0000","last_active":"2024-04-23T14:32:50.545219+0000","last_peered":"2024-04-23T14:32:50.545219+0000","last_clean":"2024-04-23T14:32:50.545219+0000","last_became_active":"2024-04-23T14:32:49.396103+0000","last_became_peered":"2024-04-23T14:32:49.396103+0000","last_unstale":"2024-04-23T14:32:50.545219+0000","last_undegraded":"2024-04-23T14:32:50.545219+0000","last_fullsized":"2024-04-23T14:32:50.545219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:32:47.055118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545045+0000","last_change":"2024-04-23T14:32:49.421894+0000","last_active":"2024-04-23T14:32:50.545045+0000","last_peered":"2024-04-23T14:32:50.545045+0000","last_clean":"2024-04-23T14:32:50.545045+0000","last_became_active":"2024-04-23T14:32:49.420281+0000","last_became_peered":"2024-04-23T14:32:49.420281+0000","last_unstale":"2024-04-23T14:32:50.545045+0000","last_undegraded":"2024-04-23T14:32:50.545045+0000","last_fullsized":"2024-04-23T14:32:50.545045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:31.762434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406931+0000","last_change":"2024-04-23T14:32:49.412161+0000","last_active":"2024-04-23T14:32:50.406931+0000","last_peered":"2024-04-23T14:32:50.406931+0000","last_clean":"2024-04-23T14:32:50.406931+0000","last_became_active":"2024-04-23T14:32:49.411609+0000","last_became_peered":"2024-04-23T14:32:49.411609+0000","last_unstale":"2024-04-23T14:32:50.406931+0000","last_undegraded":"2024-04-23T14:32:50.406931+0000","last_fullsized":"2024-04-23T14:32:50.406931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:27.732778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545453+0000","last_change":"2024-04-23T14:32:49.411075+0000","last_active":"2024-04-23T14:32:50.545453+0000","last_peered":"2024-04-23T14:32:50.545453+0000","last_clean":"2024-04-23T14:32:50.545453+0000","last_became_active":"2024-04-23T14:32:49.410648+0000","last_became_peered":"2024-04-23T14:32:49.410648+0000","last_unstale":"2024-04-23T14:32:50.545453+0000","last_undegraded":"2024-04-23T14:32:50.545453+0000","last_fullsized":"2024-04-23T14:32:50.545453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:22.451922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544151+0000","last_change":"2024-04-23T14:32:49.396761+0000","last_active":"2024-04-23T14:32:50.544151+0000","last_peered":"2024-04-23T14:32:50.544151+0000","last_clean":"2024-04-23T14:32:50.544151+0000","last_became_active":"2024-04-23T14:32:49.396598+0000","last_became_peered":"2024-04-23T14:32:49.396598+0000","last_unstale":"2024-04-23T14:32:50.544151+0000","last_undegraded":"2024-04-23T14:32:50.544151+0000","last_fullsized":"2024-04-23T14:32:50.544151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:47:45.070247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546251+0000","last_change":"2024-04-23T14:32:49.398865+0000","last_active":"2024-04-23T14:32:50.546251+0000","last_peered":"2024-04-23T14:32:50.546251+0000","last_clean":"2024-04-23T14:32:50.546251+0000","last_became_active":"2024-04-23T14:32:49.398581+0000","last_became_peered":"2024-04-23T14:32:49.398581+0000","last_unstale":"2024-04-23T14:32:50.546251+0000","last_undegraded":"2024-04-23T14:32:50.546251+0000","last_fullsized":"2024-04-23T14:32:50.546251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:27.832083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545516+0000","last_change":"2024-04-23T14:32:49.409232+0000","last_active":"2024-04-23T14:32:50.545516+0000","last_peered":"2024-04-23T14:32:50.545516+0000","last_clean":"2024-04-23T14:32:50.545516+0000","last_became_active":"2024-04-23T14:32:49.408162+0000","last_became_peered":"2024-04-23T14:32:49.408162+0000","last_unstale":"2024-04-23T14:32:50.545516+0000","last_undegraded":"2024-04-23T14:32:50.545516+0000","last_fullsized":"2024-04-23T14:32:50.545516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:07.688839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409832+0000","last_change":"2024-04-23T14:32:49.424075+0000","last_active":"2024-04-23T14:32:50.409832+0000","last_peered":"2024-04-23T14:32:50.409832+0000","last_clean":"2024-04-23T14:32:50.409832+0000","last_became_active":"2024-04-23T14:32:49.423714+0000","last_became_peered":"2024-04-23T14:32:49.423714+0000","last_unstale":"2024-04-23T14:32:50.409832+0000","last_undegraded":"2024-04-23T14:32:50.409832+0000","last_fullsized":"2024-04-23T14:32:50.409832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407192+0000","last_change":"2024-04-23T14:32:49.419419+0000","last_active":"2024-04-23T14:32:50.407192+0000","last_peered":"2024-04-23T14:32:50.407192+0000","last_clean":"2024-04-23T14:32:50.407192+0000","last_became_active":"2024-04-23T14:32:49.416916+0000","last_became_peered":"2024-04-23T14:32:49.416916+0000","last_unstale":"2024-04-23T14:32:50.407192+0000","last_undegraded":"2024-04-23T14:32:50.407192+0000","last_fullsized":"2024-04-23T14:32:50.407192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:27.832083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.547097+0000","last_change":"2024-04-23T14:32:49.422405+0000","last_active":"2024-04-23T14:32:50.547097+0000","last_peered":"2024-04-23T14:32:50.547097+0000","last_clean":"2024-04-23T14:32:50.547097+0000","last_became_active":"2024-04-23T14:32:49.419100+0000","last_became_peered":"2024-04-23T14:32:49.419100+0000","last_unstale":"2024-04-23T14:32:50.547097+0000","last_undegraded":"2024-04-23T14:32:50.547097+0000","last_fullsized":"2024-04-23T14:32:50.547097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:59:37.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544333+0000","last_change":"2024-04-23T14:32:49.414777+0000","last_active":"2024-04-23T14:32:50.544333+0000","last_peered":"2024-04-23T14:32:50.544333+0000","last_clean":"2024-04-23T14:32:50.544333+0000","last_became_active":"2024-04-23T14:32:49.414208+0000","last_became_peered":"2024-04-23T14:32:49.414208+0000","last_unstale":"2024-04-23T14:32:50.544333+0000","last_undegraded":"2024-04-23T14:32:50.544333+0000","last_fullsized":"2024-04-23T14:32:50.544333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:27.732778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545666+0000","last_change":"2024-04-23T14:32:49.420414+0000","last_active":"2024-04-23T14:32:50.545666+0000","last_peered":"2024-04-23T14:32:50.545666+0000","last_clean":"2024-04-23T14:32:50.545666+0000","last_became_active":"2024-04-23T14:32:49.420063+0000","last_became_peered":"2024-04-23T14:32:49.420063+0000","last_unstale":"2024-04-23T14:32:50.545666+0000","last_undegraded":"2024-04-23T14:32:50.545666+0000","last_fullsized":"2024-04-23T14:32:50.545666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:52.367726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544453+0000","last_change":"2024-04-23T14:32:49.409664+0000","last_active":"2024-04-23T14:32:50.544453+0000","last_peered":"2024-04-23T14:32:50.544453+0000","last_clean":"2024-04-23T14:32:50.544453+0000","last_became_active":"2024-04-23T14:32:49.409369+0000","last_became_peered":"2024-04-23T14:32:49.409369+0000","last_unstale":"2024-04-23T14:32:50.544453+0000","last_undegraded":"2024-04-23T14:32:50.544453+0000","last_fullsized":"2024-04-23T14:32:50.544453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:59:37.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.543807+0000","last_change":"2024-04-23T14:32:49.395881+0000","last_active":"2024-04-23T14:32:50.543807+0000","last_peered":"2024-04-23T14:32:50.543807+0000","last_clean":"2024-04-23T14:32:50.543807+0000","last_became_active":"2024-04-23T14:32:49.395640+0000","last_became_peered":"2024-04-23T14:32:49.395640+0000","last_unstale":"2024-04-23T14:32:50.543807+0000","last_undegraded":"2024-04-23T14:32:50.543807+0000","last_fullsized":"2024-04-23T14:32:50.543807+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:02.059103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545830+0000","last_change":"2024-04-23T14:32:49.410041+0000","last_active":"2024-04-23T14:32:50.545830+0000","last_peered":"2024-04-23T14:32:50.545830+0000","last_clean":"2024-04-23T14:32:50.545830+0000","last_became_active":"2024-04-23T14:32:49.409759+0000","last_became_peered":"2024-04-23T14:32:49.409759+0000","last_unstale":"2024-04-23T14:32:50.545830+0000","last_undegraded":"2024-04-23T14:32:50.545830+0000","last_fullsized":"2024-04-23T14:32:50.545830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:02.059103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546057+0000","last_change":"2024-04-23T14:32:49.405810+0000","last_active":"2024-04-23T14:32:50.546057+0000","last_peered":"2024-04-23T14:32:50.546057+0000","last_clean":"2024-04-23T14:32:50.546057+0000","last_became_active":"2024-04-23T14:32:49.405283+0000","last_became_peered":"2024-04-23T14:32:49.405283+0000","last_unstale":"2024-04-23T14:32:50.546057+0000","last_undegraded":"2024-04-23T14:32:50.546057+0000","last_fullsized":"2024-04-23T14:32:50.546057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:52.281569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545126+0000","last_change":"2024-04-23T14:32:49.423722+0000","last_active":"2024-04-23T14:32:50.545126+0000","last_peered":"2024-04-23T14:32:50.545126+0000","last_clean":"2024-04-23T14:32:50.545126+0000","last_became_active":"2024-04-23T14:32:49.423375+0000","last_became_peered":"2024-04-23T14:32:49.423375+0000","last_unstale":"2024-04-23T14:32:50.545126+0000","last_undegraded":"2024-04-23T14:32:50.545126+0000","last_fullsized":"2024-04-23T14:32:50.545126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:28:58.727383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544187+0000","last_change":"2024-04-23T14:32:49.420830+0000","last_active":"2024-04-23T14:32:50.544187+0000","last_peered":"2024-04-23T14:32:50.544187+0000","last_clean":"2024-04-23T14:32:50.544187+0000","last_became_active":"2024-04-23T14:32:49.420574+0000","last_became_peered":"2024-04-23T14:32:49.420574+0000","last_unstale":"2024-04-23T14:32:50.544187+0000","last_undegraded":"2024-04-23T14:32:50.544187+0000","last_fullsized":"2024-04-23T14:32:50.544187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:31.762434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545962+0000","last_change":"2024-04-23T14:32:49.408152+0000","last_active":"2024-04-23T14:32:50.545962+0000","last_peered":"2024-04-23T14:32:50.545962+0000","last_clean":"2024-04-23T14:32:50.545962+0000","last_became_active":"2024-04-23T14:32:49.407758+0000","last_became_peered":"2024-04-23T14:32:49.407758+0000","last_unstale":"2024-04-23T14:32:50.545962+0000","last_undegraded":"2024-04-23T14:32:50.545962+0000","last_fullsized":"2024-04-23T14:32:50.545962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:39.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.408974+0000","last_change":"2024-04-23T14:32:49.413362+0000","last_active":"2024-04-23T14:32:50.408974+0000","last_peered":"2024-04-23T14:32:50.408974+0000","last_clean":"2024-04-23T14:32:50.408974+0000","last_became_active":"2024-04-23T14:32:49.413082+0000","last_became_peered":"2024-04-23T14:32:49.413082+0000","last_unstale":"2024-04-23T14:32:50.408974+0000","last_undegraded":"2024-04-23T14:32:50.408974+0000","last_fullsized":"2024-04-23T14:32:50.408974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.408779+0000","last_change":"2024-04-23T14:32:49.414460+0000","last_active":"2024-04-23T14:32:50.408779+0000","last_peered":"2024-04-23T14:32:50.408779+0000","last_clean":"2024-04-23T14:32:50.408779+0000","last_became_active":"2024-04-23T14:32:49.413658+0000","last_became_peered":"2024-04-23T14:32:49.413658+0000","last_unstale":"2024-04-23T14:32:50.408779+0000","last_undegraded":"2024-04-23T14:32:50.408779+0000","last_fullsized":"2024-04-23T14:32:50.408779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403004+0000","last_change":"2024-04-23T14:32:49.404657+0000","last_active":"2024-04-23T14:32:50.403004+0000","last_peered":"2024-04-23T14:32:50.403004+0000","last_clean":"2024-04-23T14:32:50.403004+0000","last_became_active":"2024-04-23T14:32:49.404322+0000","last_became_peered":"2024-04-23T14:32:49.404322+0000","last_unstale":"2024-04-23T14:32:50.403004+0000","last_undegraded":"2024-04-23T14:32:50.403004+0000","last_fullsized":"2024-04-23T14:32:50.403004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.543591+0000","last_change":"2024-04-23T14:32:49.408981+0000","last_active":"2024-04-23T14:32:50.543591+0000","last_peered":"2024-04-23T14:32:50.543591+0000","last_clean":"2024-04-23T14:32:50.543591+0000","last_became_active":"2024-04-23T14:32:49.408732+0000","last_became_peered":"2024-04-23T14:32:49.408732+0000","last_unstale":"2024-04-23T14:32:50.543591+0000","last_undegraded":"2024-04-23T14:32:50.543591+0000","last_fullsized":"2024-04-23T14:32:50.543591+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:53.618947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407146+0000","last_change":"2024-04-23T14:32:49.411802+0000","last_active":"2024-04-23T14:32:50.407146+0000","last_peered":"2024-04-23T14:32:50.407146+0000","last_clean":"2024-04-23T14:32:50.407146+0000","last_became_active":"2024-04-23T14:32:49.411472+0000","last_became_peered":"2024-04-23T14:32:49.411472+0000","last_unstale":"2024-04-23T14:32:50.407146+0000","last_undegraded":"2024-04-23T14:32:50.407146+0000","last_fullsized":"2024-04-23T14:32:50.407146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:52.281569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546606+0000","last_change":"2024-04-23T14:32:49.408321+0000","last_active":"2024-04-23T14:32:50.546606+0000","last_peered":"2024-04-23T14:32:50.546606+0000","last_clean":"2024-04-23T14:32:50.546606+0000","last_became_active":"2024-04-23T14:32:49.407885+0000","last_became_peered":"2024-04-23T14:32:49.407885+0000","last_unstale":"2024-04-23T14:32:50.546606+0000","last_undegraded":"2024-04-23T14:32:50.546606+0000","last_fullsized":"2024-04-23T14:32:50.546606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:49:32.358448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407201+0000","last_change":"2024-04-23T14:32:49.418183+0000","last_active":"2024-04-23T14:32:50.407201+0000","last_peered":"2024-04-23T14:32:50.407201+0000","last_clean":"2024-04-23T14:32:50.407201+0000","last_became_active":"2024-04-23T14:32:49.417903+0000","last_became_peered":"2024-04-23T14:32:49.417903+0000","last_unstale":"2024-04-23T14:32:50.407201+0000","last_undegraded":"2024-04-23T14:32:50.407201+0000","last_fullsized":"2024-04-23T14:32:50.407201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:47:45.070247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544476+0000","last_change":"2024-04-23T14:32:49.396858+0000","last_active":"2024-04-23T14:32:50.544476+0000","last_peered":"2024-04-23T14:32:50.544476+0000","last_clean":"2024-04-23T14:32:50.544476+0000","last_became_active":"2024-04-23T14:32:49.396565+0000","last_became_peered":"2024-04-23T14:32:49.396565+0000","last_unstale":"2024-04-23T14:32:50.544476+0000","last_undegraded":"2024-04-23T14:32:50.544476+0000","last_fullsized":"2024-04-23T14:32:50.544476+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:27.832083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409067+0000","last_change":"2024-04-23T14:32:49.419860+0000","last_active":"2024-04-23T14:32:50.409067+0000","last_peered":"2024-04-23T14:32:50.409067+0000","last_clean":"2024-04-23T14:32:50.409067+0000","last_became_active":"2024-04-23T14:32:49.419578+0000","last_became_peered":"2024-04-23T14:32:49.419578+0000","last_unstale":"2024-04-23T14:32:50.409067+0000","last_undegraded":"2024-04-23T14:32:50.409067+0000","last_fullsized":"2024-04-23T14:32:50.409067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546132+0000","last_change":"2024-04-23T14:32:49.403367+0000","last_active":"2024-04-23T14:32:50.546132+0000","last_peered":"2024-04-23T14:32:50.546132+0000","last_clean":"2024-04-23T14:32:50.546132+0000","last_became_active":"2024-04-23T14:32:49.403043+0000","last_became_peered":"2024-04-23T14:32:49.403043+0000","last_unstale":"2024-04-23T14:32:50.546132+0000","last_undegraded":"2024-04-23T14:32:50.546132+0000","last_fullsized":"2024-04-23T14:32:50.546132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:27.732778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406136+0000","last_change":"2024-04-23T14:32:49.407909+0000","last_active":"2024-04-23T14:32:50.406136+0000","last_peered":"2024-04-23T14:32:50.406136+0000","last_clean":"2024-04-23T14:32:50.406136+0000","last_became_active":"2024-04-23T14:32:49.407653+0000","last_became_peered":"2024-04-23T14:32:49.407653+0000","last_unstale":"2024-04-23T14:32:50.406136+0000","last_undegraded":"2024-04-23T14:32:50.406136+0000","last_fullsized":"2024-04-23T14:32:50.406136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545443+0000","last_change":"2024-04-23T14:32:49.412291+0000","last_active":"2024-04-23T14:32:50.545443+0000","last_peered":"2024-04-23T14:32:50.545443+0000","last_clean":"2024-04-23T14:32:50.545443+0000","last_became_active":"2024-04-23T14:32:49.411966+0000","last_became_peered":"2024-04-23T14:32:49.411966+0000","last_unstale":"2024-04-23T14:32:50.545443+0000","last_undegraded":"2024-04-23T14:32:50.545443+0000","last_fullsized":"2024-04-23T14:32:50.545443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:01:31.163770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403278+0000","last_change":"2024-04-23T14:32:49.398365+0000","last_active":"2024-04-23T14:32:50.403278+0000","last_peered":"2024-04-23T14:32:50.403278+0000","last_clean":"2024-04-23T14:32:50.403278+0000","last_became_active":"2024-04-23T14:32:49.398036+0000","last_became_peered":"2024-04-23T14:32:49.398036+0000","last_unstale":"2024-04-23T14:32:50.403278+0000","last_undegraded":"2024-04-23T14:32:50.403278+0000","last_fullsized":"2024-04-23T14:32:50.403278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407095+0000","last_change":"2024-04-23T14:32:48.380750+0000","last_active":"2024-04-23T14:32:50.407095+0000","last_peered":"2024-04-23T14:32:50.407095+0000","last_clean":"2024-04-23T14:32:50.407095+0000","last_became_active":"2024-04-23T14:32:48.380399+0000","last_became_peered":"2024-04-23T14:32:48.380399+0000","last_unstale":"2024-04-23T14:32:50.407095+0000","last_undegraded":"2024-04-23T14:32:50.407095+0000","last_fullsized":"2024-04-23T14:32:50.407095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:31.689650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407013+0000","last_change":"2024-04-23T14:32:49.411206+0000","last_active":"2024-04-23T14:32:50.407013+0000","last_peered":"2024-04-23T14:32:50.407013+0000","last_clean":"2024-04-23T14:32:50.407013+0000","last_became_active":"2024-04-23T14:32:49.410469+0000","last_became_peered":"2024-04-23T14:32:49.410469+0000","last_unstale":"2024-04-23T14:32:50.407013+0000","last_undegraded":"2024-04-23T14:32:50.407013+0000","last_fullsized":"2024-04-23T14:32:50.407013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403444+0000","last_change":"2024-04-23T14:32:48.378555+0000","last_active":"2024-04-23T14:32:50.403444+0000","last_peered":"2024-04-23T14:32:50.403444+0000","last_clean":"2024-04-23T14:32:50.403444+0000","last_became_active":"2024-04-23T14:32:48.377594+0000","last_became_peered":"2024-04-23T14:32:48.377594+0000","last_unstale":"2024-04-23T14:32:50.403444+0000","last_undegraded":"2024-04-23T14:32:50.403444+0000","last_fullsized":"2024-04-23T14:32:50.403444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409464+0000","last_change":"2024-04-23T14:32:49.418168+0000","last_active":"2024-04-23T14:32:50.409464+0000","last_peered":"2024-04-23T14:32:50.409464+0000","last_clean":"2024-04-23T14:32:50.409464+0000","last_became_active":"2024-04-23T14:32:49.417542+0000","last_became_peered":"2024-04-23T14:32:49.417542+0000","last_unstale":"2024-04-23T14:32:50.409464+0000","last_undegraded":"2024-04-23T14:32:50.409464+0000","last_fullsized":"2024-04-23T14:32:50.409464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:29.560169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544618+0000","last_change":"2024-04-23T14:32:48.383157+0000","last_active":"2024-04-23T14:32:50.544618+0000","last_peered":"2024-04-23T14:32:50.544618+0000","last_clean":"2024-04-23T14:32:50.544618+0000","last_became_active":"2024-04-23T14:32:48.382568+0000","last_became_peered":"2024-04-23T14:32:48.382568+0000","last_unstale":"2024-04-23T14:32:50.544618+0000","last_undegraded":"2024-04-23T14:32:50.544618+0000","last_fullsized":"2024-04-23T14:32:50.544618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:35.115546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407357+0000","last_change":"2024-04-23T14:32:49.418053+0000","last_active":"2024-04-23T14:32:50.407357+0000","last_peered":"2024-04-23T14:32:50.407357+0000","last_clean":"2024-04-23T14:32:50.407357+0000","last_became_active":"2024-04-23T14:32:49.417733+0000","last_became_peered":"2024-04-23T14:32:49.417733+0000","last_unstale":"2024-04-23T14:32:50.407357+0000","last_undegraded":"2024-04-23T14:32:50.407357+0000","last_fullsized":"2024-04-23T14:32:50.407357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:31.762434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545775+0000","last_change":"2024-04-23T14:32:48.374028+0000","last_active":"2024-04-23T14:32:50.545775+0000","last_peered":"2024-04-23T14:32:50.545775+0000","last_clean":"2024-04-23T14:32:50.545775+0000","last_became_active":"2024-04-23T14:32:48.372871+0000","last_became_peered":"2024-04-23T14:32:48.372871+0000","last_unstale":"2024-04-23T14:32:50.545775+0000","last_undegraded":"2024-04-23T14:32:50.545775+0000","last_fullsized":"2024-04-23T14:32:50.545775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:14.297285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546544+0000","last_change":"2024-04-23T14:32:49.410412+0000","last_active":"2024-04-23T14:32:50.546544+0000","last_peered":"2024-04-23T14:32:50.546544+0000","last_clean":"2024-04-23T14:32:50.546544+0000","last_became_active":"2024-04-23T14:32:49.409645+0000","last_became_peered":"2024-04-23T14:32:49.409645+0000","last_unstale":"2024-04-23T14:32:50.546544+0000","last_undegraded":"2024-04-23T14:32:50.546544+0000","last_fullsized":"2024-04-23T14:32:50.546544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:31.762434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409263+0000","last_change":"2024-04-23T14:32:48.381407+0000","last_active":"2024-04-23T14:32:50.409263+0000","last_peered":"2024-04-23T14:32:50.409263+0000","last_clean":"2024-04-23T14:32:50.409263+0000","last_became_active":"2024-04-23T14:32:48.381122+0000","last_became_peered":"2024-04-23T14:32:48.381122+0000","last_unstale":"2024-04-23T14:32:50.409263+0000","last_undegraded":"2024-04-23T14:32:50.409263+0000","last_fullsized":"2024-04-23T14:32:50.409263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:16:40.112260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544214+0000","last_change":"2024-04-23T14:32:49.400326+0000","last_active":"2024-04-23T14:32:50.544214+0000","last_peered":"2024-04-23T14:32:50.544214+0000","last_clean":"2024-04-23T14:32:50.544214+0000","last_became_active":"2024-04-23T14:32:49.399716+0000","last_became_peered":"2024-04-23T14:32:49.399716+0000","last_unstale":"2024-04-23T14:32:50.544214+0000","last_undegraded":"2024-04-23T14:32:50.544214+0000","last_fullsized":"2024-04-23T14:32:50.544214+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:52.281569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.408846+0000","last_change":"2024-04-23T14:32:48.380347+0000","last_active":"2024-04-23T14:32:50.408846+0000","last_peered":"2024-04-23T14:32:50.408846+0000","last_clean":"2024-04-23T14:32:50.408846+0000","last_became_active":"2024-04-23T14:32:48.379799+0000","last_became_peered":"2024-04-23T14:32:48.379799+0000","last_unstale":"2024-04-23T14:32:50.408846+0000","last_undegraded":"2024-04-23T14:32:50.408846+0000","last_fullsized":"2024-04-23T14:32:50.408846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:13.096562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407901+0000","last_change":"2024-04-23T14:32:49.415576+0000","last_active":"2024-04-23T14:32:50.407901+0000","last_peered":"2024-04-23T14:32:50.407901+0000","last_clean":"2024-04-23T14:32:50.407901+0000","last_became_active":"2024-04-23T14:32:49.415107+0000","last_became_peered":"2024-04-23T14:32:49.415107+0000","last_unstale":"2024-04-23T14:32:50.407901+0000","last_undegraded":"2024-04-23T14:32:50.407901+0000","last_fullsized":"2024-04-23T14:32:50.407901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:41.587432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.543840+0000","last_change":"2024-04-23T14:32:48.378186+0000","last_active":"2024-04-23T14:32:50.543840+0000","last_peered":"2024-04-23T14:32:50.543840+0000","last_clean":"2024-04-23T14:32:50.543840+0000","last_became_active":"2024-04-23T14:32:48.377889+0000","last_became_peered":"2024-04-23T14:32:48.377889+0000","last_unstale":"2024-04-23T14:32:50.543840+0000","last_undegraded":"2024-04-23T14:32:50.543840+0000","last_fullsized":"2024-04-23T14:32:50.543840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406754+0000","last_change":"2024-04-23T14:32:49.418062+0000","last_active":"2024-04-23T14:32:50.406754+0000","last_peered":"2024-04-23T14:32:50.406754+0000","last_clean":"2024-04-23T14:32:50.406754+0000","last_became_active":"2024-04-23T14:32:49.417335+0000","last_became_peered":"2024-04-23T14:32:49.417335+0000","last_unstale":"2024-04-23T14:32:50.406754+0000","last_undegraded":"2024-04-23T14:32:50.406754+0000","last_fullsized":"2024-04-23T14:32:50.406754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:07.688839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407996+0000","last_change":"2024-04-23T14:32:48.376375+0000","last_active":"2024-04-23T14:32:50.407996+0000","last_peered":"2024-04-23T14:32:50.407996+0000","last_clean":"2024-04-23T14:32:50.407996+0000","last_became_active":"2024-04-23T14:32:48.375807+0000","last_became_peered":"2024-04-23T14:32:48.375807+0000","last_unstale":"2024-04-23T14:32:50.407996+0000","last_undegraded":"2024-04-23T14:32:50.407996+0000","last_fullsized":"2024-04-23T14:32:50.407996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:58:21.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545155+0000","last_change":"2024-04-23T14:32:49.414650+0000","last_active":"2024-04-23T14:32:50.545155+0000","last_peered":"2024-04-23T14:32:50.545155+0000","last_clean":"2024-04-23T14:32:50.545155+0000","last_became_active":"2024-04-23T14:32:49.414352+0000","last_became_peered":"2024-04-23T14:32:49.414352+0000","last_unstale":"2024-04-23T14:32:50.545155+0000","last_undegraded":"2024-04-23T14:32:50.545155+0000","last_fullsized":"2024-04-23T14:32:50.545155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:07.894545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546331+0000","last_change":"2024-04-23T14:32:48.380295+0000","last_active":"2024-04-23T14:32:50.546331+0000","last_peered":"2024-04-23T14:32:50.546331+0000","last_clean":"2024-04-23T14:32:50.546331+0000","last_became_active":"2024-04-23T14:32:48.379968+0000","last_became_peered":"2024-04-23T14:32:48.379968+0000","last_unstale":"2024-04-23T14:32:50.546331+0000","last_undegraded":"2024-04-23T14:32:50.546331+0000","last_fullsized":"2024-04-23T14:32:50.546331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:28.545681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546565+0000","last_change":"2024-04-23T14:32:49.408527+0000","last_active":"2024-04-23T14:32:50.546565+0000","last_peered":"2024-04-23T14:32:50.546565+0000","last_clean":"2024-04-23T14:32:50.546565+0000","last_became_active":"2024-04-23T14:32:49.408214+0000","last_became_peered":"2024-04-23T14:32:49.408214+0000","last_unstale":"2024-04-23T14:32:50.546565+0000","last_undegraded":"2024-04-23T14:32:50.546565+0000","last_fullsized":"2024-04-23T14:32:50.546565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:46:44.214978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403458+0000","last_change":"2024-04-23T14:32:48.369769+0000","last_active":"2024-04-23T14:32:50.403458+0000","last_peered":"2024-04-23T14:32:50.403458+0000","last_clean":"2024-04-23T14:32:50.403458+0000","last_became_active":"2024-04-23T14:32:48.369112+0000","last_became_peered":"2024-04-23T14:32:48.369112+0000","last_unstale":"2024-04-23T14:32:50.403458+0000","last_undegraded":"2024-04-23T14:32:50.403458+0000","last_fullsized":"2024-04-23T14:32:50.403458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:14.196347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544783+0000","last_change":"2024-04-23T14:32:49.409092+0000","last_active":"2024-04-23T14:32:50.544783+0000","last_peered":"2024-04-23T14:32:50.544783+0000","last_clean":"2024-04-23T14:32:50.544783+0000","last_became_active":"2024-04-23T14:32:49.408769+0000","last_became_peered":"2024-04-23T14:32:49.408769+0000","last_unstale":"2024-04-23T14:32:50.544783+0000","last_undegraded":"2024-04-23T14:32:50.544783+0000","last_fullsized":"2024-04-23T14:32:50.544783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:25:59.848909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544866+0000","last_change":"2024-04-23T14:32:48.379746+0000","last_active":"2024-04-23T14:32:50.544866+0000","last_peered":"2024-04-23T14:32:50.544866+0000","last_clean":"2024-04-23T14:32:50.544866+0000","last_became_active":"2024-04-23T14:32:48.378760+0000","last_became_peered":"2024-04-23T14:32:48.378760+0000","last_unstale":"2024-04-23T14:32:50.544866+0000","last_undegraded":"2024-04-23T14:32:50.544866+0000","last_fullsized":"2024-04-23T14:32:50.544866+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544563+0000","last_change":"2024-04-23T14:32:49.422110+0000","last_active":"2024-04-23T14:32:50.544563+0000","last_peered":"2024-04-23T14:32:50.544563+0000","last_clean":"2024-04-23T14:32:50.544563+0000","last_became_active":"2024-04-23T14:32:49.420945+0000","last_became_peered":"2024-04-23T14:32:49.420945+0000","last_unstale":"2024-04-23T14:32:50.544563+0000","last_undegraded":"2024-04-23T14:32:50.544563+0000","last_fullsized":"2024-04-23T14:32:50.544563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:47:45.070247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544870+0000","last_change":"2024-04-23T14:32:48.379776+0000","last_active":"2024-04-23T14:32:50.544870+0000","last_peered":"2024-04-23T14:32:50.544870+0000","last_clean":"2024-04-23T14:32:50.544870+0000","last_became_active":"2024-04-23T14:32:48.379485+0000","last_became_peered":"2024-04-23T14:32:48.379485+0000","last_unstale":"2024-04-23T14:32:50.544870+0000","last_undegraded":"2024-04-23T14:32:50.544870+0000","last_fullsized":"2024-04-23T14:32:50.544870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:00.386998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406760+0000","last_change":"2024-04-23T14:32:49.409577+0000","last_active":"2024-04-23T14:32:50.406760+0000","last_peered":"2024-04-23T14:32:50.406760+0000","last_clean":"2024-04-23T14:32:50.406760+0000","last_became_active":"2024-04-23T14:32:49.409264+0000","last_became_peered":"2024-04-23T14:32:49.409264+0000","last_unstale":"2024-04-23T14:32:50.406760+0000","last_undegraded":"2024-04-23T14:32:50.406760+0000","last_fullsized":"2024-04-23T14:32:50.406760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:53.618947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409346+0000","last_change":"2024-04-23T14:32:48.376245+0000","last_active":"2024-04-23T14:32:50.409346+0000","last_peered":"2024-04-23T14:32:50.409346+0000","last_clean":"2024-04-23T14:32:50.409346+0000","last_became_active":"2024-04-23T14:32:48.375882+0000","last_became_peered":"2024-04-23T14:32:48.375882+0000","last_unstale":"2024-04-23T14:32:50.409346+0000","last_undegraded":"2024-04-23T14:32:50.409346+0000","last_fullsized":"2024-04-23T14:32:50.409346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:27.386064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544461+0000","last_change":"2024-04-23T14:32:49.403473+0000","last_active":"2024-04-23T14:32:50.544461+0000","last_peered":"2024-04-23T14:32:50.544461+0000","last_clean":"2024-04-23T14:32:50.544461+0000","last_became_active":"2024-04-23T14:32:49.399541+0000","last_became_peered":"2024-04-23T14:32:49.399541+0000","last_unstale":"2024-04-23T14:32:50.544461+0000","last_undegraded":"2024-04-23T14:32:50.544461+0000","last_fullsized":"2024-04-23T14:32:50.544461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:07.894545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406755+0000","last_change":"2024-04-23T14:32:48.381791+0000","last_active":"2024-04-23T14:32:50.406755+0000","last_peered":"2024-04-23T14:32:50.406755+0000","last_clean":"2024-04-23T14:32:50.406755+0000","last_became_active":"2024-04-23T14:32:48.381531+0000","last_became_peered":"2024-04-23T14:32:48.381531+0000","last_unstale":"2024-04-23T14:32:50.406755+0000","last_undegraded":"2024-04-23T14:32:50.406755+0000","last_fullsized":"2024-04-23T14:32:50.406755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545585+0000","last_change":"2024-04-23T14:32:49.412273+0000","last_active":"2024-04-23T14:32:50.545585+0000","last_peered":"2024-04-23T14:32:50.545585+0000","last_clean":"2024-04-23T14:32:50.545585+0000","last_became_active":"2024-04-23T14:32:49.411939+0000","last_became_peered":"2024-04-23T14:32:49.411939+0000","last_unstale":"2024-04-23T14:32:50.545585+0000","last_undegraded":"2024-04-23T14:32:50.545585+0000","last_fullsized":"2024-04-23T14:32:50.545585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:53.618947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.543783+0000","last_change":"2024-04-23T14:32:48.383719+0000","last_active":"2024-04-23T14:32:50.543783+0000","last_peered":"2024-04-23T14:32:50.543783+0000","last_clean":"2024-04-23T14:32:50.543783+0000","last_became_active":"2024-04-23T14:32:48.382885+0000","last_became_peered":"2024-04-23T14:32:48.382885+0000","last_unstale":"2024-04-23T14:32:50.543783+0000","last_undegraded":"2024-04-23T14:32:50.543783+0000","last_fullsized":"2024-04-23T14:32:50.543783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:40.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406416+0000","last_change":"2024-04-23T14:32:49.406690+0000","last_active":"2024-04-23T14:32:50.406416+0000","last_peered":"2024-04-23T14:32:50.406416+0000","last_clean":"2024-04-23T14:32:50.406416+0000","last_became_active":"2024-04-23T14:32:49.406431+0000","last_became_peered":"2024-04-23T14:32:49.406431+0000","last_unstale":"2024-04-23T14:32:50.406416+0000","last_undegraded":"2024-04-23T14:32:50.406416+0000","last_fullsized":"2024-04-23T14:32:50.406416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:10.335907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.544129+0000","last_change":"2024-04-23T14:32:48.375851+0000","last_active":"2024-04-23T14:32:50.544129+0000","last_peered":"2024-04-23T14:32:50.544129+0000","last_clean":"2024-04-23T14:32:50.544129+0000","last_became_active":"2024-04-23T14:32:48.375572+0000","last_became_peered":"2024-04-23T14:32:48.375572+0000","last_unstale":"2024-04-23T14:32:50.544129+0000","last_undegraded":"2024-04-23T14:32:50.544129+0000","last_fullsized":"2024-04-23T14:32:50.544129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409826+0000","last_change":"2024-04-23T14:32:49.409809+0000","last_active":"2024-04-23T14:32:50.409826+0000","last_peered":"2024-04-23T14:32:50.409826+0000","last_clean":"2024-04-23T14:32:50.409826+0000","last_became_active":"2024-04-23T14:32:49.409554+0000","last_became_peered":"2024-04-23T14:32:49.409554+0000","last_unstale":"2024-04-23T14:32:50.409826+0000","last_undegraded":"2024-04-23T14:32:50.409826+0000","last_fullsized":"2024-04-23T14:32:50.409826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:28.375170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407342+0000","last_change":"2024-04-23T14:32:48.383194+0000","last_active":"2024-04-23T14:32:50.407342+0000","last_peered":"2024-04-23T14:32:50.407342+0000","last_clean":"2024-04-23T14:32:50.407342+0000","last_became_active":"2024-04-23T14:32:48.382867+0000","last_became_peered":"2024-04-23T14:32:48.382867+0000","last_unstale":"2024-04-23T14:32:50.407342+0000","last_undegraded":"2024-04-23T14:32:50.407342+0000","last_fullsized":"2024-04-23T14:32:50.407342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:09.321418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546351+0000","last_change":"2024-04-23T14:32:49.413412+0000","last_active":"2024-04-23T14:32:50.546351+0000","last_peered":"2024-04-23T14:32:50.546351+0000","last_clean":"2024-04-23T14:32:50.546351+0000","last_became_active":"2024-04-23T14:32:49.413066+0000","last_became_peered":"2024-04-23T14:32:49.413066+0000","last_unstale":"2024-04-23T14:32:50.546351+0000","last_undegraded":"2024-04-23T14:32:50.546351+0000","last_fullsized":"2024-04-23T14:32:50.546351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:59:37.858009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409907+0000","last_change":"2024-04-23T14:32:48.373256+0000","last_active":"2024-04-23T14:32:50.409907+0000","last_peered":"2024-04-23T14:32:50.409907+0000","last_clean":"2024-04-23T14:32:50.409907+0000","last_became_active":"2024-04-23T14:32:48.372720+0000","last_became_peered":"2024-04-23T14:32:48.372720+0000","last_unstale":"2024-04-23T14:32:50.409907+0000","last_undegraded":"2024-04-23T14:32:50.409907+0000","last_fullsized":"2024-04-23T14:32:50.409907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:04:15.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406958+0000","last_change":"2024-04-23T14:32:49.411786+0000","last_active":"2024-04-23T14:32:50.406958+0000","last_peered":"2024-04-23T14:32:50.406958+0000","last_clean":"2024-04-23T14:32:50.406958+0000","last_became_active":"2024-04-23T14:32:49.411468+0000","last_became_peered":"2024-04-23T14:32:49.411468+0000","last_unstale":"2024-04-23T14:32:50.406958+0000","last_undegraded":"2024-04-23T14:32:50.406958+0000","last_fullsized":"2024-04-23T14:32:50.406958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:02.059103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409989+0000","last_change":"2024-04-23T14:32:48.374462+0000","last_active":"2024-04-23T14:32:50.409989+0000","last_peered":"2024-04-23T14:32:50.409989+0000","last_clean":"2024-04-23T14:32:50.409989+0000","last_became_active":"2024-04-23T14:32:48.372060+0000","last_became_peered":"2024-04-23T14:32:48.372060+0000","last_unstale":"2024-04-23T14:32:50.409989+0000","last_undegraded":"2024-04-23T14:32:50.409989+0000","last_fullsized":"2024-04-23T14:32:50.409989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:40.111344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545895+0000","last_change":"2024-04-23T14:32:49.406897+0000","last_active":"2024-04-23T14:32:50.545895+0000","last_peered":"2024-04-23T14:32:50.545895+0000","last_clean":"2024-04-23T14:32:50.545895+0000","last_became_active":"2024-04-23T14:32:49.406604+0000","last_became_peered":"2024-04-23T14:32:49.406604+0000","last_unstale":"2024-04-23T14:32:50.545895+0000","last_undegraded":"2024-04-23T14:32:50.545895+0000","last_fullsized":"2024-04-23T14:32:50.545895+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:41.125833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.409554+0000","last_change":"2024-04-23T14:32:48.382650+0000","last_active":"2024-04-23T14:32:50.409554+0000","last_peered":"2024-04-23T14:32:50.409554+0000","last_clean":"2024-04-23T14:32:50.409554+0000","last_became_active":"2024-04-23T14:32:48.382391+0000","last_became_peered":"2024-04-23T14:32:48.382391+0000","last_unstale":"2024-04-23T14:32:50.409554+0000","last_undegraded":"2024-04-23T14:32:50.409554+0000","last_fullsized":"2024-04-23T14:32:50.409554+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:51.353237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545798+0000","last_change":"2024-04-23T14:32:49.410725+0000","last_active":"2024-04-23T14:32:50.545798+0000","last_peered":"2024-04-23T14:32:50.545798+0000","last_clean":"2024-04-23T14:32:50.545798+0000","last_became_active":"2024-04-23T14:32:49.410335+0000","last_became_peered":"2024-04-23T14:32:49.410335+0000","last_unstale":"2024-04-23T14:32:50.545798+0000","last_undegraded":"2024-04-23T14:32:50.545798+0000","last_fullsized":"2024-04-23T14:32:50.545798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:28.400553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403502+0000","last_change":"2024-04-23T14:32:48.372373+0000","last_active":"2024-04-23T14:32:50.403502+0000","last_peered":"2024-04-23T14:32:50.403502+0000","last_clean":"2024-04-23T14:32:50.403502+0000","last_became_active":"2024-04-23T14:32:48.372040+0000","last_became_peered":"2024-04-23T14:32:48.372040+0000","last_unstale":"2024-04-23T14:32:50.403502+0000","last_undegraded":"2024-04-23T14:32:50.403502+0000","last_fullsized":"2024-04-23T14:32:50.403502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:24.434665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545617+0000","last_change":"2024-04-23T14:32:49.403841+0000","last_active":"2024-04-23T14:32:50.545617+0000","last_peered":"2024-04-23T14:32:50.545617+0000","last_clean":"2024-04-23T14:32:50.545617+0000","last_became_active":"2024-04-23T14:32:49.403486+0000","last_became_peered":"2024-04-23T14:32:49.403486+0000","last_unstale":"2024-04-23T14:32:50.545617+0000","last_undegraded":"2024-04-23T14:32:50.545617+0000","last_fullsized":"2024-04-23T14:32:50.545617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:14.111050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406769+0000","last_change":"2024-04-23T14:32:48.381995+0000","last_active":"2024-04-23T14:32:50.406769+0000","last_peered":"2024-04-23T14:32:50.406769+0000","last_clean":"2024-04-23T14:32:50.406769+0000","last_became_active":"2024-04-23T14:32:48.381595+0000","last_became_peered":"2024-04-23T14:32:48.381595+0000","last_unstale":"2024-04-23T14:32:50.406769+0000","last_undegraded":"2024-04-23T14:32:50.406769+0000","last_fullsized":"2024-04-23T14:32:50.406769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:56.520711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545562+0000","last_change":"2024-04-23T14:32:49.405667+0000","last_active":"2024-04-23T14:32:50.545562+0000","last_peered":"2024-04-23T14:32:50.545562+0000","last_clean":"2024-04-23T14:32:50.545562+0000","last_became_active":"2024-04-23T14:32:49.403620+0000","last_became_peered":"2024-04-23T14:32:49.403620+0000","last_unstale":"2024-04-23T14:32:50.545562+0000","last_undegraded":"2024-04-23T14:32:50.545562+0000","last_fullsized":"2024-04-23T14:32:50.545562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:16:41.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.408274+0000","last_change":"2024-04-23T14:32:48.375422+0000","last_active":"2024-04-23T14:32:50.408274+0000","last_peered":"2024-04-23T14:32:50.408274+0000","last_clean":"2024-04-23T14:32:50.408274+0000","last_became_active":"2024-04-23T14:32:48.375126+0000","last_became_peered":"2024-04-23T14:32:48.375126+0000","last_unstale":"2024-04-23T14:32:50.408274+0000","last_undegraded":"2024-04-23T14:32:50.408274+0000","last_fullsized":"2024-04-23T14:32:50.408274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:50:18.218895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545939+0000","last_change":"2024-04-23T14:32:49.399201+0000","last_active":"2024-04-23T14:32:50.545939+0000","last_peered":"2024-04-23T14:32:50.545939+0000","last_clean":"2024-04-23T14:32:50.545939+0000","last_became_active":"2024-04-23T14:32:49.398682+0000","last_became_peered":"2024-04-23T14:32:49.398682+0000","last_unstale":"2024-04-23T14:32:50.545939+0000","last_undegraded":"2024-04-23T14:32:50.545939+0000","last_fullsized":"2024-04-23T14:32:50.545939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:07.894545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.543565+0000","last_change":"2024-04-23T14:32:48.378313+0000","last_active":"2024-04-23T14:32:50.543565+0000","last_peered":"2024-04-23T14:32:50.543565+0000","last_clean":"2024-04-23T14:32:50.543565+0000","last_became_active":"2024-04-23T14:32:48.377976+0000","last_became_peered":"2024-04-23T14:32:48.377976+0000","last_unstale":"2024-04-23T14:32:50.543565+0000","last_undegraded":"2024-04-23T14:32:50.543565+0000","last_fullsized":"2024-04-23T14:32:50.543565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545933+0000","last_change":"2024-04-23T14:32:49.408098+0000","last_active":"2024-04-23T14:32:50.545933+0000","last_peered":"2024-04-23T14:32:50.545933+0000","last_clean":"2024-04-23T14:32:50.545933+0000","last_became_active":"2024-04-23T14:32:49.407785+0000","last_became_peered":"2024-04-23T14:32:49.407785+0000","last_unstale":"2024-04-23T14:32:50.545933+0000","last_undegraded":"2024-04-23T14:32:50.545933+0000","last_fullsized":"2024-04-23T14:32:50.545933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:04:16.424567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407431+0000","last_change":"2024-04-23T14:32:48.386689+0000","last_active":"2024-04-23T14:32:50.407431+0000","last_peered":"2024-04-23T14:32:50.407431+0000","last_clean":"2024-04-23T14:32:50.407431+0000","last_became_active":"2024-04-23T14:32:48.386272+0000","last_became_peered":"2024-04-23T14:32:48.386272+0000","last_unstale":"2024-04-23T14:32:50.407431+0000","last_undegraded":"2024-04-23T14:32:50.407431+0000","last_fullsized":"2024-04-23T14:32:50.407431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:05.179093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546743+0000","last_change":"2024-04-23T14:32:49.407292+0000","last_active":"2024-04-23T14:32:50.546743+0000","last_peered":"2024-04-23T14:32:50.546743+0000","last_clean":"2024-04-23T14:32:50.546743+0000","last_became_active":"2024-04-23T14:32:49.406967+0000","last_became_peered":"2024-04-23T14:32:49.406967+0000","last_unstale":"2024-04-23T14:32:50.546743+0000","last_undegraded":"2024-04-23T14:32:50.546743+0000","last_fullsized":"2024-04-23T14:32:50.546743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:25:59.848909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403598+0000","last_change":"2024-04-23T14:32:48.372538+0000","last_active":"2024-04-23T14:32:50.403598+0000","last_peered":"2024-04-23T14:32:50.403598+0000","last_clean":"2024-04-23T14:32:50.403598+0000","last_became_active":"2024-04-23T14:32:48.372276+0000","last_became_peered":"2024-04-23T14:32:48.372276+0000","last_unstale":"2024-04-23T14:32:50.403598+0000","last_undegraded":"2024-04-23T14:32:50.403598+0000","last_fullsized":"2024-04-23T14:32:50.403598+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:57.601612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403211+0000","last_change":"2024-04-23T14:32:49.422420+0000","last_active":"2024-04-23T14:32:50.403211+0000","last_peered":"2024-04-23T14:32:50.403211+0000","last_clean":"2024-04-23T14:32:50.403211+0000","last_became_active":"2024-04-23T14:32:49.422096+0000","last_became_peered":"2024-04-23T14:32:49.422096+0000","last_unstale":"2024-04-23T14:32:50.403211+0000","last_undegraded":"2024-04-23T14:32:50.403211+0000","last_fullsized":"2024-04-23T14:32:50.403211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:01.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.407292+0000","last_change":"2024-04-23T14:32:48.373983+0000","last_active":"2024-04-23T14:32:50.407292+0000","last_peered":"2024-04-23T14:32:50.407292+0000","last_clean":"2024-04-23T14:32:50.407292+0000","last_became_active":"2024-04-23T14:32:48.373688+0000","last_became_peered":"2024-04-23T14:32:48.373688+0000","last_unstale":"2024-04-23T14:32:50.407292+0000","last_undegraded":"2024-04-23T14:32:50.407292+0000","last_fullsized":"2024-04-23T14:32:50.407292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406694+0000","last_change":"2024-04-23T14:32:49.398323+0000","last_active":"2024-04-23T14:32:50.406694+0000","last_peered":"2024-04-23T14:32:50.406694+0000","last_clean":"2024-04-23T14:32:50.406694+0000","last_became_active":"2024-04-23T14:32:49.398039+0000","last_became_peered":"2024-04-23T14:32:49.398039+0000","last_unstale":"2024-04-23T14:32:50.406694+0000","last_undegraded":"2024-04-23T14:32:50.406694+0000","last_fullsized":"2024-04-23T14:32:50.406694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:29:06.193582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403717+0000","last_change":"2024-04-23T14:32:48.384396+0000","last_active":"2024-04-23T14:32:50.403717+0000","last_peered":"2024-04-23T14:32:50.403717+0000","last_clean":"2024-04-23T14:32:50.403717+0000","last_became_active":"2024-04-23T14:32:48.383739+0000","last_became_peered":"2024-04-23T14:32:48.383739+0000","last_unstale":"2024-04-23T14:32:50.403717+0000","last_undegraded":"2024-04-23T14:32:50.403717+0000","last_fullsized":"2024-04-23T14:32:50.403717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:06.363814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.403391+0000","last_change":"2024-04-23T14:32:49.396368+0000","last_active":"2024-04-23T14:32:50.403391+0000","last_peered":"2024-04-23T14:32:50.403391+0000","last_clean":"2024-04-23T14:32:50.403391+0000","last_became_active":"2024-04-23T14:32:49.395031+0000","last_became_peered":"2024-04-23T14:32:49.395031+0000","last_unstale":"2024-04-23T14:32:50.403391+0000","last_undegraded":"2024-04-23T14:32:50.403391+0000","last_fullsized":"2024-04-23T14:32:50.403391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:50:36.130035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406928+0000","last_change":"2024-04-23T14:32:48.387008+0000","last_active":"2024-04-23T14:32:50.406928+0000","last_peered":"2024-04-23T14:32:50.406928+0000","last_clean":"2024-04-23T14:32:50.406928+0000","last_became_active":"2024-04-23T14:32:48.386260+0000","last_became_peered":"2024-04-23T14:32:48.386260+0000","last_unstale":"2024-04-23T14:32:50.406928+0000","last_undegraded":"2024-04-23T14:32:50.406928+0000","last_fullsized":"2024-04-23T14:32:50.406928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.543892+0000","last_change":"2024-04-23T14:32:49.415060+0000","last_active":"2024-04-23T14:32:50.543892+0000","last_peered":"2024-04-23T14:32:50.543892+0000","last_clean":"2024-04-23T14:32:50.543892+0000","last_became_active":"2024-04-23T14:32:49.414707+0000","last_became_peered":"2024-04-23T14:32:49.414707+0000","last_unstale":"2024-04-23T14:32:50.543892+0000","last_undegraded":"2024-04-23T14:32:50.543892+0000","last_fullsized":"2024-04-23T14:32:50.543892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:27.832083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.408612+0000","last_change":"2024-04-23T14:32:48.380600+0000","last_active":"2024-04-23T14:32:50.408612+0000","last_peered":"2024-04-23T14:32:50.408612+0000","last_clean":"2024-04-23T14:32:50.408612+0000","last_became_active":"2024-04-23T14:32:48.379900+0000","last_became_peered":"2024-04-23T14:32:48.379900+0000","last_unstale":"2024-04-23T14:32:50.408612+0000","last_undegraded":"2024-04-23T14:32:50.408612+0000","last_fullsized":"2024-04-23T14:32:50.408612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:31.916332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.408184+0000","last_change":"2024-04-23T14:32:49.417179+0000","last_active":"2024-04-23T14:32:50.408184+0000","last_peered":"2024-04-23T14:32:50.408184+0000","last_clean":"2024-04-23T14:32:50.408184+0000","last_became_active":"2024-04-23T14:32:49.414238+0000","last_became_peered":"2024-04-23T14:32:49.414238+0000","last_unstale":"2024-04-23T14:32:50.408184+0000","last_undegraded":"2024-04-23T14:32:50.408184+0000","last_fullsized":"2024-04-23T14:32:50.408184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:08.276131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.543852+0000","last_change":"2024-04-23T14:32:48.382557+0000","last_active":"2024-04-23T14:32:50.543852+0000","last_peered":"2024-04-23T14:32:50.543852+0000","last_clean":"2024-04-23T14:32:50.543852+0000","last_became_active":"2024-04-23T14:32:48.381989+0000","last_became_peered":"2024-04-23T14:32:48.381989+0000","last_unstale":"2024-04-23T14:32:50.543852+0000","last_undegraded":"2024-04-23T14:32:50.543852+0000","last_fullsized":"2024-04-23T14:32:50.543852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:07.261642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.546538+0000","last_change":"2024-04-23T14:32:49.406874+0000","last_active":"2024-04-23T14:32:50.546538+0000","last_peered":"2024-04-23T14:32:50.546538+0000","last_clean":"2024-04-23T14:32:50.546538+0000","last_became_active":"2024-04-23T14:32:49.406634+0000","last_became_peered":"2024-04-23T14:32:49.406634+0000","last_unstale":"2024-04-23T14:32:50.546538+0000","last_undegraded":"2024-04-23T14:32:50.546538+0000","last_fullsized":"2024-04-23T14:32:50.546538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:48.338071+0000","last_clean_scrub_stamp":"2024-04-23T14:32:48.338071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:47:45.070247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.406493+0000","last_change":"2024-04-23T14:32:48.381294+0000","last_active":"2024-04-23T14:32:50.406493+0000","last_peered":"2024-04-23T14:32:50.406493+0000","last_clean":"2024-04-23T14:32:50.406493+0000","last_became_active":"2024-04-23T14:32:48.380990+0000","last_became_peered":"2024-04-23T14:32:48.380990+0000","last_unstale":"2024-04-23T14:32:50.406493+0000","last_undegraded":"2024-04-23T14:32:50.406493+0000","last_fullsized":"2024-04-23T14:32:50.406493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:32:47.323582+0000","last_clean_scrub_stamp":"2024-04-23T14:32:47.323582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:20.954647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-23T14:32:50.545824+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:32:50.545824+0000","last_peered":"2024-04-23T14:32:50.545824+0000","last_clean":"2024-04-23T14:32:50.545824+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:32:50.545824+0000","last_undegraded":"2024-04-23T14:32:50.545824+0000","last_fullsized":"2024-04-23T14:32:50.545824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:32:59.409 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:32:59.410 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:32:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.671+0000 7f0de8c00640 1 Processor -- start 2024-04-23T14:32:59.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.671+0000 7f0de8c00640 1 -- start start 2024-04-23T14:32:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.673+0000 7f0de8c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de4156d10 0x7f0de415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.673+0000 7f0de8c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de406dd50 con 0x7f0de4156930 2024-04-23T14:32:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.673+0000 7f0de8c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de415d1f0 con 0x7f0de40ad880 2024-04-23T14:32:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.673+0000 7f0de8c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0de415d4d0 con 0x7f0de4156d10 2024-04-23T14:32:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.673+0000 7f0de3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0de4156930 0x7f0de40ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35196/0 (socket says 172.21.15.59:35196) 2024-04-23T14:32:59.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.673+0000 7f0de3400640 1 -- 172.21.15.59:0/53013980 learned_addr learned my addr 172.21.15.59:0/53013980 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:32:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044479045 0 0) 0x7f0de406dd50 con 0x7f0de4156930 2024-04-23T14:32:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dc8003520 con 0x7f0de4156930 2024-04-23T14:32:59.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de3e00640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de4156d10 0x7f0de415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3971473926 0 0) 0x7f0dc8003520 con 0x7f0de4156930 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de4156d10 msgr2=0x7f0de415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de4156d10 0x7f0de415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 >> v1:172.21.15.76:6789/0 conn(0x7f0de40ad880 legacy=0x7f0de406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de415d7b0 con 0x7f0de4156930 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f0dd4001670 con 0x7f0de4156930 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 >> v1:172.21.15.59:6789/0 conn(0x7f0de4156930 legacy=0x7f0de40ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0dc8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc8004800 0x7f0dc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de4156d10 0x7f0dc8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0de415d4d0 con 0x7f0dc80039e0 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc800baf0 con 0x7f0dc8004800 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.674+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc800bcf0 con 0x7f0de4156d10 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de3400640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0dc8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de3400640 1 -- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de4156d10 msgr2=0x7f0dc8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de3400640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de4156d10 0x7f0dc8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de3400640 1 -- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc8004800 msgr2=0x7f0dc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:32:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de2a00640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc8004800 0x7f0dc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de3400640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc8004800 0x7f0dc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de3400640 1 -- 172.21.15.59:0/53013980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc8003520 con 0x7f0dc80039e0 2024-04-23T14:32:59.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.675+0000 7f0de2a00640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0dc8004800 0x7f0dc8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.676+0000 7f0de3400640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0dc8003e00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400c240 tx=0x7f0dc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9f23e0295d55fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.677+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4007d10 con 0x7f0dc80039e0 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.677+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd4005ad0 con 0x7f0dc80039e0 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.677+0000 7f0de2000640 1 -- 172.21.15.59:0/53013980 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4007070 con 0x7f0dc80039e0 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.677+0000 7f0de8c00640 1 -- 172.21.15.59:0/53013980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 msgr2=0x7f0dc8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.677+0000 7f0de8c00640 1 --2- 172.21.15.59:0/53013980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0dc8003e00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400c240 tx=0x7f0dc8005830 comp rx=0 tx=0).stop 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.677+0000 7f0de3e00640 1 -- 172.21.15.59:0/53013980 reap_dead start 2024-04-23T14:32:59.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.678+0000 7f0de8c00640 1 -- 172.21.15.59:0/53013980 shutdown_connections 2024-04-23T14:32:59.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.678+0000 7f0de8c00640 1 -- 172.21.15.59:0/53013980 >> 172.21.15.59:0/53013980 conn(0x7f0de40b1e90 msgr2=0x7f0de40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.678+0000 7f0de8c00640 1 -- 172.21.15.59:0/53013980 shutdown_connections 2024-04-23T14:32:59.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.678+0000 7f0de8c00640 1 -- 172.21.15.59:0/53013980 wait complete. 2024-04-23T14:32:59.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.680+0000 7f0de8c00640 1 Processor -- start 2024-04-23T14:32:59.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.680+0000 7f0de8c00640 1 -- start start 2024-04-23T14:32:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0de4146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de8c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0de41e6270 0x7f0de41f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de8c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de40b0a40 0x7f0de40b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0de415ccd0 con 0x7f0dc80039e0 2024-04-23T14:32:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de8c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0de415cb50 con 0x7f0de41e6270 2024-04-23T14:32:59.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de8c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0de415ce50 con 0x7f0de40b0a40 2024-04-23T14:32:59.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0de4146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0de4146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45678/0 (socket says 172.21.15.59:45678) 2024-04-23T14:32:59.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.681+0000 7f0de3400640 1 -- 172.21.15.59:0/3566357106 learned_addr learned my addr 172.21.15.59:0/3566357106 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:32:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de2a00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0de41e6270 0x7f0de41f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3e00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de40b0a40 0x7f0de40b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3400640 1 -- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de40b0a40 msgr2=0x7f0de40b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3400640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de40b0a40 0x7f0de40b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3400640 1 -- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0de41e6270 msgr2=0x7f0de41f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3400640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0de41e6270 0x7f0de41f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3400640 1 -- 172.21.15.59:0/3566357106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de41f87e0 con 0x7f0dc80039e0 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3e00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de40b0a40 0x7f0de40b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0de3400640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0de4146530 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4034040 tx=0x7f0de415d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.682+0000 7f0ddb400640 1 -- 172.21.15.59:0/3566357106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4042020 con 0x7f0dc80039e0 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.683+0000 7f0ddb400640 1 -- 172.21.15.59:0/3566357106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd4004bf0 con 0x7f0dc80039e0 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.683+0000 7f0ddb400640 1 -- 172.21.15.59:0/3566357106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4007d10 con 0x7f0dc80039e0 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.683+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de41f8ac0 con 0x7f0dc80039e0 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.683+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de409c300 con 0x7f0dc80039e0 2024-04-23T14:32:59.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.684+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de404fd90 con 0x7f0dc80039e0 2024-04-23T14:32:59.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.685+0000 7f0ddb400640 1 -- 172.21.15.59:0/3566357106 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dd4045020 con 0x7f0dc80039e0 2024-04-23T14:32:59.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.686+0000 7f0ddb400640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0dc406c8f0 0x7f0dc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:32:59.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.686+0000 7f0ddb400640 1 -- 172.21.15.59:0/3566357106 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0dd40ac020 con 0x7f0dc80039e0 2024-04-23T14:32:59.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.686+0000 7f0de2a00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0dc406c8f0 0x7f0dc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:32:59.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.687+0000 7f0de2a00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0dc406c8f0 0x7f0dc406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0de409b3d0 tx=0x7f0dd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:32:59.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.691+0000 7f0ddb400640 1 -- 172.21.15.59:0/3566357106 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0dd407a4d0 con 0x7f0dc80039e0 2024-04-23T14:32:59.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.968+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0de4160020 con 0x7f0dc80039e0 2024-04-23T14:32:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.972+0000 7f0ddb400640 1 -- 172.21.15.59:0/3566357106 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f0dd4046e90 con 0x7f0dc80039e0 2024-04-23T14:32:59.974 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:32:59.974 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:32:59.974 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-23T14:32:59.974 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-23T14:32:59.974 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:32:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-23T14:32:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0dc406c8f0 msgr2=0x7f0dc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0dc406c8f0 0x7f0dc406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0de409b3d0 tx=0x7f0dd0009040 comp rx=0 tx=0).stop 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 msgr2=0x7f0de4146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0de4146530 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4034040 tx=0x7f0de415d980 comp rx=0 tx=0).stop 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 shutdown_connections 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0dc406c8f0 0x7f0dc406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0de40b0a40 0x7f0de40b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0de41e6270 0x7f0de41f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 --2- 172.21.15.59:0/3566357106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0dc80039e0 0x7f0de4146530 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 >> 172.21.15.59:0/3566357106 conn(0x7f0de40b1e90 msgr2=0x7f0de40b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 shutdown_connections 2024-04-23T14:32:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:32:59.975+0000 7f0de8c00640 1 -- 172.21.15.59:0/3566357106 wait complete. 2024-04-23T14:32:59.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:00.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.198+0000 7fde07000640 1 Processor -- start 2024-04-23T14:33:00.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.198+0000 7fde07000640 1 -- start start 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde07000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000d0b70 0x7fde000d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde07000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde0015e200 0x7fde0015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde07000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde000cd9f0 con 0x7fde000cd250 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde07000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fde0009b6f0 con 0x7fde0015e200 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde07000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fde000cdcb0 con 0x7fde000d0b70 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde05000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde0015e200 0x7fde0015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde05000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde0015e200 0x7fde0015e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54926/0 (socket says 172.21.15.59:54926) 2024-04-23T14:33:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fde05000640 1 -- 172.21.15.59:0/3386909925 learned_addr learned my addr 172.21.15.59:0/3386909925 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:00.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.200+0000 7fddffe00640 1 --2- 172.21.15.59:0/3386909925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000d0b70 0x7fde000d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.201+0000 7fde05000640 1 -- 172.21.15.59:0/3386909925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000d0b70 msgr2=0x7fde000d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:00.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.201+0000 7fde05000640 1 --2- 172.21.15.59:0/3386909925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000d0b70 0x7fde000d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.201+0000 7fde05000640 1 -- 172.21.15.59:0/3386909925 >> v1:172.21.15.59:6789/0 conn(0x7fde000cd250 legacy=0x7fde000d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.201+0000 7fde05000640 1 -- 172.21.15.59:0/3386909925 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde000cdf90 con 0x7fde0015e200 2024-04-23T14:33:00.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.202+0000 7fde05000640 1 --2- 172.21.15.59:0/3386909925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde0015e200 0x7fde0015e690 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009870 tx=0x7fddf802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa25c8f084db45f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.202+0000 7fddfea00640 1 -- 172.21.15.59:0/3386909925 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8037070 con 0x7fde0015e200 2024-04-23T14:33:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.202+0000 7fddfea00640 1 -- 172.21.15.59:0/3386909925 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf8032440 con 0x7fde0015e200 2024-04-23T14:33:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.202+0000 7fddfea00640 1 -- 172.21.15.59:0/3386909925 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8032810 con 0x7fde0015e200 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 -- 172.21.15.59:0/3386909925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde0015e200 msgr2=0x7fde0015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 --2- 172.21.15.59:0/3386909925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde0015e200 0x7fde0015e690 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009870 tx=0x7fddf802a1c0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 -- 172.21.15.59:0/3386909925 shutdown_connections 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 --2- 172.21.15.59:0/3386909925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde0015e200 0x7fde0015e690 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 --2- 172.21.15.59:0/3386909925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000d0b70 0x7fde000d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 -- 172.21.15.59:0/3386909925 >> 172.21.15.59:0/3386909925 conn(0x7fde000b4080 msgr2=0x7fde000b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 -- 172.21.15.59:0/3386909925 shutdown_connections 2024-04-23T14:33:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.203+0000 7fde07000640 1 -- 172.21.15.59:0/3386909925 wait complete. 2024-04-23T14:33:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.205+0000 7fde07000640 1 Processor -- start 2024-04-23T14:33:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.205+0000 7fde07000640 1 -- start start 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fde07000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000cd250 0x7fde0015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fde07000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde000d0b70 0x7fde001f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fde07000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0015e200 0x7fde000c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fde07000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde0015ebd0 con 0x7fde000d0b70 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fde07000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde000cd690 con 0x7fde0015e200 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fde07000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fde0015ed50 con 0x7fde000cd250 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fddff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde000d0b70 0x7fde001f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fddff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde000d0b70 0x7fde001f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54936/0 (socket says 172.21.15.59:54936) 2024-04-23T14:33:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.206+0000 7fddff400640 1 -- 172.21.15.59:0/1318490929 learned_addr learned my addr 172.21.15.59:0/1318490929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:00.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fde05000640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0015e200 0x7fde000c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddffe00640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000cd250 0x7fde0015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddff400640 1 -- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000cd250 msgr2=0x7fde0015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddff400640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000cd250 0x7fde0015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddff400640 1 -- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0015e200 msgr2=0x7fde000c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddff400640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0015e200 0x7fde000c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddff400640 1 -- 172.21.15.59:0/1318490929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0015bf90 con 0x7fde000d0b70 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddff400640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde000d0b70 0x7fde001f60f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdde800c3c0 tx=0x7fdde800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddf7e00640 1 -- 172.21.15.59:0/1318490929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde80052d0 con 0x7fde000d0b70 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddf7e00640 1 -- 172.21.15.59:0/1318490929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde8005490 con 0x7fde000d0b70 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fddf7e00640 1 -- 172.21.15.59:0/1318490929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8007260 con 0x7fde000d0b70 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.207+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde000c93e0 con 0x7fde000d0b70 2024-04-23T14:33:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.208+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde000c9100 con 0x7fde000d0b70 2024-04-23T14:33:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.209+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde00070750 con 0x7fde000d0b70 2024-04-23T14:33:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.210+0000 7fddf7e00640 1 -- 172.21.15.59:0/1318490929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdde8007400 con 0x7fde000d0b70 2024-04-23T14:33:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.211+0000 7fddf7e00640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.211+0000 7fddf7e00640 1 -- 172.21.15.59:0/1318490929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdde808aff0 con 0x7fde000d0b70 2024-04-23T14:33:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.211+0000 7fddffe00640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.212+0000 7fddffe00640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fddf00099c0 tx=0x7fddf0003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:00.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.215+0000 7fddf7e00640 1 -- 172.21.15.59:0/1318490929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde00070750 con 0x7fde000d0b70 2024-04-23T14:33:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.450+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde00068a80 con 0x7fde000d0b70 2024-04-23T14:33:00.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.451+0000 7fddf7e00640 1 -- 172.21.15.59:0/1318490929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7fdde8059420 con 0x7fde000d0b70 2024-04-23T14:33:00.452 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:00.452 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:33:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdddc06c8f0 msgr2=0x7fdddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fddf00099c0 tx=0x7fddf0003990 comp rx=0 tx=0).stop 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde000d0b70 msgr2=0x7fde001f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde000d0b70 0x7fde001f60f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdde800c3c0 tx=0x7fdde800c880 comp rx=0 tx=0).stop 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 shutdown_connections 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0015e200 0x7fde000c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde000d0b70 0x7fde001f60f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 --2- 172.21.15.59:0/1318490929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde000cd250 0x7fde0015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 >> 172.21.15.59:0/1318490929 conn(0x7fde000b4080 msgr2=0x7fde00070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 shutdown_connections 2024-04-23T14:33:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:00.454+0000 7fde07000640 1 -- 172.21.15.59:0/1318490929 wait complete. 2024-04-23T14:33:00.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:32:48.829992+0000', 'modified': '2024-04-23T14:32:51.932361+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5205}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5205': {'gid': 5205, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6836/1671400742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 1671400742}, {'type': 'v1', 'addr': '172.21.15.59:6836', 'nonce': 1671400742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5205, 'qdb_cluster': [5205]} 2024-04-23T14:33:00.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:33:00.878 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:33:01.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.094+0000 7fd7db200640 1 Processor -- start 2024-04-23T14:33:01.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.095+0000 7fd7db200640 1 -- start start 2024-04-23T14:33:01.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.096+0000 7fd7db200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd7dc0d0f80 0x7fd7dc0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.096+0000 7fd7db200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7dc15ea40 0x7fd7dc15f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.096+0000 7fd7db200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7dc0cf930 con 0x7fd7dc0d1360 2024-04-23T14:33:01.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.096+0000 7fd7db200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd7dc15fdc0 con 0x7fd7dc15ea40 2024-04-23T14:33:01.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.096+0000 7fd7db200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd7dc0cfc10 con 0x7fd7dc0d0f80 2024-04-23T14:33:01.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7dc15ea40 0x7fd7dc15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7dc15ea40 0x7fd7dc15f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54948/0 (socket says 172.21.15.59:54948) 2024-04-23T14:33:01.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9400640 1 -- 172.21.15.59:0/1254544490 learned_addr learned my addr 172.21.15.59:0/1254544490 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/1254544490 >> v2:172.21.15.76:3300/0 conn(0x7fd7dc0d0f80 0x7fd7dc0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9400640 1 -- 172.21.15.59:0/1254544490 >> v2:172.21.15.76:3300/0 conn(0x7fd7dc0d0f80 msgr2=0x7fd7dc0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9400640 1 --2- 172.21.15.59:0/1254544490 >> v2:172.21.15.76:3300/0 conn(0x7fd7dc0d0f80 0x7fd7dc0cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9400640 1 -- 172.21.15.59:0/1254544490 >> v1:172.21.15.76:6789/0 conn(0x7fd7dc0d1360 legacy=0x7fd7dc162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9400640 1 -- 172.21.15.59:0/1254544490 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc0cfef0 con 0x7fd7dc15ea40 2024-04-23T14:33:01.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.097+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/1254544490 >> v2:172.21.15.76:3300/0 conn(0x7fd7dc0d0f80 0x7fd7dc0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:01.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.098+0000 7fd7d9400640 1 --2- 172.21.15.59:0/1254544490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc15f880 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0d1e90 tx=0x7fd7c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe18d4a5a80240c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.098+0000 7fd7d8a00640 1 -- 172.21.15.59:0/1254544490 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4036070 con 0x7fd7dc15ea40 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7d8a00640 1 -- 172.21.15.59:0/1254544490 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c4031600 con 0x7fd7dc15ea40 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7d8a00640 1 -- 172.21.15.59:0/1254544490 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40319d0 con 0x7fd7dc15ea40 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7db200640 1 -- 172.21.15.59:0/1254544490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 msgr2=0x7fd7dc15f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7db200640 1 --2- 172.21.15.59:0/1254544490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc15f880 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0d1e90 tx=0x7fd7c402a660 comp rx=0 tx=0).stop 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7db200640 1 -- 172.21.15.59:0/1254544490 shutdown_connections 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7db200640 1 --2- 172.21.15.59:0/1254544490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc15f880 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7db200640 1 --2- 172.21.15.59:0/1254544490 >> v2:172.21.15.76:3300/0 conn(0x7fd7dc0d0f80 0x7fd7dc0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7db200640 1 -- 172.21.15.59:0/1254544490 >> 172.21.15.59:0/1254544490 conn(0x7fd7dc08bf90 msgr2=0x7fd7dc0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.099+0000 7fd7db200640 1 -- 172.21.15.59:0/1254544490 shutdown_connections 2024-04-23T14:33:01.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.100+0000 7fd7db200640 1 -- 172.21.15.59:0/1254544490 wait complete. 2024-04-23T14:33:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.101+0000 7fd7db200640 1 Processor -- start 2024-04-23T14:33:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.101+0000 7fd7db200640 1 -- start start 2024-04-23T14:33:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.102+0000 7fd7db200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7dc0d0f80 0x7fd7dc0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.102+0000 7fd7db200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7dc0d1360 0x7fd7dc0b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.102+0000 7fd7db200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc0b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.102+0000 7fd7db200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc0d1810 con 0x7fd7dc15ea40 2024-04-23T14:33:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.102+0000 7fd7db200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc0d04b0 con 0x7fd7dc0d0f80 2024-04-23T14:33:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.102+0000 7fd7db200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7dc0d3320 con 0x7fd7dc0d1360 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc0b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc0b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54954/0 (socket says 172.21.15.59:54954) 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 -- 172.21.15.59:0/87663958 learned_addr learned my addr 172.21.15.59:0/87663958 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7dc0d0f80 0x7fd7dc0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 -- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7dc0d1360 msgr2=0x7fd7dc0b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7dc0d1360 0x7fd7dc0b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 -- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7dc0d0f80 msgr2=0x7fd7dc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7dc0d0f80 0x7fd7dc0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 -- 172.21.15.59:0/87663958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc0c9be0 con 0x7fd7dc15ea40 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7dc0d0f80 0x7fd7dc0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.103+0000 7fd7da800640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc0b6490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400c410 tx=0x7fd7d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.104+0000 7fd7d2000640 1 -- 172.21.15.59:0/87663958 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4011790 con 0x7fd7dc15ea40 2024-04-23T14:33:01.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.104+0000 7fd7d2000640 1 -- 172.21.15.59:0/87663958 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d4017070 con 0x7fd7dc15ea40 2024-04-23T14:33:01.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.104+0000 7fd7d2000640 1 -- 172.21.15.59:0/87663958 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4011bb0 con 0x7fd7dc15ea40 2024-04-23T14:33:01.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.104+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7dc0c9ec0 con 0x7fd7dc15ea40 2024-04-23T14:33:01.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.104+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7dc0ca760 con 0x7fd7dc15ea40 2024-04-23T14:33:01.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.106+0000 7fd7d2000640 1 -- 172.21.15.59:0/87663958 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7d401c460 con 0x7fd7dc15ea40 2024-04-23T14:33:01.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.107+0000 7fd7d2000640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7b806c940 0x7fd7b806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.107+0000 7fd7d2000640 1 -- 172.21.15.59:0/87663958 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd7d408b740 con 0x7fd7dc15ea40 2024-04-23T14:33:01.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.107+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7b806c940 0x7fd7b806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.108+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7b806c940 0x7fd7b806ee00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc06cb20 tx=0x7fd7cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.108+0000 7fd7e382f740 1 Processor -- start 2024-04-23T14:33:01.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.108+0000 7fd7e382f740 1 -- start start 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f76310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa6aa0 0x55a4f5fa5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6360 0x55a4f5f7b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7e382f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5ee0b10 con 0x55a4f5fa6360 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5e872b0 con 0x55a4f5fc3980 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a4f5edcad0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7da800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6360 0x55a4f5f7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7da800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6360 0x55a4f5f7b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54970/0 (socket says 172.21.15.59:54970) 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.109+0000 7fd7da800640 1 -- 172.21.15.59:0/2979771577 learned_addr learned my addr 172.21.15.59:0/2979771577 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7da800640 1 -- 172.21.15.59:0/2979771577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa6aa0 msgr2=0x55a4f5fa5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7da800640 1 --2- 172.21.15.59:0/2979771577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa6aa0 0x55a4f5fa5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7da800640 1 -- 172.21.15.59:0/2979771577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 msgr2=0x55a4f5f76310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7da800640 1 --2- 172.21.15.59:0/2979771577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f76310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7da800640 1 -- 172.21.15.59:0/2979771577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f5bf32d0 con 0x55a4f5fa6360 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7da800640 1 --2- 172.21.15.59:0/2979771577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6360 0x55a4f5f7b170 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4003950 tx=0x7fd7d4058ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dbea8aa298605d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7ca600640 1 -- 172.21.15.59:0/2979771577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d406de60 con 0x55a4f5fa6360 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7ca600640 1 -- 172.21.15.59:0/2979771577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d4011d50 con 0x55a4f5fa6360 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.110+0000 7fd7ca600640 1 -- 172.21.15.59:0/2979771577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4061ba0 con 0x55a4f5fa6360 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 -- 172.21.15.59:0/2979771577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6360 msgr2=0x55a4f5f7b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 --2- 172.21.15.59:0/2979771577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6360 0x55a4f5f7b170 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4003950 tx=0x7fd7d4058ee0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 -- 172.21.15.59:0/2979771577 shutdown_connections 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 --2- 172.21.15.59:0/2979771577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa6aa0 0x55a4f5fa5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 --2- 172.21.15.59:0/2979771577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6360 0x55a4f5f7b170 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 --2- 172.21.15.59:0/2979771577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f76310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 -- 172.21.15.59:0/2979771577 >> 172.21.15.59:0/2979771577 conn(0x55a4f5f78580 msgr2=0x55a4f5fc0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 -- 172.21.15.59:0/2979771577 shutdown_connections 2024-04-23T14:33:01.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.111+0000 7fd7e382f740 1 -- 172.21.15.59:0/2979771577 wait complete. 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.112+0000 7fd7e382f740 1 -- start start 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.112+0000 7fd7e382f740 1 Processor -- start 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fc2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f7b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa97e0 0x55a4f5f7b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7e382f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5f7a730 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5e9a590 con 0x55a4f5fc3980 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a4f5f79b20 con 0x55a4f5fa97e0 2024-04-23T14:33:01.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7e382f740 10 client.0 _ll_register_callbacks cb 0x55a4f5e7d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fc2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fc2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54986/0 (socket says 172.21.15.59:54986) 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.113+0000 7fd7d9e00640 1 -- 172.21.15.59:0/515119688 learned_addr learned my addr 172.21.15.59:0/515119688 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9400640 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f7b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9e00640 1 -- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa97e0 msgr2=0x55a4f5f7b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa97e0 0x55a4f5f7b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9e00640 1 -- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 msgr2=0x55a4f5f7b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f7b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9e00640 1 -- 172.21.15.59:0/515119688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f5cbd6b0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9400640 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f7b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fc2c50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0126c0 tx=0x7fd7cc0087e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab18f7f13c4e886 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7c2a00640 1 -- 172.21.15.59:0/515119688 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc023060 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d4011d50 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.114+0000 7fd7c2a00640 1 -- 172.21.15.59:0/515119688 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd7cc01e600 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7c2a00640 1 -- 172.21.15.59:0/515119688 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc029050 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 10 client.5415 fetch_fsmap learned FSMap version 16 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a4f5fa82f0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7c2a00640 1 -- 172.21.15.59:0/515119688 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d4011d50 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7c2a00640 1 -- 172.21.15.59:0/515119688 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fd7cc02c020 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 10 client.5415 fetch_fsmap finished waiting for FSMap version 16 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 10 client.5415 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 20 client.5415 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f5f946d0 0x55a4f5fc1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 4 client.5415 mds_command: new command op to 5205 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.115+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 --> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f5eee530 con 0x55a4f5f946d0 2024-04-23T14:33:01.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.116+0000 7fd7d9400640 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f5f946d0 0x55a4f5fc1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.116+0000 7fd7d9400640 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f5f946d0 0x55a4f5fc1790 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d5fa5d70f88c80 server_cookie=5482b953d2a0eb91 in_seq=0 out_seq=0 2024-04-23T14:33:01.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.116+0000 7fd7c2a00640 10 client.5415 ms_handle_connect on v2:172.21.15.59:6835/1671400742 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7c2a00640 1 -- 172.21.15.59:0/515119688 <== mds.0 v2:172.21.15.59:6835/1671400742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4f5eee530 con 0x55a4f5f946d0 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7c2a00640 10 client.5415 handle_command_reply: tid=0 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 client.5415 shutdown 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 msgr2=0x55a4f5fc2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fc2c50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0126c0 tx=0x7fd7cc0087e0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 shutdown_connections 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f5f946d0 0x55a4f5fc1790 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa97e0 0x55a4f5f7b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5f7b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7c2a00640 0 client.5415 ms_handle_reset on v2:172.21.15.59:6835/1671400742 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 --2- 172.21.15.59:0/515119688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fc2c50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.117+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 >> 172.21.15.59:0/515119688 conn(0x55a4f5fa6360 msgr2=0x55a4f5f7ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.118+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 shutdown_connections 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.118+0000 7fd7e382f740 1 -- 172.21.15.59:0/515119688 wait complete. 2024-04-23T14:33:01.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.118+0000 7fd7e382f740 20 client.5415 trim_cache size 0 max 16384 2024-04-23T14:33:01.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.138+0000 7fd7e382f740 1 Processor -- start 2024-04-23T14:33:01.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.138+0000 7fd7e382f740 1 -- start start 2024-04-23T14:33:01.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fbfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5fc03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa1040 0x55a4f5fa14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7e382f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc200dd0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1f8c80 con 0x55a4f5fc3980 2024-04-23T14:33:01.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7dc1f7fe0 con 0x55a4f5fa1040 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fbfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fbfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54998/0 (socket says 172.21.15.59:54998) 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9e00640 1 -- 172.21.15.59:0/615496868 learned_addr learned my addr 172.21.15.59:0/615496868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9400640 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5fc03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9e00640 1 -- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa1040 msgr2=0x55a4f5fa14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa1040 0x55a4f5fa14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9e00640 1 -- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 msgr2=0x55a4f5fc03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.139+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5fc03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.140+0000 7fd7d9e00640 1 -- 172.21.15.59:0/615496868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f5f796c0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.140+0000 7fd7d9400640 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5fc03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.140+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fbfea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01e640 tx=0x7fd7cc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c628805a02d954 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.140+0000 7fd7c9c00640 1 -- 172.21.15.59:0/615496868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc01ed30 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.140+0000 7fd7c9c00640 1 -- 172.21.15.59:0/615496868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7cc031070 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.140+0000 7fd7c9c00640 1 -- 172.21.15.59:0/615496868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc02d5f0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 -- 172.21.15.59:0/615496868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 msgr2=0x55a4f5fbfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fbfea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01e640 tx=0x7fd7cc01c000 comp rx=0 tx=0).stop 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 -- 172.21.15.59:0/615496868 shutdown_connections 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fa1040 0x55a4f5fa14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fc3980 0x55a4f5fc03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 --2- 172.21.15.59:0/615496868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fbfea0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 -- 172.21.15.59:0/615496868 >> 172.21.15.59:0/615496868 conn(0x55a4f5fa6360 msgr2=0x55a4f5f719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 -- 172.21.15.59:0/615496868 shutdown_connections 2024-04-23T14:33:01.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.141+0000 7fd7e382f740 1 -- 172.21.15.59:0/615496868 wait complete. 2024-04-23T14:33:01.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.142+0000 7fd7e382f740 1 -- start start 2024-04-23T14:33:01.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.142+0000 7fd7e382f740 1 Processor -- start 2024-04-23T14:33:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fa1040 0x55a4f5fc02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fa2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7e382f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fc3980 0x55a4f603b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7e382f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5f94550 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1f8630 con 0x55a4f5fa1040 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7e382f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a4f5f94b40 con 0x55a4f5fc3980 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7e382f740 10 client.0 _ll_register_callbacks cb 0x7fd7cc00de50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fa2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fa2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55002/0 (socket says 172.21.15.59:55002) 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9400640 1 -- 172.21.15.59:0/159404487 learned_addr learned my addr 172.21.15.59:0/159404487 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fa1040 0x55a4f5fc02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9400640 1 -- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fc3980 msgr2=0x55a4f603b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7da800640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fc3980 0x55a4f603b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9400640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fc3980 0x55a4f603b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9400640 1 -- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fa1040 msgr2=0x55a4f5fc02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.143+0000 7fd7d9400640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fa1040 0x55a4f5fc02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.144+0000 7fd7d9400640 1 -- 172.21.15.59:0/159404487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f5fa1fd0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.144+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fa1040 0x55a4f5fc02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.144+0000 7fd7d9400640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fa2a40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4000f80 tx=0x7fd7c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfb2fbf4a0decf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.144+0000 7fd7c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7c2a00640 1 -- 172.21.15.59:0/159404487 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4042e60 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7c2a00640 1 -- 172.21.15.59:0/159404487 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd7c40422e0 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc031070 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7c2a00640 1 -- 172.21.15.59:0/159404487 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fd7c4053050 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 10 client.5421 fetch_fsmap learned FSMap version 16 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a4f5fa2f80 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7c2a00640 1 -- 172.21.15.59:0/159404487 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7cc031070 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7c2a00640 1 -- 172.21.15.59:0/159404487 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fd7c4056020 con 0x55a4f5fa6aa0 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 10 client.5421 fetch_fsmap finished waiting for FSMap version 16 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 10 client.5421 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:33:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 20 client.5421 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f6024d10 0x55a4f5fd3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 4 client.5421 mds_command: new command op to 5205 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:33:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.145+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 --> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] -- command(tid 0: {"prefix": "status"}) -- 0x55a4f5ee83a0 con 0x55a4f6024d10 2024-04-23T14:33:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.146+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f6024d10 0x55a4f5fd3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.146+0000 7fd7d9e00640 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f6024d10 0x55a4f5fd3030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98647fd5fa79ddb8 server_cookie=33aa31e986bda24a in_seq=0 out_seq=0 2024-04-23T14:33:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.146+0000 7fd7c2a00640 10 client.5421 ms_handle_connect on v2:172.21.15.59:6835/1671400742 2024-04-23T14:33:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7c2a00640 1 -- 172.21.15.59:0/159404487 <== mds.0 v2:172.21.15.59:6835/1671400742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a4f5ee83a0 con 0x55a4f6024d10 2024-04-23T14:33:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7c2a00640 10 client.5421 handle_command_reply: tid=0 2024-04-23T14:33:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 client.5421 shutdown 2024-04-23T14:33:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 msgr2=0x55a4f5fa2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fa2a40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4000f80 tx=0x7fd7c4004a40 comp rx=0 tx=0).stop 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 shutdown_connections 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:6835/1671400742,v1:172.21.15.59:6836/1671400742] conn(0x55a4f6024d10 0x55a4f5fd3030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a4f5fc3980 0x55a4f603b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f5fa6aa0 0x55a4f5fa2a40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 --2- 172.21.15.59:0/159404487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a4f5fa1040 0x55a4f5fc02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7c2a00640 0 client.5421 ms_handle_reset on v2:172.21.15.59:6835/1671400742 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 >> 172.21.15.59:0/159404487 conn(0x55a4f5fa6360 msgr2=0x55a4f5fc2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 shutdown_connections 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 1 -- 172.21.15.59:0/159404487 wait complete. 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.147+0000 7fd7e382f740 20 client.5421 trim_cache size 0 max 16384 2024-04-23T14:33:01.150 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5205, 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 11.755092663999999, 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 16, 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 33, 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 33, 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 20.684237943999999 2024-04-23T14:33:01.151 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7b806c940 msgr2=0x7fd7b806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7b806c940 0x7fd7b806ee00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc06cb20 tx=0x7fd7cc008040 comp rx=0 tx=0).stop 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 msgr2=0x7fd7dc0b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc0b6490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400c410 tx=0x7fd7d400c8d0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 shutdown_connections 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7b806c940 0x7fd7b806ee00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7dc0d1360 0x7fd7dc0b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc15ea40 0x7fd7dc0b6490 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 --2- 172.21.15.59:0/87663958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7dc0d0f80 0x7fd7dc0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 >> 172.21.15.59:0/87663958 conn(0x7fd7dc08bf90 msgr2=0x7fd7dc161630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 shutdown_connections 2024-04-23T14:33:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.148+0000 7fd7db200640 1 -- 172.21.15.59:0/87663958 wait complete. 2024-04-23T14:33:01.165 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:33:01.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.359+0000 7f731fe00640 1 Processor -- start 2024-04-23T14:33:01.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.359+0000 7f731fe00640 1 -- start start 2024-04-23T14:33:01.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f732015e7c0 0x7f73200d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200d2630 0x7f73200b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731fe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73200cf030 con 0x7f73200b6530 2024-04-23T14:33:01.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f73200b7000 con 0x7f732015e7c0 2024-04-23T14:33:01.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f73200cf620 con 0x7f73200d2630 2024-04-23T14:33:01.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f732015e7c0 0x7f73200d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f732015e7c0 0x7f73200d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55006/0 (socket says 172.21.15.59:55006) 2024-04-23T14:33:01.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.361+0000 7f731ea00640 1 -- 172.21.15.59:0/1235341279 learned_addr learned my addr 172.21.15.59:0/1235341279 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.362+0000 7f731ea00640 1 -- 172.21.15.59:0/1235341279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200d2630 msgr2=0x7f73200b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.362+0000 7f731ea00640 1 --2- 172.21.15.59:0/1235341279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200d2630 0x7f73200b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.362+0000 7f731ea00640 1 -- 172.21.15.59:0/1235341279 >> v1:172.21.15.76:6789/0 conn(0x7f73200b6530 legacy=0x7f73200d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:01.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.362+0000 7f731ea00640 1 -- 172.21.15.59:0/1235341279 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200b6a40 con 0x7f732015e7c0 2024-04-23T14:33:01.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.363+0000 7f731ea00640 1 --2- 172.21.15.59:0/1235341279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732015e7c0 0x7f73200d20f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7310009870 tx=0x7f731002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72467ec31b72afa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.363+0000 7f731e000640 1 -- 172.21.15.59:0/1235341279 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310037070 con 0x7f732015e7c0 2024-04-23T14:33:01.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.363+0000 7f731e000640 1 -- 172.21.15.59:0/1235341279 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7310032440 con 0x7f732015e7c0 2024-04-23T14:33:01.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.363+0000 7f731e000640 1 -- 172.21.15.59:0/1235341279 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310032810 con 0x7f732015e7c0 2024-04-23T14:33:01.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 -- 172.21.15.59:0/1235341279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732015e7c0 msgr2=0x7f73200d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 --2- 172.21.15.59:0/1235341279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732015e7c0 0x7f73200d20f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7310009870 tx=0x7f731002a210 comp rx=0 tx=0).stop 2024-04-23T14:33:01.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 -- 172.21.15.59:0/1235341279 shutdown_connections 2024-04-23T14:33:01.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 --2- 172.21.15.59:0/1235341279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200d2630 0x7f73200b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 --2- 172.21.15.59:0/1235341279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732015e7c0 0x7f73200d20f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 -- 172.21.15.59:0/1235341279 >> 172.21.15.59:0/1235341279 conn(0x7f732009d920 msgr2=0x7f732008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 -- 172.21.15.59:0/1235341279 shutdown_connections 2024-04-23T14:33:01.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.364+0000 7f731fe00640 1 -- 172.21.15.59:0/1235341279 wait complete. 2024-04-23T14:33:01.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.366+0000 7f731fe00640 1 Processor -- start 2024-04-23T14:33:01.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.366+0000 7f731fe00640 1 -- start start 2024-04-23T14:33:01.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f731fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200b6530 0x7f73200cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f731fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73200d2630 0x7f73200b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f731fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732015e7c0 0x7f732014c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f731fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73200d1a80 con 0x7f73200d2630 2024-04-23T14:33:01.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f731fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f73200cf2b0 con 0x7f732015e7c0 2024-04-23T14:33:01.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f731fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f73200d1c00 con 0x7f73200b6530 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f7317e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73200d2630 0x7f73200b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f7317e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73200d2630 0x7f73200b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55012/0 (socket says 172.21.15.59:55012) 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.367+0000 7f7317e00640 1 -- 172.21.15.59:0/877271350 learned_addr learned my addr 172.21.15.59:0/877271350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f731ea00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200b6530 0x7f73200cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f731f400640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732015e7c0 0x7f732014c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7317e00640 1 -- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200b6530 msgr2=0x7f73200cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7317e00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200b6530 0x7f73200cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7317e00640 1 -- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732015e7c0 msgr2=0x7f732014c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7317e00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732015e7c0 0x7f732014c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7317e00640 1 -- 172.21.15.59:0/877271350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732015bba0 con 0x7f73200d2630 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f731f400640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732015e7c0 0x7f732014c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:01.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7317e00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73200d2630 0x7f73200b4f50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f730800b450 tx=0x7f730800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7316a00640 1 -- 172.21.15.59:0/877271350 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308015020 con 0x7f73200d2630 2024-04-23T14:33:01.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7316a00640 1 -- 172.21.15.59:0/877271350 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7308011890 con 0x7f73200d2630 2024-04-23T14:33:01.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.368+0000 7f7316a00640 1 -- 172.21.15.59:0/877271350 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308011c80 con 0x7f73200d2630 2024-04-23T14:33:01.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.369+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732015be80 con 0x7f73200d2630 2024-04-23T14:33:01.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.369+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732015c160 con 0x7f73200d2630 2024-04-23T14:33:01.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.370+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732004fd90 con 0x7f73200d2630 2024-04-23T14:33:01.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.371+0000 7f7316a00640 1 -- 172.21.15.59:0/877271350 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7308018020 con 0x7f73200d2630 2024-04-23T14:33:01.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.371+0000 7f7316a00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72fc06c8f0 0x7f72fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:01.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.372+0000 7f7316a00640 1 -- 172.21.15.59:0/877271350 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f730808bd70 con 0x7f73200d2630 2024-04-23T14:33:01.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.372+0000 7f731ea00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72fc06c8f0 0x7f72fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:01.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.372+0000 7f731ea00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72fc06c8f0 0x7f72fc06edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f731002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:01.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.376+0000 7f7316a00640 1 -- 172.21.15.59:0/877271350 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f730801f020 con 0x7f73200d2630 2024-04-23T14:33:01.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.625+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f73200d04e0 con 0x7f73200d2630 2024-04-23T14:33:01.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.626+0000 7f7316a00640 1 -- 172.21.15.59:0/877271350 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f730800d040 con 0x7f73200d2630 2024-04-23T14:33:01.626 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72fc06c8f0 msgr2=0x7f72fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72fc06c8f0 0x7f72fc06edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f731002e000 comp rx=0 tx=0).stop 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73200d2630 msgr2=0x7f73200b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73200d2630 0x7f73200b4f50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f730800b450 tx=0x7f730800b910 comp rx=0 tx=0).stop 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 shutdown_connections 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72fc06c8f0 0x7f72fc06edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732015e7c0 0x7f732014c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73200d2630 0x7f73200b4f50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 --2- 172.21.15.59:0/877271350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73200b6530 0x7f73200cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.628+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 >> 172.21.15.59:0/877271350 conn(0x7f732009d920 msgr2=0x7f7320161aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.629+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 shutdown_connections 2024-04-23T14:33:01.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01.629+0000 7f731fe00640 1 -- 172.21.15.59:0/877271350 wait complete. 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:33:01.648 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:01.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:01.649 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:33:01.704 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:33:01.705 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42367sec preferred_lft 42367sec 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:33:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:33:01.707 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:01.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:01.708 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:33:01.789 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:33:01.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:01.793 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:33:01.849 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:33:01.849 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:33:01.849 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:33:01.849 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:33:01.849 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:33:01.850 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:33:01.850 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:33:01.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:33:01.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:33:01.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:33:01.850 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:33:01.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:33:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:02.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:02.078 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:33:02.129 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:33:02.129 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:33:02.130 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:02.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:02.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:33:02.218 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:33:02.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:02.218 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:02.272 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:02.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:02.273 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:02.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:33:02.414 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:02.470 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:33:02.470 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:33:02.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:33:02.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:02.473 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:02.473 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:02.473 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:02.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:02.474 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:33:02.526 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:33:02.527 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:33:02.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:33:02.604 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:02.645 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:33:02.645 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:02.646 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:02.647 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:02.648 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:02.648 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:33:02.648 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:02.648 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:02.648 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:02.648 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:02.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:02.649 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:33:02.710 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:33:02.711+0000 7fd805f20480 -1 init, newargv = 0x559984851250 newargc=15 2024-04-23T14:33:02.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[47389]: starting ceph client 2024-04-23T14:33:02.719 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[47389]: starting fuse 2024-04-23T14:33:02.737 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:33:02.737 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:33:02.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:33:02.737 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:33:02.737 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:33:02.737 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:33:02.738 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:33:02.739 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:33:02.739 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:33:02.739 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:33:02.739 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:02.739 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:33:02.739 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:02.941 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.47389.asok 2024-04-23T14:33:02.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:02.959 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47389.asok 2024-04-23T14:33:02.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:02.960 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47389.asok status 2024-04-23T14:33:03.195 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "47389", 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:33:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5430, 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 5430 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1134332958 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:33:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1134332958 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.5430 192.168.0.1:0/1134332958", 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/1134332958", 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 16, 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 33, 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:33:03.198 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:03.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.211 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:33:03.211 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:03.269 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:33:03.269 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:03.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.269 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:03.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.361 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-23T14:33:03.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.431 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-23T14:33:03.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.494 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-23T14:33:03.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.553 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-23T14:33:03.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.616 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-23T14:33:03.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:03.676 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-23T14:33:03.758 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-23T14:33:03.758 INFO:teuthology.orchestra.run.smithi059.stdout:./rfile 2024-04-23T14:33:03.758 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir 2024-04-23T14:33:03.758 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir/pfile 2024-04-23T14:33:03.758 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir/childdir 2024-04-23T14:33:03.758 INFO:teuthology.orchestra.run.smithi059.stdout:./parentdir/childdir/cfile 2024-04-23T14:33:03.760 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:33:03.760 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:03.760 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:33:03.760 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:03.760 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-23T14:33:03.760 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:03.908 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:33:03.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:03.923 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:33:03.923 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:03.923 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:33:03.923 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:03.923 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-23T14:33:03.923 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:04.115 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-23T14:33:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:04.129 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:33:04.130 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:04.130 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:33:04.130 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:04.130 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-23T14:33:04.130 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:04.324 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-23T14:33:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:04.339 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:33:04.339 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:04.339 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:33:04.339 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:04.339 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-23T14:33:04.339 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:04.533 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628279 2024-04-23T14:33:04.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:04.547 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:33:04.547 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:04.547 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:33:04.547 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:04.547 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-23T14:33:04.547 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:04.742 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-23T14:33:04.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:04.757 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:33:04.757 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:04.757 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:33:04.757 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:04.757 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-23T14:33:04.757 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:04.951 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-23T14:33:04.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:04.966 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:33:04.966 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:05.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:05.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.279+0000 7fc3a1200640 1 Processor -- start 2024-04-23T14:33:05.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.280+0000 7fc3a1200640 1 -- start start 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.281+0000 7fc3a1200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc39c0b4130 0x7fc39c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.281+0000 7fc3a1200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c0cfb90 0x7fc39c0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.281+0000 7fc3a1200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc39c06ed40 con 0x7fc39c162df0 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.281+0000 7fc3a1200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc39c06eaf0 con 0x7fc39c0b4130 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.281+0000 7fc3a1200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc39c0b3590 con 0x7fc39c0cfb90 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc39c0b4130 0x7fc39c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc39c0b4130 0x7fc39c06f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55022/0 (socket says 172.21.15.59:55022) 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39aa00640 1 -- 172.21.15.59:0/2238532342 learned_addr learned my addr 172.21.15.59:0/2238532342 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:05.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39be00640 1 --2- 172.21.15.59:0/2238532342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c0cfb90 0x7fc39c0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39aa00640 1 -- 172.21.15.59:0/2238532342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c0cfb90 msgr2=0x7fc39c0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:05.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39aa00640 1 --2- 172.21.15.59:0/2238532342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c0cfb90 0x7fc39c0c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39aa00640 1 -- 172.21.15.59:0/2238532342 >> v1:172.21.15.59:6789/0 conn(0x7fc39c162df0 legacy=0x7fc39c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.282+0000 7fc39aa00640 1 -- 172.21.15.59:0/2238532342 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c0b3220 con 0x7fc39c0b4130 2024-04-23T14:33:05.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.284+0000 7fc39aa00640 1 --2- 172.21.15.59:0/2238532342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0b4130 0x7fc39c06f680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc384009870 tx=0x7fc38402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f808c58dafa4959b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:05.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.284+0000 7fc39a000640 1 -- 172.21.15.59:0/2238532342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384037070 con 0x7fc39c0b4130 2024-04-23T14:33:05.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.284+0000 7fc39a000640 1 -- 172.21.15.59:0/2238532342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc384032440 con 0x7fc39c0b4130 2024-04-23T14:33:05.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.284+0000 7fc39a000640 1 -- 172.21.15.59:0/2238532342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384032810 con 0x7fc39c0b4130 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 -- 172.21.15.59:0/2238532342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0b4130 msgr2=0x7fc39c06f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2238532342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0b4130 0x7fc39c06f680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc384009870 tx=0x7fc38402a250 comp rx=0 tx=0).stop 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 -- 172.21.15.59:0/2238532342 shutdown_connections 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2238532342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c0cfb90 0x7fc39c0c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2238532342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0b4130 0x7fc39c06f680 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 -- 172.21.15.59:0/2238532342 >> 172.21.15.59:0/2238532342 conn(0x7fc39c0b21a0 msgr2=0x7fc39c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 -- 172.21.15.59:0/2238532342 shutdown_connections 2024-04-23T14:33:05.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.285+0000 7fc3a1200640 1 -- 172.21.15.59:0/2238532342 wait complete. 2024-04-23T14:33:05.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.287+0000 7fc3a1200640 1 Processor -- start 2024-04-23T14:33:05.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.287+0000 7fc3a1200640 1 -- start start 2024-04-23T14:33:05.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc3a1200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc39c0b4130 0x7fc39c0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc3a1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0cfb90 0x7fc39c071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc3a1200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c162df0 0x7fc39c072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc3a1200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c0c3860 con 0x7fc39c0cfb90 2024-04-23T14:33:05.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc3a1200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c0c36e0 con 0x7fc39c0b4130 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc3a1200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc39c0c39e0 con 0x7fc39c162df0 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc39aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0cfb90 0x7fc39c071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc39aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0cfb90 0x7fc39c071f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55036/0 (socket says 172.21.15.59:55036) 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.288+0000 7fc39aa00640 1 -- 172.21.15.59:0/2548703340 learned_addr learned my addr 172.21.15.59:0/2548703340 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39be00640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c162df0 0x7fc39c072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39aa00640 1 -- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c162df0 msgr2=0x7fc39c072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39b400640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc39c0b4130 0x7fc39c0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39aa00640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c162df0 0x7fc39c072fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39aa00640 1 -- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc39c0b4130 msgr2=0x7fc39c0c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39aa00640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc39c0b4130 0x7fc39c0c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39aa00640 1 -- 172.21.15.59:0/2548703340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c1633e0 con 0x7fc39c0cfb90 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39be00640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c162df0 0x7fc39c072fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39b400640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc39c0b4130 0x7fc39c0c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:05.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc39aa00640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0cfb90 0x7fc39c071f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc38402a290 tx=0x7fc3840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc393400640 1 -- 172.21.15.59:0/2548703340 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384041020 con 0x7fc39c0cfb90 2024-04-23T14:33:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc393400640 1 -- 172.21.15.59:0/2548703340 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3840306c0 con 0x7fc39c0cfb90 2024-04-23T14:33:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc393400640 1 -- 172.21.15.59:0/2548703340 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384037040 con 0x7fc39c0cfb90 2024-04-23T14:33:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.289+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c1636c0 con 0x7fc39c0cfb90 2024-04-23T14:33:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.290+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c072ab0 con 0x7fc39c0cfb90 2024-04-23T14:33:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.291+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39c04fd90 con 0x7fc39c0cfb90 2024-04-23T14:33:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.292+0000 7fc393400640 1 -- 172.21.15.59:0/2548703340 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc384044020 con 0x7fc39c0cfb90 2024-04-23T14:33:05.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.293+0000 7fc393400640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc37806c8f0 0x7fc37806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.293+0000 7fc393400640 1 -- 172.21.15.59:0/2548703340 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc3840ac370 con 0x7fc39c0cfb90 2024-04-23T14:33:05.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.293+0000 7fc39b400640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc37806c8f0 0x7fc37806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.294+0000 7fc39b400640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc37806c8f0 0x7fc37806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0015e0 tx=0x7fc38c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.298+0000 7fc393400640 1 -- 172.21.15.59:0/2548703340 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc38404b030 con 0x7fc39c0cfb90 2024-04-23T14:33:05.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.510+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc39c0cee90 con 0x7fc39c0cfb90 2024-04-23T14:33:05.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.512+0000 7fc393400640 1 -- 172.21.15.59:0/2548703340 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7fc38407a790 con 0x7fc39c0cfb90 2024-04-23T14:33:05.513 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:05.513 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:50.389436+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:05.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc37806c8f0 msgr2=0x7fc37806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc37806c8f0 0x7fc37806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0015e0 tx=0x7fc38c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0cfb90 msgr2=0x7fc39c071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0cfb90 0x7fc39c071f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc38402a290 tx=0x7fc3840046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 shutdown_connections 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc37806c8f0 0x7fc37806edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc39c162df0 0x7fc39c072fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc39c0cfb90 0x7fc39c071f80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 --2- 172.21.15.59:0/2548703340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc39c0b4130 0x7fc39c0c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.515+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 >> 172.21.15.59:0/2548703340 conn(0x7fc39c0b21a0 msgr2=0x7fc39c0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.516+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 shutdown_connections 2024-04-23T14:33:05.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.516+0000 7fc3a1200640 1 -- 172.21.15.59:0/2548703340 wait complete. 2024-04-23T14:33:05.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:46.759888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:47.810840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:05.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:05.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.745+0000 7f8ec9800640 1 Processor -- start 2024-04-23T14:33:05.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.746+0000 7f8ec9800640 1 -- start start 2024-04-23T14:33:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.747+0000 7f8ec9800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ec4145e10 0x7f8ec406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.747+0000 7f8ec9800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ec40d0cc0 0x7f8ec40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.747+0000 7f8ec9800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec40cfc20 con 0x7f8ec4071d70 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.747+0000 7f8ec9800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec40cff00 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.747+0000 7f8ec9800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec40d01e0 con 0x7f8ec4145e10 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.748+0000 7f8ec2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ec40d0cc0 0x7f8ec40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.748+0000 7f8ec2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ec40d0cc0 0x7f8ec40cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55044/0 (socket says 172.21.15.59:55044) 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.748+0000 7f8ec2000640 1 -- 172.21.15.59:0/2324057430 learned_addr learned my addr 172.21.15.59:0/2324057430 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.748+0000 7f8ec2000640 1 -- 172.21.15.59:0/2324057430 >> v2:172.21.15.76:3300/0 conn(0x7f8ec4145e10 msgr2=0x7f8ec406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.748+0000 7f8ec2000640 1 --2- 172.21.15.59:0/2324057430 >> v2:172.21.15.76:3300/0 conn(0x7f8ec4145e10 0x7f8ec406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.748+0000 7f8ec2000640 1 -- 172.21.15.59:0/2324057430 >> v1:172.21.15.76:6789/0 conn(0x7f8ec4071d70 legacy=0x7f8ec415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.748+0000 7f8ec2000640 1 -- 172.21.15.59:0/2324057430 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec40d04c0 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.749+0000 7f8ec2000640 1 --2- 172.21.15.59:0/2324057430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40cf340 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009870 tx=0x7f8eac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa879356abcdabe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:05.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.750+0000 7f8ec1600640 1 -- 172.21.15.59:0/2324057430 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac037070 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.750+0000 7f8ec1600640 1 -- 172.21.15.59:0/2324057430 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac032440 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.750+0000 7f8ec1600640 1 -- 172.21.15.59:0/2324057430 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac032810 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.750+0000 7f8ec9800640 1 -- 172.21.15.59:0/2324057430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 msgr2=0x7f8ec40cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.750+0000 7f8ec9800640 1 --2- 172.21.15.59:0/2324057430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40cf340 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009870 tx=0x7f8eac02a270 comp rx=0 tx=0).stop 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.751+0000 7f8ec9800640 1 -- 172.21.15.59:0/2324057430 shutdown_connections 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.751+0000 7f8ec9800640 1 --2- 172.21.15.59:0/2324057430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40cf340 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.751+0000 7f8ec9800640 1 --2- 172.21.15.59:0/2324057430 >> v2:172.21.15.76:3300/0 conn(0x7f8ec4145e10 0x7f8ec406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.751+0000 7f8ec9800640 1 -- 172.21.15.59:0/2324057430 >> 172.21.15.59:0/2324057430 conn(0x7f8ec4067d90 msgr2=0x7f8ec406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.751+0000 7f8ec9800640 1 -- 172.21.15.59:0/2324057430 shutdown_connections 2024-04-23T14:33:05.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.751+0000 7f8ec9800640 1 -- 172.21.15.59:0/2324057430 wait complete. 2024-04-23T14:33:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.752+0000 7f8ec9800640 1 Processor -- start 2024-04-23T14:33:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.753+0000 7f8ec9800640 1 -- start start 2024-04-23T14:33:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec9800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec40cfa00 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec9800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec40cf880 con 0x7f8ec4145e10 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec9800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec41461f0 con 0x7f8ec4071d70 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55048/0 (socket says 172.21.15.59:55048) 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec2000640 1 -- 172.21.15.59:0/4230594635 learned_addr learned my addr 172.21.15.59:0/4230594635 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec3400640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.754+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec2000640 1 -- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 msgr2=0x7f8ec415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec2000640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec2000640 1 -- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 msgr2=0x7f8ec41e6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec2000640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec2000640 1 -- 172.21.15.59:0/4230594635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec40af030 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec2000640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40b0cd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009870 tx=0x7f8eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:05.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ebaa00640 1 -- 172.21.15.59:0/4230594635 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac037040 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ebaa00640 1 -- 172.21.15.59:0/4230594635 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac004230 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ebaa00640 1 -- 172.21.15.59:0/4230594635 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac03c670 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec40af310 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.755+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec40af8d0 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.757+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec4068a80 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.757+0000 7f8ebaa00640 1 -- 172.21.15.59:0/4230594635 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eac03a070 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.759+0000 7f8ebaa00640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea006c8f0 0x7f8ea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:05.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.759+0000 7f8ebaa00640 1 -- 172.21.15.59:0/4230594635 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8eac0ac4b0 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.759+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea006c8f0 0x7f8ea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:05.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.760+0000 7f8ec2a00640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea006c8f0 0x7f8ea006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009f20 tx=0x7f8eb40094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:05.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.764+0000 7f8ebaa00640 1 -- 172.21.15.59:0/4230594635 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8eac041020 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.967+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ec40d1190 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.968+0000 7f8ebaa00640 1 -- 172.21.15.59:0/4230594635 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f8eac07a840 con 0x7f8ec40d0cc0 2024-04-23T14:33:05.968 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea006c8f0 msgr2=0x7f8ea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea006c8f0 0x7f8ea006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009f20 tx=0x7f8eb40094f0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 msgr2=0x7f8ec40b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40b0cd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009870 tx=0x7f8eac0046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 shutdown_connections 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ea006c8f0 0x7f8ea006edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ec4145e10 0x7f8ec41e6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ec40d0cc0 0x7f8ec40b0cd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 --2- 172.21.15.59:0/4230594635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ec4071d70 0x7f8ec415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:05.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 >> 172.21.15.59:0/4230594635 conn(0x7f8ec4067d90 msgr2=0x7f8ec415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:05.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 shutdown_connections 2024-04-23T14:33:05.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:05.971+0000 7f8ec9800640 1 -- 172.21.15.59:0/4230594635 wait complete. 2024-04-23T14:33:05.991 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:06.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.204+0000 7f080c800640 1 Processor -- start 2024-04-23T14:33:06.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.204+0000 7f080c800640 1 -- start start 2024-04-23T14:33:06.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.206+0000 7f080c800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0804145e10 0x7f080406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.206+0000 7f080c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 0x7f08040cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.206+0000 7f080c800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08040cfc20 con 0x7f0804071d70 2024-04-23T14:33:06.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.206+0000 7f080c800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f08040cff00 con 0x7f0804145e10 2024-04-23T14:33:06.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.206+0000 7f080c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f08040d01e0 con 0x7f08040d0cc0 2024-04-23T14:33:06.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.206+0000 7f080a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0804071d70 0x7f080415cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33230/0 (socket says 172.21.15.59:33230) 2024-04-23T14:33:06.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.206+0000 7f080a800640 1 -- 172.21.15.59:0/1023670351 learned_addr learned my addr 172.21.15.59:0/1023670351 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:06.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0809400640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 0x7f08040cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342951035 0 0) 0x7f08040cfc20 con 0x7f0804071d70 2024-04-23T14:33:06.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07ec003520 con 0x7f0804071d70 2024-04-23T14:33:06.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0809e00640 1 --2- 172.21.15.59:0/1023670351 >> v2:172.21.15.76:3300/0 conn(0x7f0804145e10 0x7f080406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 139972897 0 0) 0x7f07ec003520 con 0x7f0804071d70 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 msgr2=0x7f08040cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 0x7f08040cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 >> v2:172.21.15.76:3300/0 conn(0x7f0804145e10 msgr2=0x7f080406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 --2- 172.21.15.59:0/1023670351 >> v2:172.21.15.76:3300/0 conn(0x7f0804145e10 0x7f080406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08040d04c0 con 0x7f0804071d70 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f08000019f0 con 0x7f0804071d70 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 >> v1:172.21.15.59:6789/0 conn(0x7f0804071d70 legacy=0x7f080415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 0x7f07ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07ec004030 0x7f07ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07ec004d40 0x7f07ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08040cff00 con 0x7f07ec004030 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08040d01e0 con 0x7f07ec004d40 2024-04-23T14:33:06.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.207+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f07ec00bb20 con 0x7f08040d0cc0 2024-04-23T14:33:06.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f0809400640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07ec004030 0x7f07ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f0809e00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 0x7f07ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f080a800640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07ec004d40 0x7f07ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f0809e00640 1 -- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07ec004d40 msgr2=0x7f07ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f0809e00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07ec004d40 0x7f07ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f0809e00640 1 -- 172.21.15.59:0/1023670351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07ec004030 msgr2=0x7f07ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f0809e00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07ec004030 0x7f07ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.208+0000 7f0809e00640 1 -- 172.21.15.59:0/1023670351 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec003520 con 0x7f08040d0cc0 2024-04-23T14:33:06.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.210+0000 7f0809e00640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 0x7f07ec0039b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07f800ba30 tx=0x7f07ec005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96daa9e800293b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.210+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8030070 con 0x7f08040d0cc0 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.210+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f802ae10 con 0x7f08040d0cc0 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.210+0000 7f0808a00640 1 -- 172.21.15.59:0/1023670351 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f800b600 con 0x7f08040d0cc0 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.211+0000 7f080c800640 1 -- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 msgr2=0x7f07ec0039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.211+0000 7f080c800640 1 --2- 172.21.15.59:0/1023670351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040d0cc0 0x7f07ec0039b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07f800ba30 tx=0x7f07ec005830 comp rx=0 tx=0).stop 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.211+0000 7f080a800640 1 -- 172.21.15.59:0/1023670351 reap_dead start 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.211+0000 7f080c800640 1 -- 172.21.15.59:0/1023670351 shutdown_connections 2024-04-23T14:33:06.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.211+0000 7f080c800640 1 -- 172.21.15.59:0/1023670351 >> 172.21.15.59:0/1023670351 conn(0x7f0804067d90 msgr2=0x7f080406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.211+0000 7f080c800640 1 -- 172.21.15.59:0/1023670351 shutdown_connections 2024-04-23T14:33:06.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.211+0000 7f080c800640 1 -- 172.21.15.59:0/1023670351 wait complete. 2024-04-23T14:33:06.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.213+0000 7f080c800640 1 Processor -- start 2024-04-23T14:33:06.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.213+0000 7f080c800640 1 -- start start 2024-04-23T14:33:06.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f080c800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08040d0cc0 0x7f08041f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f080c800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08041e7f30 0x7f08041346b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f080c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040b4bd0 0x7f080415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f080c800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08041461f0 con 0x7f08041e7f30 2024-04-23T14:33:06.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f080c800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0804072200 con 0x7f08040d0cc0 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f080c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f08041e92e0 con 0x7f08040b4bd0 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f0809400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08041e7f30 0x7f08041346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f0809400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08041e7f30 0x7f08041346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55070/0 (socket says 172.21.15.59:55070) 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f0809400640 1 -- 172.21.15.59:0/4032216470 learned_addr learned my addr 172.21.15.59:0/4032216470 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.214+0000 7f080a800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040b4bd0 0x7f080415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f0809e00640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08040d0cc0 0x7f08041f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f0809400640 1 -- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040b4bd0 msgr2=0x7f080415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f0809400640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040b4bd0 0x7f080415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f0809400640 1 -- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08040d0cc0 msgr2=0x7f08041f8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f0809400640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08040d0cc0 0x7f08041f8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f0809400640 1 -- 172.21.15.59:0/4032216470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0804134bf0 con 0x7f08041e7f30 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f080a800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040b4bd0 0x7f080415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f0809400640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08041e7f30 0x7f08041346b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f07f400b770 tx=0x7f07f400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f07fe000640 1 -- 172.21.15.59:0/4032216470 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4018070 con 0x7f08041e7f30 2024-04-23T14:33:06.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f07fe000640 1 -- 172.21.15.59:0/4032216470 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f4010830 con 0x7f08041e7f30 2024-04-23T14:33:06.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f07fe000640 1 -- 172.21.15.59:0/4032216470 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4010c20 con 0x7f08041e7f30 2024-04-23T14:33:06.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.215+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08041e86d0 con 0x7f08041e7f30 2024-04-23T14:33:06.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.216+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08041e8c90 con 0x7f08041e7f30 2024-04-23T14:33:06.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.217+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0804068a80 con 0x7f08041e7f30 2024-04-23T14:33:06.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.218+0000 7f07fe000640 1 -- 172.21.15.59:0/4032216470 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07f401b070 con 0x7f08041e7f30 2024-04-23T14:33:06.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.219+0000 7f07fe000640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f07e806c8f0 0x7f07e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.219+0000 7f07fe000640 1 -- 172.21.15.59:0/4032216470 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f07f408bf00 con 0x7f08041e7f30 2024-04-23T14:33:06.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.219+0000 7f0809e00640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f07e806c8f0 0x7f07e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.220+0000 7f0809e00640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f07e806c8f0 0x7f07e806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07f800ba00 tx=0x7f07f8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.224+0000 7f07fe000640 1 -- 172.21.15.59:0/4032216470 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07f405a330 con 0x7f08041e7f30 2024-04-23T14:33:06.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.432+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f08040ce5a0 con 0x7f08041e7f30 2024-04-23T14:33:06.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.433+0000 7f07fe000640 1 -- 172.21.15.59:0/4032216470 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f07f4059c30 con 0x7f08041e7f30 2024-04-23T14:33:06.433 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f07e806c8f0 msgr2=0x7f07e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f07e806c8f0 0x7f07e806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07f800ba00 tx=0x7f07f8030000 comp rx=0 tx=0).stop 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08041e7f30 msgr2=0x7f08041346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08041e7f30 0x7f08041346b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f07f400b770 tx=0x7f07f400bc30 comp rx=0 tx=0).stop 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 shutdown_connections 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f07e806c8f0 0x7f07e806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08040b4bd0 0x7f080415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08041e7f30 0x7f08041346b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 --2- 172.21.15.59:0/4032216470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08040d0cc0 0x7f08041f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.436+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 >> 172.21.15.59:0/4032216470 conn(0x7f0804067d90 msgr2=0x7f08040a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.437+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 shutdown_connections 2024-04-23T14:33:06.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.437+0000 7f080c800640 1 -- 172.21.15.59:0/4032216470 wait complete. 2024-04-23T14:33:06.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:06.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.660+0000 7f6dc6000640 1 Processor -- start 2024-04-23T14:33:06.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.660+0000 7f6dc6000640 1 -- start start 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.661+0000 7f6dc6000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6dc015d300 0x7f6dc0071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.661+0000 7f6dc6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6dc0167180 0x7f6dc0163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.661+0000 7f6dc6000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc0163320 con 0x7f6dc00d0600 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.661+0000 7f6dc6000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc0163650 con 0x7f6dc0167180 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.661+0000 7f6dc6000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc0072380 con 0x7f6dc015d300 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dbfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6dc0167180 0x7f6dc0163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dbfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6dc0167180 0x7f6dc0163ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55074/0 (socket says 172.21.15.59:55074) 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dbfe00640 1 -- 172.21.15.59:0/3676685709 learned_addr learned my addr 172.21.15.59:0/3676685709 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dc4c00640 1 --2- 172.21.15.59:0/3676685709 >> v2:172.21.15.76:3300/0 conn(0x7f6dc015d300 0x7f6dc0071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dbfe00640 1 -- 172.21.15.59:0/3676685709 >> v2:172.21.15.76:3300/0 conn(0x7f6dc015d300 msgr2=0x7f6dc0071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dbfe00640 1 --2- 172.21.15.59:0/3676685709 >> v2:172.21.15.76:3300/0 conn(0x7f6dc015d300 0x7f6dc0071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dbfe00640 1 -- 172.21.15.59:0/3676685709 >> v1:172.21.15.76:6789/0 conn(0x7f6dc00d0600 legacy=0x7f6dc006e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.662+0000 7f6dbfe00640 1 -- 172.21.15.59:0/3676685709 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc0072660 con 0x7f6dc0167180 2024-04-23T14:33:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.663+0000 7f6dbfe00640 1 --2- 172.21.15.59:0/3676685709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc0167180 0x7f6dc0163ae0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6dc01637d0 tx=0x7f6dac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8065251cdf74031 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.664+0000 7f6dbf400640 1 -- 172.21.15.59:0/3676685709 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac037070 con 0x7f6dc0167180 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.664+0000 7f6dbf400640 1 -- 172.21.15.59:0/3676685709 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dac032440 con 0x7f6dc0167180 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.664+0000 7f6dbf400640 1 -- 172.21.15.59:0/3676685709 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac032810 con 0x7f6dc0167180 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 -- 172.21.15.59:0/3676685709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc0167180 msgr2=0x7f6dc0163ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 --2- 172.21.15.59:0/3676685709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc0167180 0x7f6dc0163ae0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6dc01637d0 tx=0x7f6dac02a210 comp rx=0 tx=0).stop 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 -- 172.21.15.59:0/3676685709 shutdown_connections 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 --2- 172.21.15.59:0/3676685709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc0167180 0x7f6dc0163ae0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 --2- 172.21.15.59:0/3676685709 >> v2:172.21.15.76:3300/0 conn(0x7f6dc015d300 0x7f6dc0071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 -- 172.21.15.59:0/3676685709 >> 172.21.15.59:0/3676685709 conn(0x7f6dc006bdf0 msgr2=0x7f6dc006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 -- 172.21.15.59:0/3676685709 shutdown_connections 2024-04-23T14:33:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.665+0000 7f6dc6000640 1 -- 172.21.15.59:0/3676685709 wait complete. 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.667+0000 7f6dc6000640 1 Processor -- start 2024-04-23T14:33:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.667+0000 7f6dc6000640 1 -- start start 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc00d0600 0x7f6dc0156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc6000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6dc015d300 0x7f6dc01ee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc6000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dc0167180 0x7f6dc01ee630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc0164020 con 0x7f6dc00d0600 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc6000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc01675e0 con 0x7f6dc0167180 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc6000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc006e8f0 con 0x7f6dc015d300 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc00d0600 0x7f6dc0156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc00d0600 0x7f6dc0156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55080/0 (socket says 172.21.15.59:55080) 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.668+0000 7f6dc4c00640 1 -- 172.21.15.59:0/1485103728 learned_addr learned my addr 172.21.15.59:0/1485103728 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dbfe00640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6dc015d300 0x7f6dc01ee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc4c00640 1 -- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6dc015d300 msgr2=0x7f6dc01ee0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc4c00640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6dc015d300 0x7f6dc01ee0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc4c00640 1 -- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dc0167180 msgr2=0x7f6dc01ee630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc5600640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dc0167180 0x7f6dc01ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc4c00640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dc0167180 0x7f6dc01ee630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc4c00640 1 -- 172.21.15.59:0/1485103728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc009cc40 con 0x7f6dc00d0600 2024-04-23T14:33:06.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc4c00640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc00d0600 0x7f6dc0156200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6db400d830 tx=0x7f6db400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dbcc00640 1 -- 172.21.15.59:0/1485103728 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400b6c0 con 0x7f6dc00d0600 2024-04-23T14:33:06.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dbcc00640 1 -- 172.21.15.59:0/1485103728 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db4019070 con 0x7f6dc00d0600 2024-04-23T14:33:06.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dbcc00640 1 -- 172.21.15.59:0/1485103728 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400ba80 con 0x7f6dc00d0600 2024-04-23T14:33:06.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.669+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc009cee0 con 0x7f6dc00d0600 2024-04-23T14:33:06.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.670+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc009d4a0 con 0x7f6dc00d0600 2024-04-23T14:33:06.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.671+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc004fd90 con 0x7f6dc00d0600 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.671+0000 7f6dbcc00640 1 -- 172.21.15.59:0/1485103728 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6db401e460 con 0x7f6dc00d0600 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.672+0000 7f6dbcc00640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6da006c8f0 0x7f6da006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.672+0000 7f6dbcc00640 1 -- 172.21.15.59:0/1485103728 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6db408c360 con 0x7f6dc00d0600 2024-04-23T14:33:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.673+0000 7f6dbfe00640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6da006c8f0 0x7f6da006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:06.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.674+0000 7f6dbfe00640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6da006c8f0 0x7f6da006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6dac034000 tx=0x7f6dac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:06.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.677+0000 7f6dbcc00640 1 -- 172.21.15.59:0/1485103728 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6db405a660 con 0x7f6dc00d0600 2024-04-23T14:33:06.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.884+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6dc00cff50 con 0x7f6dc00d0600 2024-04-23T14:33:06.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.885+0000 7f6dbcc00640 1 -- 172.21.15.59:0/1485103728 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f6db4059f60 con 0x7f6dc00d0600 2024-04-23T14:33:06.885 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:33:06.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6da006c8f0 msgr2=0x7f6da006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6da006c8f0 0x7f6da006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6dac034000 tx=0x7f6dac02e000 comp rx=0 tx=0).stop 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc00d0600 msgr2=0x7f6dc0156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc00d0600 0x7f6dc0156200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6db400d830 tx=0x7f6db400dcf0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 shutdown_connections 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6da006c8f0 0x7f6da006edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6dc015d300 0x7f6dc01ee0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dc0167180 0x7f6dc01ee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 --2- 172.21.15.59:0/1485103728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dc00d0600 0x7f6dc0156200 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 >> 172.21.15.59:0/1485103728 conn(0x7f6dc006bdf0 msgr2=0x7f6dc00af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 shutdown_connections 2024-04-23T14:33:06.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:06.888+0000 7f6dc6000640 1 -- 172.21.15.59:0/1485103728 wait complete. 2024-04-23T14:33:06.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.120+0000 7f2772c00640 1 Processor -- start 2024-04-23T14:33:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.120+0000 7f2772c00640 1 -- start start 2024-04-23T14:33:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.122+0000 7f2772c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f276c162de0 0x7f276c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.122+0000 7f2772c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276c1632e0 con 0x7f276c161e60 2024-04-23T14:33:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.122+0000 7f2772c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276c0adcd0 con 0x7f276c0b44c0 2024-04-23T14:33:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.122+0000 7f2772c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f276c06ea40 con 0x7f276c162de0 2024-04-23T14:33:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.122+0000 7f276be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f276c161e60 0x7f276c0b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33240/0 (socket says 172.21.15.59:33240) 2024-04-23T14:33:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.122+0000 7f276be00640 1 -- 172.21.15.59:0/3850610018 learned_addr learned my addr 172.21.15.59:0/3850610018 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3853305621 0 0) 0x7f276c1632e0 con 0x7f276c161e60 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f2770c00640 1 --2- 172.21.15.59:0/3850610018 >> v2:172.21.15.76:3300/0 conn(0x7f276c162de0 0x7f276c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2750003520 con 0x7f276c161e60 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1379633653 0 0) 0x7f2750003520 con 0x7f276c161e60 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 >> v2:172.21.15.76:3300/0 conn(0x7f276c162de0 msgr2=0x7f276c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 --2- 172.21.15.59:0/3850610018 >> v2:172.21.15.76:3300/0 conn(0x7f276c162de0 0x7f276c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 >> v1:172.21.15.76:6789/0 conn(0x7f276c0b44c0 legacy=0x7f276c06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c0adfb0 con 0x7f276c161e60 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f275c0019f0 con 0x7f276c161e60 2024-04-23T14:33:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 >> v1:172.21.15.59:6789/0 conn(0x7f276c161e60 legacy=0x7f276c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.123+0000 7f276aa00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2750003a00 0x7f2750003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276aa00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750004800 0x7f2750004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276aa00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27500052e0 0x7f275000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f276c06ea40 con 0x7f2750003a00 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f275000c200 con 0x7f27500052e0 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f275000c400 con 0x7f2750004800 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276be00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2750003a00 0x7f2750003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276be00640 1 -- 172.21.15.59:0/3850610018 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750004800 msgr2=0x7f2750004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276be00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750004800 0x7f2750004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276be00640 1 -- 172.21.15.59:0/3850610018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27500052e0 msgr2=0x7f275000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276be00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27500052e0 0x7f275000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f276be00640 1 -- 172.21.15.59:0/3850610018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2750003520 con 0x7f2750003a00 2024-04-23T14:33:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.124+0000 7f2770c00640 1 -- 172.21.15.59:0/3850610018 reap_dead start 2024-04-23T14:33:07.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.125+0000 7f276be00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2750003a00 0x7f2750003e20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f275c00a7b0 tx=0x7f2750005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b82b722d78036fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c00a3d0 con 0x7f2750003a00 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c02f070 con 0x7f2750003a00 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f276aa00640 1 -- 172.21.15.59:0/3850610018 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c02bcc0 con 0x7f2750003a00 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f2772c00640 1 -- 172.21.15.59:0/3850610018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2750003a00 msgr2=0x7f2750003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f2772c00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2750003a00 0x7f2750003e20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f275c00a7b0 tx=0x7f2750005830 comp rx=0 tx=0).stop 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f2772c00640 1 -- 172.21.15.59:0/3850610018 shutdown_connections 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f2772c00640 1 --2- 172.21.15.59:0/3850610018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2750003a00 0x7f2750003e20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f2772c00640 1 -- 172.21.15.59:0/3850610018 >> 172.21.15.59:0/3850610018 conn(0x7f276c0b2130 msgr2=0x7f276c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.126+0000 7f2772c00640 1 -- 172.21.15.59:0/3850610018 shutdown_connections 2024-04-23T14:33:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.127+0000 7f2772c00640 1 -- 172.21.15.59:0/3850610018 wait complete. 2024-04-23T14:33:07.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.128+0000 7f2772c00640 1 Processor -- start 2024-04-23T14:33:07.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.128+0000 7f2772c00640 1 -- start start 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.129+0000 7f2772c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750003a00 0x7f276c148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.129+0000 7f2772c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c156fe0 0x7f276c157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.129+0000 7f2772c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c1f79f0 0x7f276c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.129+0000 7f2772c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f276c0af480 con 0x7f276c156fe0 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.129+0000 7f2772c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f276c0ae570 con 0x7f276c1f79f0 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.129+0000 7f2772c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f276c0af600 con 0x7f2750003a00 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c156fe0 0x7f276c157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c156fe0 0x7f276c157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55102/0 (socket says 172.21.15.59:55102) 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 -- 172.21.15.59:0/2643657230 learned_addr learned my addr 172.21.15.59:0/2643657230 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276be00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750003a00 0x7f276c148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f2770c00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c1f79f0 0x7f276c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 -- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750003a00 msgr2=0x7f276c148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750003a00 0x7f276c148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 -- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c1f79f0 msgr2=0x7f276c1f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c1f79f0 0x7f276c1f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.130+0000 7f276b400640 1 -- 172.21.15.59:0/2643657230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c149310 con 0x7f276c156fe0 2024-04-23T14:33:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.131+0000 7f276b400640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c156fe0 0x7f276c157460 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f275400c460 tx=0x7f275400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.131+0000 7f2763e00640 1 -- 172.21.15.59:0/2643657230 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540052d0 con 0x7f276c156fe0 2024-04-23T14:33:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.131+0000 7f2763e00640 1 -- 172.21.15.59:0/2643657230 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2754005490 con 0x7f276c156fe0 2024-04-23T14:33:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.131+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c1f85d0 con 0x7f276c156fe0 2024-04-23T14:33:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.131+0000 7f2763e00640 1 -- 172.21.15.59:0/2643657230 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754012870 con 0x7f276c156fe0 2024-04-23T14:33:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.131+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c15bd50 con 0x7f276c156fe0 2024-04-23T14:33:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.132+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c068a80 con 0x7f276c156fe0 2024-04-23T14:33:07.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.133+0000 7f2763e00640 1 -- 172.21.15.59:0/2643657230 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2754012a10 con 0x7f276c156fe0 2024-04-23T14:33:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.134+0000 7f2763e00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274806c8f0 0x7f274806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.134+0000 7f2763e00640 1 -- 172.21.15.59:0/2643657230 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f275408b720 con 0x7f276c156fe0 2024-04-23T14:33:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.135+0000 7f276be00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274806c8f0 0x7f274806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.135+0000 7f276be00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274806c8f0 0x7f274806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f275c00a780 tx=0x7f275c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.139+0000 7f2763e00640 1 -- 172.21.15.59:0/2643657230 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f276c149310 con 0x7f276c156fe0 2024-04-23T14:33:07.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.360+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f276c0b1f10 con 0x7f276c156fe0 2024-04-23T14:33:07.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.361+0000 7f2763e00640 1 -- 172.21.15.59:0/2643657230 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f2754059b40 con 0x7f276c156fe0 2024-04-23T14:33:07.361 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.363+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274806c8f0 msgr2=0x7f274806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.363+0000 7f2772c00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274806c8f0 0x7f274806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f275c00a780 tx=0x7f275c009030 comp rx=0 tx=0).stop 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.363+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c156fe0 msgr2=0x7f276c157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.363+0000 7f2772c00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c156fe0 0x7f276c157460 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f275400c460 tx=0x7f275400c920 comp rx=0 tx=0).stop 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 shutdown_connections 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f274806c8f0 0x7f274806edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f276c1f79f0 0x7f276c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c156fe0 0x7f276c157460 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 --2- 172.21.15.59:0/2643657230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2750003a00 0x7f276c148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 >> 172.21.15.59:0/2643657230 conn(0x7f276c0b2130 msgr2=0x7f276c1580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 shutdown_connections 2024-04-23T14:33:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.364+0000 7f2772c00640 1 -- 172.21.15.59:0/2643657230 wait complete. 2024-04-23T14:33:07.385 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:33:07.385 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:07.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:07.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.655+0000 7f054cc00640 1 Processor -- start 2024-04-23T14:33:07.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.656+0000 7f054cc00640 1 -- start start 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.657+0000 7f054cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f054815fe90 0x7f05480b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.657+0000 7f054cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05480cd220 con 0x7f0548160cb0 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.657+0000 7f054cc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f054809b410 con 0x7f05480b34c0 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.657+0000 7f054cc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f05480d14e0 con 0x7f054815fe90 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.658+0000 7f0545e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f054815fe90 0x7f05480b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.658+0000 7f0546800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0548160cb0 0x7f05480cd9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33248/0 (socket says 172.21.15.59:33248) 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.658+0000 7f0545e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f054815fe90 0x7f05480b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55116/0 (socket says 172.21.15.59:55116) 2024-04-23T14:33:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.658+0000 7f0546800640 1 -- 172.21.15.59:0/3010277614 learned_addr learned my addr 172.21.15.59:0/3010277614 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:07.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.658+0000 7f0545e00640 1 -- 172.21.15.59:0/3010277614 >> v1:172.21.15.76:6789/0 conn(0x7f05480b34c0 legacy=0x7f05481605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:07.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.658+0000 7f0545e00640 1 -- 172.21.15.59:0/3010277614 >> v1:172.21.15.59:6789/0 conn(0x7f0548160cb0 legacy=0x7f05480cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.658+0000 7f0545e00640 1 -- 172.21.15.59:0/3010277614 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054809b6f0 con 0x7f054815fe90 2024-04-23T14:33:07.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.659+0000 7f0545e00640 1 --2- 172.21.15.59:0/3010277614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f05480b53e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0538009d90 tx=0x7f053802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4228eb46903a6057 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f0544a00640 1 -- 172.21.15.59:0/3010277614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538007740 con 0x7f054815fe90 2024-04-23T14:33:07.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f0544a00640 1 -- 172.21.15.59:0/3010277614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05380078e0 con 0x7f054815fe90 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f0544a00640 1 -- 172.21.15.59:0/3010277614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538007cb0 con 0x7f054815fe90 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f054cc00640 1 -- 172.21.15.59:0/3010277614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 msgr2=0x7f05480b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f054cc00640 1 --2- 172.21.15.59:0/3010277614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f05480b53e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0538009d90 tx=0x7f053802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f054cc00640 1 -- 172.21.15.59:0/3010277614 shutdown_connections 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f054cc00640 1 --2- 172.21.15.59:0/3010277614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f05480b53e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f054cc00640 1 -- 172.21.15.59:0/3010277614 >> 172.21.15.59:0/3010277614 conn(0x7f05480b4080 msgr2=0x7f05480b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f054cc00640 1 -- 172.21.15.59:0/3010277614 shutdown_connections 2024-04-23T14:33:07.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.660+0000 7f054cc00640 1 -- 172.21.15.59:0/3010277614 wait complete. 2024-04-23T14:33:07.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.662+0000 7f054cc00640 1 Processor -- start 2024-04-23T14:33:07.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.662+0000 7f054cc00640 1 -- start start 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.663+0000 7f054cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05480b34c0 0x7f054815d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.663+0000 7f054cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f0548154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.663+0000 7f054cc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0548160cb0 0x7f054806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.663+0000 7f054cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f054809bcb0 con 0x7f054815fe90 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.663+0000 7f054cc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f05480cd490 con 0x7f0548160cb0 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.663+0000 7f054cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f05480b5920 con 0x7f05480b34c0 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.663+0000 7f0545400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f0548154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f0548154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55128/0 (socket says 172.21.15.59:55128) 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 -- 172.21.15.59:0/1861680736 learned_addr learned my addr 172.21.15.59:0/1861680736 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545e00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05480b34c0 0x7f054815d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 -- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05480b34c0 msgr2=0x7f054815d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0546800640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0548160cb0 0x7f054806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05480b34c0 0x7f054815d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 -- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0548160cb0 msgr2=0x7f054806cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0548160cb0 0x7f054806cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 -- 172.21.15.59:0/1861680736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054806e770 con 0x7f054815fe90 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545e00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05480b34c0 0x7f054815d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.664+0000 7f0545400640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f0548154c50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f053000b420 tx=0x7f053000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.665+0000 7f053e000640 1 -- 172.21.15.59:0/1861680736 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530011830 con 0x7f054815fe90 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.665+0000 7f053e000640 1 -- 172.21.15.59:0/1861680736 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0530016070 con 0x7f054815fe90 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.665+0000 7f053e000640 1 -- 172.21.15.59:0/1861680736 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530011c10 con 0x7f054815fe90 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.665+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054806d6d0 con 0x7f054815fe90 2024-04-23T14:33:07.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.665+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054806dc90 con 0x7f054815fe90 2024-04-23T14:33:07.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.666+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0548068a80 con 0x7f054815fe90 2024-04-23T14:33:07.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.667+0000 7f053e000640 1 -- 172.21.15.59:0/1861680736 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f053001b460 con 0x7f054815fe90 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.668+0000 7f053e000640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f052406c8f0 0x7f052406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.668+0000 7f053e000640 1 -- 172.21.15.59:0/1861680736 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f053008b650 con 0x7f054815fe90 2024-04-23T14:33:07.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.668+0000 7f0545e00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f052406c8f0 0x7f052406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.669+0000 7f0545e00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f052406c8f0 0x7f052406edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0538034000 tx=0x7f053802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.672+0000 7f053e000640 1 -- 172.21.15.59:0/1861680736 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0530020020 con 0x7f054815fe90 2024-04-23T14:33:07.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.879+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05480b4d10 con 0x7f054815fe90 2024-04-23T14:33:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.881+0000 7f053e000640 1 -- 172.21.15.59:0/1861680736 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f0530059a70 con 0x7f054815fe90 2024-04-23T14:33:07.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:07.881 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:50.389436+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f052406c8f0 msgr2=0x7f052406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f052406c8f0 0x7f052406edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0538034000 tx=0x7f053802f000 comp rx=0 tx=0).stop 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 msgr2=0x7f0548154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f0548154c50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f053000b420 tx=0x7f053000b8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 shutdown_connections 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f052406c8f0 0x7f052406edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0548160cb0 0x7f054806cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f054815fe90 0x7f0548154c50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 --2- 172.21.15.59:0/1861680736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05480b34c0 0x7f054815d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 >> 172.21.15.59:0/1861680736 conn(0x7f05480b4080 msgr2=0x7f054809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 shutdown_connections 2024-04-23T14:33:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:07.884+0000 7f054cc00640 1 -- 172.21.15.59:0/1861680736 wait complete. 2024-04-23T14:33:07.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:46.759888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:47.810840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:07.905 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:08.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.122+0000 7f29f5e00640 1 Processor -- start 2024-04-23T14:33:08.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.123+0000 7f29f5e00640 1 -- start start 2024-04-23T14:33:08.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.124+0000 7f29f5e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f29f00d15b0 0x7f29f00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.124+0000 7f29f5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29f00b4f20 0x7f29f00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.124+0000 7f29f5e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f00b5da0 con 0x7f29f015ed20 2024-04-23T14:33:08.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.124+0000 7f29f5e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f29f015f100 con 0x7f29f00b4f20 2024-04-23T14:33:08.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.124+0000 7f29f5e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f29f00d0a20 con 0x7f29f00d15b0 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29f4a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f29f015ed20 0x7f29f00d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33252/0 (socket says 172.21.15.59:33252) 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29f4a00640 1 -- 172.21.15.59:0/848431245 learned_addr learned my addr 172.21.15.59:0/848431245 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29f5400640 1 --2- 172.21.15.59:0/848431245 >> v2:172.21.15.59:3300/0 conn(0x7f29f00b4f20 0x7f29f00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29efe00640 1 --2- 172.21.15.59:0/848431245 >> v2:172.21.15.76:3300/0 conn(0x7f29f00d15b0 0x7f29f00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29ef400640 1 -- 172.21.15.59:0/848431245 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365639713 0 0) 0x7f29f00b5da0 con 0x7f29f015ed20 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29ef400640 1 -- 172.21.15.59:0/848431245 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29d4003520 con 0x7f29f015ed20 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29f5400640 1 -- 172.21.15.59:0/848431245 >> v2:172.21.15.76:3300/0 conn(0x7f29f00d15b0 msgr2=0x7f29f00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29f5400640 1 --2- 172.21.15.59:0/848431245 >> v2:172.21.15.76:3300/0 conn(0x7f29f00d15b0 0x7f29f00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29f5400640 1 -- 172.21.15.59:0/848431245 >> v1:172.21.15.59:6789/0 conn(0x7f29f015ed20 legacy=0x7f29f00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29f5400640 1 -- 172.21.15.59:0/848431245 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00d2ed0 con 0x7f29f00b4f20 2024-04-23T14:33:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.125+0000 7f29efe00640 1 --2- 172.21.15.59:0/848431245 >> v2:172.21.15.76:3300/0 conn(0x7f29f00d15b0 0x7f29f00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.126+0000 7f29f5400640 1 --2- 172.21.15.59:0/848431245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f00b4f20 0x7f29f00cf7c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29f00d4520 tx=0x7f29e802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa551d70cac7332 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29ef400640 1 -- 172.21.15.59:0/848431245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8037070 con 0x7f29f00b4f20 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29ef400640 1 -- 172.21.15.59:0/848431245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e802add0 con 0x7f29f00b4f20 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29ef400640 1 -- 172.21.15.59:0/848431245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8032660 con 0x7f29f00b4f20 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29f5e00640 1 -- 172.21.15.59:0/848431245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f00b4f20 msgr2=0x7f29f00cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29f5e00640 1 --2- 172.21.15.59:0/848431245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f00b4f20 0x7f29f00cf7c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29f00d4520 tx=0x7f29e802a080 comp rx=0 tx=0).stop 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29f5e00640 1 -- 172.21.15.59:0/848431245 shutdown_connections 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29f5e00640 1 --2- 172.21.15.59:0/848431245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f00b4f20 0x7f29f00cf7c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29f5e00640 1 --2- 172.21.15.59:0/848431245 >> v2:172.21.15.76:3300/0 conn(0x7f29f00d15b0 0x7f29f00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.127+0000 7f29f5e00640 1 -- 172.21.15.59:0/848431245 >> 172.21.15.59:0/848431245 conn(0x7f29f008bf90 msgr2=0x7f29f00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.128+0000 7f29f5e00640 1 -- 172.21.15.59:0/848431245 shutdown_connections 2024-04-23T14:33:08.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.128+0000 7f29f5e00640 1 -- 172.21.15.59:0/848431245 wait complete. 2024-04-23T14:33:08.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.129+0000 7f29f5e00640 1 Processor -- start 2024-04-23T14:33:08.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.130+0000 7f29f5e00640 1 -- start start 2024-04-23T14:33:08.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.130+0000 7f29f5e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29f00b4f20 0x7f29f01507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.130+0000 7f29f5e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f29f00d15b0 0x7f29f00cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f015ed20 0x7f29f00b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00b3600 con 0x7f29f015ed20 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29f00d3490 con 0x7f29f00b4f20 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f29f00b3780 con 0x7f29f00d15b0 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f015ed20 0x7f29f00b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f015ed20 0x7f29f00b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55140/0 (socket says 172.21.15.59:55140) 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 -- 172.21.15.59:0/3136467654 learned_addr learned my addr 172.21.15.59:0/3136467654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f4a00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29f00b4f20 0x7f29f01507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29efe00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f29f00d15b0 0x7f29f00cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 -- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f29f00d15b0 msgr2=0x7f29f00cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f29f00d15b0 0x7f29f00cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 -- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29f00b4f20 msgr2=0x7f29f01507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29f00b4f20 0x7f29f01507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f5400640 1 -- 172.21.15.59:0/3136467654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f00ccb60 con 0x7f29f015ed20 2024-04-23T14:33:08.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29efe00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f29f00d15b0 0x7f29f00cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:08.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.131+0000 7f29f4a00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29f00b4f20 0x7f29f01507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:08.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.132+0000 7f29f5400640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f015ed20 0x7f29f00b7790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f29e8035040 tx=0x7f29e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.132+0000 7f29ecc00640 1 -- 172.21.15.59:0/3136467654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8037040 con 0x7f29f015ed20 2024-04-23T14:33:08.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.132+0000 7f29ecc00640 1 -- 172.21.15.59:0/3136467654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e8042070 con 0x7f29f015ed20 2024-04-23T14:33:08.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.132+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f00cce40 con 0x7f29f015ed20 2024-04-23T14:33:08.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.132+0000 7f29ecc00640 1 -- 172.21.15.59:0/3136467654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e803d680 con 0x7f29f015ed20 2024-04-23T14:33:08.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.132+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f00cd400 con 0x7f29f015ed20 2024-04-23T14:33:08.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.133+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f004fd90 con 0x7f29f015ed20 2024-04-23T14:33:08.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.134+0000 7f29ecc00640 1 -- 172.21.15.59:0/3136467654 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29e803d820 con 0x7f29f015ed20 2024-04-23T14:33:08.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.135+0000 7f29ecc00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f29d006c8f0 0x7f29d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.136+0000 7f29ecc00640 1 -- 172.21.15.59:0/3136467654 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f29e80ac280 con 0x7f29f015ed20 2024-04-23T14:33:08.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.136+0000 7f29f4a00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f29d006c8f0 0x7f29d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.137+0000 7f29f4a00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f29d006c8f0 0x7f29d006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005c00 tx=0x7f29e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.140+0000 7f29ecc00640 1 -- 172.21.15.59:0/3136467654 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29e807a580 con 0x7f29f015ed20 2024-04-23T14:33:08.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.342+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29f00cfac0 con 0x7f29f015ed20 2024-04-23T14:33:08.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.343+0000 7f29ecc00640 1 -- 172.21.15.59:0/3136467654 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f29e8079e80 con 0x7f29f015ed20 2024-04-23T14:33:08.343 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.345+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f29d006c8f0 msgr2=0x7f29d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.345+0000 7f29f5e00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f29d006c8f0 0x7f29d006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005c00 tx=0x7f29e0009040 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.345+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f015ed20 msgr2=0x7f29f00b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.345+0000 7f29f5e00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f015ed20 0x7f29f00b7790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f29e8035040 tx=0x7f29e80046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.345+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 shutdown_connections 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.346+0000 7f29f5e00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f29d006c8f0 0x7f29d006edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.346+0000 7f29f5e00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f29f00d15b0 0x7f29f00cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.346+0000 7f29f5e00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f015ed20 0x7f29f00b7790 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.346+0000 7f29f5e00640 1 --2- 172.21.15.59:0/3136467654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29f00b4f20 0x7f29f01507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.346+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 >> 172.21.15.59:0/3136467654 conn(0x7f29f008bf90 msgr2=0x7f29f00d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.346+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 shutdown_connections 2024-04-23T14:33:08.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.346+0000 7f29f5e00640 1 -- 172.21.15.59:0/3136467654 wait complete. 2024-04-23T14:33:08.366 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:08.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.575+0000 7fb94d400640 1 Processor -- start 2024-04-23T14:33:08.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.575+0000 7fb94d400640 1 -- start start 2024-04-23T14:33:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.577+0000 7fb94d400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb948145e10 0x7fb94806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.577+0000 7fb94d400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb948072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.577+0000 7fb94d400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9480cfc20 con 0x7fb9480d0cc0 2024-04-23T14:33:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.577+0000 7fb94d400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb9480cff00 con 0x7fb948145e10 2024-04-23T14:33:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.577+0000 7fb94d400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb9480d01e0 con 0x7fb948071d70 2024-04-23T14:33:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.578+0000 7fb947200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb948072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.578+0000 7fb947200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb948072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:38408/0 (socket says 172.21.15.59:38408) 2024-04-23T14:33:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.578+0000 7fb947200640 1 -- 172.21.15.59:0/2139247628 learned_addr learned my addr 172.21.15.59:0/2139247628 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.578+0000 7fb946800640 1 --2- 172.21.15.59:0/2139247628 >> v2:172.21.15.76:3300/0 conn(0x7fb948145e10 0x7fb94806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.579+0000 7fb947200640 1 -- 172.21.15.59:0/2139247628 >> v2:172.21.15.76:3300/0 conn(0x7fb948145e10 msgr2=0x7fb94806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.579+0000 7fb947200640 1 --2- 172.21.15.59:0/2139247628 >> v2:172.21.15.76:3300/0 conn(0x7fb948145e10 0x7fb94806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.579+0000 7fb947200640 1 -- 172.21.15.59:0/2139247628 >> v1:172.21.15.76:6789/0 conn(0x7fb9480d0cc0 legacy=0x7fb9480cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.579+0000 7fb947200640 1 -- 172.21.15.59:0/2139247628 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9480d04c0 con 0x7fb948071d70 2024-04-23T14:33:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.580+0000 7fb947200640 1 --2- 172.21.15.59:0/2139247628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb948072200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb940009e50 tx=0x7fb94002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146cc90d12541b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.580+0000 7fb945e00640 1 -- 172.21.15.59:0/2139247628 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940036070 con 0x7fb948071d70 2024-04-23T14:33:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.580+0000 7fb945e00640 1 -- 172.21.15.59:0/2139247628 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940031600 con 0x7fb948071d70 2024-04-23T14:33:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.580+0000 7fb945e00640 1 -- 172.21.15.59:0/2139247628 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9400319d0 con 0x7fb948071d70 2024-04-23T14:33:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 -- 172.21.15.59:0/2139247628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 msgr2=0x7fb948072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 --2- 172.21.15.59:0/2139247628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb948072200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb940009e50 tx=0x7fb94002a710 comp rx=0 tx=0).stop 2024-04-23T14:33:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 -- 172.21.15.59:0/2139247628 shutdown_connections 2024-04-23T14:33:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 --2- 172.21.15.59:0/2139247628 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb948072200 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 --2- 172.21.15.59:0/2139247628 >> v2:172.21.15.76:3300/0 conn(0x7fb948145e10 0x7fb94806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 -- 172.21.15.59:0/2139247628 >> 172.21.15.59:0/2139247628 conn(0x7fb948067d90 msgr2=0x7fb94806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 -- 172.21.15.59:0/2139247628 shutdown_connections 2024-04-23T14:33:08.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.581+0000 7fb94d400640 1 -- 172.21.15.59:0/2139247628 wait complete. 2024-04-23T14:33:08.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.583+0000 7fb94d400640 1 Processor -- start 2024-04-23T14:33:08.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.584+0000 7fb94d400640 1 -- start start 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb94d400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb94815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb94d400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9480d0cc0 0x7fb9481e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb94d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb948145e10 0x7fb9481e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb94d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9481461f0 con 0x7fb948145e10 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb94d400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb94815cdd0 con 0x7fb9480d0cc0 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb94d400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb9481e69b0 con 0x7fb948071d70 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb947200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb948145e10 0x7fb9481e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb947200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb948145e10 0x7fb9481e9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55142/0 (socket says 172.21.15.59:55142) 2024-04-23T14:33:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb947200640 1 -- 172.21.15.59:0/4076443747 learned_addr learned my addr 172.21.15.59:0/4076443747 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb93fe00640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9480d0cc0 0x7fb9481e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.585+0000 7fb946800640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb94815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb947200640 1 -- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 msgr2=0x7fb94815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb947200640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb94815c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb947200640 1 -- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9480d0cc0 msgr2=0x7fb9481e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb947200640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9480d0cc0 0x7fb9481e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb947200640 1 -- 172.21.15.59:0/4076443747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9481e66d0 con 0x7fb948145e10 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb93fe00640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9480d0cc0 0x7fb9481e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb947200640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb948145e10 0x7fb9481e9dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb940034040 tx=0x7fb940004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb93ea00640 1 -- 172.21.15.59:0/4076443747 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940036050 con 0x7fb948145e10 2024-04-23T14:33:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb93ea00640 1 -- 172.21.15.59:0/4076443747 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940042070 con 0x7fb948145e10 2024-04-23T14:33:08.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.586+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9481e73c0 con 0x7fb948145e10 2024-04-23T14:33:08.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.587+0000 7fb93ea00640 1 -- 172.21.15.59:0/4076443747 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940009510 con 0x7fb948145e10 2024-04-23T14:33:08.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.587+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481e7980 con 0x7fb948145e10 2024-04-23T14:33:08.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.588+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94806f130 con 0x7fb948145e10 2024-04-23T14:33:08.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.589+0000 7fb93ea00640 1 -- 172.21.15.59:0/4076443747 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb940007bf0 con 0x7fb948145e10 2024-04-23T14:33:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.590+0000 7fb93ea00640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb92406c8f0 0x7fb92406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.590+0000 7fb93ea00640 1 -- 172.21.15.59:0/4076443747 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb9400aca20 con 0x7fb948145e10 2024-04-23T14:33:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.590+0000 7fb946800640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb92406c8f0 0x7fb92406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.591+0000 7fb946800640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb92406c8f0 0x7fb92406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb948144a10 tx=0x7fb938008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.595+0000 7fb93ea00640 1 -- 172.21.15.59:0/4076443747 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb94806f130 con 0x7fb948145e10 2024-04-23T14:33:08.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.800+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb948068a80 con 0x7fb948145e10 2024-04-23T14:33:08.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.801+0000 7fb93ea00640 1 -- 172.21.15.59:0/4076443747 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fb940045360 con 0x7fb948145e10 2024-04-23T14:33:08.801 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:33:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb92406c8f0 msgr2=0x7fb92406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb92406c8f0 0x7fb92406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb948144a10 tx=0x7fb938008040 comp rx=0 tx=0).stop 2024-04-23T14:33:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb948145e10 msgr2=0x7fb9481e9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb948145e10 0x7fb9481e9dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb940034040 tx=0x7fb940004a40 comp rx=0 tx=0).stop 2024-04-23T14:33:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 shutdown_connections 2024-04-23T14:33:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb92406c8f0 0x7fb92406edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb948145e10 0x7fb9481e9dc0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9480d0cc0 0x7fb9481e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 --2- 172.21.15.59:0/4076443747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb948071d70 0x7fb94815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 >> 172.21.15.59:0/4076443747 conn(0x7fb948067d90 msgr2=0x7fb94815de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 shutdown_connections 2024-04-23T14:33:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:08.804+0000 7fb94d400640 1 -- 172.21.15.59:0/4076443747 wait complete. 2024-04-23T14:33:08.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:09.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.048+0000 7f2b68000640 1 Processor -- start 2024-04-23T14:33:09.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.048+0000 7f2b68000640 1 -- start start 2024-04-23T14:33:09.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b68000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b6009ac70 0x7f2b600704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b68000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b60160400 0x7f2b600ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b68000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b60070ae0 con 0x7f2b6009b050 2024-04-23T14:33:09.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b68000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2b60070dc0 con 0x7f2b60160400 2024-04-23T14:33:09.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b68000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b600710a0 con 0x7f2b6009ac70 2024-04-23T14:33:09.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b64c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b60160400 0x7f2b600ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b64c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b60160400 0x7f2b600ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55150/0 (socket says 172.21.15.59:55150) 2024-04-23T14:33:09.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.050+0000 7f2b64c00640 1 -- 172.21.15.59:0/1980705621 learned_addr learned my addr 172.21.15.59:0/1980705621 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:09.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.051+0000 7f2b65600640 1 --2- 172.21.15.59:0/1980705621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b6009ac70 0x7f2b600704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.051+0000 7f2b64c00640 1 -- 172.21.15.59:0/1980705621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b6009ac70 msgr2=0x7f2b600704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.051+0000 7f2b64c00640 1 --2- 172.21.15.59:0/1980705621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b6009ac70 0x7f2b600704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.051+0000 7f2b64c00640 1 -- 172.21.15.59:0/1980705621 >> v1:172.21.15.76:6789/0 conn(0x7f2b6009b050 legacy=0x7f2b600cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.051+0000 7f2b64c00640 1 -- 172.21.15.59:0/1980705621 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b60071380 con 0x7f2b60160400 2024-04-23T14:33:09.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.052+0000 7f2b64c00640 1 --2- 172.21.15.59:0/1980705621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b60160400 0x7f2b600ccd30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009870 tx=0x7f2b4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b7235274073a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.052+0000 7f2b5fe00640 1 -- 172.21.15.59:0/1980705621 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c037070 con 0x7f2b60160400 2024-04-23T14:33:09.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.052+0000 7f2b5fe00640 1 -- 172.21.15.59:0/1980705621 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4c032440 con 0x7f2b60160400 2024-04-23T14:33:09.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.052+0000 7f2b5fe00640 1 -- 172.21.15.59:0/1980705621 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c032810 con 0x7f2b60160400 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 -- 172.21.15.59:0/1980705621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b60160400 msgr2=0x7f2b600ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 --2- 172.21.15.59:0/1980705621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b60160400 0x7f2b600ccd30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009870 tx=0x7f2b4c02a180 comp rx=0 tx=0).stop 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 -- 172.21.15.59:0/1980705621 shutdown_connections 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 --2- 172.21.15.59:0/1980705621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b60160400 0x7f2b600ccd30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 --2- 172.21.15.59:0/1980705621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b6009ac70 0x7f2b600704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 -- 172.21.15.59:0/1980705621 >> 172.21.15.59:0/1980705621 conn(0x7f2b600b40d0 msgr2=0x7f2b600b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 -- 172.21.15.59:0/1980705621 shutdown_connections 2024-04-23T14:33:09.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.053+0000 7f2b68000640 1 -- 172.21.15.59:0/1980705621 wait complete. 2024-04-23T14:33:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.055+0000 7f2b68000640 1 Processor -- start 2024-04-23T14:33:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.055+0000 7f2b68000640 1 -- start start 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.056+0000 7f2b68000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6009ac70 0x7f2b600b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.056+0000 7f2b68000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b6009b050 0x7f2b600d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.056+0000 7f2b68000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b60160400 0x7f2b6014b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.056+0000 7f2b68000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6015d8c0 con 0x7f2b6009b050 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.056+0000 7f2b68000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6009bfb0 con 0x7f2b6009ac70 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.056+0000 7f2b68000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6015da40 con 0x7f2b60160400 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.056+0000 7f2b64c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b6009b050 0x7f2b600d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b6009b050 0x7f2b600d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55160/0 (socket says 172.21.15.59:55160) 2024-04-23T14:33:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 -- 172.21.15.59:0/1422498338 learned_addr learned my addr 172.21.15.59:0/1422498338 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b65600640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6009ac70 0x7f2b600b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 -- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b60160400 msgr2=0x7f2b6014b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b66000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b60160400 0x7f2b6014b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b60160400 0x7f2b6014b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 -- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6009ac70 msgr2=0x7f2b600b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6009ac70 0x7f2b600b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 -- 172.21.15.59:0/1422498338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b600d0890 con 0x7f2b6009b050 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b65600640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6009ac70 0x7f2b600b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b66000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b60160400 0x7f2b6014b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.057+0000 7f2b64c00640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b6009b050 0x7f2b600d0040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0090b0 tx=0x7f2b4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.058+0000 7f2b5d600640 1 -- 172.21.15.59:0/1422498338 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c041070 con 0x7f2b6009b050 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.058+0000 7f2b5d600640 1 -- 172.21.15.59:0/1422498338 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4c03c8b0 con 0x7f2b6009b050 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.058+0000 7f2b5d600640 1 -- 172.21.15.59:0/1422498338 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c037040 con 0x7f2b6009b050 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.058+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b600d0b70 con 0x7f2b6009b050 2024-04-23T14:33:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.058+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b600d1130 con 0x7f2b6009b050 2024-04-23T14:33:09.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.060+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b60069190 con 0x7f2b6009b050 2024-04-23T14:33:09.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.061+0000 7f2b5d600640 1 -- 172.21.15.59:0/1422498338 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b4c044070 con 0x7f2b6009b050 2024-04-23T14:33:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.061+0000 7f2b5d600640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b4006c8f0 0x7f2b4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.062+0000 7f2b5d600640 1 -- 172.21.15.59:0/1422498338 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2b4c0ac990 con 0x7f2b6009b050 2024-04-23T14:33:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.062+0000 7f2b65600640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b4006c8f0 0x7f2b4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.063+0000 7f2b65600640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b4006c8f0 0x7f2b4006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2b54006fd0 tx=0x7f2b54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.066+0000 7f2b5d600640 1 -- 172.21.15.59:0/1422498338 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b4c07adc0 con 0x7f2b6009b050 2024-04-23T14:33:09.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.269+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b600b1e20 con 0x7f2b6009b050 2024-04-23T14:33:09.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.270+0000 7f2b5d600640 1 -- 172.21.15.59:0/1422498338 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f2b4c07a6c0 con 0x7f2b6009b050 2024-04-23T14:33:09.270 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b4006c8f0 msgr2=0x7f2b4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b4006c8f0 0x7f2b4006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2b54006fd0 tx=0x7f2b54008040 comp rx=0 tx=0).stop 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b6009b050 msgr2=0x7f2b600d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b6009b050 0x7f2b600d0040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0090b0 tx=0x7f2b4c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 shutdown_connections 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b4006c8f0 0x7f2b4006edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b60160400 0x7f2b6014b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b6009b050 0x7f2b600d0040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 --2- 172.21.15.59:0/1422498338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b6009ac70 0x7f2b600b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 >> 172.21.15.59:0/1422498338 conn(0x7f2b600b40d0 msgr2=0x7f2b600b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 shutdown_connections 2024-04-23T14:33:09.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.273+0000 7f2b68000640 1 -- 172.21.15.59:0/1422498338 wait complete. 2024-04-23T14:33:09.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:33:09.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.511+0000 7fa2c7e00640 1 Processor -- start 2024-04-23T14:33:09.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.512+0000 7fa2c7e00640 1 -- start start 2024-04-23T14:33:09.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.513+0000 7fa2c7e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2c80d0f80 0x7fa2c80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.513+0000 7fa2c7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c815fd30 con 0x7fa2c815ea40 2024-04-23T14:33:09.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.513+0000 7fa2c7e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c80cf930 con 0x7fa2c815f880 2024-04-23T14:33:09.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.513+0000 7fa2c7e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c80cfc10 con 0x7fa2c80d0f80 2024-04-23T14:33:09.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.514+0000 7fa2c6000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa2c815ea40 0x7fa2c80d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33262/0 (socket says 172.21.15.59:33262) 2024-04-23T14:33:09.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.514+0000 7fa2c6000640 1 -- 172.21.15.59:0/146223152 learned_addr learned my addr 172.21.15.59:0/146223152 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.514+0000 7fa2c6a00640 1 --2- 172.21.15.59:0/146223152 >> v2:172.21.15.76:3300/0 conn(0x7fa2c80d0f80 0x7fa2c80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.514+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352923775 0 0) 0x7fa2c815fd30 con 0x7fa2c815ea40 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.514+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a8003520 con 0x7fa2c815ea40 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4245314892 0 0) 0x7fa2a8003520 con 0x7fa2c815ea40 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 >> v2:172.21.15.76:3300/0 conn(0x7fa2c80d0f80 msgr2=0x7fa2c80cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 --2- 172.21.15.59:0/146223152 >> v2:172.21.15.76:3300/0 conn(0x7fa2c80d0f80 0x7fa2c80cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 >> v1:172.21.15.76:6789/0 conn(0x7fa2c815f880 legacy=0x7fa2c80d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c80cfef0 con 0x7fa2c815ea40 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa2b0001670 con 0x7fa2c815ea40 2024-04-23T14:33:09.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 >> v1:172.21.15.59:6789/0 conn(0x7fa2c815ea40 legacy=0x7fa2c80d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2a8003a00 0x7fa2a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a8004800 0x7fa2a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c80cfc10 con 0x7fa2a8005210 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a800c160 con 0x7fa2a8004800 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a800c360 con 0x7fa2a8003a00 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c7400640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.515+0000 7fa2c6000640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a8004800 0x7fa2a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.516+0000 7fa2c6a00640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2a8003a00 0x7fa2a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.516+0000 7fa2c7400640 1 -- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2a8003a00 msgr2=0x7fa2a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.516+0000 7fa2c7400640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2a8003a00 0x7fa2a8003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.516+0000 7fa2c7400640 1 -- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a8004800 msgr2=0x7fa2a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.516+0000 7fa2c7400640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a8004800 0x7fa2a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.516+0000 7fa2c7400640 1 -- 172.21.15.59:0/146223152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8003520 con 0x7fa2a8005210 2024-04-23T14:33:09.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.516+0000 7fa2c6000640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2a8004800 0x7fa2a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:09.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.517+0000 7fa2c7400640 1 -- 172.21.15.59:0/146223152 reap_dead start 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.517+0000 7fa2c7400640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2a800b9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0001e20 tx=0x7fa2a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5954d3a8bad60c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.517+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0001a70 con 0x7fa2a8005210 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.517+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c002ae40 con 0x7fa2a8005210 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.517+0000 7fa2c5600640 1 -- 172.21.15.59:0/146223152 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0003b90 con 0x7fa2a8005210 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.518+0000 7fa2c7e00640 1 -- 172.21.15.59:0/146223152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 msgr2=0x7fa2a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.518+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2a800b9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0001e20 tx=0x7fa2a8005830 comp rx=0 tx=0).stop 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.518+0000 7fa2c7e00640 1 -- 172.21.15.59:0/146223152 shutdown_connections 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.518+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/146223152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2a800b9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.518+0000 7fa2c7e00640 1 -- 172.21.15.59:0/146223152 >> 172.21.15.59:0/146223152 conn(0x7fa2c808bf90 msgr2=0x7fa2c80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.518+0000 7fa2c7e00640 1 -- 172.21.15.59:0/146223152 shutdown_connections 2024-04-23T14:33:09.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.518+0000 7fa2c7e00640 1 -- 172.21.15.59:0/146223152 wait complete. 2024-04-23T14:33:09.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.520+0000 7fa2c7e00640 1 Processor -- start 2024-04-23T14:33:09.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.520+0000 7fa2c7e00640 1 -- start start 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2c8151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c80ce5a0 0x7fa2c809e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2c80cbd80 0x7fa2c80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c80cf250 con 0x7fa2a8005210 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c80d04b0 con 0x7fa2c80ce5a0 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c80cf3d0 con 0x7fa2c80cbd80 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2c8151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2c8151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55184/0 (socket says 172.21.15.59:55184) 2024-04-23T14:33:09.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.521+0000 7fa2c6a00640 1 -- 172.21.15.59:0/742388076 learned_addr learned my addr 172.21.15.59:0/742388076 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c6000640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c80ce5a0 0x7fa2c809e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c7400640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2c80cbd80 0x7fa2c80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c6a00640 1 -- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2c80cbd80 msgr2=0x7fa2c80cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c6a00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2c80cbd80 0x7fa2c80cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c6a00640 1 -- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c80ce5a0 msgr2=0x7fa2c809e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c6a00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c80ce5a0 0x7fa2c809e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c6a00640 1 -- 172.21.15.59:0/742388076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c80c9c80 con 0x7fa2a8005210 2024-04-23T14:33:09.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c6a00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2c8151a70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8010860 tx=0x7fa2b8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2bea00640 1 -- 172.21.15.59:0/742388076 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800f040 con 0x7fa2a8005210 2024-04-23T14:33:09.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2bea00640 1 -- 172.21.15.59:0/742388076 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b800eb60 con 0x7fa2a8005210 2024-04-23T14:33:09.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2bea00640 1 -- 172.21.15.59:0/742388076 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b801c400 con 0x7fa2a8005210 2024-04-23T14:33:09.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.522+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c80c9f60 con 0x7fa2a8005210 2024-04-23T14:33:09.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.523+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c80ca520 con 0x7fa2a8005210 2024-04-23T14:33:09.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.524+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c804fd90 con 0x7fa2a8005210 2024-04-23T14:33:09.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.525+0000 7fa2bea00640 1 -- 172.21.15.59:0/742388076 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2b801c5a0 con 0x7fa2a8005210 2024-04-23T14:33:09.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.526+0000 7fa2bea00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2a406c8f0 0x7fa2a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:09.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.526+0000 7fa2bea00640 1 -- 172.21.15.59:0/742388076 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa2b808c210 con 0x7fa2a8005210 2024-04-23T14:33:09.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.526+0000 7fa2c6000640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2a406c8f0 0x7fa2a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:09.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.527+0000 7fa2c6000640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2a406c8f0 0x7fa2a406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80d52f0 tx=0x7fa2b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.530+0000 7fa2bea00640 1 -- 172.21.15.59:0/742388076 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2b8018030 con 0x7fa2a8005210 2024-04-23T14:33:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.750+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2c80d1ec0 con 0x7fa2a8005210 2024-04-23T14:33:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.750+0000 7fa2bea00640 1 -- 172.21.15.59:0/742388076 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7fa2b8059fe0 con 0x7fa2a8005210 2024-04-23T14:33:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.753+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2a406c8f0 msgr2=0x7fa2a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.753+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2a406c8f0 0x7fa2a406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80d52f0 tx=0x7fa2b0009040 comp rx=0 tx=0).stop 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.753+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 msgr2=0x7fa2c8151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.753+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2c8151a70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8010860 tx=0x7fa2b8010d20 comp rx=0 tx=0).stop 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 shutdown_connections 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2a406c8f0 0x7fa2a406edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2c80cbd80 0x7fa2c80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2c80ce5a0 0x7fa2c809e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 --2- 172.21.15.59:0/742388076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2a8005210 0x7fa2c8151a70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 >> 172.21.15.59:0/742388076 conn(0x7fa2c808bf90 msgr2=0x7fa2c806cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 shutdown_connections 2024-04-23T14:33:09.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:09.754+0000 7fa2c7e00640 1 -- 172.21.15.59:0/742388076 wait complete. 2024-04-23T14:33:09.775 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:33:09.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:09.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:09.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[47389]: fuse finished with error 0 and tester_r 0 2024-04-23T14:33:09.864 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:15.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:33:15.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:15.967 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:33:16.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.246+0000 7f4786200640 1 Processor -- start 2024-04-23T14:33:16.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.246+0000 7f4786200640 1 -- start start 2024-04-23T14:33:16.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.248+0000 7f4786200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f47801454e0 0x7f4780145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.248+0000 7f4786200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.248+0000 7f4786200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47800cd520 con 0x7f47800b3e40 2024-04-23T14:33:16.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.248+0000 7f4786200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f47800cddc0 con 0x7f47801454e0 2024-04-23T14:33:16.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.248+0000 7f4786200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f47800cd800 con 0x7f47800cd020 2024-04-23T14:33:16.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.248+0000 7f477fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f47800b3e40 0x7f4780160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55964/0 (socket says 172.21.15.59:55964) 2024-04-23T14:33:16.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477fe00640 1 -- 172.21.15.59:0/4107603013 learned_addr learned my addr 172.21.15.59:0/4107603013 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:16.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.248+0000 7f477f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f47801454e0 0x7f4780145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477ea00640 1 --2- 172.21.15.59:0/4107603013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477f400640 1 -- 172.21.15.59:0/4107603013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 msgr2=0x7f47800b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:16.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477e000640 1 -- 172.21.15.59:0/4107603013 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052911958 0 0) 0x7f47800cd520 con 0x7f47800b3e40 2024-04-23T14:33:16.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477f400640 1 --2- 172.21.15.59:0/4107603013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477f400640 1 -- 172.21.15.59:0/4107603013 >> v1:172.21.15.59:6789/0 conn(0x7f47800b3e40 legacy=0x7f4780160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477f400640 1 -- 172.21.15.59:0/4107603013 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47800cdae0 con 0x7f47801454e0 2024-04-23T14:33:16.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.249+0000 7f477ea00640 1 --2- 172.21.15.59:0/4107603013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:16.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.250+0000 7f477f400640 1 --2- 172.21.15.59:0/4107603013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47801454e0 0x7f4780145bb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f47700097a0 tx=0x7f477002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c487f15a5ff1827 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f477e000640 1 -- 172.21.15.59:0/4107603013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770037070 con 0x7f47801454e0 2024-04-23T14:33:16.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f477e000640 1 -- 172.21.15.59:0/4107603013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477002ae50 con 0x7f47801454e0 2024-04-23T14:33:16.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f477e000640 1 -- 172.21.15.59:0/4107603013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770032660 con 0x7f47801454e0 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f4786200640 1 -- 172.21.15.59:0/4107603013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47801454e0 msgr2=0x7f4780145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f4786200640 1 --2- 172.21.15.59:0/4107603013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47801454e0 0x7f4780145bb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f47700097a0 tx=0x7f477002a100 comp rx=0 tx=0).stop 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f4786200640 1 -- 172.21.15.59:0/4107603013 shutdown_connections 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f4786200640 1 --2- 172.21.15.59:0/4107603013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f4786200640 1 --2- 172.21.15.59:0/4107603013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47801454e0 0x7f4780145bb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.251+0000 7f4786200640 1 -- 172.21.15.59:0/4107603013 >> 172.21.15.59:0/4107603013 conn(0x7f4780067ea0 msgr2=0x7f478006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.252+0000 7f4786200640 1 -- 172.21.15.59:0/4107603013 shutdown_connections 2024-04-23T14:33:16.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.252+0000 7f4786200640 1 -- 172.21.15.59:0/4107603013 wait complete. 2024-04-23T14:33:16.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.253+0000 7f4786200640 1 Processor -- start 2024-04-23T14:33:16.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.253+0000 7f4786200640 1 -- start start 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.254+0000 7f4786200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47800b3e40 0x7f478015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.254+0000 7f4786200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.254+0000 7f4786200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47801454e0 0x7f47800b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.254+0000 7f4786200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4780146270 con 0x7f47800b3e40 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.254+0000 7f4786200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f47801460f0 con 0x7f47801454e0 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.254+0000 7f4786200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4780161020 con 0x7f47800cd020 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47800b3e40 0x7f478015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47800b3e40 0x7f478015c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40710/0 (socket says 172.21.15.59:40710) 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 -- 172.21.15.59:0/3764698669 learned_addr learned my addr 172.21.15.59:0/3764698669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477fe00640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47801454e0 0x7f47800b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477ea00640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 -- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 msgr2=0x7f47800b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 -- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47801454e0 msgr2=0x7f47800b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47801454e0 0x7f47800b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477f400640 1 -- 172.21.15.59:0/3764698669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47800a9600 con 0x7f47800b3e40 2024-04-23T14:33:16.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.255+0000 7f477fe00640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47801454e0 0x7f47800b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:16.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.256+0000 7f477f400640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47800b3e40 0x7f478015c070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f47700097a0 tx=0x7f47700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.256+0000 7f4777400640 1 -- 172.21.15.59:0/3764698669 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770037040 con 0x7f47800b3e40 2024-04-23T14:33:16.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.256+0000 7f4777400640 1 -- 172.21.15.59:0/3764698669 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4770004250 con 0x7f47800b3e40 2024-04-23T14:33:16.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.256+0000 7f4777400640 1 -- 172.21.15.59:0/3764698669 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770003610 con 0x7f47800b3e40 2024-04-23T14:33:16.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.256+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47800a98a0 con 0x7f47800b3e40 2024-04-23T14:33:16.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.256+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47800a9e60 con 0x7f47800b3e40 2024-04-23T14:33:16.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.257+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4780068a80 con 0x7f47800b3e40 2024-04-23T14:33:16.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.258+0000 7f4777400640 1 -- 172.21.15.59:0/3764698669 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47700037b0 con 0x7f47800b3e40 2024-04-23T14:33:16.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.259+0000 7f4777400640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f476006c8f0 0x7f476006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.259+0000 7f4777400640 1 -- 172.21.15.59:0/3764698669 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f47700acb00 con 0x7f47800b3e40 2024-04-23T14:33:16.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.260+0000 7f477ea00640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f476006c8f0 0x7f476006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.261+0000 7f477ea00640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f476006c8f0 0x7f476006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4780069260 tx=0x7f476c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.264+0000 7f4777400640 1 -- 172.21.15.59:0/3764698669 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f477007ae90 con 0x7f47800b3e40 2024-04-23T14:33:16.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.513+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47800af6f0 con 0x7f47800b3e40 2024-04-23T14:33:16.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.514+0000 7f4777400640 1 -- 172.21.15.59:0/3764698669 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f477007a790 con 0x7f47800b3e40 2024-04-23T14:33:16.515 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:16.515 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:33:16.515 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:33:16.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f476006c8f0 msgr2=0x7f476006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f476006c8f0 0x7f476006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4780069260 tx=0x7f476c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47800b3e40 msgr2=0x7f478015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47800b3e40 0x7f478015c070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f47700097a0 tx=0x7f47700046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 shutdown_connections 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f476006c8f0 0x7f476006edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47800cd020 0x7f47800b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47801454e0 0x7f47800b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 --2- 172.21.15.59:0/3764698669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47800b3e40 0x7f478015c070 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 >> 172.21.15.59:0/3764698669 conn(0x7f4780067ea0 msgr2=0x7f47800b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 shutdown_connections 2024-04-23T14:33:16.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.517+0000 7f4786200640 1 -- 172.21.15.59:0/3764698669 wait complete. 2024-04-23T14:33:16.537 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-23T14:33:16.776 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:33:16.777 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:33:16.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:16.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.978+0000 7f7d5b200640 1 Processor -- start 2024-04-23T14:33:16.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.979+0000 7f7d5b200640 1 -- start start 2024-04-23T14:33:16.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.980+0000 7f7d5b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 0x7f7d5c0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.980+0000 7f7d5b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d5c0cf030 con 0x7f7d5c0b5ff0 2024-04-23T14:33:16.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.980+0000 7f7d5b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d5c0b6a40 con 0x7f7d5c15e7c0 2024-04-23T14:33:16.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.981+0000 7f7d5b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c0b7000 con 0x7f7d5c0d14a0 2024-04-23T14:33:16.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.981+0000 7f7d59400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7d5c0b5ff0 0x7f7d5c0d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55972/0 (socket says 172.21.15.59:55972) 2024-04-23T14:33:16.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.981+0000 7f7d59400640 1 -- 172.21.15.59:0/1913537175 learned_addr learned my addr 172.21.15.59:0/1913537175 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.981+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3363269718 0 0) 0x7f7d44001490 con 0x7f7d5c0b5ff0 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.981+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d3c003520 con 0x7f7d5c0b5ff0 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.981+0000 7f7d5a800640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 0x7f7d5c0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4164564541 0 0) 0x7f7d3c003520 con 0x7f7d5c0b5ff0 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 msgr2=0x7f7d5c0b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 0x7f7d5c0b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 >> v1:172.21.15.76:6789/0 conn(0x7f7d5c15e7c0 legacy=0x7f7d5c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c0b6d20 con 0x7f7d5c0b5ff0 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f7d44001bc0 con 0x7f7d5c0b5ff0 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 >> v1:172.21.15.59:6789/0 conn(0x7f7d5c0b5ff0 legacy=0x7f7d5c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c0039e0 0x7f7d3c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c004800 0x7f7d3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 0x7f7d3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0b7000 con 0x7f7d3c004800 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c00baf0 con 0x7f7d3c0039e0 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c00bcf0 con 0x7f7d5c0d14a0 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.982+0000 7f7d59400640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c004800 0x7f7d3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.983+0000 7f7d59e00640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c0039e0 0x7f7d3c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.983+0000 7f7d5a800640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 0x7f7d3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.983+0000 7f7d59400640 1 -- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 msgr2=0x7f7d3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.983+0000 7f7d59400640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c0d14a0 0x7f7d3c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.983+0000 7f7d59400640 1 -- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c0039e0 msgr2=0x7f7d3c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.983+0000 7f7d59400640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c0039e0 0x7f7d3c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.983+0000 7f7d59400640 1 -- 172.21.15.59:0/1913537175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c003520 con 0x7f7d3c004800 2024-04-23T14:33:16.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.984+0000 7f7d59400640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c004800 0x7f7d3c004c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400a9a0 tx=0x7f7d3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3ff26fd9903ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.984+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400a5c0 con 0x7f7d3c004800 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.984+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d4402f070 con 0x7f7d3c004800 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.984+0000 7f7d58a00640 1 -- 172.21.15.59:0/1913537175 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4402bdd0 con 0x7f7d3c004800 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.985+0000 7f7d5b200640 1 -- 172.21.15.59:0/1913537175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c004800 msgr2=0x7f7d3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.985+0000 7f7d5b200640 1 --2- 172.21.15.59:0/1913537175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c004800 0x7f7d3c004c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400a9a0 tx=0x7f7d3c005830 comp rx=0 tx=0).stop 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.985+0000 7f7d5a800640 1 -- 172.21.15.59:0/1913537175 reap_dead start 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.985+0000 7f7d5b200640 1 -- 172.21.15.59:0/1913537175 shutdown_connections 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.985+0000 7f7d5b200640 1 -- 172.21.15.59:0/1913537175 >> 172.21.15.59:0/1913537175 conn(0x7f7d5c09d920 msgr2=0x7f7d5c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.985+0000 7f7d5b200640 1 -- 172.21.15.59:0/1913537175 shutdown_connections 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.985+0000 7f7d5b200640 1 -- 172.21.15.59:0/1913537175 wait complete. 2024-04-23T14:33:16.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.987+0000 7f7d5b200640 1 Processor -- start 2024-04-23T14:33:16.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.987+0000 7f7d5b200640 1 -- start start 2024-04-23T14:33:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c004800 0x7f7d5c0b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c159e30 0x7f7d5c0b9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c0b76a0 0x7f7d5c0b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d2800 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c0cf620 con 0x7f7d3c004800 2024-04-23T14:33:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5c0d1a80 con 0x7f7d5c159e30 2024-04-23T14:33:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c0b76a0 0x7f7d5c0b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c0b76a0 0x7f7d5c0b7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40732/0 (socket says 172.21.15.59:40732) 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.988+0000 7f7d5a800640 1 -- 172.21.15.59:0/293857454 learned_addr learned my addr 172.21.15.59:0/293857454 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d59e00640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c004800 0x7f7d5c0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d59400640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c159e30 0x7f7d5c0b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d5a800640 1 -- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c159e30 msgr2=0x7f7d5c0b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d5a800640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c159e30 0x7f7d5c0b9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d5a800640 1 -- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c004800 msgr2=0x7f7d5c0b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d5a800640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c004800 0x7f7d5c0b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d5a800640 1 -- 172.21.15.59:0/293857454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c0ce070 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d5a800640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c0b76a0 0x7f7d5c0b7b20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e420 tx=0x7f7d5400e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.989+0000 7f7d52000640 1 -- 172.21.15.59:0/293857454 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54017020 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.990+0000 7f7d52000640 1 -- 172.21.15.59:0/293857454 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d54013630 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.990+0000 7f7d52000640 1 -- 172.21.15.59:0/293857454 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400f040 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.990+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c0b8280 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.990+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c0cb720 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.991+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c04fd90 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.992+0000 7f7d52000640 1 -- 172.21.15.59:0/293857454 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d5401a020 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.993+0000 7f7d52000640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d3806c8f0 0x7f7d3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:16.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.993+0000 7f7d52000640 1 -- 172.21.15.59:0/293857454 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7d5408b2a0 con 0x7f7d5c0b76a0 2024-04-23T14:33:16.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.993+0000 7f7d59e00640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d3806c8f0 0x7f7d3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:16.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.994+0000 7f7d59e00640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d3806c8f0 0x7f7d3806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c005540 tx=0x7f7d4c007c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:16.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:16.997+0000 7f7d52000640 1 -- 172.21.15.59:0/293857454 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d54021030 con 0x7f7d5c0b76a0 2024-04-23T14:33:17.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.242+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d5c0b7480 con 0x7f7d5c0b76a0 2024-04-23T14:33:17.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.243+0000 7f7d52000640 1 -- 172.21.15.59:0/293857454 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f7d540596c0 con 0x7f7d5c0b76a0 2024-04-23T14:33:17.245 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:17.245 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:33:17.245 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:33:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d3806c8f0 msgr2=0x7f7d3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d3806c8f0 0x7f7d3806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c005540 tx=0x7f7d4c007c50 comp rx=0 tx=0).stop 2024-04-23T14:33:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c0b76a0 msgr2=0x7f7d5c0b7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c0b76a0 0x7f7d5c0b7b20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e420 tx=0x7f7d5400e8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 shutdown_connections 2024-04-23T14:33:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d3806c8f0 0x7f7d3806edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d5c159e30 0x7f7d5c0b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c0b76a0 0x7f7d5c0b7b20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 --2- 172.21.15.59:0/293857454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c004800 0x7f7d5c0b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 >> 172.21.15.59:0/293857454 conn(0x7f7d5c09d920 msgr2=0x7f7d5c06cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 shutdown_connections 2024-04-23T14:33:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:17.246+0000 7f7d5b200640 1 -- 172.21.15.59:0/293857454 wait complete. 2024-04-23T14:33:17.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-04-23T14:33:17.505 INFO:teuthology.orchestra.run.smithi059.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-23T14:33:17.505 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-23T14:33:18.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:18.834 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:27.506 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-23T14:33:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.624+0000 7f72dc7e6f40 1 Processor -- start 2024-04-23T14:33:27.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.625+0000 7f72dc7e6f40 1 -- start start 2024-04-23T14:33:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.625+0000 7f72dc7e6f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c9755431d0 0x55c975573910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72dc7e6f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c975602ac0 0x55c975618b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72dc7e6f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c97561dad0 0x55c97561df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72dc7e6f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c9755459c0 con 0x55c9755431d0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72dc7e6f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c9754eccc0 con 0x55c975602ac0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72dc7e6f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c975543b20 con 0x55c97561dad0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c9755431d0 0x55c975573910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c9755431d0 0x55c975573910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35838/0 (socket says 172.21.15.59:35838) 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 -- 172.21.15.59:0/2889406655 learned_addr learned my addr 172.21.15.59:0/2889406655 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9000640 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c975602ac0 0x55c975618b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72da400640 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c97561dad0 0x55c97561df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 -- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c97561dad0 msgr2=0x55c97561df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c97561dad0 0x55c97561df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 -- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c975602ac0 msgr2=0x55c975618b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c975602ac0 0x55c975618b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72d9a00640 1 -- 172.21.15.59:0/2889406655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c975573430 con 0x55c9755431d0 2024-04-23T14:33:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.626+0000 7f72da400640 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c97561dad0 0x55c97561df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.627+0000 7f72d9a00640 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c9755431d0 0x55c975573910 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b2e0 tx=0x7f72cc00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901f8a784cfc0fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.627+0000 7f72d3400640 1 -- 172.21.15.59:0/2889406655 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc008070 con 0x55c9755431d0 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.627+0000 7f72d3400640 1 -- 172.21.15.59:0/2889406655 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc00d500 con 0x55c9755431d0 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.627+0000 7f72d3400640 1 -- 172.21.15.59:0/2889406655 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00d8f0 con 0x55c9755431d0 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2889406655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c9755431d0 msgr2=0x55c975573910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c9755431d0 0x55c975573910 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b2e0 tx=0x7f72cc00b7a0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2889406655 shutdown_connections 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c97561dad0 0x55c97561df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c975602ac0 0x55c975618b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2889406655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c9755431d0 0x55c975573910 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2889406655 >> 172.21.15.59:0/2889406655 conn(0x55c97558fcc0 msgr2=0x55c9756203b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2889406655 shutdown_connections 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.628+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2889406655 wait complete. 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.629+0000 7f72dc7e6f40 1 Processor -- start 2024-04-23T14:33:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.629+0000 7f72dc7e6f40 1 -- start start 2024-04-23T14:33:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72dc7e6f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c9755431d0 0x55c9755fe880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72dc7e6f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c975602ac0 0x55c9756203b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72dc7e6f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c97561dad0 0x55c9756208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72dc7e6f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c97552fd80 con 0x55c975602ac0 2024-04-23T14:33:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72dc7e6f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c975530060 con 0x55c97561dad0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72dc7e6f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c975530340 con 0x55c9755431d0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72d9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c975602ac0 0x55c9756203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72d9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c975602ac0 0x55c9756203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35846/0 (socket says 172.21.15.59:35846) 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72d9000640 1 -- 172.21.15.59:0/2656291777 learned_addr learned my addr 172.21.15.59:0/2656291777 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72d9a00640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c9755431d0 0x55c9755fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72da400640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c97561dad0 0x55c9756208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72d9000640 1 -- 172.21.15.59:0/2656291777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c9755431d0 msgr2=0x55c9755fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.630+0000 7f72d9000640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c9755431d0 0x55c9755fe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72d9000640 1 -- 172.21.15.59:0/2656291777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c97561dad0 msgr2=0x55c9756208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72d9000640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c97561dad0 0x55c9756208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72d9000640 1 -- 172.21.15.59:0/2656291777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c975577230 con 0x55c975602ac0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72d9000640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c975602ac0 0x55c9756203b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f72c400a440 tx=0x7f72c4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72d2000640 1 -- 172.21.15.59:0/2656291777 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c405fb50 con 0x55c975602ac0 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72d2000640 1 -- 172.21.15.59:0/2656291777 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c405fd10 con 0x55c975602ac0 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72d2000640 1 -- 172.21.15.59:0/2656291777 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c405d650 con 0x55c975602ac0 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.631+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c975577510 con 0x55c975602ac0 2024-04-23T14:33:27.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.632+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9755777f0 con 0x55c975602ac0 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.634+0000 7f72d2000640 1 -- 172.21.15.59:0/2656291777 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72c4070060 con 0x55c975602ac0 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.634+0000 7f72d2000640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72b806b320 0x7f72b806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.635+0000 7f72d2000640 1 -- 172.21.15.59:0/2656291777 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f72c40cebe0 con 0x55c975602ac0 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.635+0000 7f72d9a00640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72b806b320 0x7f72b806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.635+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55c97561b310 0x55c97561b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.635+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97561bc60 con 0x55c97561b310 2024-04-23T14:33:27.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.635+0000 7f72da400640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55c97561b310 0x55c97561b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.636+0000 7f72d9a00640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72b806b320 0x7f72b806d7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b2e0 tx=0x7f72cc008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.636+0000 7f72da400640 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55c97561b310 0x55c97561b720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.636+0000 7f72da400640 1 -- 172.21.15.59:0/2656291777 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f72d400c7f0 con 0x55c97561b310 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.636+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55c97561b310 msgr2=0x55c97561b720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.636+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55c97561b310 0x55c97561b720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72b806b320 msgr2=0x7f72b806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72b806b320 0x7f72b806d7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b2e0 tx=0x7f72cc008000 comp rx=0 tx=0).stop 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c975602ac0 msgr2=0x55c9756203b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 --2- 172.21.15.59:0/2656291777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c975602ac0 0x55c9756203b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f72c400a440 tx=0x7f72c4002fb0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72da400640 1 -- 172.21.15.59:0/2656291777 reap_dead start 2024-04-23T14:33:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 shutdown_connections 2024-04-23T14:33:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 >> 172.21.15.59:0/2656291777 conn(0x55c97558fcc0 msgr2=0x55c9755900a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 shutdown_connections 2024-04-23T14:33:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.637+0000 7f72dc7e6f40 1 -- 172.21.15.59:0/2656291777 wait complete. 2024-04-23T14:33:27.643 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-23T14:33:27.643 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-23T14:33:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.751+0000 7f60cd014f40 1 Processor -- start 2024-04-23T14:33:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.751+0000 7f60cd014f40 1 -- start start 2024-04-23T14:33:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.752+0000 7f60cd014f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 0x55ebdedfcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.752+0000 7f60cd014f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b470 0x55ebdee91560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.752+0000 7f60cd014f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b850 0x55ebdee91aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.752+0000 7f60cd014f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ebded76cc0 con 0x55ebdee9b470 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.752+0000 7f60cd014f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ebded7ad00 con 0x55ebdedff500 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.752+0000 7f60cd014f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ebdee1a420 con 0x55ebdee9b850 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.752+0000 7f60c0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b470 0x55ebdee91560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b470 0x55ebdee91560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35872/0 (socket says 172.21.15.59:35872) 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 -- 172.21.15.59:0/3767670092 learned_addr learned my addr 172.21.15.59:0/3767670092 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60ca000640 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b850 0x55ebdee91aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 -- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b850 msgr2=0x55ebdee91aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c9600640 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 0x55ebdedfcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b850 0x55ebdee91aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 -- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 msgr2=0x55ebdedfcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 0x55ebdedfcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 -- 172.21.15.59:0/3767670092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebdee9abf0 con 0x55ebdee9b470 2024-04-23T14:33:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c9600640 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 0x55ebdedfcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.753+0000 7f60c0c00640 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b470 0x55ebdee91560 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b2c0 tx=0x7f60b400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8c7648d724fdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60c3e00640 1 -- 172.21.15.59:0/3767670092 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4008070 con 0x55ebdee9b470 2024-04-23T14:33:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60c3e00640 1 -- 172.21.15.59:0/3767670092 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b400d500 con 0x55ebdee9b470 2024-04-23T14:33:27.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60c3e00640 1 -- 172.21.15.59:0/3767670092 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b400d8f0 con 0x55ebdee9b470 2024-04-23T14:33:27.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 -- 172.21.15.59:0/3767670092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b470 msgr2=0x55ebdee91560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b470 0x55ebdee91560 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f60b400b2c0 tx=0x7f60b400b780 comp rx=0 tx=0).stop 2024-04-23T14:33:27.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 -- 172.21.15.59:0/3767670092 shutdown_connections 2024-04-23T14:33:27.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b850 0x55ebdee91aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b470 0x55ebdee91560 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3767670092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 0x55ebdedfcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 -- 172.21.15.59:0/3767670092 >> 172.21.15.59:0/3767670092 conn(0x55ebdee1b290 msgr2=0x55ebdedd1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 -- 172.21.15.59:0/3767670092 shutdown_connections 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.754+0000 7f60cd014f40 1 -- 172.21.15.59:0/3767670092 wait complete. 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.755+0000 7f60cd014f40 1 Processor -- start 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.755+0000 7f60cd014f40 1 -- start start 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.756+0000 7f60cd014f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 0x55ebdee8a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.756+0000 7f60cd014f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b470 0x55ebdef32410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.756+0000 7f60cd014f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b850 0x55ebdef32950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.756+0000 7f60cd014f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ebdee91fe0 con 0x55ebdee9b850 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.756+0000 7f60cd014f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ebdeea8c90 con 0x55ebdedff500 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.756+0000 7f60cd014f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ebdee8ad60 con 0x55ebdee9b470 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b850 0x55ebdef32950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b850 0x55ebdef32950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35876/0 (socket says 172.21.15.59:35876) 2024-04-23T14:33:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 -- 172.21.15.59:0/3796384296 learned_addr learned my addr 172.21.15.59:0/3796384296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 -- 172.21.15.59:0/3796384296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b470 msgr2=0x55ebdef32410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ebdee9b470 0x55ebdef32410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 -- 172.21.15.59:0/3796384296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 msgr2=0x55ebdee8a820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:33:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ebdedff500 0x55ebdee8a820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.757+0000 7f60ca000640 1 -- 172.21.15.59:0/3796384296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebdedce380 con 0x55ebdee9b850 2024-04-23T14:33:27.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.758+0000 7f60ca000640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b850 0x55ebdef32950 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f60c4055040 tx=0x7f60c4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.758+0000 7f60c2a00640 1 -- 172.21.15.59:0/3796384296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4056040 con 0x55ebdee9b850 2024-04-23T14:33:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.758+0000 7f60c2a00640 1 -- 172.21.15.59:0/3796384296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c4052820 con 0x55ebdee9b850 2024-04-23T14:33:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.758+0000 7f60c2a00640 1 -- 172.21.15.59:0/3796384296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4052c10 con 0x55ebdee9b850 2024-04-23T14:33:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.758+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ebdedce620 con 0x55ebdee9b850 2024-04-23T14:33:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.758+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebdedcebe0 con 0x55ebdee9b850 2024-04-23T14:33:27.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.760+0000 7f60c2a00640 1 -- 172.21.15.59:0/3796384296 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60c4059020 con 0x55ebdee9b850 2024-04-23T14:33:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.761+0000 7f60c2a00640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60a806b2d0 0x7f60a806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.761+0000 7f60c2a00640 1 -- 172.21.15.59:0/3796384296 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f60c40ceb60 con 0x55ebdee9b850 2024-04-23T14:33:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.761+0000 7f60c9600640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60a806b2d0 0x7f60a806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.762+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ebdedcfff0 0x55ebdee9bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.762+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ebdee90760 con 0x55ebdedcfff0 2024-04-23T14:33:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.762+0000 7f60c0c00640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ebdedcfff0 0x55ebdee9bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.762+0000 7f60c9600640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60a806b2d0 0x7f60a806d790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0012e0 tx=0x7f60bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60c0c00640 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ebdedcfff0 0x55ebdee9bc60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60c0c00640 1 -- 172.21.15.59:0/3796384296 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f60b400d7c0 con 0x55ebdedcfff0 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ebdedcfff0 msgr2=0x55ebdee9bc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ebdedcfff0 0x55ebdee9bc60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60a806b2d0 msgr2=0x7f60a806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60a806b2d0 0x7f60a806d790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0012e0 tx=0x7f60bc008040 comp rx=0 tx=0).stop 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b850 msgr2=0x55ebdef32950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 --2- 172.21.15.59:0/3796384296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ebdee9b850 0x55ebdef32950 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f60c4055040 tx=0x7f60c4002fb0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60ca000640 1 -- 172.21.15.59:0/3796384296 reap_dead start 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 shutdown_connections 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.763+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 >> 172.21.15.59:0/3796384296 conn(0x55ebdee1b290 msgr2=0x55ebdeea2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.764+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 shutdown_connections 2024-04-23T14:33:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.764+0000 7f60cd014f40 1 -- 172.21.15.59:0/3796384296 wait complete. 2024-04-23T14:33:27.769 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-23T14:33:27.769 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-23T14:33:27.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.877+0000 7fa40754df40 1 Processor -- start 2024-04-23T14:33:27.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.877+0000 7fa40754df40 1 -- start start 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa40754df40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dd49500 0x563e0dd46be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa40754df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dde5470 0x563e0ddf0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa40754df40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5850 0x563e0ddf09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa40754df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e0dd64750 con 0x563e0dde5470 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa40754df40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563e0dcc0cc0 con 0x563e0dde5850 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa40754df40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563e0ddef5d0 con 0x563e0dd49500 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa403e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dde5470 0x563e0ddf0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa403e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dde5470 0x563e0ddf0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35892/0 (socket says 172.21.15.59:35892) 2024-04-23T14:33:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.878+0000 7fa403e00640 1 -- 172.21.15.59:0/2066677177 learned_addr learned my addr 172.21.15.59:0/2066677177 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa405200640 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5850 0x563e0ddf09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa404800640 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dd49500 0x563e0dd46be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa403e00640 1 -- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dd49500 msgr2=0x563e0dd46be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa403e00640 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dd49500 0x563e0dd46be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa403e00640 1 -- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5850 msgr2=0x563e0ddf09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa403e00640 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5850 0x563e0ddf09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa403e00640 1 -- 172.21.15.59:0/2066677177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e0dd48df0 con 0x563e0dde5470 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa403e00640 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dde5470 0x563e0ddf0460 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00a4b0 tx=0x7fa3ec050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa17cff45a4ef5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa402a00640 1 -- 172.21.15.59:0/2066677177 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec055030 con 0x563e0dde5470 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa402a00640 1 -- 172.21.15.59:0/2066677177 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ec051cd0 con 0x563e0dde5470 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.879+0000 7fa402a00640 1 -- 172.21.15.59:0/2066677177 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec05b670 con 0x563e0dde5470 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 -- 172.21.15.59:0/2066677177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dde5470 msgr2=0x563e0ddf0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dde5470 0x563e0ddf0460 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00a4b0 tx=0x7fa3ec050000 comp rx=0 tx=0).stop 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 -- 172.21.15.59:0/2066677177 shutdown_connections 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5850 0x563e0ddf09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dde5470 0x563e0ddf0460 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 --2- 172.21.15.59:0/2066677177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dd49500 0x563e0dd46be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 -- 172.21.15.59:0/2066677177 >> 172.21.15.59:0/2066677177 conn(0x563e0dd65290 msgr2=0x563e0ddf14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 -- 172.21.15.59:0/2066677177 shutdown_connections 2024-04-23T14:33:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.880+0000 7fa40754df40 1 -- 172.21.15.59:0/2066677177 wait complete. 2024-04-23T14:33:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.881+0000 7fa40754df40 1 Processor -- start 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.881+0000 7fa40754df40 1 -- start start 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa40754df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dd49500 0x563e0dd1a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa40754df40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5470 0x563e0de869c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa40754df40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dde5850 0x563e0dded8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa40754df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563e0dd1bf50 con 0x563e0dd49500 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa40754df40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563e0dd44120 con 0x563e0dde5470 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa40754df40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563e0ddefd90 con 0x563e0dde5850 2024-04-23T14:33:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa404800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dd49500 0x563e0dd1a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa404800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dd49500 0x563e0dd1a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35898/0 (socket says 172.21.15.59:35898) 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.882+0000 7fa404800640 1 -- 172.21.15.59:0/1956292027 learned_addr learned my addr 172.21.15.59:0/1956292027 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa403e00640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5470 0x563e0de869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa405200640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dde5850 0x563e0dded8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa404800640 1 -- 172.21.15.59:0/1956292027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dde5850 msgr2=0x563e0dded8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa404800640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563e0dde5850 0x563e0dded8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa404800640 1 -- 172.21.15.59:0/1956292027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5470 msgr2=0x563e0de869c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa404800640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563e0dde5470 0x563e0de869c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa404800640 1 -- 172.21.15.59:0/1956292027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e0dd18930 con 0x563e0dd49500 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.883+0000 7fa404800640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dd49500 0x563e0dd1a3a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4001e70 tx=0x7fa3f400a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.884+0000 7fa401600640 1 -- 172.21.15.59:0/1956292027 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4010470 con 0x563e0dd49500 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.884+0000 7fa401600640 1 -- 172.21.15.59:0/1956292027 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f4010610 con 0x563e0dd49500 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.884+0000 7fa401600640 1 -- 172.21.15.59:0/1956292027 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40109e0 con 0x563e0dd49500 2024-04-23T14:33:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.884+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e0dd198c0 con 0x563e0dd49500 2024-04-23T14:33:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.884+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0ddeffb0 con 0x563e0dd49500 2024-04-23T14:33:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.886+0000 7fa401600640 1 -- 172.21.15.59:0/1956292027 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3f400e040 con 0x563e0dd49500 2024-04-23T14:33:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.887+0000 7fa401600640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3e006b2d0 0x7fa3e006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.887+0000 7fa401600640 1 -- 172.21.15.59:0/1956292027 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa3f408a680 con 0x563e0dd49500 2024-04-23T14:33:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.887+0000 7fa403e00640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3e006b2d0 0x7fa3e006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.887+0000 7fa40754df40 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563e0dd47190 0x563e0dde0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.887+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e0ddee2d0 con 0x563e0dd47190 2024-04-23T14:33:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.888+0000 7fa405200640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563e0dd47190 0x563e0dde0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.888+0000 7fa403e00640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3e006b2d0 0x7fa3e006d790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec062000 tx=0x7fa3ec051000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.888+0000 7fa405200640 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563e0dd47190 0x563e0dde0870 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa405200640 1 -- 172.21.15.59:0/1956292027 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fa3fc0095c0 con 0x563e0dd47190 2024-04-23T14:33:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563e0dd47190 msgr2=0x563e0dde0870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563e0dd47190 0x563e0dde0870 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3e006b2d0 msgr2=0x7fa3e006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3e006b2d0 0x7fa3e006d790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec062000 tx=0x7fa3ec051000 comp rx=0 tx=0).stop 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dd49500 msgr2=0x563e0dd1a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 --2- 172.21.15.59:0/1956292027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563e0dd49500 0x563e0dd1a3a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4001e70 tx=0x7fa3f400a5a0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa405200640 1 -- 172.21.15.59:0/1956292027 reap_dead start 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 shutdown_connections 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 >> 172.21.15.59:0/1956292027 conn(0x563e0dd65290 msgr2=0x563e0dd1b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 shutdown_connections 2024-04-23T14:33:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.889+0000 7fa40754df40 1 -- 172.21.15.59:0/1956292027 wait complete. 2024-04-23T14:33:27.895 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-23T14:33:27.895 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-23T14:33:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.990+0000 7ff5041cdf40 1 Processor -- start 2024-04-23T14:33:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.990+0000 7ff5041cdf40 1 -- start start 2024-04-23T14:33:27.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.991+0000 7ff5041cdf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f82810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.991+0000 7ff5041cdf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8eb9bb0 0x556ed8ee6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.991+0000 7ff5041cdf40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8f8d340 0x556ed8ee6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.991+0000 7ff5041cdf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ed8f03750 con 0x556ed8eb9bb0 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.991+0000 7ff5041cdf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556ed8e5fcc0 con 0x7ff4e80022a0 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.991+0000 7ff5041cdf40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556ed8f8e090 con 0x556ed8f8d340 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.991+0000 7ff500a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8eb9bb0 0x556ed8ee6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8eb9bb0 0x556ed8ee6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35920/0 (socket says 172.21.15.59:35920) 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 -- 172.21.15.59:0/892904481 learned_addr learned my addr 172.21.15.59:0/892904481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff501400640 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f82810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff501e00640 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8f8d340 0x556ed8ee6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 -- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8f8d340 msgr2=0x556ed8ee6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8f8d340 0x556ed8ee6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 -- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 msgr2=0x556ed8f82810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f82810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 -- 172.21.15.59:0/892904481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed8ee4a70 con 0x556ed8eb9bb0 2024-04-23T14:33:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff501e00640 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8f8d340 0x556ed8ee6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff500a00640 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8eb9bb0 0x556ed8ee6ab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00b500 tx=0x7ff4ec00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b147653ac7733717 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.992+0000 7ff4fb400640 1 -- 172.21.15.59:0/892904481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec008070 con 0x556ed8eb9bb0 2024-04-23T14:33:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff4fb400640 1 -- 172.21.15.59:0/892904481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ec00d7e0 con 0x556ed8eb9bb0 2024-04-23T14:33:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff4fb400640 1 -- 172.21.15.59:0/892904481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00dbd0 con 0x556ed8eb9bb0 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 -- 172.21.15.59:0/892904481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8eb9bb0 msgr2=0x556ed8ee6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8eb9bb0 0x556ed8ee6ab0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00b500 tx=0x7ff4ec00b9c0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 -- 172.21.15.59:0/892904481 shutdown_connections 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8f8d340 0x556ed8ee6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8eb9bb0 0x556ed8ee6ab0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/892904481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f82810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 -- 172.21.15.59:0/892904481 >> 172.21.15.59:0/892904481 conn(0x556ed8f04290 msgr2=0x556ed8ee7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 -- 172.21.15.59:0/892904481 shutdown_connections 2024-04-23T14:33:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.993+0000 7ff5041cdf40 1 -- 172.21.15.59:0/892904481 wait complete. 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.994+0000 7ff5041cdf40 1 Processor -- start 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.994+0000 7ff5041cdf40 1 -- start start 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.995+0000 7ff5041cdf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8f8d340 0x556ed8ebb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.995+0000 7ff5041cdf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f81e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.995+0000 7ff5041cdf40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8ee6020 0x556ed8ee7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.995+0000 7ff5041cdf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ed8ee55a0 con 0x556ed8f8d340 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.995+0000 7ff5041cdf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556ed8f90ad0 con 0x7ff4e80022a0 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.995+0000 7ff5041cdf40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556ed8f82d50 con 0x556ed8ee6020 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.995+0000 7ff501400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8f8d340 0x556ed8ebb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8f8d340 0x556ed8ebb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35936/0 (socket says 172.21.15.59:35936) 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 -- 172.21.15.59:0/1072326002 learned_addr learned my addr 172.21.15.59:0/1072326002 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff500a00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f81e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501e00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8ee6020 0x556ed8ee7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 -- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8ee6020 msgr2=0x556ed8ee7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8ee6020 0x556ed8ee7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 -- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 msgr2=0x556ed8f81e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f81e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 -- 172.21.15.59:0/1072326002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed8f8cb30 con 0x556ed8f8d340 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff500a00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4e80022a0 0x556ed8f81e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501e00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556ed8ee6020 0x556ed8ee7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.996+0000 7ff501400640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8f8d340 0x556ed8ebb010 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009760 tx=0x7ff4f4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.997+0000 7ff4fa000640 1 -- 172.21.15.59:0/1072326002 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4069070 con 0x556ed8f8d340 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.997+0000 7ff4fa000640 1 -- 172.21.15.59:0/1072326002 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f404f8a0 con 0x556ed8f8d340 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.997+0000 7ff4fa000640 1 -- 172.21.15.59:0/1072326002 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f404fc90 con 0x556ed8f8d340 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.997+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ed8f84d00 con 0x556ed8f8d340 2024-04-23T14:33:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.997+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed8f852c0 con 0x556ed8f8d340 2024-04-23T14:33:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:27.999+0000 7ff4fa000640 1 -- 172.21.15.59:0/1072326002 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4f4058400 con 0x556ed8f8d340 2024-04-23T14:33:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.000+0000 7ff4fa000640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff4e006b2d0 0x7ff4e006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.000+0000 7ff4fa000640 1 -- 172.21.15.59:0/1072326002 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff4f40cdc90 con 0x556ed8f8d340 2024-04-23T14:33:28.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.000+0000 7ff500a00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff4e006b2d0 0x7ff4e006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.000+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x556ed8ee1e20 0x556ed8ee2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.000+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556ed9032f70 con 0x556ed8ee1e20 2024-04-23T14:33:28.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.001+0000 7ff501e00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x556ed8ee1e20 0x556ed8ee2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.001+0000 7ff500a00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff4e006b2d0 0x7ff4e006d790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00b500 tx=0x7ff4ec008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.001+0000 7ff501e00640 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x556ed8ee1e20 0x556ed8ee2280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff501e00640 1 -- 172.21.15.59:0/1072326002 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7ff4fc011040 con 0x556ed8ee1e20 2024-04-23T14:33:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x556ed8ee1e20 msgr2=0x556ed8ee2280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x556ed8ee1e20 0x556ed8ee2280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff4e006b2d0 msgr2=0x7ff4e006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff4e006b2d0 0x7ff4e006d790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00b500 tx=0x7ff4ec008000 comp rx=0 tx=0).stop 2024-04-23T14:33:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8f8d340 msgr2=0x556ed8ebb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 --2- 172.21.15.59:0/1072326002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ed8f8d340 0x556ed8ebb010 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009760 tx=0x7ff4f4003020 comp rx=0 tx=0).stop 2024-04-23T14:33:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff501e00640 1 -- 172.21.15.59:0/1072326002 reap_dead start 2024-04-23T14:33:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 shutdown_connections 2024-04-23T14:33:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 >> 172.21.15.59:0/1072326002 conn(0x556ed8f04290 msgr2=0x556ed8ee4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 shutdown_connections 2024-04-23T14:33:28.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.002+0000 7ff5041cdf40 1 -- 172.21.15.59:0/1072326002 wait complete. 2024-04-23T14:33:28.008 INFO:teuthology.orchestra.run.smithi059.stdout:mytag 2024-04-23T14:33:28.008 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-23T14:33:28.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.113+0000 7fb6593edf40 1 Processor -- start 2024-04-23T14:33:28.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.113+0000 7fb6593edf40 1 -- start start 2024-04-23T14:33:28.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb6593edf40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 0x55c7ef6c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb6593edf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef7619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb6593edf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 0x55c7ef6e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb6593edf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7ef63ccc0 con 0x55c7ef76a070 2024-04-23T14:33:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb6593edf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c7ef640d00 con 0x55c7ef6de9a0 2024-04-23T14:33:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb6593edf40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c7ef6e0ec0 con 0x7fb63c0030f0 2024-04-23T14:33:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef7619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.114+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef7619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35960/0 (socket says 172.21.15.59:35960) 2024-04-23T14:33:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb64fe00640 1 -- 172.21.15.59:0/2966882547 learned_addr learned my addr 172.21.15.59:0/2966882547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:28.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb64fe00640 1 -- 172.21.15.59:0/2966882547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 msgr2=0x55c7ef6c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:28.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb64fe00640 1 --2- 172.21.15.59:0/2966882547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 0x55c7ef6c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb64fe00640 1 -- 172.21.15.59:0/2966882547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 msgr2=0x55c7ef6e1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:28.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb64fe00640 1 --2- 172.21.15.59:0/2966882547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 0x55c7ef6e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb64fe00640 1 -- 172.21.15.59:0/2966882547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ef76a740 con 0x55c7ef76a070 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb64fe00640 1 --2- 172.21.15.59:0/2966882547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef7619c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb64000a770 tx=0x7fb64004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c2571fe8c554d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.115+0000 7fb655200640 1 -- 172.21.15.59:0/2966882547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64000a290 con 0x55c7ef76a070 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb655200640 1 -- 172.21.15.59:0/2966882547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64004d7d0 con 0x55c7ef76a070 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb655200640 1 -- 172.21.15.59:0/2966882547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64004dbc0 con 0x55c7ef76a070 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 -- 172.21.15.59:0/2966882547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 msgr2=0x55c7ef7619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 --2- 172.21.15.59:0/2966882547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef7619c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb64000a770 tx=0x7fb64004f000 comp rx=0 tx=0).stop 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 -- 172.21.15.59:0/2966882547 shutdown_connections 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 --2- 172.21.15.59:0/2966882547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 0x55c7ef6e1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 --2- 172.21.15.59:0/2966882547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef7619c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 --2- 172.21.15.59:0/2966882547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 0x55c7ef6c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 -- 172.21.15.59:0/2966882547 >> 172.21.15.59:0/2966882547 conn(0x55c7ef6dfcc0 msgr2=0x55c7ef76ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:28.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 -- 172.21.15.59:0/2966882547 shutdown_connections 2024-04-23T14:33:28.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.116+0000 7fb6593edf40 1 -- 172.21.15.59:0/2966882547 wait complete. 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.117+0000 7fb6593edf40 1 Processor -- start 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.117+0000 7fb6593edf40 1 -- start start 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb6593edf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 0x55c7ef763800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb6593edf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef6e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb6593edf40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 0x55c7ef6e2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb6593edf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7ef67fbe0 con 0x55c7ef76a070 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb6593edf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c7ef769d10 con 0x55c7ef6de9a0 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb6593edf40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c7ef76f160 con 0x7fb63c0030f0 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef6e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef6e1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35966/0 (socket says 172.21.15.59:35966) 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.118+0000 7fb64fe00640 1 -- 172.21.15.59:0/3808544954 learned_addr learned my addr 172.21.15.59:0/3808544954 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:28.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb656600640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 0x55c7ef763800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb657000640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 0x55c7ef6e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb64fe00640 1 -- 172.21.15.59:0/3808544954 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 msgr2=0x55c7ef6e2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb64fe00640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 0x55c7ef6e2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb64fe00640 1 -- 172.21.15.59:0/3808544954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 msgr2=0x55c7ef763800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb64fe00640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7ef6de9a0 0x55c7ef763800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb64fe00640 1 -- 172.21.15.59:0/3808544954 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ef76ce20 con 0x55c7ef76a070 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb657000640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb63c0030f0 0x55c7ef6e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.119+0000 7fb64fe00640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef6e1ad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb640025f90 tx=0x7fb640003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.120+0000 7fb64ea00640 1 -- 172.21.15.59:0/3808544954 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640071070 con 0x55c7ef76a070 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.120+0000 7fb64ea00640 1 -- 172.21.15.59:0/3808544954 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64004d980 con 0x55c7ef76a070 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.120+0000 7fb64ea00640 1 -- 172.21.15.59:0/3808544954 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64004dd70 con 0x55c7ef76a070 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.120+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c7ef762a40 con 0x55c7ef76a070 2024-04-23T14:33:28.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.120+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ef763000 con 0x55c7ef76a070 2024-04-23T14:33:28.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.122+0000 7fb64ea00640 1 -- 172.21.15.59:0/3808544954 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb640074070 con 0x55c7ef76a070 2024-04-23T14:33:28.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.123+0000 7fb64ea00640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63406b2d0 0x7fb63406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.123+0000 7fb64ea00640 1 -- 172.21.15.59:0/3808544954 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb6400cf9d0 con 0x55c7ef76a070 2024-04-23T14:33:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.123+0000 7fb656600640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63406b2d0 0x7fb63406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.123+0000 7fb6593edf40 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55c7ef7520e0 0x55c7ef803e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.123+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c7ef8043c0 con 0x55c7ef7520e0 2024-04-23T14:33:28.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.123+0000 7fb657000640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55c7ef7520e0 0x55c7ef803e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:28.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb656600640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63406b2d0 0x7fb63406d790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb648001bf0 tx=0x7fb648000f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb657000640 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55c7ef7520e0 0x55c7ef803e80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:28.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb657000640 1 -- 172.21.15.59:0/3808544954 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fb65000e230 con 0x55c7ef7520e0 2024-04-23T14:33:28.126 INFO:teuthology.orchestra.run.smithi059.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-23T14:33:28.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55c7ef7520e0 msgr2=0x55c7ef803e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb6593edf40 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55c7ef7520e0 0x55c7ef803e80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:28.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63406b2d0 msgr2=0x7fb63406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb6593edf40 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63406b2d0 0x7fb63406d790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb648001bf0 tx=0x7fb648000f50 comp rx=0 tx=0).stop 2024-04-23T14:33:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.124+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 msgr2=0x55c7ef6e1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.125+0000 7fb6593edf40 1 --2- 172.21.15.59:0/3808544954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7ef76a070 0x55c7ef6e1ad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb640025f90 tx=0x7fb640003020 comp rx=0 tx=0).stop 2024-04-23T14:33:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.125+0000 7fb657000640 1 -- 172.21.15.59:0/3808544954 reap_dead start 2024-04-23T14:33:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.125+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 shutdown_connections 2024-04-23T14:33:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.125+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 >> 172.21.15.59:0/3808544954 conn(0x55c7ef6dfcc0 msgr2=0x55c7ef76df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.125+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 shutdown_connections 2024-04-23T14:33:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:28.125+0000 7fb6593edf40 1 -- 172.21.15.59:0/3808544954 wait complete. 2024-04-23T14:33:28.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:33:28.131 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:33:28.131 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:33:28.219 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:33:28.343 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:33:28.343 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:33:28.344 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:33:28.344 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:28.401 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:33:28.402 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:33:28.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:28.402 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:33:28.448 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-23T14:33:28.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:33:28.449 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:33:28.449 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:33:28.449 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:28.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:33:28.506 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:33:28.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:28.506 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:33:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:33:28.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:33:28.560 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:33:28.560 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:33:28.560 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:28.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:33:28.621 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:33:28.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:28.622 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:33:28.679 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:33:28.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:33:28.680 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:33:28.681 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:33:28.681 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:28.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:33:28.739 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:33:28.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:28.739 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:33:28.791 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:33:28.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:33:28.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:29.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.007+0000 7fa68ae00640 1 Processor -- start 2024-04-23T14:33:29.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.008+0000 7fa68ae00640 1 -- start start 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.009+0000 7fa68ae00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa68406b9d0 0x7fa6840cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.009+0000 7fa68ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa68406bdb0 0x7fa68415c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.009+0000 7fa68ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6840d0770 0x7fa684160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.009+0000 7fa68ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa6840cff90 con 0x7fa68406bdb0 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.009+0000 7fa68ae00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa684160290 con 0x7fa68406b9d0 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.009+0000 7fa68ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa68415cec0 con 0x7fa6840d0770 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa68406bdb0 0x7fa68415c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa68406bdb0 0x7fa68415c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35980/0 (socket says 172.21.15.59:35980) 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 -- 172.21.15.59:0/3085527534 learned_addr learned my addr 172.21.15.59:0/3085527534 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa688e00640 1 --2- 172.21.15.59:0/3085527534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6840d0770 0x7fa684160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa683e00640 1 --2- 172.21.15.59:0/3085527534 >> v2:172.21.15.76:3300/0 conn(0x7fa68406b9d0 0x7fa6840cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 -- 172.21.15.59:0/3085527534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6840d0770 msgr2=0x7fa684160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:29.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 --2- 172.21.15.59:0/3085527534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6840d0770 0x7fa684160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 -- 172.21.15.59:0/3085527534 >> v2:172.21.15.76:3300/0 conn(0x7fa68406b9d0 msgr2=0x7fa6840cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:29.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 --2- 172.21.15.59:0/3085527534 >> v2:172.21.15.76:3300/0 conn(0x7fa68406b9d0 0x7fa6840cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa67be00640 1 -- 172.21.15.59:0/3085527534 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68415d180 con 0x7fa68406bdb0 2024-04-23T14:33:29.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.010+0000 7fa683e00640 1 --2- 172.21.15.59:0/3085527534 >> v2:172.21.15.76:3300/0 conn(0x7fa68406b9d0 0x7fa6840cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:29.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.011+0000 7fa67be00640 1 --2- 172.21.15.59:0/3085527534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68406bdb0 0x7fa68415c980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa684073e20 tx=0x7fa66c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a855191a3865b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.012+0000 7fa683400640 1 -- 172.21.15.59:0/3085527534 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c037070 con 0x7fa68406bdb0 2024-04-23T14:33:29.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.012+0000 7fa683400640 1 -- 172.21.15.59:0/3085527534 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66c032440 con 0x7fa68406bdb0 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.012+0000 7fa683400640 1 -- 172.21.15.59:0/3085527534 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c032810 con 0x7fa68406bdb0 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.012+0000 7fa68ae00640 1 -- 172.21.15.59:0/3085527534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68406bdb0 msgr2=0x7fa68415c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.012+0000 7fa68ae00640 1 --2- 172.21.15.59:0/3085527534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68406bdb0 0x7fa68415c980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa684073e20 tx=0x7fa66c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.013+0000 7fa68ae00640 1 -- 172.21.15.59:0/3085527534 shutdown_connections 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.013+0000 7fa68ae00640 1 --2- 172.21.15.59:0/3085527534 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6840d0770 0x7fa684160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.013+0000 7fa68ae00640 1 --2- 172.21.15.59:0/3085527534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68406bdb0 0x7fa68415c980 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.013+0000 7fa68ae00640 1 --2- 172.21.15.59:0/3085527534 >> v2:172.21.15.76:3300/0 conn(0x7fa68406b9d0 0x7fa6840cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.013+0000 7fa68ae00640 1 -- 172.21.15.59:0/3085527534 >> 172.21.15.59:0/3085527534 conn(0x7fa684073430 msgr2=0x7fa68409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.013+0000 7fa68ae00640 1 -- 172.21.15.59:0/3085527534 shutdown_connections 2024-04-23T14:33:29.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.013+0000 7fa68ae00640 1 -- 172.21.15.59:0/3085527534 wait complete. 2024-04-23T14:33:29.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.014+0000 7fa68ae00640 1 Processor -- start 2024-04-23T14:33:29.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.015+0000 7fa68ae00640 1 -- start start 2024-04-23T14:33:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.015+0000 7fa68ae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68406b9d0 0x7fa684070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa68ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68406bdb0 0x7fa684149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa68ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6840d0770 0x7fa684149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa68ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa68415da20 con 0x7fa6840d0770 2024-04-23T14:33:29.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa68ae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa68415d740 con 0x7fa68406b9d0 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa68ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa6840cecb0 con 0x7fa68406bdb0 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6840d0770 0x7fa684149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6840d0770 0x7fa684149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35990/0 (socket says 172.21.15.59:35990) 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 -- 172.21.15.59:0/291277188 learned_addr learned my addr 172.21.15.59:0/291277188 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa67be00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68406bdb0 0x7fa684149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 -- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68406bdb0 msgr2=0x7fa684149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa683e00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68406b9d0 0x7fa684070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68406bdb0 0x7fa684149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 -- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68406b9d0 msgr2=0x7fa684070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68406b9d0 0x7fa684070380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa688e00640 1 -- 172.21.15.59:0/291277188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6840c9760 con 0x7fa6840d0770 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.016+0000 7fa67be00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68406bdb0 0x7fa684149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.017+0000 7fa688e00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6840d0770 0x7fa684149740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00cda0 tx=0x7fa67c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.017+0000 7fa680c00640 1 -- 172.21.15.59:0/291277188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c003b90 con 0x7fa6840d0770 2024-04-23T14:33:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.017+0000 7fa680c00640 1 -- 172.21.15.59:0/291277188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67c003d30 con 0x7fa6840d0770 2024-04-23T14:33:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.017+0000 7fa680c00640 1 -- 172.21.15.59:0/291277188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c004240 con 0x7fa6840d0770 2024-04-23T14:33:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.017+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6840c9a40 con 0x7fa6840d0770 2024-04-23T14:33:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.017+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6840708c0 con 0x7fa6840d0770 2024-04-23T14:33:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.019+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa684068a80 con 0x7fa6840d0770 2024-04-23T14:33:29.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.020+0000 7fa680c00640 1 -- 172.21.15.59:0/291277188 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa67c0043e0 con 0x7fa6840d0770 2024-04-23T14:33:29.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.020+0000 7fa680c00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66006c920 0x7fa66006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.021+0000 7fa680c00640 1 -- 172.21.15.59:0/291277188 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa67c08c6f0 con 0x7fa6840d0770 2024-04-23T14:33:29.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.021+0000 7fa683e00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66006c920 0x7fa66006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.022+0000 7fa683e00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66006c920 0x7fa66006ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b7c0 tx=0x7fa674008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.025+0000 7fa680c00640 1 -- 172.21.15.59:0/291277188 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa67c05aa80 con 0x7fa6840d0770 2024-04-23T14:33:29.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.269+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6840ce090 con 0x7fa6840d0770 2024-04-23T14:33:29.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.270+0000 7fa680c00640 1 -- 172.21.15.59:0/291277188 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7fa67c05a380 con 0x7fa6840d0770 2024-04-23T14:33:29.271 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:29.271 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:33:29.272 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:33:29.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66006c920 msgr2=0x7fa66006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66006c920 0x7fa66006ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b7c0 tx=0x7fa674008040 comp rx=0 tx=0).stop 2024-04-23T14:33:29.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6840d0770 msgr2=0x7fa684149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6840d0770 0x7fa684149740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00cda0 tx=0x7fa67c00a660 comp rx=0 tx=0).stop 2024-04-23T14:33:29.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 shutdown_connections 2024-04-23T14:33:29.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66006c920 0x7fa66006ede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68406bdb0 0x7fa684149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6840d0770 0x7fa684149740 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 --2- 172.21.15.59:0/291277188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68406b9d0 0x7fa684070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 >> 172.21.15.59:0/291277188 conn(0x7fa684073430 msgr2=0x7fa6840aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:29.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 shutdown_connections 2024-04-23T14:33:29.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.273+0000 7fa68ae00640 1 -- 172.21.15.59:0/291277188 wait complete. 2024-04-23T14:33:29.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.510+0000 7ff43e200640 1 Processor -- start 2024-04-23T14:33:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.510+0000 7ff43e200640 1 -- start start 2024-04-23T14:33:29.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff43e200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4380cd760 0x7ff4380b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff43e200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380b4690 0x7ff4380cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff43e200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff438160210 con 0x7ff43815c3d0 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff43e200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff4381604f0 con 0x7ff4380cd760 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff43e200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff4381607d0 con 0x7ff4380b4690 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff437e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4380cd760 0x7ff4380b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff437e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4380cd760 0x7ff4380b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36002/0 (socket says 172.21.15.59:36002) 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff437e00640 1 -- 172.21.15.59:0/3753066446 learned_addr learned my addr 172.21.15.59:0/3753066446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff437e00640 1 -- 172.21.15.59:0/3753066446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380b4690 msgr2=0x7ff4380cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff43d800640 1 --2- 172.21.15.59:0/3753066446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380b4690 0x7ff4380cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.512+0000 7ff437e00640 1 --2- 172.21.15.59:0/3753066446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380b4690 0x7ff4380cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.513+0000 7ff437e00640 1 -- 172.21.15.59:0/3753066446 >> v1:172.21.15.76:6789/0 conn(0x7ff43815c3d0 legacy=0x7ff4380cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:29.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.513+0000 7ff437e00640 1 -- 172.21.15.59:0/3753066446 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438160ab0 con 0x7ff4380cd760 2024-04-23T14:33:29.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.513+0000 7ff43d800640 1 --2- 172.21.15.59:0/3753066446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380b4690 0x7ff4380cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.514+0000 7ff437e00640 1 --2- 172.21.15.59:0/3753066446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4380cd760 0x7ff4380b4150 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff424009870 tx=0x7ff42402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572c7fec78545fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.514+0000 7ff437400640 1 -- 172.21.15.59:0/3753066446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424037070 con 0x7ff4380cd760 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.514+0000 7ff437400640 1 -- 172.21.15.59:0/3753066446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff424032440 con 0x7ff4380cd760 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.514+0000 7ff437400640 1 -- 172.21.15.59:0/3753066446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424032810 con 0x7ff4380cd760 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.514+0000 7ff43e200640 1 -- 172.21.15.59:0/3753066446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4380cd760 msgr2=0x7ff4380b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.514+0000 7ff43e200640 1 --2- 172.21.15.59:0/3753066446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4380cd760 0x7ff4380b4150 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff424009870 tx=0x7ff42402a210 comp rx=0 tx=0).stop 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.515+0000 7ff43e200640 1 -- 172.21.15.59:0/3753066446 shutdown_connections 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.515+0000 7ff43e200640 1 --2- 172.21.15.59:0/3753066446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380b4690 0x7ff4380cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.515+0000 7ff43e200640 1 --2- 172.21.15.59:0/3753066446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4380cd760 0x7ff4380b4150 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.515+0000 7ff43e200640 1 -- 172.21.15.59:0/3753066446 >> 172.21.15.59:0/3753066446 conn(0x7ff4380b21a0 msgr2=0x7ff4380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.515+0000 7ff43e200640 1 -- 172.21.15.59:0/3753066446 shutdown_connections 2024-04-23T14:33:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.515+0000 7ff43e200640 1 -- 172.21.15.59:0/3753066446 wait complete. 2024-04-23T14:33:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.516+0000 7ff43e200640 1 Processor -- start 2024-04-23T14:33:29.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.517+0000 7ff43e200640 1 -- start start 2024-04-23T14:33:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43e200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4380b4690 0x7ff4380c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43e200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380cd760 0x7ff43806e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43e200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff43815c3d0 0x7ff43806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43e200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff43815c7b0 con 0x7ff43815c3d0 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43e200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380b4b00 con 0x7ff4380b4690 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43e200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff43815c930 con 0x7ff4380cd760 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff43815c3d0 0x7ff43806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff43815c3d0 0x7ff43806eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36010/0 (socket says 172.21.15.59:36010) 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43d800640 1 -- 172.21.15.59:0/3994235872 learned_addr learned my addr 172.21.15.59:0/3994235872 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff43ce00640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4380b4690 0x7ff4380c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.518+0000 7ff437e00640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380cd760 0x7ff43806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43d800640 1 -- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380cd760 msgr2=0x7ff43806e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43d800640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380cd760 0x7ff43806e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43d800640 1 -- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4380b4690 msgr2=0x7ff4380c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43d800640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4380b4690 0x7ff4380c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43d800640 1 -- 172.21.15.59:0/3994235872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438146700 con 0x7ff43815c3d0 2024-04-23T14:33:29.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43d800640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff43815c3d0 0x7ff43806eb60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff43000cef0 tx=0x7ff43000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff434c00640 1 -- 172.21.15.59:0/3994235872 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43001a070 con 0x7ff43815c3d0 2024-04-23T14:33:29.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff434c00640 1 -- 172.21.15.59:0/3994235872 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43000d860 con 0x7ff43815c3d0 2024-04-23T14:33:29.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff434c00640 1 -- 172.21.15.59:0/3994235872 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43000dc50 con 0x7ff43815c3d0 2024-04-23T14:33:29.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4381469c0 con 0x7ff43815c3d0 2024-04-23T14:33:29.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.519+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff438146f80 con 0x7ff43815c3d0 2024-04-23T14:33:29.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.521+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43804fd90 con 0x7ff43815c3d0 2024-04-23T14:33:29.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.522+0000 7ff434c00640 1 -- 172.21.15.59:0/3994235872 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff43001d070 con 0x7ff43815c3d0 2024-04-23T14:33:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.523+0000 7ff434c00640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff41806c8f0 0x7ff41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.523+0000 7ff434c00640 1 -- 172.21.15.59:0/3994235872 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff43008c550 con 0x7ff43815c3d0 2024-04-23T14:33:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.524+0000 7ff43ce00640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff41806c8f0 0x7ff41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.524+0000 7ff43ce00640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff41806c8f0 0x7ff41806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff42c006fd0 tx=0x7ff42c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.529+0000 7ff434c00640 1 -- 172.21.15.59:0/3994235872 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff43005a910 con 0x7ff43815c3d0 2024-04-23T14:33:29.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.733+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff43806cbe0 con 0x7ff43815c3d0 2024-04-23T14:33:29.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.735+0000 7ff434c00640 1 -- 172.21.15.59:0/3994235872 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7ff43005a210 con 0x7ff43815c3d0 2024-04-23T14:33:29.736 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:29.737 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:50.389436+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff41806c8f0 msgr2=0x7ff41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff41806c8f0 0x7ff41806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff42c006fd0 tx=0x7ff42c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff43815c3d0 msgr2=0x7ff43806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff43815c3d0 0x7ff43806eb60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff43000cef0 tx=0x7ff43000a710 comp rx=0 tx=0).stop 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 shutdown_connections 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff41806c8f0 0x7ff41806edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4380cd760 0x7ff43806e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff43815c3d0 0x7ff43806eb60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 --2- 172.21.15.59:0/3994235872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4380b4690 0x7ff4380c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 >> 172.21.15.59:0/3994235872 conn(0x7ff4380b21a0 msgr2=0x7ff4380cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 shutdown_connections 2024-04-23T14:33:29.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.738+0000 7ff43e200640 1 -- 172.21.15.59:0/3994235872 wait complete. 2024-04-23T14:33:29.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:32:46.759888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:32:47.810840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:29.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:29.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.968+0000 7fda85800640 1 Processor -- start 2024-04-23T14:33:29.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.968+0000 7fda85800640 1 -- start start 2024-04-23T14:33:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.969+0000 7fda85800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda801567e0 0x7fda80156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.969+0000 7fda85800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fda8015d810 0x7fda800d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.969+0000 7fda85800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8015cab0 con 0x7fda800d0b40 2024-04-23T14:33:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.969+0000 7fda85800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fda800d0f90 con 0x7fda8015d810 2024-04-23T14:33:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.969+0000 7fda85800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fda80160e40 con 0x7fda801567e0 2024-04-23T14:33:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fda8015d810 0x7fda800d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fda8015d810 0x7fda800d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36026/0 (socket says 172.21.15.59:36026) 2024-04-23T14:33:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7f400640 1 -- 172.21.15.59:0/2088628358 learned_addr learned my addr 172.21.15.59:0/2088628358 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7fe00640 1 --2- 172.21.15.59:0/2088628358 >> v2:172.21.15.76:3300/0 conn(0x7fda801567e0 0x7fda80156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7f400640 1 -- 172.21.15.59:0/2088628358 >> v2:172.21.15.76:3300/0 conn(0x7fda801567e0 msgr2=0x7fda80156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7f400640 1 --2- 172.21.15.59:0/2088628358 >> v2:172.21.15.76:3300/0 conn(0x7fda801567e0 0x7fda80156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7f400640 1 -- 172.21.15.59:0/2088628358 >> v1:172.21.15.76:6789/0 conn(0x7fda800d0b40 legacy=0x7fda80160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.970+0000 7fda7f400640 1 -- 172.21.15.59:0/2088628358 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8015cd30 con 0x7fda8015d810 2024-04-23T14:33:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.971+0000 7fda7f400640 1 --2- 172.21.15.59:0/2088628358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda800d0600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fda800b6360 tx=0x7fda6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f70626558dd4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.972+0000 7fda7ea00640 1 -- 172.21.15.59:0/2088628358 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c037070 con 0x7fda8015d810 2024-04-23T14:33:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.972+0000 7fda7ea00640 1 -- 172.21.15.59:0/2088628358 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6c032440 con 0x7fda8015d810 2024-04-23T14:33:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.972+0000 7fda7ea00640 1 -- 172.21.15.59:0/2088628358 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c032810 con 0x7fda8015d810 2024-04-23T14:33:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 -- 172.21.15.59:0/2088628358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 msgr2=0x7fda800d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 --2- 172.21.15.59:0/2088628358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda800d0600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fda800b6360 tx=0x7fda6c02a270 comp rx=0 tx=0).stop 2024-04-23T14:33:29.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 -- 172.21.15.59:0/2088628358 shutdown_connections 2024-04-23T14:33:29.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 --2- 172.21.15.59:0/2088628358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda800d0600 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 --2- 172.21.15.59:0/2088628358 >> v2:172.21.15.76:3300/0 conn(0x7fda801567e0 0x7fda80156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 -- 172.21.15.59:0/2088628358 >> 172.21.15.59:0/2088628358 conn(0x7fda8006bdf0 msgr2=0x7fda8006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:29.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 -- 172.21.15.59:0/2088628358 shutdown_connections 2024-04-23T14:33:29.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.973+0000 7fda85800640 1 -- 172.21.15.59:0/2088628358 wait complete. 2024-04-23T14:33:29.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.975+0000 7fda85800640 1 Processor -- start 2024-04-23T14:33:29.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.975+0000 7fda85800640 1 -- start start 2024-04-23T14:33:29.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda85800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda800d0b40 0x7fda8014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda85800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda801567e0 0x7fda801f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda85800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda80158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda85800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda80160820 con 0x7fda8015d810 2024-04-23T14:33:29.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda85800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda8015d2f0 con 0x7fda801567e0 2024-04-23T14:33:29.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda85800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda801609a0 con 0x7fda800d0b40 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda84e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda80158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda84e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda80158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36032/0 (socket says 172.21.15.59:36032) 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda84e00640 1 -- 172.21.15.59:0/2028733808 learned_addr learned my addr 172.21.15.59:0/2028733808 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda7fe00640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda800d0b40 0x7fda8014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda7f400640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda801567e0 0x7fda801f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda84e00640 1 -- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda800d0b40 msgr2=0x7fda8014eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda84e00640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda800d0b40 0x7fda8014eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.976+0000 7fda84e00640 1 -- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda801567e0 msgr2=0x7fda801f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda84e00640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda801567e0 0x7fda801f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda84e00640 1 -- 172.21.15.59:0/2028733808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda801518c0 con 0x7fda8015d810 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda84e00640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda80158d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fda7800c760 tx=0x7fda7800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda77e00640 1 -- 172.21.15.59:0/2028733808 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78016020 con 0x7fda8015d810 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda77e00640 1 -- 172.21.15.59:0/2028733808 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda78006d50 con 0x7fda8015d810 2024-04-23T14:33:29.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda77e00640 1 -- 172.21.15.59:0/2028733808 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78003bb0 con 0x7fda8015d810 2024-04-23T14:33:29.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda80151ba0 con 0x7fda8015d810 2024-04-23T14:33:29.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.977+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8014f080 con 0x7fda8015d810 2024-04-23T14:33:29.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.979+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8004fd90 con 0x7fda8015d810 2024-04-23T14:33:29.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.979+0000 7fda77e00640 1 -- 172.21.15.59:0/2028733808 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda780050d0 con 0x7fda8015d810 2024-04-23T14:33:29.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.981+0000 7fda77e00640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda6006c8f0 0x7fda6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:29.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.981+0000 7fda77e00640 1 -- 172.21.15.59:0/2028733808 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fda7808b740 con 0x7fda8015d810 2024-04-23T14:33:29.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.981+0000 7fda7fe00640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda6006c8f0 0x7fda6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:29.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.982+0000 7fda7fe00640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda6006c8f0 0x7fda6006edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fda70006fd0 tx=0x7fda70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:29.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:29.986+0000 7fda77e00640 1 -- 172.21.15.59:0/2028733808 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda78020020 con 0x7fda8015d810 2024-04-23T14:33:30.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.194+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda8015c250 con 0x7fda8015d810 2024-04-23T14:33:30.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.195+0000 7fda77e00640 1 -- 172.21.15.59:0/2028733808 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fda8015cab0 con 0x7fda8015d810 2024-04-23T14:33:30.196 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:33:30.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda6006c8f0 msgr2=0x7fda6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda6006c8f0 0x7fda6006edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fda70006fd0 tx=0x7fda70008040 comp rx=0 tx=0).stop 2024-04-23T14:33:30.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 msgr2=0x7fda80158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda80158d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fda7800c760 tx=0x7fda7800cb60 comp rx=0 tx=0).stop 2024-04-23T14:33:30.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 shutdown_connections 2024-04-23T14:33:30.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda6006c8f0 0x7fda6006edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda8015d810 0x7fda80158d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda801567e0 0x7fda801f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 --2- 172.21.15.59:0/2028733808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda800d0b40 0x7fda8014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 >> 172.21.15.59:0/2028733808 conn(0x7fda8006bdf0 msgr2=0x7fda80072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:30.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.198+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 shutdown_connections 2024-04-23T14:33:30.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.199+0000 7fda85800640 1 -- 172.21.15.59:0/2028733808 wait complete. 2024-04-23T14:33:30.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:33:30.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.431+0000 7f5b96e00640 1 Processor -- start 2024-04-23T14:33:30.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.432+0000 7f5b96e00640 1 -- start start 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.433+0000 7f5b96e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b9015ed20 0x7f5b900d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.433+0000 7f5b96e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5b900b3600 0x7f5b900d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.433+0000 7f5b96e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b900b5da0 con 0x7f5b900b3f10 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b96e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5b9015f100 con 0x7f5b900b3600 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b96e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5b900d0a20 con 0x7f5b9015ed20 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b95000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5b900b3600 0x7f5b900d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b95000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5b900b3600 0x7f5b900d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44092/0 (socket says 172.21.15.59:44092) 2024-04-23T14:33:30.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b95000640 1 -- 172.21.15.59:0/3981419660 learned_addr learned my addr 172.21.15.59:0/3981419660 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:30.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b95a00640 1 --2- 172.21.15.59:0/3981419660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b9015ed20 0x7f5b900d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b95000640 1 -- 172.21.15.59:0/3981419660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b9015ed20 msgr2=0x7f5b900d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:30.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.434+0000 7f5b95000640 1 --2- 172.21.15.59:0/3981419660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b9015ed20 0x7f5b900d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.435+0000 7f5b95000640 1 -- 172.21.15.59:0/3981419660 >> v1:172.21.15.76:6789/0 conn(0x7f5b900b3f10 legacy=0x7f5b900b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.435+0000 7f5b95000640 1 -- 172.21.15.59:0/3981419660 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b900d2ed0 con 0x7f5b900b3600 2024-04-23T14:33:30.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.436+0000 7f5b95000640 1 --2- 172.21.15.59:0/3981419660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b900b3600 0x7f5b900d13e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009870 tx=0x7f5b7c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2766e3ad0a41096 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.436+0000 7f5b8fe00640 1 -- 172.21.15.59:0/3981419660 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c037070 con 0x7f5b900b3600 2024-04-23T14:33:30.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b8fe00640 1 -- 172.21.15.59:0/3981419660 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c032440 con 0x7f5b900b3600 2024-04-23T14:33:30.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b8fe00640 1 -- 172.21.15.59:0/3981419660 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c032810 con 0x7f5b900b3600 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b96e00640 1 -- 172.21.15.59:0/3981419660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b900b3600 msgr2=0x7f5b900d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b96e00640 1 --2- 172.21.15.59:0/3981419660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b900b3600 0x7f5b900d13e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009870 tx=0x7f5b7c02a130 comp rx=0 tx=0).stop 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b96e00640 1 -- 172.21.15.59:0/3981419660 shutdown_connections 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b96e00640 1 --2- 172.21.15.59:0/3981419660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b900b3600 0x7f5b900d13e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b96e00640 1 --2- 172.21.15.59:0/3981419660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b9015ed20 0x7f5b900d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b96e00640 1 -- 172.21.15.59:0/3981419660 >> 172.21.15.59:0/3981419660 conn(0x7f5b9008bf90 msgr2=0x7f5b900d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.437+0000 7f5b96e00640 1 -- 172.21.15.59:0/3981419660 shutdown_connections 2024-04-23T14:33:30.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.438+0000 7f5b96e00640 1 -- 172.21.15.59:0/3981419660 wait complete. 2024-04-23T14:33:30.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.439+0000 7f5b96e00640 1 Processor -- start 2024-04-23T14:33:30.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.439+0000 7f5b96e00640 1 -- start start 2024-04-23T14:33:30.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.440+0000 7f5b96e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b900b3600 0x7f5b9009ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.440+0000 7f5b96e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b900b3f10 0x7f5b90151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9015ed20 0x7f5b900cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b900d0290 con 0x7f5b9015ed20 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5b900d3490 con 0x7f5b900b3f10 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5b900d0410 con 0x7f5b900b3600 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9015ed20 0x7f5b900cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9015ed20 0x7f5b900cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44098/0 (socket says 172.21.15.59:44098) 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 -- 172.21.15.59:0/2025285541 learned_addr learned my addr 172.21.15.59:0/2025285541 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b95a00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b900b3600 0x7f5b9009ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b95000640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b900b3f10 0x7f5b90151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 -- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b900b3600 msgr2=0x7f5b9009ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b900b3600 0x7f5b9009ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 -- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b900b3f10 msgr2=0x7f5b90151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b900b3f10 0x7f5b90151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.441+0000 7f5b96400640 1 -- 172.21.15.59:0/2025285541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b900a16d0 con 0x7f5b9015ed20 2024-04-23T14:33:30.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.442+0000 7f5b96400640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9015ed20 0x7f5b900cf520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c7a0 tx=0x7f5b8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.442+0000 7f5b8d600640 1 -- 172.21.15.59:0/2025285541 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88003b70 con 0x7f5b9015ed20 2024-04-23T14:33:30.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.442+0000 7f5b8d600640 1 -- 172.21.15.59:0/2025285541 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b88003d30 con 0x7f5b9015ed20 2024-04-23T14:33:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.442+0000 7f5b8d600640 1 -- 172.21.15.59:0/2025285541 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800aa90 con 0x7f5b9015ed20 2024-04-23T14:33:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.442+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b900a19b0 con 0x7f5b9015ed20 2024-04-23T14:33:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.442+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b900a1f70 con 0x7f5b9015ed20 2024-04-23T14:33:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.443+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b90068de0 con 0x7f5b9015ed20 2024-04-23T14:33:30.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.444+0000 7f5b8d600640 1 -- 172.21.15.59:0/2025285541 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b8800ac30 con 0x7f5b9015ed20 2024-04-23T14:33:30.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.445+0000 7f5b8d600640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5b7006c8f0 0x7f5b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.446+0000 7f5b8d600640 1 -- 172.21.15.59:0/2025285541 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5b8808bb00 con 0x7f5b9015ed20 2024-04-23T14:33:30.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.446+0000 7f5b95a00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5b7006c8f0 0x7f5b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.447+0000 7f5b95a00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5b7006c8f0 0x7f5b7006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b84006fd0 tx=0x7f5b84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.450+0000 7f5b8d600640 1 -- 172.21.15.59:0/2025285541 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b90068de0 con 0x7f5b9015ed20 2024-04-23T14:33:30.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.652+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b9004fd90 con 0x7f5b9015ed20 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.653+0000 7f5b8d600640 1 -- 172.21.15.59:0/2025285541 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f5b8808be20 con 0x7f5b9015ed20 2024-04-23T14:33:30.654 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:33:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5b7006c8f0 msgr2=0x7f5b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5b7006c8f0 0x7f5b7006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b84006fd0 tx=0x7f5b84008040 comp rx=0 tx=0).stop 2024-04-23T14:33:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9015ed20 msgr2=0x7f5b900cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9015ed20 0x7f5b900cf520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c7a0 tx=0x7f5b8800cc60 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 shutdown_connections 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5b7006c8f0 0x7f5b7006edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9015ed20 0x7f5b900cf520 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b900b3f10 0x7f5b90151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 --2- 172.21.15.59:0/2025285541 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b900b3600 0x7f5b9009ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 >> 172.21.15.59:0/2025285541 conn(0x7f5b9008bf90 msgr2=0x7f5b900b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 shutdown_connections 2024-04-23T14:33:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.656+0000 7f5b96e00640 1 -- 172.21.15.59:0/2025285541 wait complete. 2024-04-23T14:33:30.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:33:30.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.888+0000 7f133b200640 1 Processor -- start 2024-04-23T14:33:30.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.888+0000 7f133b200640 1 -- start start 2024-04-23T14:33:30.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.889+0000 7f133b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f133c1567e0 0x7f133c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.889+0000 7f133b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 0x7f133c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.889+0000 7f133b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c15cab0 con 0x7f133c15d810 2024-04-23T14:33:30.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.889+0000 7f133b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f133c160e40 con 0x7f133c1567e0 2024-04-23T14:33:30.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.889+0000 7f133b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f133c15cd30 con 0x7f133c0d0d10 2024-04-23T14:33:30.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.889+0000 7f1333e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f133c15d810 0x7f133c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54622/0 (socket says 172.21.15.59:54622) 2024-04-23T14:33:30.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.889+0000 7f1333e00640 1 -- 172.21.15.59:0/4222029896 learned_addr learned my addr 172.21.15.59:0/4222029896 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:30.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f133a800640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 0x7f133c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339e00640 1 --2- 172.21.15.59:0/4222029896 >> v2:172.21.15.76:3300/0 conn(0x7f133c1567e0 0x7f133c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798585711 0 0) 0x7f1324001110 con 0x7f133c15d810 2024-04-23T14:33:30.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1320003520 con 0x7f133c15d810 2024-04-23T14:33:30.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3821974702 0 0) 0x7f1320003520 con 0x7f133c15d810 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 msgr2=0x7f133c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 0x7f133c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 >> v2:172.21.15.76:3300/0 conn(0x7f133c1567e0 msgr2=0x7f133c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 --2- 172.21.15.59:0/4222029896 >> v2:172.21.15.76:3300/0 conn(0x7f133c1567e0 0x7f133c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c15d010 con 0x7f133c15d810 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1324001840 con 0x7f133c15d810 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 >> v1:172.21.15.59:6789/0 conn(0x7f133c15d810 legacy=0x7f133c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.890+0000 7f1339400640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 0x7f13200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1339400640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f1320004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1339400640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1320004d40 0x7f13200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f133c160e40 con 0x7f1320003f30 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f133c15cd30 con 0x7f1320004d40 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f132000b9c0 con 0x7f133c0d0d10 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1333e00640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f1320004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1339e00640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 0x7f13200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f133a800640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1320004d40 0x7f13200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1333e00640 1 -- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 msgr2=0x7f13200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1333e00640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0d0d10 0x7f13200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1333e00640 1 -- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1320004d40 msgr2=0x7f13200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1333e00640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1320004d40 0x7f13200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.891+0000 7f1333e00640 1 -- 172.21.15.59:0/4222029896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1320003520 con 0x7f1320003f30 2024-04-23T14:33:30.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.893+0000 7f1333e00640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f1320004800 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f133c15dc20 tx=0x7f1320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99906bc94b1c744 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.893+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132400c0e0 con 0x7f1320003f30 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.893+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1324034070 con 0x7f1320003f30 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.893+0000 7f1339400640 1 -- 172.21.15.59:0/4222029896 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324005b10 con 0x7f1320003f30 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.893+0000 7f133b200640 1 -- 172.21.15.59:0/4222029896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 msgr2=0x7f1320004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.893+0000 7f133b200640 1 --2- 172.21.15.59:0/4222029896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f1320004800 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f133c15dc20 tx=0x7f1320005830 comp rx=0 tx=0).stop 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.893+0000 7f133a800640 1 -- 172.21.15.59:0/4222029896 reap_dead start 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.894+0000 7f133b200640 1 -- 172.21.15.59:0/4222029896 shutdown_connections 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.894+0000 7f133b200640 1 -- 172.21.15.59:0/4222029896 >> 172.21.15.59:0/4222029896 conn(0x7f133c06bdf0 msgr2=0x7f133c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.894+0000 7f133b200640 1 -- 172.21.15.59:0/4222029896 shutdown_connections 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.894+0000 7f133b200640 1 -- 172.21.15.59:0/4222029896 wait complete. 2024-04-23T14:33:30.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.896+0000 7f133b200640 1 Processor -- start 2024-04-23T14:33:30.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.896+0000 7f133b200640 1 -- start start 2024-04-23T14:33:30.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.897+0000 7f133b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f133c150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f133b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0b2820 0x7f133c0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f133b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c0b1910 0x7f133c1f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f133b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f133c160290 con 0x7f1320003f30 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f133b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f133c160110 con 0x7f133c0b1910 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f133b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f133c160410 con 0x7f133c0b2820 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f133c150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f133c150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44120/0 (socket says 172.21.15.59:44120) 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 -- 172.21.15.59:0/1181250871 learned_addr learned my addr 172.21.15.59:0/1181250871 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 -- 172.21.15.59:0/1181250871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0b2820 msgr2=0x7f133c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1333e00640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0b2820 0x7f133c0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0b2820 0x7f133c0b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 -- 172.21.15.59:0/1181250871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c0b1910 msgr2=0x7f133c1f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c0b1910 0x7f133c1f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.898+0000 7f1339e00640 1 -- 172.21.15.59:0/1181250871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c150f10 con 0x7f1320003f30 2024-04-23T14:33:30.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.899+0000 7f1339e00640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f133c150630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f132c011660 tx=0x7f132c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.899+0000 7f1332000640 1 -- 172.21.15.59:0/1181250871 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c00f940 con 0x7f1320003f30 2024-04-23T14:33:30.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.899+0000 7f1332000640 1 -- 172.21.15.59:0/1181250871 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132c00fae0 con 0x7f1320003f30 2024-04-23T14:33:30.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.899+0000 7f1332000640 1 -- 172.21.15.59:0/1181250871 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c00fe90 con 0x7f1320003f30 2024-04-23T14:33:30.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.899+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1511f0 con 0x7f1320003f30 2024-04-23T14:33:30.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.899+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c150b70 con 0x7f1320003f30 2024-04-23T14:33:30.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.900+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c0b1d30 con 0x7f1320003f30 2024-04-23T14:33:30.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.901+0000 7f1332000640 1 -- 172.21.15.59:0/1181250871 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f132c015460 con 0x7f1320003f30 2024-04-23T14:33:30.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.902+0000 7f1332000640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f131806ca90 0x7f131806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:30.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.902+0000 7f1332000640 1 -- 172.21.15.59:0/1181250871 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f132c08c590 con 0x7f1320003f30 2024-04-23T14:33:30.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.903+0000 7f1333e00640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f131806ca90 0x7f131806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:30.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.904+0000 7f1333e00640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f131806ca90 0x7f131806ef50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f133c0afdb0 tx=0x7f1324009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:30.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:30.907+0000 7f1332000640 1 -- 172.21.15.59:0/1181250871 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f132c019030 con 0x7f1320003f30 2024-04-23T14:33:31.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.112+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f133c0b5150 con 0x7f1320003f30 2024-04-23T14:33:31.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.113+0000 7f1332000640 1 -- 172.21.15.59:0/1181250871 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f132c05a920 con 0x7f1320003f30 2024-04-23T14:33:31.114 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:33:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f131806ca90 msgr2=0x7f131806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f131806ca90 0x7f131806ef50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f133c0afdb0 tx=0x7f1324009030 comp rx=0 tx=0).stop 2024-04-23T14:33:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 msgr2=0x7f133c150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f133c150630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f132c011660 tx=0x7f132c011b20 comp rx=0 tx=0).stop 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 shutdown_connections 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f131806ca90 0x7f131806ef50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f133c0b2820 0x7f133c0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c0b1910 0x7f133c1f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 --2- 172.21.15.59:0/1181250871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1320003f30 0x7f133c150630 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 >> 172.21.15.59:0/1181250871 conn(0x7f133c06bdf0 msgr2=0x7f133c15dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 shutdown_connections 2024-04-23T14:33:31.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.116+0000 7f133b200640 1 -- 172.21.15.59:0/1181250871 wait complete. 2024-04-23T14:33:31.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:31.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.346+0000 7f226e400640 1 Processor -- start 2024-04-23T14:33:31.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.347+0000 7f226e400640 1 -- start start 2024-04-23T14:33:31.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.348+0000 7f226e400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226815e7c0 0x7f22680d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226e400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f22680d0d90 0x7f22680d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226e400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22680cf030 con 0x7f22680d2630 2024-04-23T14:33:31.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226e400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f22680b7000 con 0x7f22680d0d90 2024-04-23T14:33:31.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226e400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22680cf620 con 0x7f226815e7c0 2024-04-23T14:33:31.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226d000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226815e7c0 0x7f22680d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226d000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226815e7c0 0x7f22680d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:47426/0 (socket says 172.21.15.59:47426) 2024-04-23T14:33:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226d000640 1 -- 172.21.15.59:0/991874293 learned_addr learned my addr 172.21.15.59:0/991874293 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.349+0000 7f226da00640 1 --2- 172.21.15.59:0/991874293 >> v2:172.21.15.76:3300/0 conn(0x7f22680d0d90 0x7f22680d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.350+0000 7f226d000640 1 -- 172.21.15.59:0/991874293 >> v2:172.21.15.76:3300/0 conn(0x7f22680d0d90 msgr2=0x7f22680d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.350+0000 7f226d000640 1 --2- 172.21.15.59:0/991874293 >> v2:172.21.15.76:3300/0 conn(0x7f22680d0d90 0x7f22680d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.350+0000 7f226d000640 1 -- 172.21.15.59:0/991874293 >> v1:172.21.15.76:6789/0 conn(0x7f22680d2630 legacy=0x7f22680b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.350+0000 7f226d000640 1 -- 172.21.15.59:0/991874293 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22680b6a40 con 0x7f226815e7c0 2024-04-23T14:33:31.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.351+0000 7f226d000640 1 --2- 172.21.15.59:0/991874293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226815e7c0 0x7f22680d20f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2258009870 tx=0x7f225802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985435d29fe4efca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.352+0000 7f2267400640 1 -- 172.21.15.59:0/991874293 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258037070 con 0x7f226815e7c0 2024-04-23T14:33:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.352+0000 7f2267400640 1 -- 172.21.15.59:0/991874293 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2258032440 con 0x7f226815e7c0 2024-04-23T14:33:31.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.352+0000 7f2267400640 1 -- 172.21.15.59:0/991874293 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258032810 con 0x7f226815e7c0 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 -- 172.21.15.59:0/991874293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226815e7c0 msgr2=0x7f22680d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 --2- 172.21.15.59:0/991874293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226815e7c0 0x7f22680d20f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2258009870 tx=0x7f225802a250 comp rx=0 tx=0).stop 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 -- 172.21.15.59:0/991874293 shutdown_connections 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 --2- 172.21.15.59:0/991874293 >> v2:172.21.15.76:3300/0 conn(0x7f22680d0d90 0x7f22680d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 --2- 172.21.15.59:0/991874293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226815e7c0 0x7f22680d20f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 -- 172.21.15.59:0/991874293 >> 172.21.15.59:0/991874293 conn(0x7f226809d920 msgr2=0x7f226808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 -- 172.21.15.59:0/991874293 shutdown_connections 2024-04-23T14:33:31.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.353+0000 7f226e400640 1 -- 172.21.15.59:0/991874293 wait complete. 2024-04-23T14:33:31.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.356+0000 7f226e400640 1 Processor -- start 2024-04-23T14:33:31.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.356+0000 7f226e400640 1 -- start start 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.357+0000 7f226e400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22680d0d90 0x7f22680b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.357+0000 7f226e400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22680d2630 0x7f22680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.357+0000 7f226e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226815e7c0 0x7f22680ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.357+0000 7f226e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22680b6700 con 0x7f226815e7c0 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.357+0000 7f226e400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22680cf2b0 con 0x7f22680d2630 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.357+0000 7f226e400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22680d1a80 con 0x7f22680d0d90 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226815e7c0 0x7f22680ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226815e7c0 0x7f22680ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44128/0 (socket says 172.21.15.59:44128) 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 -- 172.21.15.59:0/2637163922 learned_addr learned my addr 172.21.15.59:0/2637163922 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226d000640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22680d0d90 0x7f22680b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f2267e00640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22680d2630 0x7f22680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 -- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22680d0d90 msgr2=0x7f22680b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22680d0d90 0x7f22680b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 -- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22680d2630 msgr2=0x7f22680ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22680d2630 0x7f22680ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.358+0000 7f226da00640 1 -- 172.21.15.59:0/2637163922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22680c8bd0 con 0x7f226815e7c0 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.359+0000 7f226da00640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226815e7c0 0x7f22680ca8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f226000e930 tx=0x7f226000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.359+0000 7f2264c00640 1 -- 172.21.15.59:0/2637163922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260017070 con 0x7f226815e7c0 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.359+0000 7f2264c00640 1 -- 172.21.15.59:0/2637163922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226000c980 con 0x7f226815e7c0 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.359+0000 7f2264c00640 1 -- 172.21.15.59:0/2637163922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226000cd70 con 0x7f226815e7c0 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.359+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22680c8e70 con 0x7f226815e7c0 2024-04-23T14:33:31.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.359+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22680c9710 con 0x7f226815e7c0 2024-04-23T14:33:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.361+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22680c9260 con 0x7f226815e7c0 2024-04-23T14:33:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.362+0000 7f2264c00640 1 -- 172.21.15.59:0/2637163922 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f226001a070 con 0x7f226815e7c0 2024-04-23T14:33:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.363+0000 7f2264c00640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224806c8f0 0x7f224806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.363+0000 7f2264c00640 1 -- 172.21.15.59:0/2637163922 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f226008baa0 con 0x7f226815e7c0 2024-04-23T14:33:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.363+0000 7f226d000640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224806c8f0 0x7f224806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.364+0000 7f226d000640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224806c8f0 0x7f224806edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2258034000 tx=0x7f225802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.370+0000 7f2264c00640 1 -- 172.21.15.59:0/2637163922 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2260059930 con 0x7f226815e7c0 2024-04-23T14:33:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.615+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22680d2d10 con 0x7f226815e7c0 2024-04-23T14:33:31.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.616+0000 7f2264c00640 1 -- 172.21.15.59:0/2637163922 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f226005dc30 con 0x7f226815e7c0 2024-04-23T14:33:31.618 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:31.618 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:33:31.618 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:33:31.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224806c8f0 msgr2=0x7f224806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224806c8f0 0x7f224806edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2258034000 tx=0x7f225802e000 comp rx=0 tx=0).stop 2024-04-23T14:33:31.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226815e7c0 msgr2=0x7f22680ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226815e7c0 0x7f22680ca8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f226000e930 tx=0x7f226000edf0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 shutdown_connections 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f224806c8f0 0x7f224806edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226815e7c0 0x7f22680ca8f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22680d2630 0x7f22680ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 --2- 172.21.15.59:0/2637163922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22680d0d90 0x7f22680b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 >> 172.21.15.59:0/2637163922 conn(0x7f226809d920 msgr2=0x7f226806cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 shutdown_connections 2024-04-23T14:33:31.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.619+0000 7f226e400640 1 -- 172.21.15.59:0/2637163922 wait complete. 2024-04-23T14:33:31.638 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:31.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.841+0000 7f2c95600640 1 Processor -- start 2024-04-23T14:33:31.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.841+0000 7f2c95600640 1 -- start start 2024-04-23T14:33:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.842+0000 7f2c95600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c901454e0 0x7f2c90145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c95600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c900cd020 0x7f2c900b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c95600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c900cd520 con 0x7f2c900b3e40 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c95600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2c900cddc0 con 0x7f2c901454e0 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c95600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2c900cd800 con 0x7f2c900cd020 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c8ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c901454e0 0x7f2c90145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c8ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c901454e0 0x7f2c90145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44140/0 (socket says 172.21.15.59:44140) 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c8ea00640 1 -- 172.21.15.59:0/3914183670 learned_addr learned my addr 172.21.15.59:0/3914183670 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c8ea00640 1 -- 172.21.15.59:0/3914183670 >> v2:172.21.15.76:3300/0 conn(0x7f2c900cd020 msgr2=0x7f2c900b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c8ea00640 1 --2- 172.21.15.59:0/3914183670 >> v2:172.21.15.76:3300/0 conn(0x7f2c900cd020 0x7f2c900b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c8ea00640 1 -- 172.21.15.59:0/3914183670 >> v1:172.21.15.76:6789/0 conn(0x7f2c900b3e40 legacy=0x7f2c90160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:31.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.843+0000 7f2c8ea00640 1 -- 172.21.15.59:0/3914183670 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c900cdae0 con 0x7f2c901454e0 2024-04-23T14:33:31.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.845+0000 7f2c8ea00640 1 --2- 172.21.15.59:0/3914183670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c901454e0 0x7f2c90145bb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c900d1490 tx=0x7f2c8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e853b73fff0f315 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.845+0000 7f2c8d600640 1 -- 172.21.15.59:0/3914183670 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037070 con 0x7f2c901454e0 2024-04-23T14:33:31.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.845+0000 7f2c8d600640 1 -- 172.21.15.59:0/3914183670 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80032440 con 0x7f2c901454e0 2024-04-23T14:33:31.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.845+0000 7f2c8d600640 1 -- 172.21.15.59:0/3914183670 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80032810 con 0x7f2c901454e0 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 -- 172.21.15.59:0/3914183670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c901454e0 msgr2=0x7f2c90145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 --2- 172.21.15.59:0/3914183670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c901454e0 0x7f2c90145bb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c900d1490 tx=0x7f2c8002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 -- 172.21.15.59:0/3914183670 shutdown_connections 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 --2- 172.21.15.59:0/3914183670 >> v2:172.21.15.76:3300/0 conn(0x7f2c900cd020 0x7f2c900b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 --2- 172.21.15.59:0/3914183670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c901454e0 0x7f2c90145bb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 -- 172.21.15.59:0/3914183670 >> 172.21.15.59:0/3914183670 conn(0x7f2c90067ea0 msgr2=0x7f2c9006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 -- 172.21.15.59:0/3914183670 shutdown_connections 2024-04-23T14:33:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.846+0000 7f2c95600640 1 -- 172.21.15.59:0/3914183670 wait complete. 2024-04-23T14:33:31.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.848+0000 7f2c95600640 1 Processor -- start 2024-04-23T14:33:31.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.848+0000 7f2c95600640 1 -- start start 2024-04-23T14:33:31.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c95600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c900b3e40 0x7f2c901e6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c95600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c900cd020 0x7f2c900afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c95600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c901454e0 0x7f2c900b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c95600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90146270 con 0x7f2c900cd020 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c95600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901460f0 con 0x7f2c900b3e40 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c95600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2c90161020 con 0x7f2c901454e0 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c8e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c900cd020 0x7f2c900afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c8e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c900cd020 0x7f2c900afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44146/0 (socket says 172.21.15.59:44146) 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c8e000640 1 -- 172.21.15.59:0/422021293 learned_addr learned my addr 172.21.15.59:0/422021293 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c8ea00640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c900b3e40 0x7f2c901e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c8f400640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c901454e0 0x7f2c900b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c8e000640 1 -- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c901454e0 msgr2=0x7f2c900b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.849+0000 7f2c8e000640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c901454e0 0x7f2c900b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c8e000640 1 -- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c900b3e40 msgr2=0x7f2c901e6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c8e000640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c900b3e40 0x7f2c901e6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c8e000640 1 -- 172.21.15.59:0/422021293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9006fe00 con 0x7f2c900cd020 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c8f400640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c901454e0 0x7f2c900b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c8ea00640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c900b3e40 0x7f2c901e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c8e000640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c900cd020 0x7f2c900afe90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b4a0 tx=0x7f2c7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c86a00640 1 -- 172.21.15.59:0/422021293 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c014070 con 0x7f2c900cd020 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c86a00640 1 -- 172.21.15.59:0/422021293 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7c010610 con 0x7f2c900cd020 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9006ed60 con 0x7f2c900cd020 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c86a00640 1 -- 172.21.15.59:0/422021293 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c010a00 con 0x7f2c900cd020 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.850+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9006f320 con 0x7f2c900cd020 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.851+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c90068a80 con 0x7f2c900cd020 2024-04-23T14:33:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.853+0000 7f2c86a00640 1 -- 172.21.15.59:0/422021293 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c7c017070 con 0x7f2c900cd020 2024-04-23T14:33:31.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.853+0000 7f2c86a00640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c7006c8f0 0x7f2c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:31.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.854+0000 7f2c86a00640 1 -- 172.21.15.59:0/422021293 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2c7c08b510 con 0x7f2c900cd020 2024-04-23T14:33:31.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.854+0000 7f2c8ea00640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c7006c8f0 0x7f2c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:31.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.855+0000 7f2c8ea00640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c7006c8f0 0x7f2c7006edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2c80034000 tx=0x7f2c8002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:31.859+0000 7f2c86a00640 1 -- 172.21.15.59:0/422021293 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c7c0599d0 con 0x7f2c900cd020 2024-04-23T14:33:32.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.065+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c900b36e0 con 0x7f2c900cd020 2024-04-23T14:33:32.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.068+0000 7f2c86a00640 1 -- 172.21.15.59:0/422021293 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f2c7c0592d0 con 0x7f2c900cd020 2024-04-23T14:33:32.069 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:32.070 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:50.389436+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.070+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c7006c8f0 msgr2=0x7f2c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.070+0000 7f2c95600640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c7006c8f0 0x7f2c7006edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2c80034000 tx=0x7f2c8002e000 comp rx=0 tx=0).stop 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.070+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c900cd020 msgr2=0x7f2c900afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.070+0000 7f2c95600640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c900cd020 0x7f2c900afe90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b4a0 tx=0x7f2c7c00b960 comp rx=0 tx=0).stop 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 shutdown_connections 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c7006c8f0 0x7f2c7006edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c901454e0 0x7f2c900b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c900cd020 0x7f2c900afe90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 --2- 172.21.15.59:0/422021293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c900b3e40 0x7f2c901e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 >> 172.21.15.59:0/422021293 conn(0x7f2c90067ea0 msgr2=0x7f2c900cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 shutdown_connections 2024-04-23T14:33:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.071+0000 7f2c95600640 1 -- 172.21.15.59:0/422021293 wait complete. 2024-04-23T14:33:32.092 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:33:32.092 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:32.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.309+0000 7f7f4be00640 1 Processor -- start 2024-04-23T14:33:32.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.309+0000 7f7f4be00640 1 -- start start 2024-04-23T14:33:32.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.310+0000 7f7f4be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7f4c15c3d0 0x7f7f4c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.310+0000 7f7f4be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 0x7f7f4c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.310+0000 7f7f4be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c160210 con 0x7f7f4c0b4690 2024-04-23T14:33:32.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.310+0000 7f7f4be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4c1604f0 con 0x7f7f4c15c3d0 2024-04-23T14:33:32.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.310+0000 7f7f4be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c1607d0 con 0x7f7f4c0cd590 2024-04-23T14:33:32.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.311+0000 7f7f4b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7f4c0b4690 0x7f7f4c06b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54624/0 (socket says 172.21.15.59:54624) 2024-04-23T14:33:32.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.311+0000 7f7f4b400640 1 -- 172.21.15.59:0/2489496419 learned_addr learned my addr 172.21.15.59:0/2489496419 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:32.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.311+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/2489496419 >> v2:172.21.15.76:3300/0 conn(0x7f7f4c15c3d0 0x7f7f4c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.311+0000 7f7f4a000640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 0x7f7f4c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.311+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 83288627 0 0) 0x7f7f4c160210 con 0x7f7f4c0b4690 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.311+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f30003520 con 0x7f7f4c0b4690 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.311+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2535464471 0 0) 0x7f7f30003520 con 0x7f7f4c0b4690 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 msgr2=0x7f7f4c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 0x7f7f4c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 >> v2:172.21.15.76:3300/0 conn(0x7f7f4c15c3d0 msgr2=0x7f7f4c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 --2- 172.21.15.59:0/2489496419 >> v2:172.21.15.76:3300/0 conn(0x7f7f4c15c3d0 0x7f7f4c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c160ab0 con 0x7f7f4c0b4690 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f7f440019f0 con 0x7f7f4c0b4690 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 >> v1:172.21.15.59:6789/0 conn(0x7f7f4c0b4690 legacy=0x7f7f4c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 0x7f7f30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f30004d40 0x7f7f300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1604f0 con 0x7f7f300039b0 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1607d0 con 0x7f7f30004d40 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3000b9c0 con 0x7f7f4c0cd590 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f4a000640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 0x7f7f30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f4aa00640 1 -- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 msgr2=0x7f7f30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4c0cd590 0x7f7f30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f4aa00640 1 -- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f30004d40 msgr2=0x7f7f300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f30004d40 0x7f7f300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.312+0000 7f7f4aa00640 1 -- 172.21.15.59:0/2489496419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f30003520 con 0x7f7f300039b0 2024-04-23T14:33:32.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.313+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009ac0 tx=0x7f7f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f972b7f7d1c2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c036070 con 0x7f7f300039b0 2024-04-23T14:33:32.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c031400 con 0x7f7f300039b0 2024-04-23T14:33:32.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f49600640 1 -- 172.21.15.59:0/2489496419 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c009690 con 0x7f7f300039b0 2024-04-23T14:33:32.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f4be00640 1 -- 172.21.15.59:0/2489496419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f300039b0 msgr2=0x7f7f30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f4be00640 1 --2- 172.21.15.59:0/2489496419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009ac0 tx=0x7f7f30005830 comp rx=0 tx=0).stop 2024-04-23T14:33:32.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f4b400640 1 -- 172.21.15.59:0/2489496419 reap_dead start 2024-04-23T14:33:32.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f4be00640 1 -- 172.21.15.59:0/2489496419 shutdown_connections 2024-04-23T14:33:32.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f4be00640 1 -- 172.21.15.59:0/2489496419 >> 172.21.15.59:0/2489496419 conn(0x7f7f4c0b21a0 msgr2=0x7f7f4c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:32.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f4be00640 1 -- 172.21.15.59:0/2489496419 shutdown_connections 2024-04-23T14:33:32.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.314+0000 7f7f4be00640 1 -- 172.21.15.59:0/2489496419 wait complete. 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.316+0000 7f7f4be00640 1 Processor -- start 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.316+0000 7f7f4be00640 1 -- start start 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.317+0000 7f7f4be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f300039b0 0x7f7f4c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.317+0000 7f7f4be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f4c0ae450 0x7f7f4c1f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.317+0000 7f7f4be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f4c0ae8d0 0x7f7f4c1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.317+0000 7f7f4be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c15c7b0 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.317+0000 7f7f4be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c0b4b00 con 0x7f7f4c0ae450 2024-04-23T14:33:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.317+0000 7f7f4be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c15c930 con 0x7f7f300039b0 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.317+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f4c0ae8d0 0x7f7f4c1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f4c0ae8d0 0x7f7f4c1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44160/0 (socket says 172.21.15.59:44160) 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 -- 172.21.15.59:0/436802324 learned_addr learned my addr 172.21.15.59:0/436802324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4a000640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f4c0ae450 0x7f7f4c1f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f300039b0 0x7f7f4c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 -- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f300039b0 msgr2=0x7f7f4c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f300039b0 0x7f7f4c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 -- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f4c0ae450 msgr2=0x7f7f4c1f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f4c0ae450 0x7f7f4c1f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 -- 172.21.15.59:0/436802324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c1f8980 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.318+0000 7f7f4b400640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f4c0ae8d0 0x7f7f4c1f8350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06d960 tx=0x7f7f4400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.319+0000 7f7f43400640 1 -- 172.21.15.59:0/436802324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44007110 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.319+0000 7f7f43400640 1 -- 172.21.15.59:0/436802324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f440072b0 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.319+0000 7f7f43400640 1 -- 172.21.15.59:0/436802324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44007680 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.319+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c0c9530 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.319+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c0c9af0 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.320+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c0c9cf0 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.321+0000 7f7f43400640 1 -- 172.21.15.59:0/436802324 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f44007820 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.322+0000 7f7f43400640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2c06c920 0x7f7f2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.322+0000 7f7f43400640 1 -- 172.21.15.59:0/436802324 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7f4408ada0 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.322+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2c06c920 0x7f7f2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.323+0000 7f7f4aa00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2c06c920 0x7f7f2c06ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c033000 tx=0x7f7f3c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.327+0000 7f7f43400640 1 -- 172.21.15.59:0/436802324 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f4c0c9cf0 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.569+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f4c04fd90 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.569+0000 7f7f43400640 1 -- 172.21.15.59:0/436802324 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f7f4401a070 con 0x7f7f4c0ae8d0 2024-04-23T14:33:32.571 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:32.571 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:33:32.571 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2c06c920 msgr2=0x7f7f2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2c06c920 0x7f7f2c06ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c033000 tx=0x7f7f3c008030 comp rx=0 tx=0).stop 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f4c0ae8d0 msgr2=0x7f7f4c1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f4c0ae8d0 0x7f7f4c1f8350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06d960 tx=0x7f7f4400cf00 comp rx=0 tx=0).stop 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 shutdown_connections 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2c06c920 0x7f7f2c06ede0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f4c0ae8d0 0x7f7f4c1f8350 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f4c0ae450 0x7f7f4c1f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 --2- 172.21.15.59:0/436802324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f300039b0 0x7f7f4c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.572+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 >> 172.21.15.59:0/436802324 conn(0x7f7f4c0b21a0 msgr2=0x7f7f4c0cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.573+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 shutdown_connections 2024-04-23T14:33:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.573+0000 7f7f4be00640 1 -- 172.21.15.59:0/436802324 wait complete. 2024-04-23T14:33:32.592 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:32.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.794+0000 7f5c09000640 1 Processor -- start 2024-04-23T14:33:32.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.794+0000 7f5c09000640 1 -- start start 2024-04-23T14:33:32.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c09000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c040b5ff0 0x7f5c040b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c09000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 0x7f5c040d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c09000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c040cf030 con 0x7f5c0415e7c0 2024-04-23T14:33:32.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c09000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5c040b7000 con 0x7f5c040b5ff0 2024-04-23T14:33:32.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c09000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5c040cf620 con 0x7f5c040d0d90 2024-04-23T14:33:32.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c03400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5c0415e7c0 0x7f5c040d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54628/0 (socket says 172.21.15.59:54628) 2024-04-23T14:33:32.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c03400640 1 -- 172.21.15.59:0/1352894863 learned_addr learned my addr 172.21.15.59:0/1352894863 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c03e00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 0x7f5c040d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5bfbe00640 1 --2- 172.21.15.59:0/1352894863 >> v2:172.21.15.76:3300/0 conn(0x7f5c040b5ff0 0x7f5c040b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.796+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1653660325 0 0) 0x7f5bf4001110 con 0x7f5c0415e7c0 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5be8003520 con 0x7f5c0415e7c0 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 187567621 0 0) 0x7f5be8003520 con 0x7f5c0415e7c0 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 msgr2=0x7f5c040d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 0x7f5c040d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 >> v2:172.21.15.76:3300/0 conn(0x7f5c040b5ff0 msgr2=0x7f5c040b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 --2- 172.21.15.59:0/1352894863 >> v2:172.21.15.76:3300/0 conn(0x7f5c040b5ff0 0x7f5c040b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040b6a40 con 0x7f5c0415e7c0 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f5bf4001840 con 0x7f5c0415e7c0 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 >> v1:172.21.15.59:6789/0 conn(0x7f5c0415e7c0 legacy=0x7f5c040d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 0x7f5be80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5be8003f80 0x7f5be8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5be80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c040b7000 con 0x7f5be8004d40 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5c040cf620 con 0x7f5be8003f80 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5be800b9c0 con 0x7f5c040d0d90 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.797+0000 7f5c03e00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5be80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5bfbe00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5be8003f80 0x7f5be8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5c03400640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 0x7f5be80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5c03e00640 1 -- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 msgr2=0x7f5be80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5c03e00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 0x7f5be80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5c03e00640 1 -- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5be8003f80 msgr2=0x7f5be8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:32.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5c03e00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5be8003f80 0x7f5be8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5c03e00640 1 -- 172.21.15.59:0/1352894863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be8003520 con 0x7f5be8004d40 2024-04-23T14:33:32.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5bfbe00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5be8003f80 0x7f5be8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:32.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.798+0000 7f5c03400640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c040d0d90 0x7f5be80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.799+0000 7f5c03e00640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5be80051c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e170 tx=0x7f5be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff6d4056bdb7fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.799+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc034020 con 0x7f5be8004d40 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.799+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bfc0305f0 con 0x7f5be8004d40 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.799+0000 7f5c02a00640 1 -- 172.21.15.59:0/1352894863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc009c10 con 0x7f5be8004d40 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.800+0000 7f5c09000640 1 -- 172.21.15.59:0/1352894863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 msgr2=0x7f5be80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.800+0000 7f5c09000640 1 --2- 172.21.15.59:0/1352894863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5be80051c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e170 tx=0x7f5be8005830 comp rx=0 tx=0).stop 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.800+0000 7f5c03e00640 1 -- 172.21.15.59:0/1352894863 reap_dead start 2024-04-23T14:33:32.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.800+0000 7f5c09000640 1 -- 172.21.15.59:0/1352894863 shutdown_connections 2024-04-23T14:33:32.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.800+0000 7f5c09000640 1 -- 172.21.15.59:0/1352894863 >> 172.21.15.59:0/1352894863 conn(0x7f5c0409d920 msgr2=0x7f5c0408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:32.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.800+0000 7f5c09000640 1 -- 172.21.15.59:0/1352894863 shutdown_connections 2024-04-23T14:33:32.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.800+0000 7f5c09000640 1 -- 172.21.15.59:0/1352894863 wait complete. 2024-04-23T14:33:32.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.802+0000 7f5c09000640 1 Processor -- start 2024-04-23T14:33:32.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.802+0000 7f5c09000640 1 -- start start 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c09000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5c041e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c09000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c041525e0 0x7f5c040b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c09000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040b2cd0 0x7f5c040cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c09000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c040d2800 con 0x7f5be8004d40 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c09000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5c040cf2b0 con 0x7f5c040b2cd0 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c09000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5c040d1a80 con 0x7f5c041525e0 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c03400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5c041e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c03400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5c041e8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44186/0 (socket says 172.21.15.59:44186) 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c03400640 1 -- 172.21.15.59:0/3262114644 learned_addr learned my addr 172.21.15.59:0/3262114644 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5bfbe00640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c041525e0 0x7f5c040b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.803+0000 7f5c03e00640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040b2cd0 0x7f5c040cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5c03400640 1 -- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c041525e0 msgr2=0x7f5c040b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5c03400640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c041525e0 0x7f5c040b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5c03400640 1 -- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040b2cd0 msgr2=0x7f5c040cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:32.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5c03400640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040b2cd0 0x7f5c040cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:32.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5c03400640 1 -- 172.21.15.59:0/3262114644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040ce160 con 0x7f5be8004d40 2024-04-23T14:33:32.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5c03400640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5c041e8c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400d8b0 tx=0x7f5bf400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5bfb400640 1 -- 172.21.15.59:0/3262114644 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4019070 con 0x7f5be8004d40 2024-04-23T14:33:32.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5bfb400640 1 -- 172.21.15.59:0/3262114644 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf40074e0 con 0x7f5be8004d40 2024-04-23T14:33:32.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5bfb400640 1 -- 172.21.15.59:0/3262114644 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40049e0 con 0x7f5be8004d40 2024-04-23T14:33:32.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.804+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c041e8170 con 0x7f5be8004d40 2024-04-23T14:33:32.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.805+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c04152a60 con 0x7f5be8004d40 2024-04-23T14:33:32.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.806+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0404fd90 con 0x7f5be8004d40 2024-04-23T14:33:32.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.806+0000 7f5bfb400640 1 -- 172.21.15.59:0/3262114644 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bf4004b80 con 0x7f5be8004d40 2024-04-23T14:33:32.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.807+0000 7f5bfb400640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5be006c8f0 0x7f5be006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:32.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.808+0000 7f5bfb400640 1 -- 172.21.15.59:0/3262114644 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5bf408b620 con 0x7f5be8004d40 2024-04-23T14:33:32.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.808+0000 7f5bfbe00640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5be006c8f0 0x7f5be006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:32.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.809+0000 7f5bfbe00640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5be006c8f0 0x7f5be006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c0415a2b0 tx=0x7f5bec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:32.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:32.812+0000 7f5bfb400640 1 -- 172.21.15.59:0/3262114644 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c040cf030 con 0x7f5be8004d40 2024-04-23T14:33:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.055+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c040b30b0 con 0x7f5be8004d40 2024-04-23T14:33:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.056+0000 7f5bfb400640 1 -- 172.21.15.59:0/3262114644 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f5bf4059a40 con 0x7f5be8004d40 2024-04-23T14:33:33.057 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:33.057 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:32:48.829992+0000","modified":"2024-04-23T14:32:51.932361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5205":{"gid":5205,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1671400742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1671400742},{"type":"v1","addr":"172.21.15.59:6836","nonce":1671400742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5205,"qdb_cluster":[5205]},"id":2}]} 2024-04-23T14:33:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2024-04-23T14:33:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.058+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5be006c8f0 msgr2=0x7f5be006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.058+0000 7f5c09000640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5be006c8f0 0x7f5be006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c0415a2b0 tx=0x7f5bec008040 comp rx=0 tx=0).stop 2024-04-23T14:33:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.058+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 msgr2=0x7f5c041e8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.058+0000 7f5c09000640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5c041e8c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400d8b0 tx=0x7f5bf400dd70 comp rx=0 tx=0).stop 2024-04-23T14:33:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 shutdown_connections 2024-04-23T14:33:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5be006c8f0 0x7f5be006edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c041525e0 0x7f5c040b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040b2cd0 0x7f5c040cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 --2- 172.21.15.59:0/3262114644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be8004d40 0x7f5c041e8c50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 >> 172.21.15.59:0/3262114644 conn(0x7f5c0409d920 msgr2=0x7f5c040b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 shutdown_connections 2024-04-23T14:33:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.059+0000 7f5c09000640 1 -- 172.21.15.59:0/3262114644 wait complete. 2024-04-23T14:33:33.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:33.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.292+0000 7f2423200640 1 Processor -- start 2024-04-23T14:33:33.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.293+0000 7f2423200640 1 -- start start 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2423200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2423200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24240d0d90 0x7f24240d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2423200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24240cf030 con 0x7f24240d2630 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2423200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f24240b7000 con 0x7f24240d0d90 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2423200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f24240cf620 con 0x7f242415e7c0 2024-04-23T14:33:33.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2421400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f24240d2630 0x7f24240b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54636/0 (socket says 172.21.15.59:54636) 2024-04-23T14:33:33.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2422800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24240d0d90 0x7f24240d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.294+0000 7f2421400640 1 -- 172.21.15.59:0/554327558 learned_addr learned my addr 172.21.15.59:0/554327558 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:33.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.295+0000 7f2421e00640 1 --2- 172.21.15.59:0/554327558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.295+0000 7f2422800640 1 -- 172.21.15.59:0/554327558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 msgr2=0x7f24240d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:33.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.295+0000 7f2422800640 1 --2- 172.21.15.59:0/554327558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.295+0000 7f2422800640 1 -- 172.21.15.59:0/554327558 >> v1:172.21.15.59:6789/0 conn(0x7f24240d2630 legacy=0x7f24240b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.295+0000 7f2422800640 1 -- 172.21.15.59:0/554327558 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240b6a40 con 0x7f24240d0d90 2024-04-23T14:33:33.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.296+0000 7f2422800640 1 --2- 172.21.15.59:0/554327558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d0d90 0x7f24240d1220 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f241c0097a0 tx=0x7f241c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b126b62407a9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.296+0000 7f2420a00640 1 -- 172.21.15.59:0/554327558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c037070 con 0x7f24240d0d90 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.296+0000 7f2420a00640 1 -- 172.21.15.59:0/554327558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c032440 con 0x7f24240d0d90 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.296+0000 7f2420a00640 1 -- 172.21.15.59:0/554327558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c032810 con 0x7f24240d0d90 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 -- 172.21.15.59:0/554327558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d0d90 msgr2=0x7f24240d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 --2- 172.21.15.59:0/554327558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d0d90 0x7f24240d1220 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f241c0097a0 tx=0x7f241c02a180 comp rx=0 tx=0).stop 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 -- 172.21.15.59:0/554327558 shutdown_connections 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 --2- 172.21.15.59:0/554327558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d0d90 0x7f24240d1220 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 --2- 172.21.15.59:0/554327558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 -- 172.21.15.59:0/554327558 >> 172.21.15.59:0/554327558 conn(0x7f242409d920 msgr2=0x7f242408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 -- 172.21.15.59:0/554327558 shutdown_connections 2024-04-23T14:33:33.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.297+0000 7f2423200640 1 -- 172.21.15.59:0/554327558 wait complete. 2024-04-23T14:33:33.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.299+0000 7f2423200640 1 Processor -- start 2024-04-23T14:33:33.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.299+0000 7f2423200640 1 -- start start 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2423200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24240d0d90 0x7f24240cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2423200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d2630 0x7f24240cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2423200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2423200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24240b6700 con 0x7f24240d2630 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2423200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f24240cf2b0 con 0x7f24240d0d90 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2423200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f24240d1a80 con 0x7f242415e7c0 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2421400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d2630 0x7f24240cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2421400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d2630 0x7f24240cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44206/0 (socket says 172.21.15.59:44206) 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2421400640 1 -- 172.21.15.59:0/352889057 learned_addr learned my addr 172.21.15.59:0/352889057 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.300+0000 7f2422800640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2421e00640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24240d0d90 0x7f24240cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2421400640 1 -- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 msgr2=0x7f24240cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2421400640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2421400640 1 -- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24240d0d90 msgr2=0x7f24240cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2421400640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24240d0d90 0x7f24240cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2421400640 1 -- 172.21.15.59:0/352889057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240b5720 con 0x7f24240d2630 2024-04-23T14:33:33.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2422800640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:33.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2421400640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d2630 0x7f24240cd8a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f240c00cae0 tx=0x7f240c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f241a000640 1 -- 172.21.15.59:0/352889057 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c017070 con 0x7f24240d2630 2024-04-23T14:33:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f241a000640 1 -- 172.21.15.59:0/352889057 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240c00ac40 con 0x7f24240d2630 2024-04-23T14:33:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f241a000640 1 -- 172.21.15.59:0/352889057 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c012620 con 0x7f24240d2630 2024-04-23T14:33:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24240cc2e0 con 0x7f24240d2630 2024-04-23T14:33:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.301+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24240cc8a0 con 0x7f24240d2630 2024-04-23T14:33:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.302+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24240d3d50 con 0x7f24240d2630 2024-04-23T14:33:33.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.304+0000 7f241a000640 1 -- 172.21.15.59:0/352889057 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f240c01a070 con 0x7f24240d2630 2024-04-23T14:33:33.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.304+0000 7f241a000640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f240006c920 0x7f240006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.305+0000 7f241a000640 1 -- 172.21.15.59:0/352889057 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f240c08b960 con 0x7f24240d2630 2024-04-23T14:33:33.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.305+0000 7f2421e00640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f240006c920 0x7f240006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.305+0000 7f2421e00640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f240006c920 0x7f240006ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2414006fd0 tx=0x7f2414008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.309+0000 7f241a000640 1 -- 172.21.15.59:0/352889057 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24240d3d50 con 0x7f24240d2630 2024-04-23T14:33:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.515+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2424073470 con 0x7f24240d2630 2024-04-23T14:33:33.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.517+0000 7f241a000640 1 -- 172.21.15.59:0/352889057 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f2424073470 con 0x7f24240d2630 2024-04-23T14:33:33.518 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:33.519 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:32:50.389436+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:32:46.759888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-23T14:32:47.810840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f240006c920 msgr2=0x7f240006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f240006c920 0x7f240006ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2414006fd0 tx=0x7f2414008040 comp rx=0 tx=0).stop 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d2630 msgr2=0x7f24240cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d2630 0x7f24240cd8a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f240c00cae0 tx=0x7f240c00a060 comp rx=0 tx=0).stop 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 shutdown_connections 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f240006c920 0x7f240006ede0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f242415e7c0 0x7f24240cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24240d2630 0x7f24240cd8a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 --2- 172.21.15.59:0/352889057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24240d0d90 0x7f24240cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 >> 172.21.15.59:0/352889057 conn(0x7f242409d920 msgr2=0x7f24241617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 shutdown_connections 2024-04-23T14:33:33.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.520+0000 7f2423200640 1 -- 172.21.15.59:0/352889057 wait complete. 2024-04-23T14:33:33.542 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:33:33.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.754+0000 7fe6eae00640 1 Processor -- start 2024-04-23T14:33:33.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.754+0000 7fe6eae00640 1 -- start start 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6eae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 0x7fe6e415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6eae00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6e40af890 0x7fe6e4144a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6eae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6e40b3010 0x7fe6e40d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6eae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e4144f60 con 0x7fe6e40b3010 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6eae00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e415f3f0 con 0x7fe6e40af890 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6eae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e415f6d0 con 0x7fe6e40af4b0 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6ea400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6e40b3010 0x7fe6e40d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6ea400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6e40b3010 0x7fe6e40d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44214/0 (socket says 172.21.15.59:44214) 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.756+0000 7fe6ea400640 1 -- 172.21.15.59:0/1087686432 learned_addr learned my addr 172.21.15.59:0/1087686432 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:33.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.757+0000 7fe6e9000640 1 --2- 172.21.15.59:0/1087686432 >> v2:172.21.15.76:3300/0 conn(0x7fe6e40af890 0x7fe6e4144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.757+0000 7fe6e9a00640 1 --2- 172.21.15.59:0/1087686432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 0x7fe6e415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.757+0000 7fe6ea400640 1 -- 172.21.15.59:0/1087686432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 msgr2=0x7fe6e415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.757+0000 7fe6ea400640 1 --2- 172.21.15.59:0/1087686432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 0x7fe6e415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.757+0000 7fe6ea400640 1 -- 172.21.15.59:0/1087686432 >> v2:172.21.15.76:3300/0 conn(0x7fe6e40af890 msgr2=0x7fe6e4144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.757+0000 7fe6ea400640 1 --2- 172.21.15.59:0/1087686432 >> v2:172.21.15.76:3300/0 conn(0x7fe6e40af890 0x7fe6e4144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.757+0000 7fe6ea400640 1 -- 172.21.15.59:0/1087686432 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e415f9b0 con 0x7fe6e40b3010 2024-04-23T14:33:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.758+0000 7fe6ea400640 1 --2- 172.21.15.59:0/1087686432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e40b3010 0x7fe6e40d05c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe6e406ca20 tx=0x7fe6dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b09697d61b57d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.758+0000 7fe6e3e00640 1 -- 172.21.15.59:0/1087686432 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc037070 con 0x7fe6e40b3010 2024-04-23T14:33:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.758+0000 7fe6e3e00640 1 -- 172.21.15.59:0/1087686432 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc032440 con 0x7fe6e40b3010 2024-04-23T14:33:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.758+0000 7fe6e3e00640 1 -- 172.21.15.59:0/1087686432 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc032810 con 0x7fe6e40b3010 2024-04-23T14:33:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 -- 172.21.15.59:0/1087686432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e40b3010 msgr2=0x7fe6e40d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 --2- 172.21.15.59:0/1087686432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e40b3010 0x7fe6e40d05c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe6e406ca20 tx=0x7fe6dc02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 -- 172.21.15.59:0/1087686432 shutdown_connections 2024-04-23T14:33:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 --2- 172.21.15.59:0/1087686432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e40b3010 0x7fe6e40d05c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 --2- 172.21.15.59:0/1087686432 >> v2:172.21.15.76:3300/0 conn(0x7fe6e40af890 0x7fe6e4144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 --2- 172.21.15.59:0/1087686432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 0x7fe6e415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 -- 172.21.15.59:0/1087686432 >> 172.21.15.59:0/1087686432 conn(0x7fe6e406bdf0 msgr2=0x7fe6e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 -- 172.21.15.59:0/1087686432 shutdown_connections 2024-04-23T14:33:33.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.759+0000 7fe6eae00640 1 -- 172.21.15.59:0/1087686432 wait complete. 2024-04-23T14:33:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.761+0000 7fe6eae00640 1 Processor -- start 2024-04-23T14:33:33.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.761+0000 7fe6eae00640 1 -- start start 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6eae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 0x7fe6e415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6eae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6e40b3010 0x7fe6e4160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6eae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e4145a40 0x7fe6e41474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6eae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40b3640 con 0x7fe6e4145a40 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6eae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40b34c0 con 0x7fe6e40b3010 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6eae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e40d0b00 con 0x7fe6e40af4b0 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6ea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e4145a40 0x7fe6e41474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6ea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e4145a40 0x7fe6e41474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44224/0 (socket says 172.21.15.59:44224) 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.762+0000 7fe6ea400640 1 -- 172.21.15.59:0/3359622780 learned_addr learned my addr 172.21.15.59:0/3359622780 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6e9000640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6e40b3010 0x7fe6e4160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6ea400640 1 -- 172.21.15.59:0/3359622780 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 msgr2=0x7fe6e415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6ea400640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 0x7fe6e415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6ea400640 1 -- 172.21.15.59:0/3359622780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6e40b3010 msgr2=0x7fe6e4160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6ea400640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6e40b3010 0x7fe6e4160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:33.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6ea400640 1 -- 172.21.15.59:0/3359622780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e4161830 con 0x7fe6e4145a40 2024-04-23T14:33:33.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6ea400640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e4145a40 0x7fe6e41474a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc02a220 tx=0x7fe6dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.763+0000 7fe6e1600640 1 -- 172.21.15.59:0/3359622780 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc041020 con 0x7fe6e4145a40 2024-04-23T14:33:33.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.764+0000 7fe6e1600640 1 -- 172.21.15.59:0/3359622780 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc0306e0 con 0x7fe6e4145a40 2024-04-23T14:33:33.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.764+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e4161270 con 0x7fe6e4145a40 2024-04-23T14:33:33.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.764+0000 7fe6e1600640 1 -- 172.21.15.59:0/3359622780 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc037040 con 0x7fe6e4145a40 2024-04-23T14:33:33.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.764+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e40ad400 con 0x7fe6e4145a40 2024-04-23T14:33:33.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.765+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e404fd90 con 0x7fe6e4145a40 2024-04-23T14:33:33.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.766+0000 7fe6e1600640 1 -- 172.21.15.59:0/3359622780 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6dc044020 con 0x7fe6e4145a40 2024-04-23T14:33:33.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.767+0000 7fe6e1600640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6c406c8f0 0x7fe6c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:33.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.767+0000 7fe6e1600640 1 -- 172.21.15.59:0/3359622780 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe6dc0ac340 con 0x7fe6e4145a40 2024-04-23T14:33:33.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.767+0000 7fe6e9a00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6c406c8f0 0x7fe6c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:33.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.768+0000 7fe6e9a00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6c406c8f0 0x7fe6c406edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80070b0 tx=0x7fe6d8007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:33.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:33.772+0000 7fe6e1600640 1 -- 172.21.15.59:0/3359622780 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6dc04b030 con 0x7fe6e4145a40 2024-04-23T14:33:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.021+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe6e40d08e0 con 0x7fe6e4145a40 2024-04-23T14:33:34.024 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:33:34.023+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:33:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.026+0000 7fe6e1600640 1 -- 172.21.15.59:0/3359622780 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7fe6dc07a760 con 0x7fe6e4145a40 2024-04-23T14:33:34.028 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:33:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6c406c8f0 msgr2=0x7fe6c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6c406c8f0 0x7fe6c406edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80070b0 tx=0x7fe6d8007040 comp rx=0 tx=0).stop 2024-04-23T14:33:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e4145a40 msgr2=0x7fe6e41474a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e4145a40 0x7fe6e41474a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc02a220 tx=0x7fe6dc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 shutdown_connections 2024-04-23T14:33:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6c406c8f0 0x7fe6c406edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6e4145a40 0x7fe6e41474a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6e40b3010 0x7fe6e4160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 --2- 172.21.15.59:0/3359622780 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6e40af4b0 0x7fe6e415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 >> 172.21.15.59:0/3359622780 conn(0x7fe6e406bdf0 msgr2=0x7fe6e406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 shutdown_connections 2024-04-23T14:33:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.030+0000 7fe6eae00640 1 -- 172.21.15.59:0/3359622780 wait complete. 2024-04-23T14:33:34.044 INFO:tasks.ceph.mds.c.smithi059.stderr: -6441> 2024-04-23T14:32:51.237+0000 7fe562400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:33:34.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:33:34.101 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:33:34.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.245+0000 7f4a8ee00640 1 Processor -- start 2024-04-23T14:33:34.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.245+0000 7f4a8ee00640 1 -- start start 2024-04-23T14:33:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8ee00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a8815ed20 0x7f4a880d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8ee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4a880b3600 0x7f4a880d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a880b5da0 con 0x7f4a880b3f10 2024-04-23T14:33:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8ee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8815f100 con 0x7f4a880b3600 2024-04-23T14:33:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8ee00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4a880d0a20 con 0x7f4a8815ed20 2024-04-23T14:33:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4a880b3600 0x7f4a880d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8e400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4a880b3f10 0x7f4a880b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54638/0 (socket says 172.21.15.59:54638) 2024-04-23T14:33:34.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.247+0000 7f4a8e400640 1 -- 172.21.15.59:0/3296424097 learned_addr learned my addr 172.21.15.59:0/3296424097 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:34.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.248+0000 7f4a8da00640 1 --2- 172.21.15.59:0/3296424097 >> v2:172.21.15.76:3300/0 conn(0x7f4a8815ed20 0x7f4a880d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.248+0000 7f4a8d000640 1 -- 172.21.15.59:0/3296424097 >> v2:172.21.15.76:3300/0 conn(0x7f4a8815ed20 msgr2=0x7f4a880d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.248+0000 7f4a8d000640 1 --2- 172.21.15.59:0/3296424097 >> v2:172.21.15.76:3300/0 conn(0x7f4a8815ed20 0x7f4a880d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.248+0000 7f4a8d000640 1 -- 172.21.15.59:0/3296424097 >> v1:172.21.15.59:6789/0 conn(0x7f4a880b3f10 legacy=0x7f4a880b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.248+0000 7f4a8d000640 1 -- 172.21.15.59:0/3296424097 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a880d2ed0 con 0x7f4a880b3600 2024-04-23T14:33:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.248+0000 7f4a87e00640 1 -- 172.21.15.59:0/3296424097 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 611640927 0 0) 0x7f4a880b5da0 con 0x7f4a880b3f10 2024-04-23T14:33:34.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.249+0000 7f4a8d000640 1 --2- 172.21.15.59:0/3296424097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a880b3600 0x7f4a880d13e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4a8808e2a0 tx=0x7f4a7402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723d873201f967e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.249+0000 7f4a87e00640 1 -- 172.21.15.59:0/3296424097 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037070 con 0x7f4a880b3600 2024-04-23T14:33:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.249+0000 7f4a87e00640 1 -- 172.21.15.59:0/3296424097 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7402add0 con 0x7f4a880b3600 2024-04-23T14:33:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.249+0000 7f4a87e00640 1 -- 172.21.15.59:0/3296424097 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74032660 con 0x7f4a880b3600 2024-04-23T14:33:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 -- 172.21.15.59:0/3296424097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a880b3600 msgr2=0x7f4a880d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/3296424097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a880b3600 0x7f4a880d13e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4a8808e2a0 tx=0x7f4a7402a080 comp rx=0 tx=0).stop 2024-04-23T14:33:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 -- 172.21.15.59:0/3296424097 shutdown_connections 2024-04-23T14:33:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/3296424097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a880b3600 0x7f4a880d13e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/3296424097 >> v2:172.21.15.76:3300/0 conn(0x7f4a8815ed20 0x7f4a880d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 -- 172.21.15.59:0/3296424097 >> 172.21.15.59:0/3296424097 conn(0x7f4a8808bf90 msgr2=0x7f4a880d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 -- 172.21.15.59:0/3296424097 shutdown_connections 2024-04-23T14:33:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.250+0000 7f4a8ee00640 1 -- 172.21.15.59:0/3296424097 wait complete. 2024-04-23T14:33:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.252+0000 7f4a8ee00640 1 Processor -- start 2024-04-23T14:33:34.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.252+0000 7f4a8ee00640 1 -- start start 2024-04-23T14:33:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a880b3600 0x7f4a881f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a880b3f10 0x7f4a880cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a8815ed20 0x7f4a880cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880d0290 con 0x7f4a8815ed20 2024-04-23T14:33:34.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4a880d3490 con 0x7f4a880b3600 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4a880d0410 con 0x7f4a880b3f10 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a8815ed20 0x7f4a880cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a8815ed20 0x7f4a880cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44266/0 (socket says 172.21.15.59:44266) 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8e400640 1 -- 172.21.15.59:0/1370825785 learned_addr learned my addr 172.21.15.59:0/1370825785 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8da00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a880b3600 0x7f4a881f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.253+0000 7f4a8d000640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a880b3f10 0x7f4a880cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8e400640 1 -- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a880b3f10 msgr2=0x7f4a880cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8e400640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a880b3f10 0x7f4a880cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8e400640 1 -- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a880b3600 msgr2=0x7f4a881f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8e400640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a880b3600 0x7f4a881f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8e400640 1 -- 172.21.15.59:0/1370825785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a881f8b80 con 0x7f4a8815ed20 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8d000640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a880b3f10 0x7f4a880cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8e400640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a8815ed20 0x7f4a880cbbd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000c480 tx=0x7f4a8000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a85600640 1 -- 172.21.15.59:0/1370825785 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80018070 con 0x7f4a8815ed20 2024-04-23T14:33:34.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a85600640 1 -- 172.21.15.59:0/1370825785 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a80007500 con 0x7f4a8815ed20 2024-04-23T14:33:34.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a85600640 1 -- 172.21.15.59:0/1370825785 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80004a20 con 0x7f4a8815ed20 2024-04-23T14:33:34.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.254+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a881f82e0 con 0x7f4a8815ed20 2024-04-23T14:33:34.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.255+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a881f88a0 con 0x7f4a8815ed20 2024-04-23T14:33:34.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.256+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8804fd90 con 0x7f4a8815ed20 2024-04-23T14:33:34.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.257+0000 7f4a85600640 1 -- 172.21.15.59:0/1370825785 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a8001b070 con 0x7f4a8815ed20 2024-04-23T14:33:34.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.258+0000 7f4a85600640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a6806c8f0 0x7f4a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:34.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.258+0000 7f4a85600640 1 -- 172.21.15.59:0/1370825785 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4a8008ae30 con 0x7f4a8815ed20 2024-04-23T14:33:34.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.258+0000 7f4a8da00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a6806c8f0 0x7f4a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:34.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.259+0000 7f4a8da00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a6806c8f0 0x7f4a6806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0099c0 tx=0x7f4a7c003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:34.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.262+0000 7f4a85600640 1 -- 172.21.15.59:0/1370825785 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a80059290 con 0x7f4a8815ed20 2024-04-23T14:33:34.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:34.509+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4a8815da80 con 0x7f4a8815ed20 2024-04-23T14:33:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.030+0000 7f4a85600640 1 -- 172.21.15.59:0/1370825785 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f4a80058b90 con 0x7f4a8815ed20 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a6806c8f0 msgr2=0x7f4a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a6806c8f0 0x7f4a6806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0099c0 tx=0x7f4a7c003990 comp rx=0 tx=0).stop 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a8815ed20 msgr2=0x7f4a880cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a8815ed20 0x7f4a880cbbd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000c480 tx=0x7f4a8000c940 comp rx=0 tx=0).stop 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 shutdown_connections 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a6806c8f0 0x7f4a6806edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a880b3f10 0x7f4a880cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a8815ed20 0x7f4a880cbbd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 --2- 172.21.15.59:0/1370825785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a880b3600 0x7f4a881f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.034+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 >> 172.21.15.59:0/1370825785 conn(0x7f4a8808bf90 msgr2=0x7f4a8806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.035+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 shutdown_connections 2024-04-23T14:33:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.035+0000 7f4a8ee00640 1 -- 172.21.15.59:0/1370825785 wait complete. 2024-04-23T14:33:35.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:33:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.262+0000 7f626cc00640 1 Processor -- start 2024-04-23T14:33:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.263+0000 7f626cc00640 1 -- start start 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.264+0000 7f626cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f626815e8e0 0x7f62680cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.264+0000 7f626cc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f626806cbb0 0x7f626806d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.264+0000 7f626cc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62680cf070 con 0x7f62680d4210 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.264+0000 7f626cc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f62680cf2f0 con 0x7f626815e8e0 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.264+0000 7f626cc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f62680d1c60 con 0x7f626806cbb0 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f626815e8e0 0x7f62680cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f626815e8e0 0x7f62680cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44276/0 (socket says 172.21.15.59:44276) 2024-04-23T14:33:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267400640 1 -- 172.21.15.59:0/973509358 learned_addr learned my addr 172.21.15.59:0/973509358 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267400640 1 -- 172.21.15.59:0/973509358 >> v2:172.21.15.76:3300/0 conn(0x7f626806cbb0 msgr2=0x7f626806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267e00640 1 --2- 172.21.15.59:0/973509358 >> v2:172.21.15.76:3300/0 conn(0x7f626806cbb0 0x7f626806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267400640 1 --2- 172.21.15.59:0/973509358 >> v2:172.21.15.76:3300/0 conn(0x7f626806cbb0 0x7f626806d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267400640 1 -- 172.21.15.59:0/973509358 >> v1:172.21.15.76:6789/0 conn(0x7f62680d4210 legacy=0x7f62680d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267400640 1 -- 172.21.15.59:0/973509358 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62680cf5b0 con 0x7f626815e8e0 2024-04-23T14:33:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.265+0000 7f6267e00640 1 --2- 172.21.15.59:0/973509358 >> v2:172.21.15.76:3300/0 conn(0x7f626806cbb0 0x7f626806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.266+0000 7f6267400640 1 --2- 172.21.15.59:0/973509358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f626815e8e0 0x7f62680cfdb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f626806c960 tx=0x7f625802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79def82c539bc58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f6266000640 1 -- 172.21.15.59:0/973509358 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258037070 con 0x7f626815e8e0 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f6266000640 1 -- 172.21.15.59:0/973509358 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6258032440 con 0x7f626815e8e0 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f6266000640 1 -- 172.21.15.59:0/973509358 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258032810 con 0x7f626815e8e0 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f626cc00640 1 -- 172.21.15.59:0/973509358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f626815e8e0 msgr2=0x7f62680cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f626cc00640 1 --2- 172.21.15.59:0/973509358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f626815e8e0 0x7f62680cfdb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f626806c960 tx=0x7f625802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f626cc00640 1 -- 172.21.15.59:0/973509358 shutdown_connections 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f626cc00640 1 --2- 172.21.15.59:0/973509358 >> v2:172.21.15.76:3300/0 conn(0x7f626806cbb0 0x7f626806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f626cc00640 1 --2- 172.21.15.59:0/973509358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f626815e8e0 0x7f62680cfdb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.267+0000 7f626cc00640 1 -- 172.21.15.59:0/973509358 >> 172.21.15.59:0/973509358 conn(0x7f626808e060 msgr2=0x7f626809d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.268+0000 7f626cc00640 1 -- 172.21.15.59:0/973509358 shutdown_connections 2024-04-23T14:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.268+0000 7f626cc00640 1 -- 172.21.15.59:0/973509358 wait complete. 2024-04-23T14:33:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.269+0000 7f626cc00640 1 Processor -- start 2024-04-23T14:33:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.269+0000 7f626cc00640 1 -- start start 2024-04-23T14:33:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.270+0000 7f626cc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806cbb0 0x7f626815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.270+0000 7f626cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680d4210 0x7f626809f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.270+0000 7f626cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f626815e8e0 0x7f62680cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.270+0000 7f626cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f626815ecc0 con 0x7f62680d4210 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.270+0000 7f626cc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f626806d580 con 0x7f626806cbb0 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.270+0000 7f626cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f62680d02f0 con 0x7f626815e8e0 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680d4210 0x7f626809f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680d4210 0x7f626809f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44282/0 (socket says 172.21.15.59:44282) 2024-04-23T14:33:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 -- 172.21.15.59:0/619765412 learned_addr learned my addr 172.21.15.59:0/619765412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6267400640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806cbb0 0x7f626815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6267e00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f626815e8e0 0x7f62680cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 -- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f626815e8e0 msgr2=0x7f62680cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f626815e8e0 0x7f62680cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 -- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806cbb0 msgr2=0x7f626815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806cbb0 0x7f626815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.271+0000 7f6266a00640 1 -- 172.21.15.59:0/619765412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62680ce620 con 0x7f62680d4210 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.272+0000 7f6266a00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680d4210 0x7f626809f330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6250004640 tx=0x7f625000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.272+0000 7f625f400640 1 -- 172.21.15.59:0/619765412 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250011790 con 0x7f62680d4210 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.272+0000 7f625f400640 1 -- 172.21.15.59:0/619765412 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6250017070 con 0x7f62680d4210 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.272+0000 7f625f400640 1 -- 172.21.15.59:0/619765412 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250011bb0 con 0x7f62680d4210 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.272+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626809f870 con 0x7f62680d4210 2024-04-23T14:33:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.272+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62680cbd00 con 0x7f62680d4210 2024-04-23T14:33:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.273+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626804fd90 con 0x7f62680d4210 2024-04-23T14:33:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.274+0000 7f625f400640 1 -- 172.21.15.59:0/619765412 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6250011d50 con 0x7f62680d4210 2024-04-23T14:33:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.275+0000 7f625f400640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624406c8f0 0x7f624406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.275+0000 7f625f400640 1 -- 172.21.15.59:0/619765412 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f625008b7e0 con 0x7f62680d4210 2024-04-23T14:33:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.276+0000 7f6267400640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624406c8f0 0x7f624406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.276+0000 7f6267400640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624406c8f0 0x7f624406edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6258034000 tx=0x7f625802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.280+0000 7f625f400640 1 -- 172.21.15.59:0/619765412 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6250059b10 con 0x7f62680d4210 2024-04-23T14:33:35.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:35.486+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f62680a2320 con 0x7f62680d4210 2024-04-23T14:33:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.149+0000 7f625f400640 1 -- 172.21.15.59:0/619765412 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f6250059410 con 0x7f62680d4210 2024-04-23T14:33:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:33:36.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.152+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624406c8f0 msgr2=0x7f624406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624406c8f0 0x7f624406edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6258034000 tx=0x7f625802e000 comp rx=0 tx=0).stop 2024-04-23T14:33:36.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680d4210 msgr2=0x7f626809f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680d4210 0x7f626809f330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6250004640 tx=0x7f625000c8e0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 shutdown_connections 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624406c8f0 0x7f624406edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f626815e8e0 0x7f62680cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680d4210 0x7f626809f330 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 --2- 172.21.15.59:0/619765412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f626806cbb0 0x7f626815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.153+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 >> 172.21.15.59:0/619765412 conn(0x7f626808e060 msgr2=0x7f626806d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.154+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 shutdown_connections 2024-04-23T14:33:36.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.154+0000 7f626cc00640 1 -- 172.21.15.59:0/619765412 wait complete. 2024-04-23T14:33:36.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:33:36.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.399+0000 7f7050800640 1 Processor -- start 2024-04-23T14:33:36.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.400+0000 7f7050800640 1 -- start start 2024-04-23T14:33:36.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.401+0000 7f7050800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f704815eca0 0x7f70480cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.401+0000 7f7050800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 0x7f704806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.401+0000 7f7050800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704815f080 con 0x7f70480d07c0 2024-04-23T14:33:36.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.401+0000 7f7050800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f70480b0cb0 con 0x7f704815eca0 2024-04-23T14:33:36.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.401+0000 7f7050800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f704806c200 con 0x7f70480cd7a0 2024-04-23T14:33:36.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.402+0000 7f704e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f70480d07c0 0x7f70480d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54646/0 (socket says 172.21.15.59:54646) 2024-04-23T14:33:36.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.402+0000 7f704e800640 1 -- 172.21.15.59:0/191870189 learned_addr learned my addr 172.21.15.59:0/191870189 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:36.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.402+0000 7f704d400640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 0x7f704806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.402+0000 7f704de00640 1 --2- 172.21.15.59:0/191870189 >> v2:172.21.15.76:3300/0 conn(0x7f704815eca0 0x7f70480cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.402+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1113020047 0 0) 0x7f704815f080 con 0x7f70480d07c0 2024-04-23T14:33:36.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.402+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7030003520 con 0x7f70480d07c0 2024-04-23T14:33:36.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3887344161 0 0) 0x7f7030003520 con 0x7f70480d07c0 2024-04-23T14:33:36.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 msgr2=0x7f704806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:36.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 0x7f704806b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 >> v2:172.21.15.76:3300/0 conn(0x7f704815eca0 msgr2=0x7f70480cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:36.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 --2- 172.21.15.59:0/191870189 >> v2:172.21.15.76:3300/0 conn(0x7f704815eca0 0x7f70480cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704806c4e0 con 0x7f70480d07c0 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f70440019f0 con 0x7f70480d07c0 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 >> v1:172.21.15.59:6789/0 conn(0x7f70480d07c0 legacy=0x7f70480d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 0x7f70300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7030003f30 0x7f7030004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7030004d40 0x7f70300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70480b0cb0 con 0x7f7030004d40 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f704806c200 con 0x7f7030003f30 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f703000baa0 con 0x7f70480cd7a0 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.403+0000 7f704e800640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7030004d40 0x7f70300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.404+0000 7f704de00640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 0x7f70300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.404+0000 7f704d400640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7030003f30 0x7f7030004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.404+0000 7f704e800640 1 -- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 msgr2=0x7f70300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.404+0000 7f704e800640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70480cd7a0 0x7f70300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.404+0000 7f704e800640 1 -- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7030003f30 msgr2=0x7f7030004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.404+0000 7f704e800640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7030003f30 0x7f7030004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.404+0000 7f704e800640 1 -- 172.21.15.59:0/191870189 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7030003520 con 0x7f7030004d40 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.405+0000 7f704e800640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7030004d40 0x7f70300051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f704400a920 tx=0x7f7030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f166d69767a0521 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.405+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704400a4c0 con 0x7f7030004d40 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.405+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704402f070 con 0x7f7030004d40 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.405+0000 7f704ca00640 1 -- 172.21.15.59:0/191870189 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704402bd40 con 0x7f7030004d40 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.406+0000 7f7050800640 1 -- 172.21.15.59:0/191870189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7030004d40 msgr2=0x7f70300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.406+0000 7f7050800640 1 --2- 172.21.15.59:0/191870189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7030004d40 0x7f70300051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f704400a920 tx=0x7f7030005830 comp rx=0 tx=0).stop 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.406+0000 7f704e800640 1 -- 172.21.15.59:0/191870189 reap_dead start 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.406+0000 7f7050800640 1 -- 172.21.15.59:0/191870189 shutdown_connections 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.406+0000 7f7050800640 1 -- 172.21.15.59:0/191870189 >> 172.21.15.59:0/191870189 conn(0x7f70480d14a0 msgr2=0x7f70480d18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.406+0000 7f7050800640 1 -- 172.21.15.59:0/191870189 shutdown_connections 2024-04-23T14:33:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.406+0000 7f7050800640 1 -- 172.21.15.59:0/191870189 wait complete. 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.408+0000 7f7050800640 1 Processor -- start 2024-04-23T14:33:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.408+0000 7f7050800640 1 -- start start 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f7050800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f704809b5e0 0x7f70480b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f7050800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f704809b9c0 0x7f704809dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f7050800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7048151510 0x7f70481519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f7050800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f704806b9d0 con 0x7f704809b9c0 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f7050800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70480b0f50 con 0x7f704809b5e0 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f7050800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f704806bb50 con 0x7f7048151510 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f704d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f704809b9c0 0x7f704809dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f704d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f704809b9c0 0x7f704809dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44298/0 (socket says 172.21.15.59:44298) 2024-04-23T14:33:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.409+0000 7f704d400640 1 -- 172.21.15.59:0/2482700984 learned_addr learned my addr 172.21.15.59:0/2482700984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704de00640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f704809b5e0 0x7f70480b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704e800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7048151510 0x7f70481519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704d400640 1 -- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7048151510 msgr2=0x7f70481519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704d400640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7048151510 0x7f70481519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704d400640 1 -- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f704809b5e0 msgr2=0x7f70480b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704d400640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f704809b5e0 0x7f70480b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704d400640 1 -- 172.21.15.59:0/2482700984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704809e230 con 0x7f704809b9c0 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704e800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7048151510 0x7f70481519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f704d400640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f704809b9c0 0x7f704809dcf0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7038011540 tx=0x7f7038011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f7042000640 1 -- 172.21.15.59:0/2482700984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038015930 con 0x7f704809b9c0 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.410+0000 7f7042000640 1 -- 172.21.15.59:0/2482700984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7038015ad0 con 0x7f704809b9c0 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.411+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704809e4d0 con 0x7f704809b9c0 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.411+0000 7f7042000640 1 -- 172.21.15.59:0/2482700984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800f460 con 0x7f704809b9c0 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.411+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704809ea90 con 0x7f704809b9c0 2024-04-23T14:33:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.412+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7048068a80 con 0x7f704809b9c0 2024-04-23T14:33:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.413+0000 7f7042000640 1 -- 172.21.15.59:0/2482700984 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f703800f600 con 0x7f704809b9c0 2024-04-23T14:33:36.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.414+0000 7f7042000640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:36.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.414+0000 7f7042000640 1 -- 172.21.15.59:0/2482700984 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7f703808c270 con 0x7f704809b9c0 2024-04-23T14:33:36.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.414+0000 7f704de00640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:36.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.415+0000 7f704de00640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f702c06c920 0x7f702c06ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f703c009a90 tx=0x7f703c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:36.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.419+0000 7f7042000640 1 -- 172.21.15.59:0/2482700984 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7038019030 con 0x7f704809b9c0 2024-04-23T14:33:36.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:36.623+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f70480b26a0 con 0x7f704809b9c0 2024-04-23T14:33:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.176+0000 7f7042000640 1 -- 172.21.15.59:0/2482700984 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f703805a7f0 con 0x7f704809b9c0 2024-04-23T14:33:37.178 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:33:37.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.180+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f702c06c920 msgr2=0x7f702c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.180+0000 7f7050800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f702c06c920 0x7f702c06ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f703c009a90 tx=0x7f703c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:37.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.180+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f704809b9c0 msgr2=0x7f704809dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.180+0000 7f7050800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f704809b9c0 0x7f704809dcf0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7038011540 tx=0x7f7038011a00 comp rx=0 tx=0).stop 2024-04-23T14:33:37.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 shutdown_connections 2024-04-23T14:33:37.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7048151510 0x7f70481519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f704809b9c0 0x7f704809dcf0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 --2- 172.21.15.59:0/2482700984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f704809b5e0 0x7f70480b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 >> 172.21.15.59:0/2482700984 conn(0x7f70480d14a0 msgr2=0x7f704806fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 shutdown_connections 2024-04-23T14:33:37.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.181+0000 7f7050800640 1 -- 172.21.15.59:0/2482700984 wait complete. 2024-04-23T14:33:37.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:33:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.414+0000 7fd115200640 1 Processor -- start 2024-04-23T14:33:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.414+0000 7fd115200640 1 -- start start 2024-04-23T14:33:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.415+0000 7fd115200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd11015c9e0 0x7fd11015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd115200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd1100708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd115200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd11015de00 con 0x7fd110070e20 2024-04-23T14:33:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd115200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd110071250 con 0x7fd11015c9e0 2024-04-23T14:33:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd115200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd1100701b0 con 0x7fd1100cd040 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd10ea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd1100708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd10fe00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd110070e20 0x7fd1101603f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:37866/0 (socket says 172.21.15.59:37866) 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd10ea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd1100708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:47590/0 (socket says 172.21.15.59:47590) 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.416+0000 7fd10fe00640 1 -- 172.21.15.59:0/1660352521 learned_addr learned my addr 172.21.15.59:0/1660352521 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.417+0000 7fd10f400640 1 --2- 172.21.15.59:0/1660352521 >> v2:172.21.15.76:3300/0 conn(0x7fd11015c9e0 0x7fd11015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.417+0000 7fd10ea00640 1 -- 172.21.15.59:0/1660352521 >> v2:172.21.15.76:3300/0 conn(0x7fd11015c9e0 msgr2=0x7fd11015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.417+0000 7fd10ea00640 1 --2- 172.21.15.59:0/1660352521 >> v2:172.21.15.76:3300/0 conn(0x7fd11015c9e0 0x7fd11015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.417+0000 7fd10ea00640 1 -- 172.21.15.59:0/1660352521 >> v1:172.21.15.76:6789/0 conn(0x7fd110070e20 legacy=0x7fd1101603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.417+0000 7fd10ea00640 1 -- 172.21.15.59:0/1660352521 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1100cd5d0 con 0x7fd1100cd040 2024-04-23T14:33:37.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.419+0000 7fd10ea00640 1 --2- 172.21.15.59:0/1660352521 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd1100708e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009d90 tx=0x7fd0f802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ffa465a105bfb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.419+0000 7fd10e000640 1 -- 172.21.15.59:0/1660352521 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8007740 con 0x7fd1100cd040 2024-04-23T14:33:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.419+0000 7fd10e000640 1 -- 172.21.15.59:0/1660352521 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f80078e0 con 0x7fd1100cd040 2024-04-23T14:33:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.419+0000 7fd10e000640 1 -- 172.21.15.59:0/1660352521 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8007cb0 con 0x7fd1100cd040 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 -- 172.21.15.59:0/1660352521 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 msgr2=0x7fd1100708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 --2- 172.21.15.59:0/1660352521 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd1100708e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009d90 tx=0x7fd0f802a530 comp rx=0 tx=0).stop 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 -- 172.21.15.59:0/1660352521 shutdown_connections 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 --2- 172.21.15.59:0/1660352521 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd1100708e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 --2- 172.21.15.59:0/1660352521 >> v2:172.21.15.76:3300/0 conn(0x7fd11015c9e0 0x7fd11015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 -- 172.21.15.59:0/1660352521 >> 172.21.15.59:0/1660352521 conn(0x7fd1100b3f20 msgr2=0x7fd1100b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 -- 172.21.15.59:0/1660352521 shutdown_connections 2024-04-23T14:33:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.420+0000 7fd115200640 1 -- 172.21.15.59:0/1660352521 wait complete. 2024-04-23T14:33:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.422+0000 7fd115200640 1 Processor -- start 2024-04-23T14:33:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.422+0000 7fd115200640 1 -- start start 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.423+0000 7fd115200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd110070e20 0x7fd1100cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.423+0000 7fd115200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd11015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.423+0000 7fd115200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd11015c9e0 0x7fd1101548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.423+0000 7fd115200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1100cde70 con 0x7fd110070e20 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.423+0000 7fd115200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1100cdb90 con 0x7fd11015c9e0 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.423+0000 7fd115200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd1100ce150 con 0x7fd1100cd040 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd110070e20 0x7fd1100cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd110070e20 0x7fd1100cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44306/0 (socket says 172.21.15.59:44306) 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 -- 172.21.15.59:0/2311401138 learned_addr learned my addr 172.21.15.59:0/2311401138 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10fe00640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd11015c9e0 0x7fd1101548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10ea00640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd11015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 -- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 msgr2=0x7fd11015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd11015dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 -- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd11015c9e0 msgr2=0x7fd1101548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd11015c9e0 0x7fd1101548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.424+0000 7fd10f400640 1 -- 172.21.15.59:0/2311401138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1100b0fe0 con 0x7fd110070e20 2024-04-23T14:33:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.425+0000 7fd10f400640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd110070e20 0x7fd1100cb5c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e870 tx=0x7fd10000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.425+0000 7fd107400640 1 -- 172.21.15.59:0/2311401138 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100017070 con 0x7fd110070e20 2024-04-23T14:33:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.425+0000 7fd107400640 1 -- 172.21.15.59:0/2311401138 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd10000c940 con 0x7fd110070e20 2024-04-23T14:33:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.425+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11006ccf0 con 0x7fd110070e20 2024-04-23T14:33:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.425+0000 7fd107400640 1 -- 172.21.15.59:0/2311401138 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10000cd30 con 0x7fd110070e20 2024-04-23T14:33:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.425+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11006d2b0 con 0x7fd110070e20 2024-04-23T14:33:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.426+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11006cae0 con 0x7fd110070e20 2024-04-23T14:33:37.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.427+0000 7fd107400640 1 -- 172.21.15.59:0/2311401138 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd10001a070 con 0x7fd110070e20 2024-04-23T14:33:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.428+0000 7fd107400640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0ec06c8f0 0x7fd0ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.428+0000 7fd107400640 1 -- 172.21.15.59:0/2311401138 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd10008bab0 con 0x7fd110070e20 2024-04-23T14:33:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.428+0000 7fd10ea00640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0ec06c8f0 0x7fd0ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.429+0000 7fd10ea00640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0ec06c8f0 0x7fd0ec06edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8034000 tx=0x7fd0f8033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.433+0000 7fd107400640 1 -- 172.21.15.59:0/2311401138 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd10005a1d0 con 0x7fd110070e20 2024-04-23T14:33:37.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.664+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd1100714d0 con 0x7fd110070e20 2024-04-23T14:33:37.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.671+0000 7fd107400640 1 -- 172.21.15.59:0/2311401138 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fd100059ad0 con 0x7fd110070e20 2024-04-23T14:33:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0ec06c8f0 msgr2=0x7fd0ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0ec06c8f0 0x7fd0ec06edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8034000 tx=0x7fd0f8033040 comp rx=0 tx=0).stop 2024-04-23T14:33:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd110070e20 msgr2=0x7fd1100cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd110070e20 0x7fd1100cb5c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e870 tx=0x7fd10000ed30 comp rx=0 tx=0).stop 2024-04-23T14:33:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 shutdown_connections 2024-04-23T14:33:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0ec06c8f0 0x7fd0ec06edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1100cd040 0x7fd11015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd11015c9e0 0x7fd1101548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 --2- 172.21.15.59:0/2311401138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd110070e20 0x7fd1100cb5c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 >> 172.21.15.59:0/2311401138 conn(0x7fd1100b3f20 msgr2=0x7fd1100c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:37.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 shutdown_connections 2024-04-23T14:33:37.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.674+0000 7fd115200640 1 -- 172.21.15.59:0/2311401138 wait complete. 2024-04-23T14:33:37.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-23T14:33:37.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.908+0000 7f069be00640 1 Processor -- start 2024-04-23T14:33:37.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.908+0000 7f069be00640 1 -- start start 2024-04-23T14:33:37.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.909+0000 7f069be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f069c0cd050 0x7f069c06f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f069c15ece0 0x7f069c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0b3080 0x7f069c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f069c0ced20 con 0x7f069c15ece0 2024-04-23T14:33:37.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f069c15fcd0 con 0x7f069c0cd050 2024-04-23T14:33:37.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f069c0cd970 con 0x7f069c0b3080 2024-04-23T14:33:37.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f069c15ece0 0x7f069c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f069c15ece0 0x7f069c0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44322/0 (socket says 172.21.15.59:44322) 2024-04-23T14:33:37.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069a000640 1 -- 172.21.15.59:0/3323726366 learned_addr learned my addr 172.21.15.59:0/3323726366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:37.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069aa00640 1 --2- 172.21.15.59:0/3323726366 >> v2:172.21.15.76:3300/0 conn(0x7f069c0cd050 0x7f069c06f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.910+0000 7f069b400640 1 --2- 172.21.15.59:0/3323726366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0b3080 0x7f069c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.911+0000 7f069a000640 1 -- 172.21.15.59:0/3323726366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0b3080 msgr2=0x7f069c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.911+0000 7f069a000640 1 --2- 172.21.15.59:0/3323726366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0b3080 0x7f069c0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.911+0000 7f069a000640 1 -- 172.21.15.59:0/3323726366 >> v2:172.21.15.76:3300/0 conn(0x7f069c0cd050 msgr2=0x7f069c06f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.911+0000 7f069a000640 1 --2- 172.21.15.59:0/3323726366 >> v2:172.21.15.76:3300/0 conn(0x7f069c0cd050 0x7f069c06f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.911+0000 7f069a000640 1 -- 172.21.15.59:0/3323726366 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c15f160 con 0x7f069c15ece0 2024-04-23T14:33:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.912+0000 7f069a000640 1 --2- 172.21.15.59:0/3323726366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0cd430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f069c06e080 tx=0x7f068802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82670dbf6dd50783 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.912+0000 7f0699600640 1 -- 172.21.15.59:0/3323726366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688036070 con 0x7f069c15ece0 2024-04-23T14:33:37.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.912+0000 7f0699600640 1 -- 172.21.15.59:0/3323726366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0688031600 con 0x7f069c15ece0 2024-04-23T14:33:37.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.912+0000 7f0699600640 1 -- 172.21.15.59:0/3323726366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880319d0 con 0x7f069c15ece0 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.913+0000 7f069be00640 1 -- 172.21.15.59:0/3323726366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 msgr2=0x7f069c0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.913+0000 7f069be00640 1 --2- 172.21.15.59:0/3323726366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0cd430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f069c06e080 tx=0x7f068802a610 comp rx=0 tx=0).stop 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.913+0000 7f069be00640 1 -- 172.21.15.59:0/3323726366 shutdown_connections 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.913+0000 7f069be00640 1 --2- 172.21.15.59:0/3323726366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0b3080 0x7f069c0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.913+0000 7f069be00640 1 --2- 172.21.15.59:0/3323726366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0cd430 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.913+0000 7f069be00640 1 --2- 172.21.15.59:0/3323726366 >> v2:172.21.15.76:3300/0 conn(0x7f069c0cd050 0x7f069c06f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.913+0000 7f069be00640 1 -- 172.21.15.59:0/3323726366 >> 172.21.15.59:0/3323726366 conn(0x7f069c0b21a0 msgr2=0x7f069c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.914+0000 7f069be00640 1 -- 172.21.15.59:0/3323726366 shutdown_connections 2024-04-23T14:33:37.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.914+0000 7f069be00640 1 -- 172.21.15.59:0/3323726366 wait complete. 2024-04-23T14:33:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.915+0000 7f069be00640 1 Processor -- start 2024-04-23T14:33:37.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.916+0000 7f069be00640 1 -- start start 2024-04-23T14:33:37.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f069c0b3080 0x7f069c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0cd050 0x7f069c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f069c15f9f0 con 0x7f069c15ece0 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f069c15f710 con 0x7f069c0b3080 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f069c0cef50 con 0x7f069c0cd050 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44326/0 (socket says 172.21.15.59:44326) 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 -- 172.21.15.59:0/2855093987 learned_addr learned my addr 172.21.15.59:0/2855093987 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069aa00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f069c0b3080 0x7f069c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069a000640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0cd050 0x7f069c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 -- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0cd050 msgr2=0x7f069c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0cd050 0x7f069c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 -- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f069c0b3080 msgr2=0x7f069c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f069c0b3080 0x7f069c0cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:37.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.917+0000 7f069b400640 1 -- 172.21.15.59:0/2855093987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c1f8010 con 0x7f069c15ece0 2024-04-23T14:33:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.918+0000 7f069aa00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f069c0b3080 0x7f069c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:33:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.918+0000 7f069b400640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0c51d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f069400d5d0 tx=0x7f069400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.918+0000 7f0692a00640 1 -- 172.21.15.59:0/2855093987 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940125f0 con 0x7f069c15ece0 2024-04-23T14:33:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.918+0000 7f0692a00640 1 -- 172.21.15.59:0/2855093987 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0694019070 con 0x7f069c15ece0 2024-04-23T14:33:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.918+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c1f82b0 con 0x7f069c15ece0 2024-04-23T14:33:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.918+0000 7f0692a00640 1 -- 172.21.15.59:0/2855093987 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940129f0 con 0x7f069c15ece0 2024-04-23T14:33:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.918+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c1f8870 con 0x7f069c15ece0 2024-04-23T14:33:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.920+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069c04fd90 con 0x7f069c15ece0 2024-04-23T14:33:37.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.921+0000 7f0692a00640 1 -- 172.21.15.59:0/2855093987 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0694012b90 con 0x7f069c15ece0 2024-04-23T14:33:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.922+0000 7f0692a00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f067c06c920 0x7f067c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.922+0000 7f0692a00640 1 -- 172.21.15.59:0/2855093987 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f069408bca0 con 0x7f069c15ece0 2024-04-23T14:33:37.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.922+0000 7f069aa00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f067c06c920 0x7f067c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:37.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.923+0000 7f069aa00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f067c06c920 0x7f067c06ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f068c009a70 tx=0x7f068c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:37.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:37.927+0000 7f0692a00640 1 -- 172.21.15.59:0/2855093987 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f069405a340 con 0x7f069c15ece0 2024-04-23T14:33:38.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:38.217+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f069c09e140 con 0x7f069c15ece0 2024-04-23T14:33:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.190+0000 7f0692a00640 1 -- 172.21.15.59:0/2855093987 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v137) ==== 142+0+0 (secure 0 0 0) 0x7f0694059c40 con 0x7f069c15ece0 2024-04-23T14:33:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f067c06c920 msgr2=0x7f067c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f067c06c920 0x7f067c06ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f068c009a70 tx=0x7f068c008040 comp rx=0 tx=0).stop 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 msgr2=0x7f069c0c51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0c51d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f069400d5d0 tx=0x7f069400da90 comp rx=0 tx=0).stop 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 shutdown_connections 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f067c06c920 0x7f067c06ede0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f069c0cd050 0x7f069c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f069c15ece0 0x7f069c0c51d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 --2- 172.21.15.59:0/2855093987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f069c0b3080 0x7f069c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.194+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 >> 172.21.15.59:0/2855093987 conn(0x7f069c0b21a0 msgr2=0x7f069c0ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.195+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 shutdown_connections 2024-04-23T14:33:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.195+0000 7f069be00640 1 -- 172.21.15.59:0/2855093987 wait complete. 2024-04-23T14:33:39.217 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-23T14:33:39.218 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-23T14:33:39.218 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-23T14:33:39.218 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:39.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.429+0000 7f3b39c00640 1 Processor -- start 2024-04-23T14:33:39.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.429+0000 7f3b39c00640 1 -- start start 2024-04-23T14:33:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.430+0000 7f3b39c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b340b1f60 0x7f3b340b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.430+0000 7f3b39c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b340d0ca0 con 0x7f3b340cce70 2024-04-23T14:33:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.430+0000 7f3b39c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b340cdb40 con 0x7f3b340d07c0 2024-04-23T14:33:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.430+0000 7f3b39c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3b340ce190 con 0x7f3b340b1f60 2024-04-23T14:33:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.431+0000 7f3b33e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3b340cce70 0x7f3b34162fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54648/0 (socket says 172.21.15.59:54648) 2024-04-23T14:33:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.431+0000 7f3b33e00640 1 -- 172.21.15.59:0/2115581809 learned_addr learned my addr 172.21.15.59:0/2115581809 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.431+0000 7f3b39200640 1 --2- 172.21.15.59:0/2115581809 >> v2:172.21.15.76:3300/0 conn(0x7f3b340b1f60 0x7f3b340b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.431+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141908178 0 0) 0x7f3b340d0ca0 con 0x7f3b340cce70 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.431+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b18003520 con 0x7f3b340cce70 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1573413324 0 0) 0x7f3b18003520 con 0x7f3b340cce70 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 >> v2:172.21.15.76:3300/0 conn(0x7f3b340b1f60 msgr2=0x7f3b340b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 --2- 172.21.15.59:0/2115581809 >> v2:172.21.15.76:3300/0 conn(0x7f3b340b1f60 0x7f3b340b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 >> v1:172.21.15.76:6789/0 conn(0x7f3b340d07c0 legacy=0x7f3b340ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b340cde20 con 0x7f3b340cce70 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f3b24001670 con 0x7f3b340cce70 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 >> v1:172.21.15.59:6789/0 conn(0x7f3b340cce70 legacy=0x7f3b34162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b180039b0 0x7f3b18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b18004800 0x7f3b18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340ce190 con 0x7f3b180051c0 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1800c060 con 0x7f3b180039b0 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1800c260 con 0x7f3b18004800 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.432+0000 7f3b39200640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b39200640 1 -- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b18004800 msgr2=0x7f3b18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b2b400640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b18004800 0x7f3b18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b33e00640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b180039b0 0x7f3b18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b39200640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b18004800 0x7f3b18004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b39200640 1 -- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b180039b0 msgr2=0x7f3b18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b39200640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b180039b0 0x7f3b18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b39200640 1 -- 172.21.15.59:0/2115581809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b18003520 con 0x7f3b180051c0 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.433+0000 7f3b33e00640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b180039b0 0x7f3b18003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.434+0000 7f3b39200640 1 -- 172.21.15.59:0/2115581809 reap_dead start 2024-04-23T14:33:39.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.434+0000 7f3b39200640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b1800b9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009fc0 tx=0x7f3b18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f64f0a41a8a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.434+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c035020 con 0x7f3b180051c0 2024-04-23T14:33:39.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.434+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2c003bd0 con 0x7f3b180051c0 2024-04-23T14:33:39.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.434+0000 7f3b33400640 1 -- 172.21.15.59:0/2115581809 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c009c10 con 0x7f3b180051c0 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.435+0000 7f3b39c00640 1 -- 172.21.15.59:0/2115581809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 msgr2=0x7f3b1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.435+0000 7f3b39c00640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b1800b9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009fc0 tx=0x7f3b18005830 comp rx=0 tx=0).stop 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.435+0000 7f3b39c00640 1 -- 172.21.15.59:0/2115581809 shutdown_connections 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.435+0000 7f3b39c00640 1 --2- 172.21.15.59:0/2115581809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b1800b9c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.435+0000 7f3b39c00640 1 -- 172.21.15.59:0/2115581809 >> 172.21.15.59:0/2115581809 conn(0x7f3b3409aa00 msgr2=0x7f3b340d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.435+0000 7f3b39c00640 1 -- 172.21.15.59:0/2115581809 shutdown_connections 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.435+0000 7f3b39c00640 1 -- 172.21.15.59:0/2115581809 wait complete. 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.437+0000 7f3b39c00640 1 Processor -- start 2024-04-23T14:33:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.437+0000 7f3b39c00640 1 -- start start 2024-04-23T14:33:39.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b39c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b341f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b39c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b340b23f0 0x7f3b340b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b39c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3415e840 0x7f3b3415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b39c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340d1db0 con 0x7f3b180051c0 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b39c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340cd250 con 0x7f3b3415e840 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b39c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3b340d1f30 con 0x7f3b340b23f0 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b341f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b341f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44346/0 (socket says 172.21.15.59:44346) 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.438+0000 7f3b33e00640 1 -- 172.21.15.59:0/4049648031 learned_addr learned my addr 172.21.15.59:0/4049648031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b33e00640 1 -- 172.21.15.59:0/4049648031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b340b23f0 msgr2=0x7f3b340b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b33e00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b340b23f0 0x7f3b340b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b33e00640 1 -- 172.21.15.59:0/4049648031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3415e840 msgr2=0x7f3b3415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b33e00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3415e840 0x7f3b3415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b33e00640 1 -- 172.21.15.59:0/4049648031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3415f5b0 con 0x7f3b180051c0 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b33e00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b341f3d00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400d840 tx=0x7f3b2400dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b30c00640 1 -- 172.21.15.59:0/4049648031 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24019070 con 0x7f3b180051c0 2024-04-23T14:33:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b30c00640 1 -- 172.21.15.59:0/4049648031 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b240074e0 con 0x7f3b180051c0 2024-04-23T14:33:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b30c00640 1 -- 172.21.15.59:0/4049648031 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240049e0 con 0x7f3b180051c0 2024-04-23T14:33:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b340ca490 con 0x7f3b180051c0 2024-04-23T14:33:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.439+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341f4240 con 0x7f3b180051c0 2024-04-23T14:33:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.441+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3404fd90 con 0x7f3b180051c0 2024-04-23T14:33:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.442+0000 7f3b30c00640 1 -- 172.21.15.59:0/4049648031 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b24004b80 con 0x7f3b180051c0 2024-04-23T14:33:39.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.443+0000 7f3b30c00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b1006c8f0 0x7f3b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.443+0000 7f3b30c00640 1 -- 172.21.15.59:0/4049648031 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f3b2408b210 con 0x7f3b180051c0 2024-04-23T14:33:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.443+0000 7f3b2b400640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b1006c8f0 0x7f3b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.444+0000 7f3b2b400640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b1006c8f0 0x7f3b1006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000f60 tx=0x7f3b1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.447+0000 7f3b30c00640 1 -- 172.21.15.59:0/4049648031 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b24059930 con 0x7f3b180051c0 2024-04-23T14:33:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.653+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b340d1b90 con 0x7f3b180051c0 2024-04-23T14:33:39.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.655+0000 7f3b30c00640 1 -- 172.21.15.59:0/4049648031 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+12960 (secure 0 0 0) 0x7f3b24059230 con 0x7f3b180051c0 2024-04-23T14:33:39.656 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:39.657 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:33:37.167994+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/1671400742":"2024-04-24T14:33:34.023740+0000","172.21.15.59:6835/1671400742":"2024-04-24T14:33:34.023740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b1006c8f0 msgr2=0x7f3b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b1006c8f0 0x7f3b1006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c000f60 tx=0x7f3b1c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 msgr2=0x7f3b341f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b341f3d00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400d840 tx=0x7f3b2400dd00 comp rx=0 tx=0).stop 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 shutdown_connections 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b1006c8f0 0x7f3b1006edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b340b23f0 0x7f3b340b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3415e840 0x7f3b3415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 --2- 172.21.15.59:0/4049648031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b180051c0 0x7f3b341f3d00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 >> 172.21.15.59:0/4049648031 conn(0x7f3b3409aa00 msgr2=0x7f3b3406ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 shutdown_connections 2024-04-23T14:33:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.658+0000 7f3b39c00640 1 -- 172.21.15.59:0/4049648031 wait complete. 2024-04-23T14:33:39.679 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:39.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.897+0000 7f2b5fc00640 1 Processor -- start 2024-04-23T14:33:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.897+0000 7f2b5fc00640 1 -- start start 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5fc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b5809aa00 0x7f2b5809ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b580ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5fc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b581593e0 con 0x7f2b5806cae0 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5fc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2b580ccbf0 con 0x7f2b5809aa00 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5809b320 con 0x7f2b5806cf60 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b5809aa00 0x7f2b5809ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b5809aa00 0x7f2b5809ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44356/0 (socket says 172.21.15.59:44356) 2024-04-23T14:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.899+0000 7f2b5d200640 1 -- 172.21.15.59:0/2161662397 learned_addr learned my addr 172.21.15.59:0/2161662397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:39.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.900+0000 7f2b5dc00640 1 --2- 172.21.15.59:0/2161662397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b580ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.900+0000 7f2b5d200640 1 -- 172.21.15.59:0/2161662397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 msgr2=0x7f2b580ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:39.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.900+0000 7f2b5d200640 1 --2- 172.21.15.59:0/2161662397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b580ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.900+0000 7f2b5d200640 1 -- 172.21.15.59:0/2161662397 >> v1:172.21.15.76:6789/0 conn(0x7f2b5806cae0 legacy=0x7f2b580cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.900+0000 7f2b5d200640 1 -- 172.21.15.59:0/2161662397 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b580ce9e0 con 0x7f2b5809aa00 2024-04-23T14:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.901+0000 7f2b5d200640 1 --2- 172.21.15.59:0/2161662397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5809aa00 0x7f2b5809ade0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009870 tx=0x7f2b4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c7f2bcc90b09e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.901+0000 7f2b57400640 1 -- 172.21.15.59:0/2161662397 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48037070 con 0x7f2b5809aa00 2024-04-23T14:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.901+0000 7f2b57400640 1 -- 172.21.15.59:0/2161662397 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b48032440 con 0x7f2b5809aa00 2024-04-23T14:33:39.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.901+0000 7f2b57400640 1 -- 172.21.15.59:0/2161662397 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48032810 con 0x7f2b5809aa00 2024-04-23T14:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 -- 172.21.15.59:0/2161662397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5809aa00 msgr2=0x7f2b5809ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/2161662397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5809aa00 0x7f2b5809ade0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009870 tx=0x7f2b4802a210 comp rx=0 tx=0).stop 2024-04-23T14:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 -- 172.21.15.59:0/2161662397 shutdown_connections 2024-04-23T14:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/2161662397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b580ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/2161662397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5809aa00 0x7f2b5809ade0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 -- 172.21.15.59:0/2161662397 >> 172.21.15.59:0/2161662397 conn(0x7f2b580b4090 msgr2=0x7f2b580b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 -- 172.21.15.59:0/2161662397 shutdown_connections 2024-04-23T14:33:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.902+0000 7f2b5fc00640 1 -- 172.21.15.59:0/2161662397 wait complete. 2024-04-23T14:33:39.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.905+0000 7f2b5fc00640 1 Processor -- start 2024-04-23T14:33:39.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.905+0000 7f2b5fc00640 1 -- start start 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.906+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5806cae0 0x7f2b5815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.906+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b58151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.906+0000 7f2b5fc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b5809aa00 0x7f2b58151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.906+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5806db20 con 0x7f2b5806cae0 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.906+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5806d840 con 0x7f2b5809aa00 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.906+0000 7f2b5fc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5806de00 con 0x7f2b5806cf60 2024-04-23T14:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5806cae0 0x7f2b5815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5806cae0 0x7f2b5815e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44366/0 (socket says 172.21.15.59:44366) 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 -- 172.21.15.59:0/823771125 learned_addr learned my addr 172.21.15.59:0/823771125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5dc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b5809aa00 0x7f2b58151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 -- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 msgr2=0x7f2b58151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b57e00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b58151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b58151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 -- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b5809aa00 msgr2=0x7f2b58151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b5809aa00 0x7f2b58151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5d200640 1 -- 172.21.15.59:0/823771125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b580ca680 con 0x7f2b5806cae0 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b5dc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b5809aa00 0x7f2b58151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.907+0000 7f2b57e00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b58151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.908+0000 7f2b5d200640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5806cae0 0x7f2b5815e390 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097c0 tx=0x7f2b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.908+0000 7f2b54c00640 1 -- 172.21.15.59:0/823771125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48041020 con 0x7f2b5806cae0 2024-04-23T14:33:39.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.908+0000 7f2b54c00640 1 -- 172.21.15.59:0/823771125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b480306c0 con 0x7f2b5806cae0 2024-04-23T14:33:39.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.908+0000 7f2b54c00640 1 -- 172.21.15.59:0/823771125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48037040 con 0x7f2b5806cae0 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.908+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b580ca3a0 con 0x7f2b5806cae0 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.908+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b580cac80 con 0x7f2b5806cae0 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.909+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b58068a80 con 0x7f2b5806cae0 2024-04-23T14:33:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.910+0000 7f2b54c00640 1 -- 172.21.15.59:0/823771125 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b48044020 con 0x7f2b5806cae0 2024-04-23T14:33:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.912+0000 7f2b54c00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b3806c8f0 0x7f2b3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.912+0000 7f2b54c00640 1 -- 172.21.15.59:0/823771125 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f2b480ac010 con 0x7f2b5806cae0 2024-04-23T14:33:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.912+0000 7f2b57e00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b3806c8f0 0x7f2b3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.913+0000 7f2b57e00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b3806c8f0 0x7f2b3806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2b581597d0 tx=0x7f2b44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:39.917+0000 7f2b54c00640 1 -- 172.21.15.59:0/823771125 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b4804b030 con 0x7f2b5806cae0 2024-04-23T14:33:40.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.125+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b5809b530 con 0x7f2b5806cae0 2024-04-23T14:33:40.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.126+0000 7f2b54c00640 1 -- 172.21.15.59:0/823771125 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f2b4807a740 con 0x7f2b5806cae0 2024-04-23T14:33:40.128 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:33:40.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b3806c8f0 msgr2=0x7f2b3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b3806c8f0 0x7f2b3806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2b581597d0 tx=0x7f2b44008040 comp rx=0 tx=0).stop 2024-04-23T14:33:40.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5806cae0 msgr2=0x7f2b5815e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5806cae0 0x7f2b5815e390 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2b480097c0 tx=0x7f2b480046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 shutdown_connections 2024-04-23T14:33:40.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b3806c8f0 0x7f2b3806edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5806cf60 0x7f2b58151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b5809aa00 0x7f2b58151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 --2- 172.21.15.59:0/823771125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5806cae0 0x7f2b5815e390 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 >> 172.21.15.59:0/823771125 conn(0x7f2b580b4090 msgr2=0x7f2b58070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:40.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 shutdown_connections 2024-04-23T14:33:40.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.129+0000 7f2b5fc00640 1 -- 172.21.15.59:0/823771125 wait complete. 2024-04-23T14:33:40.150 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-23T14:33:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.353+0000 7f2d10a00640 1 Processor -- start 2024-04-23T14:33:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.353+0000 7f2d10a00640 1 -- start start 2024-04-23T14:33:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.354+0000 7f2d10a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d0c06b9d0 0x7f2d0c0cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.355+0000 7f2d10a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d0c15d180 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.355+0000 7f2d10a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d0c15d460 con 0x7f2d0c0d0770 2024-04-23T14:33:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.355+0000 7f2d10a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2d0c0cff90 con 0x7f2d0c06b9d0 2024-04-23T14:33:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.355+0000 7f2d09e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d0c06b9d0 0x7f2d0c0cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.355+0000 7f2d09400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2d0c06bdb0 0x7f2d0c15c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39248/0 (socket says 172.21.15.59:39248) 2024-04-23T14:33:40.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.355+0000 7f2d09400640 1 -- 172.21.15.59:0/3034843489 learned_addr learned my addr 172.21.15.59:0/3034843489 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:40.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.356+0000 7f2d09e00640 1 -- 172.21.15.59:0/3034843489 >> v1:172.21.15.76:6789/0 conn(0x7f2d0c0d0770 legacy=0x7f2d0c160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.356+0000 7f2d09e00640 1 -- 172.21.15.59:0/3034843489 >> v1:172.21.15.59:6789/0 conn(0x7f2d0c06bdb0 legacy=0x7f2d0c15c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.356+0000 7f2d09e00640 1 -- 172.21.15.59:0/3034843489 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c15d740 con 0x7f2d0c06b9d0 2024-04-23T14:33:40.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.357+0000 7f2d09e00640 1 --2- 172.21.15.59:0/3034843489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06b9d0 0x7f2d0c0cfa50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009d90 tx=0x7f2cfc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b5bfacb59ff656 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.357+0000 7f2d08a00640 1 -- 172.21.15.59:0/3034843489 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007740 con 0x7f2d0c06b9d0 2024-04-23T14:33:40.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.357+0000 7f2d08a00640 1 -- 172.21.15.59:0/3034843489 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cfc0078e0 con 0x7f2d0c06b9d0 2024-04-23T14:33:40.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.357+0000 7f2d08a00640 1 -- 172.21.15.59:0/3034843489 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007cb0 con 0x7f2d0c06b9d0 2024-04-23T14:33:40.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.358+0000 7f2d10a00640 1 -- 172.21.15.59:0/3034843489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06b9d0 msgr2=0x7f2d0c0cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.358+0000 7f2d10a00640 1 --2- 172.21.15.59:0/3034843489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06b9d0 0x7f2d0c0cfa50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009d90 tx=0x7f2cfc02a6f0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.358+0000 7f2d10a00640 1 -- 172.21.15.59:0/3034843489 shutdown_connections 2024-04-23T14:33:40.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.358+0000 7f2d10a00640 1 --2- 172.21.15.59:0/3034843489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06b9d0 0x7f2d0c0cfa50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.358+0000 7f2d10a00640 1 -- 172.21.15.59:0/3034843489 >> 172.21.15.59:0/3034843489 conn(0x7f2d0c073430 msgr2=0x7f2d0c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:40.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.358+0000 7f2d10a00640 1 -- 172.21.15.59:0/3034843489 shutdown_connections 2024-04-23T14:33:40.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.358+0000 7f2d10a00640 1 -- 172.21.15.59:0/3034843489 wait complete. 2024-04-23T14:33:40.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.360+0000 7f2d10a00640 1 Processor -- start 2024-04-23T14:33:40.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.360+0000 7f2d10a00640 1 -- start start 2024-04-23T14:33:40.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d10a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c06b9d0 0x7f2d0c0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d10a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06bdb0 0x7f2d0c157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d10a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c0d0770 0x7f2d0c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d10a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c0cecb0 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d10a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c06c260 con 0x7f2d0c0d0770 2024-04-23T14:33:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d10a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0c0d0b80 con 0x7f2d0c06b9d0 2024-04-23T14:33:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d09400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06bdb0 0x7f2d0c157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d09400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06bdb0 0x7f2d0c157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60256/0 (socket says 172.21.15.59:60256) 2024-04-23T14:33:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.361+0000 7f2d09400640 1 -- 172.21.15.59:0/2255991301 learned_addr learned my addr 172.21.15.59:0/2255991301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d09e00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c06b9d0 0x7f2d0c0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d0a800640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c0d0770 0x7f2d0c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d09400640 1 -- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c06b9d0 msgr2=0x7f2d0c0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d09400640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c06b9d0 0x7f2d0c0cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d09400640 1 -- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c0d0770 msgr2=0x7f2d0c0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d09400640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c0d0770 0x7f2d0c0af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d09400640 1 -- 172.21.15.59:0/2255991301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c0b55e0 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d0a800640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c0d0770 0x7f2d0c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.362+0000 7f2d09400640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06bdb0 0x7f2d0c157b30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c4a0 tx=0x7f2cf400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.363+0000 7f2d02000640 1 -- 172.21.15.59:0/2255991301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4017070 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.363+0000 7f2d02000640 1 -- 172.21.15.59:0/2255991301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4005ce0 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.363+0000 7f2d02000640 1 -- 172.21.15.59:0/2255991301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007270 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.363+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c0b58c0 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.363+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c0b6160 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.364+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c09af30 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.365+0000 7f2d02000640 1 -- 172.21.15.59:0/2255991301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cf401a070 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.366+0000 7f2d02000640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ce806c8f0 0x7f2ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.366+0000 7f2d02000640 1 -- 172.21.15.59:0/2255991301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f2cf408ab90 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.366+0000 7f2d09e00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ce806c8f0 0x7f2ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.367+0000 7f2d09e00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ce806c8f0 0x7f2ce806edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034000 tx=0x7f2cfc02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.371+0000 7f2d02000640 1 -- 172.21.15.59:0/2255991301 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d0c09af30 con 0x7f2d0c06bdb0 2024-04-23T14:33:40.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:40.651+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f2d0c068a80 con 0x7f2d0c06bdb0 2024-04-23T14:33:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.206+0000 7f2d02000640 1 -- 172.21.15.59:0/2255991301 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v139) ==== 152+0+0 (secure 0 0 0) 0x7f2cf4058bd0 con 0x7f2d0c06bdb0 2024-04-23T14:33:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.209+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ce806c8f0 msgr2=0x7f2ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.209+0000 7f2d10a00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ce806c8f0 0x7f2ce806edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034000 tx=0x7f2cfc02f000 comp rx=0 tx=0).stop 2024-04-23T14:33:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06bdb0 msgr2=0x7f2d0c157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06bdb0 0x7f2d0c157b30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c4a0 tx=0x7f2cf400c960 comp rx=0 tx=0).stop 2024-04-23T14:33:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 shutdown_connections 2024-04-23T14:33:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ce806c8f0 0x7f2ce806edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c0d0770 0x7f2d0c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c06bdb0 0x7f2d0c157b30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 --2- 172.21.15.59:0/2255991301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c06b9d0 0x7f2d0c0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 >> 172.21.15.59:0/2255991301 conn(0x7f2d0c073430 msgr2=0x7f2d0c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:41.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 shutdown_connections 2024-04-23T14:33:41.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.210+0000 7f2d10a00640 1 -- 172.21.15.59:0/2255991301 wait complete. 2024-04-23T14:33:41.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:33:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.450+0000 7f7332a00640 1 Processor -- start 2024-04-23T14:33:41.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.450+0000 7f7332a00640 1 -- start start 2024-04-23T14:33:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.452+0000 7f7332a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c0d0d10 0x7f732c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.452+0000 7f7332a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f732c15d810 0x7f732c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.452+0000 7f7332a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732c15cab0 con 0x7f732c1567e0 2024-04-23T14:33:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.452+0000 7f7332a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f732c160e40 con 0x7f732c15d810 2024-04-23T14:33:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.452+0000 7f7332a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f732c15cd30 con 0x7f732c0d0d10 2024-04-23T14:33:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.453+0000 7f7330c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c0d0d10 0x7f732c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.453+0000 7f7332000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f732c15d810 0x7f732c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.453+0000 7f7330c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c0d0d10 0x7f732c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:50332/0 (socket says 172.21.15.59:50332) 2024-04-23T14:33:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.453+0000 7f7330c00640 1 -- 172.21.15.59:0/2427429002 learned_addr learned my addr 172.21.15.59:0/2427429002 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:41.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.453+0000 7f7331600640 1 --1- 172.21.15.59:0/2427429002 >> v1:172.21.15.76:6789/0 conn(0x7f732c1567e0 0x7f732c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:49416/0 (socket says 172.21.15.59:49416) 2024-04-23T14:33:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.454+0000 7f7332000640 1 -- 172.21.15.59:0/2427429002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c0d0d10 msgr2=0x7f732c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.454+0000 7f7332000640 1 --2- 172.21.15.59:0/2427429002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c0d0d10 0x7f732c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.454+0000 7f7332000640 1 -- 172.21.15.59:0/2427429002 >> v1:172.21.15.76:6789/0 conn(0x7f732c1567e0 legacy=0x7f732c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.454+0000 7f7332000640 1 -- 172.21.15.59:0/2427429002 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c15d010 con 0x7f732c15d810 2024-04-23T14:33:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.455+0000 7f7332000640 1 --2- 172.21.15.59:0/2427429002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c15d810 0x7f732c160110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7324009d90 tx=0x7f732402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1992280b2f6712 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f732be00640 1 -- 172.21.15.59:0/2427429002 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324007740 con 0x7f732c15d810 2024-04-23T14:33:41.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f732be00640 1 -- 172.21.15.59:0/2427429002 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73240078e0 con 0x7f732c15d810 2024-04-23T14:33:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f732be00640 1 -- 172.21.15.59:0/2427429002 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324007cb0 con 0x7f732c15d810 2024-04-23T14:33:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f7332a00640 1 -- 172.21.15.59:0/2427429002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c15d810 msgr2=0x7f732c160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f7332a00640 1 --2- 172.21.15.59:0/2427429002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c15d810 0x7f732c160110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7324009d90 tx=0x7f732402a660 comp rx=0 tx=0).stop 2024-04-23T14:33:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f7332a00640 1 -- 172.21.15.59:0/2427429002 shutdown_connections 2024-04-23T14:33:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f7332a00640 1 --2- 172.21.15.59:0/2427429002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c15d810 0x7f732c160110 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f7332a00640 1 --2- 172.21.15.59:0/2427429002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c0d0d10 0x7f732c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f7332a00640 1 -- 172.21.15.59:0/2427429002 >> 172.21.15.59:0/2427429002 conn(0x7f732c06bdf0 msgr2=0x7f732c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.456+0000 7f7332a00640 1 -- 172.21.15.59:0/2427429002 shutdown_connections 2024-04-23T14:33:41.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.457+0000 7f7332a00640 1 -- 172.21.15.59:0/2427429002 wait complete. 2024-04-23T14:33:41.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.458+0000 7f7332a00640 1 Processor -- start 2024-04-23T14:33:41.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.458+0000 7f7332a00640 1 -- start start 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.459+0000 7f7332a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c0d0d10 0x7f732c1e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.459+0000 7f7332a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c1567e0 0x7f732c09cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.459+0000 7f7332a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c15d810 0x7f732c099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.459+0000 7f7332a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f732c1607d0 con 0x7f732c0d0d10 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.459+0000 7f7332a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f732c160650 con 0x7f732c1567e0 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.459+0000 7f7332a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f732c160950 con 0x7f732c15d810 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c0d0d10 0x7f732c1e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c0d0d10 0x7f732c1e5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60264/0 (socket says 172.21.15.59:60264) 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 -- 172.21.15.59:0/2605145854 learned_addr learned my addr 172.21.15.59:0/2605145854 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:41.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7330c00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c1567e0 0x7f732c09cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7332000640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c15d810 0x7f732c099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 -- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c15d810 msgr2=0x7f732c099520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c15d810 0x7f732c099520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 -- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c1567e0 msgr2=0x7f732c09cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c1567e0 0x7f732c09cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.460+0000 7f7331600640 1 -- 172.21.15.59:0/2605145854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c06f580 con 0x7f732c0d0d10 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.461+0000 7f7331600640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c0d0d10 0x7f732c1e5f70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f732000c400 tx=0x7f732000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.461+0000 7f7329600640 1 -- 172.21.15.59:0/2605145854 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200074e0 con 0x7f732c0d0d10 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.461+0000 7f7329600640 1 -- 172.21.15.59:0/2605145854 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7320007680 con 0x7f732c0d0d10 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.461+0000 7f7329600640 1 -- 172.21.15.59:0/2605145854 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320004a00 con 0x7f732c0d0d10 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.461+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c06e4e0 con 0x7f732c0d0d10 2024-04-23T14:33:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.461+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c06ed80 con 0x7f732c0d0d10 2024-04-23T14:33:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.463+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c04fd90 con 0x7f732c0d0d10 2024-04-23T14:33:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.463+0000 7f7329600640 1 -- 172.21.15.59:0/2605145854 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f732001a020 con 0x7f732c0d0d10 2024-04-23T14:33:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.464+0000 7f7329600640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f730c06c8f0 0x7f730c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.465+0000 7f7329600640 1 -- 172.21.15.59:0/2605145854 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f732008afe0 con 0x7f732c0d0d10 2024-04-23T14:33:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.465+0000 7f7330c00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f730c06c8f0 0x7f730c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.465+0000 7f7330c00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f730c06c8f0 0x7f730c06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f732c156540 tx=0x7f7318009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:41.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.469+0000 7f7329600640 1 -- 172.21.15.59:0/2605145854 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f732c15cab0 con 0x7f732c0d0d10 2024-04-23T14:33:41.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.676+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f732c0b5150 con 0x7f732c0d0d10 2024-04-23T14:33:41.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.683+0000 7f7329600640 1 -- 172.21.15.59:0/2605145854 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f7320059760 con 0x7f732c0d0d10 2024-04-23T14:33:41.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f730c06c8f0 msgr2=0x7f730c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f730c06c8f0 0x7f730c06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f732c156540 tx=0x7f7318009240 comp rx=0 tx=0).stop 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c0d0d10 msgr2=0x7f732c1e5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c0d0d10 0x7f732c1e5f70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f732000c400 tx=0x7f732000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 shutdown_connections 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f730c06c8f0 0x7f730c06edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f732c15d810 0x7f732c099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f732c1567e0 0x7f732c09cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 --2- 172.21.15.59:0/2605145854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c0d0d10 0x7f732c1e5f70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 >> 172.21.15.59:0/2605145854 conn(0x7f732c06bdf0 msgr2=0x7f732c09a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.686+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 shutdown_connections 2024-04-23T14:33:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:41.687+0000 7f7332a00640 1 -- 172.21.15.59:0/2605145854 wait complete. 2024-04-23T14:33:41.708 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:33:41.708 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:33:41.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:33:41.908 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:33:41.908 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:33:41.908 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:41.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:33:41.970 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:33:41.970 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:42.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:33:42.028 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:33:42.028 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:42.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:33:42.087 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:33:42.087 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:33:42.144 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:33:42.145 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:33:42.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:42.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.362+0000 7ff341a00640 1 Processor -- start 2024-04-23T14:33:42.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.363+0000 7ff341a00640 1 -- start start 2024-04-23T14:33:42.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.364+0000 7ff341a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c1454e0 0x7ff33c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.364+0000 7ff341a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff33c160910 0x7ff33c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.364+0000 7ff341a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff33c0cd520 con 0x7ff33c0cd020 2024-04-23T14:33:42.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.364+0000 7ff341a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff33c0cddc0 con 0x7ff33c160910 2024-04-23T14:33:42.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.364+0000 7ff341a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff33c0cd800 con 0x7ff33c1454e0 2024-04-23T14:33:42.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.364+0000 7ff33b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff33c160910 0x7ff33c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff33c160910 0x7ff33c160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60274/0 (socket says 172.21.15.59:60274) 2024-04-23T14:33:42.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff33c0cd020 0x7ff33c0b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39252/0 (socket says 172.21.15.59:39252) 2024-04-23T14:33:42.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33b400640 1 -- 172.21.15.59:0/3851526618 learned_addr learned my addr 172.21.15.59:0/3851526618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:42.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33aa00640 1 --2- 172.21.15.59:0/3851526618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c1454e0 0x7ff33c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33b400640 1 -- 172.21.15.59:0/3851526618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c1454e0 msgr2=0x7ff33c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:42.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33b400640 1 --2- 172.21.15.59:0/3851526618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c1454e0 0x7ff33c145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33b400640 1 -- 172.21.15.59:0/3851526618 >> v1:172.21.15.59:6789/0 conn(0x7ff33c0cd020 legacy=0x7ff33c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff33b400640 1 -- 172.21.15.59:0/3851526618 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c0cdae0 con 0x7ff33c160910 2024-04-23T14:33:42.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.365+0000 7ff339600640 1 -- 172.21.15.59:0/3851526618 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974107797 0 0) 0x7ff33c0cd520 con 0x7ff33c0cd020 2024-04-23T14:33:42.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.366+0000 7ff33b400640 1 --2- 172.21.15.59:0/3851526618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c160910 0x7ff33c160da0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff334009870 tx=0x7ff33402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce72cdf66ca18e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.366+0000 7ff339600640 1 -- 172.21.15.59:0/3851526618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334037070 con 0x7ff33c160910 2024-04-23T14:33:42.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.366+0000 7ff339600640 1 -- 172.21.15.59:0/3851526618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff334032440 con 0x7ff33c160910 2024-04-23T14:33:42.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.366+0000 7ff339600640 1 -- 172.21.15.59:0/3851526618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334032810 con 0x7ff33c160910 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 -- 172.21.15.59:0/3851526618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c160910 msgr2=0x7ff33c160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 --2- 172.21.15.59:0/3851526618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c160910 0x7ff33c160da0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff334009870 tx=0x7ff33402a1c0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 -- 172.21.15.59:0/3851526618 shutdown_connections 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 --2- 172.21.15.59:0/3851526618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c160910 0x7ff33c160da0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 --2- 172.21.15.59:0/3851526618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c1454e0 0x7ff33c145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 -- 172.21.15.59:0/3851526618 >> 172.21.15.59:0/3851526618 conn(0x7ff33c067ea0 msgr2=0x7ff33c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 -- 172.21.15.59:0/3851526618 shutdown_connections 2024-04-23T14:33:42.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.367+0000 7ff341a00640 1 -- 172.21.15.59:0/3851526618 wait complete. 2024-04-23T14:33:42.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.369+0000 7ff341a00640 1 Processor -- start 2024-04-23T14:33:42.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.369+0000 7ff341a00640 1 -- start start 2024-04-23T14:33:42.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff341a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c0cd020 0x7ff33c142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff341a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff33c1454e0 0x7ff33c15db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff341a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c160910 0x7ff33c15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff341a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c0b4010 con 0x7ff33c0cd020 2024-04-23T14:33:42.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff341a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1460f0 con 0x7ff33c1454e0 2024-04-23T14:33:42.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff341a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff33c144690 con 0x7ff33c160910 2024-04-23T14:33:42.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff33aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c0cd020 0x7ff33c142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff33aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c0cd020 0x7ff33c142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60280/0 (socket says 172.21.15.59:60280) 2024-04-23T14:33:42.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.370+0000 7ff33aa00640 1 -- 172.21.15.59:0/197126309 learned_addr learned my addr 172.21.15.59:0/197126309 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33b400640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c160910 0x7ff33c15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33a000640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff33c1454e0 0x7ff33c15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33aa00640 1 -- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c160910 msgr2=0x7ff33c15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33aa00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c160910 0x7ff33c15e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33aa00640 1 -- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff33c1454e0 msgr2=0x7ff33c15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33aa00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff33c1454e0 0x7ff33c15db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33aa00640 1 -- 172.21.15.59:0/197126309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c1435f0 con 0x7ff33c0cd020 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33b400640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c160910 0x7ff33c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33a000640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff33c1454e0 0x7ff33c15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff33aa00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c0cd020 0x7ff33c142d30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff32c007f20 tx=0x7ff32c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.371+0000 7ff332a00640 1 -- 172.21.15.59:0/197126309 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c014620 con 0x7ff33c0cd020 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.372+0000 7ff332a00640 1 -- 172.21.15.59:0/197126309 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c019070 con 0x7ff33c0cd020 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.372+0000 7ff332a00640 1 -- 172.21.15.59:0/197126309 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c014a00 con 0x7ff33c0cd020 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.372+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c1438d0 con 0x7ff33c0cd020 2024-04-23T14:33:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.372+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c143e90 con 0x7ff33c0cd020 2024-04-23T14:33:42.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.373+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c0ce8e0 con 0x7ff33c0cd020 2024-04-23T14:33:42.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.374+0000 7ff332a00640 1 -- 172.21.15.59:0/197126309 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff32c014ba0 con 0x7ff33c0cd020 2024-04-23T14:33:42.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.375+0000 7ff332a00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff31806c8f0 0x7ff31806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.375+0000 7ff332a00640 1 -- 172.21.15.59:0/197126309 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ff32c08c180 con 0x7ff33c0cd020 2024-04-23T14:33:42.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.375+0000 7ff33a000640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff31806c8f0 0x7ff31806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.376+0000 7ff33a000640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff31806c8f0 0x7ff31806edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0cc670 tx=0x7ff324009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.380+0000 7ff332a00640 1 -- 172.21.15.59:0/197126309 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff33c0ce8e0 con 0x7ff33c0cd020 2024-04-23T14:33:42.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.625+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff33c068a80 con 0x7ff33c0cd020 2024-04-23T14:33:42.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.626+0000 7ff332a00640 1 -- 172.21.15.59:0/197126309 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3866 (secure 0 0 0) 0x7ff32c05a8a0 con 0x7ff33c0cd020 2024-04-23T14:33:42.627 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:42.628 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4181462383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4181462383},{"type":"v1","addr":"172.21.15.59:6837","nonce":4181462383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1383160131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1383160131},{"type":"v1","addr":"172.21.15.76:6833","nonce":1383160131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/927451554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":927451554},{"type":"v1","addr":"172.21.15.76:6835","nonce":927451554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2317720503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2317720503},{"type":"v1","addr":"172.21.15.59:6836","nonce":2317720503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-23T14:33:42.628 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 18 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff31806c8f0 msgr2=0x7ff31806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff31806c8f0 0x7ff31806edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0cc670 tx=0x7ff324009040 comp rx=0 tx=0).stop 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c0cd020 msgr2=0x7ff33c142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c0cd020 0x7ff33c142d30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff32c007f20 tx=0x7ff32c00e570 comp rx=0 tx=0).stop 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 shutdown_connections 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff31806c8f0 0x7ff31806edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff33c160910 0x7ff33c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff33c1454e0 0x7ff33c15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 --2- 172.21.15.59:0/197126309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff33c0cd020 0x7ff33c142d30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 >> 172.21.15.59:0/197126309 conn(0x7ff33c067ea0 msgr2=0x7ff33c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:42.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 shutdown_connections 2024-04-23T14:33:42.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.629+0000 7ff341a00640 1 -- 172.21.15.59:0/197126309 wait complete. 2024-04-23T14:33:42.649 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:33:42.649 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:33:42.649 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:33:42.649 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:42.650 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:33:42.650 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:33:42.650 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:33:42.650 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:42.650 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:33:42.650 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:33:42.650 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:33:42.650 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:42.651 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:33:42.651 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:33:42.651 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:33:42.651 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:33:42.670 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:33:42.670 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:33:42.672 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:33:42.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:33:42.673 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:33:42.673 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:33:42.677 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:33:42.677 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:33:42.717 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:33:42.717 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:42.721 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:33:42.722 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:42.723 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:33:42.723 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:42.727 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:33:42.728 DEBUG:teuthology.parallel:result is None 2024-04-23T14:33:42.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:33:42.869 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:33:42.870 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:33:42.876 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:33:42.876 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:33:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.911+0000 7f01da200640 1 Processor -- start 2024-04-23T14:33:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.911+0000 7f01da200640 1 -- start start 2024-04-23T14:33:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.912+0000 7f01da200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01d415e9c0 0x7f01d40d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.912+0000 7f01da200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d40cf3c0 con 0x7f01d40d1800 2024-04-23T14:33:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.912+0000 7f01da200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d40cf640 con 0x7f01d40d2d70 2024-04-23T14:33:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.912+0000 7f01da200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f01d40cf900 con 0x7f01d415e9c0 2024-04-23T14:33:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.912+0000 7f01d3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f01d40d1800 0x7f01d406cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39278/0 (socket says 172.21.15.59:39278) 2024-04-23T14:33:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.912+0000 7f01d3e00640 1 -- 172.21.15.59:0/1374261741 learned_addr learned my addr 172.21.15.59:0/1374261741 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.912+0000 7f01d3400640 1 --2- 172.21.15.59:0/1374261741 >> v2:172.21.15.59:3300/0 conn(0x7f01d415e9c0 0x7f01d40d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.913+0000 7f01d3400640 1 -- 172.21.15.59:0/1374261741 >> v1:172.21.15.76:6789/0 conn(0x7f01d40d2d70 legacy=0x7f01d40d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.913+0000 7f01d2000640 1 -- 172.21.15.59:0/1374261741 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825149128 0 0) 0x7f01d40cf640 con 0x7f01d40d2d70 2024-04-23T14:33:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.913+0000 7f01d3400640 1 -- 172.21.15.59:0/1374261741 >> v1:172.21.15.59:6789/0 conn(0x7f01d40d1800 legacy=0x7f01d406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.913+0000 7f01d3400640 1 -- 172.21.15.59:0/1374261741 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d40cfbe0 con 0x7f01d415e9c0 2024-04-23T14:33:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.914+0000 7f01d3400640 1 --2- 172.21.15.59:0/1374261741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40d03e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009790 tx=0x7f01c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95f424654770dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.914+0000 7f01d2000640 1 -- 172.21.15.59:0/1374261741 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4037070 con 0x7f01d415e9c0 2024-04-23T14:33:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.914+0000 7f01d2000640 1 -- 172.21.15.59:0/1374261741 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c4032440 con 0x7f01d415e9c0 2024-04-23T14:33:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.914+0000 7f01d2000640 1 -- 172.21.15.59:0/1374261741 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4032810 con 0x7f01d415e9c0 2024-04-23T14:33:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.915+0000 7f01da200640 1 -- 172.21.15.59:0/1374261741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 msgr2=0x7f01d40d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.915+0000 7f01da200640 1 --2- 172.21.15.59:0/1374261741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40d03e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009790 tx=0x7f01c402a180 comp rx=0 tx=0).stop 2024-04-23T14:33:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.915+0000 7f01da200640 1 -- 172.21.15.59:0/1374261741 shutdown_connections 2024-04-23T14:33:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.915+0000 7f01da200640 1 --2- 172.21.15.59:0/1374261741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40d03e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.915+0000 7f01da200640 1 -- 172.21.15.59:0/1374261741 >> 172.21.15.59:0/1374261741 conn(0x7f01d408bf90 msgr2=0x7f01d40d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.915+0000 7f01da200640 1 -- 172.21.15.59:0/1374261741 shutdown_connections 2024-04-23T14:33:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.915+0000 7f01da200640 1 -- 172.21.15.59:0/1374261741 wait complete. 2024-04-23T14:33:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.917+0000 7f01da200640 1 Processor -- start 2024-04-23T14:33:42.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.917+0000 7f01da200640 1 -- start start 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01da200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d40d1800 0x7f01d40b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01da200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d40d2d70 0x7f01d4153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01da200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01da200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01d40d3380 con 0x7f01d415e9c0 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01da200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01d40d3200 con 0x7f01d40d1800 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01da200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f01d40d3500 con 0x7f01d40d2d70 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60316/0 (socket says 172.21.15.59:60316) 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.918+0000 7f01d3e00640 1 -- 172.21.15.59:0/79896244 learned_addr learned my addr 172.21.15.59:0/79896244 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d3400640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d40d1800 0x7f01d40b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d2a00640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d40d2d70 0x7f01d4153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d3e00640 1 -- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d40d2d70 msgr2=0x7f01d4153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d3e00640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d40d2d70 0x7f01d4153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d3e00640 1 -- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d40d1800 msgr2=0x7f01d40b9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d3e00640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d40d1800 0x7f01d40b9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d3e00640 1 -- 172.21.15.59:0/79896244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d40cab40 con 0x7f01d415e9c0 2024-04-23T14:33:42.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.919+0000 7f01d3e00640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40c9ba0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00ca80 tx=0x7f01cc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.920+0000 7f01cb400640 1 -- 172.21.15.59:0/79896244 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc017070 con 0x7f01d415e9c0 2024-04-23T14:33:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.920+0000 7f01cb400640 1 -- 172.21.15.59:0/79896244 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc00ac30 con 0x7f01d415e9c0 2024-04-23T14:33:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.920+0000 7f01cb400640 1 -- 172.21.15.59:0/79896244 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc012600 con 0x7f01d415e9c0 2024-04-23T14:33:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.920+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d40cae20 con 0x7f01d415e9c0 2024-04-23T14:33:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.920+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d40b7430 con 0x7f01d415e9c0 2024-04-23T14:33:42.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.921+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d4068a80 con 0x7f01d415e9c0 2024-04-23T14:33:42.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.922+0000 7f01cb400640 1 -- 172.21.15.59:0/79896244 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01cc01a070 con 0x7f01d415e9c0 2024-04-23T14:33:42.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.923+0000 7f01cb400640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b006c8f0 0x7f01b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:42.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.923+0000 7f01cb400640 1 -- 172.21.15.59:0/79896244 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f01cc08b580 con 0x7f01d415e9c0 2024-04-23T14:33:42.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.923+0000 7f01d3400640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b006c8f0 0x7f01b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:42.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.924+0000 7f01d3400640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b006c8f0 0x7f01b006edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01c4034000 tx=0x7f01c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:42.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:42.928+0000 7f01cb400640 1 -- 172.21.15.59:0/79896244 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01d40cf3c0 con 0x7f01d415e9c0 2024-04-23T14:33:43.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.143+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f01d40d2630 con 0x7f01d415e9c0 2024-04-23T14:33:43.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.230+0000 7f01cb400640 1 -- 172.21.15.59:0/79896244 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7f01cc00ae20 con 0x7f01d415e9c0 2024-04-23T14:33:43.232 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.234+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b006c8f0 msgr2=0x7f01b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b006c8f0 0x7f01b006edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01c4034000 tx=0x7f01c402e000 comp rx=0 tx=0).stop 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 msgr2=0x7f01d40c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40c9ba0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00ca80 tx=0x7f01cc00cf40 comp rx=0 tx=0).stop 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 shutdown_connections 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b006c8f0 0x7f01b006edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d40d2d70 0x7f01d4153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d415e9c0 0x7f01d40c9ba0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 --2- 172.21.15.59:0/79896244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d40d1800 0x7f01d40b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 >> 172.21.15.59:0/79896244 conn(0x7f01d408bf90 msgr2=0x7f01d415f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 shutdown_connections 2024-04-23T14:33:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.235+0000 7f01da200640 1 -- 172.21.15.59:0/79896244 wait complete. 2024-04-23T14:33:43.258 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.459+0000 7fecfb200640 1 Processor -- start 2024-04-23T14:33:43.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.460+0000 7fecfb200640 1 -- start start 2024-04-23T14:33:43.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.461+0000 7fecfb200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fecfc0b5ff0 0x7fecfc0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.461+0000 7fecfb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 0x7fecfc0d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.461+0000 7fecfb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecfc0cf030 con 0x7fecfc15e7c0 2024-04-23T14:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.461+0000 7fecfb200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fecfc0b7000 con 0x7fecfc0b5ff0 2024-04-23T14:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.461+0000 7fecfb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fecfc0cf620 con 0x7fecfc0d0d90 2024-04-23T14:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.462+0000 7fecf9e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fecfc15e7c0 0x7fecfc0d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39280/0 (socket says 172.21.15.59:39280) 2024-04-23T14:33:43.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.462+0000 7fecf9e00640 1 -- 172.21.15.59:0/2263679916 learned_addr learned my addr 172.21.15.59:0/2263679916 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.462+0000 7fecfa800640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 0x7fecfc0d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.462+0000 7fecf3e00640 1 --2- 172.21.15.59:0/2263679916 >> v2:172.21.15.76:3300/0 conn(0x7fecfc0b5ff0 0x7fecfc0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.462+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771357736 0 0) 0x7fecec001110 con 0x7fecfc15e7c0 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.462+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fece0003520 con 0x7fecfc15e7c0 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1446919906 0 0) 0x7fece0003520 con 0x7fecfc15e7c0 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 msgr2=0x7fecfc0d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 0x7fecfc0d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 >> v2:172.21.15.76:3300/0 conn(0x7fecfc0b5ff0 msgr2=0x7fecfc0b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 --2- 172.21.15.59:0/2263679916 >> v2:172.21.15.76:3300/0 conn(0x7fecfc0b5ff0 0x7fecfc0b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecfc0b6a40 con 0x7fecfc15e7c0 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fecec001840 con 0x7fecfc15e7c0 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 >> v1:172.21.15.59:6789/0 conn(0x7fecfc15e7c0 legacy=0x7fecfc0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 0x7fece00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fece0003f80 0x7fece0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0004d40 0x7fece00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc0b7000 con 0x7fece0003f80 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc0cf620 con 0x7fece0004d40 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fece000b9c0 con 0x7fecfc0d0d90 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf3e00640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fece0003f80 0x7fece0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecfa800640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0004d40 0x7fece00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.463+0000 7fecf9e00640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 0x7fece00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.464+0000 7fecf3e00640 1 -- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 msgr2=0x7fece00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:43.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.464+0000 7fecf3e00640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 0x7fece00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.464+0000 7fecf3e00640 1 -- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0004d40 msgr2=0x7fece00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.464+0000 7fecf3e00640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0004d40 0x7fece00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.464+0000 7fecf3e00640 1 -- 172.21.15.59:0/2263679916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece0003520 con 0x7fece0003f80 2024-04-23T14:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.464+0000 7fecf9e00640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0d0d90 0x7fece00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:43.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.464+0000 7fecfa800640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0004d40 0x7fece00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.465+0000 7fecf3e00640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fece0003f80 0x7fece0004800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fece4009af0 tx=0x7fece0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb160382386f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.465+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4036070 con 0x7fece0003f80 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.465+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece4031400 con 0x7fece0003f80 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.465+0000 7fecf9400640 1 -- 172.21.15.59:0/2263679916 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4009690 con 0x7fece0003f80 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.465+0000 7fecfb200640 1 -- 172.21.15.59:0/2263679916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fece0003f80 msgr2=0x7fece0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.465+0000 7fecfb200640 1 --2- 172.21.15.59:0/2263679916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fece0003f80 0x7fece0004800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fece4009af0 tx=0x7fece0005830 comp rx=0 tx=0).stop 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.465+0000 7fecfa800640 1 -- 172.21.15.59:0/2263679916 reap_dead start 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.466+0000 7fecfb200640 1 -- 172.21.15.59:0/2263679916 shutdown_connections 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.466+0000 7fecfb200640 1 -- 172.21.15.59:0/2263679916 >> 172.21.15.59:0/2263679916 conn(0x7fecfc09d920 msgr2=0x7fecfc08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.466+0000 7fecfb200640 1 -- 172.21.15.59:0/2263679916 shutdown_connections 2024-04-23T14:33:43.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.466+0000 7fecfb200640 1 -- 172.21.15.59:0/2263679916 wait complete. 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.467+0000 7fecfb200640 1 Processor -- start 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.468+0000 7fecfb200640 1 -- start start 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.468+0000 7fecfb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0003f80 0x7fecfc152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.468+0000 7fecfb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0cc110 0x7fecfc1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecfc0cc580 0x7fecfc0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc0d2800 con 0x7fecfc0cc580 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc0cf2b0 con 0x7fece0003f80 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fecfc0d1a80 con 0x7fecfc0cc110 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecfc0cc580 0x7fecfc0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecfc0cc580 0x7fecfc0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60336/0 (socket says 172.21.15.59:60336) 2024-04-23T14:33:43.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 -- 172.21.15.59:0/77583977 learned_addr learned my addr 172.21.15.59:0/77583977 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecf3e00640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0cc110 0x7fecfc1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecf9e00640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0003f80 0x7fecfc152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 -- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0cc110 msgr2=0x7fecfc1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0cc110 0x7fecfc1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 -- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0003f80 msgr2=0x7fecfc152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0003f80 0x7fecfc152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecfa800640 1 -- 172.21.15.59:0/77583977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecfc153170 con 0x7fecfc0cc580 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.469+0000 7fecf3e00640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0cc110 0x7fecfc1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.470+0000 7fecfa800640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecfc0cc580 0x7fecfc0b5150 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fecfc08c6a0 tx=0x7fecf400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.470+0000 7fecf2000640 1 -- 172.21.15.59:0/77583977 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf401c070 con 0x7fecfc0cc580 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.470+0000 7fecf2000640 1 -- 172.21.15.59:0/77583977 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf4017400 con 0x7fecfc0cc580 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.470+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecfc0ccf60 con 0x7fecfc0cc580 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.470+0000 7fecf2000640 1 -- 172.21.15.59:0/77583977 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf40177d0 con 0x7fecfc0cc580 2024-04-23T14:33:43.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.470+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecfc1582a0 con 0x7fecfc0cc580 2024-04-23T14:33:43.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.471+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecfc04fd90 con 0x7fecfc0cc580 2024-04-23T14:33:43.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.472+0000 7fecf2000640 1 -- 172.21.15.59:0/77583977 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecf4005d10 con 0x7fecfc0cc580 2024-04-23T14:33:43.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.473+0000 7fecf2000640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fecd806c920 0x7fecd806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.474+0000 7fecf2000640 1 -- 172.21.15.59:0/77583977 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7fecf408b940 con 0x7fecfc0cc580 2024-04-23T14:33:43.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.474+0000 7fecf9e00640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fecd806c920 0x7fecd806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.474+0000 7fecf9e00640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fecd806c920 0x7fecd806ede0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fecec0013d0 tx=0x7fecec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:43.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.478+0000 7fecf2000640 1 -- 172.21.15.59:0/77583977 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fecfc153170 con 0x7fecfc0cc580 2024-04-23T14:33:43.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.767+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fecfc15d670 con 0x7fecfc0cc580 2024-04-23T14:33:43.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.768+0000 7fecf2000640 1 -- 172.21.15.59:0/77583977 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fecf4012070 con 0x7fecfc0cc580 2024-04-23T14:33:43.769 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:43.770 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-23T14:33:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-23T14:33:43.770 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A==", 2024-04-23T14:33:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:33:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:33:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.771 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-23T14:33:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA==", 2024-04-23T14:33:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.772 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:33:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:33:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-23T14:33:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw==", 2024-04-23T14:33:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:33:43.774 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:33:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-23T14:33:43.775 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg==", 2024-04-23T14:33:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:33:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:33:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-23T14:33:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow==", 2024-04-23T14:33:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.777 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-23T14:33:43.778 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ==", 2024-04-23T14:33:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-23T14:33:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ==", 2024-04-23T14:33:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-23T14:33:43.781 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g==", 2024-04-23T14:33:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-23T14:33:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA==", 2024-04-23T14:33:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.783 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA==", 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.784 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA==", 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-23T14:33:43.785 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ==", 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ==", 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q==", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg==", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:33:43.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg==", 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmEteHAhAAwfUHmvRmEMcPIphwikq6sw==", 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.788 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA==", 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:43.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ==", 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:33:43.790 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fecd806c920 msgr2=0x7fecd806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fecd806c920 0x7fecd806ede0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fecec0013d0 tx=0x7fecec009040 comp rx=0 tx=0).stop 2024-04-23T14:33:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecfc0cc580 msgr2=0x7fecfc0b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecfc0cc580 0x7fecfc0b5150 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fecfc08c6a0 tx=0x7fecf400f4a0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 shutdown_connections 2024-04-23T14:33:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fecd806c920 0x7fecd806ede0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecfc0cc110 0x7fecfc1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecfc0cc580 0x7fecfc0b5150 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 --2- 172.21.15.59:0/77583977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fece0003f80 0x7fecfc152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 >> 172.21.15.59:0/77583977 conn(0x7fecfc09d920 msgr2=0x7fecfc06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 shutdown_connections 2024-04-23T14:33:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.771+0000 7fecfb200640 1 -- 172.21.15.59:0/77583977 wait complete. 2024-04-23T14:33:43.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.996+0000 7f4f4e400640 1 Processor -- start 2024-04-23T14:33:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.996+0000 7f4f4e400640 1 -- start start 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.997+0000 7f4f4e400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4f4809b410 0x7f4f480af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.997+0000 7f4f4e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4f480d09e0 0x7f4f480d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.997+0000 7f4f4e400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f480b35f0 con 0x7f4f4815c9d0 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.997+0000 7f4f4e400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4f480cd2d0 con 0x7f4f480d09e0 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.997+0000 7f4f4e400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4f480d13b0 con 0x7f4f4809b410 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f4da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4f480d09e0 0x7f4f480d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f4d000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4f4815c9d0 0x7f4f480b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39288/0 (socket says 172.21.15.59:39288) 2024-04-23T14:33:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f4d000640 1 -- 172.21.15.59:0/2427343894 learned_addr learned my addr 172.21.15.59:0/2427343894 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f47e00640 1 --2- 172.21.15.59:0/2427343894 >> v2:172.21.15.76:3300/0 conn(0x7f4f4809b410 0x7f4f480af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f4da00640 1 -- 172.21.15.59:0/2427343894 >> v2:172.21.15.76:3300/0 conn(0x7f4f4809b410 msgr2=0x7f4f480af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f4da00640 1 --2- 172.21.15.59:0/2427343894 >> v2:172.21.15.76:3300/0 conn(0x7f4f4809b410 0x7f4f480af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f4da00640 1 -- 172.21.15.59:0/2427343894 >> v1:172.21.15.59:6789/0 conn(0x7f4f4815c9d0 legacy=0x7f4f480b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.998+0000 7f4f4da00640 1 -- 172.21.15.59:0/2427343894 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f480b3890 con 0x7f4f480d09e0 2024-04-23T14:33:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:43.999+0000 7f4f4da00640 1 --2- 172.21.15.59:0/2427343894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f480d09e0 0x7f4f480d0e70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4f4806c490 tx=0x7f4f4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10f65672f5a5b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f47400640 1 -- 172.21.15.59:0/2427343894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40037070 con 0x7f4f480d09e0 2024-04-23T14:33:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f47400640 1 -- 172.21.15.59:0/2427343894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f40032440 con 0x7f4f480d09e0 2024-04-23T14:33:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f47400640 1 -- 172.21.15.59:0/2427343894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40032810 con 0x7f4f480d09e0 2024-04-23T14:33:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 -- 172.21.15.59:0/2427343894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f480d09e0 msgr2=0x7f4f480d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 --2- 172.21.15.59:0/2427343894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f480d09e0 0x7f4f480d0e70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4f4806c490 tx=0x7f4f4002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 -- 172.21.15.59:0/2427343894 shutdown_connections 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 --2- 172.21.15.59:0/2427343894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f480d09e0 0x7f4f480d0e70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 --2- 172.21.15.59:0/2427343894 >> v2:172.21.15.76:3300/0 conn(0x7f4f4809b410 0x7f4f480af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 -- 172.21.15.59:0/2427343894 >> 172.21.15.59:0/2427343894 conn(0x7f4f480b3ef0 msgr2=0x7f4f480b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 -- 172.21.15.59:0/2427343894 shutdown_connections 2024-04-23T14:33:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.000+0000 7f4f4e400640 1 -- 172.21.15.59:0/2427343894 wait complete. 2024-04-23T14:33:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.002+0000 7f4f4e400640 1 Processor -- start 2024-04-23T14:33:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.002+0000 7f4f4e400640 1 -- start start 2024-04-23T14:33:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.003+0000 7f4f4e400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4f4809b410 0x7f4f480c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.003+0000 7f4f4e400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f480d09e0 0x7f4f480c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.003+0000 7f4f4e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f4815c9d0 0x7f4f4809f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.003+0000 7f4f4e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480ccef0 con 0x7f4f4815c9d0 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.003+0000 7f4f4e400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480af5d0 con 0x7f4f480d09e0 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.003+0000 7f4f4e400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4815cdb0 con 0x7f4f4809b410 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.003+0000 7f4f4da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f4815c9d0 0x7f4f4809f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f4815c9d0 0x7f4f4809f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60348/0 (socket says 172.21.15.59:60348) 2024-04-23T14:33:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 -- 172.21.15.59:0/123213183 learned_addr learned my addr 172.21.15.59:0/123213183 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 -- 172.21.15.59:0/123213183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4f4809b410 msgr2=0x7f4f480c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4f4809b410 0x7f4f480c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 -- 172.21.15.59:0/123213183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f480d09e0 msgr2=0x7f4f480c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f47e00640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f480d09e0 0x7f4f480c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f480d09e0 0x7f4f480c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 -- 172.21.15.59:0/123213183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f480c74c0 con 0x7f4f4815c9d0 2024-04-23T14:33:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f4da00640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f4815c9d0 0x7f4f4809f9e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4f40035040 tx=0x7f4f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f44c00640 1 -- 172.21.15.59:0/123213183 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40037040 con 0x7f4f4815c9d0 2024-04-23T14:33:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.004+0000 7f4f44c00640 1 -- 172.21.15.59:0/123213183 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f40042070 con 0x7f4f4815c9d0 2024-04-23T14:33:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.005+0000 7f4f44c00640 1 -- 172.21.15.59:0/123213183 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40030b30 con 0x7f4f4815c9d0 2024-04-23T14:33:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.005+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f480ca3c0 con 0x7f4f4815c9d0 2024-04-23T14:33:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.005+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f480c7920 con 0x7f4f4815c9d0 2024-04-23T14:33:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.006+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f480ca8e0 con 0x7f4f4815c9d0 2024-04-23T14:33:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.007+0000 7f4f44c00640 1 -- 172.21.15.59:0/123213183 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f40030cd0 con 0x7f4f4815c9d0 2024-04-23T14:33:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.007+0000 7f4f44c00640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4f2406c8f0 0x7f4f2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.008+0000 7f4f44c00640 1 -- 172.21.15.59:0/123213183 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f4f400ac320 con 0x7f4f4815c9d0 2024-04-23T14:33:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.008+0000 7f4f4d000640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4f2406c8f0 0x7f4f2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.009+0000 7f4f4d000640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4f2406c8f0 0x7f4f2406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000f60 tx=0x7f4f30007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.012+0000 7f4f44c00640 1 -- 172.21.15.59:0/123213183 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f40032e20 con 0x7f4f4815c9d0 2024-04-23T14:33:44.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.216+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f4804fd90 con 0x7f4f4815c9d0 2024-04-23T14:33:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.218+0000 7f4f44c00640 1 -- 172.21.15.59:0/123213183 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+12831 (secure 0 0 0) 0x7f4f40045360 con 0x7f4f4815c9d0 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:44.219 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":40,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:33:43.245808+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4f2406c8f0 msgr2=0x7f4f2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4f2406c8f0 0x7f4f2406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000f60 tx=0x7f4f30007a80 comp rx=0 tx=0).stop 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f4815c9d0 msgr2=0x7f4f4809f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f4815c9d0 0x7f4f4809f9e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4f40035040 tx=0x7f4f400046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 shutdown_connections 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4f2406c8f0 0x7f4f2406edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f4815c9d0 0x7f4f4809f9e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f480d09e0 0x7f4f480c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 --2- 172.21.15.59:0/123213183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4f4809b410 0x7f4f480c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 >> 172.21.15.59:0/123213183 conn(0x7f4f480b3ef0 msgr2=0x7f4f480cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.220+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 shutdown_connections 2024-04-23T14:33:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.221+0000 7f4f4e400640 1 -- 172.21.15.59:0/123213183 wait complete. 2024-04-23T14:33:44.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:33:44.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:33:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.446+0000 7f057be00640 1 Processor -- start 2024-04-23T14:33:44.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.446+0000 7f057be00640 1 -- start start 2024-04-23T14:33:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f057be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f057c0d0d10 0x7f057c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f057be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f057c15d810 0x7f057c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f057be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057c15cab0 con 0x7f057c1567e0 2024-04-23T14:33:44.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f057be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f057c160e40 con 0x7f057c0d0d10 2024-04-23T14:33:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f057be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f057c15cd30 con 0x7f057c15d810 2024-04-23T14:33:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f057aa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f057c1567e0 0x7f057c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39298/0 (socket says 172.21.15.59:39298) 2024-04-23T14:33:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f057aa00640 1 -- 172.21.15.59:0/2550395106 learned_addr learned my addr 172.21.15.59:0/2550395106 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.448+0000 7f0573e00640 1 --2- 172.21.15.59:0/2550395106 >> v2:172.21.15.59:3300/0 conn(0x7f057c0d0d10 0x7f057c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.449+0000 7f057b400640 1 --2- 172.21.15.59:0/2550395106 >> v2:172.21.15.76:3300/0 conn(0x7f057c15d810 0x7f057c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.449+0000 7f0573e00640 1 -- 172.21.15.59:0/2550395106 >> v2:172.21.15.76:3300/0 conn(0x7f057c15d810 msgr2=0x7f057c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.449+0000 7f0573e00640 1 --2- 172.21.15.59:0/2550395106 >> v2:172.21.15.76:3300/0 conn(0x7f057c15d810 0x7f057c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.449+0000 7f0573e00640 1 -- 172.21.15.59:0/2550395106 >> v1:172.21.15.59:6789/0 conn(0x7f057c1567e0 legacy=0x7f057c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.449+0000 7f0573e00640 1 -- 172.21.15.59:0/2550395106 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c15d010 con 0x7f057c0d0d10 2024-04-23T14:33:44.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.450+0000 7f0573e00640 1 --2- 172.21.15.59:0/2550395106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c0d0d10 0x7f057c156bc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f057c0b4cb0 tx=0x7f056402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbb90768725be23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.450+0000 7f057a000640 1 -- 172.21.15.59:0/2550395106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564037070 con 0x7f057c0d0d10 2024-04-23T14:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.450+0000 7f057a000640 1 -- 172.21.15.59:0/2550395106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564032440 con 0x7f057c0d0d10 2024-04-23T14:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.450+0000 7f057a000640 1 -- 172.21.15.59:0/2550395106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564032810 con 0x7f057c0d0d10 2024-04-23T14:33:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 -- 172.21.15.59:0/2550395106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c0d0d10 msgr2=0x7f057c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 --2- 172.21.15.59:0/2550395106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c0d0d10 0x7f057c156bc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f057c0b4cb0 tx=0x7f056402a1a0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 -- 172.21.15.59:0/2550395106 shutdown_connections 2024-04-23T14:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 --2- 172.21.15.59:0/2550395106 >> v2:172.21.15.76:3300/0 conn(0x7f057c15d810 0x7f057c160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 --2- 172.21.15.59:0/2550395106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c0d0d10 0x7f057c156bc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 -- 172.21.15.59:0/2550395106 >> 172.21.15.59:0/2550395106 conn(0x7f057c06bdf0 msgr2=0x7f057c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 -- 172.21.15.59:0/2550395106 shutdown_connections 2024-04-23T14:33:44.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.451+0000 7f057be00640 1 -- 172.21.15.59:0/2550395106 wait complete. 2024-04-23T14:33:44.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.453+0000 7f057be00640 1 Processor -- start 2024-04-23T14:33:44.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.453+0000 7f057be00640 1 -- start start 2024-04-23T14:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.454+0000 7f057be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f057c0d0d10 0x7f057c150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.454+0000 7f057be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c1567e0 0x7f057c157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.454+0000 7f057be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f057c15d810 0x7f057c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.454+0000 7f057be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1607d0 con 0x7f057c1567e0 2024-04-23T14:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.454+0000 7f057be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f057c160650 con 0x7f057c15d810 2024-04-23T14:33:44.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.454+0000 7f057be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f057c160950 con 0x7f057c0d0d10 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.454+0000 7f0573e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c1567e0 0x7f057c157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c1567e0 0x7f057c157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60378/0 (socket says 172.21.15.59:60378) 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 -- 172.21.15.59:0/2754266485 learned_addr learned my addr 172.21.15.59:0/2754266485 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f057aa00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f057c0d0d10 0x7f057c150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f057b400640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f057c15d810 0x7f057c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 -- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f057c0d0d10 msgr2=0x7f057c150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f057c0d0d10 0x7f057c150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 -- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f057c15d810 msgr2=0x7f057c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f057c15d810 0x7f057c06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 -- 172.21.15.59:0/2754266485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c159180 con 0x7f057c1567e0 2024-04-23T14:33:44.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.455+0000 7f0573e00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c1567e0 0x7f057c157170 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0564009890 tx=0x7f05640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.456+0000 7f0572a00640 1 -- 172.21.15.59:0/2754266485 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564037040 con 0x7f057c1567e0 2024-04-23T14:33:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.456+0000 7f0572a00640 1 -- 172.21.15.59:0/2754266485 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564042070 con 0x7f057c1567e0 2024-04-23T14:33:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.456+0000 7f0572a00640 1 -- 172.21.15.59:0/2754266485 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564030bd0 con 0x7f057c1567e0 2024-04-23T14:33:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.456+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c159460 con 0x7f057c1567e0 2024-04-23T14:33:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.456+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c159a20 con 0x7f057c1567e0 2024-04-23T14:33:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.457+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057c04fd90 con 0x7f057c1567e0 2024-04-23T14:33:44.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.458+0000 7f0572a00640 1 -- 172.21.15.59:0/2754266485 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0564047460 con 0x7f057c1567e0 2024-04-23T14:33:44.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.460+0000 7f0572a00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f055806c8f0 0x7f055806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.460+0000 7f0572a00640 1 -- 172.21.15.59:0/2754266485 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f05640ac510 con 0x7f057c1567e0 2024-04-23T14:33:44.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.460+0000 7f057aa00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f055806c8f0 0x7f055806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.461+0000 7f057aa00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f055806c8f0 0x7f055806edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f056c000f60 tx=0x7f056c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.464+0000 7f0572a00640 1 -- 172.21.15.59:0/2754266485 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0564032e20 con 0x7f057c1567e0 2024-04-23T14:33:44.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.671+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f057c0cf560 con 0x7f057c1567e0 2024-04-23T14:33:44.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.672+0000 7f0572a00640 1 -- 172.21.15.59:0/2754266485 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f056407ab60 con 0x7f057c1567e0 2024-04-23T14:33:44.673 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f055806c8f0 msgr2=0x7f055806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f055806c8f0 0x7f055806edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f056c000f60 tx=0x7f056c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c1567e0 msgr2=0x7f057c157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c1567e0 0x7f057c157170 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0564009890 tx=0x7f05640046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 shutdown_connections 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f055806c8f0 0x7f055806edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f057c15d810 0x7f057c06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f057c1567e0 0x7f057c157170 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 --2- 172.21.15.59:0/2754266485 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f057c0d0d10 0x7f057c150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 >> 172.21.15.59:0/2754266485 conn(0x7f057c06bdf0 msgr2=0x7f057c15a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 shutdown_connections 2024-04-23T14:33:44.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.675+0000 7f057be00640 1 -- 172.21.15.59:0/2754266485 wait complete. 2024-04-23T14:33:44.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:33:44.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.910+0000 7fa43de00640 1 Processor -- start 2024-04-23T14:33:44.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.910+0000 7fa43de00640 1 -- start start 2024-04-23T14:33:44.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.911+0000 7fa43de00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 0x7fa4380b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.911+0000 7fa43de00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4380d0d90 0x7fa4380d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.911+0000 7fa43de00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4380cf030 con 0x7fa43815e7c0 2024-04-23T14:33:44.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.911+0000 7fa43de00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa4380b7000 con 0x7fa4380d0d90 2024-04-23T14:33:44.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.911+0000 7fa43de00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa4380cf620 con 0x7fa4380b5ff0 2024-04-23T14:33:44.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.912+0000 7fa43ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa43815e7c0 0x7fa4380d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39310/0 (socket says 172.21.15.59:39310) 2024-04-23T14:33:44.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.912+0000 7fa43ca00640 1 -- 172.21.15.59:0/2355327045 learned_addr learned my addr 172.21.15.59:0/2355327045 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:44.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.912+0000 7fa42fe00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 0x7fa4380b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.912+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3362067446 0 0) 0x7fa428001490 con 0x7fa43815e7c0 2024-04-23T14:33:44.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.912+0000 7fa43d400640 1 --2- 172.21.15.59:0/2355327045 >> v2:172.21.15.76:3300/0 conn(0x7fa4380d0d90 0x7fa4380d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.912+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa41c003520 con 0x7fa43815e7c0 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3086374145 0 0) 0x7fa41c003520 con 0x7fa43815e7c0 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 msgr2=0x7fa4380b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 0x7fa4380b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 >> v2:172.21.15.76:3300/0 conn(0x7fa4380d0d90 msgr2=0x7fa4380d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 --2- 172.21.15.59:0/2355327045 >> v2:172.21.15.76:3300/0 conn(0x7fa4380d0d90 0x7fa4380d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4380b6a40 con 0x7fa43815e7c0 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa428001bc0 con 0x7fa43815e7c0 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 >> v1:172.21.15.59:6789/0 conn(0x7fa43815e7c0 legacy=0x7fa4380d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41c0039b0 0x7fa41c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 0x7fa41c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c004d40 0x7fa41c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4380b7000 con 0x7fa41c0039b0 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4380cf620 con 0x7fa41c004d40 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c00ba70 con 0x7fa4380b5ff0 2024-04-23T14:33:44.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.913+0000 7fa43ca00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41c0039b0 0x7fa41c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.914+0000 7fa42fe00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 0x7fa41c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.914+0000 7fa43d400640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c004d40 0x7fa41c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.914+0000 7fa43ca00640 1 -- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 msgr2=0x7fa41c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.914+0000 7fa43ca00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b5ff0 0x7fa41c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.914+0000 7fa43ca00640 1 -- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c004d40 msgr2=0x7fa41c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.914+0000 7fa43ca00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c004d40 0x7fa41c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.914+0000 7fa43ca00640 1 -- 172.21.15.59:0/2355327045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c003520 con 0x7fa41c0039b0 2024-04-23T14:33:44.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.915+0000 7fa43ca00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41c0039b0 0x7fa41c003dd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa42800aaa0 tx=0x7fa41c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8201b158723f9c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.915+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42800a690 con 0x7fa41c0039b0 2024-04-23T14:33:44.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.915+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa42802f070 con 0x7fa41c0039b0 2024-04-23T14:33:44.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.915+0000 7fa437e00640 1 -- 172.21.15.59:0/2355327045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428034470 con 0x7fa41c0039b0 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.916+0000 7fa43de00640 1 -- 172.21.15.59:0/2355327045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41c0039b0 msgr2=0x7fa41c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.916+0000 7fa43de00640 1 --2- 172.21.15.59:0/2355327045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa41c0039b0 0x7fa41c003dd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa42800aaa0 tx=0x7fa41c005830 comp rx=0 tx=0).stop 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.916+0000 7fa43d400640 1 -- 172.21.15.59:0/2355327045 reap_dead start 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.916+0000 7fa43de00640 1 -- 172.21.15.59:0/2355327045 shutdown_connections 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.916+0000 7fa43de00640 1 -- 172.21.15.59:0/2355327045 >> 172.21.15.59:0/2355327045 conn(0x7fa43809d920 msgr2=0x7fa43808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.916+0000 7fa43de00640 1 -- 172.21.15.59:0/2355327045 shutdown_connections 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.916+0000 7fa43de00640 1 -- 172.21.15.59:0/2355327045 wait complete. 2024-04-23T14:33:44.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.918+0000 7fa43de00640 1 Processor -- start 2024-04-23T14:33:44.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.918+0000 7fa43de00640 1 -- start start 2024-04-23T14:33:44.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa43de00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c0039b0 0x7fa438152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa43de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4381619b0 0x7fa4380b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa43de00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b9490 0x7fa4380b9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa43de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4380d2800 con 0x7fa4381619b0 2024-04-23T14:33:44.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa43de00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4380cf2b0 con 0x7fa41c0039b0 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa43de00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa4380d1a80 con 0x7fa4380b9490 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa42fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4381619b0 0x7fa4380b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa42fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4381619b0 0x7fa4380b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60388/0 (socket says 172.21.15.59:60388) 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.919+0000 7fa42fe00640 1 -- 172.21.15.59:0/2657397885 learned_addr learned my addr 172.21.15.59:0/2657397885 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa42fe00640 1 -- 172.21.15.59:0/2657397885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b9490 msgr2=0x7fa4380b9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa42fe00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b9490 0x7fa4380b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa42fe00640 1 -- 172.21.15.59:0/2657397885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c0039b0 msgr2=0x7fa438152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa42fe00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c0039b0 0x7fa438152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa42fe00640 1 -- 172.21.15.59:0/2657397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4380b83f0 con 0x7fa4381619b0 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa42fe00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4381619b0 0x7fa4380b7bb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e810 tx=0x7fa42000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa435600640 1 -- 172.21.15.59:0/2657397885 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420017070 con 0x7fa4381619b0 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa435600640 1 -- 172.21.15.59:0/2657397885 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa42000c6d0 con 0x7fa4381619b0 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa435600640 1 -- 172.21.15.59:0/2657397885 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42000f040 con 0x7fa4381619b0 2024-04-23T14:33:44.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.920+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4380b86d0 con 0x7fa4381619b0 2024-04-23T14:33:44.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.921+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4380b8c90 con 0x7fa4381619b0 2024-04-23T14:33:44.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.922+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43804fd90 con 0x7fa4381619b0 2024-04-23T14:33:44.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.923+0000 7fa435600640 1 -- 172.21.15.59:0/2657397885 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa42001a070 con 0x7fa4381619b0 2024-04-23T14:33:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.924+0000 7fa435600640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa41406c920 0x7fa41406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.924+0000 7fa435600640 1 -- 172.21.15.59:0/2657397885 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7fa42008b470 con 0x7fa4381619b0 2024-04-23T14:33:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.924+0000 7fa43ca00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa41406c920 0x7fa41406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:44.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.925+0000 7fa43ca00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa41406c920 0x7fa41406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa428033000 tx=0x7fa428009050 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:44.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:44.928+0000 7fa435600640 1 -- 172.21.15.59:0/2657397885 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa420059b50 con 0x7fa4381619b0 2024-04-23T14:33:45.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.137+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa43815cf70 con 0x7fa4381619b0 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.138+0000 7fa435600640 1 -- 172.21.15.59:0/2657397885 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) ==== 84+0+65 (secure 0 0 0) 0x7fa420059450 con 0x7fa4381619b0 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:45.139 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:33:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa41406c920 msgr2=0x7fa41406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa41406c920 0x7fa41406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa428033000 tx=0x7fa428009050 comp rx=0 tx=0).stop 2024-04-23T14:33:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4381619b0 msgr2=0x7fa4380b7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4381619b0 0x7fa4380b7bb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e810 tx=0x7fa42000ecd0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 shutdown_connections 2024-04-23T14:33:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa41406c920 0x7fa41406ede0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4380b9490 0x7fa4380b9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4381619b0 0x7fa4380b7bb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 --2- 172.21.15.59:0/2657397885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa41c0039b0 0x7fa438152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 >> 172.21.15.59:0/2657397885 conn(0x7fa43809d920 msgr2=0x7fa4380b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 shutdown_connections 2024-04-23T14:33:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.141+0000 7fa43de00640 1 -- 172.21.15.59:0/2657397885 wait complete. 2024-04-23T14:33:45.162 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:33:45.163 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.379+0000 7f551b200640 1 Processor -- start 2024-04-23T14:33:45.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.379+0000 7f551b200640 1 -- start start 2024-04-23T14:33:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c06bdf0 0x7f551c06c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f551c160100 0x7f551c160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551c157000 con 0x7f551c156c20 2024-04-23T14:33:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f551c0afb00 con 0x7f551c160100 2024-04-23T14:33:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f551c15cc20 con 0x7f551c06bdf0 2024-04-23T14:33:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f551c160100 0x7f551c160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f551c160100 0x7f551c160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60400/0 (socket says 172.21.15.59:60400) 2024-04-23T14:33:45.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.381+0000 7f551a800640 1 -- 172.21.15.59:0/1996030865 learned_addr learned my addr 172.21.15.59:0/1996030865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:45.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.382+0000 7f5519400640 1 --2- 172.21.15.59:0/1996030865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c06bdf0 0x7f551c06c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.382+0000 7f551a800640 1 -- 172.21.15.59:0/1996030865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c06bdf0 msgr2=0x7f551c06c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.382+0000 7f551a800640 1 --2- 172.21.15.59:0/1996030865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c06bdf0 0x7f551c06c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.382+0000 7f551a800640 1 -- 172.21.15.59:0/1996030865 >> v1:172.21.15.76:6789/0 conn(0x7f551c156c20 legacy=0x7f551c0aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:45.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.382+0000 7f551a800640 1 -- 172.21.15.59:0/1996030865 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c15cf00 con 0x7f551c160100 2024-04-23T14:33:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.383+0000 7f551a800640 1 --2- 172.21.15.59:0/1996030865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c160590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e50 tx=0x7f551402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e640cb7fba451e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.383+0000 7f5518a00640 1 -- 172.21.15.59:0/1996030865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514036070 con 0x7f551c160100 2024-04-23T14:33:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.383+0000 7f5518a00640 1 -- 172.21.15.59:0/1996030865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5514031600 con 0x7f551c160100 2024-04-23T14:33:45.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.383+0000 7f5518a00640 1 -- 172.21.15.59:0/1996030865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55140319d0 con 0x7f551c160100 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 -- 172.21.15.59:0/1996030865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 msgr2=0x7f551c160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 --2- 172.21.15.59:0/1996030865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c160590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e50 tx=0x7f551402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 -- 172.21.15.59:0/1996030865 shutdown_connections 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 --2- 172.21.15.59:0/1996030865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c160590 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 --2- 172.21.15.59:0/1996030865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c06bdf0 0x7f551c06c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 -- 172.21.15.59:0/1996030865 >> 172.21.15.59:0/1996030865 conn(0x7f551c0b2200 msgr2=0x7f551c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 -- 172.21.15.59:0/1996030865 shutdown_connections 2024-04-23T14:33:45.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.384+0000 7f551b200640 1 -- 172.21.15.59:0/1996030865 wait complete. 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.386+0000 7f551b200640 1 Processor -- start 2024-04-23T14:33:45.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.386+0000 7f551b200640 1 -- start start 2024-04-23T14:33:45.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551c06bdf0 0x7f551c1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c156c20 0x7f551c0998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c06efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f551c15d9e0 con 0x7f551c160100 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f551c15d4c0 con 0x7f551c06bdf0 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f551c073ba0 con 0x7f551c156c20 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c06efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c06efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60414/0 (socket says 172.21.15.59:60414) 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 -- 172.21.15.59:0/2160587236 learned_addr learned my addr 172.21.15.59:0/2160587236 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f5519e00640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551c06bdf0 0x7f551c1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f5519400640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c156c20 0x7f551c0998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 -- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c156c20 msgr2=0x7f551c0998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c156c20 0x7f551c0998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 -- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551c06bdf0 msgr2=0x7f551c1489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:45.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551c06bdf0 0x7f551c1489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f551a800640 1 -- 172.21.15.59:0/2160587236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c06fa00 con 0x7f551c160100 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.388+0000 7f5519e00640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551c06bdf0 0x7f551c1489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.389+0000 7f551a800640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c06efd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e50 tx=0x7f5514004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.389+0000 7f5512000640 1 -- 172.21.15.59:0/2160587236 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514041020 con 0x7f551c160100 2024-04-23T14:33:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.389+0000 7f5512000640 1 -- 172.21.15.59:0/2160587236 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5514037040 con 0x7f551c160100 2024-04-23T14:33:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.389+0000 7f5512000640 1 -- 172.21.15.59:0/2160587236 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514036070 con 0x7f551c160100 2024-04-23T14:33:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.389+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c147ed0 con 0x7f551c160100 2024-04-23T14:33:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.389+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c148490 con 0x7f551c160100 2024-04-23T14:33:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.390+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551c04fd90 con 0x7f551c160100 2024-04-23T14:33:45.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.391+0000 7f5512000640 1 -- 172.21.15.59:0/2160587236 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5514044020 con 0x7f551c160100 2024-04-23T14:33:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.392+0000 7f5512000640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f54fc06c8f0 0x7f54fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:45.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.393+0000 7f5512000640 1 -- 172.21.15.59:0/2160587236 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f55140ac530 con 0x7f551c160100 2024-04-23T14:33:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.393+0000 7f5519e00640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f54fc06c8f0 0x7f54fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:45.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.394+0000 7f5519e00640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f54fc06c8f0 0x7f54fc06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f550c007fd0 tx=0x7f550c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:45.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.397+0000 7f5512000640 1 -- 172.21.15.59:0/2160587236 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f551404b030 con 0x7f551c160100 2024-04-23T14:33:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:45.601+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f551c0b5090 con 0x7f551c160100 2024-04-23T14:33:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.258+0000 7f5512000640 1 -- 172.21.15.59:0/2160587236 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) ==== 138+0+0 (secure 0 0 0) 0x7f551407acd0 con 0x7f551c160100 2024-04-23T14:33:46.260 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.262+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f54fc06c8f0 msgr2=0x7f54fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.262+0000 7f551b200640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f54fc06c8f0 0x7f54fc06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f550c007fd0 tx=0x7f550c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.262+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 msgr2=0x7f551c06efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.262+0000 7f551b200640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c06efd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5514009e50 tx=0x7f5514004a40 comp rx=0 tx=0).stop 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 shutdown_connections 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f54fc06c8f0 0x7f54fc06edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f551c156c20 0x7f551c0998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551c160100 0x7f551c06efd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 --2- 172.21.15.59:0/2160587236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f551c06bdf0 0x7f551c1489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 >> 172.21.15.59:0/2160587236 conn(0x7f551c0b2200 msgr2=0x7f551c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:46.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 shutdown_connections 2024-04-23T14:33:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.263+0000 7f551b200640 1 -- 172.21.15.59:0/2160587236 wait complete. 2024-04-23T14:33:46.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:33:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.501+0000 7fd044e00640 1 Processor -- start 2024-04-23T14:33:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.502+0000 7fd044e00640 1 -- start start 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd044e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd040162b20 0x7fd040158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd044e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040158f80 0x7fd0400b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd044e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd04006f640 con 0x7fd040162740 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd044e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd040159410 con 0x7fd040162b20 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd044e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd04006f1b0 con 0x7fd040158f80 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd037e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd040162b20 0x7fd040158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd037e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd040162b20 0x7fd040158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60424/0 (socket says 172.21.15.59:60424) 2024-04-23T14:33:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.503+0000 7fd037e00640 1 -- 172.21.15.59:0/478712489 learned_addr learned my addr 172.21.15.59:0/478712489 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.504+0000 7fd03fe00640 1 --2- 172.21.15.59:0/478712489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040158f80 0x7fd0400b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.504+0000 7fd037e00640 1 -- 172.21.15.59:0/478712489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040158f80 msgr2=0x7fd0400b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.504+0000 7fd037e00640 1 --2- 172.21.15.59:0/478712489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040158f80 0x7fd0400b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.504+0000 7fd037e00640 1 -- 172.21.15.59:0/478712489 >> v1:172.21.15.76:6789/0 conn(0x7fd040162740 legacy=0x7fd04006dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.504+0000 7fd037e00640 1 -- 172.21.15.59:0/478712489 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04006e5a0 con 0x7fd040162b20 2024-04-23T14:33:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.505+0000 7fd037e00640 1 --2- 172.21.15.59:0/478712489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162b20 0x7fd040158a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd028009870 tx=0x7fd02802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192a4909e999965b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.505+0000 7fd03ea00640 1 -- 172.21.15.59:0/478712489 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028037070 con 0x7fd040162b20 2024-04-23T14:33:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.505+0000 7fd03ea00640 1 -- 172.21.15.59:0/478712489 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd028032440 con 0x7fd040162b20 2024-04-23T14:33:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.505+0000 7fd03ea00640 1 -- 172.21.15.59:0/478712489 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028032810 con 0x7fd040162b20 2024-04-23T14:33:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 -- 172.21.15.59:0/478712489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162b20 msgr2=0x7fd040158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 --2- 172.21.15.59:0/478712489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162b20 0x7fd040158a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd028009870 tx=0x7fd02802a210 comp rx=0 tx=0).stop 2024-04-23T14:33:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 -- 172.21.15.59:0/478712489 shutdown_connections 2024-04-23T14:33:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 --2- 172.21.15.59:0/478712489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040158f80 0x7fd0400b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 --2- 172.21.15.59:0/478712489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162b20 0x7fd040158a40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 -- 172.21.15.59:0/478712489 >> 172.21.15.59:0/478712489 conn(0x7fd04006bbb0 msgr2=0x7fd04006bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 -- 172.21.15.59:0/478712489 shutdown_connections 2024-04-23T14:33:46.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.506+0000 7fd044e00640 1 -- 172.21.15.59:0/478712489 wait complete. 2024-04-23T14:33:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.508+0000 7fd044e00640 1 Processor -- start 2024-04-23T14:33:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.508+0000 7fd044e00640 1 -- start start 2024-04-23T14:33:46.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd044e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd040158f80 0x7fd0400c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:46.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd044e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162740 0x7fd04009dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:46.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd044e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040162b20 0x7fd04009a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd044e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd04006ee40 con 0x7fd040162740 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd044e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd04006eb60 con 0x7fd040158f80 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd044e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd0400b44b0 con 0x7fd040162b20 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd037e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162740 0x7fd04009dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd037e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162740 0x7fd04009dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60426/0 (socket says 172.21.15.59:60426) 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.509+0000 7fd037e00640 1 -- 172.21.15.59:0/2529677243 learned_addr learned my addr 172.21.15.59:0/2529677243 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd03f400640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd040158f80 0x7fd0400c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd03fe00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040162b20 0x7fd04009a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037e00640 1 -- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040162b20 msgr2=0x7fd04009a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040162b20 0x7fd04009a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037e00640 1 -- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd040158f80 msgr2=0x7fd0400c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd040158f80 0x7fd0400c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037e00640 1 -- 172.21.15.59:0/2529677243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0400c5790 con 0x7fd040162740 2024-04-23T14:33:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162740 0x7fd04009dab0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd0280090b0 tx=0x7fd0280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037400640 1 -- 172.21.15.59:0/2529677243 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028041020 con 0x7fd040162740 2024-04-23T14:33:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037400640 1 -- 172.21.15.59:0/2529677243 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd028030990 con 0x7fd040162740 2024-04-23T14:33:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.510+0000 7fd037400640 1 -- 172.21.15.59:0/2529677243 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028037040 con 0x7fd040162740 2024-04-23T14:33:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.511+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0400c5a70 con 0x7fd040162740 2024-04-23T14:33:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.511+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0400c6310 con 0x7fd040162740 2024-04-23T14:33:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.512+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0400c5c40 con 0x7fd040162740 2024-04-23T14:33:46.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.514+0000 7fd037400640 1 -- 172.21.15.59:0/2529677243 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd028044020 con 0x7fd040162740 2024-04-23T14:33:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.515+0000 7fd037400640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd01c06c8f0 0x7fd01c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.515+0000 7fd037400640 1 -- 172.21.15.59:0/2529677243 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5227+0+0 (secure 0 0 0) 0x7fd0280ac7d0 con 0x7fd040162740 2024-04-23T14:33:46.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.515+0000 7fd03f400640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd01c06c8f0 0x7fd01c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.517+0000 7fd03f400640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd01c06c8f0 0x7fd01c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd030003d90 tx=0x7fd030009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.520+0000 7fd037400640 1 -- 172.21.15.59:0/2529677243 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd02804b030 con 0x7fd040162740 2024-04-23T14:33:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:46.721+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd04004fd90 con 0x7fd040162740 2024-04-23T14:33:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.274+0000 7fd037400640 1 -- 172.21.15.59:0/2529677243 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) ==== 170+0+0 (secure 0 0 0) 0x7fd02807af70 con 0x7fd040162740 2024-04-23T14:33:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:33:47.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd01c06c8f0 msgr2=0x7fd01c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd01c06c8f0 0x7fd01c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd030003d90 tx=0x7fd030009040 comp rx=0 tx=0).stop 2024-04-23T14:33:47.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162740 msgr2=0x7fd04009dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162740 0x7fd04009dab0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd0280090b0 tx=0x7fd0280046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 shutdown_connections 2024-04-23T14:33:47.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd01c06c8f0 0x7fd01c06edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd040162b20 0x7fd04009a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd040162740 0x7fd04009dab0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 --2- 172.21.15.59:0/2529677243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd040158f80 0x7fd0400c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.278+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 >> 172.21.15.59:0/2529677243 conn(0x7fd04006bbb0 msgr2=0x7fd04015ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:47.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.279+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 shutdown_connections 2024-04-23T14:33:47.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.279+0000 7fd044e00640 1 -- 172.21.15.59:0/2529677243 wait complete. 2024-04-23T14:33:47.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:33:47.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.496+0000 7fd68be00640 1 Processor -- start 2024-04-23T14:33:47.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.496+0000 7fd68be00640 1 -- start start 2024-04-23T14:33:47.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.498+0000 7fd68be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd68c0d0d10 0x7fd68c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.498+0000 7fd68be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd68c15d810 0x7fd68c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.498+0000 7fd68be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd68c15cab0 con 0x7fd68c1567e0 2024-04-23T14:33:47.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.498+0000 7fd68be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd68c160e40 con 0x7fd68c15d810 2024-04-23T14:33:47.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.498+0000 7fd68be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd68c15cd30 con 0x7fd68c0d0d10 2024-04-23T14:33:47.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.498+0000 7fd68b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd68c15d810 0x7fd68c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd68c15d810 0x7fd68c160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60428/0 (socket says 172.21.15.59:60428) 2024-04-23T14:33:47.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68b400640 1 -- 172.21.15.59:0/3396892988 learned_addr learned my addr 172.21.15.59:0/3396892988 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:47.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68a000640 1 --2- 172.21.15.59:0/3396892988 >> v2:172.21.15.76:3300/0 conn(0x7fd68c0d0d10 0x7fd68c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68b400640 1 -- 172.21.15.59:0/3396892988 >> v2:172.21.15.76:3300/0 conn(0x7fd68c0d0d10 msgr2=0x7fd68c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:47.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68b400640 1 --2- 172.21.15.59:0/3396892988 >> v2:172.21.15.76:3300/0 conn(0x7fd68c0d0d10 0x7fd68c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68b400640 1 -- 172.21.15.59:0/3396892988 >> v1:172.21.15.76:6789/0 conn(0x7fd68c1567e0 legacy=0x7fd68c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:47.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68b400640 1 -- 172.21.15.59:0/3396892988 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c15d010 con 0x7fd68c15d810 2024-04-23T14:33:47.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.499+0000 7fd68a000640 1 --2- 172.21.15.59:0/3396892988 >> v2:172.21.15.76:3300/0 conn(0x7fd68c0d0d10 0x7fd68c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:47.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.500+0000 7fd68b400640 1 --2- 172.21.15.59:0/3396892988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c15d810 0x7fd68c160110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd68c074140 tx=0x7fd68402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dd1206d02439f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.501+0000 7fd689600640 1 -- 172.21.15.59:0/3396892988 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684036070 con 0x7fd68c15d810 2024-04-23T14:33:47.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.501+0000 7fd689600640 1 -- 172.21.15.59:0/3396892988 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd684031600 con 0x7fd68c15d810 2024-04-23T14:33:47.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.501+0000 7fd689600640 1 -- 172.21.15.59:0/3396892988 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840319d0 con 0x7fd68c15d810 2024-04-23T14:33:47.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.501+0000 7fd68be00640 1 -- 172.21.15.59:0/3396892988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c15d810 msgr2=0x7fd68c160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.501+0000 7fd68be00640 1 --2- 172.21.15.59:0/3396892988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c15d810 0x7fd68c160110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd68c074140 tx=0x7fd68402a6f0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.502+0000 7fd68be00640 1 -- 172.21.15.59:0/3396892988 shutdown_connections 2024-04-23T14:33:47.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.502+0000 7fd68be00640 1 --2- 172.21.15.59:0/3396892988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c15d810 0x7fd68c160110 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.502+0000 7fd68be00640 1 --2- 172.21.15.59:0/3396892988 >> v2:172.21.15.76:3300/0 conn(0x7fd68c0d0d10 0x7fd68c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.502+0000 7fd68be00640 1 -- 172.21.15.59:0/3396892988 >> 172.21.15.59:0/3396892988 conn(0x7fd68c06bdf0 msgr2=0x7fd68c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:47.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.502+0000 7fd68be00640 1 -- 172.21.15.59:0/3396892988 shutdown_connections 2024-04-23T14:33:47.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.502+0000 7fd68be00640 1 -- 172.21.15.59:0/3396892988 wait complete. 2024-04-23T14:33:47.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.504+0000 7fd68be00640 1 Processor -- start 2024-04-23T14:33:47.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.504+0000 7fd68be00640 1 -- start start 2024-04-23T14:33:47.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd68c0d0d10 0x7fd68c14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c1567e0 0x7fd68c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68c15d810 0x7fd68c06f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1607d0 con 0x7fd68c1567e0 2024-04-23T14:33:47.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c160650 con 0x7fd68c15d810 2024-04-23T14:33:47.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd68c160950 con 0x7fd68c0d0d10 2024-04-23T14:33:47.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c1567e0 0x7fd68c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c1567e0 0x7fd68c1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60434/0 (socket says 172.21.15.59:60434) 2024-04-23T14:33:47.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.505+0000 7fd68a000640 1 -- 172.21.15.59:0/2846939056 learned_addr learned my addr 172.21.15.59:0/2846939056 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68b400640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68c15d810 0x7fd68c06f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68a000640 1 -- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd68c0d0d10 msgr2=0x7fd68c14b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68aa00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd68c0d0d10 0x7fd68c14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68a000640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd68c0d0d10 0x7fd68c14b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68a000640 1 -- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68c15d810 msgr2=0x7fd68c06f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68a000640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68c15d810 0x7fd68c06f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:47.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68a000640 1 -- 172.21.15.59:0/2846939056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c0735f0 con 0x7fd68c1567e0 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68b400640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68c15d810 0x7fd68c06f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68aa00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd68c0d0d10 0x7fd68c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68a000640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c1567e0 0x7fd68c1595b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c7f0 tx=0x7fd67400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd682a00640 1 -- 172.21.15.59:0/2846939056 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674015070 con 0x7fd68c1567e0 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd682a00640 1 -- 172.21.15.59:0/2846939056 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6740077c0 con 0x7fd68c1567e0 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd682a00640 1 -- 172.21.15.59:0/2846939056 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674007bd0 con 0x7fd68c1567e0 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.506+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c073310 con 0x7fd68c1567e0 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.507+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c06f040 con 0x7fd68c1567e0 2024-04-23T14:33:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.508+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68c04fd90 con 0x7fd68c1567e0 2024-04-23T14:33:47.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.509+0000 7fd682a00640 1 -- 172.21.15.59:0/2846939056 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd674018070 con 0x7fd68c1567e0 2024-04-23T14:33:47.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.510+0000 7fd682a00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd66806c8f0 0x7fd66806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:47.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.510+0000 7fd682a00640 1 -- 172.21.15.59:0/2846939056 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5609+0+0 (secure 0 0 0) 0x7fd67408bb30 con 0x7fd68c1567e0 2024-04-23T14:33:47.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.510+0000 7fd68aa00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd66806c8f0 0x7fd66806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:47.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.511+0000 7fd68aa00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd66806c8f0 0x7fd66806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1e8770 tx=0x7fd67c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:47.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.515+0000 7fd682a00640 1 -- 172.21.15.59:0/2846939056 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd674059fb0 con 0x7fd68c1567e0 2024-04-23T14:33:47.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:47.761+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd68c0b5150 con 0x7fd68c1567e0 2024-04-23T14:33:48.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.299+0000 7fd682a00640 1 -- 172.21.15.59:0/2846939056 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) ==== 172+0+0 (secure 0 0 0) 0x7fd6740598b0 con 0x7fd68c1567e0 2024-04-23T14:33:48.303 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 6 and data pool 7 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.311+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd66806c8f0 msgr2=0x7fd66806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.311+0000 7fd68be00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd66806c8f0 0x7fd66806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1e8770 tx=0x7fd67c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.311+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c1567e0 msgr2=0x7fd68c1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.311+0000 7fd68be00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c1567e0 0x7fd68c1595b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c7f0 tx=0x7fd67400ccb0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.312+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 shutdown_connections 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.312+0000 7fd68be00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd66806c8f0 0x7fd66806edb0 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1e8770 tx=0x7fd67c009040 comp rx=0 tx=0).stop 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.312+0000 7fd68be00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd68c15d810 0x7fd68c06f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.312+0000 7fd68be00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd68c1567e0 0x7fd68c1595b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.312+0000 7fd68be00640 1 --2- 172.21.15.59:0/2846939056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd68c0d0d10 0x7fd68c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.312+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 >> 172.21.15.59:0/2846939056 conn(0x7fd68c06bdf0 msgr2=0x7fd68c155440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:48.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.317+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 shutdown_connections 2024-04-23T14:33:48.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.317+0000 7fd68be00640 1 -- 172.21.15.59:0/2846939056 wait complete. 2024-04-23T14:33:48.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.524+0000 7fb5d3800640 1 Processor -- start 2024-04-23T14:33:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.524+0000 7fb5d3800640 1 -- start start 2024-04-23T14:33:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.526+0000 7fb5d3800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5cc1454e0 0x7fb5cc145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.526+0000 7fb5d3800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc0cd020 0x7fb5cc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.526+0000 7fb5d3800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5cc0cd520 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.526+0000 7fb5d3800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb5cc0cddc0 con 0x7fb5cc1454e0 2024-04-23T14:33:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.526+0000 7fb5d3800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb5cc0cd800 con 0x7fb5cc0cd020 2024-04-23T14:33:48.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.526+0000 7fb5cbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc0cd020 0x7fb5cc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.526+0000 7fb5d0e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5cc1454e0 0x7fb5cc145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5cbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc0cd020 0x7fb5cc0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:50550/0 (socket says 172.21.15.59:50550) 2024-04-23T14:33:48.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5cbe00640 1 -- 172.21.15.59:0/2277406566 learned_addr learned my addr 172.21.15.59:0/2277406566 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5d0e00640 1 -- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc0cd020 msgr2=0x7fb5cc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5d0e00640 1 --2- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc0cd020 0x7fb5cc0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5d0e00640 1 -- 172.21.15.59:0/2277406566 >> v1:172.21.15.76:6789/0 conn(0x7fb5cc0b3e40 legacy=0x7fb5cc160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5cb400640 1 -- 172.21.15.59:0/2277406566 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036711533 0 0) 0x7fb5cc0cd520 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5d0e00640 1 -- 172.21.15.59:0/2277406566 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc0cdae0 con 0x7fb5cc1454e0 2024-04-23T14:33:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.527+0000 7fb5cbe00640 1 --2- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc0cd020 0x7fb5cc0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.529+0000 7fb5d0e00640 1 --2- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc1454e0 0x7fb5cc145bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009d90 tx=0x7fb5bc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a37ee65732998db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.529+0000 7fb5cb400640 1 -- 172.21.15.59:0/2277406566 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc031600 con 0x7fb5cc1454e0 2024-04-23T14:33:48.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.529+0000 7fb5cb400640 1 -- 172.21.15.59:0/2277406566 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5bc0317a0 con 0x7fb5cc1454e0 2024-04-23T14:33:48.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.529+0000 7fb5cb400640 1 -- 172.21.15.59:0/2277406566 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc031b70 con 0x7fb5cc1454e0 2024-04-23T14:33:48.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 -- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc1454e0 msgr2=0x7fb5cc145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 --2- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc1454e0 0x7fb5cc145bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009d90 tx=0x7fb5bc02a5d0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 -- 172.21.15.59:0/2277406566 shutdown_connections 2024-04-23T14:33:48.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 --2- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc0cd020 0x7fb5cc0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 --2- 172.21.15.59:0/2277406566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc1454e0 0x7fb5cc145bb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 -- 172.21.15.59:0/2277406566 >> 172.21.15.59:0/2277406566 conn(0x7fb5cc067ea0 msgr2=0x7fb5cc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 -- 172.21.15.59:0/2277406566 shutdown_connections 2024-04-23T14:33:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.530+0000 7fb5d3800640 1 -- 172.21.15.59:0/2277406566 wait complete. 2024-04-23T14:33:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.532+0000 7fb5d3800640 1 Processor -- start 2024-04-23T14:33:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.532+0000 7fb5d3800640 1 -- start start 2024-04-23T14:33:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.533+0000 7fb5d3800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b3e40 0x7fb5cc15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.533+0000 7fb5d3800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0cd020 0x7fb5cc0a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.533+0000 7fb5d3800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc1454e0 0x7fb5cc13f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.533+0000 7fb5d3800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc146270 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.533+0000 7fb5d3800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1460f0 con 0x7fb5cc0cd020 2024-04-23T14:33:48.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.533+0000 7fb5d3800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb5cc161020 con 0x7fb5cc1454e0 2024-04-23T14:33:48.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.533+0000 7fb5d0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b3e40 0x7fb5cc15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b3e40 0x7fb5cc15ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60444/0 (socket says 172.21.15.59:60444) 2024-04-23T14:33:48.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 -- 172.21.15.59:0/3731111903 learned_addr learned my addr 172.21.15.59:0/3731111903 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 -- 172.21.15.59:0/3731111903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc1454e0 msgr2=0x7fb5cc13f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc1454e0 0x7fb5cc13f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 -- 172.21.15.59:0/3731111903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0cd020 msgr2=0x7fb5cc0a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0cd020 0x7fb5cc0a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 -- 172.21.15.59:0/3731111903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc15e400 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5d0e00640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b3e40 0x7fb5cc15ddb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009d90 tx=0x7fb5bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.534+0000 7fb5c8c00640 1 -- 172.21.15.59:0/3731111903 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc03f070 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.535+0000 7fb5c8c00640 1 -- 172.21.15.59:0/3731111903 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5bc030b60 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.535+0000 7fb5c8c00640 1 -- 172.21.15.59:0/3731111903 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc043410 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.535+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc15e6e0 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.535+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc15ef80 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.536+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5cc068a80 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.537+0000 7fb5c8c00640 1 -- 172.21.15.59:0/3731111903 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5bc0435b0 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.538+0000 7fb5c8c00640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb5a806c8f0 0x7fb5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.538+0000 7fb5c8c00640 1 -- 172.21.15.59:0/3731111903 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb5bc0acdf0 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.538+0000 7fb5cbe00640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb5a806c8f0 0x7fb5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.539+0000 7fb5cbe00640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb5a806c8f0 0x7fb5a806edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009630 tx=0x7fb5b4006e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.542+0000 7fb5c8c00640 1 -- 172.21.15.59:0/3731111903 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5bc07b180 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.746+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5cc160030 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.750+0000 7fb5c8c00640 1 -- 172.21.15.59:0/3731111903 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16566 (secure 0 0 0) 0x7fb5bc07aa80 con 0x7fb5cc0b3e40 2024-04-23T14:33:48.751 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:48.751 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":43,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:33:47.763694+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:45.619774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:46.736010+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb5a806c8f0 msgr2=0x7fb5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb5a806c8f0 0x7fb5a806edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009630 tx=0x7fb5b4006e90 comp rx=0 tx=0).stop 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b3e40 msgr2=0x7fb5cc15ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b3e40 0x7fb5cc15ddb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009d90 tx=0x7fb5bc004a40 comp rx=0 tx=0).stop 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 shutdown_connections 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb5a806c8f0 0x7fb5a806edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc1454e0 0x7fb5cc13f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc0cd020 0x7fb5cc0a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 --2- 172.21.15.59:0/3731111903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b3e40 0x7fb5cc15ddb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.752+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 >> 172.21.15.59:0/3731111903 conn(0x7fb5cc067ea0 msgr2=0x7fb5cc0b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.753+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 shutdown_connections 2024-04-23T14:33:48.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.753+0000 7fb5d3800640 1 -- 172.21.15.59:0/3731111903 wait complete. 2024-04-23T14:33:48.771 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:48.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.976+0000 7f30e4000640 1 Processor -- start 2024-04-23T14:33:48.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.976+0000 7f30e4000640 1 -- start start 2024-04-23T14:33:48.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.978+0000 7f30e4000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc0afaf0 0x7f30dc0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.978+0000 7f30e4000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30dc0d06e0 0x7f30dc1624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.978+0000 7f30e4000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30dc15f0f0 con 0x7f30dc15c5e0 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.978+0000 7f30e4000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f30dc15fa20 con 0x7f30dc0d06e0 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.978+0000 7f30e4000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30dc15f3d0 con 0x7f30dc0afaf0 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.979+0000 7f30e2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30dc0d06e0 0x7f30dc1624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.979+0000 7f30e0c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc0afaf0 0x7f30dc0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.979+0000 7f30e1600640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f30dc15c5e0 0x7f30dc0af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:49494/0 (socket says 172.21.15.59:49494) 2024-04-23T14:33:48.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.979+0000 7f30e2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30dc0d06e0 0x7f30dc1624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60448/0 (socket says 172.21.15.59:60448) 2024-04-23T14:33:48.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.979+0000 7f30e1600640 1 -- 172.21.15.59:0/3462696757 learned_addr learned my addr 172.21.15.59:0/3462696757 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:48.983 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:33:48.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.980+0000 7f30e2000640 1 -- 172.21.15.59:0/3462696757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc0afaf0 msgr2=0x7f30dc0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.980+0000 7f30dbe00640 1 -- 172.21.15.59:0/3462696757 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989451255 0 0) 0x7f30dc15f0f0 con 0x7f30dc15c5e0 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.980+0000 7f30e2000640 1 --2- 172.21.15.59:0/3462696757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc0afaf0 0x7f30dc0d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.980+0000 7f30e2000640 1 -- 172.21.15.59:0/3462696757 >> v1:172.21.15.76:6789/0 conn(0x7f30dc15c5e0 legacy=0x7f30dc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.980+0000 7f30e2000640 1 -- 172.21.15.59:0/3462696757 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30dc15f6b0 con 0x7f30dc0d06e0 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.980+0000 7f30e0c00640 1 --2- 172.21.15.59:0/3462696757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc0afaf0 0x7f30dc0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.981+0000 7f30e2000640 1 --2- 172.21.15.59:0/3462696757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0d06e0 0x7f30dc1624d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009d90 tx=0x7f30d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813c18d71c1a557 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.982+0000 7f30dbe00640 1 -- 172.21.15.59:0/3462696757 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4007740 con 0x7f30dc0d06e0 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.982+0000 7f30dbe00640 1 -- 172.21.15.59:0/3462696757 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d40078e0 con 0x7f30dc0d06e0 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.982+0000 7f30dbe00640 1 -- 172.21.15.59:0/3462696757 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4007cb0 con 0x7f30dc0d06e0 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.982+0000 7f30e4000640 1 -- 172.21.15.59:0/3462696757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0d06e0 msgr2=0x7f30dc1624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.982+0000 7f30e4000640 1 --2- 172.21.15.59:0/3462696757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0d06e0 0x7f30dc1624d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009d90 tx=0x7f30d402a660 comp rx=0 tx=0).stop 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.983+0000 7f30e4000640 1 -- 172.21.15.59:0/3462696757 shutdown_connections 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.983+0000 7f30e4000640 1 --2- 172.21.15.59:0/3462696757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0d06e0 0x7f30dc1624d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.983+0000 7f30e4000640 1 --2- 172.21.15.59:0/3462696757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc0afaf0 0x7f30dc0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.983+0000 7f30e4000640 1 -- 172.21.15.59:0/3462696757 >> 172.21.15.59:0/3462696757 conn(0x7f30dc067ea0 msgr2=0x7f30dc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:48.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.983+0000 7f30e4000640 1 -- 172.21.15.59:0/3462696757 shutdown_connections 2024-04-23T14:33:48.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.983+0000 7f30e4000640 1 -- 172.21.15.59:0/3462696757 wait complete. 2024-04-23T14:33:48.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.984+0000 7f30e4000640 1 Processor -- start 2024-04-23T14:33:48.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.985+0000 7f30e4000640 1 -- start start 2024-04-23T14:33:48.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.985+0000 7f30e4000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0afaf0 0x7f30dc0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.985+0000 7f30e4000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30dc0d06e0 0x7f30dc0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e4000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc15c5e0 0x7f30dc150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e4000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc06e600 con 0x7f30dc0afaf0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e4000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc06e480 con 0x7f30dc0d06e0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e4000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30dc162a10 con 0x7f30dc15c5e0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0afaf0 0x7f30dc0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0afaf0 0x7f30dc0aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60456/0 (socket says 172.21.15.59:60456) 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 -- 172.21.15.59:0/3633196811 learned_addr learned my addr 172.21.15.59:0/3633196811 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e0c00640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30dc0d06e0 0x7f30dc0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e2000640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc15c5e0 0x7f30dc150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 -- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc15c5e0 msgr2=0x7f30dc150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc15c5e0 0x7f30dc150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 -- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30dc0d06e0 msgr2=0x7f30dc0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30dc0d06e0 0x7f30dc0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e1600640 1 -- 172.21.15.59:0/3633196811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30dc0b05a0 con 0x7f30dc0afaf0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.986+0000 7f30e0c00640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30dc0d06e0 0x7f30dc0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.987+0000 7f30e1600640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0afaf0 0x7f30dc0aff10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c460 tx=0x7f30d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.987+0000 7f30d9600640 1 -- 172.21.15.59:0/3633196811 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0018070 con 0x7f30dc0afaf0 2024-04-23T14:33:48.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.987+0000 7f30d9600640 1 -- 172.21.15.59:0/3633196811 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d0007500 con 0x7f30dc0afaf0 2024-04-23T14:33:48.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.987+0000 7f30d9600640 1 -- 172.21.15.59:0/3633196811 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0004a20 con 0x7f30dc0afaf0 2024-04-23T14:33:48.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.987+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30dc0b0880 con 0x7f30dc0afaf0 2024-04-23T14:33:48.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.987+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30dc151220 con 0x7f30dc0afaf0 2024-04-23T14:33:48.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.988+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30dc068a80 con 0x7f30dc0afaf0 2024-04-23T14:33:48.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.990+0000 7f30d9600640 1 -- 172.21.15.59:0/3633196811 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30d001b070 con 0x7f30dc0afaf0 2024-04-23T14:33:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.992+0000 7f30d9600640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30bc06c8f0 0x7f30bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.992+0000 7f30d9600640 1 -- 172.21.15.59:0/3633196811 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f30d008aea0 con 0x7f30dc0afaf0 2024-04-23T14:33:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.992+0000 7f30e0c00640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30bc06c8f0 0x7f30bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:48.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.993+0000 7f30e0c00640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30bc06c8f0 0x7f30bc06edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0cc730 tx=0x7f30c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:48.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:48.997+0000 7f30d9600640 1 -- 172.21.15.59:0/3633196811 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30d0059230 con 0x7f30dc0afaf0 2024-04-23T14:33:49.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.208+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30dc06e260 con 0x7f30dc0afaf0 2024-04-23T14:33:49.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.210+0000 7f30d9600640 1 -- 172.21.15.59:0/3633196811 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16566 (secure 0 0 0) 0x7f30d0058b30 con 0x7f30dc0afaf0 2024-04-23T14:33:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:49.211 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":43,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:33:47.763694+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:45.619774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:46.736010+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.212+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30bc06c8f0 msgr2=0x7f30bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.212+0000 7f30e4000640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30bc06c8f0 0x7f30bc06edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0cc730 tx=0x7f30c8008040 comp rx=0 tx=0).stop 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.212+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0afaf0 msgr2=0x7f30dc0aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.212+0000 7f30e4000640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0afaf0 0x7f30dc0aff10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c460 tx=0x7f30d000c920 comp rx=0 tx=0).stop 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 shutdown_connections 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30bc06c8f0 0x7f30bc06edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30dc15c5e0 0x7f30dc150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30dc0d06e0 0x7f30dc0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 --2- 172.21.15.59:0/3633196811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30dc0afaf0 0x7f30dc0aff10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 >> 172.21.15.59:0/3633196811 conn(0x7f30dc067ea0 msgr2=0x7f30dc0adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 shutdown_connections 2024-04-23T14:33:49.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.213+0000 7f30e4000640 1 -- 172.21.15.59:0/3633196811 wait complete. 2024-04-23T14:33:49.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:33:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.482+0000 7f22d1c00640 1 Processor -- start 2024-04-23T14:33:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.483+0000 7f22d1c00640 1 -- start start 2024-04-23T14:33:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.484+0000 7f22d1c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc161c50 0x7f22cc0d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.484+0000 7f22d1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f22cc0d0de0 0x7f22cc0d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.484+0000 7f22d1c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22cc0d22e0 con 0x7f22cc0cfc50 2024-04-23T14:33:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.484+0000 7f22d1c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f22cc0d2ba0 con 0x7f22cc0d0de0 2024-04-23T14:33:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.484+0000 7f22d1c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22cc0d2550 con 0x7f22cc161c50 2024-04-23T14:33:49.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.484+0000 7f22cb400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f22cc0cfc50 0x7f22cc0d3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39314/0 (socket says 172.21.15.59:39314) 2024-04-23T14:33:49.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.484+0000 7f22cb400640 1 -- 172.21.15.59:0/2139704467 learned_addr learned my addr 172.21.15.59:0/2139704467 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:49.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22d1200640 1 --2- 172.21.15.59:0/2139704467 >> v2:172.21.15.59:3300/0 conn(0x7f22cc0d0de0 0x7f22cc0d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22cbe00640 1 --2- 172.21.15.59:0/2139704467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc161c50 0x7f22cc0d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22caa00640 1 -- 172.21.15.59:0/2139704467 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849934265 0 0) 0x7f22cc0d22e0 con 0x7f22cc0cfc50 2024-04-23T14:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22caa00640 1 -- 172.21.15.59:0/2139704467 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22b0003520 con 0x7f22cc0cfc50 2024-04-23T14:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22d1200640 1 -- 172.21.15.59:0/2139704467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc161c50 msgr2=0x7f22cc0d5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22d1200640 1 --2- 172.21.15.59:0/2139704467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc161c50 0x7f22cc0d5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22d1200640 1 -- 172.21.15.59:0/2139704467 >> v1:172.21.15.59:6789/0 conn(0x7f22cc0cfc50 legacy=0x7f22cc0d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.485+0000 7f22d1200640 1 -- 172.21.15.59:0/2139704467 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc0d2830 con 0x7f22cc0d0de0 2024-04-23T14:33:49.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.487+0000 7f22d1200640 1 --2- 172.21.15.59:0/2139704467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc0d0de0 0x7f22cc0d1270 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009d90 tx=0x7f22c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fcab4e3cb859a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.487+0000 7f22caa00640 1 -- 172.21.15.59:0/2139704467 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4007740 con 0x7f22cc0d0de0 2024-04-23T14:33:49.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.487+0000 7f22caa00640 1 -- 172.21.15.59:0/2139704467 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c40078e0 con 0x7f22cc0d0de0 2024-04-23T14:33:49.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.487+0000 7f22caa00640 1 -- 172.21.15.59:0/2139704467 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4007c90 con 0x7f22cc0d0de0 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 -- 172.21.15.59:0/2139704467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc0d0de0 msgr2=0x7f22cc0d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 --2- 172.21.15.59:0/2139704467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc0d0de0 0x7f22cc0d1270 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009d90 tx=0x7f22c402a660 comp rx=0 tx=0).stop 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 -- 172.21.15.59:0/2139704467 shutdown_connections 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 --2- 172.21.15.59:0/2139704467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc0d0de0 0x7f22cc0d1270 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 --2- 172.21.15.59:0/2139704467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc161c50 0x7f22cc0d5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 -- 172.21.15.59:0/2139704467 >> 172.21.15.59:0/2139704467 conn(0x7f22cc08bf90 msgr2=0x7f22cc0d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 -- 172.21.15.59:0/2139704467 shutdown_connections 2024-04-23T14:33:49.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.488+0000 7f22d1c00640 1 -- 172.21.15.59:0/2139704467 wait complete. 2024-04-23T14:33:49.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.490+0000 7f22d1c00640 1 Processor -- start 2024-04-23T14:33:49.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.490+0000 7f22d1c00640 1 -- start start 2024-04-23T14:33:49.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22cc0cfc50 0x7f22cc0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc0d0de0 0x7f22cc15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc161c50 0x7f22cc0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc0d0090 con 0x7f22cc161c50 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc0d0460 con 0x7f22cc0cfc50 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22cc0d0210 con 0x7f22cc0d0de0 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc161c50 0x7f22cc0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc161c50 0x7f22cc0ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60478/0 (socket says 172.21.15.59:60478) 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22d1200640 1 -- 172.21.15.59:0/3460052134 learned_addr learned my addr 172.21.15.59:0/3460052134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22cbe00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22cc0cfc50 0x7f22cc0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.491+0000 7f22cb400640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc0d0de0 0x7f22cc15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1200640 1 -- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc0d0de0 msgr2=0x7f22cc15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1200640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc0d0de0 0x7f22cc15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1200640 1 -- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22cc0cfc50 msgr2=0x7f22cc0c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1200640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22cc0cfc50 0x7f22cc0c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:49.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1200640 1 -- 172.21.15.59:0/3460052134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc0c8e00 con 0x7f22cc161c50 2024-04-23T14:33:49.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1200640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc161c50 0x7f22cc0ccd20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009eb0 tx=0x7f22c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22c3e00640 1 -- 172.21.15.59:0/3460052134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4040070 con 0x7f22cc161c50 2024-04-23T14:33:49.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22c3e00640 1 -- 172.21.15.59:0/3460052134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4031c60 con 0x7f22cc161c50 2024-04-23T14:33:49.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22c3e00640 1 -- 172.21.15.59:0/3460052134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4009450 con 0x7f22cc161c50 2024-04-23T14:33:49.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc0c90e0 con 0x7f22cc161c50 2024-04-23T14:33:49.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.492+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc0c96a0 con 0x7f22cc161c50 2024-04-23T14:33:49.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.493+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22cc04fd90 con 0x7f22cc161c50 2024-04-23T14:33:49.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.495+0000 7f22c3e00640 1 -- 172.21.15.59:0/3460052134 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22c4039040 con 0x7f22cc161c50 2024-04-23T14:33:49.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.495+0000 7f22c3e00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22ac06c920 0x7f22ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:49.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.496+0000 7f22c3e00640 1 -- 172.21.15.59:0/3460052134 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f22c40acfe0 con 0x7f22cc161c50 2024-04-23T14:33:49.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.496+0000 7f22cbe00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22ac06c920 0x7f22ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:49.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.496+0000 7f22cbe00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22ac06c920 0x7f22ac06ede0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f22bc006fd0 tx=0x7f22bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:49.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.500+0000 7f22c3e00640 1 -- 172.21.15.59:0/3460052134 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22c407b420 con 0x7f22cc161c50 2024-04-23T14:33:49.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:49.749+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f22cc0c9870 con 0x7f22cc161c50 2024-04-23T14:33:50.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.321+0000 7f22c3e00640 1 -- 172.21.15.59:0/3460052134 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f22c407ad20 con 0x7f22cc161c50 2024-04-23T14:33:50.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22ac06c920 msgr2=0x7f22ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22ac06c920 0x7f22ac06ede0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f22bc006fd0 tx=0x7f22bc008040 comp rx=0 tx=0).stop 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc161c50 msgr2=0x7f22cc0ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc161c50 0x7f22cc0ccd20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009eb0 tx=0x7f22c4004a40 comp rx=0 tx=0).stop 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 shutdown_connections 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22ac06c920 0x7f22ac06ede0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22cc0d0de0 0x7f22cc15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22cc161c50 0x7f22cc0ccd20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 --2- 172.21.15.59:0/3460052134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22cc0cfc50 0x7f22cc0c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.325+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 >> 172.21.15.59:0/3460052134 conn(0x7f22cc08bf90 msgr2=0x7f22cc0b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.326+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 shutdown_connections 2024-04-23T14:33:50.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.326+0000 7f22d1c00640 1 -- 172.21.15.59:0/3460052134 wait complete. 2024-04-23T14:33:50.350 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:33:50.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:33:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.567+0000 7f72b4e00640 1 Processor -- start 2024-04-23T14:33:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.567+0000 7f72b4e00640 1 -- start start 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72b4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f72b00d0d10 0x7f72b0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72b4e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b015d810 0x7f72b0160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72b4e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b015cab0 con 0x7f72b01567e0 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72b4e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f72b0160e40 con 0x7f72b00d0d10 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72b4e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f72b015cd30 con 0x7f72b015d810 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72af400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f72b01567e0 0x7f72b00d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37854/0 (socket says 172.21.15.59:37854) 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72af400640 1 -- 172.21.15.59:0/3759819296 learned_addr learned my addr 172.21.15.59:0/3759819296 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.569+0000 7f72aea00640 1 --2- 172.21.15.59:0/3759819296 >> v2:172.21.15.59:3300/0 conn(0x7f72b00d0d10 0x7f72b0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.570+0000 7f72afe00640 1 --2- 172.21.15.59:0/3759819296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b015d810 0x7f72b0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.570+0000 7f72aea00640 1 -- 172.21.15.59:0/3759819296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b015d810 msgr2=0x7f72b0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.570+0000 7f72aea00640 1 --2- 172.21.15.59:0/3759819296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b015d810 0x7f72b0160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.570+0000 7f72aea00640 1 -- 172.21.15.59:0/3759819296 >> v1:172.21.15.59:6789/0 conn(0x7f72b01567e0 legacy=0x7f72b00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.570+0000 7f72aea00640 1 -- 172.21.15.59:0/3759819296 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b015d010 con 0x7f72b00d0d10 2024-04-23T14:33:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.570+0000 7f72ae000640 1 -- 172.21.15.59:0/3759819296 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2899220795 0 0) 0x7f72a0001110 con 0x7f72b01567e0 2024-04-23T14:33:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.571+0000 7f72aea00640 1 --2- 172.21.15.59:0/3759819296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b00d0d10 0x7f72b0156bc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f729c0097a0 tx=0x7f729c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f26e493c86f914 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.571+0000 7f72ae000640 1 -- 172.21.15.59:0/3759819296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c037070 con 0x7f72b00d0d10 2024-04-23T14:33:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72ae000640 1 -- 172.21.15.59:0/3759819296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729c032440 con 0x7f72b00d0d10 2024-04-23T14:33:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72ae000640 1 -- 172.21.15.59:0/3759819296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c032810 con 0x7f72b00d0d10 2024-04-23T14:33:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 -- 172.21.15.59:0/3759819296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b00d0d10 msgr2=0x7f72b0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 --2- 172.21.15.59:0/3759819296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b00d0d10 0x7f72b0156bc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f729c0097a0 tx=0x7f729c02a180 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 -- 172.21.15.59:0/3759819296 shutdown_connections 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 --2- 172.21.15.59:0/3759819296 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b015d810 0x7f72b0160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 --2- 172.21.15.59:0/3759819296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b00d0d10 0x7f72b0156bc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 -- 172.21.15.59:0/3759819296 >> 172.21.15.59:0/3759819296 conn(0x7f72b006bdf0 msgr2=0x7f72b006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 -- 172.21.15.59:0/3759819296 shutdown_connections 2024-04-23T14:33:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.572+0000 7f72b4e00640 1 -- 172.21.15.59:0/3759819296 wait complete. 2024-04-23T14:33:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.574+0000 7f72b4e00640 1 Processor -- start 2024-04-23T14:33:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.574+0000 7f72b4e00640 1 -- start start 2024-04-23T14:33:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.575+0000 7f72b4e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b00d0d10 0x7f72b006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.575+0000 7f72b4e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b01567e0 0x7f72b01f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.575+0000 7f72b4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b015d810 0x7f72b01f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.575+0000 7f72b4e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72b01607d0 con 0x7f72b015d810 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.575+0000 7f72b4e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72b0160650 con 0x7f72b00d0d10 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.575+0000 7f72b4e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f72b0160950 con 0x7f72b01567e0 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b015d810 0x7f72b01f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b015d810 0x7f72b01f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41368/0 (socket says 172.21.15.59:41368) 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 -- 172.21.15.59:0/1131212181 learned_addr learned my addr 172.21.15.59:0/1131212181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72af400640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b00d0d10 0x7f72b006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72aea00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b01567e0 0x7f72b01f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 -- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b01567e0 msgr2=0x7f72b01f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b01567e0 0x7f72b01f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 -- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b00d0d10 msgr2=0x7f72b006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b00d0d10 0x7f72b006e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 -- 172.21.15.59:0/1131212181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b00af920 con 0x7f72b015d810 2024-04-23T14:33:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72aea00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b01567e0 0x7f72b01f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.576+0000 7f72afe00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b015d810 0x7f72b01f4910 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f72a8007e40 tx=0x7f72a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.577+0000 7f72a7400640 1 -- 172.21.15.59:0/1131212181 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8014620 con 0x7f72b015d810 2024-04-23T14:33:50.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.577+0000 7f72a7400640 1 -- 172.21.15.59:0/1131212181 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a8019070 con 0x7f72b015d810 2024-04-23T14:33:50.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.577+0000 7f72a7400640 1 -- 172.21.15.59:0/1131212181 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8014a20 con 0x7f72b015d810 2024-04-23T14:33:50.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.577+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b006eba0 con 0x7f72b015d810 2024-04-23T14:33:50.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.577+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b006f160 con 0x7f72b015d810 2024-04-23T14:33:50.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.578+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b004fd90 con 0x7f72b015d810 2024-04-23T14:33:50.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.579+0000 7f72a7400640 1 -- 172.21.15.59:0/1131212181 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72a8014bc0 con 0x7f72b015d810 2024-04-23T14:33:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.580+0000 7f72a7400640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f729006c8f0 0x7f729006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.580+0000 7f72a7400640 1 -- 172.21.15.59:0/1131212181 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f72a808c500 con 0x7f72b015d810 2024-04-23T14:33:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.580+0000 7f72af400640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f729006c8f0 0x7f729006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.581+0000 7f72af400640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f729006c8f0 0x7f729006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000f60 tx=0x7f72a0007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:50.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.585+0000 7f72a7400640 1 -- 172.21.15.59:0/1131212181 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72b00af920 con 0x7f72b015d810 2024-04-23T14:33:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:50.836+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f72b015be50 con 0x7f72b015d810 2024-04-23T14:33:51.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.333+0000 7f72a7400640 1 -- 172.21.15.59:0/1131212181 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f72a805a890 con 0x7f72b015d810 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.337+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f729006c8f0 msgr2=0x7f729006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.337+0000 7f72b4e00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f729006c8f0 0x7f729006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000f60 tx=0x7f72a0007970 comp rx=0 tx=0).stop 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.337+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b015d810 msgr2=0x7f72b01f4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.337+0000 7f72b4e00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b015d810 0x7f72b01f4910 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f72a8007e40 tx=0x7f72a800e4b0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 shutdown_connections 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f729006c8f0 0x7f729006edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72b01567e0 0x7f72b01f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72b015d810 0x7f72b01f4910 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 --2- 172.21.15.59:0/1131212181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b00d0d10 0x7f72b006e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 >> 172.21.15.59:0/1131212181 conn(0x7f72b006bdf0 msgr2=0x7f72b0072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 shutdown_connections 2024-04-23T14:33:51.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.338+0000 7f72b4e00640 1 -- 172.21.15.59:0/1131212181 wait complete. 2024-04-23T14:33:51.363 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:33:51.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.579+0000 7fc603200640 1 Processor -- start 2024-04-23T14:33:51.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.579+0000 7fc603200640 1 -- start start 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.580+0000 7fc603200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc6040cd760 0x7fc6040b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.580+0000 7fc603200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6040b4690 0x7fc6040cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.580+0000 7fc603200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc604160210 con 0x7fc60415c3d0 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc603200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc6041604f0 con 0x7fc6040cd760 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc603200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc6041607d0 con 0x7fc6040b4690 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc601e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc60415c3d0 0x7fc6040cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37870/0 (socket says 172.21.15.59:37870) 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc601e00640 1 -- 172.21.15.59:0/54032 learned_addr learned my addr 172.21.15.59:0/54032 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:51.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc601400640 1 --2- 172.21.15.59:0/54032 >> v2:172.21.15.59:3300/0 conn(0x7fc6040cd760 0x7fc6040b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc602800640 1 --2- 172.21.15.59:0/54032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6040b4690 0x7fc6040cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc600a00640 1 -- 172.21.15.59:0/54032 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 562885862 0 0) 0x7fc604160210 con 0x7fc60415c3d0 2024-04-23T14:33:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc600a00640 1 -- 172.21.15.59:0/54032 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5e8003520 con 0x7fc60415c3d0 2024-04-23T14:33:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc601400640 1 -- 172.21.15.59:0/54032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6040b4690 msgr2=0x7fc6040cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc601400640 1 --2- 172.21.15.59:0/54032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6040b4690 0x7fc6040cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.581+0000 7fc601400640 1 -- 172.21.15.59:0/54032 >> v1:172.21.15.59:6789/0 conn(0x7fc60415c3d0 legacy=0x7fc6040cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.582+0000 7fc601400640 1 -- 172.21.15.59:0/54032 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc604160ab0 con 0x7fc6040cd760 2024-04-23T14:33:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.583+0000 7fc601400640 1 --2- 172.21.15.59:0/54032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc6040b4150 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00097a0 tx=0x7fc5f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e938a42966703db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.583+0000 7fc600a00640 1 -- 172.21.15.59:0/54032 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0037070 con 0x7fc6040cd760 2024-04-23T14:33:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.583+0000 7fc600a00640 1 -- 172.21.15.59:0/54032 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f0032440 con 0x7fc6040cd760 2024-04-23T14:33:51.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.583+0000 7fc600a00640 1 -- 172.21.15.59:0/54032 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00327f0 con 0x7fc6040cd760 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 -- 172.21.15.59:0/54032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 msgr2=0x7fc6040b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 --2- 172.21.15.59:0/54032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc6040b4150 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00097a0 tx=0x7fc5f002a180 comp rx=0 tx=0).stop 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 -- 172.21.15.59:0/54032 shutdown_connections 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 --2- 172.21.15.59:0/54032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6040b4690 0x7fc6040cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 --2- 172.21.15.59:0/54032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc6040b4150 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 -- 172.21.15.59:0/54032 >> 172.21.15.59:0/54032 conn(0x7fc6040b21a0 msgr2=0x7fc6040b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 -- 172.21.15.59:0/54032 shutdown_connections 2024-04-23T14:33:51.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.584+0000 7fc603200640 1 -- 172.21.15.59:0/54032 wait complete. 2024-04-23T14:33:51.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.586+0000 7fc603200640 1 Processor -- start 2024-04-23T14:33:51.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.586+0000 7fc603200640 1 -- start start 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc603200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6040b4690 0x7fc604151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc603200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc60415d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc603200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc60415c3d0 0x7fc6040ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc603200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc60415c7b0 con 0x7fc6040cd760 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc603200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6040b4b00 con 0x7fc6040b4690 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc603200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc60415c930 con 0x7fc60415c3d0 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc601400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc60415d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc601400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc60415d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41392/0 (socket says 172.21.15.59:41392) 2024-04-23T14:33:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.587+0000 7fc601400640 1 -- 172.21.15.59:0/2754724233 learned_addr learned my addr 172.21.15.59:0/2754724233 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:51.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601400640 1 -- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc60415c3d0 msgr2=0x7fc6040ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601e00640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6040b4690 0x7fc604151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc602800640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc60415c3d0 0x7fc6040ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601400640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc60415c3d0 0x7fc6040ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601400640 1 -- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6040b4690 msgr2=0x7fc604151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601400640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6040b4690 0x7fc604151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601400640 1 -- 172.21.15.59:0/2754724233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60415dea0 con 0x7fc6040cd760 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601e00640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6040b4690 0x7fc604151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc601400640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc60415d470 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0035040 tx=0x7fc5f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.588+0000 7fc5fa000640 1 -- 172.21.15.59:0/2754724233 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0037040 con 0x7fc6040cd760 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.589+0000 7fc5fa000640 1 -- 172.21.15.59:0/2754724233 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f00044c0 con 0x7fc6040cd760 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.589+0000 7fc5fa000640 1 -- 172.21.15.59:0/2754724233 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00325f0 con 0x7fc6040cd760 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.589+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60415e180 con 0x7fc6040cd760 2024-04-23T14:33:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.589+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60415e740 con 0x7fc6040cd760 2024-04-23T14:33:51.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.590+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60404fd90 con 0x7fc6040cd760 2024-04-23T14:33:51.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.591+0000 7fc5fa000640 1 -- 172.21.15.59:0/2754724233 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5f0044020 con 0x7fc6040cd760 2024-04-23T14:33:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.592+0000 7fc5fa000640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc5e406c940 0x7fc5e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.592+0000 7fc5fa000640 1 -- 172.21.15.59:0/2754724233 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc5f00abdb0 con 0x7fc6040cd760 2024-04-23T14:33:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.593+0000 7fc601e00640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc5e406c940 0x7fc5e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.593+0000 7fc601e00640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc5e406c940 0x7fc5e406ee00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005880 tx=0x7fc5f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:51.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.597+0000 7fc5fa000640 1 -- 172.21.15.59:0/2754724233 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5f007a2c0 con 0x7fc6040cd760 2024-04-23T14:33:51.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.841+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6040cdd80 con 0x7fc6040cd760 2024-04-23T14:33:51.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.843+0000 7fc5fa000640 1 -- 172.21.15.59:0/2754724233 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7fc5f0079bc0 con 0x7fc6040cd760 2024-04-23T14:33:51.844 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:51.844 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:33:51.844 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:33:51.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc5e406c940 msgr2=0x7fc5e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc5e406c940 0x7fc5e406ee00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4005880 tx=0x7fc5f4009040 comp rx=0 tx=0).stop 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 msgr2=0x7fc60415d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc60415d470 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0035040 tx=0x7fc5f00046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 shutdown_connections 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc5e406c940 0x7fc5e406ee00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc60415c3d0 0x7fc6040ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6040cd760 0x7fc60415d470 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 --2- 172.21.15.59:0/2754724233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6040b4690 0x7fc604151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.845+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 >> 172.21.15.59:0/2754724233 conn(0x7fc6040b21a0 msgr2=0x7fc604073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.846+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 shutdown_connections 2024-04-23T14:33:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:51.846+0000 7fc603200640 1 -- 172.21.15.59:0/2754724233 wait complete. 2024-04-23T14:33:51.866 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:33:52.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.069+0000 7f6d90400640 1 Processor -- start 2024-04-23T14:33:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.069+0000 7f6d90400640 1 -- start start 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.070+0000 7f6d90400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d880afaf0 0x7f6d880d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.070+0000 7f6d90400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880d06e0 0x7f6d881624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.071+0000 7f6d90400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d8815f0f0 con 0x7f6d8815c5e0 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.071+0000 7f6d90400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8815fa20 con 0x7f6d880afaf0 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.071+0000 7f6d90400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8815f3d0 con 0x7f6d880d06e0 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.071+0000 7f6d8e400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880d06e0 0x7f6d881624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.071+0000 7f6d8e400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880d06e0 0x7f6d881624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:43568/0 (socket says 172.21.15.59:43568) 2024-04-23T14:33:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.071+0000 7f6d8e400640 1 -- 172.21.15.59:0/1911699285 learned_addr learned my addr 172.21.15.59:0/1911699285 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.071+0000 7f6d8d000640 1 --2- 172.21.15.59:0/1911699285 >> v2:172.21.15.76:3300/0 conn(0x7f6d880afaf0 0x7f6d880d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.072+0000 7f6d8e400640 1 -- 172.21.15.59:0/1911699285 >> v2:172.21.15.76:3300/0 conn(0x7f6d880afaf0 msgr2=0x7f6d880d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.072+0000 7f6d8e400640 1 --2- 172.21.15.59:0/1911699285 >> v2:172.21.15.76:3300/0 conn(0x7f6d880afaf0 0x7f6d880d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.072+0000 7f6d8e400640 1 -- 172.21.15.59:0/1911699285 >> v1:172.21.15.76:6789/0 conn(0x7f6d8815c5e0 legacy=0x7f6d880af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.072+0000 7f6d8e400640 1 -- 172.21.15.59:0/1911699285 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8815f6b0 con 0x7f6d880d06e0 2024-04-23T14:33:52.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.073+0000 7f6d8e400640 1 --2- 172.21.15.59:0/1911699285 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880d06e0 0x7f6d881624d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009870 tx=0x7f6d8002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d259438efb631c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.074+0000 7f6d87e00640 1 -- 172.21.15.59:0/1911699285 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80037070 con 0x7f6d880d06e0 2024-04-23T14:33:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.074+0000 7f6d87e00640 1 -- 172.21.15.59:0/1911699285 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d80032440 con 0x7f6d880d06e0 2024-04-23T14:33:52.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.074+0000 7f6d87e00640 1 -- 172.21.15.59:0/1911699285 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80032810 con 0x7f6d880d06e0 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 -- 172.21.15.59:0/1911699285 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880d06e0 msgr2=0x7f6d881624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 --2- 172.21.15.59:0/1911699285 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880d06e0 0x7f6d881624d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009870 tx=0x7f6d8002a250 comp rx=0 tx=0).stop 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 -- 172.21.15.59:0/1911699285 shutdown_connections 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 --2- 172.21.15.59:0/1911699285 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880d06e0 0x7f6d881624d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 --2- 172.21.15.59:0/1911699285 >> v2:172.21.15.76:3300/0 conn(0x7f6d880afaf0 0x7f6d880d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 -- 172.21.15.59:0/1911699285 >> 172.21.15.59:0/1911699285 conn(0x7f6d88067ea0 msgr2=0x7f6d8806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 -- 172.21.15.59:0/1911699285 shutdown_connections 2024-04-23T14:33:52.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.075+0000 7f6d90400640 1 -- 172.21.15.59:0/1911699285 wait complete. 2024-04-23T14:33:52.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.077+0000 7f6d90400640 1 Processor -- start 2024-04-23T14:33:52.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.077+0000 7f6d90400640 1 -- start start 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d90400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880afaf0 0x7f6d881f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d90400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d880d06e0 0x7f6d8814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d90400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8815c5e0 0x7f6d8814ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d90400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8806e600 con 0x7f6d8815c5e0 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d90400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8806e480 con 0x7f6d880d06e0 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d90400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6d88162a10 con 0x7f6d880afaf0 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8815c5e0 0x7f6d8814ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8815c5e0 0x7f6d8814ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41394/0 (socket says 172.21.15.59:41394) 2024-04-23T14:33:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.078+0000 7f6d8e400640 1 -- 172.21.15.59:0/1441453330 learned_addr learned my addr 172.21.15.59:0/1441453330 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d8e400640 1 -- 172.21.15.59:0/1441453330 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880afaf0 msgr2=0x7f6d881f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d8e400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880afaf0 0x7f6d881f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d8e400640 1 -- 172.21.15.59:0/1441453330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d880d06e0 msgr2=0x7f6d8814a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d8e400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d880d06e0 0x7f6d8814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d8e400640 1 -- 172.21.15.59:0/1441453330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d880cf710 con 0x7f6d8815c5e0 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d8e400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8815c5e0 0x7f6d8814ab90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009990 tx=0x7f6d800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d85600640 1 -- 172.21.15.59:0/1441453330 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80037040 con 0x7f6d8815c5e0 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d85600640 1 -- 172.21.15.59:0/1441453330 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d80042070 con 0x7f6d8815c5e0 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d85600640 1 -- 172.21.15.59:0/1441453330 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8003b930 con 0x7f6d8815c5e0 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.079+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d880cee70 con 0x7f6d8815c5e0 2024-04-23T14:33:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.080+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d880cf430 con 0x7f6d8815c5e0 2024-04-23T14:33:52.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.081+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d88068a80 con 0x7f6d8815c5e0 2024-04-23T14:33:52.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.082+0000 7f6d85600640 1 -- 172.21.15.59:0/1441453330 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d8003bad0 con 0x7f6d8815c5e0 2024-04-23T14:33:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.083+0000 7f6d85600640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d6806c8f0 0x7f6d6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.083+0000 7f6d85600640 1 -- 172.21.15.59:0/1441453330 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f6d800ac400 con 0x7f6d8815c5e0 2024-04-23T14:33:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.083+0000 7f6d8da00640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d6806c8f0 0x7f6d6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.084+0000 7f6d8da00640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d6806c8f0 0x7f6d6806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0012e0 tx=0x7f6d7c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.088+0000 7f6d85600640 1 -- 172.21.15.59:0/1441453330 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d8007a790 con 0x7f6d8815c5e0 2024-04-23T14:33:52.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.296+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d88161ec0 con 0x7f6d8815c5e0 2024-04-23T14:33:52.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.300+0000 7f6d85600640 1 -- 172.21.15.59:0/1441453330 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16553 (secure 0 0 0) 0x7f6d8007a090 con 0x7f6d8815c5e0 2024-04-23T14:33:52.301 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:52.301 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:33:49.299668+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:45.619774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:46.736010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:33:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d6806c8f0 msgr2=0x7f6d6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d6806c8f0 0x7f6d6806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0012e0 tx=0x7f6d7c007d50 comp rx=0 tx=0).stop 2024-04-23T14:33:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8815c5e0 msgr2=0x7f6d8814ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8815c5e0 0x7f6d8814ab90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009990 tx=0x7f6d800046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 shutdown_connections 2024-04-23T14:33:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d6806c8f0 0x7f6d6806edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8815c5e0 0x7f6d8814ab90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d880d06e0 0x7f6d8814a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 --2- 172.21.15.59:0/1441453330 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d880afaf0 0x7f6d881f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 >> 172.21.15.59:0/1441453330 conn(0x7f6d88067ea0 msgr2=0x7f6d8806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 shutdown_connections 2024-04-23T14:33:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.303+0000 7f6d90400640 1 -- 172.21.15.59:0/1441453330 wait complete. 2024-04-23T14:33:52.324 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:33:52.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:33:52.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.528+0000 7f2e11200640 1 Processor -- start 2024-04-23T14:33:52.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.528+0000 7f2e11200640 1 -- start start 2024-04-23T14:33:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e11200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 0x7f2e0c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e11200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e0c0cd590 0x7f2e0c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e11200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e0c0b4690 0x7f2e0c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e11200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0c0cd9f0 con 0x7f2e0c0cd590 2024-04-23T14:33:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e11200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0c160210 con 0x7f2e0c0b4690 2024-04-23T14:33:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e11200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c1604f0 con 0x7f2e0c15c3d0 2024-04-23T14:33:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e03e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e0c0cd590 0x7f2e0c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e03e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e0c0cd590 0x7f2e0c0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41400/0 (socket says 172.21.15.59:41400) 2024-04-23T14:33:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.530+0000 7f2e03e00640 1 -- 172.21.15.59:0/1833146901 learned_addr learned my addr 172.21.15.59:0/1833146901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.531+0000 7f2e0be00640 1 --2- 172.21.15.59:0/1833146901 >> v2:172.21.15.76:3300/0 conn(0x7f2e0c0b4690 0x7f2e0c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.531+0000 7f2e0b400640 1 --2- 172.21.15.59:0/1833146901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 0x7f2e0c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.531+0000 7f2e03e00640 1 -- 172.21.15.59:0/1833146901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 msgr2=0x7f2e0c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:52.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.531+0000 7f2e03e00640 1 --2- 172.21.15.59:0/1833146901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 0x7f2e0c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.531+0000 7f2e03e00640 1 -- 172.21.15.59:0/1833146901 >> v2:172.21.15.76:3300/0 conn(0x7f2e0c0b4690 msgr2=0x7f2e0c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:52.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.531+0000 7f2e03e00640 1 --2- 172.21.15.59:0/1833146901 >> v2:172.21.15.76:3300/0 conn(0x7f2e0c0b4690 0x7f2e0c06b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.531+0000 7f2e03e00640 1 -- 172.21.15.59:0/1833146901 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c1607d0 con 0x7f2e0c0cd590 2024-04-23T14:33:52.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.532+0000 7f2e03e00640 1 --2- 172.21.15.59:0/1833146901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c0b4150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c071b70 tx=0x7f2df402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ec43184aa24830 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e0aa00640 1 -- 172.21.15.59:0/1833146901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4037070 con 0x7f2e0c0cd590 2024-04-23T14:33:52.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e0aa00640 1 -- 172.21.15.59:0/1833146901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4032440 con 0x7f2e0c0cd590 2024-04-23T14:33:52.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e0aa00640 1 -- 172.21.15.59:0/1833146901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4032810 con 0x7f2e0c0cd590 2024-04-23T14:33:52.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e11200640 1 -- 172.21.15.59:0/1833146901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 msgr2=0x7f2e0c0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e11200640 1 --2- 172.21.15.59:0/1833146901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c0b4150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c071b70 tx=0x7f2df402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e11200640 1 -- 172.21.15.59:0/1833146901 shutdown_connections 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e11200640 1 --2- 172.21.15.59:0/1833146901 >> v2:172.21.15.76:3300/0 conn(0x7f2e0c0b4690 0x7f2e0c06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e11200640 1 --2- 172.21.15.59:0/1833146901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c0b4150 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e11200640 1 --2- 172.21.15.59:0/1833146901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 0x7f2e0c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.533+0000 7f2e11200640 1 -- 172.21.15.59:0/1833146901 >> 172.21.15.59:0/1833146901 conn(0x7f2e0c0b21a0 msgr2=0x7f2e0c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.534+0000 7f2e11200640 1 -- 172.21.15.59:0/1833146901 shutdown_connections 2024-04-23T14:33:52.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.534+0000 7f2e11200640 1 -- 172.21.15.59:0/1833146901 wait complete. 2024-04-23T14:33:52.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.535+0000 7f2e11200640 1 Processor -- start 2024-04-23T14:33:52.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.536+0000 7f2e11200640 1 -- start start 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.536+0000 7f2e11200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0c0b4690 0x7f2e0c15f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.536+0000 7f2e11200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c1e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.536+0000 7f2e11200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 0x7f2e0c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.536+0000 7f2e11200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c0b4ae0 con 0x7f2e0c0cd590 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.536+0000 7f2e11200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c160d90 con 0x7f2e0c0b4690 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.536+0000 7f2e11200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c15c7b0 con 0x7f2e0c15c3d0 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c1e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c1e68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41408/0 (socket says 172.21.15.59:41408) 2024-04-23T14:33:52.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 -- 172.21.15.59:0/2539307432 learned_addr learned my addr 172.21.15.59:0/2539307432 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:52.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 -- 172.21.15.59:0/2539307432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 msgr2=0x7f2e0c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 0x7f2e0c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 -- 172.21.15.59:0/2539307432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0c0b4690 msgr2=0x7f2e0c15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0c0b4690 0x7f2e0c15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 -- 172.21.15.59:0/2539307432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c1e6e20 con 0x7f2e0c0cd590 2024-04-23T14:33:52.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.537+0000 7f2e03e00640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c1e68e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2df402a220 tx=0x7f2df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.538+0000 7f2e03400640 1 -- 172.21.15.59:0/2539307432 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4041020 con 0x7f2e0c0cd590 2024-04-23T14:33:52.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.538+0000 7f2e03400640 1 -- 172.21.15.59:0/2539307432 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df40306c0 con 0x7f2e0c0cd590 2024-04-23T14:33:52.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.538+0000 7f2e03400640 1 -- 172.21.15.59:0/2539307432 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4037040 con 0x7f2e0c0cd590 2024-04-23T14:33:52.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.538+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c1e70c0 con 0x7f2e0c0cd590 2024-04-23T14:33:52.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.538+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c0c7490 con 0x7f2e0c0cd590 2024-04-23T14:33:52.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.539+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c04fd90 con 0x7f2e0c0cd590 2024-04-23T14:33:52.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.540+0000 7f2e03400640 1 -- 172.21.15.59:0/2539307432 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2df4044020 con 0x7f2e0c0cd590 2024-04-23T14:33:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.541+0000 7f2e03400640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2de806c8f0 0x7f2de806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.541+0000 7f2e03400640 1 -- 172.21.15.59:0/2539307432 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2df40ac450 con 0x7f2e0c0cd590 2024-04-23T14:33:52.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.541+0000 7f2e0b400640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2de806c8f0 0x7f2de806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:52.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.542+0000 7f2e0b400640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2de806c8f0 0x7f2de806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009870 tx=0x7f2dfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.545+0000 7f2e03400640 1 -- 172.21.15.59:0/2539307432 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2df404b030 con 0x7f2e0c0cd590 2024-04-23T14:33:52.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.741+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e0c1e74c0 con 0x7f2de806c8f0 2024-04-23T14:33:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.761+0000 7f2e03400640 1 -- 172.21.15.59:0/2539307432 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f2dfc009df0 con 0x7f2de806c8f0 2024-04-23T14:33:52.766 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:52.767 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:33:52.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2de806c8f0 msgr2=0x7f2de806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2de806c8f0 0x7f2de806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009870 tx=0x7f2dfc008040 comp rx=0 tx=0).stop 2024-04-23T14:33:52.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 msgr2=0x7f2e0c1e68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:52.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c1e68e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2df402a220 tx=0x7f2df40046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 shutdown_connections 2024-04-23T14:33:52.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2de806c8f0 0x7f2de806edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e0c15c3d0 0x7f2e0c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e0c0cd590 0x7f2e0c1e68e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 --2- 172.21.15.59:0/2539307432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0c0b4690 0x7f2e0c15f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:52.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 >> 172.21.15.59:0/2539307432 conn(0x7f2e0c0b21a0 msgr2=0x7f2e0c0b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:52.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 shutdown_connections 2024-04-23T14:33:52.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:52.767+0000 7f2e11200640 1 -- 172.21.15.59:0/2539307432 wait complete. 2024-04-23T14:33:52.786 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-04-23T14:33:52.389738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225492,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749228,"statfs":{"total":773094113280,"available":772863209472,"internally_reserved":0,"allocated":5283840,"data_stored":2813556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001410"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.328107+0000","last_change":"2024-04-23T14:33:48.321926+0000","last_active":"2024-04-23T14:33:48.328107+0000","last_peered":"2024-04-23T14:33:48.328107+0000","last_clean":"2024-04-23T14:33:48.328107+0000","last_became_active":"2024-04-23T14:33:48.321423+0000","last_became_peered":"2024-04-23T14:33:48.321423+0000","last_unstale":"2024-04-23T14:33:48.328107+0000","last_undegraded":"2024-04-23T14:33:48.328107+0000","last_fullsized":"2024-04-23T14:33:48.328107+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:22.631287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345783+0000","last_change":"2024-04-23T14:33:48.342873+0000","last_active":"2024-04-23T14:33:48.345783+0000","last_peered":"2024-04-23T14:33:48.345783+0000","last_clean":"2024-04-23T14:33:48.345783+0000","last_became_active":"2024-04-23T14:33:48.342459+0000","last_became_peered":"2024-04-23T14:33:48.342459+0000","last_unstale":"2024-04-23T14:33:48.345783+0000","last_undegraded":"2024-04-23T14:33:48.345783+0000","last_fullsized":"2024-04-23T14:33:48.345783+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:54.066369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324896+0000","last_change":"2024-04-23T14:33:48.312037+0000","last_active":"2024-04-23T14:33:49.324896+0000","last_peered":"2024-04-23T14:33:49.324896+0000","last_clean":"2024-04-23T14:33:49.324896+0000","last_became_active":"2024-04-23T14:33:48.310067+0000","last_became_peered":"2024-04-23T14:33:48.310067+0000","last_unstale":"2024-04-23T14:33:49.324896+0000","last_undegraded":"2024-04-23T14:33:49.324896+0000","last_fullsized":"2024-04-23T14:33:49.324896+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.341028+0000","last_change":"2024-04-23T14:33:48.332031+0000","last_active":"2024-04-23T14:33:48.341028+0000","last_peered":"2024-04-23T14:33:48.341028+0000","last_clean":"2024-04-23T14:33:48.341028+0000","last_became_active":"2024-04-23T14:33:48.331754+0000","last_became_peered":"2024-04-23T14:33:48.331754+0000","last_unstale":"2024-04-23T14:33:48.341028+0000","last_undegraded":"2024-04-23T14:33:48.341028+0000","last_fullsized":"2024-04-23T14:33:48.341028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:44.494032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.316735+0000","last_change":"2024-04-23T14:33:48.313724+0000","last_active":"2024-04-23T14:33:48.316735+0000","last_peered":"2024-04-23T14:33:48.316735+0000","last_clean":"2024-04-23T14:33:48.316735+0000","last_became_active":"2024-04-23T14:33:48.312688+0000","last_became_peered":"2024-04-23T14:33:48.312688+0000","last_unstale":"2024-04-23T14:33:48.316735+0000","last_undegraded":"2024-04-23T14:33:48.316735+0000","last_fullsized":"2024-04-23T14:33:48.316735+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:36:58.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317603+0000","last_change":"2024-04-23T14:33:48.325815+0000","last_active":"2024-04-23T14:33:49.317603+0000","last_peered":"2024-04-23T14:33:49.317603+0000","last_clean":"2024-04-23T14:33:49.317603+0000","last_became_active":"2024-04-23T14:33:48.324428+0000","last_became_peered":"2024-04-23T14:33:48.324428+0000","last_unstale":"2024-04-23T14:33:49.317603+0000","last_undegraded":"2024-04-23T14:33:49.317603+0000","last_fullsized":"2024-04-23T14:33:49.317603+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:20:39.048353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333655+0000","last_change":"2024-04-23T14:33:48.329353+0000","last_active":"2024-04-23T14:33:48.333655+0000","last_peered":"2024-04-23T14:33:48.333655+0000","last_clean":"2024-04-23T14:33:48.333655+0000","last_became_active":"2024-04-23T14:33:48.328514+0000","last_became_peered":"2024-04-23T14:33:48.328514+0000","last_unstale":"2024-04-23T14:33:48.333655+0000","last_undegraded":"2024-04-23T14:33:48.333655+0000","last_fullsized":"2024-04-23T14:33:48.333655+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:06.651665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345268+0000","last_change":"2024-04-23T14:33:48.337833+0000","last_active":"2024-04-23T14:33:48.345268+0000","last_peered":"2024-04-23T14:33:48.345268+0000","last_clean":"2024-04-23T14:33:48.345268+0000","last_became_active":"2024-04-23T14:33:48.337524+0000","last_became_peered":"2024-04-23T14:33:48.337524+0000","last_unstale":"2024-04-23T14:33:48.345268+0000","last_undegraded":"2024-04-23T14:33:48.345268+0000","last_fullsized":"2024-04-23T14:33:48.345268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:03.844780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345936+0000","last_change":"2024-04-23T14:33:48.343905+0000","last_active":"2024-04-23T14:33:48.345936+0000","last_peered":"2024-04-23T14:33:48.345936+0000","last_clean":"2024-04-23T14:33:48.345936+0000","last_became_active":"2024-04-23T14:33:48.341293+0000","last_became_peered":"2024-04-23T14:33:48.341293+0000","last_unstale":"2024-04-23T14:33:48.345936+0000","last_undegraded":"2024-04-23T14:33:48.345936+0000","last_fullsized":"2024-04-23T14:33:48.345936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:46.678623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.344104+0000","last_change":"2024-04-23T14:33:48.340522+0000","last_active":"2024-04-23T14:33:48.344104+0000","last_peered":"2024-04-23T14:33:48.344104+0000","last_clean":"2024-04-23T14:33:48.344104+0000","last_became_active":"2024-04-23T14:33:48.340036+0000","last_became_peered":"2024-04-23T14:33:48.340036+0000","last_unstale":"2024-04-23T14:33:48.344104+0000","last_undegraded":"2024-04-23T14:33:48.344104+0000","last_fullsized":"2024-04-23T14:33:48.344104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:26:13.191488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.343006+0000","last_change":"2024-04-23T14:33:48.329993+0000","last_active":"2024-04-23T14:33:48.343006+0000","last_peered":"2024-04-23T14:33:48.343006+0000","last_clean":"2024-04-23T14:33:48.343006+0000","last_became_active":"2024-04-23T14:33:48.329697+0000","last_became_peered":"2024-04-23T14:33:48.329697+0000","last_unstale":"2024-04-23T14:33:48.343006+0000","last_undegraded":"2024-04-23T14:33:48.343006+0000","last_fullsized":"2024-04-23T14:33:48.343006+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:51:19.145785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.321275+0000","last_change":"2024-04-23T14:33:48.317001+0000","last_active":"2024-04-23T14:33:48.321275+0000","last_peered":"2024-04-23T14:33:48.321275+0000","last_clean":"2024-04-23T14:33:48.321275+0000","last_became_active":"2024-04-23T14:33:48.316610+0000","last_became_peered":"2024-04-23T14:33:48.316610+0000","last_unstale":"2024-04-23T14:33:48.321275+0000","last_undegraded":"2024-04-23T14:33:48.321275+0000","last_fullsized":"2024-04-23T14:33:48.321275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:22.631287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313354+0000","last_change":"2024-04-23T14:33:48.325217+0000","last_active":"2024-04-23T14:33:49.313354+0000","last_peered":"2024-04-23T14:33:49.313354+0000","last_clean":"2024-04-23T14:33:49.313354+0000","last_became_active":"2024-04-23T14:33:48.324778+0000","last_became_peered":"2024-04-23T14:33:48.324778+0000","last_unstale":"2024-04-23T14:33:49.313354+0000","last_undegraded":"2024-04-23T14:33:49.313354+0000","last_fullsized":"2024-04-23T14:33:49.313354+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:43.137905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.341136+0000","last_change":"2024-04-23T14:33:48.332619+0000","last_active":"2024-04-23T14:33:48.341136+0000","last_peered":"2024-04-23T14:33:48.341136+0000","last_clean":"2024-04-23T14:33:48.341136+0000","last_became_active":"2024-04-23T14:33:48.332272+0000","last_became_peered":"2024-04-23T14:33:48.332272+0000","last_unstale":"2024-04-23T14:33:48.341136+0000","last_undegraded":"2024-04-23T14:33:48.341136+0000","last_fullsized":"2024-04-23T14:33:48.341136+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:18:18.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317495+0000","last_change":"2024-04-23T14:33:48.326660+0000","last_active":"2024-04-23T14:33:49.317495+0000","last_peered":"2024-04-23T14:33:49.317495+0000","last_clean":"2024-04-23T14:33:49.317495+0000","last_became_active":"2024-04-23T14:33:48.326311+0000","last_became_peered":"2024-04-23T14:33:48.326311+0000","last_unstale":"2024-04-23T14:33:49.317495+0000","last_undegraded":"2024-04-23T14:33:49.317495+0000","last_fullsized":"2024-04-23T14:33:49.317495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:14:26.838855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316307+0000","last_change":"2024-04-23T14:33:48.315387+0000","last_active":"2024-04-23T14:33:49.316307+0000","last_peered":"2024-04-23T14:33:49.316307+0000","last_clean":"2024-04-23T14:33:49.316307+0000","last_became_active":"2024-04-23T14:33:48.314997+0000","last_became_peered":"2024-04-23T14:33:48.314997+0000","last_unstale":"2024-04-23T14:33:49.316307+0000","last_undegraded":"2024-04-23T14:33:49.316307+0000","last_fullsized":"2024-04-23T14:33:49.316307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.597982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325442+0000","last_change":"2024-04-23T14:33:48.313695+0000","last_active":"2024-04-23T14:33:49.325442+0000","last_peered":"2024-04-23T14:33:49.325442+0000","last_clean":"2024-04-23T14:33:49.325442+0000","last_became_active":"2024-04-23T14:33:48.313399+0000","last_became_peered":"2024-04-23T14:33:48.313399+0000","last_unstale":"2024-04-23T14:33:49.325442+0000","last_undegraded":"2024-04-23T14:33:49.325442+0000","last_fullsized":"2024-04-23T14:33:49.325442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:36:58.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325292+0000","last_change":"2024-04-23T14:33:48.343875+0000","last_active":"2024-04-23T14:33:49.325292+0000","last_peered":"2024-04-23T14:33:49.325292+0000","last_clean":"2024-04-23T14:33:49.325292+0000","last_became_active":"2024-04-23T14:33:48.343598+0000","last_became_peered":"2024-04-23T14:33:48.343598+0000","last_unstale":"2024-04-23T14:33:49.325292+0000","last_undegraded":"2024-04-23T14:33:49.325292+0000","last_fullsized":"2024-04-23T14:33:49.325292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:12.241975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318012+0000","last_change":"2024-04-23T14:33:48.317712+0000","last_active":"2024-04-23T14:33:49.318012+0000","last_peered":"2024-04-23T14:33:49.318012+0000","last_clean":"2024-04-23T14:33:49.318012+0000","last_became_active":"2024-04-23T14:33:48.317428+0000","last_became_peered":"2024-04-23T14:33:48.317428+0000","last_unstale":"2024-04-23T14:33:49.318012+0000","last_undegraded":"2024-04-23T14:33:49.318012+0000","last_fullsized":"2024-04-23T14:33:49.318012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313368+0000","last_change":"2024-04-23T14:33:48.311951+0000","last_active":"2024-04-23T14:33:49.313368+0000","last_peered":"2024-04-23T14:33:49.313368+0000","last_clean":"2024-04-23T14:33:49.313368+0000","last_became_active":"2024-04-23T14:33:48.311660+0000","last_became_peered":"2024-04-23T14:33:48.311660+0000","last_unstale":"2024-04-23T14:33:49.313368+0000","last_undegraded":"2024-04-23T14:33:49.313368+0000","last_fullsized":"2024-04-23T14:33:49.313368+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:31.281376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317795+0000","last_change":"2024-04-23T14:33:48.328839+0000","last_active":"2024-04-23T14:33:49.317795+0000","last_peered":"2024-04-23T14:33:49.317795+0000","last_clean":"2024-04-23T14:33:49.317795+0000","last_became_active":"2024-04-23T14:33:48.328534+0000","last_became_peered":"2024-04-23T14:33:48.328534+0000","last_unstale":"2024-04-23T14:33:49.317795+0000","last_undegraded":"2024-04-23T14:33:49.317795+0000","last_fullsized":"2024-04-23T14:33:49.317795+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:57.739584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.343122+0000","last_change":"2024-04-23T14:33:48.338168+0000","last_active":"2024-04-23T14:33:48.343122+0000","last_peered":"2024-04-23T14:33:48.343122+0000","last_clean":"2024-04-23T14:33:48.343122+0000","last_became_active":"2024-04-23T14:33:48.337844+0000","last_became_peered":"2024-04-23T14:33:48.337844+0000","last_unstale":"2024-04-23T14:33:48.343122+0000","last_undegraded":"2024-04-23T14:33:48.343122+0000","last_fullsized":"2024-04-23T14:33:48.343122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:07:23.859553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316685+0000","last_change":"2024-04-23T14:33:48.324242+0000","last_active":"2024-04-23T14:33:49.316685+0000","last_peered":"2024-04-23T14:33:49.316685+0000","last_clean":"2024-04-23T14:33:49.316685+0000","last_became_active":"2024-04-23T14:33:48.323951+0000","last_became_peered":"2024-04-23T14:33:48.323951+0000","last_unstale":"2024-04-23T14:33:49.316685+0000","last_undegraded":"2024-04-23T14:33:49.316685+0000","last_fullsized":"2024-04-23T14:33:49.316685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:18:18.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316997+0000","last_change":"2024-04-23T14:33:48.321349+0000","last_active":"2024-04-23T14:33:49.316997+0000","last_peered":"2024-04-23T14:33:49.316997+0000","last_clean":"2024-04-23T14:33:49.316997+0000","last_became_active":"2024-04-23T14:33:48.321076+0000","last_became_peered":"2024-04-23T14:33:48.321076+0000","last_unstale":"2024-04-23T14:33:49.316997+0000","last_undegraded":"2024-04-23T14:33:49.316997+0000","last_fullsized":"2024-04-23T14:33:49.316997+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:44.494032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317463+0000","last_change":"2024-04-23T14:33:48.308977+0000","last_active":"2024-04-23T14:33:49.317463+0000","last_peered":"2024-04-23T14:33:49.317463+0000","last_clean":"2024-04-23T14:33:49.317463+0000","last_became_active":"2024-04-23T14:33:48.308401+0000","last_became_peered":"2024-04-23T14:33:48.308401+0000","last_unstale":"2024-04-23T14:33:49.317463+0000","last_undegraded":"2024-04-23T14:33:49.317463+0000","last_fullsized":"2024-04-23T14:33:49.317463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:07:23.859553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313592+0000","last_change":"2024-04-23T14:33:48.325154+0000","last_active":"2024-04-23T14:33:49.313592+0000","last_peered":"2024-04-23T14:33:49.313592+0000","last_clean":"2024-04-23T14:33:49.313592+0000","last_became_active":"2024-04-23T14:33:48.323045+0000","last_became_peered":"2024-04-23T14:33:48.323045+0000","last_unstale":"2024-04-23T14:33:49.313592+0000","last_undegraded":"2024-04-23T14:33:49.313592+0000","last_fullsized":"2024-04-23T14:33:49.313592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:27:12.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325125+0000","last_change":"2024-04-23T14:33:48.312852+0000","last_active":"2024-04-23T14:33:49.325125+0000","last_peered":"2024-04-23T14:33:49.325125+0000","last_clean":"2024-04-23T14:33:49.325125+0000","last_became_active":"2024-04-23T14:33:48.312497+0000","last_became_peered":"2024-04-23T14:33:48.312497+0000","last_unstale":"2024-04-23T14:33:49.325125+0000","last_undegraded":"2024-04-23T14:33:49.325125+0000","last_fullsized":"2024-04-23T14:33:49.325125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:46.678623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.316860+0000","last_change":"2024-04-23T14:33:48.312397+0000","last_active":"2024-04-23T14:33:48.316860+0000","last_peered":"2024-04-23T14:33:48.316860+0000","last_clean":"2024-04-23T14:33:48.316860+0000","last_became_active":"2024-04-23T14:33:48.311439+0000","last_became_peered":"2024-04-23T14:33:48.311439+0000","last_unstale":"2024-04-23T14:33:48.316860+0000","last_undegraded":"2024-04-23T14:33:48.316860+0000","last_fullsized":"2024-04-23T14:33:48.316860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:54.040413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317491+0000","last_change":"2024-04-23T14:33:48.300661+0000","last_active":"2024-04-23T14:33:49.317491+0000","last_peered":"2024-04-23T14:33:49.317491+0000","last_clean":"2024-04-23T14:33:49.317491+0000","last_became_active":"2024-04-23T14:33:48.299830+0000","last_became_peered":"2024-04-23T14:33:48.299830+0000","last_unstale":"2024-04-23T14:33:49.317491+0000","last_undegraded":"2024-04-23T14:33:49.317491+0000","last_fullsized":"2024-04-23T14:33:49.317491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:26:13.191488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324297+0000","last_change":"2024-04-23T14:33:48.321861+0000","last_active":"2024-04-23T14:33:49.324297+0000","last_peered":"2024-04-23T14:33:49.324297+0000","last_clean":"2024-04-23T14:33:49.324297+0000","last_became_active":"2024-04-23T14:33:48.321539+0000","last_became_peered":"2024-04-23T14:33:48.321539+0000","last_unstale":"2024-04-23T14:33:49.324297+0000","last_undegraded":"2024-04-23T14:33:49.324297+0000","last_fullsized":"2024-04-23T14:33:49.324297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:07.788298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317977+0000","last_change":"2024-04-23T14:33:48.329009+0000","last_active":"2024-04-23T14:33:49.317977+0000","last_peered":"2024-04-23T14:33:49.317977+0000","last_clean":"2024-04-23T14:33:49.317977+0000","last_became_active":"2024-04-23T14:33:48.327929+0000","last_became_peered":"2024-04-23T14:33:48.327929+0000","last_unstale":"2024-04-23T14:33:49.317977+0000","last_undegraded":"2024-04-23T14:33:49.317977+0000","last_fullsized":"2024-04-23T14:33:49.317977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:42:43.613375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313222+0000","last_change":"2024-04-23T14:33:48.304570+0000","last_active":"2024-04-23T14:33:49.313222+0000","last_peered":"2024-04-23T14:33:49.313222+0000","last_clean":"2024-04-23T14:33:49.313222+0000","last_became_active":"2024-04-23T14:33:48.304128+0000","last_became_peered":"2024-04-23T14:33:48.304128+0000","last_unstale":"2024-04-23T14:33:49.313222+0000","last_undegraded":"2024-04-23T14:33:49.313222+0000","last_fullsized":"2024-04-23T14:33:49.313222+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:45.978046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315803+0000","last_change":"2024-04-23T14:33:48.326863+0000","last_active":"2024-04-23T14:33:49.315803+0000","last_peered":"2024-04-23T14:33:49.315803+0000","last_clean":"2024-04-23T14:33:49.315803+0000","last_became_active":"2024-04-23T14:33:48.326501+0000","last_became_peered":"2024-04-23T14:33:48.326501+0000","last_unstale":"2024-04-23T14:33:49.315803+0000","last_undegraded":"2024-04-23T14:33:49.315803+0000","last_fullsized":"2024-04-23T14:33:49.315803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318386+0000","last_change":"2024-04-23T14:33:47.305121+0000","last_active":"2024-04-23T14:33:49.318386+0000","last_peered":"2024-04-23T14:33:49.318386+0000","last_clean":"2024-04-23T14:33:49.318386+0000","last_became_active":"2024-04-23T14:33:47.304801+0000","last_became_peered":"2024-04-23T14:33:47.304801+0000","last_unstale":"2024-04-23T14:33:49.318386+0000","last_undegraded":"2024-04-23T14:33:49.318386+0000","last_fullsized":"2024-04-23T14:33:49.318386+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:26.039708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317792+0000","last_change":"2024-04-23T14:33:48.308703+0000","last_active":"2024-04-23T14:33:49.317792+0000","last_peered":"2024-04-23T14:33:49.317792+0000","last_clean":"2024-04-23T14:33:49.317792+0000","last_became_active":"2024-04-23T14:33:48.307980+0000","last_became_peered":"2024-04-23T14:33:48.307980+0000","last_unstale":"2024-04-23T14:33:49.317792+0000","last_undegraded":"2024-04-23T14:33:49.317792+0000","last_fullsized":"2024-04-23T14:33:49.317792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:03.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325648+0000","last_change":"2024-04-23T14:33:47.318201+0000","last_active":"2024-04-23T14:33:49.325648+0000","last_peered":"2024-04-23T14:33:49.325648+0000","last_clean":"2024-04-23T14:33:49.325648+0000","last_became_active":"2024-04-23T14:33:47.317331+0000","last_became_peered":"2024-04-23T14:33:47.317331+0000","last_unstale":"2024-04-23T14:33:49.325648+0000","last_undegraded":"2024-04-23T14:33:49.325648+0000","last_fullsized":"2024-04-23T14:33:49.325648+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:33:30.339190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.331632+0000","last_change":"2024-04-23T14:33:48.325624+0000","last_active":"2024-04-23T14:33:48.331632+0000","last_peered":"2024-04-23T14:33:48.331632+0000","last_clean":"2024-04-23T14:33:48.331632+0000","last_became_active":"2024-04-23T14:33:48.325348+0000","last_became_peered":"2024-04-23T14:33:48.325348+0000","last_unstale":"2024-04-23T14:33:48.331632+0000","last_undegraded":"2024-04-23T14:33:48.331632+0000","last_fullsized":"2024-04-23T14:33:48.331632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:52.757461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315985+0000","last_change":"2024-04-23T14:33:47.314246+0000","last_active":"2024-04-23T14:33:49.315985+0000","last_peered":"2024-04-23T14:33:49.315985+0000","last_clean":"2024-04-23T14:33:49.315985+0000","last_became_active":"2024-04-23T14:33:47.313919+0000","last_became_peered":"2024-04-23T14:33:47.313919+0000","last_unstale":"2024-04-23T14:33:49.315985+0000","last_undegraded":"2024-04-23T14:33:49.315985+0000","last_fullsized":"2024-04-23T14:33:49.315985+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:07:22.844922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315873+0000","last_change":"2024-04-23T14:33:48.301837+0000","last_active":"2024-04-23T14:33:49.315873+0000","last_peered":"2024-04-23T14:33:49.315873+0000","last_clean":"2024-04-23T14:33:49.315873+0000","last_became_active":"2024-04-23T14:33:48.301403+0000","last_became_peered":"2024-04-23T14:33:48.301403+0000","last_unstale":"2024-04-23T14:33:49.315873+0000","last_undegraded":"2024-04-23T14:33:49.315873+0000","last_fullsized":"2024-04-23T14:33:49.315873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:46.678623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.332026+0000","last_change":"2024-04-23T14:33:47.316110+0000","last_active":"2024-04-23T14:33:48.332026+0000","last_peered":"2024-04-23T14:33:48.332026+0000","last_clean":"2024-04-23T14:33:48.332026+0000","last_became_active":"2024-04-23T14:33:47.314849+0000","last_became_peered":"2024-04-23T14:33:47.314849+0000","last_unstale":"2024-04-23T14:33:48.332026+0000","last_undegraded":"2024-04-23T14:33:48.332026+0000","last_fullsized":"2024-04-23T14:33:48.332026+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:03.505844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324369+0000","last_change":"2024-04-23T14:33:48.317452+0000","last_active":"2024-04-23T14:33:49.324369+0000","last_peered":"2024-04-23T14:33:49.324369+0000","last_clean":"2024-04-23T14:33:49.324369+0000","last_became_active":"2024-04-23T14:33:48.316171+0000","last_became_peered":"2024-04-23T14:33:48.316171+0000","last_unstale":"2024-04-23T14:33:49.324369+0000","last_undegraded":"2024-04-23T14:33:49.324369+0000","last_fullsized":"2024-04-23T14:33:49.324369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:03.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317917+0000","last_change":"2024-04-23T14:33:47.314100+0000","last_active":"2024-04-23T14:33:49.317917+0000","last_peered":"2024-04-23T14:33:49.317917+0000","last_clean":"2024-04-23T14:33:49.317917+0000","last_became_active":"2024-04-23T14:33:47.313569+0000","last_became_peered":"2024-04-23T14:33:47.313569+0000","last_unstale":"2024-04-23T14:33:49.317917+0000","last_undegraded":"2024-04-23T14:33:49.317917+0000","last_fullsized":"2024-04-23T14:33:49.317917+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:05.637034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.343601+0000","last_change":"2024-04-23T14:33:48.330366+0000","last_active":"2024-04-23T14:33:48.343601+0000","last_peered":"2024-04-23T14:33:48.343601+0000","last_clean":"2024-04-23T14:33:48.343601+0000","last_became_active":"2024-04-23T14:33:48.330016+0000","last_became_peered":"2024-04-23T14:33:48.330016+0000","last_unstale":"2024-04-23T14:33:48.343601+0000","last_undegraded":"2024-04-23T14:33:48.343601+0000","last_fullsized":"2024-04-23T14:33:48.343601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.597982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324527+0000","last_change":"2024-04-23T14:33:47.313664+0000","last_active":"2024-04-23T14:33:49.324527+0000","last_peered":"2024-04-23T14:33:49.324527+0000","last_clean":"2024-04-23T14:33:49.324527+0000","last_became_active":"2024-04-23T14:33:47.313357+0000","last_became_peered":"2024-04-23T14:33:47.313357+0000","last_unstale":"2024-04-23T14:33:49.324527+0000","last_undegraded":"2024-04-23T14:33:49.324527+0000","last_fullsized":"2024-04-23T14:33:49.324527+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:00.482836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345123+0000","last_change":"2024-04-23T14:33:48.337306+0000","last_active":"2024-04-23T14:33:48.345123+0000","last_peered":"2024-04-23T14:33:48.345123+0000","last_clean":"2024-04-23T14:33:48.345123+0000","last_became_active":"2024-04-23T14:33:48.331057+0000","last_became_peered":"2024-04-23T14:33:48.331057+0000","last_unstale":"2024-04-23T14:33:48.345123+0000","last_undegraded":"2024-04-23T14:33:48.345123+0000","last_fullsized":"2024-04-23T14:33:48.345123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:59.470476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.305195+0000","last_change":"2024-04-23T14:33:47.309849+0000","last_active":"2024-04-23T14:33:48.305195+0000","last_peered":"2024-04-23T14:33:48.305195+0000","last_clean":"2024-04-23T14:33:48.305195+0000","last_became_active":"2024-04-23T14:33:47.309531+0000","last_became_peered":"2024-04-23T14:33:47.309531+0000","last_unstale":"2024-04-23T14:33:48.305195+0000","last_undegraded":"2024-04-23T14:33:48.305195+0000","last_fullsized":"2024-04-23T14:33:48.305195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:02.239629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316341+0000","last_change":"2024-04-23T14:33:48.323236+0000","last_active":"2024-04-23T14:33:49.316341+0000","last_peered":"2024-04-23T14:33:49.316341+0000","last_clean":"2024-04-23T14:33:49.316341+0000","last_became_active":"2024-04-23T14:33:48.322974+0000","last_became_peered":"2024-04-23T14:33:48.322974+0000","last_unstale":"2024-04-23T14:33:49.316341+0000","last_undegraded":"2024-04-23T14:33:49.316341+0000","last_fullsized":"2024-04-23T14:33:49.316341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.644777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.314186+0000","last_change":"2024-04-23T14:33:47.309224+0000","last_active":"2024-04-23T14:33:48.314186+0000","last_peered":"2024-04-23T14:33:48.314186+0000","last_clean":"2024-04-23T14:33:48.314186+0000","last_became_active":"2024-04-23T14:33:47.308924+0000","last_became_peered":"2024-04-23T14:33:47.308924+0000","last_unstale":"2024-04-23T14:33:48.314186+0000","last_undegraded":"2024-04-23T14:33:48.314186+0000","last_fullsized":"2024-04-23T14:33:48.314186+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:00.482836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325305+0000","last_change":"2024-04-23T14:33:48.341446+0000","last_active":"2024-04-23T14:33:49.325305+0000","last_peered":"2024-04-23T14:33:49.325305+0000","last_clean":"2024-04-23T14:33:49.325305+0000","last_became_active":"2024-04-23T14:33:48.341061+0000","last_became_peered":"2024-04-23T14:33:48.341061+0000","last_unstale":"2024-04-23T14:33:49.325305+0000","last_undegraded":"2024-04-23T14:33:49.325305+0000","last_fullsized":"2024-04-23T14:33:49.325305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:54.040413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325735+0000","last_change":"2024-04-23T14:33:47.312006+0000","last_active":"2024-04-23T14:33:49.325735+0000","last_peered":"2024-04-23T14:33:49.325735+0000","last_clean":"2024-04-23T14:33:49.325735+0000","last_became_active":"2024-04-23T14:33:47.311248+0000","last_became_peered":"2024-04-23T14:33:47.311248+0000","last_unstale":"2024-04-23T14:33:49.325735+0000","last_undegraded":"2024-04-23T14:33:49.325735+0000","last_fullsized":"2024-04-23T14:33:49.325735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:34.096326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.328063+0000","last_change":"2024-04-23T14:33:48.320593+0000","last_active":"2024-04-23T14:33:48.328063+0000","last_peered":"2024-04-23T14:33:48.328063+0000","last_clean":"2024-04-23T14:33:48.328063+0000","last_became_active":"2024-04-23T14:33:48.320317+0000","last_became_peered":"2024-04-23T14:33:48.320317+0000","last_unstale":"2024-04-23T14:33:48.328063+0000","last_undegraded":"2024-04-23T14:33:48.328063+0000","last_fullsized":"2024-04-23T14:33:48.328063+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:36:58.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.330807+0000","last_change":"2024-04-23T14:33:47.312763+0000","last_active":"2024-04-23T14:33:48.330807+0000","last_peered":"2024-04-23T14:33:48.330807+0000","last_clean":"2024-04-23T14:33:48.330807+0000","last_became_active":"2024-04-23T14:33:47.312439+0000","last_became_peered":"2024-04-23T14:33:47.312439+0000","last_unstale":"2024-04-23T14:33:48.330807+0000","last_undegraded":"2024-04-23T14:33:48.330807+0000","last_fullsized":"2024-04-23T14:33:48.330807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:15.630146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318073+0000","last_change":"2024-04-23T14:33:48.332776+0000","last_active":"2024-04-23T14:33:49.318073+0000","last_peered":"2024-04-23T14:33:49.318073+0000","last_clean":"2024-04-23T14:33:49.318073+0000","last_became_active":"2024-04-23T14:33:48.332491+0000","last_became_peered":"2024-04-23T14:33:48.332491+0000","last_unstale":"2024-04-23T14:33:49.318073+0000","last_undegraded":"2024-04-23T14:33:49.318073+0000","last_fullsized":"2024-04-23T14:33:49.318073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:13.732846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333668+0000","last_change":"2024-04-23T14:33:47.315536+0000","last_active":"2024-04-23T14:33:48.333668+0000","last_peered":"2024-04-23T14:33:48.333668+0000","last_clean":"2024-04-23T14:33:48.333668+0000","last_became_active":"2024-04-23T14:33:47.314869+0000","last_became_peered":"2024-04-23T14:33:47.314869+0000","last_unstale":"2024-04-23T14:33:48.333668+0000","last_undegraded":"2024-04-23T14:33:48.333668+0000","last_fullsized":"2024-04-23T14:33:48.333668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:11.227344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.328063+0000","last_change":"2024-04-23T14:33:48.322586+0000","last_active":"2024-04-23T14:33:48.328063+0000","last_peered":"2024-04-23T14:33:48.328063+0000","last_clean":"2024-04-23T14:33:48.328063+0000","last_became_active":"2024-04-23T14:33:48.321940+0000","last_became_peered":"2024-04-23T14:33:48.321940+0000","last_unstale":"2024-04-23T14:33:48.328063+0000","last_undegraded":"2024-04-23T14:33:48.328063+0000","last_fullsized":"2024-04-23T14:33:48.328063+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:52.757461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313361+0000","last_change":"2024-04-23T14:33:47.313051+0000","last_active":"2024-04-23T14:33:49.313361+0000","last_peered":"2024-04-23T14:33:49.313361+0000","last_clean":"2024-04-23T14:33:49.313361+0000","last_became_active":"2024-04-23T14:33:47.311724+0000","last_became_peered":"2024-04-23T14:33:47.311724+0000","last_unstale":"2024-04-23T14:33:49.313361+0000","last_undegraded":"2024-04-23T14:33:49.313361+0000","last_fullsized":"2024-04-23T14:33:49.313361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:48:25.641075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316891+0000","last_change":"2024-04-23T14:33:48.336022+0000","last_active":"2024-04-23T14:33:49.316891+0000","last_peered":"2024-04-23T14:33:49.316891+0000","last_clean":"2024-04-23T14:33:49.316891+0000","last_became_active":"2024-04-23T14:33:48.335781+0000","last_became_peered":"2024-04-23T14:33:48.335781+0000","last_unstale":"2024-04-23T14:33:49.316891+0000","last_undegraded":"2024-04-23T14:33:49.316891+0000","last_fullsized":"2024-04-23T14:33:49.316891+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:59.470476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318036+0000","last_change":"2024-04-23T14:33:47.305908+0000","last_active":"2024-04-23T14:33:49.318036+0000","last_peered":"2024-04-23T14:33:49.318036+0000","last_clean":"2024-04-23T14:33:49.318036+0000","last_became_active":"2024-04-23T14:33:47.305523+0000","last_became_peered":"2024-04-23T14:33:47.305523+0000","last_unstale":"2024-04-23T14:33:49.318036+0000","last_undegraded":"2024-04-23T14:33:49.318036+0000","last_fullsized":"2024-04-23T14:33:49.318036+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:17.332622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345252+0000","last_change":"2024-04-23T14:33:48.343162+0000","last_active":"2024-04-23T14:33:48.345252+0000","last_peered":"2024-04-23T14:33:48.345252+0000","last_clean":"2024-04-23T14:33:48.345252+0000","last_became_active":"2024-04-23T14:33:48.338678+0000","last_became_peered":"2024-04-23T14:33:48.338678+0000","last_unstale":"2024-04-23T14:33:48.345252+0000","last_undegraded":"2024-04-23T14:33:48.345252+0000","last_fullsized":"2024-04-23T14:33:48.345252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.308862+0000","last_change":"2024-04-23T14:33:47.317355+0000","last_active":"2024-04-23T14:33:48.308862+0000","last_peered":"2024-04-23T14:33:48.308862+0000","last_clean":"2024-04-23T14:33:48.308862+0000","last_became_active":"2024-04-23T14:33:47.314263+0000","last_became_peered":"2024-04-23T14:33:47.314263+0000","last_unstale":"2024-04-23T14:33:48.308862+0000","last_undegraded":"2024-04-23T14:33:48.308862+0000","last_fullsized":"2024-04-23T14:33:48.308862+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:45.663992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317503+0000","last_change":"2024-04-23T14:33:48.328866+0000","last_active":"2024-04-23T14:33:49.317503+0000","last_peered":"2024-04-23T14:33:49.317503+0000","last_clean":"2024-04-23T14:33:49.317503+0000","last_became_active":"2024-04-23T14:33:48.328570+0000","last_became_peered":"2024-04-23T14:33:48.328570+0000","last_unstale":"2024-04-23T14:33:49.317503+0000","last_undegraded":"2024-04-23T14:33:49.317503+0000","last_fullsized":"2024-04-23T14:33:49.317503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:05:06.502293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318098+0000","last_change":"2024-04-23T14:33:47.301809+0000","last_active":"2024-04-23T14:33:49.318098+0000","last_peered":"2024-04-23T14:33:49.318098+0000","last_clean":"2024-04-23T14:33:49.318098+0000","last_became_active":"2024-04-23T14:33:47.301501+0000","last_became_peered":"2024-04-23T14:33:47.301501+0000","last_unstale":"2024-04-23T14:33:49.318098+0000","last_undegraded":"2024-04-23T14:33:49.318098+0000","last_fullsized":"2024-04-23T14:33:49.318098+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:21.616656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333675+0000","last_change":"2024-04-23T14:33:48.329804+0000","last_active":"2024-04-23T14:33:48.333675+0000","last_peered":"2024-04-23T14:33:48.333675+0000","last_clean":"2024-04-23T14:33:48.333675+0000","last_became_active":"2024-04-23T14:33:48.329463+0000","last_became_peered":"2024-04-23T14:33:48.329463+0000","last_unstale":"2024-04-23T14:33:48.333675+0000","last_undegraded":"2024-04-23T14:33:48.333675+0000","last_fullsized":"2024-04-23T14:33:48.333675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.347253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.348759+0000","last_change":"2024-04-23T14:33:47.311375+0000","last_active":"2024-04-23T14:33:48.348759+0000","last_peered":"2024-04-23T14:33:48.348759+0000","last_clean":"2024-04-23T14:33:48.348759+0000","last_became_active":"2024-04-23T14:33:47.311067+0000","last_became_peered":"2024-04-23T14:33:47.311067+0000","last_unstale":"2024-04-23T14:33:48.348759+0000","last_undegraded":"2024-04-23T14:33:48.348759+0000","last_fullsized":"2024-04-23T14:33:48.348759+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:36:52.874227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313769+0000","last_change":"2024-04-23T14:33:48.321061+0000","last_active":"2024-04-23T14:33:49.313769+0000","last_peered":"2024-04-23T14:33:49.313769+0000","last_clean":"2024-04-23T14:33:49.313769+0000","last_became_active":"2024-04-23T14:33:48.313410+0000","last_became_peered":"2024-04-23T14:33:48.313410+0000","last_unstale":"2024-04-23T14:33:49.313769+0000","last_undegraded":"2024-04-23T14:33:49.313769+0000","last_fullsized":"2024-04-23T14:33:49.313769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:20:20.248968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313717+0000","last_change":"2024-04-23T14:33:47.300054+0000","last_active":"2024-04-23T14:33:49.313717+0000","last_peered":"2024-04-23T14:33:49.313717+0000","last_clean":"2024-04-23T14:33:49.313717+0000","last_became_active":"2024-04-23T14:33:47.299022+0000","last_became_peered":"2024-04-23T14:33:47.299022+0000","last_unstale":"2024-04-23T14:33:49.313717+0000","last_undegraded":"2024-04-23T14:33:49.313717+0000","last_fullsized":"2024-04-23T14:33:49.313717+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:50.189865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345246+0000","last_change":"2024-04-23T14:33:48.343338+0000","last_active":"2024-04-23T14:33:48.345246+0000","last_peered":"2024-04-23T14:33:48.345246+0000","last_clean":"2024-04-23T14:33:48.345246+0000","last_became_active":"2024-04-23T14:33:48.341024+0000","last_became_peered":"2024-04-23T14:33:48.341024+0000","last_unstale":"2024-04-23T14:33:48.345246+0000","last_undegraded":"2024-04-23T14:33:48.345246+0000","last_fullsized":"2024-04-23T14:33:48.345246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325711+0000","last_change":"2024-04-23T14:33:47.312500+0000","last_active":"2024-04-23T14:33:49.325711+0000","last_peered":"2024-04-23T14:33:49.325711+0000","last_clean":"2024-04-23T14:33:49.325711+0000","last_became_active":"2024-04-23T14:33:47.312153+0000","last_became_peered":"2024-04-23T14:33:47.312153+0000","last_unstale":"2024-04-23T14:33:49.325711+0000","last_undegraded":"2024-04-23T14:33:49.325711+0000","last_fullsized":"2024-04-23T14:33:49.325711+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:13.546252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317466+0000","last_change":"2024-04-23T14:33:48.327638+0000","last_active":"2024-04-23T14:33:49.317466+0000","last_peered":"2024-04-23T14:33:49.317466+0000","last_clean":"2024-04-23T14:33:49.317466+0000","last_became_active":"2024-04-23T14:33:48.327332+0000","last_became_peered":"2024-04-23T14:33:48.327332+0000","last_unstale":"2024-04-23T14:33:49.317466+0000","last_undegraded":"2024-04-23T14:33:49.317466+0000","last_fullsized":"2024-04-23T14:33:49.317466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:03.844780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.326203+0000","last_change":"2024-04-23T14:33:47.310484+0000","last_active":"2024-04-23T14:33:48.326203+0000","last_peered":"2024-04-23T14:33:48.326203+0000","last_clean":"2024-04-23T14:33:48.326203+0000","last_became_active":"2024-04-23T14:33:47.308659+0000","last_became_peered":"2024-04-23T14:33:47.308659+0000","last_unstale":"2024-04-23T14:33:48.326203+0000","last_undegraded":"2024-04-23T14:33:48.326203+0000","last_fullsized":"2024-04-23T14:33:48.326203+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:03.505844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317588+0000","last_change":"2024-04-23T14:33:48.310942+0000","last_active":"2024-04-23T14:33:49.317588+0000","last_peered":"2024-04-23T14:33:49.317588+0000","last_clean":"2024-04-23T14:33:49.317588+0000","last_became_active":"2024-04-23T14:33:48.309375+0000","last_became_peered":"2024-04-23T14:33:48.309375+0000","last_unstale":"2024-04-23T14:33:49.317588+0000","last_undegraded":"2024-04-23T14:33:49.317588+0000","last_fullsized":"2024-04-23T14:33:49.317588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:07.788298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318196+0000","last_change":"2024-04-23T14:33:47.301010+0000","last_active":"2024-04-23T14:33:49.318196+0000","last_peered":"2024-04-23T14:33:49.318196+0000","last_clean":"2024-04-23T14:33:49.318196+0000","last_became_active":"2024-04-23T14:33:47.300663+0000","last_became_peered":"2024-04-23T14:33:47.300663+0000","last_unstale":"2024-04-23T14:33:49.318196+0000","last_undegraded":"2024-04-23T14:33:49.318196+0000","last_fullsized":"2024-04-23T14:33:49.318196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:51.742830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317581+0000","last_change":"2024-04-23T14:33:48.335470+0000","last_active":"2024-04-23T14:33:49.317581+0000","last_peered":"2024-04-23T14:33:49.317581+0000","last_clean":"2024-04-23T14:33:49.317581+0000","last_became_active":"2024-04-23T14:33:48.334744+0000","last_became_peered":"2024-04-23T14:33:48.334744+0000","last_unstale":"2024-04-23T14:33:49.317581+0000","last_undegraded":"2024-04-23T14:33:49.317581+0000","last_fullsized":"2024-04-23T14:33:49.317581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:21.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318296+0000","last_change":"2024-04-23T14:33:47.313414+0000","last_active":"2024-04-23T14:33:49.318296+0000","last_peered":"2024-04-23T14:33:49.318296+0000","last_clean":"2024-04-23T14:33:49.318296+0000","last_became_active":"2024-04-23T14:33:47.312929+0000","last_became_peered":"2024-04-23T14:33:47.312929+0000","last_unstale":"2024-04-23T14:33:49.318296+0000","last_undegraded":"2024-04-23T14:33:49.318296+0000","last_fullsized":"2024-04-23T14:33:49.318296+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:59.935442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315496+0000","last_change":"2024-04-23T14:33:48.313195+0000","last_active":"2024-04-23T14:33:49.315496+0000","last_peered":"2024-04-23T14:33:49.315496+0000","last_clean":"2024-04-23T14:33:49.315496+0000","last_became_active":"2024-04-23T14:33:48.312897+0000","last_became_peered":"2024-04-23T14:33:48.312897+0000","last_unstale":"2024-04-23T14:33:49.315496+0000","last_undegraded":"2024-04-23T14:33:49.315496+0000","last_fullsized":"2024-04-23T14:33:49.315496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.305861+0000","last_change":"2024-04-23T14:33:47.306796+0000","last_active":"2024-04-23T14:33:48.305861+0000","last_peered":"2024-04-23T14:33:48.305861+0000","last_clean":"2024-04-23T14:33:48.305861+0000","last_became_active":"2024-04-23T14:33:47.305772+0000","last_became_peered":"2024-04-23T14:33:47.305772+0000","last_unstale":"2024-04-23T14:33:48.305861+0000","last_undegraded":"2024-04-23T14:33:48.305861+0000","last_fullsized":"2024-04-23T14:33:48.305861+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:51.670869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.312992+0000","last_change":"2024-04-23T14:33:48.325085+0000","last_active":"2024-04-23T14:33:49.312992+0000","last_peered":"2024-04-23T14:33:49.312992+0000","last_clean":"2024-04-23T14:33:49.312992+0000","last_became_active":"2024-04-23T14:33:48.323038+0000","last_became_peered":"2024-04-23T14:33:48.323038+0000","last_unstale":"2024-04-23T14:33:49.312992+0000","last_undegraded":"2024-04-23T14:33:49.312992+0000","last_fullsized":"2024-04-23T14:33:49.312992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:29:57.650311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.322117+0000","last_change":"2024-04-23T14:33:47.314972+0000","last_active":"2024-04-23T14:33:48.322117+0000","last_peered":"2024-04-23T14:33:48.322117+0000","last_clean":"2024-04-23T14:33:48.322117+0000","last_became_active":"2024-04-23T14:33:47.314059+0000","last_became_peered":"2024-04-23T14:33:47.314059+0000","last_unstale":"2024-04-23T14:33:48.322117+0000","last_undegraded":"2024-04-23T14:33:48.322117+0000","last_fullsized":"2024-04-23T14:33:48.322117+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:34.096326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.342200+0000","last_change":"2024-04-23T14:33:48.322006+0000","last_active":"2024-04-23T14:33:48.342200+0000","last_peered":"2024-04-23T14:33:48.342200+0000","last_clean":"2024-04-23T14:33:48.342200+0000","last_became_active":"2024-04-23T14:33:48.319068+0000","last_became_peered":"2024-04-23T14:33:48.319068+0000","last_unstale":"2024-04-23T14:33:48.342200+0000","last_undegraded":"2024-04-23T14:33:48.342200+0000","last_fullsized":"2024-04-23T14:33:48.342200+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:14.560883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.308354+0000","last_change":"2024-04-23T14:33:47.314865+0000","last_active":"2024-04-23T14:33:48.308354+0000","last_peered":"2024-04-23T14:33:48.308354+0000","last_clean":"2024-04-23T14:33:48.308354+0000","last_became_active":"2024-04-23T14:33:47.314174+0000","last_became_peered":"2024-04-23T14:33:47.314174+0000","last_unstale":"2024-04-23T14:33:48.308354+0000","last_undegraded":"2024-04-23T14:33:48.308354+0000","last_fullsized":"2024-04-23T14:33:48.308354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:51.670869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318016+0000","last_change":"2024-04-23T14:33:48.344466+0000","last_active":"2024-04-23T14:33:49.318016+0000","last_peered":"2024-04-23T14:33:49.318016+0000","last_clean":"2024-04-23T14:33:49.318016+0000","last_became_active":"2024-04-23T14:33:48.344254+0000","last_became_peered":"2024-04-23T14:33:48.344254+0000","last_unstale":"2024-04-23T14:33:49.318016+0000","last_undegraded":"2024-04-23T14:33:49.318016+0000","last_fullsized":"2024-04-23T14:33:49.318016+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.322067+0000","last_change":"2024-04-23T14:33:47.314372+0000","last_active":"2024-04-23T14:33:49.322067+0000","last_peered":"2024-04-23T14:33:49.322067+0000","last_clean":"2024-04-23T14:33:49.322067+0000","last_became_active":"2024-04-23T14:33:47.313851+0000","last_became_peered":"2024-04-23T14:33:47.313851+0000","last_unstale":"2024-04-23T14:33:49.322067+0000","last_undegraded":"2024-04-23T14:33:49.322067+0000","last_fullsized":"2024-04-23T14:33:49.322067+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:53.051738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.344074+0000","last_change":"2024-04-23T14:33:48.340662+0000","last_active":"2024-04-23T14:33:48.344074+0000","last_peered":"2024-04-23T14:33:48.344074+0000","last_clean":"2024-04-23T14:33:48.344074+0000","last_became_active":"2024-04-23T14:33:48.339508+0000","last_became_peered":"2024-04-23T14:33:48.339508+0000","last_unstale":"2024-04-23T14:33:48.344074+0000","last_undegraded":"2024-04-23T14:33:48.344074+0000","last_fullsized":"2024-04-23T14:33:48.344074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:03.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.309897+0000","last_change":"2024-04-23T14:33:47.315815+0000","last_active":"2024-04-23T14:33:48.309897+0000","last_peered":"2024-04-23T14:33:48.309897+0000","last_clean":"2024-04-23T14:33:48.309897+0000","last_became_active":"2024-04-23T14:33:47.315259+0000","last_became_peered":"2024-04-23T14:33:47.315259+0000","last_unstale":"2024-04-23T14:33:48.309897+0000","last_undegraded":"2024-04-23T14:33:48.309897+0000","last_fullsized":"2024-04-23T14:33:48.309897+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:53.025782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.331027+0000","last_change":"2024-04-23T14:33:48.323498+0000","last_active":"2024-04-23T14:33:48.331027+0000","last_peered":"2024-04-23T14:33:48.331027+0000","last_clean":"2024-04-23T14:33:48.331027+0000","last_became_active":"2024-04-23T14:33:48.322296+0000","last_became_peered":"2024-04-23T14:33:48.322296+0000","last_unstale":"2024-04-23T14:33:48.331027+0000","last_undegraded":"2024-04-23T14:33:48.331027+0000","last_fullsized":"2024-04-23T14:33:48.331027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:54.066369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315486+0000","last_change":"2024-04-23T14:33:47.313626+0000","last_active":"2024-04-23T14:33:49.315486+0000","last_peered":"2024-04-23T14:33:49.315486+0000","last_clean":"2024-04-23T14:33:49.315486+0000","last_became_active":"2024-04-23T14:33:47.313361+0000","last_became_peered":"2024-04-23T14:33:47.313361+0000","last_unstale":"2024-04-23T14:33:49.315486+0000","last_undegraded":"2024-04-23T14:33:49.315486+0000","last_fullsized":"2024-04-23T14:33:49.315486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:58.455845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317853+0000","last_change":"2024-04-23T14:33:48.323375+0000","last_active":"2024-04-23T14:33:49.317853+0000","last_peered":"2024-04-23T14:33:49.317853+0000","last_clean":"2024-04-23T14:33:49.317853+0000","last_became_active":"2024-04-23T14:33:48.322959+0000","last_became_peered":"2024-04-23T14:33:48.322959+0000","last_unstale":"2024-04-23T14:33:49.317853+0000","last_undegraded":"2024-04-23T14:33:49.317853+0000","last_fullsized":"2024-04-23T14:33:49.317853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:04.978844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.306717+0000","last_change":"2024-04-23T14:33:47.315351+0000","last_active":"2024-04-23T14:33:48.306717+0000","last_peered":"2024-04-23T14:33:48.306717+0000","last_clean":"2024-04-23T14:33:48.306717+0000","last_became_active":"2024-04-23T14:33:47.315046+0000","last_became_peered":"2024-04-23T14:33:47.315046+0000","last_unstale":"2024-04-23T14:33:48.306717+0000","last_undegraded":"2024-04-23T14:33:48.306717+0000","last_fullsized":"2024-04-23T14:33:48.306717+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:02.239629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317801+0000","last_change":"2024-04-23T14:33:48.334896+0000","last_active":"2024-04-23T14:33:49.317801+0000","last_peered":"2024-04-23T14:33:49.317801+0000","last_clean":"2024-04-23T14:33:49.317801+0000","last_became_active":"2024-04-23T14:33:48.334595+0000","last_became_peered":"2024-04-23T14:33:48.334595+0000","last_unstale":"2024-04-23T14:33:49.317801+0000","last_undegraded":"2024-04-23T14:33:49.317801+0000","last_fullsized":"2024-04-23T14:33:49.317801+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:38:23.086095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324635+0000","last_change":"2024-04-23T14:33:47.311791+0000","last_active":"2024-04-23T14:33:49.324635+0000","last_peered":"2024-04-23T14:33:49.324635+0000","last_clean":"2024-04-23T14:33:49.324635+0000","last_became_active":"2024-04-23T14:33:47.311197+0000","last_became_peered":"2024-04-23T14:33:47.311197+0000","last_unstale":"2024-04-23T14:33:49.324635+0000","last_undegraded":"2024-04-23T14:33:49.324635+0000","last_fullsized":"2024-04-23T14:33:49.324635+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:36:52.874227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.314070+0000","last_change":"2024-04-23T14:33:48.310610+0000","last_active":"2024-04-23T14:33:48.314070+0000","last_peered":"2024-04-23T14:33:48.314070+0000","last_clean":"2024-04-23T14:33:48.314070+0000","last_became_active":"2024-04-23T14:33:48.310094+0000","last_became_peered":"2024-04-23T14:33:48.310094+0000","last_unstale":"2024-04-23T14:33:48.314070+0000","last_undegraded":"2024-04-23T14:33:48.314070+0000","last_fullsized":"2024-04-23T14:33:48.314070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:12.241975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315211+0000","last_change":"2024-04-23T14:33:47.312868+0000","last_active":"2024-04-23T14:33:49.315211+0000","last_peered":"2024-04-23T14:33:49.315211+0000","last_clean":"2024-04-23T14:33:49.315211+0000","last_became_active":"2024-04-23T14:33:47.312480+0000","last_became_peered":"2024-04-23T14:33:47.312480+0000","last_unstale":"2024-04-23T14:33:49.315211+0000","last_undegraded":"2024-04-23T14:33:49.315211+0000","last_fullsized":"2024-04-23T14:33:49.315211+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:02.830149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324395+0000","last_change":"2024-04-23T14:33:48.323232+0000","last_active":"2024-04-23T14:33:49.324395+0000","last_peered":"2024-04-23T14:33:49.324395+0000","last_clean":"2024-04-23T14:33:49.324395+0000","last_became_active":"2024-04-23T14:33:48.322942+0000","last_became_peered":"2024-04-23T14:33:48.322942+0000","last_unstale":"2024-04-23T14:33:49.324395+0000","last_undegraded":"2024-04-23T14:33:49.324395+0000","last_fullsized":"2024-04-23T14:33:49.324395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.314456+0000","last_change":"2024-04-23T14:33:47.301910+0000","last_active":"2024-04-23T14:33:48.314456+0000","last_peered":"2024-04-23T14:33:48.314456+0000","last_clean":"2024-04-23T14:33:48.314456+0000","last_became_active":"2024-04-23T14:33:47.301637+0000","last_became_peered":"2024-04-23T14:33:47.301637+0000","last_unstale":"2024-04-23T14:33:48.314456+0000","last_undegraded":"2024-04-23T14:33:48.314456+0000","last_fullsized":"2024-04-23T14:33:48.314456+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:33:30.339190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.344126+0000","last_change":"2024-04-23T14:33:48.339664+0000","last_active":"2024-04-23T14:33:48.344126+0000","last_peered":"2024-04-23T14:33:48.344126+0000","last_clean":"2024-04-23T14:33:48.344126+0000","last_became_active":"2024-04-23T14:33:48.339334+0000","last_became_peered":"2024-04-23T14:33:48.339334+0000","last_unstale":"2024-04-23T14:33:48.344126+0000","last_undegraded":"2024-04-23T14:33:48.344126+0000","last_fullsized":"2024-04-23T14:33:48.344126+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:07.788298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333964+0000","last_change":"2024-04-23T14:33:47.298769+0000","last_active":"2024-04-23T14:33:48.333964+0000","last_peered":"2024-04-23T14:33:48.333964+0000","last_clean":"2024-04-23T14:33:48.333964+0000","last_became_active":"2024-04-23T14:33:47.298048+0000","last_became_peered":"2024-04-23T14:33:47.298048+0000","last_unstale":"2024-04-23T14:33:48.333964+0000","last_undegraded":"2024-04-23T14:33:48.333964+0000","last_fullsized":"2024-04-23T14:33:48.333964+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:45.663992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.308422+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:33:48.308422+0000","last_peered":"2024-04-23T14:33:48.308422+0000","last_clean":"2024-04-23T14:33:48.308422+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:33:48.308422+0000","last_undegraded":"2024-04-23T14:33:48.308422+0000","last_fullsized":"2024-04-23T14:33:48.308422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1101824,"data_stored":793958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":204172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":516096,"data_stored":205112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":204138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":205134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":793902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:33:52.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:33:53.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.040+0000 7f3058e00640 1 Processor -- start 2024-04-23T14:33:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.041+0000 7f3058e00640 1 -- start start 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.042+0000 7f3058e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f305415b450 0x7f30540b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.042+0000 7f3058e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f305406f3f0 0x7f30540cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.042+0000 7f3058e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30540b3080 0x7f30540cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.042+0000 7f3058e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f30540b41a0 con 0x7f305406f3f0 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.042+0000 7f3058e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f30540b3870 con 0x7f305415b450 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.042+0000 7f3058e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30540b3b50 con 0x7f30540b3080 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f305406f3f0 0x7f30540cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f305406f3f0 0x7f30540cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41424/0 (socket says 172.21.15.59:41424) 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 -- 172.21.15.59:0/976291307 learned_addr learned my addr 172.21.15.59:0/976291307 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f3053e00640 1 --2- 172.21.15.59:0/976291307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30540b3080 0x7f30540cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f3053400640 1 --2- 172.21.15.59:0/976291307 >> v2:172.21.15.76:3300/0 conn(0x7f305415b450 0x7f30540b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 -- 172.21.15.59:0/976291307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30540b3080 msgr2=0x7f30540cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 --2- 172.21.15.59:0/976291307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30540b3080 0x7f30540cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 -- 172.21.15.59:0/976291307 >> v2:172.21.15.76:3300/0 conn(0x7f305415b450 msgr2=0x7f30540b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 --2- 172.21.15.59:0/976291307 >> v2:172.21.15.76:3300/0 conn(0x7f305415b450 0x7f30540b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.043+0000 7f304be00640 1 -- 172.21.15.59:0/976291307 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30540b3e30 con 0x7f305406f3f0 2024-04-23T14:33:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.045+0000 7f304be00640 1 --2- 172.21.15.59:0/976291307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f305406f3f0 0x7f30540cd050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f305406cb20 tx=0x7f303c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671d0184a62aefd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.045+0000 7f3052a00640 1 -- 172.21.15.59:0/976291307 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c037070 con 0x7f305406f3f0 2024-04-23T14:33:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.045+0000 7f3052a00640 1 -- 172.21.15.59:0/976291307 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f303c032440 con 0x7f305406f3f0 2024-04-23T14:33:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.045+0000 7f3052a00640 1 -- 172.21.15.59:0/976291307 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c032810 con 0x7f305406f3f0 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 -- 172.21.15.59:0/976291307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f305406f3f0 msgr2=0x7f30540cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 --2- 172.21.15.59:0/976291307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f305406f3f0 0x7f30540cd050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f305406cb20 tx=0x7f303c02a270 comp rx=0 tx=0).stop 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 -- 172.21.15.59:0/976291307 shutdown_connections 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 --2- 172.21.15.59:0/976291307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30540b3080 0x7f30540cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 --2- 172.21.15.59:0/976291307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f305406f3f0 0x7f30540cd050 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 --2- 172.21.15.59:0/976291307 >> v2:172.21.15.76:3300/0 conn(0x7f305415b450 0x7f30540b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 -- 172.21.15.59:0/976291307 >> 172.21.15.59:0/976291307 conn(0x7f30540b21a0 msgr2=0x7f30540b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 -- 172.21.15.59:0/976291307 shutdown_connections 2024-04-23T14:33:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.046+0000 7f3058e00640 1 -- 172.21.15.59:0/976291307 wait complete. 2024-04-23T14:33:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.047+0000 7f3058e00640 1 Processor -- start 2024-04-23T14:33:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.048+0000 7f3058e00640 1 -- start start 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3058e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f305406f3f0 0x7f30540c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3058e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30540b3080 0x7f3054071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3058e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305415b450 0x7f30540999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3058e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30540b4ff0 con 0x7f30540b3080 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3058e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f305415b830 con 0x7f305415b450 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3058e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30540ad060 con 0x7f305406f3f0 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30540b3080 0x7f3054071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30540b3080 0x7f3054071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41430/0 (socket says 172.21.15.59:41430) 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 -- 172.21.15.59:0/676734662 learned_addr learned my addr 172.21.15.59:0/676734662 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3053400640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f305406f3f0 0x7f30540c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3053e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305415b450 0x7f30540999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 -- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f305406f3f0 msgr2=0x7f30540c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f305406f3f0 0x7f30540c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 -- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305415b450 msgr2=0x7f30540999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305415b450 0x7f30540999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f304be00640 1 -- 172.21.15.59:0/676734662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3054099f00 con 0x7f30540b3080 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3053e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305415b450 0x7f30540999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.049+0000 7f3053400640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f305406f3f0 0x7f30540c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.050+0000 7f304be00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30540b3080 0x7f3054071ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f303c000c00 tx=0x7f303c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.050+0000 7f304b400640 1 -- 172.21.15.59:0/676734662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c041020 con 0x7f30540b3080 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.050+0000 7f304b400640 1 -- 172.21.15.59:0/676734662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f303c0306c0 con 0x7f30540b3080 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.050+0000 7f304b400640 1 -- 172.21.15.59:0/676734662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c037040 con 0x7f30540b3080 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.050+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30540c6640 con 0x7f30540b3080 2024-04-23T14:33:53.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.050+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3054072540 con 0x7f30540b3080 2024-04-23T14:33:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.051+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3054072330 con 0x7f30540b3080 2024-04-23T14:33:53.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.052+0000 7f304b400640 1 -- 172.21.15.59:0/676734662 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f303c044020 con 0x7f30540b3080 2024-04-23T14:33:53.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.053+0000 7f304b400640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f303006c8f0 0x7f303006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.053+0000 7f304b400640 1 -- 172.21.15.59:0/676734662 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f303c0ac440 con 0x7f30540b3080 2024-04-23T14:33:53.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.054+0000 7f3053400640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f303006c8f0 0x7f303006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.054+0000 7f3053400640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f303006c8f0 0x7f303006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3044006fd0 tx=0x7f3044008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.058+0000 7f304b400640 1 -- 172.21.15.59:0/676734662 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f303c04b030 con 0x7f30540b3080 2024-04-23T14:33:53.238 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:33:53.237+0000 7fc2aa400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:33:53.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.257+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f305404fd90 con 0x7f303006c8f0 2024-04-23T14:33:53.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.276+0000 7f304b400640 1 -- 172.21.15.59:0/676734662 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f305404fd90 con 0x7f303006c8f0 2024-04-23T14:33:53.281 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:53.282 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f303006c8f0 msgr2=0x7f303006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f303006c8f0 0x7f303006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3044006fd0 tx=0x7f3044008040 comp rx=0 tx=0).stop 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30540b3080 msgr2=0x7f3054071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30540b3080 0x7f3054071ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f303c000c00 tx=0x7f303c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 shutdown_connections 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f303006c8f0 0x7f303006edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305415b450 0x7f30540999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30540b3080 0x7f3054071ca0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.282+0000 7f3058e00640 1 --2- 172.21.15.59:0/676734662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f305406f3f0 0x7f30540c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.283+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 >> 172.21.15.59:0/676734662 conn(0x7f30540b21a0 msgr2=0x7f305406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.283+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 shutdown_connections 2024-04-23T14:33:53.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.283+0000 7f3058e00640 1 -- 172.21.15.59:0/676734662 wait complete. 2024-04-23T14:33:53.303 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-04-23T14:33:52.389738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225492,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749228,"statfs":{"total":773094113280,"available":772863209472,"internally_reserved":0,"allocated":5283840,"data_stored":2813556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001410"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.328107+0000","last_change":"2024-04-23T14:33:48.321926+0000","last_active":"2024-04-23T14:33:48.328107+0000","last_peered":"2024-04-23T14:33:48.328107+0000","last_clean":"2024-04-23T14:33:48.328107+0000","last_became_active":"2024-04-23T14:33:48.321423+0000","last_became_peered":"2024-04-23T14:33:48.321423+0000","last_unstale":"2024-04-23T14:33:48.328107+0000","last_undegraded":"2024-04-23T14:33:48.328107+0000","last_fullsized":"2024-04-23T14:33:48.328107+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:22.631287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345783+0000","last_change":"2024-04-23T14:33:48.342873+0000","last_active":"2024-04-23T14:33:48.345783+0000","last_peered":"2024-04-23T14:33:48.345783+0000","last_clean":"2024-04-23T14:33:48.345783+0000","last_became_active":"2024-04-23T14:33:48.342459+0000","last_became_peered":"2024-04-23T14:33:48.342459+0000","last_unstale":"2024-04-23T14:33:48.345783+0000","last_undegraded":"2024-04-23T14:33:48.345783+0000","last_fullsized":"2024-04-23T14:33:48.345783+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:54.066369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324896+0000","last_change":"2024-04-23T14:33:48.312037+0000","last_active":"2024-04-23T14:33:49.324896+0000","last_peered":"2024-04-23T14:33:49.324896+0000","last_clean":"2024-04-23T14:33:49.324896+0000","last_became_active":"2024-04-23T14:33:48.310067+0000","last_became_peered":"2024-04-23T14:33:48.310067+0000","last_unstale":"2024-04-23T14:33:49.324896+0000","last_undegraded":"2024-04-23T14:33:49.324896+0000","last_fullsized":"2024-04-23T14:33:49.324896+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.341028+0000","last_change":"2024-04-23T14:33:48.332031+0000","last_active":"2024-04-23T14:33:48.341028+0000","last_peered":"2024-04-23T14:33:48.341028+0000","last_clean":"2024-04-23T14:33:48.341028+0000","last_became_active":"2024-04-23T14:33:48.331754+0000","last_became_peered":"2024-04-23T14:33:48.331754+0000","last_unstale":"2024-04-23T14:33:48.341028+0000","last_undegraded":"2024-04-23T14:33:48.341028+0000","last_fullsized":"2024-04-23T14:33:48.341028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:44.494032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.316735+0000","last_change":"2024-04-23T14:33:48.313724+0000","last_active":"2024-04-23T14:33:48.316735+0000","last_peered":"2024-04-23T14:33:48.316735+0000","last_clean":"2024-04-23T14:33:48.316735+0000","last_became_active":"2024-04-23T14:33:48.312688+0000","last_became_peered":"2024-04-23T14:33:48.312688+0000","last_unstale":"2024-04-23T14:33:48.316735+0000","last_undegraded":"2024-04-23T14:33:48.316735+0000","last_fullsized":"2024-04-23T14:33:48.316735+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:36:58.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317603+0000","last_change":"2024-04-23T14:33:48.325815+0000","last_active":"2024-04-23T14:33:49.317603+0000","last_peered":"2024-04-23T14:33:49.317603+0000","last_clean":"2024-04-23T14:33:49.317603+0000","last_became_active":"2024-04-23T14:33:48.324428+0000","last_became_peered":"2024-04-23T14:33:48.324428+0000","last_unstale":"2024-04-23T14:33:49.317603+0000","last_undegraded":"2024-04-23T14:33:49.317603+0000","last_fullsized":"2024-04-23T14:33:49.317603+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:20:39.048353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333655+0000","last_change":"2024-04-23T14:33:48.329353+0000","last_active":"2024-04-23T14:33:48.333655+0000","last_peered":"2024-04-23T14:33:48.333655+0000","last_clean":"2024-04-23T14:33:48.333655+0000","last_became_active":"2024-04-23T14:33:48.328514+0000","last_became_peered":"2024-04-23T14:33:48.328514+0000","last_unstale":"2024-04-23T14:33:48.333655+0000","last_undegraded":"2024-04-23T14:33:48.333655+0000","last_fullsized":"2024-04-23T14:33:48.333655+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:06.651665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345268+0000","last_change":"2024-04-23T14:33:48.337833+0000","last_active":"2024-04-23T14:33:48.345268+0000","last_peered":"2024-04-23T14:33:48.345268+0000","last_clean":"2024-04-23T14:33:48.345268+0000","last_became_active":"2024-04-23T14:33:48.337524+0000","last_became_peered":"2024-04-23T14:33:48.337524+0000","last_unstale":"2024-04-23T14:33:48.345268+0000","last_undegraded":"2024-04-23T14:33:48.345268+0000","last_fullsized":"2024-04-23T14:33:48.345268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:03.844780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345936+0000","last_change":"2024-04-23T14:33:48.343905+0000","last_active":"2024-04-23T14:33:48.345936+0000","last_peered":"2024-04-23T14:33:48.345936+0000","last_clean":"2024-04-23T14:33:48.345936+0000","last_became_active":"2024-04-23T14:33:48.341293+0000","last_became_peered":"2024-04-23T14:33:48.341293+0000","last_unstale":"2024-04-23T14:33:48.345936+0000","last_undegraded":"2024-04-23T14:33:48.345936+0000","last_fullsized":"2024-04-23T14:33:48.345936+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:46.678623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.344104+0000","last_change":"2024-04-23T14:33:48.340522+0000","last_active":"2024-04-23T14:33:48.344104+0000","last_peered":"2024-04-23T14:33:48.344104+0000","last_clean":"2024-04-23T14:33:48.344104+0000","last_became_active":"2024-04-23T14:33:48.340036+0000","last_became_peered":"2024-04-23T14:33:48.340036+0000","last_unstale":"2024-04-23T14:33:48.344104+0000","last_undegraded":"2024-04-23T14:33:48.344104+0000","last_fullsized":"2024-04-23T14:33:48.344104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:26:13.191488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.343006+0000","last_change":"2024-04-23T14:33:48.329993+0000","last_active":"2024-04-23T14:33:48.343006+0000","last_peered":"2024-04-23T14:33:48.343006+0000","last_clean":"2024-04-23T14:33:48.343006+0000","last_became_active":"2024-04-23T14:33:48.329697+0000","last_became_peered":"2024-04-23T14:33:48.329697+0000","last_unstale":"2024-04-23T14:33:48.343006+0000","last_undegraded":"2024-04-23T14:33:48.343006+0000","last_fullsized":"2024-04-23T14:33:48.343006+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:51:19.145785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.321275+0000","last_change":"2024-04-23T14:33:48.317001+0000","last_active":"2024-04-23T14:33:48.321275+0000","last_peered":"2024-04-23T14:33:48.321275+0000","last_clean":"2024-04-23T14:33:48.321275+0000","last_became_active":"2024-04-23T14:33:48.316610+0000","last_became_peered":"2024-04-23T14:33:48.316610+0000","last_unstale":"2024-04-23T14:33:48.321275+0000","last_undegraded":"2024-04-23T14:33:48.321275+0000","last_fullsized":"2024-04-23T14:33:48.321275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:22.631287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313354+0000","last_change":"2024-04-23T14:33:48.325217+0000","last_active":"2024-04-23T14:33:49.313354+0000","last_peered":"2024-04-23T14:33:49.313354+0000","last_clean":"2024-04-23T14:33:49.313354+0000","last_became_active":"2024-04-23T14:33:48.324778+0000","last_became_peered":"2024-04-23T14:33:48.324778+0000","last_unstale":"2024-04-23T14:33:49.313354+0000","last_undegraded":"2024-04-23T14:33:49.313354+0000","last_fullsized":"2024-04-23T14:33:49.313354+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:43.137905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.341136+0000","last_change":"2024-04-23T14:33:48.332619+0000","last_active":"2024-04-23T14:33:48.341136+0000","last_peered":"2024-04-23T14:33:48.341136+0000","last_clean":"2024-04-23T14:33:48.341136+0000","last_became_active":"2024-04-23T14:33:48.332272+0000","last_became_peered":"2024-04-23T14:33:48.332272+0000","last_unstale":"2024-04-23T14:33:48.341136+0000","last_undegraded":"2024-04-23T14:33:48.341136+0000","last_fullsized":"2024-04-23T14:33:48.341136+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:18:18.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317495+0000","last_change":"2024-04-23T14:33:48.326660+0000","last_active":"2024-04-23T14:33:49.317495+0000","last_peered":"2024-04-23T14:33:49.317495+0000","last_clean":"2024-04-23T14:33:49.317495+0000","last_became_active":"2024-04-23T14:33:48.326311+0000","last_became_peered":"2024-04-23T14:33:48.326311+0000","last_unstale":"2024-04-23T14:33:49.317495+0000","last_undegraded":"2024-04-23T14:33:49.317495+0000","last_fullsized":"2024-04-23T14:33:49.317495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:14:26.838855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316307+0000","last_change":"2024-04-23T14:33:48.315387+0000","last_active":"2024-04-23T14:33:49.316307+0000","last_peered":"2024-04-23T14:33:49.316307+0000","last_clean":"2024-04-23T14:33:49.316307+0000","last_became_active":"2024-04-23T14:33:48.314997+0000","last_became_peered":"2024-04-23T14:33:48.314997+0000","last_unstale":"2024-04-23T14:33:49.316307+0000","last_undegraded":"2024-04-23T14:33:49.316307+0000","last_fullsized":"2024-04-23T14:33:49.316307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.597982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325442+0000","last_change":"2024-04-23T14:33:48.313695+0000","last_active":"2024-04-23T14:33:49.325442+0000","last_peered":"2024-04-23T14:33:49.325442+0000","last_clean":"2024-04-23T14:33:49.325442+0000","last_became_active":"2024-04-23T14:33:48.313399+0000","last_became_peered":"2024-04-23T14:33:48.313399+0000","last_unstale":"2024-04-23T14:33:49.325442+0000","last_undegraded":"2024-04-23T14:33:49.325442+0000","last_fullsized":"2024-04-23T14:33:49.325442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:36:58.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325292+0000","last_change":"2024-04-23T14:33:48.343875+0000","last_active":"2024-04-23T14:33:49.325292+0000","last_peered":"2024-04-23T14:33:49.325292+0000","last_clean":"2024-04-23T14:33:49.325292+0000","last_became_active":"2024-04-23T14:33:48.343598+0000","last_became_peered":"2024-04-23T14:33:48.343598+0000","last_unstale":"2024-04-23T14:33:49.325292+0000","last_undegraded":"2024-04-23T14:33:49.325292+0000","last_fullsized":"2024-04-23T14:33:49.325292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:12.241975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318012+0000","last_change":"2024-04-23T14:33:48.317712+0000","last_active":"2024-04-23T14:33:49.318012+0000","last_peered":"2024-04-23T14:33:49.318012+0000","last_clean":"2024-04-23T14:33:49.318012+0000","last_became_active":"2024-04-23T14:33:48.317428+0000","last_became_peered":"2024-04-23T14:33:48.317428+0000","last_unstale":"2024-04-23T14:33:49.318012+0000","last_undegraded":"2024-04-23T14:33:49.318012+0000","last_fullsized":"2024-04-23T14:33:49.318012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313368+0000","last_change":"2024-04-23T14:33:48.311951+0000","last_active":"2024-04-23T14:33:49.313368+0000","last_peered":"2024-04-23T14:33:49.313368+0000","last_clean":"2024-04-23T14:33:49.313368+0000","last_became_active":"2024-04-23T14:33:48.311660+0000","last_became_peered":"2024-04-23T14:33:48.311660+0000","last_unstale":"2024-04-23T14:33:49.313368+0000","last_undegraded":"2024-04-23T14:33:49.313368+0000","last_fullsized":"2024-04-23T14:33:49.313368+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:31.281376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317795+0000","last_change":"2024-04-23T14:33:48.328839+0000","last_active":"2024-04-23T14:33:49.317795+0000","last_peered":"2024-04-23T14:33:49.317795+0000","last_clean":"2024-04-23T14:33:49.317795+0000","last_became_active":"2024-04-23T14:33:48.328534+0000","last_became_peered":"2024-04-23T14:33:48.328534+0000","last_unstale":"2024-04-23T14:33:49.317795+0000","last_undegraded":"2024-04-23T14:33:49.317795+0000","last_fullsized":"2024-04-23T14:33:49.317795+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:57.739584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.343122+0000","last_change":"2024-04-23T14:33:48.338168+0000","last_active":"2024-04-23T14:33:48.343122+0000","last_peered":"2024-04-23T14:33:48.343122+0000","last_clean":"2024-04-23T14:33:48.343122+0000","last_became_active":"2024-04-23T14:33:48.337844+0000","last_became_peered":"2024-04-23T14:33:48.337844+0000","last_unstale":"2024-04-23T14:33:48.343122+0000","last_undegraded":"2024-04-23T14:33:48.343122+0000","last_fullsized":"2024-04-23T14:33:48.343122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:07:23.859553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316685+0000","last_change":"2024-04-23T14:33:48.324242+0000","last_active":"2024-04-23T14:33:49.316685+0000","last_peered":"2024-04-23T14:33:49.316685+0000","last_clean":"2024-04-23T14:33:49.316685+0000","last_became_active":"2024-04-23T14:33:48.323951+0000","last_became_peered":"2024-04-23T14:33:48.323951+0000","last_unstale":"2024-04-23T14:33:49.316685+0000","last_undegraded":"2024-04-23T14:33:49.316685+0000","last_fullsized":"2024-04-23T14:33:49.316685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:18:18.578677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316997+0000","last_change":"2024-04-23T14:33:48.321349+0000","last_active":"2024-04-23T14:33:49.316997+0000","last_peered":"2024-04-23T14:33:49.316997+0000","last_clean":"2024-04-23T14:33:49.316997+0000","last_became_active":"2024-04-23T14:33:48.321076+0000","last_became_peered":"2024-04-23T14:33:48.321076+0000","last_unstale":"2024-04-23T14:33:49.316997+0000","last_undegraded":"2024-04-23T14:33:49.316997+0000","last_fullsized":"2024-04-23T14:33:49.316997+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:44.494032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317463+0000","last_change":"2024-04-23T14:33:48.308977+0000","last_active":"2024-04-23T14:33:49.317463+0000","last_peered":"2024-04-23T14:33:49.317463+0000","last_clean":"2024-04-23T14:33:49.317463+0000","last_became_active":"2024-04-23T14:33:48.308401+0000","last_became_peered":"2024-04-23T14:33:48.308401+0000","last_unstale":"2024-04-23T14:33:49.317463+0000","last_undegraded":"2024-04-23T14:33:49.317463+0000","last_fullsized":"2024-04-23T14:33:49.317463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:07:23.859553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313592+0000","last_change":"2024-04-23T14:33:48.325154+0000","last_active":"2024-04-23T14:33:49.313592+0000","last_peered":"2024-04-23T14:33:49.313592+0000","last_clean":"2024-04-23T14:33:49.313592+0000","last_became_active":"2024-04-23T14:33:48.323045+0000","last_became_peered":"2024-04-23T14:33:48.323045+0000","last_unstale":"2024-04-23T14:33:49.313592+0000","last_undegraded":"2024-04-23T14:33:49.313592+0000","last_fullsized":"2024-04-23T14:33:49.313592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:27:12.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325125+0000","last_change":"2024-04-23T14:33:48.312852+0000","last_active":"2024-04-23T14:33:49.325125+0000","last_peered":"2024-04-23T14:33:49.325125+0000","last_clean":"2024-04-23T14:33:49.325125+0000","last_became_active":"2024-04-23T14:33:48.312497+0000","last_became_peered":"2024-04-23T14:33:48.312497+0000","last_unstale":"2024-04-23T14:33:49.325125+0000","last_undegraded":"2024-04-23T14:33:49.325125+0000","last_fullsized":"2024-04-23T14:33:49.325125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:46.678623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.316860+0000","last_change":"2024-04-23T14:33:48.312397+0000","last_active":"2024-04-23T14:33:48.316860+0000","last_peered":"2024-04-23T14:33:48.316860+0000","last_clean":"2024-04-23T14:33:48.316860+0000","last_became_active":"2024-04-23T14:33:48.311439+0000","last_became_peered":"2024-04-23T14:33:48.311439+0000","last_unstale":"2024-04-23T14:33:48.316860+0000","last_undegraded":"2024-04-23T14:33:48.316860+0000","last_fullsized":"2024-04-23T14:33:48.316860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:54.040413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317491+0000","last_change":"2024-04-23T14:33:48.300661+0000","last_active":"2024-04-23T14:33:49.317491+0000","last_peered":"2024-04-23T14:33:49.317491+0000","last_clean":"2024-04-23T14:33:49.317491+0000","last_became_active":"2024-04-23T14:33:48.299830+0000","last_became_peered":"2024-04-23T14:33:48.299830+0000","last_unstale":"2024-04-23T14:33:49.317491+0000","last_undegraded":"2024-04-23T14:33:49.317491+0000","last_fullsized":"2024-04-23T14:33:49.317491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:26:13.191488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324297+0000","last_change":"2024-04-23T14:33:48.321861+0000","last_active":"2024-04-23T14:33:49.324297+0000","last_peered":"2024-04-23T14:33:49.324297+0000","last_clean":"2024-04-23T14:33:49.324297+0000","last_became_active":"2024-04-23T14:33:48.321539+0000","last_became_peered":"2024-04-23T14:33:48.321539+0000","last_unstale":"2024-04-23T14:33:49.324297+0000","last_undegraded":"2024-04-23T14:33:49.324297+0000","last_fullsized":"2024-04-23T14:33:49.324297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:07.788298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317977+0000","last_change":"2024-04-23T14:33:48.329009+0000","last_active":"2024-04-23T14:33:49.317977+0000","last_peered":"2024-04-23T14:33:49.317977+0000","last_clean":"2024-04-23T14:33:49.317977+0000","last_became_active":"2024-04-23T14:33:48.327929+0000","last_became_peered":"2024-04-23T14:33:48.327929+0000","last_unstale":"2024-04-23T14:33:49.317977+0000","last_undegraded":"2024-04-23T14:33:49.317977+0000","last_fullsized":"2024-04-23T14:33:49.317977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:42:43.613375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313222+0000","last_change":"2024-04-23T14:33:48.304570+0000","last_active":"2024-04-23T14:33:49.313222+0000","last_peered":"2024-04-23T14:33:49.313222+0000","last_clean":"2024-04-23T14:33:49.313222+0000","last_became_active":"2024-04-23T14:33:48.304128+0000","last_became_peered":"2024-04-23T14:33:48.304128+0000","last_unstale":"2024-04-23T14:33:49.313222+0000","last_undegraded":"2024-04-23T14:33:49.313222+0000","last_fullsized":"2024-04-23T14:33:49.313222+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:33:45.978046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315803+0000","last_change":"2024-04-23T14:33:48.326863+0000","last_active":"2024-04-23T14:33:49.315803+0000","last_peered":"2024-04-23T14:33:49.315803+0000","last_clean":"2024-04-23T14:33:49.315803+0000","last_became_active":"2024-04-23T14:33:48.326501+0000","last_became_peered":"2024-04-23T14:33:48.326501+0000","last_unstale":"2024-04-23T14:33:49.315803+0000","last_undegraded":"2024-04-23T14:33:49.315803+0000","last_fullsized":"2024-04-23T14:33:49.315803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318386+0000","last_change":"2024-04-23T14:33:47.305121+0000","last_active":"2024-04-23T14:33:49.318386+0000","last_peered":"2024-04-23T14:33:49.318386+0000","last_clean":"2024-04-23T14:33:49.318386+0000","last_became_active":"2024-04-23T14:33:47.304801+0000","last_became_peered":"2024-04-23T14:33:47.304801+0000","last_unstale":"2024-04-23T14:33:49.318386+0000","last_undegraded":"2024-04-23T14:33:49.318386+0000","last_fullsized":"2024-04-23T14:33:49.318386+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:26.039708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317792+0000","last_change":"2024-04-23T14:33:48.308703+0000","last_active":"2024-04-23T14:33:49.317792+0000","last_peered":"2024-04-23T14:33:49.317792+0000","last_clean":"2024-04-23T14:33:49.317792+0000","last_became_active":"2024-04-23T14:33:48.307980+0000","last_became_peered":"2024-04-23T14:33:48.307980+0000","last_unstale":"2024-04-23T14:33:49.317792+0000","last_undegraded":"2024-04-23T14:33:49.317792+0000","last_fullsized":"2024-04-23T14:33:49.317792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:03.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325648+0000","last_change":"2024-04-23T14:33:47.318201+0000","last_active":"2024-04-23T14:33:49.325648+0000","last_peered":"2024-04-23T14:33:49.325648+0000","last_clean":"2024-04-23T14:33:49.325648+0000","last_became_active":"2024-04-23T14:33:47.317331+0000","last_became_peered":"2024-04-23T14:33:47.317331+0000","last_unstale":"2024-04-23T14:33:49.325648+0000","last_undegraded":"2024-04-23T14:33:49.325648+0000","last_fullsized":"2024-04-23T14:33:49.325648+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:33:30.339190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.331632+0000","last_change":"2024-04-23T14:33:48.325624+0000","last_active":"2024-04-23T14:33:48.331632+0000","last_peered":"2024-04-23T14:33:48.331632+0000","last_clean":"2024-04-23T14:33:48.331632+0000","last_became_active":"2024-04-23T14:33:48.325348+0000","last_became_peered":"2024-04-23T14:33:48.325348+0000","last_unstale":"2024-04-23T14:33:48.331632+0000","last_undegraded":"2024-04-23T14:33:48.331632+0000","last_fullsized":"2024-04-23T14:33:48.331632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:52.757461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315985+0000","last_change":"2024-04-23T14:33:47.314246+0000","last_active":"2024-04-23T14:33:49.315985+0000","last_peered":"2024-04-23T14:33:49.315985+0000","last_clean":"2024-04-23T14:33:49.315985+0000","last_became_active":"2024-04-23T14:33:47.313919+0000","last_became_peered":"2024-04-23T14:33:47.313919+0000","last_unstale":"2024-04-23T14:33:49.315985+0000","last_undegraded":"2024-04-23T14:33:49.315985+0000","last_fullsized":"2024-04-23T14:33:49.315985+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:07:22.844922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315873+0000","last_change":"2024-04-23T14:33:48.301837+0000","last_active":"2024-04-23T14:33:49.315873+0000","last_peered":"2024-04-23T14:33:49.315873+0000","last_clean":"2024-04-23T14:33:49.315873+0000","last_became_active":"2024-04-23T14:33:48.301403+0000","last_became_peered":"2024-04-23T14:33:48.301403+0000","last_unstale":"2024-04-23T14:33:49.315873+0000","last_undegraded":"2024-04-23T14:33:49.315873+0000","last_fullsized":"2024-04-23T14:33:49.315873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:46.678623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.332026+0000","last_change":"2024-04-23T14:33:47.316110+0000","last_active":"2024-04-23T14:33:48.332026+0000","last_peered":"2024-04-23T14:33:48.332026+0000","last_clean":"2024-04-23T14:33:48.332026+0000","last_became_active":"2024-04-23T14:33:47.314849+0000","last_became_peered":"2024-04-23T14:33:47.314849+0000","last_unstale":"2024-04-23T14:33:48.332026+0000","last_undegraded":"2024-04-23T14:33:48.332026+0000","last_fullsized":"2024-04-23T14:33:48.332026+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:03.505844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324369+0000","last_change":"2024-04-23T14:33:48.317452+0000","last_active":"2024-04-23T14:33:49.324369+0000","last_peered":"2024-04-23T14:33:49.324369+0000","last_clean":"2024-04-23T14:33:49.324369+0000","last_became_active":"2024-04-23T14:33:48.316171+0000","last_became_peered":"2024-04-23T14:33:48.316171+0000","last_unstale":"2024-04-23T14:33:49.324369+0000","last_undegraded":"2024-04-23T14:33:49.324369+0000","last_fullsized":"2024-04-23T14:33:49.324369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:03.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317917+0000","last_change":"2024-04-23T14:33:47.314100+0000","last_active":"2024-04-23T14:33:49.317917+0000","last_peered":"2024-04-23T14:33:49.317917+0000","last_clean":"2024-04-23T14:33:49.317917+0000","last_became_active":"2024-04-23T14:33:47.313569+0000","last_became_peered":"2024-04-23T14:33:47.313569+0000","last_unstale":"2024-04-23T14:33:49.317917+0000","last_undegraded":"2024-04-23T14:33:49.317917+0000","last_fullsized":"2024-04-23T14:33:49.317917+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:05.637034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.343601+0000","last_change":"2024-04-23T14:33:48.330366+0000","last_active":"2024-04-23T14:33:48.343601+0000","last_peered":"2024-04-23T14:33:48.343601+0000","last_clean":"2024-04-23T14:33:48.343601+0000","last_became_active":"2024-04-23T14:33:48.330016+0000","last_became_peered":"2024-04-23T14:33:48.330016+0000","last_unstale":"2024-04-23T14:33:48.343601+0000","last_undegraded":"2024-04-23T14:33:48.343601+0000","last_fullsized":"2024-04-23T14:33:48.343601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:19:54.597982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324527+0000","last_change":"2024-04-23T14:33:47.313664+0000","last_active":"2024-04-23T14:33:49.324527+0000","last_peered":"2024-04-23T14:33:49.324527+0000","last_clean":"2024-04-23T14:33:49.324527+0000","last_became_active":"2024-04-23T14:33:47.313357+0000","last_became_peered":"2024-04-23T14:33:47.313357+0000","last_unstale":"2024-04-23T14:33:49.324527+0000","last_undegraded":"2024-04-23T14:33:49.324527+0000","last_fullsized":"2024-04-23T14:33:49.324527+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:00.482836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345123+0000","last_change":"2024-04-23T14:33:48.337306+0000","last_active":"2024-04-23T14:33:48.345123+0000","last_peered":"2024-04-23T14:33:48.345123+0000","last_clean":"2024-04-23T14:33:48.345123+0000","last_became_active":"2024-04-23T14:33:48.331057+0000","last_became_peered":"2024-04-23T14:33:48.331057+0000","last_unstale":"2024-04-23T14:33:48.345123+0000","last_undegraded":"2024-04-23T14:33:48.345123+0000","last_fullsized":"2024-04-23T14:33:48.345123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:59.470476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.305195+0000","last_change":"2024-04-23T14:33:47.309849+0000","last_active":"2024-04-23T14:33:48.305195+0000","last_peered":"2024-04-23T14:33:48.305195+0000","last_clean":"2024-04-23T14:33:48.305195+0000","last_became_active":"2024-04-23T14:33:47.309531+0000","last_became_peered":"2024-04-23T14:33:47.309531+0000","last_unstale":"2024-04-23T14:33:48.305195+0000","last_undegraded":"2024-04-23T14:33:48.305195+0000","last_fullsized":"2024-04-23T14:33:48.305195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:02.239629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316341+0000","last_change":"2024-04-23T14:33:48.323236+0000","last_active":"2024-04-23T14:33:49.316341+0000","last_peered":"2024-04-23T14:33:49.316341+0000","last_clean":"2024-04-23T14:33:49.316341+0000","last_became_active":"2024-04-23T14:33:48.322974+0000","last_became_peered":"2024-04-23T14:33:48.322974+0000","last_unstale":"2024-04-23T14:33:49.316341+0000","last_undegraded":"2024-04-23T14:33:49.316341+0000","last_fullsized":"2024-04-23T14:33:49.316341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.644777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.314186+0000","last_change":"2024-04-23T14:33:47.309224+0000","last_active":"2024-04-23T14:33:48.314186+0000","last_peered":"2024-04-23T14:33:48.314186+0000","last_clean":"2024-04-23T14:33:48.314186+0000","last_became_active":"2024-04-23T14:33:47.308924+0000","last_became_peered":"2024-04-23T14:33:47.308924+0000","last_unstale":"2024-04-23T14:33:48.314186+0000","last_undegraded":"2024-04-23T14:33:48.314186+0000","last_fullsized":"2024-04-23T14:33:48.314186+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:00.482836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325305+0000","last_change":"2024-04-23T14:33:48.341446+0000","last_active":"2024-04-23T14:33:49.325305+0000","last_peered":"2024-04-23T14:33:49.325305+0000","last_clean":"2024-04-23T14:33:49.325305+0000","last_became_active":"2024-04-23T14:33:48.341061+0000","last_became_peered":"2024-04-23T14:33:48.341061+0000","last_unstale":"2024-04-23T14:33:49.325305+0000","last_undegraded":"2024-04-23T14:33:49.325305+0000","last_fullsized":"2024-04-23T14:33:49.325305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:54.040413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325735+0000","last_change":"2024-04-23T14:33:47.312006+0000","last_active":"2024-04-23T14:33:49.325735+0000","last_peered":"2024-04-23T14:33:49.325735+0000","last_clean":"2024-04-23T14:33:49.325735+0000","last_became_active":"2024-04-23T14:33:47.311248+0000","last_became_peered":"2024-04-23T14:33:47.311248+0000","last_unstale":"2024-04-23T14:33:49.325735+0000","last_undegraded":"2024-04-23T14:33:49.325735+0000","last_fullsized":"2024-04-23T14:33:49.325735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:34.096326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.328063+0000","last_change":"2024-04-23T14:33:48.320593+0000","last_active":"2024-04-23T14:33:48.328063+0000","last_peered":"2024-04-23T14:33:48.328063+0000","last_clean":"2024-04-23T14:33:48.328063+0000","last_became_active":"2024-04-23T14:33:48.320317+0000","last_became_peered":"2024-04-23T14:33:48.320317+0000","last_unstale":"2024-04-23T14:33:48.328063+0000","last_undegraded":"2024-04-23T14:33:48.328063+0000","last_fullsized":"2024-04-23T14:33:48.328063+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:36:58.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.330807+0000","last_change":"2024-04-23T14:33:47.312763+0000","last_active":"2024-04-23T14:33:48.330807+0000","last_peered":"2024-04-23T14:33:48.330807+0000","last_clean":"2024-04-23T14:33:48.330807+0000","last_became_active":"2024-04-23T14:33:47.312439+0000","last_became_peered":"2024-04-23T14:33:47.312439+0000","last_unstale":"2024-04-23T14:33:48.330807+0000","last_undegraded":"2024-04-23T14:33:48.330807+0000","last_fullsized":"2024-04-23T14:33:48.330807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:15.630146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318073+0000","last_change":"2024-04-23T14:33:48.332776+0000","last_active":"2024-04-23T14:33:49.318073+0000","last_peered":"2024-04-23T14:33:49.318073+0000","last_clean":"2024-04-23T14:33:49.318073+0000","last_became_active":"2024-04-23T14:33:48.332491+0000","last_became_peered":"2024-04-23T14:33:48.332491+0000","last_unstale":"2024-04-23T14:33:49.318073+0000","last_undegraded":"2024-04-23T14:33:49.318073+0000","last_fullsized":"2024-04-23T14:33:49.318073+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:13.732846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333668+0000","last_change":"2024-04-23T14:33:47.315536+0000","last_active":"2024-04-23T14:33:48.333668+0000","last_peered":"2024-04-23T14:33:48.333668+0000","last_clean":"2024-04-23T14:33:48.333668+0000","last_became_active":"2024-04-23T14:33:47.314869+0000","last_became_peered":"2024-04-23T14:33:47.314869+0000","last_unstale":"2024-04-23T14:33:48.333668+0000","last_undegraded":"2024-04-23T14:33:48.333668+0000","last_fullsized":"2024-04-23T14:33:48.333668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:11.227344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.328063+0000","last_change":"2024-04-23T14:33:48.322586+0000","last_active":"2024-04-23T14:33:48.328063+0000","last_peered":"2024-04-23T14:33:48.328063+0000","last_clean":"2024-04-23T14:33:48.328063+0000","last_became_active":"2024-04-23T14:33:48.321940+0000","last_became_peered":"2024-04-23T14:33:48.321940+0000","last_unstale":"2024-04-23T14:33:48.328063+0000","last_undegraded":"2024-04-23T14:33:48.328063+0000","last_fullsized":"2024-04-23T14:33:48.328063+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:52.757461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313361+0000","last_change":"2024-04-23T14:33:47.313051+0000","last_active":"2024-04-23T14:33:49.313361+0000","last_peered":"2024-04-23T14:33:49.313361+0000","last_clean":"2024-04-23T14:33:49.313361+0000","last_became_active":"2024-04-23T14:33:47.311724+0000","last_became_peered":"2024-04-23T14:33:47.311724+0000","last_unstale":"2024-04-23T14:33:49.313361+0000","last_undegraded":"2024-04-23T14:33:49.313361+0000","last_fullsized":"2024-04-23T14:33:49.313361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:48:25.641075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.316891+0000","last_change":"2024-04-23T14:33:48.336022+0000","last_active":"2024-04-23T14:33:49.316891+0000","last_peered":"2024-04-23T14:33:49.316891+0000","last_clean":"2024-04-23T14:33:49.316891+0000","last_became_active":"2024-04-23T14:33:48.335781+0000","last_became_peered":"2024-04-23T14:33:48.335781+0000","last_unstale":"2024-04-23T14:33:49.316891+0000","last_undegraded":"2024-04-23T14:33:49.316891+0000","last_fullsized":"2024-04-23T14:33:49.316891+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:59.470476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318036+0000","last_change":"2024-04-23T14:33:47.305908+0000","last_active":"2024-04-23T14:33:49.318036+0000","last_peered":"2024-04-23T14:33:49.318036+0000","last_clean":"2024-04-23T14:33:49.318036+0000","last_became_active":"2024-04-23T14:33:47.305523+0000","last_became_peered":"2024-04-23T14:33:47.305523+0000","last_unstale":"2024-04-23T14:33:49.318036+0000","last_undegraded":"2024-04-23T14:33:49.318036+0000","last_fullsized":"2024-04-23T14:33:49.318036+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:17.332622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345252+0000","last_change":"2024-04-23T14:33:48.343162+0000","last_active":"2024-04-23T14:33:48.345252+0000","last_peered":"2024-04-23T14:33:48.345252+0000","last_clean":"2024-04-23T14:33:48.345252+0000","last_became_active":"2024-04-23T14:33:48.338678+0000","last_became_peered":"2024-04-23T14:33:48.338678+0000","last_unstale":"2024-04-23T14:33:48.345252+0000","last_undegraded":"2024-04-23T14:33:48.345252+0000","last_fullsized":"2024-04-23T14:33:48.345252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.308862+0000","last_change":"2024-04-23T14:33:47.317355+0000","last_active":"2024-04-23T14:33:48.308862+0000","last_peered":"2024-04-23T14:33:48.308862+0000","last_clean":"2024-04-23T14:33:48.308862+0000","last_became_active":"2024-04-23T14:33:47.314263+0000","last_became_peered":"2024-04-23T14:33:47.314263+0000","last_unstale":"2024-04-23T14:33:48.308862+0000","last_undegraded":"2024-04-23T14:33:48.308862+0000","last_fullsized":"2024-04-23T14:33:48.308862+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:45.663992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317503+0000","last_change":"2024-04-23T14:33:48.328866+0000","last_active":"2024-04-23T14:33:49.317503+0000","last_peered":"2024-04-23T14:33:49.317503+0000","last_clean":"2024-04-23T14:33:49.317503+0000","last_became_active":"2024-04-23T14:33:48.328570+0000","last_became_peered":"2024-04-23T14:33:48.328570+0000","last_unstale":"2024-04-23T14:33:49.317503+0000","last_undegraded":"2024-04-23T14:33:49.317503+0000","last_fullsized":"2024-04-23T14:33:49.317503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:05:06.502293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318098+0000","last_change":"2024-04-23T14:33:47.301809+0000","last_active":"2024-04-23T14:33:49.318098+0000","last_peered":"2024-04-23T14:33:49.318098+0000","last_clean":"2024-04-23T14:33:49.318098+0000","last_became_active":"2024-04-23T14:33:47.301501+0000","last_became_peered":"2024-04-23T14:33:47.301501+0000","last_unstale":"2024-04-23T14:33:49.318098+0000","last_undegraded":"2024-04-23T14:33:49.318098+0000","last_fullsized":"2024-04-23T14:33:49.318098+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:21.616656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333675+0000","last_change":"2024-04-23T14:33:48.329804+0000","last_active":"2024-04-23T14:33:48.333675+0000","last_peered":"2024-04-23T14:33:48.333675+0000","last_clean":"2024-04-23T14:33:48.333675+0000","last_became_active":"2024-04-23T14:33:48.329463+0000","last_became_peered":"2024-04-23T14:33:48.329463+0000","last_unstale":"2024-04-23T14:33:48.333675+0000","last_undegraded":"2024-04-23T14:33:48.333675+0000","last_fullsized":"2024-04-23T14:33:48.333675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.347253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.348759+0000","last_change":"2024-04-23T14:33:47.311375+0000","last_active":"2024-04-23T14:33:48.348759+0000","last_peered":"2024-04-23T14:33:48.348759+0000","last_clean":"2024-04-23T14:33:48.348759+0000","last_became_active":"2024-04-23T14:33:47.311067+0000","last_became_peered":"2024-04-23T14:33:47.311067+0000","last_unstale":"2024-04-23T14:33:48.348759+0000","last_undegraded":"2024-04-23T14:33:48.348759+0000","last_fullsized":"2024-04-23T14:33:48.348759+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:36:52.874227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313769+0000","last_change":"2024-04-23T14:33:48.321061+0000","last_active":"2024-04-23T14:33:49.313769+0000","last_peered":"2024-04-23T14:33:49.313769+0000","last_clean":"2024-04-23T14:33:49.313769+0000","last_became_active":"2024-04-23T14:33:48.313410+0000","last_became_peered":"2024-04-23T14:33:48.313410+0000","last_unstale":"2024-04-23T14:33:49.313769+0000","last_undegraded":"2024-04-23T14:33:49.313769+0000","last_fullsized":"2024-04-23T14:33:49.313769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:20:20.248968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.313717+0000","last_change":"2024-04-23T14:33:47.300054+0000","last_active":"2024-04-23T14:33:49.313717+0000","last_peered":"2024-04-23T14:33:49.313717+0000","last_clean":"2024-04-23T14:33:49.313717+0000","last_became_active":"2024-04-23T14:33:47.299022+0000","last_became_peered":"2024-04-23T14:33:47.299022+0000","last_unstale":"2024-04-23T14:33:49.313717+0000","last_undegraded":"2024-04-23T14:33:49.313717+0000","last_fullsized":"2024-04-23T14:33:49.313717+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:50.189865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.345246+0000","last_change":"2024-04-23T14:33:48.343338+0000","last_active":"2024-04-23T14:33:48.345246+0000","last_peered":"2024-04-23T14:33:48.345246+0000","last_clean":"2024-04-23T14:33:48.345246+0000","last_became_active":"2024-04-23T14:33:48.341024+0000","last_became_peered":"2024-04-23T14:33:48.341024+0000","last_unstale":"2024-04-23T14:33:48.345246+0000","last_undegraded":"2024-04-23T14:33:48.345246+0000","last_fullsized":"2024-04-23T14:33:48.345246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:04.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.325711+0000","last_change":"2024-04-23T14:33:47.312500+0000","last_active":"2024-04-23T14:33:49.325711+0000","last_peered":"2024-04-23T14:33:49.325711+0000","last_clean":"2024-04-23T14:33:49.325711+0000","last_became_active":"2024-04-23T14:33:47.312153+0000","last_became_peered":"2024-04-23T14:33:47.312153+0000","last_unstale":"2024-04-23T14:33:49.325711+0000","last_undegraded":"2024-04-23T14:33:49.325711+0000","last_fullsized":"2024-04-23T14:33:49.325711+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:13.546252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317466+0000","last_change":"2024-04-23T14:33:48.327638+0000","last_active":"2024-04-23T14:33:49.317466+0000","last_peered":"2024-04-23T14:33:49.317466+0000","last_clean":"2024-04-23T14:33:49.317466+0000","last_became_active":"2024-04-23T14:33:48.327332+0000","last_became_peered":"2024-04-23T14:33:48.327332+0000","last_unstale":"2024-04-23T14:33:49.317466+0000","last_undegraded":"2024-04-23T14:33:49.317466+0000","last_fullsized":"2024-04-23T14:33:49.317466+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:03.844780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.326203+0000","last_change":"2024-04-23T14:33:47.310484+0000","last_active":"2024-04-23T14:33:48.326203+0000","last_peered":"2024-04-23T14:33:48.326203+0000","last_clean":"2024-04-23T14:33:48.326203+0000","last_became_active":"2024-04-23T14:33:47.308659+0000","last_became_peered":"2024-04-23T14:33:47.308659+0000","last_unstale":"2024-04-23T14:33:48.326203+0000","last_undegraded":"2024-04-23T14:33:48.326203+0000","last_fullsized":"2024-04-23T14:33:48.326203+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:53:03.505844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317588+0000","last_change":"2024-04-23T14:33:48.310942+0000","last_active":"2024-04-23T14:33:49.317588+0000","last_peered":"2024-04-23T14:33:49.317588+0000","last_clean":"2024-04-23T14:33:49.317588+0000","last_became_active":"2024-04-23T14:33:48.309375+0000","last_became_peered":"2024-04-23T14:33:48.309375+0000","last_unstale":"2024-04-23T14:33:49.317588+0000","last_undegraded":"2024-04-23T14:33:49.317588+0000","last_fullsized":"2024-04-23T14:33:49.317588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:07.788298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318196+0000","last_change":"2024-04-23T14:33:47.301010+0000","last_active":"2024-04-23T14:33:49.318196+0000","last_peered":"2024-04-23T14:33:49.318196+0000","last_clean":"2024-04-23T14:33:49.318196+0000","last_became_active":"2024-04-23T14:33:47.300663+0000","last_became_peered":"2024-04-23T14:33:47.300663+0000","last_unstale":"2024-04-23T14:33:49.318196+0000","last_undegraded":"2024-04-23T14:33:49.318196+0000","last_fullsized":"2024-04-23T14:33:49.318196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:19:51.742830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317581+0000","last_change":"2024-04-23T14:33:48.335470+0000","last_active":"2024-04-23T14:33:49.317581+0000","last_peered":"2024-04-23T14:33:49.317581+0000","last_clean":"2024-04-23T14:33:49.317581+0000","last_became_active":"2024-04-23T14:33:48.334744+0000","last_became_peered":"2024-04-23T14:33:48.334744+0000","last_unstale":"2024-04-23T14:33:49.317581+0000","last_undegraded":"2024-04-23T14:33:49.317581+0000","last_fullsized":"2024-04-23T14:33:49.317581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:21.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318296+0000","last_change":"2024-04-23T14:33:47.313414+0000","last_active":"2024-04-23T14:33:49.318296+0000","last_peered":"2024-04-23T14:33:49.318296+0000","last_clean":"2024-04-23T14:33:49.318296+0000","last_became_active":"2024-04-23T14:33:47.312929+0000","last_became_peered":"2024-04-23T14:33:47.312929+0000","last_unstale":"2024-04-23T14:33:49.318296+0000","last_undegraded":"2024-04-23T14:33:49.318296+0000","last_fullsized":"2024-04-23T14:33:49.318296+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:59.935442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315496+0000","last_change":"2024-04-23T14:33:48.313195+0000","last_active":"2024-04-23T14:33:49.315496+0000","last_peered":"2024-04-23T14:33:49.315496+0000","last_clean":"2024-04-23T14:33:49.315496+0000","last_became_active":"2024-04-23T14:33:48.312897+0000","last_became_peered":"2024-04-23T14:33:48.312897+0000","last_unstale":"2024-04-23T14:33:49.315496+0000","last_undegraded":"2024-04-23T14:33:49.315496+0000","last_fullsized":"2024-04-23T14:33:49.315496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.305861+0000","last_change":"2024-04-23T14:33:47.306796+0000","last_active":"2024-04-23T14:33:48.305861+0000","last_peered":"2024-04-23T14:33:48.305861+0000","last_clean":"2024-04-23T14:33:48.305861+0000","last_became_active":"2024-04-23T14:33:47.305772+0000","last_became_peered":"2024-04-23T14:33:47.305772+0000","last_unstale":"2024-04-23T14:33:48.305861+0000","last_undegraded":"2024-04-23T14:33:48.305861+0000","last_fullsized":"2024-04-23T14:33:48.305861+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:51.670869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.312992+0000","last_change":"2024-04-23T14:33:48.325085+0000","last_active":"2024-04-23T14:33:49.312992+0000","last_peered":"2024-04-23T14:33:49.312992+0000","last_clean":"2024-04-23T14:33:49.312992+0000","last_became_active":"2024-04-23T14:33:48.323038+0000","last_became_peered":"2024-04-23T14:33:48.323038+0000","last_unstale":"2024-04-23T14:33:49.312992+0000","last_undegraded":"2024-04-23T14:33:49.312992+0000","last_fullsized":"2024-04-23T14:33:49.312992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:29:57.650311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.322117+0000","last_change":"2024-04-23T14:33:47.314972+0000","last_active":"2024-04-23T14:33:48.322117+0000","last_peered":"2024-04-23T14:33:48.322117+0000","last_clean":"2024-04-23T14:33:48.322117+0000","last_became_active":"2024-04-23T14:33:47.314059+0000","last_became_peered":"2024-04-23T14:33:47.314059+0000","last_unstale":"2024-04-23T14:33:48.322117+0000","last_undegraded":"2024-04-23T14:33:48.322117+0000","last_fullsized":"2024-04-23T14:33:48.322117+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:34.096326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.342200+0000","last_change":"2024-04-23T14:33:48.322006+0000","last_active":"2024-04-23T14:33:48.342200+0000","last_peered":"2024-04-23T14:33:48.342200+0000","last_clean":"2024-04-23T14:33:48.342200+0000","last_became_active":"2024-04-23T14:33:48.319068+0000","last_became_peered":"2024-04-23T14:33:48.319068+0000","last_unstale":"2024-04-23T14:33:48.342200+0000","last_undegraded":"2024-04-23T14:33:48.342200+0000","last_fullsized":"2024-04-23T14:33:48.342200+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:14.560883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.308354+0000","last_change":"2024-04-23T14:33:47.314865+0000","last_active":"2024-04-23T14:33:48.308354+0000","last_peered":"2024-04-23T14:33:48.308354+0000","last_clean":"2024-04-23T14:33:48.308354+0000","last_became_active":"2024-04-23T14:33:47.314174+0000","last_became_peered":"2024-04-23T14:33:47.314174+0000","last_unstale":"2024-04-23T14:33:48.308354+0000","last_undegraded":"2024-04-23T14:33:48.308354+0000","last_fullsized":"2024-04-23T14:33:48.308354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:51.670869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.318016+0000","last_change":"2024-04-23T14:33:48.344466+0000","last_active":"2024-04-23T14:33:49.318016+0000","last_peered":"2024-04-23T14:33:49.318016+0000","last_clean":"2024-04-23T14:33:49.318016+0000","last_became_active":"2024-04-23T14:33:48.344254+0000","last_became_peered":"2024-04-23T14:33:48.344254+0000","last_unstale":"2024-04-23T14:33:49.318016+0000","last_undegraded":"2024-04-23T14:33:49.318016+0000","last_fullsized":"2024-04-23T14:33:49.318016+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.322067+0000","last_change":"2024-04-23T14:33:47.314372+0000","last_active":"2024-04-23T14:33:49.322067+0000","last_peered":"2024-04-23T14:33:49.322067+0000","last_clean":"2024-04-23T14:33:49.322067+0000","last_became_active":"2024-04-23T14:33:47.313851+0000","last_became_peered":"2024-04-23T14:33:47.313851+0000","last_unstale":"2024-04-23T14:33:49.322067+0000","last_undegraded":"2024-04-23T14:33:49.322067+0000","last_fullsized":"2024-04-23T14:33:49.322067+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:53.051738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.344074+0000","last_change":"2024-04-23T14:33:48.340662+0000","last_active":"2024-04-23T14:33:48.344074+0000","last_peered":"2024-04-23T14:33:48.344074+0000","last_clean":"2024-04-23T14:33:48.344074+0000","last_became_active":"2024-04-23T14:33:48.339508+0000","last_became_peered":"2024-04-23T14:33:48.339508+0000","last_unstale":"2024-04-23T14:33:48.344074+0000","last_undegraded":"2024-04-23T14:33:48.344074+0000","last_fullsized":"2024-04-23T14:33:48.344074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:03.254260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.309897+0000","last_change":"2024-04-23T14:33:47.315815+0000","last_active":"2024-04-23T14:33:48.309897+0000","last_peered":"2024-04-23T14:33:48.309897+0000","last_clean":"2024-04-23T14:33:48.309897+0000","last_became_active":"2024-04-23T14:33:47.315259+0000","last_became_peered":"2024-04-23T14:33:47.315259+0000","last_unstale":"2024-04-23T14:33:48.309897+0000","last_undegraded":"2024-04-23T14:33:48.309897+0000","last_fullsized":"2024-04-23T14:33:48.309897+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:19:53.025782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.331027+0000","last_change":"2024-04-23T14:33:48.323498+0000","last_active":"2024-04-23T14:33:48.331027+0000","last_peered":"2024-04-23T14:33:48.331027+0000","last_clean":"2024-04-23T14:33:48.331027+0000","last_became_active":"2024-04-23T14:33:48.322296+0000","last_became_peered":"2024-04-23T14:33:48.322296+0000","last_unstale":"2024-04-23T14:33:48.331027+0000","last_undegraded":"2024-04-23T14:33:48.331027+0000","last_fullsized":"2024-04-23T14:33:48.331027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:54.066369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315486+0000","last_change":"2024-04-23T14:33:47.313626+0000","last_active":"2024-04-23T14:33:49.315486+0000","last_peered":"2024-04-23T14:33:49.315486+0000","last_clean":"2024-04-23T14:33:49.315486+0000","last_became_active":"2024-04-23T14:33:47.313361+0000","last_became_peered":"2024-04-23T14:33:47.313361+0000","last_unstale":"2024-04-23T14:33:49.315486+0000","last_undegraded":"2024-04-23T14:33:49.315486+0000","last_fullsized":"2024-04-23T14:33:49.315486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:58.455845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317853+0000","last_change":"2024-04-23T14:33:48.323375+0000","last_active":"2024-04-23T14:33:49.317853+0000","last_peered":"2024-04-23T14:33:49.317853+0000","last_clean":"2024-04-23T14:33:49.317853+0000","last_became_active":"2024-04-23T14:33:48.322959+0000","last_became_peered":"2024-04-23T14:33:48.322959+0000","last_unstale":"2024-04-23T14:33:49.317853+0000","last_undegraded":"2024-04-23T14:33:49.317853+0000","last_fullsized":"2024-04-23T14:33:49.317853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:04.978844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.306717+0000","last_change":"2024-04-23T14:33:47.315351+0000","last_active":"2024-04-23T14:33:48.306717+0000","last_peered":"2024-04-23T14:33:48.306717+0000","last_clean":"2024-04-23T14:33:48.306717+0000","last_became_active":"2024-04-23T14:33:47.315046+0000","last_became_peered":"2024-04-23T14:33:47.315046+0000","last_unstale":"2024-04-23T14:33:48.306717+0000","last_undegraded":"2024-04-23T14:33:48.306717+0000","last_fullsized":"2024-04-23T14:33:48.306717+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:02.239629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.317801+0000","last_change":"2024-04-23T14:33:48.334896+0000","last_active":"2024-04-23T14:33:49.317801+0000","last_peered":"2024-04-23T14:33:49.317801+0000","last_clean":"2024-04-23T14:33:49.317801+0000","last_became_active":"2024-04-23T14:33:48.334595+0000","last_became_peered":"2024-04-23T14:33:48.334595+0000","last_unstale":"2024-04-23T14:33:49.317801+0000","last_undegraded":"2024-04-23T14:33:49.317801+0000","last_fullsized":"2024-04-23T14:33:49.317801+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:38:23.086095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324635+0000","last_change":"2024-04-23T14:33:47.311791+0000","last_active":"2024-04-23T14:33:49.324635+0000","last_peered":"2024-04-23T14:33:49.324635+0000","last_clean":"2024-04-23T14:33:49.324635+0000","last_became_active":"2024-04-23T14:33:47.311197+0000","last_became_peered":"2024-04-23T14:33:47.311197+0000","last_unstale":"2024-04-23T14:33:49.324635+0000","last_undegraded":"2024-04-23T14:33:49.324635+0000","last_fullsized":"2024-04-23T14:33:49.324635+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:36:52.874227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.314070+0000","last_change":"2024-04-23T14:33:48.310610+0000","last_active":"2024-04-23T14:33:48.314070+0000","last_peered":"2024-04-23T14:33:48.314070+0000","last_clean":"2024-04-23T14:33:48.314070+0000","last_became_active":"2024-04-23T14:33:48.310094+0000","last_became_peered":"2024-04-23T14:33:48.310094+0000","last_unstale":"2024-04-23T14:33:48.314070+0000","last_undegraded":"2024-04-23T14:33:48.314070+0000","last_fullsized":"2024-04-23T14:33:48.314070+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:44:12.241975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.315211+0000","last_change":"2024-04-23T14:33:47.312868+0000","last_active":"2024-04-23T14:33:49.315211+0000","last_peered":"2024-04-23T14:33:49.315211+0000","last_clean":"2024-04-23T14:33:49.315211+0000","last_became_active":"2024-04-23T14:33:47.312480+0000","last_became_peered":"2024-04-23T14:33:47.312480+0000","last_unstale":"2024-04-23T14:33:49.315211+0000","last_undegraded":"2024-04-23T14:33:49.315211+0000","last_fullsized":"2024-04-23T14:33:49.315211+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:02.830149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-23T14:33:49.324395+0000","last_change":"2024-04-23T14:33:48.323232+0000","last_active":"2024-04-23T14:33:49.324395+0000","last_peered":"2024-04-23T14:33:49.324395+0000","last_clean":"2024-04-23T14:33:49.324395+0000","last_became_active":"2024-04-23T14:33:48.322942+0000","last_became_peered":"2024-04-23T14:33:48.322942+0000","last_unstale":"2024-04-23T14:33:49.324395+0000","last_undegraded":"2024-04-23T14:33:49.324395+0000","last_fullsized":"2024-04-23T14:33:49.324395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:36:52.685500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.314456+0000","last_change":"2024-04-23T14:33:47.301910+0000","last_active":"2024-04-23T14:33:48.314456+0000","last_peered":"2024-04-23T14:33:48.314456+0000","last_clean":"2024-04-23T14:33:48.314456+0000","last_became_active":"2024-04-23T14:33:47.301637+0000","last_became_peered":"2024-04-23T14:33:47.301637+0000","last_unstale":"2024-04-23T14:33:48.314456+0000","last_undegraded":"2024-04-23T14:33:48.314456+0000","last_fullsized":"2024-04-23T14:33:48.314456+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:33:30.339190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.344126+0000","last_change":"2024-04-23T14:33:48.339664+0000","last_active":"2024-04-23T14:33:48.344126+0000","last_peered":"2024-04-23T14:33:48.344126+0000","last_clean":"2024-04-23T14:33:48.344126+0000","last_became_active":"2024-04-23T14:33:48.339334+0000","last_became_peered":"2024-04-23T14:33:48.339334+0000","last_unstale":"2024-04-23T14:33:48.344126+0000","last_undegraded":"2024-04-23T14:33:48.344126+0000","last_fullsized":"2024-04-23T14:33:48.344126+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:47.260998+0000","last_clean_scrub_stamp":"2024-04-23T14:33:47.260998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:07.788298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.333964+0000","last_change":"2024-04-23T14:33:47.298769+0000","last_active":"2024-04-23T14:33:48.333964+0000","last_peered":"2024-04-23T14:33:48.333964+0000","last_clean":"2024-04-23T14:33:48.333964+0000","last_became_active":"2024-04-23T14:33:47.298048+0000","last_became_peered":"2024-04-23T14:33:47.298048+0000","last_unstale":"2024-04-23T14:33:48.333964+0000","last_undegraded":"2024-04-23T14:33:48.333964+0000","last_fullsized":"2024-04-23T14:33:48.333964+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:33:46.246367+0000","last_clean_scrub_stamp":"2024-04-23T14:33:46.246367+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:45.663992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-23T14:33:48.308422+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:33:48.308422+0000","last_peered":"2024-04-23T14:33:48.308422+0000","last_clean":"2024-04-23T14:33:48.308422+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:33:48.308422+0000","last_undegraded":"2024-04-23T14:33:48.308422+0000","last_fullsized":"2024-04-23T14:33:48.308422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1101824,"data_stored":793958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":204172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":516096,"data_stored":205112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":204138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":205134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":793902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:33:53.307 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:33:53.307 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.565+0000 7f189ee00640 1 Processor -- start 2024-04-23T14:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.565+0000 7f189ee00640 1 -- start start 2024-04-23T14:33:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.567+0000 7f189ee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f189809aa00 0x7f18980cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.567+0000 7f189ee00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1898070b30 0x7f1898070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.567+0000 7f189ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18980d0450 con 0x7f18980cffc0 2024-04-23T14:33:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.567+0000 7f189ee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f189806c820 con 0x7f189809aa00 2024-04-23T14:33:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.567+0000 7f189ee00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f18980ce230 con 0x7f1898070b30 2024-04-23T14:33:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f189809aa00 0x7f18980cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f189809aa00 0x7f18980cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41440/0 (socket says 172.21.15.59:41440) 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189da00640 1 -- 172.21.15.59:0/2669822821 learned_addr learned my addr 172.21.15.59:0/2669822821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189e400640 1 --2- 172.21.15.59:0/2669822821 >> v2:172.21.15.76:3300/0 conn(0x7f1898070b30 0x7f1898070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189da00640 1 -- 172.21.15.59:0/2669822821 >> v2:172.21.15.76:3300/0 conn(0x7f1898070b30 msgr2=0x7f1898070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189da00640 1 --2- 172.21.15.59:0/2669822821 >> v2:172.21.15.76:3300/0 conn(0x7f1898070b30 0x7f1898070f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189da00640 1 -- 172.21.15.59:0/2669822821 >> v1:172.21.15.59:6789/0 conn(0x7f18980cffc0 legacy=0x7f189809ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189da00640 1 -- 172.21.15.59:0/2669822821 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1898070050 con 0x7f189809aa00 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f1897e00640 1 -- 172.21.15.59:0/2669822821 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2512702361 0 0) 0x7f18980d0450 con 0x7f18980cffc0 2024-04-23T14:33:53.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.568+0000 7f189e400640 1 --2- 172.21.15.59:0/2669822821 >> v2:172.21.15.76:3300/0 conn(0x7f1898070b30 0x7f1898070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.569+0000 7f189da00640 1 --2- 172.21.15.59:0/2669822821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f18980cdbc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1888009870 tx=0x7f188802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc928dcb00eeac4c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.569+0000 7f1897e00640 1 -- 172.21.15.59:0/2669822821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888037070 con 0x7f189809aa00 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f1897e00640 1 -- 172.21.15.59:0/2669822821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1888032440 con 0x7f189809aa00 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f1897e00640 1 -- 172.21.15.59:0/2669822821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888032810 con 0x7f189809aa00 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 -- 172.21.15.59:0/2669822821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 msgr2=0x7f18980cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 --2- 172.21.15.59:0/2669822821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f18980cdbc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1888009870 tx=0x7f188802a210 comp rx=0 tx=0).stop 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 -- 172.21.15.59:0/2669822821 shutdown_connections 2024-04-23T14:33:53.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 --2- 172.21.15.59:0/2669822821 >> v2:172.21.15.76:3300/0 conn(0x7f1898070b30 0x7f1898070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 --2- 172.21.15.59:0/2669822821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f18980cdbc0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 -- 172.21.15.59:0/2669822821 >> 172.21.15.59:0/2669822821 conn(0x7f18980b3ee0 msgr2=0x7f18980b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 -- 172.21.15.59:0/2669822821 shutdown_connections 2024-04-23T14:33:53.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.570+0000 7f189ee00640 1 -- 172.21.15.59:0/2669822821 wait complete. 2024-04-23T14:33:53.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.572+0000 7f189ee00640 1 Processor -- start 2024-04-23T14:33:53.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.572+0000 7f189ee00640 1 -- start start 2024-04-23T14:33:53.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1898070b30 0x7f18980c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f189814e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18980cffc0 0x7f1898151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18980714a0 con 0x7f189809aa00 2024-04-23T14:33:53.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1898070610 con 0x7f1898070b30 2024-04-23T14:33:53.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1898071620 con 0x7f18980cffc0 2024-04-23T14:33:53.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f189814e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f189814e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41456/0 (socket says 172.21.15.59:41456) 2024-04-23T14:33:53.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.573+0000 7f189d000640 1 -- 172.21.15.59:0/4002254028 learned_addr learned my addr 172.21.15.59:0/4002254028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:53.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189d000640 1 -- 172.21.15.59:0/4002254028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18980cffc0 msgr2=0x7f1898151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189d000640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18980cffc0 0x7f1898151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189d000640 1 -- 172.21.15.59:0/4002254028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1898070b30 msgr2=0x7f18980c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189d000640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1898070b30 0x7f18980c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189d000640 1 -- 172.21.15.59:0/4002254028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18981507e0 con 0x7f189809aa00 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189d000640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f189814e720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f188400c780 tx=0x7f188400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f1895600640 1 -- 172.21.15.59:0/4002254028 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884003da0 con 0x7f189809aa00 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f1895600640 1 -- 172.21.15.59:0/4002254028 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1884005070 con 0x7f189809aa00 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1898150ac0 con 0x7f189809aa00 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f1895600640 1 -- 172.21.15.59:0/4002254028 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884005420 con 0x7f189809aa00 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.574+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1898151080 con 0x7f189809aa00 2024-04-23T14:33:53.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.575+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189804fd90 con 0x7f189809aa00 2024-04-23T14:33:53.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.577+0000 7f1895600640 1 -- 172.21.15.59:0/4002254028 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18840055c0 con 0x7f189809aa00 2024-04-23T14:33:53.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.577+0000 7f1895600640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f187806c920 0x7f187806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:53.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.578+0000 7f1895600640 1 -- 172.21.15.59:0/4002254028 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f188408ba70 con 0x7f189809aa00 2024-04-23T14:33:53.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.578+0000 7f189da00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f187806c920 0x7f187806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:53.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.578+0000 7f189da00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f187806c920 0x7f187806ede0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1888034000 tx=0x7f188802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:53.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.582+0000 7f1895600640 1 -- 172.21.15.59:0/4002254028 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1884059ea0 con 0x7f189809aa00 2024-04-23T14:33:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.856+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f18980c5740 con 0x7f189809aa00 2024-04-23T14:33:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.861+0000 7f1895600640 1 -- 172.21.15.59:0/4002254028 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f18840597a0 con 0x7f189809aa00 2024-04-23T14:33:53.862 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:33:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:33:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-23T14:33:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-23T14:33:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:33:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f187806c920 msgr2=0x7f187806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f187806c920 0x7f187806ede0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1888034000 tx=0x7f188802c000 comp rx=0 tx=0).stop 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 msgr2=0x7f189814e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f189814e720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f188400c780 tx=0x7f188400cc40 comp rx=0 tx=0).stop 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 shutdown_connections 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f187806c920 0x7f187806ede0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18980cffc0 0x7f1898151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f189809aa00 0x7f189814e720 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 --2- 172.21.15.59:0/4002254028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1898070b30 0x7f18980c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.863+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 >> 172.21.15.59:0/4002254028 conn(0x7f18980b3ee0 msgr2=0x7f189806c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.864+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 shutdown_connections 2024-04-23T14:33:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:53.864+0000 7f189ee00640 1 -- 172.21.15.59:0/4002254028 wait complete. 2024-04-23T14:33:53.883 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:54.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.091+0000 7fb4a3200640 1 Processor -- start 2024-04-23T14:33:54.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.092+0000 7fb4a3200640 1 -- start start 2024-04-23T14:33:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.093+0000 7fb4a3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 0x7fb4a406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.093+0000 7fb4a3200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a406ed40 con 0x7fb4a4162df0 2024-04-23T14:33:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.093+0000 7fb4a3200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a40b3220 con 0x7fb4a40cfb90 2024-04-23T14:33:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.093+0000 7fb4a3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a406eaf0 con 0x7fb4a40b4130 2024-04-23T14:33:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.094+0000 7fb4a1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb4a4162df0 0x7fb4a40b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37892/0 (socket says 172.21.15.59:37892) 2024-04-23T14:33:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.094+0000 7fb4a1e00640 1 -- 172.21.15.59:0/2920385021 learned_addr learned my addr 172.21.15.59:0/2920385021 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.094+0000 7fb499400640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 0x7fb4a406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.094+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3277442033 0 0) 0x7fb4a406ed40 con 0x7fb4a4162df0 2024-04-23T14:33:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.094+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb488003520 con 0x7fb4a4162df0 2024-04-23T14:33:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4061194894 0 0) 0x7fb488003520 con 0x7fb4a4162df0 2024-04-23T14:33:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 msgr2=0x7fb4a406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 0x7fb4a406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 >> v1:172.21.15.76:6789/0 conn(0x7fb4a40cfb90 legacy=0x7fb4a40c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a40af4c0 con 0x7fb4a4162df0 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fb4940019f0 con 0x7fb4a4162df0 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 >> v1:172.21.15.59:6789/0 conn(0x7fb4a4162df0 legacy=0x7fb4a40b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4880039e0 0x7fb488003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 0x7fb488004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb488004d40 0x7fb4880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a406eaf0 con 0x7fb4880039e0 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb48800baa0 con 0x7fb488004d40 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb48800bca0 con 0x7fb4a40b4130 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a1e00640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4880039e0 0x7fb488003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb499400640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 0x7fb488004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.095+0000 7fb4a2800640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb488004d40 0x7fb4880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.096+0000 7fb4a1e00640 1 -- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 msgr2=0x7fb488004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.096+0000 7fb4a1e00640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4a40b4130 0x7fb488004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.096+0000 7fb4a1e00640 1 -- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb488004d40 msgr2=0x7fb4880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.096+0000 7fb4a1e00640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb488004d40 0x7fb4880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.096+0000 7fb4a1e00640 1 -- 172.21.15.59:0/2920385021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb488003520 con 0x7fb4880039e0 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.097+0000 7fb4a1e00640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4880039e0 0x7fb488003e00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb49400a780 tx=0x7fb488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9391b6e6b8b976b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.097+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49400a3d0 con 0x7fb4880039e0 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.097+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49402f070 con 0x7fb4880039e0 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.097+0000 7fb4a1400640 1 -- 172.21.15.59:0/2920385021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49402bc20 con 0x7fb4880039e0 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.097+0000 7fb4a3200640 1 -- 172.21.15.59:0/2920385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4880039e0 msgr2=0x7fb488003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.097+0000 7fb4a3200640 1 --2- 172.21.15.59:0/2920385021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4880039e0 0x7fb488003e00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb49400a780 tx=0x7fb488005830 comp rx=0 tx=0).stop 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.098+0000 7fb4a2800640 1 -- 172.21.15.59:0/2920385021 reap_dead start 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.098+0000 7fb4a3200640 1 -- 172.21.15.59:0/2920385021 shutdown_connections 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.098+0000 7fb4a3200640 1 -- 172.21.15.59:0/2920385021 >> 172.21.15.59:0/2920385021 conn(0x7fb4a40b21a0 msgr2=0x7fb4a40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.098+0000 7fb4a3200640 1 -- 172.21.15.59:0/2920385021 shutdown_connections 2024-04-23T14:33:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.098+0000 7fb4a3200640 1 -- 172.21.15.59:0/2920385021 wait complete. 2024-04-23T14:33:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.099+0000 7fb4a3200640 1 Processor -- start 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.100+0000 7fb4a3200640 1 -- start start 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.100+0000 7fb4a3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4880039e0 0x7fb4a40af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.100+0000 7fb4a3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4a40c9930 0x7fb4a40c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb4a3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4a40c7280 0x7fb4a40c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb4a3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a406f680 con 0x7fb4a40c9930 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb4a3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a40b3590 con 0x7fb4a40c7280 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb4a3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a406f800 con 0x7fb4880039e0 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4a40c9930 0x7fb4a40c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4a40c9930 0x7fb4a40c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41468/0 (socket says 172.21.15.59:41468) 2024-04-23T14:33:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 -- 172.21.15.59:0/413652477 learned_addr learned my addr 172.21.15.59:0/413652477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb4a1e00640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4880039e0 0x7fb4a40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb4a2800640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4a40c7280 0x7fb4a40c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 -- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4880039e0 msgr2=0x7fb4a40af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4880039e0 0x7fb4a40af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 -- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4a40c7280 msgr2=0x7fb4a40c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4a40c7280 0x7fb4a40c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb499400640 1 -- 172.21.15.59:0/413652477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a40b0000 con 0x7fb4a40c9930 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.101+0000 7fb4a2800640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4a40c7280 0x7fb4a40c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.102+0000 7fb499400640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4a40c9930 0x7fb4a40c9db0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb48c011600 tx=0x7fb48c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.102+0000 7fb49aa00640 1 -- 172.21.15.59:0/413652477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c015870 con 0x7fb4a40c9930 2024-04-23T14:33:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.102+0000 7fb49aa00640 1 -- 172.21.15.59:0/413652477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb48c015a10 con 0x7fb4a40c9930 2024-04-23T14:33:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.102+0000 7fb49aa00640 1 -- 172.21.15.59:0/413652477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c015dc0 con 0x7fb4a40c9930 2024-04-23T14:33:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.102+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a40b02e0 con 0x7fb4a40c9930 2024-04-23T14:33:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.102+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a40b08a0 con 0x7fb4a40c9930 2024-04-23T14:33:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.103+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a404fd90 con 0x7fb4a40c9930 2024-04-23T14:33:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.104+0000 7fb49aa00640 1 -- 172.21.15.59:0/413652477 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb48c00f450 con 0x7fb4a40c9930 2024-04-23T14:33:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.105+0000 7fb49aa00640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb48006c920 0x7fb48006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.105+0000 7fb49aa00640 1 -- 172.21.15.59:0/413652477 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fb48c08c420 con 0x7fb4a40c9930 2024-04-23T14:33:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.106+0000 7fb4a1e00640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb48006c920 0x7fb48006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.106+0000 7fb4a1e00640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb48006c920 0x7fb48006ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb4880057e0 tx=0x7fb494009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:54.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.109+0000 7fb49aa00640 1 -- 172.21.15.59:0/413652477 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb48c019030 con 0x7fb4a40c9930 2024-04-23T14:33:54.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.349+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4a40c6dc0 con 0x7fb4a40c9930 2024-04-23T14:33:54.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.350+0000 7fb49aa00640 1 -- 172.21.15.59:0/413652477 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7fb48c05a160 con 0x7fb4a40c9930 2024-04-23T14:33:54.351 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:54.351 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:33:54.352 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:33:54.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb48006c920 msgr2=0x7fb48006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb48006c920 0x7fb48006ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb4880057e0 tx=0x7fb494009030 comp rx=0 tx=0).stop 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4a40c9930 msgr2=0x7fb4a40c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4a40c9930 0x7fb4a40c9db0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb48c011600 tx=0x7fb48c011ac0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 shutdown_connections 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb48006c920 0x7fb48006ede0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4a40c7280 0x7fb4a40c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4a40c9930 0x7fb4a40c9db0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 --2- 172.21.15.59:0/413652477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4880039e0 0x7fb4a40af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 >> 172.21.15.59:0/413652477 conn(0x7fb4a40b21a0 msgr2=0x7fb4a40cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.354+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 shutdown_connections 2024-04-23T14:33:54.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.355+0000 7fb4a3200640 1 -- 172.21.15.59:0/413652477 wait complete. 2024-04-23T14:33:54.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:33:47.763614+0000', 'modified': '2024-04-23T14:33:50.838863+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5712': {'gid': 5712, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/2804599006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2804599006}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2804599006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5712, 'qdb_cluster': [5712]} 2024-04-23T14:33:54.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:33:54.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-23T14:33:55.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.998+0000 7f0b4b000640 1 Processor -- start 2024-04-23T14:33:55.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:54.999+0000 7f0b4b000640 1 -- start start 2024-04-23T14:33:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.000+0000 7f0b4b000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.000+0000 7f0b4b000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b4415f060 0x7f0b440d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.000+0000 7f0b4b000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b440d0ee0 con 0x7f0b4415c250 2024-04-23T14:33:55.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.000+0000 7f0b4b000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0b44141c20 con 0x7f0b4415f060 2024-04-23T14:33:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.000+0000 7f0b4b000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4415bdc0 con 0x7f0b440d05c0 2024-04-23T14:33:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.001+0000 7f0b49c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.001+0000 7f0b49200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b4415f060 0x7f0b440d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.001+0000 7f0b49200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b4415f060 0x7f0b440d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:46052/0 (socket says 172.21.15.59:46052) 2024-04-23T14:33:55.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.001+0000 7f0b49200640 1 -- 172.21.15.59:0/2002571460 learned_addr learned my addr 172.21.15.59:0/2002571460 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.002+0000 7f0b49200640 1 -- 172.21.15.59:0/2002571460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 msgr2=0x7f0b441416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.002+0000 7f0b49200640 1 --2- 172.21.15.59:0/2002571460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.002+0000 7f0b49200640 1 -- 172.21.15.59:0/2002571460 >> v1:172.21.15.76:6789/0 conn(0x7f0b4415c250 legacy=0x7f0b441555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.002+0000 7f0b49200640 1 -- 172.21.15.59:0/2002571460 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4415ba70 con 0x7f0b4415f060 2024-04-23T14:33:55.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.003+0000 7f0b49200640 1 --2- 172.21.15.59:0/2002571460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415f060 0x7f0b440d09a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009870 tx=0x7f0b3802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=485d89fac8a662bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b43e00640 1 -- 172.21.15.59:0/2002571460 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38037070 con 0x7f0b4415f060 2024-04-23T14:33:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b43e00640 1 -- 172.21.15.59:0/2002571460 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b38032440 con 0x7f0b4415f060 2024-04-23T14:33:55.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b43e00640 1 -- 172.21.15.59:0/2002571460 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38032810 con 0x7f0b4415f060 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b4b000640 1 -- 172.21.15.59:0/2002571460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415f060 msgr2=0x7f0b440d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b4b000640 1 --2- 172.21.15.59:0/2002571460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415f060 0x7f0b440d09a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009870 tx=0x7f0b3802a250 comp rx=0 tx=0).stop 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b4b000640 1 -- 172.21.15.59:0/2002571460 shutdown_connections 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b4b000640 1 --2- 172.21.15.59:0/2002571460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415f060 0x7f0b440d09a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b4b000640 1 --2- 172.21.15.59:0/2002571460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.004+0000 7f0b4b000640 1 -- 172.21.15.59:0/2002571460 >> 172.21.15.59:0/2002571460 conn(0x7f0b4406bdf0 msgr2=0x7f0b4406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.005+0000 7f0b4b000640 1 -- 172.21.15.59:0/2002571460 shutdown_connections 2024-04-23T14:33:55.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.005+0000 7f0b4b000640 1 -- 172.21.15.59:0/2002571460 wait complete. 2024-04-23T14:33:55.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.006+0000 7f0b4b000640 1 Processor -- start 2024-04-23T14:33:55.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.006+0000 7f0b4b000640 1 -- start start 2024-04-23T14:33:55.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.007+0000 7f0b4b000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.007+0000 7f0b4b000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415c250 0x7f0b4409c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.007+0000 7f0b4b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b4415f060 0x7f0b4409c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.007+0000 7f0b4b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4415fdc0 con 0x7f0b4415f060 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.007+0000 7f0b4b000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4415fae0 con 0x7f0b4415c250 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.007+0000 7f0b4b000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4415f4c0 con 0x7f0b440d05c0 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b4415f060 0x7f0b4409c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b4415f060 0x7f0b4409c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41484/0 (socket says 172.21.15.59:41484) 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 -- 172.21.15.59:0/1970956768 learned_addr learned my addr 172.21.15.59:0/1970956768 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b49c00640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b49200640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415c250 0x7f0b4409c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 -- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 msgr2=0x7f0b441f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 -- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415c250 msgr2=0x7f0b4409c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415c250 0x7f0b4409c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.008+0000 7f0b4a600640 1 -- 172.21.15.59:0/1970956768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b441520f0 con 0x7f0b4415f060 2024-04-23T14:33:55.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.009+0000 7f0b4a600640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b4415f060 0x7f0b4409c870 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c7a0 tx=0x7f0b3c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.009+0000 7f0b41600640 1 -- 172.21.15.59:0/1970956768 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c003b70 con 0x7f0b4415f060 2024-04-23T14:33:55.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.009+0000 7f0b41600640 1 -- 172.21.15.59:0/1970956768 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c003d30 con 0x7f0b4415f060 2024-04-23T14:33:55.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.009+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b441523d0 con 0x7f0b4415f060 2024-04-23T14:33:55.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.009+0000 7f0b41600640 1 -- 172.21.15.59:0/1970956768 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00aa90 con 0x7f0b4415f060 2024-04-23T14:33:55.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.009+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b44152990 con 0x7f0b4415f060 2024-04-23T14:33:55.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.012+0000 7f0b41600640 1 -- 172.21.15.59:0/1970956768 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b3c00ac30 con 0x7f0b4415f060 2024-04-23T14:33:55.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.013+0000 7f0b41600640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b2806c920 0x7f0b2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.013+0000 7f0b41600640 1 -- 172.21.15.59:0/1970956768 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f0b3c08ba30 con 0x7f0b4415f060 2024-04-23T14:33:55.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.013+0000 7f0b49c00640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b2806c920 0x7f0b2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.013+0000 7f0b4f3e2740 1 Processor -- start 2024-04-23T14:33:55.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.013+0000 7f0b4f3e2740 1 -- start start 2024-04-23T14:33:55.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.014+0000 7f0b49c00640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b2806c920 0x7f0b2806ede0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0b440d0b80 tx=0x7f0b34009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.014+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d4b980 0x55e744cfe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.014+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d2de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.014+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2e360 0x55e744d03170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.014+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e744c68b10 con 0x55e744d4b980 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.014+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e744c0f2b0 con 0x55e744d2eaa0 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.014+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e744c64ad0 con 0x55e744d2e360 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d4b980 0x55e744cfe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d4b980 0x55e744cfe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41486/0 (socket says 172.21.15.59:41486) 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 -- 172.21.15.59:0/313874064 learned_addr learned my addr 172.21.15.59:0/313874064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49200640 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d2de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b4a600640 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2e360 0x55e744d03170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 -- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2e360 msgr2=0x55e744d03170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2e360 0x55e744d03170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 -- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 msgr2=0x55e744d2de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d2de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 -- 172.21.15.59:0/313874064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74497b2d0 con 0x55e744d4b980 2024-04-23T14:33:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.015+0000 7f0b49c00640 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d4b980 0x55e744cfe310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0b34011ee0 tx=0x7f0b34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86643eb02539cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.016+0000 7f0b32000640 1 -- 172.21.15.59:0/313874064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3401b070 con 0x55e744d4b980 2024-04-23T14:33:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.016+0000 7f0b32000640 1 -- 172.21.15.59:0/313874064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34027070 con 0x55e744d4b980 2024-04-23T14:33:55.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.016+0000 7f0b32000640 1 -- 172.21.15.59:0/313874064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34003a20 con 0x55e744d4b980 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.016+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/313874064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d4b980 msgr2=0x55e744cfe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.016+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d4b980 0x55e744cfe310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0b34011ee0 tx=0x7f0b34014000 comp rx=0 tx=0).stop 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/313874064 shutdown_connections 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2e360 0x55e744d03170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d2de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/313874064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d4b980 0x55e744cfe310 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/313874064 >> 172.21.15.59:0/313874064 conn(0x55e744d00580 msgr2=0x55e744d48e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/313874064 shutdown_connections 2024-04-23T14:33:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/313874064 wait complete. 2024-04-23T14:33:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.017+0000 7f0b4f3e2740 1 -- start start 2024-04-23T14:33:55.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.018+0000 7f0b4f3e2740 1 Processor -- start 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d1eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d4b980 0x55e744d315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d48e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e744d02730 con 0x55e744d404a0 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e744c22590 con 0x55e744d2eaa0 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e744d01b20 con 0x55e744d4b980 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4f3e2740 10 client.0 _ll_register_callbacks cb 0x55e744c053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d48e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d48e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41494/0 (socket says 172.21.15.59:41494) 2024-04-23T14:33:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4a600640 1 -- 172.21.15.59:0/2529102800 learned_addr learned my addr 172.21.15.59:0/2529102800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b49c00640 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d1eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b49200640 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d4b980 0x55e744d315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4a600640 1 -- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d4b980 msgr2=0x55e744d315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4a600640 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d4b980 0x55e744d315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4a600640 1 -- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 msgr2=0x55e744d1eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.019+0000 7f0b4a600640 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d1eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b4a600640 1 -- 172.21.15.59:0/2529102800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e744a456b0 con 0x55e744d404a0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b4a600640 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d48e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c061d70 tx=0x7f0b3c059020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ae06a97fb95b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b26a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b26a00640 1 -- 172.21.15.59:0/2529102800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c070070 con 0x55e744d404a0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b26a00640 1 -- 172.21.15.59:0/2529102800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f0b3c058b10 con 0x55e744d404a0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b34027070 con 0x55e744d404a0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b26a00640 1 -- 172.21.15.59:0/2529102800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b3c05da70 con 0x55e744d404a0 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b4f3e2740 10 client.5853 fetch_fsmap learned FSMap version 25 2024-04-23T14:33:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.020+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55e744d41700 con 0x55e744d404a0 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b26a00640 1 -- 172.21.15.59:0/2529102800 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34027070 con 0x55e744d404a0 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b26a00640 1 -- 172.21.15.59:0/2529102800 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f0b3c055410 con 0x55e744d404a0 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b4f3e2740 10 client.5853 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b4f3e2740 10 client.5853 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b4f3e2740 20 client.5853 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1c390 0x55e744d306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b4f3e2740 4 client.5853 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e744c76530 con 0x55e744d1c390 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.021+0000 7f0b49c00640 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1c390 0x55e744d306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.022+0000 7f0b49c00640 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1c390 0x55e744d306f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236b822640adb89d server_cookie=cb9e2206c7228aba in_seq=0 out_seq=0 2024-04-23T14:33:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.022+0000 7f0b26a00640 10 client.5853 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b26a00640 1 -- 172.21.15.59:0/2529102800 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e744c76530 con 0x55e744d1c390 2024-04-23T14:33:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b26a00640 10 client.5853 handle_command_reply: tid=0 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 client.5853 shutdown 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 msgr2=0x55e744d48e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d48e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c061d70 tx=0x7f0b3c059020 comp rx=0 tx=0).stop 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 shutdown_connections 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1c390 0x55e744d306f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d4b980 0x55e744d315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d48e20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/2529102800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d2eaa0 0x55e744d1eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 >> 172.21.15.59:0/2529102800 conn(0x55e744d2e360 msgr2=0x55e744cfd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b26a00640 0 client.5853 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 shutdown_connections 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.023+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/2529102800 wait complete. 2024-04-23T14:33:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.024+0000 7f0b4f3e2740 20 client.5853 trim_cache size 0 max 16384 2024-04-23T14:33:55.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.043+0000 7f0b4f3e2740 1 Processor -- start 2024-04-23T14:33:55.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.043+0000 7f0b4f3e2740 1 -- start start 2024-04-23T14:33:55.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.044+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2eaa0 0x55e744d62a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.044+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.044+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744cfe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.044+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441ffea0 con 0x55e744d404a0 2024-04-23T14:33:55.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.044+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44151520 con 0x55e744d4b980 2024-04-23T14:33:55.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.044+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0b44151b70 con 0x55e744d2eaa0 2024-04-23T14:33:55.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d29090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41508/0 (socket says 172.21.15.59:41508) 2024-04-23T14:33:55.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 -- 172.21.15.59:0/920498697 learned_addr learned my addr 172.21.15.59:0/920498697 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 -- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2eaa0 msgr2=0x55e744d62a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b4a600640 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744cfe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49c00640 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2eaa0 0x55e744d62a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2eaa0 0x55e744d62a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 -- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 msgr2=0x55e744cfe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744cfe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49200640 1 -- 172.21.15.59:0/920498697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e744d42880 con 0x55e744d404a0 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b49c00640 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2eaa0 0x55e744d62a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.045+0000 7f0b4a600640 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744cfe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:55.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.046+0000 7f0b49200640 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d29090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b3802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a79eb990ae2045 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.046+0000 7f0b31600640 1 -- 172.21.15.59:0/920498697 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38042070 con 0x55e744d404a0 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.046+0000 7f0b31600640 1 -- 172.21.15.59:0/920498697 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b38032780 con 0x55e744d404a0 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.046+0000 7f0b31600640 1 -- 172.21.15.59:0/920498697 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38037040 con 0x55e744d404a0 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.046+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/920498697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 msgr2=0x55e744d29090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.046+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d29090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b3802c000 comp rx=0 tx=0).stop 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.046+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/920498697 shutdown_connections 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.047+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744cfe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.047+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d404a0 0x55e744d29090 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.047+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/920498697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d2eaa0 0x55e744d62a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.047+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/920498697 >> 172.21.15.59:0/920498697 conn(0x55e744d2e360 msgr2=0x55e744d2ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.047+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/920498697 shutdown_connections 2024-04-23T14:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.047+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/920498697 wait complete. 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.047+0000 7f0b4f3e2740 1 -- start start 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.048+0000 7f0b4f3e2740 1 Processor -- start 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d2eaa0 0x55e744cfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d404a0 0x55e744d28d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4f3e2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744da9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e744d1c210 con 0x55e744d2eaa0 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e744d1cd80 con 0x55e744d4b980 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4f3e2740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e744d1c800 con 0x55e744d404a0 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4f3e2740 10 client.0 _ll_register_callbacks cb 0x7f0b3c059490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d2eaa0 0x55e744cfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d2eaa0 0x55e744cfdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41512/0 (socket says 172.21.15.59:41512) 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 -- 172.21.15.59:0/3755447808 learned_addr learned my addr 172.21.15.59:0/3755447808 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b4a600640 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744da9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49200640 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d404a0 0x55e744d28d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 -- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d404a0 msgr2=0x55e744d28d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d404a0 0x55e744d28d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 -- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 msgr2=0x55e744da9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744da9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.049+0000 7f0b49c00640 1 -- 172.21.15.59:0/3755447808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e744d292d0 con 0x55e744d2eaa0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b49c00640 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d2eaa0 0x55e744cfdb80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400a010 tx=0x7f0b34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607d2ae47122bd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b26a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b26a00640 1 -- 172.21.15.59:0/3755447808 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3401b040 con 0x55e744d2eaa0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b26a00640 1 -- 172.21.15.59:0/3755447808 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f0b34003920 con 0x55e744d2eaa0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b38032780 con 0x55e744d2eaa0 2024-04-23T14:33:55.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b26a00640 1 -- 172.21.15.59:0/3755447808 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b34044050 con 0x55e744d2eaa0 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b4f3e2740 10 client.5859 fetch_fsmap learned FSMap version 25 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55e744d4a760 con 0x55e744d2eaa0 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.050+0000 7f0b26a00640 1 -- 172.21.15.59:0/3755447808 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b38032780 con 0x55e744d2eaa0 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b26a00640 1 -- 172.21.15.59:0/3755447808 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f0b34047020 con 0x55e744d2eaa0 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b4f3e2740 10 client.5859 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b4f3e2740 10 client.5859 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b4f3e2740 20 client.5859 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1de10 0x55e744d4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b4f3e2740 4 client.5859 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "status"}) -- 0x55e744cff9d0 con 0x55e744d1de10 2024-04-23T14:33:55.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b49200640 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1de10 0x55e744d4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b49200640 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1de10 0x55e744d4c240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b6f16dae73111 server_cookie=17fdde52d8271741 in_seq=0 out_seq=0 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.051+0000 7f0b26a00640 10 client.5859 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b26a00640 1 -- 172.21.15.59:0/3755447808 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e744cff9d0 con 0x55e744d1de10 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b26a00640 10 client.5859 handle_command_reply: tid=0 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 client.5859 shutdown 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d2eaa0 msgr2=0x55e744cfdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d2eaa0 0x55e744cfdb80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400a010 tx=0x7f0b34004730 comp rx=0 tx=0).stop 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 shutdown_connections 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x55e744d1de10 0x55e744d4c240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e744d404a0 0x55e744d28d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e744d4b980 0x55e744da9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b26a00640 0 client.5859 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 --2- 172.21.15.59:0/3755447808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e744d2eaa0 0x55e744cfdb80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 >> 172.21.15.59:0/3755447808 conn(0x55e744d2e360 msgr2=0x55e744d3fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 shutdown_connections 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 1 -- 172.21.15.59:0/3755447808 wait complete. 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.052+0000 7f0b4f3e2740 20 client.5859 trim_cache size 0 max 16384 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5712, 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:33:55.057 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7514651600000004, 2024-04-23T14:33:55.058 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 25, 2024-04-23T14:33:55.058 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 44, 2024-04-23T14:33:55.058 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 44, 2024-04-23T14:33:55.058 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.178820875 2024-04-23T14:33:55.058 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b2806c920 msgr2=0x7f0b2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b2806c920 0x7f0b2806ede0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0b440d0b80 tx=0x7f0b34009190 comp rx=0 tx=0).stop 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b4415f060 msgr2=0x7f0b4409c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b4415f060 0x7f0b4409c870 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00c7a0 tx=0x7f0b3c00cc60 comp rx=0 tx=0).stop 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 shutdown_connections 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0b2806c920 0x7f0b2806ede0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b4415f060 0x7f0b4409c870 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b4415c250 0x7f0b4409c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 --2- 172.21.15.59:0/1970956768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b440d05c0 0x7f0b441f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 >> 172.21.15.59:0/1970956768 conn(0x7f0b4406bdf0 msgr2=0x7f0b4415a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 shutdown_connections 2024-04-23T14:33:55.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.053+0000 7f0b4b000640 1 -- 172.21.15.59:0/1970956768 wait complete. 2024-04-23T14:33:55.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:33:55.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.280+0000 7f70e1a00640 1 Processor -- start 2024-04-23T14:33:55.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.280+0000 7f70e1a00640 1 -- start start 2024-04-23T14:33:55.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70e1a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70dc06f7c0 0x7f70dc0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70e1a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc15b840 0x7f70dc06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70e1a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70dc06df00 0x7f70dc06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70e1a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f70dc0700e0 con 0x7f70dc06f7c0 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70e1a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f70dc06ec60 con 0x7f70dc06df00 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70e1a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0b46e0 con 0x7f70dc15b840 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70daa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70dc06f7c0 0x7f70dc0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70daa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70dc06f7c0 0x7f70dc0b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41516/0 (socket says 172.21.15.59:41516) 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.282+0000 7f70daa00640 1 -- 172.21.15.59:0/3970775689 learned_addr learned my addr 172.21.15.59:0/3970775689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.283+0000 7f70da000640 1 --2- 172.21.15.59:0/3970775689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc15b840 0x7f70dc06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.283+0000 7f70db400640 1 --2- 172.21.15.59:0/3970775689 >> v2:172.21.15.76:3300/0 conn(0x7f70dc06df00 0x7f70dc06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.283+0000 7f70daa00640 1 -- 172.21.15.59:0/3970775689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc15b840 msgr2=0x7f70dc06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.283+0000 7f70daa00640 1 --2- 172.21.15.59:0/3970775689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc15b840 0x7f70dc06fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.283+0000 7f70daa00640 1 -- 172.21.15.59:0/3970775689 >> v2:172.21.15.76:3300/0 conn(0x7f70dc06df00 msgr2=0x7f70dc06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:55.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.283+0000 7f70daa00640 1 --2- 172.21.15.59:0/3970775689 >> v2:172.21.15.76:3300/0 conn(0x7f70dc06df00 0x7f70dc06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.283+0000 7f70daa00640 1 -- 172.21.15.59:0/3970775689 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0b49c0 con 0x7f70dc06f7c0 2024-04-23T14:33:55.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.284+0000 7f70daa00640 1 --2- 172.21.15.59:0/3970775689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc06f7c0 0x7f70dc0b54a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0b3180 tx=0x7f70cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c621fb26295a66b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70d9600640 1 -- 172.21.15.59:0/3970775689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc037070 con 0x7f70dc06f7c0 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70d9600640 1 -- 172.21.15.59:0/3970775689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70cc032440 con 0x7f70dc06f7c0 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70d9600640 1 -- 172.21.15.59:0/3970775689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc032810 con 0x7f70dc06f7c0 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70e1a00640 1 -- 172.21.15.59:0/3970775689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc06f7c0 msgr2=0x7f70dc0b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70e1a00640 1 --2- 172.21.15.59:0/3970775689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc06f7c0 0x7f70dc0b54a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0b3180 tx=0x7f70cc02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70e1a00640 1 -- 172.21.15.59:0/3970775689 shutdown_connections 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70e1a00640 1 --2- 172.21.15.59:0/3970775689 >> v2:172.21.15.76:3300/0 conn(0x7f70dc06df00 0x7f70dc06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70e1a00640 1 --2- 172.21.15.59:0/3970775689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc15b840 0x7f70dc06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70e1a00640 1 --2- 172.21.15.59:0/3970775689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc06f7c0 0x7f70dc0b54a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.285+0000 7f70e1a00640 1 -- 172.21.15.59:0/3970775689 >> 172.21.15.59:0/3970775689 conn(0x7f70dc067ea0 msgr2=0x7f70dc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.286+0000 7f70e1a00640 1 -- 172.21.15.59:0/3970775689 shutdown_connections 2024-04-23T14:33:55.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.286+0000 7f70e1a00640 1 -- 172.21.15.59:0/3970775689 wait complete. 2024-04-23T14:33:55.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.287+0000 7f70e1a00640 1 Processor -- start 2024-04-23T14:33:55.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.288+0000 7f70e1a00640 1 -- start start 2024-04-23T14:33:55.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.288+0000 7f70e1a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70dc06df00 0x7f70dc1424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.288+0000 7f70e1a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc06f7c0 0x7f70dc142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.288+0000 7f70e1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc1430b0 0x7f70dc1f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.288+0000 7f70e1a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc06e8d0 con 0x7f70dc1430b0 2024-04-23T14:33:55.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.288+0000 7f70e1a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0b4f80 con 0x7f70dc06df00 2024-04-23T14:33:55.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.288+0000 7f70e1a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0ce1c0 con 0x7f70dc06f7c0 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc1430b0 0x7f70dc1f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc1430b0 0x7f70dc1f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41518/0 (socket says 172.21.15.59:41518) 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 -- 172.21.15.59:0/173653836 learned_addr learned my addr 172.21.15.59:0/173653836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70da000640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc06f7c0 0x7f70dc142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70daa00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70dc06df00 0x7f70dc1424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 -- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc06f7c0 msgr2=0x7f70dc142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc06f7c0 0x7f70dc142a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 -- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70dc06df00 msgr2=0x7f70dc1424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70dc06df00 0x7f70dc1424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.289+0000 7f70db400640 1 -- 172.21.15.59:0/173653836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc15ce20 con 0x7f70dc1430b0 2024-04-23T14:33:55.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.290+0000 7f70db400640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc1430b0 0x7f70dc1f7fa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f70d4007e40 tx=0x7f70d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.290+0000 7f70d2a00640 1 -- 172.21.15.59:0/173653836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4014620 con 0x7f70dc1430b0 2024-04-23T14:33:55.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.290+0000 7f70d2a00640 1 -- 172.21.15.59:0/173653836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d4019070 con 0x7f70dc1430b0 2024-04-23T14:33:55.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.290+0000 7f70d2a00640 1 -- 172.21.15.59:0/173653836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4014a20 con 0x7f70dc1430b0 2024-04-23T14:33:55.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.290+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc15d100 con 0x7f70dc1430b0 2024-04-23T14:33:55.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.290+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc15d6c0 con 0x7f70dc1430b0 2024-04-23T14:33:55.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.291+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc068a80 con 0x7f70dc1430b0 2024-04-23T14:33:55.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.292+0000 7f70d2a00640 1 -- 172.21.15.59:0/173653836 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70d4014bc0 con 0x7f70dc1430b0 2024-04-23T14:33:55.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.294+0000 7f70d2a00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70bc06c920 0x7f70bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:55.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.294+0000 7f70d2a00640 1 -- 172.21.15.59:0/173653836 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f70d408c3c0 con 0x7f70dc1430b0 2024-04-23T14:33:55.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.294+0000 7f70daa00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70bc06c920 0x7f70bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:55.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.295+0000 7f70daa00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70bc06c920 0x7f70bc06ede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70cc034000 tx=0x7f70cc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:55.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.299+0000 7f70d2a00640 1 -- 172.21.15.59:0/173653836 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70d4011260 con 0x7f70dc1430b0 2024-04-23T14:33:55.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.549+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70dc157a70 con 0x7f70dc1430b0 2024-04-23T14:33:55.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.550+0000 7f70d2a00640 1 -- 172.21.15.59:0/173653836 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f70d405a7e0 con 0x7f70dc1430b0 2024-04-23T14:33:55.551 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.552+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70bc06c920 msgr2=0x7f70bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.552+0000 7f70e1a00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70bc06c920 0x7f70bc06ede0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70cc034000 tx=0x7f70cc02e000 comp rx=0 tx=0).stop 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.552+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc1430b0 msgr2=0x7f70dc1f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.552+0000 7f70e1a00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc1430b0 0x7f70dc1f7fa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f70d4007e40 tx=0x7f70d400e4b0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 shutdown_connections 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70bc06c920 0x7f70bc06ede0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70dc06f7c0 0x7f70dc142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70dc1430b0 0x7f70dc1f7fa0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 --2- 172.21.15.59:0/173653836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70dc06df00 0x7f70dc1424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 >> 172.21.15.59:0/173653836 conn(0x7f70dc067ea0 msgr2=0x7f70dc0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:55.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 shutdown_connections 2024-04-23T14:33:55.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55.553+0000 7f70e1a00640 1 -- 172.21.15.59:0/173653836 wait complete. 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:33:55.574 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:55.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.575 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:33:55.629 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:33:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:33:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:33:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:33:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:55.629 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42313sec preferred_lft 42313sec 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:55.630 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:33:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:33:55.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.633 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:33:55.720 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:33:55.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:55.725 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:33:55.781 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:33:55.781 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:33:55.781 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:33:55.781 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:33:55.782 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:33:55.782 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:33:55.783 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:33:55.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:33:55.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:33:55.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:33:55.783 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:33:55.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:33:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:56.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:56.006 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:33:56.060 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:33:56.060 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:33:56.061 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:56.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:56.061 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:33:56.144 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:33:56.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:56.145 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:56.202 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:33:56.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:33:56.203 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:56.257 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:33:56.344 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:56.397 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:33:56.398 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:33:56.399 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:56.400 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:56.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:56.401 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:33:56.452 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:33:56.452 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:33:56.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:33:56.524 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:33:56.569 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:33:56.569 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:33:56.569 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:33:56.570 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:33:56.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:56.572 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:33:56.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:33:56.627+0000 7f6d1c52b480 -1 init, newargv = 0x55b0f99cf920 newargc=15 2024-04-23T14:33:56.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[50804]: starting ceph client 2024-04-23T14:33:56.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[50804]: starting fuse 2024-04-23T14:33:56.658 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:33:56.658 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:33:56.658 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:56.659 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:33:56.660 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:56.851 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.50804.asok 2024-04-23T14:33:56.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:56.868 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50804.asok 2024-04-23T14:33:56.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:56.868 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50804.asok status 2024-04-23T14:33:57.100 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "50804", 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5874, 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:33:57.101 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 5874 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2859999596 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2859999596 2024-04-23T14:33:57.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.5874 192.168.0.1:0/2859999596", 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2859999596", 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 25, 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 44, 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:33:57.103 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:57.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:57.116 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:33:57.116 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:57.173 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:33:57.173 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:57.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:57.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:33:57.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:57.264 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-23T14:33:57.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:57.333 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-23T14:33:57.395 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:33:57.395 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:33:57.395 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:33:57.395 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:33:57.395 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-23T14:33:57.395 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:33:57.539 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-23T14:33:57.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:33:57.553 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:33:57.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.812+0000 7faf29400640 1 Processor -- start 2024-04-23T14:33:57.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.812+0000 7faf29400640 1 -- start start 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.813+0000 7faf29400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf2415e9d0 0x7faf2415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.813+0000 7faf29400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf240d42c0 0x7faf240d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf29400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf2415edb0 con 0x7faf240ba0e0 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf29400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faf240d3e30 con 0x7faf240d42c0 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf29400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faf2415ff20 con 0x7faf2415e9d0 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf23200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faf240ba0e0 0x7faf240b6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37908/0 (socket says 172.21.15.59:37908) 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf23200640 1 -- 172.21.15.59:0/3089223984 learned_addr learned my addr 172.21.15.59:0/3089223984 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf21e00640 1 --2- 172.21.15.59:0/3089223984 >> v2:172.21.15.59:3300/0 conn(0x7faf240d42c0 0x7faf240d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf22800640 1 --2- 172.21.15.59:0/3089223984 >> v2:172.21.15.76:3300/0 conn(0x7faf2415e9d0 0x7faf2415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.815+0000 7faf21e00640 1 -- 172.21.15.59:0/3089223984 >> v2:172.21.15.76:3300/0 conn(0x7faf2415e9d0 msgr2=0x7faf2415f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.815+0000 7faf21e00640 1 --2- 172.21.15.59:0/3089223984 >> v2:172.21.15.76:3300/0 conn(0x7faf2415e9d0 0x7faf2415f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.814+0000 7faf21400640 1 -- 172.21.15.59:0/3089223984 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971978096 0 0) 0x7faf2415edb0 con 0x7faf240ba0e0 2024-04-23T14:33:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.815+0000 7faf21e00640 1 -- 172.21.15.59:0/3089223984 >> v1:172.21.15.59:6789/0 conn(0x7faf240ba0e0 legacy=0x7faf240b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.815+0000 7faf21e00640 1 -- 172.21.15.59:0/3089223984 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf240ba630 con 0x7faf240d42c0 2024-04-23T14:33:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.816+0000 7faf21e00640 1 --2- 172.21.15.59:0/3089223984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf240d42c0 0x7faf240d0f60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7faf240d3fb0 tx=0x7faf0c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6f64a422b5c27f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.816+0000 7faf21400640 1 -- 172.21.15.59:0/3089223984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c031600 con 0x7faf240d42c0 2024-04-23T14:33:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.816+0000 7faf21400640 1 -- 172.21.15.59:0/3089223984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c0317a0 con 0x7faf240d42c0 2024-04-23T14:33:57.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.816+0000 7faf21400640 1 -- 172.21.15.59:0/3089223984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c031b70 con 0x7faf240d42c0 2024-04-23T14:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 -- 172.21.15.59:0/3089223984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf240d42c0 msgr2=0x7faf240d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 --2- 172.21.15.59:0/3089223984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf240d42c0 0x7faf240d0f60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7faf240d3fb0 tx=0x7faf0c02a5d0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 -- 172.21.15.59:0/3089223984 shutdown_connections 2024-04-23T14:33:57.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 --2- 172.21.15.59:0/3089223984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf240d42c0 0x7faf240d0f60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 --2- 172.21.15.59:0/3089223984 >> v2:172.21.15.76:3300/0 conn(0x7faf2415e9d0 0x7faf2415f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 -- 172.21.15.59:0/3089223984 >> 172.21.15.59:0/3089223984 conn(0x7faf2406cbd0 msgr2=0x7faf2406cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 -- 172.21.15.59:0/3089223984 shutdown_connections 2024-04-23T14:33:57.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.817+0000 7faf29400640 1 -- 172.21.15.59:0/3089223984 wait complete. 2024-04-23T14:33:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.819+0000 7faf29400640 1 Processor -- start 2024-04-23T14:33:57.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.819+0000 7faf29400640 1 -- start start 2024-04-23T14:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.820+0000 7faf29400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf240ba0e0 0x7faf241f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.820+0000 7faf29400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf240d42c0 0x7faf241f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.820+0000 7faf29400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf2415e9d0 0x7faf24154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.820+0000 7faf29400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf240cfd70 con 0x7faf2415e9d0 2024-04-23T14:33:57.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.820+0000 7faf29400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faf240cfa90 con 0x7faf240ba0e0 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.820+0000 7faf29400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faf240d14a0 con 0x7faf240d42c0 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf2415e9d0 0x7faf24154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf2415e9d0 0x7faf24154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41546/0 (socket says 172.21.15.59:41546) 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 -- 172.21.15.59:0/1542111440 learned_addr learned my addr 172.21.15.59:0/1542111440 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf22800640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf240ba0e0 0x7faf241f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf21e00640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf240d42c0 0x7faf241f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 -- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf240d42c0 msgr2=0x7faf241f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf240d42c0 0x7faf241f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 -- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf240ba0e0 msgr2=0x7faf241f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf240ba0e0 0x7faf241f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 -- 172.21.15.59:0/1542111440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf241f5a80 con 0x7faf2415e9d0 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf21e00640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf240d42c0 0x7faf241f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:57.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.821+0000 7faf23200640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf2415e9d0 0x7faf24154df0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c480 tx=0x7faf1c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.822+0000 7faf1aa00640 1 -- 172.21.15.59:0/1542111440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c018070 con 0x7faf2415e9d0 2024-04-23T14:33:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.822+0000 7faf1aa00640 1 -- 172.21.15.59:0/1542111440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1c007500 con 0x7faf2415e9d0 2024-04-23T14:33:57.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.822+0000 7faf1aa00640 1 -- 172.21.15.59:0/1542111440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c004a20 con 0x7faf2415e9d0 2024-04-23T14:33:57.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.822+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf241f5340 con 0x7faf2415e9d0 2024-04-23T14:33:57.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.822+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf240b43c0 con 0x7faf2415e9d0 2024-04-23T14:33:57.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.823+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf24069190 con 0x7faf2415e9d0 2024-04-23T14:33:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.825+0000 7faf1aa00640 1 -- 172.21.15.59:0/1542111440 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf1c004bc0 con 0x7faf2415e9d0 2024-04-23T14:33:57.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.826+0000 7faf1aa00640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faf0006c8f0 0x7faf0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.826+0000 7faf1aa00640 1 -- 172.21.15.59:0/1542111440 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7faf1c08acd0 con 0x7faf2415e9d0 2024-04-23T14:33:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.826+0000 7faf22800640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faf0006c8f0 0x7faf0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:57.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.827+0000 7faf22800640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faf0006c8f0 0x7faf0006edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf14006fd0 tx=0x7faf14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:57.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:57.831+0000 7faf1aa00640 1 -- 172.21.15.59:0/1542111440 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf241f5340 con 0x7faf2415e9d0 2024-04-23T14:33:58.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.080+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf2406bbe0 con 0x7faf2415e9d0 2024-04-23T14:33:58.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.081+0000 7faf1aa00640 1 -- 172.21.15.59:0/1542111440 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7faf1c059190 con 0x7faf2415e9d0 2024-04-23T14:33:58.083 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:58.083 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:33:58.083 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.084+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faf0006c8f0 msgr2=0x7faf0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.084+0000 7faf29400640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faf0006c8f0 0x7faf0006edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf14006fd0 tx=0x7faf14008040 comp rx=0 tx=0).stop 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.084+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf2415e9d0 msgr2=0x7faf24154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.084+0000 7faf29400640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf2415e9d0 0x7faf24154df0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00c480 tx=0x7faf1c00c940 comp rx=0 tx=0).stop 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 shutdown_connections 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faf0006c8f0 0x7faf0006edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf240d42c0 0x7faf241f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf2415e9d0 0x7faf24154df0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 --2- 172.21.15.59:0/1542111440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf240ba0e0 0x7faf241f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 >> 172.21.15.59:0/1542111440 conn(0x7faf2406cbd0 msgr2=0x7faf241600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 shutdown_connections 2024-04-23T14:33:58.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.085+0000 7faf29400640 1 -- 172.21.15.59:0/1542111440 wait complete. 2024-04-23T14:33:58.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-23T14:33:58.346 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:33:58.346 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:33:58.347 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-23T14:33:58.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.461+0000 7fcca8940f40 1 Processor -- start 2024-04-23T14:33:58.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.461+0000 7fcca8940f40 1 -- start start 2024-04-23T14:33:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcca8940f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc8c003ab0 0x564479319260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcca8940f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x56447931c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcca8940f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564479332660 0x564479332ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcca8940f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564479291cc0 con 0x5644793b7840 2024-04-23T14:33:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcca8940f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564479295d00 con 0x7fcc8c003ab0 2024-04-23T14:33:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcca8940f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x564479335ec0 con 0x564479332660 2024-04-23T14:33:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcc9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x56447931c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcc9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x56447931c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41558/0 (socket says 172.21.15.59:41558) 2024-04-23T14:33:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.462+0000 7fcc9d200640 1 -- 172.21.15.59:0/2145546071 learned_addr learned my addr 172.21.15.59:0/2145546071 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcca6600640 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564479332660 0x564479332ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcca5c00640 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc8c003ab0 0x564479319260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcc9d200640 1 -- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564479332660 msgr2=0x564479332ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcc9d200640 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564479332660 0x564479332ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcc9d200640 1 -- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc8c003ab0 msgr2=0x564479319260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcc9d200640 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc8c003ab0 0x564479319260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcc9d200640 1 -- 172.21.15.59:0/2145546071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644793c36f0 con 0x5644793b7840 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcc9d200640 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x56447931c0b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009760 tx=0x7fcc90005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb44a61f8316c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.463+0000 7fcc9fe00640 1 -- 172.21.15.59:0/2145546071 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90008870 con 0x5644793b7840 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcc9fe00640 1 -- 172.21.15.59:0/2145546071 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90018070 con 0x5644793b7840 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcc9fe00640 1 -- 172.21.15.59:0/2145546071 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90008c50 con 0x5644793b7840 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 -- 172.21.15.59:0/2145546071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 msgr2=0x56447931c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x56447931c0b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009760 tx=0x7fcc90005f90 comp rx=0 tx=0).stop 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 -- 172.21.15.59:0/2145546071 shutdown_connections 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564479332660 0x564479332ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x56447931c0b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 --2- 172.21.15.59:0/2145546071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc8c003ab0 0x564479319260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 -- 172.21.15.59:0/2145546071 >> 172.21.15.59:0/2145546071 conn(0x564479334cc0 msgr2=0x5644793c1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 -- 172.21.15.59:0/2145546071 shutdown_connections 2024-04-23T14:33:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.464+0000 7fcca8940f40 1 -- 172.21.15.59:0/2145546071 wait complete. 2024-04-23T14:33:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.465+0000 7fcca8940f40 1 Processor -- start 2024-04-23T14:33:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.465+0000 7fcca8940f40 1 -- start start 2024-04-23T14:33:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.466+0000 7fcca8940f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564479332660 0x5644793194a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.466+0000 7fcca8940f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x564479452ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.466+0000 7fcca8940f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc8c003ab0 0x5644793a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.466+0000 7fcca8940f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5644793beda0 con 0x5644793b7840 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.466+0000 7fcca8940f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644793c3480 con 0x564479332660 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.466+0000 7fcca8940f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5644793c31a0 con 0x7fcc8c003ab0 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x564479452ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x564479452ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41572/0 (socket says 172.21.15.59:41572) 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 -- 172.21.15.59:0/3563816783 learned_addr learned my addr 172.21.15.59:0/3563816783 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcca6600640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc8c003ab0 0x5644793a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcca5c00640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564479332660 0x5644793194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 -- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc8c003ab0 msgr2=0x5644793a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc8c003ab0 0x5644793a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 -- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564479332660 msgr2=0x5644793194a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564479332660 0x5644793194a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.467+0000 7fcc9d200640 1 -- 172.21.15.59:0/3563816783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644793a4eb0 con 0x5644793b7840 2024-04-23T14:33:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.468+0000 7fcc9d200640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x564479452ad0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcc90000c00 tx=0x7fcc90002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.468+0000 7fcc9ea00640 1 -- 172.21.15.59:0/3563816783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9001c620 con 0x5644793b7840 2024-04-23T14:33:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.468+0000 7fcc9ea00640 1 -- 172.21.15.59:0/3563816783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90021030 con 0x5644793b7840 2024-04-23T14:33:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.468+0000 7fcc9ea00640 1 -- 172.21.15.59:0/3563816783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9001ca20 con 0x5644793b7840 2024-04-23T14:33:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.468+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5644793bfba0 con 0x5644793b7840 2024-04-23T14:33:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.468+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644793c0120 con 0x5644793b7840 2024-04-23T14:33:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.470+0000 7fcc9ea00640 1 -- 172.21.15.59:0/3563816783 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc9001cbc0 con 0x5644793b7840 2024-04-23T14:33:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.471+0000 7fcc9ea00640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406b2d0 0x7fcc8406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.471+0000 7fcc9ea00640 1 -- 172.21.15.59:0/3563816783 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fcc9008c390 con 0x5644793b7840 2024-04-23T14:33:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.471+0000 7fcca5c00640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406b2d0 0x7fcc8406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.472+0000 7fcca8940f40 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x564479453010 0x5644793c0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.472+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x5644793c0e60 con 0x564479453010 2024-04-23T14:33:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.472+0000 7fcca6600640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x564479453010 0x5644793c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.472+0000 7fcca5c00640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406b2d0 0x7fcc8406d790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x564479331740 tx=0x7fcc98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.473+0000 7fcca6600640 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x564479453010 0x5644793c0920 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.474+0000 7fcca6600640 1 -- 172.21.15.59:0/3563816783 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fcca00013b0 con 0x564479453010 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.474+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x564479453010 msgr2=0x5644793c0920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.474+0000 7fcca8940f40 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x564479453010 0x5644793c0920 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406b2d0 msgr2=0x7fcc8406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406b2d0 0x7fcc8406d790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x564479331740 tx=0x7fcc98008040 comp rx=0 tx=0).stop 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 msgr2=0x564479452ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 --2- 172.21.15.59:0/3563816783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644793b7840 0x564479452ad0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcc90000c00 tx=0x7fcc90002fb0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca6600640 1 -- 172.21.15.59:0/3563816783 reap_dead start 2024-04-23T14:33:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 shutdown_connections 2024-04-23T14:33:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 >> 172.21.15.59:0/3563816783 conn(0x564479334cc0 msgr2=0x5644792d4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 shutdown_connections 2024-04-23T14:33:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.475+0000 7fcca8940f40 1 -- 172.21.15.59:0/3563816783 wait complete. 2024-04-23T14:33:58.480 INFO:teuthology.orchestra.run.smithi059.stdout:Yalphaparent_a 2024-04-23T14:33:58.481 DEBUG:teuthology.orchestra.run.smithi059:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: "ancestors": [ 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1099511627776, 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "alpha", 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 2 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:33:58.650 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1, 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "parent_a", 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 7, 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout: "old_pools": [] 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:58.651 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:33:58.665 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-23T14:33:58.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.781+0000 7f46abf4df40 1 Processor -- start 2024-04-23T14:33:58.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.781+0000 7f46abf4df40 1 -- start start 2024-04-23T14:33:58.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.782+0000 7f46abf4df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd3acdb0 0x55afbd43b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.782+0000 7f46abf4df40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3ae450 0x55afbd38f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.782+0000 7f46abf4df40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd41f5b0 0x55afbd393e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.782+0000 7f46abf4df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55afbd393840 con 0x55afbd3acdb0 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.782+0000 7f46abf4df40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55afbd30bcc0 con 0x55afbd41f5b0 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.782+0000 7f46abf4df40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55afbd3939c0 con 0x55afbd3ae450 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.782+0000 7f46a9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd3acdb0 0x55afbd43b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd3acdb0 0x55afbd43b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41592/0 (socket says 172.21.15.59:41592) 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 -- 172.21.15.59:0/142399134 learned_addr learned my addr 172.21.15.59:0/142399134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a8800640 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3ae450 0x55afbd38f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9c00640 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd41f5b0 0x55afbd393e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 -- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3ae450 msgr2=0x55afbd38f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3ae450 0x55afbd38f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 -- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd41f5b0 msgr2=0x55afbd393e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:58.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd41f5b0 0x55afbd393e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 -- 172.21.15.59:0/142399134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afbd43baf0 con 0x55afbd3acdb0 2024-04-23T14:33:58.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a8800640 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3ae450 0x55afbd38f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:58.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.783+0000 7f46a9200640 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd3acdb0 0x55afbd43b5b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f469800a4f0 tx=0x7f4698051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793c8907059ff5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46a7400640 1 -- 172.21.15.59:0/142399134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698069070 con 0x55afbd3acdb0 2024-04-23T14:33:58.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46a7400640 1 -- 172.21.15.59:0/142399134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4698052a90 con 0x55afbd3acdb0 2024-04-23T14:33:58.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46a7400640 1 -- 172.21.15.59:0/142399134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698056070 con 0x55afbd3acdb0 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 -- 172.21.15.59:0/142399134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd3acdb0 msgr2=0x55afbd43b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd3acdb0 0x55afbd43b5b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f469800a4f0 tx=0x7f4698051000 comp rx=0 tx=0).stop 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 -- 172.21.15.59:0/142399134 shutdown_connections 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3ae450 0x55afbd38f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd41f5b0 0x55afbd393e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 --2- 172.21.15.59:0/142399134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd3acdb0 0x55afbd43b5b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 -- 172.21.15.59:0/142399134 >> 172.21.15.59:0/142399134 conn(0x55afbd391e20 msgr2=0x55afbd392200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.784+0000 7f46abf4df40 1 -- 172.21.15.59:0/142399134 shutdown_connections 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.785+0000 7f46abf4df40 1 -- 172.21.15.59:0/142399134 wait complete. 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.785+0000 7f46abf4df40 1 Processor -- start 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.785+0000 7f46abf4df40 1 -- start start 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.786+0000 7f46abf4df40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3acdb0 0x55afbd3ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.786+0000 7f46abf4df40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd3ae450 0x55afbd4cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.786+0000 7f46abf4df40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd41f5b0 0x55afbd439010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.786+0000 7f46abf4df40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55afbd3a5300 con 0x55afbd41f5b0 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.786+0000 7f46abf4df40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55afbd43c080 con 0x55afbd3ae450 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.786+0000 7f46abf4df40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55afbd43bda0 con 0x55afbd3acdb0 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd41f5b0 0x55afbd439010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd41f5b0 0x55afbd439010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41602/0 (socket says 172.21.15.59:41602) 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 -- 172.21.15.59:0/2180024623 learned_addr learned my addr 172.21.15.59:0/2180024623 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9200640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3acdb0 0x55afbd3ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a8800640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd3ae450 0x55afbd4cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 -- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3acdb0 msgr2=0x55afbd3ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55afbd3acdb0 0x55afbd3ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 -- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd3ae450 msgr2=0x55afbd4cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd3ae450 0x55afbd4cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a9c00640 1 -- 172.21.15.59:0/2180024623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afbd433240 con 0x55afbd41f5b0 2024-04-23T14:33:58.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.787+0000 7f46a8800640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55afbd3ae450 0x55afbd4cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:58.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.788+0000 7f46a9c00640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd41f5b0 0x55afbd439010 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f46a000a470 tx=0x7f46a000a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.788+0000 7f46a6000640 1 -- 172.21.15.59:0/2180024623 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0011870 con 0x55afbd41f5b0 2024-04-23T14:33:58.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.788+0000 7f46a6000640 1 -- 172.21.15.59:0/2180024623 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a0011a50 con 0x55afbd41f5b0 2024-04-23T14:33:58.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.788+0000 7f46a6000640 1 -- 172.21.15.59:0/2180024623 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0011e40 con 0x55afbd41f5b0 2024-04-23T14:33:58.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.788+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55afbd3aaf80 con 0x55afbd41f5b0 2024-04-23T14:33:58.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.788+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afbd4337c0 con 0x55afbd41f5b0 2024-04-23T14:33:58.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.790+0000 7f46a6000640 1 -- 172.21.15.59:0/2180024623 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46a00103c0 con 0x55afbd41f5b0 2024-04-23T14:33:58.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.792+0000 7f46a6000640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f468c06b300 0x7f468c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.792+0000 7f46a6000640 1 -- 172.21.15.59:0/2180024623 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f46a008acb0 con 0x55afbd41f5b0 2024-04-23T14:33:58.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.792+0000 7f46a9200640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f468c06b300 0x7f468c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.792+0000 7f46abf4df40 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55afbd4cfb70 0x55afbd4dc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:58.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.792+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x55afbd392200 con 0x55afbd4cfb70 2024-04-23T14:33:58.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.793+0000 7f46a9200640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f468c06b300 0x7f468c06d7c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55afbd3a8d50 tx=0x7f4698053000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.793+0000 7f46a8800640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55afbd4cfb70 0x55afbd4dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:58.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.794+0000 7f46a8800640 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55afbd4cfb70 0x55afbd4dc460 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:58.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46a8800640 1 -- 172.21.15.59:0/2180024623 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v44'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f469c00a6e0 con 0x55afbd4cfb70 2024-04-23T14:33:58.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55afbd4cfb70 msgr2=0x55afbd4dc460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55afbd4cfb70 0x55afbd4dc460 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:58.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f468c06b300 msgr2=0x7f468c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f468c06b300 0x7f468c06d7c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55afbd3a8d50 tx=0x7f4698053000 comp rx=0 tx=0).stop 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd41f5b0 msgr2=0x55afbd439010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 --2- 172.21.15.59:0/2180024623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afbd41f5b0 0x55afbd439010 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f46a000a470 tx=0x7f46a000a930 comp rx=0 tx=0).stop 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46a9c00640 1 -- 172.21.15.59:0/2180024623 reap_dead start 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 shutdown_connections 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 >> 172.21.15.59:0/2180024623 conn(0x55afbd391e20 msgr2=0x55afbd38ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 shutdown_connections 2024-04-23T14:33:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.797+0000 7f46abf4df40 1 -- 172.21.15.59:0/2180024623 wait complete. 2024-04-23T14:33:58.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-23T14:33:58.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:58.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-23T14:33:58.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.997+0000 7f8e4d600640 1 Processor -- start 2024-04-23T14:33:58.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.998+0000 7f8e4d600640 1 -- start start 2024-04-23T14:33:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e4d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e4815e990 0x7f8e480d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e4d600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e480cec40 0x7f8e480d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e4d600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e480cfc70 con 0x7f8e480d4d00 2024-04-23T14:33:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e4d600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8e480d0230 con 0x7f8e4815e990 2024-04-23T14:33:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e4d600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8e480d21c0 con 0x7f8e480cec40 2024-04-23T14:33:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e47e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e4815e990 0x7f8e480d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e47e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e4815e990 0x7f8e480d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41608/0 (socket says 172.21.15.59:41608) 2024-04-23T14:33:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:58.999+0000 7f8e47e00640 1 -- 172.21.15.59:0/190097840 learned_addr learned my addr 172.21.15.59:0/190097840 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.000+0000 7f8e47400640 1 --2- 172.21.15.59:0/190097840 >> v2:172.21.15.76:3300/0 conn(0x7f8e480cec40 0x7f8e480d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.000+0000 7f8e47e00640 1 -- 172.21.15.59:0/190097840 >> v2:172.21.15.76:3300/0 conn(0x7f8e480cec40 msgr2=0x7f8e480d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.000+0000 7f8e47e00640 1 --2- 172.21.15.59:0/190097840 >> v2:172.21.15.76:3300/0 conn(0x7f8e480cec40 0x7f8e480d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.000+0000 7f8e47e00640 1 -- 172.21.15.59:0/190097840 >> v1:172.21.15.76:6789/0 conn(0x7f8e480d4d00 legacy=0x7f8e4806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.000+0000 7f8e47e00640 1 -- 172.21.15.59:0/190097840 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e480cff50 con 0x7f8e4815e990 2024-04-23T14:33:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.000+0000 7f8e47400640 1 --2- 172.21.15.59:0/190097840 >> v2:172.21.15.76:3300/0 conn(0x7f8e480cec40 0x7f8e480d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.001+0000 7f8e47e00640 1 --2- 172.21.15.59:0/190097840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4815e990 0x7f8e480d1000 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8e480d0fd0 tx=0x7f8e3802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fca22d12041b482 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.002+0000 7f8e46a00640 1 -- 172.21.15.59:0/190097840 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38037070 con 0x7f8e4815e990 2024-04-23T14:33:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.002+0000 7f8e46a00640 1 -- 172.21.15.59:0/190097840 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e38032440 con 0x7f8e4815e990 2024-04-23T14:33:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.002+0000 7f8e46a00640 1 -- 172.21.15.59:0/190097840 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38032810 con 0x7f8e4815e990 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.002+0000 7f8e4d600640 1 -- 172.21.15.59:0/190097840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4815e990 msgr2=0x7f8e480d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.002+0000 7f8e4d600640 1 --2- 172.21.15.59:0/190097840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4815e990 0x7f8e480d1000 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8e480d0fd0 tx=0x7f8e3802a200 comp rx=0 tx=0).stop 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.003+0000 7f8e4d600640 1 -- 172.21.15.59:0/190097840 shutdown_connections 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.003+0000 7f8e4d600640 1 --2- 172.21.15.59:0/190097840 >> v2:172.21.15.76:3300/0 conn(0x7f8e480cec40 0x7f8e480d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.003+0000 7f8e4d600640 1 --2- 172.21.15.59:0/190097840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4815e990 0x7f8e480d1000 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.003+0000 7f8e4d600640 1 -- 172.21.15.59:0/190097840 >> 172.21.15.59:0/190097840 conn(0x7f8e4808bf90 msgr2=0x7f8e480d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.003+0000 7f8e4d600640 1 -- 172.21.15.59:0/190097840 shutdown_connections 2024-04-23T14:33:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.003+0000 7f8e4d600640 1 -- 172.21.15.59:0/190097840 wait complete. 2024-04-23T14:33:59.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.005+0000 7f8e4d600640 1 Processor -- start 2024-04-23T14:33:59.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.005+0000 7f8e4d600640 1 -- start start 2024-04-23T14:33:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e4d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e480cec40 0x7f8e4814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e4d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e480d4d00 0x7f8e480c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e4d600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e4815e990 0x7f8e480c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e4d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4806e1f0 con 0x7f8e480cec40 2024-04-23T14:33:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e4d600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e480d26f0 con 0x7f8e4815e990 2024-04-23T14:33:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e4d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8e48158400 con 0x7f8e480d4d00 2024-04-23T14:33:59.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e480cec40 0x7f8e4814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e480cec40 0x7f8e4814ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41620/0 (socket says 172.21.15.59:41620) 2024-04-23T14:33:59.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.006+0000 7f8e47e00640 1 -- 172.21.15.59:0/1758597322 learned_addr learned my addr 172.21.15.59:0/1758597322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e47e00640 1 -- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e480d4d00 msgr2=0x7f8e480c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e480d4d00 0x7f8e480c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e47e00640 1 -- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e4815e990 msgr2=0x7f8e480c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e4815e990 0x7f8e480c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e4815e990 0x7f8e480c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e47e00640 1 -- 172.21.15.59:0/1758597322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e480b8e40 con 0x7f8e480cec40 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e4815e990 0x7f8e480c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.007+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e480cec40 0x7f8e4814ea30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8e38000c00 tx=0x7f8e380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.008+0000 7f8e3fe00640 1 -- 172.21.15.59:0/1758597322 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38041020 con 0x7f8e480cec40 2024-04-23T14:33:59.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.008+0000 7f8e3fe00640 1 -- 172.21.15.59:0/1758597322 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e38030970 con 0x7f8e480cec40 2024-04-23T14:33:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.008+0000 7f8e3fe00640 1 -- 172.21.15.59:0/1758597322 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38037040 con 0x7f8e480cec40 2024-04-23T14:33:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.008+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e480b9120 con 0x7f8e480cec40 2024-04-23T14:33:59.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.008+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e480b96e0 con 0x7f8e480cec40 2024-04-23T14:33:59.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.011+0000 7f8e3fe00640 1 -- 172.21.15.59:0/1758597322 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e38044020 con 0x7f8e480cec40 2024-04-23T14:33:59.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.012+0000 7f8e51a3e740 1 Processor -- start 2024-04-23T14:33:59.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.012+0000 7f8e3fe00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8e2406c8f0 0x7f8e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.012+0000 7f8e51a3e740 1 -- start start 2024-04-23T14:33:59.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.012+0000 7f8e3fe00640 1 -- 172.21.15.59:0/1758597322 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8e380ac260 con 0x7f8e480cec40 2024-04-23T14:33:59.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8e2406c8f0 0x7f8e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9420640 0x557cb93d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9403760 0x557cb9402ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403020 0x557cb93d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8e2406c8f0 0x7f8e2406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e480b5410 tx=0x7f8e40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557cb933ce70 con 0x557cb9403020 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557cb92e3380 con 0x557cb9403760 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.013+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557cb9338e30 con 0x557cb9420640 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403020 0x557cb93d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403020 0x557cb93d7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41636/0 (socket says 172.21.15.59:41636) 2024-04-23T14:33:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 -- 172.21.15.59:0/3847562039 learned_addr learned my addr 172.21.15.59:0/3847562039 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e47e00640 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9420640 0x557cb93d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e47400640 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9403760 0x557cb9402ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 -- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9420640 msgr2=0x557cb93d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9420640 0x557cb93d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 -- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9403760 msgr2=0x557cb9402ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9403760 0x557cb9402ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 -- 172.21.15.59:0/3847562039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb91177e0 con 0x557cb9403020 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.014+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403020 0x557cb93d7e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000da90 tx=0x7f8e40010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c9d42051e6091a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.015+0000 7f8e37e00640 1 -- 172.21.15.59:0/3847562039 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40024070 con 0x557cb9403020 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.015+0000 7f8e37e00640 1 -- 172.21.15.59:0/3847562039 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e40014db0 con 0x557cb9403020 2024-04-23T14:33:59.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.015+0000 7f8e37e00640 1 -- 172.21.15.59:0/3847562039 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4001f640 con 0x557cb9403020 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3847562039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403020 msgr2=0x557cb93d7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403020 0x557cb93d7e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000da90 tx=0x7f8e40010000 comp rx=0 tx=0).stop 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3847562039 shutdown_connections 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403020 0x557cb93d7e30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9403760 0x557cb9402ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3847562039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9420640 0x557cb93d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3847562039 >> 172.21.15.59:0/3847562039 conn(0x557cb93d5240 msgr2=0x557cb941dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3847562039 shutdown_connections 2024-04-23T14:33:59.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3847562039 wait complete. 2024-04-23T14:33:59.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.016+0000 7f8e51a3e740 1 -- start start 2024-04-23T14:33:59.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.017+0000 7f8e51a3e740 1 Processor -- start 2024-04-23T14:33:59.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb93fe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb94161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb94166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557cb93d73f0 con 0x557cb9420640 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557cb92f6660 con 0x557cb94147a0 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557cb93d67e0 con 0x557cb9403760 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e51a3e740 10 client.0 _ll_register_callbacks cb 0x557cb92e4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e47400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb94161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e47400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb94161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41640/0 (socket says 172.21.15.59:41640) 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e47400640 1 -- 172.21.15.59:0/3983016839 learned_addr learned my addr 172.21.15.59:0/3983016839 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e47e00640 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb93fe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb94166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.018+0000 7f8e47400640 1 -- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 msgr2=0x557cb93fe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e47400640 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb93fe6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e47400640 1 -- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 msgr2=0x557cb94166f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e47400640 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb94166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e47400640 1 -- 172.21.15.59:0/3983016839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb9177f70 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e47400640 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb94161b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000dbf0 tx=0x7f8e3000dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca891012b130bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e34c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e34c00640 1 -- 172.21.15.59:0/3983016839 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3001d060 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e34c00640 1 -- 172.21.15.59:0/3983016839 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8e3000b980 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.019+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e40014db0 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e34c00640 1 -- 172.21.15.59:0/3983016839 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e30023050 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 10 client.5913 fetch_fsmap learned FSMap version 25 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x557cb900f170 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e34c00640 1 -- 172.21.15.59:0/3983016839 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e40014db0 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e34c00640 1 -- 172.21.15.59:0/3983016839 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f8e30026020 con 0x557cb9420640 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 10 client.5913 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 10 client.5913 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 20 client.5913 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9431660 0x557cb9431a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 4 client.5913 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:33:59.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cb934a890 con 0x557cb9431660 2024-04-23T14:33:59.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.020+0000 7f8e47e00640 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9431660 0x557cb9431a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.021+0000 7f8e47e00640 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9431660 0x557cb9431a70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88cac7d54ee21a6d server_cookie=d81ef6cfd143364d in_seq=0 out_seq=0 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.021+0000 7f8e34c00640 10 client.5913 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e34c00640 1 -- 172.21.15.59:0/3983016839 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557cb934a890 con 0x557cb9431660 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e34c00640 10 client.5913 handle_command_reply: tid=0 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 client.5913 shutdown 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 msgr2=0x557cb94161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb94161b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000dbf0 tx=0x7f8e3000dce0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 shutdown_connections 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9431660 0x557cb9431a70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb94166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb94161b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/3983016839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb93fe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e34c00640 0 client.5913 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 >> 172.21.15.59:0/3983016839 conn(0x557cb9403020 msgr2=0x557cb93d3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 shutdown_connections 2024-04-23T14:33:59.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.022+0000 7f8e51a3e740 1 -- 172.21.15.59:0/3983016839 wait complete. 2024-04-23T14:33:59.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.023+0000 7f8e51a3e740 20 client.5913 trim_cache size 0 max 16384 2024-04-23T14:33:59.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.023+0000 7fd904600640 1 Processor -- start 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.023+0000 7fd904600640 1 -- start start 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.024+0000 7fd904600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc146000 0x7fd8fc15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd904600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8fc0cd090 0x7fd8fc0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd904600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8fc0cf330 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd904600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd8fc0cfc00 con 0x7fd8fc0cd090 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd904600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc06f370 con 0x7fd8fc146000 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd902600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8fc0cd090 0x7fd8fc0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd902600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8fc0cd090 0x7fd8fc0cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41644/0 (socket says 172.21.15.59:41644) 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd902600640 1 -- 172.21.15.59:0/1286744560 learned_addr learned my addr 172.21.15.59:0/1286744560 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd901200640 1 --2- 172.21.15.59:0/1286744560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc146000 0x7fd8fc15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd902600640 1 -- 172.21.15.59:0/1286744560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc146000 msgr2=0x7fd8fc15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd902600640 1 --2- 172.21.15.59:0/1286744560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc146000 0x7fd8fc15cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd902600640 1 -- 172.21.15.59:0/1286744560 >> v1:172.21.15.76:6789/0 conn(0x7fd8fc15fdb0 legacy=0x7fd8fc06ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.025+0000 7fd902600640 1 -- 172.21.15.59:0/1286744560 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc0cf5b0 con 0x7fd8fc0cd090 2024-04-23T14:33:59.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.026+0000 7fd902600640 1 --2- 172.21.15.59:0/1286744560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc0cd090 0x7fd8fc0cd520 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009e50 tx=0x7fd8f402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436a8ed703e69ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.027+0000 7fd8fbe00640 1 -- 172.21.15.59:0/1286744560 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4036070 con 0x7fd8fc0cd090 2024-04-23T14:33:59.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.027+0000 7fd8fbe00640 1 -- 172.21.15.59:0/1286744560 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f4031600 con 0x7fd8fc0cd090 2024-04-23T14:33:59.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.027+0000 7fd8fbe00640 1 -- 172.21.15.59:0/1286744560 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40319d0 con 0x7fd8fc0cd090 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.027+0000 7fd904600640 1 -- 172.21.15.59:0/1286744560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc0cd090 msgr2=0x7fd8fc0cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.027+0000 7fd904600640 1 --2- 172.21.15.59:0/1286744560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc0cd090 0x7fd8fc0cd520 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009e50 tx=0x7fd8f402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.028+0000 7fd904600640 1 -- 172.21.15.59:0/1286744560 shutdown_connections 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.028+0000 7fd904600640 1 --2- 172.21.15.59:0/1286744560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc0cd090 0x7fd8fc0cd520 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.028+0000 7fd904600640 1 --2- 172.21.15.59:0/1286744560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc146000 0x7fd8fc15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.028+0000 7fd904600640 1 -- 172.21.15.59:0/1286744560 >> 172.21.15.59:0/1286744560 conn(0x7fd8fc067ea0 msgr2=0x7fd8fc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.028+0000 7fd904600640 1 -- 172.21.15.59:0/1286744560 shutdown_connections 2024-04-23T14:33:59.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.028+0000 7fd904600640 1 -- 172.21.15.59:0/1286744560 wait complete. 2024-04-23T14:33:59.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.029+0000 7fd904600640 1 Processor -- start 2024-04-23T14:33:59.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.030+0000 7fd904600640 1 -- start start 2024-04-23T14:33:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd904600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc0cd090 0x7fd8fc1f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd904600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc146000 0x7fd8fc0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd904600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc15fdb0 0x7fd8fc0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd904600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0d0090 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd904600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0cf890 con 0x7fd8fc146000 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd904600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc0d0210 con 0x7fd8fc0cd090 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd902600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc15fdb0 0x7fd8fc0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd902600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc15fdb0 0x7fd8fc0b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41658/0 (socket says 172.21.15.59:41658) 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd902600640 1 -- 172.21.15.59:0/3913791185 learned_addr learned my addr 172.21.15.59:0/3913791185 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd901c00640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc0cd090 0x7fd8fc1f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd901200640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc146000 0x7fd8fc0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd902600640 1 -- 172.21.15.59:0/3913791185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc0cd090 msgr2=0x7fd8fc1f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd902600640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8fc0cd090 0x7fd8fc1f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd902600640 1 -- 172.21.15.59:0/3913791185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc146000 msgr2=0x7fd8fc0b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.031+0000 7fd902600640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc146000 0x7fd8fc0b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.032+0000 7fd902600640 1 -- 172.21.15.59:0/3913791185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc0c55b0 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.032+0000 7fd902600640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8fc15fdb0 0x7fd8fc0b44d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4034040 tx=0x7fd8f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.032+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4036050 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.032+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f4042070 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.032+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4009510 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.032+0000 7fd904600640 1 -- 172.21.15.59:0/3913791185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc0c5890 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.033+0000 7fd904600640 1 -- 172.21.15.59:0/3913791185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc0c5e50 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.033+0000 7fd904600640 1 -- 172.21.15.59:0/3913791185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd8fc073800 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.035+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8f4007bb0 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.035+0000 7fd8f9600640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd8dc06c940 0x7fd8dc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.036+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fd8f40ad520 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.036+0000 7fd901c00640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd8dc06c940 0x7fd8dc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.036+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7fd8fc0cf330 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.036+0000 7fd904600640 1 -- 172.21.15.59:0/3913791185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fd8fc04fe70 con 0x7fd8fc15fdb0 2024-04-23T14:33:59.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.037+0000 7fd901c00640 1 --2- 172.21.15.59:0/3913791185 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd8dc06c940 0x7fd8dc06ee00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0006fd0 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.043+0000 7f8e51a3e740 1 Processor -- start 2024-04-23T14:33:59.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.043+0000 7f8e51a3e740 1 -- start start 2024-04-23T14:33:59.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403760 0x557cb9441730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb94147a0 0x557cb9405530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9420640 0x557cb942fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557cb9417b30 con 0x557cb9403760 2024-04-23T14:33:59.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48200cd0 con 0x557cb9420640 2024-04-23T14:33:59.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8e481ffd40 con 0x557cb94147a0 2024-04-23T14:33:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403760 0x557cb9441730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403760 0x557cb9441730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41666/0 (socket says 172.21.15.59:41666) 2024-04-23T14:33:59.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.044+0000 7f8e47e00640 1 -- 172.21.15.59:0/1875985722 learned_addr learned my addr 172.21.15.59:0/1875985722 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9420640 0x557cb942fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e47e00640 1 -- 172.21.15.59:0/1875985722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb94147a0 msgr2=0x557cb9405530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb94147a0 0x557cb9405530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e47e00640 1 -- 172.21.15.59:0/1875985722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9420640 msgr2=0x557cb942fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9420640 0x557cb942fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e47e00640 1 -- 172.21.15.59:0/1875985722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb942d670 con 0x557cb9403760 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403760 0x557cb9441730 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e38079f20 tx=0x7f8e3803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c06904c95fb9b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e37400640 1 -- 172.21.15.59:0/1875985722 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38092070 con 0x557cb9403760 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e37400640 1 -- 172.21.15.59:0/1875985722 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e38082b70 con 0x557cb9403760 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.045+0000 7f8e37400640 1 -- 172.21.15.59:0/1875985722 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38078030 con 0x557cb9403760 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1875985722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403760 msgr2=0x557cb9441730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403760 0x557cb9441730 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e38079f20 tx=0x7f8e3803f040 comp rx=0 tx=0).stop 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1875985722 shutdown_connections 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb94147a0 0x557cb9405530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb9420640 0x557cb942fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1875985722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9403760 0x557cb9441730 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1875985722 >> 172.21.15.59:0/1875985722 conn(0x557cb9403020 msgr2=0x557cb93d3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1875985722 shutdown_connections 2024-04-23T14:33:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.046+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1875985722 wait complete. 2024-04-23T14:33:59.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.047+0000 7f8e51a3e740 1 -- start start 2024-04-23T14:33:59.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.047+0000 7f8e51a3e740 1 Processor -- start 2024-04-23T14:33:59.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb94977f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb949b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e51a3e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb9405c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e480ccf20 con 0x557cb9420640 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e480cc8d0 con 0x557cb94147a0 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e51a3e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557cb93d2820 con 0x557cb9403760 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e51a3e740 10 client.0 _ll_register_callbacks cb 0x7f8e300087a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.048+0000 7f8e4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb9405c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb9405c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41682/0 (socket says 172.21.15.59:41682) 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 -- 172.21.15.59:0/1226239373 learned_addr learned my addr 172.21.15.59:0/1226239373 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb94977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e47400640 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb949b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 -- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 msgr2=0x557cb94977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb94977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 -- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 msgr2=0x557cb949b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb949b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 -- 172.21.15.59:0/1226239373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb9497d30 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e4cc00640 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb9405c60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005410 tx=0x7f8e400036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751c7063e7bd6947 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.049+0000 7f8e34c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e34c00640 1 -- 172.21.15.59:0/1226239373 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4002d020 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e34c00640 1 -- 172.21.15.59:0/1226239373 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8e40007240 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e38082b70 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e34c00640 1 -- 172.21.15.59:0/1226239373 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e40024040 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 10 client.5925 fetch_fsmap learned FSMap version 25 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x557cb9406db0 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e34c00640 1 -- 172.21.15.59:0/1226239373 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e38082b70 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e34c00640 1 -- 172.21.15.59:0/1226239373 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f8e40007440 con 0x557cb9420640 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 10 client.5925 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:33:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 10 client.5925 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-23T14:33:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 20 client.5925 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9430d20 0x557cb942c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 4 client.5925 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-23T14:33:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.050+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x557cb9344700 con 0x557cb9430d20 2024-04-23T14:33:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.051+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9430d20 0x557cb942c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.051+0000 7f8e47e00640 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9430d20 0x557cb942c200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4144680b5f04fdb9 server_cookie=6814fffb5511228f in_seq=0 out_seq=0 2024-04-23T14:33:59.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.051+0000 7f8e34c00640 10 client.5925 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e34c00640 1 -- 172.21.15.59:0/1226239373 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557cb9344700 con 0x557cb9430d20 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e34c00640 10 client.5925 handle_command_reply: tid=0 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 client.5925 shutdown 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 msgr2=0x557cb9405c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb9405c60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005410 tx=0x7f8e400036f0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 shutdown_connections 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x557cb9430d20 0x557cb942c200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557cb9420640 0x557cb9405c60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cb94147a0 0x557cb949b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.052+0000 7f8e51a3e740 1 --2- 172.21.15.59:0/1226239373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557cb9403760 0x557cb94977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.053+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 >> 172.21.15.59:0/1226239373 conn(0x557cb9403020 msgr2=0x557cb93d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.053+0000 7f8e34c00640 0 client.5925 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.053+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 shutdown_connections 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.053+0000 7f8e51a3e740 1 -- 172.21.15.59:0/1226239373 wait complete. 2024-04-23T14:33:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.053+0000 7f8e51a3e740 20 client.5925 trim_cache size 0 max 16384 2024-04-23T14:33:59.057 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:33:59.053+0000 7fc2a5400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5874=0-4194304@1} caps={5874=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5874 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560a15d32c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:33:59.057 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:33:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "e9e224e8-3bf7-492a-a1fa-99ba1220dcd8", 2024-04-23T14:33:59.057 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:33:59.057 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8e2406c8f0 msgr2=0x7f8e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8e2406c8f0 0x7f8e2406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e480b5410 tx=0x7f8e40009040 comp rx=0 tx=0).stop 2024-04-23T14:33:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e480cec40 msgr2=0x7f8e4814ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e480cec40 0x7f8e4814ea30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8e38000c00 tx=0x7f8e380046c0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 shutdown_connections 2024-04-23T14:33:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8e2406c8f0 0x7f8e2406edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e480d4d00 0x7f8e480c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e4815e990 0x7f8e480c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 --2- 172.21.15.59:0/1758597322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e480cec40 0x7f8e4814ea30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 >> 172.21.15.59:0/1758597322 conn(0x7f8e4808bf90 msgr2=0x7f8e480b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 shutdown_connections 2024-04-23T14:33:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.054+0000 7f8e4d600640 1 -- 172.21.15.59:0/1758597322 wait complete. 2024-04-23T14:33:59.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:33:59.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-23T14:33:59.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.258+0000 7fc1df000640 1 Processor -- start 2024-04-23T14:33:59.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.258+0000 7fc1df000640 1 -- start start 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.259+0000 7fc1df000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc1d815ed20 0x7fc1d80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.259+0000 7fc1df000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1d80d13e0 0x7fc1d80d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.260+0000 7fc1df000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d80b5da0 con 0x7fc1d80b3600 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.260+0000 7fc1df000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d815f100 con 0x7fc1d80d13e0 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.260+0000 7fc1df000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d80d07d0 con 0x7fc1d815ed20 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.260+0000 7fc1de600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1d80d13e0 0x7fc1d80d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.260+0000 7fc1de600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1d80d13e0 0x7fc1d80d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41698/0 (socket says 172.21.15.59:41698) 2024-04-23T14:33:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.260+0000 7fc1de600640 1 -- 172.21.15.59:0/451604746 learned_addr learned my addr 172.21.15.59:0/451604746 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.260+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/451604746 >> v2:172.21.15.76:3300/0 conn(0x7fc1d815ed20 0x7fc1d80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.261+0000 7fc1de600640 1 -- 172.21.15.59:0/451604746 >> v2:172.21.15.76:3300/0 conn(0x7fc1d815ed20 msgr2=0x7fc1d80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.261+0000 7fc1de600640 1 --2- 172.21.15.59:0/451604746 >> v2:172.21.15.76:3300/0 conn(0x7fc1d815ed20 0x7fc1d80d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.261+0000 7fc1de600640 1 -- 172.21.15.59:0/451604746 >> v1:172.21.15.76:6789/0 conn(0x7fc1d80b3600 legacy=0x7fc1d80b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.261+0000 7fc1de600640 1 -- 172.21.15.59:0/451604746 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80d0a20 con 0x7fc1d80d13e0 2024-04-23T14:33:59.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.262+0000 7fc1de600640 1 --2- 172.21.15.59:0/451604746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80d0290 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc1d808e2a0 tx=0x7fc1d002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ff57d84dc39ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.262+0000 7fc1d7e00640 1 -- 172.21.15.59:0/451604746 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0036070 con 0x7fc1d80d13e0 2024-04-23T14:33:59.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.262+0000 7fc1d7e00640 1 -- 172.21.15.59:0/451604746 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d0031600 con 0x7fc1d80d13e0 2024-04-23T14:33:59.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.262+0000 7fc1d7e00640 1 -- 172.21.15.59:0/451604746 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00319d0 con 0x7fc1d80d13e0 2024-04-23T14:33:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 -- 172.21.15.59:0/451604746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80d13e0 msgr2=0x7fc1d80d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 --2- 172.21.15.59:0/451604746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80d0290 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc1d808e2a0 tx=0x7fc1d002a5d0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 -- 172.21.15.59:0/451604746 shutdown_connections 2024-04-23T14:33:59.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 --2- 172.21.15.59:0/451604746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80d0290 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 --2- 172.21.15.59:0/451604746 >> v2:172.21.15.76:3300/0 conn(0x7fc1d815ed20 0x7fc1d80d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 -- 172.21.15.59:0/451604746 >> 172.21.15.59:0/451604746 conn(0x7fc1d808bf90 msgr2=0x7fc1d80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 -- 172.21.15.59:0/451604746 shutdown_connections 2024-04-23T14:33:59.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.263+0000 7fc1df000640 1 -- 172.21.15.59:0/451604746 wait complete. 2024-04-23T14:33:59.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.265+0000 7fc1df000640 1 Processor -- start 2024-04-23T14:33:59.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.265+0000 7fc1df000640 1 -- start start 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1df000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80b3600 0x7fc1d80b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1df000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1df000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1d815ed20 0x7fc1d8151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1df000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d80d3490 con 0x7fc1d80b3600 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1df000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d80d31b0 con 0x7fc1d80d13e0 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1df000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d80b4620 con 0x7fc1d815ed20 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80b3600 0x7fc1d80b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80b3600 0x7fc1d80b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41712/0 (socket says 172.21.15.59:41712) 2024-04-23T14:33:59.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1ddc00640 1 -- 172.21.15.59:0/155701607 learned_addr learned my addr 172.21.15.59:0/155701607 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.266+0000 7fc1dd200640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1ddc00640 1 -- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1d815ed20 msgr2=0x7fc1d8151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1d815ed20 0x7fc1d8151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1ddc00640 1 -- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1d80d13e0 msgr2=0x7fc1d80b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1ddc00640 1 -- 172.21.15.59:0/155701607 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80b90b0 con 0x7fc1d80b3600 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1dd200640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80b3600 0x7fc1d80b8780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00d5f0 tx=0x7fc1cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1d5600640 1 -- 172.21.15.59:0/155701607 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0125f0 con 0x7fc1d80b3600 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1d5600640 1 -- 172.21.15.59:0/155701607 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc019070 con 0x7fc1d80b3600 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1d5600640 1 -- 172.21.15.59:0/155701607 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0129d0 con 0x7fc1d80b3600 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.267+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d80b9390 con 0x7fc1d80b3600 2024-04-23T14:33:59.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.268+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d80b9c30 con 0x7fc1d80b3600 2024-04-23T14:33:59.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.270+0000 7fc1d5600640 1 -- 172.21.15.59:0/155701607 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1cc012b70 con 0x7fc1d80b3600 2024-04-23T14:33:59.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.272+0000 7fc1d5600640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1b806c940 0x7fc1b806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.272+0000 7fc1d5600640 1 -- 172.21.15.59:0/155701607 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc1cc08bef0 con 0x7fc1d80b3600 2024-04-23T14:33:59.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.272+0000 7fc1dd200640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1b806c940 0x7fc1b806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.272+0000 7fc1e3414740 1 Processor -- start 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.272+0000 7fc1e3414740 1 -- start start 2024-04-23T14:33:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.273+0000 7fc1dd200640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1b806c940 0x7fc1b806ee00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80d4f30 tx=0x7fc1c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.273+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.273+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.273+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4cb7e0 0x56110e4a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.273+0000 7fc1e3414740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56110e4051f0 con 0x56110e4cbf20 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.273+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56110e3ab350 con 0x56110e4cb7e0 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.273+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x56110e4011b0 con 0x56110e4e8e00 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41714/0 (socket says 172.21.15.59:41714) 2024-04-23T14:33:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 -- 172.21.15.59:0/515645875 learned_addr learned my addr 172.21.15.59:0/515645875 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1de600640 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4cb7e0 0x56110e4a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 -- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 msgr2=0x56110e49b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 -- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4cb7e0 msgr2=0x56110e4a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4cb7e0 0x56110e4a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1dd200640 1 -- 172.21.15.59:0/515645875 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110e3fff50 con 0x56110e4cbf20 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:59.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.274+0000 7fc1de600640 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4cb7e0 0x56110e4a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:33:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.275+0000 7fc1dd200640 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4cb2a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400dac0 tx=0x7fc1c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebbfc7f948de4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.275+0000 7fc1ca000640 1 -- 172.21.15.59:0/515645875 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4024070 con 0x56110e4cbf20 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.275+0000 7fc1ca000640 1 -- 172.21.15.59:0/515645875 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c40149a0 con 0x56110e4cbf20 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.275+0000 7fc1ca000640 1 -- 172.21.15.59:0/515645875 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4014d90 con 0x56110e4cbf20 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 -- 172.21.15.59:0/515645875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 msgr2=0x56110e4cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4cb2a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400dac0 tx=0x7fc1c4010000 comp rx=0 tx=0).stop 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 -- 172.21.15.59:0/515645875 shutdown_connections 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4cb7e0 0x56110e4a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4cb2a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 --2- 172.21.15.59:0/515645875 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 -- 172.21.15.59:0/515645875 >> 172.21.15.59:0/515645875 conn(0x56110e49da00 msgr2=0x56110e4e62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 -- 172.21.15.59:0/515645875 shutdown_connections 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.276+0000 7fc1e3414740 1 -- 172.21.15.59:0/515645875 wait complete. 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.277+0000 7fc1e3414740 1 -- start start 2024-04-23T14:33:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.277+0000 7fc1e3414740 1 Processor -- start 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e4dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4e8e00 0x56110e4cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4f9e50 0x56110e4e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1e3414740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56110e49fbb0 con 0x56110e4f9e50 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56110e3be690 con 0x56110e4e8e00 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x56110e49efa0 con 0x56110e4cbf20 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1e3414740 10 client.0 _ll_register_callbacks cb 0x56110e3fa650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1de600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4f9e50 0x56110e4e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1de600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4f9e50 0x56110e4e2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41730/0 (socket says 172.21.15.59:41730) 2024-04-23T14:33:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.278+0000 7fc1de600640 1 -- 172.21.15.59:0/4213065735 learned_addr learned my addr 172.21.15.59:0/4213065735 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e4dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1dd200640 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4e8e00 0x56110e4cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1de600640 1 -- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 msgr2=0x56110e4dd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1de600640 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e4dd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1de600640 1 -- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4e8e00 msgr2=0x56110e4cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1de600640 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4e8e00 0x56110e4cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1de600640 1 -- 172.21.15.59:0/4213065735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110e3bd920 con 0x56110e4f9e50 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1de600640 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4f9e50 0x56110e4e2ca0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc1d002abc0 tx=0x7fc1d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9493867b70787625 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1bea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1bea00640 1 -- 172.21.15.59:0/4213065735 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0041020 con 0x56110e4f9e50 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1bea00640 1 -- 172.21.15.59:0/4213065735 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc1d0031da0 con 0x56110e4f9e50 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.279+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c40149a0 con 0x56110e4f9e50 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1bea00640 1 -- 172.21.15.59:0/4213065735 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc1d004b050 con 0x56110e4f9e50 2024-04-23T14:33:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 10 client.5937 fetch_fsmap learned FSMap version 25 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x56110e1e17d0 con 0x56110e4f9e50 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1bea00640 1 -- 172.21.15.59:0/4213065735 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c40149a0 con 0x56110e4f9e50 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1bea00640 1 -- 172.21.15.59:0/4213065735 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fc1d004e020 con 0x56110e4f9e50 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 10 client.5937 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 10 client.5937 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 20 client.5937 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e4e36b0 0x56110e4c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 4 client.5937 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:33:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56110e412c10 con 0x56110e4e36b0 2024-04-23T14:33:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.280+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e4e36b0 0x56110e4c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.281+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e4e36b0 0x56110e4c9260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e017caf4f581cf server_cookie=b2a6ea6d8e389098 in_seq=0 out_seq=0 2024-04-23T14:33:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.281+0000 7fc1bea00640 10 client.5937 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1bea00640 1 -- 172.21.15.59:0/4213065735 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56110e412c10 con 0x56110e4e36b0 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1bea00640 10 client.5937 handle_command_reply: tid=0 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 client.5937 shutdown 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4f9e50 msgr2=0x56110e4e2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4f9e50 0x56110e4e2ca0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc1d002abc0 tx=0x7fc1d0004a40 comp rx=0 tx=0).stop 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 shutdown_connections 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e4e36b0 0x56110e4c9260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4f9e50 0x56110e4e2ca0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4e8e00 0x56110e4cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 --2- 172.21.15.59:0/4213065735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e4dd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1bea00640 0 client.5937 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 >> 172.21.15.59:0/4213065735 conn(0x56110e4cb7e0 msgr2=0x56110e4e62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 shutdown_connections 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 1 -- 172.21.15.59:0/4213065735 wait complete. 2024-04-23T14:33:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.282+0000 7fc1e3414740 20 client.5937 trim_cache size 0 max 16384 2024-04-23T14:33:59.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.302+0000 7fc1e3414740 1 Processor -- start 2024-04-23T14:33:59.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.302+0000 7fc1e3414740 1 -- start start 2024-04-23T14:33:59.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.303+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e49a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.303+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4e8e00 0x56110e4f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.303+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e4e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.303+0000 7fc1e3414740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8200d90 con 0x56110e4e8e00 2024-04-23T14:33:59.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.303+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81fff70 con 0x56110e4f9e50 2024-04-23T14:33:59.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.303+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d81f89b0 con 0x56110e4cbf20 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4e8e00 0x56110e4f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4e8e00 0x56110e4f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41734/0 (socket says 172.21.15.59:41734) 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 -- 172.21.15.59:0/316849228 learned_addr learned my addr 172.21.15.59:0/316849228 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e49a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1de600640 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e4e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 -- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 msgr2=0x56110e49a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e49a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 -- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 msgr2=0x56110e4e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e4e75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 -- 172.21.15.59:0/316849228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110e1173f0 con 0x56110e4e8e00 2024-04-23T14:33:59.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.304+0000 7fc1dd200640 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4e8e00 0x56110e4f5e10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc1c401e000 tx=0x7fc1c4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae2b88289f7f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1c9600640 1 -- 172.21.15.59:0/316849228 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c402e070 con 0x56110e4e8e00 2024-04-23T14:33:59.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1c9600640 1 -- 172.21.15.59:0/316849228 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4014ba0 con 0x56110e4e8e00 2024-04-23T14:33:59.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1c9600640 1 -- 172.21.15.59:0/316849228 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4024040 con 0x56110e4e8e00 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 -- 172.21.15.59:0/316849228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4e8e00 msgr2=0x56110e4f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4e8e00 0x56110e4f5e10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc1c401e000 tx=0x7fc1c4012000 comp rx=0 tx=0).stop 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 -- 172.21.15.59:0/316849228 shutdown_connections 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e4e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4e8e00 0x56110e4f5e10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 --2- 172.21.15.59:0/316849228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4cbf20 0x56110e49a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 -- 172.21.15.59:0/316849228 >> 172.21.15.59:0/316849228 conn(0x56110e4cb7e0 msgr2=0x56110e4f9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 -- 172.21.15.59:0/316849228 shutdown_connections 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.305+0000 7fc1e3414740 1 -- 172.21.15.59:0/316849228 wait complete. 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.306+0000 7fc1e3414740 1 -- start start 2024-04-23T14:33:59.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.306+0000 7fc1e3414740 1 Processor -- start 2024-04-23T14:33:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.307+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.307+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.307+0000 7fc1e3414740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e49f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.307+0000 7fc1e3414740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d81530a0 con 0x56110e4cbf20 2024-04-23T14:33:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.307+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56110e4bc6c0 con 0x56110e4f9e50 2024-04-23T14:33:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.307+0000 7fc1e3414740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x56110e5610f0 con 0x56110e4e8e00 2024-04-23T14:33:59.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.307+0000 7fc1e3414740 10 client.0 _ll_register_callbacks cb 0x7fc1d0034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41742/0 (socket says 172.21.15.59:41742) 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 -- 172.21.15.59:0/1023395390 learned_addr learned my addr 172.21.15.59:0/1023395390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1de600640 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e49f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1dd200640 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 -- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 msgr2=0x56110e49efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 -- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 msgr2=0x56110e49f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e49f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.308+0000 7fc1ddc00640 1 -- 172.21.15.59:0/1023395390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110e0d70f0 con 0x56110e4cbf20 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1ddc00640 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4c9b90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc062340 tx=0x7fc1cc056050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bfdf005896d154 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1bea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1bea00640 1 -- 172.21.15.59:0/1023395390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0721f0 con 0x56110e4cbf20 2024-04-23T14:33:59.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1bea00640 1 -- 172.21.15.59:0/1023395390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc1cc057d30 con 0x56110e4cbf20 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c4014ba0 con 0x56110e4cbf20 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1bea00640 1 -- 172.21.15.59:0/1023395390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc1cc057f90 con 0x56110e4cbf20 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1e3414740 10 client.5943 fetch_fsmap learned FSMap version 25 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x56110e3ca810 con 0x56110e4cbf20 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1bea00640 1 -- 172.21.15.59:0/1023395390 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4014ba0 con 0x56110e4cbf20 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1bea00640 1 -- 172.21.15.59:0/1023395390 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fc1cc013070 con 0x56110e4cbf20 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.309+0000 7fc1e3414740 10 client.5943 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1e3414740 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1e3414740 20 client.5943 populate_metadata read hostname 'smithi059' 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1e3414740 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e509b00 0x56110e560190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1e3414740 4 client.5943 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56110e40ca80 con 0x56110e509b00 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1dd200640 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e509b00 0x56110e560190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:33:59.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1dd200640 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e509b00 0x56110e560190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4f3d82c76551b3 server_cookie=c32107e7ce9b180f in_seq=0 out_seq=0 2024-04-23T14:33:59.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.310+0000 7fc1bea00640 10 client.5943 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1bea00640 1 -- 172.21.15.59:0/1023395390 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x56110e40ca80 con 0x56110e509b00 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1bea00640 10 client.5943 handle_command_reply: tid=0 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 client.5943 shutdown 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 msgr2=0x56110e4c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4c9b90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc062340 tx=0x7fc1cc056050 comp rx=0 tx=0).stop 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 shutdown_connections 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x56110e509b00 0x56110e560190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56110e4e8e00 0x56110e49efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56110e4f9e50 0x56110e49f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 --2- 172.21.15.59:0/1023395390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56110e4cbf20 0x56110e4c9b90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1bea00640 0 client.5943 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 >> 172.21.15.59:0/1023395390 conn(0x56110e4cb7e0 msgr2=0x56110e4cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 shutdown_connections 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 1 -- 172.21.15.59:0/1023395390 wait complete. 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.311+0000 7fc1e3414740 20 client.5943 trim_cache size 0 max 16384 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stdout: "e9e224e8-3bf7-492a-a1fa-99ba1220dcd8": { 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "e9e224e8-3bf7-492a-a1fa-99ba1220dcd8", 2024-04-23T14:33:59.316 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-23T14:33:59.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:59.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:33:59.317 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:33:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1b806c940 msgr2=0x7fc1b806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1b806c940 0x7fc1b806ee00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80d4f30 tx=0x7fc1c4009040 comp rx=0 tx=0).stop 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80b3600 msgr2=0x7fc1d80b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80b3600 0x7fc1d80b8780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00d5f0 tx=0x7fc1cc00dab0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 shutdown_connections 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1b806c940 0x7fc1b806ee00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1d815ed20 0x7fc1d8151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1d80d13e0 0x7fc1d80b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 --2- 172.21.15.59:0/155701607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1d80b3600 0x7fc1d80b8780 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 >> 172.21.15.59:0/155701607 conn(0x7fc1d808bf90 msgr2=0x7fc1d806d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 shutdown_connections 2024-04-23T14:33:59.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:33:59.312+0000 7fc1df000640 1 -- 172.21.15.59:0/155701607 wait complete. 2024-04-23T14:33:59.331 INFO:tasks.cephfs.filesystem:scrub status for tag:e9e224e8-3bf7-492a-a1fa-99ba1220dcd8 - {'path': '/', 'tag': 'e9e224e8-3bf7-492a-a1fa-99ba1220dcd8', 'options': 'recursive,repair'} 2024-04-23T14:34:00.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:00.394+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-23T14:33:59.053115+0000) ==== 1078+0+0 (secure 0 0 0) 0x7fd8f4079410 con 0x7fd8fc15fdb0 2024-04-23T14:34:00.395 INFO:teuthology.orchestra.run.smithi059.stdout: cluster: 2024-04-23T14:34:00.395 INFO:teuthology.orchestra.run.smithi059.stdout: id: b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: health: HEALTH_OK 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: services: 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: mgr: y(active, since 2m), standbys: x 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: mds: 1/1 daemons up, 3 standby 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: data: 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: volumes: 1/1 healthy 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: pools: 3 pools, 97 pgs 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: objects: 24 objects, 579 KiB 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: usage: 220 MiB used, 720 GiB / 720 GiB avail 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: pgs: 97 active+clean 2024-04-23T14:34:00.396 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout: io: 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:33:59.053115+0000 mds.a [INF] scrub queued for path: / 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:33:59.053119+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:33:59.053125+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:33:59.054763+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:33:59.054771+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-23T14:34:00.397 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:33:59.054800+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-23T14:34:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:03.411+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-23T14:34:02.877944+0000) ==== 189+0+0 (secure 0 0 0) 0x7fd8f4079670 con 0x7fd8fc15fdb0 2024-04-23T14:34:03.413 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:34:02.877944+0000 mds.a [INF] scrub summary: idle 2024-04-23T14:34:07.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:07.435+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 9 ==== log(4 entries from seq 121 at 2024-04-23T14:34:06.543201+0000) ==== 1120+0+0 (secure 0 0 0) 0x7fd8f40779e0 con 0x7fd8fc15fdb0 2024-04-23T14:34:08.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:08.447+0000 7fd8f9600640 1 -- 172.21.15.59:0/3913791185 <== mon.0 v2:172.21.15.59:3300/0 10 ==== log(2 entries from seq 467 at 2024-04-23T14:34:07.423485+0000) ==== 558+0+0 (secure 0 0 0) 0x7fd8f407b5d0 con 0x7fd8fc15fdb0 2024-04-23T14:34:19.179 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:19.188 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:29.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:29.332 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-23T14:34:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.563+0000 7fbb09e00640 1 Processor -- start 2024-04-23T14:34:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.563+0000 7fbb09e00640 1 -- start start 2024-04-23T14:34:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.565+0000 7fbb09e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 0x7fbb040cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.565+0000 7fbb09e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb040d02a0 0x7fbb040d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.565+0000 7fbb09e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb040d23f0 con 0x7fbb040d14e0 2024-04-23T14:34:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.565+0000 7fbb09e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbb040d1950 con 0x7fbb040d02a0 2024-04-23T14:34:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.565+0000 7fbb09e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040d21a0 con 0x7fbb0415e870 2024-04-23T14:34:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.565+0000 7fbb09400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbb040d14e0 0x7fbb04162440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36330/0 (socket says 172.21.15.59:36330) 2024-04-23T14:34:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.565+0000 7fbb09400640 1 -- 172.21.15.59:0/3920199935 learned_addr learned my addr 172.21.15.59:0/3920199935 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03e00640 1 --2- 172.21.15.59:0/3920199935 >> v2:172.21.15.76:3300/0 conn(0x7fbb040d02a0 0x7fbb040d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 0x7fbb040cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388891507 0 0) 0x7fbb040d23f0 con 0x7fbb040d14e0 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaec003520 con 0x7fbb040d14e0 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3809657217 0 0) 0x7fbaec003520 con 0x7fbb040d14e0 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 msgr2=0x7fbb040cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 0x7fbb040cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 >> v2:172.21.15.76:3300/0 conn(0x7fbb040d02a0 msgr2=0x7fbb040d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 --2- 172.21.15.59:0/3920199935 >> v2:172.21.15.76:3300/0 conn(0x7fbb040d02a0 0x7fbb040d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb040d26d0 con 0x7fbb040d14e0 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fbafc0019f0 con 0x7fbb040d14e0 2024-04-23T14:34:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 >> v1:172.21.15.59:6789/0 conn(0x7fbb040d14e0 legacy=0x7fbb04162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbaec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 0x7fbaec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec004d40 0x7fbaec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040d1950 con 0x7fbaec0039b0 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040d21a0 con 0x7fbaec004d40 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.566+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec00b9c0 con 0x7fbb0415e870 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.567+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbaec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.567+0000 7fbb03e00640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 0x7fbaec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.567+0000 7fbb08a00640 1 -- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 msgr2=0x7fbaec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.567+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415e870 0x7fbaec004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.567+0000 7fbb08a00640 1 -- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec004d40 msgr2=0x7fbaec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.567+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec004d40 0x7fbaec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.567+0000 7fbb08a00640 1 -- 172.21.15.59:0/3920199935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec003520 con 0x7fbaec0039b0 2024-04-23T14:34:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.568+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbaec003dd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009ac0 tx=0x7fbaec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c848db322d3d5062 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8036070 con 0x7fbaec0039b0 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf8031400 con 0x7fbaec0039b0 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb03400640 1 -- 172.21.15.59:0/3920199935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8009690 con 0x7fbaec0039b0 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb09e00640 1 -- 172.21.15.59:0/3920199935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 msgr2=0x7fbaec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb09e00640 1 --2- 172.21.15.59:0/3920199935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbaec003dd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009ac0 tx=0x7fbaec005830 comp rx=0 tx=0).stop 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb09400640 1 -- 172.21.15.59:0/3920199935 reap_dead start 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb09e00640 1 -- 172.21.15.59:0/3920199935 shutdown_connections 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.569+0000 7fbb09e00640 1 -- 172.21.15.59:0/3920199935 >> 172.21.15.59:0/3920199935 conn(0x7fbb0408bf90 msgr2=0x7fbb040d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.570+0000 7fbb09e00640 1 -- 172.21.15.59:0/3920199935 shutdown_connections 2024-04-23T14:34:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.570+0000 7fbb09e00640 1 -- 172.21.15.59:0/3920199935 wait complete. 2024-04-23T14:34:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.571+0000 7fbb09e00640 1 Processor -- start 2024-04-23T14:34:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.572+0000 7fbb09e00640 1 -- start start 2024-04-23T14:34:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb09e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbb041f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb09e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0415aa60 0x7fbb0409e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb09e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415aee0 0x7fbb04160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb09e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040b8430 con 0x7fbaec0039b0 2024-04-23T14:34:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb09e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040d3000 con 0x7fbb0415aa60 2024-04-23T14:34:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb09e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040d2c90 con 0x7fbb0415aee0 2024-04-23T14:34:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbb041f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbb041f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40610/0 (socket says 172.21.15.59:40610) 2024-04-23T14:34:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 -- 172.21.15.59:0/1857440559 learned_addr learned my addr 172.21.15.59:0/1857440559 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0415aa60 0x7fbb0409e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 -- 172.21.15.59:0/1857440559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415aee0 msgr2=0x7fbb04160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415aee0 0x7fbb04160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 -- 172.21.15.59:0/1857440559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0415aa60 msgr2=0x7fbb0409e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0415aa60 0x7fbb0409e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.573+0000 7fbb08a00640 1 -- 172.21.15.59:0/1857440559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041f8c00 con 0x7fbaec0039b0 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.574+0000 7fbb08a00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbb041f86c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009ac0 tx=0x7fbaf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.574+0000 7fbb00c00640 1 -- 172.21.15.59:0/1857440559 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8040020 con 0x7fbaec0039b0 2024-04-23T14:34:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.574+0000 7fbb00c00640 1 -- 172.21.15.59:0/1857440559 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf8003bc0 con 0x7fbaec0039b0 2024-04-23T14:34:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.574+0000 7fbb00c00640 1 -- 172.21.15.59:0/1857440559 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8036040 con 0x7fbaec0039b0 2024-04-23T14:34:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.574+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb040cb9d0 con 0x7fbaec0039b0 2024-04-23T14:34:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.574+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb040cbf90 con 0x7fbaec0039b0 2024-04-23T14:34:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.577+0000 7fbb00c00640 1 -- 172.21.15.59:0/1857440559 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaf8043020 con 0x7fbaec0039b0 2024-04-23T14:34:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.578+0000 7fbb00c00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.578+0000 7fbb00c00640 1 -- 172.21.15.59:0/1857440559 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fbaf80abd00 con 0x7fbaec0039b0 2024-04-23T14:34:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.578+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.578+0000 7fbb0e39e740 1 Processor -- start 2024-04-23T14:34:29.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb0e39e740 1 -- start start 2024-04-23T14:34:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb040c9cb0 tx=0x7fbaf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad0591790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad05c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c17e0 0x560ad05965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560ad04fb1f0 con 0x560ad05c1f20 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ad04a1350 con 0x560ad05dee00 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.579+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560ad04f71b0 con 0x560ad05c17e0 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad05c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad05c12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40616/0 (socket says 172.21.15.59:40616) 2024-04-23T14:34:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 -- 172.21.15.59:0/4198117845 learned_addr learned my addr 172.21.15.59:0/4198117845 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb08a00640 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad0591790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb09400640 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c17e0 0x560ad05965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 -- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c17e0 msgr2=0x560ad05965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c17e0 0x560ad05965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 -- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 msgr2=0x560ad0591790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad0591790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb03e00640 1 -- 172.21.15.59:0/4198117845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad04f5f50 con 0x560ad05c1f20 2024-04-23T14:34:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb09400640 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c17e0 0x560ad05965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.580+0000 7fbb08a00640 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad0591790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.581+0000 7fbb03e00640 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad05c12a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0011c30 tx=0x7fbaf0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b2caab2b6723bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.581+0000 7fbaf5200640 1 -- 172.21.15.59:0/4198117845 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0015ac0 con 0x560ad05c1f20 2024-04-23T14:34:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.581+0000 7fbaf5200640 1 -- 172.21.15.59:0/4198117845 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0026070 con 0x560ad05c1f20 2024-04-23T14:34:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.581+0000 7fbaf5200640 1 -- 172.21.15.59:0/4198117845 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0015e60 con 0x560ad05c1f20 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 -- 172.21.15.59:0/4198117845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 msgr2=0x560ad05c12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad05c12a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0011c30 tx=0x7fbaf0014000 comp rx=0 tx=0).stop 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 -- 172.21.15.59:0/4198117845 shutdown_connections 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c17e0 0x560ad05965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad05c12a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/4198117845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad0591790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 -- 172.21.15.59:0/4198117845 >> 172.21.15.59:0/4198117845 conn(0x560ad0593a00 msgr2=0x560ad05dc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 -- 172.21.15.59:0/4198117845 shutdown_connections 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 -- 172.21.15.59:0/4198117845 wait complete. 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.582+0000 7fbb0e39e740 1 -- start start 2024-04-23T14:34:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.583+0000 7fbb0e39e740 1 Processor -- start 2024-04-23T14:34:29.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0597490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 0x560ad05ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 0x560ad05aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560ad0595bb0 con 0x560ad05c1f20 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ad04b4690 con 0x560ad05f0a30 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560ad0594fa0 con 0x560ad05dee00 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb0e39e740 10 client.0 _ll_register_callbacks cb 0x560ad04f0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0597490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0597490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40620/0 (socket says 172.21.15.59:40620) 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 -- 172.21.15.59:0/1705656944 learned_addr learned my addr 172.21.15.59:0/1705656944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 -- 172.21.15.59:0/1705656944 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 msgr2=0x560ad05ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 0x560ad05ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 -- 172.21.15.59:0/1705656944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 msgr2=0x560ad05aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 0x560ad05aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.584+0000 7fbb08a00640 1 -- 172.21.15.59:0/1705656944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad04b3920 con 0x560ad05c1f20 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbb08a00640 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0597490 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8082090 tx=0x7fbaf807a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a292efedaf42971a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbae9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:34:29.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbae9600640 1 -- 172.21.15.59:0/1705656944 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8079e40 con 0x560ad05c1f20 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbae9600640 1 -- 172.21.15.59:0/1705656944 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fbaf8082130 con 0x560ad05c1f20 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf0026070 con 0x560ad05c1f20 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbae9600640 1 -- 172.21.15.59:0/1705656944 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbaf80825c0 con 0x560ad05c1f20 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbb0e39e740 10 client.5955 fetch_fsmap learned FSMap version 25 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x560ad02d77d0 con 0x560ad05c1f20 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.585+0000 7fbae9600640 1 -- 172.21.15.59:0/1705656944 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0026070 con 0x560ad05c1f20 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbae9600640 1 -- 172.21.15.59:0/1705656944 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fbaf80a3b40 con 0x560ad05c1f20 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbb0e39e740 10 client.5955 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbb0e39e740 10 client.5955 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbb0e39e740 20 client.5955 populate_metadata read hostname 'smithi059' 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05ad110 0x560ad05aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbb0e39e740 4 client.5955 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad0508c10 con 0x560ad05ad110 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.586+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05ad110 0x560ad05aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.587+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05ad110 0x560ad05aca30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4880c926dc2a2998 server_cookie=c5d2e41d19b422ca in_seq=0 out_seq=0 2024-04-23T14:34:29.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.587+0000 7fbae9600640 10 client.5955 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.587+0000 7fbae9600640 1 -- 172.21.15.59:0/1705656944 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ad0508c10 con 0x560ad05ad110 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.587+0000 7fbae9600640 10 client.5955 handle_command_reply: tid=0 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.587+0000 7fbb0e39e740 1 client.5955 shutdown 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 msgr2=0x560ad0597490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0597490 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8082090 tx=0x7fbaf807a0e0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 shutdown_connections 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05ad110 0x560ad05aca30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 0x560ad05ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 0x560ad05aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/1705656944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0597490 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbae9600640 0 client.5955 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:34:29.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 >> 172.21.15.59:0/1705656944 conn(0x560ad05c17e0 msgr2=0x560ad0595180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 shutdown_connections 2024-04-23T14:34:29.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 1 -- 172.21.15.59:0/1705656944 wait complete. 2024-04-23T14:34:29.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.588+0000 7fbb0e39e740 20 client.5955 trim_cache size 0 max 16384 2024-04-23T14:34:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.607+0000 7fbb0e39e740 1 Processor -- start 2024-04-23T14:34:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.608+0000 7fbb0e39e740 1 -- start start 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.608+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c1f20 0x560ad05973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.608+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad05dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.608+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05f0a30 0x560ad05db720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.608+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04201620 con 0x560ad05f0a30 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.608+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb042006b0 con 0x560ad05dee00 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.608+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040c8aa0 con 0x560ad05c1f20 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05f0a30 0x560ad05db720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05f0a30 0x560ad05db720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40628/0 (socket says 172.21.15.59:40628) 2024-04-23T14:34:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 -- 172.21.15.59:0/2034287929 learned_addr learned my addr 172.21.15.59:0/2034287929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb08a00640 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c1f20 0x560ad05973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb03e00640 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad05dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 -- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c1f20 msgr2=0x560ad05973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c1f20 0x560ad05973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 -- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 msgr2=0x560ad05dd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad05dd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.609+0000 7fbb09400640 1 -- 172.21.15.59:0/2034287929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad020d3f0 con 0x560ad05f0a30 2024-04-23T14:34:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.610+0000 7fbb09400640 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05f0a30 0x560ad05db720 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbafc002030 tx=0x7fbafc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cb60b51548f77d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.610+0000 7fbaebe00640 1 -- 172.21.15.59:0/2034287929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007e40 con 0x560ad05f0a30 2024-04-23T14:34:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.610+0000 7fbaebe00640 1 -- 172.21.15.59:0/2034287929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc004b60 con 0x560ad05f0a30 2024-04-23T14:34:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.610+0000 7fbaebe00640 1 -- 172.21.15.59:0/2034287929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007110 con 0x560ad05f0a30 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.610+0000 7fbb0e39e740 1 -- 172.21.15.59:0/2034287929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05f0a30 msgr2=0x560ad05db720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.610+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05f0a30 0x560ad05db720 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbafc002030 tx=0x7fbafc00d520 comp rx=0 tx=0).stop 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 -- 172.21.15.59:0/2034287929 shutdown_connections 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05f0a30 0x560ad05db720 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05dee00 0x560ad05dd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/2034287929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05c1f20 0x560ad05973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 -- 172.21.15.59:0/2034287929 >> 172.21.15.59:0/2034287929 conn(0x560ad05c17e0 msgr2=0x560ad05ab100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 -- 172.21.15.59:0/2034287929 shutdown_connections 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 -- 172.21.15.59:0/2034287929 wait complete. 2024-04-23T14:34:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.611+0000 7fbb0e39e740 1 -- start start 2024-04-23T14:34:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.612+0000 7fbb0e39e740 1 Processor -- start 2024-04-23T14:34:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0656060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 0x560ad05df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb0e39e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 0x560ad05dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560ad0600a70 con 0x560ad05c1f20 2024-04-23T14:34:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb042000c0 con 0x560ad05f0a30 2024-04-23T14:34:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb0e39e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560ad05dbc60 con 0x560ad05dee00 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb0e39e740 10 client.0 _ll_register_callbacks cb 0x7fbaf8079830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0656060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0656060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40640/0 (socket says 172.21.15.59:40640) 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 -- 172.21.15.59:0/3260527073 learned_addr learned my addr 172.21.15.59:0/3260527073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 -- 172.21.15.59:0/3260527073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 msgr2=0x560ad05df520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 0x560ad05df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 -- 172.21.15.59:0/3260527073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 msgr2=0x560ad05dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 0x560ad05dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.613+0000 7fbb08a00640 1 -- 172.21.15.59:0/3260527073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad01cd0f0 con 0x560ad05c1f20 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbb08a00640 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0656060 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003fd0 tx=0x7fbaf80aa0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1060f4ebd95be2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbae9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:34:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbae9600640 1 -- 172.21.15.59:0/3260527073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8079e40 con 0x560ad05c1f20 2024-04-23T14:34:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbae9600640 1 -- 172.21.15.59:0/3260527073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fbaf8078e30 con 0x560ad05c1f20 2024-04-23T14:34:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc004b60 con 0x560ad05c1f20 2024-04-23T14:34:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbae9600640 1 -- 172.21.15.59:0/3260527073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbaf8009750 con 0x560ad05c1f20 2024-04-23T14:34:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbb0e39e740 10 client.5961 fetch_fsmap learned FSMap version 25 2024-04-23T14:34:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.614+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x560ad04c0810 con 0x560ad05c1f20 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbae9600640 1 -- 172.21.15.59:0/3260527073 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc004b60 con 0x560ad05c1f20 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbae9600640 1 -- 172.21.15.59:0/3260527073 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fbaf8075730 con 0x560ad05c1f20 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbb0e39e740 10 client.5961 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbb0e39e740 10 client.5961 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbb0e39e740 20 client.5961 populate_metadata read hostname 'smithi059' 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05973b0 0x560ad0590b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbb0e39e740 4 client.5961 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 --> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ad05d42f0 con 0x560ad05973b0 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.615+0000 7fbb03e00640 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05973b0 0x560ad0590b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb03e00640 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05973b0 0x560ad0590b20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0dda33b0ad34bb server_cookie=cad2428b38d7bdb6 in_seq=0 out_seq=0 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbae9600640 10 client.5961 ms_handle_connect on v2:172.21.15.59:6834/2804599006 2024-04-23T14:34:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbae9600640 1 -- 172.21.15.59:0/3260527073 <== mds.0 v2:172.21.15.59:6834/2804599006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560ad05d42f0 con 0x560ad05973b0 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbae9600640 10 client.5961 handle_command_reply: tid=0 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 client.5961 shutdown 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 msgr2=0x560ad0656060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0656060 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003fd0 tx=0x7fbaf80aa0b0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 shutdown_connections 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:6834/2804599006,v1:172.21.15.59:6835/2804599006] conn(0x560ad05973b0 0x560ad0590b20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560ad05dee00 0x560ad05df520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ad05f0a30 0x560ad05dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 --2- 172.21.15.59:0/3260527073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560ad05c1f20 0x560ad0656060 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbae9600640 0 client.5961 ms_handle_reset on v2:172.21.15.59:6834/2804599006 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 >> 172.21.15.59:0/3260527073 conn(0x560ad05c17e0 msgr2=0x560ad05da650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.616+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 shutdown_connections 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb0e39e740 1 -- 172.21.15.59:0/3260527073 wait complete. 2024-04-23T14:34:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb0e39e740 20 client.5961 trim_cache size 0 max 16384 2024-04-23T14:34:29.621 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:34:29.621 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:34:29.621 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:34:29.621 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 msgr2=0x7fbae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb040c9cb0 tx=0x7fbaf0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 msgr2=0x7fbb041f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbb041f86c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009ac0 tx=0x7fbaf8004730 comp rx=0 tx=0).stop 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 shutdown_connections 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0415aee0 0x7fbb04160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0415aa60 0x7fbb0409e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 --2- 172.21.15.59:0/1857440559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaec0039b0 0x7fbb041f86c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.617+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 >> 172.21.15.59:0/1857440559 conn(0x7fbb0408bf90 msgr2=0x7fbb040b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.618+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 shutdown_connections 2024-04-23T14:34:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.618+0000 7fbb09e00640 1 -- 172.21.15.59:0/1857440559 wait complete. 2024-04-23T14:34:29.635 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:34:29.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.950+0000 7fbc54c00640 1 Processor -- start 2024-04-23T14:34:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.950+0000 7fbc54c00640 1 -- start start 2024-04-23T14:34:29.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.951+0000 7fbc54c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc5006f3f0 0x7fbc500cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.951+0000 7fbc54c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc500b3870 con 0x7fbc5015b450 2024-04-23T14:34:29.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.951+0000 7fbc54c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc500b3b50 con 0x7fbc500b3080 2024-04-23T14:34:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.951+0000 7fbc54c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbc500b41a0 con 0x7fbc5006f3f0 2024-04-23T14:34:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.951+0000 7fbc4f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbc5015b450 0x7fbc500b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36338/0 (socket says 172.21.15.59:36338) 2024-04-23T14:34:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.952+0000 7fbc4f400640 1 -- 172.21.15.59:0/357058892 learned_addr learned my addr 172.21.15.59:0/357058892 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.952+0000 7fbc4ea00640 1 --2- 172.21.15.59:0/357058892 >> v2:172.21.15.59:3300/0 conn(0x7fbc5006f3f0 0x7fbc500cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.952+0000 7fbc4e000640 1 -- 172.21.15.59:0/357058892 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036511477 0 0) 0x7fbc500b3870 con 0x7fbc5015b450 2024-04-23T14:34:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.952+0000 7fbc4ea00640 1 -- 172.21.15.59:0/357058892 >> v1:172.21.15.76:6789/0 conn(0x7fbc500b3080 legacy=0x7fbc500ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.952+0000 7fbc4ea00640 1 -- 172.21.15.59:0/357058892 >> v1:172.21.15.59:6789/0 conn(0x7fbc5015b450 legacy=0x7fbc500b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.952+0000 7fbc4ea00640 1 -- 172.21.15.59:0/357058892 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc500b3e30 con 0x7fbc5006f3f0 2024-04-23T14:34:29.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.953+0000 7fbc4ea00640 1 --2- 172.21.15.59:0/357058892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc500cd050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbc500b4320 tx=0x7fbc3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9682cafc6fe7ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc4e000640 1 -- 172.21.15.59:0/357058892 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c037070 con 0x7fbc5006f3f0 2024-04-23T14:34:29.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc4e000640 1 -- 172.21.15.59:0/357058892 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3c032440 con 0x7fbc5006f3f0 2024-04-23T14:34:29.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc4e000640 1 -- 172.21.15.59:0/357058892 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c032810 con 0x7fbc5006f3f0 2024-04-23T14:34:29.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc54c00640 1 -- 172.21.15.59:0/357058892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 msgr2=0x7fbc500cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc54c00640 1 --2- 172.21.15.59:0/357058892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc500cd050 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbc500b4320 tx=0x7fbc3c02a270 comp rx=0 tx=0).stop 2024-04-23T14:34:29.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc54c00640 1 -- 172.21.15.59:0/357058892 shutdown_connections 2024-04-23T14:34:29.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc54c00640 1 --2- 172.21.15.59:0/357058892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc500cd050 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc54c00640 1 -- 172.21.15.59:0/357058892 >> 172.21.15.59:0/357058892 conn(0x7fbc500b21a0 msgr2=0x7fbc500b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:29.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc54c00640 1 -- 172.21.15.59:0/357058892 shutdown_connections 2024-04-23T14:34:29.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.954+0000 7fbc54c00640 1 -- 172.21.15.59:0/357058892 wait complete. 2024-04-23T14:34:29.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.956+0000 7fbc54c00640 1 Processor -- start 2024-04-23T14:34:29.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.956+0000 7fbc54c00640 1 -- start start 2024-04-23T14:34:29.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.957+0000 7fbc54c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc5009d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.957+0000 7fbc54c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc500b3080 0x7fbc5015d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.957+0000 7fbc54c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc5015b450 0x7fbc500c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.957+0000 7fbc54c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc500cd590 con 0x7fbc5006f3f0 2024-04-23T14:34:29.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.957+0000 7fbc54c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbc500b4ff0 con 0x7fbc5015b450 2024-04-23T14:34:29.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.957+0000 7fbc54c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbc500cd710 con 0x7fbc500b3080 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc5009d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc5009d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40666/0 (socket says 172.21.15.59:40666) 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 -- 172.21.15.59:0/617802409 learned_addr learned my addr 172.21.15.59:0/617802409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4ea00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc500b3080 0x7fbc5015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4fe00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc5015b450 0x7fbc500c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 -- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc500b3080 msgr2=0x7fbc5015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc500b3080 0x7fbc5015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 -- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc5015b450 msgr2=0x7fbc500c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc5015b450 0x7fbc500c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 -- 172.21.15.59:0/617802409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5009d7a0 con 0x7fbc5006f3f0 2024-04-23T14:34:29.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.958+0000 7fbc4f400640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc5009d260 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000c460 tx=0x7fbc4000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.959+0000 7fbc47400640 1 -- 172.21.15.59:0/617802409 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40018070 con 0x7fbc5006f3f0 2024-04-23T14:34:29.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.959+0000 7fbc47400640 1 -- 172.21.15.59:0/617802409 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc40007500 con 0x7fbc5006f3f0 2024-04-23T14:34:29.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.959+0000 7fbc47400640 1 -- 172.21.15.59:0/617802409 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40004a20 con 0x7fbc5006f3f0 2024-04-23T14:34:29.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.959+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5015c4f0 con 0x7fbc5006f3f0 2024-04-23T14:34:29.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.959+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5015ca70 con 0x7fbc5006f3f0 2024-04-23T14:34:29.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.960+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5004fd90 con 0x7fbc5006f3f0 2024-04-23T14:34:29.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.961+0000 7fbc47400640 1 -- 172.21.15.59:0/617802409 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc40004bc0 con 0x7fbc5006f3f0 2024-04-23T14:34:29.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.963+0000 7fbc47400640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbc3006c8f0 0x7fbc3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:29.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.963+0000 7fbc47400640 1 -- 172.21.15.59:0/617802409 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fbc4008afc0 con 0x7fbc5006f3f0 2024-04-23T14:34:29.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.963+0000 7fbc4ea00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbc3006c8f0 0x7fbc3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:29.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.964+0000 7fbc4ea00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbc3006c8f0 0x7fbc3006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c034000 tx=0x7fbc3c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:29.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:29.968+0000 7fbc47400640 1 -- 172.21.15.59:0/617802409 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc40059280 con 0x7fbc5006f3f0 2024-04-23T14:34:30.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.212+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc5006cbe0 con 0x7fbc5006f3f0 2024-04-23T14:34:30.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.213+0000 7fbc47400640 1 -- 172.21.15.59:0/617802409 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7fbc40058b80 con 0x7fbc5006f3f0 2024-04-23T14:34:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:30.214 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:34:30.215 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:34:30.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbc3006c8f0 msgr2=0x7fbc3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbc3006c8f0 0x7fbc3006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c034000 tx=0x7fbc3c02e000 comp rx=0 tx=0).stop 2024-04-23T14:34:30.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 msgr2=0x7fbc5009d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc5009d260 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000c460 tx=0x7fbc4000c920 comp rx=0 tx=0).stop 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 shutdown_connections 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbc3006c8f0 0x7fbc3006edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc500b3080 0x7fbc5015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc5015b450 0x7fbc500c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 --2- 172.21.15.59:0/617802409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc5006f3f0 0x7fbc5009d260 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 >> 172.21.15.59:0/617802409 conn(0x7fbc500b21a0 msgr2=0x7fbc5006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 shutdown_connections 2024-04-23T14:34:30.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.216+0000 7fbc54c00640 1 -- 172.21.15.59:0/617802409 wait complete. 2024-04-23T14:34:30.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-23T14:34:30.485 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:34:30.485 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:34:30.485 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-23T14:34:30.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.584+0000 7fed58f74f40 1 Processor -- start 2024-04-23T14:34:30.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.584+0000 7fed58f74f40 1 -- start start 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed58f74f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa3e41d0 0x55fdfa42f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed58f74f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4a3ac0 0x55fdfa4ba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed58f74f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa4bead0 0x55fdfa4c0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed58f74f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fdfa3e69c0 con 0x55fdfa4bead0 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed58f74f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fdfa38dcc0 con 0x55fdfa4a3ac0 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed58f74f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fdfa3e4b20 con 0x55fdfa3e41d0 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed56c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa4bead0 0x55fdfa4c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed56c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa4bead0 0x55fdfa4c0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59762/0 (socket says 172.21.15.59:59762) 2024-04-23T14:34:30.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.585+0000 7fed56c00640 1 -- 172.21.15.59:0/3838915171 learned_addr learned my addr 172.21.15.59:0/3838915171 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:30.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed56200640 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa3e41d0 0x55fdfa42f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed55800640 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4a3ac0 0x55fdfa4ba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed56c00640 1 -- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa3e41d0 msgr2=0x55fdfa42f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed56c00640 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa3e41d0 0x55fdfa42f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed56c00640 1 -- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4a3ac0 msgr2=0x55fdfa4ba2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed56c00640 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4a3ac0 0x55fdfa4ba2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed56c00640 1 -- 172.21.15.59:0/3838915171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfa4bae70 con 0x55fdfa4bead0 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed55800640 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4a3ac0 0x55fdfa4ba2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed56c00640 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa4bead0 0x55fdfa4c0740 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fed5000a4b0 tx=0x7fed50050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4292c81a5f005 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed4fe00640 1 -- 172.21.15.59:0/3838915171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50055030 con 0x55fdfa4bead0 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed4fe00640 1 -- 172.21.15.59:0/3838915171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed50051bc0 con 0x55fdfa4bead0 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.586+0000 7fed4fe00640 1 -- 172.21.15.59:0/3838915171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5005b430 con 0x55fdfa4bead0 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 -- 172.21.15.59:0/3838915171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa4bead0 msgr2=0x55fdfa4c0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa4bead0 0x55fdfa4c0740 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fed5000a4b0 tx=0x7fed50050000 comp rx=0 tx=0).stop 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 -- 172.21.15.59:0/3838915171 shutdown_connections 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa4bead0 0x55fdfa4c0740 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4a3ac0 0x55fdfa4ba2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 --2- 172.21.15.59:0/3838915171 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa3e41d0 0x55fdfa42f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 -- 172.21.15.59:0/3838915171 >> 172.21.15.59:0/3838915171 conn(0x55fdfa430cc0 msgr2=0x55fdfa413a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 -- 172.21.15.59:0/3838915171 shutdown_connections 2024-04-23T14:34:30.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.587+0000 7fed58f74f40 1 -- 172.21.15.59:0/3838915171 wait complete. 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.588+0000 7fed58f74f40 1 Processor -- start 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.588+0000 7fed58f74f40 1 -- start start 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed58f74f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa3e41d0 0x55fdfa49e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed58f74f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa4a3ac0 0x55fdfa4b9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed58f74f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4bead0 0x55fdfa4145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed58f74f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fdfa4babd0 con 0x55fdfa3e41d0 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed58f74f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fdfa4bb450 con 0x55fdfa4bead0 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed58f74f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fdfa4bb1b0 con 0x55fdfa4a3ac0 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed56200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa3e41d0 0x55fdfa49e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed56200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa3e41d0 0x55fdfa49e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59776/0 (socket says 172.21.15.59:59776) 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.589+0000 7fed56200640 1 -- 172.21.15.59:0/2424990393 learned_addr learned my addr 172.21.15.59:0/2424990393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed55800640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa4a3ac0 0x55fdfa4b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed56200640 1 -- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa4a3ac0 msgr2=0x55fdfa4b9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed56c00640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4bead0 0x55fdfa4145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed56200640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fdfa4a3ac0 0x55fdfa4b9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed56200640 1 -- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4bead0 msgr2=0x55fdfa4145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed56200640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fdfa4bead0 0x55fdfa4145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed56200640 1 -- 172.21.15.59:0/2424990393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfa49f570 con 0x55fdfa3e41d0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed56200640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa3e41d0 0x55fdfa49e570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fed4800bde0 tx=0x7fed4800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed4ea00640 1 -- 172.21.15.59:0/2424990393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48007af0 con 0x55fdfa3e41d0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed4ea00640 1 -- 172.21.15.59:0/2424990393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed48007c90 con 0x55fdfa3e41d0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed4ea00640 1 -- 172.21.15.59:0/2424990393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48014630 con 0x55fdfa3e41d0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.590+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fdfa49edd0 con 0x55fdfa3e41d0 2024-04-23T14:34:30.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.591+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdfa4a2cc0 con 0x55fdfa3e41d0 2024-04-23T14:34:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.593+0000 7fed4ea00640 1 -- 172.21.15.59:0/2424990393 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed480147d0 con 0x55fdfa3e41d0 2024-04-23T14:34:30.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.594+0000 7fed4ea00640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed3806b2d0 0x7fed3806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.594+0000 7fed4ea00640 1 -- 172.21.15.59:0/2424990393 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fed48089c90 con 0x55fdfa3e41d0 2024-04-23T14:34:30.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.594+0000 7fed55800640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed3806b2d0 0x7fed3806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.594+0000 7fed58f74f40 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fdfa49f0b0 0x55fdfa4b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:30.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.594+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) -- 0x55fdfa4b99a0 con 0x55fdfa49f0b0 2024-04-23T14:34:30.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.595+0000 7fed55800640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed3806b2d0 0x7fed3806d790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55fdfa4b4d60 tx=0x7fed440055b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.595+0000 7fed56c00640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fdfa49f0b0 0x55fdfa4b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:30.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.596+0000 7fed56c00640 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fdfa49f0b0 0x55fdfa4b5320 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:30.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed56c00640 1 -- 172.21.15.59:0/2424990393 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fed50051d70 con 0x55fdfa49f0b0 2024-04-23T14:34:30.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fdfa49f0b0 msgr2=0x55fdfa4b5320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fdfa49f0b0 0x55fdfa4b5320 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed3806b2d0 msgr2=0x7fed3806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed3806b2d0 0x7fed3806d790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55fdfa4b4d60 tx=0x7fed440055b0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa3e41d0 msgr2=0x55fdfa49e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 --2- 172.21.15.59:0/2424990393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fdfa3e41d0 0x55fdfa49e570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fed4800bde0 tx=0x7fed4800bed0 comp rx=0 tx=0).stop 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed56c00640 1 -- 172.21.15.59:0/2424990393 reap_dead start 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 shutdown_connections 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 >> 172.21.15.59:0/2424990393 conn(0x55fdfa430cc0 msgr2=0x55fdfa4c13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 shutdown_connections 2024-04-23T14:34:30.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:30.597+0000 7fed58f74f40 1 -- 172.21.15.59:0/2424990393 wait complete. 2024-04-23T14:34:30.603 INFO:teuthology.orchestra.run.smithi059.stdout:Yalphaparent_a 2024-04-23T14:34:30.603 DEBUG:teuthology.orchestra.run.smithi059:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "ancestors": [ 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1099511627776, 2024-04-23T14:34:30.691 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "alpha", 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1, 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "parent_a", 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 7 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 7, 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout: "old_pools": [] 2024-04-23T14:34:30.692 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:34:30.693 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:30.698 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:34:30.699 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:30.850 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:30.966 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:34:30.967 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:30.967 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:34:30.967 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:31.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.228+0000 7f6112800640 1 Processor -- start 2024-04-23T14:34:31.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.228+0000 7f6112800640 1 -- start start 2024-04-23T14:34:31.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.230+0000 7f6112800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f610c0cd020 0x7f610c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.230+0000 7f6112800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.230+0000 7f6112800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f610c0cd520 con 0x7f610c1454e0 2024-04-23T14:34:31.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.230+0000 7f6112800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f610c0cddc0 con 0x7f610c0cd020 2024-04-23T14:34:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.230+0000 7f6112800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f610c1462c0 con 0x7f610c0b3e40 2024-04-23T14:34:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.230+0000 7f610aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f610c0cd020 0x7f610c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.230+0000 7f610b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f610c1454e0 0x7f610c145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42794/0 (socket says 172.21.15.59:42794) 2024-04-23T14:34:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.231+0000 7f610b400640 1 -- 172.21.15.59:0/917882615 learned_addr learned my addr 172.21.15.59:0/917882615 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.231+0000 7f610be00640 1 --2- 172.21.15.59:0/917882615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.231+0000 7f610aa00640 1 -- 172.21.15.59:0/917882615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 msgr2=0x7f610c160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.231+0000 7f610aa00640 1 --2- 172.21.15.59:0/917882615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.231+0000 7f610aa00640 1 -- 172.21.15.59:0/917882615 >> v1:172.21.15.59:6789/0 conn(0x7f610c1454e0 legacy=0x7f610c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.231+0000 7f610aa00640 1 -- 172.21.15.59:0/917882615 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c0cd800 con 0x7f610c0cd020 2024-04-23T14:34:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.232+0000 7f610aa00640 1 --2- 172.21.15.59:0/917882615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c0cd020 0x7f610c0b3900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f60f80097a0 tx=0x7f60f802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b27df52e0ceb68f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.232+0000 7f610a000640 1 -- 172.21.15.59:0/917882615 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8037070 con 0x7f610c0cd020 2024-04-23T14:34:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.232+0000 7f610a000640 1 -- 172.21.15.59:0/917882615 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f802ae40 con 0x7f610c0cd020 2024-04-23T14:34:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.232+0000 7f610a000640 1 -- 172.21.15.59:0/917882615 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8032660 con 0x7f610c0cd020 2024-04-23T14:34:31.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 -- 172.21.15.59:0/917882615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c0cd020 msgr2=0x7f610c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 --2- 172.21.15.59:0/917882615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c0cd020 0x7f610c0b3900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f60f80097a0 tx=0x7f60f802a0f0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 -- 172.21.15.59:0/917882615 shutdown_connections 2024-04-23T14:34:31.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 --2- 172.21.15.59:0/917882615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 --2- 172.21.15.59:0/917882615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c0cd020 0x7f610c0b3900 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 -- 172.21.15.59:0/917882615 >> 172.21.15.59:0/917882615 conn(0x7f610c067ea0 msgr2=0x7f610c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:31.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 -- 172.21.15.59:0/917882615 shutdown_connections 2024-04-23T14:34:31.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.233+0000 7f6112800640 1 -- 172.21.15.59:0/917882615 wait complete. 2024-04-23T14:34:31.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.235+0000 7f6112800640 1 Processor -- start 2024-04-23T14:34:31.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.235+0000 7f6112800640 1 -- start start 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.236+0000 7f6112800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c14f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.236+0000 7f6112800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f610c0cd020 0x7f610c13ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.236+0000 7f6112800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c1454e0 0x7f610c15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.236+0000 7f6112800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f610c160e50 con 0x7f610c1454e0 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.236+0000 7f6112800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1603f0 con 0x7f610c0cd020 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.236+0000 7f6112800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f610c160fd0 con 0x7f610c0b3e40 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.236+0000 7f610be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c1454e0 0x7f610c15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c1454e0 0x7f610c15cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59788/0 (socket says 172.21.15.59:59788) 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 -- 172.21.15.59:0/3195974119 learned_addr learned my addr 172.21.15.59:0/3195974119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610b400640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c14f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 -- 172.21.15.59:0/3195974119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 msgr2=0x7f610c14f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c14f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 -- 172.21.15.59:0/3195974119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f610c0cd020 msgr2=0x7f610c13ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f610c0cd020 0x7f610c13ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 -- 172.21.15.59:0/3195974119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c15d310 con 0x7f610c1454e0 2024-04-23T14:34:31.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.237+0000 7f610be00640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c1454e0 0x7f610c15cdd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6104007e70 tx=0x7f610400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:31.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.238+0000 7f6103400640 1 -- 172.21.15.59:0/3195974119 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104014620 con 0x7f610c1454e0 2024-04-23T14:34:31.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.238+0000 7f6103400640 1 -- 172.21.15.59:0/3195974119 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6104019070 con 0x7f610c1454e0 2024-04-23T14:34:31.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.238+0000 7f6103400640 1 -- 172.21.15.59:0/3195974119 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104014a00 con 0x7f610c1454e0 2024-04-23T14:34:31.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.238+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c14fd30 con 0x7f610c1454e0 2024-04-23T14:34:31.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.238+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1502f0 con 0x7f610c1454e0 2024-04-23T14:34:31.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.239+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610c068a80 con 0x7f610c1454e0 2024-04-23T14:34:31.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.240+0000 7f6103400640 1 -- 172.21.15.59:0/3195974119 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6104014ba0 con 0x7f610c1454e0 2024-04-23T14:34:31.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.241+0000 7f6103400640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60ec06c920 0x7f60ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.241+0000 7f6103400640 1 -- 172.21.15.59:0/3195974119 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f610408c520 con 0x7f610c1454e0 2024-04-23T14:34:31.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.241+0000 7f610b400640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60ec06c920 0x7f60ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.242+0000 7f610b400640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60ec06c920 0x7f60ec06ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f60fc000f60 tx=0x7f60fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:31.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.246+0000 7f6103400640 1 -- 172.21.15.59:0/3195974119 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f610405a870 con 0x7f610c1454e0 2024-04-23T14:34:31.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.454+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f610c1609a0 con 0x7f610c1454e0 2024-04-23T14:34:31.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.457+0000 7f6103400640 1 -- 172.21.15.59:0/3195974119 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7f610405a170 con 0x7f610c1454e0 2024-04-23T14:34:31.459 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:31.459 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":46,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:08.435362+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:45.619774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:46.736010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60ec06c920 msgr2=0x7f60ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60ec06c920 0x7f60ec06ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f60fc000f60 tx=0x7f60fc009040 comp rx=0 tx=0).stop 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c1454e0 msgr2=0x7f610c15cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c1454e0 0x7f610c15cdd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6104007e70 tx=0x7f610400e4b0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 shutdown_connections 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f60ec06c920 0x7f60ec06ede0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f610c1454e0 0x7f610c15cdd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f610c0cd020 0x7f610c13ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 --2- 172.21.15.59:0/3195974119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f610c0b3e40 0x7f610c14f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 >> 172.21.15.59:0/3195974119 conn(0x7f610c067ea0 msgr2=0x7f610c0b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:31.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 shutdown_connections 2024-04-23T14:34:31.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.461+0000 7f6112800640 1 -- 172.21.15.59:0/3195974119 wait complete. 2024-04-23T14:34:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:45.619774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:46.736010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:31.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.696+0000 7faad0a00640 1 Processor -- start 2024-04-23T14:34:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.697+0000 7faad0a00640 1 -- start start 2024-04-23T14:34:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.698+0000 7faad0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 0x7faacc156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.698+0000 7faad0a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faacc15cab0 con 0x7faacc15d810 2024-04-23T14:34:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.698+0000 7faad0a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faacc160e40 con 0x7faacc0d0d10 2024-04-23T14:34:31.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.698+0000 7faad0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faacc15cd30 con 0x7faacc1567e0 2024-04-23T14:34:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.698+0000 7faac3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faacc15d810 0x7faacc0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42802/0 (socket says 172.21.15.59:42802) 2024-04-23T14:34:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.698+0000 7faac3e00640 1 -- 172.21.15.59:0/1734165600 learned_addr learned my addr 172.21.15.59:0/1734165600 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacb400640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 0x7faacc156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2658614892 0 0) 0x7faab4001110 con 0x7faacc15d810 2024-04-23T14:34:31.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faab0003520 con 0x7faacc15d810 2024-04-23T14:34:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1061073978 0 0) 0x7faab0003520 con 0x7faacc15d810 2024-04-23T14:34:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 msgr2=0x7faacc156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:31.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 0x7faacc156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 >> v1:172.21.15.76:6789/0 conn(0x7faacc0d0d10 legacy=0x7faacc160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc15d010 con 0x7faacc15d810 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7faab4001840 con 0x7faacc15d810 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 >> v1:172.21.15.59:6789/0 conn(0x7faacc15d810 legacy=0x7faacc0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faab00039e0 0x7faab0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 0x7faab0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faab00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faacc15cd30 con 0x7faab0004d40 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faab000ba70 con 0x7faab00039e0 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.699+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faab000bc70 con 0x7faacc1567e0 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faacbe00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faab00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faac3e00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 0x7faab0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faacb400640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faab00039e0 0x7faab0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faacbe00640 1 -- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 msgr2=0x7faab0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faacbe00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1567e0 0x7faab0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faacbe00640 1 -- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faab00039e0 msgr2=0x7faab0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:31.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faacbe00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faab00039e0 0x7faab0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.700+0000 7faacbe00640 1 -- 172.21.15.59:0/1734165600 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab0003520 con 0x7faab0004d40 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.702+0000 7faacbe00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faab00051c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faacc15a6a0 tx=0x7faab0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312c878380382c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.702+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4001a70 con 0x7faab0004d40 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.702+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac402ae10 con 0x7faab0004d40 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.702+0000 7faacaa00640 1 -- 172.21.15.59:0/1734165600 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4003bb0 con 0x7faab0004d40 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.702+0000 7faad0a00640 1 -- 172.21.15.59:0/1734165600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 msgr2=0x7faab00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.702+0000 7faad0a00640 1 --2- 172.21.15.59:0/1734165600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faab00051c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faacc15a6a0 tx=0x7faab0005830 comp rx=0 tx=0).stop 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.703+0000 7faacbe00640 1 -- 172.21.15.59:0/1734165600 reap_dead start 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.703+0000 7faad0a00640 1 -- 172.21.15.59:0/1734165600 shutdown_connections 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.703+0000 7faad0a00640 1 -- 172.21.15.59:0/1734165600 >> 172.21.15.59:0/1734165600 conn(0x7faacc06bdf0 msgr2=0x7faacc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.703+0000 7faad0a00640 1 -- 172.21.15.59:0/1734165600 shutdown_connections 2024-04-23T14:34:31.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.703+0000 7faad0a00640 1 -- 172.21.15.59:0/1734165600 wait complete. 2024-04-23T14:34:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.704+0000 7faad0a00640 1 Processor -- start 2024-04-23T14:34:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.705+0000 7faad0a00640 1 -- start start 2024-04-23T14:34:31.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faad0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faacc0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faad0a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faacc0b2820 0x7faacc1f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faad0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1551c0 0x7faacc099060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faad0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faacc156d40 con 0x7faab0004d40 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faad0a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faacc156bc0 con 0x7faacc0b2820 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faad0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faacc156ec0 con 0x7faacc1551c0 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faacc0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faacc0b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59798/0 (socket says 172.21.15.59:59798) 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 -- 172.21.15.59:0/2994557816 learned_addr learned my addr 172.21.15.59:0/2994557816 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faac3e00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faacc0b2820 0x7faacc1f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 -- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1551c0 msgr2=0x7faacc099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1551c0 0x7faacc099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 -- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faacc0b2820 msgr2=0x7faacc1f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faacc0b2820 0x7faacc1f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faacb400640 1 -- 172.21.15.59:0/2994557816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc0aff80 con 0x7faab0004d40 2024-04-23T14:34:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.706+0000 7faac3e00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faacc0b2820 0x7faacc1f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.707+0000 7faacb400640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faacc0b0fe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7faabc011600 tx=0x7faabc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.707+0000 7faac3400640 1 -- 172.21.15.59:0/2994557816 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc015870 con 0x7faab0004d40 2024-04-23T14:34:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.707+0000 7faac3400640 1 -- 172.21.15.59:0/2994557816 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faabc015a10 con 0x7faab0004d40 2024-04-23T14:34:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.707+0000 7faac3400640 1 -- 172.21.15.59:0/2994557816 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc015dc0 con 0x7faab0004d40 2024-04-23T14:34:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.707+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc0b0220 con 0x7faab0004d40 2024-04-23T14:34:31.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.707+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc0b07e0 con 0x7faab0004d40 2024-04-23T14:34:31.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.708+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faacc04fd90 con 0x7faab0004d40 2024-04-23T14:34:31.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.709+0000 7faac3400640 1 -- 172.21.15.59:0/2994557816 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faabc00f490 con 0x7faab0004d40 2024-04-23T14:34:31.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.710+0000 7faac3400640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa806c940 0x7faaa806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:31.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.710+0000 7faac3400640 1 -- 172.21.15.59:0/2994557816 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7faabc08c420 con 0x7faab0004d40 2024-04-23T14:34:31.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.711+0000 7faac3e00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa806c940 0x7faaa806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:31.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.711+0000 7faac3e00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa806c940 0x7faaa806ee00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faab4001050 tx=0x7faab4001600 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:31.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.715+0000 7faac3400640 1 -- 172.21.15.59:0/2994557816 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faabc019030 con 0x7faab0004d40 2024-04-23T14:34:31.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.918+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faacc0b5150 con 0x7faab0004d40 2024-04-23T14:34:31.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.920+0000 7faac3400640 1 -- 172.21.15.59:0/2994557816 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7faabc05a880 con 0x7faab0004d40 2024-04-23T14:34:31.921 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:34:31.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa806c940 msgr2=0x7faaa806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa806c940 0x7faaa806ee00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faab4001050 tx=0x7faab4001600 comp rx=0 tx=0).stop 2024-04-23T14:34:31.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 msgr2=0x7faacc0b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:31.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faacc0b0fe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7faabc011600 tx=0x7faabc011ac0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 shutdown_connections 2024-04-23T14:34:31.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa806c940 0x7faaa806ee00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faacc1551c0 0x7faacc099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faacc0b2820 0x7faacc1f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 --2- 172.21.15.59:0/2994557816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faab0004d40 0x7faacc0b0fe0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:31.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 >> 172.21.15.59:0/2994557816 conn(0x7faacc06bdf0 msgr2=0x7faacc15ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:31.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 shutdown_connections 2024-04-23T14:34:31.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:31.922+0000 7faad0a00640 1 -- 172.21.15.59:0/2994557816 wait complete. 2024-04-23T14:34:31.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:32.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.148+0000 7f2853200640 1 Processor -- start 2024-04-23T14:34:32.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.149+0000 7f2853200640 1 -- start start 2024-04-23T14:34:32.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.150+0000 7f2853200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f285415ed20 0x7f28540d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.150+0000 7f2853200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28540b3600 0x7f28540d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.150+0000 7f2853200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28540b5da0 con 0x7f28540b3f10 2024-04-23T14:34:32.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.150+0000 7f2853200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f285415f100 con 0x7f28540b3600 2024-04-23T14:34:32.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.150+0000 7f2853200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f28540d0a20 con 0x7f285415ed20 2024-04-23T14:34:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.151+0000 7f2851400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28540b3600 0x7f28540d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.151+0000 7f2851e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f285415ed20 0x7f28540d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.151+0000 7f2851400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28540b3600 0x7f28540d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:42834/0 (socket says 172.21.15.59:42834) 2024-04-23T14:34:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.151+0000 7f2851400640 1 -- 172.21.15.59:0/1633195696 learned_addr learned my addr 172.21.15.59:0/1633195696 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.151+0000 7f2851e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f285415ed20 0x7f28540d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:53746/0 (socket says 172.21.15.59:53746) 2024-04-23T14:34:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.152+0000 7f2851400640 1 -- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f285415ed20 msgr2=0x7f28540d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.152+0000 7f2851400640 1 --2- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f285415ed20 0x7f28540d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.152+0000 7f2851400640 1 -- 172.21.15.59:0/1633195696 >> v1:172.21.15.76:6789/0 conn(0x7f28540b3f10 legacy=0x7f28540b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.152+0000 7f2851400640 1 -- 172.21.15.59:0/1633195696 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28540d2ed0 con 0x7f28540b3600 2024-04-23T14:34:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.152+0000 7f2851e00640 1 --2- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f285415ed20 0x7f28540d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.153+0000 7f2851400640 1 --2- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3600 0x7f28540d13e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f283c009870 tx=0x7f283c02a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67cfb41b4b62f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2850a00640 1 -- 172.21.15.59:0/1633195696 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c037070 con 0x7f28540b3600 2024-04-23T14:34:32.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2850a00640 1 -- 172.21.15.59:0/1633195696 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c032440 con 0x7f28540b3600 2024-04-23T14:34:32.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2850a00640 1 -- 172.21.15.59:0/1633195696 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c032810 con 0x7f28540b3600 2024-04-23T14:34:32.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2853200640 1 -- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3600 msgr2=0x7f28540d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2853200640 1 --2- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3600 0x7f28540d13e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f283c009870 tx=0x7f283c02a140 comp rx=0 tx=0).stop 2024-04-23T14:34:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2853200640 1 -- 172.21.15.59:0/1633195696 shutdown_connections 2024-04-23T14:34:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2853200640 1 --2- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3600 0x7f28540d13e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2853200640 1 --2- 172.21.15.59:0/1633195696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f285415ed20 0x7f28540d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2853200640 1 -- 172.21.15.59:0/1633195696 >> 172.21.15.59:0/1633195696 conn(0x7f285408bf90 msgr2=0x7f28540d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.154+0000 7f2853200640 1 -- 172.21.15.59:0/1633195696 shutdown_connections 2024-04-23T14:34:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.155+0000 7f2853200640 1 -- 172.21.15.59:0/1633195696 wait complete. 2024-04-23T14:34:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.156+0000 7f2853200640 1 Processor -- start 2024-04-23T14:34:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.156+0000 7f2853200640 1 -- start start 2024-04-23T14:34:32.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.157+0000 7f2853200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28540b3600 0x7f285409e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.157+0000 7f2853200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3f10 0x7f28540a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.157+0000 7f2853200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f285415ed20 0x7f28540a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.157+0000 7f2853200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28540d0290 con 0x7f285415ed20 2024-04-23T14:34:32.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.157+0000 7f2853200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28540d3490 con 0x7f28540b3f10 2024-04-23T14:34:32.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.157+0000 7f2853200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f28540d0410 con 0x7f28540b3600 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f285415ed20 0x7f28540a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f285415ed20 0x7f28540a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59808/0 (socket says 172.21.15.59:59808) 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 -- 172.21.15.59:0/1401498577 learned_addr learned my addr 172.21.15.59:0/1401498577 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2851e00640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28540b3600 0x7f285409e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2851400640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3f10 0x7f28540a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 -- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28540b3600 msgr2=0x7f285409e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28540b3600 0x7f285409e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 -- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3f10 msgr2=0x7f28540a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3f10 0x7f28540a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.158+0000 7f2852800640 1 -- 172.21.15.59:0/1401498577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28540a1040 con 0x7f285415ed20 2024-04-23T14:34:32.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.159+0000 7f2852800640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f285415ed20 0x7f28540a0ad0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f284c001eb0 tx=0x7f284c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:32.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.159+0000 7f284a000640 1 -- 172.21.15.59:0/1401498577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c017070 con 0x7f285415ed20 2024-04-23T14:34:32.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.159+0000 7f284a000640 1 -- 172.21.15.59:0/1401498577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c0052d0 con 0x7f285415ed20 2024-04-23T14:34:32.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.159+0000 7f284a000640 1 -- 172.21.15.59:0/1401498577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c003810 con 0x7f285415ed20 2024-04-23T14:34:32.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.159+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28540a15c0 con 0x7f285415ed20 2024-04-23T14:34:32.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.159+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28540a1b80 con 0x7f285415ed20 2024-04-23T14:34:32.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.160+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285404fd90 con 0x7f285415ed20 2024-04-23T14:34:32.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.161+0000 7f284a000640 1 -- 172.21.15.59:0/1401498577 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f284c01a070 con 0x7f285415ed20 2024-04-23T14:34:32.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.162+0000 7f284a000640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f283006c8f0 0x7f283006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.162+0000 7f284a000640 1 -- 172.21.15.59:0/1401498577 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f284c08afe0 con 0x7f285415ed20 2024-04-23T14:34:32.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.163+0000 7f2851e00640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f283006c8f0 0x7f283006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.163+0000 7f2851e00640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f283006c8f0 0x7f283006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2844007440 tx=0x7f284400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:32.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.167+0000 7f284a000640 1 -- 172.21.15.59:0/1401498577 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f284c059330 con 0x7f285415ed20 2024-04-23T14:34:32.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.371+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f285415e040 con 0x7f285415ed20 2024-04-23T14:34:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.372+0000 7f284a000640 1 -- 172.21.15.59:0/1401498577 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f284c058c30 con 0x7f285415ed20 2024-04-23T14:34:32.373 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:34:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f283006c8f0 msgr2=0x7f283006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f283006c8f0 0x7f283006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2844007440 tx=0x7f284400b040 comp rx=0 tx=0).stop 2024-04-23T14:34:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f285415ed20 msgr2=0x7f28540a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f285415ed20 0x7f28540a0ad0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f284c001eb0 tx=0x7f284c00c890 comp rx=0 tx=0).stop 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 shutdown_connections 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f283006c8f0 0x7f283006edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f285415ed20 0x7f28540a0ad0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28540b3f10 0x7f28540a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 --2- 172.21.15.59:0/1401498577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28540b3600 0x7f285409e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 >> 172.21.15.59:0/1401498577 conn(0x7f285408bf90 msgr2=0x7f285406df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 shutdown_connections 2024-04-23T14:34:32.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.375+0000 7f2853200640 1 -- 172.21.15.59:0/1401498577 wait complete. 2024-04-23T14:34:32.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:32.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.618+0000 7efd8dc00640 1 Processor -- start 2024-04-23T14:34:32.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.618+0000 7efd8dc00640 1 -- start start 2024-04-23T14:34:32.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.619+0000 7efd8dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 0x7efd8809ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.619+0000 7efd8dc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efd8809b740 0x7efd88160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.619+0000 7efd8dc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd880ccfe0 con 0x7efd8809bba0 2024-04-23T14:34:32.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.619+0000 7efd8dc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7efd880cde90 con 0x7efd8809b740 2024-04-23T14:34:32.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.619+0000 7efd8dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7efd8809b320 con 0x7efd8809aa00 2024-04-23T14:34:32.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.620+0000 7efd8d200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efd8809bba0 0x7efd88160760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42804/0 (socket says 172.21.15.59:42804) 2024-04-23T14:34:32.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.620+0000 7efd8d200640 1 -- 172.21.15.59:0/2238344543 learned_addr learned my addr 172.21.15.59:0/2238344543 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:32.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.620+0000 7efd87400640 1 --2- 172.21.15.59:0/2238344543 >> v2:172.21.15.76:3300/0 conn(0x7efd8809b740 0x7efd88160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.620+0000 7efd87e00640 1 --2- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 0x7efd8809ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.621+0000 7efd86a00640 1 -- 172.21.15.59:0/2238344543 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630149728 0 0) 0x7efd880ccfe0 con 0x7efd8809bba0 2024-04-23T14:34:32.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.621+0000 7efd86a00640 1 -- 172.21.15.59:0/2238344543 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd6c003520 con 0x7efd8809bba0 2024-04-23T14:34:32.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.621+0000 7efd87400640 1 -- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 msgr2=0x7efd8809ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:32.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.621+0000 7efd87400640 1 --2- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 0x7efd8809ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.621+0000 7efd87400640 1 -- 172.21.15.59:0/2238344543 >> v1:172.21.15.59:6789/0 conn(0x7efd8809bba0 legacy=0x7efd88160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.621+0000 7efd86a00640 1 -- 172.21.15.59:0/2238344543 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 338038816 0 0) 0x7efd6c003520 con 0x7efd8809bba0 2024-04-23T14:34:32.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.621+0000 7efd87400640 1 -- 172.21.15.59:0/2238344543 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880cd280 con 0x7efd8809b740 2024-04-23T14:34:32.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.623+0000 7efd87400640 1 --2- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 0x7efd88160220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd700097a0 tx=0x7efd7002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf4ce017a3007ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:32.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.623+0000 7efd86a00640 1 -- 172.21.15.59:0/2238344543 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70037070 con 0x7efd8809b740 2024-04-23T14:34:32.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.623+0000 7efd86a00640 1 -- 172.21.15.59:0/2238344543 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7002ab10 con 0x7efd8809b740 2024-04-23T14:34:32.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.623+0000 7efd86a00640 1 -- 172.21.15.59:0/2238344543 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800019f0 con 0x7efd8809b740 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 -- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 msgr2=0x7efd88160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 --2- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 0x7efd88160220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd700097a0 tx=0x7efd7002a080 comp rx=0 tx=0).stop 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 -- 172.21.15.59:0/2238344543 shutdown_connections 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 --2- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 0x7efd88160220 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 --2- 172.21.15.59:0/2238344543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 0x7efd8809ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 -- 172.21.15.59:0/2238344543 >> 172.21.15.59:0/2238344543 conn(0x7efd880b3f30 msgr2=0x7efd880b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 -- 172.21.15.59:0/2238344543 shutdown_connections 2024-04-23T14:34:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.624+0000 7efd8dc00640 1 -- 172.21.15.59:0/2238344543 wait complete. 2024-04-23T14:34:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.626+0000 7efd8dc00640 1 Processor -- start 2024-04-23T14:34:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.626+0000 7efd8dc00640 1 -- start start 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 0x7efd881598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8dc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 0x7efd88157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd8809bba0 0x7efd88153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd880cdb20 con 0x7efd8809bba0 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8dc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd880cd840 con 0x7efd8809b740 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7efd880ce320 con 0x7efd8809aa00 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd8809bba0 0x7efd88153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd8809bba0 0x7efd88153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59814/0 (socket says 172.21.15.59:59814) 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.627+0000 7efd8d200640 1 -- 172.21.15.59:0/1500997960 learned_addr learned my addr 172.21.15.59:0/1500997960 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8d200640 1 -- 172.21.15.59:0/1500997960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 msgr2=0x7efd881598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8d200640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 0x7efd881598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8d200640 1 -- 172.21.15.59:0/1500997960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 msgr2=0x7efd88157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8d200640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 0x7efd88157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8d200640 1 -- 172.21.15.59:0/1500997960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd88159f30 con 0x7efd8809bba0 2024-04-23T14:34:32.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8d200640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd8809bba0 0x7efd88153730 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efd8000cc30 tx=0x7efd8000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:32.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd7fe00640 1 -- 172.21.15.59:0/1500997960 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80017070 con 0x7efd8809bba0 2024-04-23T14:34:32.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd7fe00640 1 -- 172.21.15.59:0/1500997960 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8000ce40 con 0x7efd8809bba0 2024-04-23T14:34:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd7fe00640 1 -- 172.21.15.59:0/1500997960 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80012640 con 0x7efd8809bba0 2024-04-23T14:34:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8815a210 con 0x7efd8809bba0 2024-04-23T14:34:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.628+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8815a7d0 con 0x7efd8809bba0 2024-04-23T14:34:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.630+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8804fd90 con 0x7efd8809bba0 2024-04-23T14:34:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.631+0000 7efd7fe00640 1 -- 172.21.15.59:0/1500997960 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd8001a070 con 0x7efd8809bba0 2024-04-23T14:34:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.632+0000 7efd7fe00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efd6406c8f0 0x7efd6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.632+0000 7efd7fe00640 1 -- 172.21.15.59:0/1500997960 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7efd8008bba0 con 0x7efd8809bba0 2024-04-23T14:34:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.632+0000 7efd87e00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efd6406c8f0 0x7efd6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.633+0000 7efd87e00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efd6406c8f0 0x7efd6406edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efd78006a90 tx=0x7efd78006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:32.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.636+0000 7efd7fe00640 1 -- 172.21.15.59:0/1500997960 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd80059f80 con 0x7efd8809bba0 2024-04-23T14:34:32.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.843+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd880b3d10 con 0x7efd8809bba0 2024-04-23T14:34:32.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.844+0000 7efd7fe00640 1 -- 172.21.15.59:0/1500997960 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7efd80059880 con 0x7efd8809bba0 2024-04-23T14:34:32.845 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:34:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efd6406c8f0 msgr2=0x7efd6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efd6406c8f0 0x7efd6406edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efd78006a90 tx=0x7efd78006a20 comp rx=0 tx=0).stop 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd8809bba0 msgr2=0x7efd88153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd8809bba0 0x7efd88153730 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efd8000cc30 tx=0x7efd8000a430 comp rx=0 tx=0).stop 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 shutdown_connections 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efd6406c8f0 0x7efd6406edb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd8809bba0 0x7efd88153730 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd8809b740 0x7efd88157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 --2- 172.21.15.59:0/1500997960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efd8809aa00 0x7efd881598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 >> 172.21.15.59:0/1500997960 conn(0x7efd880b3f30 msgr2=0x7efd880c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.846+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 shutdown_connections 2024-04-23T14:34:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:32.847+0000 7efd8dc00640 1 -- 172.21.15.59:0/1500997960 wait complete. 2024-04-23T14:34:32.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:34:33.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.077+0000 7f7db4200640 1 Processor -- start 2024-04-23T14:34:33.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.077+0000 7f7db4200640 1 -- start start 2024-04-23T14:34:33.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.079+0000 7f7db4200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7dac0cf410 0x7f7dac15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.079+0000 7f7db4200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac15dda0 con 0x7f7dac06ddf0 2024-04-23T14:34:33.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.079+0000 7f7db4200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac0cd880 con 0x7f7dac06e250 2024-04-23T14:34:33.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.079+0000 7f7db4200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7dac071ec0 con 0x7f7dac0cf410 2024-04-23T14:34:33.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.079+0000 7f7db0e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7dac06ddf0 0x7f7dac072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42810/0 (socket says 172.21.15.59:42810) 2024-04-23T14:34:33.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.079+0000 7f7db0e00640 1 -- 172.21.15.59:0/3622624410 learned_addr learned my addr 172.21.15.59:0/3622624410 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:33.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7db1800640 1 --2- 172.21.15.59:0/3622624410 >> v2:172.21.15.76:3300/0 conn(0x7f7dac0cf410 0x7f7dac15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417814367 0 0) 0x7f7d98001110 con 0x7f7dac06ddf0 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d94003520 con 0x7f7dac06ddf0 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 674967501 0 0) 0x7f7d94003520 con 0x7f7dac06ddf0 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 >> v2:172.21.15.76:3300/0 conn(0x7f7dac0cf410 msgr2=0x7f7dac15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 --2- 172.21.15.59:0/3622624410 >> v2:172.21.15.76:3300/0 conn(0x7f7dac0cf410 0x7f7dac15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 >> v1:172.21.15.76:6789/0 conn(0x7f7dac06e250 legacy=0x7f7dac0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac0cdb20 con 0x7f7dac06ddf0 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.080+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f7d98001840 con 0x7f7dac06ddf0 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 >> v1:172.21.15.59:6789/0 conn(0x7f7dac06ddf0 legacy=0x7f7dac072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7dabe00640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94003a00 0x7f7d94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7dabe00640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d94004800 0x7f7d94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7dabe00640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94005210 0x7f7d9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac071ec0 con 0x7f7d94005210 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9400c160 con 0x7f7d94004800 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9400c360 con 0x7f7d94003a00 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db2200640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94005210 0x7f7d9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db0e00640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d94004800 0x7f7d94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db1800640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94003a00 0x7f7d94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db2200640 1 -- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94003a00 msgr2=0x7f7d94003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db2200640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94003a00 0x7f7d94003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db2200640 1 -- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d94004800 msgr2=0x7f7d94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db2200640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d94004800 0x7f7d94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db2200640 1 -- 172.21.15.59:0/3622624410 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d94003520 con 0x7f7d94005210 2024-04-23T14:34:33.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db1800640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94003a00 0x7f7d94003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:33.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.081+0000 7f7db0e00640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d94004800 0x7f7d94004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:33.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.082+0000 7f7db2200640 1 -- 172.21.15.59:0/3622624410 reap_dead start 2024-04-23T14:34:33.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.083+0000 7f7db2200640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94005210 0x7f7d9400b9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7da4001e20 tx=0x7f7d94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08e86c2982bd1ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:33.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.083+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4001a70 con 0x7f7d94005210 2024-04-23T14:34:33.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.083+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da402ae40 con 0x7f7d94005210 2024-04-23T14:34:33.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.083+0000 7f7dabe00640 1 -- 172.21.15.59:0/3622624410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4003bb0 con 0x7f7d94005210 2024-04-23T14:34:33.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.083+0000 7f7db4200640 1 -- 172.21.15.59:0/3622624410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94005210 msgr2=0x7f7d9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.083+0000 7f7db4200640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94005210 0x7f7d9400b9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7da4001e20 tx=0x7f7d94005830 comp rx=0 tx=0).stop 2024-04-23T14:34:33.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.084+0000 7f7db4200640 1 -- 172.21.15.59:0/3622624410 shutdown_connections 2024-04-23T14:34:33.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.084+0000 7f7db4200640 1 --2- 172.21.15.59:0/3622624410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94005210 0x7f7d9400b9c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.084+0000 7f7db4200640 1 -- 172.21.15.59:0/3622624410 >> 172.21.15.59:0/3622624410 conn(0x7f7dac0b2130 msgr2=0x7f7dac0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:33.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.084+0000 7f7db4200640 1 -- 172.21.15.59:0/3622624410 shutdown_connections 2024-04-23T14:34:33.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.084+0000 7f7db4200640 1 -- 172.21.15.59:0/3622624410 wait complete. 2024-04-23T14:34:33.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.086+0000 7f7db4200640 1 Processor -- start 2024-04-23T14:34:33.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.086+0000 7f7db4200640 1 -- start start 2024-04-23T14:34:33.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db4200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94005210 0x7f7dac159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db4200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dac1e7090 0x7f7dac143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db4200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dac1573f0 0x7f7dac157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db4200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac15a6a0 con 0x7f7dac1573f0 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db4200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac0cde00 con 0x7f7dac1e7090 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db4200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac15a820 con 0x7f7d94005210 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dac1573f0 0x7f7dac157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dac1573f0 0x7f7dac157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59834/0 (socket says 172.21.15.59:59834) 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db2200640 1 -- 172.21.15.59:0/3193782484 learned_addr learned my addr 172.21.15.59:0/3193782484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db1800640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94005210 0x7f7dac159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.087+0000 7f7db0e00640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dac1e7090 0x7f7dac143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db2200640 1 -- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94005210 msgr2=0x7f7dac159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db2200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94005210 0x7f7dac159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db2200640 1 -- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dac1e7090 msgr2=0x7f7dac143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db2200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dac1e7090 0x7f7dac143f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db2200640 1 -- 172.21.15.59:0/3193782484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac1e8300 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db2200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dac1573f0 0x7f7dac157870 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7da402a8f0 tx=0x7f7da40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7da9600640 1 -- 172.21.15.59:0/3193782484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4001250 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7da9600640 1 -- 172.21.15.59:0/3193782484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da4005610 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7da9600640 1 -- 172.21.15.59:0/3193782484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4032440 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac1e85e0 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.088+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac0b01f0 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.089+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac068a80 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.091+0000 7f7da9600640 1 -- 172.21.15.59:0/3193782484 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7da403c070 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.091+0000 7f7da9600640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d8c06ca90 0x7f7d8c06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.092+0000 7f7da9600640 1 -- 172.21.15.59:0/3193782484 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f7da40ab2e0 con 0x7f7dac1573f0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.092+0000 7f7db1800640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d8c06ca90 0x7f7d8c06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:33.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.092+0000 7f7db1800640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d8c06ca90 0x7f7d8c06ef50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009aa0 tx=0x7f7da0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:33.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.096+0000 7f7da9600640 1 -- 172.21.15.59:0/3193782484 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7da4044060 con 0x7f7dac1573f0 2024-04-23T14:34:33.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.318+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7dac0700b0 con 0x7f7dac1573f0 2024-04-23T14:34:33.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.319+0000 7f7da9600640 1 -- 172.21.15.59:0/3193782484 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f7da40796c0 con 0x7f7dac1573f0 2024-04-23T14:34:33.320 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d8c06ca90 msgr2=0x7f7d8c06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d8c06ca90 0x7f7d8c06ef50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009aa0 tx=0x7f7da0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dac1573f0 msgr2=0x7f7dac157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dac1573f0 0x7f7dac157870 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7da402a8f0 tx=0x7f7da40014a0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 shutdown_connections 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d8c06ca90 0x7f7d8c06ef50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dac1573f0 0x7f7dac157870 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dac1e7090 0x7f7dac143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 --2- 172.21.15.59:0/3193782484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d94005210 0x7f7dac159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.322+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 >> 172.21.15.59:0/3193782484 conn(0x7f7dac0b2130 msgr2=0x7f7dac0724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.323+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 shutdown_connections 2024-04-23T14:34:33.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:33.323+0000 7f7db4200640 1 -- 172.21.15.59:0/3193782484 wait complete. 2024-04-23T14:34:33.341 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:34:33.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:34:33.421 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[50804]: fuse finished with error 0 and tester_r 0 2024-04-23T14:34:33.428 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:34:33.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.429 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:34:33.628 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:33.628 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:34:33.628 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:33.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:33.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:34:33.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.681 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:34:33.777 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:34:33.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:33.778 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:33.778 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:34:33.778 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:33.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:33.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:34:33.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.840 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:34:33.896 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:34:33.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:33.898 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:34:33.898 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:34:33.898 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:33.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:33.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:34:33.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:34:33.956 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:34:34.007 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:34:34.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:34:34.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:34.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.224+0000 7f15aa200640 1 Processor -- start 2024-04-23T14:34:34.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.225+0000 7f15aa200640 1 -- start start 2024-04-23T14:34:34.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.226+0000 7f15aa200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15a40d3260 0x7f15a40d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.226+0000 7f15aa200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a40cf520 con 0x7f15a415e9d0 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.226+0000 7f15aa200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a40cf800 con 0x7f15a40d02e0 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.226+0000 7f15aa200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f15a40cfae0 con 0x7f15a40d3260 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.226+0000 7f15a3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15a40d3260 0x7f15a40d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.226+0000 7f15a8e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f15a415e9d0 0x7f15a40d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42812/0 (socket says 172.21.15.59:42812) 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.226+0000 7f15a8e00640 1 -- 172.21.15.59:0/3278226317 learned_addr learned my addr 172.21.15.59:0/3278226317 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.227+0000 7f15a3e00640 1 -- 172.21.15.59:0/3278226317 >> v1:172.21.15.76:6789/0 conn(0x7f15a40d02e0 legacy=0x7f15a40d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.227+0000 7f15a3e00640 1 -- 172.21.15.59:0/3278226317 >> v1:172.21.15.59:6789/0 conn(0x7f15a415e9d0 legacy=0x7f15a40d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.227+0000 7f15a3e00640 1 -- 172.21.15.59:0/3278226317 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a40cfdc0 con 0x7f15a40d3260 2024-04-23T14:34:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.227+0000 7f15a3400640 1 -- 172.21.15.59:0/3278226317 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915868396 0 0) 0x7f15a40cf520 con 0x7f15a415e9d0 2024-04-23T14:34:34.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.228+0000 7f15a3e00640 1 --2- 172.21.15.59:0/3278226317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a40d3260 0x7f15a40d0f70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f15900097a0 tx=0x7f159002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13997d78b59385bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.228+0000 7f15a3400640 1 -- 172.21.15.59:0/3278226317 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590037070 con 0x7f15a40d3260 2024-04-23T14:34:34.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.228+0000 7f15a3400640 1 -- 172.21.15.59:0/3278226317 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1590032440 con 0x7f15a40d3260 2024-04-23T14:34:34.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.228+0000 7f15a3400640 1 -- 172.21.15.59:0/3278226317 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590032810 con 0x7f15a40d3260 2024-04-23T14:34:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.229+0000 7f15aa200640 1 -- 172.21.15.59:0/3278226317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a40d3260 msgr2=0x7f15a40d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.229+0000 7f15aa200640 1 --2- 172.21.15.59:0/3278226317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a40d3260 0x7f15a40d0f70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f15900097a0 tx=0x7f159002a1e0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.229+0000 7f15aa200640 1 -- 172.21.15.59:0/3278226317 shutdown_connections 2024-04-23T14:34:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.229+0000 7f15aa200640 1 --2- 172.21.15.59:0/3278226317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a40d3260 0x7f15a40d0f70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.229+0000 7f15aa200640 1 -- 172.21.15.59:0/3278226317 >> 172.21.15.59:0/3278226317 conn(0x7f15a408bf90 msgr2=0x7f15a40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.229+0000 7f15aa200640 1 -- 172.21.15.59:0/3278226317 shutdown_connections 2024-04-23T14:34:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.229+0000 7f15aa200640 1 -- 172.21.15.59:0/3278226317 wait complete. 2024-04-23T14:34:34.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.231+0000 7f15aa200640 1 Processor -- start 2024-04-23T14:34:34.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.231+0000 7f15aa200640 1 -- start start 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15aa200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a40d02e0 0x7f15a415c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15aa200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15a40d3260 0x7f15a415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15aa200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a4156d10 0x7f15a40b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15aa200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41697e0 con 0x7f15a4156d10 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15aa200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4169660 con 0x7f15a40d02e0 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15aa200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f15a4169960 con 0x7f15a40d3260 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15a9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a4156d10 0x7f15a40b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15a9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a4156d10 0x7f15a40b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59846/0 (socket says 172.21.15.59:59846) 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.232+0000 7f15a9800640 1 -- 172.21.15.59:0/2138215793 learned_addr learned my addr 172.21.15.59:0/2138215793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a8e00640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a40d02e0 0x7f15a415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a3e00640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15a40d3260 0x7f15a415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a9800640 1 -- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15a40d3260 msgr2=0x7f15a415a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a9800640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15a40d3260 0x7f15a415a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a9800640 1 -- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a40d02e0 msgr2=0x7f15a415c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a9800640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a40d02e0 0x7f15a415c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a9800640 1 -- 172.21.15.59:0/2138215793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a415afb0 con 0x7f15a4156d10 2024-04-23T14:34:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a9800640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a4156d10 0x7f15a40b6a20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c730 tx=0x7f159c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.233+0000 7f15a0c00640 1 -- 172.21.15.59:0/2138215793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c016020 con 0x7f15a4156d10 2024-04-23T14:34:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.234+0000 7f15a0c00640 1 -- 172.21.15.59:0/2138215793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c00aa50 con 0x7f15a4156d10 2024-04-23T14:34:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.234+0000 7f15a0c00640 1 -- 172.21.15.59:0/2138215793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c00ce40 con 0x7f15a4156d10 2024-04-23T14:34:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.234+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a415b290 con 0x7f15a4156d10 2024-04-23T14:34:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.234+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a415b850 con 0x7f15a4156d10 2024-04-23T14:34:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.235+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a404fd90 con 0x7f15a4156d10 2024-04-23T14:34:34.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.236+0000 7f15a0c00640 1 -- 172.21.15.59:0/2138215793 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f159c019020 con 0x7f15a4156d10 2024-04-23T14:34:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.237+0000 7f15a0c00640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f158406c8f0 0x7f158406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.237+0000 7f15a0c00640 1 -- 172.21.15.59:0/2138215793 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f159c08bb70 con 0x7f15a4156d10 2024-04-23T14:34:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.237+0000 7f15a8e00640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f158406c8f0 0x7f158406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.238+0000 7f15a8e00640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f158406c8f0 0x7f158406edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1598000f60 tx=0x7f1598007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.241+0000 7f15a0c00640 1 -- 172.21.15.59:0/2138215793 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f159c020030 con 0x7f15a4156d10 2024-04-23T14:34:34.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.486+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15a41570f0 con 0x7f15a4156d10 2024-04-23T14:34:34.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.488+0000 7f15a0c00640 1 -- 172.21.15.59:0/2138215793 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7f159c0212b0 con 0x7f15a4156d10 2024-04-23T14:34:34.489 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:34.489 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:34:34.489 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:34:34.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f158406c8f0 msgr2=0x7f158406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f158406c8f0 0x7f158406edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1598000f60 tx=0x7f1598007c00 comp rx=0 tx=0).stop 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a4156d10 msgr2=0x7f15a40b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a4156d10 0x7f15a40b6a20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c730 tx=0x7f159c00cbf0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 shutdown_connections 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f158406c8f0 0x7f158406edb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15a40d3260 0x7f15a415a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15a4156d10 0x7f15a40b6a20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 --2- 172.21.15.59:0/2138215793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a40d02e0 0x7f15a415c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 >> 172.21.15.59:0/2138215793 conn(0x7f15a408bf90 msgr2=0x7f15a40b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 shutdown_connections 2024-04-23T14:34:34.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.490+0000 7f15aa200640 1 -- 172.21.15.59:0/2138215793 wait complete. 2024-04-23T14:34:34.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:34.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.713+0000 7fefc5200640 1 Processor -- start 2024-04-23T14:34:34.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.714+0000 7fefc5200640 1 -- start start 2024-04-23T14:34:34.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.715+0000 7fefc5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefc015c9d0 0x7fefc009b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.715+0000 7fefc5200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc00af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.715+0000 7fefc5200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc00b35f0 con 0x7fefc00d09e0 2024-04-23T14:34:34.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.715+0000 7fefc5200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fefc00cd2d0 con 0x7fefc015c9d0 2024-04-23T14:34:34.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.715+0000 7fefc5200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefc00b3890 con 0x7fefc00b2e20 2024-04-23T14:34:34.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefbf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefc015c9d0 0x7fefc009b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefbf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefc015c9d0 0x7fefc009b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59848/0 (socket says 172.21.15.59:59848) 2024-04-23T14:34:34.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefbf400640 1 -- 172.21.15.59:0/1051467202 learned_addr learned my addr 172.21.15.59:0/1051467202 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:34.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefbf400640 1 -- 172.21.15.59:0/1051467202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 msgr2=0x7fefc00af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:34.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefb7e00640 1 --2- 172.21.15.59:0/1051467202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc00af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefbf400640 1 --2- 172.21.15.59:0/1051467202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc00af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefbf400640 1 -- 172.21.15.59:0/1051467202 >> v1:172.21.15.76:6789/0 conn(0x7fefc00d09e0 legacy=0x7fefc00d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:34.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.716+0000 7fefbf400640 1 -- 172.21.15.59:0/1051467202 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc00b3b70 con 0x7fefc015c9d0 2024-04-23T14:34:34.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.717+0000 7fefbf400640 1 --2- 172.21.15.59:0/1051467202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc015c9d0 0x7fefc009b410 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009870 tx=0x7fefb002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f265f28dbc9a91d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefbea00640 1 -- 172.21.15.59:0/1051467202 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0037070 con 0x7fefc015c9d0 2024-04-23T14:34:34.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefbea00640 1 -- 172.21.15.59:0/1051467202 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb0032440 con 0x7fefc015c9d0 2024-04-23T14:34:34.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefbea00640 1 -- 172.21.15.59:0/1051467202 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0032810 con 0x7fefc015c9d0 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefc5200640 1 -- 172.21.15.59:0/1051467202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc015c9d0 msgr2=0x7fefc009b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefc5200640 1 --2- 172.21.15.59:0/1051467202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc015c9d0 0x7fefc009b410 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009870 tx=0x7fefb002a230 comp rx=0 tx=0).stop 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefc5200640 1 -- 172.21.15.59:0/1051467202 shutdown_connections 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefc5200640 1 --2- 172.21.15.59:0/1051467202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc00af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefc5200640 1 --2- 172.21.15.59:0/1051467202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc015c9d0 0x7fefc009b410 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.718+0000 7fefc5200640 1 -- 172.21.15.59:0/1051467202 >> 172.21.15.59:0/1051467202 conn(0x7fefc00b3ef0 msgr2=0x7fefc00b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.719+0000 7fefc5200640 1 -- 172.21.15.59:0/1051467202 shutdown_connections 2024-04-23T14:34:34.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.719+0000 7fefc5200640 1 -- 172.21.15.59:0/1051467202 wait complete. 2024-04-23T14:34:34.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.720+0000 7fefc5200640 1 Processor -- start 2024-04-23T14:34:34.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.720+0000 7fefc5200640 1 -- start start 2024-04-23T14:34:34.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.721+0000 7fefc5200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc014e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.721+0000 7fefc5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc00d09e0 0x7fefc015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.721+0000 7fefc5200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc015c9d0 0x7fefc015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.721+0000 7fefc5200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00b3280 con 0x7fefc00d09e0 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.721+0000 7fefc5200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00ccef0 con 0x7fefc015c9d0 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.721+0000 7fefc5200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefc015cdb0 con 0x7fefc00b2e20 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc00d09e0 0x7fefc015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc00d09e0 0x7fefc015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59850/0 (socket says 172.21.15.59:59850) 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 -- 172.21.15.59:0/2933990045 learned_addr learned my addr 172.21.15.59:0/2933990045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefbf400640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefbfe00640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc015c9d0 0x7fefc015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 -- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 msgr2=0x7fefc014e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc014e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 -- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc015c9d0 msgr2=0x7fefc015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc015c9d0 0x7fefc015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefb7e00640 1 -- 172.21.15.59:0/2933990045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc015d160 con 0x7fefc00d09e0 2024-04-23T14:34:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.722+0000 7fefbfe00640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc015c9d0 0x7fefc015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.723+0000 7fefb7e00640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc00d09e0 0x7fefc015abf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fefa800c7f0 tx=0x7fefa800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.723+0000 7fefb7400640 1 -- 172.21.15.59:0/2933990045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8015070 con 0x7fefc00d09e0 2024-04-23T14:34:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.723+0000 7fefb7400640 1 -- 172.21.15.59:0/2933990045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa8004240 con 0x7fefc00d09e0 2024-04-23T14:34:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.723+0000 7fefb7400640 1 -- 172.21.15.59:0/2933990045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8007620 con 0x7fefc00d09e0 2024-04-23T14:34:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.723+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc015d440 con 0x7fefc00d09e0 2024-04-23T14:34:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.723+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc015da00 con 0x7fefc00d09e0 2024-04-23T14:34:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.724+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc004fd90 con 0x7fefc00d09e0 2024-04-23T14:34:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.725+0000 7fefb7400640 1 -- 172.21.15.59:0/2933990045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefa80079d0 con 0x7fefc00d09e0 2024-04-23T14:34:34.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.726+0000 7fefb7400640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef9c06c8f0 0x7fef9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:34.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.726+0000 7fefb7400640 1 -- 172.21.15.59:0/2933990045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fefa808bc60 con 0x7fefc00d09e0 2024-04-23T14:34:34.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.727+0000 7fefbf400640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef9c06c8f0 0x7fef9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.727+0000 7fefbf400640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef9c06c8f0 0x7fef9c06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fefb0034000 tx=0x7fefb002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:34.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.731+0000 7fefb7400640 1 -- 172.21.15.59:0/2933990045 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefa805a040 con 0x7fefc00d09e0 2024-04-23T14:34:34.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.936+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefc00d07c0 con 0x7fefc00d09e0 2024-04-23T14:34:34.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.938+0000 7fefb7400640 1 -- 172.21.15.59:0/2933990045 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7fefa8059940 con 0x7fefc00d09e0 2024-04-23T14:34:34.940 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:34.940 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":46,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:08.435362+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:45.619774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:46.736010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef9c06c8f0 msgr2=0x7fef9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef9c06c8f0 0x7fef9c06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fefb0034000 tx=0x7fefb002e000 comp rx=0 tx=0).stop 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc00d09e0 msgr2=0x7fefc015abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc00d09e0 0x7fefc015abf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fefa800c7f0 tx=0x7fefa800ccb0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 shutdown_connections 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fef9c06c8f0 0x7fef9c06edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc015c9d0 0x7fefc015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefc00d09e0 0x7fefc015abf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 --2- 172.21.15.59:0/2933990045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefc00b2e20 0x7fefc014e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 >> 172.21.15.59:0/2933990045 conn(0x7fefc00b3ef0 msgr2=0x7fefc00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 shutdown_connections 2024-04-23T14:34:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:34.941+0000 7fefc5200640 1 -- 172.21.15.59:0/2933990045 wait complete. 2024-04-23T14:34:34.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:33:45.619774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:33:46.736010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:34.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:35.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.175+0000 7fe9ee400640 1 Processor -- start 2024-04-23T14:34:35.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.175+0000 7fe9ee400640 1 -- start start 2024-04-23T14:34:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.177+0000 7fe9ee400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e806f6f0 0x7fe9e80aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.177+0000 7fe9ee400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9e8071d70 0x7fe9e8072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.177+0000 7fe9ee400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9e80adc80 con 0x7fe9e815e790 2024-04-23T14:34:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.177+0000 7fe9ee400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe9e80aef40 con 0x7fe9e8071d70 2024-04-23T14:34:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.177+0000 7fe9ee400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e80afd80 con 0x7fe9e806f6f0 2024-04-23T14:34:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.177+0000 7fe9e7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe9e815e790 0x7fe9e8162e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42814/0 (socket says 172.21.15.59:42814) 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.177+0000 7fe9e7400640 1 -- 172.21.15.59:0/1017317199 learned_addr learned my addr 172.21.15.59:0/1017317199 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9e7e00640 1 --2- 172.21.15.59:0/1017317199 >> v2:172.21.15.59:3300/0 conn(0x7fe9e8071d70 0x7fe9e8072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9dfe00640 1 --2- 172.21.15.59:0/1017317199 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e806f6f0 0x7fe9e80aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9e6a00640 1 -- 172.21.15.59:0/1017317199 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2624343043 0 0) 0x7fe9e80adc80 con 0x7fe9e815e790 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9e6a00640 1 -- 172.21.15.59:0/1017317199 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9cc003520 con 0x7fe9e815e790 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9e7e00640 1 -- 172.21.15.59:0/1017317199 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e806f6f0 msgr2=0x7fe9e80aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9e7e00640 1 --2- 172.21.15.59:0/1017317199 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e806f6f0 0x7fe9e80aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9e7e00640 1 -- 172.21.15.59:0/1017317199 >> v1:172.21.15.59:6789/0 conn(0x7fe9e815e790 legacy=0x7fe9e8162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.178+0000 7fe9e7e00640 1 -- 172.21.15.59:0/1017317199 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e80adf20 con 0x7fe9e8071d70 2024-04-23T14:34:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.180+0000 7fe9e7e00640 1 --2- 172.21.15.59:0/1017317199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8071d70 0x7fe9e8072200 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009d90 tx=0x7fe9e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee553cb602b87537 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.180+0000 7fe9e6a00640 1 -- 172.21.15.59:0/1017317199 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0007740 con 0x7fe9e8071d70 2024-04-23T14:34:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.180+0000 7fe9e6a00640 1 -- 172.21.15.59:0/1017317199 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e00078e0 con 0x7fe9e8071d70 2024-04-23T14:34:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.180+0000 7fe9e6a00640 1 -- 172.21.15.59:0/1017317199 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0007cb0 con 0x7fe9e8071d70 2024-04-23T14:34:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 -- 172.21.15.59:0/1017317199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8071d70 msgr2=0x7fe9e8072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 --2- 172.21.15.59:0/1017317199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8071d70 0x7fe9e8072200 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009d90 tx=0x7fe9e002a660 comp rx=0 tx=0).stop 2024-04-23T14:34:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 -- 172.21.15.59:0/1017317199 shutdown_connections 2024-04-23T14:34:35.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 --2- 172.21.15.59:0/1017317199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8071d70 0x7fe9e8072200 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 --2- 172.21.15.59:0/1017317199 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e806f6f0 0x7fe9e80aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 -- 172.21.15.59:0/1017317199 >> 172.21.15.59:0/1017317199 conn(0x7fe9e8067d90 msgr2=0x7fe9e806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 -- 172.21.15.59:0/1017317199 shutdown_connections 2024-04-23T14:34:35.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.181+0000 7fe9ee400640 1 -- 172.21.15.59:0/1017317199 wait complete. 2024-04-23T14:34:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.183+0000 7fe9ee400640 1 Processor -- start 2024-04-23T14:34:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.183+0000 7fe9ee400640 1 -- start start 2024-04-23T14:34:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.184+0000 7fe9ee400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e806f6f0 0x7fe9e8159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.184+0000 7fe9ee400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e8071d70 0x7fe9e81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.184+0000 7fe9ee400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e815e790 0x7fe9e815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.184+0000 7fe9ee400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e806fbb0 con 0x7fe9e815e790 2024-04-23T14:34:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.184+0000 7fe9ee400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e80ae4e0 con 0x7fe9e806f6f0 2024-04-23T14:34:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.184+0000 7fe9ee400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e815eb70 con 0x7fe9e8071d70 2024-04-23T14:34:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.184+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e815e790 0x7fe9e815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e815e790 0x7fe9e815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59864/0 (socket says 172.21.15.59:59864) 2024-04-23T14:34:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 -- 172.21.15.59:0/840335703 learned_addr learned my addr 172.21.15.59:0/840335703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9dfe00640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e8071d70 0x7fe9e81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e806f6f0 0x7fe9e8159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 -- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e8071d70 msgr2=0x7fe9e81597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e8071d70 0x7fe9e81597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 -- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e806f6f0 msgr2=0x7fe9e8159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e806f6f0 0x7fe9e8159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 -- 172.21.15.59:0/840335703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e81605a0 con 0x7fe9e815e790 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9dfe00640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e8071d70 0x7fe9e81597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e806f6f0 0x7fe9e8159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.185+0000 7fe9e7e00640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e815e790 0x7fe9e815c220 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009eb0 tx=0x7fe9e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.186+0000 7fe9df400640 1 -- 172.21.15.59:0/840335703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0040070 con 0x7fe9e815e790 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.186+0000 7fe9df400640 1 -- 172.21.15.59:0/840335703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e0031bc0 con 0x7fe9e815e790 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.186+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e8160880 con 0x7fe9e815e790 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.186+0000 7fe9df400640 1 -- 172.21.15.59:0/840335703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0009450 con 0x7fe9e815e790 2024-04-23T14:34:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.186+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e8160e40 con 0x7fe9e815e790 2024-04-23T14:34:35.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.187+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e8068a80 con 0x7fe9e815e790 2024-04-23T14:34:35.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.188+0000 7fe9df400640 1 -- 172.21.15.59:0/840335703 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9e0039040 con 0x7fe9e815e790 2024-04-23T14:34:35.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.189+0000 7fe9df400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9c406c8f0 0x7fe9c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.189+0000 7fe9df400640 1 -- 172.21.15.59:0/840335703 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe9e00ad460 con 0x7fe9e815e790 2024-04-23T14:34:35.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.189+0000 7fe9e7400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9c406c8f0 0x7fe9c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.190+0000 7fe9e7400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9c406c8f0 0x7fe9c406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005880 tx=0x7fe9d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.193+0000 7fe9df400640 1 -- 172.21.15.59:0/840335703 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9e007b8d0 con 0x7fe9e815e790 2024-04-23T14:34:35.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.400+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9e8162190 con 0x7fe9e815e790 2024-04-23T14:34:35.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.401+0000 7fe9df400640 1 -- 172.21.15.59:0/840335703 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fe9e007b1d0 con 0x7fe9e815e790 2024-04-23T14:34:35.402 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:34:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9c406c8f0 msgr2=0x7fe9c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9c406c8f0 0x7fe9c406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005880 tx=0x7fe9d8009040 comp rx=0 tx=0).stop 2024-04-23T14:34:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e815e790 msgr2=0x7fe9e815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e815e790 0x7fe9e815c220 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009eb0 tx=0x7fe9e0004a40 comp rx=0 tx=0).stop 2024-04-23T14:34:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 shutdown_connections 2024-04-23T14:34:35.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9c406c8f0 0x7fe9c406edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9e8071d70 0x7fe9e81597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e815e790 0x7fe9e815c220 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 --2- 172.21.15.59:0/840335703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e806f6f0 0x7fe9e8159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 >> 172.21.15.59:0/840335703 conn(0x7fe9e8067d90 msgr2=0x7fe9e8158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 shutdown_connections 2024-04-23T14:34:35.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.404+0000 7fe9ee400640 1 -- 172.21.15.59:0/840335703 wait complete. 2024-04-23T14:34:35.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:34:35.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.630+0000 7f312e600640 1 Processor -- start 2024-04-23T14:34:35.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.630+0000 7f312e600640 1 -- start start 2024-04-23T14:34:35.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.632+0000 7f312e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f312815e870 0x7f31280cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.632+0000 7f312e600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31280d16b0 0x7f31280b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.632+0000 7f312e600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31280d23f0 con 0x7f31280d02a0 2024-04-23T14:34:35.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.632+0000 7f312e600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f31280d21a0 con 0x7f312815e870 2024-04-23T14:34:35.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.632+0000 7f312e600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f31280d3000 con 0x7f31280d16b0 2024-04-23T14:34:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f3127e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f31280d02a0 0x7f31280d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42826/0 (socket says 172.21.15.59:42826) 2024-04-23T14:34:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f3127e00640 1 -- 172.21.15.59:0/3102896986 learned_addr learned my addr 172.21.15.59:0/3102896986 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f312d200640 1 --2- 172.21.15.59:0/3102896986 >> v2:172.21.15.59:3300/0 conn(0x7f312815e870 0x7f31280cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f312dc00640 1 --2- 172.21.15.59:0/3102896986 >> v2:172.21.15.76:3300/0 conn(0x7f31280d16b0 0x7f31280b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f312d200640 1 -- 172.21.15.59:0/3102896986 >> v2:172.21.15.76:3300/0 conn(0x7f31280d16b0 msgr2=0x7f31280b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f312d200640 1 --2- 172.21.15.59:0/3102896986 >> v2:172.21.15.76:3300/0 conn(0x7f31280d16b0 0x7f31280b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f312d200640 1 -- 172.21.15.59:0/3102896986 >> v1:172.21.15.59:6789/0 conn(0x7f31280d02a0 legacy=0x7f31280d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.633+0000 7f312d200640 1 -- 172.21.15.59:0/3102896986 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31280d26d0 con 0x7f312815e870 2024-04-23T14:34:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.635+0000 7f312d200640 1 --2- 172.21.15.59:0/3102896986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f312815e870 0x7f31280cfd60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f31280d0d40 tx=0x7f311c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d505a73d7abc1a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.635+0000 7f3127400640 1 -- 172.21.15.59:0/3102896986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c037070 con 0x7f312815e870 2024-04-23T14:34:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.635+0000 7f3127400640 1 -- 172.21.15.59:0/3102896986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c032440 con 0x7f312815e870 2024-04-23T14:34:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.635+0000 7f3127400640 1 -- 172.21.15.59:0/3102896986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c032810 con 0x7f312815e870 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 -- 172.21.15.59:0/3102896986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f312815e870 msgr2=0x7f31280cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 --2- 172.21.15.59:0/3102896986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f312815e870 0x7f31280cfd60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f31280d0d40 tx=0x7f311c02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 -- 172.21.15.59:0/3102896986 shutdown_connections 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 --2- 172.21.15.59:0/3102896986 >> v2:172.21.15.76:3300/0 conn(0x7f31280d16b0 0x7f31280b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 --2- 172.21.15.59:0/3102896986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f312815e870 0x7f31280cfd60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 -- 172.21.15.59:0/3102896986 >> 172.21.15.59:0/3102896986 conn(0x7f312808bf90 msgr2=0x7f31280d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 -- 172.21.15.59:0/3102896986 shutdown_connections 2024-04-23T14:34:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.636+0000 7f312e600640 1 -- 172.21.15.59:0/3102896986 wait complete. 2024-04-23T14:34:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.638+0000 7f312e600640 1 Processor -- start 2024-04-23T14:34:35.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.638+0000 7f312e600640 1 -- start start 2024-04-23T14:34:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31280d02a0 0x7f31280cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312e600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f31280d16b0 0x7f3128160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312e600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f312815e870 0x7f3128160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f312815ec50 con 0x7f31280d02a0 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312e600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f31280d2c90 con 0x7f312815e870 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312e600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f31280d3490 con 0x7f31280d16b0 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31280d02a0 0x7f31280cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31280d02a0 0x7f31280cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59868/0 (socket says 172.21.15.59:59868) 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 -- 172.21.15.59:0/3388109735 learned_addr learned my addr 172.21.15.59:0/3388109735 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 -- 172.21.15.59:0/3388109735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f31280d16b0 msgr2=0x7f3128160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f31280d16b0 0x7f3128160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 -- 172.21.15.59:0/3388109735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f312815e870 msgr2=0x7f3128160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f312815e870 0x7f3128160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.639+0000 7f312d200640 1 -- 172.21.15.59:0/3388109735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31281f58a0 con 0x7f31280d02a0 2024-04-23T14:34:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.640+0000 7f312d200640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31280d02a0 0x7f31280cddc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f311c02a790 tx=0x7f311c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.640+0000 7f3124c00640 1 -- 172.21.15.59:0/3388109735 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c037050 con 0x7f31280d02a0 2024-04-23T14:34:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.640+0000 7f3124c00640 1 -- 172.21.15.59:0/3388109735 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c038040 con 0x7f31280d02a0 2024-04-23T14:34:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.640+0000 7f3124c00640 1 -- 172.21.15.59:0/3388109735 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c03cb50 con 0x7f31280d02a0 2024-04-23T14:34:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.640+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31281f5ab0 con 0x7f31280d02a0 2024-04-23T14:34:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.640+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31281f5000 con 0x7f31280d02a0 2024-04-23T14:34:35.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.641+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312804fd90 con 0x7f31280d02a0 2024-04-23T14:34:35.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.642+0000 7f3124c00640 1 -- 172.21.15.59:0/3388109735 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f311c03ccf0 con 0x7f31280d02a0 2024-04-23T14:34:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.643+0000 7f3124c00640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f310806c8f0 0x7f310806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.643+0000 7f3124c00640 1 -- 172.21.15.59:0/3388109735 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f311c0ac560 con 0x7f31280d02a0 2024-04-23T14:34:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.644+0000 7f3127e00640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f310806c8f0 0x7f310806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.644+0000 7f3127e00640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f310806c8f0 0x7f310806edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3114000f60 tx=0x7f3114007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:35.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.648+0000 7f3124c00640 1 -- 172.21.15.59:0/3388109735 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f311c07aa60 con 0x7f31280d02a0 2024-04-23T14:34:35.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.851+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f31280c7f30 con 0x7f31280d02a0 2024-04-23T14:34:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.852+0000 7f3124c00640 1 -- 172.21.15.59:0/3388109735 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f311c07a360 con 0x7f31280d02a0 2024-04-23T14:34:35.853 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:34:35.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f310806c8f0 msgr2=0x7f310806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f310806c8f0 0x7f310806edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3114000f60 tx=0x7f3114007a80 comp rx=0 tx=0).stop 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31280d02a0 msgr2=0x7f31280cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31280d02a0 0x7f31280cddc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f311c02a790 tx=0x7f311c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 shutdown_connections 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f310806c8f0 0x7f310806edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f31280d16b0 0x7f3128160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f312815e870 0x7f3128160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 --2- 172.21.15.59:0/3388109735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31280d02a0 0x7f31280cddc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 >> 172.21.15.59:0/3388109735 conn(0x7f312808bf90 msgr2=0x7f312815ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.855+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 shutdown_connections 2024-04-23T14:34:35.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:35.856+0000 7f312e600640 1 -- 172.21.15.59:0/3388109735 wait complete. 2024-04-23T14:34:35.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:34:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.087+0000 7fa0bea00640 1 Processor -- start 2024-04-23T14:34:36.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.087+0000 7fa0bea00640 1 -- start start 2024-04-23T14:34:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.089+0000 7fa0bea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0b8145e10 0x7fa0b806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.089+0000 7fa0bea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8071d70 0x7fa0b8072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.089+0000 7fa0bea00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b80cfc20 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.089+0000 7fa0bea00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b80cff00 con 0x7fa0b8145e10 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.089+0000 7fa0bea00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b80d01e0 con 0x7fa0b8071d70 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0b7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0b8145e10 0x7fa0b806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0b7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0b8145e10 0x7fa0b806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59880/0 (socket says 172.21.15.59:59880) 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0b7e00640 1 -- 172.21.15.59:0/3286150625 learned_addr learned my addr 172.21.15.59:0/3286150625 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0bca00640 1 --2- 172.21.15.59:0/3286150625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8071d70 0x7fa0b8072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0b7e00640 1 -- 172.21.15.59:0/3286150625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8071d70 msgr2=0x7fa0b8072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:36.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0b7e00640 1 --2- 172.21.15.59:0/3286150625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8071d70 0x7fa0b8072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0b7e00640 1 -- 172.21.15.59:0/3286150625 >> v1:172.21.15.76:6789/0 conn(0x7fa0b80d0cc0 legacy=0x7fa0b80cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:36.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.090+0000 7fa0b7e00640 1 -- 172.21.15.59:0/3286150625 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b80d04c0 con 0x7fa0b8145e10 2024-04-23T14:34:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.091+0000 7fa0b7e00640 1 --2- 172.21.15.59:0/3286150625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b8145e10 0x7fa0b806f130 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009870 tx=0x7fa0a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13f9b55a8e36066 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.092+0000 7fa0b7400640 1 -- 172.21.15.59:0/3286150625 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8037070 con 0x7fa0b8145e10 2024-04-23T14:34:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.092+0000 7fa0b7400640 1 -- 172.21.15.59:0/3286150625 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a8032440 con 0x7fa0b8145e10 2024-04-23T14:34:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.092+0000 7fa0b7400640 1 -- 172.21.15.59:0/3286150625 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8032810 con 0x7fa0b8145e10 2024-04-23T14:34:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.092+0000 7fa0bea00640 1 -- 172.21.15.59:0/3286150625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b8145e10 msgr2=0x7fa0b806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.092+0000 7fa0bea00640 1 --2- 172.21.15.59:0/3286150625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b8145e10 0x7fa0b806f130 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009870 tx=0x7fa0a802a210 comp rx=0 tx=0).stop 2024-04-23T14:34:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.093+0000 7fa0bea00640 1 -- 172.21.15.59:0/3286150625 shutdown_connections 2024-04-23T14:34:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.093+0000 7fa0bea00640 1 --2- 172.21.15.59:0/3286150625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8071d70 0x7fa0b8072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.093+0000 7fa0bea00640 1 --2- 172.21.15.59:0/3286150625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b8145e10 0x7fa0b806f130 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.093+0000 7fa0bea00640 1 -- 172.21.15.59:0/3286150625 >> 172.21.15.59:0/3286150625 conn(0x7fa0b8067d90 msgr2=0x7fa0b806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.093+0000 7fa0bea00640 1 -- 172.21.15.59:0/3286150625 shutdown_connections 2024-04-23T14:34:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.093+0000 7fa0bea00640 1 -- 172.21.15.59:0/3286150625 wait complete. 2024-04-23T14:34:36.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.095+0000 7fa0bea00640 1 Processor -- start 2024-04-23T14:34:36.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.095+0000 7fa0bea00640 1 -- start start 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0bea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8071d70 0x7fa0b80b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0bea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b80d0cc0 0x7fa0b81f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0bea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8145e10 0x7fa0b815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0bea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b81461f0 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0bea00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b815cdd0 con 0x7fa0b8071d70 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0bea00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b81f8ab0 con 0x7fa0b8145e10 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b80d0cc0 0x7fa0b81f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b80d0cc0 0x7fa0b81f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59892/0 (socket says 172.21.15.59:59892) 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.096+0000 7fa0afe00640 1 -- 172.21.15.59:0/2734442832 learned_addr learned my addr 172.21.15.59:0/2734442832 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0b7e00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8071d70 0x7fa0b80b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0bca00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8145e10 0x7fa0b815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0afe00640 1 -- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8145e10 msgr2=0x7fa0b815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0afe00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8145e10 0x7fa0b815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0afe00640 1 -- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8071d70 msgr2=0x7fa0b80b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0afe00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8071d70 0x7fa0b80b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0afe00640 1 -- 172.21.15.59:0/2734442832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b81f84f0 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0bca00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8145e10 0x7fa0b815f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0b7e00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8071d70 0x7fa0b80b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.097+0000 7fa0afe00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b80d0cc0 0x7fa0b81f7ba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c430 tx=0x7fa0a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.098+0000 7fa0b4c00640 1 -- 172.21.15.59:0/2734442832 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0011790 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.098+0000 7fa0b4c00640 1 -- 172.21.15.59:0/2734442832 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0017070 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.098+0000 7fa0b4c00640 1 -- 172.21.15.59:0/2734442832 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0011bb0 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.098+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b81f87d0 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.098+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b81e5400 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.099+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b8068300 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.100+0000 7fa0b4c00640 1 -- 172.21.15.59:0/2734442832 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0a001c460 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.101+0000 7fa0b4c00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa09406c8f0 0x7fa09406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.101+0000 7fa0b4c00640 1 -- 172.21.15.59:0/2734442832 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fa0a008b960 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.101+0000 7fa0b7e00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa09406c8f0 0x7fa09406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.102+0000 7fa0b7e00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa09406c8f0 0x7fa09406edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8034000 tx=0x7fa0a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.108+0000 7fa0b4c00640 1 -- 172.21.15.59:0/2734442832 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0b8068300 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.316+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa0b8068a80 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.317+0000 7fa0b4c00640 1 -- 172.21.15.59:0/2734442832 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7fa0a0059de0 con 0x7fa0b80d0cc0 2024-04-23T14:34:36.318 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa09406c8f0 msgr2=0x7fa09406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa09406c8f0 0x7fa09406edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8034000 tx=0x7fa0a802e000 comp rx=0 tx=0).stop 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b80d0cc0 msgr2=0x7fa0b81f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b80d0cc0 0x7fa0b81f7ba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c430 tx=0x7fa0a000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 shutdown_connections 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa09406c8f0 0x7fa09406edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0b8145e10 0x7fa0b815f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0b80d0cc0 0x7fa0b81f7ba0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 --2- 172.21.15.59:0/2734442832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8071d70 0x7fa0b80b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.319+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 >> 172.21.15.59:0/2734442832 conn(0x7fa0b8067d90 msgr2=0x7fa0b81443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.320+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 shutdown_connections 2024-04-23T14:34:36.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.320+0000 7fa0bea00640 1 -- 172.21.15.59:0/2734442832 wait complete. 2024-04-23T14:34:36.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.542+0000 7f2ad5400640 1 Processor -- start 2024-04-23T14:34:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.542+0000 7f2ad5400640 1 -- start start 2024-04-23T14:34:36.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.543+0000 7f2ad5400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ad015e8c0 0x7f2ad00b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.543+0000 7f2ad5400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00b4480 0x7f2ad00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.543+0000 7f2ad5400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad00b4910 con 0x7f2ad00d2390 2024-04-23T14:34:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.543+0000 7f2ad5400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad00b5ef0 con 0x7f2ad015e8c0 2024-04-23T14:34:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.543+0000 7f2ad5400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad00b4bf0 con 0x7f2ad00b4480 2024-04-23T14:34:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2acfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ad015e8c0 0x7f2ad00b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2ad4a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2ad00d2390 0x7f2ad00cf3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42836/0 (socket says 172.21.15.59:42836) 2024-04-23T14:34:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2ad4a00640 1 -- 172.21.15.59:0/875816575 learned_addr learned my addr 172.21.15.59:0/875816575 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2acf400640 1 --2- 172.21.15.59:0/875816575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00b4480 0x7f2ad00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2acea00640 1 -- 172.21.15.59:0/875816575 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087991182 0 0) 0x7f2ad00b4910 con 0x7f2ad00d2390 2024-04-23T14:34:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2acfe00640 1 -- 172.21.15.59:0/875816575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00b4480 msgr2=0x7f2ad00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2acfe00640 1 --2- 172.21.15.59:0/875816575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00b4480 0x7f2ad00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2acfe00640 1 -- 172.21.15.59:0/875816575 >> v1:172.21.15.59:6789/0 conn(0x7f2ad00d2390 legacy=0x7f2ad00cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.544+0000 7f2acfe00640 1 -- 172.21.15.59:0/875816575 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad00b4ed0 con 0x7f2ad015e8c0 2024-04-23T14:34:36.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.546+0000 7f2acfe00640 1 --2- 172.21.15.59:0/875816575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad015e8c0 0x7f2ad00b59b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009790 tx=0x7f2ac002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c012392ada2c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.546+0000 7f2acea00640 1 -- 172.21.15.59:0/875816575 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2ad015e8c0 2024-04-23T14:34:36.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.546+0000 7f2acea00640 1 -- 172.21.15.59:0/875816575 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0032440 con 0x7f2ad015e8c0 2024-04-23T14:34:36.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.546+0000 7f2acea00640 1 -- 172.21.15.59:0/875816575 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0032810 con 0x7f2ad015e8c0 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 -- 172.21.15.59:0/875816575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad015e8c0 msgr2=0x7f2ad00b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 --2- 172.21.15.59:0/875816575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad015e8c0 0x7f2ad00b59b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009790 tx=0x7f2ac002a200 comp rx=0 tx=0).stop 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 -- 172.21.15.59:0/875816575 shutdown_connections 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 --2- 172.21.15.59:0/875816575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00b4480 0x7f2ad00d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 --2- 172.21.15.59:0/875816575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad015e8c0 0x7f2ad00b59b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 -- 172.21.15.59:0/875816575 >> 172.21.15.59:0/875816575 conn(0x7f2ad008bf90 msgr2=0x7f2ad00d45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 -- 172.21.15.59:0/875816575 shutdown_connections 2024-04-23T14:34:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.547+0000 7f2ad5400640 1 -- 172.21.15.59:0/875816575 wait complete. 2024-04-23T14:34:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.549+0000 7f2ad5400640 1 Processor -- start 2024-04-23T14:34:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.549+0000 7f2ad5400640 1 -- start start 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2ad5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad00d2390 0x7f2ad00b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2ad5400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ad015e8c0 0x7f2ad00d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2ad5400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00ce1a0 0x7f2ad00d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2ad5400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad015eca0 con 0x7f2ad00d2390 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2ad5400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad00b5490 con 0x7f2ad015e8c0 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2ad5400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad015ee20 con 0x7f2ad00ce1a0 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad00d2390 0x7f2ad00b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad00d2390 0x7f2ad00b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59906/0 (socket says 172.21.15.59:59906) 2024-04-23T14:34:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2acfe00640 1 -- 172.21.15.59:0/2201099743 learned_addr learned my addr 172.21.15.59:0/2201099743 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.550+0000 7f2ad4a00640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00ce1a0 0x7f2ad00d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.551+0000 7f2acfe00640 1 -- 172.21.15.59:0/2201099743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00ce1a0 msgr2=0x7f2ad00d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.551+0000 7f2acfe00640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00ce1a0 0x7f2ad00d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.551+0000 7f2acfe00640 1 -- 172.21.15.59:0/2201099743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ad015e8c0 msgr2=0x7f2ad00d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.551+0000 7f2acfe00640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ad015e8c0 0x7f2ad00d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.551+0000 7f2acfe00640 1 -- 172.21.15.59:0/2201099743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad00b9a90 con 0x7f2ad00d2390 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.551+0000 7f2acfe00640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad00d2390 0x7f2ad00b7bc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0035040 tx=0x7f2ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.551+0000 7f2ac7e00640 1 -- 172.21.15.59:0/2201099743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037040 con 0x7f2ad00d2390 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.552+0000 7f2ac7e00640 1 -- 172.21.15.59:0/2201099743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0005bd0 con 0x7f2ad00d2390 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.552+0000 7f2ac7e00640 1 -- 172.21.15.59:0/2201099743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0030770 con 0x7f2ad00d2390 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.552+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad00b9d70 con 0x7f2ad00d2390 2024-04-23T14:34:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.552+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad00ca8b0 con 0x7f2ad00d2390 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.553+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad004fd90 con 0x7f2ad00d2390 2024-04-23T14:34:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.554+0000 7f2ac7e00640 1 -- 172.21.15.59:0/2201099743 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ac0030910 con 0x7f2ad00d2390 2024-04-23T14:34:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.554+0000 7f2ac7e00640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.555+0000 7f2ac7e00640 1 -- 172.21.15.59:0/2201099743 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f2ac00ac690 con 0x7f2ad00d2390 2024-04-23T14:34:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.555+0000 7f2acf400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.556+0000 7f2acf400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.559+0000 7f2ac7e00640 1 -- 172.21.15.59:0/2201099743 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ac0032e20 con 0x7f2ad00d2390 2024-04-23T14:34:36.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.807+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ad01f59e0 con 0x7f2ad00d2390 2024-04-23T14:34:36.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.808+0000 7f2ac7e00640 1 -- 172.21.15.59:0/2201099743 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7f2ac007a9e0 con 0x7f2ad00d2390 2024-04-23T14:34:36.810 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:36.810 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:34:36.810 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:34:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 msgr2=0x7f2aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8008040 comp rx=0 tx=0).stop 2024-04-23T14:34:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad00d2390 msgr2=0x7f2ad00b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad00d2390 0x7f2ad00b7bc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0035040 tx=0x7f2ac00046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 shutdown_connections 2024-04-23T14:34:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ad00ce1a0 0x7f2ad00d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ad015e8c0 0x7f2ad00d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 --2- 172.21.15.59:0/2201099743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ad00d2390 0x7f2ad00b7bc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 >> 172.21.15.59:0/2201099743 conn(0x7f2ad008bf90 msgr2=0x7f2ad0162490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.811+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 shutdown_connections 2024-04-23T14:34:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:36.812+0000 7f2ad5400640 1 -- 172.21.15.59:0/2201099743 wait complete. 2024-04-23T14:34:36.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:37.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.040+0000 7f57d0e00640 1 Processor -- start 2024-04-23T14:34:37.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.040+0000 7f57d0e00640 1 -- start start 2024-04-23T14:34:37.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.041+0000 7f57d0e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc06f3f0 0x7f57cc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.041+0000 7f57d0e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57cc0b3080 0x7f57cc0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57d0e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57cc0b3870 con 0x7f57cc15b450 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57d0e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc0b41a0 con 0x7f57cc0b3080 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57d0e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc0b3b50 con 0x7f57cc06f3f0 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57cbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57cc0b3080 0x7f57cc0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57cbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57cc0b3080 0x7f57cc0cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59912/0 (socket says 172.21.15.59:59912) 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57cbe00640 1 -- 172.21.15.59:0/65145440 learned_addr learned my addr 172.21.15.59:0/65145440 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57c2a00640 1 --2- 172.21.15.59:0/65145440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc06f3f0 0x7f57cc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57cbe00640 1 -- 172.21.15.59:0/65145440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc06f3f0 msgr2=0x7f57cc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57cbe00640 1 --2- 172.21.15.59:0/65145440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc06f3f0 0x7f57cc0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57cbe00640 1 -- 172.21.15.59:0/65145440 >> v1:172.21.15.76:6789/0 conn(0x7f57cc15b450 legacy=0x7f57cc0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:37.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.042+0000 7f57cbe00640 1 -- 172.21.15.59:0/65145440 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc0b3e30 con 0x7f57cc0b3080 2024-04-23T14:34:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.043+0000 7f57cbe00640 1 --2- 172.21.15.59:0/65145440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc0cd590 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009e50 tx=0x7f57c402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04f20b2549e41ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.044+0000 7f57caa00640 1 -- 172.21.15.59:0/65145440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4036070 con 0x7f57cc0b3080 2024-04-23T14:34:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.044+0000 7f57caa00640 1 -- 172.21.15.59:0/65145440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c4031600 con 0x7f57cc0b3080 2024-04-23T14:34:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.044+0000 7f57caa00640 1 -- 172.21.15.59:0/65145440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40319d0 con 0x7f57cc0b3080 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.044+0000 7f57d0e00640 1 -- 172.21.15.59:0/65145440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 msgr2=0x7f57cc0cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.044+0000 7f57d0e00640 1 --2- 172.21.15.59:0/65145440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc0cd590 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009e50 tx=0x7f57c402a6d0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.044+0000 7f57d0e00640 1 -- 172.21.15.59:0/65145440 shutdown_connections 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.044+0000 7f57d0e00640 1 --2- 172.21.15.59:0/65145440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc0cd590 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.045+0000 7f57d0e00640 1 --2- 172.21.15.59:0/65145440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc06f3f0 0x7f57cc0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.045+0000 7f57d0e00640 1 -- 172.21.15.59:0/65145440 >> 172.21.15.59:0/65145440 conn(0x7f57cc0b21a0 msgr2=0x7f57cc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.045+0000 7f57d0e00640 1 -- 172.21.15.59:0/65145440 shutdown_connections 2024-04-23T14:34:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.045+0000 7f57d0e00640 1 -- 172.21.15.59:0/65145440 wait complete. 2024-04-23T14:34:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.046+0000 7f57d0e00640 1 Processor -- start 2024-04-23T14:34:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.047+0000 7f57d0e00640 1 -- start start 2024-04-23T14:34:37.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57d0e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57cc06f3f0 0x7f57cc09d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57d0e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57d0e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc15b450 0x7f57cc0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57d0e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0ad060 con 0x7f57cc0b3080 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57d0e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0b4ff0 con 0x7f57cc06f3f0 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57d0e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc0ad1e0 con 0x7f57cc15b450 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc15bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59914/0 (socket says 172.21.15.59:59914) 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57c2a00640 1 -- 172.21.15.59:0/1293349302 learned_addr learned my addr 172.21.15.59:0/1293349302 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57cb400640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57cc06f3f0 0x7f57cc09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.048+0000 7f57cbe00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc15b450 0x7f57cc0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c2a00640 1 -- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc15b450 msgr2=0x7f57cc0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c2a00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc15b450 0x7f57cc0c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c2a00640 1 -- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57cc06f3f0 msgr2=0x7f57cc09d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c2a00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57cc06f3f0 0x7f57cc09d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c2a00640 1 -- 172.21.15.59:0/1293349302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc15c560 con 0x7f57cc0b3080 2024-04-23T14:34:37.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c2a00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc15bb10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c7f0 tx=0x7f57b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c3e00640 1 -- 172.21.15.59:0/1293349302 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4015070 con 0x7f57cc0b3080 2024-04-23T14:34:37.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c3e00640 1 -- 172.21.15.59:0/1293349302 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b4004240 con 0x7f57cc0b3080 2024-04-23T14:34:37.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc15c840 con 0x7f57cc0b3080 2024-04-23T14:34:37.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57c3e00640 1 -- 172.21.15.59:0/1293349302 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4007620 con 0x7f57cc0b3080 2024-04-23T14:34:37.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.049+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc15ce00 con 0x7f57cc0b3080 2024-04-23T14:34:37.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.051+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc04fd90 con 0x7f57cc0b3080 2024-04-23T14:34:37.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.052+0000 7f57c3e00640 1 -- 172.21.15.59:0/1293349302 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57b40079d0 con 0x7f57cc0b3080 2024-04-23T14:34:37.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.053+0000 7f57c3e00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f57a806c8f0 0x7f57a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.053+0000 7f57c3e00640 1 -- 172.21.15.59:0/1293349302 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f57b408c0c0 con 0x7f57cc0b3080 2024-04-23T14:34:37.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.053+0000 7f57cb400640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f57a806c8f0 0x7f57a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.054+0000 7f57cb400640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f57a806c8f0 0x7f57a806edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.057+0000 7f57c3e00640 1 -- 172.21.15.59:0/1293349302 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57b405a410 con 0x7f57cc0b3080 2024-04-23T14:34:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.265+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57cc0c36f0 con 0x7f57cc0b3080 2024-04-23T14:34:37.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.268+0000 7f57c3e00640 1 -- 172.21.15.59:0/1293349302 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7f57b4059d10 con 0x7f57cc0b3080 2024-04-23T14:34:37.269 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:37.270 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":46,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:08.435362+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:45.619774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:46.736010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:37.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f57a806c8f0 msgr2=0x7f57a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f57a806c8f0 0x7f57a806edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57bc006fd0 tx=0x7f57bc008040 comp rx=0 tx=0).stop 2024-04-23T14:34:37.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 msgr2=0x7f57cc15bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc15bb10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f57b400c7f0 tx=0x7f57b400ccb0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 shutdown_connections 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f57a806c8f0 0x7f57a806edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57cc15b450 0x7f57cc0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57cc0b3080 0x7f57cc15bb10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 --2- 172.21.15.59:0/1293349302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57cc06f3f0 0x7f57cc09d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 >> 172.21.15.59:0/1293349302 conn(0x7f57cc0b21a0 msgr2=0x7f57cc0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 shutdown_connections 2024-04-23T14:34:37.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.271+0000 7f57d0e00640 1 -- 172.21.15.59:0/1293349302 wait complete. 2024-04-23T14:34:37.291 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:34:37.291 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:37.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.500+0000 7ff28ee00640 1 Processor -- start 2024-04-23T14:34:37.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.500+0000 7ff28ee00640 1 -- start start 2024-04-23T14:34:37.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.501+0000 7ff28ee00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2880b3820 0x7ff28809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.501+0000 7ff28ee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2881625c0 0x7ff28809af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.501+0000 7ff28ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2880cd2b0 con 0x7ff28809ba00 2024-04-23T14:34:37.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.501+0000 7ff28ee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff2880d06e0 con 0x7ff2881625c0 2024-04-23T14:34:37.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.501+0000 7ff28ee00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff2880cd550 con 0x7ff2880b3820 2024-04-23T14:34:37.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.502+0000 7ff28d000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff28809ba00 0x7ff28815e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42840/0 (socket says 172.21.15.59:42840) 2024-04-23T14:34:37.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.502+0000 7ff28d000640 1 -- 172.21.15.59:0/272786355 learned_addr learned my addr 172.21.15.59:0/272786355 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.502+0000 7ff28e400640 1 --2- 172.21.15.59:0/272786355 >> v2:172.21.15.59:3300/0 conn(0x7ff2881625c0 0x7ff28809af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.502+0000 7ff28da00640 1 --2- 172.21.15.59:0/272786355 >> v2:172.21.15.76:3300/0 conn(0x7ff2880b3820 0x7ff28809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.503+0000 7ff287e00640 1 -- 172.21.15.59:0/272786355 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736465640 0 0) 0x7ff2880cd2b0 con 0x7ff28809ba00 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.503+0000 7ff287e00640 1 -- 172.21.15.59:0/272786355 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff26c003520 con 0x7ff28809ba00 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.503+0000 7ff28e400640 1 -- 172.21.15.59:0/272786355 >> v2:172.21.15.76:3300/0 conn(0x7ff2880b3820 msgr2=0x7ff28809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.503+0000 7ff28e400640 1 --2- 172.21.15.59:0/272786355 >> v2:172.21.15.76:3300/0 conn(0x7ff2880b3820 0x7ff28809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.503+0000 7ff28e400640 1 -- 172.21.15.59:0/272786355 >> v1:172.21.15.59:6789/0 conn(0x7ff28809ba00 legacy=0x7ff28815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.503+0000 7ff28e400640 1 -- 172.21.15.59:0/272786355 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2880cd830 con 0x7ff2881625c0 2024-04-23T14:34:37.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.504+0000 7ff28e400640 1 --2- 172.21.15.59:0/272786355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2881625c0 0x7ff28809af40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff288072fc0 tx=0x7ff28002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e21bc09d92e196 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.504+0000 7ff287e00640 1 -- 172.21.15.59:0/272786355 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280007740 con 0x7ff2881625c0 2024-04-23T14:34:37.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.504+0000 7ff287e00640 1 -- 172.21.15.59:0/272786355 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2800078e0 con 0x7ff2881625c0 2024-04-23T14:34:37.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.504+0000 7ff287e00640 1 -- 172.21.15.59:0/272786355 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280007c90 con 0x7ff2881625c0 2024-04-23T14:34:37.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 -- 172.21.15.59:0/272786355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2881625c0 msgr2=0x7ff28809af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 --2- 172.21.15.59:0/272786355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2881625c0 0x7ff28809af40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff288072fc0 tx=0x7ff28002a5f0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 -- 172.21.15.59:0/272786355 shutdown_connections 2024-04-23T14:34:37.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 --2- 172.21.15.59:0/272786355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2881625c0 0x7ff28809af40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 --2- 172.21.15.59:0/272786355 >> v2:172.21.15.76:3300/0 conn(0x7ff2880b3820 0x7ff28809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 -- 172.21.15.59:0/272786355 >> 172.21.15.59:0/272786355 conn(0x7ff2880b3ef0 msgr2=0x7ff2880b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:37.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 -- 172.21.15.59:0/272786355 shutdown_connections 2024-04-23T14:34:37.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.505+0000 7ff28ee00640 1 -- 172.21.15.59:0/272786355 wait complete. 2024-04-23T14:34:37.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.507+0000 7ff28ee00640 1 Processor -- start 2024-04-23T14:34:37.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.508+0000 7ff28ee00640 1 -- start start 2024-04-23T14:34:37.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff28809ba00 0x7ff28815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2880b3820 0x7ff288156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2881625c0 0x7ff28815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880ce0d0 con 0x7ff28809ba00 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880cddf0 con 0x7ff2881625c0 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2880ce5f0 con 0x7ff2880b3820 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff28809ba00 0x7ff28815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff28809ba00 0x7ff28815bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59944/0 (socket says 172.21.15.59:59944) 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.509+0000 7ff28da00640 1 -- 172.21.15.59:0/1798119251 learned_addr learned my addr 172.21.15.59:0/1798119251 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff28da00640 1 -- 172.21.15.59:0/1798119251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2880b3820 msgr2=0x7ff288156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff28da00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2880b3820 0x7ff288156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff28da00640 1 -- 172.21.15.59:0/1798119251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2881625c0 msgr2=0x7ff28815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff28da00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2881625c0 0x7ff28815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff28da00640 1 -- 172.21.15.59:0/1798119251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28815b770 con 0x7ff28809ba00 2024-04-23T14:34:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff28da00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff28809ba00 0x7ff28815bae0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff27800e9b0 tx=0x7ff27800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff285600640 1 -- 172.21.15.59:0/1798119251 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278017070 con 0x7ff28809ba00 2024-04-23T14:34:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff285600640 1 -- 172.21.15.59:0/1798119251 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27800c9a0 con 0x7ff28809ba00 2024-04-23T14:34:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff285600640 1 -- 172.21.15.59:0/1798119251 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27800cd90 con 0x7ff28809ba00 2024-04-23T14:34:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.510+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2880c9050 con 0x7ff28809ba00 2024-04-23T14:34:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.511+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2880cb340 con 0x7ff28809ba00 2024-04-23T14:34:37.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.512+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28804fd90 con 0x7ff28809ba00 2024-04-23T14:34:37.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.513+0000 7ff285600640 1 -- 172.21.15.59:0/1798119251 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff27801a070 con 0x7ff28809ba00 2024-04-23T14:34:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.514+0000 7ff285600640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff26806c8f0 0x7ff26806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.514+0000 7ff285600640 1 -- 172.21.15.59:0/1798119251 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7ff27808bbb0 con 0x7ff28809ba00 2024-04-23T14:34:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.515+0000 7ff28d000640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff26806c8f0 0x7ff26806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.515+0000 7ff28d000640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff26806c8f0 0x7ff26806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff274000f60 tx=0x7ff2740078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.519+0000 7ff285600640 1 -- 172.21.15.59:0/1798119251 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff278059f90 con 0x7ff28809ba00 2024-04-23T14:34:37.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.763+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2880cffd0 con 0x7ff28809ba00 2024-04-23T14:34:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.764+0000 7ff285600640 1 -- 172.21.15.59:0/1798119251 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7ff278059890 con 0x7ff28809ba00 2024-04-23T14:34:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:37.765 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:34:37.766 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff26806c8f0 msgr2=0x7ff26806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff26806c8f0 0x7ff26806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff274000f60 tx=0x7ff2740078f0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff28809ba00 msgr2=0x7ff28815bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff28809ba00 0x7ff28815bae0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff27800e9b0 tx=0x7ff27800ee70 comp rx=0 tx=0).stop 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 shutdown_connections 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff26806c8f0 0x7ff26806edb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2880b3820 0x7ff288156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2881625c0 0x7ff28815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 --2- 172.21.15.59:0/1798119251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff28809ba00 0x7ff28815bae0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 >> 172.21.15.59:0/1798119251 conn(0x7ff2880b3ef0 msgr2=0x7ff28806c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 shutdown_connections 2024-04-23T14:34:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.767+0000 7ff28ee00640 1 -- 172.21.15.59:0/1798119251 wait complete. 2024-04-23T14:34:37.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:38.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:37.999+0000 7f2ae2a00640 1 Processor -- start 2024-04-23T14:34:38.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.000+0000 7f2ae2a00640 1 -- start start 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.001+0000 7f2ae2a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2adc0d0ec0 0x7f2adc0d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.001+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0d28b0 0x7f2adc15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.001+0000 7f2ae2a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2adc0d17e0 con 0x7f2adc0cff10 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.001+0000 7f2ae2a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2adc0b3b60 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.001+0000 7f2ae2a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc15fd00 con 0x7f2adc0d28b0 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.002+0000 7f2ae2000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0d28b0 0x7f2adc15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.002+0000 7f2ae1600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2adc0d0ec0 0x7f2adc0d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.002+0000 7f2ae2000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0d28b0 0x7f2adc15eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:53932/0 (socket says 172.21.15.59:53932) 2024-04-23T14:34:38.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.002+0000 7f2ae2000640 1 -- 172.21.15.59:0/2292594614 learned_addr learned my addr 172.21.15.59:0/2292594614 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:38.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.003+0000 7f2ae2000640 1 -- 172.21.15.59:0/2292594614 >> v2:172.21.15.76:3300/0 conn(0x7f2adc0d0ec0 msgr2=0x7f2adc0d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:38.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.003+0000 7f2ae2000640 1 --2- 172.21.15.59:0/2292594614 >> v2:172.21.15.76:3300/0 conn(0x7f2adc0d0ec0 0x7f2adc0d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.003+0000 7f2ae2000640 1 -- 172.21.15.59:0/2292594614 >> v1:172.21.15.76:6789/0 conn(0x7f2adc0cff10 legacy=0x7f2adc0d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.003+0000 7f2ae2000640 1 -- 172.21.15.59:0/2292594614 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc15fa20 con 0x7f2adc0d28b0 2024-04-23T14:34:38.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.003+0000 7f2ae1600640 1 --2- 172.21.15.59:0/2292594614 >> v2:172.21.15.76:3300/0 conn(0x7f2adc0d0ec0 0x7f2adc0d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:38.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.005+0000 7f2ae2000640 1 --2- 172.21.15.59:0/2292594614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0d28b0 0x7f2adc15eb10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009e50 tx=0x7f2ad402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc775f0710251d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.005+0000 7f2adbe00640 1 -- 172.21.15.59:0/2292594614 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4036070 con 0x7f2adc0d28b0 2024-04-23T14:34:38.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.005+0000 7f2adbe00640 1 -- 172.21.15.59:0/2292594614 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad4031400 con 0x7f2adc0d28b0 2024-04-23T14:34:38.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2adbe00640 1 -- 172.21.15.59:0/2292594614 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40317d0 con 0x7f2adc0d28b0 2024-04-23T14:34:38.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2ae2a00640 1 -- 172.21.15.59:0/2292594614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0d28b0 msgr2=0x7f2adc15eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/2292594614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0d28b0 0x7f2adc15eb10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009e50 tx=0x7f2ad402a6a0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2ae2a00640 1 -- 172.21.15.59:0/2292594614 shutdown_connections 2024-04-23T14:34:38.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/2292594614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0d28b0 0x7f2adc15eb10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/2292594614 >> v2:172.21.15.76:3300/0 conn(0x7f2adc0d0ec0 0x7f2adc0d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2ae2a00640 1 -- 172.21.15.59:0/2292594614 >> 172.21.15.59:0/2292594614 conn(0x7f2adc08bf90 msgr2=0x7f2adc0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:38.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.006+0000 7f2ae2a00640 1 -- 172.21.15.59:0/2292594614 shutdown_connections 2024-04-23T14:34:38.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.007+0000 7f2ae2a00640 1 -- 172.21.15.59:0/2292594614 wait complete. 2024-04-23T14:34:38.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.008+0000 7f2ae2a00640 1 Processor -- start 2024-04-23T14:34:38.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.008+0000 7f2ae2a00640 1 -- start start 2024-04-23T14:34:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.009+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0cff10 0x7f2adc0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.009+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2adc0d0ec0 0x7f2adc15c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.009+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2adc0d28b0 0x7f2adc0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae2a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0cf9f0 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae2a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0cf710 con 0x7f2adc0d28b0 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae2a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc0d0330 con 0x7f2adc0cff10 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2adc0d0ec0 0x7f2adc15c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2adc0d0ec0 0x7f2adc15c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59952/0 (socket says 172.21.15.59:59952) 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae0c00640 1 -- 172.21.15.59:0/1374363777 learned_addr learned my addr 172.21.15.59:0/1374363777 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae2000640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2adc0d28b0 0x7f2adc0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae1600640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0cff10 0x7f2adc0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae0c00640 1 -- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0cff10 msgr2=0x7f2adc0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.010+0000 7f2ae0c00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0cff10 0x7f2adc0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ae0c00640 1 -- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2adc0d28b0 msgr2=0x7f2adc0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ae0c00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2adc0d28b0 0x7f2adc0cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ae0c00640 1 -- 172.21.15.59:0/1374363777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc15b250 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ae0c00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2adc0d0ec0 0x7f2adc15c2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c780 tx=0x7f2acc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ad9600640 1 -- 172.21.15.59:0/1374363777 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc017070 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ad9600640 1 -- 172.21.15.59:0/1374363777 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc005290 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc15b530 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ad9600640 1 -- 172.21.15.59:0/1374363777 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc00a880 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.011+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc15baf0 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.013+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc04fd90 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.014+0000 7f2ad9600640 1 -- 172.21.15.59:0/1374363777 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2acc01a070 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.016+0000 7f2ad9600640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.016+0000 7f2ad9600640 1 -- 172.21.15.59:0/1374363777 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f2acc08b1a0 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.016+0000 7f2ae1600640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.017+0000 7f2ae1600640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ac006c8f0 0x7f2ac006edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009560 tx=0x7f2ad0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.020+0000 7f2ad9600640 1 -- 172.21.15.59:0/1374363777 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2acc0594f0 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.269+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2adc06ead0 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.270+0000 7f2ad9600640 1 -- 172.21.15.59:0/1374363777 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5172 (secure 0 0 0) 0x7f2acc058df0 con 0x7f2adc0d0ec0 2024-04-23T14:34:38.272 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:38.272 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:33:47.763614+0000","modified":"2024-04-23T14:33:50.838863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2804599006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2804599006},{"type":"v1","addr":"172.21.15.59:6835","nonce":2804599006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":3}]} 2024-04-23T14:34:38.272 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2024-04-23T14:34:38.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ac006c8f0 msgr2=0x7f2ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ac006c8f0 0x7f2ac006edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009560 tx=0x7f2ad0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:38.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2adc0d0ec0 msgr2=0x7f2adc15c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2adc0d0ec0 0x7f2adc15c2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c780 tx=0x7f2acc00cc40 comp rx=0 tx=0).stop 2024-04-23T14:34:38.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 shutdown_connections 2024-04-23T14:34:38.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2ac006c8f0 0x7f2ac006edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2adc0d28b0 0x7f2adc0cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2adc0d0ec0 0x7f2adc15c2f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 --2- 172.21.15.59:0/1374363777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2adc0cff10 0x7f2adc0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 >> 172.21.15.59:0/1374363777 conn(0x7f2adc08bf90 msgr2=0x7f2adc15ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:38.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.273+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 shutdown_connections 2024-04-23T14:34:38.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.274+0000 7f2ae2a00640 1 -- 172.21.15.59:0/1374363777 wait complete. 2024-04-23T14:34:38.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:38.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.505+0000 7f83eb200640 1 Processor -- start 2024-04-23T14:34:38.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.505+0000 7f83eb200640 1 -- start start 2024-04-23T14:34:38.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.507+0000 7f83eb200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f83ec15b450 0x7f83ec0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.507+0000 7f83eb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec06f3f0 0x7f83ec0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.507+0000 7f83eb200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ec0b3870 con 0x7f83ec0b3080 2024-04-23T14:34:38.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.507+0000 7f83eb200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f83ec0b41a0 con 0x7f83ec15b450 2024-04-23T14:34:38.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.507+0000 7f83eb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec0b3b50 con 0x7f83ec06f3f0 2024-04-23T14:34:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.508+0000 7f83e9e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f83ec15b450 0x7f83ec0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.508+0000 7f83e3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec06f3f0 0x7f83ec0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.508+0000 7f83e9e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f83ec15b450 0x7f83ec0b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:42998/0 (socket says 172.21.15.59:42998) 2024-04-23T14:34:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.508+0000 7f83e3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec06f3f0 0x7f83ec0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:53948/0 (socket says 172.21.15.59:53948) 2024-04-23T14:34:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.508+0000 7f83e9e00640 1 -- 172.21.15.59:0/2334583308 learned_addr learned my addr 172.21.15.59:0/2334583308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.509+0000 7f83e9e00640 1 -- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec06f3f0 msgr2=0x7f83ec0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.509+0000 7f83e9e00640 1 --2- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec06f3f0 0x7f83ec0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.509+0000 7f83e9e00640 1 -- 172.21.15.59:0/2334583308 >> v1:172.21.15.76:6789/0 conn(0x7f83ec0b3080 legacy=0x7f83ec0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.509+0000 7f83e9e00640 1 -- 172.21.15.59:0/2334583308 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec0b3e30 con 0x7f83ec15b450 2024-04-23T14:34:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.509+0000 7f83e9400640 1 -- 172.21.15.59:0/2334583308 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1610843400 0 0) 0x7f83ec0b3870 con 0x7f83ec0b3080 2024-04-23T14:34:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.509+0000 7f83e3e00640 1 --2- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec06f3f0 0x7f83ec0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:38.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.510+0000 7f83e9e00640 1 --2- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec15b450 0x7f83ec0b4630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009870 tx=0x7f83dc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aeb1764f4efcc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.511+0000 7f83e9400640 1 -- 172.21.15.59:0/2334583308 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc037070 con 0x7f83ec15b450 2024-04-23T14:34:38.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.511+0000 7f83e9400640 1 -- 172.21.15.59:0/2334583308 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83dc032440 con 0x7f83ec15b450 2024-04-23T14:34:38.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.511+0000 7f83e9400640 1 -- 172.21.15.59:0/2334583308 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc032810 con 0x7f83ec15b450 2024-04-23T14:34:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 -- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec15b450 msgr2=0x7f83ec0b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 --2- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec15b450 0x7f83ec0b4630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009870 tx=0x7f83dc02a250 comp rx=0 tx=0).stop 2024-04-23T14:34:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 -- 172.21.15.59:0/2334583308 shutdown_connections 2024-04-23T14:34:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 --2- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec06f3f0 0x7f83ec0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 --2- 172.21.15.59:0/2334583308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec15b450 0x7f83ec0b4630 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 -- 172.21.15.59:0/2334583308 >> 172.21.15.59:0/2334583308 conn(0x7f83ec0b21a0 msgr2=0x7f83ec0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 -- 172.21.15.59:0/2334583308 shutdown_connections 2024-04-23T14:34:38.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.512+0000 7f83eb200640 1 -- 172.21.15.59:0/2334583308 wait complete. 2024-04-23T14:34:38.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.514+0000 7f83eb200640 1 Processor -- start 2024-04-23T14:34:38.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.514+0000 7f83eb200640 1 -- start start 2024-04-23T14:34:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.515+0000 7f83eb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec06f3f0 0x7f83ec0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.515+0000 7f83eb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b3080 0x7f83ec15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.515+0000 7f83eb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec15b450 0x7f83ec1f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.515+0000 7f83eb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec0b4ff0 con 0x7f83ec0b3080 2024-04-23T14:34:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.515+0000 7f83eb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec15b830 con 0x7f83ec06f3f0 2024-04-23T14:34:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.515+0000 7f83eb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec0cd590 con 0x7f83ec15b450 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b3080 0x7f83ec15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b3080 0x7f83ec15de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59954/0 (socket says 172.21.15.59:59954) 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 -- 172.21.15.59:0/164139200 learned_addr learned my addr 172.21.15.59:0/164139200 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e9e00640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec06f3f0 0x7f83ec0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83ea800640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec15b450 0x7f83ec1f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 -- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec15b450 msgr2=0x7f83ec1f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec15b450 0x7f83ec1f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 -- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec06f3f0 msgr2=0x7f83ec0c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec06f3f0 0x7f83ec0c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e3e00640 1 -- 172.21.15.59:0/164139200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec0c4c60 con 0x7f83ec0b3080 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.516+0000 7f83e9e00640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec06f3f0 0x7f83ec0c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:38.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.517+0000 7f83e3e00640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b3080 0x7f83ec15de30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f83d400ec10 tx=0x7f83d400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.517+0000 7f83e2000640 1 -- 172.21.15.59:0/164139200 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4017070 con 0x7f83ec0b3080 2024-04-23T14:34:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.517+0000 7f83e2000640 1 -- 172.21.15.59:0/164139200 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d400cc30 con 0x7f83ec0b3080 2024-04-23T14:34:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.517+0000 7f83e2000640 1 -- 172.21.15.59:0/164139200 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4013470 con 0x7f83ec0b3080 2024-04-23T14:34:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.517+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec0c4f40 con 0x7f83ec0b3080 2024-04-23T14:34:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.517+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec0c5500 con 0x7f83ec0b3080 2024-04-23T14:34:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.518+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ec04fd90 con 0x7f83ec0b3080 2024-04-23T14:34:38.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.520+0000 7f83e2000640 1 -- 172.21.15.59:0/164139200 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83d401a070 con 0x7f83ec0b3080 2024-04-23T14:34:38.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.521+0000 7f83e2000640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f83c806c8f0 0x7f83c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.521+0000 7f83e2000640 1 -- 172.21.15.59:0/164139200 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f83d408c200 con 0x7f83ec0b3080 2024-04-23T14:34:38.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.522+0000 7f83e9e00640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f83c806c8f0 0x7f83c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.522+0000 7f83e9e00640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f83c806c8f0 0x7f83c806edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.525+0000 7f83e2000640 1 -- 172.21.15.59:0/164139200 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83d405a6d0 con 0x7f83ec0b3080 2024-04-23T14:34:38.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.735+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83ec0c5930 con 0x7f83ec0b3080 2024-04-23T14:34:38.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.738+0000 7f83e2000640 1 -- 172.21.15.59:0/164139200 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16705 (secure 0 0 0) 0x7f83d4059fd0 con 0x7f83ec0b3080 2024-04-23T14:34:38.740 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:38.740 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":46,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:08.435362+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:33:45.619774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-23T14:33:46.736010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:38.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f83c806c8f0 msgr2=0x7f83c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f83c806c8f0 0x7f83c806edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc02e000 comp rx=0 tx=0).stop 2024-04-23T14:34:38.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b3080 msgr2=0x7f83ec15de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b3080 0x7f83ec15de30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f83d400ec10 tx=0x7f83d400c520 comp rx=0 tx=0).stop 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 shutdown_connections 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f83c806c8f0 0x7f83c806edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83ec15b450 0x7f83ec1f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b3080 0x7f83ec15de30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 --2- 172.21.15.59:0/164139200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83ec06f3f0 0x7f83ec0c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 >> 172.21.15.59:0/164139200 conn(0x7f83ec0b21a0 msgr2=0x7f83ec159320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 shutdown_connections 2024-04-23T14:34:38.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.741+0000 7f83eb200640 1 -- 172.21.15.59:0/164139200 wait complete. 2024-04-23T14:34:38.762 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:34:38.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.969+0000 7f1f1f200640 1 Processor -- start 2024-04-23T14:34:38.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.970+0000 7f1f1f200640 1 -- start start 2024-04-23T14:34:38.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.971+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015ea40 0x7f1f200cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.971+0000 7f1f1f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1f2015f880 0x7f1f200d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.971+0000 7f1f1f200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f200cf930 con 0x7f1f200d0f80 2024-04-23T14:34:38.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.971+0000 7f1f1f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1f200cfc10 con 0x7f1f2015f880 2024-04-23T14:34:38.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.971+0000 7f1f1f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f200cfef0 con 0x7f1f2015ea40 2024-04-23T14:34:38.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.972+0000 7f1f1e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1f2015f880 0x7f1f200d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.972+0000 7f1f1e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1f2015f880 0x7f1f200d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59960/0 (socket says 172.21.15.59:59960) 2024-04-23T14:34:38.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.972+0000 7f1f1e800640 1 -- 172.21.15.59:0/3140830766 learned_addr learned my addr 172.21.15.59:0/3140830766 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:38.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.972+0000 7f1f1d400640 1 --2- 172.21.15.59:0/3140830766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015ea40 0x7f1f200cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.972+0000 7f1f1e800640 1 -- 172.21.15.59:0/3140830766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015ea40 msgr2=0x7f1f200cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:38.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.972+0000 7f1f1e800640 1 --2- 172.21.15.59:0/3140830766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015ea40 0x7f1f200cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.972+0000 7f1f1e800640 1 -- 172.21.15.59:0/3140830766 >> v1:172.21.15.76:6789/0 conn(0x7f1f200d0f80 legacy=0x7f1f200d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:38.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.973+0000 7f1f1e800640 1 -- 172.21.15.59:0/3140830766 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f200d01d0 con 0x7f1f2015f880 2024-04-23T14:34:38.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.974+0000 7f1f1e800640 1 --2- 172.21.15.59:0/3140830766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015f880 0x7f1f200d2310 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009870 tx=0x7f1f1802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5659cc74e99657fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.974+0000 7f1f1ca00640 1 -- 172.21.15.59:0/3140830766 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18037070 con 0x7f1f2015f880 2024-04-23T14:34:38.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.974+0000 7f1f1ca00640 1 -- 172.21.15.59:0/3140830766 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f18032440 con 0x7f1f2015f880 2024-04-23T14:34:38.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.974+0000 7f1f1ca00640 1 -- 172.21.15.59:0/3140830766 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18032810 con 0x7f1f2015f880 2024-04-23T14:34:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 -- 172.21.15.59:0/3140830766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015f880 msgr2=0x7f1f200d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 --2- 172.21.15.59:0/3140830766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015f880 0x7f1f200d2310 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009870 tx=0x7f1f1802a1c0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 -- 172.21.15.59:0/3140830766 shutdown_connections 2024-04-23T14:34:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 --2- 172.21.15.59:0/3140830766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015f880 0x7f1f200d2310 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 --2- 172.21.15.59:0/3140830766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015ea40 0x7f1f200cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 -- 172.21.15.59:0/3140830766 >> 172.21.15.59:0/3140830766 conn(0x7f1f2008bf90 msgr2=0x7f1f200d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 -- 172.21.15.59:0/3140830766 shutdown_connections 2024-04-23T14:34:38.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.975+0000 7f1f1f200640 1 -- 172.21.15.59:0/3140830766 wait complete. 2024-04-23T14:34:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.977+0000 7f1f1f200640 1 Processor -- start 2024-04-23T14:34:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.977+0000 7f1f1f200640 1 -- start start 2024-04-23T14:34:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200d0f80 0x7f1f200b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015ea40 0x7f1f201508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015f880 0x7f1f200cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200d2850 con 0x7f1f2015ea40 2024-04-23T14:34:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2015fd00 con 0x7f1f200d0f80 2024-04-23T14:34:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f200d29d0 con 0x7f1f2015f880 2024-04-23T14:34:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015ea40 0x7f1f201508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015ea40 0x7f1f201508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59970/0 (socket says 172.21.15.59:59970) 2024-04-23T14:34:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.978+0000 7f1f1d400640 1 -- 172.21.15.59:0/2845477041 learned_addr learned my addr 172.21.15.59:0/2845477041 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1e800640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015f880 0x7f1f200cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1de00640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200d0f80 0x7f1f200b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1d400640 1 -- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015f880 msgr2=0x7f1f200cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1d400640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015f880 0x7f1f200cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1d400640 1 -- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200d0f80 msgr2=0x7f1f200b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1d400640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200d0f80 0x7f1f200b9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1d400640 1 -- 172.21.15.59:0/2845477041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f200ba080 con 0x7f1f2015ea40 2024-04-23T14:34:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.979+0000 7f1f1d400640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015ea40 0x7f1f201508c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c007ea0 tx=0x7f1f0c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.980+0000 7f1f16000640 1 -- 172.21.15.59:0/2845477041 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c01a070 con 0x7f1f2015ea40 2024-04-23T14:34:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.980+0000 7f1f16000640 1 -- 172.21.15.59:0/2845477041 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0c003990 con 0x7f1f2015ea40 2024-04-23T14:34:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.980+0000 7f1f16000640 1 -- 172.21.15.59:0/2845477041 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c003d40 con 0x7f1f2015ea40 2024-04-23T14:34:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.980+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f200cbc20 con 0x7f1f2015ea40 2024-04-23T14:34:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.980+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f200cc1e0 con 0x7f1f2015ea40 2024-04-23T14:34:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.981+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2004fd90 con 0x7f1f2015ea40 2024-04-23T14:34:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.982+0000 7f1f16000640 1 -- 172.21.15.59:0/2845477041 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f0c01d070 con 0x7f1f2015ea40 2024-04-23T14:34:38.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.984+0000 7f1f16000640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f0006c8f0 0x7f1f0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:38.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.984+0000 7f1f16000640 1 -- 172.21.15.59:0/2845477041 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f1f0c08bcc0 con 0x7f1f2015ea40 2024-04-23T14:34:38.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.984+0000 7f1f1de00640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f0006c8f0 0x7f1f0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.985+0000 7f1f1de00640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f0006c8f0 0x7f1f0006edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f100015a0 tx=0x7f1f10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:38.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:38.988+0000 7f1f16000640 1 -- 172.21.15.59:0/2845477041 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f0c092080 con 0x7f1f2015ea40 2024-04-23T14:34:39.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.234+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1f200d1ec0 con 0x7f1f2015ea40 2024-04-23T14:34:39.236 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:34:39.235+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:34:39.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.239+0000 7f1f16000640 1 -- 172.21.15.59:0/2845477041 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f1f0c059a50 con 0x7f1f2015ea40 2024-04-23T14:34:39.240 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:34:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f0006c8f0 msgr2=0x7f1f0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f0006c8f0 0x7f1f0006edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f100015a0 tx=0x7f1f10009040 comp rx=0 tx=0).stop 2024-04-23T14:34:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015ea40 msgr2=0x7f1f201508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015ea40 0x7f1f201508c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c007ea0 tx=0x7f1f0c007f90 comp rx=0 tx=0).stop 2024-04-23T14:34:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 shutdown_connections 2024-04-23T14:34:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f0006c8f0 0x7f1f0006edb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f2015f880 0x7f1f200cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f2015ea40 0x7f1f201508c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 --2- 172.21.15.59:0/2845477041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f200d0f80 0x7f1f200b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 >> 172.21.15.59:0/2845477041 conn(0x7f1f2008bf90 msgr2=0x7f1f2015d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 shutdown_connections 2024-04-23T14:34:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.243+0000 7f1f1f200640 1 -- 172.21.15.59:0/2845477041 wait complete. 2024-04-23T14:34:39.261 INFO:tasks.ceph.mds.a.smithi059.stderr: -5011> 2024-04-23T14:33:53.237+0000 7fc2aa400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:34:39.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:34:39.276 INFO:tasks.ceph.mds.a.smithi059.stderr: -3239> 2024-04-23T14:33:59.053+0000 7fc2a5400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5874=0-4194304@1} caps={5874=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5874 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560a15d32c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:34:39.339 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:34:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.460+0000 7fcc0c800640 1 Processor -- start 2024-04-23T14:34:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.460+0000 7fcc0c800640 1 -- start start 2024-04-23T14:34:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.461+0000 7fcc0c800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcc0415ebe0 0x7fcc040b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.461+0000 7fcc0c800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc040d05a0 0x7fcc0409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.461+0000 7fcc0c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d0c60 0x7fcc040d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.461+0000 7fcc0c800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0409af40 con 0x7fcc0415ebe0 2024-04-23T14:34:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.461+0000 7fcc0c800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0406ce80 con 0x7fcc040d05a0 2024-04-23T14:34:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.461+0000 7fcc0c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0409b170 con 0x7fcc040d0c60 2024-04-23T14:34:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcc0415ebe0 0x7fcc040b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcc0415ebe0 0x7fcc040b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59996/0 (socket says 172.21.15.59:59996) 2024-04-23T14:34:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 -- 172.21.15.59:0/3505710366 learned_addr learned my addr 172.21.15.59:0/3505710366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09400640 1 --2- 172.21.15.59:0/3505710366 >> v2:172.21.15.76:3300/0 conn(0x7fcc040d05a0 0x7fcc0409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc0a800640 1 --2- 172.21.15.59:0/3505710366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d0c60 0x7fcc040d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 -- 172.21.15.59:0/3505710366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d0c60 msgr2=0x7fcc040d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 --2- 172.21.15.59:0/3505710366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d0c60 0x7fcc040d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 -- 172.21.15.59:0/3505710366 >> v2:172.21.15.76:3300/0 conn(0x7fcc040d05a0 msgr2=0x7fcc0409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 --2- 172.21.15.59:0/3505710366 >> v2:172.21.15.76:3300/0 conn(0x7fcc040d05a0 0x7fcc0409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.462+0000 7fcc09e00640 1 -- 172.21.15.59:0/3505710366 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0409b450 con 0x7fcc0415ebe0 2024-04-23T14:34:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.464+0000 7fcc09e00640 1 --2- 172.21.15.59:0/3505710366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc0415ebe0 0x7fcc040b23d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcc0406fa70 tx=0x7fcbf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276eff8a32da27a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:39.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.464+0000 7fcc08a00640 1 -- 172.21.15.59:0/3505710366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8037070 con 0x7fcc0415ebe0 2024-04-23T14:34:39.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.464+0000 7fcc08a00640 1 -- 172.21.15.59:0/3505710366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf8032440 con 0x7fcc0415ebe0 2024-04-23T14:34:39.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.464+0000 7fcc08a00640 1 -- 172.21.15.59:0/3505710366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8032810 con 0x7fcc0415ebe0 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.464+0000 7fcc0c800640 1 -- 172.21.15.59:0/3505710366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc0415ebe0 msgr2=0x7fcc040b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.464+0000 7fcc0c800640 1 --2- 172.21.15.59:0/3505710366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc0415ebe0 0x7fcc040b23d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcc0406fa70 tx=0x7fcbf802a1e0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.465+0000 7fcc0c800640 1 -- 172.21.15.59:0/3505710366 shutdown_connections 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.465+0000 7fcc0c800640 1 --2- 172.21.15.59:0/3505710366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d0c60 0x7fcc040d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.465+0000 7fcc0c800640 1 --2- 172.21.15.59:0/3505710366 >> v2:172.21.15.76:3300/0 conn(0x7fcc040d05a0 0x7fcc0409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.465+0000 7fcc0c800640 1 --2- 172.21.15.59:0/3505710366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc0415ebe0 0x7fcc040b23d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.465+0000 7fcc0c800640 1 -- 172.21.15.59:0/3505710366 >> 172.21.15.59:0/3505710366 conn(0x7fcc040d2680 msgr2=0x7fcc040d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.465+0000 7fcc0c800640 1 -- 172.21.15.59:0/3505710366 shutdown_connections 2024-04-23T14:34:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.465+0000 7fcc0c800640 1 -- 172.21.15.59:0/3505710366 wait complete. 2024-04-23T14:34:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.466+0000 7fcc0c800640 1 Processor -- start 2024-04-23T14:34:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.467+0000 7fcc0c800640 1 -- start start 2024-04-23T14:34:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.467+0000 7fcc0c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d05a0 0x7fcc0415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc0c800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc040d0c60 0x7fcc040b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc0c800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0415ebe0 0x7fcc040b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc0c800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc040d0a00 con 0x7fcc040d0c60 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc0c800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0406d120 con 0x7fcc0415ebe0 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc0c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0406d640 con 0x7fcc040d05a0 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc040d0c60 0x7fcc040b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc040d0c60 0x7fcc040b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59998/0 (socket says 172.21.15.59:59998) 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 -- 172.21.15.59:0/2430208450 learned_addr learned my addr 172.21.15.59:0/2430208450 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09e00640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d05a0 0x7fcc0415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc0a800640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0415ebe0 0x7fcc040b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 -- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d05a0 msgr2=0x7fcc0415cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d05a0 0x7fcc0415cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 -- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0415ebe0 msgr2=0x7fcc040b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0415ebe0 0x7fcc040b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.468+0000 7fcc09400640 1 -- 172.21.15.59:0/2430208450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc040cb400 con 0x7fcc040d0c60 2024-04-23T14:34:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.469+0000 7fcc09400640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc040d0c60 0x7fcc040b51e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400d800 tx=0x7fcbf400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.469+0000 7fcbfe000640 1 -- 172.21.15.59:0/2430208450 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4018020 con 0x7fcc040d0c60 2024-04-23T14:34:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.469+0000 7fcbfe000640 1 -- 172.21.15.59:0/2430208450 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf40096a0 con 0x7fcc040d0c60 2024-04-23T14:34:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.469+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc040cb6e0 con 0x7fcc040d0c60 2024-04-23T14:34:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.469+0000 7fcbfe000640 1 -- 172.21.15.59:0/2430208450 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4009a70 con 0x7fcc040d0c60 2024-04-23T14:34:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.469+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0415d420 con 0x7fcc040d0c60 2024-04-23T14:34:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.470+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc04069190 con 0x7fcc040d0c60 2024-04-23T14:34:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.471+0000 7fcbfe000640 1 -- 172.21.15.59:0/2430208450 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbf401b020 con 0x7fcc040d0c60 2024-04-23T14:34:39.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.473+0000 7fcbfe000640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcbe806c920 0x7fcbe806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:39.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.474+0000 7fcbfe000640 1 -- 172.21.15.59:0/2430208450 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fcbf408be40 con 0x7fcc040d0c60 2024-04-23T14:34:39.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.474+0000 7fcc09e00640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcbe806c920 0x7fcbe806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:39.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.475+0000 7fcc09e00640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcbe806c920 0x7fcbe806ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8034000 tx=0x7fcbf802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:39.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.478+0000 7fcbfe000640 1 -- 172.21.15.59:0/2430208450 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbf4022030 con 0x7fcc040d0c60 2024-04-23T14:34:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:39.726+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc04075230 con 0x7fcc040d0c60 2024-04-23T14:34:40.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.242+0000 7fcbfe000640 1 -- 172.21.15.59:0/2430208450 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fcbf405a140 con 0x7fcc040d0c60 2024-04-23T14:34:40.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcbe806c920 msgr2=0x7fcbe806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:40.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcbe806c920 0x7fcbe806ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8034000 tx=0x7fcbf802e000 comp rx=0 tx=0).stop 2024-04-23T14:34:40.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc040d0c60 msgr2=0x7fcc040b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:40.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc040d0c60 0x7fcc040b51e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400d800 tx=0x7fcbf400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 shutdown_connections 2024-04-23T14:34:40.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcbe806c920 0x7fcbe806ede0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc0415ebe0 0x7fcc040b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc040d0c60 0x7fcc040b51e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 --2- 172.21.15.59:0/2430208450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc040d05a0 0x7fcc0415cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 >> 172.21.15.59:0/2430208450 conn(0x7fcc040d2680 msgr2=0x7fcc040cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.246+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 shutdown_connections 2024-04-23T14:34:40.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.247+0000 7fcc0c800640 1 -- 172.21.15.59:0/2430208450 wait complete. 2024-04-23T14:34:40.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:34:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.537+0000 7fc787e00640 1 Processor -- start 2024-04-23T14:34:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.538+0000 7fc787e00640 1 -- start start 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc787e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc780162e60 0x7fc780071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc787e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc78006eef0 0x7fc7800722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc787e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7801641e0 con 0x7fc780163ce0 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc787e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc78006e130 con 0x7fc78006eef0 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc787e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc78006e410 con 0x7fc780162e60 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc785e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc78006eef0 0x7fc7800722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc785e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc78006eef0 0x7fc7800722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55374/0 (socket says 172.21.15.59:55374) 2024-04-23T14:34:40.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc785e00640 1 -- 172.21.15.59:0/1647601322 learned_addr learned my addr 172.21.15.59:0/1647601322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:40.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.540+0000 7fc785400640 1 --2- 172.21.15.59:0/1647601322 >> v2:172.21.15.76:3300/0 conn(0x7fc780162e60 0x7fc780071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:40.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.541+0000 7fc785e00640 1 -- 172.21.15.59:0/1647601322 >> v2:172.21.15.76:3300/0 conn(0x7fc780162e60 msgr2=0x7fc780071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.541+0000 7fc785e00640 1 --2- 172.21.15.59:0/1647601322 >> v2:172.21.15.76:3300/0 conn(0x7fc780162e60 0x7fc780071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.541+0000 7fc785e00640 1 -- 172.21.15.59:0/1647601322 >> v1:172.21.15.76:6789/0 conn(0x7fc780163ce0 legacy=0x7fc7800b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.541+0000 7fc785e00640 1 -- 172.21.15.59:0/1647601322 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78006e6f0 con 0x7fc78006eef0 2024-04-23T14:34:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.541+0000 7fc785400640 1 --2- 172.21.15.59:0/1647601322 >> v2:172.21.15.76:3300/0 conn(0x7fc780162e60 0x7fc780071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.542+0000 7fc785e00640 1 --2- 172.21.15.59:0/1647601322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc78006eef0 0x7fc7800722d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc7800700e0 tx=0x7fc77802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa798b6c997ee792 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.542+0000 7fc77fe00640 1 -- 172.21.15.59:0/1647601322 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778037070 con 0x7fc78006eef0 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.542+0000 7fc77fe00640 1 -- 172.21.15.59:0/1647601322 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77802ae20 con 0x7fc78006eef0 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.542+0000 7fc77fe00640 1 -- 172.21.15.59:0/1647601322 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778032660 con 0x7fc78006eef0 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 -- 172.21.15.59:0/1647601322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc78006eef0 msgr2=0x7fc7800722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 --2- 172.21.15.59:0/1647601322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc78006eef0 0x7fc7800722d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc7800700e0 tx=0x7fc77802a110 comp rx=0 tx=0).stop 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 -- 172.21.15.59:0/1647601322 shutdown_connections 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 --2- 172.21.15.59:0/1647601322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc78006eef0 0x7fc7800722d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 --2- 172.21.15.59:0/1647601322 >> v2:172.21.15.76:3300/0 conn(0x7fc780162e60 0x7fc780071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 -- 172.21.15.59:0/1647601322 >> 172.21.15.59:0/1647601322 conn(0x7fc78006b9d0 msgr2=0x7fc78006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 -- 172.21.15.59:0/1647601322 shutdown_connections 2024-04-23T14:34:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.543+0000 7fc787e00640 1 -- 172.21.15.59:0/1647601322 wait complete. 2024-04-23T14:34:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.545+0000 7fc787e00640 1 Processor -- start 2024-04-23T14:34:40.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.545+0000 7fc787e00640 1 -- start start 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc787e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78006eef0 0x7fc780150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc787e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc780162e60 0x7fc7800cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc787e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc780163ce0 0x7fc7800d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc787e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7800afe80 con 0x7fc780163ce0 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc787e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7800afd00 con 0x7fc78006eef0 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc787e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc78006f5d0 con 0x7fc780162e60 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc785e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc780163ce0 0x7fc7800d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc785e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc780163ce0 0x7fc7800d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55378/0 (socket says 172.21.15.59:55378) 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc785e00640 1 -- 172.21.15.59:0/960170324 learned_addr learned my addr 172.21.15.59:0/960170324 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.546+0000 7fc785400640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78006eef0 0x7fc780150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc785e00640 1 -- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc780162e60 msgr2=0x7fc7800cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc785e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc780162e60 0x7fc7800cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc785e00640 1 -- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78006eef0 msgr2=0x7fc780150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc785e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78006eef0 0x7fc780150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc785e00640 1 -- 172.21.15.59:0/960170324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7800aa980 con 0x7fc780163ce0 2024-04-23T14:34:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc785400640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78006eef0 0x7fc780150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc785e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc780163ce0 0x7fc7800d0230 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc77802a150 tx=0x7fc7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc77d600640 1 -- 172.21.15.59:0/960170324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778041070 con 0x7fc780163ce0 2024-04-23T14:34:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc77d600640 1 -- 172.21.15.59:0/960170324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77803c670 con 0x7fc780163ce0 2024-04-23T14:34:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc77d600640 1 -- 172.21.15.59:0/960170324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778037040 con 0x7fc780163ce0 2024-04-23T14:34:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.547+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7800aa0e0 con 0x7fc780163ce0 2024-04-23T14:34:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.548+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7800aa6a0 con 0x7fc780163ce0 2024-04-23T14:34:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.549+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc780068a80 con 0x7fc780163ce0 2024-04-23T14:34:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.550+0000 7fc77d600640 1 -- 172.21.15.59:0/960170324 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc778044070 con 0x7fc780163ce0 2024-04-23T14:34:40.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.551+0000 7fc77d600640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76006c8f0 0x7fc76006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:40.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.551+0000 7fc77d600640 1 -- 172.21.15.59:0/960170324 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fc7780ac290 con 0x7fc780163ce0 2024-04-23T14:34:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.551+0000 7fc785400640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76006c8f0 0x7fc76006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.552+0000 7fc785400640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76006c8f0 0x7fc76006edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc77400d430 tx=0x7fc774007530 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:40.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.555+0000 7fc77d600640 1 -- 172.21.15.59:0/960170324 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc77802ae20 con 0x7fc780163ce0 2024-04-23T14:34:40.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:40.762+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc7800cc620 con 0x7fc780163ce0 2024-04-23T14:34:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.590+0000 7fc77d600640 1 -- 172.21.15.59:0/960170324 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7fc77807a620 con 0x7fc780163ce0 2024-04-23T14:34:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:34:41.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.594+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76006c8f0 msgr2=0x7fc76006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:41.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.594+0000 7fc787e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76006c8f0 0x7fc76006edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc77400d430 tx=0x7fc774007530 comp rx=0 tx=0).stop 2024-04-23T14:34:41.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.594+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc780163ce0 msgr2=0x7fc7800d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:41.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.594+0000 7fc787e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc780163ce0 0x7fc7800d0230 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc77802a150 tx=0x7fc7780046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.595+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 shutdown_connections 2024-04-23T14:34:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.595+0000 7fc787e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76006c8f0 0x7fc76006edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.595+0000 7fc787e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc780162e60 0x7fc7800cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.595+0000 7fc787e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc780163ce0 0x7fc7800d0230 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.595+0000 7fc787e00640 1 --2- 172.21.15.59:0/960170324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc78006eef0 0x7fc780150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.595+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 >> 172.21.15.59:0/960170324 conn(0x7fc78006b9d0 msgr2=0x7fc7800b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.596+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 shutdown_connections 2024-04-23T14:34:41.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.598+0000 7fc787e00640 1 -- 172.21.15.59:0/960170324 wait complete. 2024-04-23T14:34:41.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:34:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.822+0000 7fefd7e00640 1 Processor -- start 2024-04-23T14:34:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.822+0000 7fefd7e00640 1 -- start start 2024-04-23T14:34:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.823+0000 7fefd7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 0x7fefd806f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.823+0000 7fefd7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd806f7c0 con 0x7fefd8161bc0 2024-04-23T14:34:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.824+0000 7fefd7e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd806faa0 con 0x7fefd80cd110 2024-04-23T14:34:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.824+0000 7fefd7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefd806f590 con 0x7fefd8072fd0 2024-04-23T14:34:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.824+0000 7fefd6a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fefd8161bc0 0x7fefd80cca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58784/0 (socket says 172.21.15.59:58784) 2024-04-23T14:34:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.824+0000 7fefd6a00640 1 -- 172.21.15.59:0/1945417795 learned_addr learned my addr 172.21.15.59:0/1945417795 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:41.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.824+0000 7fefd7400640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 0x7fefd806f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.824+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155753425 0 0) 0x7fefd806f7c0 con 0x7fefd8161bc0 2024-04-23T14:34:41.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.824+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefbc003520 con 0x7fefd8161bc0 2024-04-23T14:34:41.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2494817404 0 0) 0x7fefbc003520 con 0x7fefd8161bc0 2024-04-23T14:34:41.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 msgr2=0x7fefd806f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:41.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 0x7fefd806f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 >> v1:172.21.15.76:6789/0 conn(0x7fefd80cd110 legacy=0x7fefd80ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd806fd80 con 0x7fefd8161bc0 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fefc8001670 con 0x7fefd8161bc0 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 >> v1:172.21.15.59:6789/0 conn(0x7fefd8161bc0 legacy=0x7fefd80cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefbc004800 0x7fefbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 0x7fefbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc00ba70 con 0x7fefbc0039e0 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc00bbf0 con 0x7fefbc004800 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc00bdf0 con 0x7fefd8072fd0 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.825+0000 7fefd6a00640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.826+0000 7fefcfe00640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefbc004800 0x7fefbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.826+0000 7fefd7400640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 0x7fefbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.826+0000 7fefd6a00640 1 -- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 msgr2=0x7fefbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.826+0000 7fefd6a00640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd8072fd0 0x7fefbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.826+0000 7fefd6a00640 1 -- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefbc004800 msgr2=0x7fefbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.826+0000 7fefd6a00640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefbc004800 0x7fefbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.826+0000 7fefd6a00640 1 -- 172.21.15.59:0/1945417795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc003520 con 0x7fefbc0039e0 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.827+0000 7fefd6a00640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c210 tx=0x7fefbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e19cb3466a9244 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.827+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8007cf0 con 0x7fefbc0039e0 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.827+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8005ad0 con 0x7fefbc0039e0 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.827+0000 7fefd6000640 1 -- 172.21.15.59:0/1945417795 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8007070 con 0x7fefbc0039e0 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.828+0000 7fefd7e00640 1 -- 172.21.15.59:0/1945417795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefbc0039e0 msgr2=0x7fefbc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.828+0000 7fefd7e00640 1 --2- 172.21.15.59:0/1945417795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefbc0039e0 0x7fefbc003e00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c210 tx=0x7fefbc005830 comp rx=0 tx=0).stop 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.828+0000 7fefd7400640 1 -- 172.21.15.59:0/1945417795 reap_dead start 2024-04-23T14:34:41.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.828+0000 7fefd7e00640 1 -- 172.21.15.59:0/1945417795 shutdown_connections 2024-04-23T14:34:41.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.828+0000 7fefd7e00640 1 -- 172.21.15.59:0/1945417795 >> 172.21.15.59:0/1945417795 conn(0x7fefd80b1f90 msgr2=0x7fefd80b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:41.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.828+0000 7fefd7e00640 1 -- 172.21.15.59:0/1945417795 shutdown_connections 2024-04-23T14:34:41.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.828+0000 7fefd7e00640 1 -- 172.21.15.59:0/1945417795 wait complete. 2024-04-23T14:34:41.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.830+0000 7fefd7e00640 1 Processor -- start 2024-04-23T14:34:41.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.830+0000 7fefd7e00640 1 -- start start 2024-04-23T14:34:41.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.831+0000 7fefd7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefbc0039e0 0x7fefd814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.831+0000 7fefd7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd80cf790 0x7fefd80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.831+0000 7fefd7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefd81f4780 0x7fefd81f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.831+0000 7fefd7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefd806f050 con 0x7fefd80cf790 2024-04-23T14:34:41.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.831+0000 7fefd7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8161fa0 con 0x7fefd81f4780 2024-04-23T14:34:41.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.831+0000 7fefd7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefd806f1d0 con 0x7fefbc0039e0 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd80cf790 0x7fefd80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd80cf790 0x7fefd80cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55386/0 (socket says 172.21.15.59:55386) 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 -- 172.21.15.59:0/2708608611 learned_addr learned my addr 172.21.15.59:0/2708608611 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefd6a00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefbc0039e0 0x7fefd814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefd7400640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefd81f4780 0x7fefd81f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 -- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefbc0039e0 msgr2=0x7fefd814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefbc0039e0 0x7fefd814ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 -- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefd81f4780 msgr2=0x7fefd81f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:41.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefd81f4780 0x7fefd81f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 -- 172.21.15.59:0/2708608611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd81f5510 con 0x7fefd80cf790 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.832+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd80cf790 0x7fefd80cfc10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fefc000e530 tx=0x7fefc000e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.833+0000 7fefcf400640 1 -- 172.21.15.59:0/2708608611 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc00052e0 con 0x7fefd80cf790 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.833+0000 7fefcf400640 1 -- 172.21.15.59:0/2708608611 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0018070 con 0x7fefd80cf790 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.833+0000 7fefcf400640 1 -- 172.21.15.59:0/2708608611 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0003810 con 0x7fefd80cf790 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.833+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd81f5c00 con 0x7fefd80cf790 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.833+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd81f61c0 con 0x7fefd80cf790 2024-04-23T14:34:41.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.834+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd80d0150 con 0x7fefd80cf790 2024-04-23T14:34:41.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.835+0000 7fefcf400640 1 -- 172.21.15.59:0/2708608611 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefc00039b0 con 0x7fefd80cf790 2024-04-23T14:34:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.836+0000 7fefcf400640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefb406c920 0x7fefb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.836+0000 7fefcf400640 1 -- 172.21.15.59:0/2708608611 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5415+0+0 (secure 0 0 0) 0x7fefc008b630 con 0x7fefd80cf790 2024-04-23T14:34:41.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.836+0000 7fefd6a00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefb406c920 0x7fefb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:41.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.837+0000 7fefd6a00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefb406c920 0x7fefb406ede0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fefc8032000 tx=0x7fefc8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:41.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:41.841+0000 7fefcf400640 1 -- 172.21.15.59:0/2708608611 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefc0059b70 con 0x7fefd80cf790 2024-04-23T14:34:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.045+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fefd804fd90 con 0x7fefd80cf790 2024-04-23T14:34:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.594+0000 7fefcf400640 1 -- 172.21.15.59:0/2708608611 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7fefc0059470 con 0x7fefd80cf790 2024-04-23T14:34:42.595 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:34:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefb406c920 msgr2=0x7fefb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefb406c920 0x7fefb406ede0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fefc8032000 tx=0x7fefc8009030 comp rx=0 tx=0).stop 2024-04-23T14:34:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd80cf790 msgr2=0x7fefd80cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd80cf790 0x7fefd80cfc10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fefc000e530 tx=0x7fefc000e9f0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 shutdown_connections 2024-04-23T14:34:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefb406c920 0x7fefb406ede0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefd81f4780 0x7fefd81f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd80cf790 0x7fefd80cfc10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 --2- 172.21.15.59:0/2708608611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefbc0039e0 0x7fefd814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.598+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 >> 172.21.15.59:0/2708608611 conn(0x7fefd80b1f90 msgr2=0x7fefd80733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.599+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 shutdown_connections 2024-04-23T14:34:42.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.599+0000 7fefd7e00640 1 -- 172.21.15.59:0/2708608611 wait complete. 2024-04-23T14:34:42.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:34:42.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.827+0000 7fac27c00640 1 Processor -- start 2024-04-23T14:34:42.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.827+0000 7fac27c00640 1 -- start start 2024-04-23T14:34:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.828+0000 7fac27c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac2015e790 0x7fac200aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.828+0000 7fac27c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20071d70 0x7fac200721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.828+0000 7fac27c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac200adc80 con 0x7fac2006f6f0 2024-04-23T14:34:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.828+0000 7fac27c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fac200aef40 con 0x7fac2015e790 2024-04-23T14:34:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.828+0000 7fac27c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fac20072720 con 0x7fac20071d70 2024-04-23T14:34:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.829+0000 7fac25200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac2015e790 0x7fac200aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:42.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.829+0000 7fac25c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20071d70 0x7fac200721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:42.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.829+0000 7fac25c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20071d70 0x7fac200721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:51238/0 (socket says 172.21.15.59:51238) 2024-04-23T14:34:42.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.829+0000 7fac25200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac2015e790 0x7fac200aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:53422/0 (socket says 172.21.15.59:53422) 2024-04-23T14:34:42.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.829+0000 7fac25c00640 1 -- 172.21.15.59:0/207029068 learned_addr learned my addr 172.21.15.59:0/207029068 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:42.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.829+0000 7fac1fe00640 1 --1- 172.21.15.59:0/207029068 >> v1:172.21.15.76:6789/0 conn(0x7fac2006f6f0 0x7fac20162e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:58184/0 (socket says 172.21.15.59:58184) 2024-04-23T14:34:42.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.830+0000 7fac25c00640 1 -- 172.21.15.59:0/207029068 >> v2:172.21.15.76:3300/0 conn(0x7fac2015e790 msgr2=0x7fac200aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.830+0000 7fac25c00640 1 --2- 172.21.15.59:0/207029068 >> v2:172.21.15.76:3300/0 conn(0x7fac2015e790 0x7fac200aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.830+0000 7fac25c00640 1 -- 172.21.15.59:0/207029068 >> v1:172.21.15.76:6789/0 conn(0x7fac2006f6f0 legacy=0x7fac20162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.830+0000 7fac25c00640 1 -- 172.21.15.59:0/207029068 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac200adf20 con 0x7fac20071d70 2024-04-23T14:34:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.830+0000 7fac25200640 1 --2- 172.21.15.59:0/207029068 >> v2:172.21.15.76:3300/0 conn(0x7fac2015e790 0x7fac200aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.832+0000 7fac25c00640 1 --2- 172.21.15.59:0/207029068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20071d70 0x7fac200721e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac18009e50 tx=0x7fac1802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0c7d73a9d0ba1c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.832+0000 7fac1f400640 1 -- 172.21.15.59:0/207029068 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18036070 con 0x7fac20071d70 2024-04-23T14:34:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.832+0000 7fac1f400640 1 -- 172.21.15.59:0/207029068 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18031600 con 0x7fac20071d70 2024-04-23T14:34:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.832+0000 7fac1f400640 1 -- 172.21.15.59:0/207029068 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180319d0 con 0x7fac20071d70 2024-04-23T14:34:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 -- 172.21.15.59:0/207029068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20071d70 msgr2=0x7fac200721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 --2- 172.21.15.59:0/207029068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20071d70 0x7fac200721e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac18009e50 tx=0x7fac1802a710 comp rx=0 tx=0).stop 2024-04-23T14:34:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 -- 172.21.15.59:0/207029068 shutdown_connections 2024-04-23T14:34:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 --2- 172.21.15.59:0/207029068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20071d70 0x7fac200721e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 --2- 172.21.15.59:0/207029068 >> v2:172.21.15.76:3300/0 conn(0x7fac2015e790 0x7fac200aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 -- 172.21.15.59:0/207029068 >> 172.21.15.59:0/207029068 conn(0x7fac20067d90 msgr2=0x7fac2006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 -- 172.21.15.59:0/207029068 shutdown_connections 2024-04-23T14:34:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.833+0000 7fac27c00640 1 -- 172.21.15.59:0/207029068 wait complete. 2024-04-23T14:34:42.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.835+0000 7fac27c00640 1 Processor -- start 2024-04-23T14:34:42.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.835+0000 7fac27c00640 1 -- start start 2024-04-23T14:34:42.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.836+0000 7fac27c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006f6f0 0x7fac20156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:42.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.836+0000 7fac27c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac20071d70 0x7fac200ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:42.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.836+0000 7fac27c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e790 0x7fac200cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:42.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.836+0000 7fac27c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac200ae4e0 con 0x7fac20071d70 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.836+0000 7fac27c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac2006fbb0 con 0x7fac2006f6f0 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.836+0000 7fac27c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fac2015eb70 con 0x7fac2015e790 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac20071d70 0x7fac200ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac20071d70 0x7fac200ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55396/0 (socket says 172.21.15.59:55396) 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 -- 172.21.15.59:0/2402318667 learned_addr learned my addr 172.21.15.59:0/2402318667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac25c00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e790 0x7fac200cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac25200640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006f6f0 0x7fac20156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 -- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e790 msgr2=0x7fac200cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e790 0x7fac200cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 -- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006f6f0 msgr2=0x7fac20156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006f6f0 0x7fac20156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.837+0000 7fac1fe00640 1 -- 172.21.15.59:0/2402318667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac200cd400 con 0x7fac20071d70 2024-04-23T14:34:42.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.838+0000 7fac1fe00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac20071d70 0x7fac200ce4a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c400 tx=0x7fac0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.838+0000 7fac1cc00640 1 -- 172.21.15.59:0/2402318667 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0074e0 con 0x7fac20071d70 2024-04-23T14:34:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.838+0000 7fac1cc00640 1 -- 172.21.15.59:0/2402318667 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c007680 con 0x7fac20071d70 2024-04-23T14:34:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.838+0000 7fac1cc00640 1 -- 172.21.15.59:0/2402318667 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004a00 con 0x7fac20071d70 2024-04-23T14:34:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.838+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac200cd6e0 con 0x7fac20071d70 2024-04-23T14:34:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.838+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac200cdca0 con 0x7fac20071d70 2024-04-23T14:34:42.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.839+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac200ce190 con 0x7fac20071d70 2024-04-23T14:34:42.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.840+0000 7fac1cc00640 1 -- 172.21.15.59:0/2402318667 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac0c01a020 con 0x7fac20071d70 2024-04-23T14:34:42.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.841+0000 7fac1cc00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac0006c8f0 0x7fac0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:42.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.841+0000 7fac1cc00640 1 -- 172.21.15.59:0/2402318667 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fac0c08b0b0 con 0x7fac20071d70 2024-04-23T14:34:42.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.841+0000 7fac25200640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac0006c8f0 0x7fac0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:42.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.842+0000 7fac25200640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac0006c8f0 0x7fac0006edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fac14009950 tx=0x7fac14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:42.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:42.847+0000 7fac1cc00640 1 -- 172.21.15.59:0/2402318667 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac0c0597e0 con 0x7fac20071d70 2024-04-23T14:34:43.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.046+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fac20068a80 con 0x7fac20071d70 2024-04-23T14:34:43.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.052+0000 7fac1cc00640 1 -- 172.21.15.59:0/2402318667 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fac0c0590e0 con 0x7fac20071d70 2024-04-23T14:34:43.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac0006c8f0 msgr2=0x7fac0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac0006c8f0 0x7fac0006edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fac14009950 tx=0x7fac14008040 comp rx=0 tx=0).stop 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac20071d70 msgr2=0x7fac200ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac20071d70 0x7fac200ce4a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c400 tx=0x7fac0c00c8c0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 shutdown_connections 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac0006c8f0 0x7fac0006edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e790 0x7fac200cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac20071d70 0x7fac200ce4a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 --2- 172.21.15.59:0/2402318667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac2006f6f0 0x7fac20156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 >> 172.21.15.59:0/2402318667 conn(0x7fac20067d90 msgr2=0x7fac200b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.054+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 shutdown_connections 2024-04-23T14:34:43.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.055+0000 7fac27c00640 1 -- 172.21.15.59:0/2402318667 wait complete. 2024-04-23T14:34:43.074 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-23T14:34:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.279+0000 7f9a90400640 1 Processor -- start 2024-04-23T14:34:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.280+0000 7f9a90400640 1 -- start start 2024-04-23T14:34:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.281+0000 7f9a90400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9a881454e0 0x7f9a88145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.281+0000 7f9a90400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880cd020 0x7f9a880b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.281+0000 7f9a90400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a880b3e40 0x7f9a88160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.281+0000 7f9a90400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9a880cddc0 con 0x7f9a880b3e40 2024-04-23T14:34:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a90400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9a880cd520 con 0x7f9a881454e0 2024-04-23T14:34:43.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a90400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9a880cd800 con 0x7f9a880cd020 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a880b3e40 0x7f9a88160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a880b3e40 0x7f9a88160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55404/0 (socket says 172.21.15.59:55404) 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 -- 172.21.15.59:0/3980528219 learned_addr learned my addr 172.21.15.59:0/3980528219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 -- 172.21.15.59:0/3980528219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880cd020 msgr2=0x7f9a880b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 --2- 172.21.15.59:0/3980528219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880cd020 0x7f9a880b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 -- 172.21.15.59:0/3980528219 >> v2:172.21.15.76:3300/0 conn(0x7f9a881454e0 msgr2=0x7f9a88145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 --2- 172.21.15.59:0/3980528219 >> v2:172.21.15.76:3300/0 conn(0x7f9a881454e0 0x7f9a88145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.282+0000 7f9a8e400640 1 -- 172.21.15.59:0/3980528219 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a880cdae0 con 0x7f9a880b3e40 2024-04-23T14:34:43.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.283+0000 7f9a8e400640 1 --2- 172.21.15.59:0/3980528219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b3e40 0x7f9a88160910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9a88073280 tx=0x7f9a8002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8364a2507aacc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.284+0000 7f9a87e00640 1 -- 172.21.15.59:0/3980528219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80036070 con 0x7f9a880b3e40 2024-04-23T14:34:43.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.284+0000 7f9a87e00640 1 -- 172.21.15.59:0/3980528219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a80031600 con 0x7f9a880b3e40 2024-04-23T14:34:43.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.284+0000 7f9a87e00640 1 -- 172.21.15.59:0/3980528219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800319d0 con 0x7f9a880b3e40 2024-04-23T14:34:43.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.284+0000 7f9a90400640 1 -- 172.21.15.59:0/3980528219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b3e40 msgr2=0x7f9a88160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.284+0000 7f9a90400640 1 --2- 172.21.15.59:0/3980528219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b3e40 0x7f9a88160910 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9a88073280 tx=0x7f9a8002a610 comp rx=0 tx=0).stop 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.285+0000 7f9a90400640 1 -- 172.21.15.59:0/3980528219 shutdown_connections 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.285+0000 7f9a90400640 1 --2- 172.21.15.59:0/3980528219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b3e40 0x7f9a88160910 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.285+0000 7f9a90400640 1 --2- 172.21.15.59:0/3980528219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880cd020 0x7f9a880b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.285+0000 7f9a90400640 1 --2- 172.21.15.59:0/3980528219 >> v2:172.21.15.76:3300/0 conn(0x7f9a881454e0 0x7f9a88145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.285+0000 7f9a90400640 1 -- 172.21.15.59:0/3980528219 >> 172.21.15.59:0/3980528219 conn(0x7f9a88067ea0 msgr2=0x7f9a8806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.285+0000 7f9a90400640 1 -- 172.21.15.59:0/3980528219 shutdown_connections 2024-04-23T14:34:43.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.285+0000 7f9a90400640 1 -- 172.21.15.59:0/3980528219 wait complete. 2024-04-23T14:34:43.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.287+0000 7f9a90400640 1 Processor -- start 2024-04-23T14:34:43.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.287+0000 7f9a90400640 1 -- start start 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a90400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880b3e40 0x7f9a88140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a90400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a880cd020 0x7f9a881e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a90400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a881454e0 0x7f9a8813e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a90400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a88146270 con 0x7f9a881454e0 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a90400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a881460f0 con 0x7f9a880cd020 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a90400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9a88160e50 con 0x7f9a880b3e40 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a881454e0 0x7f9a8813e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a881454e0 0x7f9a8813e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55406/0 (socket says 172.21.15.59:55406) 2024-04-23T14:34:43.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a8e400640 1 -- 172.21.15.59:0/862205013 learned_addr learned my addr 172.21.15.59:0/862205013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:43.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.289+0000 7f9a8d000640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a880cd020 0x7f9a881e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.290+0000 7f9a8da00640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880b3e40 0x7f9a88140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.290+0000 7f9a8e400640 1 -- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880b3e40 msgr2=0x7f9a88140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.290+0000 7f9a8e400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880b3e40 0x7f9a88140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.290+0000 7f9a8e400640 1 -- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a880cd020 msgr2=0x7f9a881e6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.290+0000 7f9a8e400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a880cd020 0x7f9a881e6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.290+0000 7f9a8e400640 1 -- 172.21.15.59:0/862205013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8813f550 con 0x7f9a881454e0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.290+0000 7f9a8e400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a881454e0 0x7f9a8813e950 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9a80034040 tx=0x7f9a80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.291+0000 7f9a85600640 1 -- 172.21.15.59:0/862205013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80041020 con 0x7f9a881454e0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.291+0000 7f9a85600640 1 -- 172.21.15.59:0/862205013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a800077a0 con 0x7f9a881454e0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.291+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8813f7f0 con 0x7f9a881454e0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.291+0000 7f9a85600640 1 -- 172.21.15.59:0/862205013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80036040 con 0x7f9a881454e0 2024-04-23T14:34:43.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.291+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8813fdb0 con 0x7f9a881454e0 2024-04-23T14:34:43.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.292+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a88068a80 con 0x7f9a881454e0 2024-04-23T14:34:43.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.293+0000 7f9a85600640 1 -- 172.21.15.59:0/862205013 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a80044020 con 0x7f9a881454e0 2024-04-23T14:34:43.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.294+0000 7f9a85600640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a6806c8f0 0x7f9a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.295+0000 7f9a85600640 1 -- 172.21.15.59:0/862205013 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f9a800ac5e0 con 0x7f9a881454e0 2024-04-23T14:34:43.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.295+0000 7f9a8da00640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a6806c8f0 0x7f9a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.296+0000 7f9a8da00640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a6806c8f0 0x7f9a6806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00b810 tx=0x7f9a7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.299+0000 7f9a85600640 1 -- 172.21.15.59:0/862205013 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a8004b030 con 0x7f9a881454e0 2024-04-23T14:34:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.577+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f9a88160030 con 0x7f9a881454e0 2024-04-23T14:34:43.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.609+0000 7f9a85600640 1 -- 172.21.15.59:0/862205013 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v188) ==== 149+0+0 (secure 0 0 0) 0x7f9a8007ad00 con 0x7f9a881454e0 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a6806c8f0 msgr2=0x7f9a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a6806c8f0 0x7f9a6806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00b810 tx=0x7f9a7c008040 comp rx=0 tx=0).stop 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a881454e0 msgr2=0x7f9a8813e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a881454e0 0x7f9a8813e950 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9a80034040 tx=0x7f9a80004a40 comp rx=0 tx=0).stop 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 shutdown_connections 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a6806c8f0 0x7f9a6806edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a881454e0 0x7f9a8813e950 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a880cd020 0x7f9a881e6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 --2- 172.21.15.59:0/862205013 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a880b3e40 0x7f9a88140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 >> 172.21.15.59:0/862205013 conn(0x7f9a88067ea0 msgr2=0x7f9a880b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:43.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.612+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 shutdown_connections 2024-04-23T14:34:43.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.613+0000 7f9a90400640 1 -- 172.21.15.59:0/862205013 wait complete. 2024-04-23T14:34:43.632 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-23T14:34:43.632 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-23T14:34:43.632 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-23T14:34:43.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:43.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.838+0000 7f7e5f200640 1 Processor -- start 2024-04-23T14:34:43.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.839+0000 7f7e5f200640 1 -- start start 2024-04-23T14:34:43.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.840+0000 7f7e5f200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7e600cd050 0x7f7e6006f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.840+0000 7f7e5f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600b3080 0x7f7e600cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.840+0000 7f7e5f200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e6015f160 con 0x7f7e6015ece0 2024-04-23T14:34:43.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.840+0000 7f7e5f200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7e600ced20 con 0x7f7e600cd050 2024-04-23T14:34:43.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.840+0000 7f7e5f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6015fcd0 con 0x7f7e600b3080 2024-04-23T14:34:43.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.841+0000 7f7e5de00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7e600cd050 0x7f7e6006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.841+0000 7f7e5e800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600b3080 0x7f7e600cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.841+0000 7f7e5de00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7e600cd050 0x7f7e6006f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:53452/0 (socket says 172.21.15.59:53452) 2024-04-23T14:34:43.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.841+0000 7f7e5de00640 1 -- 172.21.15.59:0/3758069141 learned_addr learned my addr 172.21.15.59:0/3758069141 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:43.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.842+0000 7f7e5de00640 1 -- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600b3080 msgr2=0x7f7e600cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:43.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.842+0000 7f7e5de00640 1 --2- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600b3080 0x7f7e600cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.842+0000 7f7e5de00640 1 -- 172.21.15.59:0/3758069141 >> v1:172.21.15.76:6789/0 conn(0x7f7e6015ece0 legacy=0x7f7e600cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.842+0000 7f7e5de00640 1 -- 172.21.15.59:0/3758069141 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6015f430 con 0x7f7e600cd050 2024-04-23T14:34:43.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.842+0000 7f7e5d400640 1 -- 172.21.15.59:0/3758069141 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160252031 0 0) 0x7f7e6015f160 con 0x7f7e6015ece0 2024-04-23T14:34:43.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.842+0000 7f7e5e800640 1 --2- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600b3080 0x7f7e600cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:43.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.843+0000 7f7e5de00640 1 --2- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e600cd050 0x7f7e6006f620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009870 tx=0x7f7e5002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2723ea9eb1a93e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.844+0000 7f7e5d400640 1 -- 172.21.15.59:0/3758069141 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50037070 con 0x7f7e600cd050 2024-04-23T14:34:43.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.844+0000 7f7e5d400640 1 -- 172.21.15.59:0/3758069141 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e50032440 con 0x7f7e600cd050 2024-04-23T14:34:43.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.844+0000 7f7e5d400640 1 -- 172.21.15.59:0/3758069141 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50032810 con 0x7f7e600cd050 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 -- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e600cd050 msgr2=0x7f7e6006f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 --2- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e600cd050 0x7f7e6006f620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009870 tx=0x7f7e5002a1c0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 -- 172.21.15.59:0/3758069141 shutdown_connections 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 --2- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600b3080 0x7f7e600cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 --2- 172.21.15.59:0/3758069141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e600cd050 0x7f7e6006f620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 -- 172.21.15.59:0/3758069141 >> 172.21.15.59:0/3758069141 conn(0x7f7e600b21a0 msgr2=0x7f7e600b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 -- 172.21.15.59:0/3758069141 shutdown_connections 2024-04-23T14:34:43.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.845+0000 7f7e5f200640 1 -- 172.21.15.59:0/3758069141 wait complete. 2024-04-23T14:34:43.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.847+0000 7f7e5f200640 1 Processor -- start 2024-04-23T14:34:43.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.847+0000 7f7e5f200640 1 -- start start 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.849+0000 7f7e5f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e600b3080 0x7f7e601ee530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.849+0000 7f7e5f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600cd050 0x7f7e6015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.849+0000 7f7e5f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e6015ece0 0x7f7e6015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.849+0000 7f7e5f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e600cef50 con 0x7f7e600b3080 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.849+0000 7f7e5f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6015f9f0 con 0x7f7e6015ece0 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.849+0000 7f7e5f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7e60159110 con 0x7f7e600cd050 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e600b3080 0x7f7e601ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e600b3080 0x7f7e601ee530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55422/0 (socket says 172.21.15.59:55422) 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 -- 172.21.15.59:0/1776569202 learned_addr learned my addr 172.21.15.59:0/1776569202 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5e800640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e6015ece0 0x7f7e6015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e57e00640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600cd050 0x7f7e6015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 -- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600cd050 msgr2=0x7f7e6015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600cd050 0x7f7e6015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 -- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e6015ece0 msgr2=0x7f7e6015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e6015ece0 0x7f7e6015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:43.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 -- 172.21.15.59:0/1776569202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e600cc040 con 0x7f7e600b3080 2024-04-23T14:34:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.850+0000 7f7e5de00640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e600b3080 0x7f7e601ee530 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7e50000c00 tx=0x7f7e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.851+0000 7f7e56000640 1 -- 172.21.15.59:0/1776569202 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50037040 con 0x7f7e600b3080 2024-04-23T14:34:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.851+0000 7f7e56000640 1 -- 172.21.15.59:0/1776569202 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e50042070 con 0x7f7e600b3080 2024-04-23T14:34:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.851+0000 7f7e56000640 1 -- 172.21.15.59:0/1776569202 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50030940 con 0x7f7e600b3080 2024-04-23T14:34:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.851+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e600cc320 con 0x7f7e600b3080 2024-04-23T14:34:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.851+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6009b600 con 0x7f7e600b3080 2024-04-23T14:34:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.852+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6004fd90 con 0x7f7e600b3080 2024-04-23T14:34:43.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.853+0000 7f7e56000640 1 -- 172.21.15.59:0/1776569202 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e5002ad10 con 0x7f7e600b3080 2024-04-23T14:34:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.854+0000 7f7e56000640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.855+0000 7f7e56000640 1 -- 172.21.15.59:0/1776569202 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f7e500abd50 con 0x7f7e600b3080 2024-04-23T14:34:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.855+0000 7f7e57e00640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.856+0000 7f7e57e00640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e48007fd0 tx=0x7f7e48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:43.859+0000 7f7e56000640 1 -- 172.21.15.59:0/1776569202 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e5007a470 con 0x7f7e600b3080 2024-04-23T14:34:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.061+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e600cddf0 con 0x7f7e600b3080 2024-04-23T14:34:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.063+0000 7f7e56000640 1 -- 172.21.15.59:0/1776569202 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+12960 (secure 0 0 0) 0x7f7e50079d70 con 0x7f7e600b3080 2024-04-23T14:34:44.064 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:44.064 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":49,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:42.591207+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/2804599006":"2024-04-24T14:34:39.236200+0000","172.21.15.59:6834/2804599006":"2024-04-24T14:34:39.236200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.065+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e3c06c8f0 msgr2=0x7f7e3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.065+0000 7f7e5f200640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e48007fd0 tx=0x7f7e48009040 comp rx=0 tx=0).stop 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.065+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e600b3080 msgr2=0x7f7e601ee530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.065+0000 7f7e5f200640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e600b3080 0x7f7e601ee530 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7e50000c00 tx=0x7f7e500046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 shutdown_connections 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e600cd050 0x7f7e6015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e6015ece0 0x7f7e6015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 --2- 172.21.15.59:0/1776569202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e600b3080 0x7f7e601ee530 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 >> 172.21.15.59:0/1776569202 conn(0x7f7e600b21a0 msgr2=0x7f7e600cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 shutdown_connections 2024-04-23T14:34:44.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.066+0000 7f7e5f200640 1 -- 172.21.15.59:0/1776569202 wait complete. 2024-04-23T14:34:44.086 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:44.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.290+0000 7f35d9800640 1 Processor -- start 2024-04-23T14:34:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.290+0000 7f35d9800640 1 -- start start 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.291+0000 7f35d9800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35d40d0600 0x7f35d40d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.291+0000 7f35d9800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40ae290 0x7f35d40ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.291+0000 7f35d9800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d415ef20 con 0x7f35d4155a80 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.291+0000 7f35d9800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f35d415f1c0 con 0x7f35d40d0600 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.291+0000 7f35d9800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f35d415f4a0 con 0x7f35d40ae290 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f35d4155a80 0x7f35d406f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58786/0 (socket says 172.21.15.59:58786) 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d3e00640 1 -- 172.21.15.59:0/513929568 learned_addr learned my addr 172.21.15.59:0/513929568 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d3400640 1 --2- 172.21.15.59:0/513929568 >> v2:172.21.15.59:3300/0 conn(0x7f35d40d0600 0x7f35d40d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d8e00640 1 --2- 172.21.15.59:0/513929568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40ae290 0x7f35d40ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d2a00640 1 -- 172.21.15.59:0/513929568 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797746234 0 0) 0x7f35d415ef20 con 0x7f35d4155a80 2024-04-23T14:34:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d2a00640 1 -- 172.21.15.59:0/513929568 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35b8003520 con 0x7f35d4155a80 2024-04-23T14:34:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d3400640 1 -- 172.21.15.59:0/513929568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40ae290 msgr2=0x7f35d40ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d3400640 1 --2- 172.21.15.59:0/513929568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40ae290 0x7f35d40ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.292+0000 7f35d3400640 1 -- 172.21.15.59:0/513929568 >> v1:172.21.15.59:6789/0 conn(0x7f35d4155a80 legacy=0x7f35d406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.293+0000 7f35d3400640 1 -- 172.21.15.59:0/513929568 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d415f780 con 0x7f35d40d0600 2024-04-23T14:34:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.294+0000 7f35d3400640 1 --2- 172.21.15.59:0/513929568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d0600 0x7f35d40d0a90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0097a0 tx=0x7f35bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccba72a9969543fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.294+0000 7f35d2a00640 1 -- 172.21.15.59:0/513929568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc037070 con 0x7f35d40d0600 2024-04-23T14:34:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.294+0000 7f35d2a00640 1 -- 172.21.15.59:0/513929568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35bc032440 con 0x7f35d40d0600 2024-04-23T14:34:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.294+0000 7f35d2a00640 1 -- 172.21.15.59:0/513929568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc032810 con 0x7f35d40d0600 2024-04-23T14:34:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.294+0000 7f35d9800640 1 -- 172.21.15.59:0/513929568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d0600 msgr2=0x7f35d40d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.294+0000 7f35d9800640 1 --2- 172.21.15.59:0/513929568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d0600 0x7f35d40d0a90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0097a0 tx=0x7f35bc02a180 comp rx=0 tx=0).stop 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.295+0000 7f35d9800640 1 -- 172.21.15.59:0/513929568 shutdown_connections 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.295+0000 7f35d9800640 1 --2- 172.21.15.59:0/513929568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40ae290 0x7f35d40ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.295+0000 7f35d9800640 1 --2- 172.21.15.59:0/513929568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d0600 0x7f35d40d0a90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.295+0000 7f35d9800640 1 -- 172.21.15.59:0/513929568 >> 172.21.15.59:0/513929568 conn(0x7f35d406bdf0 msgr2=0x7f35d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.295+0000 7f35d9800640 1 -- 172.21.15.59:0/513929568 shutdown_connections 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.295+0000 7f35d9800640 1 -- 172.21.15.59:0/513929568 wait complete. 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.296+0000 7f35d9800640 1 Processor -- start 2024-04-23T14:34:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.297+0000 7f35d9800640 1 -- start start 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.297+0000 7f35d9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40ae290 0x7f35d4140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.297+0000 7f35d9800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d0600 0x7f35d40cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.297+0000 7f35d9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d4155a80 0x7f35d4151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.297+0000 7f35d9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35d406ef80 con 0x7f35d4155a80 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.297+0000 7f35d9800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35d415fd40 con 0x7f35d40ae290 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.297+0000 7f35d9800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f35d406f100 con 0x7f35d40d0600 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d4155a80 0x7f35d4151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d4155a80 0x7f35d4151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55442/0 (socket says 172.21.15.59:55442) 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 -- 172.21.15.59:0/650652322 learned_addr learned my addr 172.21.15.59:0/650652322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d3e00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40ae290 0x7f35d4140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d3400640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d0600 0x7f35d40cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 -- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d0600 msgr2=0x7f35d40cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d0600 0x7f35d40cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 -- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40ae290 msgr2=0x7f35d4140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40ae290 0x7f35d4140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d8e00640 1 -- 172.21.15.59:0/650652322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d40cf550 con 0x7f35d4155a80 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d3400640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d0600 0x7f35d40cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.298+0000 7f35d3e00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40ae290 0x7f35d4140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.299+0000 7f35d8e00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d4155a80 0x7f35d4151af0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00d830 tx=0x7f35cc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.299+0000 7f35cbe00640 1 -- 172.21.15.59:0/650652322 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00b6c0 con 0x7f35d4155a80 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.299+0000 7f35cbe00640 1 -- 172.21.15.59:0/650652322 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35cc019070 con 0x7f35d4155a80 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.299+0000 7f35cbe00640 1 -- 172.21.15.59:0/650652322 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00bac0 con 0x7f35d4155a80 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.299+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d40cf830 con 0x7f35d4155a80 2024-04-23T14:34:44.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.299+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d40cf0d0 con 0x7f35d4155a80 2024-04-23T14:34:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.300+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d4068290 con 0x7f35d4155a80 2024-04-23T14:34:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.301+0000 7f35cbe00640 1 -- 172.21.15.59:0/650652322 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35cc01e460 con 0x7f35d4155a80 2024-04-23T14:34:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.302+0000 7f35cbe00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35b006c940 0x7f35b006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.302+0000 7f35cbe00640 1 -- 172.21.15.59:0/650652322 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f35cc08bdb0 con 0x7f35d4155a80 2024-04-23T14:34:44.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.303+0000 7f35d3e00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35b006c940 0x7f35b006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.303+0000 7f35d3e00640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35b006c940 0x7f35b006ee00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f35c4005c00 tx=0x7f35c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.307+0000 7f35cbe00640 1 -- 172.21.15.59:0/650652322 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35d4068290 con 0x7f35d4155a80 2024-04-23T14:34:44.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.515+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35d404fd90 con 0x7f35d4155a80 2024-04-23T14:34:44.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.516+0000 7f35cbe00640 1 -- 172.21.15.59:0/650652322 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f35cc05a650 con 0x7f35d4155a80 2024-04-23T14:34:44.517 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:34:44.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35b006c940 msgr2=0x7f35b006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35b006c940 0x7f35b006ee00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f35c4005c00 tx=0x7f35c4009040 comp rx=0 tx=0).stop 2024-04-23T14:34:44.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d4155a80 msgr2=0x7f35d4151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d4155a80 0x7f35d4151af0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00d830 tx=0x7f35cc00dcf0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 shutdown_connections 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35b006c940 0x7f35b006ee00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d0600 0x7f35d40cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d4155a80 0x7f35d4151af0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 --2- 172.21.15.59:0/650652322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40ae290 0x7f35d4140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.518+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 >> 172.21.15.59:0/650652322 conn(0x7f35d406bdf0 msgr2=0x7f35d406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.519+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 shutdown_connections 2024-04-23T14:34:44.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.519+0000 7f35d9800640 1 -- 172.21.15.59:0/650652322 wait complete. 2024-04-23T14:34:44.538 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-23T14:34:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.755+0000 7fe567200640 1 Processor -- start 2024-04-23T14:34:44.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.755+0000 7fe567200640 1 -- start start 2024-04-23T14:34:44.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.756+0000 7fe567200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe56815e7c0 0x7fe5680d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe567200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5680d0d90 0x7fe5680d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe567200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5680cf030 con 0x7fe5680d2630 2024-04-23T14:34:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe567200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe5680b7000 con 0x7fe5680d0d90 2024-04-23T14:34:44.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe567200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe5680cf620 con 0x7fe56815e7c0 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe565400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe5680d2630 0x7fe5680b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58802/0 (socket says 172.21.15.59:58802) 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe566800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5680d0d90 0x7fe5680d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe565400640 1 -- 172.21.15.59:0/181036177 learned_addr learned my addr 172.21.15.59:0/181036177 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.757+0000 7fe565e00640 1 --2- 172.21.15.59:0/181036177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe56815e7c0 0x7fe5680d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.758+0000 7fe566800640 1 -- 172.21.15.59:0/181036177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe56815e7c0 msgr2=0x7fe5680d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.758+0000 7fe566800640 1 --2- 172.21.15.59:0/181036177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe56815e7c0 0x7fe5680d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.758+0000 7fe566800640 1 -- 172.21.15.59:0/181036177 >> v1:172.21.15.59:6789/0 conn(0x7fe5680d2630 legacy=0x7fe5680b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.758+0000 7fe566800640 1 -- 172.21.15.59:0/181036177 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5680b6a40 con 0x7fe5680d0d90 2024-04-23T14:34:44.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.759+0000 7fe566800640 1 --2- 172.21.15.59:0/181036177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d0d90 0x7fe5680d1220 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe560009d90 tx=0x7fe56002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a438bd2481495f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:44.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.759+0000 7fe564a00640 1 -- 172.21.15.59:0/181036177 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560007740 con 0x7fe5680d0d90 2024-04-23T14:34:44.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.759+0000 7fe564a00640 1 -- 172.21.15.59:0/181036177 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5600078e0 con 0x7fe5680d0d90 2024-04-23T14:34:44.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.759+0000 7fe564a00640 1 -- 172.21.15.59:0/181036177 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560007cb0 con 0x7fe5680d0d90 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 -- 172.21.15.59:0/181036177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d0d90 msgr2=0x7fe5680d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 --2- 172.21.15.59:0/181036177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d0d90 0x7fe5680d1220 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe560009d90 tx=0x7fe56002a660 comp rx=0 tx=0).stop 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 -- 172.21.15.59:0/181036177 shutdown_connections 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 --2- 172.21.15.59:0/181036177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d0d90 0x7fe5680d1220 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 --2- 172.21.15.59:0/181036177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe56815e7c0 0x7fe5680d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 -- 172.21.15.59:0/181036177 >> 172.21.15.59:0/181036177 conn(0x7fe56809d920 msgr2=0x7fe56808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 -- 172.21.15.59:0/181036177 shutdown_connections 2024-04-23T14:34:44.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.760+0000 7fe567200640 1 -- 172.21.15.59:0/181036177 wait complete. 2024-04-23T14:34:44.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.762+0000 7fe567200640 1 Processor -- start 2024-04-23T14:34:44.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.762+0000 7fe567200640 1 -- start start 2024-04-23T14:34:44.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe567200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5680d0d90 0x7fe5680b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe567200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d2630 0x7fe5680cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe567200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe56815e7c0 0x7fe5680cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe567200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680b6700 con 0x7fe5680d2630 2024-04-23T14:34:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe567200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680cf2b0 con 0x7fe56815e7c0 2024-04-23T14:34:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe567200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe5680d1a80 con 0x7fe5680d0d90 2024-04-23T14:34:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe565400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d2630 0x7fe5680cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe565400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d2630 0x7fe5680cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55450/0 (socket says 172.21.15.59:55450) 2024-04-23T14:34:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe565400640 1 -- 172.21.15.59:0/729782430 learned_addr learned my addr 172.21.15.59:0/729782430 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.763+0000 7fe565e00640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5680d0d90 0x7fe5680b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe566800640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe56815e7c0 0x7fe5680cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe565400640 1 -- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5680d0d90 msgr2=0x7fe5680b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe565400640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5680d0d90 0x7fe5680b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe565400640 1 -- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe56815e7c0 msgr2=0x7fe5680cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe565400640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe56815e7c0 0x7fe5680cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe565400640 1 -- 172.21.15.59:0/729782430 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5680cd090 con 0x7fe5680d2630 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe565400640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d2630 0x7fe5680cf900 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c440 tx=0x7fe558001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.764+0000 7fe55e000640 1 -- 172.21.15.59:0/729782430 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558017070 con 0x7fe5680d2630 2024-04-23T14:34:44.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.765+0000 7fe55e000640 1 -- 172.21.15.59:0/729782430 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe558005ce0 con 0x7fe5680d2630 2024-04-23T14:34:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.765+0000 7fe55e000640 1 -- 172.21.15.59:0/729782430 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580072f0 con 0x7fe5680d2630 2024-04-23T14:34:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.765+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5680cd370 con 0x7fe5680d2630 2024-04-23T14:34:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.765+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5680cd930 con 0x7fe5680d2630 2024-04-23T14:34:44.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.766+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56804fd90 con 0x7fe5680d2630 2024-04-23T14:34:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.767+0000 7fe55e000640 1 -- 172.21.15.59:0/729782430 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe55801a070 con 0x7fe5680d2630 2024-04-23T14:34:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.768+0000 7fe55e000640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe54406c920 0x7fe54406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.768+0000 7fe55e000640 1 -- 172.21.15.59:0/729782430 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fe55808ac80 con 0x7fe5680d2630 2024-04-23T14:34:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.768+0000 7fe565e00640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe54406c920 0x7fe54406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:44.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.769+0000 7fe565e00640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe54406c920 0x7fe54406ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe5500079a0 tx=0x7fe550008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:44.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:44.773+0000 7fe55e000640 1 -- 172.21.15.59:0/729782430 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5680cf030 con 0x7fe5680d2630 2024-04-23T14:34:45.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.049+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7fe56806e8b0 con 0x7fe5680d2630 2024-04-23T14:34:45.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.624+0000 7fe55e000640 1 -- 172.21.15.59:0/729782430 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v190) ==== 172+0+0 (secure 0 0 0) 0x7fe558058c30 con 0x7fe5680d2630 2024-04-23T14:34:45.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe54406c920 msgr2=0x7fe54406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe54406c920 0x7fe54406ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe5500079a0 tx=0x7fe550008040 comp rx=0 tx=0).stop 2024-04-23T14:34:45.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d2630 msgr2=0x7fe5680cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d2630 0x7fe5680cf900 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c440 tx=0x7fe558001e70 comp rx=0 tx=0).stop 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 shutdown_connections 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe54406c920 0x7fe54406ede0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe56815e7c0 0x7fe5680cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5680d2630 0x7fe5680cf900 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 --2- 172.21.15.59:0/729782430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5680d0d90 0x7fe5680b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 >> 172.21.15.59:0/729782430 conn(0x7fe56809d920 msgr2=0x7fe56815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 shutdown_connections 2024-04-23T14:34:45.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.630+0000 7fe567200640 1 -- 172.21.15.59:0/729782430 wait complete. 2024-04-23T14:34:45.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:34:45.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.872+0000 7f2ad4600640 1 Processor -- start 2024-04-23T14:34:45.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.872+0000 7f2ad4600640 1 -- start start 2024-04-23T14:34:45.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad4600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2acc145e10 0x7f2acc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad4600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2acc0d0cc0 0x7f2acc0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad4600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2acc0cfc20 con 0x7f2acc071d70 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad4600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc0cff00 con 0x7f2acc145e10 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad4600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc0d01e0 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2acc145e10 0x7f2acc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad2600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2acc071d70 0x7f2acc15cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58818/0 (socket says 172.21.15.59:58818) 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.874+0000 7f2ad2600640 1 -- 172.21.15.59:0/4129758437 learned_addr learned my addr 172.21.15.59:0/4129758437 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.875+0000 7f2ad1c00640 1 -- 172.21.15.59:0/4129758437 >> v2:172.21.15.76:3300/0 conn(0x7f2acc0d0cc0 msgr2=0x7f2acc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.875+0000 7f2ad1c00640 1 --2- 172.21.15.59:0/4129758437 >> v2:172.21.15.76:3300/0 conn(0x7f2acc0d0cc0 0x7f2acc0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.875+0000 7f2ad1c00640 1 -- 172.21.15.59:0/4129758437 >> v1:172.21.15.59:6789/0 conn(0x7f2acc071d70 legacy=0x7f2acc15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.875+0000 7f2ad1c00640 1 -- 172.21.15.59:0/4129758437 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc0d04c0 con 0x7f2acc145e10 2024-04-23T14:34:45.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.876+0000 7f2ad1c00640 1 --2- 172.21.15.59:0/4129758437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc145e10 0x7f2acc06f130 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00097a0 tx=0x7f2ac002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41b27b4b52e8361 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.876+0000 7f2acbe00640 1 -- 172.21.15.59:0/4129758437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2acc145e10 2024-04-23T14:34:45.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.876+0000 7f2acbe00640 1 -- 172.21.15.59:0/4129758437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0032440 con 0x7f2acc145e10 2024-04-23T14:34:45.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.876+0000 7f2acbe00640 1 -- 172.21.15.59:0/4129758437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0032810 con 0x7f2acc145e10 2024-04-23T14:34:45.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 -- 172.21.15.59:0/4129758437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc145e10 msgr2=0x7f2acc06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 --2- 172.21.15.59:0/4129758437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc145e10 0x7f2acc06f130 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00097a0 tx=0x7f2ac002a1e0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 -- 172.21.15.59:0/4129758437 shutdown_connections 2024-04-23T14:34:45.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 --2- 172.21.15.59:0/4129758437 >> v2:172.21.15.76:3300/0 conn(0x7f2acc0d0cc0 0x7f2acc0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 --2- 172.21.15.59:0/4129758437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc145e10 0x7f2acc06f130 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 -- 172.21.15.59:0/4129758437 >> 172.21.15.59:0/4129758437 conn(0x7f2acc067d90 msgr2=0x7f2acc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:45.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 -- 172.21.15.59:0/4129758437 shutdown_connections 2024-04-23T14:34:45.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.877+0000 7f2ad4600640 1 -- 172.21.15.59:0/4129758437 wait complete. 2024-04-23T14:34:45.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.879+0000 7f2ad4600640 1 Processor -- start 2024-04-23T14:34:45.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.879+0000 7f2ad4600640 1 -- start start 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad4600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc071d70 0x7f2acc1407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad4600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0d0cc0 0x7f2acc1436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad4600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2acc145e10 0x7f2acc143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad4600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc0cfa00 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad4600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc0cf880 con 0x7f2acc145e10 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad4600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc1461f0 con 0x7f2acc071d70 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0d0cc0 0x7f2acc1436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0d0cc0 0x7f2acc1436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55466/0 (socket says 172.21.15.59:55466) 2024-04-23T14:34:45.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad1200640 1 -- 172.21.15.59:0/2280168454 learned_addr learned my addr 172.21.15.59:0/2280168454 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad2600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2acc145e10 0x7f2acc143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.880+0000 7f2ad1c00640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc071d70 0x7f2acc1407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad1200640 1 -- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc071d70 msgr2=0x7f2acc1407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad1200640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc071d70 0x7f2acc1407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad1200640 1 -- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2acc145e10 msgr2=0x7f2acc143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad1200640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2acc145e10 0x7f2acc143be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad1200640 1 -- 172.21.15.59:0/2280168454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc1e71a0 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad1c00640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc071d70 0x7f2acc1407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad1200640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0d0cc0 0x7f2acc1436a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800b4a0 tx=0x7f2ab800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ac9600640 1 -- 172.21.15.59:0/2280168454 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8014070 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ac9600640 1 -- 172.21.15.59:0/2280168454 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab8010610 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ac9600640 1 -- 172.21.15.59:0/2280168454 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8010a00 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.881+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc1e7480 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.882+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc1e7a40 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.883+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc068a80 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.883+0000 7f2ac9600640 1 -- 172.21.15.59:0/2280168454 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ab8017070 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.884+0000 7f2ac9600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:45.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.885+0000 7f2ac9600640 1 -- 172.21.15.59:0/2280168454 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f2ab808b2b0 con 0x7f2acc0d0cc0 2024-04-23T14:34:45.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.885+0000 7f2ad1c00640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:45.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.885+0000 7f2ad1c00640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0034000 tx=0x7f2ac002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:45.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:45.889+0000 7f2ac9600640 1 -- 172.21.15.59:0/2280168454 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ab80599d0 con 0x7f2acc0d0cc0 2024-04-23T14:34:46.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.086+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2acc06ebf0 con 0x7f2acc0d0cc0 2024-04-23T14:34:46.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.093+0000 7f2ac9600640 1 -- 172.21.15.59:0/2280168454 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2ab80592d0 con 0x7f2acc0d0cc0 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.095+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 msgr2=0x7f2aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.095+0000 7f2ad4600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0034000 tx=0x7f2ac002e000 comp rx=0 tx=0).stop 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.095+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0d0cc0 msgr2=0x7f2acc1436a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.095+0000 7f2ad4600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0d0cc0 0x7f2acc1436a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800b4a0 tx=0x7f2ab800b960 comp rx=0 tx=0).stop 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.095+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 shutdown_connections 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.095+0000 7f2ad4600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.095+0000 7f2ad4600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2acc145e10 0x7f2acc143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.096+0000 7f2ad4600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2acc0d0cc0 0x7f2acc1436a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.096+0000 7f2ad4600640 1 --2- 172.21.15.59:0/2280168454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2acc071d70 0x7f2acc1407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.096+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 >> 172.21.15.59:0/2280168454 conn(0x7f2acc067d90 msgr2=0x7f2acc0b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.096+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 shutdown_connections 2024-04-23T14:34:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.096+0000 7f2ad4600640 1 -- 172.21.15.59:0/2280168454 wait complete. 2024-04-23T14:34:46.116 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:34:46.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:46.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:34:46.338 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:34:46.338 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:34:46.338 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:46.399 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:46.399 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:34:46.399 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:46.458 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:46.458 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:34:46.458 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:46.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:34:46.521 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:34:46.521 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:34:46.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:34:46.578 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:34:46.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:46.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.800+0000 7f1010c00640 1 Processor -- start 2024-04-23T14:34:46.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.801+0000 7f1010c00640 1 -- start start 2024-04-23T14:34:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.802+0000 7f1010c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f100c15e7c0 0x7f100c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.802+0000 7f1010c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0d2630 0x7f100c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.802+0000 7f1010c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f100c0b6530 0x7f100c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.802+0000 7f1010c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f100c0b7000 con 0x7f100c0b6530 2024-04-23T14:34:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.802+0000 7f1010c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f100c0cf620 con 0x7f100c15e7c0 2024-04-23T14:34:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.802+0000 7f1010c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f100c0cf030 con 0x7f100c0d2630 2024-04-23T14:34:46.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f100c0b6530 0x7f100c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f100c0b6530 0x7f100c0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55476/0 (socket says 172.21.15.59:55476) 2024-04-23T14:34:46.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 -- 172.21.15.59:0/3662583163 learned_addr learned my addr 172.21.15.59:0/3662583163 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:46.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100aa00640 1 --2- 172.21.15.59:0/3662583163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0d2630 0x7f100c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100b400640 1 --2- 172.21.15.59:0/3662583163 >> v2:172.21.15.76:3300/0 conn(0x7f100c15e7c0 0x7f100c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 -- 172.21.15.59:0/3662583163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0d2630 msgr2=0x7f100c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:46.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 --2- 172.21.15.59:0/3662583163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0d2630 0x7f100c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 -- 172.21.15.59:0/3662583163 >> v2:172.21.15.76:3300/0 conn(0x7f100c15e7c0 msgr2=0x7f100c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:46.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 --2- 172.21.15.59:0/3662583163 >> v2:172.21.15.76:3300/0 conn(0x7f100c15e7c0 0x7f100c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.803+0000 7f100be00640 1 -- 172.21.15.59:0/3662583163 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c0b6a40 con 0x7f100c0b6530 2024-04-23T14:34:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.805+0000 7f100be00640 1 --2- 172.21.15.59:0/3662583163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c0b6530 0x7f100c0d0d90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f100c08c6d0 tx=0x7f100402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f432c5ea92034a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.805+0000 7f100a000640 1 -- 172.21.15.59:0/3662583163 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004037070 con 0x7f100c0b6530 2024-04-23T14:34:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.805+0000 7f100a000640 1 -- 172.21.15.59:0/3662583163 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1004032440 con 0x7f100c0b6530 2024-04-23T14:34:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.805+0000 7f100a000640 1 -- 172.21.15.59:0/3662583163 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004032810 con 0x7f100c0b6530 2024-04-23T14:34:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 -- 172.21.15.59:0/3662583163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c0b6530 msgr2=0x7f100c0d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 --2- 172.21.15.59:0/3662583163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c0b6530 0x7f100c0d0d90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f100c08c6d0 tx=0x7f100402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 -- 172.21.15.59:0/3662583163 shutdown_connections 2024-04-23T14:34:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 --2- 172.21.15.59:0/3662583163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c0b6530 0x7f100c0d0d90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 --2- 172.21.15.59:0/3662583163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0d2630 0x7f100c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 --2- 172.21.15.59:0/3662583163 >> v2:172.21.15.76:3300/0 conn(0x7f100c15e7c0 0x7f100c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 -- 172.21.15.59:0/3662583163 >> 172.21.15.59:0/3662583163 conn(0x7f100c09d920 msgr2=0x7f100c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 -- 172.21.15.59:0/3662583163 shutdown_connections 2024-04-23T14:34:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.806+0000 7f1010c00640 1 -- 172.21.15.59:0/3662583163 wait complete. 2024-04-23T14:34:46.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.808+0000 7f1010c00640 1 Processor -- start 2024-04-23T14:34:46.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.808+0000 7f1010c00640 1 -- start start 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.809+0000 7f1010c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0b6530 0x7f100c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.809+0000 7f1010c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100c0d2630 0x7f100c0b9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.809+0000 7f1010c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c15e7c0 0x7f100c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.809+0000 7f1010c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f100c0d1a80 con 0x7f100c15e7c0 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.809+0000 7f1010c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f100c0cf2b0 con 0x7f100c0d2630 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.809+0000 7f1010c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f100c0d1c00 con 0x7f100c0b6530 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.809+0000 7f100be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c15e7c0 0x7f100c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c15e7c0 0x7f100c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55482/0 (socket says 172.21.15.59:55482) 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 -- 172.21.15.59:0/1980619049 learned_addr learned my addr 172.21.15.59:0/1980619049 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100aa00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100c0d2630 0x7f100c0b9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100b400640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0b6530 0x7f100c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 -- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0b6530 msgr2=0x7f100c14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0b6530 0x7f100c14ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 -- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100c0d2630 msgr2=0x7f100c0b9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:46.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100c0d2630 0x7f100c0b9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:46.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 -- 172.21.15.59:0/1980619049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c14d1f0 con 0x7f100c15e7c0 2024-04-23T14:34:46.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.810+0000 7f100be00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c15e7c0 0x7f100c0cd7c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1004009960 tx=0x7f10040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.811+0000 7f1003400640 1 -- 172.21.15.59:0/1980619049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004041070 con 0x7f100c15e7c0 2024-04-23T14:34:46.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.811+0000 7f1003400640 1 -- 172.21.15.59:0/1980619049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100403c870 con 0x7f100c15e7c0 2024-04-23T14:34:46.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.811+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c14d4d0 con 0x7f100c15e7c0 2024-04-23T14:34:46.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.811+0000 7f1003400640 1 -- 172.21.15.59:0/1980619049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004037040 con 0x7f100c15e7c0 2024-04-23T14:34:46.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.811+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c14da90 con 0x7f100c15e7c0 2024-04-23T14:34:46.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.812+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100c04fd90 con 0x7f100c15e7c0 2024-04-23T14:34:46.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.813+0000 7f1003400640 1 -- 172.21.15.59:0/1980619049 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1004044070 con 0x7f100c15e7c0 2024-04-23T14:34:46.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.814+0000 7f1003400640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0fe806c8f0 0x7f0fe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:46.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.814+0000 7f1003400640 1 -- 172.21.15.59:0/1980619049 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f10040ac390 con 0x7f100c15e7c0 2024-04-23T14:34:46.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.814+0000 7f100b400640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0fe806c8f0 0x7f0fe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:46.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.815+0000 7f100b400640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0fe806c8f0 0x7f0fe806edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006fd0 tx=0x7f0ff4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:46.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:46.819+0000 7f1003400640 1 -- 172.21.15.59:0/1980619049 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f100407abd0 con 0x7f100c15e7c0 2024-04-23T14:34:47.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.064+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f100c0d1eb0 con 0x7f100c15e7c0 2024-04-23T14:34:47.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.065+0000 7f1003400640 1 -- 172.21.15.59:0/1980619049 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3860 (secure 0 0 0) 0x7f100407a4d0 con 0x7f100c15e7c0 2024-04-23T14:34:47.067 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:47.067 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2511274845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2511274845},{"type":"v1","addr":"172.21.15.59:6837","nonce":2511274845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4059582609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4059582609},{"type":"v1","addr":"172.21.15.76:6834","nonce":4059582609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5176,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/7610717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":7610717},{"type":"v1","addr":"172.21.15.76:6835","nonce":7610717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":6090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2448753697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2448753697},{"type":"v1","addr":"172.21.15.59:6835","nonce":2448753697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-04-23T14:34:47.067 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 27 2024-04-23T14:34:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0fe806c8f0 msgr2=0x7f0fe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0fe806c8f0 0x7f0fe806edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006fd0 tx=0x7f0ff4008040 comp rx=0 tx=0).stop 2024-04-23T14:34:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c15e7c0 msgr2=0x7f100c0cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c15e7c0 0x7f100c0cd7c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1004009960 tx=0x7f10040046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 shutdown_connections 2024-04-23T14:34:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0fe806c8f0 0x7f0fe806edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f100c15e7c0 0x7f100c0cd7c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100c0d2630 0x7f100c0b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 --2- 172.21.15.59:0/1980619049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f100c0b6530 0x7f100c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.068+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 >> 172.21.15.59:0/1980619049 conn(0x7f100c09d920 msgr2=0x7f100c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.069+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 shutdown_connections 2024-04-23T14:34:47.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.069+0000 7f1010c00640 1 -- 172.21.15.59:0/1980619049 wait complete. 2024-04-23T14:34:47.090 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:34:47.090 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:34:47.090 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:34:47.090 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:47.090 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:34:47.091 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:34:47.091 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:34:47.091 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:47.091 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:34:47.091 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:34:47.091 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:34:47.091 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:47.092 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:34:47.092 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:34:47.092 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:34:47.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:34:47.113 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:34:47.113 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:34:47.114 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:34:47.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:34:47.115 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:34:47.116 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:34:47.119 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:34:47.120 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:34:47.161 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:34:47.161 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:47.164 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:34:47.165 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:47.167 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:34:47.167 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:47.171 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:34:47.171 DEBUG:teuthology.parallel:result is None 2024-04-23T14:34:47.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:34:47.315 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:34:47.315 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:34:47.318 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:34:47.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-23T14:34:47.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-23T14:34:47.325 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:34:47.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.353+0000 7f0afde00640 1 Processor -- start 2024-04-23T14:34:47.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.353+0000 7f0afde00640 1 -- start start 2024-04-23T14:34:47.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.354+0000 7f0afde00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af806ddf0 0x7f0af806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.354+0000 7f0afde00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0af8072680 0x7f0af8072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.354+0000 7f0afde00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af80cd880 con 0x7f0af80cf410 2024-04-23T14:34:47.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.354+0000 7f0afde00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0af8071ec0 con 0x7f0af8072680 2024-04-23T14:34:47.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.354+0000 7f0afde00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0af80cdb20 con 0x7f0af806ddf0 2024-04-23T14:34:47.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.354+0000 7f0af6a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0af80cf410 0x7f0af815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58850/0 (socket says 172.21.15.59:58850) 2024-04-23T14:34:47.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.354+0000 7f0af6a00640 1 -- 172.21.15.59:0/1845615290 learned_addr learned my addr 172.21.15.59:0/1845615290 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:47.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0aee000640 1 --2- 172.21.15.59:0/1845615290 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af806ddf0 0x7f0af806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0af7400640 1 --2- 172.21.15.59:0/1845615290 >> v2:172.21.15.76:3300/0 conn(0x7f0af8072680 0x7f0af8072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0af6000640 1 -- 172.21.15.59:0/1845615290 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364040989 0 0) 0x7f0af80cd880 con 0x7f0af80cf410 2024-04-23T14:34:47.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0af6000640 1 -- 172.21.15.59:0/1845615290 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0adc003520 con 0x7f0af80cf410 2024-04-23T14:34:47.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0aee000640 1 -- 172.21.15.59:0/1845615290 >> v2:172.21.15.76:3300/0 conn(0x7f0af8072680 msgr2=0x7f0af8072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:47.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0aee000640 1 --2- 172.21.15.59:0/1845615290 >> v2:172.21.15.76:3300/0 conn(0x7f0af8072680 0x7f0af8072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0aee000640 1 -- 172.21.15.59:0/1845615290 >> v1:172.21.15.59:6789/0 conn(0x7f0af80cf410 legacy=0x7f0af815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0aee000640 1 -- 172.21.15.59:0/1845615290 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af8072160 con 0x7f0af806ddf0 2024-04-23T14:34:47.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.355+0000 7f0af7400640 1 --2- 172.21.15.59:0/1845615290 >> v2:172.21.15.76:3300/0 conn(0x7f0af8072680 0x7f0af8072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:34:47.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.356+0000 7f0aee000640 1 --2- 172.21.15.59:0/1845615290 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af806ddf0 0x7f0af806e250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097d0 tx=0x7f0ae002a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf53ebe88030251 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0af6000640 1 -- 172.21.15.59:0/1845615290 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0037070 con 0x7f0af806ddf0 2024-04-23T14:34:47.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0af6000640 1 -- 172.21.15.59:0/1845615290 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae002ae30 con 0x7f0af806ddf0 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0af6000640 1 -- 172.21.15.59:0/1845615290 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0032640 con 0x7f0af806ddf0 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 -- 172.21.15.59:0/1845615290 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af806ddf0 msgr2=0x7f0af806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 --2- 172.21.15.59:0/1845615290 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af806ddf0 0x7f0af806e250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097d0 tx=0x7f0ae002a120 comp rx=0 tx=0).stop 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 -- 172.21.15.59:0/1845615290 shutdown_connections 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 --2- 172.21.15.59:0/1845615290 >> v2:172.21.15.76:3300/0 conn(0x7f0af8072680 0x7f0af8072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 --2- 172.21.15.59:0/1845615290 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af806ddf0 0x7f0af806e250 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 -- 172.21.15.59:0/1845615290 >> 172.21.15.59:0/1845615290 conn(0x7f0af80b2130 msgr2=0x7f0af80b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 -- 172.21.15.59:0/1845615290 shutdown_connections 2024-04-23T14:34:47.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.357+0000 7f0afde00640 1 -- 172.21.15.59:0/1845615290 wait complete. 2024-04-23T14:34:47.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.359+0000 7f0afde00640 1 Processor -- start 2024-04-23T14:34:47.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.359+0000 7f0afde00640 1 -- start start 2024-04-23T14:34:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.360+0000 7f0afde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af806ddf0 0x7f0af81f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.360+0000 7f0afde00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af8072680 0x7f0af8145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.360+0000 7f0afde00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af80cf410 0x7f0af8148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.360+0000 7f0afde00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0af815a820 con 0x7f0af806ddf0 2024-04-23T14:34:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.360+0000 7f0afde00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0af815a6a0 con 0x7f0af8072680 2024-04-23T14:34:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.360+0000 7f0afde00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0af80cf7f0 con 0x7f0af80cf410 2024-04-23T14:34:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.360+0000 7f0af6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af806ddf0 0x7f0af81f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af806ddf0 0x7f0af81f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55512/0 (socket says 172.21.15.59:55512) 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 -- 172.21.15.59:0/1676449932 learned_addr learned my addr 172.21.15.59:0/1676449932 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af7400640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af80cf410 0x7f0af8148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0aee000640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af8072680 0x7f0af8145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 -- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af80cf410 msgr2=0x7f0af8148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af80cf410 0x7f0af8148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 -- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af8072680 msgr2=0x7f0af8145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af8072680 0x7f0af8145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 -- 172.21.15.59:0/1676449932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af8146740 con 0x7f0af806ddf0 2024-04-23T14:34:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.361+0000 7f0af6a00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af806ddf0 0x7f0af81f56d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c3c0 tx=0x7f0ae800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.362+0000 7f0aefe00640 1 -- 172.21.15.59:0/1676449932 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8017070 con 0x7f0af806ddf0 2024-04-23T14:34:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.362+0000 7f0aefe00640 1 -- 172.21.15.59:0/1676449932 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae8005ce0 con 0x7f0af806ddf0 2024-04-23T14:34:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.362+0000 7f0aefe00640 1 -- 172.21.15.59:0/1676449932 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80072d0 con 0x7f0af806ddf0 2024-04-23T14:34:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.362+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af8146a20 con 0x7f0af806ddf0 2024-04-23T14:34:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.362+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af8146fe0 con 0x7f0af806ddf0 2024-04-23T14:34:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.363+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af8068a80 con 0x7f0af806ddf0 2024-04-23T14:34:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.364+0000 7f0aefe00640 1 -- 172.21.15.59:0/1676449932 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ae801a070 con 0x7f0af806ddf0 2024-04-23T14:34:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.365+0000 7f0aefe00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.365+0000 7f0aefe00640 1 -- 172.21.15.59:0/1676449932 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f0ae808abb0 con 0x7f0af806ddf0 2024-04-23T14:34:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.365+0000 7f0aee000640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.366+0000 7f0aee000640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ad406c8f0 0x7f0ad406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0034000 tx=0x7f0ae002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.370+0000 7f0aefe00640 1 -- 172.21.15.59:0/1676449932 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ae8058bf0 con 0x7f0af806ddf0 2024-04-23T14:34:47.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:47.586+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0af80d0e10 con 0x7f0af806ddf0 2024-04-23T14:34:48.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.668+0000 7f0aefe00640 1 -- 172.21.15.59:0/1676449932 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7f0ae805cef0 con 0x7f0af806ddf0 2024-04-23T14:34:48.670 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ad406c8f0 msgr2=0x7f0ad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ad406c8f0 0x7f0ad406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0034000 tx=0x7f0ae002e000 comp rx=0 tx=0).stop 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af806ddf0 msgr2=0x7f0af81f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af806ddf0 0x7f0af81f56d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c3c0 tx=0x7f0ae800c7c0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 shutdown_connections 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ad406c8f0 0x7f0ad406edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af80cf410 0x7f0af8148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af8072680 0x7f0af8145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 --2- 172.21.15.59:0/1676449932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af806ddf0 0x7f0af81f56d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 >> 172.21.15.59:0/1676449932 conn(0x7f0af80b2130 msgr2=0x7f0af80aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 shutdown_connections 2024-04-23T14:34:48.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.672+0000 7f0afde00640 1 -- 172.21.15.59:0/1676449932 wait complete. 2024-04-23T14:34:48.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:34:48.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.907+0000 7f3d72200640 1 Processor -- start 2024-04-23T14:34:48.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.907+0000 7f3d72200640 1 -- start start 2024-04-23T14:34:48.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d72200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c06f3f0 0x7f3d6c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d72200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3d6c0b3080 0x7f3d6c0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d72200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6c0b3870 con 0x7f3d6c15b450 2024-04-23T14:34:48.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d72200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c0b41a0 con 0x7f3d6c06f3f0 2024-04-23T14:34:48.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d72200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c0b3b50 con 0x7f3d6c0b3080 2024-04-23T14:34:48.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d6be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c06f3f0 0x7f3d6c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d6be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3d6c06f3f0 0x7f3d6c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55524/0 (socket says 172.21.15.59:55524) 2024-04-23T14:34:48.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d6be00640 1 -- 172.21.15.59:0/924192399 learned_addr learned my addr 172.21.15.59:0/924192399 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:48.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.908+0000 7f3d70e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3d6c15b450 0x7f3d6c0b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58866/0 (socket says 172.21.15.59:58866) 2024-04-23T14:34:48.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.909+0000 7f3d6be00640 1 -- 172.21.15.59:0/924192399 >> v2:172.21.15.76:3300/0 conn(0x7f3d6c0b3080 msgr2=0x7f3d6c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:48.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.909+0000 7f3d71800640 1 --2- 172.21.15.59:0/924192399 >> v2:172.21.15.76:3300/0 conn(0x7f3d6c0b3080 0x7f3d6c0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.909+0000 7f3d6be00640 1 --2- 172.21.15.59:0/924192399 >> v2:172.21.15.76:3300/0 conn(0x7f3d6c0b3080 0x7f3d6c0cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.909+0000 7f3d6be00640 1 -- 172.21.15.59:0/924192399 >> v1:172.21.15.59:6789/0 conn(0x7f3d6c15b450 legacy=0x7f3d6c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.909+0000 7f3d6be00640 1 -- 172.21.15.59:0/924192399 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0b3e30 con 0x7f3d6c06f3f0 2024-04-23T14:34:48.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.909+0000 7f3d71800640 1 --2- 172.21.15.59:0/924192399 >> v2:172.21.15.76:3300/0 conn(0x7f3d6c0b3080 0x7f3d6c0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:48.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.910+0000 7f3d6be00640 1 --2- 172.21.15.59:0/924192399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c06f3f0 0x7f3d6c0cd050 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0af020 tx=0x7f3d5402c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d6a5a774d35b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.910+0000 7f3d6b400640 1 -- 172.21.15.59:0/924192399 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54038070 con 0x7f3d6c06f3f0 2024-04-23T14:34:48.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.910+0000 7f3d6b400640 1 -- 172.21.15.59:0/924192399 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d54031710 con 0x7f3d6c06f3f0 2024-04-23T14:34:48.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.910+0000 7f3d6b400640 1 -- 172.21.15.59:0/924192399 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54031ae0 con 0x7f3d6c06f3f0 2024-04-23T14:34:48.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 -- 172.21.15.59:0/924192399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c06f3f0 msgr2=0x7f3d6c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 --2- 172.21.15.59:0/924192399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c06f3f0 0x7f3d6c0cd050 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0af020 tx=0x7f3d5402c510 comp rx=0 tx=0).stop 2024-04-23T14:34:48.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 -- 172.21.15.59:0/924192399 shutdown_connections 2024-04-23T14:34:48.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 --2- 172.21.15.59:0/924192399 >> v2:172.21.15.76:3300/0 conn(0x7f3d6c0b3080 0x7f3d6c0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 --2- 172.21.15.59:0/924192399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c06f3f0 0x7f3d6c0cd050 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 -- 172.21.15.59:0/924192399 >> 172.21.15.59:0/924192399 conn(0x7f3d6c0b21a0 msgr2=0x7f3d6c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:48.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 -- 172.21.15.59:0/924192399 shutdown_connections 2024-04-23T14:34:48.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.911+0000 7f3d72200640 1 -- 172.21.15.59:0/924192399 wait complete. 2024-04-23T14:34:48.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.913+0000 7f3d72200640 1 Processor -- start 2024-04-23T14:34:48.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.913+0000 7f3d72200640 1 -- start start 2024-04-23T14:34:48.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d72200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3d6c06f3f0 0x7f3d6c0c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d72200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d6c0b3080 0x7f3d6c155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d72200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c15b450 0x7f3d6c0c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d72200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0ad060 con 0x7f3d6c15b450 2024-04-23T14:34:48.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d72200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0b4ff0 con 0x7f3d6c0b3080 2024-04-23T14:34:48.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d72200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c0ad1e0 con 0x7f3d6c06f3f0 2024-04-23T14:34:48.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d71800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c15b450 0x7f3d6c0c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d71800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c15b450 0x7f3d6c0c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55526/0 (socket says 172.21.15.59:55526) 2024-04-23T14:34:48.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.914+0000 7f3d71800640 1 -- 172.21.15.59:0/2420983843 learned_addr learned my addr 172.21.15.59:0/2420983843 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d70e00640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3d6c06f3f0 0x7f3d6c0c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d6be00640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d6c0b3080 0x7f3d6c155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d71800640 1 -- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3d6c06f3f0 msgr2=0x7f3d6c0c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d71800640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3d6c06f3f0 0x7f3d6c0c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d71800640 1 -- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d6c0b3080 msgr2=0x7f3d6c155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d71800640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d6c0b3080 0x7f3d6c155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d71800640 1 -- 172.21.15.59:0/2420983843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0b14b0 con 0x7f3d6c15b450 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d71800640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c15b450 0x7f3d6c0c56b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400cef0 tx=0x7f3d6400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d68c00640 1 -- 172.21.15.59:0/2420983843 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6401a070 con 0x7f3d6c15b450 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d68c00640 1 -- 172.21.15.59:0/2420983843 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6400d860 con 0x7f3d6c15b450 2024-04-23T14:34:48.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d68c00640 1 -- 172.21.15.59:0/2420983843 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400dc50 con 0x7f3d6c15b450 2024-04-23T14:34:48.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c0c4220 con 0x7f3d6c15b450 2024-04-23T14:34:48.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.915+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c0b11d0 con 0x7f3d6c15b450 2024-04-23T14:34:48.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.917+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c04fd90 con 0x7f3d6c15b450 2024-04-23T14:34:48.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.918+0000 7f3d68c00640 1 -- 172.21.15.59:0/2420983843 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d6401d070 con 0x7f3d6c15b450 2024-04-23T14:34:48.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.919+0000 7f3d68c00640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:48.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.919+0000 7f3d68c00640 1 -- 172.21.15.59:0/2420983843 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f3d6408c000 con 0x7f3d6c15b450 2024-04-23T14:34:48.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.919+0000 7f3d70e00640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:48.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.920+0000 7f3d70e00640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d4806c8f0 0x7f3d4806edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c000f60 tx=0x7f3d5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:48.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:48.923+0000 7f3d68c00640 1 -- 172.21.15.59:0/2420983843 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d6405a8d0 con 0x7f3d6c15b450 2024-04-23T14:34:49.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.201+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3d6c0cdfb0 con 0x7f3d6c15b450 2024-04-23T14:34:49.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.203+0000 7f3d68c00640 1 -- 172.21.15.59:0/2420983843 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f3d6405a1d0 con 0x7f3d6c15b450 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A==", 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA==", 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw==", 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg==", 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:34:49.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow==", 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ==", 2024-04-23T14:34:49.207 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ==", 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.208 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g==", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA==", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA==", 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA==", 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ==", 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-23T14:34:49.211 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ==", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q==", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:34:49.212 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg==", 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.213 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg==", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmEteHAhAAwfUHmvRmEMcPIphwikq6sw==", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:34:49.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA==", 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:34:49.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ==", 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:34:49.216 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:34:49.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d4806c8f0 msgr2=0x7f3d4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d4806c8f0 0x7f3d4806edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c000f60 tx=0x7f3d5c009040 comp rx=0 tx=0).stop 2024-04-23T14:34:49.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c15b450 msgr2=0x7f3d6c0c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c15b450 0x7f3d6c0c56b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400cef0 tx=0x7f3d6400a710 comp rx=0 tx=0).stop 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 shutdown_connections 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d6c15b450 0x7f3d6c0c56b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d6c0b3080 0x7f3d6c155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 --2- 172.21.15.59:0/2420983843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3d6c06f3f0 0x7f3d6c0c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.205+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 >> 172.21.15.59:0/2420983843 conn(0x7f3d6c0b21a0 msgr2=0x7f3d6c0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.206+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 shutdown_connections 2024-04-23T14:34:49.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.206+0000 7f3d72200640 1 -- 172.21.15.59:0/2420983843 wait complete. 2024-04-23T14:34:49.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:49.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:49.337 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:34:49.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.445+0000 7fc8e3200640 1 Processor -- start 2024-04-23T14:34:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.445+0000 7fc8e3200640 1 -- start start 2024-04-23T14:34:49.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e415c3d0 0x7fc8e40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e40cd590 0x7fc8e40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e3200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e4160210 con 0x7fc8e40b4690 2024-04-23T14:34:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e40cd9f0 con 0x7fc8e40cd590 2024-04-23T14:34:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e41604f0 con 0x7fc8e415c3d0 2024-04-23T14:34:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e1400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e40cd590 0x7fc8e40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e1400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8e40cd590 0x7fc8e40b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55534/0 (socket says 172.21.15.59:55534) 2024-04-23T14:34:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e1400640 1 -- 172.21.15.59:0/2719802570 learned_addr learned my addr 172.21.15.59:0/2719802570 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:49.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.447+0000 7fc8e1e00640 1 --2- 172.21.15.59:0/2719802570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e415c3d0 0x7fc8e40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.448+0000 7fc8e1400640 1 -- 172.21.15.59:0/2719802570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e415c3d0 msgr2=0x7fc8e40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.448+0000 7fc8e1400640 1 --2- 172.21.15.59:0/2719802570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e415c3d0 0x7fc8e40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.448+0000 7fc8e1400640 1 -- 172.21.15.59:0/2719802570 >> v1:172.21.15.76:6789/0 conn(0x7fc8e40b4690 legacy=0x7fc8e406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.448+0000 7fc8e1400640 1 -- 172.21.15.59:0/2719802570 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e41607d0 con 0x7fc8e40cd590 2024-04-23T14:34:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.448+0000 7fc8e1e00640 1 --2- 172.21.15.59:0/2719802570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e415c3d0 0x7fc8e40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:49.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.449+0000 7fc8e1400640 1 --2- 172.21.15.59:0/2719802570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40b4150 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009870 tx=0x7fc8d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b8c4cbac1d5927 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:49.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.449+0000 7fc8e0a00640 1 -- 172.21.15.59:0/2719802570 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0037070 con 0x7fc8e40cd590 2024-04-23T14:34:49.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.449+0000 7fc8e0a00640 1 -- 172.21.15.59:0/2719802570 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d0032440 con 0x7fc8e40cd590 2024-04-23T14:34:49.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.449+0000 7fc8e0a00640 1 -- 172.21.15.59:0/2719802570 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0032810 con 0x7fc8e40cd590 2024-04-23T14:34:49.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 -- 172.21.15.59:0/2719802570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 msgr2=0x7fc8e40b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 --2- 172.21.15.59:0/2719802570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40b4150 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009870 tx=0x7fc8d002a210 comp rx=0 tx=0).stop 2024-04-23T14:34:49.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 -- 172.21.15.59:0/2719802570 shutdown_connections 2024-04-23T14:34:49.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 --2- 172.21.15.59:0/2719802570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40b4150 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 --2- 172.21.15.59:0/2719802570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e415c3d0 0x7fc8e40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 -- 172.21.15.59:0/2719802570 >> 172.21.15.59:0/2719802570 conn(0x7fc8e40b21a0 msgr2=0x7fc8e40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:49.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 -- 172.21.15.59:0/2719802570 shutdown_connections 2024-04-23T14:34:49.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.450+0000 7fc8e3200640 1 -- 172.21.15.59:0/2719802570 wait complete. 2024-04-23T14:34:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.452+0000 7fc8e3200640 1 Processor -- start 2024-04-23T14:34:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.452+0000 7fc8e3200640 1 -- start start 2024-04-23T14:34:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e40b4690 0x7fc8e40ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e415c3d0 0x7fc8e40c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40b4ae0 con 0x7fc8e40cd590 2024-04-23T14:34:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e4160d90 con 0x7fc8e415c3d0 2024-04-23T14:34:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e415c7b0 con 0x7fc8e40b4690 2024-04-23T14:34:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55550/0 (socket says 172.21.15.59:55550) 2024-04-23T14:34:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.453+0000 7fc8e1400640 1 -- 172.21.15.59:0/1319364301 learned_addr learned my addr 172.21.15.59:0/1319364301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e1e00640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e40b4690 0x7fc8e40ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e1400640 1 -- 172.21.15.59:0/1319364301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e40b4690 msgr2=0x7fc8e40ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e1400640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e40b4690 0x7fc8e40ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e1400640 1 -- 172.21.15.59:0/1319364301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e415c3d0 msgr2=0x7fc8e40c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e1400640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e415c3d0 0x7fc8e40c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e1400640 1 -- 172.21.15.59:0/1319364301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e40adb30 con 0x7fc8e40cd590 2024-04-23T14:34:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e1400640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40c7c90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009870 tx=0x7fc8d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8da000640 1 -- 172.21.15.59:0/1319364301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0041020 con 0x7fc8e40cd590 2024-04-23T14:34:49.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8da000640 1 -- 172.21.15.59:0/1319364301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d00306c0 con 0x7fc8e40cd590 2024-04-23T14:34:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8da000640 1 -- 172.21.15.59:0/1319364301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0037040 con 0x7fc8e40cd590 2024-04-23T14:34:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.454+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e40ade10 con 0x7fc8e40cd590 2024-04-23T14:34:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.455+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e40ae3d0 con 0x7fc8e40cd590 2024-04-23T14:34:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.456+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e404fd90 con 0x7fc8e40cd590 2024-04-23T14:34:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.457+0000 7fc8da000640 1 -- 172.21.15.59:0/1319364301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8d0044020 con 0x7fc8e40cd590 2024-04-23T14:34:49.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.458+0000 7fc8da000640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.458+0000 7fc8da000640 1 -- 172.21.15.59:0/1319364301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fc8d00ac0a0 con 0x7fc8e40cd590 2024-04-23T14:34:49.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.458+0000 7fc8e1e00640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.459+0000 7fc8e1e00640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8c406c8f0 0x7fc8c406edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006fd0 tx=0x7fc8d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:49.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.463+0000 7fc8da000640 1 -- 172.21.15.59:0/1319364301 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8d004b030 con 0x7fc8e40cd590 2024-04-23T14:34:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.670+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8e406e400 con 0x7fc8e40cd590 2024-04-23T14:34:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.673+0000 7fc8da000640 1 -- 172.21.15.59:0/1319364301 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+12831 (secure 0 0 0) 0x7fc8d007a780 con 0x7fc8e40cd590 2024-04-23T14:34:49.674 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:49.674 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:48.665692+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.675+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8c406c8f0 msgr2=0x7fc8c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.675+0000 7fc8e3200640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8c406c8f0 0x7fc8c406edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006fd0 tx=0x7fc8d4008040 comp rx=0 tx=0).stop 2024-04-23T14:34:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.675+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 msgr2=0x7fc8e40c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.675+0000 7fc8e3200640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40c7c90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009870 tx=0x7fc8d00046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 shutdown_connections 2024-04-23T14:34:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e415c3d0 0x7fc8e40c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8e40cd590 0x7fc8e40c7c90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 --2- 172.21.15.59:0/1319364301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8e40b4690 0x7fc8e40ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 >> 172.21.15.59:0/1319364301 conn(0x7fc8e40b21a0 msgr2=0x7fc8e40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 shutdown_connections 2024-04-23T14:34:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.676+0000 7fc8e3200640 1 -- 172.21.15.59:0/1319364301 wait complete. 2024-04-23T14:34:49.697 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:34:49.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:34:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.955+0000 7fe0f3e00640 1 Processor -- start 2024-04-23T14:34:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.955+0000 7fe0f3e00640 1 -- start start 2024-04-23T14:34:49.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.957+0000 7fe0f3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0f41567e0 0x7fe0f4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.957+0000 7fe0f3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f415cab0 con 0x7fe0f415d810 2024-04-23T14:34:49.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.957+0000 7fe0f3e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f4160e40 con 0x7fe0f40d0d10 2024-04-23T14:34:49.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.957+0000 7fe0f3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f415cd30 con 0x7fe0f41567e0 2024-04-23T14:34:49.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.957+0000 7fe0f2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0f41567e0 0x7fe0f4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.958+0000 7fe0f2a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0f41567e0 0x7fe0f4156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55562/0 (socket says 172.21.15.59:55562) 2024-04-23T14:34:49.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.958+0000 7fe0f2a00640 1 -- 172.21.15.59:0/447985900 learned_addr learned my addr 172.21.15.59:0/447985900 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:49.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.958+0000 7fe0f2a00640 1 -- 172.21.15.59:0/447985900 >> v1:172.21.15.76:6789/0 conn(0x7fe0f40d0d10 legacy=0x7fe0f4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.958+0000 7fe0f2a00640 1 -- 172.21.15.59:0/447985900 >> v1:172.21.15.59:6789/0 conn(0x7fe0f415d810 legacy=0x7fe0f40d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.958+0000 7fe0f2a00640 1 -- 172.21.15.59:0/447985900 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f415d010 con 0x7fe0f41567e0 2024-04-23T14:34:49.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.959+0000 7fe0f2a00640 1 --2- 172.21.15.59:0/447985900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f41567e0 0x7fe0f4156bc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009e50 tx=0x7fe0e402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b427f0aa86563b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:49.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f2000640 1 -- 172.21.15.59:0/447985900 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4036070 con 0x7fe0f41567e0 2024-04-23T14:34:49.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f2000640 1 -- 172.21.15.59:0/447985900 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e4031600 con 0x7fe0f41567e0 2024-04-23T14:34:49.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f2000640 1 -- 172.21.15.59:0/447985900 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40319f0 con 0x7fe0f41567e0 2024-04-23T14:34:49.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f3e00640 1 -- 172.21.15.59:0/447985900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f41567e0 msgr2=0x7fe0f4156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/447985900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f41567e0 0x7fe0f4156bc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009e50 tx=0x7fe0e402a7c0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f3e00640 1 -- 172.21.15.59:0/447985900 shutdown_connections 2024-04-23T14:34:49.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/447985900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f41567e0 0x7fe0f4156bc0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f3e00640 1 -- 172.21.15.59:0/447985900 >> 172.21.15.59:0/447985900 conn(0x7fe0f406bdf0 msgr2=0x7fe0f406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:49.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f3e00640 1 -- 172.21.15.59:0/447985900 shutdown_connections 2024-04-23T14:34:49.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.960+0000 7fe0f3e00640 1 -- 172.21.15.59:0/447985900 wait complete. 2024-04-23T14:34:49.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.962+0000 7fe0f3e00640 1 Processor -- start 2024-04-23T14:34:49.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.962+0000 7fe0f3e00640 1 -- start start 2024-04-23T14:34:49.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe0f40d0d10 0x7fe0f40992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f41567e0 0x7fe0f414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f415d810 0x7fe0f408bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f41609a0 con 0x7fe0f415d810 2024-04-23T14:34:49.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f4160820 con 0x7fe0f41567e0 2024-04-23T14:34:49.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f4160b20 con 0x7fe0f40d0d10 2024-04-23T14:34:49.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f415d810 0x7fe0f408bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f415d810 0x7fe0f408bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55576/0 (socket says 172.21.15.59:55576) 2024-04-23T14:34:49.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.963+0000 7fe0f3400640 1 -- 172.21.15.59:0/3811139246 learned_addr learned my addr 172.21.15.59:0/3811139246 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:49.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0f2a00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe0f40d0d10 0x7fe0f40992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0ea000640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f41567e0 0x7fe0f414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0f3400640 1 -- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe0f40d0d10 msgr2=0x7fe0f40992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0f3400640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe0f40d0d10 0x7fe0f40992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0f3400640 1 -- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f41567e0 msgr2=0x7fe0f414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0f3400640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f41567e0 0x7fe0f414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0f3400640 1 -- 172.21.15.59:0/3811139246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f414dad0 con 0x7fe0f415d810 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.964+0000 7fe0f3400640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f415d810 0x7fe0f408bf90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c390 tx=0x7fe0ec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.965+0000 7fe0eb400640 1 -- 172.21.15.59:0/3811139246 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec017070 con 0x7fe0f415d810 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.965+0000 7fe0eb400640 1 -- 172.21.15.59:0/3811139246 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ec0052d0 con 0x7fe0f415d810 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.965+0000 7fe0eb400640 1 -- 172.21.15.59:0/3811139246 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec003810 con 0x7fe0f415d810 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.965+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f414ddb0 con 0x7fe0f415d810 2024-04-23T14:34:49.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.965+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f414e370 con 0x7fe0f415d810 2024-04-23T14:34:49.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.966+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f404fd90 con 0x7fe0f415d810 2024-04-23T14:34:49.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.967+0000 7fe0eb400640 1 -- 172.21.15.59:0/3811139246 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0ec01a070 con 0x7fe0f415d810 2024-04-23T14:34:49.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.968+0000 7fe0eb400640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:49.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.968+0000 7fe0eb400640 1 -- 172.21.15.59:0/3811139246 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fe0ec08ab20 con 0x7fe0f415d810 2024-04-23T14:34:49.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.968+0000 7fe0f2a00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:49.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.969+0000 7fe0f2a00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe0d006c8f0 0x7fe0d006edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4033000 tx=0x7fe0e4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:49.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:49.972+0000 7fe0eb400640 1 -- 172.21.15.59:0/3811139246 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe0ec0592a0 con 0x7fe0f415d810 2024-04-23T14:34:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.177+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0f40b5150 con 0x7fe0f415d810 2024-04-23T14:34:50.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.178+0000 7fe0eb400640 1 -- 172.21.15.59:0/3811139246 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7fe0ec058ba0 con 0x7fe0f415d810 2024-04-23T14:34:50.179 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:34:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.180+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe0d006c8f0 msgr2=0x7fe0d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe0d006c8f0 0x7fe0d006edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4033000 tx=0x7fe0e4009000 comp rx=0 tx=0).stop 2024-04-23T14:34:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f415d810 msgr2=0x7fe0f408bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f415d810 0x7fe0f408bf90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c390 tx=0x7fe0ec00c850 comp rx=0 tx=0).stop 2024-04-23T14:34:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 shutdown_connections 2024-04-23T14:34:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0f415d810 0x7fe0f408bf90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0f41567e0 0x7fe0f414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 --2- 172.21.15.59:0/3811139246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe0f40d0d10 0x7fe0f40992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 >> 172.21.15.59:0/3811139246 conn(0x7fe0f406bdf0 msgr2=0x7fe0f415a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 shutdown_connections 2024-04-23T14:34:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.181+0000 7fe0f3e00640 1 -- 172.21.15.59:0/3811139246 wait complete. 2024-04-23T14:34:50.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:34:50.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.406+0000 7fe34e600640 1 Processor -- start 2024-04-23T14:34:50.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.406+0000 7fe34e600640 1 -- start start 2024-04-23T14:34:50.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.407+0000 7fe34e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe34815c3d0 0x7fe3480cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.407+0000 7fe34e600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3480cd590 0x7fe3480b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.407+0000 7fe34e600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe348160210 con 0x7fe3480b4690 2024-04-23T14:34:50.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.407+0000 7fe34e600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe3481604f0 con 0x7fe34815c3d0 2024-04-23T14:34:50.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.407+0000 7fe34e600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe3481607d0 con 0x7fe3480cd590 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.408+0000 7fe34dc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe3480b4690 0x7fe34806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58044/0 (socket says 172.21.15.59:58044) 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.408+0000 7fe34dc00640 1 -- 172.21.15.59:0/4275436613 learned_addr learned my addr 172.21.15.59:0/4275436613 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.408+0000 7fe34d200640 1 --2- 172.21.15.59:0/4275436613 >> v2:172.21.15.59:3300/0 conn(0x7fe34815c3d0 0x7fe3480cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.408+0000 7fe347e00640 1 --2- 172.21.15.59:0/4275436613 >> v2:172.21.15.76:3300/0 conn(0x7fe3480cd590 0x7fe3480b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.408+0000 7fe347400640 1 -- 172.21.15.59:0/4275436613 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 324301064 0 0) 0x7fe348160210 con 0x7fe3480b4690 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.408+0000 7fe347400640 1 -- 172.21.15.59:0/4275436613 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe32c003520 con 0x7fe3480b4690 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.409+0000 7fe34d200640 1 -- 172.21.15.59:0/4275436613 >> v2:172.21.15.76:3300/0 conn(0x7fe3480cd590 msgr2=0x7fe3480b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.409+0000 7fe34d200640 1 --2- 172.21.15.59:0/4275436613 >> v2:172.21.15.76:3300/0 conn(0x7fe3480cd590 0x7fe3480b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.409+0000 7fe34d200640 1 -- 172.21.15.59:0/4275436613 >> v1:172.21.15.59:6789/0 conn(0x7fe3480b4690 legacy=0x7fe34806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.409+0000 7fe34d200640 1 -- 172.21.15.59:0/4275436613 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe348160ab0 con 0x7fe34815c3d0 2024-04-23T14:34:50.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.410+0000 7fe34d200640 1 --2- 172.21.15.59:0/4275436613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe3480cd050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe3480cff90 tx=0x7fe33002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764723b317d6ac4f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:50.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.410+0000 7fe347400640 1 -- 172.21.15.59:0/4275436613 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330037070 con 0x7fe34815c3d0 2024-04-23T14:34:50.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.410+0000 7fe347400640 1 -- 172.21.15.59:0/4275436613 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330032440 con 0x7fe34815c3d0 2024-04-23T14:34:50.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.410+0000 7fe347400640 1 -- 172.21.15.59:0/4275436613 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3300327f0 con 0x7fe34815c3d0 2024-04-23T14:34:50.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 -- 172.21.15.59:0/4275436613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 msgr2=0x7fe3480cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 --2- 172.21.15.59:0/4275436613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe3480cd050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe3480cff90 tx=0x7fe33002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 -- 172.21.15.59:0/4275436613 shutdown_connections 2024-04-23T14:34:50.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 --2- 172.21.15.59:0/4275436613 >> v2:172.21.15.76:3300/0 conn(0x7fe3480cd590 0x7fe3480b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 --2- 172.21.15.59:0/4275436613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe3480cd050 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 -- 172.21.15.59:0/4275436613 >> 172.21.15.59:0/4275436613 conn(0x7fe3480b21a0 msgr2=0x7fe3480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:50.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 -- 172.21.15.59:0/4275436613 shutdown_connections 2024-04-23T14:34:50.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.411+0000 7fe34e600640 1 -- 172.21.15.59:0/4275436613 wait complete. 2024-04-23T14:34:50.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.413+0000 7fe34e600640 1 Processor -- start 2024-04-23T14:34:50.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.413+0000 7fe34e600640 1 -- start start 2024-04-23T14:34:50.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34e600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3480b4690 0x7fe34815d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34e600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3480cd590 0x7fe3480c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe348156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe34815c7b0 con 0x7fe34815c3d0 2024-04-23T14:34:50.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34e600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe3480b4ac0 con 0x7fe3480b4690 2024-04-23T14:34:50.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34e600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe34815c930 con 0x7fe3480cd590 2024-04-23T14:34:50.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe348156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe348156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52970/0 (socket says 172.21.15.59:52970) 2024-04-23T14:34:50.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.414+0000 7fe34dc00640 1 -- 172.21.15.59:0/3152259684 learned_addr learned my addr 172.21.15.59:0/3152259684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34dc00640 1 -- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3480cd590 msgr2=0x7fe3480c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe347e00640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3480cd590 0x7fe3480c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34dc00640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3480cd590 0x7fe3480c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34dc00640 1 -- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3480b4690 msgr2=0x7fe34815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34d200640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3480b4690 0x7fe34815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34dc00640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3480b4690 0x7fe34815d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34dc00640 1 -- 172.21.15.59:0/3152259684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3480c8740 con 0x7fe34815c3d0 2024-04-23T14:34:50.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34dc00640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe348156050 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe34000c430 tx=0x7fe34000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:50.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe344c00640 1 -- 172.21.15.59:0/3152259684 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340017070 con 0x7fe34815c3d0 2024-04-23T14:34:50.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe344c00640 1 -- 172.21.15.59:0/3152259684 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe340005ce0 con 0x7fe34815c3d0 2024-04-23T14:34:50.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34815e160 con 0x7fe34815c3d0 2024-04-23T14:34:50.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.415+0000 7fe344c00640 1 -- 172.21.15.59:0/3152259684 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340007310 con 0x7fe34815c3d0 2024-04-23T14:34:50.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.416+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe348156590 con 0x7fe34815c3d0 2024-04-23T14:34:50.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.417+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34804fd90 con 0x7fe34815c3d0 2024-04-23T14:34:50.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.418+0000 7fe344c00640 1 -- 172.21.15.59:0/3152259684 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe34001a070 con 0x7fe34815c3d0 2024-04-23T14:34:50.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.419+0000 7fe344c00640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe32406c8f0 0x7fe32406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.419+0000 7fe344c00640 1 -- 172.21.15.59:0/3152259684 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fe34008ace0 con 0x7fe34815c3d0 2024-04-23T14:34:50.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.419+0000 7fe34d200640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe32406c8f0 0x7fe32406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.420+0000 7fe34d200640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe32406c8f0 0x7fe32406edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe330034000 tx=0x7fe33002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:50.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.423+0000 7fe344c00640 1 -- 172.21.15.59:0/3152259684 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe340059450 con 0x7fe34815c3d0 2024-04-23T14:34:50.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.629+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe34806e400 con 0x7fe34815c3d0 2024-04-23T14:34:50.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.630+0000 7fe344c00640 1 -- 172.21.15.59:0/3152259684 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7fe340058d50 con 0x7fe34815c3d0 2024-04-23T14:34:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:50.631 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:34:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:34:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:34:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:34:50.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:34:50.632 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.632+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe32406c8f0 msgr2=0x7fe32406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.632+0000 7fe34e600640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe32406c8f0 0x7fe32406edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe330034000 tx=0x7fe33002e000 comp rx=0 tx=0).stop 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.632+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 msgr2=0x7fe348156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.632+0000 7fe34e600640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe348156050 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe34000c430 tx=0x7fe34000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.632+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 shutdown_connections 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.633+0000 7fe34e600640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe32406c8f0 0x7fe32406edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.633+0000 7fe34e600640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3480cd590 0x7fe3480c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.633+0000 7fe34e600640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe34815c3d0 0x7fe348156050 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.633+0000 7fe34e600640 1 --2- 172.21.15.59:0/3152259684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3480b4690 0x7fe34815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.633+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 >> 172.21.15.59:0/3152259684 conn(0x7fe3480b21a0 msgr2=0x7fe3480b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.633+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 shutdown_connections 2024-04-23T14:34:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.633+0000 7fe34e600640 1 -- 172.21.15.59:0/3152259684 wait complete. 2024-04-23T14:34:50.652 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:34:50.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:34:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.866+0000 7f2543e00640 1 Processor -- start 2024-04-23T14:34:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.866+0000 7f2543e00640 1 -- start start 2024-04-23T14:34:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.868+0000 7f2543e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2544162a60 0x7f25440ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.868+0000 7f2543e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25440ad870 0x7f25440cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.868+0000 7f2543e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254406ba70 con 0x7f254415e7e0 2024-04-23T14:34:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.868+0000 7f2543e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f254406c050 con 0x7f25440ad870 2024-04-23T14:34:50.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.868+0000 7f2543e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f254415ebc0 con 0x7f2544162a60 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2543400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25440ad870 0x7f25440cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2543400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25440ad870 0x7f25440cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52972/0 (socket says 172.21.15.59:52972) 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2543400640 1 -- 172.21.15.59:0/416884682 learned_addr learned my addr 172.21.15.59:0/416884682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2542000640 1 --2- 172.21.15.59:0/416884682 >> v2:172.21.15.76:3300/0 conn(0x7f2544162a60 0x7f25440ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2543400640 1 -- 172.21.15.59:0/416884682 >> v2:172.21.15.76:3300/0 conn(0x7f2544162a60 msgr2=0x7f25440ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2543400640 1 --2- 172.21.15.59:0/416884682 >> v2:172.21.15.76:3300/0 conn(0x7f2544162a60 0x7f25440ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2543400640 1 -- 172.21.15.59:0/416884682 >> v1:172.21.15.76:6789/0 conn(0x7f254415e7e0 legacy=0x7f2544162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:50.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.869+0000 7f2543400640 1 -- 172.21.15.59:0/416884682 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2544163120 con 0x7f25440ad870 2024-04-23T14:34:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.870+0000 7f2543400640 1 --2- 172.21.15.59:0/416884682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25440ad870 0x7f25440cf090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f25441622b0 tx=0x7f253c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b13027d8dd220a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.871+0000 7f2541600640 1 -- 172.21.15.59:0/416884682 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c036070 con 0x7f25440ad870 2024-04-23T14:34:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.871+0000 7f2541600640 1 -- 172.21.15.59:0/416884682 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c031600 con 0x7f25440ad870 2024-04-23T14:34:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.871+0000 7f2541600640 1 -- 172.21.15.59:0/416884682 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0319d0 con 0x7f25440ad870 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 -- 172.21.15.59:0/416884682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25440ad870 msgr2=0x7f25440cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 --2- 172.21.15.59:0/416884682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25440ad870 0x7f25440cf090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f25441622b0 tx=0x7f253c02a6f0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 -- 172.21.15.59:0/416884682 shutdown_connections 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 --2- 172.21.15.59:0/416884682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25440ad870 0x7f25440cf090 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 --2- 172.21.15.59:0/416884682 >> v2:172.21.15.76:3300/0 conn(0x7f2544162a60 0x7f25440ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 -- 172.21.15.59:0/416884682 >> 172.21.15.59:0/416884682 conn(0x7f25440b2830 msgr2=0x7f25440b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 -- 172.21.15.59:0/416884682 shutdown_connections 2024-04-23T14:34:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.872+0000 7f2543e00640 1 -- 172.21.15.59:0/416884682 wait complete. 2024-04-23T14:34:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.874+0000 7f2543e00640 1 Processor -- start 2024-04-23T14:34:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.874+0000 7f2543e00640 1 -- start start 2024-04-23T14:34:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25440ad870 0x7f2544161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254415e7e0 0x7f25441f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2544162a60 0x7f25441f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f254406f790 con 0x7f2544162a60 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f254406bce0 con 0x7f254415e7e0 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f254406f910 con 0x7f25440ad870 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2544162a60 0x7f25441f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2544162a60 0x7f25441f88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52980/0 (socket says 172.21.15.59:52980) 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2543400640 1 -- 172.21.15.59:0/959896313 learned_addr learned my addr 172.21.15.59:0/959896313 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2542000640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254415e7e0 0x7f25441f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.875+0000 7f2542a00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25440ad870 0x7f2544161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543400640 1 -- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25440ad870 msgr2=0x7f2544161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543400640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25440ad870 0x7f2544161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543400640 1 -- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254415e7e0 msgr2=0x7f25441f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543400640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254415e7e0 0x7f25441f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543400640 1 -- 172.21.15.59:0/959896313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25441f5fd0 con 0x7f2544162a60 2024-04-23T14:34:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2542000640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254415e7e0 0x7f25441f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543400640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2544162a60 0x7f25441f88a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f253c009f40 tx=0x7f253c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f253aa00640 1 -- 172.21.15.59:0/959896313 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c036050 con 0x7f2544162a60 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f253aa00640 1 -- 172.21.15.59:0/959896313 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c042070 con 0x7f2544162a60 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2544162110 con 0x7f2544162a60 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f253aa00640 1 -- 172.21.15.59:0/959896313 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c009510 con 0x7f2544162a60 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.876+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441f6240 con 0x7f2544162a60 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.878+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25440d07c0 con 0x7f2544162a60 2024-04-23T14:34:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.879+0000 7f253aa00640 1 -- 172.21.15.59:0/959896313 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f253c007bf0 con 0x7f2544162a60 2024-04-23T14:34:50.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.880+0000 7f253aa00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f252406c8f0 0x7f252406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:50.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.880+0000 7f253aa00640 1 -- 172.21.15.59:0/959896313 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f253c0ac620 con 0x7f2544162a60 2024-04-23T14:34:50.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.880+0000 7f2542a00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f252406c8f0 0x7f252406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:50.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.881+0000 7f2542a00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f252406c8f0 0x7f252406edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2534003d90 tx=0x7f2534009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:50.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:50.884+0000 7f253aa00640 1 -- 172.21.15.59:0/959896313 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25440d07c0 con 0x7f2544162a60 2024-04-23T14:34:51.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.088+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f25440ce930 con 0x7f2544162a60 2024-04-23T14:34:51.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.697+0000 7f253aa00640 1 -- 172.21.15.59:0/959896313 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f2544162110 con 0x7f2544162a60 2024-04-23T14:34:51.699 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.701+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f252406c8f0 msgr2=0x7f252406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.701+0000 7f2543e00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f252406c8f0 0x7f252406edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2534003d90 tx=0x7f2534009040 comp rx=0 tx=0).stop 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.701+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2544162a60 msgr2=0x7f25441f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.701+0000 7f2543e00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2544162a60 0x7f25441f88a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f253c009f40 tx=0x7f253c004a40 comp rx=0 tx=0).stop 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 shutdown_connections 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f252406c8f0 0x7f252406edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2544162a60 0x7f25441f88a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254415e7e0 0x7f25441f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 --2- 172.21.15.59:0/959896313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25440ad870 0x7f2544161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 >> 172.21.15.59:0/959896313 conn(0x7f25440b2830 msgr2=0x7f25440adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:51.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 shutdown_connections 2024-04-23T14:34:51.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.702+0000 7f2543e00640 1 -- 172.21.15.59:0/959896313 wait complete. 2024-04-23T14:34:51.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:34:51.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.949+0000 7fd127e00640 1 Processor -- start 2024-04-23T14:34:51.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.949+0000 7fd127e00640 1 -- start start 2024-04-23T14:34:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.950+0000 7fd127e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 0x7fd12806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.951+0000 7fd127e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd12806f640 con 0x7fd128159150 2024-04-23T14:34:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.951+0000 7fd127e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd12806e2d0 con 0x7fd128162b20 2024-04-23T14:34:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.951+0000 7fd127e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd12806f1b0 con 0x7fd128162740 2024-04-23T14:34:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.951+0000 7fd127400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd128159150 0x7fd1280b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58054/0 (socket says 172.21.15.59:58054) 2024-04-23T14:34:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.951+0000 7fd127400640 1 -- 172.21.15.59:0/3148486715 learned_addr learned my addr 172.21.15.59:0/3148486715 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:51.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.951+0000 7fd126a00640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 0x7fd12806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626894415 0 0) 0x7fd12806f640 con 0x7fd128159150 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd10c003520 con 0x7fd128159150 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1133250962 0 0) 0x7fd10c003520 con 0x7fd128159150 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 msgr2=0x7fd12806dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 0x7fd12806dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 >> v1:172.21.15.76:6789/0 conn(0x7fd128162b20 legacy=0x7fd128158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12806e5a0 con 0x7fd128159150 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fd1200019f0 con 0x7fd128159150 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 >> v1:172.21.15.59:6789/0 conn(0x7fd128159150 legacy=0x7fd1280b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd10c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd10c004800 0x7fd10c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 0x7fd10c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd12806f1b0 con 0x7fd10c0039e0 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c00ba70 con 0x7fd10c004800 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.952+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd10c00bc70 con 0x7fd128162740 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd126a00640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd10c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd126000640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd10c004800 0x7fd10c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd127400640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 0x7fd10c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd126a00640 1 -- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 msgr2=0x7fd10c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd126a00640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 0x7fd10c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd126a00640 1 -- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd10c004800 msgr2=0x7fd10c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd126a00640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd10c004800 0x7fd10c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd126a00640 1 -- 172.21.15.59:0/3148486715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c003520 con 0x7fd10c0039e0 2024-04-23T14:34:51.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.953+0000 7fd127400640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128162740 0x7fd10c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.955+0000 7fd126a00640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd10c003e00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd118009a90 tx=0x7fd10c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365e55f1fd406905 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.955+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118036070 con 0x7fd10c0039e0 2024-04-23T14:34:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.955+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd118031400 con 0x7fd10c0039e0 2024-04-23T14:34:51.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.955+0000 7fd125600640 1 -- 172.21.15.59:0/3148486715 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118009690 con 0x7fd10c0039e0 2024-04-23T14:34:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.956+0000 7fd127e00640 1 -- 172.21.15.59:0/3148486715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 msgr2=0x7fd10c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.956+0000 7fd127e00640 1 --2- 172.21.15.59:0/3148486715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd10c003e00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd118009a90 tx=0x7fd10c005830 comp rx=0 tx=0).stop 2024-04-23T14:34:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.956+0000 7fd127400640 1 -- 172.21.15.59:0/3148486715 reap_dead start 2024-04-23T14:34:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.956+0000 7fd127e00640 1 -- 172.21.15.59:0/3148486715 shutdown_connections 2024-04-23T14:34:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.956+0000 7fd127e00640 1 -- 172.21.15.59:0/3148486715 >> 172.21.15.59:0/3148486715 conn(0x7fd12806bbb0 msgr2=0x7fd12806bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.956+0000 7fd127e00640 1 -- 172.21.15.59:0/3148486715 shutdown_connections 2024-04-23T14:34:51.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.956+0000 7fd127e00640 1 -- 172.21.15.59:0/3148486715 wait complete. 2024-04-23T14:34:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.958+0000 7fd127e00640 1 Processor -- start 2024-04-23T14:34:51.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.959+0000 7fd127e00640 1 -- start start 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd127e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd128158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd127e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1280c5ee0 0x7fd1280c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd127e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1280cc1c0 0x7fd1280cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd127e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd12806ee40 con 0x7fd10c0039e0 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd127e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd12806eb60 con 0x7fd1280cc1c0 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd127e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd128072c10 con 0x7fd1280c5ee0 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd126a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd128158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd126a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd128158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52996/0 (socket says 172.21.15.59:52996) 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.960+0000 7fd126a00640 1 -- 172.21.15.59:0/3534956474 learned_addr learned my addr 172.21.15.59:0/3534956474 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126000640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1280c5ee0 0x7fd1280c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd127400640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1280cc1c0 0x7fd1280cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126a00640 1 -- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1280c5ee0 msgr2=0x7fd1280c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126a00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1280c5ee0 0x7fd1280c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126a00640 1 -- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1280cc1c0 msgr2=0x7fd1280cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126a00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1280cc1c0 0x7fd1280cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126a00640 1 -- 172.21.15.59:0/3534956474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1280c4c20 con 0x7fd10c0039e0 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126000640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1280c5ee0 0x7fd1280c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd127400640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1280cc1c0 0x7fd1280cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:51.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.961+0000 7fd126a00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd128158510 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd118009bb0 tx=0x7fd118004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.962+0000 7fd11ea00640 1 -- 172.21.15.59:0/3534956474 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118040020 con 0x7fd10c0039e0 2024-04-23T14:34:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.962+0000 7fd11ea00640 1 -- 172.21.15.59:0/3534956474 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd118003940 con 0x7fd10c0039e0 2024-04-23T14:34:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.962+0000 7fd11ea00640 1 -- 172.21.15.59:0/3534956474 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118036040 con 0x7fd10c0039e0 2024-04-23T14:34:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.962+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1280c7c50 con 0x7fd10c0039e0 2024-04-23T14:34:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.962+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1280c5180 con 0x7fd10c0039e0 2024-04-23T14:34:51.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.963+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12804fd90 con 0x7fd10c0039e0 2024-04-23T14:34:51.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.964+0000 7fd11ea00640 1 -- 172.21.15.59:0/3534956474 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd118043020 con 0x7fd10c0039e0 2024-04-23T14:34:51.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.965+0000 7fd11ea00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd10806c8f0 0x7fd10806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:51.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.965+0000 7fd11ea00640 1 -- 172.21.15.59:0/3534956474 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7fd1180abba0 con 0x7fd10c0039e0 2024-04-23T14:34:51.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.966+0000 7fd126000640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd10806c8f0 0x7fd10806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:51.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.966+0000 7fd126000640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd10806c8f0 0x7fd10806edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd1280b1800 tx=0x7fd114009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:51.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:51.971+0000 7fd11ea00640 1 -- 172.21.15.59:0/3534956474 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd11804a070 con 0x7fd10c0039e0 2024-04-23T14:34:52.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.176+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd1280ce7d0 con 0x7fd10c0039e0 2024-04-23T14:34:52.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.714+0000 7fd11ea00640 1 -- 172.21.15.59:0/3534956474 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7fd1280c4c20 con 0x7fd10c0039e0 2024-04-23T14:34:52.716 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:34:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd10806c8f0 msgr2=0x7fd10806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd10806c8f0 0x7fd10806edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd1280b1800 tx=0x7fd114009040 comp rx=0 tx=0).stop 2024-04-23T14:34:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 msgr2=0x7fd128158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:52.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd128158510 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd118009bb0 tx=0x7fd118004730 comp rx=0 tx=0).stop 2024-04-23T14:34:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 shutdown_connections 2024-04-23T14:34:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd10806c8f0 0x7fd10806edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1280c5ee0 0x7fd1280c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1280cc1c0 0x7fd1280cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 --2- 172.21.15.59:0/3534956474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd10c0039e0 0x7fd128158510 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.719+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 >> 172.21.15.59:0/3534956474 conn(0x7fd12806bbb0 msgr2=0x7fd128161080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.722+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 shutdown_connections 2024-04-23T14:34:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.722+0000 7fd127e00640 1 -- 172.21.15.59:0/3534956474 wait complete. 2024-04-23T14:34:52.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:34:52.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.941+0000 7fa300e00640 1 Processor -- start 2024-04-23T14:34:52.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.941+0000 7fa300e00640 1 -- start start 2024-04-23T14:34:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.942+0000 7fa300e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2fc15ed20 0x7fa2fc0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.942+0000 7fa300e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0d13e0 0x7fa2fc0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.942+0000 7fa300e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2fc0b5da0 con 0x7fa2fc0b3600 2024-04-23T14:34:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.942+0000 7fa300e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa2fc15f100 con 0x7fa2fc15ed20 2024-04-23T14:34:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.942+0000 7fa300e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa2fc0d07d0 con 0x7fa2fc0d13e0 2024-04-23T14:34:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fb400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2fc15ed20 0x7fa2fc0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fb400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2fc15ed20 0x7fa2fc0d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53012/0 (socket says 172.21.15.59:53012) 2024-04-23T14:34:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fb400640 1 -- 172.21.15.59:0/2190483715 learned_addr learned my addr 172.21.15.59:0/2190483715 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fbe00640 1 --2- 172.21.15.59:0/2190483715 >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0d13e0 0x7fa2fc0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fb400640 1 -- 172.21.15.59:0/2190483715 >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0d13e0 msgr2=0x7fa2fc0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fb400640 1 --2- 172.21.15.59:0/2190483715 >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0d13e0 0x7fa2fc0d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fb400640 1 -- 172.21.15.59:0/2190483715 >> v1:172.21.15.76:6789/0 conn(0x7fa2fc0b3600 legacy=0x7fa2fc0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fb400640 1 -- 172.21.15.59:0/2190483715 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc0d0a20 con 0x7fa2fc15ed20 2024-04-23T14:34:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.943+0000 7fa2fbe00640 1 --2- 172.21.15.59:0/2190483715 >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0d13e0 0x7fa2fc0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.944+0000 7fa2fb400640 1 --2- 172.21.15.59:0/2190483715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc15ed20 0x7fa2fc0d0ea0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc08e630 tx=0x7fa2ec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47737520bc50ccc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.945+0000 7fa2fa000640 1 -- 172.21.15.59:0/2190483715 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec037070 con 0x7fa2fc15ed20 2024-04-23T14:34:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.945+0000 7fa2fa000640 1 -- 172.21.15.59:0/2190483715 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec02ae20 con 0x7fa2fc15ed20 2024-04-23T14:34:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.945+0000 7fa2fa000640 1 -- 172.21.15.59:0/2190483715 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec032660 con 0x7fa2fc15ed20 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.945+0000 7fa300e00640 1 -- 172.21.15.59:0/2190483715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc15ed20 msgr2=0x7fa2fc0d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.945+0000 7fa300e00640 1 --2- 172.21.15.59:0/2190483715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc15ed20 0x7fa2fc0d0ea0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc08e630 tx=0x7fa2ec02a110 comp rx=0 tx=0).stop 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.946+0000 7fa300e00640 1 -- 172.21.15.59:0/2190483715 shutdown_connections 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.946+0000 7fa300e00640 1 --2- 172.21.15.59:0/2190483715 >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0d13e0 0x7fa2fc0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.946+0000 7fa300e00640 1 --2- 172.21.15.59:0/2190483715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc15ed20 0x7fa2fc0d0ea0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.946+0000 7fa300e00640 1 -- 172.21.15.59:0/2190483715 >> 172.21.15.59:0/2190483715 conn(0x7fa2fc08bf90 msgr2=0x7fa2fc0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.946+0000 7fa300e00640 1 -- 172.21.15.59:0/2190483715 shutdown_connections 2024-04-23T14:34:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.946+0000 7fa300e00640 1 -- 172.21.15.59:0/2190483715 wait complete. 2024-04-23T14:34:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.947+0000 7fa300e00640 1 Processor -- start 2024-04-23T14:34:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.948+0000 7fa300e00640 1 -- start start 2024-04-23T14:34:52.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.948+0000 7fa300e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc0b3600 0x7fa2fc0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:52.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa300e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0d13e0 0x7fa2fc0b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa300e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2fc15ed20 0x7fa2fc153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa300e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc0d3490 con 0x7fa2fc0b3600 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa300e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc0d31b0 con 0x7fa2fc0d13e0 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa300e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa2fc0b4620 con 0x7fa2fc15ed20 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc0b3600 0x7fa2fc0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc0b3600 0x7fa2fc0b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53014/0 (socket says 172.21.15.59:53014) 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 -- 172.21.15.59:0/4083372527 learned_addr learned my addr 172.21.15.59:0/4083372527 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2faa00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0d13e0 0x7fa2fc0b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fbe00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2fc15ed20 0x7fa2fc153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 -- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2fc15ed20 msgr2=0x7fa2fc153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2fc15ed20 0x7fa2fc153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 -- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0d13e0 msgr2=0x7fa2fc0b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0d13e0 0x7fa2fc0b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.949+0000 7fa2fb400640 1 -- 172.21.15.59:0/4083372527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc15c260 con 0x7fa2fc0b3600 2024-04-23T14:34:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.950+0000 7fa2fb400640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc0b3600 0x7fa2fc0b6090 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000c00 tx=0x7fa2ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:52.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.950+0000 7fa2f3400640 1 -- 172.21.15.59:0/4083372527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec041070 con 0x7fa2fc0b3600 2024-04-23T14:34:52.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.950+0000 7fa2f3400640 1 -- 172.21.15.59:0/4083372527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec03c670 con 0x7fa2fc0b3600 2024-04-23T14:34:52.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.950+0000 7fa2f3400640 1 -- 172.21.15.59:0/4083372527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec037040 con 0x7fa2fc0b3600 2024-04-23T14:34:52.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.950+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc0b6670 con 0x7fa2fc0b3600 2024-04-23T14:34:52.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.950+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc0b6c30 con 0x7fa2fc0b3600 2024-04-23T14:34:52.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.951+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2fc04fd90 con 0x7fa2fc0b3600 2024-04-23T14:34:52.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.953+0000 7fa2f3400640 1 -- 172.21.15.59:0/4083372527 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2ec044070 con 0x7fa2fc0b3600 2024-04-23T14:34:52.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.954+0000 7fa2f3400640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2d806c8f0 0x7fa2d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:52.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.954+0000 7fa2f3400640 1 -- 172.21.15.59:0/4083372527 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7fa2ec0ac380 con 0x7fa2fc0b3600 2024-04-23T14:34:52.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.954+0000 7fa2faa00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2d806c8f0 0x7fa2d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:52.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.955+0000 7fa2faa00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2d806c8f0 0x7fa2d806edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc1f7bb0 tx=0x7fa2e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:52.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:52.959+0000 7fa2f3400640 1 -- 172.21.15.59:0/4083372527 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2fc15c260 con 0x7fa2fc0b3600 2024-04-23T14:34:53.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.203+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa2fc0d0350 con 0x7fa2fc0b3600 2024-04-23T14:34:53.736 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 8 and data pool 9 2024-04-23T14:34:53.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.734+0000 7fa2f3400640 1 -- 172.21.15.59:0/4083372527 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) ==== 172+0+0 (secure 0 0 0) 0x7fa2ec02ae20 con 0x7fa2fc0b3600 2024-04-23T14:34:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.746+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2d806c8f0 msgr2=0x7fa2d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.746+0000 7fa300e00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2d806c8f0 0x7fa2d806edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc1f7bb0 tx=0x7fa2e4009040 comp rx=0 tx=0).stop 2024-04-23T14:34:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.746+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc0b3600 msgr2=0x7fa2fc0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:53.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.746+0000 7fa300e00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc0b3600 0x7fa2fc0b6090 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000c00 tx=0x7fa2ec0046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.747+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 shutdown_connections 2024-04-23T14:34:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.747+0000 7fa300e00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa2d806c8f0 0x7fa2d806edb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.747+0000 7fa300e00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2fc15ed20 0x7fa2fc153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.747+0000 7fa300e00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0d13e0 0x7fa2fc0b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.747+0000 7fa300e00640 1 --2- 172.21.15.59:0/4083372527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2fc0b3600 0x7fa2fc0b6090 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.747+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 >> 172.21.15.59:0/4083372527 conn(0x7fa2fc08bf90 msgr2=0x7fa2fc0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:53.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.747+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 shutdown_connections 2024-04-23T14:34:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.748+0000 7fa300e00640 1 -- 172.21.15.59:0/4083372527 wait complete. 2024-04-23T14:34:53.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.961+0000 7f6d0f600640 1 Processor -- start 2024-04-23T14:34:53.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.961+0000 7f6d0f600640 1 -- start start 2024-04-23T14:34:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0f600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d081454e0 0x7f6d08145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0f600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d080cd020 0x7f6d080b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0f600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d080cd520 con 0x7f6d080b3e40 2024-04-23T14:34:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0f600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6d080cddc0 con 0x7f6d081454e0 2024-04-23T14:34:53.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0f600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6d080cd800 con 0x7f6d080cd020 2024-04-23T14:34:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d081454e0 0x7f6d08145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0d600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6d080b3e40 0x7f6d08160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58066/0 (socket says 172.21.15.59:58066) 2024-04-23T14:34:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0d600640 1 -- 172.21.15.59:0/944553171 learned_addr learned my addr 172.21.15.59:0/944553171 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.963+0000 7f6d0cc00640 1 --2- 172.21.15.59:0/944553171 >> v2:172.21.15.59:3300/0 conn(0x7f6d081454e0 0x7f6d08145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53022/0 (socket says 172.21.15.59:53022) 2024-04-23T14:34:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.964+0000 7f6d07e00640 1 --2- 172.21.15.59:0/944553171 >> v2:172.21.15.76:3300/0 conn(0x7f6d080cd020 0x7f6d080b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.964+0000 7f6d0cc00640 1 -- 172.21.15.59:0/944553171 >> v2:172.21.15.76:3300/0 conn(0x7f6d080cd020 msgr2=0x7f6d080b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.964+0000 7f6d0cc00640 1 --2- 172.21.15.59:0/944553171 >> v2:172.21.15.76:3300/0 conn(0x7f6d080cd020 0x7f6d080b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.964+0000 7f6d0cc00640 1 -- 172.21.15.59:0/944553171 >> v1:172.21.15.59:6789/0 conn(0x7f6d080b3e40 legacy=0x7f6d08160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.964+0000 7f6d0cc00640 1 -- 172.21.15.59:0/944553171 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d080cdae0 con 0x7f6d081454e0 2024-04-23T14:34:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.964+0000 7f6d07400640 1 -- 172.21.15.59:0/944553171 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494805775 0 0) 0x7f6d080cd520 con 0x7f6d080b3e40 2024-04-23T14:34:53.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.965+0000 7f6d0cc00640 1 --2- 172.21.15.59:0/944553171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d081454e0 0x7f6d08145bb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6d080d1490 tx=0x7f6cf002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfbd907c0df477a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.965+0000 7f6d07400640 1 -- 172.21.15.59:0/944553171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0037070 con 0x7f6d081454e0 2024-04-23T14:34:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.965+0000 7f6d07400640 1 -- 172.21.15.59:0/944553171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf0032440 con 0x7f6d081454e0 2024-04-23T14:34:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.965+0000 7f6d07400640 1 -- 172.21.15.59:0/944553171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf002ae60 con 0x7f6d081454e0 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 -- 172.21.15.59:0/944553171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d081454e0 msgr2=0x7f6d08145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 --2- 172.21.15.59:0/944553171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d081454e0 0x7f6d08145bb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6d080d1490 tx=0x7f6cf002a110 comp rx=0 tx=0).stop 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 -- 172.21.15.59:0/944553171 shutdown_connections 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 --2- 172.21.15.59:0/944553171 >> v2:172.21.15.76:3300/0 conn(0x7f6d080cd020 0x7f6d080b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 --2- 172.21.15.59:0/944553171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d081454e0 0x7f6d08145bb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 -- 172.21.15.59:0/944553171 >> 172.21.15.59:0/944553171 conn(0x7f6d08067ea0 msgr2=0x7f6d0806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 -- 172.21.15.59:0/944553171 shutdown_connections 2024-04-23T14:34:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.966+0000 7f6d0f600640 1 -- 172.21.15.59:0/944553171 wait complete. 2024-04-23T14:34:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.968+0000 7f6d0f600640 1 Processor -- start 2024-04-23T14:34:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.968+0000 7f6d0f600640 1 -- start start 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0f600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d080b3e40 0x7f6d0815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0f600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d080cd020 0x7f6d080b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0f600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d081454e0 0x7f6d080b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0f600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d08146270 con 0x7f6d080b3e40 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0f600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6d081460f0 con 0x7f6d080cd020 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0f600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6d08161020 con 0x7f6d081454e0 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d080b3e40 0x7f6d0815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d080b3e40 0x7f6d0815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53032/0 (socket says 172.21.15.59:53032) 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.969+0000 7f6d0cc00640 1 -- 172.21.15.59:0/1749250115 learned_addr learned my addr 172.21.15.59:0/1749250115 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d07e00640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d080cd020 0x7f6d080b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d0d600640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d081454e0 0x7f6d080b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d0cc00640 1 -- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d081454e0 msgr2=0x7f6d080b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d0cc00640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d081454e0 0x7f6d080b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d0cc00640 1 -- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d080cd020 msgr2=0x7f6d080b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d0cc00640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d080cd020 0x7f6d080b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d0cc00640 1 -- 172.21.15.59:0/1749250115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d080af0a0 con 0x7f6d080b3e40 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.970+0000 7f6d0cc00640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d080b3e40 0x7f6d0815c070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.971+0000 7f6d04c00640 1 -- 172.21.15.59:0/1749250115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0037050 con 0x7f6d080b3e40 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.971+0000 7f6d04c00640 1 -- 172.21.15.59:0/1749250115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf0038040 con 0x7f6d080b3e40 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.971+0000 7f6d04c00640 1 -- 172.21.15.59:0/1749250115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004470 con 0x7f6d080b3e40 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.971+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d081f8130 con 0x7f6d080b3e40 2024-04-23T14:34:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.971+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d081f86f0 con 0x7f6d080b3e40 2024-04-23T14:34:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.972+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d08068a80 con 0x7f6d080b3e40 2024-04-23T14:34:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.973+0000 7f6d04c00640 1 -- 172.21.15.59:0/1749250115 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cf0030900 con 0x7f6d080b3e40 2024-04-23T14:34:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.974+0000 7f6d04c00640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ce406c920 0x7f6ce406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.974+0000 7f6d04c00640 1 -- 172.21.15.59:0/1749250115 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6cf00ac5d0 con 0x7f6d080b3e40 2024-04-23T14:34:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.974+0000 7f6d07e00640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ce406c920 0x7f6ce406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.975+0000 7f6d07e00640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ce406c920 0x7f6ce406ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d08144d70 tx=0x7f6cf8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:53.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:53.978+0000 7f6d04c00640 1 -- 172.21.15.59:0/1749250115 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cf007aa90 con 0x7f6d080b3e40 2024-04-23T14:34:54.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-23T14:34:54.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-23T14:34:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.186+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d080af780 con 0x7f6d080b3e40 2024-04-23T14:34:54.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.188+0000 7f6d04c00640 1 -- 172.21.15.59:0/1749250115 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16618 (secure 0 0 0) 0x7f6cf007a390 con 0x7f6d080b3e40 2024-04-23T14:34:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:54.190 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":57,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:53.205036+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:51.101961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:52.192516+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:54.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ce406c920 msgr2=0x7f6ce406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ce406c920 0x7f6ce406ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d08144d70 tx=0x7f6cf8008040 comp rx=0 tx=0).stop 2024-04-23T14:34:54.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d080b3e40 msgr2=0x7f6d0815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d080b3e40 0x7f6d0815c070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 shutdown_connections 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ce406c920 0x7f6ce406ede0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d081454e0 0x7f6d080b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d080cd020 0x7f6d080b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 --2- 172.21.15.59:0/1749250115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d080b3e40 0x7f6d0815c070 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 >> 172.21.15.59:0/1749250115 conn(0x7f6d08067ea0 msgr2=0x7f6d0806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 shutdown_connections 2024-04-23T14:34:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.191+0000 7f6d0f600640 1 -- 172.21.15.59:0/1749250115 wait complete. 2024-04-23T14:34:54.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:54.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.421+0000 7fa68ec00640 1 Processor -- start 2024-04-23T14:34:54.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.422+0000 7fa68ec00640 1 -- start start 2024-04-23T14:34:54.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.423+0000 7fa68ec00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 0x7fa68815e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.423+0000 7fa68ec00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68806de30 con 0x7fa68806d4f0 2024-04-23T14:34:54.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.423+0000 7fa68ec00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6880d0840 con 0x7fa6880b3bb0 2024-04-23T14:34:54.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.423+0000 7fa68ec00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880cdc60 con 0x7fa68806d9a0 2024-04-23T14:34:54.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.424+0000 7fa687400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa68806d4f0 0x7fa68815e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58078/0 (socket says 172.21.15.59:58078) 2024-04-23T14:34:54.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.424+0000 7fa687400640 1 -- 172.21.15.59:0/3421987906 learned_addr learned my addr 172.21.15.59:0/3421987906 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:54.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.424+0000 7fa68cc00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 0x7fa68815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.424+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 254388696 0 0) 0x7fa68806de30 con 0x7fa68806d4f0 2024-04-23T14:34:54.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.424+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66c003520 con 0x7fa68806d4f0 2024-04-23T14:34:54.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1777691868 0 0) 0x7fa66c003520 con 0x7fa68806d4f0 2024-04-23T14:34:54.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 msgr2=0x7fa68815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:54.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 0x7fa68815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 >> v1:172.21.15.76:6789/0 conn(0x7fa6880b3bb0 legacy=0x7fa68809b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68809bdb0 con 0x7fa68806d4f0 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa674001670 con 0x7fa68806d4f0 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 >> v1:172.21.15.59:6789/0 conn(0x7fa68806d4f0 legacy=0x7fa68815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa66c0039e0 0x7fa66c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa66c004390 0x7fa66c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 0x7fa66c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880cdc60 con 0x7fa66c0039e0 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c00b9c0 con 0x7fa66c004390 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c00bbc0 con 0x7fa68806d9a0 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa687e00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa66c0039e0 0x7fa66c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.425+0000 7fa687400640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa66c004390 0x7fa66c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.426+0000 7fa68cc00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 0x7fa66c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.426+0000 7fa687e00640 1 -- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 msgr2=0x7fa66c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:54.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.426+0000 7fa687e00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa68806d9a0 0x7fa66c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.426+0000 7fa687e00640 1 -- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa66c004390 msgr2=0x7fa66c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:54.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.426+0000 7fa687e00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa66c004390 0x7fa66c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.426+0000 7fa687e00640 1 -- 172.21.15.59:0/3421987906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c003520 con 0x7fa66c0039e0 2024-04-23T14:34:54.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.428+0000 7fa687e00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa66c0039e0 0x7fa66c003dc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa678001860 tx=0x7fa66c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27309f333585c93d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:54.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.428+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678037070 con 0x7fa66c0039e0 2024-04-23T14:34:54.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.428+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67802ac30 con 0x7fa66c0039e0 2024-04-23T14:34:54.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.428+0000 7fa686a00640 1 -- 172.21.15.59:0/3421987906 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678001430 con 0x7fa66c0039e0 2024-04-23T14:34:54.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.428+0000 7fa68ec00640 1 -- 172.21.15.59:0/3421987906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa66c0039e0 msgr2=0x7fa66c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.428+0000 7fa68ec00640 1 --2- 172.21.15.59:0/3421987906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa66c0039e0 0x7fa66c003dc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa678001860 tx=0x7fa66c005830 comp rx=0 tx=0).stop 2024-04-23T14:34:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.429+0000 7fa68cc00640 1 -- 172.21.15.59:0/3421987906 reap_dead start 2024-04-23T14:34:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.429+0000 7fa68ec00640 1 -- 172.21.15.59:0/3421987906 shutdown_connections 2024-04-23T14:34:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.429+0000 7fa68ec00640 1 -- 172.21.15.59:0/3421987906 >> 172.21.15.59:0/3421987906 conn(0x7fa6880b40f0 msgr2=0x7fa6880b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.429+0000 7fa68ec00640 1 -- 172.21.15.59:0/3421987906 shutdown_connections 2024-04-23T14:34:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.429+0000 7fa68ec00640 1 -- 172.21.15.59:0/3421987906 wait complete. 2024-04-23T14:34:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.431+0000 7fa68ec00640 1 Processor -- start 2024-04-23T14:34:54.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.431+0000 7fa68ec00640 1 -- start start 2024-04-23T14:34:54.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68ec00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa688157a80 0x7fa68815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68ec00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa688157e60 0x7fa68815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68ec00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6881590a0 0x7fa688159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68ec00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880d10a0 con 0x7fa6881590a0 2024-04-23T14:34:54.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68ec00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880d0dc0 con 0x7fa688157e60 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68ec00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880d1380 con 0x7fa688157a80 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6881590a0 0x7fa688159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6881590a0 0x7fa688159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53040/0 (socket says 172.21.15.59:53040) 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 -- 172.21.15.59:0/4185399542 learned_addr learned my addr 172.21.15.59:0/4185399542 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa687400640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa688157e60 0x7fa68815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa687e00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa688157a80 0x7fa68815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 -- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa688157a80 msgr2=0x7fa68815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa688157a80 0x7fa68815c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 -- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa688157e60 msgr2=0x7fa68815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa688157e60 0x7fa68815b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa68cc00640 1 -- 172.21.15.59:0/4185399542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68815f520 con 0x7fa6881590a0 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.432+0000 7fa687400640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa688157e60 0x7fa68815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:54.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.433+0000 7fa68cc00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6881590a0 0x7fa688159530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa6880736e0 tx=0x7fa6800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:54.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.433+0000 7fa67fe00640 1 -- 172.21.15.59:0/4185399542 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680003990 con 0x7fa6881590a0 2024-04-23T14:34:54.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.433+0000 7fa67fe00640 1 -- 172.21.15.59:0/4185399542 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68001b070 con 0x7fa6881590a0 2024-04-23T14:34:54.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.433+0000 7fa67fe00640 1 -- 172.21.15.59:0/4185399542 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680003b70 con 0x7fa6881590a0 2024-04-23T14:34:54.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.433+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68815e760 con 0x7fa6881590a0 2024-04-23T14:34:54.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.433+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68815ed20 con 0x7fa6881590a0 2024-04-23T14:34:54.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.434+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa688068a80 con 0x7fa6881590a0 2024-04-23T14:34:54.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.436+0000 7fa67fe00640 1 -- 172.21.15.59:0/4185399542 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa680005d10 con 0x7fa6881590a0 2024-04-23T14:34:54.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.437+0000 7fa67fe00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66806c8f0 0x7fa66806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.437+0000 7fa67fe00640 1 -- 172.21.15.59:0/4185399542 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa68008c060 con 0x7fa6881590a0 2024-04-23T14:34:54.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.437+0000 7fa687e00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66806c8f0 0x7fa66806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.438+0000 7fa687e00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66806c8f0 0x7fa66806edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa678001860 tx=0x7fa678009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:54.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.442+0000 7fa67fe00640 1 -- 172.21.15.59:0/4185399542 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa68005a360 con 0x7fa6881590a0 2024-04-23T14:34:54.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.651+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6880ce1f0 con 0x7fa6881590a0 2024-04-23T14:34:54.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.652+0000 7fa67fe00640 1 -- 172.21.15.59:0/4185399542 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16618 (secure 0 0 0) 0x7fa680059c60 con 0x7fa6881590a0 2024-04-23T14:34:54.654 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:54.654 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":57,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:53.205036+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:51.101961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:52.192516+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:54.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.655+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66806c8f0 msgr2=0x7fa66806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.655+0000 7fa68ec00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66806c8f0 0x7fa66806edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa678001860 tx=0x7fa678009030 comp rx=0 tx=0).stop 2024-04-23T14:34:54.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.655+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6881590a0 msgr2=0x7fa688159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.655+0000 7fa68ec00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6881590a0 0x7fa688159530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa6880736e0 tx=0x7fa6800115a0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 shutdown_connections 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa66806c8f0 0x7fa66806edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6881590a0 0x7fa688159530 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa688157e60 0x7fa68815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 --2- 172.21.15.59:0/4185399542 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa688157a80 0x7fa68815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 >> 172.21.15.59:0/4185399542 conn(0x7fa6880b40f0 msgr2=0x7fa688074ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 shutdown_connections 2024-04-23T14:34:54.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.656+0000 7fa68ec00640 1 -- 172.21.15.59:0/4185399542 wait complete. 2024-04-23T14:34:54.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:34:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.879+0000 7fd103e00640 1 Processor -- start 2024-04-23T14:34:54.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.879+0000 7fd103e00640 1 -- start start 2024-04-23T14:34:54.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.881+0000 7fd103e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.881+0000 7fd103e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1040b3600 0x7fd1040d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.881+0000 7fd103e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1040b5da0 con 0x7fd1040b3f10 2024-04-23T14:34:54.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.881+0000 7fd103e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd10415f100 con 0x7fd1040b3600 2024-04-23T14:34:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.881+0000 7fd103e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd1040d0a20 con 0x7fd10415ed20 2024-04-23T14:34:54.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.882+0000 7fd102a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.882+0000 7fd102a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:47258/0 (socket says 172.21.15.59:47258) 2024-04-23T14:34:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.882+0000 7fd102a00640 1 -- 172.21.15.59:0/3281333407 learned_addr learned my addr 172.21.15.59:0/3281333407 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.882+0000 7fd102000640 1 --2- 172.21.15.59:0/3281333407 >> v2:172.21.15.76:3300/0 conn(0x7fd1040b3600 0x7fd1040d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.883+0000 7fd102a00640 1 -- 172.21.15.59:0/3281333407 >> v2:172.21.15.76:3300/0 conn(0x7fd1040b3600 msgr2=0x7fd1040d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.883+0000 7fd102a00640 1 --2- 172.21.15.59:0/3281333407 >> v2:172.21.15.76:3300/0 conn(0x7fd1040b3600 0x7fd1040d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.883+0000 7fd102a00640 1 -- 172.21.15.59:0/3281333407 >> v1:172.21.15.76:6789/0 conn(0x7fd1040b3f10 legacy=0x7fd1040b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.883+0000 7fd102a00640 1 -- 172.21.15.59:0/3281333407 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1040d2ed0 con 0x7fd10415ed20 2024-04-23T14:34:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.884+0000 7fd102a00640 1 --2- 172.21.15.59:0/3281333407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040d0ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009870 tx=0x7fd0f402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e28df0981f290e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:54.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.884+0000 7fd101600640 1 -- 172.21.15.59:0/3281333407 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4037070 con 0x7fd10415ed20 2024-04-23T14:34:54.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd101600640 1 -- 172.21.15.59:0/3281333407 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f4032440 con 0x7fd10415ed20 2024-04-23T14:34:54.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd101600640 1 -- 172.21.15.59:0/3281333407 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4032810 con 0x7fd10415ed20 2024-04-23T14:34:54.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd103e00640 1 -- 172.21.15.59:0/3281333407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 msgr2=0x7fd1040d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd103e00640 1 --2- 172.21.15.59:0/3281333407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040d0ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009870 tx=0x7fd0f402a130 comp rx=0 tx=0).stop 2024-04-23T14:34:54.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd103e00640 1 -- 172.21.15.59:0/3281333407 shutdown_connections 2024-04-23T14:34:54.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd103e00640 1 --2- 172.21.15.59:0/3281333407 >> v2:172.21.15.76:3300/0 conn(0x7fd1040b3600 0x7fd1040d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd103e00640 1 --2- 172.21.15.59:0/3281333407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040d0ea0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.885+0000 7fd103e00640 1 -- 172.21.15.59:0/3281333407 >> 172.21.15.59:0/3281333407 conn(0x7fd10408bf90 msgr2=0x7fd1040d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:54.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.886+0000 7fd103e00640 1 -- 172.21.15.59:0/3281333407 shutdown_connections 2024-04-23T14:34:54.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.886+0000 7fd103e00640 1 -- 172.21.15.59:0/3281333407 wait complete. 2024-04-23T14:34:54.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.887+0000 7fd103e00640 1 Processor -- start 2024-04-23T14:34:54.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.888+0000 7fd103e00640 1 -- start start 2024-04-23T14:34:54.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.888+0000 7fd103e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1040b3600 0x7fd1040b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd103e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1040b3f10 0x7fd104150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd103e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd103e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040d0290 con 0x7fd1040b3600 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd103e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1040d3490 con 0x7fd1040b3f10 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd103e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd1040d0410 con 0x7fd10415ed20 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1040b3600 0x7fd1040b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1040b3600 0x7fd1040b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53050/0 (socket says 172.21.15.59:53050) 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 -- 172.21.15.59:0/2086560565 learned_addr learned my addr 172.21.15.59:0/2086560565 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102000640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1040b3f10 0x7fd104150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd103400640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 -- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 msgr2=0x7fd1040cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 -- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1040b3f10 msgr2=0x7fd104150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1040b3f10 0x7fd104150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:54.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd102a00640 1 -- 172.21.15.59:0/2086560565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd104151200 con 0x7fd1040b3600 2024-04-23T14:34:54.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.889+0000 7fd103400640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:54.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.890+0000 7fd102a00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1040b3600 0x7fd1040b6090 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402a170 tx=0x7fd0f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:54.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.890+0000 7fd0faa00640 1 -- 172.21.15.59:0/2086560565 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4037040 con 0x7fd1040b3600 2024-04-23T14:34:54.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.890+0000 7fd0faa00640 1 -- 172.21.15.59:0/2086560565 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f4042070 con 0x7fd1040b3600 2024-04-23T14:34:54.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.890+0000 7fd0faa00640 1 -- 172.21.15.59:0/2086560565 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4030860 con 0x7fd1040b3600 2024-04-23T14:34:54.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.890+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041514a0 con 0x7fd1040b3600 2024-04-23T14:34:54.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.890+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd104151a60 con 0x7fd1040b3600 2024-04-23T14:34:54.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.892+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10404fd90 con 0x7fd1040b3600 2024-04-23T14:34:54.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.893+0000 7fd0faa00640 1 -- 172.21.15.59:0/2086560565 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0f4030a00 con 0x7fd1040b3600 2024-04-23T14:34:54.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.894+0000 7fd0faa00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0e406c8f0 0x7fd0e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:54.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.894+0000 7fd0faa00640 1 -- 172.21.15.59:0/2086560565 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd0f40ac380 con 0x7fd1040b3600 2024-04-23T14:34:54.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.894+0000 7fd102000640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0e406c8f0 0x7fd0e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:54.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.895+0000 7fd102000640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0e406c8f0 0x7fd0e406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd10409e860 tx=0x7fd0f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:54.899+0000 7fd0faa00640 1 -- 172.21.15.59:0/2086560565 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0f407a840 con 0x7fd1040b3600 2024-04-23T14:34:55.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.150+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd10406ba80 con 0x7fd1040b3600 2024-04-23T14:34:55.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.745+0000 7fd0faa00640 1 -- 172.21.15.59:0/2086560565 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7fd0f407a140 con 0x7fd1040b3600 2024-04-23T14:34:55.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.748+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0e406c8f0 msgr2=0x7fd0e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0e406c8f0 0x7fd0e406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd10409e860 tx=0x7fd0f0008040 comp rx=0 tx=0).stop 2024-04-23T14:34:55.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1040b3600 msgr2=0x7fd1040b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1040b3600 0x7fd1040b6090 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402a170 tx=0x7fd0f40046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 shutdown_connections 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd0e406c8f0 0x7fd0e406edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd10415ed20 0x7fd1040cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1040b3f10 0x7fd104150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 --2- 172.21.15.59:0/2086560565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1040b3600 0x7fd1040b6090 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 >> 172.21.15.59:0/2086560565 conn(0x7fd10408bf90 msgr2=0x7fd1040d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 shutdown_connections 2024-04-23T14:34:55.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.749+0000 7fd103e00640 1 -- 172.21.15.59:0/2086560565 wait complete. 2024-04-23T14:34:55.772 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:34:55.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:34:55.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.982+0000 7ff242c00640 1 Processor -- start 2024-04-23T14:34:55.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.982+0000 7ff242c00640 1 -- start start 2024-04-23T14:34:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff242c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c15e790 0x7ff23c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff242c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff23c071d70 0x7ff23c072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff242c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c0adc80 con 0x7ff23c06f6f0 2024-04-23T14:34:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff242c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c0aef40 con 0x7ff23c071d70 2024-04-23T14:34:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff242c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c0afd80 con 0x7ff23c15e790 2024-04-23T14:34:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff240c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff23c071d70 0x7ff23c072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff240c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff23c071d70 0x7ff23c072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53052/0 (socket says 172.21.15.59:53052) 2024-04-23T14:34:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.984+0000 7ff240c00640 1 -- 172.21.15.59:0/1761338121 learned_addr learned my addr 172.21.15.59:0/1761338121 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.985+0000 7ff23be00640 1 --2- 172.21.15.59:0/1761338121 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c15e790 0x7ff23c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.985+0000 7ff240c00640 1 -- 172.21.15.59:0/1761338121 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c15e790 msgr2=0x7ff23c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.985+0000 7ff240c00640 1 --2- 172.21.15.59:0/1761338121 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c15e790 0x7ff23c0aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.985+0000 7ff240c00640 1 -- 172.21.15.59:0/1761338121 >> v1:172.21.15.76:6789/0 conn(0x7ff23c06f6f0 legacy=0x7ff23c162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.985+0000 7ff240c00640 1 -- 172.21.15.59:0/1761338121 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c0adf20 con 0x7ff23c071d70 2024-04-23T14:34:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.986+0000 7ff240c00640 1 --2- 172.21.15.59:0/1761338121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c071d70 0x7ff23c072200 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff234009e50 tx=0x7ff23402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3873cee794963f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.986+0000 7ff23aa00640 1 -- 172.21.15.59:0/1761338121 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234036070 con 0x7ff23c071d70 2024-04-23T14:34:55.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.986+0000 7ff23aa00640 1 -- 172.21.15.59:0/1761338121 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff234031600 con 0x7ff23c071d70 2024-04-23T14:34:55.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.986+0000 7ff23aa00640 1 -- 172.21.15.59:0/1761338121 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340319d0 con 0x7ff23c071d70 2024-04-23T14:34:55.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 -- 172.21.15.59:0/1761338121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c071d70 msgr2=0x7ff23c072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 --2- 172.21.15.59:0/1761338121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c071d70 0x7ff23c072200 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff234009e50 tx=0x7ff23402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 -- 172.21.15.59:0/1761338121 shutdown_connections 2024-04-23T14:34:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 --2- 172.21.15.59:0/1761338121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c071d70 0x7ff23c072200 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 --2- 172.21.15.59:0/1761338121 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c15e790 0x7ff23c0aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 -- 172.21.15.59:0/1761338121 >> 172.21.15.59:0/1761338121 conn(0x7ff23c067d90 msgr2=0x7ff23c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 -- 172.21.15.59:0/1761338121 shutdown_connections 2024-04-23T14:34:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.987+0000 7ff242c00640 1 -- 172.21.15.59:0/1761338121 wait complete. 2024-04-23T14:34:55.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.989+0000 7ff242c00640 1 Processor -- start 2024-04-23T14:34:55.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.990+0000 7ff242c00640 1 -- start start 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff242c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c06f6f0 0x7ff23c145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff242c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff23c071d70 0x7ff23c15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff242c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c15e790 0x7ff23c15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff242c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c06fbb0 con 0x7ff23c15e790 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff242c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0ae4e0 con 0x7ff23c071d70 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff242c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c15eb70 con 0x7ff23c06f6f0 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c15e790 0x7ff23c15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c15e790 0x7ff23c15d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53062/0 (socket says 172.21.15.59:53062) 2024-04-23T14:34:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 -- 172.21.15.59:0/1410884096 learned_addr learned my addr 172.21.15.59:0/1410884096 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:55.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 -- 172.21.15.59:0/1410884096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c06f6f0 msgr2=0x7ff23c145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c06f6f0 0x7ff23c145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 -- 172.21.15.59:0/1410884096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff23c071d70 msgr2=0x7ff23c15d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff23c071d70 0x7ff23c15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.991+0000 7ff240c00640 1 -- 172.21.15.59:0/1410884096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c0b33d0 con 0x7ff23c15e790 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.992+0000 7ff240c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c15e790 0x7ff23c15d6e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff234009e50 tx=0x7ff234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.992+0000 7ff233e00640 1 -- 172.21.15.59:0/1410884096 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234036050 con 0x7ff23c15e790 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.992+0000 7ff233e00640 1 -- 172.21.15.59:0/1410884096 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff234042070 con 0x7ff23c15e790 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.992+0000 7ff233e00640 1 -- 172.21.15.59:0/1410884096 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234009510 con 0x7ff23c15e790 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.992+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c0b36b0 con 0x7ff23c15e790 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.992+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c145990 con 0x7ff23c15e790 2024-04-23T14:34:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.993+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c0b53b0 con 0x7ff23c15e790 2024-04-23T14:34:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.994+0000 7ff233e00640 1 -- 172.21.15.59:0/1410884096 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff234007bd0 con 0x7ff23c15e790 2024-04-23T14:34:55.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.995+0000 7ff233e00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21c06c8f0 0x7ff21c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.995+0000 7ff233e00640 1 -- 172.21.15.59:0/1410884096 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff2340aca40 con 0x7ff23c15e790 2024-04-23T14:34:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.996+0000 7ff23be00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21c06c8f0 0x7ff21c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:55.996+0000 7ff23be00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21c06c8f0 0x7ff21c06edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff22c006fd0 tx=0x7ff22c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:56.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.000+0000 7ff233e00640 1 -- 172.21.15.59:0/1410884096 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff23c0b53b0 con 0x7ff23c15e790 2024-04-23T14:34:56.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.246+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff23c068a80 con 0x7ff23c15e790 2024-04-23T14:34:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.755+0000 7ff233e00640 1 -- 172.21.15.59:0/1410884096 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7ff234045360 con 0x7ff23c15e790 2024-04-23T14:34:56.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21c06c8f0 msgr2=0x7ff21c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:56.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21c06c8f0 0x7ff21c06edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff22c006fd0 tx=0x7ff22c008040 comp rx=0 tx=0).stop 2024-04-23T14:34:56.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c15e790 msgr2=0x7ff23c15d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:56.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c15e790 0x7ff23c15d6e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff234009e50 tx=0x7ff234004a40 comp rx=0 tx=0).stop 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 shutdown_connections 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21c06c8f0 0x7ff21c06edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23c15e790 0x7ff23c15d6e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff23c071d70 0x7ff23c15d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 --2- 172.21.15.59:0/1410884096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff23c06f6f0 0x7ff23c145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.759+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 >> 172.21.15.59:0/1410884096 conn(0x7ff23c067d90 msgr2=0x7ff23c06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.760+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 shutdown_connections 2024-04-23T14:34:56.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.760+0000 7ff242c00640 1 -- 172.21.15.59:0/1410884096 wait complete. 2024-04-23T14:34:56.781 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:34:56.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:56.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.990+0000 7fbcf4800640 1 Processor -- start 2024-04-23T14:34:56.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.990+0000 7fbcf4800640 1 -- start start 2024-04-23T14:34:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf4800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf4800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcec0cd020 0x7fbcec0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf4800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcec0b3e40 0x7fbcec160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf4800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbcec0cddc0 con 0x7fbcec0cd020 2024-04-23T14:34:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf4800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbcec0cd520 con 0x7fbcec0b3e40 2024-04-23T14:34:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf4800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbcec0cd800 con 0x7fbcec1454e0 2024-04-23T14:34:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf1400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcec0cd020 0x7fbcec0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf1400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcec0cd020 0x7fbcec0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53072/0 (socket says 172.21.15.59:53072) 2024-04-23T14:34:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.992+0000 7fbcf1400640 1 -- 172.21.15.59:0/2783693512 learned_addr learned my addr 172.21.15.59:0/2783693512 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf1e00640 1 --2- 172.21.15.59:0/2783693512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf1400640 1 -- 172.21.15.59:0/2783693512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 msgr2=0x7fbcec145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf1400640 1 --2- 172.21.15.59:0/2783693512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf1400640 1 -- 172.21.15.59:0/2783693512 >> v2:172.21.15.76:3300/0 conn(0x7fbcec0b3e40 msgr2=0x7fbcec160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf2800640 1 --2- 172.21.15.59:0/2783693512 >> v2:172.21.15.76:3300/0 conn(0x7fbcec0b3e40 0x7fbcec160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf1400640 1 --2- 172.21.15.59:0/2783693512 >> v2:172.21.15.76:3300/0 conn(0x7fbcec0b3e40 0x7fbcec160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf1400640 1 -- 172.21.15.59:0/2783693512 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec0cdae0 con 0x7fbcec0cd020 2024-04-23T14:34:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.993+0000 7fbcf2800640 1 --2- 172.21.15.59:0/2783693512 >> v2:172.21.15.76:3300/0 conn(0x7fbcec0b3e40 0x7fbcec160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.994+0000 7fbcf1400640 1 --2- 172.21.15.59:0/2783693512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0cd020 0x7fbcec0b3900 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0d0270 tx=0x7fbcdc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd84dc74c53a6869 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.995+0000 7fbcf0a00640 1 -- 172.21.15.59:0/2783693512 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc037070 con 0x7fbcec0cd020 2024-04-23T14:34:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.995+0000 7fbcf0a00640 1 -- 172.21.15.59:0/2783693512 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcdc032440 con 0x7fbcec0cd020 2024-04-23T14:34:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.995+0000 7fbcf0a00640 1 -- 172.21.15.59:0/2783693512 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc032810 con 0x7fbcec0cd020 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 -- 172.21.15.59:0/2783693512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0cd020 msgr2=0x7fbcec0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 --2- 172.21.15.59:0/2783693512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0cd020 0x7fbcec0b3900 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0d0270 tx=0x7fbcdc02a140 comp rx=0 tx=0).stop 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 -- 172.21.15.59:0/2783693512 shutdown_connections 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 --2- 172.21.15.59:0/2783693512 >> v2:172.21.15.76:3300/0 conn(0x7fbcec0b3e40 0x7fbcec160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 --2- 172.21.15.59:0/2783693512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0cd020 0x7fbcec0b3900 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 --2- 172.21.15.59:0/2783693512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 -- 172.21.15.59:0/2783693512 >> 172.21.15.59:0/2783693512 conn(0x7fbcec067ea0 msgr2=0x7fbcec06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 -- 172.21.15.59:0/2783693512 shutdown_connections 2024-04-23T14:34:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.996+0000 7fbcf4800640 1 -- 172.21.15.59:0/2783693512 wait complete. 2024-04-23T14:34:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.998+0000 7fbcf4800640 1 Processor -- start 2024-04-23T14:34:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.998+0000 7fbcf4800640 1 -- start start 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf4800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0b3e40 0x7fbcec1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf4800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0cd020 0x7fbcec15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf4800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec15d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf4800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec146270 con 0x7fbcec0b3e40 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf4800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1460f0 con 0x7fbcec0cd020 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf4800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbcec160e50 con 0x7fbcec1454e0 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0b3e40 0x7fbcec1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0b3e40 0x7fbcec1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53088/0 (socket says 172.21.15.59:53088) 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:56.999+0000 7fbcf1e00640 1 -- 172.21.15.59:0/3110649573 learned_addr learned my addr 172.21.15.59:0/3110649573 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf2800640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf1e00640 1 -- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 msgr2=0x7fbcec15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf1e00640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec15d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf1e00640 1 -- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0cd020 msgr2=0x7fbcec15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf1400640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0cd020 0x7fbcec15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf1e00640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0cd020 0x7fbcec15cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf1e00640 1 -- 172.21.15.59:0/3110649573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec13edc0 con 0x7fbcec0b3e40 2024-04-23T14:34:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbcf1e00640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0b3e40 0x7fbcec1f56c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbce000d5f0 tx=0x7fbce000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.000+0000 7fbce6000640 1 -- 172.21.15.59:0/3110649573 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00125f0 con 0x7fbcec0b3e40 2024-04-23T14:34:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.001+0000 7fbce6000640 1 -- 172.21.15.59:0/3110649573 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce0019070 con 0x7fbcec0b3e40 2024-04-23T14:34:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.001+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec13f0a0 con 0x7fbcec0b3e40 2024-04-23T14:34:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.001+0000 7fbce6000640 1 -- 172.21.15.59:0/3110649573 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce00129d0 con 0x7fbcec0b3e40 2024-04-23T14:34:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.001+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec13f660 con 0x7fbcec0b3e40 2024-04-23T14:34:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.002+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec068a80 con 0x7fbcec0b3e40 2024-04-23T14:34:57.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.003+0000 7fbce6000640 1 -- 172.21.15.59:0/3110649573 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbce0012b70 con 0x7fbcec0b3e40 2024-04-23T14:34:57.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.004+0000 7fbce6000640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c8f0 0x7fbcd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.004+0000 7fbce6000640 1 -- 172.21.15.59:0/3110649573 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fbce008bfa0 con 0x7fbcec0b3e40 2024-04-23T14:34:57.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.004+0000 7fbcf1400640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c8f0 0x7fbcd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.005+0000 7fbcf1400640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c8f0 0x7fbcd006edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc034000 tx=0x7fbcdc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.009+0000 7fbce6000640 1 -- 172.21.15.59:0/3110649573 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbce005a330 con 0x7fbcec0b3e40 2024-04-23T14:34:57.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.250+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcec160030 con 0x7fbcec0b3e40 2024-04-23T14:34:57.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.251+0000 7fbce6000640 1 -- 172.21.15.59:0/3110649573 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7fbce0059c30 con 0x7fbcec0b3e40 2024-04-23T14:34:57.253 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:57.253 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:34:56.248634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:34:57.253 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c8f0 msgr2=0x7fbcd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c8f0 0x7fbcd006edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc034000 tx=0x7fbcdc02e000 comp rx=0 tx=0).stop 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0b3e40 msgr2=0x7fbcec1f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0b3e40 0x7fbcec1f56c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbce000d5f0 tx=0x7fbce000dab0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 shutdown_connections 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbcd006c8f0 0x7fbcd006edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbcec1454e0 0x7fbcec15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0cd020 0x7fbcec15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 --2- 172.21.15.59:0/3110649573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcec0b3e40 0x7fbcec1f56c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 >> 172.21.15.59:0/3110649573 conn(0x7fbcec067ea0 msgr2=0x7fbcec0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 shutdown_connections 2024-04-23T14:34:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.254+0000 7fbcf4800640 1 -- 172.21.15.59:0/3110649573 wait complete. 2024-04-23T14:34:57.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:34:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.465+0000 7f66ffe00640 1 Processor -- start 2024-04-23T14:34:57.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.465+0000 7f66ffe00640 1 -- start start 2024-04-23T14:34:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.466+0000 7f66ffe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f670015c3d0 0x7f67000cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.466+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67000cd590 0x7f67000b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.466+0000 7f66ffe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6700160210 con 0x7f67000b4690 2024-04-23T14:34:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.466+0000 7f66ffe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f67001604f0 con 0x7f670015c3d0 2024-04-23T14:34:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.466+0000 7f66ffe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f67001607d0 con 0x7f67000cd590 2024-04-23T14:34:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.467+0000 7f66f6000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67000cd590 0x7f67000b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.467+0000 7f66f6000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67000cd590 0x7f67000b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:47318/0 (socket says 172.21.15.59:47318) 2024-04-23T14:34:57.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.467+0000 7f66f6000640 1 -- 172.21.15.59:0/2424391474 learned_addr learned my addr 172.21.15.59:0/2424391474 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:57.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.467+0000 7f66fea00640 1 --2- 172.21.15.59:0/2424391474 >> v2:172.21.15.76:3300/0 conn(0x7f670015c3d0 0x7f67000cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.468+0000 7f66f6000640 1 -- 172.21.15.59:0/2424391474 >> v2:172.21.15.76:3300/0 conn(0x7f670015c3d0 msgr2=0x7f67000cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:57.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.468+0000 7f66f6000640 1 --2- 172.21.15.59:0/2424391474 >> v2:172.21.15.76:3300/0 conn(0x7f670015c3d0 0x7f67000cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.468+0000 7f66f6000640 1 -- 172.21.15.59:0/2424391474 >> v1:172.21.15.76:6789/0 conn(0x7f67000b4690 legacy=0x7f670006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.468+0000 7f66f6000640 1 -- 172.21.15.59:0/2424391474 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6700160ab0 con 0x7f67000cd590 2024-04-23T14:34:57.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.469+0000 7f66f6000640 1 --2- 172.21.15.59:0/2424391474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67000cd590 0x7f67000b4150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009870 tx=0x7f66e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cad33a2eb979bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66fe000640 1 -- 172.21.15.59:0/2424391474 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8037070 con 0x7f67000cd590 2024-04-23T14:34:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66fe000640 1 -- 172.21.15.59:0/2424391474 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e8032440 con 0x7f67000cd590 2024-04-23T14:34:57.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66fe000640 1 -- 172.21.15.59:0/2424391474 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8032810 con 0x7f67000cd590 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66ffe00640 1 -- 172.21.15.59:0/2424391474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67000cd590 msgr2=0x7f67000b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66ffe00640 1 --2- 172.21.15.59:0/2424391474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67000cd590 0x7f67000b4150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009870 tx=0x7f66e802a270 comp rx=0 tx=0).stop 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66ffe00640 1 -- 172.21.15.59:0/2424391474 shutdown_connections 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66ffe00640 1 --2- 172.21.15.59:0/2424391474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67000cd590 0x7f67000b4150 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66ffe00640 1 --2- 172.21.15.59:0/2424391474 >> v2:172.21.15.76:3300/0 conn(0x7f670015c3d0 0x7f67000cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.470+0000 7f66ffe00640 1 -- 172.21.15.59:0/2424391474 >> 172.21.15.59:0/2424391474 conn(0x7f67000b21a0 msgr2=0x7f67000b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.471+0000 7f66ffe00640 1 -- 172.21.15.59:0/2424391474 shutdown_connections 2024-04-23T14:34:57.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.471+0000 7f66ffe00640 1 -- 172.21.15.59:0/2424391474 wait complete. 2024-04-23T14:34:57.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.472+0000 7f66ffe00640 1 Processor -- start 2024-04-23T14:34:57.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.472+0000 7f66ffe00640 1 -- start start 2024-04-23T14:34:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.473+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67000b4690 0x7f67000ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.473+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67000cd590 0x7f670015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.473+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f670015c3d0 0x7f6700158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.473+0000 7f66ffe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f670015c7b0 con 0x7f67000b4690 2024-04-23T14:34:57.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.473+0000 7f66ffe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67000b4b00 con 0x7f67000cd590 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.473+0000 7f66ffe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f670015c930 con 0x7f670015c3d0 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67000b4690 0x7f67000ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67000b4690 0x7f67000ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53104/0 (socket says 172.21.15.59:53104) 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 -- 172.21.15.59:0/1608414504 learned_addr learned my addr 172.21.15.59:0/1608414504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66f6000640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67000cd590 0x7f670015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66ff400640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f670015c3d0 0x7f6700158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 -- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f670015c3d0 msgr2=0x7f6700158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f670015c3d0 0x7f6700158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 -- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67000cd590 msgr2=0x7f670015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67000cd590 0x7f670015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66fea00640 1 -- 172.21.15.59:0/1608414504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670015d550 con 0x7f67000b4690 2024-04-23T14:34:57.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.474+0000 7f66f6000640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67000cd590 0x7f670015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:34:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.475+0000 7f66fea00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67000b4690 0x7f67000ae320 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f66f000e870 tx=0x7f66f000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.475+0000 7f66f7400640 1 -- 172.21.15.59:0/1608414504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f67000b4690 2024-04-23T14:34:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.475+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670015d830 con 0x7f67000b4690 2024-04-23T14:34:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.475+0000 7f66f7400640 1 -- 172.21.15.59:0/1608414504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f000c940 con 0x7f67000b4690 2024-04-23T14:34:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.475+0000 7f66f7400640 1 -- 172.21.15.59:0/1608414504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f000cd30 con 0x7f67000b4690 2024-04-23T14:34:57.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.475+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67000cd9c0 con 0x7f67000b4690 2024-04-23T14:34:57.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.476+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670004fd90 con 0x7f67000b4690 2024-04-23T14:34:57.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.477+0000 7f66f7400640 1 -- 172.21.15.59:0/1608414504 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66f001a070 con 0x7f67000b4690 2024-04-23T14:34:57.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.479+0000 7f66f7400640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f66dc06c8f0 0x7f66dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.479+0000 7f66f7400640 1 -- 172.21.15.59:0/1608414504 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f66f008bec0 con 0x7f67000b4690 2024-04-23T14:34:57.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.479+0000 7f66f6000640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f66dc06c8f0 0x7f66dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.480+0000 7f66f6000640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f66dc06c8f0 0x7f66dc06edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66e8034000 tx=0x7f66e802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.484+0000 7f66f7400640 1 -- 172.21.15.59:0/1608414504 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66f005a1d0 con 0x7f67000b4690 2024-04-23T14:34:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.690+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67000cd0b0 con 0x7f67000b4690 2024-04-23T14:34:57.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.691+0000 7f66f7400640 1 -- 172.21.15.59:0/1608414504 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7f66f0059ad0 con 0x7f67000b4690 2024-04-23T14:34:57.693 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:57.693 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:54.707671+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:51.101961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:52.192516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:34:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f66dc06c8f0 msgr2=0x7f66dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f66dc06c8f0 0x7f66dc06edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66e8034000 tx=0x7f66e802e000 comp rx=0 tx=0).stop 2024-04-23T14:34:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67000b4690 msgr2=0x7f67000ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67000b4690 0x7f67000ae320 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f66f000e870 tx=0x7f66f000ed30 comp rx=0 tx=0).stop 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 shutdown_connections 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f66dc06c8f0 0x7f66dc06edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f670015c3d0 0x7f6700158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67000cd590 0x7f670015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 --2- 172.21.15.59:0/1608414504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67000b4690 0x7f67000ae320 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 >> 172.21.15.59:0/1608414504 conn(0x7f67000b21a0 msgr2=0x7f67000af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 shutdown_connections 2024-04-23T14:34:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.694+0000 7f66ffe00640 1 -- 172.21.15.59:0/1608414504 wait complete. 2024-04-23T14:34:57.715 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:34:57.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:34:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.937+0000 7f7c71e00640 1 Processor -- start 2024-04-23T14:34:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.938+0000 7f7c71e00640 1 -- start start 2024-04-23T14:34:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.939+0000 7f7c71e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7c6c15e7c0 0x7f7c6c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.939+0000 7f7c71e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7c6c0d2630 0x7f7c6c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.939+0000 7f7c71e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c6c0cf030 con 0x7f7c6c0b6530 2024-04-23T14:34:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.939+0000 7f7c71e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7c6c0b7000 con 0x7f7c6c15e7c0 2024-04-23T14:34:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.939+0000 7f7c71e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7c6c0cf620 con 0x7f7c6c0d2630 2024-04-23T14:34:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.939+0000 7f7c70a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7c6c15e7c0 0x7f7c6c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c70a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7c6c15e7c0 0x7f7c6c0d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53116/0 (socket says 172.21.15.59:53116) 2024-04-23T14:34:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c70a00640 1 -- 172.21.15.59:0/205212970 learned_addr learned my addr 172.21.15.59:0/205212970 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c6be00640 1 --2- 172.21.15.59:0/205212970 >> v2:172.21.15.76:3300/0 conn(0x7f7c6c0d2630 0x7f7c6c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c70a00640 1 -- 172.21.15.59:0/205212970 >> v2:172.21.15.76:3300/0 conn(0x7f7c6c0d2630 msgr2=0x7f7c6c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c70a00640 1 --2- 172.21.15.59:0/205212970 >> v2:172.21.15.76:3300/0 conn(0x7f7c6c0d2630 0x7f7c6c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c70a00640 1 -- 172.21.15.59:0/205212970 >> v1:172.21.15.76:6789/0 conn(0x7f7c6c0b6530 legacy=0x7f7c6c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c70a00640 1 -- 172.21.15.59:0/205212970 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c0b6a40 con 0x7f7c6c15e7c0 2024-04-23T14:34:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.940+0000 7f7c6be00640 1 --2- 172.21.15.59:0/205212970 >> v2:172.21.15.76:3300/0 conn(0x7f7c6c0d2630 0x7f7c6c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.941+0000 7f7c70a00640 1 --2- 172.21.15.59:0/205212970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c15e7c0 0x7f7c6c0d20f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0d55c0 tx=0x7f7c5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25f2cc3236d6d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c6b400640 1 -- 172.21.15.59:0/205212970 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c037070 con 0x7f7c6c15e7c0 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c6b400640 1 -- 172.21.15.59:0/205212970 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5c032440 con 0x7f7c6c15e7c0 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c6b400640 1 -- 172.21.15.59:0/205212970 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c032810 con 0x7f7c6c15e7c0 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 -- 172.21.15.59:0/205212970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c15e7c0 msgr2=0x7f7c6c0d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 --2- 172.21.15.59:0/205212970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c15e7c0 0x7f7c6c0d20f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0d55c0 tx=0x7f7c5c02a230 comp rx=0 tx=0).stop 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 -- 172.21.15.59:0/205212970 shutdown_connections 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 --2- 172.21.15.59:0/205212970 >> v2:172.21.15.76:3300/0 conn(0x7f7c6c0d2630 0x7f7c6c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 --2- 172.21.15.59:0/205212970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c15e7c0 0x7f7c6c0d20f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 -- 172.21.15.59:0/205212970 >> 172.21.15.59:0/205212970 conn(0x7f7c6c09d920 msgr2=0x7f7c6c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 -- 172.21.15.59:0/205212970 shutdown_connections 2024-04-23T14:34:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.942+0000 7f7c71e00640 1 -- 172.21.15.59:0/205212970 wait complete. 2024-04-23T14:34:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.944+0000 7f7c71e00640 1 Processor -- start 2024-04-23T14:34:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.944+0000 7f7c71e00640 1 -- start start 2024-04-23T14:34:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c71e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c6c0b6530 0x7f7c6c14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c71e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c0d2630 0x7f7c6c14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c71e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c6c15e7c0 0x7f7c6c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c71e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c0d1a80 con 0x7f7c6c0d2630 2024-04-23T14:34:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c71e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c0cf2b0 con 0x7f7c6c15e7c0 2024-04-23T14:34:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c71e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c0d1c00 con 0x7f7c6c0b6530 2024-04-23T14:34:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c6be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c0d2630 0x7f7c6c14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c6be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c0d2630 0x7f7c6c14e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53126/0 (socket says 172.21.15.59:53126) 2024-04-23T14:34:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.945+0000 7f7c6be00640 1 -- 172.21.15.59:0/3253441567 learned_addr learned my addr 172.21.15.59:0/3253441567 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:57.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c70a00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c6c0b6530 0x7f7c6c14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c71400640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c6c15e7c0 0x7f7c6c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c6be00640 1 -- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c6c0b6530 msgr2=0x7f7c6c14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c6be00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c6c0b6530 0x7f7c6c14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c6be00640 1 -- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c6c15e7c0 msgr2=0x7f7c6c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:57.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c6be00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c6c15e7c0 0x7f7c6c0c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:57.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c6be00640 1 -- 172.21.15.59:0/3253441567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c161150 con 0x7f7c6c0d2630 2024-04-23T14:34:57.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.946+0000 7f7c6be00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c0d2630 0x7f7c6c14e8c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c820 tx=0x7f7c5800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.947+0000 7f7c68c00640 1 -- 172.21.15.59:0/3253441567 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58015070 con 0x7f7c6c0d2630 2024-04-23T14:34:57.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.947+0000 7f7c68c00640 1 -- 172.21.15.59:0/3253441567 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c580079d0 con 0x7f7c6c0d2630 2024-04-23T14:34:57.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.947+0000 7f7c68c00640 1 -- 172.21.15.59:0/3253441567 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58007da0 con 0x7f7c6c0d2630 2024-04-23T14:34:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.947+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c161430 con 0x7f7c6c0d2630 2024-04-23T14:34:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.947+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c1619f0 con 0x7f7c6c0d2630 2024-04-23T14:34:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.948+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6c04fd90 con 0x7f7c6c0d2630 2024-04-23T14:34:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.949+0000 7f7c68c00640 1 -- 172.21.15.59:0/3253441567 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c58018070 con 0x7f7c6c0d2630 2024-04-23T14:34:57.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.950+0000 7f7c68c00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:57.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.950+0000 7f7c68c00640 1 -- 172.21.15.59:0/3253441567 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7c5808bb00 con 0x7f7c6c0d2630 2024-04-23T14:34:57.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.950+0000 7f7c70a00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.951+0000 7f7c70a00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c034000 tx=0x7f7c5c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:57.954+0000 7f7c68c00640 1 -- 172.21.15.59:0/3253441567 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c58059e90 con 0x7f7c6c0d2630 2024-04-23T14:34:58.110 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:34:58.109+0000 7f3119800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:34:58.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.160+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c6c14f980 con 0x7f7c4c06c8f0 2024-04-23T14:34:58.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.176+0000 7f7c68c00640 1 -- 172.21.15.59:0/3253441567 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251887 (secure 0 0 0) 0x7f7c5c07f800 con 0x7f7c4c06c8f0 2024-04-23T14:34:58.179 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:34:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.182+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c4c06c8f0 msgr2=0x7f7c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.182+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c034000 tx=0x7f7c5c02e000 comp rx=0 tx=0).stop 2024-04-23T14:34:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c0d2630 msgr2=0x7f7c6c14e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c0d2630 0x7f7c6c14e8c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c820 tx=0x7f7c5800cce0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 shutdown_connections 2024-04-23T14:34:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c6c15e7c0 0x7f7c6c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c6c0d2630 0x7f7c6c14e8c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 --2- 172.21.15.59:0/3253441567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7c6c0b6530 0x7f7c6c14f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 >> 172.21.15.59:0/3253441567 conn(0x7f7c6c09d920 msgr2=0x7f7c6c06e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 shutdown_connections 2024-04-23T14:34:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.183+0000 7f7c71e00640 1 -- 172.21.15.59:0/3253441567 wait complete. 2024-04-23T14:34:58.205 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-04-23T14:34:56.411954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239384,"kb_used_data":6520,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735336,"statfs":{"total":773094113280,"available":772848984064,"internally_reserved":0,"allocated":6676480,"data_stored":3438341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.708301"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763675+0000","last_change":"2024-04-23T14:34:53.755861+0000","last_active":"2024-04-23T14:34:53.763675+0000","last_peered":"2024-04-23T14:34:53.763675+0000","last_clean":"2024-04-23T14:34:53.763675+0000","last_became_active":"2024-04-23T14:34:53.755417+0000","last_became_peered":"2024-04-23T14:34:53.755417+0000","last_unstale":"2024-04-23T14:34:53.763675+0000","last_undegraded":"2024-04-23T14:34:53.763675+0000","last_fullsized":"2024-04-23T14:34:53.763675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:58.195992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756198+0000","last_change":"2024-04-23T14:34:53.750957+0000","last_active":"2024-04-23T14:34:53.756198+0000","last_peered":"2024-04-23T14:34:53.756198+0000","last_clean":"2024-04-23T14:34:53.756198+0000","last_became_active":"2024-04-23T14:34:53.750638+0000","last_became_peered":"2024-04-23T14:34:53.750638+0000","last_unstale":"2024-04-23T14:34:53.756198+0000","last_undegraded":"2024-04-23T14:34:53.756198+0000","last_fullsized":"2024-04-23T14:34:53.756198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:12.147348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748749+0000","last_change":"2024-04-23T14:34:53.742424+0000","last_active":"2024-04-23T14:34:53.748749+0000","last_peered":"2024-04-23T14:34:53.748749+0000","last_clean":"2024-04-23T14:34:53.748749+0000","last_became_active":"2024-04-23T14:34:53.742111+0000","last_became_peered":"2024-04-23T14:34:53.742111+0000","last_unstale":"2024-04-23T14:34:53.748749+0000","last_undegraded":"2024-04-23T14:34:53.748749+0000","last_fullsized":"2024-04-23T14:34:53.748749+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:49.272884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763230+0000","last_change":"2024-04-23T14:34:53.756960+0000","last_active":"2024-04-23T14:34:53.763230+0000","last_peered":"2024-04-23T14:34:53.763230+0000","last_clean":"2024-04-23T14:34:53.763230+0000","last_became_active":"2024-04-23T14:34:53.756605+0000","last_became_peered":"2024-04-23T14:34:53.756605+0000","last_unstale":"2024-04-23T14:34:53.763230+0000","last_undegraded":"2024-04-23T14:34:53.763230+0000","last_fullsized":"2024-04-23T14:34:53.763230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:09.293204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762662+0000","last_change":"2024-04-23T14:34:53.752994+0000","last_active":"2024-04-23T14:34:53.762662+0000","last_peered":"2024-04-23T14:34:53.762662+0000","last_clean":"2024-04-23T14:34:53.762662+0000","last_became_active":"2024-04-23T14:34:53.752624+0000","last_became_peered":"2024-04-23T14:34:53.752624+0000","last_unstale":"2024-04-23T14:34:53.762662+0000","last_undegraded":"2024-04-23T14:34:53.762662+0000","last_fullsized":"2024-04-23T14:34:53.762662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:59.478944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753798+0000","last_change":"2024-04-23T14:34:53.749956+0000","last_active":"2024-04-23T14:34:53.753798+0000","last_peered":"2024-04-23T14:34:53.753798+0000","last_clean":"2024-04-23T14:34:53.753798+0000","last_became_active":"2024-04-23T14:34:53.749643+0000","last_became_peered":"2024-04-23T14:34:53.749643+0000","last_unstale":"2024-04-23T14:34:53.753798+0000","last_undegraded":"2024-04-23T14:34:53.753798+0000","last_fullsized":"2024-04-23T14:34:53.753798+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:53.519921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755988+0000","last_change":"2024-04-23T14:34:53.749975+0000","last_active":"2024-04-23T14:34:53.755988+0000","last_peered":"2024-04-23T14:34:53.755988+0000","last_clean":"2024-04-23T14:34:53.755988+0000","last_became_active":"2024-04-23T14:34:53.749335+0000","last_became_peered":"2024-04-23T14:34:53.749335+0000","last_unstale":"2024-04-23T14:34:53.755988+0000","last_undegraded":"2024-04-23T14:34:53.755988+0000","last_fullsized":"2024-04-23T14:34:53.755988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:16.193142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753452+0000","last_change":"2024-04-23T14:34:53.746610+0000","last_active":"2024-04-23T14:34:53.753452+0000","last_peered":"2024-04-23T14:34:53.753452+0000","last_clean":"2024-04-23T14:34:53.753452+0000","last_became_active":"2024-04-23T14:34:53.745711+0000","last_became_peered":"2024-04-23T14:34:53.745711+0000","last_unstale":"2024-04-23T14:34:53.753452+0000","last_undegraded":"2024-04-23T14:34:53.753452+0000","last_fullsized":"2024-04-23T14:34:53.753452+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.276721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762988+0000","last_change":"2024-04-23T14:34:53.759549+0000","last_active":"2024-04-23T14:34:53.762988+0000","last_peered":"2024-04-23T14:34:53.762988+0000","last_clean":"2024-04-23T14:34:53.762988+0000","last_became_active":"2024-04-23T14:34:53.759032+0000","last_became_peered":"2024-04-23T14:34:53.759032+0000","last_unstale":"2024-04-23T14:34:53.762988+0000","last_undegraded":"2024-04-23T14:34:53.762988+0000","last_fullsized":"2024-04-23T14:34:53.762988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:35:18.587240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735319+0000","last_change":"2024-04-23T14:34:53.724827+0000","last_active":"2024-04-23T14:34:53.735319+0000","last_peered":"2024-04-23T14:34:53.735319+0000","last_clean":"2024-04-23T14:34:53.735319+0000","last_became_active":"2024-04-23T14:34:53.723963+0000","last_became_peered":"2024-04-23T14:34:53.723963+0000","last_unstale":"2024-04-23T14:34:53.735319+0000","last_undegraded":"2024-04-23T14:34:53.735319+0000","last_fullsized":"2024-04-23T14:34:53.735319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:43:49.051906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755209+0000","last_change":"2024-04-23T14:34:53.749641+0000","last_active":"2024-04-23T14:34:53.755209+0000","last_peered":"2024-04-23T14:34:53.755209+0000","last_clean":"2024-04-23T14:34:53.755209+0000","last_became_active":"2024-04-23T14:34:53.749331+0000","last_became_peered":"2024-04-23T14:34:53.749331+0000","last_unstale":"2024-04-23T14:34:53.755209+0000","last_undegraded":"2024-04-23T14:34:53.755209+0000","last_fullsized":"2024-04-23T14:34:53.755209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:40.822873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754165+0000","last_change":"2024-04-23T14:34:53.750866+0000","last_active":"2024-04-23T14:34:53.754165+0000","last_peered":"2024-04-23T14:34:53.754165+0000","last_clean":"2024-04-23T14:34:53.754165+0000","last_became_active":"2024-04-23T14:34:53.750584+0000","last_became_peered":"2024-04-23T14:34:53.750584+0000","last_unstale":"2024-04-23T14:34:53.754165+0000","last_undegraded":"2024-04-23T14:34:53.754165+0000","last_fullsized":"2024-04-23T14:34:53.754165+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:53.519921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756673+0000","last_change":"2024-04-23T14:34:53.753913+0000","last_active":"2024-04-23T14:34:53.756673+0000","last_peered":"2024-04-23T14:34:53.756673+0000","last_clean":"2024-04-23T14:34:53.756673+0000","last_became_active":"2024-04-23T14:34:53.753634+0000","last_became_peered":"2024-04-23T14:34:53.753634+0000","last_unstale":"2024-04-23T14:34:53.756673+0000","last_undegraded":"2024-04-23T14:34:53.756673+0000","last_fullsized":"2024-04-23T14:34:53.756673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:10.417375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.745361+0000","last_change":"2024-04-23T14:34:53.742913+0000","last_active":"2024-04-23T14:34:53.745361+0000","last_peered":"2024-04-23T14:34:53.745361+0000","last_clean":"2024-04-23T14:34:53.745361+0000","last_became_active":"2024-04-23T14:34:53.742589+0000","last_became_peered":"2024-04-23T14:34:53.742589+0000","last_unstale":"2024-04-23T14:34:53.745361+0000","last_undegraded":"2024-04-23T14:34:53.745361+0000","last_fullsized":"2024-04-23T14:34:53.745361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:13.090777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.743241+0000","last_change":"2024-04-23T14:34:53.740259+0000","last_active":"2024-04-23T14:34:53.743241+0000","last_peered":"2024-04-23T14:34:53.743241+0000","last_clean":"2024-04-23T14:34:53.743241+0000","last_became_active":"2024-04-23T14:34:53.739451+0000","last_became_peered":"2024-04-23T14:34:53.739451+0000","last_unstale":"2024-04-23T14:34:53.743241+0000","last_undegraded":"2024-04-23T14:34:53.743241+0000","last_fullsized":"2024-04-23T14:34:53.743241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:18.589002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754458+0000","last_change":"2024-04-23T14:34:53.748499+0000","last_active":"2024-04-23T14:34:53.754458+0000","last_peered":"2024-04-23T14:34:53.754458+0000","last_clean":"2024-04-23T14:34:53.754458+0000","last_became_active":"2024-04-23T14:34:53.748170+0000","last_became_peered":"2024-04-23T14:34:53.748170+0000","last_unstale":"2024-04-23T14:34:53.754458+0000","last_undegraded":"2024-04-23T14:34:53.754458+0000","last_fullsized":"2024-04-23T14:34:53.754458+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:24:17.505869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756222+0000","last_change":"2024-04-23T14:34:53.752021+0000","last_active":"2024-04-23T14:34:53.756222+0000","last_peered":"2024-04-23T14:34:53.756222+0000","last_clean":"2024-04-23T14:34:53.756222+0000","last_became_active":"2024-04-23T14:34:53.751670+0000","last_became_peered":"2024-04-23T14:34:53.751670+0000","last_unstale":"2024-04-23T14:34:53.756222+0000","last_undegraded":"2024-04-23T14:34:53.756222+0000","last_fullsized":"2024-04-23T14:34:53.756222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:22.562112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.759616+0000","last_change":"2024-04-23T14:34:53.753147+0000","last_active":"2024-04-23T14:34:53.759616+0000","last_peered":"2024-04-23T14:34:53.759616+0000","last_clean":"2024-04-23T14:34:53.759616+0000","last_became_active":"2024-04-23T14:34:53.752761+0000","last_became_peered":"2024-04-23T14:34:53.752761+0000","last_unstale":"2024-04-23T14:34:53.759616+0000","last_undegraded":"2024-04-23T14:34:53.759616+0000","last_fullsized":"2024-04-23T14:34:53.759616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:14.986148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.760294+0000","last_change":"2024-04-23T14:34:53.753477+0000","last_active":"2024-04-23T14:34:53.760294+0000","last_peered":"2024-04-23T14:34:53.760294+0000","last_clean":"2024-04-23T14:34:53.760294+0000","last_became_active":"2024-04-23T14:34:53.752640+0000","last_became_peered":"2024-04-23T14:34:53.752640+0000","last_unstale":"2024-04-23T14:34:53.760294+0000","last_undegraded":"2024-04-23T14:34:53.760294+0000","last_fullsized":"2024-04-23T14:34:53.760294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:28.069818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.768875+0000","last_change":"2024-04-23T14:34:53.761723+0000","last_active":"2024-04-23T14:34:53.768875+0000","last_peered":"2024-04-23T14:34:53.768875+0000","last_clean":"2024-04-23T14:34:53.768875+0000","last_became_active":"2024-04-23T14:34:53.761194+0000","last_became_peered":"2024-04-23T14:34:53.761194+0000","last_unstale":"2024-04-23T14:34:53.768875+0000","last_undegraded":"2024-04-23T14:34:53.768875+0000","last_fullsized":"2024-04-23T14:34:53.768875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:25:35.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755268+0000","last_change":"2024-04-23T14:34:53.749269+0000","last_active":"2024-04-23T14:34:53.755268+0000","last_peered":"2024-04-23T14:34:53.755268+0000","last_clean":"2024-04-23T14:34:53.755268+0000","last_became_active":"2024-04-23T14:34:53.748998+0000","last_became_peered":"2024-04-23T14:34:53.748998+0000","last_unstale":"2024-04-23T14:34:53.755268+0000","last_undegraded":"2024-04-23T14:34:53.755268+0000","last_fullsized":"2024-04-23T14:34:53.755268+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:37.928012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.770385+0000","last_change":"2024-04-23T14:34:53.765699+0000","last_active":"2024-04-23T14:34:53.770385+0000","last_peered":"2024-04-23T14:34:53.770385+0000","last_clean":"2024-04-23T14:34:53.770385+0000","last_became_active":"2024-04-23T14:34:53.758512+0000","last_became_peered":"2024-04-23T14:34:53.758512+0000","last_unstale":"2024-04-23T14:34:53.770385+0000","last_undegraded":"2024-04-23T14:34:53.770385+0000","last_fullsized":"2024-04-23T14:34:53.770385+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:15:05.011298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753050+0000","last_change":"2024-04-23T14:34:53.747107+0000","last_active":"2024-04-23T14:34:53.753050+0000","last_peered":"2024-04-23T14:34:53.753050+0000","last_clean":"2024-04-23T14:34:53.753050+0000","last_became_active":"2024-04-23T14:34:53.746786+0000","last_became_peered":"2024-04-23T14:34:53.746786+0000","last_unstale":"2024-04-23T14:34:53.753050+0000","last_undegraded":"2024-04-23T14:34:53.753050+0000","last_fullsized":"2024-04-23T14:34:53.753050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:12.270345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756739+0000","last_change":"2024-04-23T14:34:53.753986+0000","last_active":"2024-04-23T14:34:53.756739+0000","last_peered":"2024-04-23T14:34:53.756739+0000","last_clean":"2024-04-23T14:34:53.756739+0000","last_became_active":"2024-04-23T14:34:53.753413+0000","last_became_peered":"2024-04-23T14:34:53.753413+0000","last_unstale":"2024-04-23T14:34:53.756739+0000","last_undegraded":"2024-04-23T14:34:53.756739+0000","last_fullsized":"2024-04-23T14:34:53.756739+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:43:49.051906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735836+0000","last_change":"2024-04-23T14:34:53.731093+0000","last_active":"2024-04-23T14:34:53.735836+0000","last_peered":"2024-04-23T14:34:53.735836+0000","last_clean":"2024-04-23T14:34:53.735836+0000","last_became_active":"2024-04-23T14:34:53.730744+0000","last_became_peered":"2024-04-23T14:34:53.730744+0000","last_unstale":"2024-04-23T14:34:53.735836+0000","last_undegraded":"2024-04-23T14:34:53.735836+0000","last_fullsized":"2024-04-23T14:34:53.735836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:17.680506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752311+0000","last_change":"2024-04-23T14:34:53.746566+0000","last_active":"2024-04-23T14:34:53.752311+0000","last_peered":"2024-04-23T14:34:53.752311+0000","last_clean":"2024-04-23T14:34:53.752311+0000","last_became_active":"2024-04-23T14:34:53.746281+0000","last_became_peered":"2024-04-23T14:34:53.746281+0000","last_unstale":"2024-04-23T14:34:53.752311+0000","last_undegraded":"2024-04-23T14:34:53.752311+0000","last_fullsized":"2024-04-23T14:34:53.752311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:12.423796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.757963+0000","last_change":"2024-04-23T14:34:53.751200+0000","last_active":"2024-04-23T14:34:53.757963+0000","last_peered":"2024-04-23T14:34:53.757963+0000","last_clean":"2024-04-23T14:34:53.757963+0000","last_became_active":"2024-04-23T14:34:53.750508+0000","last_became_peered":"2024-04-23T14:34:53.750508+0000","last_unstale":"2024-04-23T14:34:53.757963+0000","last_undegraded":"2024-04-23T14:34:53.757963+0000","last_fullsized":"2024-04-23T14:34:53.757963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:09.293204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756288+0000","last_change":"2024-04-23T14:34:53.753121+0000","last_active":"2024-04-23T14:34:53.756288+0000","last_peered":"2024-04-23T14:34:53.756288+0000","last_clean":"2024-04-23T14:34:53.756288+0000","last_became_active":"2024-04-23T14:34:53.752789+0000","last_became_peered":"2024-04-23T14:34:53.752789+0000","last_unstale":"2024-04-23T14:34:53.756288+0000","last_undegraded":"2024-04-23T14:34:53.756288+0000","last_fullsized":"2024-04-23T14:34:53.756288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:38:03.627993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.758370+0000","last_change":"2024-04-23T14:34:53.749200+0000","last_active":"2024-04-23T14:34:53.758370+0000","last_peered":"2024-04-23T14:34:53.758370+0000","last_clean":"2024-04-23T14:34:53.758370+0000","last_became_active":"2024-04-23T14:34:53.748892+0000","last_became_peered":"2024-04-23T14:34:53.748892+0000","last_unstale":"2024-04-23T14:34:53.758370+0000","last_undegraded":"2024-04-23T14:34:53.758370+0000","last_fullsized":"2024-04-23T14:34:53.758370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:01.770335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752915+0000","last_change":"2024-04-23T14:34:53.745893+0000","last_active":"2024-04-23T14:34:53.752915+0000","last_peered":"2024-04-23T14:34:53.752915+0000","last_clean":"2024-04-23T14:34:53.752915+0000","last_became_active":"2024-04-23T14:34:53.745620+0000","last_became_peered":"2024-04-23T14:34:53.745620+0000","last_unstale":"2024-04-23T14:34:53.752915+0000","last_undegraded":"2024-04-23T14:34:53.752915+0000","last_fullsized":"2024-04-23T14:34:53.752915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:27.733914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.734796+0000","last_change":"2024-04-23T14:34:53.727870+0000","last_active":"2024-04-23T14:34:53.734796+0000","last_peered":"2024-04-23T14:34:53.734796+0000","last_clean":"2024-04-23T14:34:53.734796+0000","last_became_active":"2024-04-23T14:34:53.726533+0000","last_became_peered":"2024-04-23T14:34:53.726533+0000","last_unstale":"2024-04-23T14:34:53.734796+0000","last_undegraded":"2024-04-23T14:34:53.734796+0000","last_fullsized":"2024-04-23T14:34:53.734796+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:52.117154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.764304+0000","last_change":"2024-04-23T14:34:53.759116+0000","last_active":"2024-04-23T14:34:53.764304+0000","last_peered":"2024-04-23T14:34:53.764304+0000","last_clean":"2024-04-23T14:34:53.764304+0000","last_became_active":"2024-04-23T14:34:53.754682+0000","last_became_peered":"2024-04-23T14:34:53.754682+0000","last_unstale":"2024-04-23T14:34:53.764304+0000","last_undegraded":"2024-04-23T14:34:53.764304+0000","last_fullsized":"2024-04-23T14:34:53.764304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:25.697374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755843+0000","last_change":"2024-04-23T14:34:53.742151+0000","last_active":"2024-04-23T14:34:53.755843+0000","last_peered":"2024-04-23T14:34:53.755843+0000","last_clean":"2024-04-23T14:34:53.755843+0000","last_became_active":"2024-04-23T14:34:53.741840+0000","last_became_peered":"2024-04-23T14:34:53.741840+0000","last_unstale":"2024-04-23T14:34:53.755843+0000","last_undegraded":"2024-04-23T14:34:53.755843+0000","last_fullsized":"2024-04-23T14:34:53.755843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:55:34.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.771486+0000","last_change":"2024-04-23T14:34:52.754641+0000","last_active":"2024-04-23T14:34:53.771486+0000","last_peered":"2024-04-23T14:34:53.771486+0000","last_clean":"2024-04-23T14:34:53.771486+0000","last_became_active":"2024-04-23T14:34:52.754335+0000","last_became_peered":"2024-04-23T14:34:52.754335+0000","last_unstale":"2024-04-23T14:34:53.771486+0000","last_undegraded":"2024-04-23T14:34:53.771486+0000","last_fullsized":"2024-04-23T14:34:53.771486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:16.182883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.764251+0000","last_change":"2024-04-23T14:34:53.752296+0000","last_active":"2024-04-23T14:34:53.764251+0000","last_peered":"2024-04-23T14:34:53.764251+0000","last_clean":"2024-04-23T14:34:53.764251+0000","last_became_active":"2024-04-23T14:34:53.748511+0000","last_became_peered":"2024-04-23T14:34:53.748511+0000","last_unstale":"2024-04-23T14:34:53.764251+0000","last_undegraded":"2024-04-23T14:34:53.764251+0000","last_fullsized":"2024-04-23T14:34:53.764251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:13.090777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.767211+0000","last_change":"2024-04-23T14:34:52.754260+0000","last_active":"2024-04-23T14:34:53.767211+0000","last_peered":"2024-04-23T14:34:53.767211+0000","last_clean":"2024-04-23T14:34:53.767211+0000","last_became_active":"2024-04-23T14:34:52.753847+0000","last_became_peered":"2024-04-23T14:34:52.753847+0000","last_unstale":"2024-04-23T14:34:53.767211+0000","last_undegraded":"2024-04-23T14:34:53.767211+0000","last_fullsized":"2024-04-23T14:34:53.767211+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:24.681850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.757942+0000","last_change":"2024-04-23T14:34:53.753795+0000","last_active":"2024-04-23T14:34:53.757942+0000","last_peered":"2024-04-23T14:34:53.757942+0000","last_clean":"2024-04-23T14:34:53.757942+0000","last_became_active":"2024-04-23T14:34:53.753341+0000","last_became_peered":"2024-04-23T14:34:53.753341+0000","last_unstale":"2024-04-23T14:34:53.757942+0000","last_undegraded":"2024-04-23T14:34:53.757942+0000","last_fullsized":"2024-04-23T14:34:53.757942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:29.344998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.772698+0000","last_change":"2024-04-23T14:34:52.739160+0000","last_active":"2024-04-23T14:34:53.772698+0000","last_peered":"2024-04-23T14:34:53.772698+0000","last_clean":"2024-04-23T14:34:53.772698+0000","last_became_active":"2024-04-23T14:34:52.738868+0000","last_became_peered":"2024-04-23T14:34:52.738868+0000","last_unstale":"2024-04-23T14:34:53.772698+0000","last_undegraded":"2024-04-23T14:34:53.772698+0000","last_fullsized":"2024-04-23T14:34:53.772698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:36.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763099+0000","last_change":"2024-04-23T14:34:53.758066+0000","last_active":"2024-04-23T14:34:53.763099+0000","last_peered":"2024-04-23T14:34:53.763099+0000","last_clean":"2024-04-23T14:34:53.763099+0000","last_became_active":"2024-04-23T14:34:53.756071+0000","last_became_peered":"2024-04-23T14:34:53.756071+0000","last_unstale":"2024-04-23T14:34:53.763099+0000","last_undegraded":"2024-04-23T14:34:53.763099+0000","last_fullsized":"2024-04-23T14:34:53.763099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:27.011070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763179+0000","last_change":"2024-04-23T14:34:52.758549+0000","last_active":"2024-04-23T14:34:53.763179+0000","last_peered":"2024-04-23T14:34:53.763179+0000","last_clean":"2024-04-23T14:34:53.763179+0000","last_became_active":"2024-04-23T14:34:52.758146+0000","last_became_peered":"2024-04-23T14:34:52.758146+0000","last_unstale":"2024-04-23T14:34:53.763179+0000","last_undegraded":"2024-04-23T14:34:53.763179+0000","last_fullsized":"2024-04-23T14:34:53.763179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:26.718389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769678+0000","last_change":"2024-04-23T14:34:53.764746+0000","last_active":"2024-04-23T14:34:53.769678+0000","last_peered":"2024-04-23T14:34:53.769678+0000","last_clean":"2024-04-23T14:34:53.769678+0000","last_became_active":"2024-04-23T14:34:53.758086+0000","last_became_peered":"2024-04-23T14:34:53.758086+0000","last_unstale":"2024-04-23T14:34:53.769678+0000","last_undegraded":"2024-04-23T14:34:53.769678+0000","last_fullsized":"2024-04-23T14:34:53.769678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:35.252357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762449+0000","last_change":"2024-04-23T14:34:52.750060+0000","last_active":"2024-04-23T14:34:53.762449+0000","last_peered":"2024-04-23T14:34:53.762449+0000","last_clean":"2024-04-23T14:34:53.762449+0000","last_became_active":"2024-04-23T14:34:52.749799+0000","last_became_peered":"2024-04-23T14:34:52.749799+0000","last_unstale":"2024-04-23T14:34:53.762449+0000","last_undegraded":"2024-04-23T14:34:53.762449+0000","last_fullsized":"2024-04-23T14:34:53.762449+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:14.826352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769299+0000","last_change":"2024-04-23T14:34:53.763289+0000","last_active":"2024-04-23T14:34:53.769299+0000","last_peered":"2024-04-23T14:34:53.769299+0000","last_clean":"2024-04-23T14:34:53.769299+0000","last_became_active":"2024-04-23T14:34:53.762954+0000","last_became_peered":"2024-04-23T14:34:53.762954+0000","last_unstale":"2024-04-23T14:34:53.769299+0000","last_undegraded":"2024-04-23T14:34:53.769299+0000","last_fullsized":"2024-04-23T14:34:53.769299+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:16:41.227970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769764+0000","last_change":"2024-04-23T14:34:52.743081+0000","last_active":"2024-04-23T14:34:53.769764+0000","last_peered":"2024-04-23T14:34:53.769764+0000","last_clean":"2024-04-23T14:34:53.769764+0000","last_became_active":"2024-04-23T14:34:52.742112+0000","last_became_peered":"2024-04-23T14:34:52.742112+0000","last_unstale":"2024-04-23T14:34:53.769764+0000","last_undegraded":"2024-04-23T14:34:53.769764+0000","last_fullsized":"2024-04-23T14:34:53.769764+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:11.408271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.746522+0000","last_change":"2024-04-23T14:34:53.743066+0000","last_active":"2024-04-23T14:34:53.746522+0000","last_peered":"2024-04-23T14:34:53.746522+0000","last_clean":"2024-04-23T14:34:53.746522+0000","last_became_active":"2024-04-23T14:34:53.742759+0000","last_became_peered":"2024-04-23T14:34:53.742759+0000","last_unstale":"2024-04-23T14:34:53.746522+0000","last_undegraded":"2024-04-23T14:34:53.746522+0000","last_fullsized":"2024-04-23T14:34:53.746522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:03.178115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.737828+0000","last_change":"2024-04-23T14:34:52.749807+0000","last_active":"2024-04-23T14:34:53.737828+0000","last_peered":"2024-04-23T14:34:53.737828+0000","last_clean":"2024-04-23T14:34:53.737828+0000","last_became_active":"2024-04-23T14:34:52.749455+0000","last_became_peered":"2024-04-23T14:34:52.749455+0000","last_unstale":"2024-04-23T14:34:53.737828+0000","last_undegraded":"2024-04-23T14:34:53.737828+0000","last_fullsized":"2024-04-23T14:34:53.737828+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:21.131326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754222+0000","last_change":"2024-04-23T14:34:53.744628+0000","last_active":"2024-04-23T14:34:53.754222+0000","last_peered":"2024-04-23T14:34:53.754222+0000","last_clean":"2024-04-23T14:34:53.754222+0000","last_became_active":"2024-04-23T14:34:53.744071+0000","last_became_peered":"2024-04-23T14:34:53.744071+0000","last_unstale":"2024-04-23T14:34:53.754222+0000","last_undegraded":"2024-04-23T14:34:53.754222+0000","last_fullsized":"2024-04-23T14:34:53.754222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:39.309228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.725375+0000","last_change":"2024-04-23T14:34:52.744553+0000","last_active":"2024-04-23T14:34:53.725375+0000","last_peered":"2024-04-23T14:34:53.725375+0000","last_clean":"2024-04-23T14:34:53.725375+0000","last_became_active":"2024-04-23T14:34:52.744022+0000","last_became_peered":"2024-04-23T14:34:52.744022+0000","last_unstale":"2024-04-23T14:34:53.725375+0000","last_undegraded":"2024-04-23T14:34:53.725375+0000","last_fullsized":"2024-04-23T14:34:53.725375+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:43:48.036381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.745258+0000","last_change":"2024-04-23T14:34:53.742007+0000","last_active":"2024-04-23T14:34:53.745258+0000","last_peered":"2024-04-23T14:34:53.745258+0000","last_clean":"2024-04-23T14:34:53.745258+0000","last_became_active":"2024-04-23T14:34:53.741679+0000","last_became_peered":"2024-04-23T14:34:53.741679+0000","last_unstale":"2024-04-23T14:34:53.745258+0000","last_undegraded":"2024-04-23T14:34:53.745258+0000","last_fullsized":"2024-04-23T14:34:53.745258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:40.822873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735474+0000","last_change":"2024-04-23T14:34:52.751589+0000","last_active":"2024-04-23T14:34:53.735474+0000","last_peered":"2024-04-23T14:34:53.735474+0000","last_clean":"2024-04-23T14:34:53.735474+0000","last_became_active":"2024-04-23T14:34:52.751143+0000","last_became_peered":"2024-04-23T14:34:52.751143+0000","last_unstale":"2024-04-23T14:34:53.735474+0000","last_undegraded":"2024-04-23T14:34:53.735474+0000","last_fullsized":"2024-04-23T14:34:53.735474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:48.657971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755102+0000","last_change":"2024-04-23T14:34:53.748022+0000","last_active":"2024-04-23T14:34:53.755102+0000","last_peered":"2024-04-23T14:34:53.755102+0000","last_clean":"2024-04-23T14:34:53.755102+0000","last_became_active":"2024-04-23T14:34:53.747091+0000","last_became_peered":"2024-04-23T14:34:53.747091+0000","last_unstale":"2024-04-23T14:34:53.755102+0000","last_undegraded":"2024-04-23T14:34:53.755102+0000","last_fullsized":"2024-04-23T14:34:53.755102+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:12.423796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.741101+0000","last_change":"2024-04-23T14:34:52.749259+0000","last_active":"2024-04-23T14:34:53.741101+0000","last_peered":"2024-04-23T14:34:53.741101+0000","last_clean":"2024-04-23T14:34:53.741101+0000","last_became_active":"2024-04-23T14:34:52.746963+0000","last_became_peered":"2024-04-23T14:34:52.746963+0000","last_unstale":"2024-04-23T14:34:53.741101+0000","last_undegraded":"2024-04-23T14:34:53.741101+0000","last_fullsized":"2024-04-23T14:34:53.741101+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:24.801128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.746995+0000","last_change":"2024-04-23T14:34:53.741889+0000","last_active":"2024-04-23T14:34:53.746995+0000","last_peered":"2024-04-23T14:34:53.746995+0000","last_clean":"2024-04-23T14:34:53.746995+0000","last_became_active":"2024-04-23T14:34:53.741562+0000","last_became_peered":"2024-04-23T14:34:53.741562+0000","last_unstale":"2024-04-23T14:34:53.746995+0000","last_undegraded":"2024-04-23T14:34:53.746995+0000","last_fullsized":"2024-04-23T14:34:53.746995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:18.589002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.766642+0000","last_change":"2024-04-23T14:34:52.754780+0000","last_active":"2024-04-23T14:34:53.766642+0000","last_peered":"2024-04-23T14:34:53.766642+0000","last_clean":"2024-04-23T14:34:53.766642+0000","last_became_active":"2024-04-23T14:34:52.754441+0000","last_became_peered":"2024-04-23T14:34:52.754441+0000","last_unstale":"2024-04-23T14:34:53.766642+0000","last_undegraded":"2024-04-23T14:34:53.766642+0000","last_fullsized":"2024-04-23T14:34:53.766642+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:35.307312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748768+0000","last_change":"2024-04-23T14:34:53.743468+0000","last_active":"2024-04-23T14:34:53.748768+0000","last_peered":"2024-04-23T14:34:53.748768+0000","last_clean":"2024-04-23T14:34:53.748768+0000","last_became_active":"2024-04-23T14:34:53.743191+0000","last_became_peered":"2024-04-23T14:34:53.743191+0000","last_unstale":"2024-04-23T14:34:53.748768+0000","last_undegraded":"2024-04-23T14:34:53.748768+0000","last_fullsized":"2024-04-23T14:34:53.748768+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:22.146850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.771300+0000","last_change":"2024-04-23T14:34:52.741588+0000","last_active":"2024-04-23T14:34:53.771300+0000","last_peered":"2024-04-23T14:34:53.771300+0000","last_clean":"2024-04-23T14:34:53.771300+0000","last_became_active":"2024-04-23T14:34:52.738712+0000","last_became_peered":"2024-04-23T14:34:52.738712+0000","last_unstale":"2024-04-23T14:34:53.771300+0000","last_undegraded":"2024-04-23T14:34:53.771300+0000","last_fullsized":"2024-04-23T14:34:53.771300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:05.295838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754125+0000","last_change":"2024-04-23T14:34:53.749735+0000","last_active":"2024-04-23T14:34:53.754125+0000","last_peered":"2024-04-23T14:34:53.754125+0000","last_clean":"2024-04-23T14:34:53.754125+0000","last_became_active":"2024-04-23T14:34:53.749437+0000","last_became_peered":"2024-04-23T14:34:53.749437+0000","last_unstale":"2024-04-23T14:34:53.754125+0000","last_undegraded":"2024-04-23T14:34:53.754125+0000","last_fullsized":"2024-04-23T14:34:53.754125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:55:34.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.770928+0000","last_change":"2024-04-23T14:34:52.748818+0000","last_active":"2024-04-23T14:34:53.770928+0000","last_peered":"2024-04-23T14:34:53.770928+0000","last_clean":"2024-04-23T14:34:53.770928+0000","last_became_active":"2024-04-23T14:34:52.748325+0000","last_became_peered":"2024-04-23T14:34:52.748325+0000","last_unstale":"2024-04-23T14:34:53.770928+0000","last_undegraded":"2024-04-23T14:34:53.770928+0000","last_fullsized":"2024-04-23T14:34:53.770928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:25.995545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.746918+0000","last_change":"2024-04-23T14:34:53.739935+0000","last_active":"2024-04-23T14:34:53.746918+0000","last_peered":"2024-04-23T14:34:53.746918+0000","last_clean":"2024-04-23T14:34:53.746918+0000","last_became_active":"2024-04-23T14:34:53.739593+0000","last_became_peered":"2024-04-23T14:34:53.739593+0000","last_unstale":"2024-04-23T14:34:53.746918+0000","last_undegraded":"2024-04-23T14:34:53.746918+0000","last_fullsized":"2024-04-23T14:34:53.746918+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:17:16.116178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.715915+0000","last_change":"2024-04-23T14:34:52.750971+0000","last_active":"2024-04-23T14:34:53.715915+0000","last_peered":"2024-04-23T14:34:53.715915+0000","last_clean":"2024-04-23T14:34:53.715915+0000","last_became_active":"2024-04-23T14:34:52.750515+0000","last_became_peered":"2024-04-23T14:34:52.750515+0000","last_unstale":"2024-04-23T14:34:53.715915+0000","last_undegraded":"2024-04-23T14:34:53.715915+0000","last_fullsized":"2024-04-23T14:34:53.715915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:43.471360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752816+0000","last_change":"2024-04-23T14:34:53.745932+0000","last_active":"2024-04-23T14:34:53.752816+0000","last_peered":"2024-04-23T14:34:53.752816+0000","last_clean":"2024-04-23T14:34:53.752816+0000","last_became_active":"2024-04-23T14:34:53.745611+0000","last_became_peered":"2024-04-23T14:34:53.745611+0000","last_unstale":"2024-04-23T14:34:53.752816+0000","last_undegraded":"2024-04-23T14:34:53.752816+0000","last_fullsized":"2024-04-23T14:34:53.752816+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:29.517466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.737982+0000","last_change":"2024-04-23T14:34:52.756406+0000","last_active":"2024-04-23T14:34:53.737982+0000","last_peered":"2024-04-23T14:34:53.737982+0000","last_clean":"2024-04-23T14:34:53.737982+0000","last_became_active":"2024-04-23T14:34:52.755977+0000","last_became_peered":"2024-04-23T14:34:52.755977+0000","last_unstale":"2024-04-23T14:34:53.737982+0000","last_undegraded":"2024-04-23T14:34:53.737982+0000","last_fullsized":"2024-04-23T14:34:53.737982+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:16.182883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755310+0000","last_change":"2024-04-23T14:34:53.748982+0000","last_active":"2024-04-23T14:34:53.755310+0000","last_peered":"2024-04-23T14:34:53.755310+0000","last_clean":"2024-04-23T14:34:53.755310+0000","last_became_active":"2024-04-23T14:34:53.748702+0000","last_became_peered":"2024-04-23T14:34:53.748702+0000","last_unstale":"2024-04-23T14:34:53.755310+0000","last_undegraded":"2024-04-23T14:34:53.755310+0000","last_fullsized":"2024-04-23T14:34:53.755310+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:55:34.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.766502+0000","last_change":"2024-04-23T14:34:52.755934+0000","last_active":"2024-04-23T14:34:53.766502+0000","last_peered":"2024-04-23T14:34:53.766502+0000","last_clean":"2024-04-23T14:34:53.766502+0000","last_became_active":"2024-04-23T14:34:52.755637+0000","last_became_peered":"2024-04-23T14:34:52.755637+0000","last_unstale":"2024-04-23T14:34:53.766502+0000","last_undegraded":"2024-04-23T14:34:53.766502+0000","last_fullsized":"2024-04-23T14:34:53.766502+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:34.568285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748704+0000","last_change":"2024-04-23T14:34:53.743383+0000","last_active":"2024-04-23T14:34:53.748704+0000","last_peered":"2024-04-23T14:34:53.748704+0000","last_clean":"2024-04-23T14:34:53.748704+0000","last_became_active":"2024-04-23T14:34:53.743103+0000","last_became_peered":"2024-04-23T14:34:53.743103+0000","last_unstale":"2024-04-23T14:34:53.748704+0000","last_undegraded":"2024-04-23T14:34:53.748704+0000","last_fullsized":"2024-04-23T14:34:53.748704+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:05.389774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769124+0000","last_change":"2024-04-23T14:34:52.748487+0000","last_active":"2024-04-23T14:34:53.769124+0000","last_peered":"2024-04-23T14:34:53.769124+0000","last_clean":"2024-04-23T14:34:53.769124+0000","last_became_active":"2024-04-23T14:34:52.748104+0000","last_became_peered":"2024-04-23T14:34:52.748104+0000","last_unstale":"2024-04-23T14:34:53.769124+0000","last_undegraded":"2024-04-23T14:34:53.769124+0000","last_fullsized":"2024-04-23T14:34:53.769124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:42:50.690580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753871+0000","last_change":"2024-04-23T14:34:53.747569+0000","last_active":"2024-04-23T14:34:53.753871+0000","last_peered":"2024-04-23T14:34:53.753871+0000","last_clean":"2024-04-23T14:34:53.753871+0000","last_became_active":"2024-04-23T14:34:53.744824+0000","last_became_peered":"2024-04-23T14:34:53.744824+0000","last_unstale":"2024-04-23T14:34:53.753871+0000","last_undegraded":"2024-04-23T14:34:53.753871+0000","last_fullsized":"2024-04-23T14:34:53.753871+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:37.928012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.771088+0000","last_change":"2024-04-23T14:34:52.748439+0000","last_active":"2024-04-23T14:34:53.771088+0000","last_peered":"2024-04-23T14:34:53.771088+0000","last_clean":"2024-04-23T14:34:53.771088+0000","last_became_active":"2024-04-23T14:34:52.748144+0000","last_became_peered":"2024-04-23T14:34:52.748144+0000","last_unstale":"2024-04-23T14:34:53.771088+0000","last_undegraded":"2024-04-23T14:34:53.771088+0000","last_fullsized":"2024-04-23T14:34:53.771088+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:21.131326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756819+0000","last_change":"2024-04-23T14:34:53.751880+0000","last_active":"2024-04-23T14:34:53.756819+0000","last_peered":"2024-04-23T14:34:53.756819+0000","last_clean":"2024-04-23T14:34:53.756819+0000","last_became_active":"2024-04-23T14:34:53.751573+0000","last_became_peered":"2024-04-23T14:34:53.751573+0000","last_unstale":"2024-04-23T14:34:53.756819+0000","last_undegraded":"2024-04-23T14:34:53.756819+0000","last_fullsized":"2024-04-23T14:34:53.756819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:44.654615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.766116+0000","last_change":"2024-04-23T14:34:52.756623+0000","last_active":"2024-04-23T14:34:53.766116+0000","last_peered":"2024-04-23T14:34:53.766116+0000","last_clean":"2024-04-23T14:34:53.766116+0000","last_became_active":"2024-04-23T14:34:52.756310+0000","last_became_peered":"2024-04-23T14:34:52.756310+0000","last_unstale":"2024-04-23T14:34:53.766116+0000","last_undegraded":"2024-04-23T14:34:53.766116+0000","last_fullsized":"2024-04-23T14:34:53.766116+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:48.257360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748786+0000","last_change":"2024-04-23T14:34:53.744189+0000","last_active":"2024-04-23T14:34:53.748786+0000","last_peered":"2024-04-23T14:34:53.748786+0000","last_clean":"2024-04-23T14:34:53.748786+0000","last_became_active":"2024-04-23T14:34:53.743916+0000","last_became_peered":"2024-04-23T14:34:53.743916+0000","last_unstale":"2024-04-23T14:34:53.748786+0000","last_undegraded":"2024-04-23T14:34:53.748786+0000","last_fullsized":"2024-04-23T14:34:53.748786+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:17.198408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.730478+0000","last_change":"2024-04-23T14:34:52.745673+0000","last_active":"2024-04-23T14:34:53.730478+0000","last_peered":"2024-04-23T14:34:53.730478+0000","last_clean":"2024-04-23T14:34:53.730478+0000","last_became_active":"2024-04-23T14:34:52.745391+0000","last_became_peered":"2024-04-23T14:34:52.745391+0000","last_unstale":"2024-04-23T14:34:53.730478+0000","last_undegraded":"2024-04-23T14:34:53.730478+0000","last_fullsized":"2024-04-23T14:34:53.730478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:11.254821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754150+0000","last_change":"2024-04-23T14:34:53.749948+0000","last_active":"2024-04-23T14:34:53.754150+0000","last_peered":"2024-04-23T14:34:53.754150+0000","last_clean":"2024-04-23T14:34:53.754150+0000","last_became_active":"2024-04-23T14:34:53.749646+0000","last_became_peered":"2024-04-23T14:34:53.749646+0000","last_unstale":"2024-04-23T14:34:53.754150+0000","last_undegraded":"2024-04-23T14:34:53.754150+0000","last_fullsized":"2024-04-23T14:34:53.754150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:14.986148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.767318+0000","last_change":"2024-04-23T14:34:52.747951+0000","last_active":"2024-04-23T14:34:53.767318+0000","last_peered":"2024-04-23T14:34:53.767318+0000","last_clean":"2024-04-23T14:34:53.767318+0000","last_became_active":"2024-04-23T14:34:52.746215+0000","last_became_peered":"2024-04-23T14:34:52.746215+0000","last_unstale":"2024-04-23T14:34:53.767318+0000","last_undegraded":"2024-04-23T14:34:53.767318+0000","last_fullsized":"2024-04-23T14:34:53.767318+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:32:49.137873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756093+0000","last_change":"2024-04-23T14:34:53.750651+0000","last_active":"2024-04-23T14:34:53.756093+0000","last_peered":"2024-04-23T14:34:53.756093+0000","last_clean":"2024-04-23T14:34:53.756093+0000","last_became_active":"2024-04-23T14:34:53.748821+0000","last_became_peered":"2024-04-23T14:34:53.748821+0000","last_unstale":"2024-04-23T14:34:53.756093+0000","last_undegraded":"2024-04-23T14:34:53.756093+0000","last_fullsized":"2024-04-23T14:34:53.756093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:48.866199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.721895+0000","last_change":"2024-04-23T14:34:52.746841+0000","last_active":"2024-04-23T14:34:53.721895+0000","last_peered":"2024-04-23T14:34:53.721895+0000","last_clean":"2024-04-23T14:34:53.721895+0000","last_became_active":"2024-04-23T14:34:52.745713+0000","last_became_peered":"2024-04-23T14:34:52.745713+0000","last_unstale":"2024-04-23T14:34:53.721895+0000","last_undegraded":"2024-04-23T14:34:53.721895+0000","last_fullsized":"2024-04-23T14:34:53.721895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:39:27.509101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752928+0000","last_change":"2024-04-23T14:34:53.742793+0000","last_active":"2024-04-23T14:34:53.752928+0000","last_peered":"2024-04-23T14:34:53.752928+0000","last_clean":"2024-04-23T14:34:53.752928+0000","last_became_active":"2024-04-23T14:34:53.742481+0000","last_became_peered":"2024-04-23T14:34:53.742481+0000","last_unstale":"2024-04-23T14:34:53.752928+0000","last_undegraded":"2024-04-23T14:34:53.752928+0000","last_fullsized":"2024-04-23T14:34:53.752928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:14.986148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.731954+0000","last_change":"2024-04-23T14:34:52.748894+0000","last_active":"2024-04-23T14:34:53.731954+0000","last_peered":"2024-04-23T14:34:53.731954+0000","last_clean":"2024-04-23T14:34:53.731954+0000","last_became_active":"2024-04-23T14:34:52.748008+0000","last_became_peered":"2024-04-23T14:34:52.748008+0000","last_unstale":"2024-04-23T14:34:53.731954+0000","last_undegraded":"2024-04-23T14:34:53.731954+0000","last_fullsized":"2024-04-23T14:34:53.731954+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:04.374250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763032+0000","last_change":"2024-04-23T14:34:53.756864+0000","last_active":"2024-04-23T14:34:53.763032+0000","last_peered":"2024-04-23T14:34:53.763032+0000","last_clean":"2024-04-23T14:34:53.763032+0000","last_became_active":"2024-04-23T14:34:53.756398+0000","last_became_peered":"2024-04-23T14:34:53.756398+0000","last_unstale":"2024-04-23T14:34:53.763032+0000","last_undegraded":"2024-04-23T14:34:53.763032+0000","last_fullsized":"2024-04-23T14:34:53.763032+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:48.866199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769993+0000","last_change":"2024-04-23T14:34:52.745471+0000","last_active":"2024-04-23T14:34:53.769993+0000","last_peered":"2024-04-23T14:34:53.769993+0000","last_clean":"2024-04-23T14:34:53.769993+0000","last_became_active":"2024-04-23T14:34:52.744180+0000","last_became_peered":"2024-04-23T14:34:52.744180+0000","last_unstale":"2024-04-23T14:34:53.769993+0000","last_undegraded":"2024-04-23T14:34:53.769993+0000","last_fullsized":"2024-04-23T14:34:53.769993+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:18.155852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754126+0000","last_change":"2024-04-23T14:34:53.750141+0000","last_active":"2024-04-23T14:34:53.754126+0000","last_peered":"2024-04-23T14:34:53.754126+0000","last_clean":"2024-04-23T14:34:53.754126+0000","last_became_active":"2024-04-23T14:34:53.749846+0000","last_became_peered":"2024-04-23T14:34:53.749846+0000","last_unstale":"2024-04-23T14:34:53.754126+0000","last_undegraded":"2024-04-23T14:34:53.754126+0000","last_fullsized":"2024-04-23T14:34:53.754126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:24:17.505869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.730394+0000","last_change":"2024-04-23T14:34:52.744767+0000","last_active":"2024-04-23T14:34:53.730394+0000","last_peered":"2024-04-23T14:34:53.730394+0000","last_clean":"2024-04-23T14:34:53.730394+0000","last_became_active":"2024-04-23T14:34:52.744492+0000","last_became_peered":"2024-04-23T14:34:52.744492+0000","last_unstale":"2024-04-23T14:34:53.730394+0000","last_undegraded":"2024-04-23T14:34:53.730394+0000","last_fullsized":"2024-04-23T14:34:53.730394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:26.718389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.743972+0000","last_change":"2024-04-23T14:34:53.740242+0000","last_active":"2024-04-23T14:34:53.743972+0000","last_peered":"2024-04-23T14:34:53.743972+0000","last_clean":"2024-04-23T14:34:53.743972+0000","last_became_active":"2024-04-23T14:34:53.739746+0000","last_became_peered":"2024-04-23T14:34:53.739746+0000","last_unstale":"2024-04-23T14:34:53.743972+0000","last_undegraded":"2024-04-23T14:34:53.743972+0000","last_fullsized":"2024-04-23T14:34:53.743972+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:06.311362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.716044+0000","last_change":"2024-04-23T14:34:52.747290+0000","last_active":"2024-04-23T14:34:53.716044+0000","last_peered":"2024-04-23T14:34:53.716044+0000","last_clean":"2024-04-23T14:34:53.716044+0000","last_became_active":"2024-04-23T14:34:52.746973+0000","last_became_peered":"2024-04-23T14:34:52.746973+0000","last_unstale":"2024-04-23T14:34:53.716044+0000","last_undegraded":"2024-04-23T14:34:53.716044+0000","last_fullsized":"2024-04-23T14:34:53.716044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:39:27.509101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762432+0000","last_change":"2024-04-23T14:34:53.756637+0000","last_active":"2024-04-23T14:34:53.762432+0000","last_peered":"2024-04-23T14:34:53.762432+0000","last_clean":"2024-04-23T14:34:53.762432+0000","last_became_active":"2024-04-23T14:34:53.750890+0000","last_became_peered":"2024-04-23T14:34:53.750890+0000","last_unstale":"2024-04-23T14:34:53.762432+0000","last_undegraded":"2024-04-23T14:34:53.762432+0000","last_fullsized":"2024-04-23T14:34:53.762432+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:59.344785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.742567+0000","last_change":"2024-04-23T14:34:52.750145+0000","last_active":"2024-04-23T14:34:53.742567+0000","last_peered":"2024-04-23T14:34:53.742567+0000","last_clean":"2024-04-23T14:34:53.742567+0000","last_became_active":"2024-04-23T14:34:52.749444+0000","last_became_peered":"2024-04-23T14:34:52.749444+0000","last_unstale":"2024-04-23T14:34:53.742567+0000","last_undegraded":"2024-04-23T14:34:53.742567+0000","last_fullsized":"2024-04-23T14:34:53.742567+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:11.254821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.757717+0000","last_change":"2024-04-23T14:34:53.750897+0000","last_active":"2024-04-23T14:34:53.757717+0000","last_peered":"2024-04-23T14:34:53.757717+0000","last_clean":"2024-04-23T14:34:53.757717+0000","last_became_active":"2024-04-23T14:34:53.750504+0000","last_became_peered":"2024-04-23T14:34:53.750504+0000","last_unstale":"2024-04-23T14:34:53.757717+0000","last_undegraded":"2024-04-23T14:34:53.757717+0000","last_fullsized":"2024-04-23T14:34:53.757717+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:25.588167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.722099+0000","last_change":"2024-04-23T14:34:52.751871+0000","last_active":"2024-04-23T14:34:53.722099+0000","last_peered":"2024-04-23T14:34:53.722099+0000","last_clean":"2024-04-23T14:34:53.722099+0000","last_became_active":"2024-04-23T14:34:52.751561+0000","last_became_peered":"2024-04-23T14:34:52.751561+0000","last_unstale":"2024-04-23T14:34:53.722099+0000","last_undegraded":"2024-04-23T14:34:53.722099+0000","last_fullsized":"2024-04-23T14:34:53.722099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:09.401850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762595+0000","last_change":"2024-04-23T14:34:53.755439+0000","last_active":"2024-04-23T14:34:53.762595+0000","last_peered":"2024-04-23T14:34:53.762595+0000","last_clean":"2024-04-23T14:34:53.762595+0000","last_became_active":"2024-04-23T14:34:53.754590+0000","last_became_peered":"2024-04-23T14:34:53.754590+0000","last_unstale":"2024-04-23T14:34:53.762595+0000","last_undegraded":"2024-04-23T14:34:53.762595+0000","last_fullsized":"2024-04-23T14:34:53.762595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:24:17.505869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.725859+0000","last_change":"2024-04-23T14:34:52.740214+0000","last_active":"2024-04-23T14:34:53.725859+0000","last_peered":"2024-04-23T14:34:53.725859+0000","last_clean":"2024-04-23T14:34:53.725859+0000","last_became_active":"2024-04-23T14:34:52.739823+0000","last_became_peered":"2024-04-23T14:34:52.739823+0000","last_unstale":"2024-04-23T14:34:53.725859+0000","last_undegraded":"2024-04-23T14:34:53.725859+0000","last_fullsized":"2024-04-23T14:34:53.725859+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:18.155852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735413+0000","last_change":"2024-04-23T14:34:53.727259+0000","last_active":"2024-04-23T14:34:53.735413+0000","last_peered":"2024-04-23T14:34:53.735413+0000","last_clean":"2024-04-23T14:34:53.735413+0000","last_became_active":"2024-04-23T14:34:53.725262+0000","last_became_peered":"2024-04-23T14:34:53.725262+0000","last_unstale":"2024-04-23T14:34:53.735413+0000","last_undegraded":"2024-04-23T14:34:53.735413+0000","last_fullsized":"2024-04-23T14:34:53.735413+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:15.841877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.722082+0000","last_change":"2024-04-23T14:34:52.740404+0000","last_active":"2024-04-23T14:34:53.722082+0000","last_peered":"2024-04-23T14:34:53.722082+0000","last_clean":"2024-04-23T14:34:53.722082+0000","last_became_active":"2024-04-23T14:34:52.739708+0000","last_became_peered":"2024-04-23T14:34:52.739708+0000","last_unstale":"2024-04-23T14:34:53.722082+0000","last_undegraded":"2024-04-23T14:34:53.722082+0000","last_fullsized":"2024-04-23T14:34:53.722082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:02.162590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755144+0000","last_change":"2024-04-23T14:34:53.749008+0000","last_active":"2024-04-23T14:34:53.755144+0000","last_peered":"2024-04-23T14:34:53.755144+0000","last_clean":"2024-04-23T14:34:53.755144+0000","last_became_active":"2024-04-23T14:34:53.748626+0000","last_became_peered":"2024-04-23T14:34:53.748626+0000","last_unstale":"2024-04-23T14:34:53.755144+0000","last_undegraded":"2024-04-23T14:34:53.755144+0000","last_fullsized":"2024-04-23T14:34:53.755144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:28.524041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.739865+0000","last_change":"2024-04-23T14:34:52.752248+0000","last_active":"2024-04-23T14:34:53.739865+0000","last_peered":"2024-04-23T14:34:53.739865+0000","last_clean":"2024-04-23T14:34:53.739865+0000","last_became_active":"2024-04-23T14:34:52.751493+0000","last_became_peered":"2024-04-23T14:34:52.751493+0000","last_unstale":"2024-04-23T14:34:53.739865+0000","last_undegraded":"2024-04-23T14:34:53.739865+0000","last_fullsized":"2024-04-23T14:34:53.739865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:58.329260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754080+0000","last_change":"2024-04-23T14:34:53.747473+0000","last_active":"2024-04-23T14:34:53.754080+0000","last_peered":"2024-04-23T14:34:53.754080+0000","last_clean":"2024-04-23T14:34:53.754080+0000","last_became_active":"2024-04-23T14:34:53.743201+0000","last_became_peered":"2024-04-23T14:34:53.743201+0000","last_unstale":"2024-04-23T14:34:53.754080+0000","last_undegraded":"2024-04-23T14:34:53.754080+0000","last_fullsized":"2024-04-23T14:34:53.754080+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:22:59.700456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.715473+0000","last_change":"2024-04-23T14:34:52.753618+0000","last_active":"2024-04-23T14:34:53.715473+0000","last_peered":"2024-04-23T14:34:53.715473+0000","last_clean":"2024-04-23T14:34:53.715473+0000","last_became_active":"2024-04-23T14:34:52.753325+0000","last_became_peered":"2024-04-23T14:34:52.753325+0000","last_unstale":"2024-04-23T14:34:53.715473+0000","last_undegraded":"2024-04-23T14:34:53.715473+0000","last_fullsized":"2024-04-23T14:34:53.715473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:25.564056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.731161+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:34:53.731161+0000","last_peered":"2024-04-23T14:34:53.731161+0000","last_clean":"2024-04-23T14:34:53.731161+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:34:53.731161+0000","last_undegraded":"2024-04-23T14:34:53.731161+0000","last_fullsized":"2024-04-23T14:34:53.731161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":874685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":277382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":284809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":285769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":284213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":671744,"data_stored":277382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":277382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":876719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:34:58.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:34:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.468+0000 7f1234e00640 1 Processor -- start 2024-04-23T14:34:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.468+0000 7f1234e00640 1 -- start start 2024-04-23T14:34:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.469+0000 7f1234e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12300d42c0 0x7f123015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.469+0000 7f1234e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 0x7f12300d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.469+0000 7f1234e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f123015edb0 con 0x7f123015e9d0 2024-04-23T14:34:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.469+0000 7f1234e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f12300d3e30 con 0x7f12300d42c0 2024-04-23T14:34:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.469+0000 7f1234e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f123015ff20 con 0x7f12300d1670 2024-04-23T14:34:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.470+0000 7f122de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f123015e9d0 0x7f12300d0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58094/0 (socket says 172.21.15.59:58094) 2024-04-23T14:34:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.470+0000 7f122de00640 1 -- 172.21.15.59:0/2017093842 learned_addr learned my addr 172.21.15.59:0/2017093842 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.470+0000 7f122e800640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 0x7f12300d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.470+0000 7f122d400640 1 --2- 172.21.15.59:0/2017093842 >> v2:172.21.15.76:3300/0 conn(0x7f12300d42c0 0x7f123015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.470+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742128709 0 0) 0x7f123015edb0 con 0x7f123015e9d0 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.470+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1214003520 con 0x7f123015e9d0 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3913096858 0 0) 0x7f1214003520 con 0x7f123015e9d0 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 msgr2=0x7f12300d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 0x7f12300d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 >> v2:172.21.15.76:3300/0 conn(0x7f12300d42c0 msgr2=0x7f123015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 --2- 172.21.15.59:0/2017093842 >> v2:172.21.15.76:3300/0 conn(0x7f12300d42c0 0x7f123015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12300ba630 con 0x7f123015e9d0 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1220001670 con 0x7f123015e9d0 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 >> v1:172.21.15.59:6789/0 conn(0x7f123015e9d0 legacy=0x7f12300d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 0x7f12140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1214003f30 0x7f1214004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1214004d40 0x7f12140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12300d3e30 con 0x7f1214003f30 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f123015ff20 con 0x7f1214004d40 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f121400b9c0 con 0x7f12300d1670 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.471+0000 7f122d400640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1214003f30 0x7f1214004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.472+0000 7f122de00640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 0x7f12140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.472+0000 7f122e800640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1214004d40 0x7f12140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.472+0000 7f122d400640 1 -- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 msgr2=0x7f12140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.472+0000 7f122d400640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12300d1670 0x7f12140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.472+0000 7f122d400640 1 -- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1214004d40 msgr2=0x7f12140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.472+0000 7f122d400640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1214004d40 0x7f12140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.472+0000 7f122d400640 1 -- 172.21.15.59:0/2017093842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1214003520 con 0x7f1214003f30 2024-04-23T14:34:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.473+0000 7f122d400640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1214003f30 0x7f1214004800 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f121c009ac0 tx=0x7f1214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f786e94d966ea10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.473+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c036070 con 0x7f1214003f30 2024-04-23T14:34:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.473+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121c031400 con 0x7f1214003f30 2024-04-23T14:34:58.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.473+0000 7f122ca00640 1 -- 172.21.15.59:0/2017093842 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c009690 con 0x7f1214003f30 2024-04-23T14:34:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.474+0000 7f1234e00640 1 -- 172.21.15.59:0/2017093842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1214003f30 msgr2=0x7f1214004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.474+0000 7f1234e00640 1 --2- 172.21.15.59:0/2017093842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1214003f30 0x7f1214004800 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f121c009ac0 tx=0x7f1214005830 comp rx=0 tx=0).stop 2024-04-23T14:34:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.474+0000 7f122e800640 1 -- 172.21.15.59:0/2017093842 reap_dead start 2024-04-23T14:34:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.474+0000 7f1234e00640 1 -- 172.21.15.59:0/2017093842 shutdown_connections 2024-04-23T14:34:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.474+0000 7f1234e00640 1 -- 172.21.15.59:0/2017093842 >> 172.21.15.59:0/2017093842 conn(0x7f123006cbd0 msgr2=0x7f123006cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.474+0000 7f1234e00640 1 -- 172.21.15.59:0/2017093842 shutdown_connections 2024-04-23T14:34:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.474+0000 7f1234e00640 1 -- 172.21.15.59:0/2017093842 wait complete. 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.476+0000 7f1234e00640 1 Processor -- start 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.476+0000 7f1234e00640 1 -- start start 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f1234e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1214003f30 0x7f12300b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f1234e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12300b49a0 0x7f1230153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f1234e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12300b3440 0x7f12300b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f1234e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12300cfd70 con 0x7f12300b3440 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f1234e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f12300cfa90 con 0x7f12300b49a0 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f1234e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f12300ba0e0 con 0x7f1214003f30 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f122e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12300b3440 0x7f12300b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f122e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12300b3440 0x7f12300b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53150/0 (socket says 172.21.15.59:53150) 2024-04-23T14:34:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.477+0000 7f122e800640 1 -- 172.21.15.59:0/1821915495 learned_addr learned my addr 172.21.15.59:0/1821915495 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122de00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1214003f30 0x7f12300b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122d400640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12300b49a0 0x7f1230153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122e800640 1 -- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1214003f30 msgr2=0x7f12300b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122e800640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1214003f30 0x7f12300b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122e800640 1 -- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12300b49a0 msgr2=0x7f1230153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122e800640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12300b49a0 0x7f1230153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122e800640 1 -- 172.21.15.59:0/1821915495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12300b5180 con 0x7f12300b3440 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f122e800640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12300b3440 0x7f12300b38c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f12300ba910 tx=0x7f122800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f1226000640 1 -- 172.21.15.59:0/1821915495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122801c070 con 0x7f12300b3440 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f1226000640 1 -- 172.21.15.59:0/1821915495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1228017400 con 0x7f12300b3440 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.478+0000 7f1226000640 1 -- 172.21.15.59:0/1821915495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12280177f0 con 0x7f12300b3440 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.479+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12300b5460 con 0x7f12300b3440 2024-04-23T14:34:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.479+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12300c9bb0 con 0x7f12300b3440 2024-04-23T14:34:58.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.480+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1230069190 con 0x7f12300b3440 2024-04-23T14:34:58.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.481+0000 7f1226000640 1 -- 172.21.15.59:0/1821915495 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f122801f070 con 0x7f12300b3440 2024-04-23T14:34:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.482+0000 7f1226000640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f121006c920 0x7f121006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.482+0000 7f1226000640 1 -- 172.21.15.59:0/1821915495 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f122808bb30 con 0x7f12300b3440 2024-04-23T14:34:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.482+0000 7f122de00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f121006c920 0x7f121006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.483+0000 7f122de00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f121006c920 0x7f121006ede0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1220001eb0 tx=0x7f1220000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.486+0000 7f1226000640 1 -- 172.21.15.59:0/1821915495 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1228012070 con 0x7f12300b3440 2024-04-23T14:34:58.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.688+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12300c9d80 con 0x7f121006c920 2024-04-23T14:34:58.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.696+0000 7f1226000640 1 -- 172.21.15.59:0/1821915495 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251887 (secure 0 0 0) 0x7f12300c9d80 con 0x7f121006c920 2024-04-23T14:34:58.699 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:58.702 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.702+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f121006c920 msgr2=0x7f121006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.702+0000 7f1234e00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f121006c920 0x7f121006ede0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1220001eb0 tx=0x7f1220000ed0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.702+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12300b3440 msgr2=0x7f12300b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.702+0000 7f1234e00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12300b3440 0x7f12300b38c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f12300ba910 tx=0x7f122800f4a0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 shutdown_connections 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f121006c920 0x7f121006ede0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12300b3440 0x7f12300b38c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12300b49a0 0x7f1230153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 --2- 172.21.15.59:0/1821915495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1214003f30 0x7f12300b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 >> 172.21.15.59:0/1821915495 conn(0x7f123006cbd0 msgr2=0x7f12301600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 shutdown_connections 2024-04-23T14:34:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.703+0000 7f1234e00640 1 -- 172.21.15.59:0/1821915495 wait complete. 2024-04-23T14:34:58.722 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-04-23T14:34:58.412527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239384,"kb_used_data":6520,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735336,"statfs":{"total":773094113280,"available":772848984064,"internally_reserved":0,"allocated":6676480,"data_stored":3438341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001754"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763675+0000","last_change":"2024-04-23T14:34:53.755861+0000","last_active":"2024-04-23T14:34:53.763675+0000","last_peered":"2024-04-23T14:34:53.763675+0000","last_clean":"2024-04-23T14:34:53.763675+0000","last_became_active":"2024-04-23T14:34:53.755417+0000","last_became_peered":"2024-04-23T14:34:53.755417+0000","last_unstale":"2024-04-23T14:34:53.763675+0000","last_undegraded":"2024-04-23T14:34:53.763675+0000","last_fullsized":"2024-04-23T14:34:53.763675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:58.195992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756198+0000","last_change":"2024-04-23T14:34:53.750957+0000","last_active":"2024-04-23T14:34:53.756198+0000","last_peered":"2024-04-23T14:34:53.756198+0000","last_clean":"2024-04-23T14:34:53.756198+0000","last_became_active":"2024-04-23T14:34:53.750638+0000","last_became_peered":"2024-04-23T14:34:53.750638+0000","last_unstale":"2024-04-23T14:34:53.756198+0000","last_undegraded":"2024-04-23T14:34:53.756198+0000","last_fullsized":"2024-04-23T14:34:53.756198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:43:12.147348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748749+0000","last_change":"2024-04-23T14:34:53.742424+0000","last_active":"2024-04-23T14:34:53.748749+0000","last_peered":"2024-04-23T14:34:53.748749+0000","last_clean":"2024-04-23T14:34:53.748749+0000","last_became_active":"2024-04-23T14:34:53.742111+0000","last_became_peered":"2024-04-23T14:34:53.742111+0000","last_unstale":"2024-04-23T14:34:53.748749+0000","last_undegraded":"2024-04-23T14:34:53.748749+0000","last_fullsized":"2024-04-23T14:34:53.748749+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:49.272884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763230+0000","last_change":"2024-04-23T14:34:53.756960+0000","last_active":"2024-04-23T14:34:53.763230+0000","last_peered":"2024-04-23T14:34:53.763230+0000","last_clean":"2024-04-23T14:34:53.763230+0000","last_became_active":"2024-04-23T14:34:53.756605+0000","last_became_peered":"2024-04-23T14:34:53.756605+0000","last_unstale":"2024-04-23T14:34:53.763230+0000","last_undegraded":"2024-04-23T14:34:53.763230+0000","last_fullsized":"2024-04-23T14:34:53.763230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:09.293204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762662+0000","last_change":"2024-04-23T14:34:53.752994+0000","last_active":"2024-04-23T14:34:53.762662+0000","last_peered":"2024-04-23T14:34:53.762662+0000","last_clean":"2024-04-23T14:34:53.762662+0000","last_became_active":"2024-04-23T14:34:53.752624+0000","last_became_peered":"2024-04-23T14:34:53.752624+0000","last_unstale":"2024-04-23T14:34:53.762662+0000","last_undegraded":"2024-04-23T14:34:53.762662+0000","last_fullsized":"2024-04-23T14:34:53.762662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:20:59.478944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753798+0000","last_change":"2024-04-23T14:34:53.749956+0000","last_active":"2024-04-23T14:34:53.753798+0000","last_peered":"2024-04-23T14:34:53.753798+0000","last_clean":"2024-04-23T14:34:53.753798+0000","last_became_active":"2024-04-23T14:34:53.749643+0000","last_became_peered":"2024-04-23T14:34:53.749643+0000","last_unstale":"2024-04-23T14:34:53.753798+0000","last_undegraded":"2024-04-23T14:34:53.753798+0000","last_fullsized":"2024-04-23T14:34:53.753798+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:53.519921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755988+0000","last_change":"2024-04-23T14:34:53.749975+0000","last_active":"2024-04-23T14:34:53.755988+0000","last_peered":"2024-04-23T14:34:53.755988+0000","last_clean":"2024-04-23T14:34:53.755988+0000","last_became_active":"2024-04-23T14:34:53.749335+0000","last_became_peered":"2024-04-23T14:34:53.749335+0000","last_unstale":"2024-04-23T14:34:53.755988+0000","last_undegraded":"2024-04-23T14:34:53.755988+0000","last_fullsized":"2024-04-23T14:34:53.755988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:16.193142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753452+0000","last_change":"2024-04-23T14:34:53.746610+0000","last_active":"2024-04-23T14:34:53.753452+0000","last_peered":"2024-04-23T14:34:53.753452+0000","last_clean":"2024-04-23T14:34:53.753452+0000","last_became_active":"2024-04-23T14:34:53.745711+0000","last_became_peered":"2024-04-23T14:34:53.745711+0000","last_unstale":"2024-04-23T14:34:53.753452+0000","last_undegraded":"2024-04-23T14:34:53.753452+0000","last_fullsized":"2024-04-23T14:34:53.753452+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.276721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762988+0000","last_change":"2024-04-23T14:34:53.759549+0000","last_active":"2024-04-23T14:34:53.762988+0000","last_peered":"2024-04-23T14:34:53.762988+0000","last_clean":"2024-04-23T14:34:53.762988+0000","last_became_active":"2024-04-23T14:34:53.759032+0000","last_became_peered":"2024-04-23T14:34:53.759032+0000","last_unstale":"2024-04-23T14:34:53.762988+0000","last_undegraded":"2024-04-23T14:34:53.762988+0000","last_fullsized":"2024-04-23T14:34:53.762988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:35:18.587240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735319+0000","last_change":"2024-04-23T14:34:53.724827+0000","last_active":"2024-04-23T14:34:53.735319+0000","last_peered":"2024-04-23T14:34:53.735319+0000","last_clean":"2024-04-23T14:34:53.735319+0000","last_became_active":"2024-04-23T14:34:53.723963+0000","last_became_peered":"2024-04-23T14:34:53.723963+0000","last_unstale":"2024-04-23T14:34:53.735319+0000","last_undegraded":"2024-04-23T14:34:53.735319+0000","last_fullsized":"2024-04-23T14:34:53.735319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:43:49.051906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755209+0000","last_change":"2024-04-23T14:34:53.749641+0000","last_active":"2024-04-23T14:34:53.755209+0000","last_peered":"2024-04-23T14:34:53.755209+0000","last_clean":"2024-04-23T14:34:53.755209+0000","last_became_active":"2024-04-23T14:34:53.749331+0000","last_became_peered":"2024-04-23T14:34:53.749331+0000","last_unstale":"2024-04-23T14:34:53.755209+0000","last_undegraded":"2024-04-23T14:34:53.755209+0000","last_fullsized":"2024-04-23T14:34:53.755209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:40.822873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754165+0000","last_change":"2024-04-23T14:34:53.750866+0000","last_active":"2024-04-23T14:34:53.754165+0000","last_peered":"2024-04-23T14:34:53.754165+0000","last_clean":"2024-04-23T14:34:53.754165+0000","last_became_active":"2024-04-23T14:34:53.750584+0000","last_became_peered":"2024-04-23T14:34:53.750584+0000","last_unstale":"2024-04-23T14:34:53.754165+0000","last_undegraded":"2024-04-23T14:34:53.754165+0000","last_fullsized":"2024-04-23T14:34:53.754165+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:53.519921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756673+0000","last_change":"2024-04-23T14:34:53.753913+0000","last_active":"2024-04-23T14:34:53.756673+0000","last_peered":"2024-04-23T14:34:53.756673+0000","last_clean":"2024-04-23T14:34:53.756673+0000","last_became_active":"2024-04-23T14:34:53.753634+0000","last_became_peered":"2024-04-23T14:34:53.753634+0000","last_unstale":"2024-04-23T14:34:53.756673+0000","last_undegraded":"2024-04-23T14:34:53.756673+0000","last_fullsized":"2024-04-23T14:34:53.756673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:10.417375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.745361+0000","last_change":"2024-04-23T14:34:53.742913+0000","last_active":"2024-04-23T14:34:53.745361+0000","last_peered":"2024-04-23T14:34:53.745361+0000","last_clean":"2024-04-23T14:34:53.745361+0000","last_became_active":"2024-04-23T14:34:53.742589+0000","last_became_peered":"2024-04-23T14:34:53.742589+0000","last_unstale":"2024-04-23T14:34:53.745361+0000","last_undegraded":"2024-04-23T14:34:53.745361+0000","last_fullsized":"2024-04-23T14:34:53.745361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:13.090777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.743241+0000","last_change":"2024-04-23T14:34:53.740259+0000","last_active":"2024-04-23T14:34:53.743241+0000","last_peered":"2024-04-23T14:34:53.743241+0000","last_clean":"2024-04-23T14:34:53.743241+0000","last_became_active":"2024-04-23T14:34:53.739451+0000","last_became_peered":"2024-04-23T14:34:53.739451+0000","last_unstale":"2024-04-23T14:34:53.743241+0000","last_undegraded":"2024-04-23T14:34:53.743241+0000","last_fullsized":"2024-04-23T14:34:53.743241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:18.589002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754458+0000","last_change":"2024-04-23T14:34:53.748499+0000","last_active":"2024-04-23T14:34:53.754458+0000","last_peered":"2024-04-23T14:34:53.754458+0000","last_clean":"2024-04-23T14:34:53.754458+0000","last_became_active":"2024-04-23T14:34:53.748170+0000","last_became_peered":"2024-04-23T14:34:53.748170+0000","last_unstale":"2024-04-23T14:34:53.754458+0000","last_undegraded":"2024-04-23T14:34:53.754458+0000","last_fullsized":"2024-04-23T14:34:53.754458+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:24:17.505869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756222+0000","last_change":"2024-04-23T14:34:53.752021+0000","last_active":"2024-04-23T14:34:53.756222+0000","last_peered":"2024-04-23T14:34:53.756222+0000","last_clean":"2024-04-23T14:34:53.756222+0000","last_became_active":"2024-04-23T14:34:53.751670+0000","last_became_peered":"2024-04-23T14:34:53.751670+0000","last_unstale":"2024-04-23T14:34:53.756222+0000","last_undegraded":"2024-04-23T14:34:53.756222+0000","last_fullsized":"2024-04-23T14:34:53.756222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:22.562112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.759616+0000","last_change":"2024-04-23T14:34:53.753147+0000","last_active":"2024-04-23T14:34:53.759616+0000","last_peered":"2024-04-23T14:34:53.759616+0000","last_clean":"2024-04-23T14:34:53.759616+0000","last_became_active":"2024-04-23T14:34:53.752761+0000","last_became_peered":"2024-04-23T14:34:53.752761+0000","last_unstale":"2024-04-23T14:34:53.759616+0000","last_undegraded":"2024-04-23T14:34:53.759616+0000","last_fullsized":"2024-04-23T14:34:53.759616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:14.986148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.760294+0000","last_change":"2024-04-23T14:34:53.753477+0000","last_active":"2024-04-23T14:34:53.760294+0000","last_peered":"2024-04-23T14:34:53.760294+0000","last_clean":"2024-04-23T14:34:53.760294+0000","last_became_active":"2024-04-23T14:34:53.752640+0000","last_became_peered":"2024-04-23T14:34:53.752640+0000","last_unstale":"2024-04-23T14:34:53.760294+0000","last_undegraded":"2024-04-23T14:34:53.760294+0000","last_fullsized":"2024-04-23T14:34:53.760294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:41:28.069818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.768875+0000","last_change":"2024-04-23T14:34:53.761723+0000","last_active":"2024-04-23T14:34:53.768875+0000","last_peered":"2024-04-23T14:34:53.768875+0000","last_clean":"2024-04-23T14:34:53.768875+0000","last_became_active":"2024-04-23T14:34:53.761194+0000","last_became_peered":"2024-04-23T14:34:53.761194+0000","last_unstale":"2024-04-23T14:34:53.768875+0000","last_undegraded":"2024-04-23T14:34:53.768875+0000","last_fullsized":"2024-04-23T14:34:53.768875+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:25:35.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755268+0000","last_change":"2024-04-23T14:34:53.749269+0000","last_active":"2024-04-23T14:34:53.755268+0000","last_peered":"2024-04-23T14:34:53.755268+0000","last_clean":"2024-04-23T14:34:53.755268+0000","last_became_active":"2024-04-23T14:34:53.748998+0000","last_became_peered":"2024-04-23T14:34:53.748998+0000","last_unstale":"2024-04-23T14:34:53.755268+0000","last_undegraded":"2024-04-23T14:34:53.755268+0000","last_fullsized":"2024-04-23T14:34:53.755268+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:37.928012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.770385+0000","last_change":"2024-04-23T14:34:53.765699+0000","last_active":"2024-04-23T14:34:53.770385+0000","last_peered":"2024-04-23T14:34:53.770385+0000","last_clean":"2024-04-23T14:34:53.770385+0000","last_became_active":"2024-04-23T14:34:53.758512+0000","last_became_peered":"2024-04-23T14:34:53.758512+0000","last_unstale":"2024-04-23T14:34:53.770385+0000","last_undegraded":"2024-04-23T14:34:53.770385+0000","last_fullsized":"2024-04-23T14:34:53.770385+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:15:05.011298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753050+0000","last_change":"2024-04-23T14:34:53.747107+0000","last_active":"2024-04-23T14:34:53.753050+0000","last_peered":"2024-04-23T14:34:53.753050+0000","last_clean":"2024-04-23T14:34:53.753050+0000","last_became_active":"2024-04-23T14:34:53.746786+0000","last_became_peered":"2024-04-23T14:34:53.746786+0000","last_unstale":"2024-04-23T14:34:53.753050+0000","last_undegraded":"2024-04-23T14:34:53.753050+0000","last_fullsized":"2024-04-23T14:34:53.753050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:12.270345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756739+0000","last_change":"2024-04-23T14:34:53.753986+0000","last_active":"2024-04-23T14:34:53.756739+0000","last_peered":"2024-04-23T14:34:53.756739+0000","last_clean":"2024-04-23T14:34:53.756739+0000","last_became_active":"2024-04-23T14:34:53.753413+0000","last_became_peered":"2024-04-23T14:34:53.753413+0000","last_unstale":"2024-04-23T14:34:53.756739+0000","last_undegraded":"2024-04-23T14:34:53.756739+0000","last_fullsized":"2024-04-23T14:34:53.756739+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:43:49.051906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735836+0000","last_change":"2024-04-23T14:34:53.731093+0000","last_active":"2024-04-23T14:34:53.735836+0000","last_peered":"2024-04-23T14:34:53.735836+0000","last_clean":"2024-04-23T14:34:53.735836+0000","last_became_active":"2024-04-23T14:34:53.730744+0000","last_became_peered":"2024-04-23T14:34:53.730744+0000","last_unstale":"2024-04-23T14:34:53.735836+0000","last_undegraded":"2024-04-23T14:34:53.735836+0000","last_fullsized":"2024-04-23T14:34:53.735836+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:17.680506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752311+0000","last_change":"2024-04-23T14:34:53.746566+0000","last_active":"2024-04-23T14:34:53.752311+0000","last_peered":"2024-04-23T14:34:53.752311+0000","last_clean":"2024-04-23T14:34:53.752311+0000","last_became_active":"2024-04-23T14:34:53.746281+0000","last_became_peered":"2024-04-23T14:34:53.746281+0000","last_unstale":"2024-04-23T14:34:53.752311+0000","last_undegraded":"2024-04-23T14:34:53.752311+0000","last_fullsized":"2024-04-23T14:34:53.752311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:12.423796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.757963+0000","last_change":"2024-04-23T14:34:53.751200+0000","last_active":"2024-04-23T14:34:53.757963+0000","last_peered":"2024-04-23T14:34:53.757963+0000","last_clean":"2024-04-23T14:34:53.757963+0000","last_became_active":"2024-04-23T14:34:53.750508+0000","last_became_peered":"2024-04-23T14:34:53.750508+0000","last_unstale":"2024-04-23T14:34:53.757963+0000","last_undegraded":"2024-04-23T14:34:53.757963+0000","last_fullsized":"2024-04-23T14:34:53.757963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:09.293204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756288+0000","last_change":"2024-04-23T14:34:53.753121+0000","last_active":"2024-04-23T14:34:53.756288+0000","last_peered":"2024-04-23T14:34:53.756288+0000","last_clean":"2024-04-23T14:34:53.756288+0000","last_became_active":"2024-04-23T14:34:53.752789+0000","last_became_peered":"2024-04-23T14:34:53.752789+0000","last_unstale":"2024-04-23T14:34:53.756288+0000","last_undegraded":"2024-04-23T14:34:53.756288+0000","last_fullsized":"2024-04-23T14:34:53.756288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:38:03.627993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.758370+0000","last_change":"2024-04-23T14:34:53.749200+0000","last_active":"2024-04-23T14:34:53.758370+0000","last_peered":"2024-04-23T14:34:53.758370+0000","last_clean":"2024-04-23T14:34:53.758370+0000","last_became_active":"2024-04-23T14:34:53.748892+0000","last_became_peered":"2024-04-23T14:34:53.748892+0000","last_unstale":"2024-04-23T14:34:53.758370+0000","last_undegraded":"2024-04-23T14:34:53.758370+0000","last_fullsized":"2024-04-23T14:34:53.758370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:01.770335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752915+0000","last_change":"2024-04-23T14:34:53.745893+0000","last_active":"2024-04-23T14:34:53.752915+0000","last_peered":"2024-04-23T14:34:53.752915+0000","last_clean":"2024-04-23T14:34:53.752915+0000","last_became_active":"2024-04-23T14:34:53.745620+0000","last_became_peered":"2024-04-23T14:34:53.745620+0000","last_unstale":"2024-04-23T14:34:53.752915+0000","last_undegraded":"2024-04-23T14:34:53.752915+0000","last_fullsized":"2024-04-23T14:34:53.752915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:27.733914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.734796+0000","last_change":"2024-04-23T14:34:53.727870+0000","last_active":"2024-04-23T14:34:53.734796+0000","last_peered":"2024-04-23T14:34:53.734796+0000","last_clean":"2024-04-23T14:34:53.734796+0000","last_became_active":"2024-04-23T14:34:53.726533+0000","last_became_peered":"2024-04-23T14:34:53.726533+0000","last_unstale":"2024-04-23T14:34:53.734796+0000","last_undegraded":"2024-04-23T14:34:53.734796+0000","last_fullsized":"2024-04-23T14:34:53.734796+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:52.117154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.764304+0000","last_change":"2024-04-23T14:34:53.759116+0000","last_active":"2024-04-23T14:34:53.764304+0000","last_peered":"2024-04-23T14:34:53.764304+0000","last_clean":"2024-04-23T14:34:53.764304+0000","last_became_active":"2024-04-23T14:34:53.754682+0000","last_became_peered":"2024-04-23T14:34:53.754682+0000","last_unstale":"2024-04-23T14:34:53.764304+0000","last_undegraded":"2024-04-23T14:34:53.764304+0000","last_fullsized":"2024-04-23T14:34:53.764304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:25.697374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755843+0000","last_change":"2024-04-23T14:34:53.742151+0000","last_active":"2024-04-23T14:34:53.755843+0000","last_peered":"2024-04-23T14:34:53.755843+0000","last_clean":"2024-04-23T14:34:53.755843+0000","last_became_active":"2024-04-23T14:34:53.741840+0000","last_became_peered":"2024-04-23T14:34:53.741840+0000","last_unstale":"2024-04-23T14:34:53.755843+0000","last_undegraded":"2024-04-23T14:34:53.755843+0000","last_fullsized":"2024-04-23T14:34:53.755843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:55:34.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.771486+0000","last_change":"2024-04-23T14:34:52.754641+0000","last_active":"2024-04-23T14:34:53.771486+0000","last_peered":"2024-04-23T14:34:53.771486+0000","last_clean":"2024-04-23T14:34:53.771486+0000","last_became_active":"2024-04-23T14:34:52.754335+0000","last_became_peered":"2024-04-23T14:34:52.754335+0000","last_unstale":"2024-04-23T14:34:53.771486+0000","last_undegraded":"2024-04-23T14:34:53.771486+0000","last_fullsized":"2024-04-23T14:34:53.771486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:16.182883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.764251+0000","last_change":"2024-04-23T14:34:53.752296+0000","last_active":"2024-04-23T14:34:53.764251+0000","last_peered":"2024-04-23T14:34:53.764251+0000","last_clean":"2024-04-23T14:34:53.764251+0000","last_became_active":"2024-04-23T14:34:53.748511+0000","last_became_peered":"2024-04-23T14:34:53.748511+0000","last_unstale":"2024-04-23T14:34:53.764251+0000","last_undegraded":"2024-04-23T14:34:53.764251+0000","last_fullsized":"2024-04-23T14:34:53.764251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:13.090777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.767211+0000","last_change":"2024-04-23T14:34:52.754260+0000","last_active":"2024-04-23T14:34:53.767211+0000","last_peered":"2024-04-23T14:34:53.767211+0000","last_clean":"2024-04-23T14:34:53.767211+0000","last_became_active":"2024-04-23T14:34:52.753847+0000","last_became_peered":"2024-04-23T14:34:52.753847+0000","last_unstale":"2024-04-23T14:34:53.767211+0000","last_undegraded":"2024-04-23T14:34:53.767211+0000","last_fullsized":"2024-04-23T14:34:53.767211+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:40:24.681850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.757942+0000","last_change":"2024-04-23T14:34:53.753795+0000","last_active":"2024-04-23T14:34:53.757942+0000","last_peered":"2024-04-23T14:34:53.757942+0000","last_clean":"2024-04-23T14:34:53.757942+0000","last_became_active":"2024-04-23T14:34:53.753341+0000","last_became_peered":"2024-04-23T14:34:53.753341+0000","last_unstale":"2024-04-23T14:34:53.757942+0000","last_undegraded":"2024-04-23T14:34:53.757942+0000","last_fullsized":"2024-04-23T14:34:53.757942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:29.344998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.772698+0000","last_change":"2024-04-23T14:34:52.739160+0000","last_active":"2024-04-23T14:34:53.772698+0000","last_peered":"2024-04-23T14:34:53.772698+0000","last_clean":"2024-04-23T14:34:53.772698+0000","last_became_active":"2024-04-23T14:34:52.738868+0000","last_became_peered":"2024-04-23T14:34:52.738868+0000","last_unstale":"2024-04-23T14:34:53.772698+0000","last_undegraded":"2024-04-23T14:34:53.772698+0000","last_fullsized":"2024-04-23T14:34:53.772698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:36.912488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763099+0000","last_change":"2024-04-23T14:34:53.758066+0000","last_active":"2024-04-23T14:34:53.763099+0000","last_peered":"2024-04-23T14:34:53.763099+0000","last_clean":"2024-04-23T14:34:53.763099+0000","last_became_active":"2024-04-23T14:34:53.756071+0000","last_became_peered":"2024-04-23T14:34:53.756071+0000","last_unstale":"2024-04-23T14:34:53.763099+0000","last_undegraded":"2024-04-23T14:34:53.763099+0000","last_fullsized":"2024-04-23T14:34:53.763099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:27.011070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763179+0000","last_change":"2024-04-23T14:34:52.758549+0000","last_active":"2024-04-23T14:34:53.763179+0000","last_peered":"2024-04-23T14:34:53.763179+0000","last_clean":"2024-04-23T14:34:53.763179+0000","last_became_active":"2024-04-23T14:34:52.758146+0000","last_became_peered":"2024-04-23T14:34:52.758146+0000","last_unstale":"2024-04-23T14:34:53.763179+0000","last_undegraded":"2024-04-23T14:34:53.763179+0000","last_fullsized":"2024-04-23T14:34:53.763179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:26.718389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769678+0000","last_change":"2024-04-23T14:34:53.764746+0000","last_active":"2024-04-23T14:34:53.769678+0000","last_peered":"2024-04-23T14:34:53.769678+0000","last_clean":"2024-04-23T14:34:53.769678+0000","last_became_active":"2024-04-23T14:34:53.758086+0000","last_became_peered":"2024-04-23T14:34:53.758086+0000","last_unstale":"2024-04-23T14:34:53.769678+0000","last_undegraded":"2024-04-23T14:34:53.769678+0000","last_fullsized":"2024-04-23T14:34:53.769678+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:54:35.252357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762449+0000","last_change":"2024-04-23T14:34:52.750060+0000","last_active":"2024-04-23T14:34:53.762449+0000","last_peered":"2024-04-23T14:34:53.762449+0000","last_clean":"2024-04-23T14:34:53.762449+0000","last_became_active":"2024-04-23T14:34:52.749799+0000","last_became_peered":"2024-04-23T14:34:52.749799+0000","last_unstale":"2024-04-23T14:34:53.762449+0000","last_undegraded":"2024-04-23T14:34:53.762449+0000","last_fullsized":"2024-04-23T14:34:53.762449+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:14.826352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769299+0000","last_change":"2024-04-23T14:34:53.763289+0000","last_active":"2024-04-23T14:34:53.769299+0000","last_peered":"2024-04-23T14:34:53.769299+0000","last_clean":"2024-04-23T14:34:53.769299+0000","last_became_active":"2024-04-23T14:34:53.762954+0000","last_became_peered":"2024-04-23T14:34:53.762954+0000","last_unstale":"2024-04-23T14:34:53.769299+0000","last_undegraded":"2024-04-23T14:34:53.769299+0000","last_fullsized":"2024-04-23T14:34:53.769299+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:16:41.227970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769764+0000","last_change":"2024-04-23T14:34:52.743081+0000","last_active":"2024-04-23T14:34:53.769764+0000","last_peered":"2024-04-23T14:34:53.769764+0000","last_clean":"2024-04-23T14:34:53.769764+0000","last_became_active":"2024-04-23T14:34:52.742112+0000","last_became_peered":"2024-04-23T14:34:52.742112+0000","last_unstale":"2024-04-23T14:34:53.769764+0000","last_undegraded":"2024-04-23T14:34:53.769764+0000","last_fullsized":"2024-04-23T14:34:53.769764+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:11.408271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.746522+0000","last_change":"2024-04-23T14:34:53.743066+0000","last_active":"2024-04-23T14:34:53.746522+0000","last_peered":"2024-04-23T14:34:53.746522+0000","last_clean":"2024-04-23T14:34:53.746522+0000","last_became_active":"2024-04-23T14:34:53.742759+0000","last_became_peered":"2024-04-23T14:34:53.742759+0000","last_unstale":"2024-04-23T14:34:53.746522+0000","last_undegraded":"2024-04-23T14:34:53.746522+0000","last_fullsized":"2024-04-23T14:34:53.746522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:03.178115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.737828+0000","last_change":"2024-04-23T14:34:52.749807+0000","last_active":"2024-04-23T14:34:53.737828+0000","last_peered":"2024-04-23T14:34:53.737828+0000","last_clean":"2024-04-23T14:34:53.737828+0000","last_became_active":"2024-04-23T14:34:52.749455+0000","last_became_peered":"2024-04-23T14:34:52.749455+0000","last_unstale":"2024-04-23T14:34:53.737828+0000","last_undegraded":"2024-04-23T14:34:53.737828+0000","last_fullsized":"2024-04-23T14:34:53.737828+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:21.131326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754222+0000","last_change":"2024-04-23T14:34:53.744628+0000","last_active":"2024-04-23T14:34:53.754222+0000","last_peered":"2024-04-23T14:34:53.754222+0000","last_clean":"2024-04-23T14:34:53.754222+0000","last_became_active":"2024-04-23T14:34:53.744071+0000","last_became_peered":"2024-04-23T14:34:53.744071+0000","last_unstale":"2024-04-23T14:34:53.754222+0000","last_undegraded":"2024-04-23T14:34:53.754222+0000","last_fullsized":"2024-04-23T14:34:53.754222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:39.309228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.725375+0000","last_change":"2024-04-23T14:34:52.744553+0000","last_active":"2024-04-23T14:34:53.725375+0000","last_peered":"2024-04-23T14:34:53.725375+0000","last_clean":"2024-04-23T14:34:53.725375+0000","last_became_active":"2024-04-23T14:34:52.744022+0000","last_became_peered":"2024-04-23T14:34:52.744022+0000","last_unstale":"2024-04-23T14:34:53.725375+0000","last_undegraded":"2024-04-23T14:34:53.725375+0000","last_fullsized":"2024-04-23T14:34:53.725375+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:43:48.036381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.745258+0000","last_change":"2024-04-23T14:34:53.742007+0000","last_active":"2024-04-23T14:34:53.745258+0000","last_peered":"2024-04-23T14:34:53.745258+0000","last_clean":"2024-04-23T14:34:53.745258+0000","last_became_active":"2024-04-23T14:34:53.741679+0000","last_became_peered":"2024-04-23T14:34:53.741679+0000","last_unstale":"2024-04-23T14:34:53.745258+0000","last_undegraded":"2024-04-23T14:34:53.745258+0000","last_fullsized":"2024-04-23T14:34:53.745258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:27:40.822873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735474+0000","last_change":"2024-04-23T14:34:52.751589+0000","last_active":"2024-04-23T14:34:53.735474+0000","last_peered":"2024-04-23T14:34:53.735474+0000","last_clean":"2024-04-23T14:34:53.735474+0000","last_became_active":"2024-04-23T14:34:52.751143+0000","last_became_peered":"2024-04-23T14:34:52.751143+0000","last_unstale":"2024-04-23T14:34:53.735474+0000","last_undegraded":"2024-04-23T14:34:53.735474+0000","last_fullsized":"2024-04-23T14:34:53.735474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:48.657971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755102+0000","last_change":"2024-04-23T14:34:53.748022+0000","last_active":"2024-04-23T14:34:53.755102+0000","last_peered":"2024-04-23T14:34:53.755102+0000","last_clean":"2024-04-23T14:34:53.755102+0000","last_became_active":"2024-04-23T14:34:53.747091+0000","last_became_peered":"2024-04-23T14:34:53.747091+0000","last_unstale":"2024-04-23T14:34:53.755102+0000","last_undegraded":"2024-04-23T14:34:53.755102+0000","last_fullsized":"2024-04-23T14:34:53.755102+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:07:12.423796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.741101+0000","last_change":"2024-04-23T14:34:52.749259+0000","last_active":"2024-04-23T14:34:53.741101+0000","last_peered":"2024-04-23T14:34:53.741101+0000","last_clean":"2024-04-23T14:34:53.741101+0000","last_became_active":"2024-04-23T14:34:52.746963+0000","last_became_peered":"2024-04-23T14:34:52.746963+0000","last_unstale":"2024-04-23T14:34:53.741101+0000","last_undegraded":"2024-04-23T14:34:53.741101+0000","last_fullsized":"2024-04-23T14:34:53.741101+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:24.801128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.746995+0000","last_change":"2024-04-23T14:34:53.741889+0000","last_active":"2024-04-23T14:34:53.746995+0000","last_peered":"2024-04-23T14:34:53.746995+0000","last_clean":"2024-04-23T14:34:53.746995+0000","last_became_active":"2024-04-23T14:34:53.741562+0000","last_became_peered":"2024-04-23T14:34:53.741562+0000","last_unstale":"2024-04-23T14:34:53.746995+0000","last_undegraded":"2024-04-23T14:34:53.746995+0000","last_fullsized":"2024-04-23T14:34:53.746995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:18.589002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.766642+0000","last_change":"2024-04-23T14:34:52.754780+0000","last_active":"2024-04-23T14:34:53.766642+0000","last_peered":"2024-04-23T14:34:53.766642+0000","last_clean":"2024-04-23T14:34:53.766642+0000","last_became_active":"2024-04-23T14:34:52.754441+0000","last_became_peered":"2024-04-23T14:34:52.754441+0000","last_unstale":"2024-04-23T14:34:53.766642+0000","last_undegraded":"2024-04-23T14:34:53.766642+0000","last_fullsized":"2024-04-23T14:34:53.766642+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:13:35.307312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748768+0000","last_change":"2024-04-23T14:34:53.743468+0000","last_active":"2024-04-23T14:34:53.748768+0000","last_peered":"2024-04-23T14:34:53.748768+0000","last_clean":"2024-04-23T14:34:53.748768+0000","last_became_active":"2024-04-23T14:34:53.743191+0000","last_became_peered":"2024-04-23T14:34:53.743191+0000","last_unstale":"2024-04-23T14:34:53.748768+0000","last_undegraded":"2024-04-23T14:34:53.748768+0000","last_fullsized":"2024-04-23T14:34:53.748768+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:22.146850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.771300+0000","last_change":"2024-04-23T14:34:52.741588+0000","last_active":"2024-04-23T14:34:53.771300+0000","last_peered":"2024-04-23T14:34:53.771300+0000","last_clean":"2024-04-23T14:34:53.771300+0000","last_became_active":"2024-04-23T14:34:52.738712+0000","last_became_peered":"2024-04-23T14:34:52.738712+0000","last_unstale":"2024-04-23T14:34:53.771300+0000","last_undegraded":"2024-04-23T14:34:53.771300+0000","last_fullsized":"2024-04-23T14:34:53.771300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:05.295838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754125+0000","last_change":"2024-04-23T14:34:53.749735+0000","last_active":"2024-04-23T14:34:53.754125+0000","last_peered":"2024-04-23T14:34:53.754125+0000","last_clean":"2024-04-23T14:34:53.754125+0000","last_became_active":"2024-04-23T14:34:53.749437+0000","last_became_peered":"2024-04-23T14:34:53.749437+0000","last_unstale":"2024-04-23T14:34:53.754125+0000","last_undegraded":"2024-04-23T14:34:53.754125+0000","last_fullsized":"2024-04-23T14:34:53.754125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:55:34.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.770928+0000","last_change":"2024-04-23T14:34:52.748818+0000","last_active":"2024-04-23T14:34:53.770928+0000","last_peered":"2024-04-23T14:34:53.770928+0000","last_clean":"2024-04-23T14:34:53.770928+0000","last_became_active":"2024-04-23T14:34:52.748325+0000","last_became_peered":"2024-04-23T14:34:52.748325+0000","last_unstale":"2024-04-23T14:34:53.770928+0000","last_undegraded":"2024-04-23T14:34:53.770928+0000","last_fullsized":"2024-04-23T14:34:53.770928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:25.995545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.746918+0000","last_change":"2024-04-23T14:34:53.739935+0000","last_active":"2024-04-23T14:34:53.746918+0000","last_peered":"2024-04-23T14:34:53.746918+0000","last_clean":"2024-04-23T14:34:53.746918+0000","last_became_active":"2024-04-23T14:34:53.739593+0000","last_became_peered":"2024-04-23T14:34:53.739593+0000","last_unstale":"2024-04-23T14:34:53.746918+0000","last_undegraded":"2024-04-23T14:34:53.746918+0000","last_fullsized":"2024-04-23T14:34:53.746918+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:17:16.116178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.715915+0000","last_change":"2024-04-23T14:34:52.750971+0000","last_active":"2024-04-23T14:34:53.715915+0000","last_peered":"2024-04-23T14:34:53.715915+0000","last_clean":"2024-04-23T14:34:53.715915+0000","last_became_active":"2024-04-23T14:34:52.750515+0000","last_became_peered":"2024-04-23T14:34:52.750515+0000","last_unstale":"2024-04-23T14:34:53.715915+0000","last_undegraded":"2024-04-23T14:34:53.715915+0000","last_fullsized":"2024-04-23T14:34:53.715915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:43.471360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752816+0000","last_change":"2024-04-23T14:34:53.745932+0000","last_active":"2024-04-23T14:34:53.752816+0000","last_peered":"2024-04-23T14:34:53.752816+0000","last_clean":"2024-04-23T14:34:53.752816+0000","last_became_active":"2024-04-23T14:34:53.745611+0000","last_became_peered":"2024-04-23T14:34:53.745611+0000","last_unstale":"2024-04-23T14:34:53.752816+0000","last_undegraded":"2024-04-23T14:34:53.752816+0000","last_fullsized":"2024-04-23T14:34:53.752816+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:29.517466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.737982+0000","last_change":"2024-04-23T14:34:52.756406+0000","last_active":"2024-04-23T14:34:53.737982+0000","last_peered":"2024-04-23T14:34:53.737982+0000","last_clean":"2024-04-23T14:34:53.737982+0000","last_became_active":"2024-04-23T14:34:52.755977+0000","last_became_peered":"2024-04-23T14:34:52.755977+0000","last_unstale":"2024-04-23T14:34:53.737982+0000","last_undegraded":"2024-04-23T14:34:53.737982+0000","last_fullsized":"2024-04-23T14:34:53.737982+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:16.182883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755310+0000","last_change":"2024-04-23T14:34:53.748982+0000","last_active":"2024-04-23T14:34:53.755310+0000","last_peered":"2024-04-23T14:34:53.755310+0000","last_clean":"2024-04-23T14:34:53.755310+0000","last_became_active":"2024-04-23T14:34:53.748702+0000","last_became_peered":"2024-04-23T14:34:53.748702+0000","last_unstale":"2024-04-23T14:34:53.755310+0000","last_undegraded":"2024-04-23T14:34:53.755310+0000","last_fullsized":"2024-04-23T14:34:53.755310+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:55:34.278627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.766502+0000","last_change":"2024-04-23T14:34:52.755934+0000","last_active":"2024-04-23T14:34:53.766502+0000","last_peered":"2024-04-23T14:34:53.766502+0000","last_clean":"2024-04-23T14:34:53.766502+0000","last_became_active":"2024-04-23T14:34:52.755637+0000","last_became_peered":"2024-04-23T14:34:52.755637+0000","last_unstale":"2024-04-23T14:34:53.766502+0000","last_undegraded":"2024-04-23T14:34:53.766502+0000","last_fullsized":"2024-04-23T14:34:53.766502+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:34.568285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748704+0000","last_change":"2024-04-23T14:34:53.743383+0000","last_active":"2024-04-23T14:34:53.748704+0000","last_peered":"2024-04-23T14:34:53.748704+0000","last_clean":"2024-04-23T14:34:53.748704+0000","last_became_active":"2024-04-23T14:34:53.743103+0000","last_became_peered":"2024-04-23T14:34:53.743103+0000","last_unstale":"2024-04-23T14:34:53.748704+0000","last_undegraded":"2024-04-23T14:34:53.748704+0000","last_fullsized":"2024-04-23T14:34:53.748704+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:05.389774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769124+0000","last_change":"2024-04-23T14:34:52.748487+0000","last_active":"2024-04-23T14:34:53.769124+0000","last_peered":"2024-04-23T14:34:53.769124+0000","last_clean":"2024-04-23T14:34:53.769124+0000","last_became_active":"2024-04-23T14:34:52.748104+0000","last_became_peered":"2024-04-23T14:34:52.748104+0000","last_unstale":"2024-04-23T14:34:53.769124+0000","last_undegraded":"2024-04-23T14:34:53.769124+0000","last_fullsized":"2024-04-23T14:34:53.769124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:42:50.690580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.753871+0000","last_change":"2024-04-23T14:34:53.747569+0000","last_active":"2024-04-23T14:34:53.753871+0000","last_peered":"2024-04-23T14:34:53.753871+0000","last_clean":"2024-04-23T14:34:53.753871+0000","last_became_active":"2024-04-23T14:34:53.744824+0000","last_became_peered":"2024-04-23T14:34:53.744824+0000","last_unstale":"2024-04-23T14:34:53.753871+0000","last_undegraded":"2024-04-23T14:34:53.753871+0000","last_fullsized":"2024-04-23T14:34:53.753871+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:37.928012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.771088+0000","last_change":"2024-04-23T14:34:52.748439+0000","last_active":"2024-04-23T14:34:53.771088+0000","last_peered":"2024-04-23T14:34:53.771088+0000","last_clean":"2024-04-23T14:34:53.771088+0000","last_became_active":"2024-04-23T14:34:52.748144+0000","last_became_peered":"2024-04-23T14:34:52.748144+0000","last_unstale":"2024-04-23T14:34:53.771088+0000","last_undegraded":"2024-04-23T14:34:53.771088+0000","last_fullsized":"2024-04-23T14:34:53.771088+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:21.131326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756819+0000","last_change":"2024-04-23T14:34:53.751880+0000","last_active":"2024-04-23T14:34:53.756819+0000","last_peered":"2024-04-23T14:34:53.756819+0000","last_clean":"2024-04-23T14:34:53.756819+0000","last_became_active":"2024-04-23T14:34:53.751573+0000","last_became_peered":"2024-04-23T14:34:53.751573+0000","last_unstale":"2024-04-23T14:34:53.756819+0000","last_undegraded":"2024-04-23T14:34:53.756819+0000","last_fullsized":"2024-04-23T14:34:53.756819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:44.654615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.766116+0000","last_change":"2024-04-23T14:34:52.756623+0000","last_active":"2024-04-23T14:34:53.766116+0000","last_peered":"2024-04-23T14:34:53.766116+0000","last_clean":"2024-04-23T14:34:53.766116+0000","last_became_active":"2024-04-23T14:34:52.756310+0000","last_became_peered":"2024-04-23T14:34:52.756310+0000","last_unstale":"2024-04-23T14:34:53.766116+0000","last_undegraded":"2024-04-23T14:34:53.766116+0000","last_fullsized":"2024-04-23T14:34:53.766116+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:03:48.257360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.748786+0000","last_change":"2024-04-23T14:34:53.744189+0000","last_active":"2024-04-23T14:34:53.748786+0000","last_peered":"2024-04-23T14:34:53.748786+0000","last_clean":"2024-04-23T14:34:53.748786+0000","last_became_active":"2024-04-23T14:34:53.743916+0000","last_became_peered":"2024-04-23T14:34:53.743916+0000","last_unstale":"2024-04-23T14:34:53.748786+0000","last_undegraded":"2024-04-23T14:34:53.748786+0000","last_fullsized":"2024-04-23T14:34:53.748786+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:17.198408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.730478+0000","last_change":"2024-04-23T14:34:52.745673+0000","last_active":"2024-04-23T14:34:53.730478+0000","last_peered":"2024-04-23T14:34:53.730478+0000","last_clean":"2024-04-23T14:34:53.730478+0000","last_became_active":"2024-04-23T14:34:52.745391+0000","last_became_peered":"2024-04-23T14:34:52.745391+0000","last_unstale":"2024-04-23T14:34:53.730478+0000","last_undegraded":"2024-04-23T14:34:53.730478+0000","last_fullsized":"2024-04-23T14:34:53.730478+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:11.254821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754150+0000","last_change":"2024-04-23T14:34:53.749948+0000","last_active":"2024-04-23T14:34:53.754150+0000","last_peered":"2024-04-23T14:34:53.754150+0000","last_clean":"2024-04-23T14:34:53.754150+0000","last_became_active":"2024-04-23T14:34:53.749646+0000","last_became_peered":"2024-04-23T14:34:53.749646+0000","last_unstale":"2024-04-23T14:34:53.754150+0000","last_undegraded":"2024-04-23T14:34:53.754150+0000","last_fullsized":"2024-04-23T14:34:53.754150+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:14.986148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.767318+0000","last_change":"2024-04-23T14:34:52.747951+0000","last_active":"2024-04-23T14:34:53.767318+0000","last_peered":"2024-04-23T14:34:53.767318+0000","last_clean":"2024-04-23T14:34:53.767318+0000","last_became_active":"2024-04-23T14:34:52.746215+0000","last_became_peered":"2024-04-23T14:34:52.746215+0000","last_unstale":"2024-04-23T14:34:53.767318+0000","last_undegraded":"2024-04-23T14:34:53.767318+0000","last_fullsized":"2024-04-23T14:34:53.767318+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:32:49.137873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.756093+0000","last_change":"2024-04-23T14:34:53.750651+0000","last_active":"2024-04-23T14:34:53.756093+0000","last_peered":"2024-04-23T14:34:53.756093+0000","last_clean":"2024-04-23T14:34:53.756093+0000","last_became_active":"2024-04-23T14:34:53.748821+0000","last_became_peered":"2024-04-23T14:34:53.748821+0000","last_unstale":"2024-04-23T14:34:53.756093+0000","last_undegraded":"2024-04-23T14:34:53.756093+0000","last_fullsized":"2024-04-23T14:34:53.756093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:48.866199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.721895+0000","last_change":"2024-04-23T14:34:52.746841+0000","last_active":"2024-04-23T14:34:53.721895+0000","last_peered":"2024-04-23T14:34:53.721895+0000","last_clean":"2024-04-23T14:34:53.721895+0000","last_became_active":"2024-04-23T14:34:52.745713+0000","last_became_peered":"2024-04-23T14:34:52.745713+0000","last_unstale":"2024-04-23T14:34:53.721895+0000","last_undegraded":"2024-04-23T14:34:53.721895+0000","last_fullsized":"2024-04-23T14:34:53.721895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:39:27.509101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.752928+0000","last_change":"2024-04-23T14:34:53.742793+0000","last_active":"2024-04-23T14:34:53.752928+0000","last_peered":"2024-04-23T14:34:53.752928+0000","last_clean":"2024-04-23T14:34:53.752928+0000","last_became_active":"2024-04-23T14:34:53.742481+0000","last_became_peered":"2024-04-23T14:34:53.742481+0000","last_unstale":"2024-04-23T14:34:53.752928+0000","last_undegraded":"2024-04-23T14:34:53.752928+0000","last_fullsized":"2024-04-23T14:34:53.752928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:14.986148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.731954+0000","last_change":"2024-04-23T14:34:52.748894+0000","last_active":"2024-04-23T14:34:53.731954+0000","last_peered":"2024-04-23T14:34:53.731954+0000","last_clean":"2024-04-23T14:34:53.731954+0000","last_became_active":"2024-04-23T14:34:52.748008+0000","last_became_peered":"2024-04-23T14:34:52.748008+0000","last_unstale":"2024-04-23T14:34:53.731954+0000","last_undegraded":"2024-04-23T14:34:53.731954+0000","last_fullsized":"2024-04-23T14:34:53.731954+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:04.374250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.763032+0000","last_change":"2024-04-23T14:34:53.756864+0000","last_active":"2024-04-23T14:34:53.763032+0000","last_peered":"2024-04-23T14:34:53.763032+0000","last_clean":"2024-04-23T14:34:53.763032+0000","last_became_active":"2024-04-23T14:34:53.756398+0000","last_became_peered":"2024-04-23T14:34:53.756398+0000","last_unstale":"2024-04-23T14:34:53.763032+0000","last_undegraded":"2024-04-23T14:34:53.763032+0000","last_fullsized":"2024-04-23T14:34:53.763032+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:48.866199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.769993+0000","last_change":"2024-04-23T14:34:52.745471+0000","last_active":"2024-04-23T14:34:53.769993+0000","last_peered":"2024-04-23T14:34:53.769993+0000","last_clean":"2024-04-23T14:34:53.769993+0000","last_became_active":"2024-04-23T14:34:52.744180+0000","last_became_peered":"2024-04-23T14:34:52.744180+0000","last_unstale":"2024-04-23T14:34:53.769993+0000","last_undegraded":"2024-04-23T14:34:53.769993+0000","last_fullsized":"2024-04-23T14:34:53.769993+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:18.155852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754126+0000","last_change":"2024-04-23T14:34:53.750141+0000","last_active":"2024-04-23T14:34:53.754126+0000","last_peered":"2024-04-23T14:34:53.754126+0000","last_clean":"2024-04-23T14:34:53.754126+0000","last_became_active":"2024-04-23T14:34:53.749846+0000","last_became_peered":"2024-04-23T14:34:53.749846+0000","last_unstale":"2024-04-23T14:34:53.754126+0000","last_undegraded":"2024-04-23T14:34:53.754126+0000","last_fullsized":"2024-04-23T14:34:53.754126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:24:17.505869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.730394+0000","last_change":"2024-04-23T14:34:52.744767+0000","last_active":"2024-04-23T14:34:53.730394+0000","last_peered":"2024-04-23T14:34:53.730394+0000","last_clean":"2024-04-23T14:34:53.730394+0000","last_became_active":"2024-04-23T14:34:52.744492+0000","last_became_peered":"2024-04-23T14:34:52.744492+0000","last_unstale":"2024-04-23T14:34:53.730394+0000","last_undegraded":"2024-04-23T14:34:53.730394+0000","last_fullsized":"2024-04-23T14:34:53.730394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:40:26.718389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.743972+0000","last_change":"2024-04-23T14:34:53.740242+0000","last_active":"2024-04-23T14:34:53.743972+0000","last_peered":"2024-04-23T14:34:53.743972+0000","last_clean":"2024-04-23T14:34:53.743972+0000","last_became_active":"2024-04-23T14:34:53.739746+0000","last_became_peered":"2024-04-23T14:34:53.739746+0000","last_unstale":"2024-04-23T14:34:53.743972+0000","last_undegraded":"2024-04-23T14:34:53.743972+0000","last_fullsized":"2024-04-23T14:34:53.743972+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:06.311362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.716044+0000","last_change":"2024-04-23T14:34:52.747290+0000","last_active":"2024-04-23T14:34:53.716044+0000","last_peered":"2024-04-23T14:34:53.716044+0000","last_clean":"2024-04-23T14:34:53.716044+0000","last_became_active":"2024-04-23T14:34:52.746973+0000","last_became_peered":"2024-04-23T14:34:52.746973+0000","last_unstale":"2024-04-23T14:34:53.716044+0000","last_undegraded":"2024-04-23T14:34:53.716044+0000","last_fullsized":"2024-04-23T14:34:53.716044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:39:27.509101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762432+0000","last_change":"2024-04-23T14:34:53.756637+0000","last_active":"2024-04-23T14:34:53.762432+0000","last_peered":"2024-04-23T14:34:53.762432+0000","last_clean":"2024-04-23T14:34:53.762432+0000","last_became_active":"2024-04-23T14:34:53.750890+0000","last_became_peered":"2024-04-23T14:34:53.750890+0000","last_unstale":"2024-04-23T14:34:53.762432+0000","last_undegraded":"2024-04-23T14:34:53.762432+0000","last_fullsized":"2024-04-23T14:34:53.762432+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:59.344785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.742567+0000","last_change":"2024-04-23T14:34:52.750145+0000","last_active":"2024-04-23T14:34:53.742567+0000","last_peered":"2024-04-23T14:34:53.742567+0000","last_clean":"2024-04-23T14:34:53.742567+0000","last_became_active":"2024-04-23T14:34:52.749444+0000","last_became_peered":"2024-04-23T14:34:52.749444+0000","last_unstale":"2024-04-23T14:34:53.742567+0000","last_undegraded":"2024-04-23T14:34:53.742567+0000","last_fullsized":"2024-04-23T14:34:53.742567+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:31:11.254821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.757717+0000","last_change":"2024-04-23T14:34:53.750897+0000","last_active":"2024-04-23T14:34:53.757717+0000","last_peered":"2024-04-23T14:34:53.757717+0000","last_clean":"2024-04-23T14:34:53.757717+0000","last_became_active":"2024-04-23T14:34:53.750504+0000","last_became_peered":"2024-04-23T14:34:53.750504+0000","last_unstale":"2024-04-23T14:34:53.757717+0000","last_undegraded":"2024-04-23T14:34:53.757717+0000","last_fullsized":"2024-04-23T14:34:53.757717+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:23:25.588167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.722099+0000","last_change":"2024-04-23T14:34:52.751871+0000","last_active":"2024-04-23T14:34:53.722099+0000","last_peered":"2024-04-23T14:34:53.722099+0000","last_clean":"2024-04-23T14:34:53.722099+0000","last_became_active":"2024-04-23T14:34:52.751561+0000","last_became_peered":"2024-04-23T14:34:52.751561+0000","last_unstale":"2024-04-23T14:34:53.722099+0000","last_undegraded":"2024-04-23T14:34:53.722099+0000","last_fullsized":"2024-04-23T14:34:53.722099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:09.401850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.762595+0000","last_change":"2024-04-23T14:34:53.755439+0000","last_active":"2024-04-23T14:34:53.762595+0000","last_peered":"2024-04-23T14:34:53.762595+0000","last_clean":"2024-04-23T14:34:53.762595+0000","last_became_active":"2024-04-23T14:34:53.754590+0000","last_became_peered":"2024-04-23T14:34:53.754590+0000","last_unstale":"2024-04-23T14:34:53.762595+0000","last_undegraded":"2024-04-23T14:34:53.762595+0000","last_fullsized":"2024-04-23T14:34:53.762595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:24:17.505869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.725859+0000","last_change":"2024-04-23T14:34:52.740214+0000","last_active":"2024-04-23T14:34:53.725859+0000","last_peered":"2024-04-23T14:34:53.725859+0000","last_clean":"2024-04-23T14:34:53.725859+0000","last_became_active":"2024-04-23T14:34:52.739823+0000","last_became_peered":"2024-04-23T14:34:52.739823+0000","last_unstale":"2024-04-23T14:34:53.725859+0000","last_undegraded":"2024-04-23T14:34:53.725859+0000","last_fullsized":"2024-04-23T14:34:53.725859+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:18.155852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.735413+0000","last_change":"2024-04-23T14:34:53.727259+0000","last_active":"2024-04-23T14:34:53.735413+0000","last_peered":"2024-04-23T14:34:53.735413+0000","last_clean":"2024-04-23T14:34:53.735413+0000","last_became_active":"2024-04-23T14:34:53.725262+0000","last_became_peered":"2024-04-23T14:34:53.725262+0000","last_unstale":"2024-04-23T14:34:53.735413+0000","last_undegraded":"2024-04-23T14:34:53.735413+0000","last_fullsized":"2024-04-23T14:34:53.735413+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:15.841877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.722082+0000","last_change":"2024-04-23T14:34:52.740404+0000","last_active":"2024-04-23T14:34:53.722082+0000","last_peered":"2024-04-23T14:34:53.722082+0000","last_clean":"2024-04-23T14:34:53.722082+0000","last_became_active":"2024-04-23T14:34:52.739708+0000","last_became_peered":"2024-04-23T14:34:52.739708+0000","last_unstale":"2024-04-23T14:34:53.722082+0000","last_undegraded":"2024-04-23T14:34:53.722082+0000","last_fullsized":"2024-04-23T14:34:53.722082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:55:02.162590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.755144+0000","last_change":"2024-04-23T14:34:53.749008+0000","last_active":"2024-04-23T14:34:53.755144+0000","last_peered":"2024-04-23T14:34:53.755144+0000","last_clean":"2024-04-23T14:34:53.755144+0000","last_became_active":"2024-04-23T14:34:53.748626+0000","last_became_peered":"2024-04-23T14:34:53.748626+0000","last_unstale":"2024-04-23T14:34:53.755144+0000","last_undegraded":"2024-04-23T14:34:53.755144+0000","last_fullsized":"2024-04-23T14:34:53.755144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:28.524041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.739865+0000","last_change":"2024-04-23T14:34:52.752248+0000","last_active":"2024-04-23T14:34:53.739865+0000","last_peered":"2024-04-23T14:34:53.739865+0000","last_clean":"2024-04-23T14:34:53.739865+0000","last_became_active":"2024-04-23T14:34:52.751493+0000","last_became_peered":"2024-04-23T14:34:52.751493+0000","last_unstale":"2024-04-23T14:34:53.739865+0000","last_undegraded":"2024-04-23T14:34:53.739865+0000","last_fullsized":"2024-04-23T14:34:53.739865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:58.329260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.754080+0000","last_change":"2024-04-23T14:34:53.747473+0000","last_active":"2024-04-23T14:34:53.754080+0000","last_peered":"2024-04-23T14:34:53.754080+0000","last_clean":"2024-04-23T14:34:53.754080+0000","last_became_active":"2024-04-23T14:34:53.743201+0000","last_became_peered":"2024-04-23T14:34:53.743201+0000","last_unstale":"2024-04-23T14:34:53.754080+0000","last_undegraded":"2024-04-23T14:34:53.754080+0000","last_fullsized":"2024-04-23T14:34:53.754080+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:52.699529+0000","last_clean_scrub_stamp":"2024-04-23T14:34:52.699529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:22:59.700456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.715473+0000","last_change":"2024-04-23T14:34:52.753618+0000","last_active":"2024-04-23T14:34:53.715473+0000","last_peered":"2024-04-23T14:34:53.715473+0000","last_clean":"2024-04-23T14:34:53.715473+0000","last_became_active":"2024-04-23T14:34:52.753325+0000","last_became_peered":"2024-04-23T14:34:52.753325+0000","last_unstale":"2024-04-23T14:34:53.715473+0000","last_undegraded":"2024-04-23T14:34:53.715473+0000","last_fullsized":"2024-04-23T14:34:53.715473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:34:51.684005+0000","last_clean_scrub_stamp":"2024-04-23T14:34:51.684005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:21:25.564056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-23T14:34:53.731161+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:34:53.731161+0000","last_peered":"2024-04-23T14:34:53.731161+0000","last_clean":"2024-04-23T14:34:53.731161+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:34:53.731161+0000","last_undegraded":"2024-04-23T14:34:53.731161+0000","last_fullsized":"2024-04-23T14:34:53.731161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":874685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":277382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":284809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":285769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":284213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":671744,"data_stored":277382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":277382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":876719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:34:58.726 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:34:58.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:34:58.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.988+0000 7fcc65e00640 1 Processor -- start 2024-04-23T14:34:58.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.988+0000 7fcc65e00640 1 -- start start 2024-04-23T14:34:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc60146120 0x7fcc60160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc65e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc6006b9d0 0x7fcc6006be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc65e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc60160450 con 0x7fcc600cd790 2024-04-23T14:34:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc65e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcc6006c380 con 0x7fcc6006b9d0 2024-04-23T14:34:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc65e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcc600b2b60 con 0x7fcc60146120 2024-04-23T14:34:58.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc5ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcc600cd790 0x7fcc600b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58106/0 (socket says 172.21.15.59:58106) 2024-04-23T14:34:58.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc5ea00640 1 -- 172.21.15.59:0/306316111 learned_addr learned my addr 172.21.15.59:0/306316111 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:34:58.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc5e000640 1 --2- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc60146120 0x7fcc60160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.990+0000 7fcc5f400640 1 --2- 172.21.15.59:0/306316111 >> v2:172.21.15.76:3300/0 conn(0x7fcc6006b9d0 0x7fcc6006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.991+0000 7fcc5d600640 1 -- 172.21.15.59:0/306316111 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3422457760 0 0) 0x7fcc60160450 con 0x7fcc600cd790 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.991+0000 7fcc5d600640 1 -- 172.21.15.59:0/306316111 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc44003520 con 0x7fcc600cd790 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.991+0000 7fcc5f400640 1 -- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc60146120 msgr2=0x7fcc60160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.991+0000 7fcc5d600640 1 -- 172.21.15.59:0/306316111 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1738083911 0 0) 0x7fcc44003520 con 0x7fcc600cd790 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.991+0000 7fcc5f400640 1 --2- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc60146120 0x7fcc60160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.991+0000 7fcc5f400640 1 -- 172.21.15.59:0/306316111 >> v1:172.21.15.59:6789/0 conn(0x7fcc600cd790 legacy=0x7fcc600b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.991+0000 7fcc5f400640 1 -- 172.21.15.59:0/306316111 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc601606f0 con 0x7fcc6006b9d0 2024-04-23T14:34:58.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.993+0000 7fcc5f400640 1 --2- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc6006be40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097a0 tx=0x7fcc5802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fabc1aa53c0637ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:58.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.993+0000 7fcc5d600640 1 -- 172.21.15.59:0/306316111 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58037070 con 0x7fcc6006b9d0 2024-04-23T14:34:58.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.993+0000 7fcc5d600640 1 -- 172.21.15.59:0/306316111 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5802aba0 con 0x7fcc6006b9d0 2024-04-23T14:34:58.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.993+0000 7fcc5d600640 1 -- 172.21.15.59:0/306316111 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50001a10 con 0x7fcc6006b9d0 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 -- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 msgr2=0x7fcc6006be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 --2- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc6006be40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097a0 tx=0x7fcc5802a110 comp rx=0 tx=0).stop 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 -- 172.21.15.59:0/306316111 shutdown_connections 2024-04-23T14:34:58.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 --2- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc6006be40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 --2- 172.21.15.59:0/306316111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc60146120 0x7fcc60160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:58.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 -- 172.21.15.59:0/306316111 >> 172.21.15.59:0/306316111 conn(0x7fcc60073430 msgr2=0x7fcc6009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:58.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 -- 172.21.15.59:0/306316111 shutdown_connections 2024-04-23T14:34:58.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.994+0000 7fcc65e00640 1 -- 172.21.15.59:0/306316111 wait complete. 2024-04-23T14:34:58.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.996+0000 7fcc65e00640 1 Processor -- start 2024-04-23T14:34:58.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.996+0000 7fcc65e00640 1 -- start start 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.997+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc600afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.997+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc600cd790 0x7fcc6015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.997+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60146120 0x7fcc600c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.997+0000 7fcc65e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc600cdb70 con 0x7fcc60146120 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.997+0000 7fcc65e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc600b2e40 con 0x7fcc6006b9d0 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.997+0000 7fcc65e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcc600cdcf0 con 0x7fcc600cd790 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60146120 0x7fcc600c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60146120 0x7fcc600c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53158/0 (socket says 172.21.15.59:53158) 2024-04-23T14:34:58.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 -- 172.21.15.59:0/2506564504 learned_addr learned my addr 172.21.15.59:0/2506564504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5e000640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc600cd790 0x7fcc6015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5ea00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc600afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 -- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc600cd790 msgr2=0x7fcc6015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc600cd790 0x7fcc6015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 -- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 msgr2=0x7fcc600afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc600afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5f400640 1 -- 172.21.15.59:0/2506564504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc600ae490 con 0x7fcc60146120 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5ea00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc600afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:59.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.998+0000 7fcc5e000640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc600cd790 0x7fcc6015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.999+0000 7fcc5f400640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60146120 0x7fcc600c6f90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097a0 tx=0x7fcc580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.999+0000 7fcc57400640 1 -- 172.21.15.59:0/2506564504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58042070 con 0x7fcc60146120 2024-04-23T14:34:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.999+0000 7fcc57400640 1 -- 172.21.15.59:0/2506564504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5803d640 con 0x7fcc60146120 2024-04-23T14:34:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.999+0000 7fcc57400640 1 -- 172.21.15.59:0/2506564504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58037040 con 0x7fcc60146120 2024-04-23T14:34:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.999+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc600ae730 con 0x7fcc60146120 2024-04-23T14:34:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:58.999+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc600aecf0 con 0x7fcc60146120 2024-04-23T14:34:59.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.000+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc60069190 con 0x7fcc60146120 2024-04-23T14:34:59.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.001+0000 7fcc57400640 1 -- 172.21.15.59:0/2506564504 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc58045070 con 0x7fcc60146120 2024-04-23T14:34:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.003+0000 7fcc57400640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc3c06c8f0 0x7fcc3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.003+0000 7fcc57400640 1 -- 172.21.15.59:0/2506564504 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fcc580ac090 con 0x7fcc60146120 2024-04-23T14:34:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.003+0000 7fcc5ea00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc3c06c8f0 0x7fcc3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.004+0000 7fcc5ea00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc3c06c8f0 0x7fcc3c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc50001580 tx=0x7fcc50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.007+0000 7fcc57400640 1 -- 172.21.15.59:0/2506564504 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc5807a540 con 0x7fcc60146120 2024-04-23T14:34:59.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.287+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcc6009ab80 con 0x7fcc60146120 2024-04-23T14:34:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.292+0000 7fcc57400640 1 -- 172.21.15.59:0/2506564504 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fcc58032e20 con 0x7fcc60146120 2024-04-23T14:34:59.293 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:34:59.293 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:34:59.293 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-23T14:34:59.293 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-23T14:34:59.293 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:34:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc3c06c8f0 msgr2=0x7fcc3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc3c06c8f0 0x7fcc3c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc50001580 tx=0x7fcc50009040 comp rx=0 tx=0).stop 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60146120 msgr2=0x7fcc600c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60146120 0x7fcc600c6f90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097a0 tx=0x7fcc580046c0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 shutdown_connections 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc3c06c8f0 0x7fcc3c06edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcc600cd790 0x7fcc6015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60146120 0x7fcc600c6f90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 --2- 172.21.15.59:0/2506564504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc6006b9d0 0x7fcc600afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.294+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 >> 172.21.15.59:0/2506564504 conn(0x7fcc60073430 msgr2=0x7fcc600ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.295+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 shutdown_connections 2024-04-23T14:34:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.295+0000 7fcc65e00640 1 -- 172.21.15.59:0/2506564504 wait complete. 2024-04-23T14:34:59.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:34:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.526+0000 7fc29a000640 1 Processor -- start 2024-04-23T14:34:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.526+0000 7fc29a000640 1 -- start start 2024-04-23T14:34:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.528+0000 7fc29a000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2940cd050 0x7fc29406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.528+0000 7fc29a000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc2940b3080 0x7fc2940cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.528+0000 7fc29a000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc29415f160 con 0x7fc29415ece0 2024-04-23T14:34:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.528+0000 7fc29a000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc2940ced20 con 0x7fc2940b3080 2024-04-23T14:34:59.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.528+0000 7fc29a000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc29415fcd0 con 0x7fc2940cd050 2024-04-23T14:34:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.528+0000 7fc299600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc2940b3080 0x7fc2940cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.529+0000 7fc299600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc2940b3080 0x7fc2940cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53160/0 (socket says 172.21.15.59:53160) 2024-04-23T14:34:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.529+0000 7fc299600640 1 -- 172.21.15.59:0/605567771 learned_addr learned my addr 172.21.15.59:0/605567771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.529+0000 7fc298c00640 1 --2- 172.21.15.59:0/605567771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2940cd050 0x7fc29406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.529+0000 7fc299600640 1 -- 172.21.15.59:0/605567771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2940cd050 msgr2=0x7fc29406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:59.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.529+0000 7fc299600640 1 --2- 172.21.15.59:0/605567771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2940cd050 0x7fc29406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.529+0000 7fc299600640 1 -- 172.21.15.59:0/605567771 >> v1:172.21.15.76:6789/0 conn(0x7fc29415ece0 legacy=0x7fc2940cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:34:59.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.529+0000 7fc299600640 1 -- 172.21.15.59:0/605567771 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29415f430 con 0x7fc2940b3080 2024-04-23T14:34:59.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.530+0000 7fc299600640 1 --2- 172.21.15.59:0/605567771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940b3080 0x7fc2940cf470 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009e50 tx=0x7fc28c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a93cf4271f225 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc293400640 1 -- 172.21.15.59:0/605567771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c036070 con 0x7fc2940b3080 2024-04-23T14:34:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc293400640 1 -- 172.21.15.59:0/605567771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc28c031600 con 0x7fc2940b3080 2024-04-23T14:34:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc293400640 1 -- 172.21.15.59:0/605567771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0319d0 con 0x7fc2940b3080 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc29a000640 1 -- 172.21.15.59:0/605567771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940b3080 msgr2=0x7fc2940cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc29a000640 1 --2- 172.21.15.59:0/605567771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940b3080 0x7fc2940cf470 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009e50 tx=0x7fc28c02a680 comp rx=0 tx=0).stop 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc29a000640 1 -- 172.21.15.59:0/605567771 shutdown_connections 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc29a000640 1 --2- 172.21.15.59:0/605567771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940b3080 0x7fc2940cf470 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc29a000640 1 --2- 172.21.15.59:0/605567771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2940cd050 0x7fc29406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc29a000640 1 -- 172.21.15.59:0/605567771 >> 172.21.15.59:0/605567771 conn(0x7fc2940b21a0 msgr2=0x7fc2940b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.531+0000 7fc29a000640 1 -- 172.21.15.59:0/605567771 shutdown_connections 2024-04-23T14:34:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.532+0000 7fc29a000640 1 -- 172.21.15.59:0/605567771 wait complete. 2024-04-23T14:34:59.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.533+0000 7fc29a000640 1 Processor -- start 2024-04-23T14:34:59.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.534+0000 7fc29a000640 1 -- start start 2024-04-23T14:34:59.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.534+0000 7fc29a000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2940b3080 0x7fc294158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc29a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940cd050 0x7fc2940cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc29a000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc29415ece0 0x7fc2940b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc29a000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2940cef50 con 0x7fc2940cd050 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc29a000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc29415f9f0 con 0x7fc2940b3080 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc29a000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc294159110 con 0x7fc29415ece0 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc293e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940cd050 0x7fc2940cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc293e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940cd050 0x7fc2940cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53174/0 (socket says 172.21.15.59:53174) 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc293e00640 1 -- 172.21.15.59:0/823137509 learned_addr learned my addr 172.21.15.59:0/823137509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc299600640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc29415ece0 0x7fc2940b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc298c00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2940b3080 0x7fc294158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc293e00640 1 -- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc29415ece0 msgr2=0x7fc2940b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.535+0000 7fc293e00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc29415ece0 0x7fc2940b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc293e00640 1 -- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2940b3080 msgr2=0x7fc294158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc293e00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2940b3080 0x7fc294158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc293e00640 1 -- 172.21.15.59:0/823137509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2940c5f60 con 0x7fc2940cd050 2024-04-23T14:34:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc298c00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2940b3080 0x7fc294158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:34:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc293e00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940cd050 0x7fc2940cb870 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc28000c400 tx=0x7fc28000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc290c00640 1 -- 172.21.15.59:0/823137509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2800052d0 con 0x7fc2940cd050 2024-04-23T14:34:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc290c00640 1 -- 172.21.15.59:0/823137509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc280005490 con 0x7fc2940cd050 2024-04-23T14:34:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc290c00640 1 -- 172.21.15.59:0/823137509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280007260 con 0x7fc2940cd050 2024-04-23T14:34:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2940c6240 con 0x7fc2940cd050 2024-04-23T14:34:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.536+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2940c6800 con 0x7fc2940cd050 2024-04-23T14:34:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.538+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29404fd90 con 0x7fc2940cd050 2024-04-23T14:34:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.539+0000 7fc290c00640 1 -- 172.21.15.59:0/823137509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc280007400 con 0x7fc2940cd050 2024-04-23T14:34:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.539+0000 7fc290c00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc27406c8f0 0x7fc27406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:34:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.540+0000 7fc290c00640 1 -- 172.21.15.59:0/823137509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc28008b090 con 0x7fc2940cd050 2024-04-23T14:34:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.540+0000 7fc298c00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc27406c8f0 0x7fc27406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:34:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.540+0000 7fc298c00640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc27406c8f0 0x7fc27406edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc288006fd0 tx=0x7fc288008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:34:59.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.544+0000 7fc290c00640 1 -- 172.21.15.59:0/823137509 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2800594b0 con 0x7fc2940cd050 2024-04-23T14:34:59.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.788+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc294071de0 con 0x7fc2940cd050 2024-04-23T14:34:59.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.789+0000 7fc290c00640 1 -- 172.21.15.59:0/823137509 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7fc280058db0 con 0x7fc2940cd050 2024-04-23T14:34:59.791 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:34:59.791 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:34:56.248634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:34:59.792 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-04-23T14:34:59.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc27406c8f0 msgr2=0x7fc27406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc27406c8f0 0x7fc27406edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc288006fd0 tx=0x7fc288008040 comp rx=0 tx=0).stop 2024-04-23T14:34:59.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940cd050 msgr2=0x7fc2940cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:34:59.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940cd050 0x7fc2940cb870 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc28000c400 tx=0x7fc28000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 shutdown_connections 2024-04-23T14:34:59.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc27406c8f0 0x7fc27406edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc29415ece0 0x7fc2940b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2940cd050 0x7fc2940cb870 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 --2- 172.21.15.59:0/823137509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2940b3080 0x7fc294158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:34:59.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 >> 172.21.15.59:0/823137509 conn(0x7fc2940b21a0 msgr2=0x7fc29406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:34:59.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.792+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 shutdown_connections 2024-04-23T14:34:59.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:34:59.793+0000 7fc29a000640 1 -- 172.21.15.59:0/823137509 wait complete. 2024-04-23T14:35:00.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:34:53.204986+0000', 'modified': '2024-04-23T14:34:56.248634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6165}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6165': {'gid': 6165, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6836/1087917409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 1087917409}, {'type': 'v1', 'addr': '172.21.15.59:6836', 'nonce': 1087917409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6165, 'qdb_cluster': [6165]} 2024-04-23T14:35:00.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:35:00.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:35:00.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.437+0000 7f5acd200640 1 Processor -- start 2024-04-23T14:35:00.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.437+0000 7f5acd200640 1 -- start start 2024-04-23T14:35:00.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.439+0000 7f5acd200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 0x7f5ac81416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.439+0000 7f5acd200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac815ba70 con 0x7f5ac815c250 2024-04-23T14:35:00.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.439+0000 7f5acd200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac815f800 con 0x7f5ac815f060 2024-04-23T14:35:00.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.439+0000 7f5acd200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8141c20 con 0x7f5ac80d05c0 2024-04-23T14:35:00.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.439+0000 7f5ac7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5ac815c250 0x7f5ac81555d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:41210/0 (socket says 172.21.15.59:41210) 2024-04-23T14:35:00.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.439+0000 7f5ac7e00640 1 -- 172.21.15.59:0/1629237067 learned_addr learned my addr 172.21.15.59:0/1629237067 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:35:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac7400640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 0x7f5ac81416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129789946 0 0) 0x7f5ac815ba70 con 0x7f5ac815c250 2024-04-23T14:35:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5aac003520 con 0x7f5ac815c250 2024-04-23T14:35:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 695339785 0 0) 0x7f5aac003520 con 0x7f5ac815c250 2024-04-23T14:35:00.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 msgr2=0x7f5ac81416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 0x7f5ac81416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 >> v1:172.21.15.76:6789/0 conn(0x7f5ac815f060 legacy=0x7f5ac80d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac815fae0 con 0x7f5ac815c250 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f5ac00019f0 con 0x7f5ac815c250 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 >> v1:172.21.15.59:6789/0 conn(0x7f5ac815c250 legacy=0x7f5ac81555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aac0039e0 0x7f5aac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 0x7f5aac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac004d40 0x7f5aac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8141c20 con 0x7f5aac0039e0 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac00baa0 con 0x7f5aac004d40 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.440+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac00bca0 con 0x7f5ac80d05c0 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7400640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aac0039e0 0x7f5aac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 0x7f5aac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7e00640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac004d40 0x7f5aac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7400640 1 -- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 msgr2=0x7f5aac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7400640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac80d05c0 0x7f5aac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7400640 1 -- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac004d40 msgr2=0x7f5aac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7400640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac004d40 0x7f5aac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7400640 1 -- 172.21.15.59:0/1629237067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac003520 con 0x7f5aac0039e0 2024-04-23T14:35:00.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.441+0000 7f5ac7e00640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac004d40 0x7f5aac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:00.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.442+0000 7f5ac7400640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aac0039e0 0x7f5aac003e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009a90 tx=0x7f5aac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431ac9981ae8595e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8036070 con 0x7f5aac0039e0 2024-04-23T14:35:00.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab8031400 con 0x7f5aac0039e0 2024-04-23T14:35:00.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5ac6000640 1 -- 172.21.15.59:0/1629237067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8009690 con 0x7f5aac0039e0 2024-04-23T14:35:00.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5acd200640 1 -- 172.21.15.59:0/1629237067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aac0039e0 msgr2=0x7f5aac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5acd200640 1 --2- 172.21.15.59:0/1629237067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aac0039e0 0x7f5aac003e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009a90 tx=0x7f5aac005830 comp rx=0 tx=0).stop 2024-04-23T14:35:00.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5ac7e00640 1 -- 172.21.15.59:0/1629237067 reap_dead start 2024-04-23T14:35:00.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5acd200640 1 -- 172.21.15.59:0/1629237067 shutdown_connections 2024-04-23T14:35:00.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5acd200640 1 -- 172.21.15.59:0/1629237067 >> 172.21.15.59:0/1629237067 conn(0x7f5ac806bdf0 msgr2=0x7f5ac806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5acd200640 1 -- 172.21.15.59:0/1629237067 shutdown_connections 2024-04-23T14:35:00.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.443+0000 7f5acd200640 1 -- 172.21.15.59:0/1629237067 wait complete. 2024-04-23T14:35:00.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.445+0000 7f5acd200640 1 Processor -- start 2024-04-23T14:35:00.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.445+0000 7f5acd200640 1 -- start start 2024-04-23T14:35:00.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.446+0000 7f5acd200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0039e0 0x7f5ac8153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.446+0000 7f5acd200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ac8156c90 0x7f5ac809cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.446+0000 7f5acd200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac809d910 0x7f5ac80b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.446+0000 7f5acd200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac815f4c0 con 0x7f5ac8156c90 2024-04-23T14:35:00.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.446+0000 7f5acd200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac815bdc0 con 0x7f5aac0039e0 2024-04-23T14:35:00.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.446+0000 7f5acd200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8155ce0 con 0x7f5ac809d910 2024-04-23T14:35:00.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ac8156c90 0x7f5ac809cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ac8156c90 0x7f5ac809cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54602/0 (socket says 172.21.15.59:54602) 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 -- 172.21.15.59:0/782009243 learned_addr learned my addr 172.21.15.59:0/782009243 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac7400640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0039e0 0x7f5ac8153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 -- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac809d910 msgr2=0x7f5ac80b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac809d910 0x7f5ac80b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 -- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0039e0 msgr2=0x7f5ac8153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0039e0 0x7f5ac8153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 -- 172.21.15.59:0/782009243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac81539d0 con 0x7f5ac8156c90 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac7400640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0039e0 0x7f5ac8153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:00.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.447+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ac8156c90 0x7f5ac809cb60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000e550 tx=0x7f5ab000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.448+0000 7f5abf400640 1 -- 172.21.15.59:0/782009243 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00052e0 con 0x7f5ac8156c90 2024-04-23T14:35:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.448+0000 7f5abf400640 1 -- 172.21.15.59:0/782009243 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0018070 con 0x7f5ac8156c90 2024-04-23T14:35:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.448+0000 7f5abf400640 1 -- 172.21.15.59:0/782009243 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0003810 con 0x7f5ac8156c90 2024-04-23T14:35:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.448+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac809de60 con 0x7f5ac8156c90 2024-04-23T14:35:00.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.448+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac8156490 con 0x7f5ac8156c90 2024-04-23T14:35:00.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.450+0000 7f5abf400640 1 -- 172.21.15.59:0/782009243 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ab00039b0 con 0x7f5ac8156c90 2024-04-23T14:35:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.451+0000 7f5abf400640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa406c920 0x7f5aa406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.451+0000 7f5abf400640 1 -- 172.21.15.59:0/782009243 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5ab008b800 con 0x7f5ac8156c90 2024-04-23T14:35:00.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.451+0000 7f5ac7400640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa406c920 0x7f5aa406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.452+0000 7f5ad15a4740 1 Processor -- start 2024-04-23T14:35:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.452+0000 7f5ad15a4740 1 -- start start 2024-04-23T14:35:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.452+0000 7f5ac7400640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa406c920 0x7f5aa406ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8033000 tx=0x7f5ab8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d41a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d15980 0x5573a4d3e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d40e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5573a4c7bb10 con 0x5573a4d15980 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5573a4c222b0 con 0x5573a4d3dc70 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5573a4c77ad0 con 0x5573a4d3e590 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ac6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d15980 0x5573a4d3e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ac6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d15980 0x5573a4d3e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54612/0 (socket says 172.21.15.59:54612) 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ac6a00640 1 -- 172.21.15.59:0/4169487582 learned_addr learned my addr 172.21.15.59:0/4169487582 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.453+0000 7f5ac7400640 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d41a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ac7e00640 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d40e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ac6a00640 1 -- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 msgr2=0x5573a4d40e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d40e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ac6a00640 1 -- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 msgr2=0x5573a4d41a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d41a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ac6a00640 1 -- 172.21.15.59:0/4169487582 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a498e2d0 con 0x5573a4d15980 2024-04-23T14:35:00.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d15980 0x5573a4d3e050 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0061bd0 tx=0x7f5ab0005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95cd3ca9539a9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ab7e00640 1 -- 172.21.15.59:0/4169487582 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab005d8d0 con 0x5573a4d15980 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ab7e00640 1 -- 172.21.15.59:0/4169487582 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0062100 con 0x5573a4d15980 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.454+0000 7f5ab7e00640 1 -- 172.21.15.59:0/4169487582 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0055b90 con 0x5573a4d15980 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 -- 172.21.15.59:0/4169487582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d15980 msgr2=0x5573a4d3e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d15980 0x5573a4d3e050 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0061bd0 tx=0x7f5ab0005480 comp rx=0 tx=0).stop 2024-04-23T14:35:00.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 -- 172.21.15.59:0/4169487582 shutdown_connections 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d40e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d15980 0x5573a4d3e050 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/4169487582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d41a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 -- 172.21.15.59:0/4169487582 >> 172.21.15.59:0/4169487582 conn(0x5573a4d13580 msgr2=0x5573a4d5be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 -- 172.21.15.59:0/4169487582 shutdown_connections 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.455+0000 7f5ad15a4740 1 -- 172.21.15.59:0/4169487582 wait complete. 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.456+0000 7f5ad15a4740 1 -- start start 2024-04-23T14:35:00.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.456+0000 7f5ad15a4740 1 Processor -- start 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.457+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3dc70 0x5573a4d30da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.457+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.457+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d352a0 0x5573a4d32610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.457+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5573a4d118d0 con 0x5573a4d3dc70 2024-04-23T14:35:00.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.457+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5573a4c35590 con 0x5573a4d352a0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.457+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5573a4d11310 con 0x5573a4d3e590 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.457+0000 7f5ad15a4740 10 client.0 _ll_register_callbacks cb 0x5573a4c183a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3dc70 0x5573a4d30da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3dc70 0x5573a4d30da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54620/0 (socket says 172.21.15.59:54620) 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 -- 172.21.15.59:0/3260919683 learned_addr learned my addr 172.21.15.59:0/3260919683 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7e00640 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d352a0 0x5573a4d32610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 -- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 msgr2=0x5573a4d320d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 -- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d352a0 msgr2=0x5573a4d32610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d352a0 0x5573a4d32610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 -- 172.21.15.59:0/3260919683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a4a586b0 con 0x5573a4d3dc70 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3dc70 0x5573a4d30da0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5ab802ab70 tx=0x7f5ab802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce818589f8f0138 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.458+0000 7f5ab4c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3260919683 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8036050 con 0x5573a4d3dc70 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3260919683 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5ab8003b60 con 0x5573a4d3dc70 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab0062100 con 0x5573a4d3dc70 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3260919683 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5ab8057050 con 0x5573a4d3dc70 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ad15a4740 10 client.6300 fetch_fsmap learned FSMap version 33 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5573a4d344e0 con 0x5573a4d3dc70 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3260919683 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0062100 con 0x5573a4d3dc70 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.459+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3260919683 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f5ab805a020 con 0x5573a4d3dc70 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ad15a4740 10 client.6300 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ad15a4740 10 client.6300 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ad15a4740 20 client.6300 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4d5d580 0x5573a4d3bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ad15a4740 4 client.6300 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:35:00.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573a4c89530 con 0x5573a4d5d580 2024-04-23T14:35:00.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4d5d580 0x5573a4d3bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4d5d580 0x5573a4d3bcb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c376b2de3deda13 server_cookie=f4227759350d18ee in_seq=0 out_seq=0 2024-04-23T14:35:00.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.460+0000 7f5ab4c00640 10 client.6300 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.461+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3260919683 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573a4c89530 con 0x5573a4d5d580 2024-04-23T14:35:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.461+0000 7f5ab4c00640 10 client.6300 handle_command_reply: tid=0 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.461+0000 7f5ad15a4740 1 client.6300 shutdown 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3dc70 msgr2=0x5573a4d30da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3dc70 0x5573a4d30da0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5ab802ab70 tx=0x7f5ab802c000 comp rx=0 tx=0).stop 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 shutdown_connections 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4d5d580 0x5573a4d3bcb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d352a0 0x5573a4d32610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3260919683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3dc70 0x5573a4d30da0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ab4c00640 0 client.6300 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:00.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 >> 172.21.15.59:0/3260919683 conn(0x5573a4d15980 msgr2=0x5573a4d16ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 shutdown_connections 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3260919683 wait complete. 2024-04-23T14:35:00.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.462+0000 7f5ad15a4740 20 client.6300 trim_cache size 0 max 16384 2024-04-23T14:35:00.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.483+0000 7f5ad15a4740 1 Processor -- start 2024-04-23T14:35:00.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.483+0000 7f5ad15a4740 1 -- start start 2024-04-23T14:35:00.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d352a0 0x5573a4d84380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d16900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d5e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac809b9a0 con 0x5573a4d352a0 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81f6500 con 0x5573a4d3dc70 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac809b380 con 0x5573a4d3e590 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d352a0 0x5573a4d84380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d352a0 0x5573a4d84380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54634/0 (socket says 172.21.15.59:54634) 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 -- 172.21.15.59:0/3873647424 learned_addr learned my addr 172.21.15.59:0/3873647424 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 -- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 msgr2=0x5573a4d5e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d5e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 -- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 msgr2=0x5573a4d16900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d16900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d16900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac7400640 1 -- 172.21.15.59:0/3873647424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a4d146c0 con 0x5573a4d352a0 2024-04-23T14:35:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.484+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d16900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.485+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d352a0 0x5573a4d84380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5ab803e4a0 tx=0x7f5ab803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc97f9a368b0361d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.485+0000 7f5ab7400640 1 -- 172.21.15.59:0/3873647424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8036040 con 0x5573a4d352a0 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.485+0000 7f5ab7400640 1 -- 172.21.15.59:0/3873647424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab804f070 con 0x5573a4d352a0 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.485+0000 7f5ab7400640 1 -- 172.21.15.59:0/3873647424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab803e5d0 con 0x5573a4d352a0 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3873647424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d352a0 msgr2=0x5573a4d84380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d352a0 0x5573a4d84380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5ab803e4a0 tx=0x7f5ab803b000 comp rx=0 tx=0).stop 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3873647424 shutdown_connections 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d3e590 0x5573a4d5e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d16900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3873647424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d352a0 0x5573a4d84380 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3873647424 >> 172.21.15.59:0/3873647424 conn(0x5573a4d15980 msgr2=0x5573a4d30bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3873647424 shutdown_connections 2024-04-23T14:35:00.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.486+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3873647424 wait complete. 2024-04-23T14:35:00.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.487+0000 7f5ad15a4740 1 -- start start 2024-04-23T14:35:00.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.487+0000 7f5ad15a4740 1 Processor -- start 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d352a0 0x5573a4dda410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ad15a4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3e590 0x5573a4d7dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5573a4d5d400 con 0x5573a4d3e590 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5573a4d347c0 con 0x5573a4d3dc70 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ad15a4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5573a4d5d9f0 con 0x5573a4d352a0 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ad15a4740 10 client.0 _ll_register_callbacks cb 0x7f5ab8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ac7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3e590 0x5573a4d7dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ac7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3e590 0x5573a4d7dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54644/0 (socket says 172.21.15.59:54644) 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ac7e00640 1 -- 172.21.15.59:0/3467465658 learned_addr learned my addr 172.21.15.59:0/3467465658 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.488+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d352a0 0x5573a4dda410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac7e00640 1 -- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d352a0 msgr2=0x5573a4dda410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac7e00640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d352a0 0x5573a4dda410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac7e00640 1 -- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 msgr2=0x5573a4d3a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac7e00640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d3a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac7e00640 1 -- 172.21.15.59:0/3467465658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a4d3a750 con 0x5573a4d3e590 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac6a00640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d3a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ac7e00640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3e590 0x5573a4d7dfb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0039b0 tx=0x7f5ac000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf25d6f6de1e0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ab4c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3467465658 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0004d70 con 0x5573a4d3e590 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3467465658 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5ac0015720 con 0x5573a4d3e590 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.489+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab804f070 con 0x5573a4d3e590 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3467465658 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5ac0015920 con 0x5573a4d3e590 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 10 client.6306 fetch_fsmap learned FSMap version 33 2024-04-23T14:35:00.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5573a4ddab20 con 0x5573a4d3e590 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3467465658 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab804f070 con 0x5573a4d3e590 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3467465658 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f5ac0015ac0 con 0x5573a4d3e590 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 10 client.6306 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 10 client.6306 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 20 client.6306 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4dd42e0 0x5573a4d43d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 4 client.6306 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "status"}) -- 0x5573a4c833a0 con 0x5573a4dd42e0 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.490+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4dd42e0 0x5573a4d43d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ac7400640 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4dd42e0 0x5573a4d43d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9ae69dd4cb9bbe server_cookie=3415a9db242ee307 in_seq=0 out_seq=0 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ab4c00640 10 client.6306 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ab4c00640 1 -- 172.21.15.59:0/3467465658 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5573a4c833a0 con 0x5573a4dd42e0 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ab4c00640 10 client.6306 handle_command_reply: tid=0 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 client.6306 shutdown 2024-04-23T14:35:00.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3e590 msgr2=0x5573a4d7dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3e590 0x5573a4d7dfb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0039b0 tx=0x7f5ac000fcc0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 shutdown_connections 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x5573a4dd42e0 0x5573a4d43d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5573a4d3e590 0x5573a4d7dfb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5573a4d3dc70 0x5573a4d3a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ab4c00640 0 client.6306 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.491+0000 7f5ad15a4740 1 --2- 172.21.15.59:0/3467465658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5573a4d352a0 0x5573a4dda410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 >> 172.21.15.59:0/3467465658 conn(0x5573a4d15980 msgr2=0x5573a4d5e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 shutdown_connections 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5ad15a4740 1 -- 172.21.15.59:0/3467465658 wait complete. 2024-04-23T14:35:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5ad15a4740 20 client.6306 trim_cache size 0 max 16384 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6165, 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7557472220000001, 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 33, 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 58, 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 58, 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 13.173153756 2024-04-23T14:35:00.497 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:00.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa406c920 msgr2=0x7f5aa406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5acd200640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa406c920 0x7f5aa406ede0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8033000 tx=0x7f5ab8008030 comp rx=0 tx=0).stop 2024-04-23T14:35:00.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ac8156c90 msgr2=0x7f5ac809cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.492+0000 7f5acd200640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ac8156c90 0x7f5ac809cb60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000e550 tx=0x7f5ab000ea10 comp rx=0 tx=0).stop 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 shutdown_connections 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa406c920 0x7f5aa406ede0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac809d910 0x7f5ac80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ac8156c90 0x7f5ac809cb60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 --2- 172.21.15.59:0/782009243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0039e0 0x7f5ac8153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 >> 172.21.15.59:0/782009243 conn(0x7f5ac806bdf0 msgr2=0x7f5ac815e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 shutdown_connections 2024-04-23T14:35:00.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.493+0000 7f5acd200640 1 -- 172.21.15.59:0/782009243 wait complete. 2024-04-23T14:35:00.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:35:00.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.703+0000 7f5dfbe00640 1 Processor -- start 2024-04-23T14:35:00.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.704+0000 7f5dfbe00640 1 -- start start 2024-04-23T14:35:00.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.705+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 0x7f5dfc0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.705+0000 7f5dfbe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dfc0cf030 con 0x7f5dfc0d0d90 2024-04-23T14:35:00.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.705+0000 7f5dfbe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dfc0b6a40 con 0x7f5dfc0d2630 2024-04-23T14:35:00.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.705+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5dfc0b7000 con 0x7f5dfc15e7c0 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.705+0000 7f5dfb400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5dfc0d0d90 0x7f5dfc0cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:41226/0 (socket says 172.21.15.59:41226) 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.705+0000 7f5dfb400640 1 -- 172.21.15.59:0/3589653165 learned_addr learned my addr 172.21.15.59:0/3589653165 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269968680 0 0) 0x7f5df4001110 con 0x7f5dfc0d0d90 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5de0003520 con 0x7f5dfc0d0d90 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5dfaa00640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 0x7f5dfc0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 926717879 0 0) 0x7f5de0003520 con 0x7f5dfc0d0d90 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 msgr2=0x7f5dfc0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 0x7f5dfc0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 >> v1:172.21.15.76:6789/0 conn(0x7f5dfc0d2630 legacy=0x7f5dfc0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc0b6d20 con 0x7f5dfc0d0d90 2024-04-23T14:35:00.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f5df4001840 con 0x7f5dfc0d0d90 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 >> v1:172.21.15.59:6789/0 conn(0x7f5dfc0d0d90 legacy=0x7f5dfc0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 0x7f5de00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004010 0x7f5de0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc0b7000 con 0x7f5de0004d40 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5de000bb80 con 0x7f5de0004010 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.706+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5de000bd80 con 0x7f5dfc15e7c0 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfb400640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfa000640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004010 0x7f5de0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfaa00640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 0x7f5de00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfb400640 1 -- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 msgr2=0x7f5de00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfb400640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc15e7c0 0x7f5de00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfb400640 1 -- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004010 msgr2=0x7f5de0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfb400640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004010 0x7f5de0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.707+0000 7f5dfb400640 1 -- 172.21.15.59:0/3589653165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de0003520 con 0x7f5de0004d40 2024-04-23T14:35:00.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.708+0000 7f5dfb400640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c4c0 tx=0x7f5de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91639c1dffec9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.708+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df400c0e0 con 0x7f5de0004d40 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.708+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df4034070 con 0x7f5de0004d40 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.708+0000 7f5df9600640 1 -- 172.21.15.59:0/3589653165 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4005d70 con 0x7f5de0004d40 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.709+0000 7f5dfbe00640 1 -- 172.21.15.59:0/3589653165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0004d40 msgr2=0x7f5de00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.709+0000 7f5dfbe00640 1 --2- 172.21.15.59:0/3589653165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0004d40 0x7f5de00051c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c4c0 tx=0x7f5de0005830 comp rx=0 tx=0).stop 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.709+0000 7f5dfb400640 1 -- 172.21.15.59:0/3589653165 reap_dead start 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.709+0000 7f5dfbe00640 1 -- 172.21.15.59:0/3589653165 shutdown_connections 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.709+0000 7f5dfbe00640 1 -- 172.21.15.59:0/3589653165 >> 172.21.15.59:0/3589653165 conn(0x7f5dfc09d920 msgr2=0x7f5dfc08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.709+0000 7f5dfbe00640 1 -- 172.21.15.59:0/3589653165 shutdown_connections 2024-04-23T14:35:00.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.709+0000 7f5dfbe00640 1 -- 172.21.15.59:0/3589653165 wait complete. 2024-04-23T14:35:00.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.711+0000 7f5dfbe00640 1 Processor -- start 2024-04-23T14:35:00.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.711+0000 7f5dfbe00640 1 -- start start 2024-04-23T14:35:00.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004d40 0x7f5dfc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc0cb660 0x7f5dfc14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dfc0b2440 0x7f5dfc0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc0b6700 con 0x7f5dfc0b2440 2024-04-23T14:35:00.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc0cf620 con 0x7f5de0004d40 2024-04-23T14:35:00.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5dfc0d1a80 con 0x7f5dfc0cb660 2024-04-23T14:35:00.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dfc0b2440 0x7f5dfc0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dfc0b2440 0x7f5dfc0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54666/0 (socket says 172.21.15.59:54666) 2024-04-23T14:35:00.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.712+0000 7f5dfb400640 1 -- 172.21.15.59:0/2616839007 learned_addr learned my addr 172.21.15.59:0/2616839007 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:00.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfa000640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc0cb660 0x7f5dfc14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfb400640 1 -- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc0cb660 msgr2=0x7f5dfc14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfb400640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc0cb660 0x7f5dfc14c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfaa00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004d40 0x7f5dfc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfb400640 1 -- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004d40 msgr2=0x7f5dfc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfb400640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004d40 0x7f5dfc0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfb400640 1 -- 172.21.15.59:0/2616839007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc14d300 con 0x7f5dfc0b2440 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfaa00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004d40 0x7f5dfc0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfb400640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dfc0b2440 0x7f5dfc0b28f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5df4031040 tx=0x7f5df4001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5df2a00640 1 -- 172.21.15.59:0/2616839007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4007480 con 0x7f5dfc0b2440 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5df2a00640 1 -- 172.21.15.59:0/2616839007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df403f030 con 0x7f5dfc0b2440 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5df2a00640 1 -- 172.21.15.59:0/2616839007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4000ed0 con 0x7f5dfc0b2440 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.713+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc0cc1b0 con 0x7f5dfc0b2440 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.714+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc0b1f40 con 0x7f5dfc0b2440 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.715+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dfc0cc6d0 con 0x7f5dfc0b2440 2024-04-23T14:35:00.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.716+0000 7f5df2a00640 1 -- 172.21.15.59:0/2616839007 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5df402f480 con 0x7f5dfc0b2440 2024-04-23T14:35:00.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.717+0000 7f5df2a00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5dd806ca90 0x7f5dd806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:00.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.717+0000 7f5df2a00640 1 -- 172.21.15.59:0/2616839007 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5df40ac5e0 con 0x7f5dfc0b2440 2024-04-23T14:35:00.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.717+0000 7f5dfaa00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5dd806ca90 0x7f5dd806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:00.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.718+0000 7f5dfaa00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5dd806ca90 0x7f5dd806ef50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009f10 tx=0x7f5dec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:00.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.721+0000 7f5df2a00640 1 -- 172.21.15.59:0/2616839007 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dfc0cc6d0 con 0x7f5dfc0b2440 2024-04-23T14:35:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.974+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5dfc04fd90 con 0x7f5dfc0b2440 2024-04-23T14:35:00.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.975+0000 7f5df2a00640 1 -- 172.21.15.59:0/2616839007 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f5dfc0cf030 con 0x7f5dfc0b2440 2024-04-23T14:35:00.976 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:35:00.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.977+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5dd806ca90 msgr2=0x7f5dd806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.977+0000 7f5dfbe00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5dd806ca90 0x7f5dd806ef50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009f10 tx=0x7f5dec00b040 comp rx=0 tx=0).stop 2024-04-23T14:35:00.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.977+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dfc0b2440 msgr2=0x7f5dfc0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:00.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.977+0000 7f5dfbe00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dfc0b2440 0x7f5dfc0b28f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5df4031040 tx=0x7f5df4001c10 comp rx=0 tx=0).stop 2024-04-23T14:35:00.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 shutdown_connections 2024-04-23T14:35:00.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5dd806ca90 0x7f5dd806ef50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dfc0cb660 0x7f5dfc14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dfc0b2440 0x7f5dfc0b28f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 --2- 172.21.15.59:0/2616839007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5de0004d40 0x7f5dfc0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:00.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 >> 172.21.15.59:0/2616839007 conn(0x7f5dfc09d920 msgr2=0x7f5dfc160850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:00.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 shutdown_connections 2024-04-23T14:35:00.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:00.978+0000 7f5dfbe00640 1 -- 172.21.15.59:0/2616839007 wait complete. 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:35:00.998 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:35:00.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:00.999 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:35:01.050 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42248sec preferred_lft 42248sec 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:35:01.051 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:35:01.052 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:35:01.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:01.054 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:35:01.139 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:35:01.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:01.144 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:35:01.200 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:35:01.200 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:35:01.201 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:35:01.201 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:35:01.201 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:35:01.201 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:35:01.201 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:35:01.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:35:01.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:35:01.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:35:01.201 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:35:01.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:35:01.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:35:01.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:01.422 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:35:01.472 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:35:01.472 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:35:01.473 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:35:01.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:01.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:35:01.560 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:35:01.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:35:01.560 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:35:01.616 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:35:01.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:35:01.616 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:35:01.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:35:01.769 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:35:01.823 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:35:01.824 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:35:01.825 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:35:01.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:01.826 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:35:01.878 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:35:01.879 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:35:01.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:35:01.966 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:35:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:35:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:35:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:35:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:35:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:35:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:35:02.009 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:35:02.010 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:35:02.011 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:35:02.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:02.012 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:35:02.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:35:02.078+0000 7f4488b77480 -1 init, newargv = 0x555f93ed84e0 newargc=15 2024-04-23T14:35:02.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[54012]: starting ceph client 2024-04-23T14:35:02.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[54012]: starting fuse 2024-04-23T14:35:02.095 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:35:02.095 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:35:02.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:35:02.095 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:35:02.095 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:35:02.096 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:35:02.097 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:35:02.291 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.54012.asok 2024-04-23T14:35:02.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:35:02.308 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54012.asok 2024-04-23T14:35:02.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:02.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54012.asok status 2024-04-23T14:35:02.548 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:02.548 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:35:02.548 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "54012", 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6318, 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 6318 2024-04-23T14:35:02.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3334866086 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3334866086 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.6318 192.168.0.1:0/3334866086", 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3334866086", 2024-04-23T14:35:02.550 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:35:02.551 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 33, 2024-04-23T14:35:02.551 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 58, 2024-04-23T14:35:02.551 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:35:02.551 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:35:02.551 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:35:02.551 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:02.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:02.564 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:35:02.564 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:35:02.621 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:35:02.622 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:35:02.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:02.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:35:02.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:02.711 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-23T14:35:02.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:02.779 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-23T14:35:02.848 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:35:02.848 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:35:02.848 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:35:02.848 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:35:02.848 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-23T14:35:02.848 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:35:02.995 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-23T14:35:03.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:35:03.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:35:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.269+0000 7ffbba400640 1 Processor -- start 2024-04-23T14:35:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.269+0000 7ffbba400640 1 -- start start 2024-04-23T14:35:03.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.271+0000 7ffbba400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffbb40b5ff0 0x7ffbb40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.271+0000 7ffbba400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40cf030 con 0x7ffbb40d0d90 2024-04-23T14:35:03.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.271+0000 7ffbba400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40b6a40 con 0x7ffbb415e7c0 2024-04-23T14:35:03.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.271+0000 7ffbba400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40b7000 con 0x7ffbb40b5ff0 2024-04-23T14:35:03.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.271+0000 7ffbb9a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffbb40d0d90 0x7ffbb40cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:41230/0 (socket says 172.21.15.59:41230) 2024-04-23T14:35:03.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.271+0000 7ffbb9a00640 1 -- 172.21.15.59:0/594253921 learned_addr learned my addr 172.21.15.59:0/594253921 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:35:03.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857551633 0 0) 0x7ffbac001490 con 0x7ffbb40d0d90 2024-04-23T14:35:03.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3e00640 1 --2- 172.21.15.59:0/594253921 >> v2:172.21.15.76:3300/0 conn(0x7ffbb40b5ff0 0x7ffbb40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb98003520 con 0x7ffbb40d0d90 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 470898184 0 0) 0x7ffb98003520 con 0x7ffbb40d0d90 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 >> v2:172.21.15.76:3300/0 conn(0x7ffbb40b5ff0 msgr2=0x7ffbb40b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 --2- 172.21.15.59:0/594253921 >> v2:172.21.15.76:3300/0 conn(0x7ffbb40b5ff0 0x7ffbb40b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 >> v1:172.21.15.76:6789/0 conn(0x7ffbb415e7c0 legacy=0x7ffbb40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b6d20 con 0x7ffbb40d0d90 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7ffbac001bc0 con 0x7ffbb40d0d90 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 >> v1:172.21.15.59:6789/0 conn(0x7ffbb40d0d90 legacy=0x7ffbb40cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffb98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffb98004800 0x7ffb98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb980051c0 0x7ffb9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b7000 con 0x7ffb980039b0 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9800c060 con 0x7ffb980051c0 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.272+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9800c260 con 0x7ffb98004800 2024-04-23T14:35:03.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9000640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffb98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb3e00640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffb98004800 0x7ffb98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9000640 1 -- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffb98004800 msgr2=0x7ffb98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9000640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffb98004800 0x7ffb98004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9000640 1 -- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb980051c0 msgr2=0x7ffb9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9a00640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb980051c0 0x7ffb9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9000640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb980051c0 0x7ffb9800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9a00640 1 -- 172.21.15.59:0/594253921 reap_dead start 2024-04-23T14:35:03.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.273+0000 7ffbb9000640 1 -- 172.21.15.59:0/594253921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb98003520 con 0x7ffb980039b0 2024-04-23T14:35:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.274+0000 7ffbb9000640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffb98003dd0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001750 tx=0x7ffb98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedfe80440773660 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4001370 con 0x7ffb980039b0 2024-04-23T14:35:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba40052d0 con 0x7ffb980039b0 2024-04-23T14:35:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbb3400640 1 -- 172.21.15.59:0/594253921 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba402abc0 con 0x7ffb980039b0 2024-04-23T14:35:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbba400640 1 -- 172.21.15.59:0/594253921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 msgr2=0x7ffb98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbba400640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffb98003dd0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001750 tx=0x7ffb98005830 comp rx=0 tx=0).stop 2024-04-23T14:35:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbba400640 1 -- 172.21.15.59:0/594253921 shutdown_connections 2024-04-23T14:35:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbba400640 1 --2- 172.21.15.59:0/594253921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffb98003dd0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.275+0000 7ffbba400640 1 -- 172.21.15.59:0/594253921 >> 172.21.15.59:0/594253921 conn(0x7ffbb409d920 msgr2=0x7ffbb408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.276+0000 7ffbba400640 1 -- 172.21.15.59:0/594253921 shutdown_connections 2024-04-23T14:35:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.276+0000 7ffbba400640 1 -- 172.21.15.59:0/594253921 wait complete. 2024-04-23T14:35:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.277+0000 7ffbba400640 1 Processor -- start 2024-04-23T14:35:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.278+0000 7ffbba400640 1 -- start start 2024-04-23T14:35:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbba400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffbb40c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbba400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbb40c9bc0 0x7ffbb40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbba400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbb40b2730 0x7ffbb40d4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbba400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40d1a80 con 0x7ffb980039b0 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbba400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40cf620 con 0x7ffbb40c9bc0 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbba400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40d1c00 con 0x7ffbb40b2730 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffbb40c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffbb40c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54680/0 (socket says 172.21.15.59:54680) 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9000640 1 -- 172.21.15.59:0/2477400632 learned_addr learned my addr 172.21.15.59:0/2477400632 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb3e00640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbb40c9bc0 0x7ffbb40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9a00640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbb40b2730 0x7ffbb40d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9000640 1 -- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbb40b2730 msgr2=0x7ffbb40d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9000640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbb40b2730 0x7ffbb40d4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9000640 1 -- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbb40c9bc0 msgr2=0x7ffbb40b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.279+0000 7ffbb9000640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbb40c9bc0 0x7ffbb40b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbb9000640 1 -- 172.21.15.59:0/2477400632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b2bb0 con 0x7ffb980039b0 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbb9a00640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbb40b2730 0x7ffbb40d4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbb9000640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffbb40c7f00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffba4000c00 tx=0x7ffba4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbb0c00640 1 -- 172.21.15.59:0/2477400632 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba403f030 con 0x7ffb980039b0 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbb0c00640 1 -- 172.21.15.59:0/2477400632 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba4005480 con 0x7ffb980039b0 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb40b2e90 con 0x7ffb980039b0 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbb0c00640 1 -- 172.21.15.59:0/2477400632 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4037040 con 0x7ffb980039b0 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.280+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb40cc6a0 con 0x7ffb980039b0 2024-04-23T14:35:03.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.282+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb404fd90 con 0x7ffb980039b0 2024-04-23T14:35:03.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.283+0000 7ffbb0c00640 1 -- 172.21.15.59:0/2477400632 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffba4042070 con 0x7ffb980039b0 2024-04-23T14:35:03.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.284+0000 7ffbb0c00640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffb9006c8f0 0x7ffb9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.284+0000 7ffbb0c00640 1 -- 172.21.15.59:0/2477400632 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ffba40ab670 con 0x7ffb980039b0 2024-04-23T14:35:03.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.284+0000 7ffbb3e00640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffb9006c8f0 0x7ffb9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.285+0000 7ffbb3e00640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffb9006c8f0 0x7ffb9006edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40b3870 tx=0x7ffb9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.289+0000 7ffbb0c00640 1 -- 172.21.15.59:0/2477400632 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffba4079a00 con 0x7ffb980039b0 2024-04-23T14:35:03.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.540+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb415d770 con 0x7ffb980039b0 2024-04-23T14:35:03.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.542+0000 7ffbb0c00640 1 -- 172.21.15.59:0/2477400632 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7ffba4079300 con 0x7ffb980039b0 2024-04-23T14:35:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:35:03.543 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:34:56.248634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:35:03.543 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-04-23T14:35:03.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffb9006c8f0 msgr2=0x7ffb9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffb9006c8f0 0x7ffb9006edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40b3870 tx=0x7ffb9c008040 comp rx=0 tx=0).stop 2024-04-23T14:35:03.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 msgr2=0x7ffbb40c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffbb40c7f00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffba4000c00 tx=0x7ffba4001120 comp rx=0 tx=0).stop 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 shutdown_connections 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffb9006c8f0 0x7ffb9006edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbb40b2730 0x7ffbb40d4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbb40c9bc0 0x7ffbb40b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 --2- 172.21.15.59:0/2477400632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb980039b0 0x7ffbb40c7f00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.544+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 >> 172.21.15.59:0/2477400632 conn(0x7ffbb409d920 msgr2=0x7ffbb406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.545+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 shutdown_connections 2024-04-23T14:35:03.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.545+0000 7ffbba400640 1 -- 172.21.15.59:0/2477400632 wait complete. 2024-04-23T14:35:03.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-23T14:35:03.819 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:35:03.820 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:35:03.820 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-23T14:35:03.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.929+0000 7fa33610ff40 1 Processor -- start 2024-04-23T14:35:03.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.929+0000 7fa33610ff40 1 -- start start 2024-04-23T14:35:03.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.930+0000 7fa33610ff40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faebf7330 0x562faec8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.930+0000 7fa33610ff40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 0x562faec9b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.930+0000 7fa33610ff40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faec93850 0x562faec9b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.930+0000 7fa33610ff40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562faec12750 con 0x562faebf7330 2024-04-23T14:35:03.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.930+0000 7fa33610ff40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562faeb6ecc0 con 0x562faec93850 2024-04-23T14:35:03.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.930+0000 7fa33610ff40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x562faec9d7e0 con 0x562faec93470 2024-04-23T14:35:03.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.930+0000 7fa333400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faebf7330 0x562faec8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faebf7330 0x562faec8da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54706/0 (socket says 172.21.15.59:54706) 2024-04-23T14:35:03.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 -- 172.21.15.59:0/2813530375 learned_addr learned my addr 172.21.15.59:0/2813530375 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 -- 172.21.15.59:0/2813530375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 msgr2=0x562faec9b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 --2- 172.21.15.59:0/2813530375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 0x562faec9b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 -- 172.21.15.59:0/2813530375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faec93850 msgr2=0x562faec9b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 --2- 172.21.15.59:0/2813530375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faec93850 0x562faec9b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 -- 172.21.15.59:0/2813530375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562faebf39d0 con 0x562faebf7330 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.931+0000 7fa333400640 1 --2- 172.21.15.59:0/2813530375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faebf7330 0x562faec8da00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa32400a4f0 tx=0x7fa32404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55ce353b5311da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa332000640 1 -- 172.21.15.59:0/2813530375 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324065070 con 0x562faebf7330 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa332000640 1 -- 172.21.15.59:0/2813530375 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324054a10 con 0x562faebf7330 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa332000640 1 -- 172.21.15.59:0/2813530375 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324059040 con 0x562faebf7330 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 -- 172.21.15.59:0/2813530375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faebf7330 msgr2=0x562faec8da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 --2- 172.21.15.59:0/2813530375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faebf7330 0x562faec8da00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa32400a4f0 tx=0x7fa32404e000 comp rx=0 tx=0).stop 2024-04-23T14:35:03.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 -- 172.21.15.59:0/2813530375 shutdown_connections 2024-04-23T14:35:03.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 --2- 172.21.15.59:0/2813530375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 0x562faec9b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 --2- 172.21.15.59:0/2813530375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faec93850 0x562faec9b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 --2- 172.21.15.59:0/2813530375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faebf7330 0x562faec8da00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 -- 172.21.15.59:0/2813530375 >> 172.21.15.59:0/2813530375 conn(0x562faec13290 msgr2=0x562faebf3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 -- 172.21.15.59:0/2813530375 shutdown_connections 2024-04-23T14:35:03.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.932+0000 7fa33610ff40 1 -- 172.21.15.59:0/2813530375 wait complete. 2024-04-23T14:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.933+0000 7fa33610ff40 1 Processor -- start 2024-04-23T14:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.933+0000 7fa33610ff40 1 -- start start 2024-04-23T14:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.934+0000 7fa33610ff40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faebf7330 0x562faec9a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.934+0000 7fa33610ff40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 0x562faec89920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.934+0000 7fa33610ff40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faec93850 0x562faed35210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.934+0000 7fa33610ff40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562faec9a970 con 0x562faec93850 2024-04-23T14:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.934+0000 7fa33610ff40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562faebf3c70 con 0x562faebf7330 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.934+0000 7fa33610ff40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x562faec9ac50 con 0x562faec93470 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faec93850 0x562faed35210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faec93850 0x562faed35210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54716/0 (socket says 172.21.15.59:54716) 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 -- 172.21.15.59:0/3044492946 learned_addr learned my addr 172.21.15.59:0/3044492946 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa32be00640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 0x562faec89920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 -- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 msgr2=0x562faec89920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333400640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faebf7330 0x562faec9a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562faec93470 0x562faec89920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 -- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faebf7330 msgr2=0x562faec9a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562faebf7330 0x562faec9a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.935+0000 7fa333e00640 1 -- 172.21.15.59:0/3044492946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562faebc6cc0 con 0x562faec93850 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.936+0000 7fa333e00640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faec93850 0x562faed35210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0020c0 tx=0x7fa32c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.936+0000 7fa330c00640 1 -- 172.21.15.59:0/3044492946 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c016070 con 0x562faec93850 2024-04-23T14:35:03.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.936+0000 7fa330c00640 1 -- 172.21.15.59:0/3044492946 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa32c0062f0 con 0x562faec93850 2024-04-23T14:35:03.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.936+0000 7fa330c00640 1 -- 172.21.15.59:0/3044492946 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c004760 con 0x562faec93850 2024-04-23T14:35:03.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.936+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562faec9a750 con 0x562faec93850 2024-04-23T14:35:03.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.936+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562faebc6f40 con 0x562faec93850 2024-04-23T14:35:03.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.938+0000 7fa330c00640 1 -- 172.21.15.59:0/3044492946 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa32c019070 con 0x562faec93850 2024-04-23T14:35:03.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.939+0000 7fa330c00640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa31006b2d0 0x7fa31006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.939+0000 7fa330c00640 1 -- 172.21.15.59:0/3044492946 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa32c089a80 con 0x562faec93850 2024-04-23T14:35:03.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.939+0000 7fa333400640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa31006b2d0 0x7fa31006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.939+0000 7fa33610ff40 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x562faebc7f30 0x562faec98230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:03.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.939+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58) -- 0x562faec98770 con 0x562faebc7f30 2024-04-23T14:35:03.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.940+0000 7fa333400640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa31006b2d0 0x7fa31006d790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa324057040 tx=0x7fa324051000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.940+0000 7fa32be00640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x562faebc7f30 0x562faec98230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:03.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.941+0000 7fa32be00640 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x562faebc7f30 0x562faec98230 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa32be00640 1 -- 172.21.15.59:0/3044492946 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fa31c00b040 con 0x562faebc7f30 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x562faebc7f30 msgr2=0x562faec98230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x562faebc7f30 0x562faec98230 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa31006b2d0 msgr2=0x7fa31006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa31006b2d0 0x7fa31006d790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa324057040 tx=0x7fa324051000 comp rx=0 tx=0).stop 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faec93850 msgr2=0x562faed35210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 --2- 172.21.15.59:0/3044492946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562faec93850 0x562faed35210 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0020c0 tx=0x7fa32c00b4b0 comp rx=0 tx=0).stop 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa333e00640 1 -- 172.21.15.59:0/3044492946 reap_dead start 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 shutdown_connections 2024-04-23T14:35:03.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 >> 172.21.15.59:0/3044492946 conn(0x562faec13290 msgr2=0x562faec9af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:03.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.942+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 shutdown_connections 2024-04-23T14:35:03.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:03.943+0000 7fa33610ff40 1 -- 172.21.15.59:0/3044492946 wait complete. 2024-04-23T14:35:03.948 INFO:teuthology.orchestra.run.smithi059.stdout:Xfiledir_test 2024-04-23T14:35:03.948 DEBUG:teuthology.orchestra.run.smithi059:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-23T14:35:04.039 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-23T14:35:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "ancestors": [ 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1099511627776, 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "file", 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 2 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "dirino": 1, 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "dir_test", 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 9, 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "old_pools": [] 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:35:04.045 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-23T14:35:04.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.166+0000 7fbf9b966f40 1 Processor -- start 2024-04-23T14:35:04.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.166+0000 7fbf9b966f40 1 -- start start 2024-04-23T14:35:04.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.167+0000 7fbf9b966f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76675d0 0x55eac76e8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.167+0000 7fbf9b966f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76f9e20 0x55eac76fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.167+0000 7fbf9b966f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 0x55eac76fd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.167+0000 7fbf9b966f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eac764d450 con 0x55eac76675d0 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.167+0000 7fbf9b966f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eac764d6f0 con 0x55eac76f9e20 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.167+0000 7fbf9b966f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55eac764d990 con 0x55eac76fa740 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76675d0 0x55eac76e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76675d0 0x55eac76e8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54730/0 (socket says 172.21.15.59:54730) 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 -- 172.21.15.59:0/3025556256 learned_addr learned my addr 172.21.15.59:0/3025556256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf99600640 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 0x55eac76fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98200640 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76f9e20 0x55eac76fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 -- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 msgr2=0x55eac76fd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 0x55eac76fd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 -- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76f9e20 msgr2=0x55eac76fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76f9e20 0x55eac76fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98c00640 1 -- 172.21.15.59:0/3025556256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac7549b30 con 0x55eac76675d0 2024-04-23T14:35:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.168+0000 7fbf98200640 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76f9e20 0x55eac76fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.169+0000 7fbf98c00640 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76675d0 0x55eac76e8970 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800a4b0 tx=0x7fbf88050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517e3dafa80ffd6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.169+0000 7fbf96e00640 1 -- 172.21.15.59:0/3025556256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8806a070 con 0x55eac76675d0 2024-04-23T14:35:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.169+0000 7fbf96e00640 1 -- 172.21.15.59:0/3025556256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8805db40 con 0x55eac76675d0 2024-04-23T14:35:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.169+0000 7fbf96e00640 1 -- 172.21.15.59:0/3025556256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf880653f0 con 0x55eac76675d0 2024-04-23T14:35:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 -- 172.21.15.59:0/3025556256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76675d0 msgr2=0x55eac76e8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76675d0 0x55eac76e8970 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800a4b0 tx=0x7fbf88050000 comp rx=0 tx=0).stop 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 -- 172.21.15.59:0/3025556256 shutdown_connections 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 0x55eac76fd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76f9e20 0x55eac76fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/3025556256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76675d0 0x55eac76e8970 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 -- 172.21.15.59:0/3025556256 >> 172.21.15.59:0/3025556256 conn(0x55eac764be20 msgr2=0x55eac764e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 -- 172.21.15.59:0/3025556256 shutdown_connections 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.170+0000 7fbf9b966f40 1 -- 172.21.15.59:0/3025556256 wait complete. 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.171+0000 7fbf9b966f40 1 Processor -- start 2024-04-23T14:35:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.171+0000 7fbf9b966f40 1 -- start start 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf9b966f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76675d0 0x55eac76ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf9b966f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76f9e20 0x55eac761ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf9b966f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 0x55eac761f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf9b966f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eac764b4c0 con 0x55eac76f9e20 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf9b966f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55eac764b1e0 con 0x55eac76675d0 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf9b966f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55eac76ec880 con 0x55eac76fa740 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76f9e20 0x55eac761ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76f9e20 0x55eac761ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54746/0 (socket says 172.21.15.59:54746) 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 -- 172.21.15.59:0/119408523 learned_addr learned my addr 172.21.15.59:0/119408523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf99600640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 0x55eac761f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 -- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 msgr2=0x55eac761f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55eac76fa740 0x55eac761f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 -- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76675d0 msgr2=0x55eac76ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eac76675d0 0x55eac76ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.172+0000 7fbf98200640 1 -- 172.21.15.59:0/119408523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac7667370 con 0x55eac76f9e20 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.173+0000 7fbf98200640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76f9e20 0x55eac761ec80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400be10 tx=0x7fbf8400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.173+0000 7fbf95a00640 1 -- 172.21.15.59:0/119408523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84019070 con 0x55eac76f9e20 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.173+0000 7fbf95a00640 1 -- 172.21.15.59:0/119408523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf84007b50 con 0x55eac76f9e20 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.173+0000 7fbf95a00640 1 -- 172.21.15.59:0/119408523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84014410 con 0x55eac76f9e20 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.173+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55eac76ed860 con 0x55eac76f9e20 2024-04-23T14:35:04.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.173+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac76e5ea0 con 0x55eac76f9e20 2024-04-23T14:35:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.175+0000 7fbf95a00640 1 -- 172.21.15.59:0/119408523 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf8401c070 con 0x55eac76f9e20 2024-04-23T14:35:04.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.176+0000 7fbf95a00640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf7806b300 0x7fbf7806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.176+0000 7fbf95a00640 1 -- 172.21.15.59:0/119408523 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fbf84089880 con 0x55eac76f9e20 2024-04-23T14:35:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.177+0000 7fbf98c00640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf7806b300 0x7fbf7806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.177+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55eac76eea30 0x55eac779d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.177+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x55eac76e6410 con 0x55eac76eea30 2024-04-23T14:35:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.177+0000 7fbf98c00640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf7806b300 0x7fbf7806d7c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbf88067000 tx=0x7fbf88005e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.177+0000 7fbf99600640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55eac76eea30 0x55eac779d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.179+0000 7fbf99600640 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55eac76eea30 0x55eac779d110 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.182+0000 7fbf99600640 1 -- 172.21.15.59:0/119408523 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v58'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbf90007550 con 0x55eac76eea30 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55eac76eea30 msgr2=0x55eac779d110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55eac76eea30 0x55eac779d110 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf7806b300 msgr2=0x7fbf7806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf7806b300 0x7fbf7806d7c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbf88067000 tx=0x7fbf88005e40 comp rx=0 tx=0).stop 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76f9e20 msgr2=0x55eac761ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 --2- 172.21.15.59:0/119408523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eac76f9e20 0x55eac761ec80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400be10 tx=0x7fbf8400bf00 comp rx=0 tx=0).stop 2024-04-23T14:35:04.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf99600640 1 -- 172.21.15.59:0/119408523 reap_dead start 2024-04-23T14:35:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 shutdown_connections 2024-04-23T14:35:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 >> 172.21.15.59:0/119408523 conn(0x55eac764be20 msgr2=0x55eac76f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 shutdown_connections 2024-04-23T14:35:04.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.183+0000 7fbf9b966f40 1 -- 172.21.15.59:0/119408523 wait complete. 2024-04-23T14:35:04.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:04.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-23T14:35:04.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.399+0000 7f62e5000640 1 Processor -- start 2024-04-23T14:35:04.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.399+0000 7f62e5000640 1 -- start start 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62e5000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62e0156c20 0x7f62e00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62e5000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e0160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62e5000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e0157000 con 0x7f62e00af4c0 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62e5000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f62e00afb00 con 0x7f62e0156c20 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62e5000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f62e015cc20 con 0x7f62e006c500 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62df400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62e0156c20 0x7f62e00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62df400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62e0156c20 0x7f62e00aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54758/0 (socket says 172.21.15.59:54758) 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.401+0000 7f62df400640 1 -- 172.21.15.59:0/1951736864 learned_addr learned my addr 172.21.15.59:0/1951736864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.402+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1951736864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e0160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.402+0000 7f62df400640 1 -- 172.21.15.59:0/1951736864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 msgr2=0x7f62e0160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.402+0000 7f62df400640 1 --2- 172.21.15.59:0/1951736864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e0160100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.402+0000 7f62df400640 1 -- 172.21.15.59:0/1951736864 >> v1:172.21.15.76:6789/0 conn(0x7f62e00af4c0 legacy=0x7f62e006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.402+0000 7f62df400640 1 -- 172.21.15.59:0/1951736864 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e015cf00 con 0x7f62e0156c20 2024-04-23T14:35:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.403+0000 7f62df400640 1 --2- 172.21.15.59:0/1951736864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e00aef80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009870 tx=0x7f62d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f312bbe918526 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.403+0000 7f62dea00640 1 -- 172.21.15.59:0/1951736864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0037070 con 0x7f62e0156c20 2024-04-23T14:35:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.403+0000 7f62dea00640 1 -- 172.21.15.59:0/1951736864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d0032440 con 0x7f62e0156c20 2024-04-23T14:35:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.403+0000 7f62dea00640 1 -- 172.21.15.59:0/1951736864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0032810 con 0x7f62e0156c20 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 -- 172.21.15.59:0/1951736864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 msgr2=0x7f62e00aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 --2- 172.21.15.59:0/1951736864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e00aef80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009870 tx=0x7f62d002a210 comp rx=0 tx=0).stop 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 -- 172.21.15.59:0/1951736864 shutdown_connections 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 --2- 172.21.15.59:0/1951736864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e0160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 --2- 172.21.15.59:0/1951736864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e00aef80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 -- 172.21.15.59:0/1951736864 >> 172.21.15.59:0/1951736864 conn(0x7f62e00b2200 msgr2=0x7f62e00b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 -- 172.21.15.59:0/1951736864 shutdown_connections 2024-04-23T14:35:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.404+0000 7f62e5000640 1 -- 172.21.15.59:0/1951736864 wait complete. 2024-04-23T14:35:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.407+0000 7f62e5000640 1 Processor -- start 2024-04-23T14:35:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.407+0000 7f62e5000640 1 -- start start 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62e5000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e00c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62e5000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e00af4c0 0x7f62e014b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62e5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e006ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62e5000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62e015d9e0 con 0x7f62e0156c20 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62e5000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f62e015d4c0 con 0x7f62e00af4c0 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62e5000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f62e0073ba0 con 0x7f62e006c500 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e006ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e006ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54764/0 (socket says 172.21.15.59:54764) 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.408+0000 7f62dfe00640 1 -- 172.21.15.59:0/3068779518 learned_addr learned my addr 172.21.15.59:0/3068779518 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62df400640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e00c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62d7e00640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e00af4c0 0x7f62e014b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62dfe00640 1 -- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 msgr2=0x7f62e00c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62dfe00640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e00c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62dfe00640 1 -- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e00af4c0 msgr2=0x7f62e014b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62dfe00640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e00af4c0 0x7f62e014b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62dfe00640 1 -- 172.21.15.59:0/3068779518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e0157590 con 0x7f62e0156c20 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62d7e00640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e00af4c0 0x7f62e014b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62df400640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e00c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62dfe00640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e006ec90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f62d800cd40 tx=0x7f62d800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62d7400640 1 -- 172.21.15.59:0/3068779518 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8003b90 con 0x7f62e0156c20 2024-04-23T14:35:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62d7400640 1 -- 172.21.15.59:0/3068779518 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8003d30 con 0x7f62e0156c20 2024-04-23T14:35:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.409+0000 7f62d7400640 1 -- 172.21.15.59:0/3068779518 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8004240 con 0x7f62e0156c20 2024-04-23T14:35:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.410+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e0157870 con 0x7f62e0156c20 2024-04-23T14:35:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.410+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e0157e30 con 0x7f62e0156c20 2024-04-23T14:35:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.412+0000 7f62d7400640 1 -- 172.21.15.59:0/3068779518 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62d80043e0 con 0x7f62e0156c20 2024-04-23T14:35:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.413+0000 7f62d7400640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62bc06c8f0 0x7f62bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.413+0000 7f62d7400640 1 -- 172.21.15.59:0/3068779518 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f62d808c740 con 0x7f62e0156c20 2024-04-23T14:35:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.413+0000 7f62df400640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62bc06c8f0 0x7f62bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.414+0000 7f62e9444740 1 Processor -- start 2024-04-23T14:35:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.414+0000 7f62e9444740 1 -- start start 2024-04-23T14:35:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.414+0000 7f62df400640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62bc06c8f0 0x7f62bc06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f62d0034000 tx=0x7f62d002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62e9444740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc2379f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc23a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23a9fc0 0x55dcc237edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62e9444740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcc22e2f50 con 0x55dcc23c75e0 2024-04-23T14:35:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dcc22892c0 con 0x55dcc23a9fc0 2024-04-23T14:35:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55dcc22def10 con 0x55dcc23aa700 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc2379f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62df400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc2379f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54768/0 (socket says 172.21.15.59:54768) 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.415+0000 7f62df400640 1 -- 172.21.15.59:0/3614813895 learned_addr learned my addr 172.21.15.59:0/3614813895 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62d7e00640 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc23a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62df400640 1 -- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 msgr2=0x55dcc23a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62df400640 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc23a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62df400640 1 -- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23a9fc0 msgr2=0x55dcc237edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62dfe00640 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23a9fc0 0x55dcc237edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62df400640 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23a9fc0 0x55dcc237edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62df400640 1 -- 172.21.15.59:0/3614813895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc20c17b0 con 0x55dcc23c75e0 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62dfe00640 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23a9fc0 0x55dcc237edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.416+0000 7f62df400640 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc2379f70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009960 tx=0x7f62d002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ebdedee13be417 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.417+0000 7f62cfe00640 1 -- 172.21.15.59:0/3614813895 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d004b070 con 0x55dcc23c75e0 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.417+0000 7f62cfe00640 1 -- 172.21.15.59:0/3614813895 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d003c460 con 0x55dcc23c75e0 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.417+0000 7f62cfe00640 1 -- 172.21.15.59:0/3614813895 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0037040 con 0x55dcc23c75e0 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.417+0000 7f62e9444740 1 -- 172.21.15.59:0/3614813895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 msgr2=0x55dcc2379f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.417+0000 7f62e9444740 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc2379f70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009960 tx=0x7f62d002c000 comp rx=0 tx=0).stop 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 -- 172.21.15.59:0/3614813895 shutdown_connections 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc23a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23a9fc0 0x55dcc237edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 --2- 172.21.15.59:0/3614813895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc2379f70 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 -- 172.21.15.59:0/3614813895 >> 172.21.15.59:0/3614813895 conn(0x55dcc237c1e0 msgr2=0x55dcc23c4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 -- 172.21.15.59:0/3614813895 shutdown_connections 2024-04-23T14:35:04.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 -- 172.21.15.59:0/3614813895 wait complete. 2024-04-23T14:35:04.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.418+0000 7f62e9444740 1 -- start start 2024-04-23T14:35:04.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.419+0000 7f62e9444740 1 Processor -- start 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62e9444740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc23acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62e9444740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcc237e390 con 0x55dcc23c75e0 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dcc229c680 con 0x55dcc23aa700 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55dcc237d780 con 0x55dcc23c3dc0 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62e9444740 10 client.0 _ll_register_callbacks cb 0x55dcc22d94f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:04.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23bd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54782/0 (socket says 172.21.15.59:54782) 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 -- 172.21.15.59:0/1110937870 learned_addr learned my addr 172.21.15.59:0/1110937870 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62df400640 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc23acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 -- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 msgr2=0x55dcc23acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc23acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 -- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 msgr2=0x55dcc23a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.420+0000 7f62d7e00640 1 -- 172.21.15.59:0/1110937870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc2121f40 con 0x55dcc23c75e0 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.421+0000 7f62d7e00640 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23bd890 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005500 tx=0x7f62c800e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2ac1616f517c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.421+0000 7f62ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.421+0000 7f62ccc00640 1 -- 172.21.15.59:0/1110937870 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8003a60 con 0x55dcc23c75e0 2024-04-23T14:35:04.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.421+0000 7f62ccc00640 1 -- 172.21.15.59:0/1110937870 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f62c8003c00 con 0x55dcc23c75e0 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.421+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d003c460 con 0x55dcc23c75e0 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62ccc00640 1 -- 172.21.15.59:0/1110937870 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62c8020050 con 0x55dcc23c75e0 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 10 client.6357 fetch_fsmap learned FSMap version 33 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55dcc1fb50a0 con 0x55dcc23c75e0 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62ccc00640 1 -- 172.21.15.59:0/1110937870 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d003c460 con 0x55dcc23c75e0 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62ccc00640 1 -- 172.21.15.59:0/1110937870 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f62c8007020 con 0x55dcc23c75e0 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 10 client.6357 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 10 client.6357 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 20 client.6357 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23ac020 0x55dcc23bb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 4 client.6357 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.422+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcc22f0970 con 0x55dcc23ac020 2024-04-23T14:35:04.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.423+0000 7f62df400640 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23ac020 0x55dcc23bb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.423+0000 7f62df400640 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23ac020 0x55dcc23bb740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e478ec9bec550c server_cookie=a706f331567717ae in_seq=0 out_seq=0 2024-04-23T14:35:04.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.423+0000 7f62ccc00640 10 client.6357 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62ccc00640 1 -- 172.21.15.59:0/1110937870 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcc22f0970 con 0x55dcc23ac020 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62ccc00640 10 client.6357 handle_command_reply: tid=0 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 client.6357 shutdown 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 msgr2=0x55dcc23bd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23bd890 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005500 tx=0x7f62c800e590 comp rx=0 tx=0).stop 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 shutdown_connections 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23ac020 0x55dcc23bb740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc23acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23bd890 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 --2- 172.21.15.59:0/1110937870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62ccc00640 0 client.6357 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.424+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 >> 172.21.15.59:0/1110937870 conn(0x55dcc23a9fc0 msgr2=0x55dcc23c7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.425+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 shutdown_connections 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.425+0000 7f62e9444740 1 -- 172.21.15.59:0/1110937870 wait complete. 2024-04-23T14:35:04.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.425+0000 7f62e9444740 20 client.6357 trim_cache size 0 max 16384 2024-04-23T14:35:04.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.446+0000 7f62e9444740 1 Processor -- start 2024-04-23T14:35:04.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.446+0000 7f62e9444740 1 -- start start 2024-04-23T14:35:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.447+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.447+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc237acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.447+0000 7f62e9444740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc237b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.447+0000 7f62e9444740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0200c60 con 0x55dcc23c75e0 2024-04-23T14:35:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.447+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01ffcc0 con 0x55dcc23aa700 2024-04-23T14:35:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.447+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f62e0154450 con 0x55dcc23c3dc0 2024-04-23T14:35:04.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc237b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc237b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54798/0 (socket says 172.21.15.59:54798) 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 -- 172.21.15.59:0/1382895108 learned_addr learned my addr 172.21.15.59:0/1382895108 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62d7e00640 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc237acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62df400640 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 -- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 msgr2=0x55dcc237acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc237acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 -- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 msgr2=0x55dcc23de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23de7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 -- 172.21.15.59:0/1382895108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc23e8d00 con 0x55dcc23c75e0 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62df400640 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.448+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc237b1f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f62d8062a80 tx=0x7f62d80566f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319d64bc620c4a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.449+0000 7f62cf400640 1 -- 172.21.15.59:0/1382895108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8088020 con 0x55dcc23c75e0 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.449+0000 7f62cf400640 1 -- 172.21.15.59:0/1382895108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8062fb0 con 0x55dcc23c75e0 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.449+0000 7f62cf400640 1 -- 172.21.15.59:0/1382895108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d80046d0 con 0x55dcc23c75e0 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.449+0000 7f62e9444740 1 -- 172.21.15.59:0/1382895108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 msgr2=0x55dcc237b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.449+0000 7f62e9444740 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc237b1f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f62d8062a80 tx=0x7f62d80566f0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 -- 172.21.15.59:0/1382895108 shutdown_connections 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23c3dc0 0x55dcc237acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc237b1f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 --2- 172.21.15.59:0/1382895108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23aa700 0x55dcc23de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 -- 172.21.15.59:0/1382895108 >> 172.21.15.59:0/1382895108 conn(0x55dcc23a9fc0 msgr2=0x55dcc23c5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 -- 172.21.15.59:0/1382895108 shutdown_connections 2024-04-23T14:35:04.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 -- 172.21.15.59:0/1382895108 wait complete. 2024-04-23T14:35:04.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.450+0000 7f62e9444740 1 -- start start 2024-04-23T14:35:04.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.451+0000 7f62e9444740 1 Processor -- start 2024-04-23T14:35:04.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.451+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc243ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62e9444740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23c3dc0 0x55dcc23dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62e9444740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62e9444740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcc23a74e0 con 0x55dcc23c75e0 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0154aa0 con 0x55dcc23c3dc0 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62e9444740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55dcc237bc90 con 0x55dcc23aa700 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62e9444740 10 client.0 _ll_register_callbacks cb 0x7f62c8007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23dda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54810/0 (socket says 172.21.15.59:54810) 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 -- 172.21.15.59:0/1232023668 learned_addr learned my addr 172.21.15.59:0/1232023668 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62df400640 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc243ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62d7e00640 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23c3dc0 0x55dcc23dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 -- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 msgr2=0x55dcc243ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc243ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 -- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23c3dc0 msgr2=0x55dcc23dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23c3dc0 0x55dcc23dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.452+0000 7f62dfe00640 1 -- 172.21.15.59:0/1232023668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc237b3f0 con 0x55dcc23c75e0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62dfe00640 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23dda10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f62d8056830 tx=0x7f62d800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0e293c7b35a542 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62ccc00640 1 -- 172.21.15.59:0/1232023668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8088030 con 0x55dcc23c75e0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62ccc00640 1 -- 172.21.15.59:0/1232023668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f62d8062b20 con 0x55dcc23c75e0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d8062fb0 con 0x55dcc23c75e0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62ccc00640 1 -- 172.21.15.59:0/1232023668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62d8081050 con 0x55dcc23c75e0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62e9444740 10 client.6363 fetch_fsmap learned FSMap version 33 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55dcc23755d0 con 0x55dcc23c75e0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.453+0000 7f62ccc00640 1 -- 172.21.15.59:0/1232023668 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8062fb0 con 0x55dcc23c75e0 2024-04-23T14:35:04.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62ccc00640 1 -- 172.21.15.59:0/1232023668 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f62d8084020 con 0x55dcc23c75e0 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62e9444740 10 client.6363 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62e9444740 10 client.6363 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62e9444740 20 client.6363 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62e9444740 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23de670 0x55dcc23a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62e9444740 4 client.6363 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55dcc22ea7e0 con 0x55dcc23de670 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62df400640 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23de670 0x55dcc23a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62df400640 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23de670 0x55dcc23a3960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf23cfd313b61ee4 server_cookie=3963a7d095aef1ef in_seq=0 out_seq=0 2024-04-23T14:35:04.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.454+0000 7f62ccc00640 10 client.6363 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.455+0000 7f62ccc00640 1 -- 172.21.15.59:0/1232023668 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcc22ea7e0 con 0x55dcc23de670 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.455+0000 7f62ccc00640 10 client.6363 handle_command_reply: tid=0 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 client.6363 shutdown 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 msgr2=0x55dcc23dda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23dda10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f62d8056830 tx=0x7f62d800cf70 comp rx=0 tx=0).stop 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 shutdown_connections 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55dcc23de670 0x55dcc23a3960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcc23c75e0 0x55dcc23dda10 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dcc23c3dc0 0x55dcc23dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 --2- 172.21.15.59:0/1232023668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55dcc23aa700 0x55dcc243ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 >> 172.21.15.59:0/1232023668 conn(0x55dcc23a9fc0 msgr2=0x55dcc23796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62ccc00640 0 client.6363 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 shutdown_connections 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 1 -- 172.21.15.59:0/1232023668 wait complete. 2024-04-23T14:35:04.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.456+0000 7f62e9444740 20 client.6363 trim_cache size 0 max 16384 2024-04-23T14:35:04.461 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:04.461 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:35:04.461 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "9ff57d84-a83a-4561-be95-ad8b5ef2d769", 2024-04-23T14:35:04.461 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:35:04.461 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:04.461 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f3114800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6318=0-4194304@1} caps={6318=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6318 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cd9aa38b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:35:04.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62bc06c8f0 msgr2=0x7f62bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62bc06c8f0 0x7f62bc06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f62d0034000 tx=0x7f62d002e000 comp rx=0 tx=0).stop 2024-04-23T14:35:04.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 msgr2=0x7f62e006ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e006ec90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f62d800cd40 tx=0x7f62d800a660 comp rx=0 tx=0).stop 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 shutdown_connections 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62bc06c8f0 0x7f62bc06edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e0156c20 0x7f62e006ec90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e00af4c0 0x7f62e014b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 --2- 172.21.15.59:0/3068779518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62e006c500 0x7f62e00c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 >> 172.21.15.59:0/3068779518 conn(0x7f62e00b2200 msgr2=0x7f62e015bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 shutdown_connections 2024-04-23T14:35:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.457+0000 7f62e5000640 1 -- 172.21.15.59:0/3068779518 wait complete. 2024-04-23T14:35:04.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:04.476 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-23T14:35:04.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.683+0000 7f0d12e00640 1 Processor -- start 2024-04-23T14:35:04.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.684+0000 7f0d12e00640 1 -- start start 2024-04-23T14:35:04.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.685+0000 7f0d12e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0d0c15ed20 0x7f0d0c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.685+0000 7f0d12e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c0d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.685+0000 7f0d12e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d0c0b3f10 0x7f0d0c0b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.685+0000 7f0d12e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0c15f100 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.685+0000 7f0d12e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0c0b5da0 con 0x7f0d0c15ed20 2024-04-23T14:35:04.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.685+0000 7f0d12e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0c0d0a20 con 0x7f0d0c0b3600 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d0c0b3f10 0x7f0d0c0b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d0c0b3f10 0x7f0d0c0b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54820/0 (socket says 172.21.15.59:54820) 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 -- 172.21.15.59:0/3214079601 learned_addr learned my addr 172.21.15.59:0/3214079601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d11a00640 1 --2- 172.21.15.59:0/3214079601 >> v2:172.21.15.76:3300/0 conn(0x7f0d0c15ed20 0x7f0d0c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d11000640 1 --2- 172.21.15.59:0/3214079601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c0d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 -- 172.21.15.59:0/3214079601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 msgr2=0x7f0d0c0d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 --2- 172.21.15.59:0/3214079601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c0d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 -- 172.21.15.59:0/3214079601 >> v2:172.21.15.76:3300/0 conn(0x7f0d0c15ed20 msgr2=0x7f0d0c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 --2- 172.21.15.59:0/3214079601 >> v2:172.21.15.76:3300/0 conn(0x7f0d0c15ed20 0x7f0d0c0d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.686+0000 7f0d12400640 1 -- 172.21.15.59:0/3214079601 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c0d2ed0 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.687+0000 7f0d12400640 1 --2- 172.21.15.59:0/3214079601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0b43a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c08e2a0 tx=0x7f0d0402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5907de94c8c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d0be00640 1 -- 172.21.15.59:0/3214079601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04037070 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d0be00640 1 -- 172.21.15.59:0/3214079601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0402ae00 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d0be00640 1 -- 172.21.15.59:0/3214079601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04032660 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d12e00640 1 -- 172.21.15.59:0/3214079601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 msgr2=0x7f0d0c0b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d12e00640 1 --2- 172.21.15.59:0/3214079601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0b43a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c08e2a0 tx=0x7f0d0402a0b0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d12e00640 1 -- 172.21.15.59:0/3214079601 shutdown_connections 2024-04-23T14:35:04.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d12e00640 1 --2- 172.21.15.59:0/3214079601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0b43a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d12e00640 1 --2- 172.21.15.59:0/3214079601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c0d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d12e00640 1 --2- 172.21.15.59:0/3214079601 >> v2:172.21.15.76:3300/0 conn(0x7f0d0c15ed20 0x7f0d0c0d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.688+0000 7f0d12e00640 1 -- 172.21.15.59:0/3214079601 >> 172.21.15.59:0/3214079601 conn(0x7f0d0c08bf90 msgr2=0x7f0d0c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.689+0000 7f0d12e00640 1 -- 172.21.15.59:0/3214079601 shutdown_connections 2024-04-23T14:35:04.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.689+0000 7f0d12e00640 1 -- 172.21.15.59:0/3214079601 wait complete. 2024-04-23T14:35:04.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.690+0000 7f0d12e00640 1 Processor -- start 2024-04-23T14:35:04.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.690+0000 7f0d12e00640 1 -- start start 2024-04-23T14:35:04.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.691+0000 7f0d12e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.691+0000 7f0d12e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.691+0000 7f0d12e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d0c15ed20 0x7f0d0c1612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.691+0000 7f0d12e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c0b48e0 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.691+0000 7f0d12e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c0d3490 con 0x7f0d0c15ed20 2024-04-23T14:35:04.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.691+0000 7f0d12e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0c0b4a60 con 0x7f0d0c0b3600 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54826/0 (socket says 172.21.15.59:54826) 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 -- 172.21.15.59:0/921799337 learned_addr learned my addr 172.21.15.59:0/921799337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11a00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 -- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 msgr2=0x7f0d0c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d12400640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d0c15ed20 0x7f0d0c1612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 -- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d0c15ed20 msgr2=0x7f0d0c1612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d0c15ed20 0x7f0d0c1612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 -- 172.21.15.59:0/921799337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c0cbc30 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.692+0000 7f0d11000640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0ca960 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800d800 tx=0x7f0cf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.693+0000 7f0d09600640 1 -- 172.21.15.59:0/921799337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8018020 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.693+0000 7f0d09600640 1 -- 172.21.15.59:0/921799337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8009640 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.693+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c0cbf10 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.693+0000 7f0d09600640 1 -- 172.21.15.59:0/921799337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8009a10 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.693+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c0cc4d0 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.695+0000 7f0d09600640 1 -- 172.21.15.59:0/921799337 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cf801b020 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.696+0000 7f0d09600640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0cec06c8f0 0x7f0cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.696+0000 7f0d09600640 1 -- 172.21.15.59:0/921799337 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0cf808bdc0 con 0x7f0d0c0b3f10 2024-04-23T14:35:04.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.696+0000 7f0d11a00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0cec06c8f0 0x7f0cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.697+0000 7f0d173c0740 1 Processor -- start 2024-04-23T14:35:04.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.697+0000 7f0d173c0740 1 -- start start 2024-04-23T14:35:04.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.697+0000 7f0d11a00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0cec06c8f0 0x7f0cec06edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0cd600 tx=0x7f0d0000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071b68790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b987e0 0x560071b6d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d173c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560071ad21f0 con 0x560071b98f20 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560071a78350 con 0x560071b987e0 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560071ace1b0 con 0x560071bb5e00 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54830/0 (socket says 172.21.15.59:54830) 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d11000640 1 -- 172.21.15.59:0/774551484 learned_addr learned my addr 172.21.15.59:0/774551484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.698+0000 7f0d11a00640 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071b68790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d11000640 1 -- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 msgr2=0x560071b68790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d11000640 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071b68790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d12400640 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b987e0 0x560071b6d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d11000640 1 -- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b987e0 msgr2=0x560071b6d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d11000640 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b987e0 0x560071b6d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d11000640 1 -- 172.21.15.59:0/774551484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560071accf50 con 0x560071b98f20 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d11a00640 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071b68790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0d11000640 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b982a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0cf808a1c0 tx=0x7f0cf805ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47fea7901f96e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0cfe000640 1 -- 172.21.15.59:0/774551484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8057d50 con 0x560071b98f20 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0cfe000640 1 -- 172.21.15.59:0/774551484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8024d80 con 0x560071b98f20 2024-04-23T14:35:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.699+0000 7f0cfe000640 1 -- 172.21.15.59:0/774551484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80600b0 con 0x560071b98f20 2024-04-23T14:35:04.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 -- 172.21.15.59:0/774551484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 msgr2=0x560071b982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b982a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0cf808a1c0 tx=0x7f0cf805ef90 comp rx=0 tx=0).stop 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 -- 172.21.15.59:0/774551484 shutdown_connections 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b987e0 0x560071b6d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b982a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 --2- 172.21.15.59:0/774551484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071b68790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 -- 172.21.15.59:0/774551484 >> 172.21.15.59:0/774551484 conn(0x560071b6aa00 msgr2=0x560071bb32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 -- 172.21.15.59:0/774551484 shutdown_connections 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.700+0000 7f0d173c0740 1 -- 172.21.15.59:0/774551484 wait complete. 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.701+0000 7f0d173c0740 1 -- start start 2024-04-23T14:35:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.701+0000 7f0d173c0740 1 Processor -- start 2024-04-23T14:35:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b98f20 0x560071b939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071bb5e00 0x560071bb5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b94060 0x560071b97820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d173c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560071b6cbb0 con 0x560071b94060 2024-04-23T14:35:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560071a8b690 con 0x560071bb5e00 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560071b6bfa0 con 0x560071b98f20 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d173c0740 10 client.0 _ll_register_callbacks cb 0x560071ac7650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.702+0000 7f0d12400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b94060 0x560071b97820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b94060 0x560071b97820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54840/0 (socket says 172.21.15.59:54840) 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 -- 172.21.15.59:0/4013337280 learned_addr learned my addr 172.21.15.59:0/4013337280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d11a00640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b98f20 0x560071b939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d11000640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071bb5e00 0x560071bb5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 -- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b98f20 msgr2=0x560071b939f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b98f20 0x560071b939f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 -- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071bb5e00 msgr2=0x560071bb5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071bb5e00 0x560071bb5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 -- 172.21.15.59:0/4013337280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560071a8a920 con 0x560071b94060 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d11000640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071bb5e00 0x560071bb5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d11a00640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b98f20 0x560071b939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0d12400640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b94060 0x560071b97820 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009960 tx=0x7f0d04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f67681c3e5df6e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.703+0000 7f0cf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0cf2a00640 1 -- 172.21.15.59:0/4013337280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04041020 con 0x560071b94060 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0cf2a00640 1 -- 172.21.15.59:0/4013337280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0d04032df0 con 0x560071b94060 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf8024d80 con 0x560071b94060 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0cf2a00640 1 -- 172.21.15.59:0/4013337280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d0404b050 con 0x560071b94060 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0d173c0740 10 client.6375 fetch_fsmap learned FSMap version 33 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5600718ae7d0 con 0x560071b94060 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0cf2a00640 1 -- 172.21.15.59:0/4013337280 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8024d80 con 0x560071b94060 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0cf2a00640 1 -- 172.21.15.59:0/4013337280 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f0d0404e020 con 0x560071b94060 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0d173c0740 10 client.6375 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0d173c0740 10 client.6375 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0d173c0740 20 client.6375 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.704+0000 7f0d173c0740 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b9a8f0 0x560071b9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.705+0000 7f0d173c0740 4 client.6375 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.705+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560071adfc10 con 0x560071b9a8f0 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.705+0000 7f0d11a00640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b9a8f0 0x560071b9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.705+0000 7f0d11a00640 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b9a8f0 0x560071b9acf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd2367666a2c0f8 server_cookie=c599a6b6bebf3e5 in_seq=0 out_seq=0 2024-04-23T14:35:04.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.705+0000 7f0cf2a00640 10 client.6375 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.706+0000 7f0cf2a00640 1 -- 172.21.15.59:0/4013337280 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560071adfc10 con 0x560071b9a8f0 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.706+0000 7f0cf2a00640 10 client.6375 handle_command_reply: tid=0 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.706+0000 7f0d173c0740 1 client.6375 shutdown 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b94060 msgr2=0x560071b97820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b94060 0x560071b97820 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009960 tx=0x7f0d04004730 comp rx=0 tx=0).stop 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 shutdown_connections 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b9a8f0 0x560071b9acf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b94060 0x560071b97820 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071bb5e00 0x560071bb5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 --2- 172.21.15.59:0/4013337280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b98f20 0x560071b939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 >> 172.21.15.59:0/4013337280 conn(0x560071b987e0 msgr2=0x560071b6c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0cf2a00640 0 client.6375 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 shutdown_connections 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 1 -- 172.21.15.59:0/4013337280 wait complete. 2024-04-23T14:35:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.707+0000 7f0d173c0740 20 client.6375 trim_cache size 0 max 16384 2024-04-23T14:35:04.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.727+0000 7f0d173c0740 1 Processor -- start 2024-04-23T14:35:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.727+0000 7f0d173c0740 1 -- start start 2024-04-23T14:35:04.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b94060 0x560071baa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b98f20 0x560071bb66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071bb5e00 0x560071b6dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d173c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c200e00 con 0x560071bb5e00 2024-04-23T14:35:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1ffe90 con 0x560071b98f20 2024-04-23T14:35:04.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0c0c96d0 con 0x560071b94060 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d12400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071bb5e00 0x560071b6dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d12400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071bb5e00 0x560071b6dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54850/0 (socket says 172.21.15.59:54850) 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d12400640 1 -- 172.21.15.59:0/1045600568 learned_addr learned my addr 172.21.15.59:0/1045600568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d11000640 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b98f20 0x560071bb66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.728+0000 7f0d12400640 1 -- 172.21.15.59:0/1045600568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b94060 msgr2=0x560071baa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0d12400640 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b94060 0x560071baa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0d12400640 1 -- 172.21.15.59:0/1045600568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b98f20 msgr2=0x560071bb66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0d12400640 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b98f20 0x560071bb66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0d12400640 1 -- 172.21.15.59:0/1045600568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600717e43f0 con 0x560071bb5e00 2024-04-23T14:35:04.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0d12400640 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071bb5e00 0x560071b6dfb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0d040090b0 tx=0x7f0d0403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218a8a8016ec2eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0cfd600640 1 -- 172.21.15.59:0/1045600568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04041070 con 0x560071bb5e00 2024-04-23T14:35:04.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0cfd600640 1 -- 172.21.15.59:0/1045600568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d04004040 con 0x560071bb5e00 2024-04-23T14:35:04.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.729+0000 7f0cfd600640 1 -- 172.21.15.59:0/1045600568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04037040 con 0x560071bb5e00 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 -- 172.21.15.59:0/1045600568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071bb5e00 msgr2=0x560071b6dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071bb5e00 0x560071b6dfb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0d040090b0 tx=0x7f0d0403a000 comp rx=0 tx=0).stop 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 -- 172.21.15.59:0/1045600568 shutdown_connections 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071bb5e00 0x560071b6dfb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b98f20 0x560071bb66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 --2- 172.21.15.59:0/1045600568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071b94060 0x560071baa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 -- 172.21.15.59:0/1045600568 >> 172.21.15.59:0/1045600568 conn(0x560071b987e0 msgr2=0x560071b97590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 -- 172.21.15.59:0/1045600568 shutdown_connections 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.730+0000 7f0d173c0740 1 -- 172.21.15.59:0/1045600568 wait complete. 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.731+0000 7f0d173c0740 1 -- start start 2024-04-23T14:35:04.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.731+0000 7f0d173c0740 1 Processor -- start 2024-04-23T14:35:04.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b94060 0x560071b94800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b94d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d173c0740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071bb3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d173c0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560071b69560 con 0x560071b98f20 2024-04-23T14:35:04.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c0c9d20 con 0x560071b94060 2024-04-23T14:35:04.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d173c0740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560071b9b2d0 con 0x560071bb5e00 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d173c0740 10 client.0 _ll_register_callbacks cb 0x7f0d04035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b94d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d11000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b94d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54856/0 (socket says 172.21.15.59:54856) 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.732+0000 7f0d11000640 1 -- 172.21.15.59:0/3980042185 learned_addr learned my addr 172.21.15.59:0/3980042185 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d11a00640 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b94060 0x560071b94800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d11000640 1 -- 172.21.15.59:0/3980042185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 msgr2=0x560071bb3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d11000640 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071bb3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d11000640 1 -- 172.21.15.59:0/3980042185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b94060 msgr2=0x560071b94800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d11000640 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b94060 0x560071b94800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d11000640 1 -- 172.21.15.59:0/3980042185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600717a40f0 con 0x560071b98f20 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d11000640 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b94d40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0cf805a300 tx=0x7f0cf8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89e23a649f852c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0cf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0cf2a00640 1 -- 172.21.15.59:0/3980042185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8079080 con 0x560071b98f20 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0cf2a00640 1 -- 172.21.15.59:0/3980042185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0cf80560c0 con 0x560071b98f20 2024-04-23T14:35:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.733+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d04004040 con 0x560071b98f20 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0cf2a00640 1 -- 172.21.15.59:0/3980042185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0cf80600b0 con 0x560071b98f20 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 10 client.6381 fetch_fsmap learned FSMap version 33 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x560071a97810 con 0x560071b98f20 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0cf2a00640 1 -- 172.21.15.59:0/3980042185 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d04004040 con 0x560071b98f20 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0cf2a00640 1 -- 172.21.15.59:0/3980042185 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f0cf8013050 con 0x560071b98f20 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 10 client.6381 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 10 client.6381 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 20 client.6381 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b99540 0x560071bb37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 4 client.6381 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560071ad9a80 con 0x560071b99540 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.734+0000 7f0d11a00640 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b99540 0x560071bb37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d11a00640 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b99540 0x560071bb37d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd098223028736b server_cookie=183d62246af76691 in_seq=0 out_seq=0 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0cf2a00640 10 client.6381 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0cf2a00640 1 -- 172.21.15.59:0/3980042185 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x560071ad9a80 con 0x560071b99540 2024-04-23T14:35:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0cf2a00640 10 client.6381 handle_command_reply: tid=0 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 client.6381 shutdown 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 msgr2=0x560071b94d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b94d40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0cf805a300 tx=0x7f0cf8015000 comp rx=0 tx=0).stop 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 shutdown_connections 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560071b99540 0x560071bb37d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560071bb5e00 0x560071bb3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560071b98f20 0x560071b94d40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 --2- 172.21.15.59:0/3980042185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560071b94060 0x560071b94800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0cf2a00640 0 client.6381 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.735+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 >> 172.21.15.59:0/3980042185 conn(0x560071b987e0 msgr2=0x560071b6dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 shutdown_connections 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d173c0740 1 -- 172.21.15.59:0/3980042185 wait complete. 2024-04-23T14:35:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d173c0740 20 client.6381 trim_cache size 0 max 16384 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: "9ff57d84-a83a-4561-be95-ad8b5ef2d769": { 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "9ff57d84-a83a-4561-be95-ad8b5ef2d769", 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive" 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:35:04.741 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0cec06c8f0 msgr2=0x7f0cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0cec06c8f0 0x7f0cec06edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0cd600 tx=0x7f0d0000b040 comp rx=0 tx=0).stop 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 msgr2=0x7f0d0c0ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0ca960 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800d800 tx=0x7f0cf800dcc0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 shutdown_connections 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0cec06c8f0 0x7f0cec06edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d0c15ed20 0x7f0d0c1612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d0c0b3f10 0x7f0d0c0ca960 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 --2- 172.21.15.59:0/921799337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d0c0b3600 0x7f0d0c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.736+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 >> 172.21.15.59:0/921799337 conn(0x7f0d0c08bf90 msgr2=0x7f0d0c0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.737+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 shutdown_connections 2024-04-23T14:35:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:04.737+0000 7f0d12e00640 1 -- 172.21.15.59:0/921799337 wait complete. 2024-04-23T14:35:04.754 INFO:tasks.cephfs.filesystem:scrub status for tag:9ff57d84-a83a-4561-be95-ad8b5ef2d769 - {'path': '/', 'tag': '9ff57d84-a83a-4561-be95-ad8b5ef2d769', 'options': 'recursive'} 2024-04-23T14:35:09.816 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:35:09.815+0000 7f3222800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-23T14:35:19.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:19.477 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:34.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:34.755 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-23T14:35:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.972+0000 7f3213200640 1 Processor -- start 2024-04-23T14:35:34.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.973+0000 7f3213200640 1 -- start start 2024-04-23T14:35:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.974+0000 7f3213200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c0d15b0 0x7f320c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.974+0000 7f3213200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f320c0b5da0 con 0x7f320c0b4f20 2024-04-23T14:35:34.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.974+0000 7f3213200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f320c0d0a20 con 0x7f320c15ed20 2024-04-23T14:35:34.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.974+0000 7f3213200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f320c15f100 con 0x7f320c0d15b0 2024-04-23T14:35:34.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.975+0000 7f3212800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f320c0b4f20 0x7f320c0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60794/0 (socket says 172.21.15.59:60794) 2024-04-23T14:35:34.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.975+0000 7f3212800640 1 -- 172.21.15.59:0/1879012611 learned_addr learned my addr 172.21.15.59:0/1879012611 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.975+0000 7f3211400640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c0d15b0 0x7f320c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.975+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813972333 0 0) 0x7f320c0b5da0 con 0x7f320c0b4f20 2024-04-23T14:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.975+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31f4003520 con 0x7f320c0b4f20 2024-04-23T14:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2334886891 0 0) 0x7f31f4003520 con 0x7f320c0b4f20 2024-04-23T14:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c0d15b0 msgr2=0x7f320c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c0d15b0 0x7f320c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 >> v1:172.21.15.76:6789/0 conn(0x7f320c15ed20 legacy=0x7f320c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c0d2ed0 con 0x7f320c0b4f20 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f32080019f0 con 0x7f320c0b4f20 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 >> v1:172.21.15.59:6789/0 conn(0x7f320c0b4f20 legacy=0x7f320c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31f40039e0 0x7f31f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c0d15b0 0x7f31f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f31f40051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f31f400bba0 con 0x7f31f4004d40 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f31f400bd50 con 0x7f31f40039e0 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f31f400bf50 con 0x7f320c0d15b0 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.976+0000 7f3212800640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f31f40051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.977+0000 7f3211e00640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31f40039e0 0x7f31f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.977+0000 7f3212800640 1 -- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c0d15b0 msgr2=0x7f31f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.977+0000 7f3212800640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c0d15b0 0x7f31f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.977+0000 7f3212800640 1 -- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31f40039e0 msgr2=0x7f31f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.977+0000 7f3212800640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31f40039e0 0x7f31f4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.977+0000 7f3212800640 1 -- 172.21.15.59:0/1879012611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f4003520 con 0x7f31f4004d40 2024-04-23T14:35:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.977+0000 7f3211e00640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31f40039e0 0x7f31f4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.978+0000 7f3212800640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f31f40051a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f320800a780 tx=0x7f31f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1d7f21b8958452 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.978+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800a3d0 con 0x7f31f4004d40 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.978+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320802f070 con 0x7f31f4004d40 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.978+0000 7f3210a00640 1 -- 172.21.15.59:0/1879012611 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320802bca0 con 0x7f31f4004d40 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.979+0000 7f3213200640 1 -- 172.21.15.59:0/1879012611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 msgr2=0x7f31f40051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.979+0000 7f3213200640 1 --2- 172.21.15.59:0/1879012611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f31f40051a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f320800a780 tx=0x7f31f4005830 comp rx=0 tx=0).stop 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.979+0000 7f3212800640 1 -- 172.21.15.59:0/1879012611 reap_dead start 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.979+0000 7f3213200640 1 -- 172.21.15.59:0/1879012611 shutdown_connections 2024-04-23T14:35:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.979+0000 7f3213200640 1 -- 172.21.15.59:0/1879012611 >> 172.21.15.59:0/1879012611 conn(0x7f320c08bf90 msgr2=0x7f320c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.979+0000 7f3213200640 1 -- 172.21.15.59:0/1879012611 shutdown_connections 2024-04-23T14:35:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.979+0000 7f3213200640 1 -- 172.21.15.59:0/1879012611 wait complete. 2024-04-23T14:35:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.981+0000 7f3213200640 1 Processor -- start 2024-04-23T14:35:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.981+0000 7f3213200640 1 -- start start 2024-04-23T14:35:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3213200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f320c1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3213200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c1f4830 0x7f320c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3213200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c15d7d0 0x7f320c15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3213200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f320c0b3600 con 0x7f31f4004d40 2024-04-23T14:35:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3213200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f320c0d3490 con 0x7f320c1f4830 2024-04-23T14:35:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3213200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f320c0b3780 con 0x7f320c15d7d0 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3211e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f320c1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3211e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f320c1519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47884/0 (socket says 172.21.15.59:47884) 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.982+0000 7f3211e00640 1 -- 172.21.15.59:0/2215187567 learned_addr learned my addr 172.21.15.59:0/2215187567 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3211e00640 1 -- 172.21.15.59:0/2215187567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c15d7d0 msgr2=0x7f320c15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3211e00640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c15d7d0 0x7f320c15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3211e00640 1 -- 172.21.15.59:0/2215187567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c1f4830 msgr2=0x7f320c0b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3211e00640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c1f4830 0x7f320c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3211e00640 1 -- 172.21.15.59:0/2215187567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c15b2b0 con 0x7f31f4004d40 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3211e00640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f320c1519a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f320000d7a0 tx=0x7f320000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3206000640 1 -- 172.21.15.59:0/2215187567 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200005070 con 0x7f31f4004d40 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3206000640 1 -- 172.21.15.59:0/2215187567 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3200005210 con 0x7f31f4004d40 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3206000640 1 -- 172.21.15.59:0/2215187567 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200007030 con 0x7f31f4004d40 2024-04-23T14:35:34.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c0c8f30 con 0x7f31f4004d40 2024-04-23T14:35:34.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.983+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c0c94f0 con 0x7f31f4004d40 2024-04-23T14:35:34.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.985+0000 7f3206000640 1 -- 172.21.15.59:0/2215187567 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f320001a020 con 0x7f31f4004d40 2024-04-23T14:35:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.986+0000 7f3206000640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31ec06c8f0 0x7f31ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.986+0000 7f3206000640 1 -- 172.21.15.59:0/2215187567 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f320008b370 con 0x7f31f4004d40 2024-04-23T14:35:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.987+0000 7f3211400640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31ec06c8f0 0x7f31ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.987+0000 7f32175f4740 1 Processor -- start 2024-04-23T14:35:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.987+0000 7f3211400640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31ec06c8f0 0x7f31ec06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f320c0c8b50 tx=0x7f31f8009500 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.988+0000 7f32175f4740 1 -- start start 2024-04-23T14:35:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.988+0000 7f32175f4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x557868044790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.988+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x5578680742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.988+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5578680747e0 0x5578680495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.988+0000 7f32175f4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557867fae1f0 con 0x557868091e00 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.988+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557867f54350 con 0x557868074f20 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.988+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557867faa1b0 con 0x5578680747e0 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x557868044790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x557868044790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47898/0 (socket says 172.21.15.59:47898) 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 -- 172.21.15.59:0/1044299721 learned_addr learned my addr 172.21.15.59:0/1044299721 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3212800640 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5578680747e0 0x5578680495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211400640 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x5578680742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 -- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5578680747e0 msgr2=0x5578680495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5578680747e0 0x5578680495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 -- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 msgr2=0x5578680742a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x5578680742a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 -- 172.21.15.59:0/1044299721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557867fa8f50 con 0x557868091e00 2024-04-23T14:35:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.989+0000 7f3211e00640 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x557868044790 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3200055460 tx=0x7f32000597c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2185fb5d4083e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.990+0000 7f31fe600640 1 -- 172.21.15.59:0/1044299721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200061be0 con 0x557868091e00 2024-04-23T14:35:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.990+0000 7f31fe600640 1 -- 172.21.15.59:0/1044299721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3200057300 con 0x557868091e00 2024-04-23T14:35:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.990+0000 7f31fe600640 1 -- 172.21.15.59:0/1044299721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200055670 con 0x557868091e00 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.990+0000 7f32175f4740 1 -- 172.21.15.59:0/1044299721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 msgr2=0x557868044790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.990+0000 7f32175f4740 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x557868044790 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3200055460 tx=0x7f32000597c0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 -- 172.21.15.59:0/1044299721 shutdown_connections 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5578680747e0 0x5578680495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x5578680742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 --2- 172.21.15.59:0/1044299721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x557868044790 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 -- 172.21.15.59:0/1044299721 >> 172.21.15.59:0/1044299721 conn(0x557868046a00 msgr2=0x55786808f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 -- 172.21.15.59:0/1044299721 shutdown_connections 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 -- 172.21.15.59:0/1044299721 wait complete. 2024-04-23T14:35:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.991+0000 7f32175f4740 1 -- start start 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.992+0000 7f32175f4740 1 Processor -- start 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557868074f20 0x557868077390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868091e00 0x55786806fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f32175f4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55786808e630 0x55786808f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f32175f4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557868048bb0 con 0x55786808e630 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557867f67690 con 0x557868091e00 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557868047fa0 con 0x557868074f20 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f32175f4740 10 client.0 _ll_register_callbacks cb 0x557867fa3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55786808e630 0x55786808f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55786808e630 0x55786808f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47902/0 (socket says 172.21.15.59:47902) 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 -- 172.21.15.59:0/4053530736 learned_addr learned my addr 172.21.15.59:0/4053530736 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3211e00640 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557868074f20 0x557868077390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3211400640 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868091e00 0x55786806fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 -- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557868074f20 msgr2=0x557868077390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557868074f20 0x557868077390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 -- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868091e00 msgr2=0x55786806fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868091e00 0x55786806fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.993+0000 7f3212800640 1 -- 172.21.15.59:0/4053530736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557867f66920 con 0x55786808e630 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f3212800640 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55786808e630 0x55786808f2a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f320800a870 tx=0x7f3208001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc45a92cd1b86071 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f31f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f31f2a00640 1 -- 172.21.15.59:0/4053530736 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320802bbb0 con 0x55786808e630 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f31f2a00640 1 -- 172.21.15.59:0/4053530736 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f320802bd50 con 0x55786808e630 2024-04-23T14:35:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3200057300 con 0x55786808e630 2024-04-23T14:35:34.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f31f2a00640 1 -- 172.21.15.59:0/4053530736 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3208047050 con 0x55786808e630 2024-04-23T14:35:34.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f32175f4740 10 client.6393 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.994+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x557867d8a7d0 con 0x55786808e630 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f31f2a00640 1 -- 172.21.15.59:0/4053530736 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3200057300 con 0x55786808e630 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f31f2a00640 1 -- 172.21.15.59:0/4053530736 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f320804a020 con 0x55786808e630 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f32175f4740 10 client.6393 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f32175f4740 10 client.6393 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f32175f4740 20 client.6393 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f32175f4740 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868086800 0x557868086c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f32175f4740 4 client.6393 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557867fbbc10 con 0x557868086800 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.995+0000 7f3211e00640 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868086800 0x557868086c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.996+0000 7f3211e00640 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868086800 0x557868086c10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0f27c27bbbd04c server_cookie=64bcdf73c4f26176 in_seq=0 out_seq=0 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.996+0000 7f31f2a00640 10 client.6393 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.996+0000 7f31f2a00640 1 -- 172.21.15.59:0/4053530736 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557867fbbc10 con 0x557868086800 2024-04-23T14:35:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.996+0000 7f31f2a00640 10 client.6393 handle_command_reply: tid=0 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 client.6393 shutdown 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55786808e630 msgr2=0x55786808f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55786808e630 0x55786808f2a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f320800a870 tx=0x7f3208001cf0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 shutdown_connections 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868086800 0x557868086c10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55786808e630 0x55786808f2a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868091e00 0x55786806fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 --2- 172.21.15.59:0/4053530736 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557868074f20 0x557868077390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 >> 172.21.15.59:0/4053530736 conn(0x5578680747e0 msgr2=0x5578680912e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f31f2a00640 0 client.6393 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 shutdown_connections 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 1 -- 172.21.15.59:0/4053530736 wait complete. 2024-04-23T14:35:35.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:34.997+0000 7f32175f4740 20 client.6393 trim_cache size 0 max 16384 2024-04-23T14:35:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.017+0000 7f32175f4740 1 Processor -- start 2024-04-23T14:35:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.017+0000 7f32175f4740 1 -- start start 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x55786809ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 0x5578680475a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f32175f4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f32175f4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f320c201330 con 0x557868091e00 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f320c2003e0 con 0x557868074f20 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f320c0ca8e0 con 0x55786808e630 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f3212800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f3212800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680903d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47910/0 (socket says 172.21.15.59:47910) 2024-04-23T14:35:35.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f3212800640 1 -- 172.21.15.59:0/1150909950 learned_addr learned my addr 172.21.15.59:0/1150909950 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.018+0000 7f3211400640 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 0x5578680475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f3211e00640 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x55786809ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f3212800640 1 -- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 msgr2=0x5578680475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f3212800640 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 0x5578680475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f3212800640 1 -- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 msgr2=0x55786809ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f3212800640 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x55786809ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f3212800640 1 -- 172.21.15.59:0/1150909950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557867cc03f0 con 0x557868091e00 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f3212800640 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680903d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3208001320 tx=0x7f320802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7cbd5664f75fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f31fdc00640 1 -- 172.21.15.59:0/1150909950 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208050070 con 0x557868091e00 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f31fdc00640 1 -- 172.21.15.59:0/1150909950 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320802f070 con 0x557868091e00 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.019+0000 7f31fdc00640 1 -- 172.21.15.59:0/1150909950 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208004d60 con 0x557868091e00 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 -- 172.21.15.59:0/1150909950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 msgr2=0x5578680903d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680903d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3208001320 tx=0x7f320802f000 comp rx=0 tx=0).stop 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 -- 172.21.15.59:0/1150909950 shutdown_connections 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 0x5578680475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680903d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 --2- 172.21.15.59:0/1150909950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x55786809ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 -- 172.21.15.59:0/1150909950 >> 172.21.15.59:0/1150909950 conn(0x5578680747e0 msgr2=0x55786808ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 -- 172.21.15.59:0/1150909950 shutdown_connections 2024-04-23T14:35:35.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.020+0000 7f32175f4740 1 -- 172.21.15.59:0/1150909950 wait complete. 2024-04-23T14:35:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.021+0000 7f32175f4740 1 -- start start 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.021+0000 7f32175f4740 1 Processor -- start 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x55786810bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f32175f4740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 0x557868087b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f32175f4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f32175f4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55786804b180 con 0x557868091e00 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f320c0caf30 con 0x557868074f20 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f32175f4740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5578680766a0 con 0x55786808e630 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f32175f4740 10 client.0 _ll_register_callbacks cb 0x7f31f40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f3212800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f3212800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680880a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47918/0 (socket says 172.21.15.59:47918) 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.022+0000 7f3212800640 1 -- 172.21.15.59:0/115078604 learned_addr learned my addr 172.21.15.59:0/115078604 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f3212800640 1 -- 172.21.15.59:0/115078604 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 msgr2=0x557868087b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f3212800640 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 0x557868087b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f3212800640 1 -- 172.21.15.59:0/115078604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 msgr2=0x55786810bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f3212800640 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x55786810bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f3212800640 1 -- 172.21.15.59:0/115078604 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557867c800f0 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f3212800640 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680880a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f320800a780 tx=0x7f320802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1663481f469b5039 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f31f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f31f2a00640 1 -- 172.21.15.59:0/115078604 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320805b020 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f31f2a00640 1 -- 172.21.15.59:0/115078604 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3208034680 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.023+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f320802f070 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f31f2a00640 1 -- 172.21.15.59:0/115078604 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3208065050 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 10 client.6399 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x557867f73810 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f31f2a00640 1 -- 172.21.15.59:0/115078604 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320802f070 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f31f2a00640 1 -- 172.21.15.59:0/115078604 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f3208068020 con 0x557868091e00 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 10 client.6399 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 10 client.6399 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 20 client.6399 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868084c30 0x557868085050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 4 client.6399 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:35:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55786804a8a0 con 0x557868084c30 2024-04-23T14:35:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.024+0000 7f3211e00640 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868084c30 0x557868085050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f3211e00640 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868084c30 0x557868085050 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d065c309503ba380 server_cookie=93b4e8aa455ce3d7 in_seq=0 out_seq=0 2024-04-23T14:35:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f31f2a00640 10 client.6399 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:35.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f31f2a00640 1 -- 172.21.15.59:0/115078604 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55786804a8a0 con 0x557868084c30 2024-04-23T14:35:35.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f31f2a00640 10 client.6399 handle_command_reply: tid=0 2024-04-23T14:35:35.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 client.6399 shutdown 2024-04-23T14:35:35.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 msgr2=0x5578680880a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680880a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f320800a780 tx=0x7f320802e000 comp rx=0 tx=0).stop 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 shutdown_connections 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x557868084c30 0x557868085050 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55786808e630 0x557868087b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557868091e00 0x5578680880a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 --2- 172.21.15.59:0/115078604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557868074f20 0x55786810bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f31f2a00640 0 client.6399 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.025+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 >> 172.21.15.59:0/115078604 conn(0x5578680747e0 msgr2=0x55786808dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.026+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 shutdown_connections 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.026+0000 7f32175f4740 1 -- 172.21.15.59:0/115078604 wait complete. 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.026+0000 7f32175f4740 20 client.6399 trim_cache size 0 max 16384 2024-04-23T14:35:35.029 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:35.030 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:35:35.030 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:35:35.030 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.026+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31ec06c8f0 msgr2=0x7f31ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.026+0000 7f3213200640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31ec06c8f0 0x7f31ec06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f320c0c8b50 tx=0x7f31f8009500 comp rx=0 tx=0).stop 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.026+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 msgr2=0x7f320c1519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.026+0000 7f3213200640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f320c1519a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f320000d7a0 tx=0x7f320000dc60 comp rx=0 tx=0).stop 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 shutdown_connections 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31ec06c8f0 0x7f31ec06edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f320c15d7d0 0x7f320c15dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f320c1f4830 0x7f320c0b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 --2- 172.21.15.59:0/2215187567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f31f4004d40 0x7f320c1519a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 >> 172.21.15.59:0/2215187567 conn(0x7f320c08bf90 msgr2=0x7f320c06de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 shutdown_connections 2024-04-23T14:35:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.027+0000 7f3213200640 1 -- 172.21.15.59:0/2215187567 wait complete. 2024-04-23T14:35:35.046 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:35:35.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:35:35.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.257+0000 7fbb0ae00640 1 Processor -- start 2024-04-23T14:35:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.258+0000 7fbb0ae00640 1 -- start start 2024-04-23T14:35:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.259+0000 7fbb0ae00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb0415e7c0 0x7fbb040d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.259+0000 7fbb0ae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbb040d0d90 0x7fbb040d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.259+0000 7fbb0ae00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb040cf030 con 0x7fbb040d2630 2024-04-23T14:35:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.259+0000 7fbb0ae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbb040b7000 con 0x7fbb040d0d90 2024-04-23T14:35:35.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.259+0000 7fbb0ae00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbb040cf620 con 0x7fbb0415e7c0 2024-04-23T14:35:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb09000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbb040d2630 0x7fbb040b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60798/0 (socket says 172.21.15.59:60798) 2024-04-23T14:35:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb09000640 1 -- 172.21.15.59:0/1335070079 learned_addr learned my addr 172.21.15.59:0/1335070079 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:35:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb0a400640 1 --2- 172.21.15.59:0/1335070079 >> v2:172.21.15.59:3300/0 conn(0x7fbb040d0d90 0x7fbb040d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb09a00640 1 --2- 172.21.15.59:0/1335070079 >> v2:172.21.15.76:3300/0 conn(0x7fbb0415e7c0 0x7fbb040d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb0a400640 1 -- 172.21.15.59:0/1335070079 >> v2:172.21.15.76:3300/0 conn(0x7fbb0415e7c0 msgr2=0x7fbb040d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb0a400640 1 --2- 172.21.15.59:0/1335070079 >> v2:172.21.15.76:3300/0 conn(0x7fbb0415e7c0 0x7fbb040d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb0a400640 1 -- 172.21.15.59:0/1335070079 >> v1:172.21.15.59:6789/0 conn(0x7fbb040d2630 legacy=0x7fbb040b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb0a400640 1 -- 172.21.15.59:0/1335070079 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb040b6a40 con 0x7fbb040d0d90 2024-04-23T14:35:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.260+0000 7fbb03e00640 1 -- 172.21.15.59:0/1335070079 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521038724 0 0) 0x7fbaf0001110 con 0x7fbb040d2630 2024-04-23T14:35:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.261+0000 7fbb0a400640 1 --2- 172.21.15.59:0/1335070079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb040d0d90 0x7fbb040d11c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb0408c6d0 tx=0x7fbafc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c9e19738cc79d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb03e00640 1 -- 172.21.15.59:0/1335070079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007740 con 0x7fbb040d0d90 2024-04-23T14:35:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb03e00640 1 -- 172.21.15.59:0/1335070079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc0078e0 con 0x7fbb040d0d90 2024-04-23T14:35:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb03e00640 1 -- 172.21.15.59:0/1335070079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007cb0 con 0x7fbb040d0d90 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb0ae00640 1 -- 172.21.15.59:0/1335070079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb040d0d90 msgr2=0x7fbb040d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/1335070079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb040d0d90 0x7fbb040d11c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb0408c6d0 tx=0x7fbafc02a680 comp rx=0 tx=0).stop 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb0ae00640 1 -- 172.21.15.59:0/1335070079 shutdown_connections 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/1335070079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb040d0d90 0x7fbb040d11c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/1335070079 >> v2:172.21.15.76:3300/0 conn(0x7fbb0415e7c0 0x7fbb040d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.262+0000 7fbb0ae00640 1 -- 172.21.15.59:0/1335070079 >> 172.21.15.59:0/1335070079 conn(0x7fbb0409d920 msgr2=0x7fbb0408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.263+0000 7fbb0ae00640 1 -- 172.21.15.59:0/1335070079 shutdown_connections 2024-04-23T14:35:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.263+0000 7fbb0ae00640 1 -- 172.21.15.59:0/1335070079 wait complete. 2024-04-23T14:35:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.264+0000 7fbb0ae00640 1 Processor -- start 2024-04-23T14:35:35.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.264+0000 7fbb0ae00640 1 -- start start 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.265+0000 7fbb0ae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb040d0d90 0x7fbb040b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.265+0000 7fbb0ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb040d2630 0x7fbb0415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.265+0000 7fbb0ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415e7c0 0x7fbb0415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.265+0000 7fbb0ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040cf2b0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.265+0000 7fbb0ae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040d1700 con 0x7fbb040d0d90 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.265+0000 7fbb0ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040b6700 con 0x7fbb040d2630 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415e7c0 0x7fbb0415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415e7c0 0x7fbb0415d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47928/0 (socket says 172.21.15.59:47928) 2024-04-23T14:35:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 -- 172.21.15.59:0/3997892207 learned_addr learned my addr 172.21.15.59:0/3997892207 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb09a00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb040d0d90 0x7fbb040b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb09000640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb040d2630 0x7fbb0415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 -- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb040d2630 msgr2=0x7fbb0415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb040d2630 0x7fbb0415d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 -- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb040d0d90 msgr2=0x7fbb040b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb040d0d90 0x7fbb040b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.266+0000 7fbb0a400640 1 -- 172.21.15.59:0/3997892207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb040c9600 con 0x7fbb0415e7c0 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.267+0000 7fbb0a400640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415e7c0 0x7fbb0415d6d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009e80 tx=0x7fbafc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.267+0000 7fbb01600640 1 -- 172.21.15.59:0/3997892207 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc03f070 con 0x7fbb0415e7c0 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.267+0000 7fbb01600640 1 -- 172.21.15.59:0/3997892207 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc031bc0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.267+0000 7fbb01600640 1 -- 172.21.15.59:0/3997892207 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc009450 con 0x7fbb0415e7c0 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.267+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb040c98e0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.267+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb040c9ea0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.269+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0404fd90 con 0x7fbb0415e7c0 2024-04-23T14:35:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.270+0000 7fbb01600640 1 -- 172.21.15.59:0/3997892207 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbafc003990 con 0x7fbb0415e7c0 2024-04-23T14:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.270+0000 7fbb01600640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.271+0000 7fbb01600640 1 -- 172.21.15.59:0/3997892207 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fbafc0ac9b0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.271+0000 7fbb09a00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.272+0000 7fbb09a00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8006fd0 tx=0x7fbaf8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.276+0000 7fbb01600640 1 -- 172.21.15.59:0/3997892207 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbafc07add0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.520+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb040d1eb0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.521+0000 7fbb01600640 1 -- 172.21.15.59:0/3997892207 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5181 (secure 0 0 0) 0x7fbafc07a6d0 con 0x7fbb0415e7c0 2024-04-23T14:35:35.522 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:35:35.522 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:35:09.775675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:35:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 34 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 msgr2=0x7fbae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8006fd0 tx=0x7fbaf8008040 comp rx=0 tx=0).stop 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415e7c0 msgr2=0x7fbb0415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415e7c0 0x7fbb0415d6d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009e80 tx=0x7fbafc004a40 comp rx=0 tx=0).stop 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 shutdown_connections 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb040d2630 0x7fbb0415d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415e7c0 0x7fbb0415d6d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 --2- 172.21.15.59:0/3997892207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb040d0d90 0x7fbb040b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.524+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 >> 172.21.15.59:0/3997892207 conn(0x7fbb0409d920 msgr2=0x7fbb0415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.525+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 shutdown_connections 2024-04-23T14:35:35.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.525+0000 7fbb0ae00640 1 -- 172.21.15.59:0/3997892207 wait complete. 2024-04-23T14:35:35.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-23T14:35:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.757+0000 7f3326e00640 1 Processor -- start 2024-04-23T14:35:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.757+0000 7f3326e00640 1 -- start start 2024-04-23T14:35:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.759+0000 7f3326e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332015b220 0x7f33200af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.759+0000 7f3326e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33200d05d0 0x7f33200d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.759+0000 7f3326e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332006fd30 con 0x7f332006f8d0 2024-04-23T14:35:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.759+0000 7f3326e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f33200d0f60 con 0x7f33200d05d0 2024-04-23T14:35:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.759+0000 7f3326e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f33200ae6b0 con 0x7f332015b220 2024-04-23T14:35:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.759+0000 7f3325000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f332006f8d0 0x7f33200adeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60802/0 (socket says 172.21.15.59:60802) 2024-04-23T14:35:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.759+0000 7f3325000640 1 -- 172.21.15.59:0/40621160 learned_addr learned my addr 172.21.15.59:0/40621160 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:35:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f3325a00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332015b220 0x7f33200af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f3326400640 1 --2- 172.21.15.59:0/40621160 >> v2:172.21.15.76:3300/0 conn(0x7f33200d05d0 0x7f33200d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455293587 0 0) 0x7f332006fd30 con 0x7f332006f8d0 2024-04-23T14:35:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3308003520 con 0x7f332006f8d0 2024-04-23T14:35:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 245228936 0 0) 0x7f3308003520 con 0x7f332006f8d0 2024-04-23T14:35:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332015b220 msgr2=0x7f33200af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332015b220 0x7f33200af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 >> v2:172.21.15.76:3300/0 conn(0x7f33200d05d0 msgr2=0x7f33200d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 --2- 172.21.15.59:0/40621160 >> v2:172.21.15.76:3300/0 conn(0x7f33200d05d0 0x7f33200d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33200ae990 con 0x7f332006f8d0 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f330c001670 con 0x7f332006f8d0 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.760+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 >> v1:172.21.15.59:6789/0 conn(0x7f332006f8d0 legacy=0x7f33200adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f331fe00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080039b0 0x7f3308003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f331fe00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332015b220 0x7f3308004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f331fe00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3308004d40 0x7f33080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33200d0f60 con 0x7f3308004d40 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33200ae6b0 con 0x7f33080039b0 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f330800dbd0 con 0x7f332015b220 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3326400640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3308004d40 0x7f33080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3325a00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080039b0 0x7f3308003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3326400640 1 -- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332015b220 msgr2=0x7f3308004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3326400640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332015b220 0x7f3308004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3326400640 1 -- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080039b0 msgr2=0x7f3308003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3326400640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080039b0 0x7f3308003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3326400640 1 -- 172.21.15.59:0/40621160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3308003520 con 0x7f3308004d40 2024-04-23T14:35:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.761+0000 7f3325a00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080039b0 0x7f3308003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.762+0000 7f3326400640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3308004d40 0x7f33080051c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3318009fc0 tx=0x7f3308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a7743568e1be76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318009c10 con 0x7f3308004d40 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3318036070 con 0x7f3308004d40 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f331fe00640 1 -- 172.21.15.59:0/40621160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331802d070 con 0x7f3308004d40 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f3326e00640 1 -- 172.21.15.59:0/40621160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3308004d40 msgr2=0x7f33080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f3326e00640 1 --2- 172.21.15.59:0/40621160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3308004d40 0x7f33080051c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3318009fc0 tx=0x7f3308005830 comp rx=0 tx=0).stop 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f3326400640 1 -- 172.21.15.59:0/40621160 reap_dead start 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f3326e00640 1 -- 172.21.15.59:0/40621160 shutdown_connections 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f3326e00640 1 -- 172.21.15.59:0/40621160 >> 172.21.15.59:0/40621160 conn(0x7f332006bdf0 msgr2=0x7f332006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f3326e00640 1 -- 172.21.15.59:0/40621160 shutdown_connections 2024-04-23T14:35:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.763+0000 7f3326e00640 1 -- 172.21.15.59:0/40621160 wait complete. 2024-04-23T14:35:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.765+0000 7f3326e00640 1 Processor -- start 2024-04-23T14:35:35.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.765+0000 7f3326e00640 1 -- start start 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.766+0000 7f3326e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3308004d40 0x7f33201f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.766+0000 7f3326e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33200b2940 0x7f332009bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.766+0000 7f3326e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332009c170 0x7f332014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.766+0000 7f3326e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33200af230 con 0x7f33200b2940 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.766+0000 7f3326e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33200aef50 con 0x7f3308004d40 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.766+0000 7f3326e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f332015b600 con 0x7f332009c170 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33200b2940 0x7f332009bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33200b2940 0x7f332009bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47950/0 (socket says 172.21.15.59:47950) 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 -- 172.21.15.59:0/1859732307 learned_addr learned my addr 172.21.15.59:0/1859732307 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3326400640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332009c170 0x7f332014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325a00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3308004d40 0x7f33201f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 -- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332009c170 msgr2=0x7f332014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332009c170 0x7f332014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 -- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3308004d40 msgr2=0x7f33201f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3308004d40 0x7f33201f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 -- 172.21.15.59:0/1859732307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33200b1600 con 0x7f33200b2940 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3326400640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332009c170 0x7f332014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.767+0000 7f3325000640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33200b2940 0x7f332009bc30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c770 tx=0x7f330c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.768+0000 7f331d600640 1 -- 172.21.15.59:0/1859732307 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c018070 con 0x7f33200b2940 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.768+0000 7f331d600640 1 -- 172.21.15.59:0/1859732307 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330c0074e0 con 0x7f33200b2940 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.768+0000 7f331d600640 1 -- 172.21.15.59:0/1859732307 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c0049e0 con 0x7f33200b2940 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.768+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33201f41c0 con 0x7f33200b2940 2024-04-23T14:35:35.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.768+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33200b1b80 con 0x7f33200b2940 2024-04-23T14:35:35.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.770+0000 7f331d600640 1 -- 172.21.15.59:0/1859732307 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f330c01b070 con 0x7f33200b2940 2024-04-23T14:35:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.771+0000 7f331d600640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f330006c920 0x7f330006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.771+0000 7f331d600640 1 -- 172.21.15.59:0/1859732307 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f330c08b230 con 0x7f33200b2940 2024-04-23T14:35:35.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.771+0000 7f3325a00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f330006c920 0x7f330006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.772+0000 7f332b22d740 1 Processor -- start 2024-04-23T14:35:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.772+0000 7f3325a00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f330006c920 0x7f330006ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3320073700 tx=0x7f3314008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.772+0000 7f332b22d740 1 -- start start 2024-04-23T14:35:35.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f332b22d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b87220 0x559fa6b87600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6ba4ac0 0x559fa6ba5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b5cf00 0x559fa6ba3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f332b22d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559fa6ac1190 con 0x559fa6b87220 2024-04-23T14:35:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559fa6a67030 con 0x559fa6b5cf00 2024-04-23T14:35:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x559fa6abd150 con 0x559fa6ba4ac0 2024-04-23T14:35:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f3325a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b87220 0x559fa6b87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.773+0000 7f3325a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b87220 0x559fa6b87600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47964/0 (socket says 172.21.15.59:47964) 2024-04-23T14:35:35.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3325a00640 1 -- 172.21.15.59:0/4027910375 learned_addr learned my addr 172.21.15.59:0/4027910375 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3326400640 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b5cf00 0x559fa6ba3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3325a00640 1 -- 172.21.15.59:0/4027910375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6ba4ac0 msgr2=0x559fa6ba5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3325a00640 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6ba4ac0 0x559fa6ba5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3325a00640 1 -- 172.21.15.59:0/4027910375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b5cf00 msgr2=0x559fa6ba3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3325a00640 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b5cf00 0x559fa6ba3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3325a00640 1 -- 172.21.15.59:0/4027910375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa6a4fb60 con 0x559fa6b87220 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.774+0000 7f3325a00640 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b87220 0x559fa6b87600 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3314011ca0 tx=0x7f3314014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7684fd425043e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f3312000640 1 -- 172.21.15.59:0/4027910375 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314011cd0 con 0x559fa6b87220 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f3312000640 1 -- 172.21.15.59:0/4027910375 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314026070 con 0x559fa6b87220 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f3312000640 1 -- 172.21.15.59:0/4027910375 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314021640 con 0x559fa6b87220 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f332b22d740 1 -- 172.21.15.59:0/4027910375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b87220 msgr2=0x559fa6b87600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f332b22d740 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b87220 0x559fa6b87600 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3314011ca0 tx=0x7f3314014000 comp rx=0 tx=0).stop 2024-04-23T14:35:35.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f332b22d740 1 -- 172.21.15.59:0/4027910375 shutdown_connections 2024-04-23T14:35:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f332b22d740 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6ba4ac0 0x559fa6ba5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f332b22d740 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b5cf00 0x559fa6ba3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f332b22d740 1 --2- 172.21.15.59:0/4027910375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b87220 0x559fa6b87600 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.775+0000 7f332b22d740 1 -- 172.21.15.59:0/4027910375 >> 172.21.15.59:0/4027910375 conn(0x559fa6b59830 msgr2=0x559fa6ba2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.776+0000 7f332b22d740 1 -- 172.21.15.59:0/4027910375 shutdown_connections 2024-04-23T14:35:35.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.776+0000 7f332b22d740 1 -- 172.21.15.59:0/4027910375 wait complete. 2024-04-23T14:35:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.776+0000 7f332b22d740 1 -- start start 2024-04-23T14:35:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.777+0000 7f332b22d740 1 Processor -- start 2024-04-23T14:35:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6ba2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f332b22d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6ba3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f332b22d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559fa6b5dca0 con 0x559fa6b82a80 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559fa6a7a500 con 0x559fa6ba4ac0 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x559fa6b57760 con 0x559fa6b87220 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f332b22d740 10 client.0 _ll_register_callbacks cb 0x559fa6ab7270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6ba3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6ba3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47980/0 (socket says 172.21.15.59:47980) 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 -- 172.21.15.59:0/1935947081 learned_addr learned my addr 172.21.15.59:0/1935947081 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3325a00640 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3325000640 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6ba2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 -- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 msgr2=0x559fa6b9a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b9a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 -- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 msgr2=0x559fa6ba2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6ba2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.778+0000 7f3326400640 1 -- 172.21.15.59:0/1935947081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa689f7c0 con 0x559fa6b82a80 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f3326400640 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6ba3480 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f331800e230 tx=0x7f3318004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b311f6d1f60657 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f3306a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f3306a00640 1 -- 172.21.15.59:0/1935947081 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318004450 con 0x559fa6b82a80 2024-04-23T14:35:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f3306a00640 1 -- 172.21.15.59:0/1935947081 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f33180045f0 con 0x559fa6b82a80 2024-04-23T14:35:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3314026070 con 0x559fa6b82a80 2024-04-23T14:35:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f3306a00640 1 -- 172.21.15.59:0/1935947081 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3318009cd0 con 0x559fa6b82a80 2024-04-23T14:35:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f332b22d740 10 client.6420 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:35.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.779+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x559fa68fff50 con 0x559fa6b82a80 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f3306a00640 1 -- 172.21.15.59:0/1935947081 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314026070 con 0x559fa6b82a80 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f3306a00640 1 -- 172.21.15.59:0/1935947081 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f3318046470 con 0x559fa6b82a80 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f332b22d740 10 client.6420 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f332b22d740 10 client.6420 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f332b22d740 20 client.6420 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f332b22d740 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6bb6080 0x559fa6bb6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f332b22d740 4 client.6420 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa6acebb0 con 0x559fa6bb6080 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.780+0000 7f3325a00640 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6bb6080 0x559fa6bb6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.781+0000 7f3325a00640 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6bb6080 0x559fa6bb6460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b44fa73fe19edea5 server_cookie=e43204f58f02cbac in_seq=0 out_seq=0 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.781+0000 7f3306a00640 10 client.6420 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.781+0000 7f3306a00640 1 -- 172.21.15.59:0/1935947081 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fa6acebb0 con 0x559fa6bb6080 2024-04-23T14:35:35.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.781+0000 7f3306a00640 10 client.6420 handle_command_reply: tid=0 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 client.6420 shutdown 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 msgr2=0x559fa6ba3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6ba3480 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f331800e230 tx=0x7f3318004ab0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 shutdown_connections 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6bb6080 0x559fa6bb6460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6ba3480 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6ba2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 --2- 172.21.15.59:0/1935947081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 >> 172.21.15.59:0/1935947081 conn(0x559fa6b5cf00 msgr2=0x559fa6ba5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f3306a00640 0 client.6420 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 shutdown_connections 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 1 -- 172.21.15.59:0/1935947081 wait complete. 2024-04-23T14:35:35.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.782+0000 7f332b22d740 20 client.6420 trim_cache size 0 max 16384 2024-04-23T14:35:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.797+0000 7f332b22d740 1 Processor -- start 2024-04-23T14:35:35.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.797+0000 7f332b22d740 1 -- start start 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f332b22d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6b5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b97da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6b982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f332b22d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3320200260 con 0x559fa6b82a80 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f332009b4f0 con 0x559fa6ba4ac0 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f33201ffc10 con 0x559fa6b87220 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f3325a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6b5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f3325a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6b5d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47988/0 (socket says 172.21.15.59:47988) 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.798+0000 7f3325a00640 1 -- 172.21.15.59:0/1245284000 learned_addr learned my addr 172.21.15.59:0/1245284000 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3325a00640 1 -- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 msgr2=0x559fa6b97da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3325000640 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b97da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3325a00640 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b97da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3326400640 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6b982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3325a00640 1 -- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 msgr2=0x559fa6b982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3325a00640 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6b982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3325a00640 1 -- 172.21.15.59:0/1245284000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa67930b0 con 0x559fa6b82a80 2024-04-23T14:35:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3325a00640 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6b5d4d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3314015fd0 tx=0x7f3314016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ab790c6c8621dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3311600640 1 -- 172.21.15.59:0/1245284000 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314037410 con 0x559fa6b82a80 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3311600640 1 -- 172.21.15.59:0/1245284000 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314026040 con 0x559fa6b82a80 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.799+0000 7f3311600640 1 -- 172.21.15.59:0/1245284000 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33140377b0 con 0x559fa6b82a80 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 -- 172.21.15.59:0/1245284000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 msgr2=0x559fa6b5d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6b5d4d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3314015fd0 tx=0x7f3314016000 comp rx=0 tx=0).stop 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 -- 172.21.15.59:0/1245284000 shutdown_connections 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b97da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6ba4ac0 0x559fa6b982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 --2- 172.21.15.59:0/1245284000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6b82a80 0x559fa6b5d4d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 -- 172.21.15.59:0/1245284000 >> 172.21.15.59:0/1245284000 conn(0x559fa6b5cf00 msgr2=0x559fa6b8af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 -- 172.21.15.59:0/1245284000 shutdown_connections 2024-04-23T14:35:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.800+0000 7f332b22d740 1 -- 172.21.15.59:0/1245284000 wait complete. 2024-04-23T14:35:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.801+0000 7f332b22d740 1 -- start start 2024-04-23T14:35:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.801+0000 7f332b22d740 1 Processor -- start 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b82a80 0x559fa6b85460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f332b22d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b979c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f332b22d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6ba4ac0 0x559fa6b97f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f332b22d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559fa6b88820 con 0x559fa6ba4ac0 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559fa6b88260 con 0x559fa6b82a80 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f332b22d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x559fa6b998c0 con 0x559fa6b87220 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f332b22d740 10 client.0 _ll_register_callbacks cb 0x7f33080036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f3326400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6ba4ac0 0x559fa6b97f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f3326400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6ba4ac0 0x559fa6b97f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47998/0 (socket says 172.21.15.59:47998) 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.802+0000 7f3326400640 1 -- 172.21.15.59:0/475561481 learned_addr learned my addr 172.21.15.59:0/475561481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3325000640 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3325a00640 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b82a80 0x559fa6b85460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3326400640 1 -- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 msgr2=0x559fa6b979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3326400640 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3326400640 1 -- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b82a80 msgr2=0x559fa6b85460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3326400640 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b82a80 0x559fa6b85460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3326400640 1 -- 172.21.15.59:0/475561481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa6b8b250 con 0x559fa6ba4ac0 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3326400640 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6ba4ac0 0x559fa6b97f00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f331802aef0 tx=0x7f331800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c2aa7d3587083a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3306a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3306a00640 1 -- 172.21.15.59:0/475561481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318004250 con 0x559fa6ba4ac0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.803+0000 7f3306a00640 1 -- 172.21.15.59:0/475561481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f331803ead0 con 0x559fa6ba4ac0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3314026040 con 0x559fa6ba4ac0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f3306a00640 1 -- 172.21.15.59:0/475561481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f331803ecd0 con 0x559fa6ba4ac0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 10 client.6426 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x559fa6c00dd0 con 0x559fa6ba4ac0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f3306a00640 1 -- 172.21.15.59:0/475561481 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314026040 con 0x559fa6ba4ac0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f3306a00640 1 -- 172.21.15.59:0/475561481 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f33180043f0 con 0x559fa6ba4ac0 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 10 client.6426 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 10 client.6426 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 20 client.6426 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6b80f10 0x559fa6b812f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 4 client.6426 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-23T14:35:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.804+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x559fa6ac8a20 con 0x559fa6b80f10 2024-04-23T14:35:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.805+0000 7f3325a00640 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6b80f10 0x559fa6b812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.805+0000 7f3325a00640 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6b80f10 0x559fa6b812f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee271bd6dab99db server_cookie=a598d42492beb71a in_seq=0 out_seq=0 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.805+0000 7f3306a00640 10 client.6426 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.805+0000 7f3306a00640 1 -- 172.21.15.59:0/475561481 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x559fa6ac8a20 con 0x559fa6b80f10 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.805+0000 7f3306a00640 10 client.6426 handle_command_reply: tid=0 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 client.6426 shutdown 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6ba4ac0 msgr2=0x559fa6b97f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6ba4ac0 0x559fa6b97f00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f331802aef0 tx=0x7f331800b040 comp rx=0 tx=0).stop 2024-04-23T14:35:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 shutdown_connections 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x559fa6b80f10 0x559fa6b812f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559fa6b87220 0x559fa6b979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f3306a00640 0 client.6426 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559fa6ba4ac0 0x559fa6b97f00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 --2- 172.21.15.59:0/475561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559fa6b82a80 0x559fa6b85460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 >> 172.21.15.59:0/475561481 conn(0x559fa6b5cf00 msgr2=0x559fa6bb1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 shutdown_connections 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 1 -- 172.21.15.59:0/475561481 wait complete. 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.806+0000 7f332b22d740 20 client.6426 trim_cache size 0 max 16384 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:35:35.809 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-23T14:35:35.810 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 1647722543, 2024-04-23T14:35:35.810 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-23T14:35:35.810 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir_test/file" 2024-04-23T14:35:35.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:35:35.810 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:35:35.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f330006c920 msgr2=0x7f330006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f330006c920 0x7f330006ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3320073700 tx=0x7f3314008040 comp rx=0 tx=0).stop 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33200b2940 msgr2=0x7f332009bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33200b2940 0x7f332009bc30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c770 tx=0x7f330c00cc30 comp rx=0 tx=0).stop 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 shutdown_connections 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f330006c920 0x7f330006ede0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f332009c170 0x7f332014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33200b2940 0x7f332009bc30 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 --2- 172.21.15.59:0/1859732307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3308004d40 0x7f33201f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 >> 172.21.15.59:0/1859732307 conn(0x7f332006bdf0 msgr2=0x7f3320158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 shutdown_connections 2024-04-23T14:35:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:35.807+0000 7f3326e00640 1 -- 172.21.15.59:0/1859732307 wait complete. 2024-04-23T14:35:35.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:35:36.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.032+0000 7f02b5e00640 1 Processor -- start 2024-04-23T14:35:36.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.032+0000 7f02b5e00640 1 -- start start 2024-04-23T14:35:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.034+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 0x7f02b006f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.034+0000 7f02b5e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02b00b3080 0x7f02b00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.034+0000 7f02b5e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b015f160 con 0x7f02b015ece0 2024-04-23T14:35:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.034+0000 7f02b5e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f02b00ced20 con 0x7f02b00b3080 2024-04-23T14:35:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.034+0000 7f02b5e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f02b015fcd0 con 0x7f02b00cd050 2024-04-23T14:35:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.034+0000 7f02afe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f02b015ece0 0x7f02b00cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60808/0 (socket says 172.21.15.59:60808) 2024-04-23T14:35:36.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.034+0000 7f02afe00640 1 -- 172.21.15.59:0/3558732935 learned_addr learned my addr 172.21.15.59:0/3558732935 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02b4a00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 0x7f02b006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02b5400640 1 --2- 172.21.15.59:0/3558732935 >> v2:172.21.15.76:3300/0 conn(0x7f02b00b3080 0x7f02b00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165821976 0 0) 0x7f02b015f160 con 0x7f02b015ece0 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0298003520 con 0x7f02b015ece0 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 824673473 0 0) 0x7f0298003520 con 0x7f02b015ece0 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 msgr2=0x7f02b006f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 0x7f02b006f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 >> v2:172.21.15.76:3300/0 conn(0x7f02b00b3080 msgr2=0x7f02b00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 --2- 172.21.15.59:0/3558732935 >> v2:172.21.15.76:3300/0 conn(0x7f02b00b3080 0x7f02b00cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b015f430 con 0x7f02b015ece0 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f029c001670 con 0x7f02b015ece0 2024-04-23T14:35:36.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 >> v1:172.21.15.59:6789/0 conn(0x7f02b015ece0 legacy=0x7f02b00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.035+0000 7f02af400640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02980039b0 0x7f0298003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02af400640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0298004800 0x7f0298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02af400640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 0x7f02980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02b015fcd0 con 0x7f0298004800 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f029800b9c0 con 0x7f02980039b0 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f029800bb40 con 0x7f02b00cd050 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02afe00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0298004800 0x7f0298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02b4a00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02980039b0 0x7f0298003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02b5400640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 0x7f02980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02afe00640 1 -- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 msgr2=0x7f02980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02afe00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00cd050 0x7f02980051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02afe00640 1 -- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02980039b0 msgr2=0x7f0298003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02afe00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02980039b0 0x7f0298003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02afe00640 1 -- 172.21.15.59:0/3558732935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0298003520 con 0x7f0298004800 2024-04-23T14:35:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.036+0000 7f02b4a00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02980039b0 0x7f0298003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.037+0000 7f02afe00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0298004800 0x7f0298004c80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f029c00c240 tx=0x7f0298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b55b19ec0ab193f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.038+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c007d50 con 0x7f0298004800 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.038+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c02e070 con 0x7f0298004800 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.038+0000 7f02af400640 1 -- 172.21.15.59:0/3558732935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c005d50 con 0x7f0298004800 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.038+0000 7f02b5e00640 1 -- 172.21.15.59:0/3558732935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0298004800 msgr2=0x7f0298004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.038+0000 7f02b5e00640 1 --2- 172.21.15.59:0/3558732935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0298004800 0x7f0298004c80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f029c00c240 tx=0x7f0298005830 comp rx=0 tx=0).stop 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.039+0000 7f02b5400640 1 -- 172.21.15.59:0/3558732935 reap_dead start 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.039+0000 7f02b5e00640 1 -- 172.21.15.59:0/3558732935 shutdown_connections 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.039+0000 7f02b5e00640 1 -- 172.21.15.59:0/3558732935 >> 172.21.15.59:0/3558732935 conn(0x7f02b00b21a0 msgr2=0x7f02b00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.039+0000 7f02b5e00640 1 -- 172.21.15.59:0/3558732935 shutdown_connections 2024-04-23T14:35:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.039+0000 7f02b5e00640 1 -- 172.21.15.59:0/3558732935 wait complete. 2024-04-23T14:35:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.041+0000 7f02b5e00640 1 Processor -- start 2024-04-23T14:35:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.041+0000 7f02b5e00640 1 -- start start 2024-04-23T14:35:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0298004800 0x7f02b01f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b009d8a0 0x7f02b00c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00c46b0 0x7f02b00cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b5e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02b00cef50 con 0x7f02b009d8a0 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b5e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02b015f9f0 con 0x7f0298004800 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b5e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f02b0159110 con 0x7f02b00c46b0 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b009d8a0 0x7f02b00c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b009d8a0 0x7f02b00c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48014/0 (socket says 172.21.15.59:48014) 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02afe00640 1 -- 172.21.15.59:0/3484660316 learned_addr learned my addr 172.21.15.59:0/3484660316 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b4a00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0298004800 0x7f02b01f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.042+0000 7f02b5400640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00c46b0 0x7f02b00cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02afe00640 1 -- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00c46b0 msgr2=0x7f02b00cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02afe00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00c46b0 0x7f02b00cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02afe00640 1 -- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0298004800 msgr2=0x7f02b01f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02afe00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0298004800 0x7f02b01f7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02afe00640 1 -- 172.21.15.59:0/3484660316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b00ca960 con 0x7f02b009d8a0 2024-04-23T14:35:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02b5400640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00c46b0 0x7f02b00cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02afe00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b009d8a0 0x7f02b00c9960 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f029c034040 tx=0x7f029c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02acc00640 1 -- 172.21.15.59:0/3484660316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c042020 con 0x7f02b009d8a0 2024-04-23T14:35:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02acc00640 1 -- 172.21.15.59:0/3484660316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c004bf0 con 0x7f02b009d8a0 2024-04-23T14:35:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02acc00640 1 -- 172.21.15.59:0/3484660316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c001c90 con 0x7f02b009d8a0 2024-04-23T14:35:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.043+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b00cac40 con 0x7f02b009d8a0 2024-04-23T14:35:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.044+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b00cb200 con 0x7f02b009d8a0 2024-04-23T14:35:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.045+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b004fd90 con 0x7f02b009d8a0 2024-04-23T14:35:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.046+0000 7f02acc00640 1 -- 172.21.15.59:0/3484660316 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f029c045020 con 0x7f02b009d8a0 2024-04-23T14:35:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.047+0000 7f02acc00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f029006c920 0x7f029006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.047+0000 7f02acc00640 1 -- 172.21.15.59:0/3484660316 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f029c0ac400 con 0x7f02b009d8a0 2024-04-23T14:35:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.047+0000 7f02b4a00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f029006c920 0x7f029006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.048+0000 7f02b4a00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f029006c920 0x7f029006ede0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f02a40099c0 tx=0x7f02a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.051+0000 7f02acc00640 1 -- 172.21.15.59:0/3484660316 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f029c04c030 con 0x7f02b009d8a0 2024-04-23T14:35:36.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.297+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f02b00ce3c0 con 0x7f02b009d8a0 2024-04-23T14:35:36.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.298+0000 7f02acc00640 1 -- 172.21.15.59:0/3484660316 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f029c07a960 con 0x7f02b009d8a0 2024-04-23T14:35:36.298 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:35:36.298 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-23T14:35:36.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f029006c920 msgr2=0x7f029006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f029006c920 0x7f029006ede0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f02a40099c0 tx=0x7f02a4008040 comp rx=0 tx=0).stop 2024-04-23T14:35:36.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b009d8a0 msgr2=0x7f02b00c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b009d8a0 0x7f02b00c9960 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f029c034040 tx=0x7f029c001a40 comp rx=0 tx=0).stop 2024-04-23T14:35:36.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 shutdown_connections 2024-04-23T14:35:36.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f029006c920 0x7f029006ede0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02b00c46b0 0x7f02b00cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b009d8a0 0x7f02b00c9960 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 --2- 172.21.15.59:0/3484660316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0298004800 0x7f02b01f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 >> 172.21.15.59:0/3484660316 conn(0x7f02b00b21a0 msgr2=0x7f02b006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 shutdown_connections 2024-04-23T14:35:36.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.301+0000 7f02b5e00640 1 -- 172.21.15.59:0/3484660316 wait complete. 2024-04-23T14:35:36.320 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-23T14:35:36.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:36.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-23T14:35:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.539+0000 7f517d600640 1 Processor -- start 2024-04-23T14:35:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.539+0000 7f517d600640 1 -- start start 2024-04-23T14:35:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.541+0000 7f517d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5178162350 0x7f517806cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.541+0000 7f517d600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f517815e990 0x7f51780cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.541+0000 7f517d600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f517806be60 con 0x7f517806b9d0 2024-04-23T14:35:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.541+0000 7f517d600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f517806ca70 con 0x7f517815e990 2024-04-23T14:35:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.541+0000 7f517d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f517806c140 con 0x7f5178162350 2024-04-23T14:35:36.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.541+0000 7f5177e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5178162350 0x7f517806cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.542+0000 7f5177e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5178162350 0x7f517806cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:43560/0 (socket says 172.21.15.59:43560) 2024-04-23T14:35:36.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.542+0000 7f5177e00640 1 -- 172.21.15.59:0/1640727184 learned_addr learned my addr 172.21.15.59:0/1640727184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.542+0000 7f517cc00640 1 --2- 172.21.15.59:0/1640727184 >> v2:172.21.15.76:3300/0 conn(0x7f517815e990 0x7f51780cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.542+0000 7f5177e00640 1 -- 172.21.15.59:0/1640727184 >> v2:172.21.15.76:3300/0 conn(0x7f517815e990 msgr2=0x7f51780cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.542+0000 7f5177e00640 1 --2- 172.21.15.59:0/1640727184 >> v2:172.21.15.76:3300/0 conn(0x7f517815e990 0x7f51780cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.543+0000 7f5177e00640 1 -- 172.21.15.59:0/1640727184 >> v1:172.21.15.76:6789/0 conn(0x7f517806b9d0 legacy=0x7f5178162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.543+0000 7f5177e00640 1 -- 172.21.15.59:0/1640727184 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517806c420 con 0x7f5178162350 2024-04-23T14:35:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.543+0000 7f517cc00640 1 --2- 172.21.15.59:0/1640727184 >> v2:172.21.15.76:3300/0 conn(0x7f517815e990 0x7f51780cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:35:36.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.544+0000 7f5177e00640 1 --2- 172.21.15.59:0/1640727184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5178162350 0x7f517806cf00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5168009870 tx=0x7f516802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa09d78f502a9098 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.544+0000 7f5176a00640 1 -- 172.21.15.59:0/1640727184 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168037070 con 0x7f5178162350 2024-04-23T14:35:36.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.544+0000 7f5176a00640 1 -- 172.21.15.59:0/1640727184 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5168032440 con 0x7f5178162350 2024-04-23T14:35:36.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.544+0000 7f5176a00640 1 -- 172.21.15.59:0/1640727184 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168032810 con 0x7f5178162350 2024-04-23T14:35:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 -- 172.21.15.59:0/1640727184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5178162350 msgr2=0x7f517806cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 --2- 172.21.15.59:0/1640727184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5178162350 0x7f517806cf00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5168009870 tx=0x7f516802a250 comp rx=0 tx=0).stop 2024-04-23T14:35:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 -- 172.21.15.59:0/1640727184 shutdown_connections 2024-04-23T14:35:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 --2- 172.21.15.59:0/1640727184 >> v2:172.21.15.76:3300/0 conn(0x7f517815e990 0x7f51780cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 --2- 172.21.15.59:0/1640727184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5178162350 0x7f517806cf00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 -- 172.21.15.59:0/1640727184 >> 172.21.15.59:0/1640727184 conn(0x7f51780b2830 msgr2=0x7f51780b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 -- 172.21.15.59:0/1640727184 shutdown_connections 2024-04-23T14:35:36.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.545+0000 7f517d600640 1 -- 172.21.15.59:0/1640727184 wait complete. 2024-04-23T14:35:36.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.547+0000 7f517d600640 1 Processor -- start 2024-04-23T14:35:36.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.547+0000 7f517d600640 1 -- start start 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f517d600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f517815e990 0x7f517815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f517d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5178162350 0x7f51781483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f517d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f517815d7d0 0x7f51780af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f517d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51781631c0 con 0x7f5178162350 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f517d600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5178163040 con 0x7f517815e990 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f517d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f51780ad870 con 0x7f517815d7d0 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f5177400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5178162350 0x7f51781483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f5177400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5178162350 0x7f51781483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48020/0 (socket says 172.21.15.59:48020) 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.548+0000 7f5177400640 1 -- 172.21.15.59:0/1503285095 learned_addr learned my addr 172.21.15.59:0/1503285095 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f5177e00640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f517815e990 0x7f517815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f517cc00640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f517815d7d0 0x7f51780af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f5177400640 1 -- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f517815d7d0 msgr2=0x7f51780af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f5177400640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f517815d7d0 0x7f51780af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f5177400640 1 -- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f517815e990 msgr2=0x7f517815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f5177400640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f517815e990 0x7f517815b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f5177400640 1 -- 172.21.15.59:0/1503285095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5178158c00 con 0x7f5178162350 2024-04-23T14:35:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.549+0000 7f5177400640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5178162350 0x7f51781483e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f516400c400 tx=0x7f516400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.550+0000 7f516fe00640 1 -- 172.21.15.59:0/1503285095 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51640052d0 con 0x7f5178162350 2024-04-23T14:35:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.550+0000 7f516fe00640 1 -- 172.21.15.59:0/1503285095 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5164005490 con 0x7f5178162350 2024-04-23T14:35:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.550+0000 7f516fe00640 1 -- 172.21.15.59:0/1503285095 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164007260 con 0x7f5178162350 2024-04-23T14:35:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.550+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517815bf50 con 0x7f5178162350 2024-04-23T14:35:36.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.550+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517815c510 con 0x7f5178162350 2024-04-23T14:35:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.552+0000 7f516fe00640 1 -- 172.21.15.59:0/1503285095 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5164007400 con 0x7f5178162350 2024-04-23T14:35:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.553+0000 7f516fe00640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f515806c8f0 0x7f515806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.553+0000 7f516fe00640 1 -- 172.21.15.59:0/1503285095 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f516408b210 con 0x7f5178162350 2024-04-23T14:35:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.553+0000 7f5177e00640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f515806c8f0 0x7f515806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.554+0000 7f5181ba3740 1 Processor -- start 2024-04-23T14:35:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.554+0000 7f5181ba3740 1 -- start start 2024-04-23T14:35:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.554+0000 7f5177e00640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f515806c8f0 0x7f515806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5168034000 tx=0x7f516802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a272df0 0x560f3a29ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a2731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a2715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5181ba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f3a1d7ff0 con 0x560f3a272df0 2024-04-23T14:35:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560f3a17df40 con 0x560f3a29b170 2024-04-23T14:35:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560f3a1d3fb0 con 0x560f3a2bba60 2024-04-23T14:35:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a272df0 0x560f3a29ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a272df0 0x560f3a29ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48030/0 (socket says 172.21.15.59:48030) 2024-04-23T14:35:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 -- 172.21.15.59:0/2689836890 learned_addr learned my addr 172.21.15.59:0/2689836890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 -- 172.21.15.59:0/2689836890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 msgr2=0x560f3a2715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 --2- 172.21.15.59:0/2689836890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a2715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 -- 172.21.15.59:0/2689836890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 msgr2=0x560f3a2731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 --2- 172.21.15.59:0/2689836890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a2731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.555+0000 7f5177e00640 1 -- 172.21.15.59:0/2689836890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f39fb6940 con 0x560f3a272df0 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.556+0000 7f5177e00640 1 --2- 172.21.15.59:0/2689836890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a272df0 0x560f3a29ac30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5168000c00 tx=0x7f516802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8982d962351254ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.556+0000 7f5163e00640 1 -- 172.21.15.59:0/2689836890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516804e070 con 0x560f3a272df0 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.556+0000 7f5163e00640 1 -- 172.21.15.59:0/2689836890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5168032d40 con 0x560f3a272df0 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.556+0000 7f5163e00640 1 -- 172.21.15.59:0/2689836890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168037040 con 0x560f3a272df0 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 -- 172.21.15.59:0/2689836890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a272df0 msgr2=0x560f3a29ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2689836890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a272df0 0x560f3a29ac30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5168000c00 tx=0x7f516802c000 comp rx=0 tx=0).stop 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 -- 172.21.15.59:0/2689836890 shutdown_connections 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2689836890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a2715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2689836890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a2731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2689836890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a272df0 0x560f3a29ac30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 -- 172.21.15.59:0/2689836890 >> 172.21.15.59:0/2689836890 conn(0x560f3a270010 msgr2=0x560f3a2b89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 -- 172.21.15.59:0/2689836890 shutdown_connections 2024-04-23T14:35:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.557+0000 7f5181ba3740 1 -- 172.21.15.59:0/2689836890 wait complete. 2024-04-23T14:35:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.558+0000 7f5181ba3740 1 -- start start 2024-04-23T14:35:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.558+0000 7f5181ba3740 1 Processor -- start 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.559+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a295f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.559+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a29ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.559+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a29d4b0 0x560f3a29e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.559+0000 7f5181ba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f3a29a7a0 con 0x560f3a29d4b0 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.559+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560f3a1912c0 con 0x560f3a29b170 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.559+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560f3a29a190 con 0x560f3a2bba60 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.559+0000 7f5181ba3740 10 client.0 _ll_register_callbacks cb 0x560f3a15fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a29d4b0 0x560f3a29e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a29d4b0 0x560f3a29e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48042/0 (socket says 172.21.15.59:48042) 2024-04-23T14:35:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 -- 172.21.15.59:0/1307554705 learned_addr learned my addr 172.21.15.59:0/1307554705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f5177400640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a29ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f5177e00640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a295f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 -- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 msgr2=0x560f3a29ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a29ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 -- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 msgr2=0x560f3a295f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a295f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 -- 172.21.15.59:0/1307554705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3a0170d0 con 0x560f3a29d4b0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f5177400640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a29ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f5177e00640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a295f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f517cc00640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a29d4b0 0x560f3a29e3a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5170007de0 tx=0x7f5170010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ce3a67d3808592 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.560+0000 7f5160c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5160c00640 1 -- 172.21.15.59:0/1307554705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517001a020 con 0x560f3a29d4b0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5160c00640 1 -- 172.21.15.59:0/1307554705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5170015440 con 0x560f3a29d4b0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5168032d40 con 0x560f3a29d4b0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5160c00640 1 -- 172.21.15.59:0/1307554705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5170024050 con 0x560f3a29d4b0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5181ba3740 10 client.6441 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560f39eaa230 con 0x560f3a29d4b0 2024-04-23T14:35:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5160c00640 1 -- 172.21.15.59:0/1307554705 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5168032d40 con 0x560f3a29d4b0 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.561+0000 7f5160c00640 1 -- 172.21.15.59:0/1307554705 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f5170027020 con 0x560f3a29d4b0 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5181ba3740 10 client.6441 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5181ba3740 10 client.6441 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5181ba3740 20 client.6441 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5181ba3740 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a29c850 0x560f3a29cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5181ba3740 4 client.6441 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3a1e5a10 con 0x560f3a29c850 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5177e00640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a29c850 0x560f3a29cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5177e00640 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a29c850 0x560f3a29cc30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98a60bff1850302 server_cookie=4d3eb2ae699047c6 in_seq=0 out_seq=0 2024-04-23T14:35:36.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.562+0000 7f5160c00640 10 client.6441 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.563+0000 7f5160c00640 1 -- 172.21.15.59:0/1307554705 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f3a1e5a10 con 0x560f3a29c850 2024-04-23T14:35:36.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.563+0000 7f5160c00640 10 client.6441 handle_command_reply: tid=0 2024-04-23T14:35:36.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.563+0000 7f5181ba3740 1 client.6441 shutdown 2024-04-23T14:35:36.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.563+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a29d4b0 msgr2=0x560f3a29e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.563+0000 7f5181ba3740 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a29d4b0 0x560f3a29e3a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5170007de0 tx=0x7f5170010590 comp rx=0 tx=0).stop 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 shutdown_connections 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a29c850 0x560f3a29cc30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a2bba60 0x560f3a29ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a29d4b0 0x560f3a29e3a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 --2- 172.21.15.59:0/1307554705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a29b170 0x560f3a295f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5160c00640 0 client.6441 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 >> 172.21.15.59:0/1307554705 conn(0x560f3a272df0 msgr2=0x560f3a26cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 shutdown_connections 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 1 -- 172.21.15.59:0/1307554705 wait complete. 2024-04-23T14:35:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.564+0000 7f5181ba3740 20 client.6441 trim_cache size 0 max 16384 2024-04-23T14:35:36.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.584+0000 7f5181ba3740 1 Processor -- start 2024-04-23T14:35:36.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.584+0000 7f5181ba3740 1 -- start start 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a26ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a2716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a271c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5181ba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560f3a296f60 con 0x560f3a26d360 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5178200c60 con 0x560f3a2bba60 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f51781ffcd0 con 0x560f3a29b170 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a271c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a271c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48048/0 (socket says 172.21.15.59:48048) 2024-04-23T14:35:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 -- 172.21.15.59:0/2965954124 learned_addr learned my addr 172.21.15.59:0/2965954124 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5177e00640 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a26ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 -- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 msgr2=0x560f3a26ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a26ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 -- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 msgr2=0x560f3a2716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a2716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f517cc00640 1 -- 172.21.15.59:0/2965954124 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3a270750 con 0x560f3a26d360 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.585+0000 7f5177e00640 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a26ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.586+0000 7f517cc00640 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a271c20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f517000b010 tx=0x7f5170016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43ceace05c6b3bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.586+0000 7f5163400640 1 -- 172.21.15.59:0/2965954124 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517002d070 con 0x560f3a26d360 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.586+0000 7f5163400640 1 -- 172.21.15.59:0/2965954124 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51700154c0 con 0x560f3a26d360 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.586+0000 7f5163400640 1 -- 172.21.15.59:0/2965954124 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517001a050 con 0x560f3a26d360 2024-04-23T14:35:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 -- 172.21.15.59:0/2965954124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 msgr2=0x560f3a271c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a271c20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f517000b010 tx=0x7f5170016000 comp rx=0 tx=0).stop 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 -- 172.21.15.59:0/2965954124 shutdown_connections 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a271c20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a2716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2965954124 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a26ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 -- 172.21.15.59:0/2965954124 >> 172.21.15.59:0/2965954124 conn(0x560f3a272df0 msgr2=0x560f3a29f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 -- 172.21.15.59:0/2965954124 shutdown_connections 2024-04-23T14:35:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.587+0000 7f5181ba3740 1 -- 172.21.15.59:0/2965954124 wait complete. 2024-04-23T14:35:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.588+0000 7f5181ba3740 1 -- start start 2024-04-23T14:35:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.588+0000 7f5181ba3740 1 Processor -- start 2024-04-23T14:35:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a2bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a333c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5181ba3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a3341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5181ba3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51781ff680 con 0x560f3a26d360 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f51780b0310 con 0x560f3a2bba60 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5181ba3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x560f3a3349f0 con 0x560f3a29b170 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5181ba3740 10 client.0 _ll_register_callbacks cb 0x7f5170007a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a2bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5177e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a2bc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48060/0 (socket says 172.21.15.59:48060) 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f5177e00640 1 -- 172.21.15.59:0/2558662946 learned_addr learned my addr 172.21.15.59:0/2558662946 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.589+0000 7f517cc00640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a3341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177400640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a333c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177e00640 1 -- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 msgr2=0x560f3a333c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177e00640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a333c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177e00640 1 -- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 msgr2=0x560f3a3341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177e00640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a3341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177e00640 1 -- 172.21.15.59:0/2558662946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3a2bcdd0 con 0x560f3a26d360 2024-04-23T14:35:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f517cc00640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a3341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177400640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a333c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5177e00640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a2bc890 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5168009990 tx=0x7f5168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bab5e4070ef925 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5160c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5160c00640 1 -- 172.21.15.59:0/2558662946 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516804e040 con 0x560f3a26d360 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5160c00640 1 -- 172.21.15.59:0/2558662946 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5168005990 con 0x560f3a26d360 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.590+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51700154c0 con 0x560f3a26d360 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5160c00640 1 -- 172.21.15.59:0/2558662946 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5168037040 con 0x560f3a26d360 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 10 client.6447 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560f3a2b8bd0 con 0x560f3a26d360 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5160c00640 1 -- 172.21.15.59:0/2558662946 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51700154c0 con 0x560f3a26d360 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5160c00640 1 -- 172.21.15.59:0/2558662946 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f5168051070 con 0x560f3a26d360 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 10 client.6447 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 10 client.6447 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 20 client.6447 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a31d360 0x560f3a2cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 4 client.6447 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-23T14:35:36.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x560f3a1df880 con 0x560f3a31d360 2024-04-23T14:35:36.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.591+0000 7f5177400640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a31d360 0x560f3a2cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.592+0000 7f5177400640 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a31d360 0x560f3a2cb590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1523b34f6138c9 server_cookie=80bd6b8fa4cdd594 in_seq=0 out_seq=0 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.592+0000 7f5160c00640 10 client.6447 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5160c00640 1 -- 172.21.15.59:0/2558662946 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f3a1df880 con 0x560f3a31d360 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5160c00640 10 client.6447 handle_command_reply: tid=0 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 client.6447 shutdown 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 msgr2=0x560f3a2bc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a2bc890 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5168009990 tx=0x7f5168004730 comp rx=0 tx=0).stop 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 shutdown_connections 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x560f3a31d360 0x560f3a2cb590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x560f3a29b170 0x560f3a333c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560f3a2bba60 0x560f3a3341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 --2- 172.21.15.59:0/2558662946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560f3a26d360 0x560f3a2bc890 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5160c00640 0 client.6447 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 >> 172.21.15.59:0/2558662946 conn(0x560f3a272df0 msgr2=0x560f3a29de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 shutdown_connections 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.593+0000 7f5181ba3740 1 -- 172.21.15.59:0/2558662946 wait complete. 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f5181ba3740 20 client.6447 trim_cache size 0 max 16384 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:36.596 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:35:36.597 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "58a57886-6808-4c83-a0a6-bcdf67f58a72", 2024-04-23T14:35:36.597 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:35:36.597 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:36.597 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f3114800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-23T14:35:02.847146+0000 1=1+0) caps={6318=pAsLsXsFs/-@2},l=6318(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55cd9aa38b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:35:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f515806c8f0 msgr2=0x7f515806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f515806c8f0 0x7f515806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5168034000 tx=0x7f516802e000 comp rx=0 tx=0).stop 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5178162350 msgr2=0x7f51781483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5178162350 0x7f51781483e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f516400c400 tx=0x7f516400c8c0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 shutdown_connections 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f515806c8f0 0x7f515806edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f517815d7d0 0x7f51780af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5178162350 0x7f51781483e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 --2- 172.21.15.59:0/1503285095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f517815e990 0x7f517815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.594+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 >> 172.21.15.59:0/1503285095 conn(0x7f51780b2830 msgr2=0x7f51781591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.595+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 shutdown_connections 2024-04-23T14:35:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.595+0000 7f517d600640 1 -- 172.21.15.59:0/1503285095 wait complete. 2024-04-23T14:35:36.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:35:36.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-23T14:35:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.811+0000 7ff187e00640 1 Processor -- start 2024-04-23T14:35:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.812+0000 7ff187e00640 1 -- start start 2024-04-23T14:35:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.813+0000 7ff187e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff18009aa00 0x7ff18009ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.813+0000 7ff187e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff18006cf60 0x7ff1800ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.813+0000 7ff187e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1801593e0 con 0x7ff18006cae0 2024-04-23T14:35:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.813+0000 7ff187e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff1800ccbf0 con 0x7ff18009aa00 2024-04-23T14:35:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.813+0000 7ff187e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff18009b320 con 0x7ff18006cf60 2024-04-23T14:35:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.814+0000 7ff185400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff18009aa00 0x7ff18009ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.814+0000 7ff185400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff18009aa00 0x7ff18009ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48064/0 (socket says 172.21.15.59:48064) 2024-04-23T14:35:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.814+0000 7ff185400640 1 -- 172.21.15.59:0/782641826 learned_addr learned my addr 172.21.15.59:0/782641826 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.814+0000 7ff185400640 1 -- 172.21.15.59:0/782641826 >> v2:172.21.15.76:3300/0 conn(0x7ff18006cf60 msgr2=0x7ff1800ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.814+0000 7ff185400640 1 --2- 172.21.15.59:0/782641826 >> v2:172.21.15.76:3300/0 conn(0x7ff18006cf60 0x7ff1800ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.814+0000 7ff185400640 1 -- 172.21.15.59:0/782641826 >> v1:172.21.15.76:6789/0 conn(0x7ff18006cae0 legacy=0x7ff1800cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.814+0000 7ff185400640 1 -- 172.21.15.59:0/782641826 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1800ce9e0 con 0x7ff18009aa00 2024-04-23T14:35:36.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.815+0000 7ff185400640 1 --2- 172.21.15.59:0/782641826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff18009ade0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff180073b20 tx=0x7ff17402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db6c7db9f1cc052 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.816+0000 7ff17fe00640 1 -- 172.21.15.59:0/782641826 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174037070 con 0x7ff18009aa00 2024-04-23T14:35:36.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.816+0000 7ff17fe00640 1 -- 172.21.15.59:0/782641826 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff174032440 con 0x7ff18009aa00 2024-04-23T14:35:36.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.816+0000 7ff17fe00640 1 -- 172.21.15.59:0/782641826 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174032810 con 0x7ff18009aa00 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.816+0000 7ff187e00640 1 -- 172.21.15.59:0/782641826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 msgr2=0x7ff18009ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.816+0000 7ff187e00640 1 --2- 172.21.15.59:0/782641826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff18009ade0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff180073b20 tx=0x7ff17402a230 comp rx=0 tx=0).stop 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.817+0000 7ff187e00640 1 -- 172.21.15.59:0/782641826 shutdown_connections 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.817+0000 7ff187e00640 1 --2- 172.21.15.59:0/782641826 >> v2:172.21.15.76:3300/0 conn(0x7ff18006cf60 0x7ff1800ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.817+0000 7ff187e00640 1 --2- 172.21.15.59:0/782641826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff18009ade0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.817+0000 7ff187e00640 1 -- 172.21.15.59:0/782641826 >> 172.21.15.59:0/782641826 conn(0x7ff1800b4090 msgr2=0x7ff1800b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.817+0000 7ff187e00640 1 -- 172.21.15.59:0/782641826 shutdown_connections 2024-04-23T14:35:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.817+0000 7ff187e00640 1 -- 172.21.15.59:0/782641826 wait complete. 2024-04-23T14:35:36.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.818+0000 7ff187e00640 1 Processor -- start 2024-04-23T14:35:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.819+0000 7ff187e00640 1 -- start start 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.819+0000 7ff187e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff18006cae0 0x7ff1800cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff187e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff18006cf60 0x7ff1800cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff187e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff1800cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff187e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff18006db20 con 0x7ff18009aa00 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff187e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff18006d840 con 0x7ff18006cae0 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff187e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff18006de00 con 0x7ff18006cf60 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff1800cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff1800cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48080/0 (socket says 172.21.15.59:48080) 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185e00640 1 -- 172.21.15.59:0/2281689157 learned_addr learned my addr 172.21.15.59:0/2281689157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185400640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff18006cae0 0x7ff1800cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff184a00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff18006cf60 0x7ff1800cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185e00640 1 -- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff18006cf60 msgr2=0x7ff1800cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff18006cf60 0x7ff1800cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185e00640 1 -- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff18006cae0 msgr2=0x7ff1800cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.820+0000 7ff185e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff18006cae0 0x7ff1800cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff185e00640 1 -- 172.21.15.59:0/2281689157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1801629c0 con 0x7ff18009aa00 2024-04-23T14:35:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff185400640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff18006cae0 0x7ff1800cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:35:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff185e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff1800cb510 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff17800b9a0 tx=0x7ff17800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff17d600640 1 -- 172.21.15.59:0/2281689157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178019070 con 0x7ff18009aa00 2024-04-23T14:35:36.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff17d600640 1 -- 172.21.15.59:0/2281689157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17800cbc0 con 0x7ff18009aa00 2024-04-23T14:35:36.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff180162ca0 con 0x7ff18009aa00 2024-04-23T14:35:36.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff17d600640 1 -- 172.21.15.59:0/2281689157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178014440 con 0x7ff18009aa00 2024-04-23T14:35:36.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.821+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1800b5380 con 0x7ff18009aa00 2024-04-23T14:35:36.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.824+0000 7ff17d600640 1 -- 172.21.15.59:0/2281689157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff17801c070 con 0x7ff18009aa00 2024-04-23T14:35:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.825+0000 7ff17d600640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff16006c8f0 0x7ff16006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.825+0000 7ff17d600640 1 -- 172.21.15.59:0/2281689157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff17808b980 con 0x7ff18009aa00 2024-04-23T14:35:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.825+0000 7ff185400640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff16006c8f0 0x7ff16006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.825+0000 7ff18ab83740 1 Processor -- start 2024-04-23T14:35:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.825+0000 7ff18ab83740 1 -- start start 2024-04-23T14:35:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.826+0000 7ff185400640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff16006c8f0 0x7ff16006edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff17402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.826+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9473ac30 0x55ba946ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.826+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471dd50 0x55ba9471d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.826+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471d610 0x55ba946f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.826+0000 7ff18ab83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ba946571f0 con 0x55ba9473ac30 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.826+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ba945fd350 con 0x55ba9471dd50 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.826+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ba946531b0 con 0x55ba9471d610 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9473ac30 0x55ba946ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9473ac30 0x55ba946ed5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48096/0 (socket says 172.21.15.59:48096) 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 -- 172.21.15.59:0/2213120853 learned_addr learned my addr 172.21.15.59:0/2213120853 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff184a00640 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471dd50 0x55ba9471d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 -- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471d610 msgr2=0x55ba946f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471d610 0x55ba946f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 -- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471dd50 msgr2=0x55ba9471d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471dd50 0x55ba9471d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 -- 172.21.15.59:0/2213120853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba94651f50 con 0x55ba9473ac30 2024-04-23T14:35:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff184a00640 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471dd50 0x55ba9471d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.827+0000 7ff185400640 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9473ac30 0x55ba946ed5c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff17402a820 tx=0x7ff174008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b831e086c80d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff171c00640 1 -- 172.21.15.59:0/2213120853 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17404e070 con 0x55ba9473ac30 2024-04-23T14:35:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff171c00640 1 -- 172.21.15.59:0/2213120853 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff174032de0 con 0x55ba9473ac30 2024-04-23T14:35:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff171c00640 1 -- 172.21.15.59:0/2213120853 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174037040 con 0x55ba9473ac30 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff18ab83740 1 -- 172.21.15.59:0/2213120853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9473ac30 msgr2=0x55ba946ed5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff18ab83740 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9473ac30 0x55ba946ed5c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff17402a820 tx=0x7ff174008030 comp rx=0 tx=0).stop 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff18ab83740 1 -- 172.21.15.59:0/2213120853 shutdown_connections 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff18ab83740 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471d610 0x55ba946f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff18ab83740 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471dd50 0x55ba9471d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff18ab83740 1 --2- 172.21.15.59:0/2213120853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9473ac30 0x55ba946ed5c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.828+0000 7ff18ab83740 1 -- 172.21.15.59:0/2213120853 >> 172.21.15.59:0/2213120853 conn(0x55ba946ef830 msgr2=0x55ba947380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.829+0000 7ff18ab83740 1 -- 172.21.15.59:0/2213120853 shutdown_connections 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.829+0000 7ff18ab83740 1 -- 172.21.15.59:0/2213120853 wait complete. 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.829+0000 7ff18ab83740 1 -- start start 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.830+0000 7ff18ab83740 1 Processor -- start 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.830+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba9470aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.830+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9473ac30 0x55ba9471e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471e940 0x55ba946f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff18ab83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ba946f19e0 con 0x55ba9471dd50 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ba94610690 con 0x55ba9471e940 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ba946f0dd0 con 0x55ba9473ac30 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff18ab83740 10 client.0 _ll_register_callbacks cb 0x55ba9464c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba9470aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba9470aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48104/0 (socket says 172.21.15.59:48104) 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 -- 172.21.15.59:0/1977393933 learned_addr learned my addr 172.21.15.59:0/1977393933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 -- 172.21.15.59:0/1977393933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9473ac30 msgr2=0x55ba9471e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9473ac30 0x55ba9471e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 -- 172.21.15.59:0/1977393933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471e940 msgr2=0x55ba946f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471e940 0x55ba946f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.831+0000 7ff185400640 1 -- 172.21.15.59:0/1977393933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9460f920 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff185400640 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba9470aac0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff17404c040 tx=0x7ff174004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53cc568dc9946fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff166000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff166000640 1 -- 172.21.15.59:0/1977393933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174057060 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff166000640 1 -- 172.21.15.59:0/1977393933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff17403cc80 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff174032de0 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff166000640 1 -- 172.21.15.59:0/1977393933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff17404e070 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff18ab83740 10 client.6459 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55ba944337d0 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.832+0000 7ff166000640 1 -- 172.21.15.59:0/1977393933 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff174032de0 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff166000640 1 -- 172.21.15.59:0/1977393933 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7ff174060020 con 0x55ba9471dd50 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff18ab83740 10 client.6459 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff18ab83740 10 client.6459 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff18ab83740 20 client.6459 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba94720300 0x55ba94720770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff18ab83740 4 client.6459 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:35:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba94664c10 con 0x55ba94720300 2024-04-23T14:35:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.833+0000 7ff184a00640 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba94720300 0x55ba94720770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.834+0000 7ff184a00640 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba94720300 0x55ba94720770 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c555f47986f63287 server_cookie=c32c0f1ff67ea1e in_seq=0 out_seq=0 2024-04-23T14:35:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.834+0000 7ff166000640 10 client.6459 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.834+0000 7ff166000640 1 -- 172.21.15.59:0/1977393933 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba94664c10 con 0x55ba94720300 2024-04-23T14:35:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.834+0000 7ff166000640 10 client.6459 handle_command_reply: tid=0 2024-04-23T14:35:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.834+0000 7ff18ab83740 1 client.6459 shutdown 2024-04-23T14:35:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 msgr2=0x55ba9470aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba9470aac0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff17404c040 tx=0x7ff174004730 comp rx=0 tx=0).stop 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 shutdown_connections 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba94720300 0x55ba94720770 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9473ac30 0x55ba9471e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9471e940 0x55ba946f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff166000640 0 client.6459 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1977393933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba9470aac0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 >> 172.21.15.59:0/1977393933 conn(0x55ba9471d610 msgr2=0x55ba946ecc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 shutdown_connections 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 1 -- 172.21.15.59:0/1977393933 wait complete. 2024-04-23T14:35:36.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.835+0000 7ff18ab83740 20 client.6459 trim_cache size 0 max 16384 2024-04-23T14:35:36.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.854+0000 7ff18ab83740 1 Processor -- start 2024-04-23T14:35:36.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.855+0000 7ff18ab83740 1 -- start start 2024-04-23T14:35:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.855+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba94731730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.855+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 0x55ba94738200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.855+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba94738740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.855+0000 7ff18ab83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff180200f90 con 0x55ba9471dd50 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.855+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801622e0 con 0x55ba9473ac30 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.855+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800b6d70 con 0x55ba9471e940 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba94731730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba94731730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48112/0 (socket says 172.21.15.59:48112) 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 -- 172.21.15.59:0/1206309630 learned_addr learned my addr 172.21.15.59:0/1206309630 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 -- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 msgr2=0x55ba94738200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 0x55ba94738200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185e00640 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba94738740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 -- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 msgr2=0x55ba94738740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba94738740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185400640 1 -- 172.21.15.59:0/1206309630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba943693f0 con 0x55ba9471dd50 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.856+0000 7ff185e00640 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba94738740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:35:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.857+0000 7ff185400640 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba94731730 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff17402a820 tx=0x7ff17404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725f7bc7ebf2dd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.857+0000 7ff171200640 1 -- 172.21.15.59:0/1206309630 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174037040 con 0x55ba9471dd50 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.857+0000 7ff171200640 1 -- 172.21.15.59:0/1206309630 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1740524f0 con 0x55ba9471dd50 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.857+0000 7ff171200640 1 -- 172.21.15.59:0/1206309630 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17403cb20 con 0x55ba9471dd50 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.857+0000 7ff18ab83740 1 -- 172.21.15.59:0/1206309630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 msgr2=0x55ba94731730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.857+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba94731730 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff17402a820 tx=0x7ff17404e000 comp rx=0 tx=0).stop 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 -- 172.21.15.59:0/1206309630 shutdown_connections 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 0x55ba94738200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba94738740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 --2- 172.21.15.59:0/1206309630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba94731730 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 -- 172.21.15.59:0/1206309630 >> 172.21.15.59:0/1206309630 conn(0x55ba9471d610 msgr2=0x55ba9474a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 -- 172.21.15.59:0/1206309630 shutdown_connections 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 -- 172.21.15.59:0/1206309630 wait complete. 2024-04-23T14:35:36.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.858+0000 7ff18ab83740 1 -- start start 2024-04-23T14:35:36.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.859+0000 7ff18ab83740 1 Processor -- start 2024-04-23T14:35:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba947b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 0x55ba9475b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff18ab83740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba9479ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff18ab83740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ba9473a390 con 0x55ba9471dd50 2024-04-23T14:35:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff180161c90 con 0x55ba9473ac30 2024-04-23T14:35:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff18ab83740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ba9471d450 con 0x55ba9471e940 2024-04-23T14:35:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff18ab83740 10 client.0 _ll_register_callbacks cb 0x7ff17404c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba947b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba947b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48120/0 (socket says 172.21.15.59:48120) 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 -- 172.21.15.59:0/186977318 learned_addr learned my addr 172.21.15.59:0/186977318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff184a00640 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 0x55ba9475b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 -- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 msgr2=0x55ba9475b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 0x55ba9475b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185e00640 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba9479ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 -- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 msgr2=0x55ba9479ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba9479ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.860+0000 7ff185400640 1 -- 172.21.15.59:0/186977318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba943290f0 con 0x55ba9471dd50 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff185400640 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba947b19d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff1740090b0 tx=0x7ff174009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcd2a3d0decc516 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:35:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff166000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff166000640 1 -- 172.21.15.59:0/186977318 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174037070 con 0x55ba9471dd50 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff166000640 1 -- 172.21.15.59:0/186977318 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff1740528a0 con 0x55ba9471dd50 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1740524f0 con 0x55ba9471dd50 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff166000640 1 -- 172.21.15.59:0/186977318 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff17406f050 con 0x55ba9471dd50 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff18ab83740 10 client.6465 fetch_fsmap learned FSMap version 34 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.861+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55ba9461c810 con 0x55ba9471dd50 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff166000640 1 -- 172.21.15.59:0/186977318 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1740524f0 con 0x55ba9471dd50 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff166000640 1 -- 172.21.15.59:0/186977318 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7ff174072020 con 0x55ba9471dd50 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff18ab83740 10 client.6465 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff18ab83740 10 client.6465 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff18ab83740 20 client.6465 populate_metadata read hostname 'smithi059' 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff18ab83740 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba9475bfd0 0x55ba946ec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff18ab83740 4 client.6465 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba946eec80 con 0x55ba9475bfd0 2024-04-23T14:35:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.862+0000 7ff184a00640 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba9475bfd0 0x55ba946ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff184a00640 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba9475bfd0 0x55ba946ec390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91aa5ff34a5bb7d9 server_cookie=d7398d4629f15723 in_seq=0 out_seq=0 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff166000640 10 client.6465 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff166000640 1 -- 172.21.15.59:0/186977318 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55ba946eec80 con 0x55ba9475bfd0 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff166000640 10 client.6465 handle_command_reply: tid=0 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 client.6465 shutdown 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 msgr2=0x55ba947b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba947b19d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff1740090b0 tx=0x7ff174009240 comp rx=0 tx=0).stop 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 shutdown_connections 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55ba9475bfd0 0x55ba946ec390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ba9471e940 0x55ba9475b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9473ac30 0x55ba9479ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 --2- 172.21.15.59:0/186977318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba9471dd50 0x55ba947b19d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff166000640 0 client.6465 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 >> 172.21.15.59:0/186977318 conn(0x55ba9471d610 msgr2=0x55ba94738200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 shutdown_connections 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.863+0000 7ff18ab83740 1 -- 172.21.15.59:0/186977318 wait complete. 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff18ab83740 20 client.6465 trim_cache size 0 max 16384 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "58a57886-6808-4c83-a0a6-bcdf67f58a72": { 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "58a57886-6808-4c83-a0a6-bcdf67f58a72", 2024-04-23T14:35:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair,force" 2024-04-23T14:35:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:35:36.868 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:35:36.868 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:35:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff16006c8f0 msgr2=0x7ff16006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff16006c8f0 0x7ff16006edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff17402e000 comp rx=0 tx=0).stop 2024-04-23T14:35:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 msgr2=0x7ff1800cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:35:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff1800cb510 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff17800b9a0 tx=0x7ff17800be60 comp rx=0 tx=0).stop 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 shutdown_connections 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff16006c8f0 0x7ff16006edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff18006cf60 0x7ff1800cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff18009aa00 0x7ff1800cb510 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 --2- 172.21.15.59:0/2281689157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff18006cae0 0x7ff1800cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.864+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 >> 172.21.15.59:0/2281689157 conn(0x7ff1800b4090 msgr2=0x7ff180158990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.865+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 shutdown_connections 2024-04-23T14:35:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:35:36.865+0000 7ff187e00640 1 -- 172.21.15.59:0/2281689157 wait complete. 2024-04-23T14:35:36.882 INFO:tasks.cephfs.filesystem:scrub status for tag:58a57886-6808-4c83-a0a6-bcdf67f58a72 - {'path': '/', 'tag': '58a57886-6808-4c83-a0a6-bcdf67f58a72', 'options': 'recursive,repair,force'} 2024-04-23T14:35:49.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:35:49.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:36:06.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:06.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-23T14:36:07.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.115+0000 7fa449400640 1 Processor -- start 2024-04-23T14:36:07.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.115+0000 7fa449400640 1 -- start start 2024-04-23T14:36:07.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.117+0000 7fa449400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4440d0ce0 0x7fa4440af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.117+0000 7fa449400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4440af5c0 0x7fa444071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.117+0000 7fa449400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa444160120 con 0x7fa44415c410 2024-04-23T14:36:07.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.117+0000 7fa449400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa444160400 con 0x7fa4440af5c0 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.117+0000 7fa449400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa4441606e0 con 0x7fa4440d0ce0 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa448a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4440af5c0 0x7fa444071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa443e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa44415c410 0x7fa4440d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56130/0 (socket says 172.21.15.59:56130) 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa443e00640 1 -- 172.21.15.59:0/1855584730 learned_addr learned my addr 172.21.15.59:0/1855584730 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa443400640 1 --2- 172.21.15.59:0/1855584730 >> v2:172.21.15.76:3300/0 conn(0x7fa4440d0ce0 0x7fa4440af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa448a00640 1 -- 172.21.15.59:0/1855584730 >> v2:172.21.15.76:3300/0 conn(0x7fa4440d0ce0 msgr2=0x7fa4440af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa448a00640 1 --2- 172.21.15.59:0/1855584730 >> v2:172.21.15.76:3300/0 conn(0x7fa4440d0ce0 0x7fa4440af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa448a00640 1 -- 172.21.15.59:0/1855584730 >> v1:172.21.15.59:6789/0 conn(0x7fa44415c410 legacy=0x7fa4440d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.119+0000 7fa448a00640 1 -- 172.21.15.59:0/1855584730 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4441609c0 con 0x7fa4440af5c0 2024-04-23T14:36:07.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.118+0000 7fa442a00640 1 -- 172.21.15.59:0/1855584730 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 21343124 0 0) 0x7fa444160120 con 0x7fa44415c410 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.120+0000 7fa448a00640 1 --2- 172.21.15.59:0/1855584730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440af5c0 0x7fa444071c70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa4440b4560 tx=0x7fa43402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74dc1d236a8b56e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.120+0000 7fa442a00640 1 -- 172.21.15.59:0/1855584730 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434037070 con 0x7fa4440af5c0 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.120+0000 7fa442a00640 1 -- 172.21.15.59:0/1855584730 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa434032440 con 0x7fa4440af5c0 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.120+0000 7fa442a00640 1 -- 172.21.15.59:0/1855584730 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43402ae60 con 0x7fa4440af5c0 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 -- 172.21.15.59:0/1855584730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440af5c0 msgr2=0x7fa444071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 --2- 172.21.15.59:0/1855584730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440af5c0 0x7fa444071c70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa4440b4560 tx=0x7fa43402a110 comp rx=0 tx=0).stop 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 -- 172.21.15.59:0/1855584730 shutdown_connections 2024-04-23T14:36:07.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 --2- 172.21.15.59:0/1855584730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440af5c0 0x7fa444071c70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 --2- 172.21.15.59:0/1855584730 >> v2:172.21.15.76:3300/0 conn(0x7fa4440d0ce0 0x7fa4440af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 -- 172.21.15.59:0/1855584730 >> 172.21.15.59:0/1855584730 conn(0x7fa44406bdf0 msgr2=0x7fa44406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 -- 172.21.15.59:0/1855584730 shutdown_connections 2024-04-23T14:36:07.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.121+0000 7fa449400640 1 -- 172.21.15.59:0/1855584730 wait complete. 2024-04-23T14:36:07.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.123+0000 7fa449400640 1 Processor -- start 2024-04-23T14:36:07.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.123+0000 7fa449400640 1 -- start start 2024-04-23T14:36:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa449400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4440af5c0 0x7fa4441565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa449400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4440d0ce0 0x7fa44415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa449400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa44415c410 0x7fa44415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa449400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440afb80 con 0x7fa44415c410 2024-04-23T14:36:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa449400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440afa00 con 0x7fa4440af5c0 2024-04-23T14:36:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa449400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa4440721b0 con 0x7fa4440d0ce0 2024-04-23T14:36:07.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa448a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa44415c410 0x7fa44415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa448a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa44415c410 0x7fa44415b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55284/0 (socket says 172.21.15.59:55284) 2024-04-23T14:36:07.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa448a00640 1 -- 172.21.15.59:0/4139413460 learned_addr learned my addr 172.21.15.59:0/4139413460 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa443e00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4440af5c0 0x7fa4441565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.124+0000 7fa448a00640 1 -- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4440d0ce0 msgr2=0x7fa44415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa443400640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4440d0ce0 0x7fa44415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa448a00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4440d0ce0 0x7fa44415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa448a00640 1 -- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4440af5c0 msgr2=0x7fa4441565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa448a00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4440af5c0 0x7fa4441565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa448a00640 1 -- 172.21.15.59:0/4139413460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa444157020 con 0x7fa44415c410 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa443e00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4440af5c0 0x7fa4441565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa448a00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa44415c410 0x7fa44415b0c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa434009890 tx=0x7fa4340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa43be00640 1 -- 172.21.15.59:0/4139413460 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434037040 con 0x7fa44415c410 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa43be00640 1 -- 172.21.15.59:0/4139413460 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa434042070 con 0x7fa44415c410 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa444157300 con 0x7fa44415c410 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa43be00640 1 -- 172.21.15.59:0/4139413460 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434030b30 con 0x7fa44415c410 2024-04-23T14:36:07.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.125+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa444156ae0 con 0x7fa44415c410 2024-04-23T14:36:07.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.128+0000 7fa43be00640 1 -- 172.21.15.59:0/4139413460 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa434030cd0 con 0x7fa44415c410 2024-04-23T14:36:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.129+0000 7fa43be00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa42006c920 0x7fa42006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.130+0000 7fa43be00640 1 -- 172.21.15.59:0/4139413460 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa4340ac6b0 con 0x7fa44415c410 2024-04-23T14:36:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.130+0000 7fa443e00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa42006c920 0x7fa42006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.130+0000 7fa44d81d740 1 Processor -- start 2024-04-23T14:36:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.130+0000 7fa44d81d740 1 -- start start 2024-04-23T14:36:07.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa443e00640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa42006c920 0x7fa42006ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007fd0 tx=0x7fa43c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c2392790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 0x55e6c23c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23c27e0 0x55e6c23975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa44d81d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e6c22fc1f0 con 0x55e6c23dfe00 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e6c22a2350 con 0x55e6c23c27e0 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e6c22f81b0 con 0x55e6c23c2f20 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.131+0000 7fa443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c2392790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c2392790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55294/0 (socket says 172.21.15.59:55294) 2024-04-23T14:36:07.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 -- 172.21.15.59:0/2602661271 learned_addr learned my addr 172.21.15.59:0/2602661271 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443400640 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 0x55e6c23c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 -- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 msgr2=0x55e6c23c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 0x55e6c23c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 -- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23c27e0 msgr2=0x55e6c23975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa448a00640 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23c27e0 0x55e6c23975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23c27e0 0x55e6c23975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 -- 172.21.15.59:0/2602661271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c22f6f50 con 0x55e6c23dfe00 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.132+0000 7fa443e00640 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c2392790 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00d8a0 tx=0x7fa43c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d35836173e8a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.133+0000 7fa433e00640 1 -- 172.21.15.59:0/2602661271 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c007850 con 0x55e6c23dfe00 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.133+0000 7fa433e00640 1 -- 172.21.15.59:0/2602661271 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c020450 con 0x55e6c23dfe00 2024-04-23T14:36:07.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.133+0000 7fa433e00640 1 -- 172.21.15.59:0/2602661271 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0079f0 con 0x55e6c23dfe00 2024-04-23T14:36:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.133+0000 7fa44d81d740 1 -- 172.21.15.59:0/2602661271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 msgr2=0x55e6c2392790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.133+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c2392790 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00d8a0 tx=0x7fa43c010000 comp rx=0 tx=0).stop 2024-04-23T14:36:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 -- 172.21.15.59:0/2602661271 shutdown_connections 2024-04-23T14:36:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 0x55e6c23c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23c27e0 0x55e6c23975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2602661271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c2392790 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 -- 172.21.15.59:0/2602661271 >> 172.21.15.59:0/2602661271 conn(0x55e6c2394a00 msgr2=0x55e6c23dd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 -- 172.21.15.59:0/2602661271 shutdown_connections 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 -- 172.21.15.59:0/2602661271 wait complete. 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.134+0000 7fa44d81d740 1 -- start start 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.135+0000 7fa44d81d740 1 Processor -- start 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.135+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.135+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 0x55e6c23f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa44d81d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e6c2396bb0 con 0x55e6c23c2f20 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e6c22b5690 con 0x55e6c23dfe00 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e6c2395fa0 con 0x55e6c23de910 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa44d81d740 10 client.0 _ll_register_callbacks cb 0x55e6c22f1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55310/0 (socket says 172.21.15.59:55310) 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 -- 172.21.15.59:0/1742841408 learned_addr learned my addr 172.21.15.59:0/1742841408 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443400640 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 -- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 msgr2=0x55e6c23f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 0x55e6c23f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 -- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 msgr2=0x55e6c23de3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23de3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443e00640 1 -- 172.21.15.59:0/1742841408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c22b4920 con 0x55e6c23c2f20 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.136+0000 7fa443400640 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23de3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa443e00640 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23ac590 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0074a0 tx=0x7fa43c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cab1a17d7fb055 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa430c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa430c00640 1 -- 172.21.15.59:0/1742841408 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0222e0 con 0x55e6c23c2f20 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa430c00640 1 -- 172.21.15.59:0/1742841408 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa43c022a00 con 0x55e6c23c2f20 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43c020450 con 0x55e6c23c2f20 2024-04-23T14:36:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa430c00640 1 -- 172.21.15.59:0/1742841408 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa43c034050 con 0x55e6c23c2f20 2024-04-23T14:36:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa44d81d740 10 client.6480 fetch_fsmap learned FSMap version 35 2024-04-23T14:36:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55e6c20d87d0 con 0x55e6c23c2f20 2024-04-23T14:36:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.137+0000 7fa430c00640 1 -- 172.21.15.59:0/1742841408 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c020450 con 0x55e6c23c2f20 2024-04-23T14:36:07.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa430c00640 1 -- 172.21.15.59:0/1742841408 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fa43c037020 con 0x55e6c23c2f20 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa44d81d740 10 client.6480 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa44d81d740 10 client.6480 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa44d81d740 20 client.6480 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa44d81d740 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23d4430 0x55e6c23d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa44d81d740 4 client.6480 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6c2309c10 con 0x55e6c23d4430 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.138+0000 7fa443400640 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23d4430 0x55e6c23d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.139+0000 7fa443400640 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23d4430 0x55e6c23d4870 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11c54cdf1995488 server_cookie=4e4fffe1131170f7 in_seq=0 out_seq=0 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.139+0000 7fa430c00640 10 client.6480 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.139+0000 7fa430c00640 1 -- 172.21.15.59:0/1742841408 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6c2309c10 con 0x55e6c23d4430 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.139+0000 7fa430c00640 10 client.6480 handle_command_reply: tid=0 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 client.6480 shutdown 2024-04-23T14:36:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 msgr2=0x55e6c23ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23ac590 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0074a0 tx=0x7fa43c012000 comp rx=0 tx=0).stop 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 shutdown_connections 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23d4430 0x55e6c23d4870 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 0x55e6c23f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23de3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 --2- 172.21.15.59:0/1742841408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23ac590 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa430c00640 0 client.6480 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 >> 172.21.15.59:0/1742841408 conn(0x55e6c23c27e0 msgr2=0x55e6c23e06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 shutdown_connections 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 1 -- 172.21.15.59:0/1742841408 wait complete. 2024-04-23T14:36:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.140+0000 7fa44d81d740 20 client.6480 trim_cache size 0 max 16384 2024-04-23T14:36:07.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.160+0000 7fa44d81d740 1 Processor -- start 2024-04-23T14:36:07.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.160+0000 7fa44d81d740 1 -- start start 2024-04-23T14:36:07.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.160+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 0x55e6c23d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.160+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.160+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa44d81d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa444200b30 con 0x55e6c23dfe00 2024-04-23T14:36:07.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4441ffb90 con 0x55e6c23de910 2024-04-23T14:36:07.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa44415e240 con 0x55e6c23c2f20 2024-04-23T14:36:07.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa448a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa448a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55316/0 (socket says 172.21.15.59:55316) 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa448a00640 1 -- 172.21.15.59:0/2872147936 learned_addr learned my addr 172.21.15.59:0/2872147936 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa448a00640 1 -- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 msgr2=0x55e6c23d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa448a00640 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 0x55e6c23d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa443400640 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa448a00640 1 -- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23de910 msgr2=0x55e6c23e06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.161+0000 7fa448a00640 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.162+0000 7fa448a00640 1 -- 172.21.15.59:0/2872147936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c200e3f0 con 0x55e6c23dfe00 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.162+0000 7fa443400640 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.162+0000 7fa448a00640 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa43407a2d0 tx=0x7fa43407a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d698389bed16cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.162+0000 7fa433400640 1 -- 172.21.15.59:0/2872147936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4340a9070 con 0x55e6c23dfe00 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.162+0000 7fa433400640 1 -- 172.21.15.59:0/2872147936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43407aa40 con 0x55e6c23dfe00 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.162+0000 7fa433400640 1 -- 172.21.15.59:0/2872147936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43407a340 con 0x55e6c23dfe00 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 -- 172.21.15.59:0/2872147936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 msgr2=0x55e6c23e0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa43407a2d0 tx=0x7fa43407a1d0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 -- 172.21.15.59:0/2872147936 shutdown_connections 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 --2- 172.21.15.59:0/2872147936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23c2f20 0x55e6c23d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 -- 172.21.15.59:0/2872147936 >> 172.21.15.59:0/2872147936 conn(0x55e6c23c27e0 msgr2=0x55e6c238de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 -- 172.21.15.59:0/2872147936 shutdown_connections 2024-04-23T14:36:07.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 -- 172.21.15.59:0/2872147936 wait complete. 2024-04-23T14:36:07.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.163+0000 7fa44d81d740 1 -- start start 2024-04-23T14:36:07.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.164+0000 7fa44d81d740 1 Processor -- start 2024-04-23T14:36:07.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa44d81d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa44d81d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e6c22a1790 con 0x55e6c23c2f20 2024-04-23T14:36:07.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440cdf70 con 0x55e6c23dfe00 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa44d81d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e6c22f8330 con 0x55e6c23de910 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa44d81d740 10 client.0 _ll_register_callbacks cb 0x7fa43c007a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23bd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55318/0 (socket says 172.21.15.59:55318) 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443e00640 1 -- 172.21.15.59:0/132862729 learned_addr learned my addr 172.21.15.59:0/132862729 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa448a00640 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443e00640 1 -- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 msgr2=0x55e6c23e06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443400640 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443e00640 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443e00640 1 -- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 msgr2=0x55e6c23e0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.165+0000 7fa443e00640 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa443e00640 1 -- 172.21.15.59:0/132862729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c1fce0f0 con 0x55e6c23c2f20 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa443e00640 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23bd7f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0016e0 tx=0x7fa43c001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8efb92d81108a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa430c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:07.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa430c00640 1 -- 172.21.15.59:0/132862729 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c02c670 con 0x55e6c23c2f20 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa430c00640 1 -- 172.21.15.59:0/132862729 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa43c02c810 con 0x55e6c23c2f20 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43407aa40 con 0x55e6c23c2f20 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa430c00640 1 -- 172.21.15.59:0/132862729 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa43c02ca50 con 0x55e6c23c2f20 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa44d81d740 10 client.6486 fetch_fsmap learned FSMap version 35 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.166+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55e6c22c1810 con 0x55e6c23c2f20 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa430c00640 1 -- 172.21.15.59:0/132862729 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43407aa40 con 0x55e6c23c2f20 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa430c00640 1 -- 172.21.15.59:0/132862729 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fa43c037040 con 0x55e6c23c2f20 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa44d81d740 10 client.6486 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa44d81d740 10 client.6486 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa44d81d740 20 client.6486 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa44d81d740 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23c4130 0x55e6c24535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa44d81d740 4 client.6486 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6c2303a80 con 0x55e6c23c4130 2024-04-23T14:36:07.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.167+0000 7fa443400640 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23c4130 0x55e6c24535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa443400640 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23c4130 0x55e6c24535b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d12dd956edf8e4 server_cookie=64b72119d2e14d6c in_seq=0 out_seq=0 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa430c00640 10 client.6486 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa430c00640 1 -- 172.21.15.59:0/132862729 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e6c2303a80 con 0x55e6c23c4130 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa430c00640 10 client.6486 handle_command_reply: tid=0 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 client.6486 shutdown 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 msgr2=0x55e6c23bd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23bd7f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0016e0 tx=0x7fa43c001400 comp rx=0 tx=0).stop 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 shutdown_connections 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x55e6c23c4130 0x55e6c24535b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e6c23de910 0x55e6c23e06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e6c23dfe00 0x55e6c23e0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 --2- 172.21.15.59:0/132862729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c23c2f20 0x55e6c23bd7f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa430c00640 0 client.6486 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 >> 172.21.15.59:0/132862729 conn(0x55e6c23c27e0 msgr2=0x55e6c23d5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 shutdown_connections 2024-04-23T14:36:07.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.168+0000 7fa44d81d740 1 -- 172.21.15.59:0/132862729 wait complete. 2024-04-23T14:36:07.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa44d81d740 20 client.6486 trim_cache size 0 max 16384 2024-04-23T14:36:07.173 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:36:07.173 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:36:07.173 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:36:07.173 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa42006c920 msgr2=0x7fa42006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa42006c920 0x7fa42006ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007fd0 tx=0x7fa43c009040 comp rx=0 tx=0).stop 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa44415c410 msgr2=0x7fa44415b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa44415c410 0x7fa44415b0c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa434009890 tx=0x7fa4340046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 shutdown_connections 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa42006c920 0x7fa42006ede0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4440d0ce0 0x7fa44415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa44415c410 0x7fa44415b0c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 --2- 172.21.15.59:0/4139413460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4440af5c0 0x7fa4441565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 >> 172.21.15.59:0/4139413460 conn(0x7fa44406bdf0 msgr2=0x7fa44406f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 shutdown_connections 2024-04-23T14:36:07.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.169+0000 7fa449400640 1 -- 172.21.15.59:0/4139413460 wait complete. 2024-04-23T14:36:07.188 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:36:07.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.394+0000 7f0a36e00640 1 Processor -- start 2024-04-23T14:36:07.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.395+0000 7f0a36e00640 1 -- start start 2024-04-23T14:36:07.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.396+0000 7f0a36e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a3015e7c0 0x7f0a300d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.396+0000 7f0a36e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a300d0d90 0x7f0a300d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.396+0000 7f0a36e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a300cf030 con 0x7f0a300d2630 2024-04-23T14:36:07.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.396+0000 7f0a36e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0a300b7000 con 0x7f0a3015e7c0 2024-04-23T14:36:07.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.396+0000 7f0a36e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0a300cf620 con 0x7f0a300d0d90 2024-04-23T14:36:07.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a35a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a3015e7c0 0x7f0a300d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a35a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0a3015e7c0 0x7f0a300d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55330/0 (socket says 172.21.15.59:55330) 2024-04-23T14:36:07.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a35a00640 1 -- 172.21.15.59:0/1330320961 learned_addr learned my addr 172.21.15.59:0/1330320961 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a36400640 1 --2- 172.21.15.59:0/1330320961 >> v2:172.21.15.76:3300/0 conn(0x7f0a300d0d90 0x7f0a300d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a35a00640 1 -- 172.21.15.59:0/1330320961 >> v2:172.21.15.76:3300/0 conn(0x7f0a300d0d90 msgr2=0x7f0a300d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a35a00640 1 --2- 172.21.15.59:0/1330320961 >> v2:172.21.15.76:3300/0 conn(0x7f0a300d0d90 0x7f0a300d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a35a00640 1 -- 172.21.15.59:0/1330320961 >> v1:172.21.15.76:6789/0 conn(0x7f0a300d2630 legacy=0x7f0a300b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a35a00640 1 -- 172.21.15.59:0/1330320961 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a300b6a40 con 0x7f0a3015e7c0 2024-04-23T14:36:07.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.397+0000 7f0a36400640 1 --2- 172.21.15.59:0/1330320961 >> v2:172.21.15.76:3300/0 conn(0x7f0a300d0d90 0x7f0a300d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:07.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.398+0000 7f0a35a00640 1 --2- 172.21.15.59:0/1330320961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300d20f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0a300cb580 tx=0x7f0a2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eb7d800b4b733f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.399+0000 7f0a2fe00640 1 -- 172.21.15.59:0/1330320961 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24037070 con 0x7f0a3015e7c0 2024-04-23T14:36:07.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.399+0000 7f0a2fe00640 1 -- 172.21.15.59:0/1330320961 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a24032440 con 0x7f0a3015e7c0 2024-04-23T14:36:07.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.399+0000 7f0a2fe00640 1 -- 172.21.15.59:0/1330320961 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24032810 con 0x7f0a3015e7c0 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.399+0000 7f0a36e00640 1 -- 172.21.15.59:0/1330320961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 msgr2=0x7f0a300d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.399+0000 7f0a36e00640 1 --2- 172.21.15.59:0/1330320961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300d20f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0a300cb580 tx=0x7f0a2402a230 comp rx=0 tx=0).stop 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.400+0000 7f0a36e00640 1 -- 172.21.15.59:0/1330320961 shutdown_connections 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.400+0000 7f0a36e00640 1 --2- 172.21.15.59:0/1330320961 >> v2:172.21.15.76:3300/0 conn(0x7f0a300d0d90 0x7f0a300d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.400+0000 7f0a36e00640 1 --2- 172.21.15.59:0/1330320961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300d20f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.400+0000 7f0a36e00640 1 -- 172.21.15.59:0/1330320961 >> 172.21.15.59:0/1330320961 conn(0x7f0a3009d920 msgr2=0x7f0a3008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.400+0000 7f0a36e00640 1 -- 172.21.15.59:0/1330320961 shutdown_connections 2024-04-23T14:36:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.400+0000 7f0a36e00640 1 -- 172.21.15.59:0/1330320961 wait complete. 2024-04-23T14:36:07.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.402+0000 7f0a36e00640 1 Processor -- start 2024-04-23T14:36:07.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.402+0000 7f0a36e00640 1 -- start start 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.403+0000 7f0a36e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a300d0d90 0x7f0a301f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.403+0000 7f0a36e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a300d2630 0x7f0a300cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.403+0000 7f0a36e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.403+0000 7f0a36e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300cf2b0 con 0x7f0a3015e7c0 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.403+0000 7f0a36e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300d1700 con 0x7f0a300d0d90 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.403+0000 7f0a36e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0a300b6700 con 0x7f0a300d2630 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55334/0 (socket says 172.21.15.59:55334) 2024-04-23T14:36:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 -- 172.21.15.59:0/102118621 learned_addr learned my addr 172.21.15.59:0/102118621 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a35a00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a300d0d90 0x7f0a301f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a35000640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a300d2630 0x7f0a300cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 -- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a300d2630 msgr2=0x7f0a300cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a300d2630 0x7f0a300cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 -- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a300d0d90 msgr2=0x7f0a301f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a300d0d90 0x7f0a301f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a36400640 1 -- 172.21.15.59:0/102118621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a301f21d0 con 0x7f0a3015e7c0 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.404+0000 7f0a35000640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a300d2630 0x7f0a300cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.405+0000 7f0a36400640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300cbb30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800cd50 tx=0x7f0a2800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.405+0000 7f0a2d600640 1 -- 172.21.15.59:0/102118621 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800f680 con 0x7f0a3015e7c0 2024-04-23T14:36:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.405+0000 7f0a2d600640 1 -- 172.21.15.59:0/102118621 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2800f840 con 0x7f0a3015e7c0 2024-04-23T14:36:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.405+0000 7f0a2d600640 1 -- 172.21.15.59:0/102118621 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800fc30 con 0x7f0a3015e7c0 2024-04-23T14:36:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.405+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a300cc170 con 0x7f0a3015e7c0 2024-04-23T14:36:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.405+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a300cc730 con 0x7f0a3015e7c0 2024-04-23T14:36:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.406+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3004fd90 con 0x7f0a3015e7c0 2024-04-23T14:36:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.407+0000 7f0a2d600640 1 -- 172.21.15.59:0/102118621 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a280213f0 con 0x7f0a3015e7c0 2024-04-23T14:36:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.409+0000 7f0a2d600640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a1406c8f0 0x7f0a1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.409+0000 7f0a2d600640 1 -- 172.21.15.59:0/102118621 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0a2808c8d0 con 0x7f0a3015e7c0 2024-04-23T14:36:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.409+0000 7f0a35a00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a1406c8f0 0x7f0a1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.410+0000 7f0a35a00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a1406c8f0 0x7f0a1406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a24034000 tx=0x7f0a2402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.414+0000 7f0a2d600640 1 -- 172.21.15.59:0/102118621 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a28014030 con 0x7f0a3015e7c0 2024-04-23T14:36:07.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.657+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a300d23c0 con 0x7f0a3015e7c0 2024-04-23T14:36:07.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.658+0000 7f0a2d600640 1 -- 172.21.15.59:0/102118621 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5182 (secure 0 0 0) 0x7f0a2805acf0 con 0x7f0a3015e7c0 2024-04-23T14:36:07.660 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:07.660 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:35:37.778594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:36:07.660 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a1406c8f0 msgr2=0x7f0a1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a1406c8f0 0x7f0a1406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a24034000 tx=0x7f0a2402e000 comp rx=0 tx=0).stop 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 msgr2=0x7f0a300cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300cbb30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800cd50 tx=0x7f0a2800a650 comp rx=0 tx=0).stop 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 shutdown_connections 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a1406c8f0 0x7f0a1406edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0a300d2630 0x7f0a300cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0a3015e7c0 0x7f0a300cbb30 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 --2- 172.21.15.59:0/102118621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a300d0d90 0x7f0a301f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.661+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 >> 172.21.15.59:0/102118621 conn(0x7f0a3009d920 msgr2=0x7f0a3015fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.662+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 shutdown_connections 2024-04-23T14:36:07.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.662+0000 7f0a36e00640 1 -- 172.21.15.59:0/102118621 wait complete. 2024-04-23T14:36:07.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-23T14:36:07.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.896+0000 7f10d3e00640 1 Processor -- start 2024-04-23T14:36:07.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.897+0000 7f10d3e00640 1 -- start start 2024-04-23T14:36:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.898+0000 7f10d3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 0x7f10cc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.898+0000 7f10d3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0cfc20 con 0x7f10cc071d70 2024-04-23T14:36:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.898+0000 7f10d3e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0cff00 con 0x7f10cc145e10 2024-04-23T14:36:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.898+0000 7f10d3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc0d01e0 con 0x7f10cc0d0cc0 2024-04-23T14:36:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.899+0000 7f10d1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f10cc071d70 0x7f10cc15cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56136/0 (socket says 172.21.15.59:56136) 2024-04-23T14:36:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.899+0000 7f10d1e00640 1 -- 172.21.15.59:0/1450984466 learned_addr learned my addr 172.21.15.59:0/1450984466 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:07.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.899+0000 7f10d0a00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 0x7f10cc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.899+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407880029 0 0) 0x7f10cc0cfc20 con 0x7f10cc071d70 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.899+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10b4003520 con 0x7f10cc071d70 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 968522932 0 0) 0x7f10b4003520 con 0x7f10cc071d70 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 msgr2=0x7f10cc06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 0x7f10cc06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 >> v1:172.21.15.76:6789/0 conn(0x7f10cc145e10 legacy=0x7f10cc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc0d04c0 con 0x7f10cc071d70 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f10c40019f0 con 0x7f10cc071d70 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 >> v1:172.21.15.59:6789/0 conn(0x7f10cc071d70 legacy=0x7f10cc15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b40039e0 0x7f10b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 0x7f10b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b4004d40 0x7f10b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0d01e0 con 0x7f10b4004d40 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10b400baa0 con 0x7f10b40039e0 2024-04-23T14:36:07.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10b400bca0 con 0x7f10cc0d0cc0 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.900+0000 7f10d1e00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b4004d40 0x7f10b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.901+0000 7f10d0a00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 0x7f10b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.901+0000 7f10d1400640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b40039e0 0x7f10b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.901+0000 7f10d1e00640 1 -- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 msgr2=0x7f10b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.901+0000 7f10d1e00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10cc0d0cc0 0x7f10b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.901+0000 7f10d1e00640 1 -- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b40039e0 msgr2=0x7f10b4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.901+0000 7f10d1e00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b40039e0 0x7f10b4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.901+0000 7f10d1e00640 1 -- 172.21.15.59:0/1450984466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b4003520 con 0x7f10b4004d40 2024-04-23T14:36:07.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.902+0000 7f10d1e00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b4004d40 0x7f10b40051c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a810 tx=0x7f10b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a985a0e061f61d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.902+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c400a460 con 0x7f10b4004d40 2024-04-23T14:36:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.902+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c402f070 con 0x7f10b4004d40 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.902+0000 7f10cbe00640 1 -- 172.21.15.59:0/1450984466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c402bc90 con 0x7f10b4004d40 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.903+0000 7f10d3e00640 1 -- 172.21.15.59:0/1450984466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b4004d40 msgr2=0x7f10b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.903+0000 7f10d3e00640 1 --2- 172.21.15.59:0/1450984466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10b4004d40 0x7f10b40051c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a810 tx=0x7f10b4005830 comp rx=0 tx=0).stop 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.903+0000 7f10d1e00640 1 -- 172.21.15.59:0/1450984466 reap_dead start 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.903+0000 7f10d3e00640 1 -- 172.21.15.59:0/1450984466 shutdown_connections 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.903+0000 7f10d3e00640 1 -- 172.21.15.59:0/1450984466 >> 172.21.15.59:0/1450984466 conn(0x7f10cc067d90 msgr2=0x7f10cc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.903+0000 7f10d3e00640 1 -- 172.21.15.59:0/1450984466 shutdown_connections 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.903+0000 7f10d3e00640 1 -- 172.21.15.59:0/1450984466 wait complete. 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.905+0000 7f10d3e00640 1 Processor -- start 2024-04-23T14:36:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.905+0000 7f10d3e00640 1 -- start start 2024-04-23T14:36:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10b4004d40 0x7f10cc130ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc0cb820 0x7f10cc0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10cc1e7540 0x7f10cc1e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1461f0 con 0x7f10cc0cb820 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc06f130 con 0x7f10cc1e7540 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc12ecf0 con 0x7f10b4004d40 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc0cb820 0x7f10cc0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc0cb820 0x7f10cc0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55352/0 (socket says 172.21.15.59:55352) 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d0a00640 1 -- 172.21.15.59:0/4049277868 learned_addr learned my addr 172.21.15.59:0/4049277868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.906+0000 7f10d1400640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10b4004d40 0x7f10cc130ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d1e00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10cc1e7540 0x7f10cc1e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d0a00640 1 -- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10b4004d40 msgr2=0x7f10cc130ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d0a00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10b4004d40 0x7f10cc130ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d0a00640 1 -- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10cc1e7540 msgr2=0x7f10cc1e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d0a00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10cc1e7540 0x7f10cc1e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d0a00640 1 -- 172.21.15.59:0/4049277868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc1312c0 con 0x7f10cc0cb820 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d0a00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc0cb820 0x7f10cc0cd930 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f10b8011600 tx=0x7f10b8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10c9600640 1 -- 172.21.15.59:0/4049277868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b800f8c0 con 0x7f10cc0cb820 2024-04-23T14:36:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10c9600640 1 -- 172.21.15.59:0/4049277868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b800fa60 con 0x7f10cc0cb820 2024-04-23T14:36:07.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10c9600640 1 -- 172.21.15.59:0/4049277868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b800fe10 con 0x7f10cc0cb820 2024-04-23T14:36:07.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.907+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc1315a0 con 0x7f10cc0cb820 2024-04-23T14:36:07.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.908+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc131b60 con 0x7f10cc0cb820 2024-04-23T14:36:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.910+0000 7f10c9600640 1 -- 172.21.15.59:0/4049277868 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10b8015460 con 0x7f10cc0cb820 2024-04-23T14:36:07.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.911+0000 7f10c9600640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10ac06c8f0 0x7f10ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.911+0000 7f10c9600640 1 -- 172.21.15.59:0/4049277868 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f10b808c500 con 0x7f10cc0cb820 2024-04-23T14:36:07.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.911+0000 7f10d1400640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10ac06c8f0 0x7f10ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.911+0000 7f10d6c39740 1 Processor -- start 2024-04-23T14:36:07.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.912+0000 7f10d6c39740 1 -- start start 2024-04-23T14:36:07.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.912+0000 7f10d1400640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10ac06c8f0 0x7f10ac06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007fd0 tx=0x7f10c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.912+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e849dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.912+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849f9890 0x556e849f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849b1cd0 0x556e849f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d6c39740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e84916190 con 0x556e849dbff0 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556e848bc030 con 0x556e849f9890 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556e84912150 con 0x556e849b1cd0 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e849dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e849dc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55364/0 (socket says 172.21.15.59:55364) 2024-04-23T14:36:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 -- 172.21.15.59:0/928466210 learned_addr learned my addr 172.21.15.59:0/928466210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d0a00640 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849f9890 0x556e849f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 -- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849b1cd0 msgr2=0x556e849f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1e00640 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849b1cd0 0x556e849f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849b1cd0 0x556e849f7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 -- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849f9890 msgr2=0x556e849f9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849f9890 0x556e849f9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.913+0000 7f10d1400640 1 -- 172.21.15.59:0/928466210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e848a4b60 con 0x556e849dbff0 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.914+0000 7f10d1400640 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e849dc3d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f10c00015b0 tx=0x7f10c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb5022b6a6f974f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.914+0000 7f10bdc00640 1 -- 172.21.15.59:0/928466210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0027070 con 0x556e849dbff0 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.914+0000 7f10bdc00640 1 -- 172.21.15.59:0/928466210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0014740 con 0x556e849dbff0 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.914+0000 7f10bdc00640 1 -- 172.21.15.59:0/928466210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0014af0 con 0x556e849dbff0 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 -- 172.21.15.59:0/928466210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 msgr2=0x556e849dc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e849dc3d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f10c00015b0 tx=0x7f10c0013000 comp rx=0 tx=0).stop 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 -- 172.21.15.59:0/928466210 shutdown_connections 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849b1cd0 0x556e849f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849f9890 0x556e849f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 --2- 172.21.15.59:0/928466210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e849dc3d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 -- 172.21.15.59:0/928466210 >> 172.21.15.59:0/928466210 conn(0x556e849ae600 msgr2=0x556e849f6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 -- 172.21.15.59:0/928466210 shutdown_connections 2024-04-23T14:36:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 -- 172.21.15.59:0/928466210 wait complete. 2024-04-23T14:36:07.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.915+0000 7f10d6c39740 1 -- start start 2024-04-23T14:36:07.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.916+0000 7f10d6c39740 1 Processor -- start 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e84a07330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849f9890 0x556e849ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e849dada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d6c39740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e849b2a70 con 0x556e849dbff0 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556e848cf500 con 0x556e849da940 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556e849ac530 con 0x556e849f9890 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d6c39740 10 client.0 _ll_register_callbacks cb 0x556e8490c270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e84a07330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e84a07330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55372/0 (socket says 172.21.15.59:55372) 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 -- 172.21.15.59:0/2621260318 learned_addr learned my addr 172.21.15.59:0/2621260318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d0a00640 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849f9890 0x556e849ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 -- 172.21.15.59:0/2621260318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849f9890 msgr2=0x556e849ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849f9890 0x556e849ef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 -- 172.21.15.59:0/2621260318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 msgr2=0x556e849dada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e849dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.917+0000 7f10d1400640 1 -- 172.21.15.59:0/2621260318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e846f47c0 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10d1400640 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e84a07330 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f10c0003e50 tx=0x7f10c0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dd33aad7622d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10b2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10b2000640 1 -- 172.21.15.59:0/2621260318 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0027070 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10b2000640 1 -- 172.21.15.59:0/2621260318 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f10c0021830 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c0014740 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10b2000640 1 -- 172.21.15.59:0/2621260318 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f10c003a050 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10d6c39740 10 client.6504 fetch_fsmap learned FSMap version 35 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.918+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556e84754f50 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10b2000640 1 -- 172.21.15.59:0/2621260318 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0014740 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10b2000640 1 -- 172.21.15.59:0/2621260318 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f10c003d020 con 0x556e849dbff0 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10d6c39740 10 client.6504 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10d6c39740 10 client.6504 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10d6c39740 20 client.6504 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:07.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a131b0 0x556e84a155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10d6c39740 4 client.6504 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:36:07.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e84923bb0 con 0x556e84a131b0 2024-04-23T14:36:07.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.919+0000 7f10d0a00640 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a131b0 0x556e84a155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.920+0000 7f10b2000640 10 client.6504 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.920+0000 7f10d0a00640 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a131b0 0x556e84a155a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2c3275609b87c0 server_cookie=2097bf6292cd5eb4 in_seq=0 out_seq=0 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.920+0000 7f10b2000640 1 -- 172.21.15.59:0/2621260318 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e84923bb0 con 0x556e84a131b0 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.920+0000 7f10b2000640 10 client.6504 handle_command_reply: tid=0 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.920+0000 7f10d6c39740 1 client.6504 shutdown 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 msgr2=0x556e84a07330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e84a07330 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f10c0003e50 tx=0x7f10c0001190 comp rx=0 tx=0).stop 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 shutdown_connections 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a131b0 0x556e84a155a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849f9890 0x556e849ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e849dada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2621260318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849dbff0 0x556e84a07330 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 >> 172.21.15.59:0/2621260318 conn(0x556e849b1cd0 msgr2=0x556e849fac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10b2000640 0 client.6504 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 shutdown_connections 2024-04-23T14:36:07.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 1 -- 172.21.15.59:0/2621260318 wait complete. 2024-04-23T14:36:07.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.921+0000 7f10d6c39740 20 client.6504 trim_cache size 0 max 16384 2024-04-23T14:36:07.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.936+0000 7f10d6c39740 1 Processor -- start 2024-04-23T14:36:07.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.936+0000 7f10d6c39740 1 -- start start 2024-04-23T14:36:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e849f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e84a10060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e849ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d6c39740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0b4d90 con 0x556e849f9890 2024-04-23T14:36:07.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc12ff10 con 0x556e849da940 2024-04-23T14:36:07.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc0b4740 con 0x556e849dbff0 2024-04-23T14:36:07.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e849ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e849ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55376/0 (socket says 172.21.15.59:55376) 2024-04-23T14:36:07.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.937+0000 7f10d1e00640 1 -- 172.21.15.59:0/1204675515 learned_addr learned my addr 172.21.15.59:0/1204675515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10d1e00640 1 -- 172.21.15.59:0/1204675515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 msgr2=0x556e84a10060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10d1e00640 1 --2- 172.21.15.59:0/1204675515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e84a10060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10d1e00640 1 -- 172.21.15.59:0/1204675515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 msgr2=0x556e849f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10d1e00640 1 --2- 172.21.15.59:0/1204675515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e849f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10d1e00640 1 -- 172.21.15.59:0/1204675515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e845e80b0 con 0x556e849f9890 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10d1e00640 1 --2- 172.21.15.59:0/1204675515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e849ab2e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a900 tx=0x7f10c402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b949988c3bacf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10bd200640 1 -- 172.21.15.59:0/1204675515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4045070 con 0x556e849f9890 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10bd200640 1 -- 172.21.15.59:0/1204675515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c402f040 con 0x556e849f9890 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.938+0000 7f10bd200640 1 -- 172.21.15.59:0/1204675515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c40019b0 con 0x556e849f9890 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 -- 172.21.15.59:0/1204675515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 msgr2=0x556e849ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 --2- 172.21.15.59:0/1204675515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e849ab2e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a900 tx=0x7f10c402d070 comp rx=0 tx=0).stop 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 -- 172.21.15.59:0/1204675515 shutdown_connections 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 --2- 172.21.15.59:0/1204675515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e84a10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 --2- 172.21.15.59:0/1204675515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e849ab2e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 --2- 172.21.15.59:0/1204675515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e849f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 -- 172.21.15.59:0/1204675515 >> 172.21.15.59:0/1204675515 conn(0x556e849b1cd0 msgr2=0x556e849a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 -- 172.21.15.59:0/1204675515 shutdown_connections 2024-04-23T14:36:07.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.939+0000 7f10d6c39740 1 -- 172.21.15.59:0/1204675515 wait complete. 2024-04-23T14:36:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.940+0000 7f10d6c39740 1 -- start start 2024-04-23T14:36:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.940+0000 7f10d6c39740 1 Processor -- start 2024-04-23T14:36:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e84a76dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e849f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d6c39740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e84a7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d6c39740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e849ef720 con 0x556e849f9890 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556e849dee30 con 0x556e849da940 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d6c39740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556e849dfda0 con 0x556e849dbff0 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d6c39740 10 client.0 _ll_register_callbacks cb 0x7f10c0025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e84a7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e84a7d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55382/0 (socket says 172.21.15.59:55382) 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.941+0000 7f10d1e00640 1 -- 172.21.15.59:0/2830303461 learned_addr learned my addr 172.21.15.59:0/2830303461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d0a00640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e849f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d1e00640 1 -- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 msgr2=0x556e849f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d1400640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e84a76dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d1e00640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e849f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d1e00640 1 -- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 msgr2=0x556e84a76dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d1e00640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e84a76dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d1e00640 1 -- 172.21.15.59:0/2830303461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e84a7a720 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d0a00640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e849f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10d1e00640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e84a7d440 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a930 tx=0x7f10cc0cfdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f5c8e8f6ddaf9d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.942+0000 7f10b2000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10b2000640 1 -- 172.21.15.59:0/2830303461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4051060 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c402f040 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10b2000640 1 -- 172.21.15.59:0/2830303461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f10c402b830 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10b2000640 1 -- 172.21.15.59:0/2830303461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f10c4057050 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 10 client.6510 fetch_fsmap learned FSMap version 35 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556e849f35f0 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10b2000640 1 -- 172.21.15.59:0/2830303461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c402f040 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10b2000640 1 -- 172.21.15.59:0/2830303461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f10c405a020 con 0x556e849f9890 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 10 client.6510 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 10 client.6510 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 20 client.6510 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a1ac10 0x556e849ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:07.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 4 client.6510 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-23T14:36:07.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.943+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 --> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x556e849f0550 con 0x556e84a1ac10 2024-04-23T14:36:07.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.944+0000 7f10d1400640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a1ac10 0x556e849ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:07.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.944+0000 7f10b2000640 10 client.6510 ms_handle_connect on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.944+0000 7f10d1400640 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a1ac10 0x556e849ec790 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3626d367859de92 server_cookie=df929d88a703211 in_seq=0 out_seq=0 2024-04-23T14:36:07.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.944+0000 7f10b2000640 1 -- 172.21.15.59:0/2830303461 <== mds.0 v2:172.21.15.59:6834/1087917409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e849f0550 con 0x556e84a1ac10 2024-04-23T14:36:07.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.944+0000 7f10b2000640 10 client.6510 handle_command_reply: tid=0 2024-04-23T14:36:07.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.944+0000 7f10d6c39740 1 client.6510 shutdown 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 msgr2=0x556e84a7d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e84a7d440 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f10c400a930 tx=0x7f10cc0cfdf0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 shutdown_connections 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:6834/1087917409,v1:172.21.15.59:6836/1087917409] conn(0x556e84a1ac10 0x556e849ec790 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556e849dbff0 0x556e849f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e849f9890 0x556e84a7d440 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 --2- 172.21.15.59:0/2830303461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556e849da940 0x556e84a76dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 >> 172.21.15.59:0/2830303461 conn(0x556e849b1cd0 msgr2=0x556e849a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10b2000640 0 client.6510 ms_handle_reset on v2:172.21.15.59:6834/1087917409 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 shutdown_connections 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 1 -- 172.21.15.59:0/2830303461 wait complete. 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.945+0000 7f10d6c39740 20 client.6510 trim_cache size 0 max 16384 2024-04-23T14:36:07.949 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:07.950 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-23T14:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10ac06c8f0 msgr2=0x7f10ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10ac06c8f0 0x7f10ac06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007fd0 tx=0x7f10c0009040 comp rx=0 tx=0).stop 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc0cb820 msgr2=0x7f10cc0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc0cb820 0x7f10cc0cd930 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f10b8011600 tx=0x7f10b8011ac0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 shutdown_connections 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10ac06c8f0 0x7f10ac06edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10cc1e7540 0x7f10cc1e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10cc0cb820 0x7f10cc0cd930 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 --2- 172.21.15.59:0/4049277868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10b4004d40 0x7f10cc130ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 >> 172.21.15.59:0/4049277868 conn(0x7f10cc067d90 msgr2=0x7f10cc0721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 shutdown_connections 2024-04-23T14:36:07.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:07.946+0000 7f10d3e00640 1 -- 172.21.15.59:0/4049277868 wait complete. 2024-04-23T14:36:07.964 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:36:08.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.158+0000 7f3daf200640 1 Processor -- start 2024-04-23T14:36:08.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.158+0000 7f3daf200640 1 -- start start 2024-04-23T14:36:08.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.159+0000 7f3daf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3da815ece0 0x7f3da80cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.160+0000 7f3daf200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da815f430 con 0x7f3da806f620 2024-04-23T14:36:08.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.160+0000 7f3daf200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da815f710 con 0x7f3da80cd050 2024-04-23T14:36:08.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.160+0000 7f3daf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3da80ced20 con 0x7f3da815ece0 2024-04-23T14:36:08.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.160+0000 7f3dae800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3da815ece0 0x7f3da80cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.160+0000 7f3dad400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3da806f620 0x7f3da80b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56138/0 (socket says 172.21.15.59:56138) 2024-04-23T14:36:08.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.160+0000 7f3dad400640 1 -- 172.21.15.59:0/1694249117 learned_addr learned my addr 172.21.15.59:0/1694249117 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:08.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.161+0000 7f3dae800640 1 -- 172.21.15.59:0/1694249117 >> v1:172.21.15.76:6789/0 conn(0x7f3da80cd050 legacy=0x7f3da80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:08.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.161+0000 7f3dae800640 1 -- 172.21.15.59:0/1694249117 >> v1:172.21.15.59:6789/0 conn(0x7f3da806f620 legacy=0x7f3da80b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.161+0000 7f3dae800640 1 -- 172.21.15.59:0/1694249117 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da815f9f0 con 0x7f3da815ece0 2024-04-23T14:36:08.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.162+0000 7f3dae800640 1 --2- 172.21.15.59:0/1694249117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da815ece0 0x7f3da80cf470 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22bed6f5b35229f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:08.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.162+0000 7f3daca00640 1 -- 172.21.15.59:0/1694249117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4031600 con 0x7f3da815ece0 2024-04-23T14:36:08.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.162+0000 7f3daca00640 1 -- 172.21.15.59:0/1694249117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da40317a0 con 0x7f3da815ece0 2024-04-23T14:36:08.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.162+0000 7f3daca00640 1 -- 172.21.15.59:0/1694249117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4031b70 con 0x7f3da815ece0 2024-04-23T14:36:08.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.163+0000 7f3daf200640 1 -- 172.21.15.59:0/1694249117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da815ece0 msgr2=0x7f3da80cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.163+0000 7f3daf200640 1 --2- 172.21.15.59:0/1694249117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da815ece0 0x7f3da80cf470 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da402a610 comp rx=0 tx=0).stop 2024-04-23T14:36:08.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.163+0000 7f3daf200640 1 -- 172.21.15.59:0/1694249117 shutdown_connections 2024-04-23T14:36:08.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.163+0000 7f3daf200640 1 --2- 172.21.15.59:0/1694249117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da815ece0 0x7f3da80cf470 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.163+0000 7f3daf200640 1 -- 172.21.15.59:0/1694249117 >> 172.21.15.59:0/1694249117 conn(0x7f3da80b21a0 msgr2=0x7f3da80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:08.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.163+0000 7f3daf200640 1 -- 172.21.15.59:0/1694249117 shutdown_connections 2024-04-23T14:36:08.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.163+0000 7f3daf200640 1 -- 172.21.15.59:0/1694249117 wait complete. 2024-04-23T14:36:08.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.164+0000 7f3daf200640 1 Processor -- start 2024-04-23T14:36:08.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.165+0000 7f3daf200640 1 -- start start 2024-04-23T14:36:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3daf200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da806f620 0x7f3da81f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3daf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da80cd050 0x7f3da80af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3daf200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da815ece0 0x7f3da80afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3daf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3da806faa0 con 0x7f3da80cd050 2024-04-23T14:36:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3daf200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3da815fcd0 con 0x7f3da815ece0 2024-04-23T14:36:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3daf200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3da815f130 con 0x7f3da806f620 2024-04-23T14:36:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3dad400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da80cd050 0x7f3da80af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3dad400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da80cd050 0x7f3da80af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55402/0 (socket says 172.21.15.59:55402) 2024-04-23T14:36:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3dad400640 1 -- 172.21.15.59:0/898131148 learned_addr learned my addr 172.21.15.59:0/898131148 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3dae800640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da815ece0 0x7f3da80afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.166+0000 7f3dade00640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da806f620 0x7f3da81f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3dad400640 1 -- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da806f620 msgr2=0x7f3da81f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3dad400640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da806f620 0x7f3da81f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3dad400640 1 -- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da815ece0 msgr2=0x7f3da80afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3dad400640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da815ece0 0x7f3da80afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3dad400640 1 -- 172.21.15.59:0/898131148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da80c7730 con 0x7f3da80cd050 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3dad400640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da80cd050 0x7f3da80af900 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c400 tx=0x7f3d9800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3da2000640 1 -- 172.21.15.59:0/898131148 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98017070 con 0x7f3da80cd050 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3da2000640 1 -- 172.21.15.59:0/898131148 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d98005ce0 con 0x7f3da80cd050 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3da2000640 1 -- 172.21.15.59:0/898131148 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980072f0 con 0x7f3da80cd050 2024-04-23T14:36:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da80c79d0 con 0x7f3da80cd050 2024-04-23T14:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.167+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da80c7f90 con 0x7f3da80cd050 2024-04-23T14:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.169+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da804fd90 con 0x7f3da80cd050 2024-04-23T14:36:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.170+0000 7f3da2000640 1 -- 172.21.15.59:0/898131148 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d9801a070 con 0x7f3da80cd050 2024-04-23T14:36:08.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.170+0000 7f3da2000640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8c06c8f0 0x7f3d8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.171+0000 7f3da2000640 1 -- 172.21.15.59:0/898131148 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3d9808afe0 con 0x7f3da80cd050 2024-04-23T14:36:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.171+0000 7f3dade00640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8c06c8f0 0x7f3d8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.171+0000 7f3dade00640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8c06c8f0 0x7f3d8c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c004ea0 tx=0x7f3d9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.175+0000 7f3da2000640 1 -- 172.21.15.59:0/898131148 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d98059390 con 0x7f3da80cd050 2024-04-23T14:36:08.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.426+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3da80ca340 con 0x7f3da80cd050 2024-04-23T14:36:08.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.427+0000 7f3da2000640 1 -- 172.21.15.59:0/898131148 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3d98058c90 con 0x7f3da80cd050 2024-04-23T14:36:08.428 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:08.429 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T14:36:08.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8c06c8f0 msgr2=0x7f3d8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8c06c8f0 0x7f3d8c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c004ea0 tx=0x7f3d9c009040 comp rx=0 tx=0).stop 2024-04-23T14:36:08.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da80cd050 msgr2=0x7f3da80af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da80cd050 0x7f3da80af900 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c400 tx=0x7f3d9800c8c0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 shutdown_connections 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3d8c06c8f0 0x7f3d8c06edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3da815ece0 0x7f3da80afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3da80cd050 0x7f3da80af900 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 --2- 172.21.15.59:0/898131148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3da806f620 0x7f3da81f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 >> 172.21.15.59:0/898131148 conn(0x7f3da80b21a0 msgr2=0x7f3da80cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.430+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 shutdown_connections 2024-04-23T14:36:08.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.431+0000 7f3daf200640 1 -- 172.21.15.59:0/898131148 wait complete. 2024-04-23T14:36:08.452 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-23T14:36:08.452 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:36:08.452 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:36:08.573 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:36:08.676 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:36:08.676 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:36:08.677 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:36:08.677 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:08.738 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:08.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.953+0000 7fc57e800640 1 Processor -- start 2024-04-23T14:36:08.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.953+0000 7fc57e800640 1 -- start start 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57e800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc5780cd760 0x7fc5780b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5780b4690 0x7fc5780cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57e800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc578160210 con 0x7fc57815c3d0 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57e800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc5781604f0 con 0x7fc5780b4690 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57e800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc5781607d0 con 0x7fc5780cd760 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57d400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc57815c3d0 0x7fc5780cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56144/0 (socket says 172.21.15.59:56144) 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57d400640 1 -- 172.21.15.59:0/1944093250 learned_addr learned my addr 172.21.15.59:0/1944093250 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:08.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57de00640 1 --2- 172.21.15.59:0/1944093250 >> v2:172.21.15.59:3300/0 conn(0x7fc5780b4690 0x7fc5780cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.955+0000 7fc57ca00640 1 --2- 172.21.15.59:0/1944093250 >> v2:172.21.15.76:3300/0 conn(0x7fc5780cd760 0x7fc5780b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.956+0000 7fc577e00640 1 -- 172.21.15.59:0/1944093250 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912108691 0 0) 0x7fc578160210 con 0x7fc57815c3d0 2024-04-23T14:36:08.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.956+0000 7fc577e00640 1 -- 172.21.15.59:0/1944093250 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc560003520 con 0x7fc57815c3d0 2024-04-23T14:36:08.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.956+0000 7fc57de00640 1 -- 172.21.15.59:0/1944093250 >> v2:172.21.15.76:3300/0 conn(0x7fc5780cd760 msgr2=0x7fc5780b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:08.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.956+0000 7fc57de00640 1 --2- 172.21.15.59:0/1944093250 >> v2:172.21.15.76:3300/0 conn(0x7fc5780cd760 0x7fc5780b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.956+0000 7fc57de00640 1 -- 172.21.15.59:0/1944093250 >> v1:172.21.15.59:6789/0 conn(0x7fc57815c3d0 legacy=0x7fc5780cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.956+0000 7fc57de00640 1 -- 172.21.15.59:0/1944093250 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc578160ab0 con 0x7fc5780b4690 2024-04-23T14:36:08.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.957+0000 7fc57de00640 1 --2- 172.21.15.59:0/1944093250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5780b4690 0x7fc5780cb7d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc57806e620 tx=0x7fc57002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde75dc169de8221 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:08.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.957+0000 7fc577e00640 1 -- 172.21.15.59:0/1944093250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570007740 con 0x7fc5780b4690 2024-04-23T14:36:08.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.957+0000 7fc577e00640 1 -- 172.21.15.59:0/1944093250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5700078e0 con 0x7fc5780b4690 2024-04-23T14:36:08.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.957+0000 7fc577e00640 1 -- 172.21.15.59:0/1944093250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570007cb0 con 0x7fc5780b4690 2024-04-23T14:36:08.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 -- 172.21.15.59:0/1944093250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5780b4690 msgr2=0x7fc5780cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 --2- 172.21.15.59:0/1944093250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5780b4690 0x7fc5780cb7d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc57806e620 tx=0x7fc57002a680 comp rx=0 tx=0).stop 2024-04-23T14:36:08.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 -- 172.21.15.59:0/1944093250 shutdown_connections 2024-04-23T14:36:08.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 --2- 172.21.15.59:0/1944093250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5780b4690 0x7fc5780cb7d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 --2- 172.21.15.59:0/1944093250 >> v2:172.21.15.76:3300/0 conn(0x7fc5780cd760 0x7fc5780b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 -- 172.21.15.59:0/1944093250 >> 172.21.15.59:0/1944093250 conn(0x7fc5780b21a0 msgr2=0x7fc5780b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:08.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 -- 172.21.15.59:0/1944093250 shutdown_connections 2024-04-23T14:36:08.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.958+0000 7fc57e800640 1 -- 172.21.15.59:0/1944093250 wait complete. 2024-04-23T14:36:08.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.960+0000 7fc57e800640 1 Processor -- start 2024-04-23T14:36:08.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.960+0000 7fc57e800640 1 -- start start 2024-04-23T14:36:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57e800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5780b4690 0x7fc5780c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57e800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5780cd760 0x7fc57815f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc57815c3d0 0x7fc5781f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57e800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc57815c7b0 con 0x7fc57815c3d0 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57e800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc5780b4ae0 con 0x7fc5780cd760 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57e800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc57815c930 con 0x7fc5780b4690 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc57815c3d0 0x7fc5781f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc57815c3d0 0x7fc5781f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55416/0 (socket says 172.21.15.59:55416) 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.961+0000 7fc57de00640 1 -- 172.21.15.59:0/545894705 learned_addr learned my addr 172.21.15.59:0/545894705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57ca00640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5780cd760 0x7fc57815f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57d400640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5780b4690 0x7fc5780c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57de00640 1 -- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5780b4690 msgr2=0x7fc5780c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57de00640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5780b4690 0x7fc5780c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57de00640 1 -- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5780cd760 msgr2=0x7fc57815f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57de00640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5780cd760 0x7fc57815f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57de00640 1 -- 172.21.15.59:0/545894705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57815de20 con 0x7fc57815c3d0 2024-04-23T14:36:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57de00640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc57815c3d0 0x7fc5781f37f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc57002ac70 tx=0x7fc570004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc575600640 1 -- 172.21.15.59:0/545894705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570040070 con 0x7fc57815c3d0 2024-04-23T14:36:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc575600640 1 -- 172.21.15.59:0/545894705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc570031bc0 con 0x7fc57815c3d0 2024-04-23T14:36:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc575600640 1 -- 172.21.15.59:0/545894705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570009450 con 0x7fc57815c3d0 2024-04-23T14:36:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.962+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5781f3d30 con 0x7fc57815c3d0 2024-04-23T14:36:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.963+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57815e3a0 con 0x7fc57815c3d0 2024-04-23T14:36:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.964+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57804fd90 con 0x7fc57815c3d0 2024-04-23T14:36:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.965+0000 7fc575600640 1 -- 172.21.15.59:0/545894705 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc570039040 con 0x7fc57815c3d0 2024-04-23T14:36:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.966+0000 7fc575600640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc55806c8f0 0x7fc55806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.966+0000 7fc575600640 1 -- 172.21.15.59:0/545894705 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc5700ad480 con 0x7fc57815c3d0 2024-04-23T14:36:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.966+0000 7fc57d400640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc55806c8f0 0x7fc55806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.967+0000 7fc57d400640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc55806c8f0 0x7fc55806edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc56c005880 tx=0x7fc56c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:08.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:08.970+0000 7fc575600640 1 -- 172.21.15.59:0/545894705 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc57007b930 con 0x7fc57815c3d0 2024-04-23T14:36:09.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.179+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc57806cbe0 con 0x7fc57815c3d0 2024-04-23T14:36:09.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.181+0000 7fc575600640 1 -- 172.21.15.59:0/545894705 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7fc57007b230 con 0x7fc57815c3d0 2024-04-23T14:36:09.183 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:09.183 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:54.707671+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:51.101961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:52.192516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc55806c8f0 msgr2=0x7fc55806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc55806c8f0 0x7fc55806edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc56c005880 tx=0x7fc56c009040 comp rx=0 tx=0).stop 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc57815c3d0 msgr2=0x7fc5781f37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc57815c3d0 0x7fc5781f37f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc57002ac70 tx=0x7fc570004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 shutdown_connections 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc55806c8f0 0x7fc55806edb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc57815c3d0 0x7fc5781f37f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5780cd760 0x7fc57815f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 --2- 172.21.15.59:0/545894705 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5780b4690 0x7fc5780c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 >> 172.21.15.59:0/545894705 conn(0x7fc5780b21a0 msgr2=0x7fc5780c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 shutdown_connections 2024-04-23T14:36:09.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.184+0000 7fc57e800640 1 -- 172.21.15.59:0/545894705 wait complete. 2024-04-23T14:36:09.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:51.101961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:52.192516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:36:09.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:36:09.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.425+0000 7fb045400640 1 Processor -- start 2024-04-23T14:36:09.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.425+0000 7fb045400640 1 -- start start 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb045400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb0400d0d10 0x7fb040156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb045400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb04015d810 0x7fb040160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb045400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04015cab0 con 0x7fb0401567e0 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb045400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb040160e40 con 0x7fb04015d810 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb045400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb04015cd30 con 0x7fb0400d0d10 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb044a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb04015d810 0x7fb040160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb044a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb04015d810 0x7fb040160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55420/0 (socket says 172.21.15.59:55420) 2024-04-23T14:36:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb044a00640 1 -- 172.21.15.59:0/4167534460 learned_addr learned my addr 172.21.15.59:0/4167534460 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb03f400640 1 --2- 172.21.15.59:0/4167534460 >> v2:172.21.15.76:3300/0 conn(0x7fb0400d0d10 0x7fb040156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb044a00640 1 -- 172.21.15.59:0/4167534460 >> v2:172.21.15.76:3300/0 conn(0x7fb0400d0d10 msgr2=0x7fb040156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb044a00640 1 --2- 172.21.15.59:0/4167534460 >> v2:172.21.15.76:3300/0 conn(0x7fb0400d0d10 0x7fb040156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.427+0000 7fb044a00640 1 -- 172.21.15.59:0/4167534460 >> v1:172.21.15.76:6789/0 conn(0x7fb0401567e0 legacy=0x7fb0400d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.428+0000 7fb044a00640 1 -- 172.21.15.59:0/4167534460 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04015d010 con 0x7fb04015d810 2024-04-23T14:36:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.428+0000 7fb03f400640 1 --2- 172.21.15.59:0/4167534460 >> v2:172.21.15.76:3300/0 conn(0x7fb0400d0d10 0x7fb040156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:09.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.429+0000 7fb044a00640 1 --2- 172.21.15.59:0/4167534460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040160110 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb040074140 tx=0x7fb03802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7708bef1e279008 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:09.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.429+0000 7fb03ea00640 1 -- 172.21.15.59:0/4167534460 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038037070 con 0x7fb04015d810 2024-04-23T14:36:09.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.429+0000 7fb03ea00640 1 -- 172.21.15.59:0/4167534460 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb038032440 con 0x7fb04015d810 2024-04-23T14:36:09.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.429+0000 7fb03ea00640 1 -- 172.21.15.59:0/4167534460 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038032810 con 0x7fb04015d810 2024-04-23T14:36:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 -- 172.21.15.59:0/4167534460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 msgr2=0x7fb040160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 --2- 172.21.15.59:0/4167534460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040160110 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb040074140 tx=0x7fb03802a230 comp rx=0 tx=0).stop 2024-04-23T14:36:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 -- 172.21.15.59:0/4167534460 shutdown_connections 2024-04-23T14:36:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 --2- 172.21.15.59:0/4167534460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040160110 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 --2- 172.21.15.59:0/4167534460 >> v2:172.21.15.76:3300/0 conn(0x7fb0400d0d10 0x7fb040156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 -- 172.21.15.59:0/4167534460 >> 172.21.15.59:0/4167534460 conn(0x7fb04006bdf0 msgr2=0x7fb04006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 -- 172.21.15.59:0/4167534460 shutdown_connections 2024-04-23T14:36:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.430+0000 7fb045400640 1 -- 172.21.15.59:0/4167534460 wait complete. 2024-04-23T14:36:09.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.432+0000 7fb045400640 1 Processor -- start 2024-04-23T14:36:09.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.432+0000 7fb045400640 1 -- start start 2024-04-23T14:36:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb045400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0400d0d10 0x7fb040148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb045400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401567e0 0x7fb040151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb045400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb045400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401607d0 con 0x7fb04015d810 2024-04-23T14:36:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb045400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb040160650 con 0x7fb0401567e0 2024-04-23T14:36:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb045400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb040160950 con 0x7fb0400d0d10 2024-04-23T14:36:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb044a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb044a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55422/0 (socket says 172.21.15.59:55422) 2024-04-23T14:36:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb044a00640 1 -- 172.21.15.59:0/2829349781 learned_addr learned my addr 172.21.15.59:0/2829349781 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb03fe00640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0400d0d10 0x7fb040148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.433+0000 7fb03f400640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401567e0 0x7fb040151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb044a00640 1 -- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0400d0d10 msgr2=0x7fb040148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb044a00640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0400d0d10 0x7fb040148360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb044a00640 1 -- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401567e0 msgr2=0x7fb040151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb044a00640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401567e0 0x7fb040151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb044a00640 1 -- 172.21.15.59:0/2829349781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0401e5e60 con 0x7fb04015d810 2024-04-23T14:36:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb044a00640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040073130 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb0380090b0 tx=0x7fb0380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb037e00640 1 -- 172.21.15.59:0/2829349781 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038041070 con 0x7fb04015d810 2024-04-23T14:36:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb037e00640 1 -- 172.21.15.59:0/2829349781 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03803c950 con 0x7fb04015d810 2024-04-23T14:36:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.434+0000 7fb037e00640 1 -- 172.21.15.59:0/2829349781 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038037040 con 0x7fb04015d810 2024-04-23T14:36:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.435+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb040151f70 con 0x7fb04015d810 2024-04-23T14:36:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.435+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb040152530 con 0x7fb04015d810 2024-04-23T14:36:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.436+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04004fd90 con 0x7fb04015d810 2024-04-23T14:36:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.437+0000 7fb037e00640 1 -- 172.21.15.59:0/2829349781 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb038044070 con 0x7fb04015d810 2024-04-23T14:36:09.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.438+0000 7fb037e00640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb01c06c920 0x7fb01c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.438+0000 7fb037e00640 1 -- 172.21.15.59:0/2829349781 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb0380ac8e0 con 0x7fb04015d810 2024-04-23T14:36:09.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.438+0000 7fb03fe00640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb01c06c920 0x7fb01c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.439+0000 7fb03fe00640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb01c06c920 0x7fb01c06ede0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb030005520 tx=0x7fb030009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:09.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.442+0000 7fb037e00640 1 -- 172.21.15.59:0/2829349781 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb04015cab0 con 0x7fb04015d810 2024-04-23T14:36:09.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.651+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0400b5150 con 0x7fb04015d810 2024-04-23T14:36:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.652+0000 7fb037e00640 1 -- 172.21.15.59:0/2829349781 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7fb03807ad90 con 0x7fb04015d810 2024-04-23T14:36:09.653 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:36:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.654+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb01c06c920 msgr2=0x7fb01c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.654+0000 7fb045400640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb01c06c920 0x7fb01c06ede0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb030005520 tx=0x7fb030009040 comp rx=0 tx=0).stop 2024-04-23T14:36:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 msgr2=0x7fb040073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040073130 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb0380090b0 tx=0x7fb0380046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 shutdown_connections 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb01c06c920 0x7fb01c06ede0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04015d810 0x7fb040073130 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0401567e0 0x7fb040151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 --2- 172.21.15.59:0/2829349781 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0400d0d10 0x7fb040148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 >> 172.21.15.59:0/2829349781 conn(0x7fb04006bdf0 msgr2=0x7fb040155440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 shutdown_connections 2024-04-23T14:36:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.655+0000 7fb045400640 1 -- 172.21.15.59:0/2829349781 wait complete. 2024-04-23T14:36:09.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:36:09.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.888+0000 7f8f27200640 1 Processor -- start 2024-04-23T14:36:09.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.888+0000 7f8f27200640 1 -- start start 2024-04-23T14:36:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f27200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f2806fea0 0x7f8f280b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f27200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f280b6490 0x7f8f2815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f27200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f28157090 con 0x7f8f28156cb0 2024-04-23T14:36:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f27200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8f28160e70 con 0x7f8f280b6490 2024-04-23T14:36:09.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f27200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f28160260 con 0x7f8f2806fea0 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f26800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f280b6490 0x7f8f2815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f26800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f280b6490 0x7f8f2815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55436/0 (socket says 172.21.15.59:55436) 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f26800640 1 -- 172.21.15.59:0/534988311 learned_addr learned my addr 172.21.15.59:0/534988311 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.890+0000 7f8f25400640 1 --2- 172.21.15.59:0/534988311 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f2806fea0 0x7f8f280b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.891+0000 7f8f26800640 1 -- 172.21.15.59:0/534988311 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f2806fea0 msgr2=0x7f8f280b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.891+0000 7f8f26800640 1 --2- 172.21.15.59:0/534988311 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f2806fea0 0x7f8f280b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.891+0000 7f8f26800640 1 -- 172.21.15.59:0/534988311 >> v1:172.21.15.76:6789/0 conn(0x7f8f28156cb0 legacy=0x7f8f2806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:09.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.891+0000 7f8f26800640 1 -- 172.21.15.59:0/534988311 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f28160540 con 0x7f8f280b6490 2024-04-23T14:36:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.892+0000 7f8f26800640 1 --2- 172.21.15.59:0/534988311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f280b6490 0x7f8f2815dd90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009870 tx=0x7f8f2002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52db7fc645d6de78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.892+0000 7f8f24a00640 1 -- 172.21.15.59:0/534988311 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20037070 con 0x7f8f280b6490 2024-04-23T14:36:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.892+0000 7f8f24a00640 1 -- 172.21.15.59:0/534988311 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f20032440 con 0x7f8f280b6490 2024-04-23T14:36:09.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.892+0000 7f8f24a00640 1 -- 172.21.15.59:0/534988311 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20032810 con 0x7f8f280b6490 2024-04-23T14:36:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 -- 172.21.15.59:0/534988311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f280b6490 msgr2=0x7f8f2815dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 --2- 172.21.15.59:0/534988311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f280b6490 0x7f8f2815dd90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009870 tx=0x7f8f2002a200 comp rx=0 tx=0).stop 2024-04-23T14:36:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 -- 172.21.15.59:0/534988311 shutdown_connections 2024-04-23T14:36:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 --2- 172.21.15.59:0/534988311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f280b6490 0x7f8f2815dd90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 --2- 172.21.15.59:0/534988311 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f2806fea0 0x7f8f280b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 -- 172.21.15.59:0/534988311 >> 172.21.15.59:0/534988311 conn(0x7f8f280733e0 msgr2=0x7f8f2806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 -- 172.21.15.59:0/534988311 shutdown_connections 2024-04-23T14:36:09.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.893+0000 7f8f27200640 1 -- 172.21.15.59:0/534988311 wait complete. 2024-04-23T14:36:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.895+0000 7f8f27200640 1 Processor -- start 2024-04-23T14:36:09.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.895+0000 7f8f27200640 1 -- start start 2024-04-23T14:36:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.896+0000 7f8f27200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2806fea0 0x7f8f28153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.896+0000 7f8f27200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f280b6490 0x7f8f28153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.896+0000 7f8f27200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f28156cb0 0x7f8f28157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.896+0000 7f8f27200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f280b6920 con 0x7f8f2806fea0 2024-04-23T14:36:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.896+0000 7f8f27200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28160b00 con 0x7f8f28156cb0 2024-04-23T14:36:09.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.896+0000 7f8f27200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2815e2d0 con 0x7f8f280b6490 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.896+0000 7f8f25e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2806fea0 0x7f8f28153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2806fea0 0x7f8f28153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55452/0 (socket says 172.21.15.59:55452) 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 -- 172.21.15.59:0/651296876 learned_addr learned my addr 172.21.15.59:0/651296876 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 -- 172.21.15.59:0/651296876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f280b6490 msgr2=0x7f8f28153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f280b6490 0x7f8f28153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 -- 172.21.15.59:0/651296876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f28156cb0 msgr2=0x7f8f28157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f26800640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f28156cb0 0x7f8f28157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f28156cb0 0x7f8f28157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 -- 172.21.15.59:0/651296876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f28154260 con 0x7f8f2806fea0 2024-04-23T14:36:09.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.897+0000 7f8f25e00640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2806fea0 0x7f8f28153050 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800ba80 tx=0x7f8f1800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.898+0000 7f8f1e000640 1 -- 172.21.15.59:0/651296876 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800cba0 con 0x7f8f2806fea0 2024-04-23T14:36:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.898+0000 7f8f1e000640 1 -- 172.21.15.59:0/651296876 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1800cd40 con 0x7f8f2806fea0 2024-04-23T14:36:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.898+0000 7f8f1e000640 1 -- 172.21.15.59:0/651296876 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18003bb0 con 0x7f8f2806fea0 2024-04-23T14:36:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.898+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f28154540 con 0x7f8f2806fea0 2024-04-23T14:36:09.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.898+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f28157790 con 0x7f8f2806fea0 2024-04-23T14:36:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.899+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2804fd90 con 0x7f8f2806fea0 2024-04-23T14:36:09.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.900+0000 7f8f1e000640 1 -- 172.21.15.59:0/651296876 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f18014440 con 0x7f8f2806fea0 2024-04-23T14:36:09.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.901+0000 7f8f1e000640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f0406c8f0 0x7f8f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.901+0000 7f8f1e000640 1 -- 172.21.15.59:0/651296876 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8f1808bb10 con 0x7f8f2806fea0 2024-04-23T14:36:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.902+0000 7f8f25400640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f0406c8f0 0x7f8f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:09.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.902+0000 7f8f25400640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f0406c8f0 0x7f8f0406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8f281f78a0 tx=0x7f8f10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:09.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:09.905+0000 7f8f1e000640 1 -- 172.21.15.59:0/651296876 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f18059ea0 con 0x7f8f2806fea0 2024-04-23T14:36:10.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.108+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f28067b50 con 0x7f8f2806fea0 2024-04-23T14:36:10.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.109+0000 7f8f1e000640 1 -- 172.21.15.59:0/651296876 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f8f180597a0 con 0x7f8f2806fea0 2024-04-23T14:36:10.110 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:36:10.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f0406c8f0 msgr2=0x7f8f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f0406c8f0 0x7f8f0406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8f281f78a0 tx=0x7f8f10008040 comp rx=0 tx=0).stop 2024-04-23T14:36:10.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2806fea0 msgr2=0x7f8f28153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2806fea0 0x7f8f28153050 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800ba80 tx=0x7f8f1800bf40 comp rx=0 tx=0).stop 2024-04-23T14:36:10.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 shutdown_connections 2024-04-23T14:36:10.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f0406c8f0 0x7f8f0406edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f280b6490 0x7f8f28153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f28156cb0 0x7f8f28157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 --2- 172.21.15.59:0/651296876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f2806fea0 0x7f8f28153050 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 >> 172.21.15.59:0/651296876 conn(0x7f8f280733e0 msgr2=0x7f8f280b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:10.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 shutdown_connections 2024-04-23T14:36:10.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.112+0000 7f8f27200640 1 -- 172.21.15.59:0/651296876 wait complete. 2024-04-23T14:36:10.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:36:10.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.346+0000 7f8797200640 1 Processor -- start 2024-04-23T14:36:10.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.346+0000 7f8797200640 1 -- start start 2024-04-23T14:36:10.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.347+0000 7f8797200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f87981567e0 0x7f8798156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.347+0000 7f8797200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879815cab0 con 0x7f87980d0d10 2024-04-23T14:36:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.347+0000 7f8797200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8798160e40 con 0x7f879815d810 2024-04-23T14:36:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.347+0000 7f8797200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f879815cd30 con 0x7f87981567e0 2024-04-23T14:36:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.348+0000 7f8795e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f87981567e0 0x7f8798156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.348+0000 7f8795e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f87981567e0 0x7f8798156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35378/0 (socket says 172.21.15.59:35378) 2024-04-23T14:36:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.348+0000 7f8795e00640 1 -- 172.21.15.59:0/1364721550 learned_addr learned my addr 172.21.15.59:0/1364721550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.349+0000 7f8795e00640 1 -- 172.21.15.59:0/1364721550 >> v1:172.21.15.76:6789/0 conn(0x7f879815d810 legacy=0x7f87980d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.349+0000 7f8795e00640 1 -- 172.21.15.59:0/1364721550 >> v1:172.21.15.59:6789/0 conn(0x7f87980d0d10 legacy=0x7f8798160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.349+0000 7f8795e00640 1 -- 172.21.15.59:0/1364721550 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879815d010 con 0x7f87981567e0 2024-04-23T14:36:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.350+0000 7f8795e00640 1 --2- 172.21.15.59:0/1364721550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798156bc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8788009870 tx=0x7f878802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2540f59303bc2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.350+0000 7f8794a00640 1 -- 172.21.15.59:0/1364721550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788037070 con 0x7f87981567e0 2024-04-23T14:36:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.350+0000 7f8794a00640 1 -- 172.21.15.59:0/1364721550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878802ae50 con 0x7f87981567e0 2024-04-23T14:36:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.350+0000 7f8794a00640 1 -- 172.21.15.59:0/1364721550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788032860 con 0x7f87981567e0 2024-04-23T14:36:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.351+0000 7f8797200640 1 -- 172.21.15.59:0/1364721550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 msgr2=0x7f8798156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.351+0000 7f8797200640 1 --2- 172.21.15.59:0/1364721550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798156bc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8788009870 tx=0x7f878802a300 comp rx=0 tx=0).stop 2024-04-23T14:36:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.351+0000 7f8797200640 1 -- 172.21.15.59:0/1364721550 shutdown_connections 2024-04-23T14:36:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.351+0000 7f8797200640 1 --2- 172.21.15.59:0/1364721550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798156bc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.351+0000 7f8797200640 1 -- 172.21.15.59:0/1364721550 >> 172.21.15.59:0/1364721550 conn(0x7f879806bdf0 msgr2=0x7f879806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.351+0000 7f8797200640 1 -- 172.21.15.59:0/1364721550 shutdown_connections 2024-04-23T14:36:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.351+0000 7f8797200640 1 -- 172.21.15.59:0/1364721550 wait complete. 2024-04-23T14:36:10.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.353+0000 7f8797200640 1 Processor -- start 2024-04-23T14:36:10.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.353+0000 7f8797200640 1 -- start start 2024-04-23T14:36:10.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8797200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87980d0d10 0x7f8798099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8797200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8797200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f879815d810 0x7f87981eab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8797200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87981609a0 con 0x7f87981567e0 2024-04-23T14:36:10.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8797200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8798160820 con 0x7f879815d810 2024-04-23T14:36:10.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8797200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8798160b20 con 0x7f87980d0d10 2024-04-23T14:36:10.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8795400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8795400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35390/0 (socket says 172.21.15.59:35390) 2024-04-23T14:36:10.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.354+0000 7f8795400640 1 -- 172.21.15.59:0/2014773305 learned_addr learned my addr 172.21.15.59:0/2014773305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8796800640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f879815d810 0x7f87981eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8795400640 1 -- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87980d0d10 msgr2=0x7f8798099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8795400640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87980d0d10 0x7f8798099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8795400640 1 -- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f879815d810 msgr2=0x7f87981eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8795400640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f879815d810 0x7f87981eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8795400640 1 -- 172.21.15.59:0/2014773305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87981eb0c0 con 0x7f87981567e0 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8796800640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f879815d810 0x7f87981eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8795400640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798154250 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f878400c370 tx=0x7f878400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f878e000640 1 -- 172.21.15.59:0/2014773305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87840052d0 con 0x7f87981567e0 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f878e000640 1 -- 172.21.15.59:0/2014773305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8784005470 con 0x7f87981567e0 2024-04-23T14:36:10.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8798154930 con 0x7f87981567e0 2024-04-23T14:36:10.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f878e000640 1 -- 172.21.15.59:0/2014773305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784007260 con 0x7f87981567e0 2024-04-23T14:36:10.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.355+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87981e88e0 con 0x7f87981567e0 2024-04-23T14:36:10.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.356+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879804fd90 con 0x7f87981567e0 2024-04-23T14:36:10.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.358+0000 7f878e000640 1 -- 172.21.15.59:0/2014773305 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8784007400 con 0x7f87981567e0 2024-04-23T14:36:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.359+0000 7f878e000640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f877806c8f0 0x7f877806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.359+0000 7f878e000640 1 -- 172.21.15.59:0/2014773305 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f878408b090 con 0x7f87981567e0 2024-04-23T14:36:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.359+0000 7f8795e00640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f877806c8f0 0x7f877806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.360+0000 7f8795e00640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f877806c8f0 0x7f877806edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8788034000 tx=0x7f878802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:10.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.363+0000 7f878e000640 1 -- 172.21.15.59:0/2014773305 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8784059420 con 0x7f87981567e0 2024-04-23T14:36:10.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.566+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87980af9a0 con 0x7f87981567e0 2024-04-23T14:36:10.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.567+0000 7f878e000640 1 -- 172.21.15.59:0/2014773305 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7f8784058d20 con 0x7f87981567e0 2024-04-23T14:36:10.568 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f877806c8f0 msgr2=0x7f877806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f877806c8f0 0x7f877806edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8788034000 tx=0x7f878802c000 comp rx=0 tx=0).stop 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 msgr2=0x7f8798154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798154250 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f878400c370 tx=0x7f878400c830 comp rx=0 tx=0).stop 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 shutdown_connections 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f877806c8f0 0x7f877806edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f879815d810 0x7f87981eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87981567e0 0x7f8798154250 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.569+0000 7f8797200640 1 --2- 172.21.15.59:0/2014773305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87980d0d10 0x7f8798099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.570+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 >> 172.21.15.59:0/2014773305 conn(0x7f879806bdf0 msgr2=0x7f8798072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.570+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 shutdown_connections 2024-04-23T14:36:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.570+0000 7f8797200640 1 -- 172.21.15.59:0/2014773305 wait complete. 2024-04-23T14:36:10.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:36:10.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.785+0000 7f4bbbe00640 1 Processor -- start 2024-04-23T14:36:10.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.785+0000 7f4bbbe00640 1 -- start start 2024-04-23T14:36:10.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.787+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0cd8b0 0x7f4bbc0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.787+0000 7f4bbbe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bbc0b4cf0 con 0x7f4bbc15c670 2024-04-23T14:36:10.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.787+0000 7f4bbbe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bbc0ccdd0 con 0x7f4bbc0d0b00 2024-04-23T14:36:10.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.787+0000 7f4bbbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4bbc15ca50 con 0x7f4bbc0cd8b0 2024-04-23T14:36:10.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.787+0000 7f4bbaa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4bbc15c670 0x7f4bbc0b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55170/0 (socket says 172.21.15.59:55170) 2024-04-23T14:36:10.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.787+0000 7f4bbaa00640 1 -- 172.21.15.59:0/597352295 learned_addr learned my addr 172.21.15.59:0/597352295 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:10.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bb2000640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0cd8b0 0x7f4bbc0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540331703 0 0) 0x7f4bbc0b4cf0 con 0x7f4bbc15c670 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ba0003520 con 0x7f4bbc15c670 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 808892281 0 0) 0x7f4ba0003520 con 0x7f4bbc15c670 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0cd8b0 msgr2=0x7f4bbc0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0cd8b0 0x7f4bbc0d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 >> v1:172.21.15.76:6789/0 conn(0x7f4bbc0d0b00 legacy=0x7f4bbc06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc0cd0b0 con 0x7f4bbc15c670 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f4bac001670 con 0x7f4bbc15c670 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 >> v1:172.21.15.59:6789/0 conn(0x7f4bbc15c670 legacy=0x7f4bbc0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4ba0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.788+0000 7f4bba000640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0cd8b0 0x7f4ba0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bba000640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba0004d40 0x7f4ba00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba000bba0 con 0x7f4ba00039e0 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba000bd20 con 0x7f4ba0004d40 2024-04-23T14:36:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba000bf20 con 0x7f4bbc0cd8b0 2024-04-23T14:36:10.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bbaa00640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4ba0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bbaa00640 1 -- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0cd8b0 msgr2=0x7f4ba0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:10.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bbaa00640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0cd8b0 0x7f4ba0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bbaa00640 1 -- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba0004d40 msgr2=0x7f4ba00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:10.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bbaa00640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba0004d40 0x7f4ba00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.789+0000 7f4bbaa00640 1 -- 172.21.15.59:0/597352295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba0003520 con 0x7f4ba00039e0 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.790+0000 7f4bbaa00640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4ba0003e00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00c0e0 tx=0x7f4ba0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314ea516ff373386 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.790+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac007c20 con 0x7f4ba00039e0 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.790+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac005af0 con 0x7f4ba00039e0 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.790+0000 7f4bba000640 1 -- 172.21.15.59:0/597352295 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac007070 con 0x7f4ba00039e0 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.791+0000 7f4bbbe00640 1 -- 172.21.15.59:0/597352295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 msgr2=0x7f4ba0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.791+0000 7f4bbbe00640 1 --2- 172.21.15.59:0/597352295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4ba0003e00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00c0e0 tx=0x7f4ba0005830 comp rx=0 tx=0).stop 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.791+0000 7f4bbb400640 1 -- 172.21.15.59:0/597352295 reap_dead start 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.791+0000 7f4bbbe00640 1 -- 172.21.15.59:0/597352295 shutdown_connections 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.791+0000 7f4bbbe00640 1 -- 172.21.15.59:0/597352295 >> 172.21.15.59:0/597352295 conn(0x7f4bbc06bdf0 msgr2=0x7f4bbc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:10.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.791+0000 7f4bbbe00640 1 -- 172.21.15.59:0/597352295 shutdown_connections 2024-04-23T14:36:10.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.791+0000 7f4bbbe00640 1 -- 172.21.15.59:0/597352295 wait complete. 2024-04-23T14:36:10.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.793+0000 7f4bbbe00640 1 Processor -- start 2024-04-23T14:36:10.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.793+0000 7f4bbbe00640 1 -- start start 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4bbc1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc153770 0x7f4bbc0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0b08a0 0x7f4bbc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc0b3e60 con 0x7f4ba00039e0 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbbe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc0ccb80 con 0x7f4bbc153770 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4bbc0b3fe0 con 0x7f4bbc0b08a0 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4bbc1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4bbc1525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35400/0 (socket says 172.21.15.59:35400) 2024-04-23T14:36:10.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.794+0000 7f4bbaa00640 1 -- 172.21.15.59:0/85289366 learned_addr learned my addr 172.21.15.59:0/85289366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbaa00640 1 -- 172.21.15.59:0/85289366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0b08a0 msgr2=0x7f4bbc15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbaa00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0b08a0 0x7f4bbc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbaa00640 1 -- 172.21.15.59:0/85289366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc153770 msgr2=0x7f4bbc0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbaa00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc153770 0x7f4bbc0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbaa00640 1 -- 172.21.15.59:0/85289366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc0ae290 con 0x7f4ba00039e0 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbaa00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4bbc1525d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4bac000f60 tx=0x7f4bbc0b4ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bb3400640 1 -- 172.21.15.59:0/85289366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0015b0 con 0x7f4ba00039e0 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bb3400640 1 -- 172.21.15.59:0/85289366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac042070 con 0x7f4ba00039e0 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bb3400640 1 -- 172.21.15.59:0/85289366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac007270 con 0x7f4ba00039e0 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc0ae570 con 0x7f4ba00039e0 2024-04-23T14:36:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.795+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc0aeb30 con 0x7f4ba00039e0 2024-04-23T14:36:10.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.797+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bbc04fd90 con 0x7f4ba00039e0 2024-04-23T14:36:10.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.797+0000 7f4bb3400640 1 -- 172.21.15.59:0/85289366 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bac007410 con 0x7f4ba00039e0 2024-04-23T14:36:10.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.798+0000 7f4bb3400640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b9806c920 0x7f4b9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:10.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.798+0000 7f4bb3400640 1 -- 172.21.15.59:0/85289366 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4bac0abf20 con 0x7f4ba00039e0 2024-04-23T14:36:10.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.799+0000 7f4bb2000640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b9806c920 0x7f4b9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:10.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.799+0000 7f4bb2000640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b9806c920 0x7f4b9806ede0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006ce0 tx=0x7f4ba4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:10.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:10.803+0000 7f4bb3400640 1 -- 172.21.15.59:0/85289366 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bac07a2b0 con 0x7f4ba00039e0 2024-04-23T14:36:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.009+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4bbc0ceab0 con 0x7f4ba00039e0 2024-04-23T14:36:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.010+0000 7f4bb3400640 1 -- 172.21.15.59:0/85289366 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7f4bac079bb0 con 0x7f4ba00039e0 2024-04-23T14:36:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:36:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b9806c920 msgr2=0x7f4b9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b9806c920 0x7f4b9806ede0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4006ce0 tx=0x7f4ba4008040 comp rx=0 tx=0).stop 2024-04-23T14:36:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 msgr2=0x7f4bbc1525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4bbc1525d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4bac000f60 tx=0x7f4bbc0b4ec0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 shutdown_connections 2024-04-23T14:36:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b9806c920 0x7f4b9806ede0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4bbc0b08a0 0x7f4bbc15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bbc153770 0x7f4bbc0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 --2- 172.21.15.59:0/85289366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba00039e0 0x7f4bbc1525d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.012+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 >> 172.21.15.59:0/85289366 conn(0x7f4bbc06bdf0 msgr2=0x7f4bbc0cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.013+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 shutdown_connections 2024-04-23T14:36:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.013+0000 7f4bbbe00640 1 -- 172.21.15.59:0/85289366 wait complete. 2024-04-23T14:36:11.030 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:36:11.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:11.031 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:11.110 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[54012]: fuse finished with error 0 and tester_r 0 2024-04-23T14:36:11.114 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:36:11.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:11.114 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:36:11.193 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:36:11.193 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:36:11.193 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:11.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:36:11.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:36:11.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:11.233 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:36:11.319 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:36:11.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:36:11.320 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:36:11.320 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:36:11.321 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:11.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:36:11.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:36:11.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:11.371 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:36:11.466 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:36:11.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:36:11.468 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:36:11.468 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:36:11.468 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:11.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:36:11.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:36:11.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:11.526 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:36:11.580 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:36:11.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:36:11.581 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:11.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.803+0000 7f0011600640 1 Processor -- start 2024-04-23T14:36:11.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.804+0000 7f0011600640 1 -- start start 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0011600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 0x7f000c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0011600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f000c06f3f0 0x7f000c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0011600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000c0b3870 con 0x7f000c0b3080 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0011600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f000c0b41a0 con 0x7f000c06f3f0 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0011600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f000c0b3b50 con 0x7f000c15b450 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0003400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f000c06f3f0 0x7f000c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0003400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f000c06f3f0 0x7f000c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35412/0 (socket says 172.21.15.59:35412) 2024-04-23T14:36:11.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0010c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f000c0b3080 0x7f000c0ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55184/0 (socket says 172.21.15.59:55184) 2024-04-23T14:36:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.805+0000 7f0003400640 1 -- 172.21.15.59:0/3016166569 learned_addr learned my addr 172.21.15.59:0/3016166569 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.806+0000 7f000be00640 1 --2- 172.21.15.59:0/3016166569 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 0x7f000c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.806+0000 7f0003400640 1 -- 172.21.15.59:0/3016166569 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 msgr2=0x7f000c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.806+0000 7f0003400640 1 --2- 172.21.15.59:0/3016166569 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 0x7f000c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.806+0000 7f0003400640 1 -- 172.21.15.59:0/3016166569 >> v1:172.21.15.59:6789/0 conn(0x7f000c0b3080 legacy=0x7f000c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:11.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.806+0000 7f0003400640 1 -- 172.21.15.59:0/3016166569 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c0b3e30 con 0x7f000c06f3f0 2024-04-23T14:36:11.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.807+0000 7f0003400640 1 --2- 172.21.15.59:0/3016166569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c06f3f0 0x7f000c0cd050 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86c87f258f4348e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.807+0000 7f000b400640 1 -- 172.21.15.59:0/3016166569 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4037070 con 0x7f000c06f3f0 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.807+0000 7f000b400640 1 -- 172.21.15.59:0/3016166569 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff4032440 con 0x7f000c06f3f0 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.807+0000 7f000b400640 1 -- 172.21.15.59:0/3016166569 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4032810 con 0x7f000c06f3f0 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 -- 172.21.15.59:0/3016166569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c06f3f0 msgr2=0x7f000c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 --2- 172.21.15.59:0/3016166569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c06f3f0 0x7f000c0cd050 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff402a250 comp rx=0 tx=0).stop 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 -- 172.21.15.59:0/3016166569 shutdown_connections 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 --2- 172.21.15.59:0/3016166569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c06f3f0 0x7f000c0cd050 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 --2- 172.21.15.59:0/3016166569 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 0x7f000c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 -- 172.21.15.59:0/3016166569 >> 172.21.15.59:0/3016166569 conn(0x7f000c0b21a0 msgr2=0x7f000c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 -- 172.21.15.59:0/3016166569 shutdown_connections 2024-04-23T14:36:11.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.808+0000 7f0011600640 1 -- 172.21.15.59:0/3016166569 wait complete. 2024-04-23T14:36:11.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.810+0000 7f0011600640 1 Processor -- start 2024-04-23T14:36:11.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.810+0000 7f0011600640 1 -- start start 2024-04-23T14:36:11.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0011600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f000c06f3f0 0x7f000c0c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0011600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c0b3080 0x7f000c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0011600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 0x7f000c1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0011600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f000c0b4ff0 con 0x7f000c0b3080 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0011600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f000c15b830 con 0x7f000c06f3f0 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0011600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f000c0cd590 con 0x7f000c15b450 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0003400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c0b3080 0x7f000c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0003400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c0b3080 0x7f000c15bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35418/0 (socket says 172.21.15.59:35418) 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.811+0000 7f0003400640 1 -- 172.21.15.59:0/1303693730 learned_addr learned my addr 172.21.15.59:0/1303693730 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0003400640 1 -- 172.21.15.59:0/1303693730 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 msgr2=0x7f000c1f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0003400640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 0x7f000c1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0003400640 1 -- 172.21.15.59:0/1303693730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f000c06f3f0 msgr2=0x7f000c0c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0003400640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f000c06f3f0 0x7f000c0c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:11.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0003400640 1 -- 172.21.15.59:0/1303693730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c14b850 con 0x7f000c0b3080 2024-04-23T14:36:11.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0003400640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c0b3080 0x7f000c15bce0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efff4000c00 tx=0x7efff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:11.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0008c00640 1 -- 172.21.15.59:0/1303693730 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4041020 con 0x7f000c0b3080 2024-04-23T14:36:11.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0008c00640 1 -- 172.21.15.59:0/1303693730 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff40306c0 con 0x7f000c0b3080 2024-04-23T14:36:11.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0008c00640 1 -- 172.21.15.59:0/1303693730 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4037040 con 0x7f000c0b3080 2024-04-23T14:36:11.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.812+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c0c5970 con 0x7f000c0b3080 2024-04-23T14:36:11.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.813+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c0c5f30 con 0x7f000c0b3080 2024-04-23T14:36:11.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.814+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c04fd90 con 0x7f000c0b3080 2024-04-23T14:36:11.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.815+0000 7f0008c00640 1 -- 172.21.15.59:0/1303693730 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efff4044020 con 0x7f000c0b3080 2024-04-23T14:36:11.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.816+0000 7f0008c00640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effe806c8f0 0x7effe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:11.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.816+0000 7f0008c00640 1 -- 172.21.15.59:0/1303693730 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7efff40ac0e0 con 0x7f000c0b3080 2024-04-23T14:36:11.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.816+0000 7f000be00640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effe806c8f0 0x7effe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:11.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.817+0000 7f000be00640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effe806c8f0 0x7effe806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efffc0070b0 tx=0x7efffc007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:11.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:11.820+0000 7f0008c00640 1 -- 172.21.15.59:0/1303693730 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efff404b070 con 0x7f000c0b3080 2024-04-23T14:36:12.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.072+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f000c0d0780 con 0x7f000c0b3080 2024-04-23T14:36:12.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.073+0000 7f0008c00640 1 -- 172.21.15.59:0/1303693730 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5182 (secure 0 0 0) 0x7efff407a470 con 0x7f000c0b3080 2024-04-23T14:36:12.075 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:12.075 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:35:37.778594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:36:12.075 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.076+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effe806c8f0 msgr2=0x7effe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.076+0000 7f0011600640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effe806c8f0 0x7effe806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efffc0070b0 tx=0x7efffc007040 comp rx=0 tx=0).stop 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.076+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c0b3080 msgr2=0x7f000c15bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.076+0000 7f0011600640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c0b3080 0x7f000c15bce0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efff4000c00 tx=0x7efff40046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 shutdown_connections 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7effe806c8f0 0x7effe806edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f000c15b450 0x7f000c1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f000c0b3080 0x7f000c15bce0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 --2- 172.21.15.59:0/1303693730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f000c06f3f0 0x7f000c0c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 >> 172.21.15.59:0/1303693730 conn(0x7f000c0b21a0 msgr2=0x7f000c0d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 shutdown_connections 2024-04-23T14:36:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.077+0000 7f0011600640 1 -- 172.21.15.59:0/1303693730 wait complete. 2024-04-23T14:36:12.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:12.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.307+0000 7f3edf200640 1 Processor -- start 2024-04-23T14:36:12.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.307+0000 7f3edf200640 1 -- start start 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3ed80b5ff0 0x7f3ed80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edf200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3ed80d0d90 0x7f3ed80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edf200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed80cf030 con 0x7f3ed815e7c0 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed80b7000 con 0x7f3ed80b5ff0 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edf200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed80d1720 con 0x7f3ed80d0d90 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edd400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3ed80b5ff0 0x7f3ed80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edde00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3ed815e7c0 0x7f3ed80d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55190/0 (socket says 172.21.15.59:55190) 2024-04-23T14:36:12.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.308+0000 7f3edde00640 1 -- 172.21.15.59:0/1832509203 learned_addr learned my addr 172.21.15.59:0/1832509203 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:12.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.309+0000 7f3edd400640 1 -- 172.21.15.59:0/1832509203 >> v2:172.21.15.76:3300/0 conn(0x7f3ed80d0d90 msgr2=0x7f3ed80d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:12.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.309+0000 7f3edd400640 1 --2- 172.21.15.59:0/1832509203 >> v2:172.21.15.76:3300/0 conn(0x7f3ed80d0d90 0x7f3ed80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.309+0000 7f3edd400640 1 -- 172.21.15.59:0/1832509203 >> v1:172.21.15.59:6789/0 conn(0x7f3ed815e7c0 legacy=0x7f3ed80d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.309+0000 7f3edd400640 1 -- 172.21.15.59:0/1832509203 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed80b6a40 con 0x7f3ed80b5ff0 2024-04-23T14:36:12.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.309+0000 7f3edca00640 1 -- 172.21.15.59:0/1832509203 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3550348302 0 0) 0x7f3ed4001490 con 0x7f3ed815e7c0 2024-04-23T14:36:12.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.310+0000 7f3edd400640 1 --2- 172.21.15.59:0/1832509203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80b5ff0 0x7f3ed80b6450 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3ed808da50 tx=0x7f3ec802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0969b2c1d2a852 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:12.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.310+0000 7f3edca00640 1 -- 172.21.15.59:0/1832509203 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8037070 con 0x7f3ed80b5ff0 2024-04-23T14:36:12.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.310+0000 7f3edca00640 1 -- 172.21.15.59:0/1832509203 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec8032440 con 0x7f3ed80b5ff0 2024-04-23T14:36:12.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.310+0000 7f3edca00640 1 -- 172.21.15.59:0/1832509203 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8032810 con 0x7f3ed80b5ff0 2024-04-23T14:36:12.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 -- 172.21.15.59:0/1832509203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80b5ff0 msgr2=0x7f3ed80b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 --2- 172.21.15.59:0/1832509203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80b5ff0 0x7f3ed80b6450 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3ed808da50 tx=0x7f3ec802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 -- 172.21.15.59:0/1832509203 shutdown_connections 2024-04-23T14:36:12.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 --2- 172.21.15.59:0/1832509203 >> v2:172.21.15.76:3300/0 conn(0x7f3ed80d0d90 0x7f3ed80d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 --2- 172.21.15.59:0/1832509203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80b5ff0 0x7f3ed80b6450 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 -- 172.21.15.59:0/1832509203 >> 172.21.15.59:0/1832509203 conn(0x7f3ed809d920 msgr2=0x7f3ed808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:12.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 -- 172.21.15.59:0/1832509203 shutdown_connections 2024-04-23T14:36:12.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.311+0000 7f3edf200640 1 -- 172.21.15.59:0/1832509203 wait complete. 2024-04-23T14:36:12.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.313+0000 7f3edf200640 1 Processor -- start 2024-04-23T14:36:12.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.313+0000 7f3edf200640 1 -- start start 2024-04-23T14:36:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edf200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed80b5ff0 0x7f3ed80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80d0d90 0x7f3ed80b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edf200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed815e7c0 0x7f3ed80b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed80cf2b0 con 0x7f3ed80d0d90 2024-04-23T14:36:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edf200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed80cf620 con 0x7f3ed80b5ff0 2024-04-23T14:36:12.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edf200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed80d2800 con 0x7f3ed815e7c0 2024-04-23T14:36:12.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80d0d90 0x7f3ed80b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80d0d90 0x7f3ed80b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35424/0 (socket says 172.21.15.59:35424) 2024-04-23T14:36:12.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.314+0000 7f3edd400640 1 -- 172.21.15.59:0/3058228716 learned_addr learned my addr 172.21.15.59:0/3058228716 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:12.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3edde00640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed80b5ff0 0x7f3ed80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3ede800640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed815e7c0 0x7f3ed80b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3edd400640 1 -- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed815e7c0 msgr2=0x7f3ed80b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3edd400640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed815e7c0 0x7f3ed80b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3edd400640 1 -- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed80b5ff0 msgr2=0x7f3ed80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3edd400640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed80b5ff0 0x7f3ed80cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3edd400640 1 -- 172.21.15.59:0/3058228716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed81f8810 con 0x7f3ed80d0d90 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3ede800640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed815e7c0 0x7f3ed80b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3edd400640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80d0d90 0x7f3ed80b84a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8035040 tx=0x7f3ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.315+0000 7f3ece000640 1 -- 172.21.15.59:0/3058228716 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8037040 con 0x7f3ed80d0d90 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.316+0000 7f3ece000640 1 -- 172.21.15.59:0/3058228716 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec8004230 con 0x7f3ed80d0d90 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.316+0000 7f3ece000640 1 -- 172.21.15.59:0/3058228716 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8032e20 con 0x7f3ed80d0d90 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.316+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed81f7c90 con 0x7f3ed80d0d90 2024-04-23T14:36:12.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.316+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81f8250 con 0x7f3ed80d0d90 2024-04-23T14:36:12.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.317+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed81f43d0 con 0x7f3ed80d0d90 2024-04-23T14:36:12.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.318+0000 7f3ece000640 1 -- 172.21.15.59:0/3058228716 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ec8030770 con 0x7f3ed80d0d90 2024-04-23T14:36:12.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.319+0000 7f3ece000640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.319+0000 7f3ece000640 1 -- 172.21.15.59:0/3058228716 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3ec80acf90 con 0x7f3ed80d0d90 2024-04-23T14:36:12.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.319+0000 7f3edde00640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.320+0000 7f3edde00640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40012e0 tx=0x7f3ed4005eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:12.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.324+0000 7f3ece000640 1 -- 172.21.15.59:0/3058228716 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ec807b320 con 0x7f3ed80d0d90 2024-04-23T14:36:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.530+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ed804fd90 con 0x7f3ed80d0d90 2024-04-23T14:36:12.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.532+0000 7f3ece000640 1 -- 172.21.15.59:0/3058228716 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7f3ec807ac20 con 0x7f3ed80d0d90 2024-04-23T14:36:12.533 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:12.534 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:54.707671+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:51.101961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:52.192516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.534+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3ebc06c8f0 msgr2=0x7f3ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.534+0000 7f3edf200640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40012e0 tx=0x7f3ed4005eb0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.534+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80d0d90 msgr2=0x7f3ed80b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.534+0000 7f3edf200640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80d0d90 0x7f3ed80b84a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8035040 tx=0x7f3ec80046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 shutdown_connections 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed815e7c0 0x7f3ed80b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed80d0d90 0x7f3ed80b84a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 --2- 172.21.15.59:0/3058228716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed80b5ff0 0x7f3ed80cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 >> 172.21.15.59:0/3058228716 conn(0x7f3ed809d920 msgr2=0x7f3ed815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:12.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 shutdown_connections 2024-04-23T14:36:12.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.535+0000 7f3edf200640 1 -- 172.21.15.59:0/3058228716 wait complete. 2024-04-23T14:36:12.556 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:34:51.101961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:34:52.192516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:36:12.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:36:12.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.773+0000 7f32e6e00640 1 Processor -- start 2024-04-23T14:36:12.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.773+0000 7f32e6e00640 1 -- start start 2024-04-23T14:36:12.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.776+0000 7f32e6e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32e00d0600 0x7f32e00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.776+0000 7f32e6e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e0071e10 con 0x7f32e006dcb0 2024-04-23T14:36:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.776+0000 7f32e6e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e00720f0 con 0x7f32e00d09e0 2024-04-23T14:36:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.776+0000 7f32e6e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f32e0072a20 con 0x7f32e00d0600 2024-04-23T14:36:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.778+0000 7f32e5a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32e00d0600 0x7f32e00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.778+0000 7f32e5a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32e00d0600 0x7f32e00ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35436/0 (socket says 172.21.15.59:35436) 2024-04-23T14:36:12.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.778+0000 7f32e5a00640 1 -- 172.21.15.59:0/1558870222 learned_addr learned my addr 172.21.15.59:0/1558870222 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.784+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331670676 0 0) 0x7f32e0071e10 con 0x7f32e006dcb0 2024-04-23T14:36:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.784+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32c8003520 con 0x7f32e006dcb0 2024-04-23T14:36:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.784+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019556205 0 0) 0x7f32e00720f0 con 0x7f32e00d09e0 2024-04-23T14:36:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.784+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32e0071e10 con 0x7f32e00d09e0 2024-04-23T14:36:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.785+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3702611337 0 0) 0x7f32c8003520 con 0x7f32e006dcb0 2024-04-23T14:36:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.785+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 >> v2:172.21.15.59:3300/0 conn(0x7f32e00d0600 msgr2=0x7f32e00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.785+0000 7f32dfe00640 1 --2- 172.21.15.59:0/1558870222 >> v2:172.21.15.59:3300/0 conn(0x7f32e00d0600 0x7f32e00ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.785+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 >> v1:172.21.15.76:6789/0 conn(0x7f32e00d09e0 legacy=0x7f32e006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.785+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e00723d0 con 0x7f32e006dcb0 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f32cc001670 con 0x7f32e006dcb0 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 >> v1:172.21.15.59:6789/0 conn(0x7f32e006dcb0 legacy=0x7f32e0162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32c8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c8004800 0x7f32c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32c80051c0 0x7f32c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0072a20 con 0x7f32c8003b40 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32c800c0f0 con 0x7f32c8004800 2024-04-23T14:36:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.786+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32c800c2f0 con 0x7f32c80051c0 2024-04-23T14:36:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.787+0000 7f32e6400640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32c80051c0 0x7f32c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.787+0000 7f32e5000640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c8004800 0x7f32c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.788+0000 7f32e5a00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32c8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.788+0000 7f32e5a00640 1 -- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32c80051c0 msgr2=0x7f32c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:12.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.788+0000 7f32e5a00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32c80051c0 0x7f32c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.788+0000 7f32e5a00640 1 -- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c8004800 msgr2=0x7f32c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:12.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.788+0000 7f32e5a00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32c8004800 0x7f32c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.788+0000 7f32e5a00640 1 -- 172.21.15.59:0/1558870222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c8003520 con 0x7f32c8003b40 2024-04-23T14:36:12.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.788+0000 7f32e6400640 1 -- 172.21.15.59:0/1558870222 reap_dead start 2024-04-23T14:36:12.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.789+0000 7f32e5a00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32c8003f40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f32d400fc50 tx=0x7f32c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415a3a638ff714e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:12.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.789+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4041070 con 0x7f32c8003b40 2024-04-23T14:36:12.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.789+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d4036030 con 0x7f32c8003b40 2024-04-23T14:36:12.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.789+0000 7f32dfe00640 1 -- 172.21.15.59:0/1558870222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d400f850 con 0x7f32c8003b40 2024-04-23T14:36:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.791+0000 7f32e6e00640 1 -- 172.21.15.59:0/1558870222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 msgr2=0x7f32c8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.791+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32c8003f40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f32d400fc50 tx=0x7f32c8005830 comp rx=0 tx=0).stop 2024-04-23T14:36:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.791+0000 7f32e6e00640 1 -- 172.21.15.59:0/1558870222 shutdown_connections 2024-04-23T14:36:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.791+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1558870222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32c8003f40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.791+0000 7f32e6e00640 1 -- 172.21.15.59:0/1558870222 >> 172.21.15.59:0/1558870222 conn(0x7f32e006bdf0 msgr2=0x7f32e006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.791+0000 7f32e6e00640 1 -- 172.21.15.59:0/1558870222 shutdown_connections 2024-04-23T14:36:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.791+0000 7f32e6e00640 1 -- 172.21.15.59:0/1558870222 wait complete. 2024-04-23T14:36:12.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.793+0000 7f32e6e00640 1 Processor -- start 2024-04-23T14:36:12.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.793+0000 7f32e6e00640 1 -- start start 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32e015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e6e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e015c720 0x7f32e015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e6e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e015d0e0 0x7f32e00b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32e00d0e90 con 0x7f32c8003b40 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e6e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32e015eaf0 con 0x7f32e015d0e0 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e6e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32e00ad170 con 0x7f32e015c720 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32e015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32e015a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35450/0 (socket says 172.21.15.59:35450) 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.797+0000 7f32e5a00640 1 -- 172.21.15.59:0/1811063876 learned_addr learned my addr 172.21.15.59:0/1811063876 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e5000640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e015c720 0x7f32e015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e5a00640 1 -- 172.21.15.59:0/1811063876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e015c720 msgr2=0x7f32e015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e5a00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e015c720 0x7f32e015cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e5a00640 1 -- 172.21.15.59:0/1811063876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e015d0e0 msgr2=0x7f32e00b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:12.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e5a00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e015d0e0 0x7f32e00b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:12.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e5a00640 1 -- 172.21.15.59:0/1811063876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e015add0 con 0x7f32c8003b40 2024-04-23T14:36:12.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e5a00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32e015a740 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f32d403f040 tx=0x7f32d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32dd600640 1 -- 172.21.15.59:0/1811063876 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4041040 con 0x7f32c8003b40 2024-04-23T14:36:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32dd600640 1 -- 172.21.15.59:0/1811063876 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d4047630 con 0x7f32c8003b40 2024-04-23T14:36:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e00afe60 con 0x7f32c8003b40 2024-04-23T14:36:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.798+0000 7f32dd600640 1 -- 172.21.15.59:0/1811063876 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d400f850 con 0x7f32c8003b40 2024-04-23T14:36:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.799+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e00b0420 con 0x7f32c8003b40 2024-04-23T14:36:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.800+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e004fd90 con 0x7f32c8003b40 2024-04-23T14:36:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.801+0000 7f32dd600640 1 -- 172.21.15.59:0/1811063876 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32d40479e0 con 0x7f32c8003b40 2024-04-23T14:36:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.802+0000 7f32dd600640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c006c8f0 0x7f32c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.802+0000 7f32dd600640 1 -- 172.21.15.59:0/1811063876 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f32d40b3ee0 con 0x7f32c8003b40 2024-04-23T14:36:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.802+0000 7f32e5000640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c006c8f0 0x7f32c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.803+0000 7f32e5000640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c006c8f0 0x7f32c006edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f32e00b27e0 tx=0x7f32cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:12.806+0000 7f32dd600640 1 -- 172.21.15.59:0/1811063876 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32d4081c20 con 0x7f32c8003b40 2024-04-23T14:36:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.010+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32e015e380 con 0x7f32c8003b40 2024-04-23T14:36:13.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.010+0000 7f32dd600640 1 -- 172.21.15.59:0/1811063876 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f32d4085f20 con 0x7f32c8003b40 2024-04-23T14:36:13.011 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:36:13.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c006c8f0 msgr2=0x7f32c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c006c8f0 0x7f32c006edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f32e00b27e0 tx=0x7f32cc009040 comp rx=0 tx=0).stop 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 msgr2=0x7f32e015a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32e015a740 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f32d403f040 tx=0x7f32d40046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 shutdown_connections 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c006c8f0 0x7f32c006edb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e015c720 0x7f32e015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e015d0e0 0x7f32e00b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 --2- 172.21.15.59:0/1811063876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c8003b40 0x7f32e015a740 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 >> 172.21.15.59:0/1811063876 conn(0x7f32e006bdf0 msgr2=0x7f32e015d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 shutdown_connections 2024-04-23T14:36:13.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.013+0000 7f32e6e00640 1 -- 172.21.15.59:0/1811063876 wait complete. 2024-04-23T14:36:13.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:36:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.260+0000 7f4444a00640 1 Processor -- start 2024-04-23T14:36:13.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.260+0000 7f4444a00640 1 -- start start 2024-04-23T14:36:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.262+0000 7f4444a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 0x7f44400cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.262+0000 7f4444a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44400b5da0 con 0x7f44400d15b0 2024-04-23T14:36:13.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.262+0000 7f4444a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44400d0a20 con 0x7f444015ed20 2024-04-23T14:36:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.262+0000 7f4444a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f444015f100 con 0x7f44400b4f20 2024-04-23T14:36:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.262+0000 7f443ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f44400d15b0 0x7f44400cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55214/0 (socket says 172.21.15.59:55214) 2024-04-23T14:36:13.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.262+0000 7f443ea00640 1 -- 172.21.15.59:0/2232491601 learned_addr learned my addr 172.21.15.59:0/2232491601 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443fe00640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 0x7f44400cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1647023838 0 0) 0x7f44400b5da0 con 0x7f44400d15b0 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4424003520 con 0x7f44400d15b0 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1352972730 0 0) 0x7f4424003520 con 0x7f44400d15b0 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 msgr2=0x7f44400cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 0x7f44400cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 >> v1:172.21.15.76:6789/0 conn(0x7f444015ed20 legacy=0x7f44400d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44400d2ed0 con 0x7f44400d15b0 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f442c001670 con 0x7f44400d15b0 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 >> v1:172.21.15.59:6789/0 conn(0x7f44400d15b0 legacy=0x7f44400cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240039e0 0x7f4424003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 0x7f4424004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f442400bb80 con 0x7f4424004d40 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.263+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f442400bd30 con 0x7f44240039e0 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f442400bf30 con 0x7f44400b4f20 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443fe00640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443ea00640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 0x7f4424004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443f400640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240039e0 0x7f4424003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443fe00640 1 -- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 msgr2=0x7f4424004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443fe00640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44400b4f20 0x7f4424004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443fe00640 1 -- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240039e0 msgr2=0x7f4424003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443fe00640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240039e0 0x7f4424003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.264+0000 7f443fe00640 1 -- 172.21.15.59:0/2232491601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4424003520 con 0x7f4424004d40 2024-04-23T14:36:13.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.265+0000 7f443f400640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240039e0 0x7f4424003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.266+0000 7f443fe00640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44240051c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4438009f90 tx=0x7f4424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2287d272fbf454f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.266+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438009c10 con 0x7f4424004d40 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.266+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4438036070 con 0x7f4424004d40 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.266+0000 7f443e000640 1 -- 172.21.15.59:0/2232491601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438031810 con 0x7f4424004d40 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.266+0000 7f4444a00640 1 -- 172.21.15.59:0/2232491601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 msgr2=0x7f44240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.266+0000 7f4444a00640 1 --2- 172.21.15.59:0/2232491601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44240051c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4438009f90 tx=0x7f4424005830 comp rx=0 tx=0).stop 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.267+0000 7f443fe00640 1 -- 172.21.15.59:0/2232491601 reap_dead start 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.267+0000 7f4444a00640 1 -- 172.21.15.59:0/2232491601 shutdown_connections 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.267+0000 7f4444a00640 1 -- 172.21.15.59:0/2232491601 >> 172.21.15.59:0/2232491601 conn(0x7f444008bf90 msgr2=0x7f44400d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.267+0000 7f4444a00640 1 -- 172.21.15.59:0/2232491601 shutdown_connections 2024-04-23T14:36:13.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.267+0000 7f4444a00640 1 -- 172.21.15.59:0/2232491601 wait complete. 2024-04-23T14:36:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.269+0000 7f4444a00640 1 Processor -- start 2024-04-23T14:36:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.269+0000 7f4444a00640 1 -- start start 2024-04-23T14:36:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f4444a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44400c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f4444a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44401f7920 0x7f4440158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f4444a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44400ccb90 0x7f44400cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f4444a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44400b3600 con 0x7f4424004d40 2024-04-23T14:36:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f4444a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f44400d3490 con 0x7f44400ccb90 2024-04-23T14:36:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f4444a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f44400b3780 con 0x7f44401f7920 2024-04-23T14:36:13.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44400c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44400c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35476/0 (socket says 172.21.15.59:35476) 2024-04-23T14:36:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 -- 172.21.15.59:0/4188025086 learned_addr learned my addr 172.21.15.59:0/4188025086 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 -- 172.21.15.59:0/4188025086 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44401f7920 msgr2=0x7f4440158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44401f7920 0x7f4440158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 -- 172.21.15.59:0/4188025086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44400ccb90 msgr2=0x7f44400cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44400ccb90 0x7f44400cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.270+0000 7f443f400640 1 -- 172.21.15.59:0/4188025086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4440159390 con 0x7f4424004d40 2024-04-23T14:36:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.271+0000 7f443f400640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44400c8960 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f443000d7d0 tx=0x7f443000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.271+0000 7f4437400640 1 -- 172.21.15.59:0/4188025086 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430005070 con 0x7f4424004d40 2024-04-23T14:36:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.271+0000 7f4437400640 1 -- 172.21.15.59:0/4188025086 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4430005230 con 0x7f4424004d40 2024-04-23T14:36:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.271+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44401f8090 con 0x7f4424004d40 2024-04-23T14:36:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.271+0000 7f4437400640 1 -- 172.21.15.59:0/4188025086 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430007030 con 0x7f4424004d40 2024-04-23T14:36:13.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.271+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401f8650 con 0x7f4424004d40 2024-04-23T14:36:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.273+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444004fd90 con 0x7f4424004d40 2024-04-23T14:36:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.274+0000 7f4437400640 1 -- 172.21.15.59:0/4188025086 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f443001a020 con 0x7f4424004d40 2024-04-23T14:36:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.274+0000 7f4437400640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f442006c8f0 0x7f442006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.275+0000 7f4437400640 1 -- 172.21.15.59:0/4188025086 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f443008b2e0 con 0x7f4424004d40 2024-04-23T14:36:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.275+0000 7f443ea00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f442006c8f0 0x7f442006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.276+0000 7f443ea00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f442006c8f0 0x7f442006edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f442c001eb0 tx=0x7f442c001da0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.279+0000 7f4437400640 1 -- 172.21.15.59:0/4188025086 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4430059700 con 0x7f4424004d40 2024-04-23T14:36:13.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.487+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44401f8820 con 0x7f4424004d40 2024-04-23T14:36:13.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.488+0000 7f4437400640 1 -- 172.21.15.59:0/4188025086 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f4430059000 con 0x7f4424004d40 2024-04-23T14:36:13.489 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:36:13.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f442006c8f0 msgr2=0x7f442006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f442006c8f0 0x7f442006edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f442c001eb0 tx=0x7f442c001da0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 msgr2=0x7f44400c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44400c8960 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f443000d7d0 tx=0x7f443000dc90 comp rx=0 tx=0).stop 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 shutdown_connections 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f442006c8f0 0x7f442006edb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44401f7920 0x7f4440158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44400ccb90 0x7f44400cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 --2- 172.21.15.59:0/4188025086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4424004d40 0x7f44400c8960 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 >> 172.21.15.59:0/4188025086 conn(0x7f444008bf90 msgr2=0x7f44400cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 shutdown_connections 2024-04-23T14:36:13.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.490+0000 7f4444a00640 1 -- 172.21.15.59:0/4188025086 wait complete. 2024-04-23T14:36:13.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:36:13.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.723+0000 7f1cdfe00640 1 Processor -- start 2024-04-23T14:36:13.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.723+0000 7f1cdfe00640 1 -- start start 2024-04-23T14:36:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.725+0000 7f1cdfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ce0146030 0x7f1ce015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.725+0000 7f1cdfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce0146410 con 0x7f1ce0160980 2024-04-23T14:36:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.725+0000 7f1cdfe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce00b10f0 con 0x7f1ce00b06a0 2024-04-23T14:36:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.725+0000 7f1cdfe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce00b13d0 con 0x7f1ce0146030 2024-04-23T14:36:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.725+0000 7f1cdf400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1ce0160980 0x7f1ce00b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55228/0 (socket says 172.21.15.59:55228) 2024-04-23T14:36:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.725+0000 7f1cdf400640 1 -- 172.21.15.59:0/158762834 learned_addr learned my addr 172.21.15.59:0/158762834 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.726+0000 7f1cdea00640 1 --2- 172.21.15.59:0/158762834 >> v2:172.21.15.76:3300/0 conn(0x7f1ce0146030 0x7f1ce015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.726+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3004529738 0 0) 0x7f1ce0146410 con 0x7f1ce0160980 2024-04-23T14:36:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.726+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cc4003520 con 0x7f1ce0160980 2024-04-23T14:36:13.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4258702520 0 0) 0x7f1cc4003520 con 0x7f1ce0160980 2024-04-23T14:36:13.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 >> v2:172.21.15.76:3300/0 conn(0x7f1ce0146030 msgr2=0x7f1ce015ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 --2- 172.21.15.59:0/158762834 >> v2:172.21.15.76:3300/0 conn(0x7f1ce0146030 0x7f1ce015ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 >> v1:172.21.15.76:6789/0 conn(0x7f1ce00b06a0 legacy=0x7f1ce0160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce00b16b0 con 0x7f1ce0160980 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1cd8001690 con 0x7f1ce0160980 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 >> v1:172.21.15.59:6789/0 conn(0x7f1ce0160980 legacy=0x7f1ce00b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc40039b0 0x7f1cc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc4004800 0x7f1cc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc40051c0 0x7f1cc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce00b13d0 con 0x7f1cc40039b0 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc400c060 con 0x7f1cc40051c0 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc400c260 con 0x7f1cc4004800 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.727+0000 7f1cdea00640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc40039b0 0x7f1cc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cdf400640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc40051c0 0x7f1cc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cde000640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc4004800 0x7f1cc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cdea00640 1 -- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc4004800 msgr2=0x7f1cc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cdea00640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc4004800 0x7f1cc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cdea00640 1 -- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc40051c0 msgr2=0x7f1cc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cdea00640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc40051c0 0x7f1cc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cdea00640 1 -- 172.21.15.59:0/158762834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc4003520 con 0x7f1cc40039b0 2024-04-23T14:36:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.728+0000 7f1cdf400640 1 -- 172.21.15.59:0/158762834 reap_dead start 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.729+0000 7f1cdea00640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc40039b0 0x7f1cc4003dd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009fc0 tx=0x7f1cc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f738ca358600c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.729+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd003c070 con 0x7f1cc40039b0 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.729+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd002f800 con 0x7f1cc40039b0 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.729+0000 7f1cdd600640 1 -- 172.21.15.59:0/158762834 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0009c10 con 0x7f1cc40039b0 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.730+0000 7f1cdfe00640 1 -- 172.21.15.59:0/158762834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc40039b0 msgr2=0x7f1cc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.730+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc40039b0 0x7f1cc4003dd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009fc0 tx=0x7f1cc4005830 comp rx=0 tx=0).stop 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.730+0000 7f1cdfe00640 1 -- 172.21.15.59:0/158762834 shutdown_connections 2024-04-23T14:36:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.730+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/158762834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc40039b0 0x7f1cc4003dd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.730+0000 7f1cdfe00640 1 -- 172.21.15.59:0/158762834 >> 172.21.15.59:0/158762834 conn(0x7f1ce00b6910 msgr2=0x7f1ce00b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.730+0000 7f1cdfe00640 1 -- 172.21.15.59:0/158762834 shutdown_connections 2024-04-23T14:36:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.730+0000 7f1cdfe00640 1 -- 172.21.15.59:0/158762834 wait complete. 2024-04-23T14:36:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.732+0000 7f1cdfe00640 1 Processor -- start 2024-04-23T14:36:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.732+0000 7f1cdfe00640 1 -- start start 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc40039b0 0x7f1ce015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce0141d80 0x7f1ce0136970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce013d2e0 0x7f1ce013d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0160fc0 con 0x7f1ce013d2e0 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdfe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce00b1c70 con 0x7f1ce0141d80 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce0161140 con 0x7f1cc40039b0 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce013d2e0 0x7f1ce013d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce013d2e0 0x7f1ce013d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35488/0 (socket says 172.21.15.59:35488) 2024-04-23T14:36:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.733+0000 7f1cdf400640 1 -- 172.21.15.59:0/3873061155 learned_addr learned my addr 172.21.15.59:0/3873061155 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdea00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc40039b0 0x7f1ce015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cde000640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce0141d80 0x7f1ce0136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdf400640 1 -- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc40039b0 msgr2=0x7f1ce015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdf400640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc40039b0 0x7f1ce015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdf400640 1 -- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce0141d80 msgr2=0x7f1ce0136970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdf400640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce0141d80 0x7f1ce0136970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdf400640 1 -- 172.21.15.59:0/3873061155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce01422f0 con 0x7f1ce013d2e0 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cde000640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce0141d80 0x7f1ce0136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdf400640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce013d2e0 0x7f1ce013d790 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000f60 tx=0x7f1cd800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cd6a00640 1 -- 172.21.15.59:0/3873061155 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8019070 con 0x7f1ce013d2e0 2024-04-23T14:36:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cd6a00640 1 -- 172.21.15.59:0/3873061155 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd8010510 con 0x7f1ce013d2e0 2024-04-23T14:36:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cd6a00640 1 -- 172.21.15.59:0/3873061155 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd80108c0 con 0x7f1ce013d2e0 2024-04-23T14:36:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.734+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce01425d0 con 0x7f1ce013d2e0 2024-04-23T14:36:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.735+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce0142b90 con 0x7f1ce013d2e0 2024-04-23T14:36:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.736+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce004fd90 con 0x7f1ce013d2e0 2024-04-23T14:36:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.737+0000 7f1cd6a00640 1 -- 172.21.15.59:0/3873061155 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cd801c070 con 0x7f1ce013d2e0 2024-04-23T14:36:13.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.738+0000 7f1cd6a00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cbc06c920 0x7f1cbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:13.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.738+0000 7f1cd6a00640 1 -- 172.21.15.59:0/3873061155 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1cd808aff0 con 0x7f1ce013d2e0 2024-04-23T14:36:13.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.738+0000 7f1cdea00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cbc06c920 0x7f1cbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:13.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.739+0000 7f1cdea00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cbc06c920 0x7f1cbc06ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0039000 tx=0x7f1cd0031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:13.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.743+0000 7f1cd6a00640 1 -- 172.21.15.59:0/3873061155 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cd8058d30 con 0x7f1ce013d2e0 2024-04-23T14:36:13.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.954+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ce00b22e0 con 0x7f1ce013d2e0 2024-04-23T14:36:13.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.955+0000 7f1cd6a00640 1 -- 172.21.15.59:0/3873061155 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7f1cd805d030 con 0x7f1ce013d2e0 2024-04-23T14:36:13.956 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:36:13.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.957+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cbc06c920 msgr2=0x7f1cbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.957+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cbc06c920 0x7f1cbc06ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0039000 tx=0x7f1cd0031000 comp rx=0 tx=0).stop 2024-04-23T14:36:13.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.957+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce013d2e0 msgr2=0x7f1ce013d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:13.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.957+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce013d2e0 0x7f1ce013d790 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000f60 tx=0x7f1cd800e960 comp rx=0 tx=0).stop 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 shutdown_connections 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cbc06c920 0x7f1cbc06ede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ce013d2e0 0x7f1ce013d790 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ce0141d80 0x7f1ce0136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 --2- 172.21.15.59:0/3873061155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cc40039b0 0x7f1ce015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 >> 172.21.15.59:0/3873061155 conn(0x7f1ce00b6910 msgr2=0x7f1ce00704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 shutdown_connections 2024-04-23T14:36:13.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:13.958+0000 7f1cdfe00640 1 -- 172.21.15.59:0/3873061155 wait complete. 2024-04-23T14:36:13.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:14.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.191+0000 7fae06400640 1 Processor -- start 2024-04-23T14:36:14.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.192+0000 7fae06400640 1 -- start start 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.193+0000 7fae06400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 0x7fae000b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.193+0000 7fae06400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae000d0d90 0x7fae000d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.193+0000 7fae06400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae000cf030 con 0x7fae0015e7c0 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.193+0000 7fae06400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fae000b7000 con 0x7fae000d0d90 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.193+0000 7fae06400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae000cf620 con 0x7fae000b5ff0 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.194+0000 7fae05000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fae0015e7c0 0x7fae000d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55236/0 (socket says 172.21.15.59:55236) 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.194+0000 7fae05000640 1 -- 172.21.15.59:0/1288357587 learned_addr learned my addr 172.21.15.59:0/1288357587 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:14.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.194+0000 7fadffe00640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 0x7fae000b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.194+0000 7fae05a00640 1 --2- 172.21.15.59:0/1288357587 >> v2:172.21.15.76:3300/0 conn(0x7fae000d0d90 0x7fae000d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.194+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1482938394 0 0) 0x7fadf0001490 con 0x7fae0015e7c0 2024-04-23T14:36:14.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.194+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fade4003520 con 0x7fae0015e7c0 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2715535885 0 0) 0x7fade4003520 con 0x7fae0015e7c0 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 msgr2=0x7fae000b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 0x7fae000b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 >> v2:172.21.15.76:3300/0 conn(0x7fae000d0d90 msgr2=0x7fae000d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 --2- 172.21.15.59:0/1288357587 >> v2:172.21.15.76:3300/0 conn(0x7fae000d0d90 0x7fae000d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000b6a40 con 0x7fae0015e7c0 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fadf0001bc0 con 0x7fae0015e7c0 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 >> v1:172.21.15.59:6789/0 conn(0x7fae0015e7c0 legacy=0x7fae000d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039b0 0x7fade4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 0x7fade4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 0x7fade40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae000b7000 con 0x7fade40039b0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae000cf620 con 0x7fade4004d40 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fade400b9c0 con 0x7fae000b5ff0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fae05000640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039b0 0x7fade4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fae05000640 1 -- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 msgr2=0x7fade4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fadffe00640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 0x7fade4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fae05a00640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 0x7fade40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.195+0000 7fae05000640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae000b5ff0 0x7fade4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.196+0000 7fae05000640 1 -- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 msgr2=0x7fade40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.196+0000 7fae05000640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 0x7fade40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.196+0000 7fae05000640 1 -- 172.21.15.59:0/1288357587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade4003520 con 0x7fade40039b0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fae05000640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039b0 0x7fade4003dd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fadf000aa70 tx=0x7fade4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa346c17269439a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf000a690 con 0x7fade40039b0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf002f070 con 0x7fade40039b0 2024-04-23T14:36:14.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fadff400640 1 -- 172.21.15.59:0/1288357587 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0034470 con 0x7fade40039b0 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fae06400640 1 -- 172.21.15.59:0/1288357587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039b0 msgr2=0x7fade4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fae06400640 1 --2- 172.21.15.59:0/1288357587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039b0 0x7fade4003dd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fadf000aa70 tx=0x7fade4005830 comp rx=0 tx=0).stop 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fae05a00640 1 -- 172.21.15.59:0/1288357587 reap_dead start 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fae06400640 1 -- 172.21.15.59:0/1288357587 shutdown_connections 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.197+0000 7fae06400640 1 -- 172.21.15.59:0/1288357587 >> 172.21.15.59:0/1288357587 conn(0x7fae0009d920 msgr2=0x7fae0008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.198+0000 7fae06400640 1 -- 172.21.15.59:0/1288357587 shutdown_connections 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.198+0000 7fae06400640 1 -- 172.21.15.59:0/1288357587 wait complete. 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.199+0000 7fae06400640 1 Processor -- start 2024-04-23T14:36:14.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.200+0000 7fae06400640 1 -- start start 2024-04-23T14:36:14.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.200+0000 7fae06400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039b0 0x7fae001561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae06400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae001f5b00 0x7fae000b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae06400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae001f5f80 0x7fae000b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae06400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae000d2800 con 0x7fae001f5b00 2024-04-23T14:36:14.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae06400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae000cf2b0 con 0x7fae001f5f80 2024-04-23T14:36:14.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae06400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae000d1a80 con 0x7fade40039b0 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae001f5b00 0x7fae000b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae001f5b00 0x7fae000b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35500/0 (socket says 172.21.15.59:35500) 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 -- 172.21.15.59:0/1824338785 learned_addr learned my addr 172.21.15.59:0/1824338785 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae05a00640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae001f5f80 0x7fae000b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae05000640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039b0 0x7fae001561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 -- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039b0 msgr2=0x7fae001561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039b0 0x7fae001561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 -- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae001f5f80 msgr2=0x7fae000b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae001f5f80 0x7fae000b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fadffe00640 1 -- 172.21.15.59:0/1824338785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae00156700 con 0x7fae001f5b00 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.201+0000 7fae05a00640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae001f5f80 0x7fae000b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.202+0000 7fadffe00640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae001f5b00 0x7fae000b2690 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fadec010890 tx=0x7fadec010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.202+0000 7fadfcc00640 1 -- 172.21.15.59:0/1824338785 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec00f040 con 0x7fae001f5b00 2024-04-23T14:36:14.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.202+0000 7fadfcc00640 1 -- 172.21.15.59:0/1824338785 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadec00eb60 con 0x7fae001f5b00 2024-04-23T14:36:14.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.202+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae000b1630 con 0x7fae001f5b00 2024-04-23T14:36:14.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.202+0000 7fadfcc00640 1 -- 172.21.15.59:0/1824338785 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec01c400 con 0x7fae001f5b00 2024-04-23T14:36:14.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.202+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae000b1bb0 con 0x7fae001f5b00 2024-04-23T14:36:14.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.203+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0004fd90 con 0x7fae001f5b00 2024-04-23T14:36:14.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.204+0000 7fadfcc00640 1 -- 172.21.15.59:0/1824338785 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadec01c5a0 con 0x7fae001f5b00 2024-04-23T14:36:14.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.205+0000 7fadfcc00640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.205+0000 7fadfcc00640 1 -- 172.21.15.59:0/1824338785 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fadec08c240 con 0x7fae001f5b00 2024-04-23T14:36:14.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.206+0000 7fae05000640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.206+0000 7fae05000640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fadf0033000 tx=0x7fadf002d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:14.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.210+0000 7fadfcc00640 1 -- 172.21.15.59:0/1824338785 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae00156700 con 0x7fae001f5b00 2024-04-23T14:36:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.457+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae000d01a0 con 0x7fae001f5b00 2024-04-23T14:36:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.459+0000 7fadfcc00640 1 -- 172.21.15.59:0/1824338785 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5182 (secure 0 0 0) 0x7fadec018030 con 0x7fae001f5b00 2024-04-23T14:36:14.460 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:14.460 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:35:37.778594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:36:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-23T14:36:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 msgr2=0x7fade006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fadf0033000 tx=0x7fadf002d000 comp rx=0 tx=0).stop 2024-04-23T14:36:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae001f5b00 msgr2=0x7fae000b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae001f5b00 0x7fae000b2690 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fadec010890 tx=0x7fadec010d50 comp rx=0 tx=0).stop 2024-04-23T14:36:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 shutdown_connections 2024-04-23T14:36:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae001f5f80 0x7fae000b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae001f5b00 0x7fae000b2690 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 --2- 172.21.15.59:0/1824338785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039b0 0x7fae001561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 >> 172.21.15.59:0/1824338785 conn(0x7fae0009d920 msgr2=0x7fae000b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 shutdown_connections 2024-04-23T14:36:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.462+0000 7fae06400640 1 -- 172.21.15.59:0/1824338785 wait complete. 2024-04-23T14:36:14.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.689+0000 7f599aa00640 1 Processor -- start 2024-04-23T14:36:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.690+0000 7f599aa00640 1 -- start start 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.691+0000 7f599aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59940b5ff0 0x7f59940b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.691+0000 7f599aa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940cf030 con 0x7f599415e7c0 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.691+0000 7f599aa00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940b6a40 con 0x7f59940d0d90 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.691+0000 7f599aa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f59940b7000 con 0x7f59940b5ff0 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.692+0000 7f5998c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59940b5ff0 0x7f59940b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.692+0000 7f5998c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59940b5ff0 0x7f59940b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35506/0 (socket says 172.21.15.59:35506) 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.692+0000 7f5998c00640 1 -- 172.21.15.59:0/2371213194 learned_addr learned my addr 172.21.15.59:0/2371213194 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:14.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.692+0000 7f5999600640 1 --1- 172.21.15.59:0/2371213194 >> v1:172.21.15.59:6789/0 conn(0x7f599415e7c0 0x7f59940d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55246/0 (socket says 172.21.15.59:55246) 2024-04-23T14:36:14.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.692+0000 7f5998c00640 1 -- 172.21.15.59:0/2371213194 >> v1:172.21.15.76:6789/0 conn(0x7f59940d0d90 legacy=0x7f59940cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:14.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.692+0000 7f5998c00640 1 -- 172.21.15.59:0/2371213194 >> v1:172.21.15.59:6789/0 conn(0x7f599415e7c0 legacy=0x7f59940d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.692+0000 7f5998c00640 1 -- 172.21.15.59:0/2371213194 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940b6d20 con 0x7f59940b5ff0 2024-04-23T14:36:14.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.694+0000 7f5998c00640 1 --2- 172.21.15.59:0/2371213194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b6450 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5984009870 tx=0x7f598402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61516d92cc9d4fea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:14.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.694+0000 7f5993e00640 1 -- 172.21.15.59:0/2371213194 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984037070 con 0x7f59940b5ff0 2024-04-23T14:36:14.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.694+0000 7f5993e00640 1 -- 172.21.15.59:0/2371213194 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5984032440 con 0x7f59940b5ff0 2024-04-23T14:36:14.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.694+0000 7f5993e00640 1 -- 172.21.15.59:0/2371213194 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984032810 con 0x7f59940b5ff0 2024-04-23T14:36:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.695+0000 7f599aa00640 1 -- 172.21.15.59:0/2371213194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 msgr2=0x7f59940b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.695+0000 7f599aa00640 1 --2- 172.21.15.59:0/2371213194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b6450 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5984009870 tx=0x7f598402a2a0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.695+0000 7f599aa00640 1 -- 172.21.15.59:0/2371213194 shutdown_connections 2024-04-23T14:36:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.695+0000 7f599aa00640 1 --2- 172.21.15.59:0/2371213194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b6450 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.695+0000 7f599aa00640 1 -- 172.21.15.59:0/2371213194 >> 172.21.15.59:0/2371213194 conn(0x7f599409d920 msgr2=0x7f599408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.695+0000 7f599aa00640 1 -- 172.21.15.59:0/2371213194 shutdown_connections 2024-04-23T14:36:14.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.695+0000 7f599aa00640 1 -- 172.21.15.59:0/2371213194 wait complete. 2024-04-23T14:36:14.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.697+0000 7f599aa00640 1 Processor -- start 2024-04-23T14:36:14.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.697+0000 7f599aa00640 1 -- start start 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f599aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f599aa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59940d0d90 0x7f59940b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f599aa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f599415e7c0 0x7f59941e8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f599aa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f59940d2800 con 0x7f59940b5ff0 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f599aa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59940cf620 con 0x7f59940d0d90 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f599aa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f59940d1a80 con 0x7f599415e7c0 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f5999600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f5999600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35518/0 (socket says 172.21.15.59:35518) 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.698+0000 7f5999600640 1 -- 172.21.15.59:0/4153820038 learned_addr learned my addr 172.21.15.59:0/4153820038 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5998c00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59940d0d90 0x7f59940b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5999600640 1 -- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f599415e7c0 msgr2=0x7f59941e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f599a000640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f599415e7c0 0x7f59941e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5999600640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f599415e7c0 0x7f59941e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5999600640 1 -- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59940d0d90 msgr2=0x7f59940b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5999600640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59940d0d90 0x7f59940b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5999600640 1 -- 172.21.15.59:0/4153820038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940cc9b0 con 0x7f59940b5ff0 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f599a000640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f599415e7c0 0x7f59941e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5998c00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59940d0d90 0x7f59940b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.699+0000 7f5999600640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b71d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f598800c400 tx=0x7f598800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.700+0000 7f5991600640 1 -- 172.21.15.59:0/4153820038 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59880074e0 con 0x7f59940b5ff0 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.700+0000 7f5991600640 1 -- 172.21.15.59:0/4153820038 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5988007680 con 0x7f59940b5ff0 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.700+0000 7f5991600640 1 -- 172.21.15.59:0/4153820038 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988004a00 con 0x7f59940b5ff0 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.700+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59940ccc90 con 0x7f59940b5ff0 2024-04-23T14:36:14.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.700+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59940cd250 con 0x7f59940b5ff0 2024-04-23T14:36:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.701+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599404fd90 con 0x7f59940b5ff0 2024-04-23T14:36:14.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.702+0000 7f5991600640 1 -- 172.21.15.59:0/4153820038 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f598801a020 con 0x7f59940b5ff0 2024-04-23T14:36:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.703+0000 7f5991600640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597806c8f0 0x7f597806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:14.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.703+0000 7f5991600640 1 -- 172.21.15.59:0/4153820038 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f598808b3c0 con 0x7f59940b5ff0 2024-04-23T14:36:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.704+0000 7f5998c00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597806c8f0 0x7f597806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:14.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.704+0000 7f5998c00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597806c8f0 0x7f597806edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5984034000 tx=0x7f598402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.707+0000 7f5991600640 1 -- 172.21.15.59:0/4153820038 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59940cf030 con 0x7f59940b5ff0 2024-04-23T14:36:14.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.912+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f599415c5b0 con 0x7f59940b5ff0 2024-04-23T14:36:14.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.915+0000 7f5991600640 1 -- 172.21.15.59:0/4153820038 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7f59880597e0 con 0x7f59940b5ff0 2024-04-23T14:36:14.917 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:14.917 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:54.707671+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:51.101961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:52.192516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.917+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597806c8f0 msgr2=0x7f597806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597806c8f0 0x7f597806edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5984034000 tx=0x7f598402e000 comp rx=0 tx=0).stop 2024-04-23T14:36:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 msgr2=0x7f59940b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b71d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f598800c400 tx=0x7f598800c8c0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 shutdown_connections 2024-04-23T14:36:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f597806c8f0 0x7f597806edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f599415e7c0 0x7f59941e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59940d0d90 0x7f59940b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 --2- 172.21.15.59:0/4153820038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59940b5ff0 0x7f59940b71d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 >> 172.21.15.59:0/4153820038 conn(0x7f599409d920 msgr2=0x7f59940b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 shutdown_connections 2024-04-23T14:36:14.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:14.918+0000 7f599aa00640 1 -- 172.21.15.59:0/4153820038 wait complete. 2024-04-23T14:36:14.938 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:36:14.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:15.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.149+0000 7f8f1ac00640 1 Processor -- start 2024-04-23T14:36:15.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.149+0000 7f8f1ac00640 1 -- start start 2024-04-23T14:36:15.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f1ac00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f140cd760 0x7f8f140b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f1ac00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f140b4690 0x7f8f140cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f1ac00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f14160210 con 0x7f8f1415c3d0 2024-04-23T14:36:15.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f1ac00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8f141604f0 con 0x7f8f140cd760 2024-04-23T14:36:15.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f1ac00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8f141607d0 con 0x7f8f140b4690 2024-04-23T14:36:15.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f19800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8f1415c3d0 0x7f8f140cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55250/0 (socket says 172.21.15.59:55250) 2024-04-23T14:36:15.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f19800640 1 -- 172.21.15.59:0/2829922172 learned_addr learned my addr 172.21.15.59:0/2829922172 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:15.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.151+0000 7f8f18e00640 1 --2- 172.21.15.59:0/2829922172 >> v2:172.21.15.59:3300/0 conn(0x7f8f140cd760 0x7f8f140b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.152+0000 7f8f1a200640 1 --2- 172.21.15.59:0/2829922172 >> v2:172.21.15.76:3300/0 conn(0x7f8f140b4690 0x7f8f140cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.152+0000 7f8f13e00640 1 -- 172.21.15.59:0/2829922172 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257558871 0 0) 0x7f8f14160210 con 0x7f8f1415c3d0 2024-04-23T14:36:15.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.152+0000 7f8f13e00640 1 -- 172.21.15.59:0/2829922172 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8efc003520 con 0x7f8f1415c3d0 2024-04-23T14:36:15.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.152+0000 7f8f18e00640 1 -- 172.21.15.59:0/2829922172 >> v2:172.21.15.76:3300/0 conn(0x7f8f140b4690 msgr2=0x7f8f140cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:15.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.152+0000 7f8f18e00640 1 --2- 172.21.15.59:0/2829922172 >> v2:172.21.15.76:3300/0 conn(0x7f8f140b4690 0x7f8f140cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.152+0000 7f8f18e00640 1 -- 172.21.15.59:0/2829922172 >> v1:172.21.15.59:6789/0 conn(0x7f8f1415c3d0 legacy=0x7f8f140cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.152+0000 7f8f18e00640 1 -- 172.21.15.59:0/2829922172 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f14160ab0 con 0x7f8f140cd760 2024-04-23T14:36:15.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.153+0000 7f8f18e00640 1 --2- 172.21.15.59:0/2829922172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f140b4150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8f140b2de0 tx=0x7f8f0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8d9c30f5df9e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:15.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.153+0000 7f8f13e00640 1 -- 172.21.15.59:0/2829922172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00037070 con 0x7f8f140cd760 2024-04-23T14:36:15.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.153+0000 7f8f13e00640 1 -- 172.21.15.59:0/2829922172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f00032440 con 0x7f8f140cd760 2024-04-23T14:36:15.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.153+0000 7f8f13e00640 1 -- 172.21.15.59:0/2829922172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00032810 con 0x7f8f140cd760 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2829922172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 msgr2=0x7f8f140b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2829922172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f140b4150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8f140b2de0 tx=0x7f8f0002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2829922172 shutdown_connections 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2829922172 >> v2:172.21.15.76:3300/0 conn(0x7f8f140b4690 0x7f8f140cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2829922172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f140b4150 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2829922172 >> 172.21.15.59:0/2829922172 conn(0x7f8f140b21a0 msgr2=0x7f8f140b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2829922172 shutdown_connections 2024-04-23T14:36:15.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.154+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2829922172 wait complete. 2024-04-23T14:36:15.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.156+0000 7f8f1ac00640 1 Processor -- start 2024-04-23T14:36:15.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.156+0000 7f8f1ac00640 1 -- start start 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f1ac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f140b4690 0x7f8f140c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f1ac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f1415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f1ac00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f1415c3d0 0x7f8f1415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f1ac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1415c7b0 con 0x7f8f140cd760 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f1ac00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140b4ae0 con 0x7f8f1415c3d0 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f1ac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1415c930 con 0x7f8f140b4690 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f18e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f1415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f18e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f1415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35528/0 (socket says 172.21.15.59:35528) 2024-04-23T14:36:15.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f18e00640 1 -- 172.21.15.59:0/2682786252 learned_addr learned my addr 172.21.15.59:0/2682786252 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.157+0000 7f8f19800640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f140b4690 0x7f8f140c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f1a200640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f1415c3d0 0x7f8f1415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f18e00640 1 -- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f140b4690 msgr2=0x7f8f140c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f18e00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f140b4690 0x7f8f140c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f18e00640 1 -- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f1415c3d0 msgr2=0x7f8f1415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f18e00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f1415c3d0 0x7f8f1415a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f18e00640 1 -- 172.21.15.59:0/2682786252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f140c6330 con 0x7f8f140cd760 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f19800640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f140b4690 0x7f8f140c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:15.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f18e00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f1415a1c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009890 tx=0x7f8f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f11600640 1 -- 172.21.15.59:0/2682786252 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00037040 con 0x7f8f140cd760 2024-04-23T14:36:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f11600640 1 -- 172.21.15.59:0/2682786252 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f00042070 con 0x7f8f140cd760 2024-04-23T14:36:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f11600640 1 -- 172.21.15.59:0/2682786252 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00030960 con 0x7f8f140cd760 2024-04-23T14:36:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.158+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f140c84e0 con 0x7f8f140cd760 2024-04-23T14:36:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.159+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f140c8aa0 con 0x7f8f140cd760 2024-04-23T14:36:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.160+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1404fd90 con 0x7f8f140cd760 2024-04-23T14:36:15.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.161+0000 7f8f11600640 1 -- 172.21.15.59:0/2682786252 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f00030b00 con 0x7f8f140cd760 2024-04-23T14:36:15.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.162+0000 7f8f11600640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c940 0x7f8ef406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.162+0000 7f8f11600640 1 -- 172.21.15.59:0/2682786252 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8f000ac3c0 con 0x7f8f140cd760 2024-04-23T14:36:15.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.162+0000 7f8f19800640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c940 0x7f8ef406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.163+0000 7f8f19800640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c940 0x7f8ef406ee00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8f08005880 tx=0x7f8f08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:15.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.167+0000 7f8f11600640 1 -- 172.21.15.59:0/2682786252 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f0007a860 con 0x7f8f140cd760 2024-04-23T14:36:15.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.414+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f1406e400 con 0x7f8f140cd760 2024-04-23T14:36:15.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.416+0000 7f8f11600640 1 -- 172.21.15.59:0/2682786252 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5182 (secure 0 0 0) 0x7f8f0007a160 con 0x7f8f140cd760 2024-04-23T14:36:15.417 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:15.417 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:35:37.778594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:36:15.417 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c940 msgr2=0x7f8ef406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c940 0x7f8ef406ee00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8f08005880 tx=0x7f8f08009040 comp rx=0 tx=0).stop 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 msgr2=0x7f8f1415a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f1415a1c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009890 tx=0x7f8f000046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 shutdown_connections 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c940 0x7f8ef406ee00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f1415c3d0 0x7f8f1415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.418+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f140cd760 0x7f8f1415a1c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.419+0000 7f8f1ac00640 1 --2- 172.21.15.59:0/2682786252 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f140b4690 0x7f8f140c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.419+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 >> 172.21.15.59:0/2682786252 conn(0x7f8f140b21a0 msgr2=0x7f8f140af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:15.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.419+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 shutdown_connections 2024-04-23T14:36:15.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.419+0000 7f8f1ac00640 1 -- 172.21.15.59:0/2682786252 wait complete. 2024-04-23T14:36:15.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:15.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.647+0000 7f3515000640 1 Processor -- start 2024-04-23T14:36:15.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.647+0000 7f3515000640 1 -- start start 2024-04-23T14:36:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.648+0000 7f3515000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f351015ae90 0x7f35100cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.648+0000 7f3515000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351006b9d0 0x7f351006be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.648+0000 7f3515000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f351015b270 con 0x7f351006f350 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f3515000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f35100b3080 con 0x7f351015ae90 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f3515000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f35100b4930 con 0x7f351006b9d0 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f350f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f351015ae90 0x7f35100cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f350f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f351015ae90 0x7f35100cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35530/0 (socket says 172.21.15.59:35530) 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f350f400640 1 -- 172.21.15.59:0/1656781758 learned_addr learned my addr 172.21.15.59:0/1656781758 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f350f400640 1 -- 172.21.15.59:0/1656781758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351006b9d0 msgr2=0x7f351006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f350f400640 1 --2- 172.21.15.59:0/1656781758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351006b9d0 0x7f351006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f350f400640 1 -- 172.21.15.59:0/1656781758 >> v1:172.21.15.76:6789/0 conn(0x7f351006f350 legacy=0x7f3510071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:15.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.649+0000 7f350f400640 1 -- 172.21.15.59:0/1656781758 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35100b4c10 con 0x7f351015ae90 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.650+0000 7f350f400640 1 --2- 172.21.15.59:0/1656781758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351015ae90 0x7f35100cdd80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3500009870 tx=0x7f350002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f635887a905f3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.651+0000 7f350e000640 1 -- 172.21.15.59:0/1656781758 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500037070 con 0x7f351015ae90 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.651+0000 7f350e000640 1 -- 172.21.15.59:0/1656781758 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3500032440 con 0x7f351015ae90 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.651+0000 7f350e000640 1 -- 172.21.15.59:0/1656781758 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500032810 con 0x7f351015ae90 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.651+0000 7f3515000640 1 -- 172.21.15.59:0/1656781758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351015ae90 msgr2=0x7f35100cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.651+0000 7f3515000640 1 --2- 172.21.15.59:0/1656781758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351015ae90 0x7f35100cdd80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3500009870 tx=0x7f350002a210 comp rx=0 tx=0).stop 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.652+0000 7f3515000640 1 -- 172.21.15.59:0/1656781758 shutdown_connections 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.652+0000 7f3515000640 1 --2- 172.21.15.59:0/1656781758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351006b9d0 0x7f351006be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.652+0000 7f3515000640 1 --2- 172.21.15.59:0/1656781758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351015ae90 0x7f35100cdd80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.652+0000 7f3515000640 1 -- 172.21.15.59:0/1656781758 >> 172.21.15.59:0/1656781758 conn(0x7f35100b21a0 msgr2=0x7f35100b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.652+0000 7f3515000640 1 -- 172.21.15.59:0/1656781758 shutdown_connections 2024-04-23T14:36:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.652+0000 7f3515000640 1 -- 172.21.15.59:0/1656781758 wait complete. 2024-04-23T14:36:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.653+0000 7f3515000640 1 Processor -- start 2024-04-23T14:36:15.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.654+0000 7f3515000640 1 -- start start 2024-04-23T14:36:15.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.654+0000 7f3515000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f351006b9d0 0x7f351015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f3515000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351006f350 0x7f351015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f3515000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351015ae90 0x7f35101e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f3515000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35100722f0 con 0x7f351006f350 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f3515000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35100b32d0 con 0x7f351006b9d0 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f3515000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f35100b45c0 con 0x7f351015ae90 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351006f350 0x7f351015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351006f350 0x7f351015d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35546/0 (socket says 172.21.15.59:35546) 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 -- 172.21.15.59:0/1035020677 learned_addr learned my addr 172.21.15.59:0/1035020677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350f400640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f351006b9d0 0x7f351015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350fe00640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351015ae90 0x7f35101e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 -- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351015ae90 msgr2=0x7f35101e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351015ae90 0x7f35101e5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 -- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f351006b9d0 msgr2=0x7f351015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f351006b9d0 0x7f351015c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350ea00640 1 -- 172.21.15.59:0/1035020677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35101e63c0 con 0x7f351006f350 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.655+0000 7f350fe00640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351015ae90 0x7f35101e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.656+0000 7f350ea00640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351006f350 0x7f351015d890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c460 tx=0x7f34f800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.656+0000 7f3507400640 1 -- 172.21.15.59:0/1035020677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8011790 con 0x7f351006f350 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.656+0000 7f3507400640 1 -- 172.21.15.59:0/1035020677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f8017070 con 0x7f351006f350 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.656+0000 7f3507400640 1 -- 172.21.15.59:0/1035020677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8011bb0 con 0x7f351006f350 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.656+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351015c7f0 con 0x7f351006f350 2024-04-23T14:36:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.656+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351015d090 con 0x7f351006f350 2024-04-23T14:36:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.657+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3510068290 con 0x7f351006f350 2024-04-23T14:36:15.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.659+0000 7f3507400640 1 -- 172.21.15.59:0/1035020677 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34f801c460 con 0x7f351006f350 2024-04-23T14:36:15.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.659+0000 7f3507400640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f34ec06c8f0 0x7f34ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:15.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.659+0000 7f3507400640 1 -- 172.21.15.59:0/1035020677 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f34f808b980 con 0x7f351006f350 2024-04-23T14:36:15.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.660+0000 7f350f400640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f34ec06c8f0 0x7f34ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:15.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.661+0000 7f350f400640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f34ec06c8f0 0x7f34ec06edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3500034000 tx=0x7f350002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.664+0000 7f3507400640 1 -- 172.21.15.59:0/1035020677 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3510068290 con 0x7f351006f350 2024-04-23T14:36:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.908+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f351004fd90 con 0x7f351006f350 2024-04-23T14:36:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.910+0000 7f3507400640 1 -- 172.21.15.59:0/1035020677 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5182 (secure 0 0 0) 0x7f34f8059da0 con 0x7f351006f350 2024-04-23T14:36:15.911 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:15.911 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:34:53.204986+0000","modified":"2024-04-23T14:35:37.778594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6836/1087917409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1087917409},{"type":"v1","addr":"172.21.15.59:6836","nonce":1087917409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-04-23T14:36:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2024-04-23T14:36:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f34ec06c8f0 msgr2=0x7f34ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f34ec06c8f0 0x7f34ec06edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3500034000 tx=0x7f350002e000 comp rx=0 tx=0).stop 2024-04-23T14:36:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351006f350 msgr2=0x7f351015d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351006f350 0x7f351015d890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c460 tx=0x7f34f800c920 comp rx=0 tx=0).stop 2024-04-23T14:36:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 shutdown_connections 2024-04-23T14:36:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f34ec06c8f0 0x7f34ec06edb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f351015ae90 0x7f35101e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f351006f350 0x7f351015d890 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 --2- 172.21.15.59:0/1035020677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f351006b9d0 0x7f351015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 >> 172.21.15.59:0/1035020677 conn(0x7f35100b21a0 msgr2=0x7f351015a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 shutdown_connections 2024-04-23T14:36:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:15.913+0000 7f3515000640 1 -- 172.21.15.59:0/1035020677 wait complete. 2024-04-23T14:36:15.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:16.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.139+0000 7f4dd4a00640 1 Processor -- start 2024-04-23T14:36:16.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.139+0000 7f4dd4a00640 1 -- start start 2024-04-23T14:36:16.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.141+0000 7f4dd4a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4dd01454e0 0x7f4dd0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.141+0000 7f4dd4a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd00cd520 con 0x7f4dd0160910 2024-04-23T14:36:16.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.141+0000 7f4dd4a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd00cd800 con 0x7f4dd00cd020 2024-04-23T14:36:16.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.141+0000 7f4dd4a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd00cddc0 con 0x7f4dd01454e0 2024-04-23T14:36:16.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.141+0000 7f4dce800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4dd0160910 0x7f4dd00d0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55254/0 (socket says 172.21.15.59:55254) 2024-04-23T14:36:16.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.141+0000 7f4dce800640 1 -- 172.21.15.59:0/2053331372 learned_addr learned my addr 172.21.15.59:0/2053331372 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:16.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369178358 0 0) 0x7f4dd00cd520 con 0x7f4dd0160910 2024-04-23T14:36:16.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcde00640 1 --2- 172.21.15.59:0/2053331372 >> v2:172.21.15.76:3300/0 conn(0x7f4dd01454e0 0x7f4dd0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4db4003520 con 0x7f4dd0160910 2024-04-23T14:36:16.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2402642836 0 0) 0x7f4db4003520 con 0x7f4dd0160910 2024-04-23T14:36:16.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 >> v2:172.21.15.76:3300/0 conn(0x7f4dd01454e0 msgr2=0x7f4dd0145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 --2- 172.21.15.59:0/2053331372 >> v2:172.21.15.76:3300/0 conn(0x7f4dd01454e0 0x7f4dd0145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 >> v1:172.21.15.76:6789/0 conn(0x7f4dd00cd020 legacy=0x7f4dd00b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd00cdae0 con 0x7f4dd0160910 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f4dc80019f0 con 0x7f4dd0160910 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 >> v1:172.21.15.59:6789/0 conn(0x7f4dd0160910 legacy=0x7f4dd00d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db40039b0 0x7f4db4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4db4004800 0x7f4db4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40051c0 0x7f4db400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd00cddc0 con 0x7f4db40039b0 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4db400c160 con 0x7f4db40051c0 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.142+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4db400c360 con 0x7f4db4004800 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.143+0000 7f4dcde00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db40039b0 0x7f4db4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.143+0000 7f4dcde00640 1 -- 172.21.15.59:0/2053331372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4db4004800 msgr2=0x7f4db4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.143+0000 7f4dcde00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4db4004800 0x7f4db4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.143+0000 7f4dcde00640 1 -- 172.21.15.59:0/2053331372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40051c0 msgr2=0x7f4db400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:16.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.143+0000 7f4dcde00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40051c0 0x7f4db400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.143+0000 7f4dcde00640 1 -- 172.21.15.59:0/2053331372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db4003520 con 0x7f4db40039b0 2024-04-23T14:36:16.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.143+0000 7f4dce800640 1 -- 172.21.15.59:0/2053331372 reap_dead start 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.144+0000 7f4dcde00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db40039b0 0x7f4db4003dd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009ac0 tx=0x7f4db4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b7af6407da8c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.144+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0036070 con 0x7f4db40039b0 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.144+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc0031400 con 0x7f4db40039b0 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.144+0000 7f4dcca00640 1 -- 172.21.15.59:0/2053331372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0009690 con 0x7f4db40039b0 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.145+0000 7f4dd4a00640 1 -- 172.21.15.59:0/2053331372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db40039b0 msgr2=0x7f4db4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.145+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db40039b0 0x7f4db4003dd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009ac0 tx=0x7f4db4005830 comp rx=0 tx=0).stop 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.145+0000 7f4dd4a00640 1 -- 172.21.15.59:0/2053331372 shutdown_connections 2024-04-23T14:36:16.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.145+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/2053331372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4db40039b0 0x7f4db4003dd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.145+0000 7f4dd4a00640 1 -- 172.21.15.59:0/2053331372 >> 172.21.15.59:0/2053331372 conn(0x7f4dd0067ea0 msgr2=0x7f4dd006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:16.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.145+0000 7f4dd4a00640 1 -- 172.21.15.59:0/2053331372 shutdown_connections 2024-04-23T14:36:16.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.145+0000 7f4dd4a00640 1 -- 172.21.15.59:0/2053331372 wait complete. 2024-04-23T14:36:16.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.147+0000 7f4dd4a00640 1 Processor -- start 2024-04-23T14:36:16.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.147+0000 7f4dd4a00640 1 -- start start 2024-04-23T14:36:16.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dd4a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40039b0 0x7f4dd0072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dd4a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4dd015bf60 0x7f4dd0072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dd4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4dd01f6240 0x7f4dd01f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dd4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0160f00 con 0x7f4dd01f6240 2024-04-23T14:36:16.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dd4a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0160d80 con 0x7f4db40039b0 2024-04-23T14:36:16.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dd4a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd0161080 con 0x7f4dd015bf60 2024-04-23T14:36:16.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4dd01f6240 0x7f4dd01f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dce800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4dd01f6240 0x7f4dd01f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35560/0 (socket says 172.21.15.59:35560) 2024-04-23T14:36:16.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.148+0000 7f4dce800640 1 -- 172.21.15.59:0/3535646426 learned_addr learned my addr 172.21.15.59:0/3535646426 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:16.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dcd400640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4dd015bf60 0x7f4dd0072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dcde00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40039b0 0x7f4dd0072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dce800640 1 -- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4dd015bf60 msgr2=0x7f4dd0072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dce800640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4dd015bf60 0x7f4dd0072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dce800640 1 -- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40039b0 msgr2=0x7f4dd0072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dce800640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40039b0 0x7f4dd0072410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dce800640 1 -- 172.21.15.59:0/3535646426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd0139500 con 0x7f4dd01f6240 2024-04-23T14:36:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dce800640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4dd01f6240 0x7f4dd01f66f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8000f80 tx=0x7f4dc800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.149+0000 7f4dc6000640 1 -- 172.21.15.59:0/3535646426 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800a5b0 con 0x7f4dd01f6240 2024-04-23T14:36:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.150+0000 7f4dc6000640 1 -- 172.21.15.59:0/3535646426 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc800a750 con 0x7f4dd01f6240 2024-04-23T14:36:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.150+0000 7f4dc6000640 1 -- 172.21.15.59:0/3535646426 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800ab40 con 0x7f4dd01f6240 2024-04-23T14:36:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.150+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd01397e0 con 0x7f4dd01f6240 2024-04-23T14:36:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.150+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd0139da0 con 0x7f4dd01f6240 2024-04-23T14:36:16.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.151+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd0068a80 con 0x7f4dd01f6240 2024-04-23T14:36:16.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.152+0000 7f4dc6000640 1 -- 172.21.15.59:0/3535646426 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4dc8011070 con 0x7f4dd01f6240 2024-04-23T14:36:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.153+0000 7f4dc6000640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4db006c920 0x7f4db006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.153+0000 7f4dc6000640 1 -- 172.21.15.59:0/3535646426 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4dc808b220 con 0x7f4dd01f6240 2024-04-23T14:36:16.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.153+0000 7f4dcde00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4db006c920 0x7f4db006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.154+0000 7f4dcde00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4db006c920 0x7f4db006ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0033000 tx=0x7f4dc0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:16.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.158+0000 7f4dc6000640 1 -- 172.21.15.59:0/3535646426 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dc801a060 con 0x7f4dd01f6240 2024-04-23T14:36:16.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.361+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dd0160030 con 0x7f4dd01f6240 2024-04-23T14:36:16.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.363+0000 7f4dc6000640 1 -- 172.21.15.59:0/3535646426 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16605 (secure 0 0 0) 0x7f4dc800f260 con 0x7f4dd01f6240 2024-04-23T14:36:16.364 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:16.364 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":58,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:34:54.707671+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:34:51.101961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-23T14:34:52.192516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4db006c920 msgr2=0x7f4db006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4db006c920 0x7f4db006ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0033000 tx=0x7f4dc0008030 comp rx=0 tx=0).stop 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4dd01f6240 msgr2=0x7f4dd01f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4dd01f6240 0x7f4dd01f66f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8000f80 tx=0x7f4dc800ccd0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 shutdown_connections 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4db006c920 0x7f4db006ede0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4dd015bf60 0x7f4dd0072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4dd01f6240 0x7f4dd01f66f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 --2- 172.21.15.59:0/3535646426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db40039b0 0x7f4dd0072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 >> 172.21.15.59:0/3535646426 conn(0x7f4dd0067ea0 msgr2=0x7f4dd00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 shutdown_connections 2024-04-23T14:36:16.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.365+0000 7f4dd4a00640 1 -- 172.21.15.59:0/3535646426 wait complete. 2024-04-23T14:36:16.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:36:16.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.589+0000 7f7af9600640 1 Processor -- start 2024-04-23T14:36:16.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.589+0000 7f7af9600640 1 -- start start 2024-04-23T14:36:16.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.590+0000 7f7af9600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7af415bc00 0x7f7af415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.590+0000 7f7af9600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af40ce7a0 0x7f7af40cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.590+0000 7f7af9600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af4160140 con 0x7f7af406ef00 2024-04-23T14:36:16.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.590+0000 7f7af9600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7af40b2f50 con 0x7f7af415bc00 2024-04-23T14:36:16.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.590+0000 7f7af9600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7af415c180 con 0x7f7af40ce7a0 2024-04-23T14:36:16.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7aea000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7af415bc00 0x7f7af415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7af2a00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7af406ef00 0x7f7af406f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:44656/0 (socket says 172.21.15.59:44656) 2024-04-23T14:36:16.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7af2a00640 1 -- 172.21.15.59:0/3849937689 learned_addr learned my addr 172.21.15.59:0/3849937689 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:16.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7af3400640 1 --2- 172.21.15.59:0/3849937689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af40ce7a0 0x7f7af40cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7aea000640 1 -- 172.21.15.59:0/3849937689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af40ce7a0 msgr2=0x7f7af40cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:16.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7aea000640 1 --2- 172.21.15.59:0/3849937689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af40ce7a0 0x7f7af40cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7aea000640 1 -- 172.21.15.59:0/3849937689 >> v1:172.21.15.76:6789/0 conn(0x7f7af406ef00 legacy=0x7f7af406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.591+0000 7f7aea000640 1 -- 172.21.15.59:0/3849937689 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af415c460 con 0x7f7af415bc00 2024-04-23T14:36:16.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.592+0000 7f7aea000640 1 --2- 172.21.15.59:0/3849937689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af415bc00 0x7f7af415fc00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ce900b036e194 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:16.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af2000640 1 -- 172.21.15.59:0/3849937689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc007740 con 0x7f7af415bc00 2024-04-23T14:36:16.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af2000640 1 -- 172.21.15.59:0/3849937689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc0078e0 con 0x7f7af415bc00 2024-04-23T14:36:16.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af2000640 1 -- 172.21.15.59:0/3849937689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc007cb0 con 0x7f7af415bc00 2024-04-23T14:36:16.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 -- 172.21.15.59:0/3849937689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af415bc00 msgr2=0x7f7af415fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 --2- 172.21.15.59:0/3849937689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af415bc00 0x7f7af415fc00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc02a660 comp rx=0 tx=0).stop 2024-04-23T14:36:16.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 -- 172.21.15.59:0/3849937689 shutdown_connections 2024-04-23T14:36:16.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 --2- 172.21.15.59:0/3849937689 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af40ce7a0 0x7f7af40cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 --2- 172.21.15.59:0/3849937689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af415bc00 0x7f7af415fc00 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 -- 172.21.15.59:0/3849937689 >> 172.21.15.59:0/3849937689 conn(0x7f7af40b2130 msgr2=0x7f7af40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:16.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 -- 172.21.15.59:0/3849937689 shutdown_connections 2024-04-23T14:36:16.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.593+0000 7f7af9600640 1 -- 172.21.15.59:0/3849937689 wait complete. 2024-04-23T14:36:16.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.595+0000 7f7af9600640 1 Processor -- start 2024-04-23T14:36:16.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.595+0000 7f7af9600640 1 -- start start 2024-04-23T14:36:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.596+0000 7f7af9600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7af406ef00 0x7f7af413eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.596+0000 7f7af9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af40ce7a0 0x7f7af40b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.596+0000 7f7af9600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af415bc00 0x7f7af415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.596+0000 7f7af9600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7af40ce210 con 0x7f7af40ce7a0 2024-04-23T14:36:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.596+0000 7f7af9600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7af415ca20 con 0x7f7af406ef00 2024-04-23T14:36:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.596+0000 7f7af9600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7af4155ff0 con 0x7f7af415bc00 2024-04-23T14:36:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af40ce7a0 0x7f7af40b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af40ce7a0 0x7f7af40b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35574/0 (socket says 172.21.15.59:35574) 2024-04-23T14:36:16.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 -- 172.21.15.59:0/4091440275 learned_addr learned my addr 172.21.15.59:0/4091440275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:16.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7af2a00640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7af406ef00 0x7f7af413eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7af3400640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af415bc00 0x7f7af415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 -- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af415bc00 msgr2=0x7f7af415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af415bc00 0x7f7af415a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 -- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7af406ef00 msgr2=0x7f7af413eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7af406ef00 0x7f7af413eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 -- 172.21.15.59:0/4091440275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af40b4e20 con 0x7f7af40ce7a0 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.597+0000 7f7aea000640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af40ce7a0 0x7f7af40b3220 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.598+0000 7f7aeb400640 1 -- 172.21.15.59:0/4091440275 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc03f070 con 0x7f7af40ce7a0 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.598+0000 7f7aeb400640 1 -- 172.21.15.59:0/4091440275 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc031bc0 con 0x7f7af40ce7a0 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.598+0000 7f7aeb400640 1 -- 172.21.15.59:0/4091440275 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc044460 con 0x7f7af40ce7a0 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.598+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af40b50c0 con 0x7f7af40ce7a0 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.598+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af40b5680 con 0x7f7af40ce7a0 2024-04-23T14:36:16.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.599+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af4068a80 con 0x7f7af40ce7a0 2024-04-23T14:36:16.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.600+0000 7f7aeb400640 1 -- 172.21.15.59:0/4091440275 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7adc03b410 con 0x7f7af40ce7a0 2024-04-23T14:36:16.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.601+0000 7f7aeb400640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ad006c920 0x7f7ad006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:16.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.601+0000 7f7aeb400640 1 -- 172.21.15.59:0/4091440275 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7adc0ad580 con 0x7f7af40ce7a0 2024-04-23T14:36:16.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.601+0000 7f7af2a00640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ad006c920 0x7f7ad006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:16.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.602+0000 7f7af2a00640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ad006c920 0x7f7ad006ede0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000f60 tx=0x7f7ae4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:16.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.606+0000 7f7aeb400640 1 -- 172.21.15.59:0/4091440275 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7adc07b2c0 con 0x7f7af40ce7a0 2024-04-23T14:36:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.850+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7af40cd650 con 0x7f7af40ce7a0 2024-04-23T14:36:16.851 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:36:16.851+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:36:16.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.854+0000 7f7aeb400640 1 -- 172.21.15.59:0/4091440275 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f7adc07f5c0 con 0x7f7af40ce7a0 2024-04-23T14:36:16.855 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:36:16.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ad006c920 msgr2=0x7f7ad006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ad006c920 0x7f7ad006ede0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000f60 tx=0x7f7ae4009040 comp rx=0 tx=0).stop 2024-04-23T14:36:16.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af40ce7a0 msgr2=0x7f7af40b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:16.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af40ce7a0 0x7f7af40b3220 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009d90 tx=0x7f7adc004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 shutdown_connections 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ad006c920 0x7f7ad006ede0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7af415bc00 0x7f7af415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7af40ce7a0 0x7f7af40b3220 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 --2- 172.21.15.59:0/4091440275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7af406ef00 0x7f7af413eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 >> 172.21.15.59:0/4091440275 conn(0x7f7af40b2130 msgr2=0x7f7af40b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.858+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 shutdown_connections 2024-04-23T14:36:16.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:16.859+0000 7f7af9600640 1 -- 172.21.15.59:0/4091440275 wait complete. 2024-04-23T14:36:16.882 INFO:tasks.ceph.mds.c.smithi059.stderr: -6698> 2024-04-23T14:34:58.109+0000 7f3119800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:36:16.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:36:16.889 INFO:tasks.ceph.mds.c.smithi059.stderr: -4923> 2024-04-23T14:35:04.457+0000 7f3114800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6318=0-4194304@1} caps={6318=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6318 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cd9aa38b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:36:16.892 INFO:tasks.ceph.mds.c.smithi059.stderr: -2992> 2024-04-23T14:35:36.594+0000 7f3114800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-23T14:35:02.847146+0000 1=1+0) caps={6318=pAsLsXsFs/-@2},l=6318(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55cd9aa38b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:36:16.952 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:36:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.074+0000 7fa017e00640 1 Processor -- start 2024-04-23T14:36:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.074+0000 7fa017e00640 1 -- start start 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.075+0000 7fa017e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa01815c410 0x7fa0180d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.075+0000 7fa017e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0180d0b10 0x7fa0180af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa017e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa018160120 con 0x7fa0180af5c0 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa017e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa0180d0f70 con 0x7fa0180d0b10 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa017e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa018160400 con 0x7fa01815c410 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa016000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0180d0b10 0x7fa0180af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa017400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa0180af5c0 0x7fa0180b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55272/0 (socket says 172.21.15.59:55272) 2024-04-23T14:36:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa017400640 1 -- 172.21.15.59:0/786377854 learned_addr learned my addr 172.21.15.59:0/786377854 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa016a00640 1 --2- 172.21.15.59:0/786377854 >> v2:172.21.15.76:3300/0 conn(0x7fa01815c410 0x7fa0180d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa015600640 1 -- 172.21.15.59:0/786377854 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345664553 0 0) 0x7fa018160120 con 0x7fa0180af5c0 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.076+0000 7fa015600640 1 -- 172.21.15.59:0/786377854 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ffc003520 con 0x7fa0180af5c0 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.077+0000 7fa016000640 1 -- 172.21.15.59:0/786377854 >> v2:172.21.15.76:3300/0 conn(0x7fa01815c410 msgr2=0x7fa0180d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.077+0000 7fa016000640 1 --2- 172.21.15.59:0/786377854 >> v2:172.21.15.76:3300/0 conn(0x7fa01815c410 0x7fa0180d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.077+0000 7fa016000640 1 -- 172.21.15.59:0/786377854 >> v1:172.21.15.59:6789/0 conn(0x7fa0180af5c0 legacy=0x7fa0180b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.077+0000 7fa016000640 1 -- 172.21.15.59:0/786377854 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0181606e0 con 0x7fa0180d0b10 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.078+0000 7fa016000640 1 --2- 172.21.15.59:0/786377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180d0b10 0x7fa0180af080 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa0180741a0 tx=0x7fa00402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f582a173d064d580 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.078+0000 7fa015600640 1 -- 172.21.15.59:0/786377854 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004037070 con 0x7fa0180d0b10 2024-04-23T14:36:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.078+0000 7fa015600640 1 -- 172.21.15.59:0/786377854 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa004032440 con 0x7fa0180d0b10 2024-04-23T14:36:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.078+0000 7fa015600640 1 -- 172.21.15.59:0/786377854 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004032810 con 0x7fa0180d0b10 2024-04-23T14:36:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.078+0000 7fa017e00640 1 -- 172.21.15.59:0/786377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180d0b10 msgr2=0x7fa0180af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.079+0000 7fa017e00640 1 --2- 172.21.15.59:0/786377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180d0b10 0x7fa0180af080 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa0180741a0 tx=0x7fa00402a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.079+0000 7fa017e00640 1 -- 172.21.15.59:0/786377854 shutdown_connections 2024-04-23T14:36:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.079+0000 7fa017e00640 1 --2- 172.21.15.59:0/786377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180d0b10 0x7fa0180af080 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.079+0000 7fa017e00640 1 --2- 172.21.15.59:0/786377854 >> v2:172.21.15.76:3300/0 conn(0x7fa01815c410 0x7fa0180d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.079+0000 7fa017e00640 1 -- 172.21.15.59:0/786377854 >> 172.21.15.59:0/786377854 conn(0x7fa01806bdf0 msgr2=0x7fa01806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.079+0000 7fa017e00640 1 -- 172.21.15.59:0/786377854 shutdown_connections 2024-04-23T14:36:17.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.079+0000 7fa017e00640 1 -- 172.21.15.59:0/786377854 wait complete. 2024-04-23T14:36:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.080+0000 7fa017e00640 1 Processor -- start 2024-04-23T14:36:17.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.081+0000 7fa017e00640 1 -- start start 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.081+0000 7fa017e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180af5c0 0x7fa018157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.081+0000 7fa017e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0180d0b10 0x7fa018073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.081+0000 7fa017e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa01815c410 0x7fa018073840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.081+0000 7fa017e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa01815c7f0 con 0x7fa0180af5c0 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.081+0000 7fa017e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180afa30 con 0x7fa0180d0b10 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.081+0000 7fa017e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa018157d40 con 0x7fa01815c410 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180af5c0 0x7fa018157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180af5c0 0x7fa018157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35612/0 (socket says 172.21.15.59:35612) 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 -- 172.21.15.59:0/3805675338 learned_addr learned my addr 172.21.15.59:0/3805675338 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:17.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016000640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0180d0b10 0x7fa018073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 -- 172.21.15.59:0/3805675338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa01815c410 msgr2=0x7fa018073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa01815c410 0x7fa018073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 -- 172.21.15.59:0/3805675338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0180d0b10 msgr2=0x7fa018073300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0180d0b10 0x7fa018073300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.082+0000 7fa016a00640 1 -- 172.21.15.59:0/3805675338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0180b1600 con 0x7fa0180af5c0 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.083+0000 7fa016a00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180af5c0 0x7fa018157800 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d5f0 tx=0x7fa00800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.083+0000 7fa00ea00640 1 -- 172.21.15.59:0/3805675338 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0080125f0 con 0x7fa0180af5c0 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.083+0000 7fa00ea00640 1 -- 172.21.15.59:0/3805675338 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa008019070 con 0x7fa0180af5c0 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.083+0000 7fa00ea00640 1 -- 172.21.15.59:0/3805675338 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0080129d0 con 0x7fa0180af5c0 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.083+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0180b18e0 con 0x7fa0180af5c0 2024-04-23T14:36:17.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.083+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0180b1ea0 con 0x7fa0180af5c0 2024-04-23T14:36:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.084+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa01806c730 con 0x7fa0180af5c0 2024-04-23T14:36:17.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.085+0000 7fa00ea00640 1 -- 172.21.15.59:0/3805675338 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa008012b70 con 0x7fa0180af5c0 2024-04-23T14:36:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.086+0000 7fa00ea00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ff806c8f0 0x7f9ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.086+0000 7fa00ea00640 1 -- 172.21.15.59:0/3805675338 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7fa00808c010 con 0x7fa0180af5c0 2024-04-23T14:36:17.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.087+0000 7fa016000640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ff806c8f0 0x7f9ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:17.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.087+0000 7fa016000640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ff806c8f0 0x7f9ff806edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa004034000 tx=0x7fa00402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:17.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.090+0000 7fa00ea00640 1 -- 172.21.15.59:0/3805675338 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa01806c730 con 0x7fa0180af5c0 2024-04-23T14:36:17.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.344+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa01804fd90 con 0x7fa0180af5c0 2024-04-23T14:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.858+0000 7fa00ea00640 1 -- 172.21.15.59:0/3805675338 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7fa00805a340 con 0x7fa0180af5c0 2024-04-23T14:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ff806c8f0 msgr2=0x7f9ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ff806c8f0 0x7f9ff806edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa004034000 tx=0x7fa00402e000 comp rx=0 tx=0).stop 2024-04-23T14:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180af5c0 msgr2=0x7fa018157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180af5c0 0x7fa018157800 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d5f0 tx=0x7fa00800dab0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 shutdown_connections 2024-04-23T14:36:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ff806c8f0 0x7f9ff806edb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa01815c410 0x7fa018073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0180d0b10 0x7fa018073300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 --2- 172.21.15.59:0/3805675338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0180af5c0 0x7fa018157800 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 >> 172.21.15.59:0/3805675338 conn(0x7fa01806bdf0 msgr2=0x7fa018158470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 shutdown_connections 2024-04-23T14:36:17.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:17.862+0000 7fa017e00640 1 -- 172.21.15.59:0/3805675338 wait complete. 2024-04-23T14:36:17.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:36:18.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.153+0000 7f84dae00640 1 Processor -- start 2024-04-23T14:36:18.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.153+0000 7f84dae00640 1 -- start start 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.154+0000 7f84dae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d415e8e0 0x7f84d40cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.154+0000 7f84dae00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f84d406cbb0 0x7f84d406d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.154+0000 7f84dae00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d40cf070 con 0x7f84d40d4210 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.154+0000 7f84dae00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f84d40cf2f0 con 0x7f84d406cbb0 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.154+0000 7f84dae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f84d40d1c60 con 0x7f84d415e8e0 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.155+0000 7f84d9000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f84d40d4210 0x7f84d40d4690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55288/0 (socket says 172.21.15.59:55288) 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.155+0000 7f84d9000640 1 -- 172.21.15.59:0/2605706610 learned_addr learned my addr 172.21.15.59:0/2605706610 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.155+0000 7f84da400640 1 --2- 172.21.15.59:0/2605706610 >> v2:172.21.15.59:3300/0 conn(0x7f84d406cbb0 0x7f84d406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:18.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.155+0000 7f84d9a00640 1 --2- 172.21.15.59:0/2605706610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d415e8e0 0x7f84d40cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.155+0000 7f84d3e00640 1 -- 172.21.15.59:0/2605706610 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2261656381 0 0) 0x7f84c0001290 con 0x7f84d40d4210 2024-04-23T14:36:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.156+0000 7f84d3e00640 1 -- 172.21.15.59:0/2605706610 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84bc003520 con 0x7f84d40d4210 2024-04-23T14:36:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.156+0000 7f84da400640 1 -- 172.21.15.59:0/2605706610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d415e8e0 msgr2=0x7f84d40cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.156+0000 7f84da400640 1 --2- 172.21.15.59:0/2605706610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d415e8e0 0x7f84d40cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.156+0000 7f84da400640 1 -- 172.21.15.59:0/2605706610 >> v1:172.21.15.59:6789/0 conn(0x7f84d40d4210 legacy=0x7f84d40d4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:18.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.156+0000 7f84da400640 1 -- 172.21.15.59:0/2605706610 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d40cf5b0 con 0x7f84d406cbb0 2024-04-23T14:36:18.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.157+0000 7f84da400640 1 --2- 172.21.15.59:0/2605706610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d406cbb0 0x7f84d406d040 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009dc0 tx=0x7f84cc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfa60bfe9e3f08e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:18.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.157+0000 7f84d3e00640 1 -- 172.21.15.59:0/2605706610 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc007740 con 0x7f84d406cbb0 2024-04-23T14:36:18.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.157+0000 7f84d3e00640 1 -- 172.21.15.59:0/2605706610 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84cc0078e0 con 0x7f84d406cbb0 2024-04-23T14:36:18.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.157+0000 7f84d3e00640 1 -- 172.21.15.59:0/2605706610 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc007cb0 con 0x7f84d406cbb0 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 -- 172.21.15.59:0/2605706610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d406cbb0 msgr2=0x7f84d406d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 --2- 172.21.15.59:0/2605706610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d406cbb0 0x7f84d406d040 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009dc0 tx=0x7f84cc02a5f0 comp rx=0 tx=0).stop 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 -- 172.21.15.59:0/2605706610 shutdown_connections 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 --2- 172.21.15.59:0/2605706610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d406cbb0 0x7f84d406d040 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 --2- 172.21.15.59:0/2605706610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d415e8e0 0x7f84d40cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 -- 172.21.15.59:0/2605706610 >> 172.21.15.59:0/2605706610 conn(0x7f84d408e060 msgr2=0x7f84d409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 -- 172.21.15.59:0/2605706610 shutdown_connections 2024-04-23T14:36:18.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.158+0000 7f84dae00640 1 -- 172.21.15.59:0/2605706610 wait complete. 2024-04-23T14:36:18.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.160+0000 7f84dae00640 1 Processor -- start 2024-04-23T14:36:18.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.160+0000 7f84dae00640 1 -- start start 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84dae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d406cbb0 0x7f84d41584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84dae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d40d4210 0x7f84d40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84dae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d415e8e0 0x7f84d40b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84dae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84d415ecc0 con 0x7f84d415e8e0 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84dae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84d406d580 con 0x7f84d40d4210 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84dae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f84d40d02f0 con 0x7f84d406cbb0 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84da400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d415e8e0 0x7f84d40b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84da400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d415e8e0 0x7f84d40b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35640/0 (socket says 172.21.15.59:35640) 2024-04-23T14:36:18.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84da400640 1 -- 172.21.15.59:0/3280311181 learned_addr learned my addr 172.21.15.59:0/3280311181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84d9a00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d406cbb0 0x7f84d41584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.161+0000 7f84d9000640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d40d4210 0x7f84d40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84da400640 1 -- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d406cbb0 msgr2=0x7f84d41584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84da400640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d406cbb0 0x7f84d41584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84da400640 1 -- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d40d4210 msgr2=0x7f84d40b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84da400640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d40d4210 0x7f84d40b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84da400640 1 -- 172.21.15.59:0/3280311181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d415f090 con 0x7f84d415e8e0 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84da400640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d415e8e0 0x7f84d40b2470 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02abe0 tx=0x7f84cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84d1600640 1 -- 172.21.15.59:0/3280311181 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc040070 con 0x7f84d415e8e0 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84d1600640 1 -- 172.21.15.59:0/3280311181 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84cc031bc0 con 0x7f84d415e8e0 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84d1600640 1 -- 172.21.15.59:0/3280311181 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc009450 con 0x7f84d415e8e0 2024-04-23T14:36:18.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d415f370 con 0x7f84d415e8e0 2024-04-23T14:36:18.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.162+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d415f930 con 0x7f84d415e8e0 2024-04-23T14:36:18.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.164+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d404fd90 con 0x7f84d415e8e0 2024-04-23T14:36:18.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.165+0000 7f84d1600640 1 -- 172.21.15.59:0/3280311181 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84cc039040 con 0x7f84d415e8e0 2024-04-23T14:36:18.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.165+0000 7f84d1600640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84b406c8f0 0x7f84b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:18.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.166+0000 7f84d1600640 1 -- 172.21.15.59:0/3280311181 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7f84cc0ad3d0 con 0x7f84d415e8e0 2024-04-23T14:36:18.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.166+0000 7f84d9a00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84b406c8f0 0x7f84b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:18.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.167+0000 7f84d9a00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84b406c8f0 0x7f84b406edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009a30 tx=0x7f84c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:18.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.170+0000 7f84d1600640 1 -- 172.21.15.59:0/3280311181 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84cc04d350 con 0x7f84d415e8e0 2024-04-23T14:36:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:18.374+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f84d415fb00 con 0x7f84d415e8e0 2024-04-23T14:36:19.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.115+0000 7f84d1600640 1 -- 172.21.15.59:0/3280311181 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7f84cc07b700 con 0x7f84d415e8e0 2024-04-23T14:36:19.116 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:36:19.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84b406c8f0 msgr2=0x7f84b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:19.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84b406c8f0 0x7f84b406edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009a30 tx=0x7f84c8008040 comp rx=0 tx=0).stop 2024-04-23T14:36:19.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d415e8e0 msgr2=0x7f84d40b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:19.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d415e8e0 0x7f84d40b2470 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f84cc02abe0 tx=0x7f84cc004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 shutdown_connections 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f84b406c8f0 0x7f84b406edb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84d415e8e0 0x7f84d40b2470 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d40d4210 0x7f84d40b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 --2- 172.21.15.59:0/3280311181 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84d406cbb0 0x7f84d41584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 >> 172.21.15.59:0/3280311181 conn(0x7f84d408e060 msgr2=0x7f84d406d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.119+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 shutdown_connections 2024-04-23T14:36:19.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.120+0000 7f84dae00640 1 -- 172.21.15.59:0/3280311181 wait complete. 2024-04-23T14:36:19.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:36:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.349+0000 7f1fe9a00640 1 Processor -- start 2024-04-23T14:36:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.349+0000 7f1fe9a00640 1 -- start start 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe9a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1fe40cd110 0x7f1fe4072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1fe406f050 0x7f1fe40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe9a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe406f7c0 con 0x7f1fe4161bc0 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe9a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe406faa0 con 0x7f1fe406f050 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe9a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe406fd80 con 0x7f1fe40cd110 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1fe4161bc0 0x7f1fe40cca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55300/0 (socket says 172.21.15.59:55300) 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe3e00640 1 -- 172.21.15.59:0/975852129 learned_addr learned my addr 172.21.15.59:0/975852129 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe9000640 1 --2- 172.21.15.59:0/975852129 >> v2:172.21.15.59:3300/0 conn(0x7f1fe406f050 0x7f1fe40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.351+0000 7f1fe2a00640 1 -- 172.21.15.59:0/975852129 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538020391 0 0) 0x7f1fe406f7c0 con 0x7f1fe4161bc0 2024-04-23T14:36:19.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.352+0000 7f1fe3400640 1 --2- 172.21.15.59:0/975852129 >> v2:172.21.15.76:3300/0 conn(0x7f1fe40cd110 0x7f1fe4072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.352+0000 7f1fe2a00640 1 -- 172.21.15.59:0/975852129 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fc8003520 con 0x7f1fe4161bc0 2024-04-23T14:36:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.352+0000 7f1fe9000640 1 -- 172.21.15.59:0/975852129 >> v2:172.21.15.76:3300/0 conn(0x7f1fe40cd110 msgr2=0x7f1fe4072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.352+0000 7f1fe9000640 1 --2- 172.21.15.59:0/975852129 >> v2:172.21.15.76:3300/0 conn(0x7f1fe40cd110 0x7f1fe4072fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.352+0000 7f1fe9000640 1 -- 172.21.15.59:0/975852129 >> v1:172.21.15.59:6789/0 conn(0x7f1fe4161bc0 legacy=0x7f1fe40cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.352+0000 7f1fe9000640 1 -- 172.21.15.59:0/975852129 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe40ce320 con 0x7f1fe406f050 2024-04-23T14:36:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.353+0000 7f1fe9000640 1 --2- 172.21.15.59:0/975852129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe406f050 0x7f1fe40ce840 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4074200 tx=0x7f1fdc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732839075ed11109 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.353+0000 7f1fe2a00640 1 -- 172.21.15.59:0/975852129 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc037070 con 0x7f1fe406f050 2024-04-23T14:36:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.353+0000 7f1fe2a00640 1 -- 172.21.15.59:0/975852129 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc032440 con 0x7f1fe406f050 2024-04-23T14:36:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.353+0000 7f1fe2a00640 1 -- 172.21.15.59:0/975852129 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc032810 con 0x7f1fe406f050 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 -- 172.21.15.59:0/975852129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe406f050 msgr2=0x7f1fe40ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/975852129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe406f050 0x7f1fe40ce840 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4074200 tx=0x7f1fdc02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 -- 172.21.15.59:0/975852129 shutdown_connections 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/975852129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe406f050 0x7f1fe40ce840 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/975852129 >> v2:172.21.15.76:3300/0 conn(0x7f1fe40cd110 0x7f1fe4072fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 -- 172.21.15.59:0/975852129 >> 172.21.15.59:0/975852129 conn(0x7f1fe40b1f90 msgr2=0x7f1fe40b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 -- 172.21.15.59:0/975852129 shutdown_connections 2024-04-23T14:36:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.354+0000 7f1fe9a00640 1 -- 172.21.15.59:0/975852129 wait complete. 2024-04-23T14:36:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.356+0000 7f1fe9a00640 1 Processor -- start 2024-04-23T14:36:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.356+0000 7f1fe9a00640 1 -- start start 2024-04-23T14:36:19.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe9a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fe406f050 0x7f1fe41596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:19.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe40cd110 0x7f1fe415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:19.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe9a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fe4161bc0 0x7f1fe415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe40ced80 con 0x7f1fe40cd110 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe9a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4161fa0 con 0x7f1fe4161bc0 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe9a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe40cef00 con 0x7f1fe406f050 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe40cd110 0x7f1fe415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe40cd110 0x7f1fe415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35662/0 (socket says 172.21.15.59:35662) 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 -- 172.21.15.59:0/3274665478 learned_addr learned my addr 172.21.15.59:0/3274665478 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 -- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fe406f050 msgr2=0x7f1fe41596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3e00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fe406f050 0x7f1fe41596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe9000640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fe4161bc0 0x7f1fe415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fe406f050 0x7f1fe41596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 -- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fe4161bc0 msgr2=0x7f1fe415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fe4161bc0 0x7f1fe415f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.357+0000 7f1fe3400640 1 -- 172.21.15.59:0/3274665478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe415bb50 con 0x7f1fe40cd110 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.358+0000 7f1fe3400640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe40cd110 0x7f1fe415f080 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00d830 tx=0x7f1fcc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.358+0000 7f1fdbe00640 1 -- 172.21.15.59:0/3274665478 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00b6c0 con 0x7f1fe40cd110 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.358+0000 7f1fdbe00640 1 -- 172.21.15.59:0/3274665478 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fcc019070 con 0x7f1fe40cd110 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.358+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe4159c00 con 0x7f1fe40cd110 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.358+0000 7f1fdbe00640 1 -- 172.21.15.59:0/3274665478 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00ba80 con 0x7f1fe40cd110 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.358+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe40b5620 con 0x7f1fe40cd110 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.359+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe404fd90 con 0x7f1fe40cd110 2024-04-23T14:36:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.360+0000 7f1fdbe00640 1 -- 172.21.15.59:0/3274665478 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fcc01e460 con 0x7f1fe40cd110 2024-04-23T14:36:19.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.362+0000 7f1fdbe00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fc006c920 0x7f1fc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:19.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.362+0000 7f1fdbe00640 1 -- 172.21.15.59:0/3274665478 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5357+0+0 (secure 0 0 0) 0x7f1fcc08c140 con 0x7f1fe40cd110 2024-04-23T14:36:19.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.362+0000 7f1fe3e00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fc006c920 0x7f1fc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:19.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.363+0000 7f1fe3e00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fc006c920 0x7f1fc006ede0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40012e0 tx=0x7f1fd4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:19.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.367+0000 7f1fdbe00640 1 -- 172.21.15.59:0/3274665478 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fcc05a630 con 0x7f1fe40cd110 2024-04-23T14:36:19.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:19.577+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1fe40731c0 con 0x7f1fe40cd110 2024-04-23T14:36:19.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:36:19.769 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:36:20.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.119+0000 7f1fdbe00640 1 -- 172.21.15.59:0/3274665478 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7f1fcc059f30 con 0x7f1fe40cd110 2024-04-23T14:36:20.120 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:36:20.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.124+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fc006c920 msgr2=0x7f1fc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.124+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fc006c920 0x7f1fc006ede0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40012e0 tx=0x7f1fd4009040 comp rx=0 tx=0).stop 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.124+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe40cd110 msgr2=0x7f1fe415f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.124+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe40cd110 0x7f1fe415f080 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00d830 tx=0x7f1fcc00dcf0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.125+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 shutdown_connections 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.125+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fc006c920 0x7f1fc006ede0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.125+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fe4161bc0 0x7f1fe415f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.126+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fe40cd110 0x7f1fe415f080 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.126+0000 7f1fe9a00640 1 --2- 172.21.15.59:0/3274665478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fe406f050 0x7f1fe41596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.126+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 >> 172.21.15.59:0/3274665478 conn(0x7f1fe40b1f90 msgr2=0x7f1fe40afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.126+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 shutdown_connections 2024-04-23T14:36:20.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.126+0000 7f1fe9a00640 1 -- 172.21.15.59:0/3274665478 wait complete. 2024-04-23T14:36:20.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:36:20.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.348+0000 7fc178a00640 1 Processor -- start 2024-04-23T14:36:20.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.349+0000 7fc178a00640 1 -- start start 2024-04-23T14:36:20.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.350+0000 7fc178a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc17415ed20 0x7fc1740d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.350+0000 7fc178a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1740b3600 0x7fc1740d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.350+0000 7fc178a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1740b5da0 con 0x7fc1740b3f10 2024-04-23T14:36:20.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.350+0000 7fc178a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc17415f100 con 0x7fc1740b3600 2024-04-23T14:36:20.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.350+0000 7fc178a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc1740d0a20 con 0x7fc17415ed20 2024-04-23T14:36:20.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc171e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1740b3600 0x7fc1740d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc171e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1740b3600 0x7fc1740d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40418/0 (socket says 172.21.15.59:40418) 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc171e00640 1 -- 172.21.15.59:0/2803595291 learned_addr learned my addr 172.21.15.59:0/2803595291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc171e00640 1 -- 172.21.15.59:0/2803595291 >> v2:172.21.15.76:3300/0 conn(0x7fc17415ed20 msgr2=0x7fc1740d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc172800640 1 --2- 172.21.15.59:0/2803595291 >> v2:172.21.15.76:3300/0 conn(0x7fc17415ed20 0x7fc1740d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc171e00640 1 --2- 172.21.15.59:0/2803595291 >> v2:172.21.15.76:3300/0 conn(0x7fc17415ed20 0x7fc1740d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc171e00640 1 -- 172.21.15.59:0/2803595291 >> v1:172.21.15.76:6789/0 conn(0x7fc1740b3f10 legacy=0x7fc1740b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc171e00640 1 -- 172.21.15.59:0/2803595291 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1740d2ed0 con 0x7fc1740b3600 2024-04-23T14:36:20.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.351+0000 7fc172800640 1 --2- 172.21.15.59:0/2803595291 >> v2:172.21.15.76:3300/0 conn(0x7fc17415ed20 0x7fc1740d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:20.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.352+0000 7fc171e00640 1 --2- 172.21.15.59:0/2803595291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1740b3600 0x7fc1740d13e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc17408e2a0 tx=0x7fc16402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46aeded3eabc359 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:20.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc171400640 1 -- 172.21.15.59:0/2803595291 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164036070 con 0x7fc1740b3600 2024-04-23T14:36:20.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc171400640 1 -- 172.21.15.59:0/2803595291 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc164031600 con 0x7fc1740b3600 2024-04-23T14:36:20.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc171400640 1 -- 172.21.15.59:0/2803595291 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1640319d0 con 0x7fc1740b3600 2024-04-23T14:36:20.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc178a00640 1 -- 172.21.15.59:0/2803595291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1740b3600 msgr2=0x7fc1740d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc178a00640 1 --2- 172.21.15.59:0/2803595291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1740b3600 0x7fc1740d13e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc17408e2a0 tx=0x7fc16402a5d0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc178a00640 1 -- 172.21.15.59:0/2803595291 shutdown_connections 2024-04-23T14:36:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc178a00640 1 --2- 172.21.15.59:0/2803595291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1740b3600 0x7fc1740d13e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc178a00640 1 --2- 172.21.15.59:0/2803595291 >> v2:172.21.15.76:3300/0 conn(0x7fc17415ed20 0x7fc1740d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.353+0000 7fc178a00640 1 -- 172.21.15.59:0/2803595291 >> 172.21.15.59:0/2803595291 conn(0x7fc17408bf90 msgr2=0x7fc1740d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.354+0000 7fc178a00640 1 -- 172.21.15.59:0/2803595291 shutdown_connections 2024-04-23T14:36:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.354+0000 7fc178a00640 1 -- 172.21.15.59:0/2803595291 wait complete. 2024-04-23T14:36:20.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.355+0000 7fc178a00640 1 Processor -- start 2024-04-23T14:36:20.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.355+0000 7fc178a00640 1 -- start start 2024-04-23T14:36:20.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.356+0000 7fc178a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1740b3600 0x7fc1740caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.356+0000 7fc178a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1740b3f10 0x7fc1740b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.356+0000 7fc178a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc17415ed20 0x7fc17414c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.356+0000 7fc178a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1740d0290 con 0x7fc17415ed20 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.356+0000 7fc178a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc1740d3490 con 0x7fc1740b3f10 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.356+0000 7fc178a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1740d0410 con 0x7fc1740b3600 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc17415ed20 0x7fc17414c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc17415ed20 0x7fc17414c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40422/0 (socket says 172.21.15.59:40422) 2024-04-23T14:36:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 -- 172.21.15.59:0/1346259662 learned_addr learned my addr 172.21.15.59:0/1346259662 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc171e00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1740b3f10 0x7fc1740b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc172800640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1740b3600 0x7fc1740caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 -- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1740b3600 msgr2=0x7fc1740caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1740b3600 0x7fc1740caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 -- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1740b3f10 msgr2=0x7fc1740b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1740b3f10 0x7fc1740b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 -- 172.21.15.59:0/1346259662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1740b1610 con 0x7fc17415ed20 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.357+0000 7fc173200640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc17415ed20 0x7fc17414c8c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00c440 tx=0x7fc16c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.358+0000 7fc16aa00640 1 -- 172.21.15.59:0/1346259662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c011790 con 0x7fc17415ed20 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.358+0000 7fc16aa00640 1 -- 172.21.15.59:0/1346259662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc16c017070 con 0x7fc17415ed20 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.358+0000 7fc16aa00640 1 -- 172.21.15.59:0/1346259662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c011b70 con 0x7fc17415ed20 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.358+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1740b18f0 con 0x7fc17415ed20 2024-04-23T14:36:20.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.358+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1740b1eb0 con 0x7fc17415ed20 2024-04-23T14:36:20.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.359+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17404fd90 con 0x7fc17415ed20 2024-04-23T14:36:20.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.360+0000 7fc16aa00640 1 -- 172.21.15.59:0/1346259662 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc16c011d10 con 0x7fc17415ed20 2024-04-23T14:36:20.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.361+0000 7fc16aa00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc15006c8f0 0x7fc15006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.362+0000 7fc16aa00640 1 -- 172.21.15.59:0/1346259662 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fc16c08b630 con 0x7fc17415ed20 2024-04-23T14:36:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.362+0000 7fc172800640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc15006c8f0 0x7fc15006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.363+0000 7fc172800640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc15006c8f0 0x7fc15006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00b840 tx=0x7fc15c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:20.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.366+0000 7fc16aa00640 1 -- 172.21.15.59:0/1346259662 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc16c059d60 con 0x7fc17415ed20 2024-04-23T14:36:20.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.576+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc1740b2bf0 con 0x7fc17415ed20 2024-04-23T14:36:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.583+0000 7fc16aa00640 1 -- 172.21.15.59:0/1346259662 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fc16c059660 con 0x7fc17415ed20 2024-04-23T14:36:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc15006c8f0 msgr2=0x7fc15006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc15006c8f0 0x7fc15006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00b840 tx=0x7fc15c008040 comp rx=0 tx=0).stop 2024-04-23T14:36:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc17415ed20 msgr2=0x7fc17414c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc17415ed20 0x7fc17414c8c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00c440 tx=0x7fc16c00c900 comp rx=0 tx=0).stop 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 shutdown_connections 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc15006c8f0 0x7fc15006edb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc17415ed20 0x7fc17414c8c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1740b3f10 0x7fc1740b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 --2- 172.21.15.59:0/1346259662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1740b3600 0x7fc1740caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 >> 172.21.15.59:0/1346259662 conn(0x7fc17408bf90 msgr2=0x7fc17415d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 shutdown_connections 2024-04-23T14:36:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.586+0000 7fc178a00640 1 -- 172.21.15.59:0/1346259662 wait complete. 2024-04-23T14:36:20.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-23T14:36:20.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.817+0000 7f22c8e00640 1 Processor -- start 2024-04-23T14:36:20.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.817+0000 7f22c8e00640 1 -- start start 2024-04-23T14:36:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.818+0000 7f22c8e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.818+0000 7f22c8e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f22c406c0e0 0x7f22c40b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.818+0000 7f22c8e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c4160870 con 0x7f22c41603d0 2024-04-23T14:36:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.818+0000 7f22c8e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f22c40b2560 con 0x7f22c406c0e0 2024-04-23T14:36:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.818+0000 7f22c8e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22c4160b30 con 0x7f22c40b0ff0 2024-04-23T14:36:20.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.819+0000 7f22c1e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.819+0000 7f22c1e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:42732/0 (socket says 172.21.15.59:42732) 2024-04-23T14:36:20.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.819+0000 7f22c1e00640 1 -- 172.21.15.59:0/940740937 learned_addr learned my addr 172.21.15.59:0/940740937 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:20.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.819+0000 7f22c2800640 1 --2- 172.21.15.59:0/940740937 >> v2:172.21.15.76:3300/0 conn(0x7f22c406c0e0 0x7f22c40b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.820+0000 7f22c1e00640 1 -- 172.21.15.59:0/940740937 >> v2:172.21.15.76:3300/0 conn(0x7f22c406c0e0 msgr2=0x7f22c40b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.820+0000 7f22c1e00640 1 --2- 172.21.15.59:0/940740937 >> v2:172.21.15.76:3300/0 conn(0x7f22c406c0e0 0x7f22c40b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.820+0000 7f22c1e00640 1 -- 172.21.15.59:0/940740937 >> v1:172.21.15.76:6789/0 conn(0x7f22c41603d0 legacy=0x7f22c406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.820+0000 7f22c1e00640 1 -- 172.21.15.59:0/940740937 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c4160e10 con 0x7f22c40b0ff0 2024-04-23T14:36:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.820+0000 7f22c0a00640 1 -- 172.21.15.59:0/940740937 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49229170 0 0) 0x7f22c4160870 con 0x7f22c41603d0 2024-04-23T14:36:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.822+0000 7f22c1e00640 1 --2- 172.21.15.59:0/940740937 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40cd050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009870 tx=0x7f22ac02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57d5c107c5ccbce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.822+0000 7f22c0a00640 1 -- 172.21.15.59:0/940740937 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac037070 con 0x7f22c40b0ff0 2024-04-23T14:36:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.822+0000 7f22c0a00640 1 -- 172.21.15.59:0/940740937 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac032440 con 0x7f22c40b0ff0 2024-04-23T14:36:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.822+0000 7f22c0a00640 1 -- 172.21.15.59:0/940740937 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac032810 con 0x7f22c40b0ff0 2024-04-23T14:36:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 -- 172.21.15.59:0/940740937 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 msgr2=0x7f22c40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 --2- 172.21.15.59:0/940740937 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40cd050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009870 tx=0x7f22ac02a270 comp rx=0 tx=0).stop 2024-04-23T14:36:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 -- 172.21.15.59:0/940740937 shutdown_connections 2024-04-23T14:36:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 --2- 172.21.15.59:0/940740937 >> v2:172.21.15.76:3300/0 conn(0x7f22c406c0e0 0x7f22c40b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 --2- 172.21.15.59:0/940740937 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40cd050 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 -- 172.21.15.59:0/940740937 >> 172.21.15.59:0/940740937 conn(0x7f22c40733c0 msgr2=0x7f22c409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 -- 172.21.15.59:0/940740937 shutdown_connections 2024-04-23T14:36:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.823+0000 7f22c8e00640 1 -- 172.21.15.59:0/940740937 wait complete. 2024-04-23T14:36:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.825+0000 7f22c8e00640 1 Processor -- start 2024-04-23T14:36:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.825+0000 7f22c8e00640 1 -- start start 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.826+0000 7f22c8e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22c406c0e0 0x7f22c41519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.826+0000 7f22c8e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.826+0000 7f22c8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22c41603d0 0x7f22c40b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.826+0000 7f22c8e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22c40cd710 con 0x7f22c41603d0 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.826+0000 7f22c8e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22c40cd590 con 0x7f22c406c0e0 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.826+0000 7f22c8e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22c40b0850 con 0x7f22c40b0ff0 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22c41603d0 0x7f22c40b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22c41603d0 0x7f22c40b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40434/0 (socket says 172.21.15.59:40434) 2024-04-23T14:36:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 -- 172.21.15.59:0/55653865 learned_addr learned my addr 172.21.15.59:0/55653865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c1400640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c1e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22c406c0e0 0x7f22c41519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 -- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 msgr2=0x7f22c40b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 -- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22c406c0e0 msgr2=0x7f22c41519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22c406c0e0 0x7f22c41519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.827+0000 7f22c2800640 1 -- 172.21.15.59:0/55653865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c40c5b20 con 0x7f22c41603d0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.828+0000 7f22c2800640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22c41603d0 0x7f22c40b13d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00e850 tx=0x7f22bc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.828+0000 7f22ba000640 1 -- 172.21.15.59:0/55653865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc017070 con 0x7f22c41603d0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.828+0000 7f22ba000640 1 -- 172.21.15.59:0/55653865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22bc00c940 con 0x7f22c41603d0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.828+0000 7f22ba000640 1 -- 172.21.15.59:0/55653865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00cd30 con 0x7f22c41603d0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.828+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c40b4a40 con 0x7f22c41603d0 2024-04-23T14:36:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.828+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c40c5de0 con 0x7f22c41603d0 2024-04-23T14:36:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.829+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c4069190 con 0x7f22c41603d0 2024-04-23T14:36:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.830+0000 7f22ba000640 1 -- 172.21.15.59:0/55653865 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22bc01a070 con 0x7f22c41603d0 2024-04-23T14:36:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.831+0000 7f22ba000640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22a006c8f0 0x7f22a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.831+0000 7f22ba000640 1 -- 172.21.15.59:0/55653865 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f22bc08b810 con 0x7f22c41603d0 2024-04-23T14:36:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.831+0000 7f22c1e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22a006c8f0 0x7f22a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:20.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.832+0000 7f22c1e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22a006c8f0 0x7f22a006edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f22ac034000 tx=0x7f22ac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:20.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:20.836+0000 7f22ba000640 1 -- 172.21.15.59:0/55653865 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22bc059f30 con 0x7f22c41603d0 2024-04-23T14:36:21.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.123+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f22c406c880 con 0x7f22c41603d0 2024-04-23T14:36:21.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.133+0000 7f22ba000640 1 -- 172.21.15.59:0/55653865 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v256) ==== 169+0+0 (secure 0 0 0) 0x7f22bc059830 con 0x7f22c41603d0 2024-04-23T14:36:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22a006c8f0 msgr2=0x7f22a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22a006c8f0 0x7f22a006edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f22ac034000 tx=0x7f22ac02e000 comp rx=0 tx=0).stop 2024-04-23T14:36:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22c41603d0 msgr2=0x7f22c40b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22c41603d0 0x7f22c40b13d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00e850 tx=0x7f22bc00ed10 comp rx=0 tx=0).stop 2024-04-23T14:36:21.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 shutdown_connections 2024-04-23T14:36:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22a006c8f0 0x7f22a006edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22c40b0ff0 0x7f22c40b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22c41603d0 0x7f22c40b13d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 --2- 172.21.15.59:0/55653865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22c406c0e0 0x7f22c41519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.135+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 >> 172.21.15.59:0/55653865 conn(0x7f22c40733c0 msgr2=0x7f22c40aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.136+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 shutdown_connections 2024-04-23T14:36:21.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.136+0000 7f22c8e00640 1 -- 172.21.15.59:0/55653865 wait complete. 2024-04-23T14:36:21.155 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-23T14:36:21.156 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-23T14:36:21.156 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-23T14:36:21.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:21.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.355+0000 7fb09b200640 1 Processor -- start 2024-04-23T14:36:21.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.356+0000 7fb09b200640 1 -- start start 2024-04-23T14:36:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.357+0000 7fb09b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb09c15e7c0 0x7fb09c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.357+0000 7fb09b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.357+0000 7fb09b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c0cf030 con 0x7fb09c0d2630 2024-04-23T14:36:21.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.357+0000 7fb09b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb09c0d1740 con 0x7fb09c15e7c0 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.357+0000 7fb09b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb09c0b7000 con 0x7fb09c0d0d90 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb099e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb09c15e7c0 0x7fb09c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb099400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb09c0d2630 0x7fb09c0b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48334/0 (socket says 172.21.15.59:48334) 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb099400640 1 -- 172.21.15.59:0/4137332617 learned_addr learned my addr 172.21.15.59:0/4137332617 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb09a800640 1 --2- 172.21.15.59:0/4137332617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb099e00640 1 -- 172.21.15.59:0/4137332617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 msgr2=0x7fb09c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb099e00640 1 --2- 172.21.15.59:0/4137332617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb099e00640 1 -- 172.21.15.59:0/4137332617 >> v1:172.21.15.59:6789/0 conn(0x7fb09c0d2630 legacy=0x7fb09c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb099e00640 1 -- 172.21.15.59:0/4137332617 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c0b6a40 con 0x7fb09c15e7c0 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb098a00640 1 -- 172.21.15.59:0/4137332617 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138685286 0 0) 0x7fb088001110 con 0x7fb09c0d2630 2024-04-23T14:36:21.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.358+0000 7fb09a800640 1 --2- 172.21.15.59:0/4137332617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:21.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.359+0000 7fb099e00640 1 --2- 172.21.15.59:0/4137332617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0d20f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0097a0 tx=0x7fb08c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6104b02e7184230e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb098a00640 1 -- 172.21.15.59:0/4137332617 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c037070 con 0x7fb09c15e7c0 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb098a00640 1 -- 172.21.15.59:0/4137332617 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb08c032440 con 0x7fb09c15e7c0 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb098a00640 1 -- 172.21.15.59:0/4137332617 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c032810 con 0x7fb09c15e7c0 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 -- 172.21.15.59:0/4137332617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 msgr2=0x7fb09c0d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 --2- 172.21.15.59:0/4137332617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0d20f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0097a0 tx=0x7fb08c02a180 comp rx=0 tx=0).stop 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 -- 172.21.15.59:0/4137332617 shutdown_connections 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 --2- 172.21.15.59:0/4137332617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 --2- 172.21.15.59:0/4137332617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0d20f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 -- 172.21.15.59:0/4137332617 >> 172.21.15.59:0/4137332617 conn(0x7fb09c09d920 msgr2=0x7fb09c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 -- 172.21.15.59:0/4137332617 shutdown_connections 2024-04-23T14:36:21.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.360+0000 7fb09b200640 1 -- 172.21.15.59:0/4137332617 wait complete. 2024-04-23T14:36:21.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.362+0000 7fb09b200640 1 Processor -- start 2024-04-23T14:36:21.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.362+0000 7fb09b200640 1 -- start start 2024-04-23T14:36:21.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.363+0000 7fb09b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.363+0000 7fb09b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb09c0d2630 0x7fb09c0c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.363+0000 7fb09b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.363+0000 7fb09b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c0cf2b0 con 0x7fb09c15e7c0 2024-04-23T14:36:21.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.363+0000 7fb09b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c0cf620 con 0x7fb09c0d2630 2024-04-23T14:36:21.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.363+0000 7fb09b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb09c0b6700 con 0x7fb09c0d0d90 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40448/0 (socket says 172.21.15.59:40448) 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 -- 172.21.15.59:0/3880648451 learned_addr learned my addr 172.21.15.59:0/3880648451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb099e00640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 -- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 msgr2=0x7fb09c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb099400640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb09c0d2630 0x7fb09c0c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 -- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb09c0d2630 msgr2=0x7fb09c0c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb09c0d2630 0x7fb09c0c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 -- 172.21.15.59:0/3880648451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c0b7510 con 0x7fb09c15e7c0 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb099e00640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb099400640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb09c0d2630 0x7fb09c0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:21.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.364+0000 7fb09a800640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0b2a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb09400cef0 tx=0x7fb09400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.365+0000 7fb092000640 1 -- 172.21.15.59:0/3880648451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09401a070 con 0x7fb09c15e7c0 2024-04-23T14:36:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.365+0000 7fb092000640 1 -- 172.21.15.59:0/3880648451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09400d860 con 0x7fb09c15e7c0 2024-04-23T14:36:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.365+0000 7fb092000640 1 -- 172.21.15.59:0/3880648451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09400dc50 con 0x7fb09c15e7c0 2024-04-23T14:36:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.365+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c0c86e0 con 0x7fb09c15e7c0 2024-04-23T14:36:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.365+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c0c8ca0 con 0x7fb09c15e7c0 2024-04-23T14:36:21.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.366+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09c04fd90 con 0x7fb09c15e7c0 2024-04-23T14:36:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.367+0000 7fb092000640 1 -- 172.21.15.59:0/3880648451 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb09401d070 con 0x7fb09c15e7c0 2024-04-23T14:36:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.369+0000 7fb092000640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb07c06c920 0x7fb07c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.369+0000 7fb092000640 1 -- 172.21.15.59:0/3880648451 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fb09408c0f0 con 0x7fb09c15e7c0 2024-04-23T14:36:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.369+0000 7fb099e00640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb07c06c920 0x7fb07c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.370+0000 7fb099e00640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb07c06c920 0x7fb07c06ede0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb08c034000 tx=0x7fb08c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.374+0000 7fb092000640 1 -- 172.21.15.59:0/3880648451 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb09405a8d0 con 0x7fb09c15e7c0 2024-04-23T14:36:21.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.589+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb09c0c9230 con 0x7fb09c15e7c0 2024-04-23T14:36:21.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.592+0000 7fb092000640 1 -- 172.21.15.59:0/3880648451 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+12960 (secure 0 0 0) 0x7fb09405a1d0 con 0x7fb09c15e7c0 2024-04-23T14:36:21.593 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:21.593 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:20.116105+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/1087917409":"2024-04-24T14:36:16.851944+0000","172.21.15.59:6834/1087917409":"2024-04-24T14:36:16.851944+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:21.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.594+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb07c06c920 msgr2=0x7fb07c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb07c06c920 0x7fb07c06ede0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb08c034000 tx=0x7fb08c02e000 comp rx=0 tx=0).stop 2024-04-23T14:36:21.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 msgr2=0x7fb09c0b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0b2a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb09400cef0 tx=0x7fb09400a710 comp rx=0 tx=0).stop 2024-04-23T14:36:21.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 shutdown_connections 2024-04-23T14:36:21.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb07c06c920 0x7fb07c06ede0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb09c15e7c0 0x7fb09c0b2a30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb09c0d2630 0x7fb09c0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 --2- 172.21.15.59:0/3880648451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb09c0d0d90 0x7fb09c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 >> 172.21.15.59:0/3880648451 conn(0x7fb09c09d920 msgr2=0x7fb09c15fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:21.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 shutdown_connections 2024-04-23T14:36:21.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.595+0000 7fb09b200640 1 -- 172.21.15.59:0/3880648451 wait complete. 2024-04-23T14:36:21.616 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:36:21.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:36:21.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.828+0000 7f122b200640 1 Processor -- start 2024-04-23T14:36:21.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.829+0000 7f122b200640 1 -- start start 2024-04-23T14:36:21.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.830+0000 7f122b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f122c0d0d10 0x7f122c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.830+0000 7f122b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f122c15d810 0x7f122c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.830+0000 7f122b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122c15cab0 con 0x7f122c1567e0 2024-04-23T14:36:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.830+0000 7f122b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f122c160e40 con 0x7f122c0d0d10 2024-04-23T14:36:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.830+0000 7f122b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f122c15cd30 con 0x7f122c15d810 2024-04-23T14:36:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.831+0000 7f1229e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f122c1567e0 0x7f122c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48342/0 (socket says 172.21.15.59:48342) 2024-04-23T14:36:21.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.831+0000 7f1229e00640 1 -- 172.21.15.59:0/1607909451 learned_addr learned my addr 172.21.15.59:0/1607909451 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.831+0000 7f122a800640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f122c15d810 0x7f122c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.831+0000 7f1223e00640 1 --2- 172.21.15.59:0/1607909451 >> v2:172.21.15.76:3300/0 conn(0x7f122c0d0d10 0x7f122c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.831+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575162180 0 0) 0x7f121c001110 con 0x7f122c1567e0 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.831+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1210003520 con 0x7f122c1567e0 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4243719394 0 0) 0x7f1210003520 con 0x7f122c1567e0 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f122c15d810 msgr2=0x7f122c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f122c15d810 0x7f122c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 >> v2:172.21.15.76:3300/0 conn(0x7f122c0d0d10 msgr2=0x7f122c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 --2- 172.21.15.59:0/1607909451 >> v2:172.21.15.76:3300/0 conn(0x7f122c0d0d10 0x7f122c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c15d010 con 0x7f122c1567e0 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f121c001840 con 0x7f122c1567e0 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 >> v1:172.21.15.59:6789/0 conn(0x7f122c1567e0 legacy=0x7f122c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12100039b0 0x7f1210003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1210004800 0x7f1210004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f122c15d810 0x7f12100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f122c160e40 con 0x7f1210004800 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f122c15cd30 con 0x7f12100039b0 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f121000b9c0 con 0x7f122c15d810 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.832+0000 7f1223e00640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1210004800 0x7f1210004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.833+0000 7f1229e00640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12100039b0 0x7f1210003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.833+0000 7f1223e00640 1 -- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f122c15d810 msgr2=0x7f12100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.833+0000 7f1223e00640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f122c15d810 0x7f12100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.833+0000 7f1223e00640 1 -- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12100039b0 msgr2=0x7f1210003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.833+0000 7f1223e00640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12100039b0 0x7f1210003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.833+0000 7f1223e00640 1 -- 172.21.15.59:0/1607909451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1210003520 con 0x7f1210004800 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.834+0000 7f1223e00640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1210004800 0x7f1210004c80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f122c0afdb0 tx=0x7f1210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d80d90136547cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.834+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214036070 con 0x7f1210004800 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.834+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1214031400 con 0x7f1210004800 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.834+0000 7f1229400640 1 -- 172.21.15.59:0/1607909451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214009690 con 0x7f1210004800 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.834+0000 7f122b200640 1 -- 172.21.15.59:0/1607909451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1210004800 msgr2=0x7f1210004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.834+0000 7f122b200640 1 --2- 172.21.15.59:0/1607909451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1210004800 0x7f1210004c80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f122c0afdb0 tx=0x7f1210005830 comp rx=0 tx=0).stop 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.835+0000 7f122a800640 1 -- 172.21.15.59:0/1607909451 reap_dead start 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.835+0000 7f122b200640 1 -- 172.21.15.59:0/1607909451 shutdown_connections 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.835+0000 7f122b200640 1 -- 172.21.15.59:0/1607909451 >> 172.21.15.59:0/1607909451 conn(0x7f122c06bdf0 msgr2=0x7f122c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.835+0000 7f122b200640 1 -- 172.21.15.59:0/1607909451 shutdown_connections 2024-04-23T14:36:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.835+0000 7f122b200640 1 -- 172.21.15.59:0/1607909451 wait complete. 2024-04-23T14:36:21.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.836+0000 7f122b200640 1 Processor -- start 2024-04-23T14:36:21.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.837+0000 7f122b200640 1 -- start start 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.837+0000 7f122b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1210004800 0x7f122c09da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.837+0000 7f122b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c0b0d90 0x7f122c0b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f122c0b1210 0x7f122c0b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f122c160290 con 0x7f122c0b1210 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f122c160110 con 0x7f122c0b0d90 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f122c160410 con 0x7f1210004800 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f122c0b1210 0x7f122c0b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f122c0b1210 0x7f122c0b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40476/0 (socket says 172.21.15.59:40476) 2024-04-23T14:36:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 -- 172.21.15.59:0/584294548 learned_addr learned my addr 172.21.15.59:0/584294548 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f1229e00640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1210004800 0x7f122c09da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 -- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1210004800 msgr2=0x7f122c09da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1210004800 0x7f122c09da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f1223e00640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c0b0d90 0x7f122c0b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 -- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c0b0d90 msgr2=0x7f122c0b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c0b0d90 0x7f122c0b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f122a800640 1 -- 172.21.15.59:0/584294548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c152450 con 0x7f122c0b1210 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.838+0000 7f1229e00640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1210004800 0x7f122c09da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.839+0000 7f122a800640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f122c0b1210 0x7f122c0b3420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f122c15a680 tx=0x7f122400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.839+0000 7f1222000640 1 -- 172.21.15.59:0/584294548 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224018020 con 0x7f122c0b1210 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.839+0000 7f1222000640 1 -- 172.21.15.59:0/584294548 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122400bcb0 con 0x7f122c0b1210 2024-04-23T14:36:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.839+0000 7f1222000640 1 -- 172.21.15.59:0/584294548 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122400be60 con 0x7f122c0b1210 2024-04-23T14:36:21.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.839+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c09dfb0 con 0x7f122c0b1210 2024-04-23T14:36:21.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.839+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c151690 con 0x7f122c0b1210 2024-04-23T14:36:21.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.840+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c04fd90 con 0x7f122c0b1210 2024-04-23T14:36:21.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.841+0000 7f1222000640 1 -- 172.21.15.59:0/584294548 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1224005d10 con 0x7f122c0b1210 2024-04-23T14:36:21.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.843+0000 7f1222000640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f120806c920 0x7f120806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:21.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.843+0000 7f1222000640 1 -- 172.21.15.59:0/584294548 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f122408be70 con 0x7f122c0b1210 2024-04-23T14:36:21.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.843+0000 7f1229e00640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f120806c920 0x7f120806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.844+0000 7f1229e00640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f120806c920 0x7f120806ede0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f121c001eb0 tx=0x7f121c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:21.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:21.847+0000 7f1222000640 1 -- 172.21.15.59:0/584294548 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1224022030 con 0x7f122c0b1210 2024-04-23T14:36:22.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.052+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f122c155d60 con 0x7f122c0b1210 2024-04-23T14:36:22.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.053+0000 7f1222000640 1 -- 172.21.15.59:0/584294548 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7f122405a590 con 0x7f122c0b1210 2024-04-23T14:36:22.055 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:36:22.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f120806c920 msgr2=0x7f120806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:22.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f120806c920 0x7f120806ede0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f121c001eb0 tx=0x7f121c009040 comp rx=0 tx=0).stop 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f122c0b1210 msgr2=0x7f122c0b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f122c0b1210 0x7f122c0b3420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f122c15a680 tx=0x7f122400b480 comp rx=0 tx=0).stop 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 shutdown_connections 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f120806c920 0x7f120806ede0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f122c0b1210 0x7f122c0b3420 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f122c0b0d90 0x7f122c0b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 --2- 172.21.15.59:0/584294548 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1210004800 0x7f122c09da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 >> 172.21.15.59:0/584294548 conn(0x7f122c06bdf0 msgr2=0x7f122c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 shutdown_connections 2024-04-23T14:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.057+0000 7f122b200640 1 -- 172.21.15.59:0/584294548 wait complete. 2024-04-23T14:36:22.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-23T14:36:22.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.285+0000 7f4c3aa00640 1 Processor -- start 2024-04-23T14:36:22.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.285+0000 7f4c3aa00640 1 -- start start 2024-04-23T14:36:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.286+0000 7f4c3aa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 0x7f4c340cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.286+0000 7f4c3aa00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4c340cd590 0x7f4c340b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.286+0000 7f4c3aa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c34160210 con 0x7f4c340b4690 2024-04-23T14:36:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.286+0000 7f4c3aa00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4c340cd9f0 con 0x7f4c340cd590 2024-04-23T14:36:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.286+0000 7f4c3aa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4c341604f0 con 0x7f4c3415c3d0 2024-04-23T14:36:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.287+0000 7f4c3a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4c340b4690 0x7f4c3406b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48348/0 (socket says 172.21.15.59:48348) 2024-04-23T14:36:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.287+0000 7f4c3a000640 1 -- 172.21.15.59:0/592279196 learned_addr learned my addr 172.21.15.59:0/592279196 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.287+0000 7f4c39600640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 0x7f4c340cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.287+0000 7f4c38c00640 1 --2- 172.21.15.59:0/592279196 >> v2:172.21.15.76:3300/0 conn(0x7f4c340cd590 0x7f4c340b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.287+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 598521678 0 0) 0x7f4c34160210 con 0x7f4c340b4690 2024-04-23T14:36:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.287+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c1c003520 con 0x7f4c340b4690 2024-04-23T14:36:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 593766378 0 0) 0x7f4c1c003520 con 0x7f4c340b4690 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 msgr2=0x7f4c340cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 0x7f4c340cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 >> v2:172.21.15.76:3300/0 conn(0x7f4c340cd590 msgr2=0x7f4c340b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 --2- 172.21.15.59:0/592279196 >> v2:172.21.15.76:3300/0 conn(0x7f4c340cd590 0x7f4c340b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c341607d0 con 0x7f4c340b4690 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f4c2c0019f0 con 0x7f4c340b4690 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 >> v1:172.21.15.59:6789/0 conn(0x7f4c340b4690 legacy=0x7f4c3406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 0x7f4c1c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1c003f80 0x7f4c1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341604f0 con 0x7f4c1c004d40 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c00b9c0 con 0x7f4c1c003f80 2024-04-23T14:36:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1c00bb40 con 0x7f4c3415c3d0 2024-04-23T14:36:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.288+0000 7f4c3a000640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.289+0000 7f4c39600640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 0x7f4c1c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.289+0000 7f4c3a000640 1 -- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 msgr2=0x7f4c1c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.289+0000 7f4c3a000640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415c3d0 0x7f4c1c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.289+0000 7f4c3a000640 1 -- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1c003f80 msgr2=0x7f4c1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.289+0000 7f4c3a000640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1c003f80 0x7f4c1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.289+0000 7f4c3a000640 1 -- 172.21.15.59:0/592279196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c003520 con 0x7f4c1c004d40 2024-04-23T14:36:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.290+0000 7f4c3a000640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c1c0051c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00a7d0 tx=0x7f4c1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cc56418735c2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.290+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00a3f0 con 0x7f4c1c004d40 2024-04-23T14:36:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.290+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c02f070 con 0x7f4c1c004d40 2024-04-23T14:36:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.290+0000 7f4c33e00640 1 -- 172.21.15.59:0/592279196 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c02bc50 con 0x7f4c1c004d40 2024-04-23T14:36:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.291+0000 7f4c3aa00640 1 -- 172.21.15.59:0/592279196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 msgr2=0x7f4c1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.291+0000 7f4c3aa00640 1 --2- 172.21.15.59:0/592279196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c1c0051c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00a7d0 tx=0x7f4c1c005830 comp rx=0 tx=0).stop 2024-04-23T14:36:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.291+0000 7f4c3a000640 1 -- 172.21.15.59:0/592279196 reap_dead start 2024-04-23T14:36:22.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.291+0000 7f4c3aa00640 1 -- 172.21.15.59:0/592279196 shutdown_connections 2024-04-23T14:36:22.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.291+0000 7f4c3aa00640 1 -- 172.21.15.59:0/592279196 >> 172.21.15.59:0/592279196 conn(0x7f4c340b21a0 msgr2=0x7f4c340b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:22.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.291+0000 7f4c3aa00640 1 -- 172.21.15.59:0/592279196 shutdown_connections 2024-04-23T14:36:22.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.291+0000 7f4c3aa00640 1 -- 172.21.15.59:0/592279196 wait complete. 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.293+0000 7f4c3aa00640 1 Processor -- start 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.293+0000 7f4c3aa00640 1 -- start start 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c3aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c34159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c3aa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415adb0 0x7f4c340ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c3aa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c3415b230 0x7f4c340ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c3aa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c340b4ae0 con 0x7f4c1c004d40 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c3aa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34160d90 con 0x7f4c3415b230 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c3aa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3415c7b0 con 0x7f4c3415adb0 2024-04-23T14:36:22.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c34159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c34159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40486/0 (socket says 172.21.15.59:40486) 2024-04-23T14:36:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 -- 172.21.15.59:0/358505249 learned_addr learned my addr 172.21.15.59:0/358505249 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 -- 172.21.15.59:0/358505249 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415adb0 msgr2=0x7f4c340ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415adb0 0x7f4c340ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 -- 172.21.15.59:0/358505249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c3415b230 msgr2=0x7f4c340ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c3415b230 0x7f4c340ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.294+0000 7f4c39600640 1 -- 172.21.15.59:0/358505249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c34159fe0 con 0x7f4c1c004d40 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.295+0000 7f4c39600640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c34159aa0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4c28011630 tx=0x7f4c28011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.295+0000 7f4c31600640 1 -- 172.21.15.59:0/358505249 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28015870 con 0x7f4c1c004d40 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.295+0000 7f4c31600640 1 -- 172.21.15.59:0/358505249 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c28015a10 con 0x7f4c1c004d40 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.295+0000 7f4c31600640 1 -- 172.21.15.59:0/358505249 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28015dc0 con 0x7f4c1c004d40 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.295+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c341f82a0 con 0x7f4c1c004d40 2024-04-23T14:36:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.295+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3415a2d0 con 0x7f4c1c004d40 2024-04-23T14:36:22.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.297+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3404fd90 con 0x7f4c1c004d40 2024-04-23T14:36:22.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.298+0000 7f4c31600640 1 -- 172.21.15.59:0/358505249 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c28013430 con 0x7f4c1c004d40 2024-04-23T14:36:22.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.299+0000 7f4c31600640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c1406c920 0x7f4c1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:22.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.299+0000 7f4c31600640 1 -- 172.21.15.59:0/358505249 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f4c2808bd20 con 0x7f4c1c004d40 2024-04-23T14:36:22.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.300+0000 7f4c38c00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c1406c920 0x7f4c1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:22.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.300+0000 7f4c38c00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c1406c920 0x7f4c1406ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4c20006a90 tx=0x7f4c20006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:22.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.303+0000 7f4c31600640 1 -- 172.21.15.59:0/358505249 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c28019030 con 0x7f4c1c004d40 2024-04-23T14:36:22.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:22.585+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f4c3406e400 con 0x7f4c1c004d40 2024-04-23T14:36:23.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.145+0000 7f4c31600640 1 -- 172.21.15.59:0/358505249 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v258) ==== 169+0+0 (secure 0 0 0) 0x7f4c28059d60 con 0x7f4c1c004d40 2024-04-23T14:36:23.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c1406c920 msgr2=0x7f4c1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c1406c920 0x7f4c1406ede0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4c20006a90 tx=0x7f4c20006a20 comp rx=0 tx=0).stop 2024-04-23T14:36:23.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 msgr2=0x7f4c34159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c34159aa0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4c28011630 tx=0x7f4c28011af0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 shutdown_connections 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c1406c920 0x7f4c1406ede0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c3415adb0 0x7f4c340ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c3415b230 0x7f4c340ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 --2- 172.21.15.59:0/358505249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1c004d40 0x7f4c34159aa0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 >> 172.21.15.59:0/358505249 conn(0x7f4c340b21a0 msgr2=0x7f4c340cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 shutdown_connections 2024-04-23T14:36:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.149+0000 7f4c3aa00640 1 -- 172.21.15.59:0/358505249 wait complete. 2024-04-23T14:36:23.172 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:36:23.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.376+0000 7f2309200640 1 Processor -- start 2024-04-23T14:36:23.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.376+0000 7f2309200640 1 -- start start 2024-04-23T14:36:23.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f2309200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:23.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f2309200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23040b99a0 0x7f23040b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:23.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f2309200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23040ba1e0 con 0x7f23040d2d60 2024-04-23T14:36:23.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f2309200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f23040ba7a0 con 0x7f23040b99a0 2024-04-23T14:36:23.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f2309200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f23040ba4c0 con 0x7f230415eb60 2024-04-23T14:36:23.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f22fbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f23040d2d60 0x7f23040b9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48354/0 (socket says 172.21.15.59:48354) 2024-04-23T14:36:23.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f22fbe00640 1 -- 172.21.15.59:0/4138779481 learned_addr learned my addr 172.21.15.59:0/4138779481 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:23.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.378+0000 7f2302800640 1 --2- 172.21.15.59:0/4138779481 >> v2:172.21.15.59:3300/0 conn(0x7f23040b99a0 0x7f23040b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:23.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2301400640 1 -- 172.21.15.59:0/4138779481 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 998488173 0 0) 0x7f23040ba1e0 con 0x7f23040d2d60 2024-04-23T14:36:23.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2301400640 1 -- 172.21.15.59:0/4138779481 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22e8003520 con 0x7f23040d2d60 2024-04-23T14:36:23.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2301e00640 1 --2- 172.21.15.59:0/4138779481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:23.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2302800640 1 -- 172.21.15.59:0/4138779481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 msgr2=0x7f23040d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:23.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2302800640 1 --2- 172.21.15.59:0/4138779481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2302800640 1 -- 172.21.15.59:0/4138779481 >> v1:172.21.15.59:6789/0 conn(0x7f23040d2d60 legacy=0x7f23040b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2301400640 1 -- 172.21.15.59:0/4138779481 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 6055162 0 0) 0x7f22e8003520 con 0x7f23040d2d60 2024-04-23T14:36:23.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.379+0000 7f2302800640 1 -- 172.21.15.59:0/4138779481 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040b8a90 con 0x7f23040b99a0 2024-04-23T14:36:23.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.380+0000 7f2302800640 1 --2- 172.21.15.59:0/4138779481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040b99a0 0x7f23040b77e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009d90 tx=0x7f22fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41aeac355ecacf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:23.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2301400640 1 -- 172.21.15.59:0/4138779481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc007740 con 0x7f23040b99a0 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2301400640 1 -- 172.21.15.59:0/4138779481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22fc0078e0 con 0x7f23040b99a0 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2301400640 1 -- 172.21.15.59:0/4138779481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc007cb0 con 0x7f23040b99a0 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2309200640 1 -- 172.21.15.59:0/4138779481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040b99a0 msgr2=0x7f23040b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2309200640 1 --2- 172.21.15.59:0/4138779481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040b99a0 0x7f23040b77e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009d90 tx=0x7f22fc02a660 comp rx=0 tx=0).stop 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2309200640 1 -- 172.21.15.59:0/4138779481 shutdown_connections 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2309200640 1 --2- 172.21.15.59:0/4138779481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040b99a0 0x7f23040b77e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2309200640 1 --2- 172.21.15.59:0/4138779481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.381+0000 7f2309200640 1 -- 172.21.15.59:0/4138779481 >> 172.21.15.59:0/4138779481 conn(0x7f2304067b50 msgr2=0x7f2304067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.382+0000 7f2309200640 1 -- 172.21.15.59:0/4138779481 shutdown_connections 2024-04-23T14:36:23.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.382+0000 7f2309200640 1 -- 172.21.15.59:0/4138779481 wait complete. 2024-04-23T14:36:23.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.383+0000 7f2309200640 1 Processor -- start 2024-04-23T14:36:23.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.383+0000 7f2309200640 1 -- start start 2024-04-23T14:36:23.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.384+0000 7f2309200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23040b99a0 0x7f23040c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:23.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.384+0000 7f2309200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:23.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.384+0000 7f2309200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040d4810 0x7f23040ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:23.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.384+0000 7f2309200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23040b3490 con 0x7f23040d4810 2024-04-23T14:36:23.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.384+0000 7f2309200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23040b3c80 con 0x7f23040b99a0 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.384+0000 7f2309200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f23040b3610 con 0x7f230415eb60 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040d4810 0x7f23040ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040d4810 0x7f23040ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40492/0 (socket says 172.21.15.59:40492) 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 -- 172.21.15.59:0/1935135246 learned_addr learned my addr 172.21.15.59:0/1935135246 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2301e00640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23040b99a0 0x7f23040c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f22fbe00640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 -- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 msgr2=0x7f23040d42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 -- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23040b99a0 msgr2=0x7f23040c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23040b99a0 0x7f23040c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.385+0000 7f2302800640 1 -- 172.21.15.59:0/1935135246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040c90f0 con 0x7f23040d4810 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.386+0000 7f2302800640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040d4810 0x7f23040ca550 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009eb0 tx=0x7f22fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.386+0000 7f22fa000640 1 -- 172.21.15.59:0/1935135246 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc040070 con 0x7f23040d4810 2024-04-23T14:36:23.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.386+0000 7f22fa000640 1 -- 172.21.15.59:0/1935135246 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22fc031bc0 con 0x7f23040d4810 2024-04-23T14:36:23.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.386+0000 7f22fa000640 1 -- 172.21.15.59:0/1935135246 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc009450 con 0x7f23040d4810 2024-04-23T14:36:23.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.386+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23041f4490 con 0x7f23040d4810 2024-04-23T14:36:23.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.386+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041f4a50 con 0x7f23040d4810 2024-04-23T14:36:23.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.387+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2304068a80 con 0x7f23040d4810 2024-04-23T14:36:23.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.388+0000 7f22fa000640 1 -- 172.21.15.59:0/1935135246 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22fc039040 con 0x7f23040d4810 2024-04-23T14:36:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.389+0000 7f22fa000640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22e006c8f0 0x7f22e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.389+0000 7f22fa000640 1 -- 172.21.15.59:0/1935135246 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f22fc0ad150 con 0x7f23040d4810 2024-04-23T14:36:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.389+0000 7f2301e00640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22e006c8f0 0x7f22e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:23.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.390+0000 7f2301e00640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22e006c8f0 0x7f22e006edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f22f4006fd0 tx=0x7f22f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.393+0000 7f22fa000640 1 -- 172.21.15.59:0/1935135246 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23040c90f0 con 0x7f23040d4810 2024-04-23T14:36:23.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.591+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f230406bc40 con 0x7f23040d4810 2024-04-23T14:36:23.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.596+0000 7f22fa000640 1 -- 172.21.15.59:0/1935135246 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f22fc07b220 con 0x7f23040d4810 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.598+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22e006c8f0 msgr2=0x7f22e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.598+0000 7f2309200640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22e006c8f0 0x7f22e006edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f22f4006fd0 tx=0x7f22f4008040 comp rx=0 tx=0).stop 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.598+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040d4810 msgr2=0x7f23040ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.598+0000 7f2309200640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040d4810 0x7f23040ca550 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009eb0 tx=0x7f22fc004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 shutdown_connections 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f22e006c8f0 0x7f22e006edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f230415eb60 0x7f23040d42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23040d4810 0x7f23040ca550 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 --2- 172.21.15.59:0/1935135246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23040b99a0 0x7f23040c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 >> 172.21.15.59:0/1935135246 conn(0x7f2304067b50 msgr2=0x7f2304067f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:23.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 shutdown_connections 2024-04-23T14:36:23.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:23.599+0000 7f2309200640 1 -- 172.21.15.59:0/1935135246 wait complete. 2024-04-23T14:36:23.617 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:36:23.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:36:23.718 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:36:23.827 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:36:23.827 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:36:23.827 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:23.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:36:23.885 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:36:23.886 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:23.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:36:23.939 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:36:23.939 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:23.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:36:23.999 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:36:23.999 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:24.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:36:24.056 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:36:24.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.276+0000 7f4ca8200640 1 Processor -- start 2024-04-23T14:36:24.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.276+0000 7f4ca8200640 1 -- start start 2024-04-23T14:36:24.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.278+0000 7f4ca8200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ca01454e0 0x7f4ca0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.278+0000 7f4ca8200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ca00cd020 0x7f4ca00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.278+0000 7f4ca8200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca00cd520 con 0x7f4ca00b3e40 2024-04-23T14:36:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.278+0000 7f4ca8200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca00cddc0 con 0x7f4ca00cd020 2024-04-23T14:36:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.278+0000 7f4ca8200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca00cd800 con 0x7f4ca01454e0 2024-04-23T14:36:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca6200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4ca00b3e40 0x7f4ca0160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48366/0 (socket says 172.21.15.59:48366) 2024-04-23T14:36:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca6200640 1 -- 172.21.15.59:0/4294428182 learned_addr learned my addr 172.21.15.59:0/4294428182 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca4e00640 1 --2- 172.21.15.59:0/4294428182 >> v2:172.21.15.59:3300/0 conn(0x7f4ca00cd020 0x7f4ca00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca5800640 1 --2- 172.21.15.59:0/4294428182 >> v2:172.21.15.76:3300/0 conn(0x7f4ca01454e0 0x7f4ca0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca4e00640 1 -- 172.21.15.59:0/4294428182 >> v2:172.21.15.76:3300/0 conn(0x7f4ca01454e0 msgr2=0x7f4ca0145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:24.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca4e00640 1 --2- 172.21.15.59:0/4294428182 >> v2:172.21.15.76:3300/0 conn(0x7f4ca01454e0 0x7f4ca0145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca4e00640 1 -- 172.21.15.59:0/4294428182 >> v1:172.21.15.59:6789/0 conn(0x7f4ca00b3e40 legacy=0x7f4ca0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.279+0000 7f4ca4e00640 1 -- 172.21.15.59:0/4294428182 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca00cdae0 con 0x7f4ca00cd020 2024-04-23T14:36:24.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.280+0000 7f4ca4e00640 1 --2- 172.21.15.59:0/4294428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00cd020 0x7f4ca00b3900 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0073280 tx=0x7f4c8c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954d5a7fbdae0db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:24.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4c9fe00640 1 -- 172.21.15.59:0/4294428182 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c037070 con 0x7f4ca00cd020 2024-04-23T14:36:24.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4c9fe00640 1 -- 172.21.15.59:0/4294428182 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8c032440 con 0x7f4ca00cd020 2024-04-23T14:36:24.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4c9fe00640 1 -- 172.21.15.59:0/4294428182 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c02ae60 con 0x7f4ca00cd020 2024-04-23T14:36:24.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 -- 172.21.15.59:0/4294428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00cd020 msgr2=0x7f4ca00b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 --2- 172.21.15.59:0/4294428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00cd020 0x7f4ca00b3900 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0073280 tx=0x7f4c8c02a110 comp rx=0 tx=0).stop 2024-04-23T14:36:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 -- 172.21.15.59:0/4294428182 shutdown_connections 2024-04-23T14:36:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 --2- 172.21.15.59:0/4294428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00cd020 0x7f4ca00b3900 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 --2- 172.21.15.59:0/4294428182 >> v2:172.21.15.76:3300/0 conn(0x7f4ca01454e0 0x7f4ca0145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 -- 172.21.15.59:0/4294428182 >> 172.21.15.59:0/4294428182 conn(0x7f4ca0067ea0 msgr2=0x7f4ca006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 -- 172.21.15.59:0/4294428182 shutdown_connections 2024-04-23T14:36:24.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.281+0000 7f4ca8200640 1 -- 172.21.15.59:0/4294428182 wait complete. 2024-04-23T14:36:24.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.283+0000 7f4ca8200640 1 Processor -- start 2024-04-23T14:36:24.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.283+0000 7f4ca8200640 1 -- start start 2024-04-23T14:36:24.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.284+0000 7f4ca8200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00b3e40 0x7f4ca01f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.284+0000 7f4ca8200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ca00cd020 0x7f4ca01443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.284+0000 7f4ca8200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ca01454e0 0x7f4ca0141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.284+0000 7f4ca8200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0146270 con 0x7f4ca00b3e40 2024-04-23T14:36:24.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.284+0000 7f4ca8200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01460f0 con 0x7f4ca00cd020 2024-04-23T14:36:24.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.284+0000 7f4ca8200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca0161020 con 0x7f4ca01454e0 2024-04-23T14:36:24.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00b3e40 0x7f4ca01f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00b3e40 0x7f4ca01f05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40498/0 (socket says 172.21.15.59:40498) 2024-04-23T14:36:24.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 -- 172.21.15.59:0/3848037835 learned_addr learned my addr 172.21.15.59:0/3848037835 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:24.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca6200640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ca01454e0 0x7f4ca0141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca4e00640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ca00cd020 0x7f4ca01443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 -- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ca01454e0 msgr2=0x7f4ca0141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ca01454e0 0x7f4ca0141790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 -- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ca00cd020 msgr2=0x7f4ca01443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ca00cd020 0x7f4ca01443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.285+0000 7f4ca5800640 1 -- 172.21.15.59:0/3848037835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca0142010 con 0x7f4ca00b3e40 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.286+0000 7f4ca5800640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00b3e40 0x7f4ca01f05c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400d5f0 tx=0x7f4c9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.286+0000 7f4c9d600640 1 -- 172.21.15.59:0/3848037835 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940125f0 con 0x7f4ca00b3e40 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.286+0000 7f4c9d600640 1 -- 172.21.15.59:0/3848037835 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c94019070 con 0x7f4ca00b3e40 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.286+0000 7f4c9d600640 1 -- 172.21.15.59:0/3848037835 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940129d0 con 0x7f4ca00b3e40 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.286+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca01422f0 con 0x7f4ca00b3e40 2024-04-23T14:36:24.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.286+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca015dc20 con 0x7f4ca00b3e40 2024-04-23T14:36:24.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.287+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0068a80 con 0x7f4ca00b3e40 2024-04-23T14:36:24.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.288+0000 7f4c9d600640 1 -- 172.21.15.59:0/3848037835 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c94012b70 con 0x7f4ca00b3e40 2024-04-23T14:36:24.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.289+0000 7f4c9d600640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c8006c8f0 0x7f4c8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.290+0000 7f4c9d600640 1 -- 172.21.15.59:0/3848037835 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f4c9408bc60 con 0x7f4ca00b3e40 2024-04-23T14:36:24.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.290+0000 7f4ca4e00640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c8006c8f0 0x7f4c8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.290+0000 7f4ca4e00640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c8006c8f0 0x7f4c8006edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c034000 tx=0x7f4c8c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:24.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.294+0000 7f4c9d600640 1 -- 172.21.15.59:0/3848037835 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c9405a380 con 0x7f4ca00b3e40 2024-04-23T14:36:24.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.540+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ca0160030 con 0x7f4ca00b3e40 2024-04-23T14:36:24.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.541+0000 7f4c9d600640 1 -- 172.21.15.59:0/3848037835 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3869 (secure 0 0 0) 0x7f4c94059c80 con 0x7f4ca00b3e40 2024-04-23T14:36:24.542 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:24.542 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1090261812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":1090261812},{"type":"v1","addr":"172.21.15.59:6837","nonce":1090261812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2100119895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2100119895},{"type":"v1","addr":"172.21.15.76:6835","nonce":2100119895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2516030883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2516030883},{"type":"v1","addr":"172.21.15.76:6833","nonce":2516030883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2966128827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2966128827},{"type":"v1","addr":"172.21.15.59:6836","nonce":2966128827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-04-23T14:36:24.543 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 37 2024-04-23T14:36:24.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.543+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c8006c8f0 msgr2=0x7f4c8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.543+0000 7f4ca8200640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c8006c8f0 0x7f4c8006edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c034000 tx=0x7f4c8c02e000 comp rx=0 tx=0).stop 2024-04-23T14:36:24.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00b3e40 msgr2=0x7f4ca01f05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00b3e40 0x7f4ca01f05c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400d5f0 tx=0x7f4c9400dab0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 shutdown_connections 2024-04-23T14:36:24.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c8006c8f0 0x7f4c8006edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ca01454e0 0x7f4ca0141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ca00cd020 0x7f4ca01443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 --2- 172.21.15.59:0/3848037835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ca00b3e40 0x7f4ca01f05c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 >> 172.21.15.59:0/3848037835 conn(0x7f4ca0067ea0 msgr2=0x7f4ca00afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:24.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 shutdown_connections 2024-04-23T14:36:24.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.544+0000 7f4ca8200640 1 -- 172.21.15.59:0/3848037835 wait complete. 2024-04-23T14:36:24.565 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:36:24.565 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:36:24.566 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:36:24.566 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:36:24.566 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:36:24.566 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:36:24.566 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:36:24.566 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:36:24.567 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:36:24.567 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:36:24.567 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:36:24.567 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:36:24.567 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:36:24.567 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:36:24.567 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:36:24.567 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:36:24.586 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:36:24.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:36:24.587 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:36:24.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:36:24.589 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:36:24.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:36:24.596 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:36:24.596 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:36:24.632 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:36:24.632 DEBUG:teuthology.parallel:result is None 2024-04-23T14:36:24.636 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:36:24.636 DEBUG:teuthology.parallel:result is None 2024-04-23T14:36:24.644 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:36:24.644 DEBUG:teuthology.parallel:result is None 2024-04-23T14:36:24.650 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:36:24.650 DEBUG:teuthology.parallel:result is None 2024-04-23T14:36:24.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:36:24.789 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:36:24.790 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:36:24.800 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:36:24.800 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:36:24.801 INFO:tasks.ceph.mds.d.smithi076.stderr:2024-04-23T14:36:24.801+0000 7f45f8a00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-23T14:36:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.842+0000 7f61a2200640 1 Processor -- start 2024-04-23T14:36:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.842+0000 7f61a2200640 1 -- start start 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f61a2200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f619c06ddf0 0x7f619c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f61a2200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f619c0cd880 con 0x7f619c0cf410 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f61a2200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f619c071ec0 con 0x7f619c072680 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f61a2200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f619c0cdb20 con 0x7f619c06ddf0 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f619aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f619c06ddf0 0x7f619c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f619b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f619c0cf410 0x7f619c15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48400/0 (socket says 172.21.15.59:48400) 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f619aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f619c06ddf0 0x7f619c06e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40550/0 (socket says 172.21.15.59:40550) 2024-04-23T14:36:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.844+0000 7f619b400640 1 -- 172.21.15.59:0/549653648 learned_addr learned my addr 172.21.15.59:0/549653648 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.845+0000 7f619aa00640 1 -- 172.21.15.59:0/549653648 >> v1:172.21.15.76:6789/0 conn(0x7f619c072680 legacy=0x7f619c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.845+0000 7f619aa00640 1 -- 172.21.15.59:0/549653648 >> v1:172.21.15.59:6789/0 conn(0x7f619c0cf410 legacy=0x7f619c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.845+0000 7f619aa00640 1 -- 172.21.15.59:0/549653648 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c072160 con 0x7f619c06ddf0 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.846+0000 7f619aa00640 1 --2- 172.21.15.59:0/549653648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c06ddf0 0x7f619c06e250 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f61880097a0 tx=0x7f618802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42c356f34234d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.846+0000 7f619a000640 1 -- 172.21.15.59:0/549653648 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188037070 con 0x7f619c06ddf0 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.846+0000 7f619a000640 1 -- 172.21.15.59:0/549653648 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6188032440 con 0x7f619c06ddf0 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.846+0000 7f619a000640 1 -- 172.21.15.59:0/549653648 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188032810 con 0x7f619c06ddf0 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.846+0000 7f61a2200640 1 -- 172.21.15.59:0/549653648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c06ddf0 msgr2=0x7f619c06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.846+0000 7f61a2200640 1 --2- 172.21.15.59:0/549653648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c06ddf0 0x7f619c06e250 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f61880097a0 tx=0x7f618802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.847+0000 7f61a2200640 1 -- 172.21.15.59:0/549653648 shutdown_connections 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.847+0000 7f61a2200640 1 --2- 172.21.15.59:0/549653648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c06ddf0 0x7f619c06e250 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.847+0000 7f61a2200640 1 -- 172.21.15.59:0/549653648 >> 172.21.15.59:0/549653648 conn(0x7f619c0b2130 msgr2=0x7f619c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.847+0000 7f61a2200640 1 -- 172.21.15.59:0/549653648 shutdown_connections 2024-04-23T14:36:24.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.847+0000 7f61a2200640 1 -- 172.21.15.59:0/549653648 wait complete. 2024-04-23T14:36:24.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.848+0000 7f61a2200640 1 Processor -- start 2024-04-23T14:36:24.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.849+0000 7f61a2200640 1 -- start start 2024-04-23T14:36:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.849+0000 7f61a2200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f619c06ddf0 0x7f619c0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.849+0000 7f61a2200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c072680 0x7f619c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.849+0000 7f61a2200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f619c0cf410 0x7f619c14d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.849+0000 7f61a2200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f619c15a820 con 0x7f619c072680 2024-04-23T14:36:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.849+0000 7f61a2200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f619c15a6a0 con 0x7f619c06ddf0 2024-04-23T14:36:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.849+0000 7f61a2200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f619c0cff00 con 0x7f619c0cf410 2024-04-23T14:36:24.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c072680 0x7f619c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c072680 0x7f619c155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40558/0 (socket says 172.21.15.59:40558) 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 -- 172.21.15.59:0/3132395988 learned_addr learned my addr 172.21.15.59:0/3132395988 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619b400640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f619c06ddf0 0x7f619c0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619be00640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f619c0cf410 0x7f619c14d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 -- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f619c0cf410 msgr2=0x7f619c14d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f619c0cf410 0x7f619c14d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 -- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f619c06ddf0 msgr2=0x7f619c0af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f619c06ddf0 0x7f619c0af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 -- 172.21.15.59:0/3132395988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c155cd0 con 0x7f619c072680 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619b400640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f619c06ddf0 0x7f619c0af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619be00640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f619c0cf410 0x7f619c14d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.850+0000 7f619aa00640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c072680 0x7f619c155790 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6188035040 tx=0x7f61880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:24.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.851+0000 7f6193400640 1 -- 172.21.15.59:0/3132395988 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188037040 con 0x7f619c072680 2024-04-23T14:36:24.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.851+0000 7f6193400640 1 -- 172.21.15.59:0/3132395988 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6188030770 con 0x7f619c072680 2024-04-23T14:36:24.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.851+0000 7f6193400640 1 -- 172.21.15.59:0/3132395988 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188030b80 con 0x7f619c072680 2024-04-23T14:36:24.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.851+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c154710 con 0x7f619c072680 2024-04-23T14:36:24.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.851+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c154cb0 con 0x7f619c072680 2024-04-23T14:36:24.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.852+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c068a80 con 0x7f619c072680 2024-04-23T14:36:24.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.853+0000 7f6193400640 1 -- 172.21.15.59:0/3132395988 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6188044020 con 0x7f619c072680 2024-04-23T14:36:24.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.854+0000 7f6193400640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f617c06c8f0 0x7f617c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:24.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.854+0000 7f6193400640 1 -- 172.21.15.59:0/3132395988 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f61880abee0 con 0x7f619c072680 2024-04-23T14:36:24.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.854+0000 7f619b400640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f617c06c8f0 0x7f617c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:24.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.855+0000 7f619b400640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f617c06c8f0 0x7f617c06edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f618c005880 tx=0x7f618c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:24.859+0000 7f6193400640 1 -- 172.21.15.59:0/3132395988 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6188032e20 con 0x7f619c072680 2024-04-23T14:36:25.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.077+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f619c15cf70 con 0x7f619c072680 2024-04-23T14:36:25.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.175+0000 7f6193400640 1 -- 172.21.15.59:0/3132395988 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7f618807a570 con 0x7f619c072680 2024-04-23T14:36:25.176 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.178+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f617c06c8f0 msgr2=0x7f617c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.178+0000 7f61a2200640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f617c06c8f0 0x7f617c06edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f618c005880 tx=0x7f618c009040 comp rx=0 tx=0).stop 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.178+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c072680 msgr2=0x7f619c155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.178+0000 7f61a2200640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c072680 0x7f619c155790 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6188035040 tx=0x7f61880046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 shutdown_connections 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f617c06c8f0 0x7f617c06edb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f619c0cf410 0x7f619c14d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f619c072680 0x7f619c155790 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 --2- 172.21.15.59:0/3132395988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f619c06ddf0 0x7f619c0af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 >> 172.21.15.59:0/3132395988 conn(0x7f619c0b2130 msgr2=0x7f619c15aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:25.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 shutdown_connections 2024-04-23T14:36:25.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.179+0000 7f61a2200640 1 -- 172.21.15.59:0/3132395988 wait complete. 2024-04-23T14:36:25.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:36:25.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.392+0000 7f8add600640 1 Processor -- start 2024-04-23T14:36:25.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.392+0000 7f8add600640 1 -- start start 2024-04-23T14:36:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.394+0000 7f8add600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ad815c5e0 0x7f8ad80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.394+0000 7f8add600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ad80af920 0x7f8ad80d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.394+0000 7f8add600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad815f0f0 con 0x7f8ad80d06e0 2024-04-23T14:36:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.394+0000 7f8add600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad80afd80 con 0x7f8ad80af920 2024-04-23T14:36:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.394+0000 7f8add600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad815fa20 con 0x7f8ad815c5e0 2024-04-23T14:36:25.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.394+0000 7f8ad6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ad80af920 0x7f8ad80d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.395+0000 7f8ad6a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ad815c5e0 0x7f8ad80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.395+0000 7f8ad6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ad80af920 0x7f8ad80d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40574/0 (socket says 172.21.15.59:40574) 2024-04-23T14:36:25.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.395+0000 7f8ad6000640 1 -- 172.21.15.59:0/2966569147 learned_addr learned my addr 172.21.15.59:0/2966569147 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.395+0000 7f8ad6000640 1 -- 172.21.15.59:0/2966569147 >> v2:172.21.15.76:3300/0 conn(0x7f8ad815c5e0 msgr2=0x7f8ad80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.395+0000 7f8ad6000640 1 --2- 172.21.15.59:0/2966569147 >> v2:172.21.15.76:3300/0 conn(0x7f8ad815c5e0 0x7f8ad80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.395+0000 7f8ad6000640 1 -- 172.21.15.59:0/2966569147 >> v1:172.21.15.76:6789/0 conn(0x7f8ad80d06e0 legacy=0x7f8ad8071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.395+0000 7f8ad6000640 1 -- 172.21.15.59:0/2966569147 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad815f3d0 con 0x7f8ad80af920 2024-04-23T14:36:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.397+0000 7f8ad6000640 1 --2- 172.21.15.59:0/2966569147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80af920 0x7f8ad80d01a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8072e60 tx=0x7f8ac402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6169da0d532f5aec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:25.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.397+0000 7f8ad5600640 1 -- 172.21.15.59:0/2966569147 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4037070 con 0x7f8ad80af920 2024-04-23T14:36:25.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.397+0000 7f8ad5600640 1 -- 172.21.15.59:0/2966569147 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac4032440 con 0x7f8ad80af920 2024-04-23T14:36:25.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.397+0000 7f8ad5600640 1 -- 172.21.15.59:0/2966569147 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4032810 con 0x7f8ad80af920 2024-04-23T14:36:25.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 -- 172.21.15.59:0/2966569147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80af920 msgr2=0x7f8ad80d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 --2- 172.21.15.59:0/2966569147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80af920 0x7f8ad80d01a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8072e60 tx=0x7f8ac402a270 comp rx=0 tx=0).stop 2024-04-23T14:36:25.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 -- 172.21.15.59:0/2966569147 shutdown_connections 2024-04-23T14:36:25.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 --2- 172.21.15.59:0/2966569147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80af920 0x7f8ad80d01a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 --2- 172.21.15.59:0/2966569147 >> v2:172.21.15.76:3300/0 conn(0x7f8ad815c5e0 0x7f8ad80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 -- 172.21.15.59:0/2966569147 >> 172.21.15.59:0/2966569147 conn(0x7f8ad8067ea0 msgr2=0x7f8ad806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:25.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 -- 172.21.15.59:0/2966569147 shutdown_connections 2024-04-23T14:36:25.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.398+0000 7f8add600640 1 -- 172.21.15.59:0/2966569147 wait complete. 2024-04-23T14:36:25.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.400+0000 7f8add600640 1 Processor -- start 2024-04-23T14:36:25.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.400+0000 7f8add600640 1 -- start start 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.401+0000 7f8add600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad80af920 0x7f8ad8155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.401+0000 7f8add600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80d06e0 0x7f8ad80b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.401+0000 7f8add600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ad815c5e0 0x7f8ad80b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.401+0000 7f8add600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80d0cf0 con 0x7f8ad80d06e0 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.401+0000 7f8add600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80d0b70 con 0x7f8ad80af920 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.401+0000 7f8add600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad806e480 con 0x7f8ad815c5e0 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80d06e0 0x7f8ad80b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80d06e0 0x7f8ad80b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40586/0 (socket says 172.21.15.59:40586) 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 -- 172.21.15.59:0/1859643367 learned_addr learned my addr 172.21.15.59:0/1859643367 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad7400640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ad815c5e0 0x7f8ad80b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6a00640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad80af920 0x7f8ad8155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 -- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ad815c5e0 msgr2=0x7f8ad80b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ad815c5e0 0x7f8ad80b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 -- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad80af920 msgr2=0x7f8ad8155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad80af920 0x7f8ad8155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 -- 172.21.15.59:0/1859643367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad81562a0 con 0x7f8ad80d06e0 2024-04-23T14:36:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.402+0000 7f8ad6000640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80d06e0 0x7f8ad80b07e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4000c00 tx=0x7f8ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.403+0000 7f8acea00640 1 -- 172.21.15.59:0/1859643367 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4041070 con 0x7f8ad80d06e0 2024-04-23T14:36:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.403+0000 7f8acea00640 1 -- 172.21.15.59:0/1859643367 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac403c870 con 0x7f8ad80d06e0 2024-04-23T14:36:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.403+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad814a510 con 0x7f8ad80d06e0 2024-04-23T14:36:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.403+0000 7f8acea00640 1 -- 172.21.15.59:0/1859643367 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4037040 con 0x7f8ad80d06e0 2024-04-23T14:36:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.403+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad80cceb0 con 0x7f8ad80d06e0 2024-04-23T14:36:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.404+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad8068a80 con 0x7f8ad80d06e0 2024-04-23T14:36:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.405+0000 7f8acea00640 1 -- 172.21.15.59:0/1859643367 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ac4044070 con 0x7f8ad80d06e0 2024-04-23T14:36:25.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.407+0000 7f8acea00640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ab806c8f0 0x7f8ab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.407+0000 7f8acea00640 1 -- 172.21.15.59:0/1859643367 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f8ac40ac220 con 0x7f8ad80d06e0 2024-04-23T14:36:25.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.407+0000 7f8ad6a00640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ab806c8f0 0x7f8ab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.408+0000 7f8ad6a00640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ab806c8f0 0x7f8ab806edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009a30 tx=0x7f8ac8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.411+0000 7f8acea00640 1 -- 172.21.15.59:0/1859643367 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ac407aa20 con 0x7f8ad80d06e0 2024-04-23T14:36:25.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.691+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8ad8161e10 con 0x7f8ad80d06e0 2024-04-23T14:36:25.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.692+0000 7f8acea00640 1 -- 172.21.15.59:0/1859643367 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f8ac40aae20 con 0x7f8ad80d06e0 2024-04-23T14:36:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A==", 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA==", 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw==", 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.695 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg==", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow==", 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ==", 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ==", 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-23T14:36:25.698 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g==", 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA==", 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA==", 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-23T14:36:25.700 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA==", 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ==", 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ==", 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-23T14:36:25.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q==", 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg==", 2024-04-23T14:36:25.703 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg==", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:36:25.704 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmEteHAhAAwfUHmvRmEMcPIphwikq6sw==", 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-23T14:36:25.705 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA==", 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ==", 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:36:25.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:25.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:36:25.707 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ab806c8f0 msgr2=0x7f8ab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ab806c8f0 0x7f8ab806edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009a30 tx=0x7f8ac8008040 comp rx=0 tx=0).stop 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80d06e0 msgr2=0x7f8ad80b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80d06e0 0x7f8ad80b07e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4000c00 tx=0x7f8ac40046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 shutdown_connections 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ab806c8f0 0x7f8ab806edb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ad815c5e0 0x7f8ad80b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ad80d06e0 0x7f8ad80b07e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 --2- 172.21.15.59:0/1859643367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad80af920 0x7f8ad8155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 >> 172.21.15.59:0/1859643367 conn(0x7f8ad8067ea0 msgr2=0x7f8ad80cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:25.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 shutdown_connections 2024-04-23T14:36:25.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.695+0000 7f8add600640 1 -- 172.21.15.59:0/1859643367 wait complete. 2024-04-23T14:36:25.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.920+0000 7f9751a00640 1 Processor -- start 2024-04-23T14:36:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.920+0000 7f9751a00640 1 -- start start 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.921+0000 7f9751a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f974c0cd050 0x7f974c06f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.921+0000 7f9751a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f974c0b3080 0x7f974c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.921+0000 7f9751a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974c15f160 con 0x7f974c15ece0 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.921+0000 7f9751a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f974c0ced20 con 0x7f974c0cd050 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.921+0000 7f9751a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f974c15fcd0 con 0x7f974c0b3080 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f974c0cd050 0x7f974c06f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f974c0cd050 0x7f974c06f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40590/0 (socket says 172.21.15.59:40590) 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974be00640 1 -- 172.21.15.59:0/3284044693 learned_addr learned my addr 172.21.15.59:0/3284044693 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f974c15ece0 0x7f974c0cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:48402/0 (socket says 172.21.15.59:48402) 2024-04-23T14:36:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974be00640 1 -- 172.21.15.59:0/3284044693 >> v2:172.21.15.76:3300/0 conn(0x7f974c0b3080 msgr2=0x7f974c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f9751000640 1 --2- 172.21.15.59:0/3284044693 >> v2:172.21.15.76:3300/0 conn(0x7f974c0b3080 0x7f974c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974be00640 1 --2- 172.21.15.59:0/3284044693 >> v2:172.21.15.76:3300/0 conn(0x7f974c0b3080 0x7f974c0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974be00640 1 -- 172.21.15.59:0/3284044693 >> v1:172.21.15.59:6789/0 conn(0x7f974c15ece0 legacy=0x7f974c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974be00640 1 -- 172.21.15.59:0/3284044693 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c15f430 con 0x7f974c0cd050 2024-04-23T14:36:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f974aa00640 1 -- 172.21.15.59:0/3284044693 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 180720221 0 0) 0x7f974c15f160 con 0x7f974c15ece0 2024-04-23T14:36:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.922+0000 7f9751000640 1 --2- 172.21.15.59:0/3284044693 >> v2:172.21.15.76:3300/0 conn(0x7f974c0b3080 0x7f974c0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.924+0000 7f974be00640 1 --2- 172.21.15.59:0/3284044693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0cd050 0x7f974c06f620 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f974c0cc960 tx=0x7f973c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638565e887b71d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.924+0000 7f974aa00640 1 -- 172.21.15.59:0/3284044693 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c037070 con 0x7f974c0cd050 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.924+0000 7f974aa00640 1 -- 172.21.15.59:0/3284044693 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c032440 con 0x7f974c0cd050 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.924+0000 7f974aa00640 1 -- 172.21.15.59:0/3284044693 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c032810 con 0x7f974c0cd050 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 -- 172.21.15.59:0/3284044693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0cd050 msgr2=0x7f974c06f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 --2- 172.21.15.59:0/3284044693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0cd050 0x7f974c06f620 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f974c0cc960 tx=0x7f973c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 -- 172.21.15.59:0/3284044693 shutdown_connections 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 --2- 172.21.15.59:0/3284044693 >> v2:172.21.15.76:3300/0 conn(0x7f974c0b3080 0x7f974c0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 --2- 172.21.15.59:0/3284044693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0cd050 0x7f974c06f620 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 -- 172.21.15.59:0/3284044693 >> 172.21.15.59:0/3284044693 conn(0x7f974c0b21a0 msgr2=0x7f974c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 -- 172.21.15.59:0/3284044693 shutdown_connections 2024-04-23T14:36:25.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.925+0000 7f9751a00640 1 -- 172.21.15.59:0/3284044693 wait complete. 2024-04-23T14:36:25.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.927+0000 7f9751a00640 1 Processor -- start 2024-04-23T14:36:25.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.927+0000 7f9751a00640 1 -- start start 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f9751a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0b3080 0x7f974c0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f9751a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f974c0cd050 0x7f974c09d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f9751a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f974c15ece0 0x7f974c072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f9751a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f974c0cef50 con 0x7f974c0b3080 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f9751a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f974c15f9f0 con 0x7f974c15ece0 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f9751a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f974c159110 con 0x7f974c0cd050 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f974be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0b3080 0x7f974c0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f974be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0b3080 0x7f974c0c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40592/0 (socket says 172.21.15.59:40592) 2024-04-23T14:36:25.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.928+0000 7f974be00640 1 -- 172.21.15.59:0/145850066 learned_addr learned my addr 172.21.15.59:0/145850066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f974be00640 1 -- 172.21.15.59:0/145850066 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f974c0cd050 msgr2=0x7f974c09d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f974be00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f974c0cd050 0x7f974c09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f974be00640 1 -- 172.21.15.59:0/145850066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f974c15ece0 msgr2=0x7f974c072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f974be00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f974c15ece0 0x7f974c072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f974be00640 1 -- 172.21.15.59:0/145850066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c1f87a0 con 0x7f974c0b3080 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f974be00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0b3080 0x7f974c0c8f20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f973c02a220 tx=0x7f973c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f9743e00640 1 -- 172.21.15.59:0/145850066 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c037040 con 0x7f974c0b3080 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f9743e00640 1 -- 172.21.15.59:0/145850066 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c042070 con 0x7f974c0b3080 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f9743e00640 1 -- 172.21.15.59:0/145850066 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c030bb0 con 0x7f974c0b3080 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c0c8440 con 0x7f974c0b3080 2024-04-23T14:36:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.929+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c0c8a00 con 0x7f974c0b3080 2024-04-23T14:36:25.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.931+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974c04fd90 con 0x7f974c0b3080 2024-04-23T14:36:25.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.932+0000 7f9743e00640 1 -- 172.21.15.59:0/145850066 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f973c047460 con 0x7f974c0b3080 2024-04-23T14:36:25.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.933+0000 7f9743e00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f972c06c8f0 0x7f972c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:25.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.933+0000 7f9743e00640 1 -- 172.21.15.59:0/145850066 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f973c0ac510 con 0x7f974c0b3080 2024-04-23T14:36:25.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.933+0000 7f974b400640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f972c06c8f0 0x7f972c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:25.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.934+0000 7f974b400640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f972c06c8f0 0x7f972c06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9738007fd0 tx=0x7f9738007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:25.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:25.938+0000 7f9743e00640 1 -- 172.21.15.59:0/145850066 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f973c045360 con 0x7f974c0b3080 2024-04-23T14:36:26.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.144+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f974c0b3f80 con 0x7f974c0b3080 2024-04-23T14:36:26.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.146+0000 7f9743e00640 1 -- 172.21.15.59:0/145850066 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+12831 (secure 0 0 0) 0x7f973c07abf0 con 0x7f974c0b3080 2024-04-23T14:36:26.148 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:26.148 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":65,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:25.192375+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:26.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f972c06c8f0 msgr2=0x7f972c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f972c06c8f0 0x7f972c06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9738007fd0 tx=0x7f9738007960 comp rx=0 tx=0).stop 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0b3080 msgr2=0x7f974c0c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0b3080 0x7f974c0c8f20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f973c02a220 tx=0x7f973c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 shutdown_connections 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f972c06c8f0 0x7f972c06edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f974c0cd050 0x7f974c09d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f974c15ece0 0x7f974c072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 --2- 172.21.15.59:0/145850066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f974c0b3080 0x7f974c0c8f20 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 >> 172.21.15.59:0/145850066 conn(0x7f974c0b21a0 msgr2=0x7f974c0cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.149+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 shutdown_connections 2024-04-23T14:36:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.150+0000 7f9751a00640 1 -- 172.21.15.59:0/145850066 wait complete. 2024-04-23T14:36:26.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:36:26.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:36:26.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.377+0000 7f28e4000640 1 Processor -- start 2024-04-23T14:36:26.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.377+0000 7f28e4000640 1 -- start start 2024-04-23T14:36:26.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e4000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28dc145e10 0x7f28dc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e4000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e4000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28dc071d70 0x7f28dc072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e4000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f28dc0cfc20 con 0x7f28dc071d70 2024-04-23T14:36:26.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e4000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f28dc0cff00 con 0x7f28dc145e10 2024-04-23T14:36:26.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e4000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f28dc0d01e0 con 0x7f28dc0d0cc0 2024-04-23T14:36:26.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28dc071d70 0x7f28dc072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28dc071d70 0x7f28dc072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40596/0 (socket says 172.21.15.59:40596) 2024-04-23T14:36:26.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.379+0000 7f28e2000640 1 -- 172.21.15.59:0/3607892114 learned_addr learned my addr 172.21.15.59:0/3607892114 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e1600640 1 --2- 172.21.15.59:0/3607892114 >> v2:172.21.15.76:3300/0 conn(0x7f28dc145e10 0x7f28dc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e0c00640 1 --2- 172.21.15.59:0/3607892114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e2000640 1 -- 172.21.15.59:0/3607892114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 msgr2=0x7f28dc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e2000640 1 --2- 172.21.15.59:0/3607892114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc0cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e2000640 1 -- 172.21.15.59:0/3607892114 >> v2:172.21.15.76:3300/0 conn(0x7f28dc145e10 msgr2=0x7f28dc06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e2000640 1 --2- 172.21.15.59:0/3607892114 >> v2:172.21.15.76:3300/0 conn(0x7f28dc145e10 0x7f28dc06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e2000640 1 -- 172.21.15.59:0/3607892114 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc0d04c0 con 0x7f28dc071d70 2024-04-23T14:36:26.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.380+0000 7f28e0c00640 1 --2- 172.21.15.59:0/3607892114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:26.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.381+0000 7f28e2000640 1 --2- 172.21.15.59:0/3607892114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc071d70 0x7f28dc072200 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009e50 tx=0x7f28d402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dceed948181c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:26.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.381+0000 7f28dbe00640 1 -- 172.21.15.59:0/3607892114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4036070 con 0x7f28dc071d70 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.381+0000 7f28dbe00640 1 -- 172.21.15.59:0/3607892114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d4031600 con 0x7f28dc071d70 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28dbe00640 1 -- 172.21.15.59:0/3607892114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d40319d0 con 0x7f28dc071d70 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 -- 172.21.15.59:0/3607892114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc071d70 msgr2=0x7f28dc072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 --2- 172.21.15.59:0/3607892114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc071d70 0x7f28dc072200 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009e50 tx=0x7f28d402a6d0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 -- 172.21.15.59:0/3607892114 shutdown_connections 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 --2- 172.21.15.59:0/3607892114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc071d70 0x7f28dc072200 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 --2- 172.21.15.59:0/3607892114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 --2- 172.21.15.59:0/3607892114 >> v2:172.21.15.76:3300/0 conn(0x7f28dc145e10 0x7f28dc06f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 -- 172.21.15.59:0/3607892114 >> 172.21.15.59:0/3607892114 conn(0x7f28dc067d90 msgr2=0x7f28dc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.382+0000 7f28e4000640 1 -- 172.21.15.59:0/3607892114 shutdown_connections 2024-04-23T14:36:26.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.383+0000 7f28e4000640 1 -- 172.21.15.59:0/3607892114 wait complete. 2024-04-23T14:36:26.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.384+0000 7f28e4000640 1 Processor -- start 2024-04-23T14:36:26.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.384+0000 7f28e4000640 1 -- start start 2024-04-23T14:36:26.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.385+0000 7f28e4000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28dc071d70 0x7f28dc1416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.385+0000 7f28e4000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc13f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.385+0000 7f28e4000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc145e10 0x7f28dc13f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.385+0000 7f28e4000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc0cfa00 con 0x7f28dc145e10 2024-04-23T14:36:26.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.385+0000 7f28e4000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc0cf880 con 0x7f28dc071d70 2024-04-23T14:36:26.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.385+0000 7f28e4000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f28dc1461f0 con 0x7f28dc0d0cc0 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc145e10 0x7f28dc13f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc145e10 0x7f28dc13f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40602/0 (socket says 172.21.15.59:40602) 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 -- 172.21.15.59:0/4149434499 learned_addr learned my addr 172.21.15.59:0/4149434499 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e0c00640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc13f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e1600640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28dc071d70 0x7f28dc1416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 -- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 msgr2=0x7f28dc13f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc13f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 -- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28dc071d70 msgr2=0x7f28dc1416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28dc071d70 0x7f28dc1416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.386+0000 7f28e2000640 1 -- 172.21.15.59:0/4149434499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc1f26e0 con 0x7f28dc145e10 2024-04-23T14:36:26.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.387+0000 7f28e2000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc145e10 0x7f28dc13f9e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009f70 tx=0x7f28d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:26.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.387+0000 7f28d9600640 1 -- 172.21.15.59:0/4149434499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d403f020 con 0x7f28dc145e10 2024-04-23T14:36:26.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.387+0000 7f28d9600640 1 -- 172.21.15.59:0/4149434499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d4037040 con 0x7f28dc145e10 2024-04-23T14:36:26.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.387+0000 7f28d9600640 1 -- 172.21.15.59:0/4149434499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4036070 con 0x7f28dc145e10 2024-04-23T14:36:26.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.387+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc0c60d0 con 0x7f28dc145e10 2024-04-23T14:36:26.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.387+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc0c6650 con 0x7f28dc145e10 2024-04-23T14:36:26.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.388+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28dc068a80 con 0x7f28dc145e10 2024-04-23T14:36:26.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.389+0000 7f28d9600640 1 -- 172.21.15.59:0/4149434499 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28d4042020 con 0x7f28dc145e10 2024-04-23T14:36:26.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.390+0000 7f28d9600640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.390+0000 7f28d9600640 1 -- 172.21.15.59:0/4149434499 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f28d40abd10 con 0x7f28dc145e10 2024-04-23T14:36:26.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.391+0000 7f28e1600640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.391+0000 7f28e1600640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28bc06c8f0 0x7f28bc06edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f28d0006fd0 tx=0x7f28d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:26.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.395+0000 7f28d9600640 1 -- 172.21.15.59:0/4149434499 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28d40b1020 con 0x7f28dc145e10 2024-04-23T14:36:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.600+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28dc141e20 con 0x7f28dc145e10 2024-04-23T14:36:26.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.601+0000 7f28d9600640 1 -- 172.21.15.59:0/4149434499 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f28d4049030 con 0x7f28dc145e10 2024-04-23T14:36:26.602 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28bc06c8f0 msgr2=0x7f28bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28bc06c8f0 0x7f28bc06edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f28d0006fd0 tx=0x7f28d0008040 comp rx=0 tx=0).stop 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc145e10 msgr2=0x7f28dc13f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc145e10 0x7f28dc13f9e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009f70 tx=0x7f28d4004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 shutdown_connections 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28bc06c8f0 0x7f28bc06edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28dc0d0cc0 0x7f28dc13f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28dc145e10 0x7f28dc13f9e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 --2- 172.21.15.59:0/4149434499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28dc071d70 0x7f28dc1416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 >> 172.21.15.59:0/4149434499 conn(0x7f28dc067d90 msgr2=0x7f28dc0c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 shutdown_connections 2024-04-23T14:36:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.604+0000 7f28e4000640 1 -- 172.21.15.59:0/4149434499 wait complete. 2024-04-23T14:36:26.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:36:26.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.834+0000 7f626ca00640 1 Processor -- start 2024-04-23T14:36:26.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.834+0000 7f626ca00640 1 -- start start 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f626ca00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62681454e0 0x7f6268145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f626ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62680cd020 0x7f62680b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f626ca00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62680cd520 con 0x7f62680b3e40 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f626ca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f62680cddc0 con 0x7f62680cd020 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f626ca00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f62680cd800 con 0x7f62681454e0 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f6265400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62680cd020 0x7f62680b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f6265400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62680cd020 0x7f62680b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40610/0 (socket says 172.21.15.59:40610) 2024-04-23T14:36:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f6265400640 1 -- 172.21.15.59:0/2188035799 learned_addr learned my addr 172.21.15.59:0/2188035799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.836+0000 7f6265e00640 1 --2- 172.21.15.59:0/2188035799 >> v2:172.21.15.76:3300/0 conn(0x7f62681454e0 0x7f6268145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.837+0000 7f6265400640 1 -- 172.21.15.59:0/2188035799 >> v2:172.21.15.76:3300/0 conn(0x7f62681454e0 msgr2=0x7f6268145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.837+0000 7f6265400640 1 --2- 172.21.15.59:0/2188035799 >> v2:172.21.15.76:3300/0 conn(0x7f62681454e0 0x7f6268145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.837+0000 7f6265400640 1 -- 172.21.15.59:0/2188035799 >> v1:172.21.15.76:6789/0 conn(0x7f62680b3e40 legacy=0x7f6268160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.837+0000 7f6265400640 1 -- 172.21.15.59:0/2188035799 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62680cdae0 con 0x7f62680cd020 2024-04-23T14:36:26.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.838+0000 7f6265400640 1 --2- 172.21.15.59:0/2188035799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680cd020 0x7f62680b3900 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6268073280 tx=0x7f625402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfdefd35d24644e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:26.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f6264a00640 1 -- 172.21.15.59:0/2188035799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254037070 con 0x7f62680cd020 2024-04-23T14:36:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f6264a00640 1 -- 172.21.15.59:0/2188035799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6254032440 con 0x7f62680cd020 2024-04-23T14:36:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f6264a00640 1 -- 172.21.15.59:0/2188035799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254032810 con 0x7f62680cd020 2024-04-23T14:36:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f626ca00640 1 -- 172.21.15.59:0/2188035799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680cd020 msgr2=0x7f62680b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f626ca00640 1 --2- 172.21.15.59:0/2188035799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680cd020 0x7f62680b3900 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6268073280 tx=0x7f625402a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f626ca00640 1 -- 172.21.15.59:0/2188035799 shutdown_connections 2024-04-23T14:36:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f626ca00640 1 --2- 172.21.15.59:0/2188035799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680cd020 0x7f62680b3900 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f626ca00640 1 --2- 172.21.15.59:0/2188035799 >> v2:172.21.15.76:3300/0 conn(0x7f62681454e0 0x7f6268145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f626ca00640 1 -- 172.21.15.59:0/2188035799 >> 172.21.15.59:0/2188035799 conn(0x7f6268067ea0 msgr2=0x7f626806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.839+0000 7f626ca00640 1 -- 172.21.15.59:0/2188035799 shutdown_connections 2024-04-23T14:36:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.840+0000 7f626ca00640 1 -- 172.21.15.59:0/2188035799 wait complete. 2024-04-23T14:36:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.841+0000 7f626ca00640 1 Processor -- start 2024-04-23T14:36:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.841+0000 7f626ca00640 1 -- start start 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.842+0000 7f626ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62680b3e40 0x7f626815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.842+0000 7f626ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62680cd020 0x7f62680b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.842+0000 7f626ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62681454e0 0x7f62680b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.842+0000 7f626ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6268146270 con 0x7f62681454e0 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.842+0000 7f626ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f62681460f0 con 0x7f62680cd020 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.842+0000 7f626ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6268161020 con 0x7f62680b3e40 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62681454e0 0x7f62680b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62681454e0 0x7f62680b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40612/0 (socket says 172.21.15.59:40612) 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 -- 172.21.15.59:0/1311781785 learned_addr learned my addr 172.21.15.59:0/1311781785 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6265400640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62680cd020 0x7f62680b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6265e00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62680b3e40 0x7f626815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 -- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62680b3e40 msgr2=0x7f626815e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62680b3e40 0x7f626815e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 -- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62680cd020 msgr2=0x7f62680b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62680cd020 0x7f62680b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.843+0000 7f6266800640 1 -- 172.21.15.59:0/1311781785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626815da10 con 0x7f62681454e0 2024-04-23T14:36:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.844+0000 7f6266800640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62681454e0 0x7f62680b47b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f626000c780 tx=0x7f626000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.844+0000 7f625e000640 1 -- 172.21.15.59:0/1311781785 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260004b60 con 0x7f62681454e0 2024-04-23T14:36:26.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.844+0000 7f625e000640 1 -- 172.21.15.59:0/1311781785 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6260004d00 con 0x7f62681454e0 2024-04-23T14:36:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.844+0000 7f625e000640 1 -- 172.21.15.59:0/1311781785 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626000aa90 con 0x7f62681454e0 2024-04-23T14:36:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.844+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626815dcf0 con 0x7f62681454e0 2024-04-23T14:36:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.844+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6268149900 con 0x7f62681454e0 2024-04-23T14:36:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.845+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6268068a80 con 0x7f62681454e0 2024-04-23T14:36:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.846+0000 7f625e000640 1 -- 172.21.15.59:0/1311781785 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f626000ac30 con 0x7f62681454e0 2024-04-23T14:36:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.848+0000 7f625e000640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624806c8f0 0x7f624806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.848+0000 7f625e000640 1 -- 172.21.15.59:0/1311781785 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f626008bb10 con 0x7f62681454e0 2024-04-23T14:36:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.848+0000 7f6265e00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624806c8f0 0x7f624806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.849+0000 7f6265e00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624806c8f0 0x7f624806edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6258006fd0 tx=0x7f6258008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:26.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:26.852+0000 7f625e000640 1 -- 172.21.15.59:0/1311781785 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6260020070 con 0x7f62681454e0 2024-04-23T14:36:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.057+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6268160030 con 0x7f62681454e0 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.058+0000 7f625e000640 1 -- 172.21.15.59:0/1311781785 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) ==== 84+0+65 (secure 0 0 0) 0x7f626005a280 con 0x7f62681454e0 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:27.060 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:36:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624806c8f0 msgr2=0x7f624806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624806c8f0 0x7f624806edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6258006fd0 tx=0x7f6258008040 comp rx=0 tx=0).stop 2024-04-23T14:36:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62681454e0 msgr2=0x7f62680b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62681454e0 0x7f62680b47b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f626000c780 tx=0x7f626000cc40 comp rx=0 tx=0).stop 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 shutdown_connections 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f624806c8f0 0x7f624806edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62681454e0 0x7f62680b47b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62680cd020 0x7f62680b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.061+0000 7f626ca00640 1 --2- 172.21.15.59:0/1311781785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62680b3e40 0x7f626815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.062+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 >> 172.21.15.59:0/1311781785 conn(0x7f6268067ea0 msgr2=0x7f6268071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.062+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 shutdown_connections 2024-04-23T14:36:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.062+0000 7f626ca00640 1 -- 172.21.15.59:0/1311781785 wait complete. 2024-04-23T14:36:27.081 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:36:27.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:36:27.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.287+0000 7f252fe00640 1 Processor -- start 2024-04-23T14:36:27.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.287+0000 7f252fe00640 1 -- start start 2024-04-23T14:36:27.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.289+0000 7f252fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f253006c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:27.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.289+0000 7f252fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2530160100 0x7f2530160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:27.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.289+0000 7f252fe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2530157000 con 0x7f2530156c20 2024-04-23T14:36:27.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.289+0000 7f252fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f25300afb00 con 0x7f2530160100 2024-04-23T14:36:27.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.289+0000 7f252fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f253015cc20 con 0x7f253006bdf0 2024-04-23T14:36:27.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.290+0000 7f252f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2530160100 0x7f2530160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:27.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.290+0000 7f2526000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f253006c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:27.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.290+0000 7f2526000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f253006c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:42906/0 (socket says 172.21.15.59:42906) 2024-04-23T14:36:27.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.290+0000 7f2526000640 1 -- 172.21.15.59:0/2706757498 learned_addr learned my addr 172.21.15.59:0/2706757498 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:27.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.290+0000 7f252f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2530160100 0x7f2530160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:48204/0 (socket says 172.21.15.59:48204) 2024-04-23T14:36:27.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.291+0000 7f2526000640 1 -- 172.21.15.59:0/2706757498 >> v2:172.21.15.76:3300/0 conn(0x7f2530160100 msgr2=0x7f2530160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:27.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.291+0000 7f2526000640 1 --2- 172.21.15.59:0/2706757498 >> v2:172.21.15.76:3300/0 conn(0x7f2530160100 0x7f2530160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.291+0000 7f2526000640 1 -- 172.21.15.59:0/2706757498 >> v1:172.21.15.76:6789/0 conn(0x7f2530156c20 legacy=0x7f25300aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:27.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.291+0000 7f2526000640 1 -- 172.21.15.59:0/2706757498 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253015cf00 con 0x7f253006bdf0 2024-04-23T14:36:27.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.292+0000 7f2526000640 1 --2- 172.21.15.59:0/2706757498 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f253006c270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2518009870 tx=0x7f251802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98946f480fdeb9c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:27.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.293+0000 7f252e000640 1 -- 172.21.15.59:0/2706757498 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518037070 con 0x7f253006bdf0 2024-04-23T14:36:27.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.293+0000 7f252e000640 1 -- 172.21.15.59:0/2706757498 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2518032440 con 0x7f253006bdf0 2024-04-23T14:36:27.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.293+0000 7f252e000640 1 -- 172.21.15.59:0/2706757498 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518032810 con 0x7f253006bdf0 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 -- 172.21.15.59:0/2706757498 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 msgr2=0x7f253006c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 --2- 172.21.15.59:0/2706757498 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f253006c270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2518009870 tx=0x7f251802a270 comp rx=0 tx=0).stop 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 -- 172.21.15.59:0/2706757498 shutdown_connections 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 --2- 172.21.15.59:0/2706757498 >> v2:172.21.15.76:3300/0 conn(0x7f2530160100 0x7f2530160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 --2- 172.21.15.59:0/2706757498 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f253006c270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 -- 172.21.15.59:0/2706757498 >> 172.21.15.59:0/2706757498 conn(0x7f25300b2200 msgr2=0x7f25300b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 -- 172.21.15.59:0/2706757498 shutdown_connections 2024-04-23T14:36:27.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.294+0000 7f252fe00640 1 -- 172.21.15.59:0/2706757498 wait complete. 2024-04-23T14:36:27.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.297+0000 7f252fe00640 1 Processor -- start 2024-04-23T14:36:27.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.297+0000 7f252fe00640 1 -- start start 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.298+0000 7f252fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f2530145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.298+0000 7f252fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2530156c20 0x7f2530159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.298+0000 7f252fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530160100 0x7f2530099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.298+0000 7f252fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f253015d9e0 con 0x7f2530160100 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.298+0000 7f252fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f253015d4c0 con 0x7f2530156c20 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.298+0000 7f252fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2530073ba0 con 0x7f253006bdf0 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530160100 0x7f2530099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:27.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530160100 0x7f2530099820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40620/0 (socket says 172.21.15.59:40620) 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 -- 172.21.15.59:0/623686919 learned_addr learned my addr 172.21.15.59:0/623686919 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252ea00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f2530145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f2526000640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2530156c20 0x7f2530159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 -- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 msgr2=0x7f2530145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f2530145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 -- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2530156c20 msgr2=0x7f2530159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2530156c20 0x7f2530159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:27.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.299+0000 7f252f400640 1 -- 172.21.15.59:0/623686919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253015a250 con 0x7f2530160100 2024-04-23T14:36:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.300+0000 7f252f400640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530160100 0x7f2530099820 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2528009d30 tx=0x7f252800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.300+0000 7f2527400640 1 -- 172.21.15.59:0/623686919 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528018020 con 0x7f2530160100 2024-04-23T14:36:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.300+0000 7f2527400640 1 -- 172.21.15.59:0/623686919 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2528014480 con 0x7f2530160100 2024-04-23T14:36:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.300+0000 7f2527400640 1 -- 172.21.15.59:0/623686919 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528014870 con 0x7f2530160100 2024-04-23T14:36:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.300+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25300ce470 con 0x7f2530160100 2024-04-23T14:36:27.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.300+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25300cea30 con 0x7f2530160100 2024-04-23T14:36:27.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.301+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2530068c90 con 0x7f2530160100 2024-04-23T14:36:27.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.303+0000 7f2527400640 1 -- 172.21.15.59:0/623686919 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2528005d10 con 0x7f2530160100 2024-04-23T14:36:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.304+0000 7f2527400640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f250c06c8f0 0x7f250c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.304+0000 7f2527400640 1 -- 172.21.15.59:0/623686919 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f252808c370 con 0x7f2530160100 2024-04-23T14:36:27.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.304+0000 7f252ea00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f250c06c8f0 0x7f250c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:27.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.305+0000 7f252ea00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f250c06c8f0 0x7f250c06edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2520001640 tx=0x7f2520009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:27.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.309+0000 7f2527400640 1 -- 172.21.15.59:0/623686919 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2530068c90 con 0x7f2530160100 2024-04-23T14:36:27.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:27.508+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f25300af820 con 0x7f2530160100 2024-04-23T14:36:28.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.208+0000 7f2527400640 1 -- 172.21.15.59:0/623686919 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) ==== 138+0+0 (secure 0 0 0) 0x7f25300af820 con 0x7f2530160100 2024-04-23T14:36:28.210 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:36:28.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f250c06c8f0 msgr2=0x7f250c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:28.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f250c06c8f0 0x7f250c06edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2520001640 tx=0x7f2520009040 comp rx=0 tx=0).stop 2024-04-23T14:36:28.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530160100 msgr2=0x7f2530099820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:28.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530160100 0x7f2530099820 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2528009d30 tx=0x7f252800c490 comp rx=0 tx=0).stop 2024-04-23T14:36:28.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 shutdown_connections 2024-04-23T14:36:28.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f250c06c8f0 0x7f250c06edb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2530160100 0x7f2530099820 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2530156c20 0x7f2530159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 --2- 172.21.15.59:0/623686919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f253006bdf0 0x7f2530145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.212+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 >> 172.21.15.59:0/623686919 conn(0x7f25300b2200 msgr2=0x7f2530071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:28.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.213+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 shutdown_connections 2024-04-23T14:36:28.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.213+0000 7f252fe00640 1 -- 172.21.15.59:0/623686919 wait complete. 2024-04-23T14:36:28.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:36:28.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.441+0000 7fa572400640 1 Processor -- start 2024-04-23T14:36:28.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.441+0000 7fa572400640 1 -- start start 2024-04-23T14:36:28.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.442+0000 7fa572400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa56c162b20 0x7fa56c158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:28.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.442+0000 7fa572400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c06f640 con 0x7fa56c162740 2024-04-23T14:36:28.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.442+0000 7fa572400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c06e5a0 con 0x7fa56c158f80 2024-04-23T14:36:28.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.442+0000 7fa572400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa56c159410 con 0x7fa56c162b20 2024-04-23T14:36:28.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.443+0000 7fa56be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa56c162b20 0x7fa56c158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:28.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.443+0000 7fa56be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa56c162b20 0x7fa56c158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40632/0 (socket says 172.21.15.59:40632) 2024-04-23T14:36:28.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.443+0000 7fa56be00640 1 -- 172.21.15.59:0/3032706558 learned_addr learned my addr 172.21.15.59:0/3032706558 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:28.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.443+0000 7fa56be00640 1 -- 172.21.15.59:0/3032706558 >> v1:172.21.15.76:6789/0 conn(0x7fa56c158f80 legacy=0x7fa56c0b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:28.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.443+0000 7fa56be00640 1 -- 172.21.15.59:0/3032706558 >> v1:172.21.15.59:6789/0 conn(0x7fa56c162740 legacy=0x7fa56c06dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:28.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.443+0000 7fa56be00640 1 -- 172.21.15.59:0/3032706558 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c06e880 con 0x7fa56c162b20 2024-04-23T14:36:28.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.444+0000 7fa56be00640 1 --2- 172.21.15.59:0/3032706558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162b20 0x7fa56c158a40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa558009870 tx=0x7fa55802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50241a60a3f05a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:28.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa56b400640 1 -- 172.21.15.59:0/3032706558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558037070 con 0x7fa56c162b20 2024-04-23T14:36:28.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa56b400640 1 -- 172.21.15.59:0/3032706558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa558032440 con 0x7fa56c162b20 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa56b400640 1 -- 172.21.15.59:0/3032706558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558032810 con 0x7fa56c162b20 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa572400640 1 -- 172.21.15.59:0/3032706558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162b20 msgr2=0x7fa56c158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa572400640 1 --2- 172.21.15.59:0/3032706558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162b20 0x7fa56c158a40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa558009870 tx=0x7fa55802a2a0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa572400640 1 -- 172.21.15.59:0/3032706558 shutdown_connections 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa572400640 1 --2- 172.21.15.59:0/3032706558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162b20 0x7fa56c158a40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.445+0000 7fa572400640 1 -- 172.21.15.59:0/3032706558 >> 172.21.15.59:0/3032706558 conn(0x7fa56c06bbb0 msgr2=0x7fa56c06bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.446+0000 7fa572400640 1 -- 172.21.15.59:0/3032706558 shutdown_connections 2024-04-23T14:36:28.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.446+0000 7fa572400640 1 -- 172.21.15.59:0/3032706558 wait complete. 2024-04-23T14:36:28.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.448+0000 7fa572400640 1 Processor -- start 2024-04-23T14:36:28.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.448+0000 7fa572400640 1 -- start start 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa572400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa56c158f80 0x7fa56c0cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa572400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162740 0x7fa56c0c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa572400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa56c162b20 0x7fa56c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa572400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c06ee40 con 0x7fa56c162740 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa572400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c06f1b0 con 0x7fa56c158f80 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa572400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c072c10 con 0x7fa56c162b20 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162740 0x7fa56c0c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162740 0x7fa56c0c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40634/0 (socket says 172.21.15.59:40634) 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 -- 172.21.15.59:0/3321084304 learned_addr learned my addr 172.21.15.59:0/3321084304 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa571000640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa56c158f80 0x7fa56c0cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 -- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa56c162b20 msgr2=0x7fa56c0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa56c162b20 0x7fa56c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 -- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa56c158f80 msgr2=0x7fa56c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:28.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa56c158f80 0x7fa56c0cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:28.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa56be00640 1 -- 172.21.15.59:0/3321084304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0c49c0 con 0x7fa56c162740 2024-04-23T14:36:28.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.449+0000 7fa571000640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa56c158f80 0x7fa56c0cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:28.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.450+0000 7fa56be00640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162740 0x7fa56c0c4190 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa558000c00 tx=0x7fa5580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:28.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.450+0000 7fa568c00640 1 -- 172.21.15.59:0/3321084304 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558041070 con 0x7fa56c162740 2024-04-23T14:36:28.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.450+0000 7fa568c00640 1 -- 172.21.15.59:0/3321084304 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55803c890 con 0x7fa56c162740 2024-04-23T14:36:28.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.450+0000 7fa568c00640 1 -- 172.21.15.59:0/3321084304 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558037040 con 0x7fa56c162740 2024-04-23T14:36:28.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.450+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c0c4ca0 con 0x7fa56c162740 2024-04-23T14:36:28.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.450+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c0c5260 con 0x7fa56c162740 2024-04-23T14:36:28.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.452+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c0ce130 con 0x7fa56c162740 2024-04-23T14:36:28.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.454+0000 7fa568c00640 1 -- 172.21.15.59:0/3321084304 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa558044070 con 0x7fa56c162740 2024-04-23T14:36:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.455+0000 7fa568c00640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa54c06c920 0x7fa54c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.455+0000 7fa568c00640 1 -- 172.21.15.59:0/3321084304 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5227+0+0 (secure 0 0 0) 0x7fa5580ac9f0 con 0x7fa56c162740 2024-04-23T14:36:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.455+0000 7fa571000640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa54c06c920 0x7fa54c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:28.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.456+0000 7fa571000640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa54c06c920 0x7fa54c06ede0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa560001680 tx=0x7fa560009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:28.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.459+0000 7fa568c00640 1 -- 172.21.15.59:0/3321084304 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa56c0ce130 con 0x7fa56c162740 2024-04-23T14:36:28.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:28.664+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa56c04fd90 con 0x7fa56c162740 2024-04-23T14:36:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.224+0000 7fa568c00640 1 -- 172.21.15.59:0/3321084304 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) ==== 170+0+0 (secure 0 0 0) 0x7fa55807fe20 con 0x7fa56c162740 2024-04-23T14:36:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:36:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa54c06c920 msgr2=0x7fa54c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa54c06c920 0x7fa54c06ede0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa560001680 tx=0x7fa560009040 comp rx=0 tx=0).stop 2024-04-23T14:36:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162740 msgr2=0x7fa56c0c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162740 0x7fa56c0c4190 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa558000c00 tx=0x7fa5580046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 shutdown_connections 2024-04-23T14:36:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa54c06c920 0x7fa54c06ede0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa56c162b20 0x7fa56c0b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa56c162740 0x7fa56c0c4190 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 --2- 172.21.15.59:0/3321084304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa56c158f80 0x7fa56c0cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.228+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 >> 172.21.15.59:0/3321084304 conn(0x7fa56c06bbb0 msgr2=0x7fa56c072f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.229+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 shutdown_connections 2024-04-23T14:36:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.229+0000 7fa572400640 1 -- 172.21.15.59:0/3321084304 wait complete. 2024-04-23T14:36:29.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:36:29.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.445+0000 7f2dbb200640 1 Processor -- start 2024-04-23T14:36:29.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.445+0000 7f2dbb200640 1 -- start start 2024-04-23T14:36:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.446+0000 7f2dbb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2dbc1567e0 0x7f2dbc156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.446+0000 7f2dbb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc0d0d10 0x7f2dbc160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.446+0000 7f2dbb200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dbc15cab0 con 0x7f2dbc15d810 2024-04-23T14:36:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2dbb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2dbc160e40 con 0x7f2dbc1567e0 2024-04-23T14:36:29.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2dbb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc15cd30 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2db9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2dbc1567e0 0x7f2dbc156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2db9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2dbc1567e0 0x7f2dbc156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40642/0 (socket says 172.21.15.59:40642) 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2db9e00640 1 -- 172.21.15.59:0/459623581 learned_addr learned my addr 172.21.15.59:0/459623581 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2dba800640 1 --2- 172.21.15.59:0/459623581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc0d0d10 0x7f2dbc160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2db9e00640 1 -- 172.21.15.59:0/459623581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc0d0d10 msgr2=0x7f2dbc160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2db9e00640 1 --2- 172.21.15.59:0/459623581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc0d0d10 0x7f2dbc160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2db9e00640 1 -- 172.21.15.59:0/459623581 >> v1:172.21.15.76:6789/0 conn(0x7f2dbc15d810 legacy=0x7f2dbc0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2db9e00640 1 -- 172.21.15.59:0/459623581 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc15d010 con 0x7f2dbc1567e0 2024-04-23T14:36:29.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.447+0000 7f2dba800640 1 --2- 172.21.15.59:0/459623581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc0d0d10 0x7f2dbc160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:29.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.448+0000 7f2db9e00640 1 --2- 172.21.15.59:0/459623581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc156bc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f39a80487a688b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:29.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.449+0000 7f2db9400640 1 -- 172.21.15.59:0/459623581 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac037070 con 0x7f2dbc1567e0 2024-04-23T14:36:29.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.449+0000 7f2db9400640 1 -- 172.21.15.59:0/459623581 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac032440 con 0x7f2dbc1567e0 2024-04-23T14:36:29.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.449+0000 7f2db9400640 1 -- 172.21.15.59:0/459623581 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac032810 con 0x7f2dbc1567e0 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.449+0000 7f2dbb200640 1 -- 172.21.15.59:0/459623581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc1567e0 msgr2=0x7f2dbc156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.449+0000 7f2dbb200640 1 --2- 172.21.15.59:0/459623581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc156bc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac02a200 comp rx=0 tx=0).stop 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.449+0000 7f2dbb200640 1 -- 172.21.15.59:0/459623581 shutdown_connections 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.450+0000 7f2dbb200640 1 --2- 172.21.15.59:0/459623581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc0d0d10 0x7f2dbc160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.450+0000 7f2dbb200640 1 --2- 172.21.15.59:0/459623581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc156bc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.450+0000 7f2dbb200640 1 -- 172.21.15.59:0/459623581 >> 172.21.15.59:0/459623581 conn(0x7f2dbc06bdf0 msgr2=0x7f2dbc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.450+0000 7f2dbb200640 1 -- 172.21.15.59:0/459623581 shutdown_connections 2024-04-23T14:36:29.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.450+0000 7f2dbb200640 1 -- 172.21.15.59:0/459623581 wait complete. 2024-04-23T14:36:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.452+0000 7f2dbb200640 1 Processor -- start 2024-04-23T14:36:29.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.452+0000 7f2dbb200640 1 -- start start 2024-04-23T14:36:29.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2dbb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc0d0d10 0x7f2dbc1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:29.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2dbb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:29.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2dbb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc15d810 0x7f2dbc14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:29.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2dbb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1607d0 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2dbb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc160650 con 0x7f2dbc1567e0 2024-04-23T14:36:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2dbb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc160950 con 0x7f2dbc15d810 2024-04-23T14:36:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2db9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc0d0d10 0x7f2dbc1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2db9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc0d0d10 0x7f2dbc1f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40654/0 (socket says 172.21.15.59:40654) 2024-04-23T14:36:29.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.453+0000 7f2db9e00640 1 -- 172.21.15.59:0/2796571246 learned_addr learned my addr 172.21.15.59:0/2796571246 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:29.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db9e00640 1 -- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc15d810 msgr2=0x7f2dbc14fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:29.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2dba800640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc15d810 0x7f2dbc14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:29.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db3e00640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db9e00640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc15d810 0x7f2dbc14fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db9e00640 1 -- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dbc1567e0 msgr2=0x7f2dbc14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db9e00640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc14f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db9e00640 1 -- 172.21.15.59:0/2796571246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc1f6110 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db3e00640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.454+0000 7f2db9e00640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc0d0d10 0x7f2dbc1f7170 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.455+0000 7f2db2000640 1 -- 172.21.15.59:0/2796571246 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac041020 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.455+0000 7f2db2000640 1 -- 172.21.15.59:0/2796571246 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac030970 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.455+0000 7f2db2000640 1 -- 172.21.15.59:0/2796571246 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac037040 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.455+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc15c480 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.455+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc1f6690 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.456+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dbc04fd90 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.457+0000 7f2db2000640 1 -- 172.21.15.59:0/2796571246 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dac044020 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.458+0000 7f2db2000640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.459+0000 7f2db2000640 1 -- 172.21.15.59:0/2796571246 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5609+0+0 (secure 0 0 0) 0x7f2dac0ac580 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.459+0000 7f2db3e00640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.460+0000 7f2db3e00640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d9806c8f0 0x7f2d9806edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0b45c0 tx=0x7f2da4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:29.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.464+0000 7f2db2000640 1 -- 172.21.15.59:0/2796571246 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2dac04b030 con 0x7f2dbc0d0d10 2024-04-23T14:36:29.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:29.708+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2dbc0b5150 con 0x7f2dbc0d0d10 2024-04-23T14:36:30.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.229+0000 7f2db2000640 1 -- 172.21.15.59:0/2796571246 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) ==== 174+0+0 (secure 0 0 0) 0x7f2dbc15cab0 con 0x7f2dbc0d0d10 2024-04-23T14:36:30.230 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 10 and data pool 11 2024-04-23T14:36:30.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d9806c8f0 msgr2=0x7f2d9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d9806c8f0 0x7f2d9806edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0b45c0 tx=0x7f2da4009040 comp rx=0 tx=0).stop 2024-04-23T14:36:30.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc0d0d10 msgr2=0x7f2dbc1f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc0d0d10 0x7f2dbc1f7170 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac0046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 shutdown_connections 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2dbc15d810 0x7f2dbc14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dbc1567e0 0x7f2dbc14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 --2- 172.21.15.59:0/2796571246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2dbc0d0d10 0x7f2dbc1f7170 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 >> 172.21.15.59:0/2796571246 conn(0x7f2dbc06bdf0 msgr2=0x7f2dbc071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.233+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 shutdown_connections 2024-04-23T14:36:30.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.234+0000 7f2dbb200640 1 -- 172.21.15.59:0/2796571246 wait complete. 2024-04-23T14:36:30.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:30.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.451+0000 7effaa600640 1 Processor -- start 2024-04-23T14:36:30.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.452+0000 7effaa600640 1 -- start start 2024-04-23T14:36:30.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.453+0000 7effaa600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7effa41454e0 0x7effa4145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.453+0000 7effaa600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa40cd520 con 0x7effa40cd020 2024-04-23T14:36:30.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.453+0000 7effaa600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa40cd800 con 0x7effa4160910 2024-04-23T14:36:30.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.453+0000 7effaa600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7effa40cddc0 con 0x7effa41454e0 2024-04-23T14:36:30.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7eff9be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7effa40cd020 0x7effa40b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35132/0 (socket says 172.21.15.59:35132) 2024-04-23T14:36:30.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7eff9be00640 1 -- 172.21.15.59:0/3211174442 learned_addr learned my addr 172.21.15.59:0/3211174442 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:30.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa3400640 1 --2- 172.21.15.59:0/3211174442 >> v2:172.21.15.76:3300/0 conn(0x7effa41454e0 0x7effa4145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111951566 0 0) 0x7effa40cd520 con 0x7effa40cd020 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff88003520 con 0x7effa40cd020 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2373794626 0 0) 0x7eff88003520 con 0x7effa40cd020 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 >> v2:172.21.15.76:3300/0 conn(0x7effa41454e0 msgr2=0x7effa4145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 --2- 172.21.15.59:0/3211174442 >> v2:172.21.15.76:3300/0 conn(0x7effa41454e0 0x7effa4145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 >> v1:172.21.15.76:6789/0 conn(0x7effa4160910 legacy=0x7effa40d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa40cdae0 con 0x7effa40cd020 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.454+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7eff8c001670 con 0x7effa40cd020 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 >> v1:172.21.15.59:6789/0 conn(0x7effa40cd020 legacy=0x7effa40b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa2a00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff88003a00 0x7eff88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa2a00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7eff88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa2a00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff880052e0 0x7eff8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7effa40cddc0 con 0x7eff88004800 2024-04-23T14:36:30.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff8800c200 con 0x7eff88003a00 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7eff8800c400 con 0x7eff880052e0 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7eff9be00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7eff88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa3400640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff88003a00 0x7eff88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa3e00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff880052e0 0x7eff8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7eff9be00640 1 -- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff880052e0 msgr2=0x7eff8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7eff9be00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff880052e0 0x7eff8800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7eff9be00640 1 -- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff88003a00 msgr2=0x7eff88003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7eff9be00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff88003a00 0x7eff88003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7eff9be00640 1 -- 172.21.15.59:0/3211174442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff88003520 con 0x7eff88004800 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa3e00640 1 -- 172.21.15.59:0/3211174442 reap_dead start 2024-04-23T14:36:30.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.455+0000 7effa3400640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff88003a00 0x7eff88003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:30.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.456+0000 7eff9be00640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7eff88004c80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c0e0 tx=0x7eff88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e974b9ea36e43d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:30.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.457+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c007c20 con 0x7eff88004800 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.457+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8c005af0 con 0x7eff88004800 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.457+0000 7effa2a00640 1 -- 172.21.15.59:0/3211174442 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c007070 con 0x7eff88004800 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.457+0000 7effaa600640 1 -- 172.21.15.59:0/3211174442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 msgr2=0x7eff88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.457+0000 7effaa600640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7eff88004c80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c0e0 tx=0x7eff88005830 comp rx=0 tx=0).stop 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.457+0000 7effaa600640 1 -- 172.21.15.59:0/3211174442 shutdown_connections 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.457+0000 7effaa600640 1 --2- 172.21.15.59:0/3211174442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7eff88004c80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.458+0000 7effaa600640 1 -- 172.21.15.59:0/3211174442 >> 172.21.15.59:0/3211174442 conn(0x7effa4067ea0 msgr2=0x7effa406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.458+0000 7effaa600640 1 -- 172.21.15.59:0/3211174442 shutdown_connections 2024-04-23T14:36:30.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.458+0000 7effaa600640 1 -- 172.21.15.59:0/3211174442 wait complete. 2024-04-23T14:36:30.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.459+0000 7effaa600640 1 Processor -- start 2024-04-23T14:36:30.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.460+0000 7effaa600640 1 -- start start 2024-04-23T14:36:30.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effaa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7effa4141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effaa600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effa413f190 0x7effa413d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effaa600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa40aeb30 0x7effa414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effaa600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7effa4160f00 con 0x7eff88004800 2024-04-23T14:36:30.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effaa600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7effa4160d80 con 0x7effa40aeb30 2024-04-23T14:36:30.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effaa600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7effa4161080 con 0x7effa413f190 2024-04-23T14:36:30.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7effa4141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7effa4141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41360/0 (socket says 172.21.15.59:41360) 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 -- 172.21.15.59:0/3760695866 learned_addr learned my addr 172.21.15.59:0/3760695866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7eff9be00640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effa413f190 0x7effa413d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 -- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effa413f190 msgr2=0x7effa413d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3e00640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa40aeb30 0x7effa414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effa413f190 0x7effa413d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 -- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa40aeb30 msgr2=0x7effa414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa40aeb30 0x7effa414e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.461+0000 7effa3400640 1 -- 172.21.15.59:0/3760695866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa413f5f0 con 0x7eff88004800 2024-04-23T14:36:30.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.462+0000 7effa3400640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7effa4141ac0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7eff9400e590 tx=0x7eff9400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:30.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.462+0000 7eff9b400640 1 -- 172.21.15.59:0/3760695866 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94017070 con 0x7eff88004800 2024-04-23T14:36:30.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.462+0000 7eff9b400640 1 -- 172.21.15.59:0/3760695866 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff94013610 con 0x7eff88004800 2024-04-23T14:36:30.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.462+0000 7eff9b400640 1 -- 172.21.15.59:0/3760695866 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9400f040 con 0x7eff88004800 2024-04-23T14:36:30.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.462+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa413e110 con 0x7eff88004800 2024-04-23T14:36:30.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.462+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa413e6d0 con 0x7eff88004800 2024-04-23T14:36:30.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.463+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa40692b0 con 0x7eff88004800 2024-04-23T14:36:30.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.465+0000 7eff9b400640 1 -- 172.21.15.59:0/3760695866 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff9401a070 con 0x7eff88004800 2024-04-23T14:36:30.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.466+0000 7eff9b400640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7eff8006c8f0 0x7eff8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.466+0000 7eff9b400640 1 -- 172.21.15.59:0/3760695866 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7eff9408b6c0 con 0x7eff88004800 2024-04-23T14:36:30.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.466+0000 7eff9be00640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7eff8006c8f0 0x7eff8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.467+0000 7eff9be00640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7eff8006c8f0 0x7eff8006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff8c032000 tx=0x7eff8c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:30.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.470+0000 7eff9b400640 1 -- 172.21.15.59:0/3760695866 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effa40692b0 con 0x7eff88004800 2024-04-23T14:36:30.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.678+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effa4160030 con 0x7eff88004800 2024-04-23T14:36:30.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.681+0000 7eff9b400640 1 -- 172.21.15.59:0/3760695866 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16621 (secure 0 0 0) 0x7eff94059a50 con 0x7eff88004800 2024-04-23T14:36:30.682 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:30.683 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":68,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:29.710872+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7eff8006c8f0 msgr2=0x7eff8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7eff8006c8f0 0x7eff8006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff8c032000 tx=0x7eff8c009030 comp rx=0 tx=0).stop 2024-04-23T14:36:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 msgr2=0x7effa4141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7effa4141ac0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7eff9400e590 tx=0x7eff9400ea50 comp rx=0 tx=0).stop 2024-04-23T14:36:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 shutdown_connections 2024-04-23T14:36:30.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7eff8006c8f0 0x7eff8006edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7effa413f190 0x7effa413d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effa40aeb30 0x7effa414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 --2- 172.21.15.59:0/3760695866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff88004800 0x7effa4141ac0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 >> 172.21.15.59:0/3760695866 conn(0x7effa4067ea0 msgr2=0x7effa40ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:30.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 shutdown_connections 2024-04-23T14:36:30.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.684+0000 7effaa600640 1 -- 172.21.15.59:0/3760695866 wait complete. 2024-04-23T14:36:30.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:30.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.911+0000 7faeb6a00640 1 Processor -- start 2024-04-23T14:36:30.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.911+0000 7faeb6a00640 1 -- start start 2024-04-23T14:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.913+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 0x7faeb00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.913+0000 7faeb6a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeb0160210 con 0x7faeb015c3d0 2024-04-23T14:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.913+0000 7faeb6a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeb01604f0 con 0x7faeb00b4690 2024-04-23T14:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.913+0000 7faeb6a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faeb01607d0 con 0x7faeb00cd760 2024-04-23T14:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.913+0000 7faeb5600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faeb015c3d0 0x7faeb00cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35146/0 (socket says 172.21.15.59:35146) 2024-04-23T14:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.913+0000 7faeb5600640 1 -- 172.21.15.59:0/3279578996 learned_addr learned my addr 172.21.15.59:0/3279578996 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeb4c00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 0x7faeb00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 822986064 0 0) 0x7faeb0160210 con 0x7faeb015c3d0 2024-04-23T14:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae98003520 con 0x7faeb015c3d0 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3122376959 0 0) 0x7fae98003520 con 0x7faeb015c3d0 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 msgr2=0x7faeb00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 0x7faeb00b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 >> v1:172.21.15.76:6789/0 conn(0x7faeb00b4690 legacy=0x7faeb00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb0160ab0 con 0x7faeb015c3d0 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7faea4001670 con 0x7faeb015c3d0 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 >> v1:172.21.15.59:6789/0 conn(0x7faeb015c3d0 legacy=0x7faeb00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae980039e0 0x7fae98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7fae98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 0x7fae980051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faeb01607d0 con 0x7fae98004800 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae9800bb50 con 0x7fae980039e0 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.914+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae9800bd50 con 0x7faeb00cd760 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb4c00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7fae98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb5600640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae980039e0 0x7fae98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb6000640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 0x7fae980051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb4c00640 1 -- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 msgr2=0x7fae980051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb4c00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb00cd760 0x7fae980051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb4c00640 1 -- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae980039e0 msgr2=0x7fae98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb4c00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae980039e0 0x7fae98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb4c00640 1 -- 172.21.15.59:0/3279578996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae98003520 con 0x7fae98004800 2024-04-23T14:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.915+0000 7faeb5600640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae980039e0 0x7fae98003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.916+0000 7faeb4c00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7fae98004c80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00ba30 tx=0x7fae98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5277e0a12ed5a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c030070 con 0x7fae98004800 2024-04-23T14:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae9c02ade0 con 0x7fae98004800 2024-04-23T14:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeafe00640 1 -- 172.21.15.59:0/3279578996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00b600 con 0x7fae98004800 2024-04-23T14:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeb6a00640 1 -- 172.21.15.59:0/3279578996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 msgr2=0x7fae98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeb6a00640 1 --2- 172.21.15.59:0/3279578996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7fae98004c80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00ba30 tx=0x7fae98005830 comp rx=0 tx=0).stop 2024-04-23T14:36:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeb6000640 1 -- 172.21.15.59:0/3279578996 reap_dead start 2024-04-23T14:36:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeb6a00640 1 -- 172.21.15.59:0/3279578996 shutdown_connections 2024-04-23T14:36:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeb6a00640 1 -- 172.21.15.59:0/3279578996 >> 172.21.15.59:0/3279578996 conn(0x7faeb00b21a0 msgr2=0x7faeb00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeb6a00640 1 -- 172.21.15.59:0/3279578996 shutdown_connections 2024-04-23T14:36:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.917+0000 7faeb6a00640 1 -- 172.21.15.59:0/3279578996 wait complete. 2024-04-23T14:36:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.919+0000 7faeb6a00640 1 Processor -- start 2024-04-23T14:36:30.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.919+0000 7faeb6a00640 1 -- start start 2024-04-23T14:36:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.920+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7faeb015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.920+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb015cab0 0x7faeb00c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.920+0000 7faeb6a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb015aa10 0x7faeb015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.920+0000 7faeb6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faeb015c7b0 con 0x7fae98004800 2024-04-23T14:36:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.920+0000 7faeb6a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faeb00b4ac0 con 0x7faeb015cab0 2024-04-23T14:36:30.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.920+0000 7faeb6a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faeb015c930 con 0x7faeb015aa10 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7faeb015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7faeb015dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41372/0 (socket says 172.21.15.59:41372) 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 -- 172.21.15.59:0/2394775520 learned_addr learned my addr 172.21.15.59:0/2394775520 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb4c00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb015cab0 0x7faeb00c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 -- 172.21.15.59:0/2394775520 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb015aa10 msgr2=0x7faeb015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb015aa10 0x7faeb015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 -- 172.21.15.59:0/2394775520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb015cab0 msgr2=0x7faeb00c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb015cab0 0x7faeb00c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 -- 172.21.15.59:0/2394775520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb00ae330 con 0x7fae98004800 2024-04-23T14:36:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.921+0000 7faeb5600640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7faeb015dff0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7faea400d8e0 tx=0x7faea400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.922+0000 7faead600640 1 -- 172.21.15.59:0/2394775520 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4019070 con 0x7fae98004800 2024-04-23T14:36:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.922+0000 7faead600640 1 -- 172.21.15.59:0/2394775520 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea40074e0 con 0x7fae98004800 2024-04-23T14:36:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.922+0000 7faead600640 1 -- 172.21.15.59:0/2394775520 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4004a00 con 0x7fae98004800 2024-04-23T14:36:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.922+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb00ae610 con 0x7fae98004800 2024-04-23T14:36:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.922+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb00aebd0 con 0x7fae98004800 2024-04-23T14:36:30.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.923+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb004fd90 con 0x7fae98004800 2024-04-23T14:36:30.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.924+0000 7faead600640 1 -- 172.21.15.59:0/2394775520 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faea4004ba0 con 0x7fae98004800 2024-04-23T14:36:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.925+0000 7faead600640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae9006c920 0x7fae9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.925+0000 7faead600640 1 -- 172.21.15.59:0/2394775520 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7faea408b630 con 0x7fae98004800 2024-04-23T14:36:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.925+0000 7faeb4c00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae9006c920 0x7fae9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.926+0000 7faeb4c00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae9006c920 0x7fae9006ede0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00ba00 tx=0x7fae9c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:30.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:30.929+0000 7faead600640 1 -- 172.21.15.59:0/2394775520 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faea4059a50 con 0x7fae98004800 2024-04-23T14:36:31.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.134+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faeb00b3110 con 0x7fae98004800 2024-04-23T14:36:31.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.138+0000 7faead600640 1 -- 172.21.15.59:0/2394775520 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16621 (secure 0 0 0) 0x7faea4059350 con 0x7fae98004800 2024-04-23T14:36:31.139 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:31.139 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":68,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:29.710872+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.140+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae9006c920 msgr2=0x7fae9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.140+0000 7faeb6a00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae9006c920 0x7fae9006ede0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00ba00 tx=0x7fae9c030000 comp rx=0 tx=0).stop 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.140+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 msgr2=0x7faeb015dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.140+0000 7faeb6a00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7faeb015dff0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7faea400d8e0 tx=0x7faea400dda0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 shutdown_connections 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae9006c920 0x7fae9006ede0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faeb015aa10 0x7faeb015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb015cab0 0x7faeb00c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 --2- 172.21.15.59:0/2394775520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae98004800 0x7faeb015dff0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 >> 172.21.15.59:0/2394775520 conn(0x7faeb00b21a0 msgr2=0x7faeb00c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 shutdown_connections 2024-04-23T14:36:31.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.141+0000 7faeb6a00640 1 -- 172.21.15.59:0/2394775520 wait complete. 2024-04-23T14:36:31.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:36:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.361+0000 7fbf1fe00640 1 Processor -- start 2024-04-23T14:36:31.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.361+0000 7fbf1fe00640 1 -- start start 2024-04-23T14:36:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20072760 0x7fbf200b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf20072f50 0x7fbf200b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2006e510 con 0x7fbf20147650 2024-04-23T14:36:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbf20147a30 con 0x7fbf20072f50 2024-04-23T14:36:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbf200b36b0 con 0x7fbf20072760 2024-04-23T14:36:31.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf20072f50 0x7fbf200b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbf20147650 0x7fbf20072050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35150/0 (socket says 172.21.15.59:35150) 2024-04-23T14:36:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf20072f50 0x7fbf200b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41374/0 (socket says 172.21.15.59:41374) 2024-04-23T14:36:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.363+0000 7fbf1ea00640 1 -- 172.21.15.59:0/3374296004 learned_addr learned my addr 172.21.15.59:0/3374296004 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.364+0000 7fbf1e000640 1 --2- 172.21.15.59:0/3374296004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20072760 0x7fbf200b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.364+0000 7fbf1f400640 1 -- 172.21.15.59:0/3374296004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20072760 msgr2=0x7fbf200b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.364+0000 7fbf1f400640 1 --2- 172.21.15.59:0/3374296004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20072760 0x7fbf200b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.364+0000 7fbf1f400640 1 -- 172.21.15.59:0/3374296004 >> v1:172.21.15.59:6789/0 conn(0x7fbf20147650 legacy=0x7fbf20072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:31.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.364+0000 7fbf1f400640 1 -- 172.21.15.59:0/3374296004 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf200b3990 con 0x7fbf20072f50 2024-04-23T14:36:31.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.364+0000 7fbf1d600640 1 -- 172.21.15.59:0/3374296004 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430508251 0 0) 0x7fbf10001110 con 0x7fbf20147650 2024-04-23T14:36:31.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.365+0000 7fbf1f400640 1 --2- 172.21.15.59:0/3374296004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf200b4c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009d90 tx=0x7fbf1802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f08238002872142 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:31.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1d600640 1 -- 172.21.15.59:0/3374296004 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18031600 con 0x7fbf20072f50 2024-04-23T14:36:31.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1d600640 1 -- 172.21.15.59:0/3374296004 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf180317a0 con 0x7fbf20072f50 2024-04-23T14:36:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1d600640 1 -- 172.21.15.59:0/3374296004 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18031b70 con 0x7fbf20072f50 2024-04-23T14:36:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1fe00640 1 -- 172.21.15.59:0/3374296004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 msgr2=0x7fbf200b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:31.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/3374296004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf200b4c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009d90 tx=0x7fbf1802a5d0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1fe00640 1 -- 172.21.15.59:0/3374296004 shutdown_connections 2024-04-23T14:36:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/3374296004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf200b4c90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/3374296004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20072760 0x7fbf200b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.366+0000 7fbf1fe00640 1 -- 172.21.15.59:0/3374296004 >> 172.21.15.59:0/3374296004 conn(0x7fbf200b2310 msgr2=0x7fbf200b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.367+0000 7fbf1fe00640 1 -- 172.21.15.59:0/3374296004 shutdown_connections 2024-04-23T14:36:31.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.367+0000 7fbf1fe00640 1 -- 172.21.15.59:0/3374296004 wait complete. 2024-04-23T14:36:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.369+0000 7fbf1fe00640 1 Processor -- start 2024-04-23T14:36:31.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.369+0000 7fbf1fe00640 1 -- start start 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf20072760 0x7fbf20161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf20143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20147650 0x7fbf2006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200b4230 con 0x7fbf20072f50 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200b3f50 con 0x7fbf20072760 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbf20148390 con 0x7fbf20147650 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf20143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf20143cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41376/0 (socket says 172.21.15.59:41376) 2024-04-23T14:36:31.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.370+0000 7fbf1e000640 1 -- 172.21.15.59:0/4235212674 learned_addr learned my addr 172.21.15.59:0/4235212674 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:31.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1e000640 1 -- 172.21.15.59:0/4235212674 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20147650 msgr2=0x7fbf2006f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1e000640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20147650 0x7fbf2006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1ea00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf20072760 0x7fbf20161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1e000640 1 -- 172.21.15.59:0/4235212674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf20072760 msgr2=0x7fbf20161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1e000640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf20072760 0x7fbf20161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1e000640 1 -- 172.21.15.59:0/4235212674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf20161950 con 0x7fbf20072f50 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1e000640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf20143cd0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00d5f0 tx=0x7fbf0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf17400640 1 -- 172.21.15.59:0/4235212674 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0125f0 con 0x7fbf20072f50 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf17400640 1 -- 172.21.15.59:0/4235212674 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c019070 con 0x7fbf20072f50 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf17400640 1 -- 172.21.15.59:0/4235212674 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0129d0 con 0x7fbf20072f50 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.371+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf20161c30 con 0x7fbf20072f50 2024-04-23T14:36:31.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.372+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2013dd10 con 0x7fbf20072f50 2024-04-23T14:36:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.373+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2004fd90 con 0x7fbf20072f50 2024-04-23T14:36:31.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.373+0000 7fbf17400640 1 -- 172.21.15.59:0/4235212674 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf0c012b70 con 0x7fbf20072f50 2024-04-23T14:36:31.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.374+0000 7fbf17400640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf0006c8f0 0x7fbf0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:31.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.375+0000 7fbf17400640 1 -- 172.21.15.59:0/4235212674 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fbf0c08bfe0 con 0x7fbf20072f50 2024-04-23T14:36:31.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.375+0000 7fbf1ea00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf0006c8f0 0x7fbf0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:31.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.375+0000 7fbf1ea00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf0006c8f0 0x7fbf0006edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000f60 tx=0x7fbf10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:31.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.379+0000 7fbf17400640 1 -- 172.21.15.59:0/4235212674 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf0c05a3d0 con 0x7fbf20072f50 2024-04-23T14:36:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:31.623+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbf20147c60 con 0x7fbf20072f50 2024-04-23T14:36:32.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.266+0000 7fbf17400640 1 -- 172.21.15.59:0/4235212674 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7fbf0c059cd0 con 0x7fbf20072f50 2024-04-23T14:36:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf0006c8f0 msgr2=0x7fbf0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf0006c8f0 0x7fbf0006edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbf10000f60 tx=0x7fbf10009040 comp rx=0 tx=0).stop 2024-04-23T14:36:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 msgr2=0x7fbf20143cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:32.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf20143cd0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00d5f0 tx=0x7fbf0c00dab0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 shutdown_connections 2024-04-23T14:36:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf0006c8f0 0x7fbf0006edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf20147650 0x7fbf2006f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf20072f50 0x7fbf20143cd0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 --2- 172.21.15.59:0/4235212674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf20072760 0x7fbf20161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 >> 172.21.15.59:0/4235212674 conn(0x7fbf200b2310 msgr2=0x7fbf20145e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.269+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 shutdown_connections 2024-04-23T14:36:32.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.270+0000 7fbf1fe00640 1 -- 172.21.15.59:0/4235212674 wait complete. 2024-04-23T14:36:32.295 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:36:32.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:36:32.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.502+0000 7fa9c3e00640 1 Processor -- start 2024-04-23T14:36:32.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.502+0000 7fa9c3e00640 1 -- start start 2024-04-23T14:36:32.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.504+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c40ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:32.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.504+0000 7fa9c3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9c40d09e0 0x7fa9c406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:32.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.504+0000 7fa9c3e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9c4071e10 con 0x7fa9c40d0600 2024-04-23T14:36:32.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.504+0000 7fa9c3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa9c4072a20 con 0x7fa9c40d09e0 2024-04-23T14:36:32.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.504+0000 7fa9c3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c40720f0 con 0x7fa9c406dcb0 2024-04-23T14:36:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.504+0000 7fa9c3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9c40d09e0 0x7fa9c406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:32.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.504+0000 7fa9c3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9c40d09e0 0x7fa9c406e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41378/0 (socket says 172.21.15.59:41378) 2024-04-23T14:36:32.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.505+0000 7fa9c3400640 1 -- 172.21.15.59:0/151566536 learned_addr learned my addr 172.21.15.59:0/151566536 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:32.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.505+0000 7fa9bbe00640 1 --2- 172.21.15.59:0/151566536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c40ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:32.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.505+0000 7fa9c3400640 1 -- 172.21.15.59:0/151566536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 msgr2=0x7fa9c40ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:32.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.505+0000 7fa9c3400640 1 --2- 172.21.15.59:0/151566536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c40ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.505+0000 7fa9c3400640 1 -- 172.21.15.59:0/151566536 >> v1:172.21.15.76:6789/0 conn(0x7fa9c40d0600 legacy=0x7fa9c4162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:32.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.505+0000 7fa9c3400640 1 -- 172.21.15.59:0/151566536 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c40723d0 con 0x7fa9c40d09e0 2024-04-23T14:36:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.506+0000 7fa9c3400640 1 --2- 172.21.15.59:0/151566536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d09e0 0x7fa9c406e5c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009e50 tx=0x7fa9bc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e47b999d4b7d201 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c2000640 1 -- 172.21.15.59:0/151566536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc036070 con 0x7fa9c40d09e0 2024-04-23T14:36:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c2000640 1 -- 172.21.15.59:0/151566536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9bc031600 con 0x7fa9c40d09e0 2024-04-23T14:36:32.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c2000640 1 -- 172.21.15.59:0/151566536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0319d0 con 0x7fa9c40d09e0 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 -- 172.21.15.59:0/151566536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d09e0 msgr2=0x7fa9c406e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/151566536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d09e0 0x7fa9c406e5c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009e50 tx=0x7fa9bc02a6c0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 -- 172.21.15.59:0/151566536 shutdown_connections 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/151566536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d09e0 0x7fa9c406e5c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/151566536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c40ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 -- 172.21.15.59:0/151566536 >> 172.21.15.59:0/151566536 conn(0x7fa9c406bdf0 msgr2=0x7fa9c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 -- 172.21.15.59:0/151566536 shutdown_connections 2024-04-23T14:36:32.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.507+0000 7fa9c3e00640 1 -- 172.21.15.59:0/151566536 wait complete. 2024-04-23T14:36:32.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.509+0000 7fa9c3e00640 1 Processor -- start 2024-04-23T14:36:32.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.509+0000 7fa9c3e00640 1 -- start start 2024-04-23T14:36:32.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.510+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c41494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.510+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d0600 0x7fa9c40afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.510+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c40d09e0 0x7fa9c40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.510+0000 7fa9c3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c40d0e70 con 0x7fa9c40d0600 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.510+0000 7fa9c3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c415eaf0 con 0x7fa9c40d09e0 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.510+0000 7fa9c3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c406eb00 con 0x7fa9c406dcb0 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.510+0000 7fa9bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d0600 0x7fa9c40afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d0600 0x7fa9c40afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41386/0 (socket says 172.21.15.59:41386) 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 -- 172.21.15.59:0/1052779158 learned_addr learned my addr 172.21.15.59:0/1052779158 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9c3400640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c40d09e0 0x7fa9c40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9c2a00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c41494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 -- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 msgr2=0x7fa9c41494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c41494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 -- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c40d09e0 msgr2=0x7fa9c40b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c40d09e0 0x7fa9c40b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 -- 172.21.15.59:0/1052779158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c40b3c20 con 0x7fa9c40d0600 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.511+0000 7fa9bbe00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d0600 0x7fa9c40afdc0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00d830 tx=0x7fa9ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.512+0000 7fa9bb400640 1 -- 172.21.15.59:0/1052779158 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00b6c0 con 0x7fa9c40d0600 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.512+0000 7fa9bb400640 1 -- 172.21.15.59:0/1052779158 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ac019070 con 0x7fa9c40d0600 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.512+0000 7fa9bb400640 1 -- 172.21.15.59:0/1052779158 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00ba80 con 0x7fa9c40d0600 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.512+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c40b0930 con 0x7fa9c40d0600 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.512+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c40b0ef0 con 0x7fa9c40d0600 2024-04-23T14:36:32.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.513+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c404fd90 con 0x7fa9c40d0600 2024-04-23T14:36:32.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.514+0000 7fa9bb400640 1 -- 172.21.15.59:0/1052779158 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9ac01e460 con 0x7fa9c40d0600 2024-04-23T14:36:32.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.516+0000 7fa9bb400640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9a006c8f0 0x7fa9a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:32.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.516+0000 7fa9bb400640 1 -- 172.21.15.59:0/1052779158 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fa9ac08c240 con 0x7fa9c40d0600 2024-04-23T14:36:32.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.516+0000 7fa9c2a00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9a006c8f0 0x7fa9a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:32.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.517+0000 7fa9c2a00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9a006c8f0 0x7fa9a006edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4003d90 tx=0x7fa9b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:32.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.520+0000 7fa9bb400640 1 -- 172.21.15.59:0/1052779158 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9ac05a660 con 0x7fa9c40d0600 2024-04-23T14:36:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:32.767+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa9c40af880 con 0x7fa9c40d0600 2024-04-23T14:36:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.276+0000 7fa9bb400640 1 -- 172.21.15.59:0/1052779158 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fa9ac059f60 con 0x7fa9c40d0600 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.279+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9a006c8f0 msgr2=0x7fa9a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.279+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9a006c8f0 0x7fa9a006edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4003d90 tx=0x7fa9b4009040 comp rx=0 tx=0).stop 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d0600 msgr2=0x7fa9c40afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d0600 0x7fa9c40afdc0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00d830 tx=0x7fa9ac00dcf0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 shutdown_connections 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9a006c8f0 0x7fa9a006edb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9c40d09e0 0x7fa9c40b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9c40d0600 0x7fa9c40afdc0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 --2- 172.21.15.59:0/1052779158 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9c406dcb0 0x7fa9c41494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 >> 172.21.15.59:0/1052779158 conn(0x7fa9c406bdf0 msgr2=0x7fa9c40cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:33.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 shutdown_connections 2024-04-23T14:36:33.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.280+0000 7fa9c3e00640 1 -- 172.21.15.59:0/1052779158 wait complete. 2024-04-23T14:36:33.303 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:36:33.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:33.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.511+0000 7f6159800640 1 Processor -- start 2024-04-23T14:36:33.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.512+0000 7f6159800640 1 -- start start 2024-04-23T14:36:33.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.513+0000 7f6159800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f615406d440 0x7f615406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.513+0000 7f6159800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540cee50 0x7f61540cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.513+0000 7f6159800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f615406cc60 con 0x7f615415e890 2024-04-23T14:36:33.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.513+0000 7f6159800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f615406cfb0 con 0x7f615406d440 2024-04-23T14:36:33.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.513+0000 7f6159800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6154162bd0 con 0x7f61540cee50 2024-04-23T14:36:33.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.513+0000 7f6152a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f615415e890 0x7f61541624c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35162/0 (socket says 172.21.15.59:35162) 2024-04-23T14:36:33.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.513+0000 7f6152a00640 1 -- 172.21.15.59:0/334147215 learned_addr learned my addr 172.21.15.59:0/334147215 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829877731 0 0) 0x7f615406cc60 con 0x7f615415e890 2024-04-23T14:36:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6134003520 con 0x7f615415e890 2024-04-23T14:36:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6152000640 1 --2- 172.21.15.59:0/334147215 >> v2:172.21.15.76:3300/0 conn(0x7f615406d440 0x7f615406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6153400640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540cee50 0x7f61540cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 160459914 0 0) 0x7f6134003520 con 0x7f615415e890 2024-04-23T14:36:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540cee50 msgr2=0x7f61540cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:33.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540cee50 0x7f61540cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 >> v2:172.21.15.76:3300/0 conn(0x7f615406d440 msgr2=0x7f615406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 --2- 172.21.15.59:0/334147215 >> v2:172.21.15.76:3300/0 conn(0x7f615406d440 0x7f615406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6154162ea0 con 0x7f615415e890 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f6144001670 con 0x7f615415e890 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 >> v1:172.21.15.59:6789/0 conn(0x7f615415e890 legacy=0x7f61541624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540cee50 0x7f61340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6134004030 0x7f6134004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004d40 0x7f61340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f615406cfb0 con 0x7f6134004030 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6154162bd0 con 0x7f6134004d40 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.514+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f613400bbd0 con 0x7f61540cee50 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.515+0000 7f6152000640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6134004030 0x7f6134004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.515+0000 7f6153400640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004d40 0x7f61340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.515+0000 7f6152000640 1 -- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540cee50 msgr2=0x7f61340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:33.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.515+0000 7f6152000640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540cee50 0x7f61340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.515+0000 7f6152000640 1 -- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004d40 msgr2=0x7f61340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:33.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.515+0000 7f6152000640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004d40 0x7f61340051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.515+0000 7f6152000640 1 -- 172.21.15.59:0/334147215 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6134003520 con 0x7f6134004030 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.516+0000 7f6152000640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6134004030 0x7f6134004800 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f613c007360 tx=0x7f6134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223da97173887415 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.516+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c009be0 con 0x7f6134004030 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.516+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c02e620 con 0x7f6134004030 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.516+0000 7f6151600640 1 -- 172.21.15.59:0/334147215 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c02e9f0 con 0x7f6134004030 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.517+0000 7f6159800640 1 -- 172.21.15.59:0/334147215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6134004030 msgr2=0x7f6134004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.517+0000 7f6159800640 1 --2- 172.21.15.59:0/334147215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6134004030 0x7f6134004800 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f613c007360 tx=0x7f6134005830 comp rx=0 tx=0).stop 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.517+0000 7f6153400640 1 -- 172.21.15.59:0/334147215 reap_dead start 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.517+0000 7f6159800640 1 -- 172.21.15.59:0/334147215 shutdown_connections 2024-04-23T14:36:33.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.517+0000 7f6159800640 1 -- 172.21.15.59:0/334147215 >> 172.21.15.59:0/334147215 conn(0x7f61540d25d0 msgr2=0x7f61540d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:33.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.517+0000 7f6159800640 1 -- 172.21.15.59:0/334147215 shutdown_connections 2024-04-23T14:36:33.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.517+0000 7f6159800640 1 -- 172.21.15.59:0/334147215 wait complete. 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.519+0000 7f6159800640 1 Processor -- start 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.519+0000 7f6159800640 1 -- start start 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6159800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004030 0x7f61540b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6159800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61540c7370 0x7f61540d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6159800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540d1100 0x7f61541f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6159800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61540b3c20 con 0x7f61540c7370 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6159800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f615415ec70 con 0x7f6134004030 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6159800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f61540b3da0 con 0x7f61540d1100 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61540c7370 0x7f61540d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61540c7370 0x7f61540d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41406/0 (socket says 172.21.15.59:41406) 2024-04-23T14:36:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 -- 172.21.15.59:0/3782600866 learned_addr learned my addr 172.21.15.59:0/3782600866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152a00640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004030 0x7f61540b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6153400640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540d1100 0x7f61541f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 -- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540d1100 msgr2=0x7f61541f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540d1100 0x7f61541f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 -- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004030 msgr2=0x7f61540b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004030 0x7f61540b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152000640 1 -- 172.21.15.59:0/3782600866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61540afbb0 con 0x7f61540c7370 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.520+0000 7f6152a00640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004030 0x7f61540b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.521+0000 7f6152000640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61540c7370 0x7f61540d0bc0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f613c036630 tx=0x7f613c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.521+0000 7f614aa00640 1 -- 172.21.15.59:0/3782600866 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c045070 con 0x7f61540c7370 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.521+0000 7f614aa00640 1 -- 172.21.15.59:0/3782600866 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c036a20 con 0x7f61540c7370 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.521+0000 7f614aa00640 1 -- 172.21.15.59:0/3782600866 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c009be0 con 0x7f61540c7370 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.521+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61540afe90 con 0x7f61540c7370 2024-04-23T14:36:33.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.521+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61540b5710 con 0x7f61540c7370 2024-04-23T14:36:33.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.522+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6154069190 con 0x7f61540c7370 2024-04-23T14:36:33.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.523+0000 7f614aa00640 1 -- 172.21.15.59:0/3782600866 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f613c048070 con 0x7f61540c7370 2024-04-23T14:36:33.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.524+0000 7f614aa00640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f613006c8f0 0x7f613006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:33.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.524+0000 7f614aa00640 1 -- 172.21.15.59:0/3782600866 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f613c0ac510 con 0x7f61540c7370 2024-04-23T14:36:33.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.525+0000 7f6152a00640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f613006c8f0 0x7f613006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:33.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.525+0000 7f6152a00640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f613006c8f0 0x7f613006edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6144001860 tx=0x7f6144009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:33.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.529+0000 7f614aa00640 1 -- 172.21.15.59:0/3782600866 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f613c03d070 con 0x7f61540c7370 2024-04-23T14:36:33.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.773+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61540d14e0 con 0x7f61540c7370 2024-04-23T14:36:33.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.774+0000 7f614aa00640 1 -- 172.21.15.59:0/3782600866 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5183 (secure 0 0 0) 0x7f613c032030 con 0x7f61540c7370 2024-04-23T14:36:33.776 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:33.776 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:36:32.769281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6735":{"gid":6735,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6836/2883460093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2883460093},{"type":"v1","addr":"172.21.15.59:6836","nonce":2883460093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6735,"qdb_cluster":[6735]},"id":5}]} 2024-04-23T14:36:33.776 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 43 2024-04-23T14:36:33.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f613006c8f0 msgr2=0x7f613006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f613006c8f0 0x7f613006edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6144001860 tx=0x7f6144009040 comp rx=0 tx=0).stop 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61540c7370 msgr2=0x7f61540d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61540c7370 0x7f61540d0bc0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f613c036630 tx=0x7f613c004ab0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 shutdown_connections 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f613006c8f0 0x7f613006edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61540d1100 0x7f61541f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61540c7370 0x7f61540d0bc0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 --2- 172.21.15.59:0/3782600866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6134004030 0x7f61540b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 >> 172.21.15.59:0/3782600866 conn(0x7f61540d25d0 msgr2=0x7f6154070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 shutdown_connections 2024-04-23T14:36:33.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:33.777+0000 7f6159800640 1 -- 172.21.15.59:0/3782600866 wait complete. 2024-04-23T14:36:33.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.006+0000 7fc3a9c00640 1 Processor -- start 2024-04-23T14:36:34.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.006+0000 7fc3a9c00640 1 -- start start 2024-04-23T14:36:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.007+0000 7fc3a9c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3a40cd050 0x7fc3a406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.007+0000 7fc3a9c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3a415ece0 0x7fc3a40cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a9c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a40cd970 con 0x7fc3a40b3080 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a9c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a40ced20 con 0x7fc3a415ece0 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a9c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a415fcd0 con 0x7fc3a40cd050 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3a415ece0 0x7fc3a40cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3a415ece0 0x7fc3a40cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41414/0 (socket says 172.21.15.59:41414) 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a3400640 1 -- 172.21.15.59:0/330607908 learned_addr learned my addr 172.21.15.59:0/330607908 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a3e00640 1 --2- 172.21.15.59:0/330607908 >> v2:172.21.15.76:3300/0 conn(0x7fc3a40cd050 0x7fc3a406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a3400640 1 -- 172.21.15.59:0/330607908 >> v2:172.21.15.76:3300/0 conn(0x7fc3a40cd050 msgr2=0x7fc3a406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a3400640 1 --2- 172.21.15.59:0/330607908 >> v2:172.21.15.76:3300/0 conn(0x7fc3a40cd050 0x7fc3a406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.008+0000 7fc3a3400640 1 -- 172.21.15.59:0/330607908 >> v1:172.21.15.76:6789/0 conn(0x7fc3a40b3080 legacy=0x7fc3a40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.009+0000 7fc3a3400640 1 -- 172.21.15.59:0/330607908 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a415f160 con 0x7fc3a415ece0 2024-04-23T14:36:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.010+0000 7fc3a3400640 1 --2- 172.21.15.59:0/330607908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a415ece0 0x7fc3a40cd430 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc3a406e080 tx=0x7fc39002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3972b8ac4faa6597 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.010+0000 7fc3a2a00640 1 -- 172.21.15.59:0/330607908 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390037070 con 0x7fc3a415ece0 2024-04-23T14:36:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.010+0000 7fc3a2a00640 1 -- 172.21.15.59:0/330607908 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc390032440 con 0x7fc3a415ece0 2024-04-23T14:36:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.010+0000 7fc3a2a00640 1 -- 172.21.15.59:0/330607908 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390032810 con 0x7fc3a415ece0 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 -- 172.21.15.59:0/330607908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a415ece0 msgr2=0x7fc3a40cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/330607908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a415ece0 0x7fc3a40cd430 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc3a406e080 tx=0x7fc39002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 -- 172.21.15.59:0/330607908 shutdown_connections 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/330607908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a415ece0 0x7fc3a40cd430 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/330607908 >> v2:172.21.15.76:3300/0 conn(0x7fc3a40cd050 0x7fc3a406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 -- 172.21.15.59:0/330607908 >> 172.21.15.59:0/330607908 conn(0x7fc3a40b21a0 msgr2=0x7fc3a40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 -- 172.21.15.59:0/330607908 shutdown_connections 2024-04-23T14:36:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.011+0000 7fc3a9c00640 1 -- 172.21.15.59:0/330607908 wait complete. 2024-04-23T14:36:34.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.013+0000 7fc3a9c00640 1 Processor -- start 2024-04-23T14:36:34.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.013+0000 7fc3a9c00640 1 -- start start 2024-04-23T14:36:34.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a40b3080 0x7fc3a4072da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a40cd050 0x7fc3a40b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a415ece0 0x7fc3a409d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a415f9f0 con 0x7fc3a40cd050 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a415f710 con 0x7fc3a40b3080 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a40cef50 con 0x7fc3a415ece0 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a40cd050 0x7fc3a40b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a40cd050 0x7fc3a40b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41418/0 (socket says 172.21.15.59:41418) 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 -- 172.21.15.59:0/94742693 learned_addr learned my addr 172.21.15.59:0/94742693 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3e00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a40b3080 0x7fc3a4072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 -- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a415ece0 msgr2=0x7fc3a409d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9200640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a415ece0 0x7fc3a409d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a415ece0 0x7fc3a409d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 -- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a40b3080 msgr2=0x7fc3a4072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a40b3080 0x7fc3a4072da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3400640 1 -- 172.21.15.59:0/94742693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a40c7bd0 con 0x7fc3a40cd050 2024-04-23T14:36:34.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a3e00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a40b3080 0x7fc3a4072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:34.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.014+0000 7fc3a9200640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a415ece0 0x7fc3a409d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:34.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.015+0000 7fc3a3400640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a40cd050 0x7fc3a40b1060 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc390000c00 tx=0x7fc3900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.015+0000 7fc39be00640 1 -- 172.21.15.59:0/94742693 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390041020 con 0x7fc3a40cd050 2024-04-23T14:36:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.015+0000 7fc39be00640 1 -- 172.21.15.59:0/94742693 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3900306c0 con 0x7fc3a40cd050 2024-04-23T14:36:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.015+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a40afaf0 con 0x7fc3a40cd050 2024-04-23T14:36:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.015+0000 7fc39be00640 1 -- 172.21.15.59:0/94742693 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390037040 con 0x7fc3a40cd050 2024-04-23T14:36:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.015+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a40b00b0 con 0x7fc3a40cd050 2024-04-23T14:36:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.017+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a404fd90 con 0x7fc3a40cd050 2024-04-23T14:36:34.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.018+0000 7fc39be00640 1 -- 172.21.15.59:0/94742693 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc390044020 con 0x7fc3a40cd050 2024-04-23T14:36:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.019+0000 7fc39be00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc38406c8f0 0x7fc38406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.019+0000 7fc39be00640 1 -- 172.21.15.59:0/94742693 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fc3900ac3c0 con 0x7fc3a40cd050 2024-04-23T14:36:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.019+0000 7fc3a3e00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc38406c8f0 0x7fc38406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.020+0000 7fc3a3e00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc38406c8f0 0x7fc38406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc394006fd0 tx=0x7fc394008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.024+0000 7fc39be00640 1 -- 172.21.15.59:0/94742693 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc39004b030 con 0x7fc3a40cd050 2024-04-23T14:36:34.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.227+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3a40d0100 con 0x7fc3a40cd050 2024-04-23T14:36:34.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.229+0000 7fc39be00640 1 -- 172.21.15.59:0/94742693 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16608 (secure 0 0 0) 0x7fc3a40c7bd0 con 0x7fc3a40cd050 2024-04-23T14:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":69,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:31.243328+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:34.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc38406c8f0 msgr2=0x7fc38406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc38406c8f0 0x7fc38406edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc394006fd0 tx=0x7fc394008040 comp rx=0 tx=0).stop 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a40cd050 msgr2=0x7fc3a40b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a40cd050 0x7fc3a40b1060 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc390000c00 tx=0x7fc3900046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 shutdown_connections 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc38406c8f0 0x7fc38406edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a415ece0 0x7fc3a409d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a40cd050 0x7fc3a40b1060 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 --2- 172.21.15.59:0/94742693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a40b3080 0x7fc3a4072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.231+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 >> 172.21.15.59:0/94742693 conn(0x7fc3a40b21a0 msgr2=0x7fc3a40d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.232+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 shutdown_connections 2024-04-23T14:36:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.232+0000 7fc3a9c00640 1 -- 172.21.15.59:0/94742693 wait complete. 2024-04-23T14:36:34.251 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:36:34.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:36:34.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.460+0000 7f286e400640 1 Processor -- start 2024-04-23T14:36:34.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.460+0000 7f286e400640 1 -- start start 2024-04-23T14:36:34.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286e400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f286815b450 0x7f28680b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286e400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28680b3870 con 0x7f286806f3f0 2024-04-23T14:36:34.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286e400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28680b3b50 con 0x7f28680cd760 2024-04-23T14:36:34.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286e400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f28680b41a0 con 0x7f286815b450 2024-04-23T14:36:34.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f286815b450 0x7f28680b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f2867e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f286806f3f0 0x7f28680cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35178/0 (socket says 172.21.15.59:35178) 2024-04-23T14:36:34.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f286815b450 0x7f28680b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41430/0 (socket says 172.21.15.59:41430) 2024-04-23T14:36:34.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f2867e00640 1 -- 172.21.15.59:0/799667193 learned_addr learned my addr 172.21.15.59:0/799667193 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:34.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286d000640 1 -- 172.21.15.59:0/799667193 >> v1:172.21.15.76:6789/0 conn(0x7f28680cd760 legacy=0x7f28680ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:34.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286d000640 1 -- 172.21.15.59:0/799667193 >> v1:172.21.15.59:6789/0 conn(0x7f286806f3f0 legacy=0x7f28680cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.462+0000 7f286d000640 1 -- 172.21.15.59:0/799667193 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28680b3e30 con 0x7f286815b450 2024-04-23T14:36:34.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.463+0000 7f286d000640 1 --2- 172.21.15.59:0/799667193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286815b450 0x7f28680b4630 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cad2b2f672c856 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.463+0000 7f2867400640 1 -- 172.21.15.59:0/799667193 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858007740 con 0x7f286815b450 2024-04-23T14:36:34.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.463+0000 7f2867400640 1 -- 172.21.15.59:0/799667193 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28580078e0 con 0x7f286815b450 2024-04-23T14:36:34.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.463+0000 7f2867400640 1 -- 172.21.15.59:0/799667193 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858007cb0 con 0x7f286815b450 2024-04-23T14:36:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.464+0000 7f286e400640 1 -- 172.21.15.59:0/799667193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286815b450 msgr2=0x7f28680b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.464+0000 7f286e400640 1 --2- 172.21.15.59:0/799667193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286815b450 0x7f28680b4630 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285802a730 comp rx=0 tx=0).stop 2024-04-23T14:36:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.464+0000 7f286e400640 1 -- 172.21.15.59:0/799667193 shutdown_connections 2024-04-23T14:36:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.464+0000 7f286e400640 1 --2- 172.21.15.59:0/799667193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286815b450 0x7f28680b4630 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.464+0000 7f286e400640 1 -- 172.21.15.59:0/799667193 >> 172.21.15.59:0/799667193 conn(0x7f28680b21a0 msgr2=0x7f28680b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.464+0000 7f286e400640 1 -- 172.21.15.59:0/799667193 shutdown_connections 2024-04-23T14:36:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.464+0000 7f286e400640 1 -- 172.21.15.59:0/799667193 wait complete. 2024-04-23T14:36:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.466+0000 7f286e400640 1 Processor -- start 2024-04-23T14:36:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.466+0000 7f286e400640 1 -- start start 2024-04-23T14:36:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.467+0000 7f286e400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286806f3f0 0x7f286809bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.467+0000 7f286e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28680cd760 0x7f28681f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.467+0000 7f286e400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f286815b450 0x7f286806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.467+0000 7f286e400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28680b4ff0 con 0x7f28680cd760 2024-04-23T14:36:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.467+0000 7f286e400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f286815b830 con 0x7f286806f3f0 2024-04-23T14:36:34.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.467+0000 7f286e400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f28680b3080 con 0x7f286815b450 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.467+0000 7f2867e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28680cd760 0x7f28681f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28680cd760 0x7f28681f4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41442/0 (socket says 172.21.15.59:41442) 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 -- 172.21.15.59:0/3134272996 learned_addr learned my addr 172.21.15.59:0/3134272996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f286d000640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286806f3f0 0x7f286809bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 -- 172.21.15.59:0/3134272996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f286815b450 msgr2=0x7f286806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f286815b450 0x7f286806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 -- 172.21.15.59:0/3134272996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286806f3f0 msgr2=0x7f286809bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286806f3f0 0x7f286809bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 -- 172.21.15.59:0/3134272996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28680b0870 con 0x7f28680cd760 2024-04-23T14:36:34.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.468+0000 7f2867e00640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28680cd760 0x7f28681f4430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f285400c430 tx=0x7f285400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.469+0000 7f2864c00640 1 -- 172.21.15.59:0/3134272996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854017070 con 0x7f28680cd760 2024-04-23T14:36:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.469+0000 7f2864c00640 1 -- 172.21.15.59:0/3134272996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2854005ce0 con 0x7f28680cd760 2024-04-23T14:36:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.469+0000 7f2864c00640 1 -- 172.21.15.59:0/3134272996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540072f0 con 0x7f28680cd760 2024-04-23T14:36:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.469+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2868155cb0 con 0x7f28680cd760 2024-04-23T14:36:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.469+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28680b0540 con 0x7f28680cd760 2024-04-23T14:36:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.470+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286804fd90 con 0x7f28680cd760 2024-04-23T14:36:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.471+0000 7f2864c00640 1 -- 172.21.15.59:0/3134272996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f285401a070 con 0x7f28680cd760 2024-04-23T14:36:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.472+0000 7f2864c00640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f284806c8f0 0x7f284806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.472+0000 7f2864c00640 1 -- 172.21.15.59:0/3134272996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f285408af70 con 0x7f28680cd760 2024-04-23T14:36:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.472+0000 7f286d000640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f284806c8f0 0x7f284806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.473+0000 7f286d000640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f284806c8f0 0x7f284806edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2858034000 tx=0x7f2858033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.477+0000 7f2864c00640 1 -- 172.21.15.59:0/3134272996 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28540593a0 con 0x7f28680cd760 2024-04-23T14:36:34.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.678+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28680c36f0 con 0x7f284806c8f0 2024-04-23T14:36:34.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.699+0000 7f2864c00640 1 -- 172.21.15.59:0/3134272996 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247579 (secure 0 0 0) 0x7f28680c36f0 con 0x7f284806c8f0 2024-04-23T14:36:34.703 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:36:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f284806c8f0 msgr2=0x7f284806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f284806c8f0 0x7f284806edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2858034000 tx=0x7f2858033040 comp rx=0 tx=0).stop 2024-04-23T14:36:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28680cd760 msgr2=0x7f28681f4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28680cd760 0x7f28681f4430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f285400c430 tx=0x7f285400c8f0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 shutdown_connections 2024-04-23T14:36:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f284806c8f0 0x7f284806edb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f286815b450 0x7f286806b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28680cd760 0x7f28681f4430 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 --2- 172.21.15.59:0/3134272996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f286806f3f0 0x7f286809bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 >> 172.21.15.59:0/3134272996 conn(0x7f28680b21a0 msgr2=0x7f286809dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 shutdown_connections 2024-04-23T14:36:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.705+0000 7f286e400640 1 -- 172.21.15.59:0/3134272996 wait complete. 2024-04-23T14:36:34.726 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-23T14:36:34.449081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256976,"kb_used_data":7516,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717744,"statfs":{"total":773094113280,"available":772830969856,"internally_reserved":0,"allocated":7696384,"data_stored":3894979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001351"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.257806+0000","last_change":"2024-04-23T14:36:29.257806+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.257806+0000","last_undegraded":"2024-04-23T14:36:29.257806+0000","last_fullsized":"2024-04-23T14:36:29.257806+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238437+0000","last_change":"2024-04-23T14:36:29.238437+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238437+0000","last_undegraded":"2024-04-23T14:36:29.238437+0000","last_fullsized":"2024-04-23T14:36:29.238437+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239371+0000","last_change":"2024-04-23T14:36:29.239371+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239371+0000","last_undegraded":"2024-04-23T14:36:29.239371+0000","last_fullsized":"2024-04-23T14:36:29.239371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.248757+0000","last_change":"2024-04-23T14:36:29.248757+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.248757+0000","last_undegraded":"2024-04-23T14:36:29.248757+0000","last_fullsized":"2024-04-23T14:36:29.248757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256769+0000","last_change":"2024-04-23T14:36:30.271347+0000","last_active":"2024-04-23T14:36:31.256769+0000","last_peered":"2024-04-23T14:36:31.256769+0000","last_clean":"2024-04-23T14:36:31.256769+0000","last_became_active":"2024-04-23T14:36:30.270634+0000","last_became_peered":"2024-04-23T14:36:30.270634+0000","last_unstale":"2024-04-23T14:36:31.256769+0000","last_undegraded":"2024-04-23T14:36:31.256769+0000","last_fullsized":"2024-04-23T14:36:31.256769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:26.664536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243660+0000","last_change":"2024-04-23T14:36:29.243660+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243660+0000","last_undegraded":"2024-04-23T14:36:29.243660+0000","last_fullsized":"2024-04-23T14:36:29.243660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239846+0000","last_change":"2024-04-23T14:36:29.239846+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239846+0000","last_undegraded":"2024-04-23T14:36:29.239846+0000","last_fullsized":"2024-04-23T14:36:29.239846+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.237516+0000","last_change":"2024-04-23T14:36:29.237516+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.237516+0000","last_undegraded":"2024-04-23T14:36:29.237516+0000","last_fullsized":"2024-04-23T14:36:29.237516+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257521+0000","last_change":"2024-04-23T14:36:30.256388+0000","last_active":"2024-04-23T14:36:31.257521+0000","last_peered":"2024-04-23T14:36:31.257521+0000","last_clean":"2024-04-23T14:36:31.257521+0000","last_became_active":"2024-04-23T14:36:30.255042+0000","last_became_peered":"2024-04-23T14:36:30.255042+0000","last_unstale":"2024-04-23T14:36:31.257521+0000","last_undegraded":"2024-04-23T14:36:31.257521+0000","last_fullsized":"2024-04-23T14:36:31.257521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:38.281474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243637+0000","last_change":"2024-04-23T14:36:29.243637+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243637+0000","last_undegraded":"2024-04-23T14:36:29.243637+0000","last_fullsized":"2024-04-23T14:36:29.243637+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.252268+0000","last_change":"2024-04-23T14:36:29.252268+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.252268+0000","last_undegraded":"2024-04-23T14:36:29.252268+0000","last_fullsized":"2024-04-23T14:36:29.252268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.248093+0000","last_change":"2024-04-23T14:36:29.248093+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.248093+0000","last_undegraded":"2024-04-23T14:36:29.248093+0000","last_fullsized":"2024-04-23T14:36:29.248093+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242843+0000","last_change":"2024-04-23T14:36:29.242843+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242843+0000","last_undegraded":"2024-04-23T14:36:29.242843+0000","last_fullsized":"2024-04-23T14:36:29.242843+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257406+0000","last_change":"2024-04-23T14:36:30.276001+0000","last_active":"2024-04-23T14:36:31.257406+0000","last_peered":"2024-04-23T14:36:31.257406+0000","last_clean":"2024-04-23T14:36:31.257406+0000","last_became_active":"2024-04-23T14:36:30.271406+0000","last_became_peered":"2024-04-23T14:36:30.271406+0000","last_unstale":"2024-04-23T14:36:31.257406+0000","last_undegraded":"2024-04-23T14:36:31.257406+0000","last_fullsized":"2024-04-23T14:36:31.257406+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:05.035180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242605+0000","last_change":"2024-04-23T14:36:29.242605+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242605+0000","last_undegraded":"2024-04-23T14:36:29.242605+0000","last_fullsized":"2024-04-23T14:36:29.242605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.246845+0000","last_change":"2024-04-23T14:36:29.246845+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.246845+0000","last_undegraded":"2024-04-23T14:36:29.246845+0000","last_fullsized":"2024-04-23T14:36:29.246845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257673+0000","last_change":"2024-04-23T14:36:30.257389+0000","last_active":"2024-04-23T14:36:31.257673+0000","last_peered":"2024-04-23T14:36:31.257673+0000","last_clean":"2024-04-23T14:36:31.257673+0000","last_became_active":"2024-04-23T14:36:30.256095+0000","last_became_peered":"2024-04-23T14:36:30.256095+0000","last_unstale":"2024-04-23T14:36:31.257673+0000","last_undegraded":"2024-04-23T14:36:31.257673+0000","last_fullsized":"2024-04-23T14:36:31.257673+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:48.658488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247241+0000","last_change":"2024-04-23T14:36:29.247241+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247241+0000","last_undegraded":"2024-04-23T14:36:29.247241+0000","last_fullsized":"2024-04-23T14:36:29.247241+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243996+0000","last_change":"2024-04-23T14:36:29.243996+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243996+0000","last_undegraded":"2024-04-23T14:36:29.243996+0000","last_fullsized":"2024-04-23T14:36:29.243996+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239476+0000","last_change":"2024-04-23T14:36:29.239476+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239476+0000","last_undegraded":"2024-04-23T14:36:29.239476+0000","last_fullsized":"2024-04-23T14:36:29.239476+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243017+0000","last_change":"2024-04-23T14:36:29.243017+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243017+0000","last_undegraded":"2024-04-23T14:36:29.243017+0000","last_fullsized":"2024-04-23T14:36:29.243017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247488+0000","last_change":"2024-04-23T14:36:29.247488+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247488+0000","last_undegraded":"2024-04-23T14:36:29.247488+0000","last_fullsized":"2024-04-23T14:36:29.247488+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.250249+0000","last_change":"2024-04-23T14:36:29.250249+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.250249+0000","last_undegraded":"2024-04-23T14:36:29.250249+0000","last_fullsized":"2024-04-23T14:36:29.250249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.254142+0000","last_change":"2024-04-23T14:36:29.254142+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.254142+0000","last_undegraded":"2024-04-23T14:36:29.254142+0000","last_fullsized":"2024-04-23T14:36:29.254142+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247037+0000","last_change":"2024-04-23T14:36:29.247037+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247037+0000","last_undegraded":"2024-04-23T14:36:29.247037+0000","last_fullsized":"2024-04-23T14:36:29.247037+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239768+0000","last_change":"2024-04-23T14:36:29.239768+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239768+0000","last_undegraded":"2024-04-23T14:36:29.239768+0000","last_fullsized":"2024-04-23T14:36:29.239768+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238732+0000","last_change":"2024-04-23T14:36:29.238732+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238732+0000","last_undegraded":"2024-04-23T14:36:29.238732+0000","last_fullsized":"2024-04-23T14:36:29.238732+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245148+0000","last_change":"2024-04-23T14:36:29.245148+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245148+0000","last_undegraded":"2024-04-23T14:36:29.245148+0000","last_fullsized":"2024-04-23T14:36:29.245148+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241311+0000","last_change":"2024-04-23T14:36:29.241311+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241311+0000","last_undegraded":"2024-04-23T14:36:29.241311+0000","last_fullsized":"2024-04-23T14:36:29.241311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238095+0000","last_change":"2024-04-23T14:36:29.238095+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238095+0000","last_undegraded":"2024-04-23T14:36:29.238095+0000","last_fullsized":"2024-04-23T14:36:29.238095+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235704+0000","last_change":"2024-04-23T14:36:29.235704+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235704+0000","last_undegraded":"2024-04-23T14:36:29.235704+0000","last_fullsized":"2024-04-23T14:36:29.235704+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235950+0000","last_change":"2024-04-23T14:36:29.235950+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235950+0000","last_undegraded":"2024-04-23T14:36:29.235950+0000","last_fullsized":"2024-04-23T14:36:29.235950+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251241+0000","last_change":"2024-04-23T14:36:29.251241+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251241+0000","last_undegraded":"2024-04-23T14:36:29.251241+0000","last_fullsized":"2024-04-23T14:36:29.251241+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266770+0000","last_change":"2024-04-23T14:36:29.265430+0000","last_active":"2024-04-23T14:36:29.266770+0000","last_peered":"2024-04-23T14:36:29.266770+0000","last_clean":"2024-04-23T14:36:29.266770+0000","last_became_active":"2024-04-23T14:36:29.265096+0000","last_became_peered":"2024-04-23T14:36:29.265096+0000","last_unstale":"2024-04-23T14:36:29.266770+0000","last_undegraded":"2024-04-23T14:36:29.266770+0000","last_fullsized":"2024-04-23T14:36:29.266770+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:29.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257718+0000","last_change":"2024-04-23T14:36:30.257027+0000","last_active":"2024-04-23T14:36:31.257718+0000","last_peered":"2024-04-23T14:36:31.257718+0000","last_clean":"2024-04-23T14:36:31.257718+0000","last_became_active":"2024-04-23T14:36:30.255514+0000","last_became_peered":"2024-04-23T14:36:30.255514+0000","last_unstale":"2024-04-23T14:36:31.257718+0000","last_undegraded":"2024-04-23T14:36:31.257718+0000","last_fullsized":"2024-04-23T14:36:31.257718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:21.165754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259920+0000","last_change":"2024-04-23T14:36:29.255962+0000","last_active":"2024-04-23T14:36:29.259920+0000","last_peered":"2024-04-23T14:36:29.259920+0000","last_clean":"2024-04-23T14:36:29.259920+0000","last_became_active":"2024-04-23T14:36:29.255566+0000","last_became_peered":"2024-04-23T14:36:29.255566+0000","last_unstale":"2024-04-23T14:36:29.259920+0000","last_undegraded":"2024-04-23T14:36:29.259920+0000","last_fullsized":"2024-04-23T14:36:29.259920+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:53.438649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245613+0000","last_change":"2024-04-23T14:36:29.245613+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245613+0000","last_undegraded":"2024-04-23T14:36:29.245613+0000","last_fullsized":"2024-04-23T14:36:29.245613+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.268721+0000","last_change":"2024-04-23T14:36:29.264940+0000","last_active":"2024-04-23T14:36:29.268721+0000","last_peered":"2024-04-23T14:36:29.268721+0000","last_clean":"2024-04-23T14:36:29.268721+0000","last_became_active":"2024-04-23T14:36:29.264568+0000","last_became_peered":"2024-04-23T14:36:29.264568+0000","last_unstale":"2024-04-23T14:36:29.268721+0000","last_undegraded":"2024-04-23T14:36:29.268721+0000","last_fullsized":"2024-04-23T14:36:29.268721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:56:10.745359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251760+0000","last_change":"2024-04-23T14:36:29.251760+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251760+0000","last_undegraded":"2024-04-23T14:36:29.251760+0000","last_fullsized":"2024-04-23T14:36:29.251760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260946+0000","last_change":"2024-04-23T14:36:29.258194+0000","last_active":"2024-04-23T14:36:29.260946+0000","last_peered":"2024-04-23T14:36:29.260946+0000","last_clean":"2024-04-23T14:36:29.260946+0000","last_became_active":"2024-04-23T14:36:29.257888+0000","last_became_peered":"2024-04-23T14:36:29.257888+0000","last_unstale":"2024-04-23T14:36:29.260946+0000","last_undegraded":"2024-04-23T14:36:29.260946+0000","last_fullsized":"2024-04-23T14:36:29.260946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:37.263336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235815+0000","last_change":"2024-04-23T14:36:29.235815+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235815+0000","last_undegraded":"2024-04-23T14:36:29.235815+0000","last_fullsized":"2024-04-23T14:36:29.235815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260424+0000","last_change":"2024-04-23T14:36:29.259191+0000","last_active":"2024-04-23T14:36:29.260424+0000","last_peered":"2024-04-23T14:36:29.260424+0000","last_clean":"2024-04-23T14:36:29.260424+0000","last_became_active":"2024-04-23T14:36:29.258420+0000","last_became_peered":"2024-04-23T14:36:29.258420+0000","last_unstale":"2024-04-23T14:36:29.260424+0000","last_undegraded":"2024-04-23T14:36:29.260424+0000","last_fullsized":"2024-04-23T14:36:29.260424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:49.177144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241971+0000","last_change":"2024-04-23T14:36:29.241971+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241971+0000","last_undegraded":"2024-04-23T14:36:29.241971+0000","last_fullsized":"2024-04-23T14:36:29.241971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260826+0000","last_change":"2024-04-23T14:36:29.255423+0000","last_active":"2024-04-23T14:36:29.260826+0000","last_peered":"2024-04-23T14:36:29.260826+0000","last_clean":"2024-04-23T14:36:29.260826+0000","last_became_active":"2024-04-23T14:36:29.255130+0000","last_became_peered":"2024-04-23T14:36:29.255130+0000","last_unstale":"2024-04-23T14:36:29.260826+0000","last_undegraded":"2024-04-23T14:36:29.260826+0000","last_fullsized":"2024-04-23T14:36:29.260826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:35.193457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256577+0000","last_change":"2024-04-23T14:36:30.259813+0000","last_active":"2024-04-23T14:36:31.256577+0000","last_peered":"2024-04-23T14:36:31.256577+0000","last_clean":"2024-04-23T14:36:31.256577+0000","last_became_active":"2024-04-23T14:36:30.259020+0000","last_became_peered":"2024-04-23T14:36:30.259020+0000","last_unstale":"2024-04-23T14:36:31.256577+0000","last_undegraded":"2024-04-23T14:36:31.256577+0000","last_fullsized":"2024-04-23T14:36:31.256577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:28.217244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264303+0000","last_change":"2024-04-23T14:36:29.262680+0000","last_active":"2024-04-23T14:36:29.264303+0000","last_peered":"2024-04-23T14:36:29.264303+0000","last_clean":"2024-04-23T14:36:29.264303+0000","last_became_active":"2024-04-23T14:36:29.262177+0000","last_became_peered":"2024-04-23T14:36:29.262177+0000","last_unstale":"2024-04-23T14:36:29.264303+0000","last_undegraded":"2024-04-23T14:36:29.264303+0000","last_fullsized":"2024-04-23T14:36:29.264303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242083+0000","last_change":"2024-04-23T14:36:29.242083+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242083+0000","last_undegraded":"2024-04-23T14:36:29.242083+0000","last_fullsized":"2024-04-23T14:36:29.242083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256559+0000","last_change":"2024-04-23T14:36:29.255068+0000","last_active":"2024-04-23T14:36:31.256559+0000","last_peered":"2024-04-23T14:36:31.256559+0000","last_clean":"2024-04-23T14:36:31.256559+0000","last_became_active":"2024-04-23T14:36:29.254465+0000","last_became_peered":"2024-04-23T14:36:29.254465+0000","last_unstale":"2024-04-23T14:36:31.256559+0000","last_undegraded":"2024-04-23T14:36:31.256559+0000","last_fullsized":"2024-04-23T14:36:31.256559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:47.763347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236904+0000","last_change":"2024-04-23T14:36:29.236904+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236904+0000","last_undegraded":"2024-04-23T14:36:29.236904+0000","last_fullsized":"2024-04-23T14:36:29.236904+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266420+0000","last_change":"2024-04-23T14:36:29.262461+0000","last_active":"2024-04-23T14:36:29.266420+0000","last_peered":"2024-04-23T14:36:29.266420+0000","last_clean":"2024-04-23T14:36:29.266420+0000","last_became_active":"2024-04-23T14:36:29.262134+0000","last_became_peered":"2024-04-23T14:36:29.262134+0000","last_unstale":"2024-04-23T14:36:29.266420+0000","last_undegraded":"2024-04-23T14:36:29.266420+0000","last_fullsized":"2024-04-23T14:36:29.266420+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.255161+0000","last_change":"2024-04-23T14:36:29.255161+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.255161+0000","last_undegraded":"2024-04-23T14:36:29.255161+0000","last_fullsized":"2024-04-23T14:36:29.255161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260641+0000","last_change":"2024-04-23T14:36:29.252722+0000","last_active":"2024-04-23T14:36:29.260641+0000","last_peered":"2024-04-23T14:36:29.260641+0000","last_clean":"2024-04-23T14:36:29.260641+0000","last_became_active":"2024-04-23T14:36:29.252421+0000","last_became_peered":"2024-04-23T14:36:29.252421+0000","last_unstale":"2024-04-23T14:36:29.260641+0000","last_undegraded":"2024-04-23T14:36:29.260641+0000","last_fullsized":"2024-04-23T14:36:29.260641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:27.199106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243002+0000","last_change":"2024-04-23T14:36:29.243002+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243002+0000","last_undegraded":"2024-04-23T14:36:29.243002+0000","last_fullsized":"2024-04-23T14:36:29.243002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.258545+0000","last_change":"2024-04-23T14:36:29.255338+0000","last_active":"2024-04-23T14:36:29.258545+0000","last_peered":"2024-04-23T14:36:29.258545+0000","last_clean":"2024-04-23T14:36:29.258545+0000","last_became_active":"2024-04-23T14:36:29.254916+0000","last_became_peered":"2024-04-23T14:36:29.254916+0000","last_unstale":"2024-04-23T14:36:29.258545+0000","last_undegraded":"2024-04-23T14:36:29.258545+0000","last_fullsized":"2024-04-23T14:36:29.258545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:39.168711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257203+0000","last_change":"2024-04-23T14:36:30.267547+0000","last_active":"2024-04-23T14:36:31.257203+0000","last_peered":"2024-04-23T14:36:31.257203+0000","last_clean":"2024-04-23T14:36:31.257203+0000","last_became_active":"2024-04-23T14:36:30.267103+0000","last_became_peered":"2024-04-23T14:36:30.267103+0000","last_unstale":"2024-04-23T14:36:31.257203+0000","last_undegraded":"2024-04-23T14:36:31.257203+0000","last_fullsized":"2024-04-23T14:36:31.257203+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:14.787860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260627+0000","last_change":"2024-04-23T14:36:29.257996+0000","last_active":"2024-04-23T14:36:29.260627+0000","last_peered":"2024-04-23T14:36:29.260627+0000","last_clean":"2024-04-23T14:36:29.260627+0000","last_became_active":"2024-04-23T14:36:29.256591+0000","last_became_peered":"2024-04-23T14:36:29.256591+0000","last_unstale":"2024-04-23T14:36:29.260627+0000","last_undegraded":"2024-04-23T14:36:29.260627+0000","last_fullsized":"2024-04-23T14:36:29.260627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:47.640350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.249860+0000","last_change":"2024-04-23T14:36:29.249860+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.249860+0000","last_undegraded":"2024-04-23T14:36:29.249860+0000","last_fullsized":"2024-04-23T14:36:29.249860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264329+0000","last_change":"2024-04-23T14:36:29.262784+0000","last_active":"2024-04-23T14:36:29.264329+0000","last_peered":"2024-04-23T14:36:29.264329+0000","last_clean":"2024-04-23T14:36:29.264329+0000","last_became_active":"2024-04-23T14:36:29.262418+0000","last_became_peered":"2024-04-23T14:36:29.262418+0000","last_unstale":"2024-04-23T14:36:29.264329+0000","last_undegraded":"2024-04-23T14:36:29.264329+0000","last_fullsized":"2024-04-23T14:36:29.264329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238374+0000","last_change":"2024-04-23T14:36:29.238374+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238374+0000","last_undegraded":"2024-04-23T14:36:29.238374+0000","last_fullsized":"2024-04-23T14:36:29.238374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260842+0000","last_change":"2024-04-23T14:36:29.256582+0000","last_active":"2024-04-23T14:36:29.260842+0000","last_peered":"2024-04-23T14:36:29.260842+0000","last_clean":"2024-04-23T14:36:29.260842+0000","last_became_active":"2024-04-23T14:36:29.256299+0000","last_became_peered":"2024-04-23T14:36:29.256299+0000","last_unstale":"2024-04-23T14:36:29.260842+0000","last_undegraded":"2024-04-23T14:36:29.260842+0000","last_fullsized":"2024-04-23T14:36:29.260842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:04.017042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236627+0000","last_change":"2024-04-23T14:36:29.236627+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236627+0000","last_undegraded":"2024-04-23T14:36:29.236627+0000","last_fullsized":"2024-04-23T14:36:29.236627+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259444+0000","last_change":"2024-04-23T14:36:29.254735+0000","last_active":"2024-04-23T14:36:29.259444+0000","last_peered":"2024-04-23T14:36:29.259444+0000","last_clean":"2024-04-23T14:36:29.259444+0000","last_became_active":"2024-04-23T14:36:29.253176+0000","last_became_peered":"2024-04-23T14:36:29.253176+0000","last_unstale":"2024-04-23T14:36:29.259444+0000","last_undegraded":"2024-04-23T14:36:29.259444+0000","last_fullsized":"2024-04-23T14:36:29.259444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:45.728544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256827+0000","last_change":"2024-04-23T14:36:30.254759+0000","last_active":"2024-04-23T14:36:31.256827+0000","last_peered":"2024-04-23T14:36:31.256827+0000","last_clean":"2024-04-23T14:36:31.256827+0000","last_became_active":"2024-04-23T14:36:30.254477+0000","last_became_peered":"2024-04-23T14:36:30.254477+0000","last_unstale":"2024-04-23T14:36:31.256827+0000","last_undegraded":"2024-04-23T14:36:31.256827+0000","last_fullsized":"2024-04-23T14:36:31.256827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:15.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266716+0000","last_change":"2024-04-23T14:36:29.263312+0000","last_active":"2024-04-23T14:36:29.266716+0000","last_peered":"2024-04-23T14:36:29.266716+0000","last_clean":"2024-04-23T14:36:29.266716+0000","last_became_active":"2024-04-23T14:36:29.262939+0000","last_became_peered":"2024-04-23T14:36:29.262939+0000","last_unstale":"2024-04-23T14:36:29.266716+0000","last_undegraded":"2024-04-23T14:36:29.266716+0000","last_fullsized":"2024-04-23T14:36:29.266716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:18:16.720971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242509+0000","last_change":"2024-04-23T14:36:29.242509+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242509+0000","last_undegraded":"2024-04-23T14:36:29.242509+0000","last_fullsized":"2024-04-23T14:36:29.242509+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259691+0000","last_change":"2024-04-23T14:36:29.251466+0000","last_active":"2024-04-23T14:36:29.259691+0000","last_peered":"2024-04-23T14:36:29.259691+0000","last_clean":"2024-04-23T14:36:29.259691+0000","last_became_active":"2024-04-23T14:36:29.250985+0000","last_became_peered":"2024-04-23T14:36:29.250985+0000","last_unstale":"2024-04-23T14:36:29.259691+0000","last_undegraded":"2024-04-23T14:36:29.259691+0000","last_fullsized":"2024-04-23T14:36:29.259691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:14.802229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.254476+0000","last_change":"2024-04-23T14:36:29.254476+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.254476+0000","last_undegraded":"2024-04-23T14:36:29.254476+0000","last_fullsized":"2024-04-23T14:36:29.254476+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264202+0000","last_change":"2024-04-23T14:36:29.257682+0000","last_active":"2024-04-23T14:36:29.264202+0000","last_peered":"2024-04-23T14:36:29.264202+0000","last_clean":"2024-04-23T14:36:29.264202+0000","last_became_active":"2024-04-23T14:36:29.257385+0000","last_became_peered":"2024-04-23T14:36:29.257385+0000","last_unstale":"2024-04-23T14:36:29.264202+0000","last_undegraded":"2024-04-23T14:36:29.264202+0000","last_fullsized":"2024-04-23T14:36:29.264202+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247591+0000","last_change":"2024-04-23T14:36:29.247591+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247591+0000","last_undegraded":"2024-04-23T14:36:29.247591+0000","last_fullsized":"2024-04-23T14:36:29.247591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264676+0000","last_change":"2024-04-23T14:36:29.262796+0000","last_active":"2024-04-23T14:36:29.264676+0000","last_peered":"2024-04-23T14:36:29.264676+0000","last_clean":"2024-04-23T14:36:29.264676+0000","last_became_active":"2024-04-23T14:36:29.262504+0000","last_became_peered":"2024-04-23T14:36:29.262504+0000","last_unstale":"2024-04-23T14:36:29.264676+0000","last_undegraded":"2024-04-23T14:36:29.264676+0000","last_fullsized":"2024-04-23T14:36:29.264676+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257317+0000","last_change":"2024-04-23T14:36:30.260184+0000","last_active":"2024-04-23T14:36:31.257317+0000","last_peered":"2024-04-23T14:36:31.257317+0000","last_clean":"2024-04-23T14:36:31.257317+0000","last_became_active":"2024-04-23T14:36:30.259450+0000","last_became_peered":"2024-04-23T14:36:30.259450+0000","last_unstale":"2024-04-23T14:36:31.257317+0000","last_undegraded":"2024-04-23T14:36:31.257317+0000","last_fullsized":"2024-04-23T14:36:31.257317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:36.211595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.269267+0000","last_change":"2024-04-23T14:36:29.267357+0000","last_active":"2024-04-23T14:36:29.269267+0000","last_peered":"2024-04-23T14:36:29.269267+0000","last_clean":"2024-04-23T14:36:29.269267+0000","last_became_active":"2024-04-23T14:36:29.266992+0000","last_became_peered":"2024-04-23T14:36:29.266992+0000","last_unstale":"2024-04-23T14:36:29.269267+0000","last_undegraded":"2024-04-23T14:36:29.269267+0000","last_fullsized":"2024-04-23T14:36:29.269267+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:45.728544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.240681+0000","last_change":"2024-04-23T14:36:29.240681+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.240681+0000","last_undegraded":"2024-04-23T14:36:29.240681+0000","last_fullsized":"2024-04-23T14:36:29.240681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260804+0000","last_change":"2024-04-23T14:36:29.254896+0000","last_active":"2024-04-23T14:36:29.260804+0000","last_peered":"2024-04-23T14:36:29.260804+0000","last_clean":"2024-04-23T14:36:29.260804+0000","last_became_active":"2024-04-23T14:36:29.254593+0000","last_became_peered":"2024-04-23T14:36:29.254593+0000","last_unstale":"2024-04-23T14:36:29.260804+0000","last_undegraded":"2024-04-23T14:36:29.260804+0000","last_fullsized":"2024-04-23T14:36:29.260804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:13.769723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241152+0000","last_change":"2024-04-23T14:36:29.241152+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241152+0000","last_undegraded":"2024-04-23T14:36:29.241152+0000","last_fullsized":"2024-04-23T14:36:29.241152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260903+0000","last_change":"2024-04-23T14:36:29.259253+0000","last_active":"2024-04-23T14:36:29.260903+0000","last_peered":"2024-04-23T14:36:29.260903+0000","last_clean":"2024-04-23T14:36:29.260903+0000","last_became_active":"2024-04-23T14:36:29.256871+0000","last_became_peered":"2024-04-23T14:36:29.256871+0000","last_unstale":"2024-04-23T14:36:29.260903+0000","last_undegraded":"2024-04-23T14:36:29.260903+0000","last_fullsized":"2024-04-23T14:36:29.260903+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:33.990830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238310+0000","last_change":"2024-04-23T14:36:29.238310+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238310+0000","last_undegraded":"2024-04-23T14:36:29.238310+0000","last_fullsized":"2024-04-23T14:36:29.238310+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.254977+0000","last_change":"2024-04-23T14:36:29.253692+0000","last_active":"2024-04-23T14:36:29.254977+0000","last_peered":"2024-04-23T14:36:29.254977+0000","last_clean":"2024-04-23T14:36:29.254977+0000","last_became_active":"2024-04-23T14:36:29.253295+0000","last_became_peered":"2024-04-23T14:36:29.253295+0000","last_unstale":"2024-04-23T14:36:29.254977+0000","last_undegraded":"2024-04-23T14:36:29.254977+0000","last_fullsized":"2024-04-23T14:36:29.254977+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:18.617182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.240864+0000","last_change":"2024-04-23T14:36:29.240864+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.240864+0000","last_undegraded":"2024-04-23T14:36:29.240864+0000","last_fullsized":"2024-04-23T14:36:29.240864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.253828+0000","last_change":"2024-04-23T14:36:29.249580+0000","last_active":"2024-04-23T14:36:29.253828+0000","last_peered":"2024-04-23T14:36:29.253828+0000","last_clean":"2024-04-23T14:36:29.253828+0000","last_became_active":"2024-04-23T14:36:29.249248+0000","last_became_peered":"2024-04-23T14:36:29.249248+0000","last_unstale":"2024-04-23T14:36:29.253828+0000","last_undegraded":"2024-04-23T14:36:29.253828+0000","last_fullsized":"2024-04-23T14:36:29.253828+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:25.646399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239729+0000","last_change":"2024-04-23T14:36:29.239729+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239729+0000","last_undegraded":"2024-04-23T14:36:29.239729+0000","last_fullsized":"2024-04-23T14:36:29.239729+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259327+0000","last_change":"2024-04-23T14:36:29.253372+0000","last_active":"2024-04-23T14:36:29.259327+0000","last_peered":"2024-04-23T14:36:29.259327+0000","last_clean":"2024-04-23T14:36:29.259327+0000","last_became_active":"2024-04-23T14:36:29.251760+0000","last_became_peered":"2024-04-23T14:36:29.251760+0000","last_unstale":"2024-04-23T14:36:29.259327+0000","last_undegraded":"2024-04-23T14:36:29.259327+0000","last_fullsized":"2024-04-23T14:36:29.259327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:29.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251080+0000","last_change":"2024-04-23T14:36:29.251080+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251080+0000","last_undegraded":"2024-04-23T14:36:29.251080+0000","last_fullsized":"2024-04-23T14:36:29.251080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257414+0000","last_change":"2024-04-23T14:36:29.245427+0000","last_active":"2024-04-23T14:36:31.257414+0000","last_peered":"2024-04-23T14:36:31.257414+0000","last_clean":"2024-04-23T14:36:31.257414+0000","last_became_active":"2024-04-23T14:36:29.245034+0000","last_became_peered":"2024-04-23T14:36:29.245034+0000","last_unstale":"2024-04-23T14:36:31.257414+0000","last_undegraded":"2024-04-23T14:36:31.257414+0000","last_fullsized":"2024-04-23T14:36:31.257414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:13.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.249579+0000","last_change":"2024-04-23T14:36:29.249579+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.249579+0000","last_undegraded":"2024-04-23T14:36:29.249579+0000","last_fullsized":"2024-04-23T14:36:29.249579+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259982+0000","last_change":"2024-04-23T14:36:29.257966+0000","last_active":"2024-04-23T14:36:29.259982+0000","last_peered":"2024-04-23T14:36:29.259982+0000","last_clean":"2024-04-23T14:36:29.259982+0000","last_became_active":"2024-04-23T14:36:29.257687+0000","last_became_peered":"2024-04-23T14:36:29.257687+0000","last_unstale":"2024-04-23T14:36:29.259982+0000","last_undegraded":"2024-04-23T14:36:29.259982+0000","last_fullsized":"2024-04-23T14:36:29.259982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:43:30.776694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257650+0000","last_change":"2024-04-23T14:36:30.260028+0000","last_active":"2024-04-23T14:36:31.257650+0000","last_peered":"2024-04-23T14:36:31.257650+0000","last_clean":"2024-04-23T14:36:31.257650+0000","last_became_active":"2024-04-23T14:36:30.259713+0000","last_became_peered":"2024-04-23T14:36:30.259713+0000","last_unstale":"2024-04-23T14:36:31.257650+0000","last_undegraded":"2024-04-23T14:36:31.257650+0000","last_fullsized":"2024-04-23T14:36:31.257650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:02.099306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256853+0000","last_change":"2024-04-23T14:36:29.249521+0000","last_active":"2024-04-23T14:36:31.256853+0000","last_peered":"2024-04-23T14:36:31.256853+0000","last_clean":"2024-04-23T14:36:31.256853+0000","last_became_active":"2024-04-23T14:36:29.249249+0000","last_became_peered":"2024-04-23T14:36:29.249249+0000","last_unstale":"2024-04-23T14:36:31.256853+0000","last_undegraded":"2024-04-23T14:36:31.256853+0000","last_fullsized":"2024-04-23T14:36:31.256853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:08:47.916797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245640+0000","last_change":"2024-04-23T14:36:29.245640+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245640+0000","last_undegraded":"2024-04-23T14:36:29.245640+0000","last_fullsized":"2024-04-23T14:36:29.245640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264297+0000","last_change":"2024-04-23T14:36:29.262517+0000","last_active":"2024-04-23T14:36:29.264297+0000","last_peered":"2024-04-23T14:36:29.264297+0000","last_clean":"2024-04-23T14:36:29.264297+0000","last_became_active":"2024-04-23T14:36:29.262167+0000","last_became_peered":"2024-04-23T14:36:29.262167+0000","last_unstale":"2024-04-23T14:36:29.264297+0000","last_undegraded":"2024-04-23T14:36:29.264297+0000","last_fullsized":"2024-04-23T14:36:29.264297+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245791+0000","last_change":"2024-04-23T14:36:29.245791+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245791+0000","last_undegraded":"2024-04-23T14:36:29.245791+0000","last_fullsized":"2024-04-23T14:36:29.245791+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266259+0000","last_change":"2024-04-23T14:36:29.260899+0000","last_active":"2024-04-23T14:36:29.266259+0000","last_peered":"2024-04-23T14:36:29.266259+0000","last_clean":"2024-04-23T14:36:29.266259+0000","last_became_active":"2024-04-23T14:36:29.258766+0000","last_became_peered":"2024-04-23T14:36:29.258766+0000","last_unstale":"2024-04-23T14:36:29.266259+0000","last_undegraded":"2024-04-23T14:36:29.266259+0000","last_fullsized":"2024-04-23T14:36:29.266259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236208+0000","last_change":"2024-04-23T14:36:29.236208+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236208+0000","last_undegraded":"2024-04-23T14:36:29.236208+0000","last_fullsized":"2024-04-23T14:36:29.236208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264147+0000","last_change":"2024-04-23T14:36:29.261017+0000","last_active":"2024-04-23T14:36:29.264147+0000","last_peered":"2024-04-23T14:36:29.264147+0000","last_clean":"2024-04-23T14:36:29.264147+0000","last_became_active":"2024-04-23T14:36:29.258667+0000","last_became_peered":"2024-04-23T14:36:29.258667+0000","last_unstale":"2024-04-23T14:36:29.264147+0000","last_undegraded":"2024-04-23T14:36:29.264147+0000","last_fullsized":"2024-04-23T14:36:29.264147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:47.640350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.237471+0000","last_change":"2024-04-23T14:36:29.237471+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.237471+0000","last_undegraded":"2024-04-23T14:36:29.237471+0000","last_fullsized":"2024-04-23T14:36:29.237471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.261357+0000","last_change":"2024-04-23T14:36:29.258243+0000","last_active":"2024-04-23T14:36:29.261357+0000","last_peered":"2024-04-23T14:36:29.261357+0000","last_clean":"2024-04-23T14:36:29.261357+0000","last_became_active":"2024-04-23T14:36:29.257918+0000","last_became_peered":"2024-04-23T14:36:29.257918+0000","last_unstale":"2024-04-23T14:36:29.261357+0000","last_undegraded":"2024-04-23T14:36:29.261357+0000","last_fullsized":"2024-04-23T14:36:29.261357+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:01.081168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.258412+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:36:29.258412+0000","last_peered":"2024-04-23T14:36:29.258412+0000","last_clean":"2024-04-23T14:36:29.258412+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:36:29.258412+0000","last_undegraded":"2024-04-23T14:36:29.258412+0000","last_fullsized":"2024-04-23T14:36:29.258412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":924675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":334307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":334307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":851968,"data_stored":355269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":924675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:36:34.730 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:36:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.977+0000 7fbea0c00640 1 Processor -- start 2024-04-23T14:36:34.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.977+0000 7fbea0c00640 1 -- start start 2024-04-23T14:36:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbea0c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c15c3d0 0x7fbe9c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbea0c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbe9c0b4860 0x7fbe9c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbea0c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9c160210 con 0x7fbe9c0cd590 2024-04-23T14:36:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbea0c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9c0cd9f0 con 0x7fbe9c0b4860 2024-04-23T14:36:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbea0c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9c1604f0 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbe9be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbe9c0b4860 0x7fbe9c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbe9be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbe9c0b4860 0x7fbe9c0cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41458/0 (socket says 172.21.15.59:41458) 2024-04-23T14:36:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbe9be00640 1 -- 172.21.15.59:0/1651795356 learned_addr learned my addr 172.21.15.59:0/1651795356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbe9be00640 1 -- 172.21.15.59:0/1651795356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c15c3d0 msgr2=0x7fbe9c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbe9be00640 1 --2- 172.21.15.59:0/1651795356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c15c3d0 0x7fbe9c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbe9be00640 1 -- 172.21.15.59:0/1651795356 >> v1:172.21.15.59:6789/0 conn(0x7fbe9c0cd590 legacy=0x7fbe9c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.978+0000 7fbe9be00640 1 -- 172.21.15.59:0/1651795356 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1607d0 con 0x7fbe9c0b4860 2024-04-23T14:36:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.979+0000 7fbe9be00640 1 --2- 172.21.15.59:0/1651795356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c0b4860 0x7fbe9c0cb7d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009e50 tx=0x7fbe9402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a359780e01a21a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.979+0000 7fbe9aa00640 1 -- 172.21.15.59:0/1651795356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94036070 con 0x7fbe9c0b4860 2024-04-23T14:36:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.979+0000 7fbe9aa00640 1 -- 172.21.15.59:0/1651795356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe94031600 con 0x7fbe9c0b4860 2024-04-23T14:36:34.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.979+0000 7fbe9aa00640 1 -- 172.21.15.59:0/1651795356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940319d0 con 0x7fbe9c0b4860 2024-04-23T14:36:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 -- 172.21.15.59:0/1651795356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c0b4860 msgr2=0x7fbe9c0cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 --2- 172.21.15.59:0/1651795356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c0b4860 0x7fbe9c0cb7d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009e50 tx=0x7fbe9402a710 comp rx=0 tx=0).stop 2024-04-23T14:36:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 -- 172.21.15.59:0/1651795356 shutdown_connections 2024-04-23T14:36:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 --2- 172.21.15.59:0/1651795356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c0b4860 0x7fbe9c0cb7d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 --2- 172.21.15.59:0/1651795356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c15c3d0 0x7fbe9c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 -- 172.21.15.59:0/1651795356 >> 172.21.15.59:0/1651795356 conn(0x7fbe9c0b21a0 msgr2=0x7fbe9c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 -- 172.21.15.59:0/1651795356 shutdown_connections 2024-04-23T14:36:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.980+0000 7fbea0c00640 1 -- 172.21.15.59:0/1651795356 wait complete. 2024-04-23T14:36:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.982+0000 7fbea0c00640 1 Processor -- start 2024-04-23T14:36:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.982+0000 7fbea0c00640 1 -- start start 2024-04-23T14:36:34.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbea0c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c0b4860 0x7fbe9c0c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbea0c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe9c0cd590 0x7fbe9c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbea0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c15c3d0 0x7fbe9c1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbea0c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c15c7b0 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbea0c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c160d90 con 0x7fbe9c0cd590 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbea0c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9c15c930 con 0x7fbe9c0b4860 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbe9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c15c3d0 0x7fbe9c1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbe9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c15c3d0 0x7fbe9c1599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41474/0 (socket says 172.21.15.59:41474) 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.983+0000 7fbe9be00640 1 -- 172.21.15.59:0/3419148518 learned_addr learned my addr 172.21.15.59:0/3419148518 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe9b400640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c0b4860 0x7fbe9c0c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe9be00640 1 -- 172.21.15.59:0/3419148518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c0b4860 msgr2=0x7fbe9c0c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe9be00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c0b4860 0x7fbe9c0c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe9be00640 1 -- 172.21.15.59:0/3419148518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe9c0cd590 msgr2=0x7fbe9c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe9be00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe9c0cd590 0x7fbe9c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe9be00640 1 -- 172.21.15.59:0/3419148518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1f5b60 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe9be00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c15c3d0 0x7fbe9c1599c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009e50 tx=0x7fbe94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe93400640 1 -- 172.21.15.59:0/3419148518 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94036050 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe93400640 1 -- 172.21.15.59:0/3419148518 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe94042070 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c1f52c0 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.984+0000 7fbe93400640 1 -- 172.21.15.59:0/3419148518 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94009510 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.985+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1f5880 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.986+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c04fd90 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.987+0000 7fbe93400640 1 -- 172.21.15.59:0/3419148518 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe94007bd0 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.988+0000 7fbe93400640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbe7806c8f0 0x7fbe7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.988+0000 7fbe93400640 1 -- 172.21.15.59:0/3419148518 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fbe940acaf0 con 0x7fbe9c15c3d0 2024-04-23T14:36:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.989+0000 7fbe9b400640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbe7806c8f0 0x7fbe7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:34.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.989+0000 7fbe9b400640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbe7806c8f0 0x7fbe7806edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c15fd10 tx=0x7fbe8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:34.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:34.993+0000 7fbe93400640 1 -- 172.21.15.59:0/3419148518 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe9407af10 con 0x7fbe9c15c3d0 2024-04-23T14:36:35.169 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:36:35.168+0000 7ff4a7200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:36:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.195+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe9c073b00 con 0x7fbe7806c8f0 2024-04-23T14:36:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.213+0000 7fbe93400640 1 -- 172.21.15.59:0/3419148518 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247579 (secure 0 0 0) 0x7fbe9c073b00 con 0x7fbe7806c8f0 2024-04-23T14:36:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:36:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbe7806c8f0 msgr2=0x7fbe7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbe7806c8f0 0x7fbe7806edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c15fd10 tx=0x7fbe8c008040 comp rx=0 tx=0).stop 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c15c3d0 msgr2=0x7fbe9c1599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c15c3d0 0x7fbe9c1599c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009e50 tx=0x7fbe94004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 shutdown_connections 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbe7806c8f0 0x7fbe7806edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe9c15c3d0 0x7fbe9c1599c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe9c0cd590 0x7fbe9c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 --2- 172.21.15.59:0/3419148518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe9c0b4860 0x7fbe9c0c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 >> 172.21.15.59:0/3419148518 conn(0x7fbe9c0b21a0 msgr2=0x7fbe9c06e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 shutdown_connections 2024-04-23T14:36:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.219+0000 7fbea0c00640 1 -- 172.21.15.59:0/3419148518 wait complete. 2024-04-23T14:36:35.241 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-23T14:36:34.449081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256976,"kb_used_data":7516,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717744,"statfs":{"total":773094113280,"available":772830969856,"internally_reserved":0,"allocated":7696384,"data_stored":3894979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001351"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.257806+0000","last_change":"2024-04-23T14:36:29.257806+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.257806+0000","last_undegraded":"2024-04-23T14:36:29.257806+0000","last_fullsized":"2024-04-23T14:36:29.257806+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238437+0000","last_change":"2024-04-23T14:36:29.238437+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238437+0000","last_undegraded":"2024-04-23T14:36:29.238437+0000","last_fullsized":"2024-04-23T14:36:29.238437+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239371+0000","last_change":"2024-04-23T14:36:29.239371+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239371+0000","last_undegraded":"2024-04-23T14:36:29.239371+0000","last_fullsized":"2024-04-23T14:36:29.239371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.248757+0000","last_change":"2024-04-23T14:36:29.248757+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.248757+0000","last_undegraded":"2024-04-23T14:36:29.248757+0000","last_fullsized":"2024-04-23T14:36:29.248757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256769+0000","last_change":"2024-04-23T14:36:30.271347+0000","last_active":"2024-04-23T14:36:31.256769+0000","last_peered":"2024-04-23T14:36:31.256769+0000","last_clean":"2024-04-23T14:36:31.256769+0000","last_became_active":"2024-04-23T14:36:30.270634+0000","last_became_peered":"2024-04-23T14:36:30.270634+0000","last_unstale":"2024-04-23T14:36:31.256769+0000","last_undegraded":"2024-04-23T14:36:31.256769+0000","last_fullsized":"2024-04-23T14:36:31.256769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:26.664536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243660+0000","last_change":"2024-04-23T14:36:29.243660+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243660+0000","last_undegraded":"2024-04-23T14:36:29.243660+0000","last_fullsized":"2024-04-23T14:36:29.243660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239846+0000","last_change":"2024-04-23T14:36:29.239846+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239846+0000","last_undegraded":"2024-04-23T14:36:29.239846+0000","last_fullsized":"2024-04-23T14:36:29.239846+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.237516+0000","last_change":"2024-04-23T14:36:29.237516+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.237516+0000","last_undegraded":"2024-04-23T14:36:29.237516+0000","last_fullsized":"2024-04-23T14:36:29.237516+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257521+0000","last_change":"2024-04-23T14:36:30.256388+0000","last_active":"2024-04-23T14:36:31.257521+0000","last_peered":"2024-04-23T14:36:31.257521+0000","last_clean":"2024-04-23T14:36:31.257521+0000","last_became_active":"2024-04-23T14:36:30.255042+0000","last_became_peered":"2024-04-23T14:36:30.255042+0000","last_unstale":"2024-04-23T14:36:31.257521+0000","last_undegraded":"2024-04-23T14:36:31.257521+0000","last_fullsized":"2024-04-23T14:36:31.257521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:38.281474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243637+0000","last_change":"2024-04-23T14:36:29.243637+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243637+0000","last_undegraded":"2024-04-23T14:36:29.243637+0000","last_fullsized":"2024-04-23T14:36:29.243637+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.252268+0000","last_change":"2024-04-23T14:36:29.252268+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.252268+0000","last_undegraded":"2024-04-23T14:36:29.252268+0000","last_fullsized":"2024-04-23T14:36:29.252268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.248093+0000","last_change":"2024-04-23T14:36:29.248093+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.248093+0000","last_undegraded":"2024-04-23T14:36:29.248093+0000","last_fullsized":"2024-04-23T14:36:29.248093+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242843+0000","last_change":"2024-04-23T14:36:29.242843+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242843+0000","last_undegraded":"2024-04-23T14:36:29.242843+0000","last_fullsized":"2024-04-23T14:36:29.242843+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257406+0000","last_change":"2024-04-23T14:36:30.276001+0000","last_active":"2024-04-23T14:36:31.257406+0000","last_peered":"2024-04-23T14:36:31.257406+0000","last_clean":"2024-04-23T14:36:31.257406+0000","last_became_active":"2024-04-23T14:36:30.271406+0000","last_became_peered":"2024-04-23T14:36:30.271406+0000","last_unstale":"2024-04-23T14:36:31.257406+0000","last_undegraded":"2024-04-23T14:36:31.257406+0000","last_fullsized":"2024-04-23T14:36:31.257406+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:05.035180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242605+0000","last_change":"2024-04-23T14:36:29.242605+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242605+0000","last_undegraded":"2024-04-23T14:36:29.242605+0000","last_fullsized":"2024-04-23T14:36:29.242605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.246845+0000","last_change":"2024-04-23T14:36:29.246845+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.246845+0000","last_undegraded":"2024-04-23T14:36:29.246845+0000","last_fullsized":"2024-04-23T14:36:29.246845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257673+0000","last_change":"2024-04-23T14:36:30.257389+0000","last_active":"2024-04-23T14:36:31.257673+0000","last_peered":"2024-04-23T14:36:31.257673+0000","last_clean":"2024-04-23T14:36:31.257673+0000","last_became_active":"2024-04-23T14:36:30.256095+0000","last_became_peered":"2024-04-23T14:36:30.256095+0000","last_unstale":"2024-04-23T14:36:31.257673+0000","last_undegraded":"2024-04-23T14:36:31.257673+0000","last_fullsized":"2024-04-23T14:36:31.257673+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:48.658488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247241+0000","last_change":"2024-04-23T14:36:29.247241+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247241+0000","last_undegraded":"2024-04-23T14:36:29.247241+0000","last_fullsized":"2024-04-23T14:36:29.247241+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243996+0000","last_change":"2024-04-23T14:36:29.243996+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243996+0000","last_undegraded":"2024-04-23T14:36:29.243996+0000","last_fullsized":"2024-04-23T14:36:29.243996+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239476+0000","last_change":"2024-04-23T14:36:29.239476+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239476+0000","last_undegraded":"2024-04-23T14:36:29.239476+0000","last_fullsized":"2024-04-23T14:36:29.239476+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243017+0000","last_change":"2024-04-23T14:36:29.243017+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243017+0000","last_undegraded":"2024-04-23T14:36:29.243017+0000","last_fullsized":"2024-04-23T14:36:29.243017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247488+0000","last_change":"2024-04-23T14:36:29.247488+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247488+0000","last_undegraded":"2024-04-23T14:36:29.247488+0000","last_fullsized":"2024-04-23T14:36:29.247488+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.250249+0000","last_change":"2024-04-23T14:36:29.250249+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.250249+0000","last_undegraded":"2024-04-23T14:36:29.250249+0000","last_fullsized":"2024-04-23T14:36:29.250249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.254142+0000","last_change":"2024-04-23T14:36:29.254142+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.254142+0000","last_undegraded":"2024-04-23T14:36:29.254142+0000","last_fullsized":"2024-04-23T14:36:29.254142+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247037+0000","last_change":"2024-04-23T14:36:29.247037+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247037+0000","last_undegraded":"2024-04-23T14:36:29.247037+0000","last_fullsized":"2024-04-23T14:36:29.247037+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239768+0000","last_change":"2024-04-23T14:36:29.239768+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239768+0000","last_undegraded":"2024-04-23T14:36:29.239768+0000","last_fullsized":"2024-04-23T14:36:29.239768+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238732+0000","last_change":"2024-04-23T14:36:29.238732+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238732+0000","last_undegraded":"2024-04-23T14:36:29.238732+0000","last_fullsized":"2024-04-23T14:36:29.238732+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245148+0000","last_change":"2024-04-23T14:36:29.245148+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245148+0000","last_undegraded":"2024-04-23T14:36:29.245148+0000","last_fullsized":"2024-04-23T14:36:29.245148+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241311+0000","last_change":"2024-04-23T14:36:29.241311+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241311+0000","last_undegraded":"2024-04-23T14:36:29.241311+0000","last_fullsized":"2024-04-23T14:36:29.241311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238095+0000","last_change":"2024-04-23T14:36:29.238095+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238095+0000","last_undegraded":"2024-04-23T14:36:29.238095+0000","last_fullsized":"2024-04-23T14:36:29.238095+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235704+0000","last_change":"2024-04-23T14:36:29.235704+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235704+0000","last_undegraded":"2024-04-23T14:36:29.235704+0000","last_fullsized":"2024-04-23T14:36:29.235704+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235950+0000","last_change":"2024-04-23T14:36:29.235950+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235950+0000","last_undegraded":"2024-04-23T14:36:29.235950+0000","last_fullsized":"2024-04-23T14:36:29.235950+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251241+0000","last_change":"2024-04-23T14:36:29.251241+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251241+0000","last_undegraded":"2024-04-23T14:36:29.251241+0000","last_fullsized":"2024-04-23T14:36:29.251241+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266770+0000","last_change":"2024-04-23T14:36:29.265430+0000","last_active":"2024-04-23T14:36:29.266770+0000","last_peered":"2024-04-23T14:36:29.266770+0000","last_clean":"2024-04-23T14:36:29.266770+0000","last_became_active":"2024-04-23T14:36:29.265096+0000","last_became_peered":"2024-04-23T14:36:29.265096+0000","last_unstale":"2024-04-23T14:36:29.266770+0000","last_undegraded":"2024-04-23T14:36:29.266770+0000","last_fullsized":"2024-04-23T14:36:29.266770+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:29.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257718+0000","last_change":"2024-04-23T14:36:30.257027+0000","last_active":"2024-04-23T14:36:31.257718+0000","last_peered":"2024-04-23T14:36:31.257718+0000","last_clean":"2024-04-23T14:36:31.257718+0000","last_became_active":"2024-04-23T14:36:30.255514+0000","last_became_peered":"2024-04-23T14:36:30.255514+0000","last_unstale":"2024-04-23T14:36:31.257718+0000","last_undegraded":"2024-04-23T14:36:31.257718+0000","last_fullsized":"2024-04-23T14:36:31.257718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:21.165754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259920+0000","last_change":"2024-04-23T14:36:29.255962+0000","last_active":"2024-04-23T14:36:29.259920+0000","last_peered":"2024-04-23T14:36:29.259920+0000","last_clean":"2024-04-23T14:36:29.259920+0000","last_became_active":"2024-04-23T14:36:29.255566+0000","last_became_peered":"2024-04-23T14:36:29.255566+0000","last_unstale":"2024-04-23T14:36:29.259920+0000","last_undegraded":"2024-04-23T14:36:29.259920+0000","last_fullsized":"2024-04-23T14:36:29.259920+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:53.438649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245613+0000","last_change":"2024-04-23T14:36:29.245613+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245613+0000","last_undegraded":"2024-04-23T14:36:29.245613+0000","last_fullsized":"2024-04-23T14:36:29.245613+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.268721+0000","last_change":"2024-04-23T14:36:29.264940+0000","last_active":"2024-04-23T14:36:29.268721+0000","last_peered":"2024-04-23T14:36:29.268721+0000","last_clean":"2024-04-23T14:36:29.268721+0000","last_became_active":"2024-04-23T14:36:29.264568+0000","last_became_peered":"2024-04-23T14:36:29.264568+0000","last_unstale":"2024-04-23T14:36:29.268721+0000","last_undegraded":"2024-04-23T14:36:29.268721+0000","last_fullsized":"2024-04-23T14:36:29.268721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:56:10.745359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251760+0000","last_change":"2024-04-23T14:36:29.251760+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251760+0000","last_undegraded":"2024-04-23T14:36:29.251760+0000","last_fullsized":"2024-04-23T14:36:29.251760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260946+0000","last_change":"2024-04-23T14:36:29.258194+0000","last_active":"2024-04-23T14:36:29.260946+0000","last_peered":"2024-04-23T14:36:29.260946+0000","last_clean":"2024-04-23T14:36:29.260946+0000","last_became_active":"2024-04-23T14:36:29.257888+0000","last_became_peered":"2024-04-23T14:36:29.257888+0000","last_unstale":"2024-04-23T14:36:29.260946+0000","last_undegraded":"2024-04-23T14:36:29.260946+0000","last_fullsized":"2024-04-23T14:36:29.260946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:37.263336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235815+0000","last_change":"2024-04-23T14:36:29.235815+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235815+0000","last_undegraded":"2024-04-23T14:36:29.235815+0000","last_fullsized":"2024-04-23T14:36:29.235815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260424+0000","last_change":"2024-04-23T14:36:29.259191+0000","last_active":"2024-04-23T14:36:29.260424+0000","last_peered":"2024-04-23T14:36:29.260424+0000","last_clean":"2024-04-23T14:36:29.260424+0000","last_became_active":"2024-04-23T14:36:29.258420+0000","last_became_peered":"2024-04-23T14:36:29.258420+0000","last_unstale":"2024-04-23T14:36:29.260424+0000","last_undegraded":"2024-04-23T14:36:29.260424+0000","last_fullsized":"2024-04-23T14:36:29.260424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:49.177144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241971+0000","last_change":"2024-04-23T14:36:29.241971+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241971+0000","last_undegraded":"2024-04-23T14:36:29.241971+0000","last_fullsized":"2024-04-23T14:36:29.241971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260826+0000","last_change":"2024-04-23T14:36:29.255423+0000","last_active":"2024-04-23T14:36:29.260826+0000","last_peered":"2024-04-23T14:36:29.260826+0000","last_clean":"2024-04-23T14:36:29.260826+0000","last_became_active":"2024-04-23T14:36:29.255130+0000","last_became_peered":"2024-04-23T14:36:29.255130+0000","last_unstale":"2024-04-23T14:36:29.260826+0000","last_undegraded":"2024-04-23T14:36:29.260826+0000","last_fullsized":"2024-04-23T14:36:29.260826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:35.193457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256577+0000","last_change":"2024-04-23T14:36:30.259813+0000","last_active":"2024-04-23T14:36:31.256577+0000","last_peered":"2024-04-23T14:36:31.256577+0000","last_clean":"2024-04-23T14:36:31.256577+0000","last_became_active":"2024-04-23T14:36:30.259020+0000","last_became_peered":"2024-04-23T14:36:30.259020+0000","last_unstale":"2024-04-23T14:36:31.256577+0000","last_undegraded":"2024-04-23T14:36:31.256577+0000","last_fullsized":"2024-04-23T14:36:31.256577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:28.217244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264303+0000","last_change":"2024-04-23T14:36:29.262680+0000","last_active":"2024-04-23T14:36:29.264303+0000","last_peered":"2024-04-23T14:36:29.264303+0000","last_clean":"2024-04-23T14:36:29.264303+0000","last_became_active":"2024-04-23T14:36:29.262177+0000","last_became_peered":"2024-04-23T14:36:29.262177+0000","last_unstale":"2024-04-23T14:36:29.264303+0000","last_undegraded":"2024-04-23T14:36:29.264303+0000","last_fullsized":"2024-04-23T14:36:29.264303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242083+0000","last_change":"2024-04-23T14:36:29.242083+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242083+0000","last_undegraded":"2024-04-23T14:36:29.242083+0000","last_fullsized":"2024-04-23T14:36:29.242083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256559+0000","last_change":"2024-04-23T14:36:29.255068+0000","last_active":"2024-04-23T14:36:31.256559+0000","last_peered":"2024-04-23T14:36:31.256559+0000","last_clean":"2024-04-23T14:36:31.256559+0000","last_became_active":"2024-04-23T14:36:29.254465+0000","last_became_peered":"2024-04-23T14:36:29.254465+0000","last_unstale":"2024-04-23T14:36:31.256559+0000","last_undegraded":"2024-04-23T14:36:31.256559+0000","last_fullsized":"2024-04-23T14:36:31.256559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:47.763347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236904+0000","last_change":"2024-04-23T14:36:29.236904+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236904+0000","last_undegraded":"2024-04-23T14:36:29.236904+0000","last_fullsized":"2024-04-23T14:36:29.236904+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266420+0000","last_change":"2024-04-23T14:36:29.262461+0000","last_active":"2024-04-23T14:36:29.266420+0000","last_peered":"2024-04-23T14:36:29.266420+0000","last_clean":"2024-04-23T14:36:29.266420+0000","last_became_active":"2024-04-23T14:36:29.262134+0000","last_became_peered":"2024-04-23T14:36:29.262134+0000","last_unstale":"2024-04-23T14:36:29.266420+0000","last_undegraded":"2024-04-23T14:36:29.266420+0000","last_fullsized":"2024-04-23T14:36:29.266420+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.255161+0000","last_change":"2024-04-23T14:36:29.255161+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.255161+0000","last_undegraded":"2024-04-23T14:36:29.255161+0000","last_fullsized":"2024-04-23T14:36:29.255161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260641+0000","last_change":"2024-04-23T14:36:29.252722+0000","last_active":"2024-04-23T14:36:29.260641+0000","last_peered":"2024-04-23T14:36:29.260641+0000","last_clean":"2024-04-23T14:36:29.260641+0000","last_became_active":"2024-04-23T14:36:29.252421+0000","last_became_peered":"2024-04-23T14:36:29.252421+0000","last_unstale":"2024-04-23T14:36:29.260641+0000","last_undegraded":"2024-04-23T14:36:29.260641+0000","last_fullsized":"2024-04-23T14:36:29.260641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:27.199106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243002+0000","last_change":"2024-04-23T14:36:29.243002+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243002+0000","last_undegraded":"2024-04-23T14:36:29.243002+0000","last_fullsized":"2024-04-23T14:36:29.243002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.258545+0000","last_change":"2024-04-23T14:36:29.255338+0000","last_active":"2024-04-23T14:36:29.258545+0000","last_peered":"2024-04-23T14:36:29.258545+0000","last_clean":"2024-04-23T14:36:29.258545+0000","last_became_active":"2024-04-23T14:36:29.254916+0000","last_became_peered":"2024-04-23T14:36:29.254916+0000","last_unstale":"2024-04-23T14:36:29.258545+0000","last_undegraded":"2024-04-23T14:36:29.258545+0000","last_fullsized":"2024-04-23T14:36:29.258545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:39.168711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257203+0000","last_change":"2024-04-23T14:36:30.267547+0000","last_active":"2024-04-23T14:36:31.257203+0000","last_peered":"2024-04-23T14:36:31.257203+0000","last_clean":"2024-04-23T14:36:31.257203+0000","last_became_active":"2024-04-23T14:36:30.267103+0000","last_became_peered":"2024-04-23T14:36:30.267103+0000","last_unstale":"2024-04-23T14:36:31.257203+0000","last_undegraded":"2024-04-23T14:36:31.257203+0000","last_fullsized":"2024-04-23T14:36:31.257203+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:14.787860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260627+0000","last_change":"2024-04-23T14:36:29.257996+0000","last_active":"2024-04-23T14:36:29.260627+0000","last_peered":"2024-04-23T14:36:29.260627+0000","last_clean":"2024-04-23T14:36:29.260627+0000","last_became_active":"2024-04-23T14:36:29.256591+0000","last_became_peered":"2024-04-23T14:36:29.256591+0000","last_unstale":"2024-04-23T14:36:29.260627+0000","last_undegraded":"2024-04-23T14:36:29.260627+0000","last_fullsized":"2024-04-23T14:36:29.260627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:47.640350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.249860+0000","last_change":"2024-04-23T14:36:29.249860+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.249860+0000","last_undegraded":"2024-04-23T14:36:29.249860+0000","last_fullsized":"2024-04-23T14:36:29.249860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264329+0000","last_change":"2024-04-23T14:36:29.262784+0000","last_active":"2024-04-23T14:36:29.264329+0000","last_peered":"2024-04-23T14:36:29.264329+0000","last_clean":"2024-04-23T14:36:29.264329+0000","last_became_active":"2024-04-23T14:36:29.262418+0000","last_became_peered":"2024-04-23T14:36:29.262418+0000","last_unstale":"2024-04-23T14:36:29.264329+0000","last_undegraded":"2024-04-23T14:36:29.264329+0000","last_fullsized":"2024-04-23T14:36:29.264329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238374+0000","last_change":"2024-04-23T14:36:29.238374+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238374+0000","last_undegraded":"2024-04-23T14:36:29.238374+0000","last_fullsized":"2024-04-23T14:36:29.238374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260842+0000","last_change":"2024-04-23T14:36:29.256582+0000","last_active":"2024-04-23T14:36:29.260842+0000","last_peered":"2024-04-23T14:36:29.260842+0000","last_clean":"2024-04-23T14:36:29.260842+0000","last_became_active":"2024-04-23T14:36:29.256299+0000","last_became_peered":"2024-04-23T14:36:29.256299+0000","last_unstale":"2024-04-23T14:36:29.260842+0000","last_undegraded":"2024-04-23T14:36:29.260842+0000","last_fullsized":"2024-04-23T14:36:29.260842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:04.017042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236627+0000","last_change":"2024-04-23T14:36:29.236627+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236627+0000","last_undegraded":"2024-04-23T14:36:29.236627+0000","last_fullsized":"2024-04-23T14:36:29.236627+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259444+0000","last_change":"2024-04-23T14:36:29.254735+0000","last_active":"2024-04-23T14:36:29.259444+0000","last_peered":"2024-04-23T14:36:29.259444+0000","last_clean":"2024-04-23T14:36:29.259444+0000","last_became_active":"2024-04-23T14:36:29.253176+0000","last_became_peered":"2024-04-23T14:36:29.253176+0000","last_unstale":"2024-04-23T14:36:29.259444+0000","last_undegraded":"2024-04-23T14:36:29.259444+0000","last_fullsized":"2024-04-23T14:36:29.259444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:45.728544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256827+0000","last_change":"2024-04-23T14:36:30.254759+0000","last_active":"2024-04-23T14:36:31.256827+0000","last_peered":"2024-04-23T14:36:31.256827+0000","last_clean":"2024-04-23T14:36:31.256827+0000","last_became_active":"2024-04-23T14:36:30.254477+0000","last_became_peered":"2024-04-23T14:36:30.254477+0000","last_unstale":"2024-04-23T14:36:31.256827+0000","last_undegraded":"2024-04-23T14:36:31.256827+0000","last_fullsized":"2024-04-23T14:36:31.256827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:15.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266716+0000","last_change":"2024-04-23T14:36:29.263312+0000","last_active":"2024-04-23T14:36:29.266716+0000","last_peered":"2024-04-23T14:36:29.266716+0000","last_clean":"2024-04-23T14:36:29.266716+0000","last_became_active":"2024-04-23T14:36:29.262939+0000","last_became_peered":"2024-04-23T14:36:29.262939+0000","last_unstale":"2024-04-23T14:36:29.266716+0000","last_undegraded":"2024-04-23T14:36:29.266716+0000","last_fullsized":"2024-04-23T14:36:29.266716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:18:16.720971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242509+0000","last_change":"2024-04-23T14:36:29.242509+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242509+0000","last_undegraded":"2024-04-23T14:36:29.242509+0000","last_fullsized":"2024-04-23T14:36:29.242509+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259691+0000","last_change":"2024-04-23T14:36:29.251466+0000","last_active":"2024-04-23T14:36:29.259691+0000","last_peered":"2024-04-23T14:36:29.259691+0000","last_clean":"2024-04-23T14:36:29.259691+0000","last_became_active":"2024-04-23T14:36:29.250985+0000","last_became_peered":"2024-04-23T14:36:29.250985+0000","last_unstale":"2024-04-23T14:36:29.259691+0000","last_undegraded":"2024-04-23T14:36:29.259691+0000","last_fullsized":"2024-04-23T14:36:29.259691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:14.802229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.254476+0000","last_change":"2024-04-23T14:36:29.254476+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.254476+0000","last_undegraded":"2024-04-23T14:36:29.254476+0000","last_fullsized":"2024-04-23T14:36:29.254476+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264202+0000","last_change":"2024-04-23T14:36:29.257682+0000","last_active":"2024-04-23T14:36:29.264202+0000","last_peered":"2024-04-23T14:36:29.264202+0000","last_clean":"2024-04-23T14:36:29.264202+0000","last_became_active":"2024-04-23T14:36:29.257385+0000","last_became_peered":"2024-04-23T14:36:29.257385+0000","last_unstale":"2024-04-23T14:36:29.264202+0000","last_undegraded":"2024-04-23T14:36:29.264202+0000","last_fullsized":"2024-04-23T14:36:29.264202+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247591+0000","last_change":"2024-04-23T14:36:29.247591+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247591+0000","last_undegraded":"2024-04-23T14:36:29.247591+0000","last_fullsized":"2024-04-23T14:36:29.247591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264676+0000","last_change":"2024-04-23T14:36:29.262796+0000","last_active":"2024-04-23T14:36:29.264676+0000","last_peered":"2024-04-23T14:36:29.264676+0000","last_clean":"2024-04-23T14:36:29.264676+0000","last_became_active":"2024-04-23T14:36:29.262504+0000","last_became_peered":"2024-04-23T14:36:29.262504+0000","last_unstale":"2024-04-23T14:36:29.264676+0000","last_undegraded":"2024-04-23T14:36:29.264676+0000","last_fullsized":"2024-04-23T14:36:29.264676+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257317+0000","last_change":"2024-04-23T14:36:30.260184+0000","last_active":"2024-04-23T14:36:31.257317+0000","last_peered":"2024-04-23T14:36:31.257317+0000","last_clean":"2024-04-23T14:36:31.257317+0000","last_became_active":"2024-04-23T14:36:30.259450+0000","last_became_peered":"2024-04-23T14:36:30.259450+0000","last_unstale":"2024-04-23T14:36:31.257317+0000","last_undegraded":"2024-04-23T14:36:31.257317+0000","last_fullsized":"2024-04-23T14:36:31.257317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:36.211595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.269267+0000","last_change":"2024-04-23T14:36:29.267357+0000","last_active":"2024-04-23T14:36:29.269267+0000","last_peered":"2024-04-23T14:36:29.269267+0000","last_clean":"2024-04-23T14:36:29.269267+0000","last_became_active":"2024-04-23T14:36:29.266992+0000","last_became_peered":"2024-04-23T14:36:29.266992+0000","last_unstale":"2024-04-23T14:36:29.269267+0000","last_undegraded":"2024-04-23T14:36:29.269267+0000","last_fullsized":"2024-04-23T14:36:29.269267+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:45.728544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.240681+0000","last_change":"2024-04-23T14:36:29.240681+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.240681+0000","last_undegraded":"2024-04-23T14:36:29.240681+0000","last_fullsized":"2024-04-23T14:36:29.240681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260804+0000","last_change":"2024-04-23T14:36:29.254896+0000","last_active":"2024-04-23T14:36:29.260804+0000","last_peered":"2024-04-23T14:36:29.260804+0000","last_clean":"2024-04-23T14:36:29.260804+0000","last_became_active":"2024-04-23T14:36:29.254593+0000","last_became_peered":"2024-04-23T14:36:29.254593+0000","last_unstale":"2024-04-23T14:36:29.260804+0000","last_undegraded":"2024-04-23T14:36:29.260804+0000","last_fullsized":"2024-04-23T14:36:29.260804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:13.769723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241152+0000","last_change":"2024-04-23T14:36:29.241152+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241152+0000","last_undegraded":"2024-04-23T14:36:29.241152+0000","last_fullsized":"2024-04-23T14:36:29.241152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260903+0000","last_change":"2024-04-23T14:36:29.259253+0000","last_active":"2024-04-23T14:36:29.260903+0000","last_peered":"2024-04-23T14:36:29.260903+0000","last_clean":"2024-04-23T14:36:29.260903+0000","last_became_active":"2024-04-23T14:36:29.256871+0000","last_became_peered":"2024-04-23T14:36:29.256871+0000","last_unstale":"2024-04-23T14:36:29.260903+0000","last_undegraded":"2024-04-23T14:36:29.260903+0000","last_fullsized":"2024-04-23T14:36:29.260903+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:33.990830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238310+0000","last_change":"2024-04-23T14:36:29.238310+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238310+0000","last_undegraded":"2024-04-23T14:36:29.238310+0000","last_fullsized":"2024-04-23T14:36:29.238310+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.254977+0000","last_change":"2024-04-23T14:36:29.253692+0000","last_active":"2024-04-23T14:36:29.254977+0000","last_peered":"2024-04-23T14:36:29.254977+0000","last_clean":"2024-04-23T14:36:29.254977+0000","last_became_active":"2024-04-23T14:36:29.253295+0000","last_became_peered":"2024-04-23T14:36:29.253295+0000","last_unstale":"2024-04-23T14:36:29.254977+0000","last_undegraded":"2024-04-23T14:36:29.254977+0000","last_fullsized":"2024-04-23T14:36:29.254977+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:18.617182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.240864+0000","last_change":"2024-04-23T14:36:29.240864+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.240864+0000","last_undegraded":"2024-04-23T14:36:29.240864+0000","last_fullsized":"2024-04-23T14:36:29.240864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.253828+0000","last_change":"2024-04-23T14:36:29.249580+0000","last_active":"2024-04-23T14:36:29.253828+0000","last_peered":"2024-04-23T14:36:29.253828+0000","last_clean":"2024-04-23T14:36:29.253828+0000","last_became_active":"2024-04-23T14:36:29.249248+0000","last_became_peered":"2024-04-23T14:36:29.249248+0000","last_unstale":"2024-04-23T14:36:29.253828+0000","last_undegraded":"2024-04-23T14:36:29.253828+0000","last_fullsized":"2024-04-23T14:36:29.253828+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:25.646399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239729+0000","last_change":"2024-04-23T14:36:29.239729+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239729+0000","last_undegraded":"2024-04-23T14:36:29.239729+0000","last_fullsized":"2024-04-23T14:36:29.239729+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259327+0000","last_change":"2024-04-23T14:36:29.253372+0000","last_active":"2024-04-23T14:36:29.259327+0000","last_peered":"2024-04-23T14:36:29.259327+0000","last_clean":"2024-04-23T14:36:29.259327+0000","last_became_active":"2024-04-23T14:36:29.251760+0000","last_became_peered":"2024-04-23T14:36:29.251760+0000","last_unstale":"2024-04-23T14:36:29.259327+0000","last_undegraded":"2024-04-23T14:36:29.259327+0000","last_fullsized":"2024-04-23T14:36:29.259327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:29.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251080+0000","last_change":"2024-04-23T14:36:29.251080+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251080+0000","last_undegraded":"2024-04-23T14:36:29.251080+0000","last_fullsized":"2024-04-23T14:36:29.251080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257414+0000","last_change":"2024-04-23T14:36:29.245427+0000","last_active":"2024-04-23T14:36:31.257414+0000","last_peered":"2024-04-23T14:36:31.257414+0000","last_clean":"2024-04-23T14:36:31.257414+0000","last_became_active":"2024-04-23T14:36:29.245034+0000","last_became_peered":"2024-04-23T14:36:29.245034+0000","last_unstale":"2024-04-23T14:36:31.257414+0000","last_undegraded":"2024-04-23T14:36:31.257414+0000","last_fullsized":"2024-04-23T14:36:31.257414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:13.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.249579+0000","last_change":"2024-04-23T14:36:29.249579+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.249579+0000","last_undegraded":"2024-04-23T14:36:29.249579+0000","last_fullsized":"2024-04-23T14:36:29.249579+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259982+0000","last_change":"2024-04-23T14:36:29.257966+0000","last_active":"2024-04-23T14:36:29.259982+0000","last_peered":"2024-04-23T14:36:29.259982+0000","last_clean":"2024-04-23T14:36:29.259982+0000","last_became_active":"2024-04-23T14:36:29.257687+0000","last_became_peered":"2024-04-23T14:36:29.257687+0000","last_unstale":"2024-04-23T14:36:29.259982+0000","last_undegraded":"2024-04-23T14:36:29.259982+0000","last_fullsized":"2024-04-23T14:36:29.259982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:43:30.776694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257650+0000","last_change":"2024-04-23T14:36:30.260028+0000","last_active":"2024-04-23T14:36:31.257650+0000","last_peered":"2024-04-23T14:36:31.257650+0000","last_clean":"2024-04-23T14:36:31.257650+0000","last_became_active":"2024-04-23T14:36:30.259713+0000","last_became_peered":"2024-04-23T14:36:30.259713+0000","last_unstale":"2024-04-23T14:36:31.257650+0000","last_undegraded":"2024-04-23T14:36:31.257650+0000","last_fullsized":"2024-04-23T14:36:31.257650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:02.099306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256853+0000","last_change":"2024-04-23T14:36:29.249521+0000","last_active":"2024-04-23T14:36:31.256853+0000","last_peered":"2024-04-23T14:36:31.256853+0000","last_clean":"2024-04-23T14:36:31.256853+0000","last_became_active":"2024-04-23T14:36:29.249249+0000","last_became_peered":"2024-04-23T14:36:29.249249+0000","last_unstale":"2024-04-23T14:36:31.256853+0000","last_undegraded":"2024-04-23T14:36:31.256853+0000","last_fullsized":"2024-04-23T14:36:31.256853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:08:47.916797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245640+0000","last_change":"2024-04-23T14:36:29.245640+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245640+0000","last_undegraded":"2024-04-23T14:36:29.245640+0000","last_fullsized":"2024-04-23T14:36:29.245640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264297+0000","last_change":"2024-04-23T14:36:29.262517+0000","last_active":"2024-04-23T14:36:29.264297+0000","last_peered":"2024-04-23T14:36:29.264297+0000","last_clean":"2024-04-23T14:36:29.264297+0000","last_became_active":"2024-04-23T14:36:29.262167+0000","last_became_peered":"2024-04-23T14:36:29.262167+0000","last_unstale":"2024-04-23T14:36:29.264297+0000","last_undegraded":"2024-04-23T14:36:29.264297+0000","last_fullsized":"2024-04-23T14:36:29.264297+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245791+0000","last_change":"2024-04-23T14:36:29.245791+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245791+0000","last_undegraded":"2024-04-23T14:36:29.245791+0000","last_fullsized":"2024-04-23T14:36:29.245791+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266259+0000","last_change":"2024-04-23T14:36:29.260899+0000","last_active":"2024-04-23T14:36:29.266259+0000","last_peered":"2024-04-23T14:36:29.266259+0000","last_clean":"2024-04-23T14:36:29.266259+0000","last_became_active":"2024-04-23T14:36:29.258766+0000","last_became_peered":"2024-04-23T14:36:29.258766+0000","last_unstale":"2024-04-23T14:36:29.266259+0000","last_undegraded":"2024-04-23T14:36:29.266259+0000","last_fullsized":"2024-04-23T14:36:29.266259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236208+0000","last_change":"2024-04-23T14:36:29.236208+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236208+0000","last_undegraded":"2024-04-23T14:36:29.236208+0000","last_fullsized":"2024-04-23T14:36:29.236208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264147+0000","last_change":"2024-04-23T14:36:29.261017+0000","last_active":"2024-04-23T14:36:29.264147+0000","last_peered":"2024-04-23T14:36:29.264147+0000","last_clean":"2024-04-23T14:36:29.264147+0000","last_became_active":"2024-04-23T14:36:29.258667+0000","last_became_peered":"2024-04-23T14:36:29.258667+0000","last_unstale":"2024-04-23T14:36:29.264147+0000","last_undegraded":"2024-04-23T14:36:29.264147+0000","last_fullsized":"2024-04-23T14:36:29.264147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:47.640350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.237471+0000","last_change":"2024-04-23T14:36:29.237471+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.237471+0000","last_undegraded":"2024-04-23T14:36:29.237471+0000","last_fullsized":"2024-04-23T14:36:29.237471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.261357+0000","last_change":"2024-04-23T14:36:29.258243+0000","last_active":"2024-04-23T14:36:29.261357+0000","last_peered":"2024-04-23T14:36:29.261357+0000","last_clean":"2024-04-23T14:36:29.261357+0000","last_became_active":"2024-04-23T14:36:29.257918+0000","last_became_peered":"2024-04-23T14:36:29.257918+0000","last_unstale":"2024-04-23T14:36:29.261357+0000","last_undegraded":"2024-04-23T14:36:29.261357+0000","last_fullsized":"2024-04-23T14:36:29.261357+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:01.081168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.258412+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:36:29.258412+0000","last_peered":"2024-04-23T14:36:29.258412+0000","last_clean":"2024-04-23T14:36:29.258412+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:36:29.258412+0000","last_undegraded":"2024-04-23T14:36:29.258412+0000","last_fullsized":"2024-04-23T14:36:29.258412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":924675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":334307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":334307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":851968,"data_stored":355269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":924675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:36:35.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:36:35.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.492+0000 7f8f19c00640 1 Processor -- start 2024-04-23T14:36:35.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.492+0000 7f8f19c00640 1 -- start start 2024-04-23T14:36:35.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.493+0000 7f8f19c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f1415e750 0x7f8f140ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.493+0000 7f8f19c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140b8fc0 con 0x7f8f140cfbf0 2024-04-23T14:36:35.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.493+0000 7f8f19c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140b92a0 con 0x7f8f1415eb30 2024-04-23T14:36:35.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.493+0000 7f8f19c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8f140b9580 con 0x7f8f1415e750 2024-04-23T14:36:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.494+0000 7f8f13400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8f140cfbf0 0x7f8f140b7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35194/0 (socket says 172.21.15.59:35194) 2024-04-23T14:36:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.494+0000 7f8f13400640 1 -- 172.21.15.59:0/3654827861 learned_addr learned my addr 172.21.15.59:0/3654827861 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.494+0000 7f8f12a00640 1 --2- 172.21.15.59:0/3654827861 >> v2:172.21.15.76:3300/0 conn(0x7f8f1415e750 0x7f8f140ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.494+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2512229059 0 0) 0x7f8f140b8fc0 con 0x7f8f140cfbf0 2024-04-23T14:36:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.494+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ef8003520 con 0x7f8f140cfbf0 2024-04-23T14:36:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3361459275 0 0) 0x7f8ef8003520 con 0x7f8f140cfbf0 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 >> v2:172.21.15.76:3300/0 conn(0x7f8f1415e750 msgr2=0x7f8f140ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 --2- 172.21.15.59:0/3654827861 >> v2:172.21.15.76:3300/0 conn(0x7f8f1415e750 0x7f8f140ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 >> v1:172.21.15.76:6789/0 conn(0x7f8f1415eb30 legacy=0x7f8f140cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f140b9860 con 0x7f8f140cfbf0 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8f0c0019f0 con 0x7f8f140cfbf0 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 >> v1:172.21.15.59:6789/0 conn(0x7f8f140cfbf0 legacy=0x7f8f140b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef80039b0 0x7f8ef8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef8004800 0x7f8ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80051c0 0x7f8ef800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140b9580 con 0x7f8ef80039b0 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef800c060 con 0x7f8ef8004800 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef800c260 con 0x7f8ef80051c0 2024-04-23T14:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.495+0000 7f8f12a00640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef80039b0 0x7f8ef8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.496+0000 7f8f12000640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef8004800 0x7f8ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.496+0000 7f8f12a00640 1 -- 172.21.15.59:0/3654827861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80051c0 msgr2=0x7f8ef800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.496+0000 7f8f12a00640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80051c0 0x7f8ef800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.496+0000 7f8f12a00640 1 -- 172.21.15.59:0/3654827861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef8004800 msgr2=0x7f8ef8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.496+0000 7f8f12a00640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef8004800 0x7f8ef8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.496+0000 7f8f12a00640 1 -- 172.21.15.59:0/3654827861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef8003520 con 0x7f8ef80039b0 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.496+0000 7f8f13400640 1 -- 172.21.15.59:0/3654827861 reap_dead start 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.497+0000 7f8f12a00640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef80039b0 0x7f8ef8003dd0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009ac0 tx=0x7f8ef8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea6659cc21979f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.497+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04036070 con 0x7f8ef80039b0 2024-04-23T14:36:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.497+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f04031400 con 0x7f8ef80039b0 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.497+0000 7f8f11600640 1 -- 172.21.15.59:0/3654827861 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04009690 con 0x7f8ef80039b0 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.498+0000 7f8f19c00640 1 -- 172.21.15.59:0/3654827861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef80039b0 msgr2=0x7f8ef8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.498+0000 7f8f19c00640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef80039b0 0x7f8ef8003dd0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009ac0 tx=0x7f8ef8005830 comp rx=0 tx=0).stop 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.498+0000 7f8f19c00640 1 -- 172.21.15.59:0/3654827861 shutdown_connections 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.498+0000 7f8f19c00640 1 --2- 172.21.15.59:0/3654827861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef80039b0 0x7f8ef8003dd0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.498+0000 7f8f19c00640 1 -- 172.21.15.59:0/3654827861 >> 172.21.15.59:0/3654827861 conn(0x7f8f14067b50 msgr2=0x7f8f14067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.498+0000 7f8f19c00640 1 -- 172.21.15.59:0/3654827861 shutdown_connections 2024-04-23T14:36:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.498+0000 7f8f19c00640 1 -- 172.21.15.59:0/3654827861 wait complete. 2024-04-23T14:36:35.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.501+0000 7f8f19c00640 1 Processor -- start 2024-04-23T14:36:35.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.501+0000 7f8f19c00640 1 -- start start 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f19c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80039b0 0x7f8f141f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f19c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f140c8980 0x7f8f14160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f19c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f14159450 0x7f8f141598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f19c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140cf660 con 0x7f8f14159450 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f19c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140cf4e0 con 0x7f8f140c8980 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f19c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f140cf7e0 con 0x7f8ef80039b0 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f14159450 0x7f8f141598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f14159450 0x7f8f141598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41480/0 (socket says 172.21.15.59:41480) 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 -- 172.21.15.59:0/1932337156 learned_addr learned my addr 172.21.15.59:0/1932337156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f12a00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80039b0 0x7f8f141f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f12000640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f140c8980 0x7f8f14160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 -- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80039b0 msgr2=0x7f8f141f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80039b0 0x7f8f141f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 -- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f140c8980 msgr2=0x7f8f14160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f140c8980 0x7f8f14160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 -- 172.21.15.59:0/1932337156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f14160b00 con 0x7f8f14159450 2024-04-23T14:36:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f12a00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80039b0 0x7f8f141f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f12000640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f140c8980 0x7f8f14160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f13400640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f14159450 0x7f8f141598e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8f140d00e0 tx=0x7f8f0c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f0aa00640 1 -- 172.21.15.59:0/1932337156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00ce50 con 0x7f8f14159450 2024-04-23T14:36:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f0aa00640 1 -- 172.21.15.59:0/1932337156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c00a640 con 0x7f8f14159450 2024-04-23T14:36:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f14160de0 con 0x7f8f14159450 2024-04-23T14:36:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.502+0000 7f8f0aa00640 1 -- 172.21.15.59:0/1932337156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00aa30 con 0x7f8f14159450 2024-04-23T14:36:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.503+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141613a0 con 0x7f8f14159450 2024-04-23T14:36:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.505+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f14068a80 con 0x7f8f14159450 2024-04-23T14:36:35.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.506+0000 7f8f0aa00640 1 -- 172.21.15.59:0/1932337156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f0c00abd0 con 0x7f8f14159450 2024-04-23T14:36:35.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.507+0000 7f8f0aa00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c920 0x7f8ef406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:35.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.507+0000 7f8f0aa00640 1 -- 172.21.15.59:0/1932337156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f8f0c08b400 con 0x7f8f14159450 2024-04-23T14:36:35.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.507+0000 7f8f12a00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c920 0x7f8ef406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:35.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.508+0000 7f8f12a00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c920 0x7f8ef406ede0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8f04033000 tx=0x7f8f04008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.511+0000 7f8f0aa00640 1 -- 172.21.15.59:0/1932337156 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f0c059850 con 0x7f8f14159450 2024-04-23T14:36:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.716+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f140682d0 con 0x7f8ef406c920 2024-04-23T14:36:35.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.733+0000 7f8f0aa00640 1 -- 172.21.15.59:0/1932337156 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247579 (secure 0 0 0) 0x7f8f140682d0 con 0x7f8ef406c920 2024-04-23T14:36:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:36:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c920 msgr2=0x7f8ef406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c920 0x7f8ef406ede0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8f04033000 tx=0x7f8f04008030 comp rx=0 tx=0).stop 2024-04-23T14:36:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f14159450 msgr2=0x7f8f141598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f14159450 0x7f8f141598e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8f140d00e0 tx=0x7f8f0c00cc00 comp rx=0 tx=0).stop 2024-04-23T14:36:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 shutdown_connections 2024-04-23T14:36:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8ef406c920 0x7f8ef406ede0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f14159450 0x7f8f141598e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f140c8980 0x7f8f14160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 --2- 172.21.15.59:0/1932337156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ef80039b0 0x7f8f141f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 >> 172.21.15.59:0/1932337156 conn(0x7f8f14067b50 msgr2=0x7f8f1415d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 shutdown_connections 2024-04-23T14:36:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:35.740+0000 7f8f19c00640 1 -- 172.21.15.59:0/1932337156 wait complete. 2024-04-23T14:36:35.760 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-23T14:36:34.449081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256976,"kb_used_data":7516,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717744,"statfs":{"total":773094113280,"available":772830969856,"internally_reserved":0,"allocated":7696384,"data_stored":3894979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001351"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.257806+0000","last_change":"2024-04-23T14:36:29.257806+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.257806+0000","last_undegraded":"2024-04-23T14:36:29.257806+0000","last_fullsized":"2024-04-23T14:36:29.257806+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238437+0000","last_change":"2024-04-23T14:36:29.238437+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238437+0000","last_undegraded":"2024-04-23T14:36:29.238437+0000","last_fullsized":"2024-04-23T14:36:29.238437+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239371+0000","last_change":"2024-04-23T14:36:29.239371+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239371+0000","last_undegraded":"2024-04-23T14:36:29.239371+0000","last_fullsized":"2024-04-23T14:36:29.239371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.248757+0000","last_change":"2024-04-23T14:36:29.248757+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.248757+0000","last_undegraded":"2024-04-23T14:36:29.248757+0000","last_fullsized":"2024-04-23T14:36:29.248757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256769+0000","last_change":"2024-04-23T14:36:30.271347+0000","last_active":"2024-04-23T14:36:31.256769+0000","last_peered":"2024-04-23T14:36:31.256769+0000","last_clean":"2024-04-23T14:36:31.256769+0000","last_became_active":"2024-04-23T14:36:30.270634+0000","last_became_peered":"2024-04-23T14:36:30.270634+0000","last_unstale":"2024-04-23T14:36:31.256769+0000","last_undegraded":"2024-04-23T14:36:31.256769+0000","last_fullsized":"2024-04-23T14:36:31.256769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:26.664536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243660+0000","last_change":"2024-04-23T14:36:29.243660+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243660+0000","last_undegraded":"2024-04-23T14:36:29.243660+0000","last_fullsized":"2024-04-23T14:36:29.243660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239846+0000","last_change":"2024-04-23T14:36:29.239846+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239846+0000","last_undegraded":"2024-04-23T14:36:29.239846+0000","last_fullsized":"2024-04-23T14:36:29.239846+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.237516+0000","last_change":"2024-04-23T14:36:29.237516+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.237516+0000","last_undegraded":"2024-04-23T14:36:29.237516+0000","last_fullsized":"2024-04-23T14:36:29.237516+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257521+0000","last_change":"2024-04-23T14:36:30.256388+0000","last_active":"2024-04-23T14:36:31.257521+0000","last_peered":"2024-04-23T14:36:31.257521+0000","last_clean":"2024-04-23T14:36:31.257521+0000","last_became_active":"2024-04-23T14:36:30.255042+0000","last_became_peered":"2024-04-23T14:36:30.255042+0000","last_unstale":"2024-04-23T14:36:31.257521+0000","last_undegraded":"2024-04-23T14:36:31.257521+0000","last_fullsized":"2024-04-23T14:36:31.257521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:38.281474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243637+0000","last_change":"2024-04-23T14:36:29.243637+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243637+0000","last_undegraded":"2024-04-23T14:36:29.243637+0000","last_fullsized":"2024-04-23T14:36:29.243637+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.252268+0000","last_change":"2024-04-23T14:36:29.252268+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.252268+0000","last_undegraded":"2024-04-23T14:36:29.252268+0000","last_fullsized":"2024-04-23T14:36:29.252268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.248093+0000","last_change":"2024-04-23T14:36:29.248093+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.248093+0000","last_undegraded":"2024-04-23T14:36:29.248093+0000","last_fullsized":"2024-04-23T14:36:29.248093+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242843+0000","last_change":"2024-04-23T14:36:29.242843+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242843+0000","last_undegraded":"2024-04-23T14:36:29.242843+0000","last_fullsized":"2024-04-23T14:36:29.242843+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257406+0000","last_change":"2024-04-23T14:36:30.276001+0000","last_active":"2024-04-23T14:36:31.257406+0000","last_peered":"2024-04-23T14:36:31.257406+0000","last_clean":"2024-04-23T14:36:31.257406+0000","last_became_active":"2024-04-23T14:36:30.271406+0000","last_became_peered":"2024-04-23T14:36:30.271406+0000","last_unstale":"2024-04-23T14:36:31.257406+0000","last_undegraded":"2024-04-23T14:36:31.257406+0000","last_fullsized":"2024-04-23T14:36:31.257406+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:05.035180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242605+0000","last_change":"2024-04-23T14:36:29.242605+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242605+0000","last_undegraded":"2024-04-23T14:36:29.242605+0000","last_fullsized":"2024-04-23T14:36:29.242605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.246845+0000","last_change":"2024-04-23T14:36:29.246845+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.246845+0000","last_undegraded":"2024-04-23T14:36:29.246845+0000","last_fullsized":"2024-04-23T14:36:29.246845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257673+0000","last_change":"2024-04-23T14:36:30.257389+0000","last_active":"2024-04-23T14:36:31.257673+0000","last_peered":"2024-04-23T14:36:31.257673+0000","last_clean":"2024-04-23T14:36:31.257673+0000","last_became_active":"2024-04-23T14:36:30.256095+0000","last_became_peered":"2024-04-23T14:36:30.256095+0000","last_unstale":"2024-04-23T14:36:31.257673+0000","last_undegraded":"2024-04-23T14:36:31.257673+0000","last_fullsized":"2024-04-23T14:36:31.257673+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:48.658488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247241+0000","last_change":"2024-04-23T14:36:29.247241+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247241+0000","last_undegraded":"2024-04-23T14:36:29.247241+0000","last_fullsized":"2024-04-23T14:36:29.247241+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243996+0000","last_change":"2024-04-23T14:36:29.243996+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243996+0000","last_undegraded":"2024-04-23T14:36:29.243996+0000","last_fullsized":"2024-04-23T14:36:29.243996+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239476+0000","last_change":"2024-04-23T14:36:29.239476+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239476+0000","last_undegraded":"2024-04-23T14:36:29.239476+0000","last_fullsized":"2024-04-23T14:36:29.239476+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243017+0000","last_change":"2024-04-23T14:36:29.243017+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243017+0000","last_undegraded":"2024-04-23T14:36:29.243017+0000","last_fullsized":"2024-04-23T14:36:29.243017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247488+0000","last_change":"2024-04-23T14:36:29.247488+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247488+0000","last_undegraded":"2024-04-23T14:36:29.247488+0000","last_fullsized":"2024-04-23T14:36:29.247488+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.250249+0000","last_change":"2024-04-23T14:36:29.250249+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.250249+0000","last_undegraded":"2024-04-23T14:36:29.250249+0000","last_fullsized":"2024-04-23T14:36:29.250249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.254142+0000","last_change":"2024-04-23T14:36:29.254142+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.254142+0000","last_undegraded":"2024-04-23T14:36:29.254142+0000","last_fullsized":"2024-04-23T14:36:29.254142+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247037+0000","last_change":"2024-04-23T14:36:29.247037+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247037+0000","last_undegraded":"2024-04-23T14:36:29.247037+0000","last_fullsized":"2024-04-23T14:36:29.247037+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239768+0000","last_change":"2024-04-23T14:36:29.239768+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239768+0000","last_undegraded":"2024-04-23T14:36:29.239768+0000","last_fullsized":"2024-04-23T14:36:29.239768+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238732+0000","last_change":"2024-04-23T14:36:29.238732+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238732+0000","last_undegraded":"2024-04-23T14:36:29.238732+0000","last_fullsized":"2024-04-23T14:36:29.238732+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245148+0000","last_change":"2024-04-23T14:36:29.245148+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245148+0000","last_undegraded":"2024-04-23T14:36:29.245148+0000","last_fullsized":"2024-04-23T14:36:29.245148+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241311+0000","last_change":"2024-04-23T14:36:29.241311+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241311+0000","last_undegraded":"2024-04-23T14:36:29.241311+0000","last_fullsized":"2024-04-23T14:36:29.241311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238095+0000","last_change":"2024-04-23T14:36:29.238095+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238095+0000","last_undegraded":"2024-04-23T14:36:29.238095+0000","last_fullsized":"2024-04-23T14:36:29.238095+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235704+0000","last_change":"2024-04-23T14:36:29.235704+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235704+0000","last_undegraded":"2024-04-23T14:36:29.235704+0000","last_fullsized":"2024-04-23T14:36:29.235704+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235950+0000","last_change":"2024-04-23T14:36:29.235950+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235950+0000","last_undegraded":"2024-04-23T14:36:29.235950+0000","last_fullsized":"2024-04-23T14:36:29.235950+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251241+0000","last_change":"2024-04-23T14:36:29.251241+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251241+0000","last_undegraded":"2024-04-23T14:36:29.251241+0000","last_fullsized":"2024-04-23T14:36:29.251241+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266770+0000","last_change":"2024-04-23T14:36:29.265430+0000","last_active":"2024-04-23T14:36:29.266770+0000","last_peered":"2024-04-23T14:36:29.266770+0000","last_clean":"2024-04-23T14:36:29.266770+0000","last_became_active":"2024-04-23T14:36:29.265096+0000","last_became_peered":"2024-04-23T14:36:29.265096+0000","last_unstale":"2024-04-23T14:36:29.266770+0000","last_undegraded":"2024-04-23T14:36:29.266770+0000","last_fullsized":"2024-04-23T14:36:29.266770+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:29.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257718+0000","last_change":"2024-04-23T14:36:30.257027+0000","last_active":"2024-04-23T14:36:31.257718+0000","last_peered":"2024-04-23T14:36:31.257718+0000","last_clean":"2024-04-23T14:36:31.257718+0000","last_became_active":"2024-04-23T14:36:30.255514+0000","last_became_peered":"2024-04-23T14:36:30.255514+0000","last_unstale":"2024-04-23T14:36:31.257718+0000","last_undegraded":"2024-04-23T14:36:31.257718+0000","last_fullsized":"2024-04-23T14:36:31.257718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:21.165754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259920+0000","last_change":"2024-04-23T14:36:29.255962+0000","last_active":"2024-04-23T14:36:29.259920+0000","last_peered":"2024-04-23T14:36:29.259920+0000","last_clean":"2024-04-23T14:36:29.259920+0000","last_became_active":"2024-04-23T14:36:29.255566+0000","last_became_peered":"2024-04-23T14:36:29.255566+0000","last_unstale":"2024-04-23T14:36:29.259920+0000","last_undegraded":"2024-04-23T14:36:29.259920+0000","last_fullsized":"2024-04-23T14:36:29.259920+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:53.438649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245613+0000","last_change":"2024-04-23T14:36:29.245613+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245613+0000","last_undegraded":"2024-04-23T14:36:29.245613+0000","last_fullsized":"2024-04-23T14:36:29.245613+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.268721+0000","last_change":"2024-04-23T14:36:29.264940+0000","last_active":"2024-04-23T14:36:29.268721+0000","last_peered":"2024-04-23T14:36:29.268721+0000","last_clean":"2024-04-23T14:36:29.268721+0000","last_became_active":"2024-04-23T14:36:29.264568+0000","last_became_peered":"2024-04-23T14:36:29.264568+0000","last_unstale":"2024-04-23T14:36:29.268721+0000","last_undegraded":"2024-04-23T14:36:29.268721+0000","last_fullsized":"2024-04-23T14:36:29.268721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:56:10.745359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251760+0000","last_change":"2024-04-23T14:36:29.251760+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251760+0000","last_undegraded":"2024-04-23T14:36:29.251760+0000","last_fullsized":"2024-04-23T14:36:29.251760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260946+0000","last_change":"2024-04-23T14:36:29.258194+0000","last_active":"2024-04-23T14:36:29.260946+0000","last_peered":"2024-04-23T14:36:29.260946+0000","last_clean":"2024-04-23T14:36:29.260946+0000","last_became_active":"2024-04-23T14:36:29.257888+0000","last_became_peered":"2024-04-23T14:36:29.257888+0000","last_unstale":"2024-04-23T14:36:29.260946+0000","last_undegraded":"2024-04-23T14:36:29.260946+0000","last_fullsized":"2024-04-23T14:36:29.260946+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:37.263336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.235815+0000","last_change":"2024-04-23T14:36:29.235815+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.235815+0000","last_undegraded":"2024-04-23T14:36:29.235815+0000","last_fullsized":"2024-04-23T14:36:29.235815+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260424+0000","last_change":"2024-04-23T14:36:29.259191+0000","last_active":"2024-04-23T14:36:29.260424+0000","last_peered":"2024-04-23T14:36:29.260424+0000","last_clean":"2024-04-23T14:36:29.260424+0000","last_became_active":"2024-04-23T14:36:29.258420+0000","last_became_peered":"2024-04-23T14:36:29.258420+0000","last_unstale":"2024-04-23T14:36:29.260424+0000","last_undegraded":"2024-04-23T14:36:29.260424+0000","last_fullsized":"2024-04-23T14:36:29.260424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:49.177144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241971+0000","last_change":"2024-04-23T14:36:29.241971+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241971+0000","last_undegraded":"2024-04-23T14:36:29.241971+0000","last_fullsized":"2024-04-23T14:36:29.241971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260826+0000","last_change":"2024-04-23T14:36:29.255423+0000","last_active":"2024-04-23T14:36:29.260826+0000","last_peered":"2024-04-23T14:36:29.260826+0000","last_clean":"2024-04-23T14:36:29.260826+0000","last_became_active":"2024-04-23T14:36:29.255130+0000","last_became_peered":"2024-04-23T14:36:29.255130+0000","last_unstale":"2024-04-23T14:36:29.260826+0000","last_undegraded":"2024-04-23T14:36:29.260826+0000","last_fullsized":"2024-04-23T14:36:29.260826+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:35.193457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256577+0000","last_change":"2024-04-23T14:36:30.259813+0000","last_active":"2024-04-23T14:36:31.256577+0000","last_peered":"2024-04-23T14:36:31.256577+0000","last_clean":"2024-04-23T14:36:31.256577+0000","last_became_active":"2024-04-23T14:36:30.259020+0000","last_became_peered":"2024-04-23T14:36:30.259020+0000","last_unstale":"2024-04-23T14:36:31.256577+0000","last_undegraded":"2024-04-23T14:36:31.256577+0000","last_fullsized":"2024-04-23T14:36:31.256577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:28.217244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264303+0000","last_change":"2024-04-23T14:36:29.262680+0000","last_active":"2024-04-23T14:36:29.264303+0000","last_peered":"2024-04-23T14:36:29.264303+0000","last_clean":"2024-04-23T14:36:29.264303+0000","last_became_active":"2024-04-23T14:36:29.262177+0000","last_became_peered":"2024-04-23T14:36:29.262177+0000","last_unstale":"2024-04-23T14:36:29.264303+0000","last_undegraded":"2024-04-23T14:36:29.264303+0000","last_fullsized":"2024-04-23T14:36:29.264303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242083+0000","last_change":"2024-04-23T14:36:29.242083+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242083+0000","last_undegraded":"2024-04-23T14:36:29.242083+0000","last_fullsized":"2024-04-23T14:36:29.242083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256559+0000","last_change":"2024-04-23T14:36:29.255068+0000","last_active":"2024-04-23T14:36:31.256559+0000","last_peered":"2024-04-23T14:36:31.256559+0000","last_clean":"2024-04-23T14:36:31.256559+0000","last_became_active":"2024-04-23T14:36:29.254465+0000","last_became_peered":"2024-04-23T14:36:29.254465+0000","last_unstale":"2024-04-23T14:36:31.256559+0000","last_undegraded":"2024-04-23T14:36:31.256559+0000","last_fullsized":"2024-04-23T14:36:31.256559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:47.763347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236904+0000","last_change":"2024-04-23T14:36:29.236904+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236904+0000","last_undegraded":"2024-04-23T14:36:29.236904+0000","last_fullsized":"2024-04-23T14:36:29.236904+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266420+0000","last_change":"2024-04-23T14:36:29.262461+0000","last_active":"2024-04-23T14:36:29.266420+0000","last_peered":"2024-04-23T14:36:29.266420+0000","last_clean":"2024-04-23T14:36:29.266420+0000","last_became_active":"2024-04-23T14:36:29.262134+0000","last_became_peered":"2024-04-23T14:36:29.262134+0000","last_unstale":"2024-04-23T14:36:29.266420+0000","last_undegraded":"2024-04-23T14:36:29.266420+0000","last_fullsized":"2024-04-23T14:36:29.266420+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.255161+0000","last_change":"2024-04-23T14:36:29.255161+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.255161+0000","last_undegraded":"2024-04-23T14:36:29.255161+0000","last_fullsized":"2024-04-23T14:36:29.255161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260641+0000","last_change":"2024-04-23T14:36:29.252722+0000","last_active":"2024-04-23T14:36:29.260641+0000","last_peered":"2024-04-23T14:36:29.260641+0000","last_clean":"2024-04-23T14:36:29.260641+0000","last_became_active":"2024-04-23T14:36:29.252421+0000","last_became_peered":"2024-04-23T14:36:29.252421+0000","last_unstale":"2024-04-23T14:36:29.260641+0000","last_undegraded":"2024-04-23T14:36:29.260641+0000","last_fullsized":"2024-04-23T14:36:29.260641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:27.199106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.243002+0000","last_change":"2024-04-23T14:36:29.243002+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.243002+0000","last_undegraded":"2024-04-23T14:36:29.243002+0000","last_fullsized":"2024-04-23T14:36:29.243002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.258545+0000","last_change":"2024-04-23T14:36:29.255338+0000","last_active":"2024-04-23T14:36:29.258545+0000","last_peered":"2024-04-23T14:36:29.258545+0000","last_clean":"2024-04-23T14:36:29.258545+0000","last_became_active":"2024-04-23T14:36:29.254916+0000","last_became_peered":"2024-04-23T14:36:29.254916+0000","last_unstale":"2024-04-23T14:36:29.258545+0000","last_undegraded":"2024-04-23T14:36:29.258545+0000","last_fullsized":"2024-04-23T14:36:29.258545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:39.168711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257203+0000","last_change":"2024-04-23T14:36:30.267547+0000","last_active":"2024-04-23T14:36:31.257203+0000","last_peered":"2024-04-23T14:36:31.257203+0000","last_clean":"2024-04-23T14:36:31.257203+0000","last_became_active":"2024-04-23T14:36:30.267103+0000","last_became_peered":"2024-04-23T14:36:30.267103+0000","last_unstale":"2024-04-23T14:36:31.257203+0000","last_undegraded":"2024-04-23T14:36:31.257203+0000","last_fullsized":"2024-04-23T14:36:31.257203+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:14.787860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260627+0000","last_change":"2024-04-23T14:36:29.257996+0000","last_active":"2024-04-23T14:36:29.260627+0000","last_peered":"2024-04-23T14:36:29.260627+0000","last_clean":"2024-04-23T14:36:29.260627+0000","last_became_active":"2024-04-23T14:36:29.256591+0000","last_became_peered":"2024-04-23T14:36:29.256591+0000","last_unstale":"2024-04-23T14:36:29.260627+0000","last_undegraded":"2024-04-23T14:36:29.260627+0000","last_fullsized":"2024-04-23T14:36:29.260627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:47.640350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.249860+0000","last_change":"2024-04-23T14:36:29.249860+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.249860+0000","last_undegraded":"2024-04-23T14:36:29.249860+0000","last_fullsized":"2024-04-23T14:36:29.249860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264329+0000","last_change":"2024-04-23T14:36:29.262784+0000","last_active":"2024-04-23T14:36:29.264329+0000","last_peered":"2024-04-23T14:36:29.264329+0000","last_clean":"2024-04-23T14:36:29.264329+0000","last_became_active":"2024-04-23T14:36:29.262418+0000","last_became_peered":"2024-04-23T14:36:29.262418+0000","last_unstale":"2024-04-23T14:36:29.264329+0000","last_undegraded":"2024-04-23T14:36:29.264329+0000","last_fullsized":"2024-04-23T14:36:29.264329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238374+0000","last_change":"2024-04-23T14:36:29.238374+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238374+0000","last_undegraded":"2024-04-23T14:36:29.238374+0000","last_fullsized":"2024-04-23T14:36:29.238374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260842+0000","last_change":"2024-04-23T14:36:29.256582+0000","last_active":"2024-04-23T14:36:29.260842+0000","last_peered":"2024-04-23T14:36:29.260842+0000","last_clean":"2024-04-23T14:36:29.260842+0000","last_became_active":"2024-04-23T14:36:29.256299+0000","last_became_peered":"2024-04-23T14:36:29.256299+0000","last_unstale":"2024-04-23T14:36:29.260842+0000","last_undegraded":"2024-04-23T14:36:29.260842+0000","last_fullsized":"2024-04-23T14:36:29.260842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:04.017042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236627+0000","last_change":"2024-04-23T14:36:29.236627+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236627+0000","last_undegraded":"2024-04-23T14:36:29.236627+0000","last_fullsized":"2024-04-23T14:36:29.236627+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259444+0000","last_change":"2024-04-23T14:36:29.254735+0000","last_active":"2024-04-23T14:36:29.259444+0000","last_peered":"2024-04-23T14:36:29.259444+0000","last_clean":"2024-04-23T14:36:29.259444+0000","last_became_active":"2024-04-23T14:36:29.253176+0000","last_became_peered":"2024-04-23T14:36:29.253176+0000","last_unstale":"2024-04-23T14:36:29.259444+0000","last_undegraded":"2024-04-23T14:36:29.259444+0000","last_fullsized":"2024-04-23T14:36:29.259444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:45.728544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256827+0000","last_change":"2024-04-23T14:36:30.254759+0000","last_active":"2024-04-23T14:36:31.256827+0000","last_peered":"2024-04-23T14:36:31.256827+0000","last_clean":"2024-04-23T14:36:31.256827+0000","last_became_active":"2024-04-23T14:36:30.254477+0000","last_became_peered":"2024-04-23T14:36:30.254477+0000","last_unstale":"2024-04-23T14:36:31.256827+0000","last_undegraded":"2024-04-23T14:36:31.256827+0000","last_fullsized":"2024-04-23T14:36:31.256827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:15.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266716+0000","last_change":"2024-04-23T14:36:29.263312+0000","last_active":"2024-04-23T14:36:29.266716+0000","last_peered":"2024-04-23T14:36:29.266716+0000","last_clean":"2024-04-23T14:36:29.266716+0000","last_became_active":"2024-04-23T14:36:29.262939+0000","last_became_peered":"2024-04-23T14:36:29.262939+0000","last_unstale":"2024-04-23T14:36:29.266716+0000","last_undegraded":"2024-04-23T14:36:29.266716+0000","last_fullsized":"2024-04-23T14:36:29.266716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:18:16.720971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.242509+0000","last_change":"2024-04-23T14:36:29.242509+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.242509+0000","last_undegraded":"2024-04-23T14:36:29.242509+0000","last_fullsized":"2024-04-23T14:36:29.242509+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259691+0000","last_change":"2024-04-23T14:36:29.251466+0000","last_active":"2024-04-23T14:36:29.259691+0000","last_peered":"2024-04-23T14:36:29.259691+0000","last_clean":"2024-04-23T14:36:29.259691+0000","last_became_active":"2024-04-23T14:36:29.250985+0000","last_became_peered":"2024-04-23T14:36:29.250985+0000","last_unstale":"2024-04-23T14:36:29.259691+0000","last_undegraded":"2024-04-23T14:36:29.259691+0000","last_fullsized":"2024-04-23T14:36:29.259691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:14.802229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.254476+0000","last_change":"2024-04-23T14:36:29.254476+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.254476+0000","last_undegraded":"2024-04-23T14:36:29.254476+0000","last_fullsized":"2024-04-23T14:36:29.254476+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264202+0000","last_change":"2024-04-23T14:36:29.257682+0000","last_active":"2024-04-23T14:36:29.264202+0000","last_peered":"2024-04-23T14:36:29.264202+0000","last_clean":"2024-04-23T14:36:29.264202+0000","last_became_active":"2024-04-23T14:36:29.257385+0000","last_became_peered":"2024-04-23T14:36:29.257385+0000","last_unstale":"2024-04-23T14:36:29.264202+0000","last_undegraded":"2024-04-23T14:36:29.264202+0000","last_fullsized":"2024-04-23T14:36:29.264202+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.247591+0000","last_change":"2024-04-23T14:36:29.247591+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.247591+0000","last_undegraded":"2024-04-23T14:36:29.247591+0000","last_fullsized":"2024-04-23T14:36:29.247591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264676+0000","last_change":"2024-04-23T14:36:29.262796+0000","last_active":"2024-04-23T14:36:29.264676+0000","last_peered":"2024-04-23T14:36:29.264676+0000","last_clean":"2024-04-23T14:36:29.264676+0000","last_became_active":"2024-04-23T14:36:29.262504+0000","last_became_peered":"2024-04-23T14:36:29.262504+0000","last_unstale":"2024-04-23T14:36:29.264676+0000","last_undegraded":"2024-04-23T14:36:29.264676+0000","last_fullsized":"2024-04-23T14:36:29.264676+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257317+0000","last_change":"2024-04-23T14:36:30.260184+0000","last_active":"2024-04-23T14:36:31.257317+0000","last_peered":"2024-04-23T14:36:31.257317+0000","last_clean":"2024-04-23T14:36:31.257317+0000","last_became_active":"2024-04-23T14:36:30.259450+0000","last_became_peered":"2024-04-23T14:36:30.259450+0000","last_unstale":"2024-04-23T14:36:31.257317+0000","last_undegraded":"2024-04-23T14:36:31.257317+0000","last_fullsized":"2024-04-23T14:36:31.257317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:36.211595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.269267+0000","last_change":"2024-04-23T14:36:29.267357+0000","last_active":"2024-04-23T14:36:29.269267+0000","last_peered":"2024-04-23T14:36:29.269267+0000","last_clean":"2024-04-23T14:36:29.269267+0000","last_became_active":"2024-04-23T14:36:29.266992+0000","last_became_peered":"2024-04-23T14:36:29.266992+0000","last_unstale":"2024-04-23T14:36:29.269267+0000","last_undegraded":"2024-04-23T14:36:29.269267+0000","last_fullsized":"2024-04-23T14:36:29.269267+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:45.728544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.240681+0000","last_change":"2024-04-23T14:36:29.240681+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.240681+0000","last_undegraded":"2024-04-23T14:36:29.240681+0000","last_fullsized":"2024-04-23T14:36:29.240681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260804+0000","last_change":"2024-04-23T14:36:29.254896+0000","last_active":"2024-04-23T14:36:29.260804+0000","last_peered":"2024-04-23T14:36:29.260804+0000","last_clean":"2024-04-23T14:36:29.260804+0000","last_became_active":"2024-04-23T14:36:29.254593+0000","last_became_peered":"2024-04-23T14:36:29.254593+0000","last_unstale":"2024-04-23T14:36:29.260804+0000","last_undegraded":"2024-04-23T14:36:29.260804+0000","last_fullsized":"2024-04-23T14:36:29.260804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:13.769723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.241152+0000","last_change":"2024-04-23T14:36:29.241152+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.241152+0000","last_undegraded":"2024-04-23T14:36:29.241152+0000","last_fullsized":"2024-04-23T14:36:29.241152+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.260903+0000","last_change":"2024-04-23T14:36:29.259253+0000","last_active":"2024-04-23T14:36:29.260903+0000","last_peered":"2024-04-23T14:36:29.260903+0000","last_clean":"2024-04-23T14:36:29.260903+0000","last_became_active":"2024-04-23T14:36:29.256871+0000","last_became_peered":"2024-04-23T14:36:29.256871+0000","last_unstale":"2024-04-23T14:36:29.260903+0000","last_undegraded":"2024-04-23T14:36:29.260903+0000","last_fullsized":"2024-04-23T14:36:29.260903+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:33.990830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.238310+0000","last_change":"2024-04-23T14:36:29.238310+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.238310+0000","last_undegraded":"2024-04-23T14:36:29.238310+0000","last_fullsized":"2024-04-23T14:36:29.238310+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.254977+0000","last_change":"2024-04-23T14:36:29.253692+0000","last_active":"2024-04-23T14:36:29.254977+0000","last_peered":"2024-04-23T14:36:29.254977+0000","last_clean":"2024-04-23T14:36:29.254977+0000","last_became_active":"2024-04-23T14:36:29.253295+0000","last_became_peered":"2024-04-23T14:36:29.253295+0000","last_unstale":"2024-04-23T14:36:29.254977+0000","last_undegraded":"2024-04-23T14:36:29.254977+0000","last_fullsized":"2024-04-23T14:36:29.254977+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:18.617182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.240864+0000","last_change":"2024-04-23T14:36:29.240864+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.240864+0000","last_undegraded":"2024-04-23T14:36:29.240864+0000","last_fullsized":"2024-04-23T14:36:29.240864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.253828+0000","last_change":"2024-04-23T14:36:29.249580+0000","last_active":"2024-04-23T14:36:29.253828+0000","last_peered":"2024-04-23T14:36:29.253828+0000","last_clean":"2024-04-23T14:36:29.253828+0000","last_became_active":"2024-04-23T14:36:29.249248+0000","last_became_peered":"2024-04-23T14:36:29.249248+0000","last_unstale":"2024-04-23T14:36:29.253828+0000","last_undegraded":"2024-04-23T14:36:29.253828+0000","last_fullsized":"2024-04-23T14:36:29.253828+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:25.646399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.239729+0000","last_change":"2024-04-23T14:36:29.239729+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.239729+0000","last_undegraded":"2024-04-23T14:36:29.239729+0000","last_fullsized":"2024-04-23T14:36:29.239729+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259327+0000","last_change":"2024-04-23T14:36:29.253372+0000","last_active":"2024-04-23T14:36:29.259327+0000","last_peered":"2024-04-23T14:36:29.259327+0000","last_clean":"2024-04-23T14:36:29.259327+0000","last_became_active":"2024-04-23T14:36:29.251760+0000","last_became_peered":"2024-04-23T14:36:29.251760+0000","last_unstale":"2024-04-23T14:36:29.259327+0000","last_undegraded":"2024-04-23T14:36:29.259327+0000","last_fullsized":"2024-04-23T14:36:29.259327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:29.750803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.251080+0000","last_change":"2024-04-23T14:36:29.251080+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.251080+0000","last_undegraded":"2024-04-23T14:36:29.251080+0000","last_fullsized":"2024-04-23T14:36:29.251080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257414+0000","last_change":"2024-04-23T14:36:29.245427+0000","last_active":"2024-04-23T14:36:31.257414+0000","last_peered":"2024-04-23T14:36:31.257414+0000","last_clean":"2024-04-23T14:36:31.257414+0000","last_became_active":"2024-04-23T14:36:29.245034+0000","last_became_peered":"2024-04-23T14:36:29.245034+0000","last_unstale":"2024-04-23T14:36:31.257414+0000","last_undegraded":"2024-04-23T14:36:31.257414+0000","last_fullsized":"2024-04-23T14:36:31.257414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:13.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.249579+0000","last_change":"2024-04-23T14:36:29.249579+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.249579+0000","last_undegraded":"2024-04-23T14:36:29.249579+0000","last_fullsized":"2024-04-23T14:36:29.249579+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.259982+0000","last_change":"2024-04-23T14:36:29.257966+0000","last_active":"2024-04-23T14:36:29.259982+0000","last_peered":"2024-04-23T14:36:29.259982+0000","last_clean":"2024-04-23T14:36:29.259982+0000","last_became_active":"2024-04-23T14:36:29.257687+0000","last_became_peered":"2024-04-23T14:36:29.257687+0000","last_unstale":"2024-04-23T14:36:29.259982+0000","last_undegraded":"2024-04-23T14:36:29.259982+0000","last_fullsized":"2024-04-23T14:36:29.259982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:43:30.776694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257650+0000","last_change":"2024-04-23T14:36:30.260028+0000","last_active":"2024-04-23T14:36:31.257650+0000","last_peered":"2024-04-23T14:36:31.257650+0000","last_clean":"2024-04-23T14:36:31.257650+0000","last_became_active":"2024-04-23T14:36:30.259713+0000","last_became_peered":"2024-04-23T14:36:30.259713+0000","last_unstale":"2024-04-23T14:36:31.257650+0000","last_undegraded":"2024-04-23T14:36:31.257650+0000","last_fullsized":"2024-04-23T14:36:31.257650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:02.099306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256853+0000","last_change":"2024-04-23T14:36:29.249521+0000","last_active":"2024-04-23T14:36:31.256853+0000","last_peered":"2024-04-23T14:36:31.256853+0000","last_clean":"2024-04-23T14:36:31.256853+0000","last_became_active":"2024-04-23T14:36:29.249249+0000","last_became_peered":"2024-04-23T14:36:29.249249+0000","last_unstale":"2024-04-23T14:36:31.256853+0000","last_undegraded":"2024-04-23T14:36:31.256853+0000","last_fullsized":"2024-04-23T14:36:31.256853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:08:47.916797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245640+0000","last_change":"2024-04-23T14:36:29.245640+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245640+0000","last_undegraded":"2024-04-23T14:36:29.245640+0000","last_fullsized":"2024-04-23T14:36:29.245640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264297+0000","last_change":"2024-04-23T14:36:29.262517+0000","last_active":"2024-04-23T14:36:29.264297+0000","last_peered":"2024-04-23T14:36:29.264297+0000","last_clean":"2024-04-23T14:36:29.264297+0000","last_became_active":"2024-04-23T14:36:29.262167+0000","last_became_peered":"2024-04-23T14:36:29.262167+0000","last_unstale":"2024-04-23T14:36:29.264297+0000","last_undegraded":"2024-04-23T14:36:29.264297+0000","last_fullsized":"2024-04-23T14:36:29.264297+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:20.147616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.245791+0000","last_change":"2024-04-23T14:36:29.245791+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.245791+0000","last_undegraded":"2024-04-23T14:36:29.245791+0000","last_fullsized":"2024-04-23T14:36:29.245791+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.266259+0000","last_change":"2024-04-23T14:36:29.260899+0000","last_active":"2024-04-23T14:36:29.266259+0000","last_peered":"2024-04-23T14:36:29.266259+0000","last_clean":"2024-04-23T14:36:29.266259+0000","last_became_active":"2024-04-23T14:36:29.258766+0000","last_became_peered":"2024-04-23T14:36:29.258766+0000","last_unstale":"2024-04-23T14:36:29.266259+0000","last_undegraded":"2024-04-23T14:36:29.266259+0000","last_fullsized":"2024-04-23T14:36:29.266259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.236208+0000","last_change":"2024-04-23T14:36:29.236208+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.236208+0000","last_undegraded":"2024-04-23T14:36:29.236208+0000","last_fullsized":"2024-04-23T14:36:29.236208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.264147+0000","last_change":"2024-04-23T14:36:29.261017+0000","last_active":"2024-04-23T14:36:29.264147+0000","last_peered":"2024-04-23T14:36:29.264147+0000","last_clean":"2024-04-23T14:36:29.264147+0000","last_became_active":"2024-04-23T14:36:29.258667+0000","last_became_peered":"2024-04-23T14:36:29.258667+0000","last_unstale":"2024-04-23T14:36:29.264147+0000","last_undegraded":"2024-04-23T14:36:29.264147+0000","last_fullsized":"2024-04-23T14:36:29.264147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:47.640350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-23T14:36:29.237471+0000","last_change":"2024-04-23T14:36:29.237471+0000","last_active":"2024-04-23T14:36:29.210668+0000","last_peered":"2024-04-23T14:36:29.210668+0000","last_clean":"2024-04-23T14:36:29.210668+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:36:29.237471+0000","last_undegraded":"2024-04-23T14:36:29.237471+0000","last_fullsized":"2024-04-23T14:36:29.237471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.261357+0000","last_change":"2024-04-23T14:36:29.258243+0000","last_active":"2024-04-23T14:36:29.261357+0000","last_peered":"2024-04-23T14:36:29.261357+0000","last_clean":"2024-04-23T14:36:29.261357+0000","last_became_active":"2024-04-23T14:36:29.257918+0000","last_became_peered":"2024-04-23T14:36:29.257918+0000","last_unstale":"2024-04-23T14:36:29.261357+0000","last_undegraded":"2024-04-23T14:36:29.261357+0000","last_fullsized":"2024-04-23T14:36:29.261357+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:01.081168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-23T14:36:29.258412+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:36:29.258412+0000","last_peered":"2024-04-23T14:36:29.258412+0000","last_clean":"2024-04-23T14:36:29.258412+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:36:29.258412+0000","last_undegraded":"2024-04-23T14:36:29.258412+0000","last_fullsized":"2024-04-23T14:36:29.258412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":924675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":334307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":334307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":851968,"data_stored":355269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":340582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":924675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:36:35.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-04-23T14:36:35.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.257806+0000', 'last_change': '2024-04-23T14:36:29.257806+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.257806+0000', 'last_undegraded': '2024-04-23T14:36:29.257806+0000', 'last_fullsized': '2024-04-23T14:36:29.257806+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-04-23T14:36:35.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.238437+0000', 'last_change': '2024-04-23T14:36:29.238437+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.238437+0000', 'last_undegraded': '2024-04-23T14:36:29.238437+0000', 'last_fullsized': '2024-04-23T14:36:29.238437+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.239371+0000', 'last_change': '2024-04-23T14:36:29.239371+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.239371+0000', 'last_undegraded': '2024-04-23T14:36:29.239371+0000', 'last_fullsized': '2024-04-23T14:36:29.239371+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.248757+0000', 'last_change': '2024-04-23T14:36:29.248757+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.248757+0000', 'last_undegraded': '2024-04-23T14:36:29.248757+0000', 'last_fullsized': '2024-04-23T14:36:29.248757+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.243660+0000', 'last_change': '2024-04-23T14:36:29.243660+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.243660+0000', 'last_undegraded': '2024-04-23T14:36:29.243660+0000', 'last_fullsized': '2024-04-23T14:36:29.243660+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.239846+0000', 'last_change': '2024-04-23T14:36:29.239846+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.239846+0000', 'last_undegraded': '2024-04-23T14:36:29.239846+0000', 'last_fullsized': '2024-04-23T14:36:29.239846+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.237516+0000', 'last_change': '2024-04-23T14:36:29.237516+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.237516+0000', 'last_undegraded': '2024-04-23T14:36:29.237516+0000', 'last_fullsized': '2024-04-23T14:36:29.237516+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-04-23T14:36:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.243637+0000', 'last_change': '2024-04-23T14:36:29.243637+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.243637+0000', 'last_undegraded': '2024-04-23T14:36:29.243637+0000', 'last_fullsized': '2024-04-23T14:36:29.243637+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.252268+0000', 'last_change': '2024-04-23T14:36:29.252268+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.252268+0000', 'last_undegraded': '2024-04-23T14:36:29.252268+0000', 'last_fullsized': '2024-04-23T14:36:29.252268+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.248093+0000', 'last_change': '2024-04-23T14:36:29.248093+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.248093+0000', 'last_undegraded': '2024-04-23T14:36:29.248093+0000', 'last_fullsized': '2024-04-23T14:36:29.248093+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.242843+0000', 'last_change': '2024-04-23T14:36:29.242843+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.242843+0000', 'last_undegraded': '2024-04-23T14:36:29.242843+0000', 'last_fullsized': '2024-04-23T14:36:29.242843+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.242605+0000', 'last_change': '2024-04-23T14:36:29.242605+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.242605+0000', 'last_undegraded': '2024-04-23T14:36:29.242605+0000', 'last_fullsized': '2024-04-23T14:36:29.242605+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.246845+0000', 'last_change': '2024-04-23T14:36:29.246845+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.246845+0000', 'last_undegraded': '2024-04-23T14:36:29.246845+0000', 'last_fullsized': '2024-04-23T14:36:29.246845+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.247241+0000', 'last_change': '2024-04-23T14:36:29.247241+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.247241+0000', 'last_undegraded': '2024-04-23T14:36:29.247241+0000', 'last_fullsized': '2024-04-23T14:36:29.247241+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.243996+0000', 'last_change': '2024-04-23T14:36:29.243996+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.243996+0000', 'last_undegraded': '2024-04-23T14:36:29.243996+0000', 'last_fullsized': '2024-04-23T14:36:29.243996+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.239476+0000', 'last_change': '2024-04-23T14:36:29.239476+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.239476+0000', 'last_undegraded': '2024-04-23T14:36:29.239476+0000', 'last_fullsized': '2024-04-23T14:36:29.239476+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.243017+0000', 'last_change': '2024-04-23T14:36:29.243017+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.243017+0000', 'last_undegraded': '2024-04-23T14:36:29.243017+0000', 'last_fullsized': '2024-04-23T14:36:29.243017+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.247488+0000', 'last_change': '2024-04-23T14:36:29.247488+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.247488+0000', 'last_undegraded': '2024-04-23T14:36:29.247488+0000', 'last_fullsized': '2024-04-23T14:36:29.247488+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-04-23T14:36:35.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.250249+0000', 'last_change': '2024-04-23T14:36:29.250249+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.250249+0000', 'last_undegraded': '2024-04-23T14:36:29.250249+0000', 'last_fullsized': '2024-04-23T14:36:29.250249+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.254142+0000', 'last_change': '2024-04-23T14:36:29.254142+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.254142+0000', 'last_undegraded': '2024-04-23T14:36:29.254142+0000', 'last_fullsized': '2024-04-23T14:36:29.254142+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.247037+0000', 'last_change': '2024-04-23T14:36:29.247037+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.247037+0000', 'last_undegraded': '2024-04-23T14:36:29.247037+0000', 'last_fullsized': '2024-04-23T14:36:29.247037+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.239768+0000', 'last_change': '2024-04-23T14:36:29.239768+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.239768+0000', 'last_undegraded': '2024-04-23T14:36:29.239768+0000', 'last_fullsized': '2024-04-23T14:36:29.239768+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.238732+0000', 'last_change': '2024-04-23T14:36:29.238732+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.238732+0000', 'last_undegraded': '2024-04-23T14:36:29.238732+0000', 'last_fullsized': '2024-04-23T14:36:29.238732+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.245148+0000', 'last_change': '2024-04-23T14:36:29.245148+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.245148+0000', 'last_undegraded': '2024-04-23T14:36:29.245148+0000', 'last_fullsized': '2024-04-23T14:36:29.245148+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-04-23T14:36:35.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.241311+0000', 'last_change': '2024-04-23T14:36:29.241311+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.241311+0000', 'last_undegraded': '2024-04-23T14:36:29.241311+0000', 'last_fullsized': '2024-04-23T14:36:29.241311+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.238095+0000', 'last_change': '2024-04-23T14:36:29.238095+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.238095+0000', 'last_undegraded': '2024-04-23T14:36:29.238095+0000', 'last_fullsized': '2024-04-23T14:36:29.238095+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.235704+0000', 'last_change': '2024-04-23T14:36:29.235704+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.235704+0000', 'last_undegraded': '2024-04-23T14:36:29.235704+0000', 'last_fullsized': '2024-04-23T14:36:29.235704+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.235950+0000', 'last_change': '2024-04-23T14:36:29.235950+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.235950+0000', 'last_undegraded': '2024-04-23T14:36:29.235950+0000', 'last_fullsized': '2024-04-23T14:36:29.235950+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.251241+0000', 'last_change': '2024-04-23T14:36:29.251241+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.251241+0000', 'last_undegraded': '2024-04-23T14:36:29.251241+0000', 'last_fullsized': '2024-04-23T14:36:29.251241+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.245613+0000', 'last_change': '2024-04-23T14:36:29.245613+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.245613+0000', 'last_undegraded': '2024-04-23T14:36:29.245613+0000', 'last_fullsized': '2024-04-23T14:36:29.245613+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.251760+0000', 'last_change': '2024-04-23T14:36:29.251760+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.251760+0000', 'last_undegraded': '2024-04-23T14:36:29.251760+0000', 'last_fullsized': '2024-04-23T14:36:29.251760+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.235815+0000', 'last_change': '2024-04-23T14:36:29.235815+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.235815+0000', 'last_undegraded': '2024-04-23T14:36:29.235815+0000', 'last_fullsized': '2024-04-23T14:36:29.235815+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.241971+0000', 'last_change': '2024-04-23T14:36:29.241971+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.241971+0000', 'last_undegraded': '2024-04-23T14:36:29.241971+0000', 'last_fullsized': '2024-04-23T14:36:29.241971+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.242083+0000', 'last_change': '2024-04-23T14:36:29.242083+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.242083+0000', 'last_undegraded': '2024-04-23T14:36:29.242083+0000', 'last_fullsized': '2024-04-23T14:36:29.242083+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.236904+0000', 'last_change': '2024-04-23T14:36:29.236904+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.236904+0000', 'last_undegraded': '2024-04-23T14:36:29.236904+0000', 'last_fullsized': '2024-04-23T14:36:29.236904+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-04-23T14:36:35.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.255161+0000', 'last_change': '2024-04-23T14:36:29.255161+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.255161+0000', 'last_undegraded': '2024-04-23T14:36:29.255161+0000', 'last_fullsized': '2024-04-23T14:36:29.255161+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.243002+0000', 'last_change': '2024-04-23T14:36:29.243002+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.243002+0000', 'last_undegraded': '2024-04-23T14:36:29.243002+0000', 'last_fullsized': '2024-04-23T14:36:29.243002+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.249860+0000', 'last_change': '2024-04-23T14:36:29.249860+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.249860+0000', 'last_undegraded': '2024-04-23T14:36:29.249860+0000', 'last_fullsized': '2024-04-23T14:36:29.249860+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.238374+0000', 'last_change': '2024-04-23T14:36:29.238374+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.238374+0000', 'last_undegraded': '2024-04-23T14:36:29.238374+0000', 'last_fullsized': '2024-04-23T14:36:29.238374+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.236627+0000', 'last_change': '2024-04-23T14:36:29.236627+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.236627+0000', 'last_undegraded': '2024-04-23T14:36:29.236627+0000', 'last_fullsized': '2024-04-23T14:36:29.236627+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.242509+0000', 'last_change': '2024-04-23T14:36:29.242509+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.242509+0000', 'last_undegraded': '2024-04-23T14:36:29.242509+0000', 'last_fullsized': '2024-04-23T14:36:29.242509+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-04-23T14:36:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.254476+0000', 'last_change': '2024-04-23T14:36:29.254476+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.254476+0000', 'last_undegraded': '2024-04-23T14:36:29.254476+0000', 'last_fullsized': '2024-04-23T14:36:29.254476+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.247591+0000', 'last_change': '2024-04-23T14:36:29.247591+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.247591+0000', 'last_undegraded': '2024-04-23T14:36:29.247591+0000', 'last_fullsized': '2024-04-23T14:36:29.247591+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.240681+0000', 'last_change': '2024-04-23T14:36:29.240681+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.240681+0000', 'last_undegraded': '2024-04-23T14:36:29.240681+0000', 'last_fullsized': '2024-04-23T14:36:29.240681+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.241152+0000', 'last_change': '2024-04-23T14:36:29.241152+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.241152+0000', 'last_undegraded': '2024-04-23T14:36:29.241152+0000', 'last_fullsized': '2024-04-23T14:36:29.241152+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.238310+0000', 'last_change': '2024-04-23T14:36:29.238310+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.238310+0000', 'last_undegraded': '2024-04-23T14:36:29.238310+0000', 'last_fullsized': '2024-04-23T14:36:29.238310+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.240864+0000', 'last_change': '2024-04-23T14:36:29.240864+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.240864+0000', 'last_undegraded': '2024-04-23T14:36:29.240864+0000', 'last_fullsized': '2024-04-23T14:36:29.240864+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-23T14:36:35.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.239729+0000', 'last_change': '2024-04-23T14:36:29.239729+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.239729+0000', 'last_undegraded': '2024-04-23T14:36:29.239729+0000', 'last_fullsized': '2024-04-23T14:36:29.239729+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.251080+0000', 'last_change': '2024-04-23T14:36:29.251080+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.251080+0000', 'last_undegraded': '2024-04-23T14:36:29.251080+0000', 'last_fullsized': '2024-04-23T14:36:29.251080+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.249579+0000', 'last_change': '2024-04-23T14:36:29.249579+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.249579+0000', 'last_undegraded': '2024-04-23T14:36:29.249579+0000', 'last_fullsized': '2024-04-23T14:36:29.249579+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.245640+0000', 'last_change': '2024-04-23T14:36:29.245640+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.245640+0000', 'last_undegraded': '2024-04-23T14:36:29.245640+0000', 'last_fullsized': '2024-04-23T14:36:29.245640+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.245791+0000', 'last_change': '2024-04-23T14:36:29.245791+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.245791+0000', 'last_undegraded': '2024-04-23T14:36:29.245791+0000', 'last_fullsized': '2024-04-23T14:36:29.245791+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.236208+0000', 'last_change': '2024-04-23T14:36:29.236208+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.236208+0000', 'last_undegraded': '2024-04-23T14:36:29.236208+0000', 'last_fullsized': '2024-04-23T14:36:29.236208+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-04-23T14:36:35.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-04-23T14:36:29.237471+0000', 'last_change': '2024-04-23T14:36:29.237471+0000', 'last_active': '2024-04-23T14:36:29.210668+0000', 'last_peered': '2024-04-23T14:36:29.210668+0000', 'last_clean': '2024-04-23T14:36:29.210668+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-23T14:36:29.237471+0000', 'last_undegraded': '2024-04-23T14:36:29.237471+0000', 'last_fullsized': '2024-04-23T14:36:29.237471+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'last_clean_scrub_stamp': '2024-04-23T14:36:29.210668+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-23T14:36:35.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-23T14:36:36.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.012+0000 7f6f72400640 1 Processor -- start 2024-04-23T14:36:36.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.012+0000 7f6f72400640 1 -- start start 2024-04-23T14:36:36.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f72400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f6c15c5e0 0x7f6f6c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f72400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0af920 0x7f6f6c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f72400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f6c0d06e0 0x7f6f6c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f72400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c0afd80 con 0x7f6f6c15c5e0 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f72400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c15fa20 con 0x7f6f6c0d06e0 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f72400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c15f0f0 con 0x7f6f6c0af920 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f6c15c5e0 0x7f6f6c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f6c15c5e0 0x7f6f6c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41482/0 (socket says 172.21.15.59:41482) 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 -- 172.21.15.59:0/2891777691 learned_addr learned my addr 172.21.15.59:0/2891777691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:36.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 -- 172.21.15.59:0/2891777691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0af920 msgr2=0x7f6f6c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:36.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 --2- 172.21.15.59:0/2891777691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0af920 0x7f6f6c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 -- 172.21.15.59:0/2891777691 >> v2:172.21.15.76:3300/0 conn(0x7f6f6c0d06e0 msgr2=0x7f6f6c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:36.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 --2- 172.21.15.59:0/2891777691 >> v2:172.21.15.76:3300/0 conn(0x7f6f6c0d06e0 0x7f6f6c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.014+0000 7f6f6b400640 1 -- 172.21.15.59:0/2891777691 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c15f3d0 con 0x7f6f6c15c5e0 2024-04-23T14:36:36.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.016+0000 7f6f6b400640 1 --2- 172.21.15.59:0/2891777691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0af3e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c06df00 tx=0x7f6f5c02a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adda654c82fa2d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:36.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.016+0000 7f6f6aa00640 1 -- 172.21.15.59:0/2891777691 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c036070 con 0x7f6f6c15c5e0 2024-04-23T14:36:36.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.016+0000 7f6f6aa00640 1 -- 172.21.15.59:0/2891777691 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c031600 con 0x7f6f6c15c5e0 2024-04-23T14:36:36.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.016+0000 7f6f6aa00640 1 -- 172.21.15.59:0/2891777691 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0319d0 con 0x7f6f6c15c5e0 2024-04-23T14:36:36.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 -- 172.21.15.59:0/2891777691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c15c5e0 msgr2=0x7f6f6c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 --2- 172.21.15.59:0/2891777691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0af3e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c06df00 tx=0x7f6f5c02a690 comp rx=0 tx=0).stop 2024-04-23T14:36:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 -- 172.21.15.59:0/2891777691 shutdown_connections 2024-04-23T14:36:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 --2- 172.21.15.59:0/2891777691 >> v2:172.21.15.76:3300/0 conn(0x7f6f6c0d06e0 0x7f6f6c071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 --2- 172.21.15.59:0/2891777691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0af920 0x7f6f6c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 --2- 172.21.15.59:0/2891777691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0af3e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 -- 172.21.15.59:0/2891777691 >> 172.21.15.59:0/2891777691 conn(0x7f6f6c067ea0 msgr2=0x7f6f6c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 -- 172.21.15.59:0/2891777691 shutdown_connections 2024-04-23T14:36:36.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.017+0000 7f6f72400640 1 -- 172.21.15.59:0/2891777691 wait complete. 2024-04-23T14:36:36.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.020+0000 7f6f72400640 1 Processor -- start 2024-04-23T14:36:36.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.020+0000 7f6f72400640 1 -- start start 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f72400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c0af920 0x7f6f6c0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f72400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0d06e0 0x7f6f6c0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f72400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f72400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0d0cf0 con 0x7f6f6c0af920 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f72400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0d0b70 con 0x7f6f6c15c5e0 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f72400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c06e480 con 0x7f6f6c0d06e0 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c0af920 0x7f6f6c0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c0af920 0x7f6f6c0cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41488/0 (socket says 172.21.15.59:41488) 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 -- 172.21.15.59:0/887455335 learned_addr learned my addr 172.21.15.59:0/887455335 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6be00640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f62a00640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0d06e0 0x7f6f6c0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 -- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0d06e0 msgr2=0x7f6f6c0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0d06e0 0x7f6f6c0aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 -- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f6c15c5e0 msgr2=0x7f6f6c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 -- 172.21.15.59:0/887455335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c0b0450 con 0x7f6f6c0af920 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f62a00640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0d06e0 0x7f6f6c0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:36.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6be00640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:36.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.021+0000 7f6f6b400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c0af920 0x7f6f6c0cd1c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0093e0 tx=0x7f6f5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:36.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.022+0000 7f6f63e00640 1 -- 172.21.15.59:0/887455335 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c041020 con 0x7f6f6c0af920 2024-04-23T14:36:36.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.022+0000 7f6f63e00640 1 -- 172.21.15.59:0/887455335 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c007800 con 0x7f6f6c0af920 2024-04-23T14:36:36.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.022+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c0b06f0 con 0x7f6f6c0af920 2024-04-23T14:36:36.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.022+0000 7f6f63e00640 1 -- 172.21.15.59:0/887455335 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c036040 con 0x7f6f6c0af920 2024-04-23T14:36:36.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.022+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c0b0cb0 con 0x7f6f6c0af920 2024-04-23T14:36:36.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.023+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c068a80 con 0x7f6f6c0af920 2024-04-23T14:36:36.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.024+0000 7f6f63e00640 1 -- 172.21.15.59:0/887455335 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f5c044020 con 0x7f6f6c0af920 2024-04-23T14:36:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.025+0000 7f6f63e00640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6f4806c920 0x7f6f4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.025+0000 7f6f63e00640 1 -- 172.21.15.59:0/887455335 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f6f5c0acb10 con 0x7f6f6c0af920 2024-04-23T14:36:36.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.025+0000 7f6f62a00640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6f4806c920 0x7f6f4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.026+0000 7f6f62a00640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6f4806c920 0x7f6f4806ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006fd0 tx=0x7f6f54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.030+0000 7f6f63e00640 1 -- 172.21.15.59:0/887455335 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f5c04b030 con 0x7f6f6c0af920 2024-04-23T14:36:36.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.313+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6f6c0cc870 con 0x7f6f6c0af920 2024-04-23T14:36:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.314+0000 7f6f63e00640 1 -- 172.21.15.59:0/887455335 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1035 (secure 0 0 0) 0x7f6f5c07af60 con 0x7f6f6c0af920 2024-04-23T14:36:36.314 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:36.315 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":335,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":69,"num_osds":8,"num_up_osds":8,"osd_up_since":1713882673,"num_in_osds":8,"osd_in_since":1713882664,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":54},{"state_name":"active+clean","count":43}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":263143424,"bytes_avail":772830969856,"bytes_total":773094113280,"inactive_pgs_ratio":0.55670100450515747},"fsmap":{"epoch":43,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6735}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-23T14:36:08.441950+0000","services":{}},"progress_events":{}} 2024-04-23T14:36:36.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6f4806c920 msgr2=0x7f6f4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6f4806c920 0x7f6f4806ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006fd0 tx=0x7f6f54008040 comp rx=0 tx=0).stop 2024-04-23T14:36:36.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c0af920 msgr2=0x7f6f6c0cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c0af920 0x7f6f6c0cd1c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0093e0 tx=0x7f6f5c004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:36.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 shutdown_connections 2024-04-23T14:36:36.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6f4806c920 0x7f6f4806ede0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f6c0d06e0 0x7f6f6c0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f6c15c5e0 0x7f6f6c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 --2- 172.21.15.59:0/887455335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6c0af920 0x7f6f6c0cd1c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 >> 172.21.15.59:0/887455335 conn(0x7f6f6c067ea0 msgr2=0x7f6f6c162d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:36.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 shutdown_connections 2024-04-23T14:36:36.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.317+0000 7f6f72400640 1 -- 172.21.15.59:0/887455335 wait complete. 2024-04-23T14:36:36.338 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-23T14:36:36.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:36:36.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.547+0000 7f9334c00640 1 Processor -- start 2024-04-23T14:36:36.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.547+0000 7f9334c00640 1 -- start start 2024-04-23T14:36:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.549+0000 7f9334c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 0x7f9330169860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.549+0000 7f9334c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f933015f0f0 con 0x7f933015c5e0 2024-04-23T14:36:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.549+0000 7f9334c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f933015f3d0 con 0x7f93300afaf0 2024-04-23T14:36:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.549+0000 7f9334c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f933015fa20 con 0x7f93300d08b0 2024-04-23T14:36:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.549+0000 7f932de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f933015c5e0 0x7f93300af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35198/0 (socket says 172.21.15.59:35198) 2024-04-23T14:36:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.549+0000 7f932de00640 1 -- 172.21.15.59:0/3105339508 learned_addr learned my addr 172.21.15.59:0/3105339508 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932e800640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 0x7f9330169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985414422 0 0) 0x7f933015f0f0 con 0x7f933015c5e0 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9314003520 con 0x7f933015c5e0 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4286287935 0 0) 0x7f9314003520 con 0x7f933015c5e0 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 msgr2=0x7f9330169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 0x7f9330169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 >> v1:172.21.15.76:6789/0 conn(0x7f93300afaf0 legacy=0x7f93300d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933015f6b0 con 0x7f933015c5e0 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9320001670 con 0x7f933015c5e0 2024-04-23T14:36:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 >> v1:172.21.15.59:6789/0 conn(0x7f933015c5e0 legacy=0x7f93300af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93140039e0 0x7f9314003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.550+0000 7f932ca00640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9314004800 0x7f9314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932ca00640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 0x7f93140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f933015fa20 con 0x7f9314004800 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f931400baf0 con 0x7f93140039e0 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f931400bcf0 con 0x7f93300d08b0 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932d400640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9314004800 0x7f9314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932de00640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93140039e0 0x7f9314003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932e800640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 0x7f93140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932d400640 1 -- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 msgr2=0x7f93140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932d400640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93300d08b0 0x7f93140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932d400640 1 -- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93140039e0 msgr2=0x7f9314003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932d400640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93140039e0 0x7f9314003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.551+0000 7f932d400640 1 -- 172.21.15.59:0/3105339508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9314003520 con 0x7f9314004800 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.552+0000 7f932d400640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9314004800 0x7f9314004c80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f931c001750 tx=0x7f9314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be874bd24f02c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c001370 con 0x7f9314004800 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c0052d0 con 0x7f9314004800 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f932ca00640 1 -- 172.21.15.59:0/3105339508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c02abe0 con 0x7f9314004800 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f9334c00640 1 -- 172.21.15.59:0/3105339508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9314004800 msgr2=0x7f9314004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f9334c00640 1 --2- 172.21.15.59:0/3105339508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9314004800 0x7f9314004c80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f931c001750 tx=0x7f9314005830 comp rx=0 tx=0).stop 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f932e800640 1 -- 172.21.15.59:0/3105339508 reap_dead start 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f9334c00640 1 -- 172.21.15.59:0/3105339508 shutdown_connections 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f9334c00640 1 -- 172.21.15.59:0/3105339508 >> 172.21.15.59:0/3105339508 conn(0x7f9330067ea0 msgr2=0x7f933006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f9334c00640 1 -- 172.21.15.59:0/3105339508 shutdown_connections 2024-04-23T14:36:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.553+0000 7f9334c00640 1 -- 172.21.15.59:0/3105339508 wait complete. 2024-04-23T14:36:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.555+0000 7f9334c00640 1 Processor -- start 2024-04-23T14:36:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.555+0000 7f9334c00640 1 -- start start 2024-04-23T14:36:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.556+0000 7f9334c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9314004800 0x7f9330143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.556+0000 7f9334c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93301513f0 0x7f9330160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.556+0000 7f9334c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9330143ae0 0x7f93301611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.556+0000 7f9334c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93301679d0 con 0x7f93301513f0 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.556+0000 7f9334c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9330167850 con 0x7f9314004800 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.556+0000 7f9334c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9330167b50 con 0x7f9330143ae0 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93301513f0 0x7f9330160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93301513f0 0x7f9330160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41514/0 (socket says 172.21.15.59:41514) 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 -- 172.21.15.59:0/2433584583 learned_addr learned my addr 172.21.15.59:0/2433584583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932de00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9314004800 0x7f9330143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932e800640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9330143ae0 0x7f93301611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 -- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9330143ae0 msgr2=0x7f93301611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9330143ae0 0x7f93301611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 -- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9314004800 msgr2=0x7f9330143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9314004800 0x7f9330143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.557+0000 7f932d400640 1 -- 172.21.15.59:0/2433584583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93301518f0 con 0x7f93301513f0 2024-04-23T14:36:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.558+0000 7f932d400640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93301513f0 0x7f9330160c90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f931c000c00 tx=0x7f931c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.558+0000 7f9326000640 1 -- 172.21.15.59:0/2433584583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c042070 con 0x7f93301513f0 2024-04-23T14:36:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.558+0000 7f9326000640 1 -- 172.21.15.59:0/2433584583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c005480 con 0x7f93301513f0 2024-04-23T14:36:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.558+0000 7f9326000640 1 -- 172.21.15.59:0/2433584583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c037040 con 0x7f93301513f0 2024-04-23T14:36:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.558+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933014b8f0 con 0x7f93301513f0 2024-04-23T14:36:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.558+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933014beb0 con 0x7f93301513f0 2024-04-23T14:36:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.559+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330068a80 con 0x7f93301513f0 2024-04-23T14:36:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.560+0000 7f9326000640 1 -- 172.21.15.59:0/2433584583 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f931c045070 con 0x7f93301513f0 2024-04-23T14:36:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.561+0000 7f9326000640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f931006c920 0x7f931006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.561+0000 7f9326000640 1 -- 172.21.15.59:0/2433584583 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f931c0ab420 con 0x7f93301513f0 2024-04-23T14:36:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.561+0000 7f932de00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f931006c920 0x7f931006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.562+0000 7f932de00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f931006c920 0x7f931006ede0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9320005990 tx=0x7f9320009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:36.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.566+0000 7f9326000640 1 -- 172.21.15.59:0/2433584583 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f931c079720 con 0x7f93301513f0 2024-04-23T14:36:36.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.765+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9330151b60 con 0x7f931006c920 2024-04-23T14:36:36.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.776+0000 7f9326000640 1 -- 172.21.15.59:0/2433584583 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f9330151b60 con 0x7f931006c920 2024-04-23T14:36:36.780 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:36.781 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:36:36.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f931006c920 msgr2=0x7f931006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f931006c920 0x7f931006ede0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9320005990 tx=0x7f9320009040 comp rx=0 tx=0).stop 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93301513f0 msgr2=0x7f9330160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93301513f0 0x7f9330160c90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f931c000c00 tx=0x7f931c001120 comp rx=0 tx=0).stop 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 shutdown_connections 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f931006c920 0x7f931006ede0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9330143ae0 0x7f93301611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93301513f0 0x7f9330160c90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 --2- 172.21.15.59:0/2433584583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9314004800 0x7f9330143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.781+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 >> 172.21.15.59:0/2433584583 conn(0x7f9330067ea0 msgr2=0x7f9330160110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.782+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 shutdown_connections 2024-04-23T14:36:36.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:36.782+0000 7f9334c00640 1 -- 172.21.15.59:0/2433584583 wait complete. 2024-04-23T14:36:36.801 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-04-23T14:36:36.449668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257228,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717492,"statfs":{"total":773094113280,"available":772830711808,"internally_reserved":0,"allocated":7970816,"data_stored":4014812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.221404"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283926+0000","last_change":"2024-04-23T14:36:30.278982+0000","last_active":"2024-04-23T14:36:32.283926+0000","last_peered":"2024-04-23T14:36:32.283926+0000","last_clean":"2024-04-23T14:36:32.283926+0000","last_became_active":"2024-04-23T14:36:30.278687+0000","last_became_peered":"2024-04-23T14:36:30.278687+0000","last_unstale":"2024-04-23T14:36:32.283926+0000","last_undegraded":"2024-04-23T14:36:32.283926+0000","last_fullsized":"2024-04-23T14:36:32.283926+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:04.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.264404+0000","last_change":"2024-04-23T14:36:30.261145+0000","last_active":"2024-04-23T14:36:31.264404+0000","last_peered":"2024-04-23T14:36:31.264404+0000","last_clean":"2024-04-23T14:36:31.264404+0000","last_became_active":"2024-04-23T14:36:30.260396+0000","last_became_peered":"2024-04-23T14:36:30.260396+0000","last_unstale":"2024-04-23T14:36:31.264404+0000","last_undegraded":"2024-04-23T14:36:31.264404+0000","last_fullsized":"2024-04-23T14:36:31.264404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:30:39.839746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283677+0000","last_change":"2024-04-23T14:36:30.254029+0000","last_active":"2024-04-23T14:36:32.283677+0000","last_peered":"2024-04-23T14:36:32.283677+0000","last_clean":"2024-04-23T14:36:32.283677+0000","last_became_active":"2024-04-23T14:36:30.253734+0000","last_became_peered":"2024-04-23T14:36:30.253734+0000","last_unstale":"2024-04-23T14:36:32.283677+0000","last_undegraded":"2024-04-23T14:36:32.283677+0000","last_fullsized":"2024-04-23T14:36:32.283677+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:30.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282308+0000","last_change":"2024-04-23T14:36:30.277554+0000","last_active":"2024-04-23T14:36:32.282308+0000","last_peered":"2024-04-23T14:36:32.282308+0000","last_clean":"2024-04-23T14:36:32.282308+0000","last_became_active":"2024-04-23T14:36:30.277223+0000","last_became_peered":"2024-04-23T14:36:30.277223+0000","last_unstale":"2024-04-23T14:36:32.282308+0000","last_undegraded":"2024-04-23T14:36:32.282308+0000","last_fullsized":"2024-04-23T14:36:32.282308+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:20.415522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256769+0000","last_change":"2024-04-23T14:36:30.271347+0000","last_active":"2024-04-23T14:36:31.256769+0000","last_peered":"2024-04-23T14:36:31.256769+0000","last_clean":"2024-04-23T14:36:31.256769+0000","last_became_active":"2024-04-23T14:36:30.270634+0000","last_became_peered":"2024-04-23T14:36:30.270634+0000","last_unstale":"2024-04-23T14:36:31.256769+0000","last_undegraded":"2024-04-23T14:36:31.256769+0000","last_fullsized":"2024-04-23T14:36:31.256769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:26.664536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263544+0000","last_change":"2024-04-23T14:36:30.265364+0000","last_active":"2024-04-23T14:36:31.263544+0000","last_peered":"2024-04-23T14:36:31.263544+0000","last_clean":"2024-04-23T14:36:31.263544+0000","last_became_active":"2024-04-23T14:36:30.264635+0000","last_became_peered":"2024-04-23T14:36:30.264635+0000","last_unstale":"2024-04-23T14:36:31.263544+0000","last_undegraded":"2024-04-23T14:36:31.263544+0000","last_fullsized":"2024-04-23T14:36:31.263544+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:46.746682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281466+0000","last_change":"2024-04-23T14:36:30.278449+0000","last_active":"2024-04-23T14:36:32.281466+0000","last_peered":"2024-04-23T14:36:32.281466+0000","last_clean":"2024-04-23T14:36:32.281466+0000","last_became_active":"2024-04-23T14:36:30.277159+0000","last_became_peered":"2024-04-23T14:36:30.277159+0000","last_unstale":"2024-04-23T14:36:32.281466+0000","last_undegraded":"2024-04-23T14:36:32.281466+0000","last_fullsized":"2024-04-23T14:36:32.281466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:54:06.102135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261842+0000","last_change":"2024-04-23T14:36:30.273457+0000","last_active":"2024-04-23T14:36:31.261842+0000","last_peered":"2024-04-23T14:36:31.261842+0000","last_clean":"2024-04-23T14:36:31.261842+0000","last_became_active":"2024-04-23T14:36:30.272918+0000","last_became_peered":"2024-04-23T14:36:30.272918+0000","last_unstale":"2024-04-23T14:36:31.261842+0000","last_undegraded":"2024-04-23T14:36:31.261842+0000","last_fullsized":"2024-04-23T14:36:31.261842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:33:00.177900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257521+0000","last_change":"2024-04-23T14:36:30.256388+0000","last_active":"2024-04-23T14:36:31.257521+0000","last_peered":"2024-04-23T14:36:31.257521+0000","last_clean":"2024-04-23T14:36:31.257521+0000","last_became_active":"2024-04-23T14:36:30.255042+0000","last_became_peered":"2024-04-23T14:36:30.255042+0000","last_unstale":"2024-04-23T14:36:31.257521+0000","last_undegraded":"2024-04-23T14:36:31.257521+0000","last_fullsized":"2024-04-23T14:36:31.257521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:38.281474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261953+0000","last_change":"2024-04-23T14:36:30.252958+0000","last_active":"2024-04-23T14:36:31.261953+0000","last_peered":"2024-04-23T14:36:31.261953+0000","last_clean":"2024-04-23T14:36:31.261953+0000","last_became_active":"2024-04-23T14:36:30.252088+0000","last_became_peered":"2024-04-23T14:36:30.252088+0000","last_unstale":"2024-04-23T14:36:31.261953+0000","last_undegraded":"2024-04-23T14:36:31.261953+0000","last_fullsized":"2024-04-23T14:36:31.261953+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:59.184476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283596+0000","last_change":"2024-04-23T14:36:30.281268+0000","last_active":"2024-04-23T14:36:32.283596+0000","last_peered":"2024-04-23T14:36:32.283596+0000","last_clean":"2024-04-23T14:36:32.283596+0000","last_became_active":"2024-04-23T14:36:30.280759+0000","last_became_peered":"2024-04-23T14:36:30.280759+0000","last_unstale":"2024-04-23T14:36:32.283596+0000","last_undegraded":"2024-04-23T14:36:32.283596+0000","last_fullsized":"2024-04-23T14:36:32.283596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:12:49.259305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282620+0000","last_change":"2024-04-23T14:36:30.274529+0000","last_active":"2024-04-23T14:36:32.282620+0000","last_peered":"2024-04-23T14:36:32.282620+0000","last_clean":"2024-04-23T14:36:32.282620+0000","last_became_active":"2024-04-23T14:36:30.273366+0000","last_became_peered":"2024-04-23T14:36:30.273366+0000","last_unstale":"2024-04-23T14:36:32.282620+0000","last_undegraded":"2024-04-23T14:36:32.282620+0000","last_fullsized":"2024-04-23T14:36:32.282620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:27.060798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280665+0000","last_change":"2024-04-23T14:36:30.268167+0000","last_active":"2024-04-23T14:36:32.280665+0000","last_peered":"2024-04-23T14:36:32.280665+0000","last_clean":"2024-04-23T14:36:32.280665+0000","last_became_active":"2024-04-23T14:36:30.267861+0000","last_became_peered":"2024-04-23T14:36:30.267861+0000","last_unstale":"2024-04-23T14:36:32.280665+0000","last_undegraded":"2024-04-23T14:36:32.280665+0000","last_fullsized":"2024-04-23T14:36:32.280665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:01.140299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257406+0000","last_change":"2024-04-23T14:36:30.276001+0000","last_active":"2024-04-23T14:36:31.257406+0000","last_peered":"2024-04-23T14:36:31.257406+0000","last_clean":"2024-04-23T14:36:31.257406+0000","last_became_active":"2024-04-23T14:36:30.271406+0000","last_became_peered":"2024-04-23T14:36:30.271406+0000","last_unstale":"2024-04-23T14:36:31.257406+0000","last_undegraded":"2024-04-23T14:36:31.257406+0000","last_fullsized":"2024-04-23T14:36:31.257406+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:05.035180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280436+0000","last_change":"2024-04-23T14:36:30.262622+0000","last_active":"2024-04-23T14:36:32.280436+0000","last_peered":"2024-04-23T14:36:32.280436+0000","last_clean":"2024-04-23T14:36:32.280436+0000","last_became_active":"2024-04-23T14:36:30.262183+0000","last_became_peered":"2024-04-23T14:36:30.262183+0000","last_unstale":"2024-04-23T14:36:32.280436+0000","last_undegraded":"2024-04-23T14:36:32.280436+0000","last_fullsized":"2024-04-23T14:36:32.280436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:12:49.259305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.260257+0000","last_change":"2024-04-23T14:36:30.270096+0000","last_active":"2024-04-23T14:36:31.260257+0000","last_peered":"2024-04-23T14:36:31.260257+0000","last_clean":"2024-04-23T14:36:31.260257+0000","last_became_active":"2024-04-23T14:36:30.268105+0000","last_became_peered":"2024-04-23T14:36:30.268105+0000","last_unstale":"2024-04-23T14:36:31.260257+0000","last_undegraded":"2024-04-23T14:36:31.260257+0000","last_fullsized":"2024-04-23T14:36:31.260257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:51:47.737900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257673+0000","last_change":"2024-04-23T14:36:30.257389+0000","last_active":"2024-04-23T14:36:31.257673+0000","last_peered":"2024-04-23T14:36:31.257673+0000","last_clean":"2024-04-23T14:36:31.257673+0000","last_became_active":"2024-04-23T14:36:30.256095+0000","last_became_peered":"2024-04-23T14:36:30.256095+0000","last_unstale":"2024-04-23T14:36:31.257673+0000","last_undegraded":"2024-04-23T14:36:31.257673+0000","last_fullsized":"2024-04-23T14:36:31.257673+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:48.658488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282665+0000","last_change":"2024-04-23T14:36:30.283652+0000","last_active":"2024-04-23T14:36:32.282665+0000","last_peered":"2024-04-23T14:36:32.282665+0000","last_clean":"2024-04-23T14:36:32.282665+0000","last_became_active":"2024-04-23T14:36:30.283367+0000","last_became_peered":"2024-04-23T14:36:30.283367+0000","last_unstale":"2024-04-23T14:36:32.282665+0000","last_undegraded":"2024-04-23T14:36:32.282665+0000","last_fullsized":"2024-04-23T14:36:32.282665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:13.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263040+0000","last_change":"2024-04-23T14:36:30.272297+0000","last_active":"2024-04-23T14:36:31.263040+0000","last_peered":"2024-04-23T14:36:31.263040+0000","last_clean":"2024-04-23T14:36:31.263040+0000","last_became_active":"2024-04-23T14:36:30.268811+0000","last_became_peered":"2024-04-23T14:36:30.268811+0000","last_unstale":"2024-04-23T14:36:31.263040+0000","last_undegraded":"2024-04-23T14:36:31.263040+0000","last_fullsized":"2024-04-23T14:36:31.263040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:59.184476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280641+0000","last_change":"2024-04-23T14:36:30.267969+0000","last_active":"2024-04-23T14:36:32.280641+0000","last_peered":"2024-04-23T14:36:32.280641+0000","last_clean":"2024-04-23T14:36:32.280641+0000","last_became_active":"2024-04-23T14:36:30.267591+0000","last_became_peered":"2024-04-23T14:36:30.267591+0000","last_unstale":"2024-04-23T14:36:32.280641+0000","last_undegraded":"2024-04-23T14:36:32.280641+0000","last_fullsized":"2024-04-23T14:36:32.280641+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:23:34.977231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261116+0000","last_change":"2024-04-23T14:36:30.256885+0000","last_active":"2024-04-23T14:36:31.261116+0000","last_peered":"2024-04-23T14:36:31.261116+0000","last_clean":"2024-04-23T14:36:31.261116+0000","last_became_active":"2024-04-23T14:36:30.256110+0000","last_became_peered":"2024-04-23T14:36:30.256110+0000","last_unstale":"2024-04-23T14:36:31.261116+0000","last_undegraded":"2024-04-23T14:36:31.261116+0000","last_fullsized":"2024-04-23T14:36:31.261116+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:33:00.177900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261104+0000","last_change":"2024-04-23T14:36:30.253926+0000","last_active":"2024-04-23T14:36:31.261104+0000","last_peered":"2024-04-23T14:36:31.261104+0000","last_clean":"2024-04-23T14:36:31.261104+0000","last_became_active":"2024-04-23T14:36:30.253472+0000","last_became_peered":"2024-04-23T14:36:30.253472+0000","last_unstale":"2024-04-23T14:36:31.261104+0000","last_undegraded":"2024-04-23T14:36:31.261104+0000","last_fullsized":"2024-04-23T14:36:31.261104+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:53.136057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283019+0000","last_change":"2024-04-23T14:36:30.279100+0000","last_active":"2024-04-23T14:36:32.283019+0000","last_peered":"2024-04-23T14:36:32.283019+0000","last_clean":"2024-04-23T14:36:32.283019+0000","last_became_active":"2024-04-23T14:36:30.276984+0000","last_became_peered":"2024-04-23T14:36:30.276984+0000","last_unstale":"2024-04-23T14:36:32.283019+0000","last_undegraded":"2024-04-23T14:36:32.283019+0000","last_fullsized":"2024-04-23T14:36:32.283019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:23:34.977231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282250+0000","last_change":"2024-04-23T14:36:30.284096+0000","last_active":"2024-04-23T14:36:32.282250+0000","last_peered":"2024-04-23T14:36:32.282250+0000","last_clean":"2024-04-23T14:36:32.282250+0000","last_became_active":"2024-04-23T14:36:30.283751+0000","last_became_peered":"2024-04-23T14:36:30.283751+0000","last_unstale":"2024-04-23T14:36:32.282250+0000","last_undegraded":"2024-04-23T14:36:32.282250+0000","last_fullsized":"2024-04-23T14:36:32.282250+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:52.453599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262620+0000","last_change":"2024-04-23T14:36:30.273502+0000","last_active":"2024-04-23T14:36:31.262620+0000","last_peered":"2024-04-23T14:36:31.262620+0000","last_clean":"2024-04-23T14:36:31.262620+0000","last_became_active":"2024-04-23T14:36:30.273238+0000","last_became_peered":"2024-04-23T14:36:30.273238+0000","last_unstale":"2024-04-23T14:36:31.262620+0000","last_undegraded":"2024-04-23T14:36:31.262620+0000","last_fullsized":"2024-04-23T14:36:31.262620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:46.746682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282358+0000","last_change":"2024-04-23T14:36:30.276366+0000","last_active":"2024-04-23T14:36:32.282358+0000","last_peered":"2024-04-23T14:36:32.282358+0000","last_clean":"2024-04-23T14:36:32.282358+0000","last_became_active":"2024-04-23T14:36:30.276016+0000","last_became_peered":"2024-04-23T14:36:30.276016+0000","last_unstale":"2024-04-23T14:36:32.282358+0000","last_undegraded":"2024-04-23T14:36:32.282358+0000","last_fullsized":"2024-04-23T14:36:32.282358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:20.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283860+0000","last_change":"2024-04-23T14:36:30.275939+0000","last_active":"2024-04-23T14:36:32.283860+0000","last_peered":"2024-04-23T14:36:32.283860+0000","last_clean":"2024-04-23T14:36:32.283860+0000","last_became_active":"2024-04-23T14:36:30.275646+0000","last_became_peered":"2024-04-23T14:36:30.275646+0000","last_unstale":"2024-04-23T14:36:32.283860+0000","last_undegraded":"2024-04-23T14:36:32.283860+0000","last_fullsized":"2024-04-23T14:36:32.283860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:45.735264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261823+0000","last_change":"2024-04-23T14:36:30.252644+0000","last_active":"2024-04-23T14:36:31.261823+0000","last_peered":"2024-04-23T14:36:31.261823+0000","last_clean":"2024-04-23T14:36:31.261823+0000","last_became_active":"2024-04-23T14:36:30.251743+0000","last_became_peered":"2024-04-23T14:36:30.251743+0000","last_unstale":"2024-04-23T14:36:31.261823+0000","last_undegraded":"2024-04-23T14:36:31.261823+0000","last_fullsized":"2024-04-23T14:36:31.261823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:01:39.745735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283514+0000","last_change":"2024-04-23T14:36:30.264701+0000","last_active":"2024-04-23T14:36:32.283514+0000","last_peered":"2024-04-23T14:36:32.283514+0000","last_clean":"2024-04-23T14:36:32.283514+0000","last_became_active":"2024-04-23T14:36:30.264443+0000","last_became_peered":"2024-04-23T14:36:30.264443+0000","last_unstale":"2024-04-23T14:36:32.283514+0000","last_undegraded":"2024-04-23T14:36:32.283514+0000","last_fullsized":"2024-04-23T14:36:32.283514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:53.571110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281939+0000","last_change":"2024-04-23T14:36:30.276300+0000","last_active":"2024-04-23T14:36:32.281939+0000","last_peered":"2024-04-23T14:36:32.281939+0000","last_clean":"2024-04-23T14:36:32.281939+0000","last_became_active":"2024-04-23T14:36:30.275375+0000","last_became_peered":"2024-04-23T14:36:30.275375+0000","last_unstale":"2024-04-23T14:36:32.281939+0000","last_undegraded":"2024-04-23T14:36:32.281939+0000","last_fullsized":"2024-04-23T14:36:32.281939+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:52.453599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281960+0000","last_change":"2024-04-23T14:36:30.273517+0000","last_active":"2024-04-23T14:36:32.281960+0000","last_peered":"2024-04-23T14:36:32.281960+0000","last_clean":"2024-04-23T14:36:32.281960+0000","last_became_active":"2024-04-23T14:36:30.271844+0000","last_became_peered":"2024-04-23T14:36:30.271844+0000","last_unstale":"2024-04-23T14:36:32.281960+0000","last_undegraded":"2024-04-23T14:36:32.281960+0000","last_fullsized":"2024-04-23T14:36:32.281960+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:23:34.977231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262188+0000","last_change":"2024-04-23T14:36:30.260056+0000","last_active":"2024-04-23T14:36:31.262188+0000","last_peered":"2024-04-23T14:36:31.262188+0000","last_clean":"2024-04-23T14:36:31.262188+0000","last_became_active":"2024-04-23T14:36:30.259502+0000","last_became_peered":"2024-04-23T14:36:30.259502+0000","last_unstale":"2024-04-23T14:36:31.262188+0000","last_undegraded":"2024-04-23T14:36:31.262188+0000","last_fullsized":"2024-04-23T14:36:31.262188+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.191748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284052+0000","last_change":"2024-04-23T14:36:30.285282+0000","last_active":"2024-04-23T14:36:32.284052+0000","last_peered":"2024-04-23T14:36:32.284052+0000","last_clean":"2024-04-23T14:36:32.284052+0000","last_became_active":"2024-04-23T14:36:30.284931+0000","last_became_peered":"2024-04-23T14:36:30.284931+0000","last_unstale":"2024-04-23T14:36:32.284052+0000","last_undegraded":"2024-04-23T14:36:32.284052+0000","last_fullsized":"2024-04-23T14:36:32.284052+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:50.195282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284490+0000","last_change":"2024-04-23T14:36:29.265430+0000","last_active":"2024-04-23T14:36:32.284490+0000","last_peered":"2024-04-23T14:36:32.284490+0000","last_clean":"2024-04-23T14:36:32.284490+0000","last_became_active":"2024-04-23T14:36:29.265096+0000","last_became_peered":"2024-04-23T14:36:29.265096+0000","last_unstale":"2024-04-23T14:36:32.284490+0000","last_undegraded":"2024-04-23T14:36:32.284490+0000","last_fullsized":"2024-04-23T14:36:32.284490+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:18.617182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257718+0000","last_change":"2024-04-23T14:36:30.257027+0000","last_active":"2024-04-23T14:36:31.257718+0000","last_peered":"2024-04-23T14:36:31.257718+0000","last_clean":"2024-04-23T14:36:31.257718+0000","last_became_active":"2024-04-23T14:36:30.255514+0000","last_became_peered":"2024-04-23T14:36:30.255514+0000","last_unstale":"2024-04-23T14:36:31.257718+0000","last_undegraded":"2024-04-23T14:36:31.257718+0000","last_fullsized":"2024-04-23T14:36:31.257718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:21.165754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281267+0000","last_change":"2024-04-23T14:36:29.255962+0000","last_active":"2024-04-23T14:36:32.281267+0000","last_peered":"2024-04-23T14:36:32.281267+0000","last_clean":"2024-04-23T14:36:32.281267+0000","last_became_active":"2024-04-23T14:36:29.255566+0000","last_became_peered":"2024-04-23T14:36:29.255566+0000","last_unstale":"2024-04-23T14:36:32.281267+0000","last_undegraded":"2024-04-23T14:36:32.281267+0000","last_fullsized":"2024-04-23T14:36:32.281267+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:25:19.305134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261357+0000","last_change":"2024-04-23T14:36:30.271373+0000","last_active":"2024-04-23T14:36:31.261357+0000","last_peered":"2024-04-23T14:36:31.261357+0000","last_clean":"2024-04-23T14:36:31.261357+0000","last_became_active":"2024-04-23T14:36:30.269418+0000","last_became_peered":"2024-04-23T14:36:30.269418+0000","last_unstale":"2024-04-23T14:36:31.261357+0000","last_undegraded":"2024-04-23T14:36:31.261357+0000","last_fullsized":"2024-04-23T14:36:31.261357+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:53.136057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262504+0000","last_change":"2024-04-23T14:36:29.264940+0000","last_active":"2024-04-23T14:36:31.262504+0000","last_peered":"2024-04-23T14:36:31.262504+0000","last_clean":"2024-04-23T14:36:31.262504+0000","last_became_active":"2024-04-23T14:36:29.264568+0000","last_became_peered":"2024-04-23T14:36:29.264568+0000","last_unstale":"2024-04-23T14:36:31.262504+0000","last_undegraded":"2024-04-23T14:36:31.262504+0000","last_fullsized":"2024-04-23T14:36:31.262504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:31.008207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283726+0000","last_change":"2024-04-23T14:36:30.273278+0000","last_active":"2024-04-23T14:36:32.283726+0000","last_peered":"2024-04-23T14:36:32.283726+0000","last_clean":"2024-04-23T14:36:32.283726+0000","last_became_active":"2024-04-23T14:36:30.272951+0000","last_became_peered":"2024-04-23T14:36:30.272951+0000","last_unstale":"2024-04-23T14:36:32.283726+0000","last_undegraded":"2024-04-23T14:36:32.283726+0000","last_fullsized":"2024-04-23T14:36:32.283726+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:52.345173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263306+0000","last_change":"2024-04-23T14:36:29.258194+0000","last_active":"2024-04-23T14:36:31.263306+0000","last_peered":"2024-04-23T14:36:31.263306+0000","last_clean":"2024-04-23T14:36:31.263306+0000","last_became_active":"2024-04-23T14:36:29.257888+0000","last_became_peered":"2024-04-23T14:36:29.257888+0000","last_unstale":"2024-04-23T14:36:31.263306+0000","last_undegraded":"2024-04-23T14:36:31.263306+0000","last_fullsized":"2024-04-23T14:36:31.263306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:41.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282433+0000","last_change":"2024-04-23T14:36:30.274986+0000","last_active":"2024-04-23T14:36:32.282433+0000","last_peered":"2024-04-23T14:36:32.282433+0000","last_clean":"2024-04-23T14:36:32.282433+0000","last_became_active":"2024-04-23T14:36:30.274524+0000","last_became_peered":"2024-04-23T14:36:30.274524+0000","last_unstale":"2024-04-23T14:36:32.282433+0000","last_undegraded":"2024-04-23T14:36:32.282433+0000","last_fullsized":"2024-04-23T14:36:32.282433+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:12:49.259305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283810+0000","last_change":"2024-04-23T14:36:29.259191+0000","last_active":"2024-04-23T14:36:32.283810+0000","last_peered":"2024-04-23T14:36:32.283810+0000","last_clean":"2024-04-23T14:36:32.283810+0000","last_became_active":"2024-04-23T14:36:29.258420+0000","last_became_peered":"2024-04-23T14:36:29.258420+0000","last_unstale":"2024-04-23T14:36:32.283810+0000","last_undegraded":"2024-04-23T14:36:32.283810+0000","last_fullsized":"2024-04-23T14:36:32.283810+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:21.800400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261208+0000","last_change":"2024-04-23T14:36:30.277342+0000","last_active":"2024-04-23T14:36:31.261208+0000","last_peered":"2024-04-23T14:36:31.261208+0000","last_clean":"2024-04-23T14:36:31.261208+0000","last_became_active":"2024-04-23T14:36:30.272999+0000","last_became_peered":"2024-04-23T14:36:30.272999+0000","last_unstale":"2024-04-23T14:36:31.261208+0000","last_undegraded":"2024-04-23T14:36:31.261208+0000","last_fullsized":"2024-04-23T14:36:31.261208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.191748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263657+0000","last_change":"2024-04-23T14:36:29.255423+0000","last_active":"2024-04-23T14:36:31.263657+0000","last_peered":"2024-04-23T14:36:31.263657+0000","last_clean":"2024-04-23T14:36:31.263657+0000","last_became_active":"2024-04-23T14:36:29.255130+0000","last_became_peered":"2024-04-23T14:36:29.255130+0000","last_unstale":"2024-04-23T14:36:31.263657+0000","last_undegraded":"2024-04-23T14:36:31.263657+0000","last_fullsized":"2024-04-23T14:36:31.263657+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256577+0000","last_change":"2024-04-23T14:36:30.259813+0000","last_active":"2024-04-23T14:36:31.256577+0000","last_peered":"2024-04-23T14:36:31.256577+0000","last_clean":"2024-04-23T14:36:31.256577+0000","last_became_active":"2024-04-23T14:36:30.259020+0000","last_became_peered":"2024-04-23T14:36:30.259020+0000","last_unstale":"2024-04-23T14:36:31.256577+0000","last_undegraded":"2024-04-23T14:36:31.256577+0000","last_fullsized":"2024-04-23T14:36:31.256577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:28.217244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282018+0000","last_change":"2024-04-23T14:36:29.262680+0000","last_active":"2024-04-23T14:36:32.282018+0000","last_peered":"2024-04-23T14:36:32.282018+0000","last_clean":"2024-04-23T14:36:32.282018+0000","last_became_active":"2024-04-23T14:36:29.262177+0000","last_became_peered":"2024-04-23T14:36:29.262177+0000","last_unstale":"2024-04-23T14:36:32.282018+0000","last_undegraded":"2024-04-23T14:36:32.282018+0000","last_fullsized":"2024-04-23T14:36:32.282018+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:16:40.504300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280643+0000","last_change":"2024-04-23T14:36:30.272307+0000","last_active":"2024-04-23T14:36:32.280643+0000","last_peered":"2024-04-23T14:36:32.280643+0000","last_clean":"2024-04-23T14:36:32.280643+0000","last_became_active":"2024-04-23T14:36:30.271993+0000","last_became_peered":"2024-04-23T14:36:30.271993+0000","last_unstale":"2024-04-23T14:36:32.280643+0000","last_undegraded":"2024-04-23T14:36:32.280643+0000","last_fullsized":"2024-04-23T14:36:32.280643+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:20.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256559+0000","last_change":"2024-04-23T14:36:29.255068+0000","last_active":"2024-04-23T14:36:31.256559+0000","last_peered":"2024-04-23T14:36:31.256559+0000","last_clean":"2024-04-23T14:36:31.256559+0000","last_became_active":"2024-04-23T14:36:29.254465+0000","last_became_peered":"2024-04-23T14:36:29.254465+0000","last_unstale":"2024-04-23T14:36:31.256559+0000","last_undegraded":"2024-04-23T14:36:31.256559+0000","last_fullsized":"2024-04-23T14:36:31.256559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:47.763347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282438+0000","last_change":"2024-04-23T14:36:30.272697+0000","last_active":"2024-04-23T14:36:32.282438+0000","last_peered":"2024-04-23T14:36:32.282438+0000","last_clean":"2024-04-23T14:36:32.282438+0000","last_became_active":"2024-04-23T14:36:30.272332+0000","last_became_peered":"2024-04-23T14:36:30.272332+0000","last_unstale":"2024-04-23T14:36:32.282438+0000","last_undegraded":"2024-04-23T14:36:32.282438+0000","last_fullsized":"2024-04-23T14:36:32.282438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:52.345173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262506+0000","last_change":"2024-04-23T14:36:29.262461+0000","last_active":"2024-04-23T14:36:31.262506+0000","last_peered":"2024-04-23T14:36:31.262506+0000","last_clean":"2024-04-23T14:36:31.262506+0000","last_became_active":"2024-04-23T14:36:29.262134+0000","last_became_peered":"2024-04-23T14:36:29.262134+0000","last_unstale":"2024-04-23T14:36:31.262506+0000","last_undegraded":"2024-04-23T14:36:31.262506+0000","last_fullsized":"2024-04-23T14:36:31.262506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:07:00.944096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284214+0000","last_change":"2024-04-23T14:36:30.284590+0000","last_active":"2024-04-23T14:36:32.284214+0000","last_peered":"2024-04-23T14:36:32.284214+0000","last_clean":"2024-04-23T14:36:32.284214+0000","last_became_active":"2024-04-23T14:36:30.283741+0000","last_became_peered":"2024-04-23T14:36:30.283741+0000","last_unstale":"2024-04-23T14:36:32.284214+0000","last_undegraded":"2024-04-23T14:36:32.284214+0000","last_fullsized":"2024-04-23T14:36:32.284214+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:22.818538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.264495+0000","last_change":"2024-04-23T14:36:29.252722+0000","last_active":"2024-04-23T14:36:31.264495+0000","last_peered":"2024-04-23T14:36:31.264495+0000","last_clean":"2024-04-23T14:36:31.264495+0000","last_became_active":"2024-04-23T14:36:29.252421+0000","last_became_peered":"2024-04-23T14:36:29.252421+0000","last_unstale":"2024-04-23T14:36:31.264495+0000","last_undegraded":"2024-04-23T14:36:31.264495+0000","last_fullsized":"2024-04-23T14:36:31.264495+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:18:16.720971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262962+0000","last_change":"2024-04-23T14:36:30.263935+0000","last_active":"2024-04-23T14:36:31.262962+0000","last_peered":"2024-04-23T14:36:31.262962+0000","last_clean":"2024-04-23T14:36:31.262962+0000","last_became_active":"2024-04-23T14:36:30.263590+0000","last_became_peered":"2024-04-23T14:36:30.263590+0000","last_unstale":"2024-04-23T14:36:31.262962+0000","last_undegraded":"2024-04-23T14:36:31.262962+0000","last_fullsized":"2024-04-23T14:36:31.262962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:01:39.745735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283913+0000","last_change":"2024-04-23T14:36:29.255338+0000","last_active":"2024-04-23T14:36:32.283913+0000","last_peered":"2024-04-23T14:36:32.283913+0000","last_clean":"2024-04-23T14:36:32.283913+0000","last_became_active":"2024-04-23T14:36:29.254916+0000","last_became_peered":"2024-04-23T14:36:29.254916+0000","last_unstale":"2024-04-23T14:36:32.283913+0000","last_undegraded":"2024-04-23T14:36:32.283913+0000","last_fullsized":"2024-04-23T14:36:32.283913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:03.094079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257203+0000","last_change":"2024-04-23T14:36:30.267547+0000","last_active":"2024-04-23T14:36:31.257203+0000","last_peered":"2024-04-23T14:36:31.257203+0000","last_clean":"2024-04-23T14:36:31.257203+0000","last_became_active":"2024-04-23T14:36:30.267103+0000","last_became_peered":"2024-04-23T14:36:30.267103+0000","last_unstale":"2024-04-23T14:36:31.257203+0000","last_undegraded":"2024-04-23T14:36:31.257203+0000","last_fullsized":"2024-04-23T14:36:31.257203+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:14.787860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282997+0000","last_change":"2024-04-23T14:36:29.257996+0000","last_active":"2024-04-23T14:36:32.282997+0000","last_peered":"2024-04-23T14:36:32.282997+0000","last_clean":"2024-04-23T14:36:32.282997+0000","last_became_active":"2024-04-23T14:36:29.256591+0000","last_became_peered":"2024-04-23T14:36:29.256591+0000","last_unstale":"2024-04-23T14:36:32.282997+0000","last_undegraded":"2024-04-23T14:36:32.282997+0000","last_fullsized":"2024-04-23T14:36:32.282997+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:18:16.720971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283084+0000","last_change":"2024-04-23T14:36:30.279080+0000","last_active":"2024-04-23T14:36:32.283084+0000","last_peered":"2024-04-23T14:36:32.283084+0000","last_clean":"2024-04-23T14:36:32.283084+0000","last_became_active":"2024-04-23T14:36:30.278726+0000","last_became_peered":"2024-04-23T14:36:30.278726+0000","last_unstale":"2024-04-23T14:36:32.283084+0000","last_undegraded":"2024-04-23T14:36:32.283084+0000","last_fullsized":"2024-04-23T14:36:32.283084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:01.140299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282322+0000","last_change":"2024-04-23T14:36:29.262784+0000","last_active":"2024-04-23T14:36:32.282322+0000","last_peered":"2024-04-23T14:36:32.282322+0000","last_clean":"2024-04-23T14:36:32.282322+0000","last_became_active":"2024-04-23T14:36:29.262418+0000","last_became_peered":"2024-04-23T14:36:29.262418+0000","last_unstale":"2024-04-23T14:36:32.282322+0000","last_undegraded":"2024-04-23T14:36:32.282322+0000","last_fullsized":"2024-04-23T14:36:32.282322+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:41.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282301+0000","last_change":"2024-04-23T14:36:30.276664+0000","last_active":"2024-04-23T14:36:32.282301+0000","last_peered":"2024-04-23T14:36:32.282301+0000","last_clean":"2024-04-23T14:36:32.282301+0000","last_became_active":"2024-04-23T14:36:30.276372+0000","last_became_peered":"2024-04-23T14:36:30.276372+0000","last_unstale":"2024-04-23T14:36:32.282301+0000","last_undegraded":"2024-04-23T14:36:32.282301+0000","last_fullsized":"2024-04-23T14:36:32.282301+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:01.140299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263742+0000","last_change":"2024-04-23T14:36:29.256582+0000","last_active":"2024-04-23T14:36:31.263742+0000","last_peered":"2024-04-23T14:36:31.263742+0000","last_clean":"2024-04-23T14:36:31.263742+0000","last_became_active":"2024-04-23T14:36:29.256299+0000","last_became_peered":"2024-04-23T14:36:29.256299+0000","last_unstale":"2024-04-23T14:36:31.263742+0000","last_undegraded":"2024-04-23T14:36:31.263742+0000","last_fullsized":"2024-04-23T14:36:31.263742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280325+0000","last_change":"2024-04-23T14:36:30.270953+0000","last_active":"2024-04-23T14:36:32.280325+0000","last_peered":"2024-04-23T14:36:32.280325+0000","last_clean":"2024-04-23T14:36:32.280325+0000","last_became_active":"2024-04-23T14:36:30.270639+0000","last_became_peered":"2024-04-23T14:36:30.270639+0000","last_unstale":"2024-04-23T14:36:32.280325+0000","last_undegraded":"2024-04-23T14:36:32.280325+0000","last_fullsized":"2024-04-23T14:36:32.280325+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:52.453599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281535+0000","last_change":"2024-04-23T14:36:29.254735+0000","last_active":"2024-04-23T14:36:32.281535+0000","last_peered":"2024-04-23T14:36:32.281535+0000","last_clean":"2024-04-23T14:36:32.281535+0000","last_became_active":"2024-04-23T14:36:29.253176+0000","last_became_peered":"2024-04-23T14:36:29.253176+0000","last_unstale":"2024-04-23T14:36:32.281535+0000","last_undegraded":"2024-04-23T14:36:32.281535+0000","last_fullsized":"2024-04-23T14:36:32.281535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:30:38.821608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256827+0000","last_change":"2024-04-23T14:36:30.254759+0000","last_active":"2024-04-23T14:36:31.256827+0000","last_peered":"2024-04-23T14:36:31.256827+0000","last_clean":"2024-04-23T14:36:31.256827+0000","last_became_active":"2024-04-23T14:36:30.254477+0000","last_became_peered":"2024-04-23T14:36:30.254477+0000","last_unstale":"2024-04-23T14:36:31.256827+0000","last_undegraded":"2024-04-23T14:36:31.256827+0000","last_fullsized":"2024-04-23T14:36:31.256827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:15.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.260271+0000","last_change":"2024-04-23T14:36:29.263312+0000","last_active":"2024-04-23T14:36:31.260271+0000","last_peered":"2024-04-23T14:36:31.260271+0000","last_clean":"2024-04-23T14:36:31.260271+0000","last_became_active":"2024-04-23T14:36:29.262939+0000","last_became_peered":"2024-04-23T14:36:29.262939+0000","last_unstale":"2024-04-23T14:36:31.260271+0000","last_undegraded":"2024-04-23T14:36:31.260271+0000","last_fullsized":"2024-04-23T14:36:31.260271+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:41.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262105+0000","last_change":"2024-04-23T14:36:30.267339+0000","last_active":"2024-04-23T14:36:31.262105+0000","last_peered":"2024-04-23T14:36:31.262105+0000","last_clean":"2024-04-23T14:36:31.262105+0000","last_became_active":"2024-04-23T14:36:30.267009+0000","last_became_peered":"2024-04-23T14:36:30.267009+0000","last_unstale":"2024-04-23T14:36:31.262105+0000","last_undegraded":"2024-04-23T14:36:31.262105+0000","last_fullsized":"2024-04-23T14:36:31.262105+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:30.768940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263672+0000","last_change":"2024-04-23T14:36:29.251466+0000","last_active":"2024-04-23T14:36:31.263672+0000","last_peered":"2024-04-23T14:36:31.263672+0000","last_clean":"2024-04-23T14:36:31.263672+0000","last_became_active":"2024-04-23T14:36:29.250985+0000","last_became_peered":"2024-04-23T14:36:29.250985+0000","last_unstale":"2024-04-23T14:36:31.263672+0000","last_undegraded":"2024-04-23T14:36:31.263672+0000","last_fullsized":"2024-04-23T14:36:31.263672+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:56:10.745359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284367+0000","last_change":"2024-04-23T14:36:30.275700+0000","last_active":"2024-04-23T14:36:32.284367+0000","last_peered":"2024-04-23T14:36:32.284367+0000","last_clean":"2024-04-23T14:36:32.284367+0000","last_became_active":"2024-04-23T14:36:30.275441+0000","last_became_peered":"2024-04-23T14:36:30.275441+0000","last_unstale":"2024-04-23T14:36:32.284367+0000","last_undegraded":"2024-04-23T14:36:32.284367+0000","last_fullsized":"2024-04-23T14:36:32.284367+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:04.112217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262179+0000","last_change":"2024-04-23T14:36:29.257682+0000","last_active":"2024-04-23T14:36:31.262179+0000","last_peered":"2024-04-23T14:36:31.262179+0000","last_clean":"2024-04-23T14:36:31.262179+0000","last_became_active":"2024-04-23T14:36:29.257385+0000","last_became_peered":"2024-04-23T14:36:29.257385+0000","last_unstale":"2024-04-23T14:36:31.262179+0000","last_undegraded":"2024-04-23T14:36:31.262179+0000","last_fullsized":"2024-04-23T14:36:31.262179+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:00.075116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282653+0000","last_change":"2024-04-23T14:36:30.274308+0000","last_active":"2024-04-23T14:36:32.282653+0000","last_peered":"2024-04-23T14:36:32.282653+0000","last_clean":"2024-04-23T14:36:32.282653+0000","last_became_active":"2024-04-23T14:36:30.273351+0000","last_became_peered":"2024-04-23T14:36:30.273351+0000","last_unstale":"2024-04-23T14:36:32.282653+0000","last_undegraded":"2024-04-23T14:36:32.282653+0000","last_fullsized":"2024-04-23T14:36:32.282653+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:20.138211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263315+0000","last_change":"2024-04-23T14:36:29.262796+0000","last_active":"2024-04-23T14:36:31.263315+0000","last_peered":"2024-04-23T14:36:31.263315+0000","last_clean":"2024-04-23T14:36:31.263315+0000","last_became_active":"2024-04-23T14:36:29.262504+0000","last_became_peered":"2024-04-23T14:36:29.262504+0000","last_unstale":"2024-04-23T14:36:31.263315+0000","last_undegraded":"2024-04-23T14:36:31.263315+0000","last_fullsized":"2024-04-23T14:36:31.263315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:45.779744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257317+0000","last_change":"2024-04-23T14:36:30.260184+0000","last_active":"2024-04-23T14:36:31.257317+0000","last_peered":"2024-04-23T14:36:31.257317+0000","last_clean":"2024-04-23T14:36:31.257317+0000","last_became_active":"2024-04-23T14:36:30.259450+0000","last_became_peered":"2024-04-23T14:36:30.259450+0000","last_unstale":"2024-04-23T14:36:31.257317+0000","last_undegraded":"2024-04-23T14:36:31.257317+0000","last_fullsized":"2024-04-23T14:36:31.257317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:36.211595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283570+0000","last_change":"2024-04-23T14:36:29.267357+0000","last_active":"2024-04-23T14:36:32.283570+0000","last_peered":"2024-04-23T14:36:32.283570+0000","last_clean":"2024-04-23T14:36:32.283570+0000","last_became_active":"2024-04-23T14:36:29.266992+0000","last_became_peered":"2024-04-23T14:36:29.266992+0000","last_unstale":"2024-04-23T14:36:32.283570+0000","last_undegraded":"2024-04-23T14:36:32.283570+0000","last_fullsized":"2024-04-23T14:36:32.283570+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:30:38.821608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283689+0000","last_change":"2024-04-23T14:36:30.277980+0000","last_active":"2024-04-23T14:36:32.283689+0000","last_peered":"2024-04-23T14:36:32.283689+0000","last_clean":"2024-04-23T14:36:32.283689+0000","last_became_active":"2024-04-23T14:36:30.277796+0000","last_became_peered":"2024-04-23T14:36:30.277796+0000","last_unstale":"2024-04-23T14:36:32.283689+0000","last_undegraded":"2024-04-23T14:36:32.283689+0000","last_fullsized":"2024-04-23T14:36:32.283689+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:44.683693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263815+0000","last_change":"2024-04-23T14:36:29.254896+0000","last_active":"2024-04-23T14:36:31.263815+0000","last_peered":"2024-04-23T14:36:31.263815+0000","last_clean":"2024-04-23T14:36:31.263815+0000","last_became_active":"2024-04-23T14:36:29.254593+0000","last_became_peered":"2024-04-23T14:36:29.254593+0000","last_unstale":"2024-04-23T14:36:31.263815+0000","last_undegraded":"2024-04-23T14:36:31.263815+0000","last_fullsized":"2024-04-23T14:36:31.263815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:16:40.504300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283215+0000","last_change":"2024-04-23T14:36:30.263085+0000","last_active":"2024-04-23T14:36:32.283215+0000","last_peered":"2024-04-23T14:36:32.283215+0000","last_clean":"2024-04-23T14:36:32.283215+0000","last_became_active":"2024-04-23T14:36:30.262877+0000","last_became_peered":"2024-04-23T14:36:30.262877+0000","last_unstale":"2024-04-23T14:36:32.283215+0000","last_undegraded":"2024-04-23T14:36:32.283215+0000","last_fullsized":"2024-04-23T14:36:32.283215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:22.180698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281814+0000","last_change":"2024-04-23T14:36:29.259253+0000","last_active":"2024-04-23T14:36:32.281814+0000","last_peered":"2024-04-23T14:36:32.281814+0000","last_clean":"2024-04-23T14:36:32.281814+0000","last_became_active":"2024-04-23T14:36:29.256871+0000","last_became_peered":"2024-04-23T14:36:29.256871+0000","last_unstale":"2024-04-23T14:36:32.281814+0000","last_undegraded":"2024-04-23T14:36:32.281814+0000","last_fullsized":"2024-04-23T14:36:32.281814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:18.617182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283551+0000","last_change":"2024-04-23T14:36:30.268952+0000","last_active":"2024-04-23T14:36:32.283551+0000","last_peered":"2024-04-23T14:36:32.283551+0000","last_clean":"2024-04-23T14:36:32.283551+0000","last_became_active":"2024-04-23T14:36:30.268388+0000","last_became_peered":"2024-04-23T14:36:30.268388+0000","last_unstale":"2024-04-23T14:36:32.283551+0000","last_undegraded":"2024-04-23T14:36:32.283551+0000","last_fullsized":"2024-04-23T14:36:32.283551+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:42.895182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280140+0000","last_change":"2024-04-23T14:36:29.253692+0000","last_active":"2024-04-23T14:36:32.280140+0000","last_peered":"2024-04-23T14:36:32.280140+0000","last_clean":"2024-04-23T14:36:32.280140+0000","last_became_active":"2024-04-23T14:36:29.253295+0000","last_became_peered":"2024-04-23T14:36:29.253295+0000","last_unstale":"2024-04-23T14:36:32.280140+0000","last_undegraded":"2024-04-23T14:36:32.280140+0000","last_fullsized":"2024-04-23T14:36:32.280140+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:48.479785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281451+0000","last_change":"2024-04-23T14:36:30.255727+0000","last_active":"2024-04-23T14:36:32.281451+0000","last_peered":"2024-04-23T14:36:32.281451+0000","last_clean":"2024-04-23T14:36:32.281451+0000","last_became_active":"2024-04-23T14:36:30.255347+0000","last_became_peered":"2024-04-23T14:36:30.255347+0000","last_unstale":"2024-04-23T14:36:32.281451+0000","last_undegraded":"2024-04-23T14:36:32.281451+0000","last_fullsized":"2024-04-23T14:36:32.281451+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:52.345173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263834+0000","last_change":"2024-04-23T14:36:29.249580+0000","last_active":"2024-04-23T14:36:31.263834+0000","last_peered":"2024-04-23T14:36:31.263834+0000","last_clean":"2024-04-23T14:36:31.263834+0000","last_became_active":"2024-04-23T14:36:29.249248+0000","last_became_peered":"2024-04-23T14:36:29.249248+0000","last_unstale":"2024-04-23T14:36:31.263834+0000","last_undegraded":"2024-04-23T14:36:31.263834+0000","last_fullsized":"2024-04-23T14:36:31.263834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:07:00.944096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282882+0000","last_change":"2024-04-23T14:36:30.278993+0000","last_active":"2024-04-23T14:36:32.282882+0000","last_peered":"2024-04-23T14:36:32.282882+0000","last_clean":"2024-04-23T14:36:32.282882+0000","last_became_active":"2024-04-23T14:36:30.278089+0000","last_became_peered":"2024-04-23T14:36:30.278089+0000","last_unstale":"2024-04-23T14:36:32.282882+0000","last_undegraded":"2024-04-23T14:36:32.282882+0000","last_fullsized":"2024-04-23T14:36:32.282882+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:04.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282648+0000","last_change":"2024-04-23T14:36:29.253372+0000","last_active":"2024-04-23T14:36:32.282648+0000","last_peered":"2024-04-23T14:36:32.282648+0000","last_clean":"2024-04-23T14:36:32.282648+0000","last_became_active":"2024-04-23T14:36:29.251760+0000","last_became_peered":"2024-04-23T14:36:29.251760+0000","last_unstale":"2024-04-23T14:36:32.282648+0000","last_undegraded":"2024-04-23T14:36:32.282648+0000","last_fullsized":"2024-04-23T14:36:32.282648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:53.438649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283299+0000","last_change":"2024-04-23T14:36:30.284555+0000","last_active":"2024-04-23T14:36:32.283299+0000","last_peered":"2024-04-23T14:36:32.283299+0000","last_clean":"2024-04-23T14:36:32.283299+0000","last_became_active":"2024-04-23T14:36:30.284171+0000","last_became_peered":"2024-04-23T14:36:30.284171+0000","last_unstale":"2024-04-23T14:36:32.283299+0000","last_undegraded":"2024-04-23T14:36:32.283299+0000","last_fullsized":"2024-04-23T14:36:32.283299+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:20.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257414+0000","last_change":"2024-04-23T14:36:29.245427+0000","last_active":"2024-04-23T14:36:31.257414+0000","last_peered":"2024-04-23T14:36:31.257414+0000","last_clean":"2024-04-23T14:36:31.257414+0000","last_became_active":"2024-04-23T14:36:29.245034+0000","last_became_peered":"2024-04-23T14:36:29.245034+0000","last_unstale":"2024-04-23T14:36:31.257414+0000","last_undegraded":"2024-04-23T14:36:31.257414+0000","last_fullsized":"2024-04-23T14:36:31.257414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:13.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283984+0000","last_change":"2024-04-23T14:36:30.277676+0000","last_active":"2024-04-23T14:36:32.283984+0000","last_peered":"2024-04-23T14:36:32.283984+0000","last_clean":"2024-04-23T14:36:32.283984+0000","last_became_active":"2024-04-23T14:36:30.277405+0000","last_became_peered":"2024-04-23T14:36:30.277405+0000","last_unstale":"2024-04-23T14:36:32.283984+0000","last_undegraded":"2024-04-23T14:36:32.283984+0000","last_fullsized":"2024-04-23T14:36:32.283984+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:40.186848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280948+0000","last_change":"2024-04-23T14:36:29.257966+0000","last_active":"2024-04-23T14:36:32.280948+0000","last_peered":"2024-04-23T14:36:32.280948+0000","last_clean":"2024-04-23T14:36:32.280948+0000","last_became_active":"2024-04-23T14:36:29.257687+0000","last_became_peered":"2024-04-23T14:36:29.257687+0000","last_unstale":"2024-04-23T14:36:32.280948+0000","last_undegraded":"2024-04-23T14:36:32.280948+0000","last_fullsized":"2024-04-23T14:36:32.280948+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:36.303077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257650+0000","last_change":"2024-04-23T14:36:30.260028+0000","last_active":"2024-04-23T14:36:31.257650+0000","last_peered":"2024-04-23T14:36:31.257650+0000","last_clean":"2024-04-23T14:36:31.257650+0000","last_became_active":"2024-04-23T14:36:30.259713+0000","last_became_peered":"2024-04-23T14:36:30.259713+0000","last_unstale":"2024-04-23T14:36:31.257650+0000","last_undegraded":"2024-04-23T14:36:31.257650+0000","last_fullsized":"2024-04-23T14:36:31.257650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:02.099306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256853+0000","last_change":"2024-04-23T14:36:29.249521+0000","last_active":"2024-04-23T14:36:31.256853+0000","last_peered":"2024-04-23T14:36:31.256853+0000","last_clean":"2024-04-23T14:36:31.256853+0000","last_became_active":"2024-04-23T14:36:29.249249+0000","last_became_peered":"2024-04-23T14:36:29.249249+0000","last_unstale":"2024-04-23T14:36:31.256853+0000","last_undegraded":"2024-04-23T14:36:31.256853+0000","last_fullsized":"2024-04-23T14:36:31.256853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:08:47.916797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263706+0000","last_change":"2024-04-23T14:36:30.283700+0000","last_active":"2024-04-23T14:36:31.263706+0000","last_peered":"2024-04-23T14:36:31.263706+0000","last_clean":"2024-04-23T14:36:31.263706+0000","last_became_active":"2024-04-23T14:36:30.283438+0000","last_became_peered":"2024-04-23T14:36:30.283438+0000","last_unstale":"2024-04-23T14:36:31.263706+0000","last_undegraded":"2024-04-23T14:36:31.263706+0000","last_fullsized":"2024-04-23T14:36:31.263706+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:35.008967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281910+0000","last_change":"2024-04-23T14:36:29.262517+0000","last_active":"2024-04-23T14:36:32.281910+0000","last_peered":"2024-04-23T14:36:32.281910+0000","last_clean":"2024-04-23T14:36:32.281910+0000","last_became_active":"2024-04-23T14:36:29.262167+0000","last_became_peered":"2024-04-23T14:36:29.262167+0000","last_unstale":"2024-04-23T14:36:32.281910+0000","last_undegraded":"2024-04-23T14:36:32.281910+0000","last_fullsized":"2024-04-23T14:36:32.281910+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:56:10.745359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281979+0000","last_change":"2024-04-23T14:36:30.279760+0000","last_active":"2024-04-23T14:36:32.281979+0000","last_peered":"2024-04-23T14:36:32.281979+0000","last_clean":"2024-04-23T14:36:32.281979+0000","last_became_active":"2024-04-23T14:36:30.278726+0000","last_became_peered":"2024-04-23T14:36:30.278726+0000","last_unstale":"2024-04-23T14:36:32.281979+0000","last_undegraded":"2024-04-23T14:36:32.281979+0000","last_fullsized":"2024-04-23T14:36:32.281979+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:25:43.492723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.260275+0000","last_change":"2024-04-23T14:36:29.260899+0000","last_active":"2024-04-23T14:36:31.260275+0000","last_peered":"2024-04-23T14:36:31.260275+0000","last_clean":"2024-04-23T14:36:31.260275+0000","last_became_active":"2024-04-23T14:36:29.258766+0000","last_became_peered":"2024-04-23T14:36:29.258766+0000","last_unstale":"2024-04-23T14:36:31.260275+0000","last_undegraded":"2024-04-23T14:36:31.260275+0000","last_fullsized":"2024-04-23T14:36:31.260275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:45.779744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263069+0000","last_change":"2024-04-23T14:36:30.263572+0000","last_active":"2024-04-23T14:36:31.263069+0000","last_peered":"2024-04-23T14:36:31.263069+0000","last_clean":"2024-04-23T14:36:31.263069+0000","last_became_active":"2024-04-23T14:36:30.263174+0000","last_became_peered":"2024-04-23T14:36:30.263174+0000","last_unstale":"2024-04-23T14:36:31.263069+0000","last_undegraded":"2024-04-23T14:36:31.263069+0000","last_fullsized":"2024-04-23T14:36:31.263069+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:30.768940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262400+0000","last_change":"2024-04-23T14:36:29.261017+0000","last_active":"2024-04-23T14:36:31.262400+0000","last_peered":"2024-04-23T14:36:31.262400+0000","last_clean":"2024-04-23T14:36:31.262400+0000","last_became_active":"2024-04-23T14:36:29.258667+0000","last_became_peered":"2024-04-23T14:36:29.258667+0000","last_unstale":"2024-04-23T14:36:31.262400+0000","last_undegraded":"2024-04-23T14:36:31.262400+0000","last_fullsized":"2024-04-23T14:36:31.262400+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:00.075116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283337+0000","last_change":"2024-04-23T14:36:30.268831+0000","last_active":"2024-04-23T14:36:32.283337+0000","last_peered":"2024-04-23T14:36:32.283337+0000","last_clean":"2024-04-23T14:36:32.283337+0000","last_became_active":"2024-04-23T14:36:30.268534+0000","last_became_peered":"2024-04-23T14:36:30.268534+0000","last_unstale":"2024-04-23T14:36:32.283337+0000","last_undegraded":"2024-04-23T14:36:32.283337+0000","last_fullsized":"2024-04-23T14:36:32.283337+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:22.607948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263916+0000","last_change":"2024-04-23T14:36:29.258243+0000","last_active":"2024-04-23T14:36:31.263916+0000","last_peered":"2024-04-23T14:36:31.263916+0000","last_clean":"2024-04-23T14:36:31.263916+0000","last_became_active":"2024-04-23T14:36:29.257918+0000","last_became_peered":"2024-04-23T14:36:29.257918+0000","last_unstale":"2024-04-23T14:36:31.263916+0000","last_undegraded":"2024-04-23T14:36:31.263916+0000","last_fullsized":"2024-04-23T14:36:31.263916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:31.008207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283390+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:36:32.283390+0000","last_peered":"2024-04-23T14:36:32.283390+0000","last_clean":"2024-04-23T14:36:32.283390+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:36:32.283390+0000","last_undegraded":"2024-04-23T14:36:32.283390+0000","last_fullsized":"2024-04-23T14:36:32.283390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":944641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":851968,"data_stored":355269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":355885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":944025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:36:39.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:36:40.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.024+0000 7fc79f000640 1 Processor -- start 2024-04-23T14:36:40.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.025+0000 7fc79f000640 1 -- start start 2024-04-23T14:36:40.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.026+0000 7fc79f000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc798162df0 0x7fc79806f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.026+0000 7fc79f000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7980b4130 0x7fc7980c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.026+0000 7fc79f000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79806ed40 con 0x7fc7980cfb90 2024-04-23T14:36:40.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.026+0000 7fc79f000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc79806eaf0 con 0x7fc7980b4130 2024-04-23T14:36:40.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.026+0000 7fc79f000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc7980b3590 con 0x7fc798162df0 2024-04-23T14:36:40.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7980b4130 0x7fc7980c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79d200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc7980cfb90 0x7fc7980b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43972/0 (socket says 172.21.15.59:43972) 2024-04-23T14:36:40.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79e600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7980b4130 0x7fc7980c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42942/0 (socket says 172.21.15.59:42942) 2024-04-23T14:36:40.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79d200640 1 -- 172.21.15.59:0/400618572 learned_addr learned my addr 172.21.15.59:0/400618572 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:40.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79dc00640 1 --2- 172.21.15.59:0/400618572 >> v2:172.21.15.76:3300/0 conn(0x7fc798162df0 0x7fc79806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79e600640 1 -- 172.21.15.59:0/400618572 >> v2:172.21.15.76:3300/0 conn(0x7fc798162df0 msgr2=0x7fc79806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:40.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79e600640 1 --2- 172.21.15.59:0/400618572 >> v2:172.21.15.76:3300/0 conn(0x7fc798162df0 0x7fc79806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79e600640 1 -- 172.21.15.59:0/400618572 >> v1:172.21.15.59:6789/0 conn(0x7fc7980cfb90 legacy=0x7fc7980b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.027+0000 7fc79e600640 1 -- 172.21.15.59:0/400618572 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7980b3220 con 0x7fc7980b4130 2024-04-23T14:36:40.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.028+0000 7fc79e600640 1 --2- 172.21.15.59:0/400618572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980b4130 0x7fc7980c31a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc7980cd890 tx=0x7fc79002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd442ead599bba1b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:40.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.029+0000 7fc797e00640 1 -- 172.21.15.59:0/400618572 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790007740 con 0x7fc7980b4130 2024-04-23T14:36:40.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.029+0000 7fc797e00640 1 -- 172.21.15.59:0/400618572 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7900078e0 con 0x7fc7980b4130 2024-04-23T14:36:40.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.029+0000 7fc797e00640 1 -- 172.21.15.59:0/400618572 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790007cb0 con 0x7fc7980b4130 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.029+0000 7fc79f000640 1 -- 172.21.15.59:0/400618572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980b4130 msgr2=0x7fc7980c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.029+0000 7fc79f000640 1 --2- 172.21.15.59:0/400618572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980b4130 0x7fc7980c31a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc7980cd890 tx=0x7fc79002a680 comp rx=0 tx=0).stop 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.029+0000 7fc79f000640 1 -- 172.21.15.59:0/400618572 shutdown_connections 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.029+0000 7fc79f000640 1 --2- 172.21.15.59:0/400618572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980b4130 0x7fc7980c31a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.030+0000 7fc79f000640 1 --2- 172.21.15.59:0/400618572 >> v2:172.21.15.76:3300/0 conn(0x7fc798162df0 0x7fc79806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.030+0000 7fc79f000640 1 -- 172.21.15.59:0/400618572 >> 172.21.15.59:0/400618572 conn(0x7fc7980b21a0 msgr2=0x7fc7980b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.030+0000 7fc79f000640 1 -- 172.21.15.59:0/400618572 shutdown_connections 2024-04-23T14:36:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.030+0000 7fc79f000640 1 -- 172.21.15.59:0/400618572 wait complete. 2024-04-23T14:36:40.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.031+0000 7fc79f000640 1 Processor -- start 2024-04-23T14:36:40.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.031+0000 7fc79f000640 1 -- start start 2024-04-23T14:36:40.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.032+0000 7fc79f000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7980b4130 0x7fc7981fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.032+0000 7fc79f000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7980cfb90 0x7fc79815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.032+0000 7fc79f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc798162df0 0x7fc7980ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.032+0000 7fc79f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980c3860 con 0x7fc798162df0 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.032+0000 7fc79f000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980c36e0 con 0x7fc7980cfb90 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.032+0000 7fc79f000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc7980c39e0 con 0x7fc7980b4130 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc798162df0 0x7fc7980ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc798162df0 0x7fc7980ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42950/0 (socket says 172.21.15.59:42950) 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 -- 172.21.15.59:0/3291611047 learned_addr learned my addr 172.21.15.59:0/3291611047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79d200640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7980cfb90 0x7fc79815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79dc00640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7980b4130 0x7fc7981fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 -- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7980b4130 msgr2=0x7fc7981fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7980b4130 0x7fc7981fc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 -- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7980cfb90 msgr2=0x7fc79815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7980cfb90 0x7fc79815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79e600640 1 -- 172.21.15.59:0/3291611047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7981fca50 con 0x7fc798162df0 2024-04-23T14:36:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.033+0000 7fc79dc00640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7980b4130 0x7fc7981fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.034+0000 7fc79e600640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc798162df0 0x7fc7980ca7e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc79002ac70 tx=0x7fc790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.034+0000 7fc795600640 1 -- 172.21.15.59:0/3291611047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790040070 con 0x7fc798162df0 2024-04-23T14:36:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.034+0000 7fc795600640 1 -- 172.21.15.59:0/3291611047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc790031bc0 con 0x7fc798162df0 2024-04-23T14:36:40.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.034+0000 7fc795600640 1 -- 172.21.15.59:0/3291611047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790045460 con 0x7fc798162df0 2024-04-23T14:36:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.034+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7981646b0 con 0x7fc798162df0 2024-04-23T14:36:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.034+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc798164c70 con 0x7fc798162df0 2024-04-23T14:36:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.036+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79804fd90 con 0x7fc798162df0 2024-04-23T14:36:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.037+0000 7fc795600640 1 -- 172.21.15.59:0/3291611047 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc790039040 con 0x7fc798162df0 2024-04-23T14:36:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.037+0000 7fc795600640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc77806c8f0 0x7fc77806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.038+0000 7fc795600640 1 -- 172.21.15.59:0/3291611047 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fc7900ad500 con 0x7fc798162df0 2024-04-23T14:36:40.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.038+0000 7fc79dc00640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc77806c8f0 0x7fc77806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.039+0000 7fc79dc00640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc77806c8f0 0x7fc77806edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc78c006fd0 tx=0x7fc78c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.042+0000 7fc795600640 1 -- 172.21.15.59:0/3291611047 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc79007b9b0 con 0x7fc798162df0 2024-04-23T14:36:40.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.246+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc79815ca10 con 0x7fc77806c8f0 2024-04-23T14:36:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.266+0000 7fc795600640 1 -- 172.21.15.59:0/3291611047 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7fc79815ca10 con 0x7fc77806c8f0 2024-04-23T14:36:40.268 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.272+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc77806c8f0 msgr2=0x7fc77806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.272+0000 7fc79f000640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc77806c8f0 0x7fc77806edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc78c006fd0 tx=0x7fc78c008040 comp rx=0 tx=0).stop 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.272+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc798162df0 msgr2=0x7fc7980ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.272+0000 7fc79f000640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc798162df0 0x7fc7980ca7e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc79002ac70 tx=0x7fc790004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 shutdown_connections 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc77806c8f0 0x7fc77806edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc798162df0 0x7fc7980ca7e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7980cfb90 0x7fc79815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 --2- 172.21.15.59:0/3291611047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7980b4130 0x7fc7981fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 >> 172.21.15.59:0/3291611047 conn(0x7fc7980b21a0 msgr2=0x7fc7980b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 shutdown_connections 2024-04-23T14:36:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.273+0000 7fc79f000640 1 -- 172.21.15.59:0/3291611047 wait complete. 2024-04-23T14:36:40.292 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-04-23T14:36:38.450705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257228,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717492,"statfs":{"total":773094113280,"available":772830711808,"internally_reserved":0,"allocated":7970816,"data_stored":4014812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.216713"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283926+0000","last_change":"2024-04-23T14:36:30.278982+0000","last_active":"2024-04-23T14:36:32.283926+0000","last_peered":"2024-04-23T14:36:32.283926+0000","last_clean":"2024-04-23T14:36:32.283926+0000","last_became_active":"2024-04-23T14:36:30.278687+0000","last_became_peered":"2024-04-23T14:36:30.278687+0000","last_unstale":"2024-04-23T14:36:32.283926+0000","last_undegraded":"2024-04-23T14:36:32.283926+0000","last_fullsized":"2024-04-23T14:36:32.283926+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:04.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.264404+0000","last_change":"2024-04-23T14:36:30.261145+0000","last_active":"2024-04-23T14:36:31.264404+0000","last_peered":"2024-04-23T14:36:31.264404+0000","last_clean":"2024-04-23T14:36:31.264404+0000","last_became_active":"2024-04-23T14:36:30.260396+0000","last_became_peered":"2024-04-23T14:36:30.260396+0000","last_unstale":"2024-04-23T14:36:31.264404+0000","last_undegraded":"2024-04-23T14:36:31.264404+0000","last_fullsized":"2024-04-23T14:36:31.264404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:30:39.839746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283677+0000","last_change":"2024-04-23T14:36:30.254029+0000","last_active":"2024-04-23T14:36:32.283677+0000","last_peered":"2024-04-23T14:36:32.283677+0000","last_clean":"2024-04-23T14:36:32.283677+0000","last_became_active":"2024-04-23T14:36:30.253734+0000","last_became_peered":"2024-04-23T14:36:30.253734+0000","last_unstale":"2024-04-23T14:36:32.283677+0000","last_undegraded":"2024-04-23T14:36:32.283677+0000","last_fullsized":"2024-04-23T14:36:32.283677+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:30.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282308+0000","last_change":"2024-04-23T14:36:30.277554+0000","last_active":"2024-04-23T14:36:32.282308+0000","last_peered":"2024-04-23T14:36:32.282308+0000","last_clean":"2024-04-23T14:36:32.282308+0000","last_became_active":"2024-04-23T14:36:30.277223+0000","last_became_peered":"2024-04-23T14:36:30.277223+0000","last_unstale":"2024-04-23T14:36:32.282308+0000","last_undegraded":"2024-04-23T14:36:32.282308+0000","last_fullsized":"2024-04-23T14:36:32.282308+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:20.415522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256769+0000","last_change":"2024-04-23T14:36:30.271347+0000","last_active":"2024-04-23T14:36:31.256769+0000","last_peered":"2024-04-23T14:36:31.256769+0000","last_clean":"2024-04-23T14:36:31.256769+0000","last_became_active":"2024-04-23T14:36:30.270634+0000","last_became_peered":"2024-04-23T14:36:30.270634+0000","last_unstale":"2024-04-23T14:36:31.256769+0000","last_undegraded":"2024-04-23T14:36:31.256769+0000","last_fullsized":"2024-04-23T14:36:31.256769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:26.664536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263544+0000","last_change":"2024-04-23T14:36:30.265364+0000","last_active":"2024-04-23T14:36:31.263544+0000","last_peered":"2024-04-23T14:36:31.263544+0000","last_clean":"2024-04-23T14:36:31.263544+0000","last_became_active":"2024-04-23T14:36:30.264635+0000","last_became_peered":"2024-04-23T14:36:30.264635+0000","last_unstale":"2024-04-23T14:36:31.263544+0000","last_undegraded":"2024-04-23T14:36:31.263544+0000","last_fullsized":"2024-04-23T14:36:31.263544+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:46.746682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281466+0000","last_change":"2024-04-23T14:36:30.278449+0000","last_active":"2024-04-23T14:36:32.281466+0000","last_peered":"2024-04-23T14:36:32.281466+0000","last_clean":"2024-04-23T14:36:32.281466+0000","last_became_active":"2024-04-23T14:36:30.277159+0000","last_became_peered":"2024-04-23T14:36:30.277159+0000","last_unstale":"2024-04-23T14:36:32.281466+0000","last_undegraded":"2024-04-23T14:36:32.281466+0000","last_fullsized":"2024-04-23T14:36:32.281466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:54:06.102135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261842+0000","last_change":"2024-04-23T14:36:30.273457+0000","last_active":"2024-04-23T14:36:31.261842+0000","last_peered":"2024-04-23T14:36:31.261842+0000","last_clean":"2024-04-23T14:36:31.261842+0000","last_became_active":"2024-04-23T14:36:30.272918+0000","last_became_peered":"2024-04-23T14:36:30.272918+0000","last_unstale":"2024-04-23T14:36:31.261842+0000","last_undegraded":"2024-04-23T14:36:31.261842+0000","last_fullsized":"2024-04-23T14:36:31.261842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:33:00.177900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257521+0000","last_change":"2024-04-23T14:36:30.256388+0000","last_active":"2024-04-23T14:36:31.257521+0000","last_peered":"2024-04-23T14:36:31.257521+0000","last_clean":"2024-04-23T14:36:31.257521+0000","last_became_active":"2024-04-23T14:36:30.255042+0000","last_became_peered":"2024-04-23T14:36:30.255042+0000","last_unstale":"2024-04-23T14:36:31.257521+0000","last_undegraded":"2024-04-23T14:36:31.257521+0000","last_fullsized":"2024-04-23T14:36:31.257521+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:26:38.281474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261953+0000","last_change":"2024-04-23T14:36:30.252958+0000","last_active":"2024-04-23T14:36:31.261953+0000","last_peered":"2024-04-23T14:36:31.261953+0000","last_clean":"2024-04-23T14:36:31.261953+0000","last_became_active":"2024-04-23T14:36:30.252088+0000","last_became_peered":"2024-04-23T14:36:30.252088+0000","last_unstale":"2024-04-23T14:36:31.261953+0000","last_undegraded":"2024-04-23T14:36:31.261953+0000","last_fullsized":"2024-04-23T14:36:31.261953+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:59.184476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283596+0000","last_change":"2024-04-23T14:36:30.281268+0000","last_active":"2024-04-23T14:36:32.283596+0000","last_peered":"2024-04-23T14:36:32.283596+0000","last_clean":"2024-04-23T14:36:32.283596+0000","last_became_active":"2024-04-23T14:36:30.280759+0000","last_became_peered":"2024-04-23T14:36:30.280759+0000","last_unstale":"2024-04-23T14:36:32.283596+0000","last_undegraded":"2024-04-23T14:36:32.283596+0000","last_fullsized":"2024-04-23T14:36:32.283596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:12:49.259305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282620+0000","last_change":"2024-04-23T14:36:30.274529+0000","last_active":"2024-04-23T14:36:32.282620+0000","last_peered":"2024-04-23T14:36:32.282620+0000","last_clean":"2024-04-23T14:36:32.282620+0000","last_became_active":"2024-04-23T14:36:30.273366+0000","last_became_peered":"2024-04-23T14:36:30.273366+0000","last_unstale":"2024-04-23T14:36:32.282620+0000","last_undegraded":"2024-04-23T14:36:32.282620+0000","last_fullsized":"2024-04-23T14:36:32.282620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:43:27.060798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280665+0000","last_change":"2024-04-23T14:36:30.268167+0000","last_active":"2024-04-23T14:36:32.280665+0000","last_peered":"2024-04-23T14:36:32.280665+0000","last_clean":"2024-04-23T14:36:32.280665+0000","last_became_active":"2024-04-23T14:36:30.267861+0000","last_became_peered":"2024-04-23T14:36:30.267861+0000","last_unstale":"2024-04-23T14:36:32.280665+0000","last_undegraded":"2024-04-23T14:36:32.280665+0000","last_fullsized":"2024-04-23T14:36:32.280665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:01.140299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257406+0000","last_change":"2024-04-23T14:36:30.276001+0000","last_active":"2024-04-23T14:36:31.257406+0000","last_peered":"2024-04-23T14:36:31.257406+0000","last_clean":"2024-04-23T14:36:31.257406+0000","last_became_active":"2024-04-23T14:36:30.271406+0000","last_became_peered":"2024-04-23T14:36:30.271406+0000","last_unstale":"2024-04-23T14:36:31.257406+0000","last_undegraded":"2024-04-23T14:36:31.257406+0000","last_fullsized":"2024-04-23T14:36:31.257406+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:06:05.035180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280436+0000","last_change":"2024-04-23T14:36:30.262622+0000","last_active":"2024-04-23T14:36:32.280436+0000","last_peered":"2024-04-23T14:36:32.280436+0000","last_clean":"2024-04-23T14:36:32.280436+0000","last_became_active":"2024-04-23T14:36:30.262183+0000","last_became_peered":"2024-04-23T14:36:30.262183+0000","last_unstale":"2024-04-23T14:36:32.280436+0000","last_undegraded":"2024-04-23T14:36:32.280436+0000","last_fullsized":"2024-04-23T14:36:32.280436+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:12:49.259305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.260257+0000","last_change":"2024-04-23T14:36:30.270096+0000","last_active":"2024-04-23T14:36:31.260257+0000","last_peered":"2024-04-23T14:36:31.260257+0000","last_clean":"2024-04-23T14:36:31.260257+0000","last_became_active":"2024-04-23T14:36:30.268105+0000","last_became_peered":"2024-04-23T14:36:30.268105+0000","last_unstale":"2024-04-23T14:36:31.260257+0000","last_undegraded":"2024-04-23T14:36:31.260257+0000","last_fullsized":"2024-04-23T14:36:31.260257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:51:47.737900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257673+0000","last_change":"2024-04-23T14:36:30.257389+0000","last_active":"2024-04-23T14:36:31.257673+0000","last_peered":"2024-04-23T14:36:31.257673+0000","last_clean":"2024-04-23T14:36:31.257673+0000","last_became_active":"2024-04-23T14:36:30.256095+0000","last_became_peered":"2024-04-23T14:36:30.256095+0000","last_unstale":"2024-04-23T14:36:31.257673+0000","last_undegraded":"2024-04-23T14:36:31.257673+0000","last_fullsized":"2024-04-23T14:36:31.257673+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:44:48.658488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282665+0000","last_change":"2024-04-23T14:36:30.283652+0000","last_active":"2024-04-23T14:36:32.282665+0000","last_peered":"2024-04-23T14:36:32.282665+0000","last_clean":"2024-04-23T14:36:32.282665+0000","last_became_active":"2024-04-23T14:36:30.283367+0000","last_became_peered":"2024-04-23T14:36:30.283367+0000","last_unstale":"2024-04-23T14:36:32.282665+0000","last_undegraded":"2024-04-23T14:36:32.282665+0000","last_fullsized":"2024-04-23T14:36:32.282665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:36:13.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263040+0000","last_change":"2024-04-23T14:36:30.272297+0000","last_active":"2024-04-23T14:36:31.263040+0000","last_peered":"2024-04-23T14:36:31.263040+0000","last_clean":"2024-04-23T14:36:31.263040+0000","last_became_active":"2024-04-23T14:36:30.268811+0000","last_became_peered":"2024-04-23T14:36:30.268811+0000","last_unstale":"2024-04-23T14:36:31.263040+0000","last_undegraded":"2024-04-23T14:36:31.263040+0000","last_fullsized":"2024-04-23T14:36:31.263040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:59.184476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280641+0000","last_change":"2024-04-23T14:36:30.267969+0000","last_active":"2024-04-23T14:36:32.280641+0000","last_peered":"2024-04-23T14:36:32.280641+0000","last_clean":"2024-04-23T14:36:32.280641+0000","last_became_active":"2024-04-23T14:36:30.267591+0000","last_became_peered":"2024-04-23T14:36:30.267591+0000","last_unstale":"2024-04-23T14:36:32.280641+0000","last_undegraded":"2024-04-23T14:36:32.280641+0000","last_fullsized":"2024-04-23T14:36:32.280641+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:23:34.977231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261116+0000","last_change":"2024-04-23T14:36:30.256885+0000","last_active":"2024-04-23T14:36:31.261116+0000","last_peered":"2024-04-23T14:36:31.261116+0000","last_clean":"2024-04-23T14:36:31.261116+0000","last_became_active":"2024-04-23T14:36:30.256110+0000","last_became_peered":"2024-04-23T14:36:30.256110+0000","last_unstale":"2024-04-23T14:36:31.261116+0000","last_undegraded":"2024-04-23T14:36:31.261116+0000","last_fullsized":"2024-04-23T14:36:31.261116+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:33:00.177900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261104+0000","last_change":"2024-04-23T14:36:30.253926+0000","last_active":"2024-04-23T14:36:31.261104+0000","last_peered":"2024-04-23T14:36:31.261104+0000","last_clean":"2024-04-23T14:36:31.261104+0000","last_became_active":"2024-04-23T14:36:30.253472+0000","last_became_peered":"2024-04-23T14:36:30.253472+0000","last_unstale":"2024-04-23T14:36:31.261104+0000","last_undegraded":"2024-04-23T14:36:31.261104+0000","last_fullsized":"2024-04-23T14:36:31.261104+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:53.136057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283019+0000","last_change":"2024-04-23T14:36:30.279100+0000","last_active":"2024-04-23T14:36:32.283019+0000","last_peered":"2024-04-23T14:36:32.283019+0000","last_clean":"2024-04-23T14:36:32.283019+0000","last_became_active":"2024-04-23T14:36:30.276984+0000","last_became_peered":"2024-04-23T14:36:30.276984+0000","last_unstale":"2024-04-23T14:36:32.283019+0000","last_undegraded":"2024-04-23T14:36:32.283019+0000","last_fullsized":"2024-04-23T14:36:32.283019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:23:34.977231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282250+0000","last_change":"2024-04-23T14:36:30.284096+0000","last_active":"2024-04-23T14:36:32.282250+0000","last_peered":"2024-04-23T14:36:32.282250+0000","last_clean":"2024-04-23T14:36:32.282250+0000","last_became_active":"2024-04-23T14:36:30.283751+0000","last_became_peered":"2024-04-23T14:36:30.283751+0000","last_unstale":"2024-04-23T14:36:32.282250+0000","last_undegraded":"2024-04-23T14:36:32.282250+0000","last_fullsized":"2024-04-23T14:36:32.282250+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:52.453599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262620+0000","last_change":"2024-04-23T14:36:30.273502+0000","last_active":"2024-04-23T14:36:31.262620+0000","last_peered":"2024-04-23T14:36:31.262620+0000","last_clean":"2024-04-23T14:36:31.262620+0000","last_became_active":"2024-04-23T14:36:30.273238+0000","last_became_peered":"2024-04-23T14:36:30.273238+0000","last_unstale":"2024-04-23T14:36:31.262620+0000","last_undegraded":"2024-04-23T14:36:31.262620+0000","last_fullsized":"2024-04-23T14:36:31.262620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:49:46.746682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282358+0000","last_change":"2024-04-23T14:36:30.276366+0000","last_active":"2024-04-23T14:36:32.282358+0000","last_peered":"2024-04-23T14:36:32.282358+0000","last_clean":"2024-04-23T14:36:32.282358+0000","last_became_active":"2024-04-23T14:36:30.276016+0000","last_became_peered":"2024-04-23T14:36:30.276016+0000","last_unstale":"2024-04-23T14:36:32.282358+0000","last_undegraded":"2024-04-23T14:36:32.282358+0000","last_fullsized":"2024-04-23T14:36:32.282358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:20.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283860+0000","last_change":"2024-04-23T14:36:30.275939+0000","last_active":"2024-04-23T14:36:32.283860+0000","last_peered":"2024-04-23T14:36:32.283860+0000","last_clean":"2024-04-23T14:36:32.283860+0000","last_became_active":"2024-04-23T14:36:30.275646+0000","last_became_peered":"2024-04-23T14:36:30.275646+0000","last_unstale":"2024-04-23T14:36:32.283860+0000","last_undegraded":"2024-04-23T14:36:32.283860+0000","last_fullsized":"2024-04-23T14:36:32.283860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:45.735264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261823+0000","last_change":"2024-04-23T14:36:30.252644+0000","last_active":"2024-04-23T14:36:31.261823+0000","last_peered":"2024-04-23T14:36:31.261823+0000","last_clean":"2024-04-23T14:36:31.261823+0000","last_became_active":"2024-04-23T14:36:30.251743+0000","last_became_peered":"2024-04-23T14:36:30.251743+0000","last_unstale":"2024-04-23T14:36:31.261823+0000","last_undegraded":"2024-04-23T14:36:31.261823+0000","last_fullsized":"2024-04-23T14:36:31.261823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:01:39.745735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283514+0000","last_change":"2024-04-23T14:36:30.264701+0000","last_active":"2024-04-23T14:36:32.283514+0000","last_peered":"2024-04-23T14:36:32.283514+0000","last_clean":"2024-04-23T14:36:32.283514+0000","last_became_active":"2024-04-23T14:36:30.264443+0000","last_became_peered":"2024-04-23T14:36:30.264443+0000","last_unstale":"2024-04-23T14:36:32.283514+0000","last_undegraded":"2024-04-23T14:36:32.283514+0000","last_fullsized":"2024-04-23T14:36:32.283514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:22:53.571110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281939+0000","last_change":"2024-04-23T14:36:30.276300+0000","last_active":"2024-04-23T14:36:32.281939+0000","last_peered":"2024-04-23T14:36:32.281939+0000","last_clean":"2024-04-23T14:36:32.281939+0000","last_became_active":"2024-04-23T14:36:30.275375+0000","last_became_peered":"2024-04-23T14:36:30.275375+0000","last_unstale":"2024-04-23T14:36:32.281939+0000","last_undegraded":"2024-04-23T14:36:32.281939+0000","last_fullsized":"2024-04-23T14:36:32.281939+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:52.453599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281960+0000","last_change":"2024-04-23T14:36:30.273517+0000","last_active":"2024-04-23T14:36:32.281960+0000","last_peered":"2024-04-23T14:36:32.281960+0000","last_clean":"2024-04-23T14:36:32.281960+0000","last_became_active":"2024-04-23T14:36:30.271844+0000","last_became_peered":"2024-04-23T14:36:30.271844+0000","last_unstale":"2024-04-23T14:36:32.281960+0000","last_undegraded":"2024-04-23T14:36:32.281960+0000","last_fullsized":"2024-04-23T14:36:32.281960+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:23:34.977231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262188+0000","last_change":"2024-04-23T14:36:30.260056+0000","last_active":"2024-04-23T14:36:31.262188+0000","last_peered":"2024-04-23T14:36:31.262188+0000","last_clean":"2024-04-23T14:36:31.262188+0000","last_became_active":"2024-04-23T14:36:30.259502+0000","last_became_peered":"2024-04-23T14:36:30.259502+0000","last_unstale":"2024-04-23T14:36:31.262188+0000","last_undegraded":"2024-04-23T14:36:31.262188+0000","last_fullsized":"2024-04-23T14:36:31.262188+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.191748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284052+0000","last_change":"2024-04-23T14:36:30.285282+0000","last_active":"2024-04-23T14:36:32.284052+0000","last_peered":"2024-04-23T14:36:32.284052+0000","last_clean":"2024-04-23T14:36:32.284052+0000","last_became_active":"2024-04-23T14:36:30.284931+0000","last_became_peered":"2024-04-23T14:36:30.284931+0000","last_unstale":"2024-04-23T14:36:32.284052+0000","last_undegraded":"2024-04-23T14:36:32.284052+0000","last_fullsized":"2024-04-23T14:36:32.284052+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:31:50.195282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284490+0000","last_change":"2024-04-23T14:36:29.265430+0000","last_active":"2024-04-23T14:36:32.284490+0000","last_peered":"2024-04-23T14:36:32.284490+0000","last_clean":"2024-04-23T14:36:32.284490+0000","last_became_active":"2024-04-23T14:36:29.265096+0000","last_became_peered":"2024-04-23T14:36:29.265096+0000","last_unstale":"2024-04-23T14:36:32.284490+0000","last_undegraded":"2024-04-23T14:36:32.284490+0000","last_fullsized":"2024-04-23T14:36:32.284490+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:18.617182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257718+0000","last_change":"2024-04-23T14:36:30.257027+0000","last_active":"2024-04-23T14:36:31.257718+0000","last_peered":"2024-04-23T14:36:31.257718+0000","last_clean":"2024-04-23T14:36:31.257718+0000","last_became_active":"2024-04-23T14:36:30.255514+0000","last_became_peered":"2024-04-23T14:36:30.255514+0000","last_unstale":"2024-04-23T14:36:31.257718+0000","last_undegraded":"2024-04-23T14:36:31.257718+0000","last_fullsized":"2024-04-23T14:36:31.257718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:49:21.165754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281267+0000","last_change":"2024-04-23T14:36:29.255962+0000","last_active":"2024-04-23T14:36:32.281267+0000","last_peered":"2024-04-23T14:36:32.281267+0000","last_clean":"2024-04-23T14:36:32.281267+0000","last_became_active":"2024-04-23T14:36:29.255566+0000","last_became_peered":"2024-04-23T14:36:29.255566+0000","last_unstale":"2024-04-23T14:36:32.281267+0000","last_undegraded":"2024-04-23T14:36:32.281267+0000","last_fullsized":"2024-04-23T14:36:32.281267+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:25:19.305134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261357+0000","last_change":"2024-04-23T14:36:30.271373+0000","last_active":"2024-04-23T14:36:31.261357+0000","last_peered":"2024-04-23T14:36:31.261357+0000","last_clean":"2024-04-23T14:36:31.261357+0000","last_became_active":"2024-04-23T14:36:30.269418+0000","last_became_peered":"2024-04-23T14:36:30.269418+0000","last_unstale":"2024-04-23T14:36:31.261357+0000","last_undegraded":"2024-04-23T14:36:31.261357+0000","last_fullsized":"2024-04-23T14:36:31.261357+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:27:53.136057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262504+0000","last_change":"2024-04-23T14:36:29.264940+0000","last_active":"2024-04-23T14:36:31.262504+0000","last_peered":"2024-04-23T14:36:31.262504+0000","last_clean":"2024-04-23T14:36:31.262504+0000","last_became_active":"2024-04-23T14:36:29.264568+0000","last_became_peered":"2024-04-23T14:36:29.264568+0000","last_unstale":"2024-04-23T14:36:31.262504+0000","last_undegraded":"2024-04-23T14:36:31.262504+0000","last_fullsized":"2024-04-23T14:36:31.262504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:31.008207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283726+0000","last_change":"2024-04-23T14:36:30.273278+0000","last_active":"2024-04-23T14:36:32.283726+0000","last_peered":"2024-04-23T14:36:32.283726+0000","last_clean":"2024-04-23T14:36:32.283726+0000","last_became_active":"2024-04-23T14:36:30.272951+0000","last_became_peered":"2024-04-23T14:36:30.272951+0000","last_unstale":"2024-04-23T14:36:32.283726+0000","last_undegraded":"2024-04-23T14:36:32.283726+0000","last_fullsized":"2024-04-23T14:36:32.283726+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:52.345173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263306+0000","last_change":"2024-04-23T14:36:29.258194+0000","last_active":"2024-04-23T14:36:31.263306+0000","last_peered":"2024-04-23T14:36:31.263306+0000","last_clean":"2024-04-23T14:36:31.263306+0000","last_became_active":"2024-04-23T14:36:29.257888+0000","last_became_peered":"2024-04-23T14:36:29.257888+0000","last_unstale":"2024-04-23T14:36:31.263306+0000","last_undegraded":"2024-04-23T14:36:31.263306+0000","last_fullsized":"2024-04-23T14:36:31.263306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:41.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282433+0000","last_change":"2024-04-23T14:36:30.274986+0000","last_active":"2024-04-23T14:36:32.282433+0000","last_peered":"2024-04-23T14:36:32.282433+0000","last_clean":"2024-04-23T14:36:32.282433+0000","last_became_active":"2024-04-23T14:36:30.274524+0000","last_became_peered":"2024-04-23T14:36:30.274524+0000","last_unstale":"2024-04-23T14:36:32.282433+0000","last_undegraded":"2024-04-23T14:36:32.282433+0000","last_fullsized":"2024-04-23T14:36:32.282433+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:12:49.259305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283810+0000","last_change":"2024-04-23T14:36:29.259191+0000","last_active":"2024-04-23T14:36:32.283810+0000","last_peered":"2024-04-23T14:36:32.283810+0000","last_clean":"2024-04-23T14:36:32.283810+0000","last_became_active":"2024-04-23T14:36:29.258420+0000","last_became_peered":"2024-04-23T14:36:29.258420+0000","last_unstale":"2024-04-23T14:36:32.283810+0000","last_undegraded":"2024-04-23T14:36:32.283810+0000","last_fullsized":"2024-04-23T14:36:32.283810+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:21.800400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.261208+0000","last_change":"2024-04-23T14:36:30.277342+0000","last_active":"2024-04-23T14:36:31.261208+0000","last_peered":"2024-04-23T14:36:31.261208+0000","last_clean":"2024-04-23T14:36:31.261208+0000","last_became_active":"2024-04-23T14:36:30.272999+0000","last_became_peered":"2024-04-23T14:36:30.272999+0000","last_unstale":"2024-04-23T14:36:31.261208+0000","last_undegraded":"2024-04-23T14:36:31.261208+0000","last_fullsized":"2024-04-23T14:36:31.261208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:45.191748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263657+0000","last_change":"2024-04-23T14:36:29.255423+0000","last_active":"2024-04-23T14:36:31.263657+0000","last_peered":"2024-04-23T14:36:31.263657+0000","last_clean":"2024-04-23T14:36:31.263657+0000","last_became_active":"2024-04-23T14:36:29.255130+0000","last_became_peered":"2024-04-23T14:36:29.255130+0000","last_unstale":"2024-04-23T14:36:31.263657+0000","last_undegraded":"2024-04-23T14:36:31.263657+0000","last_fullsized":"2024-04-23T14:36:31.263657+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:04:58.055114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256577+0000","last_change":"2024-04-23T14:36:30.259813+0000","last_active":"2024-04-23T14:36:31.256577+0000","last_peered":"2024-04-23T14:36:31.256577+0000","last_clean":"2024-04-23T14:36:31.256577+0000","last_became_active":"2024-04-23T14:36:30.259020+0000","last_became_peered":"2024-04-23T14:36:30.259020+0000","last_unstale":"2024-04-23T14:36:31.256577+0000","last_undegraded":"2024-04-23T14:36:31.256577+0000","last_fullsized":"2024-04-23T14:36:31.256577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:44:28.217244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282018+0000","last_change":"2024-04-23T14:36:29.262680+0000","last_active":"2024-04-23T14:36:32.282018+0000","last_peered":"2024-04-23T14:36:32.282018+0000","last_clean":"2024-04-23T14:36:32.282018+0000","last_became_active":"2024-04-23T14:36:29.262177+0000","last_became_peered":"2024-04-23T14:36:29.262177+0000","last_unstale":"2024-04-23T14:36:32.282018+0000","last_undegraded":"2024-04-23T14:36:32.282018+0000","last_fullsized":"2024-04-23T14:36:32.282018+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:16:40.504300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280643+0000","last_change":"2024-04-23T14:36:30.272307+0000","last_active":"2024-04-23T14:36:32.280643+0000","last_peered":"2024-04-23T14:36:32.280643+0000","last_clean":"2024-04-23T14:36:32.280643+0000","last_became_active":"2024-04-23T14:36:30.271993+0000","last_became_peered":"2024-04-23T14:36:30.271993+0000","last_unstale":"2024-04-23T14:36:32.280643+0000","last_undegraded":"2024-04-23T14:36:32.280643+0000","last_fullsized":"2024-04-23T14:36:32.280643+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:20.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256559+0000","last_change":"2024-04-23T14:36:29.255068+0000","last_active":"2024-04-23T14:36:31.256559+0000","last_peered":"2024-04-23T14:36:31.256559+0000","last_clean":"2024-04-23T14:36:31.256559+0000","last_became_active":"2024-04-23T14:36:29.254465+0000","last_became_peered":"2024-04-23T14:36:29.254465+0000","last_unstale":"2024-04-23T14:36:31.256559+0000","last_undegraded":"2024-04-23T14:36:31.256559+0000","last_fullsized":"2024-04-23T14:36:31.256559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:47.763347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282438+0000","last_change":"2024-04-23T14:36:30.272697+0000","last_active":"2024-04-23T14:36:32.282438+0000","last_peered":"2024-04-23T14:36:32.282438+0000","last_clean":"2024-04-23T14:36:32.282438+0000","last_became_active":"2024-04-23T14:36:30.272332+0000","last_became_peered":"2024-04-23T14:36:30.272332+0000","last_unstale":"2024-04-23T14:36:32.282438+0000","last_undegraded":"2024-04-23T14:36:32.282438+0000","last_fullsized":"2024-04-23T14:36:32.282438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:52.345173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262506+0000","last_change":"2024-04-23T14:36:29.262461+0000","last_active":"2024-04-23T14:36:31.262506+0000","last_peered":"2024-04-23T14:36:31.262506+0000","last_clean":"2024-04-23T14:36:31.262506+0000","last_became_active":"2024-04-23T14:36:29.262134+0000","last_became_peered":"2024-04-23T14:36:29.262134+0000","last_unstale":"2024-04-23T14:36:31.262506+0000","last_undegraded":"2024-04-23T14:36:31.262506+0000","last_fullsized":"2024-04-23T14:36:31.262506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:07:00.944096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284214+0000","last_change":"2024-04-23T14:36:30.284590+0000","last_active":"2024-04-23T14:36:32.284214+0000","last_peered":"2024-04-23T14:36:32.284214+0000","last_clean":"2024-04-23T14:36:32.284214+0000","last_became_active":"2024-04-23T14:36:30.283741+0000","last_became_peered":"2024-04-23T14:36:30.283741+0000","last_unstale":"2024-04-23T14:36:32.284214+0000","last_undegraded":"2024-04-23T14:36:32.284214+0000","last_fullsized":"2024-04-23T14:36:32.284214+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:04:22.818538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.264495+0000","last_change":"2024-04-23T14:36:29.252722+0000","last_active":"2024-04-23T14:36:31.264495+0000","last_peered":"2024-04-23T14:36:31.264495+0000","last_clean":"2024-04-23T14:36:31.264495+0000","last_became_active":"2024-04-23T14:36:29.252421+0000","last_became_peered":"2024-04-23T14:36:29.252421+0000","last_unstale":"2024-04-23T14:36:31.264495+0000","last_undegraded":"2024-04-23T14:36:31.264495+0000","last_fullsized":"2024-04-23T14:36:31.264495+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:18:16.720971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262962+0000","last_change":"2024-04-23T14:36:30.263935+0000","last_active":"2024-04-23T14:36:31.262962+0000","last_peered":"2024-04-23T14:36:31.262962+0000","last_clean":"2024-04-23T14:36:31.262962+0000","last_became_active":"2024-04-23T14:36:30.263590+0000","last_became_peered":"2024-04-23T14:36:30.263590+0000","last_unstale":"2024-04-23T14:36:31.262962+0000","last_undegraded":"2024-04-23T14:36:31.262962+0000","last_fullsized":"2024-04-23T14:36:31.262962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:01:39.745735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283913+0000","last_change":"2024-04-23T14:36:29.255338+0000","last_active":"2024-04-23T14:36:32.283913+0000","last_peered":"2024-04-23T14:36:32.283913+0000","last_clean":"2024-04-23T14:36:32.283913+0000","last_became_active":"2024-04-23T14:36:29.254916+0000","last_became_peered":"2024-04-23T14:36:29.254916+0000","last_unstale":"2024-04-23T14:36:32.283913+0000","last_undegraded":"2024-04-23T14:36:32.283913+0000","last_fullsized":"2024-04-23T14:36:32.283913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:03.094079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257203+0000","last_change":"2024-04-23T14:36:30.267547+0000","last_active":"2024-04-23T14:36:31.257203+0000","last_peered":"2024-04-23T14:36:31.257203+0000","last_clean":"2024-04-23T14:36:31.257203+0000","last_became_active":"2024-04-23T14:36:30.267103+0000","last_became_peered":"2024-04-23T14:36:30.267103+0000","last_unstale":"2024-04-23T14:36:31.257203+0000","last_undegraded":"2024-04-23T14:36:31.257203+0000","last_fullsized":"2024-04-23T14:36:31.257203+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:27:14.787860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282997+0000","last_change":"2024-04-23T14:36:29.257996+0000","last_active":"2024-04-23T14:36:32.282997+0000","last_peered":"2024-04-23T14:36:32.282997+0000","last_clean":"2024-04-23T14:36:32.282997+0000","last_became_active":"2024-04-23T14:36:29.256591+0000","last_became_peered":"2024-04-23T14:36:29.256591+0000","last_unstale":"2024-04-23T14:36:32.282997+0000","last_undegraded":"2024-04-23T14:36:32.282997+0000","last_fullsized":"2024-04-23T14:36:32.282997+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:18:16.720971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283084+0000","last_change":"2024-04-23T14:36:30.279080+0000","last_active":"2024-04-23T14:36:32.283084+0000","last_peered":"2024-04-23T14:36:32.283084+0000","last_clean":"2024-04-23T14:36:32.283084+0000","last_became_active":"2024-04-23T14:36:30.278726+0000","last_became_peered":"2024-04-23T14:36:30.278726+0000","last_unstale":"2024-04-23T14:36:32.283084+0000","last_undegraded":"2024-04-23T14:36:32.283084+0000","last_fullsized":"2024-04-23T14:36:32.283084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:01.140299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282322+0000","last_change":"2024-04-23T14:36:29.262784+0000","last_active":"2024-04-23T14:36:32.282322+0000","last_peered":"2024-04-23T14:36:32.282322+0000","last_clean":"2024-04-23T14:36:32.282322+0000","last_became_active":"2024-04-23T14:36:29.262418+0000","last_became_peered":"2024-04-23T14:36:29.262418+0000","last_unstale":"2024-04-23T14:36:32.282322+0000","last_undegraded":"2024-04-23T14:36:32.282322+0000","last_fullsized":"2024-04-23T14:36:32.282322+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:41.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282301+0000","last_change":"2024-04-23T14:36:30.276664+0000","last_active":"2024-04-23T14:36:32.282301+0000","last_peered":"2024-04-23T14:36:32.282301+0000","last_clean":"2024-04-23T14:36:32.282301+0000","last_became_active":"2024-04-23T14:36:30.276372+0000","last_became_peered":"2024-04-23T14:36:30.276372+0000","last_unstale":"2024-04-23T14:36:32.282301+0000","last_undegraded":"2024-04-23T14:36:32.282301+0000","last_fullsized":"2024-04-23T14:36:32.282301+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:59:01.140299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263742+0000","last_change":"2024-04-23T14:36:29.256582+0000","last_active":"2024-04-23T14:36:31.263742+0000","last_peered":"2024-04-23T14:36:31.263742+0000","last_clean":"2024-04-23T14:36:31.263742+0000","last_became_active":"2024-04-23T14:36:29.256299+0000","last_became_peered":"2024-04-23T14:36:29.256299+0000","last_unstale":"2024-04-23T14:36:31.263742+0000","last_undegraded":"2024-04-23T14:36:31.263742+0000","last_fullsized":"2024-04-23T14:36:31.263742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:11.056979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280325+0000","last_change":"2024-04-23T14:36:30.270953+0000","last_active":"2024-04-23T14:36:32.280325+0000","last_peered":"2024-04-23T14:36:32.280325+0000","last_clean":"2024-04-23T14:36:32.280325+0000","last_became_active":"2024-04-23T14:36:30.270639+0000","last_became_peered":"2024-04-23T14:36:30.270639+0000","last_unstale":"2024-04-23T14:36:32.280325+0000","last_undegraded":"2024-04-23T14:36:32.280325+0000","last_fullsized":"2024-04-23T14:36:32.280325+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:25:52.453599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281535+0000","last_change":"2024-04-23T14:36:29.254735+0000","last_active":"2024-04-23T14:36:32.281535+0000","last_peered":"2024-04-23T14:36:32.281535+0000","last_clean":"2024-04-23T14:36:32.281535+0000","last_became_active":"2024-04-23T14:36:29.253176+0000","last_became_peered":"2024-04-23T14:36:29.253176+0000","last_unstale":"2024-04-23T14:36:32.281535+0000","last_undegraded":"2024-04-23T14:36:32.281535+0000","last_fullsized":"2024-04-23T14:36:32.281535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:30:38.821608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256827+0000","last_change":"2024-04-23T14:36:30.254759+0000","last_active":"2024-04-23T14:36:31.256827+0000","last_peered":"2024-04-23T14:36:31.256827+0000","last_clean":"2024-04-23T14:36:31.256827+0000","last_became_active":"2024-04-23T14:36:30.254477+0000","last_became_peered":"2024-04-23T14:36:30.254477+0000","last_unstale":"2024-04-23T14:36:31.256827+0000","last_undegraded":"2024-04-23T14:36:31.256827+0000","last_fullsized":"2024-04-23T14:36:31.256827+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:10:15.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.260271+0000","last_change":"2024-04-23T14:36:29.263312+0000","last_active":"2024-04-23T14:36:31.260271+0000","last_peered":"2024-04-23T14:36:31.260271+0000","last_clean":"2024-04-23T14:36:31.260271+0000","last_became_active":"2024-04-23T14:36:29.262939+0000","last_became_peered":"2024-04-23T14:36:29.262939+0000","last_unstale":"2024-04-23T14:36:31.260271+0000","last_undegraded":"2024-04-23T14:36:31.260271+0000","last_fullsized":"2024-04-23T14:36:31.260271+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:34:41.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262105+0000","last_change":"2024-04-23T14:36:30.267339+0000","last_active":"2024-04-23T14:36:31.262105+0000","last_peered":"2024-04-23T14:36:31.262105+0000","last_clean":"2024-04-23T14:36:31.262105+0000","last_became_active":"2024-04-23T14:36:30.267009+0000","last_became_peered":"2024-04-23T14:36:30.267009+0000","last_unstale":"2024-04-23T14:36:31.262105+0000","last_undegraded":"2024-04-23T14:36:31.262105+0000","last_fullsized":"2024-04-23T14:36:31.262105+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:30.768940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263672+0000","last_change":"2024-04-23T14:36:29.251466+0000","last_active":"2024-04-23T14:36:31.263672+0000","last_peered":"2024-04-23T14:36:31.263672+0000","last_clean":"2024-04-23T14:36:31.263672+0000","last_became_active":"2024-04-23T14:36:29.250985+0000","last_became_peered":"2024-04-23T14:36:29.250985+0000","last_unstale":"2024-04-23T14:36:31.263672+0000","last_undegraded":"2024-04-23T14:36:31.263672+0000","last_fullsized":"2024-04-23T14:36:31.263672+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:56:10.745359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.284367+0000","last_change":"2024-04-23T14:36:30.275700+0000","last_active":"2024-04-23T14:36:32.284367+0000","last_peered":"2024-04-23T14:36:32.284367+0000","last_clean":"2024-04-23T14:36:32.284367+0000","last_became_active":"2024-04-23T14:36:30.275441+0000","last_became_peered":"2024-04-23T14:36:30.275441+0000","last_unstale":"2024-04-23T14:36:32.284367+0000","last_undegraded":"2024-04-23T14:36:32.284367+0000","last_fullsized":"2024-04-23T14:36:32.284367+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:04.112217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262179+0000","last_change":"2024-04-23T14:36:29.257682+0000","last_active":"2024-04-23T14:36:31.262179+0000","last_peered":"2024-04-23T14:36:31.262179+0000","last_clean":"2024-04-23T14:36:31.262179+0000","last_became_active":"2024-04-23T14:36:29.257385+0000","last_became_peered":"2024-04-23T14:36:29.257385+0000","last_unstale":"2024-04-23T14:36:31.262179+0000","last_undegraded":"2024-04-23T14:36:31.262179+0000","last_fullsized":"2024-04-23T14:36:31.262179+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:00.075116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282653+0000","last_change":"2024-04-23T14:36:30.274308+0000","last_active":"2024-04-23T14:36:32.282653+0000","last_peered":"2024-04-23T14:36:32.282653+0000","last_clean":"2024-04-23T14:36:32.282653+0000","last_became_active":"2024-04-23T14:36:30.273351+0000","last_became_peered":"2024-04-23T14:36:30.273351+0000","last_unstale":"2024-04-23T14:36:32.282653+0000","last_undegraded":"2024-04-23T14:36:32.282653+0000","last_fullsized":"2024-04-23T14:36:32.282653+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:22:20.138211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263315+0000","last_change":"2024-04-23T14:36:29.262796+0000","last_active":"2024-04-23T14:36:31.263315+0000","last_peered":"2024-04-23T14:36:31.263315+0000","last_clean":"2024-04-23T14:36:31.263315+0000","last_became_active":"2024-04-23T14:36:29.262504+0000","last_became_peered":"2024-04-23T14:36:29.262504+0000","last_unstale":"2024-04-23T14:36:31.263315+0000","last_undegraded":"2024-04-23T14:36:31.263315+0000","last_fullsized":"2024-04-23T14:36:31.263315+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:45.779744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257317+0000","last_change":"2024-04-23T14:36:30.260184+0000","last_active":"2024-04-23T14:36:31.257317+0000","last_peered":"2024-04-23T14:36:31.257317+0000","last_clean":"2024-04-23T14:36:31.257317+0000","last_became_active":"2024-04-23T14:36:30.259450+0000","last_became_peered":"2024-04-23T14:36:30.259450+0000","last_unstale":"2024-04-23T14:36:31.257317+0000","last_undegraded":"2024-04-23T14:36:31.257317+0000","last_fullsized":"2024-04-23T14:36:31.257317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:36.211595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283570+0000","last_change":"2024-04-23T14:36:29.267357+0000","last_active":"2024-04-23T14:36:32.283570+0000","last_peered":"2024-04-23T14:36:32.283570+0000","last_clean":"2024-04-23T14:36:32.283570+0000","last_became_active":"2024-04-23T14:36:29.266992+0000","last_became_peered":"2024-04-23T14:36:29.266992+0000","last_unstale":"2024-04-23T14:36:32.283570+0000","last_undegraded":"2024-04-23T14:36:32.283570+0000","last_fullsized":"2024-04-23T14:36:32.283570+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:30:38.821608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283689+0000","last_change":"2024-04-23T14:36:30.277980+0000","last_active":"2024-04-23T14:36:32.283689+0000","last_peered":"2024-04-23T14:36:32.283689+0000","last_clean":"2024-04-23T14:36:32.283689+0000","last_became_active":"2024-04-23T14:36:30.277796+0000","last_became_peered":"2024-04-23T14:36:30.277796+0000","last_unstale":"2024-04-23T14:36:32.283689+0000","last_undegraded":"2024-04-23T14:36:32.283689+0000","last_fullsized":"2024-04-23T14:36:32.283689+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:44.683693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263815+0000","last_change":"2024-04-23T14:36:29.254896+0000","last_active":"2024-04-23T14:36:31.263815+0000","last_peered":"2024-04-23T14:36:31.263815+0000","last_clean":"2024-04-23T14:36:31.263815+0000","last_became_active":"2024-04-23T14:36:29.254593+0000","last_became_peered":"2024-04-23T14:36:29.254593+0000","last_unstale":"2024-04-23T14:36:31.263815+0000","last_undegraded":"2024-04-23T14:36:31.263815+0000","last_fullsized":"2024-04-23T14:36:31.263815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:16:40.504300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283215+0000","last_change":"2024-04-23T14:36:30.263085+0000","last_active":"2024-04-23T14:36:32.283215+0000","last_peered":"2024-04-23T14:36:32.283215+0000","last_clean":"2024-04-23T14:36:32.283215+0000","last_became_active":"2024-04-23T14:36:30.262877+0000","last_became_peered":"2024-04-23T14:36:30.262877+0000","last_unstale":"2024-04-23T14:36:32.283215+0000","last_undegraded":"2024-04-23T14:36:32.283215+0000","last_fullsized":"2024-04-23T14:36:32.283215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:22.180698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281814+0000","last_change":"2024-04-23T14:36:29.259253+0000","last_active":"2024-04-23T14:36:32.281814+0000","last_peered":"2024-04-23T14:36:32.281814+0000","last_clean":"2024-04-23T14:36:32.281814+0000","last_became_active":"2024-04-23T14:36:29.256871+0000","last_became_peered":"2024-04-23T14:36:29.256871+0000","last_unstale":"2024-04-23T14:36:32.281814+0000","last_undegraded":"2024-04-23T14:36:32.281814+0000","last_fullsized":"2024-04-23T14:36:32.281814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:18.617182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283551+0000","last_change":"2024-04-23T14:36:30.268952+0000","last_active":"2024-04-23T14:36:32.283551+0000","last_peered":"2024-04-23T14:36:32.283551+0000","last_clean":"2024-04-23T14:36:32.283551+0000","last_became_active":"2024-04-23T14:36:30.268388+0000","last_became_peered":"2024-04-23T14:36:30.268388+0000","last_unstale":"2024-04-23T14:36:32.283551+0000","last_undegraded":"2024-04-23T14:36:32.283551+0000","last_fullsized":"2024-04-23T14:36:32.283551+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:00:42.895182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280140+0000","last_change":"2024-04-23T14:36:29.253692+0000","last_active":"2024-04-23T14:36:32.280140+0000","last_peered":"2024-04-23T14:36:32.280140+0000","last_clean":"2024-04-23T14:36:32.280140+0000","last_became_active":"2024-04-23T14:36:29.253295+0000","last_became_peered":"2024-04-23T14:36:29.253295+0000","last_unstale":"2024-04-23T14:36:32.280140+0000","last_undegraded":"2024-04-23T14:36:32.280140+0000","last_fullsized":"2024-04-23T14:36:32.280140+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:48.479785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281451+0000","last_change":"2024-04-23T14:36:30.255727+0000","last_active":"2024-04-23T14:36:32.281451+0000","last_peered":"2024-04-23T14:36:32.281451+0000","last_clean":"2024-04-23T14:36:32.281451+0000","last_became_active":"2024-04-23T14:36:30.255347+0000","last_became_peered":"2024-04-23T14:36:30.255347+0000","last_unstale":"2024-04-23T14:36:32.281451+0000","last_undegraded":"2024-04-23T14:36:32.281451+0000","last_fullsized":"2024-04-23T14:36:32.281451+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:52.345173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263834+0000","last_change":"2024-04-23T14:36:29.249580+0000","last_active":"2024-04-23T14:36:31.263834+0000","last_peered":"2024-04-23T14:36:31.263834+0000","last_clean":"2024-04-23T14:36:31.263834+0000","last_became_active":"2024-04-23T14:36:29.249248+0000","last_became_peered":"2024-04-23T14:36:29.249248+0000","last_unstale":"2024-04-23T14:36:31.263834+0000","last_undegraded":"2024-04-23T14:36:31.263834+0000","last_fullsized":"2024-04-23T14:36:31.263834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:07:00.944096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282882+0000","last_change":"2024-04-23T14:36:30.278993+0000","last_active":"2024-04-23T14:36:32.282882+0000","last_peered":"2024-04-23T14:36:32.282882+0000","last_clean":"2024-04-23T14:36:32.282882+0000","last_became_active":"2024-04-23T14:36:30.278089+0000","last_became_peered":"2024-04-23T14:36:30.278089+0000","last_unstale":"2024-04-23T14:36:32.282882+0000","last_undegraded":"2024-04-23T14:36:32.282882+0000","last_fullsized":"2024-04-23T14:36:32.282882+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:04.291326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.282648+0000","last_change":"2024-04-23T14:36:29.253372+0000","last_active":"2024-04-23T14:36:32.282648+0000","last_peered":"2024-04-23T14:36:32.282648+0000","last_clean":"2024-04-23T14:36:32.282648+0000","last_became_active":"2024-04-23T14:36:29.251760+0000","last_became_peered":"2024-04-23T14:36:29.251760+0000","last_unstale":"2024-04-23T14:36:32.282648+0000","last_undegraded":"2024-04-23T14:36:32.282648+0000","last_fullsized":"2024-04-23T14:36:32.282648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:53.438649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283299+0000","last_change":"2024-04-23T14:36:30.284555+0000","last_active":"2024-04-23T14:36:32.283299+0000","last_peered":"2024-04-23T14:36:32.283299+0000","last_clean":"2024-04-23T14:36:32.283299+0000","last_became_active":"2024-04-23T14:36:30.284171+0000","last_became_peered":"2024-04-23T14:36:30.284171+0000","last_unstale":"2024-04-23T14:36:32.283299+0000","last_undegraded":"2024-04-23T14:36:32.283299+0000","last_fullsized":"2024-04-23T14:36:32.283299+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:20.075635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257414+0000","last_change":"2024-04-23T14:36:29.245427+0000","last_active":"2024-04-23T14:36:31.257414+0000","last_peered":"2024-04-23T14:36:31.257414+0000","last_clean":"2024-04-23T14:36:31.257414+0000","last_became_active":"2024-04-23T14:36:29.245034+0000","last_became_peered":"2024-04-23T14:36:29.245034+0000","last_unstale":"2024-04-23T14:36:31.257414+0000","last_undegraded":"2024-04-23T14:36:31.257414+0000","last_fullsized":"2024-04-23T14:36:31.257414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:13.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283984+0000","last_change":"2024-04-23T14:36:30.277676+0000","last_active":"2024-04-23T14:36:32.283984+0000","last_peered":"2024-04-23T14:36:32.283984+0000","last_clean":"2024-04-23T14:36:32.283984+0000","last_became_active":"2024-04-23T14:36:30.277405+0000","last_became_peered":"2024-04-23T14:36:30.277405+0000","last_unstale":"2024-04-23T14:36:32.283984+0000","last_undegraded":"2024-04-23T14:36:32.283984+0000","last_fullsized":"2024-04-23T14:36:32.283984+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:40.186848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.280948+0000","last_change":"2024-04-23T14:36:29.257966+0000","last_active":"2024-04-23T14:36:32.280948+0000","last_peered":"2024-04-23T14:36:32.280948+0000","last_clean":"2024-04-23T14:36:32.280948+0000","last_became_active":"2024-04-23T14:36:29.257687+0000","last_became_peered":"2024-04-23T14:36:29.257687+0000","last_unstale":"2024-04-23T14:36:32.280948+0000","last_undegraded":"2024-04-23T14:36:32.280948+0000","last_fullsized":"2024-04-23T14:36:32.280948+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:36.303077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.257650+0000","last_change":"2024-04-23T14:36:30.260028+0000","last_active":"2024-04-23T14:36:31.257650+0000","last_peered":"2024-04-23T14:36:31.257650+0000","last_clean":"2024-04-23T14:36:31.257650+0000","last_became_active":"2024-04-23T14:36:30.259713+0000","last_became_peered":"2024-04-23T14:36:30.259713+0000","last_unstale":"2024-04-23T14:36:31.257650+0000","last_undegraded":"2024-04-23T14:36:31.257650+0000","last_fullsized":"2024-04-23T14:36:31.257650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:02.099306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.256853+0000","last_change":"2024-04-23T14:36:29.249521+0000","last_active":"2024-04-23T14:36:31.256853+0000","last_peered":"2024-04-23T14:36:31.256853+0000","last_clean":"2024-04-23T14:36:31.256853+0000","last_became_active":"2024-04-23T14:36:29.249249+0000","last_became_peered":"2024-04-23T14:36:29.249249+0000","last_unstale":"2024-04-23T14:36:31.256853+0000","last_undegraded":"2024-04-23T14:36:31.256853+0000","last_fullsized":"2024-04-23T14:36:31.256853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:08:47.916797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263706+0000","last_change":"2024-04-23T14:36:30.283700+0000","last_active":"2024-04-23T14:36:31.263706+0000","last_peered":"2024-04-23T14:36:31.263706+0000","last_clean":"2024-04-23T14:36:31.263706+0000","last_became_active":"2024-04-23T14:36:30.283438+0000","last_became_peered":"2024-04-23T14:36:30.283438+0000","last_unstale":"2024-04-23T14:36:31.263706+0000","last_undegraded":"2024-04-23T14:36:31.263706+0000","last_fullsized":"2024-04-23T14:36:31.263706+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:35.008967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281910+0000","last_change":"2024-04-23T14:36:29.262517+0000","last_active":"2024-04-23T14:36:32.281910+0000","last_peered":"2024-04-23T14:36:32.281910+0000","last_clean":"2024-04-23T14:36:32.281910+0000","last_became_active":"2024-04-23T14:36:29.262167+0000","last_became_peered":"2024-04-23T14:36:29.262167+0000","last_unstale":"2024-04-23T14:36:32.281910+0000","last_undegraded":"2024-04-23T14:36:32.281910+0000","last_fullsized":"2024-04-23T14:36:32.281910+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:56:10.745359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.281979+0000","last_change":"2024-04-23T14:36:30.279760+0000","last_active":"2024-04-23T14:36:32.281979+0000","last_peered":"2024-04-23T14:36:32.281979+0000","last_clean":"2024-04-23T14:36:32.281979+0000","last_became_active":"2024-04-23T14:36:30.278726+0000","last_became_peered":"2024-04-23T14:36:30.278726+0000","last_unstale":"2024-04-23T14:36:32.281979+0000","last_undegraded":"2024-04-23T14:36:32.281979+0000","last_fullsized":"2024-04-23T14:36:32.281979+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:25:43.492723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.260275+0000","last_change":"2024-04-23T14:36:29.260899+0000","last_active":"2024-04-23T14:36:31.260275+0000","last_peered":"2024-04-23T14:36:31.260275+0000","last_clean":"2024-04-23T14:36:31.260275+0000","last_became_active":"2024-04-23T14:36:29.258766+0000","last_became_peered":"2024-04-23T14:36:29.258766+0000","last_unstale":"2024-04-23T14:36:31.260275+0000","last_undegraded":"2024-04-23T14:36:31.260275+0000","last_fullsized":"2024-04-23T14:36:31.260275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:01:45.779744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263069+0000","last_change":"2024-04-23T14:36:30.263572+0000","last_active":"2024-04-23T14:36:31.263069+0000","last_peered":"2024-04-23T14:36:31.263069+0000","last_clean":"2024-04-23T14:36:31.263069+0000","last_became_active":"2024-04-23T14:36:30.263174+0000","last_became_peered":"2024-04-23T14:36:30.263174+0000","last_unstale":"2024-04-23T14:36:31.263069+0000","last_undegraded":"2024-04-23T14:36:31.263069+0000","last_fullsized":"2024-04-23T14:36:31.263069+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:40:30.768940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.262400+0000","last_change":"2024-04-23T14:36:29.261017+0000","last_active":"2024-04-23T14:36:31.262400+0000","last_peered":"2024-04-23T14:36:31.262400+0000","last_clean":"2024-04-23T14:36:31.262400+0000","last_became_active":"2024-04-23T14:36:29.258667+0000","last_became_peered":"2024-04-23T14:36:29.258667+0000","last_unstale":"2024-04-23T14:36:31.262400+0000","last_undegraded":"2024-04-23T14:36:31.262400+0000","last_fullsized":"2024-04-23T14:36:31.262400+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:00.075116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283337+0000","last_change":"2024-04-23T14:36:30.268831+0000","last_active":"2024-04-23T14:36:32.283337+0000","last_peered":"2024-04-23T14:36:32.283337+0000","last_clean":"2024-04-23T14:36:32.283337+0000","last_became_active":"2024-04-23T14:36:30.268534+0000","last_became_peered":"2024-04-23T14:36:30.268534+0000","last_unstale":"2024-04-23T14:36:32.283337+0000","last_undegraded":"2024-04-23T14:36:32.283337+0000","last_fullsized":"2024-04-23T14:36:32.283337+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:29.210668+0000","last_clean_scrub_stamp":"2024-04-23T14:36:29.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:30:22.607948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:31.263916+0000","last_change":"2024-04-23T14:36:29.258243+0000","last_active":"2024-04-23T14:36:31.263916+0000","last_peered":"2024-04-23T14:36:31.263916+0000","last_clean":"2024-04-23T14:36:31.263916+0000","last_became_active":"2024-04-23T14:36:29.257918+0000","last_became_peered":"2024-04-23T14:36:29.257918+0000","last_unstale":"2024-04-23T14:36:31.263916+0000","last_undegraded":"2024-04-23T14:36:31.263916+0000","last_fullsized":"2024-04-23T14:36:31.263916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:36:28.192531+0000","last_clean_scrub_stamp":"2024-04-23T14:36:28.192531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:31.008207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-23T14:36:32.283390+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:36:32.283390+0000","last_peered":"2024-04-23T14:36:32.283390+0000","last_clean":"2024-04-23T14:36:32.283390+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:36:32.283390+0000","last_undegraded":"2024-04-23T14:36:32.283390+0000","last_fullsized":"2024-04-23T14:36:32.283390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":944641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574916,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":843776,"data_stored":353727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":353769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":851968,"data_stored":355269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":355885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":944025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:36:40.296 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:36:40.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:36:40.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.562+0000 7fee1c400640 1 Processor -- start 2024-04-23T14:36:40.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.562+0000 7fee1c400640 1 -- start start 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.563+0000 7fee1c400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee140b56f0 0x7fee1406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee1c400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee140d0a60 0x7fee1406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee1c400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1415bf60 con 0x7fee1406f340 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee1c400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fee1415c240 con 0x7fee140b56f0 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee1c400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee1415c520 con 0x7fee140d0a60 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee19a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee140b56f0 0x7fee1406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee19a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee140b56f0 0x7fee1406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42954/0 (socket says 172.21.15.59:42954) 2024-04-23T14:36:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee19a00640 1 -- 172.21.15.59:0/327619673 learned_addr learned my addr 172.21.15.59:0/327619673 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.564+0000 7fee1a400640 1 --2- 172.21.15.59:0/327619673 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee140d0a60 0x7fee1406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.565+0000 7fee19a00640 1 -- 172.21.15.59:0/327619673 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee140d0a60 msgr2=0x7fee1406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.565+0000 7fee19a00640 1 --2- 172.21.15.59:0/327619673 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee140d0a60 0x7fee1406f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.565+0000 7fee19a00640 1 -- 172.21.15.59:0/327619673 >> v1:172.21.15.76:6789/0 conn(0x7fee1406f340 legacy=0x7fee140d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.565+0000 7fee19a00640 1 -- 172.21.15.59:0/327619673 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1415c800 con 0x7fee140b56f0 2024-04-23T14:36:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.565+0000 7fee1a400640 1 --2- 172.21.15.59:0/327619673 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee140d0a60 0x7fee1406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.566+0000 7fee19a00640 1 --2- 172.21.15.59:0/327619673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140b56f0 0x7fee1406ddd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fee08009870 tx=0x7fee0802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246c65377f215134 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.566+0000 7fee13e00640 1 -- 172.21.15.59:0/327619673 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08037070 con 0x7fee140b56f0 2024-04-23T14:36:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.566+0000 7fee13e00640 1 -- 172.21.15.59:0/327619673 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee08032440 con 0x7fee140b56f0 2024-04-23T14:36:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.566+0000 7fee13e00640 1 -- 172.21.15.59:0/327619673 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08032810 con 0x7fee140b56f0 2024-04-23T14:36:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 -- 172.21.15.59:0/327619673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140b56f0 msgr2=0x7fee1406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 --2- 172.21.15.59:0/327619673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140b56f0 0x7fee1406ddd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fee08009870 tx=0x7fee0802a200 comp rx=0 tx=0).stop 2024-04-23T14:36:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 -- 172.21.15.59:0/327619673 shutdown_connections 2024-04-23T14:36:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 --2- 172.21.15.59:0/327619673 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee140d0a60 0x7fee1406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 --2- 172.21.15.59:0/327619673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140b56f0 0x7fee1406ddd0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 -- 172.21.15.59:0/327619673 >> 172.21.15.59:0/327619673 conn(0x7fee14067d90 msgr2=0x7fee1406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 -- 172.21.15.59:0/327619673 shutdown_connections 2024-04-23T14:36:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.567+0000 7fee1c400640 1 -- 172.21.15.59:0/327619673 wait complete. 2024-04-23T14:36:40.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.569+0000 7fee1c400640 1 Processor -- start 2024-04-23T14:36:40.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.569+0000 7fee1c400640 1 -- start start 2024-04-23T14:36:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.570+0000 7fee1c400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee1406f340 0x7fee14141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.570+0000 7fee1c400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee140b56f0 0x7fee14156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.570+0000 7fee1c400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140d0a60 0x7fee141574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.570+0000 7fee1c400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee140b3710 con 0x7fee140d0a60 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.570+0000 7fee1c400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee1415bc10 con 0x7fee140b56f0 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.570+0000 7fee1c400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee140b3890 con 0x7fee1406f340 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140d0a60 0x7fee141574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140d0a60 0x7fee141574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42958/0 (socket says 172.21.15.59:42958) 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 -- 172.21.15.59:0/1758770017 learned_addr learned my addr 172.21.15.59:0/1758770017 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee19a00640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee1406f340 0x7fee14141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee19000640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee140b56f0 0x7fee14156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 -- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee1406f340 msgr2=0x7fee14141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee1406f340 0x7fee14141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 -- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee140b56f0 msgr2=0x7fee14156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee140b56f0 0x7fee14156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.571+0000 7fee1a400640 1 -- 172.21.15.59:0/1758770017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee140cefc0 con 0x7fee140d0a60 2024-04-23T14:36:40.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.572+0000 7fee1a400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140d0a60 0x7fee141574b0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00cdb0 tx=0x7fee0c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:40.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.572+0000 7fee11600640 1 -- 172.21.15.59:0/1758770017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00f680 con 0x7fee140d0a60 2024-04-23T14:36:40.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.572+0000 7fee11600640 1 -- 172.21.15.59:0/1758770017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c00f840 con 0x7fee140d0a60 2024-04-23T14:36:40.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.572+0000 7fee11600640 1 -- 172.21.15.59:0/1758770017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00fc30 con 0x7fee140d0a60 2024-04-23T14:36:40.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.572+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee140cf2a0 con 0x7fee140d0a60 2024-04-23T14:36:40.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.572+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee140cf860 con 0x7fee140d0a60 2024-04-23T14:36:40.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.573+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee14068a80 con 0x7fee140d0a60 2024-04-23T14:36:40.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.574+0000 7fee11600640 1 -- 172.21.15.59:0/1758770017 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee0c0213f0 con 0x7fee140d0a60 2024-04-23T14:36:40.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.575+0000 7fee11600640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fedf406c8f0 0x7fedf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:40.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.575+0000 7fee11600640 1 -- 172.21.15.59:0/1758770017 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fee0c08c8c0 con 0x7fee140d0a60 2024-04-23T14:36:40.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.575+0000 7fee19a00640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fedf406c8f0 0x7fedf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:40.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.576+0000 7fee19a00640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fedf406c8f0 0x7fedf406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee08034000 tx=0x7fee0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:40.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.580+0000 7fee11600640 1 -- 172.21.15.59:0/1758770017 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee0c014030 con 0x7fee140d0a60 2024-04-23T14:36:40.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.859+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fee1406ef20 con 0x7fee140d0a60 2024-04-23T14:36:40.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.863+0000 7fee11600640 1 -- 172.21.15.59:0/1758770017 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fee0c05ace0 con 0x7fee140d0a60 2024-04-23T14:36:40.863 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:36:40.863 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:36:40.863 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-23T14:36:40.863 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-23T14:36:40.863 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:36:40.863 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.865+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fedf406c8f0 msgr2=0x7fedf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.865+0000 7fee1c400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fedf406c8f0 0x7fedf406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee08034000 tx=0x7fee0802e000 comp rx=0 tx=0).stop 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.865+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140d0a60 msgr2=0x7fee141574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.865+0000 7fee1c400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140d0a60 0x7fee141574b0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00cdb0 tx=0x7fee0c00a650 comp rx=0 tx=0).stop 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.865+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 shutdown_connections 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.865+0000 7fee1c400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fedf406c8f0 0x7fedf406edb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.866+0000 7fee1c400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee140d0a60 0x7fee141574b0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.866+0000 7fee1c400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee140b56f0 0x7fee14156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.866+0000 7fee1c400640 1 --2- 172.21.15.59:0/1758770017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee1406f340 0x7fee14141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.866+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 >> 172.21.15.59:0/1758770017 conn(0x7fee14067d90 msgr2=0x7fee14157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.866+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 shutdown_connections 2024-04-23T14:36:40.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:40.866+0000 7fee1c400640 1 -- 172.21.15.59:0/1758770017 wait complete. 2024-04-23T14:36:40.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:41.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.091+0000 7f2b82e00640 1 Processor -- start 2024-04-23T14:36:41.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.092+0000 7f2b82e00640 1 -- start start 2024-04-23T14:36:41.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.093+0000 7f2b82e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b7c15c9d0 0x7f2b7c09b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:41.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.093+0000 7f2b82e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0d09e0 0x7f2b7c0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:41.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.093+0000 7f2b82e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c0b35f0 con 0x7f2b7c0b2e20 2024-04-23T14:36:41.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.093+0000 7f2b82e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c0cd2d0 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.093+0000 7f2b82e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c0d13b0 con 0x7f2b7c0d09e0 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b81a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b7c15c9d0 0x7f2b7c09b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b81a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b7c15c9d0 0x7f2b7c09b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42966/0 (socket says 172.21.15.59:42966) 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b81a00640 1 -- 172.21.15.59:0/3515867353 learned_addr learned my addr 172.21.15.59:0/3515867353 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b82400640 1 --2- 172.21.15.59:0/3515867353 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0d09e0 0x7f2b7c0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b81a00640 1 -- 172.21.15.59:0/3515867353 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0d09e0 msgr2=0x7f2b7c0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b81a00640 1 --2- 172.21.15.59:0/3515867353 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0d09e0 0x7f2b7c0d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b81a00640 1 -- 172.21.15.59:0/3515867353 >> v1:172.21.15.76:6789/0 conn(0x7f2b7c0b2e20 legacy=0x7f2b7c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b81a00640 1 -- 172.21.15.59:0/3515867353 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0b3890 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.094+0000 7f2b82400640 1 --2- 172.21.15.59:0/3515867353 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0d09e0 0x7f2b7c0d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:41.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.095+0000 7f2b81a00640 1 --2- 172.21.15.59:0/3515867353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c09b410 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009870 tx=0x7f2b6c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d58012664085e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:41.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.096+0000 7f2b7be00640 1 -- 172.21.15.59:0/3515867353 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c037070 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.096+0000 7f2b7be00640 1 -- 172.21.15.59:0/3515867353 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c032440 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.096+0000 7f2b7be00640 1 -- 172.21.15.59:0/3515867353 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c032810 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.096+0000 7f2b82e00640 1 -- 172.21.15.59:0/3515867353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 msgr2=0x7f2b7c09b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.096+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3515867353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c09b410 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009870 tx=0x7f2b6c02a200 comp rx=0 tx=0).stop 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.097+0000 7f2b82e00640 1 -- 172.21.15.59:0/3515867353 shutdown_connections 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.097+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3515867353 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0d09e0 0x7f2b7c0d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.097+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3515867353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c09b410 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.097+0000 7f2b82e00640 1 -- 172.21.15.59:0/3515867353 >> 172.21.15.59:0/3515867353 conn(0x7f2b7c0b3ef0 msgr2=0x7f2b7c0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.097+0000 7f2b82e00640 1 -- 172.21.15.59:0/3515867353 shutdown_connections 2024-04-23T14:36:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.097+0000 7f2b82e00640 1 -- 172.21.15.59:0/3515867353 wait complete. 2024-04-23T14:36:41.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.098+0000 7f2b82e00640 1 Processor -- start 2024-04-23T14:36:41.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.099+0000 7f2b82e00640 1 -- start start 2024-04-23T14:36:41.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b2e20 0x7f2b7c15f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:41.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c0d09e0 0x7f2b7c0caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:41.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:41.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0b3280 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0ccef0 con 0x7f2b7c0d09e0 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c15cdb0 con 0x7f2b7c0b2e20 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c0cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42968/0 (socket says 172.21.15.59:42968) 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b82400640 1 -- 172.21.15.59:0/3751450100 learned_addr learned my addr 172.21.15.59:0/3751450100 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b81a00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b2e20 0x7f2b7c15f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.100+0000 7f2b81000640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c0d09e0 0x7f2b7c0caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82400640 1 -- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b2e20 msgr2=0x7f2b7c15f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82400640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b2e20 0x7f2b7c15f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82400640 1 -- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c0d09e0 msgr2=0x7f2b7c0caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82400640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c0d09e0 0x7f2b7c0caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82400640 1 -- 172.21.15.59:0/3751450100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c1591b0 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82400640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c0cb4c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400cef0 tx=0x7f2b7400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b79600640 1 -- 172.21.15.59:0/3751450100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7401a070 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b79600640 1 -- 172.21.15.59:0/3751450100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7400d860 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b79600640 1 -- 172.21.15.59:0/3751450100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400dc50 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c0ca080 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.101+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c0ca600 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.103+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c04fd90 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.104+0000 7f2b79600640 1 -- 172.21.15.59:0/3751450100 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b7401d070 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.106+0000 7f2b79600640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c940 0x7f2b6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.106+0000 7f2b79600640 1 -- 172.21.15.59:0/3751450100 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f2b7408c2f0 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.106+0000 7f2b81a00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c940 0x7f2b6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:41.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.107+0000 7f2b81a00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c940 0x7f2b6006ee00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c034000 tx=0x7f2b6c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:41.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.110+0000 7f2b79600640 1 -- 172.21.15.59:0/3751450100 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b7c1591b0 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.353+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b7c0d07c0 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.354+0000 7f2b79600640 1 -- 172.21.15.59:0/3751450100 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5183 (secure 0 0 0) 0x7f2b7405a950 con 0x7f2b7c15c9d0 2024-04-23T14:36:41.354 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:41.354 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:36:32.769281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6735":{"gid":6735,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6836/2883460093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2883460093},{"type":"v1","addr":"172.21.15.59:6836","nonce":2883460093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6735,"qdb_cluster":[6735]},"id":5}]} 2024-04-23T14:36:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 43 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.356+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c940 msgr2=0x7f2b6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.356+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c940 0x7f2b6006ee00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c034000 tx=0x7f2b6c02e000 comp rx=0 tx=0).stop 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.356+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 msgr2=0x7f2b7c0cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.356+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c0cb4c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400cef0 tx=0x7f2b7400a710 comp rx=0 tx=0).stop 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 shutdown_connections 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b6006c940 0x7f2b6006ee00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c15c9d0 0x7f2b7c0cb4c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c0d09e0 0x7f2b7c0caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 --2- 172.21.15.59:0/3751450100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b2e20 0x7f2b7c15f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 >> 172.21.15.59:0/3751450100 conn(0x7f2b7c0b3ef0 msgr2=0x7f2b7c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 shutdown_connections 2024-04-23T14:36:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:41.357+0000 7f2b82e00640 1 -- 172.21.15.59:0/3751450100 wait complete. 2024-04-23T14:36:41.780 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:36:29.710826+0000', 'modified': '2024-04-23T14:36:32.769281+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6735}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6735': {'gid': 6735, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6836/2883460093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2883460093}, {'type': 'v1', 'addr': '172.21.15.59:6836', 'nonce': 2883460093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6735, 'qdb_cluster': [6735]} 2024-04-23T14:36:41.780 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:36:41.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-23T14:36:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.000+0000 7f0f44200640 1 Processor -- start 2024-04-23T14:36:42.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.000+0000 7f0f44200640 1 -- start start 2024-04-23T14:36:42.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.002+0000 7f0f44200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0f3c13fa10 0x7f0f3c165a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.002+0000 7f0f44200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3c0b4670 con 0x7f0f3c072620 2024-04-23T14:36:42.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.002+0000 7f0f44200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3c0b4950 con 0x7f0f3c141290 2024-04-23T14:36:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.002+0000 7f0f44200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0f3c0b51f0 con 0x7f0f3c13fa10 2024-04-23T14:36:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.002+0000 7f0f41800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0f3c072620 0x7f0f3c072a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:43988/0 (socket says 172.21.15.59:43988) 2024-04-23T14:36:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.002+0000 7f0f41800640 1 -- 172.21.15.59:0/2169487569 learned_addr learned my addr 172.21.15.59:0/2169487569 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.002+0000 7f0f42200640 1 --2- 172.21.15.59:0/2169487569 >> v2:172.21.15.59:3300/0 conn(0x7f0f3c13fa10 0x7f0f3c165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.003+0000 7f0f42200640 1 -- 172.21.15.59:0/2169487569 >> v1:172.21.15.76:6789/0 conn(0x7f0f3c141290 legacy=0x7f0f3c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.003+0000 7f0f42200640 1 -- 172.21.15.59:0/2169487569 >> v1:172.21.15.59:6789/0 conn(0x7f0f3c072620 legacy=0x7f0f3c072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.003+0000 7f0f42200640 1 -- 172.21.15.59:0/2169487569 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0b4c30 con 0x7f0f3c13fa10 2024-04-23T14:36:42.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.004+0000 7f0f42200640 1 --2- 172.21.15.59:0/2169487569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c13fa10 0x7f0f3c165a60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009d90 tx=0x7f0f3402c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0580d67e8875bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.004+0000 7f0f3be00640 1 -- 172.21.15.59:0/2169487569 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3403a070 con 0x7f0f3c13fa10 2024-04-23T14:36:42.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.004+0000 7f0f3be00640 1 -- 172.21.15.59:0/2169487569 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f34007740 con 0x7f0f3c13fa10 2024-04-23T14:36:42.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.004+0000 7f0f3be00640 1 -- 172.21.15.59:0/2169487569 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f34007b10 con 0x7f0f3c13fa10 2024-04-23T14:36:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.005+0000 7f0f44200640 1 -- 172.21.15.59:0/2169487569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c13fa10 msgr2=0x7f0f3c165a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.005+0000 7f0f44200640 1 --2- 172.21.15.59:0/2169487569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c13fa10 0x7f0f3c165a60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009d90 tx=0x7f0f3402c830 comp rx=0 tx=0).stop 2024-04-23T14:36:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.005+0000 7f0f44200640 1 -- 172.21.15.59:0/2169487569 shutdown_connections 2024-04-23T14:36:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.005+0000 7f0f44200640 1 --2- 172.21.15.59:0/2169487569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c13fa10 0x7f0f3c165a60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.005+0000 7f0f44200640 1 -- 172.21.15.59:0/2169487569 >> 172.21.15.59:0/2169487569 conn(0x7f0f3c0b2130 msgr2=0x7f0f3c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.005+0000 7f0f44200640 1 -- 172.21.15.59:0/2169487569 shutdown_connections 2024-04-23T14:36:42.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.005+0000 7f0f44200640 1 -- 172.21.15.59:0/2169487569 wait complete. 2024-04-23T14:36:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.007+0000 7f0f44200640 1 Processor -- start 2024-04-23T14:36:42.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.007+0000 7f0f44200640 1 -- start start 2024-04-23T14:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.008+0000 7f0f44200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c072620 0x7f0f3c1f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.008+0000 7f0f44200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f3c13fa10 0x7f0f3c0c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.008+0000 7f0f44200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f3c141290 0x7f0f3c1f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.008+0000 7f0f44200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c15e4e0 con 0x7f0f3c072620 2024-04-23T14:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.008+0000 7f0f44200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c140a00 con 0x7f0f3c141290 2024-04-23T14:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.008+0000 7f0f44200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c15e660 con 0x7f0f3c13fa10 2024-04-23T14:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.008+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c072620 0x7f0f3c1f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c072620 0x7f0f3c1f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42986/0 (socket says 172.21.15.59:42986) 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 -- 172.21.15.59:0/2182511594 learned_addr learned my addr 172.21.15.59:0/2182511594 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f42200640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f3c141290 0x7f0f3c1f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f40e00640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f3c13fa10 0x7f0f3c0c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 -- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f3c13fa10 msgr2=0x7f0f3c0c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f3c13fa10 0x7f0f3c0c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 -- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f3c141290 msgr2=0x7f0f3c1f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f3c141290 0x7f0f3c1f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 -- 172.21.15.59:0/2182511594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c1f5740 con 0x7f0f3c072620 2024-04-23T14:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.009+0000 7f0f41800640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c072620 0x7f0f3c1f5200 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c3c0 tx=0x7f0f3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.010+0000 7f0f39600640 1 -- 172.21.15.59:0/2182511594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30017070 con 0x7f0f3c072620 2024-04-23T14:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.010+0000 7f0f39600640 1 -- 172.21.15.59:0/2182511594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30005ce0 con 0x7f0f3c072620 2024-04-23T14:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.010+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c143320 con 0x7f0f3c072620 2024-04-23T14:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.010+0000 7f0f39600640 1 -- 172.21.15.59:0/2182511594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f300072f0 con 0x7f0f3c072620 2024-04-23T14:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.010+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1438e0 con 0x7f0f3c072620 2024-04-23T14:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.012+0000 7f0f39600640 1 -- 172.21.15.59:0/2182511594 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f3001a070 con 0x7f0f3c072620 2024-04-23T14:36:42.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.013+0000 7f0f39600640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0f1c06c8f0 0x7f0f1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.013+0000 7f0f39600640 1 -- 172.21.15.59:0/2182511594 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3008aeb0 con 0x7f0f3c072620 2024-04-23T14:36:42.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.013+0000 7f0f40e00640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0f1c06c8f0 0x7f0f1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.014+0000 7f0f46fb5740 1 Processor -- start 2024-04-23T14:36:42.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.014+0000 7f0f46fb5740 1 -- start start 2024-04-23T14:36:42.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.014+0000 7f0f40e00640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0f1c06c8f0 0x7f0f1c06edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c1f8e30 tx=0x7f0f28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6d17b0 0x557dfa684140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6b48d0 0x557dfa6b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b4190 0x557dfa688fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557dfa5eeb10 con 0x557dfa6d17b0 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557dfa5952b0 con 0x557dfa6b48d0 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557dfa5eaad0 con 0x557dfa6b4190 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6d17b0 0x557dfa684140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6d17b0 0x557dfa684140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42996/0 (socket says 172.21.15.59:42996) 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 -- 172.21.15.59:0/1178806443 learned_addr learned my addr 172.21.15.59:0/1178806443 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f40e00640 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6b48d0 0x557dfa6b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 -- 172.21.15.59:0/1178806443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b4190 msgr2=0x557dfa688fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b4190 0x557dfa688fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 -- 172.21.15.59:0/1178806443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6b48d0 msgr2=0x557dfa6b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6b48d0 0x557dfa6b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.015+0000 7f0f41800640 1 -- 172.21.15.59:0/1178806443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dfa3012d0 con 0x557dfa6d17b0 2024-04-23T14:36:42.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.016+0000 7f0f41800640 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6d17b0 0x557dfa684140 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0f30061280 tx=0x7f0f30058820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb06ffda44f5e3c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.016+0000 7f0f2e000640 1 -- 172.21.15.59:0/1178806443 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30071180 con 0x557dfa6d17b0 2024-04-23T14:36:42.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.016+0000 7f0f2e000640 1 -- 172.21.15.59:0/1178806443 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30056ed0 con 0x557dfa6d17b0 2024-04-23T14:36:42.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.016+0000 7f0f2e000640 1 -- 172.21.15.59:0/1178806443 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30007490 con 0x557dfa6d17b0 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.016+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1178806443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6d17b0 msgr2=0x557dfa684140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.016+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6d17b0 0x557dfa684140 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0f30061280 tx=0x7f0f30058820 comp rx=0 tx=0).stop 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1178806443 shutdown_connections 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b4190 0x557dfa688fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6b48d0 0x557dfa6b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1178806443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6d17b0 0x557dfa684140 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1178806443 >> 172.21.15.59:0/1178806443 conn(0x557dfa6863b0 msgr2=0x557dfa6cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1178806443 shutdown_connections 2024-04-23T14:36:42.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1178806443 wait complete. 2024-04-23T14:36:42.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.017+0000 7f0f46fb5740 1 -- start start 2024-04-23T14:36:42.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.018+0000 7f0f46fb5740 1 Processor -- start 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6cec50 0x557dfa6cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557dfa688560 con 0x557dfa6b48d0 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557dfa5a8590 con 0x557dfa6d17b0 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557dfa687950 con 0x557dfa6cec50 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f46fb5740 10 client.0 _ll_register_callbacks cb 0x557dfa58b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43010/0 (socket says 172.21.15.59:43010) 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 -- 172.21.15.59:0/1363575117 learned_addr learned my addr 172.21.15.59:0/1363575117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 -- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6cec50 msgr2=0x557dfa6cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6cec50 0x557dfa6cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 -- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 msgr2=0x557dfa6c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f40e00640 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f41800640 1 -- 172.21.15.59:0/1363575117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dfa3cb6b0 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.019+0000 7f0f40e00640 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f41800640 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b71b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0f30089340 tx=0x7f0f30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63fb48567cddaef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f22a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f22a00640 1 -- 172.21.15.59:0/1363575117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30078070 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f22a00640 1 -- 172.21.15.59:0/1363575117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f0f300076a0 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f30056ed0 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f22a00640 1 -- 172.21.15.59:0/1363575117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0f30078210 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f46fb5740 10 client.6906 fetch_fsmap learned FSMap version 43 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x557dfa6c5910 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f22a00640 1 -- 172.21.15.59:0/1363575117 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30056ed0 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.020+0000 7f0f22a00640 1 -- 172.21.15.59:0/1363575117 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f0f30058020 con 0x557dfa6b48d0 2024-04-23T14:36:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.021+0000 7f0f46fb5740 10 client.6906 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.021+0000 7f0f46fb5740 10 client.6906 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.021+0000 7f0f46fb5740 20 client.6906 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.021+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa6a2150 0x557dfa6a2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.021+0000 7f0f46fb5740 4 client.6906 mds_command: new command op to 6735 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.021+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 --> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dfa5fc530 con 0x557dfa6a2150 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.021+0000 7f0f40e00640 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa6a2150 0x557dfa6a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.022+0000 7f0f40e00640 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa6a2150 0x557dfa6a2530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f866415e21198f1 server_cookie=4f01014a59f4d6ac in_seq=0 out_seq=0 2024-04-23T14:36:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.022+0000 7f0f22a00640 10 client.6906 ms_handle_connect on v2:172.21.15.59:6834/2883460093 2024-04-23T14:36:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.022+0000 7f0f22a00640 1 -- 172.21.15.59:0/1363575117 <== mds.0 v2:172.21.15.59:6834/2883460093 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557dfa5fc530 con 0x557dfa6a2150 2024-04-23T14:36:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.022+0000 7f0f22a00640 10 client.6906 handle_command_reply: tid=0 2024-04-23T14:36:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.022+0000 7f0f46fb5740 1 client.6906 shutdown 2024-04-23T14:36:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 msgr2=0x557dfa6b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b71b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0f30089340 tx=0x7f0f30000bc0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 shutdown_connections 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa6a2150 0x557dfa6a2530 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6cec50 0x557dfa6cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1363575117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b71b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f22a00640 0 client.6906 ms_handle_reset on v2:172.21.15.59:6834/2883460093 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 >> 172.21.15.59:0/1363575117 conn(0x557dfa6b4190 msgr2=0x557dfa6d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 shutdown_connections 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1363575117 wait complete. 2024-04-23T14:36:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.023+0000 7f0f46fb5740 20 client.6906 trim_cache size 0 max 16384 2024-04-23T14:36:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.043+0000 7f0f46fb5740 1 Processor -- start 2024-04-23T14:36:42.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.043+0000 7f0f46fb5740 1 -- start start 2024-04-23T14:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.044+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b48d0 0x557dfa6ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.044+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6cec50 0x557dfa686f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.044+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.044+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c200180 con 0x557dfa6cec50 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.044+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0add20 con 0x557dfa6d17b0 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.044+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c0ae370 con 0x557dfa6b48d0 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6cec50 0x557dfa686f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6cec50 0x557dfa686f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43016/0 (socket says 172.21.15.59:43016) 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 -- 172.21.15.59:0/2757015319 learned_addr learned my addr 172.21.15.59:0/2757015319 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f41800640 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b48d0 0x557dfa6ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f42200640 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 -- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b48d0 msgr2=0x557dfa6ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b48d0 0x557dfa6ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 -- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 msgr2=0x557dfa6ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 -- 172.21.15.59:0/2757015319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dfa685ff0 con 0x557dfa6cec50 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f42200640 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f41800640 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b48d0 0x557dfa6ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.045+0000 7f0f40e00640 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6cec50 0x557dfa686f50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x557dfa6c6430 tx=0x7f0f28014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb526355cebd4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f2d600640 1 -- 172.21.15.59:0/2757015319 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800eea0 con 0x557dfa6cec50 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f2d600640 1 -- 172.21.15.59:0/2757015319 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f28015900 con 0x557dfa6cec50 2024-04-23T14:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f2d600640 1 -- 172.21.15.59:0/2757015319 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28015cd0 con 0x557dfa6cec50 2024-04-23T14:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 -- 172.21.15.59:0/2757015319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6cec50 msgr2=0x557dfa686f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6cec50 0x557dfa686f50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x557dfa6c6430 tx=0x7f0f28014000 comp rx=0 tx=0).stop 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 -- 172.21.15.59:0/2757015319 shutdown_connections 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6d17b0 0x557dfa6ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6cec50 0x557dfa686f50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/2757015319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6b48d0 0x557dfa6ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 -- 172.21.15.59:0/2757015319 >> 172.21.15.59:0/2757015319 conn(0x557dfa6b4190 msgr2=0x557dfa68a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.046+0000 7f0f46fb5740 1 -- 172.21.15.59:0/2757015319 shutdown_connections 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.047+0000 7f0f46fb5740 1 -- 172.21.15.59:0/2757015319 wait complete. 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.047+0000 7f0f46fb5740 1 -- start start 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.047+0000 7f0f46fb5740 1 Processor -- start 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.048+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.048+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6cec50 0x557dfa6f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.048+0000 7f0f46fb5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6d17b0 0x557dfa683030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.048+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557dfa688560 con 0x557dfa6b48d0 2024-04-23T14:36:42.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.048+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557dfa6c79f0 con 0x557dfa6cec50 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.048+0000 7f0f46fb5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557dfa5a8590 con 0x557dfa6d17b0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.048+0000 7f0f46fb5740 10 client.0 _ll_register_callbacks cb 0x7f0f30057080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43024/0 (socket says 172.21.15.59:43024) 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 -- 172.21.15.59:0/1784908528 learned_addr learned my addr 172.21.15.59:0/1784908528 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 -- 172.21.15.59:0/1784908528 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6d17b0 msgr2=0x557dfa683030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6d17b0 0x557dfa683030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 -- 172.21.15.59:0/1784908528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6cec50 msgr2=0x557dfa6f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6cec50 0x557dfa6f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 -- 172.21.15.59:0/1784908528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dfa6cd240 con 0x557dfa6b48d0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f41800640 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b7bf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0f30061280 tx=0x7f0f30054f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe080aa40bc917c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f22a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.049+0000 7f0f22a00640 1 -- 172.21.15.59:0/1784908528 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30078040 con 0x557dfa6b48d0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f22a00640 1 -- 172.21.15.59:0/1784908528 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3005f3a0 con 0x557dfa6b48d0 2024-04-23T14:36:42.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f28015900 con 0x557dfa6b48d0 2024-04-23T14:36:42.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f22a00640 1 -- 172.21.15.59:0/1784908528 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0f30056ed0 con 0x557dfa6b48d0 2024-04-23T14:36:42.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 10 client.6912 fetch_fsmap learned FSMap version 43 2024-04-23T14:36:42.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x557dfa6cd240 con 0x557dfa6b48d0 2024-04-23T14:36:42.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f22a00640 1 -- 172.21.15.59:0/1784908528 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f28015900 con 0x557dfa6b48d0 2024-04-23T14:36:42.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f22a00640 1 -- 172.21.15.59:0/1784908528 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7f0f3006d0d0 con 0x557dfa6b48d0 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 10 client.6912 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 10 client.6912 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 20 client.6912 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa683a60 0x557dfa6e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 4 client.6912 mds_command: new command op to 6735 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 --> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] -- command(tid 0: {"prefix": "status"}) -- 0x557dfa685800 con 0x557dfa683a60 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.050+0000 7f0f40e00640 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa683a60 0x557dfa6e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f40e00640 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa683a60 0x557dfa6e0c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803861ebea6a1fa3 server_cookie=3e1b1eefd4ac8461 in_seq=0 out_seq=0 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f22a00640 10 client.6912 ms_handle_connect on v2:172.21.15.59:6834/2883460093 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f22a00640 1 -- 172.21.15.59:0/1784908528 <== mds.0 v2:172.21.15.59:6834/2883460093 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x557dfa685800 con 0x557dfa683a60 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f22a00640 10 client.6912 handle_command_reply: tid=0 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f46fb5740 1 client.6912 shutdown 2024-04-23T14:36:42.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 msgr2=0x557dfa6b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b7bf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0f30061280 tx=0x7f0f30054f10 comp rx=0 tx=0).stop 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 shutdown_connections 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:6834/2883460093,v1:172.21.15.59:6836/2883460093] conn(0x557dfa683a60 0x557dfa6e0c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.051+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557dfa6d17b0 0x557dfa683030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557dfa6cec50 0x557dfa6f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f46fb5740 1 --2- 172.21.15.59:0/1784908528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557dfa6b48d0 0x557dfa6b7bf0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 >> 172.21.15.59:0/1784908528 conn(0x557dfa6b4190 msgr2=0x557dfa6af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f22a00640 0 client.6912 ms_handle_reset on v2:172.21.15.59:6834/2883460093 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 shutdown_connections 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f46fb5740 1 -- 172.21.15.59:0/1784908528 wait complete. 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f46fb5740 20 client.6912 trim_cache size 0 max 16384 2024-04-23T14:36:42.055 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6735, 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 11.820022184999999, 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 43, 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 69, 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 69, 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 17.257751345999999 2024-04-23T14:36:42.056 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:36:42.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0f1c06c8f0 msgr2=0x7f0f1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f44200640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0f1c06c8f0 0x7f0f1c06edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c1f8e30 tx=0x7f0f28009040 comp rx=0 tx=0).stop 2024-04-23T14:36:42.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c072620 msgr2=0x7f0f3c1f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.052+0000 7f0f44200640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c072620 0x7f0f3c1f5200 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c3c0 tx=0x7f0f3000c880 comp rx=0 tx=0).stop 2024-04-23T14:36:42.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 shutdown_connections 2024-04-23T14:36:42.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0f1c06c8f0 0x7f0f1c06edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f3c13fa10 0x7f0f3c0c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f3c141290 0x7f0f3c1f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 --2- 172.21.15.59:0/2182511594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f3c072620 0x7f0f3c1f5200 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 >> 172.21.15.59:0/2182511594 conn(0x7f0f3c0b2130 msgr2=0x7f0f3c15d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 shutdown_connections 2024-04-23T14:36:42.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.053+0000 7f0f44200640 1 -- 172.21.15.59:0/2182511594 wait complete. 2024-04-23T14:36:42.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:36:42.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.268+0000 7f2b3b000640 1 Processor -- start 2024-04-23T14:36:42.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.269+0000 7f2b3b000640 1 -- start start 2024-04-23T14:36:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.270+0000 7f2b3b000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2b340cf560 0x7f2b3406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.270+0000 7f2b3b000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3406cc60 con 0x7f2b3406d440 2024-04-23T14:36:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.270+0000 7f2b3b000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b34162bd0 con 0x7f2b3415e890 2024-04-23T14:36:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.270+0000 7f2b3b000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2b3406cfb0 con 0x7f2b340cf560 2024-04-23T14:36:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.270+0000 7f2b33400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2b3406d440 0x7f2b340cee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44002/0 (socket says 172.21.15.59:44002) 2024-04-23T14:36:42.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.270+0000 7f2b33400640 1 -- 172.21.15.59:0/2543425092 learned_addr learned my addr 172.21.15.59:0/2543425092 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:42.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b39000640 1 --2- 172.21.15.59:0/2543425092 >> v2:172.21.15.76:3300/0 conn(0x7f2b340cf560 0x7f2b3406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1333453554 0 0) 0x7f2b3406cc60 con 0x7f2b3406d440 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b14003520 con 0x7f2b3406d440 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2202985761 0 0) 0x7f2b14003520 con 0x7f2b3406d440 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 >> v2:172.21.15.76:3300/0 conn(0x7f2b340cf560 msgr2=0x7f2b3406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 --2- 172.21.15.59:0/2543425092 >> v2:172.21.15.76:3300/0 conn(0x7f2b340cf560 0x7f2b3406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 >> v1:172.21.15.76:6789/0 conn(0x7f2b3415e890 legacy=0x7f2b341624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b34162ea0 con 0x7f2b3406d440 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2b1c0019f0 con 0x7f2b3406d440 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 >> v1:172.21.15.59:6789/0 conn(0x7f2b3406d440 legacy=0x7f2b340cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b140039b0 0x7f2b14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b14004800 0x7f2b14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b140051c0 0x7f2b1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3406cfb0 con 0x7f2b14004800 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1400c160 con 0x7f2b140039b0 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.271+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1400c360 con 0x7f2b140051c0 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b33400640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b14004800 0x7f2b14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b33e00640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b140039b0 0x7f2b14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b39000640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b140051c0 0x7f2b1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b33400640 1 -- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b140051c0 msgr2=0x7f2b1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b33400640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b140051c0 0x7f2b1400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b33400640 1 -- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b140039b0 msgr2=0x7f2b14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b33400640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b140039b0 0x7f2b14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b33400640 1 -- 172.21.15.59:0/2543425092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b14003520 con 0x7f2b14004800 2024-04-23T14:36:42.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.272+0000 7f2b39000640 1 -- 172.21.15.59:0/2543425092 reap_dead start 2024-04-23T14:36:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.273+0000 7f2b33400640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b14004800 0x7f2b14004c80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00a840 tx=0x7f2b14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9f12bc4c935601 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.273+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00a460 con 0x7f2b14004800 2024-04-23T14:36:42.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c02f070 con 0x7f2b14004800 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b32a00640 1 -- 172.21.15.59:0/2543425092 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c02bc50 con 0x7f2b14004800 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b3b000640 1 -- 172.21.15.59:0/2543425092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b14004800 msgr2=0x7f2b14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b3b000640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b14004800 0x7f2b14004c80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00a840 tx=0x7f2b14005830 comp rx=0 tx=0).stop 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b3b000640 1 -- 172.21.15.59:0/2543425092 shutdown_connections 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b3b000640 1 --2- 172.21.15.59:0/2543425092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b14004800 0x7f2b14004c80 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b3b000640 1 -- 172.21.15.59:0/2543425092 >> 172.21.15.59:0/2543425092 conn(0x7f2b340d25d0 msgr2=0x7f2b340d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b3b000640 1 -- 172.21.15.59:0/2543425092 shutdown_connections 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.274+0000 7f2b3b000640 1 -- 172.21.15.59:0/2543425092 wait complete. 2024-04-23T14:36:42.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.276+0000 7f2b3b000640 1 Processor -- start 2024-04-23T14:36:42.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.276+0000 7f2b3b000640 1 -- start start 2024-04-23T14:36:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.277+0000 7f2b3b000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b14004800 0x7f2b340cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.277+0000 7f2b3b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b340b6f90 0x7f2b340ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.277+0000 7f2b3b000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3409aa00 0x7f2b3409aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.277+0000 7f2b3b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2b340b3c20 con 0x7f2b340b6f90 2024-04-23T14:36:42.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.277+0000 7f2b3b000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3415ec70 con 0x7f2b3409aa00 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.277+0000 7f2b3b000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2b340b3da0 con 0x7f2b14004800 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b340b6f90 0x7f2b340ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b340b6f90 0x7f2b340ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43028/0 (socket says 172.21.15.59:43028) 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 -- 172.21.15.59:0/3834358255 learned_addr learned my addr 172.21.15.59:0/3834358255 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33e00640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b14004800 0x7f2b340cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b39000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3409aa00 0x7f2b3409aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 -- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b14004800 msgr2=0x7f2b340cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b14004800 0x7f2b340cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 -- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3409aa00 msgr2=0x7f2b3409aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3409aa00 0x7f2b3409aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b33400640 1 -- 172.21.15.59:0/3834358255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b340cb950 con 0x7f2b340b6f90 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.278+0000 7f2b39000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3409aa00 0x7f2b3409aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:42.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.279+0000 7f2b33400640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b340b6f90 0x7f2b340ce660 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00a960 tx=0x7f2b34163070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.279+0000 7f2b2be00640 1 -- 172.21.15.59:0/3834358255 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c044070 con 0x7f2b340b6f90 2024-04-23T14:36:42.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.279+0000 7f2b2be00640 1 -- 172.21.15.59:0/3834358255 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c02bc00 con 0x7f2b340b6f90 2024-04-23T14:36:42.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.279+0000 7f2b2be00640 1 -- 172.21.15.59:0/3834358255 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c034470 con 0x7f2b340b6f90 2024-04-23T14:36:42.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.279+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b340b7440 con 0x7f2b340b6f90 2024-04-23T14:36:42.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.279+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b340b7a00 con 0x7f2b340b6f90 2024-04-23T14:36:42.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.280+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b34069190 con 0x7f2b340b6f90 2024-04-23T14:36:42.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.281+0000 7f2b2be00640 1 -- 172.21.15.59:0/3834358255 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b1c047070 con 0x7f2b340b6f90 2024-04-23T14:36:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.282+0000 7f2b2be00640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1006c920 0x7f2b1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.282+0000 7f2b2be00640 1 -- 172.21.15.59:0/3834358255 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f2b1c0aba80 con 0x7f2b340b6f90 2024-04-23T14:36:42.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.282+0000 7f2b33e00640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1006c920 0x7f2b1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:42.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.283+0000 7f2b33e00640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1006c920 0x7f2b1006ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2b340cf310 tx=0x7f2b24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:42.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.287+0000 7f2b2be00640 1 -- 172.21.15.59:0/3834358255 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b1c0798f0 con 0x7f2b340b6f90 2024-04-23T14:36:42.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.530+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b3415c690 con 0x7f2b340b6f90 2024-04-23T14:36:42.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.531+0000 7f2b2be00640 1 -- 172.21.15.59:0/3834358255 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f2b1c07dbf0 con 0x7f2b340b6f90 2024-04-23T14:36:42.531 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1006c920 msgr2=0x7f2b1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1006c920 0x7f2b1006ede0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2b340cf310 tx=0x7f2b24009040 comp rx=0 tx=0).stop 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b340b6f90 msgr2=0x7f2b340ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b340b6f90 0x7f2b340ce660 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00a960 tx=0x7f2b34163070 comp rx=0 tx=0).stop 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 shutdown_connections 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2b1006c920 0x7f2b1006ede0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3409aa00 0x7f2b3409aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b340b6f90 0x7f2b340ce660 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.533+0000 7f2b3b000640 1 --2- 172.21.15.59:0/3834358255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b14004800 0x7f2b340cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:42.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.534+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 >> 172.21.15.59:0/3834358255 conn(0x7f2b340d25d0 msgr2=0x7f2b340713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:42.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.534+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 shutdown_connections 2024-04-23T14:36:42.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42.534+0000 7f2b3b000640 1 -- 172.21.15.59:0/3834358255 wait complete. 2024-04-23T14:36:42.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:36:42.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:36:42.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:36:42.553 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:36:42.553 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:42.554 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:36:42.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:36:42.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:36:42.554 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:36:42.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:42.554 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:42.605 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42146sec preferred_lft 42146sec 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:36:42.606 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:36:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:42.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:42.608 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:36:42.686 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:36:42.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:42.690 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:36:42.744 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:36:42.744 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:36:42.744 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:36:42.744 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:36:42.744 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:36:42.745 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:36:42.746 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:36:42.746 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:36:42.746 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:36:42.746 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:36:42.746 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:36:42.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:36:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:36:42.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:42.970 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:36:43.020 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:36:43.020 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:36:43.021 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:36:43.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:43.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:36:43.106 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:36:43.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:36:43.106 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:43.159 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:36:43.159 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:36:43.159 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:43.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:36:43.301 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:36:43.359 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:36:43.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:36:43.359 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:36:43.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:36:43.359 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:36:43.359 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:36:43.360 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:36:43.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:36:43.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:36:43.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:36:43.360 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:36:43.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:36:43.360 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:36:43.361 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:36:43.361 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:36:43.361 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:36:43.361 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:36:43.361 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:36:43.361 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:36:43.362 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:36:43.362 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:36:43.362 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:36:43.362 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:36:43.362 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:36:43.362 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:36:43.363 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:36:43.363 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:36:43.363 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:36:43.363 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:36:43.363 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:36:43.363 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:43.363 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:43.364 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:43.364 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:43.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:43.365 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:36:43.420 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:36:43.421 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:36:43.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:36:43.497 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:36:43.542 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:36:43.543 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:43.544 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:43.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:43.545 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:36:43.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:36:43.597+0000 7faf25a70480 -1 init, newargv = 0x558ca16c37d0 newargc=15 2024-04-23T14:36:43.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[57738]: starting ceph client 2024-04-23T14:36:43.605 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[57738]: starting fuse 2024-04-23T14:36:43.629 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:36:43.629 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:36:43.630 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:36:43.631 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:36:43.827 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.57738.asok 2024-04-23T14:36:43.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:36:43.844 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57738.asok 2024-04-23T14:36:43.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:43.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57738.asok status 2024-04-23T14:36:44.078 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "57738", 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6924, 2024-04-23T14:36:44.079 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 6924 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1128728677 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1128728677 2024-04-23T14:36:44.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.6924 192.168.0.1:0/1128728677", 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/1128728677", 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 43, 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 69, 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:36:44.081 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:36:44.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:44.094 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:36:44.094 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:44.148 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:36:44.149 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:44.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:44.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:44.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:44.240 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-23T14:36:44.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:44.311 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-23T14:36:44.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:44.371 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-23T14:36:44.427 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:36:44.427 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:36:44.427 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:36:44.427 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:44.427 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-23T14:36:44.427 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:36:44.562 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-23T14:36:44.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:36:44.576 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:36:44.577 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:44.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:44.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.893+0000 7fad96800640 1 Processor -- start 2024-04-23T14:36:44.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.894+0000 7fad96800640 1 -- start start 2024-04-23T14:36:44.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.895+0000 7fad96800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad9015e710 0x7fad900d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.895+0000 7fad96800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad900b4900 0x7fad90162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.895+0000 7fad96800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad900b4d90 con 0x7fad9015eaf0 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.895+0000 7fad96800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fad900b5070 con 0x7fad900b4900 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.895+0000 7fad96800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fad900b5350 con 0x7fad9015e710 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad94a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad900b4900 0x7fad90162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad94a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad900b4900 0x7fad90162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43032/0 (socket says 172.21.15.59:43032) 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad94a00640 1 -- 172.21.15.59:0/2928172495 learned_addr learned my addr 172.21.15.59:0/2928172495 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:44.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad95400640 1 --2- 172.21.15.59:0/2928172495 >> v2:172.21.15.76:3300/0 conn(0x7fad9015e710 0x7fad900d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:44.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad94a00640 1 -- 172.21.15.59:0/2928172495 >> v2:172.21.15.76:3300/0 conn(0x7fad9015e710 msgr2=0x7fad900d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:44.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad94a00640 1 --2- 172.21.15.59:0/2928172495 >> v2:172.21.15.76:3300/0 conn(0x7fad9015e710 0x7fad900d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:44.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad94a00640 1 -- 172.21.15.59:0/2928172495 >> v1:172.21.15.76:6789/0 conn(0x7fad9015eaf0 legacy=0x7fad900d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:44.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.896+0000 7fad94a00640 1 -- 172.21.15.59:0/2928172495 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad900b5630 con 0x7fad900b4900 2024-04-23T14:36:44.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.897+0000 7fad94a00640 1 --2- 172.21.15.59:0/2928172495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b4900 0x7fad90162490 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fad9008c8d0 tx=0x7fad7c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04332e4967b09aa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.898+0000 7fad8fe00640 1 -- 172.21.15.59:0/2928172495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c004030 con 0x7fad900b4900 2024-04-23T14:36:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.898+0000 7fad8fe00640 1 -- 172.21.15.59:0/2928172495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7c0041d0 con 0x7fad900b4900 2024-04-23T14:36:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.898+0000 7fad8fe00640 1 -- 172.21.15.59:0/2928172495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c003610 con 0x7fad900b4900 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.898+0000 7fad96800640 1 -- 172.21.15.59:0/2928172495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b4900 msgr2=0x7fad90162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.898+0000 7fad96800640 1 --2- 172.21.15.59:0/2928172495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b4900 0x7fad90162490 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fad9008c8d0 tx=0x7fad7c02a210 comp rx=0 tx=0).stop 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.899+0000 7fad96800640 1 -- 172.21.15.59:0/2928172495 shutdown_connections 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.899+0000 7fad96800640 1 --2- 172.21.15.59:0/2928172495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b4900 0x7fad90162490 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.899+0000 7fad96800640 1 --2- 172.21.15.59:0/2928172495 >> v2:172.21.15.76:3300/0 conn(0x7fad9015e710 0x7fad900d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.899+0000 7fad96800640 1 -- 172.21.15.59:0/2928172495 >> 172.21.15.59:0/2928172495 conn(0x7fad9008bf90 msgr2=0x7fad900d45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.899+0000 7fad96800640 1 -- 172.21.15.59:0/2928172495 shutdown_connections 2024-04-23T14:36:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.899+0000 7fad96800640 1 -- 172.21.15.59:0/2928172495 wait complete. 2024-04-23T14:36:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.901+0000 7fad96800640 1 Processor -- start 2024-04-23T14:36:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.901+0000 7fad96800640 1 -- start start 2024-04-23T14:36:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.902+0000 7fad96800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad9015e710 0x7fad900ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad96800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad9015eaf0 0x7fad900cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad96800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b1940 0x7fad900b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad96800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fad900b5e10 con 0x7fad900b1940 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad96800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fad900d5410 con 0x7fad9015eaf0 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad96800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fad900d03b0 con 0x7fad9015e710 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b1940 0x7fad900b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b1940 0x7fad900b99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43044/0 (socket says 172.21.15.59:43044) 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 -- 172.21.15.59:0/414527671 learned_addr learned my addr 172.21.15.59:0/414527671 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95400640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad9015e710 0x7fad900ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad94a00640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad9015eaf0 0x7fad900cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 -- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad9015e710 msgr2=0x7fad900ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad9015e710 0x7fad900ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 -- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad9015eaf0 msgr2=0x7fad900cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad9015eaf0 0x7fad900cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:44.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.903+0000 7fad95e00640 1 -- 172.21.15.59:0/414527671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad900b1dd0 con 0x7fad900b1940 2024-04-23T14:36:44.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.904+0000 7fad95e00640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b1940 0x7fad900b99e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fad8800ba10 tx=0x7fad8800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:44.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.904+0000 7fad8d600640 1 -- 172.21.15.59:0/414527671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88019070 con 0x7fad900b1940 2024-04-23T14:36:44.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.904+0000 7fad8d600640 1 -- 172.21.15.59:0/414527671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8800cae0 con 0x7fad900b1940 2024-04-23T14:36:44.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.904+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad900b8c20 con 0x7fad900b1940 2024-04-23T14:36:44.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.904+0000 7fad8d600640 1 -- 172.21.15.59:0/414527671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88014440 con 0x7fad900b1940 2024-04-23T14:36:44.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.904+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad900b91e0 con 0x7fad900b1940 2024-04-23T14:36:44.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.906+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9004fd90 con 0x7fad900b1940 2024-04-23T14:36:44.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.907+0000 7fad8d600640 1 -- 172.21.15.59:0/414527671 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad8801c070 con 0x7fad900b1940 2024-04-23T14:36:44.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.908+0000 7fad8d600640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fad7006c920 0x7fad7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:44.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.908+0000 7fad8d600640 1 -- 172.21.15.59:0/414527671 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fad8808b810 con 0x7fad900b1940 2024-04-23T14:36:44.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.909+0000 7fad95400640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fad7006c920 0x7fad7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:44.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.909+0000 7fad95400640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fad7006c920 0x7fad7006ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fad84006fd0 tx=0x7fad84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:44.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:44.913+0000 7fad8d600640 1 -- 172.21.15.59:0/414527671 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad88059ba0 con 0x7fad900b1940 2024-04-23T14:36:45.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.122+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad900cade0 con 0x7fad900b1940 2024-04-23T14:36:45.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.123+0000 7fad8d600640 1 -- 172.21.15.59:0/414527671 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16608 (secure 0 0 0) 0x7fad880594a0 con 0x7fad900b1940 2024-04-23T14:36:45.124 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:45.124 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":69,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:31.243328+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fad7006c920 msgr2=0x7fad7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fad7006c920 0x7fad7006ede0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fad84006fd0 tx=0x7fad84008040 comp rx=0 tx=0).stop 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b1940 msgr2=0x7fad900b99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b1940 0x7fad900b99e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fad8800ba10 tx=0x7fad8800bed0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 shutdown_connections 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fad7006c920 0x7fad7006ede0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad900b1940 0x7fad900b99e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad9015eaf0 0x7fad900cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 --2- 172.21.15.59:0/414527671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad9015e710 0x7fad900ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 >> 172.21.15.59:0/414527671 conn(0x7fad9008bf90 msgr2=0x7fad9006dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:45.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.126+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 shutdown_connections 2024-04-23T14:36:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.127+0000 7fad96800640 1 -- 172.21.15.59:0/414527671 wait complete. 2024-04-23T14:36:45.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:36:27.526201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:36:28.678324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:36:45.147 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:36:45.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.355+0000 7f9a12000640 1 Processor -- start 2024-04-23T14:36:45.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.355+0000 7f9a12000640 1 -- start start 2024-04-23T14:36:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.356+0000 7f9a12000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a0c15e920 0x7f9a0c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.356+0000 7f9a12000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9a0c06c0e0 0x7f9a0c168660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a12000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a0c0ce0d0 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a12000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0c0b0bb0 con 0x7f9a0c15e920 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a12000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0c0cec30 con 0x7f9a0c06c0e0 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a10c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a0c15e920 0x7f9a0c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a10c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a0c15e920 0x7f9a0c0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43056/0 (socket says 172.21.15.59:43056) 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a10c00640 1 -- 172.21.15.59:0/3401657449 learned_addr learned my addr 172.21.15.59:0/3401657449 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a11600640 1 --2- 172.21.15.59:0/3401657449 >> v2:172.21.15.76:3300/0 conn(0x7f9a0c06c0e0 0x7f9a0c168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a10c00640 1 -- 172.21.15.59:0/3401657449 >> v2:172.21.15.76:3300/0 conn(0x7f9a0c06c0e0 msgr2=0x7f9a0c168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a10c00640 1 --2- 172.21.15.59:0/3401657449 >> v2:172.21.15.76:3300/0 conn(0x7f9a0c06c0e0 0x7f9a0c168660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a10c00640 1 -- 172.21.15.59:0/3401657449 >> v1:172.21.15.59:6789/0 conn(0x7f9a0c0cdc40 legacy=0x7f9a0c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.357+0000 7f9a10c00640 1 -- 172.21.15.59:0/3401657449 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c0ce390 con 0x7f9a0c15e920 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.359+0000 7f9a10c00640 1 --2- 172.21.15.59:0/3401657449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c15e920 0x7f9a0c0cd5f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b74b0 tx=0x7f99fc02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e2bf07314d7cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.359+0000 7f9a0b400640 1 -- 172.21.15.59:0/3401657449 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc037070 con 0x7f9a0c15e920 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.359+0000 7f9a0b400640 1 -- 172.21.15.59:0/3401657449 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc032440 con 0x7f9a0c15e920 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.359+0000 7f9a0b400640 1 -- 172.21.15.59:0/3401657449 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc032810 con 0x7f9a0c15e920 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.359+0000 7f9a12000640 1 -- 172.21.15.59:0/3401657449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c15e920 msgr2=0x7f9a0c0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.359+0000 7f9a12000640 1 --2- 172.21.15.59:0/3401657449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c15e920 0x7f9a0c0cd5f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b74b0 tx=0x7f99fc02a160 comp rx=0 tx=0).stop 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.360+0000 7f9a12000640 1 -- 172.21.15.59:0/3401657449 shutdown_connections 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.360+0000 7f9a12000640 1 --2- 172.21.15.59:0/3401657449 >> v2:172.21.15.76:3300/0 conn(0x7f9a0c06c0e0 0x7f9a0c168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.360+0000 7f9a12000640 1 --2- 172.21.15.59:0/3401657449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c15e920 0x7f9a0c0cd5f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.360+0000 7f9a12000640 1 -- 172.21.15.59:0/3401657449 >> 172.21.15.59:0/3401657449 conn(0x7f9a0c0b2f10 msgr2=0x7f9a0c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:45.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.360+0000 7f9a12000640 1 -- 172.21.15.59:0/3401657449 shutdown_connections 2024-04-23T14:36:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.360+0000 7f9a12000640 1 -- 172.21.15.59:0/3401657449 wait complete. 2024-04-23T14:36:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.361+0000 7f9a12000640 1 Processor -- start 2024-04-23T14:36:45.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.362+0000 7f9a12000640 1 -- start start 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a12000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a0c06c0e0 0x7f9a0c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a12000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c0cdc40 0x7f9a0c1f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a12000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a0c15e920 0x7f9a0c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a12000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c0ccff0 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a12000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c0ce950 con 0x7f9a0c06c0e0 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a12000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c0cd170 con 0x7f9a0c15e920 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c0cdc40 0x7f9a0c1f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c0cdc40 0x7f9a0c1f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43060/0 (socket says 172.21.15.59:43060) 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a0be00640 1 -- 172.21.15.59:0/3134343779 learned_addr learned my addr 172.21.15.59:0/3134343779 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a10c00640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a0c06c0e0 0x7f9a0c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a11600640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a0c15e920 0x7f9a0c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a0be00640 1 -- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a0c15e920 msgr2=0x7f9a0c1f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a0be00640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a0c15e920 0x7f9a0c1f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.363+0000 7f9a0be00640 1 -- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a0c06c0e0 msgr2=0x7f9a0c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a0be00640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a0c06c0e0 0x7f9a0c0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a0be00640 1 -- 172.21.15.59:0/3134343779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c0c8a60 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a0be00640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c0cdc40 0x7f9a0c1f7930 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c460 tx=0x7f99f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a08c00640 1 -- 172.21.15.59:0/3134343779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40052d0 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a08c00640 1 -- 172.21.15.59:0/3134343779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f4012630 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a08c00640 1 -- 172.21.15.59:0/3134343779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40054b0 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c0c8d40 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.364+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c0c9300 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.365+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c04fd90 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.366+0000 7f9a08c00640 1 -- 172.21.15.59:0/3134343779 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99f4012a20 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.367+0000 7f9a08c00640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99e806c8f0 0x7f99e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.368+0000 7f9a08c00640 1 -- 172.21.15.59:0/3134343779 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f99f408b800 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.368+0000 7f9a10c00640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99e806c8f0 0x7f99e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.368+0000 7f9a10c00640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99e806c8f0 0x7f99e806edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f99fc034000 tx=0x7f99fc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:45.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.372+0000 7f9a08c00640 1 -- 172.21.15.59:0/3134343779 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99f4059b90 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.580+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a0c06f260 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.581+0000 7f9a08c00640 1 -- 172.21.15.59:0/3134343779 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f99f4059490 con 0x7f9a0c0cdc40 2024-04-23T14:36:45.581 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:36:45.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99e806c8f0 msgr2=0x7f99e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99e806c8f0 0x7f99e806edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f99fc034000 tx=0x7f99fc02e000 comp rx=0 tx=0).stop 2024-04-23T14:36:45.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c0cdc40 msgr2=0x7f9a0c1f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c0cdc40 0x7f9a0c1f7930 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c460 tx=0x7f99f400c920 comp rx=0 tx=0).stop 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 shutdown_connections 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f99e806c8f0 0x7f99e806edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a0c15e920 0x7f9a0c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a0c0cdc40 0x7f9a0c1f7930 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 --2- 172.21.15.59:0/3134343779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a0c06c0e0 0x7f9a0c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 >> 172.21.15.59:0/3134343779 conn(0x7f9a0c0b2f10 msgr2=0x7f9a0c0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.584+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 shutdown_connections 2024-04-23T14:36:45.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.585+0000 7f9a12000640 1 -- 172.21.15.59:0/3134343779 wait complete. 2024-04-23T14:36:45.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:36:45.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.813+0000 7f9ece200640 1 Processor -- start 2024-04-23T14:36:45.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.813+0000 7f9ece200640 1 -- start start 2024-04-23T14:36:45.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.814+0000 7f9ece200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ec80d0a90 0x7f9ec80ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.814+0000 7f9ece200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec815ef20 con 0x7f9ec8155a80 2024-04-23T14:36:45.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.814+0000 7f9ece200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec80d0f00 con 0x7f9ec80d0600 2024-04-23T14:36:45.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.814+0000 7f9ece200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec815f1c0 con 0x7f9ec80d0a90 2024-04-23T14:36:45.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.815+0000 7f9ecce00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9ec8155a80 0x7f9ec806f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44026/0 (socket says 172.21.15.59:44026) 2024-04-23T14:36:45.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.815+0000 7f9ecce00640 1 -- 172.21.15.59:0/1577679684 learned_addr learned my addr 172.21.15.59:0/1577679684 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:45.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.815+0000 7f9ecd800640 1 --2- 172.21.15.59:0/1577679684 >> v2:172.21.15.76:3300/0 conn(0x7f9ec80d0a90 0x7f9ec80ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.815+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 57901776 0 0) 0x7f9ec815ef20 con 0x7f9ec8155a80 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.815+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eb0005730 con 0x7f9ec8155a80 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3049357324 0 0) 0x7f9eb0005730 con 0x7f9ec8155a80 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 >> v2:172.21.15.76:3300/0 conn(0x7f9ec80d0a90 msgr2=0x7f9ec80ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 --2- 172.21.15.59:0/1577679684 >> v2:172.21.15.76:3300/0 conn(0x7f9ec80d0a90 0x7f9ec80ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 >> v1:172.21.15.76:6789/0 conn(0x7f9ec80d0600 legacy=0x7f9ec80ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec815f4a0 con 0x7f9ec8155a80 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9ebc001670 con 0x7f9ec8155a80 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 >> v1:172.21.15.59:6789/0 conn(0x7f9ec8155a80 legacy=0x7f9ec806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9eb0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eb0006a10 0x7f9eb0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eb00073d0 0x7f9eb000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec815f1c0 con 0x7f9eb0005bc0 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb000e270 con 0x7f9eb00073d0 2024-04-23T14:36:45.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb000e470 con 0x7f9eb0006a10 2024-04-23T14:36:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ecce00640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9eb0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ecce00640 1 -- 172.21.15.59:0/1577679684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eb0006a10 msgr2=0x7f9eb0006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ecce00640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eb0006a10 0x7f9eb0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ecce00640 1 -- 172.21.15.59:0/1577679684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eb00073d0 msgr2=0x7f9eb000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.816+0000 7f9ecce00640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eb00073d0 0x7f9eb000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.817+0000 7f9ecce00640 1 -- 172.21.15.59:0/1577679684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb0005730 con 0x7f9eb0005bc0 2024-04-23T14:36:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.817+0000 7f9ecd800640 1 -- 172.21.15.59:0/1577679684 reap_dead start 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.818+0000 7f9ecce00640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9eb0005fe0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c210 tx=0x7f9eb0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b133c668b8443a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.818+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc007d30 con 0x7f9eb0005bc0 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.818+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc005ad0 con 0x7f9eb0005bc0 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.818+0000 7f9ec7400640 1 -- 172.21.15.59:0/1577679684 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc007070 con 0x7f9eb0005bc0 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.819+0000 7f9ece200640 1 -- 172.21.15.59:0/1577679684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 msgr2=0x7f9eb0005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.819+0000 7f9ece200640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9eb0005fe0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c210 tx=0x7f9eb0007a40 comp rx=0 tx=0).stop 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.819+0000 7f9ece200640 1 -- 172.21.15.59:0/1577679684 shutdown_connections 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.819+0000 7f9ece200640 1 --2- 172.21.15.59:0/1577679684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9eb0005fe0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.819+0000 7f9ece200640 1 -- 172.21.15.59:0/1577679684 >> 172.21.15.59:0/1577679684 conn(0x7f9ec806bdf0 msgr2=0x7f9ec806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.819+0000 7f9ece200640 1 -- 172.21.15.59:0/1577679684 shutdown_connections 2024-04-23T14:36:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.819+0000 7f9ece200640 1 -- 172.21.15.59:0/1577679684 wait complete. 2024-04-23T14:36:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.820+0000 7f9ece200640 1 Processor -- start 2024-04-23T14:36:45.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.821+0000 7f9ece200640 1 -- start start 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.821+0000 7f9ece200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9ec815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ece200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ec814a9c0 0x7f9ec80b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ece200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec80afde0 0x7f9ec80b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ece200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec815fd40 con 0x7f9eb0005bc0 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ece200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec815fa60 con 0x7f9ec80afde0 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ece200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec806ef80 con 0x7f9ec814a9c0 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9ec815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9ec815e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43078/0 (socket says 172.21.15.59:43078) 2024-04-23T14:36:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 -- 172.21.15.59:0/1663173586 learned_addr learned my addr 172.21.15.59:0/1663173586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ec7e00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ec814a9c0 0x7f9ec80b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecd800640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec80afde0 0x7f9ec80b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 -- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ec814a9c0 msgr2=0x7f9ec80b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ec814a9c0 0x7f9ec80b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 -- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec80afde0 msgr2=0x7f9ec80b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec80afde0 0x7f9ec80b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecce00640 1 -- 172.21.15.59:0/1663173586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec80b2820 con 0x7f9eb0005bc0 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ec7e00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ec814a9c0 0x7f9ec80b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.822+0000 7f9ecd800640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec80afde0 0x7f9ec80b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.823+0000 7f9ecce00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9ec815e1a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c210 tx=0x7f9ebc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.823+0000 7f9ec4c00640 1 -- 172.21.15.59:0/1663173586 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc041070 con 0x7f9eb0005bc0 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.823+0000 7f9ec4c00640 1 -- 172.21.15.59:0/1663173586 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc000ed0 con 0x7f9eb0005bc0 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.823+0000 7f9ec4c00640 1 -- 172.21.15.59:0/1663173586 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc007290 con 0x7f9eb0005bc0 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.823+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec80b2b00 con 0x7f9eb0005bc0 2024-04-23T14:36:45.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.823+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec814e9f0 con 0x7f9eb0005bc0 2024-04-23T14:36:45.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.824+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec804fd90 con 0x7f9eb0005bc0 2024-04-23T14:36:45.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.825+0000 7f9ec4c00640 1 -- 172.21.15.59:0/1663173586 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ebc044070 con 0x7f9eb0005bc0 2024-04-23T14:36:45.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.826+0000 7f9ec4c00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ea806c8f0 0x7f9ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:45.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.826+0000 7f9ec4c00640 1 -- 172.21.15.59:0/1663173586 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f9ebc0abba0 con 0x7f9eb0005bc0 2024-04-23T14:36:45.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.827+0000 7f9ec7e00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ea806c8f0 0x7f9ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:45.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.827+0000 7f9ec7e00640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ea806c8f0 0x7f9ea806edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9ec81f3620 tx=0x7f9eb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:45.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:45.831+0000 7f9ec4c00640 1 -- 172.21.15.59:0/1663173586 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ebc04b020 con 0x7f9eb0005bc0 2024-04-23T14:36:46.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.033+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ec806eba0 con 0x7f9eb0005bc0 2024-04-23T14:36:46.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.034+0000 7f9ec4c00640 1 -- 172.21.15.59:0/1663173586 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7f9ebc044e40 con 0x7f9eb0005bc0 2024-04-23T14:36:46.035 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ea806c8f0 msgr2=0x7f9ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ea806c8f0 0x7f9ea806edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9ec81f3620 tx=0x7f9eb4009040 comp rx=0 tx=0).stop 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 msgr2=0x7f9ec815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9ec815e1a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c210 tx=0x7f9ebc001a40 comp rx=0 tx=0).stop 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 shutdown_connections 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9ea806c8f0 0x7f9ea806edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ec814a9c0 0x7f9ec80b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ec80afde0 0x7f9ec80b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 --2- 172.21.15.59:0/1663173586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0005bc0 0x7f9ec815e1a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.037+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 >> 172.21.15.59:0/1663173586 conn(0x7f9ec806bdf0 msgr2=0x7f9ec806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:46.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.038+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 shutdown_connections 2024-04-23T14:36:46.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.038+0000 7f9ece200640 1 -- 172.21.15.59:0/1663173586 wait complete. 2024-04-23T14:36:46.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:36:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.275+0000 7ff23a200640 1 Processor -- start 2024-04-23T14:36:46.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.275+0000 7ff23a200640 1 -- start start 2024-04-23T14:36:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff23a200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2340d3420 0x7ff2340d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff23a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff2340b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff23a200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2340d2380 con 0x7ff23415e920 2024-04-23T14:36:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff23a200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff2340d1890 con 0x7ff2340d3420 2024-04-23T14:36:46.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff23a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2340d2660 con 0x7ff2340d1440 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff233e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2340d3420 0x7ff2340d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff233e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2340d3420 0x7ff2340d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43092/0 (socket says 172.21.15.59:43092) 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.277+0000 7ff233e00640 1 -- 172.21.15.59:0/487110519 learned_addr learned my addr 172.21.15.59:0/487110519 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.278+0000 7ff233e00640 1 -- 172.21.15.59:0/487110519 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 msgr2=0x7ff2340b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.278+0000 7ff239800640 1 --2- 172.21.15.59:0/487110519 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff2340b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.278+0000 7ff233e00640 1 --2- 172.21.15.59:0/487110519 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff2340b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.278+0000 7ff233e00640 1 -- 172.21.15.59:0/487110519 >> v1:172.21.15.76:6789/0 conn(0x7ff23415e920 legacy=0x7ff2340cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.278+0000 7ff233e00640 1 -- 172.21.15.59:0/487110519 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2340d2940 con 0x7ff2340d3420 2024-04-23T14:36:46.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.278+0000 7ff239800640 1 --2- 172.21.15.59:0/487110519 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff2340b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:46.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.279+0000 7ff233e00640 1 --2- 172.21.15.59:0/487110519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2340d3420 0x7ff2340d0f00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff220009870 tx=0x7ff22002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2902f2abdd08d716 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff233400640 1 -- 172.21.15.59:0/487110519 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220037070 con 0x7ff2340d3420 2024-04-23T14:36:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff233400640 1 -- 172.21.15.59:0/487110519 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22002ae10 con 0x7ff2340d3420 2024-04-23T14:36:46.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff233400640 1 -- 172.21.15.59:0/487110519 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220032660 con 0x7ff2340d3420 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff23a200640 1 -- 172.21.15.59:0/487110519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2340d3420 msgr2=0x7ff2340d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff23a200640 1 --2- 172.21.15.59:0/487110519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2340d3420 0x7ff2340d0f00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff220009870 tx=0x7ff22002a100 comp rx=0 tx=0).stop 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff23a200640 1 -- 172.21.15.59:0/487110519 shutdown_connections 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff23a200640 1 --2- 172.21.15.59:0/487110519 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff2340b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.280+0000 7ff23a200640 1 --2- 172.21.15.59:0/487110519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2340d3420 0x7ff2340d0f00 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.281+0000 7ff23a200640 1 -- 172.21.15.59:0/487110519 >> 172.21.15.59:0/487110519 conn(0x7ff23408bf90 msgr2=0x7ff2340d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.281+0000 7ff23a200640 1 -- 172.21.15.59:0/487110519 shutdown_connections 2024-04-23T14:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.281+0000 7ff23a200640 1 -- 172.21.15.59:0/487110519 wait complete. 2024-04-23T14:36:46.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.282+0000 7ff23a200640 1 Processor -- start 2024-04-23T14:36:46.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.283+0000 7ff23a200640 1 -- start start 2024-04-23T14:36:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff23a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff23415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff23a200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2340d3420 0x7ff2341f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff23a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23415e920 0x7ff23406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff23a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2340d0440 con 0x7ff23415e920 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff23a200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff23415ed00 con 0x7ff2340d3420 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff23a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2340d05c0 con 0x7ff2340d1440 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff239800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23415e920 0x7ff23406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff239800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23415e920 0x7ff23406dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43098/0 (socket says 172.21.15.59:43098) 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff239800640 1 -- 172.21.15.59:0/2085321536 learned_addr learned my addr 172.21.15.59:0/2085321536 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.284+0000 7ff238e00640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff23415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff239800640 1 -- 172.21.15.59:0/2085321536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 msgr2=0x7ff23415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff239800640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff23415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff239800640 1 -- 172.21.15.59:0/2085321536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2340d3420 msgr2=0x7ff2341f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff239800640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2340d3420 0x7ff2341f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff239800640 1 -- 172.21.15.59:0/2085321536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2340cbb70 con 0x7ff23415e920 2024-04-23T14:36:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff239800640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23415e920 0x7ff23406dd50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00cef0 tx=0x7ff22c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:46.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff230c00640 1 -- 172.21.15.59:0/2085321536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c01a070 con 0x7ff23415e920 2024-04-23T14:36:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff230c00640 1 -- 172.21.15.59:0/2085321536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22c00d860 con 0x7ff23415e920 2024-04-23T14:36:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff230c00640 1 -- 172.21.15.59:0/2085321536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00dc50 con 0x7ff23415e920 2024-04-23T14:36:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2340cbe50 con 0x7ff23415e920 2024-04-23T14:36:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.285+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2340cc6f0 con 0x7ff23415e920 2024-04-23T14:36:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.286+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2340cc020 con 0x7ff23415e920 2024-04-23T14:36:46.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.288+0000 7ff230c00640 1 -- 172.21.15.59:0/2085321536 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff22c01d070 con 0x7ff23415e920 2024-04-23T14:36:46.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.288+0000 7ff230c00640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21406c8f0 0x7ff21406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.289+0000 7ff230c00640 1 -- 172.21.15.59:0/2085321536 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7ff22c08c4c0 con 0x7ff23415e920 2024-04-23T14:36:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.289+0000 7ff238e00640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21406c8f0 0x7ff21406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.290+0000 7ff238e00640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21406c8f0 0x7ff21406edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff228006fd0 tx=0x7ff228008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:46.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.293+0000 7ff230c00640 1 -- 172.21.15.59:0/2085321536 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff22c05a910 con 0x7ff23415e920 2024-04-23T14:36:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.498+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff23408d380 con 0x7ff23415e920 2024-04-23T14:36:46.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.499+0000 7ff230c00640 1 -- 172.21.15.59:0/2085321536 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7ff23408d380 con 0x7ff23415e920 2024-04-23T14:36:46.499 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.501+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21406c8f0 msgr2=0x7ff21406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.501+0000 7ff23a200640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21406c8f0 0x7ff21406edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff228006fd0 tx=0x7ff228008040 comp rx=0 tx=0).stop 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23415e920 msgr2=0x7ff23406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23415e920 0x7ff23406dd50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00cef0 tx=0x7ff22c00a710 comp rx=0 tx=0).stop 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 shutdown_connections 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff21406c8f0 0x7ff21406edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff23415e920 0x7ff23406dd50 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2340d3420 0x7ff2341f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 --2- 172.21.15.59:0/2085321536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2340d1440 0x7ff23415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 >> 172.21.15.59:0/2085321536 conn(0x7ff23408bf90 msgr2=0x7ff2340d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 shutdown_connections 2024-04-23T14:36:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.502+0000 7ff23a200640 1 -- 172.21.15.59:0/2085321536 wait complete. 2024-04-23T14:36:46.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:36:46.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.735+0000 7f2337e00640 1 Processor -- start 2024-04-23T14:36:46.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.735+0000 7f2337e00640 1 -- start start 2024-04-23T14:36:46.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.736+0000 7f2337e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f23380d0cd0 0x7f23381674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.736+0000 7f2337e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23380b4cf0 con 0x7f23380cd8b0 2024-04-23T14:36:46.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.736+0000 7f2337e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23380ccdd0 con 0x7f233815c670 2024-04-23T14:36:46.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.736+0000 7f2337e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f233815ca50 con 0x7f23380d0cd0 2024-04-23T14:36:46.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.737+0000 7f232e000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f23380cd8b0 0x7f23380d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44028/0 (socket says 172.21.15.59:44028) 2024-04-23T14:36:46.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.737+0000 7f232e000640 1 -- 172.21.15.59:0/346122372 learned_addr learned my addr 172.21.15.59:0/346122372 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.737+0000 7f2337400640 1 --2- 172.21.15.59:0/346122372 >> v2:172.21.15.76:3300/0 conn(0x7f23380d0cd0 0x7f23381674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.737+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424279345 0 0) 0x7f23380b4cf0 con 0x7f23380cd8b0 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.737+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f231c003520 con 0x7f23380cd8b0 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2883992224 0 0) 0x7f231c003520 con 0x7f23380cd8b0 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 >> v2:172.21.15.76:3300/0 conn(0x7f23380d0cd0 msgr2=0x7f23381674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 --2- 172.21.15.59:0/346122372 >> v2:172.21.15.76:3300/0 conn(0x7f23380d0cd0 0x7f23381674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 >> v1:172.21.15.76:6789/0 conn(0x7f233815c670 legacy=0x7f23380b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23380cd0b0 con 0x7f23380cd8b0 2024-04-23T14:36:46.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f23200019f0 con 0x7f23380cd8b0 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 >> v1:172.21.15.59:6789/0 conn(0x7f23380cd8b0 legacy=0x7f23380d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f231c0039b0 0x7f231c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f231c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f231c0051c0 0x7f231c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f231c00c030 con 0x7f231c004800 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f231c00c1e0 con 0x7f231c0039b0 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f231c00c3e0 con 0x7f231c0051c0 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f232e000640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f231c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.738+0000 7f2336a00640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f231c0039b0 0x7f231c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f2337400640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f231c0051c0 0x7f231c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f232e000640 1 -- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f231c0051c0 msgr2=0x7f231c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f232e000640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f231c0051c0 0x7f231c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f232e000640 1 -- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f231c0039b0 msgr2=0x7f231c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f232e000640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f231c0039b0 0x7f231c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f232e000640 1 -- 172.21.15.59:0/346122372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c003520 con 0x7f231c004800 2024-04-23T14:36:46.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f2337400640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f231c0051c0 0x7f231c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:46.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.739+0000 7f2337400640 1 -- 172.21.15.59:0/346122372 reap_dead start 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.740+0000 7f232e000640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f231c004c80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f232000a810 tx=0x7f231c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228132a416433619 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.740+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232000a460 con 0x7f231c004800 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.740+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232002f070 con 0x7f231c004800 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.740+0000 7f2336000640 1 -- 172.21.15.59:0/346122372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232002bc60 con 0x7f231c004800 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.741+0000 7f2337e00640 1 -- 172.21.15.59:0/346122372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 msgr2=0x7f231c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.741+0000 7f2337e00640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f231c004c80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f232000a810 tx=0x7f231c005830 comp rx=0 tx=0).stop 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.741+0000 7f2337e00640 1 -- 172.21.15.59:0/346122372 shutdown_connections 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.741+0000 7f2337e00640 1 --2- 172.21.15.59:0/346122372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f231c004c80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.741+0000 7f2337e00640 1 -- 172.21.15.59:0/346122372 >> 172.21.15.59:0/346122372 conn(0x7f233806bdf0 msgr2=0x7f233806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.741+0000 7f2337e00640 1 -- 172.21.15.59:0/346122372 shutdown_connections 2024-04-23T14:36:46.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.741+0000 7f2337e00640 1 -- 172.21.15.59:0/346122372 wait complete. 2024-04-23T14:36:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.743+0000 7f2337e00640 1 Processor -- start 2024-04-23T14:36:46.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.743+0000 7f2337e00640 1 -- start start 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2337e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f23380af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2337e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23380ca320 0x7f23380ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2337e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2338158f80 0x7f2338159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2337e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23380b3e60 con 0x7f231c004800 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2337e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23380ccb80 con 0x7f23380ca320 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2337e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f23380b3fe0 con 0x7f2338158f80 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2336a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f23380af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2336a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f23380af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43112/0 (socket says 172.21.15.59:43112) 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.744+0000 7f2336a00640 1 -- 172.21.15.59:0/1413825557 learned_addr learned my addr 172.21.15.59:0/1413825557 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f232e000640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23380ca320 0x7f23380ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f2337400640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2338158f80 0x7f2338159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f2336a00640 1 -- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2338158f80 msgr2=0x7f2338159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f2336a00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2338158f80 0x7f2338159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f2336a00640 1 -- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23380ca320 msgr2=0x7f23380ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f2336a00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23380ca320 0x7f23380ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f2336a00640 1 -- 172.21.15.59:0/1413825557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23381f8660 con 0x7f231c004800 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f232e000640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23380ca320 0x7f23380ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.745+0000 7f2336a00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f23380af090 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f232800e480 tx=0x7f232800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.746+0000 7f232fe00640 1 -- 172.21.15.59:0/1413825557 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23280052e0 con 0x7f231c004800 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.746+0000 7f232fe00640 1 -- 172.21.15.59:0/1413825557 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2328018070 con 0x7f231c004800 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.746+0000 7f232fe00640 1 -- 172.21.15.59:0/1413825557 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2328003830 con 0x7f231c004800 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.746+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23381f4470 con 0x7f231c004800 2024-04-23T14:36:46.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.746+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23381f4a30 con 0x7f231c004800 2024-04-23T14:36:46.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.747+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233804fd90 con 0x7f231c004800 2024-04-23T14:36:46.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.748+0000 7f232fe00640 1 -- 172.21.15.59:0/1413825557 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23280039d0 con 0x7f231c004800 2024-04-23T14:36:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.749+0000 7f232fe00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f231406c8f0 0x7f231406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.749+0000 7f232fe00640 1 -- 172.21.15.59:0/1413825557 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f232808b800 con 0x7f231c004800 2024-04-23T14:36:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.749+0000 7f232e000640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f231406c8f0 0x7f231406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:46.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.750+0000 7f232e000640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f231406c8f0 0x7f231406edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2320033000 tx=0x7f2320038040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:46.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.754+0000 7f232fe00640 1 -- 172.21.15.59:0/1413825557 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2328059cc0 con 0x7f231c004800 2024-04-23T14:36:46.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.969+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23380ceab0 con 0x7f231c004800 2024-04-23T14:36:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.970+0000 7f232fe00640 1 -- 172.21.15.59:0/1413825557 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f23280595c0 con 0x7f231c004800 2024-04-23T14:36:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f231406c8f0 msgr2=0x7f231406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f231406c8f0 0x7f231406edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2320033000 tx=0x7f2320038040 comp rx=0 tx=0).stop 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 msgr2=0x7f23380af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f23380af090 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f232800e480 tx=0x7f232800e940 comp rx=0 tx=0).stop 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 shutdown_connections 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f231406c8f0 0x7f231406edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2338158f80 0x7f2338159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23380ca320 0x7f23380ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 --2- 172.21.15.59:0/1413825557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c004800 0x7f23380af090 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:46.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 >> 172.21.15.59:0/1413825557 conn(0x7f233806bdf0 msgr2=0x7f2338071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:46.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 shutdown_connections 2024-04-23T14:36:46.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:46.973+0000 7f2337e00640 1 -- 172.21.15.59:0/1413825557 wait complete. 2024-04-23T14:36:46.992 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:36:46.992 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:36:47.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:36:47.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.302+0000 7f8fdbe00640 1 Processor -- start 2024-04-23T14:36:47.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.302+0000 7f8fdbe00640 1 -- start start 2024-04-23T14:36:47.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8fdc0cd050 0x7f8fdc06f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc15ece0 0x7f8fdc0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdbe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fdc0cd970 con 0x7f8fdc0b3080 2024-04-23T14:36:47.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdbe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8fdc0ced20 con 0x7f8fdc0cd050 2024-04-23T14:36:47.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc15fcd0 con 0x7f8fdc15ece0 2024-04-23T14:36:47.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdaa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8fdc0cd050 0x7f8fdc06f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdb400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8fdc0b3080 0x7f8fdc0cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44034/0 (socket says 172.21.15.59:44034) 2024-04-23T14:36:47.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fdb400640 1 -- 172.21.15.59:0/779068522 learned_addr learned my addr 172.21.15.59:0/779068522 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:47.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.304+0000 7f8fda000640 1 --2- 172.21.15.59:0/779068522 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc15ece0 0x7f8fdc0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.305+0000 7f8fdaa00640 1 -- 172.21.15.59:0/779068522 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc15ece0 msgr2=0x7f8fdc0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:47.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.305+0000 7f8fdaa00640 1 --2- 172.21.15.59:0/779068522 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc15ece0 0x7f8fdc0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.305+0000 7f8fdaa00640 1 -- 172.21.15.59:0/779068522 >> v1:172.21.15.59:6789/0 conn(0x7f8fdc0b3080 legacy=0x7f8fdc0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.305+0000 7f8fdaa00640 1 -- 172.21.15.59:0/779068522 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc15f160 con 0x7f8fdc0cd050 2024-04-23T14:36:47.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.306+0000 7f8fdaa00640 1 --2- 172.21.15.59:0/779068522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0cd050 0x7f8fdc06f620 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0097a0 tx=0x7f8fcc02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f40fa3488e9ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:47.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.306+0000 7f8fd9600640 1 -- 172.21.15.59:0/779068522 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc037070 con 0x7f8fdc0cd050 2024-04-23T14:36:47.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.306+0000 7f8fd9600640 1 -- 172.21.15.59:0/779068522 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc02ae40 con 0x7f8fdc0cd050 2024-04-23T14:36:47.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.306+0000 7f8fd9600640 1 -- 172.21.15.59:0/779068522 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc032660 con 0x7f8fdc0cd050 2024-04-23T14:36:47.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 -- 172.21.15.59:0/779068522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0cd050 msgr2=0x7f8fdc06f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/779068522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0cd050 0x7f8fdc06f620 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0097a0 tx=0x7f8fcc02a0f0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 -- 172.21.15.59:0/779068522 shutdown_connections 2024-04-23T14:36:47.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/779068522 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc15ece0 0x7f8fdc0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/779068522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0cd050 0x7f8fdc06f620 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 -- 172.21.15.59:0/779068522 >> 172.21.15.59:0/779068522 conn(0x7f8fdc0b21a0 msgr2=0x7f8fdc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:47.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 -- 172.21.15.59:0/779068522 shutdown_connections 2024-04-23T14:36:47.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.307+0000 7f8fdbe00640 1 -- 172.21.15.59:0/779068522 wait complete. 2024-04-23T14:36:47.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.309+0000 7f8fdbe00640 1 Processor -- start 2024-04-23T14:36:47.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.310+0000 7f8fdbe00640 1 -- start start 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0b3080 0x7f8fdc1f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc0cd050 0x7f8fdc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fdc15ece0 0x7f8fdc0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdbe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc15f9f0 con 0x7f8fdc0b3080 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdbe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc15f710 con 0x7f8fdc15ece0 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdbe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc0cef50 con 0x7f8fdc0cd050 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0b3080 0x7f8fdc1f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0b3080 0x7f8fdc1f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43124/0 (socket says 172.21.15.59:43124) 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.311+0000 7f8fdaa00640 1 -- 172.21.15.59:0/143038926 learned_addr learned my addr 172.21.15.59:0/143038926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdaa00640 1 -- 172.21.15.59:0/143038926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc0cd050 msgr2=0x7f8fdc0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdaa00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc0cd050 0x7f8fdc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdaa00640 1 -- 172.21.15.59:0/143038926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fdc15ece0 msgr2=0x7f8fdc0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdaa00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fdc15ece0 0x7f8fdc0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdaa00640 1 -- 172.21.15.59:0/143038926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc0b1050 con 0x7f8fdc0b3080 2024-04-23T14:36:47.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdaa00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0b3080 0x7f8fdc1f8490 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0098c0 tx=0x7f8fcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:47.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fd3400640 1 -- 172.21.15.59:0/143038926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc041030 con 0x7f8fdc0b3080 2024-04-23T14:36:47.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fd3400640 1 -- 172.21.15.59:0/143038926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc02ac40 con 0x7f8fdc0b3080 2024-04-23T14:36:47.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fd3400640 1 -- 172.21.15.59:0/143038926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc037040 con 0x7f8fdc0b3080 2024-04-23T14:36:47.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc0b1330 con 0x7f8fdc0b3080 2024-04-23T14:36:47.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.312+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc0b18f0 con 0x7f8fdc0b3080 2024-04-23T14:36:47.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.314+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fdc0b1b30 con 0x7f8fdc0b3080 2024-04-23T14:36:47.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.315+0000 7f8fd3400640 1 -- 172.21.15.59:0/143038926 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fcc044070 con 0x7f8fdc0b3080 2024-04-23T14:36:47.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.316+0000 7f8fd3400640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fbc06c920 0x7f8fbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.316+0000 7f8fd3400640 1 -- 172.21.15.59:0/143038926 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f8fcc0ac1b0 con 0x7f8fdc0b3080 2024-04-23T14:36:47.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.316+0000 7f8fda000640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fbc06c920 0x7f8fbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.317+0000 7f8fda000640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fbc06c920 0x7f8fbc06ede0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8006e80 tx=0x7f8fc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:47.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.321+0000 7f8fd3400640 1 -- 172.21.15.59:0/143038926 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fcc07a5d0 con 0x7f8fdc0b3080 2024-04-23T14:36:47.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.528+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fdc04fd90 con 0x7f8fdc0b3080 2024-04-23T14:36:47.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.531+0000 7f8fd3400640 1 -- 172.21.15.59:0/143038926 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16608 (secure 0 0 0) 0x7f8fcc079ed0 con 0x7f8fdc0b3080 2024-04-23T14:36:47.531 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:47.531 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":69,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:31.243328+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:36:47.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.533+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fbc06c920 msgr2=0x7f8fbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.533+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fbc06c920 0x7f8fbc06ede0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8006e80 tx=0x7f8fc8008040 comp rx=0 tx=0).stop 2024-04-23T14:36:47.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.533+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0b3080 msgr2=0x7f8fdc1f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.533+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0b3080 0x7f8fdc1f8490 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0098c0 tx=0x7f8fcc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 shutdown_connections 2024-04-23T14:36:47.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fbc06c920 0x7f8fbc06ede0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fdc0cd050 0x7f8fdc0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fdc15ece0 0x7f8fdc0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 --2- 172.21.15.59:0/143038926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fdc0b3080 0x7f8fdc1f8490 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 >> 172.21.15.59:0/143038926 conn(0x7f8fdc0b21a0 msgr2=0x7f8fdc0ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:47.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 shutdown_connections 2024-04-23T14:36:47.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.534+0000 7f8fdbe00640 1 -- 172.21.15.59:0/143038926 wait complete. 2024-04-23T14:36:47.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:36:27.526201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:36:28.678324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:36:47.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:36:47.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.770+0000 7faabf200640 1 Processor -- start 2024-04-23T14:36:47.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.770+0000 7faabf200640 1 -- start start 2024-04-23T14:36:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.772+0000 7faabf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faac0161fd0 0x7faac00b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.772+0000 7faabf200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac00cdc90 0x7faac00d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.772+0000 7faabf200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac009b880 con 0x7faac009aa00 2024-04-23T14:36:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.772+0000 7faabf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faac006dce0 con 0x7faac0161fd0 2024-04-23T14:36:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.772+0000 7faabf200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faac006d690 con 0x7faac00cdc90 2024-04-23T14:36:47.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.772+0000 7faabde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faac0161fd0 0x7faac00b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.773+0000 7faabde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faac0161fd0 0x7faac00b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43138/0 (socket says 172.21.15.59:43138) 2024-04-23T14:36:47.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.773+0000 7faabde00640 1 -- 172.21.15.59:0/3661947109 learned_addr learned my addr 172.21.15.59:0/3661947109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:47.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.773+0000 7faabe800640 1 --2- 172.21.15.59:0/3661947109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac00cdc90 0x7faac00d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.773+0000 7faabde00640 1 -- 172.21.15.59:0/3661947109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac00cdc90 msgr2=0x7faac00d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.773+0000 7faabde00640 1 --2- 172.21.15.59:0/3661947109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac00cdc90 0x7faac00d0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.773+0000 7faabde00640 1 -- 172.21.15.59:0/3661947109 >> v1:172.21.15.76:6789/0 conn(0x7faac009aa00 legacy=0x7faac009ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.773+0000 7faabde00640 1 -- 172.21.15.59:0/3661947109 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac006d970 con 0x7faac0161fd0 2024-04-23T14:36:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.775+0000 7faabde00640 1 --2- 172.21.15.59:0/3661947109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac0161fd0 0x7faac00b3340 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7faab0009870 tx=0x7faab002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeee7e711ba7204a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.775+0000 7faabca00640 1 -- 172.21.15.59:0/3661947109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0037070 con 0x7faac0161fd0 2024-04-23T14:36:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.775+0000 7faabca00640 1 -- 172.21.15.59:0/3661947109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab0032440 con 0x7faac0161fd0 2024-04-23T14:36:47.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.775+0000 7faabca00640 1 -- 172.21.15.59:0/3661947109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0032810 con 0x7faac0161fd0 2024-04-23T14:36:47.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 -- 172.21.15.59:0/3661947109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac0161fd0 msgr2=0x7faac00b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 --2- 172.21.15.59:0/3661947109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac0161fd0 0x7faac00b3340 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7faab0009870 tx=0x7faab002a130 comp rx=0 tx=0).stop 2024-04-23T14:36:47.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 -- 172.21.15.59:0/3661947109 shutdown_connections 2024-04-23T14:36:47.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 --2- 172.21.15.59:0/3661947109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac00cdc90 0x7faac00d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 --2- 172.21.15.59:0/3661947109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac0161fd0 0x7faac00b3340 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 -- 172.21.15.59:0/3661947109 >> 172.21.15.59:0/3661947109 conn(0x7faac00b3f70 msgr2=0x7faac00b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:47.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 -- 172.21.15.59:0/3661947109 shutdown_connections 2024-04-23T14:36:47.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.776+0000 7faabf200640 1 -- 172.21.15.59:0/3661947109 wait complete. 2024-04-23T14:36:47.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.778+0000 7faabf200640 1 Processor -- start 2024-04-23T14:36:47.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.778+0000 7faabf200640 1 -- start start 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.779+0000 7faabf200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac009aa00 0x7faac00cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.779+0000 7faabf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac00cdc90 0x7faac014e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.779+0000 7faabf200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac0161fd0 0x7faac014eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.779+0000 7faabf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faac00d0cd0 con 0x7faac00cdc90 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.779+0000 7faabf200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faac00d0b50 con 0x7faac009aa00 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.779+0000 7faabf200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faac00d0e50 con 0x7faac0161fd0 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac00cdc90 0x7faac014e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac00cdc90 0x7faac014e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43144/0 (socket says 172.21.15.59:43144) 2024-04-23T14:36:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 -- 172.21.15.59:0/1767455558 learned_addr learned my addr 172.21.15.59:0/1767455558 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabe800640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac0161fd0 0x7faac014eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabde00640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac009aa00 0x7faac00cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 -- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac0161fd0 msgr2=0x7faac014eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac0161fd0 0x7faac014eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 -- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac009aa00 msgr2=0x7faac00cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac009aa00 0x7faac00cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.780+0000 7faabd400640 1 -- 172.21.15.59:0/1767455558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac00cbad0 con 0x7faac00cdc90 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.781+0000 7faabd400640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac00cdc90 0x7faac014e660 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c450 tx=0x7faaac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.781+0000 7faab6000640 1 -- 172.21.15.59:0/1767455558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0052d0 con 0x7faac00cdc90 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.781+0000 7faab6000640 1 -- 172.21.15.59:0/1767455558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaac005470 con 0x7faac00cdc90 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.781+0000 7faab6000640 1 -- 172.21.15.59:0/1767455558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac012870 con 0x7faac00cdc90 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.781+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac00c98f0 con 0x7faac00cdc90 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.781+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac00cc020 con 0x7faac00cdc90 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.782+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac004fd90 con 0x7faac00cdc90 2024-04-23T14:36:47.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.783+0000 7faab6000640 1 -- 172.21.15.59:0/1767455558 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaac012a10 con 0x7faac00cdc90 2024-04-23T14:36:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.785+0000 7faab6000640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa006c8f0 0x7faaa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.785+0000 7faab6000640 1 -- 172.21.15.59:0/1767455558 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7faaac08b7f0 con 0x7faac00cdc90 2024-04-23T14:36:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.785+0000 7faabde00640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa006c8f0 0x7faaa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.786+0000 7faabde00640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa006c8f0 0x7faaa006edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faab0034000 tx=0x7faab002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:47.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.789+0000 7faab6000640 1 -- 172.21.15.59:0/1767455558 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faaac059b80 con 0x7faac00cdc90 2024-04-23T14:36:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.994+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faac009ae80 con 0x7faac00cdc90 2024-04-23T14:36:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.996+0000 7faab6000640 1 -- 172.21.15.59:0/1767455558 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7faaac059480 con 0x7faac00cdc90 2024-04-23T14:36:47.996 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa006c8f0 msgr2=0x7faaa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa006c8f0 0x7faaa006edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faab0034000 tx=0x7faab002e000 comp rx=0 tx=0).stop 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac00cdc90 msgr2=0x7faac014e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac00cdc90 0x7faac014e660 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c450 tx=0x7faaac00c910 comp rx=0 tx=0).stop 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 shutdown_connections 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaa006c8f0 0x7faaa006edb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faac0161fd0 0x7faac014eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faac00cdc90 0x7faac014e660 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 --2- 172.21.15.59:0/1767455558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faac009aa00 0x7faac00cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 >> 172.21.15.59:0/1767455558 conn(0x7faac00b3f70 msgr2=0x7faac0070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.998+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 shutdown_connections 2024-04-23T14:36:47.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:47.999+0000 7faabf200640 1 -- 172.21.15.59:0/1767455558 wait complete. 2024-04-23T14:36:48.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:36:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.235+0000 7f5b0a800640 1 Processor -- start 2024-04-23T14:36:48.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.235+0000 7f5b0a800640 1 -- start start 2024-04-23T14:36:48.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.237+0000 7f5b0a800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5b040d15b0 0x7f5b040cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.237+0000 7f5b0a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b040cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.237+0000 7f5b0a800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b040b5da0 con 0x7f5b0415ed20 2024-04-23T14:36:48.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.237+0000 7f5b0a800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5b0415f100 con 0x7f5b040d15b0 2024-04-23T14:36:48.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.237+0000 7f5b0a800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5b040d0a20 con 0x7f5b040b4f20 2024-04-23T14:36:48.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.238+0000 7f5b09e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b040cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.238+0000 7f5b08a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5b040d15b0 0x7f5b040cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.238+0000 7f5b09e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b040cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:36690/0 (socket says 172.21.15.59:36690) 2024-04-23T14:36:48.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.238+0000 7f5b09e00640 1 -- 172.21.15.59:0/2797942626 learned_addr learned my addr 172.21.15.59:0/2797942626 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:48.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.239+0000 7f5b09e00640 1 -- 172.21.15.59:0/2797942626 >> v2:172.21.15.76:3300/0 conn(0x7f5b040d15b0 msgr2=0x7f5b040cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:48.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.239+0000 7f5b09e00640 1 --2- 172.21.15.59:0/2797942626 >> v2:172.21.15.76:3300/0 conn(0x7f5b040d15b0 0x7f5b040cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.239+0000 7f5b09e00640 1 -- 172.21.15.59:0/2797942626 >> v1:172.21.15.76:6789/0 conn(0x7f5b0415ed20 legacy=0x7f5b040d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.239+0000 7f5b09e00640 1 -- 172.21.15.59:0/2797942626 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b040d2ed0 con 0x7f5b040b4f20 2024-04-23T14:36:48.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.240+0000 7f5b09e00640 1 --2- 172.21.15.59:0/2797942626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b040cf7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009e50 tx=0x7f5afc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6847fce87b3bc7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.241+0000 7f5b03e00640 1 -- 172.21.15.59:0/2797942626 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc036070 con 0x7f5b040b4f20 2024-04-23T14:36:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.241+0000 7f5b03e00640 1 -- 172.21.15.59:0/2797942626 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc031600 con 0x7f5b040b4f20 2024-04-23T14:36:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.241+0000 7f5b03e00640 1 -- 172.21.15.59:0/2797942626 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0319d0 con 0x7f5b040b4f20 2024-04-23T14:36:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.241+0000 7f5b0a800640 1 -- 172.21.15.59:0/2797942626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 msgr2=0x7f5b040cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.241+0000 7f5b0a800640 1 --2- 172.21.15.59:0/2797942626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b040cf7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009e50 tx=0x7f5afc02a5f0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.242+0000 7f5b0a800640 1 -- 172.21.15.59:0/2797942626 shutdown_connections 2024-04-23T14:36:48.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.242+0000 7f5b0a800640 1 --2- 172.21.15.59:0/2797942626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b040cf7c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.242+0000 7f5b0a800640 1 --2- 172.21.15.59:0/2797942626 >> v2:172.21.15.76:3300/0 conn(0x7f5b040d15b0 0x7f5b040cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.242+0000 7f5b0a800640 1 -- 172.21.15.59:0/2797942626 >> 172.21.15.59:0/2797942626 conn(0x7f5b0408bf90 msgr2=0x7f5b040d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:48.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.242+0000 7f5b0a800640 1 -- 172.21.15.59:0/2797942626 shutdown_connections 2024-04-23T14:36:48.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.242+0000 7f5b0a800640 1 -- 172.21.15.59:0/2797942626 wait complete. 2024-04-23T14:36:48.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.244+0000 7f5b0a800640 1 Processor -- start 2024-04-23T14:36:48.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.244+0000 7f5b0a800640 1 -- start start 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b0a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b041549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b0a800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b040d15b0 0x7f5b040b9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b0a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0415ed20 0x7f5b040cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b0a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b040b3600 con 0x7f5b0415ed20 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b0a800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5b040d3490 con 0x7f5b040d15b0 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b0a800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5b040b3780 con 0x7f5b040b4f20 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b09e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0415ed20 0x7f5b040cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b09e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0415ed20 0x7f5b040cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43160/0 (socket says 172.21.15.59:43160) 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b09e00640 1 -- 172.21.15.59:0/1768394475 learned_addr learned my addr 172.21.15.59:0/1768394475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b09400640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b041549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.246+0000 7f5b08a00640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b040d15b0 0x7f5b040b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b09e00640 1 -- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 msgr2=0x7f5b041549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b09e00640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b041549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b09e00640 1 -- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b040d15b0 msgr2=0x7f5b040b9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b09e00640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b040d15b0 0x7f5b040b9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b09e00640 1 -- 172.21.15.59:0/1768394475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b040ba400 con 0x7f5b0415ed20 2024-04-23T14:36:48.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b09e00640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0415ed20 0x7f5b040cbfc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5afc034040 tx=0x7f5afc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:48.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b01600640 1 -- 172.21.15.59:0/1768394475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc036050 con 0x7f5b0415ed20 2024-04-23T14:36:48.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b01600640 1 -- 172.21.15.59:0/1768394475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc042070 con 0x7f5b0415ed20 2024-04-23T14:36:48.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b01600640 1 -- 172.21.15.59:0/1768394475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc009510 con 0x7f5b0415ed20 2024-04-23T14:36:48.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.247+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b040b9e40 con 0x7f5b0415ed20 2024-04-23T14:36:48.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.248+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b040cb7c0 con 0x7f5b0415ed20 2024-04-23T14:36:48.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.249+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b0404fd90 con 0x7f5b0415ed20 2024-04-23T14:36:48.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.250+0000 7f5b01600640 1 -- 172.21.15.59:0/1768394475 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5afc007c70 con 0x7f5b0415ed20 2024-04-23T14:36:48.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.250+0000 7f5b01600640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ae406c8f0 0x7f5ae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.251+0000 7f5b01600640 1 -- 172.21.15.59:0/1768394475 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f5afc0ac9e0 con 0x7f5b0415ed20 2024-04-23T14:36:48.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.251+0000 7f5b09400640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ae406c8f0 0x7f5ae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.252+0000 7f5b09400640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ae406c8f0 0x7f5ae406edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5af4001640 tx=0x7f5af4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:48.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.255+0000 7f5b01600640 1 -- 172.21.15.59:0/1768394475 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5afc045360 con 0x7f5b0415ed20 2024-04-23T14:36:48.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.462+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b040d5370 con 0x7f5b0415ed20 2024-04-23T14:36:48.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.463+0000 7f5b01600640 1 -- 172.21.15.59:0/1768394475 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7f5afc07ae00 con 0x7f5b0415ed20 2024-04-23T14:36:48.463 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:36:48.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ae406c8f0 msgr2=0x7f5ae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ae406c8f0 0x7f5ae406edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5af4001640 tx=0x7f5af4009040 comp rx=0 tx=0).stop 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0415ed20 msgr2=0x7f5b040cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0415ed20 0x7f5b040cbfc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5afc034040 tx=0x7f5afc004a40 comp rx=0 tx=0).stop 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 shutdown_connections 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ae406c8f0 0x7f5ae406edb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b0415ed20 0x7f5b040cbfc0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b040d15b0 0x7f5b040b9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 --2- 172.21.15.59:0/1768394475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5b040b4f20 0x7f5b041549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 >> 172.21.15.59:0/1768394475 conn(0x7f5b0408bf90 msgr2=0x7f5b040d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.466+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 shutdown_connections 2024-04-23T14:36:48.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.467+0000 7f5b0a800640 1 -- 172.21.15.59:0/1768394475 wait complete. 2024-04-23T14:36:48.487 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:36:48.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.706+0000 7f40ae200640 1 Processor -- start 2024-04-23T14:36:48.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.707+0000 7f40ae200640 1 -- start start 2024-04-23T14:36:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.708+0000 7f40ae200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a80d15b0 0x7f40a80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.708+0000 7f40ae200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40a80b5da0 con 0x7f40a80b4f20 2024-04-23T14:36:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.708+0000 7f40ae200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40a80d0a20 con 0x7f40a815ed20 2024-04-23T14:36:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.708+0000 7f40ae200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f40a815f100 con 0x7f40a80d15b0 2024-04-23T14:36:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.709+0000 7f40ad800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f40a80b4f20 0x7f40a80cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44038/0 (socket says 172.21.15.59:44038) 2024-04-23T14:36:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.709+0000 7f40ad800640 1 -- 172.21.15.59:0/1830194248 learned_addr learned my addr 172.21.15.59:0/1830194248 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:48.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.709+0000 7f40a7e00640 1 --2- 172.21.15.59:0/1830194248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a80d15b0 0x7f40a80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.709+0000 7f40a7400640 1 -- 172.21.15.59:0/1830194248 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157477819 0 0) 0x7f40a80b5da0 con 0x7f40a80b4f20 2024-04-23T14:36:48.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.710+0000 7f40a7400640 1 -- 172.21.15.59:0/1830194248 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4090003520 con 0x7f40a80b4f20 2024-04-23T14:36:48.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.710+0000 7f40a7400640 1 -- 172.21.15.59:0/1830194248 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2617137385 0 0) 0x7f40a80d0a20 con 0x7f40a815ed20 2024-04-23T14:36:48.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.710+0000 7f40a7e00640 1 -- 172.21.15.59:0/1830194248 >> v1:172.21.15.76:6789/0 conn(0x7f40a815ed20 legacy=0x7f40a80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.710+0000 7f40a7e00640 1 -- 172.21.15.59:0/1830194248 >> v1:172.21.15.59:6789/0 conn(0x7f40a80b4f20 legacy=0x7f40a80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.710+0000 7f40a7e00640 1 -- 172.21.15.59:0/1830194248 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a80d2ed0 con 0x7f40a80d15b0 2024-04-23T14:36:48.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.711+0000 7f40a7e00640 1 --2- 172.21.15.59:0/1830194248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a80d15b0 0x7f40a80cf280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40940097d0 tx=0x7f409402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2ec0583df2a8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:48.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.712+0000 7f40a7400640 1 -- 172.21.15.59:0/1830194248 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094037070 con 0x7f40a80d15b0 2024-04-23T14:36:48.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.712+0000 7f40a7400640 1 -- 172.21.15.59:0/1830194248 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f409402ae50 con 0x7f40a80d15b0 2024-04-23T14:36:48.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.712+0000 7f40a7400640 1 -- 172.21.15.59:0/1830194248 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094032640 con 0x7f40a80d15b0 2024-04-23T14:36:48.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.712+0000 7f40ae200640 1 -- 172.21.15.59:0/1830194248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a80d15b0 msgr2=0x7f40a80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.712+0000 7f40ae200640 1 --2- 172.21.15.59:0/1830194248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a80d15b0 0x7f40a80cf280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40940097d0 tx=0x7f409402a100 comp rx=0 tx=0).stop 2024-04-23T14:36:48.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.713+0000 7f40ae200640 1 -- 172.21.15.59:0/1830194248 shutdown_connections 2024-04-23T14:36:48.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.713+0000 7f40ae200640 1 --2- 172.21.15.59:0/1830194248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a80d15b0 0x7f40a80cf280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.713+0000 7f40ae200640 1 -- 172.21.15.59:0/1830194248 >> 172.21.15.59:0/1830194248 conn(0x7f40a808bf90 msgr2=0x7f40a80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:48.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.713+0000 7f40ae200640 1 -- 172.21.15.59:0/1830194248 shutdown_connections 2024-04-23T14:36:48.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.713+0000 7f40ae200640 1 -- 172.21.15.59:0/1830194248 wait complete. 2024-04-23T14:36:48.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.715+0000 7f40ae200640 1 Processor -- start 2024-04-23T14:36:48.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.715+0000 7f40ae200640 1 -- start start 2024-04-23T14:36:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.716+0000 7f40ae200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40a80b4f20 0x7f40a80ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.716+0000 7f40ae200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a80d15b0 0x7f40a81f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.716+0000 7f40ae200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a815ed20 0x7f40a81f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.716+0000 7f40ae200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40a80b3600 con 0x7f40a80d15b0 2024-04-23T14:36:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.716+0000 7f40ae200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f40a80d3490 con 0x7f40a80b4f20 2024-04-23T14:36:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.716+0000 7f40ae200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f40a80b3780 con 0x7f40a815ed20 2024-04-23T14:36:48.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a80d15b0 0x7f40a81f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a80d15b0 0x7f40a81f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43166/0 (socket says 172.21.15.59:43166) 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 -- 172.21.15.59:0/2422099580 learned_addr learned my addr 172.21.15.59:0/2422099580 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40ace00640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40a80b4f20 0x7f40a80ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40ad800640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a815ed20 0x7f40a81f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 -- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a815ed20 msgr2=0x7f40a81f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a815ed20 0x7f40a81f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 -- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40a80b4f20 msgr2=0x7f40a80ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40a80b4f20 0x7f40a80ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 -- 172.21.15.59:0/2422099580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a81f6270 con 0x7f40a80d15b0 2024-04-23T14:36:48.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.717+0000 7f40a7e00640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a80d15b0 0x7f40a81f72f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4094035040 tx=0x7f40940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:48.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.718+0000 7f40a4c00640 1 -- 172.21.15.59:0/2422099580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094037050 con 0x7f40a80d15b0 2024-04-23T14:36:48.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.718+0000 7f40a4c00640 1 -- 172.21.15.59:0/2422099580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4094038040 con 0x7f40a80d15b0 2024-04-23T14:36:48.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.718+0000 7f40a4c00640 1 -- 172.21.15.59:0/2422099580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40940044c0 con 0x7f40a80d15b0 2024-04-23T14:36:48.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.718+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a81f6550 con 0x7f40a80d15b0 2024-04-23T14:36:48.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.718+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a81f6df0 con 0x7f40a80d15b0 2024-04-23T14:36:48.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.719+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a81f6940 con 0x7f40a80d15b0 2024-04-23T14:36:48.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.720+0000 7f40a4c00640 1 -- 172.21.15.59:0/2422099580 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4094045020 con 0x7f40a80d15b0 2024-04-23T14:36:48.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.721+0000 7f40a4c00640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f408806c8f0 0x7f408806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:48.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.721+0000 7f40a4c00640 1 -- 172.21.15.59:0/2422099580 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f40940aba30 con 0x7f40a80d15b0 2024-04-23T14:36:48.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.721+0000 7f40ace00640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f408806c8f0 0x7f408806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:48.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.722+0000 7f40ace00640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f408806c8f0 0x7f408806edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f409c000f60 tx=0x7f409c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:48.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.726+0000 7f40a4c00640 1 -- 172.21.15.59:0/2422099580 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f409404c020 con 0x7f40a80d15b0 2024-04-23T14:36:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.931+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40a81f6720 con 0x7f40a80d15b0 2024-04-23T14:36:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.932+0000 7f40a4c00640 1 -- 172.21.15.59:0/2422099580 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7f4094079e60 con 0x7f40a80d15b0 2024-04-23T14:36:48.932 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f408806c8f0 msgr2=0x7f408806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f408806c8f0 0x7f408806edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f409c000f60 tx=0x7f409c009040 comp rx=0 tx=0).stop 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a80d15b0 msgr2=0x7f40a81f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a80d15b0 0x7f40a81f72f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4094035040 tx=0x7f40940046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 shutdown_connections 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f408806c8f0 0x7f408806edb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40a815ed20 0x7f40a81f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a80d15b0 0x7f40a81f72f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 --2- 172.21.15.59:0/2422099580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40a80b4f20 0x7f40a80ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 >> 172.21.15.59:0/2422099580 conn(0x7f40a808bf90 msgr2=0x7f40a815cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 shutdown_connections 2024-04-23T14:36:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:48.935+0000 7f40ae200640 1 -- 172.21.15.59:0/2422099580 wait complete. 2024-04-23T14:36:48.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:36:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.174+0000 7f6cc6800640 1 Processor -- start 2024-04-23T14:36:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.174+0000 7f6cc6800640 1 -- start start 2024-04-23T14:36:49.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.175+0000 7f6cc6800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 0x7f6cc00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.175+0000 7f6cc6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6cc00b4230 0x7f6cc00b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc6800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cc00b4cf0 con 0x7f6cc00d05c0 2024-04-23T14:36:49.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc6800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc015ca50 con 0x7f6cc00b4230 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc6800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc00ccb80 con 0x7f6cc015c670 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6cc00b4230 0x7f6cc00b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6cc00b4230 0x7f6cc00b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43180/0 (socket says 172.21.15.59:43180) 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc4a00640 1 -- 172.21.15.59:0/3667103312 learned_addr learned my addr 172.21.15.59:0/3667103312 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc5400640 1 --2- 172.21.15.59:0/3667103312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 0x7f6cc00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc4a00640 1 -- 172.21.15.59:0/3667103312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 msgr2=0x7f6cc00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc4a00640 1 --2- 172.21.15.59:0/3667103312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 0x7f6cc00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc4a00640 1 -- 172.21.15.59:0/3667103312 >> v1:172.21.15.76:6789/0 conn(0x7f6cc00d05c0 legacy=0x7f6cc0160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.176+0000 7f6cc4a00640 1 -- 172.21.15.59:0/3667103312 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc00ccdd0 con 0x7f6cc00b4230 2024-04-23T14:36:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.178+0000 7f6cc4a00640 1 --2- 172.21.15.59:0/3667103312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc00b4690 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009870 tx=0x7f6cac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c47a1db7488ccab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.178+0000 7f6cbfe00640 1 -- 172.21.15.59:0/3667103312 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac037070 con 0x7f6cc00b4230 2024-04-23T14:36:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.178+0000 7f6cbfe00640 1 -- 172.21.15.59:0/3667103312 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cac032440 con 0x7f6cc00b4230 2024-04-23T14:36:49.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.178+0000 7f6cbfe00640 1 -- 172.21.15.59:0/3667103312 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac032810 con 0x7f6cc00b4230 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.178+0000 7f6cc6800640 1 -- 172.21.15.59:0/3667103312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 msgr2=0x7f6cc00b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.178+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3667103312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc00b4690 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009870 tx=0x7f6cac02a210 comp rx=0 tx=0).stop 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.179+0000 7f6cc6800640 1 -- 172.21.15.59:0/3667103312 shutdown_connections 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.179+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3667103312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc00b4690 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.179+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3667103312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 0x7f6cc00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.179+0000 7f6cc6800640 1 -- 172.21.15.59:0/3667103312 >> 172.21.15.59:0/3667103312 conn(0x7f6cc006bdf0 msgr2=0x7f6cc006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.179+0000 7f6cc6800640 1 -- 172.21.15.59:0/3667103312 shutdown_connections 2024-04-23T14:36:49.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.179+0000 7f6cc6800640 1 -- 172.21.15.59:0/3667103312 wait complete. 2024-04-23T14:36:49.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.180+0000 7f6cc6800640 1 Processor -- start 2024-04-23T14:36:49.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.181+0000 7f6cc6800640 1 -- start start 2024-04-23T14:36:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc0155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc6800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc00d05c0 0x7f6cc0156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc6800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 0x7f6cc01546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc6800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc00b3e60 con 0x7f6cc00b4230 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc6800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc00cd390 con 0x7f6cc00d05c0 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc6800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc00b3fe0 con 0x7f6cc015c670 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc0155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc0155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43184/0 (socket says 172.21.15.59:43184) 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 -- 172.21.15.59:0/3213813017 learned_addr learned my addr 172.21.15.59:0/3213813017 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc4a00640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc00d05c0 0x7f6cc0156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 -- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 msgr2=0x7f6cc01546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 0x7f6cc01546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 -- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc00d05c0 msgr2=0x7f6cc0156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc00d05c0 0x7f6cc0156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc5400640 1 -- 172.21.15.59:0/3213813017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc0156740 con 0x7f6cc00b4230 2024-04-23T14:36:49.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.182+0000 7f6cc4a00640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc00d05c0 0x7f6cc0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:49.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.183+0000 7f6cc5400640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc0155fd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d620 tx=0x7f6cb400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:49.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.183+0000 7f6cbd600640 1 -- 172.21.15.59:0/3213813017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400b620 con 0x7f6cc00b4230 2024-04-23T14:36:49.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.183+0000 7f6cbd600640 1 -- 172.21.15.59:0/3213813017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4019070 con 0x7f6cc00b4230 2024-04-23T14:36:49.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.183+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc0072a80 con 0x7f6cc00b4230 2024-04-23T14:36:49.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.183+0000 7f6cbd600640 1 -- 172.21.15.59:0/3213813017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400ba00 con 0x7f6cc00b4230 2024-04-23T14:36:49.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.183+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc0073040 con 0x7f6cc00b4230 2024-04-23T14:36:49.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.184+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc004fd90 con 0x7f6cc00b4230 2024-04-23T14:36:49.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.185+0000 7f6cbd600640 1 -- 172.21.15.59:0/3213813017 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cb400bba0 con 0x7f6cc00b4230 2024-04-23T14:36:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.186+0000 7f6cbd600640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ca006c8f0 0x7f6ca006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.186+0000 7f6cbd600640 1 -- 172.21.15.59:0/3213813017 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f6cb408bf50 con 0x7f6cc00b4230 2024-04-23T14:36:49.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.187+0000 7f6cc4a00640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ca006c8f0 0x7f6ca006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.188+0000 7f6cc4a00640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ca006c8f0 0x7f6ca006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6cac034000 tx=0x7f6cac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.191+0000 7f6cbd600640 1 -- 172.21.15.59:0/3213813017 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cb405a370 con 0x7f6cc00b4230 2024-04-23T14:36:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.409+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6cc00ceab0 con 0x7f6cc00b4230 2024-04-23T14:36:49.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.409+0000 7f6cbd600640 1 -- 172.21.15.59:0/3213813017 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f6cb4060030 con 0x7f6cc00b4230 2024-04-23T14:36:49.410 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:36:49.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ca006c8f0 msgr2=0x7f6ca006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ca006c8f0 0x7f6ca006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6cac034000 tx=0x7f6cac02e000 comp rx=0 tx=0).stop 2024-04-23T14:36:49.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 msgr2=0x7f6cc0155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc0155fd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d620 tx=0x7f6cb400dae0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 shutdown_connections 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6ca006c8f0 0x7f6ca006edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6cc015c670 0x7f6cc01546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc00d05c0 0x7f6cc0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 --2- 172.21.15.59:0/3213813017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cc00b4230 0x7f6cc0155fd0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 >> 172.21.15.59:0/3213813017 conn(0x7f6cc006bdf0 msgr2=0x7f6cc0155860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 shutdown_connections 2024-04-23T14:36:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.412+0000 7f6cc6800640 1 -- 172.21.15.59:0/3213813017 wait complete. 2024-04-23T14:36:49.431 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:36:49.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:49.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:49.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[57738]: fuse finished with error 0 and tester_r 0 2024-04-23T14:36:49.513 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:36:49.587 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:36:49.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:49.588 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:36:49.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:49.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.842+0000 7fea47e00640 1 Processor -- start 2024-04-23T14:36:49.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.842+0000 7fea47e00640 1 -- start start 2024-04-23T14:36:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea47e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea4815c3d0 0x7fea480cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea47e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea480cd590 0x7fea480b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea47e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea48160210 con 0x7fea480b4690 2024-04-23T14:36:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea47e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fea481604f0 con 0x7fea4815c3d0 2024-04-23T14:36:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea47e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fea481607d0 con 0x7fea480cd590 2024-04-23T14:36:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea46a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea4815c3d0 0x7fea480cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea46a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea4815c3d0 0x7fea480cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43198/0 (socket says 172.21.15.59:43198) 2024-04-23T14:36:49.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea47400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fea480b4690 0x7fea4806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44050/0 (socket says 172.21.15.59:44050) 2024-04-23T14:36:49.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.844+0000 7fea46a00640 1 -- 172.21.15.59:0/924385085 learned_addr learned my addr 172.21.15.59:0/924385085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.845+0000 7fea46a00640 1 -- 172.21.15.59:0/924385085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea480cd590 msgr2=0x7fea480b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.845+0000 7fea46a00640 1 --2- 172.21.15.59:0/924385085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea480cd590 0x7fea480b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.845+0000 7fea46a00640 1 -- 172.21.15.59:0/924385085 >> v1:172.21.15.59:6789/0 conn(0x7fea480b4690 legacy=0x7fea4806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.845+0000 7fea46a00640 1 -- 172.21.15.59:0/924385085 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea48160ab0 con 0x7fea4815c3d0 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.846+0000 7fea46a00640 1 --2- 172.21.15.59:0/924385085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea4815c3d0 0x7fea480cd050 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea3802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e5629057c1d816 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.846+0000 7fea45600640 1 -- 172.21.15.59:0/924385085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38037070 con 0x7fea4815c3d0 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.846+0000 7fea45600640 1 -- 172.21.15.59:0/924385085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38032440 con 0x7fea4815c3d0 2024-04-23T14:36:49.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.846+0000 7fea45600640 1 -- 172.21.15.59:0/924385085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38032810 con 0x7fea4815c3d0 2024-04-23T14:36:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.846+0000 7fea47e00640 1 -- 172.21.15.59:0/924385085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea4815c3d0 msgr2=0x7fea480cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.846+0000 7fea47e00640 1 --2- 172.21.15.59:0/924385085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea4815c3d0 0x7fea480cd050 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea3802a250 comp rx=0 tx=0).stop 2024-04-23T14:36:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.847+0000 7fea47e00640 1 -- 172.21.15.59:0/924385085 shutdown_connections 2024-04-23T14:36:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.847+0000 7fea47e00640 1 --2- 172.21.15.59:0/924385085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea480cd590 0x7fea480b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.847+0000 7fea47e00640 1 --2- 172.21.15.59:0/924385085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea4815c3d0 0x7fea480cd050 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.847+0000 7fea47e00640 1 -- 172.21.15.59:0/924385085 >> 172.21.15.59:0/924385085 conn(0x7fea480b21a0 msgr2=0x7fea480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.847+0000 7fea47e00640 1 -- 172.21.15.59:0/924385085 shutdown_connections 2024-04-23T14:36:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.847+0000 7fea47e00640 1 -- 172.21.15.59:0/924385085 wait complete. 2024-04-23T14:36:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.848+0000 7fea47e00640 1 Processor -- start 2024-04-23T14:36:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.849+0000 7fea47e00640 1 -- start start 2024-04-23T14:36:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.849+0000 7fea47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea480b4690 0x7fea4815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea47e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea480cd590 0x7fea48152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea47e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea4815c3d0 0x7fea4815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea47e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea4815c7b0 con 0x7fea480b4690 2024-04-23T14:36:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea47e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea480b4b00 con 0x7fea480cd590 2024-04-23T14:36:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea47e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fea4815c930 con 0x7fea4815c3d0 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea480b4690 0x7fea4815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea46a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea480b4690 0x7fea4815cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43210/0 (socket says 172.21.15.59:43210) 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea46a00640 1 -- 172.21.15.59:0/2637330799 learned_addr learned my addr 172.21.15.59:0/2637330799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea47400640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea4815c3d0 0x7fea4815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.850+0000 7fea46000640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea480cd590 0x7fea48152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea46a00640 1 -- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea4815c3d0 msgr2=0x7fea4815e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea46a00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea4815c3d0 0x7fea4815e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea46a00640 1 -- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea480cd590 msgr2=0x7fea48152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea46a00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea480cd590 0x7fea48152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea46a00640 1 -- 172.21.15.59:0/2637330799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea480cf7d0 con 0x7fea480b4690 2024-04-23T14:36:49.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea46a00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea480b4690 0x7fea4815cce0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea3f400640 1 -- 172.21.15.59:0/2637330799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38041020 con 0x7fea480b4690 2024-04-23T14:36:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea3f400640 1 -- 172.21.15.59:0/2637330799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea38030970 con 0x7fea480b4690 2024-04-23T14:36:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea480c74d0 con 0x7fea480b4690 2024-04-23T14:36:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea3f400640 1 -- 172.21.15.59:0/2637330799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38037040 con 0x7fea480b4690 2024-04-23T14:36:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.851+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea480cf4f0 con 0x7fea480b4690 2024-04-23T14:36:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.853+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4804fd90 con 0x7fea480b4690 2024-04-23T14:36:49.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.853+0000 7fea3f400640 1 -- 172.21.15.59:0/2637330799 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea38044020 con 0x7fea480b4690 2024-04-23T14:36:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.854+0000 7fea3f400640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea2806c8f0 0x7fea2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.854+0000 7fea3f400640 1 -- 172.21.15.59:0/2637330799 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fea380ac550 con 0x7fea480b4690 2024-04-23T14:36:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.855+0000 7fea46000640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea2806c8f0 0x7fea2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:49.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.855+0000 7fea46000640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea2806c8f0 0x7fea2806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fea4815e370 tx=0x7fea34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:49.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:49.859+0000 7fea3f400640 1 -- 172.21.15.59:0/2637330799 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea3804b030 con 0x7fea480b4690 2024-04-23T14:36:49.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:36:49.899 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:36:50.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.102+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea480b0c90 con 0x7fea480b4690 2024-04-23T14:36:50.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.104+0000 7fea3f400640 1 -- 172.21.15.59:0/2637330799 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5183 (secure 0 0 0) 0x7fea3807a970 con 0x7fea480b4690 2024-04-23T14:36:50.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:50.104 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:36:32.769281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6735":{"gid":6735,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6836/2883460093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2883460093},{"type":"v1","addr":"172.21.15.59:6836","nonce":2883460093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6735,"qdb_cluster":[6735]},"id":5}]} 2024-04-23T14:36:50.104 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 43 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea2806c8f0 msgr2=0x7fea2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea2806c8f0 0x7fea2806edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fea4815e370 tx=0x7fea34008040 comp rx=0 tx=0).stop 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea480b4690 msgr2=0x7fea4815cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea480b4690 0x7fea4815cce0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea380046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 shutdown_connections 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea2806c8f0 0x7fea2806edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea4815c3d0 0x7fea4815e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.106+0000 7fea47e00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea480cd590 0x7fea48152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.107+0000 7fea47e00640 1 --2- 172.21.15.59:0/2637330799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea480b4690 0x7fea4815cce0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.107+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 >> 172.21.15.59:0/2637330799 conn(0x7fea480b21a0 msgr2=0x7fea480b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.107+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 shutdown_connections 2024-04-23T14:36:50.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.107+0000 7fea47e00640 1 -- 172.21.15.59:0/2637330799 wait complete. 2024-04-23T14:36:50.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-23T14:36:50.363 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:36:50.364 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:36:50.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:36:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.565+0000 7f0d87e00640 1 Processor -- start 2024-04-23T14:36:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.565+0000 7f0d87e00640 1 -- start start 2024-04-23T14:36:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.566+0000 7f0d87e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d880b2630 0x7f0d8806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.566+0000 7f0d87e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0d8806bf10 0x7f0d880cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.566+0000 7f0d87e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d8806c700 con 0x7f0d88159140 2024-04-23T14:36:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.566+0000 7f0d87e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0d88159520 con 0x7f0d880b2630 2024-04-23T14:36:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.566+0000 7f0d87e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0d8815f1b0 con 0x7f0d8806bf10 2024-04-23T14:36:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d86000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d880b2630 0x7f0d8806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d86000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d880b2630 0x7f0d8806b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36638/0 (socket says 172.21.15.59:36638) 2024-04-23T14:36:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d86000640 1 -- 172.21.15.59:0/59305251 learned_addr learned my addr 172.21.15.59:0/59305251 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d87400640 1 --2- 172.21.15.59:0/59305251 >> v2:172.21.15.76:3300/0 conn(0x7f0d8806bf10 0x7f0d880cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d86000640 1 -- 172.21.15.59:0/59305251 >> v2:172.21.15.76:3300/0 conn(0x7f0d8806bf10 msgr2=0x7f0d880cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d86000640 1 --2- 172.21.15.59:0/59305251 >> v2:172.21.15.76:3300/0 conn(0x7f0d8806bf10 0x7f0d880cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d86000640 1 -- 172.21.15.59:0/59305251 >> v1:172.21.15.76:6789/0 conn(0x7f0d88159140 legacy=0x7f0d880b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:50.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.567+0000 7f0d86000640 1 -- 172.21.15.59:0/59305251 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8815f490 con 0x7f0d880b2630 2024-04-23T14:36:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.568+0000 7f0d86000640 1 --2- 172.21.15.59:0/59305251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8806b9d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0d880cf3f0 tx=0x7f0d7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6448c56f3a128e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.569+0000 7f0d85600640 1 -- 172.21.15.59:0/59305251 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70037070 con 0x7f0d880b2630 2024-04-23T14:36:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.569+0000 7f0d85600640 1 -- 172.21.15.59:0/59305251 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d70032440 con 0x7f0d880b2630 2024-04-23T14:36:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.569+0000 7f0d85600640 1 -- 172.21.15.59:0/59305251 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70032810 con 0x7f0d880b2630 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.569+0000 7f0d87e00640 1 -- 172.21.15.59:0/59305251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 msgr2=0x7f0d8806b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.569+0000 7f0d87e00640 1 --2- 172.21.15.59:0/59305251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8806b9d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0d880cf3f0 tx=0x7f0d7002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.570+0000 7f0d87e00640 1 -- 172.21.15.59:0/59305251 shutdown_connections 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.570+0000 7f0d87e00640 1 --2- 172.21.15.59:0/59305251 >> v2:172.21.15.76:3300/0 conn(0x7f0d8806bf10 0x7f0d880cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.570+0000 7f0d87e00640 1 --2- 172.21.15.59:0/59305251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8806b9d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.570+0000 7f0d87e00640 1 -- 172.21.15.59:0/59305251 >> 172.21.15.59:0/59305251 conn(0x7f0d8809aa00 msgr2=0x7f0d880d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.570+0000 7f0d87e00640 1 -- 172.21.15.59:0/59305251 shutdown_connections 2024-04-23T14:36:50.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.570+0000 7f0d87e00640 1 -- 172.21.15.59:0/59305251 wait complete. 2024-04-23T14:36:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.571+0000 7f0d87e00640 1 Processor -- start 2024-04-23T14:36:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.572+0000 7f0d87e00640 1 -- start start 2024-04-23T14:36:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d8806bf10 0x7f0d881f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d88159140 0x7f0d880cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d87e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8815ff70 con 0x7f0d880b2630 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d87e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8815fa50 con 0x7f0d88159140 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d87e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8806c360 con 0x7f0d8806bf10 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8815c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36646/0 (socket says 172.21.15.59:36646) 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 -- 172.21.15.59:0/1002537986 learned_addr learned my addr 172.21.15.59:0/1002537986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86a00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d8806bf10 0x7f0d881f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 -- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d8806bf10 msgr2=0x7f0d881f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d87400640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d88159140 0x7f0d880cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d8806bf10 0x7f0d881f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 -- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d88159140 msgr2=0x7f0d880cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d88159140 0x7f0d880cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.573+0000 7f0d86000640 1 -- 172.21.15.59:0/1002537986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d881f4fa0 con 0x7f0d880b2630 2024-04-23T14:36:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.574+0000 7f0d86a00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d8806bf10 0x7f0d881f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.574+0000 7f0d86000640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8815c060 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0d70000c00 tx=0x7f0d700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.575+0000 7f0d7ea00640 1 -- 172.21.15.59:0/1002537986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70041020 con 0x7f0d880b2630 2024-04-23T14:36:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.575+0000 7f0d7ea00640 1 -- 172.21.15.59:0/1002537986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d700306c0 con 0x7f0d880b2630 2024-04-23T14:36:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.575+0000 7f0d7ea00640 1 -- 172.21.15.59:0/1002537986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70037040 con 0x7f0d880b2630 2024-04-23T14:36:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.575+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d881f5280 con 0x7f0d880b2630 2024-04-23T14:36:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.575+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d880cba70 con 0x7f0d880b2630 2024-04-23T14:36:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.576+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d881f57a0 con 0x7f0d880b2630 2024-04-23T14:36:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.577+0000 7f0d7ea00640 1 -- 172.21.15.59:0/1002537986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d70044020 con 0x7f0d880b2630 2024-04-23T14:36:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.578+0000 7f0d7ea00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d6406c8f0 0x7f0d6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.578+0000 7f0d7ea00640 1 -- 172.21.15.59:0/1002537986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f0d700ac420 con 0x7f0d880b2630 2024-04-23T14:36:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.578+0000 7f0d86a00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d6406c8f0 0x7f0d6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.579+0000 7f0d86a00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d6406c8f0 0x7f0d6406edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0d78003eb0 tx=0x7f0d78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:50.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.583+0000 7f0d7ea00640 1 -- 172.21.15.59:0/1002537986 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d881f57a0 con 0x7f0d880b2630 2024-04-23T14:36:50.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.828+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0d8804fd90 con 0x7f0d880b2630 2024-04-23T14:36:50.829 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:36:50.830+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:36:50.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.833+0000 7f0d7ea00640 1 -- 172.21.15.59:0/1002537986 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7f0d7004b030 con 0x7f0d880b2630 2024-04-23T14:36:50.833 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.836+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d6406c8f0 msgr2=0x7f0d6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.836+0000 7f0d87e00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d6406c8f0 0x7f0d6406edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0d78003eb0 tx=0x7f0d78009040 comp rx=0 tx=0).stop 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.836+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 msgr2=0x7f0d8815c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.836+0000 7f0d87e00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8815c060 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0d70000c00 tx=0x7f0d700046c0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 shutdown_connections 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d6406c8f0 0x7f0d6406edb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d88159140 0x7f0d880cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d880b2630 0x7f0d8815c060 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 --2- 172.21.15.59:0/1002537986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d8806bf10 0x7f0d881f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 >> 172.21.15.59:0/1002537986 conn(0x7f0d8809aa00 msgr2=0x7f0d8815cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 shutdown_connections 2024-04-23T14:36:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.837+0000 7f0d87e00640 1 -- 172.21.15.59:0/1002537986 wait complete. 2024-04-23T14:36:50.857 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-23T14:36:50.861 INFO:tasks.ceph.mds.a.smithi059.stderr: -3592> 2024-04-23T14:36:35.168+0000 7ff4a7200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:36:50.932 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:36:50.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.948+0000 7ffa88750f40 1 Processor -- start 2024-04-23T14:36:50.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.949+0000 7ffa88750f40 1 -- start start 2024-04-23T14:36:50.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.949+0000 7ffa88750f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c0022a0 0x55960a4bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.949+0000 7ffa88750f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a5720 0x55960a4a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.949+0000 7ffa88750f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55960a4a17c0 0x55960a4311a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.949+0000 7ffa88750f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55960a38fcc0 con 0x55960a4a5720 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.949+0000 7ffa88750f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55960a393d00 con 0x7ffa6c0022a0 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.949+0000 7ffa88750f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55960a4341f0 con 0x55960a4a17c0 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a5720 0x55960a4a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a5720 0x55960a4a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36690/0 (socket says 172.21.15.59:36690) 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 -- 172.21.15.59:0/2426261455 learned_addr learned my addr 172.21.15.59:0/2426261455 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85a00640 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c0022a0 0x55960a4bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa86400640 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55960a4a17c0 0x55960a4311a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 -- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55960a4a17c0 msgr2=0x55960a4311a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55960a4a17c0 0x55960a4311a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 -- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c0022a0 msgr2=0x55960a4bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c0022a0 0x55960a4bc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 -- 172.21.15.59:0/2426261455 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960a3e7250 con 0x55960a4a5720 2024-04-23T14:36:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85a00640 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c0022a0 0x55960a4bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.950+0000 7ffa85000640 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a5720 0x55960a4a2430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400d750 tx=0x7ffa7400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80c1450a1e6c084 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa7f400640 1 -- 172.21.15.59:0/2426261455 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400b8d0 con 0x55960a4a5720 2024-04-23T14:36:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa7f400640 1 -- 172.21.15.59:0/2426261455 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa74014020 con 0x55960a4a5720 2024-04-23T14:36:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa7f400640 1 -- 172.21.15.59:0/2426261455 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400bcb0 con 0x55960a4a5720 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 -- 172.21.15.59:0/2426261455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a5720 msgr2=0x55960a4a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a5720 0x55960a4a2430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400d750 tx=0x7ffa7400dc10 comp rx=0 tx=0).stop 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 -- 172.21.15.59:0/2426261455 shutdown_connections 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55960a4a17c0 0x55960a4311a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a5720 0x55960a4a2430 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2426261455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c0022a0 0x55960a4bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 -- 172.21.15.59:0/2426261455 >> 172.21.15.59:0/2426261455 conn(0x55960a417f40 msgr2=0x55960a4350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 -- 172.21.15.59:0/2426261455 shutdown_connections 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.951+0000 7ffa88750f40 1 -- 172.21.15.59:0/2426261455 wait complete. 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.952+0000 7ffa88750f40 1 Processor -- start 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.952+0000 7ffa88750f40 1 -- start start 2024-04-23T14:36:50.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.953+0000 7ffa88750f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a17c0 0x55960a41b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.953+0000 7ffa88750f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55960a4a5720 0x55960a54fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.953+0000 7ffa88750f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa6c0022a0 0x55960a4a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.953+0000 7ffa88750f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55960a4a51d0 con 0x55960a4a17c0 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.953+0000 7ffa88750f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55960a3e74f0 con 0x55960a4a5720 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.953+0000 7ffa88750f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55960a4a4c70 con 0x7ffa6c0022a0 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.953+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a17c0 0x55960a41b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a17c0 0x55960a41b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36700/0 (socket says 172.21.15.59:36700) 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 -- 172.21.15.59:0/2363488895 learned_addr learned my addr 172.21.15.59:0/2363488895 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa86400640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa6c0022a0 0x55960a4a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85000640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55960a4a5720 0x55960a54fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 -- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa6c0022a0 msgr2=0x55960a4a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa6c0022a0 0x55960a4a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 -- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55960a4a5720 msgr2=0x55960a54fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55960a4a5720 0x55960a54fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 -- 172.21.15.59:0/2363488895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960a4a13a0 con 0x55960a4a17c0 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85000640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55960a4a5720 0x55960a54fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.954+0000 7ffa85a00640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a17c0 0x55960a41b9e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ffa78002e70 tx=0x7ffa78001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.955+0000 7ffa7e000640 1 -- 172.21.15.59:0/2363488895 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78063070 con 0x55960a4a17c0 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.955+0000 7ffa7e000640 1 -- 172.21.15.59:0/2363488895 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa78054440 con 0x55960a4a17c0 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.955+0000 7ffa7e000640 1 -- 172.21.15.59:0/2363488895 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78059040 con 0x55960a4a17c0 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.955+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55960a54ecc0 con 0x55960a4a17c0 2024-04-23T14:36:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.955+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960a54f280 con 0x55960a4a17c0 2024-04-23T14:36:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.957+0000 7ffa7e000640 1 -- 172.21.15.59:0/2363488895 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa78066070 con 0x55960a4a17c0 2024-04-23T14:36:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.957+0000 7ffa7e000640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa6806b300 0x7ffa6806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.958+0000 7ffa7e000640 1 -- 172.21.15.59:0/2363488895 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7ffa780ce220 con 0x55960a4a17c0 2024-04-23T14:36:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.958+0000 7ffa85000640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa6806b300 0x7ffa6806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.958+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55960a419a60 0x55960a562130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.958+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70) -- 0x55960a562670 con 0x55960a419a60 2024-04-23T14:36:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.958+0000 7ffa85000640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa6806b300 0x7ffa6806d7c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400d750 tx=0x7ffa74009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.959+0000 7ffa86400640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55960a419a60 0x55960a562130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.960+0000 7ffa86400640 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55960a419a60 0x55960a562130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa86400640 1 -- 172.21.15.59:0/2363488895 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v70'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa80011440 con 0x55960a419a60 2024-04-23T14:36:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55960a419a60 msgr2=0x55960a562130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55960a419a60 0x55960a562130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa6806b300 msgr2=0x7ffa6806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa6806b300 0x7ffa6806d7c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400d750 tx=0x7ffa74009000 comp rx=0 tx=0).stop 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a17c0 msgr2=0x55960a41b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 --2- 172.21.15.59:0/2363488895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55960a4a17c0 0x55960a41b9e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ffa78002e70 tx=0x7ffa78001fe0 comp rx=0 tx=0).stop 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa86400640 1 -- 172.21.15.59:0/2363488895 reap_dead start 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 shutdown_connections 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 >> 172.21.15.59:0/2363488895 conn(0x55960a417f40 msgr2=0x55960a4350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 shutdown_connections 2024-04-23T14:36:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:50.964+0000 7ffa88750f40 1 -- 172.21.15.59:0/2363488895 wait complete. 2024-04-23T14:36:50.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-23T14:36:51.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.155+0000 7f0d60800640 1 Processor -- start 2024-04-23T14:36:51.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.155+0000 7f0d60800640 1 -- start start 2024-04-23T14:36:51.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d60800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5815e790 0x7f0d580aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d60800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d58071d70 0x7f0d58072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d60800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d580adc80 con 0x7f0d5806f6f0 2024-04-23T14:36:51.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d60800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0d580aef40 con 0x7f0d58071d70 2024-04-23T14:36:51.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d60800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580afd80 con 0x7f0d5815e790 2024-04-23T14:36:51.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d5e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d58071d70 0x7f0d58072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:51.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d5e800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d58071d70 0x7f0d58072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36712/0 (socket says 172.21.15.59:36712) 2024-04-23T14:36:51.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.157+0000 7f0d5e800640 1 -- 172.21.15.59:0/3445886605 learned_addr learned my addr 172.21.15.59:0/3445886605 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:51.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.158+0000 7f0d5e800640 1 -- 172.21.15.59:0/3445886605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5815e790 msgr2=0x7f0d580aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:51.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.158+0000 7f0d5e800640 1 --2- 172.21.15.59:0/3445886605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5815e790 0x7f0d580aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.158+0000 7f0d5e800640 1 -- 172.21.15.59:0/3445886605 >> v1:172.21.15.76:6789/0 conn(0x7f0d5806f6f0 legacy=0x7f0d58162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:51.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.158+0000 7f0d5e800640 1 -- 172.21.15.59:0/3445886605 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d580adf20 con 0x7f0d58071d70 2024-04-23T14:36:51.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.159+0000 7f0d5e800640 1 --2- 172.21.15.59:0/3445886605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d58072200 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009e50 tx=0x7f0d5402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c3013215fd4020 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:51.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.159+0000 7f0d5ca00640 1 -- 172.21.15.59:0/3445886605 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54036070 con 0x7f0d58071d70 2024-04-23T14:36:51.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.159+0000 7f0d5ca00640 1 -- 172.21.15.59:0/3445886605 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d54031600 con 0x7f0d58071d70 2024-04-23T14:36:51.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.159+0000 7f0d5ca00640 1 -- 172.21.15.59:0/3445886605 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540319d0 con 0x7f0d58071d70 2024-04-23T14:36:51.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 -- 172.21.15.59:0/3445886605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 msgr2=0x7f0d58072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:51.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 --2- 172.21.15.59:0/3445886605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d58072200 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009e50 tx=0x7f0d5402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 -- 172.21.15.59:0/3445886605 shutdown_connections 2024-04-23T14:36:51.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 --2- 172.21.15.59:0/3445886605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d58072200 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 --2- 172.21.15.59:0/3445886605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5815e790 0x7f0d580aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 -- 172.21.15.59:0/3445886605 >> 172.21.15.59:0/3445886605 conn(0x7f0d58067d90 msgr2=0x7f0d5806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:51.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 -- 172.21.15.59:0/3445886605 shutdown_connections 2024-04-23T14:36:51.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.160+0000 7f0d60800640 1 -- 172.21.15.59:0/3445886605 wait complete. 2024-04-23T14:36:51.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.162+0000 7f0d60800640 1 Processor -- start 2024-04-23T14:36:51.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.162+0000 7f0d60800640 1 -- start start 2024-04-23T14:36:51.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d60800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5806f6f0 0x7f0d5815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d60800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d580b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d60800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d5815e790 0x7f0d5814b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d60800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5806fbb0 con 0x7f0d58071d70 2024-04-23T14:36:51.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d60800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580ae4e0 con 0x7f0d5815e790 2024-04-23T14:36:51.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d60800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5815eb70 con 0x7f0d5806f6f0 2024-04-23T14:36:51.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d5d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d580b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:51.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d5d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d580b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36714/0 (socket says 172.21.15.59:36714) 2024-04-23T14:36:51.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.163+0000 7f0d5d400640 1 -- 172.21.15.59:0/541333641 learned_addr learned my addr 172.21.15.59:0/541333641 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5e800640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d5815e790 0x7f0d5814b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5de00640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5806f6f0 0x7f0d5815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5d400640 1 -- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5806f6f0 msgr2=0x7f0d5815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5d400640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5806f6f0 0x7f0d5815fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5d400640 1 -- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d5815e790 msgr2=0x7f0d5814b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5d400640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d5815e790 0x7f0d5814b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5d400640 1 -- 172.21.15.59:0/541333641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d580b08f0 con 0x7f0d58071d70 2024-04-23T14:36:51.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5de00640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5806f6f0 0x7f0d5815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:51.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d5d400640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d580b00a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800b3f0 tx=0x7f0d4800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:51.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d52000640 1 -- 172.21.15.59:0/541333641 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48015020 con 0x7f0d58071d70 2024-04-23T14:36:51.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d52000640 1 -- 172.21.15.59:0/541333641 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d48011830 con 0x7f0d58071d70 2024-04-23T14:36:51.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d52000640 1 -- 172.21.15.59:0/541333641 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48011c20 con 0x7f0d58071d70 2024-04-23T14:36:51.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.164+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d580b0bd0 con 0x7f0d58071d70 2024-04-23T14:36:51.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.165+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d580b1190 con 0x7f0d58071d70 2024-04-23T14:36:51.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.166+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d58068a80 con 0x7f0d58071d70 2024-04-23T14:36:51.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.167+0000 7f0d52000640 1 -- 172.21.15.59:0/541333641 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d48018020 con 0x7f0d58071d70 2024-04-23T14:36:51.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.168+0000 7f0d52000640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d3c06c8f0 0x7f0d3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.168+0000 7f0d52000640 1 -- 172.21.15.59:0/541333641 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f0d4808b870 con 0x7f0d58071d70 2024-04-23T14:36:51.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.168+0000 7f0d5de00640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d3c06c8f0 0x7f0d3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:51.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.169+0000 7f0d5de00640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d3c06c8f0 0x7f0d3c06edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006fd0 tx=0x7f0d4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:51.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.172+0000 7f0d52000640 1 -- 172.21.15.59:0/541333641 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d4801f030 con 0x7f0d58071d70 2024-04-23T14:36:51.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.419+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f0d581e9720 con 0x7f0d58071d70 2024-04-23T14:36:51.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.836+0000 7f0d52000640 1 -- 172.21.15.59:0/541333641 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v45) ==== 162+0+0 (secure 0 0 0) 0x7f0d480172c0 con 0x7f0d58071d70 2024-04-23T14:36:51.838 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-23T14:36:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d3c06c8f0 msgr2=0x7f0d3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d3c06c8f0 0x7f0d3c06edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006fd0 tx=0x7f0d4c008040 comp rx=0 tx=0).stop 2024-04-23T14:36:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 msgr2=0x7f0d580b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d580b00a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800b3f0 tx=0x7f0d4800b8b0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 shutdown_connections 2024-04-23T14:36:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0d3c06c8f0 0x7f0d3c06edb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d5815e790 0x7f0d5814b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d58071d70 0x7f0d580b00a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 --2- 172.21.15.59:0/541333641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d5806f6f0 0x7f0d5815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 >> 172.21.15.59:0/541333641 conn(0x7f0d58067d90 msgr2=0x7f0d5815c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 shutdown_connections 2024-04-23T14:36:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:51.840+0000 7f0d60800640 1 -- 172.21.15.59:0/541333641 wait complete. 2024-04-23T14:36:51.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.045+0000 7f98c2600640 1 Processor -- start 2024-04-23T14:36:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.045+0000 7f98c2600640 1 -- start start 2024-04-23T14:36:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.046+0000 7f98c2600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98bc0afaf0 0x7f98bc0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:52.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.046+0000 7f98c2600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98bc15f0f0 con 0x7f98bc15c5e0 2024-04-23T14:36:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.046+0000 7f98c2600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98bc15f3d0 con 0x7f98bc0d06e0 2024-04-23T14:36:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.046+0000 7f98c2600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f98bc15fa20 con 0x7f98bc0afaf0 2024-04-23T14:36:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.046+0000 7f98b3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98bc0afaf0 0x7f98bc0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.046+0000 7f98bb400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f98bc15c5e0 0x7f98bc0af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54320/0 (socket says 172.21.15.59:54320) 2024-04-23T14:36:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.047+0000 7f98bb400640 1 -- 172.21.15.59:0/63571131 learned_addr learned my addr 172.21.15.59:0/63571131 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:52.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.047+0000 7f98b3e00640 1 -- 172.21.15.59:0/63571131 >> v1:172.21.15.76:6789/0 conn(0x7f98bc0d06e0 legacy=0x7f98bc1624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.047+0000 7f98b3e00640 1 -- 172.21.15.59:0/63571131 >> v1:172.21.15.59:6789/0 conn(0x7f98bc15c5e0 legacy=0x7f98bc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.047+0000 7f98b3e00640 1 -- 172.21.15.59:0/63571131 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc15f6b0 con 0x7f98bc0afaf0 2024-04-23T14:36:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.047+0000 7f98baa00640 1 -- 172.21.15.59:0/63571131 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724819945 0 0) 0x7f98bc15f0f0 con 0x7f98bc15c5e0 2024-04-23T14:36:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.048+0000 7f98b3e00640 1 --2- 172.21.15.59:0/63571131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc0afaf0 0x7f98bc0d01a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f98a40097a0 tx=0x7f98a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152e20a13a086a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.048+0000 7f98baa00640 1 -- 172.21.15.59:0/63571131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4037070 con 0x7f98bc0afaf0 2024-04-23T14:36:52.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.048+0000 7f98baa00640 1 -- 172.21.15.59:0/63571131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a4032440 con 0x7f98bc0afaf0 2024-04-23T14:36:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.048+0000 7f98baa00640 1 -- 172.21.15.59:0/63571131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4032810 con 0x7f98bc0afaf0 2024-04-23T14:36:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.049+0000 7f98c2600640 1 -- 172.21.15.59:0/63571131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc0afaf0 msgr2=0x7f98bc0d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.049+0000 7f98c2600640 1 --2- 172.21.15.59:0/63571131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc0afaf0 0x7f98bc0d01a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f98a40097a0 tx=0x7f98a402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.049+0000 7f98c2600640 1 -- 172.21.15.59:0/63571131 shutdown_connections 2024-04-23T14:36:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.049+0000 7f98c2600640 1 --2- 172.21.15.59:0/63571131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc0afaf0 0x7f98bc0d01a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.049+0000 7f98c2600640 1 -- 172.21.15.59:0/63571131 >> 172.21.15.59:0/63571131 conn(0x7f98bc067ea0 msgr2=0x7f98bc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.049+0000 7f98c2600640 1 -- 172.21.15.59:0/63571131 shutdown_connections 2024-04-23T14:36:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.049+0000 7f98c2600640 1 -- 172.21.15.59:0/63571131 wait complete. 2024-04-23T14:36:52.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.051+0000 7f98c2600640 1 Processor -- start 2024-04-23T14:36:52.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.051+0000 7f98c2600640 1 -- start start 2024-04-23T14:36:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98c2600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98bc0afaf0 0x7f98bc1f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98c2600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98bc0d06e0 0x7f98bc14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98c2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc15c5e0 0x7f98bc0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98c2600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc06e600 con 0x7f98bc15c5e0 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98c2600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc06e480 con 0x7f98bc0d06e0 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98c2600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc162be0 con 0x7f98bc0afaf0 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc15c5e0 0x7f98bc0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98bbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc15c5e0 0x7f98bc0cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36744/0 (socket says 172.21.15.59:36744) 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98bbe00640 1 -- 172.21.15.59:0/833064174 learned_addr learned my addr 172.21.15.59:0/833064174 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.052+0000 7f98b3e00640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98bc0d06e0 0x7f98bc14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98bb400640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98bc0afaf0 0x7f98bc1f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98bbe00640 1 -- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98bc0afaf0 msgr2=0x7f98bc1f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98bbe00640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98bc0afaf0 0x7f98bc1f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98bbe00640 1 -- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98bc0d06e0 msgr2=0x7f98bc14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98bbe00640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98bc0d06e0 0x7f98bc14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98bbe00640 1 -- 172.21.15.59:0/833064174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc14e230 con 0x7f98bc15c5e0 2024-04-23T14:36:52.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98bbe00640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc15c5e0 0x7f98bc0cfd90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c780 tx=0x7f98b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98b3400640 1 -- 172.21.15.59:0/833064174 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4017070 con 0x7f98bc15c5e0 2024-04-23T14:36:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98b3400640 1 -- 172.21.15.59:0/833064174 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b4003da0 con 0x7f98bc15c5e0 2024-04-23T14:36:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98b3400640 1 -- 172.21.15.59:0/833064174 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40052b0 con 0x7f98bc15c5e0 2024-04-23T14:36:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.053+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc0d02d0 con 0x7f98bc15c5e0 2024-04-23T14:36:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.054+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc158d60 con 0x7f98bc15c5e0 2024-04-23T14:36:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.055+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc068a80 con 0x7f98bc15c5e0 2024-04-23T14:36:52.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.056+0000 7f98b3400640 1 -- 172.21.15.59:0/833064174 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98b4005450 con 0x7f98bc15c5e0 2024-04-23T14:36:52.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.056+0000 7f98b3400640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f989806c8f0 0x7f989806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:52.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.057+0000 7f98b3400640 1 -- 172.21.15.59:0/833064174 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f98b408b1d0 con 0x7f98bc15c5e0 2024-04-23T14:36:52.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.057+0000 7f98bb400640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f989806c8f0 0x7f989806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:52.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.057+0000 7f98bb400640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f989806c8f0 0x7f989806edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f98ac000f60 tx=0x7f98ac007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:52.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.061+0000 7f98b3400640 1 -- 172.21.15.59:0/833064174 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98b4059470 con 0x7f98bc15c5e0 2024-04-23T14:36:52.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.300+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98bc06bb00 con 0x7f98bc15c5e0 2024-04-23T14:36:52.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.301+0000 7f98b3400640 1 -- 172.21.15.59:0/833064174 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5177 (secure 0 0 0) 0x7f98b4058d70 con 0x7f98bc15c5e0 2024-04-23T14:36:52.302 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:52.302 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:36:51.841139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-23T14:36:52.302 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 46 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f989806c8f0 msgr2=0x7f989806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f989806c8f0 0x7f989806edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f98ac000f60 tx=0x7f98ac007c00 comp rx=0 tx=0).stop 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc15c5e0 msgr2=0x7f98bc0cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc15c5e0 0x7f98bc0cfd90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c780 tx=0x7f98b400cc40 comp rx=0 tx=0).stop 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 shutdown_connections 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f989806c8f0 0x7f989806edb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98bc15c5e0 0x7f98bc0cfd90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98bc0d06e0 0x7f98bc14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 --2- 172.21.15.59:0/833064174 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98bc0afaf0 0x7f98bc1f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 >> 172.21.15.59:0/833064174 conn(0x7f98bc067ea0 msgr2=0x7f98bc1610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 shutdown_connections 2024-04-23T14:36:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:52.304+0000 7f98c2600640 1 -- 172.21.15.59:0/833064174 wait complete. 2024-04-23T14:36:52.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:36:29.710826+0000', 'modified': '2024-04-23T14:36:51.841139+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7008}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7008': {'gid': 7008, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6836/1773251382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 1773251382}, {'type': 'v1', 'addr': '172.21.15.59:6836', 'nonce': 1773251382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:36:52.723 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7008:up:replay 2024-04-23T14:36:53.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:53.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.939+0000 7f7aee800640 1 Processor -- start 2024-04-23T14:36:53.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.939+0000 7f7aee800640 1 -- start start 2024-04-23T14:36:53.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.940+0000 7f7aee800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ae80cd760 0x7f7ae80b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:53.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.940+0000 7f7aee800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80b4690 0x7f7ae80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:53.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.940+0000 7f7aee800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae8160210 con 0x7f7ae815c3d0 2024-04-23T14:36:53.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.940+0000 7f7aee800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae81604f0 con 0x7f7ae80cd760 2024-04-23T14:36:53.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.940+0000 7f7aee800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae81607d0 con 0x7f7ae80b4690 2024-04-23T14:36:53.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.941+0000 7f7aeca00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ae80cd760 0x7f7ae80b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:53.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.941+0000 7f7aede00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80b4690 0x7f7ae80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:53.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.941+0000 7f7aeca00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ae80cd760 0x7f7ae80b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37770/0 (socket says 172.21.15.59:37770) 2024-04-23T14:36:53.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.941+0000 7f7aeca00640 1 -- 172.21.15.59:0/3485363131 learned_addr learned my addr 172.21.15.59:0/3485363131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.942+0000 7f7aeca00640 1 -- 172.21.15.59:0/3485363131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80b4690 msgr2=0x7f7ae80cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.942+0000 7f7aeca00640 1 --2- 172.21.15.59:0/3485363131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80b4690 0x7f7ae80cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.942+0000 7f7aeca00640 1 -- 172.21.15.59:0/3485363131 >> v1:172.21.15.76:6789/0 conn(0x7f7ae815c3d0 legacy=0x7f7ae80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:53.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.942+0000 7f7aeca00640 1 -- 172.21.15.59:0/3485363131 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae8160ab0 con 0x7f7ae80cd760 2024-04-23T14:36:53.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.943+0000 7f7aeca00640 1 --2- 172.21.15.59:0/3485363131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae80cd760 0x7f7ae80b4150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009870 tx=0x7f7ad402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff36106c7020a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:53.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7ae7e00640 1 -- 172.21.15.59:0/3485363131 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4037070 con 0x7f7ae80cd760 2024-04-23T14:36:53.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7ae7e00640 1 -- 172.21.15.59:0/3485363131 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4032440 con 0x7f7ae80cd760 2024-04-23T14:36:53.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7ae7e00640 1 -- 172.21.15.59:0/3485363131 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4032810 con 0x7f7ae80cd760 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 -- 172.21.15.59:0/3485363131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae80cd760 msgr2=0x7f7ae80b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 --2- 172.21.15.59:0/3485363131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae80cd760 0x7f7ae80b4150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009870 tx=0x7f7ad402a250 comp rx=0 tx=0).stop 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 -- 172.21.15.59:0/3485363131 shutdown_connections 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 --2- 172.21.15.59:0/3485363131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80b4690 0x7f7ae80cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 --2- 172.21.15.59:0/3485363131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae80cd760 0x7f7ae80b4150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 -- 172.21.15.59:0/3485363131 >> 172.21.15.59:0/3485363131 conn(0x7f7ae80b21a0 msgr2=0x7f7ae80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 -- 172.21.15.59:0/3485363131 shutdown_connections 2024-04-23T14:36:53.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.944+0000 7f7aee800640 1 -- 172.21.15.59:0/3485363131 wait complete. 2024-04-23T14:36:53.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.946+0000 7f7aee800640 1 Processor -- start 2024-04-23T14:36:53.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.946+0000 7f7aee800640 1 -- start start 2024-04-23T14:36:53.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.947+0000 7f7aee800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ae80b4690 0x7f7ae80c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:53.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.947+0000 7f7aee800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80cd760 0x7f7ae815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:53.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.947+0000 7f7aee800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae815c3d0 0x7f7ae80c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:53.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.947+0000 7f7aee800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae815c7b0 con 0x7f7ae80b4690 2024-04-23T14:36:53.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.947+0000 7f7aee800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae80b4b00 con 0x7f7ae815c3d0 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.947+0000 7f7aee800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae815c930 con 0x7f7ae80cd760 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ae80b4690 0x7f7ae80c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ae80b4690 0x7f7ae80c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36756/0 (socket says 172.21.15.59:36756) 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 -- 172.21.15.59:0/117540716 learned_addr learned my addr 172.21.15.59:0/117540716 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aede00640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae815c3d0 0x7f7ae80c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aeca00640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80cd760 0x7f7ae815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 -- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80cd760 msgr2=0x7f7ae815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80cd760 0x7f7ae815da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 -- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae815c3d0 msgr2=0x7f7ae80c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae815c3d0 0x7f7ae80c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 -- 172.21.15.59:0/117540716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae815e430 con 0x7f7ae80b4690 2024-04-23T14:36:53.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.948+0000 7f7aed400640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ae80b4690 0x7f7ae80c8c70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c3c0 tx=0x7f7adc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:53.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.949+0000 7f7ae5600640 1 -- 172.21.15.59:0/117540716 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0052d0 con 0x7f7ae80b4690 2024-04-23T14:36:53.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.949+0000 7f7ae5600640 1 -- 172.21.15.59:0/117540716 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc005490 con 0x7f7ae80b4690 2024-04-23T14:36:53.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.949+0000 7f7ae5600640 1 -- 172.21.15.59:0/117540716 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc007260 con 0x7f7ae80b4690 2024-04-23T14:36:53.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.949+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae815e710 con 0x7f7ae80b4690 2024-04-23T14:36:53.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.949+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae80c91b0 con 0x7f7ae80b4690 2024-04-23T14:36:53.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.950+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae804fd90 con 0x7f7ae80b4690 2024-04-23T14:36:53.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.951+0000 7f7ae5600640 1 -- 172.21.15.59:0/117540716 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7adc007400 con 0x7f7ae80b4690 2024-04-23T14:36:53.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.952+0000 7f7ae5600640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ac806c8f0 0x7f7ac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:53.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.952+0000 7f7ae5600640 1 -- 172.21.15.59:0/117540716 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f7adc08b1e0 con 0x7f7ae80b4690 2024-04-23T14:36:53.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.952+0000 7f7aeca00640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ac806c8f0 0x7f7ac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:53.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.953+0000 7f7aeca00640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ac806c8f0 0x7f7ac806edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4034000 tx=0x7f7ad402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:53.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:53.956+0000 7f7ae5600640 1 -- 172.21.15.59:0/117540716 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7adc059480 con 0x7f7ae80b4690 2024-04-23T14:36:54.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.189+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ae806e400 con 0x7f7ae80b4690 2024-04-23T14:36:54.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.190+0000 7f7ae5600640 1 -- 172.21.15.59:0/117540716 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5177 (secure 0 0 0) 0x7f7adc058d80 con 0x7f7ae80b4690 2024-04-23T14:36:54.192 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:54.192 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:36:52.859363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-23T14:36:54.192 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 48 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ac806c8f0 msgr2=0x7f7ac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ac806c8f0 0x7f7ac806edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4034000 tx=0x7f7ad402e000 comp rx=0 tx=0).stop 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ae80b4690 msgr2=0x7f7ae80c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ae80b4690 0x7f7ae80c8c70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c3c0 tx=0x7f7adc00c880 comp rx=0 tx=0).stop 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 shutdown_connections 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ac806c8f0 0x7f7ac806edb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ae80cd760 0x7f7ae815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ae815c3d0 0x7f7ae80c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 --2- 172.21.15.59:0/117540716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ae80b4690 0x7f7ae80c8c70 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 >> 172.21.15.59:0/117540716 conn(0x7f7ae80b21a0 msgr2=0x7f7ae8071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 shutdown_connections 2024-04-23T14:36:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:54.193+0000 7f7aee800640 1 -- 172.21.15.59:0/117540716 wait complete. 2024-04-23T14:36:54.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:36:29.710826+0000', 'modified': '2024-04-23T14:36:52.859363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7008}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7008': {'gid': 7008, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.59:6836/1773251382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 1773251382}, {'type': 'v1', 'addr': '172.21.15.59:6836', 'nonce': 1773251382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:36:54.616 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7008:up:rejoin 2024-04-23T14:36:55.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:36:55.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.829+0000 7f6e43200640 1 Processor -- start 2024-04-23T14:36:55.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.830+0000 7f6e43200640 1 -- start start 2024-04-23T14:36:55.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.831+0000 7f6e43200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6e3c15c3d0 0x7f6e3c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:55.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.831+0000 7f6e43200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c0cd590 0x7f6e3c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:55.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.831+0000 7f6e43200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e3c160210 con 0x7f6e3c0b4690 2024-04-23T14:36:55.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.831+0000 7f6e43200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6e3c1604f0 con 0x7f6e3c15c3d0 2024-04-23T14:36:55.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.831+0000 7f6e43200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c1607d0 con 0x7f6e3c0cd590 2024-04-23T14:36:55.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.831+0000 7f6e41e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6e3c15c3d0 0x7f6e3c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:55.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.831+0000 7f6e41e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6e3c15c3d0 0x7f6e3c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36772/0 (socket says 172.21.15.59:36772) 2024-04-23T14:36:55.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e41e00640 1 -- 172.21.15.59:0/1139073719 learned_addr learned my addr 172.21.15.59:0/1139073719 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e41e00640 1 -- 172.21.15.59:0/1139073719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c0cd590 msgr2=0x7f6e3c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e41400640 1 --2- 172.21.15.59:0/1139073719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c0cd590 0x7f6e3c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e41e00640 1 --2- 172.21.15.59:0/1139073719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c0cd590 0x7f6e3c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e41e00640 1 -- 172.21.15.59:0/1139073719 >> v1:172.21.15.59:6789/0 conn(0x7f6e3c0b4690 legacy=0x7f6e3c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e41e00640 1 -- 172.21.15.59:0/1139073719 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c160ab0 con 0x7f6e3c15c3d0 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e41400640 1 --2- 172.21.15.59:0/1139073719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c0cd590 0x7f6e3c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.832+0000 7f6e40a00640 1 -- 172.21.15.59:0/1139073719 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 616155077 0 0) 0x7f6e3c160210 con 0x7f6e3c0b4690 2024-04-23T14:36:55.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.833+0000 7f6e41e00640 1 --2- 172.21.15.59:0/1139073719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c15c3d0 0x7f6e3c0cd050 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009870 tx=0x7f6e3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffe1ce1083e8fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:55.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e40a00640 1 -- 172.21.15.59:0/1139073719 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30037070 con 0x7f6e3c15c3d0 2024-04-23T14:36:55.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e40a00640 1 -- 172.21.15.59:0/1139073719 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e30032440 con 0x7f6e3c15c3d0 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e40a00640 1 -- 172.21.15.59:0/1139073719 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30032810 con 0x7f6e3c15c3d0 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 -- 172.21.15.59:0/1139073719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c15c3d0 msgr2=0x7f6e3c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 --2- 172.21.15.59:0/1139073719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c15c3d0 0x7f6e3c0cd050 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009870 tx=0x7f6e3002a270 comp rx=0 tx=0).stop 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 -- 172.21.15.59:0/1139073719 shutdown_connections 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 --2- 172.21.15.59:0/1139073719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c0cd590 0x7f6e3c0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 --2- 172.21.15.59:0/1139073719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c15c3d0 0x7f6e3c0cd050 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 -- 172.21.15.59:0/1139073719 >> 172.21.15.59:0/1139073719 conn(0x7f6e3c0b21a0 msgr2=0x7f6e3c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 -- 172.21.15.59:0/1139073719 shutdown_connections 2024-04-23T14:36:55.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.834+0000 7f6e43200640 1 -- 172.21.15.59:0/1139073719 wait complete. 2024-04-23T14:36:55.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.836+0000 7f6e43200640 1 Processor -- start 2024-04-23T14:36:55.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.837+0000 7f6e43200640 1 -- start start 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.837+0000 7f6e43200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e3c0b4690 0x7f6e3c0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e43200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c0cd590 0x7f6e3c1f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e43200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c15c3d0 0x7f6e3c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e43200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c15c7b0 con 0x7f6e3c0cd590 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e43200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0b4b00 con 0x7f6e3c0b4690 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e43200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c15c930 con 0x7f6e3c15c3d0 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c0cd590 0x7f6e3c1f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c0cd590 0x7f6e3c1f7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36784/0 (socket says 172.21.15.59:36784) 2024-04-23T14:36:55.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 -- 172.21.15.59:0/246552415 learned_addr learned my addr 172.21.15.59:0/246552415 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e42800640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c15c3d0 0x7f6e3c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41e00640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e3c0b4690 0x7f6e3c0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 -- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c15c3d0 msgr2=0x7f6e3c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c15c3d0 0x7f6e3c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 -- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e3c0b4690 msgr2=0x7f6e3c0c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e3c0b4690 0x7f6e3c0c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41400640 1 -- 172.21.15.59:0/246552415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c1f79a0 con 0x7f6e3c0cd590 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e42800640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c15c3d0 0x7f6e3c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.838+0000 7f6e41e00640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e3c0b4690 0x7f6e3c0c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.839+0000 7f6e41400640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c0cd590 0x7f6e3c1f7460 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00c7f0 tx=0x7f6e2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.839+0000 7f6e36000640 1 -- 172.21.15.59:0/246552415 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c015070 con 0x7f6e3c0cd590 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.839+0000 7f6e36000640 1 -- 172.21.15.59:0/246552415 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e2c007a30 con 0x7f6e3c0cd590 2024-04-23T14:36:55.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.839+0000 7f6e36000640 1 -- 172.21.15.59:0/246552415 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c007e00 con 0x7f6e3c0cd590 2024-04-23T14:36:55.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.839+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c1f7c40 con 0x7f6e3c0cd590 2024-04-23T14:36:55.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.839+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1f8200 con 0x7f6e3c0cd590 2024-04-23T14:36:55.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.840+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c04fd90 con 0x7f6e3c0cd590 2024-04-23T14:36:55.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.841+0000 7f6e36000640 1 -- 172.21.15.59:0/246552415 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e2c018070 con 0x7f6e3c0cd590 2024-04-23T14:36:55.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.842+0000 7f6e36000640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e2006c8f0 0x7f6e2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:55.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.842+0000 7f6e36000640 1 -- 172.21.15.59:0/246552415 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f6e2c08bb30 con 0x7f6e3c0cd590 2024-04-23T14:36:55.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.843+0000 7f6e41e00640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e2006c8f0 0x7f6e2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:55.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.843+0000 7f6e41e00640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e2006c8f0 0x7f6e2006edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e3002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:55.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:55.847+0000 7f6e36000640 1 -- 172.21.15.59:0/246552415 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e2c059ef0 con 0x7f6e3c0cd590 2024-04-23T14:36:56.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.098+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e3c06e400 con 0x7f6e3c0cd590 2024-04-23T14:36:56.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.100+0000 7f6e36000640 1 -- 172.21.15.59:0/246552415 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5184 (secure 0 0 0) 0x7f6e2c0597f0 con 0x7f6e3c0cd590 2024-04-23T14:36:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:36:56.101 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:36:54.866886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":5}]} 2024-04-23T14:36:56.101 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 49 2024-04-23T14:36:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e2006c8f0 msgr2=0x7f6e2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e2006c8f0 0x7f6e2006edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e3002e000 comp rx=0 tx=0).stop 2024-04-23T14:36:56.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c0cd590 msgr2=0x7f6e3c1f7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c0cd590 0x7f6e3c1f7460 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00c7f0 tx=0x7f6e2c00cbf0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 shutdown_connections 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e2006c8f0 0x7f6e2006edb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e3c15c3d0 0x7f6e3c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c0cd590 0x7f6e3c1f7460 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 --2- 172.21.15.59:0/246552415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e3c0b4690 0x7f6e3c0c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.102+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 >> 172.21.15.59:0/246552415 conn(0x7f6e3c0b21a0 msgr2=0x7f6e3c0d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.103+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 shutdown_connections 2024-04-23T14:36:56.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.103+0000 7f6e43200640 1 -- 172.21.15.59:0/246552415 wait complete. 2024-04-23T14:36:56.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:36:29.710826+0000', 'modified': '2024-04-23T14:36:54.866886+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7008}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7008': {'gid': 7008, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6836/1773251382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 1773251382}, {'type': 'v1', 'addr': '172.21.15.59:6836', 'nonce': 1773251382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7008, 'qdb_cluster': [7008]} 2024-04-23T14:36:56.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:36:56.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-23T14:36:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.743+0000 7f8f41800640 1 Processor -- start 2024-04-23T14:36:56.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.743+0000 7f8f41800640 1 -- start start 2024-04-23T14:36:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.744+0000 7f8f41800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 0x7f8f3c15f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.744+0000 7f8f41800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f3c15ba70 con 0x7f8f3c0d05c0 2024-04-23T14:36:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.744+0000 7f8f41800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f3c15f800 con 0x7f8f3c15c250 2024-04-23T14:36:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.744+0000 7f8f41800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c15bdc0 con 0x7f8f3c1416e0 2024-04-23T14:36:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.745+0000 7f8f3be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8f3c0d05c0 0x7f8f3c0d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54348/0 (socket says 172.21.15.59:54348) 2024-04-23T14:36:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.745+0000 7f8f3be00640 1 -- 172.21.15.59:0/1910563748 learned_addr learned my addr 172.21.15.59:0/1910563748 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.745+0000 7f8f33e00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 0x7f8f3c15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.745+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165052570 0 0) 0x7f8f3c15ba70 con 0x7f8f3c0d05c0 2024-04-23T14:36:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.745+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f20003520 con 0x7f8f3c0d05c0 2024-04-23T14:36:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3233920790 0 0) 0x7f8f20003520 con 0x7f8f3c0d05c0 2024-04-23T14:36:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 msgr2=0x7f8f3c15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 0x7f8f3c15f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 >> v1:172.21.15.76:6789/0 conn(0x7f8f3c15c250 legacy=0x7f8f3c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c15fae0 con 0x7f8f3c0d05c0 2024-04-23T14:36:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8f2c001670 con 0x7f8f3c0d05c0 2024-04-23T14:36:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 >> v1:172.21.15.59:6789/0 conn(0x7f8f3c0d05c0 legacy=0x7f8f3c0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 0x7f8f200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c15bdc0 con 0x7f8f200039e0 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2000baf0 con 0x7f8f20004800 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2000bcf0 con 0x7f8f3c1416e0 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f3be00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f33e00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.746+0000 7f8f40e00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 0x7f8f200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.747+0000 7f8f3be00640 1 -- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 msgr2=0x7f8f200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.747+0000 7f8f3be00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c1416e0 0x7f8f200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.747+0000 7f8f3be00640 1 -- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20004800 msgr2=0x7f8f20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.747+0000 7f8f3be00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.747+0000 7f8f3be00640 1 -- 172.21.15.59:0/1910563748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f20003520 con 0x7f8f200039e0 2024-04-23T14:36:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.747+0000 7f8f33e00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.748+0000 7f8f3be00640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f20003e20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c110 tx=0x7f8f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a233f17ff79b0292 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.748+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c007c40 con 0x7f8f200039e0 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.748+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c005ad0 con 0x7f8f200039e0 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.748+0000 7f8f3b400640 1 -- 172.21.15.59:0/1910563748 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c007070 con 0x7f8f200039e0 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.749+0000 7f8f41800640 1 -- 172.21.15.59:0/1910563748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 msgr2=0x7f8f20003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.749+0000 7f8f41800640 1 --2- 172.21.15.59:0/1910563748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f20003e20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c110 tx=0x7f8f20005830 comp rx=0 tx=0).stop 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.749+0000 7f8f40e00640 1 -- 172.21.15.59:0/1910563748 reap_dead start 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.749+0000 7f8f41800640 1 -- 172.21.15.59:0/1910563748 shutdown_connections 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.749+0000 7f8f41800640 1 -- 172.21.15.59:0/1910563748 >> 172.21.15.59:0/1910563748 conn(0x7f8f3c06bdf0 msgr2=0x7f8f3c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:56.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.749+0000 7f8f41800640 1 -- 172.21.15.59:0/1910563748 shutdown_connections 2024-04-23T14:36:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.749+0000 7f8f41800640 1 -- 172.21.15.59:0/1910563748 wait complete. 2024-04-23T14:36:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.751+0000 7f8f41800640 1 Processor -- start 2024-04-23T14:36:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.751+0000 7f8f41800640 1 -- start start 2024-04-23T14:36:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f41800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f3c1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f41800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f3c148a40 0x7f8f3c1e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f41800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c0ad990 0x7f8f3c0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f41800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c155ce0 con 0x7f8f200039e0 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f41800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c141b40 con 0x7f8f3c148a40 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f41800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c155e60 con 0x7f8f3c0ad990 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f3c1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f3c1482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36808/0 (socket says 172.21.15.59:36808) 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.752+0000 7f8f3be00640 1 -- 172.21.15.59:0/150391986 learned_addr learned my addr 172.21.15.59:0/150391986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f33e00640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f3c148a40 0x7f8f3c1e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f3be00640 1 -- 172.21.15.59:0/150391986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c0ad990 msgr2=0x7f8f3c0ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f3be00640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c0ad990 0x7f8f3c0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f3be00640 1 -- 172.21.15.59:0/150391986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f3c148a40 msgr2=0x7f8f3c1e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f3be00640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f3c148a40 0x7f8f3c1e5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f3be00640 1 -- 172.21.15.59:0/150391986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c1e61c0 con 0x7f8f200039e0 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f3be00640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f3c1482c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c230 tx=0x7f8f3c073610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.753+0000 7f8f38c00640 1 -- 172.21.15.59:0/150391986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0015b0 con 0x7f8f200039e0 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.754+0000 7f8f38c00640 1 -- 172.21.15.59:0/150391986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c038040 con 0x7f8f200039e0 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.754+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c1e64a0 con 0x7f8f200039e0 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.754+0000 7f8f38c00640 1 -- 172.21.15.59:0/150391986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c007290 con 0x7f8f200039e0 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.754+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c0afcb0 con 0x7f8f200039e0 2024-04-23T14:36:56.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.756+0000 7f8f38c00640 1 -- 172.21.15.59:0/150391986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f2c007430 con 0x7f8f200039e0 2024-04-23T14:36:56.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.757+0000 7f8f38c00640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f1806c920 0x7f8f1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.758+0000 7f8f38c00640 1 -- 172.21.15.59:0/150391986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f8f2c0abe80 con 0x7f8f200039e0 2024-04-23T14:36:56.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.758+0000 7f8f33e00640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f1806c920 0x7f8f1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.758+0000 7f8f45b78740 1 Processor -- start 2024-04-23T14:36:56.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.758+0000 7f8f33e00640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f1806c920 0x7f8f1806ede0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c148e60 tx=0x7f8f24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:56.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.758+0000 7f8f45b78740 1 -- start start 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.759+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6cdc70 0x5555ef6d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.759+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6a5980 0x5555ef6ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.759+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.759+0000 7f8f45b78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555ef60bb10 con 0x5555ef6cdc70 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.759+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555ef5b22b0 con 0x5555ef6ce590 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.759+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5555ef607ad0 con 0x5555ef6a5980 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6cdc70 0x5555ef6d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6cdc70 0x5555ef6d1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36814/0 (socket says 172.21.15.59:36814) 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 -- 172.21.15.59:0/332243848 learned_addr learned my addr 172.21.15.59:0/332243848 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f33e00640 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6a5980 0x5555ef6ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 -- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6a5980 msgr2=0x5555ef6ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f40e00640 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6a5980 0x5555ef6ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 -- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 msgr2=0x5555ef6d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6d0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 -- 172.21.15.59:0/332243848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ef31e2d0 con 0x5555ef6cdc70 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f33e00640 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6a5980 0x5555ef6ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.760+0000 7f8f3be00640 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6cdc70 0x5555ef6d1a70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0aa1f0 tx=0x7f8f2c0799b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436a824f7d79a53b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f30c00640 1 -- 172.21.15.59:0/332243848 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0a8070 con 0x5555ef6cdc70 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f30c00640 1 -- 172.21.15.59:0/332243848 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c092030 con 0x5555ef6cdc70 2024-04-23T14:36:56.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f30c00640 1 -- 172.21.15.59:0/332243848 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c07a1b0 con 0x5555ef6cdc70 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 -- 172.21.15.59:0/332243848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6cdc70 msgr2=0x5555ef6d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6cdc70 0x5555ef6d1a70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0aa1f0 tx=0x7f8f2c0799b0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 -- 172.21.15.59:0/332243848 shutdown_connections 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6a5980 0x5555ef6ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 --2- 172.21.15.59:0/332243848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6cdc70 0x5555ef6d1a70 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 -- 172.21.15.59:0/332243848 >> 172.21.15.59:0/332243848 conn(0x5555ef6a3580 msgr2=0x5555ef6ebe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:56.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.761+0000 7f8f45b78740 1 -- 172.21.15.59:0/332243848 shutdown_connections 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.762+0000 7f8f45b78740 1 -- 172.21.15.59:0/332243848 wait complete. 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.762+0000 7f8f45b78740 1 -- start start 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.762+0000 7f8f45b78740 1 Processor -- start 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.763+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.763+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.763+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6c2a40 0x5555ef6c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.763+0000 7f8f45b78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555ef5b22b0 con 0x5555ef6c2a40 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.763+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555ef5c5590 con 0x5555ef6ce590 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.763+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5555ef60bb10 con 0x5555ef6cdc70 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.763+0000 7f8f45b78740 10 client.0 _ll_register_callbacks cb 0x5555ef5a83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6c2a40 0x5555ef6c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6c2a40 0x5555ef6c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36822/0 (socket says 172.21.15.59:36822) 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 -- 172.21.15.59:0/653150717 learned_addr learned my addr 172.21.15.59:0/653150717 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:56.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f3be00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f33e00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 -- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 msgr2=0x5555ef6ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 -- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 msgr2=0x5555ef6c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f40e00640 1 -- 172.21.15.59:0/653150717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ef3e86b0 con 0x5555ef6c2a40 2024-04-23T14:36:56.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.764+0000 7f8f33e00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:56.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f40e00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6c2a40 0x5555ef6c4540 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c15bd30 tx=0x7f8f3400f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d000ced10a142143 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f29600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f29600640 1 -- 172.21.15.59:0/653150717 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003b80 con 0x5555ef6c2a40 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f29600640 1 -- 172.21.15.59:0/653150717 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f8f34003d20 con 0x5555ef6c2a40 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f2c092030 con 0x5555ef6c2a40 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f29600640 1 -- 172.21.15.59:0/653150717 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8f34020050 con 0x5555ef6c2a40 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f45b78740 10 client.7056 fetch_fsmap learned FSMap version 49 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x5555ef6c4de0 con 0x5555ef6c2a40 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.765+0000 7f8f29600640 1 -- 172.21.15.59:0/653150717 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c092030 con 0x5555ef6c2a40 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f29600640 1 -- 172.21.15.59:0/653150717 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f8f34006b50 con 0x5555ef6c2a40 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f45b78740 10 client.7056 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f45b78740 10 client.7056 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f45b78740 20 client.7056 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:56.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f45b78740 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6cc450 0x5555ef70e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f45b78740 4 client.7056 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ef619530 con 0x5555ef6cc450 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f3be00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6cc450 0x5555ef70e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.766+0000 7f8f3be00640 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6cc450 0x5555ef70e170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00a581b8e174265 server_cookie=b7ae700b5bc5512 in_seq=0 out_seq=0 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.767+0000 7f8f29600640 10 client.7056 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.767+0000 7f8f29600640 1 -- 172.21.15.59:0/653150717 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555ef619530 con 0x5555ef6cc450 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.767+0000 7f8f29600640 10 client.7056 handle_command_reply: tid=0 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.767+0000 7f8f45b78740 1 client.7056 shutdown 2024-04-23T14:36:56.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6c2a40 msgr2=0x5555ef6c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6c2a40 0x5555ef6c4540 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c15bd30 tx=0x7f8f3400f890 comp rx=0 tx=0).stop 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 shutdown_connections 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6cc450 0x5555ef70e170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6c2a40 0x5555ef6c4540 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6ce590 0x5555ef6c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 --2- 172.21.15.59:0/653150717 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f29600640 0 client.7056 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:36:56.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 >> 172.21.15.59:0/653150717 conn(0x5555ef6a5980 msgr2=0x5555ef6a00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:56.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 shutdown_connections 2024-04-23T14:36:56.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 1 -- 172.21.15.59:0/653150717 wait complete. 2024-04-23T14:36:56.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.768+0000 7f8f45b78740 20 client.7056 trim_cache size 0 max 16384 2024-04-23T14:36:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.788+0000 7f8f45b78740 1 Processor -- start 2024-04-23T14:36:56.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.788+0000 7f8f45b78740 1 -- start start 2024-04-23T14:36:56.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.789+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6c2a40 0x5555ef708df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.789+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.789+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.789+0000 7f8f45b78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c06f7e0 con 0x5555ef6ce590 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.789+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c0b1410 con 0x5555ef6c2a40 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.789+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c06f190 con 0x5555ef6cdc70 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6a62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36828/0 (socket says 172.21.15.59:36828) 2024-04-23T14:36:56.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 -- 172.21.15.59:0/392292344 learned_addr learned my addr 172.21.15.59:0/392292344 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f3be00640 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6c2a40 0x5555ef708df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f33e00640 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 -- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 msgr2=0x5555ef6a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 -- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6c2a40 msgr2=0x5555ef708df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6c2a40 0x5555ef708df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 -- 172.21.15.59:0/392292344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ef6edcb0 con 0x5555ef6ce590 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f33e00640 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:56.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f3be00640 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6c2a40 0x5555ef708df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.790+0000 7f8f40e00640 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6a62a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8f34008f20 tx=0x7f8f3400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5323fc197404de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f2be00640 1 -- 172.21.15.59:0/392292344 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003d80 con 0x5555ef6ce590 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f2be00640 1 -- 172.21.15.59:0/392292344 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f34018070 con 0x5555ef6ce590 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f2be00640 1 -- 172.21.15.59:0/392292344 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34005360 con 0x5555ef6ce590 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f45b78740 1 -- 172.21.15.59:0/392292344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 msgr2=0x5555ef6a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f45b78740 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6a62a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8f34008f20 tx=0x7f8f3400a000 comp rx=0 tx=0).stop 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f45b78740 1 -- 172.21.15.59:0/392292344 shutdown_connections 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f45b78740 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6cdc70 0x5555ef6a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f45b78740 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6a62a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f45b78740 1 --2- 172.21.15.59:0/392292344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6c2a40 0x5555ef708df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.791+0000 7f8f45b78740 1 -- 172.21.15.59:0/392292344 >> 172.21.15.59:0/392292344 conn(0x5555ef6a5980 msgr2=0x5555ef6c22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:56.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.792+0000 7f8f45b78740 1 -- 172.21.15.59:0/392292344 shutdown_connections 2024-04-23T14:36:56.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.792+0000 7f8f45b78740 1 -- 172.21.15.59:0/392292344 wait complete. 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.792+0000 7f8f45b78740 1 -- start start 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.793+0000 7f8f45b78740 1 Processor -- start 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6c2a40 0x5555ef6d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6cdc70 0x5555ef7693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f45b78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f45b78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555ef6cc2d0 con 0x5555ef6ce590 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555ef6cbe00 con 0x5555ef6cdc70 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f45b78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5555ef6cc920 con 0x5555ef6c2a40 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f45b78740 10 client.0 _ll_register_callbacks cb 0x7f8f34001f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36830/0 (socket says 172.21.15.59:36830) 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f40e00640 1 -- 172.21.15.59:0/964505510 learned_addr learned my addr 172.21.15.59:0/964505510 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f3be00640 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6c2a40 0x5555ef6d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f33e00640 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6cdc70 0x5555ef7693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f40e00640 1 -- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6c2a40 msgr2=0x5555ef6d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f40e00640 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6c2a40 0x5555ef6d44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f40e00640 1 -- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6cdc70 msgr2=0x5555ef7693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.794+0000 7f8f40e00640 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6cdc70 0x5555ef7693a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f40e00640 1 -- 172.21.15.59:0/964505510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ef6c39f0 con 0x5555ef6ce590 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f40e00640 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6e25b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8f340020a0 tx=0x7f8f340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8712287d55f9bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f29600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f29600640 1 -- 172.21.15.59:0/964505510 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34005320 con 0x5555ef6ce590 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f29600640 1 -- 172.21.15.59:0/964505510 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f8f340054c0 con 0x5555ef6ce590 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f34018070 con 0x5555ef6ce590 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f29600640 1 -- 172.21.15.59:0/964505510 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8f3403f050 con 0x5555ef6ce590 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f45b78740 10 client.7062 fetch_fsmap learned FSMap version 49 2024-04-23T14:36:56.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.795+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x5555ef7698e0 con 0x5555ef6ce590 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f29600640 1 -- 172.21.15.59:0/964505510 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f34018070 con 0x5555ef6ce590 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f29600640 1 -- 172.21.15.59:0/964505510 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f8f34042020 con 0x5555ef6ce590 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f45b78740 10 client.7062 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f45b78740 10 client.7062 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f45b78740 20 client.7062 populate_metadata read hostname 'smithi059' 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f45b78740 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6a6420 0x5555ef6beca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f45b78740 4 client.7062 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "status"}) -- 0x5555ef6133a0 con 0x5555ef6a6420 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.796+0000 7f8f3be00640 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6a6420 0x5555ef6beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f3be00640 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6a6420 0x5555ef6beca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4db1f61c868acf9 server_cookie=58c6224e0cd53634 in_seq=0 out_seq=0 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f29600640 10 client.7062 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f29600640 1 -- 172.21.15.59:0/964505510 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5555ef6133a0 con 0x5555ef6a6420 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f29600640 10 client.7062 handle_command_reply: tid=0 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 client.7062 shutdown 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 msgr2=0x5555ef6e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6e25b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8f340020a0 tx=0x7f8f340014a0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 shutdown_connections 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5555ef6a6420 0x5555ef6beca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555ef6ce590 0x5555ef6e25b0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555ef6cdc70 0x5555ef7693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 --2- 172.21.15.59:0/964505510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555ef6c2a40 0x5555ef6d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f29600640 0 client.7062 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 >> 172.21.15.59:0/964505510 conn(0x5555ef6a5980 msgr2=0x5555ef708cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 shutdown_connections 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.797+0000 7f8f45b78740 1 -- 172.21.15.59:0/964505510 wait complete. 2024-04-23T14:36:56.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f45b78740 20 client.7062 trim_cache size 0 max 16384 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7008, 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 4.9481670219999998, 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 49, 2024-04-23T14:36:56.803 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 70, 2024-04-23T14:36:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 70, 2024-04-23T14:36:56.804 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 5.8617780509999999 2024-04-23T14:36:56.804 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:36:56.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f1806c920 msgr2=0x7f8f1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f1806c920 0x7f8f1806ede0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c148e60 tx=0x7f8f24008040 comp rx=0 tx=0).stop 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 msgr2=0x7f8f3c1482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f3c1482c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c230 tx=0x7f8f3c073610 comp rx=0 tx=0).stop 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 shutdown_connections 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f1806c920 0x7f8f1806ede0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f3c0ad990 0x7f8f3c0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f3c148a40 0x7f8f3c1e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 --2- 172.21.15.59:0/150391986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f200039e0 0x7f8f3c1482c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.798+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 >> 172.21.15.59:0/150391986 conn(0x7f8f3c06bdf0 msgr2=0x7f8f3c1554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.799+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 shutdown_connections 2024-04-23T14:36:56.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:56.799+0000 7f8f41800640 1 -- 172.21.15.59:0/150391986 wait complete. 2024-04-23T14:36:56.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:36:56.853 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:36:56.852+0000 7f4f35000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:36:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.020+0000 7f1816800640 1 Processor -- start 2024-04-23T14:36:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.020+0000 7f1816800640 1 -- start start 2024-04-23T14:36:57.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.022+0000 7f1816800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f18100b5ff0 0x7f18100b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.022+0000 7f1816800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18100cf030 con 0x7f181015e7c0 2024-04-23T14:36:57.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.022+0000 7f1816800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18100b6a40 con 0x7f18100d0d90 2024-04-23T14:36:57.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.022+0000 7f1816800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f18100b7000 con 0x7f18100b5ff0 2024-04-23T14:36:57.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.022+0000 7f1815400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f181015e7c0 0x7f18100d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54352/0 (socket says 172.21.15.59:54352) 2024-04-23T14:36:57.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.022+0000 7f1815400640 1 -- 172.21.15.59:0/2405264527 learned_addr learned my addr 172.21.15.59:0/2405264527 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:36:57.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f1814a00640 1 --2- 172.21.15.59:0/2405264527 >> v2:172.21.15.76:3300/0 conn(0x7f18100b5ff0 0x7f18100b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4066627335 0 0) 0x7f1804001110 con 0x7f181015e7c0 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17f8003520 con 0x7f181015e7c0 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3246996611 0 0) 0x7f17f8003520 con 0x7f181015e7c0 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 >> v2:172.21.15.76:3300/0 conn(0x7f18100b5ff0 msgr2=0x7f18100b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 --2- 172.21.15.59:0/2405264527 >> v2:172.21.15.76:3300/0 conn(0x7f18100b5ff0 0x7f18100b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 >> v1:172.21.15.76:6789/0 conn(0x7f18100d0d90 legacy=0x7f18100cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18100b6d20 con 0x7f181015e7c0 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1804001840 con 0x7f181015e7c0 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 >> v1:172.21.15.59:6789/0 conn(0x7f181015e7c0 legacy=0x7f18100d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.023+0000 7f180fe00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f80039b0 0x7f17f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f180fe00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f8004800 0x7f17f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f180fe00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f17f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18100b7000 con 0x7f17f80051c0 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17f800c010 con 0x7f17f80039b0 2024-04-23T14:36:57.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f17f800c210 con 0x7f17f8004800 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1815e00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f17f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1815400640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f80039b0 0x7f17f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1814a00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f8004800 0x7f17f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1815e00640 1 -- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f8004800 msgr2=0x7f17f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1815e00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f8004800 0x7f17f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1815e00640 1 -- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f80039b0 msgr2=0x7f17f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1815e00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f80039b0 0x7f17f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.024+0000 7f1815e00640 1 -- 172.21.15.59:0/2405264527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18100b6d20 con 0x7f17f80051c0 2024-04-23T14:36:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.025+0000 7f1815e00640 1 -- 172.21.15.59:0/2405264527 reap_dead start 2024-04-23T14:36:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.025+0000 7f1815e00640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f17f800b9c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1808001e50 tx=0x7f17f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bec872e7b97756 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.026+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808001a70 con 0x7f17f80051c0 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.026+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1808003970 con 0x7f17f80051c0 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.026+0000 7f180fe00640 1 -- 172.21.15.59:0/2405264527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808003d60 con 0x7f17f80051c0 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.026+0000 7f1816800640 1 -- 172.21.15.59:0/2405264527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 msgr2=0x7f17f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.027+0000 7f1816800640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f17f800b9c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1808001e50 tx=0x7f17f8005830 comp rx=0 tx=0).stop 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.027+0000 7f1816800640 1 -- 172.21.15.59:0/2405264527 shutdown_connections 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.027+0000 7f1816800640 1 --2- 172.21.15.59:0/2405264527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f17f800b9c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.027+0000 7f1816800640 1 -- 172.21.15.59:0/2405264527 >> 172.21.15.59:0/2405264527 conn(0x7f181009d920 msgr2=0x7f181008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.027+0000 7f1816800640 1 -- 172.21.15.59:0/2405264527 shutdown_connections 2024-04-23T14:36:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.027+0000 7f1816800640 1 -- 172.21.15.59:0/2405264527 wait complete. 2024-04-23T14:36:57.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.029+0000 7f1816800640 1 Processor -- start 2024-04-23T14:36:57.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.030+0000 7f1816800640 1 -- start start 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1816800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f181015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1816800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18101529c0 0x7f181015f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1816800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18100b21f0 0x7f18100b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1816800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18100d1a80 con 0x7f17f80051c0 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1816800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f18100cf620 con 0x7f18100b21f0 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1816800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f18100d1c00 con 0x7f18101529c0 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1815400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f181015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1815400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f181015edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36844/0 (socket says 172.21.15.59:36844) 2024-04-23T14:36:57.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.031+0000 7f1815400640 1 -- 172.21.15.59:0/3481722177 learned_addr learned my addr 172.21.15.59:0/3481722177 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1814a00640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18101529c0 0x7f181015f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1815400640 1 -- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18101529c0 msgr2=0x7f181015f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1815e00640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18100b21f0 0x7f18100b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1815400640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18101529c0 0x7f181015f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1815400640 1 -- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18100b21f0 msgr2=0x7f18100b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1815400640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18100b21f0 0x7f18100b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1815400640 1 -- 172.21.15.59:0/3481722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1810161430 con 0x7f17f80051c0 2024-04-23T14:36:57.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.032+0000 7f1815400640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f181015edb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f180400e8c0 tx=0x7f180400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:57.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.033+0000 7f180d600640 1 -- 172.21.15.59:0/3481722177 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400c870 con 0x7f17f80051c0 2024-04-23T14:36:57.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.033+0000 7f180d600640 1 -- 172.21.15.59:0/3481722177 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1804018070 con 0x7f17f80051c0 2024-04-23T14:36:57.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.033+0000 7f180d600640 1 -- 172.21.15.59:0/3481722177 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400cc50 con 0x7f17f80051c0 2024-04-23T14:36:57.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.033+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1810161710 con 0x7f17f80051c0 2024-04-23T14:36:57.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.033+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18100cd210 con 0x7f17f80051c0 2024-04-23T14:36:57.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.034+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181004fd90 con 0x7f17f80051c0 2024-04-23T14:36:57.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.035+0000 7f180d600640 1 -- 172.21.15.59:0/3481722177 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f180401d460 con 0x7f17f80051c0 2024-04-23T14:36:57.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.036+0000 7f180d600640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f17f006c8f0 0x7f17f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:57.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.036+0000 7f180d600640 1 -- 172.21.15.59:0/3481722177 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f180408bae0 con 0x7f17f80051c0 2024-04-23T14:36:57.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.036+0000 7f1814a00640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f17f006c8f0 0x7f17f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:57.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.037+0000 7f1814a00640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f17f006c8f0 0x7f17f006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f181006e6f0 tx=0x7f17fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.041+0000 7f180d600640 1 -- 172.21.15.59:0/3481722177 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18100cf030 con 0x7f17f80051c0 2024-04-23T14:36:57.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.287+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f181015d480 con 0x7f17f80051c0 2024-04-23T14:36:57.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.287+0000 7f180d600640 1 -- 172.21.15.59:0/3481722177 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7f1804059f40 con 0x7f17f80051c0 2024-04-23T14:36:57.288 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:36:57.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f17f006c8f0 msgr2=0x7f17f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:57.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f17f006c8f0 0x7f17f006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f181006e6f0 tx=0x7f17fc008040 comp rx=0 tx=0).stop 2024-04-23T14:36:57.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 msgr2=0x7f181015edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:57.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f181015edb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f180400e8c0 tx=0x7f180400ed80 comp rx=0 tx=0).stop 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 shutdown_connections 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f17f006c8f0 0x7f17f006edb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18101529c0 0x7f181015f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18100b21f0 0x7f18100b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 --2- 172.21.15.59:0/3481722177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f80051c0 0x7f181015edb0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 >> 172.21.15.59:0/3481722177 conn(0x7f181009d920 msgr2=0x7f181006dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 shutdown_connections 2024-04-23T14:36:57.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57.290+0000 7f1816800640 1 -- 172.21.15.59:0/3481722177 wait complete. 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:36:57.311 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:36:57.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:57.311 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:36:57.410 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:36:57.410 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:36:57.410 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:36:57.410 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:57.410 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:36:57.410 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:57.410 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42131sec preferred_lft 42131sec 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:57.411 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:36:57.412 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:36:57.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:57.413 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:36:57.500 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:36:57.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:57.504 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:36:57.560 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:36:57.560 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:36:57.560 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:36:57.560 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:36:57.560 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:36:57.561 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:36:57.562 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:36:57.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:36:57.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:36:57.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:36:57.562 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:36:57.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:36:57.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:36:57.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:57.788 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:36:57.836 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:36:57.836 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:36:57.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:36:57.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:57.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:36:57.919 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:36:57.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:36:57.919 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:57.975 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:36:57.975 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:36:57.976 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:58.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:36:58.116 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:36:58.174 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:36:58.175 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:58.176 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:58.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:58.177 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:36:58.232 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:36:58.232 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:36:58.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:36:58.312 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:36:58.357 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:36:58.357 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:36:58.357 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:36:58.357 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:36:58.357 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:36:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:58.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:58.360 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:58.360 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:58.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:58.361 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:36:58.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:36:58.419+0000 7f4dd9922480 -1 init, newargv = 0x56117df27490 newargc=15 2024-04-23T14:36:58.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[59144]: starting ceph client 2024-04-23T14:36:58.428 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[59144]: starting fuse 2024-04-23T14:36:59.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:36:59.603 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:36:59.665 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:36:59.666 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:36:59.667 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:36:59.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:59.669 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:36:59.723 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:36:59.724 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:36:59.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:36:59.724 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:36:59.724 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:36:59.724 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:36:59.724 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:36:59.724 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:36:59.725 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:36:59.726 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:36:59.726 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:36:59.726 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:36:59.902 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.59144.asok 2024-04-23T14:36:59.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:36:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:36:59.921 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.59144.asok 2024-04-23T14:36:59.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:36:59.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.59144.asok status 2024-04-23T14:37:00.156 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "59144", 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:37:00.157 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7074, 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7074 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2181881594 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:37:00.158 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2181881594 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7074 192.168.0.1:0/2181881594", 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2181881594", 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 49, 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 70, 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:37:00.159 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:37:00.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:00.172 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:37:00.172 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:37:00.230 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:37:00.230 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:37:00.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:00.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:37:00.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:00.322 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-23T14:37:00.388 INFO:teuthology.orchestra.run.smithi059.stdout:file_undamaged 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> import json 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> import sys 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> try: 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> except OSError as e: 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> sys.exit(e.errno) 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> print(json.dumps( 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-23T14:37:00.391 DEBUG:teuthology.orchestra.run.smithi059:> indent=2)) 2024-04-23T14:37:00.392 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:37:00.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-23T14:37:00.561 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-23T14:37:00.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:00.561 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-23T14:37:00.666 INFO:teuthology.orchestra.run.smithi059.stdout:2 2024-04-23T14:37:00.666 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:37:00.666 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:37:00.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:37:00.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.920+0000 7f82ece00640 1 Processor -- start 2024-04-23T14:37:00.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.920+0000 7f82ece00640 1 -- start start 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82ece00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e815c3d0 0x7f82e80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82ece00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f82e80b4860 0x7f82e80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82ece00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e8160210 con 0x7f82e80cd590 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82ece00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f82e80cd9f0 con 0x7f82e80b4860 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82ece00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f82e81604f0 con 0x7f82e815c3d0 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82e7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f82e80b4860 0x7f82e80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82e7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f82e80b4860 0x7f82e80cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52534/0 (socket says 172.21.15.59:52534) 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.922+0000 7f82e7e00640 1 -- 172.21.15.59:0/3464470714 learned_addr learned my addr 172.21.15.59:0/3464470714 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.923+0000 7f82e7400640 1 --2- 172.21.15.59:0/3464470714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e815c3d0 0x7f82e80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:00.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.923+0000 7f82e7e00640 1 -- 172.21.15.59:0/3464470714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e815c3d0 msgr2=0x7f82e80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:00.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.923+0000 7f82e7e00640 1 --2- 172.21.15.59:0/3464470714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e815c3d0 0x7f82e80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:00.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.923+0000 7f82e7e00640 1 -- 172.21.15.59:0/3464470714 >> v1:172.21.15.76:6789/0 conn(0x7f82e80cd590 legacy=0x7f82e80b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:00.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.923+0000 7f82e7e00640 1 -- 172.21.15.59:0/3464470714 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e81607d0 con 0x7f82e80b4860 2024-04-23T14:37:00.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.924+0000 7f82e7e00640 1 --2- 172.21.15.59:0/3464470714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e80b4860 0x7f82e80cb7d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009e50 tx=0x7f82e002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9d69ebd2b7d501 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:00.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.924+0000 7f82e6a00640 1 -- 172.21.15.59:0/3464470714 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0036070 con 0x7f82e80b4860 2024-04-23T14:37:00.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.924+0000 7f82e6a00640 1 -- 172.21.15.59:0/3464470714 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e0031600 con 0x7f82e80b4860 2024-04-23T14:37:00.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.924+0000 7f82e6a00640 1 -- 172.21.15.59:0/3464470714 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00319d0 con 0x7f82e80b4860 2024-04-23T14:37:00.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 -- 172.21.15.59:0/3464470714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e80b4860 msgr2=0x7f82e80cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:00.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 --2- 172.21.15.59:0/3464470714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e80b4860 0x7f82e80cb7d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009e50 tx=0x7f82e002a6c0 comp rx=0 tx=0).stop 2024-04-23T14:37:00.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 -- 172.21.15.59:0/3464470714 shutdown_connections 2024-04-23T14:37:00.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 --2- 172.21.15.59:0/3464470714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e80b4860 0x7f82e80cb7d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:00.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 --2- 172.21.15.59:0/3464470714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e815c3d0 0x7f82e80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:00.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 -- 172.21.15.59:0/3464470714 >> 172.21.15.59:0/3464470714 conn(0x7f82e80b21a0 msgr2=0x7f82e80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:00.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 -- 172.21.15.59:0/3464470714 shutdown_connections 2024-04-23T14:37:00.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.925+0000 7f82ece00640 1 -- 172.21.15.59:0/3464470714 wait complete. 2024-04-23T14:37:00.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.928+0000 7f82ece00640 1 Processor -- start 2024-04-23T14:37:00.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.928+0000 7f82ece00640 1 -- start start 2024-04-23T14:37:00.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.929+0000 7f82ece00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82e80b4860 0x7f82e8159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:00.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.929+0000 7f82ece00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e80cd590 0x7f82e80724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:00.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.929+0000 7f82ece00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e815c3d0 0x7f82e80add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:00.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.929+0000 7f82ece00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82e815c7b0 con 0x7f82e815c3d0 2024-04-23T14:37:00.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.929+0000 7f82ece00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f82e8160d90 con 0x7f82e80b4860 2024-04-23T14:37:00.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.929+0000 7f82ece00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f82e815c930 con 0x7f82e80cd590 2024-04-23T14:37:00.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e815c3d0 0x7f82e80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e815c3d0 0x7f82e80add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52542/0 (socket says 172.21.15.59:52542) 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 -- 172.21.15.59:0/1665520187 learned_addr learned my addr 172.21.15.59:0/1665520187 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7400640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82e80b4860 0x7f82e8159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82dfe00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e80cd590 0x7f82e80724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 -- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e80cd590 msgr2=0x7f82e80724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e80cd590 0x7f82e80724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 -- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82e80b4860 msgr2=0x7f82e8159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82e80b4860 0x7f82e8159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82e7e00640 1 -- 172.21.15.59:0/1665520187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e8072a30 con 0x7f82e815c3d0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.930+0000 7f82dfe00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e80cd590 0x7f82e80724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.931+0000 7f82e7e00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e815c3d0 0x7f82e80add80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009e50 tx=0x7f82e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.931+0000 7f82df400640 1 -- 172.21.15.59:0/1665520187 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0036050 con 0x7f82e815c3d0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.931+0000 7f82df400640 1 -- 172.21.15.59:0/1665520187 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e0042070 con 0x7f82e815c3d0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.931+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e8072cd0 con 0x7f82e815c3d0 2024-04-23T14:37:00.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.931+0000 7f82df400640 1 -- 172.21.15.59:0/1665520187 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0009510 con 0x7f82e815c3d0 2024-04-23T14:37:00.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.931+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e8073290 con 0x7f82e815c3d0 2024-04-23T14:37:00.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.932+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e804fd90 con 0x7f82e815c3d0 2024-04-23T14:37:00.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.933+0000 7f82df400640 1 -- 172.21.15.59:0/1665520187 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82e0007bd0 con 0x7f82e815c3d0 2024-04-23T14:37:00.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.935+0000 7f82df400640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f82c406c8f0 0x7f82c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:00.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.935+0000 7f82df400640 1 -- 172.21.15.59:0/1665520187 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f82e00acb40 con 0x7f82e815c3d0 2024-04-23T14:37:00.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.935+0000 7f82e7400640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f82c406c8f0 0x7f82c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:00.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.936+0000 7f82e7400640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f82c406c8f0 0x7f82c406edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f82d8006fd0 tx=0x7f82d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:00.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:00.939+0000 7f82df400640 1 -- 172.21.15.59:0/1665520187 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82e0045360 con 0x7f82e815c3d0 2024-04-23T14:37:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.145+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82e806e400 con 0x7f82e815c3d0 2024-04-23T14:37:01.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.148+0000 7f82df400640 1 -- 172.21.15.59:0/1665520187 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16737 (secure 0 0 0) 0x7f82e007ae70 con 0x7f82e815c3d0 2024-04-23T14:37:01.150 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:37:01.150 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":70,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:50.829874+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/2883460093":"2024-04-24T14:36:50.829845+0000","172.21.15.59:6834/2883460093":"2024-04-24T14:36:50.829845+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:37:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f82c406c8f0 msgr2=0x7f82c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f82c406c8f0 0x7f82c406edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f82d8006fd0 tx=0x7f82d8008040 comp rx=0 tx=0).stop 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e815c3d0 msgr2=0x7f82e80add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e815c3d0 0x7f82e80add80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009e50 tx=0x7f82e0004a40 comp rx=0 tx=0).stop 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 shutdown_connections 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f82c406c8f0 0x7f82c406edb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82e80cd590 0x7f82e80724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82e815c3d0 0x7f82e80add80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 --2- 172.21.15.59:0/1665520187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82e80b4860 0x7f82e8159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 >> 172.21.15.59:0/1665520187 conn(0x7f82e80b21a0 msgr2=0x7f82e80af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 shutdown_connections 2024-04-23T14:37:01.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.151+0000 7f82ece00640 1 -- 172.21.15.59:0/1665520187 wait complete. 2024-04-23T14:37:01.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:36:27.526201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:36:28.678324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:37:01.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:37:01.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.388+0000 7fa51a000640 1 Processor -- start 2024-04-23T14:37:01.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.388+0000 7fa51a000640 1 -- start start 2024-04-23T14:37:01.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.389+0000 7fa51a000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa51415d810 0x7fa514160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.389+0000 7fa51a000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa51415cab0 con 0x7fa5140d0d10 2024-04-23T14:37:01.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.390+0000 7fa51a000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa514160e40 con 0x7fa5141567e0 2024-04-23T14:37:01.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.390+0000 7fa51a000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa51415cd30 con 0x7fa51415d810 2024-04-23T14:37:01.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.390+0000 7fa513e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa5140d0d10 0x7fa514160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56306/0 (socket says 172.21.15.59:56306) 2024-04-23T14:37:01.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.390+0000 7fa513e00640 1 -- 172.21.15.59:0/3952210851 learned_addr learned my addr 172.21.15.59:0/3952210851 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:01.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.390+0000 7fa519600640 1 --2- 172.21.15.59:0/3952210851 >> v2:172.21.15.76:3300/0 conn(0x7fa51415d810 0x7fa514160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.390+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68486946 0 0) 0x7fa500001110 con 0x7fa5140d0d10 2024-04-23T14:37:01.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.390+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4fc003520 con 0x7fa5140d0d10 2024-04-23T14:37:01.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 478836944 0 0) 0x7fa4fc003520 con 0x7fa5140d0d10 2024-04-23T14:37:01.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 >> v2:172.21.15.76:3300/0 conn(0x7fa51415d810 msgr2=0x7fa514160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 --2- 172.21.15.59:0/3952210851 >> v2:172.21.15.76:3300/0 conn(0x7fa51415d810 0x7fa514160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 >> v1:172.21.15.76:6789/0 conn(0x7fa5141567e0 legacy=0x7fa5140d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51415d010 con 0x7fa5140d0d10 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa500001840 con 0x7fa5140d0d10 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 >> v1:172.21.15.59:6789/0 conn(0x7fa5140d0d10 legacy=0x7fa514160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4fc0039b0 0x7fa4fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4fc004800 0x7fa4fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0051c0 0x7fa4fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa51415cd30 con 0x7fa4fc0039b0 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc00c060 con 0x7fa4fc0051c0 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc00c260 con 0x7fa4fc004800 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.391+0000 7fa518c00640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4fc0039b0 0x7fa4fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa519600640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0051c0 0x7fa4fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa518c00640 1 -- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4fc004800 msgr2=0x7fa4fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa518c00640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4fc004800 0x7fa4fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa518c00640 1 -- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0051c0 msgr2=0x7fa4fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa518c00640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0051c0 0x7fa4fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa518c00640 1 -- 172.21.15.59:0/3952210851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc003520 con 0x7fa4fc0039b0 2024-04-23T14:37:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa519600640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0051c0 0x7fa4fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:01.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.392+0000 7fa519600640 1 -- 172.21.15.59:0/3952210851 reap_dead start 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.393+0000 7fa518c00640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4fc0039b0 0x7fa4fc003dd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa508001750 tx=0x7fa4fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d96f17dcb37b59d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.393+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508001370 con 0x7fa4fc0039b0 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.393+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5080052d0 con 0x7fa4fc0039b0 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.393+0000 7fa513400640 1 -- 172.21.15.59:0/3952210851 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50802abc0 con 0x7fa4fc0039b0 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.394+0000 7fa51a000640 1 -- 172.21.15.59:0/3952210851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4fc0039b0 msgr2=0x7fa4fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.394+0000 7fa51a000640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4fc0039b0 0x7fa4fc003dd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa508001750 tx=0x7fa4fc005830 comp rx=0 tx=0).stop 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.394+0000 7fa51a000640 1 -- 172.21.15.59:0/3952210851 shutdown_connections 2024-04-23T14:37:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.394+0000 7fa51a000640 1 --2- 172.21.15.59:0/3952210851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4fc0039b0 0x7fa4fc003dd0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.394+0000 7fa51a000640 1 -- 172.21.15.59:0/3952210851 >> 172.21.15.59:0/3952210851 conn(0x7fa51406bdf0 msgr2=0x7fa51406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.394+0000 7fa51a000640 1 -- 172.21.15.59:0/3952210851 shutdown_connections 2024-04-23T14:37:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.394+0000 7fa51a000640 1 -- 172.21.15.59:0/3952210851 wait complete. 2024-04-23T14:37:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.396+0000 7fa51a000640 1 Processor -- start 2024-04-23T14:37:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.396+0000 7fa51a000640 1 -- start start 2024-04-23T14:37:01.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa51a000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0039b0 0x7fa514159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa51a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514099990 0x7fa514159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa51a000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5140b0e30 0x7fa5140b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa51a000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa514156d40 con 0x7fa514099990 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa51a000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa514156bc0 con 0x7fa4fc0039b0 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa51a000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa514156ec0 con 0x7fa5140b0e30 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa513e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514099990 0x7fa514159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa513e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514099990 0x7fa514159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52550/0 (socket says 172.21.15.59:52550) 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa513e00640 1 -- 172.21.15.59:0/342237256 learned_addr learned my addr 172.21.15.59:0/342237256 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.397+0000 7fa518c00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0039b0 0x7fa514159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa519600640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5140b0e30 0x7fa5140b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa513e00640 1 -- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5140b0e30 msgr2=0x7fa5140b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa513e00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5140b0e30 0x7fa5140b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa513e00640 1 -- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0039b0 msgr2=0x7fa514159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa513e00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0039b0 0x7fa514159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa513e00640 1 -- 172.21.15.59:0/342237256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa514157e40 con 0x7fa514099990 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa519600640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5140b0e30 0x7fa5140b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa518c00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0039b0 0x7fa514159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:01.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa513e00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514099990 0x7fa514159980 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa5000013b0 tx=0x7fa50000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa510c00640 1 -- 172.21.15.59:0/342237256 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500018070 con 0x7fa514099990 2024-04-23T14:37:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa510c00640 1 -- 172.21.15.59:0/342237256 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5000074e0 con 0x7fa514099990 2024-04-23T14:37:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa510c00640 1 -- 172.21.15.59:0/342237256 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500004a20 con 0x7fa514099990 2024-04-23T14:37:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.398+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa514158120 con 0x7fa514099990 2024-04-23T14:37:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.399+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141586e0 con 0x7fa514099990 2024-04-23T14:37:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.400+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5141588e0 con 0x7fa514099990 2024-04-23T14:37:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.401+0000 7fa510c00640 1 -- 172.21.15.59:0/342237256 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa500004bc0 con 0x7fa514099990 2024-04-23T14:37:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.401+0000 7fa510c00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4f406c920 0x7fa4f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.402+0000 7fa510c00640 1 -- 172.21.15.59:0/342237256 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7fa50008ae60 con 0x7fa514099990 2024-04-23T14:37:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.402+0000 7fa518c00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4f406c920 0x7fa4f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.402+0000 7fa518c00640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4f406c920 0x7fa4f406ede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa508032000 tx=0x7fa50802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.406+0000 7fa510c00640 1 -- 172.21.15.59:0/342237256 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5141588e0 con 0x7fa514099990 2024-04-23T14:37:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.609+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa51404fd90 con 0x7fa514099990 2024-04-23T14:37:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.610+0000 7fa510c00640 1 -- 172.21.15.59:0/342237256 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fa51415cab0 con 0x7fa514099990 2024-04-23T14:37:01.611 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:37:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4f406c920 msgr2=0x7fa4f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4f406c920 0x7fa4f406ede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa508032000 tx=0x7fa50802e000 comp rx=0 tx=0).stop 2024-04-23T14:37:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514099990 msgr2=0x7fa514159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514099990 0x7fa514159980 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa5000013b0 tx=0x7fa50000c9e0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 shutdown_connections 2024-04-23T14:37:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4f406c920 0x7fa4f406ede0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5140b0e30 0x7fa5140b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514099990 0x7fa514159980 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 --2- 172.21.15.59:0/342237256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4fc0039b0 0x7fa514159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 >> 172.21.15.59:0/342237256 conn(0x7fa51406bdf0 msgr2=0x7fa51415b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 shutdown_connections 2024-04-23T14:37:01.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.613+0000 7fa51a000640 1 -- 172.21.15.59:0/342237256 wait complete. 2024-04-23T14:37:01.633 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:37:01.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.837+0000 7fcaa8a00640 1 Processor -- start 2024-04-23T14:37:01.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.837+0000 7fcaa8a00640 1 -- start start 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fcaa8a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaa40d0600 0x7fcaa40d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fcaa8a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcaa40ae290 0x7fcaa40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fcaa8a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa415ef20 con 0x7fcaa4155a80 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fcaa8a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa415f1c0 con 0x7fcaa40d0600 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fcaa8a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa415f4a0 con 0x7fcaa40ae290 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fca9aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaa40d0600 0x7fcaa40d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fca9aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaa40d0600 0x7fcaa40d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52560/0 (socket says 172.21.15.59:52560) 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fca9aa00640 1 -- 172.21.15.59:0/1542378170 learned_addr learned my addr 172.21.15.59:0/1542378170 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.839+0000 7fcaa3e00640 1 --2- 172.21.15.59:0/1542378170 >> v2:172.21.15.76:3300/0 conn(0x7fcaa40ae290 0x7fcaa40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.840+0000 7fca9aa00640 1 -- 172.21.15.59:0/1542378170 >> v2:172.21.15.76:3300/0 conn(0x7fcaa40ae290 msgr2=0x7fcaa40ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.840+0000 7fca9aa00640 1 --2- 172.21.15.59:0/1542378170 >> v2:172.21.15.76:3300/0 conn(0x7fcaa40ae290 0x7fcaa40ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.840+0000 7fca9aa00640 1 -- 172.21.15.59:0/1542378170 >> v1:172.21.15.76:6789/0 conn(0x7fcaa4155a80 legacy=0x7fcaa406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.840+0000 7fca9aa00640 1 -- 172.21.15.59:0/1542378170 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa415f780 con 0x7fcaa40d0600 2024-04-23T14:37:01.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.840+0000 7fcaa3e00640 1 --2- 172.21.15.59:0/1542378170 >> v2:172.21.15.76:3300/0 conn(0x7fcaa40ae290 0x7fcaa40ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:01.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.841+0000 7fca9aa00640 1 --2- 172.21.15.59:0/1542378170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d0600 0x7fcaa40d0a90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406df60 tx=0x7fca8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f2ea28f20cfdbe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.841+0000 7fcaa2a00640 1 -- 172.21.15.59:0/1542378170 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c037070 con 0x7fcaa40d0600 2024-04-23T14:37:01.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.841+0000 7fcaa2a00640 1 -- 172.21.15.59:0/1542378170 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8c032440 con 0x7fcaa40d0600 2024-04-23T14:37:01.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.841+0000 7fcaa2a00640 1 -- 172.21.15.59:0/1542378170 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c032810 con 0x7fcaa40d0600 2024-04-23T14:37:01.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1542378170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d0600 msgr2=0x7fcaa40d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1542378170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d0600 0x7fcaa40d0a90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406df60 tx=0x7fca8c02a230 comp rx=0 tx=0).stop 2024-04-23T14:37:01.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1542378170 shutdown_connections 2024-04-23T14:37:01.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1542378170 >> v2:172.21.15.76:3300/0 conn(0x7fcaa40ae290 0x7fcaa40ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1542378170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40d0600 0x7fcaa40d0a90 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1542378170 >> 172.21.15.59:0/1542378170 conn(0x7fcaa406bdf0 msgr2=0x7fcaa406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:01.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1542378170 shutdown_connections 2024-04-23T14:37:01.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.842+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1542378170 wait complete. 2024-04-23T14:37:01.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.844+0000 7fcaa8a00640 1 Processor -- start 2024-04-23T14:37:01.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.844+0000 7fcaa8a00640 1 -- start start 2024-04-23T14:37:01.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.845+0000 7fcaa8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40ae290 0x7fcaa41481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.845+0000 7fcaa8a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa40d0600 0x7fcaa40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.845+0000 7fcaa8a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaa4155a80 0x7fcaa415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.845+0000 7fcaa8a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa406ef80 con 0x7fcaa40ae290 2024-04-23T14:37:01.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.845+0000 7fcaa8a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa415fd40 con 0x7fcaa40d0600 2024-04-23T14:37:01.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.845+0000 7fcaa8a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa406f100 con 0x7fcaa4155a80 2024-04-23T14:37:01.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.845+0000 7fcaa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40ae290 0x7fcaa41481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40ae290 0x7fcaa41481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52576/0 (socket says 172.21.15.59:52576) 2024-04-23T14:37:01.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 -- 172.21.15.59:0/1718413157 learned_addr learned my addr 172.21.15.59:0/1718413157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:01.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3e00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaa4155a80 0x7fcaa415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fca9aa00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa40d0600 0x7fcaa40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 -- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaa4155a80 msgr2=0x7fcaa415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaa4155a80 0x7fcaa415d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 -- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa40d0600 msgr2=0x7fcaa40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa40d0600 0x7fcaa40cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 -- 172.21.15.59:0/1718413157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa40b13a0 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.846+0000 7fcaa3400640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40ae290 0x7fcaa41481b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fca9400ba50 tx=0x7fca9400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.847+0000 7fca9be00640 1 -- 172.21.15.59:0/1718413157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94019070 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.847+0000 7fca9be00640 1 -- 172.21.15.59:0/1718413157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca9400cc20 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.847+0000 7fca9be00640 1 -- 172.21.15.59:0/1718413157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94014680 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.847+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa414c1a0 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.847+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa414c760 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.848+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa404fd90 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.849+0000 7fca9be00640 1 -- 172.21.15.59:0/1718413157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca9401c070 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.851+0000 7fca9be00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fca8006c920 0x7fca8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.851+0000 7fca9be00640 1 -- 172.21.15.59:0/1718413157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7fca9408b9b0 con 0x7fcaa40ae290 2024-04-23T14:37:01.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.851+0000 7fca9aa00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fca8006c920 0x7fca8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.852+0000 7fca9aa00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fca8006c920 0x7fca8006ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca8c034000 tx=0x7fca8c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:01.855+0000 7fca9be00640 1 -- 172.21.15.59:0/1718413157 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca94059600 con 0x7fcaa40ae290 2024-04-23T14:37:01.891 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:37:01.890+0000 7f3222800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-23T14:37:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.060+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcaa406eba0 con 0x7fcaa40ae290 2024-04-23T14:37:02.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.060+0000 7fca9be00640 1 -- 172.21.15.59:0/1718413157 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7fca9405d900 con 0x7fcaa40ae290 2024-04-23T14:37:02.062 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fca8006c920 msgr2=0x7fca8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fca8006c920 0x7fca8006ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca8c034000 tx=0x7fca8c02e000 comp rx=0 tx=0).stop 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40ae290 msgr2=0x7fcaa41481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40ae290 0x7fcaa41481b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fca9400ba50 tx=0x7fca9400bf10 comp rx=0 tx=0).stop 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 shutdown_connections 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fca8006c920 0x7fca8006ede0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaa4155a80 0x7fcaa415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaa40d0600 0x7fcaa40cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 --2- 172.21.15.59:0/1718413157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaa40ae290 0x7fcaa41481b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 >> 172.21.15.59:0/1718413157 conn(0x7fcaa406bdf0 msgr2=0x7fcaa40aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:02.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.063+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 shutdown_connections 2024-04-23T14:37:02.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.064+0000 7fcaa8a00640 1 -- 172.21.15.59:0/1718413157 wait complete. 2024-04-23T14:37:02.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:37:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.306+0000 7f1be2000640 1 Processor -- start 2024-04-23T14:37:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.307+0000 7f1be2000640 1 -- start start 2024-04-23T14:37:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1be2000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc155a80 0x7f1bdc06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1be2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1bdc0d0600 0x7f1bdc0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1be2000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bdc15ef20 con 0x7f1bdc0b43a0 2024-04-23T14:37:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1be2000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1bdc0d0fa0 con 0x7f1bdc0d0600 2024-04-23T14:37:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1be2000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1bdc15f1c0 con 0x7f1bdc155a80 2024-04-23T14:37:02.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1bdbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1bdc0d0600 0x7f1bdc0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1be1600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1bdc0b43a0 0x7f1bdc0b4830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56312/0 (socket says 172.21.15.59:56312) 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.308+0000 7f1be1600640 1 -- 172.21.15.59:0/3582865114 learned_addr learned my addr 172.21.15.59:0/3582865114 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.309+0000 7f1be0c00640 1 --2- 172.21.15.59:0/3582865114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc155a80 0x7f1bdc06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.309+0000 7f1bdbe00640 1 -- 172.21.15.59:0/3582865114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc155a80 msgr2=0x7f1bdc06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.309+0000 7f1bdbe00640 1 --2- 172.21.15.59:0/3582865114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc155a80 0x7f1bdc06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.309+0000 7f1bdbe00640 1 -- 172.21.15.59:0/3582865114 >> v1:172.21.15.59:6789/0 conn(0x7f1bdc0b43a0 legacy=0x7f1bdc0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.309+0000 7f1bdbe00640 1 -- 172.21.15.59:0/3582865114 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc15f4a0 con 0x7f1bdc0d0600 2024-04-23T14:37:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.309+0000 7f1bdb400640 1 -- 172.21.15.59:0/3582865114 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897058017 0 0) 0x7f1bdc15ef20 con 0x7f1bdc0b43a0 2024-04-23T14:37:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.310+0000 7f1bdbe00640 1 --2- 172.21.15.59:0/3582865114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc0d0600 0x7f1bdc0d0a60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80097a0 tx=0x7f1bc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e1f74928de8f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:02.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.310+0000 7f1bdb400640 1 -- 172.21.15.59:0/3582865114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037070 con 0x7f1bdc0d0600 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1bdb400640 1 -- 172.21.15.59:0/3582865114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8032440 con 0x7f1bdc0d0600 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1bdb400640 1 -- 172.21.15.59:0/3582865114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8032810 con 0x7f1bdc0d0600 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 -- 172.21.15.59:0/3582865114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc0d0600 msgr2=0x7f1bdc0d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 --2- 172.21.15.59:0/3582865114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc0d0600 0x7f1bdc0d0a60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80097a0 tx=0x7f1bc802a180 comp rx=0 tx=0).stop 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 -- 172.21.15.59:0/3582865114 shutdown_connections 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 --2- 172.21.15.59:0/3582865114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc0d0600 0x7f1bdc0d0a60 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 --2- 172.21.15.59:0/3582865114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc155a80 0x7f1bdc06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 -- 172.21.15.59:0/3582865114 >> 172.21.15.59:0/3582865114 conn(0x7f1bdc06bdf0 msgr2=0x7f1bdc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 -- 172.21.15.59:0/3582865114 shutdown_connections 2024-04-23T14:37:02.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.311+0000 7f1be2000640 1 -- 172.21.15.59:0/3582865114 wait complete. 2024-04-23T14:37:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.313+0000 7f1be2000640 1 Processor -- start 2024-04-23T14:37:02.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.313+0000 7f1be2000640 1 -- start start 2024-04-23T14:37:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.314+0000 7f1be2000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc0b43a0 0x7f1bdc0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.314+0000 7f1be2000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bdc0d0600 0x7f1bdc154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.314+0000 7f1be2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc155a80 0x7f1bdc1f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.314+0000 7f1be2000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc15fd40 con 0x7f1bdc155a80 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.314+0000 7f1be2000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc15fa60 con 0x7f1bdc0d0600 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.314+0000 7f1be2000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1bdc06ff00 con 0x7f1bdc0b43a0 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc155a80 0x7f1bdc1f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc155a80 0x7f1bdc1f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52582/0 (socket says 172.21.15.59:52582) 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 -- 172.21.15.59:0/3341650352 learned_addr learned my addr 172.21.15.59:0/3341650352 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be0c00640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc0b43a0 0x7f1bdc0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1bdbe00640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bdc0d0600 0x7f1bdc154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 -- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc0b43a0 msgr2=0x7f1bdc0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc0b43a0 0x7f1bdc0ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 -- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bdc0d0600 msgr2=0x7f1bdc154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bdc0d0600 0x7f1bdc154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.315+0000 7f1be1600640 1 -- 172.21.15.59:0/3341650352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc0b0130 con 0x7f1bdc155a80 2024-04-23T14:37:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.316+0000 7f1be1600640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc155a80 0x7f1bdc1f8300 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400c7a0 tx=0x7f1bd400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:02.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.316+0000 7f1bd8c00640 1 -- 172.21.15.59:0/3341650352 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4018070 con 0x7f1bdc155a80 2024-04-23T14:37:02.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.316+0000 7f1bd8c00640 1 -- 172.21.15.59:0/3341650352 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd400a640 con 0x7f1bdc155a80 2024-04-23T14:37:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.316+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc0b0470 con 0x7f1bdc155a80 2024-04-23T14:37:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.316+0000 7f1bd8c00640 1 -- 172.21.15.59:0/3341650352 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400aab0 con 0x7f1bdc155a80 2024-04-23T14:37:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.316+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc0b0cb0 con 0x7f1bdc155a80 2024-04-23T14:37:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.317+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc04fd90 con 0x7f1bdc155a80 2024-04-23T14:37:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.318+0000 7f1bd8c00640 1 -- 172.21.15.59:0/3341650352 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bd401b070 con 0x7f1bdc155a80 2024-04-23T14:37:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.319+0000 7f1bd8c00640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bbc06c8f0 0x7f1bbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.319+0000 7f1bd8c00640 1 -- 172.21.15.59:0/3341650352 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f1bd408b280 con 0x7f1bdc155a80 2024-04-23T14:37:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.320+0000 7f1be0c00640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bbc06c8f0 0x7f1bbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.320+0000 7f1be0c00640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bbc06c8f0 0x7f1bbc06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006fd0 tx=0x7f1bd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:02.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.324+0000 7f1bd8c00640 1 -- 172.21.15.59:0/3341650352 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bdc0b0470 con 0x7f1bdc155a80 2024-04-23T14:37:02.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.529+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1bdc0b0f40 con 0x7f1bdc155a80 2024-04-23T14:37:02.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.530+0000 7f1bd8c00640 1 -- 172.21.15.59:0/3341650352 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7f1bd40595b0 con 0x7f1bdc155a80 2024-04-23T14:37:02.531 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:37:02.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bbc06c8f0 msgr2=0x7f1bbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bbc06c8f0 0x7f1bbc06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006fd0 tx=0x7f1bd0008040 comp rx=0 tx=0).stop 2024-04-23T14:37:02.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc155a80 msgr2=0x7f1bdc1f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc155a80 0x7f1bdc1f8300 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400c7a0 tx=0x7f1bd400cc60 comp rx=0 tx=0).stop 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 shutdown_connections 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bbc06c8f0 0x7f1bbc06edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bdc155a80 0x7f1bdc1f8300 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bdc0d0600 0x7f1bdc154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 --2- 172.21.15.59:0/3341650352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bdc0b43a0 0x7f1bdc0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 >> 172.21.15.59:0/3341650352 conn(0x7f1bdc06bdf0 msgr2=0x7f1bdc072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 shutdown_connections 2024-04-23T14:37:02.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.533+0000 7f1be2000640 1 -- 172.21.15.59:0/3341650352 wait complete. 2024-04-23T14:37:02.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:37:02.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.768+0000 7f6609c00640 1 Processor -- start 2024-04-23T14:37:02.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.768+0000 7f6609c00640 1 -- start start 2024-04-23T14:37:02.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6609c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f66041416e0 0x7f660415f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6609c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f660415ba70 con 0x7f66040d05c0 2024-04-23T14:37:02.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6609c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f660415f800 con 0x7f660415c250 2024-04-23T14:37:02.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6609c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f660415bdc0 con 0x7f66041416e0 2024-04-23T14:37:02.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6603e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f66040d05c0 0x7f66040d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56320/0 (socket says 172.21.15.59:56320) 2024-04-23T14:37:02.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6603e00640 1 -- 172.21.15.59:0/2279904094 learned_addr learned my addr 172.21.15.59:0/2279904094 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:02.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6603400640 1 --2- 172.21.15.59:0/2279904094 >> v2:172.21.15.76:3300/0 conn(0x7f66041416e0 0x7f660415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.770+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167414525 0 0) 0x7f660415ba70 con 0x7f66040d05c0 2024-04-23T14:37:02.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65e8003520 con 0x7f66040d05c0 2024-04-23T14:37:02.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2063509262 0 0) 0x7f65e8003520 con 0x7f66040d05c0 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 >> v2:172.21.15.76:3300/0 conn(0x7f66041416e0 msgr2=0x7f660415f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 --2- 172.21.15.59:0/2279904094 >> v2:172.21.15.76:3300/0 conn(0x7f66041416e0 0x7f660415f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 >> v1:172.21.15.76:6789/0 conn(0x7f660415c250 legacy=0x7f66041555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660415fae0 con 0x7f66040d05c0 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f65f4001670 con 0x7f66040d05c0 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 >> v1:172.21.15.59:6789/0 conn(0x7f66040d05c0 legacy=0x7f66040d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65e80039b0 0x7f65e8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8004800 0x7f65e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8005210 0x7f65e800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f660415bdc0 con 0x7f65e8005210 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65e800e350 con 0x7f65e8004800 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f65e800e550 con 0x7f65e80039b0 2024-04-23T14:37:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.771+0000 7f6609200640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8005210 0x7f65e800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.772+0000 7f6603e00640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65e80039b0 0x7f65e8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.772+0000 7f6603400640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8004800 0x7f65e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.772+0000 7f6609200640 1 -- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65e80039b0 msgr2=0x7f65e8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.772+0000 7f6609200640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65e80039b0 0x7f65e8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.772+0000 7f6609200640 1 -- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8004800 msgr2=0x7f65e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.772+0000 7f6609200640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8004800 0x7f65e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.772+0000 7f6609200640 1 -- 172.21.15.59:0/2279904094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e8003520 con 0x7f65e8005210 2024-04-23T14:37:02.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.773+0000 7f6609200640 1 -- 172.21.15.59:0/2279904094 reap_dead start 2024-04-23T14:37:02.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.773+0000 7f6609200640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8005210 0x7f65e800dbd0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f65fc001e50 tx=0x7f65e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd69d4b4688e5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:02.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.773+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc001a70 con 0x7f65e8005210 2024-04-23T14:37:02.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.773+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc003970 con 0x7f65e8005210 2024-04-23T14:37:02.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.773+0000 7f6602a00640 1 -- 172.21.15.59:0/2279904094 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc003d60 con 0x7f65e8005210 2024-04-23T14:37:02.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.774+0000 7f6609c00640 1 -- 172.21.15.59:0/2279904094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8005210 msgr2=0x7f65e800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.774+0000 7f6609c00640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8005210 0x7f65e800dbd0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f65fc001e50 tx=0x7f65e8005830 comp rx=0 tx=0).stop 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.774+0000 7f6609c00640 1 -- 172.21.15.59:0/2279904094 shutdown_connections 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.774+0000 7f6609c00640 1 --2- 172.21.15.59:0/2279904094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e8005210 0x7f65e800dbd0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.774+0000 7f6609c00640 1 -- 172.21.15.59:0/2279904094 >> 172.21.15.59:0/2279904094 conn(0x7f660406bdf0 msgr2=0x7f660406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.774+0000 7f6609c00640 1 -- 172.21.15.59:0/2279904094 shutdown_connections 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.774+0000 7f6609c00640 1 -- 172.21.15.59:0/2279904094 wait complete. 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.776+0000 7f6609c00640 1 Processor -- start 2024-04-23T14:37:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.776+0000 7f6609c00640 1 -- start start 2024-04-23T14:37:02.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6609c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8005210 0x7f66041e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6609c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66040994b0 0x7f6604099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6609c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f660409cfa0 0x7f660409d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6609c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f660415c6e0 con 0x7f66040994b0 2024-04-23T14:37:02.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6609c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6604141b70 con 0x7f65e8005210 2024-04-23T14:37:02.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6609c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f660415c860 con 0x7f660409cfa0 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6603400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66040994b0 0x7f6604099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6603400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66040994b0 0x7f6604099930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52598/0 (socket says 172.21.15.59:52598) 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.777+0000 7f6603400640 1 -- 172.21.15.59:0/3563835230 learned_addr learned my addr 172.21.15.59:0/3563835230 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603e00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8005210 0x7f66041e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6609200640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f660409cfa0 0x7f660409d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603400640 1 -- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f660409cfa0 msgr2=0x7f660409d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603400640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f660409cfa0 0x7f660409d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603400640 1 -- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8005210 msgr2=0x7f66041e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603400640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8005210 0x7f66041e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603400640 1 -- 172.21.15.59:0/3563835230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6604099e70 con 0x7f66040994b0 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603e00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8005210 0x7f66041e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:02.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f6603400640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66040994b0 0x7f6604099930 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f65f000f580 tx=0x7f65f000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.778+0000 7f65fbe00640 1 -- 172.21.15.59:0/3563835230 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0013810 con 0x7f66040994b0 2024-04-23T14:37:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.779+0000 7f65fbe00640 1 -- 172.21.15.59:0/3563835230 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f001a070 con 0x7f66040994b0 2024-04-23T14:37:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.779+0000 7f65fbe00640 1 -- 172.21.15.59:0/3563835230 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0013bb0 con 0x7f66040994b0 2024-04-23T14:37:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.779+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66041e5350 con 0x7f66040994b0 2024-04-23T14:37:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.779+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66041e58f0 con 0x7f66040994b0 2024-04-23T14:37:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.780+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660404fd90 con 0x7f66040994b0 2024-04-23T14:37:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.781+0000 7f65fbe00640 1 -- 172.21.15.59:0/3563835230 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65f0012070 con 0x7f66040994b0 2024-04-23T14:37:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.782+0000 7f65fbe00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65e406c920 0x7f65e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.782+0000 7f65fbe00640 1 -- 172.21.15.59:0/3563835230 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f65f008bbf0 con 0x7f66040994b0 2024-04-23T14:37:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.782+0000 7f6603e00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65e406c920 0x7f65e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.783+0000 7f6603e00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65e406c920 0x7f65e406ede0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f65f4001200 tx=0x7f65f4001170 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:02.786+0000 7f65fbe00640 1 -- 172.21.15.59:0/3563835230 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65f0059f30 con 0x7f66040994b0 2024-04-23T14:37:03.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.001+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66040cfec0 con 0x7f66040994b0 2024-04-23T14:37:03.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.002+0000 7f65fbe00640 1 -- 172.21.15.59:0/3563835230 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) ==== 80+0+122 (secure 0 0 0) 0x7f65f0059830 con 0x7f66040994b0 2024-04-23T14:37:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6836/2883460093 2024-04-24T14:36:50.829845+0000 2024-04-23T14:37:03.003 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6834/2883460093 2024-04-24T14:36:50.829845+0000 2024-04-23T14:37:03.003 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.004+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65e406c920 msgr2=0x7f65e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.004+0000 7f6609c00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65e406c920 0x7f65e406ede0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f65f4001200 tx=0x7f65f4001170 comp rx=0 tx=0).stop 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.004+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66040994b0 msgr2=0x7f6604099930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.004+0000 7f6609c00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66040994b0 0x7f6604099930 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f65f000f580 tx=0x7f65f000fa40 comp rx=0 tx=0).stop 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 shutdown_connections 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f65e406c920 0x7f65e406ede0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f660409cfa0 0x7f660409d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66040994b0 0x7f6604099930 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 --2- 172.21.15.59:0/3563835230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65e8005210 0x7f66041e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 >> 172.21.15.59:0/3563835230 conn(0x7f660406bdf0 msgr2=0x7f66040c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:03.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 shutdown_connections 2024-04-23T14:37:03.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.005+0000 7f6609c00640 1 -- 172.21.15.59:0/3563835230 wait complete. 2024-04-23T14:37:03.024 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:37:03.024 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:37:03.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:37:03.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.326+0000 7f02a9000640 1 Processor -- start 2024-04-23T14:37:03.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.326+0000 7f02a9000640 1 -- start start 2024-04-23T14:37:03.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a9000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a40b5ff0 0x7f02a40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a9000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02a40d0d90 0x7f02a40d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a9000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a40cf030 con 0x7f02a415e7c0 2024-04-23T14:37:03.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a9000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f02a40b7000 con 0x7f02a40d0d90 2024-04-23T14:37:03.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a9000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f02a40cf620 con 0x7f02a40b5ff0 2024-04-23T14:37:03.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02a40d0d90 0x7f02a40d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f02a415e7c0 0x7f02a40d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56332/0 (socket says 172.21.15.59:56332) 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a3400640 1 -- 172.21.15.59:0/3463223432 learned_addr learned my addr 172.21.15.59:0/3463223432 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.328+0000 7f02a3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02a40d0d90 0x7f02a40d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52604/0 (socket says 172.21.15.59:52604) 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.329+0000 7f029aa00640 1 --2- 172.21.15.59:0/3463223432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a40b5ff0 0x7f02a40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.329+0000 7f02a3e00640 1 -- 172.21.15.59:0/3463223432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a40b5ff0 msgr2=0x7f02a40b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.329+0000 7f02a3e00640 1 --2- 172.21.15.59:0/3463223432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a40b5ff0 0x7f02a40b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.329+0000 7f02a3e00640 1 -- 172.21.15.59:0/3463223432 >> v1:172.21.15.59:6789/0 conn(0x7f02a415e7c0 legacy=0x7f02a40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.329+0000 7f02a3e00640 1 -- 172.21.15.59:0/3463223432 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a40b6a40 con 0x7f02a40d0d90 2024-04-23T14:37:03.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.330+0000 7f02a3e00640 1 --2- 172.21.15.59:0/3463223432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40d1220 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f029c009d90 tx=0x7f029c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1578dd6f8e406868 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:03.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.331+0000 7f02a2a00640 1 -- 172.21.15.59:0/3463223432 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c007740 con 0x7f02a40d0d90 2024-04-23T14:37:03.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.331+0000 7f02a2a00640 1 -- 172.21.15.59:0/3463223432 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c0078e0 con 0x7f02a40d0d90 2024-04-23T14:37:03.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.331+0000 7f02a2a00640 1 -- 172.21.15.59:0/3463223432 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c007cb0 con 0x7f02a40d0d90 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.331+0000 7f02a9000640 1 -- 172.21.15.59:0/3463223432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 msgr2=0x7f02a40d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.331+0000 7f02a9000640 1 --2- 172.21.15.59:0/3463223432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40d1220 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f029c009d90 tx=0x7f029c02a660 comp rx=0 tx=0).stop 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.332+0000 7f02a9000640 1 -- 172.21.15.59:0/3463223432 shutdown_connections 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.332+0000 7f02a9000640 1 --2- 172.21.15.59:0/3463223432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40d1220 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.332+0000 7f02a9000640 1 --2- 172.21.15.59:0/3463223432 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a40b5ff0 0x7f02a40b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.332+0000 7f02a9000640 1 -- 172.21.15.59:0/3463223432 >> 172.21.15.59:0/3463223432 conn(0x7f02a409d920 msgr2=0x7f02a408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.332+0000 7f02a9000640 1 -- 172.21.15.59:0/3463223432 shutdown_connections 2024-04-23T14:37:03.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.332+0000 7f02a9000640 1 -- 172.21.15.59:0/3463223432 wait complete. 2024-04-23T14:37:03.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.333+0000 7f02a9000640 1 Processor -- start 2024-04-23T14:37:03.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.334+0000 7f02a9000640 1 -- start start 2024-04-23T14:37:03.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a9000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a40b5ff0 0x7f02a41f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a9000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a415e7c0 0x7f02a4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a9000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f02a40d2800 con 0x7f02a40d0d90 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a9000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02a40cf2b0 con 0x7f02a40b5ff0 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a9000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f02a40d1a80 con 0x7f02a415e7c0 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52606/0 (socket says 172.21.15.59:52606) 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 -- 172.21.15.59:0/1890884043 learned_addr learned my addr 172.21.15.59:0/1890884043 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a3400640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a40b5ff0 0x7f02a41f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 -- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a415e7c0 msgr2=0x7f02a4159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a415e7c0 0x7f02a4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 -- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a40b5ff0 msgr2=0x7f02a41f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a40b5ff0 0x7f02a41f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f029aa00640 1 -- 172.21.15.59:0/1890884043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a41f7ef0 con 0x7f02a40d0d90 2024-04-23T14:37:03.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.335+0000 7f02a3400640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a40b5ff0 0x7f02a41f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:03.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.336+0000 7f029aa00640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40cfc20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d800 tx=0x7f028c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:03.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.336+0000 7f029be00640 1 -- 172.21.15.59:0/1890884043 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00b640 con 0x7f02a40d0d90 2024-04-23T14:37:03.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.336+0000 7f029be00640 1 -- 172.21.15.59:0/1890884043 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c019070 con 0x7f02a40d0d90 2024-04-23T14:37:03.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.336+0000 7f029be00640 1 -- 172.21.15.59:0/1890884043 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00ba00 con 0x7f02a40d0d90 2024-04-23T14:37:03.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.336+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a41f81d0 con 0x7f02a40d0d90 2024-04-23T14:37:03.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.336+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41f8790 con 0x7f02a40d0d90 2024-04-23T14:37:03.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.337+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a404fd90 con 0x7f02a40d0d90 2024-04-23T14:37:03.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.338+0000 7f029be00640 1 -- 172.21.15.59:0/1890884043 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f028c00bba0 con 0x7f02a40d0d90 2024-04-23T14:37:03.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.339+0000 7f029be00640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f028006c920 0x7f028006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.339+0000 7f029be00640 1 -- 172.21.15.59:0/1890884043 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f028c08bfd0 con 0x7f02a40d0d90 2024-04-23T14:37:03.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.339+0000 7f02a3400640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f028006c920 0x7f028006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.340+0000 7f02a3400640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f028006c920 0x7f028006ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0294000f60 tx=0x7f0294009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:03.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.343+0000 7f029be00640 1 -- 172.21.15.59:0/1890884043 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f028c05a390 con 0x7f02a40d0d90 2024-04-23T14:37:03.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.550+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02a415e090 con 0x7f02a40d0d90 2024-04-23T14:37:03.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.553+0000 7f029be00640 1 -- 172.21.15.59:0/1890884043 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16737 (secure 0 0 0) 0x7f028c059c90 con 0x7f02a40d0d90 2024-04-23T14:37:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:37:03.555 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":70,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:36:50.829874+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/2883460093":"2024-04-24T14:36:50.829845+0000","172.21.15.59:6834/2883460093":"2024-04-24T14:36:50.829845+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f028006c920 msgr2=0x7f028006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f028006c920 0x7f028006ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0294000f60 tx=0x7f0294009040 comp rx=0 tx=0).stop 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 msgr2=0x7f02a40cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40cfc20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d800 tx=0x7f028c00dcc0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 shutdown_connections 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f028006c920 0x7f028006ede0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a415e7c0 0x7f02a4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a40d0d90 0x7f02a40cfc20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 --2- 172.21.15.59:0/1890884043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a40b5ff0 0x7f02a41f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 >> 172.21.15.59:0/1890884043 conn(0x7f02a409d920 msgr2=0x7f02a40b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 shutdown_connections 2024-04-23T14:37:03.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.556+0000 7f02a9000640 1 -- 172.21.15.59:0/1890884043 wait complete. 2024-04-23T14:37:03.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:36:27.526201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:36:28.678324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:37:03.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:37:03.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.792+0000 7ff96b200640 1 Processor -- start 2024-04-23T14:37:03.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.792+0000 7ff96b200640 1 -- start start 2024-04-23T14:37:03.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.794+0000 7ff96b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff96c1567e0 0x7ff96c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.794+0000 7ff96b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c0d0d10 0x7ff96c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.794+0000 7ff96b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c15cab0 con 0x7ff96c15d810 2024-04-23T14:37:03.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.794+0000 7ff96b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c160e40 con 0x7ff96c1567e0 2024-04-23T14:37:03.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.794+0000 7ff96b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c15cd30 con 0x7ff96c0d0d10 2024-04-23T14:37:03.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.795+0000 7ff96a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c0d0d10 0x7ff96c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.795+0000 7ff969e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff96c1567e0 0x7ff96c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.795+0000 7ff96a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c0d0d10 0x7ff96c160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:39450/0 (socket says 172.21.15.59:39450) 2024-04-23T14:37:03.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.795+0000 7ff96a800640 1 -- 172.21.15.59:0/196221986 learned_addr learned my addr 172.21.15.59:0/196221986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:03.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.796+0000 7ff96a800640 1 -- 172.21.15.59:0/196221986 >> v2:172.21.15.76:3300/0 conn(0x7ff96c1567e0 msgr2=0x7ff96c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.796+0000 7ff96a800640 1 --2- 172.21.15.59:0/196221986 >> v2:172.21.15.76:3300/0 conn(0x7ff96c1567e0 0x7ff96c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.796+0000 7ff96a800640 1 -- 172.21.15.59:0/196221986 >> v1:172.21.15.76:6789/0 conn(0x7ff96c15d810 legacy=0x7ff96c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.796+0000 7ff96a800640 1 -- 172.21.15.59:0/196221986 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c15d010 con 0x7ff96c0d0d10 2024-04-23T14:37:03.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.796+0000 7ff969e00640 1 --2- 172.21.15.59:0/196221986 >> v2:172.21.15.76:3300/0 conn(0x7ff96c1567e0 0x7ff96c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:37:03.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.797+0000 7ff96a800640 1 --2- 172.21.15.59:0/196221986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c0d0d10 0x7ff96c160110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff964009e50 tx=0x7ff96402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee35452c1de40ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:03.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff969400640 1 -- 172.21.15.59:0/196221986 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964036070 con 0x7ff96c0d0d10 2024-04-23T14:37:03.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff969400640 1 -- 172.21.15.59:0/196221986 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff964031600 con 0x7ff96c0d0d10 2024-04-23T14:37:03.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff969400640 1 -- 172.21.15.59:0/196221986 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9640319d0 con 0x7ff96c0d0d10 2024-04-23T14:37:03.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff96b200640 1 -- 172.21.15.59:0/196221986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c0d0d10 msgr2=0x7ff96c160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff96b200640 1 --2- 172.21.15.59:0/196221986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c0d0d10 0x7ff96c160110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff964009e50 tx=0x7ff96402a710 comp rx=0 tx=0).stop 2024-04-23T14:37:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff96b200640 1 -- 172.21.15.59:0/196221986 shutdown_connections 2024-04-23T14:37:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff96b200640 1 --2- 172.21.15.59:0/196221986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c0d0d10 0x7ff96c160110 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff96b200640 1 --2- 172.21.15.59:0/196221986 >> v2:172.21.15.76:3300/0 conn(0x7ff96c1567e0 0x7ff96c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.798+0000 7ff96b200640 1 -- 172.21.15.59:0/196221986 >> 172.21.15.59:0/196221986 conn(0x7ff96c06bdf0 msgr2=0x7ff96c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.799+0000 7ff96b200640 1 -- 172.21.15.59:0/196221986 shutdown_connections 2024-04-23T14:37:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.799+0000 7ff96b200640 1 -- 172.21.15.59:0/196221986 wait complete. 2024-04-23T14:37:03.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.800+0000 7ff96b200640 1 Processor -- start 2024-04-23T14:37:03.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.800+0000 7ff96b200640 1 -- start start 2024-04-23T14:37:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.801+0000 7ff96b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c0d0d10 0x7ff96c1f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.801+0000 7ff96b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96c1567e0 0x7ff96c1f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.801+0000 7ff96b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c15d810 0x7ff96c1f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.801+0000 7ff96b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1607d0 con 0x7ff96c0d0d10 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.801+0000 7ff96b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c160650 con 0x7ff96c1567e0 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.801+0000 7ff96b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c160950 con 0x7ff96c15d810 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c0d0d10 0x7ff96c1f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c0d0d10 0x7ff96c1f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52622/0 (socket says 172.21.15.59:52622) 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 -- 172.21.15.59:0/3589258397 learned_addr learned my addr 172.21.15.59:0/3589258397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff961400640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96c1567e0 0x7ff96c1f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff96a800640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c15d810 0x7ff96c1f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 -- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c15d810 msgr2=0x7ff96c1f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c15d810 0x7ff96c1f83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 -- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96c1567e0 msgr2=0x7ff96c1f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96c1567e0 0x7ff96c1f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.802+0000 7ff969e00640 1 -- 172.21.15.59:0/3589258397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c1f8b60 con 0x7ff96c0d0d10 2024-04-23T14:37:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.803+0000 7ff969e00640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c0d0d10 0x7ff96c1f4210 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00ebe0 tx=0x7ff95c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:03.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.803+0000 7ff962a00640 1 -- 172.21.15.59:0/3589258397 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c017070 con 0x7ff96c0d0d10 2024-04-23T14:37:03.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.803+0000 7ff962a00640 1 -- 172.21.15.59:0/3589258397 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c00cc30 con 0x7ff96c0d0d10 2024-04-23T14:37:03.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.803+0000 7ff962a00640 1 -- 172.21.15.59:0/3589258397 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c013470 con 0x7ff96c0d0d10 2024-04-23T14:37:03.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.803+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c159330 con 0x7ff96c0d0d10 2024-04-23T14:37:03.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.803+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1f4750 con 0x7ff96c0d0d10 2024-04-23T14:37:03.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.804+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c04fd90 con 0x7ff96c0d0d10 2024-04-23T14:37:03.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.805+0000 7ff962a00640 1 -- 172.21.15.59:0/3589258397 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff95c01a070 con 0x7ff96c0d0d10 2024-04-23T14:37:03.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.806+0000 7ff962a00640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:03.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.806+0000 7ff962a00640 1 -- 172.21.15.59:0/3589258397 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7ff95c08bf00 con 0x7ff96c0d0d10 2024-04-23T14:37:03.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.806+0000 7ff961400640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:03.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.807+0000 7ff961400640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff96c15a0b0 tx=0x7ff954009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:03.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:03.811+0000 7ff962a00640 1 -- 172.21.15.59:0/3589258397 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff95c05a230 con 0x7ff96c0d0d10 2024-04-23T14:37:04.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.014+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff96c071c70 con 0x7ff96c0d0d10 2024-04-23T14:37:04.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.015+0000 7ff962a00640 1 -- 172.21.15.59:0/3589258397 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7ff95c059b30 con 0x7ff96c0d0d10 2024-04-23T14:37:04.016 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff94806c8f0 msgr2=0x7ff94806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff96c15a0b0 tx=0x7ff954009040 comp rx=0 tx=0).stop 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c0d0d10 msgr2=0x7ff96c1f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c0d0d10 0x7ff96c1f4210 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00ebe0 tx=0x7ff95c00c520 comp rx=0 tx=0).stop 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 shutdown_connections 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96c15d810 0x7ff96c1f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.017+0000 7ff96b200640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96c1567e0 0x7ff96c1f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.018+0000 7ff96b200640 1 --2- 172.21.15.59:0/3589258397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96c0d0d10 0x7ff96c1f4210 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.018+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 >> 172.21.15.59:0/3589258397 conn(0x7ff96c06bdf0 msgr2=0x7ff96c072bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.018+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 shutdown_connections 2024-04-23T14:37:04.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.018+0000 7ff96b200640 1 -- 172.21.15.59:0/3589258397 wait complete. 2024-04-23T14:37:04.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:37:04.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.245+0000 7f88c5600640 1 Processor -- start 2024-04-23T14:37:04.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.245+0000 7f88c5600640 1 -- start start 2024-04-23T14:37:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.247+0000 7f88c5600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 0x7f88c00aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.247+0000 7f88c5600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c00adc80 con 0x7f88c0071d70 2024-04-23T14:37:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.247+0000 7f88c5600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c00adf20 con 0x7f88c015e790 2024-04-23T14:37:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.247+0000 7f88c5600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f88c00aef40 con 0x7f88c006f6f0 2024-04-23T14:37:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.247+0000 7f88bf400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f88c0071d70 0x7f88c00b3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56344/0 (socket says 172.21.15.59:56344) 2024-04-23T14:37:04.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.247+0000 7f88bf400640 1 -- 172.21.15.59:0/3680595852 learned_addr learned my addr 172.21.15.59:0/3680595852 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:04.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88be000640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 0x7f88c00aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992904684 0 0) 0x7f88c00adc80 con 0x7f88c0071d70 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88a4003520 con 0x7f88c0071d70 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1343450568 0 0) 0x7f88a4003520 con 0x7f88c0071d70 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 msgr2=0x7f88c00aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 0x7f88c00aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 >> v1:172.21.15.76:6789/0 conn(0x7f88c015e790 legacy=0x7f88c0162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c00ae200 con 0x7f88c0071d70 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f88b80019f0 con 0x7f88c0071d70 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 >> v1:172.21.15.59:6789/0 conn(0x7f88c0071d70 legacy=0x7f88c00b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 0x7f88a4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4003fd0 0x7f88a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a4004d40 0x7f88a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88c00aef40 con 0x7f88a4004d40 2024-04-23T14:37:04.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f88a400bb20 con 0x7f88a4003fd0 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.248+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f88a400bd20 con 0x7f88c006f6f0 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88bf400640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a4004d40 0x7f88a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88bea00640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 0x7f88a4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88be000640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4003fd0 0x7f88a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88bf400640 1 -- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 msgr2=0x7f88a4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88bf400640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88c006f6f0 0x7f88a4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88bf400640 1 -- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4003fd0 msgr2=0x7f88a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88bf400640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4003fd0 0x7f88a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.249+0000 7f88bf400640 1 -- 172.21.15.59:0/3680595852 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a4003520 con 0x7f88a4004d40 2024-04-23T14:37:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.250+0000 7f88bf400640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a4004d40 0x7f88a40051c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f88b800a770 tx=0x7f88a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e54282834e16a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.250+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b800a390 con 0x7f88a4004d40 2024-04-23T14:37:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.250+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b802f070 con 0x7f88a4004d40 2024-04-23T14:37:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.250+0000 7f88bd600640 1 -- 172.21.15.59:0/3680595852 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b802bc20 con 0x7f88a4004d40 2024-04-23T14:37:04.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.251+0000 7f88c5600640 1 -- 172.21.15.59:0/3680595852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a4004d40 msgr2=0x7f88a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.251+0000 7f88c5600640 1 --2- 172.21.15.59:0/3680595852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88a4004d40 0x7f88a40051c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f88b800a770 tx=0x7f88a4005830 comp rx=0 tx=0).stop 2024-04-23T14:37:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.251+0000 7f88bf400640 1 -- 172.21.15.59:0/3680595852 reap_dead start 2024-04-23T14:37:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.251+0000 7f88c5600640 1 -- 172.21.15.59:0/3680595852 shutdown_connections 2024-04-23T14:37:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.251+0000 7f88c5600640 1 -- 172.21.15.59:0/3680595852 >> 172.21.15.59:0/3680595852 conn(0x7f88c0067d90 msgr2=0x7f88c006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.251+0000 7f88c5600640 1 -- 172.21.15.59:0/3680595852 shutdown_connections 2024-04-23T14:37:04.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.251+0000 7f88c5600640 1 -- 172.21.15.59:0/3680595852 wait complete. 2024-04-23T14:37:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.253+0000 7f88c5600640 1 Processor -- start 2024-04-23T14:37:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.253+0000 7f88c5600640 1 -- start start 2024-04-23T14:37:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88c5600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88a4004d40 0x7f88c00b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88c5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c01f4660 0x7f88c0156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88c5600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88c01570c0 0x7f88c0160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88c5600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88c006fbb0 con 0x7f88c01f4660 2024-04-23T14:37:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88c5600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f88c015eb70 con 0x7f88c01570c0 2024-04-23T14:37:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88c5600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f88c015ecf0 con 0x7f88a4004d40 2024-04-23T14:37:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88be000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c01f4660 0x7f88c0156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88be000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c01f4660 0x7f88c0156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52642/0 (socket says 172.21.15.59:52642) 2024-04-23T14:37:04.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.254+0000 7f88be000640 1 -- 172.21.15.59:0/1283640201 learned_addr learned my addr 172.21.15.59:0/1283640201 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88bea00640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88a4004d40 0x7f88c00b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88bf400640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88c01570c0 0x7f88c0160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88be000640 1 -- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88a4004d40 msgr2=0x7f88c00b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88be000640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88a4004d40 0x7f88c00b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88be000640 1 -- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88c01570c0 msgr2=0x7f88c0160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88be000640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88c01570c0 0x7f88c0160e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88be000640 1 -- 172.21.15.59:0/1283640201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c01f3bc0 con 0x7f88c01f4660 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88bf400640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88c01570c0 0x7f88c0160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88be000640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c01f4660 0x7f88c0156b80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f88ac011630 tx=0x7f88ac011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88b6a00640 1 -- 172.21.15.59:0/1283640201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00f920 con 0x7f88c01f4660 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88b6a00640 1 -- 172.21.15.59:0/1283640201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ac00fac0 con 0x7f88c01f4660 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c0157520 con 0x7f88c01f4660 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.255+0000 7f88b6a00640 1 -- 172.21.15.59:0/1283640201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00fe70 con 0x7f88c01f4660 2024-04-23T14:37:04.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.256+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c01f3e60 con 0x7f88c01f4660 2024-04-23T14:37:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.257+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c0068a80 con 0x7f88c01f4660 2024-04-23T14:37:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.258+0000 7f88b6a00640 1 -- 172.21.15.59:0/1283640201 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88ac015460 con 0x7f88c01f4660 2024-04-23T14:37:04.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.258+0000 7f88b6a00640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88a006c8f0 0x7f88a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.259+0000 7f88b6a00640 1 -- 172.21.15.59:0/1283640201 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f88ac08c5d0 con 0x7f88c01f4660 2024-04-23T14:37:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.259+0000 7f88bea00640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88a006c8f0 0x7f88a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.259+0000 7f88bea00640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88a006c8f0 0x7f88a006edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007fd0 tx=0x7f88b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.263+0000 7f88b6a00640 1 -- 172.21.15.59:0/1283640201 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88ac019030 con 0x7f88c01f4660 2024-04-23T14:37:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.462+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88c0162190 con 0x7f88c01f4660 2024-04-23T14:37:04.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.463+0000 7f88b6a00640 1 -- 172.21.15.59:0/1283640201 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) ==== 104+0+11 (secure 0 0 0) 0x7f88ac08ae20 con 0x7f88c01f4660 2024-04-23T14:37:04.464 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:37:04.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.465+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88a006c8f0 msgr2=0x7f88a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.465+0000 7f88c5600640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88a006c8f0 0x7f88a006edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007fd0 tx=0x7f88b0009040 comp rx=0 tx=0).stop 2024-04-23T14:37:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.465+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c01f4660 msgr2=0x7f88c0156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.465+0000 7f88c5600640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c01f4660 0x7f88c0156b80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f88ac011630 tx=0x7f88ac011af0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 shutdown_connections 2024-04-23T14:37:04.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88a006c8f0 0x7f88a006edb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88c01570c0 0x7f88c0160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c01f4660 0x7f88c0156b80 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 --2- 172.21.15.59:0/1283640201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88a4004d40 0x7f88c00b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 >> 172.21.15.59:0/1283640201 conn(0x7f88c0067d90 msgr2=0x7f88c00b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 shutdown_connections 2024-04-23T14:37:04.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.466+0000 7f88c5600640 1 -- 172.21.15.59:0/1283640201 wait complete. 2024-04-23T14:37:04.486 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:37:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.699+0000 7f1d1dc00640 1 Processor -- start 2024-04-23T14:37:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.699+0000 7f1d1dc00640 1 -- start start 2024-04-23T14:37:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.701+0000 7f1d1dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d1815d300 0x7f1d18071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.701+0000 7f1d1dc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1d180d0600 0x7f1d180d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.701+0000 7f1d1dc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d18163320 con 0x7f1d18167180 2024-04-23T14:37:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.701+0000 7f1d1dc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1d18163650 con 0x7f1d180d0600 2024-04-23T14:37:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.701+0000 7f1d1dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1d18072380 con 0x7f1d1815d300 2024-04-23T14:37:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d1d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1d180d0600 0x7f1d180d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d1d200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1d180d0600 0x7f1d180d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52644/0 (socket says 172.21.15.59:52644) 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d1d200640 1 -- 172.21.15.59:0/501283196 learned_addr learned my addr 172.21.15.59:0/501283196 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d17e00640 1 --2- 172.21.15.59:0/501283196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d1815d300 0x7f1d18071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d1d200640 1 -- 172.21.15.59:0/501283196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d1815d300 msgr2=0x7f1d18071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d1d200640 1 --2- 172.21.15.59:0/501283196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d1815d300 0x7f1d18071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d1d200640 1 -- 172.21.15.59:0/501283196 >> v1:172.21.15.76:6789/0 conn(0x7f1d18167180 legacy=0x7f1d18163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.702+0000 7f1d1d200640 1 -- 172.21.15.59:0/501283196 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d18072660 con 0x7f1d180d0600 2024-04-23T14:37:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.704+0000 7f1d1d200640 1 --2- 172.21.15.59:0/501283196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d180d0600 0x7f1d180d0a90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1d181637d0 tx=0x7f1d1002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45678a750bc9329c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.704+0000 7f1d17400640 1 -- 172.21.15.59:0/501283196 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10036070 con 0x7f1d180d0600 2024-04-23T14:37:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.704+0000 7f1d17400640 1 -- 172.21.15.59:0/501283196 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d10031600 con 0x7f1d180d0600 2024-04-23T14:37:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.704+0000 7f1d17400640 1 -- 172.21.15.59:0/501283196 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d100319d0 con 0x7f1d180d0600 2024-04-23T14:37:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.704+0000 7f1d1dc00640 1 -- 172.21.15.59:0/501283196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d180d0600 msgr2=0x7f1d180d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.704+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/501283196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d180d0600 0x7f1d180d0a90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1d181637d0 tx=0x7f1d1002a730 comp rx=0 tx=0).stop 2024-04-23T14:37:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.705+0000 7f1d1dc00640 1 -- 172.21.15.59:0/501283196 shutdown_connections 2024-04-23T14:37:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.705+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/501283196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d180d0600 0x7f1d180d0a90 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.705+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/501283196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d1815d300 0x7f1d18071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.705+0000 7f1d1dc00640 1 -- 172.21.15.59:0/501283196 >> 172.21.15.59:0/501283196 conn(0x7f1d1806bdf0 msgr2=0x7f1d1806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.705+0000 7f1d1dc00640 1 -- 172.21.15.59:0/501283196 shutdown_connections 2024-04-23T14:37:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.705+0000 7f1d1dc00640 1 -- 172.21.15.59:0/501283196 wait complete. 2024-04-23T14:37:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.707+0000 7f1d1dc00640 1 Processor -- start 2024-04-23T14:37:04.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.707+0000 7f1d1dc00640 1 -- start start 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d180d0600 0x7f1d181561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1dc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1815d300 0x7f1d181ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18167180 0x7f1d181eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1806e1e0 con 0x7f1d18167180 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1dc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181675e0 con 0x7f1d1815d300 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1806e360 con 0x7f1d180d0600 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18167180 0x7f1d181eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18167180 0x7f1d181eeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52652/0 (socket says 172.21.15.59:52652) 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1d200640 1 -- 172.21.15.59:0/2362543120 learned_addr learned my addr 172.21.15.59:0/2362543120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d0fe00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1815d300 0x7f1d181ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d17e00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d180d0600 0x7f1d181561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1d200640 1 -- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d180d0600 msgr2=0x7f1d181561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.708+0000 7f1d1d200640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d180d0600 0x7f1d181561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d1d200640 1 -- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1815d300 msgr2=0x7f1d181ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d1d200640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1815d300 0x7f1d181ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d1d200640 1 -- 172.21.15.59:0/2362543120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d18157550 con 0x7f1d18167180 2024-04-23T14:37:04.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d1d200640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18167180 0x7f1d181eeb80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009f70 tx=0x7f1d10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d14c00640 1 -- 172.21.15.59:0/2362543120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1003f020 con 0x7f1d18167180 2024-04-23T14:37:04.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d14c00640 1 -- 172.21.15.59:0/2362543120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d10037040 con 0x7f1d18167180 2024-04-23T14:37:04.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d14c00640 1 -- 172.21.15.59:0/2362543120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10036070 con 0x7f1d18167180 2024-04-23T14:37:04.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d18157830 con 0x7f1d18167180 2024-04-23T14:37:04.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.709+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d18157df0 con 0x7f1d18167180 2024-04-23T14:37:04.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.711+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1804fd90 con 0x7f1d18167180 2024-04-23T14:37:04.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.711+0000 7f1d14c00640 1 -- 172.21.15.59:0/2362543120 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d10042020 con 0x7f1d18167180 2024-04-23T14:37:04.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.712+0000 7f1d14c00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cf406c8f0 0x7f1cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.713+0000 7f1d14c00640 1 -- 172.21.15.59:0/2362543120 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7f1d100ac100 con 0x7f1d18167180 2024-04-23T14:37:04.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.713+0000 7f1d17e00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cf406c8f0 0x7f1cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.714+0000 7f1d17e00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cf406c8f0 0x7f1cf406edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006fd0 tx=0x7f1d08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.717+0000 7f1d14c00640 1 -- 172.21.15.59:0/2362543120 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d10049030 con 0x7f1d18167180 2024-04-23T14:37:04.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.914+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d180cff50 con 0x7f1d18167180 2024-04-23T14:37:04.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.915+0000 7f1d14c00640 1 -- 172.21.15.59:0/2362543120 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) ==== 100+0+11 (secure 0 0 0) 0x7f1d1007a430 con 0x7f1d18167180 2024-04-23T14:37:04.916 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:37:04.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cf406c8f0 msgr2=0x7f1cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cf406c8f0 0x7f1cf406edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006fd0 tx=0x7f1d08008040 comp rx=0 tx=0).stop 2024-04-23T14:37:04.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18167180 msgr2=0x7f1d181eeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18167180 0x7f1d181eeb80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009f70 tx=0x7f1d10004a40 comp rx=0 tx=0).stop 2024-04-23T14:37:04.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 shutdown_connections 2024-04-23T14:37:04.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1cf406c8f0 0x7f1cf406edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d18167180 0x7f1d181eeb80 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1815d300 0x7f1d181ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 --2- 172.21.15.59:0/2362543120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d180d0600 0x7f1d181561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.918+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 >> 172.21.15.59:0/2362543120 conn(0x7f1d1806bdf0 msgr2=0x7f1d181647f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:04.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.919+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 shutdown_connections 2024-04-23T14:37:04.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:04.919+0000 7f1d1dc00640 1 -- 172.21.15.59:0/2362543120 wait complete. 2024-04-23T14:37:04.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:37:05.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.142+0000 7ffa56a00640 1 Processor -- start 2024-04-23T14:37:05.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.142+0000 7ffa56a00640 1 -- start start 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.144+0000 7ffa56a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 0x7ffa500d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.144+0000 7ffa56a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa500b4520 0x7ffa500adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.144+0000 7ffa56a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5006f080 con 0x7ffa5006ec20 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.144+0000 7ffa56a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ffa501486a0 con 0x7ffa500b4520 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.144+0000 7ffa56a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5006f320 con 0x7ffa501482c0 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa56000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa500b4520 0x7ffa500adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa54c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffa5006ec20 0x7ffa500b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56354/0 (socket says 172.21.15.59:56354) 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa54c00640 1 -- 172.21.15.59:0/1411433193 learned_addr learned my addr 172.21.15.59:0/1411433193 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:05.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa56000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa500b4520 0x7ffa500adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52662/0 (socket says 172.21.15.59:52662) 2024-04-23T14:37:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa55600640 1 --2- 172.21.15.59:0/1411433193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 0x7ffa500d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:05.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa56000640 1 -- 172.21.15.59:0/1411433193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 msgr2=0x7ffa500d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa56000640 1 --2- 172.21.15.59:0/1411433193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 0x7ffa500d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa56000640 1 -- 172.21.15.59:0/1411433193 >> v1:172.21.15.59:6789/0 conn(0x7ffa5006ec20 legacy=0x7ffa500b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa56000640 1 -- 172.21.15.59:0/1411433193 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5006f600 con 0x7ffa500b4520 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.145+0000 7ffa4fe00640 1 -- 172.21.15.59:0/1411433193 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974016970 0 0) 0x7ffa5006f080 con 0x7ffa5006ec20 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.146+0000 7ffa56000640 1 --2- 172.21.15.59:0/1411433193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa500adcc0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009d90 tx=0x7ffa4802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be74663e9dfe32e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa4fe00640 1 -- 172.21.15.59:0/1411433193 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48007740 con 0x7ffa500b4520 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa4fe00640 1 -- 172.21.15.59:0/1411433193 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa480078e0 con 0x7ffa500b4520 2024-04-23T14:37:05.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa4fe00640 1 -- 172.21.15.59:0/1411433193 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48007cb0 con 0x7ffa500b4520 2024-04-23T14:37:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 -- 172.21.15.59:0/1411433193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 msgr2=0x7ffa500adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 --2- 172.21.15.59:0/1411433193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa500adcc0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009d90 tx=0x7ffa4802a6f0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 -- 172.21.15.59:0/1411433193 shutdown_connections 2024-04-23T14:37:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 --2- 172.21.15.59:0/1411433193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa500adcc0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 --2- 172.21.15.59:0/1411433193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 0x7ffa500d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 -- 172.21.15.59:0/1411433193 >> 172.21.15.59:0/1411433193 conn(0x7ffa50067b80 msgr2=0x7ffa5006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 -- 172.21.15.59:0/1411433193 shutdown_connections 2024-04-23T14:37:05.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.147+0000 7ffa56a00640 1 -- 172.21.15.59:0/1411433193 wait complete. 2024-04-23T14:37:05.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.149+0000 7ffa56a00640 1 Processor -- start 2024-04-23T14:37:05.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.149+0000 7ffa56a00640 1 -- start start 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa56a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5006ec20 0x7ffa50131d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa56a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa501e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa56a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 0x7ffa501f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa56a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5006fea0 con 0x7ffa500b4520 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa56a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5006fbc0 con 0x7ffa5006ec20 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa56a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffa500ae200 con 0x7ffa501482c0 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa54c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa501e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa54c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa501e6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52664/0 (socket says 172.21.15.59:52664) 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.150+0000 7ffa54c00640 1 -- 172.21.15.59:0/4289547686 learned_addr learned my addr 172.21.15.59:0/4289547686 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa54c00640 1 -- 172.21.15.59:0/4289547686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 msgr2=0x7ffa501f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa54c00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 0x7ffa501f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa55600640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5006ec20 0x7ffa50131d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa54c00640 1 -- 172.21.15.59:0/4289547686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5006ec20 msgr2=0x7ffa50131d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa54c00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5006ec20 0x7ffa50131d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa54c00640 1 -- 172.21.15.59:0/4289547686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa501e6ff0 con 0x7ffa500b4520 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa54c00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa501e6530 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00c4a0 tx=0x7ffa3c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa4d600640 1 -- 172.21.15.59:0/4289547686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c017070 con 0x7ffa500b4520 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa4d600640 1 -- 172.21.15.59:0/4289547686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c005ce0 con 0x7ffa500b4520 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa4d600640 1 -- 172.21.15.59:0/4289547686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0072b0 con 0x7ffa500b4520 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.151+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa501e72d0 con 0x7ffa500b4520 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.152+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501324f0 con 0x7ffa500b4520 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.153+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5006c960 con 0x7ffa500b4520 2024-04-23T14:37:05.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.154+0000 7ffa4d600640 1 -- 172.21.15.59:0/4289547686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa3c01a070 con 0x7ffa500b4520 2024-04-23T14:37:05.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.155+0000 7ffa4d600640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa3006c8f0 0x7ffa3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:05.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.155+0000 7ffa4d600640 1 -- 172.21.15.59:0/4289547686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5795+0+0 (secure 0 0 0) 0x7ffa3c08aeb0 con 0x7ffa500b4520 2024-04-23T14:37:05.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.155+0000 7ffa55600640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa3006c8f0 0x7ffa3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:05.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.156+0000 7ffa55600640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa3006c8f0 0x7ffa3006edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006fd0 tx=0x7ffa44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:05.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.159+0000 7ffa4d600640 1 -- 172.21.15.59:0/4289547686 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa3c059270 con 0x7ffa500b4520 2024-04-23T14:37:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.377+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa5004fd90 con 0x7ffa500b4520 2024-04-23T14:37:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.378+0000 7ffa4d600640 1 -- 172.21.15.59:0/4289547686 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) ==== 80+0+122 (secure 0 0 0) 0x7ffa3c058b70 con 0x7ffa500b4520 2024-04-23T14:37:05.380 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6836/2883460093 2024-04-24T14:36:50.829845+0000 2024-04-23T14:37:05.380 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6834/2883460093 2024-04-24T14:36:50.829845+0000 2024-04-23T14:37:05.380 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.382+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa3006c8f0 msgr2=0x7ffa3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.382+0000 7ffa56a00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa3006c8f0 0x7ffa3006edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006fd0 tx=0x7ffa44008040 comp rx=0 tx=0).stop 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.382+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 msgr2=0x7ffa501e6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.382+0000 7ffa56a00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa501e6530 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00c4a0 tx=0x7ffa3c00c960 comp rx=0 tx=0).stop 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.382+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 shutdown_connections 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.383+0000 7ffa56a00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffa3006c8f0 0x7ffa3006edb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.383+0000 7ffa56a00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa501482c0 0x7ffa501f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.383+0000 7ffa56a00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa500b4520 0x7ffa501e6530 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.383+0000 7ffa56a00640 1 --2- 172.21.15.59:0/4289547686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa5006ec20 0x7ffa50131d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.383+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 >> 172.21.15.59:0/4289547686 conn(0x7ffa50067b80 msgr2=0x7ffa5006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:05.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.383+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 shutdown_connections 2024-04-23T14:37:05.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:05.383+0000 7ffa56a00640 1 -- 172.21.15.59:0/4289547686 wait complete. 2024-04-23T14:37:05.403 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:37:05.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:05.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:37:05.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[59144]: fuse finished with error 0 and tester_r 0 2024-04-23T14:37:05.481 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:37:11.583 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:37:11.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:11.583 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:37:11.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:11.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-23T14:37:11.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.869+0000 7f2f69a00640 1 Processor -- start 2024-04-23T14:37:11.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.870+0000 7f2f69a00640 1 -- start start 2024-04-23T14:37:11.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.871+0000 7f2f69a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 0x7f2f6406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.871+0000 7f2f69a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f64071e90 con 0x7f2f64072d00 2024-04-23T14:37:11.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.871+0000 7f2f69a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6406eac0 con 0x7f2f6415bcf0 2024-04-23T14:37:11.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.871+0000 7f2f69a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6406eda0 con 0x7f2f641552c0 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.872+0000 7f2f69000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2f64072d00 0x7f2f640b30b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37098/0 (socket says 172.21.15.59:37098) 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.872+0000 7f2f69000640 1 -- 172.21.15.59:0/3084029324 learned_addr learned my addr 172.21.15.59:0/3084029324 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.872+0000 7f2f63e00640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 0x7f2f6406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.872+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767822455 0 0) 0x7f2f64071e90 con 0x7f2f64072d00 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.872+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f48003520 con 0x7f2f64072d00 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1217624492 0 0) 0x7f2f48003520 con 0x7f2f64072d00 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 msgr2=0x7f2f6406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 0x7f2f6406fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 >> v1:172.21.15.76:6789/0 conn(0x7f2f6415bcf0 legacy=0x7f2f640725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6406f080 con 0x7f2f64072d00 2024-04-23T14:37:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2f5c001670 con 0x7f2f64072d00 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 >> v1:172.21.15.59:6789/0 conn(0x7f2f64072d00 legacy=0x7f2f640b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 0x7f2f48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f48004d40 0x7f2f480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6406eda0 con 0x7f2f480039e0 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4800dc80 con 0x7f2f48004d40 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4800de80 con 0x7f2f641552c0 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.873+0000 7f2f63e00640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f63e00640 1 -- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 msgr2=0x7f2f48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f5b400640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 0x7f2f48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f69000640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f48004d40 0x7f2f480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f63e00640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f641552c0 0x7f2f48004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f63e00640 1 -- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f48004d40 msgr2=0x7f2f480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f63e00640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f48004d40 0x7f2f480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f63e00640 1 -- 172.21.15.59:0/3084029324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f48003520 con 0x7f2f480039e0 2024-04-23T14:37:11.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.874+0000 7f2f69000640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f48004d40 0x7f2f480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.875+0000 7f2f63e00640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f48003e00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400a220 tx=0x7f2f48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e128f3800a749c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.875+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400adf0 con 0x7f2f480039e0 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.875+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f54037070 con 0x7f2f480039e0 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.875+0000 7f2f63400640 1 -- 172.21.15.59:0/3084029324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54032830 con 0x7f2f480039e0 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.876+0000 7f2f69a00640 1 -- 172.21.15.59:0/3084029324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 msgr2=0x7f2f48003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.876+0000 7f2f69a00640 1 --2- 172.21.15.59:0/3084029324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f48003e00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400a220 tx=0x7f2f48005830 comp rx=0 tx=0).stop 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.876+0000 7f2f69000640 1 -- 172.21.15.59:0/3084029324 reap_dead start 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.876+0000 7f2f69a00640 1 -- 172.21.15.59:0/3084029324 shutdown_connections 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.876+0000 7f2f69a00640 1 -- 172.21.15.59:0/3084029324 >> 172.21.15.59:0/3084029324 conn(0x7f2f640b2200 msgr2=0x7f2f640b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.876+0000 7f2f69a00640 1 -- 172.21.15.59:0/3084029324 shutdown_connections 2024-04-23T14:37:11.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.876+0000 7f2f69a00640 1 -- 172.21.15.59:0/3084029324 wait complete. 2024-04-23T14:37:11.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.878+0000 7f2f69a00640 1 Processor -- start 2024-04-23T14:37:11.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.878+0000 7f2f69a00640 1 -- start start 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.879+0000 7f2f69a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f6414c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.879+0000 7f2f69a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f640c9470 0x7f2f640cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.879+0000 7f2f69a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f64158e90 0x7f2f64159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.879+0000 7f2f69a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641556a0 con 0x7f2f480039e0 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.879+0000 7f2f69a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6406f640 con 0x7f2f64158e90 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.879+0000 7f2f69a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6415c170 con 0x7f2f640c9470 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f6414c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f6414c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43786/0 (socket says 172.21.15.59:43786) 2024-04-23T14:37:11.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 -- 172.21.15.59:0/948062516 learned_addr learned my addr 172.21.15.59:0/948062516 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f5b400640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f640c9470 0x7f2f640cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f69000640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f64158e90 0x7f2f64159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 -- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f640c9470 msgr2=0x7f2f640cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f640c9470 0x7f2f640cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 -- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f64158e90 msgr2=0x7f2f64159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f64158e90 0x7f2f64159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f63e00640 1 -- 172.21.15.59:0/948062516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f640ca480 con 0x7f2f480039e0 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.880+0000 7f2f5b400640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f640c9470 0x7f2f640cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.881+0000 7f2f63e00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f6414c630 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2f54034040 tx=0x7f2f54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.881+0000 7f2f60c00640 1 -- 172.21.15.59:0/948062516 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400adf0 con 0x7f2f480039e0 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.881+0000 7f2f60c00640 1 -- 172.21.15.59:0/948062516 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f54003b90 con 0x7f2f480039e0 2024-04-23T14:37:11.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.881+0000 7f2f60c00640 1 -- 172.21.15.59:0/948062516 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5403c610 con 0x7f2f480039e0 2024-04-23T14:37:11.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.881+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f640c9900 con 0x7f2f480039e0 2024-04-23T14:37:11.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.881+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f640c9ec0 con 0x7f2f480039e0 2024-04-23T14:37:11.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.884+0000 7f2f60c00640 1 -- 172.21.15.59:0/948062516 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f54044070 con 0x7f2f480039e0 2024-04-23T14:37:11.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.885+0000 7f2f60c00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f4006c8f0 0x7f2f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.885+0000 7f2f60c00640 1 -- 172.21.15.59:0/948062516 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f2f540aca30 con 0x7f2f480039e0 2024-04-23T14:37:11.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.885+0000 7f2f5b400640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f4006c8f0 0x7f2f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.885+0000 7f2f6de3a740 1 Processor -- start 2024-04-23T14:37:11.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.886+0000 7f2f6de3a740 1 -- start start 2024-04-23T14:37:11.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.886+0000 7f2f5b400640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f4006c8f0 0x7f2f4006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2f640b16c0 tx=0x7f2f4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da9084f90 0x558da90cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558da8fe83a0 con 0x558da9084f90 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558da8f8e240 con 0x558da90af2b0 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558da8fe4360 con 0x558da90ccb50 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da9084f90 0x558da90cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da9084f90 0x558da90cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43794/0 (socket says 172.21.15.59:43794) 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 -- 172.21.15.59:0/689096679 learned_addr learned my addr 172.21.15.59:0/689096679 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 -- 172.21.15.59:0/689096679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 msgr2=0x558da90cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 --2- 172.21.15.59:0/689096679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 -- 172.21.15.59:0/689096679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 msgr2=0x558da90af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 --2- 172.21.15.59:0/689096679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.887+0000 7f2f69000640 1 -- 172.21.15.59:0/689096679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558da8dc6a20 con 0x558da9084f90 2024-04-23T14:37:11.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.888+0000 7f2f69000640 1 --2- 172.21.15.59:0/689096679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da9084f90 0x558da90cb210 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2f6415e7c0 tx=0x7f2f5c007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de46d53a7fa0702 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.888+0000 7f2f58c00640 1 -- 172.21.15.59:0/689096679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c018070 con 0x558da9084f90 2024-04-23T14:37:11.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.888+0000 7f2f58c00640 1 -- 172.21.15.59:0/689096679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0072d0 con 0x558da9084f90 2024-04-23T14:37:11.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.888+0000 7f2f58c00640 1 -- 172.21.15.59:0/689096679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0047e0 con 0x558da9084f90 2024-04-23T14:37:11.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 -- 172.21.15.59:0/689096679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da9084f90 msgr2=0x558da90cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/689096679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da9084f90 0x558da90cb210 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2f6415e7c0 tx=0x7f2f5c007be0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 -- 172.21.15.59:0/689096679 shutdown_connections 2024-04-23T14:37:11.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/689096679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/689096679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da9084f90 0x558da90cb210 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/689096679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 -- 172.21.15.59:0/689096679 >> 172.21.15.59:0/689096679 conn(0x558da90818c0 msgr2=0x558da90ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 -- 172.21.15.59:0/689096679 shutdown_connections 2024-04-23T14:37:11.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.889+0000 7f2f6de3a740 1 -- 172.21.15.59:0/689096679 wait complete. 2024-04-23T14:37:11.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.890+0000 7f2f6de3a740 1 -- start start 2024-04-23T14:37:11.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.890+0000 7f2f6de3a740 1 Processor -- start 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da90b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558da9085d30 con 0x558da90b04b0 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558da8fa17e0 con 0x558da90af2b0 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558da907f7f0 con 0x558da90ccb50 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f6de3a740 10 client.0 _ll_register_callbacks cb 0x558da8fdeb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da90b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da90b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43806/0 (socket says 172.21.15.59:43806) 2024-04-23T14:37:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f69000640 1 -- 172.21.15.59:0/2870011392 learned_addr learned my addr 172.21.15.59:0/2870011392 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f5b400640 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.891+0000 7f2f69000640 1 -- 172.21.15.59:0/2870011392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 msgr2=0x558da90b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f69000640 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f69000640 1 -- 172.21.15.59:0/2870011392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 msgr2=0x558da90aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f69000640 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f69000640 1 -- 172.21.15.59:0/2870011392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558da8cba310 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f69000640 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da90b2cf0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c000 tx=0x7f2f6406ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7796cc10bc0572d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f51600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f51600640 1 -- 172.21.15.59:0/2870011392 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c018030 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f51600640 1 -- 172.21.15.59:0/2870011392 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f2f5c00e710 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f5c0072d0 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f51600640 1 -- 172.21.15.59:0/2870011392 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c02c050 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f6de3a740 10 client.7152 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.892+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x558da8e271b0 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f51600640 1 -- 172.21.15.59:0/2870011392 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0072d0 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f51600640 1 -- 172.21.15.59:0/2870011392 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f2f5c02f020 con 0x558da90b04b0 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f6de3a740 10 client.7152 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f6de3a740 10 client.7152 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f6de3a740 20 client.7152 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:11.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c1640 0x558da90c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f6de3a740 4 client.7152 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:37:11.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558da8ff5dc0 con 0x558da90c1640 2024-04-23T14:37:11.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.893+0000 7f2f63e00640 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c1640 0x558da90c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.894+0000 7f2f51600640 10 client.7152 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:11.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.894+0000 7f2f63e00640 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c1640 0x558da90c1a90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7cbf3501e79a28e server_cookie=999ed3d0ae4f9689 in_seq=0 out_seq=0 2024-04-23T14:37:11.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f51600640 1 -- 172.21.15.59:0/2870011392 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558da8ff5dc0 con 0x558da90c1640 2024-04-23T14:37:11.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f51600640 10 client.7152 handle_command_reply: tid=0 2024-04-23T14:37:11.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 client.7152 shutdown 2024-04-23T14:37:11.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 msgr2=0x558da90b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da90b2cf0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c000 tx=0x7f2f6406ec90 comp rx=0 tx=0).stop 2024-04-23T14:37:11.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 shutdown_connections 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c1640 0x558da90c1a90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da90b2cf0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/2870011392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 >> 172.21.15.59:0/2870011392 conn(0x558da9084f90 msgr2=0x558da90cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f51600640 0 client.7152 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 shutdown_connections 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 1 -- 172.21.15.59:0/2870011392 wait complete. 2024-04-23T14:37:11.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.895+0000 7f2f6de3a740 20 client.7152 trim_cache size 0 max 16384 2024-04-23T14:37:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.915+0000 7f2f6de3a740 1 Processor -- start 2024-04-23T14:37:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.915+0000 7f2f6de3a740 1 -- start start 2024-04-23T14:37:11.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da9084500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f642018d0 con 0x558da90b04b0 2024-04-23T14:37:11.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64200940 con 0x558da90af2b0 2024-04-23T14:37:11.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2f640ced20 con 0x558da90ccb50 2024-04-23T14:37:11.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f5b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da9084500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f5b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da9084500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43810/0 (socket says 172.21.15.59:43810) 2024-04-23T14:37:11.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.916+0000 7f2f5b400640 1 -- 172.21.15.59:0/620107038 learned_addr learned my addr 172.21.15.59:0/620107038 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f5b400640 1 -- 172.21.15.59:0/620107038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 msgr2=0x558da90d96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f5b400640 1 --2- 172.21.15.59:0/620107038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f5b400640 1 -- 172.21.15.59:0/620107038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 msgr2=0x558da90ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f5b400640 1 --2- 172.21.15.59:0/620107038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f5b400640 1 -- 172.21.15.59:0/620107038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558da90b02e0 con 0x558da90b04b0 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f5b400640 1 --2- 172.21.15.59:0/620107038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da9084500 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00e8a0 tx=0x7f2f4c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2439afa9fe2ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f53e00640 1 -- 172.21.15.59:0/620107038 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c013850 con 0x558da90b04b0 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f53e00640 1 -- 172.21.15.59:0/620107038 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c024070 con 0x558da90b04b0 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.917+0000 7f2f53e00640 1 -- 172.21.15.59:0/620107038 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c013bf0 con 0x558da90b04b0 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 -- 172.21.15.59:0/620107038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 msgr2=0x558da9084500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/620107038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da9084500 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00e8a0 tx=0x7f2f4c011000 comp rx=0 tx=0).stop 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 -- 172.21.15.59:0/620107038 shutdown_connections 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/620107038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90ccb50 0x558da90d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/620107038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90b04b0 0x558da9084500 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/620107038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90af2b0 0x558da90ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 -- 172.21.15.59:0/620107038 >> 172.21.15.59:0/620107038 conn(0x558da9084f90 msgr2=0x558da90c2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 -- 172.21.15.59:0/620107038 shutdown_connections 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.918+0000 7f2f6de3a740 1 -- 172.21.15.59:0/620107038 wait complete. 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.919+0000 7f2f6de3a740 1 -- start start 2024-04-23T14:37:11.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.919+0000 7f2f6de3a740 1 Processor -- start 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90af2b0 0x558da9146790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90b04b0 0x558da90dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f6de3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90ccb50 0x558da90db1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64200420 con 0x558da90ccb50 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558da90cb3f0 con 0x558da90b04b0 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f6de3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558da9084aa0 con 0x558da90af2b0 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f6de3a740 10 client.0 _ll_register_callbacks cb 0x7f2f5c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90ccb50 0x558da90db1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f69000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90ccb50 0x558da90db1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43816/0 (socket says 172.21.15.59:43816) 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.920+0000 7f2f69000640 1 -- 172.21.15.59:0/1047526193 learned_addr learned my addr 172.21.15.59:0/1047526193 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f63e00640 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90af2b0 0x558da9146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f69000640 1 -- 172.21.15.59:0/1047526193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90af2b0 msgr2=0x558da9146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f69000640 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90af2b0 0x558da9146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f69000640 1 -- 172.21.15.59:0/1047526193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90b04b0 msgr2=0x558da90dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f69000640 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90b04b0 0x558da90dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f69000640 1 -- 172.21.15.59:0/1047526193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558da90afa00 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f69000640 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90ccb50 0x558da90db1a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0018a0 tx=0x7f2f6406f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3c6e5895efcf58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f51600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f51600640 1 -- 172.21.15.59:0/1047526193 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c036060 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f51600640 1 -- 172.21.15.59:0/1047526193 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f2f5c00e960 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.921+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4c024070 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f51600640 1 -- 172.21.15.59:0/1047526193 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c03c050 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 10 client.7158 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x558da8c93490 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f51600640 1 -- 172.21.15.59:0/1047526193 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c024070 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f51600640 1 -- 172.21.15.59:0/1047526193 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f2f5c03f020 con 0x558da90ccb50 2024-04-23T14:37:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 10 client.7158 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 10 client.7158 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 20 client.7158 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c92e0 0x558da90ed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 4 client.7158 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x558da8fefc30 con 0x558da90c92e0 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.922+0000 7f2f63e00640 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c92e0 0x558da90ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.923+0000 7f2f63e00640 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c92e0 0x558da90ed270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1908ec4ecfb377 server_cookie=3eef6483b1a35e03 in_seq=0 out_seq=0 2024-04-23T14:37:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.923+0000 7f2f51600640 10 client.7158 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.923+0000 7f2f51600640 1 -- 172.21.15.59:0/1047526193 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558da8fefc30 con 0x558da90c92e0 2024-04-23T14:37:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f51600640 10 client.7158 handle_command_reply: tid=0 2024-04-23T14:37:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 client.7158 shutdown 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90ccb50 msgr2=0x558da90db1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90ccb50 0x558da90db1a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0018a0 tx=0x7f2f6406f250 comp rx=0 tx=0).stop 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 shutdown_connections 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558da90c92e0 0x558da90ed270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558da90ccb50 0x558da90db1a0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558da90b04b0 0x558da90dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f51600640 0 client.7158 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 --2- 172.21.15.59:0/1047526193 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558da90af2b0 0x558da9146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 >> 172.21.15.59:0/1047526193 conn(0x558da9084f90 msgr2=0x558da90c38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 shutdown_connections 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 1 -- 172.21.15.59:0/1047526193 wait complete. 2024-04-23T14:37:11.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.924+0000 7f2f6de3a740 20 client.7158 trim_cache size 0 max 16384 2024-04-23T14:37:11.929 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:37:11.929 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:37:11.929 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "e6c12528-9c9c-4dfa-9eed-67f64f686768", 2024-04-23T14:37:11.929 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:37:11.929 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:37:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f4006c8f0 msgr2=0x7f2f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f4006c8f0 0x7f2f4006edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2f640b16c0 tx=0x7f2f4c009040 comp rx=0 tx=0).stop 2024-04-23T14:37:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 msgr2=0x7f2f6414c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f6414c630 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2f54034040 tx=0x7f2f54004a40 comp rx=0 tx=0).stop 2024-04-23T14:37:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 shutdown_connections 2024-04-23T14:37:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f4006c8f0 0x7f2f4006edb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f640c9470 0x7f2f640cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f64158e90 0x7f2f64159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 --2- 172.21.15.59:0/948062516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f480039e0 0x7f2f6414c630 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 >> 172.21.15.59:0/948062516 conn(0x7f2f640b2200 msgr2=0x7f2f640725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 shutdown_connections 2024-04-23T14:37:11.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:11.925+0000 7f2f69a00640 1 -- 172.21.15.59:0/948062516 wait complete. 2024-04-23T14:37:11.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:11.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-23T14:37:12.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.144+0000 7f04c9800640 1 Processor -- start 2024-04-23T14:37:12.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.145+0000 7f04c9800640 1 -- start start 2024-04-23T14:37:12.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.146+0000 7f04c9800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c415ed20 0x7f04c40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.146+0000 7f04c9800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04c40d13e0 0x7f04c40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.146+0000 7f04c9800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c40b5da0 con 0x7f04c40b3600 2024-04-23T14:37:12.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.146+0000 7f04c9800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f04c415f100 con 0x7f04c40d13e0 2024-04-23T14:37:12.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.146+0000 7f04c9800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40d07d0 con 0x7f04c415ed20 2024-04-23T14:37:12.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.147+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c415ed20 0x7f04c40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.147+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c415ed20 0x7f04c40d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:60556/0 (socket says 172.21.15.59:60556) 2024-04-23T14:37:12.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.147+0000 7f04c3e00640 1 -- 172.21.15.59:0/3591793709 learned_addr learned my addr 172.21.15.59:0/3591793709 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:12.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.147+0000 7f04c8e00640 1 --2- 172.21.15.59:0/3591793709 >> v2:172.21.15.76:3300/0 conn(0x7f04c40d13e0 0x7f04c40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.148+0000 7f04c3e00640 1 -- 172.21.15.59:0/3591793709 >> v2:172.21.15.76:3300/0 conn(0x7f04c40d13e0 msgr2=0x7f04c40d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.148+0000 7f04c3e00640 1 --2- 172.21.15.59:0/3591793709 >> v2:172.21.15.76:3300/0 conn(0x7f04c40d13e0 0x7f04c40d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.148+0000 7f04c3e00640 1 -- 172.21.15.59:0/3591793709 >> v1:172.21.15.76:6789/0 conn(0x7f04c40b3600 legacy=0x7f04c40b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.148+0000 7f04c3e00640 1 -- 172.21.15.59:0/3591793709 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c40d0a20 con 0x7f04c415ed20 2024-04-23T14:37:12.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.149+0000 7f04c3e00640 1 --2- 172.21.15.59:0/3591793709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c415ed20 0x7f04c40d0ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009870 tx=0x7f04b402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e9b1bf6384d17b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.150+0000 7f04c2a00640 1 -- 172.21.15.59:0/3591793709 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4037070 con 0x7f04c415ed20 2024-04-23T14:37:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.150+0000 7f04c2a00640 1 -- 172.21.15.59:0/3591793709 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b4032440 con 0x7f04c415ed20 2024-04-23T14:37:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.150+0000 7f04c2a00640 1 -- 172.21.15.59:0/3591793709 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4032810 con 0x7f04c415ed20 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 -- 172.21.15.59:0/3591793709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c415ed20 msgr2=0x7f04c40d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 --2- 172.21.15.59:0/3591793709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c415ed20 0x7f04c40d0ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009870 tx=0x7f04b402a130 comp rx=0 tx=0).stop 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 -- 172.21.15.59:0/3591793709 shutdown_connections 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 --2- 172.21.15.59:0/3591793709 >> v2:172.21.15.76:3300/0 conn(0x7f04c40d13e0 0x7f04c40d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 --2- 172.21.15.59:0/3591793709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c415ed20 0x7f04c40d0ea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 -- 172.21.15.59:0/3591793709 >> 172.21.15.59:0/3591793709 conn(0x7f04c408bf90 msgr2=0x7f04c40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 -- 172.21.15.59:0/3591793709 shutdown_connections 2024-04-23T14:37:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.151+0000 7f04c9800640 1 -- 172.21.15.59:0/3591793709 wait complete. 2024-04-23T14:37:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.153+0000 7f04c9800640 1 Processor -- start 2024-04-23T14:37:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.153+0000 7f04c9800640 1 -- start start 2024-04-23T14:37:12.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c40b3600 0x7f04c4155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c9800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c40d13e0 0x7f04c40b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04c415ed20 0x7f04c41563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40d3490 con 0x7f04c40b3600 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c9800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40d31b0 con 0x7f04c415ed20 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c9800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40b4620 con 0x7f04c40d13e0 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c40b3600 0x7f04c4155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c40b3600 0x7f04c4155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43828/0 (socket says 172.21.15.59:43828) 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 -- 172.21.15.59:0/1411655393 learned_addr learned my addr 172.21.15.59:0/1411655393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 -- 172.21.15.59:0/1411655393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c40d13e0 msgr2=0x7f04c40b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c40d13e0 0x7f04c40b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 -- 172.21.15.59:0/1411655393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04c415ed20 msgr2=0x7f04c41563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04c415ed20 0x7f04c41563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.155+0000 7f04c3e00640 1 -- 172.21.15.59:0/1411655393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c4155fb0 con 0x7f04c40b3600 2024-04-23T14:37:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.156+0000 7f04c3e00640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c40b3600 0x7f04c4155a70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f04b402a170 tx=0x7f04b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.156+0000 7f04bbe00640 1 -- 172.21.15.59:0/1411655393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4037040 con 0x7f04c40b3600 2024-04-23T14:37:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.156+0000 7f04bbe00640 1 -- 172.21.15.59:0/1411655393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b4042070 con 0x7f04c40b3600 2024-04-23T14:37:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.156+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c40b8650 con 0x7f04c40b3600 2024-04-23T14:37:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.156+0000 7f04bbe00640 1 -- 172.21.15.59:0/1411655393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4030880 con 0x7f04c40b3600 2024-04-23T14:37:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.156+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c40b8c10 con 0x7f04c40b3600 2024-04-23T14:37:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.159+0000 7f04bbe00640 1 -- 172.21.15.59:0/1411655393 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04b4030a20 con 0x7f04c40b3600 2024-04-23T14:37:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.160+0000 7f04bbe00640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f04a006c8f0 0x7f04a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.160+0000 7f04bbe00640 1 -- 172.21.15.59:0/1411655393 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f04b40ac4a0 con 0x7f04c40b3600 2024-04-23T14:37:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.160+0000 7f04c3400640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f04a006c8f0 0x7f04a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.160+0000 7f04cdc2e740 1 Processor -- start 2024-04-23T14:37:12.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.161+0000 7f04cdc2e740 1 -- start start 2024-04-23T14:37:12.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.161+0000 7f04c3400640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f04a006c8f0 0x7f04a006edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0012e0 tx=0x7f04ac007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:12.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf40eb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411b7e0 0x55ddf40f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ddf40551f0 con 0x55ddf4138e00 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ddf3ffb350 con 0x55ddf411b7e0 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ddf40511b0 con 0x55ddf411bf20 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf40eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf40eb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43842/0 (socket says 172.21.15.59:43842) 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 -- 172.21.15.59:0/3809020493 learned_addr learned my addr 172.21.15.59:0/3809020493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 -- 172.21.15.59:0/3809020493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 msgr2=0x55ddf411b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 --2- 172.21.15.59:0/3809020493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 -- 172.21.15.59:0/3809020493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411b7e0 msgr2=0x55ddf40f05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 --2- 172.21.15.59:0/3809020493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411b7e0 0x55ddf40f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.162+0000 7f04c3e00640 1 -- 172.21.15.59:0/3809020493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf404ff50 con 0x55ddf4138e00 2024-04-23T14:37:12.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.163+0000 7f04c3e00640 1 --2- 172.21.15.59:0/3809020493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf40eb790 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f04b4076540 tx=0x7f04b4079a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e309bf3d1db139a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:12.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.163+0000 7f04b8c00640 1 -- 172.21.15.59:0/3809020493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b407a130 con 0x55ddf4138e00 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.163+0000 7f04b8c00640 1 -- 172.21.15.59:0/3809020493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b407e4e0 con 0x55ddf4138e00 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.163+0000 7f04b8c00640 1 -- 172.21.15.59:0/3809020493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4082d10 con 0x55ddf4138e00 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 -- 172.21.15.59:0/3809020493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 msgr2=0x55ddf40eb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/3809020493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf40eb790 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f04b4076540 tx=0x7f04b4079a90 comp rx=0 tx=0).stop 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 -- 172.21.15.59:0/3809020493 shutdown_connections 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/3809020493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/3809020493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411b7e0 0x55ddf40f05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/3809020493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf40eb790 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 -- 172.21.15.59:0/3809020493 >> 172.21.15.59:0/3809020493 conn(0x55ddf40eda00 msgr2=0x55ddf41362a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 -- 172.21.15.59:0/3809020493 shutdown_connections 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.164+0000 7f04cdc2e740 1 -- 172.21.15.59:0/3809020493 wait complete. 2024-04-23T14:37:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.165+0000 7f04cdc2e740 1 -- start start 2024-04-23T14:37:12.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.165+0000 7f04cdc2e740 1 Processor -- start 2024-04-23T14:37:12.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411bf20 0x55ddf4138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf410b6b0 0x55ddf410baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ddf40efbb0 con 0x55ddf4138e00 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ddf400e690 con 0x55ddf411bf20 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ddf40eefa0 con 0x55ddf410b6b0 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04cdc2e740 10 client.0 _ll_register_callbacks cb 0x55ddf404a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04c3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43856/0 (socket says 172.21.15.59:43856) 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.166+0000 7f04c3400640 1 -- 172.21.15.59:0/1752146370 learned_addr learned my addr 172.21.15.59:0/1752146370 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c3e00640 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411bf20 0x55ddf4138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c8e00640 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf410b6b0 0x55ddf410baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c3400640 1 -- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf410b6b0 msgr2=0x55ddf410baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c3400640 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf410b6b0 0x55ddf410baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c3400640 1 -- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411bf20 msgr2=0x55ddf4138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c3400640 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411bf20 0x55ddf4138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c3400640 1 -- 172.21.15.59:0/1752146370 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf400d920 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04c3400640 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411de80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f04c40a24e0 tx=0x7f04ac016a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f8f6dc804aebe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04b1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.167+0000 7f04b1600640 1 -- 172.21.15.59:0/1752146370 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac003970 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04b1600640 1 -- 172.21.15.59:0/1752146370 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f04ac003b10 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b407e4e0 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04b1600640 1 -- 172.21.15.59:0/1752146370 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f04ac003d10 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 10 client.7167 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55ddf3e317d0 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04b1600640 1 -- 172.21.15.59:0/1752146370 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b407e4e0 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04b1600640 1 -- 172.21.15.59:0/1752146370 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f04ac0050d0 con 0x55ddf4138e00 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 10 client.7167 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 10 client.7167 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 20 client.7167 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41464d0 0x55ddf411c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 4 client.7167 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:37:12.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.168+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf4062c10 con 0x55ddf41464d0 2024-04-23T14:37:12.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.169+0000 7f04c3e00640 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41464d0 0x55ddf411c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.169+0000 7f04c3e00640 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41464d0 0x55ddf411c410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53da675480037889 server_cookie=9b166b5a4d6cc249 in_seq=0 out_seq=0 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.169+0000 7f04b1600640 10 client.7167 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04b1600640 1 -- 172.21.15.59:0/1752146370 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddf4062c10 con 0x55ddf41464d0 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04b1600640 10 client.7167 handle_command_reply: tid=0 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 client.7167 shutdown 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 msgr2=0x55ddf411de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411de80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f04c40a24e0 tx=0x7f04ac016a90 comp rx=0 tx=0).stop 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 shutdown_connections 2024-04-23T14:37:12.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41464d0 0x55ddf411c410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf410b6b0 0x55ddf410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411de80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/1752146370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf411bf20 0x55ddf4138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 >> 172.21.15.59:0/1752146370 conn(0x55ddf411b7e0 msgr2=0x55ddf41373d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.170+0000 7f04b1600640 0 client.7167 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.171+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 shutdown_connections 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.171+0000 7f04cdc2e740 1 -- 172.21.15.59:0/1752146370 wait complete. 2024-04-23T14:37:12.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.171+0000 7f04cdc2e740 20 client.7167 trim_cache size 0 max 16384 2024-04-23T14:37:12.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.190+0000 7f04cdc2e740 1 Processor -- start 2024-04-23T14:37:12.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.190+0000 7f04cdc2e740 1 -- start start 2024-04-23T14:37:12.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf410b6b0 0x55ddf41375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf412e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf4138e00 0x55ddf412e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4205520 con 0x55ddf410b6b0 2024-04-23T14:37:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4204540 con 0x55ddf4138e00 2024-04-23T14:37:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f04c415a5e0 con 0x55ddf411bf20 2024-04-23T14:37:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf410b6b0 0x55ddf41375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf410b6b0 0x55ddf41375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43860/0 (socket says 172.21.15.59:43860) 2024-04-23T14:37:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.191+0000 7f04c3e00640 1 -- 172.21.15.59:0/220392112 learned_addr learned my addr 172.21.15.59:0/220392112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.192+0000 7f04c3e00640 1 -- 172.21.15.59:0/220392112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 msgr2=0x55ddf412e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.192+0000 7f04c3e00640 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf412e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.192+0000 7f04c8e00640 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf4138e00 0x55ddf412e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.192+0000 7f04c3e00640 1 -- 172.21.15.59:0/220392112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf4138e00 msgr2=0x55ddf412e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.192+0000 7f04c3e00640 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf4138e00 0x55ddf412e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.192+0000 7f04c3e00640 1 -- 172.21.15.59:0/220392112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf3d673f0 con 0x55ddf410b6b0 2024-04-23T14:37:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.192+0000 7f04c3e00640 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf410b6b0 0x55ddf41375f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f04b4076540 tx=0x7f04b407d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b303a8b99e6876ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04b3e00640 1 -- 172.21.15.59:0/220392112 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b407e4e0 con 0x55ddf410b6b0 2024-04-23T14:37:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04b3e00640 1 -- 172.21.15.59:0/220392112 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b40781d0 con 0x55ddf410b6b0 2024-04-23T14:37:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04b3e00640 1 -- 172.21.15.59:0/220392112 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4030d10 con 0x55ddf410b6b0 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 -- 172.21.15.59:0/220392112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf410b6b0 msgr2=0x55ddf41375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf410b6b0 0x55ddf41375f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f04b4076540 tx=0x7f04b407d000 comp rx=0 tx=0).stop 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 -- 172.21.15.59:0/220392112 shutdown_connections 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf412e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf4138e00 0x55ddf412e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/220392112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf410b6b0 0x55ddf41375f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 -- 172.21.15.59:0/220392112 >> 172.21.15.59:0/220392112 conn(0x55ddf411b7e0 msgr2=0x55ddf4115b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:12.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 -- 172.21.15.59:0/220392112 shutdown_connections 2024-04-23T14:37:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.193+0000 7f04cdc2e740 1 -- 172.21.15.59:0/220392112 wait complete. 2024-04-23T14:37:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.194+0000 7f04cdc2e740 1 -- start start 2024-04-23T14:37:12.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.194+0000 7f04cdc2e740 1 Processor -- start 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.195+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf410b6b0 0x55ddf4198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.195+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.195+0000 7f04cdc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.195+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40b6d80 con 0x55ddf4138e00 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.195+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ddf4062c10 con 0x55ddf410b6b0 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.195+0000 7f04cdc2e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ddf40eb270 con 0x55ddf411bf20 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.195+0000 7f04cdc2e740 10 client.0 _ll_register_callbacks cb 0x7f04ac001910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:12.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43862/0 (socket says 172.21.15.59:43862) 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 -- 172.21.15.59:0/2616771837 learned_addr learned my addr 172.21.15.59:0/2616771837 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c3e00640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf410b6b0 0x55ddf4198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c3400640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 -- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 msgr2=0x55ddf411e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 -- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf410b6b0 msgr2=0x55ddf4198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf410b6b0 0x55ddf4198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c8e00640 1 -- 172.21.15.59:0/2616771837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf3d270f0 con 0x55ddf4138e00 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.196+0000 7f04c3400640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04c8e00640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411eed0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f04bc012550 tx=0x7f04bc012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc71f906cb87c9a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04b1600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:12.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04b1600640 1 -- 172.21.15.59:0/2616771837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc01a070 con 0x55ddf4138e00 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04b1600640 1 -- 172.21.15.59:0/2616771837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f04bc014880 con 0x55ddf4138e00 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b40781d0 con 0x55ddf4138e00 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04b1600640 1 -- 172.21.15.59:0/2616771837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f04bc014a80 con 0x55ddf4138e00 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04cdc2e740 10 client.7173 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55ddf401a810 con 0x55ddf4138e00 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.197+0000 7f04b1600640 1 -- 172.21.15.59:0/2616771837 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b40781d0 con 0x55ddf4138e00 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04b1600640 1 -- 172.21.15.59:0/2616771837 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f04bc028020 con 0x55ddf4138e00 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04cdc2e740 10 client.7173 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04cdc2e740 10 client.7173 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04cdc2e740 20 client.7173 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41468b0 0x55ddf411c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04cdc2e740 4 client.7173 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ddf405ca80 con 0x55ddf41468b0 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04c3e00640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41468b0 0x55ddf411c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:12.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04c3e00640 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41468b0 0x55ddf411c410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a337a250a63f2e0 server_cookie=3b8586ae8884fa4b in_seq=0 out_seq=0 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.198+0000 7f04b1600640 10 client.7173 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04b1600640 1 -- 172.21.15.59:0/2616771837 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ddf405ca80 con 0x55ddf41468b0 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04b1600640 10 client.7173 handle_command_reply: tid=0 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 client.7173 shutdown 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 msgr2=0x55ddf411eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411eed0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f04bc012550 tx=0x7f04bc012a10 comp rx=0 tx=0).stop 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 shutdown_connections 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55ddf41468b0 0x55ddf411c410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ddf411bf20 0x55ddf411e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ddf4138e00 0x55ddf411eed0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 --2- 172.21.15.59:0/2616771837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ddf410b6b0 0x55ddf4198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04b1600640 0 client.7173 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 >> 172.21.15.59:0/2616771837 conn(0x55ddf411b7e0 msgr2=0x55ddf41396b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 shutdown_connections 2024-04-23T14:37:12.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 1 -- 172.21.15.59:0/2616771837 wait complete. 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.199+0000 7f04cdc2e740 20 client.7173 trim_cache size 0 max 16384 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: "e6c12528-9c9c-4dfa-9eed-67f64f686768": { 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/subdir", 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "e6c12528-9c9c-4dfa-9eed-67f64f686768", 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive" 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:37:12.209 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:37:12.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f04a006c8f0 msgr2=0x7f04a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f04a006c8f0 0x7f04a006edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0012e0 tx=0x7f04ac007d30 comp rx=0 tx=0).stop 2024-04-23T14:37:12.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c40b3600 msgr2=0x7f04c4155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c40b3600 0x7f04c4155a70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f04b402a170 tx=0x7f04b40046c0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 shutdown_connections 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f04a006c8f0 0x7f04a006edb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04c40d13e0 0x7f04c40b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04c415ed20 0x7f04c41563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 --2- 172.21.15.59:0/1411655393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c40b3600 0x7f04c4155a70 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 >> 172.21.15.59:0/1411655393 conn(0x7f04c408bf90 msgr2=0x7f04c40d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 shutdown_connections 2024-04-23T14:37:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:12.200+0000 7f04c9800640 1 -- 172.21.15.59:0/1411655393 wait complete. 2024-04-23T14:37:12.220 INFO:tasks.cephfs.filesystem:scrub status for tag:e6c12528-9c9c-4dfa-9eed-67f64f686768 - {'path': '/subdir', 'tag': 'e6c12528-9c9c-4dfa-9eed-67f64f686768', 'options': 'recursive'} 2024-04-23T14:37:20.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:37:20.035 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:37:42.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:42.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-23T14:37:42.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.450+0000 7fd67ca00640 1 Processor -- start 2024-04-23T14:37:42.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.451+0000 7fd67ca00640 1 -- start start 2024-04-23T14:37:42.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.452+0000 7fd67ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd67815ed20 0x7fd6780d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.452+0000 7fd67ca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6780b5da0 con 0x7fd6780d13e0 2024-04-23T14:37:42.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.452+0000 7fd67ca00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6780d0a20 con 0x7fd6780b3600 2024-04-23T14:37:42.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.452+0000 7fd67ca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd67815f100 con 0x7fd67815ed20 2024-04-23T14:37:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.453+0000 7fd677200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd6780d13e0 0x7fd6780b4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35886/0 (socket says 172.21.15.59:35886) 2024-04-23T14:37:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.453+0000 7fd677200640 1 -- 172.21.15.59:0/196828561 learned_addr learned my addr 172.21.15.59:0/196828561 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.453+0000 7fd676800640 1 --2- 172.21.15.59:0/196828561 >> v2:172.21.15.59:3300/0 conn(0x7fd67815ed20 0x7fd6780d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.453+0000 7fd675400640 1 -- 172.21.15.59:0/196828561 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151844960 0 0) 0x7fd6780b5da0 con 0x7fd6780d13e0 2024-04-23T14:37:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.453+0000 7fd676800640 1 -- 172.21.15.59:0/196828561 >> v1:172.21.15.76:6789/0 conn(0x7fd6780b3600 legacy=0x7fd6780b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.453+0000 7fd676800640 1 -- 172.21.15.59:0/196828561 >> v1:172.21.15.59:6789/0 conn(0x7fd6780d13e0 legacy=0x7fd6780b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.453+0000 7fd676800640 1 -- 172.21.15.59:0/196828561 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6780d2ed0 con 0x7fd67815ed20 2024-04-23T14:37:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.455+0000 7fd676800640 1 --2- 172.21.15.59:0/196828561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd67815ed20 0x7fd6780d0ea0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd6680097a0 tx=0x7fd66802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd54a45c8a96ae3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.455+0000 7fd675400640 1 -- 172.21.15.59:0/196828561 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668037070 con 0x7fd67815ed20 2024-04-23T14:37:42.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.455+0000 7fd675400640 1 -- 172.21.15.59:0/196828561 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668032440 con 0x7fd67815ed20 2024-04-23T14:37:42.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.455+0000 7fd675400640 1 -- 172.21.15.59:0/196828561 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66802ae60 con 0x7fd67815ed20 2024-04-23T14:37:42.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.455+0000 7fd67ca00640 1 -- 172.21.15.59:0/196828561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd67815ed20 msgr2=0x7fd6780d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.455+0000 7fd67ca00640 1 --2- 172.21.15.59:0/196828561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd67815ed20 0x7fd6780d0ea0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd6680097a0 tx=0x7fd66802a110 comp rx=0 tx=0).stop 2024-04-23T14:37:42.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.456+0000 7fd67ca00640 1 -- 172.21.15.59:0/196828561 shutdown_connections 2024-04-23T14:37:42.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.456+0000 7fd67ca00640 1 --2- 172.21.15.59:0/196828561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd67815ed20 0x7fd6780d0ea0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.456+0000 7fd67ca00640 1 -- 172.21.15.59:0/196828561 >> 172.21.15.59:0/196828561 conn(0x7fd67808bf90 msgr2=0x7fd6780d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.456+0000 7fd67ca00640 1 -- 172.21.15.59:0/196828561 shutdown_connections 2024-04-23T14:37:42.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.456+0000 7fd67ca00640 1 -- 172.21.15.59:0/196828561 wait complete. 2024-04-23T14:37:42.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.457+0000 7fd67ca00640 1 Processor -- start 2024-04-23T14:37:42.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.458+0000 7fd67ca00640 1 -- start start 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.458+0000 7fd67ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b3600 0x7fd6781f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.458+0000 7fd67ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6780d13e0 0x7fd678159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.458+0000 7fd67ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd67815ed20 0x7fd6780ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd67ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780b4d30 con 0x7fd6780d13e0 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd67ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780d3490 con 0x7fd6780b3600 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd67ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd6780b4eb0 con 0x7fd67815ed20 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6780d13e0 0x7fd678159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6780d13e0 0x7fd678159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47410/0 (socket says 172.21.15.59:47410) 2024-04-23T14:37:42.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 -- 172.21.15.59:0/2639346120 learned_addr learned my addr 172.21.15.59:0/2639346120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:42.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd677200640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd67815ed20 0x7fd6780ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd676800640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b3600 0x7fd6781f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 -- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd67815ed20 msgr2=0x7fd6780ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd67815ed20 0x7fd6780ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 -- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b3600 msgr2=0x7fd6781f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b3600 0x7fd6781f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd675e00640 1 -- 172.21.15.59:0/2639346120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6780cae20 con 0x7fd6780d13e0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.459+0000 7fd677200640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd67815ed20 0x7fd6780ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.460+0000 7fd675e00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6780d13e0 0x7fd678159e60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c790 tx=0x7fd66000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.460+0000 7fd66ea00640 1 -- 172.21.15.59:0/2639346120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660016020 con 0x7fd6780d13e0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.460+0000 7fd66ea00640 1 -- 172.21.15.59:0/2639346120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660006d50 con 0x7fd6780d13e0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.460+0000 7fd66ea00640 1 -- 172.21.15.59:0/2639346120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660003bb0 con 0x7fd6780d13e0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.460+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67815a5e0 con 0x7fd6780d13e0 2024-04-23T14:37:42.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.460+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67815aba0 con 0x7fd6780d13e0 2024-04-23T14:37:42.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.463+0000 7fd66ea00640 1 -- 172.21.15.59:0/2639346120 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd66000a850 con 0x7fd6780d13e0 2024-04-23T14:37:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.463+0000 7fd66ea00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd65406c920 0x7fd65406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.464+0000 7fd66ea00640 1 -- 172.21.15.59:0/2639346120 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fd66008b910 con 0x7fd6780d13e0 2024-04-23T14:37:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.464+0000 7fd676800640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd65406c920 0x7fd65406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.464+0000 7fd6803e5740 1 Processor -- start 2024-04-23T14:37:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.464+0000 7fd676800640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd65406c920 0x7fd65406ede0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd668034000 tx=0x7fd66802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.465+0000 7fd6803e5740 1 -- start start 2024-04-23T14:37:42.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.465+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d1c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.465+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.465+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4c7e0 0x55a5d4d215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.465+0000 7fd6803e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5d4c861f0 con 0x55a5d4d4c7e0 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.465+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a5d4c2c350 con 0x55a5d4d4cf20 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.465+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a5d4c821b0 con 0x55a5d4d69e00 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4c7e0 0x55a5d4d215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4c7e0 0x55a5d4d215f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47414/0 (socket says 172.21.15.59:47414) 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 -- 172.21.15.59:0/76235802 learned_addr learned my addr 172.21.15.59:0/76235802 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd676800640 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d1c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd675e00640 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 -- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 msgr2=0x55a5d4d1c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d1c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 -- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 msgr2=0x55a5d4d4c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 -- 172.21.15.59:0/76235802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d4c80f50 con 0x55a5d4d4c7e0 2024-04-23T14:37:42.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.466+0000 7fd677200640 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4c7e0 0x55a5d4d215f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd670001800 tx=0x7fd670001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dc4aca6bf592e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.467+0000 7fd667000640 1 -- 172.21.15.59:0/76235802 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700074e0 con 0x55a5d4d4c7e0 2024-04-23T14:37:42.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.467+0000 7fd667000640 1 -- 172.21.15.59:0/76235802 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6700047e0 con 0x55a5d4d4c7e0 2024-04-23T14:37:42.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.467+0000 7fd667000640 1 -- 172.21.15.59:0/76235802 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670004b90 con 0x55a5d4d4c7e0 2024-04-23T14:37:42.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.467+0000 7fd6803e5740 1 -- 172.21.15.59:0/76235802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4c7e0 msgr2=0x55a5d4d215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.467+0000 7fd6803e5740 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4c7e0 0x55a5d4d215f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd670001800 tx=0x7fd670001cc0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 -- 172.21.15.59:0/76235802 shutdown_connections 2024-04-23T14:37:42.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4c7e0 0x55a5d4d215f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 --2- 172.21.15.59:0/76235802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 -- 172.21.15.59:0/76235802 >> 172.21.15.59:0/76235802 conn(0x55a5d4d1ea00 msgr2=0x55a5d4d672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 -- 172.21.15.59:0/76235802 shutdown_connections 2024-04-23T14:37:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 -- 172.21.15.59:0/76235802 wait complete. 2024-04-23T14:37:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.468+0000 7fd6803e5740 1 -- start start 2024-04-23T14:37:42.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.469+0000 7fd6803e5740 1 Processor -- start 2024-04-23T14:37:42.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d22000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4d410 0x55a5d4d22540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd6803e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a5d4d20bb0 con 0x55a5d4d4cf20 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a5d4c3f690 con 0x55a5d4d4d410 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a5d4d1ffa0 con 0x55a5d4d69e00 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd6803e5740 10 client.0 _ll_register_callbacks cb 0x55a5d4c7b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd676800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd676800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47428/0 (socket says 172.21.15.59:47428) 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.470+0000 7fd676800640 1 -- 172.21.15.59:0/1779267501 learned_addr learned my addr 172.21.15.59:0/1779267501 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd677200640 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4d410 0x55a5d4d22540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd675e00640 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d22000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd676800640 1 -- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 msgr2=0x55a5d4d22000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd676800640 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d22000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd676800640 1 -- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4d410 msgr2=0x55a5d4d22540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd676800640 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4d410 0x55a5d4d22540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd676800640 1 -- 172.21.15.59:0/1779267501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d4c3e920 con 0x55a5d4d4cf20 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd676800640 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4ff30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd6680098c0 tx=0x7fd66802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dff576e15c36e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd65f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd65f400640 1 -- 172.21.15.59:0/1779267501 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668037050 con 0x55a5d4d4cf20 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd65f400640 1 -- 172.21.15.59:0/1779267501 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fd668032850 con 0x55a5d4d4cf20 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.471+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6700047e0 con 0x55a5d4d4cf20 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd65f400640 1 -- 172.21.15.59:0/1779267501 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd668057050 con 0x55a5d4d4cf20 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 10 client.7188 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:42.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55a5d4a627d0 con 0x55a5d4d4cf20 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd65f400640 1 -- 172.21.15.59:0/1779267501 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6700047e0 con 0x55a5d4d4cf20 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd65f400640 1 -- 172.21.15.59:0/1779267501 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fd66805a020 con 0x55a5d4d4cf20 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 10 client.7188 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 10 client.7188 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 20 client.7188 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d4a730 0x55a5d4d4ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 4 client.7188 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.472+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d4c93c10 con 0x55a5d4d4a730 2024-04-23T14:37:42.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.473+0000 7fd675e00640 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d4a730 0x55a5d4d4ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.473+0000 7fd675e00640 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d4a730 0x55a5d4d4ab90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294beb7dc973f86f server_cookie=af1b230900c87c68 in_seq=0 out_seq=0 2024-04-23T14:37:42.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.473+0000 7fd65f400640 10 client.7188 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:42.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd65f400640 1 -- 172.21.15.59:0/1779267501 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5d4c93c10 con 0x55a5d4d4a730 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd65f400640 10 client.7188 handle_command_reply: tid=0 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 client.7188 shutdown 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 msgr2=0x55a5d4d4ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4ff30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd6680098c0 tx=0x7fd66802c070 comp rx=0 tx=0).stop 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 shutdown_connections 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d4a730 0x55a5d4d4ab90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d22000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4d410 0x55a5d4d22540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 --2- 172.21.15.59:0/1779267501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d4ff30 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd65f400640 0 client.7188 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 >> 172.21.15.59:0/1779267501 conn(0x55a5d4d4c7e0 msgr2=0x55a5d4d672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 shutdown_connections 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.474+0000 7fd6803e5740 1 -- 172.21.15.59:0/1779267501 wait complete. 2024-04-23T14:37:42.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.475+0000 7fd6803e5740 20 client.7188 trim_cache size 0 max 16384 2024-04-23T14:37:42.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.494+0000 7fd6803e5740 1 Processor -- start 2024-04-23T14:37:42.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.495+0000 7fd6803e5740 1 -- start start 2024-04-23T14:37:42.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.495+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d6a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.495+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4d410 0x55a5d4d1b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.495+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d77750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.495+0000 7fd6803e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd678200ea0 con 0x55a5d4d4d410 2024-04-23T14:37:42.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.495+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781e5020 con 0x55a5d4d4cf20 2024-04-23T14:37:42.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.495+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd6781f87b0 con 0x55a5d4d69e00 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4d410 0x55a5d4d1b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4d410 0x55a5d4d1b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47430/0 (socket says 172.21.15.59:47430) 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 -- 172.21.15.59:0/64804134 learned_addr learned my addr 172.21.15.59:0/64804134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd677200640 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d77750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd676800640 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d6a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 -- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 msgr2=0x55a5d4d77750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d77750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 -- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 msgr2=0x55a5d4d6a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d6a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 -- 172.21.15.59:0/64804134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d49983f0 con 0x55a5d4d4d410 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.496+0000 7fd675e00640 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4d410 0x55a5d4d1b930 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd660007f20 tx=0x7fd660001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb857bd6c359b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd666600640 1 -- 172.21.15.59:0/64804134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660076070 con 0x55a5d4d4d410 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd666600640 1 -- 172.21.15.59:0/64804134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66005d950 con 0x55a5d4d4d410 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd666600640 1 -- 172.21.15.59:0/64804134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66008b040 con 0x55a5d4d4d410 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd6803e5740 1 -- 172.21.15.59:0/64804134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4d410 msgr2=0x55a5d4d1b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd6803e5740 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4d410 0x55a5d4d1b930 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd660007f20 tx=0x7fd660001250 comp rx=0 tx=0).stop 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd6803e5740 1 -- 172.21.15.59:0/64804134 shutdown_connections 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd6803e5740 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d69e00 0x55a5d4d77750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd6803e5740 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4d410 0x55a5d4d1b930 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd6803e5740 1 --2- 172.21.15.59:0/64804134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d4cf20 0x55a5d4d6a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.497+0000 7fd6803e5740 1 -- 172.21.15.59:0/64804134 >> 172.21.15.59:0/64804134 conn(0x55a5d4d4c7e0 msgr2=0x55a5d4d670a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.498+0000 7fd6803e5740 1 -- 172.21.15.59:0/64804134 shutdown_connections 2024-04-23T14:37:42.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.498+0000 7fd6803e5740 1 -- 172.21.15.59:0/64804134 wait complete. 2024-04-23T14:37:42.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.498+0000 7fd6803e5740 1 -- start start 2024-04-23T14:37:42.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.499+0000 7fd6803e5740 1 Processor -- start 2024-04-23T14:37:42.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.499+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4de0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d4d410 0x55a5d4d779d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd6803e5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d69e00 0x55a5d4d77f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd6803e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781e49d0 con 0x55a5d4d4cf20 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a5d4d4ac40 con 0x55a5d4d69e00 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd6803e5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a5d4d49720 con 0x55a5d4d4d410 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd6803e5740 10 client.0 _ll_register_callbacks cb 0x7fd668035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4de0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4de0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47444/0 (socket says 172.21.15.59:47444) 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 -- 172.21.15.59:0/2135859079 learned_addr learned my addr 172.21.15.59:0/2135859079 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 -- 172.21.15.59:0/2135859079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d4d410 msgr2=0x55a5d4d779d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d4d410 0x55a5d4d779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd677200640 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d69e00 0x55a5d4d77f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 -- 172.21.15.59:0/2135859079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d69e00 msgr2=0x55a5d4d77f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d69e00 0x55a5d4d77f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.500+0000 7fd676800640 1 -- 172.21.15.59:0/2135859079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d49580f0 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd676800640 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4de0820 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd668009890 tx=0x7fd668004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7730c3a668641c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd65f400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd65f400640 1 -- 172.21.15.59:0/2135859079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66805f070 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd65f400640 1 -- 172.21.15.59:0/2135859079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fd66803f630 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd66005d950 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd65f400640 1 -- 172.21.15.59:0/2135859079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd66806a050 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd6803e5740 10 client.7194 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.501+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55a5d4c4b810 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd65f400640 1 -- 172.21.15.59:0/2135859079 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66005d950 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd65f400640 1 -- 172.21.15.59:0/2135859079 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fd66806d020 con 0x55a5d4d4cf20 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd6803e5740 10 client.7194 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd6803e5740 10 client.7194 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd6803e5740 20 client.7194 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:42.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd6803e5740 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d1c2d0 0x55a5d4d48800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd6803e5740 4 client.7194 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:37:42.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5d4c8da80 con 0x55a5d4d1c2d0 2024-04-23T14:37:42.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.502+0000 7fd675e00640 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d1c2d0 0x55a5d4d48800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd675e00640 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d1c2d0 0x55a5d4d48800 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33bd32e4c5f5084 server_cookie=dc28118a249263da in_seq=0 out_seq=0 2024-04-23T14:37:42.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd65f400640 10 client.7194 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd65f400640 1 -- 172.21.15.59:0/2135859079 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a5d4c8da80 con 0x55a5d4d1c2d0 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd65f400640 10 client.7194 handle_command_reply: tid=0 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 client.7194 shutdown 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 msgr2=0x55a5d4de0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4de0820 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd668009890 tx=0x7fd668004730 comp rx=0 tx=0).stop 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 shutdown_connections 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x55a5d4d1c2d0 0x55a5d4d48800 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a5d4d4d410 0x55a5d4d779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a5d4d69e00 0x55a5d4d77f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 --2- 172.21.15.59:0/2135859079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a5d4d4cf20 0x55a5d4de0820 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd65f400640 0 client.7194 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:42.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 >> 172.21.15.59:0/2135859079 conn(0x55a5d4d4c7e0 msgr2=0x55a5d4d6a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 shutdown_connections 2024-04-23T14:37:42.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 1 -- 172.21.15.59:0/2135859079 wait complete. 2024-04-23T14:37:42.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.503+0000 7fd6803e5740 20 client.7194 trim_cache size 0 max 16384 2024-04-23T14:37:42.509 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:37:42.509 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:37:42.509 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:37:42.509 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:37:42.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd65406c920 msgr2=0x7fd65406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd65406c920 0x7fd65406ede0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd668034000 tx=0x7fd66802e000 comp rx=0 tx=0).stop 2024-04-23T14:37:42.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6780d13e0 msgr2=0x7fd678159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6780d13e0 0x7fd678159e60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c790 tx=0x7fd66000cc50 comp rx=0 tx=0).stop 2024-04-23T14:37:42.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 shutdown_connections 2024-04-23T14:37:42.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd65406c920 0x7fd65406ede0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd67815ed20 0x7fd6780ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6780d13e0 0x7fd678159e60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 --2- 172.21.15.59:0/2639346120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6780b3600 0x7fd6781f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 >> 172.21.15.59:0/2639346120 conn(0x7fd67808bf90 msgr2=0x7fd6780d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 shutdown_connections 2024-04-23T14:37:42.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.504+0000 7fd67ca00640 1 -- 172.21.15.59:0/2639346120 wait complete. 2024-04-23T14:37:42.523 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:37:42.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:37:42.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.724+0000 7fee39800640 1 Processor -- start 2024-04-23T14:37:42.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.725+0000 7fee39800640 1 -- start start 2024-04-23T14:37:42.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.726+0000 7fee39800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee340d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.726+0000 7fee39800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fee340d2630 0x7fee340b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.726+0000 7fee39800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee340cf030 con 0x7fee340b6530 2024-04-23T14:37:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.726+0000 7fee39800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fee340b7000 con 0x7fee340d2630 2024-04-23T14:37:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.726+0000 7fee39800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee340cf620 con 0x7fee3415e7c0 2024-04-23T14:37:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.727+0000 7fee33e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee340d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.727+0000 7fee33400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fee340d2630 0x7fee340b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.727+0000 7fee33e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee340d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:36652/0 (socket says 172.21.15.59:36652) 2024-04-23T14:37:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.727+0000 7fee33e00640 1 -- 172.21.15.59:0/4227810930 learned_addr learned my addr 172.21.15.59:0/4227810930 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.728+0000 7fee33400640 1 -- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 msgr2=0x7fee340d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.728+0000 7fee33400640 1 --2- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee340d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.728+0000 7fee33400640 1 -- 172.21.15.59:0/4227810930 >> v1:172.21.15.76:6789/0 conn(0x7fee340b6530 legacy=0x7fee340d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.728+0000 7fee33400640 1 -- 172.21.15.59:0/4227810930 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee340b6a40 con 0x7fee340d2630 2024-04-23T14:37:42.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.728+0000 7fee33e00640 1 --2- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee340d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:37:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.729+0000 7fee33400640 1 --2- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 0x7fee340b5ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0097a0 tx=0x7fee1c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7c1befaa2f53fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.730+0000 7fee32a00640 1 -- 172.21.15.59:0/4227810930 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c037070 con 0x7fee340d2630 2024-04-23T14:37:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.730+0000 7fee32a00640 1 -- 172.21.15.59:0/4227810930 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1c032440 con 0x7fee340d2630 2024-04-23T14:37:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.730+0000 7fee32a00640 1 -- 172.21.15.59:0/4227810930 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c032810 con 0x7fee340d2630 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.730+0000 7fee39800640 1 -- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 msgr2=0x7fee340b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.730+0000 7fee39800640 1 --2- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 0x7fee340b5ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0097a0 tx=0x7fee1c02a180 comp rx=0 tx=0).stop 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.731+0000 7fee39800640 1 -- 172.21.15.59:0/4227810930 shutdown_connections 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.731+0000 7fee39800640 1 --2- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 0x7fee340b5ff0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.731+0000 7fee39800640 1 --2- 172.21.15.59:0/4227810930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee340d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.731+0000 7fee39800640 1 -- 172.21.15.59:0/4227810930 >> 172.21.15.59:0/4227810930 conn(0x7fee3409d920 msgr2=0x7fee3408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.731+0000 7fee39800640 1 -- 172.21.15.59:0/4227810930 shutdown_connections 2024-04-23T14:37:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.731+0000 7fee39800640 1 -- 172.21.15.59:0/4227810930 wait complete. 2024-04-23T14:37:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.733+0000 7fee39800640 1 Processor -- start 2024-04-23T14:37:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.733+0000 7fee39800640 1 -- start start 2024-04-23T14:37:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee39800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee340b6530 0x7fee34160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee39800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 0x7fee34153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee39800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee34161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee39800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee340d1a80 con 0x7fee340b6530 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee39800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee340cf2b0 con 0x7fee340d2630 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee39800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee340d1c00 con 0x7fee3415e7c0 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee33e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee340b6530 0x7fee34160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee33e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee340b6530 0x7fee34160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47456/0 (socket says 172.21.15.59:47456) 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee33e00640 1 -- 172.21.15.59:0/3978531111 learned_addr learned my addr 172.21.15.59:0/3978531111 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee38e00640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee34161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee33400640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 0x7fee34153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee33e00640 1 -- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 msgr2=0x7fee34161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.735+0000 7fee33e00640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee34161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee33e00640 1 -- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 msgr2=0x7fee34153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee33e00640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 0x7fee34153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee33e00640 1 -- 172.21.15.59:0/3978531111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee34159e20 con 0x7fee340b6530 2024-04-23T14:37:42.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee33e00640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee340b6530 0x7fee34160ca0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fee2400efc0 tx=0x7fee2400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee2be00640 1 -- 172.21.15.59:0/3978531111 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2400ce60 con 0x7fee340b6530 2024-04-23T14:37:42.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee2be00640 1 -- 172.21.15.59:0/3978531111 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee24019070 con 0x7fee340b6530 2024-04-23T14:37:42.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3415fee0 con 0x7fee340b6530 2024-04-23T14:37:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee2be00640 1 -- 172.21.15.59:0/3978531111 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24014680 con 0x7fee340b6530 2024-04-23T14:37:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.736+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee341604a0 con 0x7fee340b6530 2024-04-23T14:37:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.737+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3404fd90 con 0x7fee340b6530 2024-04-23T14:37:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.739+0000 7fee2be00640 1 -- 172.21.15.59:0/3978531111 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee24005d10 con 0x7fee340b6530 2024-04-23T14:37:42.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.739+0000 7fee2be00640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.740+0000 7fee2be00640 1 -- 172.21.15.59:0/3978531111 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fee2408c0d0 con 0x7fee340b6530 2024-04-23T14:37:42.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.740+0000 7fee33400640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.741+0000 7fee33400640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee1006c8f0 0x7fee1006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fee1c034000 tx=0x7fee1c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.745+0000 7fee2be00640 1 -- 172.21.15.59:0/3978531111 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee24010040 con 0x7fee340b6530 2024-04-23T14:37:42.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.994+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee340d1eb0 con 0x7fee340b6530 2024-04-23T14:37:42.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.995+0000 7fee2be00640 1 -- 172.21.15.59:0/3978531111 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5184 (secure 0 0 0) 0x7fee2405a460 con 0x7fee340b6530 2024-04-23T14:37:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:37:42.997 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:37:01.867542+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":5}]} 2024-04-23T14:37:42.997 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 50 2024-04-23T14:37:42.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee1006c8f0 msgr2=0x7fee1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee1006c8f0 0x7fee1006edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fee1c034000 tx=0x7fee1c02e000 comp rx=0 tx=0).stop 2024-04-23T14:37:42.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee340b6530 msgr2=0x7fee34160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee340b6530 0x7fee34160ca0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fee2400efc0 tx=0x7fee2400c490 comp rx=0 tx=0).stop 2024-04-23T14:37:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 shutdown_connections 2024-04-23T14:37:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee3415e7c0 0x7fee34161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee340d2630 0x7fee34153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 --2- 172.21.15.59:0/3978531111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee340b6530 0x7fee34160ca0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.998+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 >> 172.21.15.59:0/3978531111 conn(0x7fee3409d920 msgr2=0x7fee3415c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.999+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 shutdown_connections 2024-04-23T14:37:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:42.999+0000 7fee39800640 1 -- 172.21.15.59:0/3978531111 wait complete. 2024-04-23T14:37:43.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-23T14:37:43.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.226+0000 7fc675a00640 1 Processor -- start 2024-04-23T14:37:43.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.226+0000 7fc675a00640 1 -- start start 2024-04-23T14:37:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.228+0000 7fc675a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc670072c80 0x7fc6700af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.228+0000 7fc675a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc670160120 con 0x7fc67015c410 2024-04-23T14:37:43.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.228+0000 7fc675a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc670160400 con 0x7fc6700d0ce0 2024-04-23T14:37:43.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.228+0000 7fc675a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc6701606e0 con 0x7fc670072c80 2024-04-23T14:37:43.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.228+0000 7fc66fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc67015c410 0x7fc6700d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35898/0 (socket says 172.21.15.59:35898) 2024-04-23T14:37:43.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.228+0000 7fc66fe00640 1 -- 172.21.15.59:0/1460416337 learned_addr learned my addr 172.21.15.59:0/1460416337 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:43.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc675000640 1 --2- 172.21.15.59:0/1460416337 >> v2:172.21.15.76:3300/0 conn(0x7fc670072c80 0x7fc6700af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525141659 0 0) 0x7fc670160120 con 0x7fc67015c410 2024-04-23T14:37:43.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc654003520 con 0x7fc67015c410 2024-04-23T14:37:43.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1973438192 0 0) 0x7fc654003520 con 0x7fc67015c410 2024-04-23T14:37:43.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 >> v2:172.21.15.76:3300/0 conn(0x7fc670072c80 msgr2=0x7fc6700af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 --2- 172.21.15.59:0/1460416337 >> v2:172.21.15.76:3300/0 conn(0x7fc670072c80 0x7fc6700af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 >> v1:172.21.15.76:6789/0 conn(0x7fc6700d0ce0 legacy=0x7fc6700af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6701609c0 con 0x7fc67015c410 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fc6600019f0 con 0x7fc67015c410 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 >> v1:172.21.15.59:6789/0 conn(0x7fc67015c410 legacy=0x7fc6700d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6540039b0 0x7fc654003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6540051c0 0x7fc65400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701606e0 con 0x7fc6540039b0 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc65400c040 con 0x7fc654004800 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.229+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc65400c240 con 0x7fc6540051c0 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc66fe00640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6540039b0 0x7fc654003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc675000640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6540051c0 0x7fc65400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc66fe00640 1 -- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6540051c0 msgr2=0x7fc65400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc66fe00640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6540051c0 0x7fc65400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc667400640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc66fe00640 1 -- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654004800 msgr2=0x7fc654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc66fe00640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc675000640 1 -- 172.21.15.59:0/1460416337 reap_dead start 2024-04-23T14:37:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc66fe00640 1 -- 172.21.15.59:0/1460416337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc654003520 con 0x7fc6540039b0 2024-04-23T14:37:43.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.230+0000 7fc667400640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc654004800 0x7fc654004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.231+0000 7fc66fe00640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6540039b0 0x7fc654003db0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc66000a860 tx=0x7fc654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc61632289e4c204 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.231+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66000a480 con 0x7fc6540039b0 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.231+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66002f070 con 0x7fc6540039b0 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.231+0000 7fc66f400640 1 -- 172.21.15.59:0/1460416337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66002bcd0 con 0x7fc6540039b0 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.232+0000 7fc675a00640 1 -- 172.21.15.59:0/1460416337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6540039b0 msgr2=0x7fc654003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.232+0000 7fc675a00640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6540039b0 0x7fc654003db0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc66000a860 tx=0x7fc654005830 comp rx=0 tx=0).stop 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.232+0000 7fc675a00640 1 -- 172.21.15.59:0/1460416337 shutdown_connections 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.232+0000 7fc675a00640 1 --2- 172.21.15.59:0/1460416337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6540039b0 0x7fc654003db0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.232+0000 7fc675a00640 1 -- 172.21.15.59:0/1460416337 >> 172.21.15.59:0/1460416337 conn(0x7fc67006bdf0 msgr2=0x7fc67006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.232+0000 7fc675a00640 1 -- 172.21.15.59:0/1460416337 shutdown_connections 2024-04-23T14:37:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.232+0000 7fc675a00640 1 -- 172.21.15.59:0/1460416337 wait complete. 2024-04-23T14:37:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.234+0000 7fc675a00640 1 Processor -- start 2024-04-23T14:37:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.234+0000 7fc675a00640 1 -- start start 2024-04-23T14:37:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc675a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6540039b0 0x7fc6700cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc675a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6701f4cd0 0x7fc6700c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc675a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6700b2610 0x7fc6700cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc675a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6700d0750 con 0x7fc6701f4cd0 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc675a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6700d05d0 con 0x7fc6540039b0 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc675a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc6700d08d0 con 0x7fc6700b2610 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc667400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6701f4cd0 0x7fc6700c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.235+0000 7fc667400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6701f4cd0 0x7fc6700c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47478/0 (socket says 172.21.15.59:47478) 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc667400640 1 -- 172.21.15.59:0/2222443797 learned_addr learned my addr 172.21.15.59:0/2222443797 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc66fe00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6540039b0 0x7fc6700cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc675000640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6700b2610 0x7fc6700cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc667400640 1 -- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6700b2610 msgr2=0x7fc6700cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc667400640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6700b2610 0x7fc6700cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc667400640 1 -- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6540039b0 msgr2=0x7fc6700cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc667400640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6540039b0 0x7fc6700cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc667400640 1 -- 172.21.15.59:0/2222443797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6700cf230 con 0x7fc6701f4cd0 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc66fe00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6540039b0 0x7fc6700cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc667400640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6701f4cd0 0x7fc6700c9ee0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc65800d460 tx=0x7fc65800d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc66cc00640 1 -- 172.21.15.59:0/2222443797 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658016030 con 0x7fc6701f4cd0 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.236+0000 7fc66cc00640 1 -- 172.21.15.59:0/2222443797 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc658007020 con 0x7fc6701f4cd0 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.237+0000 7fc66cc00640 1 -- 172.21.15.59:0/2222443797 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658007410 con 0x7fc6701f4cd0 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.237+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6700ca570 con 0x7fc6701f4cd0 2024-04-23T14:37:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.237+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6700cab30 con 0x7fc6701f4cd0 2024-04-23T14:37:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.239+0000 7fc66cc00640 1 -- 172.21.15.59:0/2222443797 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc658019070 con 0x7fc6701f4cd0 2024-04-23T14:37:43.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.240+0000 7fc66cc00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc64c06c8f0 0x7fc64c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.240+0000 7fc66cc00640 1 -- 172.21.15.59:0/2222443797 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fc65808b950 con 0x7fc6701f4cd0 2024-04-23T14:37:43.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.240+0000 7fc66fe00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc64c06c8f0 0x7fc64c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.240+0000 7fc679f77740 1 Processor -- start 2024-04-23T14:37:43.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.241+0000 7fc679f77740 1 -- start start 2024-04-23T14:37:43.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.241+0000 7fc66fe00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc64c06c8f0 0x7fc64c06edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc660033000 tx=0x7fc660007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.241+0000 7fc679f77740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e1e220 0x561e77e1e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.241+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e3c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77df3f00 0x561e77e3a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc679f77740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561e77d58190 con 0x561e77e1e220 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561e77cfe030 con 0x561e77e3bac0 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x561e77d54150 con 0x561e77df3f00 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e1e220 0x561e77e1e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e1e220 0x561e77e1e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47480/0 (socket says 172.21.15.59:47480) 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 -- 172.21.15.59:0/2402160632 learned_addr learned my addr 172.21.15.59:0/2402160632 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc675000640 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77df3f00 0x561e77e3a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc667400640 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e3c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 -- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77df3f00 msgr2=0x561e77e3a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77df3f00 0x561e77e3a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 -- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 msgr2=0x561e77e3c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e3c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.242+0000 7fc66fe00640 1 -- 172.21.15.59:0/2402160632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e77ce6b60 con 0x561e77e1e220 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.243+0000 7fc66fe00640 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e1e220 0x561e77e1e600 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc66002b4c0 tx=0x7fc66002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20e0abb078f7e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.243+0000 7fc664c00640 1 -- 172.21.15.59:0/2402160632 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66002be20 con 0x561e77e1e220 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.243+0000 7fc664c00640 1 -- 172.21.15.59:0/2402160632 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66002f040 con 0x561e77e1e220 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.243+0000 7fc664c00640 1 -- 172.21.15.59:0/2402160632 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660035bd0 con 0x561e77e1e220 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 -- 172.21.15.59:0/2402160632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e1e220 msgr2=0x561e77e1e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e1e220 0x561e77e1e600 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc66002b4c0 tx=0x7fc66002d000 comp rx=0 tx=0).stop 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 -- 172.21.15.59:0/2402160632 shutdown_connections 2024-04-23T14:37:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77df3f00 0x561e77e3a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e3c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 --2- 172.21.15.59:0/2402160632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e1e220 0x561e77e1e600 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 -- 172.21.15.59:0/2402160632 >> 172.21.15.59:0/2402160632 conn(0x561e77df0830 msgr2=0x561e77e39030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 -- 172.21.15.59:0/2402160632 shutdown_connections 2024-04-23T14:37:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.244+0000 7fc679f77740 1 -- 172.21.15.59:0/2402160632 wait complete. 2024-04-23T14:37:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.245+0000 7fc679f77740 1 -- start start 2024-04-23T14:37:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.245+0000 7fc679f77740 1 Processor -- start 2024-04-23T14:37:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e1e220 0x561e77e36480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc679f77740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc679f77740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561e77df4ca0 con 0x561e77e31ac0 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561e77d11500 con 0x561e77e3bac0 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x561e77dee760 con 0x561e77e1e220 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc679f77740 10 client.0 _ll_register_callbacks cb 0x561e77d4e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc675000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc675000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e48f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47492/0 (socket says 172.21.15.59:47492) 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.246+0000 7fc675000640 1 -- 172.21.15.59:0/1563604620 learned_addr learned my addr 172.21.15.59:0/1563604620 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc675000640 1 -- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e1e220 msgr2=0x561e77e36480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc66fe00640 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e1e220 0x561e77e36480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc667400640 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc675000640 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e1e220 0x561e77e36480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc675000640 1 -- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 msgr2=0x561e77e489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc675000640 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc675000640 1 -- 172.21.15.59:0/1563604620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e77b367c0 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc675000640 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e48f20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc668009a10 tx=0x7fc668009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6221f307bd693b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc65d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc65d600640 1 -- 172.21.15.59:0/1563604620 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66801a070 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc65d600640 1 -- 172.21.15.59:0/1563604620 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fc668013960 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.247+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc66002f040 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc65d600640 1 -- 172.21.15.59:0/1563604620 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc668013b60 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 10 client.7209 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x561e77b96f50 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc65d600640 1 -- 172.21.15.59:0/1563604620 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66002f040 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc65d600640 1 -- 172.21.15.59:0/1563604620 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fc668013d00 con 0x561e77e31ac0 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 10 client.7209 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 10 client.7209 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:37:43.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 20 client.7209 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e4d190 0x561e77e34b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 4 client.7209 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e77d65bb0 con 0x561e77e4d190 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.248+0000 7fc66fe00640 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e4d190 0x561e77e34b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.249+0000 7fc66fe00640 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e4d190 0x561e77e34b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b56561595400db server_cookie=4db1c461ece2409f in_seq=0 out_seq=0 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.249+0000 7fc65d600640 10 client.7209 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc65d600640 1 -- 172.21.15.59:0/1563604620 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e77d65bb0 con 0x561e77e4d190 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc65d600640 10 client.7209 handle_command_reply: tid=0 2024-04-23T14:37:43.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 client.7209 shutdown 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 msgr2=0x561e77e48f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e48f20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc668009a10 tx=0x7fc668009ed0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 shutdown_connections 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e4d190 0x561e77e34b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e48f20 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e3bac0 0x561e77e489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 --2- 172.21.15.59:0/1563604620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e1e220 0x561e77e36480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 >> 172.21.15.59:0/1563604620 conn(0x561e77df3f00 msgr2=0x561e77e3ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc65d600640 0 client.7209 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 shutdown_connections 2024-04-23T14:37:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 1 -- 172.21.15.59:0/1563604620 wait complete. 2024-04-23T14:37:43.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.250+0000 7fc679f77740 20 client.7209 trim_cache size 0 max 16384 2024-04-23T14:37:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.265+0000 7fc679f77740 1 Processor -- start 2024-04-23T14:37:43.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.265+0000 7fc679f77740 1 -- start start 2024-04-23T14:37:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 0x561e77e20ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc679f77740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e3bac0 0x561e77e2e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc679f77740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6702003b0 con 0x561e77e31ac0 2024-04-23T14:37:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6701ff710 con 0x561e77e1e220 2024-04-23T14:37:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc6701ffd60 con 0x561e77e3bac0 2024-04-23T14:37:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc667400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc667400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47494/0 (socket says 172.21.15.59:47494) 2024-04-23T14:37:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.266+0000 7fc667400640 1 -- 172.21.15.59:0/3888654973 learned_addr learned my addr 172.21.15.59:0/3888654973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc667400640 1 -- 172.21.15.59:0/3888654973 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e3bac0 msgr2=0x561e77e2e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc667400640 1 --2- 172.21.15.59:0/3888654973 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e3bac0 0x561e77e2e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc667400640 1 -- 172.21.15.59:0/3888654973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 msgr2=0x561e77e20ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc667400640 1 --2- 172.21.15.59:0/3888654973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 0x561e77e20ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc667400640 1 -- 172.21.15.59:0/3888654973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e77a2a0b0 con 0x561e77e31ac0 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc667400640 1 --2- 172.21.15.59:0/3888654973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e485d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc658059250 tx=0x7fc658058fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9279827584e43841 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc65fe00640 1 -- 172.21.15.59:0/3888654973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6580575f0 con 0x561e77e31ac0 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc65fe00640 1 -- 172.21.15.59:0/3888654973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc658071070 con 0x561e77e31ac0 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.267+0000 7fc65fe00640 1 -- 172.21.15.59:0/3888654973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658057850 con 0x561e77e31ac0 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 -- 172.21.15.59:0/3888654973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 msgr2=0x561e77e485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 --2- 172.21.15.59:0/3888654973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e485d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc658059250 tx=0x7fc658058fe0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 -- 172.21.15.59:0/3888654973 shutdown_connections 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 --2- 172.21.15.59:0/3888654973 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e3bac0 0x561e77e2e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 --2- 172.21.15.59:0/3888654973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e31ac0 0x561e77e485d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 --2- 172.21.15.59:0/3888654973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 0x561e77e20ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 -- 172.21.15.59:0/3888654973 >> 172.21.15.59:0/3888654973 conn(0x561e77df3f00 msgr2=0x561e77df42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 -- 172.21.15.59:0/3888654973 shutdown_connections 2024-04-23T14:37:43.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.268+0000 7fc679f77740 1 -- 172.21.15.59:0/3888654973 wait complete. 2024-04-23T14:37:43.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.269+0000 7fc679f77740 1 -- start start 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.269+0000 7fc679f77740 1 Processor -- start 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 0x561e77ded370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc679f77740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e31ac0 0x561e77dedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc679f77740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e3bac0 0x561e77e485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc679f77740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561e77e4d010 con 0x561e77e3bac0 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561e77e33c00 con 0x561e77e1e220 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc679f77740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x561e77e4d660 con 0x561e77e31ac0 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc679f77740 10 client.0 _ll_register_callbacks cb 0x7fc668006c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc675000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e3bac0 0x561e77e485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc675000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e3bac0 0x561e77e485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47498/0 (socket says 172.21.15.59:47498) 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.270+0000 7fc675000640 1 -- 172.21.15.59:0/3151055789 learned_addr learned my addr 172.21.15.59:0/3151055789 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc667400640 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e31ac0 0x561e77dedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc66fe00640 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 0x561e77ded370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc675000640 1 -- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e31ac0 msgr2=0x561e77dedd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc675000640 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e31ac0 0x561e77dedd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc675000640 1 -- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 msgr2=0x561e77ded370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc675000640 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 0x561e77ded370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc675000640 1 -- 172.21.15.59:0/3151055789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e77e537c0 con 0x561e77e3bac0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc675000640 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e3bac0 0x561e77e485d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc668013a40 tx=0x7fc6680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347c958eec8c2179 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc65d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc65d600640 1 -- 172.21.15.59:0/3151055789 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66801a050 con 0x561e77e3bac0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc65d600640 1 -- 172.21.15.59:0/3151055789 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fc668005b90 con 0x561e77e3bac0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc658071070 con 0x561e77e3bac0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.271+0000 7fc65d600640 1 -- 172.21.15.59:0/3151055789 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc668025020 con 0x561e77e3bac0 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 10 client.7215 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x561e77e215e0 con 0x561e77e3bac0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc65d600640 1 -- 172.21.15.59:0/3151055789 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc658071070 con 0x561e77e3bac0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc65d600640 1 -- 172.21.15.59:0/3151055789 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fc66803b020 con 0x561e77e3bac0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 10 client.7215 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 10 client.7215 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 20 client.7215 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e5bff0 0x561e77e5c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 4 client.7215 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x561e77d5fa20 con 0x561e77e5bff0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.272+0000 7fc66fe00640 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e5bff0 0x561e77e5c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc66fe00640 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e5bff0 0x561e77e5c3d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3938ecdfababd6bf server_cookie=a9d3791bce1ccfd6 in_seq=0 out_seq=0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc65d600640 10 client.7215 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc65d600640 1 -- 172.21.15.59:0/3151055789 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x561e77d5fa20 con 0x561e77e5bff0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc65d600640 10 client.7215 handle_command_reply: tid=0 2024-04-23T14:37:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 client.7215 shutdown 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e3bac0 msgr2=0x561e77e485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e3bac0 0x561e77e485d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc668013a40 tx=0x7fc6680046c0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 shutdown_connections 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x561e77e5bff0 0x561e77e5c3d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e77e31ac0 0x561e77dedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e77e3bac0 0x561e77e485d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc65d600640 0 client.7215 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 --2- 172.21.15.59:0/3151055789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e77e1e220 0x561e77ded370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 >> 172.21.15.59:0/3151055789 conn(0x561e77df3f00 msgr2=0x561e77e5d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.273+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 shutdown_connections 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc679f77740 1 -- 172.21.15.59:0/3151055789 wait complete. 2024-04-23T14:37:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc679f77740 20 client.7215 trim_cache size 0 max 16384 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "dentry", 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 1225652500, 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627776, 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "frag": "*", 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "dname": "file_to_be_damaged", 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_id": "head", 2024-04-23T14:37:43.279 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-23T14:37:43.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:37:43.280 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:37:43.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc64c06c8f0 msgr2=0x7fc64c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc64c06c8f0 0x7fc64c06edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc660033000 tx=0x7fc660007000 comp rx=0 tx=0).stop 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6701f4cd0 msgr2=0x7fc6700c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6701f4cd0 0x7fc6700c9ee0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc65800d460 tx=0x7fc65800d920 comp rx=0 tx=0).stop 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 shutdown_connections 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc64c06c8f0 0x7fc64c06edb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6700b2610 0x7fc6700cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6701f4cd0 0x7fc6700c9ee0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 --2- 172.21.15.59:0/2222443797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6540039b0 0x7fc6700cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 >> 172.21.15.59:0/2222443797 conn(0x7fc67006bdf0 msgr2=0x7fc6700ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.274+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 shutdown_connections 2024-04-23T14:37:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.275+0000 7fc675a00640 1 -- 172.21.15.59:0/2222443797 wait complete. 2024-04-23T14:37:43.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:37:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.488+0000 7fb6fb200640 1 Processor -- start 2024-04-23T14:37:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.488+0000 7fb6fb200640 1 -- start start 2024-04-23T14:37:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.489+0000 7fb6fb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6fc15c3d0 0x7fb6fc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.489+0000 7fb6fb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6fc160210 con 0x7fb6fc0b4860 2024-04-23T14:37:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.489+0000 7fb6fb200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6fc1604f0 con 0x7fb6fc0cd590 2024-04-23T14:37:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.489+0000 7fb6fb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb6fc1607d0 con 0x7fb6fc15c3d0 2024-04-23T14:37:43.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6fa800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb6fc0b4860 0x7fb6fc0cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35900/0 (socket says 172.21.15.59:35900) 2024-04-23T14:37:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6fa800640 1 -- 172.21.15.59:0/3390510220 learned_addr learned my addr 172.21.15.59:0/3390510220 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6f9e00640 1 --2- 172.21.15.59:0/3390510220 >> v2:172.21.15.59:3300/0 conn(0x7fb6fc15c3d0 0x7fb6fc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6f8a00640 1 -- 172.21.15.59:0/3390510220 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 83542680 0 0) 0x7fb6fc160210 con 0x7fb6fc0b4860 2024-04-23T14:37:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6f8a00640 1 -- 172.21.15.59:0/3390510220 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6e0003520 con 0x7fb6fc0b4860 2024-04-23T14:37:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6f9e00640 1 -- 172.21.15.59:0/3390510220 >> v1:172.21.15.76:6789/0 conn(0x7fb6fc0cd590 legacy=0x7fb6fc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6f9e00640 1 -- 172.21.15.59:0/3390510220 >> v1:172.21.15.59:6789/0 conn(0x7fb6fc0b4860 legacy=0x7fb6fc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.490+0000 7fb6f9e00640 1 -- 172.21.15.59:0/3390510220 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc160ab0 con 0x7fb6fc15c3d0 2024-04-23T14:37:43.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.492+0000 7fb6f9e00640 1 --2- 172.21.15.59:0/3390510220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc15c3d0 0x7fb6fc0cd050 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097a0 tx=0x7fb6ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8ef3f63c2a6c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.492+0000 7fb6f8a00640 1 -- 172.21.15.59:0/3390510220 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec037070 con 0x7fb6fc15c3d0 2024-04-23T14:37:43.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.492+0000 7fb6f8a00640 1 -- 172.21.15.59:0/3390510220 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec032440 con 0x7fb6fc15c3d0 2024-04-23T14:37:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.492+0000 7fb6f8a00640 1 -- 172.21.15.59:0/3390510220 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec032810 con 0x7fb6fc15c3d0 2024-04-23T14:37:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.493+0000 7fb6fb200640 1 -- 172.21.15.59:0/3390510220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc15c3d0 msgr2=0x7fb6fc0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.493+0000 7fb6fb200640 1 --2- 172.21.15.59:0/3390510220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc15c3d0 0x7fb6fc0cd050 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0097a0 tx=0x7fb6ec02a230 comp rx=0 tx=0).stop 2024-04-23T14:37:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.493+0000 7fb6fb200640 1 -- 172.21.15.59:0/3390510220 shutdown_connections 2024-04-23T14:37:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.493+0000 7fb6fb200640 1 --2- 172.21.15.59:0/3390510220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc15c3d0 0x7fb6fc0cd050 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.493+0000 7fb6fb200640 1 -- 172.21.15.59:0/3390510220 >> 172.21.15.59:0/3390510220 conn(0x7fb6fc0b21a0 msgr2=0x7fb6fc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.493+0000 7fb6fb200640 1 -- 172.21.15.59:0/3390510220 shutdown_connections 2024-04-23T14:37:43.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.493+0000 7fb6fb200640 1 -- 172.21.15.59:0/3390510220 wait complete. 2024-04-23T14:37:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.495+0000 7fb6fb200640 1 Processor -- start 2024-04-23T14:37:43.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.495+0000 7fb6fb200640 1 -- start start 2024-04-23T14:37:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6fb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6fc0b4860 0x7fb6fc0c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6fb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc0cd590 0x7fb6fc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6fb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6fc15c3d0 0x7fb6fc0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6fb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc15c930 con 0x7fb6fc0cd590 2024-04-23T14:37:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6fb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc15c7b0 con 0x7fb6fc15c3d0 2024-04-23T14:37:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6fb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb6fc0cbee0 con 0x7fb6fc0b4860 2024-04-23T14:37:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6f9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc0cd590 0x7fb6fc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6f9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc0cd590 0x7fb6fc0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47522/0 (socket says 172.21.15.59:47522) 2024-04-23T14:37:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.496+0000 7fb6f9400640 1 -- 172.21.15.59:0/1565138458 learned_addr learned my addr 172.21.15.59:0/1565138458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f9e00640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6fc0b4860 0x7fb6fc0c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6fa800640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6fc15c3d0 0x7fb6fc0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f9400640 1 -- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6fc0b4860 msgr2=0x7fb6fc0c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f9400640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6fc0b4860 0x7fb6fc0c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f9400640 1 -- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6fc15c3d0 msgr2=0x7fb6fc0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f9400640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6fc15c3d0 0x7fb6fc0c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f9400640 1 -- 172.21.15.59:0/1565138458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc0cedd0 con 0x7fb6fc0cd590 2024-04-23T14:37:43.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f9400640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc0cd590 0x7fb6fc0ce740 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400c790 tx=0x7fb6e400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f2000640 1 -- 172.21.15.59:0/1565138458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4016020 con 0x7fb6fc0cd590 2024-04-23T14:37:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f2000640 1 -- 172.21.15.59:0/1565138458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e4006d50 con 0x7fb6fc0cd590 2024-04-23T14:37:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6f2000640 1 -- 172.21.15.59:0/1565138458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4003bb0 con 0x7fb6fc0cd590 2024-04-23T14:37:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.497+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc0cf0b0 con 0x7fb6fc0cd590 2024-04-23T14:37:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.498+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc0cf670 con 0x7fb6fc0cd590 2024-04-23T14:37:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.499+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6fc04fd90 con 0x7fb6fc0cd590 2024-04-23T14:37:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.500+0000 7fb6f2000640 1 -- 172.21.15.59:0/1565138458 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6e400a850 con 0x7fb6fc0cd590 2024-04-23T14:37:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.501+0000 7fb6f2000640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d806c8f0 0x7fb6d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.501+0000 7fb6f2000640 1 -- 172.21.15.59:0/1565138458 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fb6e408b940 con 0x7fb6fc0cd590 2024-04-23T14:37:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.501+0000 7fb6f9e00640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d806c8f0 0x7fb6d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.502+0000 7fb6f9e00640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d806c8f0 0x7fb6d806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec034000 tx=0x7fb6ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.506+0000 7fb6f2000640 1 -- 172.21.15.59:0/1565138458 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6e4020020 con 0x7fb6fc0cd590 2024-04-23T14:37:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.739+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb6fc06f580 con 0x7fb6fc0cd590 2024-04-23T14:37:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.739+0000 7fb6f2000640 1 -- 172.21.15.59:0/1565138458 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fb6e4019310 con 0x7fb6fc0cd590 2024-04-23T14:37:43.741 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:37:43.741 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-23T14:37:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d806c8f0 msgr2=0x7fb6d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d806c8f0 0x7fb6d806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec034000 tx=0x7fb6ec02e000 comp rx=0 tx=0).stop 2024-04-23T14:37:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc0cd590 msgr2=0x7fb6fc0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc0cd590 0x7fb6fc0ce740 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400c790 tx=0x7fb6e400cc50 comp rx=0 tx=0).stop 2024-04-23T14:37:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 shutdown_connections 2024-04-23T14:37:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d806c8f0 0x7fb6d806edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6fc15c3d0 0x7fb6fc0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6fc0cd590 0x7fb6fc0ce740 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 --2- 172.21.15.59:0/1565138458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6fc0b4860 0x7fb6fc0c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 >> 172.21.15.59:0/1565138458 conn(0x7fb6fc0b21a0 msgr2=0x7fb6fc0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 shutdown_connections 2024-04-23T14:37:43.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.742+0000 7fb6fb200640 1 -- 172.21.15.59:0/1565138458 wait complete. 2024-04-23T14:37:43.760 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-23T14:37:43.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:43.760 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-23T14:37:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.973+0000 7f19f5a00640 1 Processor -- start 2024-04-23T14:37:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.973+0000 7f19f5a00640 1 -- start start 2024-04-23T14:37:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19f5a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19f00737e0 0x7f19f00adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19f5a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f015b8d0 0x7f19f00722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19f5a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f00b5170 con 0x7f19f006f530 2024-04-23T14:37:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19f5a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f19f015c0f0 con 0x7f19f00737e0 2024-04-23T14:37:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19f5a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f19f015c3d0 con 0x7f19f015b8d0 2024-04-23T14:37:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19e7400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19f00737e0 0x7f19f00adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19e7400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19f00737e0 0x7f19f00adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:54518/0 (socket says 172.21.15.59:54518) 2024-04-23T14:37:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19e7400640 1 -- 172.21.15.59:0/1536265560 learned_addr learned my addr 172.21.15.59:0/1536265560 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.975+0000 7f19f5000640 1 --2- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f015b8d0 0x7f19f00722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.976+0000 7f19e7400640 1 -- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f015b8d0 msgr2=0x7f19f00722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.976+0000 7f19e7400640 1 --2- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f015b8d0 0x7f19f00722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.976+0000 7f19e7400640 1 -- 172.21.15.59:0/1536265560 >> v1:172.21.15.76:6789/0 conn(0x7f19f006f530 legacy=0x7f19f006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.976+0000 7f19e7400640 1 -- 172.21.15.59:0/1536265560 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f015c6b0 con 0x7f19f00737e0 2024-04-23T14:37:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.978+0000 7f19e7400640 1 --2- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f00737e0 0x7f19f00adef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009870 tx=0x7f19d802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856de386cd33ecc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.978+0000 7f19ef400640 1 -- 172.21.15.59:0/1536265560 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8037070 con 0x7f19f00737e0 2024-04-23T14:37:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.978+0000 7f19ef400640 1 -- 172.21.15.59:0/1536265560 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d8032440 con 0x7f19f00737e0 2024-04-23T14:37:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.978+0000 7f19ef400640 1 -- 172.21.15.59:0/1536265560 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8032810 con 0x7f19f00737e0 2024-04-23T14:37:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 -- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f00737e0 msgr2=0x7f19f00adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 --2- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f00737e0 0x7f19f00adef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009870 tx=0x7f19d802a270 comp rx=0 tx=0).stop 2024-04-23T14:37:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 -- 172.21.15.59:0/1536265560 shutdown_connections 2024-04-23T14:37:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 --2- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f015b8d0 0x7f19f00722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 --2- 172.21.15.59:0/1536265560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f00737e0 0x7f19f00adef0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 -- 172.21.15.59:0/1536265560 >> 172.21.15.59:0/1536265560 conn(0x7f19f00d0210 msgr2=0x7f19f00d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 -- 172.21.15.59:0/1536265560 shutdown_connections 2024-04-23T14:37:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.979+0000 7f19f5a00640 1 -- 172.21.15.59:0/1536265560 wait complete. 2024-04-23T14:37:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.981+0000 7f19f5a00640 1 Processor -- start 2024-04-23T14:37:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.982+0000 7f19f5a00640 1 -- start start 2024-04-23T14:37:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f006f530 0x7f19f01f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f00737e0 0x7f19f009d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19f015b8d0 0x7f19f009be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0155ce0 con 0x7f19f015b8d0 2024-04-23T14:37:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00727e0 con 0x7f19f006f530 2024-04-23T14:37:43.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f19f009b070 con 0x7f19f00737e0 2024-04-23T14:37:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19f015b8d0 0x7f19f009be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19f015b8d0 0x7f19f009be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47534/0 (socket says 172.21.15.59:47534) 2024-04-23T14:37:43.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.983+0000 7f19f5000640 1 -- 172.21.15.59:0/4047748833 learned_addr learned my addr 172.21.15.59:0/4047748833 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19e7400640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f00737e0 0x7f19f009d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19efe00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f006f530 0x7f19f01f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19f5000640 1 -- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f00737e0 msgr2=0x7f19f009d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19f5000640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f00737e0 0x7f19f009d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19f5000640 1 -- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f006f530 msgr2=0x7f19f01f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19f5000640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f006f530 0x7f19f01f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19f5000640 1 -- 172.21.15.59:0/4047748833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f009b350 con 0x7f19f015b8d0 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19efe00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f006f530 0x7f19f01f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19f5000640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19f015b8d0 0x7f19f009be30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e7f0 tx=0x7f19e800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19ecc00640 1 -- 172.21.15.59:0/4047748833 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8017070 con 0x7f19f015b8d0 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19ecc00640 1 -- 172.21.15.59:0/4047748833 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e800ee40 con 0x7f19f015b8d0 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19ecc00640 1 -- 172.21.15.59:0/4047748833 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800cb80 con 0x7f19f015b8d0 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.984+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f009b630 con 0x7f19f015b8d0 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.985+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01f1490 con 0x7f19f015b8d0 2024-04-23T14:37:43.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.987+0000 7f19ecc00640 1 -- 172.21.15.59:0/4047748833 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19e801a070 con 0x7f19f015b8d0 2024-04-23T14:37:43.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.988+0000 7f19ecc00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f19cc06c8f0 0x7f19cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.988+0000 7f19ecc00640 1 -- 172.21.15.59:0/4047748833 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f19e808be20 con 0x7f19f015b8d0 2024-04-23T14:37:43.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.988+0000 7f19efe00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f19cc06c8f0 0x7f19cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.989+0000 7f19f9fe3740 1 Processor -- start 2024-04-23T14:37:43.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.989+0000 7f19efe00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f19cc06c8f0 0x7f19cc06edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f19e0001d70 tx=0x7f19e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.989+0000 7f19f9fe3740 1 -- start start 2024-04-23T14:37:43.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c1bdf0 0x564104bce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bfe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104bfe7d0 0x564104bd35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564104b38070 con 0x564104bfe7d0 2024-04-23T14:37:43.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564104ade580 con 0x564104bfef10 2024-04-23T14:37:43.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x564104b34030 con 0x564104c1bdf0 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104bfe7d0 0x564104bd35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104bfe7d0 0x564104bd35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47548/0 (socket says 172.21.15.59:47548) 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.990+0000 7f19f5000640 1 -- 172.21.15.59:0/550561631 learned_addr learned my addr 172.21.15.59:0/550561631 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19efe00640 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c1bdf0 0x564104bce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19f5000640 1 -- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c1bdf0 msgr2=0x564104bce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19f5000640 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c1bdf0 0x564104bce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19f5000640 1 -- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 msgr2=0x564104bfe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19f5000640 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bfe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19f5000640 1 -- 172.21.15.59:0/550561631 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564104912a50 con 0x564104bfe7d0 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19efe00640 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c1bdf0 0x564104bce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19f5000640 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104bfe7d0 0x564104bd35e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f19e808a220 tx=0x7f19e8059540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af9e8e1ee69b71f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:43.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19e4c00640 1 -- 172.21.15.59:0/550561631 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8021070 con 0x564104bfe7d0 2024-04-23T14:37:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19e4c00640 1 -- 172.21.15.59:0/550561631 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e800cd20 con 0x564104bfe7d0 2024-04-23T14:37:43.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.991+0000 7f19e4c00640 1 -- 172.21.15.59:0/550561631 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8058f00 con 0x564104bfe7d0 2024-04-23T14:37:43.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 -- 172.21.15.59:0/550561631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104bfe7d0 msgr2=0x564104bd35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104bfe7d0 0x564104bd35e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f19e808a220 tx=0x7f19e8059540 comp rx=0 tx=0).stop 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 -- 172.21.15.59:0/550561631 shutdown_connections 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104bfe7d0 0x564104bd35e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bfe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/550561631 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c1bdf0 0x564104bce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 -- 172.21.15.59:0/550561631 >> 172.21.15.59:0/550561631 conn(0x564104bd09f0 msgr2=0x564104c19290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 -- 172.21.15.59:0/550561631 shutdown_connections 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.992+0000 7f19f9fe3740 1 -- 172.21.15.59:0/550561631 wait complete. 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.993+0000 7f19f9fe3740 1 -- start start 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.993+0000 7f19f9fe3740 1 Processor -- start 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bfa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104bffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c00350 0x564104c017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564104bd2ba0 con 0x564104c1bdf0 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564104af1860 con 0x564104c00350 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x564104bd1f90 con 0x564104bfef10 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19f9fe3740 10 client.0 _ll_register_callbacks cb 0x564104adfca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104bffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104bffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47556/0 (socket says 172.21.15.59:47556) 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.994+0000 7f19e7400640 1 -- 172.21.15.59:0/3821511791 learned_addr learned my addr 172.21.15.59:0/3821511791 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19efe00640 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bfa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19f5000640 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c00350 0x564104c017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19e7400640 1 -- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 msgr2=0x564104bfa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19e7400640 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bfa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19e7400640 1 -- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c00350 msgr2=0x564104c017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19e7400640 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c00350 0x564104c017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19e7400640 1 -- 172.21.15.59:0/3821511791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56410480a3e0 con 0x564104c1bdf0 2024-04-23T14:37:43.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19e7400640 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104bffe10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009990 tx=0x7f19d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1076cc098d1968 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19dd600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19dd600640 1 -- 172.21.15.59:0/3821511791 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8037040 con 0x564104c1bdf0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e800cd20 con 0x564104c1bdf0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.995+0000 7f19dd600640 1 -- 172.21.15.59:0/3821511791 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f19d8032d40 con 0x564104c1bdf0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19dd600640 1 -- 172.21.15.59:0/3821511791 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f19d8047050 con 0x564104c1bdf0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 10 client.7233 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5641049731e0 con 0x564104c1bdf0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19dd600640 1 -- 172.21.15.59:0/3821511791 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e800cd20 con 0x564104c1bdf0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19dd600640 1 -- 172.21.15.59:0/3821511791 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f19d804a020 con 0x564104c1bdf0 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 10 client.7233 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 10 client.7233 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 20 client.7233 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104c19fc0 0x564104c1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 4 client.7233 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:37:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.996+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564104b45a90 con 0x564104c19fc0 2024-04-23T14:37:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.997+0000 7f19efe00640 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104c19fc0 0x564104c1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.997+0000 7f19efe00640 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104c19fc0 0x564104c1a3d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c15b91d7c03628d server_cookie=6f79a233f72b399f in_seq=0 out_seq=0 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.997+0000 7f19dd600640 10 client.7233 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19dd600640 1 -- 172.21.15.59:0/3821511791 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564104b45a90 con 0x564104c19fc0 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19dd600640 10 client.7233 handle_command_reply: tid=0 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 client.7233 shutdown 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 msgr2=0x564104bffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104bffe10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009990 tx=0x7f19d80046c0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 shutdown_connections 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104c19fc0 0x564104c1a3d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c00350 0x564104c017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104bffe10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3821511791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bfa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 >> 172.21.15.59:0/3821511791 conn(0x564104bfe7d0 msgr2=0x564104c1c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19dd600640 0 client.7233 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.998+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 shutdown_connections 2024-04-23T14:37:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.999+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3821511791 wait complete. 2024-04-23T14:37:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:43.999+0000 7f19f9fe3740 20 client.7233 trim_cache size 0 max 16384 2024-04-23T14:37:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.018+0000 7f19f9fe3740 1 Processor -- start 2024-04-23T14:37:44.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.018+0000 7f19f9fe3740 1 -- start start 2024-04-23T14:37:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c00350 0x564104bf8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104c1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01fcab0 con 0x564104c1bdf0 2024-04-23T14:37:44.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01fbae0 con 0x564104bfef10 2024-04-23T14:37:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00b04a0 con 0x564104c00350 2024-04-23T14:37:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104c1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104c1a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47570/0 (socket says 172.21.15.59:47570) 2024-04-23T14:37:44.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.019+0000 7f19f5000640 1 -- 172.21.15.59:0/3815205814 learned_addr learned my addr 172.21.15.59:0/3815205814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:44.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19f5000640 1 -- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c00350 msgr2=0x564104bf8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19f5000640 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c00350 0x564104bf8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19f5000640 1 -- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 msgr2=0x564104bce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19efe00640 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19f5000640 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19f5000640 1 -- 172.21.15.59:0/3815205814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641047e34a0 con 0x564104c1bdf0 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19efe00640 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19f5000640 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104c1a3d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f19e8055f10 tx=0x7f19e805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cb12ca9f5314db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19dfe00640 1 -- 172.21.15.59:0/3815205814 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800cd20 con 0x564104c1bdf0 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19dfe00640 1 -- 172.21.15.59:0/3815205814 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8087070 con 0x564104c1bdf0 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.020+0000 7f19dfe00640 1 -- 172.21.15.59:0/3815205814 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8058f00 con 0x564104c1bdf0 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3815205814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 msgr2=0x564104c1a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104c1a3d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f19e8055f10 tx=0x7f19e805c000 comp rx=0 tx=0).stop 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3815205814 shutdown_connections 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104c00350 0x564104bf8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c1bdf0 0x564104c1a3d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/3815205814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104bfef10 0x564104bce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3815205814 >> 172.21.15.59:0/3815205814 conn(0x564104bfe7d0 msgr2=0x564104bc9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3815205814 shutdown_connections 2024-04-23T14:37:44.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.021+0000 7f19f9fe3740 1 -- 172.21.15.59:0/3815205814 wait complete. 2024-04-23T14:37:44.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.022+0000 7f19f9fe3740 1 -- start start 2024-04-23T14:37:44.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.022+0000 7f19f9fe3740 1 Processor -- start 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c00350 0x564104c81eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19f9fe3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c1bdf0 0x564104bfad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564104c10190 con 0x564104c00350 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01fb4f0 con 0x564104c1bdf0 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19f9fe3740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x564104c33700 con 0x564104bfef10 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19f9fe3740 10 client.0 _ll_register_callbacks cb 0x7f19d8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c00350 0x564104c81eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c00350 0x564104c81eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47584/0 (socket says 172.21.15.59:47584) 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.023+0000 7f19e7400640 1 -- 172.21.15.59:0/1678076710 learned_addr learned my addr 172.21.15.59:0/1678076710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19efe00640 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19f5000640 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c1bdf0 0x564104bfad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19e7400640 1 -- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 msgr2=0x564104bcdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19e7400640 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bcdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19e7400640 1 -- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c1bdf0 msgr2=0x564104bfad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19e7400640 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c1bdf0 0x564104bfad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19e7400640 1 -- 172.21.15.59:0/1678076710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564104bf8f90 con 0x564104c00350 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19e7400640 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c00350 0x564104c81eb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009210 tx=0x7f19d8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef70f7960581439 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19dd600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.024+0000 7f19dd600640 1 -- 172.21.15.59:0/1678076710 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8037070 con 0x564104c00350 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19dd600640 1 -- 172.21.15.59:0/1678076710 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f19d8005b90 con 0x564104c00350 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e8087070 con 0x564104c00350 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19dd600640 1 -- 172.21.15.59:0/1678076710 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f19d8005d90 con 0x564104c00350 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 10 client.7239 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x564104c88e50 con 0x564104c00350 2024-04-23T14:37:44.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19dd600640 1 -- 172.21.15.59:0/1678076710 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8087070 con 0x564104c00350 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19dd600640 1 -- 172.21.15.59:0/1678076710 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f19d804a070 con 0x564104c00350 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 10 client.7239 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 10 client.7239 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 20 client.7239 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104bfa720 0x564104bfb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 4 client.7239 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.025+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x564104bcfe40 con 0x564104bfa720 2024-04-23T14:37:44.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.026+0000 7f19efe00640 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104bfa720 0x564104bfb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.026+0000 7f19efe00640 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104bfa720 0x564104bfb240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6dc5938fd75cf6 server_cookie=7047ce473351b0f0 in_seq=0 out_seq=0 2024-04-23T14:37:44.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.026+0000 7f19dd600640 10 client.7239 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19dd600640 1 -- 172.21.15.59:0/1678076710 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564104bcfe40 con 0x564104bfa720 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19dd600640 10 client.7239 handle_command_reply: tid=0 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 client.7239 shutdown 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c00350 msgr2=0x564104c81eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c00350 0x564104c81eb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009210 tx=0x7f19d8009240 comp rx=0 tx=0).stop 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 shutdown_connections 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x564104bfa720 0x564104bfb240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564104c1bdf0 0x564104bfad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564104c00350 0x564104c81eb0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 --2- 172.21.15.59:0/1678076710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564104bfef10 0x564104bcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 >> 172.21.15.59:0/1678076710 conn(0x564104bfe7d0 msgr2=0x564104c01010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19dd600640 0 client.7239 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 shutdown_connections 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 1 -- 172.21.15.59:0/1678076710 wait complete. 2024-04-23T14:37:44.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.027+0000 7f19f9fe3740 20 client.7239 trim_cache size 0 max 16384 2024-04-23T14:37:44.032 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:37:44.032 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:37:44.032 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "4fad4cd5-2780-4561-92ed-d53eedbdca97", 2024-04-23T14:37:44.032 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:37:44.032 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f19cc06c8f0 msgr2=0x7f19cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f19cc06c8f0 0x7f19cc06edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f19e0001d70 tx=0x7f19e0009040 comp rx=0 tx=0).stop 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19f015b8d0 msgr2=0x7f19f009be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19f015b8d0 0x7f19f009be30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e7f0 tx=0x7f19e800ebf0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 shutdown_connections 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f19cc06c8f0 0x7f19cc06edb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19f00737e0 0x7f19f009d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19f015b8d0 0x7f19f009be30 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 --2- 172.21.15.59:0/4047748833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19f006f530 0x7f19f01f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 >> 172.21.15.59:0/4047748833 conn(0x7f19f00d0210 msgr2=0x7f19f00b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 shutdown_connections 2024-04-23T14:37:44.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.028+0000 7f19f5a00640 1 -- 172.21.15.59:0/4047748833 wait complete. 2024-04-23T14:37:44.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:37:44.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-23T14:37:44.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.240+0000 7f846be00640 1 Processor -- start 2024-04-23T14:37:44.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.241+0000 7f846be00640 1 -- start start 2024-04-23T14:37:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.242+0000 7f846be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f846409aa00 0x7f846409ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.242+0000 7f846be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846415cfb0 con 0x7f846409b750 2024-04-23T14:37:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.242+0000 7f846be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846409b320 con 0x7f84640cd120 2024-04-23T14:37:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.242+0000 7f846be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f846415dd30 con 0x7f846409aa00 2024-04-23T14:37:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8468a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f846409b750 0x7f84640d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35906/0 (socket says 172.21.15.59:35906) 2024-04-23T14:37:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8468a00640 1 -- 172.21.15.59:0/1869344004 learned_addr learned my addr 172.21.15.59:0/1869344004 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:37:44.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8469400640 1 --2- 172.21.15.59:0/1869344004 >> v2:172.21.15.59:3300/0 conn(0x7f846409aa00 0x7f846409ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8463e00640 1 -- 172.21.15.59:0/1869344004 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 799518048 0 0) 0x7f846415cfb0 con 0x7f846409b750 2024-04-23T14:37:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8463e00640 1 -- 172.21.15.59:0/1869344004 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f844c003520 con 0x7f846409b750 2024-04-23T14:37:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8469400640 1 -- 172.21.15.59:0/1869344004 >> v1:172.21.15.76:6789/0 conn(0x7f84640cd120 legacy=0x7f84640cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8469400640 1 -- 172.21.15.59:0/1869344004 >> v1:172.21.15.59:6789/0 conn(0x7f846409b750 legacy=0x7f84640d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.243+0000 7f8469400640 1 -- 172.21.15.59:0/1869344004 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846415d250 con 0x7f846409aa00 2024-04-23T14:37:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.244+0000 7f8469400640 1 --2- 172.21.15.59:0/1869344004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f846409ade0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f84580097a0 tx=0x7f845802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b343cd9986855e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.245+0000 7f8463e00640 1 -- 172.21.15.59:0/1869344004 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458037070 con 0x7f846409aa00 2024-04-23T14:37:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.245+0000 7f8463e00640 1 -- 172.21.15.59:0/1869344004 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845802ae50 con 0x7f846409aa00 2024-04-23T14:37:44.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.245+0000 7f8463e00640 1 -- 172.21.15.59:0/1869344004 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458032660 con 0x7f846409aa00 2024-04-23T14:37:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.245+0000 7f846be00640 1 -- 172.21.15.59:0/1869344004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 msgr2=0x7f846409ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.245+0000 7f846be00640 1 --2- 172.21.15.59:0/1869344004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f846409ade0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f84580097a0 tx=0x7f845802a140 comp rx=0 tx=0).stop 2024-04-23T14:37:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.245+0000 7f846be00640 1 -- 172.21.15.59:0/1869344004 shutdown_connections 2024-04-23T14:37:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.245+0000 7f846be00640 1 --2- 172.21.15.59:0/1869344004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f846409ade0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.246+0000 7f846be00640 1 -- 172.21.15.59:0/1869344004 >> 172.21.15.59:0/1869344004 conn(0x7f84640b4090 msgr2=0x7f84640b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.246+0000 7f846be00640 1 -- 172.21.15.59:0/1869344004 shutdown_connections 2024-04-23T14:37:44.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.246+0000 7f846be00640 1 -- 172.21.15.59:0/1869344004 wait complete. 2024-04-23T14:37:44.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.247+0000 7f846be00640 1 Processor -- start 2024-04-23T14:37:44.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.248+0000 7f846be00640 1 -- start start 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f846be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f84640cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f846be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f846409b750 0x7f84640cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f846be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84640cd120 0x7f84640c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f846be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f846415d8a0 con 0x7f846409aa00 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f846be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84640d09c0 con 0x7f84640cd120 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f846be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f846415cbe0 con 0x7f846409b750 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f84640cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f84640cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47596/0 (socket says 172.21.15.59:47596) 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469400640 1 -- 172.21.15.59:0/314684163 learned_addr learned my addr 172.21.15.59:0/314684163 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8468a00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f846409b750 0x7f84640cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469400640 1 -- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f846409b750 msgr2=0x7f84640cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469400640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f846409b750 0x7f84640cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469400640 1 -- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84640cd120 msgr2=0x7f84640c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469e00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84640cd120 0x7f84640c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.249+0000 7f8469400640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84640cd120 0x7f84640c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f8469400640 1 -- 172.21.15.59:0/314684163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84640afee0 con 0x7f846409aa00 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f8468a00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f846409b750 0x7f84640cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f8469e00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84640cd120 0x7f84640c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f8469400640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f84640cbac0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8458000c00 tx=0x7f84580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f8461600640 1 -- 172.21.15.59:0/314684163 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458042070 con 0x7f846409aa00 2024-04-23T14:37:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f8461600640 1 -- 172.21.15.59:0/314684163 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845803e460 con 0x7f846409aa00 2024-04-23T14:37:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f8461600640 1 -- 172.21.15.59:0/314684163 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458037040 con 0x7f846409aa00 2024-04-23T14:37:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84640b01c0 con 0x7f846409aa00 2024-04-23T14:37:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.250+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84640b0780 con 0x7f846409aa00 2024-04-23T14:37:44.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.253+0000 7f8461600640 1 -- 172.21.15.59:0/314684163 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8458045070 con 0x7f846409aa00 2024-04-23T14:37:44.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.255+0000 7f8461600640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f844806c8f0 0x7f844806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.255+0000 7f8461600640 1 -- 172.21.15.59:0/314684163 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f84580ac4e0 con 0x7f846409aa00 2024-04-23T14:37:44.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.255+0000 7f8468a00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f844806c8f0 0x7f844806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.255+0000 7f846eb7c740 1 Processor -- start 2024-04-23T14:37:44.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.255+0000 7f846eb7c740 1 -- start start 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f8468a00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f844806c8f0 0x7f844806edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8464156fb0 tx=0x7f8454009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e55610 0x562f78e2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f846eb7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f78d8f1f0 con 0x562f78e55d50 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562f78d35350 con 0x562f78e55610 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x562f78d8b1b0 con 0x562f78e72c30 2024-04-23T14:37:44.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f8468a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f8468a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47598/0 (socket says 172.21.15.59:47598) 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.256+0000 7f8468a00640 1 -- 172.21.15.59:0/1291540372 learned_addr learned my addr 172.21.15.59:0/1291540372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8469400640 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8468a00640 1 -- 172.21.15.59:0/1291540372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 msgr2=0x562f78e255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8468a00640 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8468a00640 1 -- 172.21.15.59:0/1291540372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e55610 msgr2=0x562f78e2a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8468a00640 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e55610 0x562f78e2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8468a00640 1 -- 172.21.15.59:0/1291540372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f78d89f50 con 0x562f78e55d50 2024-04-23T14:37:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8468a00640 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e550d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f845400df30 tx=0x7f8454010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba851b858792119a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8451c00640 1 -- 172.21.15.59:0/1291540372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454012820 con 0x562f78e55d50 2024-04-23T14:37:44.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8451c00640 1 -- 172.21.15.59:0/1291540372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84540129c0 con 0x562f78e55d50 2024-04-23T14:37:44.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.257+0000 7f8451c00640 1 -- 172.21.15.59:0/1291540372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454012d70 con 0x562f78e55d50 2024-04-23T14:37:44.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 -- 172.21.15.59:0/1291540372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 msgr2=0x562f78e550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e550d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f845400df30 tx=0x7f8454010000 comp rx=0 tx=0).stop 2024-04-23T14:37:44.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 -- 172.21.15.59:0/1291540372 shutdown_connections 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e55610 0x562f78e2a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e550d0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1291540372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 -- 172.21.15.59:0/1291540372 >> 172.21.15.59:0/1291540372 conn(0x562f78e27830 msgr2=0x562f78e700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 -- 172.21.15.59:0/1291540372 shutdown_connections 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.258+0000 7f846eb7c740 1 -- 172.21.15.59:0/1291540372 wait complete. 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.259+0000 7f846eb7c740 1 -- start start 2024-04-23T14:37:44.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.259+0000 7f846eb7c740 1 Processor -- start 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e7fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78e66d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f846eb7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f78e299e0 con 0x562f78e55d50 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562f78d48690 con 0x562f78e538e0 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x562f78e28dd0 con 0x562f78e72c30 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f846eb7c740 10 client.0 _ll_register_callbacks cb 0x562f78d84650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.260+0000 7f8469400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e7fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e7fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47612/0 (socket says 172.21.15.59:47612) 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 -- 172.21.15.59:0/2962069601 learned_addr learned my addr 172.21.15.59:0/2962069601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 -- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 msgr2=0x562f78e533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8468a00640 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469e00640 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78e66d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 -- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 msgr2=0x562f78e66d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78e66d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 -- 172.21.15.59:0/2962069601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f78d47920 con 0x562f78e55d50 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8469400640 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e7fe80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8458082820 tx=0x7f845807a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f0c3d0e1cf0225 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8446000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.261+0000 7f8446000640 1 -- 172.21.15.59:0/2962069601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845804c040 con 0x562f78e55d50 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f8446000640 1 -- 172.21.15.59:0/2962069601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f84580ac0c0 con 0x562f78e55d50 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84540129c0 con 0x562f78e55d50 2024-04-23T14:37:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f8446000640 1 -- 172.21.15.59:0/2962069601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f845807e320 con 0x562f78e55d50 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 10 client.7254 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x562f78b6b7d0 con 0x562f78e55d50 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f8446000640 1 -- 172.21.15.59:0/2962069601 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84540129c0 con 0x562f78e55d50 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f8446000640 1 -- 172.21.15.59:0/2962069601 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f84580828c0 con 0x562f78e55d50 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 10 client.7254 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 10 client.7254 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 20 client.7254 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e6d1f0 0x562f78e6d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 4 client.7254 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.262+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f78d9cc10 con 0x562f78e6d1f0 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.263+0000 7f8468a00640 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e6d1f0 0x562f78e6d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.263+0000 7f8468a00640 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e6d1f0 0x562f78e6d640 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7ef509f21adb9 server_cookie=46d8b74ae0716a4d in_seq=0 out_seq=0 2024-04-23T14:37:44.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.263+0000 7f8446000640 10 client.7254 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f8446000640 1 -- 172.21.15.59:0/2962069601 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f78d9cc10 con 0x562f78e6d1f0 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f8446000640 10 client.7254 handle_command_reply: tid=0 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 client.7254 shutdown 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 msgr2=0x562f78e7fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e7fe80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8458082820 tx=0x7f845807a9b0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 shutdown_connections 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e6d1f0 0x562f78e6d640 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e72c30 0x562f78e533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78e66d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f8446000640 0 client.7254 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 --2- 172.21.15.59:0/2962069601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e55d50 0x562f78e7fe80 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.264+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 >> 172.21.15.59:0/2962069601 conn(0x562f78e55610 msgr2=0x562f78e51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.265+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 shutdown_connections 2024-04-23T14:37:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.265+0000 7f846eb7c740 1 -- 172.21.15.59:0/2962069601 wait complete. 2024-04-23T14:37:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.265+0000 7f846eb7c740 20 client.7254 trim_cache size 0 max 16384 2024-04-23T14:37:44.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.284+0000 7f846eb7c740 1 Processor -- start 2024-04-23T14:37:44.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.284+0000 7f846eb7c740 1 -- start start 2024-04-23T14:37:44.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.285+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ee15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.285+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 0x562f78e28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.285+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.285+0000 7f846eb7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8464201110 con 0x562f78e72c30 2024-04-23T14:37:44.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.285+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8464200110 con 0x562f78e538e0 2024-04-23T14:37:44.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.285+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f84640cb680 con 0x562f78e55d50 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47628/0 (socket says 172.21.15.59:47628) 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 -- 172.21.15.59:0/1394240884 learned_addr learned my addr 172.21.15.59:0/1394240884 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469400640 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ee15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8468a00640 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 0x562f78e28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 -- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 msgr2=0x562f78e28dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 0x562f78e28dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 -- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 msgr2=0x562f78ee15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ee15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 -- 172.21.15.59:0/1394240884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f78aa13f0 con 0x562f78e72c30 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.286+0000 7f8469e00640 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d6a0 tx=0x7f845c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53bf51371c088d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f8451200640 1 -- 172.21.15.59:0/1394240884 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c016020 con 0x562f78e72c30 2024-04-23T14:37:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f8451200640 1 -- 172.21.15.59:0/1394240884 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c00b920 con 0x562f78e72c30 2024-04-23T14:37:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f8451200640 1 -- 172.21.15.59:0/1394240884 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c00bcf0 con 0x562f78e72c30 2024-04-23T14:37:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 -- 172.21.15.59:0/1394240884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 msgr2=0x562f78e29310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d6a0 tx=0x7f845c00db60 comp rx=0 tx=0).stop 2024-04-23T14:37:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 -- 172.21.15.59:0/1394240884 shutdown_connections 2024-04-23T14:37:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 0x562f78e28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 --2- 172.21.15.59:0/1394240884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ee15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 -- 172.21.15.59:0/1394240884 >> 172.21.15.59:0/1394240884 conn(0x562f78e55610 msgr2=0x562f78e6fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 -- 172.21.15.59:0/1394240884 shutdown_connections 2024-04-23T14:37:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.287+0000 7f846eb7c740 1 -- 172.21.15.59:0/1394240884 wait complete. 2024-04-23T14:37:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.288+0000 7f846eb7c740 1 -- start start 2024-04-23T14:37:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.288+0000 7f846eb7c740 1 Processor -- start 2024-04-23T14:37:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ecce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 0x562f78e28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f846eb7c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f846eb7c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f78e68540 con 0x562f78e72c30 2024-04-23T14:37:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ffaf0 con 0x562f78e538e0 2024-04-23T14:37:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f846eb7c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x562f78ecd3d0 con 0x562f78e55d50 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f846eb7c740 10 client.0 _ll_register_callbacks cb 0x7f8458009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.289+0000 7f8469e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47644/0 (socket says 172.21.15.59:47644) 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 -- 172.21.15.59:0/3471842456 learned_addr learned my addr 172.21.15.59:0/3471842456 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 -- 172.21.15.59:0/3471842456 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 msgr2=0x562f78e28dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 0x562f78e28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 -- 172.21.15.59:0/3471842456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 msgr2=0x562f78ecce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469400640 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ecce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ecce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 -- 172.21.15.59:0/3471842456 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f78a610f0 con 0x562f78e72c30 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8469e00640 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b570 tx=0x7f845c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4874342c68d14a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.290+0000 7f8446000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f8446000640 1 -- 172.21.15.59:0/3471842456 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c004260 con 0x562f78e72c30 2024-04-23T14:37:44.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f8446000640 1 -- 172.21.15.59:0/3471842456 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f845c00d230 con 0x562f78e72c30 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845c00b920 con 0x562f78e72c30 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f8446000640 1 -- 172.21.15.59:0/3471842456 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f845c004400 con 0x562f78e72c30 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 10 client.7260 fetch_fsmap learned FSMap version 50 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x562f78d54810 con 0x562f78e72c30 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f8446000640 1 -- 172.21.15.59:0/3471842456 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c00b920 con 0x562f78e72c30 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f8446000640 1 -- 172.21.15.59:0/3471842456 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f845c030080 con 0x562f78e72c30 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 10 client.7260 fetch_fsmap finished waiting for FSMap version 50 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 10 client.7260 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 20 client.7260 populate_metadata read hostname 'smithi059' 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e805d0 0x562f78eea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 4 client.7260 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:37:44.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.291+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f78d96a80 con 0x562f78e805d0 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.292+0000 7f8469400640 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e805d0 0x562f78eea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.292+0000 7f8469400640 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e805d0 0x562f78eea7e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3186ee885fdbb2 server_cookie=561278d0d52b3e4d in_seq=0 out_seq=0 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.292+0000 7f8446000640 10 client.7260 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.292+0000 7f8446000640 1 -- 172.21.15.59:0/3471842456 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x562f78d96a80 con 0x562f78e805d0 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f8446000640 10 client.7260 handle_command_reply: tid=0 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 client.7260 shutdown 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 msgr2=0x562f78e29310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b570 tx=0x7f845c004680 comp rx=0 tx=0).stop 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 shutdown_connections 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x562f78e805d0 0x562f78eea7e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x562f78e55d50 0x562f78e28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f78e72c30 0x562f78e29310 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 --2- 172.21.15.59:0/3471842456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562f78e538e0 0x562f78ecce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f8446000640 0 client.7260 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 >> 172.21.15.59:0/3471842456 conn(0x562f78e55610 msgr2=0x562f78ee0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 shutdown_connections 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 1 -- 172.21.15.59:0/3471842456 wait complete. 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.293+0000 7f846eb7c740 20 client.7260 trim_cache size 0 max 16384 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stdout: "4fad4cd5-2780-4561-92ed-d53eedbdca97": { 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/subdir", 2024-04-23T14:37:44.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "4fad4cd5-2780-4561-92ed-d53eedbdca97", 2024-04-23T14:37:44.298 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-23T14:37:44.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:37:44.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:37:44.298 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:37:44.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f844806c8f0 msgr2=0x7f844806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f844806c8f0 0x7f844806edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8464156fb0 tx=0x7f8454009040 comp rx=0 tx=0).stop 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 msgr2=0x7f84640cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f84640cbac0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8458000c00 tx=0x7f84580046c0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 shutdown_connections 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f844806c8f0 0x7f844806edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f846409b750 0x7f84640cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84640cd120 0x7f84640c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 --2- 172.21.15.59:0/314684163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846409aa00 0x7f84640cbac0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 >> 172.21.15.59:0/314684163 conn(0x7f84640b4090 msgr2=0x7f84640d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 shutdown_connections 2024-04-23T14:37:44.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:37:44.294+0000 7f846be00640 1 -- 172.21.15.59:0/314684163 wait complete. 2024-04-23T14:37:44.313 INFO:tasks.cephfs.filesystem:scrub status for tag:4fad4cd5-2780-4561-92ed-d53eedbdca97 - {'path': '/subdir', 'tag': '4fad4cd5-2780-4561-92ed-d53eedbdca97', 'options': 'recursive,repair'} 2024-04-23T14:37:50.163 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:37:50.172 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:38:14.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:14.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-23T14:38:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.536+0000 7fa5aac00640 1 Processor -- start 2024-04-23T14:38:14.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.536+0000 7fa5aac00640 1 -- start start 2024-04-23T14:38:14.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.538+0000 7fa5aac00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5a40d0cc0 0x7fa5a406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.538+0000 7fa5aac00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a40cfc20 con 0x7fa5a4071d70 2024-04-23T14:38:14.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.538+0000 7fa5aac00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a40cff00 con 0x7fa5a4145e10 2024-04-23T14:38:14.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.538+0000 7fa5aac00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a40d01e0 con 0x7fa5a40d0cc0 2024-04-23T14:38:14.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.538+0000 7fa5a8c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa5a4071d70 0x7fa5a415cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39660/0 (socket says 172.21.15.59:39660) 2024-04-23T14:38:14.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1691987098 learned_addr learned my addr 172.21.15.59:0/1691987098 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1691987098 >> v2:172.21.15.76:3300/0 conn(0x7fa5a40d0cc0 0x7fa5a406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 389977065 0 0) 0x7fa5a40cfc20 con 0x7fa5a4071d70 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa588003520 con 0x7fa5a4071d70 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3182435087 0 0) 0x7fa588003520 con 0x7fa5a4071d70 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 >> v2:172.21.15.76:3300/0 conn(0x7fa5a40d0cc0 msgr2=0x7fa5a406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a2a00640 1 --2- 172.21.15.59:0/1691987098 >> v2:172.21.15.76:3300/0 conn(0x7fa5a40d0cc0 0x7fa5a406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 >> v1:172.21.15.76:6789/0 conn(0x7fa5a4145e10 legacy=0x7fa5a40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.539+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40d04c0 con 0x7fa5a4071d70 2024-04-23T14:38:14.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa59c001670 con 0x7fa5a4071d70 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 >> v1:172.21.15.59:6789/0 conn(0x7fa5a4071d70 legacy=0x7fa5a415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5880039b0 0x7fa588003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa588004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5880051c0 0x7fa58800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40d01e0 con 0x7fa588004800 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa58800c060 con 0x7fa5880051c0 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa58800c260 con 0x7fa5880039b0 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa588004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5880039b0 0x7fa588003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a3400640 1 -- 172.21.15.59:0/1691987098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5880039b0 msgr2=0x7fa588003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5880039b0 0x7fa588003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a3400640 1 -- 172.21.15.59:0/1691987098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5880051c0 msgr2=0x7fa58800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5880051c0 0x7fa58800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a3400640 1 -- 172.21.15.59:0/1691987098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa588003520 con 0x7fa588004800 2024-04-23T14:38:14.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.540+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1691987098 reap_dead start 2024-04-23T14:38:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.541+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa588004c80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa590009ac0 tx=0x7fa588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c54f170c8407d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590036070 con 0x7fa588004800 2024-04-23T14:38:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590031400 con 0x7fa588004800 2024-04-23T14:38:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5a2a00640 1 -- 172.21.15.59:0/1691987098 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590009690 con 0x7fa588004800 2024-04-23T14:38:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5aac00640 1 -- 172.21.15.59:0/1691987098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 msgr2=0x7fa588004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5aac00640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa588004c80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa590009ac0 tx=0x7fa588005830 comp rx=0 tx=0).stop 2024-04-23T14:38:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5aac00640 1 -- 172.21.15.59:0/1691987098 shutdown_connections 2024-04-23T14:38:14.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5aac00640 1 --2- 172.21.15.59:0/1691987098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa588004c80 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5aac00640 1 -- 172.21.15.59:0/1691987098 >> 172.21.15.59:0/1691987098 conn(0x7fa5a4067d90 msgr2=0x7fa5a406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:14.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.542+0000 7fa5aac00640 1 -- 172.21.15.59:0/1691987098 shutdown_connections 2024-04-23T14:38:14.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.543+0000 7fa5aac00640 1 -- 172.21.15.59:0/1691987098 wait complete. 2024-04-23T14:38:14.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.544+0000 7fa5aac00640 1 Processor -- start 2024-04-23T14:38:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.545+0000 7fa5aac00640 1 -- start start 2024-04-23T14:38:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.545+0000 7fa5aac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa5a413f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.545+0000 7fa5aac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5a41e6180 0x7fa5a41e6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5aac00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40ad250 0x7fa5a40ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5aac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41461f0 con 0x7fa588004800 2024-04-23T14:38:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5aac00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a406f130 con 0x7fa5a40ad250 2024-04-23T14:38:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5aac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a41f2f40 con 0x7fa5a41e6180 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa5a413f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa5a413f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45164/0 (socket says 172.21.15.59:45164) 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 -- 172.21.15.59:0/14275409 learned_addr learned my addr 172.21.15.59:0/14275409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40ad250 0x7fa5a40ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3400640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5a41e6180 0x7fa5a41e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 -- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5a41e6180 msgr2=0x7fa5a41e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5a41e6180 0x7fa5a41e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 -- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40ad250 msgr2=0x7fa5a40ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40ad250 0x7fa5a40ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.546+0000 7fa5a3e00640 1 -- 172.21.15.59:0/14275409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a41e6d20 con 0x7fa588004800 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.547+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa5a413f9f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa59400d830 tx=0x7fa59400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.547+0000 7fa59be00640 1 -- 172.21.15.59:0/14275409 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594003da0 con 0x7fa588004800 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.547+0000 7fa59be00640 1 -- 172.21.15.59:0/14275409 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594017070 con 0x7fa588004800 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.547+0000 7fa59be00640 1 -- 172.21.15.59:0/14275409 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400b8b0 con 0x7fa588004800 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.547+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a41e7000 con 0x7fa588004800 2024-04-23T14:38:14.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.547+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a40afaa0 con 0x7fa588004800 2024-04-23T14:38:14.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.550+0000 7fa59be00640 1 -- 172.21.15.59:0/14275409 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa59400ba50 con 0x7fa588004800 2024-04-23T14:38:14.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.550+0000 7fa59be00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa58406c920 0x7fa58406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.551+0000 7fa59be00640 1 -- 172.21.15.59:0/14275409 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fa59408bcb0 con 0x7fa588004800 2024-04-23T14:38:14.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.551+0000 7fa5a3400640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa58406c920 0x7fa58406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.551+0000 7fa5ada0e740 1 Processor -- start 2024-04-23T14:38:14.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.551+0000 7fa5a3400640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa58406c920 0x7fa58406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.552+0000 7fa5ada0e740 1 -- start start 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.552+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b169df20 0x5559b16a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.552+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b1675c30 0x5559b169e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.552+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.552+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5559b15db1f0 con 0x5559b169e840 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.552+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5559b1581350 con 0x5559b169df20 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.552+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5559b15d71b0 con 0x5559b1675c30 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45178/0 (socket says 172.21.15.59:45178) 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1747920220 learned_addr learned my addr 172.21.15.59:0/1747920220 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b169df20 0x5559b16a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b1675c30 0x5559b169e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b1675c30 msgr2=0x5559b169e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b1675c30 0x5559b169e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b169df20 msgr2=0x5559b16a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b169df20 0x5559b16a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1747920220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b15d5f50 con 0x5559b169e840 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.553+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b169df20 0x5559b16a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.554+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16a10d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40cec80 tx=0x7fa59c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28df141777c48284 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.554+0000 7fa58fe00640 1 -- 172.21.15.59:0/1747920220 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00e8f0 con 0x5559b169e840 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.554+0000 7fa58fe00640 1 -- 172.21.15.59:0/1747920220 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c00ea90 con 0x5559b169e840 2024-04-23T14:38:14.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.554+0000 7fa58fe00640 1 -- 172.21.15.59:0/1747920220 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00ee40 con 0x5559b169e840 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.554+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1747920220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 msgr2=0x5559b16a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.554+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16a10d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40cec80 tx=0x7fa59c005bc0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1747920220 shutdown_connections 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b1675c30 0x5559b169e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16a10d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1747920220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b169df20 0x5559b16a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1747920220 >> 172.21.15.59:0/1747920220 conn(0x5559b1673830 msgr2=0x5559b16bc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1747920220 shutdown_connections 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1747920220 wait complete. 2024-04-23T14:38:14.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.555+0000 7fa5ada0e740 1 -- start start 2024-04-23T14:38:14.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.556+0000 7fa5ada0e740 1 Processor -- start 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b168ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 0x5559b16bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b16bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5559b1671b80 con 0x5559b169df20 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5559b1594690 con 0x5559b168f090 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5559b16715c0 con 0x5559b169e840 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5ada0e740 10 client.0 _ll_register_callbacks cb 0x5559b15d0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b168ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b168ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45182/0 (socket says 172.21.15.59:45182) 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.557+0000 7fa5a3e00640 1 -- 172.21.15.59:0/1439315557 learned_addr learned my addr 172.21.15.59:0/1439315557 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5a3e00640 1 -- 172.21.15.59:0/1439315557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 msgr2=0x5559b16bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 0x5559b16bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5a3e00640 1 -- 172.21.15.59:0/1439315557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 msgr2=0x5559b16bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b16bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5a3e00640 1 -- 172.21.15.59:0/1439315557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b1593920 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b168ea00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa5940620b0 tx=0x7fa594059a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c90ba30b11df8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa58cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa58cc00640 1 -- 172.21.15.59:0/1439315557 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594074060 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa58cc00640 1 -- 172.21.15.59:0/1439315557 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fa594059c60 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c00ea90 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa58cc00640 1 -- 172.21.15.59:0/1439315557 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa594059e60 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5ada0e740 10 client.7272 fetch_fsmap learned FSMap version 51 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.558+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5559b13b77d0 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa58cc00640 1 -- 172.21.15.59:0/1439315557 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c00ea90 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa58cc00640 1 -- 172.21.15.59:0/1439315557 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fa594062150 con 0x5559b169df20 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa5ada0e740 10 client.7272 fetch_fsmap finished waiting for FSMap version 51 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa5ada0e740 10 client.7272 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa5ada0e740 20 client.7272 populate_metadata read hostname 'smithi059' 2024-04-23T14:38:14.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16940a0 0x5559b169b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa5ada0e740 4 client.7272 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:38:14.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559b15e8c10 con 0x5559b16940a0 2024-04-23T14:38:14.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.559+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16940a0 0x5559b169b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.560+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16940a0 0x5559b169b470 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da23fea479b9394c server_cookie=deb48c1643e376ef in_seq=0 out_seq=0 2024-04-23T14:38:14.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.560+0000 7fa58cc00640 10 client.7272 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.560+0000 7fa58cc00640 1 -- 172.21.15.59:0/1439315557 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559b15e8c10 con 0x5559b16940a0 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.560+0000 7fa58cc00640 10 client.7272 handle_command_reply: tid=0 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 client.7272 shutdown 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 msgr2=0x5559b168ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b168ea00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa5940620b0 tx=0x7fa594059a00 comp rx=0 tx=0).stop 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 shutdown_connections 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16940a0 0x5559b169b470 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 0x5559b16bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b16bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1439315557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b168ea00 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa58cc00640 0 client.7272 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:14.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 >> 172.21.15.59:0/1439315557 conn(0x5559b1675c30 msgr2=0x5559b1676d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:14.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 shutdown_connections 2024-04-23T14:38:14.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1439315557 wait complete. 2024-04-23T14:38:14.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.561+0000 7fa5ada0e740 20 client.7272 trim_cache size 0 max 16384 2024-04-23T14:38:14.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 Processor -- start 2024-04-23T14:38:14.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 -- start start 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1671640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b1676b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 0x5559b1677090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40b53f0 con 0x5559b169df20 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40b1230 con 0x5559b168f090 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.581+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40008d0 con 0x5559b169e840 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b1676b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b1676b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45190/0 (socket says 172.21.15.59:45190) 2024-04-23T14:38:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 -- 172.21.15.59:0/2310131955 learned_addr learned my addr 172.21.15.59:0/2310131955 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 -- 172.21.15.59:0/2310131955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 msgr2=0x5559b1677090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 --2- 172.21.15.59:0/2310131955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 0x5559b1677090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 -- 172.21.15.59:0/2310131955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 msgr2=0x5559b1671640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 --2- 172.21.15.59:0/2310131955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1671640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 -- 172.21.15.59:0/2310131955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b12ed3f0 con 0x5559b169df20 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.582+0000 7fa5a3400640 1 --2- 172.21.15.59:0/2310131955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b1676b50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa59002abb0 tx=0x7fa59002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660a634ccfd30de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa58f400640 1 -- 172.21.15.59:0/2310131955 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590036040 con 0x5559b169df20 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa58f400640 1 -- 172.21.15.59:0/2310131955 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590003ce0 con 0x5559b169df20 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa58f400640 1 -- 172.21.15.59:0/2310131955 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59004c340 con 0x5559b169df20 2024-04-23T14:38:14.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 -- 172.21.15.59:0/2310131955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 msgr2=0x5559b1676b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/2310131955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b1676b50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa59002abb0 tx=0x7fa59002e000 comp rx=0 tx=0).stop 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 -- 172.21.15.59:0/2310131955 shutdown_connections 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/2310131955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169e840 0x5559b1677090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/2310131955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169df20 0x5559b1676b50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/2310131955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1671640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 -- 172.21.15.59:0/2310131955 >> 172.21.15.59:0/2310131955 conn(0x5559b1675c30 msgr2=0x5559b16906b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 -- 172.21.15.59:0/2310131955 shutdown_connections 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.583+0000 7fa5ada0e740 1 -- 172.21.15.59:0/2310131955 wait complete. 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.584+0000 7fa5ada0e740 1 -- start start 2024-04-23T14:38:14.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.584+0000 7fa5ada0e740 1 Processor -- start 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.585+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1676b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.585+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169df20 0x5559b1677090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.585+0000 7fa5ada0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16e3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.585+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40b0be0 con 0x5559b169e840 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.585+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5559b168e810 con 0x5559b168f090 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.585+0000 7fa5ada0e740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5559b16e4ca0 con 0x5559b169df20 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.585+0000 7fa5ada0e740 10 client.0 _ll_register_callbacks cb 0x7fa594055d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16e3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16e3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45196/0 (socket says 172.21.15.59:45196) 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1941327443 learned_addr learned my addr 172.21.15.59:0/1941327443 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1676b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a3400640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169df20 0x5559b1677090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169df20 msgr2=0x5559b1677090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169df20 0x5559b1677090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 msgr2=0x5559b1676b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1676b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 -- 172.21.15.59:0/1941327443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b12ad0f0 con 0x5559b169e840 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1676b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa5a8c00640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16e3c70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000ea0 tx=0x7fa59c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ea8cd391e05a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.586+0000 7fa58cc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa58cc00640 1 -- 172.21.15.59:0/1941327443 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c014430 con 0x5559b169e840 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa58cc00640 1 -- 172.21.15.59:0/1941327443 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fa59c0145d0 con 0x5559b169e840 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa590003ce0 con 0x5559b169e840 2024-04-23T14:38:14.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa58cc00640 1 -- 172.21.15.59:0/1941327443 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa59c025050 con 0x5559b169e840 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa5ada0e740 10 client.7278 fetch_fsmap learned FSMap version 51 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5559b15a0810 con 0x5559b169e840 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa58cc00640 1 -- 172.21.15.59:0/1941327443 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590003ce0 con 0x5559b169e840 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa58cc00640 1 -- 172.21.15.59:0/1941327443 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fa59c028020 con 0x5559b169e840 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa5ada0e740 10 client.7278 fetch_fsmap finished waiting for FSMap version 51 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa5ada0e740 10 client.7278 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.587+0000 7fa5ada0e740 20 client.7278 populate_metadata read hostname 'smithi059' 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.588+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16e41b0 0x5559b16bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.588+0000 7fa5ada0e740 4 client.7278 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.588+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559b15e2a80 con 0x5559b16e41b0 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.588+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16e41b0 0x5559b16bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.588+0000 7fa5a3e00640 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16e41b0 0x5559b16bd1d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0dc73952ef58c2 server_cookie=351cb9408d1a47ec in_seq=0 out_seq=0 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.588+0000 7fa58cc00640 10 client.7278 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.588+0000 7fa58cc00640 1 -- 172.21.15.59:0/1941327443 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5559b15e2a80 con 0x5559b16e41b0 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa58cc00640 10 client.7278 handle_command_reply: tid=0 2024-04-23T14:38:14.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 client.7278 shutdown 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 msgr2=0x5559b16e3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16e3c70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000ea0 tx=0x7fa59c00ebf0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 shutdown_connections 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x5559b16e41b0 0x5559b16bd1d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5559b169df20 0x5559b1677090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5559b169e840 0x5559b16e3c70 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 --2- 172.21.15.59:0/1941327443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5559b168f090 0x5559b1676b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa58cc00640 0 client.7278 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 >> 172.21.15.59:0/1941327443 conn(0x5559b1675c30 msgr2=0x5559b16997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 shutdown_connections 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 1 -- 172.21.15.59:0/1941327443 wait complete. 2024-04-23T14:38:14.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.589+0000 7fa5ada0e740 20 client.7278 trim_cache size 0 max 16384 2024-04-23T14:38:14.595 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:38:14.595 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:38:14.595 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:38:14.595 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa58406c920 msgr2=0x7fa58406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa58406c920 0x7fa58406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590008030 comp rx=0 tx=0).stop 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 msgr2=0x7fa5a413f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa5a413f9f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa59400d830 tx=0x7fa59400dcf0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 shutdown_connections 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa58406c920 0x7fa58406ede0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5a41e6180 0x7fa5a41e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40ad250 0x7fa5a40ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 --2- 172.21.15.59:0/14275409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa588004800 0x7fa5a413f9f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 >> 172.21.15.59:0/14275409 conn(0x7fa5a4067d90 msgr2=0x7fa5a415e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:14.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 shutdown_connections 2024-04-23T14:38:14.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.590+0000 7fa5aac00640 1 -- 172.21.15.59:0/14275409 wait complete. 2024-04-23T14:38:14.608 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:38:14.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.805+0000 7fb1a4a00640 1 Processor -- start 2024-04-23T14:38:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.805+0000 7fb1a4a00640 1 -- start start 2024-04-23T14:38:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.806+0000 7fb1a4a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1a00b5ff0 0x7fb1a00b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.807+0000 7fb1a4a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1a00cf030 con 0x7fb1a00d0d90 2024-04-23T14:38:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.807+0000 7fb1a4a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1a00b6a40 con 0x7fb1a015e7c0 2024-04-23T14:38:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.807+0000 7fb1a4a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb1a00b7000 con 0x7fb1a00b5ff0 2024-04-23T14:38:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.807+0000 7fb19f200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb1a00d0d90 0x7fb1a00cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39670/0 (socket says 172.21.15.59:39670) 2024-04-23T14:38:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.807+0000 7fb19f200640 1 -- 172.21.15.59:0/2128862348 learned_addr learned my addr 172.21.15.59:0/2128862348 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.807+0000 7fb19de00640 1 --2- 172.21.15.59:0/2128862348 >> v2:172.21.15.76:3300/0 conn(0x7fb1a00b5ff0 0x7fb1a00b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.807+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314654741 0 0) 0x7fb198001490 con 0x7fb1a00d0d90 2024-04-23T14:38:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb184003520 con 0x7fb1a00d0d90 2024-04-23T14:38:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3302826191 0 0) 0x7fb184003520 con 0x7fb1a00d0d90 2024-04-23T14:38:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 >> v2:172.21.15.76:3300/0 conn(0x7fb1a00b5ff0 msgr2=0x7fb1a00b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 --2- 172.21.15.59:0/2128862348 >> v2:172.21.15.76:3300/0 conn(0x7fb1a00b5ff0 0x7fb1a00b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 >> v1:172.21.15.76:6789/0 conn(0x7fb1a015e7c0 legacy=0x7fb1a00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a00b6d20 con 0x7fb1a00d0d90 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fb198001bc0 con 0x7fb1a00d0d90 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 >> v1:172.21.15.59:6789/0 conn(0x7fb1a00d0d90 legacy=0x7fb1a00cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb184003a00 0x7fb184003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb184004800 0x7fb184004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1840051c0 0x7fb18400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00b7000 con 0x7fb1840051c0 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb18400c0c0 con 0x7fb184003a00 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb18400c2c0 con 0x7fb184004800 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.808+0000 7fb19f200640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1840051c0 0x7fb18400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.809+0000 7fb19e800640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb184003a00 0x7fb184003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.809+0000 7fb19de00640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb184004800 0x7fb184004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.809+0000 7fb19f200640 1 -- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb184004800 msgr2=0x7fb184004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.809+0000 7fb19f200640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb184004800 0x7fb184004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.809+0000 7fb19f200640 1 -- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb184003a00 msgr2=0x7fb184003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.809+0000 7fb19f200640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb184003a00 0x7fb184003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.809+0000 7fb19f200640 1 -- 172.21.15.59:0/2128862348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb184003520 con 0x7fb1840051c0 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.810+0000 7fb19f200640 1 -- 172.21.15.59:0/2128862348 reap_dead start 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.810+0000 7fb19f200640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1840051c0 0x7fb18400b9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb19800a950 tx=0x7fb184005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f518e7b117c659b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.810+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19800a5a0 con 0x7fb1840051c0 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.810+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19802f070 con 0x7fb1840051c0 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.810+0000 7fb19d400640 1 -- 172.21.15.59:0/2128862348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19802bdf0 con 0x7fb1840051c0 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.811+0000 7fb1a4a00640 1 -- 172.21.15.59:0/2128862348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1840051c0 msgr2=0x7fb18400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.811+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1840051c0 0x7fb18400b9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb19800a950 tx=0x7fb184005830 comp rx=0 tx=0).stop 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.811+0000 7fb1a4a00640 1 -- 172.21.15.59:0/2128862348 shutdown_connections 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.811+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/2128862348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1840051c0 0x7fb18400b9c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.811+0000 7fb1a4a00640 1 -- 172.21.15.59:0/2128862348 >> 172.21.15.59:0/2128862348 conn(0x7fb1a009d920 msgr2=0x7fb1a008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.811+0000 7fb1a4a00640 1 -- 172.21.15.59:0/2128862348 shutdown_connections 2024-04-23T14:38:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.811+0000 7fb1a4a00640 1 -- 172.21.15.59:0/2128862348 wait complete. 2024-04-23T14:38:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.813+0000 7fb1a4a00640 1 Processor -- start 2024-04-23T14:38:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.813+0000 7fb1a4a00640 1 -- start start 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb1a4a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1840051c0 0x7fb1a015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb1a4a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a014c350 0x7fb1a00b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb1a4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00b2860 0x7fb1a01f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb1a4a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00d1a80 con 0x7fb1a00b2860 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb1a4a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00cf620 con 0x7fb1a014c350 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb1a4a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a00d1c00 con 0x7fb1840051c0 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb19f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00b2860 0x7fb1a01f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb19f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00b2860 0x7fb1a01f8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45214/0 (socket says 172.21.15.59:45214) 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.814+0000 7fb19f200640 1 -- 172.21.15.59:0/3894164667 learned_addr learned my addr 172.21.15.59:0/3894164667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19e800640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1840051c0 0x7fb1a015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19de00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a014c350 0x7fb1a00b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19f200640 1 -- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1840051c0 msgr2=0x7fb1a015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19f200640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1840051c0 0x7fb1a015eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19f200640 1 -- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a014c350 msgr2=0x7fb1a00b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:14.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19f200640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a014c350 0x7fb1a00b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:14.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19f200640 1 -- 172.21.15.59:0/3894164667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a00cba60 con 0x7fb1a00b2860 2024-04-23T14:38:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.815+0000 7fb19f200640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00b2860 0x7fb1a01f8180 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb19802b620 tx=0x7fb19802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.816+0000 7fb196a00640 1 -- 172.21.15.59:0/3894164667 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198001490 con 0x7fb1a00b2860 2024-04-23T14:38:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.816+0000 7fb196a00640 1 -- 172.21.15.59:0/3894164667 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19802bda0 con 0x7fb1a00b2860 2024-04-23T14:38:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.816+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a00cbd40 con 0x7fb1a00b2860 2024-04-23T14:38:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.816+0000 7fb196a00640 1 -- 172.21.15.59:0/3894164667 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198034690 con 0x7fb1a00b2860 2024-04-23T14:38:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.816+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a00cc300 con 0x7fb1a00b2860 2024-04-23T14:38:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.817+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a00b5e10 con 0x7fb1a00b2860 2024-04-23T14:38:14.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.818+0000 7fb196a00640 1 -- 172.21.15.59:0/3894164667 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb198034830 con 0x7fb1a00b2860 2024-04-23T14:38:14.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.819+0000 7fb196a00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb18006c920 0x7fb18006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:14.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.820+0000 7fb196a00640 1 -- 172.21.15.59:0/3894164667 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fb1980abd70 con 0x7fb1a00b2860 2024-04-23T14:38:14.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.820+0000 7fb19e800640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb18006c920 0x7fb18006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:14.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.820+0000 7fb19e800640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb18006c920 0x7fb18006ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb190005490 tx=0x7fb190009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:14.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:14.825+0000 7fb196a00640 1 -- 172.21.15.59:0/3894164667 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1a00b5e10 con 0x7fb1a00b2860 2024-04-23T14:38:15.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.072+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1a004fd90 con 0x7fb1a00b2860 2024-04-23T14:38:15.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.073+0000 7fb196a00640 1 -- 172.21.15.59:0/3894164667 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5185 (secure 0 0 0) 0x7fb1a00cf030 con 0x7fb1a00b2860 2024-04-23T14:38:15.075 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:15.075 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:37:49.879970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":18,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":5}]} 2024-04-23T14:38:15.075 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 51 2024-04-23T14:38:15.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.076+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb18006c920 msgr2=0x7fb18006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.076+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb18006c920 0x7fb18006ede0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb190005490 tx=0x7fb190009040 comp rx=0 tx=0).stop 2024-04-23T14:38:15.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.076+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00b2860 msgr2=0x7fb1a01f8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.076+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00b2860 0x7fb1a01f8180 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb19802b620 tx=0x7fb19802d070 comp rx=0 tx=0).stop 2024-04-23T14:38:15.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 shutdown_connections 2024-04-23T14:38:15.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb18006c920 0x7fb18006ede0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00b2860 0x7fb1a01f8180 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a014c350 0x7fb1a00b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 --2- 172.21.15.59:0/3894164667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1840051c0 0x7fb1a015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 >> 172.21.15.59:0/3894164667 conn(0x7fb1a009d920 msgr2=0x7fb1a015cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 shutdown_connections 2024-04-23T14:38:15.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.077+0000 7fb1a4a00640 1 -- 172.21.15.59:0/3894164667 wait complete. 2024-04-23T14:38:15.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-23T14:38:15.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.307+0000 7f03b2600640 1 Processor -- start 2024-04-23T14:38:15.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.307+0000 7f03b2600640 1 -- start start 2024-04-23T14:38:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.309+0000 7f03b2600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f03ac0cf6d0 0x7f03ac165820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.309+0000 7f03b2600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03ac1448f0 con 0x7f03ac144510 2024-04-23T14:38:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.309+0000 7f03b2600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03ac0cfca0 con 0x7f03ac0cd420 2024-04-23T14:38:15.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.309+0000 7f03b2600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f03ac0b4850 con 0x7f03ac0cf6d0 2024-04-23T14:38:15.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.309+0000 7f03b1200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f03ac144510 0x7f03ac0ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39674/0 (socket says 172.21.15.59:39674) 2024-04-23T14:38:15.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.310+0000 7f03b1c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f03ac0cf6d0 0x7f03ac165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.310+0000 7f03b1200640 1 -- 172.21.15.59:0/957702775 learned_addr learned my addr 172.21.15.59:0/957702775 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:15.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.310+0000 7f03b1c00640 1 -- 172.21.15.59:0/957702775 >> v1:172.21.15.76:6789/0 conn(0x7f03ac0cd420 legacy=0x7f03ac0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.310+0000 7f03b1c00640 1 -- 172.21.15.59:0/957702775 >> v1:172.21.15.59:6789/0 conn(0x7f03ac144510 legacy=0x7f03ac0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.310+0000 7f03b1c00640 1 -- 172.21.15.59:0/957702775 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac0cff80 con 0x7f03ac0cf6d0 2024-04-23T14:38:15.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.311+0000 7f03b1c00640 1 --2- 172.21.15.59:0/957702775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac0cf6d0 0x7f03ac165820 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009d90 tx=0x7f03a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45e938848d8e949 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.312+0000 7f03ab400640 1 -- 172.21.15.59:0/957702775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4007740 con 0x7f03ac0cf6d0 2024-04-23T14:38:15.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.312+0000 7f03ab400640 1 -- 172.21.15.59:0/957702775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a40078e0 con 0x7f03ac0cf6d0 2024-04-23T14:38:15.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.312+0000 7f03ab400640 1 -- 172.21.15.59:0/957702775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4007cb0 con 0x7f03ac0cf6d0 2024-04-23T14:38:15.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.312+0000 7f03b2600640 1 -- 172.21.15.59:0/957702775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac0cf6d0 msgr2=0x7f03ac165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.312+0000 7f03b2600640 1 --2- 172.21.15.59:0/957702775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac0cf6d0 0x7f03ac165820 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009d90 tx=0x7f03a402a660 comp rx=0 tx=0).stop 2024-04-23T14:38:15.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.313+0000 7f03b2600640 1 -- 172.21.15.59:0/957702775 shutdown_connections 2024-04-23T14:38:15.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.313+0000 7f03b2600640 1 --2- 172.21.15.59:0/957702775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac0cf6d0 0x7f03ac165820 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.313+0000 7f03b2600640 1 -- 172.21.15.59:0/957702775 >> 172.21.15.59:0/957702775 conn(0x7f03ac06b9d0 msgr2=0x7f03ac06bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.313+0000 7f03b2600640 1 -- 172.21.15.59:0/957702775 shutdown_connections 2024-04-23T14:38:15.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.313+0000 7f03b2600640 1 -- 172.21.15.59:0/957702775 wait complete. 2024-04-23T14:38:15.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.314+0000 7f03b2600640 1 Processor -- start 2024-04-23T14:38:15.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.315+0000 7f03b2600640 1 -- start start 2024-04-23T14:38:15.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b2600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03ac0cd420 0x7f03ac13c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b2600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03ac0cf6d0 0x7f03ac1f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac144510 0x7f03ac0c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b2600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac06e570 con 0x7f03ac144510 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b2600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac0b4070 con 0x7f03ac0cd420 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b2600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f03ac06e6f0 con 0x7f03ac0cf6d0 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac144510 0x7f03ac0c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac144510 0x7f03ac0c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45236/0 (socket says 172.21.15.59:45236) 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1c00640 1 -- 172.21.15.59:0/1518721323 learned_addr learned my addr 172.21.15.59:0/1518721323 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1200640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03ac0cd420 0x7f03ac13c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03abe00640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03ac0cf6d0 0x7f03ac1f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1c00640 1 -- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03ac0cf6d0 msgr2=0x7f03ac1f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1c00640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03ac0cf6d0 0x7f03ac1f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1c00640 1 -- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03ac0cd420 msgr2=0x7f03ac13c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.316+0000 7f03b1c00640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03ac0cd420 0x7f03ac13c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03b1c00640 1 -- 172.21.15.59:0/1518721323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac1f8970 con 0x7f03ac144510 2024-04-23T14:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03abe00640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03ac0cf6d0 0x7f03ac1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03b1c00640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac144510 0x7f03ac0c5d80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f03a402ac50 tx=0x7f03a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03a8c00640 1 -- 172.21.15.59:0/1518721323 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a403f070 con 0x7f03ac144510 2024-04-23T14:38:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03a8c00640 1 -- 172.21.15.59:0/1518721323 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a4031c90 con 0x7f03ac144510 2024-04-23T14:38:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03a8c00640 1 -- 172.21.15.59:0/1518721323 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4044680 con 0x7f03ac144510 2024-04-23T14:38:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac13e170 con 0x7f03ac144510 2024-04-23T14:38:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.317+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac13e730 con 0x7f03ac144510 2024-04-23T14:38:15.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.320+0000 7f03a8c00640 1 -- 172.21.15.59:0/1518721323 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03a403b410 con 0x7f03ac144510 2024-04-23T14:38:15.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.321+0000 7f03a8c00640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f038c06c8f0 0x7f038c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.322+0000 7f03a8c00640 1 -- 172.21.15.59:0/1518721323 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f03a40ad510 con 0x7f03ac144510 2024-04-23T14:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.322+0000 7f03b1200640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f038c06c8f0 0x7f038c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.322+0000 7f03b69d8740 1 Processor -- start 2024-04-23T14:38:15.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.322+0000 7f03b69d8740 1 -- start start 2024-04-23T14:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b1200640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f038c06c8f0 0x7f038c06edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005880 tx=0x7f03a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cc4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5ce2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5c99f00 0x558ce5ce0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b69d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ce5bfe190 con 0x558ce5cc4220 2024-04-23T14:38:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558ce5ba4030 con 0x558ce5ce1ac0 2024-04-23T14:38:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558ce5bfa150 con 0x558ce5c99f00 2024-04-23T14:38:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cc4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cc4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45252/0 (socket says 172.21.15.59:45252) 2024-04-23T14:38:15.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.323+0000 7f03b1200640 1 -- 172.21.15.59:0/2366460183 learned_addr learned my addr 172.21.15.59:0/2366460183 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:15.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03b1c00640 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5c99f00 0x558ce5ce0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03b1200640 1 -- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5c99f00 msgr2=0x558ce5ce0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03abe00640 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5ce2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03b1200640 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5c99f00 0x558ce5ce0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03b1200640 1 -- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 msgr2=0x558ce5ce2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03b1200640 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5ce2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03b1200640 1 -- 172.21.15.59:0/2366460183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce5b8cb60 con 0x558ce5cc4220 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f03b1200640 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cc4600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005b60 tx=0x7f03a0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115ae7ee08f62840 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.324+0000 7f039d600640 1 -- 172.21.15.59:0/2366460183 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000d9a0 con 0x558ce5cc4220 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f039d600640 1 -- 172.21.15.59:0/2366460183 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a000db40 con 0x558ce5cc4220 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f039d600640 1 -- 172.21.15.59:0/2366460183 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0020450 con 0x558ce5cc4220 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 -- 172.21.15.59:0/2366460183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 msgr2=0x558ce5cc4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cc4600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005b60 tx=0x7f03a0010000 comp rx=0 tx=0).stop 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 -- 172.21.15.59:0/2366460183 shutdown_connections 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5c99f00 0x558ce5ce0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5ce2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 --2- 172.21.15.59:0/2366460183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cc4600 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 -- 172.21.15.59:0/2366460183 >> 172.21.15.59:0/2366460183 conn(0x558ce5c96830 msgr2=0x558ce5cdf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 -- 172.21.15.59:0/2366460183 shutdown_connections 2024-04-23T14:38:15.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.325+0000 7f03b69d8740 1 -- 172.21.15.59:0/2366460183 wait complete. 2024-04-23T14:38:15.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.326+0000 7f03b69d8740 1 -- start start 2024-04-23T14:38:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.326+0000 7f03b69d8740 1 Processor -- start 2024-04-23T14:38:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.327+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cbc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.327+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5ce1ac0 0x558ce5cd76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.327+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5cc35b0 0x558ce5cc71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.327+0000 7f03b69d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ce5c9aca0 con 0x558ce5ce1ac0 2024-04-23T14:38:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.327+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558ce5bb7500 con 0x558ce5cc35b0 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.327+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558ce5c94760 con 0x558ce5cc4220 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.327+0000 7f03b69d8740 10 client.0 _ll_register_callbacks cb 0x558ce5bf4270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5ce1ac0 0x558ce5cd76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5ce1ac0 0x558ce5cd76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45262/0 (socket says 172.21.15.59:45262) 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 -- 172.21.15.59:0/1654885964 learned_addr learned my addr 172.21.15.59:0/1654885964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03b1200640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cbc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03b1c00640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5cc35b0 0x558ce5cc71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 -- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 msgr2=0x558ce5cbc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cbc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 -- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5cc35b0 msgr2=0x558ce5cc71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5cc35b0 0x558ce5cc71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 -- 172.21.15.59:0/1654885964 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce59dc7c0 con 0x558ce5ce1ac0 2024-04-23T14:38:15.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03b1200640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cbc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f03abe00640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5ce1ac0 0x558ce5cd76a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f039800edc0 tx=0x7f0398001430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8915639f9260966 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.328+0000 7f0392000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f0392000640 1 -- 172.21.15.59:0/1654885964 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398007230 con 0x558ce5ce1ac0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f0392000640 1 -- 172.21.15.59:0/1654885964 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f03980073d0 con 0x558ce5ce1ac0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a000db40 con 0x558ce5ce1ac0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f0392000640 1 -- 172.21.15.59:0/1654885964 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f039801f050 con 0x558ce5ce1ac0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f03b69d8740 10 client.7299 fetch_fsmap learned FSMap version 51 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x558ce5a3cf50 con 0x558ce5ce1ac0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f0392000640 1 -- 172.21.15.59:0/1654885964 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a000db40 con 0x558ce5ce1ac0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f0392000640 1 -- 172.21.15.59:0/1654885964 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f0398022020 con 0x558ce5ce1ac0 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f03b69d8740 10 client.7299 fetch_fsmap finished waiting for FSMap version 51 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f03b69d8740 10 client.7299 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f03b69d8740 20 client.7299 populate_metadata read hostname 'smithi059' 2024-04-23T14:38:15.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.329+0000 7f03b69d8740 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cc0b80 0x558ce5cfdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.330+0000 7f03b69d8740 4 client.7299 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:38:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.330+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce5c0bbb0 con 0x558ce5cc0b80 2024-04-23T14:38:15.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.330+0000 7f03b1200640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cc0b80 0x558ce5cfdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.330+0000 7f03b1200640 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cc0b80 0x558ce5cfdcd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3dacd7b34690200 server_cookie=28c5cb56803ab7a0 in_seq=0 out_seq=0 2024-04-23T14:38:15.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.330+0000 7f0392000640 10 client.7299 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:15.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.331+0000 7f0392000640 1 -- 172.21.15.59:0/1654885964 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ce5c0bbb0 con 0x558ce5cc0b80 2024-04-23T14:38:15.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.331+0000 7f0392000640 10 client.7299 handle_command_reply: tid=0 2024-04-23T14:38:15.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.331+0000 7f03b69d8740 1 client.7299 shutdown 2024-04-23T14:38:15.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.331+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5ce1ac0 msgr2=0x558ce5cd76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.331+0000 7f03b69d8740 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5ce1ac0 0x558ce5cd76a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f039800edc0 tx=0x7f0398001430 comp rx=0 tx=0).stop 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 shutdown_connections 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cc0b80 0x558ce5cfdcd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5cc35b0 0x558ce5cc71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5ce1ac0 0x558ce5cd76a0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 --2- 172.21.15.59:0/1654885964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cbc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 >> 172.21.15.59:0/1654885964 conn(0x558ce5c99f00 msgr2=0x558ce5ce25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f0392000640 0 client.7299 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 shutdown_connections 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 1 -- 172.21.15.59:0/1654885964 wait complete. 2024-04-23T14:38:15.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.332+0000 7f03b69d8740 20 client.7299 trim_cache size 0 max 16384 2024-04-23T14:38:15.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.347+0000 7f03b69d8740 1 Processor -- start 2024-04-23T14:38:15.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.347+0000 7f03b69d8740 1 -- start start 2024-04-23T14:38:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc35b0 0x558ce5cde6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cdf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5cdf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03b69d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac200120 con 0x558ce5cc4220 2024-04-23T14:38:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac13b560 con 0x558ce5ce1ac0 2024-04-23T14:38:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f03ac13bbb0 con 0x558ce5cc35b0 2024-04-23T14:38:15.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cdf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03abe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cdf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45264/0 (socket says 172.21.15.59:45264) 2024-04-23T14:38:15.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03abe00640 1 -- 172.21.15.59:0/3201095453 learned_addr learned my addr 172.21.15.59:0/3201095453 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:15.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.348+0000 7f03b1200640 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc35b0 0x558ce5cde6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03b1c00640 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5cdf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03abe00640 1 -- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc35b0 msgr2=0x558ce5cde6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03abe00640 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc35b0 0x558ce5cde6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03abe00640 1 -- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 msgr2=0x558ce5cdf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03abe00640 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5cdf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03abe00640 1 -- 172.21.15.59:0/3201095453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce58d00b0 con 0x558ce5cc4220 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03b1200640 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc35b0 0x558ce5cde6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f03abe00640 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cdf030 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0398006880 tx=0x7f0398012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed798a3c56e2b0b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f039cc00640 1 -- 172.21.15.59:0/3201095453 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980050d0 con 0x558ce5cc4220 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f039cc00640 1 -- 172.21.15.59:0/3201095453 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039800c070 con 0x558ce5cc4220 2024-04-23T14:38:15.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.349+0000 7f039cc00640 1 -- 172.21.15.59:0/3201095453 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398005490 con 0x558ce5cc4220 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 -- 172.21.15.59:0/3201095453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 msgr2=0x558ce5cdf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cdf030 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0398006880 tx=0x7f0398012000 comp rx=0 tx=0).stop 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 -- 172.21.15.59:0/3201095453 shutdown_connections 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5cdf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc4220 0x558ce5cdf030 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 --2- 172.21.15.59:0/3201095453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc35b0 0x558ce5cde6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 -- 172.21.15.59:0/3201095453 >> 172.21.15.59:0/3201095453 conn(0x558ce5c99f00 msgr2=0x558ce5c9a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 -- 172.21.15.59:0/3201095453 shutdown_connections 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.350+0000 7f03b69d8740 1 -- 172.21.15.59:0/3201095453 wait complete. 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.351+0000 7f03b69d8740 1 -- start start 2024-04-23T14:38:15.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.351+0000 7f03b69d8740 1 Processor -- start 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc35b0 0x558ce5ceda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cf8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b69d8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5cf8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b69d8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ce5cc1ef0 con 0x558ce5cc35b0 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558ce5cc3090 con 0x558ce5ce1ac0 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b69d8740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558ce5cefab0 con 0x558ce5cc4220 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b69d8740 10 client.0 _ll_register_callbacks cb 0x7f0398007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc35b0 0x558ce5ceda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc35b0 0x558ce5ceda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45266/0 (socket says 172.21.15.59:45266) 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.352+0000 7f03b1200640 1 -- 172.21.15.59:0/702284636 learned_addr learned my addr 172.21.15.59:0/702284636 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f03b1200640 1 -- 172.21.15.59:0/702284636 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 msgr2=0x558ce5cf8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f03b1200640 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cf8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f03b1200640 1 -- 172.21.15.59:0/702284636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 msgr2=0x558ce5cf8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f03b1200640 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5cf8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f03b1200640 1 -- 172.21.15.59:0/702284636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce5cc1ba0 con 0x558ce5cc35b0 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f03b1200640 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc35b0 0x558ce5ceda70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f03a0007d10 tx=0x7f03a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3562eaed0d08544 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f0392000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f0392000640 1 -- 172.21.15.59:0/702284636 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000d9a0 con 0x558ce5cc35b0 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f0392000640 1 -- 172.21.15.59:0/702284636 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f03a002f070 con 0x558ce5cc35b0 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039800c070 con 0x558ce5cc35b0 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.353+0000 7f0392000640 1 -- 172.21.15.59:0/702284636 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f03a003e050 con 0x558ce5cc35b0 2024-04-23T14:38:15.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 10 client.7305 fetch_fsmap learned FSMap version 51 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x558ce5ce2e10 con 0x558ce5cc35b0 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f0392000640 1 -- 172.21.15.59:0/702284636 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039800c070 con 0x558ce5cc35b0 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f0392000640 1 -- 172.21.15.59:0/702284636 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f03a0041020 con 0x558ce5cc35b0 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 10 client.7305 fetch_fsmap finished waiting for FSMap version 51 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 10 client.7305 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 20 client.7305 populate_metadata read hostname 'smithi059' 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cbe430 0x558ce5cbe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 4 client.7305 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 --> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x558ce5c05a20 con 0x558ce5cbe430 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.354+0000 7f03abe00640 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cbe430 0x558ce5cbe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.355+0000 7f03abe00640 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cbe430 0x558ce5cbe810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd88760e9ddfa389 server_cookie=fea3817d468a7e1c in_seq=0 out_seq=0 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.355+0000 7f0392000640 10 client.7305 ms_handle_connect on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.355+0000 7f0392000640 1 -- 172.21.15.59:0/702284636 <== mds.0 v2:172.21.15.59:6834/1773251382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ce5c05a20 con 0x558ce5cbe430 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.355+0000 7f0392000640 10 client.7305 handle_command_reply: tid=0 2024-04-23T14:38:15.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.355+0000 7f03b69d8740 1 client.7305 shutdown 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc35b0 msgr2=0x558ce5ceda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc35b0 0x558ce5ceda70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f03a0007d10 tx=0x7f03a0000ed0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 shutdown_connections 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:6834/1773251382,v1:172.21.15.59:6836/1773251382] conn(0x558ce5cbe430 0x558ce5cbe810 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558ce5cc4220 0x558ce5cf8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558ce5ce1ac0 0x558ce5cf8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f0392000640 0 client.7305 ms_handle_reset on v2:172.21.15.59:6834/1773251382 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 --2- 172.21.15.59:0/702284636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ce5cc35b0 0x558ce5ceda70 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 >> 172.21.15.59:0/702284636 conn(0x558ce5c99f00 msgr2=0x558ce5cdf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 shutdown_connections 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 1 -- 172.21.15.59:0/702284636 wait complete. 2024-04-23T14:38:15.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.356+0000 7f03b69d8740 20 client.7305 trim_cache size 0 max 16384 2024-04-23T14:38:15.361 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:15.361 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f038c06c8f0 msgr2=0x7f038c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f038c06c8f0 0x7f038c06edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005880 tx=0x7f03a0009040 comp rx=0 tx=0).stop 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac144510 msgr2=0x7f03ac0c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac144510 0x7f03ac0c5d80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f03a402ac50 tx=0x7f03a4004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 shutdown_connections 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f038c06c8f0 0x7f038c06edb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03ac0cf6d0 0x7f03ac1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ac144510 0x7f03ac0c5d80 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 --2- 172.21.15.59:0/1518721323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03ac0cd420 0x7f03ac13c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 >> 172.21.15.59:0/1518721323 conn(0x7f03ac06b9d0 msgr2=0x7f03ac1429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 shutdown_connections 2024-04-23T14:38:15.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.357+0000 7f03b2600640 1 -- 172.21.15.59:0/1518721323 wait complete. 2024-04-23T14:38:15.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:38:15.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.582+0000 7f2487200640 1 Processor -- start 2024-04-23T14:38:15.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.582+0000 7f2487200640 1 -- start start 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.583+0000 7f2487200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 0x7f24880ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.583+0000 7f2487200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24880b3870 con 0x7f248806f3f0 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.583+0000 7f2487200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24880b3b50 con 0x7f248815b450 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.583+0000 7f2487200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f24880b41a0 con 0x7f24880cd760 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2485400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f248806f3f0 0x7f24880cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39678/0 (socket says 172.21.15.59:39678) 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2485400640 1 -- 172.21.15.59:0/3159048666 learned_addr learned my addr 172.21.15.59:0/3159048666 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2486800640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 0x7f24880ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342456763 0 0) 0x7f24880b3870 con 0x7f248806f3f0 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f246c003520 con 0x7f248806f3f0 2024-04-23T14:38:15.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 484060952 0 0) 0x7f246c003520 con 0x7f248806f3f0 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 msgr2=0x7f24880ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.584+0000 7f2484a00640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 0x7f24880ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 >> v1:172.21.15.76:6789/0 conn(0x7f248815b450 legacy=0x7f24880b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24880b3e30 con 0x7f248806f3f0 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2474001670 con 0x7f248806f3f0 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 >> v1:172.21.15.59:6789/0 conn(0x7f248806f3f0 legacy=0x7f24880cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c0039e0 0x7f246c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246c004800 0x7f246c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 0x7f246c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24880b41a0 con 0x7f246c004800 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f246c00dd10 con 0x7f246c0039e0 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f246c00df10 con 0x7f24880cd760 2024-04-23T14:38:15.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2485400640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246c004800 0x7f246c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2485e00640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c0039e0 0x7f246c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2486800640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 0x7f246c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.585+0000 7f2485400640 1 -- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 msgr2=0x7f246c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.586+0000 7f2485400640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24880cd760 0x7f246c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.586+0000 7f2485400640 1 -- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c0039e0 msgr2=0x7f246c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:15.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.586+0000 7f2485400640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c0039e0 0x7f246c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.586+0000 7f2485400640 1 -- 172.21.15.59:0/3159048666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c003520 con 0x7f246c004800 2024-04-23T14:38:15.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.587+0000 7f2485400640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246c004800 0x7f246c004c80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f247400c210 tx=0x7f246c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9756d8172fe5b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.587+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474007cf0 con 0x7f246c004800 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.587+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2474005ad0 con 0x7f246c004800 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.587+0000 7f2484a00640 1 -- 172.21.15.59:0/3159048666 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474007070 con 0x7f246c004800 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.587+0000 7f2487200640 1 -- 172.21.15.59:0/3159048666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246c004800 msgr2=0x7f246c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.587+0000 7f2487200640 1 --2- 172.21.15.59:0/3159048666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246c004800 0x7f246c004c80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f247400c210 tx=0x7f246c005830 comp rx=0 tx=0).stop 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.587+0000 7f2486800640 1 -- 172.21.15.59:0/3159048666 reap_dead start 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.588+0000 7f2487200640 1 -- 172.21.15.59:0/3159048666 shutdown_connections 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.588+0000 7f2487200640 1 -- 172.21.15.59:0/3159048666 >> 172.21.15.59:0/3159048666 conn(0x7f24880b21a0 msgr2=0x7f24880b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.588+0000 7f2487200640 1 -- 172.21.15.59:0/3159048666 shutdown_connections 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.588+0000 7f2487200640 1 -- 172.21.15.59:0/3159048666 wait complete. 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.590+0000 7f2487200640 1 Processor -- start 2024-04-23T14:38:15.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.590+0000 7f2487200640 1 -- start start 2024-04-23T14:38:15.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2487200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c004800 0x7f24880cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2487200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2488157d40 0x7f24881581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2487200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24880c4ca0 0x7f24880c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2487200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24880b3080 con 0x7f24880c4ca0 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2487200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f24880b4ff0 con 0x7f246c004800 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2487200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f24880b3200 con 0x7f2488157d40 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2486800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24880c4ca0 0x7f24880c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2486800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24880c4ca0 0x7f24880c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45288/0 (socket says 172.21.15.59:45288) 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2486800640 1 -- 172.21.15.59:0/1650642006 learned_addr learned my addr 172.21.15.59:0/1650642006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2485400640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2488157d40 0x7f24881581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.591+0000 7f2485e00640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c004800 0x7f24880cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2486800640 1 -- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2488157d40 msgr2=0x7f24881581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2486800640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2488157d40 0x7f24881581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2486800640 1 -- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c004800 msgr2=0x7f24880cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2486800640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c004800 0x7f24880cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2486800640 1 -- 172.21.15.59:0/1650642006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24880c3c00 con 0x7f24880c4ca0 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2486800640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24880c4ca0 0x7f24880c5120 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f248806ee90 tx=0x7f24800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f247e000640 1 -- 172.21.15.59:0/1650642006 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480003990 con 0x7f24880c4ca0 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f247e000640 1 -- 172.21.15.59:0/1650642006 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248001b070 con 0x7f24880c4ca0 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24880c3ee0 con 0x7f24880c4ca0 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f247e000640 1 -- 172.21.15.59:0/1650642006 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480003b70 con 0x7f24880c4ca0 2024-04-23T14:38:15.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.592+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24880c44a0 con 0x7f24880c4ca0 2024-04-23T14:38:15.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.593+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248804fd90 con 0x7f24880c4ca0 2024-04-23T14:38:15.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.595+0000 7f247e000640 1 -- 172.21.15.59:0/1650642006 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2480005d10 con 0x7f24880c4ca0 2024-04-23T14:38:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.596+0000 7f247e000640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f246806c8f0 0x7f246806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.596+0000 7f247e000640 1 -- 172.21.15.59:0/1650642006 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f248008c330 con 0x7f24880c4ca0 2024-04-23T14:38:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.596+0000 7f2485e00640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f246806c8f0 0x7f246806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.597+0000 7f2485e00640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f246806c8f0 0x7f246806edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2478005490 tx=0x7f2478009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.600+0000 7f247e000640 1 -- 172.21.15.59:0/1650642006 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2480025020 con 0x7f24880c4ca0 2024-04-23T14:38:15.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.843+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2488159540 con 0x7f24880c4ca0 2024-04-23T14:38:15.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.844+0000 7f247e000640 1 -- 172.21.15.59:0/1650642006 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f248005a630 con 0x7f24880c4ca0 2024-04-23T14:38:15.845 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:15.845 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T14:38:15.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f246806c8f0 msgr2=0x7f246806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f246806c8f0 0x7f246806edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2478005490 tx=0x7f2478009040 comp rx=0 tx=0).stop 2024-04-23T14:38:15.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24880c4ca0 msgr2=0x7f24880c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24880c4ca0 0x7f24880c5120 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f248806ee90 tx=0x7f24800115a0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 shutdown_connections 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f246806c8f0 0x7f246806edb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2488157d40 0x7f24881581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24880c4ca0 0x7f24880c5120 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 --2- 172.21.15.59:0/1650642006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246c004800 0x7f24880cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 >> 172.21.15.59:0/1650642006 conn(0x7f24880b21a0 msgr2=0x7f2488159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.846+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 shutdown_connections 2024-04-23T14:38:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:15.847+0000 7f2487200640 1 -- 172.21.15.59:0/1650642006 wait complete. 2024-04-23T14:38:15.864 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-23T14:38:15.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.066+0000 7f12d6000640 1 Processor -- start 2024-04-23T14:38:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.066+0000 7f12d6000640 1 -- start start 2024-04-23T14:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.068+0000 7f12d6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f12d00afaf0 0x7f12d00d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.068+0000 7f12d6000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12d00d06e0 0x7f12d01624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.068+0000 7f12d6000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d015f0f0 con 0x7f12d015c5e0 2024-04-23T14:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.068+0000 7f12d6000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f12d015fa20 con 0x7f12d00afaf0 2024-04-23T14:38:16.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.068+0000 7f12d6000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f12d015f3d0 con 0x7f12d00d06e0 2024-04-23T14:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f12d00afaf0 0x7f12d00d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12d00d06e0 0x7f12d01624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12d00d06e0 0x7f12d01624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:33024/0 (socket says 172.21.15.59:33024) 2024-04-23T14:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cfe00640 1 -- 172.21.15.59:0/3921604489 learned_addr learned my addr 172.21.15.59:0/3921604489 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cea00640 1 -- 172.21.15.59:0/3921604489 >> v2:172.21.15.76:3300/0 conn(0x7f12d00d06e0 msgr2=0x7f12d01624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cea00640 1 --2- 172.21.15.59:0/3921604489 >> v2:172.21.15.76:3300/0 conn(0x7f12d00d06e0 0x7f12d01624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cea00640 1 -- 172.21.15.59:0/3921604489 >> v1:172.21.15.76:6789/0 conn(0x7f12d015c5e0 legacy=0x7f12d00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.069+0000 7f12cea00640 1 -- 172.21.15.59:0/3921604489 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d015f6b0 con 0x7f12d00afaf0 2024-04-23T14:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.070+0000 7f12cea00640 1 --2- 172.21.15.59:0/3921604489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00afaf0 0x7f12d00d01a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f12d006c7c0 tx=0x7f12b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13303ad53a39b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12ce000640 1 -- 172.21.15.59:0/3921604489 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8037070 con 0x7f12d00afaf0 2024-04-23T14:38:16.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12ce000640 1 -- 172.21.15.59:0/3921604489 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b8032440 con 0x7f12d00afaf0 2024-04-23T14:38:16.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12ce000640 1 -- 172.21.15.59:0/3921604489 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8032810 con 0x7f12d00afaf0 2024-04-23T14:38:16.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 -- 172.21.15.59:0/3921604489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00afaf0 msgr2=0x7f12d00d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:16.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 --2- 172.21.15.59:0/3921604489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00afaf0 0x7f12d00d01a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f12d006c7c0 tx=0x7f12b802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 -- 172.21.15.59:0/3921604489 shutdown_connections 2024-04-23T14:38:16.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 --2- 172.21.15.59:0/3921604489 >> v2:172.21.15.76:3300/0 conn(0x7f12d00d06e0 0x7f12d01624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 --2- 172.21.15.59:0/3921604489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00afaf0 0x7f12d00d01a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 -- 172.21.15.59:0/3921604489 >> 172.21.15.59:0/3921604489 conn(0x7f12d0067ea0 msgr2=0x7f12d006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:16.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 -- 172.21.15.59:0/3921604489 shutdown_connections 2024-04-23T14:38:16.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.071+0000 7f12d6000640 1 -- 172.21.15.59:0/3921604489 wait complete. 2024-04-23T14:38:16.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.073+0000 7f12d6000640 1 Processor -- start 2024-04-23T14:38:16.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.073+0000 7f12d6000640 1 -- start start 2024-04-23T14:38:16.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.074+0000 7f12d6000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12d00afaf0 0x7f12d0149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.074+0000 7f12d6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00d06e0 0x7f12d0159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.074+0000 7f12d6000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d015c5e0 0x7f12d015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.074+0000 7f12d6000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12d006e600 con 0x7f12d00d06e0 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.074+0000 7f12d6000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f12d006e480 con 0x7f12d015c5e0 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.074+0000 7f12d6000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f12d0162a10 con 0x7f12d00afaf0 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00d06e0 0x7f12d0159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00d06e0 0x7f12d0159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45302/0 (socket says 172.21.15.59:45302) 2024-04-23T14:38:16.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 -- 172.21.15.59:0/2358482096 learned_addr learned my addr 172.21.15.59:0/2358482096 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:16.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cf400640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12d00afaf0 0x7f12d0149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:16.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cfe00640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d015c5e0 0x7f12d015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:16.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 -- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12d00afaf0 msgr2=0x7f12d0149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12d00afaf0 0x7f12d0149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 -- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d015c5e0 msgr2=0x7f12d015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d015c5e0 0x7f12d015b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cea00640 1 -- 172.21.15.59:0/2358482096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d014a390 con 0x7f12d00d06e0 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.075+0000 7f12cf400640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12d00afaf0 0x7f12d0149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.076+0000 7f12cea00640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00d06e0 0x7f12d0159920 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009890 tx=0x7f12b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.076+0000 7f12c7400640 1 -- 172.21.15.59:0/2358482096 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8041070 con 0x7f12d00d06e0 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.076+0000 7f12c7400640 1 -- 172.21.15.59:0/2358482096 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b803c890 con 0x7f12d00d06e0 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.076+0000 7f12c7400640 1 -- 172.21.15.59:0/2358482096 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8037040 con 0x7f12d00d06e0 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.076+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d0159e60 con 0x7f12d00d06e0 2024-04-23T14:38:16.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.076+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d015a410 con 0x7f12d00d06e0 2024-04-23T14:38:16.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.077+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d0068a80 con 0x7f12d00d06e0 2024-04-23T14:38:16.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.078+0000 7f12c7400640 1 -- 172.21.15.59:0/2358482096 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12b8044070 con 0x7f12d00d06e0 2024-04-23T14:38:16.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.079+0000 7f12c7400640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f12ac06c8f0 0x7f12ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:16.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.079+0000 7f12c7400640 1 -- 172.21.15.59:0/2358482096 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f12b80ac5d0 con 0x7f12d00d06e0 2024-04-23T14:38:16.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.079+0000 7f12cf400640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f12ac06c8f0 0x7f12ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:16.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.080+0000 7f12cf400640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f12ac06c8f0 0x7f12ac06edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12d01f7c00 tx=0x7f12c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:16.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.084+0000 7f12c7400640 1 -- 172.21.15.59:0/2358482096 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12d014a390 con 0x7f12d00d06e0 2024-04-23T14:38:16.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.333+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f12d00d0080 con 0x7f12d00d06e0 2024-04-23T14:38:16.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.334+0000 7f12c7400640 1 -- 172.21.15.59:0/2358482096 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f12b807a960 con 0x7f12d00d06e0 2024-04-23T14:38:16.335 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.336+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f12ac06c8f0 msgr2=0x7f12ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.336+0000 7f12d6000640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f12ac06c8f0 0x7f12ac06edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12d01f7c00 tx=0x7f12c0009040 comp rx=0 tx=0).stop 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.336+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00d06e0 msgr2=0x7f12d0159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.336+0000 7f12d6000640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00d06e0 0x7f12d0159920 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009890 tx=0x7f12b80046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 shutdown_connections 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f12ac06c8f0 0x7f12ac06edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d015c5e0 0x7f12d015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12d00d06e0 0x7f12d0159920 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 --2- 172.21.15.59:0/2358482096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f12d00afaf0 0x7f12d0149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 >> 172.21.15.59:0/2358482096 conn(0x7f12d0067ea0 msgr2=0x7f12d00b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 shutdown_connections 2024-04-23T14:38:16.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16.337+0000 7f12d6000640 1 -- 172.21.15.59:0/2358482096 wait complete. 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:38:16.357 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:38:16.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:16.358 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:38:16.412 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42052sec preferred_lft 42052sec 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:38:16.413 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:38:16.414 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:16.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:16.415 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:38:16.502 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:38:16.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:16.507 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:38:16.563 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:38:16.563 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:38:16.564 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:38:16.564 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:38:16.564 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:38:16.564 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:38:16.564 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:38:16.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:38:16.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:38:16.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:38:16.564 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:38:16.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:38:16.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:38:16.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:16.787 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:38:16.838 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:38:16.838 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:38:16.839 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:38:16.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:16.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:38:16.924 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:38:16.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:38:16.925 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:16.978 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:38:16.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:38:16.978 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:17.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:38:17.122 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:38:17.182 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:38:17.183 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:38:17.184 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:38:17.185 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:38:17.185 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:38:17.185 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:38:17.185 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:17.185 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:17.185 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:17.185 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:17.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:17.186 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:38:17.240 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:38:17.241 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:38:17.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:38:17.316 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:38:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:38:17.360 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:17.361 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:17.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:17.362 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:38:17.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:38:17.423+0000 7f4547466480 -1 init, newargv = 0x56431f555660 newargc=15 2024-04-23T14:38:17.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[60879]: starting ceph client 2024-04-23T14:38:17.433 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[60879]: starting fuse 2024-04-23T14:38:17.450 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:38:17.450 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:38:17.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:38:17.451 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:38:17.452 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:38:17.643 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.60879.asok 2024-04-23T14:38:17.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:38:17.661 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60879.asok 2024-04-23T14:38:17.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:17.661 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60879.asok status 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "60879", 2024-04-23T14:38:17.899 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7323, 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7323 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 106274086 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:17.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 106274086 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7323 192.168.0.1:0/106274086", 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/106274086", 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 51, 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 72, 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:38:17.901 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:38:17.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:17.914 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:38:17.914 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:17.972 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:38:17.972 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:17.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:17.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:18.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:18.064 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-23T14:38:18.123 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:38:18.123 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-23T14:38:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.239+0000 7f9db4b91f40 1 Processor -- start 2024-04-23T14:38:18.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.240+0000 7f9db4b91f40 1 -- start start 2024-04-23T14:38:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.240+0000 7f9db4b91f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5592289c61d0 0x559228a15320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.240+0000 7f9db4b91f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559228a88c90 0x5592289fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.240+0000 7f9db4b91f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a9f980 0x559228a86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.240+0000 7f9db4b91f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559228a9fd60 con 0x559228a88c90 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.240+0000 7f9db4b91f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55922896fcc0 con 0x559228a9f980 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.240+0000 7f9db4b91f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x559228a141f0 con 0x5592289c61d0 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559228a88c90 0x5592289fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559228a88c90 0x5592289fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45330/0 (socket says 172.21.15.59:45330) 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 -- 172.21.15.59:0/3837982434 learned_addr learned my addr 172.21.15.59:0/3837982434 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1e00640 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5592289c61d0 0x559228a15320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db2800640 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a9f980 0x559228a86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 -- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5592289c61d0 msgr2=0x559228a15320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5592289c61d0 0x559228a15320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 -- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a9f980 msgr2=0x559228a86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a9f980 0x559228a86760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1400640 1 -- 172.21.15.59:0/3837982434 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559228a85420 con 0x559228a88c90 2024-04-23T14:38:18.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.241+0000 7f9db1e00640 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5592289c61d0 0x559228a15320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:18.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.242+0000 7f9db1400640 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559228a88c90 0x5592289fb360 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00a4f0 tx=0x7f9d9c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dac35a4133cca45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.242+0000 7f9dabe00640 1 -- 172.21.15.59:0/3837982434 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c053b30 con 0x559228a88c90 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.242+0000 7f9dabe00640 1 -- 172.21.15.59:0/3837982434 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c053cd0 con 0x559228a88c90 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.242+0000 7f9dabe00640 1 -- 172.21.15.59:0/3837982434 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c067670 con 0x559228a88c90 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.242+0000 7f9db4b91f40 1 -- 172.21.15.59:0/3837982434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559228a88c90 msgr2=0x5592289fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.242+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559228a88c90 0x5592289fb360 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00a4f0 tx=0x7f9d9c051000 comp rx=0 tx=0).stop 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 -- 172.21.15.59:0/3837982434 shutdown_connections 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a9f980 0x559228a86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559228a88c90 0x5592289fb360 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/3837982434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5592289c61d0 0x559228a15320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 -- 172.21.15.59:0/3837982434 >> 172.21.15.59:0/3837982434 conn(0x5592289f7f40 msgr2=0x559228a113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:18.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 -- 172.21.15.59:0/3837982434 shutdown_connections 2024-04-23T14:38:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 -- 172.21.15.59:0/3837982434 wait complete. 2024-04-23T14:38:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.243+0000 7f9db4b91f40 1 Processor -- start 2024-04-23T14:38:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.244+0000 7f9db4b91f40 1 -- start start 2024-04-23T14:38:18.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db4b91f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592289c61d0 0x5592289f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db4b91f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a88c90 0x559228a83f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db4b91f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559228a9f980 0x559228b2e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db4b91f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559228a856c0 con 0x5592289c61d0 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db4b91f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559228a11c80 con 0x559228a88c90 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db4b91f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5592289fae40 con 0x559228a9f980 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592289c61d0 0x5592289f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592289c61d0 0x5592289f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45332/0 (socket says 172.21.15.59:45332) 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 -- 172.21.15.59:0/2812751123 learned_addr learned my addr 172.21.15.59:0/2812751123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db2800640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559228a9f980 0x559228b2e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1400640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a88c90 0x559228a83f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 -- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559228a9f980 msgr2=0x559228b2e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559228a9f980 0x559228b2e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 -- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a88c90 msgr2=0x559228a83f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a88c90 0x559228a83f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1e00640 1 -- 172.21.15.59:0/2812751123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559228a886e0 con 0x5592289c61d0 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db1400640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559228a88c90 0x559228a83f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.245+0000 7f9db2800640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559228a9f980 0x559228b2e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.246+0000 7f9db1e00640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592289c61d0 0x5592289f52b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009430 tx=0x7f9da40098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.246+0000 7f9daaa00640 1 -- 172.21.15.59:0/2812751123 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da400f780 con 0x5592289c61d0 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.246+0000 7f9daaa00640 1 -- 172.21.15.59:0/2812751123 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da400f920 con 0x5592289c61d0 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.246+0000 7f9daaa00640 1 -- 172.21.15.59:0/2812751123 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da400fd10 con 0x5592289c61d0 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.246+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559228a848f0 con 0x5592289c61d0 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.246+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559228a84eb0 con 0x5592289c61d0 2024-04-23T14:38:18.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.248+0000 7f9daaa00640 1 -- 172.21.15.59:0/2812751123 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9da401b470 con 0x5592289c61d0 2024-04-23T14:38:18.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.249+0000 7f9daaa00640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d9006b2d0 0x7f9d9006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.249+0000 7f9daaa00640 1 -- 172.21.15.59:0/2812751123 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f9da408acb0 con 0x5592289c61d0 2024-04-23T14:38:18.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.250+0000 7f9db1400640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d9006b2d0 0x7f9d9006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.250+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x559228a84450 0x559228b2f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.250+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72) -- 0x559228a116f0 con 0x559228a84450 2024-04-23T14:38:18.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.250+0000 7f9db1400640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d9006b2d0 0x7f9d9006d790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c064000 tx=0x7f9d9c053000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:18.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.250+0000 7f9db2800640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x559228a84450 0x559228b2f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.252+0000 7f9db2800640 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x559228a84450 0x559228b2f830 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:18.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db2800640 1 -- 172.21.15.59:0/2812751123 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v72'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dac007560 con 0x559228a84450 2024-04-23T14:38:18.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x559228a84450 msgr2=0x559228b2f830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x559228a84450 0x559228b2f830 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d9006b2d0 msgr2=0x7f9d9006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d9006b2d0 0x7f9d9006d790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c064000 tx=0x7f9d9c053000 comp rx=0 tx=0).stop 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592289c61d0 msgr2=0x5592289f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 --2- 172.21.15.59:0/2812751123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5592289c61d0 0x5592289f52b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009430 tx=0x7f9da40098f0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db2800640 1 -- 172.21.15.59:0/2812751123 reap_dead start 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 shutdown_connections 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 >> 172.21.15.59:0/2812751123 conn(0x5592289f7f40 msgr2=0x559228a9e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 shutdown_connections 2024-04-23T14:38:18.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.256+0000 7f9db4b91f40 1 -- 172.21.15.59:0/2812751123 wait complete. 2024-04-23T14:38:18.263 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:38:18.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:38:18.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:38:18.477 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:38:18.477 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:38:18.477 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:38:18.477 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:18.537 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:18.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.748+0000 7fb69fe00640 1 Processor -- start 2024-04-23T14:38:18.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.749+0000 7fb69fe00640 1 -- start start 2024-04-23T14:38:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb69fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6a00cd760 0x7fb6a00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb69fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb6a00b4690 0x7fb6a00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb69fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6a0160210 con 0x7fb6a015c3d0 2024-04-23T14:38:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb69fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a01604f0 con 0x7fb6a00cd760 2024-04-23T14:38:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb69fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a01607d0 con 0x7fb6a00b4690 2024-04-23T14:38:18.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb69ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb6a015c3d0 0x7fb6a00cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39682/0 (socket says 172.21.15.59:39682) 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb69ea00640 1 -- 172.21.15.59:0/4281603932 learned_addr learned my addr 172.21.15.59:0/4281603932 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.750+0000 7fb696000640 1 --2- 172.21.15.59:0/4281603932 >> v2:172.21.15.59:3300/0 conn(0x7fb6a00cd760 0x7fb6a00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.751+0000 7fb69f400640 1 --2- 172.21.15.59:0/4281603932 >> v2:172.21.15.76:3300/0 conn(0x7fb6a00b4690 0x7fb6a00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.751+0000 7fb69e000640 1 -- 172.21.15.59:0/4281603932 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2620189325 0 0) 0x7fb6a0160210 con 0x7fb6a015c3d0 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.751+0000 7fb69e000640 1 -- 172.21.15.59:0/4281603932 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb684003520 con 0x7fb6a015c3d0 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.751+0000 7fb696000640 1 -- 172.21.15.59:0/4281603932 >> v2:172.21.15.76:3300/0 conn(0x7fb6a00b4690 msgr2=0x7fb6a00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.751+0000 7fb696000640 1 --2- 172.21.15.59:0/4281603932 >> v2:172.21.15.76:3300/0 conn(0x7fb6a00b4690 0x7fb6a00cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.751+0000 7fb696000640 1 -- 172.21.15.59:0/4281603932 >> v1:172.21.15.59:6789/0 conn(0x7fb6a015c3d0 legacy=0x7fb6a00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.751+0000 7fb696000640 1 -- 172.21.15.59:0/4281603932 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a0160ab0 con 0x7fb6a00cd760 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.752+0000 7fb696000640 1 --2- 172.21.15.59:0/4281603932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a00b4150 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00b2de0 tx=0x7fb68802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18074158a8b51fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.752+0000 7fb69e000640 1 -- 172.21.15.59:0/4281603932 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688037070 con 0x7fb6a00cd760 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.752+0000 7fb69e000640 1 -- 172.21.15.59:0/4281603932 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb688032440 con 0x7fb6a00cd760 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.752+0000 7fb69e000640 1 -- 172.21.15.59:0/4281603932 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688032810 con 0x7fb6a00cd760 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 -- 172.21.15.59:0/4281603932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 msgr2=0x7fb6a00b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 --2- 172.21.15.59:0/4281603932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a00b4150 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00b2de0 tx=0x7fb68802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 -- 172.21.15.59:0/4281603932 shutdown_connections 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 --2- 172.21.15.59:0/4281603932 >> v2:172.21.15.76:3300/0 conn(0x7fb6a00b4690 0x7fb6a00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 --2- 172.21.15.59:0/4281603932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a00b4150 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 -- 172.21.15.59:0/4281603932 >> 172.21.15.59:0/4281603932 conn(0x7fb6a00b21a0 msgr2=0x7fb6a00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 -- 172.21.15.59:0/4281603932 shutdown_connections 2024-04-23T14:38:18.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.753+0000 7fb69fe00640 1 -- 172.21.15.59:0/4281603932 wait complete. 2024-04-23T14:38:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.755+0000 7fb69fe00640 1 Processor -- start 2024-04-23T14:38:18.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.755+0000 7fb69fe00640 1 -- start start 2024-04-23T14:38:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.756+0000 7fb69fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6a00b4690 0x7fb6a00c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.756+0000 7fb69fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.756+0000 7fb69fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6a015c3d0 0x7fb6a01f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.756+0000 7fb69fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a015c7b0 con 0x7fb6a00cd760 2024-04-23T14:38:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.756+0000 7fb69fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00b4ae0 con 0x7fb6a015c3d0 2024-04-23T14:38:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.756+0000 7fb69fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a015c930 con 0x7fb6a00b4690 2024-04-23T14:38:18.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a015cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45354/0 (socket says 172.21.15.59:45354) 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 -- 172.21.15.59:0/560802228 learned_addr learned my addr 172.21.15.59:0/560802228 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb69f400640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6a015c3d0 0x7fb6a01f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb69ea00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6a00b4690 0x7fb6a00c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 -- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6a00b4690 msgr2=0x7fb6a00c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6a00b4690 0x7fb6a00c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 -- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6a015c3d0 msgr2=0x7fb6a01f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6a015c3d0 0x7fb6a01f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 -- 172.21.15.59:0/560802228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a015d490 con 0x7fb6a00cd760 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb69ea00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6a00b4690 0x7fb6a00c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:18.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.757+0000 7fb696000640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a015cab0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb688009890 tx=0x7fb6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.758+0000 7fb697e00640 1 -- 172.21.15.59:0/560802228 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688037040 con 0x7fb6a00cd760 2024-04-23T14:38:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.758+0000 7fb697e00640 1 -- 172.21.15.59:0/560802228 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb688042070 con 0x7fb6a00cd760 2024-04-23T14:38:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.758+0000 7fb697e00640 1 -- 172.21.15.59:0/560802228 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688030960 con 0x7fb6a00cd760 2024-04-23T14:38:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.758+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a015d770 con 0x7fb6a00cd760 2024-04-23T14:38:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.758+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a015dd30 con 0x7fb6a00cd760 2024-04-23T14:38:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.759+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a00b1b80 con 0x7fb6a00cd760 2024-04-23T14:38:18.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.760+0000 7fb697e00640 1 -- 172.21.15.59:0/560802228 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb688030b00 con 0x7fb6a00cd760 2024-04-23T14:38:18.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.761+0000 7fb697e00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67c06c8f0 0x7fb67c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:18.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.761+0000 7fb697e00640 1 -- 172.21.15.59:0/560802228 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fb6880ac510 con 0x7fb6a00cd760 2024-04-23T14:38:18.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.761+0000 7fb69ea00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67c06c8f0 0x7fb67c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:18.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.762+0000 7fb69ea00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67c06c8f0 0x7fb67c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb690005880 tx=0x7fb690009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:18.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.766+0000 7fb697e00640 1 -- 172.21.15.59:0/560802228 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6a00b1b80 con 0x7fb6a00cd760 2024-04-23T14:38:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.973+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6a004fd90 con 0x7fb6a00cd760 2024-04-23T14:38:18.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.975+0000 7fb697e00640 1 -- 172.21.15.59:0/560802228 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16834 (secure 0 0 0) 0x7fb68807a8a0 con 0x7fb6a00cd760 2024-04-23T14:38:18.977 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:18.977 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:37:08.944193+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/2883460093":"2024-04-24T14:36:50.829845+0000","172.21.15.59:6834/2883460093":"2024-04-24T14:36:50.829845+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67c06c8f0 msgr2=0x7fb67c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67c06c8f0 0x7fb67c06edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb690005880 tx=0x7fb690009040 comp rx=0 tx=0).stop 2024-04-23T14:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 msgr2=0x7fb6a015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a015cab0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb688009890 tx=0x7fb6880046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 shutdown_connections 2024-04-23T14:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67c06c8f0 0x7fb67c06edb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6a015c3d0 0x7fb6a01f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6a00cd760 0x7fb6a015cab0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 --2- 172.21.15.59:0/560802228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6a00b4690 0x7fb6a00c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.978+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 >> 172.21.15.59:0/560802228 conn(0x7fb6a00b21a0 msgr2=0x7fb6a00b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.979+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 shutdown_connections 2024-04-23T14:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:18.979+0000 7fb69fe00640 1 -- 172.21.15.59:0/560802228 wait complete. 2024-04-23T14:38:18.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:36:27.526201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:36:28.678324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:38:19.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:38:19.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.213+0000 7f11d6200640 1 Processor -- start 2024-04-23T14:38:19.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.213+0000 7f11d6200640 1 -- start start 2024-04-23T14:38:19.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.215+0000 7f11d6200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11d00d0600 0x7f11d00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.215+0000 7f11d6200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d0071e10 con 0x7f11d00d09e0 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.215+0000 7f11d6200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d00720f0 con 0x7f11d006dcb0 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.215+0000 7f11d6200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f11d0072a20 con 0x7f11d00d0600 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.215+0000 7f11d4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11d00d0600 0x7f11d00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.215+0000 7f11d4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11d00d0600 0x7f11d00ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45368/0 (socket says 172.21.15.59:45368) 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.215+0000 7f11d4e00640 1 -- 172.21.15.59:0/4134185966 learned_addr learned my addr 172.21.15.59:0/4134185966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.216+0000 7f11d4e00640 1 -- 172.21.15.59:0/4134185966 >> v1:172.21.15.76:6789/0 conn(0x7f11d006dcb0 legacy=0x7f11d0162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.216+0000 7f11d4e00640 1 -- 172.21.15.59:0/4134185966 >> v1:172.21.15.59:6789/0 conn(0x7f11d00d09e0 legacy=0x7f11d006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.216+0000 7f11d4e00640 1 -- 172.21.15.59:0/4134185966 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d00723d0 con 0x7f11d00d0600 2024-04-23T14:38:19.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.217+0000 7f11d4e00640 1 --2- 172.21.15.59:0/4134185966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d00d0600 0x7f11d00ad170 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009870 tx=0x7f11c402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223700d58954249c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.219+0000 7f11cf400640 1 -- 172.21.15.59:0/4134185966 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4037070 con 0x7f11d00d0600 2024-04-23T14:38:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.219+0000 7f11cf400640 1 -- 172.21.15.59:0/4134185966 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c4032440 con 0x7f11d00d0600 2024-04-23T14:38:19.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.219+0000 7f11cf400640 1 -- 172.21.15.59:0/4134185966 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4032810 con 0x7f11d00d0600 2024-04-23T14:38:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.220+0000 7f11d6200640 1 -- 172.21.15.59:0/4134185966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d00d0600 msgr2=0x7f11d00ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.220+0000 7f11d6200640 1 --2- 172.21.15.59:0/4134185966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d00d0600 0x7f11d00ad170 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009870 tx=0x7f11c402a2a0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.220+0000 7f11d6200640 1 -- 172.21.15.59:0/4134185966 shutdown_connections 2024-04-23T14:38:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.220+0000 7f11d6200640 1 --2- 172.21.15.59:0/4134185966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d00d0600 0x7f11d00ad170 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.220+0000 7f11d6200640 1 -- 172.21.15.59:0/4134185966 >> 172.21.15.59:0/4134185966 conn(0x7f11d006bdf0 msgr2=0x7f11d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.220+0000 7f11d6200640 1 -- 172.21.15.59:0/4134185966 shutdown_connections 2024-04-23T14:38:19.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.220+0000 7f11d6200640 1 -- 172.21.15.59:0/4134185966 wait complete. 2024-04-23T14:38:19.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.222+0000 7f11d6200640 1 Processor -- start 2024-04-23T14:38:19.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.222+0000 7f11d6200640 1 -- start start 2024-04-23T14:38:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d6200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d006dcb0 0x7f11d01e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d6200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11d00d0600 0x7f11d0159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d6200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11d00d09e0 0x7f11d015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d6200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11d00d0e90 con 0x7f11d006dcb0 2024-04-23T14:38:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d6200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11d015eaf0 con 0x7f11d00d0600 2024-04-23T14:38:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d6200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f11d006ecd0 con 0x7f11d00d09e0 2024-04-23T14:38:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d006dcb0 0x7f11d01e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d006dcb0 0x7f11d01e8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45378/0 (socket says 172.21.15.59:45378) 2024-04-23T14:38:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.223+0000 7f11d4e00640 1 -- 172.21.15.59:0/1700468294 learned_addr learned my addr 172.21.15.59:0/1700468294 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.224+0000 7f11d4e00640 1 -- 172.21.15.59:0/1700468294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11d00d09e0 msgr2=0x7f11d015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.224+0000 7f11d4e00640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11d00d09e0 0x7f11d015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.224+0000 7f11cfe00640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11d00d0600 0x7f11d0159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.224+0000 7f11d4e00640 1 -- 172.21.15.59:0/1700468294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11d00d0600 msgr2=0x7f11d0159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.224+0000 7f11d4e00640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11d00d0600 0x7f11d0159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.224+0000 7f11d4e00640 1 -- 172.21.15.59:0/1700468294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d00cb880 con 0x7f11d006dcb0 2024-04-23T14:38:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.224+0000 7f11d4e00640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d006dcb0 0x7f11d01e8250 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.225+0000 7f11ccc00640 1 -- 172.21.15.59:0/1700468294 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4037040 con 0x7f11d006dcb0 2024-04-23T14:38:19.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.225+0000 7f11ccc00640 1 -- 172.21.15.59:0/1700468294 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c4042070 con 0x7f11d006dcb0 2024-04-23T14:38:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.225+0000 7f11ccc00640 1 -- 172.21.15.59:0/1700468294 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4030bd0 con 0x7f11d006dcb0 2024-04-23T14:38:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.225+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d00cbb60 con 0x7f11d006dcb0 2024-04-23T14:38:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.225+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d00cc120 con 0x7f11d006dcb0 2024-04-23T14:38:19.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.227+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d004fd90 con 0x7f11d006dcb0 2024-04-23T14:38:19.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.227+0000 7f11ccc00640 1 -- 172.21.15.59:0/1700468294 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11c4047460 con 0x7f11d006dcb0 2024-04-23T14:38:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.228+0000 7f11ccc00640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f11b006c940 0x7f11b006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.228+0000 7f11ccc00640 1 -- 172.21.15.59:0/1700468294 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f11c40ac790 con 0x7f11d006dcb0 2024-04-23T14:38:19.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.229+0000 7f11cfe00640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f11b006c940 0x7f11b006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.229+0000 7f11cfe00640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f11b006c940 0x7f11b006ee00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f11d0161a30 tx=0x7f11bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.233+0000 7f11ccc00640 1 -- 172.21.15.59:0/1700468294 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11c4045360 con 0x7f11d006dcb0 2024-04-23T14:38:19.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.436+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f11d015ad20 con 0x7f11d006dcb0 2024-04-23T14:38:19.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.436+0000 7f11ccc00640 1 -- 172.21.15.59:0/1700468294 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f11c407aba0 con 0x7f11d006dcb0 2024-04-23T14:38:19.438 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f11b006c940 msgr2=0x7f11b006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f11b006c940 0x7f11b006ee00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f11d0161a30 tx=0x7f11bc009040 comp rx=0 tx=0).stop 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d006dcb0 msgr2=0x7f11d01e8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d006dcb0 0x7f11d01e8250 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c40046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 shutdown_connections 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f11b006c940 0x7f11b006ee00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11d00d09e0 0x7f11d015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11d00d0600 0x7f11d0159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 --2- 172.21.15.59:0/1700468294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d006dcb0 0x7f11d01e8250 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.439+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 >> 172.21.15.59:0/1700468294 conn(0x7f11d006bdf0 msgr2=0x7f11d00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.440+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 shutdown_connections 2024-04-23T14:38:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.440+0000 7f11d6200640 1 -- 172.21.15.59:0/1700468294 wait complete. 2024-04-23T14:38:19.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:38:19.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.671+0000 7fac70e00640 1 Processor -- start 2024-04-23T14:38:19.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.672+0000 7fac70e00640 1 -- start start 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.673+0000 7fac70e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac6c0d0d10 0x7fac6c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.673+0000 7fac70e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fac6c15d810 0x7fac6c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.673+0000 7fac70e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac6c15cab0 con 0x7fac6c1567e0 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.673+0000 7fac70e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c160e40 con 0x7fac6c15d810 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.673+0000 7fac70e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c15cd30 con 0x7fac6c0d0d10 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.674+0000 7fac6be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fac6c15d810 0x7fac6c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.674+0000 7fac6be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fac6c15d810 0x7fac6c160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45380/0 (socket says 172.21.15.59:45380) 2024-04-23T14:38:19.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.674+0000 7fac6be00640 1 -- 172.21.15.59:0/2537605045 learned_addr learned my addr 172.21.15.59:0/2537605045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:19.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.674+0000 7fac6aa00640 1 --2- 172.21.15.59:0/2537605045 >> v2:172.21.15.76:3300/0 conn(0x7fac6c0d0d10 0x7fac6c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.675+0000 7fac6be00640 1 -- 172.21.15.59:0/2537605045 >> v2:172.21.15.76:3300/0 conn(0x7fac6c0d0d10 msgr2=0x7fac6c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:19.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.675+0000 7fac6be00640 1 --2- 172.21.15.59:0/2537605045 >> v2:172.21.15.76:3300/0 conn(0x7fac6c0d0d10 0x7fac6c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.675+0000 7fac6be00640 1 -- 172.21.15.59:0/2537605045 >> v1:172.21.15.76:6789/0 conn(0x7fac6c1567e0 legacy=0x7fac6c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:19.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.675+0000 7fac6be00640 1 -- 172.21.15.59:0/2537605045 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c15d010 con 0x7fac6c15d810 2024-04-23T14:38:19.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.676+0000 7fac6be00640 1 --2- 172.21.15.59:0/2537605045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c15d810 0x7fac6c160110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fac6c074140 tx=0x7fac6402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ab441f6960e621 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.676+0000 7fac6a000640 1 -- 172.21.15.59:0/2537605045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64036070 con 0x7fac6c15d810 2024-04-23T14:38:19.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.676+0000 7fac6a000640 1 -- 172.21.15.59:0/2537605045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac64031600 con 0x7fac6c15d810 2024-04-23T14:38:19.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.676+0000 7fac6a000640 1 -- 172.21.15.59:0/2537605045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac640319d0 con 0x7fac6c15d810 2024-04-23T14:38:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 -- 172.21.15.59:0/2537605045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c15d810 msgr2=0x7fac6c160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 --2- 172.21.15.59:0/2537605045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c15d810 0x7fac6c160110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fac6c074140 tx=0x7fac6402a6f0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 -- 172.21.15.59:0/2537605045 shutdown_connections 2024-04-23T14:38:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 --2- 172.21.15.59:0/2537605045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c15d810 0x7fac6c160110 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 --2- 172.21.15.59:0/2537605045 >> v2:172.21.15.76:3300/0 conn(0x7fac6c0d0d10 0x7fac6c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 -- 172.21.15.59:0/2537605045 >> 172.21.15.59:0/2537605045 conn(0x7fac6c06bdf0 msgr2=0x7fac6c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:19.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 -- 172.21.15.59:0/2537605045 shutdown_connections 2024-04-23T14:38:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.677+0000 7fac70e00640 1 -- 172.21.15.59:0/2537605045 wait complete. 2024-04-23T14:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.679+0000 7fac70e00640 1 Processor -- start 2024-04-23T14:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.679+0000 7fac70e00640 1 -- start start 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac70e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac6c0d0d10 0x7fac6c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac70e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c1567e0 0x7fac6c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac70e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac6c15d810 0x7fac6c151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac70e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1607d0 con 0x7fac6c1567e0 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac70e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c160650 con 0x7fac6c0d0d10 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac70e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c160950 con 0x7fac6c15d810 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac6aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c1567e0 0x7fac6c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac6aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c1567e0 0x7fac6c1511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45386/0 (socket says 172.21.15.59:45386) 2024-04-23T14:38:19.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac6aa00640 1 -- 172.21.15.59:0/863930287 learned_addr learned my addr 172.21.15.59:0/863930287 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac6b400640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac6c0d0d10 0x7fac6c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.680+0000 7fac6be00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac6c15d810 0x7fac6c151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac6aa00640 1 -- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac6c15d810 msgr2=0x7fac6c151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac6aa00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac6c15d810 0x7fac6c151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac6aa00640 1 -- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac6c0d0d10 msgr2=0x7fac6c14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac6aa00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac6c0d0d10 0x7fac6c14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac6aa00640 1 -- 172.21.15.59:0/863930287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c151d40 con 0x7fac6c1567e0 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac6b400640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac6c0d0d10 0x7fac6c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:19.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac6aa00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c1567e0 0x7fac6c1511f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fac5800d600 tx=0x7fac5800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac63400640 1 -- 172.21.15.59:0/863930287 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac580125f0 con 0x7fac6c1567e0 2024-04-23T14:38:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac63400640 1 -- 172.21.15.59:0/863930287 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac58019070 con 0x7fac6c1567e0 2024-04-23T14:38:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac63400640 1 -- 172.21.15.59:0/863930287 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac580129b0 con 0x7fac6c1567e0 2024-04-23T14:38:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c152020 con 0x7fac6c1567e0 2024-04-23T14:38:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.681+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c1525e0 con 0x7fac6c1567e0 2024-04-23T14:38:19.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.683+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c04fd90 con 0x7fac6c1567e0 2024-04-23T14:38:19.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.683+0000 7fac63400640 1 -- 172.21.15.59:0/863930287 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac58012b50 con 0x7fac6c1567e0 2024-04-23T14:38:19.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.684+0000 7fac63400640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac4c06c8f0 0x7fac4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.684+0000 7fac63400640 1 -- 172.21.15.59:0/863930287 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fac5808c080 con 0x7fac6c1567e0 2024-04-23T14:38:19.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.685+0000 7fac6b400640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac4c06c8f0 0x7fac4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.685+0000 7fac6b400640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac4c06c8f0 0x7fac4c06edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fac5c003eb0 tx=0x7fac5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.689+0000 7fac63400640 1 -- 172.21.15.59:0/863930287 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac5805a390 con 0x7fac6c1567e0 2024-04-23T14:38:19.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.890+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fac6c0b5150 con 0x7fac6c1567e0 2024-04-23T14:38:19.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.891+0000 7fac63400640 1 -- 172.21.15.59:0/863930287 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7fac58059c90 con 0x7fac6c1567e0 2024-04-23T14:38:19.892 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac4c06c8f0 msgr2=0x7fac4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac4c06c8f0 0x7fac4c06edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fac5c003eb0 tx=0x7fac5c009040 comp rx=0 tx=0).stop 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c1567e0 msgr2=0x7fac6c1511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c1567e0 0x7fac6c1511f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fac5800d600 tx=0x7fac5800dac0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 shutdown_connections 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fac4c06c8f0 0x7fac4c06edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac6c15d810 0x7fac6c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c1567e0 0x7fac6c1511f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 --2- 172.21.15.59:0/863930287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac6c0d0d10 0x7fac6c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 >> 172.21.15.59:0/863930287 conn(0x7fac6c06bdf0 msgr2=0x7fac6c155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 shutdown_connections 2024-04-23T14:38:19.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:19.894+0000 7fac70e00640 1 -- 172.21.15.59:0/863930287 wait complete. 2024-04-23T14:38:19.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:38:20.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.130+0000 7f9f6c800640 1 Processor -- start 2024-04-23T14:38:20.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.130+0000 7f9f6c800640 1 -- start start 2024-04-23T14:38:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.132+0000 7f9f6c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f6415eb60 0x7f9f640d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.132+0000 7f9f6c800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f640b99a0 0x7f9f640b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.132+0000 7f9f6c800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f640ba1e0 con 0x7f9f640d2d60 2024-04-23T14:38:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.132+0000 7f9f6c800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9f640ba7a0 con 0x7f9f640b99a0 2024-04-23T14:38:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.132+0000 7f9f6c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9f640ba4c0 con 0x7f9f6415eb60 2024-04-23T14:38:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.133+0000 7f9f6a800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f640b99a0 0x7f9f640b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.133+0000 7f9f69e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f6415eb60 0x7f9f640d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.133+0000 7f9f69e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f6415eb60 0x7f9f640d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:48978/0 (socket says 172.21.15.59:48978) 2024-04-23T14:38:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.133+0000 7f9f69e00640 1 -- 172.21.15.59:0/871680676 learned_addr learned my addr 172.21.15.59:0/871680676 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.133+0000 7f9f6a800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f640b99a0 0x7f9f640b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37506/0 (socket says 172.21.15.59:37506) 2024-04-23T14:38:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.134+0000 7f9f69e00640 1 -- 172.21.15.59:0/871680676 >> v2:172.21.15.76:3300/0 conn(0x7f9f640b99a0 msgr2=0x7f9f640b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.134+0000 7f9f69e00640 1 --2- 172.21.15.59:0/871680676 >> v2:172.21.15.76:3300/0 conn(0x7f9f640b99a0 0x7f9f640b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.134+0000 7f9f69e00640 1 -- 172.21.15.59:0/871680676 >> v1:172.21.15.76:6789/0 conn(0x7f9f640d2d60 legacy=0x7f9f640b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.134+0000 7f9f69e00640 1 -- 172.21.15.59:0/871680676 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640b8a90 con 0x7f9f6415eb60 2024-04-23T14:38:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.135+0000 7f9f69e00640 1 --2- 172.21.15.59:0/871680676 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f6415eb60 0x7f9f640d2820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009870 tx=0x7f9f5802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cd8664a72eeb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.135+0000 7f9f68a00640 1 -- 172.21.15.59:0/871680676 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58037070 con 0x7f9f6415eb60 2024-04-23T14:38:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f68a00640 1 -- 172.21.15.59:0/871680676 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f58032440 con 0x7f9f6415eb60 2024-04-23T14:38:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f68a00640 1 -- 172.21.15.59:0/871680676 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58032810 con 0x7f9f6415eb60 2024-04-23T14:38:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 -- 172.21.15.59:0/871680676 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f6415eb60 msgr2=0x7f9f640d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 --2- 172.21.15.59:0/871680676 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f6415eb60 0x7f9f640d2820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009870 tx=0x7f9f5802a270 comp rx=0 tx=0).stop 2024-04-23T14:38:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 -- 172.21.15.59:0/871680676 shutdown_connections 2024-04-23T14:38:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 --2- 172.21.15.59:0/871680676 >> v2:172.21.15.76:3300/0 conn(0x7f9f640b99a0 0x7f9f640b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 --2- 172.21.15.59:0/871680676 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f6415eb60 0x7f9f640d2820 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 -- 172.21.15.59:0/871680676 >> 172.21.15.59:0/871680676 conn(0x7f9f64067b50 msgr2=0x7f9f64067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:20.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 -- 172.21.15.59:0/871680676 shutdown_connections 2024-04-23T14:38:20.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.136+0000 7f9f6c800640 1 -- 172.21.15.59:0/871680676 wait complete. 2024-04-23T14:38:20.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.138+0000 7f9f6c800640 1 Processor -- start 2024-04-23T14:38:20.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.138+0000 7f9f6c800640 1 -- start start 2024-04-23T14:38:20.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.139+0000 7f9f6c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f640b99a0 0x7f9f6414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.139+0000 7f9f6c800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f6415eb60 0x7f9f640ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.139+0000 7f9f6c800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f640b4b90 0x7f9f64150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.139+0000 7f9f6c800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640b3490 con 0x7f9f6415eb60 2024-04-23T14:38:20.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.139+0000 7f9f6c800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640b3c80 con 0x7f9f640b4b90 2024-04-23T14:38:20.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.139+0000 7f9f6c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9f640b3610 con 0x7f9f640b99a0 2024-04-23T14:38:20.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f6415eb60 0x7f9f640ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f6415eb60 0x7f9f640ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58690/0 (socket says 172.21.15.59:58690) 2024-04-23T14:38:20.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 -- 172.21.15.59:0/2758569554 learned_addr learned my addr 172.21.15.59:0/2758569554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69e00640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f640b99a0 0x7f9f6414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f6a800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f640b4b90 0x7f9f64150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 -- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f640b99a0 msgr2=0x7f9f6414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f640b99a0 0x7f9f6414bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 -- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f640b4b90 msgr2=0x7f9f64150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f640b4b90 0x7f9f64150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f69400640 1 -- 172.21.15.59:0/2758569554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640b6b80 con 0x7f9f6415eb60 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.140+0000 7f9f6a800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f640b4b90 0x7f9f64150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.141+0000 7f9f69400640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f6415eb60 0x7f9f640ce040 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c400 tx=0x7f9f5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.141+0000 7f9f5e000640 1 -- 172.21.15.59:0/2758569554 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f540052d0 con 0x7f9f6415eb60 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.141+0000 7f9f5e000640 1 -- 172.21.15.59:0/2758569554 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f54005490 con 0x7f9f6415eb60 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.141+0000 7f9f5e000640 1 -- 172.21.15.59:0/2758569554 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54007260 con 0x7f9f6415eb60 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.141+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f640b6e60 con 0x7f9f6415eb60 2024-04-23T14:38:20.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.141+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f640ce580 con 0x7f9f6415eb60 2024-04-23T14:38:20.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.142+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f64068a80 con 0x7f9f6415eb60 2024-04-23T14:38:20.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.143+0000 7f9f5e000640 1 -- 172.21.15.59:0/2758569554 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f54007400 con 0x7f9f6415eb60 2024-04-23T14:38:20.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.144+0000 7f9f5e000640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9f4806c8f0 0x7f9f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.144+0000 7f9f5e000640 1 -- 172.21.15.59:0/2758569554 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f9f5408b0d0 con 0x7f9f6415eb60 2024-04-23T14:38:20.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.145+0000 7f9f69e00640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9f4806c8f0 0x7f9f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.146+0000 7f9f69e00640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9f4806c8f0 0x7f9f4806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9f58034000 tx=0x7f9f5802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:20.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.149+0000 7f9f5e000640 1 -- 172.21.15.59:0/2758569554 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f54058e10 con 0x7f9f6415eb60 2024-04-23T14:38:20.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:38:20.311 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:38:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.354+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f640c8090 con 0x7f9f6415eb60 2024-04-23T14:38:20.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.354+0000 7f9f5e000640 1 -- 172.21.15.59:0/2758569554 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7f9f5405d110 con 0x7f9f6415eb60 2024-04-23T14:38:20.356 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:38:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9f4806c8f0 msgr2=0x7f9f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9f4806c8f0 0x7f9f4806edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9f58034000 tx=0x7f9f5802e000 comp rx=0 tx=0).stop 2024-04-23T14:38:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f6415eb60 msgr2=0x7f9f640ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f6415eb60 0x7f9f640ce040 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c400 tx=0x7f9f5400c8c0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 shutdown_connections 2024-04-23T14:38:20.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9f4806c8f0 0x7f9f4806edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f640b4b90 0x7f9f64150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f6415eb60 0x7f9f640ce040 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 --2- 172.21.15.59:0/2758569554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f640b99a0 0x7f9f6414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 >> 172.21.15.59:0/2758569554 conn(0x7f9f64067b50 msgr2=0x7f9f640d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 shutdown_connections 2024-04-23T14:38:20.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.357+0000 7f9f6c800640 1 -- 172.21.15.59:0/2758569554 wait complete. 2024-04-23T14:38:20.378 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:38:20.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.578+0000 7f1750e00640 1 Processor -- start 2024-04-23T14:38:20.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.578+0000 7f1750e00640 1 -- start start 2024-04-23T14:38:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.579+0000 7f1750e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f174c15c670 0x7f174c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.579+0000 7f1750e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 0x7f174c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.579+0000 7f1750e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174c0b4cf0 con 0x7f174c0d05c0 2024-04-23T14:38:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.579+0000 7f1750e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f174c15ca50 con 0x7f174c15c670 2024-04-23T14:38:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.579+0000 7f1750e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f174c0ccb80 con 0x7f174c0b4230 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.579+0000 7f174be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f174c0d05c0 0x7f174c160160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33548/0 (socket says 172.21.15.59:33548) 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174be00640 1 -- 172.21.15.59:0/4158793366 learned_addr learned my addr 172.21.15.59:0/4158793366 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174aa00640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 0x7f174c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174b400640 1 --2- 172.21.15.59:0/4158793366 >> v2:172.21.15.76:3300/0 conn(0x7f174c15c670 0x7f174c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724543454 0 0) 0x7f174c0b4cf0 con 0x7f174c0d05c0 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1730003520 con 0x7f174c0d05c0 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3826368815 0 0) 0x7f1730003520 con 0x7f174c0d05c0 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 msgr2=0x7f174c0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 0x7f174c0b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 >> v2:172.21.15.76:3300/0 conn(0x7f174c15c670 msgr2=0x7f174c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 --2- 172.21.15.59:0/4158793366 >> v2:172.21.15.76:3300/0 conn(0x7f174c15c670 0x7f174c0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c0ccdd0 con 0x7f174c0d05c0 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1744001670 con 0x7f174c0d05c0 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.580+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 >> v1:172.21.15.59:6789/0 conn(0x7f174c0d05c0 legacy=0x7f174c160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174a000640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 0x7f17300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174a000640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1730003f80 0x7f1730004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174a000640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f17300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f174c0ccb80 con 0x7f1730004d40 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f173000b9c0 con 0x7f1730003f80 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f173000bb40 con 0x7f174c0b4230 2024-04-23T14:38:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174be00640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f17300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174b400640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 0x7f17300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174be00640 1 -- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 msgr2=0x7f17300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174be00640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b4230 0x7f17300039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174be00640 1 -- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1730003f80 msgr2=0x7f1730004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174be00640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1730003f80 0x7f1730004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.581+0000 7f174be00640 1 -- 172.21.15.59:0/4158793366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1730003520 con 0x7f1730004d40 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.582+0000 7f174be00640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f17300051c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f174400c190 tx=0x7f1730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf5e97ef5923d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.583+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744007c60 con 0x7f1730004d40 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.583+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1744005af0 con 0x7f1730004d40 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.583+0000 7f174a000640 1 -- 172.21.15.59:0/4158793366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744007070 con 0x7f1730004d40 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.583+0000 7f1750e00640 1 -- 172.21.15.59:0/4158793366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 msgr2=0x7f17300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.583+0000 7f1750e00640 1 --2- 172.21.15.59:0/4158793366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f17300051c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f174400c190 tx=0x7f1730005830 comp rx=0 tx=0).stop 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.583+0000 7f174be00640 1 -- 172.21.15.59:0/4158793366 reap_dead start 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.583+0000 7f1750e00640 1 -- 172.21.15.59:0/4158793366 shutdown_connections 2024-04-23T14:38:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.584+0000 7f1750e00640 1 -- 172.21.15.59:0/4158793366 >> 172.21.15.59:0/4158793366 conn(0x7f174c06bdf0 msgr2=0x7f174c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.584+0000 7f1750e00640 1 -- 172.21.15.59:0/4158793366 shutdown_connections 2024-04-23T14:38:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.584+0000 7f1750e00640 1 -- 172.21.15.59:0/4158793366 wait complete. 2024-04-23T14:38:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.585+0000 7f1750e00640 1 Processor -- start 2024-04-23T14:38:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.586+0000 7f1750e00640 1 -- start start 2024-04-23T14:38:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.586+0000 7f1750e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f174c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.586+0000 7f1750e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b2070 0x7f174c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.586+0000 7f1750e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f174c1600f0 0x7f174c1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.586+0000 7f1750e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f174c0b3e60 con 0x7f1730004d40 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f1750e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f174c0cd390 con 0x7f174c1600f0 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f1750e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f174c0b3fe0 con 0x7f174c0b2070 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f174c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f174c0cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58718/0 (socket says 172.21.15.59:58718) 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 -- 172.21.15.59:0/508041929 learned_addr learned my addr 172.21.15.59:0/508041929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174aa00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b2070 0x7f174c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174be00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f174c1600f0 0x7f174c1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 -- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b2070 msgr2=0x7f174c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b2070 0x7f174c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 -- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f174c1600f0 msgr2=0x7f174c1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f174c1600f0 0x7f174c1605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.587+0000 7f174b400640 1 -- 172.21.15.59:0/508041929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c0b03e0 con 0x7f1730004d40 2024-04-23T14:38:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.588+0000 7f174b400640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f174c0cb960 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f173c0108e0 tx=0x7f173c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.588+0000 7f1743400640 1 -- 172.21.15.59:0/508041929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c00f040 con 0x7f1730004d40 2024-04-23T14:38:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.588+0000 7f1743400640 1 -- 172.21.15.59:0/508041929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173c00eb60 con 0x7f1730004d40 2024-04-23T14:38:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.588+0000 7f1743400640 1 -- 172.21.15.59:0/508041929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c01c400 con 0x7f1730004d40 2024-04-23T14:38:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.588+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c0b06c0 con 0x7f1730004d40 2024-04-23T14:38:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.588+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c0b0c80 con 0x7f1730004d40 2024-04-23T14:38:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.589+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c04fd90 con 0x7f1730004d40 2024-04-23T14:38:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.590+0000 7f1743400640 1 -- 172.21.15.59:0/508041929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f173c01c5a0 con 0x7f1730004d40 2024-04-23T14:38:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.591+0000 7f1743400640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f172806c920 0x7f172806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.592+0000 7f1743400640 1 -- 172.21.15.59:0/508041929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f173c08c420 con 0x7f1730004d40 2024-04-23T14:38:20.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.592+0000 7f174aa00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f172806c920 0x7f172806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:20.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.592+0000 7f174aa00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f172806c920 0x7f172806ede0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1734004b30 tx=0x7f173400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.596+0000 7f1743400640 1 -- 172.21.15.59:0/508041929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f173c018030 con 0x7f1730004d40 2024-04-23T14:38:20.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.815+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f174c0ceab0 con 0x7f1730004d40 2024-04-23T14:38:20.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.816+0000 7f1743400640 1 -- 172.21.15.59:0/508041929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v72) ==== 80+0+122 (secure 0 0 0) 0x7f173c05a0d0 con 0x7f1730004d40 2024-04-23T14:38:20.817 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6836/2883460093 2024-04-24T14:36:50.829845+0000 2024-04-23T14:38:20.818 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6834/2883460093 2024-04-24T14:36:50.829845+0000 2024-04-23T14:38:20.818 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f172806c920 msgr2=0x7f172806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f172806c920 0x7f172806ede0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1734004b30 tx=0x7f173400d040 comp rx=0 tx=0).stop 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 msgr2=0x7f174c0cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f174c0cb960 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f173c0108e0 tx=0x7f173c010da0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 shutdown_connections 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f172806c920 0x7f172806ede0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f174c0b2070 0x7f174c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f174c1600f0 0x7f174c1605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 --2- 172.21.15.59:0/508041929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1730004d40 0x7f174c0cb960 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.819+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 >> 172.21.15.59:0/508041929 conn(0x7f174c06bdf0 msgr2=0x7f174c15b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.820+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 shutdown_connections 2024-04-23T14:38:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:20.820+0000 7f1750e00640 1 -- 172.21.15.59:0/508041929 wait complete. 2024-04-23T14:38:20.841 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:38:20.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:20.841 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:20.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[60879]: fuse finished with error 0 and tester_r 0 2024-04-23T14:38:20.931 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:38:20.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:20.931 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:38:21.009 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:38:21.009 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:38:21.009 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:21.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:38:21.067 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:38:21.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:21.068 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:38:21.118 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:38:21.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:38:21.119 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:38:21.119 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:38:21.119 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:21.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:38:21.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:38:21.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:21.180 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:38:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:38:21.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:38:21.240 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:38:21.240 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:38:21.240 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:21.297 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:38:21.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:38:21.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:21.298 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:38:21.348 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:38:21.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:38:21.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:21.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.568+0000 7fa90d400640 1 Processor -- start 2024-04-23T14:38:21.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.568+0000 7fa90d400640 1 -- start start 2024-04-23T14:38:21.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.570+0000 7fa90d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9080cd050 0x7fa90806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:21.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.570+0000 7fa90d400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa90815ece0 0x7fa9080cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:21.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.570+0000 7fa90d400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9080cd970 con 0x7fa9080b3080 2024-04-23T14:38:21.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.570+0000 7fa90d400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa9080ced20 con 0x7fa9080cd050 2024-04-23T14:38:21.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.570+0000 7fa90d400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa90815fcd0 con 0x7fa90815ece0 2024-04-23T14:38:21.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa907e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9080cd050 0x7fa90806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa90ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa9080b3080 0x7fa9080cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33552/0 (socket says 172.21.15.59:33552) 2024-04-23T14:38:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa90ca00640 1 -- 172.21.15.59:0/1443797562 learned_addr learned my addr 172.21.15.59:0/1443797562 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa8ffe00640 1 --2- 172.21.15.59:0/1443797562 >> v2:172.21.15.76:3300/0 conn(0x7fa90815ece0 0x7fa9080cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa907e00640 1 -- 172.21.15.59:0/1443797562 >> v2:172.21.15.76:3300/0 conn(0x7fa90815ece0 msgr2=0x7fa9080cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa907e00640 1 --2- 172.21.15.59:0/1443797562 >> v2:172.21.15.76:3300/0 conn(0x7fa90815ece0 0x7fa9080cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa907e00640 1 -- 172.21.15.59:0/1443797562 >> v1:172.21.15.59:6789/0 conn(0x7fa9080b3080 legacy=0x7fa9080cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:21.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.571+0000 7fa907e00640 1 -- 172.21.15.59:0/1443797562 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90815f160 con 0x7fa9080cd050 2024-04-23T14:38:21.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.572+0000 7fa907e00640 1 --2- 172.21.15.59:0/1443797562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9080cd050 0x7fa90806f620 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa90806e210 tx=0x7fa8f802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e4eaf3a2691c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:21.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa907400640 1 -- 172.21.15.59:0/1443797562 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8037070 con 0x7fa9080cd050 2024-04-23T14:38:21.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa907400640 1 -- 172.21.15.59:0/1443797562 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f8032440 con 0x7fa9080cd050 2024-04-23T14:38:21.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa907400640 1 -- 172.21.15.59:0/1443797562 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f802ae60 con 0x7fa9080cd050 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa90d400640 1 -- 172.21.15.59:0/1443797562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9080cd050 msgr2=0x7fa90806f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa90d400640 1 --2- 172.21.15.59:0/1443797562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9080cd050 0x7fa90806f620 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa90806e210 tx=0x7fa8f802a110 comp rx=0 tx=0).stop 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa90d400640 1 -- 172.21.15.59:0/1443797562 shutdown_connections 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa90d400640 1 --2- 172.21.15.59:0/1443797562 >> v2:172.21.15.76:3300/0 conn(0x7fa90815ece0 0x7fa9080cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa90d400640 1 --2- 172.21.15.59:0/1443797562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9080cd050 0x7fa90806f620 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.573+0000 7fa90d400640 1 -- 172.21.15.59:0/1443797562 >> 172.21.15.59:0/1443797562 conn(0x7fa9080b21a0 msgr2=0x7fa9080b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.574+0000 7fa90d400640 1 -- 172.21.15.59:0/1443797562 shutdown_connections 2024-04-23T14:38:21.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.574+0000 7fa90d400640 1 -- 172.21.15.59:0/1443797562 wait complete. 2024-04-23T14:38:21.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.575+0000 7fa90d400640 1 Processor -- start 2024-04-23T14:38:21.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.575+0000 7fa90d400640 1 -- start start 2024-04-23T14:38:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.576+0000 7fa90d400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9080b3080 0x7fa9080c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.576+0000 7fa90d400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9080cd050 0x7fa908156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.576+0000 7fa90d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa90815ece0 0x7fa9080cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.576+0000 7fa90d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa90815f9f0 con 0x7fa90815ece0 2024-04-23T14:38:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.576+0000 7fa90d400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa90815f710 con 0x7fa9080cd050 2024-04-23T14:38:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.576+0000 7fa90d400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9080cef50 con 0x7fa9080b3080 2024-04-23T14:38:21.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa90815ece0 0x7fa9080cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa90815ece0 0x7fa9080cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58734/0 (socket says 172.21.15.59:58734) 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 -- 172.21.15.59:0/3715525125 learned_addr learned my addr 172.21.15.59:0/3715525125 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa8ffe00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9080cd050 0x7fa908156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa907e00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9080b3080 0x7fa9080c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 -- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9080b3080 msgr2=0x7fa9080c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9080b3080 0x7fa9080c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 -- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9080cd050 msgr2=0x7fa908156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9080cd050 0x7fa908156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa90ca00640 1 -- 172.21.15.59:0/3715525125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9081f5eb0 con 0x7fa90815ece0 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.577+0000 7fa907e00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9080b3080 0x7fa9080c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.578+0000 7fa90ca00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa90815ece0 0x7fa9080cb8b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa90000cad0 tx=0x7fa90000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:21.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.578+0000 7fa904c00640 1 -- 172.21.15.59:0/3715525125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900017070 con 0x7fa90815ece0 2024-04-23T14:38:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.578+0000 7fa904c00640 1 -- 172.21.15.59:0/3715525125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90000ac30 con 0x7fa90815ece0 2024-04-23T14:38:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.578+0000 7fa904c00640 1 -- 172.21.15.59:0/3715525125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900012620 con 0x7fa90815ece0 2024-04-23T14:38:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.578+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9081f6190 con 0x7fa90815ece0 2024-04-23T14:38:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.578+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9081f6750 con 0x7fa90815ece0 2024-04-23T14:38:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.579+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90804fd90 con 0x7fa90815ece0 2024-04-23T14:38:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.580+0000 7fa904c00640 1 -- 172.21.15.59:0/3715525125 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa90001a070 con 0x7fa90815ece0 2024-04-23T14:38:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.581+0000 7fa904c00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa8e406c8f0 0x7fa8e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.581+0000 7fa904c00640 1 -- 172.21.15.59:0/3715525125 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fa90008b8e0 con 0x7fa90815ece0 2024-04-23T14:38:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.582+0000 7fa907e00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa8e406c8f0 0x7fa8e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.582+0000 7fa907e00640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa8e406c8f0 0x7fa8e406edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8034000 tx=0x7fa8f802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.587+0000 7fa904c00640 1 -- 172.21.15.59:0/3715525125 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa90000ae20 con 0x7fa90815ece0 2024-04-23T14:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.835+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9081f6bf0 con 0x7fa90815ece0 2024-04-23T14:38:21.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.836+0000 7fa904c00640 1 -- 172.21.15.59:0/3715525125 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5185 (secure 0 0 0) 0x7fa900059c70 con 0x7fa90815ece0 2024-04-23T14:38:21.837 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:21.838 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:37:49.879970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":18,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":5}]} 2024-04-23T14:38:21.838 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 51 2024-04-23T14:38:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa8e406c8f0 msgr2=0x7fa8e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa8e406c8f0 0x7fa8e406edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8034000 tx=0x7fa8f802e000 comp rx=0 tx=0).stop 2024-04-23T14:38:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa90815ece0 msgr2=0x7fa9080cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa90815ece0 0x7fa9080cb8b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa90000cad0 tx=0x7fa90000cf90 comp rx=0 tx=0).stop 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 shutdown_connections 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa8e406c8f0 0x7fa8e406edb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa90815ece0 0x7fa9080cb8b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9080cd050 0x7fa908156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 --2- 172.21.15.59:0/3715525125 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9080b3080 0x7fa9080c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 >> 172.21.15.59:0/3715525125 conn(0x7fa9080b21a0 msgr2=0x7fa90806f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 shutdown_connections 2024-04-23T14:38:21.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:21.839+0000 7fa90d400640 1 -- 172.21.15.59:0/3715525125 wait complete. 2024-04-23T14:38:21.859 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:22.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.068+0000 7f585fa00640 1 Processor -- start 2024-04-23T14:38:22.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.069+0000 7f585fa00640 1 -- start start 2024-04-23T14:38:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f585fa00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58581454e0 0x7f5858145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f585fa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f58580cd020 0x7f58580b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f585fa00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58580cd520 con 0x7f58580b3e40 2024-04-23T14:38:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f585fa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f58580cddc0 con 0x7f58580cd020 2024-04-23T14:38:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f585fa00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f58580cd800 con 0x7f58581454e0 2024-04-23T14:38:22.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f585da00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f58580b3e40 0x7f5858160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33568/0 (socket says 172.21.15.59:33568) 2024-04-23T14:38:22.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f585da00640 1 -- 172.21.15.59:0/155621006 learned_addr learned my addr 172.21.15.59:0/155621006 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:22.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.070+0000 7f5857e00640 1 --2- 172.21.15.59:0/155621006 >> v2:172.21.15.59:3300/0 conn(0x7f58580cd020 0x7f58580b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:22.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.071+0000 7f5857400640 1 -- 172.21.15.59:0/155621006 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751533706 0 0) 0x7f58580cd520 con 0x7f58580b3e40 2024-04-23T14:38:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.071+0000 7f5857400640 1 -- 172.21.15.59:0/155621006 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f583c003520 con 0x7f58580b3e40 2024-04-23T14:38:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.071+0000 7f585d000640 1 --2- 172.21.15.59:0/155621006 >> v2:172.21.15.76:3300/0 conn(0x7f58581454e0 0x7f5858145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.071+0000 7f5857e00640 1 -- 172.21.15.59:0/155621006 >> v2:172.21.15.76:3300/0 conn(0x7f58581454e0 msgr2=0x7f5858145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.071+0000 7f5857e00640 1 --2- 172.21.15.59:0/155621006 >> v2:172.21.15.76:3300/0 conn(0x7f58581454e0 0x7f5858145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.071+0000 7f5857e00640 1 -- 172.21.15.59:0/155621006 >> v1:172.21.15.59:6789/0 conn(0x7f58580b3e40 legacy=0x7f5858160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.071+0000 7f5857e00640 1 -- 172.21.15.59:0/155621006 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58580cdae0 con 0x7f58580cd020 2024-04-23T14:38:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.072+0000 7f5857e00640 1 --2- 172.21.15.59:0/155621006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f58580b3900 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5858073280 tx=0x7f584402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc00c51c3d195dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.072+0000 7f5857400640 1 -- 172.21.15.59:0/155621006 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844037070 con 0x7f58580cd020 2024-04-23T14:38:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.072+0000 7f5857400640 1 -- 172.21.15.59:0/155621006 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5844032440 con 0x7f58580cd020 2024-04-23T14:38:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f5857400640 1 -- 172.21.15.59:0/155621006 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584402ae60 con 0x7f58580cd020 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 -- 172.21.15.59:0/155621006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 msgr2=0x7f58580b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 --2- 172.21.15.59:0/155621006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f58580b3900 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5858073280 tx=0x7f584402a110 comp rx=0 tx=0).stop 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 -- 172.21.15.59:0/155621006 shutdown_connections 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 --2- 172.21.15.59:0/155621006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f58580b3900 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 --2- 172.21.15.59:0/155621006 >> v2:172.21.15.76:3300/0 conn(0x7f58581454e0 0x7f5858145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 -- 172.21.15.59:0/155621006 >> 172.21.15.59:0/155621006 conn(0x7f5858067ea0 msgr2=0x7f585806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 -- 172.21.15.59:0/155621006 shutdown_connections 2024-04-23T14:38:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.073+0000 7f585fa00640 1 -- 172.21.15.59:0/155621006 wait complete. 2024-04-23T14:38:22.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.075+0000 7f585fa00640 1 Processor -- start 2024-04-23T14:38:22.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.075+0000 7f585fa00640 1 -- start start 2024-04-23T14:38:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f585fa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58580b3e40 0x7f585815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f585fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f585806e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f585fa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58581454e0 0x7f585806e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f585fa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5858146270 con 0x7f58580cd020 2024-04-23T14:38:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f585fa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58581460f0 con 0x7f58580b3e40 2024-04-23T14:38:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f585fa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5858161020 con 0x7f58581454e0 2024-04-23T14:38:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f5857e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f585806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f5857e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f585806e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58754/0 (socket says 172.21.15.59:58754) 2024-04-23T14:38:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.076+0000 7f5857e00640 1 -- 172.21.15.59:0/2972950898 learned_addr learned my addr 172.21.15.59:0/2972950898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.077+0000 7f5857e00640 1 -- 172.21.15.59:0/2972950898 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58581454e0 msgr2=0x7f585806e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.077+0000 7f5857e00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58581454e0 0x7f585806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.077+0000 7f5857e00640 1 -- 172.21.15.59:0/2972950898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58580b3e40 msgr2=0x7f585815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.077+0000 7f5857e00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58580b3e40 0x7f585815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.077+0000 7f5857e00640 1 -- 172.21.15.59:0/2972950898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585813e050 con 0x7f58580cd020 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.077+0000 7f5857e00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f585806e050 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5844000c00 tx=0x7f58440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.078+0000 7f5854c00640 1 -- 172.21.15.59:0/2972950898 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844037040 con 0x7f58580cd020 2024-04-23T14:38:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.078+0000 7f5854c00640 1 -- 172.21.15.59:0/2972950898 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5844042070 con 0x7f58580cd020 2024-04-23T14:38:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.078+0000 7f5854c00640 1 -- 172.21.15.59:0/2972950898 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844030960 con 0x7f58580cd020 2024-04-23T14:38:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.078+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585813e330 con 0x7f58580cd020 2024-04-23T14:38:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.078+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585813e8f0 con 0x7f58580cd020 2024-04-23T14:38:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.079+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5858068a80 con 0x7f58580cd020 2024-04-23T14:38:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.080+0000 7f5854c00640 1 -- 172.21.15.59:0/2972950898 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5844030b00 con 0x7f58580cd020 2024-04-23T14:38:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.081+0000 7f5854c00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f583806c8f0 0x7f583806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.081+0000 7f5854c00640 1 -- 172.21.15.59:0/2972950898 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f58440ac5b0 con 0x7f58580cd020 2024-04-23T14:38:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.081+0000 7f585d000640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f583806c8f0 0x7f583806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:22.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.082+0000 7f585d000640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f583806c8f0 0x7f583806edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5848006fd0 tx=0x7f5848008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.085+0000 7f5854c00640 1 -- 172.21.15.59:0/2972950898 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5844045360 con 0x7f58580cd020 2024-04-23T14:38:22.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.300+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5858160030 con 0x7f58580cd020 2024-04-23T14:38:22.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.302+0000 7f5854c00640 1 -- 172.21.15.59:0/2972950898 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16834 (secure 0 0 0) 0x7f584407a8b0 con 0x7f58580cd020 2024-04-23T14:38:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:22.304 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:37:08.944193+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/2883460093":"2024-04-24T14:36:50.829845+0000","172.21.15.59:6834/2883460093":"2024-04-24T14:36:50.829845+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:22.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.304+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f583806c8f0 msgr2=0x7f583806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.304+0000 7f585fa00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f583806c8f0 0x7f583806edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5848006fd0 tx=0x7f5848008040 comp rx=0 tx=0).stop 2024-04-23T14:38:22.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.304+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 msgr2=0x7f585806e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.304+0000 7f585fa00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f585806e050 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5844000c00 tx=0x7f58440046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 shutdown_connections 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f583806c8f0 0x7f583806edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f58581454e0 0x7f585806e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58580cd020 0x7f585806e050 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 --2- 172.21.15.59:0/2972950898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58580b3e40 0x7f585815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 >> 172.21.15.59:0/2972950898 conn(0x7f5858067ea0 msgr2=0x7f58580d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 shutdown_connections 2024-04-23T14:38:22.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.305+0000 7f585fa00640 1 -- 172.21.15.59:0/2972950898 wait complete. 2024-04-23T14:38:22.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:36:27.526201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:36:28.678324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:38:22.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:38:22.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.539+0000 7f39bcc00640 1 Processor -- start 2024-04-23T14:38:22.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.539+0000 7f39bcc00640 1 -- start start 2024-04-23T14:38:22.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39bcc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39b806dcb0 0x7f39b80ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39bcc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b8071e10 con 0x7f39b80d0600 2024-04-23T14:38:22.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39bcc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b80720f0 con 0x7f39b80d09e0 2024-04-23T14:38:22.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39bcc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f39b8072a20 con 0x7f39b806dcb0 2024-04-23T14:38:22.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39b6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39b806dcb0 0x7f39b80ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:22.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39b7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f39b80d0600 0x7f39b8162d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33572/0 (socket says 172.21.15.59:33572) 2024-04-23T14:38:22.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39b7400640 1 -- 172.21.15.59:0/2515180123 learned_addr learned my addr 172.21.15.59:0/2515180123 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:22.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.541+0000 7f39b6a00640 1 -- 172.21.15.59:0/2515180123 >> v1:172.21.15.76:6789/0 conn(0x7f39b80d09e0 legacy=0x7f39b806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:22.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.542+0000 7f39b6a00640 1 -- 172.21.15.59:0/2515180123 >> v1:172.21.15.59:6789/0 conn(0x7f39b80d0600 legacy=0x7f39b8162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.542+0000 7f39b6a00640 1 -- 172.21.15.59:0/2515180123 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b80723d0 con 0x7f39b806dcb0 2024-04-23T14:38:22.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.543+0000 7f39b6a00640 1 --2- 172.21.15.59:0/2515180123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b806dcb0 0x7f39b80ad170 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f39a40097a0 tx=0x7f39a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6313a971b63af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:22.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.543+0000 7f39b6000640 1 -- 172.21.15.59:0/2515180123 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4037070 con 0x7f39b806dcb0 2024-04-23T14:38:22.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.543+0000 7f39b6000640 1 -- 172.21.15.59:0/2515180123 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a4032440 con 0x7f39b806dcb0 2024-04-23T14:38:22.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.543+0000 7f39b6000640 1 -- 172.21.15.59:0/2515180123 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4032810 con 0x7f39b806dcb0 2024-04-23T14:38:22.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.544+0000 7f39bcc00640 1 -- 172.21.15.59:0/2515180123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b806dcb0 msgr2=0x7f39b80ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.544+0000 7f39bcc00640 1 --2- 172.21.15.59:0/2515180123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b806dcb0 0x7f39b80ad170 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f39a40097a0 tx=0x7f39a402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.544+0000 7f39bcc00640 1 -- 172.21.15.59:0/2515180123 shutdown_connections 2024-04-23T14:38:22.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.544+0000 7f39bcc00640 1 --2- 172.21.15.59:0/2515180123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b806dcb0 0x7f39b80ad170 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.544+0000 7f39bcc00640 1 -- 172.21.15.59:0/2515180123 >> 172.21.15.59:0/2515180123 conn(0x7f39b806bdf0 msgr2=0x7f39b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:22.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.544+0000 7f39bcc00640 1 -- 172.21.15.59:0/2515180123 shutdown_connections 2024-04-23T14:38:22.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.544+0000 7f39bcc00640 1 -- 172.21.15.59:0/2515180123 wait complete. 2024-04-23T14:38:22.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.546+0000 7f39bcc00640 1 Processor -- start 2024-04-23T14:38:22.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.546+0000 7f39bcc00640 1 -- start start 2024-04-23T14:38:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39bcc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39b806dcb0 0x7f39b80cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39bcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b80d0600 0x7f39b81494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39bcc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39b80d09e0 0x7f39b815d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39bcc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80d0e90 con 0x7f39b80d0600 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39bcc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f39b815eaf0 con 0x7f39b806dcb0 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39bcc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f39b806ecd0 con 0x7f39b80d09e0 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b80d0600 0x7f39b81494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b80d0600 0x7f39b81494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58778/0 (socket says 172.21.15.59:58778) 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 -- 172.21.15.59:0/1469579702 learned_addr learned my addr 172.21.15.59:0/1469579702 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 -- 172.21.15.59:0/1469579702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39b80d09e0 msgr2=0x7f39b815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39b80d09e0 0x7f39b815d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 -- 172.21.15.59:0/1469579702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39b806dcb0 msgr2=0x7f39b80cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39b806dcb0 0x7f39b80cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.547+0000 7f39b6a00640 1 -- 172.21.15.59:0/1469579702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b80ca3d0 con 0x7f39b80d0600 2024-04-23T14:38:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.548+0000 7f39b6a00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b80d0600 0x7f39b81494e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f39a40098c0 tx=0x7f39a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.548+0000 7f39af400640 1 -- 172.21.15.59:0/1469579702 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4037040 con 0x7f39b80d0600 2024-04-23T14:38:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.548+0000 7f39af400640 1 -- 172.21.15.59:0/1469579702 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a4042070 con 0x7f39b80d0600 2024-04-23T14:38:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.548+0000 7f39af400640 1 -- 172.21.15.59:0/1469579702 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4030a70 con 0x7f39b80d0600 2024-04-23T14:38:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.548+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b80cb390 con 0x7f39b80d0600 2024-04-23T14:38:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.548+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b8149a20 con 0x7f39b80d0600 2024-04-23T14:38:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.549+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b804fd90 con 0x7f39b80d0600 2024-04-23T14:38:22.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.551+0000 7f39af400640 1 -- 172.21.15.59:0/1469579702 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39a4030c10 con 0x7f39b80d0600 2024-04-23T14:38:22.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.551+0000 7f39af400640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f399806c8f0 0x7f399806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:22.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.552+0000 7f39af400640 1 -- 172.21.15.59:0/1469579702 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f39a40ac6d0 con 0x7f39b80d0600 2024-04-23T14:38:22.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.552+0000 7f39b7400640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f399806c8f0 0x7f399806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:22.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.553+0000 7f39b7400640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f399806c8f0 0x7f399806edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39a8000f60 tx=0x7f39a8007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:22.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.557+0000 7f39af400640 1 -- 172.21.15.59:0/1469579702 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39a4045360 con 0x7f39b80d0600 2024-04-23T14:38:22.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.764+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39b80cff50 con 0x7f39b80d0600 2024-04-23T14:38:22.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.765+0000 7f39af400640 1 -- 172.21.15.59:0/1469579702 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f39a407aa60 con 0x7f39b80d0600 2024-04-23T14:38:22.767 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f399806c8f0 msgr2=0x7f399806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f399806c8f0 0x7f399806edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39a8000f60 tx=0x7f39a8007a80 comp rx=0 tx=0).stop 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b80d0600 msgr2=0x7f39b81494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b80d0600 0x7f39b81494e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f39a40098c0 tx=0x7f39a40046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 shutdown_connections 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f399806c8f0 0x7f399806edb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39b80d09e0 0x7f39b815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b80d0600 0x7f39b81494e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 --2- 172.21.15.59:0/1469579702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39b806dcb0 0x7f39b80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:22.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.768+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 >> 172.21.15.59:0/1469579702 conn(0x7f39b806bdf0 msgr2=0x7f39b80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:22.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.769+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 shutdown_connections 2024-04-23T14:38:22.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.769+0000 7f39bcc00640 1 -- 172.21.15.59:0/1469579702 wait complete. 2024-04-23T14:38:22.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:38:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.997+0000 7fc7f9600640 1 Processor -- start 2024-04-23T14:38:22.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.997+0000 7fc7f9600640 1 -- start start 2024-04-23T14:38:23.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.999+0000 7fc7f9600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc7f4072480 0x7fc7f416d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.999+0000 7fc7f9600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f40b35d0 con 0x7fc7f4162c10 2024-04-23T14:38:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.999+0000 7fc7f9600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f40b38b0 con 0x7fc7f40b4670 2024-04-23T14:38:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.999+0000 7fc7f9600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f40b3b90 con 0x7fc7f4072480 2024-04-23T14:38:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.999+0000 7fc7f2a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc7f4162c10 0x7fc7f40d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33574/0 (socket says 172.21.15.59:33574) 2024-04-23T14:38:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:22.999+0000 7fc7f2a00640 1 -- 172.21.15.59:0/996501996 learned_addr learned my addr 172.21.15.59:0/996501996 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f3400640 1 --2- 172.21.15.59:0/996501996 >> v2:172.21.15.76:3300/0 conn(0x7fc7f4072480 0x7fc7f416d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375874399 0 0) 0x7fc7f40b35d0 con 0x7fc7f4162c10 2024-04-23T14:38:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7d8003520 con 0x7fc7f4162c10 2024-04-23T14:38:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1105132529 0 0) 0x7fc7d8003520 con 0x7fc7f4162c10 2024-04-23T14:38:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 >> v2:172.21.15.76:3300/0 conn(0x7fc7f4072480 msgr2=0x7fc7f416d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 --2- 172.21.15.59:0/996501996 >> v2:172.21.15.76:3300/0 conn(0x7fc7f4072480 0x7fc7f416d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 >> v1:172.21.15.76:6789/0 conn(0x7fc7f40b4670 legacy=0x7fc7f4071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f40b3e70 con 0x7fc7f4162c10 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.000+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fc7dc001670 con 0x7fc7f4162c10 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 >> v1:172.21.15.59:6789/0 conn(0x7fc7f4162c10 legacy=0x7fc7f40d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2000640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2000640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2000640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7d80051c0 0x7fc7d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f40b3b90 con 0x7fc7d8003a00 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d800c0c0 con 0x7fc7d8004800 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d800c2c0 con 0x7fc7d80051c0 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f2a00640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7f3400640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7d80051c0 0x7fc7d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.001+0000 7fc7ea000640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7f2a00640 1 -- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7d80051c0 msgr2=0x7fc7d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7f2a00640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7d80051c0 0x7fc7d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7f2a00640 1 -- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8004800 msgr2=0x7fc7d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7f2a00640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7f2a00640 1 -- 172.21.15.59:0/996501996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d8003520 con 0x7fc7d8003a00 2024-04-23T14:38:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7f3400640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7d80051c0 0x7fc7d800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:38:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7ea000640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7d8004800 0x7fc7d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:38:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.002+0000 7fc7f3400640 1 -- 172.21.15.59:0/996501996 reap_dead start 2024-04-23T14:38:23.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.003+0000 7fc7f2a00640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7d8003e20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00c0e0 tx=0x7fc7d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24337314cac369db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc007c20 con 0x7fc7d8003a00 2024-04-23T14:38:23.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc005ad0 con 0x7fc7d8003a00 2024-04-23T14:38:23.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f2000640 1 -- 172.21.15.59:0/996501996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc007070 con 0x7fc7d8003a00 2024-04-23T14:38:23.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f9600640 1 -- 172.21.15.59:0/996501996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 msgr2=0x7fc7d8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f9600640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7d8003e20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00c0e0 tx=0x7fc7d8005830 comp rx=0 tx=0).stop 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f9600640 1 -- 172.21.15.59:0/996501996 shutdown_connections 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f9600640 1 --2- 172.21.15.59:0/996501996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7d8003e20 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f9600640 1 -- 172.21.15.59:0/996501996 >> 172.21.15.59:0/996501996 conn(0x7fc7f4067d90 msgr2=0x7fc7f406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f9600640 1 -- 172.21.15.59:0/996501996 shutdown_connections 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.004+0000 7fc7f9600640 1 -- 172.21.15.59:0/996501996 wait complete. 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.006+0000 7fc7f9600640 1 Processor -- start 2024-04-23T14:38:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.006+0000 7fc7f9600640 1 -- start start 2024-04-23T14:38:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.007+0000 7fc7f9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7f4159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.007+0000 7fc7f9600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7f41650b0 0x7fc7f4165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.007+0000 7fc7f9600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7f415a0c0 0x7fc7f415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.007+0000 7fc7f9600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4163170 con 0x7fc7d8003a00 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.007+0000 7fc7f9600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4162ff0 con 0x7fc7f415a0c0 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.007+0000 7fc7f9600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f40cfba0 con 0x7fc7f41650b0 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7f4159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7f4159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58800/0 (socket says 172.21.15.59:58800) 2024-04-23T14:38:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 -- 172.21.15.59:0/4150451374 learned_addr learned my addr 172.21.15.59:0/4150451374 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:23.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7ea000640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7f41650b0 0x7fc7f4165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f3400640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7f415a0c0 0x7fc7f415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 -- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7f41650b0 msgr2=0x7fc7f4165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7f41650b0 0x7fc7f4165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 -- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7f415a0c0 msgr2=0x7fc7f415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7f415a0c0 0x7fc7f415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7f2a00640 1 -- 172.21.15.59:0/4150451374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f415acd0 con 0x7fc7d8003a00 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.008+0000 7fc7ea000640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7f41650b0 0x7fc7f4165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.009+0000 7fc7f2a00640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7f4159980 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc02acb0 tx=0x7fc7f40b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.009+0000 7fc7eb400640 1 -- 172.21.15.59:0/4150451374 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc03f020 con 0x7fc7d8003a00 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.009+0000 7fc7eb400640 1 -- 172.21.15.59:0/4150451374 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc005c80 con 0x7fc7d8003a00 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.009+0000 7fc7eb400640 1 -- 172.21.15.59:0/4150451374 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc001670 con 0x7fc7d8003a00 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.009+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f415afb0 con 0x7fc7d8003a00 2024-04-23T14:38:23.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.009+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f415b570 con 0x7fc7d8003a00 2024-04-23T14:38:23.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.010+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f4068a80 con 0x7fc7d8003a00 2024-04-23T14:38:23.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.011+0000 7fc7eb400640 1 -- 172.21.15.59:0/4150451374 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7dc042020 con 0x7fc7d8003a00 2024-04-23T14:38:23.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.012+0000 7fc7eb400640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7d006c920 0x7fc7d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.012+0000 7fc7eb400640 1 -- 172.21.15.59:0/4150451374 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fc7dc0ab6e0 con 0x7fc7d8003a00 2024-04-23T14:38:23.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.012+0000 7fc7ea000640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7d006c920 0x7fc7d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.013+0000 7fc7ea000640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7d006c920 0x7fc7d006ede0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7f41eb590 tx=0x7fc7e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.017+0000 7fc7eb400640 1 -- 172.21.15.59:0/4150451374 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7dc079a70 con 0x7fc7d8003a00 2024-04-23T14:38:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.221+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7f4068a80 con 0x7fc7d8003a00 2024-04-23T14:38:23.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.222+0000 7fc7eb400640 1 -- 172.21.15.59:0/4150451374 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7fc7dc079370 con 0x7fc7d8003a00 2024-04-23T14:38:23.223 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:38:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7d006c920 msgr2=0x7fc7d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7d006c920 0x7fc7d006ede0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7f41eb590 tx=0x7fc7e4009040 comp rx=0 tx=0).stop 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 msgr2=0x7fc7f4159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7f4159980 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc02acb0 tx=0x7fc7f40b4040 comp rx=0 tx=0).stop 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 shutdown_connections 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7d006c920 0x7fc7d006ede0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7f41650b0 0x7fc7f4165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7f415a0c0 0x7fc7f415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 --2- 172.21.15.59:0/4150451374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d8003a00 0x7fc7f4159980 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 >> 172.21.15.59:0/4150451374 conn(0x7fc7f4067d90 msgr2=0x7fc7f40d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 shutdown_connections 2024-04-23T14:38:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.225+0000 7fc7f9600640 1 -- 172.21.15.59:0/4150451374 wait complete. 2024-04-23T14:38:23.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:38:23.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.447+0000 7f0ef1400640 1 Processor -- start 2024-04-23T14:38:23.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.447+0000 7f0ef1400640 1 -- start start 2024-04-23T14:38:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.448+0000 7f0ef1400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 0x7f0eec156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.448+0000 7f0ef1400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eec15cab0 con 0x7f0eec0d0d10 2024-04-23T14:38:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.449+0000 7f0ef1400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eec160e40 con 0x7f0eec15d810 2024-04-23T14:38:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.449+0000 7f0ef1400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0eec15cd30 con 0x7f0eec1567e0 2024-04-23T14:38:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.449+0000 7f0ef0a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0eec0d0d10 0x7f0eec160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33584/0 (socket says 172.21.15.59:33584) 2024-04-23T14:38:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.449+0000 7f0ef0a00640 1 -- 172.21.15.59:0/4229203154 learned_addr learned my addr 172.21.15.59:0/4229203154 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.449+0000 7f0eebe00640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 0x7f0eec156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.449+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671081348 0 0) 0x7f0ee4001490 con 0x7f0eec0d0d10 2024-04-23T14:38:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.449+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ed0003520 con 0x7f0eec0d0d10 2024-04-23T14:38:23.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 767429690 0 0) 0x7f0ed0003520 con 0x7f0eec0d0d10 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 msgr2=0x7f0eec156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 0x7f0eec156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 >> v1:172.21.15.76:6789/0 conn(0x7f0eec15d810 legacy=0x7f0eec0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec15d010 con 0x7f0eec0d0d10 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f0ee4001bc0 con 0x7f0eec0d0d10 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 >> v1:172.21.15.59:6789/0 conn(0x7f0eec0d0d10 legacy=0x7f0eec160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed0004800 0x7f0ed0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 0x7f0ed00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec15cd30 con 0x7f0ed00039e0 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed000baa0 con 0x7f0ed0004800 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed000bca0 con 0x7f0eec1567e0 2024-04-23T14:38:23.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.450+0000 7f0eebe00640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.451+0000 7f0ee3e00640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed0004800 0x7f0ed0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.451+0000 7f0ef0a00640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 0x7f0ed00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.451+0000 7f0eebe00640 1 -- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 msgr2=0x7f0ed00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.451+0000 7f0eebe00640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec1567e0 0x7f0ed00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.451+0000 7f0eebe00640 1 -- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed0004800 msgr2=0x7f0ed0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.451+0000 7f0eebe00640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ed0004800 0x7f0ed0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.451+0000 7f0eebe00640 1 -- 172.21.15.59:0/4229203154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed0003520 con 0x7f0ed00039e0 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.452+0000 7f0eebe00640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003e00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009a90 tx=0x7f0ed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553cf8b7d6e76b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.452+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc036070 con 0x7f0ed00039e0 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.452+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0edc031400 con 0x7f0ed00039e0 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.452+0000 7f0eeb400640 1 -- 172.21.15.59:0/4229203154 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc009690 con 0x7f0ed00039e0 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.453+0000 7f0ef1400640 1 -- 172.21.15.59:0/4229203154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 msgr2=0x7f0ed0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.453+0000 7f0ef1400640 1 --2- 172.21.15.59:0/4229203154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003e00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009a90 tx=0x7f0ed0005830 comp rx=0 tx=0).stop 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.453+0000 7f0ef0a00640 1 -- 172.21.15.59:0/4229203154 reap_dead start 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.453+0000 7f0ef1400640 1 -- 172.21.15.59:0/4229203154 shutdown_connections 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.453+0000 7f0ef1400640 1 -- 172.21.15.59:0/4229203154 >> 172.21.15.59:0/4229203154 conn(0x7f0eec06bdf0 msgr2=0x7f0eec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.453+0000 7f0ef1400640 1 -- 172.21.15.59:0/4229203154 shutdown_connections 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.453+0000 7f0ef1400640 1 -- 172.21.15.59:0/4229203154 wait complete. 2024-04-23T14:38:23.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.455+0000 7f0ef1400640 1 Processor -- start 2024-04-23T14:38:23.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.455+0000 7f0ef1400640 1 -- start start 2024-04-23T14:38:23.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0ef1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0eec159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0ef1400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec0ce910 0x7f0eec0b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0ef1400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eec0b3420 0x7f0eec158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0ef1400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec156d40 con 0x7f0ed00039e0 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0ef1400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec156bc0 con 0x7f0eec0b3420 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0ef1400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0eec156ec0 con 0x7f0eec0ce910 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0eebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0eec159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0eebe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0eec159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58808/0 (socket says 172.21.15.59:58808) 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.456+0000 7f0eebe00640 1 -- 172.21.15.59:0/1158792414 learned_addr learned my addr 172.21.15.59:0/1158792414 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0ee3e00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec0ce910 0x7f0eec0b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0eebe00640 1 -- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec0ce910 msgr2=0x7f0eec0b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0ef0a00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eec0b3420 0x7f0eec158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0eebe00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec0ce910 0x7f0eec0b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0eebe00640 1 -- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eec0b3420 msgr2=0x7f0eec158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0eebe00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eec0b3420 0x7f0eec158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0eebe00640 1 -- 172.21.15.59:0/1158792414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec159910 con 0x7f0ed00039e0 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0eebe00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0eec159d30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009bb0 tx=0x7f0edc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0ee8c00640 1 -- 172.21.15.59:0/1158792414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc040020 con 0x7f0ed00039e0 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0ee8c00640 1 -- 172.21.15.59:0/1158792414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0edc003bc0 con 0x7f0ed00039e0 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0ee8c00640 1 -- 172.21.15.59:0/1158792414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc036040 con 0x7f0ed00039e0 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.457+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eec15a430 con 0x7f0ed00039e0 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.458+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eec0b1ba0 con 0x7f0ed00039e0 2024-04-23T14:38:23.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.459+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eec04fd90 con 0x7f0ed00039e0 2024-04-23T14:38:23.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.460+0000 7f0ee8c00640 1 -- 172.21.15.59:0/1158792414 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0edc043020 con 0x7f0ed00039e0 2024-04-23T14:38:23.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.461+0000 7f0ee8c00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ec806c920 0x7f0ec806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.461+0000 7f0ee8c00640 1 -- 172.21.15.59:0/1158792414 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f0edc0abfd0 con 0x7f0ed00039e0 2024-04-23T14:38:23.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.461+0000 7f0ee3e00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ec806c920 0x7f0ec806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.462+0000 7f0ee3e00640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ec806c920 0x7f0ec806ede0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0cf030 tx=0x7f0ed4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.465+0000 7f0ee8c00640 1 -- 172.21.15.59:0/1158792414 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0edc04a030 con 0x7f0ed00039e0 2024-04-23T14:38:23.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.666+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0eec0b3800 con 0x7f0ed00039e0 2024-04-23T14:38:23.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.667+0000 7f0ee8c00640 1 -- 172.21.15.59:0/1158792414 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7f0edc0044e0 con 0x7f0ed00039e0 2024-04-23T14:38:23.668 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.669+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ec806c920 msgr2=0x7f0ec806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.669+0000 7f0ef1400640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ec806c920 0x7f0ec806ede0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0cf030 tx=0x7f0ed4009040 comp rx=0 tx=0).stop 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.669+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 msgr2=0x7f0eec159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.669+0000 7f0ef1400640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0eec159d30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009bb0 tx=0x7f0edc004730 comp rx=0 tx=0).stop 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 shutdown_connections 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0ec806c920 0x7f0ec806ede0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eec0ce910 0x7f0eec0b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eec0b3420 0x7f0eec158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 --2- 172.21.15.59:0/1158792414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ed00039e0 0x7f0eec159d30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 >> 172.21.15.59:0/1158792414 conn(0x7f0eec06bdf0 msgr2=0x7f0eec15b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:23.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 shutdown_connections 2024-04-23T14:38:23.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.670+0000 7f0ef1400640 1 -- 172.21.15.59:0/1158792414 wait complete. 2024-04-23T14:38:23.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.900+0000 7f67c8e00640 1 Processor -- start 2024-04-23T14:38:23.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.901+0000 7f67c8e00640 1 -- start start 2024-04-23T14:38:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.902+0000 7f67c8e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 0x7f67c40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.902+0000 7f67c8e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f67c40d0d90 0x7f67c40d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.902+0000 7f67c8e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c40cf030 con 0x7f67c415e7c0 2024-04-23T14:38:23.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.902+0000 7f67c8e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f67c40b7000 con 0x7f67c40d0d90 2024-04-23T14:38:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.902+0000 7f67c8e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f67c40cf620 con 0x7f67c40b5ff0 2024-04-23T14:38:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.902+0000 7f67c3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f67c415e7c0 0x7f67c40d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33594/0 (socket says 172.21.15.59:33594) 2024-04-23T14:38:23.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.902+0000 7f67c3400640 1 -- 172.21.15.59:0/837615172 learned_addr learned my addr 172.21.15.59:0/837615172 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c3e00640 1 --2- 172.21.15.59:0/837615172 >> v2:172.21.15.76:3300/0 conn(0x7f67c40d0d90 0x7f67c40d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2a00640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 0x7f67c40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491564611 0 0) 0x7f67b4001110 con 0x7f67c415e7c0 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67a8003520 con 0x7f67c415e7c0 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2664999897 0 0) 0x7f67a8003520 con 0x7f67c415e7c0 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 msgr2=0x7f67c40b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 0x7f67c40b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 >> v2:172.21.15.76:3300/0 conn(0x7f67c40d0d90 msgr2=0x7f67c40d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 --2- 172.21.15.59:0/837615172 >> v2:172.21.15.76:3300/0 conn(0x7f67c40d0d90 0x7f67c40d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c40b6a40 con 0x7f67c415e7c0 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.903+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f67b4001840 con 0x7f67c415e7c0 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 >> v1:172.21.15.59:6789/0 conn(0x7f67c415e7c0 legacy=0x7f67c40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2000640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 0x7f67a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2000640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8003f30 0x7f67a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2000640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67a8004d40 0x7f67a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67c40b7000 con 0x7f67a8004d40 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67c40cf620 con 0x7f67a8003f30 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f67a800b9c0 con 0x7f67c40b5ff0 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c3e00640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67a8004d40 0x7f67a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c3400640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 0x7f67a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.904+0000 7f67c2a00640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8003f30 0x7f67a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.905+0000 7f67c3e00640 1 -- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 msgr2=0x7f67a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.905+0000 7f67c3e00640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b5ff0 0x7f67a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.905+0000 7f67c3e00640 1 -- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8003f30 msgr2=0x7f67a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:23.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.905+0000 7f67c3e00640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8003f30 0x7f67a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.905+0000 7f67c3e00640 1 -- 172.21.15.59:0/837615172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a8003520 con 0x7f67a8004d40 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.906+0000 7f67c3e00640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67a8004d40 0x7f67a80051c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009fc0 tx=0x7f67a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccc301fcfa8b5a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.906+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc009c10 con 0x7f67a8004d40 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.906+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67bc036070 con 0x7f67a8004d40 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.906+0000 7f67c2000640 1 -- 172.21.15.59:0/837615172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc02d070 con 0x7f67a8004d40 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.906+0000 7f67c8e00640 1 -- 172.21.15.59:0/837615172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67a8004d40 msgr2=0x7f67a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.906+0000 7f67c8e00640 1 --2- 172.21.15.59:0/837615172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67a8004d40 0x7f67a80051c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009fc0 tx=0x7f67a8005830 comp rx=0 tx=0).stop 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.906+0000 7f67c3e00640 1 -- 172.21.15.59:0/837615172 reap_dead start 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.907+0000 7f67c8e00640 1 -- 172.21.15.59:0/837615172 shutdown_connections 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.907+0000 7f67c8e00640 1 -- 172.21.15.59:0/837615172 >> 172.21.15.59:0/837615172 conn(0x7f67c409d920 msgr2=0x7f67c408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.907+0000 7f67c8e00640 1 -- 172.21.15.59:0/837615172 shutdown_connections 2024-04-23T14:38:23.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.907+0000 7f67c8e00640 1 -- 172.21.15.59:0/837615172 wait complete. 2024-04-23T14:38:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.908+0000 7f67c8e00640 1 Processor -- start 2024-04-23T14:38:23.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.909+0000 7f67c8e00640 1 -- start start 2024-04-23T14:38:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.909+0000 7f67c8e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8004d40 0x7f67c406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.909+0000 7f67c8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c40b8af0 0x7f67c40ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c8e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b31b0 0x7f67c40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c8e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67c40d2800 con 0x7f67c40b8af0 2024-04-23T14:38:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c8e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f67c40cf2b0 con 0x7f67a8004d40 2024-04-23T14:38:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c8e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f67c40d1a80 con 0x7f67c40b31b0 2024-04-23T14:38:23.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c40b8af0 0x7f67c40ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c40b8af0 0x7f67c40ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58822/0 (socket says 172.21.15.59:58822) 2024-04-23T14:38:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c2a00640 1 -- 172.21.15.59:0/2869110451 learned_addr learned my addr 172.21.15.59:0/2869110451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c3e00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b31b0 0x7f67c40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.910+0000 7f67c3400640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8004d40 0x7f67c406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c2a00640 1 -- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b31b0 msgr2=0x7f67c40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c2a00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b31b0 0x7f67c40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c2a00640 1 -- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8004d40 msgr2=0x7f67c406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c2a00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8004d40 0x7f67c406e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c2a00640 1 -- 172.21.15.59:0/2869110451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c406e6e0 con 0x7f67c40b8af0 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c2a00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c40b8af0 0x7f67c40ca2c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f67ac011570 tx=0x7f67ac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67bb400640 1 -- 172.21.15.59:0/2869110451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac015930 con 0x7f67c40b8af0 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67bb400640 1 -- 172.21.15.59:0/2869110451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67ac015ad0 con 0x7f67c40b8af0 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67bb400640 1 -- 172.21.15.59:0/2869110451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac01d400 con 0x7f67c40b8af0 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c40ca800 con 0x7f67c40b8af0 2024-04-23T14:38:23.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.911+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c40b8010 con 0x7f67c40b8af0 2024-04-23T14:38:23.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.913+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c404fd90 con 0x7f67c40b8af0 2024-04-23T14:38:23.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.914+0000 7f67bb400640 1 -- 172.21.15.59:0/2869110451 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67ac01d5a0 con 0x7f67c40b8af0 2024-04-23T14:38:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.915+0000 7f67bb400640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f67a006c920 0x7f67a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.915+0000 7f67bb400640 1 -- 172.21.15.59:0/2869110451 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f67ac08c5b0 con 0x7f67c40b8af0 2024-04-23T14:38:23.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.915+0000 7f67c3400640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f67a006c920 0x7f67a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:23.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.916+0000 7f67c3400640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f67a006c920 0x7f67a006ede0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f67b4001050 tx=0x7f67b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:23.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:23.920+0000 7f67bb400640 1 -- 172.21.15.59:0/2869110451 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67c406e6e0 con 0x7f67c40b8af0 2024-04-23T14:38:24.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.168+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67c406d840 con 0x7f67c40b8af0 2024-04-23T14:38:24.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.169+0000 7f67bb400640 1 -- 172.21.15.59:0/2869110451 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5185 (secure 0 0 0) 0x7f67ac019030 con 0x7f67c40b8af0 2024-04-23T14:38:24.171 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:24.171 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:37:49.879970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":18,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":5}]} 2024-04-23T14:38:24.171 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 51 2024-04-23T14:38:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f67a006c920 msgr2=0x7f67a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f67a006c920 0x7f67a006ede0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f67b4001050 tx=0x7f67b4009040 comp rx=0 tx=0).stop 2024-04-23T14:38:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c40b8af0 msgr2=0x7f67c40ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c40b8af0 0x7f67c40ca2c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f67ac011570 tx=0x7f67ac011a30 comp rx=0 tx=0).stop 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 shutdown_connections 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f67a006c920 0x7f67a006ede0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67c40b31b0 0x7f67c40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c40b8af0 0x7f67c40ca2c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 --2- 172.21.15.59:0/2869110451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8004d40 0x7f67c406e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 >> 172.21.15.59:0/2869110451 conn(0x7f67c409d920 msgr2=0x7f67c406cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 shutdown_connections 2024-04-23T14:38:24.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.172+0000 7f67c8e00640 1 -- 172.21.15.59:0/2869110451 wait complete. 2024-04-23T14:38:24.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:24.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.406+0000 7fd48a800640 1 Processor -- start 2024-04-23T14:38:24.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.406+0000 7fd48a800640 1 -- start start 2024-04-23T14:38:24.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.407+0000 7fd48a800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd48415c3d0 0x7fd4840cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.407+0000 7fd48a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4840b4860 0x7fd4840cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.407+0000 7fd48a800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd484160210 con 0x7fd4840cd590 2024-04-23T14:38:24.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.407+0000 7fd48a800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd4841604f0 con 0x7fd48415c3d0 2024-04-23T14:38:24.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.407+0000 7fd48a800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd4841607d0 con 0x7fd4840b4860 2024-04-23T14:38:24.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.408+0000 7fd489e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4840b4860 0x7fd4840cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.408+0000 7fd489e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4840b4860 0x7fd4840cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49108/0 (socket says 172.21.15.59:49108) 2024-04-23T14:38:24.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.408+0000 7fd489e00640 1 -- 172.21.15.59:0/1964491703 learned_addr learned my addr 172.21.15.59:0/1964491703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:24.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.409+0000 7fd489400640 1 --2- 172.21.15.59:0/1964491703 >> v2:172.21.15.76:3300/0 conn(0x7fd48415c3d0 0x7fd4840cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.409+0000 7fd489e00640 1 -- 172.21.15.59:0/1964491703 >> v2:172.21.15.76:3300/0 conn(0x7fd48415c3d0 msgr2=0x7fd4840cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:24.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.409+0000 7fd489e00640 1 --2- 172.21.15.59:0/1964491703 >> v2:172.21.15.76:3300/0 conn(0x7fd48415c3d0 0x7fd4840cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.409+0000 7fd489e00640 1 -- 172.21.15.59:0/1964491703 >> v1:172.21.15.76:6789/0 conn(0x7fd4840cd590 legacy=0x7fd4840b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.409+0000 7fd489e00640 1 -- 172.21.15.59:0/1964491703 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd484160ab0 con 0x7fd4840b4860 2024-04-23T14:38:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.410+0000 7fd489e00640 1 --2- 172.21.15.59:0/1964491703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4840b4860 0x7fd4840cb7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009e50 tx=0x7fd47c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769d474144eedf59 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd483e00640 1 -- 172.21.15.59:0/1964491703 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c036070 con 0x7fd4840b4860 2024-04-23T14:38:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd483e00640 1 -- 172.21.15.59:0/1964491703 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c031600 con 0x7fd4840b4860 2024-04-23T14:38:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd483e00640 1 -- 172.21.15.59:0/1964491703 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0319d0 con 0x7fd4840b4860 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd48a800640 1 -- 172.21.15.59:0/1964491703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4840b4860 msgr2=0x7fd4840cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd48a800640 1 --2- 172.21.15.59:0/1964491703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4840b4860 0x7fd4840cb7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009e50 tx=0x7fd47c02a710 comp rx=0 tx=0).stop 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd48a800640 1 -- 172.21.15.59:0/1964491703 shutdown_connections 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd48a800640 1 --2- 172.21.15.59:0/1964491703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4840b4860 0x7fd4840cb7d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd48a800640 1 --2- 172.21.15.59:0/1964491703 >> v2:172.21.15.76:3300/0 conn(0x7fd48415c3d0 0x7fd4840cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.411+0000 7fd48a800640 1 -- 172.21.15.59:0/1964491703 >> 172.21.15.59:0/1964491703 conn(0x7fd4840b21a0 msgr2=0x7fd4840b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.412+0000 7fd48a800640 1 -- 172.21.15.59:0/1964491703 shutdown_connections 2024-04-23T14:38:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.412+0000 7fd48a800640 1 -- 172.21.15.59:0/1964491703 wait complete. 2024-04-23T14:38:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.413+0000 7fd48a800640 1 Processor -- start 2024-04-23T14:38:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.413+0000 7fd48a800640 1 -- start start 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.414+0000 7fd48a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4840b4860 0x7fd48415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.414+0000 7fd48a800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4840cd590 0x7fd484152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.414+0000 7fd48a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd48415c3d0 0x7fd484153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.414+0000 7fd48a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd48415c930 con 0x7fd4840b4860 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.414+0000 7fd48a800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd48415c7b0 con 0x7fd4840cd590 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.414+0000 7fd48a800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd4840cbd10 con 0x7fd48415c3d0 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4840b4860 0x7fd48415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4840b4860 0x7fd48415cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58824/0 (socket says 172.21.15.59:58824) 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 -- 172.21.15.59:0/1992797306 learned_addr learned my addr 172.21.15.59:0/1992797306 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489e00640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd48415c3d0 0x7fd484153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd488a00640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4840cd590 0x7fd484152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 -- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd48415c3d0 msgr2=0x7fd484153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd48415c3d0 0x7fd484153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 -- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4840cd590 msgr2=0x7fd484152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4840cd590 0x7fd484152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd489400640 1 -- 172.21.15.59:0/1992797306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4841539b0 con 0x7fd4840b4860 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.415+0000 7fd488a00640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4840cd590 0x7fd484152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.416+0000 7fd489400640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4840b4860 0x7fd48415cab0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e8a0 tx=0x7fd47800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.416+0000 7fd481600640 1 -- 172.21.15.59:0/1992797306 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478017070 con 0x7fd4840b4860 2024-04-23T14:38:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.416+0000 7fd481600640 1 -- 172.21.15.59:0/1992797306 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47800c940 con 0x7fd4840b4860 2024-04-23T14:38:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.416+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48415f5b0 con 0x7fd4840b4860 2024-04-23T14:38:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.416+0000 7fd481600640 1 -- 172.21.15.59:0/1992797306 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47800cd30 con 0x7fd4840b4860 2024-04-23T14:38:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.416+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4840c73b0 con 0x7fd4840b4860 2024-04-23T14:38:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.417+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48404fd90 con 0x7fd4840b4860 2024-04-23T14:38:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.418+0000 7fd481600640 1 -- 172.21.15.59:0/1992797306 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd47801a070 con 0x7fd4840b4860 2024-04-23T14:38:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.419+0000 7fd481600640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd46406c8f0 0x7fd46406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.420+0000 7fd481600640 1 -- 172.21.15.59:0/1992797306 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fd47808ba20 con 0x7fd4840b4860 2024-04-23T14:38:24.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.420+0000 7fd488a00640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd46406c8f0 0x7fd46406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.421+0000 7fd488a00640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd46406c8f0 0x7fd46406edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b4fd0 tx=0x7fd470009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:24.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.424+0000 7fd481600640 1 -- 172.21.15.59:0/1992797306 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4841539b0 con 0x7fd4840b4860 2024-04-23T14:38:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.635+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd48406e400 con 0x7fd4840b4860 2024-04-23T14:38:24.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.638+0000 7fd481600640 1 -- 172.21.15.59:0/1992797306 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16834 (secure 0 0 0) 0x7fd478059e50 con 0x7fd4840b4860 2024-04-23T14:38:24.640 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:24.640 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:37:08.944193+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/2883460093":"2024-04-24T14:36:50.829845+0000","172.21.15.59:6834/2883460093":"2024-04-24T14:36:50.829845+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:24.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd46406c8f0 msgr2=0x7fd46406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd46406c8f0 0x7fd46406edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b4fd0 tx=0x7fd470009040 comp rx=0 tx=0).stop 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4840b4860 msgr2=0x7fd48415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4840b4860 0x7fd48415cab0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e8a0 tx=0x7fd47800ed60 comp rx=0 tx=0).stop 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 shutdown_connections 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd46406c8f0 0x7fd46406edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd48415c3d0 0x7fd484153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4840cd590 0x7fd484152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 --2- 172.21.15.59:0/1992797306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4840b4860 0x7fd48415cab0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 >> 172.21.15.59:0/1992797306 conn(0x7fd4840b21a0 msgr2=0x7fd4840cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.641+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 shutdown_connections 2024-04-23T14:38:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.642+0000 7fd48a800640 1 -- 172.21.15.59:0/1992797306 wait complete. 2024-04-23T14:38:24.663 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:38:24.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.876+0000 7fc1e6800640 1 Processor -- start 2024-04-23T14:38:24.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.876+0000 7fc1e6800640 1 -- start start 2024-04-23T14:38:24.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.877+0000 7fc1e6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1e00cd050 0x7fc1e006f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.877+0000 7fc1e6800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e015ece0 0x7fc1e00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.877+0000 7fc1e6800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e00cd970 con 0x7fc1e00b3080 2024-04-23T14:38:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.877+0000 7fc1e6800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e00ced20 con 0x7fc1e00cd050 2024-04-23T14:38:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.877+0000 7fc1e6800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e015fcd0 con 0x7fc1e015ece0 2024-04-23T14:38:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e5400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1e00cd050 0x7fc1e006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e5e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc1e00b3080 0x7fc1e00cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33600/0 (socket says 172.21.15.59:33600) 2024-04-23T14:38:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e5e00640 1 -- 172.21.15.59:0/895353900 learned_addr learned my addr 172.21.15.59:0/895353900 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e4a00640 1 --2- 172.21.15.59:0/895353900 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e015ece0 0x7fc1e00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e5400640 1 -- 172.21.15.59:0/895353900 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e015ece0 msgr2=0x7fc1e00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e5400640 1 --2- 172.21.15.59:0/895353900 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e015ece0 0x7fc1e00cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e5400640 1 -- 172.21.15.59:0/895353900 >> v1:172.21.15.59:6789/0 conn(0x7fc1e00b3080 legacy=0x7fc1e00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1dfe00640 1 -- 172.21.15.59:0/895353900 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2433812921 0 0) 0x7fc1e00cd970 con 0x7fc1e00b3080 2024-04-23T14:38:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.878+0000 7fc1e5400640 1 -- 172.21.15.59:0/895353900 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e015f160 con 0x7fc1e00cd050 2024-04-23T14:38:24.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.879+0000 7fc1e5400640 1 --2- 172.21.15.59:0/895353900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e006f620 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097a0 tx=0x7fc1d002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7efced6222f4a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:24.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1dfe00640 1 -- 172.21.15.59:0/895353900 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0037070 con 0x7fc1e00cd050 2024-04-23T14:38:24.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1dfe00640 1 -- 172.21.15.59:0/895353900 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d002ae50 con 0x7fc1e00cd050 2024-04-23T14:38:24.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1dfe00640 1 -- 172.21.15.59:0/895353900 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0032660 con 0x7fc1e00cd050 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1e6800640 1 -- 172.21.15.59:0/895353900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 msgr2=0x7fc1e006f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1e6800640 1 --2- 172.21.15.59:0/895353900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e006f620 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097a0 tx=0x7fc1d002a100 comp rx=0 tx=0).stop 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1e6800640 1 -- 172.21.15.59:0/895353900 shutdown_connections 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1e6800640 1 --2- 172.21.15.59:0/895353900 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e015ece0 0x7fc1e00cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1e6800640 1 --2- 172.21.15.59:0/895353900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e006f620 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.880+0000 7fc1e6800640 1 -- 172.21.15.59:0/895353900 >> 172.21.15.59:0/895353900 conn(0x7fc1e00b21a0 msgr2=0x7fc1e00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.881+0000 7fc1e6800640 1 -- 172.21.15.59:0/895353900 shutdown_connections 2024-04-23T14:38:24.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.881+0000 7fc1e6800640 1 -- 172.21.15.59:0/895353900 wait complete. 2024-04-23T14:38:24.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.882+0000 7fc1e6800640 1 Processor -- start 2024-04-23T14:38:24.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.882+0000 7fc1e6800640 1 -- start start 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.883+0000 7fc1e6800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e00b3080 0x7fc1e009d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.883+0000 7fc1e6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e00cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.883+0000 7fc1e6800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1e015ece0 0x7fc1e0072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.883+0000 7fc1e6800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e015f9f0 con 0x7fc1e00cd050 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.883+0000 7fc1e6800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e015f710 con 0x7fc1e015ece0 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.883+0000 7fc1e6800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e00cef50 con 0x7fc1e00b3080 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e00cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e00cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58836/0 (socket says 172.21.15.59:58836) 2024-04-23T14:38:24.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 -- 172.21.15.59:0/2765167941 learned_addr learned my addr 172.21.15.59:0/2765167941 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:24.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 -- 172.21.15.59:0/2765167941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e00b3080 msgr2=0x7fc1e009d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e00b3080 0x7fc1e009d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 -- 172.21.15.59:0/2765167941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1e015ece0 msgr2=0x7fc1e0072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1e015ece0 0x7fc1e0072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 -- 172.21.15.59:0/2765167941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e00cc6c0 con 0x7fc1e00cd050 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.884+0000 7fc1e4a00640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e00cbd70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00cd00 tx=0x7fc1cc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.885+0000 7fc1dd600640 1 -- 172.21.15.59:0/2765167941 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc003990 con 0x7fc1e00cd050 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.885+0000 7fc1dd600640 1 -- 172.21.15.59:0/2765167941 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc003b30 con 0x7fc1e00cd050 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.885+0000 7fc1dd600640 1 -- 172.21.15.59:0/2765167941 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc004020 con 0x7fc1e00cd050 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.885+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e015aeb0 con 0x7fc1e00cd050 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.885+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e015b470 con 0x7fc1e00cd050 2024-04-23T14:38:24.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.886+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e004fd90 con 0x7fc1e00cd050 2024-04-23T14:38:24.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.887+0000 7fc1dd600640 1 -- 172.21.15.59:0/2765167941 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1cc0041c0 con 0x7fc1e00cd050 2024-04-23T14:38:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.888+0000 7fc1dd600640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1c006c8f0 0x7fc1c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.888+0000 7fc1dd600640 1 -- 172.21.15.59:0/2765167941 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fc1cc08c1a0 con 0x7fc1e00cd050 2024-04-23T14:38:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.889+0000 7fc1e5400640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1c006c8f0 0x7fc1c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.889+0000 7fc1e5400640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1c006c8f0 0x7fc1c006edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0034000 tx=0x7fc1d002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:24.893+0000 7fc1dd600640 1 -- 172.21.15.59:0/2765167941 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1cc014030 con 0x7fc1e00cd050 2024-04-23T14:38:25.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.139+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1e015c9a0 con 0x7fc1e00cd050 2024-04-23T14:38:25.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.140+0000 7fc1dd600640 1 -- 172.21.15.59:0/2765167941 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5185 (secure 0 0 0) 0x7fc1cc05a4a0 con 0x7fc1e00cd050 2024-04-23T14:38:25.141 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:25.141 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:37:49.879970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":18,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":5}]} 2024-04-23T14:38:25.142 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 51 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1c006c8f0 msgr2=0x7fc1c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1c006c8f0 0x7fc1c006edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0034000 tx=0x7fc1d002c000 comp rx=0 tx=0).stop 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 msgr2=0x7fc1e00cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e00cbd70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00cd00 tx=0x7fc1cc00a5c0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 shutdown_connections 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc1c006c8f0 0x7fc1c006edb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1e015ece0 0x7fc1e0072b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1e00cd050 0x7fc1e00cbd70 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.142+0000 7fc1e6800640 1 --2- 172.21.15.59:0/2765167941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc1e00b3080 0x7fc1e009d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.143+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 >> 172.21.15.59:0/2765167941 conn(0x7fc1e00b21a0 msgr2=0x7fc1e00ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:25.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.143+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 shutdown_connections 2024-04-23T14:38:25.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.143+0000 7fc1e6800640 1 -- 172.21.15.59:0/2765167941 wait complete. 2024-04-23T14:38:25.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:25.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.380+0000 7f6fd9800640 1 Processor -- start 2024-04-23T14:38:25.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.380+0000 7f6fd9800640 1 -- start start 2024-04-23T14:38:25.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.382+0000 7f6fd9800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fd40cc7e0 0x7f6fd4167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.382+0000 7f6fd9800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd4160210 con 0x7f6fd40cd760 2024-04-23T14:38:25.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.382+0000 7f6fd9800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd41604f0 con 0x7f6fd415c3d0 2024-04-23T14:38:25.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.382+0000 7f6fd9800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd41607d0 con 0x7f6fd40cc7e0 2024-04-23T14:38:25.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.382+0000 7f6fcbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6fd40cd760 0x7f6fd40b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33604/0 (socket says 172.21.15.59:33604) 2024-04-23T14:38:25.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.382+0000 7f6fcbe00640 1 -- 172.21.15.59:0/3978583724 learned_addr learned my addr 172.21.15.59:0/3978583724 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:25.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd8e00640 1 --2- 172.21.15.59:0/3978583724 >> v2:172.21.15.76:3300/0 conn(0x7f6fd40cc7e0 0x7f6fd4167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1998151209 0 0) 0x7f6fd4160210 con 0x7f6fd40cd760 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fb8003520 con 0x7f6fd40cd760 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216138861 0 0) 0x7f6fb8003520 con 0x7f6fd40cd760 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 >> v2:172.21.15.76:3300/0 conn(0x7f6fd40cc7e0 msgr2=0x7f6fd4167400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 --2- 172.21.15.59:0/3978583724 >> v2:172.21.15.76:3300/0 conn(0x7f6fd40cc7e0 0x7f6fd4167400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 >> v1:172.21.15.76:6789/0 conn(0x7f6fd415c3d0 legacy=0x7f6fd40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd4160ab0 con 0x7f6fd40cd760 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f6fbc001670 con 0x7f6fd40cd760 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.383+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 >> v1:172.21.15.59:6789/0 conn(0x7f6fd40cd760 legacy=0x7f6fd40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd3400640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb8003a00 0x7f6fb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd3400640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fb8004800 0x7f6fb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd3400640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb80051c0 0x7f6fb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41607d0 con 0x7f6fb80051c0 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb800c0c0 con 0x7f6fb8003a00 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb800c2c0 con 0x7f6fb8004800 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd8e00640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb80051c0 0x7f6fb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fcbe00640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fb8004800 0x7f6fb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd3e00640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb8003a00 0x7f6fb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd8e00640 1 -- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fb8004800 msgr2=0x7f6fb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd8e00640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fb8004800 0x7f6fb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd8e00640 1 -- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb8003a00 msgr2=0x7f6fb8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd8e00640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb8003a00 0x7f6fb8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.384+0000 7f6fd8e00640 1 -- 172.21.15.59:0/3978583724 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb8003520 con 0x7f6fb80051c0 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.385+0000 7f6fd8e00640 1 -- 172.21.15.59:0/3978583724 reap_dead start 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.386+0000 7f6fd8e00640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb80051c0 0x7f6fb800b9c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009fc0 tx=0x7f6fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f806019a449256b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.386+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc035020 con 0x7f6fb80051c0 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.386+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc003bd0 con 0x7f6fb80051c0 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.386+0000 7f6fd3400640 1 -- 172.21.15.59:0/3978583724 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc009c10 con 0x7f6fb80051c0 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.387+0000 7f6fd9800640 1 -- 172.21.15.59:0/3978583724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb80051c0 msgr2=0x7f6fb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.387+0000 7f6fd9800640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb80051c0 0x7f6fb800b9c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009fc0 tx=0x7f6fb8005830 comp rx=0 tx=0).stop 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.387+0000 7f6fd9800640 1 -- 172.21.15.59:0/3978583724 shutdown_connections 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.387+0000 7f6fd9800640 1 --2- 172.21.15.59:0/3978583724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb80051c0 0x7f6fb800b9c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.387+0000 7f6fd9800640 1 -- 172.21.15.59:0/3978583724 >> 172.21.15.59:0/3978583724 conn(0x7f6fd40b21a0 msgr2=0x7f6fd40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:25.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.387+0000 7f6fd9800640 1 -- 172.21.15.59:0/3978583724 shutdown_connections 2024-04-23T14:38:25.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.387+0000 7f6fd9800640 1 -- 172.21.15.59:0/3978583724 wait complete. 2024-04-23T14:38:25.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.390+0000 7f6fd9800640 1 Processor -- start 2024-04-23T14:38:25.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.390+0000 7f6fd9800640 1 -- start start 2024-04-23T14:38:25.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fd9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb80051c0 0x7f6fd40c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fd9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd415cf60 0x7f6fd4152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fd9800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fd415d8f0 0x7f6fd415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fd9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd415c930 con 0x7f6fd415cf60 2024-04-23T14:38:25.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fd9800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd415c7b0 con 0x7f6fb80051c0 2024-04-23T14:38:25.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fd9800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd40cd050 con 0x7f6fd415d8f0 2024-04-23T14:38:25.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fcbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd415cf60 0x7f6fd4152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fcbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd415cf60 0x7f6fd4152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58850/0 (socket says 172.21.15.59:58850) 2024-04-23T14:38:25.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.391+0000 7f6fcbe00640 1 -- 172.21.15.59:0/1305868864 learned_addr learned my addr 172.21.15.59:0/1305868864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:25.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fd3e00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb80051c0 0x7f6fd40c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fd8e00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fd415d8f0 0x7f6fd415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fcbe00640 1 -- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fd415d8f0 msgr2=0x7f6fd415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fcbe00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fd415d8f0 0x7f6fd415dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fcbe00640 1 -- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb80051c0 msgr2=0x7f6fd40c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fcbe00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb80051c0 0x7f6fd40c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fcbe00640 1 -- 172.21.15.59:0/1305868864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd41534f0 con 0x7f6fd415cf60 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fcbe00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd415cf60 0x7f6fd4152fb0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e830 tx=0x7f6fbc00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.392+0000 7f6fd0c00640 1 -- 172.21.15.59:0/1305868864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00c660 con 0x7f6fd415cf60 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.393+0000 7f6fd0c00640 1 -- 172.21.15.59:0/1305868864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fbc018070 con 0x7f6fd415cf60 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.393+0000 7f6fd0c00640 1 -- 172.21.15.59:0/1305868864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00ca40 con 0x7f6fd415cf60 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.393+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd40ca0d0 con 0x7f6fd415cf60 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.393+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd40c8e40 con 0x7f6fd415cf60 2024-04-23T14:38:25.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.394+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd40ca5f0 con 0x7f6fd415cf60 2024-04-23T14:38:25.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.395+0000 7f6fd0c00640 1 -- 172.21.15.59:0/1305868864 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fbc013420 con 0x7f6fd415cf60 2024-04-23T14:38:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.396+0000 7f6fd0c00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6fb006c920 0x7f6fb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.396+0000 7f6fd0c00640 1 -- 172.21.15.59:0/1305868864 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f6fbc08b910 con 0x7f6fd415cf60 2024-04-23T14:38:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.396+0000 7f6fd3e00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6fb006c920 0x7f6fb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.397+0000 7f6fd3e00640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6fb006c920 0x7f6fb006ede0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005410 tx=0x7f6fc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:25.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.400+0000 7f6fd0c00640 1 -- 172.21.15.59:0/1305868864 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fd40ca5f0 con 0x7f6fd415cf60 2024-04-23T14:38:25.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.641+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fd404fd90 con 0x7f6fd415cf60 2024-04-23T14:38:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.642+0000 7f6fd0c00640 1 -- 172.21.15.59:0/1305868864 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5185 (secure 0 0 0) 0x7f6fbc059b80 con 0x7f6fd415cf60 2024-04-23T14:38:25.644 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:25.644 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:36:29.710826+0000","modified":"2024-04-23T14:37:49.879970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":18,"addr":"172.21.15.59:6836/1773251382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1773251382},{"type":"v1","addr":"172.21.15.59:6836","nonce":1773251382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":5}]} 2024-04-23T14:38:25.644 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 51 2024-04-23T14:38:25.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6fb006c920 msgr2=0x7f6fb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6fb006c920 0x7f6fb006ede0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005410 tx=0x7f6fc4009040 comp rx=0 tx=0).stop 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd415cf60 msgr2=0x7f6fd4152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd415cf60 0x7f6fd4152fb0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e830 tx=0x7f6fbc00ecf0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 shutdown_connections 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6fb006c920 0x7f6fb006ede0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6fd415d8f0 0x7f6fd415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd415cf60 0x7f6fd4152fb0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 --2- 172.21.15.59:0/1305868864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fb80051c0 0x7f6fd40c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 >> 172.21.15.59:0/1305868864 conn(0x7f6fd40b21a0 msgr2=0x7f6fd40cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 shutdown_connections 2024-04-23T14:38:25.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.645+0000 7f6fd9800640 1 -- 172.21.15.59:0/1305868864 wait complete. 2024-04-23T14:38:25.666 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.878+0000 7fdf18a00640 1 Processor -- start 2024-04-23T14:38:25.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.878+0000 7fdf18a00640 1 -- start start 2024-04-23T14:38:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf18a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf1415e7c0 0x7fdf140d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf18a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf140d0d90 0x7fdf140d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf18a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf140cf030 con 0x7fdf140d2630 2024-04-23T14:38:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf18a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdf140b7000 con 0x7fdf1415e7c0 2024-04-23T14:38:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf18a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdf140cf620 con 0x7fdf140d0d90 2024-04-23T14:38:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf12a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdf140d2630 0x7fdf140b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33612/0 (socket says 172.21.15.59:33612) 2024-04-23T14:38:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf13400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf1415e7c0 0x7fdf140d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf12a00640 1 -- 172.21.15.59:0/3444178296 learned_addr learned my addr 172.21.15.59:0/3444178296 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.880+0000 7fdf13e00640 1 --2- 172.21.15.59:0/3444178296 >> v2:172.21.15.76:3300/0 conn(0x7fdf140d0d90 0x7fdf140d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.881+0000 7fdf13400640 1 -- 172.21.15.59:0/3444178296 >> v2:172.21.15.76:3300/0 conn(0x7fdf140d0d90 msgr2=0x7fdf140d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.881+0000 7fdf13400640 1 --2- 172.21.15.59:0/3444178296 >> v2:172.21.15.76:3300/0 conn(0x7fdf140d0d90 0x7fdf140d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.881+0000 7fdf13400640 1 -- 172.21.15.59:0/3444178296 >> v1:172.21.15.59:6789/0 conn(0x7fdf140d2630 legacy=0x7fdf140b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.881+0000 7fdf13400640 1 -- 172.21.15.59:0/3444178296 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf140b6a40 con 0x7fdf1415e7c0 2024-04-23T14:38:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.881+0000 7fdf12000640 1 -- 172.21.15.59:0/3444178296 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46681981 0 0) 0x7fdf00001110 con 0x7fdf140d2630 2024-04-23T14:38:25.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.882+0000 7fdf13400640 1 --2- 172.21.15.59:0/3444178296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140d20f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fdf140cb580 tx=0x7fdf0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff7ef93dd46cee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:25.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.882+0000 7fdf12000640 1 -- 172.21.15.59:0/3444178296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04037070 con 0x7fdf1415e7c0 2024-04-23T14:38:25.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.882+0000 7fdf12000640 1 -- 172.21.15.59:0/3444178296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf04032440 con 0x7fdf1415e7c0 2024-04-23T14:38:25.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.882+0000 7fdf12000640 1 -- 172.21.15.59:0/3444178296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04032810 con 0x7fdf1415e7c0 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 -- 172.21.15.59:0/3444178296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 msgr2=0x7fdf140d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3444178296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140d20f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fdf140cb580 tx=0x7fdf0402a1a0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 -- 172.21.15.59:0/3444178296 shutdown_connections 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3444178296 >> v2:172.21.15.76:3300/0 conn(0x7fdf140d0d90 0x7fdf140d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3444178296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140d20f0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 -- 172.21.15.59:0/3444178296 >> 172.21.15.59:0/3444178296 conn(0x7fdf1409d920 msgr2=0x7fdf1408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 -- 172.21.15.59:0/3444178296 shutdown_connections 2024-04-23T14:38:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.883+0000 7fdf18a00640 1 -- 172.21.15.59:0/3444178296 wait complete. 2024-04-23T14:38:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.884+0000 7fdf18a00640 1 Processor -- start 2024-04-23T14:38:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.885+0000 7fdf18a00640 1 -- start start 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf18a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf140d0d90 0x7fdf14157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf18a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf140d2630 0x7fdf140cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf18a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf18a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf140cf2b0 con 0x7fdf1415e7c0 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf18a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf140d1700 con 0x7fdf140d0d90 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf18a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdf140b6700 con 0x7fdf140d2630 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58866/0 (socket says 172.21.15.59:58866) 2024-04-23T14:38:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 -- 172.21.15.59:0/3248259263 learned_addr learned my addr 172.21.15.59:0/3248259263 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 -- 172.21.15.59:0/3248259263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf140d2630 msgr2=0x7fdf140cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf140d2630 0x7fdf140cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 -- 172.21.15.59:0/3248259263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf140d0d90 msgr2=0x7fdf14157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13400640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf140d0d90 0x7fdf14157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf140d0d90 0x7fdf14157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.886+0000 7fdf13e00640 1 -- 172.21.15.59:0/3248259263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf140cbd90 con 0x7fdf1415e7c0 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.887+0000 7fdf13e00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140cb5f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c007e60 tx=0x7fdf0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.887+0000 7fdf0b400640 1 -- 172.21.15.59:0/3248259263 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c014420 con 0x7fdf1415e7c0 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.887+0000 7fdf0b400640 1 -- 172.21.15.59:0/3248259263 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0c019070 con 0x7fdf1415e7c0 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.887+0000 7fdf0b400640 1 -- 172.21.15.59:0/3248259263 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c014820 con 0x7fdf1415e7c0 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.887+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf140cc070 con 0x7fdf1415e7c0 2024-04-23T14:38:25.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.887+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf140cc630 con 0x7fdf1415e7c0 2024-04-23T14:38:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.888+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1404fd90 con 0x7fdf1415e7c0 2024-04-23T14:38:25.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.889+0000 7fdf0b400640 1 -- 172.21.15.59:0/3248259263 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf0c0149c0 con 0x7fdf1415e7c0 2024-04-23T14:38:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.891+0000 7fdf0b400640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdef406c8f0 0x7fdef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.891+0000 7fdf0b400640 1 -- 172.21.15.59:0/3248259263 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7fdf0c08c550 con 0x7fdf1415e7c0 2024-04-23T14:38:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.891+0000 7fdf13400640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdef406c8f0 0x7fdef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:25.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.892+0000 7fdf13400640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdef406c8f0 0x7fdef406edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdf04034000 tx=0x7fdf0402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:25.896+0000 7fdf0b400640 1 -- 172.21.15.59:0/3248259263 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf0c05a850 con 0x7fdf1415e7c0 2024-04-23T14:38:26.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.101+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf140d1eb0 con 0x7fdf1415e7c0 2024-04-23T14:38:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.103+0000 7fdf0b400640 1 -- 172.21.15.59:0/3248259263 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16834 (secure 0 0 0) 0x7fdf0c05a150 con 0x7fdf1415e7c0 2024-04-23T14:38:26.105 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:26.105 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:37:08.944193+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:36:27.526201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-23T14:36:28.678324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.8","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6836/2883460093":"2024-04-24T14:36:50.829845+0000","172.21.15.59:6834/2883460093":"2024-04-24T14:36:50.829845+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:26.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.106+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdef406c8f0 msgr2=0x7fdef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.106+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdef406c8f0 0x7fdef406edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdf04034000 tx=0x7fdf0402e000 comp rx=0 tx=0).stop 2024-04-23T14:38:26.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.106+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 msgr2=0x7fdf140cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.106+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140cb5f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c007e60 tx=0x7fdf0c00e4b0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 shutdown_connections 2024-04-23T14:38:26.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdef406c8f0 0x7fdef406edb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf140d2630 0x7fdf140cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf1415e7c0 0x7fdf140cb5f0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 --2- 172.21.15.59:0/3248259263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf140d0d90 0x7fdf14157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 >> 172.21.15.59:0/3248259263 conn(0x7fdf1409d920 msgr2=0x7fdf14160750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:26.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 shutdown_connections 2024-04-23T14:38:26.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.107+0000 7fdf18a00640 1 -- 172.21.15.59:0/3248259263 wait complete. 2024-04-23T14:38:26.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.341+0000 7f722d600640 1 Processor -- start 2024-04-23T14:38:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.341+0000 7f722d600640 1 -- start start 2024-04-23T14:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f722d600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7228161ef0 0x7f72280b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f722d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f722809aa00 0x7f722809ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f722d600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722809b340 con 0x7f722809b740 2024-04-23T14:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f722d600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f722809bbd0 con 0x7f722809aa00 2024-04-23T14:38:26.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f722d600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f722806d830 con 0x7f7228161ef0 2024-04-23T14:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f7227400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f722809aa00 0x7f722809ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f721e000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f722809b740 0x7f72280b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33628/0 (socket says 172.21.15.59:33628) 2024-04-23T14:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.343+0000 7f721e000640 1 -- 172.21.15.59:0/3807862807 learned_addr learned my addr 172.21.15.59:0/3807862807 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.344+0000 7f7226a00640 1 --2- 172.21.15.59:0/3807862807 >> v2:172.21.15.76:3300/0 conn(0x7f7228161ef0 0x7f72280b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.344+0000 7f7227400640 1 -- 172.21.15.59:0/3807862807 >> v2:172.21.15.76:3300/0 conn(0x7f7228161ef0 msgr2=0x7f72280b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.344+0000 7f7227400640 1 --2- 172.21.15.59:0/3807862807 >> v2:172.21.15.76:3300/0 conn(0x7f7228161ef0 0x7f72280b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.344+0000 7f7227400640 1 -- 172.21.15.59:0/3807862807 >> v1:172.21.15.59:6789/0 conn(0x7f722809b740 legacy=0x7f72280b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.344+0000 7f7226000640 1 -- 172.21.15.59:0/3807862807 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230049403 0 0) 0x7f722809b340 con 0x7f722809b740 2024-04-23T14:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.344+0000 7f7227400640 1 -- 172.21.15.59:0/3807862807 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722806db10 con 0x7f722809aa00 2024-04-23T14:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.345+0000 7f7227400640 1 --2- 172.21.15.59:0/3807862807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f722809aa00 0x7f722809ae00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f72280738c0 tx=0x7f722002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92503ee55b9ae14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:26.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.345+0000 7f7226000640 1 -- 172.21.15.59:0/3807862807 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220007740 con 0x7f722809aa00 2024-04-23T14:38:26.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.345+0000 7f7226000640 1 -- 172.21.15.59:0/3807862807 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72200078e0 con 0x7f722809aa00 2024-04-23T14:38:26.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.345+0000 7f7226000640 1 -- 172.21.15.59:0/3807862807 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220007cb0 con 0x7f722809aa00 2024-04-23T14:38:26.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 -- 172.21.15.59:0/3807862807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f722809aa00 msgr2=0x7f722809ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 --2- 172.21.15.59:0/3807862807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f722809aa00 0x7f722809ae00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f72280738c0 tx=0x7f722002a5f0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 -- 172.21.15.59:0/3807862807 shutdown_connections 2024-04-23T14:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 --2- 172.21.15.59:0/3807862807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f722809aa00 0x7f722809ae00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 --2- 172.21.15.59:0/3807862807 >> v2:172.21.15.76:3300/0 conn(0x7f7228161ef0 0x7f72280b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 -- 172.21.15.59:0/3807862807 >> 172.21.15.59:0/3807862807 conn(0x7f72280b4030 msgr2=0x7f72280b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 -- 172.21.15.59:0/3807862807 shutdown_connections 2024-04-23T14:38:26.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.346+0000 7f722d600640 1 -- 172.21.15.59:0/3807862807 wait complete. 2024-04-23T14:38:26.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.348+0000 7f722d600640 1 Processor -- start 2024-04-23T14:38:26.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.348+0000 7f722d600640 1 -- start start 2024-04-23T14:38:26.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.349+0000 7f722d600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f722809aa00 0x7f7228160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.349+0000 7f722d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f722809b740 0x7f7228157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.349+0000 7f722d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7228161ef0 0x7f7228157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.349+0000 7f722d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72281622d0 con 0x7f7228161ef0 2024-04-23T14:38:26.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.349+0000 7f722d600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72280d0c00 con 0x7f722809aa00 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.349+0000 7f722d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f72280d08b0 con 0x7f722809b740 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7228161ef0 0x7f7228157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7228161ef0 0x7f7228157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58882/0 (socket says 172.21.15.59:58882) 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 -- 172.21.15.59:0/2500857218 learned_addr learned my addr 172.21.15.59:0/2500857218 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7226a00640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f722809aa00 0x7f7228160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f721e000640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f722809b740 0x7f7228157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 -- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f722809b740 msgr2=0x7f7228157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f722809b740 0x7f7228157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 -- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f722809aa00 msgr2=0x7f7228160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f722809aa00 0x7f7228160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 -- 172.21.15.59:0/2500857218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72280c8380 con 0x7f7228161ef0 2024-04-23T14:38:26.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.350+0000 7f7227400640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7228161ef0 0x7f7228157b60 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f722002abe0 tx=0x7f7220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:26.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.351+0000 7f721f400640 1 -- 172.21.15.59:0/2500857218 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722003f070 con 0x7f7228161ef0 2024-04-23T14:38:26.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.351+0000 7f721f400640 1 -- 172.21.15.59:0/2500857218 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7220031bc0 con 0x7f7228161ef0 2024-04-23T14:38:26.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.351+0000 7f721f400640 1 -- 172.21.15.59:0/2500857218 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220044460 con 0x7f7228161ef0 2024-04-23T14:38:26.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.351+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72280c8660 con 0x7f7228161ef0 2024-04-23T14:38:26.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.351+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72280c8c20 con 0x7f7228161ef0 2024-04-23T14:38:26.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.352+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72280d2030 con 0x7f7228161ef0 2024-04-23T14:38:26.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.353+0000 7f721f400640 1 -- 172.21.15.59:0/2500857218 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7220007790 con 0x7f7228161ef0 2024-04-23T14:38:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.354+0000 7f721f400640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f720406c8f0 0x7f720406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.354+0000 7f721f400640 1 -- 172.21.15.59:0/2500857218 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5853+0+0 (secure 0 0 0) 0x7f72200ad4e0 con 0x7f7228161ef0 2024-04-23T14:38:26.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.354+0000 7f7226a00640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f720406c8f0 0x7f720406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.355+0000 7f7226a00640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f720406c8f0 0x7f720406edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7218006fd0 tx=0x7f7218008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:26.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.359+0000 7f721f400640 1 -- 172.21.15.59:0/2500857218 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72280d2030 con 0x7f7228161ef0 2024-04-23T14:38:26.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.610+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7228068a80 con 0x7f7228161ef0 2024-04-23T14:38:26.612 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:38:26.611+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:38:26.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.614+0000 7f721f400640 1 -- 172.21.15.59:0/2500857218 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 161+0+0 (secure 0 0 0) 0x7f722004c350 con 0x7f7228161ef0 2024-04-23T14:38:26.615 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:38:26.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.617+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f720406c8f0 msgr2=0x7f720406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.617+0000 7f722d600640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f720406c8f0 0x7f720406edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7218006fd0 tx=0x7f7218008040 comp rx=0 tx=0).stop 2024-04-23T14:38:26.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.617+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7228161ef0 msgr2=0x7f7228157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.617+0000 7f722d600640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7228161ef0 0x7f7228157b60 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f722002abe0 tx=0x7f7220004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 shutdown_connections 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f720406c8f0 0x7f720406edb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f722809b740 0x7f7228157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7228161ef0 0x7f7228157b60 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 --2- 172.21.15.59:0/2500857218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f722809aa00 0x7f7228160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 >> 172.21.15.59:0/2500857218 conn(0x7f72280b4030 msgr2=0x7f7228071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 shutdown_connections 2024-04-23T14:38:26.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.618+0000 7f722d600640 1 -- 172.21.15.59:0/2500857218 wait complete. 2024-04-23T14:38:26.633 INFO:tasks.ceph.mds.a.smithi059.stderr: -4821> 2024-04-23T14:36:56.852+0000 7f4f35000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:38:26.640 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:38:26.703 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:38:26.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.830+0000 7f4ec7200640 1 Processor -- start 2024-04-23T14:38:26.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.830+0000 7f4ec7200640 1 -- start start 2024-04-23T14:38:26.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.831+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec8143ea0 0x7f4ec8144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.831+0000 7f4ec7200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ec806e570 0x7f4ec806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.831+0000 7f4ec7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec80b5a80 con 0x7f4ec80b01f0 2024-04-23T14:38:26.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.831+0000 7f4ec7200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec80b6690 con 0x7f4ec806e570 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.831+0000 7f4ec7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec80b5d60 con 0x7f4ec8143ea0 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ec806e570 0x7f4ec806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec5e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4ec80b01f0 0x7f4ec80b5170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33642/0 (socket says 172.21.15.59:33642) 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec5e00640 1 -- 172.21.15.59:0/3205264544 learned_addr learned my addr 172.21.15.59:0/3205264544 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/3205264544 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec8143ea0 0x7f4ec8144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec6800640 1 -- 172.21.15.59:0/3205264544 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec8143ea0 msgr2=0x7f4ec8144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec6800640 1 --2- 172.21.15.59:0/3205264544 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec8143ea0 0x7f4ec8144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec6800640 1 -- 172.21.15.59:0/3205264544 >> v1:172.21.15.59:6789/0 conn(0x7f4ec80b01f0 legacy=0x7f4ec80b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec6800640 1 -- 172.21.15.59:0/3205264544 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec80b6040 con 0x7f4ec806e570 2024-04-23T14:38:26.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.832+0000 7f4ec5400640 1 -- 172.21.15.59:0/3205264544 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818794227 0 0) 0x7f4ec80b5a80 con 0x7f4ec80b01f0 2024-04-23T14:38:26.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.833+0000 7f4ec6800640 1 --2- 172.21.15.59:0/3205264544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec806e570 0x7f4ec806e9a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c39d021be37ee6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec5400640 1 -- 172.21.15.59:0/3205264544 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0007740 con 0x7f4ec806e570 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec5400640 1 -- 172.21.15.59:0/3205264544 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec00078e0 con 0x7f4ec806e570 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec5400640 1 -- 172.21.15.59:0/3205264544 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0007cb0 con 0x7f4ec806e570 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec7200640 1 -- 172.21.15.59:0/3205264544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec806e570 msgr2=0x7f4ec806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec7200640 1 --2- 172.21.15.59:0/3205264544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec806e570 0x7f4ec806e9a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002a660 comp rx=0 tx=0).stop 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec7200640 1 -- 172.21.15.59:0/3205264544 shutdown_connections 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec7200640 1 --2- 172.21.15.59:0/3205264544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec806e570 0x7f4ec806e9a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec7200640 1 --2- 172.21.15.59:0/3205264544 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec8143ea0 0x7f4ec8144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec7200640 1 -- 172.21.15.59:0/3205264544 >> 172.21.15.59:0/3205264544 conn(0x7f4ec806b9d0 msgr2=0x7f4ec806bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.834+0000 7f4ec7200640 1 -- 172.21.15.59:0/3205264544 shutdown_connections 2024-04-23T14:38:26.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.835+0000 7f4ec7200640 1 -- 172.21.15.59:0/3205264544 wait complete. 2024-04-23T14:38:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.836+0000 7f4ec7200640 1 Processor -- start 2024-04-23T14:38:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.836+0000 7f4ec7200640 1 -- start start 2024-04-23T14:38:26.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.837+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ec806e570 0x7f4ec81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.837+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec80b01f0 0x7f4ec81f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.837+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec8143ea0 0x7f4ec815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.837+0000 7f4ec7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80b6ca0 con 0x7f4ec8143ea0 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.837+0000 7f4ec7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80b6b20 con 0x7f4ec806e570 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.837+0000 7f4ec7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec806eee0 con 0x7f4ec80b01f0 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.837+0000 7f4ec6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec8143ea0 0x7f4ec815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec8143ea0 0x7f4ec815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58906/0 (socket says 172.21.15.59:58906) 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 -- 172.21.15.59:0/2481109031 learned_addr learned my addr 172.21.15.59:0/2481109031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ebfe00640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec80b01f0 0x7f4ec81f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec5e00640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ec806e570 0x7f4ec81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 -- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec80b01f0 msgr2=0x7f4ec81f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec80b01f0 0x7f4ec81f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 -- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ec806e570 msgr2=0x7f4ec81586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ec806e570 0x7f4ec81586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 -- 172.21.15.59:0/2481109031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec8158c10 con 0x7f4ec8143ea0 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec5e00640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ec806e570 0x7f4ec81586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.838+0000 7f4ec6800640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec8143ea0 0x7f4ec815ad90 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4ec002ac50 tx=0x7f4ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.839+0000 7f4ebe000640 1 -- 172.21.15.59:0/2481109031 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0040070 con 0x7f4ec8143ea0 2024-04-23T14:38:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.839+0000 7f4ebe000640 1 -- 172.21.15.59:0/2481109031 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0031bc0 con 0x7f4ec8143ea0 2024-04-23T14:38:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.839+0000 7f4ebe000640 1 -- 172.21.15.59:0/2481109031 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0045460 con 0x7f4ec8143ea0 2024-04-23T14:38:26.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.839+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec815ba60 con 0x7f4ec8143ea0 2024-04-23T14:38:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.839+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec815bfe0 con 0x7f4ec8143ea0 2024-04-23T14:38:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.840+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec804fd90 con 0x7f4ec8143ea0 2024-04-23T14:38:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.841+0000 7f4ebe000640 1 -- 172.21.15.59:0/2481109031 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ec0039040 con 0x7f4ec8143ea0 2024-04-23T14:38:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.842+0000 7f4ebe000640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.842+0000 7f4ebe000640 1 -- 172.21.15.59:0/2481109031 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5939+0+0 (secure 0 0 0) 0x7f4ec00ad640 con 0x7f4ec8143ea0 2024-04-23T14:38:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.842+0000 7f4ec5e00640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:26.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.843+0000 7f4ec5e00640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8000f60 tx=0x7f4eb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:26.847+0000 7f4ebe000640 1 -- 172.21.15.59:0/2481109031 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ec0043360 con 0x7f4ec8143ea0 2024-04-23T14:38:27.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.088+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4ec8070b50 con 0x7f4ec8143ea0 2024-04-23T14:38:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.618+0000 7f4ebe000640 1 -- 172.21.15.59:0/2481109031 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7f4ec007b970 con 0x7f4ec8143ea0 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ea406c8f0 msgr2=0x7f4ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8000f60 tx=0x7f4eb8009040 comp rx=0 tx=0).stop 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec8143ea0 msgr2=0x7f4ec815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec8143ea0 0x7f4ec815ad90 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4ec002ac50 tx=0x7f4ec0004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 shutdown_connections 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ec80b01f0 0x7f4ec81f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ec8143ea0 0x7f4ec815ad90 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 --2- 172.21.15.59:0/2481109031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ec806e570 0x7f4ec81586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.621+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 >> 172.21.15.59:0/2481109031 conn(0x7f4ec806b9d0 msgr2=0x7f4ec806fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.622+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 shutdown_connections 2024-04-23T14:38:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.622+0000 7f4ec7200640 1 -- 172.21.15.59:0/2481109031 wait complete. 2024-04-23T14:38:27.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:38:27.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.858+0000 7fb519200640 1 Processor -- start 2024-04-23T14:38:27.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.858+0000 7fb519200640 1 -- start start 2024-04-23T14:38:27.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb519200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb51415e8e0 0x7fb5140cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb519200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb51406cbb0 0x7fb51406d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb519200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5140cf070 con 0x7fb5140d4210 2024-04-23T14:38:27.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb519200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb5140cf2f0 con 0x7fb51406cbb0 2024-04-23T14:38:27.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb519200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb5140d1c60 con 0x7fb51415e8e0 2024-04-23T14:38:27.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb50be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb5140d4210 0x7fb5140d46c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33658/0 (socket says 172.21.15.59:33658) 2024-04-23T14:38:27.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb50be00640 1 -- 172.21.15.59:0/3589033620 learned_addr learned my addr 172.21.15.59:0/3589033620 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:27.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.860+0000 7fb513e00640 1 --2- 172.21.15.59:0/3589033620 >> v2:172.21.15.59:3300/0 conn(0x7fb51406cbb0 0x7fb51406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.861+0000 7fb513400640 1 --2- 172.21.15.59:0/3589033620 >> v2:172.21.15.76:3300/0 conn(0x7fb51415e8e0 0x7fb5140cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.861+0000 7fb512a00640 1 -- 172.21.15.59:0/3589033620 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092414907 0 0) 0x7fb4fc001290 con 0x7fb5140d4210 2024-04-23T14:38:27.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.861+0000 7fb512a00640 1 -- 172.21.15.59:0/3589033620 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4f8003520 con 0x7fb5140d4210 2024-04-23T14:38:27.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.861+0000 7fb513e00640 1 -- 172.21.15.59:0/3589033620 >> v2:172.21.15.76:3300/0 conn(0x7fb51415e8e0 msgr2=0x7fb5140cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:27.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.861+0000 7fb513e00640 1 --2- 172.21.15.59:0/3589033620 >> v2:172.21.15.76:3300/0 conn(0x7fb51415e8e0 0x7fb5140cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.861+0000 7fb513e00640 1 -- 172.21.15.59:0/3589033620 >> v1:172.21.15.59:6789/0 conn(0x7fb5140d4210 legacy=0x7fb5140d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.861+0000 7fb513e00640 1 -- 172.21.15.59:0/3589033620 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5140cf5b0 con 0x7fb51406cbb0 2024-04-23T14:38:27.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.862+0000 7fb513e00640 1 --2- 172.21.15.59:0/3589033620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51406cbb0 0x7fb51406d040 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb51406c550 tx=0x7fb50c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b76db19a27bbc78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:27.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.862+0000 7fb512a00640 1 -- 172.21.15.59:0/3589033620 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c007740 con 0x7fb51406cbb0 2024-04-23T14:38:27.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.862+0000 7fb512a00640 1 -- 172.21.15.59:0/3589033620 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c0078e0 con 0x7fb51406cbb0 2024-04-23T14:38:27.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.862+0000 7fb512a00640 1 -- 172.21.15.59:0/3589033620 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c007cb0 con 0x7fb51406cbb0 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 -- 172.21.15.59:0/3589033620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51406cbb0 msgr2=0x7fb51406d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 --2- 172.21.15.59:0/3589033620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51406cbb0 0x7fb51406d040 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb51406c550 tx=0x7fb50c02a5f0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 -- 172.21.15.59:0/3589033620 shutdown_connections 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 --2- 172.21.15.59:0/3589033620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51406cbb0 0x7fb51406d040 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 --2- 172.21.15.59:0/3589033620 >> v2:172.21.15.76:3300/0 conn(0x7fb51415e8e0 0x7fb5140cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 -- 172.21.15.59:0/3589033620 >> 172.21.15.59:0/3589033620 conn(0x7fb51408e060 msgr2=0x7fb51409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 -- 172.21.15.59:0/3589033620 shutdown_connections 2024-04-23T14:38:27.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.863+0000 7fb519200640 1 -- 172.21.15.59:0/3589033620 wait complete. 2024-04-23T14:38:27.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.865+0000 7fb519200640 1 Processor -- start 2024-04-23T14:38:27.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.865+0000 7fb519200640 1 -- start start 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb519200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb51406cbb0 0x7fb5140ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb519200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5140d4210 0x7fb5141e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb519200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51415e8e0 0x7fb5141e7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb519200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb51415ecc0 con 0x7fb51415e8e0 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb519200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb51406d580 con 0x7fb5140d4210 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb519200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140d02f0 con 0x7fb51406cbb0 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb513e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51415e8e0 0x7fb5141e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb513e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51415e8e0 0x7fb5141e7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58922/0 (socket says 172.21.15.59:58922) 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.866+0000 7fb513e00640 1 -- 172.21.15.59:0/1841318862 learned_addr learned my addr 172.21.15.59:0/1841318862 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb50be00640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5140d4210 0x7fb5141e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513e00640 1 -- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb51406cbb0 msgr2=0x7fb5140ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513400640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb51406cbb0 0x7fb5140ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513e00640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb51406cbb0 0x7fb5140ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513e00640 1 -- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5140d4210 msgr2=0x7fb5141e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513e00640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5140d4210 0x7fb5141e70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513e00640 1 -- 172.21.15.59:0/1841318862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5141616f0 con 0x7fb51415e8e0 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb50be00640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5140d4210 0x7fb5141e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513400640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb51406cbb0 0x7fb5140ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.867+0000 7fb513e00640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51415e8e0 0x7fb5141e7620 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb50c02abe0 tx=0x7fb50c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.868+0000 7fb50b400640 1 -- 172.21.15.59:0/1841318862 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c040070 con 0x7fb51415e8e0 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.868+0000 7fb50b400640 1 -- 172.21.15.59:0/1841318862 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c031b40 con 0x7fb51415e8e0 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.868+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51414b890 con 0x7fb51415e8e0 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.868+0000 7fb50b400640 1 -- 172.21.15.59:0/1841318862 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c045460 con 0x7fb51415e8e0 2024-04-23T14:38:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.868+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51414be50 con 0x7fb51415e8e0 2024-04-23T14:38:27.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.869+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51404fd90 con 0x7fb51415e8e0 2024-04-23T14:38:27.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.871+0000 7fb50b400640 1 -- 172.21.15.59:0/1841318862 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb50c039040 con 0x7fb51415e8e0 2024-04-23T14:38:27.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.871+0000 7fb50b400640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb4f006c8f0 0x7fb4f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.872+0000 7fb50b400640 1 -- 172.21.15.59:0/1841318862 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5939+0+0 (secure 0 0 0) 0x7fb50c0ad810 con 0x7fb51415e8e0 2024-04-23T14:38:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.872+0000 7fb513400640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb4f006c8f0 0x7fb4f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.872+0000 7fb513400640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb4f006c8f0 0x7fb4f006edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb504006fd0 tx=0x7fb504008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:27.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:27.875+0000 7fb50b400640 1 -- 172.21.15.59:0/1841318862 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb50c07bb30 con 0x7fb51415e8e0 2024-04-23T14:38:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.070+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb5140cea00 con 0x7fb51415e8e0 2024-04-23T14:38:28.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.621+0000 7fb50b400640 1 -- 172.21.15.59:0/1841318862 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) ==== 181+0+0 (secure 0 0 0) 0x7fb50c07b430 con 0x7fb51415e8e0 2024-04-23T14:38:28.623 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:38:28.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb4f006c8f0 msgr2=0x7fb4f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:28.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb4f006c8f0 0x7fb4f006edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb504006fd0 tx=0x7fb504008040 comp rx=0 tx=0).stop 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51415e8e0 msgr2=0x7fb5141e7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51415e8e0 0x7fb5141e7620 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb50c02abe0 tx=0x7fb50c004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 shutdown_connections 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb4f006c8f0 0x7fb4f006edb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51415e8e0 0x7fb5141e7620 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5140d4210 0x7fb5141e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 --2- 172.21.15.59:0/1841318862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb51406cbb0 0x7fb5140ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.625+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 >> 172.21.15.59:0/1841318862 conn(0x7fb51408e060 msgr2=0x7fb5140d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.626+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 shutdown_connections 2024-04-23T14:38:28.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.626+0000 7fb519200640 1 -- 172.21.15.59:0/1841318862 wait complete. 2024-04-23T14:38:28.654 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:38:28.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.865+0000 7f5ae3e00640 1 Processor -- start 2024-04-23T14:38:28.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.865+0000 7f5ae3e00640 1 -- start start 2024-04-23T14:38:28.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.866+0000 7f5ae3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ae40d4210 0x7f5ae40d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.866+0000 7f5ae3e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ae406cbb0 0x7f5ae406d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.866+0000 7f5ae3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae40cf070 con 0x7f5ae415e8e0 2024-04-23T14:38:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.866+0000 7f5ae3e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae40d04c0 con 0x7f5ae40d4210 2024-04-23T14:38:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.866+0000 7f5ae3e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae40d4c00 con 0x7f5ae406cbb0 2024-04-23T14:38:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.866+0000 7f5ae2a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5ae415e8e0 0x7f5ae40cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33662/0 (socket says 172.21.15.59:33662) 2024-04-23T14:38:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.866+0000 7f5ae2a00640 1 -- 172.21.15.59:0/1087681690 learned_addr learned my addr 172.21.15.59:0/1087681690 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:28.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae2000640 1 --2- 172.21.15.59:0/1087681690 >> v2:172.21.15.59:3300/0 conn(0x7f5ae40d4210 0x7f5ae40d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:28.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae3400640 1 --2- 172.21.15.59:0/1087681690 >> v2:172.21.15.76:3300/0 conn(0x7f5ae406cbb0 0x7f5ae406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:28.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae1600640 1 -- 172.21.15.59:0/1087681690 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368660735 0 0) 0x7f5ad4001110 con 0x7f5ae415e8e0 2024-04-23T14:38:28.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae1600640 1 -- 172.21.15.59:0/1087681690 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ac4003520 con 0x7f5ae415e8e0 2024-04-23T14:38:28.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae2000640 1 -- 172.21.15.59:0/1087681690 >> v2:172.21.15.76:3300/0 conn(0x7f5ae406cbb0 msgr2=0x7f5ae406d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:28.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae2000640 1 --2- 172.21.15.59:0/1087681690 >> v2:172.21.15.76:3300/0 conn(0x7f5ae406cbb0 0x7f5ae406d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae2000640 1 -- 172.21.15.59:0/1087681690 >> v1:172.21.15.59:6789/0 conn(0x7f5ae415e8e0 legacy=0x7f5ae40cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:28.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.867+0000 7f5ae2000640 1 -- 172.21.15.59:0/1087681690 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae40cf2f0 con 0x7f5ae40d4210 2024-04-23T14:38:28.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.868+0000 7f5ae2000640 1 --2- 172.21.15.59:0/1087681690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae40d4210 0x7f5ae40d46c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5ae406c960 tx=0x7f5acc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b13f7fc4badecb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae1600640 1 -- 172.21.15.59:0/1087681690 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc007740 con 0x7f5ae40d4210 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae1600640 1 -- 172.21.15.59:0/1087681690 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc02ae50 con 0x7f5ae40d4210 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae1600640 1 -- 172.21.15.59:0/1087681690 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc007b00 con 0x7f5ae40d4210 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 -- 172.21.15.59:0/1087681690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae40d4210 msgr2=0x7f5ae40d46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/1087681690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae40d4210 0x7f5ae40d46c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5ae406c960 tx=0x7f5acc02a630 comp rx=0 tx=0).stop 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 -- 172.21.15.59:0/1087681690 shutdown_connections 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/1087681690 >> v2:172.21.15.76:3300/0 conn(0x7f5ae406cbb0 0x7f5ae406d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/1087681690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae40d4210 0x7f5ae40d46c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 -- 172.21.15.59:0/1087681690 >> 172.21.15.59:0/1087681690 conn(0x7f5ae408e060 msgr2=0x7f5ae409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:28.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 -- 172.21.15.59:0/1087681690 shutdown_connections 2024-04-23T14:38:28.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.869+0000 7f5ae3e00640 1 -- 172.21.15.59:0/1087681690 wait complete. 2024-04-23T14:38:28.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.871+0000 7f5ae3e00640 1 Processor -- start 2024-04-23T14:38:28.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.871+0000 7f5ae3e00640 1 -- start start 2024-04-23T14:38:28.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae406cbb0 0x7f5ae40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:28.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae40d4210 0x7f5ae41f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ae415e8e0 0x7f5ae4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae40cf890 con 0x7f5ae406cbb0 2024-04-23T14:38:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae40cf5b0 con 0x7f5ae40d4210 2024-04-23T14:38:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae406d580 con 0x7f5ae415e8e0 2024-04-23T14:38:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae406cbb0 0x7f5ae40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.872+0000 7f5ae2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae406cbb0 0x7f5ae40cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58948/0 (socket says 172.21.15.59:58948) 2024-04-23T14:38:28.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2a00640 1 -- 172.21.15.59:0/2662050240 learned_addr learned my addr 172.21.15.59:0/2662050240 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2000640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae40d4210 0x7f5ae41f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2a00640 1 -- 172.21.15.59:0/2662050240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ae415e8e0 msgr2=0x7f5ae4161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2a00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ae415e8e0 0x7f5ae4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2a00640 1 -- 172.21.15.59:0/2662050240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae40d4210 msgr2=0x7f5ae41f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2a00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae40d4210 0x7f5ae41f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:28.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2a00640 1 -- 172.21.15.59:0/2662050240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae41f8060 con 0x7f5ae406cbb0 2024-04-23T14:38:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.873+0000 7f5ae2a00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae406cbb0 0x7f5ae40cde80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c3a0 tx=0x7f5ad400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.874+0000 7f5adaa00640 1 -- 172.21.15.59:0/2662050240 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4017070 con 0x7f5ae406cbb0 2024-04-23T14:38:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.874+0000 7f5adaa00640 1 -- 172.21.15.59:0/2662050240 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4005ce0 con 0x7f5ae406cbb0 2024-04-23T14:38:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.874+0000 7f5adaa00640 1 -- 172.21.15.59:0/2662050240 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4007310 con 0x7f5ae406cbb0 2024-04-23T14:38:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.874+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae4162090 con 0x7f5ae406cbb0 2024-04-23T14:38:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.874+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae41f85e0 con 0x7f5ae406cbb0 2024-04-23T14:38:28.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.875+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae404fd90 con 0x7f5ae406cbb0 2024-04-23T14:38:28.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.876+0000 7f5adaa00640 1 -- 172.21.15.59:0/2662050240 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ad401a070 con 0x7f5ae406cbb0 2024-04-23T14:38:28.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.877+0000 7f5adaa00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ac006c8f0 0x7f5ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:28.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.877+0000 7f5adaa00640 1 -- 172.21.15.59:0/2662050240 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5501+0+0 (secure 0 0 0) 0x7f5ad408aed0 con 0x7f5ae406cbb0 2024-04-23T14:38:28.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.877+0000 7f5ae2000640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ac006c8f0 0x7f5ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:28.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.878+0000 7f5ae2000640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ac006c8f0 0x7f5ac006edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5acc034000 tx=0x7f5acc02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:28.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:28.881+0000 7f5adaa00640 1 -- 172.21.15.59:0/2662050240 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ae40cf070 con 0x7f5ae406cbb0 2024-04-23T14:38:29.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.085+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ae406c990 con 0x7f5ae406cbb0 2024-04-23T14:38:29.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.634+0000 7f5adaa00640 1 -- 172.21.15.59:0/2662050240 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) ==== 169+0+0 (secure 0 0 0) 0x7f5ad40593c0 con 0x7f5ae406cbb0 2024-04-23T14:38:29.636 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:38:29.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.639+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ac006c8f0 msgr2=0x7f5ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.639+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ac006c8f0 0x7f5ac006edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5acc034000 tx=0x7f5acc02f000 comp rx=0 tx=0).stop 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.640+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae406cbb0 msgr2=0x7f5ae40cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.640+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae406cbb0 0x7f5ae40cde80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c3a0 tx=0x7f5ad400c860 comp rx=0 tx=0).stop 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.641+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 shutdown_connections 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.641+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ac006c8f0 0x7f5ac006edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.641+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ae415e8e0 0x7f5ae4161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.641+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ae40d4210 0x7f5ae41f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.641+0000 7f5ae3e00640 1 --2- 172.21.15.59:0/2662050240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ae406cbb0 0x7f5ae40cde80 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.641+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 >> 172.21.15.59:0/2662050240 conn(0x7f5ae408e060 msgr2=0x7f5ae40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:29.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.642+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 shutdown_connections 2024-04-23T14:38:29.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.642+0000 7f5ae3e00640 1 -- 172.21.15.59:0/2662050240 wait complete. 2024-04-23T14:38:29.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:38:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.865+0000 7f5d96200640 1 Processor -- start 2024-04-23T14:38:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.865+0000 7f5d96200640 1 -- start start 2024-04-23T14:38:29.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.867+0000 7f5d96200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d900d0ce0 0x7f5d900af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.867+0000 7f5d96200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d90160120 con 0x7f5d9015c410 2024-04-23T14:38:29.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.867+0000 7f5d96200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d90160400 con 0x7f5d900af5c0 2024-04-23T14:38:29.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.867+0000 7f5d96200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5d901606e0 con 0x7f5d900d0ce0 2024-04-23T14:38:29.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.867+0000 7f5d94e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5d9015c410 0x7f5d900d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33664/0 (socket says 172.21.15.59:33664) 2024-04-23T14:38:29.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.867+0000 7f5d94e00640 1 -- 172.21.15.59:0/1363196047 learned_addr learned my addr 172.21.15.59:0/1363196047 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:29.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122282598 0 0) 0x7f5d90160120 con 0x7f5d9015c410 2024-04-23T14:38:29.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d78003520 con 0x7f5d9015c410 2024-04-23T14:38:29.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8fe00640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d900d0ce0 0x7f5d900af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:29.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3756683156 0 0) 0x7f5d78003520 con 0x7f5d9015c410 2024-04-23T14:38:29.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d900d0ce0 msgr2=0x7f5d900af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:29.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d900d0ce0 0x7f5d900af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 >> v1:172.21.15.76:6789/0 conn(0x7f5d900af5c0 legacy=0x7f5d90071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d901609c0 con 0x7f5d9015c410 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f5d84001670 con 0x7f5d9015c410 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 >> v1:172.21.15.59:6789/0 conn(0x7f5d9015c410 legacy=0x7f5d900d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d78004800 0x7f5d78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d780051c0 0x7f5d7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901606e0 con 0x7f5d780039e0 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7800c0c0 con 0x7f5d780051c0 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7800c2c0 con 0x7f5d78004800 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.868+0000 7f5d8fe00640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d900d0ce0 0x7f5d900af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d94e00640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d95800640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d780051c0 0x7f5d7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:29.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d94e00640 1 -- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d78004800 msgr2=0x7f5d78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:29.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d94e00640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d78004800 0x7f5d78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d94e00640 1 -- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d780051c0 msgr2=0x7f5d7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:29.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d94e00640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d780051c0 0x7f5d7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d94e00640 1 -- 172.21.15.59:0/1363196047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d78003520 con 0x7f5d780039e0 2024-04-23T14:38:29.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.869+0000 7f5d95800640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d780051c0 0x7f5d7800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:29.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.870+0000 7f5d94e00640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d78003e00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c0e0 tx=0x7f5d78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7278409b9fd3fd1c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:29.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84007c20 con 0x7f5d780039e0 2024-04-23T14:38:29.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d84005af0 con 0x7f5d780039e0 2024-04-23T14:38:29.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d8f400640 1 -- 172.21.15.59:0/1363196047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84007070 con 0x7f5d780039e0 2024-04-23T14:38:29.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d96200640 1 -- 172.21.15.59:0/1363196047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 msgr2=0x7f5d78003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:29.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d96200640 1 --2- 172.21.15.59:0/1363196047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d78003e00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c0e0 tx=0x7f5d78005830 comp rx=0 tx=0).stop 2024-04-23T14:38:29.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d95800640 1 -- 172.21.15.59:0/1363196047 reap_dead start 2024-04-23T14:38:29.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d96200640 1 -- 172.21.15.59:0/1363196047 shutdown_connections 2024-04-23T14:38:29.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d96200640 1 -- 172.21.15.59:0/1363196047 >> 172.21.15.59:0/1363196047 conn(0x7f5d9006bdf0 msgr2=0x7f5d9006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:29.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d96200640 1 -- 172.21.15.59:0/1363196047 shutdown_connections 2024-04-23T14:38:29.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.871+0000 7f5d96200640 1 -- 172.21.15.59:0/1363196047 wait complete. 2024-04-23T14:38:29.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.873+0000 7f5d96200640 1 Processor -- start 2024-04-23T14:38:29.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.873+0000 7f5d96200640 1 -- start start 2024-04-23T14:38:29.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.874+0000 7f5d96200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d900b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.874+0000 7f5d96200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d900cf010 0x7f5d9015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.874+0000 7f5d96200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d901f7dd0 0x7f5d901f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.874+0000 7f5d96200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d900afbb0 con 0x7f5d780039e0 2024-04-23T14:38:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.874+0000 7f5d96200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5d900afa30 con 0x7f5d900cf010 2024-04-23T14:38:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.874+0000 7f5d96200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5d900af080 con 0x7f5d901f7dd0 2024-04-23T14:38:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.874+0000 7f5d94e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d900b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d900b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58954/0 (socket says 172.21.15.59:58954) 2024-04-23T14:38:29.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 -- 172.21.15.59:0/2188701168 learned_addr learned my addr 172.21.15.59:0/2188701168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d8fe00640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d900cf010 0x7f5d9015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d95800640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d901f7dd0 0x7f5d901f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 -- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d901f7dd0 msgr2=0x7f5d901f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d901f7dd0 0x7f5d901f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 -- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d900cf010 msgr2=0x7f5d9015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d900cf010 0x7f5d9015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 -- 172.21.15.59:0/2188701168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d90157d90 con 0x7f5d780039e0 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.875+0000 7f5d94e00640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d900b2800 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c200 tx=0x7f5d90160b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.876+0000 7f5d8cc00640 1 -- 172.21.15.59:0/2188701168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84001630 con 0x7f5d780039e0 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.876+0000 7f5d8cc00640 1 -- 172.21.15.59:0/2188701168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d84042070 con 0x7f5d780039e0 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.876+0000 7f5d8cc00640 1 -- 172.21.15.59:0/2188701168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d840072b0 con 0x7f5d780039e0 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.876+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d900cd5e0 con 0x7f5d780039e0 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.876+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d900ceaf0 con 0x7f5d780039e0 2024-04-23T14:38:29.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.877+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9004fd90 con 0x7f5d780039e0 2024-04-23T14:38:29.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.878+0000 7f5d8cc00640 1 -- 172.21.15.59:0/2188701168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d84007450 con 0x7f5d780039e0 2024-04-23T14:38:29.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.879+0000 7f5d8cc00640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d7006c8f0 0x7f5d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:29.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.880+0000 7f5d8cc00640 1 -- 172.21.15.59:0/2188701168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f5d840ab9e0 con 0x7f5d780039e0 2024-04-23T14:38:29.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.880+0000 7f5d8fe00640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d7006c8f0 0x7f5d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:29.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.880+0000 7f5d8fe00640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d7006c8f0 0x7f5d7006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5d900cf3f0 tx=0x7f5d7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:29.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:29.884+0000 7f5d8cc00640 1 -- 172.21.15.59:0/2188701168 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d8407a020 con 0x7f5d780039e0 2024-04-23T14:38:30.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.090+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5d900b5170 con 0x7f5d780039e0 2024-04-23T14:38:30.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.097+0000 7f5d8cc00640 1 -- 172.21.15.59:0/2188701168 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f5d84079920 con 0x7f5d780039e0 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d7006c8f0 msgr2=0x7f5d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d7006c8f0 0x7f5d7006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5d900cf3f0 tx=0x7f5d7c008040 comp rx=0 tx=0).stop 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 msgr2=0x7f5d900b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d900b2800 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c200 tx=0x7f5d90160b90 comp rx=0 tx=0).stop 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 shutdown_connections 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d7006c8f0 0x7f5d7006edb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d901f7dd0 0x7f5d901f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d900cf010 0x7f5d9015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 --2- 172.21.15.59:0/2188701168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d780039e0 0x7f5d900b2800 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.100+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 >> 172.21.15.59:0/2188701168 conn(0x7f5d9006bdf0 msgr2=0x7f5d90159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:30.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.101+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 shutdown_connections 2024-04-23T14:38:30.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.101+0000 7f5d96200640 1 -- 172.21.15.59:0/2188701168 wait complete. 2024-04-23T14:38:30.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-23T14:38:30.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.334+0000 7f200be00640 1 Processor -- start 2024-04-23T14:38:30.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.334+0000 7f200be00640 1 -- start start 2024-04-23T14:38:30.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f200c0cd050 0x7f200c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f200c15f620 0x7f200c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f200c15be00 con 0x7f200c0af540 2024-04-23T14:38:30.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f200c06fa50 con 0x7f200c0cd050 2024-04-23T14:38:30.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f200c0cd970 con 0x7f200c15f620 2024-04-23T14:38:30.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f200c0cd050 0x7f200c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f200c0af540 0x7f200c15b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46446/0 (socket says 172.21.15.59:46446) 2024-04-23T14:38:30.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.336+0000 7f200a000640 1 -- 172.21.15.59:0/2059950410 learned_addr learned my addr 172.21.15.59:0/2059950410 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.337+0000 7f200b400640 1 --2- 172.21.15.59:0/2059950410 >> v2:172.21.15.76:3300/0 conn(0x7f200c15f620 0x7f200c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.337+0000 7f200aa00640 1 -- 172.21.15.59:0/2059950410 >> v2:172.21.15.76:3300/0 conn(0x7f200c15f620 msgr2=0x7f200c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.337+0000 7f200aa00640 1 --2- 172.21.15.59:0/2059950410 >> v2:172.21.15.76:3300/0 conn(0x7f200c15f620 0x7f200c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.337+0000 7f200aa00640 1 -- 172.21.15.59:0/2059950410 >> v1:172.21.15.59:6789/0 conn(0x7f200c0af540 legacy=0x7f200c15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.337+0000 7f200aa00640 1 -- 172.21.15.59:0/2059950410 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c15c070 con 0x7f200c0cd050 2024-04-23T14:38:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.337+0000 7f2009600640 1 -- 172.21.15.59:0/2059950410 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073633128 0 0) 0x7f200c15be00 con 0x7f200c0af540 2024-04-23T14:38:30.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.337+0000 7f200b400640 1 --2- 172.21.15.59:0/2059950410 >> v2:172.21.15.76:3300/0 conn(0x7f200c15f620 0x7f200c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:30.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.338+0000 7f200aa00640 1 --2- 172.21.15.59:0/2059950410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c0cd430 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f200c0d0160 tx=0x7f1ffc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adf0d6f085b64f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.338+0000 7f2009600640 1 -- 172.21.15.59:0/2059950410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc007740 con 0x7f200c0cd050 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.338+0000 7f2009600640 1 -- 172.21.15.59:0/2059950410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ffc0078e0 con 0x7f200c0cd050 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.338+0000 7f2009600640 1 -- 172.21.15.59:0/2059950410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc007cb0 con 0x7f200c0cd050 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 -- 172.21.15.59:0/2059950410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 msgr2=0x7f200c0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 --2- 172.21.15.59:0/2059950410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c0cd430 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f200c0d0160 tx=0x7f1ffc02a680 comp rx=0 tx=0).stop 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 -- 172.21.15.59:0/2059950410 shutdown_connections 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 --2- 172.21.15.59:0/2059950410 >> v2:172.21.15.76:3300/0 conn(0x7f200c15f620 0x7f200c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 --2- 172.21.15.59:0/2059950410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c0cd430 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 -- 172.21.15.59:0/2059950410 >> 172.21.15.59:0/2059950410 conn(0x7f200c0b21a0 msgr2=0x7f200c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 -- 172.21.15.59:0/2059950410 shutdown_connections 2024-04-23T14:38:30.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.339+0000 7f200be00640 1 -- 172.21.15.59:0/2059950410 wait complete. 2024-04-23T14:38:30.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.341+0000 7f200be00640 1 Processor -- start 2024-04-23T14:38:30.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.341+0000 7f200be00640 1 -- start start 2024-04-23T14:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.342+0000 7f200be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f200c0af540 0x7f200c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.342+0000 7f200be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.342+0000 7f200be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f200c15f620 0x7f200c0c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.342+0000 7f200be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f200c15f100 con 0x7f200c0cd050 2024-04-23T14:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.342+0000 7f200be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f200c15c630 con 0x7f200c15f620 2024-04-23T14:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.342+0000 7f200be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f200c0b3560 con 0x7f200c0af540 2024-04-23T14:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45864/0 (socket says 172.21.15.59:45864) 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 -- 172.21.15.59:0/1594786122 learned_addr learned my addr 172.21.15.59:0/1594786122 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200aa00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f200c0af540 0x7f200c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200b400640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f200c15f620 0x7f200c0c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 -- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f200c0af540 msgr2=0x7f200c0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f200c0af540 0x7f200c0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 -- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f200c15f620 msgr2=0x7f200c0c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f200c15f620 0x7f200c0c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 -- 172.21.15.59:0/1594786122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c0c4aa0 con 0x7f200c0cd050 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200aa00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f200c0af540 0x7f200c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:30.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.343+0000 7f200a000640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c06b9d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c500 tx=0x7f1ff400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.344+0000 7f2002a00640 1 -- 172.21.15.59:0/1594786122 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4018070 con 0x7f200c0cd050 2024-04-23T14:38:30.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.344+0000 7f2002a00640 1 -- 172.21.15.59:0/1594786122 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff4007580 con 0x7f200c0cd050 2024-04-23T14:38:30.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.344+0000 7f2002a00640 1 -- 172.21.15.59:0/1594786122 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4004a20 con 0x7f200c0cd050 2024-04-23T14:38:30.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.344+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c0c4d80 con 0x7f200c0cd050 2024-04-23T14:38:30.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.344+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c1f4fe0 con 0x7f200c0cd050 2024-04-23T14:38:30.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.345+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200c04fd90 con 0x7f200c0cd050 2024-04-23T14:38:30.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.346+0000 7f2002a00640 1 -- 172.21.15.59:0/1594786122 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ff4004bc0 con 0x7f200c0cd050 2024-04-23T14:38:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.347+0000 7f2002a00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fe806c8f0 0x7f1fe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.347+0000 7f2002a00640 1 -- 172.21.15.59:0/1594786122 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f1ff408ab30 con 0x7f200c0cd050 2024-04-23T14:38:30.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.347+0000 7f200aa00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fe806c8f0 0x7f1fe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.348+0000 7f200aa00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fe806c8f0 0x7f1fe806edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc034000 tx=0x7f1ffc02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.352+0000 7f2002a00640 1 -- 172.21.15.59:0/1594786122 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ff4059290 con 0x7f200c0cd050 2024-04-23T14:38:30.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.632+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f200c06f5a0 con 0x7f200c0cd050 2024-04-23T14:38:30.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.637+0000 7f2002a00640 1 -- 172.21.15.59:0/1594786122 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v348) ==== 166+0+0 (secure 0 0 0) 0x7f1ff4058b90 con 0x7f200c0cd050 2024-04-23T14:38:30.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fe806c8f0 msgr2=0x7f1fe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fe806c8f0 0x7f1fe806edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc034000 tx=0x7f1ffc02f000 comp rx=0 tx=0).stop 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 msgr2=0x7f200c06b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c06b9d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c500 tx=0x7f1ff400c9c0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 shutdown_connections 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fe806c8f0 0x7f1fe806edb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f200c15f620 0x7f200c0c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f200c0cd050 0x7f200c06b9d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 --2- 172.21.15.59:0/1594786122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f200c0af540 0x7f200c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 >> 172.21.15.59:0/1594786122 conn(0x7f200c0b21a0 msgr2=0x7f200c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 shutdown_connections 2024-04-23T14:38:30.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.640+0000 7f200be00640 1 -- 172.21.15.59:0/1594786122 wait complete. 2024-04-23T14:38:30.660 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-23T14:38:30.661 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-23T14:38:30.661 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-23T14:38:30.661 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:30.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.860+0000 7fcca7200640 1 Processor -- start 2024-04-23T14:38:30.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.860+0000 7fcca7200640 1 -- start start 2024-04-23T14:38:30.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca7200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcca00b3080 0x7fcca00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca7200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca015f430 con 0x7fcca00cd050 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca7200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcca00ced20 con 0x7fcca00b3080 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcca015fcd0 con 0x7fcca006f620 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcca00b3080 0x7fcca00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca6800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcca00b3080 0x7fcca00cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45870/0 (socket says 172.21.15.59:45870) 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.862+0000 7fcca6800640 1 -- 172.21.15.59:0/919918250 learned_addr learned my addr 172.21.15.59:0/919918250 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.863+0000 7fcca5400640 1 --2- 172.21.15.59:0/919918250 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.863+0000 7fcca6800640 1 -- 172.21.15.59:0/919918250 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 msgr2=0x7fcca015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:30.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.863+0000 7fcca6800640 1 --2- 172.21.15.59:0/919918250 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.863+0000 7fcca6800640 1 -- 172.21.15.59:0/919918250 >> v1:172.21.15.76:6789/0 conn(0x7fcca00cd050 legacy=0x7fcca00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.863+0000 7fcca6800640 1 -- 172.21.15.59:0/919918250 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca015f710 con 0x7fcca00b3080 2024-04-23T14:38:30.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.864+0000 7fcca6800640 1 --2- 172.21.15.59:0/919918250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00cf470 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009e50 tx=0x7fcc9c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd34454b7095802f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.864+0000 7fcca4a00640 1 -- 172.21.15.59:0/919918250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c036070 con 0x7fcca00b3080 2024-04-23T14:38:30.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.864+0000 7fcca4a00640 1 -- 172.21.15.59:0/919918250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c031600 con 0x7fcca00b3080 2024-04-23T14:38:30.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.864+0000 7fcca4a00640 1 -- 172.21.15.59:0/919918250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0319d0 con 0x7fcca00b3080 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 -- 172.21.15.59:0/919918250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 msgr2=0x7fcca00cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 --2- 172.21.15.59:0/919918250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00cf470 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009e50 tx=0x7fcc9c02a680 comp rx=0 tx=0).stop 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 -- 172.21.15.59:0/919918250 shutdown_connections 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 --2- 172.21.15.59:0/919918250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00cf470 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 --2- 172.21.15.59:0/919918250 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 -- 172.21.15.59:0/919918250 >> 172.21.15.59:0/919918250 conn(0x7fcca00b21a0 msgr2=0x7fcca00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 -- 172.21.15.59:0/919918250 shutdown_connections 2024-04-23T14:38:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.865+0000 7fcca7200640 1 -- 172.21.15.59:0/919918250 wait complete. 2024-04-23T14:38:30.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.867+0000 7fcca7200640 1 Processor -- start 2024-04-23T14:38:30.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.867+0000 7fcca7200640 1 -- start start 2024-04-23T14:38:30.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.868+0000 7fcca7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca01f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.868+0000 7fcca7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.868+0000 7fcca7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca00cd050 0x7fcca015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.868+0000 7fcca7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcca00cef50 con 0x7fcca00b3080 2024-04-23T14:38:30.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.868+0000 7fcca7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcca006faa0 con 0x7fcca00cd050 2024-04-23T14:38:30.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.868+0000 7fcca7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcca0159110 con 0x7fcca006f620 2024-04-23T14:38:30.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45878/0 (socket says 172.21.15.59:45878) 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 -- 172.21.15.59:0/2947813398 learned_addr learned my addr 172.21.15.59:0/2947813398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5e00640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca01f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca6800640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca00cd050 0x7fcca015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 -- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 msgr2=0x7fcca01f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca01f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 -- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca00cd050 msgr2=0x7fcca015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca00cd050 0x7fcca015b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 -- 172.21.15.59:0/2947813398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca01f8ad0 con 0x7fcca00b3080 2024-04-23T14:38:30.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.869+0000 7fcca5400640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00ad3d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000d800 tx=0x7fcc9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.870+0000 7fcc9a000640 1 -- 172.21.15.59:0/2947813398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000b6c0 con 0x7fcca00b3080 2024-04-23T14:38:30.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.870+0000 7fcc9a000640 1 -- 172.21.15.59:0/2947813398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90019070 con 0x7fcca00b3080 2024-04-23T14:38:30.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.870+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca015a600 con 0x7fcca00b3080 2024-04-23T14:38:30.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.870+0000 7fcc9a000640 1 -- 172.21.15.59:0/2947813398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000bac0 con 0x7fcca00b3080 2024-04-23T14:38:30.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.870+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca015ab40 con 0x7fcca00b3080 2024-04-23T14:38:30.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.871+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca015a950 con 0x7fcca00b3080 2024-04-23T14:38:30.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.872+0000 7fcc9a000640 1 -- 172.21.15.59:0/2947813398 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc9000bc60 con 0x7fcca00b3080 2024-04-23T14:38:30.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.873+0000 7fcc9a000640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.873+0000 7fcc9a000640 1 -- 172.21.15.59:0/2947813398 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7fcc9008be80 con 0x7fcca00b3080 2024-04-23T14:38:30.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.874+0000 7fcca5e00640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.874+0000 7fcca5e00640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406c8f0 0x7fcc8406edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcc94007fd0 tx=0x7fcc94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:30.878+0000 7fcc9a000640 1 -- 172.21.15.59:0/2947813398 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcca01f8ad0 con 0x7fcca00b3080 2024-04-23T14:38:31.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.084+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcca00ca5c0 con 0x7fcca00b3080 2024-04-23T14:38:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.086+0000 7fcc9a000640 1 -- 172.21.15.59:0/2947813398 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+13091 (secure 0 0 0) 0x7fcc9005a4b0 con 0x7fcca00b3080 2024-04-23T14:38:31.087 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:31.087 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:38:29.623134+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6834/1773251382":"2024-04-24T14:38:26.611932+0000","172.21.15.59:6836/2883460093":"2024-04-24T14:36:50.829845+0000","172.21.15.59:6836/1773251382":"2024-04-24T14:38:26.611932+0000","172.21.15.59:6834/2883460093":"2024-04-24T14:36:50.829845+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.088+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406c8f0 msgr2=0x7fcc8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.088+0000 7fcca7200640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406c8f0 0x7fcc8406edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcc94007fd0 tx=0x7fcc94009040 comp rx=0 tx=0).stop 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.088+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 msgr2=0x7fcca00ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.088+0000 7fcca7200640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00ad3d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000d800 tx=0x7fcc9000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 shutdown_connections 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcca00cd050 0x7fcca015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca00b3080 0x7fcca00ad3d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 --2- 172.21.15.59:0/2947813398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcca006f620 0x7fcca01f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 >> 172.21.15.59:0/2947813398 conn(0x7fcca00b21a0 msgr2=0x7fcca00cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 shutdown_connections 2024-04-23T14:38:31.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.089+0000 7fcca7200640 1 -- 172.21.15.59:0/2947813398 wait complete. 2024-04-23T14:38:31.110 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:38:31.110 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:38:31.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.323+0000 7fc745400640 1 Processor -- start 2024-04-23T14:38:31.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.323+0000 7fc745400640 1 -- start start 2024-04-23T14:38:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.325+0000 7fc745400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7401567e0 0x7fc740156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.325+0000 7fc745400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc74015cab0 con 0x7fc74015d810 2024-04-23T14:38:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.325+0000 7fc745400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc740160e40 con 0x7fc7400d0d10 2024-04-23T14:38:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.325+0000 7fc745400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc74015cd30 con 0x7fc7401567e0 2024-04-23T14:38:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.325+0000 7fc73f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc74015d810 0x7fc7400d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46452/0 (socket says 172.21.15.59:46452) 2024-04-23T14:38:31.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.325+0000 7fc73f400640 1 -- 172.21.15.59:0/2827238958 learned_addr learned my addr 172.21.15.59:0/2827238958 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:31.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73fe00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7401567e0 0x7fc740156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049748721 0 0) 0x7fc728001110 con 0x7fc74015d810 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc724003520 con 0x7fc74015d810 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3453252831 0 0) 0x7fc724003520 con 0x7fc74015d810 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7401567e0 msgr2=0x7fc740156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7401567e0 0x7fc740156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 >> v1:172.21.15.76:6789/0 conn(0x7fc7400d0d10 legacy=0x7fc740160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74015d010 con 0x7fc74015d810 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fc728001840 con 0x7fc74015d810 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.326+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 >> v1:172.21.15.59:6789/0 conn(0x7fc74015d810 legacy=0x7fc7400d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73ea00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7240039e0 0x7fc724003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73ea00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc724004800 0x7fc724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73ea00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7401567e0 0x7fc7240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc74015cd30 con 0x7fc7240039e0 2024-04-23T14:38:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc72400ba70 con 0x7fc724004800 2024-04-23T14:38:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc72400bc70 con 0x7fc7401567e0 2024-04-23T14:38:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73fe00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7240039e0 0x7fc724003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73fe00640 1 -- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7401567e0 msgr2=0x7fc7240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73fe00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7401567e0 0x7fc7240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73fe00640 1 -- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc724004800 msgr2=0x7fc724004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73fe00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc724004800 0x7fc724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.327+0000 7fc73fe00640 1 -- 172.21.15.59:0/2827238958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc724003520 con 0x7fc7240039e0 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.328+0000 7fc73fe00640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7240039e0 0x7fc724003e00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc730009a90 tx=0x7fc724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89603493f0b988f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.328+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730036070 con 0x7fc7240039e0 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc730031400 con 0x7fc7240039e0 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc73ea00640 1 -- 172.21.15.59:0/2827238958 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730009690 con 0x7fc7240039e0 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc745400640 1 -- 172.21.15.59:0/2827238958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7240039e0 msgr2=0x7fc724003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc745400640 1 --2- 172.21.15.59:0/2827238958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7240039e0 0x7fc724003e00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc730009a90 tx=0x7fc724005830 comp rx=0 tx=0).stop 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc744a00640 1 -- 172.21.15.59:0/2827238958 reap_dead start 2024-04-23T14:38:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc745400640 1 -- 172.21.15.59:0/2827238958 shutdown_connections 2024-04-23T14:38:31.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc745400640 1 -- 172.21.15.59:0/2827238958 >> 172.21.15.59:0/2827238958 conn(0x7fc74006bdf0 msgr2=0x7fc74006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:31.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc745400640 1 -- 172.21.15.59:0/2827238958 shutdown_connections 2024-04-23T14:38:31.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.329+0000 7fc745400640 1 -- 172.21.15.59:0/2827238958 wait complete. 2024-04-23T14:38:31.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.331+0000 7fc745400640 1 Processor -- start 2024-04-23T14:38:31.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.331+0000 7fc745400640 1 -- start start 2024-04-23T14:38:31.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc745400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240039e0 0x7fc7400b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc745400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7400b2500 0x7fc7400b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc745400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7400b2960 0x7fc7401f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc745400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc740156d40 con 0x7fc7400b2960 2024-04-23T14:38:31.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc745400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc740156bc0 con 0x7fc7240039e0 2024-04-23T14:38:31.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc745400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc740156ec0 con 0x7fc7400b2500 2024-04-23T14:38:31.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc744a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7400b2960 0x7fc7401f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc744a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7400b2960 0x7fc7401f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45892/0 (socket says 172.21.15.59:45892) 2024-04-23T14:38:31.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.332+0000 7fc744a00640 1 -- 172.21.15.59:0/3067356684 learned_addr learned my addr 172.21.15.59:0/3067356684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc73f400640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7400b2500 0x7fc7400b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc73fe00640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240039e0 0x7fc7400b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc744a00640 1 -- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7400b2500 msgr2=0x7fc7400b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc744a00640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7400b2500 0x7fc7400b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc744a00640 1 -- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240039e0 msgr2=0x7fc7400b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc744a00640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240039e0 0x7fc7400b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc744a00640 1 -- 172.21.15.59:0/3067356684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7401593d0 con 0x7fc7400b2960 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.333+0000 7fc744a00640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7400b2960 0x7fc7401f60c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc740071c00 tx=0x7fc73800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.334+0000 7fc737e00640 1 -- 172.21.15.59:0/3067356684 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738017070 con 0x7fc7400b2960 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.334+0000 7fc737e00640 1 -- 172.21.15.59:0/3067356684 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc738005d60 con 0x7fc7400b2960 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.334+0000 7fc737e00640 1 -- 172.21.15.59:0/3067356684 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73800a850 con 0x7fc7400b2960 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.334+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc740097850 con 0x7fc7400b2960 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.334+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc740159950 con 0x7fc7400b2960 2024-04-23T14:38:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.335+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74004fd90 con 0x7fc7400b2960 2024-04-23T14:38:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.336+0000 7fc737e00640 1 -- 172.21.15.59:0/3067356684 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc73801a070 con 0x7fc7400b2960 2024-04-23T14:38:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.337+0000 7fc737e00640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.337+0000 7fc737e00640 1 -- 172.21.15.59:0/3067356684 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7fc73808ab80 con 0x7fc7400b2960 2024-04-23T14:38:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.337+0000 7fc73fe00640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.338+0000 7fc73fe00640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc71c06c8f0 0x7fc71c06edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc730033000 tx=0x7fc730008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:31.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.342+0000 7fc737e00640 1 -- 172.21.15.59:0/3067356684 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc738059240 con 0x7fc7400b2960 2024-04-23T14:38:31.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.547+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7400b40e0 con 0x7fc7400b2960 2024-04-23T14:38:31.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.548+0000 7fc737e00640 1 -- 172.21.15.59:0/3067356684 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7fc738058b40 con 0x7fc7400b2960 2024-04-23T14:38:31.550 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:38:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc71c06c8f0 msgr2=0x7fc71c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc71c06c8f0 0x7fc71c06edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc730033000 tx=0x7fc730008030 comp rx=0 tx=0).stop 2024-04-23T14:38:31.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7400b2960 msgr2=0x7fc7401f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7400b2960 0x7fc7401f60c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc740071c00 tx=0x7fc73800cb90 comp rx=0 tx=0).stop 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 shutdown_connections 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc71c06c8f0 0x7fc71c06edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7400b2500 0x7fc7400b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7400b2960 0x7fc7401f60c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 --2- 172.21.15.59:0/3067356684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7240039e0 0x7fc7400b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 >> 172.21.15.59:0/3067356684 conn(0x7fc74006bdf0 msgr2=0x7fc7400af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 shutdown_connections 2024-04-23T14:38:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.551+0000 7fc745400640 1 -- 172.21.15.59:0/3067356684 wait complete. 2024-04-23T14:38:31.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-23T14:38:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.787+0000 7f1486200640 1 Processor -- start 2024-04-23T14:38:31.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.787+0000 7f1486200640 1 -- start start 2024-04-23T14:38:31.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.788+0000 7f1486200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 0x7f14800cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.788+0000 7f1486200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14800b3080 0x7f14800cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.788+0000 7f1486200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14800b3870 con 0x7f148015b450 2024-04-23T14:38:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.788+0000 7f1486200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f14800b41a0 con 0x7f14800b3080 2024-04-23T14:38:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.788+0000 7f1486200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f14800b3b50 con 0x7f148006f3f0 2024-04-23T14:38:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.789+0000 7f1484e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f148015b450 0x7f14800b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46466/0 (socket says 172.21.15.59:46466) 2024-04-23T14:38:31.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.789+0000 7f1484e00640 1 -- 172.21.15.59:0/3566532722 learned_addr learned my addr 172.21.15.59:0/3566532722 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.789+0000 7f1485800640 1 --2- 172.21.15.59:0/3566532722 >> v2:172.21.15.76:3300/0 conn(0x7f14800b3080 0x7f14800cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.789+0000 7f147fe00640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 0x7f14800cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.789+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283923645 0 0) 0x7f14800b3870 con 0x7f148015b450 2024-04-23T14:38:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.789+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1464003520 con 0x7f148015b450 2024-04-23T14:38:31.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2737616889 0 0) 0x7f1464003520 con 0x7f148015b450 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 msgr2=0x7f14800cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 0x7f14800cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 >> v2:172.21.15.76:3300/0 conn(0x7f14800b3080 msgr2=0x7f14800cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 --2- 172.21.15.59:0/3566532722 >> v2:172.21.15.76:3300/0 conn(0x7f14800b3080 0x7f14800cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14800b3e30 con 0x7f148015b450 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1470001670 con 0x7f148015b450 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 >> v1:172.21.15.59:6789/0 conn(0x7f148015b450 legacy=0x7f14800b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1464003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 0x7f1464004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1464004d40 0x7f14640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14800b41a0 con 0x7f14640039b0 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f14800b3b50 con 0x7f1464004d40 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f146400dbd0 con 0x7f148006f3f0 2024-04-23T14:38:31.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.790+0000 7f1484e00640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1464003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f147fe00640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 0x7f1464004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f1484e00640 1 -- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 msgr2=0x7f1464004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f1485800640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1464004d40 0x7f14640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f1484e00640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 0x7f1464004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f1484e00640 1 -- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1464004d40 msgr2=0x7f14640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f1484e00640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1464004d40 0x7f14640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f1484e00640 1 -- 172.21.15.59:0/3566532722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1464003520 con 0x7f14640039b0 2024-04-23T14:38:31.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.791+0000 7f147fe00640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148006f3f0 0x7f1464004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.792+0000 7f1484e00640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1464003dd0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f147000c190 tx=0x7f1464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780bef644ef563ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.792+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470007c60 con 0x7f14640039b0 2024-04-23T14:38:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.792+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470005af0 con 0x7f14640039b0 2024-04-23T14:38:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.792+0000 7f147f400640 1 -- 172.21.15.59:0/3566532722 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470007070 con 0x7f14640039b0 2024-04-23T14:38:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.793+0000 7f1486200640 1 -- 172.21.15.59:0/3566532722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 msgr2=0x7f1464003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.793+0000 7f1486200640 1 --2- 172.21.15.59:0/3566532722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1464003dd0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f147000c190 tx=0x7f1464005830 comp rx=0 tx=0).stop 2024-04-23T14:38:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.793+0000 7f1485800640 1 -- 172.21.15.59:0/3566532722 reap_dead start 2024-04-23T14:38:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.793+0000 7f1486200640 1 -- 172.21.15.59:0/3566532722 shutdown_connections 2024-04-23T14:38:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.793+0000 7f1486200640 1 -- 172.21.15.59:0/3566532722 >> 172.21.15.59:0/3566532722 conn(0x7f14800b21a0 msgr2=0x7f14800b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.793+0000 7f1486200640 1 -- 172.21.15.59:0/3566532722 shutdown_connections 2024-04-23T14:38:31.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.793+0000 7f1486200640 1 -- 172.21.15.59:0/3566532722 wait complete. 2024-04-23T14:38:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.795+0000 7f1486200640 1 Processor -- start 2024-04-23T14:38:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.795+0000 7f1486200640 1 -- start start 2024-04-23T14:38:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.796+0000 7f1486200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1480073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.796+0000 7f1486200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f148015e1d0 0x7f148009d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.796+0000 7f1486200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148015e630 0x7f148009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.796+0000 7f1486200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f14800cd050 con 0x7f14640039b0 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.796+0000 7f1486200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f14800b4ff0 con 0x7f148015e1d0 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.796+0000 7f1486200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f14800cd1d0 con 0x7f148015e630 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1480073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1480073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45918/0 (socket says 172.21.15.59:45918) 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 -- 172.21.15.59:0/2580657391 learned_addr learned my addr 172.21.15.59:0/2580657391 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f147fe00640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f148015e1d0 0x7f148009d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1485800640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148015e630 0x7f148009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 -- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148015e630 msgr2=0x7f148009db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148015e630 0x7f148009db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 -- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f148015e1d0 msgr2=0x7f148009d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f148015e1d0 0x7f148009d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1484e00640 1 -- 172.21.15.59:0/2580657391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148014d980 con 0x7f14640039b0 2024-04-23T14:38:31.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.797+0000 7f1485800640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148015e630 0x7f148009db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:31.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.798+0000 7f1484e00640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1480073620 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f147002ad60 tx=0x7f14800b3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:31.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.798+0000 7f147cc00640 1 -- 172.21.15.59:0/2580657391 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470001630 con 0x7f14640039b0 2024-04-23T14:38:31.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.798+0000 7f147cc00640 1 -- 172.21.15.59:0/2580657391 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470042070 con 0x7f14640039b0 2024-04-23T14:38:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.798+0000 7f147cc00640 1 -- 172.21.15.59:0/2580657391 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470007270 con 0x7f14640039b0 2024-04-23T14:38:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.798+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148014dc20 con 0x7f14640039b0 2024-04-23T14:38:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.798+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148014e1e0 con 0x7f14640039b0 2024-04-23T14:38:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.799+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148004fd90 con 0x7f14640039b0 2024-04-23T14:38:31.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.800+0000 7f147cc00640 1 -- 172.21.15.59:0/2580657391 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1470007410 con 0x7f14640039b0 2024-04-23T14:38:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.801+0000 7f147cc00640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f146006c920 0x7f146006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.801+0000 7f147cc00640 1 -- 172.21.15.59:0/2580657391 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f14700ab9d0 con 0x7f14640039b0 2024-04-23T14:38:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.801+0000 7f147fe00640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f146006c920 0x7f146006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:31.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.802+0000 7f147fe00640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f146006c920 0x7f146006ede0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f14800cd6e0 tx=0x7f146c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:31.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:31.806+0000 7f147cc00640 1 -- 172.21.15.59:0/2580657391 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f147007a010 con 0x7f14640039b0 2024-04-23T14:38:32.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.083+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f14800cc0a0 con 0x7f14640039b0 2024-04-23T14:38:32.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.651+0000 7f147cc00640 1 -- 172.21.15.59:0/2580657391 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v350) ==== 170+0+0 (secure 0 0 0) 0x7f1470079910 con 0x7f14640039b0 2024-04-23T14:38:32.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f146006c920 msgr2=0x7f146006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:32.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f146006c920 0x7f146006ede0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f14800cd6e0 tx=0x7f146c00b040 comp rx=0 tx=0).stop 2024-04-23T14:38:32.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 msgr2=0x7f1480073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1480073620 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f147002ad60 tx=0x7f14800b3a40 comp rx=0 tx=0).stop 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 shutdown_connections 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f146006c920 0x7f146006ede0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f148015e630 0x7f148009db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f148015e1d0 0x7f148009d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 --2- 172.21.15.59:0/2580657391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14640039b0 0x7f1480073620 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 >> 172.21.15.59:0/2580657391 conn(0x7f14800b21a0 msgr2=0x7f14800d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 shutdown_connections 2024-04-23T14:38:32.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.655+0000 7f1486200640 1 -- 172.21.15.59:0/2580657391 wait complete. 2024-04-23T14:38:32.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:38:32.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.885+0000 7f4709600640 1 Processor -- start 2024-04-23T14:38:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.885+0000 7f4709600640 1 -- start start 2024-04-23T14:38:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.887+0000 7f4709600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 0x7f47040ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.887+0000 7f4709600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47040d09e0 0x7f470406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.887+0000 7f4709600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4704071e10 con 0x7f47040d0600 2024-04-23T14:38:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.887+0000 7f4709600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4704072a20 con 0x7f47040d09e0 2024-04-23T14:38:32.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.887+0000 7f4709600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f47040720f0 con 0x7f470406dcb0 2024-04-23T14:38:32.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.887+0000 7f4703e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f47040d0600 0x7f4704162d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46478/0 (socket says 172.21.15.59:46478) 2024-04-23T14:38:32.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.887+0000 7f4703e00640 1 -- 172.21.15.59:0/180440935 learned_addr learned my addr 172.21.15.59:0/180440935 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4708c00640 1 --2- 172.21.15.59:0/180440935 >> v2:172.21.15.76:3300/0 conn(0x7f47040d09e0 0x7f470406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f46fb400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 0x7f47040ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738847458 0 0) 0x7f4704071e10 con 0x7f47040d0600 2024-04-23T14:38:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e8003520 con 0x7f47040d0600 2024-04-23T14:38:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3822499194 0 0) 0x7f46e8003520 con 0x7f47040d0600 2024-04-23T14:38:32.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 msgr2=0x7f47040ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 0x7f47040ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 >> v2:172.21.15.76:3300/0 conn(0x7f47040d09e0 msgr2=0x7f470406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 --2- 172.21.15.59:0/180440935 >> v2:172.21.15.76:3300/0 conn(0x7f47040d09e0 0x7f470406e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47040723d0 con 0x7f47040d0600 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f46f40019f0 con 0x7f47040d0600 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 >> v1:172.21.15.59:6789/0 conn(0x7f47040d0600 legacy=0x7f4704162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80039b0 0x7f46e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f46e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 0x7f46e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4704072a20 con 0x7f46e8004800 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f47040720f0 con 0x7f46e80039b0 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.888+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f46e800b9c0 con 0x7f470406dcb0 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f46fb400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f46e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f4708c00640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 0x7f46e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f4703e00640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80039b0 0x7f46e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f46fb400640 1 -- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 msgr2=0x7f46e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f46fb400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f470406dcb0 0x7f46e80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f46fb400640 1 -- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80039b0 msgr2=0x7f46e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f46fb400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80039b0 0x7f46e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f46fb400640 1 -- 172.21.15.59:0/180440935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8003520 con 0x7f46e8004800 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.889+0000 7f4703e00640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80039b0 0x7f46e8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.890+0000 7f46fb400640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f46e8004c80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009ac0 tx=0x7f46e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa85f0f5d50c5fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec036070 con 0x7f46e8004800 2024-04-23T14:38:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec031400 con 0x7f46e8004800 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4703400640 1 -- 172.21.15.59:0/180440935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec009690 con 0x7f46e8004800 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4709600640 1 -- 172.21.15.59:0/180440935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 msgr2=0x7f46e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4709600640 1 --2- 172.21.15.59:0/180440935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f46e8004c80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009ac0 tx=0x7f46e8005830 comp rx=0 tx=0).stop 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4708c00640 1 -- 172.21.15.59:0/180440935 reap_dead start 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4709600640 1 -- 172.21.15.59:0/180440935 shutdown_connections 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4709600640 1 -- 172.21.15.59:0/180440935 >> 172.21.15.59:0/180440935 conn(0x7f470406bdf0 msgr2=0x7f470406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4709600640 1 -- 172.21.15.59:0/180440935 shutdown_connections 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.891+0000 7f4709600640 1 -- 172.21.15.59:0/180440935 wait complete. 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.893+0000 7f4709600640 1 Processor -- start 2024-04-23T14:38:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.893+0000 7f4709600640 1 -- start start 2024-04-23T14:38:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.894+0000 7f4709600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f47041f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.894+0000 7f4709600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47041432a0 0x7f470415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.894+0000 7f4709600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4704143720 0x7f470415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.894+0000 7f4709600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47040d0e70 con 0x7f46e8004800 2024-04-23T14:38:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.894+0000 7f4709600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f470415eaf0 con 0x7f4704143720 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.894+0000 7f4709600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f47040ad170 con 0x7f47041432a0 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f47041f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f47041f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45930/0 (socket says 172.21.15.59:45930) 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 -- 172.21.15.59:0/2683009254 learned_addr learned my addr 172.21.15.59:0/2683009254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f46fb400640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47041432a0 0x7f470415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4708c00640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4704143720 0x7f470415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 -- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47041432a0 msgr2=0x7f470415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47041432a0 0x7f470415ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 -- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4704143720 msgr2=0x7f470415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4704143720 0x7f470415f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4703e00640 1 -- 172.21.15.59:0/2683009254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47041581d0 con 0x7f46e8004800 2024-04-23T14:38:32.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.895+0000 7f4708c00640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4704143720 0x7f470415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.896+0000 7f4703e00640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f47041f1560 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f46f40012e0 tx=0x7f46f400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.896+0000 7f4700c00640 1 -- 172.21.15.59:0/2683009254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4019070 con 0x7f46e8004800 2024-04-23T14:38:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.896+0000 7f4700c00640 1 -- 172.21.15.59:0/2683009254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f4004d60 con 0x7f46e8004800 2024-04-23T14:38:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.896+0000 7f4700c00640 1 -- 172.21.15.59:0/2683009254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f40072d0 con 0x7f46e8004800 2024-04-23T14:38:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.896+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47041584b0 con 0x7f46e8004800 2024-04-23T14:38:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.896+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4704158a70 con 0x7f46e8004800 2024-04-23T14:38:32.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.897+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470404fd90 con 0x7f46e8004800 2024-04-23T14:38:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.898+0000 7f4700c00640 1 -- 172.21.15.59:0/2683009254 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46f4007470 con 0x7f46e8004800 2024-04-23T14:38:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.900+0000 7f4700c00640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46e006c8f0 0x7f46e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.900+0000 7f4700c00640 1 -- 172.21.15.59:0/2683009254 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f46f408baa0 con 0x7f46e8004800 2024-04-23T14:38:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.900+0000 7f46fb400640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46e006c8f0 0x7f46e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.901+0000 7f46fb400640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46e006c8f0 0x7f46e006edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f46ec033000 tx=0x7f46ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:32.904+0000 7f4700c00640 1 -- 172.21.15.59:0/2683009254 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46f405a160 con 0x7f46e8004800 2024-04-23T14:38:33.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.107+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f47040cff50 con 0x7f46e8004800 2024-04-23T14:38:33.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.112+0000 7f4700c00640 1 -- 172.21.15.59:0/2683009254 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f46f4059a60 con 0x7f46e8004800 2024-04-23T14:38:33.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46e006c8f0 msgr2=0x7f46e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46e006c8f0 0x7f46e006edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f46ec033000 tx=0x7f46ec008030 comp rx=0 tx=0).stop 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 msgr2=0x7f47041f1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f47041f1560 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f46f40012e0 tx=0x7f46f400b560 comp rx=0 tx=0).stop 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 shutdown_connections 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46e006c8f0 0x7f46e006edb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47041432a0 0x7f470415ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4704143720 0x7f470415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 --2- 172.21.15.59:0/2683009254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e8004800 0x7f47041f1560 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 >> 172.21.15.59:0/2683009254 conn(0x7f470406bdf0 msgr2=0x7f47040ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:33.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 shutdown_connections 2024-04-23T14:38:33.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.115+0000 7f4709600640 1 -- 172.21.15.59:0/2683009254 wait complete. 2024-04-23T14:38:33.136 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:38:33.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:38:33.249 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:38:33.357 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:38:33.357 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:38:33.357 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:33.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:38:33.418 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:38:33.418 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:33.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:38:33.476 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:38:33.476 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:33.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:38:33.539 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:38:33.539 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:33.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:38:33.597 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:38:33.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:33.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.810+0000 7f1a86200640 1 Processor -- start 2024-04-23T14:38:33.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.810+0000 7f1a86200640 1 -- start start 2024-04-23T14:38:33.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.812+0000 7f1a86200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 0x7f1a800b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.812+0000 7f1a86200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8006ec60 con 0x7f1a8006f7c0 2024-04-23T14:38:33.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.812+0000 7f1a86200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a800b46e0 con 0x7f1a8015b840 2024-04-23T14:38:33.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.812+0000 7f1a86200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a800b49c0 con 0x7f1a8006e610 2024-04-23T14:38:33.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.812+0000 7f1a7f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1a8006f7c0 0x7f1a8006fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46480/0 (socket says 172.21.15.59:46480) 2024-04-23T14:38:33.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.812+0000 7f1a7f400640 1 -- 172.21.15.59:0/2903243947 learned_addr learned my addr 172.21.15.59:0/2903243947 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382680472 0 0) 0x7f1a8006ec60 con 0x7f1a8006f7c0 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a64003520 con 0x7f1a8006f7c0 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7fe00640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 0x7f1a800b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3726793623 0 0) 0x7f1a64003520 con 0x7f1a8006f7c0 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 msgr2=0x7f1a800b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 0x7f1a800b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 >> v1:172.21.15.76:6789/0 conn(0x7f1a8015b840 legacy=0x7f1a8006df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:33.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a800b4ca0 con 0x7f1a8006f7c0 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1a70001670 con 0x7f1a8006f7c0 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 >> v1:172.21.15.59:6789/0 conn(0x7f1a8006f7c0 legacy=0x7f1a8006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a640039e0 0x7f1a64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 0x7f1a64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a64004d40 0x7f1a640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800b49c0 con 0x7f1a640039e0 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6400dd60 con 0x7f1a64004d40 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.813+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6400df60 con 0x7f1a8006e610 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7f400640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a640039e0 0x7f1a64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7ea00640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 0x7f1a64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7f400640 1 -- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 msgr2=0x7f1a64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7f400640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 0x7f1a64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7f400640 1 -- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a64004d40 msgr2=0x7f1a640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7fe00640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a64004d40 0x7f1a640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7f400640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a64004d40 0x7f1a640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7f400640 1 -- 172.21.15.59:0/2903243947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a64003520 con 0x7f1a640039e0 2024-04-23T14:38:33.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7ea00640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a8006e610 0x7f1a64004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:33.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.814+0000 7f1a7fe00640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a64004d40 0x7f1a640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:33.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.815+0000 7f1a7f400640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a640039e0 0x7f1a64003e00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c210 tx=0x7f1a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193f4c3d304fddd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70007d30 con 0x7f1a640039e0 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a70005ad0 con 0x7f1a640039e0 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a7e000640 1 -- 172.21.15.59:0/2903243947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70007070 con 0x7f1a640039e0 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a86200640 1 -- 172.21.15.59:0/2903243947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a640039e0 msgr2=0x7f1a64003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a86200640 1 --2- 172.21.15.59:0/2903243947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a640039e0 0x7f1a64003e00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c210 tx=0x7f1a64005830 comp rx=0 tx=0).stop 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a7fe00640 1 -- 172.21.15.59:0/2903243947 reap_dead start 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a86200640 1 -- 172.21.15.59:0/2903243947 shutdown_connections 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a86200640 1 -- 172.21.15.59:0/2903243947 >> 172.21.15.59:0/2903243947 conn(0x7f1a80067ea0 msgr2=0x7f1a8006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a86200640 1 -- 172.21.15.59:0/2903243947 shutdown_connections 2024-04-23T14:38:33.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.816+0000 7f1a86200640 1 -- 172.21.15.59:0/2903243947 wait complete. 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.818+0000 7f1a86200640 1 Processor -- start 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.818+0000 7f1a86200640 1 -- start start 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.819+0000 7f1a86200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a640039e0 0x7f1a800b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.819+0000 7f1a86200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a800d04e0 0x7f1a80149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.819+0000 7f1a86200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a8015bc30 0x7f1a80153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a86200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800b5620 con 0x7f1a8015bc30 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a86200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a800b54a0 con 0x7f1a640039e0 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a86200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a800b57a0 con 0x7f1a800d04e0 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a8015bc30 0x7f1a80153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a8015bc30 0x7f1a80153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45960/0 (socket says 172.21.15.59:45960) 2024-04-23T14:38:33.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 -- 172.21.15.59:0/3017730915 learned_addr learned my addr 172.21.15.59:0/3017730915 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7f400640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a640039e0 0x7f1a800b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7ea00640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a800d04e0 0x7f1a80149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 -- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a800d04e0 msgr2=0x7f1a80149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a800d04e0 0x7f1a80149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 -- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a640039e0 msgr2=0x7f1a800b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a640039e0 0x7f1a800b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7fe00640 1 -- 172.21.15.59:0/3017730915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a800b0150 con 0x7f1a8015bc30 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.820+0000 7f1a7ea00640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a800d04e0 0x7f1a80149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.821+0000 7f1a7fe00640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a8015bc30 0x7f1a80153d80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1a800b0000 tx=0x7f1a78009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.821+0000 7f1a77400640 1 -- 172.21.15.59:0/3017730915 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78019070 con 0x7f1a8015bc30 2024-04-23T14:38:33.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.821+0000 7f1a77400640 1 -- 172.21.15.59:0/3017730915 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a78007ce0 con 0x7f1a8015bc30 2024-04-23T14:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.821+0000 7f1a77400640 1 -- 172.21.15.59:0/3017730915 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78014640 con 0x7f1a8015bc30 2024-04-23T14:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.821+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a800b0a00 con 0x7f1a8015bc30 2024-04-23T14:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.821+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a800ce990 con 0x7f1a8015bc30 2024-04-23T14:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.822+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a800b0f20 con 0x7f1a8015bc30 2024-04-23T14:38:33.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.824+0000 7f1a77400640 1 -- 172.21.15.59:0/3017730915 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a7801c070 con 0x7f1a8015bc30 2024-04-23T14:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.825+0000 7f1a77400640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5c06c920 0x7f1a5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.825+0000 7f1a77400640 1 -- 172.21.15.59:0/3017730915 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f1a7808b240 con 0x7f1a8015bc30 2024-04-23T14:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.825+0000 7f1a7f400640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5c06c920 0x7f1a5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.826+0000 7f1a7f400640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5c06c920 0x7f1a5c06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c210 tx=0x7f1a7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:33.829+0000 7f1a77400640 1 -- 172.21.15.59:0/3017730915 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a780599c0 con 0x7f1a8015bc30 2024-04-23T14:38:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.078+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a80068a80 con 0x7f1a8015bc30 2024-04-23T14:38:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.078+0000 7f1a77400640 1 -- 172.21.15.59:0/3017730915 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+3866 (secure 0 0 0) 0x7f1a780592c0 con 0x7f1a8015bc30 2024-04-23T14:38:34.080 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:34.080 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3181614909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3181614909},{"type":"v1","addr":"172.21.15.59:6837","nonce":3181614909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2600552971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2600552971},{"type":"v1","addr":"172.21.15.76:6835","nonce":2600552971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/4226919297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4226919297},{"type":"v1","addr":"172.21.15.76:6834","nonce":4226919297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":7446,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/710136321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":710136321},{"type":"v1","addr":"172.21.15.59:6836","nonce":710136321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[]} 2024-04-23T14:38:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 53 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.081+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5c06c920 msgr2=0x7f1a5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.081+0000 7f1a86200640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5c06c920 0x7f1a5c06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c210 tx=0x7f1a7002e000 comp rx=0 tx=0).stop 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.081+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a8015bc30 msgr2=0x7f1a80153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.081+0000 7f1a86200640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a8015bc30 0x7f1a80153d80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1a800b0000 tx=0x7f1a78009d50 comp rx=0 tx=0).stop 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 shutdown_connections 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5c06c920 0x7f1a5c06ede0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a800d04e0 0x7f1a80149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a8015bc30 0x7f1a80153d80 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 --2- 172.21.15.59:0/3017730915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a640039e0 0x7f1a800b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 >> 172.21.15.59:0/3017730915 conn(0x7f1a80067ea0 msgr2=0x7f1a800b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:34.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 shutdown_connections 2024-04-23T14:38:34.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.082+0000 7f1a86200640 1 -- 172.21.15.59:0/3017730915 wait complete. 2024-04-23T14:38:34.104 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:38:34.104 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:38:34.105 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:38:34.105 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:38:34.105 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:38:34.105 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:38:34.105 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:38:34.106 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:38:34.106 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:38:34.106 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:38:34.106 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:38:34.106 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:38:34.106 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:38:34.107 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:38:34.107 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:38:34.107 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:38:34.127 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:38:34.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:38:34.130 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:38:34.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:38:34.140 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:38:34.140 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:38:34.141 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:38:34.141 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:38:34.178 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:38:34.179 DEBUG:teuthology.parallel:result is None 2024-04-23T14:38:34.185 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:38:34.185 DEBUG:teuthology.parallel:result is None 2024-04-23T14:38:34.188 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:38:34.189 DEBUG:teuthology.parallel:result is None 2024-04-23T14:38:34.196 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:38:34.196 DEBUG:teuthology.parallel:result is None 2024-04-23T14:38:34.197 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:38:34.331 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:38:34.332 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:38:34.348 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:38:34.349 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:38:34.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.379+0000 7f2d76c00640 1 Processor -- start 2024-04-23T14:38:34.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.379+0000 7f2d76c00640 1 -- start start 2024-04-23T14:38:34.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.380+0000 7f2d76c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2d700d1800 0x7f2d700d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.380+0000 7f2d76c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d700cf3c0 con 0x7f2d700d3480 2024-04-23T14:38:34.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.380+0000 7f2d76c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d700cf640 con 0x7f2d7015e9c0 2024-04-23T14:38:34.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.380+0000 7f2d76c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2d700cf900 con 0x7f2d700d1800 2024-04-23T14:38:34.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d6fe00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2d7015e9c0 0x7f2d700d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:59894/0 (socket says 172.21.15.59:59894) 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d6f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2d700d3480 0x7f2d700d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46522/0 (socket says 172.21.15.59:46522) 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d6fe00640 1 -- 172.21.15.59:0/1082733475 learned_addr learned my addr 172.21.15.59:0/1082733475 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d74c00640 1 --2- 172.21.15.59:0/1082733475 >> v2:172.21.15.76:3300/0 conn(0x7f2d700d1800 0x7f2d700d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d6ea00640 1 -- 172.21.15.59:0/1082733475 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476176477 0 0) 0x7f2d700cf640 con 0x7f2d7015e9c0 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d6ea00640 1 -- 172.21.15.59:0/1082733475 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d54003520 con 0x7f2d7015e9c0 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d6ea00640 1 -- 172.21.15.59:0/1082733475 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186676936 0 0) 0x7f2d5c0011d0 con 0x7f2d700d3480 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d6ea00640 1 -- 172.21.15.59:0/1082733475 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d700cf640 con 0x7f2d700d3480 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d74c00640 1 -- 172.21.15.59:0/1082733475 >> v1:172.21.15.76:6789/0 conn(0x7f2d7015e9c0 legacy=0x7f2d700d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d74c00640 1 -- 172.21.15.59:0/1082733475 >> v1:172.21.15.59:6789/0 conn(0x7f2d700d3480 legacy=0x7f2d700d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.381+0000 7f2d74c00640 1 -- 172.21.15.59:0/1082733475 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d700cfbe0 con 0x7f2d700d1800 2024-04-23T14:38:34.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.382+0000 7f2d74c00640 1 --2- 172.21.15.59:0/1082733475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d700d1800 0x7f2d700d03e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009d90 tx=0x7f2d6802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4452e7a02d2133 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:34.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d6ea00640 1 -- 172.21.15.59:0/1082733475 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68007740 con 0x7f2d700d1800 2024-04-23T14:38:34.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d6ea00640 1 -- 172.21.15.59:0/1082733475 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d680078e0 con 0x7f2d700d1800 2024-04-23T14:38:34.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d6ea00640 1 -- 172.21.15.59:0/1082733475 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68007c90 con 0x7f2d700d1800 2024-04-23T14:38:34.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d76c00640 1 -- 172.21.15.59:0/1082733475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d700d1800 msgr2=0x7f2d700d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1082733475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d700d1800 0x7f2d700d03e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009d90 tx=0x7f2d6802a660 comp rx=0 tx=0).stop 2024-04-23T14:38:34.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d76c00640 1 -- 172.21.15.59:0/1082733475 shutdown_connections 2024-04-23T14:38:34.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1082733475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d700d1800 0x7f2d700d03e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.383+0000 7f2d76c00640 1 -- 172.21.15.59:0/1082733475 >> 172.21.15.59:0/1082733475 conn(0x7f2d7008bf90 msgr2=0x7f2d700d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:34.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.384+0000 7f2d76c00640 1 -- 172.21.15.59:0/1082733475 shutdown_connections 2024-04-23T14:38:34.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.384+0000 7f2d76c00640 1 -- 172.21.15.59:0/1082733475 wait complete. 2024-04-23T14:38:34.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.385+0000 7f2d76c00640 1 Processor -- start 2024-04-23T14:38:34.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.385+0000 7f2d76c00640 1 -- start start 2024-04-23T14:38:34.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.386+0000 7f2d76c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d700d1800 0x7f2d7014bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.386+0000 7f2d76c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d700d3480 0x7f2d700b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.386+0000 7f2d76c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015e9c0 0x7f2d700b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.386+0000 7f2d76c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7006cd10 con 0x7f2d700d1800 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.386+0000 7f2d76c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7006cb90 con 0x7f2d7015e9c0 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.386+0000 7f2d76c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7006ce90 con 0x7f2d700d3480 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d700d1800 0x7f2d7014bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d700d1800 0x7f2d7014bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46000/0 (socket says 172.21.15.59:46000) 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 -- 172.21.15.59:0/1016012392 learned_addr learned my addr 172.21.15.59:0/1016012392 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d74c00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015e9c0 0x7f2d700b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6f400640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d700d3480 0x7f2d700b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 -- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d700d3480 msgr2=0x7f2d700b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d700d3480 0x7f2d700b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 -- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015e9c0 msgr2=0x7f2d700b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015e9c0 0x7f2d700b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 -- 172.21.15.59:0/1016012392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d700cbad0 con 0x7f2d700d1800 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.387+0000 7f2d6fe00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d700d1800 0x7f2d7014bf20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c420 tx=0x7f2d6000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.388+0000 7f2d67e00640 1 -- 172.21.15.59:0/1016012392 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60017070 con 0x7f2d700d1800 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.388+0000 7f2d67e00640 1 -- 172.21.15.59:0/1016012392 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d60005d30 con 0x7f2d700d1800 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.388+0000 7f2d67e00640 1 -- 172.21.15.59:0/1016012392 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600072f0 con 0x7f2d700d1800 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.388+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d700cbdb0 con 0x7f2d700d1800 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.388+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d700b5970 con 0x7f2d700d1800 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.389+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d700cb860 con 0x7f2d700d1800 2024-04-23T14:38:34.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.390+0000 7f2d67e00640 1 -- 172.21.15.59:0/1016012392 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d6001a070 con 0x7f2d700d1800 2024-04-23T14:38:34.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.391+0000 7f2d67e00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d5006c940 0x7f2d5006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.391+0000 7f2d67e00640 1 -- 172.21.15.59:0/1016012392 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5029+0+0 (secure 0 0 0) 0x7f2d6008ac50 con 0x7f2d700d1800 2024-04-23T14:38:34.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.392+0000 7f2d6f400640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d5006c940 0x7f2d5006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.392+0000 7f2d6f400640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d5006c940 0x7f2d5006ee00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c007fd0 tx=0x7f2d5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:34.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.396+0000 7f2d67e00640 1 -- 172.21.15.59:0/1016012392 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d60058d40 con 0x7f2d700d1800 2024-04-23T14:38:34.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.621+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2d70068a80 con 0x7f2d700d1800 2024-04-23T14:38:34.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.677+0000 7f2d67e00640 1 -- 172.21.15.59:0/1016012392 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) ==== 97+0+0 (secure 0 0 0) 0x7f2d6005d040 con 0x7f2d700d1800 2024-04-23T14:38:34.678 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-23T14:38:34.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.679+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d5006c940 msgr2=0x7f2d5006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.679+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d5006c940 0x7f2d5006ee00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c007fd0 tx=0x7f2d5c009040 comp rx=0 tx=0).stop 2024-04-23T14:38:34.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.679+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d700d1800 msgr2=0x7f2d7014bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d700d1800 0x7f2d7014bf20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c420 tx=0x7f2d6000c8e0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 shutdown_connections 2024-04-23T14:38:34.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d5006c940 0x7f2d5006ee00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d700d3480 0x7f2d700b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015e9c0 0x7f2d700b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 --2- 172.21.15.59:0/1016012392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d700d1800 0x7f2d7014bf20 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 >> 172.21.15.59:0/1016012392 conn(0x7f2d7008bf90 msgr2=0x7f2d700b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:34.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 shutdown_connections 2024-04-23T14:38:34.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.680+0000 7f2d76c00640 1 -- 172.21.15.59:0/1016012392 wait complete. 2024-04-23T14:38:34.702 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:38:34.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.909+0000 7f1e4d600640 1 Processor -- start 2024-04-23T14:38:34.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.909+0000 7f1e4d600640 1 -- start start 2024-04-23T14:38:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.911+0000 7f1e4d600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e4815c3d0 0x7f1e480cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.911+0000 7f1e4d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e480b4860 0x7f1e480cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.911+0000 7f1e4d600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e48160210 con 0x7f1e480cd590 2024-04-23T14:38:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.911+0000 7f1e4d600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1e481604f0 con 0x7f1e4815c3d0 2024-04-23T14:38:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.911+0000 7f1e4d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1e481607d0 con 0x7f1e480b4860 2024-04-23T14:38:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e47e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e4815c3d0 0x7f1e480cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e4cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e480b4860 0x7f1e480cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e4cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e480b4860 0x7f1e480cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:41320/0 (socket says 172.21.15.59:41320) 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e4cc00640 1 -- 172.21.15.59:0/3208415258 learned_addr learned my addr 172.21.15.59:0/3208415258 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e47e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e4815c3d0 0x7f1e480cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:58830/0 (socket says 172.21.15.59:58830) 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e47e00640 1 -- 172.21.15.59:0/3208415258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e480b4860 msgr2=0x7f1e480cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e47e00640 1 --2- 172.21.15.59:0/3208415258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e480b4860 0x7f1e480cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e47e00640 1 -- 172.21.15.59:0/3208415258 >> v1:172.21.15.76:6789/0 conn(0x7f1e480cd590 legacy=0x7f1e480b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.912+0000 7f1e47e00640 1 -- 172.21.15.59:0/3208415258 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e48160ab0 con 0x7f1e4815c3d0 2024-04-23T14:38:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.914+0000 7f1e47e00640 1 --2- 172.21.15.59:0/3208415258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e4815c3d0 0x7f1e480cd050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009d90 tx=0x7f1e3802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104c01e29aefc557 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:34.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.914+0000 7f1e47400640 1 -- 172.21.15.59:0/3208415258 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38007740 con 0x7f1e4815c3d0 2024-04-23T14:38:34.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.914+0000 7f1e47400640 1 -- 172.21.15.59:0/3208415258 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e380078e0 con 0x7f1e4815c3d0 2024-04-23T14:38:34.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.914+0000 7f1e47400640 1 -- 172.21.15.59:0/3208415258 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38007cb0 con 0x7f1e4815c3d0 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 -- 172.21.15.59:0/3208415258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e4815c3d0 msgr2=0x7f1e480cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 --2- 172.21.15.59:0/3208415258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e4815c3d0 0x7f1e480cd050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009d90 tx=0x7f1e3802a660 comp rx=0 tx=0).stop 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 -- 172.21.15.59:0/3208415258 shutdown_connections 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 --2- 172.21.15.59:0/3208415258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e480b4860 0x7f1e480cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 --2- 172.21.15.59:0/3208415258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e4815c3d0 0x7f1e480cd050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 -- 172.21.15.59:0/3208415258 >> 172.21.15.59:0/3208415258 conn(0x7f1e480b21a0 msgr2=0x7f1e480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 -- 172.21.15.59:0/3208415258 shutdown_connections 2024-04-23T14:38:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.915+0000 7f1e4d600640 1 -- 172.21.15.59:0/3208415258 wait complete. 2024-04-23T14:38:34.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.917+0000 7f1e4d600640 1 Processor -- start 2024-04-23T14:38:34.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.917+0000 7f1e4d600640 1 -- start start 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e4d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e480b4860 0x7f1e480c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e4d600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e480cd590 0x7f1e4815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e4d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e4815c3d0 0x7f1e481f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e4d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4815c930 con 0x7f1e480b4860 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e4d600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4815c7b0 con 0x7f1e480cd590 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e4d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1e480cbd10 con 0x7f1e4815c3d0 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e480b4860 0x7f1e480c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e47e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e480b4860 0x7f1e480c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46012/0 (socket says 172.21.15.59:46012) 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.918+0000 7f1e47e00640 1 -- 172.21.15.59:0/1051697248 learned_addr learned my addr 172.21.15.59:0/1051697248 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e4cc00640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e4815c3d0 0x7f1e481f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e3f400640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e480cd590 0x7f1e4815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e47e00640 1 -- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e4815c3d0 msgr2=0x7f1e481f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e47e00640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e4815c3d0 0x7f1e481f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e47e00640 1 -- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e480cd590 msgr2=0x7f1e4815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:34.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e47e00640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e480cd590 0x7f1e4815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:34.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e47e00640 1 -- 172.21.15.59:0/1051697248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4815fb60 con 0x7f1e480b4860 2024-04-23T14:38:34.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e3f400640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e480cd590 0x7f1e4815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:34.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e47e00640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e480b4860 0x7f1e480c5e60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009d90 tx=0x7f1e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:34.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e44c00640 1 -- 172.21.15.59:0/1051697248 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3803f070 con 0x7f1e480b4860 2024-04-23T14:38:34.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.919+0000 7f1e44c00640 1 -- 172.21.15.59:0/1051697248 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38031bc0 con 0x7f1e480b4860 2024-04-23T14:38:34.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.920+0000 7f1e44c00640 1 -- 172.21.15.59:0/1051697248 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38044460 con 0x7f1e480b4860 2024-04-23T14:38:34.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.920+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e480c7e20 con 0x7f1e480b4860 2024-04-23T14:38:34.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.920+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481f6250 con 0x7f1e480b4860 2024-04-23T14:38:34.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.921+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4804fd90 con 0x7f1e480b4860 2024-04-23T14:38:34.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.922+0000 7f1e44c00640 1 -- 172.21.15.59:0/1051697248 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e38044600 con 0x7f1e480b4860 2024-04-23T14:38:34.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.923+0000 7f1e44c00640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e2406c940 0x7f1e2406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:34.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.923+0000 7f1e44c00640 1 -- 172.21.15.59:0/1051697248 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f1e380acfa0 con 0x7f1e480b4860 2024-04-23T14:38:34.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.923+0000 7f1e3f400640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e2406c940 0x7f1e2406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:34.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.924+0000 7f1e3f400640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e2406c940 0x7f1e2406ee00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1e4815bdc0 tx=0x7f1e30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:34.927+0000 7f1e44c00640 1 -- 172.21.15.59:0/1051697248 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e4815fb60 con 0x7f1e480b4860 2024-04-23T14:38:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.200+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1e4806cbe0 con 0x7f1e480b4860 2024-04-23T14:38:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.202+0000 7f1e44c00640 1 -- 172.21.15.59:0/1051697248 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f1e3807b700 con 0x7f1e480b4860 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A==", 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.203 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA==", 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.204 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw==", 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg==", 2024-04-23T14:38:35.205 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow==", 2024-04-23T14:38:35.206 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ==", 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ==", 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-23T14:38:35.208 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g==", 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA==", 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA==", 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA==", 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ==", 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ==", 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q==", 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg==", 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-23T14:38:35.214 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg==", 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmEteHAhAAwfUHmvRmEMcPIphwikq6sw==", 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA==", 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ==", 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:38:35.217 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:38:35.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e2406c940 msgr2=0x7f1e2406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e2406c940 0x7f1e2406ee00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1e4815bdc0 tx=0x7f1e30009040 comp rx=0 tx=0).stop 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e480b4860 msgr2=0x7f1e480c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e480b4860 0x7f1e480c5e60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009d90 tx=0x7f1e38004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 shutdown_connections 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e2406c940 0x7f1e2406ee00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e4815c3d0 0x7f1e481f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e480cd590 0x7f1e4815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 --2- 172.21.15.59:0/1051697248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e480b4860 0x7f1e480c5e60 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 >> 172.21.15.59:0/1051697248 conn(0x7f1e480b21a0 msgr2=0x7f1e480b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.204+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 shutdown_connections 2024-04-23T14:38:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.205+0000 7f1e4d600640 1 -- 172.21.15.59:0/1051697248 wait complete. 2024-04-23T14:38:35.225 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.427+0000 7fb3b4800640 1 Processor -- start 2024-04-23T14:38:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.427+0000 7fb3b4800640 1 -- start start 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.429+0000 7fb3b4800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb3ac1454e0 0x7fb3ac145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.429+0000 7fb3b4800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 0x7fb3ac0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.429+0000 7fb3b4800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ac0cd520 con 0x7fb3ac0b3e40 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.429+0000 7fb3b4800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb3ac0cddc0 con 0x7fb3ac1454e0 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.429+0000 7fb3b4800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac0cd800 con 0x7fb3ac0cd020 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.429+0000 7fb3b2800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb3ac0b3e40 0x7fb3ac160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46532/0 (socket says 172.21.15.59:46532) 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.429+0000 7fb3b2800640 1 -- 172.21.15.59:0/2385551630 learned_addr learned my addr 172.21.15.59:0/2385551630 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b1e00640 1 --2- 172.21.15.59:0/2385551630 >> v2:172.21.15.76:3300/0 conn(0x7fb3ac1454e0 0x7fb3ac145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b1400640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 0x7fb3ac0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778042152 0 0) 0x7fb3ac0cd520 con 0x7fb3ac0b3e40 2024-04-23T14:38:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb394003520 con 0x7fb3ac0b3e40 2024-04-23T14:38:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 396109156 0 0) 0x7fb394003520 con 0x7fb3ac0b3e40 2024-04-23T14:38:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 msgr2=0x7fb3ac0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 0x7fb3ac0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 >> v2:172.21.15.76:3300/0 conn(0x7fb3ac1454e0 msgr2=0x7fb3ac145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 --2- 172.21.15.59:0/2385551630 >> v2:172.21.15.76:3300/0 conn(0x7fb3ac1454e0 0x7fb3ac145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac0cdae0 con 0x7fb3ac0b3e40 2024-04-23T14:38:35.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fb3a80019f0 con 0x7fb3ac0b3e40 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 >> v1:172.21.15.59:6789/0 conn(0x7fb3ac0b3e40 legacy=0x7fb3ac160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 0x7fb3940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.430+0000 7fb3b0a00640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb394003f80 0x7fb394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b0a00640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb394004d40 0x7fb3940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0cddc0 con 0x7fb394004d40 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0cd800 con 0x7fb394003f80 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb39400ba70 con 0x7fb3ac0cd020 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b2800640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb394004d40 0x7fb3940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b1e00640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 0x7fb3940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b1400640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb394003f80 0x7fb394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b2800640 1 -- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 msgr2=0x7fb3940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b2800640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac0cd020 0x7fb3940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b2800640 1 -- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb394003f80 msgr2=0x7fb394004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b2800640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb394003f80 0x7fb394004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.431+0000 7fb3b2800640 1 -- 172.21.15.59:0/2385551630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb394003520 con 0x7fb394004d40 2024-04-23T14:38:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.432+0000 7fb3b2800640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb394004d40 0x7fb3940051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800a7d0 tx=0x7fb394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c771ab45d1836fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800a3f0 con 0x7fb394004d40 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a802f070 con 0x7fb394004d40 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b0a00640 1 -- 172.21.15.59:0/2385551630 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a802bc90 con 0x7fb394004d40 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b4800640 1 -- 172.21.15.59:0/2385551630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb394004d40 msgr2=0x7fb3940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b4800640 1 --2- 172.21.15.59:0/2385551630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb394004d40 0x7fb3940051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800a7d0 tx=0x7fb394005830 comp rx=0 tx=0).stop 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b2800640 1 -- 172.21.15.59:0/2385551630 reap_dead start 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b4800640 1 -- 172.21.15.59:0/2385551630 shutdown_connections 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b4800640 1 -- 172.21.15.59:0/2385551630 >> 172.21.15.59:0/2385551630 conn(0x7fb3ac067ea0 msgr2=0x7fb3ac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b4800640 1 -- 172.21.15.59:0/2385551630 shutdown_connections 2024-04-23T14:38:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.433+0000 7fb3b4800640 1 -- 172.21.15.59:0/2385551630 wait complete. 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.435+0000 7fb3b4800640 1 Processor -- start 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.435+0000 7fb3b4800640 1 -- start start 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.436+0000 7fb3b4800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3ac0aeb30 0x7fb3ac1f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.436+0000 7fb3b4800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac1f8220 0x7fb3ac15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.436+0000 7fb3b4800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ac0aef10 0x7fb3ac15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.436+0000 7fb3b4800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac146270 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.436+0000 7fb3b4800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac1460f0 con 0x7fb3ac0aef10 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.436+0000 7fb3b4800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac161020 con 0x7fb3ac1f8220 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3ac0aeb30 0x7fb3ac1f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3ac0aeb30 0x7fb3ac1f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46028/0 (socket says 172.21.15.59:46028) 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 -- 172.21.15.59:0/1524282699 learned_addr learned my addr 172.21.15.59:0/1524282699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b2800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ac0aef10 0x7fb3ac15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1400640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac1f8220 0x7fb3ac15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 -- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac1f8220 msgr2=0x7fb3ac15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac1f8220 0x7fb3ac15b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 -- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ac0aef10 msgr2=0x7fb3ac15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ac0aef10 0x7fb3ac15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 -- 172.21.15.59:0/1524282699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac1f87a0 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b2800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ac0aef10 0x7fb3ac15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.437+0000 7fb3b1e00640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3ac0aeb30 0x7fb3ac1f7ce0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00116d0 tx=0x7fb3a0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.438+0000 7fb3a6000640 1 -- 172.21.15.59:0/1524282699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000f9c0 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.438+0000 7fb3a6000640 1 -- 172.21.15.59:0/1524282699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a000fb60 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.438+0000 7fb3a6000640 1 -- 172.21.15.59:0/1524282699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0015460 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.438+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac1f8a80 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.438+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac13bd70 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.439+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ac068a80 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.440+0000 7fb3a6000640 1 -- 172.21.15.59:0/1524282699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3a0015600 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.441+0000 7fb3a6000640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb39006c8f0 0x7fb39006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.441+0000 7fb3a6000640 1 -- 172.21.15.59:0/1524282699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fb3a008be50 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.441+0000 7fb3b1400640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb39006c8f0 0x7fb39006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.442+0000 7fb3b1400640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb39006c8f0 0x7fb39006edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac15f040 tx=0x7fb39c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:35.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.445+0000 7fb3a6000640 1 -- 172.21.15.59:0/1524282699 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3a0019030 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.649+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3ac160030 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.651+0000 7fb3a6000640 1 -- 172.21.15.59:0/1524282699 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+12832 (secure 0 0 0) 0x7fb3a005a530 con 0x7fb3ac0aeb30 2024-04-23T14:38:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:35.652 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":79,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:38:34.688851+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:35.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.653+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb39006c8f0 msgr2=0x7fb39006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.653+0000 7fb3b4800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb39006c8f0 0x7fb39006edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac15f040 tx=0x7fb39c008040 comp rx=0 tx=0).stop 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.653+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3ac0aeb30 msgr2=0x7fb3ac1f7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.653+0000 7fb3b4800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3ac0aeb30 0x7fb3ac1f7ce0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00116d0 tx=0x7fb3a0011b90 comp rx=0 tx=0).stop 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.653+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 shutdown_connections 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.653+0000 7fb3b4800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb39006c8f0 0x7fb39006edb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.654+0000 7fb3b4800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3ac1f8220 0x7fb3ac15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.654+0000 7fb3b4800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ac0aef10 0x7fb3ac15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.654+0000 7fb3b4800640 1 --2- 172.21.15.59:0/1524282699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3ac0aeb30 0x7fb3ac1f7ce0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.654+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 >> 172.21.15.59:0/1524282699 conn(0x7fb3ac067ea0 msgr2=0x7fb3ac0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.654+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 shutdown_connections 2024-04-23T14:38:35.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.654+0000 7fb3b4800640 1 -- 172.21.15.59:0/1524282699 wait complete. 2024-04-23T14:38:35.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:38:35.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:38:35.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.857+0000 7f8140a00640 1 Processor -- start 2024-04-23T14:38:35.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.857+0000 7f8140a00640 1 -- start start 2024-04-23T14:38:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f8140a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f813c0d0600 0x7f813c0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f8140a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c071e10 con 0x7f813c0d09e0 2024-04-23T14:38:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f8140a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0720f0 con 0x7f813c06dcb0 2024-04-23T14:38:35.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f8140a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f813c072a20 con 0x7f813c0d0600 2024-04-23T14:38:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f813a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f813c0d0600 0x7f813c0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f813a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f813c0d0600 0x7f813c0ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46030/0 (socket says 172.21.15.59:46030) 2024-04-23T14:38:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f813b200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f813c0d09e0 0x7f813c06e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46536/0 (socket says 172.21.15.59:46536) 2024-04-23T14:38:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.859+0000 7f813a800640 1 -- 172.21.15.59:0/856068303 learned_addr learned my addr 172.21.15.59:0/856068303 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.860+0000 7f813a800640 1 -- 172.21.15.59:0/856068303 >> v1:172.21.15.76:6789/0 conn(0x7f813c06dcb0 legacy=0x7f813c162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.860+0000 7f813a800640 1 -- 172.21.15.59:0/856068303 >> v1:172.21.15.59:6789/0 conn(0x7f813c0d09e0 legacy=0x7f813c06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.860+0000 7f813a800640 1 -- 172.21.15.59:0/856068303 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0723d0 con 0x7f813c0d0600 2024-04-23T14:38:35.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.861+0000 7f813a800640 1 --2- 172.21.15.59:0/856068303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c0d0600 0x7f813c0ad170 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f812c009870 tx=0x7f812c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22348f70c8d46e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:35.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.861+0000 7f8139e00640 1 -- 172.21.15.59:0/856068303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c037070 con 0x7f813c0d0600 2024-04-23T14:38:35.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.861+0000 7f8139e00640 1 -- 172.21.15.59:0/856068303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812c032440 con 0x7f813c0d0600 2024-04-23T14:38:35.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.861+0000 7f8139e00640 1 -- 172.21.15.59:0/856068303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c032810 con 0x7f813c0d0600 2024-04-23T14:38:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.862+0000 7f8140a00640 1 -- 172.21.15.59:0/856068303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c0d0600 msgr2=0x7f813c0ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.862+0000 7f8140a00640 1 --2- 172.21.15.59:0/856068303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c0d0600 0x7f813c0ad170 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f812c009870 tx=0x7f812c02a2a0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.862+0000 7f8140a00640 1 -- 172.21.15.59:0/856068303 shutdown_connections 2024-04-23T14:38:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.862+0000 7f8140a00640 1 --2- 172.21.15.59:0/856068303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c0d0600 0x7f813c0ad170 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.862+0000 7f8140a00640 1 -- 172.21.15.59:0/856068303 >> 172.21.15.59:0/856068303 conn(0x7f813c06bdf0 msgr2=0x7f813c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.862+0000 7f8140a00640 1 -- 172.21.15.59:0/856068303 shutdown_connections 2024-04-23T14:38:35.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.862+0000 7f8140a00640 1 -- 172.21.15.59:0/856068303 wait complete. 2024-04-23T14:38:35.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.863+0000 7f8140a00640 1 Processor -- start 2024-04-23T14:38:35.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.864+0000 7f8140a00640 1 -- start start 2024-04-23T14:38:35.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8140a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c06dcb0 0x7f813c1e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8140a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f813c0d0600 0x7f813c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8140a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c0d09e0 0x7f813c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8140a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f813c0d0e90 con 0x7f813c06dcb0 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8140a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f813c15eaf0 con 0x7f813c0d09e0 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8140a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f813c06ecd0 con 0x7f813c0d0600 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c06dcb0 0x7f813c1e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c06dcb0 0x7f813c1e8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46032/0 (socket says 172.21.15.59:46032) 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 -- 172.21.15.59:0/2857862267 learned_addr learned my addr 172.21.15.59:0/2857862267 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813b200640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c0d09e0 0x7f813c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 -- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f813c0d0600 msgr2=0x7f813c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8133e00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f813c0d0600 0x7f813c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f813c0d0600 0x7f813c159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 -- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c0d09e0 msgr2=0x7f813c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c0d09e0 0x7f813c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f813a800640 1 -- 172.21.15.59:0/2857862267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0cb880 con 0x7f813c06dcb0 2024-04-23T14:38:35.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.865+0000 7f8133e00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f813c0d0600 0x7f813c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:35.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.866+0000 7f813a800640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c06dcb0 0x7f813c1e8250 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f812c000c00 tx=0x7f812c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:35.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.866+0000 7f8132a00640 1 -- 172.21.15.59:0/2857862267 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c037040 con 0x7f813c06dcb0 2024-04-23T14:38:35.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.866+0000 7f8132a00640 1 -- 172.21.15.59:0/2857862267 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812c042070 con 0x7f813c06dcb0 2024-04-23T14:38:35.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.866+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813c0cbb60 con 0x7f813c06dcb0 2024-04-23T14:38:35.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.866+0000 7f8132a00640 1 -- 172.21.15.59:0/2857862267 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c030bd0 con 0x7f813c06dcb0 2024-04-23T14:38:35.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.866+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813c0cc120 con 0x7f813c06dcb0 2024-04-23T14:38:35.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.867+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813c04fd90 con 0x7f813c06dcb0 2024-04-23T14:38:35.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.868+0000 7f8132a00640 1 -- 172.21.15.59:0/2857862267 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f812c047460 con 0x7f813c06dcb0 2024-04-23T14:38:35.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.869+0000 7f8132a00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f811806c8f0 0x7f811806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:35.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.869+0000 7f8132a00640 1 -- 172.21.15.59:0/2857862267 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f812c0ac440 con 0x7f813c06dcb0 2024-04-23T14:38:35.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.869+0000 7f8133e00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f811806c8f0 0x7f811806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:35.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.870+0000 7f8133e00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f811806c8f0 0x7f811806edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f813c161a30 tx=0x7f8124009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:35.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:35.874+0000 7f8132a00640 1 -- 172.21.15.59:0/2857862267 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f812c07abb0 con 0x7f813c06dcb0 2024-04-23T14:38:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.069+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f813c15ad20 con 0x7f813c06dcb0 2024-04-23T14:38:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.069+0000 7f8132a00640 1 -- 172.21.15.59:0/2857862267 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f812c07a4b0 con 0x7f813c06dcb0 2024-04-23T14:38:36.071 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:38:36.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f811806c8f0 msgr2=0x7f811806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f811806c8f0 0x7f811806edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f813c161a30 tx=0x7f8124009040 comp rx=0 tx=0).stop 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c06dcb0 msgr2=0x7f813c1e8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c06dcb0 0x7f813c1e8250 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f812c000c00 tx=0x7f812c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 shutdown_connections 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f811806c8f0 0x7f811806edb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f813c0d0600 0x7f813c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f813c0d09e0 0x7f813c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 --2- 172.21.15.59:0/2857862267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f813c06dcb0 0x7f813c1e8250 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 >> 172.21.15.59:0/2857862267 conn(0x7f813c06bdf0 msgr2=0x7f813c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.072+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 shutdown_connections 2024-04-23T14:38:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.073+0000 7f8140a00640 1 -- 172.21.15.59:0/2857862267 wait complete. 2024-04-23T14:38:36.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:38:36.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.282+0000 7fa277e00640 1 Processor -- start 2024-04-23T14:38:36.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.282+0000 7fa277e00640 1 -- start start 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.283+0000 7fa277e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780cd760 0x7fa2780b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.283+0000 7fa277e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2780b4690 0x7fa2780cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.283+0000 7fa277e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa278160210 con 0x7fa27815c3d0 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.283+0000 7fa277e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa2781604f0 con 0x7fa2780b4690 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.283+0000 7fa277e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa2781607d0 con 0x7fa2780cd760 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa277400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2780b4690 0x7fa2780cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa276a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa27815c3d0 0x7fa2780cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46542/0 (socket says 172.21.15.59:46542) 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa277400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2780b4690 0x7fa2780cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46034/0 (socket says 172.21.15.59:46034) 2024-04-23T14:38:36.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa276a00640 1 -- 172.21.15.59:0/1798503782 learned_addr learned my addr 172.21.15.59:0/1798503782 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:36.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa276000640 1 --2- 172.21.15.59:0/1798503782 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780cd760 0x7fa2780b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa277400640 1 -- 172.21.15.59:0/1798503782 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780cd760 msgr2=0x7fa2780b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:36.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa277400640 1 --2- 172.21.15.59:0/1798503782 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780cd760 0x7fa2780b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa277400640 1 -- 172.21.15.59:0/1798503782 >> v1:172.21.15.59:6789/0 conn(0x7fa27815c3d0 legacy=0x7fa2780cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.284+0000 7fa277400640 1 -- 172.21.15.59:0/1798503782 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa278160ab0 con 0x7fa2780b4690 2024-04-23T14:38:36.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.285+0000 7fa277400640 1 --2- 172.21.15.59:0/1798503782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780b4690 0x7fa2780cb7d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa270009d90 tx=0x7fa27002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cf8e8ba5f0ebc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa275600640 1 -- 172.21.15.59:0/1798503782 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270007740 con 0x7fa2780b4690 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa275600640 1 -- 172.21.15.59:0/1798503782 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2700078e0 con 0x7fa2780b4690 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa275600640 1 -- 172.21.15.59:0/1798503782 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270007cb0 con 0x7fa2780b4690 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa277e00640 1 -- 172.21.15.59:0/1798503782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780b4690 msgr2=0x7fa2780cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa277e00640 1 --2- 172.21.15.59:0/1798503782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780b4690 0x7fa2780cb7d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa270009d90 tx=0x7fa27002a660 comp rx=0 tx=0).stop 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa277e00640 1 -- 172.21.15.59:0/1798503782 shutdown_connections 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa277e00640 1 --2- 172.21.15.59:0/1798503782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780b4690 0x7fa2780cb7d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa277e00640 1 --2- 172.21.15.59:0/1798503782 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780cd760 0x7fa2780b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa277e00640 1 -- 172.21.15.59:0/1798503782 >> 172.21.15.59:0/1798503782 conn(0x7fa2780b21a0 msgr2=0x7fa2780b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.286+0000 7fa277e00640 1 -- 172.21.15.59:0/1798503782 shutdown_connections 2024-04-23T14:38:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.287+0000 7fa277e00640 1 -- 172.21.15.59:0/1798503782 wait complete. 2024-04-23T14:38:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.288+0000 7fa277e00640 1 Processor -- start 2024-04-23T14:38:36.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.288+0000 7fa277e00640 1 -- start start 2024-04-23T14:38:36.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa277e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780b4690 0x7fa2780c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa277e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780cd760 0x7fa27815b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa277e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa27815c3d0 0x7fa27815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa277e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa27815c7b0 con 0x7fa2780cd760 2024-04-23T14:38:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa277e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780b4ac0 con 0x7fa27815c3d0 2024-04-23T14:38:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa277e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa27815c930 con 0x7fa2780b4690 2024-04-23T14:38:36.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa276000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780cd760 0x7fa27815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa276000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780cd760 0x7fa27815b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46044/0 (socket says 172.21.15.59:46044) 2024-04-23T14:38:36.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.289+0000 7fa276000640 1 -- 172.21.15.59:0/3007078053 learned_addr learned my addr 172.21.15.59:0/3007078053 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:36.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa276000640 1 -- 172.21.15.59:0/3007078053 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780b4690 msgr2=0x7fa2780c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa276000640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780b4690 0x7fa2780c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa276000640 1 -- 172.21.15.59:0/3007078053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa27815c3d0 msgr2=0x7fa27815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa276000640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa27815c3d0 0x7fa27815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa276000640 1 -- 172.21.15.59:0/3007078053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27815a3a0 con 0x7fa2780cd760 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa276000640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780cd760 0x7fa27815b440 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa26400d800 tx=0x7fa26400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa26ea00640 1 -- 172.21.15.59:0/3007078053 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26400b6c0 con 0x7fa2780cd760 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa26ea00640 1 -- 172.21.15.59:0/3007078053 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa264019070 con 0x7fa2780cd760 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa26ea00640 1 -- 172.21.15.59:0/3007078053 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26400ba80 con 0x7fa2780cd760 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.290+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27815a680 con 0x7fa2780cd760 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.291+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27815ac40 con 0x7fa2780cd760 2024-04-23T14:38:36.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.292+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27806fda0 con 0x7fa2780cd760 2024-04-23T14:38:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.293+0000 7fa26ea00640 1 -- 172.21.15.59:0/3007078053 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa26400bc20 con 0x7fa2780cd760 2024-04-23T14:38:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.294+0000 7fa26ea00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa25806c8f0 0x7fa25806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.294+0000 7fa26ea00640 1 -- 172.21.15.59:0/3007078053 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fa26408bcc0 con 0x7fa2780cd760 2024-04-23T14:38:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.294+0000 7fa276a00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa25806c8f0 0x7fa25806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.295+0000 7fa276a00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa25806c8f0 0x7fa25806edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f60 tx=0x7fa268007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:36.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.299+0000 7fa26ea00640 1 -- 172.21.15.59:0/3007078053 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa27806fda0 con 0x7fa2780cd760 2024-04-23T14:38:36.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.497+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa27804fd90 con 0x7fa2780cd760 2024-04-23T14:38:36.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.498+0000 7fa26ea00640 1 -- 172.21.15.59:0/3007078053 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7fa264092080 con 0x7fa2780cd760 2024-04-23T14:38:36.499 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:36.500 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:38:36.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:38:36.500 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:38:36.500 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:38:36.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:36.500 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.500+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa25806c8f0 msgr2=0x7fa25806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.500+0000 7fa277e00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa25806c8f0 0x7fa25806edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f60 tx=0x7fa268007a80 comp rx=0 tx=0).stop 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.500+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780cd760 msgr2=0x7fa27815b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.500+0000 7fa277e00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780cd760 0x7fa27815b440 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa26400d800 tx=0x7fa26400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 shutdown_connections 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa25806c8f0 0x7fa25806edb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa27815c3d0 0x7fa27815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2780cd760 0x7fa27815b440 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 --2- 172.21.15.59:0/3007078053 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2780b4690 0x7fa2780c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 >> 172.21.15.59:0/3007078053 conn(0x7fa2780b21a0 msgr2=0x7fa27806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:36.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 shutdown_connections 2024-04-23T14:38:36.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.501+0000 7fa277e00640 1 -- 172.21.15.59:0/3007078053 wait complete. 2024-04-23T14:38:36.519 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:38:36.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:38:36.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.731+0000 7efc3b200640 1 Processor -- start 2024-04-23T14:38:36.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.731+0000 7efc3b200640 1 -- start start 2024-04-23T14:38:36.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c06bdf0 0x7efc3c06c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc3c160100 0x7efc3c160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc3c157000 con 0x7efc3c156c20 2024-04-23T14:38:36.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7efc3c0afb00 con 0x7efc3c160100 2024-04-23T14:38:36.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7efc3c15cc20 con 0x7efc3c06bdf0 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc3c160100 0x7efc3c160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc3c160100 0x7efc3c160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46054/0 (socket says 172.21.15.59:46054) 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.733+0000 7efc3a800640 1 -- 172.21.15.59:0/434611135 learned_addr learned my addr 172.21.15.59:0/434611135 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.734+0000 7efc3a800640 1 -- 172.21.15.59:0/434611135 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c06bdf0 msgr2=0x7efc3c06c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.734+0000 7efc3a800640 1 --2- 172.21.15.59:0/434611135 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c06bdf0 0x7efc3c06c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.734+0000 7efc3a800640 1 -- 172.21.15.59:0/434611135 >> v1:172.21.15.76:6789/0 conn(0x7efc3c156c20 legacy=0x7efc3c0aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.734+0000 7efc3a800640 1 -- 172.21.15.59:0/434611135 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c15cf00 con 0x7efc3c160100 2024-04-23T14:38:36.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.735+0000 7efc3a800640 1 --2- 172.21.15.59:0/434611135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c160100 0x7efc3c160590 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7efc34009e50 tx=0x7efc3402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ac86a178c4eb9c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.735+0000 7efc38a00640 1 -- 172.21.15.59:0/434611135 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34036070 con 0x7efc3c160100 2024-04-23T14:38:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.735+0000 7efc38a00640 1 -- 172.21.15.59:0/434611135 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc34031600 con 0x7efc3c160100 2024-04-23T14:38:36.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.735+0000 7efc38a00640 1 -- 172.21.15.59:0/434611135 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc340319d0 con 0x7efc3c160100 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 -- 172.21.15.59:0/434611135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c160100 msgr2=0x7efc3c160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 --2- 172.21.15.59:0/434611135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c160100 0x7efc3c160590 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7efc34009e50 tx=0x7efc3402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 -- 172.21.15.59:0/434611135 shutdown_connections 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 --2- 172.21.15.59:0/434611135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c160100 0x7efc3c160590 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 --2- 172.21.15.59:0/434611135 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c06bdf0 0x7efc3c06c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 -- 172.21.15.59:0/434611135 >> 172.21.15.59:0/434611135 conn(0x7efc3c0b2200 msgr2=0x7efc3c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 -- 172.21.15.59:0/434611135 shutdown_connections 2024-04-23T14:38:36.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.736+0000 7efc3b200640 1 -- 172.21.15.59:0/434611135 wait complete. 2024-04-23T14:38:36.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.737+0000 7efc3b200640 1 Processor -- start 2024-04-23T14:38:36.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.738+0000 7efc3b200640 1 -- start start 2024-04-23T14:38:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.738+0000 7efc3b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c06bdf0 0x7efc3c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.738+0000 7efc3b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c156c20 0x7efc3c14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc3b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c160100 0x7efc3c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc3b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c15d9e0 con 0x7efc3c156c20 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc3b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c15d4c0 con 0x7efc3c06bdf0 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc3b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7efc3c073ba0 con 0x7efc3c160100 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c156c20 0x7efc3c14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c156c20 0x7efc3c14cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46062/0 (socket says 172.21.15.59:46062) 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 -- 172.21.15.59:0/2485910216 learned_addr learned my addr 172.21.15.59:0/2485910216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39e00640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c06bdf0 0x7efc3c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc3a800640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c160100 0x7efc3c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 -- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c160100 msgr2=0x7efc3c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c160100 0x7efc3c15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 -- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c06bdf0 msgr2=0x7efc3c15f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c06bdf0 0x7efc3c15f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc39400640 1 -- 172.21.15.59:0/2485910216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c15a7b0 con 0x7efc3c156c20 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.739+0000 7efc3a800640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c160100 0x7efc3c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:36.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.740+0000 7efc39400640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c156c20 0x7efc3c14cb60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b4d0 tx=0x7efc2800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.740+0000 7efc32000640 1 -- 172.21.15.59:0/2485910216 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28014070 con 0x7efc3c156c20 2024-04-23T14:38:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.740+0000 7efc32000640 1 -- 172.21.15.59:0/2485910216 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc28010870 con 0x7efc3c156c20 2024-04-23T14:38:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.740+0000 7efc32000640 1 -- 172.21.15.59:0/2485910216 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28010c60 con 0x7efc3c156c20 2024-04-23T14:38:36.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.740+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c15aa50 con 0x7efc3c156c20 2024-04-23T14:38:36.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.740+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c15b010 con 0x7efc3c156c20 2024-04-23T14:38:36.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.741+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c04fd90 con 0x7efc3c156c20 2024-04-23T14:38:36.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.742+0000 7efc32000640 1 -- 172.21.15.59:0/2485910216 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc28017070 con 0x7efc3c156c20 2024-04-23T14:38:36.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.743+0000 7efc32000640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efc1c06c8f0 0x7efc1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:36.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.743+0000 7efc32000640 1 -- 172.21.15.59:0/2485910216 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7efc2808b5d0 con 0x7efc3c156c20 2024-04-23T14:38:36.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.743+0000 7efc39e00640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efc1c06c8f0 0x7efc1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:36.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.744+0000 7efc39e00640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efc1c06c8f0 0x7efc1c06edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efc3c06fe20 tx=0x7efc2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.747+0000 7efc32000640 1 -- 172.21.15.59:0/2485910216 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc28059d40 con 0x7efc3c156c20 2024-04-23T14:38:36.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:36.950+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efc3c0b5090 con 0x7efc3c156c20 2024-04-23T14:38:37.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.709+0000 7efc32000640 1 -- 172.21.15.59:0/2485910216 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7efc28059640 con 0x7efc3c156c20 2024-04-23T14:38:37.711 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:38:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.713+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efc1c06c8f0 msgr2=0x7efc1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:37.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.713+0000 7efc3b200640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efc1c06c8f0 0x7efc1c06edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efc3c06fe20 tx=0x7efc2c009040 comp rx=0 tx=0).stop 2024-04-23T14:38:37.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.713+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c156c20 msgr2=0x7efc3c14cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:37.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.713+0000 7efc3b200640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c156c20 0x7efc3c14cb60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b4d0 tx=0x7efc2800b990 comp rx=0 tx=0).stop 2024-04-23T14:38:37.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.714+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 shutdown_connections 2024-04-23T14:38:37.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.714+0000 7efc3b200640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efc1c06c8f0 0x7efc1c06edb0 secure :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efc3c06fe20 tx=0x7efc2c009040 comp rx=0 tx=0).stop 2024-04-23T14:38:37.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.714+0000 7efc3b200640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efc3c160100 0x7efc3c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.714+0000 7efc3b200640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc3c156c20 0x7efc3c14cb60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.714+0000 7efc3b200640 1 --2- 172.21.15.59:0/2485910216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c06bdf0 0x7efc3c15f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.714+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 >> 172.21.15.59:0/2485910216 conn(0x7efc3c0b2200 msgr2=0x7efc3c152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.714+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 shutdown_connections 2024-04-23T14:38:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.715+0000 7efc3b200640 1 -- 172.21.15.59:0/2485910216 wait complete. 2024-04-23T14:38:37.743 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:38:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.947+0000 7f4c8da00640 1 Processor -- start 2024-04-23T14:38:37.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.948+0000 7f4c8da00640 1 -- start start 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.949+0000 7f4c8da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c880b99a0 0x7f4c88169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.949+0000 7f4c8da00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c880b87f0 con 0x7f4c880b3490 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.949+0000 7f4c8da00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c880d2f30 con 0x7f4c8815eb60 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.949+0000 7f4c8da00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4c880ba7a0 con 0x7f4c880b99a0 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.950+0000 7f4c87400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c880b99a0 0x7f4c88169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.950+0000 7f4c87400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c880b99a0 0x7f4c88169510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46076/0 (socket says 172.21.15.59:46076) 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.950+0000 7f4c87400640 1 -- 172.21.15.59:0/2658656129 learned_addr learned my addr 172.21.15.59:0/2658656129 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.950+0000 7f4c86000640 1 --1- 172.21.15.59:0/2658656129 >> v1:172.21.15.59:6789/0 conn(0x7f4c880b3490 0x7f4c880b9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46554/0 (socket says 172.21.15.59:46554) 2024-04-23T14:38:37.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.950+0000 7f4c87400640 1 -- 172.21.15.59:0/2658656129 >> v1:172.21.15.76:6789/0 conn(0x7f4c8815eb60 legacy=0x7f4c880d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:37.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.950+0000 7f4c87400640 1 -- 172.21.15.59:0/2658656129 >> v1:172.21.15.59:6789/0 conn(0x7f4c880b3490 legacy=0x7f4c880b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:37.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.950+0000 7f4c87400640 1 -- 172.21.15.59:0/2658656129 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c880ba1e0 con 0x7f4c880b99a0 2024-04-23T14:38:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.951+0000 7f4c87400640 1 --2- 172.21.15.59:0/2658656129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c88169510 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009870 tx=0x7f4c8002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a52c12954ffdd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c85600640 1 -- 172.21.15.59:0/2658656129 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80037070 con 0x7f4c880b99a0 2024-04-23T14:38:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c85600640 1 -- 172.21.15.59:0/2658656129 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8002ae50 con 0x7f4c880b99a0 2024-04-23T14:38:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c85600640 1 -- 172.21.15.59:0/2658656129 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80032860 con 0x7f4c880b99a0 2024-04-23T14:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c8da00640 1 -- 172.21.15.59:0/2658656129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 msgr2=0x7f4c88169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c8da00640 1 --2- 172.21.15.59:0/2658656129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c88169510 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009870 tx=0x7f4c8002a300 comp rx=0 tx=0).stop 2024-04-23T14:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c8da00640 1 -- 172.21.15.59:0/2658656129 shutdown_connections 2024-04-23T14:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c8da00640 1 --2- 172.21.15.59:0/2658656129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c88169510 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.952+0000 7f4c8da00640 1 -- 172.21.15.59:0/2658656129 >> 172.21.15.59:0/2658656129 conn(0x7f4c88067b50 msgr2=0x7f4c88067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.953+0000 7f4c8da00640 1 -- 172.21.15.59:0/2658656129 shutdown_connections 2024-04-23T14:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.953+0000 7f4c8da00640 1 -- 172.21.15.59:0/2658656129 wait complete. 2024-04-23T14:38:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.955+0000 7f4c8da00640 1 Processor -- start 2024-04-23T14:38:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.955+0000 7f4c8da00640 1 -- start start 2024-04-23T14:38:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.956+0000 7f4c8da00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c880b3490 0x7f4c881611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.956+0000 7f4c8da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c8814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.956+0000 7f4c8da00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c8815eb60 0x7f4c880cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.956+0000 7f4c8da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880b8d70 con 0x7f4c880b99a0 2024-04-23T14:38:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.956+0000 7f4c8da00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880b8a90 con 0x7f4c880b3490 2024-04-23T14:38:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.956+0000 7f4c8da00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4c880b3c80 con 0x7f4c8815eb60 2024-04-23T14:38:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c8814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c8814c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46078/0 (socket says 172.21.15.59:46078) 2024-04-23T14:38:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 -- 172.21.15.59:0/3585454963 learned_addr learned my addr 172.21.15.59:0/3585454963 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c87400640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c8815eb60 0x7f4c880cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86a00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c880b3490 0x7f4c881611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 -- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c8815eb60 msgr2=0x7f4c880cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c8815eb60 0x7f4c880cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 -- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c880b3490 msgr2=0x7f4c881611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c880b3490 0x7f4c881611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.957+0000 7f4c86000640 1 -- 172.21.15.59:0/3585454963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8815fee0 con 0x7f4c880b99a0 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.958+0000 7f4c86000640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c8814c7d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000c7a0 tx=0x7f4c7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.958+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3585454963 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70017070 con 0x7f4c880b99a0 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.958+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3585454963 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7000a640 con 0x7f4c880b99a0 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.958+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8814cd10 con 0x7f4c880b99a0 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.958+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3585454963 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000a9f0 con 0x7f4c880b99a0 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.958+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c88160440 con 0x7f4c880b99a0 2024-04-23T14:38:37.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.959+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c88068a80 con 0x7f4c880b99a0 2024-04-23T14:38:37.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.961+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3585454963 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c7001a070 con 0x7f4c880b99a0 2024-04-23T14:38:37.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.962+0000 7f4c7ea00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c6406c8f0 0x7f4c6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:37.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.962+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3585454963 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5227+0+0 (secure 0 0 0) 0x7f4c7008ae70 con 0x7f4c880b99a0 2024-04-23T14:38:37.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.962+0000 7f4c86a00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c6406c8f0 0x7f4c6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:37.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.963+0000 7f4c86a00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c6406c8f0 0x7f4c6406edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4c78001640 tx=0x7f4c78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:37.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:37.968+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3585454963 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c8815fee0 con 0x7f4c880b99a0 2024-04-23T14:38:38.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.174+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4c880b8300 con 0x7f4c880b99a0 2024-04-23T14:38:38.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.724+0000 7f4c7ea00640 1 -- 172.21.15.59:0/3585454963 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7f4c70058e30 con 0x7f4c880b99a0 2024-04-23T14:38:38.726 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:38:38.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.728+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c6406c8f0 msgr2=0x7f4c6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.728+0000 7f4c8da00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c6406c8f0 0x7f4c6406edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4c78001640 tx=0x7f4c78009040 comp rx=0 tx=0).stop 2024-04-23T14:38:38.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 msgr2=0x7f4c8814c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c8814c7d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000c7a0 tx=0x7f4c7000cc60 comp rx=0 tx=0).stop 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 shutdown_connections 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4c6406c8f0 0x7f4c6406edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c8815eb60 0x7f4c880cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c880b99a0 0x7f4c8814c7d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 --2- 172.21.15.59:0/3585454963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c880b3490 0x7f4c881611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.729+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 >> 172.21.15.59:0/3585454963 conn(0x7f4c88067b50 msgr2=0x7f4c88159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.730+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 shutdown_connections 2024-04-23T14:38:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.730+0000 7f4c8da00640 1 -- 172.21.15.59:0/3585454963 wait complete. 2024-04-23T14:38:38.768 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:38:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.946+0000 7fae4a800640 1 Processor -- start 2024-04-23T14:38:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.947+0000 7fae4a800640 1 -- start start 2024-04-23T14:38:38.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.948+0000 7fae4a800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae44072480 0x7fae4416d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.948+0000 7fae4a800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae440b35d0 con 0x7fae44162c10 2024-04-23T14:38:38.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.948+0000 7fae4a800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae440b38b0 con 0x7fae440b4670 2024-04-23T14:38:38.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.948+0000 7fae4a800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fae440b3b90 con 0x7fae44072480 2024-04-23T14:38:38.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.949+0000 7fae43400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fae44162c10 0x7fae440d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46562/0 (socket says 172.21.15.59:46562) 2024-04-23T14:38:38.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.949+0000 7fae43400640 1 -- 172.21.15.59:0/3966137741 learned_addr learned my addr 172.21.15.59:0/3966137741 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.949+0000 7fae43e00640 1 --2- 172.21.15.59:0/3966137741 >> v2:172.21.15.76:3300/0 conn(0x7fae44072480 0x7fae4416d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.949+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1133847133 0 0) 0x7fae440b35d0 con 0x7fae44162c10 2024-04-23T14:38:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.949+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae28003520 con 0x7fae44162c10 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 873237762 0 0) 0x7fae28003520 con 0x7fae44162c10 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 >> v2:172.21.15.76:3300/0 conn(0x7fae44072480 msgr2=0x7fae4416d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 --2- 172.21.15.59:0/3966137741 >> v2:172.21.15.76:3300/0 conn(0x7fae44072480 0x7fae4416d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 >> v1:172.21.15.76:6789/0 conn(0x7fae440b4670 legacy=0x7fae44071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae440b3e70 con 0x7fae44162c10 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fae34001670 con 0x7fae44162c10 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 >> v1:172.21.15.59:6789/0 conn(0x7fae44162c10 legacy=0x7fae440d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae28004800 0x7fae28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae280051c0 0x7fae2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae440b3b90 con 0x7fae280039b0 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae2800c060 con 0x7fae28004800 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae2800c260 con 0x7fae280051c0 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.950+0000 7fae43400640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae42a00640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae28004800 0x7fae28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae43400640 1 -- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae280051c0 msgr2=0x7fae2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae43400640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae280051c0 0x7fae2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae43400640 1 -- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae28004800 msgr2=0x7fae28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae43400640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae28004800 0x7fae28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae43400640 1 -- 172.21.15.59:0/3966137741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae28003520 con 0x7fae280039b0 2024-04-23T14:38:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae42a00640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae28004800 0x7fae28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.951+0000 7fae43e00640 1 -- 172.21.15.59:0/3966137741 reap_dead start 2024-04-23T14:38:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.952+0000 7fae43400640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae28003dd0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c240 tx=0x7fae28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7851aa8511df1d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.952+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34007cf0 con 0x7fae280039b0 2024-04-23T14:38:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.952+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae34005af0 con 0x7fae280039b0 2024-04-23T14:38:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.952+0000 7fae42000640 1 -- 172.21.15.59:0/3966137741 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34007070 con 0x7fae280039b0 2024-04-23T14:38:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.953+0000 7fae4a800640 1 -- 172.21.15.59:0/3966137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 msgr2=0x7fae28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.953+0000 7fae4a800640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae28003dd0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c240 tx=0x7fae28005830 comp rx=0 tx=0).stop 2024-04-23T14:38:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.953+0000 7fae4a800640 1 -- 172.21.15.59:0/3966137741 shutdown_connections 2024-04-23T14:38:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.953+0000 7fae4a800640 1 --2- 172.21.15.59:0/3966137741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae28003dd0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.953+0000 7fae4a800640 1 -- 172.21.15.59:0/3966137741 >> 172.21.15.59:0/3966137741 conn(0x7fae44067d90 msgr2=0x7fae4406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.953+0000 7fae4a800640 1 -- 172.21.15.59:0/3966137741 shutdown_connections 2024-04-23T14:38:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.953+0000 7fae4a800640 1 -- 172.21.15.59:0/3966137741 wait complete. 2024-04-23T14:38:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.955+0000 7fae4a800640 1 Processor -- start 2024-04-23T14:38:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.955+0000 7fae4a800640 1 -- start start 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae4a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae441560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae4a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae44165d90 0x7fae440b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae4a800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae440b1460 0x7fae441599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae4a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae44163170 con 0x7fae280039b0 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae4a800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae44162ff0 con 0x7fae440b1460 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae4a800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae440cfba0 con 0x7fae44165d90 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae441560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae441560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46096/0 (socket says 172.21.15.59:46096) 2024-04-23T14:38:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.956+0000 7fae43400640 1 -- 172.21.15.59:0/840461283 learned_addr learned my addr 172.21.15.59:0/840461283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae42a00640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae44165d90 0x7fae440b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43e00640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae440b1460 0x7fae441599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43400640 1 -- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae44165d90 msgr2=0x7fae440b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43400640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae44165d90 0x7fae440b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43400640 1 -- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae440b1460 msgr2=0x7fae441599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43400640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae440b1460 0x7fae441599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43400640 1 -- 172.21.15.59:0/840461283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441565f0 con 0x7fae280039b0 2024-04-23T14:38:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43e00640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae440b1460 0x7fae441599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.957+0000 7fae43400640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae441560b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c360 tx=0x7fae440b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.958+0000 7fae3b400640 1 -- 172.21.15.59:0/840461283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34041030 con 0x7fae280039b0 2024-04-23T14:38:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.958+0000 7fae3b400640 1 -- 172.21.15.59:0/840461283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae34000ed0 con 0x7fae280039b0 2024-04-23T14:38:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.958+0000 7fae3b400640 1 -- 172.21.15.59:0/840461283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34038040 con 0x7fae280039b0 2024-04-23T14:38:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.958+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae440b0160 con 0x7fae280039b0 2024-04-23T14:38:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.958+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae440b0720 con 0x7fae280039b0 2024-04-23T14:38:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.959+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae44068a80 con 0x7fae280039b0 2024-04-23T14:38:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.960+0000 7fae3b400640 1 -- 172.21.15.59:0/840461283 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae34044070 con 0x7fae280039b0 2024-04-23T14:38:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.961+0000 7fae3b400640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae2406c8f0 0x7fae2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.961+0000 7fae3b400640 1 -- 172.21.15.59:0/840461283 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5609+0+0 (secure 0 0 0) 0x7fae340aba60 con 0x7fae280039b0 2024-04-23T14:38:38.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.962+0000 7fae42a00640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae2406c8f0 0x7fae2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.962+0000 7fae42a00640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae2406c8f0 0x7fae2406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae44161e20 tx=0x7fae30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:38.966+0000 7fae3b400640 1 -- 172.21.15.59:0/840461283 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae34079ee0 con 0x7fae280039b0 2024-04-23T14:38:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.208+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fae44068a80 con 0x7fae280039b0 2024-04-23T14:38:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.729+0000 7fae3b400640 1 -- 172.21.15.59:0/840461283 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v56) ==== 174+0+0 (secure 0 0 0) 0x7fae340797e0 con 0x7fae280039b0 2024-04-23T14:38:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 12 and data pool 13 2024-04-23T14:38:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.734+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae2406c8f0 msgr2=0x7fae2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.734+0000 7fae4a800640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae2406c8f0 0x7fae2406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae44161e20 tx=0x7fae30009040 comp rx=0 tx=0).stop 2024-04-23T14:38:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.734+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 msgr2=0x7fae441560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.734+0000 7fae4a800640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae441560b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c360 tx=0x7fae440b4040 comp rx=0 tx=0).stop 2024-04-23T14:38:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.736+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 shutdown_connections 2024-04-23T14:38:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.736+0000 7fae4a800640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae2406c8f0 0x7fae2406edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.736+0000 7fae4a800640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae44165d90 0x7fae440b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.736+0000 7fae4a800640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae440b1460 0x7fae441599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.736+0000 7fae4a800640 1 --2- 172.21.15.59:0/840461283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae280039b0 0x7fae441560b0 secure :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c360 tx=0x7fae440b4040 comp rx=0 tx=0).stop 2024-04-23T14:38:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.736+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 >> 172.21.15.59:0/840461283 conn(0x7fae44067d90 msgr2=0x7fae4415f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.738+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 shutdown_connections 2024-04-23T14:38:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.738+0000 7fae4a800640 1 -- 172.21.15.59:0/840461283 wait complete. 2024-04-23T14:38:39.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:39.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.955+0000 7f3294e00640 1 Processor -- start 2024-04-23T14:38:39.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.955+0000 7f3294e00640 1 -- start start 2024-04-23T14:38:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.956+0000 7f3294e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f329006f3f0 0x7f32900cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.956+0000 7f3294e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32900b3080 0x7f32900cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.956+0000 7f3294e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32900b3870 con 0x7f329015b450 2024-04-23T14:38:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.956+0000 7f3294e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f32900b41a0 con 0x7f329006f3f0 2024-04-23T14:38:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.956+0000 7f3294e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32900b3b50 con 0x7f32900b3080 2024-04-23T14:38:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.956+0000 7f328f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f329015b450 0x7f32900b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:46578/0 (socket says 172.21.15.59:46578) 2024-04-23T14:38:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.956+0000 7f328f400640 1 -- 172.21.15.59:0/3496528703 learned_addr learned my addr 172.21.15.59:0/3496528703 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f3287e00640 1 --2- 172.21.15.59:0/3496528703 >> v2:172.21.15.59:3300/0 conn(0x7f329006f3f0 0x7f32900cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f328ea00640 1 -- 172.21.15.59:0/3496528703 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3303315827 0 0) 0x7f32900b3870 con 0x7f329015b450 2024-04-23T14:38:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f328fe00640 1 --2- 172.21.15.59:0/3496528703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32900b3080 0x7f32900cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f328ea00640 1 -- 172.21.15.59:0/3496528703 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3274003520 con 0x7f329015b450 2024-04-23T14:38:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f3287e00640 1 -- 172.21.15.59:0/3496528703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32900b3080 msgr2=0x7f32900cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f3287e00640 1 --2- 172.21.15.59:0/3496528703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32900b3080 0x7f32900cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f3287e00640 1 -- 172.21.15.59:0/3496528703 >> v1:172.21.15.59:6789/0 conn(0x7f329015b450 legacy=0x7f32900b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.957+0000 7f3287e00640 1 -- 172.21.15.59:0/3496528703 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32900b3e30 con 0x7f329006f3f0 2024-04-23T14:38:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.958+0000 7f3287e00640 1 --2- 172.21.15.59:0/3496528703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32900cd050 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f32780097a0 tx=0x7f327802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4983c47fada55d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f328ea00640 1 -- 172.21.15.59:0/3496528703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278037070 con 0x7f329006f3f0 2024-04-23T14:38:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f328ea00640 1 -- 172.21.15.59:0/3496528703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3278032440 con 0x7f329006f3f0 2024-04-23T14:38:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f328ea00640 1 -- 172.21.15.59:0/3496528703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278032810 con 0x7f329006f3f0 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f3294e00640 1 -- 172.21.15.59:0/3496528703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 msgr2=0x7f32900cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f3294e00640 1 --2- 172.21.15.59:0/3496528703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32900cd050 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f32780097a0 tx=0x7f327802a210 comp rx=0 tx=0).stop 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f3294e00640 1 -- 172.21.15.59:0/3496528703 shutdown_connections 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f3294e00640 1 --2- 172.21.15.59:0/3496528703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32900b3080 0x7f32900cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f3294e00640 1 --2- 172.21.15.59:0/3496528703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32900cd050 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.959+0000 7f3294e00640 1 -- 172.21.15.59:0/3496528703 >> 172.21.15.59:0/3496528703 conn(0x7f32900b21a0 msgr2=0x7f32900b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.960+0000 7f3294e00640 1 -- 172.21.15.59:0/3496528703 shutdown_connections 2024-04-23T14:38:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.960+0000 7f3294e00640 1 -- 172.21.15.59:0/3496528703 wait complete. 2024-04-23T14:38:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.961+0000 7f3294e00640 1 Processor -- start 2024-04-23T14:38:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.961+0000 7f3294e00640 1 -- start start 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.962+0000 7f3294e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32901f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.962+0000 7f3294e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32900b3080 0x7f32900b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.962+0000 7f3294e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f329015b450 0x7f32900c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.962+0000 7f3294e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32900ad060 con 0x7f329006f3f0 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f3294e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32900b4ff0 con 0x7f32900b3080 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f3294e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32900ad1e0 con 0x7f329015b450 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32901f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32901f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46120/0 (socket says 172.21.15.59:46120) 2024-04-23T14:38:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 -- 172.21.15.59:0/1992179092 learned_addr learned my addr 172.21.15.59:0/1992179092 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:39.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f3287e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32900b3080 0x7f32900b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328fe00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f329015b450 0x7f32900c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 -- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f329015b450 msgr2=0x7f32900c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f329015b450 0x7f32900c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 -- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32900b3080 msgr2=0x7f32900b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32900b3080 0x7f32900b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.963+0000 7f328f400640 1 -- 172.21.15.59:0/1992179092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329015ba40 con 0x7f329006f3f0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.964+0000 7f328f400640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32901f4000 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f32800010e0 tx=0x7f328000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.964+0000 7f3287400640 1 -- 172.21.15.59:0/1992179092 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280017070 con 0x7f329006f3f0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.964+0000 7f3287400640 1 -- 172.21.15.59:0/1992179092 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3280005d00 con 0x7f329006f3f0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.964+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329015bd20 con 0x7f329006f3f0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.964+0000 7f3287400640 1 -- 172.21.15.59:0/1992179092 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280007310 con 0x7f329006f3f0 2024-04-23T14:38:39.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.964+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329015c2e0 con 0x7f329006f3f0 2024-04-23T14:38:39.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.965+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329015ca00 con 0x7f329006f3f0 2024-04-23T14:38:39.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.966+0000 7f3287400640 1 -- 172.21.15.59:0/1992179092 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f328001a070 con 0x7f329006f3f0 2024-04-23T14:38:39.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.967+0000 7f3287400640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f326c06c8f0 0x7f326c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.968+0000 7f3287400640 1 -- 172.21.15.59:0/1992179092 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f328008b0a0 con 0x7f329006f3f0 2024-04-23T14:38:39.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.968+0000 7f3287e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f326c06c8f0 0x7f326c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.969+0000 7f3287e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f326c06c8f0 0x7f326c06edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3278034000 tx=0x7f327802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:39.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:39.972+0000 7f3287400640 1 -- 172.21.15.59:0/1992179092 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32800593a0 con 0x7f329006f3f0 2024-04-23T14:38:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.178+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f329004fd90 con 0x7f329006f3f0 2024-04-23T14:38:40.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.181+0000 7f3287400640 1 -- 172.21.15.59:0/1992179092 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16613 (secure 0 0 0) 0x7f3280058ca0 con 0x7f329006f3f0 2024-04-23T14:38:40.183 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:40.183 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:38:39.211023+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f326c06c8f0 msgr2=0x7f326c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f326c06c8f0 0x7f326c06edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3278034000 tx=0x7f327802e000 comp rx=0 tx=0).stop 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 msgr2=0x7f32901f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32901f4000 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f32800010e0 tx=0x7f328000c890 comp rx=0 tx=0).stop 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 shutdown_connections 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f326c06c8f0 0x7f326c06edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f329015b450 0x7f32900c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32900b3080 0x7f32900b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 --2- 172.21.15.59:0/1992179092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f329006f3f0 0x7f32901f4000 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 >> 172.21.15.59:0/1992179092 conn(0x7f32900b21a0 msgr2=0x7f3290158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:40.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 shutdown_connections 2024-04-23T14:38:40.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.184+0000 7f3294e00640 1 -- 172.21.15.59:0/1992179092 wait complete. 2024-04-23T14:38:40.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.410+0000 7f5533200640 1 Processor -- start 2024-04-23T14:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.411+0000 7f5533200640 1 -- start start 2024-04-23T14:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.412+0000 7f5533200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f553415e7c0 0x7f55340d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.412+0000 7f5533200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55340cf030 con 0x7f55340d0d90 2024-04-23T14:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.412+0000 7f5533200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55340b6a40 con 0x7f55340d2630 2024-04-23T14:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.412+0000 7f5533200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f55340b7000 con 0x7f553415e7c0 2024-04-23T14:38:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.412+0000 7f5532800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f55340d0d90 0x7f55340cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39120/0 (socket says 172.21.15.59:39120) 2024-04-23T14:38:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.412+0000 7f5532800640 1 -- 172.21.15.59:0/2897863818 learned_addr learned my addr 172.21.15.59:0/2897863818 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.412+0000 7f5531e00640 1 --2- 172.21.15.59:0/2897863818 >> v2:172.21.15.76:3300/0 conn(0x7f553415e7c0 0x7f55340d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2816104452 0 0) 0x7f552c001110 con 0x7f55340d0d90 2024-04-23T14:38:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5518003520 con 0x7f55340d0d90 2024-04-23T14:38:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3955526658 0 0) 0x7f5518003520 con 0x7f55340d0d90 2024-04-23T14:38:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 >> v2:172.21.15.76:3300/0 conn(0x7f553415e7c0 msgr2=0x7f55340d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 --2- 172.21.15.59:0/2897863818 >> v2:172.21.15.76:3300/0 conn(0x7f553415e7c0 0x7f55340d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 >> v1:172.21.15.76:6789/0 conn(0x7f55340d2630 legacy=0x7f55340b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55340b6d20 con 0x7f55340d0d90 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f552c001840 con 0x7f55340d0d90 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 >> v1:172.21.15.59:6789/0 conn(0x7f55340d0d90 legacy=0x7f55340cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5518003a00 0x7f5518003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5518004800 0x7f5518004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55180051c0 0x7f551800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55340b7000 con 0x7f5518004800 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f551800c0c0 con 0x7f5518003a00 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.413+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f551800c2c0 con 0x7f55180051c0 2024-04-23T14:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5531400640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5518004800 0x7f5518004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5531e00640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5518003a00 0x7f5518003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5531400640 1 -- 172.21.15.59:0/2897863818 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55180051c0 msgr2=0x7f551800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5531400640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55180051c0 0x7f551800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5531400640 1 -- 172.21.15.59:0/2897863818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5518003a00 msgr2=0x7f5518003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5531400640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5518003a00 0x7f5518003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5531400640 1 -- 172.21.15.59:0/2897863818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5518003520 con 0x7f5518004800 2024-04-23T14:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.414+0000 7f5532800640 1 -- 172.21.15.59:0/2897863818 reap_dead start 2024-04-23T14:38:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.415+0000 7f5531400640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5518004800 0x7f5518004c80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5520001e20 tx=0x7f5518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf2181d20542b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520001a70 con 0x7f5518004800 2024-04-23T14:38:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552002ae10 con 0x7f5518004800 2024-04-23T14:38:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5530a00640 1 -- 172.21.15.59:0/2897863818 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520003bb0 con 0x7f5518004800 2024-04-23T14:38:40.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5533200640 1 -- 172.21.15.59:0/2897863818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5518004800 msgr2=0x7f5518004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5533200640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5518004800 0x7f5518004c80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5520001e20 tx=0x7f5518005830 comp rx=0 tx=0).stop 2024-04-23T14:38:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5533200640 1 -- 172.21.15.59:0/2897863818 shutdown_connections 2024-04-23T14:38:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5533200640 1 --2- 172.21.15.59:0/2897863818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5518004800 0x7f5518004c80 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5533200640 1 -- 172.21.15.59:0/2897863818 >> 172.21.15.59:0/2897863818 conn(0x7f553409d920 msgr2=0x7f553408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5533200640 1 -- 172.21.15.59:0/2897863818 shutdown_connections 2024-04-23T14:38:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.416+0000 7f5533200640 1 -- 172.21.15.59:0/2897863818 wait complete. 2024-04-23T14:38:40.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.418+0000 7f5533200640 1 Processor -- start 2024-04-23T14:38:40.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.418+0000 7f5533200640 1 -- start start 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.419+0000 7f5533200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5518004800 0x7f55340b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.419+0000 7f5533200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55340cc2e0 0x7f55340c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.419+0000 7f5533200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55340ccbf0 0x7f5534156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.419+0000 7f5533200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55340d1a80 con 0x7f55340ccbf0 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.419+0000 7f5533200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55340cf620 con 0x7f55340cc2e0 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.419+0000 7f5533200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f55340d1c00 con 0x7f5518004800 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.419+0000 7f5532800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55340ccbf0 0x7f5534156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55340ccbf0 0x7f5534156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43138/0 (socket says 172.21.15.59:43138) 2024-04-23T14:38:40.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 -- 172.21.15.59:0/3410165107 learned_addr learned my addr 172.21.15.59:0/3410165107 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5531e00640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5518004800 0x7f55340b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5531400640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55340cc2e0 0x7f55340c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 -- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5518004800 msgr2=0x7f55340b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5518004800 0x7f55340b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 -- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55340cc2e0 msgr2=0x7f55340c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55340cc2e0 0x7f55340c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 -- 172.21.15.59:0/3410165107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55340c88e0 con 0x7f55340ccbf0 2024-04-23T14:38:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.420+0000 7f5532800640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55340ccbf0 0x7f5534156660 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f553408c6a0 tx=0x7f552c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.421+0000 7f552a000640 1 -- 172.21.15.59:0/3410165107 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c007440 con 0x7f55340ccbf0 2024-04-23T14:38:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.421+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55340cc760 con 0x7f55340ccbf0 2024-04-23T14:38:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.421+0000 7f552a000640 1 -- 172.21.15.59:0/3410165107 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c0075e0 con 0x7f55340ccbf0 2024-04-23T14:38:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.421+0000 7f552a000640 1 -- 172.21.15.59:0/3410165107 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c004a20 con 0x7f55340ccbf0 2024-04-23T14:38:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.421+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553415b050 con 0x7f55340ccbf0 2024-04-23T14:38:40.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.422+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553404fd90 con 0x7f55340ccbf0 2024-04-23T14:38:40.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.423+0000 7f552a000640 1 -- 172.21.15.59:0/3410165107 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f552c004bc0 con 0x7f55340ccbf0 2024-04-23T14:38:40.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.424+0000 7f552a000640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f551406c8f0 0x7f551406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.424+0000 7f552a000640 1 -- 172.21.15.59:0/3410165107 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f552c08aee0 con 0x7f55340ccbf0 2024-04-23T14:38:40.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.424+0000 7f5531e00640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f551406c8f0 0x7f551406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.425+0000 7f5531e00640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f551406c8f0 0x7f551406edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f55340b1350 tx=0x7f5524008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:40.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.428+0000 7f552a000640 1 -- 172.21.15.59:0/3410165107 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f552c01b030 con 0x7f55340ccbf0 2024-04-23T14:38:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.634+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55340d1eb0 con 0x7f55340ccbf0 2024-04-23T14:38:40.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.637+0000 7f552a000640 1 -- 172.21.15.59:0/3410165107 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16613 (secure 0 0 0) 0x7f55340cf030 con 0x7f55340ccbf0 2024-04-23T14:38:40.639 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:40.639 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:38:39.211023+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:40.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f551406c8f0 msgr2=0x7f551406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f551406c8f0 0x7f551406edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f55340b1350 tx=0x7f5524008040 comp rx=0 tx=0).stop 2024-04-23T14:38:40.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55340ccbf0 msgr2=0x7f5534156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55340ccbf0 0x7f5534156660 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f553408c6a0 tx=0x7f552c00c890 comp rx=0 tx=0).stop 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 shutdown_connections 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f551406c8f0 0x7f551406edb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55340ccbf0 0x7f5534156660 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55340cc2e0 0x7f55340c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 --2- 172.21.15.59:0/3410165107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5518004800 0x7f55340b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 >> 172.21.15.59:0/3410165107 conn(0x7f553409d920 msgr2=0x7f553408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 shutdown_connections 2024-04-23T14:38:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.640+0000 7f5533200640 1 -- 172.21.15.59:0/3410165107 wait complete. 2024-04-23T14:38:40.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:38:40.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.867+0000 7fcf32200640 1 Processor -- start 2024-04-23T14:38:40.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.867+0000 7fcf32200640 1 -- start start 2024-04-23T14:38:40.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.868+0000 7fcf32200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c1567e0 0x7fcf2c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.868+0000 7fcf32200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcf2c15d810 0x7fcf2c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.868+0000 7fcf32200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf2c0d0b40 0x7fcf2c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.868+0000 7fcf32200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcf2c0d0f70 con 0x7fcf2c0d0b40 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.868+0000 7fcf32200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcf2c15cab0 con 0x7fcf2c15d810 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.868+0000 7fcf32200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c160e40 con 0x7fcf2c1567e0 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf2c0d0b40 0x7fcf2c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf2c0d0b40 0x7fcf2c160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43148/0 (socket says 172.21.15.59:43148) 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 -- 172.21.15.59:0/255224964 learned_addr learned my addr 172.21.15.59:0/255224964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf23e00640 1 --2- 172.21.15.59:0/255224964 >> v2:172.21.15.76:3300/0 conn(0x7fcf2c15d810 0x7fcf2c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 -- 172.21.15.59:0/255224964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c1567e0 msgr2=0x7fcf2c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf30e00640 1 --2- 172.21.15.59:0/255224964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c1567e0 0x7fcf2c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 --2- 172.21.15.59:0/255224964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c1567e0 0x7fcf2c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 -- 172.21.15.59:0/255224964 >> v2:172.21.15.76:3300/0 conn(0x7fcf2c15d810 msgr2=0x7fcf2c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:40.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 --2- 172.21.15.59:0/255224964 >> v2:172.21.15.76:3300/0 conn(0x7fcf2c15d810 0x7fcf2c0d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf31800640 1 -- 172.21.15.59:0/255224964 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c15cd30 con 0x7fcf2c0d0b40 2024-04-23T14:38:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf23e00640 1 --2- 172.21.15.59:0/255224964 >> v2:172.21.15.76:3300/0 conn(0x7fcf2c15d810 0x7fcf2c0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.869+0000 7fcf30e00640 1 --2- 172.21.15.59:0/255224964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c1567e0 0x7fcf2c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.870+0000 7fcf31800640 1 --2- 172.21.15.59:0/255224964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c0d0b40 0x7fcf2c160110 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0b5660 tx=0x7fcf2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9dae907fea305d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.871+0000 7fcf2be00640 1 -- 172.21.15.59:0/255224964 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24036070 con 0x7fcf2c0d0b40 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.871+0000 7fcf2be00640 1 -- 172.21.15.59:0/255224964 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf24031600 con 0x7fcf2c0d0b40 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.871+0000 7fcf2be00640 1 -- 172.21.15.59:0/255224964 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf240319d0 con 0x7fcf2c0d0b40 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.871+0000 7fcf32200640 1 -- 172.21.15.59:0/255224964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c0d0b40 msgr2=0x7fcf2c160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.871+0000 7fcf32200640 1 --2- 172.21.15.59:0/255224964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c0d0b40 0x7fcf2c160110 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0b5660 tx=0x7fcf2402a6a0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.872+0000 7fcf32200640 1 -- 172.21.15.59:0/255224964 shutdown_connections 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.872+0000 7fcf32200640 1 --2- 172.21.15.59:0/255224964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c0d0b40 0x7fcf2c160110 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.872+0000 7fcf32200640 1 --2- 172.21.15.59:0/255224964 >> v2:172.21.15.76:3300/0 conn(0x7fcf2c15d810 0x7fcf2c0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.872+0000 7fcf32200640 1 --2- 172.21.15.59:0/255224964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c1567e0 0x7fcf2c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.872+0000 7fcf32200640 1 -- 172.21.15.59:0/255224964 >> 172.21.15.59:0/255224964 conn(0x7fcf2c06bdf0 msgr2=0x7fcf2c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.872+0000 7fcf32200640 1 -- 172.21.15.59:0/255224964 shutdown_connections 2024-04-23T14:38:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.872+0000 7fcf32200640 1 -- 172.21.15.59:0/255224964 wait complete. 2024-04-23T14:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.873+0000 7fcf32200640 1 Processor -- start 2024-04-23T14:38:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.874+0000 7fcf32200640 1 -- start start 2024-04-23T14:38:40.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.874+0000 7fcf32200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c0d0b40 0x7fcf2c09a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.874+0000 7fcf32200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c1567e0 0x7fcf2c159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.874+0000 7fcf32200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf2c15d810 0x7fcf2c099290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.874+0000 7fcf32200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c160650 con 0x7fcf2c1567e0 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.874+0000 7fcf32200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c15d2f0 con 0x7fcf2c15d810 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.874+0000 7fcf32200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c1607d0 con 0x7fcf2c0d0b40 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c1567e0 0x7fcf2c159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c1567e0 0x7fcf2c159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43164/0 (socket says 172.21.15.59:43164) 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 -- 172.21.15.59:0/1763848429 learned_addr learned my addr 172.21.15.59:0/1763848429 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf31800640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf2c15d810 0x7fcf2c099290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf30e00640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c0d0b40 0x7fcf2c09a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 -- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c0d0b40 msgr2=0x7fcf2c09a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c0d0b40 0x7fcf2c09a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 -- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf2c15d810 msgr2=0x7fcf2c099290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf2c15d810 0x7fcf2c099290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:40.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.875+0000 7fcf23e00640 1 -- 172.21.15.59:0/1763848429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c14f3b0 con 0x7fcf2c1567e0 2024-04-23T14:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.876+0000 7fcf23e00640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c1567e0 0x7fcf2c159990 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400c7f0 tx=0x7fcf1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.876+0000 7fcf29600640 1 -- 172.21.15.59:0/1763848429 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14015070 con 0x7fcf2c1567e0 2024-04-23T14:38:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.876+0000 7fcf29600640 1 -- 172.21.15.59:0/1763848429 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf14007a30 con 0x7fcf2c1567e0 2024-04-23T14:38:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.876+0000 7fcf29600640 1 -- 172.21.15.59:0/1763848429 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14007e00 con 0x7fcf2c1567e0 2024-04-23T14:38:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.876+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c14f690 con 0x7fcf2c1567e0 2024-04-23T14:38:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.876+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c14fc50 con 0x7fcf2c1567e0 2024-04-23T14:38:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.877+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2c04fd90 con 0x7fcf2c1567e0 2024-04-23T14:38:40.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.878+0000 7fcf29600640 1 -- 172.21.15.59:0/1763848429 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf14018070 con 0x7fcf2c1567e0 2024-04-23T14:38:40.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.879+0000 7fcf29600640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcf0806c8f0 0x7fcf0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:40.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.879+0000 7fcf29600640 1 -- 172.21.15.59:0/1763848429 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fcf1408bbb0 con 0x7fcf2c1567e0 2024-04-23T14:38:40.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.880+0000 7fcf30e00640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcf0806c8f0 0x7fcf0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:40.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.880+0000 7fcf30e00640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcf0806c8f0 0x7fcf0806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00a990 tx=0x7fcf1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:40.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:40.884+0000 7fcf29600640 1 -- 172.21.15.59:0/1763848429 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf14059f40 con 0x7fcf2c1567e0 2024-04-23T14:38:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.123+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcf2c0b5150 con 0x7fcf2c1567e0 2024-04-23T14:38:41.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.770+0000 7fcf29600640 1 -- 172.21.15.59:0/1763848429 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) ==== 118+0+0 (secure 0 0 0) 0x7fcf14059840 con 0x7fcf2c1567e0 2024-04-23T14:38:41.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcf0806c8f0 msgr2=0x7fcf0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:41.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcf0806c8f0 0x7fcf0806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00a990 tx=0x7fcf1c008040 comp rx=0 tx=0).stop 2024-04-23T14:38:41.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c1567e0 msgr2=0x7fcf2c159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:41.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c1567e0 0x7fcf2c159990 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400c7f0 tx=0x7fcf1400cbf0 comp rx=0 tx=0).stop 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 shutdown_connections 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcf0806c8f0 0x7fcf0806edb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf2c15d810 0x7fcf2c099290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf2c1567e0 0x7fcf2c159990 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 --2- 172.21.15.59:0/1763848429 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf2c0d0b40 0x7fcf2c09a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 >> 172.21.15.59:0/1763848429 conn(0x7fcf2c06bdf0 msgr2=0x7fcf2c15ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.774+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 shutdown_connections 2024-04-23T14:38:41.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:41.775+0000 7fcf32200640 1 -- 172.21.15.59:0/1763848429 wait complete. 2024-04-23T14:38:41.797 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:38:41.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:38:42.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.017+0000 7f844fe00640 1 Processor -- start 2024-04-23T14:38:42.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.018+0000 7f844fe00640 1 -- start start 2024-04-23T14:38:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.019+0000 7f844fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f84500d15b0 0x7f84500cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.019+0000 7f844fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84500b5da0 con 0x7f84500b4f20 2024-04-23T14:38:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.019+0000 7f844fe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84500d0a20 con 0x7f845015ed20 2024-04-23T14:38:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.019+0000 7f844fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f845015f100 con 0x7f84500d15b0 2024-04-23T14:38:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.019+0000 7f844f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f84500b4f20 0x7f84500cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39134/0 (socket says 172.21.15.59:39134) 2024-04-23T14:38:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.019+0000 7f844f400640 1 -- 172.21.15.59:0/3690239323 learned_addr learned my addr 172.21.15.59:0/3690239323 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.019+0000 7f8446000640 1 --2- 172.21.15.59:0/3690239323 >> v2:172.21.15.76:3300/0 conn(0x7f84500d15b0 0x7f84500cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244419985 0 0) 0x7f84500b5da0 con 0x7f84500b4f20 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8434003520 con 0x7f84500b4f20 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2283851415 0 0) 0x7f8434003520 con 0x7f84500b4f20 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 >> v2:172.21.15.76:3300/0 conn(0x7f84500d15b0 msgr2=0x7f84500cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 --2- 172.21.15.59:0/3690239323 >> v2:172.21.15.76:3300/0 conn(0x7f84500d15b0 0x7f84500cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 >> v1:172.21.15.76:6789/0 conn(0x7f845015ed20 legacy=0x7f84500d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84500d2ed0 con 0x7f84500b4f20 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8448001670 con 0x7f84500b4f20 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 >> v1:172.21.15.59:6789/0 conn(0x7f84500b4f20 legacy=0x7f84500cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8434003a00 0x7f8434003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8434004800 0x7f8434004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f843400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f843400c1b0 con 0x7f84340052e0 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f843400c360 con 0x7f8434003a00 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.020+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f843400c560 con 0x7f8434004800 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.021+0000 7f844f400640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f843400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.021+0000 7f844ea00640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8434003a00 0x7f8434003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.021+0000 7f844f400640 1 -- 172.21.15.59:0/3690239323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8434004800 msgr2=0x7f8434004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.021+0000 7f844f400640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8434004800 0x7f8434004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.021+0000 7f844f400640 1 -- 172.21.15.59:0/3690239323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8434003a00 msgr2=0x7f8434003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.021+0000 7f844f400640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8434003a00 0x7f8434003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.021+0000 7f844f400640 1 -- 172.21.15.59:0/3690239323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8434003520 con 0x7f84340052e0 2024-04-23T14:38:42.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.022+0000 7f844f400640 1 -- 172.21.15.59:0/3690239323 reap_dead start 2024-04-23T14:38:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.022+0000 7f844f400640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f843400b9c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f844800c190 tx=0x7f8434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32898e15ff790b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.022+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448007c60 con 0x7f84340052e0 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.022+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8448005b10 con 0x7f84340052e0 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.022+0000 7f844e000640 1 -- 172.21.15.59:0/3690239323 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448007070 con 0x7f84340052e0 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.023+0000 7f844fe00640 1 -- 172.21.15.59:0/3690239323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 msgr2=0x7f843400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.023+0000 7f844fe00640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f843400b9c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f844800c190 tx=0x7f8434005830 comp rx=0 tx=0).stop 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.023+0000 7f844fe00640 1 -- 172.21.15.59:0/3690239323 shutdown_connections 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.023+0000 7f844fe00640 1 --2- 172.21.15.59:0/3690239323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f843400b9c0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.023+0000 7f844fe00640 1 -- 172.21.15.59:0/3690239323 >> 172.21.15.59:0/3690239323 conn(0x7f845008bf90 msgr2=0x7f84500d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.023+0000 7f844fe00640 1 -- 172.21.15.59:0/3690239323 shutdown_connections 2024-04-23T14:38:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.023+0000 7f844fe00640 1 -- 172.21.15.59:0/3690239323 wait complete. 2024-04-23T14:38:42.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.025+0000 7f844fe00640 1 Processor -- start 2024-04-23T14:38:42.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.025+0000 7f844fe00640 1 -- start start 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f845014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84500cf930 0x7f84500cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84500cfd90 0x7f84500ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84500b3600 con 0x7f84340052e0 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84500d3490 con 0x7f84500cfd90 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f84500b3780 con 0x7f84500cf930 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f845014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f845014f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43182/0 (socket says 172.21.15.59:43182) 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.026+0000 7f844ea00640 1 -- 172.21.15.59:0/784648283 learned_addr learned my addr 172.21.15.59:0/784648283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844f400640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84500cfd90 0x7f84500ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844ea00640 1 -- 172.21.15.59:0/784648283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84500cf930 msgr2=0x7f84500cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844ea00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84500cf930 0x7f84500cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844ea00640 1 -- 172.21.15.59:0/784648283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84500cfd90 msgr2=0x7f84500ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844ea00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84500cfd90 0x7f84500ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844ea00640 1 -- 172.21.15.59:0/784648283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845014fa00 con 0x7f84340052e0 2024-04-23T14:38:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844ea00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f845014f4c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f844000e920 tx=0x7f844000ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f8447400640 1 -- 172.21.15.59:0/784648283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440003b80 con 0x7f84340052e0 2024-04-23T14:38:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f8447400640 1 -- 172.21.15.59:0/784648283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8440018070 con 0x7f84340052e0 2024-04-23T14:38:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.027+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84501588a0 con 0x7f84340052e0 2024-04-23T14:38:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.028+0000 7f8447400640 1 -- 172.21.15.59:0/784648283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84400050d0 con 0x7f84340052e0 2024-04-23T14:38:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.028+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8450158e60 con 0x7f84340052e0 2024-04-23T14:38:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.029+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84500d2a10 con 0x7f84340052e0 2024-04-23T14:38:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.030+0000 7f8447400640 1 -- 172.21.15.59:0/784648283 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8440005270 con 0x7f84340052e0 2024-04-23T14:38:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.031+0000 7f8447400640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f842c06c920 0x7f842c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.031+0000 7f8447400640 1 -- 172.21.15.59:0/784648283 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f844008bb70 con 0x7f84340052e0 2024-04-23T14:38:42.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.031+0000 7f8446000640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f842c06c920 0x7f842c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.032+0000 7f8446000640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f842c06c920 0x7f842c06ede0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8438006a90 tx=0x7f8438006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:42.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.035+0000 7f8447400640 1 -- 172.21.15.59:0/784648283 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84500d2a10 con 0x7f84340052e0 2024-04-23T14:38:42.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.285+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f845004fd90 con 0x7f84340052e0 2024-04-23T14:38:42.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.773+0000 7f8447400640 1 -- 172.21.15.59:0/784648283 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) ==== 122+0+0 (secure 0 0 0) 0x7f844001a310 con 0x7f84340052e0 2024-04-23T14:38:42.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f842c06c920 msgr2=0x7f842c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f842c06c920 0x7f842c06ede0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8438006a90 tx=0x7f8438006a20 comp rx=0 tx=0).stop 2024-04-23T14:38:42.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 msgr2=0x7f845014f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f845014f4c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f844000e920 tx=0x7f844000ede0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 shutdown_connections 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f842c06c920 0x7f842c06ede0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84500cf930 0x7f84500cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84500cfd90 0x7f84500ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 --2- 172.21.15.59:0/784648283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84340052e0 0x7f845014f4c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 >> 172.21.15.59:0/784648283 conn(0x7f845008bf90 msgr2=0x7f84500d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 shutdown_connections 2024-04-23T14:38:42.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:42.777+0000 7f844fe00640 1 -- 172.21.15.59:0/784648283 wait complete. 2024-04-23T14:38:42.801 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:38:42.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:43.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.012+0000 7f9a99800640 1 Processor -- start 2024-04-23T14:38:43.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.012+0000 7f9a99800640 1 -- start start 2024-04-23T14:38:43.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.013+0000 7f9a99800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a940cd760 0x7f9a940b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.013+0000 7f9a99800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9a940b4690 0x7f9a940cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.013+0000 7f9a99800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a94160210 con 0x7f9a9415c3d0 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.013+0000 7f9a99800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9a941604f0 con 0x7f9a940cd760 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.013+0000 7f9a99800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9a941607d0 con 0x7f9a940b4690 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a8b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a940cd760 0x7f9a940b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a8b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a940cd760 0x7f9a940b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43184/0 (socket says 172.21.15.59:43184) 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a8b400640 1 -- 172.21.15.59:0/4154464547 learned_addr learned my addr 172.21.15.59:0/4154464547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a98e00640 1 --2- 172.21.15.59:0/4154464547 >> v2:172.21.15.76:3300/0 conn(0x7f9a940b4690 0x7f9a940cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a8b400640 1 -- 172.21.15.59:0/4154464547 >> v2:172.21.15.76:3300/0 conn(0x7f9a940b4690 msgr2=0x7f9a940cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:43.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a8b400640 1 --2- 172.21.15.59:0/4154464547 >> v2:172.21.15.76:3300/0 conn(0x7f9a940b4690 0x7f9a940cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a8b400640 1 -- 172.21.15.59:0/4154464547 >> v1:172.21.15.76:6789/0 conn(0x7f9a9415c3d0 legacy=0x7f9a940cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:43.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.014+0000 7f9a8b400640 1 -- 172.21.15.59:0/4154464547 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a94160ab0 con 0x7f9a940cd760 2024-04-23T14:38:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.015+0000 7f9a8b400640 1 --2- 172.21.15.59:0/4154464547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940b4150 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9a940b2de0 tx=0x7f9a7c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9210f1a0e48f688 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a93400640 1 -- 172.21.15.59:0/4154464547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c037070 con 0x7f9a940cd760 2024-04-23T14:38:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a93400640 1 -- 172.21.15.59:0/4154464547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c032440 con 0x7f9a940cd760 2024-04-23T14:38:43.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a93400640 1 -- 172.21.15.59:0/4154464547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c032810 con 0x7f9a940cd760 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a99800640 1 -- 172.21.15.59:0/4154464547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 msgr2=0x7f9a940b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a99800640 1 --2- 172.21.15.59:0/4154464547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940b4150 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9a940b2de0 tx=0x7f9a7c02a230 comp rx=0 tx=0).stop 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a99800640 1 -- 172.21.15.59:0/4154464547 shutdown_connections 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a99800640 1 --2- 172.21.15.59:0/4154464547 >> v2:172.21.15.76:3300/0 conn(0x7f9a940b4690 0x7f9a940cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a99800640 1 --2- 172.21.15.59:0/4154464547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940b4150 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.016+0000 7f9a99800640 1 -- 172.21.15.59:0/4154464547 >> 172.21.15.59:0/4154464547 conn(0x7f9a940b21a0 msgr2=0x7f9a940b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.017+0000 7f9a99800640 1 -- 172.21.15.59:0/4154464547 shutdown_connections 2024-04-23T14:38:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.017+0000 7f9a99800640 1 -- 172.21.15.59:0/4154464547 wait complete. 2024-04-23T14:38:43.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.019+0000 7f9a99800640 1 Processor -- start 2024-04-23T14:38:43.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.019+0000 7f9a99800640 1 -- start start 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a99800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a940b4690 0x7f9a940ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a99800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940cf570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a99800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9415c3d0 0x7f9a94152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a99800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9415c7b0 con 0x7f9a940cd760 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a99800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a940b4aa0 con 0x7f9a9415c3d0 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a99800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9415c930 con 0x7f9a940b4690 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a8b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a8b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43194/0 (socket says 172.21.15.59:43194) 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.020+0000 7f9a8b400640 1 -- 172.21.15.59:0/2828153926 learned_addr learned my addr 172.21.15.59:0/2828153926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a8b400640 1 -- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a940b4690 msgr2=0x7f9a940ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a8b400640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a940b4690 0x7f9a940ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a8b400640 1 -- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9415c3d0 msgr2=0x7f9a94152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a98e00640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9415c3d0 0x7f9a94152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a8b400640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9415c3d0 0x7f9a94152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a8b400640 1 -- 172.21.15.59:0/2828153926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a940aee90 con 0x7f9a940cd760 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a98e00640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9415c3d0 0x7f9a94152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a8b400640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940cf570 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c000c00 tx=0x7f9a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a90c00640 1 -- 172.21.15.59:0/2828153926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c041020 con 0x7f9a940cd760 2024-04-23T14:38:43.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a90c00640 1 -- 172.21.15.59:0/2828153926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0306c0 con 0x7f9a940cd760 2024-04-23T14:38:43.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.021+0000 7f9a90c00640 1 -- 172.21.15.59:0/2828153926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c037040 con 0x7f9a940cd760 2024-04-23T14:38:43.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.022+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a940af170 con 0x7f9a940cd760 2024-04-23T14:38:43.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.022+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a940af730 con 0x7f9a940cd760 2024-04-23T14:38:43.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.023+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9404fd90 con 0x7f9a940cd760 2024-04-23T14:38:43.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.024+0000 7f9a90c00640 1 -- 172.21.15.59:0/2828153926 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a7c044020 con 0x7f9a940cd760 2024-04-23T14:38:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.025+0000 7f9a90c00640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a7006c8f0 0x7f9a7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.025+0000 7f9a90c00640 1 -- 172.21.15.59:0/2828153926 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f9a7c0ac450 con 0x7f9a940cd760 2024-04-23T14:38:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.025+0000 7f9a93e00640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a7006c8f0 0x7f9a7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.026+0000 7f9a93e00640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a7006c8f0 0x7f9a7006edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9a84006fd0 tx=0x7f9a84006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.030+0000 7f9a90c00640 1 -- 172.21.15.59:0/2828153926 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a7c04b030 con 0x7f9a940cd760 2024-04-23T14:38:43.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.275+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a9406e400 con 0x7f9a940cd760 2024-04-23T14:38:43.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.276+0000 7f9a90c00640 1 -- 172.21.15.59:0/2828153926 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5183 (secure 0 0 0) 0x7f9a7c07a7e0 con 0x7f9a940cd760 2024-04-23T14:38:43.277 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:43.278 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:38:42.286820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3870021000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3870021000},{"type":"v1","addr":"172.21.15.76:6835","nonce":3870021000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":6}]} 2024-04-23T14:38:43.278 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 60 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.279+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a7006c8f0 msgr2=0x7f9a7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.279+0000 7f9a99800640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a7006c8f0 0x7f9a7006edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9a84006fd0 tx=0x7f9a84006bc0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.279+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 msgr2=0x7f9a940cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.279+0000 7f9a99800640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940cf570 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c000c00 tx=0x7f9a7c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 shutdown_connections 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a7006c8f0 0x7f9a7006edb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9415c3d0 0x7f9a94152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a940cd760 0x7f9a940cf570 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 --2- 172.21.15.59:0/2828153926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a940b4690 0x7f9a940ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 >> 172.21.15.59:0/2828153926 conn(0x7f9a940b21a0 msgr2=0x7f9a940cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 shutdown_connections 2024-04-23T14:38:43.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.280+0000 7f9a99800640 1 -- 172.21.15.59:0/2828153926 wait complete. 2024-04-23T14:38:43.301 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:43.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.510+0000 7fa16b200640 1 Processor -- start 2024-04-23T14:38:43.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.510+0000 7fa16b200640 1 -- start start 2024-04-23T14:38:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.512+0000 7fa16b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 0x7fa16c167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.512+0000 7fa16b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa16c160210 con 0x7fa16c15c3d0 2024-04-23T14:38:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.512+0000 7fa16b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa16c1604f0 con 0x7fa16c0cd760 2024-04-23T14:38:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.512+0000 7fa16b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c1607d0 con 0x7fa16c0cc7e0 2024-04-23T14:38:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.512+0000 7fa169e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa16c15c3d0 0x7fa16c0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39148/0 (socket says 172.21.15.59:39148) 2024-04-23T14:38:43.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.512+0000 7fa169e00640 1 -- 172.21.15.59:0/4002964294 learned_addr learned my addr 172.21.15.59:0/4002964294 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.512+0000 7fa16a800640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 0x7fa16c167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2327531764 0 0) 0x7fa16c160210 con 0x7fa16c15c3d0 2024-04-23T14:38:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa150003520 con 0x7fa16c15c3d0 2024-04-23T14:38:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3021604126 0 0) 0x7fa150003520 con 0x7fa16c15c3d0 2024-04-23T14:38:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 msgr2=0x7fa16c167400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:43.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 0x7fa16c167400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 >> v1:172.21.15.76:6789/0 conn(0x7fa16c0cd760 legacy=0x7fa16c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c160ab0 con 0x7fa16c15c3d0 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa15c001670 con 0x7fa16c15c3d0 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 >> v1:172.21.15.59:6789/0 conn(0x7fa16c15c3d0 legacy=0x7fa16c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa150003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 0x7fa150004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa150004d40 0x7fa1500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c1607d0 con 0x7fa1500039e0 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa15000ba70 con 0x7fa150004d40 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.513+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa15000bc40 con 0x7fa16c0cc7e0 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa169e00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa150003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa169400640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 0x7fa150004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa16a800640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa150004d40 0x7fa1500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa169e00640 1 -- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 msgr2=0x7fa150004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa169e00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cc7e0 0x7fa150004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa169e00640 1 -- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa150004d40 msgr2=0x7fa1500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa169e00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa150004d40 0x7fa1500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.514+0000 7fa169e00640 1 -- 172.21.15.59:0/4002964294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa150003520 con 0x7fa1500039e0 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.515+0000 7fa169e00640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa150003dc0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c1c0 tx=0x7fa150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259633183990b6ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c007c60 con 0x7fa1500039e0 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa15c005ad0 con 0x7fa1500039e0 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa168a00640 1 -- 172.21.15.59:0/4002964294 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c007070 con 0x7fa1500039e0 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa16b200640 1 -- 172.21.15.59:0/4002964294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 msgr2=0x7fa150003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa16b200640 1 --2- 172.21.15.59:0/4002964294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa150003dc0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c1c0 tx=0x7fa150005830 comp rx=0 tx=0).stop 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa16a800640 1 -- 172.21.15.59:0/4002964294 reap_dead start 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa16b200640 1 -- 172.21.15.59:0/4002964294 shutdown_connections 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa16b200640 1 -- 172.21.15.59:0/4002964294 >> 172.21.15.59:0/4002964294 conn(0x7fa16c0b21a0 msgr2=0x7fa16c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:43.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa16b200640 1 -- 172.21.15.59:0/4002964294 shutdown_connections 2024-04-23T14:38:43.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.516+0000 7fa16b200640 1 -- 172.21.15.59:0/4002964294 wait complete. 2024-04-23T14:38:43.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.518+0000 7fa16b200640 1 Processor -- start 2024-04-23T14:38:43.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.518+0000 7fa16b200640 1 -- start start 2024-04-23T14:38:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.519+0000 7fa16b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa16c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.519+0000 7fa16b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cec90 0x7fa16c1f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.519+0000 7fa16b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa16c1f4f60 0x7fa16c1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.519+0000 7fa16b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c15c930 con 0x7fa1500039e0 2024-04-23T14:38:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.519+0000 7fa16b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c15c7b0 con 0x7fa16c1f4f60 2024-04-23T14:38:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.519+0000 7fa16b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c0b4860 con 0x7fa16c0cec90 2024-04-23T14:38:43.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa16c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa16c0ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43204/0 (socket says 172.21.15.59:43204) 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 -- 172.21.15.59:0/125071891 learned_addr learned my addr 172.21.15.59:0/125071891 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa16a800640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa16c1f4f60 0x7fa16c1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169400640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cec90 0x7fa16c1f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 -- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cec90 msgr2=0x7fa16c1f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cec90 0x7fa16c1f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 -- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa16c1f4f60 msgr2=0x7fa16c1f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa16c1f4f60 0x7fa16c1f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.520+0000 7fa169e00640 1 -- 172.21.15.59:0/125071891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c0cfd70 con 0x7fa1500039e0 2024-04-23T14:38:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.521+0000 7fa169e00640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa16c0ce750 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c1c0 tx=0x7fa16c1603e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.521+0000 7fa162000640 1 -- 172.21.15.59:0/125071891 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c043070 con 0x7fa1500039e0 2024-04-23T14:38:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.521+0000 7fa162000640 1 -- 172.21.15.59:0/125071891 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa15c038040 con 0x7fa1500039e0 2024-04-23T14:38:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.521+0000 7fa162000640 1 -- 172.21.15.59:0/125071891 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c007070 con 0x7fa1500039e0 2024-04-23T14:38:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.521+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16c0cf1f0 con 0x7fa1500039e0 2024-04-23T14:38:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.521+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16c0cf7b0 con 0x7fa1500039e0 2024-04-23T14:38:43.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.522+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa16c04fd90 con 0x7fa1500039e0 2024-04-23T14:38:43.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.523+0000 7fa162000640 1 -- 172.21.15.59:0/125071891 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa15c007210 con 0x7fa1500039e0 2024-04-23T14:38:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.525+0000 7fa162000640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14806c8f0 0x7fa14806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.525+0000 7fa162000640 1 -- 172.21.15.59:0/125071891 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa15c0ab770 con 0x7fa1500039e0 2024-04-23T14:38:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.525+0000 7fa169400640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14806c8f0 0x7fa14806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.526+0000 7fa169400640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14806c8f0 0x7fa14806edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa154003fd0 tx=0x7fa154009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.529+0000 7fa162000640 1 -- 172.21.15.59:0/125071891 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa15c079b00 con 0x7fa1500039e0 2024-04-23T14:38:43.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.734+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa16c06e400 con 0x7fa1500039e0 2024-04-23T14:38:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.736+0000 7fa162000640 1 -- 172.21.15.59:0/125071891 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16600 (secure 0 0 0) 0x7fa15c005aa0 con 0x7fa1500039e0 2024-04-23T14:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:43.738 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:38:40.747863+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14806c8f0 msgr2=0x7fa14806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14806c8f0 0x7fa14806edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa154003fd0 tx=0x7fa154009040 comp rx=0 tx=0).stop 2024-04-23T14:38:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 msgr2=0x7fa16c0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa16c0ce750 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c1c0 tx=0x7fa16c1603e0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 shutdown_connections 2024-04-23T14:38:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14806c8f0 0x7fa14806edb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa16c0cec90 0x7fa16c1f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa16c1f4f60 0x7fa16c1f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 --2- 172.21.15.59:0/125071891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1500039e0 0x7fa16c0ce750 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.738+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 >> 172.21.15.59:0/125071891 conn(0x7fa16c0b21a0 msgr2=0x7fa16c06ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:43.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.739+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 shutdown_connections 2024-04-23T14:38:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.739+0000 7fa16b200640 1 -- 172.21.15.59:0/125071891 wait complete. 2024-04-23T14:38:43.761 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:38:43.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:38:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.968+0000 7fd44f200640 1 Processor -- start 2024-04-23T14:38:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.968+0000 7fd44f200640 1 -- start start 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.969+0000 7fd44f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd45015c3d0 0x7fd4500cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.970+0000 7fd44f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd450160210 con 0x7fd4500b4860 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.970+0000 7fd44f200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4501604f0 con 0x7fd4500cd590 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.970+0000 7fd44f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd4501607d0 con 0x7fd45015c3d0 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.970+0000 7fd44de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd45015c3d0 0x7fd4500cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.970+0000 7fd44de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd45015c3d0 0x7fd4500cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43216/0 (socket says 172.21.15.59:43216) 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.970+0000 7fd44de00640 1 -- 172.21.15.59:0/2209403227 learned_addr learned my addr 172.21.15.59:0/2209403227 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.970+0000 7fd44e800640 1 --1- 172.21.15.59:0/2209403227 >> v1:172.21.15.59:6789/0 conn(0x7fd4500b4860 0x7fd4500cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39162/0 (socket says 172.21.15.59:39162) 2024-04-23T14:38:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.971+0000 7fd44de00640 1 -- 172.21.15.59:0/2209403227 >> v1:172.21.15.76:6789/0 conn(0x7fd4500cd590 legacy=0x7fd4500b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.971+0000 7fd44de00640 1 -- 172.21.15.59:0/2209403227 >> v1:172.21.15.59:6789/0 conn(0x7fd4500b4860 legacy=0x7fd4500cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.971+0000 7fd44de00640 1 -- 172.21.15.59:0/2209403227 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd450160ab0 con 0x7fd45015c3d0 2024-04-23T14:38:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.971+0000 7fd44ca00640 1 -- 172.21.15.59:0/2209403227 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827719064 0 0) 0x7fd450160210 con 0x7fd4500b4860 2024-04-23T14:38:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.972+0000 7fd44de00640 1 --2- 172.21.15.59:0/2209403227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd45015c3d0 0x7fd4500cd050 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9230825114ea880 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.972+0000 7fd44ca00640 1 -- 172.21.15.59:0/2209403227 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440037070 con 0x7fd45015c3d0 2024-04-23T14:38:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.972+0000 7fd44ca00640 1 -- 172.21.15.59:0/2209403227 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440032440 con 0x7fd45015c3d0 2024-04-23T14:38:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.972+0000 7fd44ca00640 1 -- 172.21.15.59:0/2209403227 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440032810 con 0x7fd45015c3d0 2024-04-23T14:38:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.973+0000 7fd44f200640 1 -- 172.21.15.59:0/2209403227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd45015c3d0 msgr2=0x7fd4500cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.973+0000 7fd44f200640 1 --2- 172.21.15.59:0/2209403227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd45015c3d0 0x7fd4500cd050 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a2a0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.973+0000 7fd44f200640 1 -- 172.21.15.59:0/2209403227 shutdown_connections 2024-04-23T14:38:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.973+0000 7fd44f200640 1 --2- 172.21.15.59:0/2209403227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd45015c3d0 0x7fd4500cd050 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.973+0000 7fd44f200640 1 -- 172.21.15.59:0/2209403227 >> 172.21.15.59:0/2209403227 conn(0x7fd4500b21a0 msgr2=0x7fd4500b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.973+0000 7fd44f200640 1 -- 172.21.15.59:0/2209403227 shutdown_connections 2024-04-23T14:38:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.973+0000 7fd44f200640 1 -- 172.21.15.59:0/2209403227 wait complete. 2024-04-23T14:38:43.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.975+0000 7fd44f200640 1 Processor -- start 2024-04-23T14:38:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.975+0000 7fd44f200640 1 -- start start 2024-04-23T14:38:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4500b4860 0x7fd4500c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4500cd590 0x7fd45015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd45015c3d0 0x7fd4500d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd45015c930 con 0x7fd4500b4860 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd45015c7b0 con 0x7fd45015c3d0 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd4500cbee0 con 0x7fd4500cd590 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4500b4860 0x7fd4500c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4500b4860 0x7fd4500c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43232/0 (socket says 172.21.15.59:43232) 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 -- 172.21.15.59:0/1101202411 learned_addr learned my addr 172.21.15.59:0/1101202411 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 -- 172.21.15.59:0/1101202411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4500cd590 msgr2=0x7fd45015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4500cd590 0x7fd45015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 -- 172.21.15.59:0/1101202411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd45015c3d0 msgr2=0x7fd4500d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd45015c3d0 0x7fd4500d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.976+0000 7fd44de00640 1 -- 172.21.15.59:0/1101202411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45015e8a0 con 0x7fd4500b4860 2024-04-23T14:38:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.977+0000 7fd44de00640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4500b4860 0x7fd4500c5dc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd4400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.977+0000 7fd446000640 1 -- 172.21.15.59:0/1101202411 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440037040 con 0x7fd4500b4860 2024-04-23T14:38:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.977+0000 7fd446000640 1 -- 172.21.15.59:0/1101202411 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440004250 con 0x7fd4500b4860 2024-04-23T14:38:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.977+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45015eb80 con 0x7fd4500b4860 2024-04-23T14:38:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.977+0000 7fd446000640 1 -- 172.21.15.59:0/1101202411 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44003c670 con 0x7fd4500b4860 2024-04-23T14:38:43.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.978+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45015f140 con 0x7fd4500b4860 2024-04-23T14:38:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.979+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45004fd90 con 0x7fd4500b4860 2024-04-23T14:38:43.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.980+0000 7fd446000640 1 -- 172.21.15.59:0/1101202411 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd44003a070 con 0x7fd4500b4860 2024-04-23T14:38:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.981+0000 7fd446000640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd43006c8f0 0x7fd43006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.981+0000 7fd446000640 1 -- 172.21.15.59:0/1101202411 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fd4400ac590 con 0x7fd4500b4860 2024-04-23T14:38:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.981+0000 7fd44d400640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd43006c8f0 0x7fd43006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:43.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.982+0000 7fd44d400640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd43006c8f0 0x7fd43006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd45015a9f0 tx=0x7fd43c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:43.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:43.985+0000 7fd446000640 1 -- 172.21.15.59:0/1101202411 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd440041020 con 0x7fd4500b4860 2024-04-23T14:38:44.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.188+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd45006e400 con 0x7fd43006c8f0 2024-04-23T14:38:44.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.200+0000 7fd446000640 1 -- 172.21.15.59:0/1101202411 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246754 (secure 0 0 0) 0x7fd45006e400 con 0x7fd43006c8f0 2024-04-23T14:38:44.204 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:44.206 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:38:44.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd43006c8f0 msgr2=0x7fd43006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd43006c8f0 0x7fd43006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd45015a9f0 tx=0x7fd43c0079b0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4500b4860 msgr2=0x7fd4500c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4500b4860 0x7fd4500c5dc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd4400046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 shutdown_connections 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd43006c8f0 0x7fd43006edb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4500cd590 0x7fd45015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd45015c3d0 0x7fd4500d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 --2- 172.21.15.59:0/1101202411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4500b4860 0x7fd4500c5dc0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 >> 172.21.15.59:0/1101202411 conn(0x7fd4500b21a0 msgr2=0x7fd4500af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 shutdown_connections 2024-04-23T14:38:44.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.205+0000 7fd44f200640 1 -- 172.21.15.59:0/1101202411 wait complete. 2024-04-23T14:38:44.226 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-04-23T14:38:42.496588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258380,"kb_used_data":8912,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716340,"statfs":{"total":773094113280,"available":772829532160,"internally_reserved":0,"allocated":9125888,"data_stored":4549042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.782236"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.743185+0000","last_change":"2024-04-23T14:38:38.743185+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.743185+0000","last_undegraded":"2024-04-23T14:38:38.743185+0000","last_fullsized":"2024-04-23T14:38:38.743185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.747534+0000","last_change":"2024-04-23T14:38:38.747534+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.747534+0000","last_undegraded":"2024-04-23T14:38:38.747534+0000","last_fullsized":"2024-04-23T14:38:38.747534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.757889+0000","last_change":"2024-04-23T14:38:38.757889+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.757889+0000","last_undegraded":"2024-04-23T14:38:38.757889+0000","last_fullsized":"2024-04-23T14:38:38.757889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.742686+0000","last_change":"2024-04-23T14:38:38.742686+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.742686+0000","last_undegraded":"2024-04-23T14:38:38.742686+0000","last_fullsized":"2024-04-23T14:38:38.742686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.755255+0000","last_change":"2024-04-23T14:38:38.755255+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.755255+0000","last_undegraded":"2024-04-23T14:38:38.755255+0000","last_fullsized":"2024-04-23T14:38:38.755255+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.745777+0000","last_change":"2024-04-23T14:38:38.745777+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.745777+0000","last_undegraded":"2024-04-23T14:38:38.745777+0000","last_fullsized":"2024-04-23T14:38:38.745777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.740323+0000","last_change":"2024-04-23T14:38:38.740323+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.740323+0000","last_undegraded":"2024-04-23T14:38:38.740323+0000","last_fullsized":"2024-04-23T14:38:38.740323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.745123+0000","last_change":"2024-04-23T14:38:38.745123+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.745123+0000","last_undegraded":"2024-04-23T14:38:38.745123+0000","last_fullsized":"2024-04-23T14:38:38.745123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.747624+0000","last_change":"2024-04-23T14:38:38.747624+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.747624+0000","last_undegraded":"2024-04-23T14:38:38.747624+0000","last_fullsized":"2024-04-23T14:38:38.747624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.738915+0000","last_change":"2024-04-23T14:38:38.738915+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.738915+0000","last_undegraded":"2024-04-23T14:38:38.738915+0000","last_fullsized":"2024-04-23T14:38:38.738915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.748693+0000","last_change":"2024-04-23T14:38:38.748693+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.748693+0000","last_undegraded":"2024-04-23T14:38:38.748693+0000","last_fullsized":"2024-04-23T14:38:38.748693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.743165+0000","last_change":"2024-04-23T14:38:38.743165+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.743165+0000","last_undegraded":"2024-04-23T14:38:38.743165+0000","last_fullsized":"2024-04-23T14:38:38.743165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.756431+0000","last_change":"2024-04-23T14:38:38.756431+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.756431+0000","last_undegraded":"2024-04-23T14:38:38.756431+0000","last_fullsized":"2024-04-23T14:38:38.756431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.738852+0000","last_change":"2024-04-23T14:38:38.738852+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.738852+0000","last_undegraded":"2024-04-23T14:38:38.738852+0000","last_fullsized":"2024-04-23T14:38:38.738852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.745894+0000","last_change":"2024-04-23T14:38:38.745894+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.745894+0000","last_undegraded":"2024-04-23T14:38:38.745894+0000","last_fullsized":"2024-04-23T14:38:38.745894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.752502+0000","last_change":"2024-04-23T14:38:38.752502+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.752502+0000","last_undegraded":"2024-04-23T14:38:38.752502+0000","last_fullsized":"2024-04-23T14:38:38.752502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.754315+0000","last_change":"2024-04-23T14:38:38.754315+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.754315+0000","last_undegraded":"2024-04-23T14:38:38.754315+0000","last_fullsized":"2024-04-23T14:38:38.754315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.737388+0000","last_change":"2024-04-23T14:38:38.737388+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.737388+0000","last_undegraded":"2024-04-23T14:38:38.737388+0000","last_fullsized":"2024-04-23T14:38:38.737388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.739744+0000","last_change":"2024-04-23T14:38:38.739744+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.739744+0000","last_undegraded":"2024-04-23T14:38:38.739744+0000","last_fullsized":"2024-04-23T14:38:38.739744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.736651+0000","last_change":"2024-04-23T14:38:38.736651+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.736651+0000","last_undegraded":"2024-04-23T14:38:38.736651+0000","last_fullsized":"2024-04-23T14:38:38.736651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.735981+0000","last_change":"2024-04-23T14:38:38.735981+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.735981+0000","last_undegraded":"2024-04-23T14:38:38.735981+0000","last_fullsized":"2024-04-23T14:38:38.735981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.747780+0000","last_change":"2024-04-23T14:38:38.747780+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.747780+0000","last_undegraded":"2024-04-23T14:38:38.747780+0000","last_fullsized":"2024-04-23T14:38:38.747780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.742425+0000","last_change":"2024-04-23T14:38:38.742425+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.742425+0000","last_undegraded":"2024-04-23T14:38:38.742425+0000","last_fullsized":"2024-04-23T14:38:38.742425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.735816+0000","last_change":"2024-04-23T14:38:38.735816+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.735816+0000","last_undegraded":"2024-04-23T14:38:38.735816+0000","last_fullsized":"2024-04-23T14:38:38.735816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.747534+0000","last_change":"2024-04-23T14:38:38.747534+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.747534+0000","last_undegraded":"2024-04-23T14:38:38.747534+0000","last_fullsized":"2024-04-23T14:38:38.747534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.758009+0000","last_change":"2024-04-23T14:38:38.758009+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.758009+0000","last_undegraded":"2024-04-23T14:38:38.758009+0000","last_fullsized":"2024-04-23T14:38:38.758009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.757236+0000","last_change":"2024-04-23T14:38:38.757236+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.757236+0000","last_undegraded":"2024-04-23T14:38:38.757236+0000","last_fullsized":"2024-04-23T14:38:38.757236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.741471+0000","last_change":"2024-04-23T14:38:38.741471+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.741471+0000","last_undegraded":"2024-04-23T14:38:38.741471+0000","last_fullsized":"2024-04-23T14:38:38.741471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.753040+0000","last_change":"2024-04-23T14:38:38.753040+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.753040+0000","last_undegraded":"2024-04-23T14:38:38.753040+0000","last_fullsized":"2024-04-23T14:38:38.753040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.747272+0000","last_change":"2024-04-23T14:38:38.747272+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.747272+0000","last_undegraded":"2024-04-23T14:38:38.747272+0000","last_fullsized":"2024-04-23T14:38:38.747272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.752821+0000","last_change":"2024-04-23T14:38:38.752821+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.752821+0000","last_undegraded":"2024-04-23T14:38:38.752821+0000","last_fullsized":"2024-04-23T14:38:38.752821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.733821+0000","last_change":"2024-04-23T14:38:38.733821+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.733821+0000","last_undegraded":"2024-04-23T14:38:38.733821+0000","last_fullsized":"2024-04-23T14:38:38.733821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.759190+0000","last_change":"2024-04-23T14:38:38.759190+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.759190+0000","last_undegraded":"2024-04-23T14:38:38.759190+0000","last_fullsized":"2024-04-23T14:38:38.759190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.757860+0000","last_change":"2024-04-23T14:38:38.754157+0000","last_active":"2024-04-23T14:38:38.757860+0000","last_peered":"2024-04-23T14:38:38.757860+0000","last_clean":"2024-04-23T14:38:38.757860+0000","last_became_active":"2024-04-23T14:38:38.753093+0000","last_became_peered":"2024-04-23T14:38:38.753093+0000","last_unstale":"2024-04-23T14:38:38.757860+0000","last_undegraded":"2024-04-23T14:38:38.757860+0000","last_fullsized":"2024-04-23T14:38:38.757860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:57.269105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.742451+0000","last_change":"2024-04-23T14:38:38.742451+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.742451+0000","last_undegraded":"2024-04-23T14:38:38.742451+0000","last_fullsized":"2024-04-23T14:38:38.742451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.768362+0000","last_change":"2024-04-23T14:38:38.764941+0000","last_active":"2024-04-23T14:38:38.768362+0000","last_peered":"2024-04-23T14:38:38.768362+0000","last_clean":"2024-04-23T14:38:38.768362+0000","last_became_active":"2024-04-23T14:38:38.764556+0000","last_became_peered":"2024-04-23T14:38:38.764556+0000","last_unstale":"2024-04-23T14:38:38.768362+0000","last_undegraded":"2024-04-23T14:38:38.768362+0000","last_fullsized":"2024-04-23T14:38:38.768362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:08:14.646602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.740464+0000","last_change":"2024-04-23T14:38:38.740464+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.740464+0000","last_undegraded":"2024-04-23T14:38:38.740464+0000","last_fullsized":"2024-04-23T14:38:38.740464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.750822+0000","last_change":"2024-04-23T14:38:38.748098+0000","last_active":"2024-04-23T14:38:38.750822+0000","last_peered":"2024-04-23T14:38:38.750822+0000","last_clean":"2024-04-23T14:38:38.750822+0000","last_became_active":"2024-04-23T14:38:38.746024+0000","last_became_peered":"2024-04-23T14:38:38.746024+0000","last_unstale":"2024-04-23T14:38:38.750822+0000","last_undegraded":"2024-04-23T14:38:38.750822+0000","last_fullsized":"2024-04-23T14:38:38.750822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:31.849203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.757425+0000","last_change":"2024-04-23T14:38:38.757425+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.757425+0000","last_undegraded":"2024-04-23T14:38:38.757425+0000","last_fullsized":"2024-04-23T14:38:38.757425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.757077+0000","last_change":"2024-04-23T14:38:38.754381+0000","last_active":"2024-04-23T14:38:38.757077+0000","last_peered":"2024-04-23T14:38:38.757077+0000","last_clean":"2024-04-23T14:38:38.757077+0000","last_became_active":"2024-04-23T14:38:38.754057+0000","last_became_peered":"2024-04-23T14:38:38.754057+0000","last_unstale":"2024-04-23T14:38:38.757077+0000","last_undegraded":"2024-04-23T14:38:38.757077+0000","last_fullsized":"2024-04-23T14:38:38.757077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:56:24.819213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.746806+0000","last_change":"2024-04-23T14:38:38.746806+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.746806+0000","last_undegraded":"2024-04-23T14:38:38.746806+0000","last_fullsized":"2024-04-23T14:38:38.746806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.771080+0000","last_change":"2024-04-23T14:38:38.767124+0000","last_active":"2024-04-23T14:38:38.771080+0000","last_peered":"2024-04-23T14:38:38.771080+0000","last_clean":"2024-04-23T14:38:38.771080+0000","last_became_active":"2024-04-23T14:38:38.766544+0000","last_became_peered":"2024-04-23T14:38:38.766544+0000","last_unstale":"2024-04-23T14:38:38.771080+0000","last_undegraded":"2024-04-23T14:38:38.771080+0000","last_fullsized":"2024-04-23T14:38:38.771080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:51.502522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.754781+0000","last_change":"2024-04-23T14:38:38.754781+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.754781+0000","last_undegraded":"2024-04-23T14:38:38.754781+0000","last_fullsized":"2024-04-23T14:38:38.754781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.766686+0000","last_change":"2024-04-23T14:38:38.764480+0000","last_active":"2024-04-23T14:38:38.766686+0000","last_peered":"2024-04-23T14:38:38.766686+0000","last_clean":"2024-04-23T14:38:38.766686+0000","last_became_active":"2024-04-23T14:38:38.764049+0000","last_became_peered":"2024-04-23T14:38:38.764049+0000","last_unstale":"2024-04-23T14:38:38.766686+0000","last_undegraded":"2024-04-23T14:38:38.766686+0000","last_fullsized":"2024-04-23T14:38:38.766686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.307678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.751696+0000","last_change":"2024-04-23T14:38:38.751696+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.751696+0000","last_undegraded":"2024-04-23T14:38:38.751696+0000","last_fullsized":"2024-04-23T14:38:38.751696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.756657+0000","last_change":"2024-04-23T14:38:38.752839+0000","last_active":"2024-04-23T14:38:38.756657+0000","last_peered":"2024-04-23T14:38:38.756657+0000","last_clean":"2024-04-23T14:38:38.756657+0000","last_became_active":"2024-04-23T14:38:38.752499+0000","last_became_peered":"2024-04-23T14:38:38.752499+0000","last_unstale":"2024-04-23T14:38:38.756657+0000","last_undegraded":"2024-04-23T14:38:38.756657+0000","last_fullsized":"2024-04-23T14:38:38.756657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:39.955458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.745092+0000","last_change":"2024-04-23T14:38:38.745092+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.745092+0000","last_undegraded":"2024-04-23T14:38:38.745092+0000","last_fullsized":"2024-04-23T14:38:38.745092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.765765+0000","last_change":"2024-04-23T14:38:38.762763+0000","last_active":"2024-04-23T14:38:38.765765+0000","last_peered":"2024-04-23T14:38:38.765765+0000","last_clean":"2024-04-23T14:38:38.765765+0000","last_became_active":"2024-04-23T14:38:38.762457+0000","last_became_peered":"2024-04-23T14:38:38.762457+0000","last_unstale":"2024-04-23T14:38:38.765765+0000","last_undegraded":"2024-04-23T14:38:38.765765+0000","last_fullsized":"2024-04-23T14:38:38.765765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:59.471051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.747010+0000","last_change":"2024-04-23T14:38:38.747010+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.747010+0000","last_undegraded":"2024-04-23T14:38:38.747010+0000","last_fullsized":"2024-04-23T14:38:38.747010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.762174+0000","last_change":"2024-04-23T14:38:38.756605+0000","last_active":"2024-04-23T14:38:38.762174+0000","last_peered":"2024-04-23T14:38:38.762174+0000","last_clean":"2024-04-23T14:38:38.762174+0000","last_became_active":"2024-04-23T14:38:38.756001+0000","last_became_peered":"2024-04-23T14:38:38.756001+0000","last_unstale":"2024-04-23T14:38:38.762174+0000","last_undegraded":"2024-04-23T14:38:38.762174+0000","last_fullsized":"2024-04-23T14:38:38.762174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:10:28.754500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.746485+0000","last_change":"2024-04-23T14:38:38.746485+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.746485+0000","last_undegraded":"2024-04-23T14:38:38.746485+0000","last_fullsized":"2024-04-23T14:38:38.746485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.767490+0000","last_change":"2024-04-23T14:38:38.766098+0000","last_active":"2024-04-23T14:38:38.767490+0000","last_peered":"2024-04-23T14:38:38.767490+0000","last_clean":"2024-04-23T14:38:38.767490+0000","last_became_active":"2024-04-23T14:38:38.765736+0000","last_became_peered":"2024-04-23T14:38:38.765736+0000","last_unstale":"2024-04-23T14:38:38.767490+0000","last_undegraded":"2024-04-23T14:38:38.767490+0000","last_fullsized":"2024-04-23T14:38:38.767490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:29.263259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.739780+0000","last_change":"2024-04-23T14:38:38.739780+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.739780+0000","last_undegraded":"2024-04-23T14:38:38.739780+0000","last_fullsized":"2024-04-23T14:38:38.739780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.764022+0000","last_change":"2024-04-23T14:38:38.758516+0000","last_active":"2024-04-23T14:38:38.764022+0000","last_peered":"2024-04-23T14:38:38.764022+0000","last_clean":"2024-04-23T14:38:38.764022+0000","last_became_active":"2024-04-23T14:38:38.758197+0000","last_became_peered":"2024-04-23T14:38:38.758197+0000","last_unstale":"2024-04-23T14:38:38.764022+0000","last_undegraded":"2024-04-23T14:38:38.764022+0000","last_fullsized":"2024-04-23T14:38:38.764022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:17.317722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.756827+0000","last_change":"2024-04-23T14:38:38.756827+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.756827+0000","last_undegraded":"2024-04-23T14:38:38.756827+0000","last_fullsized":"2024-04-23T14:38:38.756827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.759926+0000","last_change":"2024-04-23T14:38:38.757183+0000","last_active":"2024-04-23T14:38:38.759926+0000","last_peered":"2024-04-23T14:38:38.759926+0000","last_clean":"2024-04-23T14:38:38.759926+0000","last_became_active":"2024-04-23T14:38:38.756775+0000","last_became_peered":"2024-04-23T14:38:38.756775+0000","last_unstale":"2024-04-23T14:38:38.759926+0000","last_undegraded":"2024-04-23T14:38:38.759926+0000","last_fullsized":"2024-04-23T14:38:38.759926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:31.849203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.739008+0000","last_change":"2024-04-23T14:38:38.739008+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.739008+0000","last_undegraded":"2024-04-23T14:38:38.739008+0000","last_fullsized":"2024-04-23T14:38:38.739008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.762635+0000","last_change":"2024-04-23T14:38:38.758385+0000","last_active":"2024-04-23T14:38:38.762635+0000","last_peered":"2024-04-23T14:38:38.762635+0000","last_clean":"2024-04-23T14:38:38.762635+0000","last_became_active":"2024-04-23T14:38:38.756673+0000","last_became_peered":"2024-04-23T14:38:38.756673+0000","last_unstale":"2024-04-23T14:38:38.762635+0000","last_undegraded":"2024-04-23T14:38:38.762635+0000","last_fullsized":"2024-04-23T14:38:38.762635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:40.667400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.756003+0000","last_change":"2024-04-23T14:38:38.756003+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.756003+0000","last_undegraded":"2024-04-23T14:38:38.756003+0000","last_fullsized":"2024-04-23T14:38:38.756003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.763841+0000","last_change":"2024-04-23T14:38:38.760828+0000","last_active":"2024-04-23T14:38:38.763841+0000","last_peered":"2024-04-23T14:38:38.763841+0000","last_clean":"2024-04-23T14:38:38.763841+0000","last_became_active":"2024-04-23T14:38:38.760535+0000","last_became_peered":"2024-04-23T14:38:38.760535+0000","last_unstale":"2024-04-23T14:38:38.763841+0000","last_undegraded":"2024-04-23T14:38:38.763841+0000","last_fullsized":"2024-04-23T14:38:38.763841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:39.397304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.749907+0000","last_change":"2024-04-23T14:38:38.749907+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.749907+0000","last_undegraded":"2024-04-23T14:38:38.749907+0000","last_fullsized":"2024-04-23T14:38:38.749907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.765803+0000","last_change":"2024-04-23T14:38:38.763751+0000","last_active":"2024-04-23T14:38:38.765803+0000","last_peered":"2024-04-23T14:38:38.765803+0000","last_clean":"2024-04-23T14:38:38.765803+0000","last_became_active":"2024-04-23T14:38:38.763447+0000","last_became_peered":"2024-04-23T14:38:38.763447+0000","last_unstale":"2024-04-23T14:38:38.765803+0000","last_undegraded":"2024-04-23T14:38:38.765803+0000","last_fullsized":"2024-04-23T14:38:38.765803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:39.955458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.746605+0000","last_change":"2024-04-23T14:38:38.746605+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.746605+0000","last_undegraded":"2024-04-23T14:38:38.746605+0000","last_fullsized":"2024-04-23T14:38:38.746605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.765847+0000","last_change":"2024-04-23T14:38:38.763686+0000","last_active":"2024-04-23T14:38:38.765847+0000","last_peered":"2024-04-23T14:38:38.765847+0000","last_clean":"2024-04-23T14:38:38.765847+0000","last_became_active":"2024-04-23T14:38:38.762993+0000","last_became_peered":"2024-04-23T14:38:38.762993+0000","last_unstale":"2024-04-23T14:38:38.765847+0000","last_undegraded":"2024-04-23T14:38:38.765847+0000","last_fullsized":"2024-04-23T14:38:38.765847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:58.686486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.757488+0000","last_change":"2024-04-23T14:38:38.757488+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.757488+0000","last_undegraded":"2024-04-23T14:38:38.757488+0000","last_fullsized":"2024-04-23T14:38:38.757488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.756858+0000","last_change":"2024-04-23T14:38:38.750732+0000","last_active":"2024-04-23T14:38:38.756858+0000","last_peered":"2024-04-23T14:38:38.756858+0000","last_clean":"2024-04-23T14:38:38.756858+0000","last_became_active":"2024-04-23T14:38:38.749763+0000","last_became_peered":"2024-04-23T14:38:38.749763+0000","last_unstale":"2024-04-23T14:38:38.756858+0000","last_undegraded":"2024-04-23T14:38:38.756858+0000","last_fullsized":"2024-04-23T14:38:38.756858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:34.199124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.738400+0000","last_change":"2024-04-23T14:38:38.738400+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.738400+0000","last_undegraded":"2024-04-23T14:38:38.738400+0000","last_fullsized":"2024-04-23T14:38:38.738400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.765988+0000","last_change":"2024-04-23T14:38:38.765461+0000","last_active":"2024-04-23T14:38:38.765988+0000","last_peered":"2024-04-23T14:38:38.765988+0000","last_clean":"2024-04-23T14:38:38.765988+0000","last_became_active":"2024-04-23T14:38:38.764564+0000","last_became_peered":"2024-04-23T14:38:38.764564+0000","last_unstale":"2024-04-23T14:38:38.765988+0000","last_undegraded":"2024-04-23T14:38:38.765988+0000","last_fullsized":"2024-04-23T14:38:38.765988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:42:25.899768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.756427+0000","last_change":"2024-04-23T14:38:38.756427+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.756427+0000","last_undegraded":"2024-04-23T14:38:38.756427+0000","last_fullsized":"2024-04-23T14:38:38.756427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.756175+0000","last_change":"2024-04-23T14:38:38.753700+0000","last_active":"2024-04-23T14:38:38.756175+0000","last_peered":"2024-04-23T14:38:38.756175+0000","last_clean":"2024-04-23T14:38:38.756175+0000","last_became_active":"2024-04-23T14:38:38.753417+0000","last_became_peered":"2024-04-23T14:38:38.753417+0000","last_unstale":"2024-04-23T14:38:38.756175+0000","last_undegraded":"2024-04-23T14:38:38.756175+0000","last_fullsized":"2024-04-23T14:38:38.756175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:51:45.001973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.735424+0000","last_change":"2024-04-23T14:38:38.735424+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.735424+0000","last_undegraded":"2024-04-23T14:38:38.735424+0000","last_fullsized":"2024-04-23T14:38:38.735424+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.762285+0000","last_change":"2024-04-23T14:38:38.758485+0000","last_active":"2024-04-23T14:38:38.762285+0000","last_peered":"2024-04-23T14:38:38.762285+0000","last_clean":"2024-04-23T14:38:38.762285+0000","last_became_active":"2024-04-23T14:38:38.758158+0000","last_became_peered":"2024-04-23T14:38:38.758158+0000","last_unstale":"2024-04-23T14:38:38.762285+0000","last_undegraded":"2024-04-23T14:38:38.762285+0000","last_fullsized":"2024-04-23T14:38:38.762285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:27.444838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.743043+0000","last_change":"2024-04-23T14:38:38.743043+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.743043+0000","last_undegraded":"2024-04-23T14:38:38.743043+0000","last_fullsized":"2024-04-23T14:38:38.743043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.749555+0000","last_change":"2024-04-23T14:38:38.742178+0000","last_active":"2024-04-23T14:38:38.749555+0000","last_peered":"2024-04-23T14:38:38.749555+0000","last_clean":"2024-04-23T14:38:38.749555+0000","last_became_active":"2024-04-23T14:38:38.741810+0000","last_became_peered":"2024-04-23T14:38:38.741810+0000","last_unstale":"2024-04-23T14:38:38.749555+0000","last_undegraded":"2024-04-23T14:38:38.749555+0000","last_fullsized":"2024-04-23T14:38:38.749555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:28:29.999652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.735096+0000","last_change":"2024-04-23T14:38:38.735096+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.735096+0000","last_undegraded":"2024-04-23T14:38:38.735096+0000","last_fullsized":"2024-04-23T14:38:38.735096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.756782+0000","last_change":"2024-04-23T14:38:38.753322+0000","last_active":"2024-04-23T14:38:38.756782+0000","last_peered":"2024-04-23T14:38:38.756782+0000","last_clean":"2024-04-23T14:38:38.756782+0000","last_became_active":"2024-04-23T14:38:38.751669+0000","last_became_peered":"2024-04-23T14:38:38.751669+0000","last_unstale":"2024-04-23T14:38:38.756782+0000","last_undegraded":"2024-04-23T14:38:38.756782+0000","last_fullsized":"2024-04-23T14:38:38.756782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:14.586218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.744786+0000","last_change":"2024-04-23T14:38:38.744786+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.744786+0000","last_undegraded":"2024-04-23T14:38:38.744786+0000","last_fullsized":"2024-04-23T14:38:38.744786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.759574+0000","last_change":"2024-04-23T14:38:38.753720+0000","last_active":"2024-04-23T14:38:38.759574+0000","last_peered":"2024-04-23T14:38:38.759574+0000","last_clean":"2024-04-23T14:38:38.759574+0000","last_became_active":"2024-04-23T14:38:38.753419+0000","last_became_peered":"2024-04-23T14:38:38.753419+0000","last_unstale":"2024-04-23T14:38:38.759574+0000","last_undegraded":"2024-04-23T14:38:38.759574+0000","last_fullsized":"2024-04-23T14:38:38.759574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:40:58.643251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.758754+0000","last_change":"2024-04-23T14:38:38.758754+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.758754+0000","last_undegraded":"2024-04-23T14:38:38.758754+0000","last_fullsized":"2024-04-23T14:38:38.758754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.766213+0000","last_change":"2024-04-23T14:38:38.764821+0000","last_active":"2024-04-23T14:38:38.766213+0000","last_peered":"2024-04-23T14:38:38.766213+0000","last_clean":"2024-04-23T14:38:38.766213+0000","last_became_active":"2024-04-23T14:38:38.764410+0000","last_became_peered":"2024-04-23T14:38:38.764410+0000","last_unstale":"2024-04-23T14:38:38.766213+0000","last_undegraded":"2024-04-23T14:38:38.766213+0000","last_fullsized":"2024-04-23T14:38:38.766213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:32:31.092031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.755434+0000","last_change":"2024-04-23T14:38:38.755434+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.755434+0000","last_undegraded":"2024-04-23T14:38:38.755434+0000","last_fullsized":"2024-04-23T14:38:38.755434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.762650+0000","last_change":"2024-04-23T14:38:38.758493+0000","last_active":"2024-04-23T14:38:38.762650+0000","last_peered":"2024-04-23T14:38:38.762650+0000","last_clean":"2024-04-23T14:38:38.762650+0000","last_became_active":"2024-04-23T14:38:38.758152+0000","last_became_peered":"2024-04-23T14:38:38.758152+0000","last_unstale":"2024-04-23T14:38:38.762650+0000","last_undegraded":"2024-04-23T14:38:38.762650+0000","last_fullsized":"2024-04-23T14:38:38.762650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:05.418663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.736949+0000","last_change":"2024-04-23T14:38:38.736949+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.736949+0000","last_undegraded":"2024-04-23T14:38:38.736949+0000","last_fullsized":"2024-04-23T14:38:38.736949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.753423+0000","last_change":"2024-04-23T14:38:38.748633+0000","last_active":"2024-04-23T14:38:38.753423+0000","last_peered":"2024-04-23T14:38:38.753423+0000","last_clean":"2024-04-23T14:38:38.753423+0000","last_became_active":"2024-04-23T14:38:38.748127+0000","last_became_peered":"2024-04-23T14:38:38.748127+0000","last_unstale":"2024-04-23T14:38:38.753423+0000","last_undegraded":"2024-04-23T14:38:38.753423+0000","last_fullsized":"2024-04-23T14:38:38.753423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:51.502522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.755981+0000","last_change":"2024-04-23T14:38:38.755981+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.755981+0000","last_undegraded":"2024-04-23T14:38:38.755981+0000","last_fullsized":"2024-04-23T14:38:38.755981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.762172+0000","last_change":"2024-04-23T14:38:38.755614+0000","last_active":"2024-04-23T14:38:38.762172+0000","last_peered":"2024-04-23T14:38:38.762172+0000","last_clean":"2024-04-23T14:38:38.762172+0000","last_became_active":"2024-04-23T14:38:38.752060+0000","last_became_peered":"2024-04-23T14:38:38.752060+0000","last_unstale":"2024-04-23T14:38:38.762172+0000","last_undegraded":"2024-04-23T14:38:38.762172+0000","last_fullsized":"2024-04-23T14:38:38.762172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:41:06.128775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.743043+0000","last_change":"2024-04-23T14:38:38.743043+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.743043+0000","last_undegraded":"2024-04-23T14:38:38.743043+0000","last_fullsized":"2024-04-23T14:38:38.743043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.764291+0000","last_change":"2024-04-23T14:38:38.763486+0000","last_active":"2024-04-23T14:38:38.764291+0000","last_peered":"2024-04-23T14:38:38.764291+0000","last_clean":"2024-04-23T14:38:38.764291+0000","last_became_active":"2024-04-23T14:38:38.763125+0000","last_became_peered":"2024-04-23T14:38:38.763125+0000","last_unstale":"2024-04-23T14:38:38.764291+0000","last_undegraded":"2024-04-23T14:38:38.764291+0000","last_fullsized":"2024-04-23T14:38:38.764291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:43:04.315920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.738192+0000","last_change":"2024-04-23T14:38:38.738192+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.738192+0000","last_undegraded":"2024-04-23T14:38:38.738192+0000","last_fullsized":"2024-04-23T14:38:38.738192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.766381+0000","last_change":"2024-04-23T14:38:38.763771+0000","last_active":"2024-04-23T14:38:38.766381+0000","last_peered":"2024-04-23T14:38:38.766381+0000","last_clean":"2024-04-23T14:38:38.766381+0000","last_became_active":"2024-04-23T14:38:38.762540+0000","last_became_peered":"2024-04-23T14:38:38.762540+0000","last_unstale":"2024-04-23T14:38:38.766381+0000","last_undegraded":"2024-04-23T14:38:38.766381+0000","last_fullsized":"2024-04-23T14:38:38.766381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:59.197642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.750969+0000","last_change":"2024-04-23T14:38:38.750969+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.750969+0000","last_undegraded":"2024-04-23T14:38:38.750969+0000","last_fullsized":"2024-04-23T14:38:38.750969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.755194+0000","last_change":"2024-04-23T14:38:38.752186+0000","last_active":"2024-04-23T14:38:38.755194+0000","last_peered":"2024-04-23T14:38:38.755194+0000","last_clean":"2024-04-23T14:38:38.755194+0000","last_became_active":"2024-04-23T14:38:38.751597+0000","last_became_peered":"2024-04-23T14:38:38.751597+0000","last_unstale":"2024-04-23T14:38:38.755194+0000","last_undegraded":"2024-04-23T14:38:38.755194+0000","last_fullsized":"2024-04-23T14:38:38.755194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:37.889533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.736452+0000","last_change":"2024-04-23T14:38:38.736452+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.736452+0000","last_undegraded":"2024-04-23T14:38:38.736452+0000","last_fullsized":"2024-04-23T14:38:38.736452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.766603+0000","last_change":"2024-04-23T14:38:38.763764+0000","last_active":"2024-04-23T14:38:38.766603+0000","last_peered":"2024-04-23T14:38:38.766603+0000","last_clean":"2024-04-23T14:38:38.766603+0000","last_became_active":"2024-04-23T14:38:38.762666+0000","last_became_peered":"2024-04-23T14:38:38.762666+0000","last_unstale":"2024-04-23T14:38:38.766603+0000","last_undegraded":"2024-04-23T14:38:38.766603+0000","last_fullsized":"2024-04-23T14:38:38.766603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:54.219347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2024-04-23T14:38:38.746458+0000","last_change":"2024-04-23T14:38:38.746458+0000","last_active":"2024-04-23T14:38:38.711438+0000","last_peered":"2024-04-23T14:38:38.711438+0000","last_clean":"2024-04-23T14:38:38.711438+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:38:38.746458+0000","last_undegraded":"2024-04-23T14:38:38.746458+0000","last_fullsized":"2024-04-23T14:38:38.746458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.768292+0000","last_change":"2024-04-23T14:38:38.765939+0000","last_active":"2024-04-23T14:38:38.768292+0000","last_peered":"2024-04-23T14:38:38.768292+0000","last_clean":"2024-04-23T14:38:38.768292+0000","last_became_active":"2024-04-23T14:38:38.765621+0000","last_became_peered":"2024-04-23T14:38:38.765621+0000","last_unstale":"2024-04-23T14:38:38.768292+0000","last_undegraded":"2024-04-23T14:38:38.768292+0000","last_fullsized":"2024-04-23T14:38:38.768292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:42:25.899768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":281,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-23T14:38:38.754396+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:38:38.754396+0000","last_peered":"2024-04-23T14:38:38.754396+0000","last_clean":"2024-04-23T14:38:38.754396+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:38:38.754396+0000","last_undegraded":"2024-04-23T14:38:38.754396+0000","last_fullsized":"2024-04-23T14:38:38.754396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":1006700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":422607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":422607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":422607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":983040,"data_stored":416332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":422607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":422607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1589248,"data_stored":1012975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:38:44.231 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:38:44.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.485+0000 7ff783e00640 1 Processor -- start 2024-04-23T14:38:44.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.485+0000 7ff783e00640 1 -- start start 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.486+0000 7ff783e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff7840d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.486+0000 7ff783e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7840d2630 0x7ff7840b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.486+0000 7ff783e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7840cf030 con 0x7ff7840b6530 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.486+0000 7ff783e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff7840b7000 con 0x7ff7840d2630 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.486+0000 7ff783e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff7840cf620 con 0x7ff78415e7c0 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7840d2630 0x7ff7840b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff783400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff7840b6530 0x7ff7840d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39170/0 (socket says 172.21.15.59:39170) 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7840d2630 0x7ff7840b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43236/0 (socket says 172.21.15.59:43236) 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff783400640 1 -- 172.21.15.59:0/347072184 learned_addr learned my addr 172.21.15.59:0/347072184 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:44.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782a00640 1 --2- 172.21.15.59:0/347072184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff7840d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782000640 1 -- 172.21.15.59:0/347072184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 msgr2=0x7ff7840d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782000640 1 --2- 172.21.15.59:0/347072184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff7840d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782000640 1 -- 172.21.15.59:0/347072184 >> v1:172.21.15.59:6789/0 conn(0x7ff7840b6530 legacy=0x7ff7840d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff781600640 1 -- 172.21.15.59:0/347072184 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856621612 0 0) 0x7ff77c001110 con 0x7ff7840b6530 2024-04-23T14:38:44.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782000640 1 -- 172.21.15.59:0/347072184 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7840b6a40 con 0x7ff7840d2630 2024-04-23T14:38:44.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.487+0000 7ff782a00640 1 --2- 172.21.15.59:0/347072184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff7840d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:38:44.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.488+0000 7ff782000640 1 --2- 172.21.15.59:0/347072184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840d2630 0x7ff7840b5ff0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009d90 tx=0x7ff76c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfeb39317878441 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.489+0000 7ff781600640 1 -- 172.21.15.59:0/347072184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c007740 con 0x7ff7840d2630 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.489+0000 7ff781600640 1 -- 172.21.15.59:0/347072184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c0078e0 con 0x7ff7840d2630 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.489+0000 7ff781600640 1 -- 172.21.15.59:0/347072184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c007cb0 con 0x7ff7840d2630 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.489+0000 7ff783e00640 1 -- 172.21.15.59:0/347072184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840d2630 msgr2=0x7ff7840b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.489+0000 7ff783e00640 1 --2- 172.21.15.59:0/347072184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840d2630 0x7ff7840b5ff0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009d90 tx=0x7ff76c02a660 comp rx=0 tx=0).stop 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.490+0000 7ff783e00640 1 -- 172.21.15.59:0/347072184 shutdown_connections 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.490+0000 7ff783e00640 1 --2- 172.21.15.59:0/347072184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840d2630 0x7ff7840b5ff0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.490+0000 7ff783e00640 1 --2- 172.21.15.59:0/347072184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff7840d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.490+0000 7ff783e00640 1 -- 172.21.15.59:0/347072184 >> 172.21.15.59:0/347072184 conn(0x7ff78409d920 msgr2=0x7ff78408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.490+0000 7ff783e00640 1 -- 172.21.15.59:0/347072184 shutdown_connections 2024-04-23T14:38:44.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.490+0000 7ff783e00640 1 -- 172.21.15.59:0/347072184 wait complete. 2024-04-23T14:38:44.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.491+0000 7ff783e00640 1 Processor -- start 2024-04-23T14:38:44.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.492+0000 7ff783e00640 1 -- start start 2024-04-23T14:38:44.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.492+0000 7ff783e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840b6530 0x7ff7840cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:44.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.492+0000 7ff783e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7840d2630 0x7ff784160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:44.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.492+0000 7ff783e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff784161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:44.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.492+0000 7ff783e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7840d1a80 con 0x7ff7840b6530 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.492+0000 7ff783e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff7840cf2b0 con 0x7ff7840d2630 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.492+0000 7ff783e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff7840d1c00 con 0x7ff78415e7c0 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840b6530 0x7ff7840cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840b6530 0x7ff7840cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43238/0 (socket says 172.21.15.59:43238) 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 -- 172.21.15.59:0/1847266797 learned_addr learned my addr 172.21.15.59:0/1847266797 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782000640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7840d2630 0x7ff784160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff783400640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff784161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 -- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 msgr2=0x7ff784161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff784161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 -- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7840d2630 msgr2=0x7ff784160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7840d2630 0x7ff784160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 -- 172.21.15.59:0/1847266797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7840b1340 con 0x7ff7840b6530 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782000640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7840d2630 0x7ff784160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:44.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.493+0000 7ff782a00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840b6530 0x7ff7840cbdc0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ff77400d7d0 tx=0x7ff77400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.494+0000 7ff77aa00640 1 -- 172.21.15.59:0/1847266797 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774004030 con 0x7ff7840b6530 2024-04-23T14:38:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.494+0000 7ff77aa00640 1 -- 172.21.15.59:0/1847266797 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7740041d0 con 0x7ff7840b6530 2024-04-23T14:38:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.494+0000 7ff77aa00640 1 -- 172.21.15.59:0/1847266797 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77400b6c0 con 0x7ff7840b6530 2024-04-23T14:38:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.494+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7840b1620 con 0x7ff7840b6530 2024-04-23T14:38:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.494+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7840b1be0 con 0x7ff7840b6530 2024-04-23T14:38:44.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.495+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78404fd90 con 0x7ff7840b6530 2024-04-23T14:38:44.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.496+0000 7ff77aa00640 1 -- 172.21.15.59:0/1847266797 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff77400b860 con 0x7ff7840b6530 2024-04-23T14:38:44.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.497+0000 7ff77aa00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff76006c8f0 0x7ff76006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:44.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.497+0000 7ff77aa00640 1 -- 172.21.15.59:0/1847266797 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7ff77408be70 con 0x7ff7840b6530 2024-04-23T14:38:44.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.498+0000 7ff782000640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff76006c8f0 0x7ff76006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:44.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.498+0000 7ff782000640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff76006c8f0 0x7ff76006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034000 tx=0x7ff76c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:44.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.502+0000 7ff77aa00640 1 -- 172.21.15.59:0/1847266797 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff77405a2a0 con 0x7ff7840b6530 2024-04-23T14:38:44.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.704+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff784161bb0 con 0x7ff76006c8f0 2024-04-23T14:38:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.716+0000 7ff77aa00640 1 -- 172.21.15.59:0/1847266797 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252085 (secure 0 0 0) 0x7ff784161bb0 con 0x7ff76006c8f0 2024-04-23T14:38:44.720 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:38:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff76006c8f0 msgr2=0x7ff76006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff76006c8f0 0x7ff76006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034000 tx=0x7ff76c02f000 comp rx=0 tx=0).stop 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840b6530 msgr2=0x7ff7840cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840b6530 0x7ff7840cbdc0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ff77400d7d0 tx=0x7ff77400dc90 comp rx=0 tx=0).stop 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 shutdown_connections 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff76006c8f0 0x7ff76006edb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff78415e7c0 0x7ff784161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7840d2630 0x7ff784160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 --2- 172.21.15.59:0/1847266797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7840b6530 0x7ff7840cbdc0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.721+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 >> 172.21.15.59:0/1847266797 conn(0x7ff78409d920 msgr2=0x7ff7840b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.722+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 shutdown_connections 2024-04-23T14:38:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.722+0000 7ff783e00640 1 -- 172.21.15.59:0/1847266797 wait complete. 2024-04-23T14:38:44.744 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-23T14:38:44.497097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258644,"kb_used_data":9192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716076,"statfs":{"total":773094113280,"available":772829261824,"internally_reserved":0,"allocated":9412608,"data_stored":4671012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001827"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331981+0000","last_change":"2024-04-23T14:38:39.766068+0000","last_active":"2024-04-23T14:38:41.331981+0000","last_peered":"2024-04-23T14:38:41.331981+0000","last_clean":"2024-04-23T14:38:41.331981+0000","last_became_active":"2024-04-23T14:38:39.765567+0000","last_became_peered":"2024-04-23T14:38:39.765567+0000","last_unstale":"2024-04-23T14:38:41.331981+0000","last_undegraded":"2024-04-23T14:38:41.331981+0000","last_fullsized":"2024-04-23T14:38:41.331981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:44.434109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332116+0000","last_change":"2024-04-23T14:38:39.752868+0000","last_active":"2024-04-23T14:38:41.332116+0000","last_peered":"2024-04-23T14:38:41.332116+0000","last_clean":"2024-04-23T14:38:41.332116+0000","last_became_active":"2024-04-23T14:38:39.752556+0000","last_became_peered":"2024-04-23T14:38:39.752556+0000","last_unstale":"2024-04-23T14:38:41.332116+0000","last_undegraded":"2024-04-23T14:38:41.332116+0000","last_fullsized":"2024-04-23T14:38:41.332116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:49.648752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764944+0000","last_change":"2024-04-23T14:38:39.773266+0000","last_active":"2024-04-23T14:38:40.764944+0000","last_peered":"2024-04-23T14:38:40.764944+0000","last_clean":"2024-04-23T14:38:40.764944+0000","last_became_active":"2024-04-23T14:38:39.772939+0000","last_became_peered":"2024-04-23T14:38:39.772939+0000","last_unstale":"2024-04-23T14:38:40.764944+0000","last_undegraded":"2024-04-23T14:38:40.764944+0000","last_fullsized":"2024-04-23T14:38:40.764944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:52.023552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.328912+0000","last_change":"2024-04-23T14:38:39.775279+0000","last_active":"2024-04-23T14:38:41.328912+0000","last_peered":"2024-04-23T14:38:41.328912+0000","last_clean":"2024-04-23T14:38:41.328912+0000","last_became_active":"2024-04-23T14:38:39.774923+0000","last_became_peered":"2024-04-23T14:38:39.774923+0000","last_unstale":"2024-04-23T14:38:41.328912+0000","last_undegraded":"2024-04-23T14:38:41.328912+0000","last_fullsized":"2024-04-23T14:38:41.328912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:00.487737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764608+0000","last_change":"2024-04-23T14:38:39.772768+0000","last_active":"2024-04-23T14:38:40.764608+0000","last_peered":"2024-04-23T14:38:40.764608+0000","last_clean":"2024-04-23T14:38:40.764608+0000","last_became_active":"2024-04-23T14:38:39.772384+0000","last_became_peered":"2024-04-23T14:38:39.772384+0000","last_unstale":"2024-04-23T14:38:40.764608+0000","last_undegraded":"2024-04-23T14:38:40.764608+0000","last_fullsized":"2024-04-23T14:38:40.764608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:32.865889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329644+0000","last_change":"2024-04-23T14:38:39.763601+0000","last_active":"2024-04-23T14:38:41.329644+0000","last_peered":"2024-04-23T14:38:41.329644+0000","last_clean":"2024-04-23T14:38:41.329644+0000","last_became_active":"2024-04-23T14:38:39.763299+0000","last_became_peered":"2024-04-23T14:38:39.763299+0000","last_unstale":"2024-04-23T14:38:41.329644+0000","last_undegraded":"2024-04-23T14:38:41.329644+0000","last_fullsized":"2024-04-23T14:38:41.329644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:21.801450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.327880+0000","last_change":"2024-04-23T14:38:39.777329+0000","last_active":"2024-04-23T14:38:41.327880+0000","last_peered":"2024-04-23T14:38:41.327880+0000","last_clean":"2024-04-23T14:38:41.327880+0000","last_became_active":"2024-04-23T14:38:39.775344+0000","last_became_peered":"2024-04-23T14:38:39.775344+0000","last_unstale":"2024-04-23T14:38:41.327880+0000","last_undegraded":"2024-04-23T14:38:41.327880+0000","last_fullsized":"2024-04-23T14:38:41.327880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:19:48.300387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329595+0000","last_change":"2024-04-23T14:38:39.763674+0000","last_active":"2024-04-23T14:38:41.329595+0000","last_peered":"2024-04-23T14:38:41.329595+0000","last_clean":"2024-04-23T14:38:41.329595+0000","last_became_active":"2024-04-23T14:38:39.763328+0000","last_became_peered":"2024-04-23T14:38:39.763328+0000","last_unstale":"2024-04-23T14:38:41.329595+0000","last_undegraded":"2024-04-23T14:38:41.329595+0000","last_fullsized":"2024-04-23T14:38:41.329595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:36.922911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330560+0000","last_change":"2024-04-23T14:38:39.766084+0000","last_active":"2024-04-23T14:38:41.330560+0000","last_peered":"2024-04-23T14:38:41.330560+0000","last_clean":"2024-04-23T14:38:41.330560+0000","last_became_active":"2024-04-23T14:38:39.765750+0000","last_became_peered":"2024-04-23T14:38:39.765750+0000","last_unstale":"2024-04-23T14:38:41.330560+0000","last_undegraded":"2024-04-23T14:38:41.330560+0000","last_fullsized":"2024-04-23T14:38:41.330560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:44.434109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331549+0000","last_change":"2024-04-23T14:38:39.757837+0000","last_active":"2024-04-23T14:38:41.331549+0000","last_peered":"2024-04-23T14:38:41.331549+0000","last_clean":"2024-04-23T14:38:41.331549+0000","last_became_active":"2024-04-23T14:38:39.756454+0000","last_became_peered":"2024-04-23T14:38:39.756454+0000","last_unstale":"2024-04-23T14:38:41.331549+0000","last_undegraded":"2024-04-23T14:38:41.331549+0000","last_fullsized":"2024-04-23T14:38:41.331549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:53.374381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331157+0000","last_change":"2024-04-23T14:38:39.768412+0000","last_active":"2024-04-23T14:38:41.331157+0000","last_peered":"2024-04-23T14:38:41.331157+0000","last_clean":"2024-04-23T14:38:41.331157+0000","last_became_active":"2024-04-23T14:38:39.767839+0000","last_became_peered":"2024-04-23T14:38:39.767839+0000","last_unstale":"2024-04-23T14:38:41.331157+0000","last_undegraded":"2024-04-23T14:38:41.331157+0000","last_fullsized":"2024-04-23T14:38:41.331157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:32:08.048643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331015+0000","last_change":"2024-04-23T14:38:39.771500+0000","last_active":"2024-04-23T14:38:41.331015+0000","last_peered":"2024-04-23T14:38:41.331015+0000","last_clean":"2024-04-23T14:38:41.331015+0000","last_became_active":"2024-04-23T14:38:39.770001+0000","last_became_peered":"2024-04-23T14:38:39.770001+0000","last_unstale":"2024-04-23T14:38:41.331015+0000","last_undegraded":"2024-04-23T14:38:41.331015+0000","last_fullsized":"2024-04-23T14:38:41.331015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:59:53.179599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764842+0000","last_change":"2024-04-23T14:38:39.759732+0000","last_active":"2024-04-23T14:38:40.764842+0000","last_peered":"2024-04-23T14:38:40.764842+0000","last_clean":"2024-04-23T14:38:40.764842+0000","last_became_active":"2024-04-23T14:38:39.759070+0000","last_became_peered":"2024-04-23T14:38:39.759070+0000","last_unstale":"2024-04-23T14:38:40.764842+0000","last_undegraded":"2024-04-23T14:38:40.764842+0000","last_fullsized":"2024-04-23T14:38:40.764842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:56.925422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.328359+0000","last_change":"2024-04-23T14:38:39.775451+0000","last_active":"2024-04-23T14:38:41.328359+0000","last_peered":"2024-04-23T14:38:41.328359+0000","last_clean":"2024-04-23T14:38:41.328359+0000","last_became_active":"2024-04-23T14:38:39.766619+0000","last_became_peered":"2024-04-23T14:38:39.766619+0000","last_unstale":"2024-04-23T14:38:41.328359+0000","last_undegraded":"2024-04-23T14:38:41.328359+0000","last_fullsized":"2024-04-23T14:38:41.328359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:30:41.684086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.761180+0000","last_change":"2024-04-23T14:38:39.761872+0000","last_active":"2024-04-23T14:38:40.761180+0000","last_peered":"2024-04-23T14:38:40.761180+0000","last_clean":"2024-04-23T14:38:40.761180+0000","last_became_active":"2024-04-23T14:38:39.761597+0000","last_became_peered":"2024-04-23T14:38:39.761597+0000","last_unstale":"2024-04-23T14:38:40.761180+0000","last_undegraded":"2024-04-23T14:38:40.761180+0000","last_fullsized":"2024-04-23T14:38:40.761180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:42:09.026220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764042+0000","last_change":"2024-04-23T14:38:39.767415+0000","last_active":"2024-04-23T14:38:40.764042+0000","last_peered":"2024-04-23T14:38:40.764042+0000","last_clean":"2024-04-23T14:38:40.764042+0000","last_became_active":"2024-04-23T14:38:39.766865+0000","last_became_peered":"2024-04-23T14:38:39.766865+0000","last_unstale":"2024-04-23T14:38:40.764042+0000","last_undegraded":"2024-04-23T14:38:40.764042+0000","last_fullsized":"2024-04-23T14:38:40.764042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:17:19.195865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.766083+0000","last_change":"2024-04-23T14:38:39.781245+0000","last_active":"2024-04-23T14:38:40.766083+0000","last_peered":"2024-04-23T14:38:40.766083+0000","last_clean":"2024-04-23T14:38:40.766083+0000","last_became_active":"2024-04-23T14:38:39.780937+0000","last_became_peered":"2024-04-23T14:38:39.780937+0000","last_unstale":"2024-04-23T14:38:40.766083+0000","last_undegraded":"2024-04-23T14:38:40.766083+0000","last_fullsized":"2024-04-23T14:38:40.766083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:20.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.328651+0000","last_change":"2024-04-23T14:38:39.767888+0000","last_active":"2024-04-23T14:38:41.328651+0000","last_peered":"2024-04-23T14:38:41.328651+0000","last_clean":"2024-04-23T14:38:41.328651+0000","last_became_active":"2024-04-23T14:38:39.767605+0000","last_became_peered":"2024-04-23T14:38:39.767605+0000","last_unstale":"2024-04-23T14:38:41.328651+0000","last_undegraded":"2024-04-23T14:38:41.328651+0000","last_fullsized":"2024-04-23T14:38:41.328651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:09:06.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.333194+0000","last_change":"2024-04-23T14:38:39.762777+0000","last_active":"2024-04-23T14:38:41.333194+0000","last_peered":"2024-04-23T14:38:41.333194+0000","last_clean":"2024-04-23T14:38:41.333194+0000","last_became_active":"2024-04-23T14:38:39.762480+0000","last_became_peered":"2024-04-23T14:38:39.762480+0000","last_unstale":"2024-04-23T14:38:41.333194+0000","last_undegraded":"2024-04-23T14:38:41.333194+0000","last_fullsized":"2024-04-23T14:38:41.333194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:52.023552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331200+0000","last_change":"2024-04-23T14:38:39.770884+0000","last_active":"2024-04-23T14:38:41.331200+0000","last_peered":"2024-04-23T14:38:41.331200+0000","last_clean":"2024-04-23T14:38:41.331200+0000","last_became_active":"2024-04-23T14:38:39.768881+0000","last_became_peered":"2024-04-23T14:38:39.768881+0000","last_unstale":"2024-04-23T14:38:41.331200+0000","last_undegraded":"2024-04-23T14:38:41.331200+0000","last_fullsized":"2024-04-23T14:38:41.331200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:31.681468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329031+0000","last_change":"2024-04-23T14:38:39.772799+0000","last_active":"2024-04-23T14:38:41.329031+0000","last_peered":"2024-04-23T14:38:41.329031+0000","last_clean":"2024-04-23T14:38:41.329031+0000","last_became_active":"2024-04-23T14:38:39.772466+0000","last_became_peered":"2024-04-23T14:38:39.772466+0000","last_unstale":"2024-04-23T14:38:41.329031+0000","last_undegraded":"2024-04-23T14:38:41.329031+0000","last_fullsized":"2024-04-23T14:38:41.329031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:32:32.108717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330150+0000","last_change":"2024-04-23T14:38:39.768573+0000","last_active":"2024-04-23T14:38:41.330150+0000","last_peered":"2024-04-23T14:38:41.330150+0000","last_clean":"2024-04-23T14:38:41.330150+0000","last_became_active":"2024-04-23T14:38:39.768258+0000","last_became_peered":"2024-04-23T14:38:39.768258+0000","last_unstale":"2024-04-23T14:38:41.330150+0000","last_undegraded":"2024-04-23T14:38:41.330150+0000","last_fullsized":"2024-04-23T14:38:41.330150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:50:58.459694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330621+0000","last_change":"2024-04-23T14:38:39.761869+0000","last_active":"2024-04-23T14:38:41.330621+0000","last_peered":"2024-04-23T14:38:41.330621+0000","last_clean":"2024-04-23T14:38:41.330621+0000","last_became_active":"2024-04-23T14:38:39.760259+0000","last_became_peered":"2024-04-23T14:38:39.760259+0000","last_unstale":"2024-04-23T14:38:41.330621+0000","last_undegraded":"2024-04-23T14:38:41.330621+0000","last_fullsized":"2024-04-23T14:38:41.330621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:40.418558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.327913+0000","last_change":"2024-04-23T14:38:39.777404+0000","last_active":"2024-04-23T14:38:41.327913+0000","last_peered":"2024-04-23T14:38:41.327913+0000","last_clean":"2024-04-23T14:38:41.327913+0000","last_became_active":"2024-04-23T14:38:39.774272+0000","last_became_peered":"2024-04-23T14:38:39.774272+0000","last_unstale":"2024-04-23T14:38:41.327913+0000","last_undegraded":"2024-04-23T14:38:41.327913+0000","last_fullsized":"2024-04-23T14:38:41.327913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:17:24.148073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332125+0000","last_change":"2024-04-23T14:38:39.756350+0000","last_active":"2024-04-23T14:38:41.332125+0000","last_peered":"2024-04-23T14:38:41.332125+0000","last_clean":"2024-04-23T14:38:41.332125+0000","last_became_active":"2024-04-23T14:38:39.753445+0000","last_became_peered":"2024-04-23T14:38:39.753445+0000","last_unstale":"2024-04-23T14:38:41.332125+0000","last_undegraded":"2024-04-23T14:38:41.332125+0000","last_fullsized":"2024-04-23T14:38:41.332125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:05.141978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765132+0000","last_change":"2024-04-23T14:38:39.770751+0000","last_active":"2024-04-23T14:38:40.765132+0000","last_peered":"2024-04-23T14:38:40.765132+0000","last_clean":"2024-04-23T14:38:40.765132+0000","last_became_active":"2024-04-23T14:38:39.770451+0000","last_became_peered":"2024-04-23T14:38:39.770451+0000","last_unstale":"2024-04-23T14:38:40.765132+0000","last_undegraded":"2024-04-23T14:38:40.765132+0000","last_fullsized":"2024-04-23T14:38:40.765132+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:00:09.478524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765042+0000","last_change":"2024-04-23T14:38:39.771587+0000","last_active":"2024-04-23T14:38:40.765042+0000","last_peered":"2024-04-23T14:38:40.765042+0000","last_clean":"2024-04-23T14:38:40.765042+0000","last_became_active":"2024-04-23T14:38:39.771257+0000","last_became_peered":"2024-04-23T14:38:39.771257+0000","last_unstale":"2024-04-23T14:38:40.765042+0000","last_undegraded":"2024-04-23T14:38:40.765042+0000","last_fullsized":"2024-04-23T14:38:40.765042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:56:25.835899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331440+0000","last_change":"2024-04-23T14:38:39.772004+0000","last_active":"2024-04-23T14:38:41.331440+0000","last_peered":"2024-04-23T14:38:41.331440+0000","last_clean":"2024-04-23T14:38:41.331440+0000","last_became_active":"2024-04-23T14:38:39.770365+0000","last_became_peered":"2024-04-23T14:38:39.770365+0000","last_unstale":"2024-04-23T14:38:41.331440+0000","last_undegraded":"2024-04-23T14:38:41.331440+0000","last_fullsized":"2024-04-23T14:38:41.331440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:36.922911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330099+0000","last_change":"2024-04-23T14:38:39.751612+0000","last_active":"2024-04-23T14:38:41.330099+0000","last_peered":"2024-04-23T14:38:41.330099+0000","last_clean":"2024-04-23T14:38:41.330099+0000","last_became_active":"2024-04-23T14:38:39.751332+0000","last_became_peered":"2024-04-23T14:38:39.751332+0000","last_unstale":"2024-04-23T14:38:41.330099+0000","last_undegraded":"2024-04-23T14:38:41.330099+0000","last_fullsized":"2024-04-23T14:38:41.330099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:29.310568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330562+0000","last_change":"2024-04-23T14:38:39.758649+0000","last_active":"2024-04-23T14:38:41.330562+0000","last_peered":"2024-04-23T14:38:41.330562+0000","last_clean":"2024-04-23T14:38:41.330562+0000","last_became_active":"2024-04-23T14:38:39.758375+0000","last_became_peered":"2024-04-23T14:38:39.758375+0000","last_unstale":"2024-04-23T14:38:41.330562+0000","last_undegraded":"2024-04-23T14:38:41.330562+0000","last_fullsized":"2024-04-23T14:38:41.330562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:04.212117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764357+0000","last_change":"2024-04-23T14:38:39.769014+0000","last_active":"2024-04-23T14:38:40.764357+0000","last_peered":"2024-04-23T14:38:40.764357+0000","last_clean":"2024-04-23T14:38:40.764357+0000","last_became_active":"2024-04-23T14:38:39.768704+0000","last_became_peered":"2024-04-23T14:38:39.768704+0000","last_unstale":"2024-04-23T14:38:40.764357+0000","last_undegraded":"2024-04-23T14:38:40.764357+0000","last_fullsized":"2024-04-23T14:38:40.764357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:52.621380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330149+0000","last_change":"2024-04-23T14:38:39.763533+0000","last_active":"2024-04-23T14:38:41.330149+0000","last_peered":"2024-04-23T14:38:41.330149+0000","last_clean":"2024-04-23T14:38:41.330149+0000","last_became_active":"2024-04-23T14:38:39.762079+0000","last_became_peered":"2024-04-23T14:38:39.762079+0000","last_unstale":"2024-04-23T14:38:41.330149+0000","last_undegraded":"2024-04-23T14:38:41.330149+0000","last_fullsized":"2024-04-23T14:38:41.330149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:41:48.419326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765882+0000","last_change":"2024-04-23T14:38:39.769143+0000","last_active":"2024-04-23T14:38:40.765882+0000","last_peered":"2024-04-23T14:38:40.765882+0000","last_clean":"2024-04-23T14:38:40.765882+0000","last_became_active":"2024-04-23T14:38:39.768852+0000","last_became_peered":"2024-04-23T14:38:39.768852+0000","last_unstale":"2024-04-23T14:38:40.765882+0000","last_undegraded":"2024-04-23T14:38:40.765882+0000","last_fullsized":"2024-04-23T14:38:40.765882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:42:26.916455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332357+0000","last_change":"2024-04-23T14:38:38.754157+0000","last_active":"2024-04-23T14:38:41.332357+0000","last_peered":"2024-04-23T14:38:41.332357+0000","last_clean":"2024-04-23T14:38:41.332357+0000","last_became_active":"2024-04-23T14:38:38.753093+0000","last_became_peered":"2024-04-23T14:38:38.753093+0000","last_unstale":"2024-04-23T14:38:41.332357+0000","last_undegraded":"2024-04-23T14:38:41.332357+0000","last_fullsized":"2024-04-23T14:38:41.332357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:32:07.031957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332093+0000","last_change":"2024-04-23T14:38:39.753710+0000","last_active":"2024-04-23T14:38:41.332093+0000","last_peered":"2024-04-23T14:38:41.332093+0000","last_clean":"2024-04-23T14:38:41.332093+0000","last_became_active":"2024-04-23T14:38:39.753294+0000","last_became_peered":"2024-04-23T14:38:39.753294+0000","last_unstale":"2024-04-23T14:38:41.332093+0000","last_undegraded":"2024-04-23T14:38:41.332093+0000","last_fullsized":"2024-04-23T14:38:41.332093+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:01.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.766300+0000","last_change":"2024-04-23T14:38:38.764941+0000","last_active":"2024-04-23T14:38:40.766300+0000","last_peered":"2024-04-23T14:38:40.766300+0000","last_clean":"2024-04-23T14:38:40.766300+0000","last_became_active":"2024-04-23T14:38:38.764556+0000","last_became_peered":"2024-04-23T14:38:38.764556+0000","last_unstale":"2024-04-23T14:38:40.766300+0000","last_undegraded":"2024-04-23T14:38:40.766300+0000","last_fullsized":"2024-04-23T14:38:40.766300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:10:28.754500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331580+0000","last_change":"2024-04-23T14:38:39.773147+0000","last_active":"2024-04-23T14:38:41.331580+0000","last_peered":"2024-04-23T14:38:41.331580+0000","last_clean":"2024-04-23T14:38:41.331580+0000","last_became_active":"2024-04-23T14:38:39.769512+0000","last_became_peered":"2024-04-23T14:38:39.769512+0000","last_unstale":"2024-04-23T14:38:41.331580+0000","last_undegraded":"2024-04-23T14:38:41.331580+0000","last_fullsized":"2024-04-23T14:38:41.331580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:21.801450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.333114+0000","last_change":"2024-04-23T14:38:38.748098+0000","last_active":"2024-04-23T14:38:41.333114+0000","last_peered":"2024-04-23T14:38:41.333114+0000","last_clean":"2024-04-23T14:38:41.333114+0000","last_became_active":"2024-04-23T14:38:38.746024+0000","last_became_peered":"2024-04-23T14:38:38.746024+0000","last_unstale":"2024-04-23T14:38:41.333114+0000","last_undegraded":"2024-04-23T14:38:41.333114+0000","last_fullsized":"2024-04-23T14:38:41.333114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:31:01.210430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765145+0000","last_change":"2024-04-23T14:38:39.776474+0000","last_active":"2024-04-23T14:38:40.765145+0000","last_peered":"2024-04-23T14:38:40.765145+0000","last_clean":"2024-04-23T14:38:40.765145+0000","last_became_active":"2024-04-23T14:38:39.775710+0000","last_became_peered":"2024-04-23T14:38:39.775710+0000","last_unstale":"2024-04-23T14:38:40.765145+0000","last_undegraded":"2024-04-23T14:38:40.765145+0000","last_fullsized":"2024-04-23T14:38:40.765145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:47:30.322752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332920+0000","last_change":"2024-04-23T14:38:38.754381+0000","last_active":"2024-04-23T14:38:41.332920+0000","last_peered":"2024-04-23T14:38:41.332920+0000","last_clean":"2024-04-23T14:38:41.332920+0000","last_became_active":"2024-04-23T14:38:38.754057+0000","last_became_peered":"2024-04-23T14:38:38.754057+0000","last_unstale":"2024-04-23T14:38:41.332920+0000","last_undegraded":"2024-04-23T14:38:41.332920+0000","last_fullsized":"2024-04-23T14:38:41.332920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:40:39.401871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330957+0000","last_change":"2024-04-23T14:38:39.759033+0000","last_active":"2024-04-23T14:38:41.330957+0000","last_peered":"2024-04-23T14:38:41.330957+0000","last_clean":"2024-04-23T14:38:41.330957+0000","last_became_active":"2024-04-23T14:38:39.758682+0000","last_became_peered":"2024-04-23T14:38:39.758682+0000","last_unstale":"2024-04-23T14:38:41.330957+0000","last_undegraded":"2024-04-23T14:38:41.330957+0000","last_fullsized":"2024-04-23T14:38:41.330957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:16:43.216365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764855+0000","last_change":"2024-04-23T14:38:38.767124+0000","last_active":"2024-04-23T14:38:40.764855+0000","last_peered":"2024-04-23T14:38:40.764855+0000","last_clean":"2024-04-23T14:38:40.764855+0000","last_became_active":"2024-04-23T14:38:38.766544+0000","last_became_peered":"2024-04-23T14:38:38.766544+0000","last_unstale":"2024-04-23T14:38:40.764855+0000","last_undegraded":"2024-04-23T14:38:40.764855+0000","last_fullsized":"2024-04-23T14:38:40.764855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:41:06.128775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764621+0000","last_change":"2024-04-23T14:38:39.774114+0000","last_active":"2024-04-23T14:38:40.764621+0000","last_peered":"2024-04-23T14:38:40.764621+0000","last_clean":"2024-04-23T14:38:40.764621+0000","last_became_active":"2024-04-23T14:38:39.773759+0000","last_became_peered":"2024-04-23T14:38:39.773759+0000","last_unstale":"2024-04-23T14:38:40.764621+0000","last_undegraded":"2024-04-23T14:38:40.764621+0000","last_fullsized":"2024-04-23T14:38:40.764621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:56:40.972144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765127+0000","last_change":"2024-04-23T14:38:38.764480+0000","last_active":"2024-04-23T14:38:40.765127+0000","last_peered":"2024-04-23T14:38:40.765127+0000","last_clean":"2024-04-23T14:38:40.765127+0000","last_became_active":"2024-04-23T14:38:38.764049+0000","last_became_peered":"2024-04-23T14:38:38.764049+0000","last_unstale":"2024-04-23T14:38:40.765127+0000","last_undegraded":"2024-04-23T14:38:40.765127+0000","last_fullsized":"2024-04-23T14:38:40.765127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:57:37.889533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764325+0000","last_change":"2024-04-23T14:38:39.774056+0000","last_active":"2024-04-23T14:38:40.764325+0000","last_peered":"2024-04-23T14:38:40.764325+0000","last_clean":"2024-04-23T14:38:40.764325+0000","last_became_active":"2024-04-23T14:38:39.772419+0000","last_became_peered":"2024-04-23T14:38:39.772419+0000","last_unstale":"2024-04-23T14:38:40.764325+0000","last_undegraded":"2024-04-23T14:38:40.764325+0000","last_fullsized":"2024-04-23T14:38:40.764325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:44.434109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330888+0000","last_change":"2024-04-23T14:38:38.752839+0000","last_active":"2024-04-23T14:38:41.330888+0000","last_peered":"2024-04-23T14:38:41.330888+0000","last_clean":"2024-04-23T14:38:41.330888+0000","last_became_active":"2024-04-23T14:38:38.752499+0000","last_became_peered":"2024-04-23T14:38:38.752499+0000","last_unstale":"2024-04-23T14:38:41.330888+0000","last_undegraded":"2024-04-23T14:38:41.330888+0000","last_fullsized":"2024-04-23T14:38:41.330888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:19.788945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.761458+0000","last_change":"2024-04-23T14:38:39.760024+0000","last_active":"2024-04-23T14:38:40.761458+0000","last_peered":"2024-04-23T14:38:40.761458+0000","last_clean":"2024-04-23T14:38:40.761458+0000","last_became_active":"2024-04-23T14:38:39.759747+0000","last_became_peered":"2024-04-23T14:38:39.759747+0000","last_unstale":"2024-04-23T14:38:40.761458+0000","last_undegraded":"2024-04-23T14:38:40.761458+0000","last_fullsized":"2024-04-23T14:38:40.761458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:20:10.670403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329370+0000","last_change":"2024-04-23T14:38:38.762763+0000","last_active":"2024-04-23T14:38:41.329370+0000","last_peered":"2024-04-23T14:38:41.329370+0000","last_clean":"2024-04-23T14:38:41.329370+0000","last_became_active":"2024-04-23T14:38:38.762457+0000","last_became_peered":"2024-04-23T14:38:38.762457+0000","last_unstale":"2024-04-23T14:38:41.329370+0000","last_undegraded":"2024-04-23T14:38:41.329370+0000","last_fullsized":"2024-04-23T14:38:41.329370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:51.379265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.762710+0000","last_change":"2024-04-23T14:38:39.764482+0000","last_active":"2024-04-23T14:38:40.762710+0000","last_peered":"2024-04-23T14:38:40.762710+0000","last_clean":"2024-04-23T14:38:40.762710+0000","last_became_active":"2024-04-23T14:38:39.763633+0000","last_became_peered":"2024-04-23T14:38:39.763633+0000","last_unstale":"2024-04-23T14:38:40.762710+0000","last_undegraded":"2024-04-23T14:38:40.762710+0000","last_fullsized":"2024-04-23T14:38:40.762710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:09.897899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332515+0000","last_change":"2024-04-23T14:38:38.756605+0000","last_active":"2024-04-23T14:38:41.332515+0000","last_peered":"2024-04-23T14:38:41.332515+0000","last_clean":"2024-04-23T14:38:41.332515+0000","last_became_active":"2024-04-23T14:38:38.756001+0000","last_became_peered":"2024-04-23T14:38:38.756001+0000","last_unstale":"2024-04-23T14:38:41.332515+0000","last_undegraded":"2024-04-23T14:38:41.332515+0000","last_fullsized":"2024-04-23T14:38:41.332515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:48.632065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330807+0000","last_change":"2024-04-23T14:38:39.776006+0000","last_active":"2024-04-23T14:38:41.330807+0000","last_peered":"2024-04-23T14:38:41.330807+0000","last_clean":"2024-04-23T14:38:41.330807+0000","last_became_active":"2024-04-23T14:38:39.775681+0000","last_became_peered":"2024-04-23T14:38:39.775681+0000","last_unstale":"2024-04-23T14:38:41.330807+0000","last_undegraded":"2024-04-23T14:38:41.330807+0000","last_fullsized":"2024-04-23T14:38:41.330807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:01.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331549+0000","last_change":"2024-04-23T14:38:38.766098+0000","last_active":"2024-04-23T14:38:41.331549+0000","last_peered":"2024-04-23T14:38:41.331549+0000","last_clean":"2024-04-23T14:38:41.331549+0000","last_became_active":"2024-04-23T14:38:38.765736+0000","last_became_peered":"2024-04-23T14:38:38.765736+0000","last_unstale":"2024-04-23T14:38:41.331549+0000","last_undegraded":"2024-04-23T14:38:41.331549+0000","last_fullsized":"2024-04-23T14:38:41.331549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:16:42.199678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332810+0000","last_change":"2024-04-23T14:38:39.756696+0000","last_active":"2024-04-23T14:38:41.332810+0000","last_peered":"2024-04-23T14:38:41.332810+0000","last_clean":"2024-04-23T14:38:41.332810+0000","last_became_active":"2024-04-23T14:38:39.756400+0000","last_became_peered":"2024-04-23T14:38:39.756400+0000","last_unstale":"2024-04-23T14:38:41.332810+0000","last_undegraded":"2024-04-23T14:38:41.332810+0000","last_fullsized":"2024-04-23T14:38:41.332810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:10.574496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331644+0000","last_change":"2024-04-23T14:38:38.758516+0000","last_active":"2024-04-23T14:38:41.331644+0000","last_peered":"2024-04-23T14:38:41.331644+0000","last_clean":"2024-04-23T14:38:41.331644+0000","last_became_active":"2024-04-23T14:38:38.758197+0000","last_became_peered":"2024-04-23T14:38:38.758197+0000","last_unstale":"2024-04-23T14:38:41.331644+0000","last_undegraded":"2024-04-23T14:38:41.331644+0000","last_fullsized":"2024-04-23T14:38:41.331644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:50:57.443007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765001+0000","last_change":"2024-04-23T14:38:39.767473+0000","last_active":"2024-04-23T14:38:40.765001+0000","last_peered":"2024-04-23T14:38:40.765001+0000","last_clean":"2024-04-23T14:38:40.765001+0000","last_became_active":"2024-04-23T14:38:39.767143+0000","last_became_peered":"2024-04-23T14:38:39.767143+0000","last_unstale":"2024-04-23T14:38:40.765001+0000","last_undegraded":"2024-04-23T14:38:40.765001+0000","last_fullsized":"2024-04-23T14:38:40.765001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:51:46.018659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330496+0000","last_change":"2024-04-23T14:38:38.757183+0000","last_active":"2024-04-23T14:38:41.330496+0000","last_peered":"2024-04-23T14:38:41.330496+0000","last_clean":"2024-04-23T14:38:41.330496+0000","last_became_active":"2024-04-23T14:38:38.756775+0000","last_became_peered":"2024-04-23T14:38:38.756775+0000","last_unstale":"2024-04-23T14:38:41.330496+0000","last_undegraded":"2024-04-23T14:38:41.330496+0000","last_fullsized":"2024-04-23T14:38:41.330496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:43:04.315920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331644+0000","last_change":"2024-04-23T14:38:39.758070+0000","last_active":"2024-04-23T14:38:41.331644+0000","last_peered":"2024-04-23T14:38:41.331644+0000","last_clean":"2024-04-23T14:38:41.331644+0000","last_became_active":"2024-04-23T14:38:39.757002+0000","last_became_peered":"2024-04-23T14:38:39.757002+0000","last_unstale":"2024-04-23T14:38:41.331644+0000","last_undegraded":"2024-04-23T14:38:41.331644+0000","last_fullsized":"2024-04-23T14:38:41.331644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:01.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330408+0000","last_change":"2024-04-23T14:38:38.758385+0000","last_active":"2024-04-23T14:38:41.330408+0000","last_peered":"2024-04-23T14:38:41.330408+0000","last_clean":"2024-04-23T14:38:41.330408+0000","last_became_active":"2024-04-23T14:38:38.756673+0000","last_became_peered":"2024-04-23T14:38:38.756673+0000","last_unstale":"2024-04-23T14:38:41.330408+0000","last_undegraded":"2024-04-23T14:38:41.330408+0000","last_fullsized":"2024-04-23T14:38:41.330408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:25:02.055193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764914+0000","last_change":"2024-04-23T14:38:39.772743+0000","last_active":"2024-04-23T14:38:40.764914+0000","last_peered":"2024-04-23T14:38:40.764914+0000","last_clean":"2024-04-23T14:38:40.764914+0000","last_became_active":"2024-04-23T14:38:39.771878+0000","last_became_peered":"2024-04-23T14:38:39.771878+0000","last_unstale":"2024-04-23T14:38:40.764914+0000","last_undegraded":"2024-04-23T14:38:40.764914+0000","last_fullsized":"2024-04-23T14:38:40.764914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:10.574496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.761744+0000","last_change":"2024-04-23T14:38:38.760828+0000","last_active":"2024-04-23T14:38:40.761744+0000","last_peered":"2024-04-23T14:38:40.761744+0000","last_clean":"2024-04-23T14:38:40.761744+0000","last_became_active":"2024-04-23T14:38:38.760535+0000","last_became_peered":"2024-04-23T14:38:38.760535+0000","last_unstale":"2024-04-23T14:38:40.761744+0000","last_undegraded":"2024-04-23T14:38:40.761744+0000","last_fullsized":"2024-04-23T14:38:40.761744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:41.354760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.328636+0000","last_change":"2024-04-23T14:38:39.776168+0000","last_active":"2024-04-23T14:38:41.328636+0000","last_peered":"2024-04-23T14:38:41.328636+0000","last_clean":"2024-04-23T14:38:41.328636+0000","last_became_active":"2024-04-23T14:38:39.775895+0000","last_became_peered":"2024-04-23T14:38:39.775895+0000","last_unstale":"2024-04-23T14:38:41.328636+0000","last_undegraded":"2024-04-23T14:38:41.328636+0000","last_fullsized":"2024-04-23T14:38:41.328636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:30.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329920+0000","last_change":"2024-04-23T14:38:38.763751+0000","last_active":"2024-04-23T14:38:41.329920+0000","last_peered":"2024-04-23T14:38:41.329920+0000","last_clean":"2024-04-23T14:38:41.329920+0000","last_became_active":"2024-04-23T14:38:38.763447+0000","last_became_peered":"2024-04-23T14:38:38.763447+0000","last_unstale":"2024-04-23T14:38:41.329920+0000","last_undegraded":"2024-04-23T14:38:41.329920+0000","last_fullsized":"2024-04-23T14:38:41.329920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:56:24.819213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329504+0000","last_change":"2024-04-23T14:38:39.758147+0000","last_active":"2024-04-23T14:38:41.329504+0000","last_peered":"2024-04-23T14:38:41.329504+0000","last_clean":"2024-04-23T14:38:41.329504+0000","last_became_active":"2024-04-23T14:38:39.753324+0000","last_became_peered":"2024-04-23T14:38:39.753324+0000","last_unstale":"2024-04-23T14:38:41.329504+0000","last_undegraded":"2024-04-23T14:38:41.329504+0000","last_fullsized":"2024-04-23T14:38:41.329504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:04.851824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330208+0000","last_change":"2024-04-23T14:38:38.763686+0000","last_active":"2024-04-23T14:38:41.330208+0000","last_peered":"2024-04-23T14:38:41.330208+0000","last_clean":"2024-04-23T14:38:41.330208+0000","last_became_active":"2024-04-23T14:38:38.762993+0000","last_became_peered":"2024-04-23T14:38:38.762993+0000","last_unstale":"2024-04-23T14:38:41.330208+0000","last_undegraded":"2024-04-23T14:38:41.330208+0000","last_fullsized":"2024-04-23T14:38:41.330208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:30:15.211062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765357+0000","last_change":"2024-04-23T14:38:39.765747+0000","last_active":"2024-04-23T14:38:40.765357+0000","last_peered":"2024-04-23T14:38:40.765357+0000","last_clean":"2024-04-23T14:38:40.765357+0000","last_became_active":"2024-04-23T14:38:39.765469+0000","last_became_peered":"2024-04-23T14:38:39.765469+0000","last_unstale":"2024-04-23T14:38:40.765357+0000","last_undegraded":"2024-04-23T14:38:40.765357+0000","last_fullsized":"2024-04-23T14:38:40.765357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:43:17.524210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332768+0000","last_change":"2024-04-23T14:38:38.750732+0000","last_active":"2024-04-23T14:38:41.332768+0000","last_peered":"2024-04-23T14:38:41.332768+0000","last_clean":"2024-04-23T14:38:41.332768+0000","last_became_active":"2024-04-23T14:38:38.749763+0000","last_became_peered":"2024-04-23T14:38:38.749763+0000","last_unstale":"2024-04-23T14:38:41.332768+0000","last_undegraded":"2024-04-23T14:38:41.332768+0000","last_fullsized":"2024-04-23T14:38:41.332768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:04.125291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332508+0000","last_change":"2024-04-23T14:38:39.765831+0000","last_active":"2024-04-23T14:38:41.332508+0000","last_peered":"2024-04-23T14:38:41.332508+0000","last_clean":"2024-04-23T14:38:41.332508+0000","last_became_active":"2024-04-23T14:38:39.765025+0000","last_became_peered":"2024-04-23T14:38:39.765025+0000","last_unstale":"2024-04-23T14:38:41.332508+0000","last_undegraded":"2024-04-23T14:38:41.332508+0000","last_fullsized":"2024-04-23T14:38:41.332508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:56.925422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.333288+0000","last_change":"2024-04-23T14:38:38.765461+0000","last_active":"2024-04-23T14:38:41.333288+0000","last_peered":"2024-04-23T14:38:41.333288+0000","last_clean":"2024-04-23T14:38:41.333288+0000","last_became_active":"2024-04-23T14:38:38.764564+0000","last_became_peered":"2024-04-23T14:38:38.764564+0000","last_unstale":"2024-04-23T14:38:41.333288+0000","last_undegraded":"2024-04-23T14:38:41.333288+0000","last_fullsized":"2024-04-23T14:38:41.333288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:35.906224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765790+0000","last_change":"2024-04-23T14:38:39.770979+0000","last_active":"2024-04-23T14:38:40.765790+0000","last_peered":"2024-04-23T14:38:40.765790+0000","last_clean":"2024-04-23T14:38:40.765790+0000","last_became_active":"2024-04-23T14:38:39.770320+0000","last_became_peered":"2024-04-23T14:38:39.770320+0000","last_unstale":"2024-04-23T14:38:40.765790+0000","last_undegraded":"2024-04-23T14:38:40.765790+0000","last_fullsized":"2024-04-23T14:38:40.765790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:43:05.332606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.333080+0000","last_change":"2024-04-23T14:38:38.753700+0000","last_active":"2024-04-23T14:38:41.333080+0000","last_peered":"2024-04-23T14:38:41.333080+0000","last_clean":"2024-04-23T14:38:41.333080+0000","last_became_active":"2024-04-23T14:38:38.753417+0000","last_became_peered":"2024-04-23T14:38:38.753417+0000","last_unstale":"2024-04-23T14:38:41.333080+0000","last_undegraded":"2024-04-23T14:38:41.333080+0000","last_fullsized":"2024-04-23T14:38:41.333080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:59:52.162913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.328339+0000","last_change":"2024-04-23T14:38:39.761259+0000","last_active":"2024-04-23T14:38:41.328339+0000","last_peered":"2024-04-23T14:38:41.328339+0000","last_clean":"2024-04-23T14:38:41.328339+0000","last_became_active":"2024-04-23T14:38:39.760960+0000","last_became_peered":"2024-04-23T14:38:39.760960+0000","last_unstale":"2024-04-23T14:38:41.328339+0000","last_undegraded":"2024-04-23T14:38:41.328339+0000","last_fullsized":"2024-04-23T14:38:41.328339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:49.858660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.761912+0000","last_change":"2024-04-23T14:38:38.758485+0000","last_active":"2024-04-23T14:38:40.761912+0000","last_peered":"2024-04-23T14:38:40.761912+0000","last_clean":"2024-04-23T14:38:40.761912+0000","last_became_active":"2024-04-23T14:38:38.758158+0000","last_became_peered":"2024-04-23T14:38:38.758158+0000","last_unstale":"2024-04-23T14:38:40.761912+0000","last_undegraded":"2024-04-23T14:38:40.761912+0000","last_fullsized":"2024-04-23T14:38:40.761912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:33.551735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330028+0000","last_change":"2024-04-23T14:38:39.763864+0000","last_active":"2024-04-23T14:38:41.330028+0000","last_peered":"2024-04-23T14:38:41.330028+0000","last_clean":"2024-04-23T14:38:41.330028+0000","last_became_active":"2024-04-23T14:38:39.763545+0000","last_became_peered":"2024-04-23T14:38:39.763545+0000","last_unstale":"2024-04-23T14:38:41.330028+0000","last_undegraded":"2024-04-23T14:38:41.330028+0000","last_fullsized":"2024-04-23T14:38:41.330028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:23:05.356934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.328619+0000","last_change":"2024-04-23T14:38:38.742178+0000","last_active":"2024-04-23T14:38:41.328619+0000","last_peered":"2024-04-23T14:38:41.328619+0000","last_clean":"2024-04-23T14:38:41.328619+0000","last_became_active":"2024-04-23T14:38:38.741810+0000","last_became_peered":"2024-04-23T14:38:38.741810+0000","last_unstale":"2024-04-23T14:38:41.328619+0000","last_undegraded":"2024-04-23T14:38:41.328619+0000","last_fullsized":"2024-04-23T14:38:41.328619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:01:09.624382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.327936+0000","last_change":"2024-04-23T14:38:39.766829+0000","last_active":"2024-04-23T14:38:41.327936+0000","last_peered":"2024-04-23T14:38:41.327936+0000","last_clean":"2024-04-23T14:38:41.327936+0000","last_became_active":"2024-04-23T14:38:39.766393+0000","last_became_peered":"2024-04-23T14:38:39.766393+0000","last_unstale":"2024-04-23T14:38:41.327936+0000","last_undegraded":"2024-04-23T14:38:41.327936+0000","last_fullsized":"2024-04-23T14:38:41.327936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:35:00.637051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.328487+0000","last_change":"2024-04-23T14:38:38.753322+0000","last_active":"2024-04-23T14:38:41.328487+0000","last_peered":"2024-04-23T14:38:41.328487+0000","last_clean":"2024-04-23T14:38:41.328487+0000","last_became_active":"2024-04-23T14:38:38.751669+0000","last_became_peered":"2024-04-23T14:38:38.751669+0000","last_unstale":"2024-04-23T14:38:41.328487+0000","last_undegraded":"2024-04-23T14:38:41.328487+0000","last_fullsized":"2024-04-23T14:38:41.328487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:25:43.461314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329596+0000","last_change":"2024-04-23T14:38:39.765289+0000","last_active":"2024-04-23T14:38:41.329596+0000","last_peered":"2024-04-23T14:38:41.329596+0000","last_clean":"2024-04-23T14:38:41.329596+0000","last_became_active":"2024-04-23T14:38:39.764962+0000","last_became_peered":"2024-04-23T14:38:39.764962+0000","last_unstale":"2024-04-23T14:38:41.329596+0000","last_undegraded":"2024-04-23T14:38:41.329596+0000","last_fullsized":"2024-04-23T14:38:41.329596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:31:02.227117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.761084+0000","last_change":"2024-04-23T14:38:38.753720+0000","last_active":"2024-04-23T14:38:40.761084+0000","last_peered":"2024-04-23T14:38:40.761084+0000","last_clean":"2024-04-23T14:38:40.761084+0000","last_became_active":"2024-04-23T14:38:38.753419+0000","last_became_peered":"2024-04-23T14:38:38.753419+0000","last_unstale":"2024-04-23T14:38:40.761084+0000","last_undegraded":"2024-04-23T14:38:40.761084+0000","last_fullsized":"2024-04-23T14:38:40.761084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:39.127658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.766082+0000","last_change":"2024-04-23T14:38:39.758440+0000","last_active":"2024-04-23T14:38:40.766082+0000","last_peered":"2024-04-23T14:38:40.766082+0000","last_clean":"2024-04-23T14:38:40.766082+0000","last_became_active":"2024-04-23T14:38:39.758127+0000","last_became_peered":"2024-04-23T14:38:39.758127+0000","last_unstale":"2024-04-23T14:38:40.766082+0000","last_undegraded":"2024-04-23T14:38:40.766082+0000","last_fullsized":"2024-04-23T14:38:40.766082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:30:16.227749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332083+0000","last_change":"2024-04-23T14:38:38.764821+0000","last_active":"2024-04-23T14:38:41.332083+0000","last_peered":"2024-04-23T14:38:41.332083+0000","last_clean":"2024-04-23T14:38:41.332083+0000","last_became_active":"2024-04-23T14:38:38.764410+0000","last_became_peered":"2024-04-23T14:38:38.764410+0000","last_unstale":"2024-04-23T14:38:41.332083+0000","last_undegraded":"2024-04-23T14:38:41.332083+0000","last_fullsized":"2024-04-23T14:38:41.332083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:38.775307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765452+0000","last_change":"2024-04-23T14:38:39.763443+0000","last_active":"2024-04-23T14:38:40.765452+0000","last_peered":"2024-04-23T14:38:40.765452+0000","last_clean":"2024-04-23T14:38:40.765452+0000","last_became_active":"2024-04-23T14:38:39.763060+0000","last_became_peered":"2024-04-23T14:38:39.763060+0000","last_unstale":"2024-04-23T14:38:40.765452+0000","last_undegraded":"2024-04-23T14:38:40.765452+0000","last_fullsized":"2024-04-23T14:38:40.765452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:38:13.983451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332486+0000","last_change":"2024-04-23T14:38:38.758493+0000","last_active":"2024-04-23T14:38:41.332486+0000","last_peered":"2024-04-23T14:38:41.332486+0000","last_clean":"2024-04-23T14:38:41.332486+0000","last_became_active":"2024-04-23T14:38:38.758152+0000","last_became_peered":"2024-04-23T14:38:38.758152+0000","last_unstale":"2024-04-23T14:38:41.332486+0000","last_undegraded":"2024-04-23T14:38:41.332486+0000","last_fullsized":"2024-04-23T14:38:41.332486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:51:30.664781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331835+0000","last_change":"2024-04-23T14:38:39.772347+0000","last_active":"2024-04-23T14:38:41.331835+0000","last_peered":"2024-04-23T14:38:41.331835+0000","last_clean":"2024-04-23T14:38:41.331835+0000","last_became_active":"2024-04-23T14:38:39.770952+0000","last_became_peered":"2024-04-23T14:38:39.770952+0000","last_unstale":"2024-04-23T14:38:41.331835+0000","last_undegraded":"2024-04-23T14:38:41.331835+0000","last_fullsized":"2024-04-23T14:38:41.331835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:23:05.356934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331100+0000","last_change":"2024-04-23T14:38:38.748633+0000","last_active":"2024-04-23T14:38:41.331100+0000","last_peered":"2024-04-23T14:38:41.331100+0000","last_clean":"2024-04-23T14:38:41.331100+0000","last_became_active":"2024-04-23T14:38:38.748127+0000","last_became_peered":"2024-04-23T14:38:38.748127+0000","last_unstale":"2024-04-23T14:38:41.331100+0000","last_undegraded":"2024-04-23T14:38:41.331100+0000","last_fullsized":"2024-04-23T14:38:41.331100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:03.195430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765837+0000","last_change":"2024-04-23T14:38:39.767485+0000","last_active":"2024-04-23T14:38:40.765837+0000","last_peered":"2024-04-23T14:38:40.765837+0000","last_clean":"2024-04-23T14:38:40.765837+0000","last_became_active":"2024-04-23T14:38:39.767086+0000","last_became_peered":"2024-04-23T14:38:39.767086+0000","last_unstale":"2024-04-23T14:38:40.765837+0000","last_undegraded":"2024-04-23T14:38:40.765837+0000","last_fullsized":"2024-04-23T14:38:40.765837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:59.703173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331976+0000","last_change":"2024-04-23T14:38:38.755614+0000","last_active":"2024-04-23T14:38:41.331976+0000","last_peered":"2024-04-23T14:38:41.331976+0000","last_clean":"2024-04-23T14:38:41.331976+0000","last_became_active":"2024-04-23T14:38:38.752060+0000","last_became_peered":"2024-04-23T14:38:38.752060+0000","last_unstale":"2024-04-23T14:38:41.331976+0000","last_undegraded":"2024-04-23T14:38:41.331976+0000","last_fullsized":"2024-04-23T14:38:41.331976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:50.247873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.329658+0000","last_change":"2024-04-23T14:38:39.761753+0000","last_active":"2024-04-23T14:38:41.329658+0000","last_peered":"2024-04-23T14:38:41.329658+0000","last_clean":"2024-04-23T14:38:41.329658+0000","last_became_active":"2024-04-23T14:38:39.761068+0000","last_became_peered":"2024-04-23T14:38:39.761068+0000","last_unstale":"2024-04-23T14:38:41.329658+0000","last_undegraded":"2024-04-23T14:38:41.329658+0000","last_fullsized":"2024-04-23T14:38:41.329658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:53.374381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332572+0000","last_change":"2024-04-23T14:38:38.763486+0000","last_active":"2024-04-23T14:38:41.332572+0000","last_peered":"2024-04-23T14:38:41.332572+0000","last_clean":"2024-04-23T14:38:41.332572+0000","last_became_active":"2024-04-23T14:38:38.763125+0000","last_became_peered":"2024-04-23T14:38:38.763125+0000","last_unstale":"2024-04-23T14:38:41.332572+0000","last_undegraded":"2024-04-23T14:38:41.332572+0000","last_fullsized":"2024-04-23T14:38:41.332572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:41:47.402639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.332665+0000","last_change":"2024-04-23T14:38:39.763937+0000","last_active":"2024-04-23T14:38:41.332665+0000","last_peered":"2024-04-23T14:38:41.332665+0000","last_clean":"2024-04-23T14:38:41.332665+0000","last_became_active":"2024-04-23T14:38:39.763198+0000","last_became_peered":"2024-04-23T14:38:39.763198+0000","last_unstale":"2024-04-23T14:38:41.332665+0000","last_undegraded":"2024-04-23T14:38:41.332665+0000","last_fullsized":"2024-04-23T14:38:41.332665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:13:56.925422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765176+0000","last_change":"2024-04-23T14:38:38.763771+0000","last_active":"2024-04-23T14:38:40.765176+0000","last_peered":"2024-04-23T14:38:40.765176+0000","last_clean":"2024-04-23T14:38:40.765176+0000","last_became_active":"2024-04-23T14:38:38.762540+0000","last_became_peered":"2024-04-23T14:38:38.762540+0000","last_unstale":"2024-04-23T14:38:40.765176+0000","last_undegraded":"2024-04-23T14:38:40.765176+0000","last_fullsized":"2024-04-23T14:38:40.765176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:57.269105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764546+0000","last_change":"2024-04-23T14:38:39.764297+0000","last_active":"2024-04-23T14:38:40.764546+0000","last_peered":"2024-04-23T14:38:40.764546+0000","last_clean":"2024-04-23T14:38:40.764546+0000","last_became_active":"2024-04-23T14:38:39.763823+0000","last_became_peered":"2024-04-23T14:38:39.763823+0000","last_unstale":"2024-04-23T14:38:40.764546+0000","last_undegraded":"2024-04-23T14:38:40.764546+0000","last_fullsized":"2024-04-23T14:38:40.764546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:20:01.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331305+0000","last_change":"2024-04-23T14:38:38.752186+0000","last_active":"2024-04-23T14:38:41.331305+0000","last_peered":"2024-04-23T14:38:41.331305+0000","last_clean":"2024-04-23T14:38:41.331305+0000","last_became_active":"2024-04-23T14:38:38.751597+0000","last_became_peered":"2024-04-23T14:38:38.751597+0000","last_unstale":"2024-04-23T14:38:41.331305+0000","last_undegraded":"2024-04-23T14:38:41.331305+0000","last_fullsized":"2024-04-23T14:38:41.331305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:28.293882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.331042+0000","last_change":"2024-04-23T14:38:39.761423+0000","last_active":"2024-04-23T14:38:41.331042+0000","last_peered":"2024-04-23T14:38:41.331042+0000","last_clean":"2024-04-23T14:38:41.331042+0000","last_became_active":"2024-04-23T14:38:39.760214+0000","last_became_peered":"2024-04-23T14:38:39.760214+0000","last_unstale":"2024-04-23T14:38:41.331042+0000","last_undegraded":"2024-04-23T14:38:41.331042+0000","last_fullsized":"2024-04-23T14:38:41.331042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:31:02.227117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764065+0000","last_change":"2024-04-23T14:38:38.763764+0000","last_active":"2024-04-23T14:38:40.764065+0000","last_peered":"2024-04-23T14:38:40.764065+0000","last_clean":"2024-04-23T14:38:40.764065+0000","last_became_active":"2024-04-23T14:38:38.762666+0000","last_became_peered":"2024-04-23T14:38:38.762666+0000","last_unstale":"2024-04-23T14:38:40.764065+0000","last_undegraded":"2024-04-23T14:38:40.764065+0000","last_fullsized":"2024-04-23T14:38:40.764065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:18:34.199124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:41.330128+0000","last_change":"2024-04-23T14:38:39.772062+0000","last_active":"2024-04-23T14:38:41.330128+0000","last_peered":"2024-04-23T14:38:41.330128+0000","last_clean":"2024-04-23T14:38:41.330128+0000","last_became_active":"2024-04-23T14:38:39.770602+0000","last_became_peered":"2024-04-23T14:38:39.770602+0000","last_unstale":"2024-04-23T14:38:41.330128+0000","last_undegraded":"2024-04-23T14:38:41.330128+0000","last_fullsized":"2024-04-23T14:38:41.330128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:38.711438+0000","last_clean_scrub_stamp":"2024-04-23T14:38:38.711438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:51.264559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.765286+0000","last_change":"2024-04-23T14:38:38.765939+0000","last_active":"2024-04-23T14:38:40.765286+0000","last_peered":"2024-04-23T14:38:40.765286+0000","last_clean":"2024-04-23T14:38:40.765286+0000","last_became_active":"2024-04-23T14:38:38.765621+0000","last_became_peered":"2024-04-23T14:38:38.765621+0000","last_unstale":"2024-04-23T14:38:40.765286+0000","last_undegraded":"2024-04-23T14:38:40.765286+0000","last_fullsized":"2024-04-23T14:38:40.765286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:38:37.694751+0000","last_clean_scrub_stamp":"2024-04-23T14:38:37.694751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:19.788945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":285,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-23T14:38:40.764679+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:38:40.764679+0000","last_peered":"2024-04-23T14:38:40.764679+0000","last_clean":"2024-04-23T14:38:40.764679+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:38:40.764679+0000","last_undegraded":"2024-04-23T14:38:40.764679+0000","last_fullsized":"2024-04-23T14:38:40.764679+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1617920,"data_stored":1026120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":436376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":437316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":435682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":437294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":436264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":435876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1617920,"data_stored":1026084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:38:44.748 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:38:44.748 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:38:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:44.999+0000 7f8965600640 1 Processor -- start 2024-04-23T14:38:45.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.000+0000 7f8965600640 1 -- start start 2024-04-23T14:38:45.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.001+0000 7f8965600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f89600732d0 0x7f89600b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.001+0000 7f8965600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896006f790 con 0x7f89600b3190 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.001+0000 7f8965600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8960162ef0 con 0x7f8960162b10 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.001+0000 7f8965600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f896006eef0 con 0x7f89600732d0 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.002+0000 7f8957e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f89600b3190 0x7f89600b3620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39180/0 (socket says 172.21.15.59:39180) 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.002+0000 7f8957e00640 1 -- 172.21.15.59:0/2918826240 learned_addr learned my addr 172.21.15.59:0/2918826240 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.002+0000 7f8964c00640 1 --2- 172.21.15.59:0/2918826240 >> v2:172.21.15.76:3300/0 conn(0x7f89600732d0 0x7f89600b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.002+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099586315 0 0) 0x7f896006f790 con 0x7f89600b3190 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.002+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8944003520 con 0x7f89600b3190 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2255640326 0 0) 0x7f8944003520 con 0x7f89600b3190 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 >> v2:172.21.15.76:3300/0 conn(0x7f89600732d0 msgr2=0x7f89600b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 --2- 172.21.15.59:0/2918826240 >> v2:172.21.15.76:3300/0 conn(0x7f89600732d0 0x7f89600b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 >> v1:172.21.15.76:6789/0 conn(0x7f8960162b10 legacy=0x7f896006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896006f1d0 con 0x7f89600b3190 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8948001670 con 0x7f89600b3190 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 >> v1:172.21.15.59:6789/0 conn(0x7f89600b3190 legacy=0x7f89600b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440039b0 0x7f8944003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8944004800 0x7f8944004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89440052e0 0x7f894400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f896006eef0 con 0x7f89440052e0 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f894400c1b0 con 0x7f8944004800 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f894400c3b0 con 0x7f89440039b0 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.003+0000 7f8964c00640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89440052e0 0x7f894400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.004+0000 7f8957e00640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8944004800 0x7f8944004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.004+0000 7f895fe00640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440039b0 0x7f8944003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.004+0000 7f8964c00640 1 -- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440039b0 msgr2=0x7f8944003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.004+0000 7f8964c00640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440039b0 0x7f8944003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.004+0000 7f8964c00640 1 -- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8944004800 msgr2=0x7f8944004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.004+0000 7f8964c00640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8944004800 0x7f8944004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.004+0000 7f8964c00640 1 -- 172.21.15.59:0/2918826240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8944003520 con 0x7f89440052e0 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.005+0000 7f8964c00640 1 -- 172.21.15.59:0/2918826240 reap_dead start 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.005+0000 7f8964c00640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89440052e0 0x7f894400b9c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8958009f90 tx=0x7f8944005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c35446d096b45bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.005+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958035020 con 0x7f89440052e0 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.005+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8958003b90 con 0x7f89440052e0 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.005+0000 7f895f400640 1 -- 172.21.15.59:0/2918826240 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958009c10 con 0x7f89440052e0 2024-04-23T14:38:45.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.006+0000 7f8965600640 1 -- 172.21.15.59:0/2918826240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89440052e0 msgr2=0x7f894400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.006+0000 7f8965600640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89440052e0 0x7f894400b9c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8958009f90 tx=0x7f8944005830 comp rx=0 tx=0).stop 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.006+0000 7f8965600640 1 -- 172.21.15.59:0/2918826240 shutdown_connections 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.006+0000 7f8965600640 1 --2- 172.21.15.59:0/2918826240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89440052e0 0x7f894400b9c0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.006+0000 7f8965600640 1 -- 172.21.15.59:0/2918826240 >> 172.21.15.59:0/2918826240 conn(0x7f89600b1e90 msgr2=0x7f89600b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.006+0000 7f8965600640 1 -- 172.21.15.59:0/2918826240 shutdown_connections 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.006+0000 7f8965600640 1 -- 172.21.15.59:0/2918826240 wait complete. 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.008+0000 7f8965600640 1 Processor -- start 2024-04-23T14:38:45.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.008+0000 7f8965600640 1 -- start start 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.009+0000 7f8965600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440052e0 0x7f896014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.009+0000 7f8965600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8960150270 0x7f896014e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.009+0000 7f8965600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896014ec30 0x7f8960150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.009+0000 7f8965600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f896006e6c0 con 0x7f8960150270 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.009+0000 7f8965600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8960071da0 con 0x7f896014ec30 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.009+0000 7f8965600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f896006e840 con 0x7f89440052e0 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8960150270 0x7f896014e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8960150270 0x7f896014e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43258/0 (socket says 172.21.15.59:43258) 2024-04-23T14:38:45.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 -- 172.21.15.59:0/991439096 learned_addr learned my addr 172.21.15.59:0/991439096 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f895fe00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440052e0 0x7f896014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8964c00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896014ec30 0x7f8960150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 -- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440052e0 msgr2=0x7f896014dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440052e0 0x7f896014dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 -- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896014ec30 msgr2=0x7f8960150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896014ec30 0x7f8960150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 -- 172.21.15.59:0/991439096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896014f1d0 con 0x7f8960150270 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f895fe00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440052e0 0x7f896014dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.010+0000 7f8957e00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8960150270 0x7f896014e6f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f894800e790 tx=0x7f894800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.011+0000 7f895cc00640 1 -- 172.21.15.59:0/991439096 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948019070 con 0x7f8960150270 2024-04-23T14:38:45.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.011+0000 7f895cc00640 1 -- 172.21.15.59:0/991439096 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894800c6c0 con 0x7f8960150270 2024-04-23T14:38:45.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.011+0000 7f895cc00640 1 -- 172.21.15.59:0/991439096 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894800ca90 con 0x7f8960150270 2024-04-23T14:38:45.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.011+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896014f4b0 con 0x7f8960150270 2024-04-23T14:38:45.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.011+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896014fa70 con 0x7f8960150270 2024-04-23T14:38:45.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.012+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896004fd90 con 0x7f8960150270 2024-04-23T14:38:45.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.013+0000 7f895cc00640 1 -- 172.21.15.59:0/991439096 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f894801c070 con 0x7f8960150270 2024-04-23T14:38:45.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.014+0000 7f895cc00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f893c06c8f0 0x7f893c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.014+0000 7f895cc00640 1 -- 172.21.15.59:0/991439096 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f894808b2b0 con 0x7f8960150270 2024-04-23T14:38:45.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.014+0000 7f895fe00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f893c06c8f0 0x7f893c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.015+0000 7f895fe00640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f893c06c8f0 0x7f893c06edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8950005580 tx=0x7f8950009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.018+0000 7f895cc00640 1 -- 172.21.15.59:0/991439096 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8948059640 con 0x7f8960150270 2024-04-23T14:38:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.299+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f89600b4a90 con 0x7f8960150270 2024-04-23T14:38:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.304+0000 7f895cc00640 1 -- 172.21.15.59:0/991439096 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f8948058f40 con 0x7f8960150270 2024-04-23T14:38:45.306 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:38:45.306 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:38:45.306 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-23T14:38:45.306 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-23T14:38:45.306 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:38:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-23T14:38:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f893c06c8f0 msgr2=0x7f893c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f893c06c8f0 0x7f893c06edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8950005580 tx=0x7f8950009040 comp rx=0 tx=0).stop 2024-04-23T14:38:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8960150270 msgr2=0x7f896014e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8960150270 0x7f896014e6f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f894800e790 tx=0x7f894800ec50 comp rx=0 tx=0).stop 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 shutdown_connections 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f893c06c8f0 0x7f893c06edb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896014ec30 0x7f8960150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8960150270 0x7f896014e6f0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 --2- 172.21.15.59:0/991439096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89440052e0 0x7f896014dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 >> 172.21.15.59:0/991439096 conn(0x7f89600b1e90 msgr2=0x7f89600b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 shutdown_connections 2024-04-23T14:38:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.307+0000 7f8965600640 1 -- 172.21.15.59:0/991439096 wait complete. 2024-04-23T14:38:45.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:38:45.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.540+0000 7fa1b8600640 1 Processor -- start 2024-04-23T14:38:45.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.540+0000 7fa1b8600640 1 -- start start 2024-04-23T14:38:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.541+0000 7fa1b8600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1b01454e0 0x7fa1b0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.541+0000 7fa1b8600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b00cd020 0x7fa1b00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.541+0000 7fa1b8600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b00cd520 con 0x7fa1b00b3e40 2024-04-23T14:38:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.541+0000 7fa1b8600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b00cddc0 con 0x7fa1b01454e0 2024-04-23T14:38:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.541+0000 7fa1b8600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b00cd800 con 0x7fa1b00cd020 2024-04-23T14:38:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.542+0000 7fa1b6600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa1b00b3e40 0x7fa1b0160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39184/0 (socket says 172.21.15.59:39184) 2024-04-23T14:38:45.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.542+0000 7fa1b6600640 1 -- 172.21.15.59:0/2191456312 learned_addr learned my addr 172.21.15.59:0/2191456312 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.542+0000 7fa1b5c00640 1 --2- 172.21.15.59:0/2191456312 >> v2:172.21.15.76:3300/0 conn(0x7fa1b01454e0 0x7fa1b0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.542+0000 7fa1b5200640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b00cd020 0x7fa1b00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.542+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2031803534 0 0) 0x7fa1b00cd520 con 0x7fa1b00b3e40 2024-04-23T14:38:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa194003520 con 0x7fa1b00b3e40 2024-04-23T14:38:45.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 715600935 0 0) 0x7fa194003520 con 0x7fa1b00b3e40 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b00cd020 msgr2=0x7fa1b00b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b00cd020 0x7fa1b00b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 >> v2:172.21.15.76:3300/0 conn(0x7fa1b01454e0 msgr2=0x7fa1b0145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 --2- 172.21.15.59:0/2191456312 >> v2:172.21.15.76:3300/0 conn(0x7fa1b01454e0 0x7fa1b0145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b00cdae0 con 0x7fa1b00b3e40 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa1a80019f0 con 0x7fa1b00b3e40 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 >> v1:172.21.15.59:6789/0 conn(0x7fa1b00b3e40 legacy=0x7fa1b0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b00cd020 0x7fa1940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa194003f80 0x7fa194004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b00cddc0 con 0x7fa194004d40 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b00cd800 con 0x7fa194003f80 2024-04-23T14:38:45.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa19400ba70 con 0x7fa1b00cd020 2024-04-23T14:38:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.543+0000 7fa1b6600640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.544+0000 7fa1b5200640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa194003f80 0x7fa194004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.544+0000 7fa1b6600640 1 -- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b00cd020 msgr2=0x7fa1940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.544+0000 7fa1b6600640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b00cd020 0x7fa1940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.544+0000 7fa1b6600640 1 -- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa194003f80 msgr2=0x7fa194004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.544+0000 7fa1b6600640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa194003f80 0x7fa194004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.544+0000 7fa1b6600640 1 -- 172.21.15.59:0/2191456312 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa194003520 con 0x7fa194004d40 2024-04-23T14:38:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.545+0000 7fa1b6600640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1940051c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800a7d0 tx=0x7fa194005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681343eb7154d793 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.545+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800a3f0 con 0x7fa194004d40 2024-04-23T14:38:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.545+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a802f070 con 0x7fa194004d40 2024-04-23T14:38:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.545+0000 7fa1afe00640 1 -- 172.21.15.59:0/2191456312 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a802bc90 con 0x7fa194004d40 2024-04-23T14:38:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.546+0000 7fa1b8600640 1 -- 172.21.15.59:0/2191456312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 msgr2=0x7fa1940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.546+0000 7fa1b8600640 1 --2- 172.21.15.59:0/2191456312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1940051c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800a7d0 tx=0x7fa194005830 comp rx=0 tx=0).stop 2024-04-23T14:38:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.546+0000 7fa1b6600640 1 -- 172.21.15.59:0/2191456312 reap_dead start 2024-04-23T14:38:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.546+0000 7fa1b8600640 1 -- 172.21.15.59:0/2191456312 shutdown_connections 2024-04-23T14:38:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.546+0000 7fa1b8600640 1 -- 172.21.15.59:0/2191456312 >> 172.21.15.59:0/2191456312 conn(0x7fa1b0067ea0 msgr2=0x7fa1b006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.546+0000 7fa1b8600640 1 -- 172.21.15.59:0/2191456312 shutdown_connections 2024-04-23T14:38:45.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.546+0000 7fa1b8600640 1 -- 172.21.15.59:0/2191456312 wait complete. 2024-04-23T14:38:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.548+0000 7fa1b8600640 1 Processor -- start 2024-04-23T14:38:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.548+0000 7fa1b8600640 1 -- start start 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b8600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1b015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b8600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1b015b380 0x7fa1b00b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b8600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b015b800 0x7fa1b00b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b8600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0146270 con 0x7fa194004d40 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b8600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b01460f0 con 0x7fa1b015b380 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b8600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b0161020 con 0x7fa1b015b800 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1b015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b5c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1b015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43278/0 (socket says 172.21.15.59:43278) 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b5c00640 1 -- 172.21.15.59:0/2467557592 learned_addr learned my addr 172.21.15.59:0/2467557592 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b5c00640 1 -- 172.21.15.59:0/2467557592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b015b800 msgr2=0x7fa1b00b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.549+0000 7fa1b5c00640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b015b800 0x7fa1b00b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1b5200640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1b015b380 0x7fa1b00b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1b5c00640 1 -- 172.21.15.59:0/2467557592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1b015b380 msgr2=0x7fa1b00b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1b5c00640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1b015b380 0x7fa1b00b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1b5c00640 1 -- 172.21.15.59:0/2467557592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b0141a30 con 0x7fa194004d40 2024-04-23T14:38:45.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1b5c00640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1b015c190 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000e820 tx=0x7fa1a000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1ad600640 1 -- 172.21.15.59:0/2467557592 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0017070 con 0x7fa194004d40 2024-04-23T14:38:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1ad600640 1 -- 172.21.15.59:0/2467557592 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a000c860 con 0x7fa194004d40 2024-04-23T14:38:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1ad600640 1 -- 172.21.15.59:0/2467557592 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000cc50 con 0x7fa194004d40 2024-04-23T14:38:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b015cca0 con 0x7fa194004d40 2024-04-23T14:38:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.550+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b015d260 con 0x7fa194004d40 2024-04-23T14:38:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.551+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b0068a80 con 0x7fa194004d40 2024-04-23T14:38:45.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.552+0000 7fa1ad600640 1 -- 172.21.15.59:0/2467557592 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1a001a070 con 0x7fa194004d40 2024-04-23T14:38:45.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.553+0000 7fa1ad600640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa19006c8f0 0x7fa19006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.553+0000 7fa1ad600640 1 -- 172.21.15.59:0/2467557592 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa1a008b9f0 con 0x7fa194004d40 2024-04-23T14:38:45.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.554+0000 7fa1b5200640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa19006c8f0 0x7fa19006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.554+0000 7fa1b5200640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa19006c8f0 0x7fa19006edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00b5510 tx=0x7fa19c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.558+0000 7fa1ad600640 1 -- 172.21.15.59:0/2467557592 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1b0141a30 con 0x7fa194004d40 2024-04-23T14:38:45.694 INFO:tasks.ceph.mds.b.smithi076.stderr:2024-04-23T14:38:45.693+0000 7f8755a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:38:45.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.801+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1b0160030 con 0x7fa194004d40 2024-04-23T14:38:45.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.803+0000 7fa1ad600640 1 -- 172.21.15.59:0/2467557592 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5183 (secure 0 0 0) 0x7fa1a0059d80 con 0x7fa194004d40 2024-04-23T14:38:45.805 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:45.805 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:38:42.286820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3870021000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3870021000},{"type":"v1","addr":"172.21.15.76:6835","nonce":3870021000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":6}]} 2024-04-23T14:38:45.805 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 60 2024-04-23T14:38:45.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa19006c8f0 msgr2=0x7fa19006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa19006c8f0 0x7fa19006edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00b5510 tx=0x7fa19c008040 comp rx=0 tx=0).stop 2024-04-23T14:38:45.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 msgr2=0x7fa1b015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1b015c190 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000e820 tx=0x7fa1a000ece0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 shutdown_connections 2024-04-23T14:38:45.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa19006c8f0 0x7fa19006edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1b015b800 0x7fa1b00b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1b015b380 0x7fa1b00b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 --2- 172.21.15.59:0/2467557592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa194004d40 0x7fa1b015c190 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 >> 172.21.15.59:0/2467557592 conn(0x7fa1b0067ea0 msgr2=0x7fa1b00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:45.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 shutdown_connections 2024-04-23T14:38:45.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:45.806+0000 7fa1b8600640 1 -- 172.21.15.59:0/2467557592 wait complete. 2024-04-23T14:38:46.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:38:39.210981+0000', 'modified': '2024-04-23T14:38:42.286820+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7545': {'gid': 7545, 'name': 'b', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/3870021000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6833', 'nonce': 3870021000}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3870021000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7545, 'qdb_cluster': [7545]} 2024-04-23T14:38:46.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:38:46.229 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T14:38:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.451+0000 7f2272400640 1 Processor -- start 2024-04-23T14:38:46.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.451+0000 7f2272400640 1 -- start start 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2272400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f226c0cd8b0 0x7f226c0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2272400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f226c0d0b00 0x7f226c06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2272400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f226c0b4cf0 con 0x7f226c15c670 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2272400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f226c15ca50 con 0x7f226c0cd8b0 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2272400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f226c0ccb80 con 0x7f226c0d0b00 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2271000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f226c15c670 0x7f226c0b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39186/0 (socket says 172.21.15.59:39186) 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2271000640 1 -- 172.21.15.59:0/578117264 learned_addr learned my addr 172.21.15.59:0/578117264 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f226be00640 1 --2- 172.21.15.59:0/578117264 >> v2:172.21.15.59:3300/0 conn(0x7f226c0cd8b0 0x7f226c0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f2271a00640 1 --2- 172.21.15.59:0/578117264 >> v2:172.21.15.76:3300/0 conn(0x7f226c0d0b00 0x7f226c06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.453+0000 7f226b400640 1 -- 172.21.15.59:0/578117264 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3850892542 0 0) 0x7f226c0b4cf0 con 0x7f226c15c670 2024-04-23T14:38:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.454+0000 7f226b400640 1 -- 172.21.15.59:0/578117264 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2254003520 con 0x7f226c15c670 2024-04-23T14:38:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.454+0000 7f226be00640 1 -- 172.21.15.59:0/578117264 >> v2:172.21.15.76:3300/0 conn(0x7f226c0d0b00 msgr2=0x7f226c06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.454+0000 7f226be00640 1 --2- 172.21.15.59:0/578117264 >> v2:172.21.15.76:3300/0 conn(0x7f226c0d0b00 0x7f226c06ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.454+0000 7f226be00640 1 -- 172.21.15.59:0/578117264 >> v1:172.21.15.59:6789/0 conn(0x7f226c15c670 legacy=0x7f226c0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.454+0000 7f226be00640 1 -- 172.21.15.59:0/578117264 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c0ccdd0 con 0x7f226c0cd8b0 2024-04-23T14:38:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.454+0000 7f2271a00640 1 --2- 172.21.15.59:0/578117264 >> v2:172.21.15.76:3300/0 conn(0x7f226c0d0b00 0x7f226c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.455+0000 7f226be00640 1 --2- 172.21.15.59:0/578117264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0cd8b0 0x7f226c0d05c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f226c06c970 tx=0x7f225c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dfaae0cf1a9f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.455+0000 7f226b400640 1 -- 172.21.15.59:0/578117264 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c037070 con 0x7f226c0cd8b0 2024-04-23T14:38:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.455+0000 7f226b400640 1 -- 172.21.15.59:0/578117264 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225c032440 con 0x7f226c0cd8b0 2024-04-23T14:38:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.455+0000 7f226b400640 1 -- 172.21.15.59:0/578117264 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c0327f0 con 0x7f226c0cd8b0 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 -- 172.21.15.59:0/578117264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0cd8b0 msgr2=0x7f226c0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 --2- 172.21.15.59:0/578117264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0cd8b0 0x7f226c0d05c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f226c06c970 tx=0x7f225c02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 -- 172.21.15.59:0/578117264 shutdown_connections 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 --2- 172.21.15.59:0/578117264 >> v2:172.21.15.76:3300/0 conn(0x7f226c0d0b00 0x7f226c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 --2- 172.21.15.59:0/578117264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0cd8b0 0x7f226c0d05c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 -- 172.21.15.59:0/578117264 >> 172.21.15.59:0/578117264 conn(0x7f226c06bdf0 msgr2=0x7f226c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 -- 172.21.15.59:0/578117264 shutdown_connections 2024-04-23T14:38:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.456+0000 7f2272400640 1 -- 172.21.15.59:0/578117264 wait complete. 2024-04-23T14:38:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.458+0000 7f2272400640 1 Processor -- start 2024-04-23T14:38:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.458+0000 7f2272400640 1 -- start start 2024-04-23T14:38:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f2272400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226c0cd8b0 0x7f226c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f2272400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0d0b00 0x7f226c1e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f2272400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c15c670 0x7f226c1ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f2272400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0b3e60 con 0x7f226c0d0b00 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f2272400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0cd390 con 0x7f226c15c670 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f2272400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f226c0b3fe0 con 0x7f226c0cd8b0 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f226be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0d0b00 0x7f226c1e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f226be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0d0b00 0x7f226c1e9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43286/0 (socket says 172.21.15.59:43286) 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f226be00640 1 -- 172.21.15.59:0/2142327872 learned_addr learned my addr 172.21.15.59:0/2142327872 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f226be00640 1 -- 172.21.15.59:0/2142327872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226c0cd8b0 msgr2=0x7f226c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f226be00640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226c0cd8b0 0x7f226c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.459+0000 7f226be00640 1 -- 172.21.15.59:0/2142327872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c15c670 msgr2=0x7f226c1ea1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f2271a00640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c15c670 0x7f226c1ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f226be00640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c15c670 0x7f226c1ea1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f226be00640 1 -- 172.21.15.59:0/2142327872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c155c20 con 0x7f226c0d0b00 2024-04-23T14:38:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f226be00640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0d0b00 0x7f226c1e9cb0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f225c000c00 tx=0x7f225c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f2268c00640 1 -- 172.21.15.59:0/2142327872 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c037040 con 0x7f226c0d0b00 2024-04-23T14:38:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f2268c00640 1 -- 172.21.15.59:0/2142327872 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225c0304c0 con 0x7f226c0d0b00 2024-04-23T14:38:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c1e66d0 con 0x7f226c0d0b00 2024-04-23T14:38:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f2268c00640 1 -- 172.21.15.59:0/2142327872 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c030890 con 0x7f226c0d0b00 2024-04-23T14:38:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.460+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c1e7690 con 0x7f226c0d0b00 2024-04-23T14:38:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.463+0000 7f2268c00640 1 -- 172.21.15.59:0/2142327872 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f225c044020 con 0x7f226c0d0b00 2024-04-23T14:38:46.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.464+0000 7f2268c00640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f225006c8f0 0x7f225006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.464+0000 7f2268c00640 1 -- 172.21.15.59:0/2142327872 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f225c0abe00 con 0x7f226c0d0b00 2024-04-23T14:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.464+0000 7f2271000640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f225006c8f0 0x7f225006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.464+0000 7f22769d1740 1 Processor -- start 2024-04-23T14:38:46.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.464+0000 7f22769d1740 1 -- start start 2024-04-23T14:38:46.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.465+0000 7f2271000640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f225006c8f0 0x7f225006edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2260001530 tx=0x7f22600078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.465+0000 7f22769d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b456a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.465+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4599e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.465+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459a360 0x55f5b456f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.465+0000 7f22769d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f5b44d4b10 con 0x55f5b45b7980 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.465+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f5b447b2b0 con 0x55f5b459a360 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.465+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55f5b44d0ad0 con 0x55f5b459aaa0 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b456a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b456a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43300/0 (socket says 172.21.15.59:43300) 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 -- 172.21.15.59:0/2769153740 learned_addr learned my addr 172.21.15.59:0/2769153740 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f226be00640 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4599e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271a00640 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459a360 0x55f5b456f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 -- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 msgr2=0x55f5b4599e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4599e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 -- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459a360 msgr2=0x55f5b456f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459a360 0x55f5b456f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 -- 172.21.15.59:0/2769153740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b41e72d0 con 0x55f5b45b7980 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.466+0000 7f2271000640 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b456a310 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2260005dd0 tx=0x7f226000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f2e55a719557c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f2259600640 1 -- 172.21.15.59:0/2769153740 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260007560 con 0x55f5b45b7980 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f2259600640 1 -- 172.21.15.59:0/2769153740 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2260007700 con 0x55f5b45b7980 2024-04-23T14:38:46.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f2259600640 1 -- 172.21.15.59:0/2769153740 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260016b90 con 0x55f5b45b7980 2024-04-23T14:38:46.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f22769d1740 1 -- 172.21.15.59:0/2769153740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 msgr2=0x55f5b456a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f22769d1740 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b456a310 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2260005dd0 tx=0x7f226000f000 comp rx=0 tx=0).stop 2024-04-23T14:38:46.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f22769d1740 1 -- 172.21.15.59:0/2769153740 shutdown_connections 2024-04-23T14:38:46.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f22769d1740 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4599e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f22769d1740 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459a360 0x55f5b456f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f22769d1740 1 --2- 172.21.15.59:0/2769153740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b456a310 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.467+0000 7f22769d1740 1 -- 172.21.15.59:0/2769153740 >> 172.21.15.59:0/2769153740 conn(0x55f5b456c580 msgr2=0x55f5b45b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.468+0000 7f22769d1740 1 -- 172.21.15.59:0/2769153740 shutdown_connections 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.468+0000 7f22769d1740 1 -- 172.21.15.59:0/2769153740 wait complete. 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.468+0000 7f22769d1740 1 -- start start 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.468+0000 7f22769d1740 1 Processor -- start 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.469+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459aaa0 0x55f5b45959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.469+0000 7f22769d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b459b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.469+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459b620 0x55f5b459d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.469+0000 7f22769d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f5b456e730 con 0x55f5b45b7980 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.469+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f5b448e590 con 0x55f5b459aaa0 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.469+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55f5b456db20 con 0x55f5b459b620 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.469+0000 7f22769d1740 10 client.0 _ll_register_callbacks cb 0x55f5b44713a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b459b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b459b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43308/0 (socket says 172.21.15.59:43308) 2024-04-23T14:38:46.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 -- 172.21.15.59:0/878472493 learned_addr learned my addr 172.21.15.59:0/878472493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 -- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459b620 msgr2=0x55f5b459d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459b620 0x55f5b459d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 -- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459aaa0 msgr2=0x55f5b45959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459aaa0 0x55f5b45959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 -- 172.21.15.59:0/878472493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5b42b16b0 con 0x55f5b45b7980 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f226be00640 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b459b0e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f225c082270 tx=0x7f225c07a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bccd92c7c9709e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f224e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.470+0000 7f224e000640 1 -- 172.21.15.59:0/878472493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c0a8060 con 0x55f5b45b7980 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f224e000640 1 -- 172.21.15.59:0/878472493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f225c076230 con 0x55f5b45b7980 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2260007700 con 0x55f5b45b7980 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f224e000640 1 -- 172.21.15.59:0/878472493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f225c092070 con 0x55f5b45b7980 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 10 client.7683 fetch_fsmap learned FSMap version 60 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55f5b459c6f0 con 0x55f5b45b7980 2024-04-23T14:38:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f224e000640 1 -- 172.21.15.59:0/878472493 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2260007700 con 0x55f5b45b7980 2024-04-23T14:38:46.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f224e000640 1 -- 172.21.15.59:0/878472493 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f225c03a070 con 0x55f5b45b7980 2024-04-23T14:38:46.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 10 client.7683 fetch_fsmap finished waiting for FSMap version 60 2024-04-23T14:38:46.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 10 client.7683 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T14:38:46.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 20 client.7683 populate_metadata read hostname 'smithi059' 2024-04-23T14:38:46.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45ac370 0x55f5b45c6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 4 client.7683 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:38:46.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.471+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 --> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5b44e2530 con 0x55f5b45ac370 2024-04-23T14:38:46.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.472+0000 7f2271000640 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45ac370 0x55f5b45c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.473+0000 7f2271000640 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45ac370 0x55f5b45c6460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9447dcf922dbae9e server_cookie=43bd70cc857c8cb2 in_seq=0 out_seq=0 2024-04-23T14:38:46.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.473+0000 7f224e000640 10 client.7683 ms_handle_connect on v2:172.21.15.76:6833/3870021000 2024-04-23T14:38:46.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f224e000640 1 -- 172.21.15.59:0/878472493 <== mds.0 v2:172.21.15.76:6833/3870021000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5b44e2530 con 0x55f5b45ac370 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f224e000640 10 client.7683 handle_command_reply: tid=0 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 client.7683 shutdown 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 msgr2=0x55f5b459b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b459b0e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f225c082270 tx=0x7f225c07a400 comp rx=0 tx=0).stop 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 shutdown_connections 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45ac370 0x55f5b45c6460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459b620 0x55f5b459d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b45b7980 0x55f5b459b0e0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 --2- 172.21.15.59:0/878472493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b459aaa0 0x55f5b45959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f224e000640 0 client.7683 ms_handle_reset on v2:172.21.15.76:6833/3870021000 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 >> 172.21.15.59:0/878472493 conn(0x55f5b459a360 msgr2=0x55f5b45b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 shutdown_connections 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 1 -- 172.21.15.59:0/878472493 wait complete. 2024-04-23T14:38:46.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.475+0000 7f22769d1740 20 client.7683 trim_cache size 0 max 16384 2024-04-23T14:38:46.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.495+0000 7f22769d1740 1 Processor -- start 2024-04-23T14:38:46.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.495+0000 7f22769d1740 1 -- start start 2024-04-23T14:38:46.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.496+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4594b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.496+0000 7f22769d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459b620 0x55f5b462a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.496+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b45b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.496+0000 7f22769d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f226c1ffa90 con 0x55f5b459b620 2024-04-23T14:38:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.496+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f226c1fede0 con 0x55f5b45b7980 2024-04-23T14:38:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.496+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f226c1ff400 con 0x55f5b459aaa0 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459b620 0x55f5b462a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459b620 0x55f5b462a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43324/0 (socket says 172.21.15.59:43324) 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 -- 172.21.15.59:0/670481069 learned_addr learned my addr 172.21.15.59:0/670481069 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f2271a00640 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b45b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f2271000640 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4594b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 -- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 msgr2=0x55f5b4594b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4594b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 -- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 msgr2=0x55f5b45b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b45b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f226be00640 1 -- 172.21.15.59:0/670481069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b4570830 con 0x55f5b459b620 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.497+0000 7f2271a00640 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b45b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.498+0000 7f226be00640 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459b620 0x55f5b462a150 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f225c077e70 tx=0x7f225c07b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4dc8d95a44c3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.498+0000 7f2258c00640 1 -- 172.21.15.59:0/670481069 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c09f070 con 0x55f5b459b620 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.498+0000 7f2258c00640 1 -- 172.21.15.59:0/670481069 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225c0827a0 con 0x55f5b459b620 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.498+0000 7f2258c00640 1 -- 172.21.15.59:0/670481069 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c082360 con 0x55f5b459b620 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 -- 172.21.15.59:0/670481069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459b620 msgr2=0x55f5b462a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459b620 0x55f5b462a150 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f225c077e70 tx=0x7f225c07b000 comp rx=0 tx=0).stop 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 -- 172.21.15.59:0/670481069 shutdown_connections 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b45b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459b620 0x55f5b462a150 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 --2- 172.21.15.59:0/670481069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b459aaa0 0x55f5b4594b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 -- 172.21.15.59:0/670481069 >> 172.21.15.59:0/670481069 conn(0x55f5b459a360 msgr2=0x55f5b45690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 -- 172.21.15.59:0/670481069 shutdown_connections 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.499+0000 7f22769d1740 1 -- 172.21.15.59:0/670481069 wait complete. 2024-04-23T14:38:46.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.500+0000 7f22769d1740 1 -- start start 2024-04-23T14:38:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.500+0000 7f22769d1740 1 Processor -- start 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f22769d1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459aaa0 0x55f5b45958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b4628c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f22769d1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b45ca1e0 0x55f5b45b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f22769d1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f5b45ac1f0 con 0x55f5b459aaa0 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f5b45c6af0 con 0x55f5b45b7980 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f22769d1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55f5b45ac880 con 0x55f5b45ca1e0 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f22769d1740 10 client.0 _ll_register_callbacks cb 0x7f225c079520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:38:46.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f2271000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459aaa0 0x55f5b45958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f2271000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459aaa0 0x55f5b45958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43326/0 (socket says 172.21.15.59:43326) 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.501+0000 7f2271000640 1 -- 172.21.15.59:0/884355204 learned_addr learned my addr 172.21.15.59:0/884355204 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f2271000640 1 -- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b45ca1e0 msgr2=0x55f5b45b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f2271000640 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b45ca1e0 0x55f5b45b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f2271000640 1 -- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 msgr2=0x55f5b4628c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f2271000640 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b4628c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f2271000640 1 -- 172.21.15.59:0/884355204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5b4596870 con 0x55f5b459aaa0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f2271000640 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459aaa0 0x55f5b45958e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f2260016890 tx=0x7f2260001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa874b59a6c269 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f224e000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f224e000640 1 -- 172.21.15.59:0/884355204 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226002e060 con 0x55f5b459aaa0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f224e000640 1 -- 172.21.15.59:0/884355204 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f2260029dc0 con 0x55f5b459aaa0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f225c0827a0 con 0x55f5b459aaa0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f224e000640 1 -- 172.21.15.59:0/884355204 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2260034050 con 0x55f5b459aaa0 2024-04-23T14:38:46.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f22769d1740 10 client.7689 fetch_fsmap learned FSMap version 60 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.502+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55f5b45cbb60 con 0x55f5b459aaa0 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f224e000640 1 -- 172.21.15.59:0/884355204 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225c0827a0 con 0x55f5b459aaa0 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f224e000640 1 -- 172.21.15.59:0/884355204 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f2260037020 con 0x55f5b459aaa0 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f22769d1740 10 client.7689 fetch_fsmap finished waiting for FSMap version 60 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f22769d1740 10 client.7689 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f22769d1740 20 client.7689 populate_metadata read hostname 'smithi059' 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f22769d1740 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45c6eb0 0x55f5b45c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f22769d1740 4 client.7689 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.503+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 --> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] -- command(tid 0: {"prefix": "status"}) -- 0x55f5b44dc3a0 con 0x55f5b45c6eb0 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.504+0000 7f226be00640 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45c6eb0 0x55f5b45c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.504+0000 7f226be00640 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45c6eb0 0x55f5b45c7290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ce549a80f041ba server_cookie=fa2c25eee83016d6 in_seq=0 out_seq=0 2024-04-23T14:38:46.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.504+0000 7f224e000640 10 client.7689 ms_handle_connect on v2:172.21.15.76:6833/3870021000 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.505+0000 7f224e000640 1 -- 172.21.15.59:0/884355204 <== mds.0 v2:172.21.15.76:6833/3870021000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f5b44dc3a0 con 0x55f5b45c6eb0 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.505+0000 7f224e000640 10 client.7689 handle_command_reply: tid=0 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.505+0000 7f22769d1740 1 client.7689 shutdown 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.505+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459aaa0 msgr2=0x55f5b45958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.505+0000 7f22769d1740 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459aaa0 0x55f5b45958e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f2260016890 tx=0x7f2260001080 comp rx=0 tx=0).stop 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 shutdown_connections 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:6833/3870021000,v1:172.21.15.76:6835/3870021000] conn(0x55f5b45c6eb0 0x55f5b45c7290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f5b45ca1e0 0x55f5b45b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5b45b7980 0x55f5b4628c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 --2- 172.21.15.59:0/884355204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5b459aaa0 0x55f5b45958e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f224e000640 0 client.7689 ms_handle_reset on v2:172.21.15.76:6833/3870021000 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 >> 172.21.15.59:0/884355204 conn(0x55f5b459a360 msgr2=0x55f5b45aa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 shutdown_connections 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 1 -- 172.21.15.59:0/884355204 wait complete. 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f22769d1740 20 client.7689 trim_cache size 0 max 16384 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7545, 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:38:46.510 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:38:46.511 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7741445239999996, 2024-04-23T14:38:46.511 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 60, 2024-04-23T14:38:46.511 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 83, 2024-04-23T14:38:46.511 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 83, 2024-04-23T14:38:46.511 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.152389978 2024-04-23T14:38:46.511 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f225006c8f0 msgr2=0x7f225006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f2272400640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f225006c8f0 0x7f225006edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2260001530 tx=0x7f22600078f0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0d0b00 msgr2=0x7f226c1e9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.506+0000 7f2272400640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0d0b00 0x7f226c1e9cb0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f225c000c00 tx=0x7f225c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 shutdown_connections 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f225006c8f0 0x7f225006edb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f226c15c670 0x7f226c1ea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f226c0d0b00 0x7f226c1e9cb0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 --2- 172.21.15.59:0/2142327872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f226c0cd8b0 0x7f226c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 >> 172.21.15.59:0/2142327872 conn(0x7f226c06bdf0 msgr2=0x7f226c15cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 shutdown_connections 2024-04-23T14:38:46.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.507+0000 7f2272400640 1 -- 172.21.15.59:0/2142327872 wait complete. 2024-04-23T14:38:46.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:38:46.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.712+0000 7f4ba9200640 1 Processor -- start 2024-04-23T14:38:46.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.712+0000 7f4ba9200640 1 -- start start 2024-04-23T14:38:46.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.714+0000 7f4ba9200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.714+0000 7f4ba9200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ba40b4690 0x7f4ba40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.714+0000 7f4ba9200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ba4160210 con 0x7f4ba415c3d0 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.714+0000 7f4ba9200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4ba41604f0 con 0x7f4ba40b4690 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.714+0000 7f4ba9200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba41607d0 con 0x7f4ba40cd760 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.714+0000 7f4ba3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ba40b4690 0x7f4ba40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.714+0000 7f4ba3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ba40b4690 0x7f4ba40cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43330/0 (socket says 172.21.15.59:43330) 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.715+0000 7f4ba3e00640 1 -- 172.21.15.59:0/1002878190 learned_addr learned my addr 172.21.15.59:0/1002878190 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:46.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.715+0000 7f4ba2a00640 1 --2- 172.21.15.59:0/1002878190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.715+0000 7f4ba3e00640 1 -- 172.21.15.59:0/1002878190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 msgr2=0x7f4ba40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:46.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.715+0000 7f4ba3e00640 1 --2- 172.21.15.59:0/1002878190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.715+0000 7f4ba3e00640 1 -- 172.21.15.59:0/1002878190 >> v1:172.21.15.76:6789/0 conn(0x7f4ba415c3d0 legacy=0x7f4ba40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.715+0000 7f4ba3e00640 1 -- 172.21.15.59:0/1002878190 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba4160ab0 con 0x7f4ba40b4690 2024-04-23T14:38:46.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.716+0000 7f4ba3e00640 1 --2- 172.21.15.59:0/1002878190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba40b4690 0x7f4ba40cb7d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009e50 tx=0x7f4b9c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9d66503cde3581 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.716+0000 7f4ba2000640 1 -- 172.21.15.59:0/1002878190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c036070 con 0x7f4ba40b4690 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba2000640 1 -- 172.21.15.59:0/1002878190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9c031600 con 0x7f4ba40b4690 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba2000640 1 -- 172.21.15.59:0/1002878190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0319d0 con 0x7f4ba40b4690 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 -- 172.21.15.59:0/1002878190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba40b4690 msgr2=0x7f4ba40cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 --2- 172.21.15.59:0/1002878190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba40b4690 0x7f4ba40cb7d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009e50 tx=0x7f4b9c02a710 comp rx=0 tx=0).stop 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 -- 172.21.15.59:0/1002878190 shutdown_connections 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 --2- 172.21.15.59:0/1002878190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba40b4690 0x7f4ba40cb7d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 --2- 172.21.15.59:0/1002878190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 -- 172.21.15.59:0/1002878190 >> 172.21.15.59:0/1002878190 conn(0x7f4ba40b21a0 msgr2=0x7f4ba40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 -- 172.21.15.59:0/1002878190 shutdown_connections 2024-04-23T14:38:46.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.717+0000 7f4ba9200640 1 -- 172.21.15.59:0/1002878190 wait complete. 2024-04-23T14:38:46.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.719+0000 7f4ba9200640 1 Processor -- start 2024-04-23T14:38:46.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.719+0000 7f4ba9200640 1 -- start start 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba9200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba40b4690 0x7f4ba41e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba9200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba9200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba415c3d0 0x7f4ba40c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba9200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba415c7b0 con 0x7f4ba415c3d0 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba9200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba40b4ac0 con 0x7f4ba40b4690 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba9200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba415c930 con 0x7f4ba40cd760 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba415c3d0 0x7f4ba40c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba415c3d0 0x7f4ba40c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43338/0 (socket says 172.21.15.59:43338) 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.720+0000 7f4ba3e00640 1 -- 172.21.15.59:0/2933701270 learned_addr learned my addr 172.21.15.59:0/2933701270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba2a00640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3400640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba40b4690 0x7f4ba41e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3e00640 1 -- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 msgr2=0x7f4ba40c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3e00640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3e00640 1 -- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba40b4690 msgr2=0x7f4ba41e6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3e00640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba40b4690 0x7f4ba41e6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3e00640 1 -- 172.21.15.59:0/2933701270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba40ae1d0 con 0x7f4ba415c3d0 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3400640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba40b4690 0x7f4ba41e6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:46.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.721+0000 7f4ba3e00640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba415c3d0 0x7f4ba40c5920 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c034040 tx=0x7f4b9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.722+0000 7f4b9b400640 1 -- 172.21.15.59:0/2933701270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c036050 con 0x7f4ba415c3d0 2024-04-23T14:38:46.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.722+0000 7f4b9b400640 1 -- 172.21.15.59:0/2933701270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9c042070 con 0x7f4ba415c3d0 2024-04-23T14:38:46.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.722+0000 7f4b9b400640 1 -- 172.21.15.59:0/2933701270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c009510 con 0x7f4ba415c3d0 2024-04-23T14:38:46.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.722+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba40ae4b0 con 0x7f4ba415c3d0 2024-04-23T14:38:46.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.722+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba40aea70 con 0x7f4ba415c3d0 2024-04-23T14:38:46.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.723+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba404fd90 con 0x7f4ba415c3d0 2024-04-23T14:38:46.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.724+0000 7f4b9b400640 1 -- 172.21.15.59:0/2933701270 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b9c007bd0 con 0x7f4ba415c3d0 2024-04-23T14:38:46.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.725+0000 7f4b9b400640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b8406c920 0x7f4b8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:46.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.725+0000 7f4b9b400640 1 -- 172.21.15.59:0/2933701270 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f4b9c0ac9e0 con 0x7f4ba415c3d0 2024-04-23T14:38:46.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.725+0000 7f4ba3400640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b8406c920 0x7f4b8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:46.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.726+0000 7f4ba3400640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b8406c920 0x7f4b8406ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4ba41e85c0 tx=0x7f4b94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:46.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.730+0000 7f4b9b400640 1 -- 172.21.15.59:0/2933701270 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b9c045360 con 0x7f4ba415c3d0 2024-04-23T14:38:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.973+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ba406cbe0 con 0x7f4ba415c3d0 2024-04-23T14:38:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.974+0000 7f4b9b400640 1 -- 172.21.15.59:0/2933701270 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7f4b9c07ae00 con 0x7f4ba415c3d0 2024-04-23T14:38:46.976 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:38:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b8406c920 msgr2=0x7f4b8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b8406c920 0x7f4b8406ede0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4ba41e85c0 tx=0x7f4b94009040 comp rx=0 tx=0).stop 2024-04-23T14:38:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba415c3d0 msgr2=0x7f4ba40c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba415c3d0 0x7f4ba40c5920 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c034040 tx=0x7f4b9c004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 shutdown_connections 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4b8406c920 0x7f4b8406ede0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ba40cd760 0x7f4ba40c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ba415c3d0 0x7f4ba40c5920 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 --2- 172.21.15.59:0/2933701270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ba40b4690 0x7f4ba41e6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 >> 172.21.15.59:0/2933701270 conn(0x7f4ba40b21a0 msgr2=0x7f4ba4071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 shutdown_connections 2024-04-23T14:38:46.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:46.977+0000 7f4ba9200640 1 -- 172.21.15.59:0/2933701270 wait complete. 2024-04-23T14:38:46.995 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:38:46.995 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:38:46.995 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:38:46.996 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:38:46.996 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:46.996 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:38:46.996 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:38:46.996 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:38:46.996 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:38:46.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:46.996 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:38:47.047 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42022sec preferred_lft 42022sec 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:38:47.048 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:47.049 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:38:47.050 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:47.050 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:38:47.050 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:38:47.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:47.051 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:38:47.129 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:38:47.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:47.134 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:38:47.188 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:38:47.188 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:38:47.188 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:38:47.188 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:38:47.188 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:38:47.189 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:38:47.189 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:38:47.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:38:47.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:38:47.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:38:47.189 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:38:47.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:38:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:38:47.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:47.401 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:38:47.449 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:38:47.449 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:38:47.450 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:38:47.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:47.451 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:38:47.536 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:38:47.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:38:47.536 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:47.592 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:38:47.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:38:47.592 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:47.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:38:47.746 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:38:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:38:47.808 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:47.809 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:47.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:47.810 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:38:47.860 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:38:47.861 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:38:47.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:38:47.934 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:38:47.978 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:38:47.979 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:47.980 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:38:47.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:47.981 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:38:48.031 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:38:48.031+0000 7fa28d023480 -1 init, newargv = 0x55f45b9756a0 newargc=15 2024-04-23T14:38:48.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[63542]: starting ceph client 2024-04-23T14:38:48.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[63542]: starting fuse 2024-04-23T14:38:48.062 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:38:48.063 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:38:48.063 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:38:48.064 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:38:48.065 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:38:48.065 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:48.065 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:38:48.065 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:38:48.262 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.63542.asok 2024-04-23T14:38:48.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:38:48.279 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.63542.asok 2024-04-23T14:38:48.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:48.280 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.63542.asok status 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "63542", 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:38:48.516 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7701, 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7701 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2053298559 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:38:48.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2053298559 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7701 192.168.0.1:0/2053298559", 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2053298559", 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 60, 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 83, 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:38:48.518 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:38:48.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:48.532 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:38:48.532 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:48.588 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:38:48.588 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:48.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:48.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:48.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:48.679 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-23T14:38:48.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:48.747 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-23T14:38:48.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:48.809 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-23T14:38:48.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:48.874 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-23T14:38:48.949 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:38:48.949 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:38:48.949 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:38:48.949 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:38:48.950 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-23T14:38:48.950 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:38:49.100 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-23T14:38:49.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:38:49.114 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:38:49.114 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:49.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:49.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.435+0000 7ff429e00640 1 Processor -- start 2024-04-23T14:38:49.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.435+0000 7ff429e00640 1 -- start start 2024-04-23T14:38:49.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.436+0000 7ff429e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff42415b450 0x7ff4240b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.436+0000 7ff429e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4240b3870 con 0x7ff42406f3f0 2024-04-23T14:38:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.437+0000 7ff429e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4240b3b50 con 0x7ff4240cd760 2024-04-23T14:38:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.437+0000 7ff429e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff4240b41a0 con 0x7ff42415b450 2024-04-23T14:38:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.437+0000 7ff423e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff42406f3f0 0x7ff4240cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39196/0 (socket says 172.21.15.59:39196) 2024-04-23T14:38:49.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.437+0000 7ff423e00640 1 -- 172.21.15.59:0/1661785191 learned_addr learned my addr 172.21.15.59:0/1661785191 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.437+0000 7ff428a00640 1 --2- 172.21.15.59:0/1661785191 >> v2:172.21.15.76:3300/0 conn(0x7ff42415b450 0x7ff4240b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.437+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2100081897 0 0) 0x7ff4240b3870 con 0x7ff42406f3f0 2024-04-23T14:38:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.437+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff408003520 con 0x7ff42406f3f0 2024-04-23T14:38:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1729753798 0 0) 0x7ff408003520 con 0x7ff42406f3f0 2024-04-23T14:38:49.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 >> v2:172.21.15.76:3300/0 conn(0x7ff42415b450 msgr2=0x7ff4240b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 --2- 172.21.15.59:0/1661785191 >> v2:172.21.15.76:3300/0 conn(0x7ff42415b450 0x7ff4240b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 >> v1:172.21.15.76:6789/0 conn(0x7ff4240cd760 legacy=0x7ff4240ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4240b3e30 con 0x7ff42406f3f0 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7ff410001670 con 0x7ff42406f3f0 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 >> v1:172.21.15.59:6789/0 conn(0x7ff42406f3f0 legacy=0x7ff4240cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080039b0 0x7ff408003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff408004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080051c0 0x7ff40800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240b41a0 con 0x7ff408004800 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff40800e280 con 0x7ff4080051c0 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff40800e480 con 0x7ff4080039b0 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.438+0000 7ff423e00640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff408004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff423e00640 1 -- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080039b0 msgr2=0x7ff408003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:49.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff428a00640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080039b0 0x7ff408003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff429400640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080051c0 0x7ff40800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff423e00640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080039b0 0x7ff408003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff423e00640 1 -- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080051c0 msgr2=0x7ff40800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff423e00640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080051c0 0x7ff40800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff423e00640 1 -- 172.21.15.59:0/1661785191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff408003520 con 0x7ff408004800 2024-04-23T14:38:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff429400640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080051c0 0x7ff40800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.439+0000 7ff429400640 1 -- 172.21.15.59:0/1661785191 reap_dead start 2024-04-23T14:38:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.440+0000 7ff423e00640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff408004c80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c240 tx=0x7ff408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6208184a7affbfd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.440+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410007cf0 con 0x7ff408004800 2024-04-23T14:38:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.440+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4100058d0 con 0x7ff408004800 2024-04-23T14:38:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.440+0000 7ff423400640 1 -- 172.21.15.59:0/1661785191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410005d20 con 0x7ff408004800 2024-04-23T14:38:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.441+0000 7ff429e00640 1 -- 172.21.15.59:0/1661785191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 msgr2=0x7ff408004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.441+0000 7ff429e00640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff408004c80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c240 tx=0x7ff408005830 comp rx=0 tx=0).stop 2024-04-23T14:38:49.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.441+0000 7ff429e00640 1 -- 172.21.15.59:0/1661785191 shutdown_connections 2024-04-23T14:38:49.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.441+0000 7ff429e00640 1 --2- 172.21.15.59:0/1661785191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff408004c80 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.441+0000 7ff429e00640 1 -- 172.21.15.59:0/1661785191 >> 172.21.15.59:0/1661785191 conn(0x7ff4240b21a0 msgr2=0x7ff4240b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:49.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.441+0000 7ff429e00640 1 -- 172.21.15.59:0/1661785191 shutdown_connections 2024-04-23T14:38:49.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.441+0000 7ff429e00640 1 -- 172.21.15.59:0/1661785191 wait complete. 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.443+0000 7ff429e00640 1 Processor -- start 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.443+0000 7ff429e00640 1 -- start start 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff429e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff4240936f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff429e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff42409df30 0x7ff424158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff429e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4241e4470 0x7ff4240cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff429e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240b3080 con 0x7ff408004800 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff429e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240b4ff0 con 0x7ff4241e4470 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff429e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff4240b3200 con 0x7ff42409df30 2024-04-23T14:38:49.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff428a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff4240936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff428a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff4240936f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43350/0 (socket says 172.21.15.59:43350) 2024-04-23T14:38:49.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff428a00640 1 -- 172.21.15.59:0/232865853 learned_addr learned my addr 172.21.15.59:0/232865853 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff429400640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4241e4470 0x7ff4240cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.444+0000 7ff423e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff42409df30 0x7ff424158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff428a00640 1 -- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff42409df30 msgr2=0x7ff424158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff428a00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff42409df30 0x7ff424158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff428a00640 1 -- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4241e4470 msgr2=0x7ff4240cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff428a00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4241e4470 0x7ff4240cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff428a00640 1 -- 172.21.15.59:0/232865853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42415bad0 con 0x7ff408004800 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff429400640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4241e4470 0x7ff4240cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:49.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff428a00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff4240936f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff41400f400 tx=0x7ff41400f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff420c00640 1 -- 172.21.15.59:0/232865853 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414016620 con 0x7ff408004800 2024-04-23T14:38:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff420c00640 1 -- 172.21.15.59:0/232865853 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41401b070 con 0x7ff408004800 2024-04-23T14:38:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42415bdb0 con 0x7ff408004800 2024-04-23T14:38:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff420c00640 1 -- 172.21.15.59:0/232865853 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414016a20 con 0x7ff408004800 2024-04-23T14:38:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.445+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42415c370 con 0x7ff408004800 2024-04-23T14:38:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.447+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42404fd90 con 0x7ff408004800 2024-04-23T14:38:49.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.448+0000 7ff420c00640 1 -- 172.21.15.59:0/232865853 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff414005d10 con 0x7ff408004800 2024-04-23T14:38:49.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.448+0000 7ff420c00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff40406c8f0 0x7ff40406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.449+0000 7ff420c00640 1 -- 172.21.15.59:0/232865853 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7ff41408c9f0 con 0x7ff408004800 2024-04-23T14:38:49.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.449+0000 7ff423e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff40406c8f0 0x7ff40406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.450+0000 7ff423e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff40406c8f0 0x7ff40406edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff410033000 tx=0x7ff410009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.453+0000 7ff420c00640 1 -- 172.21.15.59:0/232865853 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff414025020 con 0x7ff408004800 2024-04-23T14:38:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.666+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4240d0ec0 con 0x7ff408004800 2024-04-23T14:38:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.669+0000 7ff420c00640 1 -- 172.21.15.59:0/232865853 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16600 (secure 0 0 0) 0x7ff41405ad80 con 0x7ff408004800 2024-04-23T14:38:49.671 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:49.671 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:38:40.747863+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.672+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff40406c8f0 msgr2=0x7ff40406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.672+0000 7ff429e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff40406c8f0 0x7ff40406edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff410033000 tx=0x7ff410009030 comp rx=0 tx=0).stop 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 msgr2=0x7ff4240936f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff4240936f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff41400f400 tx=0x7ff41400f8c0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 shutdown_connections 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff40406c8f0 0x7ff40406edb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff42409df30 0x7ff424158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4241e4470 0x7ff4240cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 --2- 172.21.15.59:0/232865853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408004800 0x7ff4240936f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 >> 172.21.15.59:0/232865853 conn(0x7ff4240b21a0 msgr2=0x7ff42406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:49.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 shutdown_connections 2024-04-23T14:38:49.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.673+0000 7ff429e00640 1 -- 172.21.15.59:0/232865853 wait complete. 2024-04-23T14:38:49.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:38:36.964433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:38:38.188587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:38:49.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:38:49.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.912+0000 7f2203e00640 1 Processor -- start 2024-04-23T14:38:49.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.912+0000 7f2203e00640 1 -- start start 2024-04-23T14:38:49.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2203e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21fc06f6f0 0x7f21fc0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2203e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21fc071d70 0x7f21fc0721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2203e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21fc0adc80 con 0x7f21fc15e790 2024-04-23T14:38:49.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2203e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f21fc0aef40 con 0x7f21fc071d70 2024-04-23T14:38:49.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2203e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f21fc0afd80 con 0x7f21fc06f6f0 2024-04-23T14:38:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2201400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f21fc15e790 0x7f21fc162e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39208/0 (socket says 172.21.15.59:39208) 2024-04-23T14:38:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2201400640 1 -- 172.21.15.59:0/3120788513 learned_addr learned my addr 172.21.15.59:0/3120788513 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.914+0000 7f2201e00640 1 --2- 172.21.15.59:0/3120788513 >> v2:172.21.15.59:3300/0 conn(0x7f21fc071d70 0x7f21fc0721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.915+0000 7f2200a00640 1 --2- 172.21.15.59:0/3120788513 >> v2:172.21.15.76:3300/0 conn(0x7f21fc06f6f0 0x7f21fc0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.915+0000 7f2201e00640 1 -- 172.21.15.59:0/3120788513 >> v2:172.21.15.76:3300/0 conn(0x7f21fc06f6f0 msgr2=0x7f21fc0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.915+0000 7f2201e00640 1 --2- 172.21.15.59:0/3120788513 >> v2:172.21.15.76:3300/0 conn(0x7f21fc06f6f0 0x7f21fc0aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.915+0000 7f2201e00640 1 -- 172.21.15.59:0/3120788513 >> v1:172.21.15.59:6789/0 conn(0x7f21fc15e790 legacy=0x7f21fc162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.915+0000 7f2201e00640 1 -- 172.21.15.59:0/3120788513 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc0adf20 con 0x7f21fc071d70 2024-04-23T14:38:49.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.915+0000 7f21fbe00640 1 -- 172.21.15.59:0/3120788513 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439178505 0 0) 0x7f21fc0adc80 con 0x7f21fc15e790 2024-04-23T14:38:49.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.916+0000 7f2201e00640 1 --2- 172.21.15.59:0/3120788513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc071d70 0x7f21fc0721a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0b5b20 tx=0x7f21f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40876f79f567406 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f21fbe00640 1 -- 172.21.15.59:0/3120788513 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4007740 con 0x7f21fc071d70 2024-04-23T14:38:49.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f21fbe00640 1 -- 172.21.15.59:0/3120788513 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f40078e0 con 0x7f21fc071d70 2024-04-23T14:38:49.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f21fbe00640 1 -- 172.21.15.59:0/3120788513 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4007cb0 con 0x7f21fc071d70 2024-04-23T14:38:49.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 -- 172.21.15.59:0/3120788513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc071d70 msgr2=0x7f21fc0721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 --2- 172.21.15.59:0/3120788513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc071d70 0x7f21fc0721a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0b5b20 tx=0x7f21f402a680 comp rx=0 tx=0).stop 2024-04-23T14:38:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 -- 172.21.15.59:0/3120788513 shutdown_connections 2024-04-23T14:38:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 --2- 172.21.15.59:0/3120788513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc071d70 0x7f21fc0721a0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 --2- 172.21.15.59:0/3120788513 >> v2:172.21.15.76:3300/0 conn(0x7f21fc06f6f0 0x7f21fc0aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 -- 172.21.15.59:0/3120788513 >> 172.21.15.59:0/3120788513 conn(0x7f21fc067d90 msgr2=0x7f21fc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 -- 172.21.15.59:0/3120788513 shutdown_connections 2024-04-23T14:38:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.917+0000 7f2203e00640 1 -- 172.21.15.59:0/3120788513 wait complete. 2024-04-23T14:38:49.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.919+0000 7f2203e00640 1 Processor -- start 2024-04-23T14:38:49.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.919+0000 7f2203e00640 1 -- start start 2024-04-23T14:38:49.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.920+0000 7f2203e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21fc06f6f0 0x7f21fc15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.920+0000 7f2203e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21fc071d70 0x7f21fc1e7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.920+0000 7f2203e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc15e790 0x7f21fc154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.920+0000 7f2203e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0ae4e0 con 0x7f21fc15e790 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.920+0000 7f2203e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0726e0 con 0x7f21fc06f6f0 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.920+0000 7f2203e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc06fbb0 con 0x7f21fc071d70 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.920+0000 7f2201e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc15e790 0x7f21fc154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc15e790 0x7f21fc154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43362/0 (socket says 172.21.15.59:43362) 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201e00640 1 -- 172.21.15.59:0/1146420857 learned_addr learned my addr 172.21.15.59:0/1146420857 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201400640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21fc06f6f0 0x7f21fc15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2200a00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21fc071d70 0x7f21fc1e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201e00640 1 -- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21fc071d70 msgr2=0x7f21fc1e7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21fc071d70 0x7f21fc1e7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201e00640 1 -- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21fc06f6f0 msgr2=0x7f21fc15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21fc06f6f0 0x7f21fc15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2201e00640 1 -- 172.21.15.59:0/1146420857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc15dd90 con 0x7f21fc15e790 2024-04-23T14:38:49.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.921+0000 7f2200a00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21fc071d70 0x7f21fc1e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:49.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.922+0000 7f2201e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc15e790 0x7f21fc154f20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009e80 tx=0x7f21f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.922+0000 7f21f9600640 1 -- 172.21.15.59:0/1146420857 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4040070 con 0x7f21fc15e790 2024-04-23T14:38:49.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.922+0000 7f21f9600640 1 -- 172.21.15.59:0/1146420857 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f4031c40 con 0x7f21fc15e790 2024-04-23T14:38:49.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.922+0000 7f21f9600640 1 -- 172.21.15.59:0/1146420857 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4009450 con 0x7f21fc15e790 2024-04-23T14:38:49.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.922+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc15eda0 con 0x7f21fc15e790 2024-04-23T14:38:49.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.922+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc15f360 con 0x7f21fc15e790 2024-04-23T14:38:49.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.923+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc072360 con 0x7f21fc15e790 2024-04-23T14:38:49.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.924+0000 7f21f9600640 1 -- 172.21.15.59:0/1146420857 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21f4039040 con 0x7f21fc15e790 2024-04-23T14:38:49.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.925+0000 7f21f9600640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21dc06c8f0 0x7f21dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.925+0000 7f21f9600640 1 -- 172.21.15.59:0/1146420857 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f21f40ace40 con 0x7f21fc15e790 2024-04-23T14:38:49.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.926+0000 7f2201400640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21dc06c8f0 0x7f21dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.926+0000 7f2201400640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21dc06c8f0 0x7f21dc06edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000f60 tx=0x7f21f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:49.930+0000 7f21f9600640 1 -- 172.21.15.59:0/1146420857 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21fc072360 con 0x7f21fc15e790 2024-04-23T14:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.134+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21fc162190 con 0x7f21fc15e790 2024-04-23T14:38:50.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.135+0000 7f21f9600640 1 -- 172.21.15.59:0/1146420857 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f21f407b1d0 con 0x7f21fc15e790 2024-04-23T14:38:50.136 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:38:50.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.137+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21dc06c8f0 msgr2=0x7f21dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.137+0000 7f2203e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21dc06c8f0 0x7f21dc06edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000f60 tx=0x7f21f0009040 comp rx=0 tx=0).stop 2024-04-23T14:38:50.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.137+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc15e790 msgr2=0x7f21fc154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.137+0000 7f2203e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc15e790 0x7f21fc154f20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009e80 tx=0x7f21f4004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:50.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 shutdown_connections 2024-04-23T14:38:50.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21dc06c8f0 0x7f21dc06edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21fc071d70 0x7f21fc1e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21fc15e790 0x7f21fc154f20 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 --2- 172.21.15.59:0/1146420857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21fc06f6f0 0x7f21fc15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 >> 172.21.15.59:0/1146420857 conn(0x7f21fc067d90 msgr2=0x7f21fc157f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 shutdown_connections 2024-04-23T14:38:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.138+0000 7f2203e00640 1 -- 172.21.15.59:0/1146420857 wait complete. 2024-04-23T14:38:50.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:38:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.375+0000 7f2a53200640 1 Processor -- start 2024-04-23T14:38:50.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.375+0000 7f2a53200640 1 -- start start 2024-04-23T14:38:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.376+0000 7f2a53200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2a541567e0 0x7f2a54156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.376+0000 7f2a53200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 0x7f2a540d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a53200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a540d0b40 0x7f2a54160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a53200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2a540d0f70 con 0x7f2a540d0b40 2024-04-23T14:38:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a53200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5415cab0 con 0x7f2a541567e0 2024-04-23T14:38:50.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a53200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2a54160e40 con 0x7f2a5415d810 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a52800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a540d0b40 0x7f2a54160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a52800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a540d0b40 0x7f2a54160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54846/0 (socket says 172.21.15.59:54846) 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a52800640 1 -- 172.21.15.59:0/1156549032 learned_addr learned my addr 172.21.15.59:0/1156549032 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a51e00640 1 --2- 172.21.15.59:0/1156549032 >> v2:172.21.15.76:3300/0 conn(0x7f2a541567e0 0x7f2a54156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a52800640 1 -- 172.21.15.59:0/1156549032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 msgr2=0x7f2a540d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.377+0000 7f2a49400640 1 --2- 172.21.15.59:0/1156549032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 0x7f2a540d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.378+0000 7f2a52800640 1 --2- 172.21.15.59:0/1156549032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 0x7f2a540d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.378+0000 7f2a52800640 1 -- 172.21.15.59:0/1156549032 >> v2:172.21.15.76:3300/0 conn(0x7f2a541567e0 msgr2=0x7f2a54156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.378+0000 7f2a52800640 1 --2- 172.21.15.59:0/1156549032 >> v2:172.21.15.76:3300/0 conn(0x7f2a541567e0 0x7f2a54156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.378+0000 7f2a52800640 1 -- 172.21.15.59:0/1156549032 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5415cd30 con 0x7f2a540d0b40 2024-04-23T14:38:50.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.378+0000 7f2a51e00640 1 --2- 172.21.15.59:0/1156549032 >> v2:172.21.15.76:3300/0 conn(0x7f2a541567e0 0x7f2a54156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.379+0000 7f2a52800640 1 --2- 172.21.15.59:0/1156549032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a54160110 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2a540b6360 tx=0x7f2a4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11407987fad40ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.379+0000 7f2a51400640 1 -- 172.21.15.59:0/1156549032 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c037070 con 0x7f2a540d0b40 2024-04-23T14:38:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.379+0000 7f2a51400640 1 -- 172.21.15.59:0/1156549032 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4c032440 con 0x7f2a540d0b40 2024-04-23T14:38:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.379+0000 7f2a51400640 1 -- 172.21.15.59:0/1156549032 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c032810 con 0x7f2a540d0b40 2024-04-23T14:38:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 -- 172.21.15.59:0/1156549032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 msgr2=0x7f2a54160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 --2- 172.21.15.59:0/1156549032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a54160110 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2a540b6360 tx=0x7f2a4c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 -- 172.21.15.59:0/1156549032 shutdown_connections 2024-04-23T14:38:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 --2- 172.21.15.59:0/1156549032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a54160110 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 --2- 172.21.15.59:0/1156549032 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 0x7f2a540d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 --2- 172.21.15.59:0/1156549032 >> v2:172.21.15.76:3300/0 conn(0x7f2a541567e0 0x7f2a54156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 -- 172.21.15.59:0/1156549032 >> 172.21.15.59:0/1156549032 conn(0x7f2a5406bdf0 msgr2=0x7f2a5406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 -- 172.21.15.59:0/1156549032 shutdown_connections 2024-04-23T14:38:50.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.380+0000 7f2a53200640 1 -- 172.21.15.59:0/1156549032 wait complete. 2024-04-23T14:38:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.382+0000 7f2a53200640 1 Processor -- start 2024-04-23T14:38:50.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.382+0000 7f2a53200640 1 -- start start 2024-04-23T14:38:50.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a53200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a5406e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a53200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a541567e0 0x7f2a541f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a53200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 0x7f2a541512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a53200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54160650 con 0x7f2a540d0b40 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a53200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5415d2f0 con 0x7f2a541567e0 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a53200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2a541607d0 con 0x7f2a5415d810 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a5406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a5406e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54852/0 (socket says 172.21.15.59:54852) 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 -- 172.21.15.59:0/3585173568 learned_addr learned my addr 172.21.15.59:0/3585173568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a49400640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a541567e0 0x7f2a541f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 -- 172.21.15.59:0/3585173568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 msgr2=0x7f2a541512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 0x7f2a541512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 -- 172.21.15.59:0/3585173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a541567e0 msgr2=0x7f2a541f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a541567e0 0x7f2a541f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.383+0000 7f2a51e00640 1 -- 172.21.15.59:0/3585173568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5406eeb0 con 0x7f2a540d0b40 2024-04-23T14:38:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.384+0000 7f2a51e00640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a5406e680 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400cf60 tx=0x7f2a4400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:50.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.384+0000 7f2a4aa00640 1 -- 172.21.15.59:0/3585173568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4401a070 con 0x7f2a540d0b40 2024-04-23T14:38:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.384+0000 7f2a4aa00640 1 -- 172.21.15.59:0/3585173568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4400d840 con 0x7f2a540d0b40 2024-04-23T14:38:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.384+0000 7f2a4aa00640 1 -- 172.21.15.59:0/3585173568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400dc10 con 0x7f2a540d0b40 2024-04-23T14:38:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.384+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5406f190 con 0x7f2a540d0b40 2024-04-23T14:38:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.384+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5406f750 con 0x7f2a540d0b40 2024-04-23T14:38:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.385+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5404fd90 con 0x7f2a540d0b40 2024-04-23T14:38:50.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.387+0000 7f2a4aa00640 1 -- 172.21.15.59:0/3585173568 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a4401d070 con 0x7f2a540d0b40 2024-04-23T14:38:50.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.387+0000 7f2a4aa00640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a3006c920 0x7f2a3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.388+0000 7f2a4aa00640 1 -- 172.21.15.59:0/3585173568 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f2a4408c3c0 con 0x7f2a540d0b40 2024-04-23T14:38:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.388+0000 7f2a49400640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a3006c920 0x7f2a3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.389+0000 7f2a49400640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a3006c920 0x7f2a3006ede0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ae3e0 tx=0x7f2a3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:50.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.392+0000 7f2a4aa00640 1 -- 172.21.15.59:0/3585173568 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a4405a100 con 0x7f2a540d0b40 2024-04-23T14:38:50.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:38:50.446 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:38:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.602+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2a540d0700 con 0x7f2a540d0b40 2024-04-23T14:38:50.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.603+0000 7f2a4aa00640 1 -- 172.21.15.59:0/3585173568 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f2a4405e400 con 0x7f2a540d0b40 2024-04-23T14:38:50.604 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:38:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a3006c920 msgr2=0x7f2a3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a3006c920 0x7f2a3006ede0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ae3e0 tx=0x7f2a3c008040 comp rx=0 tx=0).stop 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 msgr2=0x7f2a5406e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a5406e680 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400cf60 tx=0x7f2a4400cf90 comp rx=0 tx=0).stop 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 shutdown_connections 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a3006c920 0x7f2a3006ede0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a5415d810 0x7f2a541512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a541567e0 0x7f2a541f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 --2- 172.21.15.59:0/3585173568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a540d0b40 0x7f2a5406e680 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 >> 172.21.15.59:0/3585173568 conn(0x7f2a5406bdf0 msgr2=0x7f2a5415dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 shutdown_connections 2024-04-23T14:38:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.606+0000 7f2a53200640 1 -- 172.21.15.59:0/3585173568 wait complete. 2024-04-23T14:38:50.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:38:50.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.832+0000 7f5abfe00640 1 Processor -- start 2024-04-23T14:38:50.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.832+0000 7f5abfe00640 1 -- start start 2024-04-23T14:38:50.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.834+0000 7f5abfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ac006c0e0 0x7f5ac00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.834+0000 7f5abfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac00b2ca0 con 0x7f5ac00ce860 2024-04-23T14:38:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.834+0000 7f5abfe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac00cdaa0 con 0x7f5ac015c940 2024-04-23T14:38:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.834+0000 7f5abfe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac00cdd80 con 0x7f5ac006c0e0 2024-04-23T14:38:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.835+0000 7f5abe000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5ac00ce860 0x7f5ac006b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45052/0 (socket says 172.21.15.59:45052) 2024-04-23T14:38:50.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.835+0000 7f5abe000640 1 -- 172.21.15.59:0/2873876536 learned_addr learned my addr 172.21.15.59:0/2873876536 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:50.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.835+0000 7f5abf400640 1 --2- 172.21.15.59:0/2873876536 >> v2:172.21.15.76:3300/0 conn(0x7f5ac006c0e0 0x7f5ac00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.835+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393513494 0 0) 0x7f5ac00b2ca0 con 0x7f5ac00ce860 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.835+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5aa4003520 con 0x7f5ac00ce860 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2021911984 0 0) 0x7f5aa4003520 con 0x7f5ac00ce860 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 >> v2:172.21.15.76:3300/0 conn(0x7f5ac006c0e0 msgr2=0x7f5ac00b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 --2- 172.21.15.59:0/2873876536 >> v2:172.21.15.76:3300/0 conn(0x7f5ac006c0e0 0x7f5ac00b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 >> v1:172.21.15.76:6789/0 conn(0x7f5ac015c940 legacy=0x7f5ac00b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac00ce060 con 0x7f5ac00ce860 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f5aac001670 con 0x7f5ac00ce860 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 >> v1:172.21.15.59:6789/0 conn(0x7f5ac00ce860 legacy=0x7f5ac006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5aa40039b0 0x7f5aa4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5aa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa40051c0 0x7f5aa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00cdd80 con 0x7f5aa4004800 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa400c060 con 0x7f5aa40051c0 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa400c260 con 0x7f5aa40039b0 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.836+0000 7f5abe000640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5aa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.837+0000 7f5abe000640 1 -- 172.21.15.59:0/2873876536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5aa40039b0 msgr2=0x7f5aa4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.837+0000 7f5abe000640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5aa40039b0 0x7f5aa4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.837+0000 7f5abe000640 1 -- 172.21.15.59:0/2873876536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa40051c0 msgr2=0x7f5aa400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.837+0000 7f5abe000640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa40051c0 0x7f5aa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.837+0000 7f5abe000640 1 -- 172.21.15.59:0/2873876536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa4003520 con 0x7f5aa4004800 2024-04-23T14:38:50.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.837+0000 7f5abf400640 1 -- 172.21.15.59:0/2873876536 reap_dead start 2024-04-23T14:38:50.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.838+0000 7f5abe000640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5aa4004c80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c210 tx=0x7f5aa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d22db58625262ae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.838+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac007d30 con 0x7f5aa4004800 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.838+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aac005af0 con 0x7f5aa4004800 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.838+0000 7f5abd600640 1 -- 172.21.15.59:0/2873876536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac007070 con 0x7f5aa4004800 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.839+0000 7f5abfe00640 1 -- 172.21.15.59:0/2873876536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 msgr2=0x7f5aa4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.839+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5aa4004c80 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c210 tx=0x7f5aa4005830 comp rx=0 tx=0).stop 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.839+0000 7f5abfe00640 1 -- 172.21.15.59:0/2873876536 shutdown_connections 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.839+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2873876536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5aa4004c80 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.839+0000 7f5abfe00640 1 -- 172.21.15.59:0/2873876536 >> 172.21.15.59:0/2873876536 conn(0x7f5ac00b2f10 msgr2=0x7f5ac00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.839+0000 7f5abfe00640 1 -- 172.21.15.59:0/2873876536 shutdown_connections 2024-04-23T14:38:50.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.839+0000 7f5abfe00640 1 -- 172.21.15.59:0/2873876536 wait complete. 2024-04-23T14:38:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.841+0000 7f5abfe00640 1 Processor -- start 2024-04-23T14:38:50.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.842+0000 7f5abfe00640 1 -- start start 2024-04-23T14:38:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5ac015d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac00c73d0 0x7f5ac0153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac00c5d70 0x7f5ac015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac006b9d0 con 0x7f5aa4004800 2024-04-23T14:38:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abfe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac00c4ba0 con 0x7f5ac00c5d70 2024-04-23T14:38:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac006bb50 con 0x7f5ac00c73d0 2024-04-23T14:38:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5ac015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5ac015d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54870/0 (socket says 172.21.15.59:54870) 2024-04-23T14:38:50.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abea00640 1 -- 172.21.15.59:0/2188294047 learned_addr learned my addr 172.21.15.59:0/2188294047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.843+0000 7f5abea00640 1 -- 172.21.15.59:0/2188294047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac00c73d0 msgr2=0x7f5ac0153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5abea00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac00c73d0 0x7f5ac0153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5abea00640 1 -- 172.21.15.59:0/2188294047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac00c5d70 msgr2=0x7f5ac015f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5abea00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac00c5d70 0x7f5ac015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5abea00640 1 -- 172.21.15.59:0/2188294047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac015b930 con 0x7f5aa4004800 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5abea00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5ac015d730 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c3f0 tx=0x7f5ab000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5ab6a00640 1 -- 172.21.15.59:0/2188294047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0017070 con 0x7f5aa4004800 2024-04-23T14:38:50.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5ab6a00640 1 -- 172.21.15.59:0/2188294047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab00052d0 con 0x7f5aa4004800 2024-04-23T14:38:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5ab6a00640 1 -- 172.21.15.59:0/2188294047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0003810 con 0x7f5aa4004800 2024-04-23T14:38:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac00c6330 con 0x7f5aa4004800 2024-04-23T14:38:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.844+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac00c68f0 con 0x7f5aa4004800 2024-04-23T14:38:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.846+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac004fd90 con 0x7f5aa4004800 2024-04-23T14:38:50.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.847+0000 7f5ab6a00640 1 -- 172.21.15.59:0/2188294047 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ab001a070 con 0x7f5aa4004800 2024-04-23T14:38:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.847+0000 7f5ab6a00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.848+0000 7f5ab6a00640 1 -- 172.21.15.59:0/2188294047 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f5ab008adc0 con 0x7f5aa4004800 2024-04-23T14:38:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.848+0000 7f5abe000640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:50.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.849+0000 7f5abe000640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa006c8f0 0x7f5aa006edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5aac033000 tx=0x7f5aac007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:50.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:50.853+0000 7f5ab6a00640 1 -- 172.21.15.59:0/2188294047 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ab00591e0 con 0x7f5aa4004800 2024-04-23T14:38:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.068+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5ac00cd7e0 con 0x7f5aa4004800 2024-04-23T14:38:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.069+0000 7f5ab6a00640 1 -- 172.21.15.59:0/2188294047 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f5ab0058ae0 con 0x7f5aa4004800 2024-04-23T14:38:51.070 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:38:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.071+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa006c8f0 msgr2=0x7f5aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.071+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa006c8f0 0x7f5aa006edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5aac033000 tx=0x7f5aac007000 comp rx=0 tx=0).stop 2024-04-23T14:38:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.071+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 msgr2=0x7f5ac015d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.071+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5ac015d730 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c3f0 tx=0x7f5ab000c8b0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 shutdown_connections 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ac00c73d0 0x7f5ac0153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ac00c5d70 0x7f5ac015f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 --2- 172.21.15.59:0/2188294047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa4004800 0x7f5ac015d730 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 >> 172.21.15.59:0/2188294047 conn(0x7f5ac00b2f10 msgr2=0x7f5ac00cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 shutdown_connections 2024-04-23T14:38:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.072+0000 7f5abfe00640 1 -- 172.21.15.59:0/2188294047 wait complete. 2024-04-23T14:38:51.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:38:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.305+0000 7fe396400640 1 Processor -- start 2024-04-23T14:38:51.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.305+0000 7fe396400640 1 -- start start 2024-04-23T14:38:51.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.306+0000 7fe396400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3900cf410 0x7fe39015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.306+0000 7fe396400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe39006ddf0 0x7fe39006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.307+0000 7fe396400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39015dda0 con 0x7fe390072680 2024-04-23T14:38:51.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.307+0000 7fe396400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe3900cd880 con 0x7fe39006ddf0 2024-04-23T14:38:51.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.307+0000 7fe396400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe390071ec0 con 0x7fe3900cf410 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.307+0000 7fe38fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe390072680 0x7fe3900cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45068/0 (socket says 172.21.15.59:45068) 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.307+0000 7fe38ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe39006ddf0 0x7fe39006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.307+0000 7fe38fe00640 1 -- 172.21.15.59:0/2987116683 learned_addr learned my addr 172.21.15.59:0/2987116683 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.307+0000 7fe38f400640 1 --2- 172.21.15.59:0/2987116683 >> v2:172.21.15.76:3300/0 conn(0x7fe3900cf410 0x7fe39015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.308+0000 7fe38ea00640 1 -- 172.21.15.59:0/2987116683 >> v2:172.21.15.76:3300/0 conn(0x7fe3900cf410 msgr2=0x7fe39015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.308+0000 7fe38ea00640 1 --2- 172.21.15.59:0/2987116683 >> v2:172.21.15.76:3300/0 conn(0x7fe3900cf410 0x7fe39015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.308+0000 7fe38ea00640 1 -- 172.21.15.59:0/2987116683 >> v1:172.21.15.59:6789/0 conn(0x7fe390072680 legacy=0x7fe3900cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.308+0000 7fe38ea00640 1 -- 172.21.15.59:0/2987116683 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3900cdb20 con 0x7fe39006ddf0 2024-04-23T14:38:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.309+0000 7fe38ea00640 1 --2- 172.21.15.59:0/2987116683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39006ddf0 0x7fe39006e250 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe3900507b0 tx=0x7fe37c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f830016a4e9db900 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.309+0000 7fe38e000640 1 -- 172.21.15.59:0/2987116683 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c037070 con 0x7fe39006ddf0 2024-04-23T14:38:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.309+0000 7fe38e000640 1 -- 172.21.15.59:0/2987116683 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37c032440 con 0x7fe39006ddf0 2024-04-23T14:38:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.309+0000 7fe38e000640 1 -- 172.21.15.59:0/2987116683 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c02ae60 con 0x7fe39006ddf0 2024-04-23T14:38:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 -- 172.21.15.59:0/2987116683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39006ddf0 msgr2=0x7fe39006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 --2- 172.21.15.59:0/2987116683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39006ddf0 0x7fe39006e250 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe3900507b0 tx=0x7fe37c02a110 comp rx=0 tx=0).stop 2024-04-23T14:38:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 -- 172.21.15.59:0/2987116683 shutdown_connections 2024-04-23T14:38:51.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 --2- 172.21.15.59:0/2987116683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39006ddf0 0x7fe39006e250 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 --2- 172.21.15.59:0/2987116683 >> v2:172.21.15.76:3300/0 conn(0x7fe3900cf410 0x7fe39015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 -- 172.21.15.59:0/2987116683 >> 172.21.15.59:0/2987116683 conn(0x7fe3900b2130 msgr2=0x7fe3900b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 -- 172.21.15.59:0/2987116683 shutdown_connections 2024-04-23T14:38:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.310+0000 7fe396400640 1 -- 172.21.15.59:0/2987116683 wait complete. 2024-04-23T14:38:51.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.312+0000 7fe396400640 1 Processor -- start 2024-04-23T14:38:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.312+0000 7fe396400640 1 -- start start 2024-04-23T14:38:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe396400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39006ddf0 0x7fe39013ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe396400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe390072680 0x7fe3900d0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe396400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3900cf410 0x7fe3900d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe396400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe39015a6a0 con 0x7fe390072680 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe396400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe3900cde00 con 0x7fe3900cf410 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe396400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe39015a820 con 0x7fe39006ddf0 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe390072680 0x7fe3900d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe390072680 0x7fe3900d0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54888/0 (socket says 172.21.15.59:54888) 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 -- 172.21.15.59:0/2792462680 learned_addr learned my addr 172.21.15.59:0/2792462680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 -- 172.21.15.59:0/2792462680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39006ddf0 msgr2=0x7fe39013ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38f400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39006ddf0 0x7fe39013ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39006ddf0 0x7fe39013ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 -- 172.21.15.59:0/2792462680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3900cf410 msgr2=0x7fe3900d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3900cf410 0x7fe3900d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.313+0000 7fe38ea00640 1 -- 172.21.15.59:0/2792462680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3901f2930 con 0x7fe390072680 2024-04-23T14:38:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.314+0000 7fe38ea00640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe390072680 0x7fe3900d0840 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe37c02a150 tx=0x7fe37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.314+0000 7fe387400640 1 -- 172.21.15.59:0/2792462680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c037040 con 0x7fe390072680 2024-04-23T14:38:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.314+0000 7fe387400640 1 -- 172.21.15.59:0/2792462680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37c030770 con 0x7fe390072680 2024-04-23T14:38:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.314+0000 7fe387400640 1 -- 172.21.15.59:0/2792462680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c030b40 con 0x7fe390072680 2024-04-23T14:38:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.314+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3901f2c10 con 0x7fe390072680 2024-04-23T14:38:51.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.314+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3901f36f0 con 0x7fe390072680 2024-04-23T14:38:51.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.316+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3901f3920 con 0x7fe390072680 2024-04-23T14:38:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.317+0000 7fe387400640 1 -- 172.21.15.59:0/2792462680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe37c044020 con 0x7fe390072680 2024-04-23T14:38:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.318+0000 7fe387400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe37006c8f0 0x7fe37006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.318+0000 7fe387400640 1 -- 172.21.15.59:0/2792462680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe37c0ac140 con 0x7fe390072680 2024-04-23T14:38:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.318+0000 7fe38f400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe37006c8f0 0x7fe37006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.319+0000 7fe38f400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe37006c8f0 0x7fe37006edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe380009f30 tx=0x7fe38000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:51.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.323+0000 7fe387400640 1 -- 172.21.15.59:0/2792462680 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe37c032e20 con 0x7fe390072680 2024-04-23T14:38:51.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.543+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe390068a80 con 0x7fe390072680 2024-04-23T14:38:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.544+0000 7fe387400640 1 -- 172.21.15.59:0/2792462680 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) ==== 80+0+0 (secure 0 0 0) 0x7fe37c07a4d0 con 0x7fe390072680 2024-04-23T14:38:51.545 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:38:51.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe37006c8f0 msgr2=0x7fe37006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe37006c8f0 0x7fe37006edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe380009f30 tx=0x7fe38000b040 comp rx=0 tx=0).stop 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe390072680 msgr2=0x7fe3900d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe390072680 0x7fe3900d0840 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe37c02a150 tx=0x7fe37c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 shutdown_connections 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe37006c8f0 0x7fe37006edb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3900cf410 0x7fe3900d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe390072680 0x7fe3900d0840 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 --2- 172.21.15.59:0/2792462680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39006ddf0 0x7fe39013ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 >> 172.21.15.59:0/2792462680 conn(0x7fe3900b2130 msgr2=0x7fe39015df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 shutdown_connections 2024-04-23T14:38:51.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.547+0000 7fe396400640 1 -- 172.21.15.59:0/2792462680 wait complete. 2024-04-23T14:38:51.568 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:38:51.568 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:38:51.618 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:38:51.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.827+0000 7f69e3200640 1 Processor -- start 2024-04-23T14:38:51.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.827+0000 7f69e3200640 1 -- start start 2024-04-23T14:38:51.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.828+0000 7f69e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 0x7f69e40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.828+0000 7f69e3200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69e40b4690 0x7f69e40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.828+0000 7f69e3200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e4160210 con 0x7f69e415c3d0 2024-04-23T14:38:51.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.828+0000 7f69e3200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f69e41604f0 con 0x7f69e40b4690 2024-04-23T14:38:51.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.828+0000 7f69e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f69e41607d0 con 0x7f69e40cd760 2024-04-23T14:38:51.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.829+0000 7f69e1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f69e415c3d0 0x7f69e40cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45074/0 (socket says 172.21.15.59:45074) 2024-04-23T14:38:51.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.829+0000 7f69e1e00640 1 -- 172.21.15.59:0/2431138948 learned_addr learned my addr 172.21.15.59:0/2431138948 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:51.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.829+0000 7f69e2800640 1 --2- 172.21.15.59:0/2431138948 >> v2:172.21.15.76:3300/0 conn(0x7f69e40b4690 0x7f69e40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.829+0000 7f69d9400640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 0x7f69e40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.829+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2224813568 0 0) 0x7f69e4160210 con 0x7f69e415c3d0 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.829+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69c8003520 con 0x7f69e415c3d0 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 145170023 0 0) 0x7f69c8003520 con 0x7f69e415c3d0 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 msgr2=0x7f69e40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 0x7f69e40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 >> v2:172.21.15.76:3300/0 conn(0x7f69e40b4690 msgr2=0x7f69e40cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 --2- 172.21.15.59:0/2431138948 >> v2:172.21.15.76:3300/0 conn(0x7f69e40b4690 0x7f69e40cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e4160ab0 con 0x7f69e415c3d0 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f69d4001670 con 0x7f69e415c3d0 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 >> v1:172.21.15.59:6789/0 conn(0x7f69e415c3d0 legacy=0x7f69e40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69c8004800 0x7f69c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 0x7f69c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69e41604f0 con 0x7f69c80039b0 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f69e41607d0 con 0x7f69c8004800 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f69c800b9c0 con 0x7f69e40cd760 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.830+0000 7f69e1e00640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.831+0000 7f69d9400640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69c8004800 0x7f69c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.831+0000 7f69e2800640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 0x7f69c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.831+0000 7f69e1e00640 1 -- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 msgr2=0x7f69c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.831+0000 7f69e1e00640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cd760 0x7f69c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.831+0000 7f69e1e00640 1 -- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69c8004800 msgr2=0x7f69c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.831+0000 7f69e1e00640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69c8004800 0x7f69c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.831+0000 7f69e1e00640 1 -- 172.21.15.59:0/2431138948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c8003520 con 0x7f69c80039b0 2024-04-23T14:38:51.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.832+0000 7f69e1e00640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69c8003dd0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f69d400c190 tx=0x7f69c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cfd50474183a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4007c80 con 0x7f69c80039b0 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d4005af0 con 0x7f69c80039b0 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e1400640 1 -- 172.21.15.59:0/2431138948 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4007070 con 0x7f69c80039b0 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e3200640 1 -- 172.21.15.59:0/2431138948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 msgr2=0x7f69c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e3200640 1 --2- 172.21.15.59:0/2431138948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69c8003dd0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f69d400c190 tx=0x7f69c8005830 comp rx=0 tx=0).stop 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e2800640 1 -- 172.21.15.59:0/2431138948 reap_dead start 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e3200640 1 -- 172.21.15.59:0/2431138948 shutdown_connections 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e3200640 1 -- 172.21.15.59:0/2431138948 >> 172.21.15.59:0/2431138948 conn(0x7f69e40b21a0 msgr2=0x7f69e40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e3200640 1 -- 172.21.15.59:0/2431138948 shutdown_connections 2024-04-23T14:38:51.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.834+0000 7f69e3200640 1 -- 172.21.15.59:0/2431138948 wait complete. 2024-04-23T14:38:51.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.836+0000 7f69e3200640 1 Processor -- start 2024-04-23T14:38:51.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.836+0000 7f69e3200640 1 -- start start 2024-04-23T14:38:51.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.837+0000 7f69e3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69e40c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.837+0000 7f69e3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69e40c7280 0x7f69e40cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.837+0000 7f69e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cf350 0x7f69e4151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.837+0000 7f69e3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69e415c7b0 con 0x7f69c80039b0 2024-04-23T14:38:51.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.837+0000 7f69e3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40b4ac0 con 0x7f69e40c7280 2024-04-23T14:38:51.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.837+0000 7f69e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f69e415c930 con 0x7f69e40cf350 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69e40c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69e40c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54914/0 (socket says 172.21.15.59:54914) 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 -- 172.21.15.59:0/1129489692 learned_addr learned my addr 172.21.15.59:0/1129489692 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69d9400640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69e40c7280 0x7f69e40cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 -- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cf350 msgr2=0x7f69e4151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e2800640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cf350 0x7f69e4151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cf350 0x7f69e4151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 -- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69e40c7280 msgr2=0x7f69e40cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69e40c7280 0x7f69e40cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 -- 172.21.15.59:0/1129489692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40cdd70 con 0x7f69c80039b0 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69d9400640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69e40c7280 0x7f69e40cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:51.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.838+0000 7f69e1e00640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69e40c5dd0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69e41603e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.839+0000 7f69daa00640 1 -- 172.21.15.59:0/1129489692 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4041030 con 0x7f69c80039b0 2024-04-23T14:38:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.839+0000 7f69daa00640 1 -- 172.21.15.59:0/1129489692 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d4038040 con 0x7f69c80039b0 2024-04-23T14:38:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.839+0000 7f69daa00640 1 -- 172.21.15.59:0/1129489692 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4001650 con 0x7f69c80039b0 2024-04-23T14:38:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.839+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e40ce050 con 0x7f69c80039b0 2024-04-23T14:38:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.839+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e40ce610 con 0x7f69c80039b0 2024-04-23T14:38:51.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.840+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e404fd90 con 0x7f69c80039b0 2024-04-23T14:38:51.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.842+0000 7f69daa00640 1 -- 172.21.15.59:0/1129489692 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69d4044070 con 0x7f69c80039b0 2024-04-23T14:38:51.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.843+0000 7f69daa00640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f69c006c8f0 0x7f69c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:51.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.843+0000 7f69daa00640 1 -- 172.21.15.59:0/1129489692 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f69d40ab500 con 0x7f69c80039b0 2024-04-23T14:38:51.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.843+0000 7f69d9400640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f69c006c8f0 0x7f69c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:51.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.843+0000 7f69d9400640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f69c006c8f0 0x7f69c006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f69e40b1940 tx=0x7f69cc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:51.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:51.848+0000 7f69daa00640 1 -- 172.21.15.59:0/1129489692 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69d404b020 con 0x7f69c80039b0 2024-04-23T14:38:52.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.057+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69e406e400 con 0x7f69c80039b0 2024-04-23T14:38:52.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.061+0000 7f69daa00640 1 -- 172.21.15.59:0/1129489692 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16600 (secure 0 0 0) 0x7f69d4079920 con 0x7f69c80039b0 2024-04-23T14:38:52.062 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:38:52.063 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:38:40.747863+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:38:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f69c006c8f0 msgr2=0x7f69c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f69c006c8f0 0x7f69c006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f69e40b1940 tx=0x7f69cc00b040 comp rx=0 tx=0).stop 2024-04-23T14:38:52.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 msgr2=0x7f69e40c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69e40c5dd0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69e41603e0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 shutdown_connections 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f69c006c8f0 0x7f69c006edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69e40cf350 0x7f69e4151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69e40c7280 0x7f69e40cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 --2- 172.21.15.59:0/1129489692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69c80039b0 0x7f69e40c5dd0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 >> 172.21.15.59:0/1129489692 conn(0x7f69e40b21a0 msgr2=0x7f69e40c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 shutdown_connections 2024-04-23T14:38:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.064+0000 7f69e3200640 1 -- 172.21.15.59:0/1129489692 wait complete. 2024-04-23T14:38:52.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:38:36.964433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:38:38.188587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:38:52.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:38:52.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.301+0000 7f3e6f000640 1 Processor -- start 2024-04-23T14:38:52.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.301+0000 7f3e6f000640 1 -- start start 2024-04-23T14:38:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.303+0000 7f3e6f000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e6806f6f0 0x7f3e680aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.303+0000 7f3e6f000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e680adc80 con 0x7f3e6815e790 2024-04-23T14:38:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.303+0000 7f3e6f000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e680adf20 con 0x7f3e68071d70 2024-04-23T14:38:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.303+0000 7f3e6f000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3e680aef40 con 0x7f3e6806f6f0 2024-04-23T14:38:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.303+0000 7f3e67e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3e6815e790 0x7f3e68162e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45084/0 (socket says 172.21.15.59:45084) 2024-04-23T14:38:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.303+0000 7f3e67e00640 1 -- 172.21.15.59:0/1999408497 learned_addr learned my addr 172.21.15.59:0/1999408497 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.303+0000 7f3e5f400640 1 --2- 172.21.15.59:0/1999408497 >> v2:172.21.15.59:3300/0 conn(0x7f3e6806f6f0 0x7f3e680aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.304+0000 7f3e5f400640 1 -- 172.21.15.59:0/1999408497 >> v1:172.21.15.76:6789/0 conn(0x7f3e68071d70 legacy=0x7f3e680b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.304+0000 7f3e5f400640 1 -- 172.21.15.59:0/1999408497 >> v1:172.21.15.59:6789/0 conn(0x7f3e6815e790 legacy=0x7f3e68162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.304+0000 7f3e5f400640 1 -- 172.21.15.59:0/1999408497 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e680ae200 con 0x7f3e6806f6f0 2024-04-23T14:38:52.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.305+0000 7f3e5f400640 1 --2- 172.21.15.59:0/1999408497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e6806f6f0 0x7f3e680aea00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009d90 tx=0x7f3e5002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abce1f2123859ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:52.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.305+0000 7f3e67400640 1 -- 172.21.15.59:0/1999408497 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50007740 con 0x7f3e6806f6f0 2024-04-23T14:38:52.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.305+0000 7f3e67400640 1 -- 172.21.15.59:0/1999408497 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e500078e0 con 0x7f3e6806f6f0 2024-04-23T14:38:52.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.305+0000 7f3e67400640 1 -- 172.21.15.59:0/1999408497 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50007cb0 con 0x7f3e6806f6f0 2024-04-23T14:38:52.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.306+0000 7f3e6f000640 1 -- 172.21.15.59:0/1999408497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e6806f6f0 msgr2=0x7f3e680aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.306+0000 7f3e6f000640 1 --2- 172.21.15.59:0/1999408497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e6806f6f0 0x7f3e680aea00 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009d90 tx=0x7f3e5002a6a0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.306+0000 7f3e6f000640 1 -- 172.21.15.59:0/1999408497 shutdown_connections 2024-04-23T14:38:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.306+0000 7f3e6f000640 1 --2- 172.21.15.59:0/1999408497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e6806f6f0 0x7f3e680aea00 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.306+0000 7f3e6f000640 1 -- 172.21.15.59:0/1999408497 >> 172.21.15.59:0/1999408497 conn(0x7f3e68067d90 msgr2=0x7f3e6806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.306+0000 7f3e6f000640 1 -- 172.21.15.59:0/1999408497 shutdown_connections 2024-04-23T14:38:52.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.306+0000 7f3e6f000640 1 -- 172.21.15.59:0/1999408497 wait complete. 2024-04-23T14:38:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.308+0000 7f3e6f000640 1 Processor -- start 2024-04-23T14:38:52.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.308+0000 7f3e6f000640 1 -- start start 2024-04-23T14:38:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e6f000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e6806f6f0 0x7f3e68147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e6f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e68071d70 0x7f3e68159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e6f000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6815e790 0x7f3e6815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e6f000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6806fbb0 con 0x7f3e68071d70 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e6f000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6815eb70 con 0x7f3e6806f6f0 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e6f000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6815ecf0 con 0x7f3e6815e790 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e5f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e68071d70 0x7f3e68159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e5f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e68071d70 0x7f3e68159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54934/0 (socket says 172.21.15.59:54934) 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.309+0000 7f3e5f400640 1 -- 172.21.15.59:0/2188541191 learned_addr learned my addr 172.21.15.59:0/2188541191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e67e00640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e6806f6f0 0x7f3e68147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e6d000640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6815e790 0x7f3e6815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e5f400640 1 -- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6815e790 msgr2=0x7f3e6815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e5f400640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6815e790 0x7f3e6815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e5f400640 1 -- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e6806f6f0 msgr2=0x7f3e68147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e5f400640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e6806f6f0 0x7f3e68147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e5f400640 1 -- 172.21.15.59:0/2188541191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e680cf4b0 con 0x7f3e68071d70 2024-04-23T14:38:52.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e67e00640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e6806f6f0 0x7f3e68147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:52.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.310+0000 7f3e5f400640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e68071d70 0x7f3e68159cd0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009eb0 tx=0x7f3e50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:52.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.311+0000 7f3e64c00640 1 -- 172.21.15.59:0/2188541191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5003f070 con 0x7f3e68071d70 2024-04-23T14:38:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.311+0000 7f3e64c00640 1 -- 172.21.15.59:0/2188541191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e50031d00 con 0x7f3e68071d70 2024-04-23T14:38:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.311+0000 7f3e64c00640 1 -- 172.21.15.59:0/2188541191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50044660 con 0x7f3e68071d70 2024-04-23T14:38:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.311+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e68148860 con 0x7f3e68071d70 2024-04-23T14:38:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.311+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e68148e20 con 0x7f3e68071d70 2024-04-23T14:38:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.312+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e680cfb70 con 0x7f3e68071d70 2024-04-23T14:38:52.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.313+0000 7f3e64c00640 1 -- 172.21.15.59:0/2188541191 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e5003b410 con 0x7f3e68071d70 2024-04-23T14:38:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.314+0000 7f3e64c00640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4406c8f0 0x7f3e4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.314+0000 7f3e64c00640 1 -- 172.21.15.59:0/2188541191 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f3e500ad520 con 0x7f3e68071d70 2024-04-23T14:38:52.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.314+0000 7f3e67e00640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4406c8f0 0x7f3e4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.315+0000 7f3e67e00640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4406c8f0 0x7f3e4406edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3e58000f60 tx=0x7f3e58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:52.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.319+0000 7f3e64c00640 1 -- 172.21.15.59:0/2188541191 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e680cfb70 con 0x7f3e68071d70 2024-04-23T14:38:52.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.526+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e68068a80 con 0x7f3e68071d70 2024-04-23T14:38:52.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.527+0000 7f3e64c00640 1 -- 172.21.15.59:0/2188541191 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f3e5007b940 con 0x7f3e68071d70 2024-04-23T14:38:52.528 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:38:52.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4406c8f0 msgr2=0x7f3e4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4406c8f0 0x7f3e4406edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3e58000f60 tx=0x7f3e58009040 comp rx=0 tx=0).stop 2024-04-23T14:38:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e68071d70 msgr2=0x7f3e68159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e68071d70 0x7f3e68159cd0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009eb0 tx=0x7f3e50004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 shutdown_connections 2024-04-23T14:38:52.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4406c8f0 0x7f3e4406edb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6815e790 0x7f3e6815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e68071d70 0x7f3e68159cd0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 --2- 172.21.15.59:0/2188541191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e6806f6f0 0x7f3e68147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 >> 172.21.15.59:0/2188541191 conn(0x7f3e68067d90 msgr2=0x7f3e680b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 shutdown_connections 2024-04-23T14:38:52.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.530+0000 7f3e6f000640 1 -- 172.21.15.59:0/2188541191 wait complete. 2024-04-23T14:38:52.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:38:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.752+0000 7fabace00640 1 Processor -- start 2024-04-23T14:38:52.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.752+0000 7fabace00640 1 -- start start 2024-04-23T14:38:52.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7fabace00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faba8145e10 0x7faba806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7fabace00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba80d0cc0 0x7faba80cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7fabace00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faba8071d70 0x7faba8072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7fabace00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faba80cfc20 con 0x7faba8145e10 2024-04-23T14:38:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7fabace00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faba80cff00 con 0x7faba8071d70 2024-04-23T14:38:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7fabace00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faba80d01e0 con 0x7faba80d0cc0 2024-04-23T14:38:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7faba5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faba8145e10 0x7faba806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7faba5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faba8145e10 0x7faba806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54950/0 (socket says 172.21.15.59:54950) 2024-04-23T14:38:52.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.754+0000 7faba5e00640 1 -- 172.21.15.59:0/3554344838 learned_addr learned my addr 172.21.15.59:0/3554344838 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.755+0000 7faba5400640 1 --2- 172.21.15.59:0/3554344838 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba80d0cc0 0x7faba80cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.755+0000 7faba5e00640 1 -- 172.21.15.59:0/3554344838 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba80d0cc0 msgr2=0x7faba80cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.755+0000 7faba5e00640 1 --2- 172.21.15.59:0/3554344838 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba80d0cc0 0x7faba80cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.755+0000 7faba5e00640 1 -- 172.21.15.59:0/3554344838 >> v2:172.21.15.76:3300/0 conn(0x7faba8071d70 msgr2=0x7faba8072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.755+0000 7faba5e00640 1 --2- 172.21.15.59:0/3554344838 >> v2:172.21.15.76:3300/0 conn(0x7faba8071d70 0x7faba8072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.755+0000 7faba5e00640 1 -- 172.21.15.59:0/3554344838 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba80d04c0 con 0x7faba8145e10 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.755+0000 7faba5400640 1 --2- 172.21.15.59:0/3554344838 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba80d0cc0 0x7faba80cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.756+0000 7faba5e00640 1 --2- 172.21.15.59:0/3554344838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8145e10 0x7faba806f130 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fab90009870 tx=0x7fab9002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cadc07db3e79fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.756+0000 7faba4a00640 1 -- 172.21.15.59:0/3554344838 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90037070 con 0x7faba8145e10 2024-04-23T14:38:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.756+0000 7faba4a00640 1 -- 172.21.15.59:0/3554344838 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab90032440 con 0x7faba8145e10 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.756+0000 7faba4a00640 1 -- 172.21.15.59:0/3554344838 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90032810 con 0x7faba8145e10 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 -- 172.21.15.59:0/3554344838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8145e10 msgr2=0x7faba806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 --2- 172.21.15.59:0/3554344838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8145e10 0x7faba806f130 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fab90009870 tx=0x7fab9002a200 comp rx=0 tx=0).stop 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 -- 172.21.15.59:0/3554344838 shutdown_connections 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 --2- 172.21.15.59:0/3554344838 >> v2:172.21.15.76:3300/0 conn(0x7faba8071d70 0x7faba8072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 --2- 172.21.15.59:0/3554344838 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba80d0cc0 0x7faba80cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 --2- 172.21.15.59:0/3554344838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8145e10 0x7faba806f130 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 -- 172.21.15.59:0/3554344838 >> 172.21.15.59:0/3554344838 conn(0x7faba8067d90 msgr2=0x7faba806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 -- 172.21.15.59:0/3554344838 shutdown_connections 2024-04-23T14:38:52.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.757+0000 7fabace00640 1 -- 172.21.15.59:0/3554344838 wait complete. 2024-04-23T14:38:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.759+0000 7fabace00640 1 Processor -- start 2024-04-23T14:38:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.759+0000 7fabace00640 1 -- start start 2024-04-23T14:38:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7fabace00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8071d70 0x7faba813ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7fabace00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80d0cc0 0x7faba814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7fabace00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba8145e10 0x7faba814e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7fabace00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faba80cfa00 con 0x7faba8071d70 2024-04-23T14:38:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7fabace00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faba80cf880 con 0x7faba80d0cc0 2024-04-23T14:38:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7fabace00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faba81461f0 con 0x7faba8145e10 2024-04-23T14:38:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7faba5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8071d70 0x7faba813ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7faba5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8071d70 0x7faba813ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54964/0 (socket says 172.21.15.59:54964) 2024-04-23T14:38:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.760+0000 7faba5e00640 1 -- 172.21.15.59:0/2095109205 learned_addr learned my addr 172.21.15.59:0/2095109205 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5400640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80d0cc0 0x7faba814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba6800640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba8145e10 0x7faba814e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5e00640 1 -- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba8145e10 msgr2=0x7faba814e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5e00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba8145e10 0x7faba814e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5e00640 1 -- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80d0cc0 msgr2=0x7faba814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5e00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80d0cc0 0x7faba814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5e00640 1 -- 172.21.15.59:0/2095109205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba813f470 con 0x7faba8071d70 2024-04-23T14:38:52.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba6800640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba8145e10 0x7faba814e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5400640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80d0cc0 0x7faba814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.761+0000 7faba5e00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8071d70 0x7faba813ef30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fab9002a240 tx=0x7fab900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.762+0000 7fab9e000640 1 -- 172.21.15.59:0/2095109205 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90037040 con 0x7faba8071d70 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.762+0000 7fab9e000640 1 -- 172.21.15.59:0/2095109205 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab90042070 con 0x7faba8071d70 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.762+0000 7fab9e000640 1 -- 172.21.15.59:0/2095109205 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90030bd0 con 0x7faba8071d70 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.762+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba814ec50 con 0x7faba8071d70 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.762+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba814f210 con 0x7faba8071d70 2024-04-23T14:38:52.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.763+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba8068a80 con 0x7faba8071d70 2024-04-23T14:38:52.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.764+0000 7fab9e000640 1 -- 172.21.15.59:0/2095109205 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab9002ad10 con 0x7faba8071d70 2024-04-23T14:38:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.765+0000 7fab9e000640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fab8406c8f0 0x7fab8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.765+0000 7fab9e000640 1 -- 172.21.15.59:0/2095109205 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fab900ac590 con 0x7faba8071d70 2024-04-23T14:38:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.765+0000 7faba5400640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fab8406c8f0 0x7fab8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:52.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.766+0000 7faba5400640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fab8406c8f0 0x7fab8406edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faba80b1b60 tx=0x7fab98007550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:52.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.769+0000 7fab9e000640 1 -- 172.21.15.59:0/2095109205 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faba813f470 con 0x7faba8071d70 2024-04-23T14:38:52.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.975+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faba814e960 con 0x7faba8071d70 2024-04-23T14:38:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.976+0000 7fab9e000640 1 -- 172.21.15.59:0/2095109205 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7fab9007a890 con 0x7faba8071d70 2024-04-23T14:38:52.977 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:38:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fab8406c8f0 msgr2=0x7fab8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fab8406c8f0 0x7fab8406edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faba80b1b60 tx=0x7fab98007550 comp rx=0 tx=0).stop 2024-04-23T14:38:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8071d70 msgr2=0x7faba813ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8071d70 0x7faba813ef30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fab9002a240 tx=0x7fab900046c0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 shutdown_connections 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fab8406c8f0 0x7fab8406edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faba8145e10 0x7faba814e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80d0cc0 0x7faba814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 --2- 172.21.15.59:0/2095109205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8071d70 0x7faba813ef30 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 >> 172.21.15.59:0/2095109205 conn(0x7faba8067d90 msgr2=0x7faba815f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 shutdown_connections 2024-04-23T14:38:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:52.979+0000 7fabace00640 1 -- 172.21.15.59:0/2095109205 wait complete. 2024-04-23T14:38:53.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:38:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.212+0000 7f28e7e00640 1 Processor -- start 2024-04-23T14:38:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.212+0000 7f28e7e00640 1 -- start start 2024-04-23T14:38:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.214+0000 7f28e7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28e81567e0 0x7f28e8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.214+0000 7f28e7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e815cab0 con 0x7f28e80d0d10 2024-04-23T14:38:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.214+0000 7f28e7e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e8160e40 con 0x7f28e815d810 2024-04-23T14:38:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.214+0000 7f28e7e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f28e815cd30 con 0x7f28e81567e0 2024-04-23T14:38:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.214+0000 7f28e6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28e81567e0 0x7f28e8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.214+0000 7f28e7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f28e80d0d10 0x7f28e8160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45096/0 (socket says 172.21.15.59:45096) 2024-04-23T14:38:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.214+0000 7f28e7400640 1 -- 172.21.15.59:0/3898015067 learned_addr learned my addr 172.21.15.59:0/3898015067 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.215+0000 7f28e6a00640 1 -- 172.21.15.59:0/3898015067 >> v1:172.21.15.76:6789/0 conn(0x7f28e815d810 legacy=0x7f28e80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.215+0000 7f28e6a00640 1 -- 172.21.15.59:0/3898015067 >> v1:172.21.15.59:6789/0 conn(0x7f28e80d0d10 legacy=0x7f28e8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.215+0000 7f28e6a00640 1 -- 172.21.15.59:0/3898015067 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e815d010 con 0x7f28e81567e0 2024-04-23T14:38:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.215+0000 7f28e6000640 1 -- 172.21.15.59:0/3898015067 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3115153246 0 0) 0x7f28e0001490 con 0x7f28e80d0d10 2024-04-23T14:38:53.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.216+0000 7f28e6a00640 1 --2- 172.21.15.59:0/3898015067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8156bc0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ac10513218eee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:53.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.216+0000 7f28e6000640 1 -- 172.21.15.59:0/3898015067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8037070 con 0x7f28e81567e0 2024-04-23T14:38:53.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.216+0000 7f28e6000640 1 -- 172.21.15.59:0/3898015067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d8032440 con 0x7f28e81567e0 2024-04-23T14:38:53.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.216+0000 7f28e6000640 1 -- 172.21.15.59:0/3898015067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8032810 con 0x7f28e81567e0 2024-04-23T14:38:53.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.217+0000 7f28e7e00640 1 -- 172.21.15.59:0/3898015067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 msgr2=0x7f28e8156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.217+0000 7f28e7e00640 1 --2- 172.21.15.59:0/3898015067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8156bc0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802a1e0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.217+0000 7f28e7e00640 1 -- 172.21.15.59:0/3898015067 shutdown_connections 2024-04-23T14:38:53.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.217+0000 7f28e7e00640 1 --2- 172.21.15.59:0/3898015067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8156bc0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.217+0000 7f28e7e00640 1 -- 172.21.15.59:0/3898015067 >> 172.21.15.59:0/3898015067 conn(0x7f28e806bdf0 msgr2=0x7f28e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:53.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.217+0000 7f28e7e00640 1 -- 172.21.15.59:0/3898015067 shutdown_connections 2024-04-23T14:38:53.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.217+0000 7f28e7e00640 1 -- 172.21.15.59:0/3898015067 wait complete. 2024-04-23T14:38:53.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.218+0000 7f28e7e00640 1 Processor -- start 2024-04-23T14:38:53.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.219+0000 7f28e7e00640 1 -- start start 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28e80d0d10 0x7f28e81ea810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28e815d810 0x7f28e8099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81609a0 con 0x7f28e81567e0 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8160820 con 0x7f28e815d810 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f28e8160b20 con 0x7f28e80d0d10 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8099520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54984/0 (socket says 172.21.15.59:54984) 2024-04-23T14:38:53.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28dfe00640 1 -- 172.21.15.59:0/1666795419 learned_addr learned my addr 172.21.15.59:0/1666795419 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:53.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e6a00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28e80d0d10 0x7f28e81ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28e7400640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28e815d810 0x7f28e8099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.220+0000 7f28dfe00640 1 -- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28e80d0d10 msgr2=0x7f28e81ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dfe00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28e80d0d10 0x7f28e81ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dfe00640 1 -- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28e815d810 msgr2=0x7f28e8099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dfe00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28e815d810 0x7f28e8099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dfe00640 1 -- 172.21.15.59:0/1666795419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e8154a80 con 0x7f28e81567e0 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dfe00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8099520 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f28d000c3c0 tx=0x7f28d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dea00640 1 -- 172.21.15.59:0/1666795419 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0007020 con 0x7f28e81567e0 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dea00640 1 -- 172.21.15.59:0/1666795419 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d00071c0 con 0x7f28e81567e0 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28dea00640 1 -- 172.21.15.59:0/1666795419 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d00075b0 con 0x7f28e81567e0 2024-04-23T14:38:53.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e8154d60 con 0x7f28e81567e0 2024-04-23T14:38:53.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.221+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e8155320 con 0x7f28e81567e0 2024-04-23T14:38:53.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.223+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e804fd90 con 0x7f28e81567e0 2024-04-23T14:38:53.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.223+0000 7f28dea00640 1 -- 172.21.15.59:0/1666795419 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28d0019020 con 0x7f28e81567e0 2024-04-23T14:38:53.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.224+0000 7f28dea00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28c406c8f0 0x7f28c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.224+0000 7f28dea00640 1 -- 172.21.15.59:0/1666795419 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f28d008b350 con 0x7f28e81567e0 2024-04-23T14:38:53.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.225+0000 7f28e6a00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28c406c8f0 0x7f28c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.226+0000 7f28e6a00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28c406c8f0 0x7f28c406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f28d8034000 tx=0x7f28d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:53.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.229+0000 7f28dea00640 1 -- 172.21.15.59:0/1666795419 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28d00596e0 con 0x7f28e81567e0 2024-04-23T14:38:53.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.438+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28e806dd50 con 0x7f28e81567e0 2024-04-23T14:38:53.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.439+0000 7f28dea00640 1 -- 172.21.15.59:0/1666795419 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f28d0058fe0 con 0x7f28e81567e0 2024-04-23T14:38:53.440 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.441+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28c406c8f0 msgr2=0x7f28c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.441+0000 7f28e7e00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28c406c8f0 0x7f28c406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f28d8034000 tx=0x7f28d802e000 comp rx=0 tx=0).stop 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.441+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 msgr2=0x7f28e8099520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.441+0000 7f28e7e00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8099520 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f28d000c3c0 tx=0x7f28d000c880 comp rx=0 tx=0).stop 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 shutdown_connections 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28c406c8f0 0x7f28c406edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28e815d810 0x7f28e8099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28e81567e0 0x7f28e8099520 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 --2- 172.21.15.59:0/1666795419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f28e80d0d10 0x7f28e81ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 >> 172.21.15.59:0/1666795419 conn(0x7f28e806bdf0 msgr2=0x7f28e80af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:53.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 shutdown_connections 2024-04-23T14:38:53.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.442+0000 7f28e7e00640 1 -- 172.21.15.59:0/1666795419 wait complete. 2024-04-23T14:38:53.464 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:38:53.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.679+0000 7fc8d5400640 1 Processor -- start 2024-04-23T14:38:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.679+0000 7fc8d5400640 1 -- start start 2024-04-23T14:38:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.681+0000 7fc8d5400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00cd8b0 0x7fc8d00d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.681+0000 7fc8d5400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8d00d0b00 0x7fc8d006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.681+0000 7fc8d5400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d00b4cf0 con 0x7fc8d015c670 2024-04-23T14:38:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.681+0000 7fc8d5400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d015ca50 con 0x7fc8d00d0b00 2024-04-23T14:38:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.681+0000 7fc8d5400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d00ccb80 con 0x7fc8d00cd8b0 2024-04-23T14:38:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.682+0000 7fc8d4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8d00d0b00 0x7fc8d006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.682+0000 7fc8cfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc8d015c670 0x7fc8d00b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45100/0 (socket says 172.21.15.59:45100) 2024-04-23T14:38:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.682+0000 7fc8cfe00640 1 -- 172.21.15.59:0/515503054 learned_addr learned my addr 172.21.15.59:0/515503054 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:38:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.682+0000 7fc8c7e00640 1 --2- 172.21.15.59:0/515503054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00cd8b0 0x7fc8d00d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.682+0000 7fc8d4a00640 1 -- 172.21.15.59:0/515503054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00cd8b0 msgr2=0x7fc8d00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:38:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.682+0000 7fc8d4a00640 1 --2- 172.21.15.59:0/515503054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00cd8b0 0x7fc8d00d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.682+0000 7fc8d4a00640 1 -- 172.21.15.59:0/515503054 >> v1:172.21.15.59:6789/0 conn(0x7fc8d015c670 legacy=0x7fc8d00b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.683+0000 7fc8cf400640 1 -- 172.21.15.59:0/515503054 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150341520 0 0) 0x7fc8d00b4cf0 con 0x7fc8d015c670 2024-04-23T14:38:53.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.683+0000 7fc8d4a00640 1 -- 172.21.15.59:0/515503054 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d00ccdd0 con 0x7fc8d00d0b00 2024-04-23T14:38:53.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.684+0000 7fc8d4a00640 1 --2- 172.21.15.59:0/515503054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d00d0b00 0x7fc8d006ec50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009d90 tx=0x7fc8c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969f99cbb5d83f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:53.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.684+0000 7fc8cf400640 1 -- 172.21.15.59:0/515503054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8007740 con 0x7fc8d00d0b00 2024-04-23T14:38:53.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.684+0000 7fc8cf400640 1 -- 172.21.15.59:0/515503054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c80078e0 con 0x7fc8d00d0b00 2024-04-23T14:38:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.684+0000 7fc8cf400640 1 -- 172.21.15.59:0/515503054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8007cb0 con 0x7fc8d00d0b00 2024-04-23T14:38:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 -- 172.21.15.59:0/515503054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d00d0b00 msgr2=0x7fc8d006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 --2- 172.21.15.59:0/515503054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d00d0b00 0x7fc8d006ec50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009d90 tx=0x7fc8c802a660 comp rx=0 tx=0).stop 2024-04-23T14:38:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 -- 172.21.15.59:0/515503054 shutdown_connections 2024-04-23T14:38:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 --2- 172.21.15.59:0/515503054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d00d0b00 0x7fc8d006ec50 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 --2- 172.21.15.59:0/515503054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00cd8b0 0x7fc8d00d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 -- 172.21.15.59:0/515503054 >> 172.21.15.59:0/515503054 conn(0x7fc8d006bdf0 msgr2=0x7fc8d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 -- 172.21.15.59:0/515503054 shutdown_connections 2024-04-23T14:38:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.685+0000 7fc8d5400640 1 -- 172.21.15.59:0/515503054 wait complete. 2024-04-23T14:38:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.687+0000 7fc8d5400640 1 Processor -- start 2024-04-23T14:38:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.687+0000 7fc8d5400640 1 -- start start 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d5400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cd8b0 0x7fc8d014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d5400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00d0b00 0x7fc8d00b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d015c670 0x7fc8d015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d5400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00b3e60 con 0x7fc8d015c670 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d5400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00cd390 con 0x7fc8d00cd8b0 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d5400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d00b3fe0 con 0x7fc8d00d0b00 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d015c670 0x7fc8d015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d4a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d015c670 0x7fc8d015dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55004/0 (socket says 172.21.15.59:55004) 2024-04-23T14:38:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.688+0000 7fc8d4a00640 1 -- 172.21.15.59:0/3217735747 learned_addr learned my addr 172.21.15.59:0/3217735747 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8cfe00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cd8b0 0x7fc8d014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8c7e00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00d0b00 0x7fc8d00b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8d4a00640 1 -- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00d0b00 msgr2=0x7fc8d00b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8d4a00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00d0b00 0x7fc8d00b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8d4a00640 1 -- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cd8b0 msgr2=0x7fc8d014c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8d4a00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cd8b0 0x7fc8d014c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8d4a00640 1 -- 172.21.15.59:0/3217735747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d0158fc0 con 0x7fc8d015c670 2024-04-23T14:38:53.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8cfe00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cd8b0 0x7fc8d014c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:38:53.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.689+0000 7fc8d4a00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d015c670 0x7fc8d015dc30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009eb0 tx=0x7fc8c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:53.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.690+0000 7fc8ccc00640 1 -- 172.21.15.59:0/3217735747 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8040070 con 0x7fc8d015c670 2024-04-23T14:38:53.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.690+0000 7fc8ccc00640 1 -- 172.21.15.59:0/3217735747 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c8031c40 con 0x7fc8d015c670 2024-04-23T14:38:53.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.690+0000 7fc8ccc00640 1 -- 172.21.15.59:0/3217735747 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8009450 con 0x7fc8d015c670 2024-04-23T14:38:53.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.690+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d00afdb0 con 0x7fc8d015c670 2024-04-23T14:38:53.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.690+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d00b0050 con 0x7fc8d015c670 2024-04-23T14:38:53.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.691+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d004fd90 con 0x7fc8d015c670 2024-04-23T14:38:53.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.692+0000 7fc8ccc00640 1 -- 172.21.15.59:0/3217735747 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8c8039040 con 0x7fc8d015c670 2024-04-23T14:38:53.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.693+0000 7fc8ccc00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:53.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.693+0000 7fc8ccc00640 1 -- 172.21.15.59:0/3217735747 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fc8c80ac950 con 0x7fc8d015c670 2024-04-23T14:38:53.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.693+0000 7fc8cfe00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:53.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.694+0000 7fc8cfe00640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0001030 tx=0x7fc8c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.697+0000 7fc8ccc00640 1 -- 172.21.15.59:0/3217735747 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8c807ad70 con 0x7fc8d015c670 2024-04-23T14:38:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.914+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8d0073520 con 0x7fc8d015c670 2024-04-23T14:38:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.915+0000 7fc8ccc00640 1 -- 172.21.15.59:0/3217735747 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) ==== 80+0+0 (secure 0 0 0) 0x7fc8c807a670 con 0x7fc8d015c670 2024-04-23T14:38:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.917+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8ac06c8f0 msgr2=0x7fc8ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0001030 tx=0x7fc8c0009040 comp rx=0 tx=0).stop 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d015c670 msgr2=0x7fc8d015dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d015c670 0x7fc8d015dc30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009eb0 tx=0x7fc8c8004a40 comp rx=0 tx=0).stop 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 shutdown_connections 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8ac06c8f0 0x7fc8ac06edb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8d00d0b00 0x7fc8d00b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d015c670 0x7fc8d015dc30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 --2- 172.21.15.59:0/3217735747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cd8b0 0x7fc8d014c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 >> 172.21.15.59:0/3217735747 conn(0x7fc8d006bdf0 msgr2=0x7fc8d006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 shutdown_connections 2024-04-23T14:38:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:38:53.918+0000 7fc8d5400640 1 -- 172.21.15.59:0/3217735747 wait complete. 2024-04-23T14:38:53.936 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:38:53.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:38:53.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:38:54.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[63542]: fuse finished with error 0 and tester_r 0 2024-04-23T14:38:54.019 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:39:00.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:39:00.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:00.121 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:39:00.184 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:39:00.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.398+0000 7f3bb5600640 1 Processor -- start 2024-04-23T14:39:00.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.398+0000 7f3bb5600640 1 -- start start 2024-04-23T14:39:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.399+0000 7f3bb5600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 0x7f3bb00d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.399+0000 7f3bb5600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb00cf030 con 0x7f3bb00d0d90 2024-04-23T14:39:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.399+0000 7f3bb5600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb00b6a40 con 0x7f3bb00d2630 2024-04-23T14:39:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.399+0000 7f3bb5600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb00b7000 con 0x7f3bb015e7c0 2024-04-23T14:39:00.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.399+0000 7f3bb4c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3bb00d0d90 0x7f3bb00cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50862/0 (socket says 172.21.15.59:50862) 2024-04-23T14:39:00.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.399+0000 7f3bb4c00640 1 -- 172.21.15.59:0/609566516 learned_addr learned my addr 172.21.15.59:0/609566516 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3bafe00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 0x7f3bb00d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743203744 0 0) 0x7f3ba8001490 con 0x7f3bb00d0d90 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b94003520 con 0x7f3bb00d0d90 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 30301052 0 0) 0x7f3b94003520 con 0x7f3bb00d0d90 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 msgr2=0x7f3bb00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 0x7f3bb00d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 >> v1:172.21.15.76:6789/0 conn(0x7f3bb00d2630 legacy=0x7f3bb00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb00b6d20 con 0x7f3bb00d0d90 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f3ba8001bc0 con 0x7f3bb00d0d90 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 >> v1:172.21.15.59:6789/0 conn(0x7f3bb00d0d90 legacy=0x7f3bb00cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.400+0000 7f3baea00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3baea00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 0x7f3b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3baea00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b7000 con 0x7f3b940039e0 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9400ba70 con 0x7f3b94004d40 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9400bc70 con 0x7f3bb015e7c0 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bafe00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bb4c00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bafe00640 1 -- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 msgr2=0x7f3b94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3baf400640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 0x7f3b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bafe00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 0x7f3b94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bafe00640 1 -- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b94004d40 msgr2=0x7f3b940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bafe00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bafe00640 1 -- 172.21.15.59:0/609566516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b94003520 con 0x7f3b940039e0 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3baf400640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb015e7c0 0x7f3b94004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:00.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.401+0000 7f3bb4c00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:00.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.402+0000 7f3bafe00640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009a90 tx=0x7f3b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b1c05d90d21c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0036070 con 0x7f3b940039e0 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0031400 con 0x7f3b940039e0 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3baea00640 1 -- 172.21.15.59:0/609566516 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0009690 con 0x7f3b940039e0 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3bb5600640 1 -- 172.21.15.59:0/609566516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b940039e0 msgr2=0x7f3b94003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3bb5600640 1 --2- 172.21.15.59:0/609566516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009a90 tx=0x7f3b94005830 comp rx=0 tx=0).stop 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3bb4c00640 1 -- 172.21.15.59:0/609566516 reap_dead start 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3bb5600640 1 -- 172.21.15.59:0/609566516 shutdown_connections 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3bb5600640 1 -- 172.21.15.59:0/609566516 >> 172.21.15.59:0/609566516 conn(0x7f3bb009d920 msgr2=0x7f3bb008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.403+0000 7f3bb5600640 1 -- 172.21.15.59:0/609566516 shutdown_connections 2024-04-23T14:39:00.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.404+0000 7f3bb5600640 1 -- 172.21.15.59:0/609566516 wait complete. 2024-04-23T14:39:00.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.405+0000 7f3bb5600640 1 Processor -- start 2024-04-23T14:39:00.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.405+0000 7f3bb5600640 1 -- start start 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.406+0000 7f3bb5600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b940039e0 0x7f3bb0158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.406+0000 7f3bb5600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb014e140 0x7f3bb01f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.406+0000 7f3bb5600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bb015d3d0 0x7f3bb015d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.406+0000 7f3bb5600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b6700 con 0x7f3bb015d3d0 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.406+0000 7f3bb5600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00cf620 con 0x7f3b940039e0 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.406+0000 7f3bb5600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb00d1a80 con 0x7f3bb014e140 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bb015d3d0 0x7f3bb015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bb015d3d0 0x7f3bb015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45064/0 (socket says 172.21.15.59:45064) 2024-04-23T14:39:00.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 -- 172.21.15.59:0/2502743279 learned_addr learned my addr 172.21.15.59:0/2502743279 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bafe00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b940039e0 0x7f3bb0158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3baf400640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb014e140 0x7f3bb01f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 -- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb014e140 msgr2=0x7f3bb01f7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb014e140 0x7f3bb01f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 -- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b940039e0 msgr2=0x7f3bb0158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b940039e0 0x7f3bb0158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 -- 172.21.15.59:0/2502743279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb00ce410 con 0x7f3bb015d3d0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bafe00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b940039e0 0x7f3bb0158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.407+0000 7f3bb4c00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bb015d3d0 0x7f3bb015d850 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8000f80 tx=0x7f3ba800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.408+0000 7f3ba7e00640 1 -- 172.21.15.59:0/2502743279 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800bb80 con 0x7f3bb015d3d0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.408+0000 7f3ba7e00640 1 -- 172.21.15.59:0/2502743279 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba800bd20 con 0x7f3bb015d3d0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.408+0000 7f3ba7e00640 1 -- 172.21.15.59:0/2502743279 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba80072b0 con 0x7f3bb015d3d0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.408+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb014aeb0 con 0x7f3bb015d3d0 2024-04-23T14:39:00.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.408+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb00ce130 con 0x7f3bb015d3d0 2024-04-23T14:39:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.409+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb004fd90 con 0x7f3bb015d3d0 2024-04-23T14:39:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.410+0000 7f3ba7e00640 1 -- 172.21.15.59:0/2502743279 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ba801b020 con 0x7f3bb015d3d0 2024-04-23T14:39:00.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.411+0000 7f3ba7e00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b9006c920 0x7f3b9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.412+0000 7f3ba7e00640 1 -- 172.21.15.59:0/2502743279 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f3ba808be20 con 0x7f3bb015d3d0 2024-04-23T14:39:00.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.412+0000 7f3bafe00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b9006c920 0x7f3b9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.413+0000 7f3bafe00640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b9006c920 0x7f3b9006ede0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0033000 tx=0x7f3ba0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:00.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.417+0000 7f3ba7e00640 1 -- 172.21.15.59:0/2502743279 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3bb00cf030 con 0x7f3bb015d3d0 2024-04-23T14:39:00.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.665+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb00d1eb0 con 0x7f3bb015d3d0 2024-04-23T14:39:00.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.666+0000 7f3ba7e00640 1 -- 172.21.15.59:0/2502743279 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5183 (secure 0 0 0) 0x7f3ba805a1b0 con 0x7f3bb015d3d0 2024-04-23T14:39:00.666 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:39:00.667 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:38:42.286820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3870021000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3870021000},{"type":"v1","addr":"172.21.15.76:6835","nonce":3870021000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7545,"qdb_cluster":[7545]},"id":6}]} 2024-04-23T14:39:00.667 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 60 2024-04-23T14:39:00.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b9006c920 msgr2=0x7f3b9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b9006c920 0x7f3b9006ede0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0033000 tx=0x7f3ba0008030 comp rx=0 tx=0).stop 2024-04-23T14:39:00.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bb015d3d0 msgr2=0x7f3bb015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bb015d3d0 0x7f3bb015d850 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8000f80 tx=0x7f3ba800b4a0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 shutdown_connections 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3b9006c920 0x7f3b9006ede0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3bb014e140 0x7f3bb01f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bb015d3d0 0x7f3bb015d850 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 --2- 172.21.15.59:0/2502743279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b940039e0 0x7f3bb0158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 >> 172.21.15.59:0/2502743279 conn(0x7f3bb009d920 msgr2=0x7f3bb015f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 shutdown_connections 2024-04-23T14:39:00.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:00.669+0000 7f3bb5600640 1 -- 172.21.15.59:0/2502743279 wait complete. 2024-04-23T14:39:00.689 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-23T14:39:00.943 INFO:teuthology.orchestra.run.smithi076.stdout:{"message":"","return_code":0} 2024-04-23T14:39:00.944 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:39:00.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:39:01.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.157+0000 7ffbc7e00640 1 Processor -- start 2024-04-23T14:39:01.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.157+0000 7ffbc7e00640 1 -- start start 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.158+0000 7ffbc7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 0x7ffbc815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.158+0000 7ffbc7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc80b3720 con 0x7ffbc80af890 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.158+0000 7ffbc7e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc8144ed0 con 0x7ffbc8144a20 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.158+0000 7ffbc7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc815f3f0 con 0x7ffbc80af4b0 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.158+0000 7ffbc6000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffbc80af890 0x7ffbc80b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50874/0 (socket says 172.21.15.59:50874) 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc6000640 1 -- 172.21.15.59:0/3390981802 learned_addr learned my addr 172.21.15.59:0/3390981802 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc6a00640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 0x7ffbc815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006481208 0 0) 0x7ffbc80b3720 con 0x7ffbc80af890 2024-04-23T14:39:01.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbac003520 con 0x7ffbc80af890 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3484034661 0 0) 0x7ffbac003520 con 0x7ffbc80af890 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 msgr2=0x7ffbc815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 0x7ffbc815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 >> v1:172.21.15.76:6789/0 conn(0x7ffbc8144a20 legacy=0x7ffbc80d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc815f6d0 con 0x7ffbc80af890 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.159+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7ffbb4001670 con 0x7ffbc80af890 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 >> v1:172.21.15.59:6789/0 conn(0x7ffbc80af890 legacy=0x7ffbc80b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc5600640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 0x7ffbac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc5600640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac003f60 0x7ffbac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc5600640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac004d40 0x7ffbac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc815f3f0 con 0x7ffbac003f60 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac00baa0 con 0x7ffbac004d40 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffbac00bca0 con 0x7ffbc80af4b0 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc6000640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac003f60 0x7ffbac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc6a00640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 0x7ffbac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc6000640 1 -- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 msgr2=0x7ffbac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc6000640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbc80af4b0 0x7ffbac0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc6000640 1 -- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac004d40 msgr2=0x7ffbac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.160+0000 7ffbc7400640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac004d40 0x7ffbac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.161+0000 7ffbc6000640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac004d40 0x7ffbac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.161+0000 7ffbc6000640 1 -- 172.21.15.59:0/3390981802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac003520 con 0x7ffbac003f60 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.161+0000 7ffbc7400640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbac004d40 0x7ffbac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.162+0000 7ffbc6000640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac003f60 0x7ffbac004800 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c210 tx=0x7ffbac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d98162d61521ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.162+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4007d30 con 0x7ffbac003f60 2024-04-23T14:39:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.162+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4005af0 con 0x7ffbac003f60 2024-04-23T14:39:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.162+0000 7ffbc5600640 1 -- 172.21.15.59:0/3390981802 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4007070 con 0x7ffbac003f60 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.163+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3390981802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac003f60 msgr2=0x7ffbac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.163+0000 7ffbc7e00640 1 --2- 172.21.15.59:0/3390981802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac003f60 0x7ffbac004800 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c210 tx=0x7ffbac005830 comp rx=0 tx=0).stop 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.163+0000 7ffbc7400640 1 -- 172.21.15.59:0/3390981802 reap_dead start 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.163+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3390981802 shutdown_connections 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.163+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3390981802 >> 172.21.15.59:0/3390981802 conn(0x7ffbc806bdf0 msgr2=0x7ffbc806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.163+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3390981802 shutdown_connections 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.163+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3390981802 wait complete. 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.165+0000 7ffbc7e00640 1 Processor -- start 2024-04-23T14:39:01.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.165+0000 7ffbc7e00640 1 -- start start 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbac003f60 0x7ffbc815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc8156ba0 0x7ffbc815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc81e7540 0x7ffbc81e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc80d0cd0 con 0x7ffbc81e7540 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc815fc90 con 0x7ffbc8156ba0 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc80d0e50 con 0x7ffbac003f60 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc81e7540 0x7ffbc81e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc81e7540 0x7ffbc81e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45086/0 (socket says 172.21.15.59:45086) 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc7400640 1 -- 172.21.15.59:0/3797754220 learned_addr learned my addr 172.21.15.59:0/3797754220 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc6000640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc8156ba0 0x7ffbc815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.166+0000 7ffbc6a00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbac003f60 0x7ffbc815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7400640 1 -- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbac003f60 msgr2=0x7ffbc815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7400640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbac003f60 0x7ffbc815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7400640 1 -- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc8156ba0 msgr2=0x7ffbc815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7400640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc8156ba0 0x7ffbc815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7400640 1 -- 172.21.15.59:0/3797754220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc81597b0 con 0x7ffbc81e7540 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc6000640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc8156ba0 0x7ffbc815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc6a00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbac003f60 0x7ffbc815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7400640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc81e7540 0x7ffbc81e79c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8144e00 tx=0x7ffbc000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbbea00640 1 -- 172.21.15.59:0/3797754220 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0006d50 con 0x7ffbc81e7540 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbbea00640 1 -- 172.21.15.59:0/3797754220 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc0003970 con 0x7ffbc81e7540 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc8142990 con 0x7ffbc81e7540 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbbea00640 1 -- 172.21.15.59:0/3797754220 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0003d60 con 0x7ffbc81e7540 2024-04-23T14:39:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.167+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc8142f50 con 0x7ffbc81e7540 2024-04-23T14:39:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.168+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc804fd90 con 0x7ffbc81e7540 2024-04-23T14:39:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.170+0000 7ffbbea00640 1 -- 172.21.15.59:0/3797754220 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbc001b020 con 0x7ffbc81e7540 2024-04-23T14:39:01.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.170+0000 7ffbbea00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffba806c920 0x7ffba806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.171+0000 7ffbbea00640 1 -- 172.21.15.59:0/3797754220 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7ffbc008b8c0 con 0x7ffbc81e7540 2024-04-23T14:39:01.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.171+0000 7ffbc6a00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffba806c920 0x7ffba806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.172+0000 7ffbc6a00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffba806c920 0x7ffba806ede0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80099e0 tx=0x7ffbb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.175+0000 7ffbbea00640 1 -- 172.21.15.59:0/3797754220 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbc0059bc0 con 0x7ffbc81e7540 2024-04-23T14:39:01.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.425+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffbc80cf570 con 0x7ffbc81e7540 2024-04-23T14:39:01.426 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:39:01.427+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:39:01.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.431+0000 7ffbbea00640 1 -- 172.21.15.59:0/3797754220 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) ==== 161+0+0 (secure 0 0 0) 0x7ffbc00594c0 con 0x7ffbc81e7540 2024-04-23T14:39:01.431 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:39:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.435+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffba806c920 msgr2=0x7ffba806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.435+0000 7ffbc7e00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffba806c920 0x7ffba806ede0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80099e0 tx=0x7ffbb8008040 comp rx=0 tx=0).stop 2024-04-23T14:39:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.435+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc81e7540 msgr2=0x7ffbc81e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.435+0000 7ffbc7e00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc81e7540 0x7ffbc81e79c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8144e00 tx=0x7ffbc000dc00 comp rx=0 tx=0).stop 2024-04-23T14:39:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 shutdown_connections 2024-04-23T14:39:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffba806c920 0x7ffba806ede0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc81e7540 0x7ffbc81e79c0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc8156ba0 0x7ffbc815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 --2- 172.21.15.59:0/3797754220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffbac003f60 0x7ffbc815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 >> 172.21.15.59:0/3797754220 conn(0x7ffbc806bdf0 msgr2=0x7ffbc80ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 shutdown_connections 2024-04-23T14:39:01.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.436+0000 7ffbc7e00640 1 -- 172.21.15.59:0/3797754220 wait complete. 2024-04-23T14:39:01.456 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-23T14:39:01.462 INFO:tasks.ceph.mds.b.smithi076.stderr: -4058> 2024-04-23T14:38:45.693+0000 7f8755a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:39:01.557 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:39:01.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.563+0000 7fd2d9b1ff40 1 Processor -- start 2024-04-23T14:39:01.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.563+0000 7fd2d9b1ff40 1 -- start start 2024-04-23T14:39:01.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d9b1ff40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb6fd1d0 0x55a7fb7d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d9b1ff40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d9b1ff40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb7d7ad0 0x55a7fb7d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d9b1ff40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb6ff9c0 con 0x55a7fb7bcac0 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d9b1ff40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb6a6cc0 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d9b1ff40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a7fb6fdb20 con 0x55a7fb7d7ad0 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2ce400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2ce400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45102/0 (socket says 172.21.15.59:45102) 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2ce400640 1 -- 172.21.15.59:0/1123100134 learned_addr learned my addr 172.21.15.59:0/1123100134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d6e00640 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb6fd1d0 0x55a7fb7d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.564+0000 7fd2d7800640 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb7d7ad0 0x55a7fb7d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2ce400640 1 -- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb7d7ad0 msgr2=0x55a7fb7d84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2ce400640 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb7d7ad0 0x55a7fb7d84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2ce400640 1 -- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb6fd1d0 msgr2=0x55a7fb7d9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2ce400640 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb6fd1d0 0x55a7fb7d9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2ce400640 1 -- 172.21.15.59:0/1123100134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fb6e9ce0 con 0x55a7fb7bcac0 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2ce400640 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d7f70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000a4f0 tx=0x7fd2c004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddc345770709a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2d5a00640 1 -- 172.21.15.59:0/1123100134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0065070 con 0x55a7fb7bcac0 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2d5a00640 1 -- 172.21.15.59:0/1123100134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0054a10 con 0x55a7fb7bcac0 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.565+0000 7fd2d5a00640 1 -- 172.21.15.59:0/1123100134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0059040 con 0x55a7fb7bcac0 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/1123100134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7bcac0 msgr2=0x55a7fb7d7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d7f70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000a4f0 tx=0x7fd2c004e000 comp rx=0 tx=0).stop 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/1123100134 shutdown_connections 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb7d7ad0 0x55a7fb7d84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d7f70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/1123100134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb6fd1d0 0x55a7fb7d9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/1123100134 >> 172.21.15.59:0/1123100134 conn(0x55a7fb749cc0 msgr2=0x55a7fb748990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/1123100134 shutdown_connections 2024-04-23T14:39:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.566+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/1123100134 wait complete. 2024-04-23T14:39:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.567+0000 7fd2d9b1ff40 1 Processor -- start 2024-04-23T14:39:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.567+0000 7fd2d9b1ff40 1 -- start start 2024-04-23T14:39:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d9b1ff40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb6fd1d0 0x55a7fb7bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d9b1ff40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d9b1ff40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7d7ad0 0x55a7fb7d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d9b1ff40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb7d4340 con 0x55a7fb7d7ad0 2024-04-23T14:39:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d9b1ff40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a7fb7daef0 con 0x55a7fb7bcac0 2024-04-23T14:39:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d9b1ff40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a7fb7d4060 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d6e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb6fd1d0 0x55a7fb7bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d7800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7d7ad0 0x55a7fb7d9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d6e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb6fd1d0 0x55a7fb7bddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:57618/0 (socket says 172.21.15.59:57618) 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2d6e00640 1 -- 172.21.15.59:0/589001420 learned_addr learned my addr 172.21.15.59:0/589001420 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.568+0000 7fd2ce400640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2d6e00640 1 -- 172.21.15.59:0/589001420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb7bcac0 msgr2=0x55a7fb7d9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2d6e00640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2d6e00640 1 -- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7d7ad0 msgr2=0x55a7fb7d9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2d6e00640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7d7ad0 0x55a7fb7d9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2d6e00640 1 -- 172.21.15.59:0/589001420 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fb74a140 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2d7800640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a7fb7d7ad0 0x55a7fb7d9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2ce400640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a7fb7bcac0 0x55a7fb7d9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:39:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.569+0000 7fd2d6e00640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb6fd1d0 0x55a7fb7bddc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8003e70 tx=0x7fd2c8001a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.570+0000 7fd2cfe00640 1 -- 172.21.15.59:0/589001420 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80020b0 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.570+0000 7fd2cfe00640 1 -- 172.21.15.59:0/589001420 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c8017070 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.570+0000 7fd2cfe00640 1 -- 172.21.15.59:0/589001420 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80117a0 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.570+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a7fb7bd2e0 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.570+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fb7bd8a0 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.572+0000 7fd2cfe00640 1 -- 172.21.15.59:0/589001420 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2c8011940 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.573+0000 7fd2cfe00640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406b2d0 0x7fd2b406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.573+0000 7fd2cfe00640 1 -- 172.21.15.59:0/589001420 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fd2c808a730 con 0x55a7fb6fd1d0 2024-04-23T14:39:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.573+0000 7fd2ce400640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406b2d0 0x7fd2b406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.573+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55a7fb72d6e0 0x55a7fb86d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.573+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e84) -- 0x55a7fb7d25a0 con 0x55a7fb72d6e0 2024-04-23T14:39:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.574+0000 7fd2d7800640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55a7fb72d6e0 0x55a7fb86d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.574+0000 7fd2ce400640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406b2d0 0x7fd2b406d790 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0057040 tx=0x7fd2c0051000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.574+0000 7fd2d7800640 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55a7fb72d6e0 0x55a7fb86d7a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d7800640 1 -- 172.21.15.59:0/589001420 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v84'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd2d0009870 con 0x55a7fb72d6e0 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55a7fb72d6e0 msgr2=0x55a7fb86d7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55a7fb72d6e0 0x55a7fb86d7a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406b2d0 msgr2=0x7fd2b406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406b2d0 0x7fd2b406d790 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0057040 tx=0x7fd2c0051000 comp rx=0 tx=0).stop 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb6fd1d0 msgr2=0x55a7fb7bddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 --2- 172.21.15.59:0/589001420 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a7fb6fd1d0 0x55a7fb7bddc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8003e70 tx=0x7fd2c8001a00 comp rx=0 tx=0).stop 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d7800640 1 -- 172.21.15.59:0/589001420 reap_dead start 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 shutdown_connections 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.579+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 >> 172.21.15.59:0/589001420 conn(0x55a7fb749cc0 msgr2=0x55a7fb7d7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.580+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 shutdown_connections 2024-04-23T14:39:01.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.580+0000 7fd2d9b1ff40 1 -- 172.21.15.59:0/589001420 wait complete. 2024-04-23T14:39:01.584 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-23T14:39:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.675+0000 7fe972612280 1 -- start start 2024-04-23T14:39:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.675+0000 7fe972612280 1 Processor -- start 2024-04-23T14:39:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.676+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7222c00 0x5597c7ff6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.676+0000 7fe972612280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7223800 0x5597c7ff6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.676+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7223c00 0x5597c7ff6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.676+0000 7fe972612280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fba180 con 0x5597c7223800 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.676+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fba300 con 0x5597c7222c00 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.676+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5597c7fba480 con 0x5597c7223c00 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7223800 0x5597c7ff6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7223800 0x5597c7ff6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45118/0 (socket says 172.21.15.59:45118) 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 -- 172.21.15.59:0/2649662643 learned_addr learned my addr 172.21.15.59:0/2649662643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96fa00640 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7222c00 0x5597c7ff6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe970400640 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7223c00 0x5597c7ff6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 -- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7223c00 msgr2=0x5597c7ff6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7223c00 0x5597c7ff6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 -- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7222c00 msgr2=0x5597c7ff6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7222c00 0x5597c7ff6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.677+0000 7fe96f000640 1 -- 172.21.15.59:0/2649662643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c7256780 con 0x5597c7223800 2024-04-23T14:39:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe96f000640 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7223800 0x5597c7ff6580 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x5597c8053440 tx=0x5597c7fbe960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fcb617203dc546 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe96c800640 1 -- 172.21.15.59:0/2649662643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c7248680 con 0x5597c7223800 2024-04-23T14:39:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe96c800640 1 -- 172.21.15.59:0/2649662643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5597c724e700 con 0x5597c7223800 2024-04-23T14:39:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe96c800640 1 -- 172.21.15.59:0/2649662643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c72489c0 con 0x5597c7223800 2024-04-23T14:39:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe972612280 1 -- 172.21.15.59:0/2649662643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c7257680 con 0x5597c7223800 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe972612280 1 -- 172.21.15.59:0/2649662643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5597c7256000 con 0x5597c7223800 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe972612280 4 waiting for MDS map... 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe96c800640 1 -- 172.21.15.59:0/2649662643 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x5597c800c600 con 0x5597c7223800 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe96c800640 1 -- 172.21.15.59:0/2649662643 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 61) ==== 2885+0+0 (secure 0 0 0) 0x5597c727d180 con 0x5597c7223800 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.678+0000 7fe972612280 4 Got MDS map 61 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.679+0000 7fe972612280 10 main: JournalTool::main 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.679+0000 7fe972612280 4 main: JournalTool: connecting to RADOS... 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.679+0000 7fe972612280 1 Processor -- start 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.679+0000 7fe972612280 1 -- start start 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.680+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.680+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87800 0x5597c7ff7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.680+0000 7fe972612280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.680+0000 7fe972612280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fba600 con 0x5597c7f87c00 2024-04-23T14:39:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.680+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fba780 con 0x5597c7f87400 2024-04-23T14:39:01.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.680+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5597c7fba900 con 0x5597c7f87800 2024-04-23T14:39:01.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.680+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45128/0 (socket says 172.21.15.59:45128) 2024-04-23T14:39:01.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 -- 172.21.15.59:0/1156476574 learned_addr learned my addr 172.21.15.59:0/1156476574 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 -- 172.21.15.59:0/1156476574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87800 msgr2=0x5597c7ff7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87800 0x5597c7ff7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe970400640 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 -- 172.21.15.59:0/1156476574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87400 msgr2=0x5597c7ff7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 -- 172.21.15.59:0/1156476574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c72563c0 con 0x5597c7f87c00 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.681+0000 7fe96f000640 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x5597c81028d0 tx=0x5597c7fbcae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f41b5a909300f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe96a000640 1 -- 172.21.15.59:0/1156476574 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c7248b60 con 0x5597c7f87c00 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe96a000640 1 -- 172.21.15.59:0/1156476574 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5597c724efc0 con 0x5597c7f87c00 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe96a000640 1 -- 172.21.15.59:0/1156476574 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c7248680 con 0x5597c7f87c00 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 -- 172.21.15.59:0/1156476574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 msgr2=0x5597c7ff7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x5597c81028d0 tx=0x5597c7fbcae0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 -- 172.21.15.59:0/1156476574 shutdown_connections 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87800 0x5597c7ff7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 --2- 172.21.15.59:0/1156476574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 -- 172.21.15.59:0/1156476574 >> 172.21.15.59:0/1156476574 conn(0x5597c7f87000 msgr2=0x5597c7309000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 -- 172.21.15.59:0/1156476574 shutdown_connections 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.682+0000 7fe972612280 1 -- 172.21.15.59:0/1156476574 wait complete. 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.683+0000 7fe972612280 1 Processor -- start 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.683+0000 7fe972612280 1 -- start start 2024-04-23T14:39:01.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87800 0x5597c7ff7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe972612280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe972612280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fbbc80 con 0x5597c7f87c00 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fbbb00 con 0x5597c7f87800 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5597c7fbb980 con 0x5597c7f87400 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45140/0 (socket says 172.21.15.59:45140) 2024-04-23T14:39:01.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.684+0000 7fe96f000640 1 -- 172.21.15.59:0/2236523093 learned_addr learned my addr 172.21.15.59:0/2236523093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe970400640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe96f000640 1 -- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87400 msgr2=0x5597c7ff7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe96f000640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe96f000640 1 -- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87800 msgr2=0x5597c7ff7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe96f000640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87800 0x5597c7ff7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe96f000640 1 -- 172.21.15.59:0/2236523093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c7256780 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe970400640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe96f000640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x5597c8103590 tx=0x5597c8106c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe968200640 1 -- 172.21.15.59:0/2236523093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c72489c0 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe968200640 1 -- 172.21.15.59:0/2236523093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5597c724ee00 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe968200640 1 -- 172.21.15.59:0/2236523093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c7248820 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5597c7257680 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c7256960 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.685+0000 7fe972612280 4 main: JournalTool: resolving pool 12 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.687+0000 7fe968200640 1 -- 172.21.15.59:0/2236523093 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5597c8166c00 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.688+0000 7fe968200640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5597c8161000 0x5597c7ff8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.688+0000 7fe968200640 1 -- 172.21.15.59:0/2236523093 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x5597c816a600 con 0x5597c7f87c00 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.688+0000 7fe972612280 4 main: JournalTool: creating IoCtx.. 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.688+0000 7fe972612280 4 main: Executing for rank 0 2024-04-23T14:39:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.688+0000 7fe970400640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5597c8161000 0x5597c7ff8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.689+0000 7fe970400640 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5597c8161000 0x5597c7ff8100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x5597c8053260 tx=0x5597c75a5800 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.689+0000 7fe972612280 1 -- start start 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.689+0000 7fe972612280 1 Processor -- start 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c75ee400 0x5597c7ff8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe972612280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c75ee800 0x5597c7ff8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe972612280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c75eec00 0x5597c7ff9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe972612280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fba780 con 0x5597c75eec00 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5597c7fba900 con 0x5597c75ee400 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe972612280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5597c7fbb680 con 0x5597c75ee800 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c75eec00 0x5597c7ff9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe96f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c75eec00 0x5597c7ff9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45142/0 (socket says 172.21.15.59:45142) 2024-04-23T14:39:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe96f000640 1 -- 172.21.15.59:0/4159159063 learned_addr learned my addr 172.21.15.59:0/4159159063 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe970400640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c75ee400 0x5597c7ff8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.690+0000 7fe96fa00640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c75ee800 0x5597c7ff8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe96f000640 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c75ee800 msgr2=0x5597c7ff8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe96f000640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c75ee800 0x5597c7ff8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe96f000640 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c75ee400 msgr2=0x5597c7ff8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe96f000640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c75ee400 0x5597c7ff8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe96f000640 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c7257860 con 0x5597c75eec00 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe970400640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c75ee400 0x5597c7ff8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe96f000640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c75eec00 0x5597c7ff9180 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x5597c755d5f0 tx=0x5597c75a59e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b794e9264488bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe964600640 1 -- 172.21.15.59:0/4159159063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c7533380 con 0x5597c75eec00 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe964600640 1 -- 172.21.15.59:0/4159159063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5597c724ec40 con 0x5597c75eec00 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe964600640 1 -- 172.21.15.59:0/4159159063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597c7533520 con 0x5597c75eec00 2024-04-23T14:39:01.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c7256000 con 0x5597c75eec00 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5597c72563c0 con 0x5597c75eec00 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.691+0000 7fe972612280 4 waiting for MDS map... 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.692+0000 7fe964600640 1 -- 172.21.15.59:0/4159159063 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x5597c7602400 con 0x5597c75eec00 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.692+0000 7fe964600640 1 -- 172.21.15.59:0/4159159063 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 61) ==== 2885+0+0 (secure 0 0 0) 0x5597c727d500 con 0x5597c75eec00 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.692+0000 7fe972612280 4 Got MDS map 61 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.692+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5597c75ef000 0x5597c7ff9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.692+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84) -- 0x5597c760a800 con 0x5597c75ef000 2024-04-23T14:39:01.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.693+0000 7fe970400640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5597c75ef000 0x5597c7ff9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.694+0000 7fe970400640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5597c75ef000 0x5597c7ff9700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=cd579a842f679946 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.695+0000 7fe970400640 1 -- 172.21.15.59:0/4159159063 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5597c7fac000 con 0x5597c75ef000 2024-04-23T14:39:01.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.695+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5597c760ac00 0x5597c761e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.695+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84) -- 0x5597c760b000 con 0x5597c760ac00 2024-04-23T14:39:01.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.696+0000 7fe96fa00640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5597c760ac00 0x5597c761e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.697+0000 7fe96fa00640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5597c760ac00 0x5597c761e000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=f48813b6d17024dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.704+0000 7fe96fa00640 1 -- 172.21.15.59:0/4159159063 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5597c7fac500 con 0x5597c760ac00 2024-04-23T14:39:01.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.704+0000 7fe962800640 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84) -- 0x5597c8003400 con 0x5597c760ac00 2024-04-23T14:39:01.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.704+0000 7fe962800640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5597c8003000 0x5597c761fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.704+0000 7fe962800640 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84) -- 0x5597c8002800 con 0x5597c8003000 2024-04-23T14:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.705+0000 7fe970400640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5597c8003000 0x5597c761fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.705+0000 7fe96fa00640 1 -- 172.21.15.59:0/4159159063 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv15 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x5597c7fac500 con 0x5597c760ac00 2024-04-23T14:39:01.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.706+0000 7fe970400640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5597c8003000 0x5597c761fb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=1e2d6690f49c3985 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.712+0000 7fe970400640 1 -- 172.21.15.59:0/4159159063 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac000 con 0x5597c8003000 2024-04-23T14:39:01.713 INFO:teuthology.orchestra.run.smithi059.stdout:old journal was 4209834~892 2024-04-23T14:39:01.713 INFO:teuthology.orchestra.run.smithi059.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-04-23T14:39:01.713 INFO:teuthology.orchestra.run.smithi059.stdout:writing journal head 2024-04-23T14:39:01.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.712+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c760b400 con 0x5597c760ac00 2024-04-23T14:39:01.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe96fa00640 1 -- 172.21.15.59:0/4159159063 <== osd.5 v2:172.21.15.76:6808/2782996536 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v84'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac500 con 0x5597c760ac00 2024-04-23T14:39:01.716 INFO:teuthology.orchestra.run.smithi059.stdout:writing EResetJournal entry 2024-04-23T14:39:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c760b800 con 0x5597c8003000 2024-04-23T14:39:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5597c760bc00 0x5597c761e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c760b000 con 0x5597c760bc00 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c8002c00 con 0x5597c75ef000 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c8032000 con 0x5597c8003000 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c8032400 con 0x5597c8003000 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5597c8032800 0x5597c761eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.716+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c8032c00 con 0x5597c8032800 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.717+0000 7fe96fa00640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5597c760bc00 0x5597c761e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.717+0000 7fe96f000640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5597c8032800 0x5597c761eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.718+0000 7fe96fa00640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5597c760bc00 0x5597c761e580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=f91fd4514cdce5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.718+0000 7fe96f000640 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5597c8032800 0x5597c761eb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=771f5307f68cdabf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.720+0000 7fe970400640 1 -- 172.21.15.59:0/4159159063 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== osd_op_reply(8 200.00000004 [delete] v84'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac000 con 0x5597c75ef000 2024-04-23T14:39:01.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.720+0000 7fe96fa00640 1 -- 172.21.15.59:0/4159159063 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(7 200.00000003 [delete] v84'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac500 con 0x5597c760bc00 2024-04-23T14:39:01.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.720+0000 7fe970400640 1 -- 172.21.15.59:0/4159159063 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(9 200.00000005 [delete] v84'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac000 con 0x5597c8003000 2024-04-23T14:39:01.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.720+0000 7fe970400640 1 -- 172.21.15.59:0/4159159063 <== osd.4 v2:172.21.15.76:6816/3407674963 3 ==== osd_op_reply(6 200.00000002 [delete] v84'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac000 con 0x5597c8003000 2024-04-23T14:39:01.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.720+0000 7fe962800640 1 -- 172.21.15.59:0/4159159063 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84) -- 0x5597c760a800 con 0x5597c8003000 2024-04-23T14:39:01.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.720+0000 7fe96f000640 1 -- 172.21.15.59:0/4159159063 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(11 200.00000007 [delete] v84'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac780 con 0x5597c8032800 2024-04-23T14:39:01.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.720+0000 7fe970400640 1 -- 172.21.15.59:0/4159159063 <== osd.4 v2:172.21.15.76:6816/3407674963 4 ==== osd_op_reply(10 200.00000006 [delete] v84'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac000 con 0x5597c8003000 2024-04-23T14:39:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe970400640 1 -- 172.21.15.59:0/4159159063 <== osd.4 v2:172.21.15.76:6816/3407674963 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v84'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5597c7fac000 con 0x5597c8003000 2024-04-23T14:39:01.724 INFO:teuthology.orchestra.run.smithi059.stdout:done 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5597c8032800 msgr2=0x5597c761eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5597c8032800 0x5597c761eb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5597c760bc00 msgr2=0x5597c761e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5597c760bc00 0x5597c761e580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5597c8003000 msgr2=0x5597c761fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5597c8003000 0x5597c761fb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5597c760ac00 msgr2=0x5597c761e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5597c760ac00 0x5597c761e000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5597c75ef000 msgr2=0x5597c7ff9700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5597c75ef000 0x5597c7ff9700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c75eec00 msgr2=0x5597c7ff9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 --2- 172.21.15.59:0/4159159063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c75eec00 0x5597c7ff9180 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x5597c755d5f0 tx=0x5597c75a59e0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe96fa00640 1 -- 172.21.15.59:0/4159159063 reap_dead start 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe96fa00640 1 -- 172.21.15.59:0/4159159063 reap_dead start 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe96fa00640 1 -- 172.21.15.59:0/4159159063 reap_dead start 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 shutdown_connections 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 >> 172.21.15.59:0/4159159063 conn(0x5597c75ee000 msgr2=0x5597c7309800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 shutdown_connections 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.724+0000 7fe972612280 1 -- 172.21.15.59:0/4159159063 wait complete. 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5597c8161000 msgr2=0x5597c7ff8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5597c8161000 0x5597c7ff8100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x5597c8053260 tx=0x5597c75a5800 comp rx=0 tx=0).stop 2024-04-23T14:39:01.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 msgr2=0x5597c7ff7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x5597c8103590 tx=0x5597c8106c00 comp rx=0 tx=0).stop 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 shutdown_connections 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5597c8161000 0x5597c7ff8100 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7f87c00 0x5597c7ff7b80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7f87800 0x5597c7ff7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 --2- 172.21.15.59:0/2236523093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7f87400 0x5597c7ff7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 >> 172.21.15.59:0/2236523093 conn(0x5597c7f87000 msgr2=0x5597c7309000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 shutdown_connections 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.725+0000 7fe972612280 1 -- 172.21.15.59:0/2236523093 wait complete. 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 -- 172.21.15.59:0/2649662643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7223800 msgr2=0x5597c7ff6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7223800 0x5597c7ff6580 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x5597c8053440 tx=0x5597c7fbe960 comp rx=0 tx=0).stop 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 -- 172.21.15.59:0/2649662643 shutdown_connections 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5597c7223c00 0x5597c7ff6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597c7223800 0x5597c7ff6580 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 --2- 172.21.15.59:0/2649662643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597c7222c00 0x5597c7ff6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 -- 172.21.15.59:0/2649662643 >> 172.21.15.59:0/2649662643 conn(0x5597c7223000 msgr2=0x5597c7308800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 -- 172.21.15.59:0/2649662643 shutdown_connections 2024-04-23T14:39:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.726+0000 7fe972612280 1 -- 172.21.15.59:0/2649662643 wait complete. 2024-04-23T14:39:01.731 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.146988, result: old journal was 4209834~892 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-23T14:39:01.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-23T14:39:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.942+0000 7fb0d9a00640 1 Processor -- start 2024-04-23T14:39:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.942+0000 7fb0d9a00640 1 -- start start 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.943+0000 7fb0d9a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4145e10 0x7fb0d406f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.943+0000 7fb0d9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0d40d0cc0 0x7fb0d40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.943+0000 7fb0d9a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb0d4071d70 0x7fb0d4072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.943+0000 7fb0d9a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d40cfc20 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.943+0000 7fb0d9a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d40cff00 con 0x7fb0d4071d70 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.944+0000 7fb0d9a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d40d01e0 con 0x7fb0d4145e10 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.944+0000 7fb0d2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0d40d0cc0 0x7fb0d40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.944+0000 7fb0d2000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0d40d0cc0 0x7fb0d40cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45144/0 (socket says 172.21.15.59:45144) 2024-04-23T14:39:01.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.944+0000 7fb0d2000640 1 -- 172.21.15.59:0/3126474710 learned_addr learned my addr 172.21.15.59:0/3126474710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.944+0000 7fb0d2a00640 1 --2- 172.21.15.59:0/3126474710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4145e10 0x7fb0d406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.944+0000 7fb0d3400640 1 --2- 172.21.15.59:0/3126474710 >> v2:172.21.15.76:3300/0 conn(0x7fb0d4071d70 0x7fb0d4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.945+0000 7fb0d2000640 1 -- 172.21.15.59:0/3126474710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4145e10 msgr2=0x7fb0d406f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.945+0000 7fb0d2000640 1 --2- 172.21.15.59:0/3126474710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4145e10 0x7fb0d406f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.945+0000 7fb0d2000640 1 -- 172.21.15.59:0/3126474710 >> v2:172.21.15.76:3300/0 conn(0x7fb0d4071d70 msgr2=0x7fb0d4072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:01.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.945+0000 7fb0d2000640 1 --2- 172.21.15.59:0/3126474710 >> v2:172.21.15.76:3300/0 conn(0x7fb0d4071d70 0x7fb0d4072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.945+0000 7fb0d2000640 1 -- 172.21.15.59:0/3126474710 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d40d04c0 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.946+0000 7fb0d2000640 1 --2- 172.21.15.59:0/3126474710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d40cf340 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009870 tx=0x7fb0bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285fb7f426cdb53e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.947+0000 7fb0d1600640 1 -- 172.21.15.59:0/3126474710 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc037070 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.947+0000 7fb0d1600640 1 -- 172.21.15.59:0/3126474710 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0bc032440 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.947+0000 7fb0d1600640 1 -- 172.21.15.59:0/3126474710 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc032810 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.947+0000 7fb0d9a00640 1 -- 172.21.15.59:0/3126474710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 msgr2=0x7fb0d40cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.947+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/3126474710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d40cf340 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009870 tx=0x7fb0bc02a210 comp rx=0 tx=0).stop 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.948+0000 7fb0d9a00640 1 -- 172.21.15.59:0/3126474710 shutdown_connections 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.948+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/3126474710 >> v2:172.21.15.76:3300/0 conn(0x7fb0d4071d70 0x7fb0d4072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.948+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/3126474710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d40cf340 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.948+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/3126474710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4145e10 0x7fb0d406f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.948+0000 7fb0d9a00640 1 -- 172.21.15.59:0/3126474710 >> 172.21.15.59:0/3126474710 conn(0x7fb0d4067d90 msgr2=0x7fb0d406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.948+0000 7fb0d9a00640 1 -- 172.21.15.59:0/3126474710 shutdown_connections 2024-04-23T14:39:01.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.948+0000 7fb0d9a00640 1 -- 172.21.15.59:0/3126474710 wait complete. 2024-04-23T14:39:01.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.950+0000 7fb0d9a00640 1 Processor -- start 2024-04-23T14:39:01.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.950+0000 7fb0d9a00640 1 -- start start 2024-04-23T14:39:01.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.951+0000 7fb0d9a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4071d70 0x7fb0d4140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.951+0000 7fb0d9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d415c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.951+0000 7fb0d9a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0d4145e10 0x7fb0d40c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d40cfa00 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d9a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d40cf880 con 0x7fb0d4145e10 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d9a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d41461f0 con 0x7fb0d4071d70 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d415c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45154/0 (socket says 172.21.15.59:45154) 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 -- 172.21.15.59:0/2256019923 learned_addr learned my addr 172.21.15.59:0/2256019923 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4071d70 0x7fb0d4140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d3400640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0d4145e10 0x7fb0d40c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 -- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4071d70 msgr2=0x7fb0d4140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4071d70 0x7fb0d4140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 -- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0d4145e10 msgr2=0x7fb0d40c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0d4145e10 0x7fb0d40c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:01.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.952+0000 7fb0d2000640 1 -- 172.21.15.59:0/2256019923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d4142460 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.953+0000 7fb0d2000640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d415c000 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc000c00 tx=0x7fb0bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.953+0000 7fb0caa00640 1 -- 172.21.15.59:0/2256019923 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc041020 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.953+0000 7fb0caa00640 1 -- 172.21.15.59:0/2256019923 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0bc0306c0 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.953+0000 7fb0caa00640 1 -- 172.21.15.59:0/2256019923 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc037040 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.953+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d413ff50 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.953+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d4140510 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.954+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d4068a80 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.955+0000 7fb0caa00640 1 -- 172.21.15.59:0/2256019923 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0bc044020 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.957+0000 7fb0caa00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0b006c8f0 0x7fb0b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:01.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.957+0000 7fb0caa00640 1 -- 172.21.15.59:0/2256019923 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fb0bc0ac430 con 0x7fb0d40d0cc0 2024-04-23T14:39:01.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.957+0000 7fb0d2a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0b006c8f0 0x7fb0b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:01.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.958+0000 7fb0d2a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0b006c8f0 0x7fb0b006edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006fd0 tx=0x7fb0c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:01.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:01.962+0000 7fb0caa00640 1 -- 172.21.15.59:0/2256019923 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0bc04b030 con 0x7fb0d40d0cc0 2024-04-23T14:39:02.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.208+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb0d415d210 con 0x7fb0d40d0cc0 2024-04-23T14:39:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.434+0000 7fb0caa00640 1 -- 172.21.15.59:0/2256019923 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v62) ==== 162+0+0 (secure 0 0 0) 0x7fb0bc07a760 con 0x7fb0d40d0cc0 2024-04-23T14:39:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.438+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0b006c8f0 msgr2=0x7fb0b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.438+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0b006c8f0 0x7fb0b006edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006fd0 tx=0x7fb0c4008040 comp rx=0 tx=0).stop 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 msgr2=0x7fb0d415c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d415c000 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc000c00 tx=0x7fb0bc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 shutdown_connections 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0b006c8f0 0x7fb0b006edb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0d4145e10 0x7fb0d40c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d40d0cc0 0x7fb0d415c000 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 --2- 172.21.15.59:0/2256019923 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0d4071d70 0x7fb0d4140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 >> 172.21.15.59:0/2256019923 conn(0x7fb0d4067d90 msgr2=0x7fb0d40c5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:02.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 shutdown_connections 2024-04-23T14:39:02.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.439+0000 7fb0d9a00640 1 -- 172.21.15.59:0/2256019923 wait complete. 2024-04-23T14:39:02.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:39:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.664+0000 7fca0de00640 1 Processor -- start 2024-04-23T14:39:02.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.664+0000 7fca0de00640 1 -- start start 2024-04-23T14:39:02.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.665+0000 7fca0de00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fca080cdd80 0x7fca0806b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.665+0000 7fca0de00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca080ce1f0 con 0x7fca0806f350 2024-04-23T14:39:02.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.665+0000 7fca0de00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca080b4930 con 0x7fca0815ae90 2024-04-23T14:39:02.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.665+0000 7fca0de00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fca080cd760 con 0x7fca080cdd80 2024-04-23T14:39:02.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.666+0000 7fca0ca00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fca0806f350 0x7fca080cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50902/0 (socket says 172.21.15.59:50902) 2024-04-23T14:39:02.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.666+0000 7fca0ca00640 1 -- 172.21.15.59:0/3046989374 learned_addr learned my addr 172.21.15.59:0/3046989374 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.666+0000 7fca0d400640 1 --2- 172.21.15.59:0/3046989374 >> v2:172.21.15.76:3300/0 conn(0x7fca080cdd80 0x7fca0806b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.666+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013656413 0 0) 0x7fca080ce1f0 con 0x7fca0806f350 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.666+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9ec003520 con 0x7fca0806f350 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.666+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1392549302 0 0) 0x7fc9ec003520 con 0x7fca0806f350 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 >> v2:172.21.15.76:3300/0 conn(0x7fca080cdd80 msgr2=0x7fca0806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 --2- 172.21.15.59:0/3046989374 >> v2:172.21.15.76:3300/0 conn(0x7fca080cdd80 0x7fca0806b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 >> v1:172.21.15.76:6789/0 conn(0x7fca0815ae90 legacy=0x7fca08071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca080b4c10 con 0x7fca0806f350 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fc9f8001670 con 0x7fca0806f350 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 >> v1:172.21.15.59:6789/0 conn(0x7fca0806f350 legacy=0x7fca080cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9ec0051c0 0x7fc9ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca080cd760 con 0x7fc9ec004800 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec00c060 con 0x7fc9ec0039b0 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ec00c260 con 0x7fc9ec0051c0 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca07e00640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca0ca00640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.667+0000 7fca0d400640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9ec0051c0 0x7fc9ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.668+0000 7fca07e00640 1 -- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9ec0051c0 msgr2=0x7fc9ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.668+0000 7fca07e00640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9ec0051c0 0x7fc9ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.668+0000 7fca07e00640 1 -- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec0039b0 msgr2=0x7fc9ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.668+0000 7fca07e00640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.668+0000 7fca07e00640 1 -- 172.21.15.59:0/3046989374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec003520 con 0x7fc9ec004800 2024-04-23T14:39:02.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.668+0000 7fca0d400640 1 -- 172.21.15.59:0/3046989374 reap_dead start 2024-04-23T14:39:02.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.669+0000 7fca07e00640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9ec004800 0x7fc9ec004c80 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001750 tx=0x7fc9ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f09d646e17f9d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.669+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0001370 con 0x7fc9ec004800 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.669+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f00052d0 con 0x7fc9ec004800 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.669+0000 7fca07400640 1 -- 172.21.15.59:0/3046989374 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f002abe0 con 0x7fc9ec004800 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.669+0000 7fca0de00640 1 -- 172.21.15.59:0/3046989374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9ec004800 msgr2=0x7fc9ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.669+0000 7fca0de00640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9ec004800 0x7fc9ec004c80 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001750 tx=0x7fc9ec005830 comp rx=0 tx=0).stop 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.670+0000 7fca0de00640 1 -- 172.21.15.59:0/3046989374 shutdown_connections 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.670+0000 7fca0de00640 1 --2- 172.21.15.59:0/3046989374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.670+0000 7fca0de00640 1 -- 172.21.15.59:0/3046989374 >> 172.21.15.59:0/3046989374 conn(0x7fca080b21a0 msgr2=0x7fca080b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.670+0000 7fca0de00640 1 -- 172.21.15.59:0/3046989374 shutdown_connections 2024-04-23T14:39:02.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.670+0000 7fca0de00640 1 -- 172.21.15.59:0/3046989374 wait complete. 2024-04-23T14:39:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.672+0000 7fca0de00640 1 Processor -- start 2024-04-23T14:39:02.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.672+0000 7fca0de00640 1 -- start start 2024-04-23T14:39:02.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0de00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec004800 0x7fca0809d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca0815dda0 0x7fca08159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0de00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca080b1530 0x7fca08071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca080b32d0 con 0x7fca0815dda0 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0de00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fca080b3080 con 0x7fc9ec004800 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0de00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fca080b45c0 con 0x7fca080b1530 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca0815dda0 0x7fca08159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca0815dda0 0x7fca08159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45172/0 (socket says 172.21.15.59:45172) 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca07e00640 1 -- 172.21.15.59:0/2908667941 learned_addr learned my addr 172.21.15.59:0/2908667941 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0d400640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca080b1530 0x7fca08071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.673+0000 7fca0ca00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec004800 0x7fca0809d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca07e00640 1 -- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca080b1530 msgr2=0x7fca08071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca07e00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca080b1530 0x7fca08071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca07e00640 1 -- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec004800 msgr2=0x7fca0809d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca07e00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec004800 0x7fca0809d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca07e00640 1 -- 172.21.15.59:0/2908667941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0809dc70 con 0x7fca0815dda0 2024-04-23T14:39:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca07e00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca0815dda0 0x7fca08159210 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0000c00 tx=0x7fc9f0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:02.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca04c00640 1 -- 172.21.15.59:0/2908667941 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f003f030 con 0x7fca0815dda0 2024-04-23T14:39:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca04c00640 1 -- 172.21.15.59:0/2908667941 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f0005480 con 0x7fca0815dda0 2024-04-23T14:39:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca04c00640 1 -- 172.21.15.59:0/2908667941 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0037040 con 0x7fca0815dda0 2024-04-23T14:39:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca08159750 con 0x7fca0815dda0 2024-04-23T14:39:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.674+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0809c970 con 0x7fca0815dda0 2024-04-23T14:39:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.675+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca0804fd90 con 0x7fca0815dda0 2024-04-23T14:39:02.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.677+0000 7fca04c00640 1 -- 172.21.15.59:0/2908667941 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9f0042070 con 0x7fca0815dda0 2024-04-23T14:39:02.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.678+0000 7fca04c00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc9e406c920 0x7fc9e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:02.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.678+0000 7fca04c00640 1 -- 172.21.15.59:0/2908667941 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fc9f00ab200 con 0x7fca0815dda0 2024-04-23T14:39:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.678+0000 7fca0ca00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc9e406c920 0x7fc9e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:02.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.679+0000 7fca0ca00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc9e406c920 0x7fc9e406ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8001eb0 tx=0x7fc9f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:02.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.682+0000 7fca04c00640 1 -- 172.21.15.59:0/2908667941 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9f00794a0 con 0x7fca0815dda0 2024-04-23T14:39:02.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.921+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca080b08e0 con 0x7fca0815dda0 2024-04-23T14:39:02.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.922+0000 7fca04c00640 1 -- 172.21.15.59:0/2908667941 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5174 (secure 0 0 0) 0x7fc9f0078da0 con 0x7fca0815dda0 2024-04-23T14:39:02.923 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:39:02.923 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:39:02.439427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":63,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-23T14:39:02.923 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 63 2024-04-23T14:39:02.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc9e406c920 msgr2=0x7fc9e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc9e406c920 0x7fc9e406ede0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8001eb0 tx=0x7fc9f8009040 comp rx=0 tx=0).stop 2024-04-23T14:39:02.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca0815dda0 msgr2=0x7fca08159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:02.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca0815dda0 0x7fca08159210 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0000c00 tx=0x7fc9f0001120 comp rx=0 tx=0).stop 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 shutdown_connections 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc9e406c920 0x7fc9e406ede0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca080b1530 0x7fca08071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca0815dda0 0x7fca08159210 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 --2- 172.21.15.59:0/2908667941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9ec004800 0x7fca0809d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 >> 172.21.15.59:0/2908667941 conn(0x7fca080b21a0 msgr2=0x7fca0806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 shutdown_connections 2024-04-23T14:39:02.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:02.925+0000 7fca0de00640 1 -- 172.21.15.59:0/2908667941 wait complete. 2024-04-23T14:39:03.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:38:39.210981+0000', 'modified': '2024-04-23T14:39:02.439427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7533': {'gid': 7533, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/2128549219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2128549219}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2128549219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:39:03.345 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7533:up:replay 2024-04-23T14:39:04.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:39:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.566+0000 7f9016c00640 1 Processor -- start 2024-04-23T14:39:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.567+0000 7f9016c00640 1 -- start start 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f9016c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90101454e0 0x7f9010145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f9016c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90100cd020 0x7f90100b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f9016c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90100cd520 con 0x7f90100b3e40 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f9016c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f90100cddc0 con 0x7f90100cd020 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f9016c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f90100cd800 con 0x7f90101454e0 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f900f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90100cd020 0x7f90100b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f9014c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f90100b3e40 0x7f9010160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50918/0 (socket says 172.21.15.59:50918) 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f900f400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90100cd020 0x7f90100b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45174/0 (socket says 172.21.15.59:45174) 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.569+0000 7f9014c00640 1 -- 172.21.15.59:0/3439888046 learned_addr learned my addr 172.21.15.59:0/3439888046 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.570+0000 7f900fe00640 1 --2- 172.21.15.59:0/3439888046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90101454e0 0x7f9010145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.570+0000 7f900f400640 1 -- 172.21.15.59:0/3439888046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90101454e0 msgr2=0x7f9010145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.570+0000 7f900f400640 1 --2- 172.21.15.59:0/3439888046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90101454e0 0x7f9010145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.570+0000 7f900f400640 1 -- 172.21.15.59:0/3439888046 >> v1:172.21.15.59:6789/0 conn(0x7f90100b3e40 legacy=0x7f9010160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.570+0000 7f900f400640 1 -- 172.21.15.59:0/3439888046 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90100cdae0 con 0x7f90100cd020 2024-04-23T14:39:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.570+0000 7f900fe00640 1 --2- 172.21.15.59:0/3439888046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90101454e0 0x7f9010145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.571+0000 7f900f400640 1 --2- 172.21.15.59:0/3439888046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90100b3900 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9000009d90 tx=0x7f900002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58b8befc2f4b8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.571+0000 7f900ea00640 1 -- 172.21.15.59:0/3439888046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000031600 con 0x7f90100cd020 2024-04-23T14:39:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.571+0000 7f900ea00640 1 -- 172.21.15.59:0/3439888046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90000317a0 con 0x7f90100cd020 2024-04-23T14:39:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.571+0000 7f900ea00640 1 -- 172.21.15.59:0/3439888046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000031b70 con 0x7f90100cd020 2024-04-23T14:39:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 -- 172.21.15.59:0/3439888046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 msgr2=0x7f90100b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 --2- 172.21.15.59:0/3439888046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90100b3900 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9000009d90 tx=0x7f900002a5d0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 -- 172.21.15.59:0/3439888046 shutdown_connections 2024-04-23T14:39:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 --2- 172.21.15.59:0/3439888046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90100b3900 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 --2- 172.21.15.59:0/3439888046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90101454e0 0x7f9010145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 -- 172.21.15.59:0/3439888046 >> 172.21.15.59:0/3439888046 conn(0x7f9010067ea0 msgr2=0x7f901006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 -- 172.21.15.59:0/3439888046 shutdown_connections 2024-04-23T14:39:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.572+0000 7f9016c00640 1 -- 172.21.15.59:0/3439888046 wait complete. 2024-04-23T14:39:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.574+0000 7f9016c00640 1 Processor -- start 2024-04-23T14:39:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.574+0000 7f9016c00640 1 -- start start 2024-04-23T14:39:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.575+0000 7f9016c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90100b3e40 0x7f90101eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.575+0000 7f9016c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90101e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.575+0000 7f9016c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90101454e0 0x7f901015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.575+0000 7f9016c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9010146270 con 0x7f90100cd020 2024-04-23T14:39:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.575+0000 7f9016c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f90101460f0 con 0x7f90101454e0 2024-04-23T14:39:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.575+0000 7f9016c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9010161020 con 0x7f90100b3e40 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90101e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90101e6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45184/0 (socket says 172.21.15.59:45184) 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 -- 172.21.15.59:0/2179131 learned_addr learned my addr 172.21.15.59:0/2179131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900fe00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90100b3e40 0x7f90101eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f9014c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90101454e0 0x7f901015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 -- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90100b3e40 msgr2=0x7f90101eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90100b3e40 0x7f90101eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 -- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90101454e0 msgr2=0x7f901015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90101454e0 0x7f901015c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 -- 172.21.15.59:0/2179131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901015c590 con 0x7f90100cd020 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f9014c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90101454e0 0x7f901015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:04.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.576+0000 7f900f400640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90101e6af0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9000009eb0 tx=0x7f9000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.577+0000 7f9007e00640 1 -- 172.21.15.59:0/2179131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000040070 con 0x7f90100cd020 2024-04-23T14:39:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.577+0000 7f9007e00640 1 -- 172.21.15.59:0/2179131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9000030be0 con 0x7f90100cd020 2024-04-23T14:39:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.577+0000 7f9007e00640 1 -- 172.21.15.59:0/2179131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000045460 con 0x7f90100cd020 2024-04-23T14:39:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.577+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90101ea010 con 0x7f90100cd020 2024-04-23T14:39:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.577+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90101ea5d0 con 0x7f90100cd020 2024-04-23T14:39:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.578+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9010068a80 con 0x7f90100cd020 2024-04-23T14:39:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.579+0000 7f9007e00640 1 -- 172.21.15.59:0/2179131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9000039040 con 0x7f90100cd020 2024-04-23T14:39:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.580+0000 7f9007e00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fec06c8f0 0x7f8fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.580+0000 7f9007e00640 1 -- 172.21.15.59:0/2179131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f90000ad4e0 con 0x7f90100cd020 2024-04-23T14:39:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.580+0000 7f900fe00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fec06c8f0 0x7f8fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.581+0000 7f900fe00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fec06c8f0 0x7f8fec06edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800a950 tx=0x7f8ff8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.585+0000 7f9007e00640 1 -- 172.21.15.59:0/2179131 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f901015c590 con 0x7f90100cd020 2024-04-23T14:39:04.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.835+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90101ebd80 con 0x7f90100cd020 2024-04-23T14:39:04.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.836+0000 7f9007e00640 1 -- 172.21.15.59:0/2179131 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5175 (secure 0 0 0) 0x7f900007ba80 con 0x7f90100cd020 2024-04-23T14:39:04.836 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:39:04.836 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:39:03.461838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":63,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-23T14:39:04.837 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 65 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fec06c8f0 msgr2=0x7f8fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fec06c8f0 0x7f8fec06edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800a950 tx=0x7f8ff8008040 comp rx=0 tx=0).stop 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 msgr2=0x7f90101e6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90101e6af0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9000009eb0 tx=0x7f9000004a40 comp rx=0 tx=0).stop 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 shutdown_connections 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8fec06c8f0 0x7f8fec06edb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90101454e0 0x7f901015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90100cd020 0x7f90101e6af0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 --2- 172.21.15.59:0/2179131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90100b3e40 0x7f90101eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 >> 172.21.15.59:0/2179131 conn(0x7f9010067ea0 msgr2=0x7f90100d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 shutdown_connections 2024-04-23T14:39:04.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:04.839+0000 7f9016c00640 1 -- 172.21.15.59:0/2179131 wait complete. 2024-04-23T14:39:05.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:38:39.210981+0000', 'modified': '2024-04-23T14:39:03.461838+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7533': {'gid': 7533, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.59:6837/2128549219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2128549219}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2128549219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:39:05.262 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7533:up:rejoin 2024-04-23T14:39:05.690 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:39:05.690+0000 7fedc1800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:39:06.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:39:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.488+0000 7f2215800640 1 Processor -- start 2024-04-23T14:39:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.488+0000 7f2215800640 1 -- start start 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f2215800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f22100cce70 0x7f2210162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f2215800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f22100b1f60 0x7f22100b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f2215800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22101634e0 con 0x7f22100d07c0 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f2215800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f22100ce190 con 0x7f22100cce70 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f2215800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f22100d0ca0 con 0x7f22100b1f60 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f220fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f22100cce70 0x7f2210162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f220fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f22100cce70 0x7f2210162fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45200/0 (socket says 172.21.15.59:45200) 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f220fe00640 1 -- 172.21.15.59:0/1624276790 learned_addr learned my addr 172.21.15.59:0/1624276790 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:06.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.490+0000 7f2207e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f22100d07c0 0x7f22100ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50926/0 (socket says 172.21.15.59:50926) 2024-04-23T14:39:06.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.491+0000 7f220fe00640 1 -- 172.21.15.59:0/1624276790 >> v2:172.21.15.76:3300/0 conn(0x7f22100b1f60 msgr2=0x7f22100b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:06.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.491+0000 7f220fe00640 1 --2- 172.21.15.59:0/1624276790 >> v2:172.21.15.76:3300/0 conn(0x7f22100b1f60 0x7f22100b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.491+0000 7f220fe00640 1 -- 172.21.15.59:0/1624276790 >> v1:172.21.15.59:6789/0 conn(0x7f22100d07c0 legacy=0x7f22100ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:06.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.491+0000 7f220fe00640 1 -- 172.21.15.59:0/1624276790 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22100cdb40 con 0x7f22100cce70 2024-04-23T14:39:06.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.492+0000 7f220fe00640 1 --2- 172.21.15.59:0/1624276790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100cce70 0x7f2210162fa0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f22101636b0 tx=0x7f220002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1993026f9ef47c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:06.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.493+0000 7f220f400640 1 -- 172.21.15.59:0/1624276790 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200037070 con 0x7f22100cce70 2024-04-23T14:39:06.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.493+0000 7f220f400640 1 -- 172.21.15.59:0/1624276790 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2200032440 con 0x7f22100cce70 2024-04-23T14:39:06.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.493+0000 7f220f400640 1 -- 172.21.15.59:0/1624276790 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200032810 con 0x7f22100cce70 2024-04-23T14:39:06.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.493+0000 7f2215800640 1 -- 172.21.15.59:0/1624276790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100cce70 msgr2=0x7f2210162fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:06.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.493+0000 7f2215800640 1 --2- 172.21.15.59:0/1624276790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100cce70 0x7f2210162fa0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f22101636b0 tx=0x7f220002a270 comp rx=0 tx=0).stop 2024-04-23T14:39:06.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.494+0000 7f2215800640 1 -- 172.21.15.59:0/1624276790 shutdown_connections 2024-04-23T14:39:06.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.494+0000 7f2215800640 1 --2- 172.21.15.59:0/1624276790 >> v2:172.21.15.76:3300/0 conn(0x7f22100b1f60 0x7f22100b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.494+0000 7f2215800640 1 --2- 172.21.15.59:0/1624276790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100cce70 0x7f2210162fa0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.494+0000 7f2215800640 1 -- 172.21.15.59:0/1624276790 >> 172.21.15.59:0/1624276790 conn(0x7f221009aa00 msgr2=0x7f22100d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:06.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.494+0000 7f2215800640 1 -- 172.21.15.59:0/1624276790 shutdown_connections 2024-04-23T14:39:06.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.494+0000 7f2215800640 1 -- 172.21.15.59:0/1624276790 wait complete. 2024-04-23T14:39:06.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.495+0000 7f2215800640 1 Processor -- start 2024-04-23T14:39:06.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.496+0000 7f2215800640 1 -- start start 2024-04-23T14:39:06.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.496+0000 7f2215800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22100b1f60 0x7f22101f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:06.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.496+0000 7f2215800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22100cce70 0x7f22100c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:06.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.496+0000 7f2215800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100d07c0 0x7f22100c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2215800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f22100cd250 con 0x7f22100d07c0 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2215800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22100d0f80 con 0x7f22100b1f60 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2215800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f22100d1db0 con 0x7f22100cce70 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100d07c0 0x7f22100c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100d07c0 0x7f22100c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45212/0 (socket says 172.21.15.59:45212) 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 -- 172.21.15.59:0/3716550484 learned_addr learned my addr 172.21.15.59:0/3716550484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f220fe00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22100b1f60 0x7f22101f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2207e00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22100cce70 0x7f22100c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 -- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22100cce70 msgr2=0x7f22100c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22100cce70 0x7f22100c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 -- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22100b1f60 msgr2=0x7f22101f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22100b1f60 0x7f22101f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.497+0000 7f2214e00640 1 -- 172.21.15.59:0/3716550484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22100af780 con 0x7f22100d07c0 2024-04-23T14:39:06.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.498+0000 7f2214e00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100d07c0 0x7f22100c5350 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f220800b970 tx=0x7f220800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:06.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.498+0000 7f220cc00640 1 -- 172.21.15.59:0/3716550484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208019070 con 0x7f22100d07c0 2024-04-23T14:39:06.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.498+0000 7f220cc00640 1 -- 172.21.15.59:0/3716550484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220800cb60 con 0x7f22100d07c0 2024-04-23T14:39:06.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.498+0000 7f220cc00640 1 -- 172.21.15.59:0/3716550484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208014440 con 0x7f22100d07c0 2024-04-23T14:39:06.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.498+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22100afa60 con 0x7f22100d07c0 2024-04-23T14:39:06.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.498+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22100b0300 con 0x7f22100d07c0 2024-04-23T14:39:06.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.499+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22100afc30 con 0x7f22100d07c0 2024-04-23T14:39:06.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.501+0000 7f220cc00640 1 -- 172.21.15.59:0/3716550484 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f220801c070 con 0x7f22100d07c0 2024-04-23T14:39:06.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.501+0000 7f220cc00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21ec06c8f0 0x7f21ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:06.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.502+0000 7f220cc00640 1 -- 172.21.15.59:0/3716550484 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f220808b980 con 0x7f22100d07c0 2024-04-23T14:39:06.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.502+0000 7f220fe00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21ec06c8f0 0x7f21ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:06.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.502+0000 7f220fe00640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21ec06c8f0 0x7f21ec06edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2200034000 tx=0x7f220002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:06.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.506+0000 7f220cc00640 1 -- 172.21.15.59:0/3716550484 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2208059cb0 con 0x7f22100d07c0 2024-04-23T14:39:06.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.756+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f221004fd90 con 0x7f22100d07c0 2024-04-23T14:39:06.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.758+0000 7f220cc00640 1 -- 172.21.15.59:0/3716550484 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5182 (secure 0 0 0) 0x7f22080595b0 con 0x7f22100d07c0 2024-04-23T14:39:06.759 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:39:06.759 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:39:05.471564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":11,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7533,"qdb_cluster":[7533]},"id":6}]} 2024-04-23T14:39:06.759 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 66 2024-04-23T14:39:06.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.760+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21ec06c8f0 msgr2=0x7f21ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:06.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.760+0000 7f2215800640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21ec06c8f0 0x7f21ec06edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2200034000 tx=0x7f220002e000 comp rx=0 tx=0).stop 2024-04-23T14:39:06.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100d07c0 msgr2=0x7f22100c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:06.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100d07c0 0x7f22100c5350 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f220800b970 tx=0x7f220800be30 comp rx=0 tx=0).stop 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 shutdown_connections 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f21ec06c8f0 0x7f21ec06edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f22100cce70 0x7f22100c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22100d07c0 0x7f22100c5350 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 --2- 172.21.15.59:0/3716550484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22100b1f60 0x7f22101f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 >> 172.21.15.59:0/3716550484 conn(0x7f221009aa00 msgr2=0x7f22100b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 shutdown_connections 2024-04-23T14:39:06.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:06.761+0000 7f2215800640 1 -- 172.21.15.59:0/3716550484 wait complete. 2024-04-23T14:39:07.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:38:39.210981+0000', 'modified': '2024-04-23T14:39:05.471564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 84, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7533': {'gid': 7533, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.59:6837/2128549219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2128549219}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2128549219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7533, 'qdb_cluster': [7533]} 2024-04-23T14:39:07.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:39:07.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:39:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.405+0000 7fd802c00640 1 Processor -- start 2024-04-23T14:39:07.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.405+0000 7fd802c00640 1 -- start start 2024-04-23T14:39:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.407+0000 7fd802c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.407+0000 7fd802c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd7fc0d0b00 0x7fd7fc06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.407+0000 7fd802c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7fc0b4cf0 con 0x7fd7fc15c670 2024-04-23T14:39:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.407+0000 7fd802c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd7fc15ca50 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.407+0000 7fd802c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc0ccb80 con 0x7fd7fc0cd8b0 2024-04-23T14:39:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd800e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd800e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc0d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:57736/0 (socket says 172.21.15.59:57736) 2024-04-23T14:39:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd800e00640 1 -- 172.21.15.59:0/623391414 learned_addr learned my addr 172.21.15.59:0/623391414 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd802200640 1 --2- 172.21.15.59:0/623391414 >> v2:172.21.15.76:3300/0 conn(0x7fd7fc0d0b00 0x7fd7fc06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd800e00640 1 -- 172.21.15.59:0/623391414 >> v2:172.21.15.76:3300/0 conn(0x7fd7fc0d0b00 msgr2=0x7fd7fc06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd800e00640 1 --2- 172.21.15.59:0/623391414 >> v2:172.21.15.76:3300/0 conn(0x7fd7fc0d0b00 0x7fd7fc06ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd800e00640 1 -- 172.21.15.59:0/623391414 >> v1:172.21.15.76:6789/0 conn(0x7fd7fc15c670 legacy=0x7fd7fc0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.408+0000 7fd800e00640 1 -- 172.21.15.59:0/623391414 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc0ccdd0 con 0x7fd7fc0cd8b0 2024-04-23T14:39:07.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.410+0000 7fd800e00640 1 --2- 172.21.15.59:0/623391414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc0d05c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009870 tx=0x7fd7e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75166cea52393ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.411+0000 7fd7fbe00640 1 -- 172.21.15.59:0/623391414 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8037070 con 0x7fd7fc0cd8b0 2024-04-23T14:39:07.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.411+0000 7fd7fbe00640 1 -- 172.21.15.59:0/623391414 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8032440 con 0x7fd7fc0cd8b0 2024-04-23T14:39:07.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.411+0000 7fd7fbe00640 1 -- 172.21.15.59:0/623391414 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8032810 con 0x7fd7fc0cd8b0 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.411+0000 7fd802c00640 1 -- 172.21.15.59:0/623391414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 msgr2=0x7fd7fc0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.411+0000 7fd802c00640 1 --2- 172.21.15.59:0/623391414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc0d05c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009870 tx=0x7fd7e802a270 comp rx=0 tx=0).stop 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.412+0000 7fd802c00640 1 -- 172.21.15.59:0/623391414 shutdown_connections 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.412+0000 7fd802c00640 1 --2- 172.21.15.59:0/623391414 >> v2:172.21.15.76:3300/0 conn(0x7fd7fc0d0b00 0x7fd7fc06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.412+0000 7fd802c00640 1 --2- 172.21.15.59:0/623391414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc0d05c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.412+0000 7fd802c00640 1 -- 172.21.15.59:0/623391414 >> 172.21.15.59:0/623391414 conn(0x7fd7fc06bdf0 msgr2=0x7fd7fc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.412+0000 7fd802c00640 1 -- 172.21.15.59:0/623391414 shutdown_connections 2024-04-23T14:39:07.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.412+0000 7fd802c00640 1 -- 172.21.15.59:0/623391414 wait complete. 2024-04-23T14:39:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.414+0000 7fd802c00640 1 Processor -- start 2024-04-23T14:39:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.414+0000 7fd802c00640 1 -- start start 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.415+0000 7fd802c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.415+0000 7fd802c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7fc0d0b00 0x7fd7fc15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.415+0000 7fd802c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7fc15c670 0x7fd7fc15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.415+0000 7fd802c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc0b3e60 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.415+0000 7fd802c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc0cd390 con 0x7fd7fc15c670 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.415+0000 7fd802c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc0b3fe0 con 0x7fd7fc0cd8b0 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.415+0000 7fd800e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7fc0d0b00 0x7fd7fc15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7fc0d0b00 0x7fd7fc15eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45224/0 (socket says 172.21.15.59:45224) 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 -- 172.21.15.59:0/1153782381 learned_addr learned my addr 172.21.15.59:0/1153782381 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd801800640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 -- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 msgr2=0x7fd7fc148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd802200640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7fc15c670 0x7fd7fc15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 -- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7fc15c670 msgr2=0x7fd7fc15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7fc15c670 0x7fd7fc15f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 -- 172.21.15.59:0/1153782381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc09d410 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.416+0000 7fd800e00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7fc0d0b00 0x7fd7fc15eda0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8000c00 tx=0x7fd7e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.417+0000 7fd7f9600640 1 -- 172.21.15.59:0/1153782381 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8037040 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.417+0000 7fd7f9600640 1 -- 172.21.15.59:0/1153782381 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8042070 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.417+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc09d6f0 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.417+0000 7fd7f9600640 1 -- 172.21.15.59:0/1153782381 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8030880 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.417+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc09dcb0 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.420+0000 7fd7f9600640 1 -- 172.21.15.59:0/1153782381 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7e8030a20 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.421+0000 7fd7f9600640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7dc06c8f0 0x7fd7dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.421+0000 7fd7f9600640 1 -- 172.21.15.59:0/1153782381 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fd7e80ac560 con 0x7fd7fc0d0b00 2024-04-23T14:39:07.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.421+0000 7fd801800640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7dc06c8f0 0x7fd7dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.422+0000 7fd806fe5740 1 Processor -- start 2024-04-23T14:39:07.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.422+0000 7fd806fe5740 1 -- start start 2024-04-23T14:39:07.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.422+0000 7fd801800640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7dc06c8f0 0x7fd7dc06edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0001500 tx=0x7fd7f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff075310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 0x55f0ff0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5360 0x55f0ff07a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd806fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f0fefdfb10 con 0x55f0ff0a5360 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f0fef862b0 con 0x55f0ff0c2980 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55f0fefdbad0 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5360 0x55f0ff07a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5360 0x55f0ff07a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45240/0 (socket says 172.21.15.59:45240) 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 -- 172.21.15.59:0/3199517721 learned_addr learned my addr 172.21.15.59:0/3199517721 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 -- 172.21.15.59:0/3199517721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 msgr2=0x55f0ff0a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 --2- 172.21.15.59:0/3199517721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 0x55f0ff0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 -- 172.21.15.59:0/3199517721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 msgr2=0x55f0ff075310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 --2- 172.21.15.59:0/3199517721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.423+0000 7fd802200640 1 -- 172.21.15.59:0/3199517721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0fecf22d0 con 0x55f0ff0a5360 2024-04-23T14:39:07.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.424+0000 7fd802200640 1 --2- 172.21.15.59:0/3199517721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5360 0x55f0ff07a170 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007550 tx=0x7fd7f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8d11c5c6e1f0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.424+0000 7fd7ee000640 1 -- 172.21.15.59:0/3199517721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4003990 con 0x55f0ff0a5360 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.424+0000 7fd7ee000640 1 -- 172.21.15.59:0/3199517721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4003b30 con 0x55f0ff0a5360 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.424+0000 7fd7ee000640 1 -- 172.21.15.59:0/3199517721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004020 con 0x55f0ff0a5360 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 -- 172.21.15.59:0/3199517721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5360 msgr2=0x55f0ff07a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3199517721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5360 0x55f0ff07a170 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007550 tx=0x7fd7f4009750 comp rx=0 tx=0).stop 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 -- 172.21.15.59:0/3199517721 shutdown_connections 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3199517721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 0x55f0ff0a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3199517721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5360 0x55f0ff07a170 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3199517721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff075310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 -- 172.21.15.59:0/3199517721 >> 172.21.15.59:0/3199517721 conn(0x55f0ff077580 msgr2=0x55f0ff0bfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 -- 172.21.15.59:0/3199517721 shutdown_connections 2024-04-23T14:39:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.425+0000 7fd806fe5740 1 -- 172.21.15.59:0/3199517721 wait complete. 2024-04-23T14:39:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.426+0000 7fd806fe5740 1 -- start start 2024-04-23T14:39:07.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.426+0000 7fd806fe5740 1 Processor -- start 2024-04-23T14:39:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0c2980 0x55f0ff07a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd806fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f0ff079730 con 0x55f0ff0c2980 2024-04-23T14:39:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f0fef99590 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55f0ff078b20 con 0x55f0ff07a950 2024-04-23T14:39:07.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd806fe5740 10 client.0 _ll_register_callbacks cb 0x55f0fef7c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd800e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0c2980 0x55f0ff07a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd800e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0c2980 0x55f0ff07a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45248/0 (socket says 172.21.15.59:45248) 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd800e00640 1 -- 172.21.15.59:0/3054800097 learned_addr learned my addr 172.21.15.59:0/3054800097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.427+0000 7fd801800640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd800e00640 1 -- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 msgr2=0x55f0ff0d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd802200640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd800e00640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd800e00640 1 -- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0a5aa0 msgr2=0x55f0ff0a6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd800e00640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0a6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd800e00640 1 -- 172.21.15.59:0/3054800097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0fedbc6b0 con 0x55f0ff0c2980 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd801800640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:07.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd802200640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd800e00640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0c2980 0x55f0ff07a410 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80827c0 tx=0x7fd7e8032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc42a5d40623402c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd7e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3054800097 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e807e4c0 con 0x55f0ff0c2980 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3054800097 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fd7e8030d10 con 0x55f0ff0c2980 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f4003b30 con 0x55f0ff0c2980 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.428+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3054800097 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd7e8082cf0 con 0x55f0ff0c2980 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 10 client.7860 fetch_fsmap learned FSMap version 66 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55f0ff0a8700 con 0x55f0ff0c2980 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3054800097 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f4003b30 con 0x55f0ff0c2980 2024-04-23T14:39:07.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3054800097 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fd7e80a5020 con 0x55f0ff0c2980 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 10 client.7860 fetch_fsmap finished waiting for FSMap version 66 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 10 client.7860 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 20 client.7860 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0a36e0 0x55f0ff0deb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 4 client.7860 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0fefed530 con 0x55f0ff0a36e0 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.429+0000 7fd801800640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0a36e0 0x55f0ff0deb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.430+0000 7fd801800640 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0a36e0 0x55f0ff0deb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7db6609aab1d6a server_cookie=9d31937eced10c7f in_seq=0 out_seq=0 2024-04-23T14:39:07.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.430+0000 7fd7e2a00640 10 client.7860 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.431+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3054800097 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0fefed530 con 0x55f0ff0a36e0 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.431+0000 7fd7e2a00640 10 client.7860 handle_command_reply: tid=0 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.431+0000 7fd806fe5740 1 client.7860 shutdown 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.431+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0c2980 msgr2=0x55f0ff07a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.431+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0c2980 0x55f0ff07a410 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80827c0 tx=0x7fd7e8032440 comp rx=0 tx=0).stop 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 shutdown_connections 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0a36e0 0x55f0ff0deb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0c2980 0x55f0ff07a410 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3054800097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 >> 172.21.15.59:0/3054800097 conn(0x55f0ff0a5360 msgr2=0x55f0ff0bfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd7e2a00640 0 client.7860 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 shutdown_connections 2024-04-23T14:39:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 1 -- 172.21.15.59:0/3054800097 wait complete. 2024-04-23T14:39:07.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.432+0000 7fd806fe5740 20 client.7860 trim_cache size 0 max 16384 2024-04-23T14:39:07.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.452+0000 7fd806fe5740 1 Processor -- start 2024-04-23T14:39:07.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.452+0000 7fd806fe5740 1 -- start start 2024-04-23T14:39:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff0bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd806fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1fcbf0 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc072f30 con 0x55f0ff0c2980 2024-04-23T14:39:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc09c840 con 0x55f0ff07a950 2024-04-23T14:39:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd800e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd800e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45260/0 (socket says 172.21.15.59:45260) 2024-04-23T14:39:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.453+0000 7fd800e00640 1 -- 172.21.15.59:0/2054345666 learned_addr learned my addr 172.21.15.59:0/2054345666 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd801800640 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd800e00640 1 -- 172.21.15.59:0/2054345666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 msgr2=0x55f0ff0a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd800e00640 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd800e00640 1 -- 172.21.15.59:0/2054345666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 msgr2=0x55f0ff0bedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd800e00640 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff0bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd800e00640 1 -- 172.21.15.59:0/2054345666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0ff0d0720 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd800e00640 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009210 tx=0x7fd7e803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a1df7dfe28d2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd7ed600640 1 -- 172.21.15.59:0/2054345666 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e807e4c0 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd7ed600640 1 -- 172.21.15.59:0/2054345666 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8030d10 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.454+0000 7fd7ed600640 1 -- 172.21.15.59:0/2054345666 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8079c50 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 -- 172.21.15.59:0/2054345666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5aa0 msgr2=0x55f0ff0cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc60 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009210 tx=0x7fd7e803e000 comp rx=0 tx=0).stop 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 -- 172.21.15.59:0/2054345666 shutdown_connections 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff0bedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc60 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 --2- 172.21.15.59:0/2054345666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff07a950 0x55f0ff0a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 -- 172.21.15.59:0/2054345666 >> 172.21.15.59:0/2054345666 conn(0x55f0ff0a5360 msgr2=0x55f0ff070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 -- 172.21.15.59:0/2054345666 shutdown_connections 2024-04-23T14:39:07.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.455+0000 7fd806fe5740 1 -- 172.21.15.59:0/2054345666 wait complete. 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.456+0000 7fd806fe5740 1 -- start start 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.456+0000 7fd806fe5740 1 Processor -- start 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.457+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff07a950 0x55f0ff0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.457+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.457+0000 7fd806fe5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff0e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.457+0000 7fd806fe5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f0ff0a8a80 con 0x55f0ff07a950 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.457+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc09ce90 con 0x55f0ff0c2980 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.457+0000 7fd806fe5740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55f0ff0d1820 con 0x55f0ff0a5aa0 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.457+0000 7fd806fe5740 10 client.0 _ll_register_callbacks cb 0x7fd7e8032f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff07a950 0x55f0ff0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff07a950 0x55f0ff0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45268/0 (socket says 172.21.15.59:45268) 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 -- 172.21.15.59:0/3368677850 learned_addr learned my addr 172.21.15.59:0/3368677850 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 -- 172.21.15.59:0/3368677850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 msgr2=0x55f0ff0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 -- 172.21.15.59:0/3368677850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 msgr2=0x55f0ff0e3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff0e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 -- 172.21.15.59:0/3368677850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0ff0ce650 con 0x55f0ff07a950 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd801800640 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff07a950 0x55f0ff0cf710 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0001330 tx=0x7fd7f0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93eb0e278a32b130 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.458+0000 7fd7e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3368677850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00245b0 con 0x55f0ff07a950 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3368677850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7fd7f0024750 con 0x55f0ff07a950 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7e8030d10 con 0x55f0ff07a950 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3368677850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd7f0035050 con 0x55f0ff07a950 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 10 client.7866 fetch_fsmap learned FSMap version 66 2024-04-23T14:39:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55f0ff0e2ed0 con 0x55f0ff07a950 2024-04-23T14:39:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3368677850 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8030d10 con 0x55f0ff07a950 2024-04-23T14:39:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3368677850 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fd7f002d430 con 0x55f0ff07a950 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 10 client.7866 fetch_fsmap finished waiting for FSMap version 66 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 10 client.7866 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 20 client.7866 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0b57b0 0x55f0ff0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 4 client.7866 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.459+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "status"}) -- 0x55f0ff0b78d0 con 0x55f0ff0b57b0 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.460+0000 7fd800e00640 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0b57b0 0x55f0ff0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.460+0000 7fd800e00640 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0b57b0 0x55f0ff0b5b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcfaa8ca24d7dd32 server_cookie=2731f765e60b60fd in_seq=0 out_seq=0 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.460+0000 7fd7e2a00640 10 client.7866 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.460+0000 7fd7e2a00640 1 -- 172.21.15.59:0/3368677850 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f0ff0b78d0 con 0x55f0ff0b57b0 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.460+0000 7fd7e2a00640 10 client.7866 handle_command_reply: tid=0 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 client.7866 shutdown 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff07a950 msgr2=0x55f0ff0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff07a950 0x55f0ff0cf710 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0001330 tx=0x7fd7f0000ed0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 shutdown_connections 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55f0ff0b57b0 0x55f0ff0b5b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55f0ff0a5aa0 0x55f0ff0cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0ff0c2980 0x55f0ff0e3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 --2- 172.21.15.59:0/3368677850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0ff07a950 0x55f0ff0cf710 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd7e2a00640 0 client.7866 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 >> 172.21.15.59:0/3368677850 conn(0x55f0ff0a5360 msgr2=0x55f0ff0bedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 shutdown_connections 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 1 -- 172.21.15.59:0/3368677850 wait complete. 2024-04-23T14:39:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.461+0000 7fd806fe5740 20 client.7866 trim_cache size 0 max 16384 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7533, 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0175965070000004, 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 66, 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 84, 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 84, 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 33.122829154000001 2024-04-23T14:39:07.465 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:39:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7dc06c8f0 msgr2=0x7fd7dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7dc06c8f0 0x7fd7dc06edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0001500 tx=0x7fd7f0009040 comp rx=0 tx=0).stop 2024-04-23T14:39:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7fc0d0b00 msgr2=0x7fd7fc15eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7fc0d0b00 0x7fd7fc15eda0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8000c00 tx=0x7fd7e80046c0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 shutdown_connections 2024-04-23T14:39:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7dc06c8f0 0x7fd7dc06edb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7fc15c670 0x7fd7fc15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7fc0d0b00 0x7fd7fc15eda0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 --2- 172.21.15.59:0/1153782381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7fc0cd8b0 0x7fd7fc148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 >> 172.21.15.59:0/1153782381 conn(0x7fd7fc06bdf0 msgr2=0x7fd7fc15b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 shutdown_connections 2024-04-23T14:39:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.462+0000 7fd802c00640 1 -- 172.21.15.59:0/1153782381 wait complete. 2024-04-23T14:39:07.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:39:07.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.693+0000 7f5dc6400640 1 Processor -- start 2024-04-23T14:39:07.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.693+0000 7f5dc6400640 1 -- start start 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.694+0000 7f5dc6400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5dc015c700 0x7f5dc00d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.694+0000 7f5dc6400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5dc006c0e0 0x7f5dc00b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc6400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc00d0c70 con 0x7f5dc0160d50 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc6400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc00b46b0 con 0x7f5dc015c700 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc6400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc00d0f50 con 0x7f5dc006c0e0 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5dc015c700 0x7f5dc00d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5dc015c700 0x7f5dc00d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45284/0 (socket says 172.21.15.59:45284) 2024-04-23T14:39:07.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5000640 1 -- 172.21.15.59:0/3541413587 learned_addr learned my addr 172.21.15.59:0/3541413587 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5a00640 1 --2- 172.21.15.59:0/3541413587 >> v2:172.21.15.76:3300/0 conn(0x7f5dc006c0e0 0x7f5dc00b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5000640 1 -- 172.21.15.59:0/3541413587 >> v2:172.21.15.76:3300/0 conn(0x7f5dc006c0e0 msgr2=0x7f5dc00b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5000640 1 --2- 172.21.15.59:0/3541413587 >> v2:172.21.15.76:3300/0 conn(0x7f5dc006c0e0 0x7f5dc00b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5000640 1 -- 172.21.15.59:0/3541413587 >> v1:172.21.15.76:6789/0 conn(0x7f5dc0160d50 legacy=0x7f5dc006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:07.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5000640 1 -- 172.21.15.59:0/3541413587 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00b4340 con 0x7f5dc015c700 2024-04-23T14:39:07.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.695+0000 7f5dc5a00640 1 --2- 172.21.15.59:0/3541413587 >> v2:172.21.15.76:3300/0 conn(0x7f5dc006c0e0 0x7f5dc00b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:07.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.696+0000 7f5dc5000640 1 --2- 172.21.15.59:0/3541413587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc00d0600 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0073150 tx=0x7f5da802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ff58402df8e077 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.697+0000 7f5dbf400640 1 -- 172.21.15.59:0/3541413587 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8037070 con 0x7f5dc015c700 2024-04-23T14:39:07.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.697+0000 7f5dbf400640 1 -- 172.21.15.59:0/3541413587 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da8032440 con 0x7f5dc015c700 2024-04-23T14:39:07.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.697+0000 7f5dbf400640 1 -- 172.21.15.59:0/3541413587 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8032810 con 0x7f5dc015c700 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.697+0000 7f5dc6400640 1 -- 172.21.15.59:0/3541413587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 msgr2=0x7f5dc00d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.697+0000 7f5dc6400640 1 --2- 172.21.15.59:0/3541413587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc00d0600 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0073150 tx=0x7f5da802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.698+0000 7f5dc6400640 1 -- 172.21.15.59:0/3541413587 shutdown_connections 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.698+0000 7f5dc6400640 1 --2- 172.21.15.59:0/3541413587 >> v2:172.21.15.76:3300/0 conn(0x7f5dc006c0e0 0x7f5dc00b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.698+0000 7f5dc6400640 1 --2- 172.21.15.59:0/3541413587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc00d0600 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.698+0000 7f5dc6400640 1 -- 172.21.15.59:0/3541413587 >> 172.21.15.59:0/3541413587 conn(0x7f5dc009aa00 msgr2=0x7f5dc00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.698+0000 7f5dc6400640 1 -- 172.21.15.59:0/3541413587 shutdown_connections 2024-04-23T14:39:07.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.698+0000 7f5dc6400640 1 -- 172.21.15.59:0/3541413587 wait complete. 2024-04-23T14:39:07.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.700+0000 7f5dc6400640 1 Processor -- start 2024-04-23T14:39:07.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.700+0000 7f5dc6400640 1 -- start start 2024-04-23T14:39:07.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.701+0000 7f5dc6400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc006c0e0 0x7f5dc00ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.701+0000 7f5dc6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc015d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dc6400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc0160d50 0x7f5dc00b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dc6400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc00b5080 con 0x7f5dc015c700 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dc6400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01611b0 con 0x7f5dc006c0e0 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dc6400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc00b5200 con 0x7f5dc0160d50 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc015d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45290/0 (socket says 172.21.15.59:45290) 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 -- 172.21.15.59:0/352589701 learned_addr learned my addr 172.21.15.59:0/352589701 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dc5a00640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc0160d50 0x7f5dc00b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dc5000640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc006c0e0 0x7f5dc00ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 -- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc0160d50 msgr2=0x7f5dc00b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc0160d50 0x7f5dc00b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 -- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc006c0e0 msgr2=0x7f5dc00ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc006c0e0 0x7f5dc00ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dbfe00640 1 -- 172.21.15.59:0/352589701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc015e020 con 0x7f5dc015c700 2024-04-23T14:39:07.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.702+0000 7f5dc5a00640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc0160d50 0x7f5dc00b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:07.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.703+0000 7f5dbfe00640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc015d780 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ba80 tx=0x7f5db000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.703+0000 7f5dbcc00640 1 -- 172.21.15.59:0/352589701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0019070 con 0x7f5dc015c700 2024-04-23T14:39:07.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.703+0000 7f5dbcc00640 1 -- 172.21.15.59:0/352589701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db000cc30 con 0x7f5dc015c700 2024-04-23T14:39:07.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.703+0000 7f5dbcc00640 1 -- 172.21.15.59:0/352589701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0014440 con 0x7f5dc015c700 2024-04-23T14:39:07.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.703+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc015e300 con 0x7f5dc015c700 2024-04-23T14:39:07.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.703+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc015e8c0 con 0x7f5dc015c700 2024-04-23T14:39:07.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.704+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc004fd90 con 0x7f5dc015c700 2024-04-23T14:39:07.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.706+0000 7f5dbcc00640 1 -- 172.21.15.59:0/352589701 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5db0007370 con 0x7f5dc015c700 2024-04-23T14:39:07.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.707+0000 7f5dbcc00640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d9c06c920 0x7f5d9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:07.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.707+0000 7f5dbcc00640 1 -- 172.21.15.59:0/352589701 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5795+0+0 (secure 0 0 0) 0x7f5db008b850 con 0x7f5dc015c700 2024-04-23T14:39:07.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.707+0000 7f5dc5000640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d9c06c920 0x7f5d9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:07.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.708+0000 7f5dc5000640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d9c06c920 0x7f5d9c06ede0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5da8034000 tx=0x7f5da802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.713+0000 7f5dbcc00640 1 -- 172.21.15.59:0/352589701 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5db0059b80 con 0x7f5dc015c700 2024-04-23T14:39:07.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.959+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5dc00cd920 con 0x7f5dc015c700 2024-04-23T14:39:07.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.960+0000 7f5dbcc00640 1 -- 172.21.15.59:0/352589701 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f5db0059480 con 0x7f5dc015c700 2024-04-23T14:39:07.961 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:39:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.963+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d9c06c920 msgr2=0x7f5d9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.963+0000 7f5dc6400640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d9c06c920 0x7f5d9c06ede0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5da8034000 tx=0x7f5da802e000 comp rx=0 tx=0).stop 2024-04-23T14:39:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.963+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 msgr2=0x7f5dc015d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.963+0000 7f5dc6400640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc015d780 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ba80 tx=0x7f5db000bf40 comp rx=0 tx=0).stop 2024-04-23T14:39:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 shutdown_connections 2024-04-23T14:39:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5d9c06c920 0x7f5d9c06ede0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc0160d50 0x7f5dc00b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc015c700 0x7f5dc015d780 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 --2- 172.21.15.59:0/352589701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc006c0e0 0x7f5dc00ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 >> 172.21.15.59:0/352589701 conn(0x7f5dc009aa00 msgr2=0x7f5dc006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 shutdown_connections 2024-04-23T14:39:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:07.964+0000 7f5dc6400640 1 -- 172.21.15.59:0/352589701 wait complete. 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:39:07.984 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:39:07.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:07.984 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:39:08.039 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:39:08.039 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:39:08.040 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42001sec preferred_lft 42001sec 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:39:08.041 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:39:08.042 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:39:08.042 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:39:08.042 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:39:08.042 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:39:08.042 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:39:08.042 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:39:08.042 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:39:08.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:08.043 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:39:08.125 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:39:08.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:08.130 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:39:08.185 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:39:08.185 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:39:08.185 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:39:08.186 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:39:08.186 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:39:08.187 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:39:08.187 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:39:08.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:39:08.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:39:08.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:39:08.187 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:39:08.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:39:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:39:08.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:08.419 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:39:08.470 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:39:08.470 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:39:08.471 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:39:08.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:08.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:39:08.553 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:39:08.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:39:08.553 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:39:08.600 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:39:08.600 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:39:08.600 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:39:08.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:39:08.743 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:39:08.803 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:39:08.804 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:39:08.805 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:39:08.806 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:39:08.806 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:39:08.806 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:39:08.806 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:39:08.806 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:39:08.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:08.807 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:39:08.865 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:39:08.866 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:39:08.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:39:08.942 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:39:08.986 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:39:08.986 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:39:08.986 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:39:08.986 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:39:08.987 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:39:08.988 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:39:08.989 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:39:08.989 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:39:08.989 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:39:08.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:08.990 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:39:09.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:39:09.051+0000 7fe70867a480 -1 init, newargv = 0x560445d85750 newargc=15 2024-04-23T14:39:09.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[64972]: starting ceph client 2024-04-23T14:39:09.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[64972]: starting fuse 2024-04-23T14:39:09.074 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:39:09.074 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:39:09.075 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:39:09.076 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:39:09.265 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.64972.asok 2024-04-23T14:39:09.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:39:09.283 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.64972.asok 2024-04-23T14:39:09.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:09.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.64972.asok status 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "64972", 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:39:09.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7878, 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 7878 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 274403673 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 274403673 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.7878 192.168.0.1:0/274403673", 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/274403673", 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 66, 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 85, 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:39:09.525 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:39:09.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:09.539 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:39:09.539 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:39:09.593 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:39:09.593 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:39:09.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:09.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:39:09.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:09.681 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-23T14:39:09.755 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:39:09.755+0000 7fedbc800640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-23T14:39:09.757 INFO:teuthology.orchestra.run.smithi059.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-23T14:39:09.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:39:09.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:09.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-23T14:39:09.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.976+0000 7f5f39600640 1 Processor -- start 2024-04-23T14:39:09.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.976+0000 7f5f39600640 1 -- start start 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.977+0000 7f5f39600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f340b3990 0x7f5f3409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.977+0000 7f5f39600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5f3409af40 0x7f5f340ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.977+0000 7f5f39600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f340b3df0 con 0x7f5f3415c8a0 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.977+0000 7f5f39600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3409b390 con 0x7f5f3409af40 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.977+0000 7f5f39600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5f340cd780 con 0x7f5f340b3990 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.978+0000 7f5f33400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5f3409af40 0x7f5f340ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.978+0000 7f5f33400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5f3409af40 0x7f5f340ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45294/0 (socket says 172.21.15.59:45294) 2024-04-23T14:39:09.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.978+0000 7f5f33400640 1 -- 172.21.15.59:0/3125312239 learned_addr learned my addr 172.21.15.59:0/3125312239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:09.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.978+0000 7f5f32a00640 1 --1- 172.21.15.59:0/3125312239 >> v1:172.21.15.59:6789/0 conn(0x7f5f3415c8a0 0x7f5f340ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50928/0 (socket says 172.21.15.59:50928) 2024-04-23T14:39:09.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.978+0000 7f5f32000640 1 --2- 172.21.15.59:0/3125312239 >> v2:172.21.15.76:3300/0 conn(0x7f5f340b3990 0x7f5f3409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.978+0000 7f5f33400640 1 -- 172.21.15.59:0/3125312239 >> v2:172.21.15.76:3300/0 conn(0x7f5f340b3990 msgr2=0x7f5f3409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:09.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.978+0000 7f5f33400640 1 --2- 172.21.15.59:0/3125312239 >> v2:172.21.15.76:3300/0 conn(0x7f5f340b3990 0x7f5f3409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.979+0000 7f5f33400640 1 -- 172.21.15.59:0/3125312239 >> v1:172.21.15.59:6789/0 conn(0x7f5f3415c8a0 legacy=0x7f5f340ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:09.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.979+0000 7f5f33400640 1 -- 172.21.15.59:0/3125312239 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f340cda60 con 0x7f5f3409af40 2024-04-23T14:39:09.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.979+0000 7f5f31600640 1 -- 172.21.15.59:0/3125312239 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398785128 0 0) 0x7f5f340b3df0 con 0x7f5f3415c8a0 2024-04-23T14:39:09.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.980+0000 7f5f33400640 1 --2- 172.21.15.59:0/3125312239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3409af40 0x7f5f340ce820 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5f340d1dd0 tx=0x7f5f2c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b54f940821cbde4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.980+0000 7f5f31600640 1 -- 172.21.15.59:0/3125312239 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c036070 con 0x7f5f3409af40 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.980+0000 7f5f31600640 1 -- 172.21.15.59:0/3125312239 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c031600 con 0x7f5f3409af40 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.980+0000 7f5f31600640 1 -- 172.21.15.59:0/3125312239 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0319d0 con 0x7f5f3409af40 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.980+0000 7f5f39600640 1 -- 172.21.15.59:0/3125312239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3409af40 msgr2=0x7f5f340ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.980+0000 7f5f39600640 1 --2- 172.21.15.59:0/3125312239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3409af40 0x7f5f340ce820 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5f340d1dd0 tx=0x7f5f2c02a730 comp rx=0 tx=0).stop 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.981+0000 7f5f39600640 1 -- 172.21.15.59:0/3125312239 shutdown_connections 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.981+0000 7f5f39600640 1 --2- 172.21.15.59:0/3125312239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3409af40 0x7f5f340ce820 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.981+0000 7f5f39600640 1 --2- 172.21.15.59:0/3125312239 >> v2:172.21.15.76:3300/0 conn(0x7f5f340b3990 0x7f5f3409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.981+0000 7f5f39600640 1 -- 172.21.15.59:0/3125312239 >> 172.21.15.59:0/3125312239 conn(0x7f5f340d2600 msgr2=0x7f5f340d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:09.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.981+0000 7f5f39600640 1 -- 172.21.15.59:0/3125312239 shutdown_connections 2024-04-23T14:39:09.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.981+0000 7f5f39600640 1 -- 172.21.15.59:0/3125312239 wait complete. 2024-04-23T14:39:09.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.982+0000 7f5f39600640 1 Processor -- start 2024-04-23T14:39:09.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.983+0000 7f5f39600640 1 -- start start 2024-04-23T14:39:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.983+0000 7f5f39600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f3409af40 0x7f5f340cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.983+0000 7f5f39600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f340b3990 0x7f5f3415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.983+0000 7f5f39600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3415c8a0 0x7f5f3414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f39600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340ce300 con 0x7f5f3415c8a0 2024-04-23T14:39:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f39600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340ce020 con 0x7f5f3409af40 2024-04-23T14:39:09.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f39600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3415cc80 con 0x7f5f340b3990 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3415c8a0 0x7f5f3414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3415c8a0 0x7f5f3414c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45306/0 (socket says 172.21.15.59:45306) 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 -- 172.21.15.59:0/2239524213 learned_addr learned my addr 172.21.15.59:0/2239524213 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f32a00640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f3409af40 0x7f5f340cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f32000640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f340b3990 0x7f5f3415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 -- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f340b3990 msgr2=0x7f5f3415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f340b3990 0x7f5f3415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 -- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f3409af40 msgr2=0x7f5f340cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f3409af40 0x7f5f340cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.984+0000 7f5f33400640 1 -- 172.21.15.59:0/2239524213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f340b5380 con 0x7f5f3415c8a0 2024-04-23T14:39:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.985+0000 7f5f33400640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3415c8a0 0x7f5f3414c610 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c02ad20 tx=0x7f5f2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.985+0000 7f5f2aa00640 1 -- 172.21.15.59:0/2239524213 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c03f020 con 0x7f5f3415c8a0 2024-04-23T14:39:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.985+0000 7f5f2aa00640 1 -- 172.21.15.59:0/2239524213 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c037040 con 0x7f5f3415c8a0 2024-04-23T14:39:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.985+0000 7f5f2aa00640 1 -- 172.21.15.59:0/2239524213 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c036070 con 0x7f5f3415c8a0 2024-04-23T14:39:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.985+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f340b5660 con 0x7f5f3415c8a0 2024-04-23T14:39:09.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.985+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f340b5c20 con 0x7f5f3415c8a0 2024-04-23T14:39:09.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.988+0000 7f5f2aa00640 1 -- 172.21.15.59:0/2239524213 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f2c042020 con 0x7f5f3415c8a0 2024-04-23T14:39:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.989+0000 7f5f2aa00640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f1006c8f0 0x7f5f1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.989+0000 7f5f2aa00640 1 -- 172.21.15.59:0/2239524213 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f5f2c0abfd0 con 0x7f5f3415c8a0 2024-04-23T14:39:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.989+0000 7f5f32a00640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f1006c8f0 0x7f5f1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.989+0000 7f5f3c37a740 1 Processor -- start 2024-04-23T14:39:09.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.989+0000 7f5f3c37a740 1 -- start start 2024-04-23T14:39:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f32a00640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f1006c8f0 0x7f5f1006edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000f60 tx=0x7f5f24007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3ab1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3ae18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae1df0 0x5650b3ab6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650b3a1af50 con 0x5650b3aff410 2024-04-23T14:39:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5650b39c12c0 con 0x5650b3ae2530 2024-04-23T14:39:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5650b3a16f10 con 0x5650b3ae1df0 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.990+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3ab1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3ab1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45314/0 (socket says 172.21.15.59:45314) 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 -- 172.21.15.59:0/3801997675 learned_addr learned my addr 172.21.15.59:0/3801997675 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32000640 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3ae18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f33400640 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae1df0 0x5650b3ab6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 -- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae1df0 msgr2=0x5650b3ab6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae1df0 0x5650b3ab6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 -- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 msgr2=0x5650b3ae18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3ae18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 -- 172.21.15.59:0/3801997675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b37f97b0 con 0x5650b3aff410 2024-04-23T14:39:09.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.991+0000 7f5f32a00640 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3ab1da0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005e10 tx=0x7f5f24012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b468ec83dac382 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:09.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f23000640 1 -- 172.21.15.59:0/3801997675 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24015110 con 0x5650b3aff410 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f23000640 1 -- 172.21.15.59:0/3801997675 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f240159c0 con 0x5650b3aff410 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f23000640 1 -- 172.21.15.59:0/3801997675 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24015d70 con 0x5650b3aff410 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f3c37a740 1 -- 172.21.15.59:0/3801997675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 msgr2=0x5650b3ab1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3ab1da0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005e10 tx=0x7f5f24012000 comp rx=0 tx=0).stop 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f3c37a740 1 -- 172.21.15.59:0/3801997675 shutdown_connections 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae1df0 0x5650b3ab6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3ae18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/3801997675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3ab1da0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.992+0000 7f5f3c37a740 1 -- 172.21.15.59:0/3801997675 >> 172.21.15.59:0/3801997675 conn(0x5650b3ab4010 msgr2=0x5650b3afc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.993+0000 7f5f3c37a740 1 -- 172.21.15.59:0/3801997675 shutdown_connections 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.993+0000 7f5f3c37a740 1 -- 172.21.15.59:0/3801997675 wait complete. 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.993+0000 7f5f3c37a740 1 -- start start 2024-04-23T14:39:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.993+0000 7f5f3c37a740 1 Processor -- start 2024-04-23T14:39:09.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.994+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2530 0x5650b3ae5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.994+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3aff410 0x5650b3afe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.994+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae2be0 0x5650b3ae3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.994+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650b3ab61c0 con 0x5650b3ae2530 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.994+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5650b39d4680 con 0x5650b3aff410 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.994+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5650b3ab55b0 con 0x5650b3ae2be0 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f3c37a740 10 client.0 _ll_register_callbacks cb 0x5650b3a114f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2530 0x5650b3ae5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2530 0x5650b3ae5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45328/0 (socket says 172.21.15.59:45328) 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 -- 172.21.15.59:0/339945912 learned_addr learned my addr 172.21.15.59:0/339945912 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32000640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3aff410 0x5650b3afe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f33400640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae2be0 0x5650b3ae3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:09.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 -- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae2be0 msgr2=0x5650b3ae3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae2be0 0x5650b3ae3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 -- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3aff410 msgr2=0x5650b3afe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3aff410 0x5650b3afe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32a00640 1 -- 172.21.15.59:0/339945912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b3859f40 con 0x5650b3ae2530 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.995+0000 7f5f32000640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3aff410 0x5650b3afe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f32a00640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2530 0x5650b3ae5330 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f5f24007400 tx=0x7f5f24000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945370eb92e98a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f1b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f1b400640 1 -- 172.21.15.59:0/339945912 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24010040 con 0x5650b3ae2530 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f1b400640 1 -- 172.21.15.59:0/339945912 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f5f2402cb60 con 0x5650b3ae2530 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f240159c0 con 0x5650b3ae2530 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f1b400640 1 -- 172.21.15.59:0/339945912 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5f2402cd60 con 0x5650b3ae2530 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f3c37a740 10 client.7893 fetch_fsmap learned FSMap version 66 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.996+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5650b36ed0a0 con 0x5650b3ae2530 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f1b400640 1 -- 172.21.15.59:0/339945912 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f240159c0 con 0x5650b3ae2530 2024-04-23T14:39:09.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f1b400640 1 -- 172.21.15.59:0/339945912 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f5f24036080 con 0x5650b3ae2530 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f3c37a740 10 client.7893 fetch_fsmap finished waiting for FSMap version 66 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f3c37a740 10 client.7893 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f3c37a740 20 client.7893 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3af4e50 0x5650b3b102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f3c37a740 4 client.7893 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b3a28970 con 0x5650b3af4e50 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.997+0000 7f5f32000640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3af4e50 0x5650b3b102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.998+0000 7f5f32000640 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3af4e50 0x5650b3b102a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1ca2889173761e server_cookie=6861b512abe88229 in_seq=0 out_seq=0 2024-04-23T14:39:10.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.998+0000 7f5f1b400640 10 client.7893 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.998+0000 7f5f1b400640 1 -- 172.21.15.59:0/339945912 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650b3a28970 con 0x5650b3af4e50 2024-04-23T14:39:10.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.998+0000 7f5f1b400640 10 client.7893 handle_command_reply: tid=0 2024-04-23T14:39:10.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 client.7893 shutdown 2024-04-23T14:39:10.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2530 msgr2=0x5650b3ae5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2530 0x5650b3ae5330 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f5f24007400 tx=0x7f5f24000ed0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 shutdown_connections 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3af4e50 0x5650b3b102a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3ae2be0 0x5650b3ae3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3aff410 0x5650b3afe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/339945912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2530 0x5650b3ae5330 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f1b400640 0 client.7893 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 >> 172.21.15.59:0/339945912 conn(0x5650b3ae1df0 msgr2=0x5650b3ab5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 shutdown_connections 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 1 -- 172.21.15.59:0/339945912 wait complete. 2024-04-23T14:39:10.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:09.999+0000 7f5f3c37a740 20 client.7893 trim_cache size 0 max 16384 2024-04-23T14:39:10.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.019+0000 7f5f3c37a740 1 Processor -- start 2024-04-23T14:39:10.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.019+0000 7f5f3c37a740 1 -- start start 2024-04-23T14:39:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.020+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3afc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.020+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2be0 0x5650b3afcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.020+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3aff410 0x5650b3ab64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.020+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650b3af4800 con 0x5650b3ae2be0 2024-04-23T14:39:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.020+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34200e30 con 0x5650b3ae2530 2024-04-23T14:39:10.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.020+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5f341ffe50 con 0x5650b3aff410 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2be0 0x5650b3afcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2be0 0x5650b3afcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35232/0 (socket says 172.21.15.59:35232) 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 -- 172.21.15.59:0/1261899625 learned_addr learned my addr 172.21.15.59:0/1261899625 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32a00640 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3afc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f33400640 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3aff410 0x5650b3ab64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 -- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3aff410 msgr2=0x5650b3ab64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3aff410 0x5650b3ab64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 -- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 msgr2=0x5650b3afc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3afc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.021+0000 7f5f32000640 1 -- 172.21.15.59:0/1261899625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b3ab22d0 con 0x5650b3ae2be0 2024-04-23T14:39:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.022+0000 7f5f32000640 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2be0 0x5650b3afcbf0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x5650b3af54c0 tx=0x7f5f1c00cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c060e50db13b7439 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.022+0000 7f5f22600640 1 -- 172.21.15.59:0/1261899625 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00f040 con 0x5650b3ae2be0 2024-04-23T14:39:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.022+0000 7f5f22600640 1 -- 172.21.15.59:0/1261899625 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c01f030 con 0x5650b3ae2be0 2024-04-23T14:39:10.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.022+0000 7f5f22600640 1 -- 172.21.15.59:0/1261899625 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c01b150 con 0x5650b3ae2be0 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.022+0000 7f5f3c37a740 1 -- 172.21.15.59:0/1261899625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2be0 msgr2=0x5650b3afcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.022+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2be0 0x5650b3afcbf0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x5650b3af54c0 tx=0x7f5f1c00cfc0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 -- 172.21.15.59:0/1261899625 shutdown_connections 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3aff410 0x5650b3ab64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3ae2be0 0x5650b3afcbf0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/1261899625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3afc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 -- 172.21.15.59:0/1261899625 >> 172.21.15.59:0/1261899625 conn(0x5650b3ae1df0 msgr2=0x5650b3b0cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 -- 172.21.15.59:0/1261899625 shutdown_connections 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 -- 172.21.15.59:0/1261899625 wait complete. 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.023+0000 7f5f3c37a740 1 -- start start 2024-04-23T14:39:10.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.024+0000 7f5f3c37a740 1 Processor -- start 2024-04-23T14:39:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3afbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3b09900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f3c37a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3b09e40 0x5650b3ade2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3415e9d0 con 0x5650b3aff410 2024-04-23T14:39:10.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340cb2d0 con 0x5650b3ae2530 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f3c37a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5650b3afc940 con 0x5650b3b09e40 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f3c37a740 10 client.0 _ll_register_callbacks cb 0x7f5f24005c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3b09900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3b09900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35234/0 (socket says 172.21.15.59:35234) 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 -- 172.21.15.59:0/412928204 learned_addr learned my addr 172.21.15.59:0/412928204 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 -- 172.21.15.59:0/412928204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3b09e40 msgr2=0x5650b3ade2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3b09e40 0x5650b3ade2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 -- 172.21.15.59:0/412928204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 msgr2=0x5650b3afbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3afbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.025+0000 7f5f32000640 1 -- 172.21.15.59:0/412928204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b3b09140 con 0x5650b3aff410 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f32000640 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3b09900 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c006fd0 tx=0x7f5f1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5336e78f2903fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f1b400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f1b400640 1 -- 172.21.15.59:0/412928204 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004240 con 0x5650b3aff410 2024-04-23T14:39:10.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f1b400640 1 -- 172.21.15.59:0/412928204 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f5f1c0043e0 con 0x5650b3aff410 2024-04-23T14:39:10.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c01f030 con 0x5650b3aff410 2024-04-23T14:39:10.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f1b400640 1 -- 172.21.15.59:0/412928204 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c01b150 con 0x5650b3aff410 2024-04-23T14:39:10.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f3c37a740 10 client.7899 fetch_fsmap learned FSMap version 66 2024-04-23T14:39:10.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.026+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5650b3afc680 con 0x5650b3aff410 2024-04-23T14:39:10.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f1b400640 1 -- 172.21.15.59:0/412928204 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c01f030 con 0x5650b3aff410 2024-04-23T14:39:10.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f1b400640 1 -- 172.21.15.59:0/412928204 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f5f1c01b2f0 con 0x5650b3aff410 2024-04-23T14:39:10.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f3c37a740 10 client.7899 fetch_fsmap finished waiting for FSMap version 66 2024-04-23T14:39:10.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f3c37a740 10 client.7899 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f3c37a740 20 client.7899 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3b76810 0x5650b3ade7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f3c37a740 4 client.7899 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x5650b3a227e0 con 0x5650b3b76810 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.027+0000 7f5f32a00640 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3b76810 0x5650b3ade7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.028+0000 7f5f32a00640 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3b76810 0x5650b3ade7f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20df22f3af6359e7 server_cookie=815eab3e57008a2a in_seq=0 out_seq=0 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.028+0000 7f5f1b400640 10 client.7899 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.028+0000 7f5f1b400640 1 -- 172.21.15.59:0/412928204 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5650b3a227e0 con 0x5650b3b76810 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.028+0000 7f5f1b400640 10 client.7899 handle_command_reply: tid=0 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.028+0000 7f5f3c37a740 1 client.7899 shutdown 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.028+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 msgr2=0x5650b3b09900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.028+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3b09900 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c006fd0 tx=0x7f5f1c004730 comp rx=0 tx=0).stop 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 shutdown_connections 2024-04-23T14:39:10.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5650b3b76810 0x5650b3ade7f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b3b09e40 0x5650b3ade2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b3aff410 0x5650b3b09900 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 --2- 172.21.15.59:0/412928204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b3ae2530 0x5650b3afbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 >> 172.21.15.59:0/412928204 conn(0x5650b3ae1df0 msgr2=0x5650b3b16a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f1b400640 0 client.7899 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.031 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:39:10.029+0000 7fedbc800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0) n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55a610a96100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 shutdown_connections 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 1 -- 172.21.15.59:0/412928204 wait complete. 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f3c37a740 20 client.7899 trim_cache size 0 max 16384 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "20e6f503-c0fb-489e-ba32-c9ca677704a0", 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:39:10.032 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:39:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.029+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f1006c8f0 msgr2=0x7f5f1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f1006c8f0 0x7f5f1006edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000f60 tx=0x7f5f24007c00 comp rx=0 tx=0).stop 2024-04-23T14:39:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3415c8a0 msgr2=0x7f5f3414c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3415c8a0 0x7f5f3414c610 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c02ad20 tx=0x7f5f2c004a40 comp rx=0 tx=0).stop 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 shutdown_connections 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5f1006c8f0 0x7f5f1006edb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f340b3990 0x7f5f3415f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f3415c8a0 0x7f5f3414c610 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 --2- 172.21.15.59:0/2239524213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f3409af40 0x7f5f340cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 >> 172.21.15.59:0/2239524213 conn(0x7f5f340d2600 msgr2=0x7f5f340ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 shutdown_connections 2024-04-23T14:39:10.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.030+0000 7f5f39600640 1 -- 172.21.15.59:0/2239524213 wait complete. 2024-04-23T14:39:10.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:10.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-23T14:39:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.247+0000 7f2c35a00640 1 Processor -- start 2024-04-23T14:39:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.248+0000 7f2c35a00640 1 -- start start 2024-04-23T14:39:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.249+0000 7f2c35a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3015ed20 0x7f2c300d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.249+0000 7f2c35a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c300d13e0 0x7f2c300d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.249+0000 7f2c35a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c300b5da0 con 0x7f2c300b3600 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.249+0000 7f2c35a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3015f100 con 0x7f2c300d13e0 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.249+0000 7f2c35a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2c300d07d0 con 0x7f2c3015ed20 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.249+0000 7f2c35000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c300d13e0 0x7f2c300d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.250+0000 7f2c35000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c300d13e0 0x7f2c300d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35240/0 (socket says 172.21.15.59:35240) 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.250+0000 7f2c35000640 1 -- 172.21.15.59:0/2017872017 learned_addr learned my addr 172.21.15.59:0/2017872017 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.250+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/2017872017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3015ed20 0x7f2c300d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.250+0000 7f2c35000640 1 -- 172.21.15.59:0/2017872017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3015ed20 msgr2=0x7f2c300d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.250+0000 7f2c35000640 1 --2- 172.21.15.59:0/2017872017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3015ed20 0x7f2c300d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.250+0000 7f2c35000640 1 -- 172.21.15.59:0/2017872017 >> v1:172.21.15.76:6789/0 conn(0x7f2c300b3600 legacy=0x7f2c300b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.250+0000 7f2c35000640 1 -- 172.21.15.59:0/2017872017 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c300d0a20 con 0x7f2c300d13e0 2024-04-23T14:39:10.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.251+0000 7f2c35000640 1 --2- 172.21.15.59:0/2017872017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c300d13e0 0x7f2c300d0290 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009870 tx=0x7f2c2802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e5c22cb4c8a269 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c2ea00640 1 -- 172.21.15.59:0/2017872017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28037070 con 0x7f2c300d13e0 2024-04-23T14:39:10.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c2ea00640 1 -- 172.21.15.59:0/2017872017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2802ae10 con 0x7f2c300d13e0 2024-04-23T14:39:10.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c2ea00640 1 -- 172.21.15.59:0/2017872017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28032660 con 0x7f2c300d13e0 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c35a00640 1 -- 172.21.15.59:0/2017872017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c300d13e0 msgr2=0x7f2c300d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c35a00640 1 --2- 172.21.15.59:0/2017872017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c300d13e0 0x7f2c300d0290 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009870 tx=0x7f2c2802a100 comp rx=0 tx=0).stop 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c35a00640 1 -- 172.21.15.59:0/2017872017 shutdown_connections 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c35a00640 1 --2- 172.21.15.59:0/2017872017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c300d13e0 0x7f2c300d0290 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c35a00640 1 --2- 172.21.15.59:0/2017872017 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3015ed20 0x7f2c300d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c35a00640 1 -- 172.21.15.59:0/2017872017 >> 172.21.15.59:0/2017872017 conn(0x7f2c3008bf90 msgr2=0x7f2c300d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.252+0000 7f2c35a00640 1 -- 172.21.15.59:0/2017872017 shutdown_connections 2024-04-23T14:39:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.253+0000 7f2c35a00640 1 -- 172.21.15.59:0/2017872017 wait complete. 2024-04-23T14:39:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.254+0000 7f2c35a00640 1 Processor -- start 2024-04-23T14:39:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.254+0000 7f2c35a00640 1 -- start start 2024-04-23T14:39:10.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.255+0000 7f2c35a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c300b3600 0x7f2c300cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.255+0000 7f2c35a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c300d13e0 0x7f2c300ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.255+0000 7f2c35a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3015ed20 0x7f2c3009e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.255+0000 7f2c35a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c300d3490 con 0x7f2c3015ed20 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.255+0000 7f2c35a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c300d31b0 con 0x7f2c300d13e0 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.255+0000 7f2c35a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2c300b4620 con 0x7f2c300b3600 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3015ed20 0x7f2c3009e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3015ed20 0x7f2c3009e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35244/0 (socket says 172.21.15.59:35244) 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 -- 172.21.15.59:0/567771734 learned_addr learned my addr 172.21.15.59:0/567771734 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c2f400640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c300d13e0 0x7f2c300ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c300b3600 0x7f2c300cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 -- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c300b3600 msgr2=0x7f2c300cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c300b3600 0x7f2c300cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 -- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c300d13e0 msgr2=0x7f2c300ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c300d13e0 0x7f2c300ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c35000640 1 -- 172.21.15.59:0/567771734 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c300b92d0 con 0x7f2c3015ed20 2024-04-23T14:39:10.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.256+0000 7f2c2f400640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c300d13e0 0x7f2c300ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:39:10.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.257+0000 7f2c35000640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3015ed20 0x7f2c3009e310 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2c2802a140 tx=0x7f2c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.257+0000 7f2c27e00640 1 -- 172.21.15.59:0/567771734 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28041020 con 0x7f2c3015ed20 2024-04-23T14:39:10.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.257+0000 7f2c27e00640 1 -- 172.21.15.59:0/567771734 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2802ac10 con 0x7f2c3015ed20 2024-04-23T14:39:10.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.257+0000 7f2c27e00640 1 -- 172.21.15.59:0/567771734 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28037040 con 0x7f2c3015ed20 2024-04-23T14:39:10.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.257+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c300b95b0 con 0x7f2c3015ed20 2024-04-23T14:39:10.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.257+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c300b9b70 con 0x7f2c3015ed20 2024-04-23T14:39:10.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.260+0000 7f2c27e00640 1 -- 172.21.15.59:0/567771734 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c28044020 con 0x7f2c3015ed20 2024-04-23T14:39:10.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.261+0000 7f2c27e00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.262+0000 7f2c27e00640 1 -- 172.21.15.59:0/567771734 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f2c280ac090 con 0x7f2c3015ed20 2024-04-23T14:39:10.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.262+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.262+0000 7f2c39d78740 1 Processor -- start 2024-04-23T14:39:10.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.262+0000 7f2c39d78740 1 -- start start 2024-04-23T14:39:10.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2c20006fd0 tx=0x7f2c20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1f92790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fc22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc27e0 0x5635f1f975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c39d78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5635f1efc1f0 con 0x5635f1fc2f20 2024-04-23T14:39:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5635f1ea2350 con 0x5635f1fdfe00 2024-04-23T14:39:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5635f1ef81b0 con 0x5635f1fc27e0 2024-04-23T14:39:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.263+0000 7f2c2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fc22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fc22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35250/0 (socket says 172.21.15.59:35250) 2024-04-23T14:39:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 -- 172.21.15.59:0/3832849530 learned_addr learned my addr 172.21.15.59:0/3832849530 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1f92790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c35000640 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc27e0 0x5635f1f975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 -- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc27e0 msgr2=0x5635f1f975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc27e0 0x5635f1f975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 -- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 msgr2=0x5635f1f92790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1f92790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 -- 172.21.15.59:0/3832849530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f1ef6f50 con 0x5635f1fc2f20 2024-04-23T14:39:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.264+0000 7f2c2f400640 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fc22a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2c18004560 tx=0x7f2c1800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecfbcdbb5b29372 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.265+0000 7f2c24c00640 1 -- 172.21.15.59:0/3832849530 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800ec40 con 0x5635f1fc2f20 2024-04-23T14:39:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.265+0000 7f2c24c00640 1 -- 172.21.15.59:0/3832849530 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1800ede0 con 0x5635f1fc2f20 2024-04-23T14:39:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.265+0000 7f2c24c00640 1 -- 172.21.15.59:0/3832849530 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18004980 con 0x5635f1fc2f20 2024-04-23T14:39:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.265+0000 7f2c39d78740 1 -- 172.21.15.59:0/3832849530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 msgr2=0x5635f1fc22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.265+0000 7f2c39d78740 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fc22a0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2c18004560 tx=0x7f2c1800e530 comp rx=0 tx=0).stop 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 -- 172.21.15.59:0/3832849530 shutdown_connections 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc27e0 0x5635f1f975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fc22a0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 --2- 172.21.15.59:0/3832849530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1f92790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 -- 172.21.15.59:0/3832849530 >> 172.21.15.59:0/3832849530 conn(0x5635f1f94a00 msgr2=0x5635f1fdd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 -- 172.21.15.59:0/3832849530 shutdown_connections 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 -- 172.21.15.59:0/3832849530 wait complete. 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.266+0000 7f2c39d78740 1 -- start start 2024-04-23T14:39:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.267+0000 7f2c39d78740 1 Processor -- start 2024-04-23T14:39:10.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fd4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fc4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc52a0 0x5635f1fed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c39d78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5635f1f96bb0 con 0x5635f1fc2f20 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5635f1eb5690 con 0x5635f1fdfe00 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5635f1f95fa0 con 0x5635f1fc52a0 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c39d78740 10 client.0 _ll_register_callbacks cb 0x5635f1ef1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fd4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fd4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35258/0 (socket says 172.21.15.59:35258) 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 -- 172.21.15.59:0/2186804912 learned_addr learned my addr 172.21.15.59:0/2186804912 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2f400640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fc4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c35000640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc52a0 0x5635f1fed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 -- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc52a0 msgr2=0x5635f1fed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc52a0 0x5635f1fed590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 -- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 msgr2=0x5635f1fc4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fc4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2fe00640 1 -- 172.21.15.59:0/2186804912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f1eb4920 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.268+0000 7f2c2f400640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fc4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fd4b50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2c20014740 tx=0x7f2c20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c7df74df20814c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c1d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c1d600640 1 -- 172.21.15.59:0/2186804912 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20025020 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c1d600640 1 -- 172.21.15.59:0/2186804912 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f2c200158e0 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c1800ede0 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c1d600640 1 -- 172.21.15.59:0/2186804912 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2c2002f050 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c39d78740 10 client.7911 fetch_fsmap learned FSMap version 66 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.269+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5635f1cd87d0 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c1d600640 1 -- 172.21.15.59:0/2186804912 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1800ede0 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c1d600640 1 -- 172.21.15.59:0/2186804912 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f2c20032020 con 0x5635f1fc2f20 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c39d78740 10 client.7911 fetch_fsmap finished waiting for FSMap version 66 2024-04-23T14:39:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c39d78740 10 client.7911 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:10.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c39d78740 20 client.7911 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:10.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c39d78740 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fdeb30 0x5635f1fc3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c39d78740 4 client.7911 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:39:10.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635f1f09c10 con 0x5635f1fdeb30 2024-04-23T14:39:10.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.270+0000 7f2c2f400640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fdeb30 0x5635f1fc3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.271+0000 7f2c2f400640 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fdeb30 0x5635f1fc3410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb521ad7297ad621 server_cookie=61e250ff9adfd501 in_seq=0 out_seq=0 2024-04-23T14:39:10.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.271+0000 7f2c1d600640 10 client.7911 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c1d600640 1 -- 172.21.15.59:0/2186804912 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635f1f09c10 con 0x5635f1fdeb30 2024-04-23T14:39:10.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c1d600640 10 client.7911 handle_command_reply: tid=0 2024-04-23T14:39:10.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 client.7911 shutdown 2024-04-23T14:39:10.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 msgr2=0x5635f1fd4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fd4b50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f2c20014740 tx=0x7f2c20004730 comp rx=0 tx=0).stop 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 shutdown_connections 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fdeb30 0x5635f1fc3410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc52a0 0x5635f1fed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fc4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 --2- 172.21.15.59:0/2186804912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc2f20 0x5635f1fd4b50 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 >> 172.21.15.59:0/2186804912 conn(0x5635f1fc27e0 msgr2=0x5635f1fdd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c1d600640 0 client.7911 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 shutdown_connections 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 1 -- 172.21.15.59:0/2186804912 wait complete. 2024-04-23T14:39:10.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.272+0000 7f2c39d78740 20 client.7911 trim_cache size 0 max 16384 2024-04-23T14:39:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.292+0000 7f2c39d78740 1 Processor -- start 2024-04-23T14:39:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.292+0000 7f2c39d78740 1 -- start start 2024-04-23T14:39:10.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fd4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc5ea0 0x5635f1fd5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c39d78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c30200ff0 con 0x5635f1fc5ea0 2024-04-23T14:39:10.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301f8cc0 con 0x5635f1fdfe00 2024-04-23T14:39:10.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2c301f8020 con 0x5635f1fc2f20 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c35000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc5ea0 0x5635f1fd5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c35000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc5ea0 0x5635f1fd5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35272/0 (socket says 172.21.15.59:35272) 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c35000640 1 -- 172.21.15.59:0/1140242471 learned_addr learned my addr 172.21.15.59:0/1140242471 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c2f400640 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.293+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fd4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c35000640 1 -- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 msgr2=0x5635f1fd4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c35000640 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fd4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c35000640 1 -- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 msgr2=0x5635f1fd52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c35000640 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c35000640 1 -- 172.21.15.59:0/1140242471 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f1c0e3f0 con 0x5635f1fc5ea0 2024-04-23T14:39:10.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c35000640 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc5ea0 0x5635f1fd5810 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2c280aa4d0 tx=0x7f2c2802a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf7d578bde31ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c1fe00640 1 -- 172.21.15.59:0/1140242471 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28092020 con 0x5635f1fc5ea0 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c1fe00640 1 -- 172.21.15.59:0/1140242471 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c28079d60 con 0x5635f1fc5ea0 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.294+0000 7f2c1fe00640 1 -- 172.21.15.59:0/1140242471 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2807e110 con 0x5635f1fc5ea0 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 -- 172.21.15.59:0/1140242471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc5ea0 msgr2=0x5635f1fd5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc5ea0 0x5635f1fd5810 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2c280aa4d0 tx=0x7f2c2802a6b0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 -- 172.21.15.59:0/1140242471 shutdown_connections 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fc5ea0 0x5635f1fd5810 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 --2- 172.21.15.59:0/1140242471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fd4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 -- 172.21.15.59:0/1140242471 >> 172.21.15.59:0/1140242471 conn(0x5635f1fc27e0 msgr2=0x5635f1f955a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 -- 172.21.15.59:0/1140242471 shutdown_connections 2024-04-23T14:39:10.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.295+0000 7f2c39d78740 1 -- 172.21.15.59:0/1140242471 wait complete. 2024-04-23T14:39:10.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.296+0000 7f2c39d78740 1 -- start start 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.296+0000 7f2c39d78740 1 Processor -- start 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.297+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fc3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.297+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.297+0000 7f2c39d78740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fdd000 0x5635f203a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.297+0000 7f2c39d78740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5635f1fd6c40 con 0x5635f1fdd000 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.297+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301f8670 con 0x5635f1fdfe00 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.297+0000 7f2c39d78740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5635f1fdc380 con 0x5635f1fc2f20 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.297+0000 7f2c39d78740 10 client.0 _ll_register_callbacks cb 0x7f2c2000be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fdd000 0x5635f203a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fdd000 0x5635f203a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35276/0 (socket says 172.21.15.59:35276) 2024-04-23T14:39:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 -- 172.21.15.59:0/267066671 learned_addr learned my addr 172.21.15.59:0/267066671 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c2f400640 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fc3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 -- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 msgr2=0x5635f1fc3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fc3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 -- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 msgr2=0x5635f1fd3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 -- 172.21.15.59:0/267066671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f1bce0f0 con 0x5635f1fdd000 2024-04-23T14:39:10.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c35000640 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fdd000 0x5635f203a080 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2c280761c0 tx=0x7f2c2803f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c52ccda8ecc1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:10.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.298+0000 7f2c1d600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:10.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c1d600640 1 -- 172.21.15.59:0/267066671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28096030 con 0x5635f1fdd000 2024-04-23T14:39:10.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c1d600640 1 -- 172.21.15.59:0/267066671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f2c28077e00 con 0x5635f1fdd000 2024-04-23T14:39:10.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c28079d60 con 0x5635f1fdd000 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c1d600640 1 -- 172.21.15.59:0/267066671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2c280a1050 con 0x5635f1fdd000 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 10 client.7917 fetch_fsmap learned FSMap version 66 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5635f1ec1810 con 0x5635f1fdd000 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c1d600640 1 -- 172.21.15.59:0/267066671 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c28079d60 con 0x5635f1fdd000 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c1d600640 1 -- 172.21.15.59:0/267066671 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f2c280824b0 con 0x5635f1fdd000 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 10 client.7917 fetch_fsmap finished waiting for FSMap version 66 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 10 client.7917 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 20 client.7917 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fd5220 0x5635f1fd5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 4 client.7917 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.299+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635f1f03a80 con 0x5635f1fd5220 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.300+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fd5220 0x5635f1fd5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.300+0000 7f2c2fe00640 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fd5220 0x5635f1fd5630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7cb451ea537e3f server_cookie=79a8352e3dd7effa in_seq=0 out_seq=0 2024-04-23T14:39:10.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.300+0000 7f2c1d600640 10 client.7917 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.300+0000 7f2c1d600640 1 -- 172.21.15.59:0/267066671 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x5635f1f03a80 con 0x5635f1fd5220 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.300+0000 7f2c1d600640 10 client.7917 handle_command_reply: tid=0 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 client.7917 shutdown 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fdd000 msgr2=0x5635f203a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fdd000 0x5635f203a080 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2c280761c0 tx=0x7f2c2803f070 comp rx=0 tx=0).stop 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 shutdown_connections 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5635f1fd5220 0x5635f1fd5630 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635f1fdd000 0x5635f203a080 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635f1fdfe00 0x5635f1fd3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c1d600640 0 client.7917 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 --2- 172.21.15.59:0/267066671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5635f1fc2f20 0x5635f1fc3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 >> 172.21.15.59:0/267066671 conn(0x5635f1fc27e0 msgr2=0x5635f1f955a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 shutdown_connections 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 1 -- 172.21.15.59:0/267066671 wait complete. 2024-04-23T14:39:10.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.301+0000 7f2c39d78740 20 client.7917 trim_cache size 0 max 16384 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: "20e6f503-c0fb-489e-ba32-c9ca677704a0": { 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir", 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "20e6f503-c0fb-489e-ba32-c9ca677704a0", 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive" 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:39:10.304 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:39:10.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c0c06c8f0 msgr2=0x7f2c0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2c20006fd0 tx=0x7f2c20008040 comp rx=0 tx=0).stop 2024-04-23T14:39:10.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3015ed20 msgr2=0x7f2c3009e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3015ed20 0x7f2c3009e310 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2c2802a140 tx=0x7f2c280046c0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 shutdown_connections 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c0c06c8f0 0x7f2c0c06edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3015ed20 0x7f2c3009e310 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c300d13e0 0x7f2c300ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 --2- 172.21.15.59:0/567771734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c300b3600 0x7f2c300cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 >> 172.21.15.59:0/567771734 conn(0x7f2c3008bf90 msgr2=0x7f2c3006cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 shutdown_connections 2024-04-23T14:39:10.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:10.302+0000 7f2c35a00640 1 -- 172.21.15.59:0/567771734 wait complete. 2024-04-23T14:39:10.319 INFO:tasks.cephfs.filesystem:scrub status for tag:20e6f503-c0fb-489e-ba32-c9ca677704a0 - {'path': '/dir', 'tag': '20e6f503-c0fb-489e-ba32-c9ca677704a0', 'options': 'recursive'} 2024-04-23T14:39:16.549 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:39:16.547+0000 7f3222800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-23T14:39:20.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:39:20.678 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:39:40.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:40.320 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-23T14:39:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.542+0000 7f86a9000640 1 Processor -- start 2024-04-23T14:39:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.542+0000 7f86a9000640 1 -- start start 2024-04-23T14:39:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.544+0000 7f86a9000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f86a4072c80 0x7f86a40af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.544+0000 7f86a9000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a4160120 con 0x7f86a40d0ce0 2024-04-23T14:39:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.544+0000 7f86a9000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a4160400 con 0x7f86a415c410 2024-04-23T14:39:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.544+0000 7f86a9000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f86a41606e0 con 0x7f86a4072c80 2024-04-23T14:39:40.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.544+0000 7f869be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f86a40d0ce0 0x7f86a40af080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44444/0 (socket says 172.21.15.59:44444) 2024-04-23T14:39:40.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.544+0000 7f869be00640 1 -- 172.21.15.59:0/3298147894 learned_addr learned my addr 172.21.15.59:0/3298147894 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:40.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.544+0000 7f86a3e00640 1 --2- 172.21.15.59:0/3298147894 >> v2:172.21.15.59:3300/0 conn(0x7f86a4072c80 0x7f86a40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.545+0000 7f86a2a00640 1 -- 172.21.15.59:0/3298147894 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708089071 0 0) 0x7f86a4160120 con 0x7f86a40d0ce0 2024-04-23T14:39:40.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.545+0000 7f86a2a00640 1 -- 172.21.15.59:0/3298147894 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8688003520 con 0x7f86a40d0ce0 2024-04-23T14:39:40.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.545+0000 7f86a3e00640 1 -- 172.21.15.59:0/3298147894 >> v1:172.21.15.76:6789/0 conn(0x7f86a415c410 legacy=0x7f86a40d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.545+0000 7f86a3e00640 1 -- 172.21.15.59:0/3298147894 >> v1:172.21.15.59:6789/0 conn(0x7f86a40d0ce0 legacy=0x7f86a40af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.545+0000 7f86a3e00640 1 -- 172.21.15.59:0/3298147894 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a41609c0 con 0x7f86a4072c80 2024-04-23T14:39:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.546+0000 7f86a3e00640 1 --2- 172.21.15.59:0/3298147894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a40af790 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f869c009d90 tx=0x7f869c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1e792f04d36736 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.546+0000 7f86a2a00640 1 -- 172.21.15.59:0/3298147894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c007740 con 0x7f86a4072c80 2024-04-23T14:39:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.546+0000 7f86a2a00640 1 -- 172.21.15.59:0/3298147894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f869c0078e0 con 0x7f86a4072c80 2024-04-23T14:39:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a2a00640 1 -- 172.21.15.59:0/3298147894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c007cb0 con 0x7f86a4072c80 2024-04-23T14:39:40.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a9000640 1 -- 172.21.15.59:0/3298147894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 msgr2=0x7f86a40af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a9000640 1 --2- 172.21.15.59:0/3298147894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a40af790 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f869c009d90 tx=0x7f869c02a6f0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a9000640 1 -- 172.21.15.59:0/3298147894 shutdown_connections 2024-04-23T14:39:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a9000640 1 --2- 172.21.15.59:0/3298147894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a40af790 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a9000640 1 -- 172.21.15.59:0/3298147894 >> 172.21.15.59:0/3298147894 conn(0x7f86a406bdf0 msgr2=0x7f86a406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a9000640 1 -- 172.21.15.59:0/3298147894 shutdown_connections 2024-04-23T14:39:40.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.547+0000 7f86a9000640 1 -- 172.21.15.59:0/3298147894 wait complete. 2024-04-23T14:39:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.549+0000 7f86a9000640 1 Processor -- start 2024-04-23T14:39:40.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.549+0000 7f86a9000640 1 -- start start 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a4098ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a9000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a40d0ce0 0x7f86a415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a9000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a415c410 0x7f86a4151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a9000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4167590 con 0x7f86a4072c80 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a9000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4167410 con 0x7f86a415c410 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a9000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f86a4167710 con 0x7f86a40d0ce0 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a4098ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a4098ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41454/0 (socket says 172.21.15.59:41454) 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.550+0000 7f86a3400640 1 -- 172.21.15.59:0/1725509640 learned_addr learned my addr 172.21.15.59:0/1725509640 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f86a3e00640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a415c410 0x7f86a4151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f86a3400640 1 -- 172.21.15.59:0/1725509640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a40d0ce0 msgr2=0x7f86a415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f86a3400640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a40d0ce0 0x7f86a415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f86a3400640 1 -- 172.21.15.59:0/1725509640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a415c410 msgr2=0x7f86a4151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f86a3400640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a415c410 0x7f86a4151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f86a3400640 1 -- 172.21.15.59:0/1725509640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a40b3f60 con 0x7f86a4072c80 2024-04-23T14:39:40.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f86a3400640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a4098ad0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f869400c3f0 tx=0x7f869400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f869b400640 1 -- 172.21.15.59:0/1725509640 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86940052d0 con 0x7f86a4072c80 2024-04-23T14:39:40.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f869b400640 1 -- 172.21.15.59:0/1725509640 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8694005490 con 0x7f86a4072c80 2024-04-23T14:39:40.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.551+0000 7f869b400640 1 -- 172.21.15.59:0/1725509640 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694007260 con 0x7f86a4072c80 2024-04-23T14:39:40.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.552+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a40b0810 con 0x7f86a4072c80 2024-04-23T14:39:40.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.552+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a40b0dd0 con 0x7f86a4072c80 2024-04-23T14:39:40.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.554+0000 7f869b400640 1 -- 172.21.15.59:0/1725509640 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8694007400 con 0x7f86a4072c80 2024-04-23T14:39:40.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.555+0000 7f869b400640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f868006c8f0 0x7f868006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.555+0000 7f869b400640 1 -- 172.21.15.59:0/1725509640 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f869408b100 con 0x7f86a4072c80 2024-04-23T14:39:40.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.556+0000 7f869be00640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f868006c8f0 0x7f868006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.556+0000 7f86ad394740 1 Processor -- start 2024-04-23T14:39:40.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.556+0000 7f86ad394740 1 -- start start 2024-04-23T14:39:40.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.556+0000 7f869be00640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f868006c8f0 0x7f868006edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f868c000f60 tx=0x7f868c0078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86ad394740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x5630511a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d8f20 0x5630511d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d87e0 0x5630511ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86ad394740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5630511121f0 con 0x5630511f5e00 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630510b8350 con 0x5630511d87e0 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x56305110e1b0 con 0x5630511d8f20 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x5630511a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x5630511a8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41464/0 (socket says 172.21.15.59:41464) 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.557+0000 7f86a3400640 1 -- 172.21.15.59:0/759797614 learned_addr learned my addr 172.21.15.59:0/759797614 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f86a3400640 1 -- 172.21.15.59:0/759797614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d8f20 msgr2=0x5630511d82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f86a3400640 1 --2- 172.21.15.59:0/759797614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d8f20 0x5630511d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f86a3400640 1 -- 172.21.15.59:0/759797614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d87e0 msgr2=0x5630511ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f86a3400640 1 --2- 172.21.15.59:0/759797614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d87e0 0x5630511ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f86a3400640 1 -- 172.21.15.59:0/759797614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56305110cf50 con 0x5630511f5e00 2024-04-23T14:39:40.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f86a3400640 1 --2- 172.21.15.59:0/759797614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x5630511a8790 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8694089480 tx=0x7f869400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ab9bc87c546681 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f8693e00640 1 -- 172.21.15.59:0/759797614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694061920 con 0x5630511f5e00 2024-04-23T14:39:40.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f8693e00640 1 -- 172.21.15.59:0/759797614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8694057040 con 0x5630511f5e00 2024-04-23T14:39:40.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.558+0000 7f8693e00640 1 -- 172.21.15.59:0/759797614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694071180 con 0x5630511f5e00 2024-04-23T14:39:40.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 -- 172.21.15.59:0/759797614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 msgr2=0x5630511a8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 --2- 172.21.15.59:0/759797614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x5630511a8790 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8694089480 tx=0x7f869400a000 comp rx=0 tx=0).stop 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 -- 172.21.15.59:0/759797614 shutdown_connections 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 --2- 172.21.15.59:0/759797614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d8f20 0x5630511d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 --2- 172.21.15.59:0/759797614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d87e0 0x5630511ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 --2- 172.21.15.59:0/759797614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x5630511a8790 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 -- 172.21.15.59:0/759797614 >> 172.21.15.59:0/759797614 conn(0x5630511aaa00 msgr2=0x5630511f32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 -- 172.21.15.59:0/759797614 shutdown_connections 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.559+0000 7f86ad394740 1 -- 172.21.15.59:0/759797614 wait complete. 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.560+0000 7f86ad394740 1 -- start start 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.560+0000 7f86ad394740 1 Processor -- start 2024-04-23T14:39:40.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86ad394740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d8f20 0x563051204580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 0x5630511ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d4040 0x5630511ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86ad394740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5630511acbb0 con 0x5630511d8f20 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5630510cb690 con 0x5630511d4040 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5630511abfa0 con 0x5630511f5e00 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86ad394740 10 client.0 _ll_register_callbacks cb 0x563051107650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d8f20 0x563051204580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d8f20 0x563051204580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41476/0 (socket says 172.21.15.59:41476) 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.561+0000 7f86a3400640 1 -- 172.21.15.59:0/325259764 learned_addr learned my addr 172.21.15.59:0/325259764 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f86a3400640 1 -- 172.21.15.59:0/325259764 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 msgr2=0x5630511ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f86a3400640 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 0x5630511ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f86a3400640 1 -- 172.21.15.59:0/325259764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d4040 msgr2=0x5630511ec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f86a3400640 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d4040 0x5630511ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f86a3400640 1 -- 172.21.15.59:0/325259764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630510ca920 con 0x5630511d8f20 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f86a3400640 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d8f20 0x563051204580 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f869406f040 tx=0x7f8694000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1eeb99477fc1aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f8690c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f8690c00640 1 -- 172.21.15.59:0/325259764 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694076060 con 0x5630511d8f20 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f8690c00640 1 -- 172.21.15.59:0/325259764 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f86940553b0 con 0x5630511d8f20 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.562+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8694057040 con 0x5630511d8f20 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f8690c00640 1 -- 172.21.15.59:0/325259764 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f869407d050 con 0x5630511d8f20 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 10 client.7932 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x563050eee7d0 con 0x5630511d8f20 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f8690c00640 1 -- 172.21.15.59:0/325259764 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8694057040 con 0x5630511d8f20 2024-04-23T14:39:40.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f8690c00640 1 -- 172.21.15.59:0/325259764 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f8694058190 con 0x5630511d8f20 2024-04-23T14:39:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 10 client.7932 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 10 client.7932 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 20 client.7932 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5630512075c0 0x563051207a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 4 client.7932 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:39:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56305111fc10 con 0x5630512075c0 2024-04-23T14:39:40.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.563+0000 7f869be00640 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5630512075c0 0x563051207a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.564+0000 7f869be00640 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5630512075c0 0x563051207a00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342c6a18cf222f0b server_cookie=38a2a6f19a7293b3 in_seq=0 out_seq=0 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.564+0000 7f8690c00640 10 client.7932 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f8690c00640 1 -- 172.21.15.59:0/325259764 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56305111fc10 con 0x5630512075c0 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f8690c00640 10 client.7932 handle_command_reply: tid=0 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 client.7932 shutdown 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d8f20 msgr2=0x563051204580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d8f20 0x563051204580 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f869406f040 tx=0x7f8694000bc0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 shutdown_connections 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x5630512075c0 0x563051207a00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 0x5630511ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d4040 0x5630511ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 --2- 172.21.15.59:0/325259764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d8f20 0x563051204580 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f8690c00640 0 client.7932 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.565+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 >> 172.21.15.59:0/325259764 conn(0x5630511d87e0 msgr2=0x5630511f54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.566+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 shutdown_connections 2024-04-23T14:39:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.566+0000 7f86ad394740 1 -- 172.21.15.59:0/325259764 wait complete. 2024-04-23T14:39:40.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.566+0000 7f86ad394740 20 client.7932 trim_cache size 0 max 16384 2024-04-23T14:39:40.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.586+0000 7f86ad394740 1 Processor -- start 2024-04-23T14:39:40.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.586+0000 7f86ad394740 1 -- start start 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86ad394740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d4040 0x5630511ed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x5630511f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 0x5630511f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86ad394740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4200e40 con 0x5630511d4040 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41fff50 con 0x5630511d8f20 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f86a4154bb0 con 0x5630511f5e00 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d4040 0x5630511ed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d4040 0x5630511ed360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41478/0 (socket says 172.21.15.59:41478) 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.587+0000 7f86a3400640 1 -- 172.21.15.59:0/3674367516 learned_addr learned my addr 172.21.15.59:0/3674367516 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f869be00640 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x5630511f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f86a3400640 1 -- 172.21.15.59:0/3674367516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 msgr2=0x5630511f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f86a3400640 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 0x5630511f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f86a3400640 1 -- 172.21.15.59:0/3674367516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 msgr2=0x5630511f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f86a3400640 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x5630511f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f86a3400640 1 -- 172.21.15.59:0/3674367516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563050e243f0 con 0x5630511d4040 2024-04-23T14:39:40.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f86a3400640 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d4040 0x5630511ed360 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f86940050a0 tx=0x7f8694084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19f08db198459df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f8693400640 1 -- 172.21.15.59:0/3674367516 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694054fe0 con 0x5630511d4040 2024-04-23T14:39:40.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f8693400640 1 -- 172.21.15.59:0/3674367516 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8694061920 con 0x5630511d4040 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.588+0000 7f8693400640 1 -- 172.21.15.59:0/3674367516 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694057040 con 0x5630511d4040 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 -- 172.21.15.59:0/3674367516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d4040 msgr2=0x5630511ed360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d4040 0x5630511ed360 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f86940050a0 tx=0x7f8694084040 comp rx=0 tx=0).stop 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 -- 172.21.15.59:0/3674367516 shutdown_connections 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511f5e00 0x5630511f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x5630511f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 --2- 172.21.15.59:0/3674367516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511d4040 0x5630511ed360 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 -- 172.21.15.59:0/3674367516 >> 172.21.15.59:0/3674367516 conn(0x5630511d87e0 msgr2=0x5630511ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 -- 172.21.15.59:0/3674367516 shutdown_connections 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.589+0000 7f86ad394740 1 -- 172.21.15.59:0/3674367516 wait complete. 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.590+0000 7f86ad394740 1 -- start start 2024-04-23T14:39:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.590+0000 7f86ad394740 1 Processor -- start 2024-04-23T14:39:40.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d4040 0x563051274f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86ad394740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x563051205b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86ad394740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x563051278880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86ad394740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5630511d3e50 con 0x5630511f5e00 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4155200 con 0x5630511d8f20 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86ad394740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5630511e9ef0 con 0x5630511d4040 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86ad394740 10 client.0 _ll_register_callbacks cb 0x7f86940594d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x563051278880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x563051278880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41484/0 (socket says 172.21.15.59:41484) 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.591+0000 7f86a3e00640 1 -- 172.21.15.59:0/3271321844 learned_addr learned my addr 172.21.15.59:0/3271321844 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f86a3e00640 1 -- 172.21.15.59:0/3271321844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d4040 msgr2=0x563051274f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f86a3e00640 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d4040 0x563051274f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f86a3e00640 1 -- 172.21.15.59:0/3271321844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 msgr2=0x563051205b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f869be00640 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x563051205b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f86a3e00640 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x563051205b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f86a3e00640 1 -- 172.21.15.59:0/3271321844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563050de40f0 con 0x5630511f5e00 2024-04-23T14:39:40.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f86a3e00640 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x563051278880 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f869c009d90 tx=0x7f869c007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e630ae77735fe3d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f8690c00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f8690c00640 1 -- 172.21.15.59:0/3271321844 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c045430 con 0x5630511f5e00 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f8690c00640 1 -- 172.21.15.59:0/3271321844 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f869c046cf0 con 0x5630511f5e00 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.592+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8694061920 con 0x5630511f5e00 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f8690c00640 1 -- 172.21.15.59:0/3271321844 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f869c03c420 con 0x5630511f5e00 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 10 client.7938 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5630510d7810 con 0x5630511f5e00 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f8690c00640 1 -- 172.21.15.59:0/3271321844 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8694061920 con 0x5630511f5e00 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f8690c00640 1 -- 172.21.15.59:0/3271321844 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f869c04d050 con 0x5630511f5e00 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 10 client.7938 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 10 client.7938 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 20 client.7938 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x563051275a90 0x5630511ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 4 client.7938 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:39:40.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.593+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563051119a80 con 0x563051275a90 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f86a3400640 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x563051275a90 0x5630511ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f86a3400640 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x563051275a90 0x5630511ebc00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbe500547fc25c server_cookie=fc2f33c28d255289 in_seq=0 out_seq=0 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f8690c00640 10 client.7938 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f8690c00640 1 -- 172.21.15.59:0/3271321844 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563051119a80 con 0x563051275a90 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f8690c00640 10 client.7938 handle_command_reply: tid=0 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f86ad394740 1 client.7938 shutdown 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 msgr2=0x563051278880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.594+0000 7f86ad394740 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x563051278880 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f869c009d90 tx=0x7f869c007c70 comp rx=0 tx=0).stop 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 shutdown_connections 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x563051275a90 0x5630511ebc00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5630511f5e00 0x563051278880 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5630511d8f20 0x563051205b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 --2- 172.21.15.59:0/3271321844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5630511d4040 0x563051274f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f8690c00640 0 client.7938 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:40.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 >> 172.21.15.59:0/3271321844 conn(0x5630511d87e0 msgr2=0x5630511ecfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:40.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 shutdown_connections 2024-04-23T14:39:40.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 1 -- 172.21.15.59:0/3271321844 wait complete. 2024-04-23T14:39:40.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.595+0000 7f86ad394740 20 client.7938 trim_cache size 0 max 16384 2024-04-23T14:39:40.600 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:39:40.600 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:39:40.600 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:39:40.600 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:39:40.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f868006c8f0 msgr2=0x7f868006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f868006c8f0 0x7f868006edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f868c000f60 tx=0x7f868c0078f0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 msgr2=0x7f86a4098ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a4098ad0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f869400c3f0 tx=0x7f869400c8b0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 shutdown_connections 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f868006c8f0 0x7f868006edb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a40d0ce0 0x7f86a415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a415c410 0x7f86a4151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 --2- 172.21.15.59:0/1725509640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a4072c80 0x7f86a4098ad0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 >> 172.21.15.59:0/1725509640 conn(0x7f86a406bdf0 msgr2=0x7f86a40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 shutdown_connections 2024-04-23T14:39:40.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.596+0000 7f86a9000640 1 -- 172.21.15.59:0/1725509640 wait complete. 2024-04-23T14:39:40.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:39:40.616 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:39:40.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.825+0000 7f3217e00640 1 Processor -- start 2024-04-23T14:39:40.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.825+0000 7f3217e00640 1 -- start start 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.826+0000 7f3217e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f321806f3f0 0x7f32180cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.826+0000 7f3217e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32180b3080 0x7f32180cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.826+0000 7f3217e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32180b3870 con 0x7f321815b450 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.826+0000 7f3217e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f32180b41a0 con 0x7f32180b3080 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.826+0000 7f3217e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32180b3b50 con 0x7f321806f3f0 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.827+0000 7f3216a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f321815b450 0x7f32180b4630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44450/0 (socket says 172.21.15.59:44450) 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.827+0000 7f3216a00640 1 -- 172.21.15.59:0/1811247067 learned_addr learned my addr 172.21.15.59:0/1811247067 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.827+0000 7f3217400640 1 --2- 172.21.15.59:0/1811247067 >> v2:172.21.15.59:3300/0 conn(0x7f32180b3080 0x7f32180cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.827+0000 7f3216000640 1 --2- 172.21.15.59:0/1811247067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f321806f3f0 0x7f32180cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.827+0000 7f3215600640 1 -- 172.21.15.59:0/1811247067 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39484424 0 0) 0x7f32180b3870 con 0x7f321815b450 2024-04-23T14:39:40.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.827+0000 7f3215600640 1 -- 172.21.15.59:0/1811247067 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31fc003520 con 0x7f321815b450 2024-04-23T14:39:40.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.828+0000 7f3217400640 1 -- 172.21.15.59:0/1811247067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f321806f3f0 msgr2=0x7f32180cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:40.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.828+0000 7f3217400640 1 --2- 172.21.15.59:0/1811247067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f321806f3f0 0x7f32180cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.828+0000 7f3217400640 1 -- 172.21.15.59:0/1811247067 >> v1:172.21.15.59:6789/0 conn(0x7f321815b450 legacy=0x7f32180b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.828+0000 7f3217400640 1 -- 172.21.15.59:0/1811247067 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32180b3e30 con 0x7f32180b3080 2024-04-23T14:39:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.829+0000 7f3217400640 1 --2- 172.21.15.59:0/1811247067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32180b3080 0x7f32180cd590 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f32100097a0 tx=0x7f321002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8302bc884579f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.829+0000 7f3215600640 1 -- 172.21.15.59:0/1811247067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210037070 con 0x7f32180b3080 2024-04-23T14:39:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.829+0000 7f3215600640 1 -- 172.21.15.59:0/1811247067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3210032440 con 0x7f32180b3080 2024-04-23T14:39:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.829+0000 7f3215600640 1 -- 172.21.15.59:0/1811247067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32100327f0 con 0x7f32180b3080 2024-04-23T14:39:40.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 -- 172.21.15.59:0/1811247067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32180b3080 msgr2=0x7f32180cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 --2- 172.21.15.59:0/1811247067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32180b3080 0x7f32180cd590 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f32100097a0 tx=0x7f321002a180 comp rx=0 tx=0).stop 2024-04-23T14:39:40.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 -- 172.21.15.59:0/1811247067 shutdown_connections 2024-04-23T14:39:40.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 --2- 172.21.15.59:0/1811247067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32180b3080 0x7f32180cd590 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 --2- 172.21.15.59:0/1811247067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f321806f3f0 0x7f32180cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 -- 172.21.15.59:0/1811247067 >> 172.21.15.59:0/1811247067 conn(0x7f32180b21a0 msgr2=0x7f32180b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:40.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 -- 172.21.15.59:0/1811247067 shutdown_connections 2024-04-23T14:39:40.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.830+0000 7f3217e00640 1 -- 172.21.15.59:0/1811247067 wait complete. 2024-04-23T14:39:40.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.832+0000 7f3217e00640 1 Processor -- start 2024-04-23T14:39:40.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.832+0000 7f3217e00640 1 -- start start 2024-04-23T14:39:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321806f3f0 0x7f32180ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32180b3080 0x7f32180c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321815b450 0x7f32180c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32180ad060 con 0x7f321815b450 2024-04-23T14:39:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32180b4ff0 con 0x7f321806f3f0 2024-04-23T14:39:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32180ad1e0 con 0x7f32180b3080 2024-04-23T14:39:40.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321815b450 0x7f32180c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321815b450 0x7f32180c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41504/0 (socket says 172.21.15.59:41504) 2024-04-23T14:39:40.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217400640 1 -- 172.21.15.59:0/743258447 learned_addr learned my addr 172.21.15.59:0/743258447 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:40.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217400640 1 -- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32180b3080 msgr2=0x7f32180c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217400640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32180b3080 0x7f32180c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.833+0000 7f3217400640 1 -- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321806f3f0 msgr2=0x7f32180ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:40.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f3216a00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321806f3f0 0x7f32180ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f3217400640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321806f3f0 0x7f32180ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:40.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f3217400640 1 -- 172.21.15.59:0/743258447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321815ba40 con 0x7f321815b450 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f3216a00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321806f3f0 0x7f32180ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f3217400640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321815b450 0x7f32180c2900 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f3210035040 tx=0x7f32100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f320ea00640 1 -- 172.21.15.59:0/743258447 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210037040 con 0x7f321815b450 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f320ea00640 1 -- 172.21.15.59:0/743258447 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321003b430 con 0x7f321815b450 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f320ea00640 1 -- 172.21.15.59:0/743258447 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321003b7e0 con 0x7f321815b450 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.834+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3218150a30 con 0x7f321815b450 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.835+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321815bce0 con 0x7f321815b450 2024-04-23T14:39:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.836+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321804fd90 con 0x7f321815b450 2024-04-23T14:39:40.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.837+0000 7f320ea00640 1 -- 172.21.15.59:0/743258447 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3210044020 con 0x7f321815b450 2024-04-23T14:39:40.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.838+0000 7f320ea00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31f806c8f0 0x7f31f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:40.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.838+0000 7f320ea00640 1 -- 172.21.15.59:0/743258447 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f32100abe30 con 0x7f321815b450 2024-04-23T14:39:40.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.838+0000 7f3216a00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31f806c8f0 0x7f31f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:40.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.839+0000 7f3216a00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31f806c8f0 0x7f31f806edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3208005c00 tx=0x7f3208009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:40.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:40.843+0000 7f320ea00640 1 -- 172.21.15.59:0/743258447 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f321007a170 con 0x7f321815b450 2024-04-23T14:39:41.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.096+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f321806e6c0 con 0x7f321815b450 2024-04-23T14:39:41.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.097+0000 7f320ea00640 1 -- 172.21.15.59:0/743258447 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5182 (secure 0 0 0) 0x7f3210079a70 con 0x7f321815b450 2024-04-23T14:39:41.098 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:39:41.098 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:39:16.472663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7533,"qdb_cluster":[7533]},"id":6}]} 2024-04-23T14:39:41.099 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 67 2024-04-23T14:39:41.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31f806c8f0 msgr2=0x7f31f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31f806c8f0 0x7f31f806edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3208005c00 tx=0x7f3208009040 comp rx=0 tx=0).stop 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321815b450 msgr2=0x7f32180c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321815b450 0x7f32180c2900 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f3210035040 tx=0x7f32100046c0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 shutdown_connections 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f31f806c8f0 0x7f31f806edb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32180b3080 0x7f32180c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321815b450 0x7f32180c2900 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 --2- 172.21.15.59:0/743258447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321806f3f0 0x7f32180ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 >> 172.21.15.59:0/743258447 conn(0x7f32180b21a0 msgr2=0x7f32180ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.100+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 shutdown_connections 2024-04-23T14:39:41.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.101+0000 7f3217e00640 1 -- 172.21.15.59:0/743258447 wait complete. 2024-04-23T14:39:41.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-23T14:39:41.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.334+0000 7fb983200640 1 Processor -- start 2024-04-23T14:39:41.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.334+0000 7fb983200640 1 -- start start 2024-04-23T14:39:41.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.336+0000 7fb983200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb984162a30 0x7fb9840d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.336+0000 7fb983200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9840b3fb0 con 0x7fb98406f5f0 2024-04-23T14:39:41.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.336+0000 7fb983200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98406f340 con 0x7fb98406fb40 2024-04-23T14:39:41.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.336+0000 7fb983200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb9840b5500 con 0x7fb984162a30 2024-04-23T14:39:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.337+0000 7fb97be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb98406f5f0 0x7fb9840b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44462/0 (socket says 172.21.15.59:44462) 2024-04-23T14:39:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.337+0000 7fb97be00640 1 -- 172.21.15.59:0/3211888491 learned_addr learned my addr 172.21.15.59:0/3211888491 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.337+0000 7fb981e00640 1 --2- 172.21.15.59:0/3211888491 >> v2:172.21.15.76:3300/0 conn(0x7fb984162a30 0x7fb9840d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.337+0000 7fb981400640 1 -- 172.21.15.59:0/3211888491 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948947300 0 0) 0x7fb9840b3fb0 con 0x7fb98406f5f0 2024-04-23T14:39:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.337+0000 7fb981400640 1 -- 172.21.15.59:0/3211888491 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb968003520 con 0x7fb98406f5f0 2024-04-23T14:39:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.338+0000 7fb981e00640 1 -- 172.21.15.59:0/3211888491 >> v1:172.21.15.76:6789/0 conn(0x7fb98406fb40 legacy=0x7fb9840b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.338+0000 7fb981400640 1 -- 172.21.15.59:0/3211888491 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2600815797 0 0) 0x7fb968003520 con 0x7fb98406f5f0 2024-04-23T14:39:41.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.338+0000 7fb981e00640 1 -- 172.21.15.59:0/3211888491 >> v1:172.21.15.59:6789/0 conn(0x7fb98406f5f0 legacy=0x7fb9840b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.338+0000 7fb981e00640 1 -- 172.21.15.59:0/3211888491 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9841668b0 con 0x7fb984162a30 2024-04-23T14:39:41.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.339+0000 7fb981e00640 1 --2- 172.21.15.59:0/3211888491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb984162a30 0x7fb9840d05d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9740097a0 tx=0x7fb97402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ff117e03afa29 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.339+0000 7fb981400640 1 -- 172.21.15.59:0/3211888491 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974037070 con 0x7fb984162a30 2024-04-23T14:39:41.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.339+0000 7fb981400640 1 -- 172.21.15.59:0/3211888491 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97402ac50 con 0x7fb984162a30 2024-04-23T14:39:41.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.339+0000 7fb981400640 1 -- 172.21.15.59:0/3211888491 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c001670 con 0x7fb984162a30 2024-04-23T14:39:41.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.340+0000 7fb983200640 1 -- 172.21.15.59:0/3211888491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb984162a30 msgr2=0x7fb9840d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.340+0000 7fb983200640 1 --2- 172.21.15.59:0/3211888491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb984162a30 0x7fb9840d05d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb9740097a0 tx=0x7fb97402a1c0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.340+0000 7fb983200640 1 -- 172.21.15.59:0/3211888491 shutdown_connections 2024-04-23T14:39:41.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.340+0000 7fb983200640 1 --2- 172.21.15.59:0/3211888491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb984162a30 0x7fb9840d05d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.340+0000 7fb983200640 1 -- 172.21.15.59:0/3211888491 >> 172.21.15.59:0/3211888491 conn(0x7fb98406bdf0 msgr2=0x7fb98406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.340+0000 7fb983200640 1 -- 172.21.15.59:0/3211888491 shutdown_connections 2024-04-23T14:39:41.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.340+0000 7fb983200640 1 -- 172.21.15.59:0/3211888491 wait complete. 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.342+0000 7fb983200640 1 Processor -- start 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.342+0000 7fb983200640 1 -- start start 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.343+0000 7fb983200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98406f5f0 0x7fb98415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.343+0000 7fb983200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98406fb40 0x7fb98415f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.343+0000 7fb983200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb984162a30 0x7fb9840b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.343+0000 7fb983200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb984166e70 con 0x7fb98406f5f0 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.343+0000 7fb983200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb984166660 con 0x7fb98406fb40 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.343+0000 7fb983200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb984167150 con 0x7fb984162a30 2024-04-23T14:39:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98406f5f0 0x7fb98415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98406f5f0 0x7fb98415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41520/0 (socket says 172.21.15.59:41520) 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 -- 172.21.15.59:0/1160759713 learned_addr learned my addr 172.21.15.59:0/1160759713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb97be00640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98406fb40 0x7fb98415f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb982800640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb984162a30 0x7fb9840b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 -- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb984162a30 msgr2=0x7fb9840b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb984162a30 0x7fb9840b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 -- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98406fb40 msgr2=0x7fb98415f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98406fb40 0x7fb98415f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb981e00640 1 -- 172.21.15.59:0/1160759713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9841659b0 con 0x7fb98406f5f0 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.344+0000 7fb97be00640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98406fb40 0x7fb98415f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.345+0000 7fb981e00640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98406f5f0 0x7fb98415c5a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb974035040 tx=0x7fb9740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.345+0000 7fb97a000640 1 -- 172.21.15.59:0/1160759713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974041070 con 0x7fb98406f5f0 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.345+0000 7fb97a000640 1 -- 172.21.15.59:0/1160759713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97403c460 con 0x7fb98406f5f0 2024-04-23T14:39:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.345+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb984165c90 con 0x7fb98406f5f0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.345+0000 7fb97a000640 1 -- 172.21.15.59:0/1160759713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974037040 con 0x7fb98406f5f0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.345+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98415e810 con 0x7fb98406f5f0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.347+0000 7fb97a000640 1 -- 172.21.15.59:0/1160759713 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb974044070 con 0x7fb98406f5f0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.348+0000 7fb97a000640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.348+0000 7fb97a000640 1 -- 172.21.15.59:0/1160759713 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fb9740ac230 con 0x7fb98406f5f0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.349+0000 7fb97be00640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.350+0000 7fb98c1ae740 1 Processor -- start 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.350+0000 7fb97be00640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96006c8f0 0x7fb96006edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005880 tx=0x7fb96c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.350+0000 7fb98c1ae740 1 -- start start 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95b4d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b6aac0 0x559b95b6b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b22f00 0x559b95b69180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b95a87190 con 0x559b95b6aac0 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559b95a2d030 con 0x559b95b22f00 2024-04-23T14:39:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x559b95a83150 con 0x559b95b4d220 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b6aac0 0x559b95b6b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b6aac0 0x559b95b6b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41534/0 (socket says 172.21.15.59:41534) 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 -- 172.21.15.59:0/103845420 learned_addr learned my addr 172.21.15.59:0/103845420 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb982800640 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b22f00 0x559b95b69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb981e00640 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95b4d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 -- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 msgr2=0x559b95b4d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95b4d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 -- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b22f00 msgr2=0x559b95b69180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b22f00 0x559b95b69180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.351+0000 7fb97be00640 1 -- 172.21.15.59:0/103845420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b95a15b60 con 0x559b95b6aac0 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.352+0000 7fb97be00640 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b6aac0 0x559b95b6b010 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005c30 tx=0x7fb96c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbc465a046f150e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.352+0000 7fb972a00640 1 -- 172.21.15.59:0/103845420 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c005c60 con 0x559b95b6aac0 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.352+0000 7fb972a00640 1 -- 172.21.15.59:0/103845420 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c005e00 con 0x559b95b6aac0 2024-04-23T14:39:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.352+0000 7fb972a00640 1 -- 172.21.15.59:0/103845420 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c020650 con 0x559b95b6aac0 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 -- 172.21.15.59:0/103845420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b6aac0 msgr2=0x559b95b6b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b6aac0 0x559b95b6b010 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005c30 tx=0x7fb96c010000 comp rx=0 tx=0).stop 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 -- 172.21.15.59:0/103845420 shutdown_connections 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b22f00 0x559b95b69180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b6aac0 0x559b95b6b010 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/103845420 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95b4d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 -- 172.21.15.59:0/103845420 >> 172.21.15.59:0/103845420 conn(0x559b95b1f830 msgr2=0x559b95b68030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 -- 172.21.15.59:0/103845420 shutdown_connections 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.353+0000 7fb98c1ae740 1 -- 172.21.15.59:0/103845420 wait complete. 2024-04-23T14:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.354+0000 7fb98c1ae740 1 -- start start 2024-04-23T14:39:41.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.354+0000 7fb98c1ae740 1 Processor -- start 2024-04-23T14:39:41.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b49cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b78ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b68fb0 0x559b95b69430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b95b23ca0 con 0x559b95b4d220 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559b95a40500 con 0x559b95b6aac0 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x559b95b1d760 con 0x559b95b68fb0 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb98c1ae740 10 client.0 _ll_register_callbacks cb 0x559b95a7d270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb981e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b49cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb981e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b49cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41542/0 (socket says 172.21.15.59:41542) 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.355+0000 7fb981e00640 1 -- 172.21.15.59:0/1130366475 learned_addr learned my addr 172.21.15.59:0/1130366475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb982800640 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b68fb0 0x559b95b69430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb981e00640 1 -- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b68fb0 msgr2=0x559b95b69430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb97be00640 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b78ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb981e00640 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b68fb0 0x559b95b69430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb981e00640 1 -- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 msgr2=0x559b95b78ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb981e00640 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b78ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb981e00640 1 -- 172.21.15.59:0/1130366475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b958657c0 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb981e00640 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b49cc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb9740aa670 tx=0x7fb97407a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4765f8b5d1c014cc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb967400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb967400640 1 -- 172.21.15.59:0/1130366475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974092020 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb967400640 1 -- 172.21.15.59:0/1130366475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fb9740923c0 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.356+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96c005e00 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb967400640 1 -- 172.21.15.59:0/1130366475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb97404b040 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 10 client.7959 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x559b958c5f50 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb967400640 1 -- 172.21.15.59:0/1130366475 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c005e00 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb967400640 1 -- 172.21.15.59:0/1130366475 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fb9740825c0 con 0x559b95b4d220 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 10 client.7959 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 10 client.7959 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 20 client.7959 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:41.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b612d0 0x559b95b616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 4 client.7959 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.357+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b95a94bb0 con 0x559b95b612d0 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.358+0000 7fb97be00640 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b612d0 0x559b95b616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.358+0000 7fb97be00640 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b612d0 0x559b95b616b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc339528cc58021 server_cookie=3d528b64f731bc65 in_seq=0 out_seq=0 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.358+0000 7fb967400640 10 client.7959 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb967400640 1 -- 172.21.15.59:0/1130366475 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b95a94bb0 con 0x559b95b612d0 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb967400640 10 client.7959 handle_command_reply: tid=0 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 client.7959 shutdown 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 msgr2=0x559b95b49cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b49cc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb9740aa670 tx=0x7fb97407a570 comp rx=0 tx=0).stop 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 shutdown_connections 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b612d0 0x559b95b616b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b68fb0 0x559b95b69430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b78ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/1130366475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b49cc0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 >> 172.21.15.59:0/1130366475 conn(0x559b95b22f00 msgr2=0x559b95b6b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.359+0000 7fb967400640 0 client.7959 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.360+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 shutdown_connections 2024-04-23T14:39:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.360+0000 7fb98c1ae740 1 -- 172.21.15.59:0/1130366475 wait complete. 2024-04-23T14:39:41.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.360+0000 7fb98c1ae740 20 client.7959 trim_cache size 0 max 16384 2024-04-23T14:39:41.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.375+0000 7fb98c1ae740 1 Processor -- start 2024-04-23T14:39:41.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.375+0000 7fb98c1ae740 1 -- start start 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.375+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b40ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.375+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b68fb0 0x559b95b18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.375+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b6aac0 0x559b95b77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.375+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9842051f0 con 0x559b95b4d220 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.375+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb984204360 con 0x559b95b68fb0 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb9840ce9d0 con 0x559b95b6aac0 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b40ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b40ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41548/0 (socket says 172.21.15.59:41548) 2024-04-23T14:39:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 -- 172.21.15.59:0/2639644881 learned_addr learned my addr 172.21.15.59:0/2639644881 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 -- 172.21.15.59:0/2639644881 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b6aac0 msgr2=0x559b95b77b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 --2- 172.21.15.59:0/2639644881 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b6aac0 0x559b95b77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 -- 172.21.15.59:0/2639644881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b68fb0 msgr2=0x559b95b18d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 --2- 172.21.15.59:0/2639644881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b68fb0 0x559b95b18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.376+0000 7fb981e00640 1 -- 172.21.15.59:0/2639644881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b957590b0 con 0x559b95b4d220 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb981e00640 1 --2- 172.21.15.59:0/2639644881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b40ee0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb974078120 tx=0x7fb974048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7658f775bc5aabc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb972000640 1 -- 172.21.15.59:0/2639644881 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740aa0d0 con 0x559b95b4d220 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb972000640 1 -- 172.21.15.59:0/2639644881 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9740aa270 con 0x559b95b4d220 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb972000640 1 -- 172.21.15.59:0/2639644881 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740794e0 con 0x559b95b4d220 2024-04-23T14:39:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb98c1ae740 1 -- 172.21.15.59:0/2639644881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 msgr2=0x559b95b40ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/2639644881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b40ee0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb974078120 tx=0x7fb974048000 comp rx=0 tx=0).stop 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb98c1ae740 1 -- 172.21.15.59:0/2639644881 shutdown_connections 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/2639644881 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b6aac0 0x559b95b77b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/2639644881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b68fb0 0x559b95b18d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/2639644881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b4d220 0x559b95b40ee0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.377+0000 7fb98c1ae740 1 -- 172.21.15.59:0/2639644881 >> 172.21.15.59:0/2639644881 conn(0x559b95b22f00 msgr2=0x559b95b49ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.378+0000 7fb98c1ae740 1 -- 172.21.15.59:0/2639644881 shutdown_connections 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.378+0000 7fb98c1ae740 1 -- 172.21.15.59:0/2639644881 wait complete. 2024-04-23T14:39:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.378+0000 7fb98c1ae740 1 -- start start 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.378+0000 7fb98c1ae740 1 Processor -- start 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.379+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95be1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.379+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b68fb0 0x559b95b78470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.379+0000 7fb98c1ae740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b5df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.379+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b95b79430 con 0x559b95b68fb0 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.379+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb9840cf020 con 0x559b95b6aac0 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.379+0000 7fb98c1ae740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x559b95b4fdf0 con 0x559b95b4d220 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.379+0000 7fb98c1ae740 10 client.0 _ll_register_callbacks cb 0x7fb974009150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b68fb0 0x559b95b78470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b68fb0 0x559b95b78470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41558/0 (socket says 172.21.15.59:41558) 2024-04-23T14:39:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 -- 172.21.15.59:0/450972930 learned_addr learned my addr 172.21.15.59:0/450972930 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb982800640 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b5df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 -- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 msgr2=0x559b95be1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb981e00640 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95be1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95be1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 -- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 msgr2=0x559b95b5df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b5df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 -- 172.21.15.59:0/450972930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b95b67c30 con 0x559b95b68fb0 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb97be00640 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b68fb0 0x559b95b78470 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb96c001e10 tx=0x7fb96c001450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e7dd867963903b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.380+0000 7fb967400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.381+0000 7fb967400640 1 -- 172.21.15.59:0/450972930 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c02d450 con 0x559b95b68fb0 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.381+0000 7fb967400640 1 -- 172.21.15.59:0/450972930 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fb96c02d5f0 con 0x559b95b68fb0 2024-04-23T14:39:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.381+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9740aa270 con 0x559b95b68fb0 2024-04-23T14:39:41.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.381+0000 7fb967400640 1 -- 172.21.15.59:0/450972930 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb96c02d7f0 con 0x559b95b68fb0 2024-04-23T14:39:41.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.381+0000 7fb98c1ae740 10 client.7965 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:41.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.381+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x559b95b78a40 con 0x559b95b68fb0 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb967400640 1 -- 172.21.15.59:0/450972930 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9740aa270 con 0x559b95b68fb0 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb967400640 1 -- 172.21.15.59:0/450972930 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fb96c020450 con 0x559b95b68fb0 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb98c1ae740 10 client.7965 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb98c1ae740 10 client.7965 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb98c1ae740 20 client.7965 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b68460 0x559b95b4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb98c1ae740 4 client.7965 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x559b95b62790 con 0x559b95b68460 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.382+0000 7fb981e00640 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b68460 0x559b95b4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb981e00640 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b68460 0x559b95b4dee0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ec7c1025dcae6 server_cookie=84e098d0d721319c in_seq=0 out_seq=0 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb967400640 10 client.7965 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb967400640 1 -- 172.21.15.59:0/450972930 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+418 (crc 0 0 0) 0x559b95b62790 con 0x559b95b68460 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb967400640 10 client.7965 handle_command_reply: tid=0 2024-04-23T14:39:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 client.7965 shutdown 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b68fb0 msgr2=0x559b95b78470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b68fb0 0x559b95b78470 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb96c001e10 tx=0x7fb96c001450 comp rx=0 tx=0).stop 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 shutdown_connections 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x559b95b68460 0x559b95b4dee0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b95b6aac0 0x559b95b5df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b95b68fb0 0x559b95b78470 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 --2- 172.21.15.59:0/450972930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559b95b4d220 0x559b95be1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb967400640 0 client.7965 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 >> 172.21.15.59:0/450972930 conn(0x559b95b22f00 msgr2=0x559b95b60050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.383+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 shutdown_connections 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb98c1ae740 1 -- 172.21.15.59:0/450972930 wait complete. 2024-04-23T14:39:41.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb98c1ae740 20 client.7965 trim_cache size 0 max 16384 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 665661408, 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627776, 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir" 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 728316110, 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-23T14:39:41.388 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/testdir/hardlink" 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "dir_frag", 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 3402197265, 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627776, 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "frag": "*", 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir" 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:39:41.389 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96006c8f0 msgr2=0x7fb96006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96006c8f0 0x7fb96006edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005880 tx=0x7fb96c009040 comp rx=0 tx=0).stop 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98406f5f0 msgr2=0x7fb98415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98406f5f0 0x7fb98415c5a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb974035040 tx=0x7fb9740046c0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 shutdown_connections 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb984162a30 0x7fb9840b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98406fb40 0x7fb98415f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 --2- 172.21.15.59:0/1160759713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98406f5f0 0x7fb98415c5a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.384+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 >> 172.21.15.59:0/1160759713 conn(0x7fb98406bdf0 msgr2=0x7fb9840ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.385+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 shutdown_connections 2024-04-23T14:39:41.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.385+0000 7fb983200640 1 -- 172.21.15.59:0/1160759713 wait complete. 2024-04-23T14:39:41.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:39:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.602+0000 7fa063200640 1 Processor -- start 2024-04-23T14:39:41.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.602+0000 7fa063200640 1 -- start start 2024-04-23T14:39:41.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.603+0000 7fa063200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa05c15c3d0 0x7fa05c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.603+0000 7fa063200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05c160210 con 0x7fa05c0cd590 2024-04-23T14:39:41.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.603+0000 7fa063200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05c1604f0 con 0x7fa05c0b4860 2024-04-23T14:39:41.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.603+0000 7fa063200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa05c1607d0 con 0x7fa05c15c3d0 2024-04-23T14:39:41.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.603+0000 7fa061400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa05c0cd590 0x7fa05c0b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44468/0 (socket says 172.21.15.59:44468) 2024-04-23T14:39:41.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.603+0000 7fa061400640 1 -- 172.21.15.59:0/2837903772 learned_addr learned my addr 172.21.15.59:0/2837903772 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa061e00640 1 --2- 172.21.15.59:0/2837903772 >> v2:172.21.15.76:3300/0 conn(0x7fa05c15c3d0 0x7fa05c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2808960291 0 0) 0x7fa05c160210 con 0x7fa05c0cd590 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa044003520 con 0x7fa05c0cd590 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 248203658 0 0) 0x7fa044003520 con 0x7fa05c0cd590 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 >> v2:172.21.15.76:3300/0 conn(0x7fa05c15c3d0 msgr2=0x7fa05c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 --2- 172.21.15.59:0/2837903772 >> v2:172.21.15.76:3300/0 conn(0x7fa05c15c3d0 0x7fa05c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 >> v1:172.21.15.76:6789/0 conn(0x7fa05c0b4860 legacy=0x7fa05c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c160ab0 con 0x7fa05c0cd590 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa04c001670 con 0x7fa05c0cd590 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 >> v1:172.21.15.59:6789/0 conn(0x7fa05c0cd590 legacy=0x7fa05c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0440039b0 0x7fa044003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044004800 0x7fa044004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.604+0000 7fa060a00640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440051c0 0x7fa04400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1607d0 con 0x7fa0440051c0 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa04400c060 con 0x7fa044004800 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa04400c260 con 0x7fa0440039b0 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa062800640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440051c0 0x7fa04400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa061400640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044004800 0x7fa044004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa061e00640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0440039b0 0x7fa044003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa062800640 1 -- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0440039b0 msgr2=0x7fa044003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa062800640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0440039b0 0x7fa044003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa062800640 1 -- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044004800 msgr2=0x7fa044004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa062800640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa044004800 0x7fa044004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa062800640 1 -- 172.21.15.59:0/2837903772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa044003520 con 0x7fa0440051c0 2024-04-23T14:39:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.605+0000 7fa061e00640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0440039b0 0x7fa044003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:41.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.606+0000 7fa062800640 1 -- 172.21.15.59:0/2837903772 reap_dead start 2024-04-23T14:39:41.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.606+0000 7fa062800640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440051c0 0x7fa04400b9c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa058001e50 tx=0x7fa044005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe3757dbce0e9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058001a70 con 0x7fa0440051c0 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa058003970 con 0x7fa0440051c0 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa060a00640 1 -- 172.21.15.59:0/2837903772 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058003d60 con 0x7fa0440051c0 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa063200640 1 -- 172.21.15.59:0/2837903772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440051c0 msgr2=0x7fa04400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa063200640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440051c0 0x7fa04400b9c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa058001e50 tx=0x7fa044005830 comp rx=0 tx=0).stop 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa063200640 1 -- 172.21.15.59:0/2837903772 shutdown_connections 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa063200640 1 --2- 172.21.15.59:0/2837903772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440051c0 0x7fa04400b9c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa063200640 1 -- 172.21.15.59:0/2837903772 >> 172.21.15.59:0/2837903772 conn(0x7fa05c0b21a0 msgr2=0x7fa05c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.607+0000 7fa063200640 1 -- 172.21.15.59:0/2837903772 shutdown_connections 2024-04-23T14:39:41.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.608+0000 7fa063200640 1 -- 172.21.15.59:0/2837903772 wait complete. 2024-04-23T14:39:41.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.609+0000 7fa063200640 1 Processor -- start 2024-04-23T14:39:41.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.609+0000 7fa063200640 1 -- start start 2024-04-23T14:39:41.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.610+0000 7fa063200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0440051c0 0x7fa05c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.610+0000 7fa063200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa05c15de60 0x7fa05c1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.610+0000 7fa063200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa05c072bf0 0x7fa05c073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.610+0000 7fa063200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c15c930 con 0x7fa05c072bf0 2024-04-23T14:39:41.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.610+0000 7fa063200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c15c7b0 con 0x7fa0440051c0 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.610+0000 7fa063200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa05c0cd050 con 0x7fa05c15de60 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa05c072bf0 0x7fa05c073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa05c072bf0 0x7fa05c073070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41590/0 (socket says 172.21.15.59:41590) 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 -- 172.21.15.59:0/1431255509 learned_addr learned my addr 172.21.15.59:0/1431255509 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa061e00640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0440051c0 0x7fa05c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa061400640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa05c15de60 0x7fa05c1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 -- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa05c15de60 msgr2=0x7fa05c1560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa05c15de60 0x7fa05c1560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 -- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0440051c0 msgr2=0x7fa05c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0440051c0 0x7fa05c0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.611+0000 7fa062800640 1 -- 172.21.15.59:0/1431255509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c15e3b0 con 0x7fa05c072bf0 2024-04-23T14:39:41.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.612+0000 7fa062800640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa05c072bf0 0x7fa05c073070 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa058001f70 tx=0x7fa0580014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.612+0000 7fa056000640 1 -- 172.21.15.59:0/1431255509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058001a70 con 0x7fa05c072bf0 2024-04-23T14:39:41.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.612+0000 7fa056000640 1 -- 172.21.15.59:0/1431255509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05802ac70 con 0x7fa05c072bf0 2024-04-23T14:39:41.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.612+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c15e690 con 0x7fa05c072bf0 2024-04-23T14:39:41.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.612+0000 7fa056000640 1 -- 172.21.15.59:0/1431255509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058007db0 con 0x7fa05c072bf0 2024-04-23T14:39:41.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.612+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c15ec50 con 0x7fa05c072bf0 2024-04-23T14:39:41.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.613+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c04fd90 con 0x7fa05c072bf0 2024-04-23T14:39:41.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.614+0000 7fa056000640 1 -- 172.21.15.59:0/1431255509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa058007030 con 0x7fa05c072bf0 2024-04-23T14:39:41.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.615+0000 7fa056000640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa04006c920 0x7fa04006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:41.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.615+0000 7fa056000640 1 -- 172.21.15.59:0/1431255509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fa0580ab150 con 0x7fa05c072bf0 2024-04-23T14:39:41.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.615+0000 7fa061e00640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa04006c920 0x7fa04006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:41.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.616+0000 7fa061e00640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa04006c920 0x7fa04006ede0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa050009aa0 tx=0x7fa050008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:41.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.620+0000 7fa056000640 1 -- 172.21.15.59:0/1431255509 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa058079490 con 0x7fa05c072bf0 2024-04-23T14:39:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.873+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa05c156cf0 con 0x7fa05c072bf0 2024-04-23T14:39:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.874+0000 7fa056000640 1 -- 172.21.15.59:0/1431255509 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fa058078d90 con 0x7fa05c072bf0 2024-04-23T14:39:41.875 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:39:41.875 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-23T14:39:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa04006c920 msgr2=0x7fa04006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa04006c920 0x7fa04006ede0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa050009aa0 tx=0x7fa050008040 comp rx=0 tx=0).stop 2024-04-23T14:39:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa05c072bf0 msgr2=0x7fa05c073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa05c072bf0 0x7fa05c073070 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa058001f70 tx=0x7fa0580014a0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 shutdown_connections 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa04006c920 0x7fa04006ede0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa05c15de60 0x7fa05c1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa05c072bf0 0x7fa05c073070 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 --2- 172.21.15.59:0/1431255509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0440051c0 0x7fa05c0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 >> 172.21.15.59:0/1431255509 conn(0x7fa05c0b21a0 msgr2=0x7fa05c0cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 shutdown_connections 2024-04-23T14:39:41.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:41.877+0000 7fa063200640 1 -- 172.21.15.59:0/1431255509 wait complete. 2024-04-23T14:39:41.898 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-23T14:39:41.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:41.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-23T14:39:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.115+0000 7f63fe200640 1 Processor -- start 2024-04-23T14:39:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.115+0000 7f63fe200640 1 -- start start 2024-04-23T14:39:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.116+0000 7f63fe200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63f80d06b0 0x7f63f80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.116+0000 7f63fe200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63f80704d0 0x7f63f806c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.116+0000 7f63fe200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f80d0fd0 con 0x7f63f806ecd0 2024-04-23T14:39:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.116+0000 7f63fe200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f63f8070d20 con 0x7f63f80d06b0 2024-04-23T14:39:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.116+0000 7f63fe200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f63f815fce0 con 0x7f63f80704d0 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63fce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63f80d06b0 0x7f63f80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63fd800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f63f806ecd0 0x7f63f806f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44478/0 (socket says 172.21.15.59:44478) 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63fd800640 1 -- 172.21.15.59:0/1297232003 learned_addr learned my addr 172.21.15.59:0/1297232003 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63f7e00640 1 --2- 172.21.15.59:0/1297232003 >> v2:172.21.15.76:3300/0 conn(0x7f63f80704d0 0x7f63f806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63fce00640 1 -- 172.21.15.59:0/1297232003 >> v2:172.21.15.76:3300/0 conn(0x7f63f80704d0 msgr2=0x7f63f806c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63fce00640 1 --2- 172.21.15.59:0/1297232003 >> v2:172.21.15.76:3300/0 conn(0x7f63f80704d0 0x7f63f806c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63fce00640 1 -- 172.21.15.59:0/1297232003 >> v1:172.21.15.59:6789/0 conn(0x7f63f806ecd0 legacy=0x7f63f806f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.118+0000 7f63fce00640 1 -- 172.21.15.59:0/1297232003 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f815ef20 con 0x7f63f80d06b0 2024-04-23T14:39:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.117+0000 7f63f7400640 1 -- 172.21.15.59:0/1297232003 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552036909 0 0) 0x7f63f80d0fd0 con 0x7f63f806ecd0 2024-04-23T14:39:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.119+0000 7f63fce00640 1 --2- 172.21.15.59:0/1297232003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80d06b0 0x7f63f80d0a90 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f63e00097a0 tx=0x7f63e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7605efba27653b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.119+0000 7f63f7400640 1 -- 172.21.15.59:0/1297232003 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0037070 con 0x7f63f80d06b0 2024-04-23T14:39:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.119+0000 7f63f7400640 1 -- 172.21.15.59:0/1297232003 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e0032440 con 0x7f63f80d06b0 2024-04-23T14:39:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.119+0000 7f63f7400640 1 -- 172.21.15.59:0/1297232003 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0032810 con 0x7f63f80d06b0 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 -- 172.21.15.59:0/1297232003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80d06b0 msgr2=0x7f63f80d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 --2- 172.21.15.59:0/1297232003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80d06b0 0x7f63f80d0a90 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f63e00097a0 tx=0x7f63e002a1e0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 -- 172.21.15.59:0/1297232003 shutdown_connections 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 --2- 172.21.15.59:0/1297232003 >> v2:172.21.15.76:3300/0 conn(0x7f63f80704d0 0x7f63f806c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 --2- 172.21.15.59:0/1297232003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80d06b0 0x7f63f80d0a90 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 -- 172.21.15.59:0/1297232003 >> 172.21.15.59:0/1297232003 conn(0x7f63f80d12e0 msgr2=0x7f63f80d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 -- 172.21.15.59:0/1297232003 shutdown_connections 2024-04-23T14:39:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.120+0000 7f63fe200640 1 -- 172.21.15.59:0/1297232003 wait complete. 2024-04-23T14:39:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.122+0000 7f63fe200640 1 Processor -- start 2024-04-23T14:39:42.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.122+0000 7f63fe200640 1 -- start start 2024-04-23T14:39:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fe200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63f806ecd0 0x7f63f814d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fe200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80704d0 0x7f63f815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fe200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63f80d06b0 0x7f63f80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fe200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63f815f7c0 con 0x7f63f80704d0 2024-04-23T14:39:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fe200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63f815f4e0 con 0x7f63f806ecd0 2024-04-23T14:39:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fe200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f63f806c7d0 con 0x7f63f80d06b0 2024-04-23T14:39:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80704d0 0x7f63f815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80704d0 0x7f63f815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41604/0 (socket says 172.21.15.59:41604) 2024-04-23T14:39:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63f7e00640 1 -- 172.21.15.59:0/2313511342 learned_addr learned my addr 172.21.15.59:0/2313511342 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fce00640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63f806ecd0 0x7f63f814d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.123+0000 7f63fd800640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63f80d06b0 0x7f63f80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f7e00640 1 -- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63f80d06b0 msgr2=0x7f63f80b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f7e00640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63f80d06b0 0x7f63f80b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f7e00640 1 -- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63f806ecd0 msgr2=0x7f63f814d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f7e00640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63f806ecd0 0x7f63f814d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f7e00640 1 -- 172.21.15.59:0/2313511342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f815b0f0 con 0x7f63f80704d0 2024-04-23T14:39:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63fd800640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63f80d06b0 0x7f63f80b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f7e00640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80704d0 0x7f63f815a8a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f63e8007ef0 tx=0x7f63e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f4c00640 1 -- 172.21.15.59:0/2313511342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8014620 con 0x7f63f80704d0 2024-04-23T14:39:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f4c00640 1 -- 172.21.15.59:0/2313511342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e8019070 con 0x7f63f80704d0 2024-04-23T14:39:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63f4c00640 1 -- 172.21.15.59:0/2313511342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8014a00 con 0x7f63f80704d0 2024-04-23T14:39:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.124+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f815b3d0 con 0x7f63f80704d0 2024-04-23T14:39:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.125+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f815b990 con 0x7f63f80704d0 2024-04-23T14:39:42.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.128+0000 7f63f4c00640 1 -- 172.21.15.59:0/2313511342 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63e8014ba0 con 0x7f63f80704d0 2024-04-23T14:39:42.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.128+0000 7f63f4c00640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63d406c920 0x7f63d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.129+0000 7f63f4c00640 1 -- 172.21.15.59:0/2313511342 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f63e808c4d0 con 0x7f63f80704d0 2024-04-23T14:39:42.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.129+0000 7f63fce00640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63d406c920 0x7f63d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.129+0000 7f640264f740 1 Processor -- start 2024-04-23T14:39:42.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.129+0000 7f640264f740 1 -- start start 2024-04-23T14:39:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f63fce00640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63d406c920 0x7f63d406ede0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f63e0034000 tx=0x7f63e002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b05ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f640264f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b090760 0x55728b08fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090020 0x55728b064e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f640264f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55728afc9e70 con 0x55728b090760 2024-04-23T14:39:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55728af70380 con 0x55728b0ad640 2024-04-23T14:39:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55728afc5e30 con 0x55728b090020 2024-04-23T14:39:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b090760 0x55728b08fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b090760 0x55728b08fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41612/0 (socket says 172.21.15.59:41612) 2024-04-23T14:39:42.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.130+0000 7f63f7e00640 1 -- 172.21.15.59:0/41063148 learned_addr learned my addr 172.21.15.59:0/41063148 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63fce00640 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b05ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63fd800640 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090020 0x55728b064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63f7e00640 1 -- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090020 msgr2=0x55728b064e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63f7e00640 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090020 0x55728b064e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63f7e00640 1 -- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 msgr2=0x55728b05ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63f7e00640 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b05ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63f7e00640 1 -- 172.21.15.59:0/41063148 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55728ada47e0 con 0x55728b090760 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63fd800640 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090020 0x55728b064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63f7e00640 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b090760 0x55728b08fae0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f63e8062850 tx=0x7f63e805a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180977cfcb0c6c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.131+0000 7f63ed600640 1 -- 172.21.15.59:0/41063148 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8062d80 con 0x55728b090760 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f63ed600640 1 -- 172.21.15.59:0/41063148 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e80584a0 con 0x55728b090760 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f63ed600640 1 -- 172.21.15.59:0/41063148 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e805e120 con 0x55728b090760 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 -- 172.21.15.59:0/41063148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b090760 msgr2=0x55728b08fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b090760 0x55728b08fae0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f63e8062850 tx=0x7f63e805a8a0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 -- 172.21.15.59:0/41063148 shutdown_connections 2024-04-23T14:39:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090020 0x55728b064e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b090760 0x55728b08fae0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 --2- 172.21.15.59:0/41063148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b05ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 -- 172.21.15.59:0/41063148 >> 172.21.15.59:0/41063148 conn(0x55728b062240 msgr2=0x55728b0aaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 -- 172.21.15.59:0/41063148 shutdown_connections 2024-04-23T14:39:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.132+0000 7f640264f740 1 -- 172.21.15.59:0/41063148 wait complete. 2024-04-23T14:39:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.133+0000 7f640264f740 1 -- start start 2024-04-23T14:39:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.133+0000 7f640264f740 1 Processor -- start 2024-04-23T14:39:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.134+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 0x55728b087ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.134+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b0becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.134+0000 7f640264f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.134+0000 7f640264f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55728b0643f0 con 0x55728b0a3250 2024-04-23T14:39:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.134+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55728af83660 con 0x55728b0ad640 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.134+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55728b0637e0 con 0x55728b090760 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f640264f740 10 client.0 _ll_register_callbacks cb 0x55728af71aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41626/0 (socket says 172.21.15.59:41626) 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 -- 172.21.15.59:0/830302667 learned_addr learned my addr 172.21.15.59:0/830302667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fce00640 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 0x55728b087ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63f7e00640 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b0becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 -- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 msgr2=0x55728b087ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 0x55728b087ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 -- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 msgr2=0x55728b0becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b0becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.135+0000 7f63fd800640 1 -- 172.21.15.59:0/830302667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728ae04f70 con 0x55728b0a3250 2024-04-23T14:39:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f63fd800640 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0a36d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f63f0001e40 tx=0x7f63f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88ed34f4f6e4284 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f63e6000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f63e6000640 1 -- 172.21.15.59:0/830302667 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0019060 con 0x55728b0a3250 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f63e6000640 1 -- 172.21.15.59:0/830302667 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f63f0014b60 con 0x55728b0a3250 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e80584a0 con 0x55728b0a3250 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f63e6000640 1 -- 172.21.15.59:0/830302667 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f63f0014d60 con 0x55728b0a3250 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f640264f740 10 client.7986 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.136+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55728ac9c170 con 0x55728b0a3250 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f63e6000640 1 -- 172.21.15.59:0/830302667 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e80584a0 con 0x55728b0a3250 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f63e6000640 1 -- 172.21.15.59:0/830302667 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f63f00072d0 con 0x55728b0a3250 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f640264f740 10 client.7986 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f640264f740 10 client.7986 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f640264f740 20 client.7986 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f640264f740 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b091120 0x55728b091530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f640264f740 4 client.7986 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:39:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55728afd7890 con 0x55728b091120 2024-04-23T14:39:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.137+0000 7f63fce00640 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b091120 0x55728b091530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.138+0000 7f63fce00640 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b091120 0x55728b091530 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990c6eb29a3fbfa1 server_cookie=2d031adc19a50910 in_seq=0 out_seq=0 2024-04-23T14:39:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.138+0000 7f63e6000640 10 client.7986 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f63e6000640 1 -- 172.21.15.59:0/830302667 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55728afd7890 con 0x55728b091120 2024-04-23T14:39:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f63e6000640 10 client.7986 handle_command_reply: tid=0 2024-04-23T14:39:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 client.7986 shutdown 2024-04-23T14:39:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 msgr2=0x55728b0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0a36d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f63f0001e40 tx=0x7f63f0001e70 comp rx=0 tx=0).stop 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 shutdown_connections 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b091120 0x55728b091530 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0a36d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b0becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 --2- 172.21.15.59:0/830302667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 0x55728b087ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 >> 172.21.15.59:0/830302667 conn(0x55728b090020 msgr2=0x55728b0adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f63e6000640 0 client.7986 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 shutdown_connections 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 1 -- 172.21.15.59:0/830302667 wait complete. 2024-04-23T14:39:42.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.139+0000 7f640264f740 20 client.7986 trim_cache size 0 max 16384 2024-04-23T14:39:42.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.159+0000 7f640264f740 1 Processor -- start 2024-04-23T14:39:42.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.159+0000 7f640264f740 1 -- start start 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 0x55728b0ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f640264f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b0be2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f640264f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8200cd0 con 0x55728b0a3250 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81ffd20 con 0x55728b0ad640 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f63f8159dd0 con 0x55728b090760 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0bddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41628/0 (socket says 172.21.15.59:41628) 2024-04-23T14:39:42.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f63f7e00640 1 -- 172.21.15.59:0/3952300023 learned_addr learned my addr 172.21.15.59:0/3952300023 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f63f7e00640 1 -- 172.21.15.59:0/3952300023 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 msgr2=0x55728b0ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f63f7e00640 1 --2- 172.21.15.59:0/3952300023 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 0x55728b0ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f63f7e00640 1 -- 172.21.15.59:0/3952300023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 msgr2=0x55728b0be2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.160+0000 7f63f7e00640 1 --2- 172.21.15.59:0/3952300023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b0be2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.161+0000 7f63f7e00640 1 -- 172.21.15.59:0/3952300023 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55728b08c3e0 con 0x55728b0a3250 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.161+0000 7f63f7e00640 1 --2- 172.21.15.59:0/3952300023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0bddb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f63e8056600 tx=0x7f63e8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a265cded09d08147 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.161+0000 7f63ecc00640 1 -- 172.21.15.59:0/3952300023 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8062d80 con 0x55728b0a3250 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.161+0000 7f63ecc00640 1 -- 172.21.15.59:0/3952300023 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e80584a0 con 0x55728b0a3250 2024-04-23T14:39:42.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.161+0000 7f63ecc00640 1 -- 172.21.15.59:0/3952300023 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e805e120 con 0x55728b0a3250 2024-04-23T14:39:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 -- 172.21.15.59:0/3952300023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 msgr2=0x55728b0bddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 --2- 172.21.15.59:0/3952300023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0bddb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f63e8056600 tx=0x7f63e8015000 comp rx=0 tx=0).stop 2024-04-23T14:39:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 -- 172.21.15.59:0/3952300023 shutdown_connections 2024-04-23T14:39:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 --2- 172.21.15.59:0/3952300023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b0ad640 0x55728b0be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 --2- 172.21.15.59:0/3952300023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0bddb0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 --2- 172.21.15.59:0/3952300023 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b090760 0x55728b0ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 -- 172.21.15.59:0/3952300023 >> 172.21.15.59:0/3952300023 conn(0x55728b090020 msgr2=0x55728b092180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 -- 172.21.15.59:0/3952300023 shutdown_connections 2024-04-23T14:39:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.162+0000 7f640264f740 1 -- 172.21.15.59:0/3952300023 wait complete. 2024-04-23T14:39:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.163+0000 7f640264f740 1 -- start start 2024-04-23T14:39:42.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.163+0000 7f640264f740 1 Processor -- start 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b090760 0x55728b0aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f640264f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f640264f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b0ad640 0x55728b0cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f640264f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55728afd7890 con 0x55728b0a3250 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81f5130 con 0x55728b090760 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f640264f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55728b0a2be0 con 0x55728b0ad640 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f640264f740 10 client.0 _ll_register_callbacks cb 0x7f63f0007bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f63f7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41634/0 (socket says 172.21.15.59:41634) 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f63f7e00640 1 -- 172.21.15.59:0/4249212357 learned_addr learned my addr 172.21.15.59:0/4249212357 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f63fce00640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b090760 0x55728b0aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f63fd800640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b0ad640 0x55728b0cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f63f7e00640 1 -- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b0ad640 msgr2=0x55728b0cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.164+0000 7f63f7e00640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b0ad640 0x55728b0cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63f7e00640 1 -- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b090760 msgr2=0x55728b0aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63f7e00640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b090760 0x55728b0aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63f7e00640 1 -- 172.21.15.59:0/4249212357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728b0a0c80 con 0x55728b0a3250 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63fce00640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b090760 0x55728b0aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63f7e00640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0aabc0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f63e8056600 tx=0x7f63e80096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d213cb45dac0fc36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63e6000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63e6000640 1 -- 172.21.15.59:0/4249212357 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e805e320 con 0x55728b0a3250 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63e6000640 1 -- 172.21.15.59:0/4249212357 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f63e808c0b0 con 0x55728b0a3250 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e80584a0 con 0x55728b0a3250 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63e6000640 1 -- 172.21.15.59:0/4249212357 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f63e808c2b0 con 0x55728b0a3250 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f640264f740 10 client.7992 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55728b0a0a00 con 0x55728b0a3250 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.165+0000 7f63e6000640 1 -- 172.21.15.59:0/4249212357 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e80584a0 con 0x55728b0a3250 2024-04-23T14:39:42.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f63e6000640 1 -- 172.21.15.59:0/4249212357 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f63e805a400 con 0x55728b0a3250 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f640264f740 10 client.7992 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f640264f740 10 client.7992 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f640264f740 20 client.7992 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f640264f740 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b1244d0 0x55728b0a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f640264f740 4 client.7992 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55728afd1700 con 0x55728b1244d0 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.166+0000 7f63fce00640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b1244d0 0x55728b0a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f63fce00640 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b1244d0 0x55728b0a2be0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adab11bcbd895551 server_cookie=b3ac2cae8aa4fb6e in_seq=0 out_seq=0 2024-04-23T14:39:42.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f63e6000640 10 client.7992 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f63e6000640 1 -- 172.21.15.59:0/4249212357 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55728afd1700 con 0x55728b1244d0 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f63e6000640 10 client.7992 handle_command_reply: tid=0 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f640264f740 1 client.7992 shutdown 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 msgr2=0x55728b0aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f640264f740 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0aabc0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f63e8056600 tx=0x7f63e80096f0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 shutdown_connections 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.167+0000 7f640264f740 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55728b1244d0 0x55728b0a2be0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f640264f740 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55728b0ad640 0x55728b0cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f640264f740 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55728b0a3250 0x55728b0aabc0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f640264f740 1 --2- 172.21.15.59:0/4249212357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55728b090760 0x55728b0aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f63e6000640 0 client.7992 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 >> 172.21.15.59:0/4249212357 conn(0x55728b090020 msgr2=0x55728b0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 shutdown_connections 2024-04-23T14:39:42.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f640264f740 1 -- 172.21.15.59:0/4249212357 wait complete. 2024-04-23T14:39:42.172 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:39:42.168+0000 7fedbc800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0)->f(v1 m2024-04-23T14:38:48.802771+0000) n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1)->n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55a610a96100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-23T14:39:42.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.168+0000 7f640264f740 20 client.7992 trim_cache size 0 max 16384 2024-04-23T14:39:42.173 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:39:42.173 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:39:42.173 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "ed4fa53a-77aa-46d5-8a36-c41329825366", 2024-04-23T14:39:42.173 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:39:42.173 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:39:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63d406c920 msgr2=0x7f63d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63d406c920 0x7f63d406ede0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f63e0034000 tx=0x7f63e002e000 comp rx=0 tx=0).stop 2024-04-23T14:39:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80704d0 msgr2=0x7f63f815a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80704d0 0x7f63f815a8a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f63e8007ef0 tx=0x7f63e800e4b0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 shutdown_connections 2024-04-23T14:39:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63d406c920 0x7f63d406ede0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63f80d06b0 0x7f63f80b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63f80704d0 0x7f63f815a8a0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 --2- 172.21.15.59:0/2313511342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63f806ecd0 0x7f63f814d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 >> 172.21.15.59:0/2313511342 conn(0x7f63f80d12e0 msgr2=0x7f63f806f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 shutdown_connections 2024-04-23T14:39:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.169+0000 7f63fe200640 1 -- 172.21.15.59:0/2313511342 wait complete. 2024-04-23T14:39:42.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:39:42.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-23T14:39:42.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.391+0000 7fe9db400640 1 Processor -- start 2024-04-23T14:39:42.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.392+0000 7fe9db400640 1 -- start start 2024-04-23T14:39:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.393+0000 7fe9db400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe9d415eb60 0x7fe9d40d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.393+0000 7fe9db400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40d2d60 0x7fe9d40b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.393+0000 7fe9db400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9d40b3490 0x7fe9d40b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.393+0000 7fe9db400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d40ba7a0 con 0x7fe9d40b3490 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.393+0000 7fe9db400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d40ba1e0 con 0x7fe9d415eb60 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.393+0000 7fe9db400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d40ba4c0 con 0x7fe9d40d2d60 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9d40b3490 0x7fe9d40b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9d40b3490 0x7fe9d40b97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41648/0 (socket says 172.21.15.59:41648) 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 -- 172.21.15.59:0/3493685743 learned_addr learned my addr 172.21.15.59:0/3493685743 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/3493685743 >> v2:172.21.15.76:3300/0 conn(0x7fe9d415eb60 0x7fe9d40d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/3493685743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40d2d60 0x7fe9d40b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 -- 172.21.15.59:0/3493685743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40d2d60 msgr2=0x7fe9d40b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 --2- 172.21.15.59:0/3493685743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40d2d60 0x7fe9d40b9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 -- 172.21.15.59:0/3493685743 >> v2:172.21.15.76:3300/0 conn(0x7fe9d415eb60 msgr2=0x7fe9d40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 --2- 172.21.15.59:0/3493685743 >> v2:172.21.15.76:3300/0 conn(0x7fe9d415eb60 0x7fe9d40d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.394+0000 7fe9d9400640 1 -- 172.21.15.59:0/3493685743 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d40b8a90 con 0x7fe9d40b3490 2024-04-23T14:39:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.395+0000 7fe9d9400640 1 --2- 172.21.15.59:0/3493685743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d40b3490 0x7fe9d40b97d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40d1d00 tx=0x7fe9cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6eab47ada3699e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.396+0000 7fe9d3400640 1 -- 172.21.15.59:0/3493685743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc036070 con 0x7fe9d40b3490 2024-04-23T14:39:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.396+0000 7fe9d3400640 1 -- 172.21.15.59:0/3493685743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc031600 con 0x7fe9d40b3490 2024-04-23T14:39:42.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.396+0000 7fe9d3400640 1 -- 172.21.15.59:0/3493685743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc0319d0 con 0x7fe9d40b3490 2024-04-23T14:39:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 -- 172.21.15.59:0/3493685743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d40b3490 msgr2=0x7fe9d40b97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 --2- 172.21.15.59:0/3493685743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d40b3490 0x7fe9d40b97d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40d1d00 tx=0x7fe9cc02a6a0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 -- 172.21.15.59:0/3493685743 shutdown_connections 2024-04-23T14:39:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 --2- 172.21.15.59:0/3493685743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d40b3490 0x7fe9d40b97d0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 --2- 172.21.15.59:0/3493685743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40d2d60 0x7fe9d40b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 --2- 172.21.15.59:0/3493685743 >> v2:172.21.15.76:3300/0 conn(0x7fe9d415eb60 0x7fe9d40d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 -- 172.21.15.59:0/3493685743 >> 172.21.15.59:0/3493685743 conn(0x7fe9d4067b50 msgr2=0x7fe9d4067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 -- 172.21.15.59:0/3493685743 shutdown_connections 2024-04-23T14:39:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.397+0000 7fe9db400640 1 -- 172.21.15.59:0/3493685743 wait complete. 2024-04-23T14:39:42.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.400+0000 7fe9db400640 1 Processor -- start 2024-04-23T14:39:42.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.400+0000 7fe9db400640 1 -- start start 2024-04-23T14:39:42.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.401+0000 7fe9db400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d40b3490 0x7fe9d414cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.401+0000 7fe9db400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d415eb60 0x7fe9d40d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.401+0000 7fe9db400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40ce100 0x7fe9d40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.401+0000 7fe9db400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d40b6280 con 0x7fe9d415eb60 2024-04-23T14:39:42.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.401+0000 7fe9db400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d40b3c80 con 0x7fe9d40b3490 2024-04-23T14:39:42.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.401+0000 7fe9db400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d40b6400 con 0x7fe9d40ce100 2024-04-23T14:39:42.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d415eb60 0x7fe9d40d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d415eb60 0x7fe9d40d4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41656/0 (socket says 172.21.15.59:41656) 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 -- 172.21.15.59:0/474364292 learned_addr learned my addr 172.21.15.59:0/474364292 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d40b3490 0x7fe9d414cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d9400640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40ce100 0x7fe9d40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 -- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40ce100 msgr2=0x7fe9d40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40ce100 0x7fe9d40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 -- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d40b3490 msgr2=0x7fe9d414cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d40b3490 0x7fe9d414cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.402+0000 7fe9d3e00640 1 -- 172.21.15.59:0/474364292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d41f7bc0 con 0x7fe9d415eb60 2024-04-23T14:39:42.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.403+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d415eb60 0x7fe9d40d4ae0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c7f0 tx=0x7fe9c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.403+0000 7fe9d0c00640 1 -- 172.21.15.59:0/474364292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0015070 con 0x7fe9d415eb60 2024-04-23T14:39:42.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.403+0000 7fe9d0c00640 1 -- 172.21.15.59:0/474364292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c0004240 con 0x7fe9d415eb60 2024-04-23T14:39:42.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.403+0000 7fe9d0c00640 1 -- 172.21.15.59:0/474364292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0007620 con 0x7fe9d415eb60 2024-04-23T14:39:42.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.403+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d41f7ea0 con 0x7fe9d415eb60 2024-04-23T14:39:42.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.403+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d40ce580 con 0x7fe9d415eb60 2024-04-23T14:39:42.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.406+0000 7fe9d0c00640 1 -- 172.21.15.59:0/474364292 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9c00079d0 con 0x7fe9d415eb60 2024-04-23T14:39:42.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.407+0000 7fe9d0c00640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9b406c920 0x7fe9b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.407+0000 7fe9d0c00640 1 -- 172.21.15.59:0/474364292 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fe9c008bcb0 con 0x7fe9d415eb60 2024-04-23T14:39:42.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.407+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9b406c920 0x7fe9b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.407+0000 7fe9de1bf740 1 Processor -- start 2024-04-23T14:39:42.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.407+0000 7fe9de1bf740 1 -- start start 2024-04-23T14:39:42.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.408+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9b406c920 0x7fe9b406ede0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.408+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4d6a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.408+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d9a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.408+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9a610 0x55c0a4d6f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.408+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c0a4cd41f0 con 0x55c0a4db7c30 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.408+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c0a4c7a350 con 0x55c0a4d9ad50 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.408+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c0a4cd01b0 con 0x55c0a4d9a610 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4d6a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4d6a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41658/0 (socket says 172.21.15.59:41658) 2024-04-23T14:39:42.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3943837595 learned_addr learned my addr 172.21.15.59:0/3943837595 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d9a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9a610 msgr2=0x55c0a4d6f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9a610 0x55c0a4d6f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 msgr2=0x55c0a4d9a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d9a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3943837595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a4ccef50 con 0x55c0a4db7c30 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d9a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.409+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4d6a5c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800f990 tx=0x7fe9c8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100793a882c3d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.410+0000 7fe9c5200640 1 -- 172.21.15.59:0/3943837595 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8023070 con 0x55c0a4db7c30 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.410+0000 7fe9c5200640 1 -- 172.21.15.59:0/3943837595 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c8019070 con 0x55c0a4db7c30 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.410+0000 7fe9c5200640 1 -- 172.21.15.59:0/3943837595 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8014de0 con 0x55c0a4db7c30 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.410+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3943837595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 msgr2=0x55c0a4d6a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.410+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4d6a5c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800f990 tx=0x7fe9c8012000 comp rx=0 tx=0).stop 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.411+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3943837595 shutdown_connections 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.411+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9a610 0x55c0a4d6f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.411+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d9a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.411+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3943837595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4d6a5c0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.411+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3943837595 >> 172.21.15.59:0/3943837595 conn(0x55c0a4d6c830 msgr2=0x55c0a4db50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.411+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3943837595 shutdown_connections 2024-04-23T14:39:42.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.411+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3943837595 wait complete. 2024-04-23T14:39:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.412+0000 7fe9de1bf740 1 -- start start 2024-04-23T14:39:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.412+0000 7fe9de1bf740 1 Processor -- start 2024-04-23T14:39:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9ad50 0x55c0a4d87a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4dabd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9da80 0x55c0a4dad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c0a4d6e9e0 con 0x55c0a4d9da80 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c0a4c8d690 con 0x55c0a4db7c30 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c0a4d6ddd0 con 0x55c0a4d9ad50 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9de1bf740 10 client.0 _ll_register_callbacks cb 0x55c0a4cc9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9da80 0x55c0a4dad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9da80 0x55c0a4dad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41666/0 (socket says 172.21.15.59:41666) 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9d9400640 1 -- 172.21.15.59:0/1213703670 learned_addr learned my addr 172.21.15.59:0/1213703670 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4dabd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9d9400640 1 -- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9ad50 msgr2=0x55c0a4d87a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9ad50 0x55c0a4d87a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.413+0000 7fe9d9400640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9ad50 0x55c0a4d87a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9d9400640 1 -- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 msgr2=0x55c0a4dabd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9d9400640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4dabd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9d9400640 1 -- 172.21.15.59:0/1213703670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a4c8c920 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4dabd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9d9400640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9da80 0x55c0a4dad760 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009f40 tx=0x7fe9cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a1b929e481c1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9b9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9b9600640 1 -- 172.21.15.59:0/1213703670 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc041070 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9b9600640 1 -- 172.21.15.59:0/1213703670 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fe9cc031dc0 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c8019070 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9b9600640 1 -- 172.21.15.59:0/1213703670 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe9cc04c050 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9de1bf740 10 client.8004 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.414+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55c0a4ab07d0 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9b9600640 1 -- 172.21.15.59:0/1213703670 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c8019070 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9b9600640 1 -- 172.21.15.59:0/1213703670 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fe9cc04f020 con 0x55c0a4d9da80 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9de1bf740 10 client.8004 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9de1bf740 10 client.8004 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9de1bf740 20 client.8004 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:42.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d70300 0x55c0a4d9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9de1bf740 4 client.8004 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:39:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0a4ce1c10 con 0x55c0a4d70300 2024-04-23T14:39:42.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.415+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d70300 0x55c0a4d9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.416+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d70300 0x55c0a4d9b240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c931f336b9caed server_cookie=2656459c3a8f9db7 in_seq=0 out_seq=0 2024-04-23T14:39:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.416+0000 7fe9b9600640 10 client.8004 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9b9600640 1 -- 172.21.15.59:0/1213703670 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0a4ce1c10 con 0x55c0a4d70300 2024-04-23T14:39:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9b9600640 10 client.8004 handle_command_reply: tid=0 2024-04-23T14:39:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 client.8004 shutdown 2024-04-23T14:39:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9da80 msgr2=0x55c0a4dad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9da80 0x55c0a4dad760 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009f40 tx=0x7fe9cc004a40 comp rx=0 tx=0).stop 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 shutdown_connections 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d70300 0x55c0a4d9b240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9da80 0x55c0a4dad760 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4dabd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/1213703670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9ad50 0x55c0a4d87a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 >> 172.21.15.59:0/1213703670 conn(0x55c0a4d9a610 msgr2=0x55c0a4db6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9b9600640 0 client.8004 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 shutdown_connections 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.417+0000 7fe9de1bf740 1 -- 172.21.15.59:0/1213703670 wait complete. 2024-04-23T14:39:42.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.418+0000 7fe9de1bf740 20 client.8004 trim_cache size 0 max 16384 2024-04-23T14:39:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.437+0000 7fe9de1bf740 1 Processor -- start 2024-04-23T14:39:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.437+0000 7fe9de1bf740 1 -- start start 2024-04-23T14:39:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4db84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 0x55c0a4dd8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4db45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4201030 con 0x55c0a4db7c30 2024-04-23T14:39:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d42000b0 con 0x55c0a4d9ad50 2024-04-23T14:39:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d40c9ff0 con 0x55c0a4d9da80 2024-04-23T14:39:42.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4db45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9d9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4db45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41682/0 (socket says 172.21.15.59:41682) 2024-04-23T14:39:42.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9d9400640 1 -- 172.21.15.59:0/661394352 learned_addr learned my addr 172.21.15.59:0/661394352 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.438+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4db84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9d9400640 1 -- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 msgr2=0x55c0a4dd8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9d9400640 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 0x55c0a4dd8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9d9400640 1 -- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 msgr2=0x55c0a4db84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9d9400640 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4db84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9d9400640 1 -- 172.21.15.59:0/661394352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a49e63f0 con 0x55c0a4db7c30 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4db84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9d9400640 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4db45a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009f40 tx=0x7fe9cc03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542d13c66627a7d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9bbe00640 1 -- 172.21.15.59:0/661394352 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc041070 con 0x55c0a4db7c30 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9bbe00640 1 -- 172.21.15.59:0/661394352 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc031dc0 con 0x55c0a4db7c30 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.439+0000 7fe9bbe00640 1 -- 172.21.15.59:0/661394352 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc036040 con 0x55c0a4db7c30 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 -- 172.21.15.59:0/661394352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 msgr2=0x55c0a4db45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4db45a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009f40 tx=0x7fe9cc03e000 comp rx=0 tx=0).stop 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 -- 172.21.15.59:0/661394352 shutdown_connections 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 0x55c0a4dd8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4db7c30 0x55c0a4db45a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/661394352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4d9ad50 0x55c0a4db84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 -- 172.21.15.59:0/661394352 >> 172.21.15.59:0/661394352 conn(0x55c0a4d9a610 msgr2=0x55c0a4dcee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 -- 172.21.15.59:0/661394352 shutdown_connections 2024-04-23T14:39:42.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.440+0000 7fe9de1bf740 1 -- 172.21.15.59:0/661394352 wait complete. 2024-04-23T14:39:42.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.441+0000 7fe9de1bf740 1 -- start start 2024-04-23T14:39:42.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.441+0000 7fe9de1bf740 1 Processor -- start 2024-04-23T14:39:42.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 0x55c0a4d6a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9de1bf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4e372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c0a4d960c0 con 0x55c0a4d9ad50 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d40ca640 con 0x55c0a4db7c30 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9de1bf740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c0a4dad1a0 con 0x55c0a4d9da80 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9de1bf740 10 client.0 _ll_register_callbacks cb 0x7fe9cc034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9d8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9d8a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d6a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41694/0 (socket says 172.21.15.59:41694) 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.442+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3474770356 learned_addr learned my addr 172.21.15.59:0/3474770356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3474770356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 msgr2=0x55c0a4d6a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 0x55c0a4d6a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3474770356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 msgr2=0x55c0a4e372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4e372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9d8a00640 1 -- 172.21.15.59:0/3474770356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a49a60f0 con 0x55c0a4d9ad50 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9d8a00640 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d6a190 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800f930 tx=0x7fe9c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b75eb066791d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:42.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9b9600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9b9600640 1 -- 172.21.15.59:0/3474770356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8023040 con 0x55c0a4d9ad50 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9b9600640 1 -- 172.21.15.59:0/3474770356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fe9c8032400 con 0x55c0a4d9ad50 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.443+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9cc031dc0 con 0x55c0a4d9ad50 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9b9600640 1 -- 172.21.15.59:0/3474770356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fe9c803f050 con 0x55c0a4d9ad50 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 10 client.8010 fetch_fsmap learned FSMap version 67 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55c0a4c99810 con 0x55c0a4d9ad50 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9b9600640 1 -- 172.21.15.59:0/3474770356 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc031dc0 con 0x55c0a4d9ad50 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9b9600640 1 -- 172.21.15.59:0/3474770356 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fe9c8042020 con 0x55c0a4d9ad50 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 10 client.8010 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 10 client.8010 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 20 client.8010 populate_metadata read hostname 'smithi059' 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d97210 0x55c0a4e37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 4 client.8010 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c0a4cdba80 con 0x55c0a4d97210 2024-04-23T14:39:42.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.444+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d97210 0x55c0a4e37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:42.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9d3e00640 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d97210 0x55c0a4e37830 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9832ab6bec6038e1 server_cookie=f6193560099d5c52 in_seq=0 out_seq=0 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9b9600640 10 client.8010 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9b9600640 1 -- 172.21.15.59:0/3474770356 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55c0a4cdba80 con 0x55c0a4d97210 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9b9600640 10 client.8010 handle_command_reply: tid=0 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 client.8010 shutdown 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9ad50 msgr2=0x55c0a4d6a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d6a190 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800f930 tx=0x7fe9c8004730 comp rx=0 tx=0).stop 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 shutdown_connections 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55c0a4d97210 0x55c0a4e37830 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c0a4d9da80 0x55c0a4d6a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c0a4db7c30 0x55c0a4e372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 --2- 172.21.15.59:0/3474770356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c0a4d9ad50 0x55c0a4d6a190 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9b9600640 0 client.8010 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.445+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 >> 172.21.15.59:0/3474770356 conn(0x55c0a4d9a610 msgr2=0x55c0a4dd8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.446+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 shutdown_connections 2024-04-23T14:39:42.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.446+0000 7fe9de1bf740 1 -- 172.21.15.59:0/3474770356 wait complete. 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.446+0000 7fe9de1bf740 20 client.8010 trim_cache size 0 max 16384 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: "ed4fa53a-77aa-46d5-8a36-c41329825366": { 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir", 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "ed4fa53a-77aa-46d5-8a36-c41329825366", 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:39:42.450 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.446+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9b406c920 msgr2=0x7fe9b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.446+0000 7fe9db400640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9b406c920 0x7fe9b406ede0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).stop 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.446+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d415eb60 msgr2=0x7fe9d40d4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.446+0000 7fe9db400640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d415eb60 0x7fe9d40d4ae0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c7f0 tx=0x7fe9c000ccb0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 shutdown_connections 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe9b406c920 0x7fe9b406ede0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d40ce100 0x7fe9d40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d415eb60 0x7fe9d40d4ae0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 --2- 172.21.15.59:0/474364292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d40b3490 0x7fe9d414cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:42.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 >> 172.21.15.59:0/474364292 conn(0x7fe9d4067b50 msgr2=0x7fe9d406d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:42.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 shutdown_connections 2024-04-23T14:39:42.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:39:42.447+0000 7fe9db400640 1 -- 172.21.15.59:0/474364292 wait complete. 2024-04-23T14:39:42.465 INFO:tasks.cephfs.filesystem:scrub status for tag:ed4fa53a-77aa-46d5-8a36-c41329825366 - {'path': '/dir', 'tag': 'ed4fa53a-77aa-46d5-8a36-c41329825366', 'options': 'recursive,repair'} 2024-04-23T14:39:50.813 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:39:50.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:40:00.000 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:39:59.999+0000 7f3222800640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-23T14:40:12.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:12.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-23T14:40:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.708+0000 7f3007000640 1 Processor -- start 2024-04-23T14:40:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.708+0000 7f3007000640 1 -- start start 2024-04-23T14:40:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.710+0000 7f3007000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.710+0000 7f3007000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30000d0b10 0x7f30000af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.710+0000 7f3007000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3000160120 con 0x7f30000af5c0 2024-04-23T14:40:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.710+0000 7f3007000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f30000d0f70 con 0x7f30000d0b10 2024-04-23T14:40:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.710+0000 7f3007000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3000160400 con 0x7f300015c410 2024-04-23T14:40:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.710+0000 7f3005c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.711+0000 7f3005c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:47432/0 (socket says 172.21.15.59:47432) 2024-04-23T14:40:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.711+0000 7f3005c00640 1 -- 172.21.15.59:0/3205474984 learned_addr learned my addr 172.21.15.59:0/3205474984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.711+0000 7f3005200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30000d0b10 0x7f30000af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.711+0000 7f3005c00640 1 -- 172.21.15.59:0/3205474984 >> v2:172.21.15.76:3300/0 conn(0x7f30000d0b10 msgr2=0x7f30000af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.711+0000 7f3005c00640 1 --2- 172.21.15.59:0/3205474984 >> v2:172.21.15.76:3300/0 conn(0x7f30000d0b10 0x7f30000af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.711+0000 7f3005c00640 1 -- 172.21.15.59:0/3205474984 >> v1:172.21.15.76:6789/0 conn(0x7f30000af5c0 legacy=0x7f30000b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.711+0000 7f3005c00640 1 -- 172.21.15.59:0/3205474984 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001606e0 con 0x7f300015c410 2024-04-23T14:40:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.713+0000 7f3005c00640 1 --2- 172.21.15.59:0/3205474984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000d05d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009870 tx=0x7f2ff402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463de1b178626075 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.713+0000 7f2fffe00640 1 -- 172.21.15.59:0/3205474984 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4037070 con 0x7f300015c410 2024-04-23T14:40:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.713+0000 7f2fffe00640 1 -- 172.21.15.59:0/3205474984 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff4032440 con 0x7f300015c410 2024-04-23T14:40:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.713+0000 7f2fffe00640 1 -- 172.21.15.59:0/3205474984 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4032810 con 0x7f300015c410 2024-04-23T14:40:12.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 -- 172.21.15.59:0/3205474984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 msgr2=0x7f30000d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 --2- 172.21.15.59:0/3205474984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000d05d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009870 tx=0x7f2ff402a270 comp rx=0 tx=0).stop 2024-04-23T14:40:12.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 -- 172.21.15.59:0/3205474984 shutdown_connections 2024-04-23T14:40:12.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 --2- 172.21.15.59:0/3205474984 >> v2:172.21.15.76:3300/0 conn(0x7f30000d0b10 0x7f30000af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 --2- 172.21.15.59:0/3205474984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000d05d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 -- 172.21.15.59:0/3205474984 >> 172.21.15.59:0/3205474984 conn(0x7f300006bdf0 msgr2=0x7f300006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:12.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 -- 172.21.15.59:0/3205474984 shutdown_connections 2024-04-23T14:40:12.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.714+0000 7f3007000640 1 -- 172.21.15.59:0/3205474984 wait complete. 2024-04-23T14:40:12.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.716+0000 7f3007000640 1 Processor -- start 2024-04-23T14:40:12.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.716+0000 7f3007000640 1 -- start start 2024-04-23T14:40:12.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3007000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30000af5c0 0x7f30000cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3007000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30000d0b10 0x7f30001f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3007000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3007000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f300015c7f0 con 0x7f30000af5c0 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3007000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30000afa50 con 0x7f30000d0b10 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3007000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30000ce510 con 0x7f300015c410 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3005c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30000af5c0 0x7f30000cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3005c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30000af5c0 0x7f30000cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57184/0 (socket says 172.21.15.59:57184) 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.717+0000 7f3005c00640 1 -- 172.21.15.59:0/2257043316 learned_addr learned my addr 172.21.15.59:0/2257043316 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3006600640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005200640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30000d0b10 0x7f30001f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005c00640 1 -- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 msgr2=0x7f30000b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005c00640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005c00640 1 -- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30000d0b10 msgr2=0x7f30001f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005c00640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30000d0b10 0x7f30001f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005c00640 1 -- 172.21.15.59:0/2257043316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30000cf3a0 con 0x7f30000af5c0 2024-04-23T14:40:12.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005200640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30000d0b10 0x7f30001f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:12.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.718+0000 7f3005c00640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30000af5c0 0x7f30000cdfd0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2ff402a2b0 tx=0x7f2ff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.719+0000 7f2ffd600640 1 -- 172.21.15.59:0/2257043316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4037040 con 0x7f30000af5c0 2024-04-23T14:40:12.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.719+0000 7f2ffd600640 1 -- 172.21.15.59:0/2257043316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff4042070 con 0x7f30000af5c0 2024-04-23T14:40:12.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.719+0000 7f2ffd600640 1 -- 172.21.15.59:0/2257043316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4030860 con 0x7f30000af5c0 2024-04-23T14:40:12.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.719+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30000ce790 con 0x7f30000af5c0 2024-04-23T14:40:12.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.719+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30000ced50 con 0x7f30000af5c0 2024-04-23T14:40:12.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.721+0000 7f2ffd600640 1 -- 172.21.15.59:0/2257043316 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ff4030a00 con 0x7f30000af5c0 2024-04-23T14:40:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.722+0000 7f2ffd600640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2fe006c8f0 0x7f2fe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.722+0000 7f2ffd600640 1 -- 172.21.15.59:0/2257043316 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f2ff40ac520 con 0x7f30000af5c0 2024-04-23T14:40:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.722+0000 7f3005200640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2fe006c8f0 0x7f2fe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.723+0000 7f300b41b740 1 Processor -- start 2024-04-23T14:40:12.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.723+0000 7f300b41b740 1 -- start start 2024-04-23T14:40:12.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.723+0000 7f3005200640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2fe006c8f0 0x7f2fe006edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30000cc430 tx=0x7f2fec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f300b41b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9346a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b9349a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349a7e0 0x558b9346f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f300b41b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b933d41f0 con 0x558b934b7e00 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558b9337a350 con 0x558b9349af20 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558b933d01b0 con 0x558b9349a7e0 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f3005c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9346a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f3005c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9346a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57192/0 (socket says 172.21.15.59:57192) 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.724+0000 7f3005c00640 1 -- 172.21.15.59:0/1082507975 learned_addr learned my addr 172.21.15.59:0/1082507975 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3006600640 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349a7e0 0x558b9346f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3005200640 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b9349a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3005c00640 1 -- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349a7e0 msgr2=0x558b9346f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3005c00640 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349a7e0 0x558b9346f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3005c00640 1 -- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 msgr2=0x558b9349a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3005c00640 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b9349a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3005c00640 1 -- 172.21.15.59:0/1082507975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b933cef50 con 0x558b934b7e00 2024-04-23T14:40:12.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f3005c00640 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9346a790 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40aa8a0 tx=0x7f2ff407a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bb8d87dfa43ece server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f2ff2000640 1 -- 172.21.15.59:0/1082507975 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff407a160 con 0x558b934b7e00 2024-04-23T14:40:12.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f2ff2000640 1 -- 172.21.15.59:0/1082507975 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff407e510 con 0x558b934b7e00 2024-04-23T14:40:12.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.725+0000 7f2ff2000640 1 -- 172.21.15.59:0/1082507975 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4082d40 con 0x558b934b7e00 2024-04-23T14:40:12.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 -- 172.21.15.59:0/1082507975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 msgr2=0x558b9346a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9346a790 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40aa8a0 tx=0x7f2ff407a060 comp rx=0 tx=0).stop 2024-04-23T14:40:12.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 -- 172.21.15.59:0/1082507975 shutdown_connections 2024-04-23T14:40:12.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349a7e0 0x558b9346f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b9349a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 --2- 172.21.15.59:0/1082507975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9346a790 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 -- 172.21.15.59:0/1082507975 >> 172.21.15.59:0/1082507975 conn(0x558b9346ca00 msgr2=0x558b934b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 -- 172.21.15.59:0/1082507975 shutdown_connections 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.726+0000 7f300b41b740 1 -- 172.21.15.59:0/1082507975 wait complete. 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.727+0000 7f300b41b740 1 -- start start 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.727+0000 7f300b41b740 1 Processor -- start 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.728+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349af20 0x558b934abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.728+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b934b7e00 0x558b934adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.728+0000 7f300b41b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b93496040 0x558b934ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.728+0000 7f300b41b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b9346ebb0 con 0x558b93496040 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.728+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558b9338d690 con 0x558b934b7e00 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.728+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558b9346dfa0 con 0x558b9349af20 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.728+0000 7f300b41b740 10 client.0 _ll_register_callbacks cb 0x558b933c9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:12.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b93496040 0x558b934ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b93496040 0x558b934ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57206/0 (socket says 172.21.15.59:57206) 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 -- 172.21.15.59:0/4068382306 learned_addr learned my addr 172.21.15.59:0/4068382306 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3005c00640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349af20 0x558b934abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3005200640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b934b7e00 0x558b934adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 -- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349af20 msgr2=0x558b934abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349af20 0x558b934abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 -- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b934b7e00 msgr2=0x558b934adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b934b7e00 0x558b934adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3006600640 1 -- 172.21.15.59:0/4068382306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9338c920 con 0x558b93496040 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.729+0000 7f3005c00640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349af20 0x558b934abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:40:12.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f3006600640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b93496040 0x558b934ae070 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800f4b0 tx=0x7f2ff800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ebe0349feaf9f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f2fe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f2fe6a00640 1 -- 172.21.15.59:0/4068382306 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8017020 con 0x558b93496040 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f2fe6a00640 1 -- 172.21.15.59:0/4068382306 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f2ff8003b80 con 0x558b93496040 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff407e510 con 0x558b93496040 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f2fe6a00640 1 -- 172.21.15.59:0/4068382306 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8021050 con 0x558b93496040 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f300b41b740 10 client.8019 fetch_fsmap learned FSMap version 67 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558b931b07d0 con 0x558b93496040 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.730+0000 7f2fe6a00640 1 -- 172.21.15.59:0/4068382306 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff407e510 con 0x558b93496040 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f2fe6a00640 1 -- 172.21.15.59:0/4068382306 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f2ff8024020 con 0x558b93496040 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f300b41b740 10 client.8019 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f300b41b740 10 client.8019 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:40:12.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f300b41b740 20 client.8019 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:12.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f300b41b740 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b9349bb00 0x558b9349bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f300b41b740 4 client.8019 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:40:12.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b933e1c10 con 0x558b9349bb00 2024-04-23T14:40:12.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.731+0000 7f3005c00640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b9349bb00 0x558b9349bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.732+0000 7f3005c00640 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b9349bb00 0x558b9349bf90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e01805d7135e51 server_cookie=1872b8c345b5cdbe in_seq=0 out_seq=0 2024-04-23T14:40:12.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.732+0000 7f2fe6a00640 10 client.8019 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:12.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.732+0000 7f2fe6a00640 1 -- 172.21.15.59:0/4068382306 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b933e1c10 con 0x558b9349bb00 2024-04-23T14:40:12.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.732+0000 7f2fe6a00640 10 client.8019 handle_command_reply: tid=0 2024-04-23T14:40:12.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.732+0000 7f300b41b740 1 client.8019 shutdown 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b93496040 msgr2=0x558b934ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b93496040 0x558b934ae070 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800f4b0 tx=0x7f2ff800f970 comp rx=0 tx=0).stop 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 shutdown_connections 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b9349bb00 0x558b9349bf90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b93496040 0x558b934ae070 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b934b7e00 0x558b934adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 --2- 172.21.15.59:0/4068382306 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b9349af20 0x558b934abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 >> 172.21.15.59:0/4068382306 conn(0x558b9349a7e0 msgr2=0x558b934b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f2fe6a00640 0 client.8019 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 shutdown_connections 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 1 -- 172.21.15.59:0/4068382306 wait complete. 2024-04-23T14:40:12.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.733+0000 7f300b41b740 20 client.8019 trim_cache size 0 max 16384 2024-04-23T14:40:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.753+0000 7f300b41b740 1 Processor -- start 2024-04-23T14:40:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.753+0000 7f300b41b740 1 -- start start 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 0x558b934701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b934ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f300b41b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b934addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f300b41b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3000200dc0 con 0x558b934b7e00 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30001ffe30 con 0x558b9349af20 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30000b07b0 con 0x558b93496040 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b934addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b934addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57216/0 (socket says 172.21.15.59:57216) 2024-04-23T14:40:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 -- 172.21.15.59:0/247564020 learned_addr learned my addr 172.21.15.59:0/247564020 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 -- 172.21.15.59:0/247564020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 msgr2=0x558b934701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 --2- 172.21.15.59:0/247564020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 0x558b934701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 -- 172.21.15.59:0/247564020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 msgr2=0x558b934ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 --2- 172.21.15.59:0/247564020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b934ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.754+0000 7f3006600640 1 -- 172.21.15.59:0/247564020 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b930e63f0 con 0x558b934b7e00 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f3006600640 1 --2- 172.21.15.59:0/247564020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b934addc0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8001220 tx=0x7f2ff8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb93dd766314755a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f2ff1600640 1 -- 172.21.15.59:0/247564020 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8017040 con 0x558b934b7e00 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f2ff1600640 1 -- 172.21.15.59:0/247564020 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8010000 con 0x558b934b7e00 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f2ff1600640 1 -- 172.21.15.59:0/247564020 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8005590 con 0x558b934b7e00 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f300b41b740 1 -- 172.21.15.59:0/247564020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 msgr2=0x558b934addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f300b41b740 1 --2- 172.21.15.59:0/247564020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b934addc0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8001220 tx=0x7f2ff8013000 comp rx=0 tx=0).stop 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f300b41b740 1 -- 172.21.15.59:0/247564020 shutdown_connections 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f300b41b740 1 --2- 172.21.15.59:0/247564020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b934addc0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f300b41b740 1 --2- 172.21.15.59:0/247564020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b934ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f300b41b740 1 --2- 172.21.15.59:0/247564020 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 0x558b934701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.755+0000 7f300b41b740 1 -- 172.21.15.59:0/247564020 >> 172.21.15.59:0/247564020 conn(0x558b9349a7e0 msgr2=0x558b934b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.756+0000 7f300b41b740 1 -- 172.21.15.59:0/247564020 shutdown_connections 2024-04-23T14:40:12.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.756+0000 7f300b41b740 1 -- 172.21.15.59:0/247564020 wait complete. 2024-04-23T14:40:12.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.756+0000 7f300b41b740 1 -- start start 2024-04-23T14:40:12.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.756+0000 7f300b41b740 1 Processor -- start 2024-04-23T14:40:12.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.757+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 0x558b9352c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.757+0000 7f300b41b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b934b7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.757+0000 7f300b41b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9349c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f300b41b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b93388e10 con 0x558b934b7e00 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30000b0dd0 con 0x558b9349af20 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f300b41b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x558b9349daa0 con 0x558b93496040 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f300b41b740 10 client.0 _ll_register_callbacks cb 0x7f2ff8001600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9349c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9349c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57224/0 (socket says 172.21.15.59:57224) 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 -- 172.21.15.59:0/3348592058 learned_addr learned my addr 172.21.15.59:0/3348592058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3005c00640 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 0x558b9352c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3005200640 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b934b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 -- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 msgr2=0x558b9352c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 0x558b9352c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 -- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 msgr2=0x558b934b7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b934b7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.758+0000 7f3006600640 1 -- 172.21.15.59:0/3348592058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b930a60f0 con 0x558b934b7e00 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f3006600640 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9349c5d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2ff801f040 tx=0x7f2ff80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146a8c086c752a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f2fe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f2fe6a00640 1 -- 172.21.15.59:0/3348592058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8032060 con 0x558b934b7e00 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f2fe6a00640 1 -- 172.21.15.59:0/3348592058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f2ff80253f0 con 0x558b934b7e00 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff8010000 con 0x558b934b7e00 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f2fe6a00640 1 -- 172.21.15.59:0/3348592058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2ff8038050 con 0x558b934b7e00 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f300b41b740 10 client.8025 fetch_fsmap learned FSMap version 67 2024-04-23T14:40:12.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.759+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558b93399810 con 0x558b934b7e00 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f2fe6a00640 1 -- 172.21.15.59:0/3348592058 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8010000 con 0x558b934b7e00 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f2fe6a00640 1 -- 172.21.15.59:0/3348592058 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f2ff803b020 con 0x558b934b7e00 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f300b41b740 10 client.8025 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f300b41b740 10 client.8025 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f300b41b740 20 client.8025 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f300b41b740 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b934b7570 0x558b9349cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f300b41b740 4 client.8025 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b934c92d0 con 0x558b934b7570 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.760+0000 7f3005c00640 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b934b7570 0x558b9349cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f3005c00640 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b934b7570 0x558b9349cb10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa20c5f9f2cae3f7 server_cookie=78430dc99371dc59 in_seq=0 out_seq=0 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f2fe6a00640 10 client.8025 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f2fe6a00640 1 -- 172.21.15.59:0/3348592058 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558b934c92d0 con 0x558b934b7570 2024-04-23T14:40:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f2fe6a00640 10 client.8025 handle_command_reply: tid=0 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 client.8025 shutdown 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 msgr2=0x558b9349c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9349c5d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2ff801f040 tx=0x7f2ff80014a0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 shutdown_connections 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x558b934b7570 0x558b9349cb10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b934b7e00 0x558b9349c5d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558b9349af20 0x558b934b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 --2- 172.21.15.59:0/3348592058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x558b93496040 0x558b9352c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f2fe6a00640 0 client.8025 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 >> 172.21.15.59:0/3348592058 conn(0x558b9349a7e0 msgr2=0x558b934aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 shutdown_connections 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.761+0000 7f300b41b740 1 -- 172.21.15.59:0/3348592058 wait complete. 2024-04-23T14:40:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f300b41b740 20 client.8025 trim_cache size 0 max 16384 2024-04-23T14:40:12.766 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:40:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:40:12.766 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:40:12.766 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:40:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2fe006c8f0 msgr2=0x7f2fe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2fe006c8f0 0x7f2fe006edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30000cc430 tx=0x7f2fec008040 comp rx=0 tx=0).stop 2024-04-23T14:40:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30000af5c0 msgr2=0x7f30000cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30000af5c0 0x7f30000cdfd0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2ff402a2b0 tx=0x7f2ff40046c0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 shutdown_connections 2024-04-23T14:40:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2fe006c8f0 0x7f2fe006edb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f300015c410 0x7f30000b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30000d0b10 0x7f30001f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 --2- 172.21.15.59:0/2257043316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30000af5c0 0x7f30000cdfd0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 >> 172.21.15.59:0/2257043316 conn(0x7f300006bdf0 msgr2=0x7f300006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 shutdown_connections 2024-04-23T14:40:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.762+0000 7f3007000640 1 -- 172.21.15.59:0/2257043316 wait complete. 2024-04-23T14:40:12.779 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:40:12.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:40:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.983+0000 7f8588200640 1 Processor -- start 2024-04-23T14:40:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.983+0000 7f8588200640 1 -- start start 2024-04-23T14:40:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.985+0000 7f8588200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f85800cd020 0x7f85800b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.985+0000 7f8588200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f8580160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.985+0000 7f8588200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85800cd520 con 0x7f85801454e0 2024-04-23T14:40:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.985+0000 7f8588200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f85800cddc0 con 0x7f85800cd020 2024-04-23T14:40:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.985+0000 7f8588200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f85801462c0 con 0x7f85800b3e40 2024-04-23T14:40:12.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.986+0000 7f8586200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f8580160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.986+0000 7f8586200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f8580160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:47480/0 (socket says 172.21.15.59:47480) 2024-04-23T14:40:12.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.986+0000 7f8586200640 1 -- 172.21.15.59:0/3586893537 learned_addr learned my addr 172.21.15.59:0/3586893537 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.986+0000 7f8584e00640 1 --2- 172.21.15.59:0/3586893537 >> v2:172.21.15.76:3300/0 conn(0x7f85800cd020 0x7f85800b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.987+0000 7f8586200640 1 -- 172.21.15.59:0/3586893537 >> v2:172.21.15.76:3300/0 conn(0x7f85800cd020 msgr2=0x7f85800b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:12.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.987+0000 7f8586200640 1 --2- 172.21.15.59:0/3586893537 >> v2:172.21.15.76:3300/0 conn(0x7f85800cd020 0x7f85800b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.987+0000 7f8586200640 1 -- 172.21.15.59:0/3586893537 >> v1:172.21.15.76:6789/0 conn(0x7f85801454e0 legacy=0x7f8580145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.987+0000 7f8586200640 1 -- 172.21.15.59:0/3586893537 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85800cd800 con 0x7f85800b3e40 2024-04-23T14:40:12.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.988+0000 7f8586200640 1 --2- 172.21.15.59:0/3586893537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f8580160910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8578009870 tx=0x7f857802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0ab744aae4bd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.989+0000 7f857fe00640 1 -- 172.21.15.59:0/3586893537 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578037070 con 0x7f85800b3e40 2024-04-23T14:40:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.989+0000 7f857fe00640 1 -- 172.21.15.59:0/3586893537 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8578032440 con 0x7f85800b3e40 2024-04-23T14:40:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.989+0000 7f857fe00640 1 -- 172.21.15.59:0/3586893537 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578032810 con 0x7f85800b3e40 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 -- 172.21.15.59:0/3586893537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 msgr2=0x7f8580160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 --2- 172.21.15.59:0/3586893537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f8580160910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8578009870 tx=0x7f857802a1c0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 -- 172.21.15.59:0/3586893537 shutdown_connections 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 --2- 172.21.15.59:0/3586893537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f8580160910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 --2- 172.21.15.59:0/3586893537 >> v2:172.21.15.76:3300/0 conn(0x7f85800cd020 0x7f85800b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 -- 172.21.15.59:0/3586893537 >> 172.21.15.59:0/3586893537 conn(0x7f8580067ea0 msgr2=0x7f858006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 -- 172.21.15.59:0/3586893537 shutdown_connections 2024-04-23T14:40:12.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.990+0000 7f8588200640 1 -- 172.21.15.59:0/3586893537 wait complete. 2024-04-23T14:40:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.992+0000 7f8588200640 1 Processor -- start 2024-04-23T14:40:12.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.992+0000 7f8588200640 1 -- start start 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.993+0000 7f8588200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f85801f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.993+0000 7f8588200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd020 0x7f858013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.993+0000 7f8588200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85801454e0 0x7f858015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.993+0000 7f8588200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8580160e50 con 0x7f85800cd020 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.993+0000 7f8588200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85801603f0 con 0x7f85801454e0 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.993+0000 7f8588200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8580160fd0 con 0x7f85800b3e40 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd020 0x7f858013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd020 0x7f858013ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57228/0 (socket says 172.21.15.59:57228) 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 -- 172.21.15.59:0/3467074085 learned_addr learned my addr 172.21.15.59:0/3467074085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8586200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85801454e0 0x7f858015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8585800640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f85801f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 -- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 msgr2=0x7f85801f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f85801f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 -- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85801454e0 msgr2=0x7f858015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:12.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85801454e0 0x7f858015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:12.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8584e00640 1 -- 172.21.15.59:0/3467074085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85801e7280 con 0x7f85800cd020 2024-04-23T14:40:12.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.994+0000 7f8586200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85801454e0 0x7f858015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:12.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.995+0000 7f8584e00640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd020 0x7f858013ed10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f856c00e9e0 tx=0x7f856c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:12.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.995+0000 7f857d600640 1 -- 172.21.15.59:0/3467074085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c017070 con 0x7f85800cd020 2024-04-23T14:40:12.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.995+0000 7f857d600640 1 -- 172.21.15.59:0/3467074085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f856c00c9d0 con 0x7f85800cd020 2024-04-23T14:40:12.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.995+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801e7560 con 0x7f85800cd020 2024-04-23T14:40:12.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.995+0000 7f857d600640 1 -- 172.21.15.59:0/3467074085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c00cdc0 con 0x7f85800cd020 2024-04-23T14:40:12.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.995+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801e7b20 con 0x7f85800cd020 2024-04-23T14:40:12.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.996+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8580068a80 con 0x7f85800cd020 2024-04-23T14:40:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.998+0000 7f857d600640 1 -- 172.21.15.59:0/3467074085 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f856c01a070 con 0x7f85800cd020 2024-04-23T14:40:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.998+0000 7f857d600640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f856006c8f0 0x7f856006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:12.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.999+0000 7f857d600640 1 -- 172.21.15.59:0/3467074085 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f856c08c080 con 0x7f85800cd020 2024-04-23T14:40:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:12.999+0000 7f8585800640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f856006c8f0 0x7f856006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:12.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.000+0000 7f8585800640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f856006c8f0 0x7f856006edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8570001ca0 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.003+0000 7f857d600640 1 -- 172.21.15.59:0/3467074085 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f856c05a340 con 0x7f85800cd020 2024-04-23T14:40:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.247+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8580160030 con 0x7f85800cd020 2024-04-23T14:40:13.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.249+0000 7f857d600640 1 -- 172.21.15.59:0/3467074085 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5182 (secure 0 0 0) 0x7f856c059c40 con 0x7f85800cd020 2024-04-23T14:40:13.249 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:13.249 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:39:16.472663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7533,"qdb_cluster":[7533]},"id":6}]} 2024-04-23T14:40:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 67 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.251+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f856006c8f0 msgr2=0x7f856006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.251+0000 7f8588200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f856006c8f0 0x7f856006edb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8570001ca0 tx=0x7f8570009040 comp rx=0 tx=0).stop 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.251+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd020 msgr2=0x7f858013ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.251+0000 7f8588200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd020 0x7f858013ed10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f856c00e9e0 tx=0x7f856c00eea0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 shutdown_connections 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f856006c8f0 0x7f856006edb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85801454e0 0x7f858015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85800cd020 0x7f858013ed10 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 --2- 172.21.15.59:0/3467074085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85800b3e40 0x7f85801f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 >> 172.21.15.59:0/3467074085 conn(0x7f8580067ea0 msgr2=0x7f85800cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 shutdown_connections 2024-04-23T14:40:13.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.252+0000 7f8588200640 1 -- 172.21.15.59:0/3467074085 wait complete. 2024-04-23T14:40:13.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-23T14:40:13.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.495+0000 7fd7d7200640 1 Processor -- start 2024-04-23T14:40:13.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.495+0000 7fd7d7200640 1 -- start start 2024-04-23T14:40:13.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.497+0000 7fd7d7200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd7d015af40 0x7fd7d0154f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.497+0000 7fd7d7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d015ed90 con 0x7fd7d006fe60 2024-04-23T14:40:13.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.497+0000 7fd7d7200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d015f070 con 0x7fd7d00b3cd0 2024-04-23T14:40:13.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.497+0000 7fd7d7200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d00b34d0 con 0x7fd7d015af40 2024-04-23T14:40:13.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.497+0000 7fd7cfe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd7d006fe60 0x7fd7d015e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58646/0 (socket says 172.21.15.59:58646) 2024-04-23T14:40:13.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.497+0000 7fd7cfe00640 1 -- 172.21.15.59:0/2475943544 learned_addr learned my addr 172.21.15.59:0/2475943544 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578928303 0 0) 0x7fd7d015ed90 con 0x7fd7d006fe60 2024-04-23T14:40:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/2475943544 >> v2:172.21.15.76:3300/0 conn(0x7fd7d015af40 0x7fd7d0154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7b4003520 con 0x7fd7d006fe60 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 115353750 0 0) 0x7fd7b4003520 con 0x7fd7d006fe60 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 >> v2:172.21.15.76:3300/0 conn(0x7fd7d015af40 msgr2=0x7fd7d0154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 --2- 172.21.15.59:0/2475943544 >> v2:172.21.15.76:3300/0 conn(0x7fd7d015af40 0x7fd7d0154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 >> v1:172.21.15.76:6789/0 conn(0x7fd7d00b3cd0 legacy=0x7fd7d00b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d015c080 con 0x7fd7d006fe60 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fd7c0001670 con 0x7fd7d006fe60 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 >> v1:172.21.15.59:6789/0 conn(0x7fd7d006fe60 legacy=0x7fd7d015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7b40039b0 0x7fd7b4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b4004800 0x7fd7b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40052e0 0x7fd7b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d00b34d0 con 0x7fd7b40052e0 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b400c190 con 0x7fd7b4004800 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.498+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b400c390 con 0x7fd7b40039b0 2024-04-23T14:40:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7d5200640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40052e0 0x7fd7b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b4004800 0x7fd7b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7d5200640 1 -- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7b40039b0 msgr2=0x7fd7b4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7d5200640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7b40039b0 0x7fd7b4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7d5200640 1 -- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b4004800 msgr2=0x7fd7b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7d5200640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b4004800 0x7fd7b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7d5200640 1 -- 172.21.15.59:0/2475943544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b4003520 con 0x7fd7b40052e0 2024-04-23T14:40:13.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.499+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b4004800 0x7fd7b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.500+0000 7fd7d5200640 1 -- 172.21.15.59:0/2475943544 reap_dead start 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.500+0000 7fd7d5200640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40052e0 0x7fd7b400b9c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8004480 tx=0x7fd7b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505a01c448ed6952 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.501+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8004080 con 0x7fd7b40052e0 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.501+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c802ad80 con 0x7fd7b40052e0 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.501+0000 7fd7cf400640 1 -- 172.21.15.59:0/2475943544 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8006d90 con 0x7fd7b40052e0 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.501+0000 7fd7d7200640 1 -- 172.21.15.59:0/2475943544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40052e0 msgr2=0x7fd7b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.501+0000 7fd7d7200640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40052e0 0x7fd7b400b9c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8004480 tx=0x7fd7b4005830 comp rx=0 tx=0).stop 2024-04-23T14:40:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.502+0000 7fd7d7200640 1 -- 172.21.15.59:0/2475943544 shutdown_connections 2024-04-23T14:40:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.502+0000 7fd7d7200640 1 --2- 172.21.15.59:0/2475943544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40052e0 0x7fd7b400b9c0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.502+0000 7fd7d7200640 1 -- 172.21.15.59:0/2475943544 >> 172.21.15.59:0/2475943544 conn(0x7fd7d0067d90 msgr2=0x7fd7d006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.502+0000 7fd7d7200640 1 -- 172.21.15.59:0/2475943544 shutdown_connections 2024-04-23T14:40:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.502+0000 7fd7d7200640 1 -- 172.21.15.59:0/2475943544 wait complete. 2024-04-23T14:40:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.504+0000 7fd7d7200640 1 Processor -- start 2024-04-23T14:40:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.504+0000 7fd7d7200640 1 -- start start 2024-04-23T14:40:13.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.505+0000 7fd7d7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b40052e0 0x7fd7d01518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7d01efcb0 0x7fd7d00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7d01f0130 0x7fd7d00b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0155110 con 0x7fd7d01f0130 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0154f90 con 0x7fd7b40052e0 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d0155290 con 0x7fd7d01efcb0 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7d01f0130 0x7fd7d00b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7d01f0130 0x7fd7d00b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57234/0 (socket says 172.21.15.59:57234) 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 -- 172.21.15.59:0/73049800 learned_addr learned my addr 172.21.15.59:0/73049800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b40052e0 0x7fd7d01518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7d01efcb0 0x7fd7d00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 -- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7d01efcb0 msgr2=0x7fd7d00b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7d01efcb0 0x7fd7d00b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 -- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b40052e0 msgr2=0x7fd7d01518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b40052e0 0x7fd7d01518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.506+0000 7fd7d5200640 1 -- 172.21.15.59:0/73049800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d01569e0 con 0x7fd7d01f0130 2024-04-23T14:40:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.507+0000 7fd7d5200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7d01f0130 0x7fd7d00b13b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80045a0 tx=0x7fd7c8003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.507+0000 7fd7ccc00640 1 -- 172.21.15.59:0/73049800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8007df0 con 0x7fd7d01f0130 2024-04-23T14:40:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.507+0000 7fd7ccc00640 1 -- 172.21.15.59:0/73049800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c8043070 con 0x7fd7d01f0130 2024-04-23T14:40:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.507+0000 7fd7ccc00640 1 -- 172.21.15.59:0/73049800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80079c0 con 0x7fd7d01f0130 2024-04-23T14:40:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.507+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d0156cc0 con 0x7fd7d01f0130 2024-04-23T14:40:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.507+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d0157280 con 0x7fd7d01f0130 2024-04-23T14:40:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.510+0000 7fd7ccc00640 1 -- 172.21.15.59:0/73049800 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7c8048460 con 0x7fd7d01f0130 2024-04-23T14:40:13.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.511+0000 7fd7ccc00640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7ac06c920 0x7fd7ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.511+0000 7fd7ccc00640 1 -- 172.21.15.59:0/73049800 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fd7c80ac0f0 con 0x7fd7d01f0130 2024-04-23T14:40:13.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.511+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7ac06c920 0x7fd7ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.512+0000 7fd7da1a1740 1 Processor -- start 2024-04-23T14:40:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.512+0000 7fd7da1a1740 1 -- start start 2024-04-23T14:40:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.512+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7ac06c920 0x7fd7ac06ede0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000fb0 tx=0x7fd7c0001c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef630485a0 0x55ef63020c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6301eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ef62f85190 con 0x55ef630485a0 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ef62f2b030 con 0x55ef63048980 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ef62f81150 con 0x55ef6304b340 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef630485a0 0x55ef63020c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef630485a0 0x55ef63020c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57250/0 (socket says 172.21.15.59:57250) 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 -- 172.21.15.59:0/546472382 learned_addr learned my addr 172.21.15.59:0/546472382 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7d5200640 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6301eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 -- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 msgr2=0x55ef6301eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6301eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 -- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 msgr2=0x55ef6304ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7cfe00640 1 -- 172.21.15.59:0/546472382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef62f13b60 con 0x55ef630485a0 2024-04-23T14:40:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7d5200640 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6301eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.513+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.514+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef630485a0 0x55ef63020c00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0004a30 tx=0x7fd7c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d818654ef300c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.514+0000 7fd7c4c00640 1 -- 172.21.15.59:0/546472382 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000e7e0 con 0x55ef630485a0 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.514+0000 7fd7c4c00640 1 -- 172.21.15.59:0/546472382 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c000e980 con 0x55ef630485a0 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.514+0000 7fd7c4c00640 1 -- 172.21.15.59:0/546472382 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000ed30 con 0x55ef630485a0 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 -- 172.21.15.59:0/546472382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef630485a0 msgr2=0x55ef63020c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef630485a0 0x55ef63020c00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0004a30 tx=0x7fd7c0011000 comp rx=0 tx=0).stop 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 -- 172.21.15.59:0/546472382 shutdown_connections 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6301eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/546472382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef630485a0 0x55ef63020c00 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 -- 172.21.15.59:0/546472382 >> 172.21.15.59:0/546472382 conn(0x55ef6301d600 msgr2=0x55ef63065e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 -- 172.21.15.59:0/546472382 shutdown_connections 2024-04-23T14:40:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.515+0000 7fd7da1a1740 1 -- 172.21.15.59:0/546472382 wait complete. 2024-04-23T14:40:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.516+0000 7fd7da1a1740 1 -- start start 2024-04-23T14:40:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.516+0000 7fd7da1a1740 1 Processor -- start 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef63064fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef6304b340 0x55ef6305ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef63032900 0x55ef6305d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ef62f3e500 con 0x55ef6304b340 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ef63063b00 con 0x55ef63048980 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ef63034c00 con 0x55ef63032900 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7da1a1740 10 client.0 _ll_register_callbacks cb 0x55ef62f7b270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.517+0000 7fd7c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef6304b340 0x55ef6305ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef6304b340 0x55ef6305ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57254/0 (socket says 172.21.15.59:57254) 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 -- 172.21.15.59:0/760667794 learned_addr learned my addr 172.21.15.59:0/760667794 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef63064fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7d5200640 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef63032900 0x55ef6305d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 -- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef63032900 msgr2=0x55ef6305d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef63032900 0x55ef6305d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 -- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 msgr2=0x55ef63064fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef63064fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 -- 172.21.15.59:0/760667794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef62d637c0 con 0x55ef6304b340 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef6304b340 0x55ef6305ce70 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8011750 tx=0x7fd7b8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136c6905971481aa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.518+0000 7fd7bd600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7bd600640 1 -- 172.21.15.59:0/760667794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800ed50 con 0x55ef6304b340 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7bd600640 1 -- 172.21.15.59:0/760667794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fd7b801bb60 con 0x55ef6304b340 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c000e980 con 0x55ef6304b340 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7bd600640 1 -- 172.21.15.59:0/760667794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd7b801bdb0 con 0x55ef6304b340 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 10 client.8040 fetch_fsmap learned FSMap version 67 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55ef62dc3f50 con 0x55ef6304b340 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7bd600640 1 -- 172.21.15.59:0/760667794 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c000e980 con 0x55ef6304b340 2024-04-23T14:40:13.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7bd600640 1 -- 172.21.15.59:0/760667794 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd7b8022050 con 0x55ef6304b340 2024-04-23T14:40:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 10 client.8040 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:40:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 10 client.8040 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:40:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 20 client.8040 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef63035750 0x55ef63089280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 4 client.8040 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:40:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.519+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef62f92bb0 con 0x55ef63035750 2024-04-23T14:40:13.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.520+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef63035750 0x55ef63089280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.520+0000 7fd7bd600640 10 client.8040 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.520+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef63035750 0x55ef63089280 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45871fb03e4a9aa server_cookie=35ce9e87c8da25ce in_seq=0 out_seq=0 2024-04-23T14:40:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7bd600640 1 -- 172.21.15.59:0/760667794 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef62f92bb0 con 0x55ef63035750 2024-04-23T14:40:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7bd600640 10 client.8040 handle_command_reply: tid=0 2024-04-23T14:40:13.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 client.8040 shutdown 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef6304b340 msgr2=0x55ef6305ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef6304b340 0x55ef6305ce70 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8011750 tx=0x7fd7b8011c10 comp rx=0 tx=0).stop 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 shutdown_connections 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef63035750 0x55ef63089280 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef63032900 0x55ef6305d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef6304b340 0x55ef6305ce70 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/760667794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef63064fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7bd600640 0 client.8040 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.521+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 >> 172.21.15.59:0/760667794 conn(0x55ef630485a0 msgr2=0x55ef63069380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.522+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 shutdown_connections 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.522+0000 7fd7da1a1740 1 -- 172.21.15.59:0/760667794 wait complete. 2024-04-23T14:40:13.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.522+0000 7fd7da1a1740 20 client.8040 trim_cache size 0 max 16384 2024-04-23T14:40:13.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.537+0000 7fd7da1a1740 1 Processor -- start 2024-04-23T14:40:13.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.537+0000 7fd7da1a1740 1 -- start start 2024-04-23T14:40:13.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6304e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6308e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01fd070 con 0x55ef63032900 2024-04-23T14:40:13.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01fc090 con 0x55ef63048980 2024-04-23T14:40:13.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d01fb440 con 0x55ef6304b340 2024-04-23T14:40:13.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6304e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6304e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57270/0 (socket says 172.21.15.59:57270) 2024-04-23T14:40:13.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.538+0000 7fd7cfe00640 1 -- 172.21.15.59:0/2271059570 learned_addr learned my addr 172.21.15.59:0/2271059570 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:13.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7cfe00640 1 -- 172.21.15.59:0/2271059570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 msgr2=0x55ef6308e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/2271059570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6308e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7cfe00640 1 -- 172.21.15.59:0/2271059570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 msgr2=0x55ef6304eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/2271059570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7cfe00640 1 -- 172.21.15.59:0/2271059570 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6301e740 con 0x55ef63032900 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/2271059570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6304e630 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0017000 tx=0x7fd7c0023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6501cbfa9de7d5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7bfe00640 1 -- 172.21.15.59:0/2271059570 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0031070 con 0x55ef63032900 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7bfe00640 1 -- 172.21.15.59:0/2271059570 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c002b3c0 con 0x55ef63032900 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.539+0000 7fd7bfe00640 1 -- 172.21.15.59:0/2271059570 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c002b770 con 0x55ef63032900 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 -- 172.21.15.59:0/2271059570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 msgr2=0x55ef6304e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/2271059570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6304e630 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0017000 tx=0x7fd7c0023040 comp rx=0 tx=0).stop 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 -- 172.21.15.59:0/2271059570 shutdown_connections 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/2271059570 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef6308e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/2271059570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6304eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/2271059570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6304e630 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 -- 172.21.15.59:0/2271059570 >> 172.21.15.59:0/2271059570 conn(0x55ef630485a0 msgr2=0x55ef63016a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 -- 172.21.15.59:0/2271059570 shutdown_connections 2024-04-23T14:40:13.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.540+0000 7fd7da1a1740 1 -- 172.21.15.59:0/2271059570 wait complete. 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.541+0000 7fd7da1a1740 1 -- start start 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.541+0000 7fd7da1a1740 1 Processor -- start 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6301ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6308e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7da1a1740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef630e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ef63033430 con 0x55ef63032900 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01fba40 con 0x55ef63048980 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7da1a1740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ef63034940 con 0x55ef6304b340 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7da1a1740 10 client.0 _ll_register_callbacks cb 0x7fd7b800b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.542+0000 7fd7cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6301ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6301ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57272/0 (socket says 172.21.15.59:57272) 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 -- 172.21.15.59:0/3870420679 learned_addr learned my addr 172.21.15.59:0/3870420679 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6308e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 -- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 msgr2=0x55ef630e3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef630e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 -- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 msgr2=0x55ef6308e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6308e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 -- 172.21.15.59:0/3870420679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef62c570b0 con 0x55ef63032900 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6308e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7cfe00640 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6301ab90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00049d0 tx=0x7fd7c0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b289d00621563a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.543+0000 7fd7bd600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7bd600640 1 -- 172.21.15.59:0/3870420679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0031070 con 0x55ef63032900 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7bd600640 1 -- 172.21.15.59:0/3870420679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fd7c0038b60 con 0x55ef63032900 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c002b3c0 con 0x55ef63032900 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7bd600640 1 -- 172.21.15.59:0/3870420679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd7c003f050 con 0x55ef63032900 2024-04-23T14:40:13.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 10 client.8046 fetch_fsmap learned FSMap version 67 2024-04-23T14:40:13.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55ef6305bfd0 con 0x55ef63032900 2024-04-23T14:40:13.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7bd600640 1 -- 172.21.15.59:0/3870420679 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c002b3c0 con 0x55ef63032900 2024-04-23T14:40:13.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7bd600640 1 -- 172.21.15.59:0/3870420679 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd7c0042020 con 0x55ef63032900 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 10 client.8046 fetch_fsmap finished waiting for FSMap version 67 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 10 client.8046 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 20 client.8046 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef6307ca80 0x55ef6307ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 4 client.8046 mds_command: new command op to 7533 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.544+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 --> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55ef62f8ca20 con 0x55ef6307ca80 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.545+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef6307ca80 0x55ef6307ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.545+0000 7fd7c7e00640 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef6307ca80 0x55ef6307ce60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6c3c1218074725 server_cookie=4b3f9b154cbfe6da in_seq=0 out_seq=0 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.545+0000 7fd7bd600640 10 client.8046 ms_handle_connect on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.545+0000 7fd7bd600640 1 -- 172.21.15.59:0/3870420679 <== mds.0 v2:172.21.15.59:6835/2128549219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+143 (crc 0 0 0) 0x55ef62f8ca20 con 0x55ef6307ca80 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.545+0000 7fd7bd600640 10 client.8046 handle_command_reply: tid=0 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 client.8046 shutdown 2024-04-23T14:40:13.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 msgr2=0x55ef6301ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6301ab90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00049d0 tx=0x7fd7c0000ed0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 shutdown_connections 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:6835/2128549219,v1:172.21.15.59:6837/2128549219] conn(0x55ef6307ca80 0x55ef6307ce60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ef6304b340 0x55ef630e3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ef63048980 0x55ef6308e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 --2- 172.21.15.59:0/3870420679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ef63032900 0x55ef6301ab90 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 >> 172.21.15.59:0/3870420679 conn(0x55ef630485a0 msgr2=0x55ef63016a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7bd600640 0 client.8046 ms_handle_reset on v2:172.21.15.59:6835/2128549219 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 shutdown_connections 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 1 -- 172.21.15.59:0/3870420679 wait complete. 2024-04-23T14:40:13.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.546+0000 7fd7da1a1740 20 client.8046 trim_cache size 0 max 16384 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "damage_type": "backtrace", 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 728316110, 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 1099511627777, 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/testdir/hardlink" 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:40:13.550 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:40:13.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7ac06c920 msgr2=0x7fd7ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7ac06c920 0x7fd7ac06ede0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000fb0 tx=0x7fd7c0001c50 comp rx=0 tx=0).stop 2024-04-23T14:40:13.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7d01f0130 msgr2=0x7fd7d00b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7d01f0130 0x7fd7d00b13b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80045a0 tx=0x7fd7c8003a70 comp rx=0 tx=0).stop 2024-04-23T14:40:13.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 shutdown_connections 2024-04-23T14:40:13.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd7ac06c920 0x7fd7ac06ede0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7d01efcb0 0x7fd7d00b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7d01f0130 0x7fd7d00b13b0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 --2- 172.21.15.59:0/73049800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b40052e0 0x7fd7d01518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 >> 172.21.15.59:0/73049800 conn(0x7fd7d0067d90 msgr2=0x7fd7d015e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 shutdown_connections 2024-04-23T14:40:13.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.547+0000 7fd7d7200640 1 -- 172.21.15.59:0/73049800 wait complete. 2024-04-23T14:40:13.567 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:40:13.567 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:40:13.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:40:13.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.821+0000 7f8e14a00640 1 Processor -- start 2024-04-23T14:40:13.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.822+0000 7f8e14a00640 1 -- start start 2024-04-23T14:40:13.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.823+0000 7f8e14a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e1006f7c0 0x7f8e100b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.823+0000 7f8e14a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e1015b840 0x7f8e1006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.823+0000 7f8e14a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e100700e0 con 0x7f8e1006df00 2024-04-23T14:40:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.823+0000 7f8e14a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8e1006ec60 con 0x7f8e1015b840 2024-04-23T14:40:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.823+0000 7f8e14a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8e100b46e0 con 0x7f8e1006f7c0 2024-04-23T14:40:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e1015b840 0x7f8e1006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e1015b840 0x7f8e1006fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57276/0 (socket says 172.21.15.59:57276) 2024-04-23T14:40:13.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0d400640 1 -- 172.21.15.59:0/1817486505 learned_addr learned my addr 172.21.15.59:0/1817486505 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0de00640 1 --2- 172.21.15.59:0/1817486505 >> v2:172.21.15.76:3300/0 conn(0x7f8e1006f7c0 0x7f8e100b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0d400640 1 -- 172.21.15.59:0/1817486505 >> v2:172.21.15.76:3300/0 conn(0x7f8e1006f7c0 msgr2=0x7f8e100b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0d400640 1 --2- 172.21.15.59:0/1817486505 >> v2:172.21.15.76:3300/0 conn(0x7f8e1006f7c0 0x7f8e100b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0d400640 1 -- 172.21.15.59:0/1817486505 >> v1:172.21.15.59:6789/0 conn(0x7f8e1006df00 legacy=0x7f8e1006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.824+0000 7f8e0d400640 1 -- 172.21.15.59:0/1817486505 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e100b49c0 con 0x7f8e1015b840 2024-04-23T14:40:13.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.825+0000 7f8e0d400640 1 --2- 172.21.15.59:0/1817486505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1015b840 0x7f8e1006fba0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8e1006c700 tx=0x7f8df802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ca93e50a3f5670 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.826+0000 7f8e0ca00640 1 -- 172.21.15.59:0/1817486505 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8037070 con 0x7f8e1015b840 2024-04-23T14:40:13.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.826+0000 7f8e0ca00640 1 -- 172.21.15.59:0/1817486505 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df8032440 con 0x7f8e1015b840 2024-04-23T14:40:13.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.826+0000 7f8e0ca00640 1 -- 172.21.15.59:0/1817486505 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8032810 con 0x7f8e1015b840 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.826+0000 7f8e14a00640 1 -- 172.21.15.59:0/1817486505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1015b840 msgr2=0x7f8e1006fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.826+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1817486505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1015b840 0x7f8e1006fba0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8e1006c700 tx=0x7f8df802a270 comp rx=0 tx=0).stop 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.827+0000 7f8e14a00640 1 -- 172.21.15.59:0/1817486505 shutdown_connections 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.827+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1817486505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1015b840 0x7f8e1006fba0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.827+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1817486505 >> v2:172.21.15.76:3300/0 conn(0x7f8e1006f7c0 0x7f8e100b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.827+0000 7f8e14a00640 1 -- 172.21.15.59:0/1817486505 >> 172.21.15.59:0/1817486505 conn(0x7f8e10067ea0 msgr2=0x7f8e1006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.827+0000 7f8e14a00640 1 -- 172.21.15.59:0/1817486505 shutdown_connections 2024-04-23T14:40:13.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.827+0000 7f8e14a00640 1 -- 172.21.15.59:0/1817486505 wait complete. 2024-04-23T14:40:13.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.829+0000 7f8e14a00640 1 Processor -- start 2024-04-23T14:40:13.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.829+0000 7f8e14a00640 1 -- start start 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e14a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1006df00 0x7f8e101f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e14a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1006f7c0 0x7f8e1015cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e14a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1014df80 0x7f8e1014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e14a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e100ce1c0 con 0x7f8e1006f7c0 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e14a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e100b4f80 con 0x7f8e1006df00 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e14a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8e100ce340 con 0x7f8e1014df80 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e0d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1006f7c0 0x7f8e1015cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e0d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1006f7c0 0x7f8e1015cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57282/0 (socket says 172.21.15.59:57282) 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.830+0000 7f8e0d400640 1 -- 172.21.15.59:0/1994655629 learned_addr learned my addr 172.21.15.59:0/1994655629 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e0de00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1006df00 0x7f8e101f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e0d400640 1 -- 172.21.15.59:0/1994655629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1014df80 msgr2=0x7f8e1014bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e0d400640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1014df80 0x7f8e1014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e0d400640 1 -- 172.21.15.59:0/1994655629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1006df00 msgr2=0x7f8e101f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e0d400640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1006df00 0x7f8e101f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e0d400640 1 -- 172.21.15.59:0/1994655629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1014c480 con 0x7f8e1006f7c0 2024-04-23T14:40:13.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e0d400640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1006f7c0 0x7f8e1015cf10 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000c00 tx=0x7f8df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e06000640 1 -- 172.21.15.59:0/1994655629 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8041070 con 0x7f8e1006f7c0 2024-04-23T14:40:13.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e06000640 1 -- 172.21.15.59:0/1994655629 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df803c870 con 0x7f8e1006f7c0 2024-04-23T14:40:13.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e06000640 1 -- 172.21.15.59:0/1994655629 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8037040 con 0x7f8e1006f7c0 2024-04-23T14:40:13.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.831+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1014e430 con 0x7f8e1006f7c0 2024-04-23T14:40:13.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.832+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101f8c20 con 0x7f8e1006f7c0 2024-04-23T14:40:13.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.833+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e10068a80 con 0x7f8e1006f7c0 2024-04-23T14:40:13.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.834+0000 7f8e06000640 1 -- 172.21.15.59:0/1994655629 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8df8044070 con 0x7f8e1006f7c0 2024-04-23T14:40:13.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.835+0000 7f8e06000640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8dec06c8f0 0x7f8dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:13.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.835+0000 7f8e06000640 1 -- 172.21.15.59:0/1994655629 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f8df80ac6e0 con 0x7f8e1006f7c0 2024-04-23T14:40:13.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.836+0000 7f8e0de00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8dec06c8f0 0x7f8dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:13.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.836+0000 7f8e0de00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8dec06c8f0 0x7f8dec06edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006fd0 tx=0x7f8e00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:13.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:13.840+0000 7f8e06000640 1 -- 172.21.15.59:0/1994655629 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8df807aa20 con 0x7f8e1006f7c0 2024-04-23T14:40:14.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.044+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e1006f370 con 0x7f8e1006f7c0 2024-04-23T14:40:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.046+0000 7f8e06000640 1 -- 172.21.15.59:0/1994655629 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16779 (secure 0 0 0) 0x7f8df807a320 con 0x7f8e1006f7c0 2024-04-23T14:40:14.046 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:14.047 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:39:09.500890+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6835/3870021000":"2024-04-24T14:39:01.426690+0000","172.21.15.76:6833/3870021000":"2024-04-24T14:39:01.426690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:40:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8dec06c8f0 msgr2=0x7f8dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8dec06c8f0 0x7f8dec06edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8e00006fd0 tx=0x7f8e00008040 comp rx=0 tx=0).stop 2024-04-23T14:40:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1006f7c0 msgr2=0x7f8e1015cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1006f7c0 0x7f8e1015cf10 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000c00 tx=0x7f8df80046c0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 shutdown_connections 2024-04-23T14:40:14.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8dec06c8f0 0x7f8dec06edb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1014df80 0x7f8e1014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e1006f7c0 0x7f8e1015cf10 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 --2- 172.21.15.59:0/1994655629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1006df00 0x7f8e101f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.048+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 >> 172.21.15.59:0/1994655629 conn(0x7f8e10067ea0 msgr2=0x7f8e100b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:14.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.049+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 shutdown_connections 2024-04-23T14:40:14.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.049+0000 7f8e14a00640 1 -- 172.21.15.59:0/1994655629 wait complete. 2024-04-23T14:40:14.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:38:36.964433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:38:38.188587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:40:14.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:40:14.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.283+0000 7f61b5200640 1 Processor -- start 2024-04-23T14:40:14.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.284+0000 7f61b5200640 1 -- start start 2024-04-23T14:40:14.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.285+0000 7f61b5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b015d810 0x7f61b0160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.285+0000 7f61b5200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b015cab0 con 0x7f61b00d0d10 2024-04-23T14:40:14.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.285+0000 7f61b5200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b0160e40 con 0x7f61b01567e0 2024-04-23T14:40:14.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.285+0000 7f61b5200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f61b015cd30 con 0x7f61b015d810 2024-04-23T14:40:14.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.286+0000 7f61afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b015d810 0x7f61b0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.286+0000 7f61aea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f61b00d0d10 0x7f61b0160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58672/0 (socket says 172.21.15.59:58672) 2024-04-23T14:40:14.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.286+0000 7f61aea00640 1 -- 172.21.15.59:0/2201602660 learned_addr learned my addr 172.21.15.59:0/2201602660 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.286+0000 7f61afe00640 1 -- 172.21.15.59:0/2201602660 >> v1:172.21.15.76:6789/0 conn(0x7f61b01567e0 legacy=0x7f61b00d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.286+0000 7f61afe00640 1 -- 172.21.15.59:0/2201602660 >> v1:172.21.15.59:6789/0 conn(0x7f61b00d0d10 legacy=0x7f61b0160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.286+0000 7f61afe00640 1 -- 172.21.15.59:0/2201602660 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b015d010 con 0x7f61b015d810 2024-04-23T14:40:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.286+0000 7f61ae000640 1 -- 172.21.15.59:0/2201602660 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3135752730 0 0) 0x7f6198001110 con 0x7f61b00d0d10 2024-04-23T14:40:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.287+0000 7f61afe00640 1 --2- 172.21.15.59:0/2201602660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b015d810 0x7f61b0160820 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009d90 tx=0x7f61a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e0f0861c1ccaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.288+0000 7f61ae000640 1 -- 172.21.15.59:0/2201602660 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8007740 con 0x7f61b015d810 2024-04-23T14:40:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.288+0000 7f61ae000640 1 -- 172.21.15.59:0/2201602660 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a80078e0 con 0x7f61b015d810 2024-04-23T14:40:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.288+0000 7f61ae000640 1 -- 172.21.15.59:0/2201602660 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8007cb0 con 0x7f61b015d810 2024-04-23T14:40:14.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.288+0000 7f61b5200640 1 -- 172.21.15.59:0/2201602660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b015d810 msgr2=0x7f61b0160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.288+0000 7f61b5200640 1 --2- 172.21.15.59:0/2201602660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b015d810 0x7f61b0160820 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009d90 tx=0x7f61a802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.289+0000 7f61b5200640 1 -- 172.21.15.59:0/2201602660 shutdown_connections 2024-04-23T14:40:14.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.289+0000 7f61b5200640 1 --2- 172.21.15.59:0/2201602660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b015d810 0x7f61b0160820 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.289+0000 7f61b5200640 1 -- 172.21.15.59:0/2201602660 >> 172.21.15.59:0/2201602660 conn(0x7f61b006bdf0 msgr2=0x7f61b006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:14.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.289+0000 7f61b5200640 1 -- 172.21.15.59:0/2201602660 shutdown_connections 2024-04-23T14:40:14.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.289+0000 7f61b5200640 1 -- 172.21.15.59:0/2201602660 wait complete. 2024-04-23T14:40:14.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.290+0000 7f61b5200640 1 Processor -- start 2024-04-23T14:40:14.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.290+0000 7f61b5200640 1 -- start start 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.291+0000 7f61b5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00d0d10 0x7f61b00998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.291+0000 7f61b5200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61b01567e0 0x7f61b014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.291+0000 7f61b5200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61b015d810 0x7f61b01f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.291+0000 7f61b5200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0156d40 con 0x7f61b00d0d10 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.291+0000 7f61b5200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0156bc0 con 0x7f61b01567e0 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.291+0000 7f61b5200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f61b0156ec0 con 0x7f61b015d810 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.292+0000 7f61af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00d0d10 0x7f61b00998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.292+0000 7f61af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00d0d10 0x7f61b00998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57296/0 (socket says 172.21.15.59:57296) 2024-04-23T14:40:14.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.292+0000 7f61af400640 1 -- 172.21.15.59:0/3411797362 learned_addr learned my addr 172.21.15.59:0/3411797362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:14.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.292+0000 7f61aea00640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61b01567e0 0x7f61b014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.292+0000 7f61afe00640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61b015d810 0x7f61b01f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61af400640 1 -- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61b015d810 msgr2=0x7f61b01f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61af400640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61b015d810 0x7f61b01f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61af400640 1 -- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61b01567e0 msgr2=0x7f61b014f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61af400640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61b01567e0 0x7f61b014f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61af400640 1 -- 172.21.15.59:0/3411797362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b0151900 con 0x7f61b00d0d10 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61af400640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00d0d10 0x7f61b00998e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f61a000c430 tx=0x7f61a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61a7400640 1 -- 172.21.15.59:0/3411797362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0011790 con 0x7f61b00d0d10 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61a7400640 1 -- 172.21.15.59:0/3411797362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a0017070 con 0x7f61b00d0d10 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b0151e70 con 0x7f61b00d0d10 2024-04-23T14:40:14.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61a7400640 1 -- 172.21.15.59:0/3411797362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0011bb0 con 0x7f61b00d0d10 2024-04-23T14:40:14.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.293+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b0152430 con 0x7f61b00d0d10 2024-04-23T14:40:14.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.295+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b00d08c0 con 0x7f61b00d0d10 2024-04-23T14:40:14.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.296+0000 7f61a7400640 1 -- 172.21.15.59:0/3411797362 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61a001c460 con 0x7f61b00d0d10 2024-04-23T14:40:14.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.297+0000 7f61a7400640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f618c06c8f0 0x7f618c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.297+0000 7f61a7400640 1 -- 172.21.15.59:0/3411797362 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f61a008bb60 con 0x7f61b00d0d10 2024-04-23T14:40:14.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.297+0000 7f61aea00640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f618c06c8f0 0x7f618c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.298+0000 7f61aea00640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f618c06c8f0 0x7f618c06edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f61b01517e0 tx=0x7f6198007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:14.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.301+0000 7f61a7400640 1 -- 172.21.15.59:0/3411797362 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61b00d08c0 con 0x7f61b00d0d10 2024-04-23T14:40:14.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.507+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61b0099e80 con 0x7f61b00d0d10 2024-04-23T14:40:14.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.508+0000 7f61a7400640 1 -- 172.21.15.59:0/3411797362 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f61a0059ea0 con 0x7f61b00d0d10 2024-04-23T14:40:14.508 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f618c06c8f0 msgr2=0x7f618c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f618c06c8f0 0x7f618c06edb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f61b01517e0 tx=0x7f6198007c00 comp rx=0 tx=0).stop 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00d0d10 msgr2=0x7f61b00998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00d0d10 0x7f61b00998e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f61a000c430 tx=0x7f61a000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 shutdown_connections 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f618c06c8f0 0x7f618c06edb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61b015d810 0x7f61b01f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61b01567e0 0x7f61b014f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 --2- 172.21.15.59:0/3411797362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00d0d10 0x7f61b00998e0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.510+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 >> 172.21.15.59:0/3411797362 conn(0x7f61b006bdf0 msgr2=0x7f61b00af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.511+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 shutdown_connections 2024-04-23T14:40:14.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.511+0000 7f61b5200640 1 -- 172.21.15.59:0/3411797362 wait complete. 2024-04-23T14:40:14.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:40:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.735+0000 7f6e6a200640 1 Processor -- start 2024-04-23T14:40:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.735+0000 7f6e6a200640 1 -- start start 2024-04-23T14:40:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.737+0000 7f6e6a200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6e6415ed20 0x7f6e640d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.737+0000 7f6e6a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 0x7f6e640d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.737+0000 7f6e6a200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e640b5da0 con 0x7f6e640b3600 2024-04-23T14:40:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.737+0000 7f6e6a200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6e6415f100 con 0x7f6e6415ed20 2024-04-23T14:40:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.737+0000 7f6e6a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6e640d07d0 con 0x7f6e640d13e0 2024-04-23T14:40:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.737+0000 7f6e63e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6e640b3600 0x7f6e640b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58678/0 (socket says 172.21.15.59:58678) 2024-04-23T14:40:14.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.737+0000 7f6e63e00640 1 -- 172.21.15.59:0/3320677201 learned_addr learned my addr 172.21.15.59:0/3320677201 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e68e00640 1 --2- 172.21.15.59:0/3320677201 >> v2:172.21.15.76:3300/0 conn(0x7f6e6415ed20 0x7f6e640d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e69800640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 0x7f6e640d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718054229 0 0) 0x7f6e640b5da0 con 0x7f6e640b3600 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e4c003520 con 0x7f6e640b3600 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2338873948 0 0) 0x7f6e4c003520 con 0x7f6e640b3600 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 msgr2=0x7f6e640d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 0x7f6e640d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 >> v2:172.21.15.76:3300/0 conn(0x7f6e6415ed20 msgr2=0x7f6e640d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 --2- 172.21.15.59:0/3320677201 >> v2:172.21.15.76:3300/0 conn(0x7f6e6415ed20 0x7f6e640d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e640d0a20 con 0x7f6e640b3600 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.738+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f6e50001670 con 0x7f6e640b3600 2024-04-23T14:40:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 >> v1:172.21.15.59:6789/0 conn(0x7f6e640b3600 legacy=0x7f6e640b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63400640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 0x7f6e4c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63400640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e4c003f30 0x7f6e4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63400640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640d07d0 con 0x7f6e4c004d40 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c00ba20 con 0x7f6e4c003f30 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4c00bc20 con 0x7f6e640d13e0 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e69800640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e63e00640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e4c003f30 0x7f6e4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e68e00640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 0x7f6e4c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e69800640 1 -- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 msgr2=0x7f6e4c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e69800640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640d13e0 0x7f6e4c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e69800640 1 -- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e4c003f30 msgr2=0x7f6e4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e69800640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e4c003f30 0x7f6e4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.739+0000 7f6e69800640 1 -- 172.21.15.59:0/3320677201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c003520 con 0x7f6e4c004d40 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.741+0000 7f6e69800640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e4c0051c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009fc0 tx=0x7f6e4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78d3bbe34693e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.741+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c009c10 con 0x7f6e4c004d40 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.741+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e5c036070 con 0x7f6e4c004d40 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.741+0000 7f6e63400640 1 -- 172.21.15.59:0/3320677201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c031880 con 0x7f6e4c004d40 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.742+0000 7f6e6a200640 1 -- 172.21.15.59:0/3320677201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 msgr2=0x7f6e4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.742+0000 7f6e6a200640 1 --2- 172.21.15.59:0/3320677201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e4c0051c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009fc0 tx=0x7f6e4c005830 comp rx=0 tx=0).stop 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.742+0000 7f6e69800640 1 -- 172.21.15.59:0/3320677201 reap_dead start 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.742+0000 7f6e6a200640 1 -- 172.21.15.59:0/3320677201 shutdown_connections 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.742+0000 7f6e6a200640 1 -- 172.21.15.59:0/3320677201 >> 172.21.15.59:0/3320677201 conn(0x7f6e6408bf90 msgr2=0x7f6e640d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.742+0000 7f6e6a200640 1 -- 172.21.15.59:0/3320677201 shutdown_connections 2024-04-23T14:40:14.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.742+0000 7f6e6a200640 1 -- 172.21.15.59:0/3320677201 wait complete. 2024-04-23T14:40:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.744+0000 7f6e6a200640 1 Processor -- start 2024-04-23T14:40:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.744+0000 7f6e6a200640 1 -- start start 2024-04-23T14:40:14.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e6a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e641f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e6a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640ce400 0x7f6e640b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e6a200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e64158dc0 0x7f6e640b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e6a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640d3490 con 0x7f6e4c004d40 2024-04-23T14:40:14.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e6a200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640d31b0 con 0x7f6e64158dc0 2024-04-23T14:40:14.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e6a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6e640d0290 con 0x7f6e640ce400 2024-04-23T14:40:14.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e68e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e641f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e68e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e641f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57310/0 (socket says 172.21.15.59:57310) 2024-04-23T14:40:14.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.745+0000 7f6e68e00640 1 -- 172.21.15.59:0/2115215369 learned_addr learned my addr 172.21.15.59:0/2115215369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e69800640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e64158dc0 0x7f6e640b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e68e00640 1 -- 172.21.15.59:0/2115215369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640ce400 msgr2=0x7f6e640b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e68e00640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640ce400 0x7f6e640b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e68e00640 1 -- 172.21.15.59:0/2115215369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e64158dc0 msgr2=0x7f6e640b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e68e00640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e64158dc0 0x7f6e640b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e68e00640 1 -- 172.21.15.59:0/2115215369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e640b85c0 con 0x7f6e4c004d40 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e68e00640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e641f5040 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6e580108e0 tx=0x7f6e58010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e60c00640 1 -- 172.21.15.59:0/2115215369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5800f040 con 0x7f6e4c004d40 2024-04-23T14:40:14.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e60c00640 1 -- 172.21.15.59:0/2115215369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e5800eb60 con 0x7f6e4c004d40 2024-04-23T14:40:14.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e60c00640 1 -- 172.21.15.59:0/2115215369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5801c400 con 0x7f6e4c004d40 2024-04-23T14:40:14.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.746+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e640b8b40 con 0x7f6e4c004d40 2024-04-23T14:40:14.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.747+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e641f4560 con 0x7f6e4c004d40 2024-04-23T14:40:14.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.748+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6404fd90 con 0x7f6e4c004d40 2024-04-23T14:40:14.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.749+0000 7f6e60c00640 1 -- 172.21.15.59:0/2115215369 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e5801c5a0 con 0x7f6e4c004d40 2024-04-23T14:40:14.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.750+0000 7f6e60c00640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:14.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.750+0000 7f6e60c00640 1 -- 172.21.15.59:0/2115215369 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f6e5808c380 con 0x7f6e4c004d40 2024-04-23T14:40:14.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.751+0000 7f6e63e00640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:14.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.751+0000 7f6e63e00640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e4406c8f0 0x7f6e4406edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000f60 tx=0x7f6e50000fe0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:14.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.755+0000 7f6e60c00640 1 -- 172.21.15.59:0/2115215369 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e58018030 con 0x7f6e4c004d40 2024-04-23T14:40:14.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.962+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6e640b4980 con 0x7f6e4c004d40 2024-04-23T14:40:14.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.963+0000 7f6e60c00640 1 -- 172.21.15.59:0/2115215369 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f6e5805a100 con 0x7f6e4c004d40 2024-04-23T14:40:14.963 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e4406c8f0 msgr2=0x7f6e4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e4406c8f0 0x7f6e4406edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000f60 tx=0x7f6e50000fe0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 msgr2=0x7f6e641f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e641f5040 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6e580108e0 tx=0x7f6e58010da0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 shutdown_connections 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6e4406c8f0 0x7f6e4406edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e640ce400 0x7f6e640b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e64158dc0 0x7f6e640b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 --2- 172.21.15.59:0/2115215369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e4c004d40 0x7f6e641f5040 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.965+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 >> 172.21.15.59:0/2115215369 conn(0x7f6e6408bf90 msgr2=0x7f6e6406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:14.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.966+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 shutdown_connections 2024-04-23T14:40:14.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:14.966+0000 7f6e6a200640 1 -- 172.21.15.59:0/2115215369 wait complete. 2024-04-23T14:40:14.984 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:40:15.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.198+0000 7faa07600640 1 Processor -- start 2024-04-23T14:40:15.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.199+0000 7faa07600640 1 -- start start 2024-04-23T14:40:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa07600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa00145e10 0x7faa0006f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa07600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa00072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa07600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa000cfc20 con 0x7faa000d0cc0 2024-04-23T14:40:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa07600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faa000cff00 con 0x7faa00145e10 2024-04-23T14:40:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa07600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faa000d01e0 con 0x7faa00071d70 2024-04-23T14:40:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa04c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa00145e10 0x7faa0006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa04c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa00145e10 0x7faa0006f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57322/0 (socket says 172.21.15.59:57322) 2024-04-23T14:40:15.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa04c00640 1 -- 172.21.15.59:0/841242142 learned_addr learned my addr 172.21.15.59:0/841242142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:15.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.201+0000 7faa05600640 1 --2- 172.21.15.59:0/841242142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa00072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.202+0000 7faa04c00640 1 -- 172.21.15.59:0/841242142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 msgr2=0x7faa00072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:15.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.202+0000 7faa04c00640 1 --2- 172.21.15.59:0/841242142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa00072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.202+0000 7faa04c00640 1 -- 172.21.15.59:0/841242142 >> v1:172.21.15.76:6789/0 conn(0x7faa000d0cc0 legacy=0x7faa000cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:15.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.202+0000 7faa04c00640 1 -- 172.21.15.59:0/841242142 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa000d04c0 con 0x7faa00145e10 2024-04-23T14:40:15.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.203+0000 7faa04c00640 1 --2- 172.21.15.59:0/841242142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa00145e10 0x7faa0006f130 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009870 tx=0x7fa9f402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bad78edc5092a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:15.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.203+0000 7fa9ff400640 1 -- 172.21.15.59:0/841242142 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4037070 con 0x7faa00145e10 2024-04-23T14:40:15.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.203+0000 7fa9ff400640 1 -- 172.21.15.59:0/841242142 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4032440 con 0x7faa00145e10 2024-04-23T14:40:15.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.203+0000 7fa9ff400640 1 -- 172.21.15.59:0/841242142 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4032810 con 0x7faa00145e10 2024-04-23T14:40:15.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 -- 172.21.15.59:0/841242142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa00145e10 msgr2=0x7faa0006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 --2- 172.21.15.59:0/841242142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa00145e10 0x7faa0006f130 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009870 tx=0x7fa9f402a200 comp rx=0 tx=0).stop 2024-04-23T14:40:15.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 -- 172.21.15.59:0/841242142 shutdown_connections 2024-04-23T14:40:15.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 --2- 172.21.15.59:0/841242142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa00072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 --2- 172.21.15.59:0/841242142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa00145e10 0x7faa0006f130 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 -- 172.21.15.59:0/841242142 >> 172.21.15.59:0/841242142 conn(0x7faa00067d90 msgr2=0x7faa0006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:15.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 -- 172.21.15.59:0/841242142 shutdown_connections 2024-04-23T14:40:15.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.204+0000 7faa07600640 1 -- 172.21.15.59:0/841242142 wait complete. 2024-04-23T14:40:15.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.206+0000 7faa07600640 1 Processor -- start 2024-04-23T14:40:15.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.206+0000 7faa07600640 1 -- start start 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7faa07600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa0013dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7faa07600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa000d0cc0 0x7faa001f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7faa07600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa00145e10 0x7faa00141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7faa07600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa001461f0 con 0x7faa000d0cc0 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7faa07600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa0015cdd0 con 0x7faa00145e10 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7faa07600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faa0013cef0 con 0x7faa00071d70 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7fa9ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa000d0cc0 0x7faa001f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7fa9ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa000d0cc0 0x7faa001f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57338/0 (socket says 172.21.15.59:57338) 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.207+0000 7fa9ffe00640 1 -- 172.21.15.59:0/522689112 learned_addr learned my addr 172.21.15.59:0/522689112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7faa04c00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa0013dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7faa05600640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa00145e10 0x7faa00141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9ffe00640 1 -- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 msgr2=0x7faa0013dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9ffe00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa0013dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9ffe00640 1 -- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa00145e10 msgr2=0x7faa00141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9ffe00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa00145e10 0x7faa00141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9ffe00640 1 -- 172.21.15.59:0/522689112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0013c2e0 con 0x7faa000d0cc0 2024-04-23T14:40:15.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7faa04c00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa0013dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:15.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9ffe00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa000d0cc0 0x7faa001f72b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c430 tx=0x7fa9ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:15.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9fcc00640 1 -- 172.21.15.59:0/522689112 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec011790 con 0x7faa000d0cc0 2024-04-23T14:40:15.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9fcc00640 1 -- 172.21.15.59:0/522689112 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ec017070 con 0x7faa000d0cc0 2024-04-23T14:40:15.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.208+0000 7fa9fcc00640 1 -- 172.21.15.59:0/522689112 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec011bb0 con 0x7faa000d0cc0 2024-04-23T14:40:15.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.209+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0013c5c0 con 0x7faa000d0cc0 2024-04-23T14:40:15.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.209+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0013cb80 con 0x7faa000d0cc0 2024-04-23T14:40:15.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.210+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0013c070 con 0x7faa000d0cc0 2024-04-23T14:40:15.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.211+0000 7fa9fcc00640 1 -- 172.21.15.59:0/522689112 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9ec01c460 con 0x7faa000d0cc0 2024-04-23T14:40:15.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.212+0000 7fa9fcc00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9e006c8f0 0x7fa9e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.212+0000 7fa9fcc00640 1 -- 172.21.15.59:0/522689112 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fa9ec08b9f0 con 0x7faa000d0cc0 2024-04-23T14:40:15.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.212+0000 7faa04c00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9e006c8f0 0x7fa9e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.213+0000 7faa04c00640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9e006c8f0 0x7fa9e006edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4034000 tx=0x7fa9f402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:15.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.216+0000 7fa9fcc00640 1 -- 172.21.15.59:0/522689112 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9ec059e40 con 0x7faa000d0cc0 2024-04-23T14:40:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.424+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa00068a80 con 0x7faa000d0cc0 2024-04-23T14:40:15.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.425+0000 7fa9fcc00640 1 -- 172.21.15.59:0/522689112 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7fa9ec059740 con 0x7faa000d0cc0 2024-04-23T14:40:15.425 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.427+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9e006c8f0 msgr2=0x7fa9e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.427+0000 7faa07600640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9e006c8f0 0x7fa9e006edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4034000 tx=0x7fa9f402e000 comp rx=0 tx=0).stop 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.427+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa000d0cc0 msgr2=0x7faa001f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.427+0000 7faa07600640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa000d0cc0 0x7faa001f72b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c430 tx=0x7fa9ec00c8f0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.427+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 shutdown_connections 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.428+0000 7faa07600640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa9e006c8f0 0x7fa9e006edb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.428+0000 7faa07600640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa00145e10 0x7faa00141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.428+0000 7faa07600640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa000d0cc0 0x7faa001f72b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.428+0000 7faa07600640 1 --2- 172.21.15.59:0/522689112 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa00071d70 0x7faa0013dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.428+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 >> 172.21.15.59:0/522689112 conn(0x7faa00067d90 msgr2=0x7faa0015f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.428+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 shutdown_connections 2024-04-23T14:40:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.428+0000 7faa07600640 1 -- 172.21.15.59:0/522689112 wait complete. 2024-04-23T14:40:15.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:40:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.659+0000 7fd57ea00640 1 Processor -- start 2024-04-23T14:40:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.659+0000 7fd57ea00640 1 -- start start 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.660+0000 7fd57ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd57815c670 0x7fd5780cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.660+0000 7fd57ea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd5780b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.661+0000 7fd57ea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5780b4cf0 con 0x7fd5780b4230 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.661+0000 7fd57ea00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd57815ca50 con 0x7fd57815c670 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.661+0000 7fd57ea00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780ccb80 con 0x7fd5780d0cd0 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.661+0000 7fd57d600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd57815c670 0x7fd5780cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.661+0000 7fd57cc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd5780b4230 0x7fd5780d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58680/0 (socket says 172.21.15.59:58680) 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.661+0000 7fd57cc00640 1 -- 172.21.15.59:0/1437215778 learned_addr learned my addr 172.21.15.59:0/1437215778 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.661+0000 7fd57e000640 1 --2- 172.21.15.59:0/1437215778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd5780b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.662+0000 7fd57d600640 1 -- 172.21.15.59:0/1437215778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 msgr2=0x7fd5780b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.662+0000 7fd577e00640 1 -- 172.21.15.59:0/1437215778 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139081238 0 0) 0x7fd5780b4cf0 con 0x7fd5780b4230 2024-04-23T14:40:15.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.662+0000 7fd57d600640 1 --2- 172.21.15.59:0/1437215778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd5780b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.662+0000 7fd57d600640 1 -- 172.21.15.59:0/1437215778 >> v1:172.21.15.59:6789/0 conn(0x7fd5780b4230 legacy=0x7fd5780d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.662+0000 7fd57d600640 1 -- 172.21.15.59:0/1437215778 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5780ccdd0 con 0x7fd57815c670 2024-04-23T14:40:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.663+0000 7fd57d600640 1 --2- 172.21.15.59:0/1437215778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd5780cd8b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd5780ccd00 tx=0x7fd56c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe03057478ae07dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.664+0000 7fd577e00640 1 -- 172.21.15.59:0/1437215778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c037070 con 0x7fd57815c670 2024-04-23T14:40:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.664+0000 7fd577e00640 1 -- 172.21.15.59:0/1437215778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c032440 con 0x7fd57815c670 2024-04-23T14:40:15.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.664+0000 7fd577e00640 1 -- 172.21.15.59:0/1437215778 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c032810 con 0x7fd57815c670 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.664+0000 7fd57ea00640 1 -- 172.21.15.59:0/1437215778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 msgr2=0x7fd5780cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.664+0000 7fd57ea00640 1 --2- 172.21.15.59:0/1437215778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd5780cd8b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd5780ccd00 tx=0x7fd56c02a180 comp rx=0 tx=0).stop 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.665+0000 7fd57ea00640 1 -- 172.21.15.59:0/1437215778 shutdown_connections 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.665+0000 7fd57ea00640 1 --2- 172.21.15.59:0/1437215778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd5780b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.665+0000 7fd57ea00640 1 --2- 172.21.15.59:0/1437215778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd5780cd8b0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.665+0000 7fd57ea00640 1 -- 172.21.15.59:0/1437215778 >> 172.21.15.59:0/1437215778 conn(0x7fd57806bdf0 msgr2=0x7fd57806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.665+0000 7fd57ea00640 1 -- 172.21.15.59:0/1437215778 shutdown_connections 2024-04-23T14:40:15.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.665+0000 7fd57ea00640 1 -- 172.21.15.59:0/1437215778 wait complete. 2024-04-23T14:40:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.667+0000 7fd57ea00640 1 Processor -- start 2024-04-23T14:40:15.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.667+0000 7fd57ea00640 1 -- start start 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57ea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5780b4230 0x7fd578152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57ea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd57809b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd578158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57ea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780b3e60 con 0x7fd57815c670 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57ea00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780cd390 con 0x7fd5780b4230 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57ea00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780b3fe0 con 0x7fd5780d0cd0 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd578158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57e000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd578158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57370/0 (socket says 172.21.15.59:57370) 2024-04-23T14:40:15.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57e000640 1 -- 172.21.15.59:0/3624558999 learned_addr learned my addr 172.21.15.59:0/3624558999 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57cc00640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd57809b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.668+0000 7fd57d600640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5780b4230 0x7fd578152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57e000640 1 -- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 msgr2=0x7fd57809b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57e000640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd57809b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57e000640 1 -- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5780b4230 msgr2=0x7fd578152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57e000640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5780b4230 0x7fd578152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57e000640 1 -- 172.21.15.59:0/3624558999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57815adc0 con 0x7fd57815c670 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57e000640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd578158c00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd570007e40 tx=0x7fd57000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:15.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd575600640 1 -- 172.21.15.59:0/3624558999 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570014620 con 0x7fd57815c670 2024-04-23T14:40:15.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd575600640 1 -- 172.21.15.59:0/3624558999 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570019070 con 0x7fd57815c670 2024-04-23T14:40:15.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57809b6f0 con 0x7fd57815c670 2024-04-23T14:40:15.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd575600640 1 -- 172.21.15.59:0/3624558999 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570014a20 con 0x7fd57815c670 2024-04-23T14:40:15.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.669+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd578158120 con 0x7fd57815c670 2024-04-23T14:40:15.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.671+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57804fd90 con 0x7fd57815c670 2024-04-23T14:40:15.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.671+0000 7fd575600640 1 -- 172.21.15.59:0/3624558999 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd570014bc0 con 0x7fd57815c670 2024-04-23T14:40:15.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.672+0000 7fd575600640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd55806c8f0 0x7fd55806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:15.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.672+0000 7fd575600640 1 -- 172.21.15.59:0/3624558999 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fd57008c480 con 0x7fd57815c670 2024-04-23T14:40:15.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.673+0000 7fd57d600640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd55806c8f0 0x7fd55806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:15.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.673+0000 7fd57d600640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd55806c8f0 0x7fd55806edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd56c034000 tx=0x7fd56c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:15.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.677+0000 7fd575600640 1 -- 172.21.15.59:0/3624558999 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd57005a840 con 0x7fd57815c670 2024-04-23T14:40:15.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.900+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd5780cfaf0 con 0x7fd57815c670 2024-04-23T14:40:15.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.901+0000 7fd575600640 1 -- 172.21.15.59:0/3624558999 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v86) ==== 80+0+122 (secure 0 0 0) 0x7fd57005a140 con 0x7fd57815c670 2024-04-23T14:40:15.901 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.76:6835/3870021000 2024-04-24T14:39:01.426690+0000 2024-04-23T14:40:15.901 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.76:6833/3870021000 2024-04-24T14:39:01.426690+0000 2024-04-23T14:40:15.902 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd55806c8f0 msgr2=0x7fd55806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd55806c8f0 0x7fd55806edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd56c034000 tx=0x7fd56c02e000 comp rx=0 tx=0).stop 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 msgr2=0x7fd578158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd578158c00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd570007e40 tx=0x7fd57000e4b0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 shutdown_connections 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd55806c8f0 0x7fd55806edb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5780d0cd0 0x7fd57809b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd57815c670 0x7fd578158c00 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 --2- 172.21.15.59:0/3624558999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5780b4230 0x7fd578152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 >> 172.21.15.59:0/3624558999 conn(0x7fd57806bdf0 msgr2=0x7fd578073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 shutdown_connections 2024-04-23T14:40:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:15.904+0000 7fd57ea00640 1 -- 172.21.15.59:0/3624558999 wait complete. 2024-04-23T14:40:15.924 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:40:15.924 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:40:15.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:40:16.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.182+0000 7f80c2400640 1 Processor -- start 2024-04-23T14:40:16.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.182+0000 7f80c2400640 1 -- start start 2024-04-23T14:40:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.183+0000 7f80c2400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 0x7f80bc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.183+0000 7f80c2400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc160210 con 0x7f80bc0cd590 2024-04-23T14:40:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.183+0000 7f80c2400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc1604f0 con 0x7f80bc0b4860 2024-04-23T14:40:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.183+0000 7f80c2400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0cd9f0 con 0x7f80bc15c3d0 2024-04-23T14:40:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.184+0000 7f80bbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f80bc0cd590 0x7f80bc0b4150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58692/0 (socket says 172.21.15.59:58692) 2024-04-23T14:40:16.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.184+0000 7f80bbe00640 1 -- 172.21.15.59:0/2338772185 learned_addr learned my addr 172.21.15.59:0/2338772185 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.184+0000 7f80c1000640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 0x7f80bc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.184+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 361691493 0 0) 0x7f80bc160210 con 0x7f80bc0cd590 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.184+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a4003520 con 0x7f80bc0cd590 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2168373228 0 0) 0x7f80a4003520 con 0x7f80bc0cd590 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 msgr2=0x7f80bc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 0x7f80bc0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 >> v1:172.21.15.76:6789/0 conn(0x7f80bc0b4860 legacy=0x7f80bc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc1607d0 con 0x7f80bc0cd590 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f80a8001670 con 0x7f80bc0cd590 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 >> v1:172.21.15.59:6789/0 conn(0x7f80bc0cd590 legacy=0x7f80bc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a40039e0 0x7f80a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 0x7f80a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80a4004d40 0x7f80a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80a400ba70 con 0x7f80a4004d40 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80a400bc20 con 0x7f80a40039e0 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f80a400be20 con 0x7f80bc15c3d0 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.185+0000 7f80c1a00640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80a4004d40 0x7f80a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80c1000640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a40039e0 0x7f80a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80bbe00640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 0x7f80a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80c1a00640 1 -- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 msgr2=0x7f80a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80c1a00640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc15c3d0 0x7f80a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80c1a00640 1 -- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a40039e0 msgr2=0x7f80a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80c1a00640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a40039e0 0x7f80a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80c1a00640 1 -- 172.21.15.59:0/2338772185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a4003520 con 0x7f80a4004d40 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.186+0000 7f80c1000640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a40039e0 0x7f80a4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:16.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.187+0000 7f80c1a00640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80a4004d40 0x7f80a40051c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f80b4001e20 tx=0x7f80a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361915b6f95da6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.187+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4001a70 con 0x7f80a4004d40 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.187+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b402ae40 con 0x7f80a4004d40 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.187+0000 7f80bb400640 1 -- 172.21.15.59:0/2338772185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4003bb0 con 0x7f80a4004d40 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.188+0000 7f80c2400640 1 -- 172.21.15.59:0/2338772185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80a4004d40 msgr2=0x7f80a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.188+0000 7f80c2400640 1 --2- 172.21.15.59:0/2338772185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80a4004d40 0x7f80a40051c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f80b4001e20 tx=0x7f80a4005830 comp rx=0 tx=0).stop 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.188+0000 7f80c1a00640 1 -- 172.21.15.59:0/2338772185 reap_dead start 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.188+0000 7f80c2400640 1 -- 172.21.15.59:0/2338772185 shutdown_connections 2024-04-23T14:40:16.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.188+0000 7f80c2400640 1 -- 172.21.15.59:0/2338772185 >> 172.21.15.59:0/2338772185 conn(0x7f80bc0b21a0 msgr2=0x7f80bc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:16.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.188+0000 7f80c2400640 1 -- 172.21.15.59:0/2338772185 shutdown_connections 2024-04-23T14:40:16.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.188+0000 7f80c2400640 1 -- 172.21.15.59:0/2338772185 wait complete. 2024-04-23T14:40:16.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.190+0000 7f80c2400640 1 Processor -- start 2024-04-23T14:40:16.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.190+0000 7f80c2400640 1 -- start start 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c2400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a4004d40 0x7f80bc0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c2400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc072dd0 0x7f80bc073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c2400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc0c5450 0x7f80bc0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c2400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc15c7b0 con 0x7f80bc0c5450 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c2400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc160d90 con 0x7f80a4004d40 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c2400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc15c930 con 0x7f80bc072dd0 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc0c5450 0x7f80bc0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc0c5450 0x7f80bc0c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57386/0 (socket says 172.21.15.59:57386) 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 -- 172.21.15.59:0/2931928435 learned_addr learned my addr 172.21.15.59:0/2931928435 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1000640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a4004d40 0x7f80bc0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 -- 172.21.15.59:0/2931928435 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc072dd0 msgr2=0x7f80bc073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc072dd0 0x7f80bc073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 -- 172.21.15.59:0/2931928435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a4004d40 msgr2=0x7f80bc0c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a4004d40 0x7f80bc0c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.191+0000 7f80c1a00640 1 -- 172.21.15.59:0/2931928435 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0c64c0 con 0x7f80bc0c5450 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.192+0000 7f80c1a00640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc0c5450 0x7f80bc0c58d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f80b4001f40 tx=0x7f80b40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.192+0000 7f80b8c00640 1 -- 172.21.15.59:0/2931928435 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4037040 con 0x7f80bc0c5450 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.192+0000 7f80b8c00640 1 -- 172.21.15.59:0/2931928435 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b4041070 con 0x7f80bc0c5450 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.192+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc0c61e0 con 0x7f80bc0c5450 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.192+0000 7f80b8c00640 1 -- 172.21.15.59:0/2931928435 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4001a70 con 0x7f80bc0c5450 2024-04-23T14:40:16.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.192+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc15ce30 con 0x7f80bc0c5450 2024-04-23T14:40:16.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.194+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc04fd90 con 0x7f80bc0c5450 2024-04-23T14:40:16.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.195+0000 7f80b8c00640 1 -- 172.21.15.59:0/2931928435 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80b40052f0 con 0x7f80bc0c5450 2024-04-23T14:40:16.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.195+0000 7f80b8c00640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f809c06c920 0x7f809c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.196+0000 7f80b8c00640 1 -- 172.21.15.59:0/2931928435 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f80b40ab970 con 0x7f80bc0c5450 2024-04-23T14:40:16.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.196+0000 7f80c1000640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f809c06c920 0x7f809c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.197+0000 7f80c1000640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f809c06c920 0x7f809c06ede0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f80b00099c0 tx=0x7f80b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:16.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.200+0000 7f80b8c00640 1 -- 172.21.15.59:0/2931928435 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80b4079cb0 con 0x7f80bc0c5450 2024-04-23T14:40:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.408+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80bc06ee80 con 0x7f80bc0c5450 2024-04-23T14:40:16.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.410+0000 7f80b8c00640 1 -- 172.21.15.59:0/2931928435 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16779 (secure 0 0 0) 0x7f80b40795b0 con 0x7f80bc0c5450 2024-04-23T14:40:16.411 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:16.411 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:39:09.500890+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6835/3870021000":"2024-04-24T14:39:01.426690+0000","172.21.15.76:6833/3870021000":"2024-04-24T14:39:01.426690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:40:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f809c06c920 msgr2=0x7f809c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f809c06c920 0x7f809c06ede0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f80b00099c0 tx=0x7f80b0008040 comp rx=0 tx=0).stop 2024-04-23T14:40:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc0c5450 msgr2=0x7f80bc0c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc0c5450 0x7f80bc0c58d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f80b4001f40 tx=0x7f80b40014a0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 shutdown_connections 2024-04-23T14:40:16.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f809c06c920 0x7f809c06ede0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80bc072dd0 0x7f80bc073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80bc0c5450 0x7f80bc0c58d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 --2- 172.21.15.59:0/2931928435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80a4004d40 0x7f80bc0c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 >> 172.21.15.59:0/2931928435 conn(0x7f80bc0b21a0 msgr2=0x7f80bc0b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 shutdown_connections 2024-04-23T14:40:16.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.413+0000 7f80c2400640 1 -- 172.21.15.59:0/2931928435 wait complete. 2024-04-23T14:40:16.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:38:36.964433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:38:38.188587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:40:16.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:40:16.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.640+0000 7f3493e00640 1 Processor -- start 2024-04-23T14:40:16.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.640+0000 7f3493e00640 1 -- start start 2024-04-23T14:40:16.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.641+0000 7f3493e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3494155280 0x7f349415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.642+0000 7f3493e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f349406efe0 con 0x7f349406e740 2024-04-23T14:40:16.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.642+0000 7f3493e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f349406f2c0 con 0x7f3494072610 2024-04-23T14:40:16.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.642+0000 7f3493e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f34940d0d10 con 0x7f3494155280 2024-04-23T14:40:16.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.642+0000 7f3492a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3494155280 0x7f349415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.642+0000 7f3492000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f349406e740 0x7f34940d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58708/0 (socket says 172.21.15.59:58708) 2024-04-23T14:40:16.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.642+0000 7f3492a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3494155280 0x7f349415bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57396/0 (socket says 172.21.15.59:57396) 2024-04-23T14:40:16.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.642+0000 7f3492000640 1 -- 172.21.15.59:0/2564389274 learned_addr learned my addr 172.21.15.59:0/2564389274 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:16.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.643+0000 7f3492a00640 1 -- 172.21.15.59:0/2564389274 >> v1:172.21.15.76:6789/0 conn(0x7f3494072610 legacy=0x7f3494165a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:16.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.643+0000 7f3492a00640 1 -- 172.21.15.59:0/2564389274 >> v1:172.21.15.59:6789/0 conn(0x7f349406e740 legacy=0x7f34940d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.643+0000 7f3492a00640 1 -- 172.21.15.59:0/2564389274 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349406f5a0 con 0x7f3494155280 2024-04-23T14:40:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.643+0000 7f3491600640 1 -- 172.21.15.59:0/2564389274 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773223804 0 0) 0x7f349406efe0 con 0x7f349406e740 2024-04-23T14:40:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.644+0000 7f3492a00640 1 --2- 172.21.15.59:0/2564389274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f349415bc80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f348400ba70 tx=0x7f348402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dac0268b8f5e472 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.644+0000 7f3491600640 1 -- 172.21.15.59:0/2564389274 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484032790 con 0x7f3494155280 2024-04-23T14:40:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.644+0000 7f3491600640 1 -- 172.21.15.59:0/2564389274 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3484038070 con 0x7f3494155280 2024-04-23T14:40:16.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.644+0000 7f3491600640 1 -- 172.21.15.59:0/2564389274 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484032b50 con 0x7f3494155280 2024-04-23T14:40:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.645+0000 7f3493e00640 1 -- 172.21.15.59:0/2564389274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 msgr2=0x7f349415bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.645+0000 7f3493e00640 1 --2- 172.21.15.59:0/2564389274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f349415bc80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f348400ba70 tx=0x7f348402c8d0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.646+0000 7f3493e00640 1 -- 172.21.15.59:0/2564389274 shutdown_connections 2024-04-23T14:40:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.646+0000 7f3493e00640 1 --2- 172.21.15.59:0/2564389274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f349415bc80 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.646+0000 7f3493e00640 1 -- 172.21.15.59:0/2564389274 >> 172.21.15.59:0/2564389274 conn(0x7f349406bdf0 msgr2=0x7f349406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.646+0000 7f3493e00640 1 -- 172.21.15.59:0/2564389274 shutdown_connections 2024-04-23T14:40:16.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.646+0000 7f3493e00640 1 -- 172.21.15.59:0/2564389274 wait complete. 2024-04-23T14:40:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.647+0000 7f3493e00640 1 Processor -- start 2024-04-23T14:40:16.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.648+0000 7f3493e00640 1 -- start start 2024-04-23T14:40:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.648+0000 7f3493e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f349406e740 0x7f3494155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3494072610 0x7f3494099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f34940b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f349406eb80 con 0x7f3494155280 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f349406fb60 con 0x7f3494072610 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f349406ed00 con 0x7f349406e740 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f34940b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f34940b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57398/0 (socket says 172.21.15.59:57398) 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 -- 172.21.15.59:0/2797309625 learned_addr learned my addr 172.21.15.59:0/2797309625 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3492a00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f349406e740 0x7f3494155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3492000640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3494072610 0x7f3494099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 -- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f349406e740 msgr2=0x7f3494155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f349406e740 0x7f3494155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 -- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3494072610 msgr2=0x7f3494099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3494072610 0x7f3494099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.649+0000 7f3493400640 1 -- 172.21.15.59:0/2797309625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3494155c40 con 0x7f3494155280 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.650+0000 7f3493400640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f34940b3570 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c780 tx=0x7f348c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.650+0000 7f348aa00640 1 -- 172.21.15.59:0/2797309625 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c004b60 con 0x7f3494155280 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.650+0000 7f348aa00640 1 -- 172.21.15.59:0/2797309625 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348c004d00 con 0x7f3494155280 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.650+0000 7f348aa00640 1 -- 172.21.15.59:0/2797309625 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c00aa90 con 0x7f3494155280 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.650+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3494155f20 con 0x7f3494155280 2024-04-23T14:40:16.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.650+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34941564e0 con 0x7f3494155280 2024-04-23T14:40:16.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.651+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349404fd90 con 0x7f3494155280 2024-04-23T14:40:16.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.652+0000 7f348aa00640 1 -- 172.21.15.59:0/2797309625 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f348c00ac30 con 0x7f3494155280 2024-04-23T14:40:16.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.653+0000 7f348aa00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f347406c8f0 0x7f347406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.653+0000 7f348aa00640 1 -- 172.21.15.59:0/2797309625 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f348c08bf40 con 0x7f3494155280 2024-04-23T14:40:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.654+0000 7f3492a00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f347406c8f0 0x7f347406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.654+0000 7f3492a00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f347406c8f0 0x7f347406edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3484037000 tx=0x7f3484032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:16.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.658+0000 7f348aa00640 1 -- 172.21.15.59:0/2797309625 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f348c020070 con 0x7f3494155280 2024-04-23T14:40:16.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.865+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f34940d0820 con 0x7f3494155280 2024-04-23T14:40:16.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.866+0000 7f348aa00640 1 -- 172.21.15.59:0/2797309625 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f348c05a280 con 0x7f3494155280 2024-04-23T14:40:16.866 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:40:16.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f347406c8f0 msgr2=0x7f347406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f347406c8f0 0x7f347406edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3484037000 tx=0x7f3484032000 comp rx=0 tx=0).stop 2024-04-23T14:40:16.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 msgr2=0x7f34940b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f34940b3570 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c780 tx=0x7f348c00cc40 comp rx=0 tx=0).stop 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 shutdown_connections 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f347406c8f0 0x7f347406edb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3494155280 0x7f34940b3570 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3494072610 0x7f3494099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 --2- 172.21.15.59:0/2797309625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f349406e740 0x7f3494155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 >> 172.21.15.59:0/2797309625 conn(0x7f349406bdf0 msgr2=0x7f349415c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 shutdown_connections 2024-04-23T14:40:16.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:16.869+0000 7f3493e00640 1 -- 172.21.15.59:0/2797309625 wait complete. 2024-04-23T14:40:16.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:40:17.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.097+0000 7fee84200640 1 Processor -- start 2024-04-23T14:40:17.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.097+0000 7fee84200640 1 -- start start 2024-04-23T14:40:17.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.099+0000 7fee84200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c145e10 0x7fee7c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.099+0000 7fee84200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fee7c071d70 0x7fee7c072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.099+0000 7fee84200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c0cfc20 con 0x7fee7c0d0cc0 2024-04-23T14:40:17.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.099+0000 7fee84200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fee7c0cff00 con 0x7fee7c071d70 2024-04-23T14:40:17.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.099+0000 7fee84200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c0d01e0 con 0x7fee7c145e10 2024-04-23T14:40:17.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.099+0000 7fee80e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fee7c0d0cc0 0x7fee7c0cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58714/0 (socket says 172.21.15.59:58714) 2024-04-23T14:40:17.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.099+0000 7fee80e00640 1 -- 172.21.15.59:0/1330744753 learned_addr learned my addr 172.21.15.59:0/1330744753 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee82200640 1 --2- 172.21.15.59:0/1330744753 >> v2:172.21.15.76:3300/0 conn(0x7fee7c071d70 0x7fee7c072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee81800640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c145e10 0x7fee7c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698291968 0 0) 0x7fee7c0cfc20 con 0x7fee7c0d0cc0 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee64003520 con 0x7fee7c0d0cc0 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1167643902 0 0) 0x7fee64003520 con 0x7fee7c0d0cc0 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c145e10 msgr2=0x7fee7c06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c145e10 0x7fee7c06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 >> v2:172.21.15.76:3300/0 conn(0x7fee7c071d70 msgr2=0x7fee7c072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:17.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 --2- 172.21.15.59:0/1330744753 >> v2:172.21.15.76:3300/0 conn(0x7fee7c071d70 0x7fee7c072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c0d04c0 con 0x7fee7c0d0cc0 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fee68001670 con 0x7fee7c0d0cc0 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 >> v1:172.21.15.59:6789/0 conn(0x7fee7c0d0cc0 legacy=0x7fee7c0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee640039b0 0x7fee64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.100+0000 7fee7be00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee7be00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c0cff00 con 0x7fee64004800 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c0d01e0 con 0x7fee640051c0 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee6400bf00 con 0x7fee640039b0 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee80e00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee81800640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee640039b0 0x7fee64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee82200640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee80e00640 1 -- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee640039b0 msgr2=0x7fee64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:17.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee80e00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee640039b0 0x7fee64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee80e00640 1 -- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee640051c0 msgr2=0x7fee6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:17.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee80e00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee80e00640 1 -- 172.21.15.59:0/1330744753 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee64003520 con 0x7fee64004800 2024-04-23T14:40:17.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.101+0000 7fee82200640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee640051c0 0x7fee6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.102+0000 7fee80e00640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee64004c80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c110 tx=0x7fee64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19127c5334f70602 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.103+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68007c20 con 0x7fee64004800 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.103+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee68005af0 con 0x7fee64004800 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.103+0000 7fee7be00640 1 -- 172.21.15.59:0/1330744753 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68007070 con 0x7fee64004800 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.103+0000 7fee84200640 1 -- 172.21.15.59:0/1330744753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 msgr2=0x7fee64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.103+0000 7fee84200640 1 --2- 172.21.15.59:0/1330744753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee64004c80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c110 tx=0x7fee64005830 comp rx=0 tx=0).stop 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.103+0000 7fee82200640 1 -- 172.21.15.59:0/1330744753 reap_dead start 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.104+0000 7fee84200640 1 -- 172.21.15.59:0/1330744753 shutdown_connections 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.104+0000 7fee84200640 1 -- 172.21.15.59:0/1330744753 >> 172.21.15.59:0/1330744753 conn(0x7fee7c067d90 msgr2=0x7fee7c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.104+0000 7fee84200640 1 -- 172.21.15.59:0/1330744753 shutdown_connections 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.104+0000 7fee84200640 1 -- 172.21.15.59:0/1330744753 wait complete. 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.105+0000 7fee84200640 1 Processor -- start 2024-04-23T14:40:17.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.106+0000 7fee84200640 1 -- start start 2024-04-23T14:40:17.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee84200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee7c13cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee84200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee7c143ed0 0x7fee7c144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee84200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c0ad870 0x7fee7c0adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee84200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1461f0 con 0x7fee64004800 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee84200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c06f130 con 0x7fee7c143ed0 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee84200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c13d450 con 0x7fee7c0ad870 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee7c13cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee7c13cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57408/0 (socket says 172.21.15.59:57408) 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 -- 172.21.15.59:0/1937155754 learned_addr learned my addr 172.21.15.59:0/1937155754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee82200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c0ad870 0x7fee7c0adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee80e00640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee7c143ed0 0x7fee7c144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 -- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c0ad870 msgr2=0x7fee7c0adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c0ad870 0x7fee7c0adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 -- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee7c143ed0 msgr2=0x7fee7c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee7c143ed0 0x7fee7c144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee81800640 1 -- 172.21.15.59:0/1937155754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c141880 con 0x7fee64004800 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee82200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c0ad870 0x7fee7c0adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.107+0000 7fee80e00640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee7c143ed0 0x7fee7c144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.108+0000 7fee81800640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee7c13cf10 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fee70007f80 tx=0x7fee70011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.108+0000 7fee79600640 1 -- 172.21.15.59:0/1937155754 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7001a020 con 0x7fee64004800 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.108+0000 7fee79600640 1 -- 172.21.15.59:0/1937155754 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee70016470 con 0x7fee64004800 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.108+0000 7fee79600640 1 -- 172.21.15.59:0/1937155754 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70016840 con 0x7fee64004800 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.108+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c0ae5a0 con 0x7fee64004800 2024-04-23T14:40:17.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.108+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c141b40 con 0x7fee64004800 2024-04-23T14:40:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.110+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c068a80 con 0x7fee64004800 2024-04-23T14:40:17.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.111+0000 7fee79600640 1 -- 172.21.15.59:0/1937155754 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee70005d10 con 0x7fee64004800 2024-04-23T14:40:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.112+0000 7fee79600640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee5c06c8f0 0x7fee5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.112+0000 7fee79600640 1 -- 172.21.15.59:0/1937155754 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fee7008caf0 con 0x7fee64004800 2024-04-23T14:40:17.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.112+0000 7fee80e00640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee5c06c8f0 0x7fee5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.113+0000 7fee80e00640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee5c06c8f0 0x7fee5c06edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fee68032000 tx=0x7fee68009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:17.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.116+0000 7fee79600640 1 -- 172.21.15.59:0/1937155754 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee70024030 con 0x7fee64004800 2024-04-23T14:40:17.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.319+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fee7c0d1190 con 0x7fee64004800 2024-04-23T14:40:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.320+0000 7fee79600640 1 -- 172.21.15.59:0/1937155754 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7fee7005ae30 con 0x7fee64004800 2024-04-23T14:40:17.323 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:40:17.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee5c06c8f0 msgr2=0x7fee5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee5c06c8f0 0x7fee5c06edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fee68032000 tx=0x7fee68009030 comp rx=0 tx=0).stop 2024-04-23T14:40:17.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 msgr2=0x7fee7c13cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee7c13cf10 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fee70007f80 tx=0x7fee70011660 comp rx=0 tx=0).stop 2024-04-23T14:40:17.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 shutdown_connections 2024-04-23T14:40:17.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee5c06c8f0 0x7fee5c06edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee7c0ad870 0x7fee7c0adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee7c143ed0 0x7fee7c144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 --2- 172.21.15.59:0/1937155754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee64004800 0x7fee7c13cf10 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 >> 172.21.15.59:0/1937155754 conn(0x7fee7c067d90 msgr2=0x7fee7c15e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:17.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 shutdown_connections 2024-04-23T14:40:17.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.323+0000 7fee84200640 1 -- 172.21.15.59:0/1937155754 wait complete. 2024-04-23T14:40:17.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:40:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.564+0000 7f64c1600640 1 Processor -- start 2024-04-23T14:40:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.564+0000 7f64c1600640 1 -- start start 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64c1600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64bc155a80 0x7f64bc06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64c1600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64c1600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64bc15ef20 con 0x7f64bc0d0600 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64c1600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f64bc15f1c0 con 0x7f64bc155a80 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64c1600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f64bc15f4a0 con 0x7f64bc0af2a0 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64bbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64bc155a80 0x7f64bc06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64bbe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64bc155a80 0x7f64bc06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57410/0 (socket says 172.21.15.59:57410) 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.566+0000 7f64bbe00640 1 -- 172.21.15.59:0/2981313791 learned_addr learned my addr 172.21.15.59:0/2981313791 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.567+0000 7f64c0c00640 1 --2- 172.21.15.59:0/2981313791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.567+0000 7f64bbe00640 1 -- 172.21.15.59:0/2981313791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 msgr2=0x7f64bc0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.567+0000 7f64bbe00640 1 --2- 172.21.15.59:0/2981313791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.567+0000 7f64bbe00640 1 -- 172.21.15.59:0/2981313791 >> v1:172.21.15.76:6789/0 conn(0x7f64bc0d0600 legacy=0x7f64bc0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.567+0000 7f64bbe00640 1 -- 172.21.15.59:0/2981313791 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc15f780 con 0x7f64bc155a80 2024-04-23T14:40:17.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.568+0000 7f64bbe00640 1 --2- 172.21.15.59:0/2981313791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc06f9c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009870 tx=0x7f64ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc395354fe53acd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64baa00640 1 -- 172.21.15.59:0/2981313791 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac037070 con 0x7f64bc155a80 2024-04-23T14:40:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64baa00640 1 -- 172.21.15.59:0/2981313791 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ac032440 con 0x7f64bc155a80 2024-04-23T14:40:17.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64baa00640 1 -- 172.21.15.59:0/2981313791 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac032810 con 0x7f64bc155a80 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64c1600640 1 -- 172.21.15.59:0/2981313791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 msgr2=0x7f64bc06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64c1600640 1 --2- 172.21.15.59:0/2981313791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc06f9c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009870 tx=0x7f64ac02a210 comp rx=0 tx=0).stop 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64c1600640 1 -- 172.21.15.59:0/2981313791 shutdown_connections 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64c1600640 1 --2- 172.21.15.59:0/2981313791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64c1600640 1 --2- 172.21.15.59:0/2981313791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc06f9c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.569+0000 7f64c1600640 1 -- 172.21.15.59:0/2981313791 >> 172.21.15.59:0/2981313791 conn(0x7f64bc06bdf0 msgr2=0x7f64bc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.570+0000 7f64c1600640 1 -- 172.21.15.59:0/2981313791 shutdown_connections 2024-04-23T14:40:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.570+0000 7f64c1600640 1 -- 172.21.15.59:0/2981313791 wait complete. 2024-04-23T14:40:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.571+0000 7f64c1600640 1 Processor -- start 2024-04-23T14:40:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.571+0000 7f64c1600640 1 -- start start 2024-04-23T14:40:17.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.572+0000 7f64c1600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.572+0000 7f64c1600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64bc0d0600 0x7f64bc0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.572+0000 7f64c1600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.572+0000 7f64c1600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc06ff00 con 0x7f64bc155a80 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.572+0000 7f64c1600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc15fd40 con 0x7f64bc0d0600 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.572+0000 7f64c1600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f64bc06ef80 con 0x7f64bc0af2a0 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57414/0 (socket says 172.21.15.59:57414) 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 -- 172.21.15.59:0/1784837933 learned_addr learned my addr 172.21.15.59:0/1784837933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64bbe00640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64bb400640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64bc0d0600 0x7f64bc0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 -- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 msgr2=0x7f64bc0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 -- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64bc0d0600 msgr2=0x7f64bc0afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64bc0d0600 0x7f64bc0afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.573+0000 7f64c0c00640 1 -- 172.21.15.59:0/1784837933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc152850 con 0x7f64bc155a80 2024-04-23T14:40:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.574+0000 7f64c0c00640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc0b03e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f64b4007e60 tx=0x7f64b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.574+0000 7f64b3e00640 1 -- 172.21.15.59:0/1784837933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4014620 con 0x7f64bc155a80 2024-04-23T14:40:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.574+0000 7f64b3e00640 1 -- 172.21.15.59:0/1784837933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4019070 con 0x7f64bc155a80 2024-04-23T14:40:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.574+0000 7f64b3e00640 1 -- 172.21.15.59:0/1784837933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4014a00 con 0x7f64bc155a80 2024-04-23T14:40:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.574+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc152b30 con 0x7f64bc155a80 2024-04-23T14:40:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.574+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc0b20b0 con 0x7f64bc155a80 2024-04-23T14:40:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.575+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc04fd90 con 0x7f64bc155a80 2024-04-23T14:40:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.576+0000 7f64b3e00640 1 -- 172.21.15.59:0/1784837933 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64b4014ba0 con 0x7f64bc155a80 2024-04-23T14:40:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.578+0000 7f64b3e00640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f649c06c8f0 0x7f649c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.578+0000 7f64b3e00640 1 -- 172.21.15.59:0/1784837933 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f64b408c4d0 con 0x7f64bc155a80 2024-04-23T14:40:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.578+0000 7f64bbe00640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f649c06c8f0 0x7f649c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.579+0000 7f64bbe00640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f649c06c8f0 0x7f649c06edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f64ac034000 tx=0x7f64ac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.582+0000 7f64b3e00640 1 -- 172.21.15.59:0/1784837933 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64b405a8a0 con 0x7f64bc155a80 2024-04-23T14:40:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.787+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64bc06eba0 con 0x7f64bc155a80 2024-04-23T14:40:17.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.788+0000 7f64b3e00640 1 -- 172.21.15.59:0/1784837933 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f64b405a1a0 con 0x7f64bc155a80 2024-04-23T14:40:17.788 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:40:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f649c06c8f0 msgr2=0x7f649c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f649c06c8f0 0x7f649c06edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f64ac034000 tx=0x7f64ac02e000 comp rx=0 tx=0).stop 2024-04-23T14:40:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 msgr2=0x7f64bc0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc0b03e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f64b4007e60 tx=0x7f64b400e4b0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 shutdown_connections 2024-04-23T14:40:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f649c06c8f0 0x7f649c06edb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc155a80 0x7f64bc0b03e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64bc0d0600 0x7f64bc0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 --2- 172.21.15.59:0/1784837933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64bc0af2a0 0x7f64bc0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 >> 172.21.15.59:0/1784837933 conn(0x7f64bc06bdf0 msgr2=0x7f64bc15d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 shutdown_connections 2024-04-23T14:40:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:17.791+0000 7f64c1600640 1 -- 172.21.15.59:0/1784837933 wait complete. 2024-04-23T14:40:17.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:40:18.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.012+0000 7fda22200640 1 Processor -- start 2024-04-23T14:40:18.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.012+0000 7fda22200640 1 -- start start 2024-04-23T14:40:18.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.013+0000 7fda22200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.013+0000 7fda22200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda1c0d0cd0 0x7fda1c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.013+0000 7fda22200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1c0b4cf0 con 0x7fda1c0b4230 2024-04-23T14:40:18.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.013+0000 7fda22200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fda1c15ca50 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.013+0000 7fda22200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c0ccb80 con 0x7fda1c15c670 2024-04-23T14:40:18.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.014+0000 7fda20e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.014+0000 7fda20e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:47710/0 (socket says 172.21.15.59:47710) 2024-04-23T14:40:18.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.014+0000 7fda20e00640 1 -- 172.21.15.59:0/1499982291 learned_addr learned my addr 172.21.15.59:0/1499982291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:18.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.014+0000 7fda21800640 1 --2- 172.21.15.59:0/1499982291 >> v2:172.21.15.76:3300/0 conn(0x7fda1c0d0cd0 0x7fda1c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.015+0000 7fda20e00640 1 -- 172.21.15.59:0/1499982291 >> v2:172.21.15.76:3300/0 conn(0x7fda1c0d0cd0 msgr2=0x7fda1c0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:18.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.015+0000 7fda20e00640 1 --2- 172.21.15.59:0/1499982291 >> v2:172.21.15.76:3300/0 conn(0x7fda1c0d0cd0 0x7fda1c0b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.015+0000 7fda20e00640 1 -- 172.21.15.59:0/1499982291 >> v1:172.21.15.76:6789/0 conn(0x7fda1c0b4230 legacy=0x7fda1c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:18.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.015+0000 7fda20e00640 1 -- 172.21.15.59:0/1499982291 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c0ccdd0 con 0x7fda1c15c670 2024-04-23T14:40:18.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.016+0000 7fda20e00640 1 --2- 172.21.15.59:0/1499982291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c0cd8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009e50 tx=0x7fda0c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22501f96eb53cb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:18.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.016+0000 7fda1b400640 1 -- 172.21.15.59:0/1499982291 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c036070 con 0x7fda1c15c670 2024-04-23T14:40:18.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.016+0000 7fda1b400640 1 -- 172.21.15.59:0/1499982291 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c031600 con 0x7fda1c15c670 2024-04-23T14:40:18.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.016+0000 7fda1b400640 1 -- 172.21.15.59:0/1499982291 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c0319d0 con 0x7fda1c15c670 2024-04-23T14:40:18.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.017+0000 7fda22200640 1 -- 172.21.15.59:0/1499982291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 msgr2=0x7fda1c0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.017+0000 7fda22200640 1 --2- 172.21.15.59:0/1499982291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c0cd8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009e50 tx=0x7fda0c02a6c0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.017+0000 7fda22200640 1 -- 172.21.15.59:0/1499982291 shutdown_connections 2024-04-23T14:40:18.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.017+0000 7fda22200640 1 --2- 172.21.15.59:0/1499982291 >> v2:172.21.15.76:3300/0 conn(0x7fda1c0d0cd0 0x7fda1c0b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.017+0000 7fda22200640 1 --2- 172.21.15.59:0/1499982291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c0cd8b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.017+0000 7fda22200640 1 -- 172.21.15.59:0/1499982291 >> 172.21.15.59:0/1499982291 conn(0x7fda1c06bdf0 msgr2=0x7fda1c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:18.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.018+0000 7fda22200640 1 -- 172.21.15.59:0/1499982291 shutdown_connections 2024-04-23T14:40:18.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.018+0000 7fda22200640 1 -- 172.21.15.59:0/1499982291 wait complete. 2024-04-23T14:40:18.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.019+0000 7fda22200640 1 Processor -- start 2024-04-23T14:40:18.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.019+0000 7fda22200640 1 -- start start 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.020+0000 7fda22200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c0b4230 0x7fda1c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.020+0000 7fda22200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1c0d0cd0 0x7fda1c147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.020+0000 7fda22200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.020+0000 7fda22200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c0b3e60 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.020+0000 7fda22200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c0cd390 con 0x7fda1c0b4230 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.020+0000 7fda22200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c0b3fe0 con 0x7fda1c15c670 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1c0d0cd0 0x7fda1c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1c0d0cd0 0x7fda1c147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57426/0 (socket says 172.21.15.59:57426) 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 -- 172.21.15.59:0/917984501 learned_addr learned my addr 172.21.15.59:0/917984501 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda21800640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda20e00640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c0b4230 0x7fda1c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 -- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 msgr2=0x7fda1c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 -- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c0b4230 msgr2=0x7fda1c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c0b4230 0x7fda1c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.021+0000 7fda1be00640 1 -- 172.21.15.59:0/917984501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c1f8e00 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.022+0000 7fda1be00640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1c0d0cd0 0x7fda1c147740 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c400 tx=0x7fda0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.022+0000 7fda18c00640 1 -- 172.21.15.59:0/917984501 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08011790 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.022+0000 7fda18c00640 1 -- 172.21.15.59:0/917984501 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda08017070 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.022+0000 7fda18c00640 1 -- 172.21.15.59:0/917984501 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08011bb0 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.022+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c1f8b20 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.022+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c0b17f0 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.023+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c04fd90 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.025+0000 7fda18c00640 1 -- 172.21.15.59:0/917984501 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda0801c460 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.025+0000 7fda18c00640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.026+0000 7fda18c00640 1 -- 172.21.15.59:0/917984501 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fda0808ba60 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.026+0000 7fda20e00640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.027+0000 7fda20e00640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fda0c033000 tx=0x7fda0c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:18.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.030+0000 7fda18c00640 1 -- 172.21.15.59:0/917984501 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda08059e20 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.242+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda1c0ceab0 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.243+0000 7fda18c00640 1 -- 172.21.15.59:0/917984501 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v86) ==== 80+0+122 (secure 0 0 0) 0x7fda08059720 con 0x7fda1c0d0cd0 2024-04-23T14:40:18.243 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.76:6835/3870021000 2024-04-24T14:39:01.426690+0000 2024-04-23T14:40:18.243 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.76:6833/3870021000 2024-04-24T14:39:01.426690+0000 2024-04-23T14:40:18.243 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.245+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 msgr2=0x7fd9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.245+0000 7fda22200640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fda0c033000 tx=0x7fda0c009000 comp rx=0 tx=0).stop 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.245+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1c0d0cd0 msgr2=0x7fda1c147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.245+0000 7fda22200640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1c0d0cd0 0x7fda1c147740 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c400 tx=0x7fda0800c8c0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 shutdown_connections 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c15c670 0x7fda1c071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1c0d0cd0 0x7fda1c147740 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 --2- 172.21.15.59:0/917984501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c0b4230 0x7fda1c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 >> 172.21.15.59:0/917984501 conn(0x7fda1c06bdf0 msgr2=0x7fda1c15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:18.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 shutdown_connections 2024-04-23T14:40:18.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:18.246+0000 7fda22200640 1 -- 172.21.15.59:0/917984501 wait complete. 2024-04-23T14:40:18.265 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:40:18.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:18.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:40:18.344 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[64972]: fuse finished with error 0 and tester_r 0 2024-04-23T14:40:18.349 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:40:20.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:40:20.951 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:40:24.451 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:40:24.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:24.451 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:40:24.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:40:24.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.729+0000 7fe3dc200640 1 Processor -- start 2024-04-23T14:40:24.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.729+0000 7fe3dc200640 1 -- start start 2024-04-23T14:40:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.731+0000 7fe3dc200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3d41644b0 0x7fe3d4163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.731+0000 7fe3dc200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3d40b4550 0x7fe3d406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.731+0000 7fe3dc200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d4160d40 con 0x7fe3d4160960 2024-04-23T14:40:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.731+0000 7fe3dc200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d4164270 con 0x7fe3d41644b0 2024-04-23T14:40:24.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.731+0000 7fe3dc200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d4166de0 con 0x7fe3d40b4550 2024-04-23T14:40:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3d8e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3d41644b0 0x7fe3d4163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3d8e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3d41644b0 0x7fe3d4163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51866/0 (socket says 172.21.15.59:51866) 2024-04-23T14:40:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3d8e00640 1 -- 172.21.15.59:0/2571659594 learned_addr learned my addr 172.21.15.59:0/2571659594 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:24.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3da200640 1 --2- 172.21.15.59:0/2571659594 >> v2:172.21.15.76:3300/0 conn(0x7fe3d40b4550 0x7fe3d406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3d8e00640 1 -- 172.21.15.59:0/2571659594 >> v2:172.21.15.76:3300/0 conn(0x7fe3d40b4550 msgr2=0x7fe3d406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3d8e00640 1 --2- 172.21.15.59:0/2571659594 >> v2:172.21.15.76:3300/0 conn(0x7fe3d40b4550 0x7fe3d406e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3d8e00640 1 -- 172.21.15.59:0/2571659594 >> v1:172.21.15.76:6789/0 conn(0x7fe3d4160960 legacy=0x7fe3d40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.732+0000 7fe3d8e00640 1 -- 172.21.15.59:0/2571659594 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d41670c0 con 0x7fe3d41644b0 2024-04-23T14:40:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.734+0000 7fe3d8e00640 1 --2- 172.21.15.59:0/2571659594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d41644b0 0x7fe3d4163d30 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe3d406df70 tx=0x7fe3c802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352cd3a54f43982f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.734+0000 7fe3d3e00640 1 -- 172.21.15.59:0/2571659594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8036070 con 0x7fe3d41644b0 2024-04-23T14:40:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.734+0000 7fe3d3e00640 1 -- 172.21.15.59:0/2571659594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c8031600 con 0x7fe3d41644b0 2024-04-23T14:40:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.734+0000 7fe3d3e00640 1 -- 172.21.15.59:0/2571659594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80319d0 con 0x7fe3d41644b0 2024-04-23T14:40:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 -- 172.21.15.59:0/2571659594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d41644b0 msgr2=0x7fe3d4163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 --2- 172.21.15.59:0/2571659594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d41644b0 0x7fe3d4163d30 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe3d406df70 tx=0x7fe3c802a6c0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 -- 172.21.15.59:0/2571659594 shutdown_connections 2024-04-23T14:40:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 --2- 172.21.15.59:0/2571659594 >> v2:172.21.15.76:3300/0 conn(0x7fe3d40b4550 0x7fe3d406e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 --2- 172.21.15.59:0/2571659594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d41644b0 0x7fe3d4163d30 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 -- 172.21.15.59:0/2571659594 >> 172.21.15.59:0/2571659594 conn(0x7fe3d4067ea0 msgr2=0x7fe3d406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:24.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 -- 172.21.15.59:0/2571659594 shutdown_connections 2024-04-23T14:40:24.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.735+0000 7fe3dc200640 1 -- 172.21.15.59:0/2571659594 wait complete. 2024-04-23T14:40:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.737+0000 7fe3dc200640 1 Processor -- start 2024-04-23T14:40:24.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.738+0000 7fe3dc200640 1 -- start start 2024-04-23T14:40:24.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3dc200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d40b4550 0x7fe3d40aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:24.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3dc200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4160960 0x7fe3d4160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:24.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3dc200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3d41644b0 0x7fe3d4161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:24.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3dc200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41668e0 con 0x7fe3d4160960 2024-04-23T14:40:24.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3dc200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4167680 con 0x7fe3d41644b0 2024-04-23T14:40:24.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3dc200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d4166a60 con 0x7fe3d40b4550 2024-04-23T14:40:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3d8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4160960 0x7fe3d4160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3d8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4160960 0x7fe3d4160f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51868/0 (socket says 172.21.15.59:51868) 2024-04-23T14:40:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3d8e00640 1 -- 172.21.15.59:0/424649454 learned_addr learned my addr 172.21.15.59:0/424649454 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3da200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3d41644b0 0x7fe3d4161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.739+0000 7fe3d9800640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d40b4550 0x7fe3d40aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d8e00640 1 -- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d40b4550 msgr2=0x7fe3d40aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d8e00640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d40b4550 0x7fe3d40aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d8e00640 1 -- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3d41644b0 msgr2=0x7fe3d4161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d8e00640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3d41644b0 0x7fe3d4161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d8e00640 1 -- 172.21.15.59:0/424649454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d40cf560 con 0x7fe3d4160960 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d9800640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d40b4550 0x7fe3d40aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3da200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3d41644b0 0x7fe3d4161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d8e00640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4160960 0x7fe3d4160f10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009f10 tx=0x7fe3c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d1600640 1 -- 172.21.15.59:0/424649454 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8036050 con 0x7fe3d4160960 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d1600640 1 -- 172.21.15.59:0/424649454 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c8042070 con 0x7fe3d4160960 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.740+0000 7fe3d1600640 1 -- 172.21.15.59:0/424649454 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8009510 con 0x7fe3d4160960 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.741+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d40cecc0 con 0x7fe3d4160960 2024-04-23T14:40:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.741+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d40cf280 con 0x7fe3d4160960 2024-04-23T14:40:24.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.742+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d4068a80 con 0x7fe3d4160960 2024-04-23T14:40:24.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.743+0000 7fe3d1600640 1 -- 172.21.15.59:0/424649454 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3c8007c50 con 0x7fe3d4160960 2024-04-23T14:40:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.744+0000 7fe3d1600640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b806c8f0 0x7fe3b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.744+0000 7fe3d1600640 1 -- 172.21.15.59:0/424649454 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7fe3c80acb00 con 0x7fe3d4160960 2024-04-23T14:40:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.744+0000 7fe3d9800640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b806c8f0 0x7fe3b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.745+0000 7fe3d9800640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b806c8f0 0x7fe3b806edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe3d415f3e0 tx=0x7fe3c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.749+0000 7fe3d1600640 1 -- 172.21.15.59:0/424649454 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3c8045360 con 0x7fe3d4160960 2024-04-23T14:40:24.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.991+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3d406e260 con 0x7fe3d4160960 2024-04-23T14:40:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.993+0000 7fe3d1600640 1 -- 172.21.15.59:0/424649454 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5182 (secure 0 0 0) 0x7fe3c807aed0 con 0x7fe3d4160960 2024-04-23T14:40:24.993 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:24.993 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:39:16.472663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":14,"addr":"172.21.15.59:6837/2128549219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2128549219},{"type":"v1","addr":"172.21.15.59:6837","nonce":2128549219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7533,"qdb_cluster":[7533]},"id":6}]} 2024-04-23T14:40:24.993 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 67 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.995+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b806c8f0 msgr2=0x7fe3b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.995+0000 7fe3dc200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b806c8f0 0x7fe3b806edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe3d415f3e0 tx=0x7fe3c4008040 comp rx=0 tx=0).stop 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.995+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4160960 msgr2=0x7fe3d4160f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.995+0000 7fe3dc200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4160960 0x7fe3d4160f10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009f10 tx=0x7fe3c8004a40 comp rx=0 tx=0).stop 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 shutdown_connections 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b806c8f0 0x7fe3b806edb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3d41644b0 0x7fe3d4161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4160960 0x7fe3d4160f10 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 --2- 172.21.15.59:0/424649454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d40b4550 0x7fe3d40aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 >> 172.21.15.59:0/424649454 conn(0x7fe3d4067ea0 msgr2=0x7fe3d40b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 shutdown_connections 2024-04-23T14:40:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:24.996+0000 7fe3dc200640 1 -- 172.21.15.59:0/424649454 wait complete. 2024-04-23T14:40:25.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-23T14:40:25.257 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:40:25.258 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:40:25.258 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:40:25.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.467+0000 7f0899c00640 1 Processor -- start 2024-04-23T14:40:25.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.467+0000 7f0899c00640 1 -- start start 2024-04-23T14:40:25.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.469+0000 7f0899c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08940d0c10 0x7f08940cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.469+0000 7f0899c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f08940af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.469+0000 7f0899c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08941676a0 con 0x7f08940d0830 2024-04-23T14:40:25.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.469+0000 7f0899c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0894166b20 con 0x7f08940d0c10 2024-04-23T14:40:25.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.469+0000 7f0899c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0894166e00 con 0x7f08940af010 2024-04-23T14:40:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.470+0000 7f0899200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f08940af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.470+0000 7f088be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08940d0c10 0x7f08940cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.470+0000 7f088be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08940d0c10 0x7f08940cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:33096/0 (socket says 172.21.15.59:33096) 2024-04-23T14:40:25.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.470+0000 7f088be00640 1 -- 172.21.15.59:0/2453871184 learned_addr learned my addr 172.21.15.59:0/2453871184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.471+0000 7f0899200640 1 -- 172.21.15.59:0/2453871184 >> v2:172.21.15.76:3300/0 conn(0x7f08940d0c10 msgr2=0x7f08940cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.471+0000 7f0899200640 1 --2- 172.21.15.59:0/2453871184 >> v2:172.21.15.76:3300/0 conn(0x7f08940d0c10 0x7f08940cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.471+0000 7f0899200640 1 -- 172.21.15.59:0/2453871184 >> v1:172.21.15.76:6789/0 conn(0x7f08940d0830 legacy=0x7f0894163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.471+0000 7f0899200640 1 -- 172.21.15.59:0/2453871184 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08941670e0 con 0x7f08940af010 2024-04-23T14:40:25.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.471+0000 7f088be00640 1 --2- 172.21.15.59:0/2453871184 >> v2:172.21.15.76:3300/0 conn(0x7f08940d0c10 0x7f08940cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:40:25.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.472+0000 7f0899200640 1 --2- 172.21.15.59:0/2453871184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f08940af450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f088c009d90 tx=0x7f088c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b007d18de280fecd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.473+0000 7f0893400640 1 -- 172.21.15.59:0/2453871184 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c031600 con 0x7f08940af010 2024-04-23T14:40:25.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.473+0000 7f0893400640 1 -- 172.21.15.59:0/2453871184 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f088c0317a0 con 0x7f08940af010 2024-04-23T14:40:25.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.473+0000 7f0893400640 1 -- 172.21.15.59:0/2453871184 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c031b70 con 0x7f08940af010 2024-04-23T14:40:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.473+0000 7f0899c00640 1 -- 172.21.15.59:0/2453871184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 msgr2=0x7f08940af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.473+0000 7f0899c00640 1 --2- 172.21.15.59:0/2453871184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f08940af450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f088c009d90 tx=0x7f088c02a5d0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.474+0000 7f0899c00640 1 -- 172.21.15.59:0/2453871184 shutdown_connections 2024-04-23T14:40:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.474+0000 7f0899c00640 1 --2- 172.21.15.59:0/2453871184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f08940af450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.474+0000 7f0899c00640 1 --2- 172.21.15.59:0/2453871184 >> v2:172.21.15.76:3300/0 conn(0x7f08940d0c10 0x7f08940cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.474+0000 7f0899c00640 1 -- 172.21.15.59:0/2453871184 >> 172.21.15.59:0/2453871184 conn(0x7f089406bdf0 msgr2=0x7f089406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:25.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.474+0000 7f0899c00640 1 -- 172.21.15.59:0/2453871184 shutdown_connections 2024-04-23T14:40:25.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.474+0000 7f0899c00640 1 -- 172.21.15.59:0/2453871184 wait complete. 2024-04-23T14:40:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.475+0000 7f0899c00640 1 Processor -- start 2024-04-23T14:40:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.476+0000 7f0899c00640 1 -- start start 2024-04-23T14:40:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.476+0000 7f0899c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f0894155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.476+0000 7f0899c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940d0830 0x7f08941f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f0899c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08940d0c10 0x7f08940b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f0899c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08940b45f0 con 0x7f08940d0830 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f0899c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08940af990 con 0x7f08940d0c10 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f0899c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0894166740 con 0x7f08940af010 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940d0830 0x7f08941f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940d0830 0x7f08941f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51878/0 (socket says 172.21.15.59:51878) 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 -- 172.21.15.59:0/2743083586 learned_addr learned my addr 172.21.15.59:0/2743083586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f0893e00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f0894155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f0899200640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08940d0c10 0x7f08940b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 -- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 msgr2=0x7f0894155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f0894155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 -- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08940d0c10 msgr2=0x7f08940b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08940d0c10 0x7f08940b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f088be00640 1 -- 172.21.15.59:0/2743083586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089409c910 con 0x7f08940d0830 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.477+0000 7f0899200640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08940d0c10 0x7f08940b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.478+0000 7f088be00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940d0830 0x7f08941f9200 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f087c00e9e0 tx=0x7f087c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.478+0000 7f0890c00640 1 -- 172.21.15.59:0/2743083586 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c017070 con 0x7f08940d0830 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.478+0000 7f0890c00640 1 -- 172.21.15.59:0/2743083586 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087c00ca30 con 0x7f08940d0830 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.478+0000 7f0890c00640 1 -- 172.21.15.59:0/2743083586 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c00ce20 con 0x7f08940d0830 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.478+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08941fc180 con 0x7f08940d0830 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.478+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08941fc740 con 0x7f08940d0830 2024-04-23T14:40:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.479+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089404fd90 con 0x7f08940d0830 2024-04-23T14:40:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.480+0000 7f0890c00640 1 -- 172.21.15.59:0/2743083586 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f087c01a070 con 0x7f08940d0830 2024-04-23T14:40:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.482+0000 7f0890c00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f087006c940 0x7f087006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.482+0000 7f0890c00640 1 -- 172.21.15.59:0/2743083586 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5824+0+0 (secure 0 0 0) 0x7f087c08ba40 con 0x7f08940d0830 2024-04-23T14:40:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.482+0000 7f0893e00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f087006c940 0x7f087006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.483+0000 7f0893e00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f087006c940 0x7f087006ee00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0884001640 tx=0x7f0884009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.487+0000 7f0890c00640 1 -- 172.21.15.59:0/2743083586 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f089409c910 con 0x7f08940d0830 2024-04-23T14:40:25.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.737+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f08940cedb0 con 0x7f08940d0830 2024-04-23T14:40:25.737 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:40:25.737+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:40:25.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.741+0000 7f0890c00640 1 -- 172.21.15.59:0/2743083586 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f087c059f00 con 0x7f08940d0830 2024-04-23T14:40:25.741 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:40:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f087006c940 msgr2=0x7f087006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f087006c940 0x7f087006ee00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0884001640 tx=0x7f0884009040 comp rx=0 tx=0).stop 2024-04-23T14:40:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940d0830 msgr2=0x7f08941f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940d0830 0x7f08941f9200 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f087c00e9e0 tx=0x7f087c00eea0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 shutdown_connections 2024-04-23T14:40:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f087006c940 0x7f087006ee00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08940d0c10 0x7f08940b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940d0830 0x7f08941f9200 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 --2- 172.21.15.59:0/2743083586 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08940af010 0x7f0894155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.744+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 >> 172.21.15.59:0/2743083586 conn(0x7f089406bdf0 msgr2=0x7f08940ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.745+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 shutdown_connections 2024-04-23T14:40:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.745+0000 7f0899c00640 1 -- 172.21.15.59:0/2743083586 wait complete. 2024-04-23T14:40:25.764 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-23T14:40:25.777 INFO:tasks.ceph.mds.c.smithi059.stderr: -5044> 2024-04-23T14:39:05.690+0000 7fedc1800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:40:25.780 INFO:tasks.ceph.mds.c.smithi059.stderr: -4572> 2024-04-23T14:39:09.755+0000 7fedbc800640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-23T14:40:25.782 INFO:tasks.ceph.mds.c.smithi059.stderr: -4406> 2024-04-23T14:39:10.029+0000 7fedbc800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0) n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55a610a96100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-23T14:40:25.790 INFO:tasks.ceph.mds.c.smithi059.stderr: -2611> 2024-04-23T14:39:42.168+0000 7fedbc800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0)->f(v1 m2024-04-23T14:38:48.802771+0000) n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1)->n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55a610a96100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-23T14:38:48.802771+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-23T14:38:48.943131+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-23T14:40:25.844 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:40:25.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.851+0000 7f7b5c010740 1 -- start start 2024-04-23T14:40:25.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.851+0000 7f7b5c010740 1 Processor -- start 2024-04-23T14:40:25.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563102fc8c00 0x563103d34000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563102fc9800 0x563103d34580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563102fc9c00 0x563103d34b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b5c010740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563103d38600 con 0x563102fc9800 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b5c010740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563103d38480 con 0x563102fc9c00 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b5c010740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563103d38300 con 0x563102fc8c00 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b58c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563102fc9800 0x563103d34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.852+0000 7f7b58c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563102fc9800 0x563103d34580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51894/0 (socket says 172.21.15.59:51894) 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b58c00640 1 -- 172.21.15.59:0/3913082209 learned_addr learned my addr 172.21.15.59:0/3913082209 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b5a000640 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563102fc9c00 0x563103d34b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b59600640 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563102fc8c00 0x563103d34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b58c00640 1 -- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563102fc8c00 msgr2=0x563103d34000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b58c00640 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563102fc8c00 0x563103d34000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b58c00640 1 -- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563102fc9c00 msgr2=0x563103d34b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b58c00640 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563102fc9c00 0x563103d34b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b58c00640 1 -- 172.21.15.59:0/3913082209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563102ff5680 con 0x563102fc9800 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b59600640 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563102fc8c00 0x563103d34000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.853+0000 7f7b58c00640 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563102fc9800 0x563103d34580 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x563103de5c50 tx=0x563103d6c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4a3207590f0ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b56400640 1 -- 172.21.15.59:0/3913082209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563102fee680 con 0x563102fc9800 2024-04-23T14:40:25.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b56400640 1 -- 172.21.15.59:0/3913082209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563103012c40 con 0x563102fc9800 2024-04-23T14:40:25.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b56400640 1 -- 172.21.15.59:0/3913082209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563102fee9c0 con 0x563102fc9800 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b5c010740 1 -- 172.21.15.59:0/3913082209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563102ff5a40 con 0x563102fc9800 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b5c010740 1 -- 172.21.15.59:0/3913082209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563102ff52c0 con 0x563102fc9800 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b5c010740 4 waiting for MDS map... 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b56400640 1 -- 172.21.15.59:0/3913082209 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x563103e9ea00 con 0x563102fc9800 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b56400640 1 -- 172.21.15.59:0/3913082209 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x563103021180 con 0x563102fc9800 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b5c010740 4 Got MDS map 68 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b5c010740 4 datascan.main: Using metadata pool output 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.854+0000 7f7b5c010740 4 datascan.main: connecting to RADOS... 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.855+0000 7f7b5c010740 1 Processor -- start 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.855+0000 7f7b5c010740 1 -- start start 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91800 0x563103d35600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91c00 0x563103d35b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b5c010740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563103d38180 con 0x563103e91800 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b5c010740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563103028480 con 0x563103e91400 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b5c010740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563103028900 con 0x563103e91c00 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b59600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91800 0x563103d35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b59600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91800 0x563103d35600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51908/0 (socket says 172.21.15.59:51908) 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.856+0000 7f7b59600640 1 -- 172.21.15.59:0/4285172219 learned_addr learned my addr 172.21.15.59:0/4285172219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b5a000640 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b59600640 1 -- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91c00 msgr2=0x563103d35b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b59600640 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91c00 0x563103d35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b59600640 1 -- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 msgr2=0x563103d35080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b59600640 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b59600640 1 -- 172.21.15.59:0/4285172219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563102ff54a0 con 0x563103e91800 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b5a000640 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b59600640 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91800 0x563103d35600 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x563103de5200 tx=0x563103d68900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a87b2ad1baef32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b53c00640 1 -- 172.21.15.59:0/4285172219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563102fee680 con 0x563103e91800 2024-04-23T14:40:25.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b53c00640 1 -- 172.21.15.59:0/4285172219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563103013340 con 0x563103e91800 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.857+0000 7f7b53c00640 1 -- 172.21.15.59:0/4285172219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563102fee9c0 con 0x563103e91800 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 -- 172.21.15.59:0/4285172219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91800 msgr2=0x563103d35600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91800 0x563103d35600 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x563103de5200 tx=0x563103d68900 comp rx=0 tx=0).stop 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 -- 172.21.15.59:0/4285172219 shutdown_connections 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91c00 0x563103d35b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91800 0x563103d35600 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 --2- 172.21.15.59:0/4285172219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 -- 172.21.15.59:0/4285172219 >> 172.21.15.59:0/4285172219 conn(0x563103e91000 msgr2=0x563103094800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 -- 172.21.15.59:0/4285172219 shutdown_connections 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.858+0000 7f7b5c010740 1 -- 172.21.15.59:0/4285172219 wait complete. 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.859+0000 7f7b5c010740 1 Processor -- start 2024-04-23T14:40:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.859+0000 7f7b5c010740 1 -- start start 2024-04-23T14:40:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91800 0x563103d35600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b5c010740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91c00 0x563103d35b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b5c010740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563103d39200 con 0x563103e91c00 2024-04-23T14:40:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b5c010740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563103d38180 con 0x563103e91400 2024-04-23T14:40:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b5c010740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563103d39380 con 0x563103e91800 2024-04-23T14:40:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b58c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91c00 0x563103d35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b58c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91c00 0x563103d35b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51918/0 (socket says 172.21.15.59:51918) 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 learned_addr learned my addr 172.21.15.59:0/2381592300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b59600640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91800 0x563103d35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.860+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91800 msgr2=0x563103d35600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b58c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563103e91800 0x563103d35600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 msgr2=0x563103d35080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b58c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563102ff5860 con 0x563103e91c00 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563103e91400 0x563103d35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b58c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91c00 0x563103d35b80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x563103db3e90 tx=0x56310307cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b51e00640 1 -- 172.21.15.59:0/2381592300 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563102fee680 con 0x563103e91c00 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b51e00640 1 -- 172.21.15.59:0/2381592300 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563103012a80 con 0x563103e91c00 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b51e00640 1 -- 172.21.15.59:0/2381592300 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563102fee9c0 con 0x563103e91c00 2024-04-23T14:40:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563102ff5680 con 0x563103e91c00 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563102ff5a40 con 0x563103e91c00 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.861+0000 7f7b5c010740 4 datascan.init: resolving metadata pool 12 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.863+0000 7f7b51e00640 1 -- 172.21.15.59:0/2381592300 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x563103dc6c00 con 0x563103e91c00 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b51e00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x563103f0d000 0x563103d36100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b51e00640 1 -- 172.21.15.59:0/2381592300 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x563103dc4c00 con 0x563103e91c00 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b5c010740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b5c010740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b5c010740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x563103f0d400 0x563103d36680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103f0d800 con 0x563103f0d400 2024-04-23T14:40:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.864+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x563103f0d000 0x563103d36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.865+0000 7f7b59600640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x563103f0d400 0x563103d36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.865+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x563103f0d000 0x563103d36100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x563103d6af60 tx=0x563103eb30e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.866+0000 7f7b59600640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x563103f0d400 0x563103d36680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.866+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(1 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.867+0000 7f7b53200640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x563103d67400 0x563103354000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.867+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103d67000 con 0x563103d67400 2024-04-23T14:40:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.867+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x563103d67400 0x563103354000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.868+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x563103d67400 0x563103354000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.874+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(2 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.874+0000 7f7b53c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563103d66800 0x563103354580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.874+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032ccc00 con 0x563103d66800 2024-04-23T14:40:25.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.875+0000 7f7b59600640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563103d66800 0x563103354580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.875+0000 7f7b59600640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563103d66800 0x563103354580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.878+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(3 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.878+0000 7f7b53200640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x563103d66c00 0x563103354b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.878+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cd000 con 0x563103d66c00 2024-04-23T14:40:25.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.879+0000 7f7b58c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x563103d66c00 0x563103354b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.880+0000 7f7b58c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x563103d66c00 0x563103354b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.881+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(4 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.881+0000 7f7b53c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x563103f0d800 0x563103355600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.881+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cdc00 con 0x563103f0d800 2024-04-23T14:40:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.881+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x563103f0d800 0x563103355600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.882+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x563103f0d800 0x563103355600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.885+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(5 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.885+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cd400 con 0x563103f0d800 2024-04-23T14:40:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.885+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== osd_op_reply(6 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.885+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cc800 con 0x563103f0d400 2024-04-23T14:40:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.886+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== osd_op_reply(7 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.886+0000 7f7b53200640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5631032cd800 0x563103355080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.886+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e000 con 0x5631032cd800 2024-04-23T14:40:25.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.887+0000 7f7b59600640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5631032cd800 0x563103355080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.888+0000 7f7b59600640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5631032cd800 0x563103355080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.894+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(8 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x5631032cd800 2024-04-23T14:40:25.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.894+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cdc00 con 0x563103d67400 2024-04-23T14:40:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.895+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== osd_op_reply(9 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.895+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e400 con 0x563103d67400 2024-04-23T14:40:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.896+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 3 ==== osd_op_reply(10 [pgnls start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.896+0000 7f7b53c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5631032cd400 0x563103355b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.896+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103d67000 con 0x5631032cd400 2024-04-23T14:40:25.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.896+0000 7f7b58c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5631032cd400 0x563103355b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.897+0000 7f7b58c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5631032cd400 0x563103355b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.900+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(11 [pgnls start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x5631032cd400 2024-04-23T14:40:25.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.900+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e800 con 0x563103d67400 2024-04-23T14:40:25.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.901+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 4 ==== osd_op_reply(12 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.901+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f000 con 0x563103d66800 2024-04-23T14:40:25.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.901+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== osd_op_reply(13 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.902+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336ec00 con 0x563103f0d800 2024-04-23T14:40:25.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.902+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 3 ==== osd_op_reply(14 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.902+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f400 con 0x563103d67400 2024-04-23T14:40:25.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.903+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 5 ==== osd_op_reply(15 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.903+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e400 con 0x563103f0d800 2024-04-23T14:40:25.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.904+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 4 ==== osd_op_reply(16 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.904+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f800 con 0x5631032cd800 2024-04-23T14:40:25.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.905+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(17 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x5631032cd800 2024-04-23T14:40:25.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.905+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e800 con 0x563103d66800 2024-04-23T14:40:25.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.905+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 3 ==== osd_op_reply(18 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.905+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336fc00 con 0x5631032cd400 2024-04-23T14:40:25.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.906+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== osd_op_reply(19 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x5631032cd400 2024-04-23T14:40:25.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.906+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336ec00 con 0x563103f0d800 2024-04-23T14:40:25.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.906+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 5 ==== osd_op_reply(20 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.906+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e000 con 0x563103f0d800 2024-04-23T14:40:25.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.906+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 6 ==== osd_op_reply(21 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.907+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f400 con 0x563103d66800 2024-04-23T14:40:25.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.907+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 4 ==== osd_op_reply(22 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.907+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f000 con 0x563103d66800 2024-04-23T14:40:25.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.907+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 5 ==== osd_op_reply(23 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.908+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cdc00 con 0x5631032cd800 2024-04-23T14:40:25.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.909+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.5 v2:172.21.15.76:6808/2782996536 3 ==== osd_op_reply(24 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x5631032cd800 2024-04-23T14:40:25.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.909+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f800 con 0x563103d66c00 2024-04-23T14:40:25.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.910+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(25 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.910+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336fc00 con 0x563103f0d400 2024-04-23T14:40:25.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.911+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 3 ==== osd_op_reply(26 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.911+0000 7f7b53c00640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x56310336e800 0x563103356100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:25.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.911+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032ccc00 con 0x56310336e800 2024-04-23T14:40:25.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.912+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x56310336e800 0x563103356100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:25.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.912+0000 7f7b5a000640 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x56310336e800 0x563103356100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:25.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.913+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(27 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x56310336e800 2024-04-23T14:40:25.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.913+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f800 con 0x563103d67400 2024-04-23T14:40:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.914+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 6 ==== osd_op_reply(28 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.914+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cc800 con 0x563103f0d400 2024-04-23T14:40:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.915+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 4 ==== osd_op_reply(29 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.915+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103d67000 con 0x563103f0d800 2024-04-23T14:40:25.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.916+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 7 ==== osd_op_reply(30 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.916+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103d67000 con 0x563103d66c00 2024-04-23T14:40:25.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.917+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 3 ==== osd_op_reply(31 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.917+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cd000 con 0x563103d66800 2024-04-23T14:40:25.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.917+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 6 ==== osd_op_reply(32 [pgnls start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.917+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e400 con 0x563103d67400 2024-04-23T14:40:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.918+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 7 ==== osd_op_reply(33 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.919+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103f0cc00 con 0x563103d66c00 2024-04-23T14:40:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.920+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 4 ==== osd_op_reply(34 [pgnls start_epoch 87 out=49b] v83'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.920+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336ec00 con 0x563103d66800 2024-04-23T14:40:25.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.920+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 7 ==== osd_op_reply(35 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.921+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103f0c800 con 0x56310336e800 2024-04-23T14:40:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.921+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== osd_op_reply(36 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x56310336e800 2024-04-23T14:40:25.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.921+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336e000 con 0x5631032cd800 2024-04-23T14:40:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.922+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.5 v2:172.21.15.76:6808/2782996536 4 ==== osd_op_reply(37 [pgnls start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x5631032cd800 2024-04-23T14:40:25.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.923+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f400 con 0x563103d67400 2024-04-23T14:40:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.924+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 8 ==== osd_op_reply(38 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.924+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f800 con 0x56310336e800 2024-04-23T14:40:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.924+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.3 v2:172.21.15.59:6800/3205049728 3 ==== osd_op_reply(39 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x56310336e800 2024-04-23T14:40:25.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.924+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336f000 con 0x563103f0d800 2024-04-23T14:40:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.925+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 8 ==== osd_op_reply(40 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.925+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032ccc00 con 0x563103d66c00 2024-04-23T14:40:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.926+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 5 ==== osd_op_reply(41 [pgnls start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.926+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336fc00 con 0x563103d66c00 2024-04-23T14:40:25.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.927+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 6 ==== osd_op_reply(42 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.928+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103388000 con 0x563103d66c00 2024-04-23T14:40:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.929+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 7 ==== osd_op_reply(43 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.929+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336ec00 con 0x563103d66800 2024-04-23T14:40:25.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.930+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 8 ==== osd_op_reply(44 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.930+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103388400 con 0x563103d66c00 2024-04-23T14:40:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.931+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 8 ==== osd_op_reply(45 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.931+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cdc00 con 0x5631032cd400 2024-04-23T14:40:25.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.931+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.0 v2:172.21.15.59:6817/2024703144 3 ==== osd_op_reply(46 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x5631032cd400 2024-04-23T14:40:25.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.932+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103388800 con 0x563103f0d400 2024-04-23T14:40:25.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.933+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 5 ==== osd_op_reply(47 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.933+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cc800 con 0x563103f0d400 2024-04-23T14:40:25.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.934+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 6 ==== osd_op_reply(48 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.934+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103388c00 con 0x5631032cd400 2024-04-23T14:40:25.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.934+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.0 v2:172.21.15.59:6817/2024703144 4 ==== osd_op_reply(49 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x5631032cd400 2024-04-23T14:40:25.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.934+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cd000 con 0x563103d66800 2024-04-23T14:40:25.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.935+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 9 ==== osd_op_reply(50 [pgnls start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.935+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103389000 con 0x563103f0d400 2024-04-23T14:40:25.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.936+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 7 ==== osd_op_reply(51 [pgnls start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.936+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032ccc00 con 0x563103f0d400 2024-04-23T14:40:25.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.937+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 8 ==== osd_op_reply(52 [pgnls start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.938+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103389400 con 0x5631032cd800 2024-04-23T14:40:25.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.939+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.5 v2:172.21.15.76:6808/2782996536 5 ==== osd_op_reply(53 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x5631032cd800 2024-04-23T14:40:25.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.939+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cdc00 con 0x563103f0d800 2024-04-23T14:40:25.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.940+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.2 v2:172.21.15.59:6816/77995666 9 ==== osd_op_reply(54 [pgnls start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103f0d800 2024-04-23T14:40:25.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.940+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103389800 con 0x563103d66c00 2024-04-23T14:40:25.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.941+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 9 ==== osd_op_reply(55 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.941+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103388000 con 0x563103d66c00 2024-04-23T14:40:25.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.942+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 10 ==== osd_op_reply(56 [pgnls start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.942+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103389c00 con 0x563103f0d400 2024-04-23T14:40:25.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.943+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 9 ==== osd_op_reply(57 [pgnls start_epoch 87 out=81b] v83'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.944+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x563103f0dc00 con 0x563103f0d400 2024-04-23T14:40:25.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.945+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.945+0000 7f7b5c010740 10 datascan.operator(): handling object 1099511627777.0 2024-04-23T14:40:25.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.945+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x563103388800 con 0x563103f0d400 2024-04-23T14:40:25.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.950+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.7 v2:172.21.15.76:6822/3281262326 11 ==== osd_op_reply(59 10000000001.00000000 [call] v87'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x563103d56000 con 0x563103f0d400 2024-04-23T14:40:25.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.950+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5631032cc800 con 0x563103d66800 2024-04-23T14:40:25.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.950+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.1 v2:172.21.15.59:6808/4218092099 10 ==== osd_op_reply(60 [pgnls start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x563103d66800 2024-04-23T14:40:25.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.951+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103389000 con 0x5631032cd800 2024-04-23T14:40:25.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.952+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.5 v2:172.21.15.76:6808/2782996536 6 ==== osd_op_reply(61 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x5631032cd800 2024-04-23T14:40:25.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.952+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103388400 con 0x5631032cd800 2024-04-23T14:40:25.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.953+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 <== osd.5 v2:172.21.15.76:6808/2782996536 7 ==== osd_op_reply(62 [pgnls start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56000 con 0x5631032cd800 2024-04-23T14:40:25.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.953+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103389400 con 0x563103d67400 2024-04-23T14:40:25.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.954+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 9 ==== osd_op_reply(63 [pgnls start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.954+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x56310336fc00 con 0x563103d66c00 2024-04-23T14:40:25.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.955+0000 7f7b58c00640 1 -- 172.21.15.59:0/2381592300 <== osd.4 v2:172.21.15.76:6816/3407674963 11 ==== osd_op_reply(64 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56780 con 0x563103d66c00 2024-04-23T14:40:25.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.956+0000 7f7b53c00640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103389400 con 0x563103d67400 2024-04-23T14:40:25.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.957+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 10 ==== osd_op_reply(65 [pgnls start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.957+0000 7f7b53200640 1 -- 172.21.15.59:0/2381592300 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x563103f0cc00 con 0x563103d67400 2024-04-23T14:40:25.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5a000640 1 -- 172.21.15.59:0/2381592300 <== osd.6 v2:172.21.15.76:6800/1890860772 11 ==== osd_op_reply(66 [pgnls start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563103d56500 con 0x563103d67400 2024-04-23T14:40:25.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5631032cd400 msgr2=0x563103355b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5631032cd400 0x563103355b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563103d66800 msgr2=0x563103354580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x563103d66800 0x563103354580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x563103f0d800 msgr2=0x563103355600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x563103f0d800 0x563103355600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x56310336e800 msgr2=0x563103356100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x56310336e800 0x563103356100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x563103d66c00 msgr2=0x563103354b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x563103d66c00 0x563103354b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5631032cd800 msgr2=0x563103355080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5631032cd800 0x563103355080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x563103d67400 msgr2=0x563103354000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 reap_dead start 2024-04-23T14:40:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x563103d67400 0x563103354000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b59600640 1 -- 172.21.15.59:0/2381592300 reap_dead start 2024-04-23T14:40:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x563103f0d400 msgr2=0x563103d36680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.958+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x563103f0d400 0x563103d36680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x563103f0d000 msgr2=0x563103d36100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x563103f0d000 0x563103d36100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x563103d6af60 tx=0x563103eb30e0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91c00 msgr2=0x563103d35b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91c00 0x563103d35b80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x563103db3e90 tx=0x56310307cf00 comp rx=0 tx=0).stop 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 shutdown_connections 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x563103d67400 0x563103354000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x563103f0d000 0x563103d36100 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x563103f0d400 0x563103d36680 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 --2- 172.21.15.59:0/2381592300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563103e91c00 0x563103d35b80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.959+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 >> 172.21.15.59:0/2381592300 conn(0x563103e91000 msgr2=0x563103094800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.960+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 shutdown_connections 2024-04-23T14:40:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.960+0000 7f7b5c010740 1 -- 172.21.15.59:0/2381592300 wait complete. 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 -- 172.21.15.59:0/3913082209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563102fc9800 msgr2=0x563103d34580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563102fc9800 0x563103d34580 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x563103de5c50 tx=0x563103d6c120 comp rx=0 tx=0).stop 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 -- 172.21.15.59:0/3913082209 shutdown_connections 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563102fc9c00 0x563103d34b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563102fc9800 0x563103d34580 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 --2- 172.21.15.59:0/3913082209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563102fc8c00 0x563103d34000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 -- 172.21.15.59:0/3913082209 >> 172.21.15.59:0/3913082209 conn(0x563102fc9000 msgr2=0x563103095000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 -- 172.21.15.59:0/3913082209 shutdown_connections 2024-04-23T14:40:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:25.961+0000 7f7b5c010740 1 -- 172.21.15.59:0/3913082209 wait complete. 2024-04-23T14:40:25.973 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.208936, result: 2024-04-23T14:40:25.973 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-23T14:40:26.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.124+0000 7fa371a15740 1 -- start start 2024-04-23T14:40:26.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.124+0000 7fa371a15740 1 Processor -- start 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa371a15740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a4732400 0x5642a4768b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa371a15740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733000 0x5642a4769080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa371a15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733400 0x5642a4768580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa371a15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5642a485e000 con 0x5642a4733400 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa371a15740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5642a485e180 con 0x5642a4733000 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa371a15740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5642a485e300 con 0x5642a4732400 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa36fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733400 0x5642a4768580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa36fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733400 0x5642a4768580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51934/0 (socket says 172.21.15.59:51934) 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.125+0000 7fa36fa00640 1 -- 172.21.15.59:0/2993808938 learned_addr learned my addr 172.21.15.59:0/2993808938 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36e600640 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733000 0x5642a4769080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36f000640 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a4732400 0x5642a4768b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36fa00640 1 -- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a4732400 msgr2=0x5642a4768b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a4732400 0x5642a4768b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36fa00640 1 -- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733000 msgr2=0x5642a4769080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733000 0x5642a4769080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36fa00640 1 -- 172.21.15.59:0/2993808938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a39c1680 con 0x5642a4733400 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36f000640 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a4732400 0x5642a4768b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.126+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733400 0x5642a4768580 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x5642a47bbaa0 tx=0x5642a47380c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad28e553fb4ab72 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa36be00640 1 -- 172.21.15.59:0/2993808938 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642a39ba9c0 con 0x5642a4733400 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa36be00640 1 -- 172.21.15.59:0/2993808938 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5642a39de540 con 0x5642a4733400 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa36be00640 1 -- 172.21.15.59:0/2993808938 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642a39bab60 con 0x5642a4733400 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa371a15740 1 -- 172.21.15.59:0/2993808938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a39c12c0 con 0x5642a4733400 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa371a15740 1 -- 172.21.15.59:0/2993808938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5642a39c1a40 con 0x5642a4733400 2024-04-23T14:40:26.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa371a15740 4 waiting for MDS map... 2024-04-23T14:40:26.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa36be00640 1 -- 172.21.15.59:0/2993808938 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x5642a4872600 con 0x5642a4733400 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa36be00640 1 -- 172.21.15.59:0/2993808938 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x5642a39ed180 con 0x5642a4733400 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa371a15740 4 Got MDS map 68 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa371a15740 4 datascan.main: Using metadata pool output 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.127+0000 7fa371a15740 4 datascan.main: connecting to RADOS... 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.128+0000 7fa371a15740 1 Processor -- start 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.128+0000 7fa371a15740 1 -- start start 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa371a15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733c00 0x5642a4769600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa371a15740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a48d6000 0x5642a4769b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa371a15740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48d6400 0x5642a476a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa371a15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5642a485e480 con 0x5642a4733c00 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa371a15740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5642a485e600 con 0x5642a48d6000 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa371a15740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5642a485e780 con 0x5642a48d6400 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa36e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733c00 0x5642a4769600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa36e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733c00 0x5642a4769600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51944/0 (socket says 172.21.15.59:51944) 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.129+0000 7fa36e600640 1 -- 172.21.15.59:0/3552428272 learned_addr learned my addr 172.21.15.59:0/3552428272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa36f000640 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a48d6000 0x5642a4769b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa36e600640 1 -- 172.21.15.59:0/3552428272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48d6400 msgr2=0x5642a476a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa36e600640 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48d6400 0x5642a476a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa36e600640 1 -- 172.21.15.59:0/3552428272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a48d6000 msgr2=0x5642a4769b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa36e600640 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a48d6000 0x5642a4769b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa36e600640 1 -- 172.21.15.59:0/3552428272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a39c1c20 con 0x5642a4733c00 2024-04-23T14:40:26.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa36e600640 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733c00 0x5642a4769600 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x5642a488ef00 tx=0x5642a47348a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cababf8a2a186a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa369600640 1 -- 172.21.15.59:0/3552428272 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642a39ba680 con 0x5642a4733c00 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa369600640 1 -- 172.21.15.59:0/3552428272 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5642a39dec40 con 0x5642a4733c00 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.130+0000 7fa369600640 1 -- 172.21.15.59:0/3552428272 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642a39ba9c0 con 0x5642a4733c00 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 -- 172.21.15.59:0/3552428272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733c00 msgr2=0x5642a4769600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733c00 0x5642a4769600 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x5642a488ef00 tx=0x5642a47348a0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 -- 172.21.15.59:0/3552428272 shutdown_connections 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48d6400 0x5642a476a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a48d6000 0x5642a4769b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 --2- 172.21.15.59:0/3552428272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733c00 0x5642a4769600 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 -- 172.21.15.59:0/3552428272 >> 172.21.15.59:0/3552428272 conn(0x5642a4733800 msgr2=0x5642a3a63000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:26.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 -- 172.21.15.59:0/3552428272 shutdown_connections 2024-04-23T14:40:26.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.131+0000 7fa371a15740 1 -- 172.21.15.59:0/3552428272 wait complete. 2024-04-23T14:40:26.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.132+0000 7fa371a15740 1 Processor -- start 2024-04-23T14:40:26.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.132+0000 7fa371a15740 1 -- start start 2024-04-23T14:40:26.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa371a15740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733800 0x5642a4769600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa371a15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a48ec000 0x5642a4769b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa371a15740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48ec400 0x5642a476a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa371a15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5642a485fc80 con 0x5642a48ec000 2024-04-23T14:40:26.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa371a15740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5642a485fb00 con 0x5642a4733800 2024-04-23T14:40:26.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa371a15740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5642a485f980 con 0x5642a48ec400 2024-04-23T14:40:26.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a48ec000 0x5642a4769b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a48ec000 0x5642a4769b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51958/0 (socket says 172.21.15.59:51958) 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 learned_addr learned my addr 172.21.15.59:0/2104586267 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733800 0x5642a4769600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48ec400 0x5642a476a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48ec400 msgr2=0x5642a476a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a48ec400 0x5642a476a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733800 msgr2=0x5642a4769600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733800 0x5642a4769600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.133+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a39c0b40 con 0x5642a48ec000 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733800 0x5642a4769600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa36f000640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a48ec000 0x5642a4769b80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x5642a488fbf0 tx=0x5642a4890c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa367800640 1 -- 172.21.15.59:0/2104586267 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642a39ba680 con 0x5642a48ec000 2024-04-23T14:40:26.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa367800640 1 -- 172.21.15.59:0/2104586267 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5642a39df340 con 0x5642a48ec000 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5642a39c03c0 con 0x5642a48ec000 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa367800640 1 -- 172.21.15.59:0/2104586267 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5642a39ba9c0 con 0x5642a48ec000 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a39c14a0 con 0x5642a48ec000 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.134+0000 7fa371a15740 4 datascan.init: resolving metadata pool 12 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.136+0000 7fa367800640 1 -- 172.21.15.59:0/2104586267 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5642a48d6c00 con 0x5642a48ec000 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa367800640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5642a48eec00 0x5642a476a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa367800640 1 -- 172.21.15.59:0/2104586267 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x5642a48dca00 con 0x5642a48ec000 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa371a15740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa371a15740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa371a15740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-23T14:40:26.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5642a48ec800 0x5642a476ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ecc00 con 0x5642a48ec800 2024-04-23T14:40:26.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.137+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5642a48eec00 0x5642a476a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.138+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5642a48eec00 0x5642a476a680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x5642a488f110 tx=0x5642a4735aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.138+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5642a48ec800 0x5642a476ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.139+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5642a48ec800 0x5642a476ac00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.139+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.140+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5642a48ed000 0x5642a476b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.140+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ed400 con 0x5642a48ed000 2024-04-23T14:40:26.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.140+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5642a48ed000 0x5642a476b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.141+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5642a48ed000 0x5642a476b180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.142+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.143+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5642a48ed800 0x5642a476b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.143+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48edc00 con 0x5642a48ed800 2024-04-23T14:40:26.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.143+0000 7fa36f000640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5642a48ed800 0x5642a476b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.144+0000 7fa36f000640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5642a48ed800 0x5642a476b700 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.145+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.145+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a48ed000 2024-04-23T14:40:26.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.146+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.147+0000 7fa369600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5642a48ef400 0x5642a3d3e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.147+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef800 con 0x5642a48ef400 2024-04-23T14:40:26.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.147+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5642a48ef400 0x5642a3d3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.148+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5642a48ef400 0x5642a3d3e000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.148+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.148+0000 7fa368c00640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5642a3d44000 0x5642a3d3e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.148+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44400 con 0x5642a3d44000 2024-04-23T14:40:26.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.149+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5642a3d44000 0x5642a3d3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.150+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5642a3d44000 0x5642a3d3e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.151+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.151+0000 7fa369600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5642a48efc00 0x5642a3d3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.151+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871800 con 0x5642a48efc00 2024-04-23T14:40:26.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.151+0000 7fa36f000640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5642a48efc00 0x5642a3d3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.152+0000 7fa36f000640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5642a48efc00 0x5642a3d3eb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.152+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.152+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44400 con 0x5642a48efc00 2024-04-23T14:40:26.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.152+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.152+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871400 con 0x5642a48ed800 2024-04-23T14:40:26.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.153+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.153+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a48ec800 2024-04-23T14:40:26.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.154+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.154+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871000 con 0x5642a48ed000 2024-04-23T14:40:26.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.155+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.155+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ed400 con 0x5642a48ed000 2024-04-23T14:40:26.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.156+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.156+0000 7fa369600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5642a48ef800 0x5642a3d3f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.157+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a48ef800 2024-04-23T14:40:26.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.157+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5642a48ef800 0x5642a3d3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.158+0000 7fa36e600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5642a48ef800 0x5642a3d3f080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.158+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.158+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871800 con 0x5642a48ed000 2024-04-23T14:40:26.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.159+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.159+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732c00 con 0x5642a48ef400 2024-04-23T14:40:26.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.160+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.160+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871400 con 0x5642a48efc00 2024-04-23T14:40:26.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.160+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.161+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732000 con 0x5642a48ed000 2024-04-23T14:40:26.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.161+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.161+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ecc00 con 0x5642a48efc00 2024-04-23T14:40:26.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.162+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.162+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44400 con 0x5642a48ec800 2024-04-23T14:40:26.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.163+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.163+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48edc00 con 0x5642a48ef400 2024-04-23T14:40:26.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.164+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.164+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44800 con 0x5642a48ef800 2024-04-23T14:40:26.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.164+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.164+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45800 con 0x5642a48efc00 2024-04-23T14:40:26.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.165+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.165+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44c00 con 0x5642a48efc00 2024-04-23T14:40:26.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.166+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.166+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45c00 con 0x5642a48ef400 2024-04-23T14:40:26.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.166+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.166+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45000 con 0x5642a48ef400 2024-04-23T14:40:26.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.167+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.167+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44400 con 0x5642a48ec800 2024-04-23T14:40:26.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.168+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.168+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45400 con 0x5642a3d44000 2024-04-23T14:40:26.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.169+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.169+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44800 con 0x5642a48ed800 2024-04-23T14:40:26.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.171+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.171+0000 7fa369600640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5642a3d45800 0x5642a3d3f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.171+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44c00 con 0x5642a3d45800 2024-04-23T14:40:26.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.171+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5642a3d45800 0x5642a3d3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.172+0000 7fa36fa00640 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5642a3d45800 0x5642a3d3f600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.173+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d45800 2024-04-23T14:40:26.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.173+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45c00 con 0x5642a48ed000 2024-04-23T14:40:26.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.174+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.174+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44800 con 0x5642a48ed800 2024-04-23T14:40:26.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.175+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.175+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48edc00 con 0x5642a48efc00 2024-04-23T14:40:26.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.176+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.176+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ecc00 con 0x5642a3d44000 2024-04-23T14:40:26.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.177+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.177+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732c00 con 0x5642a48ef400 2024-04-23T14:40:26.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.178+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.178+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871400 con 0x5642a48ed000 2024-04-23T14:40:26.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.179+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.179+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a3d44000 2024-04-23T14:40:26.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.180+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 87 out=49b] v83'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.180+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45400 con 0x5642a48ef400 2024-04-23T14:40:26.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.181+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.181+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64000 con 0x5642a3d45800 2024-04-23T14:40:26.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.182+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d45800 2024-04-23T14:40:26.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.182+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44c00 con 0x5642a48ec800 2024-04-23T14:40:26.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.183+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.183+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64400 con 0x5642a48ed000 2024-04-23T14:40:26.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.184+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.184+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45c00 con 0x5642a3d45800 2024-04-23T14:40:26.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.185+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.3 v2:172.21.15.59:6800/3205049728 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d45800 2024-04-23T14:40:26.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.185+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64800 con 0x5642a48efc00 2024-04-23T14:40:26.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.186+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.186+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732000 con 0x5642a3d44000 2024-04-23T14:40:26.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.187+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.187+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64c00 con 0x5642a3d44000 2024-04-23T14:40:26.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.188+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.189+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871000 con 0x5642a3d44000 2024-04-23T14:40:26.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.190+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.190+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65000 con 0x5642a48ef400 2024-04-23T14:40:26.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.191+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.191+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871800 con 0x5642a3d44000 2024-04-23T14:40:26.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.192+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.192+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65400 con 0x5642a48ef800 2024-04-23T14:40:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.193+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.193+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ed400 con 0x5642a48ed800 2024-04-23T14:40:26.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.194+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.194+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65800 con 0x5642a48ed800 2024-04-23T14:40:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.195+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.195+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45000 con 0x5642a48ef800 2024-04-23T14:40:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.196+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.196+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65000 con 0x5642a48ef400 2024-04-23T14:40:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.197+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.197+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44400 con 0x5642a48ed800 2024-04-23T14:40:26.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.199+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.199+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65400 con 0x5642a48ed800 2024-04-23T14:40:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.200+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.200+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45400 con 0x5642a48ec800 2024-04-23T14:40:26.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.201+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.201+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64000 con 0x5642a48efc00 2024-04-23T14:40:26.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.202+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.202+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44c00 con 0x5642a3d44000 2024-04-23T14:40:26.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.204+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.204+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64400 con 0x5642a3d44000 2024-04-23T14:40:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.205+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.205+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 87 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45c00 con 0x5642a48ed800 2024-04-23T14:40:26.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.206+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 87 out=81b] v87'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.206+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44800 con 0x5642a48ed800 2024-04-23T14:40:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.208+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.208+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a48ed000 2024-04-23T14:40:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.209+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.209+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64c00 con 0x5642a48ef400 2024-04-23T14:40:26.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.210+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.210+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871400 con 0x5642a3d44000 2024-04-23T14:40:26.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.212+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.212+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65c00 con 0x5642a48efc00 2024-04-23T14:40:26.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.213+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.213+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ecc00 con 0x5642a48efc00 2024-04-23T14:40:26.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.214+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.214+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732c00 con 0x5642a48ed800 2024-04-23T14:40:26.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.215+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.215+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732c00 con 0x5642a48ec800 2024-04-23T14:40:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.216+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.216+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732000 con 0x5642a48ed000 2024-04-23T14:40:26.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.217+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.218+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ecc00 con 0x5642a48ed000 2024-04-23T14:40:26.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.219+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.219+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68000 con 0x5642a48ef800 2024-04-23T14:40:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.220+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.220+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65400 con 0x5642a48ed000 2024-04-23T14:40:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.221+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.221+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68400 con 0x5642a48ef400 2024-04-23T14:40:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.222+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.222+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64000 con 0x5642a48efc00 2024-04-23T14:40:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.224+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.224+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68800 con 0x5642a48ed000 2024-04-23T14:40:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.225+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.225+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65c00 con 0x5642a48efc00 2024-04-23T14:40:26.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.226+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.226+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68c00 con 0x5642a48ec800 2024-04-23T14:40:26.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.227+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.227+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45c00 con 0x5642a48ef400 2024-04-23T14:40:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.228+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.228+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69000 con 0x5642a48ef800 2024-04-23T14:40:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.229+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.229+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44800 con 0x5642a48efc00 2024-04-23T14:40:26.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.230+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.231+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69400 con 0x5642a48efc00 2024-04-23T14:40:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.232+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.232+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871000 con 0x5642a48ef400 2024-04-23T14:40:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.233+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.233+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69800 con 0x5642a48ef400 2024-04-23T14:40:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.233+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 14 ==== osd_op_reply(82 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.234+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871800 con 0x5642a48ec800 2024-04-23T14:40:26.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.235+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.235+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69c00 con 0x5642a3d44000 2024-04-23T14:40:26.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.236+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.236+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871400 con 0x5642a48ed800 2024-04-23T14:40:26.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.237+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.237+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65000 con 0x5642a3d45800 2024-04-23T14:40:26.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.238+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.3 v2:172.21.15.59:6800/3205049728 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d45800 2024-04-23T14:40:26.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.239+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a48ed000 2024-04-23T14:40:26.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.240+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.240+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64c00 con 0x5642a48ed800 2024-04-23T14:40:26.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.241+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.241+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45000 con 0x5642a48efc00 2024-04-23T14:40:26.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.242+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.242+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64400 con 0x5642a3d44000 2024-04-23T14:40:26.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.243+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.244+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45400 con 0x5642a48ef400 2024-04-23T14:40:26.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.244+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 15 ==== osd_op_reply(91 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.245+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65400 con 0x5642a48ed000 2024-04-23T14:40:26.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.246+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.246+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44c00 con 0x5642a3d44000 2024-04-23T14:40:26.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.247+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 87 out=49b] v83'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.247+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ecc00 con 0x5642a48ef400 2024-04-23T14:40:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.248+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 16 ==== osd_op_reply(94 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.248+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68000 con 0x5642a3d45800 2024-04-23T14:40:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.249+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.3 v2:172.21.15.59:6800/3205049728 5 ==== osd_op_reply(95 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d45800 2024-04-23T14:40:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.249+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732c00 con 0x5642a48ec800 2024-04-23T14:40:26.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.250+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.250+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68400 con 0x5642a48ed000 2024-04-23T14:40:26.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.251+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.252+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732000 con 0x5642a3d45800 2024-04-23T14:40:26.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.252+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.3 v2:172.21.15.59:6800/3205049728 6 ==== osd_op_reply(98 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d45800 2024-04-23T14:40:26.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.253+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68c00 con 0x5642a48efc00 2024-04-23T14:40:26.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.254+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.254+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44800 con 0x5642a3d44000 2024-04-23T14:40:26.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.255+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 87 out=49b] v83'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.255+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732000 con 0x5642a3d44000 2024-04-23T14:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.256+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.256+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d45000 con 0x5642a3d44000 2024-04-23T14:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.257+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.258+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69400 con 0x5642a48ef400 2024-04-23T14:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.259+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 17 ==== osd_op_reply(103 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.259+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d44c00 con 0x5642a3d44000 2024-04-23T14:40:26.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.260+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.260+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68c00 con 0x5642a48ef800 2024-04-23T14:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.261+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.261+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64000 con 0x5642a48ed800 2024-04-23T14:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.262+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.262+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68800 con 0x5642a48ed800 2024-04-23T14:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.263+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.263+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65c00 con 0x5642a48ef800 2024-04-23T14:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.264+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.264+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69c00 con 0x5642a48ef400 2024-04-23T14:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.265+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 18 ==== osd_op_reply(109 [pgnls-filter start_epoch 87 out=49b] v83'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.266+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64c00 con 0x5642a48ed800 2024-04-23T14:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.267+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 87 out=49b] v83'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.267+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68000 con 0x5642a48ed800 2024-04-23T14:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.268+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 87 out=49b] v83'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.268+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65000 con 0x5642a48ec800 2024-04-23T14:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.269+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.269+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d68400 con 0x5642a48efc00 2024-04-23T14:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.270+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.2 v2:172.21.15.59:6816/77995666 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722780 con 0x5642a48efc00 2024-04-23T14:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.271+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64400 con 0x5642a3d44000 2024-04-23T14:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.272+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.272+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69000 con 0x5642a3d44000 2024-04-23T14:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.273+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 87 out=49b] v83'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.273+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65400 con 0x5642a48ed800 2024-04-23T14:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.275+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 87 out=81b] v87'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.275+0000 7fa371a15740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-23T14:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.275+0000 7fa371a15740 10 datascan.operator(): handling object 10000000001.0 2024-04-23T14:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.275+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d64800 con 0x5642a48ed800 2024-04-23T14:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.277+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 <== osd.7 v2:172.21.15.76:6822/3281262326 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x5642a4722780 con 0x5642a48ed800 2024-04-23T14:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.277+0000 7fa371a15740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-23T14:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.277+0000 7fa371a15740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-23T14:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.277+0000 7fa371a15740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-23T14:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.277+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d65800 con 0x5642a48ef800 2024-04-23T14:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.278+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.278+0000 7fa371a15740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-23T14:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.278+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5642a48edc00 con 0x5642a48ed000 2024-04-23T14:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.279+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.280+0000 7fa371a15740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-23T14:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.280+0000 7fa371a15740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-23T14:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.280+0000 7fa371a15740 20 datascan.get_frag_of: frag is * 2024-04-23T14:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.280+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ed400 con 0x5642a48ef800 2024-04-23T14:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.281+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.281+0000 7fa371a15740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-23T14:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.281+0000 7fa371a15740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-23T14:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.281+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d7c000 con 0x5642a48ef800 2024-04-23T14:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.282+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.282+0000 7fa371a15740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-23T14:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.282+0000 7fa371a15740 20 datascan.read_dentry: file_head not found in result 2024-04-23T14:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.282+0000 7fa371a15740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-23T14:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.282+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d7c400 con 0x5642a48ef800 2024-04-23T14:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.286+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.0 v2:172.21.15.59:6817/2024703144 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v87'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef800 2024-04-23T14:40:26.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.286+0000 7fa371a15740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-23T14:40:26.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.286+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d7c800 con 0x5642a48ef400 2024-04-23T14:40:26.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.287+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.1 v2:172.21.15.59:6808/4218092099 19 ==== osd_op_reply(123 [pgnls-filter start_epoch 87 out=49b] v83'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ef400 2024-04-23T14:40:26.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.288+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a48ec800 2024-04-23T14:40:26.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.289+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.289+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69800 con 0x5642a48ec800 2024-04-23T14:40:26.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.290+0000 7fa36e600640 1 -- 172.21.15.59:0/2104586267 <== osd.5 v2:172.21.15.76:6808/2782996536 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 87 out=49b] v83'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722000 con 0x5642a48ec800 2024-04-23T14:40:26.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.290+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4871400 con 0x5642a48ed000 2024-04-23T14:40:26.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.291+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 87 out=49b] v83'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.291+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a3d69400 con 0x5642a3d44000 2024-04-23T14:40:26.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.292+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.4 v2:172.21.15.76:6816/3407674963 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a3d44000 2024-04-23T14:40:26.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.292+0000 7fa368c00640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a48ef000 con 0x5642a48ed000 2024-04-23T14:40:26.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.294+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 87 out=49b] v83'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.294+0000 7fa369600640 1 -- 172.21.15.59:0/2104586267 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 87 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5642a4732c00 con 0x5642a48ed000 2024-04-23T14:40:26.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa36fa00640 1 -- 172.21.15.59:0/2104586267 <== osd.6 v2:172.21.15.76:6800/1890860772 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 87 out=49b] v83'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5642a4722500 con 0x5642a48ed000 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5642a48ef800 msgr2=0x5642a3d3f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5642a48ef800 0x5642a3d3f080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5642a48ef400 msgr2=0x5642a3d3e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5642a48ef400 0x5642a3d3e000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5642a48efc00 msgr2=0x5642a3d3eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5642a48efc00 0x5642a3d3eb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5642a3d45800 msgr2=0x5642a3d3f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5642a3d45800 0x5642a3d3f600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 reap_dead start 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5642a3d44000 msgr2=0x5642a3d3e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5642a3d44000 0x5642a3d3e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5642a48ec800 msgr2=0x5642a476ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5642a48ec800 0x5642a476ac00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5642a48ed000 msgr2=0x5642a476b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5642a48ed000 0x5642a476b180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5642a48ed800 msgr2=0x5642a476b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5642a48ed800 0x5642a476b700 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.295+0000 7fa36f000640 1 -- 172.21.15.59:0/2104586267 reap_dead start 2024-04-23T14:40:26.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5642a48eec00 msgr2=0x5642a476a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5642a48eec00 0x5642a476a680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x5642a488f110 tx=0x5642a4735aa0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a48ec000 msgr2=0x5642a4769b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a48ec000 0x5642a4769b80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x5642a488fbf0 tx=0x5642a4890c00 comp rx=0 tx=0).stop 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 shutdown_connections 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5642a48eec00 0x5642a476a680 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 --2- 172.21.15.59:0/2104586267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a48ec000 0x5642a4769b80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.296+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 >> 172.21.15.59:0/2104586267 conn(0x5642a4733c00 msgr2=0x5642a3a63000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:26.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.297+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 shutdown_connections 2024-04-23T14:40:26.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.297+0000 7fa371a15740 1 -- 172.21.15.59:0/2104586267 wait complete. 2024-04-23T14:40:26.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.298+0000 7fa371a15740 1 -- 172.21.15.59:0/2993808938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733400 msgr2=0x5642a4768580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.298+0000 7fa371a15740 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733400 0x5642a4768580 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x5642a47bbaa0 tx=0x5642a47380c0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.298+0000 7fa371a15740 1 -- 172.21.15.59:0/2993808938 shutdown_connections 2024-04-23T14:40:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.298+0000 7fa371a15740 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642a4733400 0x5642a4768580 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.298+0000 7fa371a15740 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5642a4733000 0x5642a4769080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.298+0000 7fa371a15740 1 --2- 172.21.15.59:0/2993808938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5642a4732400 0x5642a4768b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.298+0000 7fa371a15740 1 -- 172.21.15.59:0/2993808938 >> 172.21.15.59:0/2993808938 conn(0x5642a3995000 msgr2=0x5642a3a62000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.299+0000 7fa371a15740 1 -- 172.21.15.59:0/2993808938 shutdown_connections 2024-04-23T14:40:26.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.299+0000 7fa371a15740 1 -- 172.21.15.59:0/2993808938 wait complete. 2024-04-23T14:40:26.313 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.339352, result: 2024-04-23T14:40:26.313 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-23T14:40:26.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.426+0000 7f008142f740 1 -- start start 2024-04-23T14:40:26.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.426+0000 7f008142f740 1 Processor -- start 2024-04-23T14:40:26.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f008142f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b63bec00 0x5650b712a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f008142f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b63bf800 0x5650b712a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f008142f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b63bfc00 0x5650b712ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f008142f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650b712e600 con 0x5650b63bfc00 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f008142f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5650b712e480 con 0x5650b63bf800 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f008142f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5650b712e300 con 0x5650b63bec00 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f007f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b63bfc00 0x5650b712ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f007f000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b63bfc00 0x5650b712ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51968/0 (socket says 172.21.15.59:51968) 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.427+0000 7f007f000640 1 -- 172.21.15.59:0/2847929552 learned_addr learned my addr 172.21.15.59:0/2847929552 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007dc00640 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b63bf800 0x5650b712a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007e600640 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b63bec00 0x5650b712a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007f000640 1 -- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b63bec00 msgr2=0x5650b712a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007f000640 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b63bec00 0x5650b712a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007f000640 1 -- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b63bf800 msgr2=0x5650b712a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007f000640 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b63bf800 0x5650b712a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007f000640 1 -- 172.21.15.59:0/2847929552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b63eb680 con 0x5650b63bfc00 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007e600640 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b63bec00 0x5650b712a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:40:26.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007dc00640 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b63bf800 0x5650b712a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:40:26.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.428+0000 7f007f000640 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b63bfc00 0x5650b712ab00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x5650b71d7d40 tx=0x5650b7162120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34822aa14971b058 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f007b400640 1 -- 172.21.15.59:0/2847929552 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5650b63e4680 con 0x5650b63bfc00 2024-04-23T14:40:26.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f007b400640 1 -- 172.21.15.59:0/2847929552 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5650b64081c0 con 0x5650b63bfc00 2024-04-23T14:40:26.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f007b400640 1 -- 172.21.15.59:0/2847929552 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5650b63e49c0 con 0x5650b63bfc00 2024-04-23T14:40:26.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f008142f740 1 -- 172.21.15.59:0/2847929552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b63eba40 con 0x5650b63bfc00 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f008142f740 1 -- 172.21.15.59:0/2847929552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5650b63eb2c0 con 0x5650b63bfc00 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f008142f740 4 waiting for MDS map... 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f007b400640 1 -- 172.21.15.59:0/2847929552 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x5650b728e600 con 0x5650b63bfc00 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f007b400640 1 -- 172.21.15.59:0/2847929552 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x5650b6417180 con 0x5650b63bfc00 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.429+0000 7f008142f740 4 Got MDS map 68 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.430+0000 7f008142f740 4 datascan.main: Using metadata pool output 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.430+0000 7f008142f740 4 datascan.main: connecting to RADOS... 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.430+0000 7f008142f740 1 Processor -- start 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.431+0000 7f008142f740 1 -- start start 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.431+0000 7f008142f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.431+0000 7f008142f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291800 0x5650b712b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.431+0000 7f008142f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b7291c00 0x5650b712bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.431+0000 7f008142f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650b712e180 con 0x5650b7291400 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.431+0000 7f008142f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5650b641e480 con 0x5650b7291c00 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.431+0000 7f008142f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5650b641e900 con 0x5650b7291800 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51980/0 (socket says 172.21.15.59:51980) 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 -- 172.21.15.59:0/893477950 learned_addr learned my addr 172.21.15.59:0/893477950 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007e600640 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291800 0x5650b712b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 -- 172.21.15.59:0/893477950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291800 msgr2=0x5650b712b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291800 0x5650b712b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 -- 172.21.15.59:0/893477950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b7291c00 msgr2=0x5650b712bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b7291c00 0x5650b712bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.432+0000 7f007dc00640 1 -- 172.21.15.59:0/893477950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b63eb4a0 con 0x5650b7291400 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.433+0000 7f007dc00640 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x5650b7289020 tx=0x5650b729cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ffe2e6cf14d573 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.433+0000 7f0078c00640 1 -- 172.21.15.59:0/893477950 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5650b63e4680 con 0x5650b7291400 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.433+0000 7f0078c00640 1 -- 172.21.15.59:0/893477950 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5650b6408c40 con 0x5650b7291400 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.433+0000 7f0078c00640 1 -- 172.21.15.59:0/893477950 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5650b63e49c0 con 0x5650b7291400 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.433+0000 7f008142f740 1 -- 172.21.15.59:0/893477950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 msgr2=0x5650b712b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.433+0000 7f008142f740 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x5650b7289020 tx=0x5650b729cc00 comp rx=0 tx=0).stop 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.433+0000 7f008142f740 1 -- 172.21.15.59:0/893477950 shutdown_connections 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.434+0000 7f008142f740 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291800 0x5650b712b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.434+0000 7f008142f740 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b7291c00 0x5650b712bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.434+0000 7f008142f740 1 --2- 172.21.15.59:0/893477950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.434+0000 7f008142f740 1 -- 172.21.15.59:0/893477950 >> 172.21.15.59:0/893477950 conn(0x5650b7291000 msgr2=0x5650b648a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.434+0000 7f008142f740 1 -- 172.21.15.59:0/893477950 shutdown_connections 2024-04-23T14:40:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.434+0000 7f008142f740 1 -- 172.21.15.59:0/893477950 wait complete. 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.434+0000 7f008142f740 1 Processor -- start 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.435+0000 7f008142f740 1 -- start start 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.435+0000 7f008142f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.435+0000 7f008142f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b7291800 0x5650b712b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.435+0000 7f008142f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291c00 0x5650b712bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.435+0000 7f008142f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650b712f380 con 0x5650b7291400 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.435+0000 7f008142f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5650b712e180 con 0x5650b7291800 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.435+0000 7f008142f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5650b712f500 con 0x5650b7291c00 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51986/0 (socket says 172.21.15.59:51986) 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 learned_addr learned my addr 172.21.15.59:0/36704417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291c00 msgr2=0x5650b712bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b7291c00 0x5650b712bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b7291800 msgr2=0x5650b712b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b7291800 0x5650b712b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b63eb680 con 0x5650b7291400 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.436+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x5650b7289ce0 tx=0x5650b715e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.437+0000 7f0076e00640 1 -- 172.21.15.59:0/36704417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5650b63e4680 con 0x5650b7291400 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.437+0000 7f0076e00640 1 -- 172.21.15.59:0/36704417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5650b6408a80 con 0x5650b7291400 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.437+0000 7f0076e00640 1 -- 172.21.15.59:0/36704417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5650b63e49c0 con 0x5650b7291400 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.437+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5650b63eba40 con 0x5650b7291400 2024-04-23T14:40:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.437+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b63eb0e0 con 0x5650b7291400 2024-04-23T14:40:26.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.437+0000 7f008142f740 4 datascan.init: resolving metadata pool 12 2024-04-23T14:40:26.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.439+0000 7f0076e00640 1 -- 172.21.15.59:0/36704417 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5650b7304c00 con 0x5650b7291400 2024-04-23T14:40:26.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.439+0000 7f0076e00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5650b72fac00 0x5650b712c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.440+0000 7f0076e00640 1 -- 172.21.15.59:0/36704417 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x5650b72fc600 con 0x5650b7291400 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.440+0000 7f008142f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.440+0000 7f008142f740 4 datascan.main: resolving metadata pool 12 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.440+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5650b72fb000 0x5650b712c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.440+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb400 con 0x5650b72fb000 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.440+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5650b72fac00 0x5650b712c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.440+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5650b72fac00 0x5650b712c100 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5650b71601e0 tx=0x5650b715e900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.441+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5650b72fb000 0x5650b712c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.441+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5650b72fb000 0x5650b712c680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.442+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(1 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.442+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5650b72fb400 0x5650b6742000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.442+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66ce000 con 0x5650b72fb400 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.443+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5650b72fb400 0x5650b6742000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.443+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5650b72fb400 0x5650b6742000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.444+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(2 [pgnls start_epoch 87 out=177b] v87'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.444+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5650b66ce000 0x5650b6742580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.444+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b7290800 con 0x5650b66ce000 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.444+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5650b66ce000 0x5650b6742580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.445+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5650b66ce000 0x5650b6742580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.445+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(3 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce000 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.445+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5650b7290800 0x5650b6742b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.445+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d000 con 0x5650b7290800 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.445+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5650b7290800 0x5650b6742b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.446+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5650b7290800 0x5650b6742b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.446+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(4 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.446+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5650b715d000 0x5650b6743080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.446+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66ce400 con 0x5650b715d000 2024-04-23T14:40:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.447+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5650b715d000 0x5650b6743080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.448+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5650b715d000 0x5650b6743080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.449+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(5 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.449+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5650b66ce400 0x5650b6743b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.449+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cec00 con 0x5650b66ce400 2024-04-23T14:40:26.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.450+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5650b66ce400 0x5650b6743b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.451+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5650b66ce400 0x5650b6743b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.452+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(6 [pgnls start_epoch 87 out=149b] v86'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce400 2024-04-23T14:40:26.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.452+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cec00 con 0x5650b72fb400 2024-04-23T14:40:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.452+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== osd_op_reply(7 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.452+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cec00 con 0x5650b72fb400 2024-04-23T14:40:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.453+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 3 ==== osd_op_reply(8 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.453+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cec00 con 0x5650b72fb000 2024-04-23T14:40:26.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.454+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(9 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.454+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cec00 con 0x5650b66ce000 2024-04-23T14:40:26.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.454+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== osd_op_reply(10 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce000 2024-04-23T14:40:26.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.454+0000 7f007e600640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5650b66cec00 0x5650b6744680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.455+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b66cec00 2024-04-23T14:40:26.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.455+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5650b66cec00 0x5650b6744680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.456+0000 7f007dc00640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5650b66cec00 0x5650b6744680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.456+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(11 [pgnls start_epoch 87 out=81b] v86'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5650b714c000 con 0x5650b66cec00 2024-04-23T14:40:26.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.456+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b72fb000 2024-04-23T14:40:26.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.457+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 3 ==== osd_op_reply(12 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.457+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b7290800 2024-04-23T14:40:26.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.458+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== osd_op_reply(13 [pgnls start_epoch 87 out=77b] v86'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.458+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b715d000 2024-04-23T14:40:26.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.459+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== osd_op_reply(14 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.459+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b72fb400 2024-04-23T14:40:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.459+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 4 ==== osd_op_reply(15 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.459+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b72fb000 2024-04-23T14:40:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.460+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 4 ==== osd_op_reply(16 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.460+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b715d000 2024-04-23T14:40:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.461+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 3 ==== osd_op_reply(17 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.461+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b66ce400 2024-04-23T14:40:26.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.462+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(18 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce400 2024-04-23T14:40:26.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.462+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b66cec00 2024-04-23T14:40:26.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.463+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== osd_op_reply(19 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b66cec00 2024-04-23T14:40:26.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.463+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b72fb400 2024-04-23T14:40:26.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.463+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 5 ==== osd_op_reply(20 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.463+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b715d000 2024-04-23T14:40:26.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.464+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 4 ==== osd_op_reply(21 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.464+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b715d400 con 0x5650b715d000 2024-04-23T14:40:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.465+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 5 ==== osd_op_reply(22 [pgnls start_epoch 87 out=97b] v86'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.465+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5650b715d400 0x5650b6744100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.465+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b715d400 2024-04-23T14:40:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.466+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5650b715d400 0x5650b6744100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.467+0000 7f007f000640 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5650b715d400 0x5650b6744100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.468+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(23 [pgnls start_epoch 87 out=74b] v82'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d400 2024-04-23T14:40:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.468+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b72fb400 2024-04-23T14:40:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.469+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 6 ==== osd_op_reply(24 [pgnls start_epoch 87 out=73b] v86'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.469+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b72fb400 2024-04-23T14:40:26.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.469+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 7 ==== osd_op_reply(25 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.469+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b7290800 2024-04-23T14:40:26.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.470+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 3 ==== osd_op_reply(26 [pgnls start_epoch 87 out=98b] v86'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.470+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b66ce000 2024-04-23T14:40:26.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.470+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.3 v2:172.21.15.59:6800/3205049728 3 ==== osd_op_reply(27 [pgnls start_epoch 87 out=76b] v86'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce000 2024-04-23T14:40:26.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.470+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b72fb000 2024-04-23T14:40:26.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.472+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 5 ==== osd_op_reply(28 [pgnls start_epoch 87 out=73b] v86'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.472+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b7290800 2024-04-23T14:40:26.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.473+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 4 ==== osd_op_reply(29 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.473+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b715d000 2024-04-23T14:40:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.474+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 6 ==== osd_op_reply(30 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.474+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b715d000 2024-04-23T14:40:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.475+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 7 ==== osd_op_reply(31 [pgnls start_epoch 87 out=73b] v82'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.475+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b715d000 2024-04-23T14:40:26.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.476+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 8 ==== osd_op_reply(32 [pgnls start_epoch 87 out=101b] v86'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.476+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-23T14:40:26.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.476+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.477+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.477+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-23T14:40:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.477+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fbc00 con 0x5650b72fb400 2024-04-23T14:40:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.477+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.478+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-23T14:40:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.478+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cfc00 con 0x5650b72fb400 2024-04-23T14:40:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.478+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.478+0000 7f008142f740 20 datascan.scan_links: injected ino detected 2024-04-23T14:40:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.479+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-23T14:40:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.479+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf800 con 0x5650b72fb400 2024-04-23T14:40:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.479+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.479+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-23T14:40:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.479+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf400 con 0x5650b72fb400 2024-04-23T14:40:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-23T14:40:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-23T14:40:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-23T14:40:26.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-23T14:40:26.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-23T14:40:26.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-23T14:40:26.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-23T14:40:26.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.480+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cf000 con 0x5650b66ce400 2024-04-23T14:40:26.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.482+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.5 v2:172.21.15.76:6808/2782996536 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce400 2024-04-23T14:40:26.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.482+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-23T14:40:26.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.482+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cfc00 con 0x5650b72fb400 2024-04-23T14:40:26.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.483+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.483+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-23T14:40:26.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.483+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66cec00 2024-04-23T14:40:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.483+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.2 v2:172.21.15.59:6816/77995666 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x5650b714c000 con 0x5650b66cec00 2024-04-23T14:40:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.484+0000 7f008142f740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-23T14:40:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.484+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-23T14:40:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.484+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-23T14:40:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.484+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fbc00 con 0x5650b72fb400 2024-04-23T14:40:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.484+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.485+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-23T14:40:26.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.485+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e000 con 0x5650b715d000 2024-04-23T14:40:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.486+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.486+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-23T14:40:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.486+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-23T14:40:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.486+0000 7f008142f740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-23T14:40:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.486+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-23T14:40:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.486+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-23T14:40:26.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.486+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e400 con 0x5650b72fb400 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.487+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.487+0000 7f008142f740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.487+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.487+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.487+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e800 con 0x5650b7290800 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.488+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.488+0000 7f008142f740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.488+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-23T14:40:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.488+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.488+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.488+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.488+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676ec00 con 0x5650b7290800 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.489+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.489+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.489+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.489+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-23T14:40:26.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.489+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-23T14:40:26.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.489+0000 7f008142f740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-23T14:40:26.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.489+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f000 con 0x5650b715d000 2024-04-23T14:40:26.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.490+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.490+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f400 con 0x5650b72fb000 2024-04-23T14:40:26.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.492+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 6 ==== osd_op_reply(47 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.492+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.493+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 16 ==== osd_op_reply(48 [pgnls start_epoch 87 out=177b] v87'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.493+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66ce000 2024-04-23T14:40:26.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.493+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.3 v2:172.21.15.59:6800/3205049728 4 ==== osd_op_reply(49 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce000 2024-04-23T14:40:26.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.493+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b7290800 2024-04-23T14:40:26.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.494+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 7 ==== osd_op_reply(50 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.494+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.496+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 11 ==== osd_op_reply(51 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.496+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66ce400 2024-04-23T14:40:26.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.497+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.5 v2:172.21.15.76:6808/2782996536 4 ==== osd_op_reply(52 [pgnls start_epoch 87 out=149b] v86'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce400 2024-04-23T14:40:26.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.497+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.498+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 17 ==== osd_op_reply(53 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.498+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.499+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 18 ==== osd_op_reply(54 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.499+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb000 2024-04-23T14:40:26.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.500+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 7 ==== osd_op_reply(55 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.500+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66ce000 2024-04-23T14:40:26.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.501+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.3 v2:172.21.15.59:6800/3205049728 5 ==== osd_op_reply(56 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce000 2024-04-23T14:40:26.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.501+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66cec00 2024-04-23T14:40:26.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.502+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.2 v2:172.21.15.59:6816/77995666 4 ==== osd_op_reply(57 [pgnls start_epoch 87 out=81b] v86'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5650b714c000 con 0x5650b66cec00 2024-04-23T14:40:26.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.502+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb000 2024-04-23T14:40:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.503+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 8 ==== osd_op_reply(58 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.503+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b7290800 2024-04-23T14:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.504+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 8 ==== osd_op_reply(59 [pgnls start_epoch 87 out=77b] v86'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.504+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.506+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 12 ==== osd_op_reply(60 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.506+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.507+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 19 ==== osd_op_reply(61 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.507+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb000 2024-04-23T14:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.508+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 9 ==== osd_op_reply(62 [pgnls start_epoch 87 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.508+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.509+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 13 ==== osd_op_reply(63 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.509+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66ce400 2024-04-23T14:40:26.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.510+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.5 v2:172.21.15.76:6808/2782996536 5 ==== osd_op_reply(64 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce400 2024-04-23T14:40:26.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.510+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66cec00 2024-04-23T14:40:26.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.511+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.2 v2:172.21.15.59:6816/77995666 5 ==== osd_op_reply(65 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c000 con 0x5650b66cec00 2024-04-23T14:40:26.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.511+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.512+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 20 ==== osd_op_reply(66 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.512+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.513+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 14 ==== osd_op_reply(67 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.513+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.514+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 15 ==== osd_op_reply(68 [pgnls start_epoch 87 out=97b] v86'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.514+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d400 2024-04-23T14:40:26.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.515+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== osd_op_reply(69 [pgnls start_epoch 87 out=74b] v82'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d400 2024-04-23T14:40:26.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.515+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.516+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 21 ==== osd_op_reply(70 [pgnls start_epoch 87 out=73b] v86'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.516+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb400 2024-04-23T14:40:26.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.517+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 22 ==== osd_op_reply(71 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.517+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b7290800 2024-04-23T14:40:26.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.518+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 9 ==== osd_op_reply(72 [pgnls start_epoch 87 out=98b] v86'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.518+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b66ce000 2024-04-23T14:40:26.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.519+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.3 v2:172.21.15.59:6800/3205049728 6 ==== osd_op_reply(73 [pgnls start_epoch 87 out=76b] v86'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce000 2024-04-23T14:40:26.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.519+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b72fb000 2024-04-23T14:40:26.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.520+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.4 v2:172.21.15.76:6816/3407674963 10 ==== osd_op_reply(74 [pgnls start_epoch 87 out=73b] v86'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c000 con 0x5650b72fb000 2024-04-23T14:40:26.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.520+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b7290800 2024-04-23T14:40:26.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.521+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 10 ==== osd_op_reply(75 [pgnls start_epoch 87 out=73b] v86'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.521+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.523+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 16 ==== osd_op_reply(76 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.523+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.524+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 17 ==== osd_op_reply(77 [pgnls start_epoch 87 out=73b] v82'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.524+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.525+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 18 ==== osd_op_reply(78 [pgnls start_epoch 87 out=101b] v86'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.526+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-23T14:40:26.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.526+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f800 con 0x5650b72fb400 2024-04-23T14:40:26.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.527+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.527+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-23T14:40:26.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.527+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676fc00 con 0x5650b72fb400 2024-04-23T14:40:26.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.528+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.528+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-23T14:40:26.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.528+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e800 con 0x5650b72fb400 2024-04-23T14:40:26.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.529+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.529+0000 7f008142f740 20 datascan.scan_links: injected ino detected 2024-04-23T14:40:26.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.529+0000 7f008142f740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-23T14:40:26.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.529+0000 7f008142f740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-23T14:40:26.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.529+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-23T14:40:26.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.529+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676ec00 con 0x5650b72fb400 2024-04-23T14:40:26.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.530+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.530+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-23T14:40:26.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.530+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f400 con 0x5650b72fb400 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-23T14:40:26.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.531+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e000 con 0x5650b66ce400 2024-04-23T14:40:26.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.532+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 <== osd.5 v2:172.21.15.76:6808/2782996536 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c780 con 0x5650b66ce400 2024-04-23T14:40:26.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.532+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-23T14:40:26.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.532+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e400 con 0x5650b72fb400 2024-04-23T14:40:26.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.533+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.534+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-23T14:40:26.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.534+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f000 con 0x5650b66cec00 2024-04-23T14:40:26.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.535+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.2 v2:172.21.15.59:6816/77995666 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x5650b714c000 con 0x5650b66cec00 2024-04-23T14:40:26.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.535+0000 7f008142f740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-23T14:40:26.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.535+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-23T14:40:26.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.535+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-23T14:40:26.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.535+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fbc00 con 0x5650b72fb400 2024-04-23T14:40:26.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.536+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.536+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-23T14:40:26.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.536+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fb800 con 0x5650b715d000 2024-04-23T14:40:26.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.537+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.537+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-23T14:40:26.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.537+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-23T14:40:26.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.537+0000 7f008142f740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-23T14:40:26.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.537+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-23T14:40:26.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.537+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-23T14:40:26.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.537+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b72fbc00 con 0x5650b72fb400 2024-04-23T14:40:26.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.538+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.539+0000 7f008142f740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-23T14:40:26.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.539+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-23T14:40:26.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.539+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-23T14:40:26.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.539+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f800 con 0x5650b7290800 2024-04-23T14:40:26.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.540+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.540+0000 7f008142f740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-23T14:40:26.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.540+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-23T14:40:26.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.540+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-23T14:40:26.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.540+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-23T14:40:26.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.540+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-23T14:40:26.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.540+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676fc00 con 0x5650b7290800 2024-04-23T14:40:26.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.541+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.541+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-23T14:40:26.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.541+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-23T14:40:26.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.541+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-23T14:40:26.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.541+0000 7f008142f740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-23T14:40:26.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.541+0000 7f008142f740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-23T14:40:26.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.541+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e800 con 0x5650b715d000 2024-04-23T14:40:26.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.542+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.6 v2:172.21.15.76:6800/1890860772 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x5650b714c500 con 0x5650b715d000 2024-04-23T14:40:26.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.543+0000 7f008142f740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-23T14:40:26.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.543+0000 7f008142f740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-23T14:40:26.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.543+0000 7f008142f740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-23T14:40:26.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.543+0000 7f008142f740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-23T14:40:26.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.543+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676ec00 con 0x5650b72fb400 2024-04-23T14:40:26.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.544+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.544+0000 7f008142f740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-23T14:40:26.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.544+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f400 con 0x5650b72fb400 2024-04-23T14:40:26.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.548+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v87'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.548+0000 7f008142f740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-23T14:40:26.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.548+0000 7f008142f740 10 datascan.scan_links: processing 1 injected_inos 2024-04-23T14:40:26.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.548+0000 7f008142f740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-23T14:40:26.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.548+0000 7f008142f740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-23T14:40:26.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.548+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676e400 con 0x5650b72fb400 2024-04-23T14:40:26.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.549+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.549+0000 7f008142f740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-23T14:40:26.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.549+0000 7f008142f740 20 datascan.scan_links: first is now 2 2024-04-23T14:40:26.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.549+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f000 con 0x5650b72fb400 2024-04-23T14:40:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.554+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.0 v2:172.21.15.59:6817/2024703144 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v87'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5650b714c500 con 0x5650b72fb400 2024-04-23T14:40:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.554+0000 7f008142f740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-23T14:40:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.554+0000 7f008142f740 10 datascan.scan_links: updating inotable 2024-04-23T14:40:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.554+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b676f800 con 0x5650b7290800 2024-04-23T14:40:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.555+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.555+0000 7f008142f740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-23T14:40:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.555+0000 7f008142f740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-23T14:40:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.555+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5650b676fc00 con 0x5650b7290800 2024-04-23T14:40:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.559+0000 7f007dc00640 1 -- 172.21.15.59:0/36704417 <== osd.1 v2:172.21.15.59:6808/4218092099 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v87'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5650b714c000 con 0x5650b7290800 2024-04-23T14:40:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.559+0000 7f008142f740 10 datascan.scan_links: updating snaptable 2024-04-23T14:40:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.559+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5650b66cfc00 con 0x5650b715d400 2024-04-23T14:40:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.560+0000 7f007f000640 1 -- 172.21.15.59:0/36704417 <== osd.7 v2:172.21.15.76:6822/3281262326 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5650b714c280 con 0x5650b715d400 2024-04-23T14:40:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5650b72fb400 msgr2=0x5650b6742000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5650b72fb400 0x5650b6742000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5650b7290800 msgr2=0x5650b6742b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5650b7290800 0x5650b6742b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5650b66cec00 msgr2=0x5650b6744680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5650b66cec00 0x5650b6744680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5650b66ce000 msgr2=0x5650b6742580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5650b66ce000 0x5650b6742580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5650b72fb000 msgr2=0x5650b712c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5650b72fb000 0x5650b712c680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5650b66ce400 msgr2=0x5650b6743b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5650b66ce400 0x5650b6743b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 reap_dead start 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5650b715d000 msgr2=0x5650b6743080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5650b715d000 0x5650b6743080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 reap_dead start 2024-04-23T14:40:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5650b715d400 msgr2=0x5650b6744100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5650b715d400 0x5650b6744100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 reap_dead start 2024-04-23T14:40:26.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f007e600640 1 -- 172.21.15.59:0/36704417 reap_dead start 2024-04-23T14:40:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.561+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5650b72fac00 msgr2=0x5650b712c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5650b72fac00 0x5650b712c100 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5650b71601e0 tx=0x5650b715e900 comp rx=0 tx=0).stop 2024-04-23T14:40:26.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 msgr2=0x5650b712b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x5650b7289ce0 tx=0x5650b715e840 comp rx=0 tx=0).stop 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 shutdown_connections 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5650b72fac00 0x5650b712c100 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 --2- 172.21.15.59:0/36704417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b7291400 0x5650b712b080 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 >> 172.21.15.59:0/36704417 conn(0x5650b7291000 msgr2=0x5650b648a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 shutdown_connections 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.562+0000 7f008142f740 1 -- 172.21.15.59:0/36704417 wait complete. 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 -- 172.21.15.59:0/2847929552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b63bfc00 msgr2=0x5650b712ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b63bfc00 0x5650b712ab00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x5650b71d7d40 tx=0x5650b7162120 comp rx=0 tx=0).stop 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 -- 172.21.15.59:0/2847929552 shutdown_connections 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650b63bfc00 0x5650b712ab00 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650b63bf800 0x5650b712a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 --2- 172.21.15.59:0/2847929552 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5650b63bec00 0x5650b712a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 -- 172.21.15.59:0/2847929552 >> 172.21.15.59:0/2847929552 conn(0x5650b63bf000 msgr2=0x5650b648b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:26.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 -- 172.21.15.59:0/2847929552 shutdown_connections 2024-04-23T14:40:26.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.564+0000 7f008142f740 1 -- 172.21.15.59:0/2847929552 wait complete. 2024-04-23T14:40:26.577 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.264093, result: 2024-04-23T14:40:26.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-23T14:40:26.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.804+0000 7f7d3be00640 1 Processor -- start 2024-04-23T14:40:26.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.804+0000 7f7d3be00640 1 -- start start 2024-04-23T14:40:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d3be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d3c0d0d10 0x7f7d3c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d3be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d3c15d810 0x7f7d3c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d3be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3c15cab0 con 0x7f7d3c1567e0 2024-04-23T14:40:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d3be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c160e40 con 0x7f7d3c0d0d10 2024-04-23T14:40:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d3be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c15cd30 con 0x7f7d3c15d810 2024-04-23T14:40:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d32000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d3c0d0d10 0x7f7d3c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d32000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d3c0d0d10 0x7f7d3c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51992/0 (socket says 172.21.15.59:51992) 2024-04-23T14:40:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d32000640 1 -- 172.21.15.59:0/2561957054 learned_addr learned my addr 172.21.15.59:0/2561957054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d32000640 1 -- 172.21.15.59:0/2561957054 >> v2:172.21.15.76:3300/0 conn(0x7f7d3c15d810 msgr2=0x7f7d3c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d3b400640 1 --2- 172.21.15.59:0/2561957054 >> v2:172.21.15.76:3300/0 conn(0x7f7d3c15d810 0x7f7d3c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d32000640 1 --2- 172.21.15.59:0/2561957054 >> v2:172.21.15.76:3300/0 conn(0x7f7d3c15d810 0x7f7d3c160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.806+0000 7f7d32000640 1 -- 172.21.15.59:0/2561957054 >> v1:172.21.15.76:6789/0 conn(0x7f7d3c1567e0 legacy=0x7f7d3c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.807+0000 7f7d32000640 1 -- 172.21.15.59:0/2561957054 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c15d010 con 0x7f7d3c0d0d10 2024-04-23T14:40:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.808+0000 7f7d32000640 1 --2- 172.21.15.59:0/2561957054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0d0d10 0x7f7d3c156bc0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0b4cb0 tx=0x7f7d2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d2016d6572ee61 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.808+0000 7f7d3a000640 1 -- 172.21.15.59:0/2561957054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24037070 con 0x7f7d3c0d0d10 2024-04-23T14:40:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.808+0000 7f7d3a000640 1 -- 172.21.15.59:0/2561957054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d24032440 con 0x7f7d3c0d0d10 2024-04-23T14:40:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.808+0000 7f7d3a000640 1 -- 172.21.15.59:0/2561957054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24032810 con 0x7f7d3c0d0d10 2024-04-23T14:40:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 -- 172.21.15.59:0/2561957054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0d0d10 msgr2=0x7f7d3c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2561957054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0d0d10 0x7f7d3c156bc0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0b4cb0 tx=0x7f7d2402a230 comp rx=0 tx=0).stop 2024-04-23T14:40:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 -- 172.21.15.59:0/2561957054 shutdown_connections 2024-04-23T14:40:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2561957054 >> v2:172.21.15.76:3300/0 conn(0x7f7d3c15d810 0x7f7d3c160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2561957054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0d0d10 0x7f7d3c156bc0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 -- 172.21.15.59:0/2561957054 >> 172.21.15.59:0/2561957054 conn(0x7f7d3c06bdf0 msgr2=0x7f7d3c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 -- 172.21.15.59:0/2561957054 shutdown_connections 2024-04-23T14:40:26.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.809+0000 7f7d3be00640 1 -- 172.21.15.59:0/2561957054 wait complete. 2024-04-23T14:40:26.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.811+0000 7f7d3be00640 1 Processor -- start 2024-04-23T14:40:26.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.811+0000 7f7d3be00640 1 -- start start 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0d0d10 0x7f7d3c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1567e0 0x7f7d3c0986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c15d810 0x7f7d3c098bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1607d0 con 0x7f7d3c15d810 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c160650 con 0x7f7d3c1567e0 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c160950 con 0x7f7d3c0d0d10 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c15d810 0x7f7d3c098bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c15d810 0x7f7d3c098bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52000/0 (socket says 172.21.15.59:52000) 2024-04-23T14:40:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3b400640 1 -- 172.21.15.59:0/2036396233 learned_addr learned my addr 172.21.15.59:0/2036396233 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3aa00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0d0d10 0x7f7d3c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3b400640 1 -- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0d0d10 msgr2=0x7f7d3c151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d32000640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1567e0 0x7f7d3c0986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3b400640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0d0d10 0x7f7d3c151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3b400640 1 -- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1567e0 msgr2=0x7f7d3c0986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.812+0000 7f7d3b400640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1567e0 0x7f7d3c0986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d3b400640 1 -- 172.21.15.59:0/2036396233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c1f41a0 con 0x7f7d3c15d810 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d3aa00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0d0d10 0x7f7d3c151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d3b400640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c15d810 0x7f7d3c098bf0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400cd40 tx=0x7f7d3400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d33400640 1 -- 172.21.15.59:0/2036396233 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34003b90 con 0x7f7d3c15d810 2024-04-23T14:40:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d33400640 1 -- 172.21.15.59:0/2036396233 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d34003d30 con 0x7f7d3c15d810 2024-04-23T14:40:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d33400640 1 -- 172.21.15.59:0/2036396233 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34004240 con 0x7f7d3c15d810 2024-04-23T14:40:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c1f4480 con 0x7f7d3c15d810 2024-04-23T14:40:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.813+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c1f4a40 con 0x7f7d3c15d810 2024-04-23T14:40:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.814+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c1f4c40 con 0x7f7d3c15d810 2024-04-23T14:40:26.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.816+0000 7f7d33400640 1 -- 172.21.15.59:0/2036396233 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d340043e0 con 0x7f7d3c15d810 2024-04-23T14:40:26.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.817+0000 7f7d33400640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d1806c8f0 0x7f7d1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:26.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.817+0000 7f7d33400640 1 -- 172.21.15.59:0/2036396233 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f7d3408c7c0 con 0x7f7d3c15d810 2024-04-23T14:40:26.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.817+0000 7f7d3aa00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d1806c8f0 0x7f7d1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:26.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.818+0000 7f7d3aa00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d1806c8f0 0x7f7d1806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c1601b0 tx=0x7f7d2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:26.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:26.821+0000 7f7d33400640 1 -- 172.21.15.59:0/2036396233 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d3c1f4c40 con 0x7f7d3c15d810 2024-04-23T14:40:27.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.070+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7d3c04fd90 con 0x7f7d3c15d810 2024-04-23T14:40:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.763+0000 7f7d33400640 1 -- 172.21.15.59:0/2036396233 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v70) ==== 162+0+0 (secure 0 0 0) 0x7f7d3405aa80 con 0x7f7d3c15d810 2024-04-23T14:40:27.763 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-23T14:40:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.766+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d1806c8f0 msgr2=0x7f7d1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.766+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d1806c8f0 0x7f7d1806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c1601b0 tx=0x7f7d2c009040 comp rx=0 tx=0).stop 2024-04-23T14:40:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.766+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c15d810 msgr2=0x7f7d3c098bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.766+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c15d810 0x7f7d3c098bf0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400cd40 tx=0x7f7d3400a660 comp rx=0 tx=0).stop 2024-04-23T14:40:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 shutdown_connections 2024-04-23T14:40:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d1806c8f0 0x7f7d1806edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c15d810 0x7f7d3c098bf0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1567e0 0x7f7d3c0986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 --2- 172.21.15.59:0/2036396233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0d0d10 0x7f7d3c151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 >> 172.21.15.59:0/2036396233 conn(0x7f7d3c06bdf0 msgr2=0x7f7d3c073140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 shutdown_connections 2024-04-23T14:40:27.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.767+0000 7f7d3be00640 1 -- 172.21.15.59:0/2036396233 wait complete. 2024-04-23T14:40:27.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:40:27.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.985+0000 7f9bafe00640 1 Processor -- start 2024-04-23T14:40:27.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.986+0000 7f9bafe00640 1 -- start start 2024-04-23T14:40:27.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.987+0000 7f9bafe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9bb00cc7e0 0x7f9bb0167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:27.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.987+0000 7f9bafe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb0160210 con 0x7f9bb015c3d0 2024-04-23T14:40:27.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.987+0000 7f9bafe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb01604f0 con 0x7f9bb00cd760 2024-04-23T14:40:27.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.987+0000 7f9bafe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb01607d0 con 0x7f9bb00cc7e0 2024-04-23T14:40:27.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.987+0000 7f9baea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9bb015c3d0 0x7f9bb00cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53460/0 (socket says 172.21.15.59:53460) 2024-04-23T14:40:27.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.987+0000 7f9baea00640 1 -- 172.21.15.59:0/3444154156 learned_addr learned my addr 172.21.15.59:0/3444154156 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:27.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.987+0000 7f9baf400640 1 --2- 172.21.15.59:0/3444154156 >> v2:172.21.15.59:3300/0 conn(0x7f9bb00cc7e0 0x7f9bb0167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:27.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.988+0000 7f9baf400640 1 -- 172.21.15.59:0/3444154156 >> v1:172.21.15.76:6789/0 conn(0x7f9bb00cd760 legacy=0x7f9bb00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:27.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.988+0000 7f9baf400640 1 -- 172.21.15.59:0/3444154156 >> v1:172.21.15.59:6789/0 conn(0x7f9bb015c3d0 legacy=0x7f9bb00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:27.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.988+0000 7f9baf400640 1 -- 172.21.15.59:0/3444154156 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb0160ab0 con 0x7f9bb00cc7e0 2024-04-23T14:40:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.989+0000 7f9baf400640 1 --2- 172.21.15.59:0/3444154156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb0167400 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009d90 tx=0x7f9ba802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d290d738fc2bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.989+0000 7f9bae000640 1 -- 172.21.15.59:0/3444154156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8007740 con 0x7f9bb00cc7e0 2024-04-23T14:40:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.989+0000 7f9bae000640 1 -- 172.21.15.59:0/3444154156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba80078e0 con 0x7f9bb00cc7e0 2024-04-23T14:40:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.989+0000 7f9bae000640 1 -- 172.21.15.59:0/3444154156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8007cb0 con 0x7f9bb00cc7e0 2024-04-23T14:40:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.990+0000 7f9bafe00640 1 -- 172.21.15.59:0/3444154156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00cc7e0 msgr2=0x7f9bb0167400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.990+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3444154156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb0167400 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009d90 tx=0x7f9ba802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.990+0000 7f9bafe00640 1 -- 172.21.15.59:0/3444154156 shutdown_connections 2024-04-23T14:40:27.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.990+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3444154156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb0167400 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.990+0000 7f9bafe00640 1 -- 172.21.15.59:0/3444154156 >> 172.21.15.59:0/3444154156 conn(0x7f9bb00b21a0 msgr2=0x7f9bb00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:27.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.990+0000 7f9bafe00640 1 -- 172.21.15.59:0/3444154156 shutdown_connections 2024-04-23T14:40:27.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.990+0000 7f9bafe00640 1 -- 172.21.15.59:0/3444154156 wait complete. 2024-04-23T14:40:27.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.992+0000 7f9bafe00640 1 Processor -- start 2024-04-23T14:40:27.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.992+0000 7f9bafe00640 1 -- start start 2024-04-23T14:40:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb00c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9bb00cd760 0x7f9bb00c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9bafe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015c3d0 0x7f9bb01f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9bafe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb015c930 con 0x7f9bb015c3d0 2024-04-23T14:40:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9bafe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb015c7b0 con 0x7f9bb00cc7e0 2024-04-23T14:40:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9bafe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00b4860 con 0x7f9bb00cd760 2024-04-23T14:40:27.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9baf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015c3d0 0x7f9bb01f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9baf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015c3d0 0x7f9bb01f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52010/0 (socket says 172.21.15.59:52010) 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.993+0000 7f9baf400640 1 -- 172.21.15.59:0/3053902545 learned_addr learned my addr 172.21.15.59:0/3053902545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baea00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb00c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9ba6000640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9bb00cd760 0x7f9bb00c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baf400640 1 -- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9bb00cd760 msgr2=0x7f9bb00c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baf400640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9bb00cd760 0x7f9bb00c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baf400640 1 -- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bb00cc7e0 msgr2=0x7f9bb00c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baf400640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb00c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baf400640 1 -- 172.21.15.59:0/3053902545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb01599c0 con 0x7f9bb015c3d0 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baea00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb00c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:27.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9baf400640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015c3d0 0x7f9bb01f5be0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009d90 tx=0x7f9ba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.994+0000 7f9ba7400640 1 -- 172.21.15.59:0/3053902545 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8040070 con 0x7f9bb015c3d0 2024-04-23T14:40:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.995+0000 7f9ba7400640 1 -- 172.21.15.59:0/3053902545 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba8031c90 con 0x7f9bb015c3d0 2024-04-23T14:40:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.995+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb0159ca0 con 0x7f9bb015c3d0 2024-04-23T14:40:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.995+0000 7f9ba7400640 1 -- 172.21.15.59:0/3053902545 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba803c640 con 0x7f9bb015c3d0 2024-04-23T14:40:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.995+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb015a260 con 0x7f9bb015c3d0 2024-04-23T14:40:27.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.996+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb004fd90 con 0x7f9bb015c3d0 2024-04-23T14:40:27.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.997+0000 7f9ba7400640 1 -- 172.21.15.59:0/3053902545 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ba8039040 con 0x7f9bb015c3d0 2024-04-23T14:40:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.998+0000 7f9ba7400640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.998+0000 7f9ba7400640 1 -- 172.21.15.59:0/3053902545 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f9ba80ad270 con 0x7f9bb015c3d0 2024-04-23T14:40:27.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.998+0000 7f9baea00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:28.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:27.999+0000 7f9baea00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00cd140 tx=0x7f9ba0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:28.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.002+0000 7f9ba7400640 1 -- 172.21.15.59:0/3053902545 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ba807b690 con 0x7f9bb015c3d0 2024-04-23T14:40:28.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.246+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bb0075150 con 0x7f9bb015c3d0 2024-04-23T14:40:28.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.247+0000 7f9ba7400640 1 -- 172.21.15.59:0/3053902545 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5174 (secure 0 0 0) 0x7f9ba807af90 con 0x7f9bb015c3d0 2024-04-23T14:40:28.249 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:28.249 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:27.766850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-23T14:40:28.249 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 71 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b8c06c8f0 msgr2=0x7f9b8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00cd140 tx=0x7f9ba0009040 comp rx=0 tx=0).stop 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015c3d0 msgr2=0x7f9bb01f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015c3d0 0x7f9bb01f5be0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009d90 tx=0x7f9ba8004a40 comp rx=0 tx=0).stop 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 shutdown_connections 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b8c06c8f0 0x7f9b8c06edb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9bb00cd760 0x7f9bb00c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9bb015c3d0 0x7f9bb01f5be0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 --2- 172.21.15.59:0/3053902545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bb00cc7e0 0x7f9bb00c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 >> 172.21.15.59:0/3053902545 conn(0x7f9bb00b21a0 msgr2=0x7f9bb01591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.250+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 shutdown_connections 2024-04-23T14:40:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:28.251+0000 7f9bafe00640 1 -- 172.21.15.59:0/3053902545 wait complete. 2024-04-23T14:40:28.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:38:39.210981+0000', 'modified': '2024-04-23T14:40:27.766850+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/3466627380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 3466627380}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3466627380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:40:28.675 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8133:up:replay 2024-04-23T14:40:29.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:40:29.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.895+0000 7f8683200640 1 Processor -- start 2024-04-23T14:40:29.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.896+0000 7f8683200640 1 -- start start 2024-04-23T14:40:29.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.897+0000 7f8683200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 0x7f86840d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.897+0000 7f8683200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868415fa20 con 0x7f86840d0ec0 2024-04-23T14:40:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.897+0000 7f8683200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86840cf430 con 0x7f86840d0330 2024-04-23T14:40:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.897+0000 7f8683200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f86840b3b60 con 0x7f86840cff10 2024-04-23T14:40:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.898+0000 7f8681e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f86840d0ec0 0x7f86840d12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53468/0 (socket says 172.21.15.59:53468) 2024-04-23T14:40:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.898+0000 7f8681e00640 1 -- 172.21.15.59:0/490931555 learned_addr learned my addr 172.21.15.59:0/490931555 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.898+0000 7f8681400640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 0x7f86840d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.898+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396063118 0 0) 0x7f868415fa20 con 0x7f86840d0ec0 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.898+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8668003520 con 0x7f86840d0ec0 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3813112983 0 0) 0x7f8668003520 con 0x7f86840d0ec0 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 msgr2=0x7f86840d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 0x7f86840d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 >> v1:172.21.15.76:6789/0 conn(0x7f86840d0330 legacy=0x7f868415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86840cf710 con 0x7f86840d0ec0 2024-04-23T14:40:29.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8674001670 con 0x7f86840d0ec0 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 >> v1:172.21.15.59:6789/0 conn(0x7f86840d0ec0 legacy=0x7f86840d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86680039e0 0x7f8668003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 0x7f8668004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8668004d40 0x7f86680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f866800bb00 con 0x7f8668004d40 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f866800bcb0 con 0x7f86680039e0 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f866800beb0 con 0x7f86840cff10 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8682800640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8668004d40 0x7f86680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.899+0000 7f8681e00640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86680039e0 0x7f8668003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.900+0000 7f8681400640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 0x7f8668004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.900+0000 7f8682800640 1 -- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 msgr2=0x7f8668004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.900+0000 7f8682800640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86840cff10 0x7f8668004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:29.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.900+0000 7f8682800640 1 -- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86680039e0 msgr2=0x7f8668003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:29.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.900+0000 7f8682800640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86680039e0 0x7f8668003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:29.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.900+0000 7f8682800640 1 -- 172.21.15.59:0/490931555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8668003520 con 0x7f8668004d40 2024-04-23T14:40:29.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.901+0000 7f8682800640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8668004d40 0x7f86680051c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f867c001e20 tx=0x7f8668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30811f7e98718b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:29.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.901+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c001a70 con 0x7f8668004d40 2024-04-23T14:40:29.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.901+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867c02ae40 con 0x7f8668004d40 2024-04-23T14:40:29.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.901+0000 7f8680a00640 1 -- 172.21.15.59:0/490931555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c003bb0 con 0x7f8668004d40 2024-04-23T14:40:29.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.902+0000 7f8683200640 1 -- 172.21.15.59:0/490931555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8668004d40 msgr2=0x7f86680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:29.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.902+0000 7f8683200640 1 --2- 172.21.15.59:0/490931555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8668004d40 0x7f86680051c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f867c001e20 tx=0x7f8668005830 comp rx=0 tx=0).stop 2024-04-23T14:40:29.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.902+0000 7f8682800640 1 -- 172.21.15.59:0/490931555 reap_dead start 2024-04-23T14:40:29.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.902+0000 7f8683200640 1 -- 172.21.15.59:0/490931555 shutdown_connections 2024-04-23T14:40:29.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.902+0000 7f8683200640 1 -- 172.21.15.59:0/490931555 >> 172.21.15.59:0/490931555 conn(0x7f868408bf90 msgr2=0x7f86840d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:29.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.902+0000 7f8683200640 1 -- 172.21.15.59:0/490931555 shutdown_connections 2024-04-23T14:40:29.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.902+0000 7f8683200640 1 -- 172.21.15.59:0/490931555 wait complete. 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.904+0000 7f8683200640 1 Processor -- start 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.904+0000 7f8683200640 1 -- start start 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8683200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8668004d40 0x7f86840b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8683200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8684159850 0x7f8684159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8683200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b7180 0x7f86840b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8683200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f868415f220 con 0x7f86840b7180 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8683200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f868415fd00 con 0x7f8668004d40 2024-04-23T14:40:29.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8683200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f868415f3a0 con 0x7f8684159850 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8682800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b7180 0x7f86840b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8682800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b7180 0x7f86840b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52028/0 (socket says 172.21.15.59:52028) 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.905+0000 7f8682800640 1 -- 172.21.15.59:0/2701216050 learned_addr learned my addr 172.21.15.59:0/2701216050 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8681e00640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8668004d40 0x7f86840b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8682800640 1 -- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8684159850 msgr2=0x7f8684159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8681400640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8684159850 0x7f8684159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8682800640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8684159850 0x7f8684159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8682800640 1 -- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8668004d40 msgr2=0x7f86840b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8682800640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8668004d40 0x7f86840b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:29.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8682800640 1 -- 172.21.15.59:0/2701216050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868415a210 con 0x7f86840b7180 2024-04-23T14:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.906+0000 7f8682800640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b7180 0x7f86840b7600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f867c000f80 tx=0x7f867c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.907+0000 7f867a000640 1 -- 172.21.15.59:0/2701216050 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c007db0 con 0x7f86840b7180 2024-04-23T14:40:29.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.907+0000 7f867a000640 1 -- 172.21.15.59:0/2701216050 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867c02ac40 con 0x7f86840b7180 2024-04-23T14:40:29.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.907+0000 7f867a000640 1 -- 172.21.15.59:0/2701216050 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0050d0 con 0x7f86840b7180 2024-04-23T14:40:29.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.907+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86840b7ef0 con 0x7f86840b7180 2024-04-23T14:40:29.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.907+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86840b84b0 con 0x7f86840b7180 2024-04-23T14:40:29.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.908+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868415fa20 con 0x7f86840b7180 2024-04-23T14:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.909+0000 7f867a000640 1 -- 172.21.15.59:0/2701216050 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f867c043020 con 0x7f86840b7180 2024-04-23T14:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.910+0000 7f867a000640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f866406c920 0x7f866406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.910+0000 7f867a000640 1 -- 172.21.15.59:0/2701216050 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f867c0ab0f0 con 0x7f86840b7180 2024-04-23T14:40:29.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.910+0000 7f8681e00640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f866406c920 0x7f866406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:29.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.911+0000 7f8681e00640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f866406c920 0x7f866406ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f86840b2e20 tx=0x7f8674009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:29.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:29.914+0000 7f867a000640 1 -- 172.21.15.59:0/2701216050 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f868415fa20 con 0x7f86840b7180 2024-04-23T14:40:30.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.160+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86840ba490 con 0x7f86840b7180 2024-04-23T14:40:30.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.161+0000 7f867a000640 1 -- 172.21.15.59:0/2701216050 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5174 (secure 0 0 0) 0x7f867c0795c0 con 0x7f86840b7180 2024-04-23T14:40:30.162 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:30.163 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:28.790412+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-23T14:40:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 73 2024-04-23T14:40:30.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f866406c920 msgr2=0x7f866406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:30.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f866406c920 0x7f866406ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f86840b2e20 tx=0x7f8674009040 comp rx=0 tx=0).stop 2024-04-23T14:40:30.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b7180 msgr2=0x7f86840b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:30.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b7180 0x7f86840b7600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f867c000f80 tx=0x7f867c0014a0 comp rx=0 tx=0).stop 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 shutdown_connections 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f866406c920 0x7f866406ede0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8684159850 0x7f8684159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b7180 0x7f86840b7600 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 --2- 172.21.15.59:0/2701216050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8668004d40 0x7f86840b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 >> 172.21.15.59:0/2701216050 conn(0x7f868408bf90 msgr2=0x7f8684160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 shutdown_connections 2024-04-23T14:40:30.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:30.164+0000 7f8683200640 1 -- 172.21.15.59:0/2701216050 wait complete. 2024-04-23T14:40:30.588 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:38:39.210981+0000', 'modified': '2024-04-23T14:40:28.790412+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.59:6837/3466627380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 3466627380}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3466627380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:40:30.589 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8133:up:rejoin 2024-04-23T14:40:31.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:40:31.759 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:40:31.758+0000 7f3598e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:40:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.808+0000 7fc7e3200640 1 Processor -- start 2024-04-23T14:40:31.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.809+0000 7fc7e3200640 1 -- start start 2024-04-23T14:40:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7e415e7c0 0x7fc7e40d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d0d90 0x7fc7e40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e3200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e40cf030 con 0x7fc7e40d2630 2024-04-23T14:40:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e40d1740 con 0x7fc7e415e7c0 2024-04-23T14:40:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e40b7000 con 0x7fc7e40d0d90 2024-04-23T14:40:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e1e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7e415e7c0 0x7fc7e40d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:31.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e1e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7e415e7c0 0x7fc7e40d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51410/0 (socket says 172.21.15.59:51410) 2024-04-23T14:40:31.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7e1e00640 1 -- 172.21.15.59:0/2673292394 learned_addr learned my addr 172.21.15.59:0/2673292394 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:31.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.810+0000 7fc7dbe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc7e40d2630 0x7fc7e40b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35296/0 (socket says 172.21.15.59:35296) 2024-04-23T14:40:31.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.811+0000 7fc7e1e00640 1 -- 172.21.15.59:0/2673292394 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d0d90 msgr2=0x7fc7e40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:31.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.811+0000 7fc7e1e00640 1 --2- 172.21.15.59:0/2673292394 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d0d90 0x7fc7e40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:31.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.811+0000 7fc7e1e00640 1 -- 172.21.15.59:0/2673292394 >> v1:172.21.15.59:6789/0 conn(0x7fc7e40d2630 legacy=0x7fc7e40b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:31.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.811+0000 7fc7e1e00640 1 -- 172.21.15.59:0/2673292394 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e40b6a40 con 0x7fc7e415e7c0 2024-04-23T14:40:31.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.811+0000 7fc7e1400640 1 -- 172.21.15.59:0/2673292394 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792364955 0 0) 0x7fc7cc001110 con 0x7fc7e40d2630 2024-04-23T14:40:31.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.812+0000 7fc7e1e00640 1 --2- 172.21.15.59:0/2673292394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e415e7c0 0x7fc7e40d20f0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009870 tx=0x7fc7d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367b0d4fb762844f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:31.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.812+0000 7fc7e1400640 1 -- 172.21.15.59:0/2673292394 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4037070 con 0x7fc7e415e7c0 2024-04-23T14:40:31.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.812+0000 7fc7e1400640 1 -- 172.21.15.59:0/2673292394 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d4032440 con 0x7fc7e415e7c0 2024-04-23T14:40:31.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.812+0000 7fc7e1400640 1 -- 172.21.15.59:0/2673292394 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4032810 con 0x7fc7e415e7c0 2024-04-23T14:40:31.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 -- 172.21.15.59:0/2673292394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e415e7c0 msgr2=0x7fc7e40d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:31.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 --2- 172.21.15.59:0/2673292394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e415e7c0 0x7fc7e40d20f0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009870 tx=0x7fc7d402a270 comp rx=0 tx=0).stop 2024-04-23T14:40:31.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 -- 172.21.15.59:0/2673292394 shutdown_connections 2024-04-23T14:40:31.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 --2- 172.21.15.59:0/2673292394 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d0d90 0x7fc7e40d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:31.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 --2- 172.21.15.59:0/2673292394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e415e7c0 0x7fc7e40d20f0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:31.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 -- 172.21.15.59:0/2673292394 >> 172.21.15.59:0/2673292394 conn(0x7fc7e409d920 msgr2=0x7fc7e408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:31.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 -- 172.21.15.59:0/2673292394 shutdown_connections 2024-04-23T14:40:31.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.813+0000 7fc7e3200640 1 -- 172.21.15.59:0/2673292394 wait complete. 2024-04-23T14:40:31.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.815+0000 7fc7e3200640 1 Processor -- start 2024-04-23T14:40:31.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.815+0000 7fc7e3200640 1 -- start start 2024-04-23T14:40:31.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e40d0d90 0x7fc7e415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:31.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d2630 0x7fc7e415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7e415e7c0 0x7fc7e4157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40cf2b0 con 0x7fc7e40d0d90 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40cf620 con 0x7fc7e415e7c0 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e40b6700 con 0x7fc7e40d2630 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e40d0d90 0x7fc7e415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e40d0d90 0x7fc7e415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51422/0 (socket says 172.21.15.59:51422) 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e1e00640 1 -- 172.21.15.59:0/3056053371 learned_addr learned my addr 172.21.15.59:0/3056053371 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7dbe00640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d2630 0x7fc7e415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e1e00640 1 -- 172.21.15.59:0/3056053371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d2630 msgr2=0x7fc7e415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e1e00640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d2630 0x7fc7e415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.816+0000 7fc7e1e00640 1 -- 172.21.15.59:0/3056053371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7e415e7c0 msgr2=0x7fc7e4157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.817+0000 7fc7e1e00640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7e415e7c0 0x7fc7e4157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:31.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.817+0000 7fc7e1e00640 1 -- 172.21.15.59:0/3056053371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e40ccad0 con 0x7fc7e40d0d90 2024-04-23T14:40:31.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.817+0000 7fc7e1e00640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e40d0d90 0x7fc7e415d6f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009870 tx=0x7fc7d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:31.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.817+0000 7fc7da000640 1 -- 172.21.15.59:0/3056053371 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4037040 con 0x7fc7e40d0d90 2024-04-23T14:40:31.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.817+0000 7fc7da000640 1 -- 172.21.15.59:0/3056053371 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d4004030 con 0x7fc7e40d0d90 2024-04-23T14:40:31.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.817+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e40ccdb0 con 0x7fc7e40d0d90 2024-04-23T14:40:31.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.817+0000 7fc7da000640 1 -- 172.21.15.59:0/3056053371 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4004440 con 0x7fc7e40d0d90 2024-04-23T14:40:31.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.818+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e40cd890 con 0x7fc7e40d0d90 2024-04-23T14:40:31.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.819+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e404fd90 con 0x7fc7e40d0d90 2024-04-23T14:40:31.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.820+0000 7fc7da000640 1 -- 172.21.15.59:0/3056053371 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7d403a040 con 0x7fc7e40d0d90 2024-04-23T14:40:31.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.821+0000 7fc7da000640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7c006c8f0 0x7fc7c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:31.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.821+0000 7fc7da000640 1 -- 172.21.15.59:0/3056053371 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fc7d40ac4f0 con 0x7fc7e40d0d90 2024-04-23T14:40:31.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.821+0000 7fc7dbe00640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7c006c8f0 0x7fc7c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:31.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.822+0000 7fc7dbe00640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7c006c8f0 0x7fc7c006edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc7e415dfb0 tx=0x7fc7cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:31.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:31.826+0000 7fc7da000640 1 -- 172.21.15.59:0/3056053371 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7d4042020 con 0x7fc7e40d0d90 2024-04-23T14:40:32.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.068+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e40d2d10 con 0x7fc7e40d0d90 2024-04-23T14:40:32.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.069+0000 7fc7da000640 1 -- 172.21.15.59:0/3056053371 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5181 (secure 0 0 0) 0x7fc7d407a960 con 0x7fc7e40d0d90 2024-04-23T14:40:32.071 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:40:32.071 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:30.798278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-23T14:40:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 74 2024-04-23T14:40:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7c006c8f0 msgr2=0x7fc7c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7c006c8f0 0x7fc7c006edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc7e415dfb0 tx=0x7fc7cc009040 comp rx=0 tx=0).stop 2024-04-23T14:40:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e40d0d90 msgr2=0x7fc7e415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e40d0d90 0x7fc7e415d6f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009870 tx=0x7fc7d40046c0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 shutdown_connections 2024-04-23T14:40:32.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc7c006c8f0 0x7fc7c006edb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7e40d2630 0x7fc7e415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7e415e7c0 0x7fc7e4157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 --2- 172.21.15.59:0/3056053371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7e40d0d90 0x7fc7e415d6f0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 >> 172.21.15.59:0/3056053371 conn(0x7fc7e409d920 msgr2=0x7fc7e408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:32.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 shutdown_connections 2024-04-23T14:40:32.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.072+0000 7fc7e3200640 1 -- 172.21.15.59:0/3056053371 wait complete. 2024-04-23T14:40:32.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:38:39.210981+0000', 'modified': '2024-04-23T14:40:30.798278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8133}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8133': {'gid': 8133, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6837/3466627380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 3466627380}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3466627380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8133, 'qdb_cluster': [8133]} 2024-04-23T14:40:32.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:40:32.494 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:40:32.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.719+0000 7fefcfe00640 1 Processor -- start 2024-04-23T14:40:32.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.719+0000 7fefcfe00640 1 -- start start 2024-04-23T14:40:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.720+0000 7fefcfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fefd00cd8b0 0x7fefd00d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.720+0000 7fefcfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00d0b00 0x7fefd006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.720+0000 7fefcfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd00b4cf0 con 0x7fefd015c670 2024-04-23T14:40:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.720+0000 7fefcfe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fefd015ca50 con 0x7fefd00cd8b0 2024-04-23T14:40:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.720+0000 7fefcfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefd00ccb80 con 0x7fefd00d0b00 2024-04-23T14:40:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.721+0000 7fefcea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fefd015c670 0x7fefd00b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35312/0 (socket says 172.21.15.59:35312) 2024-04-23T14:40:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.721+0000 7fefcea00640 1 -- 172.21.15.59:0/988651524 learned_addr learned my addr 172.21.15.59:0/988651524 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:32.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.721+0000 7fefcf400640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00d0b00 0x7fefd006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.721+0000 7fefce000640 1 --2- 172.21.15.59:0/988651524 >> v2:172.21.15.76:3300/0 conn(0x7fefd00cd8b0 0x7fefd00d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.721+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860465167 0 0) 0x7fefd00b4cf0 con 0x7fefd015c670 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.721+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefb4003520 con 0x7fefd015c670 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 92410411 0 0) 0x7fefb4003520 con 0x7fefd015c670 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00d0b00 msgr2=0x7fefd006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00d0b00 0x7fefd006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 >> v2:172.21.15.76:3300/0 conn(0x7fefd00cd8b0 msgr2=0x7fefd00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 --2- 172.21.15.59:0/988651524 >> v2:172.21.15.76:3300/0 conn(0x7fefd00cd8b0 0x7fefd00d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd00ccdd0 con 0x7fefd015c670 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fefc0001670 con 0x7fefd015c670 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 >> v1:172.21.15.59:6789/0 conn(0x7fefd015c670 legacy=0x7fefd00b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00d0b00 0x7fefb40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb4003f80 0x7fefb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4004d40 0x7fefb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00ccb80 con 0x7fefb4003f80 2024-04-23T14:40:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefb400b9c0 con 0x7fefb4004d40 2024-04-23T14:40:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefb400bb40 con 0x7fefd00d0b00 2024-04-23T14:40:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.722+0000 7fefce000640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb4003f80 0x7fefb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.723+0000 7fefce000640 1 -- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00d0b00 msgr2=0x7fefb40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.723+0000 7fefce000640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00d0b00 0x7fefb40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.723+0000 7fefcf400640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4004d40 0x7fefb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.723+0000 7fefce000640 1 -- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4004d40 msgr2=0x7fefb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.723+0000 7fefce000640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4004d40 0x7fefb40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.723+0000 7fefce000640 1 -- 172.21.15.59:0/988651524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb4003520 con 0x7fefb4003f80 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.724+0000 7fefce000640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb4003f80 0x7fefb4004800 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009ac0 tx=0x7fefb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ddc5d30258e15f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.724+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8036070 con 0x7fefb4003f80 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.724+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb8031400 con 0x7fefb4003f80 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.724+0000 7fefcd600640 1 -- 172.21.15.59:0/988651524 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8009690 con 0x7fefb4003f80 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.725+0000 7fefcfe00640 1 -- 172.21.15.59:0/988651524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb4003f80 msgr2=0x7fefb4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.725+0000 7fefcfe00640 1 --2- 172.21.15.59:0/988651524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefb4003f80 0x7fefb4004800 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009ac0 tx=0x7fefb4005830 comp rx=0 tx=0).stop 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.725+0000 7fefcf400640 1 -- 172.21.15.59:0/988651524 reap_dead start 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.725+0000 7fefcfe00640 1 -- 172.21.15.59:0/988651524 shutdown_connections 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.725+0000 7fefcfe00640 1 -- 172.21.15.59:0/988651524 >> 172.21.15.59:0/988651524 conn(0x7fefd006bdf0 msgr2=0x7fefd006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.725+0000 7fefcfe00640 1 -- 172.21.15.59:0/988651524 shutdown_connections 2024-04-23T14:40:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.725+0000 7fefcfe00640 1 -- 172.21.15.59:0/988651524 wait complete. 2024-04-23T14:40:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.727+0000 7fefcfe00640 1 Processor -- start 2024-04-23T14:40:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.727+0000 7fefcfe00640 1 -- start start 2024-04-23T14:40:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefcfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4003f80 0x7fefd0099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd0157f60 0x7fefd01583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefcfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00b0330 0x7fefd00b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefcfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00b3e60 con 0x7fefd0157f60 2024-04-23T14:40:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefcfe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00cd390 con 0x7fefb4003f80 2024-04-23T14:40:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefcfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefd00b3fe0 con 0x7fefd00b0330 2024-04-23T14:40:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefce000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd0157f60 0x7fefd01583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefce000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd0157f60 0x7fefd01583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51452/0 (socket says 172.21.15.59:51452) 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.728+0000 7fefce000640 1 -- 172.21.15.59:0/2140076532 learned_addr learned my addr 172.21.15.59:0/2140076532 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefcea00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4003f80 0x7fefd0099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefcf400640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00b0330 0x7fefd00b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefce000640 1 -- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00b0330 msgr2=0x7fefd00b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefce000640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00b0330 0x7fefd00b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefce000640 1 -- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4003f80 msgr2=0x7fefd0099ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefce000640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4003f80 0x7fefd0099ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefce000640 1 -- 172.21.15.59:0/2140076532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd0151f80 con 0x7fefd0157f60 2024-04-23T14:40:32.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefcea00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4003f80 0x7fefd0099ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:32.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.729+0000 7fefce000640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd0157f60 0x7fefd01583e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009be0 tx=0x7fefb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:32.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.730+0000 7fefc6a00640 1 -- 172.21.15.59:0/2140076532 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8040020 con 0x7fefd0157f60 2024-04-23T14:40:32.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.730+0000 7fefc6a00640 1 -- 172.21.15.59:0/2140076532 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb803b680 con 0x7fefd0157f60 2024-04-23T14:40:32.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.730+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd0150ec0 con 0x7fefd0157f60 2024-04-23T14:40:32.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.730+0000 7fefc6a00640 1 -- 172.21.15.59:0/2140076532 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8036040 con 0x7fefd0157f60 2024-04-23T14:40:32.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.730+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd00b11f0 con 0x7fefd0157f60 2024-04-23T14:40:32.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.732+0000 7fefc6a00640 1 -- 172.21.15.59:0/2140076532 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefb8043020 con 0x7fefd0157f60 2024-04-23T14:40:32.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.733+0000 7fefc6a00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefac06c920 0x7fefac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.733+0000 7fefc6a00640 1 -- 172.21.15.59:0/2140076532 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fefb80ac6f0 con 0x7fefd0157f60 2024-04-23T14:40:32.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.733+0000 7fefcea00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefac06c920 0x7fefac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.734+0000 7fefd8412740 1 Processor -- start 2024-04-23T14:40:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.734+0000 7fefd8412740 1 -- start start 2024-04-23T14:40:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.734+0000 7fefcea00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefac06c920 0x7fefac06ede0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fefc00079d0 tx=0x7fefc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa440a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa414980 0x55a6fa43d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefd8412740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefd8412740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6fa37ab10 con 0x55a6fa43d590 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6fa3212b0 con 0x55a6fa43cc70 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a6fa376ad0 con 0x55a6fa414980 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51458/0 (socket says 172.21.15.59:51458) 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefcf400640 1 -- 172.21.15.59:0/3424743110 learned_addr learned my addr 172.21.15.59:0/3424743110 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefcf400640 1 -- 172.21.15.59:0/3424743110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa414980 msgr2=0x55a6fa43d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefcf400640 1 --2- 172.21.15.59:0/3424743110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa414980 0x55a6fa43d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefcf400640 1 -- 172.21.15.59:0/3424743110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 msgr2=0x55a6fa440a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.735+0000 7fefcf400640 1 --2- 172.21.15.59:0/3424743110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa440a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.736+0000 7fefcf400640 1 -- 172.21.15.59:0/3424743110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fa08d2d0 con 0x55a6fa43d590 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.736+0000 7fefcf400640 1 --2- 172.21.15.59:0/3424743110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43fe20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009fc0 tx=0x7fefc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb85a0be35eed0d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.736+0000 7fefbf000640 1 -- 172.21.15.59:0/3424743110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc801b070 con 0x55a6fa43d590 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.736+0000 7fefbf000640 1 -- 172.21.15.59:0/3424743110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8016400 con 0x55a6fa43d590 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.736+0000 7fefbf000640 1 -- 172.21.15.59:0/3424743110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc80167f0 con 0x55a6fa43d590 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 -- 172.21.15.59:0/3424743110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 msgr2=0x55a6fa43fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 --2- 172.21.15.59:0/3424743110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43fe20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009fc0 tx=0x7fefc800b040 comp rx=0 tx=0).stop 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 -- 172.21.15.59:0/3424743110 shutdown_connections 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 --2- 172.21.15.59:0/3424743110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa414980 0x55a6fa43d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 --2- 172.21.15.59:0/3424743110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43fe20 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 --2- 172.21.15.59:0/3424743110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa440a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 -- 172.21.15.59:0/3424743110 >> 172.21.15.59:0/3424743110 conn(0x55a6fa412580 msgr2=0x55a6fa45ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 -- 172.21.15.59:0/3424743110 shutdown_connections 2024-04-23T14:40:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.737+0000 7fefd8412740 1 -- 172.21.15.59:0/3424743110 wait complete. 2024-04-23T14:40:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.738+0000 7fefd8412740 1 -- start start 2024-04-23T14:40:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.738+0000 7fefd8412740 1 Processor -- start 2024-04-23T14:40:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa42d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefd8412740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 0x55a6fa434c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefd8412740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6fa4108d0 con 0x55a6fa43d590 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6fa334590 con 0x55a6fa43cc70 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a6fa410310 con 0x55a6fa433580 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefd8412740 10 client.0 _ll_register_callbacks cb 0x55a6fa3173a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefce000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefce000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51470/0 (socket says 172.21.15.59:51470) 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.739+0000 7fefce000640 1 -- 172.21.15.59:0/2098273989 learned_addr learned my addr 172.21.15.59:0/2098273989 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefce000640 1 -- 172.21.15.59:0/2098273989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 msgr2=0x55a6fa434c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefce000640 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 0x55a6fa434c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefcea00640 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa42d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefce000640 1 -- 172.21.15.59:0/2098273989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 msgr2=0x55a6fa42d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefce000640 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa42d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefce000640 1 -- 172.21.15.59:0/2098273989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fa1576b0 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefce000640 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43aaf0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fefb8076710 tx=0x7fefb807a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa7a11cae1a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefb3400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefb3400640 1 -- 172.21.15.59:0/2098273989 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8082c00 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefb3400640 1 -- 172.21.15.59:0/2098273989 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fefb80a5b90 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.740+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefc8016400 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefb3400640 1 -- 172.21.15.59:0/2098273989 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fefb807e660 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 10 client.8208 fetch_fsmap learned FSMap version 74 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a6fa43b9e0 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefb3400640 1 -- 172.21.15.59:0/2098273989 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8016400 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefb3400640 1 -- 172.21.15.59:0/2098273989 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fefb8079700 con 0x55a6fa43d590 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 10 client.8208 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 10 client.8208 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:40:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 20 client.8208 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa439520 0x55a6fa439900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 4 client.8208 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6fa388530 con 0x55a6fa439520 2024-04-23T14:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.741+0000 7fefcea00640 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa439520 0x55a6fa439900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.742+0000 7fefcea00640 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa439520 0x55a6fa439900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9cfa855792d3500 server_cookie=bdb93e39a2667a91 in_seq=0 out_seq=0 2024-04-23T14:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.742+0000 7fefb3400640 10 client.8208 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefb3400640 1 -- 172.21.15.59:0/2098273989 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6fa388530 con 0x55a6fa439520 2024-04-23T14:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefb3400640 10 client.8208 handle_command_reply: tid=0 2024-04-23T14:40:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 client.8208 shutdown 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 msgr2=0x55a6fa43aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43aaf0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fefb8076710 tx=0x7fefb807a9b0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 shutdown_connections 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa439520 0x55a6fa439900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 0x55a6fa434c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa43aaf0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 --2- 172.21.15.59:0/2098273989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa42d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 >> 172.21.15.59:0/2098273989 conn(0x55a6fa414980 msgr2=0x55a6fa415bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.743+0000 7fefb3400640 0 client.8208 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.744+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 shutdown_connections 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.744+0000 7fefd8412740 1 -- 172.21.15.59:0/2098273989 wait complete. 2024-04-23T14:40:32.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.744+0000 7fefd8412740 20 client.8208 trim_cache size 0 max 16384 2024-04-23T14:40:32.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.764+0000 7fefd8412740 1 Processor -- start 2024-04-23T14:40:32.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.764+0000 7fefd8412740 1 -- start start 2024-04-23T14:40:32.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa433580 0x55a6fa414140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa43cc70 0x55a6fa431a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefd8412740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa414ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefd8412740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00736f0 con 0x55a6fa43d590 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0157a30 con 0x55a6fa433580 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fefd00730a0 con 0x55a6fa43cc70 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa414ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa414ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51486/0 (socket says 172.21.15.59:51486) 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.765+0000 7fefcf400640 1 -- 172.21.15.59:0/3983669433 learned_addr learned my addr 172.21.15.59:0/3983669433 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefce000640 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa43cc70 0x55a6fa431a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefcf400640 1 -- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa43cc70 msgr2=0x55a6fa431a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefcf400640 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa43cc70 0x55a6fa431a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefcf400640 1 -- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa433580 msgr2=0x55a6fa414140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefcea00640 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa433580 0x55a6fa414140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefcf400640 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa433580 0x55a6fa414140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefcf400640 1 -- 172.21.15.59:0/3983669433 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fa40f0e0 con 0x55a6fa43d590 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefcf400640 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa414ea0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fefc8019040 tx=0x7fefd00b4ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24f004197faef3c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefbe600640 1 -- 172.21.15.59:0/3983669433 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc801b050 con 0x55a6fa43d590 2024-04-23T14:40:32.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefbe600640 1 -- 172.21.15.59:0/3983669433 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8025070 con 0x55a6fa43d590 2024-04-23T14:40:32.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.766+0000 7fefbe600640 1 -- 172.21.15.59:0/3983669433 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8020640 con 0x55a6fa43d590 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 -- 172.21.15.59:0/3983669433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 msgr2=0x55a6fa414ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa414ea0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fefc8019040 tx=0x7fefd00b4ec0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 -- 172.21.15.59:0/3983669433 shutdown_connections 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa43cc70 0x55a6fa431a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa414ea0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 --2- 172.21.15.59:0/3983669433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa433580 0x55a6fa414140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 -- 172.21.15.59:0/3983669433 >> 172.21.15.59:0/3983669433 conn(0x55a6fa414980 msgr2=0x55a6fa40f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 -- 172.21.15.59:0/3983669433 shutdown_connections 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.767+0000 7fefd8412740 1 -- 172.21.15.59:0/3983669433 wait complete. 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.768+0000 7fefd8412740 1 -- start start 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.768+0000 7fefd8412740 1 Processor -- start 2024-04-23T14:40:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 0x55a6fa415440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefd8412740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa45a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefd8412740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa433e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefd8412740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a6fa4108d0 con 0x55a6fa43d590 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6fa334590 con 0x55a6fa43cc70 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefd8412740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55a6fa472300 con 0x55a6fa433580 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefd8412740 10 client.0 _ll_register_callbacks cb 0x7fefb8079c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa433e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa433e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51498/0 (socket says 172.21.15.59:51498) 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.769+0000 7fefcf400640 1 -- 172.21.15.59:0/3045237145 learned_addr learned my addr 172.21.15.59:0/3045237145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:32.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefcf400640 1 -- 172.21.15.59:0/3045237145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 msgr2=0x55a6fa415440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefcf400640 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 0x55a6fa415440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefcf400640 1 -- 172.21.15.59:0/3045237145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 msgr2=0x55a6fa45a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefce000640 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa45a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefcf400640 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa45a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefcf400640 1 -- 172.21.15.59:0/3045237145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fa484220 con 0x55a6fa43d590 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefcf400640 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa433e00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fefc8016000 tx=0x7fefc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7626eff2940c3509 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefb3400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefb3400640 1 -- 172.21.15.59:0/3045237145 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc801b040 con 0x55a6fa43d590 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefb3400640 1 -- 172.21.15.59:0/3045237145 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fefc8020de0 con 0x55a6fa43d590 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.770+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefc8025070 con 0x55a6fa43d590 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefb3400640 1 -- 172.21.15.59:0/3045237145 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fefc8038050 con 0x55a6fa43d590 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 10 client.8214 fetch_fsmap learned FSMap version 74 2024-04-23T14:40:32.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a6fa4512b0 con 0x55a6fa43d590 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefb3400640 1 -- 172.21.15.59:0/3045237145 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8025070 con 0x55a6fa43d590 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefb3400640 1 -- 172.21.15.59:0/3045237145 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fefc803b020 con 0x55a6fa43d590 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 10 client.8214 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 10 client.8214 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 20 client.8214 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa482f40 0x55a6fa483320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 4 client.8214 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.771+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "status"}) -- 0x55a6fa3823a0 con 0x55a6fa482f40 2024-04-23T14:40:32.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.772+0000 7fefcea00640 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa482f40 0x55a6fa483320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.772+0000 7fefcea00640 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa482f40 0x55a6fa483320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba82b5e8cce19cdb server_cookie=5584984492406f9e in_seq=0 out_seq=0 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.772+0000 7fefb3400640 10 client.8214 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.772+0000 7fefb3400640 1 -- 172.21.15.59:0/3045237145 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a6fa3823a0 con 0x55a6fa482f40 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.772+0000 7fefb3400640 10 client.8214 handle_command_reply: tid=0 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.772+0000 7fefd8412740 1 client.8214 shutdown 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 msgr2=0x55a6fa433e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa433e00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fefc8016000 tx=0x7fefc8004a40 comp rx=0 tx=0).stop 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 shutdown_connections 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55a6fa482f40 0x55a6fa483320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a6fa43d590 0x55a6fa433e00 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6fa43cc70 0x55a6fa45a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 --2- 172.21.15.59:0/3045237145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55a6fa433580 0x55a6fa415440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 >> 172.21.15.59:0/3045237145 conn(0x55a6fa414980 msgr2=0x55a6fa431a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefb3400640 0 client.8214 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 shutdown_connections 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 1 -- 172.21.15.59:0/3045237145 wait complete. 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.773+0000 7fefd8412740 20 client.8214 trim_cache size 0 max 16384 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:40:32.779 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8133, 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0019836419999999, 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 74, 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 87, 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 6.9245401290000004 2024-04-23T14:40:32.780 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefac06c920 msgr2=0x7fefac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefac06c920 0x7fefac06ede0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fefc00079d0 tx=0x7fefc0009040 comp rx=0 tx=0).stop 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd0157f60 msgr2=0x7fefd01583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd0157f60 0x7fefd01583e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009be0 tx=0x7fefb8004730 comp rx=0 tx=0).stop 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 shutdown_connections 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fefac06c920 0x7fefac06ede0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefd00b0330 0x7fefd00b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefd0157f60 0x7fefd01583e0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 --2- 172.21.15.59:0/2140076532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefb4003f80 0x7fefd0099ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 >> 172.21.15.59:0/2140076532 conn(0x7fefd006bdf0 msgr2=0x7fefd015cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 shutdown_connections 2024-04-23T14:40:32.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.774+0000 7fefcfe00640 1 -- 172.21.15.59:0/2140076532 wait complete. 2024-04-23T14:40:32.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:40:32.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.989+0000 7fc2ae800640 1 Processor -- start 2024-04-23T14:40:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.990+0000 7fc2ae800640 1 -- start start 2024-04-23T14:40:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.991+0000 7fc2ae800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc2a80cd760 0x7fc2a80b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.991+0000 7fc2ae800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a8160210 con 0x7fc2a815c3d0 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.991+0000 7fc2ae800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a81604f0 con 0x7fc2a80b4690 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.991+0000 7fc2ae800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a81607d0 con 0x7fc2a80cd760 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.991+0000 7fc2ad400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc2a815c3d0 0x7fc2a80cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35324/0 (socket says 172.21.15.59:35324) 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.991+0000 7fc2ad400640 1 -- 172.21.15.59:0/3397753420 learned_addr learned my addr 172.21.15.59:0/3397753420 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.991+0000 7fc2aca00640 1 --2- 172.21.15.59:0/3397753420 >> v2:172.21.15.59:3300/0 conn(0x7fc2a80cd760 0x7fc2a80b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.992+0000 7fc2a7e00640 1 -- 172.21.15.59:0/3397753420 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200294173 0 0) 0x7fc2a8160210 con 0x7fc2a815c3d0 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.992+0000 7fc2a7e00640 1 -- 172.21.15.59:0/3397753420 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc290003520 con 0x7fc2a815c3d0 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.992+0000 7fc2aca00640 1 -- 172.21.15.59:0/3397753420 >> v1:172.21.15.76:6789/0 conn(0x7fc2a80b4690 legacy=0x7fc2a80cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.992+0000 7fc2aca00640 1 -- 172.21.15.59:0/3397753420 >> v1:172.21.15.59:6789/0 conn(0x7fc2a815c3d0 legacy=0x7fc2a80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.992+0000 7fc2aca00640 1 -- 172.21.15.59:0/3397753420 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a8160ab0 con 0x7fc2a80cd760 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.993+0000 7fc2aca00640 1 --2- 172.21.15.59:0/3397753420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a80cd760 0x7fc2a80b4150 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc2940097a0 tx=0x7fc29402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d83d45f9dc8f417 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.993+0000 7fc2a7e00640 1 -- 172.21.15.59:0/3397753420 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294037070 con 0x7fc2a80cd760 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.993+0000 7fc2a7e00640 1 -- 172.21.15.59:0/3397753420 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc294032440 con 0x7fc2a80cd760 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.993+0000 7fc2a7e00640 1 -- 172.21.15.59:0/3397753420 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2940327f0 con 0x7fc2a80cd760 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.994+0000 7fc2ae800640 1 -- 172.21.15.59:0/3397753420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a80cd760 msgr2=0x7fc2a80b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.994+0000 7fc2ae800640 1 --2- 172.21.15.59:0/3397753420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a80cd760 0x7fc2a80b4150 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc2940097a0 tx=0x7fc29402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.994+0000 7fc2ae800640 1 -- 172.21.15.59:0/3397753420 shutdown_connections 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.994+0000 7fc2ae800640 1 --2- 172.21.15.59:0/3397753420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a80cd760 0x7fc2a80b4150 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.994+0000 7fc2ae800640 1 -- 172.21.15.59:0/3397753420 >> 172.21.15.59:0/3397753420 conn(0x7fc2a80b21a0 msgr2=0x7fc2a80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.994+0000 7fc2ae800640 1 -- 172.21.15.59:0/3397753420 shutdown_connections 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.994+0000 7fc2ae800640 1 -- 172.21.15.59:0/3397753420 wait complete. 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.996+0000 7fc2ae800640 1 Processor -- start 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.996+0000 7fc2ae800640 1 -- start start 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.997+0000 7fc2ae800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2a80b4690 0x7fc2a815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.997+0000 7fc2ae800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2a80cd760 0x7fc2a80c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.997+0000 7fc2ae800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a815c3d0 0x7fc2a8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.997+0000 7fc2ae800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a815c7b0 con 0x7fc2a815c3d0 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.997+0000 7fc2ae800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80b4ae0 con 0x7fc2a80cd760 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.997+0000 7fc2ae800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a815c930 con 0x7fc2a80b4690 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a815c3d0 0x7fc2a8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a815c3d0 0x7fc2a8157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51518/0 (socket says 172.21.15.59:51518) 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 -- 172.21.15.59:0/1317553111 learned_addr learned my addr 172.21.15.59:0/1317553111 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2aca00640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2a80cd760 0x7fc2a80c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 -- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2a80b4690 msgr2=0x7fc2a815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ad400640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2a80b4690 0x7fc2a815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2a80b4690 0x7fc2a815f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 -- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2a80cd760 msgr2=0x7fc2a80c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2a80cd760 0x7fc2a80c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 -- 172.21.15.59:0/1317553111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a815d640 con 0x7fc2a815c3d0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2aca00640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2a80cd760 0x7fc2a80c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ad400640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2a80b4690 0x7fc2a815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.998+0000 7fc2ade00640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a815c3d0 0x7fc2a8157c20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c780 tx=0x7fc2a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.999+0000 7fc2a5600640 1 -- 172.21.15.59:0/1317553111 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0004b60 con 0x7fc2a815c3d0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.999+0000 7fc2a5600640 1 -- 172.21.15.59:0/1317553111 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a0004d00 con 0x7fc2a815c3d0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.999+0000 7fc2a5600640 1 -- 172.21.15.59:0/1317553111 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000aa30 con 0x7fc2a815c3d0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.999+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a815d900 con 0x7fc2a815c3d0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:32.999+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a815dec0 con 0x7fc2a815c3d0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.000+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a804fd90 con 0x7fc2a815c3d0 2024-04-23T14:40:33.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.001+0000 7fc2a5600640 1 -- 172.21.15.59:0/1317553111 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2a000abd0 con 0x7fc2a815c3d0 2024-04-23T14:40:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.002+0000 7fc2a5600640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc28806c8f0 0x7fc28806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.002+0000 7fc2a5600640 1 -- 172.21.15.59:0/1317553111 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fc2a008bef0 con 0x7fc2a815c3d0 2024-04-23T14:40:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.002+0000 7fc2ad400640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc28806c8f0 0x7fc28806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.003+0000 7fc2ad400640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc28806c8f0 0x7fc28806edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005880 tx=0x7fc29c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:33.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.007+0000 7fc2a5600640 1 -- 172.21.15.59:0/1317553111 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2a0020070 con 0x7fc2a815c3d0 2024-04-23T14:40:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.250+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2a806e400 con 0x7fc2a815c3d0 2024-04-23T14:40:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.251+0000 7fc2a5600640 1 -- 172.21.15.59:0/1317553111 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7fc2a005a1c0 con 0x7fc2a815c3d0 2024-04-23T14:40:33.252 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:40:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc28806c8f0 msgr2=0x7fc28806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc28806c8f0 0x7fc28806edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005880 tx=0x7fc29c009040 comp rx=0 tx=0).stop 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a815c3d0 msgr2=0x7fc2a8157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a815c3d0 0x7fc2a8157c20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c780 tx=0x7fc2a000cc40 comp rx=0 tx=0).stop 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 shutdown_connections 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc28806c8f0 0x7fc28806edb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2a815c3d0 0x7fc2a8157c20 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2a80cd760 0x7fc2a80c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 --2- 172.21.15.59:0/1317553111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc2a80b4690 0x7fc2a815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 >> 172.21.15.59:0/1317553111 conn(0x7fc2a80b21a0 msgr2=0x7fc2a8073500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 shutdown_connections 2024-04-23T14:40:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33.254+0000 7fc2ae800640 1 -- 172.21.15.59:0/1317553111 wait complete. 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:40:33.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:40:33.275 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:40:33.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:33.275 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:40:33.327 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:40:33.328 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41915sec preferred_lft 41915sec 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:40:33.329 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:40:33.330 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:40:33.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:33.331 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:40:33.410 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:40:33.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:33.415 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:40:33.471 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:40:33.471 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:40:33.471 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:40:33.471 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:40:33.471 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:40:33.472 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:40:33.472 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:40:33.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:40:33.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:40:33.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:40:33.472 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:40:33.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:40:33.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:40:33.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:33.702 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:40:33.753 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:40:33.754 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:40:33.754 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:40:33.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:33.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:40:33.843 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:40:33.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:40:33.843 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:40:33.898 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:40:33.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:40:33.899 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:40:33.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:40:34.036 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:40:34.091 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:40:34.091 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:40:34.091 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:40:34.091 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:40:34.091 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:40:34.092 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:40:34.093 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:40:34.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:34.094 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:40:34.146 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:40:34.147 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:40:34.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:40:34.222 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:40:34.267 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:40:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:40:34.268 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:40:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:40:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:40:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:40:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:40:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:40:34.269 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:40:34.270 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:40:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:40:34.271 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:40:34.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:34.272 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:40:34.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:40:34.322+0000 7fb4c9a5f480 -1 init, newargv = 0x55bcdb6fe1e0 newargc=15 2024-04-23T14:40:34.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[67104]: starting ceph client 2024-04-23T14:40:34.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[67104]: starting fuse 2024-04-23T14:40:34.354 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:40:34.354 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:40:34.354 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:40:34.354 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:40:34.355 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:40:34.356 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:40:34.551 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.67104.asok 2024-04-23T14:40:34.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:40:34.568 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67104.asok 2024-04-23T14:40:34.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:34.568 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67104.asok status 2024-04-23T14:40:34.807 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "67104", 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8232, 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:40:34.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 8232 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4224688290 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4224688290 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:40:34.809 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.8232 192.168.0.1:0/4224688290", 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/4224688290", 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 74, 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 87, 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:40:34.810 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:40:34.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:34.823 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:40:34.823 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:40:34.878 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:40:34.878 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:40:34.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:34.878 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:40:34.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:34.960 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-23T14:40:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.181+0000 7f3537200640 1 Processor -- start 2024-04-23T14:40:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.181+0000 7f3537200640 1 -- start start 2024-04-23T14:40:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.182+0000 7f3537200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35300cec40 0x7f35300d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.182+0000 7f3537200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35300cfc70 con 0x7f35300d1710 2024-04-23T14:40:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.182+0000 7f3537200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35300cff50 con 0x7f353015e990 2024-04-23T14:40:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.182+0000 7f3537200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f35300d0230 con 0x7f35300cec40 2024-04-23T14:40:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.183+0000 7f3535400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f35300d1710 0x7f35300d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:35334/0 (socket says 172.21.15.59:35334) 2024-04-23T14:40:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.183+0000 7f3535400640 1 -- 172.21.15.59:0/2654786013 learned_addr learned my addr 172.21.15.59:0/2654786013 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:40:35.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.183+0000 7f3536800640 1 --2- 172.21.15.59:0/2654786013 >> v2:172.21.15.59:3300/0 conn(0x7f35300cec40 0x7f35300d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.183+0000 7f3536800640 1 -- 172.21.15.59:0/2654786013 >> v1:172.21.15.76:6789/0 conn(0x7f353015e990 legacy=0x7f35300d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.183+0000 7f3536800640 1 -- 172.21.15.59:0/2654786013 >> v1:172.21.15.59:6789/0 conn(0x7f35300d1710 legacy=0x7f35300d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.183+0000 7f3534a00640 1 -- 172.21.15.59:0/2654786013 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656068460 0 0) 0x7f35300cfc70 con 0x7f35300d1710 2024-04-23T14:40:35.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.183+0000 7f3536800640 1 -- 172.21.15.59:0/2654786013 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35300d2410 con 0x7f35300cec40 2024-04-23T14:40:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.184+0000 7f3536800640 1 --2- 172.21.15.59:0/2654786013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35300cec40 0x7f35300d3320 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f352c0097a0 tx=0x7f352c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e00aec505a81da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3534a00640 1 -- 172.21.15.59:0/2654786013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c037070 con 0x7f35300cec40 2024-04-23T14:40:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3534a00640 1 -- 172.21.15.59:0/2654786013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c032440 con 0x7f35300cec40 2024-04-23T14:40:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3534a00640 1 -- 172.21.15.59:0/2654786013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c032810 con 0x7f35300cec40 2024-04-23T14:40:35.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3537200640 1 -- 172.21.15.59:0/2654786013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35300cec40 msgr2=0x7f35300d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3537200640 1 --2- 172.21.15.59:0/2654786013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35300cec40 0x7f35300d3320 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f352c0097a0 tx=0x7f352c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3537200640 1 -- 172.21.15.59:0/2654786013 shutdown_connections 2024-04-23T14:40:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3537200640 1 --2- 172.21.15.59:0/2654786013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35300cec40 0x7f35300d3320 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.185+0000 7f3537200640 1 -- 172.21.15.59:0/2654786013 >> 172.21.15.59:0/2654786013 conn(0x7f353008bf90 msgr2=0x7f35300d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.186+0000 7f3537200640 1 -- 172.21.15.59:0/2654786013 shutdown_connections 2024-04-23T14:40:35.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.186+0000 7f3537200640 1 -- 172.21.15.59:0/2654786013 wait complete. 2024-04-23T14:40:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.187+0000 7f3537200640 1 Processor -- start 2024-04-23T14:40:35.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.188+0000 7f3537200640 1 -- start start 2024-04-23T14:40:35.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.188+0000 7f3537200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35300cec40 0x7f3530156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.188+0000 7f3537200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35300d1710 0x7f35300c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3537200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f353015e990 0x7f35300c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3537200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f353006e1f0 con 0x7f353015e990 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3537200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35300d21c0 con 0x7f35300cec40 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3537200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f35300c8ba0 con 0x7f35300d1710 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f353015e990 0x7f35300c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f353015e990 0x7f35300c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51540/0 (socket says 172.21.15.59:51540) 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 -- 172.21.15.59:0/3006636300 learned_addr learned my addr 172.21.15.59:0/3006636300 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3535e00640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35300cec40 0x7f3530156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3535400640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35300d1710 0x7f35300c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 -- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35300d1710 msgr2=0x7f35300c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35300d1710 0x7f35300c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 -- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35300cec40 msgr2=0x7f3530156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35300cec40 0x7f3530156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3536800640 1 -- 172.21.15.59:0/3006636300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35300c8e80 con 0x7f353015e990 2024-04-23T14:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.189+0000 7f3535e00640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35300cec40 0x7f3530156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.190+0000 7f3536800640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f353015e990 0x7f35300c9ee0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f352c0097a0 tx=0x7f352c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.190+0000 7f3526000640 1 -- 172.21.15.59:0/3006636300 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c041070 con 0x7f353015e990 2024-04-23T14:40:35.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.190+0000 7f3526000640 1 -- 172.21.15.59:0/3006636300 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c03c870 con 0x7f353015e990 2024-04-23T14:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.190+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35300c9160 con 0x7f353015e990 2024-04-23T14:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.190+0000 7f3526000640 1 -- 172.21.15.59:0/3006636300 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c037040 con 0x7f353015e990 2024-04-23T14:40:35.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.190+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3530157380 con 0x7f353015e990 2024-04-23T14:40:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.193+0000 7f3526000640 1 -- 172.21.15.59:0/3006636300 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f352c044070 con 0x7f353015e990 2024-04-23T14:40:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.193+0000 7f3526000640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f351406c8f0 0x7f351406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.194+0000 7f3526000640 1 -- 172.21.15.59:0/3006636300 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f352c0ac7b0 con 0x7f353015e990 2024-04-23T14:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.194+0000 7f3535e00640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f351406c8f0 0x7f351406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.194+0000 7f353b786740 1 Processor -- start 2024-04-23T14:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.194+0000 7f3535e00640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f351406c8f0 0x7f351406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3528007e90 tx=0x7f35280073a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.195+0000 7f353b786740 1 -- start start 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.195+0000 7f353b786740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93acb3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.195+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93ace3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.195+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4020 0x55c93acb8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.195+0000 7f353b786740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c93ac1de70 con 0x55c93ad01640 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.195+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c93abc4380 con 0x55c93ace4020 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.195+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c93ac19e30 con 0x55c93ace4760 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93acb3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93acb3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51542/0 (socket says 172.21.15.59:51542) 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 -- 172.21.15.59:0/1889574647 learned_addr learned my addr 172.21.15.59:0/1889574647 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3536800640 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4020 0x55c93acb8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 -- 172.21.15.59:0/1889574647 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 msgr2=0x55c93ace3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93ace3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 -- 172.21.15.59:0/1889574647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4020 msgr2=0x55c93acb8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4020 0x55c93acb8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 -- 172.21.15.59:0/1889574647 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93a9f87e0 con 0x55c93ad01640 2024-04-23T14:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.196+0000 7f3535e00640 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93acb3fd0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f352800f610 tx=0x7f352800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8137dbc86f9f2fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f351a600640 1 -- 172.21.15.59:0/1889574647 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528003970 con 0x55c93ad01640 2024-04-23T14:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f351a600640 1 -- 172.21.15.59:0/1889574647 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528015030 con 0x55c93ad01640 2024-04-23T14:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f351a600640 1 -- 172.21.15.59:0/1889574647 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528003d50 con 0x55c93ad01640 2024-04-23T14:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 -- 172.21.15.59:0/1889574647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 msgr2=0x55c93acb3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93acb3fd0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f352800f610 tx=0x7f352800f000 comp rx=0 tx=0).stop 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 -- 172.21.15.59:0/1889574647 shutdown_connections 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93ace3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4020 0x55c93acb8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 --2- 172.21.15.59:0/1889574647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93acb3fd0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 -- 172.21.15.59:0/1889574647 >> 172.21.15.59:0/1889574647 conn(0x55c93acb6240 msgr2=0x55c93acfeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.197+0000 7f353b786740 1 -- 172.21.15.59:0/1889574647 shutdown_connections 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.198+0000 7f353b786740 1 -- 172.21.15.59:0/1889574647 wait complete. 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.198+0000 7f353b786740 1 -- start start 2024-04-23T14:40:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.198+0000 7f353b786740 1 Processor -- start 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.199+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93acdf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.199+0000 7f353b786740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93ace66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.199+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93acb9840 0x55c93acb9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.199+0000 7f353b786740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c93acb83f0 con 0x55c93ad01640 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.199+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c93abd7660 con 0x55c93acb9840 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.199+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c93acb77e0 con 0x55c93ace4760 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f353b786740 10 client.0 _ll_register_callbacks cb 0x55c93abc5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93ace66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93ace66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51546/0 (socket says 172.21.15.59:51546) 2024-04-23T14:40:35.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 -- 172.21.15.59:0/1339860893 learned_addr learned my addr 172.21.15.59:0/1339860893 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3536800640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93acb9840 0x55c93acb9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535e00640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93acdf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 -- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 msgr2=0x55c93acdf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93acdf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 -- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93acb9840 msgr2=0x55c93acb9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93acb9840 0x55c93acb9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535400640 1 -- 172.21.15.59:0/1339860893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93aa58f70 con 0x55c93ad01640 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.200+0000 7f3535e00640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93acdf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f3535400640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93ace66c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3520007e20 tx=0x7f352000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c5b949eefb3402 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f350ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f350ea00640 1 -- 172.21.15.59:0/1339860893 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520018060 con 0x55c93ad01640 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f350ea00640 1 -- 172.21.15.59:0/1339860893 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f35200072d0 con 0x55c93ad01640 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528015030 con 0x55c93ad01640 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f350ea00640 1 -- 172.21.15.59:0/1339860893 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f352001e050 con 0x55c93ad01640 2024-04-23T14:40:35.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f353b786740 10 client.8247 fetch_fsmap learned FSMap version 74 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55c93a8f0170 con 0x55c93ad01640 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.201+0000 7f350ea00640 1 -- 172.21.15.59:0/1339860893 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528015030 con 0x55c93ad01640 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f350ea00640 1 -- 172.21.15.59:0/1339860893 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f3520021020 con 0x55c93ad01640 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f353b786740 10 client.8247 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f353b786740 10 client.8247 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f353b786740 20 client.8247 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f353b786740 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad0f550 0x55c93ad0f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f353b786740 4 client.8247 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c93ac2b890 con 0x55c93ad0f550 2024-04-23T14:40:35.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.202+0000 7f3535e00640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad0f550 0x55c93ad0f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.203+0000 7f3535e00640 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad0f550 0x55c93ad0f930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6021dab01cb2f1ba server_cookie=f4e1af5e45fdda24 in_seq=0 out_seq=0 2024-04-23T14:40:35.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.203+0000 7f350ea00640 10 client.8247 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f350ea00640 1 -- 172.21.15.59:0/1339860893 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c93ac2b890 con 0x55c93ad0f550 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f350ea00640 10 client.8247 handle_command_reply: tid=0 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 client.8247 shutdown 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 msgr2=0x55c93ace66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93ace66c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3520007e20 tx=0x7f352000d520 comp rx=0 tx=0).stop 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 shutdown_connections 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad0f550 0x55c93ad0f930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93acb9840 0x55c93acb9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93ad01640 0x55c93ace66c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 --2- 172.21.15.59:0/1339860893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ace4760 0x55c93acdf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f350ea00640 0 client.8247 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 >> 172.21.15.59:0/1339860893 conn(0x55c93ace4020 msgr2=0x55c93acffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 shutdown_connections 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 1 -- 172.21.15.59:0/1339860893 wait complete. 2024-04-23T14:40:35.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.204+0000 7f353b786740 20 client.8247 trim_cache size 0 max 16384 2024-04-23T14:40:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.224+0000 7f353b786740 1 Processor -- start 2024-04-23T14:40:35.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.224+0000 7f353b786740 1 -- start start 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f353b786740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93acb90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 0x55c93ad0fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f353b786740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3530200d30 con 0x55c93acb9840 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35301ffd50 con 0x55c93ace4760 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3530154de0 con 0x55c93ad01640 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93acb90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93acb90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51554/0 (socket says 172.21.15.59:51554) 2024-04-23T14:40:35.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 -- 172.21.15.59:0/1190216997 learned_addr learned my addr 172.21.15.59:0/1190216997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 -- 172.21.15.59:0/1190216997 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 msgr2=0x55c93ad0fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 --2- 172.21.15.59:0/1190216997 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 0x55c93ad0fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 -- 172.21.15.59:0/1190216997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 msgr2=0x55c93ad0f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 --2- 172.21.15.59:0/1190216997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.225+0000 7f3535e00640 1 -- 172.21.15.59:0/1190216997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93acba030 con 0x55c93acb9840 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.226+0000 7f3535e00640 1 --2- 172.21.15.59:0/1190216997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93acb90e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f352800cb00 tx=0x7f3528011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac368698aea5af6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.226+0000 7f3519c00640 1 -- 172.21.15.59:0/1190216997 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528030bc0 con 0x55c93acb9840 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.226+0000 7f3519c00640 1 -- 172.21.15.59:0/1190216997 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528036070 con 0x55c93acb9840 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.226+0000 7f3519c00640 1 -- 172.21.15.59:0/1190216997 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528030330 con 0x55c93acb9840 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 -- 172.21.15.59:0/1190216997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 msgr2=0x55c93acb90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 --2- 172.21.15.59:0/1190216997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93acb90e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f352800cb00 tx=0x7f3528011000 comp rx=0 tx=0).stop 2024-04-23T14:40:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 -- 172.21.15.59:0/1190216997 shutdown_connections 2024-04-23T14:40:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 --2- 172.21.15.59:0/1190216997 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 0x55c93ad0fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 --2- 172.21.15.59:0/1190216997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 --2- 172.21.15.59:0/1190216997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93acb90e0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 -- 172.21.15.59:0/1190216997 >> 172.21.15.59:0/1190216997 conn(0x55c93ace4020 msgr2=0x55c93acb2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 -- 172.21.15.59:0/1190216997 shutdown_connections 2024-04-23T14:40:35.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 -- 172.21.15.59:0/1190216997 wait complete. 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.227+0000 7f353b786740 1 -- start start 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.228+0000 7f353b786740 1 Processor -- start 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.228+0000 7f353b786740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93ad7be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f353b786740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 0x55c93acff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f353b786740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3530155430 con 0x55c93acb9840 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c93aba5ec0 con 0x55c93ace4760 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f353b786740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c93ad10aa0 con 0x55c93ad01640 2024-04-23T14:40:35.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f353b786740 10 client.0 _ll_register_callbacks cb 0x7f35200079a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93ad7be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93ad7be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51556/0 (socket says 172.21.15.59:51556) 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 -- 172.21.15.59:0/4175001718 learned_addr learned my addr 172.21.15.59:0/4175001718 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535400640 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 -- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 msgr2=0x55c93acff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 0x55c93acff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 -- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 msgr2=0x55c93ad0fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535e00640 1 -- 172.21.15.59:0/4175001718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93acb7380 con 0x55c93acb9840 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.229+0000 7f3535400640 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f3535e00640 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93ad7be80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f352800f5e0 tx=0x7f3528004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49a9599af3f80e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f350ea00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f350ea00640 1 -- 172.21.15.59:0/4175001718 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352800e850 con 0x55c93acb9840 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f350ea00640 1 -- 172.21.15.59:0/4175001718 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f3528043b80 con 0x55c93acb9840 2024-04-23T14:40:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528036070 con 0x55c93acb9840 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f350ea00640 1 -- 172.21.15.59:0/4175001718 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f3528043d80 con 0x55c93acb9840 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f353b786740 10 client.8253 fetch_fsmap learned FSMap version 74 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55c93ad10530 con 0x55c93acb9840 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f350ea00640 1 -- 172.21.15.59:0/4175001718 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528036070 con 0x55c93acb9840 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.230+0000 7f350ea00640 1 -- 172.21.15.59:0/4175001718 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f352800e9f0 con 0x55c93acb9840 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f353b786740 10 client.8253 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f353b786740 10 client.8253 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f353b786740 20 client.8253 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f353b786740 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad18e80 0x55c93acdf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f353b786740 4 client.8253 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55c93ace5350 con 0x55c93ad18e80 2024-04-23T14:40:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f3535400640 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad18e80 0x55c93acdf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f3535400640 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad18e80 0x55c93acdf570 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53742ce2128dfb87 server_cookie=cea984a0fbe21e03 in_seq=0 out_seq=0 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.231+0000 7f350ea00640 10 client.8253 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f350ea00640 1 -- 172.21.15.59:0/4175001718 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c93ace5350 con 0x55c93ad18e80 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f350ea00640 10 client.8253 handle_command_reply: tid=0 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 client.8253 shutdown 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 msgr2=0x55c93ad7be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93ad7be80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f352800f5e0 tx=0x7f3528004a00 comp rx=0 tx=0).stop 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 shutdown_connections 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c93ad18e80 0x55c93acdf570 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c93ad01640 0x55c93acff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c93ace4760 0x55c93ad0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 --2- 172.21.15.59:0/4175001718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c93acb9840 0x55c93ad7be80 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 >> 172.21.15.59:0/4175001718 conn(0x55c93ace4020 msgr2=0x55c93ad0e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.232+0000 7f350ea00640 0 client.8253 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.233+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 shutdown_connections 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.233+0000 7f353b786740 1 -- 172.21.15.59:0/4175001718 wait complete. 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.233+0000 7f353b786740 20 client.8253 trim_cache size 0 max 16384 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "8e510cc2-9e8f-4094-bd67-c3cda70b5260", 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:40:35.237 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:40:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.233+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f351406c8f0 msgr2=0x7f351406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.233+0000 7f3537200640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f351406c8f0 0x7f351406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3528007e90 tx=0x7f35280073a0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.233+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f353015e990 msgr2=0x7f35300c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.233+0000 7f3537200640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f353015e990 0x7f35300c9ee0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f352c0097a0 tx=0x7f352c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 shutdown_connections 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f351406c8f0 0x7f351406edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35300d1710 0x7f35300c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f353015e990 0x7f35300c9ee0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 --2- 172.21.15.59:0/3006636300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35300cec40 0x7f3530156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 >> 172.21.15.59:0/3006636300 conn(0x7f353008bf90 msgr2=0x7f35300d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 shutdown_connections 2024-04-23T14:40:35.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3537200640 1 -- 172.21.15.59:0/3006636300 wait complete. 2024-04-23T14:40:35.239 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:40:35.234+0000 7f3593e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x560c23380680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:40:35.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:40:35.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-23T14:40:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.457+0000 7f4d3cc00640 1 Processor -- start 2024-04-23T14:40:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.457+0000 7f4d3cc00640 1 -- start start 2024-04-23T14:40:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d3cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3815eb60 0x7f4d380d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d3cc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d380d2d60 0x7f4d380b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d3cc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d380b3490 0x7f4d380b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d3cc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4d380ba7a0 con 0x7f4d380b3490 2024-04-23T14:40:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d3cc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4d380ba1e0 con 0x7f4d380d2d60 2024-04-23T14:40:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d3cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4d380ba4c0 con 0x7f4d3815eb60 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d36800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d380b3490 0x7f4d380b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d36800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d380b3490 0x7f4d380b97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51570/0 (socket says 172.21.15.59:51570) 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.458+0000 7f4d36800640 1 -- 172.21.15.59:0/4016074443 learned_addr learned my addr 172.21.15.59:0/4016074443 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.459+0000 7f4d35400640 1 --2- 172.21.15.59:0/4016074443 >> v2:172.21.15.76:3300/0 conn(0x7f4d380d2d60 0x7f4d380b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.459+0000 7f4d35e00640 1 --2- 172.21.15.59:0/4016074443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3815eb60 0x7f4d380d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.459+0000 7f4d36800640 1 -- 172.21.15.59:0/4016074443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3815eb60 msgr2=0x7f4d380d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.459+0000 7f4d36800640 1 --2- 172.21.15.59:0/4016074443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3815eb60 0x7f4d380d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.459+0000 7f4d36800640 1 -- 172.21.15.59:0/4016074443 >> v2:172.21.15.76:3300/0 conn(0x7f4d380d2d60 msgr2=0x7f4d380b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.459+0000 7f4d36800640 1 --2- 172.21.15.59:0/4016074443 >> v2:172.21.15.76:3300/0 conn(0x7f4d380d2d60 0x7f4d380b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.459+0000 7f4d36800640 1 -- 172.21.15.59:0/4016074443 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d380b8a90 con 0x7f4d380b3490 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.460+0000 7f4d36800640 1 --2- 172.21.15.59:0/4016074443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d380b3490 0x7f4d380b97d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f4d3806dce0 tx=0x7f4d3002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9f66eb21bea29d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.460+0000 7f4d34a00640 1 -- 172.21.15.59:0/4016074443 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30036070 con 0x7f4d380b3490 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.460+0000 7f4d34a00640 1 -- 172.21.15.59:0/4016074443 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d30031600 con 0x7f4d380b3490 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.460+0000 7f4d34a00640 1 -- 172.21.15.59:0/4016074443 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d300319d0 con 0x7f4d380b3490 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 -- 172.21.15.59:0/4016074443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d380b3490 msgr2=0x7f4d380b97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/4016074443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d380b3490 0x7f4d380b97d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f4d3806dce0 tx=0x7f4d3002a6a0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 -- 172.21.15.59:0/4016074443 shutdown_connections 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/4016074443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d380b3490 0x7f4d380b97d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/4016074443 >> v2:172.21.15.76:3300/0 conn(0x7f4d380d2d60 0x7f4d380b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/4016074443 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3815eb60 0x7f4d380d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 -- 172.21.15.59:0/4016074443 >> 172.21.15.59:0/4016074443 conn(0x7f4d38067b50 msgr2=0x7f4d38067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 -- 172.21.15.59:0/4016074443 shutdown_connections 2024-04-23T14:40:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.461+0000 7f4d3cc00640 1 -- 172.21.15.59:0/4016074443 wait complete. 2024-04-23T14:40:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.463+0000 7f4d3cc00640 1 Processor -- start 2024-04-23T14:40:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.463+0000 7f4d3cc00640 1 -- start start 2024-04-23T14:40:35.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.464+0000 7f4d3cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d380b3490 0x7f4d38148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.464+0000 7f4d3cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3815eb60 0x7f4d380d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.464+0000 7f4d3cc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d380d46e0 0x7f4d380cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.464+0000 7f4d3cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d380b6280 con 0x7f4d3815eb60 2024-04-23T14:40:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.464+0000 7f4d3cc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d380b3c80 con 0x7f4d380d46e0 2024-04-23T14:40:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.464+0000 7f4d3cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4d380b6400 con 0x7f4d380b3490 2024-04-23T14:40:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.464+0000 7f4d35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3815eb60 0x7f4d380d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3815eb60 0x7f4d380d41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51584/0 (socket says 172.21.15.59:51584) 2024-04-23T14:40:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 -- 172.21.15.59:0/896383363 learned_addr learned my addr 172.21.15.59:0/896383363 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d36800640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d380d46e0 0x7f4d380cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35e00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d380b3490 0x7f4d38148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 -- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d380b3490 msgr2=0x7f4d38148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d380b3490 0x7f4d38148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 -- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d380d46e0 msgr2=0x7f4d380cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d380d46e0 0x7f4d380cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 -- 172.21.15.59:0/896383363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d380ce060 con 0x7f4d3815eb60 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.465+0000 7f4d35400640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3815eb60 0x7f4d380d41a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000d830 tx=0x7f4d2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.466+0000 7f4d2e000640 1 -- 172.21.15.59:0/896383363 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000b6c0 con 0x7f4d3815eb60 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.466+0000 7f4d2e000640 1 -- 172.21.15.59:0/896383363 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d20019070 con 0x7f4d3815eb60 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.466+0000 7f4d2e000640 1 -- 172.21.15.59:0/896383363 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000ba80 con 0x7f4d3815eb60 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.466+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d380ce340 con 0x7f4d3815eb60 2024-04-23T14:40:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.466+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d380c8330 con 0x7f4d3815eb60 2024-04-23T14:40:35.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.469+0000 7f4d2e000640 1 -- 172.21.15.59:0/896383363 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d2001e460 con 0x7f4d3815eb60 2024-04-23T14:40:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.470+0000 7f4d2e000640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4d1406c920 0x7f4d1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.470+0000 7f4d2e000640 1 -- 172.21.15.59:0/896383363 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f4d2008c370 con 0x7f4d3815eb60 2024-04-23T14:40:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.470+0000 7f4d35e00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4d1406c920 0x7f4d1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.471+0000 7f4d3fa25740 1 Processor -- start 2024-04-23T14:40:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.471+0000 7f4d3fa25740 1 -- start start 2024-04-23T14:40:35.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.471+0000 7f4d35e00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4d1406c920 0x7f4d1406ede0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4d28006fd0 tx=0x7f4d28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c75040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7504610 0x5569c74d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5569c743e1f0 con 0x5569c7504d50 2024-04-23T14:40:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5569c73e4350 con 0x5569c7504610 2024-04-23T14:40:35.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5569c743a1b0 con 0x5569c7521c30 2024-04-23T14:40:35.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c75040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d35400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c75040d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51600/0 (socket says 172.21.15.59:51600) 2024-04-23T14:40:35.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.472+0000 7f4d35400640 1 -- 172.21.15.59:0/4259749073 learned_addr learned my addr 172.21.15.59:0/4259749073 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d36800640 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7504610 0x5569c74d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d35400640 1 -- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 msgr2=0x5569c74d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d35e00640 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d35400640 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d35400640 1 -- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7504610 msgr2=0x5569c74d9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d35400640 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7504610 0x5569c74d9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d35400640 1 -- 172.21.15.59:0/4259749073 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c7438f50 con 0x5569c7504d50 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d36800640 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7504610 0x5569c74d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d35400640 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c75040d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4d2008a670 tx=0x7f4d200599c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6915161a2a93324 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d26600640 1 -- 172.21.15.59:0/4259749073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20072020 con 0x5569c7504d50 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d26600640 1 -- 172.21.15.59:0/4259749073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2005e2e0 con 0x5569c7504d50 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.473+0000 7f4d26600640 1 -- 172.21.15.59:0/4259749073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20062b10 con 0x5569c7504d50 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4259749073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 msgr2=0x5569c75040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c75040d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4d2008a670 tx=0x7f4d200599c0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4259749073 shutdown_connections 2024-04-23T14:40:35.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7504610 0x5569c74d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c75040d0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4259749073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4259749073 >> 172.21.15.59:0/4259749073 conn(0x5569c74d6830 msgr2=0x5569c751f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4259749073 shutdown_connections 2024-04-23T14:40:35.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.474+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4259749073 wait complete. 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.475+0000 7f4d3fa25740 1 -- start start 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.475+0000 7f4d3fa25740 1 Processor -- start 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c74ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c752fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c75205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5569c74d89e0 con 0x5569c7504d50 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5569c73f7690 con 0x5569c7520120 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5569c74d7dd0 con 0x5569c7521c30 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d3fa25740 10 client.0 _ll_register_callbacks cb 0x5569c7433650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:35.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.476+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c74ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c74ffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51612/0 (socket says 172.21.15.59:51612) 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 -- 172.21.15.59:0/945063068 learned_addr learned my addr 172.21.15.59:0/945063068 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d36800640 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c75205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35400640 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c752fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 -- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 msgr2=0x5569c752fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c752fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 -- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 msgr2=0x5569c75205a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c75205a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 -- 172.21.15.59:0/945063068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c73f6920 con 0x5569c7504d50 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d35e00640 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c74ffb50 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800dfc0 tx=0x7f4d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f106aa1c94fd8b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.477+0000 7f4d1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d1aa00640 1 -- 172.21.15.59:0/945063068 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28023030 con 0x5569c7504d50 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d1aa00640 1 -- 172.21.15.59:0/945063068 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f4d2801f610 con 0x5569c7504d50 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d2005e2e0 con 0x5569c7504d50 2024-04-23T14:40:35.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d1aa00640 1 -- 172.21.15.59:0/945063068 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f4d2801f810 con 0x5569c7504d50 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d3fa25740 10 client.8265 fetch_fsmap learned FSMap version 74 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5569c721a7d0 con 0x5569c7504d50 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d1aa00640 1 -- 172.21.15.59:0/945063068 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2005e2e0 con 0x5569c7504d50 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d1aa00640 1 -- 172.21.15.59:0/945063068 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f4d28032020 con 0x5569c7504d50 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d3fa25740 10 client.8265 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d3fa25740 10 client.8265 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d3fa25740 20 client.8265 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.478+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7516650 0x5569c7532cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.479+0000 7f4d3fa25740 4 client.8265 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.479+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569c744bc10 con 0x5569c7516650 2024-04-23T14:40:35.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.479+0000 7f4d35400640 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7516650 0x5569c7532cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.479+0000 7f4d35400640 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7516650 0x5569c7532cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d5fa86090d0bf3c server_cookie=41a2a99f35d2aee2 in_seq=0 out_seq=0 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.479+0000 7f4d1aa00640 10 client.8265 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d1aa00640 1 -- 172.21.15.59:0/945063068 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569c744bc10 con 0x5569c7516650 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d1aa00640 10 client.8265 handle_command_reply: tid=0 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 client.8265 shutdown 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 msgr2=0x5569c74ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c74ffb50 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800dfc0 tx=0x7f4d280046c0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 shutdown_connections 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7516650 0x5569c7532cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c752fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c75205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d1aa00640 0 client.8265 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/945063068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c74ffb50 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.480+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 >> 172.21.15.59:0/945063068 conn(0x5569c7504610 msgr2=0x5569c74d3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.481+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 shutdown_connections 2024-04-23T14:40:35.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.481+0000 7f4d3fa25740 1 -- 172.21.15.59:0/945063068 wait complete. 2024-04-23T14:40:35.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.481+0000 7f4d3fa25740 20 client.8265 trim_cache size 0 max 16384 2024-04-23T14:40:35.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.500+0000 7f4d3fa25740 1 Processor -- start 2024-04-23T14:40:35.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.500+0000 7f4d3fa25740 1 -- start start 2024-04-23T14:40:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.501+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c751e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.501+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c751e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.501+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.501+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38201090 con 0x5569c7504d50 2024-04-23T14:40:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.501+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d382000f0 con 0x5569c7520120 2024-04-23T14:40:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.501+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4d380c9990 con 0x5569c7521c30 2024-04-23T14:40:35.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.501+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c751e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c751e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51614/0 (socket says 172.21.15.59:51614) 2024-04-23T14:40:35.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 -- 172.21.15.59:0/3654006446 learned_addr learned my addr 172.21.15.59:0/3654006446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d36800640 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35400640 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c751e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 -- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 msgr2=0x5569c74d9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 -- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 msgr2=0x5569c751e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c751e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 -- 172.21.15.59:0/3654006446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c71503f0 con 0x5569c7504d50 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35400640 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c751e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d36800640 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.502+0000 7f4d35e00640 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c751e160 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800e010 tx=0x7f4d2801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a83c0ee55d6f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d25c00640 1 -- 172.21.15.59:0/3654006446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28037070 con 0x5569c7504d50 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d25c00640 1 -- 172.21.15.59:0/3654006446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28019040 con 0x5569c7504d50 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d25c00640 1 -- 172.21.15.59:0/3654006446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28023040 con 0x5569c7504d50 2024-04-23T14:40:35.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 -- 172.21.15.59:0/3654006446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 msgr2=0x5569c751e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c751e160 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800e010 tx=0x7f4d2801a000 comp rx=0 tx=0).stop 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 -- 172.21.15.59:0/3654006446 shutdown_connections 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7521c30 0x5569c74d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7520120 0x5569c751e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/3654006446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c751e160 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 -- 172.21.15.59:0/3654006446 >> 172.21.15.59:0/3654006446 conn(0x5569c7504610 msgr2=0x5569c75227d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.503+0000 7f4d3fa25740 1 -- 172.21.15.59:0/3654006446 shutdown_connections 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.504+0000 7f4d3fa25740 1 -- 172.21.15.59:0/3654006446 wait complete. 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.504+0000 7f4d3fa25740 1 -- start start 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.504+0000 7f4d3fa25740 1 Processor -- start 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.505+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c7598140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7520120 0x5569c751ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d3fa25740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7521c30 0x5569c751f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5569c751d6c0 con 0x5569c7504d50 2024-04-23T14:40:35.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381ffad0 con 0x5569c7521c30 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d3fa25740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5569c75177e0 con 0x5569c7520120 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d3fa25740 10 client.0 _ll_register_callbacks cb 0x7f4d2800d5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c7598140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c7598140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51624/0 (socket says 172.21.15.59:51624) 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 -- 172.21.15.59:0/4005689264 learned_addr learned my addr 172.21.15.59:0/4005689264 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d36800640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7521c30 0x5569c751f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 -- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7520120 msgr2=0x5569c751ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35400640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7520120 0x5569c751ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7520120 0x5569c751ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 -- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7521c30 msgr2=0x5569c751f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7521c30 0x5569c751f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d35e00640 1 -- 172.21.15.59:0/4005689264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c71100f0 con 0x5569c7504d50 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.506+0000 7f4d36800640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7521c30 0x5569c751f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d35e00640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c7598140 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800deb0 tx=0x7f4d28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bfd512617d1637 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:35.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:40:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d1aa00640 1 -- 172.21.15.59:0/4005689264 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28040070 con 0x5569c7504d50 2024-04-23T14:40:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d1aa00640 1 -- 172.21.15.59:0/4005689264 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f4d2803c860 con 0x5569c7504d50 2024-04-23T14:40:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d28019040 con 0x5569c7504d50 2024-04-23T14:40:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d1aa00640 1 -- 172.21.15.59:0/4005689264 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f4d28037070 con 0x5569c7504d50 2024-04-23T14:40:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d3fa25740 10 client.8271 fetch_fsmap learned FSMap version 74 2024-04-23T14:40:35.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.507+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5569c7403810 con 0x5569c7504d50 2024-04-23T14:40:35.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d1aa00640 1 -- 172.21.15.59:0/4005689264 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28019040 con 0x5569c7504d50 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d1aa00640 1 -- 172.21.15.59:0/4005689264 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f4d2804e080 con 0x5569c7504d50 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d3fa25740 10 client.8271 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d3fa25740 10 client.8271 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d3fa25740 20 client.8271 populate_metadata read hostname 'smithi059' 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7507040 0x5569c75317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d3fa25740 4 client.8271 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569c7445a80 con 0x5569c7507040 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.508+0000 7f4d35400640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7507040 0x5569c75317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d35400640 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7507040 0x5569c75317f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3296f69fe013e4 server_cookie=8923e255014787 in_seq=0 out_seq=0 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d1aa00640 10 client.8271 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d1aa00640 1 -- 172.21.15.59:0/4005689264 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x5569c7445a80 con 0x5569c7507040 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d1aa00640 10 client.8271 handle_command_reply: tid=0 2024-04-23T14:40:35.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 client.8271 shutdown 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 msgr2=0x5569c7598140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c7598140 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800deb0 tx=0x7f4d28017000 comp rx=0 tx=0).stop 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 shutdown_connections 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x5569c7507040 0x5569c75317f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5569c7520120 0x5569c751ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5569c7521c30 0x5569c751f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d1aa00640 0 client.8271 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 --2- 172.21.15.59:0/4005689264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c7504d50 0x5569c7598140 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.509+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 >> 172.21.15.59:0/4005689264 conn(0x5569c7504610 msgr2=0x5569c751e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 shutdown_connections 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3fa25740 1 -- 172.21.15.59:0/4005689264 wait complete. 2024-04-23T14:40:35.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3fa25740 20 client.8271 trim_cache size 0 max 16384 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: "8e510cc2-9e8f-4094-bd67-c3cda70b5260": { 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/dir", 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "8e510cc2-9e8f-4094-bd67-c3cda70b5260", 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:40:35.516 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:40:35.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4d1406c920 msgr2=0x7f4d1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4d1406c920 0x7f4d1406ede0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4d28006fd0 tx=0x7f4d28008040 comp rx=0 tx=0).stop 2024-04-23T14:40:35.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3815eb60 msgr2=0x7f4d380d41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3815eb60 0x7f4d380d41a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000d830 tx=0x7f4d2000dcf0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 shutdown_connections 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4d1406c920 0x7f4d1406ede0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d380d46e0 0x7f4d380cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3815eb60 0x7f4d380d41a0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 --2- 172.21.15.59:0/896383363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d380b3490 0x7f4d38148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.510+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 >> 172.21.15.59:0/896383363 conn(0x7f4d38067b50 msgr2=0x7f4d3806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.511+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 shutdown_connections 2024-04-23T14:40:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:40:35.511+0000 7f4d3cc00640 1 -- 172.21.15.59:0/896383363 wait complete. 2024-04-23T14:40:35.531 INFO:tasks.cephfs.filesystem:scrub status for tag:8e510cc2-9e8f-4094-bd67-c3cda70b5260 - {'path': '/dir', 'tag': '8e510cc2-9e8f-4094-bd67-c3cda70b5260', 'options': 'recursive,repair'} 2024-04-23T14:40:51.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:40:51.081 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:41:05.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:05.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-23T14:41:05.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.758+0000 7f890be00640 1 Processor -- start 2024-04-23T14:41:05.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.758+0000 7f890be00640 1 -- start start 2024-04-23T14:41:05.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.759+0000 7f890be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f890c15e870 0x7f890c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.759+0000 7f890be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f890c0d02a0 0x7f890c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.759+0000 7f890be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.759+0000 7f890be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f890c0d1930 con 0x7f890c15e870 2024-04-23T14:41:05.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.759+0000 7f890be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f890c0d21a0 con 0x7f890c0d02a0 2024-04-23T14:41:05.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.759+0000 7f890be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f890c0d3000 con 0x7f890c0d14e0 2024-04-23T14:41:05.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.759+0000 7f890aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f890c15e870 0x7f890c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f890c15e870 0x7f890c0cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40048/0 (socket says 172.21.15.59:40048) 2024-04-23T14:41:05.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890aa00640 1 -- 172.21.15.59:0/1687495319 learned_addr learned my addr 172.21.15.59:0/1687495319 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890a000640 1 --2- 172.21.15.59:0/1687495319 >> v2:172.21.15.76:3300/0 conn(0x7f890c0d02a0 0x7f890c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890b400640 1 --2- 172.21.15.59:0/1687495319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890aa00640 1 -- 172.21.15.59:0/1687495319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 msgr2=0x7f890c162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890aa00640 1 --2- 172.21.15.59:0/1687495319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890aa00640 1 -- 172.21.15.59:0/1687495319 >> v2:172.21.15.76:3300/0 conn(0x7f890c0d02a0 msgr2=0x7f890c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890aa00640 1 --2- 172.21.15.59:0/1687495319 >> v2:172.21.15.76:3300/0 conn(0x7f890c0d02a0 0x7f890c0d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.760+0000 7f890aa00640 1 -- 172.21.15.59:0/1687495319 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c0d23f0 con 0x7f890c15e870 2024-04-23T14:41:05.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.762+0000 7f890aa00640 1 --2- 172.21.15.59:0/1687495319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cfd60 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f890c08da70 tx=0x7f88fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77507b9aacbd743 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:05.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.762+0000 7f8909600640 1 -- 172.21.15.59:0/1687495319 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc037070 con 0x7f890c15e870 2024-04-23T14:41:05.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.762+0000 7f8909600640 1 -- 172.21.15.59:0/1687495319 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc032440 con 0x7f890c15e870 2024-04-23T14:41:05.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.762+0000 7f8909600640 1 -- 172.21.15.59:0/1687495319 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc032810 con 0x7f890c15e870 2024-04-23T14:41:05.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 -- 172.21.15.59:0/1687495319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 msgr2=0x7f890c0cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 --2- 172.21.15.59:0/1687495319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cfd60 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f890c08da70 tx=0x7f88fc02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 -- 172.21.15.59:0/1687495319 shutdown_connections 2024-04-23T14:41:05.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 --2- 172.21.15.59:0/1687495319 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 --2- 172.21.15.59:0/1687495319 >> v2:172.21.15.76:3300/0 conn(0x7f890c0d02a0 0x7f890c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 --2- 172.21.15.59:0/1687495319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cfd60 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 -- 172.21.15.59:0/1687495319 >> 172.21.15.59:0/1687495319 conn(0x7f890c08bf90 msgr2=0x7f890c0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:05.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 -- 172.21.15.59:0/1687495319 shutdown_connections 2024-04-23T14:41:05.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.763+0000 7f890be00640 1 -- 172.21.15.59:0/1687495319 wait complete. 2024-04-23T14:41:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.765+0000 7f890be00640 1 Processor -- start 2024-04-23T14:41:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.765+0000 7f890be00640 1 -- start start 2024-04-23T14:41:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f890c0d02a0 0x7f890c14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0d2c90 con 0x7f890c15e870 2024-04-23T14:41:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0d29b0 con 0x7f890c0d02a0 2024-04-23T14:41:05.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f890c15ec50 con 0x7f890c0d14e0 2024-04-23T14:41:05.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40064/0 (socket says 172.21.15.59:40064) 2024-04-23T14:41:05.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.766+0000 7f890b400640 1 -- 172.21.15.59:0/1416377475 learned_addr learned my addr 172.21.15.59:0/1416377475 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:05.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890aa00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f890c0d02a0 0x7f890c14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890a000640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890b400640 1 -- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 msgr2=0x7f890c0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890b400640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c0cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890b400640 1 -- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f890c0d02a0 msgr2=0x7f890c14e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890b400640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f890c0d02a0 0x7f890c14e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890b400640 1 -- 172.21.15.59:0/1416377475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c15c2f0 con 0x7f890c15e870 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890b400640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cdbf0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f8904007e40 tx=0x7f890400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f8902a00640 1 -- 172.21.15.59:0/1416377475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904014620 con 0x7f890c15e870 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f8902a00640 1 -- 172.21.15.59:0/1416377475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904019070 con 0x7f890c15e870 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f8902a00640 1 -- 172.21.15.59:0/1416377475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904014a20 con 0x7f890c15e870 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.767+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c0c9e10 con 0x7f890c15e870 2024-04-23T14:41:05.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.768+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c0ca3d0 con 0x7f890c15e870 2024-04-23T14:41:05.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.770+0000 7f8902a00640 1 -- 172.21.15.59:0/1416377475 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8904014bc0 con 0x7f890c15e870 2024-04-23T14:41:05.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.771+0000 7f8902a00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88ec06c8f0 0x7f88ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.771+0000 7f8902a00640 1 -- 172.21.15.59:0/1416377475 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f890408c4f0 con 0x7f890c15e870 2024-04-23T14:41:05.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.771+0000 7f890aa00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88ec06c8f0 0x7f88ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.771+0000 7f891483d740 1 Processor -- start 2024-04-23T14:41:05.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.772+0000 7f891483d740 1 -- start start 2024-04-23T14:41:05.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.772+0000 7f890aa00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88ec06c8f0 0x7f88ec06edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f88fc034000 tx=0x7f88fc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.772+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af525790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f891483d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af5557e0 0x55c7af52a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f891483d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7af48f1f0 con 0x55c7af5557e0 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c7af435350 con 0x55c7af572e00 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c7af48b1b0 con 0x55c7af555f20 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af5557e0 0x55c7af52a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af5557e0 0x55c7af52a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40078/0 (socket says 172.21.15.59:40078) 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 -- 172.21.15.59:0/2035411745 learned_addr learned my addr 172.21.15.59:0/2035411745 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890aa00640 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af525790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890a000640 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 -- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 msgr2=0x55c7af5552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 -- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 msgr2=0x55c7af525790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af525790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.773+0000 7f890b400640 1 -- 172.21.15.59:0/2035411745 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7af489f50 con 0x55c7af5557e0 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.774+0000 7f890b400640 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af5557e0 0x55c7af52a5f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f89040627f0 tx=0x7f8904059bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb44ec5e471eea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.774+0000 7f88f7400640 1 -- 172.21.15.59:0/2035411745 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904062d20 con 0x55c7af5557e0 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.774+0000 7f88f7400640 1 -- 172.21.15.59:0/2035411745 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904058440 con 0x55c7af5557e0 2024-04-23T14:41:05.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.774+0000 7f88f7400640 1 -- 172.21.15.59:0/2035411745 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890405e120 con 0x55c7af5557e0 2024-04-23T14:41:05.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.774+0000 7f891483d740 1 -- 172.21.15.59:0/2035411745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af5557e0 msgr2=0x55c7af52a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af5557e0 0x55c7af52a5f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f89040627f0 tx=0x7f8904059bd0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 -- 172.21.15.59:0/2035411745 shutdown_connections 2024-04-23T14:41:05.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af5557e0 0x55c7af52a5f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 --2- 172.21.15.59:0/2035411745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af525790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 -- 172.21.15.59:0/2035411745 >> 172.21.15.59:0/2035411745 conn(0x55c7af527a00 msgr2=0x55c7af5702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 -- 172.21.15.59:0/2035411745 shutdown_connections 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 -- 172.21.15.59:0/2035411745 wait complete. 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.775+0000 7f891483d740 1 -- start start 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.776+0000 7f891483d740 1 Processor -- start 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af580680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af56f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f891483d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af568e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f891483d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7af529bb0 con 0x55c7af568a00 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c7af448690 con 0x55c7af572e00 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c7af528fa0 con 0x55c7af555f20 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f891483d740 10 client.0 _ll_register_callbacks cb 0x55c7af484650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af568e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af568e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40092/0 (socket says 172.21.15.59:40092) 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 -- 172.21.15.59:0/3965646123 learned_addr learned my addr 172.21.15.59:0/3965646123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890aa00640 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af580680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890a000640 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af56f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 -- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 msgr2=0x55c7af580680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af580680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 -- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 msgr2=0x55c7af56f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af56f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.777+0000 7f890b400640 1 -- 172.21.15.59:0/3965646123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7af447920 con 0x55c7af568a00 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f890b400640 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af568e80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f89040565a0 tx=0x7f890408c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da80eefea3950355 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f88ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f88ebe00640 1 -- 172.21.15.59:0/3965646123 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904076020 con 0x55c7af568a00 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f88ebe00640 1 -- 172.21.15.59:0/3965646123 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f890408c0b0 con 0x55c7af568a00 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8904058440 con 0x55c7af568a00 2024-04-23T14:41:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f88ebe00640 1 -- 172.21.15.59:0/3965646123 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f890408c2b0 con 0x55c7af568a00 2024-04-23T14:41:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f891483d740 10 client.8283 fetch_fsmap learned FSMap version 74 2024-04-23T14:41:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.778+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55c7af26b7d0 con 0x55c7af568a00 2024-04-23T14:41:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f88ebe00640 1 -- 172.21.15.59:0/3965646123 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904058440 con 0x55c7af568a00 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f88ebe00640 1 -- 172.21.15.59:0/3965646123 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f890405a3a0 con 0x55c7af568a00 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f891483d740 10 client.8283 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f891483d740 10 client.8283 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f891483d740 20 client.8283 populate_metadata read hostname 'smithi059' 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f891483d740 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af557720 0x55c7af583c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f891483d740 4 client.8283 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7af49cc10 con 0x55c7af557720 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.779+0000 7f890aa00640 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af557720 0x55c7af583c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.780+0000 7f890aa00640 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af557720 0x55c7af583c40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbdd6f6b1e2a5bd server_cookie=4810ddbc7658de82 in_seq=0 out_seq=0 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.780+0000 7f88ebe00640 10 client.8283 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.780+0000 7f88ebe00640 1 -- 172.21.15.59:0/3965646123 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7af49cc10 con 0x55c7af557720 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.780+0000 7f88ebe00640 10 client.8283 handle_command_reply: tid=0 2024-04-23T14:41:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.780+0000 7f891483d740 1 client.8283 shutdown 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 msgr2=0x55c7af568e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af568e80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f89040565a0 tx=0x7f890408c040 comp rx=0 tx=0).stop 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 shutdown_connections 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af557720 0x55c7af583c40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af568e80 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af56f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 --2- 172.21.15.59:0/3965646123 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af580680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f88ebe00640 0 client.8283 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 >> 172.21.15.59:0/3965646123 conn(0x55c7af5557e0 msgr2=0x55c7af5736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 shutdown_connections 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 1 -- 172.21.15.59:0/3965646123 wait complete. 2024-04-23T14:41:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.781+0000 7f891483d740 20 client.8283 trim_cache size 0 max 16384 2024-04-23T14:41:05.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.800+0000 7f891483d740 1 Processor -- start 2024-04-23T14:41:05.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.801+0000 7f891483d740 1 -- start start 2024-04-23T14:41:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f891483d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af555f20 0x55c7af574320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af568a00 0x55c7af594560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af572e00 0x55c7af5736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f891483d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f890c200fc0 con 0x55c7af555f20 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1fffe0 con 0x55c7af568a00 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f890c1f59f0 con 0x55c7af572e00 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af555f20 0x55c7af574320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af555f20 0x55c7af574320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40094/0 (socket says 172.21.15.59:40094) 2024-04-23T14:41:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 -- 172.21.15.59:0/459740616 learned_addr learned my addr 172.21.15.59:0/459740616 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:05.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 -- 172.21.15.59:0/459740616 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af572e00 msgr2=0x55c7af5736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af572e00 0x55c7af5736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890a000640 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af568a00 0x55c7af594560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 -- 172.21.15.59:0/459740616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af568a00 msgr2=0x55c7af594560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af568a00 0x55c7af594560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.802+0000 7f890aa00640 1 -- 172.21.15.59:0/459740616 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7af1a13f0 con 0x55c7af555f20 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.803+0000 7f890aa00640 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af555f20 0x55c7af574320 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009960 tx=0x7f88fc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f620d75073b03d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.803+0000 7f88f6a00640 1 -- 172.21.15.59:0/459740616 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc037040 con 0x55c7af555f20 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.803+0000 7f88f6a00640 1 -- 172.21.15.59:0/459740616 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc04d250 con 0x55c7af555f20 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.803+0000 7f88f6a00640 1 -- 172.21.15.59:0/459740616 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0325c0 con 0x55c7af555f20 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.803+0000 7f891483d740 1 -- 172.21.15.59:0/459740616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af555f20 msgr2=0x55c7af574320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.803+0000 7f891483d740 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af555f20 0x55c7af574320 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009960 tx=0x7f88fc02c000 comp rx=0 tx=0).stop 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 -- 172.21.15.59:0/459740616 shutdown_connections 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af572e00 0x55c7af5736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af568a00 0x55c7af594560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 --2- 172.21.15.59:0/459740616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af555f20 0x55c7af574320 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 -- 172.21.15.59:0/459740616 >> 172.21.15.59:0/459740616 conn(0x55c7af5557e0 msgr2=0x55c7af56f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 -- 172.21.15.59:0/459740616 shutdown_connections 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 -- 172.21.15.59:0/459740616 wait complete. 2024-04-23T14:41:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.804+0000 7f891483d740 1 -- start start 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.805+0000 7f891483d740 1 Processor -- start 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f891483d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af5930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f891483d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af5935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f891483d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7af56a920 con 0x55c7af568a00 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0c8c10 con 0x55c7af572e00 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f891483d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c7af56edd0 con 0x55c7af555f20 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f891483d740 10 client.0 _ll_register_callbacks cb 0x7f8904059aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af5930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af5930b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40110/0 (socket says 172.21.15.59:40110) 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 -- 172.21.15.59:0/1503987040 learned_addr learned my addr 172.21.15.59:0/1503987040 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890b400640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af5935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 -- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 msgr2=0x55c7af5d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890aa00640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 -- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 msgr2=0x55c7af5935f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af5935f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890a000640 1 -- 172.21.15.59:0/1503987040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7af1610f0 con 0x55c7af568a00 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.806+0000 7f890b400640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af5935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f890a000640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af5930b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f88f8006fd0 tx=0x7f88f8011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9e7dcee031a9d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f88ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f88ebe00640 1 -- 172.21.15.59:0/1503987040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8014850 con 0x55c7af568a00 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f88ebe00640 1 -- 172.21.15.59:0/1503987040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f88f80149f0 con 0x55c7af568a00 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc04d250 con 0x55c7af568a00 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f88ebe00640 1 -- 172.21.15.59:0/1503987040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f88f8025050 con 0x55c7af568a00 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f891483d740 10 client.8289 fetch_fsmap learned FSMap version 74 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.807+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55c7af454810 con 0x55c7af568a00 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f88ebe00640 1 -- 172.21.15.59:0/1503987040 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc04d250 con 0x55c7af568a00 2024-04-23T14:41:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f88ebe00640 1 -- 172.21.15.59:0/1503987040 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f88f8014bf0 con 0x55c7af568a00 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f891483d740 10 client.8289 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f891483d740 10 client.8289 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f891483d740 20 client.8289 populate_metadata read hostname 'smithi059' 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f891483d740 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af593b30 0x55c7af56fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f891483d740 4 client.8289 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7af496a80 con 0x55c7af593b30 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.808+0000 7f890aa00640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af593b30 0x55c7af56fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f890aa00640 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af593b30 0x55c7af56fef0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf820b244783c32 server_cookie=d8ef3bc9a0c31567 in_seq=0 out_seq=0 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f88ebe00640 10 client.8289 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f88ebe00640 1 -- 172.21.15.59:0/1503987040 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c7af496a80 con 0x55c7af593b30 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f88ebe00640 10 client.8289 handle_command_reply: tid=0 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 client.8289 shutdown 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 msgr2=0x55c7af5930b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af5930b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f88f8006fd0 tx=0x7f88f8011690 comp rx=0 tx=0).stop 2024-04-23T14:41:05.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 shutdown_connections 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c7af593b30 0x55c7af56fef0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c7af572e00 0x55c7af5935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7af568a00 0x55c7af5930b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 --2- 172.21.15.59:0/1503987040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c7af555f20 0x55c7af5d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 >> 172.21.15.59:0/1503987040 conn(0x55c7af5557e0 msgr2=0x55c7af594270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f88ebe00640 0 client.8289 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 shutdown_connections 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.809+0000 7f891483d740 1 -- 172.21.15.59:0/1503987040 wait complete. 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f891483d740 20 client.8289 trim_cache size 0 max 16384 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:41:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:41:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:41:05.817 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:41:05.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88ec06c8f0 msgr2=0x7f88ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88ec06c8f0 0x7f88ec06edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f88fc034000 tx=0x7f88fc02e000 comp rx=0 tx=0).stop 2024-04-23T14:41:05.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 msgr2=0x7f890c0cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cdbf0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f8904007e40 tx=0x7f890400e4b0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 shutdown_connections 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88ec06c8f0 0x7f88ec06edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f890c0d14e0 0x7f890c0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f890c15e870 0x7f890c0cdbf0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 --2- 172.21.15.59:0/1416377475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f890c0d02a0 0x7f890c14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 >> 172.21.15.59:0/1416377475 conn(0x7f890c08bf90 msgr2=0x7f890c074ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 shutdown_connections 2024-04-23T14:41:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:05.810+0000 7f890be00640 1 -- 172.21.15.59:0/1416377475 wait complete. 2024-04-23T14:41:05.829 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:41:05.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.034+0000 7fb82dc00640 1 Processor -- start 2024-04-23T14:41:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.035+0000 7fb82dc00640 1 -- start start 2024-04-23T14:41:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.036+0000 7fb82dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 0x7fb8280b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.036+0000 7fb82dc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8280cff40 con 0x7fb8280b35c0 2024-04-23T14:41:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.036+0000 7fb82dc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb82815b560 con 0x7fb82806fcd0 2024-04-23T14:41:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.036+0000 7fb82dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb82815e330 con 0x7fb8280b4d80 2024-04-23T14:41:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.036+0000 7fb82d200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb8280b35c0 0x7fb8280cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42618/0 (socket says 172.21.15.59:42618) 2024-04-23T14:41:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.036+0000 7fb82d200640 1 -- 172.21.15.59:0/3762596588 learned_addr learned my addr 172.21.15.59:0/3762596588 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 53490800 0 0) 0x7fb8280cff40 con 0x7fb8280b35c0 2024-04-23T14:41:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb80c003520 con 0x7fb8280b35c0 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb827400640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 0x7fb8280b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3143058683 0 0) 0x7fb80c003520 con 0x7fb8280b35c0 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 msgr2=0x7fb8280b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 0x7fb8280b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 >> v1:172.21.15.76:6789/0 conn(0x7fb82806fcd0 legacy=0x7fb82815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82815b840 con 0x7fb8280b35c0 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fb8200019f0 con 0x7fb8280b35c0 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 >> v1:172.21.15.59:6789/0 conn(0x7fb8280b35c0 legacy=0x7fb8280cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb80c0039e0 0x7fb80c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 0x7fb80c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb80c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb82815e330 con 0x7fb80c004d40 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c00bb50 con 0x7fb80c0039e0 2024-04-23T14:41:06.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.037+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb80c00bd50 con 0x7fb8280b4d80 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb82d200640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb80c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb827e00640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb80c0039e0 0x7fb80c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb827400640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 0x7fb80c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb82d200640 1 -- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 msgr2=0x7fb80c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb82d200640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280b4d80 0x7fb80c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb82d200640 1 -- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb80c0039e0 msgr2=0x7fb80c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb82d200640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb80c0039e0 0x7fb80c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.038+0000 7fb82d200640 1 -- 172.21.15.59:0/3762596588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c003520 con 0x7fb80c004d40 2024-04-23T14:41:06.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.039+0000 7fb82d200640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb80c0051c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb82000a850 tx=0x7fb80c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa48469c3508f8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.039+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82000a4a0 con 0x7fb80c004d40 2024-04-23T14:41:06.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82002f070 con 0x7fb80c004d40 2024-04-23T14:41:06.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb826a00640 1 -- 172.21.15.59:0/3762596588 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82002bce0 con 0x7fb80c004d40 2024-04-23T14:41:06.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb82dc00640 1 -- 172.21.15.59:0/3762596588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 msgr2=0x7fb80c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb82dc00640 1 --2- 172.21.15.59:0/3762596588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb80c0051c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb82000a850 tx=0x7fb80c005830 comp rx=0 tx=0).stop 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb82d200640 1 -- 172.21.15.59:0/3762596588 reap_dead start 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb82dc00640 1 -- 172.21.15.59:0/3762596588 shutdown_connections 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb82dc00640 1 -- 172.21.15.59:0/3762596588 >> 172.21.15.59:0/3762596588 conn(0x7fb8280b21a0 msgr2=0x7fb8280b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb82dc00640 1 -- 172.21.15.59:0/3762596588 shutdown_connections 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.040+0000 7fb82dc00640 1 -- 172.21.15.59:0/3762596588 wait complete. 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.042+0000 7fb82dc00640 1 Processor -- start 2024-04-23T14:41:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.042+0000 7fb82dc00640 1 -- start start 2024-04-23T14:41:06.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.043+0000 7fb82dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb8280aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.043+0000 7fb82dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280d0850 0x7fb82815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.043+0000 7fb82dc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8280cd180 0x7fb82815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.043+0000 7fb82dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8280cd890 con 0x7fb80c004d40 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.043+0000 7fb82dc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb82815e580 con 0x7fb8280cd180 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.043+0000 7fb82dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb8280cda10 con 0x7fb8280d0850 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb8280aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb8280aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40122/0 (socket says 172.21.15.59:40122) 2024-04-23T14:41:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 -- 172.21.15.59:0/2310665652 learned_addr learned my addr 172.21.15.59:0/2310665652 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827400640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280d0850 0x7fb82815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 -- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280d0850 msgr2=0x7fb82815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280d0850 0x7fb82815bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 -- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8280cd180 msgr2=0x7fb82815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8280cd180 0x7fb82815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 -- 172.21.15.59:0/2310665652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb828154430 con 0x7fb80c004d40 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827400640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280d0850 0x7fb82815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.044+0000 7fb827e00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb8280aeb40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb81800e5b0 tx=0x7fb81800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.045+0000 7fb81fe00640 1 -- 172.21.15.59:0/2310665652 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8180052e0 con 0x7fb80c004d40 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.045+0000 7fb81fe00640 1 -- 172.21.15.59:0/2310665652 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb818018070 con 0x7fb80c004d40 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.045+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82815dd50 con 0x7fb80c004d40 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.045+0000 7fb81fe00640 1 -- 172.21.15.59:0/2310665652 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818003810 con 0x7fb80c004d40 2024-04-23T14:41:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.045+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82806f4d0 con 0x7fb80c004d40 2024-04-23T14:41:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.046+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82804fd90 con 0x7fb80c004d40 2024-04-23T14:41:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.047+0000 7fb81fe00640 1 -- 172.21.15.59:0/2310665652 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8180039b0 con 0x7fb80c004d40 2024-04-23T14:41:06.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.048+0000 7fb81fe00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb80406c920 0x7fb80406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.048+0000 7fb81fe00640 1 -- 172.21.15.59:0/2310665652 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fb81808b820 con 0x7fb80c004d40 2024-04-23T14:41:06.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.048+0000 7fb827400640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb80406c920 0x7fb80406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.049+0000 7fb827400640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb80406c920 0x7fb80406ede0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb828169410 tx=0x7fb810008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.053+0000 7fb81fe00640 1 -- 172.21.15.59:0/2310665652 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb818059ae0 con 0x7fb80c004d40 2024-04-23T14:41:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.297+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb828073ea0 con 0x7fb80c004d40 2024-04-23T14:41:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.298+0000 7fb81fe00640 1 -- 172.21.15.59:0/2310665652 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5181 (secure 0 0 0) 0x7fb8180593e0 con 0x7fb80c004d40 2024-04-23T14:41:06.299 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:06.300 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:30.798278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-23T14:41:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 74 2024-04-23T14:41:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb80406c920 msgr2=0x7fb80406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb80406c920 0x7fb80406ede0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb828169410 tx=0x7fb810008040 comp rx=0 tx=0).stop 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 msgr2=0x7fb8280aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb8280aeb40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb81800e5b0 tx=0x7fb81800ea70 comp rx=0 tx=0).stop 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 shutdown_connections 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb80406c920 0x7fb80406ede0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb8280d0850 0x7fb82815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8280cd180 0x7fb82815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 --2- 172.21.15.59:0/2310665652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb80c004d40 0x7fb8280aeb40 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 >> 172.21.15.59:0/2310665652 conn(0x7fb8280b21a0 msgr2=0x7fb8280d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 shutdown_connections 2024-04-23T14:41:06.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.301+0000 7fb82dc00640 1 -- 172.21.15.59:0/2310665652 wait complete. 2024-04-23T14:41:06.322 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-23T14:41:06.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.536+0000 7fc78fe00640 1 Processor -- start 2024-04-23T14:41:06.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.536+0000 7fc78fe00640 1 -- start start 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.537+0000 7fc78fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc7900d15b0 0x7fc7900cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.537+0000 7fc78fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7900b4f20 0x7fc7900cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.537+0000 7fc78fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7900b5da0 con 0x7fc79015ed20 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.537+0000 7fc78fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc79015f100 con 0x7fc7900b4f20 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.537+0000 7fc78fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc7900d0a20 con 0x7fc7900d15b0 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.538+0000 7fc78ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc79015ed20 0x7fc7900d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42628/0 (socket says 172.21.15.59:42628) 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.538+0000 7fc78ea00640 1 -- 172.21.15.59:0/3579876513 learned_addr learned my addr 172.21.15.59:0/3579876513 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:06.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.538+0000 7fc78f400640 1 --2- 172.21.15.59:0/3579876513 >> v2:172.21.15.59:3300/0 conn(0x7fc7900b4f20 0x7fc7900cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.538+0000 7fc786000640 1 --2- 172.21.15.59:0/3579876513 >> v2:172.21.15.76:3300/0 conn(0x7fc7900d15b0 0x7fc7900cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.538+0000 7fc78e000640 1 -- 172.21.15.59:0/3579876513 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309322141 0 0) 0x7fc7900b5da0 con 0x7fc79015ed20 2024-04-23T14:41:06.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.538+0000 7fc78e000640 1 -- 172.21.15.59:0/3579876513 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc774003520 con 0x7fc79015ed20 2024-04-23T14:41:06.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.539+0000 7fc78f400640 1 -- 172.21.15.59:0/3579876513 >> v2:172.21.15.76:3300/0 conn(0x7fc7900d15b0 msgr2=0x7fc7900cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.539+0000 7fc78f400640 1 --2- 172.21.15.59:0/3579876513 >> v2:172.21.15.76:3300/0 conn(0x7fc7900d15b0 0x7fc7900cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.539+0000 7fc78f400640 1 -- 172.21.15.59:0/3579876513 >> v1:172.21.15.59:6789/0 conn(0x7fc79015ed20 legacy=0x7fc7900d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.539+0000 7fc78f400640 1 -- 172.21.15.59:0/3579876513 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7900d2ed0 con 0x7fc7900b4f20 2024-04-23T14:41:06.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.539+0000 7fc786000640 1 --2- 172.21.15.59:0/3579876513 >> v2:172.21.15.76:3300/0 conn(0x7fc7900d15b0 0x7fc7900cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:41:06.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.540+0000 7fc78f400640 1 --2- 172.21.15.59:0/3579876513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7900b4f20 0x7fc7900cf7c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc7900d4520 tx=0x7fc78802a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f775674e9bea7d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.540+0000 7fc78e000640 1 -- 172.21.15.59:0/3579876513 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788007740 con 0x7fc7900b4f20 2024-04-23T14:41:06.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.540+0000 7fc78e000640 1 -- 172.21.15.59:0/3579876513 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7880078e0 con 0x7fc7900b4f20 2024-04-23T14:41:06.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.540+0000 7fc78e000640 1 -- 172.21.15.59:0/3579876513 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788007cb0 con 0x7fc7900b4f20 2024-04-23T14:41:06.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.540+0000 7fc78fe00640 1 -- 172.21.15.59:0/3579876513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7900b4f20 msgr2=0x7fc7900cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.540+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3579876513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7900b4f20 0x7fc7900cf7c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc7900d4520 tx=0x7fc78802a550 comp rx=0 tx=0).stop 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.541+0000 7fc78fe00640 1 -- 172.21.15.59:0/3579876513 shutdown_connections 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.541+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3579876513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7900b4f20 0x7fc7900cf7c0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.541+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3579876513 >> v2:172.21.15.76:3300/0 conn(0x7fc7900d15b0 0x7fc7900cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.541+0000 7fc78fe00640 1 -- 172.21.15.59:0/3579876513 >> 172.21.15.59:0/3579876513 conn(0x7fc79008bf90 msgr2=0x7fc7900d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.541+0000 7fc78fe00640 1 -- 172.21.15.59:0/3579876513 shutdown_connections 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.541+0000 7fc78fe00640 1 -- 172.21.15.59:0/3579876513 wait complete. 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.542+0000 7fc78fe00640 1 Processor -- start 2024-04-23T14:41:06.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.543+0000 7fc78fe00640 1 -- start start 2024-04-23T14:41:06.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.543+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7900b4f20 0x7fc790155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.543+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7900d15b0 0x7fc7900c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79015ed20 0x7fc790157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7900b3600 con 0x7fc79015ed20 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7900d3490 con 0x7fc7900b4f20 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc7900b3780 con 0x7fc7900d15b0 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79015ed20 0x7fc790157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79015ed20 0x7fc790157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40146/0 (socket says 172.21.15.59:40146) 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 -- 172.21.15.59:0/3103806535 learned_addr learned my addr 172.21.15.59:0/3103806535 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78ea00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7900b4f20 0x7fc790155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc786000640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7900d15b0 0x7fc7900c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 -- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7900d15b0 msgr2=0x7fc7900c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7900d15b0 0x7fc7900c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 -- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7900b4f20 msgr2=0x7fc790155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7900b4f20 0x7fc790155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.544+0000 7fc78f400640 1 -- 172.21.15.59:0/3103806535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc790155780 con 0x7fc79015ed20 2024-04-23T14:41:06.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.545+0000 7fc78f400640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79015ed20 0x7fc790157610 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc78802ab40 tx=0x7fc788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.545+0000 7fc787400640 1 -- 172.21.15.59:0/3103806535 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78803f070 con 0x7fc79015ed20 2024-04-23T14:41:06.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.545+0000 7fc787400640 1 -- 172.21.15.59:0/3103806535 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788031bc0 con 0x7fc79015ed20 2024-04-23T14:41:06.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.545+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc790156850 con 0x7fc79015ed20 2024-04-23T14:41:06.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.545+0000 7fc787400640 1 -- 172.21.15.59:0/3103806535 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788009450 con 0x7fc79015ed20 2024-04-23T14:41:06.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.545+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc790156e10 con 0x7fc79015ed20 2024-04-23T14:41:06.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.548+0000 7fc787400640 1 -- 172.21.15.59:0/3103806535 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc78803b410 con 0x7fc79015ed20 2024-04-23T14:41:06.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.548+0000 7fc787400640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76c06c8f0 0x7fc76c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.549+0000 7fc787400640 1 -- 172.21.15.59:0/3103806535 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fc7880ad560 con 0x7fc79015ed20 2024-04-23T14:41:06.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.549+0000 7fc78ea00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76c06c8f0 0x7fc76c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.549+0000 7fc798674740 1 Processor -- start 2024-04-23T14:41:06.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.550+0000 7fc78ea00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76c06c8f0 0x7fc76c06edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc780005880 tx=0x7fc780009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.550+0000 7fc798674740 1 -- start start 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e5a220 0x55c855e5a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e77ac0 0x55c855e78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc798674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e2ff00 0x55c855e76180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc798674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c855d94190 con 0x55c855e2ff00 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c855d3a030 con 0x55c855e5a220 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c855d90150 con 0x55c855e77ac0 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e2ff00 0x55c855e76180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e2ff00 0x55c855e76180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40156/0 (socket says 172.21.15.59:40156) 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 -- 172.21.15.59:0/4087759799 learned_addr learned my addr 172.21.15.59:0/4087759799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:06.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78ea00640 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e5a220 0x55c855e5a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 -- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e77ac0 msgr2=0x55c855e78010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e77ac0 0x55c855e78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 -- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e5a220 msgr2=0x55c855e5a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e5a220 0x55c855e5a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78f400640 1 -- 172.21.15.59:0/4087759799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c855d22b60 con 0x55c855e2ff00 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.551+0000 7fc78ea00640 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e5a220 0x55c855e5a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.552+0000 7fc78f400640 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e2ff00 0x55c855e76180 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc7880837d0 tx=0x7fc788009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d97a81d89005d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.552+0000 7fc77f400640 1 -- 172.21.15.59:0/4087759799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788083d00 con 0x55c855e2ff00 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.552+0000 7fc77f400640 1 -- 172.21.15.59:0/4087759799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788079440 con 0x55c855e2ff00 2024-04-23T14:41:06.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.552+0000 7fc77f400640 1 -- 172.21.15.59:0/4087759799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7880777b0 con 0x55c855e2ff00 2024-04-23T14:41:06.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 -- 172.21.15.59:0/4087759799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e2ff00 msgr2=0x55c855e76180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e2ff00 0x55c855e76180 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc7880837d0 tx=0x7fc788009b90 comp rx=0 tx=0).stop 2024-04-23T14:41:06.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 -- 172.21.15.59:0/4087759799 shutdown_connections 2024-04-23T14:41:06.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e77ac0 0x55c855e78010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e2ff00 0x55c855e76180 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 --2- 172.21.15.59:0/4087759799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e5a220 0x55c855e5a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 -- 172.21.15.59:0/4087759799 >> 172.21.15.59:0/4087759799 conn(0x55c855e2c830 msgr2=0x55c855e75030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 -- 172.21.15.59:0/4087759799 shutdown_connections 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.553+0000 7fc798674740 1 -- 172.21.15.59:0/4087759799 wait complete. 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.554+0000 7fc798674740 1 -- start start 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.554+0000 7fc798674740 1 Processor -- start 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc798674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855e5c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 0x55c855e85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc798674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c855e30ca0 con 0x55c855e5a220 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c855d4d500 con 0x55c855e77ac0 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c855e2a760 con 0x55c855e6f130 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc798674740 10 client.0 _ll_register_callbacks cb 0x55c855d8a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc78ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855e5c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc78ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855e5c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40170/0 (socket says 172.21.15.59:40170) 2024-04-23T14:41:06.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.555+0000 7fc78ea00640 1 -- 172.21.15.59:0/3677529888 learned_addr learned my addr 172.21.15.59:0/3677529888 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc786000640 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc78ea00640 1 -- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 msgr2=0x55c855e85110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc78ea00640 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 0x55c855e85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc78ea00640 1 -- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 msgr2=0x55c855e6ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc78ea00640 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e6ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc78ea00640 1 -- 172.21.15.59:0/3677529888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c855b727c0 con 0x55c855e5a220 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc786000640 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e6ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc78ea00640 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855e5c1b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc780005f40 tx=0x7fc780000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02cf58fceaa1b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc773e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:41:06.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc773e00640 1 -- 172.21.15.59:0/3677529888 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780007560 con 0x55c855e5a220 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc773e00640 1 -- 172.21.15.59:0/3677529888 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fc780021b50 con 0x55c855e5a220 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc788079440 con 0x55c855e5a220 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.556+0000 7fc773e00640 1 -- 172.21.15.59:0/3677529888 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc780007700 con 0x55c855e5a220 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 10 client.8310 fetch_fsmap learned FSMap version 74 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55c855bd2f50 con 0x55c855e5a220 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc773e00640 1 -- 172.21.15.59:0/3677529888 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788079440 con 0x55c855e5a220 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc773e00640 1 -- 172.21.15.59:0/3677529888 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fc780021420 con 0x55c855e5a220 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 10 client.8310 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 10 client.8310 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 20 client.8310 populate_metadata read hostname 'smithi059' 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855e6cfc0 0x55c855e89420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 4 client.8310 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:41:06.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c855da1bb0 con 0x55c855e6cfc0 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.557+0000 7fc786000640 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855e6cfc0 0x55c855e89420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.558+0000 7fc786000640 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855e6cfc0 0x55c855e89420 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e30530bf484dbd server_cookie=8f6fb2779e9f1bb7 in_seq=0 out_seq=0 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.558+0000 7fc773e00640 10 client.8310 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc773e00640 1 -- 172.21.15.59:0/3677529888 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c855da1bb0 con 0x55c855e6cfc0 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc773e00640 10 client.8310 handle_command_reply: tid=0 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 client.8310 shutdown 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 msgr2=0x55c855e5c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855e5c1b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc780005f40 tx=0x7fc780000ed0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 shutdown_connections 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855e6cfc0 0x55c855e89420 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 0x55c855e85110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e6ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 --2- 172.21.15.59:0/3677529888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855e5c1b0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc773e00640 0 client.8310 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:06.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 >> 172.21.15.59:0/3677529888 conn(0x55c855e2ff00 msgr2=0x55c855e785b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 shutdown_connections 2024-04-23T14:41:06.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 1 -- 172.21.15.59:0/3677529888 wait complete. 2024-04-23T14:41:06.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.559+0000 7fc798674740 20 client.8310 trim_cache size 0 max 16384 2024-04-23T14:41:06.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.574+0000 7fc798674740 1 Processor -- start 2024-04-23T14:41:06.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.575+0000 7fc798674740 1 -- start start 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.575+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e5a220 0x55c855e84f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.575+0000 7fc798674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e6f130 0x55c855e77f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.575+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.575+0000 7fc798674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc790159850 con 0x55c855e6f130 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.575+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc790158bb0 con 0x55c855e77ac0 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.575+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc790159200 con 0x55c855e5a220 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e6f130 0x55c855e77f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e6f130 0x55c855e77f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40176/0 (socket says 172.21.15.59:40176) 2024-04-23T14:41:06.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 -- 172.21.15.59:0/1797101489 learned_addr learned my addr 172.21.15.59:0/1797101489 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:06.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc78f400640 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 -- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e5a220 msgr2=0x55c855e84f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e5a220 0x55c855e84f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 -- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 msgr2=0x55c855e2ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e2ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc786000640 1 -- 172.21.15.59:0/1797101489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c855e76300 con 0x55c855e6f130 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.576+0000 7fc78f400640 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.577+0000 7fc786000640 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e6f130 0x55c855e77f70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b940 tx=0x7fc77800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec12ef19edd7718 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.577+0000 7fc77ea00640 1 -- 172.21.15.59:0/1797101489 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77800f040 con 0x55c855e6f130 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.577+0000 7fc77ea00640 1 -- 172.21.15.59:0/1797101489 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77801e030 con 0x55c855e6f130 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.577+0000 7fc77ea00640 1 -- 172.21.15.59:0/1797101489 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77801a150 con 0x55c855e6f130 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.577+0000 7fc798674740 1 -- 172.21.15.59:0/1797101489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e6f130 msgr2=0x55c855e77f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.577+0000 7fc798674740 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e6f130 0x55c855e77f70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b940 tx=0x7fc77800bea0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 -- 172.21.15.59:0/1797101489 shutdown_connections 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e6f130 0x55c855e77f70 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 --2- 172.21.15.59:0/1797101489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e5a220 0x55c855e84f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 -- 172.21.15.59:0/1797101489 >> 172.21.15.59:0/1797101489 conn(0x55c855e2ff00 msgr2=0x55c855e2f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 -- 172.21.15.59:0/1797101489 shutdown_connections 2024-04-23T14:41:06.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 -- 172.21.15.59:0/1797101489 wait complete. 2024-04-23T14:41:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.578+0000 7fc798674740 1 -- start start 2024-04-23T14:41:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.579+0000 7fc798674740 1 Processor -- start 2024-04-23T14:41:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc798674740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855ef1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 0x55c855e302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc798674740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e30820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc798674740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c855e5daa0 con 0x55c855e5a220 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c855e5d2a0 con 0x55c855e77ac0 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc798674740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c855e6ca70 con 0x55c855e6f130 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc798674740 10 client.0 _ll_register_callbacks cb 0x7fc780001450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855ef1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855ef1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40192/0 (socket says 172.21.15.59:40192) 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 -- 172.21.15.59:0/780470328 learned_addr learned my addr 172.21.15.59:0/780470328 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 -- 172.21.15.59:0/780470328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 msgr2=0x55c855e302e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 0x55c855e302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 -- 172.21.15.59:0/780470328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 msgr2=0x55c855e30820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78f400640 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e30820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e30820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.580+0000 7fc78ea00640 1 -- 172.21.15.59:0/780470328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c855a660b0 con 0x55c855e5a220 2024-04-23T14:41:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc78ea00640 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855ef1e80 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc780021bd0 tx=0x7fc780000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ec29c3e3a61818 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc773e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc773e00640 1 -- 172.21.15.59:0/780470328 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780030070 con 0x55c855e5a220 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc773e00640 1 -- 172.21.15.59:0/780470328 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fc780022430 con 0x55c855e5a220 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77801e030 con 0x55c855e5a220 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc773e00640 1 -- 172.21.15.59:0/780470328 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc78003a050 con 0x55c855e5a220 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc798674740 10 client.8316 fetch_fsmap learned FSMap version 74 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.581+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55c855ef8a80 con 0x55c855e5a220 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc773e00640 1 -- 172.21.15.59:0/780470328 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77801e030 con 0x55c855e5a220 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc773e00640 1 -- 172.21.15.59:0/780470328 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fc7800047e0 con 0x55c855e5a220 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc798674740 10 client.8316 fetch_fsmap finished waiting for FSMap version 74 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc798674740 10 client.8316 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc798674740 20 client.8316 populate_metadata read hostname 'smithi059' 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc798674740 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855ef8510 0x55c855e5aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc798674740 4 client.8316 mds_command: new command op to 8133 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-23T14:41:06.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 --> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55c855d9ba20 con 0x55c855ef8510 2024-04-23T14:41:06.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.582+0000 7fc786000640 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855ef8510 0x55c855e5aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc786000640 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855ef8510 0x55c855e5aee0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276f7273edc91d48 server_cookie=3e787821b44f73c in_seq=0 out_seq=0 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc773e00640 10 client.8316 ms_handle_connect on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc773e00640 1 -- 172.21.15.59:0/780470328 <== mds.0 v2:172.21.15.59:6835/3466627380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c855d9ba20 con 0x55c855ef8510 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc773e00640 10 client.8316 handle_command_reply: tid=0 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 client.8316 shutdown 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 msgr2=0x55c855ef1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855ef1e80 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc780021bd0 tx=0x7fc780000bc0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 shutdown_connections 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:6835/3466627380,v1:172.21.15.59:6837/3466627380] conn(0x55c855ef8510 0x55c855e5aee0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c855e6f130 0x55c855e302e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c855e77ac0 0x55c855e30820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc773e00640 0 client.8316 ms_handle_reset on v2:172.21.15.59:6835/3466627380 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 --2- 172.21.15.59:0/780470328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c855e5a220 0x55c855ef1e80 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 >> 172.21.15.59:0/780470328 conn(0x55c855e2ff00 msgr2=0x55c855e59460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 shutdown_connections 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.583+0000 7fc798674740 1 -- 172.21.15.59:0/780470328 wait complete. 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc798674740 20 client.8316 trim_cache size 0 max 16384 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:06.588 INFO:teuthology.orchestra.run.smithi059.stdout:[] 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76c06c8f0 msgr2=0x7fc76c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76c06c8f0 0x7fc76c06edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc780005880 tx=0x7fc780009040 comp rx=0 tx=0).stop 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79015ed20 msgr2=0x7fc790157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79015ed20 0x7fc790157610 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc78802ab40 tx=0x7fc788004a40 comp rx=0 tx=0).stop 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 shutdown_connections 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc76c06c8f0 0x7fc76c06edb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc7900d15b0 0x7fc7900c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc79015ed20 0x7fc790157610 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 --2- 172.21.15.59:0/3103806535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7900b4f20 0x7fc790155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.584+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 >> 172.21.15.59:0/3103806535 conn(0x7fc79008bf90 msgr2=0x7fc79006d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.585+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 shutdown_connections 2024-04-23T14:41:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.585+0000 7fc78fe00640 1 -- 172.21.15.59:0/3103806535 wait complete. 2024-04-23T14:41:06.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-23T14:41:06.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.806+0000 7fddd3600640 1 Processor -- start 2024-04-23T14:41:06.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.806+0000 7fddd3600640 1 -- start start 2024-04-23T14:41:06.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddd3600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fddcc0cd020 0x7fddcc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddd3600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc0b3e40 0x7fddcc160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddd3600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddcc0cd520 con 0x7fddcc1454e0 2024-04-23T14:41:06.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddd3600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fddcc0cddc0 con 0x7fddcc0cd020 2024-04-23T14:41:06.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddd3600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fddcc1462c0 con 0x7fddcc0b3e40 2024-04-23T14:41:06.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddd0c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fddcc1454e0 0x7fddcc145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42632/0 (socket says 172.21.15.59:42632) 2024-04-23T14:41:06.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddd0c00640 1 -- 172.21.15.59:0/2249588495 learned_addr learned my addr 172.21.15.59:0/2249588495 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:06.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.808+0000 7fddcbe00640 1 --2- 172.21.15.59:0/2249588495 >> v2:172.21.15.76:3300/0 conn(0x7fddcc0cd020 0x7fddcc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddd1600640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc0b3e40 0x7fddcc160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854818364 0 0) 0x7fddcc0cd520 con 0x7fddcc1454e0 2024-04-23T14:41:06.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddb4003520 con 0x7fddcc1454e0 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4085758978 0 0) 0x7fddb4003520 con 0x7fddcc1454e0 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc0b3e40 msgr2=0x7fddcc160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc0b3e40 0x7fddcc160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 >> v2:172.21.15.76:3300/0 conn(0x7fddcc0cd020 msgr2=0x7fddcc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 --2- 172.21.15.59:0/2249588495 >> v2:172.21.15.76:3300/0 conn(0x7fddcc0cd020 0x7fddcc0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc0cd800 con 0x7fddcc1454e0 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fddc0001670 con 0x7fddcc1454e0 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 >> v1:172.21.15.59:6789/0 conn(0x7fddcc1454e0 legacy=0x7fddcc145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc0b3e40 0x7fddb40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb4004d40 0x7fddb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc0cddc0 con 0x7fddb4003f80 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1462c0 con 0x7fddb4004d40 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.809+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fddb400ba70 con 0x7fddcc0b3e40 2024-04-23T14:41:06.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.810+0000 7fddcbe00640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.810+0000 7fddcbe00640 1 -- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc0b3e40 msgr2=0x7fddb40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.810+0000 7fddcbe00640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc0b3e40 0x7fddb40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.810+0000 7fddcbe00640 1 -- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb4004d40 msgr2=0x7fddb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:06.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.810+0000 7fddd1600640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb4004d40 0x7fddb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.810+0000 7fddcbe00640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb4004d40 0x7fddb40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.810+0000 7fddcbe00640 1 -- 172.21.15.59:0/2249588495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb4003520 con 0x7fddb4003f80 2024-04-23T14:41:06.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.811+0000 7fddcbe00640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddb4004800 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009ac0 tx=0x7fddb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df2e134d1748ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.811+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8036070 con 0x7fddb4003f80 2024-04-23T14:41:06.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.811+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb8031400 con 0x7fddb4003f80 2024-04-23T14:41:06.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.811+0000 7fddcb400640 1 -- 172.21.15.59:0/2249588495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8009690 con 0x7fddb4003f80 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.812+0000 7fddd3600640 1 -- 172.21.15.59:0/2249588495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 msgr2=0x7fddb4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.812+0000 7fddd3600640 1 --2- 172.21.15.59:0/2249588495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddb4004800 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009ac0 tx=0x7fddb4005830 comp rx=0 tx=0).stop 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.812+0000 7fddd1600640 1 -- 172.21.15.59:0/2249588495 reap_dead start 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.812+0000 7fddd3600640 1 -- 172.21.15.59:0/2249588495 shutdown_connections 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.812+0000 7fddd3600640 1 -- 172.21.15.59:0/2249588495 >> 172.21.15.59:0/2249588495 conn(0x7fddcc067ea0 msgr2=0x7fddcc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.812+0000 7fddd3600640 1 -- 172.21.15.59:0/2249588495 shutdown_connections 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.812+0000 7fddd3600640 1 -- 172.21.15.59:0/2249588495 wait complete. 2024-04-23T14:41:06.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.814+0000 7fddd3600640 1 Processor -- start 2024-04-23T14:41:06.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.814+0000 7fddd3600640 1 -- start start 2024-04-23T14:41:06.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.815+0000 7fddd3600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddcc15bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.815+0000 7fddd3600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc142ba0 0x7fddcc1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.815+0000 7fddd3600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddcc15e500 0x7fddcc0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.815+0000 7fddd3600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc160910 con 0x7fddb4003f80 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.815+0000 7fddd3600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc1603f0 con 0x7fddcc15e500 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.815+0000 7fddd3600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fddcc160a90 con 0x7fddcc142ba0 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddcc15bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddcc15bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40200/0 (socket says 172.21.15.59:40200) 2024-04-23T14:41:06.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 -- 172.21.15.59:0/2289172030 learned_addr learned my addr 172.21.15.59:0/2289172030 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:06.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 -- 172.21.15.59:0/2289172030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc142ba0 msgr2=0x7fddcc1f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc142ba0 0x7fddcc1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 -- 172.21.15.59:0/2289172030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddcc15e500 msgr2=0x7fddcc0725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd1600640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddcc15e500 0x7fddcc0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddcc15e500 0x7fddcc0725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 -- 172.21.15.59:0/2289172030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc15c520 con 0x7fddb4003f80 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.816+0000 7fddd0c00640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddcc15bfe0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fddc000c680 tx=0x7fddc000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.817+0000 7fddc8c00640 1 -- 172.21.15.59:0/2289172030 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0018070 con 0x7fddb4003f80 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.817+0000 7fddc8c00640 1 -- 172.21.15.59:0/2289172030 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc00072f0 con 0x7fddb4003f80 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.817+0000 7fddc8c00640 1 -- 172.21.15.59:0/2289172030 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00047e0 con 0x7fddb4003f80 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.817+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddcc1430a0 con 0x7fddb4003f80 2024-04-23T14:41:06.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.817+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddcc1f3f00 con 0x7fddb4003f80 2024-04-23T14:41:06.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.818+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddcc068a80 con 0x7fddb4003f80 2024-04-23T14:41:06.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.819+0000 7fddc8c00640 1 -- 172.21.15.59:0/2289172030 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddc001b070 con 0x7fddb4003f80 2024-04-23T14:41:06.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.820+0000 7fddc8c00640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fddac06c890 0x7fddac06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:06.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.820+0000 7fddc8c00640 1 -- 172.21.15.59:0/2289172030 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fddc008b170 con 0x7fddb4003f80 2024-04-23T14:41:06.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.820+0000 7fddcbe00640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fddac06c890 0x7fddac06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:06.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.821+0000 7fddcbe00640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fddac06c890 0x7fddac06ed50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fddb8033000 tx=0x7fddb802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:06.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:06.825+0000 7fddc8c00640 1 -- 172.21.15.59:0/2289172030 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddc00594c0 con 0x7fddb4003f80 2024-04-23T14:41:07.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.072+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fddcc145fc0 con 0x7fddb4003f80 2024-04-23T14:41:07.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.073+0000 7fddc8c00640 1 -- 172.21.15.59:0/2289172030 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fddc0058dc0 con 0x7fddb4003f80 2024-04-23T14:41:07.074 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:07.074 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T14:41:07.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fddac06c890 msgr2=0x7fddac06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fddac06c890 0x7fddac06ed50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fddb8033000 tx=0x7fddb802e000 comp rx=0 tx=0).stop 2024-04-23T14:41:07.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 msgr2=0x7fddcc15bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddcc15bfe0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fddc000c680 tx=0x7fddc000cb40 comp rx=0 tx=0).stop 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 shutdown_connections 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fddac06c890 0x7fddac06ed50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fddcc142ba0 0x7fddcc1f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddcc15e500 0x7fddcc0725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 --2- 172.21.15.59:0/2289172030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddb4003f80 0x7fddcc15bfe0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 >> 172.21.15.59:0/2289172030 conn(0x7fddcc067ea0 msgr2=0x7fddcc0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 shutdown_connections 2024-04-23T14:41:07.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.076+0000 7fddd3600640 1 -- 172.21.15.59:0/2289172030 wait complete. 2024-04-23T14:41:07.098 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-23T14:41:07.098 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:41:07.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:41:07.205 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:41:07.313 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:41:07.313 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:41:07.314 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:07.314 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:07.371 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:07.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.585+0000 7f7869600640 1 Processor -- start 2024-04-23T14:41:07.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.585+0000 7f7869600640 1 -- start start 2024-04-23T14:41:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.587+0000 7f7869600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f786415d2d0 0x7f78640cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.587+0000 7f7869600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f786409b960 con 0x7f78640c5ae0 2024-04-23T14:41:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.587+0000 7f7869600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78640d0a90 con 0x7f78640d0510 2024-04-23T14:41:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.587+0000 7f7869600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f786409b4d0 con 0x7f786415d2d0 2024-04-23T14:41:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.587+0000 7f7863e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f786415d2d0 0x7f78640cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:07.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.587+0000 7f7868c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f78640c5ae0 0x7f78641630b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42640/0 (socket says 172.21.15.59:42640) 2024-04-23T14:41:07.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.587+0000 7f7868c00640 1 -- 172.21.15.59:0/1699956510 learned_addr learned my addr 172.21.15.59:0/1699956510 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:07.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.588+0000 7f7863e00640 1 -- 172.21.15.59:0/1699956510 >> v1:172.21.15.76:6789/0 conn(0x7f78640d0510 legacy=0x7f78640c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:07.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.588+0000 7f7863e00640 1 -- 172.21.15.59:0/1699956510 >> v1:172.21.15.59:6789/0 conn(0x7f78640c5ae0 legacy=0x7f78641630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.588+0000 7f7863e00640 1 -- 172.21.15.59:0/1699956510 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786409aba0 con 0x7f786415d2d0 2024-04-23T14:41:07.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.588+0000 7f7862a00640 1 -- 172.21.15.59:0/1699956510 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557902947 0 0) 0x7f786409b960 con 0x7f78640c5ae0 2024-04-23T14:41:07.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.589+0000 7f7863e00640 1 --2- 172.21.15.59:0/1699956510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f786415d2d0 0x7f78640cffd0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f78540097a0 tx=0x7f785402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84120acf68eb2c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.590+0000 7f7862a00640 1 -- 172.21.15.59:0/1699956510 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854037070 con 0x7f786415d2d0 2024-04-23T14:41:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.590+0000 7f7862a00640 1 -- 172.21.15.59:0/1699956510 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7854032440 con 0x7f786415d2d0 2024-04-23T14:41:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.590+0000 7f7862a00640 1 -- 172.21.15.59:0/1699956510 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854032810 con 0x7f786415d2d0 2024-04-23T14:41:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.590+0000 7f7869600640 1 -- 172.21.15.59:0/1699956510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f786415d2d0 msgr2=0x7f78640cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.590+0000 7f7869600640 1 --2- 172.21.15.59:0/1699956510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f786415d2d0 0x7f78640cffd0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f78540097a0 tx=0x7f785402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.591+0000 7f7869600640 1 -- 172.21.15.59:0/1699956510 shutdown_connections 2024-04-23T14:41:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.591+0000 7f7869600640 1 --2- 172.21.15.59:0/1699956510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f786415d2d0 0x7f78640cffd0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.591+0000 7f7869600640 1 -- 172.21.15.59:0/1699956510 >> 172.21.15.59:0/1699956510 conn(0x7f78640b3ef0 msgr2=0x7f78640b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.591+0000 7f7869600640 1 -- 172.21.15.59:0/1699956510 shutdown_connections 2024-04-23T14:41:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.591+0000 7f7869600640 1 -- 172.21.15.59:0/1699956510 wait complete. 2024-04-23T14:41:07.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.593+0000 7f7869600640 1 Processor -- start 2024-04-23T14:41:07.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.593+0000 7f7869600640 1 -- start start 2024-04-23T14:41:07.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7869600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78640c5ae0 0x7f78641ec670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:07.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7869600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78640d0510 0x7f786415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:07.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7869600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f786415d2d0 0x7f78640c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7869600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f78640b00a0 con 0x7f78640d0510 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7869600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f786409b160 con 0x7f786415d2d0 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7869600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f78641637c0 con 0x7f78640c5ae0 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7863400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78640d0510 0x7f786415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7863400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78640d0510 0x7f786415c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40218/0 (socket says 172.21.15.59:40218) 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7863400640 1 -- 172.21.15.59:0/4128456340 learned_addr learned my addr 172.21.15.59:0/4128456340 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7863e00640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78640c5ae0 0x7f78641ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7868c00640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f786415d2d0 0x7f78640c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7863400640 1 -- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78640c5ae0 msgr2=0x7f78641ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7863400640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78640c5ae0 0x7f78641ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.595+0000 7f7863400640 1 -- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f786415d2d0 msgr2=0x7f78640c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f7863400640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f786415d2d0 0x7f78640c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f7863400640 1 -- 172.21.15.59:0/4128456340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78640c7610 con 0x7f78640d0510 2024-04-23T14:41:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f7863400640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78640d0510 0x7f786415c960 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f785000c430 tx=0x7f785000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f785be00640 1 -- 172.21.15.59:0/4128456340 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850011790 con 0x7f78640d0510 2024-04-23T14:41:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f785be00640 1 -- 172.21.15.59:0/4128456340 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7850017070 con 0x7f78640d0510 2024-04-23T14:41:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f785be00640 1 -- 172.21.15.59:0/4128456340 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850011bb0 con 0x7f78640d0510 2024-04-23T14:41:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78640c78f0 con 0x7f78640d0510 2024-04-23T14:41:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.596+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78640c7eb0 con 0x7f78640d0510 2024-04-23T14:41:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.598+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78640bc700 con 0x7f78640d0510 2024-04-23T14:41:07.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.599+0000 7f785be00640 1 -- 172.21.15.59:0/4128456340 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f785001c460 con 0x7f78640d0510 2024-04-23T14:41:07.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.600+0000 7f785be00640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f784406c920 0x7f784406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:07.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.600+0000 7f785be00640 1 -- 172.21.15.59:0/4128456340 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f785008bb80 con 0x7f78640d0510 2024-04-23T14:41:07.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.600+0000 7f7863e00640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f784406c920 0x7f784406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.601+0000 7f7863e00640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f784406c920 0x7f784406ede0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7854034000 tx=0x7f785402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:07.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.605+0000 7f785be00640 1 -- 172.21.15.59:0/4128456340 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78640bc700 con 0x7f78640d0510 2024-04-23T14:41:07.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.816+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f786404fd90 con 0x7f78640d0510 2024-04-23T14:41:07.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.819+0000 7f785be00640 1 -- 172.21.15.59:0/4128456340 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16909 (secure 0 0 0) 0x7f7850059db0 con 0x7f78640d0510 2024-04-23T14:41:07.820 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:07.821 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:40:25.737792+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.59:6835/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.76:6835/3870021000":"2024-04-24T14:39:01.426690+0000","172.21.15.76:6833/3870021000":"2024-04-24T14:39:01.426690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f784406c920 msgr2=0x7f784406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f784406c920 0x7f784406ede0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7854034000 tx=0x7f785402e000 comp rx=0 tx=0).stop 2024-04-23T14:41:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78640d0510 msgr2=0x7f786415c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78640d0510 0x7f786415c960 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f785000c430 tx=0x7f785000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 shutdown_connections 2024-04-23T14:41:07.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f784406c920 0x7f784406ede0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f786415d2d0 0x7f78640c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78640d0510 0x7f786415c960 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 --2- 172.21.15.59:0/4128456340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78640c5ae0 0x7f78641ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 >> 172.21.15.59:0/4128456340 conn(0x7f78640b3ef0 msgr2=0x7f78640b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 shutdown_connections 2024-04-23T14:41:07.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:07.822+0000 7f7869600640 1 -- 172.21.15.59:0/4128456340 wait complete. 2024-04-23T14:41:07.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:38:36.964433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:38:38.188587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:07.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:08.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.060+0000 7f392fe00640 1 Processor -- start 2024-04-23T14:41:08.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.060+0000 7f392fe00640 1 -- start start 2024-04-23T14:41:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.062+0000 7f392fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 0x7f3930160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.062+0000 7f392fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393015cab0 con 0x7f39301567e0 2024-04-23T14:41:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.062+0000 7f392fe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3930160e40 con 0x7f39300d0d10 2024-04-23T14:41:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.062+0000 7f392fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f393015cd30 con 0x7f393015d810 2024-04-23T14:41:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.062+0000 7f392ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f39301567e0 0x7f39300d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42650/0 (socket says 172.21.15.59:42650) 2024-04-23T14:41:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.062+0000 7f392ea00640 1 -- 172.21.15.59:0/2624914074 learned_addr learned my addr 172.21.15.59:0/2624914074 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127931345 0 0) 0x7f3920001490 con 0x7f39301567e0 2024-04-23T14:41:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3914003520 con 0x7f39301567e0 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392f400640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 0x7f3930160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3698401345 0 0) 0x7f3914003520 con 0x7f39301567e0 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 msgr2=0x7f3930160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 0x7f3930160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 >> v1:172.21.15.76:6789/0 conn(0x7f39300d0d10 legacy=0x7f3930160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393015d010 con 0x7f39301567e0 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f3920001bc0 con 0x7f39301567e0 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 >> v1:172.21.15.59:6789/0 conn(0x7f39301567e0 legacy=0x7f39300d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39140039e0 0x7f3914003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f3914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 0x7f3914005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f393015cd30 con 0x7f3914004800 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f391400baf0 con 0x7f39140039e0 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f391400bcf0 con 0x7f393015d810 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.063+0000 7f392e000640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f3914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.064+0000 7f392ea00640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39140039e0 0x7f3914003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.064+0000 7f392f400640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 0x7f3914005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.064+0000 7f392e000640 1 -- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 msgr2=0x7f3914005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.064+0000 7f392e000640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393015d810 0x7f3914005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.064+0000 7f392e000640 1 -- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39140039e0 msgr2=0x7f3914003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.064+0000 7f392e000640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39140039e0 0x7f3914003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.064+0000 7f392e000640 1 -- 172.21.15.59:0/2624914074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3914003520 con 0x7f3914004800 2024-04-23T14:41:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.065+0000 7f392e000640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f3914004c80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f39300afdb0 tx=0x7f3914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2dd0c06b12a2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.065+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918001370 con 0x7f3914004800 2024-04-23T14:41:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.065+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39180052d0 con 0x7f3914004800 2024-04-23T14:41:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.065+0000 7f392d600640 1 -- 172.21.15.59:0/2624914074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391802ab90 con 0x7f3914004800 2024-04-23T14:41:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.066+0000 7f392fe00640 1 -- 172.21.15.59:0/2624914074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 msgr2=0x7f3914004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.066+0000 7f392fe00640 1 --2- 172.21.15.59:0/2624914074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f3914004c80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f39300afdb0 tx=0x7f3914005830 comp rx=0 tx=0).stop 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.066+0000 7f392f400640 1 -- 172.21.15.59:0/2624914074 reap_dead start 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.066+0000 7f392fe00640 1 -- 172.21.15.59:0/2624914074 shutdown_connections 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.066+0000 7f392fe00640 1 -- 172.21.15.59:0/2624914074 >> 172.21.15.59:0/2624914074 conn(0x7f393006bdf0 msgr2=0x7f393006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.066+0000 7f392fe00640 1 -- 172.21.15.59:0/2624914074 shutdown_connections 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.066+0000 7f392fe00640 1 -- 172.21.15.59:0/2624914074 wait complete. 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.068+0000 7f392fe00640 1 Processor -- start 2024-04-23T14:41:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.068+0000 7f392fe00640 1 -- start start 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.069+0000 7f392fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f39301f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.069+0000 7f392fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393006f140 0x7f393006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.069+0000 7f392fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393014e8e0 0x7f393014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.069+0000 7f392fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39301609a0 con 0x7f3914004800 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.069+0000 7f392fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3930160820 con 0x7f393014e8e0 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.069+0000 7f392fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3930160b20 con 0x7f393006f140 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f39301f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f39301f46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40224/0 (socket says 172.21.15.59:40224) 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 -- 172.21.15.59:0/2330911742 learned_addr learned my addr 172.21.15.59:0/2330911742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392f400640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393014e8e0 0x7f393014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392e000640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393006f140 0x7f393006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 -- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393006f140 msgr2=0x7f393006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393006f140 0x7f393006f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 -- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393014e8e0 msgr2=0x7f393014ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393014e8e0 0x7f393014ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.070+0000 7f392ea00640 1 -- 172.21.15.59:0/2330911742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393014fec0 con 0x7f3914004800 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.071+0000 7f392ea00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f39301f46c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f39200079e0 tx=0x7f392000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.071+0000 7f3927400640 1 -- 172.21.15.59:0/2330911742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920017020 con 0x7f3914004800 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.071+0000 7f3927400640 1 -- 172.21.15.59:0/2330911742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392000edf0 con 0x7f3914004800 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.071+0000 7f3927400640 1 -- 172.21.15.59:0/2330911742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920013640 con 0x7f3914004800 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.071+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393014f620 con 0x7f3914004800 2024-04-23T14:41:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.071+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393014fbe0 con 0x7f3914004800 2024-04-23T14:41:08.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.072+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393004fd90 con 0x7f3914004800 2024-04-23T14:41:08.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.073+0000 7f3927400640 1 -- 172.21.15.59:0/2330911742 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f392001a020 con 0x7f3914004800 2024-04-23T14:41:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.075+0000 7f3927400640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f390c06c920 0x7f390c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.075+0000 7f3927400640 1 -- 172.21.15.59:0/2330911742 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f392008c470 con 0x7f3914004800 2024-04-23T14:41:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.075+0000 7f392e000640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f390c06c920 0x7f390c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.076+0000 7f392e000640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f390c06c920 0x7f390c06ede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3918032000 tx=0x7f391802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.079+0000 7f3927400640 1 -- 172.21.15.59:0/2330911742 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3920021020 con 0x7f3914004800 2024-04-23T14:41:08.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.283+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f393015a460 con 0x7f3914004800 2024-04-23T14:41:08.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.284+0000 7f3927400640 1 -- 172.21.15.59:0/2330911742 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f393015cab0 con 0x7f3914004800 2024-04-23T14:41:08.285 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f390c06c920 msgr2=0x7f390c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f390c06c920 0x7f390c06ede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3918032000 tx=0x7f391802e000 comp rx=0 tx=0).stop 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 msgr2=0x7f39301f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f39301f46c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f39200079e0 tx=0x7f392000e530 comp rx=0 tx=0).stop 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 shutdown_connections 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f390c06c920 0x7f390c06ede0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393006f140 0x7f393006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393014e8e0 0x7f393014ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 --2- 172.21.15.59:0/2330911742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3914004800 0x7f39301f46c0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 >> 172.21.15.59:0/2330911742 conn(0x7f393006bdf0 msgr2=0x7f393015b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.286+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 shutdown_connections 2024-04-23T14:41:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.287+0000 7f392fe00640 1 -- 172.21.15.59:0/2330911742 wait complete. 2024-04-23T14:41:08.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:41:08.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.520+0000 7f1029400640 1 Processor -- start 2024-04-23T14:41:08.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.520+0000 7f1029400640 1 -- start start 2024-04-23T14:41:08.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.522+0000 7f1029400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1024162c10 0x7f10240d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.522+0000 7f1029400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10240b4670 0x7f1024071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.522+0000 7f1029400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10240d0fd0 con 0x7f10240cfba0 2024-04-23T14:41:08.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.522+0000 7f1029400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f10240b35d0 con 0x7f10240b4670 2024-04-23T14:41:08.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.522+0000 7f1029400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f10240b38b0 con 0x7f1024162c10 2024-04-23T14:41:08.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1021e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10240b4670 0x7f1024071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1021e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10240b4670 0x7f1024071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40236/0 (socket says 172.21.15.59:40236) 2024-04-23T14:41:08.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1021e00640 1 -- 172.21.15.59:0/3818379739 learned_addr learned my addr 172.21.15.59:0/3818379739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1022800640 1 --2- 172.21.15.59:0/3818379739 >> v2:172.21.15.76:3300/0 conn(0x7f1024162c10 0x7f10240d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1021e00640 1 -- 172.21.15.59:0/3818379739 >> v2:172.21.15.76:3300/0 conn(0x7f1024162c10 msgr2=0x7f10240d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1021e00640 1 --2- 172.21.15.59:0/3818379739 >> v2:172.21.15.76:3300/0 conn(0x7f1024162c10 0x7f10240d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1021e00640 1 -- 172.21.15.59:0/3818379739 >> v1:172.21.15.76:6789/0 conn(0x7f10240cfba0 legacy=0x7f1024166840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.523+0000 7f1021e00640 1 -- 172.21.15.59:0/3818379739 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10240b3b90 con 0x7f10240b4670 2024-04-23T14:41:08.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.525+0000 7f1021e00640 1 --2- 172.21.15.59:0/3818379739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10240b4670 0x7f1024071d70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f10240d1200 tx=0x7f100c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d04f4f26cf2185 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.525+0000 7f1021400640 1 -- 172.21.15.59:0/3818379739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c037070 con 0x7f10240b4670 2024-04-23T14:41:08.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.525+0000 7f1021400640 1 -- 172.21.15.59:0/3818379739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c032440 con 0x7f10240b4670 2024-04-23T14:41:08.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.525+0000 7f1021400640 1 -- 172.21.15.59:0/3818379739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c032810 con 0x7f10240b4670 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.525+0000 7f1029400640 1 -- 172.21.15.59:0/3818379739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10240b4670 msgr2=0x7f1024071d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.525+0000 7f1029400640 1 --2- 172.21.15.59:0/3818379739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10240b4670 0x7f1024071d70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f10240d1200 tx=0x7f100c02a270 comp rx=0 tx=0).stop 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.526+0000 7f1029400640 1 -- 172.21.15.59:0/3818379739 shutdown_connections 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.526+0000 7f1029400640 1 --2- 172.21.15.59:0/3818379739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10240b4670 0x7f1024071d70 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.526+0000 7f1029400640 1 --2- 172.21.15.59:0/3818379739 >> v2:172.21.15.76:3300/0 conn(0x7f1024162c10 0x7f10240d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.526+0000 7f1029400640 1 -- 172.21.15.59:0/3818379739 >> 172.21.15.59:0/3818379739 conn(0x7f1024067d90 msgr2=0x7f102406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.526+0000 7f1029400640 1 -- 172.21.15.59:0/3818379739 shutdown_connections 2024-04-23T14:41:08.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.526+0000 7f1029400640 1 -- 172.21.15.59:0/3818379739 wait complete. 2024-04-23T14:41:08.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.528+0000 7f1029400640 1 Processor -- start 2024-04-23T14:41:08.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.528+0000 7f1029400640 1 -- start start 2024-04-23T14:41:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1029400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10240b4670 0x7f10240b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1029400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10240cfba0 0x7f102415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1029400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1024162c10 0x7f10240b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1029400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1024162ff0 con 0x7f1024162c10 2024-04-23T14:41:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1029400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10240b4150 con 0x7f10240cfba0 2024-04-23T14:41:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1029400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1024163170 con 0x7f10240b4670 2024-04-23T14:41:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1023200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1024162c10 0x7f10240b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1023200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1024162c10 0x7f10240b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40252/0 (socket says 172.21.15.59:40252) 2024-04-23T14:41:08.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1023200640 1 -- 172.21.15.59:0/4091470251 learned_addr learned my addr 172.21.15.59:0/4091470251 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1021e00640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10240cfba0 0x7f102415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1022800640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10240b4670 0x7f10240b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1023200640 1 -- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10240b4670 msgr2=0x7f10240b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.529+0000 7f1023200640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10240b4670 0x7f10240b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f1023200640 1 -- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10240cfba0 msgr2=0x7f102415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f1023200640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10240cfba0 0x7f102415ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f1023200640 1 -- 172.21.15.59:0/4091470251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10240b2de0 con 0x7f1024162c10 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f1023200640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1024162c10 0x7f10240b0e20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c3a0 tx=0x7f101c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f101aa00640 1 -- 172.21.15.59:0/4091470251 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c017070 con 0x7f1024162c10 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f101aa00640 1 -- 172.21.15.59:0/4091470251 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c0052d0 con 0x7f1024162c10 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f101aa00640 1 -- 172.21.15.59:0/4091470251 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c003830 con 0x7f1024162c10 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10240b1800 con 0x7f1024162c10 2024-04-23T14:41:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.530+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10240b1dc0 con 0x7f1024162c10 2024-04-23T14:41:08.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.532+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1024068a80 con 0x7f1024162c10 2024-04-23T14:41:08.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.533+0000 7f101aa00640 1 -- 172.21.15.59:0/4091470251 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f101c01a070 con 0x7f1024162c10 2024-04-23T14:41:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.534+0000 7f101aa00640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f100006c940 0x7f100006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.534+0000 7f101aa00640 1 -- 172.21.15.59:0/4091470251 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f101c08afa0 con 0x7f1024162c10 2024-04-23T14:41:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.534+0000 7f1022800640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f100006c940 0x7f100006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.535+0000 7f1022800640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f100006c940 0x7f100006ee00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1014007380 tx=0x7f101400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.538+0000 7f101aa00640 1 -- 172.21.15.59:0/4091470251 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10240d0fd0 con 0x7f1024162c10 2024-04-23T14:41:08.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.738+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10240b2480 con 0x7f1024162c10 2024-04-23T14:41:08.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.739+0000 7f101aa00640 1 -- 172.21.15.59:0/4091470251 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f101c059430 con 0x7f1024162c10 2024-04-23T14:41:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f100006c940 msgr2=0x7f100006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f100006c940 0x7f100006ee00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1014007380 tx=0x7f101400b040 comp rx=0 tx=0).stop 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1024162c10 msgr2=0x7f10240b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1024162c10 0x7f10240b0e20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c3a0 tx=0x7f101c00c860 comp rx=0 tx=0).stop 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 shutdown_connections 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f100006c940 0x7f100006ee00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1024162c10 0x7f10240b0e20 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10240cfba0 0x7f102415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 --2- 172.21.15.59:0/4091470251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10240b4670 0x7f10240b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.741+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 >> 172.21.15.59:0/4091470251 conn(0x7f1024067d90 msgr2=0x7f102406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:08.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.742+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 shutdown_connections 2024-04-23T14:41:08.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.742+0000 7f1029400640 1 -- 172.21.15.59:0/4091470251 wait complete. 2024-04-23T14:41:08.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:41:08.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.960+0000 7f9d9a600640 1 Processor -- start 2024-04-23T14:41:08.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.960+0000 7f9d9a600640 1 -- start start 2024-04-23T14:41:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.962+0000 7f9d9a600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9d940cd5f0 0x7f9d940cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.962+0000 7f9d9a600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d940cd9f0 con 0x7f9d9406c0e0 2024-04-23T14:41:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.962+0000 7f9d9a600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d940ce390 con 0x7f9d9415e920 2024-04-23T14:41:08.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.962+0000 7f9d9a600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9d940b0bb0 con 0x7f9d940cd5f0 2024-04-23T14:41:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.962+0000 7f9d8be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9d9406c0e0 0x7f9d94166650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42656/0 (socket says 172.21.15.59:42656) 2024-04-23T14:41:08.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.962+0000 7f9d8be00640 1 -- 172.21.15.59:0/4105981189 learned_addr learned my addr 172.21.15.59:0/4105981189 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:08.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d99c00640 1 --2- 172.21.15.59:0/4105981189 >> v2:172.21.15.76:3300/0 conn(0x7f9d940cd5f0 0x7f9d940cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1088254273 0 0) 0x7f9d940cd9f0 con 0x7f9d9406c0e0 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d74003520 con 0x7f9d9406c0e0 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3545926116 0 0) 0x7f9d74003520 con 0x7f9d9406c0e0 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 >> v2:172.21.15.76:3300/0 conn(0x7f9d940cd5f0 msgr2=0x7f9d940cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 --2- 172.21.15.59:0/4105981189 >> v2:172.21.15.76:3300/0 conn(0x7f9d940cd5f0 0x7f9d940cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 >> v1:172.21.15.76:6789/0 conn(0x7f9d9415e920 legacy=0x7f9d9406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d940ce670 con 0x7f9d9406c0e0 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9d7c001670 con 0x7f9d9406c0e0 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 >> v1:172.21.15.59:6789/0 conn(0x7f9d9406c0e0 legacy=0x7f9d94166650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740039b0 0x7f9d74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d74004800 0x7f9d74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d740053c0 0x7f9d7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7400c220 con 0x7f9d74004800 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7400c3d0 con 0x7f9d740039b0 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.963+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7400c5d0 con 0x7f9d740053c0 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d8be00640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d74004800 0x7f9d74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d99200640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740039b0 0x7f9d74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d8be00640 1 -- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d740053c0 msgr2=0x7f9d7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d8be00640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d740053c0 0x7f9d7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d8be00640 1 -- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740039b0 msgr2=0x7f9d74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d8be00640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740039b0 0x7f9d74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d8be00640 1 -- 172.21.15.59:0/4105981189 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d74003520 con 0x7f9d74004800 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d99200640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740039b0 0x7f9d74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:08.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.964+0000 7f9d99c00640 1 -- 172.21.15.59:0/4105981189 reap_dead start 2024-04-23T14:41:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.965+0000 7f9d8be00640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d74004800 0x7f9d74004c80 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00c210 tx=0x7f9d74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7816a5d54d69f3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.965+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c007d30 con 0x7f9d74004800 2024-04-23T14:41:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.965+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c005ad0 con 0x7f9d74004800 2024-04-23T14:41:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.965+0000 7f9d93e00640 1 -- 172.21.15.59:0/4105981189 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c007070 con 0x7f9d74004800 2024-04-23T14:41:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.966+0000 7f9d9a600640 1 -- 172.21.15.59:0/4105981189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d74004800 msgr2=0x7f9d74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.966+0000 7f9d9a600640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d74004800 0x7f9d74004c80 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00c210 tx=0x7f9d74005830 comp rx=0 tx=0).stop 2024-04-23T14:41:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.966+0000 7f9d9a600640 1 -- 172.21.15.59:0/4105981189 shutdown_connections 2024-04-23T14:41:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.966+0000 7f9d9a600640 1 --2- 172.21.15.59:0/4105981189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d74004800 0x7f9d74004c80 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.966+0000 7f9d9a600640 1 -- 172.21.15.59:0/4105981189 >> 172.21.15.59:0/4105981189 conn(0x7f9d940b2f10 msgr2=0x7f9d940b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.966+0000 7f9d9a600640 1 -- 172.21.15.59:0/4105981189 shutdown_connections 2024-04-23T14:41:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.966+0000 7f9d9a600640 1 -- 172.21.15.59:0/4105981189 wait complete. 2024-04-23T14:41:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.968+0000 7f9d9a600640 1 Processor -- start 2024-04-23T14:41:08.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.968+0000 7f9d9a600640 1 -- start start 2024-04-23T14:41:08.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d9a600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d74004800 0x7f9d941f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d9a600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d940cad00 0x7f9d940af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d9a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d9414b4b0 0x7f9d9414b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d9a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9415ed00 con 0x7f9d9414b4b0 2024-04-23T14:41:08.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d9a600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d940cec30 con 0x7f9d74004800 2024-04-23T14:41:08.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d9a600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9415ee80 con 0x7f9d940cad00 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d99c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d9414b4b0 0x7f9d9414b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d99c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d9414b4b0 0x7f9d9414b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40266/0 (socket says 172.21.15.59:40266) 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.969+0000 7f9d99c00640 1 -- 172.21.15.59:0/2617072462 learned_addr learned my addr 172.21.15.59:0/2617072462 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d99200640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d74004800 0x7f9d941f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d99c00640 1 -- 172.21.15.59:0/2617072462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d940cad00 msgr2=0x7f9d940af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d99c00640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d940cad00 0x7f9d940af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d99c00640 1 -- 172.21.15.59:0/2617072462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d74004800 msgr2=0x7f9d941f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d99c00640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d74004800 0x7f9d941f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d99c00640 1 -- 172.21.15.59:0/2617072462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d940cb180 con 0x7f9d9414b4b0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d99c00640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d9414b4b0 0x7f9d9414b930 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9d940d00f0 tx=0x7f9d8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d91600640 1 -- 172.21.15.59:0/2617072462 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00ed30 con 0x7f9d9414b4b0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d91600640 1 -- 172.21.15.59:0/2617072462 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8c019070 con 0x7f9d9414b4b0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d91600640 1 -- 172.21.15.59:0/2617072462 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c014660 con 0x7f9d9414b4b0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.970+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d940c8430 con 0x7f9d9414b4b0 2024-04-23T14:41:08.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.971+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d940aeda0 con 0x7f9d9414b4b0 2024-04-23T14:41:08.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.972+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9404fd90 con 0x7f9d9414b4b0 2024-04-23T14:41:08.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.973+0000 7f9d91600640 1 -- 172.21.15.59:0/2617072462 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d8c014800 con 0x7f9d9414b4b0 2024-04-23T14:41:08.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.974+0000 7f9d91600640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d7006c920 0x7f9d7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:08.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.974+0000 7f9d91600640 1 -- 172.21.15.59:0/2617072462 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f9d8c08c1d0 con 0x7f9d9414b4b0 2024-04-23T14:41:08.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.975+0000 7f9d99200640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d7006c920 0x7f9d7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:08.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.975+0000 7f9d99200640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d7006c920 0x7f9d7006ede0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000c00 tx=0x7f9d84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:08.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:08.979+0000 7f9d91600640 1 -- 172.21.15.59:0/2617072462 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d8c05a490 con 0x7f9d9414b4b0 2024-04-23T14:41:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.187+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9d9406c880 con 0x7f9d9414b4b0 2024-04-23T14:41:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.188+0000 7f9d91600640 1 -- 172.21.15.59:0/2617072462 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f9d8c059d90 con 0x7f9d9414b4b0 2024-04-23T14:41:09.189 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:41:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.190+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d7006c920 msgr2=0x7f9d7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.190+0000 7f9d9a600640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d7006c920 0x7f9d7006ede0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000c00 tx=0x7f9d84009040 comp rx=0 tx=0).stop 2024-04-23T14:41:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.190+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d9414b4b0 msgr2=0x7f9d9414b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.190+0000 7f9d9a600640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d9414b4b0 0x7f9d9414b930 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9d940d00f0 tx=0x7f9d8c00e4b0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 shutdown_connections 2024-04-23T14:41:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9d7006c920 0x7f9d7006ede0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d940cad00 0x7f9d940af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d9414b4b0 0x7f9d9414b930 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 --2- 172.21.15.59:0/2617072462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d74004800 0x7f9d941f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 >> 172.21.15.59:0/2617072462 conn(0x7f9d940b2f10 msgr2=0x7f9d9406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 shutdown_connections 2024-04-23T14:41:09.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.191+0000 7f9d9a600640 1 -- 172.21.15.59:0/2617072462 wait complete. 2024-04-23T14:41:09.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:41:09.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.425+0000 7fe1be000640 1 Processor -- start 2024-04-23T14:41:09.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.425+0000 7fe1be000640 1 -- start start 2024-04-23T14:41:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.427+0000 7fe1be000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b80d0cd0 0x7fe1b81674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.427+0000 7fe1be000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b80b4cf0 con 0x7fe1b80cd8b0 2024-04-23T14:41:09.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.427+0000 7fe1be000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b80ccdd0 con 0x7fe1b815c670 2024-04-23T14:41:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.427+0000 7fe1be000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b815ca50 con 0x7fe1b80d0cd0 2024-04-23T14:41:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.427+0000 7fe1afe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe1b80cd8b0 0x7fe1b80d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42666/0 (socket says 172.21.15.59:42666) 2024-04-23T14:41:09.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.427+0000 7fe1afe00640 1 -- 172.21.15.59:0/660548800 learned_addr learned my addr 172.21.15.59:0/660548800 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1bd600640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b80d0cd0 0x7fe1b81674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773808922 0 0) 0x7fe1b80b4cf0 con 0x7fe1b80cd8b0 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe19c003520 con 0x7fe1b80cd8b0 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 559496336 0 0) 0x7fe19c003520 con 0x7fe1b80cd8b0 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b80d0cd0 msgr2=0x7fe1b81674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b80d0cd0 0x7fe1b81674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 >> v1:172.21.15.76:6789/0 conn(0x7fe1b815c670 legacy=0x7fe1b80b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b80cd0b0 con 0x7fe1b80cd8b0 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fe1a0001670 con 0x7fe1b80cd8b0 2024-04-23T14:41:09.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.428+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 >> v1:172.21.15.59:6789/0 conn(0x7fe1b80cd8b0 legacy=0x7fe1b80d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1b7e00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe19c0039e0 0x7fe19c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1b7e00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c004800 0x7fe19c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1b7e00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c0051c0 0x7fe19c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0103f0 con 0x7fe19c004800 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0105a0 con 0x7fe19c0051c0 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c0107a0 con 0x7fe19c0039e0 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1afe00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c004800 0x7fe19c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1bcc00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe19c0039e0 0x7fe19c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1bd600640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c0051c0 0x7fe19c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1afe00640 1 -- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe19c0039e0 msgr2=0x7fe19c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1afe00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe19c0039e0 0x7fe19c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1afe00640 1 -- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c0051c0 msgr2=0x7fe19c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1afe00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c0051c0 0x7fe19c00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1afe00640 1 -- 172.21.15.59:0/660548800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c003520 con 0x7fe19c004800 2024-04-23T14:41:09.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1bd600640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c0051c0 0x7fe19c00fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:09.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.429+0000 7fe1bcc00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe19c0039e0 0x7fe19c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:09.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.430+0000 7fe1afe00640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c004800 0x7fe19c004c80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007fd0 tx=0x7fe19c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c6a06a0d815a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0007c20 con 0x7fe19c004800 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a0005af0 con 0x7fe19c004800 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1b7e00640 1 -- 172.21.15.59:0/660548800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0007070 con 0x7fe19c004800 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1be000640 1 -- 172.21.15.59:0/660548800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c004800 msgr2=0x7fe19c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1be000640 1 --2- 172.21.15.59:0/660548800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe19c004800 0x7fe19c004c80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007fd0 tx=0x7fe19c005830 comp rx=0 tx=0).stop 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1bd600640 1 -- 172.21.15.59:0/660548800 reap_dead start 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1be000640 1 -- 172.21.15.59:0/660548800 shutdown_connections 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1be000640 1 -- 172.21.15.59:0/660548800 >> 172.21.15.59:0/660548800 conn(0x7fe1b806bdf0 msgr2=0x7fe1b806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1be000640 1 -- 172.21.15.59:0/660548800 shutdown_connections 2024-04-23T14:41:09.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.431+0000 7fe1be000640 1 -- 172.21.15.59:0/660548800 wait complete. 2024-04-23T14:41:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.433+0000 7fe1be000640 1 Processor -- start 2024-04-23T14:41:09.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.433+0000 7fe1be000640 1 -- start start 2024-04-23T14:41:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.434+0000 7fe1be000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c004800 0x7fe1b8155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.434+0000 7fe1be000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b809cb70 0x7fe1b8143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.434+0000 7fe1be000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b8141570 0x7fe1b8159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.434+0000 7fe1be000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b80b3e60 con 0x7fe1b809cb70 2024-04-23T14:41:09.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.434+0000 7fe1be000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b80ccb80 con 0x7fe19c004800 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.434+0000 7fe1be000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b80b3fe0 con 0x7fe1b8141570 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.434+0000 7fe1afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b809cb70 0x7fe1b8143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b809cb70 0x7fe1b8143870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40282/0 (socket says 172.21.15.59:40282) 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 -- 172.21.15.59:0/4065577742 learned_addr learned my addr 172.21.15.59:0/4065577742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1bd600640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b8141570 0x7fe1b8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 -- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b8141570 msgr2=0x7fe1b8159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1bcc00640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c004800 0x7fe1b8155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b8141570 0x7fe1b8159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 -- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c004800 msgr2=0x7fe1b8155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c004800 0x7fe1b8155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 -- 172.21.15.59:0/4065577742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b81588a0 con 0x7fe1b809cb70 2024-04-23T14:41:09.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.435+0000 7fe1afe00640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b809cb70 0x7fe1b8143870 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c1a0 tx=0x7fe1b80b4ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:09.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.436+0000 7fe1b5600640 1 -- 172.21.15.59:0/4065577742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0037040 con 0x7fe1b809cb70 2024-04-23T14:41:09.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.436+0000 7fe1b5600640 1 -- 172.21.15.59:0/4065577742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a0041070 con 0x7fe1b809cb70 2024-04-23T14:41:09.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.436+0000 7fe1b5600640 1 -- 172.21.15.59:0/4065577742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0004a00 con 0x7fe1b809cb70 2024-04-23T14:41:09.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.436+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b8158b80 con 0x7fe1b809cb70 2024-04-23T14:41:09.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.436+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b8159140 con 0x7fe1b809cb70 2024-04-23T14:41:09.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.437+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b804fd90 con 0x7fe1b809cb70 2024-04-23T14:41:09.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.438+0000 7fe1b5600640 1 -- 172.21.15.59:0/4065577742 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1a0004ba0 con 0x7fe1b809cb70 2024-04-23T14:41:09.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.439+0000 7fe1b5600640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:09.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.439+0000 7fe1b5600640 1 -- 172.21.15.59:0/4065577742 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fe1a00ab9f0 con 0x7fe1b809cb70 2024-04-23T14:41:09.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.439+0000 7fe1bcc00640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:09.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.440+0000 7fe1bcc00640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8000c00 tx=0x7fe1a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:09.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.443+0000 7fe1b5600640 1 -- 172.21.15.59:0/4065577742 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1a0079cb0 con 0x7fe1b809cb70 2024-04-23T14:41:09.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.666+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe1b80cfa20 con 0x7fe1b809cb70 2024-04-23T14:41:09.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.667+0000 7fe1b5600640 1 -- 172.21.15.59:0/4065577742 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v87) ==== 80+0+244 (secure 0 0 0) 0x7fe1a00795b0 con 0x7fe1b809cb70 2024-04-23T14:41:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6837/2128549219 2024-04-24T14:40:25.737762+0000 2024-04-23T14:41:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6835/2128549219 2024-04-24T14:40:25.737762+0000 2024-04-23T14:41:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.76:6835/3870021000 2024-04-24T14:39:01.426690+0000 2024-04-23T14:41:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.76:6833/3870021000 2024-04-24T14:39:01.426690+0000 2024-04-23T14:41:09.668 INFO:teuthology.orchestra.run.smithi059.stderr:listed 4 entries 2024-04-23T14:41:09.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe19406c8f0 msgr2=0x7fe19406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8000c00 tx=0x7fe1a8009040 comp rx=0 tx=0).stop 2024-04-23T14:41:09.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b809cb70 msgr2=0x7fe1b8143870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:09.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b809cb70 0x7fe1b8143870 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000c1a0 tx=0x7fe1b80b4ec0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 shutdown_connections 2024-04-23T14:41:09.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1b8141570 0x7fe1b8159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1b809cb70 0x7fe1b8143870 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 --2- 172.21.15.59:0/4065577742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe19c004800 0x7fe1b8155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:09.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 >> 172.21.15.59:0/4065577742 conn(0x7fe1b806bdf0 msgr2=0x7fe1b809dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:09.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 shutdown_connections 2024-04-23T14:41:09.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:09.670+0000 7fe1be000640 1 -- 172.21.15.59:0/4065577742 wait complete. 2024-04-23T14:41:09.690 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:41:09.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:09.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:09.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[67104]: fuse finished with error 0 and tester_r 0 2024-04-23T14:41:09.768 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:41:09.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:09.768 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:41:09.945 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:41:09.945 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:41:09.945 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:10.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:41:10.004 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:41:10.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:10.005 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:41:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:41:10.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:41:10.057 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:41:10.058 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:10.058 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:10.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:41:10.116 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:41:10.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:10.116 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:41:10.172 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:41:10.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:41:10.174 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:41:10.174 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:41:10.174 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:10.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:41:10.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:41:10.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:10.233 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:41:10.283 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:41:10.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:41:10.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:10.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.503+0000 7f79cfa00640 1 Processor -- start 2024-04-23T14:41:10.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.504+0000 7f79cfa00640 1 -- start start 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.505+0000 7f79cfa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f79c81454e0 0x7f79c8145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.505+0000 7f79cfa00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f79c8160910 0x7f79c8160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.505+0000 7f79cfa00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c80cd520 con 0x7f79c80cd020 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.505+0000 7f79cfa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f79c80cddc0 con 0x7f79c81454e0 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.505+0000 7f79cfa00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f79c80cd800 con 0x7f79c8160910 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cd000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f79c81454e0 0x7f79c8145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cd000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f79c81454e0 0x7f79c8145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46426/0 (socket says 172.21.15.59:46426) 2024-04-23T14:41:10.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cd000640 1 -- 172.21.15.59:0/2903693404 learned_addr learned my addr 172.21.15.59:0/2903693404 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cda00640 1 --2- 172.21.15.59:0/2903693404 >> v2:172.21.15.76:3300/0 conn(0x7f79c8160910 0x7f79c8160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cd000640 1 -- 172.21.15.59:0/2903693404 >> v2:172.21.15.76:3300/0 conn(0x7f79c8160910 msgr2=0x7f79c8160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cd000640 1 --2- 172.21.15.59:0/2903693404 >> v2:172.21.15.76:3300/0 conn(0x7f79c8160910 0x7f79c8160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cd000640 1 -- 172.21.15.59:0/2903693404 >> v1:172.21.15.76:6789/0 conn(0x7f79c80cd020 legacy=0x7f79c80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:10.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.506+0000 7f79cd000640 1 -- 172.21.15.59:0/2903693404 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c80cdae0 con 0x7f79c81454e0 2024-04-23T14:41:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.507+0000 7f79cd000640 1 --2- 172.21.15.59:0/2903693404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c81454e0 0x7f79c8145bb0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f79c80702d0 tx=0x7f79b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22164647c0678d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79c7400640 1 -- 172.21.15.59:0/2903693404 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8036070 con 0x7f79c81454e0 2024-04-23T14:41:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79c7400640 1 -- 172.21.15.59:0/2903693404 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b8031600 con 0x7f79c81454e0 2024-04-23T14:41:10.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79c7400640 1 -- 172.21.15.59:0/2903693404 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b80319d0 con 0x7f79c81454e0 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79cfa00640 1 -- 172.21.15.59:0/2903693404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c81454e0 msgr2=0x7f79c8145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79cfa00640 1 --2- 172.21.15.59:0/2903693404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c81454e0 0x7f79c8145bb0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f79c80702d0 tx=0x7f79b802a660 comp rx=0 tx=0).stop 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79cfa00640 1 -- 172.21.15.59:0/2903693404 shutdown_connections 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79cfa00640 1 --2- 172.21.15.59:0/2903693404 >> v2:172.21.15.76:3300/0 conn(0x7f79c8160910 0x7f79c8160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.508+0000 7f79cfa00640 1 --2- 172.21.15.59:0/2903693404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c81454e0 0x7f79c8145bb0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.509+0000 7f79cfa00640 1 -- 172.21.15.59:0/2903693404 >> 172.21.15.59:0/2903693404 conn(0x7f79c8067ea0 msgr2=0x7f79c806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.509+0000 7f79cfa00640 1 -- 172.21.15.59:0/2903693404 shutdown_connections 2024-04-23T14:41:10.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.509+0000 7f79cfa00640 1 -- 172.21.15.59:0/2903693404 wait complete. 2024-04-23T14:41:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.510+0000 7f79cfa00640 1 Processor -- start 2024-04-23T14:41:10.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.510+0000 7f79cfa00640 1 -- start start 2024-04-23T14:41:10.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.511+0000 7f79cfa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cd020 0x7f79c81f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.511+0000 7f79cfa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79c81454e0 0x7f79c815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.511+0000 7f79cfa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c8160910 0x7f79c8140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.511+0000 7f79cfa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80b4010 con 0x7f79c8160910 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.511+0000 7f79cfa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81460f0 con 0x7f79c80cd020 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.511+0000 7f79cfa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f79c81f8df0 con 0x7f79c81454e0 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c8160910 0x7f79c8140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c8160910 0x7f79c8140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46442/0 (socket says 172.21.15.59:46442) 2024-04-23T14:41:10.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 -- 172.21.15.59:0/1232230720 learned_addr learned my addr 172.21.15.59:0/1232230720 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:10.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cd000640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cd020 0x7f79c81f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:10.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 -- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79c81454e0 msgr2=0x7f79c815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79c81454e0 0x7f79c815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 -- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cd020 msgr2=0x7f79c81f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cd020 0x7f79c81f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 -- 172.21.15.59:0/1232230720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c81f8030 con 0x7f79c8160910 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cd000640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cd020 0x7f79c81f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.512+0000 7f79cda00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c8160910 0x7f79c8140640 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d5d0 tx=0x7f79c000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.513+0000 7f79c4c00640 1 -- 172.21.15.59:0/1232230720 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00125f0 con 0x7f79c8160910 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.513+0000 7f79c4c00640 1 -- 172.21.15.59:0/1232230720 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c0019070 con 0x7f79c8160910 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.513+0000 7f79c4c00640 1 -- 172.21.15.59:0/1232230720 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00129f0 con 0x7f79c8160910 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.513+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c81f8310 con 0x7f79c8160910 2024-04-23T14:41:10.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.513+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c81f88d0 con 0x7f79c8160910 2024-04-23T14:41:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.514+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c815e7a0 con 0x7f79c8160910 2024-04-23T14:41:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.515+0000 7f79c4c00640 1 -- 172.21.15.59:0/1232230720 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79c0012b90 con 0x7f79c8160910 2024-04-23T14:41:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.516+0000 7f79c4c00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79a406c8f0 0x7f79a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.516+0000 7f79c4c00640 1 -- 172.21.15.59:0/1232230720 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f79c008c100 con 0x7f79c8160910 2024-04-23T14:41:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.516+0000 7f79cd000640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79a406c8f0 0x7f79a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:10.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.517+0000 7f79cd000640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79a406c8f0 0x7f79a406edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79b8033000 tx=0x7f79b8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:10.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.520+0000 7f79c4c00640 1 -- 172.21.15.59:0/1232230720 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79c005a3d0 con 0x7f79c8160910 2024-04-23T14:41:10.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.766+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79c8068a80 con 0x7f79c8160910 2024-04-23T14:41:10.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.767+0000 7f79c4c00640 1 -- 172.21.15.59:0/1232230720 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5181 (secure 0 0 0) 0x7f79c0059cd0 con 0x7f79c8160910 2024-04-23T14:41:10.769 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:10.769 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:30.798278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-23T14:41:10.769 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 74 2024-04-23T14:41:10.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.770+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79a406c8f0 msgr2=0x7f79a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:10.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79a406c8f0 0x7f79a406edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79b8033000 tx=0x7f79b8009000 comp rx=0 tx=0).stop 2024-04-23T14:41:10.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c8160910 msgr2=0x7f79c8140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:10.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c8160910 0x7f79c8140640 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d5d0 tx=0x7f79c000da90 comp rx=0 tx=0).stop 2024-04-23T14:41:10.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 shutdown_connections 2024-04-23T14:41:10.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79a406c8f0 0x7f79a406edb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79c81454e0 0x7f79c815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79c8160910 0x7f79c8140640 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 --2- 172.21.15.59:0/1232230720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79c80cd020 0x7f79c81f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:10.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 >> 172.21.15.59:0/1232230720 conn(0x7f79c8067ea0 msgr2=0x7f79c80b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:10.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 shutdown_connections 2024-04-23T14:41:10.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.771+0000 7f79cfa00640 1 -- 172.21.15.59:0/1232230720 wait complete. 2024-04-23T14:41:10.791 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:10.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.997+0000 7f1dcba00640 1 Processor -- start 2024-04-23T14:41:10.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.998+0000 7f1dcba00640 1 -- start start 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.999+0000 7f1dcba00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dc415e750 0x7f1dc40ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.999+0000 7f1dcba00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1dc415eb30 0x7f1dc40cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.999+0000 7f1dcba00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc40cfa20 0x7f1dc40d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.999+0000 7f1dcba00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc40b8fc0 con 0x7f1dc415eb30 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.999+0000 7f1dcba00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc40b92a0 con 0x7f1dc415e750 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:10.999+0000 7f1dcba00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc40b9580 con 0x7f1dc40cfa20 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1dc415eb30 0x7f1dc40cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1dc415eb30 0x7f1dc40cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46448/0 (socket says 172.21.15.59:46448) 2024-04-23T14:41:11.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 -- 172.21.15.59:0/3466044409 learned_addr learned my addr 172.21.15.59:0/3466044409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc9a00640 1 --2- 172.21.15.59:0/3466044409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc40cfa20 0x7f1dc40d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 -- 172.21.15.59:0/3466044409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc40cfa20 msgr2=0x7f1dc40d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc9000640 1 --2- 172.21.15.59:0/3466044409 >> v2:172.21.15.76:3300/0 conn(0x7f1dc415e750 0x7f1dc40ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 --2- 172.21.15.59:0/3466044409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc40cfa20 0x7f1dc40d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 -- 172.21.15.59:0/3466044409 >> v2:172.21.15.76:3300/0 conn(0x7f1dc415e750 msgr2=0x7f1dc40ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 --2- 172.21.15.59:0/3466044409 >> v2:172.21.15.76:3300/0 conn(0x7f1dc415e750 0x7f1dc40ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc3e00640 1 -- 172.21.15.59:0/3466044409 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc40b9860 con 0x7f1dc415eb30 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc9a00640 1 --2- 172.21.15.59:0/3466044409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc40cfa20 0x7f1dc40d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.000+0000 7f1dc9000640 1 --2- 172.21.15.59:0/3466044409 >> v2:172.21.15.76:3300/0 conn(0x7f1dc415e750 0x7f1dc40ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:11.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.001+0000 7f1dc3e00640 1 --2- 172.21.15.59:0/3466044409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc415eb30 0x7f1dc40cf4e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40d11f0 tx=0x7f1db002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e69a70fdb07db79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.002+0000 7f1dc3400640 1 -- 172.21.15.59:0/3466044409 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0037070 con 0x7f1dc415eb30 2024-04-23T14:41:11.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.002+0000 7f1dc3400640 1 -- 172.21.15.59:0/3466044409 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0032440 con 0x7f1dc415eb30 2024-04-23T14:41:11.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.002+0000 7f1dc3400640 1 -- 172.21.15.59:0/3466044409 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0032810 con 0x7f1dc415eb30 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.002+0000 7f1dcba00640 1 -- 172.21.15.59:0/3466044409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc415eb30 msgr2=0x7f1dc40cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.002+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3466044409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc415eb30 0x7f1dc40cf4e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40d11f0 tx=0x7f1db002a1d0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.003+0000 7f1dcba00640 1 -- 172.21.15.59:0/3466044409 shutdown_connections 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.003+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3466044409 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc40cfa20 0x7f1dc40d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.003+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3466044409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc415eb30 0x7f1dc40cf4e0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.003+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3466044409 >> v2:172.21.15.76:3300/0 conn(0x7f1dc415e750 0x7f1dc40ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.003+0000 7f1dcba00640 1 -- 172.21.15.59:0/3466044409 >> 172.21.15.59:0/3466044409 conn(0x7f1dc4067b50 msgr2=0x7f1dc4067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.003+0000 7f1dcba00640 1 -- 172.21.15.59:0/3466044409 shutdown_connections 2024-04-23T14:41:11.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.003+0000 7f1dcba00640 1 -- 172.21.15.59:0/3466044409 wait complete. 2024-04-23T14:41:11.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.004+0000 7f1dcba00640 1 Processor -- start 2024-04-23T14:41:11.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.005+0000 7f1dcba00640 1 -- start start 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.005+0000 7f1dcba00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc40cfa20 0x7f1dc415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.005+0000 7f1dcba00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc415e750 0x7f1dc415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.005+0000 7f1dcba00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc415eb30 0x7f1dc40c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.005+0000 7f1dcba00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40d31b0 con 0x7f1dc40cfa20 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.005+0000 7f1dcba00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40ba6e0 con 0x7f1dc415eb30 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.005+0000 7f1dcba00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc40d3330 con 0x7f1dc415e750 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc40cfa20 0x7f1dc415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc40cfa20 0x7f1dc415cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46460/0 (socket says 172.21.15.59:46460) 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 -- 172.21.15.59:0/3587124792 learned_addr learned my addr 172.21.15.59:0/3587124792 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:11.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9a00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc415eb30 0x7f1dc40c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 -- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc415e750 msgr2=0x7f1dc415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc415e750 0x7f1dc415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 -- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc415eb30 msgr2=0x7f1dc40c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc415eb30 0x7f1dc40c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 -- 172.21.15.59:0/3587124792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41f81e0 con 0x7f1dc40cfa20 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9a00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc415eb30 0x7f1dc40c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.006+0000 7f1dc9000640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc40cfa20 0x7f1dc415cd50 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c7f0 tx=0x7f1db800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.007+0000 7f1dc0c00640 1 -- 172.21.15.59:0/3587124792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8015070 con 0x7f1dc40cfa20 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.007+0000 7f1dc0c00640 1 -- 172.21.15.59:0/3587124792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8007a30 con 0x7f1dc40cfa20 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.007+0000 7f1dc0c00640 1 -- 172.21.15.59:0/3587124792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007e00 con 0x7f1dc40cfa20 2024-04-23T14:41:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.007+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41f84c0 con 0x7f1dc40cfa20 2024-04-23T14:41:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.007+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41f8a80 con 0x7f1dc40cfa20 2024-04-23T14:41:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.008+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc4068a80 con 0x7f1dc40cfa20 2024-04-23T14:41:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.009+0000 7f1dc0c00640 1 -- 172.21.15.59:0/3587124792 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db8018070 con 0x7f1dc40cfa20 2024-04-23T14:41:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.010+0000 7f1dc0c00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1da406c8f0 0x7f1da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.010+0000 7f1dc0c00640 1 -- 172.21.15.59:0/3587124792 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f1db808bbe0 con 0x7f1dc40cfa20 2024-04-23T14:41:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.010+0000 7f1dc3e00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1da406c8f0 0x7f1da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.011+0000 7f1dc3e00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1da406c8f0 0x7f1da406edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1db0034000 tx=0x7f1db002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.014+0000 7f1dc0c00640 1 -- 172.21.15.59:0/3587124792 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1db8059f40 con 0x7f1dc40cfa20 2024-04-23T14:41:11.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.219+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dc40b7900 con 0x7f1dc40cfa20 2024-04-23T14:41:11.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.221+0000 7f1dc0c00640 1 -- 172.21.15.59:0/3587124792 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16909 (secure 0 0 0) 0x7f1db8059840 con 0x7f1dc40cfa20 2024-04-23T14:41:11.222 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:40:25.737792+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.59:6835/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.76:6835/3870021000":"2024-04-24T14:39:01.426690+0000","172.21.15.76:6833/3870021000":"2024-04-24T14:39:01.426690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.223+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1da406c8f0 msgr2=0x7f1da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.223+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1da406c8f0 0x7f1da406edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1db0034000 tx=0x7f1db002c000 comp rx=0 tx=0).stop 2024-04-23T14:41:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.223+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc40cfa20 msgr2=0x7f1dc415cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc40cfa20 0x7f1dc415cd50 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c7f0 tx=0x7f1db800cbf0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 shutdown_connections 2024-04-23T14:41:11.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1da406c8f0 0x7f1da406edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1dc415e750 0x7f1dc415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc415eb30 0x7f1dc40c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 --2- 172.21.15.59:0/3587124792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1dc40cfa20 0x7f1dc415cd50 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 >> 172.21.15.59:0/3587124792 conn(0x7f1dc4067b50 msgr2=0x7f1dc4067f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:11.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 shutdown_connections 2024-04-23T14:41:11.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.224+0000 7f1dcba00640 1 -- 172.21.15.59:0/3587124792 wait complete. 2024-04-23T14:41:11.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:38:36.964433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:38:38.188587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:11.245 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:11.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.455+0000 7fda46400640 1 Processor -- start 2024-04-23T14:41:11.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.455+0000 7fda46400640 1 -- start start 2024-04-23T14:41:11.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.457+0000 7fda46400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda400d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.457+0000 7fda46400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda40072480 0x7fda4006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.457+0000 7fda46400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda400d0fd0 con 0x7fda400b4670 2024-04-23T14:41:11.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.457+0000 7fda46400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fda400b35d0 con 0x7fda40072480 2024-04-23T14:41:11.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.457+0000 7fda46400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda400b38b0 con 0x7fda40162c10 2024-04-23T14:41:11.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3f400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda400d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda40072480 0x7fda4006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3f400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda400d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:36494/0 (socket says 172.21.15.59:36494) 2024-04-23T14:41:11.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3f400640 1 -- 172.21.15.59:0/185948255 learned_addr learned my addr 172.21.15.59:0/185948255 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:11.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda36a00640 1 --1- 172.21.15.59:0/185948255 >> v1:172.21.15.76:6789/0 conn(0x7fda400b4670 0x7fda40071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:44034/0 (socket says 172.21.15.59:44034) 2024-04-23T14:41:11.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3f400640 1 -- 172.21.15.59:0/185948255 >> v2:172.21.15.76:3300/0 conn(0x7fda40072480 msgr2=0x7fda4006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:11.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3f400640 1 --2- 172.21.15.59:0/185948255 >> v2:172.21.15.76:3300/0 conn(0x7fda40072480 0x7fda4006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3f400640 1 -- 172.21.15.59:0/185948255 >> v1:172.21.15.76:6789/0 conn(0x7fda400b4670 legacy=0x7fda40071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.458+0000 7fda3f400640 1 -- 172.21.15.59:0/185948255 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400b3b90 con 0x7fda40162c10 2024-04-23T14:41:11.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.460+0000 7fda3f400640 1 --2- 172.21.15.59:0/185948255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda400d0a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda30009e50 tx=0x7fda3002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3e0cc5f67ab875 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.460+0000 7fda3ea00640 1 -- 172.21.15.59:0/185948255 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30036070 con 0x7fda40162c10 2024-04-23T14:41:11.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.460+0000 7fda3ea00640 1 -- 172.21.15.59:0/185948255 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda30031600 con 0x7fda40162c10 2024-04-23T14:41:11.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.460+0000 7fda3ea00640 1 -- 172.21.15.59:0/185948255 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300319d0 con 0x7fda40162c10 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 -- 172.21.15.59:0/185948255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 msgr2=0x7fda400d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 --2- 172.21.15.59:0/185948255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda400d0a90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fda30009e50 tx=0x7fda3002a710 comp rx=0 tx=0).stop 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 -- 172.21.15.59:0/185948255 shutdown_connections 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 --2- 172.21.15.59:0/185948255 >> v2:172.21.15.76:3300/0 conn(0x7fda40072480 0x7fda4006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 --2- 172.21.15.59:0/185948255 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda400d0a90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 -- 172.21.15.59:0/185948255 >> 172.21.15.59:0/185948255 conn(0x7fda40067d90 msgr2=0x7fda4006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 -- 172.21.15.59:0/185948255 shutdown_connections 2024-04-23T14:41:11.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.461+0000 7fda46400640 1 -- 172.21.15.59:0/185948255 wait complete. 2024-04-23T14:41:11.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.463+0000 7fda46400640 1 Processor -- start 2024-04-23T14:41:11.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.463+0000 7fda46400640 1 -- start start 2024-04-23T14:41:11.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda46400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda40072480 0x7fda400ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda46400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda400b4670 0x7fda401f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda46400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda401f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda46400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda40162ff0 con 0x7fda400b4670 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda46400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda400b4150 con 0x7fda40072480 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda46400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda40163170 con 0x7fda40162c10 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda36a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda400b4670 0x7fda401f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda36a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda400b4670 0x7fda401f8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46464/0 (socket says 172.21.15.59:46464) 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.464+0000 7fda36a00640 1 -- 172.21.15.59:0/661446461 learned_addr learned my addr 172.21.15.59:0/661446461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda36a00640 1 -- 172.21.15.59:0/661446461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 msgr2=0x7fda401f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda36a00640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda401f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda3f400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda40072480 0x7fda400ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda36a00640 1 -- 172.21.15.59:0/661446461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda40072480 msgr2=0x7fda400ab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda36a00640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda40072480 0x7fda400ab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda36a00640 1 -- 172.21.15.59:0/661446461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda400ab7e0 con 0x7fda400b4670 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda36a00640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda400b4670 0x7fda401f8840 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fda2800c400 tx=0x7fda2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda37e00640 1 -- 172.21.15.59:0/661446461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda280074e0 con 0x7fda400b4670 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda37e00640 1 -- 172.21.15.59:0/661446461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda28007680 con 0x7fda400b4670 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda37e00640 1 -- 172.21.15.59:0/661446461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28004a00 con 0x7fda400b4670 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.465+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda400abac0 con 0x7fda400b4670 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.466+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda400ac080 con 0x7fda400b4670 2024-04-23T14:41:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.467+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda40068a80 con 0x7fda400b4670 2024-04-23T14:41:11.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.468+0000 7fda37e00640 1 -- 172.21.15.59:0/661446461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda2801a020 con 0x7fda400b4670 2024-04-23T14:41:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.469+0000 7fda37e00640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda1c06c920 0x7fda1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.469+0000 7fda37e00640 1 -- 172.21.15.59:0/661446461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fda2808b400 con 0x7fda400b4670 2024-04-23T14:41:11.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.470+0000 7fda3f400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda1c06c920 0x7fda1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.470+0000 7fda3f400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda1c06c920 0x7fda1c06ede0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fda30033000 tx=0x7fda30009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.474+0000 7fda37e00640 1 -- 172.21.15.59:0/661446461 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda400d0fd0 con 0x7fda400b4670 2024-04-23T14:41:11.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.676+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda4015e9d0 con 0x7fda400b4670 2024-04-23T14:41:11.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.677+0000 7fda37e00640 1 -- 172.21.15.59:0/661446461 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fda28010300 con 0x7fda400b4670 2024-04-23T14:41:11.678 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:11.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda1c06c920 msgr2=0x7fda1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda1c06c920 0x7fda1c06ede0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fda30033000 tx=0x7fda30009000 comp rx=0 tx=0).stop 2024-04-23T14:41:11.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda400b4670 msgr2=0x7fda401f8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda400b4670 0x7fda401f8840 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fda2800c400 tx=0x7fda2800c8c0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 shutdown_connections 2024-04-23T14:41:11.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fda1c06c920 0x7fda1c06ede0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda40162c10 0x7fda401f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda400b4670 0x7fda401f8840 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 --2- 172.21.15.59:0/661446461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda40072480 0x7fda400ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 >> 172.21.15.59:0/661446461 conn(0x7fda40067d90 msgr2=0x7fda40163fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 shutdown_connections 2024-04-23T14:41:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.680+0000 7fda46400640 1 -- 172.21.15.59:0/661446461 wait complete. 2024-04-23T14:41:11.701 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:41:11.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.917+0000 7fdb21000640 1 Processor -- start 2024-04-23T14:41:11.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.917+0000 7fdb21000640 1 -- start start 2024-04-23T14:41:11.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.918+0000 7fdb21000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb1c0b24b0 0x7fdb1c0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.918+0000 7fdb21000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 0x7fdb1c0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.918+0000 7fdb21000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb1c0b2dd0 con 0x7fdb1c15ed60 2024-04-23T14:41:11.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.918+0000 7fdb21000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdb1c06b9d0 con 0x7fdb1c0b24b0 2024-04-23T14:41:11.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.918+0000 7fdb21000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c06c250 con 0x7fdb1c0cbb90 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb13e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdb1c15ed60 0x7fdb1c0b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58822/0 (socket says 172.21.15.59:58822) 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb13e00640 1 -- 172.21.15.59:0/599970662 learned_addr learned my addr 172.21.15.59:0/599970662 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19e00640 1 --2- 172.21.15.59:0/599970662 >> v2:172.21.15.76:3300/0 conn(0x7fdb1c0b24b0 0x7fdb1c0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb1a800640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 0x7fdb1c0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631861369 0 0) 0x7fdb1c0b2dd0 con 0x7fdb1c15ed60 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb00003520 con 0x7fdb1c15ed60 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1191151616 0 0) 0x7fdb00003520 con 0x7fdb1c15ed60 2024-04-23T14:41:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 msgr2=0x7fdb1c0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 0x7fdb1c0d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 >> v2:172.21.15.76:3300/0 conn(0x7fdb1c0b24b0 msgr2=0x7fdb1c0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 --2- 172.21.15.59:0/599970662 >> v2:172.21.15.76:3300/0 conn(0x7fdb1c0b24b0 0x7fdb1c0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.919+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c0cea70 con 0x7fdb1c15ed60 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fdb04001670 con 0x7fdb1c15ed60 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 >> v1:172.21.15.59:6789/0 conn(0x7fdb1c15ed60 legacy=0x7fdb1c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb000039b0 0x7fdb00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb00004800 0x7fdb00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 0x7fdb000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c06c250 con 0x7fdb00004800 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0000b9c0 con 0x7fdb000039b0 2024-04-23T14:41:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0000bb40 con 0x7fdb1c0cbb90 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb13e00640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb00004800 0x7fdb00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb19e00640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb000039b0 0x7fdb00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb13e00640 1 -- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 msgr2=0x7fdb000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb1a800640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 0x7fdb000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb13e00640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cbb90 0x7fdb000051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb13e00640 1 -- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb000039b0 msgr2=0x7fdb00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.920+0000 7fdb13e00640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb000039b0 0x7fdb00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.921+0000 7fdb13e00640 1 -- 172.21.15.59:0/599970662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb00003520 con 0x7fdb00004800 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.922+0000 7fdb13e00640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb00004800 0x7fdb00004c80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c210 tx=0x7fdb00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d44b6067f1eb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.922+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04007d50 con 0x7fdb00004800 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.922+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb04005af0 con 0x7fdb00004800 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.922+0000 7fdb19400640 1 -- 172.21.15.59:0/599970662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04007070 con 0x7fdb00004800 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.922+0000 7fdb21000640 1 -- 172.21.15.59:0/599970662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb00004800 msgr2=0x7fdb00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.922+0000 7fdb21000640 1 --2- 172.21.15.59:0/599970662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb00004800 0x7fdb00004c80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c210 tx=0x7fdb00005830 comp rx=0 tx=0).stop 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.923+0000 7fdb1a800640 1 -- 172.21.15.59:0/599970662 reap_dead start 2024-04-23T14:41:11.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.923+0000 7fdb21000640 1 -- 172.21.15.59:0/599970662 shutdown_connections 2024-04-23T14:41:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.923+0000 7fdb21000640 1 -- 172.21.15.59:0/599970662 >> 172.21.15.59:0/599970662 conn(0x7fdb1c073450 msgr2=0x7fdb1c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.923+0000 7fdb21000640 1 -- 172.21.15.59:0/599970662 shutdown_connections 2024-04-23T14:41:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.923+0000 7fdb21000640 1 -- 172.21.15.59:0/599970662 wait complete. 2024-04-23T14:41:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.924+0000 7fdb21000640 1 Processor -- start 2024-04-23T14:41:11.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.925+0000 7fdb21000640 1 -- start start 2024-04-23T14:41:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb21000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb00004800 0x7fdb1c145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb21000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cafd0 0x7fdb1c1486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb21000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c148bf0 0x7fdb1c0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb21000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c0ce4d0 con 0x7fdb1c148bf0 2024-04-23T14:41:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb21000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c06bee0 con 0x7fdb00004800 2024-04-23T14:41:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb21000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c0ce650 con 0x7fdb1c0cafd0 2024-04-23T14:41:11.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c148bf0 0x7fdb1c0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c148bf0 0x7fdb1c0ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46486/0 (socket says 172.21.15.59:46486) 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 -- 172.21.15.59:0/3413045367 learned_addr learned my addr 172.21.15.59:0/3413045367 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb19e00640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb00004800 0x7fdb1c145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 -- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cafd0 msgr2=0x7fdb1c1486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb13e00640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cafd0 0x7fdb1c1486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cafd0 0x7fdb1c1486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 -- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb00004800 msgr2=0x7fdb1c145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb00004800 0x7fdb1c145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb1a800640 1 -- 172.21.15.59:0/3413045367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c0ca7f0 con 0x7fdb1c148bf0 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.926+0000 7fdb13e00640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cafd0 0x7fdb1c1486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.927+0000 7fdb1a800640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c148bf0 0x7fdb1c0ae360 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0cda50 tx=0x7fdb1400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.927+0000 7fdb12000640 1 -- 172.21.15.59:0/3413045367 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1401c070 con 0x7fdb1c148bf0 2024-04-23T14:41:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.927+0000 7fdb12000640 1 -- 172.21.15.59:0/3413045367 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb14017400 con 0x7fdb1c148bf0 2024-04-23T14:41:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.927+0000 7fdb12000640 1 -- 172.21.15.59:0/3413045367 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140177f0 con 0x7fdb1c148bf0 2024-04-23T14:41:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.927+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c149100 con 0x7fdb1c148bf0 2024-04-23T14:41:11.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.927+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c146280 con 0x7fdb1c148bf0 2024-04-23T14:41:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.928+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c068a80 con 0x7fdb1c148bf0 2024-04-23T14:41:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.929+0000 7fdb12000640 1 -- 172.21.15.59:0/3413045367 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb14005d10 con 0x7fdb1c148bf0 2024-04-23T14:41:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.930+0000 7fdb12000640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdaf806c920 0x7fdaf806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.930+0000 7fdb12000640 1 -- 172.21.15.59:0/3413045367 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fdb1408c050 con 0x7fdb1c148bf0 2024-04-23T14:41:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.931+0000 7fdb19e00640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdaf806c920 0x7fdaf806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:11.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.931+0000 7fdb19e00640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdaf806c920 0x7fdaf806ede0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009990 tx=0x7fdb0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:11.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:11.935+0000 7fdb12000640 1 -- 172.21.15.59:0/3413045367 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb14012070 con 0x7fdb1c148bf0 2024-04-23T14:41:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.139+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb1c06efb0 con 0x7fdb1c148bf0 2024-04-23T14:41:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.140+0000 7fdb12000640 1 -- 172.21.15.59:0/3413045367 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fdb1405a310 con 0x7fdb1c148bf0 2024-04-23T14:41:12.143 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:41:12.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdaf806c920 msgr2=0x7fdaf806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdaf806c920 0x7fdaf806ede0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009990 tx=0x7fdb0c008040 comp rx=0 tx=0).stop 2024-04-23T14:41:12.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c148bf0 msgr2=0x7fdb1c0ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c148bf0 0x7fdb1c0ae360 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0cda50 tx=0x7fdb1400f4a0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 shutdown_connections 2024-04-23T14:41:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdaf806c920 0x7fdaf806ede0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb1c0cafd0 0x7fdb1c1486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c148bf0 0x7fdb1c0ae360 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 --2- 172.21.15.59:0/3413045367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb00004800 0x7fdb1c145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.143+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 >> 172.21.15.59:0/3413045367 conn(0x7fdb1c073450 msgr2=0x7fdb1c15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.144+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 shutdown_connections 2024-04-23T14:41:12.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.144+0000 7fdb21000640 1 -- 172.21.15.59:0/3413045367 wait complete. 2024-04-23T14:41:12.164 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:41:12.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.374+0000 7fdcbf200640 1 Processor -- start 2024-04-23T14:41:12.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.374+0000 7fdcbf200640 1 -- start start 2024-04-23T14:41:12.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbf200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdcc015e770 0x7fdcc015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbf200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc015fce0 con 0x7fdcc00d21c0 2024-04-23T14:41:12.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbf200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc00d37e0 con 0x7fdcc00d16d0 2024-04-23T14:41:12.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbf200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc00d3350 con 0x7fdcc015e770 2024-04-23T14:41:12.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdcc015e770 0x7fdcc015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdcc015e770 0x7fdcc015f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46502/0 (socket says 172.21.15.59:46502) 2024-04-23T14:41:12.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbde00640 1 -- 172.21.15.59:0/3490346873 learned_addr learned my addr 172.21.15.59:0/3490346873 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:12.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.376+0000 7fdcbd400640 1 --1- 172.21.15.59:0/3490346873 >> v1:172.21.15.59:6789/0 conn(0x7fdcc00d21c0 0x7fdcc00d0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58830/0 (socket says 172.21.15.59:58830) 2024-04-23T14:41:12.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.377+0000 7fdcbde00640 1 -- 172.21.15.59:0/3490346873 >> v1:172.21.15.76:6789/0 conn(0x7fdcc00d16d0 legacy=0x7fdcc00cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.377+0000 7fdcbde00640 1 -- 172.21.15.59:0/3490346873 >> v1:172.21.15.59:6789/0 conn(0x7fdcc00d21c0 legacy=0x7fdcc00d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.377+0000 7fdcbde00640 1 -- 172.21.15.59:0/3490346873 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc00d2740 con 0x7fdcc015e770 2024-04-23T14:41:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.377+0000 7fdcbca00640 1 -- 172.21.15.59:0/3490346873 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176275575 0 0) 0x7fdcc015fce0 con 0x7fdcc00d21c0 2024-04-23T14:41:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.378+0000 7fdcbde00640 1 --2- 172.21.15.59:0/3490346873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc015e770 0x7fdcc015f7a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009870 tx=0x7fdcb002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3061dadd0f490fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:12.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.378+0000 7fdcbca00640 1 -- 172.21.15.59:0/3490346873 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0037070 con 0x7fdcc015e770 2024-04-23T14:41:12.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbca00640 1 -- 172.21.15.59:0/3490346873 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb0032440 con 0x7fdcc015e770 2024-04-23T14:41:12.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbca00640 1 -- 172.21.15.59:0/3490346873 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0032810 con 0x7fdcc015e770 2024-04-23T14:41:12.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbf200640 1 -- 172.21.15.59:0/3490346873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc015e770 msgr2=0x7fdcc015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3490346873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc015e770 0x7fdcc015f7a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009870 tx=0x7fdcb002a2a0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbf200640 1 -- 172.21.15.59:0/3490346873 shutdown_connections 2024-04-23T14:41:12.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3490346873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc015e770 0x7fdcc015f7a0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbf200640 1 -- 172.21.15.59:0/3490346873 >> 172.21.15.59:0/3490346873 conn(0x7fdcc008bf90 msgr2=0x7fdcc00d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:12.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbf200640 1 -- 172.21.15.59:0/3490346873 shutdown_connections 2024-04-23T14:41:12.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.379+0000 7fdcbf200640 1 -- 172.21.15.59:0/3490346873 wait complete. 2024-04-23T14:41:12.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.381+0000 7fdcbf200640 1 Processor -- start 2024-04-23T14:41:12.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.381+0000 7fdcbf200640 1 -- start start 2024-04-23T14:41:12.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbf200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc00d16d0 0x7fdcc0153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbf200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc00d21c0 0x7fdcc0160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbf200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdcc015e770 0x7fdcc0161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbf200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc00d2fe0 con 0x7fdcc00d16d0 2024-04-23T14:41:12.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbf200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc00d2d00 con 0x7fdcc00d21c0 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbf200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc015eb50 con 0x7fdcc015e770 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc00d16d0 0x7fdcc0153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbde00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc00d16d0 0x7fdcc0153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46504/0 (socket says 172.21.15.59:46504) 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.382+0000 7fdcbde00640 1 -- 172.21.15.59:0/3270983606 learned_addr learned my addr 172.21.15.59:0/3270983606 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbd400640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc00d21c0 0x7fdcc0160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbe800640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdcc015e770 0x7fdcc0161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbde00640 1 -- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdcc015e770 msgr2=0x7fdcc0161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbde00640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdcc015e770 0x7fdcc0161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbde00640 1 -- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc00d21c0 msgr2=0x7fdcc0160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbde00640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc00d21c0 0x7fdcc0160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbde00640 1 -- 172.21.15.59:0/3270983606 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc0157c70 con 0x7fdcc00d16d0 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbe800640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdcc015e770 0x7fdcc0161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:12.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.383+0000 7fdcbde00640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc00d16d0 0x7fdcc0153b10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00090b0 tx=0x7fdcb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:12.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.384+0000 7fdcb6000640 1 -- 172.21.15.59:0/3270983606 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0037040 con 0x7fdcc00d16d0 2024-04-23T14:41:12.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.384+0000 7fdcb6000640 1 -- 172.21.15.59:0/3270983606 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb0004250 con 0x7fdcc00d16d0 2024-04-23T14:41:12.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.384+0000 7fdcb6000640 1 -- 172.21.15.59:0/3270983606 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb003c670 con 0x7fdcc00d16d0 2024-04-23T14:41:12.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.384+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc0157f50 con 0x7fdcc00d16d0 2024-04-23T14:41:12.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.384+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc0158510 con 0x7fdcc00d16d0 2024-04-23T14:41:12.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.385+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc004fd90 con 0x7fdcc00d16d0 2024-04-23T14:41:12.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.386+0000 7fdcb6000640 1 -- 172.21.15.59:0/3270983606 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcb003a070 con 0x7fdcc00d16d0 2024-04-23T14:41:12.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.387+0000 7fdcb6000640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.388+0000 7fdcb6000640 1 -- 172.21.15.59:0/3270983606 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fdcb00ac5b0 con 0x7fdcc00d16d0 2024-04-23T14:41:12.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.388+0000 7fdcbd400640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.389+0000 7fdcbd400640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00ce0e0 tx=0x7fdca8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:12.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.392+0000 7fdcb6000640 1 -- 172.21.15.59:0/3270983606 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcb0041020 con 0x7fdcc00d16d0 2024-04-23T14:41:12.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.597+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdcc00b37e0 con 0x7fdcc00d16d0 2024-04-23T14:41:12.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.598+0000 7fdcb6000640 1 -- 172.21.15.59:0/3270983606 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fdcb007a900 con 0x7fdcc00d16d0 2024-04-23T14:41:12.599 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc9c06c8f0 msgr2=0x7fdc9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00ce0e0 tx=0x7fdca8009040 comp rx=0 tx=0).stop 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc00d16d0 msgr2=0x7fdcc0153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc00d16d0 0x7fdcc0153b10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00090b0 tx=0x7fdcb00046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 shutdown_connections 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdcc015e770 0x7fdcc0161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc00d21c0 0x7fdcc0160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.600+0000 7fdcbf200640 1 --2- 172.21.15.59:0/3270983606 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdcc00d16d0 0x7fdcc0153b10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.601+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 >> 172.21.15.59:0/3270983606 conn(0x7fdcc008bf90 msgr2=0x7fdcc00b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.601+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 shutdown_connections 2024-04-23T14:41:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.601+0000 7fdcbf200640 1 -- 172.21.15.59:0/3270983606 wait complete. 2024-04-23T14:41:12.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:12.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.824+0000 7fedb5800640 1 Processor -- start 2024-04-23T14:41:12.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.824+0000 7fedb5800640 1 -- start start 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.825+0000 7fedb5800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb015e750 0x7fedb00ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.825+0000 7fedb5800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fedb00cfbf0 0x7fedb00b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.825+0000 7fedb5800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb00b8fc0 con 0x7fedb015eb30 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.825+0000 7fedb5800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fedb00b92a0 con 0x7fedb00cfbf0 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.825+0000 7fedb5800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fedb00b9580 con 0x7fedb015e750 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fedb00cfbf0 0x7fedb00b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fedb00cfbf0 0x7fedb00b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46514/0 (socket says 172.21.15.59:46514) 2024-04-23T14:41:12.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaf400640 1 -- 172.21.15.59:0/820803286 learned_addr learned my addr 172.21.15.59:0/820803286 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaea00640 1 --2- 172.21.15.59:0/820803286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb015e750 0x7fedb00ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaf400640 1 -- 172.21.15.59:0/820803286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb015e750 msgr2=0x7fedb00ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaf400640 1 --2- 172.21.15.59:0/820803286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb015e750 0x7fedb00ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaf400640 1 -- 172.21.15.59:0/820803286 >> v1:172.21.15.76:6789/0 conn(0x7fedb015eb30 legacy=0x7fedb00cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:12.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.826+0000 7fedaf400640 1 -- 172.21.15.59:0/820803286 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb00b9860 con 0x7fedb00cfbf0 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.828+0000 7fedaf400640 1 --2- 172.21.15.59:0/820803286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00cfbf0 0x7fedb00b7940 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7feda8009870 tx=0x7feda802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a6043de4295710 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.828+0000 7fedad600640 1 -- 172.21.15.59:0/820803286 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8037070 con 0x7fedb00cfbf0 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.828+0000 7fedad600640 1 -- 172.21.15.59:0/820803286 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda8032440 con 0x7fedb00cfbf0 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.828+0000 7fedad600640 1 -- 172.21.15.59:0/820803286 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8032810 con 0x7fedb00cfbf0 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.828+0000 7fedb5800640 1 -- 172.21.15.59:0/820803286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00cfbf0 msgr2=0x7fedb00b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.828+0000 7fedb5800640 1 --2- 172.21.15.59:0/820803286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00cfbf0 0x7fedb00b7940 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7feda8009870 tx=0x7feda802a200 comp rx=0 tx=0).stop 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.829+0000 7fedb5800640 1 -- 172.21.15.59:0/820803286 shutdown_connections 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.829+0000 7fedb5800640 1 --2- 172.21.15.59:0/820803286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb00cfbf0 0x7fedb00b7940 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.829+0000 7fedb5800640 1 --2- 172.21.15.59:0/820803286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb015e750 0x7fedb00ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.829+0000 7fedb5800640 1 -- 172.21.15.59:0/820803286 >> 172.21.15.59:0/820803286 conn(0x7fedb0067b50 msgr2=0x7fedb0067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.829+0000 7fedb5800640 1 -- 172.21.15.59:0/820803286 shutdown_connections 2024-04-23T14:41:12.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.829+0000 7fedb5800640 1 -- 172.21.15.59:0/820803286 wait complete. 2024-04-23T14:41:12.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.831+0000 7fedb5800640 1 Processor -- start 2024-04-23T14:41:12.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.831+0000 7fedb5800640 1 -- start start 2024-04-23T14:41:12.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.832+0000 7fedb5800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb00cfbf0 0x7fedb014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.832+0000 7fedb5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb015e750 0x7fedb01610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.832+0000 7fedb5800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedb015eb30 0x7fedb00ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedb5800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fedb00d3c80 con 0x7fedb015e750 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedb5800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fedb00ba6e0 con 0x7fedb015eb30 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedb5800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fedb00d3e00 con 0x7fedb00cfbf0 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb015e750 0x7fedb01610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb015e750 0x7fedb01610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46530/0 (socket says 172.21.15.59:46530) 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 -- 172.21.15.59:0/2383612446 learned_addr learned my addr 172.21.15.59:0/2383612446 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedaea00640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb00cfbf0 0x7fedb014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedaf400640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedb015eb30 0x7fedb00ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 -- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb00cfbf0 msgr2=0x7fedb014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb00cfbf0 0x7fedb014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 -- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedb015eb30 msgr2=0x7fedb00ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedb015eb30 0x7fedb00ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.833+0000 7fedae000640 1 -- 172.21.15.59:0/2383612446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb01f4910 con 0x7fedb015e750 2024-04-23T14:41:12.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.834+0000 7fedae000640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb015e750 0x7fedb01610e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00b3f0 tx=0x7fed9c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:12.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.834+0000 7feda6a00640 1 -- 172.21.15.59:0/2383612446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c015020 con 0x7fedb015e750 2024-04-23T14:41:12.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.834+0000 7feda6a00640 1 -- 172.21.15.59:0/2383612446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c011830 con 0x7fedb015e750 2024-04-23T14:41:12.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.834+0000 7feda6a00640 1 -- 172.21.15.59:0/2383612446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c011c20 con 0x7fedb015e750 2024-04-23T14:41:12.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.834+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb01f4bb0 con 0x7fedb015e750 2024-04-23T14:41:12.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.834+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb01f5170 con 0x7fedb015e750 2024-04-23T14:41:12.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.835+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb0067f30 con 0x7fedb015e750 2024-04-23T14:41:12.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.836+0000 7feda6a00640 1 -- 172.21.15.59:0/2383612446 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed9c018020 con 0x7fedb015e750 2024-04-23T14:41:12.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.837+0000 7feda6a00640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed9006c920 0x7fed9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:12.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.837+0000 7feda6a00640 1 -- 172.21.15.59:0/2383612446 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fed9c08b7e0 con 0x7fedb015e750 2024-04-23T14:41:12.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.838+0000 7fedaea00640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed9006c920 0x7fed9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:12.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.838+0000 7fedaea00640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed9006c920 0x7fed9006ede0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feda0009a60 tx=0x7feda0003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:12.844+0000 7feda6a00640 1 -- 172.21.15.59:0/2383612446 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedb0067f30 con 0x7fedb015e750 2024-04-23T14:41:13.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.091+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedb0068a80 con 0x7fedb015e750 2024-04-23T14:41:13.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.093+0000 7feda6a00640 1 -- 172.21.15.59:0/2383612446 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5181 (secure 0 0 0) 0x7fed9c01f030 con 0x7fedb015e750 2024-04-23T14:41:13.094 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:13.094 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:30.798278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-23T14:41:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 74 2024-04-23T14:41:13.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed9006c920 msgr2=0x7fed9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed9006c920 0x7fed9006ede0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feda0009a60 tx=0x7feda0003990 comp rx=0 tx=0).stop 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb015e750 msgr2=0x7fedb01610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb015e750 0x7fedb01610e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00b3f0 tx=0x7fed9c00b8b0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 shutdown_connections 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed9006c920 0x7fed9006ede0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fedb015eb30 0x7fedb00ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedb015e750 0x7fedb01610e0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 --2- 172.21.15.59:0/2383612446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fedb00cfbf0 0x7fedb014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.095+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 >> 172.21.15.59:0/2383612446 conn(0x7fedb0067b50 msgr2=0x7fedb00d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.096+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 shutdown_connections 2024-04-23T14:41:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.096+0000 7fedb5800640 1 -- 172.21.15.59:0/2383612446 wait complete. 2024-04-23T14:41:13.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:13.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.326+0000 7f1107400640 1 Processor -- start 2024-04-23T14:41:13.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.327+0000 7f1107400640 1 -- start start 2024-04-23T14:41:13.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.328+0000 7f1107400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11001454e0 0x7f1100145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.328+0000 7f1107400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1100160910 0x7f1100160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.328+0000 7f1107400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11000cd520 con 0x7f11000cd020 2024-04-23T14:41:13.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.328+0000 7f1107400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f11000cddc0 con 0x7f11001454e0 2024-04-23T14:41:13.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.328+0000 7f1107400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f11000cd800 con 0x7f1100160910 2024-04-23T14:41:13.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.329+0000 7f1104a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11001454e0 0x7f1100145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.329+0000 7f1105400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1100160910 0x7f1100160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.329+0000 7f1104a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11001454e0 0x7f1100145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:46442/0 (socket says 172.21.15.59:46442) 2024-04-23T14:41:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.329+0000 7f1104a00640 1 -- 172.21.15.59:0/2988355418 learned_addr learned my addr 172.21.15.59:0/2988355418 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.330+0000 7f1104a00640 1 -- 172.21.15.59:0/2988355418 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1100160910 msgr2=0x7f1100160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.330+0000 7f1104a00640 1 --2- 172.21.15.59:0/2988355418 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1100160910 0x7f1100160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.330+0000 7f1104a00640 1 -- 172.21.15.59:0/2988355418 >> v1:172.21.15.76:6789/0 conn(0x7f11000cd020 legacy=0x7f11000b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.330+0000 7f1104a00640 1 -- 172.21.15.59:0/2988355418 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11000cdae0 con 0x7f11001454e0 2024-04-23T14:41:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.332+0000 7f1104a00640 1 --2- 172.21.15.59:0/2988355418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 0x7f1100145bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae47ad330e92ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:13.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.332+0000 7f10ff400640 1 -- 172.21.15.59:0/2988355418 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0037070 con 0x7f11001454e0 2024-04-23T14:41:13.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.332+0000 7f10ff400640 1 -- 172.21.15.59:0/2988355418 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f0032440 con 0x7f11001454e0 2024-04-23T14:41:13.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.332+0000 7f10ff400640 1 -- 172.21.15.59:0/2988355418 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0032810 con 0x7f11001454e0 2024-04-23T14:41:13.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.333+0000 7f1107400640 1 -- 172.21.15.59:0/2988355418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 msgr2=0x7f1100145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.333+0000 7f1107400640 1 --2- 172.21.15.59:0/2988355418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 0x7f1100145bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f002a1c0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.334+0000 7f1107400640 1 -- 172.21.15.59:0/2988355418 shutdown_connections 2024-04-23T14:41:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.334+0000 7f1107400640 1 --2- 172.21.15.59:0/2988355418 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1100160910 0x7f1100160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.334+0000 7f1107400640 1 --2- 172.21.15.59:0/2988355418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 0x7f1100145bb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.334+0000 7f1107400640 1 -- 172.21.15.59:0/2988355418 >> 172.21.15.59:0/2988355418 conn(0x7f1100067ea0 msgr2=0x7f110006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.334+0000 7f1107400640 1 -- 172.21.15.59:0/2988355418 shutdown_connections 2024-04-23T14:41:13.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.334+0000 7f1107400640 1 -- 172.21.15.59:0/2988355418 wait complete. 2024-04-23T14:41:13.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.336+0000 7f1107400640 1 Processor -- start 2024-04-23T14:41:13.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.336+0000 7f1107400640 1 -- start start 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1107400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11000cd020 0x7f110006f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1107400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 0x7f11001424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1107400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1100160910 0x7f11001e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1107400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11000b4010 con 0x7f1100160910 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1107400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11001460f0 con 0x7f11001454e0 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1107400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f110006e270 con 0x7f11000cd020 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1105400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1100160910 0x7f11001e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1105400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1100160910 0x7f11001e6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46540/0 (socket says 172.21.15.59:46540) 2024-04-23T14:41:13.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f1105400640 1 -- 172.21.15.59:0/3068199890 learned_addr learned my addr 172.21.15.59:0/3068199890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.338+0000 7f10ffe00640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 0x7f11001424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1104a00640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11000cd020 0x7f110006f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1105400640 1 -- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11000cd020 msgr2=0x7f110006f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1105400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11000cd020 0x7f110006f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1105400640 1 -- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 msgr2=0x7f11001424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1105400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 0x7f11001424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1105400640 1 -- 172.21.15.59:0/3068199890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110006e550 con 0x7f1100160910 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1105400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1100160910 0x7f11001e6b10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f10f800efc0 tx=0x7f10f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f10fcc00640 1 -- 172.21.15.59:0/3068199890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8018020 con 0x7f1100160910 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f10fcc00640 1 -- 172.21.15.59:0/3068199890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f8014480 con 0x7f1100160910 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f10fcc00640 1 -- 172.21.15.59:0/3068199890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8014870 con 0x7f1100160910 2024-04-23T14:41:13.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.339+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110006e830 con 0x7f1100160910 2024-04-23T14:41:13.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.340+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110006edf0 con 0x7f1100160910 2024-04-23T14:41:13.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.341+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110006eff0 con 0x7f1100160910 2024-04-23T14:41:13.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.342+0000 7f10fcc00640 1 -- 172.21.15.59:0/3068199890 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10f8005d10 con 0x7f1100160910 2024-04-23T14:41:13.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.343+0000 7f10fcc00640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10dc06c8f0 0x7f10dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.343+0000 7f10fcc00640 1 -- 172.21.15.59:0/3068199890 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f10f808c5f0 con 0x7f1100160910 2024-04-23T14:41:13.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.344+0000 7f1104a00640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10dc06c8f0 0x7f10dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.345+0000 7f1104a00640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10dc06c8f0 0x7f10dc06edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f10f0034000 tx=0x7f10f002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:13.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.348+0000 7f10fcc00640 1 -- 172.21.15.59:0/3068199890 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f110006eff0 con 0x7f1100160910 2024-04-23T14:41:13.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.555+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1100068a80 con 0x7f1100160910 2024-04-23T14:41:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.559+0000 7f10fcc00640 1 -- 172.21.15.59:0/3068199890 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16909 (secure 0 0 0) 0x7f10f8022070 con 0x7f1100160910 2024-04-23T14:41:13.560 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:13.560 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:40:25.737792+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.59:6835/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.76:6835/3870021000":"2024-04-24T14:39:01.426690+0000","172.21.15.76:6833/3870021000":"2024-04-24T14:39:01.426690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10dc06c8f0 msgr2=0x7f10dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10dc06c8f0 0x7f10dc06edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f10f0034000 tx=0x7f10f002e000 comp rx=0 tx=0).stop 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1100160910 msgr2=0x7f11001e6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1100160910 0x7f11001e6b10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f10f800efc0 tx=0x7f10f800c490 comp rx=0 tx=0).stop 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 shutdown_connections 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f10dc06c8f0 0x7f10dc06edb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1100160910 0x7f11001e6b10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11001454e0 0x7f11001424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 --2- 172.21.15.59:0/3068199890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11000cd020 0x7f110006f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.561+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 >> 172.21.15.59:0/3068199890 conn(0x7f1100067ea0 msgr2=0x7f1100144580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.562+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 shutdown_connections 2024-04-23T14:41:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.562+0000 7f1107400640 1 -- 172.21.15.59:0/3068199890 wait complete. 2024-04-23T14:41:13.582 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:41:13.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.794+0000 7efefa000640 1 Processor -- start 2024-04-23T14:41:13.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.794+0000 7efefa000640 1 -- start start 2024-04-23T14:41:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.796+0000 7efefa000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efef4158650 0x7efef406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.796+0000 7efefa000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 0x7efef415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.796+0000 7efefa000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef4158a30 con 0x7efef415ec10 2024-04-23T14:41:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.796+0000 7efefa000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7efef415f040 con 0x7efef4158650 2024-04-23T14:41:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.796+0000 7efefa000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7efef406cdc0 con 0x7efef406c920 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.796+0000 7efef9600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efef415ec10 0x7efef415b0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58834/0 (socket says 172.21.15.59:58834) 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.796+0000 7efef9600640 1 -- 172.21.15.59:0/606051499 learned_addr learned my addr 172.21.15.59:0/606051499 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef8c00640 1 --2- 172.21.15.59:0/606051499 >> v2:172.21.15.76:3300/0 conn(0x7efef4158650 0x7efef406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3e00640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 0x7efef415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829726510 0 0) 0x7efef4158a30 con 0x7efef415ec10 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efed8003520 con 0x7efef415ec10 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 919631605 0 0) 0x7efed8003520 con 0x7efef415ec10 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 msgr2=0x7efef415e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 0x7efef415e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 >> v2:172.21.15.76:3300/0 conn(0x7efef4158650 msgr2=0x7efef406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 --2- 172.21.15.59:0/606051499 >> v2:172.21.15.76:3300/0 conn(0x7efef4158650 0x7efef406c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef406d0a0 con 0x7efef415ec10 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef8c00640 1 --2- 172.21.15.59:0/606051499 >> v2:172.21.15.76:3300/0 conn(0x7efef4158650 0x7efef406c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7efeec0019f0 con 0x7efef415ec10 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 >> v1:172.21.15.59:6789/0 conn(0x7efef415ec10 legacy=0x7efef415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed80039e0 0x7efed8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efed8004800 0x7efed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.797+0000 7efef3400640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 0x7efed80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efef415f040 con 0x7efed80039e0 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efef406cdc0 con 0x7efed8004800 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7efed800baf0 con 0x7efef406c920 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef8c00640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed80039e0 0x7efed8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef3e00640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efed8004800 0x7efed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef9600640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 0x7efed80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef8c00640 1 -- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 msgr2=0x7efed80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef8c00640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 0x7efed80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef8c00640 1 -- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efed8004800 msgr2=0x7efed8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef8c00640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efed8004800 0x7efed8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef8c00640 1 -- 172.21.15.59:0/606051499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed8003520 con 0x7efed80039e0 2024-04-23T14:41:13.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.798+0000 7efef9600640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef406c920 0x7efed80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.799+0000 7efef8c00640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed80039e0 0x7efed8003e00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7efee400bd70 tx=0x7efed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a38c68ca247de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.800+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4038070 con 0x7efed80039e0 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.800+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee402bb00 con 0x7efed80039e0 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.800+0000 7efef3400640 1 -- 172.21.15.59:0/606051499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400b970 con 0x7efed80039e0 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.800+0000 7efefa000640 1 -- 172.21.15.59:0/606051499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed80039e0 msgr2=0x7efed8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.800+0000 7efefa000640 1 --2- 172.21.15.59:0/606051499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efed80039e0 0x7efed8003e00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7efee400bd70 tx=0x7efed8005830 comp rx=0 tx=0).stop 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.800+0000 7efef9600640 1 -- 172.21.15.59:0/606051499 reap_dead start 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.801+0000 7efefa000640 1 -- 172.21.15.59:0/606051499 shutdown_connections 2024-04-23T14:41:13.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.801+0000 7efefa000640 1 -- 172.21.15.59:0/606051499 >> 172.21.15.59:0/606051499 conn(0x7efef40b2cc0 msgr2=0x7efef40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:13.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.801+0000 7efefa000640 1 -- 172.21.15.59:0/606051499 shutdown_connections 2024-04-23T14:41:13.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.801+0000 7efefa000640 1 -- 172.21.15.59:0/606051499 wait complete. 2024-04-23T14:41:13.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.802+0000 7efefa000640 1 Processor -- start 2024-04-23T14:41:13.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.803+0000 7efefa000640 1 -- start start 2024-04-23T14:41:13.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efefa000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efed80039e0 0x7efef4159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efefa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef415b960 0x7efef41443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efefa000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef40cf6e0 0x7efef40cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efefa000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efef415f4d0 con 0x7efef415b960 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efefa000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efef415f350 con 0x7efef40cf6e0 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efefa000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7efef406ff00 con 0x7efed80039e0 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efef3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef415b960 0x7efef41443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efef3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef415b960 0x7efef41443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46560/0 (socket says 172.21.15.59:46560) 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efef3e00640 1 -- 172.21.15.59:0/3939769269 learned_addr learned my addr 172.21.15.59:0/3939769269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efef8c00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efed80039e0 0x7efef4159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.804+0000 7efef9600640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef40cf6e0 0x7efef40cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef3e00640 1 -- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efed80039e0 msgr2=0x7efef4159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef3e00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efed80039e0 0x7efef4159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef3e00640 1 -- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef40cf6e0 msgr2=0x7efef40cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef3e00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef40cf6e0 0x7efef40cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef3e00640 1 -- 172.21.15.59:0/3939769269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef415c0b0 con 0x7efef415b960 2024-04-23T14:41:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef8c00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efed80039e0 0x7efef4159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef3e00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef415b960 0x7efef41443e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7efee0011660 tx=0x7efee0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef0c00640 1 -- 172.21.15.59:0/3939769269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee000f960 con 0x7efef415b960 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef0c00640 1 -- 172.21.15.59:0/3939769269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee000fb00 con 0x7efef415b960 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efef0c00640 1 -- 172.21.15.59:0/3939769269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0015460 con 0x7efef415b960 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.805+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef415c390 con 0x7efef415b960 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.806+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef415c950 con 0x7efef415b960 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.807+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef404fd90 con 0x7efef415b960 2024-04-23T14:41:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.808+0000 7efef0c00640 1 -- 172.21.15.59:0/3939769269 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efee0015600 con 0x7efef415b960 2024-04-23T14:41:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.808+0000 7efef0c00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efed406c8f0 0x7efed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.809+0000 7efef0c00640 1 -- 172.21.15.59:0/3939769269 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7efee008c690 con 0x7efef415b960 2024-04-23T14:41:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.809+0000 7efef8c00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efed406c8f0 0x7efed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.810+0000 7efef8c00640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efed406c8f0 0x7efed406edb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efee4035000 tx=0x7efee402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:13.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:13.813+0000 7efef0c00640 1 -- 172.21.15.59:0/3939769269 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efee0019030 con 0x7efef415b960 2024-04-23T14:41:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.069+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efef40b4d70 con 0x7efef415b960 2024-04-23T14:41:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.071+0000 7efef0c00640 1 -- 172.21.15.59:0/3939769269 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5181 (secure 0 0 0) 0x7efee005a8c0 con 0x7efef415b960 2024-04-23T14:41:14.072 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:14.072 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:30.798278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-23T14:41:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 74 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efed406c8f0 msgr2=0x7efed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efed406c8f0 0x7efed406edb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efee4035000 tx=0x7efee402f000 comp rx=0 tx=0).stop 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef415b960 msgr2=0x7efef41443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef415b960 0x7efef41443e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7efee0011660 tx=0x7efee0011b20 comp rx=0 tx=0).stop 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 shutdown_connections 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7efed406c8f0 0x7efed406edb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef40cf6e0 0x7efef40cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef415b960 0x7efef41443e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 --2- 172.21.15.59:0/3939769269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efed80039e0 0x7efef4159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.073+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 >> 172.21.15.59:0/3939769269 conn(0x7efef40b2cc0 msgr2=0x7efef41577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.074+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 shutdown_connections 2024-04-23T14:41:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.074+0000 7efefa000640 1 -- 172.21.15.59:0/3939769269 wait complete. 2024-04-23T14:41:14.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:14.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.305+0000 7fb101600640 1 Processor -- start 2024-04-23T14:41:14.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.305+0000 7fb101600640 1 -- start start 2024-04-23T14:41:14.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.307+0000 7fb101600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0fc0cd020 0x7fb0fc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.307+0000 7fb101600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0fc0cd520 con 0x7fb0fc0b3e40 2024-04-23T14:41:14.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.307+0000 7fb101600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0fc0cd800 con 0x7fb0fc1454e0 2024-04-23T14:41:14.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.307+0000 7fb101600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc0cddc0 con 0x7fb0fc0cd020 2024-04-23T14:41:14.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.307+0000 7fb0fa000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0fc0cd020 0x7fb0fc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.307+0000 7fb0fa000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0fc0cd020 0x7fb0fc0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46568/0 (socket says 172.21.15.59:46568) 2024-04-23T14:41:14.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.307+0000 7fb0fa000640 1 -- 172.21.15.59:0/1805687274 learned_addr learned my addr 172.21.15.59:0/1805687274 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:14.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.308+0000 7fb0fa000640 1 -- 172.21.15.59:0/1805687274 >> v1:172.21.15.76:6789/0 conn(0x7fb0fc1454e0 legacy=0x7fb0fc145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:14.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.308+0000 7fb0fa000640 1 -- 172.21.15.59:0/1805687274 >> v1:172.21.15.59:6789/0 conn(0x7fb0fc0b3e40 legacy=0x7fb0fc160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.308+0000 7fb0fa000640 1 -- 172.21.15.59:0/1805687274 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc0cdae0 con 0x7fb0fc0cd020 2024-04-23T14:41:14.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.309+0000 7fb0fa000640 1 --2- 172.21.15.59:0/1805687274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc0b3900 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009870 tx=0x7fb0e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e28429251e3c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:14.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.309+0000 7fb0f9600640 1 -- 172.21.15.59:0/1805687274 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8037070 con 0x7fb0fc0cd020 2024-04-23T14:41:14.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.309+0000 7fb0f9600640 1 -- 172.21.15.59:0/1805687274 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e8032440 con 0x7fb0fc0cd020 2024-04-23T14:41:14.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.309+0000 7fb0f9600640 1 -- 172.21.15.59:0/1805687274 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8032810 con 0x7fb0fc0cd020 2024-04-23T14:41:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.310+0000 7fb101600640 1 -- 172.21.15.59:0/1805687274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 msgr2=0x7fb0fc0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.310+0000 7fb101600640 1 --2- 172.21.15.59:0/1805687274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc0b3900 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009870 tx=0x7fb0e802a210 comp rx=0 tx=0).stop 2024-04-23T14:41:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.310+0000 7fb101600640 1 -- 172.21.15.59:0/1805687274 shutdown_connections 2024-04-23T14:41:14.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.310+0000 7fb101600640 1 --2- 172.21.15.59:0/1805687274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc0b3900 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.310+0000 7fb101600640 1 -- 172.21.15.59:0/1805687274 >> 172.21.15.59:0/1805687274 conn(0x7fb0fc067ea0 msgr2=0x7fb0fc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:14.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.310+0000 7fb101600640 1 -- 172.21.15.59:0/1805687274 shutdown_connections 2024-04-23T14:41:14.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.310+0000 7fb101600640 1 -- 172.21.15.59:0/1805687274 wait complete. 2024-04-23T14:41:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.312+0000 7fb101600640 1 Processor -- start 2024-04-23T14:41:14.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.312+0000 7fb101600640 1 -- start start 2024-04-23T14:41:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.313+0000 7fb101600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0fc0b3e40 0x7fb0fc15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.313+0000 7fb101600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.313+0000 7fb101600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0fc1454e0 0x7fb0fc1e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.313+0000 7fb101600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc161020 con 0x7fb0fc0cd020 2024-04-23T14:41:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.313+0000 7fb101600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1462c0 con 0x7fb0fc0b3e40 2024-04-23T14:41:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.313+0000 7fb101600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc1611a0 con 0x7fb0fc1454e0 2024-04-23T14:41:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.313+0000 7fb0fa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc15bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46580/0 (socket says 172.21.15.59:46580) 2024-04-23T14:41:14.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fa000640 1 -- 172.21.15.59:0/1928844101 learned_addr learned my addr 172.21.15.59:0/1928844101 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:14.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0faa00640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0fc0b3e40 0x7fb0fc15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fa000640 1 -- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0fc1454e0 msgr2=0x7fb0fc1e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fb400640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0fc1454e0 0x7fb0fc1e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fa000640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0fc1454e0 0x7fb0fc1e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fa000640 1 -- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0fc0b3e40 msgr2=0x7fb0fc15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fa000640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0fc0b3e40 0x7fb0fc15ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fa000640 1 -- 172.21.15.59:0/1928844101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc15efd0 con 0x7fb0fc0cd020 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0faa00640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0fc0b3e40 0x7fb0fc15ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.314+0000 7fb0fb400640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0fc1454e0 0x7fb0fc1e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.315+0000 7fb0fa000640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc15bcb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009870 tx=0x7fb0e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.315+0000 7fb0f2a00640 1 -- 172.21.15.59:0/1928844101 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8041020 con 0x7fb0fc0cd020 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.315+0000 7fb0f2a00640 1 -- 172.21.15.59:0/1928844101 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e8030970 con 0x7fb0fc0cd020 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.315+0000 7fb0f2a00640 1 -- 172.21.15.59:0/1928844101 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8037040 con 0x7fb0fc0cd020 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.315+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc1e6950 con 0x7fb0fc0cd020 2024-04-23T14:41:14.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.315+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc15e290 con 0x7fb0fc0cd020 2024-04-23T14:41:14.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.316+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc068a80 con 0x7fb0fc0cd020 2024-04-23T14:41:14.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.317+0000 7fb0f2a00640 1 -- 172.21.15.59:0/1928844101 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0e8044020 con 0x7fb0fc0cd020 2024-04-23T14:41:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.319+0000 7fb0f2a00640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0dc06c8f0 0x7fb0dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.319+0000 7fb0f2a00640 1 -- 172.21.15.59:0/1928844101 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7fb0e80ac2e0 con 0x7fb0fc0cd020 2024-04-23T14:41:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.319+0000 7fb0faa00640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0dc06c8f0 0x7fb0dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.320+0000 7fb0faa00640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0dc06c8f0 0x7fb0dc06edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00a830 tx=0x7fb0ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:14.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.323+0000 7fb0f2a00640 1 -- 172.21.15.59:0/1928844101 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0e804b070 con 0x7fb0fc0cd020 2024-04-23T14:41:14.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.566+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0fc15c550 con 0x7fb0fc0cd020 2024-04-23T14:41:14.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.568+0000 7fb0f2a00640 1 -- 172.21.15.59:0/1928844101 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5181 (secure 0 0 0) 0x7fb0fc15efd0 con 0x7fb0fc0cd020 2024-04-23T14:41:14.569 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:14.569 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:38:39.210981+0000","modified":"2024-04-23T14:40:30.798278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8133},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8133":{"gid":8133,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3466627380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3466627380},{"type":"v1","addr":"172.21.15.59:6837","nonce":3466627380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8133,"qdb_cluster":[8133]},"id":6}]} 2024-04-23T14:41:14.570 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 74 2024-04-23T14:41:14.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.570+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0dc06c8f0 msgr2=0x7fb0dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.570+0000 7fb101600640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0dc06c8f0 0x7fb0dc06edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00a830 tx=0x7fb0ec008040 comp rx=0 tx=0).stop 2024-04-23T14:41:14.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.570+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 msgr2=0x7fb0fc15bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.570+0000 7fb101600640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc15bcb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009870 tx=0x7fb0e80046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 shutdown_connections 2024-04-23T14:41:14.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb0dc06c8f0 0x7fb0dc06edb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb0fc1454e0 0x7fb0fc1e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0cd020 0x7fb0fc15bcb0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 --2- 172.21.15.59:0/1928844101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0fc0b3e40 0x7fb0fc15ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 >> 172.21.15.59:0/1928844101 conn(0x7fb0fc067ea0 msgr2=0x7fb0fc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:14.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 shutdown_connections 2024-04-23T14:41:14.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.571+0000 7fb101600640 1 -- 172.21.15.59:0/1928844101 wait complete. 2024-04-23T14:41:14.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:14.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.801+0000 7f4ceca00640 1 Processor -- start 2024-04-23T14:41:14.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.801+0000 7f4ceca00640 1 -- start start 2024-04-23T14:41:14.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4ceca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ce806f3f0 0x7f4ce80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4ceca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce80b3870 con 0x7f4ce80b3080 2024-04-23T14:41:14.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4ceca00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce80b3b50 con 0x7f4ce815b450 2024-04-23T14:41:14.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4ceca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce80b41a0 con 0x7f4ce806f3f0 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4cdde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ce806f3f0 0x7f4ce80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4cdde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ce806f3f0 0x7f4ce80cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46594/0 (socket says 172.21.15.59:46594) 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4cdde00640 1 -- 172.21.15.59:0/252442742 learned_addr learned my addr 172.21.15.59:0/252442742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.803+0000 7f4ce7200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4ce80b3080 0x7f4ce80ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58852/0 (socket says 172.21.15.59:58852) 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.804+0000 7f4cdde00640 1 -- 172.21.15.59:0/252442742 >> v1:172.21.15.76:6789/0 conn(0x7f4ce815b450 legacy=0x7f4ce80b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.804+0000 7f4cdde00640 1 -- 172.21.15.59:0/252442742 >> v1:172.21.15.59:6789/0 conn(0x7f4ce80b3080 legacy=0x7f4ce80ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.804+0000 7f4cdde00640 1 -- 172.21.15.59:0/252442742 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce80b3e30 con 0x7f4ce806f3f0 2024-04-23T14:41:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.804+0000 7f4ce5e00640 1 -- 172.21.15.59:0/252442742 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1992625536 0 0) 0x7f4ce80b3870 con 0x7f4ce80b3080 2024-04-23T14:41:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.805+0000 7f4cdde00640 1 --2- 172.21.15.59:0/252442742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce806f3f0 0x7f4ce80cd050 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80b4320 tx=0x7f4cd002a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edb2ca613368a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.805+0000 7f4ce5e00640 1 -- 172.21.15.59:0/252442742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0037070 con 0x7f4ce806f3f0 2024-04-23T14:41:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.805+0000 7f4ce5e00640 1 -- 172.21.15.59:0/252442742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd0032640 con 0x7f4ce806f3f0 2024-04-23T14:41:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.805+0000 7f4ce5e00640 1 -- 172.21.15.59:0/252442742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0032a10 con 0x7f4ce806f3f0 2024-04-23T14:41:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.806+0000 7f4ceca00640 1 -- 172.21.15.59:0/252442742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce806f3f0 msgr2=0x7f4ce80cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.806+0000 7f4ceca00640 1 --2- 172.21.15.59:0/252442742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce806f3f0 0x7f4ce80cd050 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80b4320 tx=0x7f4cd002a370 comp rx=0 tx=0).stop 2024-04-23T14:41:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.806+0000 7f4ceca00640 1 -- 172.21.15.59:0/252442742 shutdown_connections 2024-04-23T14:41:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.806+0000 7f4ceca00640 1 --2- 172.21.15.59:0/252442742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce806f3f0 0x7f4ce80cd050 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.806+0000 7f4ceca00640 1 -- 172.21.15.59:0/252442742 >> 172.21.15.59:0/252442742 conn(0x7f4ce80b21a0 msgr2=0x7f4ce80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.806+0000 7f4ceca00640 1 -- 172.21.15.59:0/252442742 shutdown_connections 2024-04-23T14:41:14.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.806+0000 7f4ceca00640 1 -- 172.21.15.59:0/252442742 wait complete. 2024-04-23T14:41:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.808+0000 7f4ceca00640 1 Processor -- start 2024-04-23T14:41:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.808+0000 7f4ceca00640 1 -- start start 2024-04-23T14:41:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ce806f3f0 0x7f4ce809cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce80b3080 0x7f4ce80c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ceca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce815b450 0x7f4ce80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ceca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce80cd590 con 0x7f4ce815b450 2024-04-23T14:41:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ceca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce80b4ff0 con 0x7f4ce80b3080 2024-04-23T14:41:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ceca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce80cd710 con 0x7f4ce806f3f0 2024-04-23T14:41:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ce7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce815b450 0x7f4ce80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ce7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce815b450 0x7f4ce80cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46598/0 (socket says 172.21.15.59:46598) 2024-04-23T14:41:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.809+0000 7f4ce7200640 1 -- 172.21.15.59:0/4066253860 learned_addr learned my addr 172.21.15.59:0/4066253860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4cdde00640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce80b3080 0x7f4ce80c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4ce6800640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ce806f3f0 0x7f4ce809cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4ce7200640 1 -- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ce806f3f0 msgr2=0x7f4ce809cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4ce7200640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ce806f3f0 0x7f4ce809cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4ce7200640 1 -- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce80b3080 msgr2=0x7f4ce80c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4ce7200640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce80b3080 0x7f4ce80c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4ce7200640 1 -- 172.21.15.59:0/4066253860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce809d170 con 0x7f4ce815b450 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.810+0000 7f4ce7200640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce815b450 0x7f4ce80cb690 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c7d0 tx=0x7f4ce000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.811+0000 7f4cdf400640 1 -- 172.21.15.59:0/4066253860 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0018070 con 0x7f4ce815b450 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.811+0000 7f4cdf400640 1 -- 172.21.15.59:0/4066253860 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce000a640 con 0x7f4ce815b450 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.811+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce815d920 con 0x7f4ce815b450 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.811+0000 7f4cdf400640 1 -- 172.21.15.59:0/4066253860 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000aa70 con 0x7f4ce815b450 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.811+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce815dee0 con 0x7f4ce815b450 2024-04-23T14:41:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.812+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce804fd90 con 0x7f4ce815b450 2024-04-23T14:41:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.813+0000 7f4cdf400640 1 -- 172.21.15.59:0/4066253860 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ce001b070 con 0x7f4ce815b450 2024-04-23T14:41:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.814+0000 7f4cdf400640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.814+0000 7f4cdf400640 1 -- 172.21.15.59:0/4066253860 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f4ce008b1d0 con 0x7f4ce815b450 2024-04-23T14:41:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.814+0000 7f4ce6800640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.815+0000 7f4ce6800640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cc406c8f0 0x7f4cc406edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8000f60 tx=0x7f4cd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:14.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:14.818+0000 7f4cdf400640 1 -- 172.21.15.59:0/4066253860 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ce0059530 con 0x7f4ce815b450 2024-04-23T14:41:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.022+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ce80b1b60 con 0x7f4ce815b450 2024-04-23T14:41:15.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.026+0000 7f4cdf400640 1 -- 172.21.15.59:0/4066253860 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16909 (secure 0 0 0) 0x7f4ce0058e30 con 0x7f4ce815b450 2024-04-23T14:41:15.028 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:15.028 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:40:25.737792+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:38:36.964433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-23T14:38:38.188587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.59:6835/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.76:6835/3870021000":"2024-04-24T14:39:01.426690+0000","172.21.15.76:6833/3870021000":"2024-04-24T14:39:01.426690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:15.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.028+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cc406c8f0 msgr2=0x7f4cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.028+0000 7f4ceca00640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cc406c8f0 0x7f4cc406edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8000f60 tx=0x7f4cd8009040 comp rx=0 tx=0).stop 2024-04-23T14:41:15.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.028+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce815b450 msgr2=0x7f4ce80cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.028+0000 7f4ceca00640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce815b450 0x7f4ce80cb690 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c7d0 tx=0x7f4ce000cc90 comp rx=0 tx=0).stop 2024-04-23T14:41:15.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 shutdown_connections 2024-04-23T14:41:15.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ce815b450 0x7f4ce80cb690 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce80b3080 0x7f4ce80c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 --2- 172.21.15.59:0/4066253860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ce806f3f0 0x7f4ce809cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 >> 172.21.15.59:0/4066253860 conn(0x7f4ce80b21a0 msgr2=0x7f4ce8158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:15.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 shutdown_connections 2024-04-23T14:41:15.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.029+0000 7f4ceca00640 1 -- 172.21.15.59:0/4066253860 wait complete. 2024-04-23T14:41:15.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:41:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.258+0000 7f4843200640 1 Processor -- start 2024-04-23T14:41:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.259+0000 7f4843200640 1 -- start start 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.260+0000 7f4843200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f483c15c670 0x7f483c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.260+0000 7f4843200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c0b4cf0 con 0x7f483c0b4230 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.260+0000 7f4843200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c0ccdd0 con 0x7f483c0d0cd0 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.260+0000 7f4843200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f483c15ca50 con 0x7f483c15c670 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.261+0000 7f4841e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f483c15c670 0x7f483c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.261+0000 7f4841e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f483c15c670 0x7f483c0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46610/0 (socket says 172.21.15.59:46610) 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.261+0000 7f4841e00640 1 -- 172.21.15.59:0/1241024108 learned_addr learned my addr 172.21.15.59:0/1241024108 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:15.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.261+0000 7f4841400640 1 --1- 172.21.15.59:0/1241024108 >> v1:172.21.15.59:6789/0 conn(0x7f483c0b4230 0x7f483c0d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58856/0 (socket says 172.21.15.59:58856) 2024-04-23T14:41:15.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.261+0000 7f4841e00640 1 -- 172.21.15.59:0/1241024108 >> v1:172.21.15.76:6789/0 conn(0x7f483c0d0cd0 legacy=0x7f483c06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:15.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.261+0000 7f4841e00640 1 -- 172.21.15.59:0/1241024108 >> v1:172.21.15.59:6789/0 conn(0x7f483c0b4230 legacy=0x7f483c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.261+0000 7f4841e00640 1 -- 172.21.15.59:0/1241024108 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c0cd0b0 con 0x7f483c15c670 2024-04-23T14:41:15.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.262+0000 7f4841e00640 1 --2- 172.21.15.59:0/1241024108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c0cd8b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f4830009870 tx=0x7f483002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832acae3503320b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:15.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4840a00640 1 -- 172.21.15.59:0/1241024108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830037070 con 0x7f483c15c670 2024-04-23T14:41:15.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4840a00640 1 -- 172.21.15.59:0/1241024108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4830032440 con 0x7f483c15c670 2024-04-23T14:41:15.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4840a00640 1 -- 172.21.15.59:0/1241024108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830032810 con 0x7f483c15c670 2024-04-23T14:41:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4843200640 1 -- 172.21.15.59:0/1241024108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 msgr2=0x7f483c0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4843200640 1 --2- 172.21.15.59:0/1241024108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c0cd8b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f4830009870 tx=0x7f483002a2a0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4843200640 1 -- 172.21.15.59:0/1241024108 shutdown_connections 2024-04-23T14:41:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4843200640 1 --2- 172.21.15.59:0/1241024108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c0cd8b0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4843200640 1 -- 172.21.15.59:0/1241024108 >> 172.21.15.59:0/1241024108 conn(0x7f483c06bdf0 msgr2=0x7f483c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4843200640 1 -- 172.21.15.59:0/1241024108 shutdown_connections 2024-04-23T14:41:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.263+0000 7f4843200640 1 -- 172.21.15.59:0/1241024108 wait complete. 2024-04-23T14:41:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.266+0000 7f4843200640 1 Processor -- start 2024-04-23T14:41:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.266+0000 7f4843200640 1 -- start start 2024-04-23T14:41:15.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.267+0000 7f4843200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f483c0b4230 0x7f483c1f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.267+0000 7f4843200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f483c0d0cd0 0x7f483c14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.267+0000 7f4843200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.267+0000 7f4843200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f483c0b3e60 con 0x7f483c15c670 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.267+0000 7f4843200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f483c0ccb80 con 0x7f483c0d0cd0 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.267+0000 7f4843200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f483c0b3fe0 con 0x7f483c0b4230 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.267+0000 7f4842800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46616/0 (socket says 172.21.15.59:46616) 2024-04-23T14:41:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 -- 172.21.15.59:0/379451382 learned_addr learned my addr 172.21.15.59:0/379451382 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4841400640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f483c0d0cd0 0x7f483c14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4841e00640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f483c0b4230 0x7f483c1f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 -- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f483c0b4230 msgr2=0x7f483c1f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f483c0b4230 0x7f483c1f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 -- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f483c0d0cd0 msgr2=0x7f483c14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f483c0d0cd0 0x7f483c14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 -- 172.21.15.59:0/379451382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c158720 con 0x7f483c15c670 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4841e00640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f483c0b4230 0x7f483c1f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4841400640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f483c0d0cd0 0x7f483c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.268+0000 7f4842800640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c157360 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f483800c7b0 tx=0x7f483800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.269+0000 7f4836000640 1 -- 172.21.15.59:0/379451382 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838004b60 con 0x7f483c15c670 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.269+0000 7f4836000640 1 -- 172.21.15.59:0/379451382 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4838004d00 con 0x7f483c15c670 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.269+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c158a00 con 0x7f483c15c670 2024-04-23T14:41:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.269+0000 7f4836000640 1 -- 172.21.15.59:0/379451382 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483800aa90 con 0x7f483c15c670 2024-04-23T14:41:15.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.269+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c158fc0 con 0x7f483c15c670 2024-04-23T14:41:15.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.270+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c04fd90 con 0x7f483c15c670 2024-04-23T14:41:15.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.271+0000 7f4836000640 1 -- 172.21.15.59:0/379451382 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f483800ac30 con 0x7f483c15c670 2024-04-23T14:41:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.272+0000 7f4836000640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f482006c8f0 0x7f482006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.272+0000 7f4836000640 1 -- 172.21.15.59:0/379451382 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5910+0+0 (secure 0 0 0) 0x7f483808bf20 con 0x7f483c15c670 2024-04-23T14:41:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.272+0000 7f4841e00640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f482006c8f0 0x7f482006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.273+0000 7f4841e00640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f482006c8f0 0x7f482006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4830034000 tx=0x7f483002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.276+0000 7f4836000640 1 -- 172.21.15.59:0/379451382 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4838020070 con 0x7f483c15c670 2024-04-23T14:41:15.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.524+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f483c0ceab0 con 0x7f483c15c670 2024-04-23T14:41:15.526 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:41:15.526+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:41:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.529+0000 7f4836000640 1 -- 172.21.15.59:0/379451382 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f483805a270 con 0x7f483c15c670 2024-04-23T14:41:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:41:15.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f482006c8f0 msgr2=0x7f482006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f482006c8f0 0x7f482006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4830034000 tx=0x7f483002e000 comp rx=0 tx=0).stop 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 msgr2=0x7f483c157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c157360 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f483800c7b0 tx=0x7f483800cc70 comp rx=0 tx=0).stop 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 shutdown_connections 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f482006c8f0 0x7f482006edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f483c15c670 0x7f483c157360 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f483c0d0cd0 0x7f483c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 --2- 172.21.15.59:0/379451382 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f483c0b4230 0x7f483c1f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 >> 172.21.15.59:0/379451382 conn(0x7f483c06bdf0 msgr2=0x7f483c073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 shutdown_connections 2024-04-23T14:41:15.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.533+0000 7f4843200640 1 -- 172.21.15.59:0/379451382 wait complete. 2024-04-23T14:41:15.544 INFO:tasks.ceph.mds.c.smithi059.stderr: -2518> 2024-04-23T14:40:31.758+0000 7f3598e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:41:15.546 INFO:tasks.ceph.mds.c.smithi059.stderr: -2037> 2024-04-23T14:40:35.234+0000 7f3593e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x560c23380680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-23T14:41:15.561 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:41:15.599 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:41:15.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.744+0000 7fbaa9600640 1 Processor -- start 2024-04-23T14:41:15.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.744+0000 7fbaa9600640 1 -- start start 2024-04-23T14:41:15.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.746+0000 7fbaa9600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbaa40d15b0 0x7fbaa40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.746+0000 7fbaa9600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbaa40b4f20 0x7fbaa40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.746+0000 7fbaa9600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa40b5da0 con 0x7fbaa415ed20 2024-04-23T14:41:15.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.746+0000 7fbaa9600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa415f100 con 0x7fbaa40b4f20 2024-04-23T14:41:15.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.746+0000 7fbaa9600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa40d0a20 con 0x7fbaa40d15b0 2024-04-23T14:41:15.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fbaa8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbaa40b4f20 0x7fbaa40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fbaa8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbaa40b4f20 0x7fbaa40cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46644/0 (socket says 172.21.15.59:46644) 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fbaa8c00640 1 -- 172.21.15.59:0/3919941137 learned_addr learned my addr 172.21.15.59:0/3919941137 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fba9be00640 1 --2- 172.21.15.59:0/3919941137 >> v2:172.21.15.76:3300/0 conn(0x7fbaa40d15b0 0x7fbaa40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fbaa8c00640 1 -- 172.21.15.59:0/3919941137 >> v2:172.21.15.76:3300/0 conn(0x7fbaa40d15b0 msgr2=0x7fbaa40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fbaa8c00640 1 --2- 172.21.15.59:0/3919941137 >> v2:172.21.15.76:3300/0 conn(0x7fbaa40d15b0 0x7fbaa40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fbaa8c00640 1 -- 172.21.15.59:0/3919941137 >> v1:172.21.15.76:6789/0 conn(0x7fbaa415ed20 legacy=0x7fbaa40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.747+0000 7fbaa8c00640 1 -- 172.21.15.59:0/3919941137 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa40d2ed0 con 0x7fbaa40b4f20 2024-04-23T14:41:15.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.748+0000 7fbaa8c00640 1 --2- 172.21.15.59:0/3919941137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40b4f20 0x7fbaa40cf7c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40d4520 tx=0x7fba9c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fffabaf54c43ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:15.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa3400640 1 -- 172.21.15.59:0/3919941137 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x7fbaa40b4f20 2024-04-23T14:41:15.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa3400640 1 -- 172.21.15.59:0/3919941137 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c031600 con 0x7fbaa40b4f20 2024-04-23T14:41:15.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa3400640 1 -- 172.21.15.59:0/3919941137 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0319d0 con 0x7fbaa40b4f20 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa9600640 1 -- 172.21.15.59:0/3919941137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40b4f20 msgr2=0x7fbaa40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3919941137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40b4f20 0x7fbaa40cf7c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40d4520 tx=0x7fba9c02a5d0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa9600640 1 -- 172.21.15.59:0/3919941137 shutdown_connections 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3919941137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40b4f20 0x7fbaa40cf7c0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.749+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3919941137 >> v2:172.21.15.76:3300/0 conn(0x7fbaa40d15b0 0x7fbaa40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.750+0000 7fbaa9600640 1 -- 172.21.15.59:0/3919941137 >> 172.21.15.59:0/3919941137 conn(0x7fbaa408bf90 msgr2=0x7fbaa40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.750+0000 7fbaa9600640 1 -- 172.21.15.59:0/3919941137 shutdown_connections 2024-04-23T14:41:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.750+0000 7fbaa9600640 1 -- 172.21.15.59:0/3919941137 wait complete. 2024-04-23T14:41:15.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.751+0000 7fbaa9600640 1 Processor -- start 2024-04-23T14:41:15.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.751+0000 7fbaa9600640 1 -- start start 2024-04-23T14:41:15.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.752+0000 7fbaa9600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaa40b4f20 0x7fbaa415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.752+0000 7fbaa9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40d15b0 0x7fbaa4157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.752+0000 7fbaa9600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbaa415ed20 0x7fbaa4150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.752+0000 7fbaa9600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40b3600 con 0x7fbaa40d15b0 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.752+0000 7fbaa9600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40d3490 con 0x7fbaa40b4f20 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.752+0000 7fbaa9600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa40b3780 con 0x7fbaa415ed20 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40d15b0 0x7fbaa4157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40d15b0 0x7fbaa4157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46658/0 (socket says 172.21.15.59:46658) 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 -- 172.21.15.59:0/3214334929 learned_addr learned my addr 172.21.15.59:0/3214334929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fbaa3e00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaa40b4f20 0x7fbaa415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fbaa8c00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbaa415ed20 0x7fbaa4150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 -- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbaa415ed20 msgr2=0x7fbaa4150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbaa415ed20 0x7fbaa4150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 -- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaa40b4f20 msgr2=0x7fbaa415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaa40b4f20 0x7fbaa415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fba9be00640 1 -- 172.21.15.59:0/3214334929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa40c8c60 con 0x7fbaa40d15b0 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.753+0000 7fbaa8c00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbaa415ed20 0x7fbaa4150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:15.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.754+0000 7fba9be00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40d15b0 0x7fbaa4157450 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d5f0 tx=0x7fba8c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.754+0000 7fbaa0c00640 1 -- 172.21.15.59:0/3214334929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0125f0 con 0x7fbaa40d15b0 2024-04-23T14:41:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.754+0000 7fbaa0c00640 1 -- 172.21.15.59:0/3214334929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c019070 con 0x7fbaa40d15b0 2024-04-23T14:41:15.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.754+0000 7fbaa0c00640 1 -- 172.21.15.59:0/3214334929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0129d0 con 0x7fbaa40d15b0 2024-04-23T14:41:15.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.754+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa4157990 con 0x7fbaa40d15b0 2024-04-23T14:41:15.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.754+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa40c8f00 con 0x7fbaa40d15b0 2024-04-23T14:41:15.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.755+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa404fd90 con 0x7fbaa40d15b0 2024-04-23T14:41:15.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.756+0000 7fbaa0c00640 1 -- 172.21.15.59:0/3214334929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba8c012b70 con 0x7fbaa40d15b0 2024-04-23T14:41:15.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.757+0000 7fbaa0c00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba8006c8f0 0x7fba8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:15.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.757+0000 7fbaa0c00640 1 -- 172.21.15.59:0/3214334929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5996+0+0 (secure 0 0 0) 0x7fba8c08c0d0 con 0x7fbaa40d15b0 2024-04-23T14:41:15.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.758+0000 7fbaa3e00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba8006c8f0 0x7fba8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.758+0000 7fbaa3e00640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba8006c8f0 0x7fba8006edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fba94004ea0 tx=0x7fba94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:15.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.762+0000 7fbaa0c00640 1 -- 172.21.15.59:0/3214334929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba8c05a3d0 con 0x7fbaa40d15b0 2024-04-23T14:41:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:15.998+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbaa40008d0 con 0x7fbaa40d15b0 2024-04-23T14:41:16.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.533+0000 7fbaa0c00640 1 -- 172.21.15.59:0/3214334929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7fba8c059cd0 con 0x7fbaa40d15b0 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba8006c8f0 msgr2=0x7fba8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba8006c8f0 0x7fba8006edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fba94004ea0 tx=0x7fba94009040 comp rx=0 tx=0).stop 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40d15b0 msgr2=0x7fbaa4157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40d15b0 0x7fbaa4157450 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d5f0 tx=0x7fba8c00dab0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 shutdown_connections 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fba8006c8f0 0x7fba8006edb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbaa415ed20 0x7fbaa4150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbaa40d15b0 0x7fbaa4157450 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 --2- 172.21.15.59:0/3214334929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaa40b4f20 0x7fbaa415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.536+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 >> 172.21.15.59:0/3214334929 conn(0x7fbaa408bf90 msgr2=0x7fbaa40cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:16.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.537+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 shutdown_connections 2024-04-23T14:41:16.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.537+0000 7fbaa9600640 1 -- 172.21.15.59:0/3214334929 wait complete. 2024-04-23T14:41:16.559 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:41:16.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.817+0000 7f01dcc00640 1 Processor -- start 2024-04-23T14:41:16.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.817+0000 7f01dcc00640 1 -- start start 2024-04-23T14:41:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01dcc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01d8162940 0x7f01d80af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01dcc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01d8162d20 0x7f01d806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01dcc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d806ec60 con 0x7f01d80ccb40 2024-04-23T14:41:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01dcc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f01d8070060 con 0x7f01d8162d20 2024-04-23T14:41:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01dcc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f01d8073170 con 0x7f01d8162940 2024-04-23T14:41:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01d6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01d8162d20 0x7f01d806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01d6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01d8162d20 0x7f01d806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46672/0 (socket says 172.21.15.59:46672) 2024-04-23T14:41:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01d6a00640 1 -- 172.21.15.59:0/3038154034 learned_addr learned my addr 172.21.15.59:0/3038154034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.819+0000 7f01d7400640 1 --2- 172.21.15.59:0/3038154034 >> v2:172.21.15.76:3300/0 conn(0x7f01d8162940 0x7f01d80af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.820+0000 7f01d6a00640 1 -- 172.21.15.59:0/3038154034 >> v2:172.21.15.76:3300/0 conn(0x7f01d8162940 msgr2=0x7f01d80af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.820+0000 7f01d6a00640 1 --2- 172.21.15.59:0/3038154034 >> v2:172.21.15.76:3300/0 conn(0x7f01d8162940 0x7f01d80af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.820+0000 7f01d6a00640 1 -- 172.21.15.59:0/3038154034 >> v1:172.21.15.76:6789/0 conn(0x7f01d80ccb40 legacy=0x7f01d80ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.820+0000 7f01d6a00640 1 -- 172.21.15.59:0/3038154034 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d8073450 con 0x7f01d8162d20 2024-04-23T14:41:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.820+0000 7f01d7400640 1 --2- 172.21.15.59:0/3038154034 >> v2:172.21.15.76:3300/0 conn(0x7f01d8162940 0x7f01d80af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:41:16.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.823+0000 7f01d6a00640 1 --2- 172.21.15.59:0/3038154034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d8162d20 0x7f01d806e720 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f01d806d440 tx=0x7f01c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5184a3d5f02f3813 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:16.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01d6000640 1 -- 172.21.15.59:0/3038154034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0037070 con 0x7f01d8162d20 2024-04-23T14:41:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01d6000640 1 -- 172.21.15.59:0/3038154034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c0032440 con 0x7f01d8162d20 2024-04-23T14:41:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01d6000640 1 -- 172.21.15.59:0/3038154034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0032810 con 0x7f01d8162d20 2024-04-23T14:41:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01dcc00640 1 -- 172.21.15.59:0/3038154034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d8162d20 msgr2=0x7f01d806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:16.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3038154034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d8162d20 0x7f01d806e720 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f01d806d440 tx=0x7f01c002a270 comp rx=0 tx=0).stop 2024-04-23T14:41:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01dcc00640 1 -- 172.21.15.59:0/3038154034 shutdown_connections 2024-04-23T14:41:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3038154034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d8162d20 0x7f01d806e720 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3038154034 >> v2:172.21.15.76:3300/0 conn(0x7f01d8162940 0x7f01d80af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01dcc00640 1 -- 172.21.15.59:0/3038154034 >> 172.21.15.59:0/3038154034 conn(0x7f01d80b1f90 msgr2=0x7f01d80b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.824+0000 7f01dcc00640 1 -- 172.21.15.59:0/3038154034 shutdown_connections 2024-04-23T14:41:16.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.825+0000 7f01dcc00640 1 -- 172.21.15.59:0/3038154034 wait complete. 2024-04-23T14:41:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.826+0000 7f01dcc00640 1 Processor -- start 2024-04-23T14:41:16.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.826+0000 7f01dcc00640 1 -- start start 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.827+0000 7f01dcc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d80ccb40 0x7f01d8165c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.827+0000 7f01dcc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d8162940 0x7f01d8159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.827+0000 7f01dcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d815b9e0 0x7f01d8164200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.827+0000 7f01dcc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01d80ce180 con 0x7f01d815b9e0 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.827+0000 7f01dcc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8073a10 con 0x7f01d8162940 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.827+0000 7f01dcc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f01d80ce300 con 0x7f01d80ccb40 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d815b9e0 0x7f01d8164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d815b9e0 0x7f01d8164200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46676/0 (socket says 172.21.15.59:46676) 2024-04-23T14:41:16.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 -- 172.21.15.59:0/3064569533 learned_addr learned my addr 172.21.15.59:0/3064569533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:16.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d6a00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d8162940 0x7f01d8159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:16.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7400640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d80ccb40 0x7f01d8165c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 -- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d80ccb40 msgr2=0x7f01d8165c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d80ccb40 0x7f01d8165c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 -- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d8162940 msgr2=0x7f01d8159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d8162940 0x7f01d8159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d7e00640 1 -- 172.21.15.59:0/3064569533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d8159fa0 con 0x7f01d815b9e0 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.828+0000 7f01d6a00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d8162940 0x7f01d8159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.829+0000 7f01d7e00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d815b9e0 0x7f01d8164200 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f01d0001eb0 tx=0x7f01d000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.829+0000 7f01cf400640 1 -- 172.21.15.59:0/3064569533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0017070 con 0x7f01d815b9e0 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.829+0000 7f01cf400640 1 -- 172.21.15.59:0/3064569533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d00052d0 con 0x7f01d815b9e0 2024-04-23T14:41:16.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.829+0000 7f01cf400640 1 -- 172.21.15.59:0/3064569533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0003810 con 0x7f01d815b9e0 2024-04-23T14:41:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.829+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d8163440 con 0x7f01d815b9e0 2024-04-23T14:41:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.829+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d8163a00 con 0x7f01d815b9e0 2024-04-23T14:41:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.830+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d80d05f0 con 0x7f01d815b9e0 2024-04-23T14:41:16.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.831+0000 7f01cf400640 1 -- 172.21.15.59:0/3064569533 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01d001a070 con 0x7f01d815b9e0 2024-04-23T14:41:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.832+0000 7f01cf400640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b406c8f0 0x7f01b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:16.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.832+0000 7f01cf400640 1 -- 172.21.15.59:0/3064569533 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5996+0+0 (secure 0 0 0) 0x7f01d008afe0 con 0x7f01d815b9e0 2024-04-23T14:41:16.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.833+0000 7f01d7400640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b406c8f0 0x7f01b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:16.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.833+0000 7f01d7400640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b406c8f0 0x7f01b406edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f01c8003fd0 tx=0x7f01c8003ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:16.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:16.837+0000 7f01cf400640 1 -- 172.21.15.59:0/3064569533 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01d80d05f0 con 0x7f01d815b9e0 2024-04-23T14:41:17.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:17.045+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01d804fd90 con 0x7f01d815b9e0 2024-04-23T14:41:18.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.016+0000 7f01cf400640 1 -- 172.21.15.59:0/3064569533 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7f01d806ec60 con 0x7f01d815b9e0 2024-04-23T14:41:18.017 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b406c8f0 msgr2=0x7f01b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b406c8f0 0x7f01b406edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f01c8003fd0 tx=0x7f01c8003ce0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d815b9e0 msgr2=0x7f01d8164200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d815b9e0 0x7f01d8164200 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f01d0001eb0 tx=0x7f01d000c890 comp rx=0 tx=0).stop 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 shutdown_connections 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01b406c8f0 0x7f01b406edb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01d815b9e0 0x7f01d8164200 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01d8162940 0x7f01d8159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 --2- 172.21.15.59:0/3064569533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01d80ccb40 0x7f01d8165c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 >> 172.21.15.59:0/3064569533 conn(0x7f01d80b1f90 msgr2=0x7f01d80cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:18.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 shutdown_connections 2024-04-23T14:41:18.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.020+0000 7f01dcc00640 1 -- 172.21.15.59:0/3064569533 wait complete. 2024-04-23T14:41:18.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:41:18.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.263+0000 7f4fd1200640 1 Processor -- start 2024-04-23T14:41:18.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.263+0000 7f4fd1200640 1 -- start start 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.264+0000 7f4fd1200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4fcc15e8e0 0x7f4fcc0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.264+0000 7f4fd1200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc06d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.265+0000 7f4fd1200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fcc0cf070 con 0x7f4fcc0d4210 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.265+0000 7f4fd1200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4fcc0cf2f0 con 0x7f4fcc15e8e0 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.265+0000 7f4fd1200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc0d1c60 con 0x7f4fcc06cbb0 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.265+0000 7f4fcb400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4fcc15e8e0 0x7f4fcc0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.265+0000 7f4fcb400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4fcc15e8e0 0x7f4fcc0cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46692/0 (socket says 172.21.15.59:46692) 2024-04-23T14:41:18.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.265+0000 7f4fcb400640 1 -- 172.21.15.59:0/3495872323 learned_addr learned my addr 172.21.15.59:0/3495872323 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:18.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.265+0000 7f4fcbe00640 1 --2- 172.21.15.59:0/3495872323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc06d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:18.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.266+0000 7f4fcb400640 1 -- 172.21.15.59:0/3495872323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 msgr2=0x7f4fcc06d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:18.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.266+0000 7f4fcb400640 1 --2- 172.21.15.59:0/3495872323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc06d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.266+0000 7f4fcb400640 1 -- 172.21.15.59:0/3495872323 >> v1:172.21.15.59:6789/0 conn(0x7f4fcc0d4210 legacy=0x7f4fcc0d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:18.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.266+0000 7f4fcb400640 1 -- 172.21.15.59:0/3495872323 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc0cf5b0 con 0x7f4fcc15e8e0 2024-04-23T14:41:18.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.268+0000 7f4fcb400640 1 --2- 172.21.15.59:0/3495872323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc15e8e0 0x7f4fcc0cfdb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009870 tx=0x7f4fbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3cc11356602b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:18.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.269+0000 7f4fca000640 1 -- 172.21.15.59:0/3495872323 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc037070 con 0x7f4fcc15e8e0 2024-04-23T14:41:18.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.269+0000 7f4fca000640 1 -- 172.21.15.59:0/3495872323 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fbc032440 con 0x7f4fcc15e8e0 2024-04-23T14:41:18.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.269+0000 7f4fca000640 1 -- 172.21.15.59:0/3495872323 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc032810 con 0x7f4fcc15e8e0 2024-04-23T14:41:18.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 -- 172.21.15.59:0/3495872323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc15e8e0 msgr2=0x7f4fcc0cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:18.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 --2- 172.21.15.59:0/3495872323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc15e8e0 0x7f4fcc0cfdb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009870 tx=0x7f4fbc02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 -- 172.21.15.59:0/3495872323 shutdown_connections 2024-04-23T14:41:18.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 --2- 172.21.15.59:0/3495872323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc06d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 --2- 172.21.15.59:0/3495872323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc15e8e0 0x7f4fcc0cfdb0 secure :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009870 tx=0x7f4fbc02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 -- 172.21.15.59:0/3495872323 >> 172.21.15.59:0/3495872323 conn(0x7f4fcc08e060 msgr2=0x7f4fcc09d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:18.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 -- 172.21.15.59:0/3495872323 shutdown_connections 2024-04-23T14:41:18.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.271+0000 7f4fd1200640 1 -- 172.21.15.59:0/3495872323 wait complete. 2024-04-23T14:41:18.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.274+0000 7f4fd1200640 1 Processor -- start 2024-04-23T14:41:18.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.275+0000 7f4fd1200640 1 -- start start 2024-04-23T14:41:18.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.277+0000 7f4fd1200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:18.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.277+0000 7f4fd1200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fcc0d4210 0x7f4fcc1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.277+0000 7f4fd1200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc1608a0 0x7f4fcc160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.277+0000 7f4fd1200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc15ecc0 con 0x7f4fcc1608a0 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.277+0000 7f4fd1200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc06d580 con 0x7f4fcc0d4210 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.277+0000 7f4fd1200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4fcc0d02f0 con 0x7f4fcc06cbb0 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.278+0000 7f4fcaa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fcc0d4210 0x7f4fcc1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.278+0000 7f4fcbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc1608a0 0x7f4fcc160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.278+0000 7f4fcbe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc1608a0 0x7f4fcc160d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46694/0 (socket says 172.21.15.59:46694) 2024-04-23T14:41:18.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.278+0000 7f4fcbe00640 1 -- 172.21.15.59:0/1024705484 learned_addr learned my addr 172.21.15.59:0/1024705484 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:18.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.278+0000 7f4fcb400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:18.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.280+0000 7f4fcbe00640 1 -- 172.21.15.59:0/1024705484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 msgr2=0x7f4fcc150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:18.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.280+0000 7f4fcbe00640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.280+0000 7f4fcbe00640 1 -- 172.21.15.59:0/1024705484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fcc0d4210 msgr2=0x7f4fcc1507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:18.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.280+0000 7f4fcbe00640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fcc0d4210 0x7f4fcc1507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:18.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.280+0000 7f4fcbe00640 1 -- 172.21.15.59:0/1024705484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc151070 con 0x7f4fcc1608a0 2024-04-23T14:41:18.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.280+0000 7f4fcbe00640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc1608a0 0x7f4fcc160d50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400d800 tx=0x7f4fc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:18.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.281+0000 7f4fc3400640 1 -- 172.21.15.59:0/1024705484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400b6c0 con 0x7f4fcc1608a0 2024-04-23T14:41:18.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.281+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc161460 con 0x7f4fcc1608a0 2024-04-23T14:41:18.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.281+0000 7f4fc3400640 1 -- 172.21.15.59:0/1024705484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc4019070 con 0x7f4fcc1608a0 2024-04-23T14:41:18.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.281+0000 7f4fc3400640 1 -- 172.21.15.59:0/1024705484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400bac0 con 0x7f4fcc1608a0 2024-04-23T14:41:18.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.281+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc14ad60 con 0x7f4fcc1608a0 2024-04-23T14:41:18.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.282+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fcc04fd90 con 0x7f4fcc1608a0 2024-04-23T14:41:18.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.283+0000 7f4fc3400640 1 -- 172.21.15.59:0/1024705484 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fc400bc60 con 0x7f4fcc1608a0 2024-04-23T14:41:18.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.284+0000 7f4fc3400640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fa806c8f0 0x7f4fa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:18.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.284+0000 7f4fc3400640 1 -- 172.21.15.59:0/1024705484 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5558+0+0 (secure 0 0 0) 0x7f4fc408bea0 con 0x7f4fcc1608a0 2024-04-23T14:41:18.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.284+0000 7f4fcb400640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fa806c8f0 0x7f4fa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:18.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.285+0000 7f4fcb400640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fa806c8f0 0x7f4fa806edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc034000 tx=0x7f4fbc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.289+0000 7f4fc3400640 1 -- 172.21.15.59:0/1024705484 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4fc4059c70 con 0x7f4fcc1608a0 2024-04-23T14:41:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:18.493+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4fcc0cb050 con 0x7f4fcc1608a0 2024-04-23T14:41:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.026+0000 7f4fc3400640 1 -- 172.21.15.59:0/1024705484 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f4fc405df70 con 0x7f4fcc1608a0 2024-04-23T14:41:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.031+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fa806c8f0 msgr2=0x7f4fa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.031+0000 7f4fd1200640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fa806c8f0 0x7f4fa806edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc034000 tx=0x7f4fbc02e000 comp rx=0 tx=0).stop 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.031+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc1608a0 msgr2=0x7f4fcc160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.031+0000 7f4fd1200640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc1608a0 0x7f4fcc160d50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400d800 tx=0x7f4fc400dcc0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.032+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 shutdown_connections 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.032+0000 7f4fd1200640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4fa806c8f0 0x7f4fa806edb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.032+0000 7f4fd1200640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fcc1608a0 0x7f4fcc160d50 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.032+0000 7f4fd1200640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fcc0d4210 0x7f4fcc1507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.032+0000 7f4fd1200640 1 --2- 172.21.15.59:0/1024705484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4fcc06cbb0 0x7f4fcc150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.032+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 >> 172.21.15.59:0/1024705484 conn(0x7f4fcc08e060 msgr2=0x7f4fcc0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:19.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.033+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 shutdown_connections 2024-04-23T14:41:19.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.033+0000 7f4fd1200640 1 -- 172.21.15.59:0/1024705484 wait complete. 2024-04-23T14:41:19.058 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:41:19.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.249+0000 7fbfd3000640 1 Processor -- start 2024-04-23T14:41:19.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.250+0000 7fbfd3000640 1 -- start start 2024-04-23T14:41:19.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.251+0000 7fbfd3000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfcc0b4f20 0x7fbfcc0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.251+0000 7fbfd3000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfcc0b5da0 con 0x7fbfcc15ed20 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.251+0000 7fbfd3000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfcc0d0a20 con 0x7fbfcc0d15b0 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.251+0000 7fbfd3000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbfcc15f100 con 0x7fbfcc0b4f20 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.251+0000 7fbfd1c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbfcc15ed20 0x7fbfcc0d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58872/0 (socket says 172.21.15.59:58872) 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.251+0000 7fbfd1c00640 1 -- 172.21.15.59:0/1209151510 learned_addr learned my addr 172.21.15.59:0/1209151510 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.252+0000 7fbfd2600640 1 --2- 172.21.15.59:0/1209151510 >> v2:172.21.15.59:3300/0 conn(0x7fbfcc0b4f20 0x7fbfcc0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.252+0000 7fbfcbe00640 1 -- 172.21.15.59:0/1209151510 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1200162530 0 0) 0x7fbfcc0b5da0 con 0x7fbfcc15ed20 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.252+0000 7fbfd2600640 1 -- 172.21.15.59:0/1209151510 >> v1:172.21.15.76:6789/0 conn(0x7fbfcc0d15b0 legacy=0x7fbfcc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.252+0000 7fbfd2600640 1 -- 172.21.15.59:0/1209151510 >> v1:172.21.15.59:6789/0 conn(0x7fbfcc15ed20 legacy=0x7fbfcc0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.252+0000 7fbfd2600640 1 -- 172.21.15.59:0/1209151510 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc0d2ed0 con 0x7fbfcc0b4f20 2024-04-23T14:41:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.253+0000 7fbfd2600640 1 --2- 172.21.15.59:0/1209151510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc0b4f20 0x7fbfcc0cf990 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009d90 tx=0x7fbfc402a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7323c5429687da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.253+0000 7fbfcbe00640 1 -- 172.21.15.59:0/1209151510 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4007740 con 0x7fbfcc0b4f20 2024-04-23T14:41:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.253+0000 7fbfcbe00640 1 -- 172.21.15.59:0/1209151510 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc40078e0 con 0x7fbfcc0b4f20 2024-04-23T14:41:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.253+0000 7fbfcbe00640 1 -- 172.21.15.59:0/1209151510 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4007cb0 con 0x7fbfcc0b4f20 2024-04-23T14:41:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.254+0000 7fbfd3000640 1 -- 172.21.15.59:0/1209151510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc0b4f20 msgr2=0x7fbfcc0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.254+0000 7fbfd3000640 1 --2- 172.21.15.59:0/1209151510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc0b4f20 0x7fbfcc0cf990 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009d90 tx=0x7fbfc402a590 comp rx=0 tx=0).stop 2024-04-23T14:41:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.254+0000 7fbfd3000640 1 -- 172.21.15.59:0/1209151510 shutdown_connections 2024-04-23T14:41:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.254+0000 7fbfd3000640 1 --2- 172.21.15.59:0/1209151510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc0b4f20 0x7fbfcc0cf990 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.254+0000 7fbfd3000640 1 -- 172.21.15.59:0/1209151510 >> 172.21.15.59:0/1209151510 conn(0x7fbfcc08bf90 msgr2=0x7fbfcc0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.254+0000 7fbfd3000640 1 -- 172.21.15.59:0/1209151510 shutdown_connections 2024-04-23T14:41:19.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.254+0000 7fbfd3000640 1 -- 172.21.15.59:0/1209151510 wait complete. 2024-04-23T14:41:19.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.256+0000 7fbfd3000640 1 Processor -- start 2024-04-23T14:41:19.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.257+0000 7fbfd3000640 1 -- start start 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd3000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfcc0b4f20 0x7fbfcc06de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd3000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfcc0d15b0 0x7fbfcc06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd3000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc15ed20 0x7fbfcc06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd3000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc0b3600 con 0x7fbfcc15ed20 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd3000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc0d3490 con 0x7fbfcc0d15b0 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd3000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbfcc0b3780 con 0x7fbfcc0b4f20 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc15ed20 0x7fbfcc06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd2600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc15ed20 0x7fbfcc06d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46716/0 (socket says 172.21.15.59:46716) 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd2600640 1 -- 172.21.15.59:0/575054326 learned_addr learned my addr 172.21.15.59:0/575054326 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:19.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd1c00640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfcc0b4f20 0x7fbfcc06de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.258+0000 7fbfd1200640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfcc0d15b0 0x7fbfcc06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfd2600640 1 -- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfcc0b4f20 msgr2=0x7fbfcc06de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfd2600640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfcc0b4f20 0x7fbfcc06de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfd2600640 1 -- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfcc0d15b0 msgr2=0x7fbfcc06cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfd2600640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfcc0d15b0 0x7fbfcc06cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfd2600640 1 -- 172.21.15.59:0/575054326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc06e350 con 0x7fbfcc15ed20 2024-04-23T14:41:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfd2600640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc15ed20 0x7fbfcc06d0a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009eb0 tx=0x7fbfc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfc9600640 1 -- 172.21.15.59:0/575054326 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4040070 con 0x7fbfcc15ed20 2024-04-23T14:41:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfc9600640 1 -- 172.21.15.59:0/575054326 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc4031c90 con 0x7fbfcc15ed20 2024-04-23T14:41:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfc9600640 1 -- 172.21.15.59:0/575054326 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4045680 con 0x7fbfcc15ed20 2024-04-23T14:41:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.259+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc1f47e0 con 0x7fbfcc15ed20 2024-04-23T14:41:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.260+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc1f4da0 con 0x7fbfcc15ed20 2024-04-23T14:41:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.261+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfcc15e430 con 0x7fbfcc15ed20 2024-04-23T14:41:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.262+0000 7fbfc9600640 1 -- 172.21.15.59:0/575054326 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfc4039040 con 0x7fbfcc15ed20 2024-04-23T14:41:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.263+0000 7fbfc9600640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfac06c8f0 0x7fbfac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.263+0000 7fbfc9600640 1 -- 172.21.15.59:0/575054326 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fbfc40ad2b0 con 0x7fbfcc15ed20 2024-04-23T14:41:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.263+0000 7fbfd1c00640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfac06c8f0 0x7fbfac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.264+0000 7fbfd1c00640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfac06c8f0 0x7fbfac06edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0000f60 tx=0x7fbfc0007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.269+0000 7fbfc9600640 1 -- 172.21.15.59:0/575054326 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfcc15e430 con 0x7fbfcc15ed20 2024-04-23T14:41:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.474+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fbfcc0c8090 con 0x7fbfcc15ed20 2024-04-23T14:41:19.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.479+0000 7fbfc9600640 1 -- 172.21.15.59:0/575054326 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fbfcc06e350 con 0x7fbfcc15ed20 2024-04-23T14:41:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfac06c8f0 msgr2=0x7fbfac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfac06c8f0 0x7fbfac06edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0000f60 tx=0x7fbfc0007c00 comp rx=0 tx=0).stop 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc15ed20 msgr2=0x7fbfcc06d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc15ed20 0x7fbfcc06d0a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009eb0 tx=0x7fbfc4004a40 comp rx=0 tx=0).stop 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 shutdown_connections 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfac06c8f0 0x7fbfac06edb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfcc15ed20 0x7fbfcc06d0a0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfcc0d15b0 0x7fbfcc06cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 --2- 172.21.15.59:0/575054326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfcc0b4f20 0x7fbfcc06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 >> 172.21.15.59:0/575054326 conn(0x7fbfcc08bf90 msgr2=0x7fbfcc0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 shutdown_connections 2024-04-23T14:41:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.483+0000 7fbfd3000640 1 -- 172.21.15.59:0/575054326 wait complete. 2024-04-23T14:41:19.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-23T14:41:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.720+0000 7f43ade00640 1 Processor -- start 2024-04-23T14:41:19.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.720+0000 7f43ade00640 1 -- start start 2024-04-23T14:41:19.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.722+0000 7f43ade00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a815e7c0 0x7f43a80d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.722+0000 7f43ade00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43a80d0d90 0x7f43a80d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.722+0000 7f43ade00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a80cf030 con 0x7f43a80d2630 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.722+0000 7f43ade00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f43a80b7000 con 0x7f43a80d0d90 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.722+0000 7f43ade00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f43a80cf620 con 0x7f43a815e7c0 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.722+0000 7f439fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f43a80d2630 0x7f43a80b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58876/0 (socket says 172.21.15.59:58876) 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.722+0000 7f439fe00640 1 -- 172.21.15.59:0/3749102398 learned_addr learned my addr 172.21.15.59:0/3749102398 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.723+0000 7f43ad400640 1 --2- 172.21.15.59:0/3749102398 >> v2:172.21.15.76:3300/0 conn(0x7f43a80d0d90 0x7f43a80d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.723+0000 7f43aca00640 1 --2- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a815e7c0 0x7f43a80d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.723+0000 7f43a7e00640 1 -- 172.21.15.59:0/3749102398 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4154164471 0 0) 0x7f4390001110 con 0x7f43a80d2630 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.723+0000 7f43a7e00640 1 -- 172.21.15.59:0/3749102398 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f438c003520 con 0x7f43a80d2630 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.723+0000 7f43ad400640 1 -- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a815e7c0 msgr2=0x7f43a80d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.723+0000 7f43ad400640 1 --2- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a815e7c0 0x7f43a80d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.724+0000 7f43ad400640 1 -- 172.21.15.59:0/3749102398 >> v1:172.21.15.59:6789/0 conn(0x7f43a80d2630 legacy=0x7f43a80b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.724+0000 7f43a7e00640 1 -- 172.21.15.59:0/3749102398 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1479683797 0 0) 0x7f438c003520 con 0x7f43a80d2630 2024-04-23T14:41:19.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.724+0000 7f43ad400640 1 -- 172.21.15.59:0/3749102398 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a80b6a40 con 0x7f43a80d0d90 2024-04-23T14:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.725+0000 7f43ad400640 1 --2- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a80d0d90 0x7f43a80d1220 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009d90 tx=0x7f43a002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7eadd99b94d5ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.725+0000 7f43a7e00640 1 -- 172.21.15.59:0/3749102398 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390001840 con 0x7f43a80d0d90 2024-04-23T14:41:19.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.725+0000 7f43a7e00640 1 -- 172.21.15.59:0/3749102398 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a0007540 con 0x7f43a80d0d90 2024-04-23T14:41:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.725+0000 7f43a7e00640 1 -- 172.21.15.59:0/3749102398 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0007910 con 0x7f43a80d0d90 2024-04-23T14:41:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 -- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a80d0d90 msgr2=0x7f43a80d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 --2- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a80d0d90 0x7f43a80d1220 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009d90 tx=0x7f43a002a5f0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 -- 172.21.15.59:0/3749102398 shutdown_connections 2024-04-23T14:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 --2- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a80d0d90 0x7f43a80d1220 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 --2- 172.21.15.59:0/3749102398 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a815e7c0 0x7f43a80d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 -- 172.21.15.59:0/3749102398 >> 172.21.15.59:0/3749102398 conn(0x7f43a809d920 msgr2=0x7f43a808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 -- 172.21.15.59:0/3749102398 shutdown_connections 2024-04-23T14:41:19.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.726+0000 7f43ade00640 1 -- 172.21.15.59:0/3749102398 wait complete. 2024-04-23T14:41:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.728+0000 7f43ade00640 1 Processor -- start 2024-04-23T14:41:19.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.728+0000 7f43ade00640 1 -- start start 2024-04-23T14:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.729+0000 7f43ade00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a80d0d90 0x7f43a81e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.729+0000 7f43ade00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a80d2630 0x7f43a8157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.729+0000 7f43ade00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a815e7c0 0x7f43a815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.729+0000 7f43ade00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43a80b6700 con 0x7f43a80d0d90 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.729+0000 7f43ade00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f43a80cf2b0 con 0x7f43a815e7c0 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.729+0000 7f43ade00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f43a80d1a80 con 0x7f43a80d2630 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a80d0d90 0x7f43a81e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a80d0d90 0x7f43a81e92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46720/0 (socket says 172.21.15.59:46720) 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 -- 172.21.15.59:0/3522845258 learned_addr learned my addr 172.21.15.59:0/3522845258 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43ad400640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a815e7c0 0x7f43a815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f439fe00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a80d2630 0x7f43a8157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 -- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a80d2630 msgr2=0x7f43a8157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a80d2630 0x7f43a8157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 -- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a815e7c0 msgr2=0x7f43a815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a815e7c0 0x7f43a815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:19.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f43aca00640 1 -- 172.21.15.59:0/3522845258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a81e98d0 con 0x7f43a80d0d90 2024-04-23T14:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.730+0000 7f439fe00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a80d2630 0x7f43a8157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:19.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.731+0000 7f43aca00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a80d0d90 0x7f43a81e92f0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f439800e870 tx=0x7f439800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.731+0000 7f43a5600640 1 -- 172.21.15.59:0/3522845258 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398017070 con 0x7f43a80d0d90 2024-04-23T14:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.731+0000 7f43a5600640 1 -- 172.21.15.59:0/3522845258 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439800c940 con 0x7f43a80d0d90 2024-04-23T14:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.731+0000 7f43a5600640 1 -- 172.21.15.59:0/3522845258 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439800cd30 con 0x7f43a80d0d90 2024-04-23T14:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.731+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a81e9bb0 con 0x7f43a80d0d90 2024-04-23T14:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.731+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a81ea170 con 0x7f43a80d0d90 2024-04-23T14:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.732+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a804fd90 con 0x7f43a80d0d90 2024-04-23T14:41:19.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.733+0000 7f43a5600640 1 -- 172.21.15.59:0/3522845258 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f439801a070 con 0x7f43a80d0d90 2024-04-23T14:41:19.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.734+0000 7f43a5600640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f438406c8f0 0x7f438406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:19.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.734+0000 7f43a5600640 1 -- 172.21.15.59:0/3522845258 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f439808bbb0 con 0x7f43a80d0d90 2024-04-23T14:41:19.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.735+0000 7f439fe00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f438406c8f0 0x7f438406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:19.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.735+0000 7f439fe00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f438406c8f0 0x7f438406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4390001eb0 tx=0x7f4390009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:19.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:19.739+0000 7f43a5600640 1 -- 172.21.15.59:0/3522845258 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f439805a220 con 0x7f43a80d0d90 2024-04-23T14:41:20.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.019+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f43a815ca90 con 0x7f43a80d0d90 2024-04-23T14:41:20.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.029+0000 7f43a5600640 1 -- 172.21.15.59:0/3522845258 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v470) ==== 167+0+0 (secure 0 0 0) 0x7f4398059b20 con 0x7f43a80d0d90 2024-04-23T14:41:20.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.031+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f438406c8f0 msgr2=0x7f438406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.031+0000 7f43ade00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f438406c8f0 0x7f438406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4390001eb0 tx=0x7f4390009040 comp rx=0 tx=0).stop 2024-04-23T14:41:20.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.031+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a80d0d90 msgr2=0x7f43a81e92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.031+0000 7f43ade00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a80d0d90 0x7f43a81e92f0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f439800e870 tx=0x7f439800ec70 comp rx=0 tx=0).stop 2024-04-23T14:41:20.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 shutdown_connections 2024-04-23T14:41:20.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f438406c8f0 0x7f438406edb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43a80d2630 0x7f43a8157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43a815e7c0 0x7f43a815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 --2- 172.21.15.59:0/3522845258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43a80d0d90 0x7f43a81e92f0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 >> 172.21.15.59:0/3522845258 conn(0x7f43a809d920 msgr2=0x7f43a81616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:20.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 shutdown_connections 2024-04-23T14:41:20.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.032+0000 7f43ade00640 1 -- 172.21.15.59:0/3522845258 wait complete. 2024-04-23T14:41:20.055 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-23T14:41:20.055 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-23T14:41:20.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:20.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.253+0000 7f4ee5200640 1 Processor -- start 2024-04-23T14:41:20.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.253+0000 7f4ee5200640 1 -- start start 2024-04-23T14:41:20.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ee5200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee015e7c0 0x7f4ee00d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ee5200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ee00d2630 0x7f4ee00b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ee5200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ee00b6530 0x7f4ee00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ee5200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee00b7000 con 0x7f4ee00d2630 2024-04-23T14:41:20.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ee5200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee00cf620 con 0x7f4ee00b6530 2024-04-23T14:41:20.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ee5200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee00cf030 con 0x7f4ee015e7c0 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ed6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ee00d2630 0x7f4ee00b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ed6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ee00d2630 0x7f4ee00b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44628/0 (socket says 172.21.15.59:44628) 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.255+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2436632751 learned_addr learned my addr 172.21.15.59:0/2436632751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.256+0000 7f4edfe00640 1 --2- 172.21.15.59:0/2436632751 >> v2:172.21.15.76:3300/0 conn(0x7f4ee00b6530 0x7f4ee00d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.256+0000 7f4edf400640 1 --2- 172.21.15.59:0/2436632751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee015e7c0 0x7f4ee00d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.256+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2436632751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee015e7c0 msgr2=0x7f4ee00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.256+0000 7f4ed6a00640 1 --2- 172.21.15.59:0/2436632751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee015e7c0 0x7f4ee00d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.256+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2436632751 >> v2:172.21.15.76:3300/0 conn(0x7f4ee00b6530 msgr2=0x7f4ee00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:20.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.256+0000 7f4ed6a00640 1 --2- 172.21.15.59:0/2436632751 >> v2:172.21.15.76:3300/0 conn(0x7f4ee00b6530 0x7f4ee00d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.256+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2436632751 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee00b6a40 con 0x7f4ee00d2630 2024-04-23T14:41:20.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.257+0000 7f4ed6a00640 1 --2- 172.21.15.59:0/2436632751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b5ff0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00cf590 tx=0x7f4ec802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eda5e61c9a0306 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:20.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.258+0000 7f4edea00640 1 -- 172.21.15.59:0/2436632751 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8036070 con 0x7f4ee00d2630 2024-04-23T14:41:20.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.258+0000 7f4edea00640 1 -- 172.21.15.59:0/2436632751 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec8031600 con 0x7f4ee00d2630 2024-04-23T14:41:20.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.258+0000 7f4edea00640 1 -- 172.21.15.59:0/2436632751 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80319d0 con 0x7f4ee00d2630 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.258+0000 7f4ee5200640 1 -- 172.21.15.59:0/2436632751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 msgr2=0x7f4ee00b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.258+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2436632751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b5ff0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00cf590 tx=0x7f4ec802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.259+0000 7f4ee5200640 1 -- 172.21.15.59:0/2436632751 shutdown_connections 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.259+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2436632751 >> v2:172.21.15.76:3300/0 conn(0x7f4ee00b6530 0x7f4ee00d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.259+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2436632751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b5ff0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.259+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2436632751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee015e7c0 0x7f4ee00d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.259+0000 7f4ee5200640 1 -- 172.21.15.59:0/2436632751 >> 172.21.15.59:0/2436632751 conn(0x7f4ee009d920 msgr2=0x7f4ee008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.259+0000 7f4ee5200640 1 -- 172.21.15.59:0/2436632751 shutdown_connections 2024-04-23T14:41:20.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.259+0000 7f4ee5200640 1 -- 172.21.15.59:0/2436632751 wait complete. 2024-04-23T14:41:20.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.260+0000 7f4ee5200640 1 Processor -- start 2024-04-23T14:41:20.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.261+0000 7f4ee5200640 1 -- start start 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.261+0000 7f4ee5200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee00b6530 0x7f4ee0148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.261+0000 7f4ee5200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.261+0000 7f4ee5200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee015e7c0 0x7f4ee00b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.261+0000 7f4ee5200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee00d1a80 con 0x7f4ee00d2630 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.261+0000 7f4ee5200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee00cf2b0 con 0x7f4ee015e7c0 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.261+0000 7f4ee5200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee00d1c00 con 0x7f4ee00b6530 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44642/0 (socket says 172.21.15.59:44642) 2024-04-23T14:41:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2913476549 learned_addr learned my addr 172.21.15.59:0/2913476549 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:20.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4edf400640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee00b6530 0x7f4ee0148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4edfe00640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee015e7c0 0x7f4ee00b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee00b6530 msgr2=0x7f4ee0148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee00b6530 0x7f4ee0148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee015e7c0 msgr2=0x7f4ee00b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee015e7c0 0x7f4ee00b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4ed6a00640 1 -- 172.21.15.59:0/2913476549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee0148f60 con 0x7f4ee00d2630 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.262+0000 7f4edfe00640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee015e7c0 0x7f4ee00b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.263+0000 7f4ed6a00640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b11a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8034040 tx=0x7f4ec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.263+0000 7f4ed7e00640 1 -- 172.21.15.59:0/2913476549 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8041020 con 0x7f4ee00d2630 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.263+0000 7f4ed7e00640 1 -- 172.21.15.59:0/2913476549 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec8007820 con 0x7f4ee00d2630 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.263+0000 7f4ed7e00640 1 -- 172.21.15.59:0/2913476549 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8036040 con 0x7f4ee00d2630 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.263+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee00cdd30 con 0x7f4ee00d2630 2024-04-23T14:41:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.263+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee00ce2f0 con 0x7f4ee00d2630 2024-04-23T14:41:20.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.264+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee004fd90 con 0x7f4ee00d2630 2024-04-23T14:41:20.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.265+0000 7f4ed7e00640 1 -- 172.21.15.59:0/2913476549 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ec8044020 con 0x7f4ee00d2630 2024-04-23T14:41:20.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.266+0000 7f4ed7e00640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ebc06c920 0x7f4ebc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.266+0000 7f4ed7e00640 1 -- 172.21.15.59:0/2913476549 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f4ec80ac820 con 0x7f4ee00d2630 2024-04-23T14:41:20.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.267+0000 7f4edf400640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ebc06c920 0x7f4ebc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.267+0000 7f4edf400640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ebc06c920 0x7f4ebc06ede0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:20.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.271+0000 7f4ed7e00640 1 -- 172.21.15.59:0/2913476549 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ec804b030 con 0x7f4ee00d2630 2024-04-23T14:41:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.469+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ee015f4e0 con 0x7f4ee00d2630 2024-04-23T14:41:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.470+0000 7f4ed7e00640 1 -- 172.21.15.59:0/2913476549 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+13221 (secure 0 0 0) 0x7f4ee0148f60 con 0x7f4ee00d2630 2024-04-23T14:41:20.472 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:20.472 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:19.017711+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3466627380":"2024-04-24T14:41:15.526852+0000","172.21.15.59:6835/3466627380":"2024-04-24T14:41:15.526852+0000","172.21.15.59:6837/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.59:6835/2128549219":"2024-04-24T14:40:25.737762+0000","172.21.15.76:6835/3870021000":"2024-04-24T14:39:01.426690+0000","172.21.15.76:6833/3870021000":"2024-04-24T14:39:01.426690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ebc06c920 msgr2=0x7f4ebc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ebc06c920 0x7f4ebc06ede0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0008040 comp rx=0 tx=0).stop 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 msgr2=0x7f4ee00b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b11a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8034040 tx=0x7f4ec8004a40 comp rx=0 tx=0).stop 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 shutdown_connections 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ebc06c920 0x7f4ebc06ede0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee015e7c0 0x7f4ee00b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee00d2630 0x7f4ee00b11a0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 --2- 172.21.15.59:0/2913476549 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee00b6530 0x7f4ee0148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 >> 172.21.15.59:0/2913476549 conn(0x7f4ee009d920 msgr2=0x7f4ee015fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 shutdown_connections 2024-04-23T14:41:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.473+0000 7f4ee5200640 1 -- 172.21.15.59:0/2913476549 wait complete. 2024-04-23T14:41:20.493 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:20.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:20.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.700+0000 7f2aa0a00640 1 Processor -- start 2024-04-23T14:41:20.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.700+0000 7f2aa0a00640 1 -- start start 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.701+0000 7f2aa0a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a9c06f6f0 0x7f2a9c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2aa0a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2a9c071d70 0x7f2a9c0721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2aa0a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9c0adc80 con 0x7f2a9c15e790 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2aa0a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9c0aef40 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2aa0a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9c0afd80 con 0x7f2a9c071d70 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a99400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a9c06f6f0 0x7f2a9c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a99400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a9c06f6f0 0x7f2a9c0aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44656/0 (socket says 172.21.15.59:44656) 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a99400640 1 -- 172.21.15.59:0/1678520599 learned_addr learned my addr 172.21.15.59:0/1678520599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a9a800640 1 --2- 172.21.15.59:0/1678520599 >> v2:172.21.15.76:3300/0 conn(0x7f2a9c071d70 0x7f2a9c0721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a99400640 1 -- 172.21.15.59:0/1678520599 >> v2:172.21.15.76:3300/0 conn(0x7f2a9c071d70 msgr2=0x7f2a9c0721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a99400640 1 --2- 172.21.15.59:0/1678520599 >> v2:172.21.15.76:3300/0 conn(0x7f2a9c071d70 0x7f2a9c0721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a99400640 1 -- 172.21.15.59:0/1678520599 >> v1:172.21.15.76:6789/0 conn(0x7f2a9c15e790 legacy=0x7f2a9c162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:20.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.702+0000 7f2a99400640 1 -- 172.21.15.59:0/1678520599 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c0adf20 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.703+0000 7f2a99400640 1 --2- 172.21.15.59:0/1678520599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c0aea00 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0b5b20 tx=0x7f2a8402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e3aabeb630d64e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:20.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.704+0000 7f2a98a00640 1 -- 172.21.15.59:0/1678520599 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84037070 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.704+0000 7f2a98a00640 1 -- 172.21.15.59:0/1678520599 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84032440 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.704+0000 7f2a98a00640 1 -- 172.21.15.59:0/1678520599 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84032810 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 -- 172.21.15.59:0/1678520599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 msgr2=0x7f2a9c0aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/1678520599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c0aea00 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0b5b20 tx=0x7f2a8402a200 comp rx=0 tx=0).stop 2024-04-23T14:41:20.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 -- 172.21.15.59:0/1678520599 shutdown_connections 2024-04-23T14:41:20.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/1678520599 >> v2:172.21.15.76:3300/0 conn(0x7f2a9c071d70 0x7f2a9c0721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/1678520599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c0aea00 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 -- 172.21.15.59:0/1678520599 >> 172.21.15.59:0/1678520599 conn(0x7f2a9c067d90 msgr2=0x7f2a9c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:20.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 -- 172.21.15.59:0/1678520599 shutdown_connections 2024-04-23T14:41:20.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.705+0000 7f2aa0a00640 1 -- 172.21.15.59:0/1678520599 wait complete. 2024-04-23T14:41:20.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.706+0000 7f2aa0a00640 1 Processor -- start 2024-04-23T14:41:20.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.707+0000 7f2aa0a00640 1 -- start start 2024-04-23T14:41:20.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.707+0000 7f2aa0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c1f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.707+0000 7f2aa0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a9c071d70 0x7f2a9c0cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.707+0000 7f2aa0a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c15e790 0x7f2a9c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.707+0000 7f2aa0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c0ae4e0 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.707+0000 7f2aa0a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c0726e0 con 0x7f2a9c15e790 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.707+0000 7f2aa0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c06fbb0 con 0x7f2a9c071d70 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c1f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c1f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44670/0 (socket says 172.21.15.59:44670) 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 -- 172.21.15.59:0/631306795 learned_addr learned my addr 172.21.15.59:0/631306795 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a9a800640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c15e790 0x7f2a9c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 -- 172.21.15.59:0/631306795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a9c071d70 msgr2=0x7f2a9c0cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a9c071d70 0x7f2a9c0cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 -- 172.21.15.59:0/631306795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c15e790 msgr2=0x7f2a9c15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c15e790 0x7f2a9c15f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.708+0000 7f2a99e00640 1 -- 172.21.15.59:0/631306795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c1f7ca0 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.709+0000 7f2a99e00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c1f4660 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c7c0 tx=0x7f2a8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:20.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.709+0000 7f2a92000640 1 -- 172.21.15.59:0/631306795 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c016020 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.709+0000 7f2a92000640 1 -- 172.21.15.59:0/631306795 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a8c00aa50 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.709+0000 7f2a92000640 1 -- 172.21.15.59:0/631306795 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c012420 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.709+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c1f4f90 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.709+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c1f5550 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.710+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c068a80 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.711+0000 7f2a92000640 1 -- 172.21.15.59:0/631306795 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a8c019020 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.712+0000 7f2a92000640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a7806c940 0x7f2a7806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.712+0000 7f2a92000640 1 -- 172.21.15.59:0/631306795 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f2a8c08b5a0 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.712+0000 7f2a99400640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a7806c940 0x7f2a7806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:20.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.713+0000 7f2a99400640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a7806c940 0x7f2a7806ee00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2a84034000 tx=0x7f2a8402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:20.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.716+0000 7f2a92000640 1 -- 172.21.15.59:0/631306795 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a8c020070 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.911+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a9c162190 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.912+0000 7f2a92000640 1 -- 172.21.15.59:0/631306795 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f2a8c059c90 con 0x7f2a9c06f6f0 2024-04-23T14:41:20.913 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:20.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a7806c940 msgr2=0x7f2a7806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a7806c940 0x7f2a7806ee00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2a84034000 tx=0x7f2a8402c000 comp rx=0 tx=0).stop 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 msgr2=0x7f2a9c1f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c1f4660 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c7c0 tx=0x7f2a8c00cc80 comp rx=0 tx=0).stop 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 shutdown_connections 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2a7806c940 0x7f2a7806ee00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a9c071d70 0x7f2a9c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a9c15e790 0x7f2a9c15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 --2- 172.21.15.59:0/631306795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a9c06f6f0 0x7f2a9c1f4660 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 >> 172.21.15.59:0/631306795 conn(0x7f2a9c067d90 msgr2=0x7f2a9c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 shutdown_connections 2024-04-23T14:41:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:20.915+0000 7f2aa0a00640 1 -- 172.21.15.59:0/631306795 wait complete. 2024-04-23T14:41:20.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-23T14:41:21.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.141+0000 7fd22f800640 1 Processor -- start 2024-04-23T14:41:21.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.141+0000 7fd22f800640 1 -- start start 2024-04-23T14:41:21.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22f800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2281454e0 0x7fd228145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:21.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22f800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd228160910 0x7fd228160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:21.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22f800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2280cd520 con 0x7fd2280cd020 2024-04-23T14:41:21.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22f800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd2280cddc0 con 0x7fd2281454e0 2024-04-23T14:41:21.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22f800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd2280cd800 con 0x7fd228160910 2024-04-23T14:41:21.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2281454e0 0x7fd228145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:21.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22ce00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2281454e0 0x7fd228145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44684/0 (socket says 172.21.15.59:44684) 2024-04-23T14:41:21.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.142+0000 7fd22ce00640 1 -- 172.21.15.59:0/1043397383 learned_addr learned my addr 172.21.15.59:0/1043397383 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:21.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.143+0000 7fd22d800640 1 --2- 172.21.15.59:0/1043397383 >> v2:172.21.15.76:3300/0 conn(0x7fd228160910 0x7fd228160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:21.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.143+0000 7fd22ce00640 1 -- 172.21.15.59:0/1043397383 >> v2:172.21.15.76:3300/0 conn(0x7fd228160910 msgr2=0x7fd228160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:21.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.143+0000 7fd22ce00640 1 --2- 172.21.15.59:0/1043397383 >> v2:172.21.15.76:3300/0 conn(0x7fd228160910 0x7fd228160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:21.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.143+0000 7fd22ce00640 1 -- 172.21.15.59:0/1043397383 >> v1:172.21.15.76:6789/0 conn(0x7fd2280cd020 legacy=0x7fd2280b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:21.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.143+0000 7fd22ce00640 1 -- 172.21.15.59:0/1043397383 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2280cdae0 con 0x7fd2281454e0 2024-04-23T14:41:21.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.143+0000 7fd22d800640 1 --2- 172.21.15.59:0/1043397383 >> v2:172.21.15.76:3300/0 conn(0x7fd228160910 0x7fd228160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:21.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.144+0000 7fd22ce00640 1 --2- 172.21.15.59:0/1043397383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2281454e0 0x7fd228145bb0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fd2280702d0 tx=0x7fd21802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59f268270b9b74e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:21.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.144+0000 7fd227e00640 1 -- 172.21.15.59:0/1043397383 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218037070 con 0x7fd2281454e0 2024-04-23T14:41:21.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.144+0000 7fd227e00640 1 -- 172.21.15.59:0/1043397383 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd218032440 con 0x7fd2281454e0 2024-04-23T14:41:21.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.144+0000 7fd227e00640 1 -- 172.21.15.59:0/1043397383 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218032810 con 0x7fd2281454e0 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 -- 172.21.15.59:0/1043397383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2281454e0 msgr2=0x7fd228145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 --2- 172.21.15.59:0/1043397383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2281454e0 0x7fd228145bb0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fd2280702d0 tx=0x7fd21802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 -- 172.21.15.59:0/1043397383 shutdown_connections 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 --2- 172.21.15.59:0/1043397383 >> v2:172.21.15.76:3300/0 conn(0x7fd228160910 0x7fd228160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 --2- 172.21.15.59:0/1043397383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2281454e0 0x7fd228145bb0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 -- 172.21.15.59:0/1043397383 >> 172.21.15.59:0/1043397383 conn(0x7fd228067ea0 msgr2=0x7fd22806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 -- 172.21.15.59:0/1043397383 shutdown_connections 2024-04-23T14:41:21.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.145+0000 7fd22f800640 1 -- 172.21.15.59:0/1043397383 wait complete. 2024-04-23T14:41:21.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.147+0000 7fd22f800640 1 Processor -- start 2024-04-23T14:41:21.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.147+0000 7fd22f800640 1 -- start start 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22f800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2280cd020 0x7fd22815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22f800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2281454e0 0x7fd22815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22f800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228160910 0x7fd22813f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22f800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2280b4010 con 0x7fd2280cd020 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22f800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281460f0 con 0x7fd228160910 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22f800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd22815f160 con 0x7fd2281454e0 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2280cd020 0x7fd22815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:21.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2280cd020 0x7fd22815bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44692/0 (socket says 172.21.15.59:44692) 2024-04-23T14:41:21.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.148+0000 7fd22ce00640 1 -- 172.21.15.59:0/912873085 learned_addr learned my addr 172.21.15.59:0/912873085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.149+0000 7fd22ce00640 1 -- 172.21.15.59:0/912873085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2281454e0 msgr2=0x7fd22815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.149+0000 7fd22ce00640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2281454e0 0x7fd22815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.149+0000 7fd22d800640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228160910 0x7fd22813f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.149+0000 7fd22ce00640 1 -- 172.21.15.59:0/912873085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228160910 msgr2=0x7fd22813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.149+0000 7fd22ce00640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228160910 0x7fd22813f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.149+0000 7fd22ce00640 1 -- 172.21.15.59:0/912873085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22815f440 con 0x7fd2280cd020 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.149+0000 7fd22ce00640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2280cd020 0x7fd22815bf70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd218000c00 tx=0x7fd2180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.150+0000 7fd225600640 1 -- 172.21.15.59:0/912873085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218041020 con 0x7fd2280cd020 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.150+0000 7fd225600640 1 -- 172.21.15.59:0/912873085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd218030970 con 0x7fd2280cd020 2024-04-23T14:41:21.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.150+0000 7fd225600640 1 -- 172.21.15.59:0/912873085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218037040 con 0x7fd2280cd020 2024-04-23T14:41:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.150+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22815f720 con 0x7fd2280cd020 2024-04-23T14:41:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.150+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22815c4b0 con 0x7fd2280cd020 2024-04-23T14:41:21.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.151+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd228068a80 con 0x7fd2280cd020 2024-04-23T14:41:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.152+0000 7fd225600640 1 -- 172.21.15.59:0/912873085 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd218044020 con 0x7fd2280cd020 2024-04-23T14:41:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.153+0000 7fd225600640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd20406c8f0 0x7fd20406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.153+0000 7fd225600640 1 -- 172.21.15.59:0/912873085 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fd2180ac300 con 0x7fd2280cd020 2024-04-23T14:41:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.153+0000 7fd21fe00640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd20406c8f0 0x7fd20406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.154+0000 7fd21fe00640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd20406c8f0 0x7fd20406edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd2280b2150 tx=0x7fd2100079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.157+0000 7fd225600640 1 -- 172.21.15.59:0/912873085 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd21804b030 con 0x7fd2280cd020 2024-04-23T14:41:21.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:41:21.212 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:41:21.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:21.441+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fd228160030 con 0x7fd2280cd020 2024-04-23T14:41:22.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.041+0000 7fd225600640 1 -- 172.21.15.59:0/912873085 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v472) ==== 149+0+0 (secure 0 0 0) 0x7fd21807aa10 con 0x7fd2280cd020 2024-04-23T14:41:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.045+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd20406c8f0 msgr2=0x7fd20406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.045+0000 7fd22f800640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd20406c8f0 0x7fd20406edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd2280b2150 tx=0x7fd2100079b0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.045+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2280cd020 msgr2=0x7fd22815bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.045+0000 7fd22f800640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2280cd020 0x7fd22815bf70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd218000c00 tx=0x7fd2180046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 shutdown_connections 2024-04-23T14:41:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd20406c8f0 0x7fd20406edb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2281454e0 0x7fd22815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228160910 0x7fd22813f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 --2- 172.21.15.59:0/912873085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2280cd020 0x7fd22815bf70 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 >> 172.21.15.59:0/912873085 conn(0x7fd228067ea0 msgr2=0x7fd2280b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:22.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 shutdown_connections 2024-04-23T14:41:22.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.046+0000 7fd22f800640 1 -- 172.21.15.59:0/912873085 wait complete. 2024-04-23T14:41:22.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:41:22.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.286+0000 7f3e69600640 1 Processor -- start 2024-04-23T14:41:22.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.287+0000 7f3e69600640 1 -- start start 2024-04-23T14:41:22.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.287+0000 7f3e69600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e641567e0 0x7f3e64156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:22.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.287+0000 7f3e69600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 0x7f3e640d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:22.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.287+0000 7f3e69600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e6415cab0 con 0x7f3e640d0b40 2024-04-23T14:41:22.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.287+0000 7f3e69600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f3e640d0f70 con 0x7f3e641567e0 2024-04-23T14:41:22.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.287+0000 7f3e69600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3e64160e40 con 0x7f3e6415d810 2024-04-23T14:41:22.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e63e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e641567e0 0x7f3e64156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:22.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e68c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3e640d0b40 0x7f3e64160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36046/0 (socket says 172.21.15.59:36046) 2024-04-23T14:41:22.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e68c00640 1 -- 172.21.15.59:0/1722847056 learned_addr learned my addr 172.21.15.59:0/1722847056 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e5be00640 1 --2- 172.21.15.59:0/1722847056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 0x7f3e640d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e63e00640 1 -- 172.21.15.59:0/1722847056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 msgr2=0x7f3e640d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e63e00640 1 --2- 172.21.15.59:0/1722847056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 0x7f3e640d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e63e00640 1 -- 172.21.15.59:0/1722847056 >> v1:172.21.15.59:6789/0 conn(0x7f3e640d0b40 legacy=0x7f3e64160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e63400640 1 -- 172.21.15.59:0/1722847056 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3248627001 0 0) 0x7f3e5c001490 con 0x7f3e640d0b40 2024-04-23T14:41:22.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.288+0000 7f3e63e00640 1 -- 172.21.15.59:0/1722847056 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6415cd30 con 0x7f3e641567e0 2024-04-23T14:41:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.289+0000 7f3e63e00640 1 --2- 172.21.15.59:0/1722847056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e641567e0 0x7f3e64156bc0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f3e540097a0 tx=0x7f3e5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21c6571c42210f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e63400640 1 -- 172.21.15.59:0/1722847056 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54037070 con 0x7f3e641567e0 2024-04-23T14:41:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e63400640 1 -- 172.21.15.59:0/1722847056 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e54032440 con 0x7f3e641567e0 2024-04-23T14:41:22.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e63400640 1 -- 172.21.15.59:0/1722847056 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54032810 con 0x7f3e641567e0 2024-04-23T14:41:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e69600640 1 -- 172.21.15.59:0/1722847056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e641567e0 msgr2=0x7f3e64156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e69600640 1 --2- 172.21.15.59:0/1722847056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e641567e0 0x7f3e64156bc0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f3e540097a0 tx=0x7f3e5402a180 comp rx=0 tx=0).stop 2024-04-23T14:41:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e69600640 1 -- 172.21.15.59:0/1722847056 shutdown_connections 2024-04-23T14:41:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e69600640 1 --2- 172.21.15.59:0/1722847056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 0x7f3e640d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e69600640 1 --2- 172.21.15.59:0/1722847056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e641567e0 0x7f3e64156bc0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.290+0000 7f3e69600640 1 -- 172.21.15.59:0/1722847056 >> 172.21.15.59:0/1722847056 conn(0x7f3e6406bdf0 msgr2=0x7f3e6406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:22.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.291+0000 7f3e69600640 1 -- 172.21.15.59:0/1722847056 shutdown_connections 2024-04-23T14:41:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.291+0000 7f3e69600640 1 -- 172.21.15.59:0/1722847056 wait complete. 2024-04-23T14:41:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.292+0000 7f3e69600640 1 Processor -- start 2024-04-23T14:41:22.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.292+0000 7f3e69600640 1 -- start start 2024-04-23T14:41:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.293+0000 7f3e69600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e640d0b40 0x7f3e6406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.293+0000 7f3e69600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e641567e0 0x7f3e640998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.293+0000 7f3e69600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 0x7f3e64158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.293+0000 7f3e69600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e64160820 con 0x7f3e640d0b40 2024-04-23T14:41:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.293+0000 7f3e69600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6415d2f0 con 0x7f3e641567e0 2024-04-23T14:41:22.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.293+0000 7f3e69600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3e641609a0 con 0x7f3e6415d810 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e640d0b40 0x7f3e6406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e640d0b40 0x7f3e6406f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44712/0 (socket says 172.21.15.59:44712) 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 -- 172.21.15.59:0/1138136075 learned_addr learned my addr 172.21.15.59:0/1138136075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 -- 172.21.15.59:0/1138136075 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 msgr2=0x7f3e64158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 0x7f3e64158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 -- 172.21.15.59:0/1138136075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e641567e0 msgr2=0x7f3e640998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e641567e0 0x7f3e640998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 -- 172.21.15.59:0/1138136075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6406e7d0 con 0x7f3e640d0b40 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.294+0000 7f3e63e00640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e640d0b40 0x7f3e6406f870 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3e540098c0 tx=0x7f3e540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.295+0000 7f3e60c00640 1 -- 172.21.15.59:0/1138136075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54037040 con 0x7f3e640d0b40 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.295+0000 7f3e60c00640 1 -- 172.21.15.59:0/1138136075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e54042070 con 0x7f3e640d0b40 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.295+0000 7f3e60c00640 1 -- 172.21.15.59:0/1138136075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54030bb0 con 0x7f3e640d0b40 2024-04-23T14:41:22.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.295+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6406eab0 con 0x7f3e640d0b40 2024-04-23T14:41:22.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.295+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6406f070 con 0x7f3e640d0b40 2024-04-23T14:41:22.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.296+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6404fd90 con 0x7f3e640d0b40 2024-04-23T14:41:22.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.297+0000 7f3e60c00640 1 -- 172.21.15.59:0/1138136075 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e54047460 con 0x7f3e640d0b40 2024-04-23T14:41:22.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.298+0000 7f3e60c00640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4006c8f0 0x7f3e4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:22.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.298+0000 7f3e60c00640 1 -- 172.21.15.59:0/1138136075 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f3e540ac4b0 con 0x7f3e640d0b40 2024-04-23T14:41:22.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.298+0000 7f3e5be00640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4006c8f0 0x7f3e4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:22.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.299+0000 7f3e5be00640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4006c8f0 0x7f3e4006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3e640afaa0 tx=0x7f3e4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:22.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.304+0000 7f3e60c00640 1 -- 172.21.15.59:0/1138136075 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e54032e20 con 0x7f3e640d0b40 2024-04-23T14:41:22.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.514+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3e640736d0 con 0x7f3e640d0b40 2024-04-23T14:41:22.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.522+0000 7f3e60c00640 1 -- 172.21.15.59:0/1138136075 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f3e5407ab20 con 0x7f3e640d0b40 2024-04-23T14:41:22.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4006c8f0 msgr2=0x7f3e4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4006c8f0 0x7f3e4006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3e640afaa0 tx=0x7f3e4c008040 comp rx=0 tx=0).stop 2024-04-23T14:41:22.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e640d0b40 msgr2=0x7f3e6406f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:22.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e640d0b40 0x7f3e6406f870 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3e540098c0 tx=0x7f3e540046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 shutdown_connections 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e4006c8f0 0x7f3e4006edb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e6415d810 0x7f3e64158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e641567e0 0x7f3e640998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 --2- 172.21.15.59:0/1138136075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e640d0b40 0x7f3e6406f870 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 >> 172.21.15.59:0/1138136075 conn(0x7f3e6406bdf0 msgr2=0x7f3e6415f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 shutdown_connections 2024-04-23T14:41:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:22.526+0000 7f3e69600640 1 -- 172.21.15.59:0/1138136075 wait complete. 2024-04-23T14:41:22.547 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:41:22.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:41:22.648 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:41:22.761 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:41:22.761 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:22.762 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:22.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:41:22.821 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:41:22.821 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:22.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:41:22.878 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:22.879 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:22.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:41:22.939 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:41:22.939 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:22.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:41:22.997 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:41:22.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:23.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.215+0000 7f1bce400640 1 Processor -- start 2024-04-23T14:41:23.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.215+0000 7f1bce400640 1 -- start start 2024-04-23T14:41:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.217+0000 7f1bce400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1bc8162df0 0x7f1bc806f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.217+0000 7f1bce400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.217+0000 7f1bce400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc806ed40 con 0x7f1bc80b3f60 2024-04-23T14:41:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.217+0000 7f1bce400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc806eaf0 con 0x7f1bc8162df0 2024-04-23T14:41:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.217+0000 7f1bce400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc80b3590 con 0x7f1bc80cfb90 2024-04-23T14:41:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.218+0000 7f1bbfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.218+0000 7f1bcd000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1bc8162df0 0x7f1bc806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.218+0000 7f1bbfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:35396/0 (socket says 172.21.15.59:35396) 2024-04-23T14:41:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.218+0000 7f1bbfe00640 1 -- 172.21.15.59:0/3434216387 learned_addr learned my addr 172.21.15.59:0/3434216387 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.219+0000 7f1bbfe00640 1 -- 172.21.15.59:0/3434216387 >> v2:172.21.15.76:3300/0 conn(0x7f1bc8162df0 msgr2=0x7f1bc806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.219+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/3434216387 >> v2:172.21.15.76:3300/0 conn(0x7f1bc8162df0 0x7f1bc806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.219+0000 7f1bbfe00640 1 -- 172.21.15.59:0/3434216387 >> v1:172.21.15.76:6789/0 conn(0x7f1bc80b3f60 legacy=0x7f1bc80c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.219+0000 7f1bbfe00640 1 -- 172.21.15.59:0/3434216387 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80b3220 con 0x7f1bc80cfb90 2024-04-23T14:41:23.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.220+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/3434216387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80b3a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009870 tx=0x7f1bb802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59816b6b19623056 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:23.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.220+0000 7f1bc7e00640 1 -- 172.21.15.59:0/3434216387 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8037070 con 0x7f1bc80cfb90 2024-04-23T14:41:23.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bc7e00640 1 -- 172.21.15.59:0/3434216387 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb8032440 con 0x7f1bc80cfb90 2024-04-23T14:41:23.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bc7e00640 1 -- 172.21.15.59:0/3434216387 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8032810 con 0x7f1bc80cfb90 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bce400640 1 -- 172.21.15.59:0/3434216387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 msgr2=0x7f1bc80b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bce400640 1 --2- 172.21.15.59:0/3434216387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80b3a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009870 tx=0x7f1bb802a210 comp rx=0 tx=0).stop 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bce400640 1 -- 172.21.15.59:0/3434216387 shutdown_connections 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bce400640 1 --2- 172.21.15.59:0/3434216387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80b3a20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bce400640 1 --2- 172.21.15.59:0/3434216387 >> v2:172.21.15.76:3300/0 conn(0x7f1bc8162df0 0x7f1bc806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.221+0000 7f1bce400640 1 -- 172.21.15.59:0/3434216387 >> 172.21.15.59:0/3434216387 conn(0x7f1bc80b21a0 msgr2=0x7f1bc80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.222+0000 7f1bce400640 1 -- 172.21.15.59:0/3434216387 shutdown_connections 2024-04-23T14:41:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.222+0000 7f1bce400640 1 -- 172.21.15.59:0/3434216387 wait complete. 2024-04-23T14:41:23.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.223+0000 7f1bce400640 1 Processor -- start 2024-04-23T14:41:23.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.223+0000 7f1bce400640 1 -- start start 2024-04-23T14:41:23.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.224+0000 7f1bce400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc80b3f60 0x7f1bc815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.224+0000 7f1bce400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.224+0000 7f1bce400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc8162df0 0x7f1bc80cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.224+0000 7f1bce400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80c3a30 con 0x7f1bc80b3f60 2024-04-23T14:41:23.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.224+0000 7f1bce400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc80c38b0 con 0x7f1bc8162df0 2024-04-23T14:41:23.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.224+0000 7f1bce400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc80ce200 con 0x7f1bc80cfb90 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc80b3f60 0x7f1bc815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc80b3f60 0x7f1bc815d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44726/0 (socket says 172.21.15.59:44726) 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 -- 172.21.15.59:0/1681007929 learned_addr learned my addr 172.21.15.59:0/1681007929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcda00640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc8162df0 0x7f1bc80cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 -- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 msgr2=0x7f1bc80cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 -- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc8162df0 msgr2=0x7f1bc80cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc8162df0 0x7f1bc80cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.225+0000 7f1bcd000640 1 -- 172.21.15.59:0/1681007929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc80c9660 con 0x7f1bc80b3f60 2024-04-23T14:41:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.226+0000 7f1bcd000640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc80b3f60 0x7f1bc815d3e0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e8a0 tx=0x7f1bb000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.226+0000 7f1bc5600640 1 -- 172.21.15.59:0/1681007929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0017070 con 0x7f1bc80b3f60 2024-04-23T14:41:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.226+0000 7f1bc5600640 1 -- 172.21.15.59:0/1681007929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb000c940 con 0x7f1bc80b3f60 2024-04-23T14:41:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.226+0000 7f1bc5600640 1 -- 172.21.15.59:0/1681007929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000cd30 con 0x7f1bc80b3f60 2024-04-23T14:41:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.226+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc80c98d0 con 0x7f1bc80b3f60 2024-04-23T14:41:23.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.226+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc80c9e90 con 0x7f1bc80b3f60 2024-04-23T14:41:23.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.227+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc804fd90 con 0x7f1bc80b3f60 2024-04-23T14:41:23.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.228+0000 7f1bc5600640 1 -- 172.21.15.59:0/1681007929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bb001a070 con 0x7f1bc80b3f60 2024-04-23T14:41:23.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.230+0000 7f1bc5600640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba406c920 0x7f1ba406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.230+0000 7f1bc5600640 1 -- 172.21.15.59:0/1681007929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7f1bb008b7f0 con 0x7f1bc80b3f60 2024-04-23T14:41:23.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.230+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba406c920 0x7f1ba406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.231+0000 7f1bbfe00640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba406c920 0x7f1ba406ede0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8034000 tx=0x7f1bb802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:23.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.234+0000 7f1bc5600640 1 -- 172.21.15.59:0/1681007929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bb0059dd0 con 0x7f1bc80b3f60 2024-04-23T14:41:23.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.472+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bc80ba330 con 0x7f1bc80b3f60 2024-04-23T14:41:23.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.473+0000 7f1bc5600640 1 -- 172.21.15.59:0/1681007929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3866 (secure 0 0 0) 0x7f1bb00596d0 con 0x7f1bc80b3f60 2024-04-23T14:41:23.473 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:23.474 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2641353301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2641353301},{"type":"v1","addr":"172.21.15.76:6834","nonce":2641353301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/415209459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":415209459},{"type":"v1","addr":"172.21.15.76:6835","nonce":415209459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7530,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/3357306307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3357306307},{"type":"v1","addr":"172.21.15.59:6836","nonce":3357306307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":8433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2971389960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2971389960},{"type":"v1","addr":"172.21.15.59:6837","nonce":2971389960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-04-23T14:41:23.474 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 76 2024-04-23T14:41:23.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba406c920 msgr2=0x7f1ba406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba406c920 0x7f1ba406ede0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8034000 tx=0x7f1bb802e000 comp rx=0 tx=0).stop 2024-04-23T14:41:23.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc80b3f60 msgr2=0x7f1bc815d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc80b3f60 0x7f1bc815d3e0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e8a0 tx=0x7f1bb000ed60 comp rx=0 tx=0).stop 2024-04-23T14:41:23.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 shutdown_connections 2024-04-23T14:41:23.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1ba406c920 0x7f1ba406ede0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bc80cfb90 0x7f1bc80cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bc8162df0 0x7f1bc80cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 --2- 172.21.15.59:0/1681007929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bc80b3f60 0x7f1bc815d3e0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 >> 172.21.15.59:0/1681007929 conn(0x7f1bc80b21a0 msgr2=0x7f1bc80b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:23.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 shutdown_connections 2024-04-23T14:41:23.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.476+0000 7f1bce400640 1 -- 172.21.15.59:0/1681007929 wait complete. 2024-04-23T14:41:23.497 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:41:23.497 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:41:23.497 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:41:23.498 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:41:23.498 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:41:23.498 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:41:23.498 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:41:23.498 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:41:23.498 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:41:23.498 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:41:23.499 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:41:23.499 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:41:23.499 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:41:23.499 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:41:23.499 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:41:23.499 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:41:23.513 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:41:23.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:41:23.515 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:41:23.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:41:23.525 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:41:23.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:41:23.528 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:41:23.528 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:41:23.559 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:41:23.559 DEBUG:teuthology.parallel:result is None 2024-04-23T14:41:23.565 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:41:23.566 DEBUG:teuthology.parallel:result is None 2024-04-23T14:41:23.573 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:41:23.573 DEBUG:teuthology.parallel:result is None 2024-04-23T14:41:23.578 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:41:23.578 DEBUG:teuthology.parallel:result is None 2024-04-23T14:41:23.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:41:23.714 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:41:23.715 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:41:23.724 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:41:23.725 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:41:23.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.760+0000 7fc772c00640 1 Processor -- start 2024-04-23T14:41:23.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.760+0000 7fc772c00640 1 -- start start 2024-04-23T14:41:23.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.761+0000 7fc772c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc76c15ea50 0x7fc76c162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.761+0000 7fc772c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.761+0000 7fc772c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76c0d4e80 con 0x7fc76c0b7410 2024-04-23T14:41:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.761+0000 7fc772c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc76c0cf260 con 0x7fc76c15ea50 2024-04-23T14:41:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.761+0000 7fc772c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c162c20 con 0x7fc76c0d44b0 2024-04-23T14:41:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.761+0000 7fc770e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc76c0b7410 0x7fc76c0b5910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36054/0 (socket says 172.21.15.59:36054) 2024-04-23T14:41:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.761+0000 7fc770e00640 1 -- 172.21.15.59:0/2536373379 learned_addr learned my addr 172.21.15.59:0/2536373379 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc771800640 1 --2- 172.21.15.59:0/2536373379 >> v2:172.21.15.76:3300/0 conn(0x7fc76c15ea50 0x7fc76c162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc772200640 1 --2- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc76be00640 1 -- 172.21.15.59:0/2536373379 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3619167102 0 0) 0x7fc76c0d4e80 con 0x7fc76c0b7410 2024-04-23T14:41:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc76be00640 1 -- 172.21.15.59:0/2536373379 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc754003520 con 0x7fc76c0b7410 2024-04-23T14:41:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc771800640 1 -- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 msgr2=0x7fc76c0d4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc771800640 1 --2- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0d4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc771800640 1 -- 172.21.15.59:0/2536373379 >> v1:172.21.15.59:6789/0 conn(0x7fc76c0b7410 legacy=0x7fc76c0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.762+0000 7fc771800640 1 -- 172.21.15.59:0/2536373379 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c162f00 con 0x7fc76c15ea50 2024-04-23T14:41:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.763+0000 7fc771800640 1 --2- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 0x7fc76c162640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc764009d90 tx=0x7fc76402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f56219f821bae95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc76be00640 1 -- 172.21.15.59:0/2536373379 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764031600 con 0x7fc76c15ea50 2024-04-23T14:41:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc76be00640 1 -- 172.21.15.59:0/2536373379 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7640317a0 con 0x7fc76c15ea50 2024-04-23T14:41:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc76be00640 1 -- 172.21.15.59:0/2536373379 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764031b50 con 0x7fc76c15ea50 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc772c00640 1 -- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 msgr2=0x7fc76c162640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc772c00640 1 --2- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 0x7fc76c162640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc764009d90 tx=0x7fc76402a5d0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc772c00640 1 -- 172.21.15.59:0/2536373379 shutdown_connections 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc772c00640 1 --2- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0d4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc772c00640 1 --2- 172.21.15.59:0/2536373379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 0x7fc76c162640 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.764+0000 7fc772c00640 1 -- 172.21.15.59:0/2536373379 >> 172.21.15.59:0/2536373379 conn(0x7fc76c09d950 msgr2=0x7fc76c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.765+0000 7fc772c00640 1 -- 172.21.15.59:0/2536373379 shutdown_connections 2024-04-23T14:41:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.765+0000 7fc772c00640 1 -- 172.21.15.59:0/2536373379 wait complete. 2024-04-23T14:41:23.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.766+0000 7fc772c00640 1 Processor -- start 2024-04-23T14:41:23.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.766+0000 7fc772c00640 1 -- start start 2024-04-23T14:41:23.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.767+0000 7fc772c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc76c0b7410 0x7fc76c1f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.767+0000 7fc772c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.767+0000 7fc772c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 0x7fc76c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.767+0000 7fc772c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c15ee30 con 0x7fc76c0b7410 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.767+0000 7fc772c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1634c0 con 0x7fc76c15ea50 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.767+0000 7fc772c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c0ca690 con 0x7fc76c0d44b0 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc76c0b7410 0x7fc76c1f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc76c0b7410 0x7fc76c1f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44776/0 (socket says 172.21.15.59:44776) 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 -- 172.21.15.59:0/2256061538 learned_addr learned my addr 172.21.15.59:0/2256061538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc770e00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc772200640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 0x7fc76c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 -- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 msgr2=0x7fc76c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 -- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 msgr2=0x7fc76c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 0x7fc76c0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc771800640 1 -- 172.21.15.59:0/2256061538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c15fe00 con 0x7fc76c0b7410 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.768+0000 7fc770e00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:23.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.769+0000 7fc771800640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc76c0b7410 0x7fc76c1f47f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fc764000f80 tx=0x7fc764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:23.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.769+0000 7fc769600640 1 -- 172.21.15.59:0/2256061538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764009450 con 0x7fc76c0b7410 2024-04-23T14:41:23.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.769+0000 7fc769600640 1 -- 172.21.15.59:0/2256061538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc764031b50 con 0x7fc76c0b7410 2024-04-23T14:41:23.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.769+0000 7fc769600640 1 -- 172.21.15.59:0/2256061538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764003e10 con 0x7fc76c0b7410 2024-04-23T14:41:23.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.769+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1600e0 con 0x7fc76c0b7410 2024-04-23T14:41:23.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.769+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c0ce2b0 con 0x7fc76c0b7410 2024-04-23T14:41:23.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.771+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c15efb0 con 0x7fc76c0b7410 2024-04-23T14:41:23.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.771+0000 7fc769600640 1 -- 172.21.15.59:0/2256061538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc764033070 con 0x7fc76c0b7410 2024-04-23T14:41:23.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.773+0000 7fc769600640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc75006c940 0x7fc75006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:23.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.773+0000 7fc769600640 1 -- 172.21.15.59:0/2256061538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5115+0+0 (secure 0 0 0) 0x7fc7640acd00 con 0x7fc76c0b7410 2024-04-23T14:41:23.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.773+0000 7fc770e00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc75006c940 0x7fc75006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:23.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.774+0000 7fc770e00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc75006c940 0x7fc75006ee00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0cac80 tx=0x7fc75c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:23.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.776+0000 7fc769600640 1 -- 172.21.15.59:0/2256061538 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc76403f030 con 0x7fc76c0b7410 2024-04-23T14:41:23.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:23.994+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc76c04fd90 con 0x7fc76c0b7410 2024-04-23T14:41:25.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.082+0000 7fc769600640 1 -- 172.21.15.59:0/2256061538 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) ==== 97+0+0 (secure 0 0 0) 0x7fc76407b360 con 0x7fc76c0b7410 2024-04-23T14:41:25.082 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.085+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc75006c940 msgr2=0x7fc75006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.085+0000 7fc772c00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc75006c940 0x7fc75006ee00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0cac80 tx=0x7fc75c009040 comp rx=0 tx=0).stop 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.085+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc76c0b7410 msgr2=0x7fc76c1f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.085+0000 7fc772c00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc76c0b7410 0x7fc76c1f47f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fc764000f80 tx=0x7fc764004a40 comp rx=0 tx=0).stop 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 shutdown_connections 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc75006c940 0x7fc75006ee00 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc76c0d44b0 0x7fc76c0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc76c15ea50 0x7fc76c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 --2- 172.21.15.59:0/2256061538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc76c0b7410 0x7fc76c1f47f0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 >> 172.21.15.59:0/2256061538 conn(0x7fc76c09d950 msgr2=0x7fc76c0b9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:25.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 shutdown_connections 2024-04-23T14:41:25.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.086+0000 7fc772c00640 1 -- 172.21.15.59:0/2256061538 wait complete. 2024-04-23T14:41:25.111 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:41:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.323+0000 7f731be00640 1 Processor -- start 2024-04-23T14:41:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.324+0000 7f731be00640 1 -- start start 2024-04-23T14:41:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.325+0000 7f731be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f731c0d09e0 0x7f731c0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.325+0000 7f731be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f731c0b35f0 con 0x7f731c15c9d0 2024-04-23T14:41:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.325+0000 7f731be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f731c0d13b0 con 0x7f731c09b410 2024-04-23T14:41:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.325+0000 7f731be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f731c0cd2d0 con 0x7f731c0d09e0 2024-04-23T14:41:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.326+0000 7f731aa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f731c15c9d0 0x7f731c0b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36070/0 (socket says 172.21.15.59:36070) 2024-04-23T14:41:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.326+0000 7f731aa00640 1 -- 172.21.15.59:0/782778872 learned_addr learned my addr 172.21.15.59:0/782778872 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.326+0000 7f731b400640 1 --2- 172.21.15.59:0/782778872 >> v2:172.21.15.76:3300/0 conn(0x7f731c0d09e0 0x7f731c0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.326+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490860506 0 0) 0x7f731c0b35f0 con 0x7f731c15c9d0 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.326+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7300003520 con 0x7f731c15c9d0 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 666047272 0 0) 0x7f7300003520 con 0x7f731c15c9d0 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 >> v2:172.21.15.76:3300/0 conn(0x7f731c0d09e0 msgr2=0x7f731c0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 --2- 172.21.15.59:0/782778872 >> v2:172.21.15.76:3300/0 conn(0x7f731c0d09e0 0x7f731c0d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 >> v1:172.21.15.76:6789/0 conn(0x7f731c09b410 legacy=0x7f731c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c0b3890 con 0x7f731c15c9d0 2024-04-23T14:41:25.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f7304001670 con 0x7f731c15c9d0 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 >> v1:172.21.15.59:6789/0 conn(0x7f731c15c9d0 legacy=0x7f731c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000039b0 0x7f7300003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7300004800 0x7f7300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73000051c0 0x7f730000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f731c0cd2d0 con 0x7f73000051c0 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f730000c060 con 0x7f7300004800 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f730000c260 con 0x7f73000039b0 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.327+0000 7f731b400640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73000051c0 0x7f730000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.328+0000 7f731a000640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7300004800 0x7f7300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.328+0000 7f731aa00640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000039b0 0x7f7300003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.328+0000 7f731b400640 1 -- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000039b0 msgr2=0x7f7300003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.328+0000 7f731b400640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000039b0 0x7f7300003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.328+0000 7f731b400640 1 -- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7300004800 msgr2=0x7f7300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:25.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.328+0000 7f731b400640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7300004800 0x7f7300004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.328+0000 7f731b400640 1 -- 172.21.15.59:0/782778872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7300003520 con 0x7f73000051c0 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.329+0000 7f731b400640 1 -- 172.21.15.59:0/782778872 reap_dead start 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.329+0000 7f731b400640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73000051c0 0x7f730000b9c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7314009fc0 tx=0x7f7300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1685b80ef00dd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.329+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314035020 con 0x7f73000051c0 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.329+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7314003bd0 con 0x7f73000051c0 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.329+0000 7f7319600640 1 -- 172.21.15.59:0/782778872 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314009c10 con 0x7f73000051c0 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.330+0000 7f731be00640 1 -- 172.21.15.59:0/782778872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73000051c0 msgr2=0x7f730000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.330+0000 7f731be00640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73000051c0 0x7f730000b9c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f7314009fc0 tx=0x7f7300005830 comp rx=0 tx=0).stop 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.330+0000 7f731be00640 1 -- 172.21.15.59:0/782778872 shutdown_connections 2024-04-23T14:41:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.330+0000 7f731be00640 1 --2- 172.21.15.59:0/782778872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73000051c0 0x7f730000b9c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.330+0000 7f731be00640 1 -- 172.21.15.59:0/782778872 >> 172.21.15.59:0/782778872 conn(0x7f731c0b3ef0 msgr2=0x7f731c0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:25.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.330+0000 7f731be00640 1 -- 172.21.15.59:0/782778872 shutdown_connections 2024-04-23T14:41:25.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.330+0000 7f731be00640 1 -- 172.21.15.59:0/782778872 wait complete. 2024-04-23T14:41:25.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.332+0000 7f731be00640 1 Processor -- start 2024-04-23T14:41:25.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.332+0000 7f731be00640 1 -- start start 2024-04-23T14:41:25.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.333+0000 7f731be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000051c0 0x7f731c152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.333+0000 7f731be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0cb4e0 0x7f731c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.333+0000 7f731be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c06cae0 0x7f731c06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.333+0000 7f731be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f731c15cdb0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.333+0000 7f731be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f731c0ccef0 con 0x7f731c06cae0 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.333+0000 7f731be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f731c15cf30 con 0x7f73000051c0 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0cb4e0 0x7f731c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0cb4e0 0x7f731c0cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44786/0 (socket says 172.21.15.59:44786) 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 -- 172.21.15.59:0/4261846184 learned_addr learned my addr 172.21.15.59:0/4261846184 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731aa00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000051c0 0x7f731c152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731b400640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c06cae0 0x7f731c06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 -- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000051c0 msgr2=0x7f731c152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000051c0 0x7f731c152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 -- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c06cae0 msgr2=0x7f731c06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c06cae0 0x7f731c06cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 -- 172.21.15.59:0/4261846184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c06d810 con 0x7f731c0cb4e0 2024-04-23T14:41:25.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.334+0000 7f731a000640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0cb4e0 0x7f731c0cb960 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f730c00e500 tx=0x7f730c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:25.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.335+0000 7f7312a00640 1 -- 172.21.15.59:0/4261846184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c0052e0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.335+0000 7f7312a00640 1 -- 172.21.15.59:0/4261846184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f730c018070 con 0x7f731c0cb4e0 2024-04-23T14:41:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.335+0000 7f7312a00640 1 -- 172.21.15.59:0/4261846184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c003810 con 0x7f731c0cb4e0 2024-04-23T14:41:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.335+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c06db30 con 0x7f731c0cb4e0 2024-04-23T14:41:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.335+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c06e0f0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.336+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731c04fd90 con 0x7f731c0cb4e0 2024-04-23T14:41:25.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.337+0000 7f7312a00640 1 -- 172.21.15.59:0/4261846184 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f730c0039b0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.339+0000 7f7312a00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72f806c920 0x7f72f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.339+0000 7f7312a00640 1 -- 172.21.15.59:0/4261846184 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7f730c08b4c0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.339+0000 7f731aa00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72f806c920 0x7f72f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.340+0000 7f731aa00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72f806c920 0x7f72f806ede0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7304001050 tx=0x7f7304009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.343+0000 7f7312a00640 1 -- 172.21.15.59:0/4261846184 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f730c059bb0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.628+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f731c0d07c0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.629+0000 7f7312a00640 1 -- 172.21.15.59:0/4261846184 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f730c0594b0 con 0x7f731c0cb4e0 2024-04-23T14:41:25.629 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:25.629 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:41:25.629 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-23T14:41:25.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.629 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-23T14:41:25.629 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A==", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA==", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:41:25.630 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw==", 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.631 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg==", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow==", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.632 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ==", 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ==", 2024-04-23T14:41:25.633 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g==", 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.634 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA==", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA==", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.635 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA==", 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ==", 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.636 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ==", 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.637 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q==", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg==", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.638 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg==", 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmEteHAhAAwfUHmvRmEMcPIphwikq6sw==", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA==", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:41:25.640 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ==", 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:41:25.641 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:41:25.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72f806c920 msgr2=0x7f72f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72f806c920 0x7f72f806ede0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7304001050 tx=0x7f7304009040 comp rx=0 tx=0).stop 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0cb4e0 msgr2=0x7f731c0cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0cb4e0 0x7f731c0cb960 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f730c00e500 tx=0x7f730c00e9c0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 shutdown_connections 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f72f806c920 0x7f72f806ede0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c06cae0 0x7f731c06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f731c0cb4e0 0x7f731c0cb960 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 --2- 172.21.15.59:0/4261846184 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73000051c0 0x7f731c152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 >> 172.21.15.59:0/4261846184 conn(0x7f731c0b3ef0 msgr2=0x7f731c0b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 shutdown_connections 2024-04-23T14:41:25.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.632+0000 7f731be00640 1 -- 172.21.15.59:0/4261846184 wait complete. 2024-04-23T14:41:25.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.861+0000 7ff26ae00640 1 Processor -- start 2024-04-23T14:41:25.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.862+0000 7ff26ae00640 1 -- start start 2024-04-23T14:41:25.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.863+0000 7ff26ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2640b5ff0 0x7ff2640b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff26ae00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2640d0d90 0x7ff2640d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff26ae00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2640cf030 con 0x7ff26415e7c0 2024-04-23T14:41:25.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff26ae00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff2640b7000 con 0x7ff2640d0d90 2024-04-23T14:41:25.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff26ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2640cf620 con 0x7ff2640b5ff0 2024-04-23T14:41:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff269000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2640b5ff0 0x7ff2640b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff26a400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2640d0d90 0x7ff2640d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff269000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2640b5ff0 0x7ff2640b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:35484/0 (socket says 172.21.15.59:35484) 2024-04-23T14:41:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff26a400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2640d0d90 0x7ff2640d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:40008/0 (socket says 172.21.15.59:40008) 2024-04-23T14:41:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff269000640 1 -- 172.21.15.59:0/3454133330 learned_addr learned my addr 172.21.15.59:0/3454133330 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:25.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.864+0000 7ff269a00640 1 --1- 172.21.15.59:0/3454133330 >> v1:172.21.15.76:6789/0 conn(0x7ff26415e7c0 0x7ff2640d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:46224/0 (socket says 172.21.15.59:46224) 2024-04-23T14:41:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.865+0000 7ff269000640 1 -- 172.21.15.59:0/3454133330 >> v2:172.21.15.76:3300/0 conn(0x7ff2640d0d90 msgr2=0x7ff2640d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.865+0000 7ff269000640 1 --2- 172.21.15.59:0/3454133330 >> v2:172.21.15.76:3300/0 conn(0x7ff2640d0d90 0x7ff2640d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.865+0000 7ff269000640 1 -- 172.21.15.59:0/3454133330 >> v1:172.21.15.76:6789/0 conn(0x7ff26415e7c0 legacy=0x7ff2640d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.865+0000 7ff269000640 1 -- 172.21.15.59:0/3454133330 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2640b6a40 con 0x7ff2640b5ff0 2024-04-23T14:41:25.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.867+0000 7ff269000640 1 --2- 172.21.15.59:0/3454133330 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2640b5ff0 0x7ff2640b6450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff250009870 tx=0x7ff25002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=455200b400119dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:25.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff263e00640 1 -- 172.21.15.59:0/3454133330 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250037070 con 0x7ff2640b5ff0 2024-04-23T14:41:25.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff263e00640 1 -- 172.21.15.59:0/3454133330 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff250032440 con 0x7ff2640b5ff0 2024-04-23T14:41:25.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff263e00640 1 -- 172.21.15.59:0/3454133330 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250032810 con 0x7ff2640b5ff0 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff26ae00640 1 -- 172.21.15.59:0/3454133330 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2640b5ff0 msgr2=0x7ff2640b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3454133330 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2640b5ff0 0x7ff2640b6450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff250009870 tx=0x7ff25002a270 comp rx=0 tx=0).stop 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff26ae00640 1 -- 172.21.15.59:0/3454133330 shutdown_connections 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3454133330 >> v2:172.21.15.76:3300/0 conn(0x7ff2640d0d90 0x7ff2640d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3454133330 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2640b5ff0 0x7ff2640b6450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff26ae00640 1 -- 172.21.15.59:0/3454133330 >> 172.21.15.59:0/3454133330 conn(0x7ff26409d920 msgr2=0x7ff26408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.868+0000 7ff26ae00640 1 -- 172.21.15.59:0/3454133330 shutdown_connections 2024-04-23T14:41:25.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.869+0000 7ff26ae00640 1 -- 172.21.15.59:0/3454133330 wait complete. 2024-04-23T14:41:25.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.870+0000 7ff26ae00640 1 Processor -- start 2024-04-23T14:41:25.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.870+0000 7ff26ae00640 1 -- start start 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.871+0000 7ff26ae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2640b5ff0 0x7ff264151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.871+0000 7ff26ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2640d0d90 0x7ff2640b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.871+0000 7ff26ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff26415e7c0 0x7ff2640cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.871+0000 7ff26ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2640d2800 con 0x7ff2640d0d90 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.871+0000 7ff26ae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2640cf2b0 con 0x7ff2640b5ff0 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.871+0000 7ff26ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2640d1a80 con 0x7ff26415e7c0 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2640d0d90 0x7ff2640b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2640d0d90 0x7ff2640b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44798/0 (socket says 172.21.15.59:44798) 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 -- 172.21.15.59:0/3890944411 learned_addr learned my addr 172.21.15.59:0/3890944411 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:25.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff26a400640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff26415e7c0 0x7ff2640cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269a00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2640b5ff0 0x7ff264151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 -- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff26415e7c0 msgr2=0x7ff2640cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff26415e7c0 0x7ff2640cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 -- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2640b5ff0 msgr2=0x7ff264151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2640b5ff0 0x7ff264151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.872+0000 7ff269000640 1 -- 172.21.15.59:0/3890944411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2640c9120 con 0x7ff2640d0d90 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.873+0000 7ff269000640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2640d0d90 0x7ff2640b11a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ff250009990 tx=0x7ff2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.873+0000 7ff261600640 1 -- 172.21.15.59:0/3890944411 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250041070 con 0x7ff2640d0d90 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.873+0000 7ff261600640 1 -- 172.21.15.59:0/3890944411 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25003c870 con 0x7ff2640d0d90 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.873+0000 7ff261600640 1 -- 172.21.15.59:0/3890944411 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250037040 con 0x7ff2640d0d90 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.873+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2640c93c0 con 0x7ff2640d0d90 2024-04-23T14:41:25.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.873+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2640c9940 con 0x7ff2640d0d90 2024-04-23T14:41:25.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.875+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26415fbb0 con 0x7ff2640d0d90 2024-04-23T14:41:25.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.876+0000 7ff261600640 1 -- 172.21.15.59:0/3890944411 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff250044070 con 0x7ff2640d0d90 2024-04-23T14:41:25.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.877+0000 7ff261600640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff24406c920 0x7ff24406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:25.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.877+0000 7ff261600640 1 -- 172.21.15.59:0/3890944411 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7ff2500ac400 con 0x7ff2640d0d90 2024-04-23T14:41:25.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.877+0000 7ff269a00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff24406c920 0x7ff24406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:25.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.878+0000 7ff269a00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff24406c920 0x7ff24406ede0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff258000f60 tx=0x7ff258007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:25.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:25.882+0000 7ff261600640 1 -- 172.21.15.59:0/3890944411 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff25007ac00 con 0x7ff2640d0d90 2024-04-23T14:41:26.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.088+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2640d1eb0 con 0x7ff2640d0d90 2024-04-23T14:41:26.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.092+0000 7ff261600640 1 -- 172.21.15.59:0/3890944411 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+12832 (secure 0 0 0) 0x7ff25007a500 con 0x7ff2640d0d90 2024-04-23T14:41:26.092 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:26.092 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":95,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:25.078305+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff24406c920 msgr2=0x7ff24406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff24406c920 0x7ff24406ede0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff258000f60 tx=0x7ff258007ce0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2640d0d90 msgr2=0x7ff2640b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2640d0d90 0x7ff2640b11a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ff250009990 tx=0x7ff2500046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 shutdown_connections 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff24406c920 0x7ff24406ede0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff26415e7c0 0x7ff2640cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2640d0d90 0x7ff2640b11a0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 --2- 172.21.15.59:0/3890944411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2640b5ff0 0x7ff264151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 >> 172.21.15.59:0/3890944411 conn(0x7ff26409d920 msgr2=0x7ff2640b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 shutdown_connections 2024-04-23T14:41:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.095+0000 7ff26ae00640 1 -- 172.21.15.59:0/3890944411 wait complete. 2024-04-23T14:41:26.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:26.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:26.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.328+0000 7fb664a00640 1 Processor -- start 2024-04-23T14:41:26.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.328+0000 7fb664a00640 1 -- start start 2024-04-23T14:41:26.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.329+0000 7fb664a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 0x7fb6600d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.329+0000 7fb664a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6600b5da0 con 0x7fb6600b3600 2024-04-23T14:41:26.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.329+0000 7fb664a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6600d0a20 con 0x7fb6600d13e0 2024-04-23T14:41:26.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.329+0000 7fb664a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb66015f100 con 0x7fb66015ed20 2024-04-23T14:41:26.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.330+0000 7fb65ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb6600b3600 0x7fb6600b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36072/0 (socket says 172.21.15.59:36072) 2024-04-23T14:41:26.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.330+0000 7fb65ea00640 1 -- 172.21.15.59:0/1982726949 learned_addr learned my addr 172.21.15.59:0/1982726949 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:26.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.330+0000 7fb65f400640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 0x7fb6600d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.330+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336330438 0 0) 0x7fb6600b5da0 con 0x7fb6600b3600 2024-04-23T14:41:26.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.330+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb640003520 con 0x7fb6600b3600 2024-04-23T14:41:26.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2489599220 0 0) 0x7fb640003520 con 0x7fb6600b3600 2024-04-23T14:41:26.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 msgr2=0x7fb6600d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 0x7fb6600d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 >> v1:172.21.15.76:6789/0 conn(0x7fb6600d13e0 legacy=0x7fb6600b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6600d2ed0 con 0x7fb6600b3600 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fb6480019f0 con 0x7fb6600b3600 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 >> v1:172.21.15.59:6789/0 conn(0x7fb6600b3600 legacy=0x7fb6600b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6400039e0 0x7fb640003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb640004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 0x7fb640005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb64000bb50 con 0x7fb640004800 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb64000bd00 con 0x7fb6400039e0 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb64000bf00 con 0x7fb66015ed20 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.331+0000 7fb65ea00640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb640004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65f400640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6400039e0 0x7fb640003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65fe00640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 0x7fb640005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65ea00640 1 -- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 msgr2=0x7fb640005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65ea00640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 0x7fb640005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65ea00640 1 -- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6400039e0 msgr2=0x7fb640003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65ea00640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6400039e0 0x7fb640003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65ea00640 1 -- 172.21.15.59:0/1982726949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb640003520 con 0x7fb640004800 2024-04-23T14:41:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65f400640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6400039e0 0x7fb640003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.332+0000 7fb65fe00640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015ed20 0x7fb640005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.333+0000 7fb65ea00640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb640004c80 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb64800a850 tx=0x7fb640005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1572c2906cbc86ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.333+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64800a4a0 con 0x7fb640004800 2024-04-23T14:41:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.333+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64802f070 con 0x7fb640004800 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.333+0000 7fb65e000640 1 -- 172.21.15.59:0/1982726949 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64802bce0 con 0x7fb640004800 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.334+0000 7fb664a00640 1 -- 172.21.15.59:0/1982726949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 msgr2=0x7fb640004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.334+0000 7fb664a00640 1 --2- 172.21.15.59:0/1982726949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb640004c80 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb64800a850 tx=0x7fb640005830 comp rx=0 tx=0).stop 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.334+0000 7fb65fe00640 1 -- 172.21.15.59:0/1982726949 reap_dead start 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.334+0000 7fb664a00640 1 -- 172.21.15.59:0/1982726949 shutdown_connections 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.334+0000 7fb664a00640 1 -- 172.21.15.59:0/1982726949 >> 172.21.15.59:0/1982726949 conn(0x7fb66008bf90 msgr2=0x7fb6600d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.334+0000 7fb664a00640 1 -- 172.21.15.59:0/1982726949 shutdown_connections 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.334+0000 7fb664a00640 1 -- 172.21.15.59:0/1982726949 wait complete. 2024-04-23T14:41:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.336+0000 7fb664a00640 1 Processor -- start 2024-04-23T14:41:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.336+0000 7fb664a00640 1 -- start start 2024-04-23T14:41:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.337+0000 7fb664a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb6600b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.337+0000 7fb664a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015bbf0 0x7fb66015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.337+0000 7fb664a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6600b8ec0 0x7fb6600b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.337+0000 7fb664a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600d0ea0 con 0x7fb640004800 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.337+0000 7fb664a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600d3490 con 0x7fb6600b8ec0 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.337+0000 7fb664a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb6600d1020 con 0x7fb66015bbf0 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb6600b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb6600b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44826/0 (socket says 172.21.15.59:44826) 2024-04-23T14:41:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 -- 172.21.15.59:0/1744318654 learned_addr learned my addr 172.21.15.59:0/1744318654 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65ea00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015bbf0 0x7fb66015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65fe00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6600b8ec0 0x7fb6600b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 -- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015bbf0 msgr2=0x7fb66015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015bbf0 0x7fb66015b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 -- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6600b8ec0 msgr2=0x7fb6600b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6600b8ec0 0x7fb6600b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65f400640 1 -- 172.21.15.59:0/1744318654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6600b7870 con 0x7fb640004800 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.338+0000 7fb65fe00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6600b8ec0 0x7fb6600b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.339+0000 7fb65f400640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb6600b71e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fb6500108b0 tx=0x7fb650010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.339+0000 7fb657400640 1 -- 172.21.15.59:0/1744318654 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65000f040 con 0x7fb640004800 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.339+0000 7fb657400640 1 -- 172.21.15.59:0/1744318654 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65000eb60 con 0x7fb640004800 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.339+0000 7fb657400640 1 -- 172.21.15.59:0/1744318654 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65001c400 con 0x7fb640004800 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.339+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6600b7b50 con 0x7fb640004800 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.339+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6600b8110 con 0x7fb640004800 2024-04-23T14:41:26.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.340+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66004fd90 con 0x7fb640004800 2024-04-23T14:41:26.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.341+0000 7fb657400640 1 -- 172.21.15.59:0/1744318654 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb65001c5a0 con 0x7fb640004800 2024-04-23T14:41:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.342+0000 7fb657400640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63c06c8f0 0x7fb63c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.342+0000 7fb657400640 1 -- 172.21.15.59:0/1744318654 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7fb65008bf90 con 0x7fb640004800 2024-04-23T14:41:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.343+0000 7fb65ea00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63c06c8f0 0x7fb63c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.343+0000 7fb65ea00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63c06c8f0 0x7fb63c06edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb648033000 tx=0x7fb648009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:26.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.347+0000 7fb657400640 1 -- 172.21.15.59:0/1744318654 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb650018030 con 0x7fb640004800 2024-04-23T14:41:26.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.554+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb66015b990 con 0x7fb640004800 2024-04-23T14:41:26.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.555+0000 7fb657400640 1 -- 172.21.15.59:0/1744318654 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fb65005a020 con 0x7fb640004800 2024-04-23T14:41:26.555 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63c06c8f0 msgr2=0x7fb63c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63c06c8f0 0x7fb63c06edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb648033000 tx=0x7fb648009030 comp rx=0 tx=0).stop 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 msgr2=0x7fb6600b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb6600b71e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fb6500108b0 tx=0x7fb650010d70 comp rx=0 tx=0).stop 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 shutdown_connections 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb63c06c8f0 0x7fb63c06edb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb66015bbf0 0x7fb66015b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6600b8ec0 0x7fb6600b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 --2- 172.21.15.59:0/1744318654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb640004800 0x7fb6600b71e0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.558+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 >> 172.21.15.59:0/1744318654 conn(0x7fb66008bf90 msgr2=0x7fb66006e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:26.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.559+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 shutdown_connections 2024-04-23T14:41:26.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.559+0000 7fb664a00640 1 -- 172.21.15.59:0/1744318654 wait complete. 2024-04-23T14:41:26.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:41:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.794+0000 7ff66b200640 1 Processor -- start 2024-04-23T14:41:26.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.794+0000 7ff66b200640 1 -- start start 2024-04-23T14:41:26.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.796+0000 7ff66b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff66c162fb0 0x7ff66c1673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.796+0000 7ff66b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c0c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.796+0000 7ff66b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66c164420 con 0x7ff66c0d1a60 2024-04-23T14:41:26.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.796+0000 7ff66b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff66c163e60 con 0x7ff66c162fb0 2024-04-23T14:41:26.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.796+0000 7ff66b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c164140 con 0x7ff66c0c42b0 2024-04-23T14:41:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.796+0000 7ff66a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c0c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.797+0000 7ff669e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff66c162fb0 0x7ff66c1673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.797+0000 7ff66a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c0c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:35526/0 (socket says 172.21.15.59:35526) 2024-04-23T14:41:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.797+0000 7ff66a800640 1 -- 172.21.15.59:0/1010557450 learned_addr learned my addr 172.21.15.59:0/1010557450 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:26.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.797+0000 7ff66a800640 1 -- 172.21.15.59:0/1010557450 >> v2:172.21.15.76:3300/0 conn(0x7ff66c162fb0 msgr2=0x7ff66c1673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:26.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.797+0000 7ff66a800640 1 --2- 172.21.15.59:0/1010557450 >> v2:172.21.15.76:3300/0 conn(0x7ff66c162fb0 0x7ff66c1673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.797+0000 7ff66a800640 1 -- 172.21.15.59:0/1010557450 >> v1:172.21.15.76:6789/0 conn(0x7ff66c0d1a60 legacy=0x7ff66c163390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.797+0000 7ff66a800640 1 -- 172.21.15.59:0/1010557450 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c166bf0 con 0x7ff66c0c42b0 2024-04-23T14:41:26.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.799+0000 7ff66a800640 1 --2- 172.21.15.59:0/1010557450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c0c4740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff664009870 tx=0x7ff66402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1200471dd8793cca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:26.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.799+0000 7ff668a00640 1 -- 172.21.15.59:0/1010557450 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664037070 con 0x7ff66c0c42b0 2024-04-23T14:41:26.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.799+0000 7ff668a00640 1 -- 172.21.15.59:0/1010557450 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff664032440 con 0x7ff66c0c42b0 2024-04-23T14:41:26.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.799+0000 7ff668a00640 1 -- 172.21.15.59:0/1010557450 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664032810 con 0x7ff66c0c42b0 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 -- 172.21.15.59:0/1010557450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 msgr2=0x7ff66c0c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 --2- 172.21.15.59:0/1010557450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c0c4740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff664009870 tx=0x7ff66402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 -- 172.21.15.59:0/1010557450 shutdown_connections 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 --2- 172.21.15.59:0/1010557450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c0c4740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 --2- 172.21.15.59:0/1010557450 >> v2:172.21.15.76:3300/0 conn(0x7ff66c162fb0 0x7ff66c1673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 -- 172.21.15.59:0/1010557450 >> 172.21.15.59:0/1010557450 conn(0x7ff66c09aa00 msgr2=0x7ff66c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 -- 172.21.15.59:0/1010557450 shutdown_connections 2024-04-23T14:41:26.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.800+0000 7ff66b200640 1 -- 172.21.15.59:0/1010557450 wait complete. 2024-04-23T14:41:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.802+0000 7ff66b200640 1 Processor -- start 2024-04-23T14:41:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.802+0000 7ff66b200640 1 -- start start 2024-04-23T14:41:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66c0d1a60 0x7ff66c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff66c162fb0 0x7ff66c15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c163aa0 con 0x7ff66c162fb0 2024-04-23T14:41:26.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c167930 con 0x7ff66c0d1a60 2024-04-23T14:41:26.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c163c20 con 0x7ff66c0c42b0 2024-04-23T14:41:26.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff66c162fb0 0x7ff66c15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff66c162fb0 0x7ff66c15c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44836/0 (socket says 172.21.15.59:44836) 2024-04-23T14:41:26.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.803+0000 7ff66a800640 1 -- 172.21.15.59:0/102623866 learned_addr learned my addr 172.21.15.59:0/102623866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:26.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff669e00640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff66a800640 1 -- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 msgr2=0x7ff66c070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff66a800640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff669400640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66c0d1a60 0x7ff66c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff66a800640 1 -- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66c0d1a60 msgr2=0x7ff66c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff66a800640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66c0d1a60 0x7ff66c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff66a800640 1 -- 172.21.15.59:0/102623866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c161b20 con 0x7ff66c162fb0 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff669e00640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff669400640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66c0d1a60 0x7ff66c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.804+0000 7ff66a800640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff66c162fb0 0x7ff66c15c650 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff66402a220 tx=0x7ff6640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.805+0000 7ff662000640 1 -- 172.21.15.59:0/102623866 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664041020 con 0x7ff66c162fb0 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.805+0000 7ff662000640 1 -- 172.21.15.59:0/102623866 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66403d640 con 0x7ff66c162fb0 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.805+0000 7ff662000640 1 -- 172.21.15.59:0/102623866 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664037040 con 0x7ff66c162fb0 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.805+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c161e00 con 0x7ff66c162fb0 2024-04-23T14:41:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.805+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1623c0 con 0x7ff66c162fb0 2024-04-23T14:41:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.806+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c04fd90 con 0x7ff66c162fb0 2024-04-23T14:41:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.807+0000 7ff662000640 1 -- 172.21.15.59:0/102623866 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff664044020 con 0x7ff66c162fb0 2024-04-23T14:41:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.808+0000 7ff662000640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64806c8f0 0x7ff64806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.808+0000 7ff662000640 1 -- 172.21.15.59:0/102623866 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7ff6640abfe0 con 0x7ff66c162fb0 2024-04-23T14:41:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.809+0000 7ff669e00640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64806c8f0 0x7ff64806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.809+0000 7ff669e00640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64806c8f0 0x7ff64806edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff65c007520 tx=0x7ff65c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:26.812+0000 7ff662000640 1 -- 172.21.15.59:0/102623866 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff66404b030 con 0x7ff66c162fb0 2024-04-23T14:41:27.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.017+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff66c0b7260 con 0x7ff66c162fb0 2024-04-23T14:41:27.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.018+0000 7ff662000640 1 -- 172.21.15.59:0/102623866 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) ==== 84+0+65 (secure 0 0 0) 0x7ff66407a6d0 con 0x7ff66c162fb0 2024-04-23T14:41:27.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:27.019 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:41:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:41:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:41:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:41:27.019 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:27.019 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64806c8f0 msgr2=0x7ff64806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:27.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64806c8f0 0x7ff64806edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff65c007520 tx=0x7ff65c00b040 comp rx=0 tx=0).stop 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff66c162fb0 msgr2=0x7ff66c15c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff66c162fb0 0x7ff66c15c650 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff66402a220 tx=0x7ff6640046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 shutdown_connections 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff64806c8f0 0x7ff64806edb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff66c162fb0 0x7ff66c15c650 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66c0d1a60 0x7ff66c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 --2- 172.21.15.59:0/102623866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff66c0c42b0 0x7ff66c070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 >> 172.21.15.59:0/102623866 conn(0x7ff66c09aa00 msgr2=0x7ff66c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 shutdown_connections 2024-04-23T14:41:27.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.021+0000 7ff66b200640 1 -- 172.21.15.59:0/102623866 wait complete. 2024-04-23T14:41:27.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:41:27.041 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:41:27.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.237+0000 7facd9400640 1 Processor -- start 2024-04-23T14:41:27.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.237+0000 7facd9400640 1 -- start start 2024-04-23T14:41:27.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.238+0000 7facd9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7facd415e990 0x7facd40d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:27.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.238+0000 7facd9400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 0x7facd40d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:27.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd9400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd40cfc70 con 0x7facd40d4d00 2024-04-23T14:41:27.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd9400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7facd40d0230 con 0x7facd415e990 2024-04-23T14:41:27.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd9400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7facd40d21c0 con 0x7facd40cec40 2024-04-23T14:41:27.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7facd415e990 0x7facd40d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7facd415e990 0x7facd40d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44840/0 (socket says 172.21.15.59:44840) 2024-04-23T14:41:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd3e00640 1 -- 172.21.15.59:0/2211082070 learned_addr learned my addr 172.21.15.59:0/2211082070 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd3e00640 1 -- 172.21.15.59:0/2211082070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 msgr2=0x7facd40d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd3e00640 1 --2- 172.21.15.59:0/2211082070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 0x7facd40d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd3e00640 1 -- 172.21.15.59:0/2211082070 >> v1:172.21.15.76:6789/0 conn(0x7facd40d4d00 legacy=0x7facd406cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:27.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.239+0000 7facd3e00640 1 -- 172.21.15.59:0/2211082070 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd40cff50 con 0x7facd415e990 2024-04-23T14:41:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.240+0000 7facd3e00640 1 --2- 172.21.15.59:0/2211082070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd415e990 0x7facd40d1000 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7facd40d2340 tx=0x7facc402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52986aeb5ec6c5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd2a00640 1 -- 172.21.15.59:0/2211082070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4037070 con 0x7facd415e990 2024-04-23T14:41:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd2a00640 1 -- 172.21.15.59:0/2211082070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc4032440 con 0x7facd415e990 2024-04-23T14:41:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd2a00640 1 -- 172.21.15.59:0/2211082070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4032810 con 0x7facd415e990 2024-04-23T14:41:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 -- 172.21.15.59:0/2211082070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd415e990 msgr2=0x7facd40d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:27.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 --2- 172.21.15.59:0/2211082070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd415e990 0x7facd40d1000 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7facd40d2340 tx=0x7facc402a1f0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 -- 172.21.15.59:0/2211082070 shutdown_connections 2024-04-23T14:41:27.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 --2- 172.21.15.59:0/2211082070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 0x7facd40d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 --2- 172.21.15.59:0/2211082070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd415e990 0x7facd40d1000 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 -- 172.21.15.59:0/2211082070 >> 172.21.15.59:0/2211082070 conn(0x7facd408bf90 msgr2=0x7facd40d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:27.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 -- 172.21.15.59:0/2211082070 shutdown_connections 2024-04-23T14:41:27.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.241+0000 7facd9400640 1 -- 172.21.15.59:0/2211082070 wait complete. 2024-04-23T14:41:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.243+0000 7facd9400640 1 Processor -- start 2024-04-23T14:41:27.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.243+0000 7facd9400640 1 -- start start 2024-04-23T14:41:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.244+0000 7facd9400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 0x7facd41f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.244+0000 7facd9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd40d4d00 0x7facd41f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.244+0000 7facd9400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd415e990 0x7facd41f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.244+0000 7facd9400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7facd406e1f0 con 0x7facd40d4d00 2024-04-23T14:41:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.244+0000 7facd9400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7facd40d26f0 con 0x7facd415e990 2024-04-23T14:41:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.244+0000 7facd9400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7facd4160d80 con 0x7facd40cec40 2024-04-23T14:41:27.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.244+0000 7facd3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd40d4d00 0x7facd41f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd40d4d00 0x7facd41f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44850/0 (socket says 172.21.15.59:44850) 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 -- 172.21.15.59:0/2890422346 learned_addr learned my addr 172.21.15.59:0/2890422346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3e00640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 0x7facd41f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd8a00640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd415e990 0x7facd41f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 -- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 msgr2=0x7facd41f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 0x7facd41f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 -- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd415e990 msgr2=0x7facd41f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd415e990 0x7facd41f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:27.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 -- 172.21.15.59:0/2890422346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd41f4d10 con 0x7facd40d4d00 2024-04-23T14:41:27.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.245+0000 7facd3400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd40d4d00 0x7facd41f4440 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b4a0 tx=0x7facbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:27.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.246+0000 7faccbe00640 1 -- 172.21.15.59:0/2890422346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc014070 con 0x7facd40d4d00 2024-04-23T14:41:27.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.246+0000 7faccbe00640 1 -- 172.21.15.59:0/2890422346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc005a10 con 0x7facd40d4d00 2024-04-23T14:41:27.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.246+0000 7faccbe00640 1 -- 172.21.15.59:0/2890422346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc005e00 con 0x7facd40d4d00 2024-04-23T14:41:27.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.246+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41f4ff0 con 0x7facd40d4d00 2024-04-23T14:41:27.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.246+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd41f4980 con 0x7facd40d4d00 2024-04-23T14:41:27.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.247+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd404fd90 con 0x7facd40d4d00 2024-04-23T14:41:27.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.248+0000 7faccbe00640 1 -- 172.21.15.59:0/2890422346 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facbc010610 con 0x7facd40d4d00 2024-04-23T14:41:27.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.249+0000 7faccbe00640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7facb006c8f0 0x7facb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:27.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.249+0000 7faccbe00640 1 -- 172.21.15.59:0/2890422346 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7facbc08b220 con 0x7facd40d4d00 2024-04-23T14:41:27.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.249+0000 7facd3e00640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7facb006c8f0 0x7facb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:27.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.250+0000 7facd3e00640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7facb006c8f0 0x7facb006edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7facc4034000 tx=0x7facc402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:27.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.254+0000 7faccbe00640 1 -- 172.21.15.59:0/2890422346 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7facbc059990 con 0x7facd40d4d00 2024-04-23T14:41:27.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:27.449+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7facd40b2100 con 0x7facd40d4d00 2024-04-23T14:41:28.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.111+0000 7faccbe00640 1 -- 172.21.15.59:0/2890422346 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) ==== 138+0+0 (secure 0 0 0) 0x7facbc059290 con 0x7facd40d4d00 2024-04-23T14:41:28.112 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:41:28.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7facb006c8f0 msgr2=0x7facb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:28.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7facb006c8f0 0x7facb006edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7facc4034000 tx=0x7facc402e000 comp rx=0 tx=0).stop 2024-04-23T14:41:28.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd40d4d00 msgr2=0x7facd41f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:28.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd40d4d00 0x7facd41f4440 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b4a0 tx=0x7facbc00b960 comp rx=0 tx=0).stop 2024-04-23T14:41:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 shutdown_connections 2024-04-23T14:41:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7facb006c8f0 0x7facb006edb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd415e990 0x7facd41f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7facd40d4d00 0x7facd41f4440 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 --2- 172.21.15.59:0/2890422346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7facd40cec40 0x7facd41f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 >> 172.21.15.59:0/2890422346 conn(0x7facd408bf90 msgr2=0x7facd40b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:28.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.115+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 shutdown_connections 2024-04-23T14:41:28.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.116+0000 7facd9400640 1 -- 172.21.15.59:0/2890422346 wait complete. 2024-04-23T14:41:28.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:41:28.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.348+0000 7fa4fa000640 1 Processor -- start 2024-04-23T14:41:28.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.349+0000 7fa4fa000640 1 -- start start 2024-04-23T14:41:28.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.350+0000 7fa4fa000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4f40b5ff0 0x7fa4f40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:28.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.350+0000 7fa4fa000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4f40d0d90 0x7fa4f40d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:28.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.350+0000 7fa4fa000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40cf030 con 0x7fa4f415e7c0 2024-04-23T14:41:28.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.350+0000 7fa4fa000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f40b7000 con 0x7fa4f40d0d90 2024-04-23T14:41:28.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.350+0000 7fa4fa000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f40d1720 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.350+0000 7fa4f9600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa4f40d0d90 0x7fa4f40d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:28.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.351+0000 7fa4f8c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa4f415e7c0 0x7fa4f40d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36074/0 (socket says 172.21.15.59:36074) 2024-04-23T14:41:28.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.351+0000 7fa4f8c00640 1 -- 172.21.15.59:0/3440501482 learned_addr learned my addr 172.21.15.59:0/3440501482 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:28.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.351+0000 7fa4f3e00640 1 --2- 172.21.15.59:0/3440501482 >> v2:172.21.15.76:3300/0 conn(0x7fa4f40b5ff0 0x7fa4f40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:28.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.351+0000 7fa4f9600640 1 -- 172.21.15.59:0/3440501482 >> v2:172.21.15.76:3300/0 conn(0x7fa4f40b5ff0 msgr2=0x7fa4f40b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:28.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.351+0000 7fa4f9600640 1 --2- 172.21.15.59:0/3440501482 >> v2:172.21.15.76:3300/0 conn(0x7fa4f40b5ff0 0x7fa4f40b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.351+0000 7fa4f9600640 1 -- 172.21.15.59:0/3440501482 >> v1:172.21.15.59:6789/0 conn(0x7fa4f415e7c0 legacy=0x7fa4f40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:28.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.351+0000 7fa4f9600640 1 -- 172.21.15.59:0/3440501482 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f40b6a40 con 0x7fa4f40d0d90 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.352+0000 7fa4f9600640 1 --2- 172.21.15.59:0/3440501482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40d0d90 0x7fa4f40d11e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40d55c0 tx=0x7fa4ec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a4398652ff0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.352+0000 7fa4f3400640 1 -- 172.21.15.59:0/3440501482 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec007740 con 0x7fa4f40d0d90 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.352+0000 7fa4f3400640 1 -- 172.21.15.59:0/3440501482 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0078e0 con 0x7fa4f40d0d90 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.352+0000 7fa4f3400640 1 -- 172.21.15.59:0/3440501482 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec007cb0 con 0x7fa4f40d0d90 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 -- 172.21.15.59:0/3440501482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40d0d90 msgr2=0x7fa4f40d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 --2- 172.21.15.59:0/3440501482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40d0d90 0x7fa4f40d11e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40d55c0 tx=0x7fa4ec02a680 comp rx=0 tx=0).stop 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 -- 172.21.15.59:0/3440501482 shutdown_connections 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 --2- 172.21.15.59:0/3440501482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40d0d90 0x7fa4f40d11e0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 --2- 172.21.15.59:0/3440501482 >> v2:172.21.15.76:3300/0 conn(0x7fa4f40b5ff0 0x7fa4f40b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 -- 172.21.15.59:0/3440501482 >> 172.21.15.59:0/3440501482 conn(0x7fa4f409d920 msgr2=0x7fa4f408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:28.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 -- 172.21.15.59:0/3440501482 shutdown_connections 2024-04-23T14:41:28.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.353+0000 7fa4fa000640 1 -- 172.21.15.59:0/3440501482 wait complete. 2024-04-23T14:41:28.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.355+0000 7fa4fa000640 1 Processor -- start 2024-04-23T14:41:28.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.355+0000 7fa4fa000640 1 -- start start 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4fa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40b5ff0 0x7fa4f414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4fa000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4f40d0d90 0x7fa4f40d4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4fa000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4f415e7c0 0x7fa4f40d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4fa000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40cf2b0 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4fa000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40cf620 con 0x7fa4f415e7c0 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4fa000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f40d2800 con 0x7fa4f40d0d90 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4f8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40b5ff0 0x7fa4f414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4f8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40b5ff0 0x7fa4f414acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44876/0 (socket says 172.21.15.59:44876) 2024-04-23T14:41:28.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.356+0000 7fa4f8c00640 1 -- 172.21.15.59:0/1321724972 learned_addr learned my addr 172.21.15.59:0/1321724972 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f9600640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4f415e7c0 0x7fa4f40d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f3e00640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4f40d0d90 0x7fa4f40d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f8c00640 1 -- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4f40d0d90 msgr2=0x7fa4f40d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f8c00640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4f40d0d90 0x7fa4f40d4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f8c00640 1 -- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4f415e7c0 msgr2=0x7fa4f40d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f8c00640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4f415e7c0 0x7fa4f40d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f8c00640 1 -- 172.21.15.59:0/1321724972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f414b210 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f9600640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4f415e7c0 0x7fa4f40d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.357+0000 7fa4f8c00640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40b5ff0 0x7fa4f414acd0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c3f0 tx=0x7fa4e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.358+0000 7fa4f0c00640 1 -- 172.21.15.59:0/1321724972 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8017070 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.358+0000 7fa4f0c00640 1 -- 172.21.15.59:0/1321724972 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e8005ce0 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.358+0000 7fa4f0c00640 1 -- 172.21.15.59:0/1321724972 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e80072f0 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.358+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f40b8d80 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.358+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f40b9340 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.359+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f404fd90 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.361+0000 7fa4f0c00640 1 -- 172.21.15.59:0/1321724972 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4e801a070 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.362+0000 7fa4f0c00640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4d406c920 0x7fa4d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:28.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.362+0000 7fa4f0c00640 1 -- 172.21.15.59:0/1321724972 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5227+0+0 (secure 0 0 0) 0x7fa4e808b190 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.362+0000 7fa4f3e00640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4d406c920 0x7fa4d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:28.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.363+0000 7fa4f3e00640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4d406c920 0x7fa4d406ede0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40caf50 tx=0x7fa4e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:28.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.366+0000 7fa4f0c00640 1 -- 172.21.15.59:0/1321724972 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4e8021020 con 0x7fa4f40b5ff0 2024-04-23T14:41:28.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:28.569+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa4f40b5b00 con 0x7fa4f40b5ff0 2024-04-23T14:41:29.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.128+0000 7fa4f0c00640 1 -- 172.21.15.59:0/1321724972 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) ==== 170+0+0 (secure 0 0 0) 0x7fa4f40cf030 con 0x7fa4f40b5ff0 2024-04-23T14:41:29.128 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:41:29.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4d406c920 msgr2=0x7fa4d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:29.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4d406c920 0x7fa4d406ede0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40caf50 tx=0x7fa4e0008040 comp rx=0 tx=0).stop 2024-04-23T14:41:29.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40b5ff0 msgr2=0x7fa4f414acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:29.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40b5ff0 0x7fa4f414acd0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c3f0 tx=0x7fa4e800c8b0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 shutdown_connections 2024-04-23T14:41:29.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa4d406c920 0x7fa4d406ede0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa4f40d0d90 0x7fa4f40d4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4f415e7c0 0x7fa4f40d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 --2- 172.21.15.59:0/1321724972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f40b5ff0 0x7fa4f414acd0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.131+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 >> 172.21.15.59:0/1321724972 conn(0x7fa4f409d920 msgr2=0x7fa4f40b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:29.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.132+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 shutdown_connections 2024-04-23T14:41:29.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.132+0000 7fa4fa000640 1 -- 172.21.15.59:0/1321724972 wait complete. 2024-04-23T14:41:29.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:41:29.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.350+0000 7f63e3200640 1 Processor -- start 2024-04-23T14:41:29.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.351+0000 7f63e3200640 1 -- start start 2024-04-23T14:41:29.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.352+0000 7f63e3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63e41567e0 0x7f63e4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.352+0000 7f63e3200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63e40d0d10 0x7f63e4160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.352+0000 7f63e3200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e415cab0 con 0x7f63e415d810 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.352+0000 7f63e3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f63e4160e40 con 0x7f63e41567e0 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.352+0000 7f63e3200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f63e415cd30 con 0x7f63e40d0d10 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.352+0000 7f63e1e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63e41567e0 0x7f63e4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.352+0000 7f63e1e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63e41567e0 0x7f63e4156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44878/0 (socket says 172.21.15.59:44878) 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.353+0000 7f63e1e00640 1 -- 172.21.15.59:0/768797739 learned_addr learned my addr 172.21.15.59:0/768797739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:29.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.353+0000 7f63e2800640 1 --2- 172.21.15.59:0/768797739 >> v2:172.21.15.76:3300/0 conn(0x7f63e40d0d10 0x7f63e4160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:29.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.353+0000 7f63e1e00640 1 -- 172.21.15.59:0/768797739 >> v2:172.21.15.76:3300/0 conn(0x7f63e40d0d10 msgr2=0x7f63e4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:29.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.353+0000 7f63e1e00640 1 --2- 172.21.15.59:0/768797739 >> v2:172.21.15.76:3300/0 conn(0x7f63e40d0d10 0x7f63e4160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.353+0000 7f63e1e00640 1 -- 172.21.15.59:0/768797739 >> v1:172.21.15.76:6789/0 conn(0x7f63e415d810 legacy=0x7f63e40d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:29.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.353+0000 7f63e1e00640 1 -- 172.21.15.59:0/768797739 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e415d010 con 0x7f63e41567e0 2024-04-23T14:41:29.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.355+0000 7f63e1e00640 1 --2- 172.21.15.59:0/768797739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e41567e0 0x7f63e4156bc0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f63e40b4cb0 tx=0x7f63d402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b1e9f17a27909c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:29.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.355+0000 7f63e1400640 1 -- 172.21.15.59:0/768797739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4036070 con 0x7f63e41567e0 2024-04-23T14:41:29.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.355+0000 7f63e1400640 1 -- 172.21.15.59:0/768797739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d4031600 con 0x7f63e41567e0 2024-04-23T14:41:29.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.355+0000 7f63e1400640 1 -- 172.21.15.59:0/768797739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d40319d0 con 0x7f63e41567e0 2024-04-23T14:41:29.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 -- 172.21.15.59:0/768797739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e41567e0 msgr2=0x7f63e4156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:29.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 --2- 172.21.15.59:0/768797739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e41567e0 0x7f63e4156bc0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f63e40b4cb0 tx=0x7f63d402a730 comp rx=0 tx=0).stop 2024-04-23T14:41:29.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 -- 172.21.15.59:0/768797739 shutdown_connections 2024-04-23T14:41:29.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 --2- 172.21.15.59:0/768797739 >> v2:172.21.15.76:3300/0 conn(0x7f63e40d0d10 0x7f63e4160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 --2- 172.21.15.59:0/768797739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e41567e0 0x7f63e4156bc0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 -- 172.21.15.59:0/768797739 >> 172.21.15.59:0/768797739 conn(0x7f63e406bdf0 msgr2=0x7f63e406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:29.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 -- 172.21.15.59:0/768797739 shutdown_connections 2024-04-23T14:41:29.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.356+0000 7f63e3200640 1 -- 172.21.15.59:0/768797739 wait complete. 2024-04-23T14:41:29.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.358+0000 7f63e3200640 1 Processor -- start 2024-04-23T14:41:29.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.358+0000 7f63e3200640 1 -- start start 2024-04-23T14:41:29.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.359+0000 7f63e3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e40d0d10 0x7f63e4099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:29.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.359+0000 7f63e3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63e41567e0 0x7f63e4159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:29.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.359+0000 7f63e3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e415d810 0x7f63e4159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.359+0000 7f63e3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41607d0 con 0x7f63e40d0d10 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.359+0000 7f63e3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4160650 con 0x7f63e415d810 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.359+0000 7f63e3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f63e4160950 con 0x7f63e41567e0 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e40d0d10 0x7f63e4099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e40d0d10 0x7f63e4099820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44890/0 (socket says 172.21.15.59:44890) 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 -- 172.21.15.59:0/2594331045 learned_addr learned my addr 172.21.15.59:0/2594331045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63d9400640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63e41567e0 0x7f63e4159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e2800640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e415d810 0x7f63e4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 -- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63e41567e0 msgr2=0x7f63e4159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63e41567e0 0x7f63e4159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 -- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e415d810 msgr2=0x7f63e4159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e415d810 0x7f63e4159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:29.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.360+0000 7f63e1e00640 1 -- 172.21.15.59:0/2594331045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e4155570 con 0x7f63e40d0d10 2024-04-23T14:41:29.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.361+0000 7f63e1e00640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e40d0d10 0x7f63e4099820 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f63d4034040 tx=0x7f63d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.361+0000 7f63daa00640 1 -- 172.21.15.59:0/2594331045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4036050 con 0x7f63e40d0d10 2024-04-23T14:41:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.361+0000 7f63daa00640 1 -- 172.21.15.59:0/2594331045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d4042070 con 0x7f63e40d0d10 2024-04-23T14:41:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.361+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e4155850 con 0x7f63e40d0d10 2024-04-23T14:41:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.361+0000 7f63daa00640 1 -- 172.21.15.59:0/2594331045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4009510 con 0x7f63e40d0d10 2024-04-23T14:41:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.361+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e414fdd0 con 0x7f63e40d0d10 2024-04-23T14:41:29.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.362+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e404fd90 con 0x7f63e40d0d10 2024-04-23T14:41:29.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.364+0000 7f63daa00640 1 -- 172.21.15.59:0/2594331045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63d4007c70 con 0x7f63e40d0d10 2024-04-23T14:41:29.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.365+0000 7f63daa00640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63c006c8f0 0x7f63c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:29.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.365+0000 7f63daa00640 1 -- 172.21.15.59:0/2594331045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5609+0+0 (secure 0 0 0) 0x7f63d40ad380 con 0x7f63e40d0d10 2024-04-23T14:41:29.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.365+0000 7f63d9400640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63c006c8f0 0x7f63c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:29.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.366+0000 7f63d9400640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63c006c8f0 0x7f63c006edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f63e41f7f50 tx=0x7f63cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:29.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.369+0000 7f63daa00640 1 -- 172.21.15.59:0/2594331045 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63d407b910 con 0x7f63e40d0d10 2024-04-23T14:41:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:29.611+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f63e40b5150 con 0x7f63e40d0d10 2024-04-23T14:41:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.159+0000 7f63daa00640 1 -- 172.21.15.59:0/2594331045 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v78) ==== 174+0+0 (secure 0 0 0) 0x7f63d407b210 con 0x7f63e40d0d10 2024-04-23T14:41:30.163 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 14 and data pool 15 2024-04-23T14:41:30.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63c006c8f0 msgr2=0x7f63c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63c006c8f0 0x7f63c006edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f63e41f7f50 tx=0x7f63cc009040 comp rx=0 tx=0).stop 2024-04-23T14:41:30.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e40d0d10 msgr2=0x7f63e4099820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e40d0d10 0x7f63e4099820 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f63d4034040 tx=0x7f63d4004a40 comp rx=0 tx=0).stop 2024-04-23T14:41:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 shutdown_connections 2024-04-23T14:41:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f63c006c8f0 0x7f63c006edb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63e41567e0 0x7f63e4159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e415d810 0x7f63e4159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 --2- 172.21.15.59:0/2594331045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63e40d0d10 0x7f63e4099820 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.173+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 >> 172.21.15.59:0/2594331045 conn(0x7f63e406bdf0 msgr2=0x7f63e415b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:30.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.174+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 shutdown_connections 2024-04-23T14:41:30.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.174+0000 7f63e3200640 1 -- 172.21.15.59:0/2594331045 wait complete. 2024-04-23T14:41:30.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:30.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.375+0000 7f8bfd800640 1 Processor -- start 2024-04-23T14:41:30.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.376+0000 7f8bfd800640 1 -- start start 2024-04-23T14:41:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.377+0000 7f8bfd800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8bf80afaf0 0x7f8bf80d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.377+0000 7f8bfd800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf815f0f0 con 0x7f8bf80d06e0 2024-04-23T14:41:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.377+0000 7f8bfd800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf815f3d0 con 0x7f8bf815c5e0 2024-04-23T14:41:30.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.377+0000 7f8bfd800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8bf815fa20 con 0x7f8bf80afaf0 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8bf80afaf0 0x7f8bf80d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf6000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8bf80afaf0 0x7f8bf80d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59580/0 (socket says 172.21.15.59:59580) 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf6000640 1 -- 172.21.15.59:0/675322282 learned_addr learned my addr 172.21.15.59:0/675322282 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8bf80d06e0 0x7f8bf81624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55598/0 (socket says 172.21.15.59:55598) 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf6000640 1 -- 172.21.15.59:0/675322282 >> v1:172.21.15.76:6789/0 conn(0x7f8bf815c5e0 legacy=0x7f8bf80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf6000640 1 -- 172.21.15.59:0/675322282 >> v1:172.21.15.59:6789/0 conn(0x7f8bf80d06e0 legacy=0x7f8bf81624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf5600640 1 -- 172.21.15.59:0/675322282 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555758145 0 0) 0x7f8bf815f0f0 con 0x7f8bf80d06e0 2024-04-23T14:41:30.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.378+0000 7f8bf6000640 1 -- 172.21.15.59:0/675322282 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf815f6b0 con 0x7f8bf80afaf0 2024-04-23T14:41:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.379+0000 7f8bf6000640 1 --2- 172.21.15.59:0/675322282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80afaf0 0x7f8bf80d01a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009870 tx=0x7f8be002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9031b32d9d10731a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bf5600640 1 -- 172.21.15.59:0/675322282 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0037070 con 0x7f8bf80afaf0 2024-04-23T14:41:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bf5600640 1 -- 172.21.15.59:0/675322282 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be0032440 con 0x7f8bf80afaf0 2024-04-23T14:41:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bf5600640 1 -- 172.21.15.59:0/675322282 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0032810 con 0x7f8bf80afaf0 2024-04-23T14:41:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bfd800640 1 -- 172.21.15.59:0/675322282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80afaf0 msgr2=0x7f8bf80d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bfd800640 1 --2- 172.21.15.59:0/675322282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80afaf0 0x7f8bf80d01a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009870 tx=0x7f8be002a2a0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bfd800640 1 -- 172.21.15.59:0/675322282 shutdown_connections 2024-04-23T14:41:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bfd800640 1 --2- 172.21.15.59:0/675322282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80afaf0 0x7f8bf80d01a0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bfd800640 1 -- 172.21.15.59:0/675322282 >> 172.21.15.59:0/675322282 conn(0x7f8bf8067ea0 msgr2=0x7f8bf806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bfd800640 1 -- 172.21.15.59:0/675322282 shutdown_connections 2024-04-23T14:41:30.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.380+0000 7f8bfd800640 1 -- 172.21.15.59:0/675322282 wait complete. 2024-04-23T14:41:30.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.382+0000 7f8bfd800640 1 Processor -- start 2024-04-23T14:41:30.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.382+0000 7f8bfd800640 1 -- start start 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.383+0000 7f8bfd800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bf80afaf0 0x7f8bf8159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.383+0000 7f8bfd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80d06e0 0x7f8bf81fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.383+0000 7f8bfd800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bf815c5e0 0x7f8bf81571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.383+0000 7f8bfd800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf806e600 con 0x7f8bf80d06e0 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.383+0000 7f8bfd800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf806e480 con 0x7f8bf80afaf0 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.383+0000 7f8bfd800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf8162be0 con 0x7f8bf815c5e0 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80d06e0 0x7f8bf81fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80d06e0 0x7f8bf81fc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59594/0 (socket says 172.21.15.59:59594) 2024-04-23T14:41:30.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 -- 172.21.15.59:0/2548757797 learned_addr learned my addr 172.21.15.59:0/2548757797 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf7400640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bf815c5e0 0x7f8bf81571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6a00640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bf80afaf0 0x7f8bf8159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 -- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bf815c5e0 msgr2=0x7f8bf81571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bf815c5e0 0x7f8bf81571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 -- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bf80afaf0 msgr2=0x7f8bf8159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bf80afaf0 0x7f8bf8159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 -- 172.21.15.59:0/2548757797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf81fca70 con 0x7f8bf80d06e0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6a00640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bf80afaf0 0x7f8bf8159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.384+0000 7f8bf6000640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80d06e0 0x7f8bf81fc1b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009870 tx=0x7f8be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.385+0000 7f8beea00640 1 -- 172.21.15.59:0/2548757797 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0041020 con 0x7f8bf80d06e0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.385+0000 7f8beea00640 1 -- 172.21.15.59:0/2548757797 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be0030970 con 0x7f8bf80d06e0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.385+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf81fcd50 con 0x7f8bf80d06e0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.385+0000 7f8beea00640 1 -- 172.21.15.59:0/2548757797 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0037040 con 0x7f8bf80d06e0 2024-04-23T14:41:30.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.385+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf80cce40 con 0x7f8bf80d06e0 2024-04-23T14:41:30.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.386+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf8068a80 con 0x7f8bf80d06e0 2024-04-23T14:41:30.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.387+0000 7f8beea00640 1 -- 172.21.15.59:0/2548757797 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8be0044020 con 0x7f8bf80d06e0 2024-04-23T14:41:30.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.388+0000 7f8beea00640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8bd406c8f0 0x7f8bd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.388+0000 7f8beea00640 1 -- 172.21.15.59:0/2548757797 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f8be00ac550 con 0x7f8bf80d06e0 2024-04-23T14:41:30.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.389+0000 7f8bf6a00640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8bd406c8f0 0x7f8bd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.389+0000 7f8bf6a00640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8bd406c8f0 0x7f8bd406edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a830 tx=0x7f8be8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.393+0000 7f8beea00640 1 -- 172.21.15.59:0/2548757797 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8be004b030 con 0x7f8bf80d06e0 2024-04-23T14:41:30.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.597+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bf80cedf0 con 0x7f8bf80d06e0 2024-04-23T14:41:30.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.598+0000 7f8beea00640 1 -- 172.21.15.59:0/2548757797 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16561 (secure 0 0 0) 0x7f8be007a970 con 0x7f8bf80d06e0 2024-04-23T14:41:30.599 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:30.599 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":98,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:29.612688+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:30.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8bd406c8f0 msgr2=0x7f8bd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8bd406c8f0 0x7f8bd406edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a830 tx=0x7f8be8008040 comp rx=0 tx=0).stop 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80d06e0 msgr2=0x7f8bf81fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80d06e0 0x7f8bf81fc1b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009870 tx=0x7f8be00046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 shutdown_connections 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8bd406c8f0 0x7f8bd406edb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bf815c5e0 0x7f8bf81571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bf80d06e0 0x7f8bf81fc1b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 --2- 172.21.15.59:0/2548757797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bf80afaf0 0x7f8bf8159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 >> 172.21.15.59:0/2548757797 conn(0x7f8bf8067ea0 msgr2=0x7f8bf80ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 shutdown_connections 2024-04-23T14:41:30.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.601+0000 7f8bfd800640 1 -- 172.21.15.59:0/2548757797 wait complete. 2024-04-23T14:41:30.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:30.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.830+0000 7f0694a00640 1 Processor -- start 2024-04-23T14:41:30.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.830+0000 7f0694a00640 1 -- start start 2024-04-23T14:41:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.831+0000 7f0694a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06900b4130 0x7f069006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.831+0000 7f0694a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06900cfb90 0x7f06900c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.831+0000 7f0694a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069006ed40 con 0x7f0690162df0 2024-04-23T14:41:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.831+0000 7f0694a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f069006eaf0 con 0x7f06900b4130 2024-04-23T14:41:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.831+0000 7f0694a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f06900b3590 con 0x7f06900cfb90 2024-04-23T14:41:30.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06900b4130 0x7f069006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06900b4130 0x7f069006f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59604/0 (socket says 172.21.15.59:59604) 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068de00640 1 -- 172.21.15.59:0/3622211342 learned_addr learned my addr 172.21.15.59:0/3622211342 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068e800640 1 --1- 172.21.15.59:0/3622211342 >> v1:172.21.15.59:6789/0 conn(0x7f0690162df0 0x7f06900b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55608/0 (socket says 172.21.15.59:55608) 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068f200640 1 --2- 172.21.15.59:0/3622211342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06900cfb90 0x7f06900c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068de00640 1 -- 172.21.15.59:0/3622211342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06900cfb90 msgr2=0x7f06900c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068de00640 1 --2- 172.21.15.59:0/3622211342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06900cfb90 0x7f06900c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068de00640 1 -- 172.21.15.59:0/3622211342 >> v1:172.21.15.59:6789/0 conn(0x7f0690162df0 legacy=0x7f06900b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.832+0000 7f068de00640 1 -- 172.21.15.59:0/3622211342 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06900b3220 con 0x7f06900b4130 2024-04-23T14:41:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.833+0000 7f068de00640 1 --2- 172.21.15.59:0/3622211342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900b4130 0x7f069006f680 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0678009870 tx=0x7f067802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68126b50023ea81f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:30.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f068d400640 1 -- 172.21.15.59:0/3622211342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678037070 con 0x7f06900b4130 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f068d400640 1 -- 172.21.15.59:0/3622211342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0678032440 con 0x7f06900b4130 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f068d400640 1 -- 172.21.15.59:0/3622211342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678032810 con 0x7f06900b4130 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 -- 172.21.15.59:0/3622211342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900b4130 msgr2=0x7f069006f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 --2- 172.21.15.59:0/3622211342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900b4130 0x7f069006f680 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0678009870 tx=0x7f067802a250 comp rx=0 tx=0).stop 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 -- 172.21.15.59:0/3622211342 shutdown_connections 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 --2- 172.21.15.59:0/3622211342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06900cfb90 0x7f06900c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 --2- 172.21.15.59:0/3622211342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900b4130 0x7f069006f680 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 -- 172.21.15.59:0/3622211342 >> 172.21.15.59:0/3622211342 conn(0x7f06900b21a0 msgr2=0x7f06900b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 -- 172.21.15.59:0/3622211342 shutdown_connections 2024-04-23T14:41:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.834+0000 7f0694a00640 1 -- 172.21.15.59:0/3622211342 wait complete. 2024-04-23T14:41:30.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.836+0000 7f0694a00640 1 Processor -- start 2024-04-23T14:41:30.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.836+0000 7f0694a00640 1 -- start start 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.837+0000 7f0694a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06900b4130 0x7f06900c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.837+0000 7f0694a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900cfb90 0x7f0690071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.837+0000 7f0694a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0690162df0 0x7f0690072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.837+0000 7f0694a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06900c3860 con 0x7f06900cfb90 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.837+0000 7f0694a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f06900c36e0 con 0x7f06900b4130 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.837+0000 7f0694a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f06900c39e0 con 0x7f0690162df0 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.837+0000 7f068de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900cfb90 0x7f0690071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900cfb90 0x7f0690071f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59610/0 (socket says 172.21.15.59:59610) 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 -- 172.21.15.59:0/4002045642 learned_addr learned my addr 172.21.15.59:0/4002045642 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068e800640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06900b4130 0x7f06900c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068f200640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0690162df0 0x7f0690072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 -- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0690162df0 msgr2=0x7f0690072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0690162df0 0x7f0690072fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 -- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06900b4130 msgr2=0x7f06900c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06900b4130 0x7f06900c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 -- 172.21.15.59:0/4002045642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06901633e0 con 0x7f06900cfb90 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068e800640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06900b4130 0x7f06900c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.838+0000 7f068de00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900cfb90 0x7f0690071f80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f067802a290 tx=0x7f06780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.839+0000 7f0686a00640 1 -- 172.21.15.59:0/4002045642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678041020 con 0x7f06900cfb90 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.839+0000 7f0686a00640 1 -- 172.21.15.59:0/4002045642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06780306c0 con 0x7f06900cfb90 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.839+0000 7f0686a00640 1 -- 172.21.15.59:0/4002045642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678037040 con 0x7f06900cfb90 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.839+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06901636c0 con 0x7f06900cfb90 2024-04-23T14:41:30.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.839+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0690072ab0 con 0x7f06900cfb90 2024-04-23T14:41:30.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.840+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069004fd90 con 0x7f06900cfb90 2024-04-23T14:41:30.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.841+0000 7f0686a00640 1 -- 172.21.15.59:0/4002045642 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0678044020 con 0x7f06900cfb90 2024-04-23T14:41:30.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.842+0000 7f0686a00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f066c06c8f0 0x7f066c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:30.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.842+0000 7f0686a00640 1 -- 172.21.15.59:0/4002045642 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f06780ac360 con 0x7f06900cfb90 2024-04-23T14:41:30.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.842+0000 7f068e800640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f066c06c8f0 0x7f066c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:30.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.843+0000 7f068e800640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f066c06c8f0 0x7f066c06edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0680000f60 tx=0x7f0680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:30.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:30.846+0000 7f0686a00640 1 -- 172.21.15.59:0/4002045642 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f067804b030 con 0x7f06900cfb90 2024-04-23T14:41:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.044+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06900cee90 con 0x7f06900cfb90 2024-04-23T14:41:31.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.047+0000 7f0686a00640 1 -- 172.21.15.59:0/4002045642 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16561 (secure 0 0 0) 0x7f067807a780 con 0x7f06900cfb90 2024-04-23T14:41:31.048 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:31.048 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":98,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:29.612688+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:31.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f066c06c8f0 msgr2=0x7f066c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:31.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f066c06c8f0 0x7f066c06edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0680000f60 tx=0x7f0680009040 comp rx=0 tx=0).stop 2024-04-23T14:41:31.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900cfb90 msgr2=0x7f0690071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:31.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900cfb90 0x7f0690071f80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f067802a290 tx=0x7f06780046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 shutdown_connections 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f066c06c8f0 0x7f066c06edb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0690162df0 0x7f0690072fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06900cfb90 0x7f0690071f80 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 --2- 172.21.15.59:0/4002045642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06900b4130 0x7f06900c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 >> 172.21.15.59:0/4002045642 conn(0x7f06900b21a0 msgr2=0x7f06900b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 shutdown_connections 2024-04-23T14:41:31.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.050+0000 7f0694a00640 1 -- 172.21.15.59:0/4002045642 wait complete. 2024-04-23T14:41:31.068 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:41:31.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.271+0000 7fa96ca00640 1 Processor -- start 2024-04-23T14:41:31.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.272+0000 7fa96ca00640 1 -- start start 2024-04-23T14:41:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa96ca00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa9680d15b0 0x7fa9680cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9680b4f20 0x7fa9680cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa96ca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9680b5da0 con 0x7fa96815ed20 2024-04-23T14:41:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa96ca00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa96815f100 con 0x7fa9680d15b0 2024-04-23T14:41:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa96ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9680d0a20 con 0x7fa9680b4f20 2024-04-23T14:41:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa967400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa96815ed20 0x7fa9680d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55614/0 (socket says 172.21.15.59:55614) 2024-04-23T14:41:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa967400640 1 -- 172.21.15.59:0/1005559350 learned_addr learned my addr 172.21.15.59:0/1005559350 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.273+0000 7fa966a00640 1 --2- 172.21.15.59:0/1005559350 >> v2:172.21.15.76:3300/0 conn(0x7fa9680d15b0 0x7fa9680cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa967e00640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9680b4f20 0x7fa9680cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206323390 0 0) 0x7fa9680b5da0 con 0x7fa96815ed20 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa94c003520 con 0x7fa96815ed20 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 617661414 0 0) 0x7fa94c003520 con 0x7fa96815ed20 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9680b4f20 msgr2=0x7fa9680cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9680b4f20 0x7fa9680cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 >> v2:172.21.15.76:3300/0 conn(0x7fa9680d15b0 msgr2=0x7fa9680cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 --2- 172.21.15.59:0/1005559350 >> v2:172.21.15.76:3300/0 conn(0x7fa9680d15b0 0x7fa9680cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9680d2ed0 con 0x7fa96815ed20 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa9580019f0 con 0x7fa96815ed20 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 >> v1:172.21.15.59:6789/0 conn(0x7fa96815ed20 legacy=0x7fa9680d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa94c0039b0 0x7fa94c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9680b4f20 0x7fa94c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa94c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9680d0a20 con 0x7fa94c004d40 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c00ba40 con 0x7fa94c0039b0 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.274+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c00bc40 con 0x7fa9680b4f20 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.275+0000 7fa967e00640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa94c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.275+0000 7fa967400640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa94c0039b0 0x7fa94c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.275+0000 7fa967e00640 1 -- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9680b4f20 msgr2=0x7fa94c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.275+0000 7fa967e00640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9680b4f20 0x7fa94c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.275+0000 7fa967e00640 1 -- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa94c0039b0 msgr2=0x7fa94c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.275+0000 7fa967e00640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa94c0039b0 0x7fa94c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.275+0000 7fa967e00640 1 -- 172.21.15.59:0/1005559350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c003520 con 0x7fa94c004d40 2024-04-23T14:41:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.276+0000 7fa967e00640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa94c0051c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa960009a90 tx=0x7fa94c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19895c842c0ce7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.276+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960036070 con 0x7fa94c004d40 2024-04-23T14:41:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.276+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa960031400 con 0x7fa94c004d40 2024-04-23T14:41:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.276+0000 7fa966000640 1 -- 172.21.15.59:0/1005559350 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960009690 con 0x7fa94c004d40 2024-04-23T14:41:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.277+0000 7fa96ca00640 1 -- 172.21.15.59:0/1005559350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 msgr2=0x7fa94c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.277+0000 7fa96ca00640 1 --2- 172.21.15.59:0/1005559350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa94c0051c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa960009a90 tx=0x7fa94c005830 comp rx=0 tx=0).stop 2024-04-23T14:41:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.277+0000 7fa967e00640 1 -- 172.21.15.59:0/1005559350 reap_dead start 2024-04-23T14:41:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.277+0000 7fa96ca00640 1 -- 172.21.15.59:0/1005559350 shutdown_connections 2024-04-23T14:41:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.277+0000 7fa96ca00640 1 -- 172.21.15.59:0/1005559350 >> 172.21.15.59:0/1005559350 conn(0x7fa96808bf90 msgr2=0x7fa9680d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.277+0000 7fa96ca00640 1 -- 172.21.15.59:0/1005559350 shutdown_connections 2024-04-23T14:41:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.277+0000 7fa96ca00640 1 -- 172.21.15.59:0/1005559350 wait complete. 2024-04-23T14:41:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.279+0000 7fa96ca00640 1 Processor -- start 2024-04-23T14:41:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.279+0000 7fa96ca00640 1 -- start start 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa9681e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815b9d0 0x7fa9680cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa96815be30 0x7fa9680cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa96ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9680b3600 con 0x7fa94c004d40 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa96ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9680d3490 con 0x7fa96815b9d0 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa96ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa9680b3780 con 0x7fa96815be30 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa9681e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa9681e4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59638/0 (socket says 172.21.15.59:59638) 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 -- 172.21.15.59:0/2334153594 learned_addr learned my addr 172.21.15.59:0/2334153594 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 -- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa96815be30 msgr2=0x7fa9680cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967e00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa96815be30 0x7fa9680cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa966a00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815b9d0 0x7fa9680cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa96815be30 0x7fa9680cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 -- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815b9d0 msgr2=0x7fa9680cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815b9d0 0x7fa9680cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.280+0000 7fa967400640 1 -- 172.21.15.59:0/2334153594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9680ce2d0 con 0x7fa94c004d40 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.281+0000 7fa967400640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa9681e4a90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa958002180 tx=0x7fa95800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.281+0000 7fa95f400640 1 -- 172.21.15.59:0/2334153594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958019070 con 0x7fa94c004d40 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.281+0000 7fa95f400640 1 -- 172.21.15.59:0/2334153594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa958004d60 con 0x7fa94c004d40 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.281+0000 7fa95f400640 1 -- 172.21.15.59:0/2334153594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958007310 con 0x7fa94c004d40 2024-04-23T14:41:31.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.281+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9681e3fb0 con 0x7fa94c004d40 2024-04-23T14:41:31.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.281+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681e4570 con 0x7fa94c004d40 2024-04-23T14:41:31.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.282+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9680694e0 con 0x7fa94c004d40 2024-04-23T14:41:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.283+0000 7fa95f400640 1 -- 172.21.15.59:0/2334153594 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa95801c070 con 0x7fa94c004d40 2024-04-23T14:41:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.284+0000 7fa95f400640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa94406c8f0 0x7fa94406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.284+0000 7fa95f400640 1 -- 172.21.15.59:0/2334153594 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fa95808bdd0 con 0x7fa94c004d40 2024-04-23T14:41:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.285+0000 7fa966a00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa94406c8f0 0x7fa94406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:31.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.285+0000 7fa966a00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa94406c8f0 0x7fa94406edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa950004770 tx=0x7fa950008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:31.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.289+0000 7fa95f400640 1 -- 172.21.15.59:0/2334153594 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9680694e0 con 0x7fa94c004d40 2024-04-23T14:41:31.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:31.532+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa96804fd90 con 0x7fa94c004d40 2024-04-23T14:41:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.171+0000 7fa95f400640 1 -- 172.21.15.59:0/2334153594 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7fa95805a160 con 0x7fa94c004d40 2024-04-23T14:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.175+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa94406c8f0 msgr2=0x7fa94406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.175+0000 7fa96ca00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa94406c8f0 0x7fa94406edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa950004770 tx=0x7fa950008040 comp rx=0 tx=0).stop 2024-04-23T14:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.175+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 msgr2=0x7fa9681e4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.175+0000 7fa96ca00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa9681e4a90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa958002180 tx=0x7fa95800b560 comp rx=0 tx=0).stop 2024-04-23T14:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 shutdown_connections 2024-04-23T14:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa94406c8f0 0x7fa94406edb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa96815be30 0x7fa9680cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815b9d0 0x7fa9680cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 --2- 172.21.15.59:0/2334153594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa94c004d40 0x7fa9681e4a90 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 >> 172.21.15.59:0/2334153594 conn(0x7fa96808bf90 msgr2=0x7fa9680d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 shutdown_connections 2024-04-23T14:41:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.176+0000 7fa96ca00640 1 -- 172.21.15.59:0/2334153594 wait complete. 2024-04-23T14:41:32.197 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:41:32.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:41:32.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.421+0000 7f37eea00640 1 Processor -- start 2024-04-23T14:41:32.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.421+0000 7f37eea00640 1 -- start start 2024-04-23T14:41:32.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37eea00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f37e815d810 0x7f37e8160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37eea00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e815cab0 con 0x7f37e80d0d10 2024-04-23T14:41:32.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37eea00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e8160e40 con 0x7f37e81567e0 2024-04-23T14:41:32.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37eea00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f37e815cd30 con 0x7f37e815d810 2024-04-23T14:41:32.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37ecc00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f37e80d0d10 0x7f37e8160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55630/0 (socket says 172.21.15.59:55630) 2024-04-23T14:41:32.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37ecc00640 1 -- 172.21.15.59:0/790530699 learned_addr learned my addr 172.21.15.59:0/790530699 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:32.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37ee000640 1 --2- 172.21.15.59:0/790530699 >> v2:172.21.15.76:3300/0 conn(0x7f37e815d810 0x7f37e8160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.423+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211410164 0 0) 0x7f37d4001110 con 0x7f37e80d0d10 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37d0003520 con 0x7f37e80d0d10 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3979168662 0 0) 0x7f37d0003520 con 0x7f37e80d0d10 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 >> v2:172.21.15.76:3300/0 conn(0x7f37e815d810 msgr2=0x7f37e8160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 --2- 172.21.15.59:0/790530699 >> v2:172.21.15.76:3300/0 conn(0x7f37e815d810 0x7f37e8160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 >> v1:172.21.15.76:6789/0 conn(0x7f37e81567e0 legacy=0x7f37e80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e815d010 con 0x7f37e80d0d10 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f37d4001840 con 0x7f37e80d0d10 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 >> v1:172.21.15.59:6789/0 conn(0x7f37e80d0d10 legacy=0x7f37e8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:32.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d0003a00 0x7f37d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37d0004800 0x7f37d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37d00051c0 0x7f37d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37e815cd30 con 0x7f37d00051c0 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f37d000c0c0 con 0x7f37d0003a00 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.424+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f37d000c2c0 con 0x7f37d0004800 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ee000640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37d00051c0 0x7f37d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ed600640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d0003a00 0x7f37d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ecc00640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37d0004800 0x7f37d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ee000640 1 -- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37d0004800 msgr2=0x7f37d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ee000640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37d0004800 0x7f37d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ee000640 1 -- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d0003a00 msgr2=0x7f37d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ee000640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d0003a00 0x7f37d0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.425+0000 7f37ee000640 1 -- 172.21.15.59:0/790530699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d0003520 con 0x7f37d00051c0 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.426+0000 7f37ee000640 1 -- 172.21.15.59:0/790530699 reap_dead start 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.426+0000 7f37ee000640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37d00051c0 0x7f37d000b9c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009f90 tx=0x7f37d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33076ef30ded22e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.426+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0035020 con 0x7f37d00051c0 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.426+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e0003b90 con 0x7f37d00051c0 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.426+0000 7f37e7e00640 1 -- 172.21.15.59:0/790530699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0009c10 con 0x7f37d00051c0 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.427+0000 7f37eea00640 1 -- 172.21.15.59:0/790530699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37d00051c0 msgr2=0x7f37d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.427+0000 7f37eea00640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37d00051c0 0x7f37d000b9c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009f90 tx=0x7f37d0005830 comp rx=0 tx=0).stop 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.427+0000 7f37eea00640 1 -- 172.21.15.59:0/790530699 shutdown_connections 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.427+0000 7f37eea00640 1 --2- 172.21.15.59:0/790530699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37d00051c0 0x7f37d000b9c0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.427+0000 7f37eea00640 1 -- 172.21.15.59:0/790530699 >> 172.21.15.59:0/790530699 conn(0x7f37e806bdf0 msgr2=0x7f37e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:32.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.427+0000 7f37eea00640 1 -- 172.21.15.59:0/790530699 shutdown_connections 2024-04-23T14:41:32.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.427+0000 7f37eea00640 1 -- 172.21.15.59:0/790530699 wait complete. 2024-04-23T14:41:32.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.429+0000 7f37eea00640 1 Processor -- start 2024-04-23T14:41:32.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.429+0000 7f37eea00640 1 -- start start 2024-04-23T14:41:32.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.430+0000 7f37eea00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d00051c0 0x7f37e81f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.430+0000 7f37eea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e806e4f0 0x7f37e806e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.430+0000 7f37eea00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37e815dc50 0x7f37e815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.430+0000 7f37eea00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8156d40 con 0x7f37e806e4f0 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.430+0000 7f37eea00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8156bc0 con 0x7f37d00051c0 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.430+0000 7f37eea00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f37e8156ec0 con 0x7f37e815dc50 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e806e4f0 0x7f37e806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e806e4f0 0x7f37e806e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59666/0 (socket says 172.21.15.59:59666) 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 -- 172.21.15.59:0/4272049145 learned_addr learned my addr 172.21.15.59:0/4272049145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ed600640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d00051c0 0x7f37e81f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 -- 172.21.15.59:0/4272049145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37e815dc50 msgr2=0x7f37e815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37e815dc50 0x7f37e815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 -- 172.21.15.59:0/4272049145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d00051c0 msgr2=0x7f37e81f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:32.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d00051c0 0x7f37e81f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:32.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.431+0000 7f37ecc00640 1 -- 172.21.15.59:0/4272049145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81524d0 con 0x7f37e806e4f0 2024-04-23T14:41:32.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.432+0000 7f37ecc00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e806e4f0 0x7f37e806e970 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e700 tx=0x7f37d400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:32.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.432+0000 7f37e5600640 1 -- 172.21.15.59:0/4272049145 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d400c6c0 con 0x7f37e806e4f0 2024-04-23T14:41:32.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.432+0000 7f37e5600640 1 -- 172.21.15.59:0/4272049145 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d4018070 con 0x7f37e806e4f0 2024-04-23T14:41:32.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.432+0000 7f37e5600640 1 -- 172.21.15.59:0/4272049145 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d400caa0 con 0x7f37e806e4f0 2024-04-23T14:41:32.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.432+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e8159300 con 0x7f37e806e4f0 2024-04-23T14:41:32.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.432+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81598c0 con 0x7f37e806e4f0 2024-04-23T14:41:32.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.433+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e8159ed0 con 0x7f37e806e4f0 2024-04-23T14:41:32.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.434+0000 7f37e5600640 1 -- 172.21.15.59:0/4272049145 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37d400cc40 con 0x7f37e806e4f0 2024-04-23T14:41:32.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.435+0000 7f37e5600640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f37c806c8f0 0x7f37c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:32.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.435+0000 7f37e5600640 1 -- 172.21.15.59:0/4272049145 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f37d408b960 con 0x7f37e806e4f0 2024-04-23T14:41:32.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.436+0000 7f37ed600640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f37c806c8f0 0x7f37c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:32.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.436+0000 7f37ed600640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f37c806c8f0 0x7f37c806edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005490 tx=0x7f37dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:32.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.440+0000 7f37e5600640 1 -- 172.21.15.59:0/4272049145 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37e815cab0 con 0x7f37e806e4f0 2024-04-23T14:41:32.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:32.685+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f37e804fd90 con 0x7f37e806e4f0 2024-04-23T14:41:33.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.175+0000 7f37e5600640 1 -- 172.21.15.59:0/4272049145 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f37d4059cf0 con 0x7f37e806e4f0 2024-04-23T14:41:33.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f37c806c8f0 msgr2=0x7f37c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f37c806c8f0 0x7f37c806edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005490 tx=0x7f37dc009040 comp rx=0 tx=0).stop 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e806e4f0 msgr2=0x7f37e806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e806e4f0 0x7f37e806e970 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e700 tx=0x7f37d400ebc0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 shutdown_connections 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f37c806c8f0 0x7f37c806edb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f37e815dc50 0x7f37e815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e806e4f0 0x7f37e806e970 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 --2- 172.21.15.59:0/4272049145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37d00051c0 0x7f37e81f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 >> 172.21.15.59:0/4272049145 conn(0x7f37e806bdf0 msgr2=0x7f37e80728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.179+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 shutdown_connections 2024-04-23T14:41:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.180+0000 7f37eea00640 1 -- 172.21.15.59:0/4272049145 wait complete. 2024-04-23T14:41:33.203 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:41:33.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.413+0000 7f9746000640 1 Processor -- start 2024-04-23T14:41:33.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.414+0000 7f9746000640 1 -- start start 2024-04-23T14:41:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.415+0000 7f9746000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f97400b5ff0 0x7f97400b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.415+0000 7f9746000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97400cf030 con 0x7f974015e7c0 2024-04-23T14:41:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.415+0000 7f9746000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97400b6a40 con 0x7f97400d0d90 2024-04-23T14:41:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.416+0000 7f9746000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f97400b7000 con 0x7f97400b5ff0 2024-04-23T14:41:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.416+0000 7f9744c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f974015e7c0 0x7f97400d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55634/0 (socket says 172.21.15.59:55634) 2024-04-23T14:41:33.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.416+0000 7f9744c00640 1 -- 172.21.15.59:0/883346794 learned_addr learned my addr 172.21.15.59:0/883346794 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.416+0000 7f973fe00640 1 --2- 172.21.15.59:0/883346794 >> v2:172.21.15.76:3300/0 conn(0x7f97400b5ff0 0x7f97400b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.416+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323806686 0 0) 0x7f9728001110 con 0x7f974015e7c0 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.416+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9724003520 con 0x7f974015e7c0 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2994537324 0 0) 0x7f9724003520 con 0x7f974015e7c0 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 >> v2:172.21.15.76:3300/0 conn(0x7f97400b5ff0 msgr2=0x7f97400b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 --2- 172.21.15.59:0/883346794 >> v2:172.21.15.76:3300/0 conn(0x7f97400b5ff0 0x7f97400b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 >> v1:172.21.15.76:6789/0 conn(0x7f97400d0d90 legacy=0x7f97400cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97400b6d20 con 0x7f974015e7c0 2024-04-23T14:41:33.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9728001840 con 0x7f974015e7c0 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 >> v1:172.21.15.59:6789/0 conn(0x7f974015e7c0 legacy=0x7f97400d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9724003a00 0x7f9724003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9724004800 0x7f9724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97240051c0 0x7f972400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97400b7000 con 0x7f9724003a00 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f972400c090 con 0x7f97240051c0 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f972400c290 con 0x7f9724004800 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f9744c00640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9724003a00 0x7f9724003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.417+0000 7f9745600640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97240051c0 0x7f972400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f973fe00640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9724004800 0x7f9724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f9744c00640 1 -- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9724004800 msgr2=0x7f9724004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f9744c00640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9724004800 0x7f9724004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f9744c00640 1 -- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97240051c0 msgr2=0x7f972400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f9744c00640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97240051c0 0x7f972400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f9744c00640 1 -- 172.21.15.59:0/883346794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9724003520 con 0x7f9724003a00 2024-04-23T14:41:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f9745600640 1 -- 172.21.15.59:0/883346794 reap_dead start 2024-04-23T14:41:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.418+0000 7f973fe00640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9724004800 0x7f9724004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.419+0000 7f9744c00640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9724003a00 0x7f9724003e00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f972800c320 tx=0x7f9724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee3a6c28c246f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.419+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728005ad0 con 0x7f9724003a00 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.419+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728034070 con 0x7f9724003a00 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.419+0000 7f973f400640 1 -- 172.21.15.59:0/883346794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728007070 con 0x7f9724003a00 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.420+0000 7f9746000640 1 -- 172.21.15.59:0/883346794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9724003a00 msgr2=0x7f9724003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.420+0000 7f9746000640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9724003a00 0x7f9724003e00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f972800c320 tx=0x7f9724005830 comp rx=0 tx=0).stop 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.420+0000 7f9746000640 1 -- 172.21.15.59:0/883346794 shutdown_connections 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.420+0000 7f9746000640 1 --2- 172.21.15.59:0/883346794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9724003a00 0x7f9724003e00 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.420+0000 7f9746000640 1 -- 172.21.15.59:0/883346794 >> 172.21.15.59:0/883346794 conn(0x7f974009d920 msgr2=0x7f974008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.420+0000 7f9746000640 1 -- 172.21.15.59:0/883346794 shutdown_connections 2024-04-23T14:41:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.420+0000 7f9746000640 1 -- 172.21.15.59:0/883346794 wait complete. 2024-04-23T14:41:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.422+0000 7f9746000640 1 Processor -- start 2024-04-23T14:41:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.422+0000 7f9746000640 1 -- start start 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f9746000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9724003a00 0x7f97400cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f9746000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97400b4440 0x7f97400b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f9746000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97400b7b70 0x7f97400b8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f9746000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f97400d1a80 con 0x7f97400b4440 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f9746000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f97400cf620 con 0x7f9724003a00 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f9746000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f97400d1c00 con 0x7f97400b7b70 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f973fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97400b4440 0x7f97400b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f973fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97400b4440 0x7f97400b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59676/0 (socket says 172.21.15.59:59676) 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.423+0000 7f973fe00640 1 -- 172.21.15.59:0/2913464680 learned_addr learned my addr 172.21.15.59:0/2913464680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973fe00640 1 -- 172.21.15.59:0/2913464680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97400b7b70 msgr2=0x7f97400b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973fe00640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97400b7b70 0x7f97400b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973fe00640 1 -- 172.21.15.59:0/2913464680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9724003a00 msgr2=0x7f97400cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f9744c00640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9724003a00 0x7f97400cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973fe00640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9724003a00 0x7f97400cddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973fe00640 1 -- 172.21.15.59:0/2913464680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97400b8560 con 0x7f97400b4440 2024-04-23T14:41:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973fe00640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97400b4440 0x7f97400b48c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9730009fc0 tx=0x7f973000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973cc00640 1 -- 172.21.15.59:0/2913464680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730003990 con 0x7f97400b4440 2024-04-23T14:41:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973cc00640 1 -- 172.21.15.59:0/2913464680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9730003b30 con 0x7f97400b4440 2024-04-23T14:41:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f973cc00640 1 -- 172.21.15.59:0/2913464680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730004020 con 0x7f97400b4440 2024-04-23T14:41:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97400ce310 con 0x7f97400b4440 2024-04-23T14:41:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.424+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9740153e40 con 0x7f97400b4440 2024-04-23T14:41:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.426+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974004fd90 con 0x7f97400b4440 2024-04-23T14:41:33.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.427+0000 7f973cc00640 1 -- 172.21.15.59:0/2913464680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f973001d020 con 0x7f97400b4440 2024-04-23T14:41:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.427+0000 7f973cc00640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.428+0000 7f973cc00640 1 -- 172.21.15.59:0/2913464680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f973008c2d0 con 0x7f97400b4440 2024-04-23T14:41:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.428+0000 7f9744c00640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.429+0000 7f9744c00640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9728033000 tx=0x7f972802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:33.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.432+0000 7f973cc00640 1 -- 172.21.15.59:0/2913464680 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97400ce310 con 0x7f97400b4440 2024-04-23T14:41:33.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.678+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f974015ce40 con 0x7f97400b4440 2024-04-23T14:41:33.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.679+0000 7f973cc00640 1 -- 172.21.15.59:0/2913464680 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5180 (secure 0 0 0) 0x7f973005a6f0 con 0x7f97400b4440 2024-04-23T14:41:33.680 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:33.680 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:41:32.686386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8514":{"gid":8514,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/620420969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":620420969},{"type":"v1","addr":"172.21.15.59:6837","nonce":620420969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8514,"qdb_cluster":[8514]},"id":7}]} 2024-04-23T14:41:33.680 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 82 2024-04-23T14:41:33.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 msgr2=0x7f971c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9728033000 tx=0x7f972802e000 comp rx=0 tx=0).stop 2024-04-23T14:41:33.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97400b4440 msgr2=0x7f97400b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97400b4440 0x7f97400b48c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9730009fc0 tx=0x7f973000f520 comp rx=0 tx=0).stop 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 shutdown_connections 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f971c06c920 0x7f971c06ede0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97400b7b70 0x7f97400b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97400b4440 0x7f97400b48c0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 --2- 172.21.15.59:0/2913464680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9724003a00 0x7f97400cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 >> 172.21.15.59:0/2913464680 conn(0x7f974009d920 msgr2=0x7f974006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 shutdown_connections 2024-04-23T14:41:33.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.683+0000 7f9746000640 1 -- 172.21.15.59:0/2913464680 wait complete. 2024-04-23T14:41:33.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:33.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.905+0000 7f488a400640 1 Processor -- start 2024-04-23T14:41:33.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.905+0000 7f488a400640 1 -- start start 2024-04-23T14:41:33.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.906+0000 7f488a400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f488415cb20 0x7f48840cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.906+0000 7f488a400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884163770 0x7f48840d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.906+0000 7f488a400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48840d01c0 con 0x7f4884160180 2024-04-23T14:41:33.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.906+0000 7f488a400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f488415cf00 con 0x7f488415cb20 2024-04-23T14:41:33.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.906+0000 7f488a400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f488415f6a0 con 0x7f4884163770 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.907+0000 7f4883400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f488415cb20 0x7f48840cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.907+0000 7f4883400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f488415cb20 0x7f48840cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:34140/0 (socket says 172.21.15.59:34140) 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.907+0000 7f4883400640 1 -- 172.21.15.59:0/3502518546 learned_addr learned my addr 172.21.15.59:0/3502518546 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.907+0000 7f4882a00640 1 --2- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884163770 0x7f48840d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.908+0000 7f4883400640 1 -- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884163770 msgr2=0x7f48840d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.908+0000 7f4883400640 1 --2- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884163770 0x7f48840d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.908+0000 7f4883400640 1 -- 172.21.15.59:0/3502518546 >> v1:172.21.15.76:6789/0 conn(0x7f4884160180 legacy=0x7f48840b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.908+0000 7f4883400640 1 -- 172.21.15.59:0/3502518546 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488415f980 con 0x7f488415cb20 2024-04-23T14:41:33.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.909+0000 7f4883400640 1 --2- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488415cb20 0x7f48840cf840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4874009870 tx=0x7f487402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adbcb8344ad2dc14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:33.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f4882000640 1 -- 172.21.15.59:0/3502518546 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874037070 con 0x7f488415cb20 2024-04-23T14:41:33.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f4882000640 1 -- 172.21.15.59:0/3502518546 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4874032440 con 0x7f488415cb20 2024-04-23T14:41:33.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f4882000640 1 -- 172.21.15.59:0/3502518546 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874032810 con 0x7f488415cb20 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 -- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488415cb20 msgr2=0x7f48840cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 --2- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488415cb20 0x7f48840cf840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4874009870 tx=0x7f487402a1c0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 -- 172.21.15.59:0/3502518546 shutdown_connections 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 --2- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884163770 0x7f48840d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 --2- 172.21.15.59:0/3502518546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f488415cb20 0x7f48840cf840 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 -- 172.21.15.59:0/3502518546 >> 172.21.15.59:0/3502518546 conn(0x7f488406cc70 msgr2=0x7f488406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 -- 172.21.15.59:0/3502518546 shutdown_connections 2024-04-23T14:41:33.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.910+0000 7f488a400640 1 -- 172.21.15.59:0/3502518546 wait complete. 2024-04-23T14:41:33.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.912+0000 7f488a400640 1 Processor -- start 2024-04-23T14:41:33.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.912+0000 7f488a400640 1 -- start start 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f488a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488415cb20 0x7f4884157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f488a400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884160180 0x7f48841ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f488a400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4884163770 0x7f4884159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f488a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48840d1780 con 0x7f488415cb20 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f488a400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f48840d1900 con 0x7f4884163770 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f488a400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f48840d1a80 con 0x7f4884160180 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f4883400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488415cb20 0x7f4884157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f4883400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488415cb20 0x7f4884157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59682/0 (socket says 172.21.15.59:59682) 2024-04-23T14:41:33.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.913+0000 7f4883400640 1 -- 172.21.15.59:0/858063021 learned_addr learned my addr 172.21.15.59:0/858063021 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:33.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f4883400640 1 -- 172.21.15.59:0/858063021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884160180 msgr2=0x7f48841ea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f4883400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884160180 0x7f48841ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f4883400640 1 -- 172.21.15.59:0/858063021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4884163770 msgr2=0x7f4884159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:33.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f4883400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4884163770 0x7f4884159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:33.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f4883400640 1 -- 172.21.15.59:0/858063021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48840c8d40 con 0x7f488415cb20 2024-04-23T14:41:33.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f4883400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488415cb20 0x7f4884157cd0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c00 tx=0x7f48740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:33.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f487b400640 1 -- 172.21.15.59:0/858063021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874037040 con 0x7f488415cb20 2024-04-23T14:41:33.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f487b400640 1 -- 172.21.15.59:0/858063021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4874042070 con 0x7f488415cb20 2024-04-23T14:41:33.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48841eabc0 con 0x7f488415cb20 2024-04-23T14:41:33.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f487b400640 1 -- 172.21.15.59:0/858063021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874030880 con 0x7f488415cb20 2024-04-23T14:41:33.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.914+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841e9160 con 0x7f488415cb20 2024-04-23T14:41:33.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.915+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4884073c30 con 0x7f488415cb20 2024-04-23T14:41:33.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.916+0000 7f487b400640 1 -- 172.21.15.59:0/858063021 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f487402ad10 con 0x7f488415cb20 2024-04-23T14:41:33.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.917+0000 7f487b400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f486406c920 0x7f486406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:33.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.917+0000 7f487b400640 1 -- 172.21.15.59:0/858063021 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f48740ac0d0 con 0x7f488415cb20 2024-04-23T14:41:33.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.918+0000 7f4882a00640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f486406c920 0x7f486406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:33.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.918+0000 7f4882a00640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f486406c920 0x7f486406ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4870006a90 tx=0x7f4870006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:33.922+0000 7f487b400640 1 -- 172.21.15.59:0/858063021 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4884073c30 con 0x7f488415cb20 2024-04-23T14:41:34.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.127+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4884068a80 con 0x7f488415cb20 2024-04-23T14:41:34.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.130+0000 7f487b400640 1 -- 172.21.15.59:0/858063021 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16548 (secure 0 0 0) 0x7f48840c8d40 con 0x7f488415cb20 2024-04-23T14:41:34.131 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:34.131 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":99,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:31.158736+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:34.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.133+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f486406c920 msgr2=0x7f486406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.133+0000 7f488a400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f486406c920 0x7f486406ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4870006a90 tx=0x7f4870006a20 comp rx=0 tx=0).stop 2024-04-23T14:41:34.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.133+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488415cb20 msgr2=0x7f4884157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.133+0000 7f488a400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488415cb20 0x7f4884157cd0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c00 tx=0x7f48740046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 shutdown_connections 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f486406c920 0x7f486406ede0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4884160180 0x7f48841ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4884163770 0x7f4884159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 --2- 172.21.15.59:0/858063021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f488415cb20 0x7f4884157cd0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 >> 172.21.15.59:0/858063021 conn(0x7f488406cc70 msgr2=0x7f48840d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 shutdown_connections 2024-04-23T14:41:34.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.134+0000 7f488a400640 1 -- 172.21.15.59:0/858063021 wait complete. 2024-04-23T14:41:34.153 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:41:34.153 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:41:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.347+0000 7f41a9800640 1 Processor -- start 2024-04-23T14:41:34.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.347+0000 7f41a9800640 1 -- start start 2024-04-23T14:41:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.348+0000 7f41a9800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41a415c3d0 0x7f41a40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.348+0000 7f41a9800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f41a40cd590 0x7f41a40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.348+0000 7f41a9800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a4160210 con 0x7f41a40b4690 2024-04-23T14:41:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.348+0000 7f41a9800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f41a41604f0 con 0x7f41a415c3d0 2024-04-23T14:41:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.348+0000 7f41a9800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f41a41607d0 con 0x7f41a40cd590 2024-04-23T14:41:34.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41a415c3d0 0x7f41a40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41a415c3d0 0x7f41a40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59692/0 (socket says 172.21.15.59:59692) 2024-04-23T14:41:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3e00640 1 -- 172.21.15.59:0/594572761 learned_addr learned my addr 172.21.15.59:0/594572761 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3400640 1 --2- 172.21.15.59:0/594572761 >> v2:172.21.15.76:3300/0 conn(0x7f41a40cd590 0x7f41a40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3e00640 1 -- 172.21.15.59:0/594572761 >> v2:172.21.15.76:3300/0 conn(0x7f41a40cd590 msgr2=0x7f41a40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3e00640 1 --2- 172.21.15.59:0/594572761 >> v2:172.21.15.76:3300/0 conn(0x7f41a40cd590 0x7f41a40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3e00640 1 -- 172.21.15.59:0/594572761 >> v1:172.21.15.76:6789/0 conn(0x7f41a40b4690 legacy=0x7f41a406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:34.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.349+0000 7f41a3e00640 1 -- 172.21.15.59:0/594572761 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a4160ab0 con 0x7f41a415c3d0 2024-04-23T14:41:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.350+0000 7f41a3e00640 1 --2- 172.21.15.59:0/594572761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40cd050 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f41a40cff90 tx=0x7f419402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb306acdc2bcc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.350+0000 7f41a2a00640 1 -- 172.21.15.59:0/594572761 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194037070 con 0x7f41a415c3d0 2024-04-23T14:41:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a2a00640 1 -- 172.21.15.59:0/594572761 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4194032440 con 0x7f41a415c3d0 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a2a00640 1 -- 172.21.15.59:0/594572761 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194032810 con 0x7f41a415c3d0 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 -- 172.21.15.59:0/594572761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 msgr2=0x7f41a40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 --2- 172.21.15.59:0/594572761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40cd050 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f41a40cff90 tx=0x7f419402a230 comp rx=0 tx=0).stop 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 -- 172.21.15.59:0/594572761 shutdown_connections 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 --2- 172.21.15.59:0/594572761 >> v2:172.21.15.76:3300/0 conn(0x7f41a40cd590 0x7f41a40b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 --2- 172.21.15.59:0/594572761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40cd050 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 -- 172.21.15.59:0/594572761 >> 172.21.15.59:0/594572761 conn(0x7f41a40b21a0 msgr2=0x7f41a40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 -- 172.21.15.59:0/594572761 shutdown_connections 2024-04-23T14:41:34.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.351+0000 7f41a9800640 1 -- 172.21.15.59:0/594572761 wait complete. 2024-04-23T14:41:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.353+0000 7f41a9800640 1 Processor -- start 2024-04-23T14:41:34.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.353+0000 7f41a9800640 1 -- start start 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a9800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a40b4690 0x7f41a40c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a9800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41a40cd590 0x7f41a40cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a9800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a9800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41a40b4ac0 con 0x7f41a415c3d0 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a9800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f41a40cd9a0 con 0x7f41a40b4690 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a9800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f41a415c7b0 con 0x7f41a40cd590 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a8e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59696/0 (socket says 172.21.15.59:59696) 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.354+0000 7f41a8e00640 1 -- 172.21.15.59:0/809557364 learned_addr learned my addr 172.21.15.59:0/809557364 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a8e00640 1 -- 172.21.15.59:0/809557364 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41a40cd590 msgr2=0x7f41a40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a8e00640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41a40cd590 0x7f41a40cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a8e00640 1 -- 172.21.15.59:0/809557364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a40b4690 msgr2=0x7f41a40c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a8e00640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a40b4690 0x7f41a40c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a8e00640 1 -- 172.21.15.59:0/809557364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a40ad100 con 0x7f41a415c3d0 2024-04-23T14:41:34.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a8e00640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40ce0c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f419c00c780 tx=0x7f419c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:34.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f419be00640 1 -- 172.21.15.59:0/809557364 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c004b60 con 0x7f41a415c3d0 2024-04-23T14:41:34.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f419be00640 1 -- 172.21.15.59:0/809557364 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419c004d00 con 0x7f41a415c3d0 2024-04-23T14:41:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f419be00640 1 -- 172.21.15.59:0/809557364 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c00aa90 con 0x7f41a415c3d0 2024-04-23T14:41:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a40ad3e0 con 0x7f41a415c3d0 2024-04-23T14:41:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.355+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a40aed40 con 0x7f41a415c3d0 2024-04-23T14:41:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.357+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a404fd90 con 0x7f41a415c3d0 2024-04-23T14:41:34.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.358+0000 7f419be00640 1 -- 172.21.15.59:0/809557364 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f419c00ac30 con 0x7f41a415c3d0 2024-04-23T14:41:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.358+0000 7f419be00640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f418006c8f0 0x7f418006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.359+0000 7f419be00640 1 -- 172.21.15.59:0/809557364 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f419c08be50 con 0x7f41a415c3d0 2024-04-23T14:41:34.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.359+0000 7f41a3e00640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f418006c8f0 0x7f418006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.359+0000 7f41a3e00640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f418006c8f0 0x7f418006edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4194034000 tx=0x7f419402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:34.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.363+0000 7f419be00640 1 -- 172.21.15.59:0/809557364 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f419c020070 con 0x7f41a415c3d0 2024-04-23T14:41:34.546 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:41:34.545+0000 7f6a67000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:41:34.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.553+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41a40ad900 con 0x7f418006c8f0 2024-04-23T14:41:34.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.571+0000 7f419be00640 1 -- 172.21.15.59:0/809557364 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246769 (secure 0 0 0) 0x7f41940325c0 con 0x7f418006c8f0 2024-04-23T14:41:34.576 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:34.577 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:41:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.577+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f418006c8f0 msgr2=0x7f418006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.577+0000 7f41a9800640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f418006c8f0 0x7f418006edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4194034000 tx=0x7f419402c000 comp rx=0 tx=0).stop 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.577+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 msgr2=0x7f41a40ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.577+0000 7f41a9800640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40ce0c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f419c00c780 tx=0x7f419c00cc40 comp rx=0 tx=0).stop 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 shutdown_connections 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f418006c8f0 0x7f418006edb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41a40cd590 0x7f41a40cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41a415c3d0 0x7f41a40ce0c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 --2- 172.21.15.59:0/809557364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a40b4690 0x7f41a40c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 >> 172.21.15.59:0/809557364 conn(0x7f41a40b21a0 msgr2=0x7f41a40b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 shutdown_connections 2024-04-23T14:41:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.578+0000 7f41a9800640 1 -- 172.21.15.59:0/809557364 wait complete. 2024-04-23T14:41:34.597 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-04-23T14:41:32.559155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259936,"kb_used_data":10488,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714784,"statfs":{"total":773094113280,"available":772827938816,"internally_reserved":0,"allocated":10739712,"data_stored":5290860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.443467"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.138081+0000","last_change":"2024-04-23T14:41:29.138081+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.138081+0000","last_undegraded":"2024-04-23T14:41:29.138081+0000","last_fullsized":"2024-04-23T14:41:29.138081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.145105+0000","last_change":"2024-04-23T14:41:29.145105+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.145105+0000","last_undegraded":"2024-04-23T14:41:29.145105+0000","last_fullsized":"2024-04-23T14:41:29.145105+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.137406+0000","last_change":"2024-04-23T14:41:29.137406+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.137406+0000","last_undegraded":"2024-04-23T14:41:29.137406+0000","last_fullsized":"2024-04-23T14:41:29.137406+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.144441+0000","last_change":"2024-04-23T14:41:29.144441+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.144441+0000","last_undegraded":"2024-04-23T14:41:29.144441+0000","last_fullsized":"2024-04-23T14:41:29.144441+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.138563+0000","last_change":"2024-04-23T14:41:29.138563+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.138563+0000","last_undegraded":"2024-04-23T14:41:29.138563+0000","last_fullsized":"2024-04-23T14:41:29.138563+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.153101+0000","last_change":"2024-04-23T14:41:29.153101+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.153101+0000","last_undegraded":"2024-04-23T14:41:29.153101+0000","last_fullsized":"2024-04-23T14:41:29.153101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.157172+0000","last_change":"2024-04-23T14:41:29.157172+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.157172+0000","last_undegraded":"2024-04-23T14:41:29.157172+0000","last_fullsized":"2024-04-23T14:41:29.157172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.143384+0000","last_change":"2024-04-23T14:41:29.143384+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.143384+0000","last_undegraded":"2024-04-23T14:41:29.143384+0000","last_fullsized":"2024-04-23T14:41:29.143384+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.155966+0000","last_change":"2024-04-23T14:41:29.155966+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.155966+0000","last_undegraded":"2024-04-23T14:41:29.155966+0000","last_fullsized":"2024-04-23T14:41:29.155966+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.141304+0000","last_change":"2024-04-23T14:41:29.141304+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.141304+0000","last_undegraded":"2024-04-23T14:41:29.141304+0000","last_fullsized":"2024-04-23T14:41:29.141304+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.160744+0000","last_change":"2024-04-23T14:41:29.160744+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.160744+0000","last_undegraded":"2024-04-23T14:41:29.160744+0000","last_fullsized":"2024-04-23T14:41:29.160744+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.156432+0000","last_change":"2024-04-23T14:41:29.156432+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.156432+0000","last_undegraded":"2024-04-23T14:41:29.156432+0000","last_fullsized":"2024-04-23T14:41:29.156432+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.145253+0000","last_change":"2024-04-23T14:41:29.145253+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.145253+0000","last_undegraded":"2024-04-23T14:41:29.145253+0000","last_fullsized":"2024-04-23T14:41:29.145253+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.147332+0000","last_change":"2024-04-23T14:41:29.147332+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.147332+0000","last_undegraded":"2024-04-23T14:41:29.147332+0000","last_fullsized":"2024-04-23T14:41:29.147332+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.147084+0000","last_change":"2024-04-23T14:41:29.147084+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.147084+0000","last_undegraded":"2024-04-23T14:41:29.147084+0000","last_fullsized":"2024-04-23T14:41:29.147084+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.152262+0000","last_change":"2024-04-23T14:41:29.152262+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.152262+0000","last_undegraded":"2024-04-23T14:41:29.152262+0000","last_fullsized":"2024-04-23T14:41:29.152262+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.142625+0000","last_change":"2024-04-23T14:41:29.142625+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.142625+0000","last_undegraded":"2024-04-23T14:41:29.142625+0000","last_fullsized":"2024-04-23T14:41:29.142625+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.146204+0000","last_change":"2024-04-23T14:41:29.146204+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.146204+0000","last_undegraded":"2024-04-23T14:41:29.146204+0000","last_fullsized":"2024-04-23T14:41:29.146204+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.145333+0000","last_change":"2024-04-23T14:41:29.145333+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.145333+0000","last_undegraded":"2024-04-23T14:41:29.145333+0000","last_fullsized":"2024-04-23T14:41:29.145333+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.155496+0000","last_change":"2024-04-23T14:41:29.155496+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.155496+0000","last_undegraded":"2024-04-23T14:41:29.155496+0000","last_fullsized":"2024-04-23T14:41:29.155496+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.145896+0000","last_change":"2024-04-23T14:41:29.145896+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.145896+0000","last_undegraded":"2024-04-23T14:41:29.145896+0000","last_fullsized":"2024-04-23T14:41:29.145896+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.142641+0000","last_change":"2024-04-23T14:41:29.142641+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.142641+0000","last_undegraded":"2024-04-23T14:41:29.142641+0000","last_fullsized":"2024-04-23T14:41:29.142641+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.140502+0000","last_change":"2024-04-23T14:41:29.140502+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.140502+0000","last_undegraded":"2024-04-23T14:41:29.140502+0000","last_fullsized":"2024-04-23T14:41:29.140502+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.151156+0000","last_change":"2024-04-23T14:41:29.151156+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.151156+0000","last_undegraded":"2024-04-23T14:41:29.151156+0000","last_fullsized":"2024-04-23T14:41:29.151156+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.139861+0000","last_change":"2024-04-23T14:41:29.139861+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.139861+0000","last_undegraded":"2024-04-23T14:41:29.139861+0000","last_fullsized":"2024-04-23T14:41:29.139861+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.144072+0000","last_change":"2024-04-23T14:41:29.144072+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.144072+0000","last_undegraded":"2024-04-23T14:41:29.144072+0000","last_fullsized":"2024-04-23T14:41:29.144072+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.160258+0000","last_change":"2024-04-23T14:41:29.160258+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.160258+0000","last_undegraded":"2024-04-23T14:41:29.160258+0000","last_fullsized":"2024-04-23T14:41:29.160258+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.143935+0000","last_change":"2024-04-23T14:41:29.143935+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.143935+0000","last_undegraded":"2024-04-23T14:41:29.143935+0000","last_fullsized":"2024-04-23T14:41:29.143935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.141690+0000","last_change":"2024-04-23T14:41:29.141690+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.141690+0000","last_undegraded":"2024-04-23T14:41:29.141690+0000","last_fullsized":"2024-04-23T14:41:29.141690+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.146903+0000","last_change":"2024-04-23T14:41:29.146903+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.146903+0000","last_undegraded":"2024-04-23T14:41:29.146903+0000","last_fullsized":"2024-04-23T14:41:29.146903+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.146654+0000","last_change":"2024-04-23T14:41:29.146654+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.146654+0000","last_undegraded":"2024-04-23T14:41:29.146654+0000","last_fullsized":"2024-04-23T14:41:29.146654+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.139259+0000","last_change":"2024-04-23T14:41:29.139259+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.139259+0000","last_undegraded":"2024-04-23T14:41:29.139259+0000","last_fullsized":"2024-04-23T14:41:29.139259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.146837+0000","last_change":"2024-04-23T14:41:29.146837+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.146837+0000","last_undegraded":"2024-04-23T14:41:29.146837+0000","last_fullsized":"2024-04-23T14:41:29.146837+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.159336+0000","last_change":"2024-04-23T14:41:29.155206+0000","last_active":"2024-04-23T14:41:29.159336+0000","last_peered":"2024-04-23T14:41:29.159336+0000","last_clean":"2024-04-23T14:41:29.159336+0000","last_became_active":"2024-04-23T14:41:29.154492+0000","last_became_peered":"2024-04-23T14:41:29.154492+0000","last_unstale":"2024-04-23T14:41:29.159336+0000","last_undegraded":"2024-04-23T14:41:29.159336+0000","last_fullsized":"2024-04-23T14:41:29.159336+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:52:49.591648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.144585+0000","last_change":"2024-04-23T14:41:29.144585+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.144585+0000","last_undegraded":"2024-04-23T14:41:29.144585+0000","last_fullsized":"2024-04-23T14:41:29.144585+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.174886+0000","last_change":"2024-04-23T14:41:29.170454+0000","last_active":"2024-04-23T14:41:29.174886+0000","last_peered":"2024-04-23T14:41:29.174886+0000","last_clean":"2024-04-23T14:41:29.174886+0000","last_became_active":"2024-04-23T14:41:29.170006+0000","last_became_peered":"2024-04-23T14:41:29.170006+0000","last_unstale":"2024-04-23T14:41:29.174886+0000","last_undegraded":"2024-04-23T14:41:29.174886+0000","last_fullsized":"2024-04-23T14:41:29.174886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:41:27.119257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.138185+0000","last_change":"2024-04-23T14:41:29.138185+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.138185+0000","last_undegraded":"2024-04-23T14:41:29.138185+0000","last_fullsized":"2024-04-23T14:41:29.138185+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.160471+0000","last_change":"2024-04-23T14:41:29.156724+0000","last_active":"2024-04-23T14:41:29.160471+0000","last_peered":"2024-04-23T14:41:29.160471+0000","last_clean":"2024-04-23T14:41:29.160471+0000","last_became_active":"2024-04-23T14:41:29.156363+0000","last_became_peered":"2024-04-23T14:41:29.156363+0000","last_unstale":"2024-04-23T14:41:29.160471+0000","last_undegraded":"2024-04-23T14:41:29.160471+0000","last_fullsized":"2024-04-23T14:41:29.160471+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:41:27.119257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.153110+0000","last_change":"2024-04-23T14:41:29.153110+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.153110+0000","last_undegraded":"2024-04-23T14:41:29.153110+0000","last_fullsized":"2024-04-23T14:41:29.153110+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.167768+0000","last_change":"2024-04-23T14:41:29.166943+0000","last_active":"2024-04-23T14:41:29.167768+0000","last_peered":"2024-04-23T14:41:29.167768+0000","last_clean":"2024-04-23T14:41:29.167768+0000","last_became_active":"2024-04-23T14:41:29.166619+0000","last_became_peered":"2024-04-23T14:41:29.166619+0000","last_unstale":"2024-04-23T14:41:29.167768+0000","last_undegraded":"2024-04-23T14:41:29.167768+0000","last_fullsized":"2024-04-23T14:41:29.167768+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:44.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.156013+0000","last_change":"2024-04-23T14:41:29.156013+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.156013+0000","last_undegraded":"2024-04-23T14:41:29.156013+0000","last_fullsized":"2024-04-23T14:41:29.156013+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.153917+0000","last_change":"2024-04-23T14:41:29.150617+0000","last_active":"2024-04-23T14:41:29.153917+0000","last_peered":"2024-04-23T14:41:29.153917+0000","last_clean":"2024-04-23T14:41:29.153917+0000","last_became_active":"2024-04-23T14:41:29.150304+0000","last_became_peered":"2024-04-23T14:41:29.150304+0000","last_unstale":"2024-04-23T14:41:29.153917+0000","last_undegraded":"2024-04-23T14:41:29.153917+0000","last_fullsized":"2024-04-23T14:41:29.153917+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:33:35.898982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.152497+0000","last_change":"2024-04-23T14:41:29.152497+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.152497+0000","last_undegraded":"2024-04-23T14:41:29.152497+0000","last_fullsized":"2024-04-23T14:41:29.152497+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.153767+0000","last_change":"2024-04-23T14:41:29.148813+0000","last_active":"2024-04-23T14:41:29.153767+0000","last_peered":"2024-04-23T14:41:29.153767+0000","last_clean":"2024-04-23T14:41:29.153767+0000","last_became_active":"2024-04-23T14:41:29.148523+0000","last_became_peered":"2024-04-23T14:41:29.148523+0000","last_unstale":"2024-04-23T14:41:29.153767+0000","last_undegraded":"2024-04-23T14:41:29.153767+0000","last_fullsized":"2024-04-23T14:41:29.153767+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:47.301375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.144366+0000","last_change":"2024-04-23T14:41:29.144366+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.144366+0000","last_undegraded":"2024-04-23T14:41:29.144366+0000","last_fullsized":"2024-04-23T14:41:29.144366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.161507+0000","last_change":"2024-04-23T14:41:29.155513+0000","last_active":"2024-04-23T14:41:29.161507+0000","last_peered":"2024-04-23T14:41:29.161507+0000","last_clean":"2024-04-23T14:41:29.161507+0000","last_became_active":"2024-04-23T14:41:29.154886+0000","last_became_peered":"2024-04-23T14:41:29.154886+0000","last_unstale":"2024-04-23T14:41:29.161507+0000","last_undegraded":"2024-04-23T14:41:29.161507+0000","last_fullsized":"2024-04-23T14:41:29.161507+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:26:50.114366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.154053+0000","last_change":"2024-04-23T14:41:29.154053+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.154053+0000","last_undegraded":"2024-04-23T14:41:29.154053+0000","last_fullsized":"2024-04-23T14:41:29.154053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.160597+0000","last_change":"2024-04-23T14:41:29.158324+0000","last_active":"2024-04-23T14:41:29.160597+0000","last_peered":"2024-04-23T14:41:29.160597+0000","last_clean":"2024-04-23T14:41:29.160597+0000","last_became_active":"2024-04-23T14:41:29.158037+0000","last_became_peered":"2024-04-23T14:41:29.158037+0000","last_unstale":"2024-04-23T14:41:29.160597+0000","last_undegraded":"2024-04-23T14:41:29.160597+0000","last_fullsized":"2024-04-23T14:41:29.160597+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:30.433488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.140748+0000","last_change":"2024-04-23T14:41:29.140748+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.140748+0000","last_undegraded":"2024-04-23T14:41:29.140748+0000","last_fullsized":"2024-04-23T14:41:29.140748+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.164969+0000","last_change":"2024-04-23T14:41:29.161624+0000","last_active":"2024-04-23T14:41:29.164969+0000","last_peered":"2024-04-23T14:41:29.164969+0000","last_clean":"2024-04-23T14:41:29.164969+0000","last_became_active":"2024-04-23T14:41:29.161247+0000","last_became_peered":"2024-04-23T14:41:29.161247+0000","last_unstale":"2024-04-23T14:41:29.164969+0000","last_undegraded":"2024-04-23T14:41:29.164969+0000","last_fullsized":"2024-04-23T14:41:29.164969+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:31.321745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.155193+0000","last_change":"2024-04-23T14:41:29.155193+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.155193+0000","last_undegraded":"2024-04-23T14:41:29.155193+0000","last_fullsized":"2024-04-23T14:41:29.155193+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.156583+0000","last_change":"2024-04-23T14:41:29.155730+0000","last_active":"2024-04-23T14:41:29.156583+0000","last_peered":"2024-04-23T14:41:29.156583+0000","last_clean":"2024-04-23T14:41:29.156583+0000","last_became_active":"2024-04-23T14:41:29.155427+0000","last_became_peered":"2024-04-23T14:41:29.155427+0000","last_unstale":"2024-04-23T14:41:29.156583+0000","last_undegraded":"2024-04-23T14:41:29.156583+0000","last_fullsized":"2024-04-23T14:41:29.156583+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:20.729560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.142456+0000","last_change":"2024-04-23T14:41:29.142456+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.142456+0000","last_undegraded":"2024-04-23T14:41:29.142456+0000","last_fullsized":"2024-04-23T14:41:29.142456+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.174942+0000","last_change":"2024-04-23T14:41:29.171170+0000","last_active":"2024-04-23T14:41:29.174942+0000","last_peered":"2024-04-23T14:41:29.174942+0000","last_clean":"2024-04-23T14:41:29.174942+0000","last_became_active":"2024-04-23T14:41:29.170597+0000","last_became_peered":"2024-04-23T14:41:29.170597+0000","last_unstale":"2024-04-23T14:41:29.174942+0000","last_undegraded":"2024-04-23T14:41:29.174942+0000","last_fullsized":"2024-04-23T14:41:29.174942+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:18.481597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.158993+0000","last_change":"2024-04-23T14:41:29.158993+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.158993+0000","last_undegraded":"2024-04-23T14:41:29.158993+0000","last_fullsized":"2024-04-23T14:41:29.158993+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.172382+0000","last_change":"2024-04-23T14:41:29.168241+0000","last_active":"2024-04-23T14:41:29.172382+0000","last_peered":"2024-04-23T14:41:29.172382+0000","last_clean":"2024-04-23T14:41:29.172382+0000","last_became_active":"2024-04-23T14:41:29.167851+0000","last_became_peered":"2024-04-23T14:41:29.167851+0000","last_unstale":"2024-04-23T14:41:29.172382+0000","last_undegraded":"2024-04-23T14:41:29.172382+0000","last_fullsized":"2024-04-23T14:41:29.172382+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:58.813793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.146849+0000","last_change":"2024-04-23T14:41:29.146849+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.146849+0000","last_undegraded":"2024-04-23T14:41:29.146849+0000","last_fullsized":"2024-04-23T14:41:29.146849+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.172495+0000","last_change":"2024-04-23T14:41:29.169064+0000","last_active":"2024-04-23T14:41:29.172495+0000","last_peered":"2024-04-23T14:41:29.172495+0000","last_clean":"2024-04-23T14:41:29.172495+0000","last_became_active":"2024-04-23T14:41:29.168820+0000","last_became_peered":"2024-04-23T14:41:29.168820+0000","last_unstale":"2024-04-23T14:41:29.172495+0000","last_undegraded":"2024-04-23T14:41:29.172495+0000","last_fullsized":"2024-04-23T14:41:29.172495+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:41:59.282079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.138520+0000","last_change":"2024-04-23T14:41:29.138520+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.138520+0000","last_undegraded":"2024-04-23T14:41:29.138520+0000","last_fullsized":"2024-04-23T14:41:29.138520+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.155446+0000","last_change":"2024-04-23T14:41:29.152287+0000","last_active":"2024-04-23T14:41:29.155446+0000","last_peered":"2024-04-23T14:41:29.155446+0000","last_clean":"2024-04-23T14:41:29.155446+0000","last_became_active":"2024-04-23T14:41:29.150841+0000","last_became_peered":"2024-04-23T14:41:29.150841+0000","last_unstale":"2024-04-23T14:41:29.155446+0000","last_undegraded":"2024-04-23T14:41:29.155446+0000","last_fullsized":"2024-04-23T14:41:29.155446+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:44:58.410399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.149719+0000","last_change":"2024-04-23T14:41:29.149719+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.149719+0000","last_undegraded":"2024-04-23T14:41:29.149719+0000","last_fullsized":"2024-04-23T14:41:29.149719+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.169866+0000","last_change":"2024-04-23T14:41:29.167466+0000","last_active":"2024-04-23T14:41:29.169866+0000","last_peered":"2024-04-23T14:41:29.169866+0000","last_clean":"2024-04-23T14:41:29.169866+0000","last_became_active":"2024-04-23T14:41:29.167099+0000","last_became_peered":"2024-04-23T14:41:29.167099+0000","last_unstale":"2024-04-23T14:41:29.169866+0000","last_undegraded":"2024-04-23T14:41:29.169866+0000","last_fullsized":"2024-04-23T14:41:29.169866+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:16.245033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.140391+0000","last_change":"2024-04-23T14:41:29.140391+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.140391+0000","last_undegraded":"2024-04-23T14:41:29.140391+0000","last_fullsized":"2024-04-23T14:41:29.140391+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.164974+0000","last_change":"2024-04-23T14:41:29.161846+0000","last_active":"2024-04-23T14:41:29.164974+0000","last_peered":"2024-04-23T14:41:29.164974+0000","last_clean":"2024-04-23T14:41:29.164974+0000","last_became_active":"2024-04-23T14:41:29.161564+0000","last_became_peered":"2024-04-23T14:41:29.161564+0000","last_unstale":"2024-04-23T14:41:29.164974+0000","last_undegraded":"2024-04-23T14:41:29.164974+0000","last_fullsized":"2024-04-23T14:41:29.164974+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:56.847481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.138555+0000","last_change":"2024-04-23T14:41:29.138555+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.138555+0000","last_undegraded":"2024-04-23T14:41:29.138555+0000","last_fullsized":"2024-04-23T14:41:29.138555+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.172359+0000","last_change":"2024-04-23T14:41:29.167837+0000","last_active":"2024-04-23T14:41:29.172359+0000","last_peered":"2024-04-23T14:41:29.172359+0000","last_clean":"2024-04-23T14:41:29.172359+0000","last_became_active":"2024-04-23T14:41:29.166272+0000","last_became_peered":"2024-04-23T14:41:29.166272+0000","last_unstale":"2024-04-23T14:41:29.172359+0000","last_undegraded":"2024-04-23T14:41:29.172359+0000","last_fullsized":"2024-04-23T14:41:29.172359+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:23.943537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.156964+0000","last_change":"2024-04-23T14:41:29.156964+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.156964+0000","last_undegraded":"2024-04-23T14:41:29.156964+0000","last_fullsized":"2024-04-23T14:41:29.156964+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.168036+0000","last_change":"2024-04-23T14:41:29.162803+0000","last_active":"2024-04-23T14:41:29.168036+0000","last_peered":"2024-04-23T14:41:29.168036+0000","last_clean":"2024-04-23T14:41:29.168036+0000","last_became_active":"2024-04-23T14:41:29.162194+0000","last_became_peered":"2024-04-23T14:41:29.162194+0000","last_unstale":"2024-04-23T14:41:29.168036+0000","last_undegraded":"2024-04-23T14:41:29.168036+0000","last_fullsized":"2024-04-23T14:41:29.168036+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:47.301375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.155175+0000","last_change":"2024-04-23T14:41:29.155175+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.155175+0000","last_undegraded":"2024-04-23T14:41:29.155175+0000","last_fullsized":"2024-04-23T14:41:29.155175+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.165104+0000","last_change":"2024-04-23T14:41:29.163038+0000","last_active":"2024-04-23T14:41:29.165104+0000","last_peered":"2024-04-23T14:41:29.165104+0000","last_clean":"2024-04-23T14:41:29.165104+0000","last_became_active":"2024-04-23T14:41:29.162738+0000","last_became_peered":"2024-04-23T14:41:29.162738+0000","last_unstale":"2024-04-23T14:41:29.165104+0000","last_undegraded":"2024-04-23T14:41:29.165104+0000","last_fullsized":"2024-04-23T14:41:29.165104+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:55.871120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.146449+0000","last_change":"2024-04-23T14:41:29.146449+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.146449+0000","last_undegraded":"2024-04-23T14:41:29.146449+0000","last_fullsized":"2024-04-23T14:41:29.146449+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.160501+0000","last_change":"2024-04-23T14:41:29.156525+0000","last_active":"2024-04-23T14:41:29.160501+0000","last_peered":"2024-04-23T14:41:29.160501+0000","last_clean":"2024-04-23T14:41:29.160501+0000","last_became_active":"2024-04-23T14:41:29.156198+0000","last_became_peered":"2024-04-23T14:41:29.156198+0000","last_unstale":"2024-04-23T14:41:29.160501+0000","last_undegraded":"2024-04-23T14:41:29.160501+0000","last_fullsized":"2024-04-23T14:41:29.160501+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:22:15.255469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.144656+0000","last_change":"2024-04-23T14:41:29.144656+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.144656+0000","last_undegraded":"2024-04-23T14:41:29.144656+0000","last_fullsized":"2024-04-23T14:41:29.144656+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.171793+0000","last_change":"2024-04-23T14:41:29.163604+0000","last_active":"2024-04-23T14:41:29.171793+0000","last_peered":"2024-04-23T14:41:29.171793+0000","last_clean":"2024-04-23T14:41:29.171793+0000","last_became_active":"2024-04-23T14:41:29.163303+0000","last_became_peered":"2024-04-23T14:41:29.163303+0000","last_unstale":"2024-04-23T14:41:29.171793+0000","last_undegraded":"2024-04-23T14:41:29.171793+0000","last_fullsized":"2024-04-23T14:41:29.171793+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:26:50.114366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.140495+0000","last_change":"2024-04-23T14:41:29.140495+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.140495+0000","last_undegraded":"2024-04-23T14:41:29.140495+0000","last_fullsized":"2024-04-23T14:41:29.140495+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.171894+0000","last_change":"2024-04-23T14:41:29.165780+0000","last_active":"2024-04-23T14:41:29.171894+0000","last_peered":"2024-04-23T14:41:29.171894+0000","last_clean":"2024-04-23T14:41:29.171894+0000","last_became_active":"2024-04-23T14:41:29.165435+0000","last_became_peered":"2024-04-23T14:41:29.165435+0000","last_unstale":"2024-04-23T14:41:29.171894+0000","last_undegraded":"2024-04-23T14:41:29.171894+0000","last_fullsized":"2024-04-23T14:41:29.171894+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:03.123843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.141022+0000","last_change":"2024-04-23T14:41:29.141022+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.141022+0000","last_undegraded":"2024-04-23T14:41:29.141022+0000","last_fullsized":"2024-04-23T14:41:29.141022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.154584+0000","last_change":"2024-04-23T14:41:29.146434+0000","last_active":"2024-04-23T14:41:29.154584+0000","last_peered":"2024-04-23T14:41:29.154584+0000","last_clean":"2024-04-23T14:41:29.154584+0000","last_became_active":"2024-04-23T14:41:29.145530+0000","last_became_peered":"2024-04-23T14:41:29.145530+0000","last_unstale":"2024-04-23T14:41:29.154584+0000","last_undegraded":"2024-04-23T14:41:29.154584+0000","last_fullsized":"2024-04-23T14:41:29.154584+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:26.066571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.140081+0000","last_change":"2024-04-23T14:41:29.140081+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.140081+0000","last_undegraded":"2024-04-23T14:41:29.140081+0000","last_fullsized":"2024-04-23T14:41:29.140081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.172295+0000","last_change":"2024-04-23T14:41:29.169432+0000","last_active":"2024-04-23T14:41:29.172295+0000","last_peered":"2024-04-23T14:41:29.172295+0000","last_clean":"2024-04-23T14:41:29.172295+0000","last_became_active":"2024-04-23T14:41:29.168901+0000","last_became_peered":"2024-04-23T14:41:29.168901+0000","last_unstale":"2024-04-23T14:41:29.172295+0000","last_undegraded":"2024-04-23T14:41:29.172295+0000","last_fullsized":"2024-04-23T14:41:29.172295+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:26.066571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.137401+0000","last_change":"2024-04-23T14:41:29.137401+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.137401+0000","last_undegraded":"2024-04-23T14:41:29.137401+0000","last_fullsized":"2024-04-23T14:41:29.137401+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.160588+0000","last_change":"2024-04-23T14:41:29.157298+0000","last_active":"2024-04-23T14:41:29.160588+0000","last_peered":"2024-04-23T14:41:29.160588+0000","last_clean":"2024-04-23T14:41:29.160588+0000","last_became_active":"2024-04-23T14:41:29.156972+0000","last_became_peered":"2024-04-23T14:41:29.156972+0000","last_unstale":"2024-04-23T14:41:29.160588+0000","last_undegraded":"2024-04-23T14:41:29.160588+0000","last_fullsized":"2024-04-23T14:41:29.160588+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:51.929518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.137671+0000","last_change":"2024-04-23T14:41:29.137671+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.137671+0000","last_undegraded":"2024-04-23T14:41:29.137671+0000","last_fullsized":"2024-04-23T14:41:29.137671+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.165103+0000","last_change":"2024-04-23T14:41:29.162997+0000","last_active":"2024-04-23T14:41:29.165103+0000","last_peered":"2024-04-23T14:41:29.165103+0000","last_clean":"2024-04-23T14:41:29.165103+0000","last_became_active":"2024-04-23T14:41:29.162673+0000","last_became_peered":"2024-04-23T14:41:29.162673+0000","last_unstale":"2024-04-23T14:41:29.165103+0000","last_undegraded":"2024-04-23T14:41:29.165103+0000","last_fullsized":"2024-04-23T14:41:29.165103+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:18.481597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.140088+0000","last_change":"2024-04-23T14:41:29.140088+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.140088+0000","last_undegraded":"2024-04-23T14:41:29.140088+0000","last_fullsized":"2024-04-23T14:41:29.140088+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.167802+0000","last_change":"2024-04-23T14:41:29.162589+0000","last_active":"2024-04-23T14:41:29.167802+0000","last_peered":"2024-04-23T14:41:29.167802+0000","last_clean":"2024-04-23T14:41:29.167802+0000","last_became_active":"2024-04-23T14:41:29.162046+0000","last_became_peered":"2024-04-23T14:41:29.162046+0000","last_unstale":"2024-04-23T14:41:29.167802+0000","last_undegraded":"2024-04-23T14:41:29.167802+0000","last_fullsized":"2024-04-23T14:41:29.167802+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:26.066571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.141037+0000","last_change":"2024-04-23T14:41:29.141037+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.141037+0000","last_undegraded":"2024-04-23T14:41:29.141037+0000","last_fullsized":"2024-04-23T14:41:29.141037+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.160341+0000","last_change":"2024-04-23T14:41:29.154132+0000","last_active":"2024-04-23T14:41:29.160341+0000","last_peered":"2024-04-23T14:41:29.160341+0000","last_clean":"2024-04-23T14:41:29.160341+0000","last_became_active":"2024-04-23T14:41:29.152404+0000","last_became_peered":"2024-04-23T14:41:29.152404+0000","last_unstale":"2024-04-23T14:41:29.160341+0000","last_undegraded":"2024-04-23T14:41:29.160341+0000","last_fullsized":"2024-04-23T14:41:29.160341+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:19.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.150986+0000","last_change":"2024-04-23T14:41:29.150986+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.150986+0000","last_undegraded":"2024-04-23T14:41:29.150986+0000","last_fullsized":"2024-04-23T14:41:29.150986+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.161738+0000","last_change":"2024-04-23T14:41:29.157151+0000","last_active":"2024-04-23T14:41:29.161738+0000","last_peered":"2024-04-23T14:41:29.161738+0000","last_clean":"2024-04-23T14:41:29.161738+0000","last_became_active":"2024-04-23T14:41:29.155019+0000","last_became_peered":"2024-04-23T14:41:29.155019+0000","last_unstale":"2024-04-23T14:41:29.161738+0000","last_undegraded":"2024-04-23T14:41:29.161738+0000","last_fullsized":"2024-04-23T14:41:29.161738+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:06:03.123843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.139905+0000","last_change":"2024-04-23T14:41:29.139905+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.139905+0000","last_undegraded":"2024-04-23T14:41:29.139905+0000","last_fullsized":"2024-04-23T14:41:29.139905+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.155628+0000","last_change":"2024-04-23T14:41:29.152918+0000","last_active":"2024-04-23T14:41:29.155628+0000","last_peered":"2024-04-23T14:41:29.155628+0000","last_clean":"2024-04-23T14:41:29.155628+0000","last_became_active":"2024-04-23T14:41:29.152661+0000","last_became_peered":"2024-04-23T14:41:29.152661+0000","last_unstale":"2024-04-23T14:41:29.155628+0000","last_undegraded":"2024-04-23T14:41:29.155628+0000","last_fullsized":"2024-04-23T14:41:29.155628+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:22.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.138657+0000","last_change":"2024-04-23T14:41:29.138657+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.138657+0000","last_undegraded":"2024-04-23T14:41:29.138657+0000","last_fullsized":"2024-04-23T14:41:29.138657+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.155962+0000","last_change":"2024-04-23T14:41:29.154113+0000","last_active":"2024-04-23T14:41:29.155962+0000","last_peered":"2024-04-23T14:41:29.155962+0000","last_clean":"2024-04-23T14:41:29.155962+0000","last_became_active":"2024-04-23T14:41:29.153847+0000","last_became_peered":"2024-04-23T14:41:29.153847+0000","last_unstale":"2024-04-23T14:41:29.155962+0000","last_undegraded":"2024-04-23T14:41:29.155962+0000","last_fullsized":"2024-04-23T14:41:29.155962+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:23:00.054583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":97,"state":"creating+peering","last_fresh":"2024-04-23T14:41:29.146413+0000","last_change":"2024-04-23T14:41:29.146413+0000","last_active":"2024-04-23T14:41:29.112596+0000","last_peered":"2024-04-23T14:41:29.112596+0000","last_clean":"2024-04-23T14:41:29.112596+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-23T14:41:29.146413+0000","last_undegraded":"2024-04-23T14:41:29.146413+0000","last_fullsized":"2024-04-23T14:41:29.146413+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.174892+0000","last_change":"2024-04-23T14:41:29.170521+0000","last_active":"2024-04-23T14:41:29.174892+0000","last_peered":"2024-04-23T14:41:29.174892+0000","last_clean":"2024-04-23T14:41:29.174892+0000","last_became_active":"2024-04-23T14:41:29.170150+0000","last_became_peered":"2024-04-23T14:41:29.170150+0000","last_unstale":"2024-04-23T14:41:29.174892+0000","last_undegraded":"2024-04-23T14:41:29.174892+0000","last_fullsized":"2024-04-23T14:41:29.174892+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:51.929518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":313,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-23T14:41:29.164025+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:41:29.164025+0000","last_peered":"2024-04-23T14:41:29.164025+0000","last_clean":"2024-04-23T14:41:29.164025+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:41:29.164025+0000","last_undegraded":"2024-04-23T14:41:29.164025+0000","last_fullsized":"2024-04-23T14:41:29.164025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":1107271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":510628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":510628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":510628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":516903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1200128,"data_stored":516903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":516903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1781760,"data_stored":1100996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:41:34.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:41:34.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.849+0000 7fc006c00640 1 Processor -- start 2024-04-23T14:41:34.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.850+0000 7fc006c00640 1 -- start start 2024-04-23T14:41:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc006c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc00015c3d0 0x7fc0000cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc006c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc0000cd590 0x7fc0000b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc006c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc000160210 con 0x7fc0000b4690 2024-04-23T14:41:34.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc006c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc0001604f0 con 0x7fc0000cd590 2024-04-23T14:41:34.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc006c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc0001607d0 con 0x7fc00015c3d0 2024-04-23T14:41:34.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc006200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc0000b4690 0x7fc00006b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55644/0 (socket says 172.21.15.59:55644) 2024-04-23T14:41:34.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc006200640 1 -- 172.21.15.59:0/647098528 learned_addr learned my addr 172.21.15.59:0/647098528 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:34.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.851+0000 7fc004e00640 1 --2- 172.21.15.59:0/647098528 >> v2:172.21.15.59:3300/0 conn(0x7fc0000cd590 0x7fc0000b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.852+0000 7fc005800640 1 --2- 172.21.15.59:0/647098528 >> v2:172.21.15.76:3300/0 conn(0x7fc00015c3d0 0x7fc0000cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.852+0000 7fc004e00640 1 -- 172.21.15.59:0/647098528 >> v2:172.21.15.76:3300/0 conn(0x7fc00015c3d0 msgr2=0x7fc0000cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.852+0000 7fc004e00640 1 --2- 172.21.15.59:0/647098528 >> v2:172.21.15.76:3300/0 conn(0x7fc00015c3d0 0x7fc0000cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.852+0000 7fc004e00640 1 -- 172.21.15.59:0/647098528 >> v1:172.21.15.59:6789/0 conn(0x7fc0000b4690 legacy=0x7fc00006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.852+0000 7fc004e00640 1 -- 172.21.15.59:0/647098528 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc000160ab0 con 0x7fc0000cd590 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.852+0000 7fbfffe00640 1 -- 172.21.15.59:0/647098528 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 406977830 0 0) 0x7fc000160210 con 0x7fc0000b4690 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.852+0000 7fc005800640 1 --2- 172.21.15.59:0/647098528 >> v2:172.21.15.76:3300/0 conn(0x7fc00015c3d0 0x7fc0000cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:34.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.853+0000 7fc004e00640 1 --2- 172.21.15.59:0/647098528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc0000b4150 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc00006d9b0 tx=0x7fbfec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3927941dc9dfc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fbfffe00640 1 -- 172.21.15.59:0/647098528 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec037070 con 0x7fc0000cd590 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fbfffe00640 1 -- 172.21.15.59:0/647098528 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec032440 con 0x7fc0000cd590 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fbfffe00640 1 -- 172.21.15.59:0/647098528 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec032810 con 0x7fc0000cd590 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 -- 172.21.15.59:0/647098528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 msgr2=0x7fc0000b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 --2- 172.21.15.59:0/647098528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc0000b4150 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc00006d9b0 tx=0x7fbfec02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 -- 172.21.15.59:0/647098528 shutdown_connections 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 --2- 172.21.15.59:0/647098528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc0000b4150 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 --2- 172.21.15.59:0/647098528 >> v2:172.21.15.76:3300/0 conn(0x7fc00015c3d0 0x7fc0000cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 -- 172.21.15.59:0/647098528 >> 172.21.15.59:0/647098528 conn(0x7fc0000b21a0 msgr2=0x7fc0000b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 -- 172.21.15.59:0/647098528 shutdown_connections 2024-04-23T14:41:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.854+0000 7fc006c00640 1 -- 172.21.15.59:0/647098528 wait complete. 2024-04-23T14:41:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.856+0000 7fc006c00640 1 Processor -- start 2024-04-23T14:41:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.856+0000 7fc006c00640 1 -- start start 2024-04-23T14:41:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc006c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0000b4690 0x7fc0000c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc006c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc00015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc006c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc00015c3d0 0x7fc0000af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc006c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc00015c7b0 con 0x7fc0000cd590 2024-04-23T14:41:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc006c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc0000b4ac0 con 0x7fc0000b4690 2024-04-23T14:41:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc006c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc00015c930 con 0x7fc00015c3d0 2024-04-23T14:41:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc004e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc00015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc004e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc00015cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59714/0 (socket says 172.21.15.59:59714) 2024-04-23T14:41:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.857+0000 7fc004e00640 1 -- 172.21.15.59:0/2068767896 learned_addr learned my addr 172.21.15.59:0/2068767896 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc005800640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0000b4690 0x7fc0000c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc004e00640 1 -- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc00015c3d0 msgr2=0x7fc0000af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc006200640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc00015c3d0 0x7fc0000af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc004e00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc00015c3d0 0x7fc0000af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc004e00640 1 -- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0000b4690 msgr2=0x7fc0000c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc004e00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0000b4690 0x7fc0000c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc004e00640 1 -- 172.21.15.59:0/2068767896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0000c73b0 con 0x7fc0000cd590 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc006200640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc00015c3d0 0x7fc0000af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.858+0000 7fc004e00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc00015cbf0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.859+0000 7fbffd600640 1 -- 172.21.15.59:0/2068767896 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec037050 con 0x7fc0000cd590 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.859+0000 7fbffd600640 1 -- 172.21.15.59:0/2068767896 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec038040 con 0x7fc0000cd590 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.859+0000 7fbffd600640 1 -- 172.21.15.59:0/2068767896 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec004450 con 0x7fc0000cd590 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.859+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0000c7690 con 0x7fc0000cd590 2024-04-23T14:41:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.859+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0000c7c50 con 0x7fc0000cd590 2024-04-23T14:41:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.860+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00004fd90 con 0x7fc0000cd590 2024-04-23T14:41:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.861+0000 7fbffd600640 1 -- 172.21.15.59:0/2068767896 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfec030980 con 0x7fc0000cd590 2024-04-23T14:41:34.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.862+0000 7fbffd600640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfe006c920 0x7fbfe006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:34.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.862+0000 7fbffd600640 1 -- 172.21.15.59:0/2068767896 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fbfec0ac800 con 0x7fc0000cd590 2024-04-23T14:41:34.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.862+0000 7fc005800640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfe006c920 0x7fbfe006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:34.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.863+0000 7fc005800640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfe006c920 0x7fbfe006ede0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbff400b840 tx=0x7fbff4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:34.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:34.868+0000 7fbffd600640 1 -- 172.21.15.59:0/2068767896 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfec07ad40 con 0x7fc0000cd590 2024-04-23T14:41:35.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.068+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc00006e400 con 0x7fbfe006c920 2024-04-23T14:41:35.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.088+0000 7fbffd600640 1 -- 172.21.15.59:0/2068767896 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252086 (secure 0 0 0) 0x7fc00006e400 con 0x7fbfe006c920 2024-04-23T14:41:35.092 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:35.093 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.093+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfe006c920 msgr2=0x7fbfe006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.093+0000 7fc006c00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfe006c920 0x7fbfe006ede0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbff400b840 tx=0x7fbff4008040 comp rx=0 tx=0).stop 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.093+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 msgr2=0x7fc00015cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.093+0000 7fc006c00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc00015cbf0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec0046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 shutdown_connections 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbfe006c920 0x7fbfe006ede0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc00015c3d0 0x7fc0000af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0000cd590 0x7fc00015cbf0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 --2- 172.21.15.59:0/2068767896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0000b4690 0x7fc0000c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 >> 172.21.15.59:0/2068767896 conn(0x7fc0000b21a0 msgr2=0x7fc00006f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:35.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 shutdown_connections 2024-04-23T14:41:35.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.094+0000 7fc006c00640 1 -- 172.21.15.59:0/2068767896 wait complete. 2024-04-23T14:41:35.114 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-04-23T14:41:34.570265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260284,"kb_used_data":10792,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714436,"statfs":{"total":773094113280,"available":772827582464,"internally_reserved":0,"allocated":11051008,"data_stored":5425380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.012328"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.220436+0000","last_change":"2024-04-23T14:41:30.177793+0000","last_active":"2024-04-23T14:41:31.220436+0000","last_peered":"2024-04-23T14:41:31.220436+0000","last_clean":"2024-04-23T14:41:31.220436+0000","last_became_active":"2024-04-23T14:41:30.177486+0000","last_became_peered":"2024-04-23T14:41:30.177486+0000","last_unstale":"2024-04-23T14:41:31.220436+0000","last_undegraded":"2024-04-23T14:41:31.220436+0000","last_fullsized":"2024-04-23T14:41:31.220436+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:16.870565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177055+0000","last_change":"2024-04-23T14:41:30.176710+0000","last_active":"2024-04-23T14:41:31.177055+0000","last_peered":"2024-04-23T14:41:31.177055+0000","last_clean":"2024-04-23T14:41:31.177055+0000","last_became_active":"2024-04-23T14:41:30.175933+0000","last_became_peered":"2024-04-23T14:41:30.175933+0000","last_unstale":"2024-04-23T14:41:31.177055+0000","last_undegraded":"2024-04-23T14:41:31.177055+0000","last_fullsized":"2024-04-23T14:41:31.177055+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:16.734942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.220203+0000","last_change":"2024-04-23T14:41:30.176043+0000","last_active":"2024-04-23T14:41:31.220203+0000","last_peered":"2024-04-23T14:41:31.220203+0000","last_clean":"2024-04-23T14:41:31.220203+0000","last_became_active":"2024-04-23T14:41:30.175694+0000","last_became_peered":"2024-04-23T14:41:30.175694+0000","last_unstale":"2024-04-23T14:41:31.220203+0000","last_undegraded":"2024-04-23T14:41:31.220203+0000","last_fullsized":"2024-04-23T14:41:31.220203+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:12.158319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173702+0000","last_change":"2024-04-23T14:41:30.166277+0000","last_active":"2024-04-23T14:41:31.173702+0000","last_peered":"2024-04-23T14:41:31.173702+0000","last_clean":"2024-04-23T14:41:31.173702+0000","last_became_active":"2024-04-23T14:41:30.165900+0000","last_became_peered":"2024-04-23T14:41:30.165900+0000","last_unstale":"2024-04-23T14:41:31.173702+0000","last_undegraded":"2024-04-23T14:41:31.173702+0000","last_fullsized":"2024-04-23T14:41:31.173702+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:03.894814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177385+0000","last_change":"2024-04-23T14:41:30.164564+0000","last_active":"2024-04-23T14:41:31.177385+0000","last_peered":"2024-04-23T14:41:31.177385+0000","last_clean":"2024-04-23T14:41:31.177385+0000","last_became_active":"2024-04-23T14:41:30.164139+0000","last_became_peered":"2024-04-23T14:41:30.164139+0000","last_unstale":"2024-04-23T14:41:31.177385+0000","last_undegraded":"2024-04-23T14:41:31.177385+0000","last_fullsized":"2024-04-23T14:41:31.177385+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:23.933577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221655+0000","last_change":"2024-04-23T14:41:30.167898+0000","last_active":"2024-04-23T14:41:31.221655+0000","last_peered":"2024-04-23T14:41:31.221655+0000","last_clean":"2024-04-23T14:41:31.221655+0000","last_became_active":"2024-04-23T14:41:30.167612+0000","last_became_peered":"2024-04-23T14:41:30.167612+0000","last_unstale":"2024-04-23T14:41:31.221655+0000","last_undegraded":"2024-04-23T14:41:31.221655+0000","last_fullsized":"2024-04-23T14:41:31.221655+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.413980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221163+0000","last_change":"2024-04-23T14:41:30.169962+0000","last_active":"2024-04-23T14:41:31.221163+0000","last_peered":"2024-04-23T14:41:31.221163+0000","last_clean":"2024-04-23T14:41:31.221163+0000","last_became_active":"2024-04-23T14:41:30.169406+0000","last_became_peered":"2024-04-23T14:41:30.169406+0000","last_unstale":"2024-04-23T14:41:31.221163+0000","last_undegraded":"2024-04-23T14:41:31.221163+0000","last_fullsized":"2024-04-23T14:41:31.221163+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:14.321147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177260+0000","last_change":"2024-04-23T14:41:30.167524+0000","last_active":"2024-04-23T14:41:31.177260+0000","last_peered":"2024-04-23T14:41:31.177260+0000","last_clean":"2024-04-23T14:41:31.177260+0000","last_became_active":"2024-04-23T14:41:30.167150+0000","last_became_peered":"2024-04-23T14:41:30.167150+0000","last_unstale":"2024-04-23T14:41:31.177260+0000","last_undegraded":"2024-04-23T14:41:31.177260+0000","last_fullsized":"2024-04-23T14:41:31.177260+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:03.894814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223529+0000","last_change":"2024-04-23T14:41:30.152971+0000","last_active":"2024-04-23T14:41:31.223529+0000","last_peered":"2024-04-23T14:41:31.223529+0000","last_clean":"2024-04-23T14:41:31.223529+0000","last_became_active":"2024-04-23T14:41:30.152053+0000","last_became_peered":"2024-04-23T14:41:30.152053+0000","last_unstale":"2024-04-23T14:41:31.223529+0000","last_undegraded":"2024-04-23T14:41:31.223529+0000","last_fullsized":"2024-04-23T14:41:31.223529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:45.034433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177390+0000","last_change":"2024-04-23T14:41:30.178264+0000","last_active":"2024-04-23T14:41:31.177390+0000","last_peered":"2024-04-23T14:41:31.177390+0000","last_clean":"2024-04-23T14:41:31.177390+0000","last_became_active":"2024-04-23T14:41:30.177669+0000","last_became_peered":"2024-04-23T14:41:30.177669+0000","last_unstale":"2024-04-23T14:41:31.177390+0000","last_undegraded":"2024-04-23T14:41:31.177390+0000","last_fullsized":"2024-04-23T14:41:31.177390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:21.746538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.222247+0000","last_change":"2024-04-23T14:41:30.172427+0000","last_active":"2024-04-23T14:41:31.222247+0000","last_peered":"2024-04-23T14:41:31.222247+0000","last_clean":"2024-04-23T14:41:31.222247+0000","last_became_active":"2024-04-23T14:41:30.172009+0000","last_became_peered":"2024-04-23T14:41:30.172009+0000","last_unstale":"2024-04-23T14:41:31.222247+0000","last_undegraded":"2024-04-23T14:41:31.222247+0000","last_fullsized":"2024-04-23T14:41:31.222247+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:25:35.510457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223516+0000","last_change":"2024-04-23T14:41:30.175330+0000","last_active":"2024-04-23T14:41:31.223516+0000","last_peered":"2024-04-23T14:41:31.223516+0000","last_clean":"2024-04-23T14:41:31.223516+0000","last_became_active":"2024-04-23T14:41:30.173739+0000","last_became_peered":"2024-04-23T14:41:30.173739+0000","last_unstale":"2024-04-23T14:41:31.223516+0000","last_undegraded":"2024-04-23T14:41:31.223516+0000","last_fullsized":"2024-04-23T14:41:31.223516+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:24.427255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173847+0000","last_change":"2024-04-23T14:41:30.154528+0000","last_active":"2024-04-23T14:41:31.173847+0000","last_peered":"2024-04-23T14:41:31.173847+0000","last_clean":"2024-04-23T14:41:31.173847+0000","last_became_active":"2024-04-23T14:41:30.153902+0000","last_became_peered":"2024-04-23T14:41:30.153902+0000","last_unstale":"2024-04-23T14:41:31.173847+0000","last_undegraded":"2024-04-23T14:41:31.173847+0000","last_fullsized":"2024-04-23T14:41:31.173847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:10:51.098237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.171979+0000","last_change":"2024-04-23T14:41:30.172390+0000","last_active":"2024-04-23T14:41:31.171979+0000","last_peered":"2024-04-23T14:41:31.171979+0000","last_clean":"2024-04-23T14:41:31.171979+0000","last_became_active":"2024-04-23T14:41:30.172074+0000","last_became_peered":"2024-04-23T14:41:30.172074+0000","last_unstale":"2024-04-23T14:41:31.171979+0000","last_undegraded":"2024-04-23T14:41:31.171979+0000","last_fullsized":"2024-04-23T14:41:31.171979+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:33:36.915960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173315+0000","last_change":"2024-04-23T14:41:30.179844+0000","last_active":"2024-04-23T14:41:31.173315+0000","last_peered":"2024-04-23T14:41:31.173315+0000","last_clean":"2024-04-23T14:41:31.173315+0000","last_became_active":"2024-04-23T14:41:30.179552+0000","last_became_peered":"2024-04-23T14:41:30.179552+0000","last_unstale":"2024-04-23T14:41:31.173315+0000","last_undegraded":"2024-04-23T14:41:31.173315+0000","last_fullsized":"2024-04-23T14:41:31.173315+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:16.734942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221052+0000","last_change":"2024-04-23T14:41:30.167196+0000","last_active":"2024-04-23T14:41:31.221052+0000","last_peered":"2024-04-23T14:41:31.221052+0000","last_clean":"2024-04-23T14:41:31.221052+0000","last_became_active":"2024-04-23T14:41:30.166863+0000","last_became_peered":"2024-04-23T14:41:30.166863+0000","last_unstale":"2024-04-23T14:41:31.221052+0000","last_undegraded":"2024-04-23T14:41:31.221052+0000","last_fullsized":"2024-04-23T14:41:31.221052+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:21.746538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177012+0000","last_change":"2024-04-23T14:41:30.175187+0000","last_active":"2024-04-23T14:41:31.177012+0000","last_peered":"2024-04-23T14:41:31.177012+0000","last_clean":"2024-04-23T14:41:31.177012+0000","last_became_active":"2024-04-23T14:41:30.174786+0000","last_became_peered":"2024-04-23T14:41:30.174786+0000","last_unstale":"2024-04-23T14:41:31.177012+0000","last_undegraded":"2024-04-23T14:41:31.177012+0000","last_fullsized":"2024-04-23T14:41:31.177012+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:21:05.768325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173456+0000","last_change":"2024-04-23T14:41:30.174328+0000","last_active":"2024-04-23T14:41:31.173456+0000","last_peered":"2024-04-23T14:41:31.173456+0000","last_clean":"2024-04-23T14:41:31.173456+0000","last_became_active":"2024-04-23T14:41:30.171298+0000","last_became_peered":"2024-04-23T14:41:30.171298+0000","last_unstale":"2024-04-23T14:41:31.173456+0000","last_undegraded":"2024-04-23T14:41:31.173456+0000","last_fullsized":"2024-04-23T14:41:31.173456+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:00:42.385292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172775+0000","last_change":"2024-04-23T14:41:30.171418+0000","last_active":"2024-04-23T14:41:31.172775+0000","last_peered":"2024-04-23T14:41:31.172775+0000","last_clean":"2024-04-23T14:41:31.172775+0000","last_became_active":"2024-04-23T14:41:30.171103+0000","last_became_peered":"2024-04-23T14:41:30.171103+0000","last_unstale":"2024-04-23T14:41:31.172775+0000","last_undegraded":"2024-04-23T14:41:31.172775+0000","last_fullsized":"2024-04-23T14:41:31.172775+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:24.860540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223773+0000","last_change":"2024-04-23T14:41:30.151707+0000","last_active":"2024-04-23T14:41:31.223773+0000","last_peered":"2024-04-23T14:41:31.223773+0000","last_clean":"2024-04-23T14:41:31.223773+0000","last_became_active":"2024-04-23T14:41:30.151021+0000","last_became_peered":"2024-04-23T14:41:30.151021+0000","last_unstale":"2024-04-23T14:41:31.223773+0000","last_undegraded":"2024-04-23T14:41:31.223773+0000","last_fullsized":"2024-04-23T14:41:31.223773+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:40.782377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172433+0000","last_change":"2024-04-23T14:41:30.170062+0000","last_active":"2024-04-23T14:41:31.172433+0000","last_peered":"2024-04-23T14:41:31.172433+0000","last_clean":"2024-04-23T14:41:31.172433+0000","last_became_active":"2024-04-23T14:41:30.169737+0000","last_became_peered":"2024-04-23T14:41:30.169737+0000","last_unstale":"2024-04-23T14:41:31.172433+0000","last_undegraded":"2024-04-23T14:41:31.172433+0000","last_fullsized":"2024-04-23T14:41:31.172433+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:34.041256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176722+0000","last_change":"2024-04-23T14:41:30.182978+0000","last_active":"2024-04-23T14:41:31.176722+0000","last_peered":"2024-04-23T14:41:31.176722+0000","last_clean":"2024-04-23T14:41:31.176722+0000","last_became_active":"2024-04-23T14:41:30.180841+0000","last_became_peered":"2024-04-23T14:41:30.180841+0000","last_unstale":"2024-04-23T14:41:31.176722+0000","last_undegraded":"2024-04-23T14:41:31.176722+0000","last_fullsized":"2024-04-23T14:41:31.176722+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:52:52.141378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176659+0000","last_change":"2024-04-23T14:41:30.178398+0000","last_active":"2024-04-23T14:41:31.176659+0000","last_peered":"2024-04-23T14:41:31.176659+0000","last_clean":"2024-04-23T14:41:31.176659+0000","last_became_active":"2024-04-23T14:41:30.178019+0000","last_became_peered":"2024-04-23T14:41:30.178019+0000","last_unstale":"2024-04-23T14:41:31.176659+0000","last_undegraded":"2024-04-23T14:41:31.176659+0000","last_fullsized":"2024-04-23T14:41:31.176659+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:10:57.529185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221311+0000","last_change":"2024-04-23T14:41:30.178106+0000","last_active":"2024-04-23T14:41:31.221311+0000","last_peered":"2024-04-23T14:41:31.221311+0000","last_clean":"2024-04-23T14:41:31.221311+0000","last_became_active":"2024-04-23T14:41:30.177750+0000","last_became_peered":"2024-04-23T14:41:30.177750+0000","last_unstale":"2024-04-23T14:41:31.221311+0000","last_undegraded":"2024-04-23T14:41:31.221311+0000","last_fullsized":"2024-04-23T14:41:31.221311+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:52.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177963+0000","last_change":"2024-04-23T14:41:30.168904+0000","last_active":"2024-04-23T14:41:31.177963+0000","last_peered":"2024-04-23T14:41:31.177963+0000","last_clean":"2024-04-23T14:41:31.177963+0000","last_became_active":"2024-04-23T14:41:30.168372+0000","last_became_peered":"2024-04-23T14:41:30.168372+0000","last_unstale":"2024-04-23T14:41:31.177963+0000","last_undegraded":"2024-04-23T14:41:31.177963+0000","last_fullsized":"2024-04-23T14:41:31.177963+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:52.464734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177189+0000","last_change":"2024-04-23T14:41:30.179804+0000","last_active":"2024-04-23T14:41:31.177189+0000","last_peered":"2024-04-23T14:41:31.177189+0000","last_clean":"2024-04-23T14:41:31.177189+0000","last_became_active":"2024-04-23T14:41:30.179253+0000","last_became_peered":"2024-04-23T14:41:30.179253+0000","last_unstale":"2024-04-23T14:41:31.177189+0000","last_undegraded":"2024-04-23T14:41:31.177189+0000","last_fullsized":"2024-04-23T14:41:31.177189+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:32:27.264356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221997+0000","last_change":"2024-04-23T14:41:30.165107+0000","last_active":"2024-04-23T14:41:31.221997+0000","last_peered":"2024-04-23T14:41:31.221997+0000","last_clean":"2024-04-23T14:41:31.221997+0000","last_became_active":"2024-04-23T14:41:30.164733+0000","last_became_peered":"2024-04-23T14:41:30.164733+0000","last_unstale":"2024-04-23T14:41:31.221997+0000","last_undegraded":"2024-04-23T14:41:31.221997+0000","last_fullsized":"2024-04-23T14:41:31.221997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:20.039424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176912+0000","last_change":"2024-04-23T14:41:30.160370+0000","last_active":"2024-04-23T14:41:31.176912+0000","last_peered":"2024-04-23T14:41:31.176912+0000","last_clean":"2024-04-23T14:41:31.176912+0000","last_became_active":"2024-04-23T14:41:30.159856+0000","last_became_peered":"2024-04-23T14:41:30.159856+0000","last_unstale":"2024-04-23T14:41:31.176912+0000","last_undegraded":"2024-04-23T14:41:31.176912+0000","last_fullsized":"2024-04-23T14:41:31.176912+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:26.171712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177414+0000","last_change":"2024-04-23T14:41:30.169177+0000","last_active":"2024-04-23T14:41:31.177414+0000","last_peered":"2024-04-23T14:41:31.177414+0000","last_clean":"2024-04-23T14:41:31.177414+0000","last_became_active":"2024-04-23T14:41:30.168897+0000","last_became_peered":"2024-04-23T14:41:30.168897+0000","last_unstale":"2024-04-23T14:41:31.177414+0000","last_undegraded":"2024-04-23T14:41:31.177414+0000","last_fullsized":"2024-04-23T14:41:31.177414+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:10:51.098237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172053+0000","last_change":"2024-04-23T14:41:30.170083+0000","last_active":"2024-04-23T14:41:31.172053+0000","last_peered":"2024-04-23T14:41:31.172053+0000","last_clean":"2024-04-23T14:41:31.172053+0000","last_became_active":"2024-04-23T14:41:30.169755+0000","last_became_peered":"2024-04-23T14:41:30.169755+0000","last_unstale":"2024-04-23T14:41:31.172053+0000","last_undegraded":"2024-04-23T14:41:31.172053+0000","last_fullsized":"2024-04-23T14:41:31.172053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:31.450466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173457+0000","last_change":"2024-04-23T14:41:30.158265+0000","last_active":"2024-04-23T14:41:31.173457+0000","last_peered":"2024-04-23T14:41:31.173457+0000","last_clean":"2024-04-23T14:41:31.173457+0000","last_became_active":"2024-04-23T14:41:30.157902+0000","last_became_peered":"2024-04-23T14:41:30.157902+0000","last_unstale":"2024-04-23T14:41:31.173457+0000","last_undegraded":"2024-04-23T14:41:31.173457+0000","last_fullsized":"2024-04-23T14:41:31.173457+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:00:43.361653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.219540+0000","last_change":"2024-04-23T14:41:30.186257+0000","last_active":"2024-04-23T14:41:31.219540+0000","last_peered":"2024-04-23T14:41:31.219540+0000","last_clean":"2024-04-23T14:41:31.219540+0000","last_became_active":"2024-04-23T14:41:30.185889+0000","last_became_peered":"2024-04-23T14:41:30.185889+0000","last_unstale":"2024-04-23T14:41:31.219540+0000","last_undegraded":"2024-04-23T14:41:31.219540+0000","last_fullsized":"2024-04-23T14:41:31.219540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:03:24.960515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172894+0000","last_change":"2024-04-23T14:41:30.171117+0000","last_active":"2024-04-23T14:41:31.172894+0000","last_peered":"2024-04-23T14:41:31.172894+0000","last_clean":"2024-04-23T14:41:31.172894+0000","last_became_active":"2024-04-23T14:41:30.170805+0000","last_became_peered":"2024-04-23T14:41:30.170805+0000","last_unstale":"2024-04-23T14:41:31.172894+0000","last_undegraded":"2024-04-23T14:41:31.172894+0000","last_fullsized":"2024-04-23T14:41:31.172894+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:02:29.898259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"98'4","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177047+0000","last_change":"2024-04-23T14:41:29.155206+0000","last_active":"2024-04-23T14:41:31.177047+0000","last_peered":"2024-04-23T14:41:31.177047+0000","last_clean":"2024-04-23T14:41:31.177047+0000","last_became_active":"2024-04-23T14:41:29.154492+0000","last_became_peered":"2024-04-23T14:41:29.154492+0000","last_unstale":"2024-04-23T14:41:31.177047+0000","last_undegraded":"2024-04-23T14:41:31.177047+0000","last_fullsized":"2024-04-23T14:41:31.177047+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:04.127440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173681+0000","last_change":"2024-04-23T14:41:30.160723+0000","last_active":"2024-04-23T14:41:31.173681+0000","last_peered":"2024-04-23T14:41:31.173681+0000","last_clean":"2024-04-23T14:41:31.173681+0000","last_became_active":"2024-04-23T14:41:30.160379+0000","last_became_peered":"2024-04-23T14:41:30.160379+0000","last_unstale":"2024-04-23T14:41:31.173681+0000","last_undegraded":"2024-04-23T14:41:31.173681+0000","last_fullsized":"2024-04-23T14:41:31.173681+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:32:27.264356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221625+0000","last_change":"2024-04-23T14:41:29.170454+0000","last_active":"2024-04-23T14:41:31.221625+0000","last_peered":"2024-04-23T14:41:31.221625+0000","last_clean":"2024-04-23T14:41:31.221625+0000","last_became_active":"2024-04-23T14:41:29.170006+0000","last_became_peered":"2024-04-23T14:41:29.170006+0000","last_unstale":"2024-04-23T14:41:31.221625+0000","last_undegraded":"2024-04-23T14:41:31.221625+0000","last_fullsized":"2024-04-23T14:41:31.221625+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:55.871120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.219837+0000","last_change":"2024-04-23T14:41:30.182333+0000","last_active":"2024-04-23T14:41:31.219837+0000","last_peered":"2024-04-23T14:41:31.219837+0000","last_clean":"2024-04-23T14:41:31.219837+0000","last_became_active":"2024-04-23T14:41:30.179299+0000","last_became_peered":"2024-04-23T14:41:30.179299+0000","last_unstale":"2024-04-23T14:41:31.219837+0000","last_undegraded":"2024-04-23T14:41:31.219837+0000","last_fullsized":"2024-04-23T14:41:31.219837+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:59.830771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176798+0000","last_change":"2024-04-23T14:41:29.156724+0000","last_active":"2024-04-23T14:41:31.176798+0000","last_peered":"2024-04-23T14:41:31.176798+0000","last_clean":"2024-04-23T14:41:31.176798+0000","last_became_active":"2024-04-23T14:41:29.156363+0000","last_became_peered":"2024-04-23T14:41:29.156363+0000","last_unstale":"2024-04-23T14:41:31.176798+0000","last_undegraded":"2024-04-23T14:41:31.176798+0000","last_fullsized":"2024-04-23T14:41:31.176798+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:51:07.963029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223668+0000","last_change":"2024-04-23T14:41:30.163328+0000","last_active":"2024-04-23T14:41:31.223668+0000","last_peered":"2024-04-23T14:41:31.223668+0000","last_clean":"2024-04-23T14:41:31.223668+0000","last_became_active":"2024-04-23T14:41:30.162984+0000","last_became_peered":"2024-04-23T14:41:30.162984+0000","last_unstale":"2024-04-23T14:41:31.223668+0000","last_undegraded":"2024-04-23T14:41:31.223668+0000","last_fullsized":"2024-04-23T14:41:31.223668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:27.072869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176686+0000","last_change":"2024-04-23T14:41:29.166943+0000","last_active":"2024-04-23T14:41:31.176686+0000","last_peered":"2024-04-23T14:41:31.176686+0000","last_clean":"2024-04-23T14:41:31.176686+0000","last_became_active":"2024-04-23T14:41:29.166619+0000","last_became_peered":"2024-04-23T14:41:29.166619+0000","last_unstale":"2024-04-23T14:41:31.176686+0000","last_undegraded":"2024-04-23T14:41:31.176686+0000","last_fullsized":"2024-04-23T14:41:31.176686+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:39.765399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.178673+0000","last_change":"2024-04-23T14:41:30.172378+0000","last_active":"2024-04-23T14:41:31.178673+0000","last_peered":"2024-04-23T14:41:31.178673+0000","last_clean":"2024-04-23T14:41:31.178673+0000","last_became_active":"2024-04-23T14:41:30.172026+0000","last_became_peered":"2024-04-23T14:41:30.172026+0000","last_unstale":"2024-04-23T14:41:31.178673+0000","last_undegraded":"2024-04-23T14:41:31.178673+0000","last_fullsized":"2024-04-23T14:41:31.178673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:45.034433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.220136+0000","last_change":"2024-04-23T14:41:29.150617+0000","last_active":"2024-04-23T14:41:31.220136+0000","last_peered":"2024-04-23T14:41:31.220136+0000","last_clean":"2024-04-23T14:41:31.220136+0000","last_became_active":"2024-04-23T14:41:29.150304+0000","last_became_peered":"2024-04-23T14:41:29.150304+0000","last_unstale":"2024-04-23T14:41:31.220136+0000","last_undegraded":"2024-04-23T14:41:31.220136+0000","last_fullsized":"2024-04-23T14:41:31.220136+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:33.024277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221173+0000","last_change":"2024-04-23T14:41:30.172302+0000","last_active":"2024-04-23T14:41:31.221173+0000","last_peered":"2024-04-23T14:41:31.221173+0000","last_clean":"2024-04-23T14:41:31.221173+0000","last_became_active":"2024-04-23T14:41:30.172009+0000","last_became_peered":"2024-04-23T14:41:30.172009+0000","last_unstale":"2024-04-23T14:41:31.221173+0000","last_undegraded":"2024-04-23T14:41:31.221173+0000","last_fullsized":"2024-04-23T14:41:31.221173+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:19.781496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176625+0000","last_change":"2024-04-23T14:41:29.148813+0000","last_active":"2024-04-23T14:41:31.176625+0000","last_peered":"2024-04-23T14:41:31.176625+0000","last_clean":"2024-04-23T14:41:31.176625+0000","last_became_active":"2024-04-23T14:41:29.148523+0000","last_became_peered":"2024-04-23T14:41:29.148523+0000","last_unstale":"2024-04-23T14:41:31.176625+0000","last_undegraded":"2024-04-23T14:41:31.176625+0000","last_fullsized":"2024-04-23T14:41:31.176625+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:11:26.055891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172296+0000","last_change":"2024-04-23T14:41:30.182886+0000","last_active":"2024-04-23T14:41:31.172296+0000","last_peered":"2024-04-23T14:41:31.172296+0000","last_clean":"2024-04-23T14:41:31.172296+0000","last_became_active":"2024-04-23T14:41:30.181240+0000","last_became_peered":"2024-04-23T14:41:30.181240+0000","last_unstale":"2024-04-23T14:41:31.172296+0000","last_undegraded":"2024-04-23T14:41:31.172296+0000","last_fullsized":"2024-04-23T14:41:31.172296+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:21:05.768325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176714+0000","last_change":"2024-04-23T14:41:29.155513+0000","last_active":"2024-04-23T14:41:31.176714+0000","last_peered":"2024-04-23T14:41:31.176714+0000","last_clean":"2024-04-23T14:41:31.176714+0000","last_became_active":"2024-04-23T14:41:29.154886+0000","last_became_peered":"2024-04-23T14:41:29.154886+0000","last_unstale":"2024-04-23T14:41:31.176714+0000","last_undegraded":"2024-04-23T14:41:31.176714+0000","last_fullsized":"2024-04-23T14:41:31.176714+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:41:27.119257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.222614+0000","last_change":"2024-04-23T14:41:30.163730+0000","last_active":"2024-04-23T14:41:31.222614+0000","last_peered":"2024-04-23T14:41:31.222614+0000","last_clean":"2024-04-23T14:41:31.222614+0000","last_became_active":"2024-04-23T14:41:30.163426+0000","last_became_peered":"2024-04-23T14:41:30.163426+0000","last_unstale":"2024-04-23T14:41:31.222614+0000","last_undegraded":"2024-04-23T14:41:31.222614+0000","last_fullsized":"2024-04-23T14:41:31.222614+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:30:19.498575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.219713+0000","last_change":"2024-04-23T14:41:29.158324+0000","last_active":"2024-04-23T14:41:31.219713+0000","last_peered":"2024-04-23T14:41:31.219713+0000","last_clean":"2024-04-23T14:41:31.219713+0000","last_became_active":"2024-04-23T14:41:29.158037+0000","last_became_peered":"2024-04-23T14:41:29.158037+0000","last_unstale":"2024-04-23T14:41:31.219713+0000","last_undegraded":"2024-04-23T14:41:31.219713+0000","last_fullsized":"2024-04-23T14:41:31.219713+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:23.843562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177178+0000","last_change":"2024-04-23T14:41:30.179568+0000","last_active":"2024-04-23T14:41:31.177178+0000","last_peered":"2024-04-23T14:41:31.177178+0000","last_clean":"2024-04-23T14:41:31.177178+0000","last_became_active":"2024-04-23T14:41:30.178820+0000","last_became_peered":"2024-04-23T14:41:30.178820+0000","last_unstale":"2024-04-23T14:41:31.177178+0000","last_undegraded":"2024-04-23T14:41:31.177178+0000","last_fullsized":"2024-04-23T14:41:31.177178+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:00:43.361653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.220420+0000","last_change":"2024-04-23T14:41:29.161624+0000","last_active":"2024-04-23T14:41:31.220420+0000","last_peered":"2024-04-23T14:41:31.220420+0000","last_clean":"2024-04-23T14:41:31.220420+0000","last_became_active":"2024-04-23T14:41:29.161247+0000","last_became_peered":"2024-04-23T14:41:29.161247+0000","last_unstale":"2024-04-23T14:41:31.220420+0000","last_undegraded":"2024-04-23T14:41:31.220420+0000","last_fullsized":"2024-04-23T14:41:31.220420+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:58.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223780+0000","last_change":"2024-04-23T14:41:30.163484+0000","last_active":"2024-04-23T14:41:31.223780+0000","last_peered":"2024-04-23T14:41:31.223780+0000","last_clean":"2024-04-23T14:41:31.223780+0000","last_became_active":"2024-04-23T14:41:30.163107+0000","last_became_peered":"2024-04-23T14:41:30.163107+0000","last_unstale":"2024-04-23T14:41:31.223780+0000","last_undegraded":"2024-04-23T14:41:31.223780+0000","last_fullsized":"2024-04-23T14:41:31.223780+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:25.212919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173523+0000","last_change":"2024-04-23T14:41:29.155730+0000","last_active":"2024-04-23T14:41:31.173523+0000","last_peered":"2024-04-23T14:41:31.173523+0000","last_clean":"2024-04-23T14:41:31.173523+0000","last_became_active":"2024-04-23T14:41:29.155427+0000","last_became_peered":"2024-04-23T14:41:29.155427+0000","last_unstale":"2024-04-23T14:41:31.173523+0000","last_undegraded":"2024-04-23T14:41:31.173523+0000","last_fullsized":"2024-04-23T14:41:31.173523+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:18.764518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177713+0000","last_change":"2024-04-23T14:41:30.170047+0000","last_active":"2024-04-23T14:41:31.177713+0000","last_peered":"2024-04-23T14:41:31.177713+0000","last_clean":"2024-04-23T14:41:31.177713+0000","last_became_active":"2024-04-23T14:41:30.168919+0000","last_became_peered":"2024-04-23T14:41:30.168919+0000","last_unstale":"2024-04-23T14:41:31.177713+0000","last_undegraded":"2024-04-23T14:41:31.177713+0000","last_fullsized":"2024-04-23T14:41:31.177713+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:19.781496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"98'3","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.222676+0000","last_change":"2024-04-23T14:41:29.171170+0000","last_active":"2024-04-23T14:41:31.222676+0000","last_peered":"2024-04-23T14:41:31.222676+0000","last_clean":"2024-04-23T14:41:31.222676+0000","last_became_active":"2024-04-23T14:41:29.170597+0000","last_became_peered":"2024-04-23T14:41:29.170597+0000","last_unstale":"2024-04-23T14:41:31.222676+0000","last_undegraded":"2024-04-23T14:41:31.222676+0000","last_fullsized":"2024-04-23T14:41:31.222676+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:23.410277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.222435+0000","last_change":"2024-04-23T14:41:30.179541+0000","last_active":"2024-04-23T14:41:31.222435+0000","last_peered":"2024-04-23T14:41:31.222435+0000","last_clean":"2024-04-23T14:41:31.222435+0000","last_became_active":"2024-04-23T14:41:30.179146+0000","last_became_peered":"2024-04-23T14:41:30.179146+0000","last_unstale":"2024-04-23T14:41:31.222435+0000","last_undegraded":"2024-04-23T14:41:31.222435+0000","last_fullsized":"2024-04-23T14:41:31.222435+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:28:17.113173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"98'6","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223986+0000","last_change":"2024-04-23T14:41:29.168241+0000","last_active":"2024-04-23T14:41:31.223986+0000","last_peered":"2024-04-23T14:41:31.223986+0000","last_clean":"2024-04-23T14:41:31.223986+0000","last_became_active":"2024-04-23T14:41:29.167851+0000","last_became_peered":"2024-04-23T14:41:29.167851+0000","last_unstale":"2024-04-23T14:41:31.223986+0000","last_undegraded":"2024-04-23T14:41:31.223986+0000","last_fullsized":"2024-04-23T14:41:31.223986+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:11.141341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173982+0000","last_change":"2024-04-23T14:41:30.169459+0000","last_active":"2024-04-23T14:41:31.173982+0000","last_peered":"2024-04-23T14:41:31.173982+0000","last_clean":"2024-04-23T14:41:31.173982+0000","last_became_active":"2024-04-23T14:41:30.169135+0000","last_became_peered":"2024-04-23T14:41:30.169135+0000","last_unstale":"2024-04-23T14:41:31.173982+0000","last_undegraded":"2024-04-23T14:41:31.173982+0000","last_fullsized":"2024-04-23T14:41:31.173982+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:54.280793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172662+0000","last_change":"2024-04-23T14:41:29.169064+0000","last_active":"2024-04-23T14:41:31.172662+0000","last_peered":"2024-04-23T14:41:31.172662+0000","last_clean":"2024-04-23T14:41:31.172662+0000","last_became_active":"2024-04-23T14:41:29.168820+0000","last_became_peered":"2024-04-23T14:41:29.168820+0000","last_unstale":"2024-04-23T14:41:31.172662+0000","last_undegraded":"2024-04-23T14:41:31.172662+0000","last_fullsized":"2024-04-23T14:41:31.172662+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:40.504573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.219849+0000","last_change":"2024-04-23T14:41:30.188678+0000","last_active":"2024-04-23T14:41:31.219849+0000","last_peered":"2024-04-23T14:41:31.219849+0000","last_clean":"2024-04-23T14:41:31.219849+0000","last_became_active":"2024-04-23T14:41:30.188353+0000","last_became_peered":"2024-04-23T14:41:30.188353+0000","last_unstale":"2024-04-23T14:41:31.219849+0000","last_undegraded":"2024-04-23T14:41:31.219849+0000","last_fullsized":"2024-04-23T14:41:31.219849+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:59:29.370524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172217+0000","last_change":"2024-04-23T14:41:29.152287+0000","last_active":"2024-04-23T14:41:31.172217+0000","last_peered":"2024-04-23T14:41:31.172217+0000","last_clean":"2024-04-23T14:41:31.172217+0000","last_became_active":"2024-04-23T14:41:29.150841+0000","last_became_peered":"2024-04-23T14:41:29.150841+0000","last_unstale":"2024-04-23T14:41:31.172217+0000","last_undegraded":"2024-04-23T14:41:31.172217+0000","last_fullsized":"2024-04-23T14:41:31.172217+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:08.989967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221813+0000","last_change":"2024-04-23T14:41:30.167310+0000","last_active":"2024-04-23T14:41:31.221813+0000","last_peered":"2024-04-23T14:41:31.221813+0000","last_clean":"2024-04-23T14:41:31.221813+0000","last_became_active":"2024-04-23T14:41:30.167008+0000","last_became_peered":"2024-04-23T14:41:30.167008+0000","last_unstale":"2024-04-23T14:41:31.221813+0000","last_undegraded":"2024-04-23T14:41:31.221813+0000","last_fullsized":"2024-04-23T14:41:31.221813+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:52.946497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.222515+0000","last_change":"2024-04-23T14:41:29.167466+0000","last_active":"2024-04-23T14:41:31.222515+0000","last_peered":"2024-04-23T14:41:31.222515+0000","last_clean":"2024-04-23T14:41:31.222515+0000","last_became_active":"2024-04-23T14:41:29.167099+0000","last_became_peered":"2024-04-23T14:41:29.167099+0000","last_unstale":"2024-04-23T14:41:31.222515+0000","last_undegraded":"2024-04-23T14:41:31.222515+0000","last_fullsized":"2024-04-23T14:41:31.222515+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:59:28.353545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.171965+0000","last_change":"2024-04-23T14:41:30.165371+0000","last_active":"2024-04-23T14:41:31.171965+0000","last_peered":"2024-04-23T14:41:31.171965+0000","last_clean":"2024-04-23T14:41:31.171965+0000","last_became_active":"2024-04-23T14:41:30.165039+0000","last_became_peered":"2024-04-23T14:41:30.165039+0000","last_unstale":"2024-04-23T14:41:31.171965+0000","last_undegraded":"2024-04-23T14:41:31.171965+0000","last_fullsized":"2024-04-23T14:41:31.171965+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:49:57.066691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"98'8","reported_seq":22,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.219772+0000","last_change":"2024-04-23T14:41:29.161846+0000","last_active":"2024-04-23T14:41:31.219772+0000","last_peered":"2024-04-23T14:41:31.219772+0000","last_clean":"2024-04-23T14:41:31.219772+0000","last_became_active":"2024-04-23T14:41:29.161564+0000","last_became_peered":"2024-04-23T14:41:29.161564+0000","last_unstale":"2024-04-23T14:41:31.219772+0000","last_undegraded":"2024-04-23T14:41:31.219772+0000","last_fullsized":"2024-04-23T14:41:31.219772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:02:28.881281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.178135+0000","last_change":"2024-04-23T14:41:30.159418+0000","last_active":"2024-04-23T14:41:31.178135+0000","last_peered":"2024-04-23T14:41:31.178135+0000","last_clean":"2024-04-23T14:41:31.178135+0000","last_became_active":"2024-04-23T14:41:30.156792+0000","last_became_peered":"2024-04-23T14:41:30.156792+0000","last_unstale":"2024-04-23T14:41:31.178135+0000","last_undegraded":"2024-04-23T14:41:31.178135+0000","last_fullsized":"2024-04-23T14:41:31.178135+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.413980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223109+0000","last_change":"2024-04-23T14:41:29.167837+0000","last_active":"2024-04-23T14:41:31.223109+0000","last_peered":"2024-04-23T14:41:31.223109+0000","last_clean":"2024-04-23T14:41:31.223109+0000","last_became_active":"2024-04-23T14:41:29.166272+0000","last_became_peered":"2024-04-23T14:41:29.166272+0000","last_unstale":"2024-04-23T14:41:31.223109+0000","last_undegraded":"2024-04-23T14:41:31.223109+0000","last_fullsized":"2024-04-23T14:41:31.223109+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:18.794811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.179111+0000","last_change":"2024-04-23T14:41:30.167714+0000","last_active":"2024-04-23T14:41:31.179111+0000","last_peered":"2024-04-23T14:41:31.179111+0000","last_clean":"2024-04-23T14:41:31.179111+0000","last_became_active":"2024-04-23T14:41:30.167413+0000","last_became_peered":"2024-04-23T14:41:30.167413+0000","last_unstale":"2024-04-23T14:41:31.179111+0000","last_undegraded":"2024-04-23T14:41:31.179111+0000","last_fullsized":"2024-04-23T14:41:31.179111+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:24.427255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177516+0000","last_change":"2024-04-23T14:41:29.162803+0000","last_active":"2024-04-23T14:41:31.177516+0000","last_peered":"2024-04-23T14:41:31.177516+0000","last_clean":"2024-04-23T14:41:31.177516+0000","last_became_active":"2024-04-23T14:41:29.162194+0000","last_became_peered":"2024-04-23T14:41:29.162194+0000","last_unstale":"2024-04-23T14:41:31.177516+0000","last_undegraded":"2024-04-23T14:41:31.177516+0000","last_fullsized":"2024-04-23T14:41:31.177516+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:22:15.255469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.222676+0000","last_change":"2024-04-23T14:41:30.173516+0000","last_active":"2024-04-23T14:41:31.222676+0000","last_peered":"2024-04-23T14:41:31.222676+0000","last_clean":"2024-04-23T14:41:31.222676+0000","last_became_active":"2024-04-23T14:41:30.171897+0000","last_became_peered":"2024-04-23T14:41:30.171897+0000","last_unstale":"2024-04-23T14:41:31.222676+0000","last_undegraded":"2024-04-23T14:41:31.222676+0000","last_fullsized":"2024-04-23T14:41:31.222676+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:32:07.797373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176808+0000","last_change":"2024-04-23T14:41:29.163038+0000","last_active":"2024-04-23T14:41:31.176808+0000","last_peered":"2024-04-23T14:41:31.176808+0000","last_clean":"2024-04-23T14:41:31.176808+0000","last_became_active":"2024-04-23T14:41:29.162738+0000","last_became_peered":"2024-04-23T14:41:29.162738+0000","last_unstale":"2024-04-23T14:41:31.176808+0000","last_undegraded":"2024-04-23T14:41:31.176808+0000","last_fullsized":"2024-04-23T14:41:31.176808+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:59.934295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172130+0000","last_change":"2024-04-23T14:41:30.178825+0000","last_active":"2024-04-23T14:41:31.172130+0000","last_peered":"2024-04-23T14:41:31.172130+0000","last_clean":"2024-04-23T14:41:31.172130+0000","last_became_active":"2024-04-23T14:41:30.178539+0000","last_became_peered":"2024-04-23T14:41:30.178539+0000","last_unstale":"2024-04-23T14:41:31.172130+0000","last_undegraded":"2024-04-23T14:41:31.172130+0000","last_fullsized":"2024-04-23T14:41:31.172130+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:32.338723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177578+0000","last_change":"2024-04-23T14:41:29.156525+0000","last_active":"2024-04-23T14:41:31.177578+0000","last_peered":"2024-04-23T14:41:31.177578+0000","last_clean":"2024-04-23T14:41:31.177578+0000","last_became_active":"2024-04-23T14:41:29.156198+0000","last_became_peered":"2024-04-23T14:41:29.156198+0000","last_unstale":"2024-04-23T14:41:31.177578+0000","last_undegraded":"2024-04-23T14:41:31.177578+0000","last_fullsized":"2024-04-23T14:41:31.177578+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:33:24.195940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172575+0000","last_change":"2024-04-23T14:41:30.185004+0000","last_active":"2024-04-23T14:41:31.172575+0000","last_peered":"2024-04-23T14:41:31.172575+0000","last_clean":"2024-04-23T14:41:31.172575+0000","last_became_active":"2024-04-23T14:41:30.182407+0000","last_became_peered":"2024-04-23T14:41:30.182407+0000","last_unstale":"2024-04-23T14:41:31.172575+0000","last_undegraded":"2024-04-23T14:41:31.172575+0000","last_fullsized":"2024-04-23T14:41:31.172575+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:59.297366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.178637+0000","last_change":"2024-04-23T14:41:29.163604+0000","last_active":"2024-04-23T14:41:31.178637+0000","last_peered":"2024-04-23T14:41:31.178637+0000","last_clean":"2024-04-23T14:41:31.178637+0000","last_became_active":"2024-04-23T14:41:29.163303+0000","last_became_peered":"2024-04-23T14:41:29.163303+0000","last_unstale":"2024-04-23T14:41:31.178637+0000","last_undegraded":"2024-04-23T14:41:31.178637+0000","last_fullsized":"2024-04-23T14:41:31.178637+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:19.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172669+0000","last_change":"2024-04-23T14:41:30.160471+0000","last_active":"2024-04-23T14:41:31.172669+0000","last_peered":"2024-04-23T14:41:31.172669+0000","last_clean":"2024-04-23T14:41:31.172669+0000","last_became_active":"2024-04-23T14:41:30.160144+0000","last_became_peered":"2024-04-23T14:41:30.160144+0000","last_unstale":"2024-04-23T14:41:31.172669+0000","last_undegraded":"2024-04-23T14:41:31.172669+0000","last_fullsized":"2024-04-23T14:41:31.172669+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:35:35.827791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.178303+0000","last_change":"2024-04-23T14:41:29.165780+0000","last_active":"2024-04-23T14:41:31.178303+0000","last_peered":"2024-04-23T14:41:31.178303+0000","last_clean":"2024-04-23T14:41:31.178303+0000","last_became_active":"2024-04-23T14:41:29.165435+0000","last_became_peered":"2024-04-23T14:41:29.165435+0000","last_unstale":"2024-04-23T14:41:31.178303+0000","last_undegraded":"2024-04-23T14:41:31.178303+0000","last_fullsized":"2024-04-23T14:41:31.178303+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:52:49.591648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176421+0000","last_change":"2024-04-23T14:41:30.168851+0000","last_active":"2024-04-23T14:41:31.176421+0000","last_peered":"2024-04-23T14:41:31.176421+0000","last_clean":"2024-04-23T14:41:31.176421+0000","last_became_active":"2024-04-23T14:41:30.168533+0000","last_became_peered":"2024-04-23T14:41:30.168533+0000","last_unstale":"2024-04-23T14:41:31.176421+0000","last_undegraded":"2024-04-23T14:41:31.176421+0000","last_fullsized":"2024-04-23T14:41:31.176421+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:51:08.980007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177294+0000","last_change":"2024-04-23T14:41:29.146434+0000","last_active":"2024-04-23T14:41:31.177294+0000","last_peered":"2024-04-23T14:41:31.177294+0000","last_clean":"2024-04-23T14:41:31.177294+0000","last_became_active":"2024-04-23T14:41:29.145530+0000","last_became_peered":"2024-04-23T14:41:29.145530+0000","last_unstale":"2024-04-23T14:41:31.177294+0000","last_undegraded":"2024-04-23T14:41:31.177294+0000","last_fullsized":"2024-04-23T14:41:31.177294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:12:23.410277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177349+0000","last_change":"2024-04-23T14:41:30.176294+0000","last_active":"2024-04-23T14:41:31.177349+0000","last_peered":"2024-04-23T14:41:31.177349+0000","last_clean":"2024-04-23T14:41:31.177349+0000","last_became_active":"2024-04-23T14:41:30.175525+0000","last_became_peered":"2024-04-23T14:41:30.175525+0000","last_unstale":"2024-04-23T14:41:31.177349+0000","last_undegraded":"2024-04-23T14:41:31.177349+0000","last_fullsized":"2024-04-23T14:41:31.177349+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:00:42.385292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.179182+0000","last_change":"2024-04-23T14:41:29.169432+0000","last_active":"2024-04-23T14:41:31.179182+0000","last_peered":"2024-04-23T14:41:31.179182+0000","last_clean":"2024-04-23T14:41:31.179182+0000","last_became_active":"2024-04-23T14:41:29.168901+0000","last_became_peered":"2024-04-23T14:41:29.168901+0000","last_unstale":"2024-04-23T14:41:31.179182+0000","last_undegraded":"2024-04-23T14:41:31.179182+0000","last_fullsized":"2024-04-23T14:41:31.179182+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:22:15.255469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.220325+0000","last_change":"2024-04-23T14:41:30.185239+0000","last_active":"2024-04-23T14:41:31.220325+0000","last_peered":"2024-04-23T14:41:31.220325+0000","last_clean":"2024-04-23T14:41:31.220325+0000","last_became_active":"2024-04-23T14:41:30.184952+0000","last_became_peered":"2024-04-23T14:41:30.184952+0000","last_unstale":"2024-04-23T14:41:31.220325+0000","last_undegraded":"2024-04-23T14:41:31.220325+0000","last_fullsized":"2024-04-23T14:41:31.220325+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:19.811789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"99'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177462+0000","last_change":"2024-04-23T14:41:29.157298+0000","last_active":"2024-04-23T14:41:31.177462+0000","last_peered":"2024-04-23T14:41:31.177462+0000","last_clean":"2024-04-23T14:41:31.177462+0000","last_became_active":"2024-04-23T14:41:29.156972+0000","last_became_peered":"2024-04-23T14:41:29.156972+0000","last_unstale":"2024-04-23T14:41:31.177462+0000","last_undegraded":"2024-04-23T14:41:31.177462+0000","last_fullsized":"2024-04-23T14:41:31.177462+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:09:22.916599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.220001+0000","last_change":"2024-04-23T14:41:30.183921+0000","last_active":"2024-04-23T14:41:31.220001+0000","last_peered":"2024-04-23T14:41:31.220001+0000","last_clean":"2024-04-23T14:41:31.220001+0000","last_became_active":"2024-04-23T14:41:30.181408+0000","last_became_peered":"2024-04-23T14:41:30.181408+0000","last_unstale":"2024-04-23T14:41:31.220001+0000","last_undegraded":"2024-04-23T14:41:31.220001+0000","last_fullsized":"2024-04-23T14:41:31.220001+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:44:41.205449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176820+0000","last_change":"2024-04-23T14:41:29.162997+0000","last_active":"2024-04-23T14:41:31.176820+0000","last_peered":"2024-04-23T14:41:31.176820+0000","last_clean":"2024-04-23T14:41:31.176820+0000","last_became_active":"2024-04-23T14:41:29.162673+0000","last_became_peered":"2024-04-23T14:41:29.162673+0000","last_unstale":"2024-04-23T14:41:31.176820+0000","last_undegraded":"2024-04-23T14:41:31.176820+0000","last_fullsized":"2024-04-23T14:41:31.176820+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:38:25.154734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177504+0000","last_change":"2024-04-23T14:41:30.175184+0000","last_active":"2024-04-23T14:41:31.177504+0000","last_peered":"2024-04-23T14:41:31.177504+0000","last_clean":"2024-04-23T14:41:31.177504+0000","last_became_active":"2024-04-23T14:41:30.174509+0000","last_became_peered":"2024-04-23T14:41:30.174509+0000","last_unstale":"2024-04-23T14:41:31.177504+0000","last_undegraded":"2024-04-23T14:41:31.177504+0000","last_fullsized":"2024-04-23T14:41:31.177504+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:14.321147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177200+0000","last_change":"2024-04-23T14:41:29.162589+0000","last_active":"2024-04-23T14:41:31.177200+0000","last_peered":"2024-04-23T14:41:31.177200+0000","last_clean":"2024-04-23T14:41:31.177200+0000","last_became_active":"2024-04-23T14:41:29.162046+0000","last_became_peered":"2024-04-23T14:41:29.162046+0000","last_unstale":"2024-04-23T14:41:31.177200+0000","last_undegraded":"2024-04-23T14:41:31.177200+0000","last_fullsized":"2024-04-23T14:41:31.177200+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:52:49.591648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172193+0000","last_change":"2024-04-23T14:41:30.174733+0000","last_active":"2024-04-23T14:41:31.172193+0000","last_peered":"2024-04-23T14:41:31.172193+0000","last_clean":"2024-04-23T14:41:31.172193+0000","last_became_active":"2024-04-23T14:41:30.174398+0000","last_became_peered":"2024-04-23T14:41:30.174398+0000","last_unstale":"2024-04-23T14:41:31.172193+0000","last_undegraded":"2024-04-23T14:41:31.172193+0000","last_fullsized":"2024-04-23T14:41:31.172193+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:35:57.864459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.177677+0000","last_change":"2024-04-23T14:41:29.154132+0000","last_active":"2024-04-23T14:41:31.177677+0000","last_peered":"2024-04-23T14:41:31.177677+0000","last_clean":"2024-04-23T14:41:31.177677+0000","last_became_active":"2024-04-23T14:41:29.152404+0000","last_became_peered":"2024-04-23T14:41:29.152404+0000","last_unstale":"2024-04-23T14:41:31.177677+0000","last_undegraded":"2024-04-23T14:41:31.177677+0000","last_fullsized":"2024-04-23T14:41:31.177677+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:02:46.480574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.223365+0000","last_change":"2024-04-23T14:41:30.158906+0000","last_active":"2024-04-23T14:41:31.223365+0000","last_peered":"2024-04-23T14:41:31.223365+0000","last_clean":"2024-04-23T14:41:31.223365+0000","last_became_active":"2024-04-23T14:41:30.158257+0000","last_became_peered":"2024-04-23T14:41:30.158257+0000","last_unstale":"2024-04-23T14:41:31.223365+0000","last_undegraded":"2024-04-23T14:41:31.223365+0000","last_fullsized":"2024-04-23T14:41:31.223365+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:35:56.888098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176675+0000","last_change":"2024-04-23T14:41:29.157151+0000","last_active":"2024-04-23T14:41:31.176675+0000","last_peered":"2024-04-23T14:41:31.176675+0000","last_clean":"2024-04-23T14:41:31.176675+0000","last_became_active":"2024-04-23T14:41:29.155019+0000","last_became_peered":"2024-04-23T14:41:29.155019+0000","last_unstale":"2024-04-23T14:41:31.176675+0000","last_undegraded":"2024-04-23T14:41:31.176675+0000","last_fullsized":"2024-04-23T14:41:31.176675+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:49:19.315893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172376+0000","last_change":"2024-04-23T14:41:30.168313+0000","last_active":"2024-04-23T14:41:31.172376+0000","last_peered":"2024-04-23T14:41:31.172376+0000","last_clean":"2024-04-23T14:41:31.172376+0000","last_became_active":"2024-04-23T14:41:30.167936+0000","last_became_peered":"2024-04-23T14:41:30.167936+0000","last_unstale":"2024-04-23T14:41:31.172376+0000","last_undegraded":"2024-04-23T14:41:31.172376+0000","last_fullsized":"2024-04-23T14:41:31.172376+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:40:21.200552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.173411+0000","last_change":"2024-04-23T14:41:29.152918+0000","last_active":"2024-04-23T14:41:31.173411+0000","last_peered":"2024-04-23T14:41:31.173411+0000","last_clean":"2024-04-23T14:41:31.173411+0000","last_became_active":"2024-04-23T14:41:29.152661+0000","last_became_peered":"2024-04-23T14:41:29.152661+0000","last_unstale":"2024-04-23T14:41:31.173411+0000","last_undegraded":"2024-04-23T14:41:31.173411+0000","last_fullsized":"2024-04-23T14:41:31.173411+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:13.304169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.176588+0000","last_change":"2024-04-23T14:41:30.163121+0000","last_active":"2024-04-23T14:41:31.176588+0000","last_peered":"2024-04-23T14:41:31.176588+0000","last_clean":"2024-04-23T14:41:31.176588+0000","last_became_active":"2024-04-23T14:41:30.162074+0000","last_became_peered":"2024-04-23T14:41:30.162074+0000","last_unstale":"2024-04-23T14:41:31.176588+0000","last_undegraded":"2024-04-23T14:41:31.176588+0000","last_fullsized":"2024-04-23T14:41:31.176588+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:05.144418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172475+0000","last_change":"2024-04-23T14:41:29.154113+0000","last_active":"2024-04-23T14:41:31.172475+0000","last_peered":"2024-04-23T14:41:31.172475+0000","last_clean":"2024-04-23T14:41:31.172475+0000","last_became_active":"2024-04-23T14:41:29.153847+0000","last_became_peered":"2024-04-23T14:41:29.153847+0000","last_unstale":"2024-04-23T14:41:31.172475+0000","last_undegraded":"2024-04-23T14:41:31.172475+0000","last_fullsized":"2024-04-23T14:41:31.172475+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:37.570893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.172921+0000","last_change":"2024-04-23T14:41:30.176178+0000","last_active":"2024-04-23T14:41:31.172921+0000","last_peered":"2024-04-23T14:41:31.172921+0000","last_clean":"2024-04-23T14:41:31.172921+0000","last_became_active":"2024-04-23T14:41:30.175715+0000","last_became_peered":"2024-04-23T14:41:30.175715+0000","last_unstale":"2024-04-23T14:41:31.172921+0000","last_undegraded":"2024-04-23T14:41:31.172921+0000","last_fullsized":"2024-04-23T14:41:31.172921+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:29.112596+0000","last_clean_scrub_stamp":"2024-04-23T14:41:29.112596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:02.991000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221956+0000","last_change":"2024-04-23T14:41:29.170521+0000","last_active":"2024-04-23T14:41:31.221956+0000","last_peered":"2024-04-23T14:41:31.221956+0000","last_clean":"2024-04-23T14:41:31.221956+0000","last_became_active":"2024-04-23T14:41:29.170150+0000","last_became_peered":"2024-04-23T14:41:29.170150+0000","last_unstale":"2024-04-23T14:41:31.221956+0000","last_undegraded":"2024-04-23T14:41:31.221956+0000","last_fullsized":"2024-04-23T14:41:31.221956+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:41:28.095618+0000","last_clean_scrub_stamp":"2024-04-23T14:41:28.095618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:44.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":317,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-23T14:41:31.221568+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:41:31.221568+0000","last_peered":"2024-04-23T14:41:31.221568+0000","last_clean":"2024-04-23T14:41:31.221568+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:41:31.221568+0000","last_undegraded":"2024-04-23T14:41:31.221568+0000","last_fullsized":"2024-04-23T14:41:31.221568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1826816,"data_stored":1120458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":530630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":530048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":531714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":530012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":529978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1822720,"data_stored":1120928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:41:35.118 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:41:35.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:41:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.382+0000 7f2f96200640 1 Processor -- start 2024-04-23T14:41:35.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.382+0000 7f2f96200640 1 -- start start 2024-04-23T14:41:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.384+0000 7f2f96200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2f90156930 0x7f2f90156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.384+0000 7f2f96200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9006dd50 con 0x7f2f9015cb50 2024-04-23T14:41:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.384+0000 7f2f96200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9015d1f0 con 0x7f2f900ad880 2024-04-23T14:41:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.384+0000 7f2f96200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2f9015d4d0 con 0x7f2f90156930 2024-04-23T14:41:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.384+0000 7f2f8fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2f9015cb50 0x7f2f900ad170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55656/0 (socket says 172.21.15.59:55656) 2024-04-23T14:41:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.384+0000 7f2f8fe00640 1 -- 172.21.15.59:0/2111216670 learned_addr learned my addr 172.21.15.59:0/2111216670 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:35.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f94e00640 1 --2- 172.21.15.59:0/2111216670 >> v2:172.21.15.76:3300/0 conn(0x7f2f90156930 0x7f2f90156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3140749403 0 0) 0x7f2f9006dd50 con 0x7f2f9015cb50 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f78003520 con 0x7f2f9015cb50 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1566999972 0 0) 0x7f2f78003520 con 0x7f2f9015cb50 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 >> v2:172.21.15.76:3300/0 conn(0x7f2f90156930 msgr2=0x7f2f90156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 --2- 172.21.15.59:0/2111216670 >> v2:172.21.15.76:3300/0 conn(0x7f2f90156930 0x7f2f90156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 >> v1:172.21.15.76:6789/0 conn(0x7f2f900ad880 legacy=0x7f2f9006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9015d7b0 con 0x7f2f9015cb50 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2f7c001670 con 0x7f2f9015cb50 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.385+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 >> v1:172.21.15.59:6789/0 conn(0x7f2f9015cb50 legacy=0x7f2f900ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8f400640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f78003a00 0x7f2f78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8f400640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f78004800 0x7f2f78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8f400640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f780051c0 0x7f2f7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9015d4d0 con 0x7f2f78004800 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7800c090 con 0x7f2f780051c0 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7800c290 con 0x7f2f78003a00 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8fe00640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f78004800 0x7f2f78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f94e00640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f78003a00 0x7f2f78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8fe00640 1 -- 172.21.15.59:0/2111216670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f78003a00 msgr2=0x7f2f78003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8fe00640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f78003a00 0x7f2f78003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8fe00640 1 -- 172.21.15.59:0/2111216670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f780051c0 msgr2=0x7f2f7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8fe00640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f780051c0 0x7f2f7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f8fe00640 1 -- 172.21.15.59:0/2111216670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f78003520 con 0x7f2f78004800 2024-04-23T14:41:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.386+0000 7f2f95800640 1 -- 172.21.15.59:0/2111216670 reap_dead start 2024-04-23T14:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.387+0000 7f2f8fe00640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f78004800 0x7f2f78004c80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c110 tx=0x7f2f78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ca2dd66fe76d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c007c40 con 0x7f2f78004800 2024-04-23T14:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c005af0 con 0x7f2f78004800 2024-04-23T14:41:35.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f8f400640 1 -- 172.21.15.59:0/2111216670 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c007070 con 0x7f2f78004800 2024-04-23T14:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f96200640 1 -- 172.21.15.59:0/2111216670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f78004800 msgr2=0x7f2f78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f96200640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f78004800 0x7f2f78004c80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c110 tx=0x7f2f78005830 comp rx=0 tx=0).stop 2024-04-23T14:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f96200640 1 -- 172.21.15.59:0/2111216670 shutdown_connections 2024-04-23T14:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f96200640 1 --2- 172.21.15.59:0/2111216670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f78004800 0x7f2f78004c80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.388+0000 7f2f96200640 1 -- 172.21.15.59:0/2111216670 >> 172.21.15.59:0/2111216670 conn(0x7f2f900b1e90 msgr2=0x7f2f900b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.389+0000 7f2f96200640 1 -- 172.21.15.59:0/2111216670 shutdown_connections 2024-04-23T14:41:35.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.389+0000 7f2f96200640 1 -- 172.21.15.59:0/2111216670 wait complete. 2024-04-23T14:41:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.390+0000 7f2f96200640 1 Processor -- start 2024-04-23T14:41:35.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.391+0000 7f2f96200640 1 -- start start 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.391+0000 7f2f96200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f78004800 0x7f2f9014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f96200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f900b0e80 0x7f2f900b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f96200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f9014a1a0 0x7f2f9014a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f96200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90156e90 con 0x7f2f900b0e80 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f96200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90156d10 con 0x7f2f78004800 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f96200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2f90157010 con 0x7f2f9014a1a0 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f900b0e80 0x7f2f900b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f900b0e80 0x7f2f900b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59738/0 (socket says 172.21.15.59:59738) 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f8fe00640 1 -- 172.21.15.59:0/4063166151 learned_addr learned my addr 172.21.15.59:0/4063166151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f94e00640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f78004800 0x7f2f9014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f95800640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f9014a1a0 0x7f2f9014a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.392+0000 7f2f8fe00640 1 -- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f9014a1a0 msgr2=0x7f2f9014a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8fe00640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f9014a1a0 0x7f2f9014a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8fe00640 1 -- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f78004800 msgr2=0x7f2f9014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8fe00640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f78004800 0x7f2f9014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8fe00640 1 -- 172.21.15.59:0/4063166151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9014ae40 con 0x7f2f900b0e80 2024-04-23T14:41:35.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8fe00640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f900b0e80 0x7f2f900b1300 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c02ad30 tx=0x7f2f9015d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:35.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8cc00640 1 -- 172.21.15.59:0/4063166151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c041020 con 0x7f2f900b0e80 2024-04-23T14:41:35.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8cc00640 1 -- 172.21.15.59:0/4063166151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c038040 con 0x7f2f900b0e80 2024-04-23T14:41:35.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f8cc00640 1 -- 172.21.15.59:0/4063166151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0072b0 con 0x7f2f900b0e80 2024-04-23T14:41:35.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9014b120 con 0x7f2f900b0e80 2024-04-23T14:41:35.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.393+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9014b6e0 con 0x7f2f900b0e80 2024-04-23T14:41:35.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.395+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9004fd90 con 0x7f2f900b0e80 2024-04-23T14:41:35.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.396+0000 7f2f8cc00640 1 -- 172.21.15.59:0/4063166151 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f7c044020 con 0x7f2f900b0e80 2024-04-23T14:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.397+0000 7f2f8cc00640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f7006c920 0x7f2f7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.397+0000 7f2f8cc00640 1 -- 172.21.15.59:0/4063166151 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c0abee0 con 0x7f2f900b0e80 2024-04-23T14:41:35.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.397+0000 7f2f94e00640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f7006c920 0x7f2f7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.398+0000 7f2f94e00640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f7006c920 0x7f2f7006ede0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2f9006a7b0 tx=0x7f2f84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:35.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.401+0000 7f2f8cc00640 1 -- 172.21.15.59:0/4063166151 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f7c07a280 con 0x7f2f900b0e80 2024-04-23T14:41:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.677+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2f9006fca0 con 0x7f2f900b0e80 2024-04-23T14:41:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.682+0000 7f2f8cc00640 1 -- 172.21.15.59:0/4063166151 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f2f7c079b80 con 0x7f2f900b0e80 2024-04-23T14:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-23T14:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-23T14:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:41:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f7006c920 msgr2=0x7f2f7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f7006c920 0x7f2f7006ede0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2f9006a7b0 tx=0x7f2f84008040 comp rx=0 tx=0).stop 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f900b0e80 msgr2=0x7f2f900b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f900b0e80 0x7f2f900b1300 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c02ad30 tx=0x7f2f9015d980 comp rx=0 tx=0).stop 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 shutdown_connections 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2f7006c920 0x7f2f7006ede0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f9014a1a0 0x7f2f9014a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f900b0e80 0x7f2f900b1300 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 --2- 172.21.15.59:0/4063166151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f78004800 0x7f2f9014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.684+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 >> 172.21.15.59:0/4063166151 conn(0x7f2f900b1e90 msgr2=0x7f2f900b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.685+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 shutdown_connections 2024-04-23T14:41:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.685+0000 7f2f96200640 1 -- 172.21.15.59:0/4063166151 wait complete. 2024-04-23T14:41:35.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.911+0000 7fb5b4e00640 1 Processor -- start 2024-04-23T14:41:35.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.911+0000 7fb5b4e00640 1 -- start start 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.912+0000 7fb5b4e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b00d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.912+0000 7fb5b4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5b00d0d90 0x7fb5b00d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.912+0000 7fb5b4e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5b00cf030 con 0x7fb5b00d2630 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.912+0000 7fb5b4e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b00b7000 con 0x7fb5b00d0d90 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.912+0000 7fb5b4e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00cf620 con 0x7fb5b015e7c0 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5b00d0d90 0x7fb5b00d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5b00d0d90 0x7fb5b00d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59748/0 (socket says 172.21.15.59:59748) 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5afe00640 1 -- 172.21.15.59:0/3788889537 learned_addr learned my addr 172.21.15.59:0/3788889537 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5aea00640 1 --1- 172.21.15.59:0/3788889537 >> v1:172.21.15.59:6789/0 conn(0x7fb5b00d2630 0x7fb5b00b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55664/0 (socket says 172.21.15.59:55664) 2024-04-23T14:41:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5af400640 1 --2- 172.21.15.59:0/3788889537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b00d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5afe00640 1 -- 172.21.15.59:0/3788889537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 msgr2=0x7fb5b00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5afe00640 1 --2- 172.21.15.59:0/3788889537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b00d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5afe00640 1 -- 172.21.15.59:0/3788889537 >> v1:172.21.15.59:6789/0 conn(0x7fb5b00d2630 legacy=0x7fb5b00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5afe00640 1 -- 172.21.15.59:0/3788889537 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b00b6a40 con 0x7fb5b00d0d90 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.913+0000 7fb5ae000640 1 -- 172.21.15.59:0/3788889537 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527209142 0 0) 0x7fb598001110 con 0x7fb5b00d2630 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.914+0000 7fb5afe00640 1 --2- 172.21.15.59:0/3788889537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b00d1220 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009870 tx=0x7fb5a802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a1185d2e0c62a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5ae000640 1 -- 172.21.15.59:0/3788889537 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8037070 con 0x7fb5b00d0d90 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5ae000640 1 -- 172.21.15.59:0/3788889537 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a8032440 con 0x7fb5b00d0d90 2024-04-23T14:41:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5ae000640 1 -- 172.21.15.59:0/3788889537 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8032810 con 0x7fb5b00d0d90 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 -- 172.21.15.59:0/3788889537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 msgr2=0x7fb5b00d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/3788889537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b00d1220 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009870 tx=0x7fb5a802a250 comp rx=0 tx=0).stop 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 -- 172.21.15.59:0/3788889537 shutdown_connections 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/3788889537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b00d1220 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/3788889537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b00d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 -- 172.21.15.59:0/3788889537 >> 172.21.15.59:0/3788889537 conn(0x7fb5b009d920 msgr2=0x7fb5b008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 -- 172.21.15.59:0/3788889537 shutdown_connections 2024-04-23T14:41:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.915+0000 7fb5b4e00640 1 -- 172.21.15.59:0/3788889537 wait complete. 2024-04-23T14:41:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.917+0000 7fb5b4e00640 1 Processor -- start 2024-04-23T14:41:35.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.917+0000 7fb5b4e00640 1 -- start start 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.918+0000 7fb5b4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b0159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.918+0000 7fb5b4e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00d2630 0x7fb5b006d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.918+0000 7fb5b4e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.918+0000 7fb5b4e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00b6700 con 0x7fb5b00d0d90 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.918+0000 7fb5b4e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00cf2b0 con 0x7fb5b00d2630 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.918+0000 7fb5b4e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00d1a80 con 0x7fb5b015e7c0 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b0159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b0159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59754/0 (socket says 172.21.15.59:59754) 2024-04-23T14:41:35.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 -- 172.21.15.59:0/2741456406 learned_addr learned my addr 172.21.15.59:0/2741456406 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 -- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 msgr2=0x7fb5b006da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5afe00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5aea00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00d2630 0x7fb5b006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b006da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 -- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00d2630 msgr2=0x7fb5b006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00d2630 0x7fb5b006d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:35.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 -- 172.21.15.59:0/2741456406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b00c80d0 con 0x7fb5b00d0d90 2024-04-23T14:41:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.919+0000 7fb5af400640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b0159ba0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0007e40 tx=0x7fb5a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.920+0000 7fb5a7400640 1 -- 172.21.15.59:0/2741456406 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0014620 con 0x7fb5b00d0d90 2024-04-23T14:41:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.920+0000 7fb5a7400640 1 -- 172.21.15.59:0/2741456406 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a0019070 con 0x7fb5b00d0d90 2024-04-23T14:41:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.920+0000 7fb5a7400640 1 -- 172.21.15.59:0/2741456406 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0014a00 con 0x7fb5b00d0d90 2024-04-23T14:41:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.920+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b00c83b0 con 0x7fb5b00d0d90 2024-04-23T14:41:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.920+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b00c8970 con 0x7fb5b00d0d90 2024-04-23T14:41:35.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.921+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b004fd90 con 0x7fb5b00d0d90 2024-04-23T14:41:35.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.922+0000 7fb5a7400640 1 -- 172.21.15.59:0/2741456406 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5a0014ba0 con 0x7fb5b00d0d90 2024-04-23T14:41:35.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.923+0000 7fb5a7400640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb58c06c8f0 0x7fb58c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:35.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.923+0000 7fb5a7400640 1 -- 172.21.15.59:0/2741456406 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fb5a008c400 con 0x7fb5b00d0d90 2024-04-23T14:41:35.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.923+0000 7fb5aea00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb58c06c8f0 0x7fb58c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:35.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.924+0000 7fb5aea00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb58c06c8f0 0x7fb58c06edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00b7280 tx=0x7fb598009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:35.927+0000 7fb5a7400640 1 -- 172.21.15.59:0/2741456406 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5a005a820 con 0x7fb5b00d0d90 2024-04-23T14:41:36.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.162+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5b00b6240 con 0x7fb5b00d0d90 2024-04-23T14:41:36.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.163+0000 7fb5a7400640 1 -- 172.21.15.59:0/2741456406 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5180 (secure 0 0 0) 0x7fb5a005a120 con 0x7fb5b00d0d90 2024-04-23T14:41:36.163 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:36.163 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:41:32.686386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8514":{"gid":8514,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/620420969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":620420969},{"type":"v1","addr":"172.21.15.59:6837","nonce":620420969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8514,"qdb_cluster":[8514]},"id":7}]} 2024-04-23T14:41:36.163 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 82 2024-04-23T14:41:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb58c06c8f0 msgr2=0x7fb58c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb58c06c8f0 0x7fb58c06edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00b7280 tx=0x7fb598009040 comp rx=0 tx=0).stop 2024-04-23T14:41:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 msgr2=0x7fb5b0159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b0159ba0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0007e40 tx=0x7fb5a000e4b0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 shutdown_connections 2024-04-23T14:41:36.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb58c06c8f0 0x7fb58c06edb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5b015e7c0 0x7fb5b006da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00d2630 0x7fb5b006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 --2- 172.21.15.59:0/2741456406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5b00d0d90 0x7fb5b0159ba0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 >> 172.21.15.59:0/2741456406 conn(0x7fb5b009d920 msgr2=0x7fb5b00d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:36.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 shutdown_connections 2024-04-23T14:41:36.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.165+0000 7fb5b4e00640 1 -- 172.21.15.59:0/2741456406 wait complete. 2024-04-23T14:41:36.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:41:29.612586+0000', 'modified': '2024-04-23T14:41:32.686386+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8514': {'gid': 8514, 'name': 'a', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6837/620420969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 620420969}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 620420969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8514, 'qdb_cluster': [8514]} 2024-04-23T14:41:36.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:41:36.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-23T14:41:36.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.808+0000 7f4aa3200640 1 Processor -- start 2024-04-23T14:41:36.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.809+0000 7f4aa3200640 1 -- start start 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.810+0000 7f4aa3200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4aa415c670 0x7f4aa40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.810+0000 7f4aa3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4aa40b4230 0x7f4aa40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.810+0000 7f4aa3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40d05c0 0x7f4aa40d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.810+0000 7f4aa3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa415ca50 con 0x7f4aa40b4230 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.810+0000 7f4aa3200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa40ccb80 con 0x7f4aa415c670 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.810+0000 7f4aa3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa40b4cf0 con 0x7f4aa40d05c0 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.810+0000 7f4a99400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4aa40b4230 0x7f4aa40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4a99400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4aa40b4230 0x7f4aa40b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59768/0 (socket says 172.21.15.59:59768) 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4a99400640 1 -- 172.21.15.59:0/2939071098 learned_addr learned my addr 172.21.15.59:0/2939071098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/2939071098 >> v2:172.21.15.76:3300/0 conn(0x7f4aa415c670 0x7f4aa40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4aa2800640 1 --2- 172.21.15.59:0/2939071098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40d05c0 0x7f4aa40d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4a99400640 1 -- 172.21.15.59:0/2939071098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40d05c0 msgr2=0x7f4aa40d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4a99400640 1 --2- 172.21.15.59:0/2939071098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40d05c0 0x7f4aa40d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4a99400640 1 -- 172.21.15.59:0/2939071098 >> v2:172.21.15.76:3300/0 conn(0x7f4aa415c670 msgr2=0x7f4aa40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4a99400640 1 --2- 172.21.15.59:0/2939071098 >> v2:172.21.15.76:3300/0 conn(0x7f4aa415c670 0x7f4aa40cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.811+0000 7f4a99400640 1 -- 172.21.15.59:0/2939071098 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa40ccdd0 con 0x7f4aa40b4230 2024-04-23T14:41:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.812+0000 7f4a99400640 1 --2- 172.21.15.59:0/2939071098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa40b4230 0x7f4aa40b4690 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40d1270 tx=0x7f4a8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6947b7aa245139 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa1400640 1 -- 172.21.15.59:0/2939071098 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c037070 con 0x7f4aa40b4230 2024-04-23T14:41:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa1400640 1 -- 172.21.15.59:0/2939071098 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c032440 con 0x7f4aa40b4230 2024-04-23T14:41:36.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa1400640 1 -- 172.21.15.59:0/2939071098 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c032810 con 0x7f4aa40b4230 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 -- 172.21.15.59:0/2939071098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa40b4230 msgr2=0x7f4aa40b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 --2- 172.21.15.59:0/2939071098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa40b4230 0x7f4aa40b4690 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40d1270 tx=0x7f4a8c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 -- 172.21.15.59:0/2939071098 shutdown_connections 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 --2- 172.21.15.59:0/2939071098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40d05c0 0x7f4aa40d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 --2- 172.21.15.59:0/2939071098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa40b4230 0x7f4aa40b4690 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 --2- 172.21.15.59:0/2939071098 >> v2:172.21.15.76:3300/0 conn(0x7f4aa415c670 0x7f4aa40cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 -- 172.21.15.59:0/2939071098 >> 172.21.15.59:0/2939071098 conn(0x7f4aa406bdf0 msgr2=0x7f4aa406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 -- 172.21.15.59:0/2939071098 shutdown_connections 2024-04-23T14:41:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.813+0000 7f4aa3200640 1 -- 172.21.15.59:0/2939071098 wait complete. 2024-04-23T14:41:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.815+0000 7f4aa3200640 1 Processor -- start 2024-04-23T14:41:36.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.815+0000 7f4aa3200640 1 -- start start 2024-04-23T14:41:36.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.816+0000 7f4aa3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40b4230 0x7f4aa41f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.816+0000 7f4aa3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa40d05c0 0x7f4aa41f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.816+0000 7f4aa3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa415c670 0x7f4aa40c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.816+0000 7f4aa3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa40b3e60 con 0x7f4aa415c670 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.816+0000 7f4aa3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa40cd390 con 0x7f4aa40d05c0 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.816+0000 7f4aa3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa40b3fe0 con 0x7f4aa40b4230 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa415c670 0x7f4aa40c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa415c670 0x7f4aa40c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59782/0 (socket says 172.21.15.59:59782) 2024-04-23T14:41:36.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 -- 172.21.15.59:0/3390814968 learned_addr learned my addr 172.21.15.59:0/3390814968 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40b4230 0x7f4aa41f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4a99400640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa40d05c0 0x7f4aa41f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 -- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40b4230 msgr2=0x7f4aa41f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40b4230 0x7f4aa41f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 -- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa40d05c0 msgr2=0x7f4aa41f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa40d05c0 0x7f4aa41f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 -- 172.21.15.59:0/3390814968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa4072790 con 0x7f4aa415c670 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4a99400640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa40d05c0 0x7f4aa41f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.817+0000 7f4aa2800640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa415c670 0x7f4aa40c93d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c007e60 tx=0x7f4a9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:36.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.818+0000 7f4a9aa00640 1 -- 172.21.15.59:0/3390814968 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c014620 con 0x7f4aa415c670 2024-04-23T14:41:36.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.818+0000 7f4a9aa00640 1 -- 172.21.15.59:0/3390814968 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c019070 con 0x7f4aa415c670 2024-04-23T14:41:36.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.818+0000 7f4a9aa00640 1 -- 172.21.15.59:0/3390814968 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c014a20 con 0x7f4aa415c670 2024-04-23T14:41:36.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.818+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa4072a50 con 0x7f4aa415c670 2024-04-23T14:41:36.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.818+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa4073010 con 0x7f4aa415c670 2024-04-23T14:41:36.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.820+0000 7f4a9aa00640 1 -- 172.21.15.59:0/3390814968 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a9c014bc0 con 0x7f4aa415c670 2024-04-23T14:41:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.821+0000 7f4a9aa00640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a8006c8f0 0x7f4a8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.821+0000 7f4a9aa00640 1 -- 172.21.15.59:0/3390814968 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f4a9c08c400 con 0x7f4aa415c670 2024-04-23T14:41:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.822+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a8006c8f0 0x7f4a8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.822+0000 7f4aabe3a740 1 Processor -- start 2024-04-23T14:41:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.822+0000 7f4aabe3a740 1 -- start start 2024-04-23T14:41:36.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.822+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a8006c8f0 0x7f4a8006edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4a94006fd0 tx=0x7f4a94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d24310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 0x563de2d53e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54360 0x563de2d29170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563de2c8eb10 con 0x563de2d54360 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563de2c352b0 con 0x563de2d71980 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563de2c8aad0 con 0x563de2d54aa0 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aa2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54360 0x563de2d29170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aa2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54360 0x563de2d29170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59796/0 (socket says 172.21.15.59:59796) 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.823+0000 7f4aa2800640 1 -- 172.21.15.59:0/1324812283 learned_addr learned my addr 172.21.15.59:0/1324812283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d24310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4a99400640 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 0x563de2d53e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4aa2800640 1 -- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 msgr2=0x563de2d53e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4aa2800640 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 0x563de2d53e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4aa2800640 1 -- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 msgr2=0x563de2d24310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4aa2800640 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d24310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4aa2800640 1 -- 172.21.15.59:0/1324812283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563de29a12d0 con 0x563de2d54360 2024-04-23T14:41:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4aa2800640 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54360 0x563de2d29170 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c056580 tx=0x7f4a9c05a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734886296f9fc2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4a92a00640 1 -- 172.21.15.59:0/1324812283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c062d00 con 0x563de2d54360 2024-04-23T14:41:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4a92a00640 1 -- 172.21.15.59:0/1324812283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c058420 con 0x563de2d54360 2024-04-23T14:41:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.824+0000 7f4a92a00640 1 -- 172.21.15.59:0/1324812283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c08c0b0 con 0x563de2d54360 2024-04-23T14:41:36.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 -- 172.21.15.59:0/1324812283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54360 msgr2=0x563de2d29170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54360 0x563de2d29170 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c056580 tx=0x7f4a9c05a8e0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 -- 172.21.15.59:0/1324812283 shutdown_connections 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 0x563de2d53e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54360 0x563de2d29170 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/1324812283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d24310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 -- 172.21.15.59:0/1324812283 >> 172.21.15.59:0/1324812283 conn(0x563de2d26580 msgr2=0x563de2d6ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 -- 172.21.15.59:0/1324812283 shutdown_connections 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.825+0000 7f4aabe3a740 1 -- 172.21.15.59:0/1324812283 wait complete. 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.826+0000 7f4aabe3a740 1 -- start start 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.826+0000 7f4aabe3a740 1 Processor -- start 2024-04-23T14:41:36.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.827+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54aa0 0x563de2d6b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.827+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d832a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.827+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d70f40 0x563de2d7f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.827+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563de2d28730 con 0x563de2d54aa0 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.827+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563de2c48590 con 0x563de2d71980 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.827+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563de2d27b20 con 0x563de2d70f40 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.827+0000 7f4aabe3a740 10 client.0 _ll_register_callbacks cb 0x563de2c2b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54aa0 0x563de2d6b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54aa0 0x563de2d6b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59812/0 (socket says 172.21.15.59:59812) 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 -- 172.21.15.59:0/3029069942 learned_addr learned my addr 172.21.15.59:0/3029069942 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 -- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d70f40 msgr2=0x563de2d7f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa2800640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d70f40 0x563de2d7f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4a99400640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d70f40 0x563de2d7f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 -- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 msgr2=0x563de2d832a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d832a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 -- 172.21.15.59:0/3029069942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563de2a6b6b0 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4a99400640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d832a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54aa0 0x563de2d6b670 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f4a94014740 tx=0x7f4a94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fba31bd336f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.828+0000 7f4a87400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4a87400640 1 -- 172.21.15.59:0/3029069942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94025020 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4a87400640 1 -- 172.21.15.59:0/3029069942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f4a940158e0 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9c058420 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4a87400640 1 -- 172.21.15.59:0/3029069942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f4a9401b070 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4aabe3a740 10 client.8646 fetch_fsmap learned FSMap version 82 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x563de2d7ffd0 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4a87400640 1 -- 172.21.15.59:0/3029069942 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c058420 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.829+0000 7f4a87400640 1 -- 172.21.15.59:0/3029069942 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f4a94015ae0 con 0x563de2d54aa0 2024-04-23T14:41:36.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.830+0000 7f4aabe3a740 10 client.8646 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:41:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.830+0000 7f4aabe3a740 10 client.8646 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:41:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.830+0000 7f4aabe3a740 20 client.8646 populate_metadata read hostname 'smithi059' 2024-04-23T14:41:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.830+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d6f540 0x563de2d8dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.830+0000 7f4aabe3a740 4 client.8646 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:41:36.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.830+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563de2c9c530 con 0x563de2d6f540 2024-04-23T14:41:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.830+0000 7f4a99400640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d6f540 0x563de2d8dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.831+0000 7f4a99400640 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d6f540 0x563de2d8dd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1066b35b4bac4dc server_cookie=21c8b88e7a76eaa5 in_seq=0 out_seq=0 2024-04-23T14:41:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.831+0000 7f4a87400640 10 client.8646 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:41:36.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4a87400640 1 -- 172.21.15.59:0/3029069942 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563de2c9c530 con 0x563de2d6f540 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4a87400640 10 client.8646 handle_command_reply: tid=0 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 client.8646 shutdown 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54aa0 msgr2=0x563de2d6b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54aa0 0x563de2d6b670 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f4a94014740 tx=0x7f4a94004730 comp rx=0 tx=0).stop 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 shutdown_connections 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d70f40 0x563de2d7f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d6f540 0x563de2d8dd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d832a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/3029069942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d54aa0 0x563de2d6b670 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4a87400640 0 client.8646 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 >> 172.21.15.59:0/3029069942 conn(0x563de2d54360 msgr2=0x563de2d28bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 shutdown_connections 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 1 -- 172.21.15.59:0/3029069942 wait complete. 2024-04-23T14:41:36.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.832+0000 7f4aabe3a740 20 client.8646 trim_cache size 0 max 16384 2024-04-23T14:41:36.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.853+0000 7f4aabe3a740 1 Processor -- start 2024-04-23T14:41:36.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.853+0000 7f4aabe3a740 1 -- start start 2024-04-23T14:41:36.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d54aa0 0x563de2d83280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d6b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d71980 0x563de2d934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41ffd90 con 0x563de2d70f40 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41ff0f0 con 0x563de2d54aa0 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa41ff740 con 0x563de2d71980 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4a99400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d6b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4a99400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d6b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59818/0 (socket says 172.21.15.59:59818) 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.854+0000 7f4a99400640 1 -- 172.21.15.59:0/449650099 learned_addr learned my addr 172.21.15.59:0/449650099 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d54aa0 0x563de2d83280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a99400640 1 -- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d71980 msgr2=0x563de2d934f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a99400640 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d71980 0x563de2d934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a99400640 1 -- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d54aa0 msgr2=0x563de2d83280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a99400640 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d54aa0 0x563de2d83280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a99400640 1 -- 172.21.15.59:0/449650099 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563de2d2a0b0 con 0x563de2d70f40 2024-04-23T14:41:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d54aa0 0x563de2d83280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a99400640 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d6b620 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009960 tx=0x7f4a8c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5e485cc64c58f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a92000640 1 -- 172.21.15.59:0/449650099 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c037040 con 0x563de2d70f40 2024-04-23T14:41:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a92000640 1 -- 172.21.15.59:0/449650099 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c004380 con 0x563de2d70f40 2024-04-23T14:41:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.855+0000 7f4a92000640 1 -- 172.21.15.59:0/449650099 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0434a0 con 0x563de2d70f40 2024-04-23T14:41:36.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 -- 172.21.15.59:0/449650099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 msgr2=0x563de2d6b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d6b620 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009960 tx=0x7f4a8c02c000 comp rx=0 tx=0).stop 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 -- 172.21.15.59:0/449650099 shutdown_connections 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d71980 0x563de2d934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d6b620 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/449650099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d54aa0 0x563de2d83280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 -- 172.21.15.59:0/449650099 >> 172.21.15.59:0/449650099 conn(0x563de2d54360 msgr2=0x563de2d6f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 -- 172.21.15.59:0/449650099 shutdown_connections 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.856+0000 7f4aabe3a740 1 -- 172.21.15.59:0/449650099 wait complete. 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.857+0000 7f4aabe3a740 1 -- start start 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.857+0000 7f4aabe3a740 1 Processor -- start 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 0x563de2d52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d72240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4aabe3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d72780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563de2d7f830 con 0x563de2d70f40 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563de2d82aa0 con 0x563de2d71980 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4aabe3a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563de2d53c00 con 0x563de2d54aa0 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4aabe3a740 10 client.0 _ll_register_callbacks cb 0x7f4a9400fd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4a99400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d72240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4a99400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d72240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59832/0 (socket says 172.21.15.59:59832) 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.858+0000 7f4a99400640 1 -- 172.21.15.59:0/374345639 learned_addr learned my addr 172.21.15.59:0/374345639 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a99400640 1 -- 172.21.15.59:0/374345639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 msgr2=0x563de2d52eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a99400640 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 0x563de2d52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4aa2800640 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d72780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a99400640 1 -- 172.21.15.59:0/374345639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 msgr2=0x563de2d72780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a99400640 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d72780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a99400640 1 -- 172.21.15.59:0/374345639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563de2d23fe0 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a99400640 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d72240 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34dacf613f8a638 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a87400640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a87400640 1 -- 172.21.15.59:0/374345639 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c052060 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.859+0000 7f4a87400640 1 -- 172.21.15.59:0/374345639 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f4a8c043460 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a8c004380 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4a87400640 1 -- 172.21.15.59:0/374345639 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c058050 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 10 client.8652 fetch_fsmap learned FSMap version 82 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x563de2d92a90 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4a87400640 1 -- 172.21.15.59:0/374345639 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c004380 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4a87400640 1 -- 172.21.15.59:0/374345639 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f4a8c05b020 con 0x563de2d70f40 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 10 client.8652 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 10 client.8652 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 20 client.8652 populate_metadata read hostname 'smithi059' 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d652c0 0x563de2de8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:36.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 4 client.8652 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:41:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.860+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "status"}) -- 0x563de2d66b00 con 0x563de2d652c0 2024-04-23T14:41:36.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.861+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d652c0 0x563de2de8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.861+0000 7f4aa1e00640 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d652c0 0x563de2de8dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6058b6a33f1bdf30 server_cookie=174060a52a10f9d8 in_seq=0 out_seq=0 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.861+0000 7f4a87400640 10 client.8652 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.861+0000 7f4a87400640 1 -- 172.21.15.59:0/374345639 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563de2d66b00 con 0x563de2d652c0 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.861+0000 7f4a87400640 10 client.8652 handle_command_reply: tid=0 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.861+0000 7f4aabe3a740 1 client.8652 shutdown 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 msgr2=0x563de2d72240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d72240 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c004730 comp rx=0 tx=0).stop 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 shutdown_connections 2024-04-23T14:41:36.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563de2d652c0 0x563de2de8dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563de2d71980 0x563de2d72780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563de2d70f40 0x563de2d72240 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 --2- 172.21.15.59:0/374345639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563de2d54aa0 0x563de2d52eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4a87400640 0 client.8652 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 >> 172.21.15.59:0/374345639 conn(0x563de2d54360 msgr2=0x563de2d2a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 shutdown_connections 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 1 -- 172.21.15.59:0/374345639 wait complete. 2024-04-23T14:41:36.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.862+0000 7f4aabe3a740 20 client.8652 trim_cache size 0 max 16384 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8514, 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7008075170000003, 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 82, 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 99, 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 99, 2024-04-23T14:41:36.867 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 13.140337202 2024-04-23T14:41:36.868 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:41:36.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a8006c8f0 msgr2=0x7f4a8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a8006c8f0 0x7f4a8006edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4a94006fd0 tx=0x7f4a94008040 comp rx=0 tx=0).stop 2024-04-23T14:41:36.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa415c670 msgr2=0x7f4aa40c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:36.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa415c670 0x7f4aa40c93d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c007e60 tx=0x7f4a9c00e4b0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 shutdown_connections 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a8006c8f0 0x7f4a8006edb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aa415c670 0x7f4aa40c93d0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa40d05c0 0x7f4aa41f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 --2- 172.21.15.59:0/3390814968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4aa40b4230 0x7f4aa41f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 >> 172.21.15.59:0/3390814968 conn(0x7f4aa406bdf0 msgr2=0x7f4aa41556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 shutdown_connections 2024-04-23T14:41:36.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:36.863+0000 7f4aa3200640 1 -- 172.21.15.59:0/3390814968 wait complete. 2024-04-23T14:41:36.881 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:41:37.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.094+0000 7f851f200640 1 Processor -- start 2024-04-23T14:41:37.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.095+0000 7f851f200640 1 -- start start 2024-04-23T14:41:37.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.096+0000 7f851f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f852015e7c0 0x7f85200d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:37.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.096+0000 7f851f200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f85200d0d90 0x7f85200d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:37.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.096+0000 7f851f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85200cf030 con 0x7f85200d2630 2024-04-23T14:41:37.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.096+0000 7f851f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f85200b7000 con 0x7f852015e7c0 2024-04-23T14:41:37.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.096+0000 7f851f200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f85200cf620 con 0x7f85200d0d90 2024-04-23T14:41:37.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.096+0000 7f851de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f852015e7c0 0x7f85200d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851d400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f85200d2630 0x7f85200b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55672/0 (socket says 172.21.15.59:55672) 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851d400640 1 -- 172.21.15.59:0/2086429084 learned_addr learned my addr 172.21.15.59:0/2086429084 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851e800640 1 --2- 172.21.15.59:0/2086429084 >> v2:172.21.15.76:3300/0 conn(0x7f85200d0d90 0x7f85200d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851de00640 1 -- 172.21.15.59:0/2086429084 >> v2:172.21.15.76:3300/0 conn(0x7f85200d0d90 msgr2=0x7f85200d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851de00640 1 --2- 172.21.15.59:0/2086429084 >> v2:172.21.15.76:3300/0 conn(0x7f85200d0d90 0x7f85200d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851de00640 1 -- 172.21.15.59:0/2086429084 >> v1:172.21.15.59:6789/0 conn(0x7f85200d2630 legacy=0x7f85200b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851de00640 1 -- 172.21.15.59:0/2086429084 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85200b6a40 con 0x7f852015e7c0 2024-04-23T14:41:37.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.097+0000 7f851ca00640 1 -- 172.21.15.59:0/2086429084 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4259471024 0 0) 0x7f850c001490 con 0x7f85200d2630 2024-04-23T14:41:37.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.098+0000 7f851de00640 1 --2- 172.21.15.59:0/2086429084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200d20f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f85200cb580 tx=0x7f851002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12b618369744407 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:37.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.098+0000 7f851ca00640 1 -- 172.21.15.59:0/2086429084 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510037070 con 0x7f852015e7c0 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851ca00640 1 -- 172.21.15.59:0/2086429084 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8510032440 con 0x7f852015e7c0 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851ca00640 1 -- 172.21.15.59:0/2086429084 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510032810 con 0x7f852015e7c0 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 -- 172.21.15.59:0/2086429084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 msgr2=0x7f85200d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 --2- 172.21.15.59:0/2086429084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200d20f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f85200cb580 tx=0x7f851002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 -- 172.21.15.59:0/2086429084 shutdown_connections 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 --2- 172.21.15.59:0/2086429084 >> v2:172.21.15.76:3300/0 conn(0x7f85200d0d90 0x7f85200d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 --2- 172.21.15.59:0/2086429084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200d20f0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 -- 172.21.15.59:0/2086429084 >> 172.21.15.59:0/2086429084 conn(0x7f852009d920 msgr2=0x7f852008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 -- 172.21.15.59:0/2086429084 shutdown_connections 2024-04-23T14:41:37.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.099+0000 7f851f200640 1 -- 172.21.15.59:0/2086429084 wait complete. 2024-04-23T14:41:37.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.101+0000 7f851f200640 1 Processor -- start 2024-04-23T14:41:37.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.101+0000 7f851f200640 1 -- start start 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.102+0000 7f851f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85200d0d90 0x7f85200ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.102+0000 7f851f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85200d2630 0x7f852015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.102+0000 7f851f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.102+0000 7f851f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f85200cf2b0 con 0x7f852015e7c0 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.102+0000 7f851f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85200d1700 con 0x7f85200d2630 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.102+0000 7f851f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f85200b6700 con 0x7f85200d0d90 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:37.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59852/0 (socket says 172.21.15.59:59852) 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 -- 172.21.15.59:0/1448334998 learned_addr learned my addr 172.21.15.59:0/1448334998 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851d400640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85200d2630 0x7f852015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851de00640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85200d0d90 0x7f85200ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 -- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85200d0d90 msgr2=0x7f85200ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85200d0d90 0x7f85200ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 -- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85200d2630 msgr2=0x7f852015d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85200d2630 0x7f852015d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 -- 172.21.15.59:0/1448334998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85200ccf70 con 0x7f852015e7c0 2024-04-23T14:41:37.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.103+0000 7f851e800640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200cc590 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f851800d600 tx=0x7f851800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:37.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.104+0000 7f8516000640 1 -- 172.21.15.59:0/1448334998 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85180125f0 con 0x7f852015e7c0 2024-04-23T14:41:37.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.104+0000 7f8516000640 1 -- 172.21.15.59:0/1448334998 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8518019070 con 0x7f852015e7c0 2024-04-23T14:41:37.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.104+0000 7f8516000640 1 -- 172.21.15.59:0/1448334998 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85180129f0 con 0x7f852015e7c0 2024-04-23T14:41:37.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.104+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85200cd250 con 0x7f852015e7c0 2024-04-23T14:41:37.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.104+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85200cd810 con 0x7f852015e7c0 2024-04-23T14:41:37.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.105+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852004fd90 con 0x7f852015e7c0 2024-04-23T14:41:37.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.106+0000 7f8516000640 1 -- 172.21.15.59:0/1448334998 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8518012b90 con 0x7f852015e7c0 2024-04-23T14:41:37.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.107+0000 7f8516000640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f850006c970 0x7f850006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:37.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.107+0000 7f8516000640 1 -- 172.21.15.59:0/1448334998 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f851808bec0 con 0x7f852015e7c0 2024-04-23T14:41:37.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.108+0000 7f851de00640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f850006c970 0x7f850006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:37.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.108+0000 7f851de00640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f850006c970 0x7f850006ee30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8510034000 tx=0x7f851002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:37.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.112+0000 7f8516000640 1 -- 172.21.15.59:0/1448334998 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f851805a2e0 con 0x7f852015e7c0 2024-04-23T14:41:37.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.362+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f85200d1eb0 con 0x7f852015e7c0 2024-04-23T14:41:37.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.362+0000 7f8516000640 1 -- 172.21.15.59:0/1448334998 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f8518059be0 con 0x7f852015e7c0 2024-04-23T14:41:37.363 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:41:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f850006c970 msgr2=0x7f850006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f850006c970 0x7f850006ee30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8510034000 tx=0x7f851002e000 comp rx=0 tx=0).stop 2024-04-23T14:41:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 msgr2=0x7f85200cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200cc590 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f851800d600 tx=0x7f851800dac0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 shutdown_connections 2024-04-23T14:41:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f850006c970 0x7f850006ee30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f852015e7c0 0x7f85200cc590 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85200d2630 0x7f852015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 --2- 172.21.15.59:0/1448334998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85200d0d90 0x7f85200ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.365+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 >> 172.21.15.59:0/1448334998 conn(0x7f852009d920 msgr2=0x7f852008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:37.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.366+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 shutdown_connections 2024-04-23T14:41:37.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37.366+0000 7f851f200640 1 -- 172.21.15.59:0/1448334998 wait complete. 2024-04-23T14:41:37.386 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:41:37.386 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:41:37.386 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:41:37.386 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:41:37.386 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:37.387 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:41:37.387 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:41:37.387 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:41:37.387 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:41:37.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:37.387 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:37.443 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41851sec preferred_lft 41851sec 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:41:37.444 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:37.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:37.446 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:41:37.534 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:41:37.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:37.537 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:41:37.594 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:41:37.595 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:41:37.595 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:41:37.595 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:41:37.595 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:41:37.595 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:41:37.595 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:41:37.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:41:37.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:41:37.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:41:37.595 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:41:37.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:41:37.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:37.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:37.825 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:41:37.876 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:41:37.876 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:41:37.876 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:41:37.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:37.877 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:41:37.963 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:41:37.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:41:37.963 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:38.020 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:41:38.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:41:38.020 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:38.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:41:38.165 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:41:38.223 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:41:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:38.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:38.226 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:41:38.271 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:41:38.272 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:41:38.319 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:41:38.393 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:41:38.437 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:41:38.438 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:41:38.439 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:41:38.440 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:41:38.440 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:38.440 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:38.440 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:38.440 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:38.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:38.441 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:41:38.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:41:38.499+0000 7fbecbe14480 -1 init, newargv = 0x55f5a3133390 newargc=15 2024-04-23T14:41:38.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[70092]: starting ceph client 2024-04-23T14:41:38.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[70092]: starting fuse 2024-04-23T14:41:38.528 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:41:38.528 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:41:38.528 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:41:38.528 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:41:38.528 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:38.529 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:41:38.530 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:41:38.724 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.70092.asok 2024-04-23T14:41:38.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:38.741 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.70092.asok 2024-04-23T14:41:38.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:38.741 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.70092.asok status 2024-04-23T14:41:38.971 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:41:38.971 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:41:38.971 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:41:38.971 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "70092", 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8667, 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 8667 2024-04-23T14:41:38.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 78175599 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 78175599 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.8667 192.168.0.1:0/78175599", 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/78175599", 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:41:38.973 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 82, 2024-04-23T14:41:38.974 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 99, 2024-04-23T14:41:38.974 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:41:38.974 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:41:38.974 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:41:38.974 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:41:38.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:38.987 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:41:38.987 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:39.042 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:41:39.042 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:39.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:39.042 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:39.134 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:39.134 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:39.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:39.193 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-23T14:41:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records in 2024-04-23T14:41:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records out 2024-04-23T14:41:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.185254 s, 34.0 MB/s 2024-04-23T14:41:39.457 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:39.457 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:39.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:39.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.727+0000 7f2d4f200640 1 Processor -- start 2024-04-23T14:41:39.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.727+0000 7f2d4f200640 1 -- start start 2024-04-23T14:41:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.728+0000 7f2d4f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500b5ff0 0x7f2d500b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.728+0000 7f2d4f200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2d500d0d90 0x7f2d500d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.728+0000 7f2d4f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d500cf030 con 0x7f2d5015e7c0 2024-04-23T14:41:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.728+0000 7f2d4f200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2d500b7000 con 0x7f2d500d0d90 2024-04-23T14:41:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.728+0000 7f2d4f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2d500cf620 con 0x7f2d500b5ff0 2024-04-23T14:41:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.729+0000 7f2d4de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2d5015e7c0 0x7f2d500d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55680/0 (socket says 172.21.15.59:55680) 2024-04-23T14:41:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.729+0000 7f2d4de00640 1 -- 172.21.15.59:0/3491972861 learned_addr learned my addr 172.21.15.59:0/3491972861 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.729+0000 7f2d4d400640 1 --2- 172.21.15.59:0/3491972861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500b5ff0 0x7f2d500b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.729+0000 7f2d4e800640 1 --2- 172.21.15.59:0/3491972861 >> v2:172.21.15.76:3300/0 conn(0x7f2d500d0d90 0x7f2d500d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.729+0000 7f2d4ca00640 1 -- 172.21.15.59:0/3491972861 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339158299 0 0) 0x7f2d40001110 con 0x7f2d5015e7c0 2024-04-23T14:41:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.729+0000 7f2d4ca00640 1 -- 172.21.15.59:0/3491972861 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d30003520 con 0x7f2d5015e7c0 2024-04-23T14:41:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.730+0000 7f2d4d400640 1 -- 172.21.15.59:0/3491972861 >> v2:172.21.15.76:3300/0 conn(0x7f2d500d0d90 msgr2=0x7f2d500d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.730+0000 7f2d4d400640 1 --2- 172.21.15.59:0/3491972861 >> v2:172.21.15.76:3300/0 conn(0x7f2d500d0d90 0x7f2d500d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.730+0000 7f2d4d400640 1 -- 172.21.15.59:0/3491972861 >> v1:172.21.15.59:6789/0 conn(0x7f2d5015e7c0 legacy=0x7f2d500d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:39.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.730+0000 7f2d4ca00640 1 -- 172.21.15.59:0/3491972861 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 529031474 0 0) 0x7f2d30003520 con 0x7f2d5015e7c0 2024-04-23T14:41:39.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.730+0000 7f2d4d400640 1 -- 172.21.15.59:0/3491972861 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d500b6a40 con 0x7f2d500b5ff0 2024-04-23T14:41:39.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.731+0000 7f2d4d400640 1 --2- 172.21.15.59:0/3491972861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500b5ff0 0x7f2d500b6450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009d90 tx=0x7f2d3802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2502564e64d785ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.732+0000 7f2d4ca00640 1 -- 172.21.15.59:0/3491972861 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40001840 con 0x7f2d500b5ff0 2024-04-23T14:41:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.732+0000 7f2d4ca00640 1 -- 172.21.15.59:0/3491972861 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d38007540 con 0x7f2d500b5ff0 2024-04-23T14:41:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.732+0000 7f2d4ca00640 1 -- 172.21.15.59:0/3491972861 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38007910 con 0x7f2d500b5ff0 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.732+0000 7f2d4f200640 1 -- 172.21.15.59:0/3491972861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500b5ff0 msgr2=0x7f2d500b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.732+0000 7f2d4f200640 1 --2- 172.21.15.59:0/3491972861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500b5ff0 0x7f2d500b6450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009d90 tx=0x7f2d3802a5f0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.733+0000 7f2d4f200640 1 -- 172.21.15.59:0/3491972861 shutdown_connections 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.733+0000 7f2d4f200640 1 --2- 172.21.15.59:0/3491972861 >> v2:172.21.15.76:3300/0 conn(0x7f2d500d0d90 0x7f2d500d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.733+0000 7f2d4f200640 1 --2- 172.21.15.59:0/3491972861 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500b5ff0 0x7f2d500b6450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.733+0000 7f2d4f200640 1 -- 172.21.15.59:0/3491972861 >> 172.21.15.59:0/3491972861 conn(0x7f2d5009d920 msgr2=0x7f2d5008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.733+0000 7f2d4f200640 1 -- 172.21.15.59:0/3491972861 shutdown_connections 2024-04-23T14:41:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.733+0000 7f2d4f200640 1 -- 172.21.15.59:0/3491972861 wait complete. 2024-04-23T14:41:39.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.734+0000 7f2d4f200640 1 Processor -- start 2024-04-23T14:41:39.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.735+0000 7f2d4f200640 1 -- start start 2024-04-23T14:41:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d500b5ff0 0x7f2d500b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500d0d90 0x7f2d5014f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5015e7c0 0x7f2d500b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d500d2800 con 0x7f2d500b5ff0 2024-04-23T14:41:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d500cf2b0 con 0x7f2d5015e7c0 2024-04-23T14:41:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2d500d1a80 con 0x7f2d500d0d90 2024-04-23T14:41:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d500b5ff0 0x7f2d500b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d500b5ff0 0x7f2d500b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59854/0 (socket says 172.21.15.59:59854) 2024-04-23T14:41:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4de00640 1 -- 172.21.15.59:0/854993018 learned_addr learned my addr 172.21.15.59:0/854993018 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4d400640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500d0d90 0x7f2d5014f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4e800640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5015e7c0 0x7f2d500b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.736+0000 7f2d4de00640 1 -- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500d0d90 msgr2=0x7f2d5014f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d4de00640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500d0d90 0x7f2d5014f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d4de00640 1 -- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5015e7c0 msgr2=0x7f2d500b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d4de00640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5015e7c0 0x7f2d500b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d4de00640 1 -- 172.21.15.59:0/854993018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d500b1ef0 con 0x7f2d500b5ff0 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d4de00640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d500b5ff0 0x7f2d500b2f90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c4d0 tx=0x7f2d4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d46000640 1 -- 172.21.15.59:0/854993018 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40017070 con 0x7f2d500b5ff0 2024-04-23T14:41:39.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d46000640 1 -- 172.21.15.59:0/854993018 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d40005ce0 con 0x7f2d500b5ff0 2024-04-23T14:41:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d46000640 1 -- 172.21.15.59:0/854993018 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40007310 con 0x7f2d500b5ff0 2024-04-23T14:41:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d500b21d0 con 0x7f2d500b5ff0 2024-04-23T14:41:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.737+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d500b2790 con 0x7f2d500b5ff0 2024-04-23T14:41:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.739+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5004fd90 con 0x7f2d500b5ff0 2024-04-23T14:41:39.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.740+0000 7f2d46000640 1 -- 172.21.15.59:0/854993018 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d4001a070 con 0x7f2d500b5ff0 2024-04-23T14:41:39.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.741+0000 7f2d46000640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d2c06c920 0x7f2d2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:39.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.741+0000 7f2d46000640 1 -- 172.21.15.59:0/854993018 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f2d4008af40 con 0x7f2d500b5ff0 2024-04-23T14:41:39.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.741+0000 7f2d4d400640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d2c06c920 0x7f2d2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:39.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.742+0000 7f2d4d400640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d2c06c920 0x7f2d2c06ede0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2d38034000 tx=0x7f2d3802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:39.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.746+0000 7f2d46000640 1 -- 172.21.15.59:0/854993018 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d500cf030 con 0x7f2d500b5ff0 2024-04-23T14:41:39.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.954+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d500ba410 con 0x7f2d500b5ff0 2024-04-23T14:41:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.956+0000 7f2d46000640 1 -- 172.21.15.59:0/854993018 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16548 (secure 0 0 0) 0x7f2d40059400 con 0x7f2d500b5ff0 2024-04-23T14:41:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:39.957 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":99,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:31.158736+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d2c06c920 msgr2=0x7f2d2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d2c06c920 0x7f2d2c06ede0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2d38034000 tx=0x7f2d3802f000 comp rx=0 tx=0).stop 2024-04-23T14:41:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d500b5ff0 msgr2=0x7f2d500b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d500b5ff0 0x7f2d500b2f90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c4d0 tx=0x7f2d4000c990 comp rx=0 tx=0).stop 2024-04-23T14:41:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 shutdown_connections 2024-04-23T14:41:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2d2c06c920 0x7f2d2c06ede0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d500d0d90 0x7f2d5014f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5015e7c0 0x7f2d500b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 --2- 172.21.15.59:0/854993018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d500b5ff0 0x7f2d500b2f90 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 >> 172.21.15.59:0/854993018 conn(0x7f2d5009d920 msgr2=0x7f2d500b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 shutdown_connections 2024-04-23T14:41:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:39.959+0000 7f2d4f200640 1 -- 172.21.15.59:0/854993018 wait complete. 2024-04-23T14:41:39.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:41:27.465708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:41:28.586680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:39.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.199+0000 7f4962200640 1 Processor -- start 2024-04-23T14:41:40.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.199+0000 7f4962200640 1 -- start start 2024-04-23T14:41:40.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.201+0000 7f4962200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f495c1567e0 0x7f495c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.201+0000 7f4962200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c15cab0 con 0x7f495c0d0d10 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.201+0000 7f4962200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c160e40 con 0x7f495c15d810 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.201+0000 7f4962200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f495c15cd30 con 0x7f495c1567e0 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.202+0000 7f4960e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f495c1567e0 0x7f495c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.202+0000 7f4961800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f495c0d0d10 0x7f495c160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56444/0 (socket says 172.21.15.59:56444) 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.202+0000 7f4961800640 1 -- 172.21.15.59:0/3722041054 learned_addr learned my addr 172.21.15.59:0/3722041054 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.202+0000 7f4960e00640 1 -- 172.21.15.59:0/3722041054 >> v1:172.21.15.76:6789/0 conn(0x7f495c15d810 legacy=0x7f495c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:40.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.202+0000 7f4960e00640 1 -- 172.21.15.59:0/3722041054 >> v1:172.21.15.59:6789/0 conn(0x7f495c0d0d10 legacy=0x7f495c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.202+0000 7f4960e00640 1 -- 172.21.15.59:0/3722041054 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c15d010 con 0x7f495c1567e0 2024-04-23T14:41:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.204+0000 7f4960e00640 1 --2- 172.21.15.59:0/3722041054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c1567e0 0x7f495c156bc0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f49500097a0 tx=0x7f495002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9118abe9911c8e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.205+0000 7f495b400640 1 -- 172.21.15.59:0/3722041054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950037070 con 0x7f495c1567e0 2024-04-23T14:41:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.205+0000 7f495b400640 1 -- 172.21.15.59:0/3722041054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4950032440 con 0x7f495c1567e0 2024-04-23T14:41:40.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.205+0000 7f495b400640 1 -- 172.21.15.59:0/3722041054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950032810 con 0x7f495c1567e0 2024-04-23T14:41:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.205+0000 7f4962200640 1 -- 172.21.15.59:0/3722041054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c1567e0 msgr2=0x7f495c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.205+0000 7f4962200640 1 --2- 172.21.15.59:0/3722041054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c1567e0 0x7f495c156bc0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f49500097a0 tx=0x7f495002a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.206+0000 7f4962200640 1 -- 172.21.15.59:0/3722041054 shutdown_connections 2024-04-23T14:41:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.206+0000 7f4962200640 1 --2- 172.21.15.59:0/3722041054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c1567e0 0x7f495c156bc0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.206+0000 7f4962200640 1 -- 172.21.15.59:0/3722041054 >> 172.21.15.59:0/3722041054 conn(0x7f495c06bdf0 msgr2=0x7f495c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.206+0000 7f4962200640 1 -- 172.21.15.59:0/3722041054 shutdown_connections 2024-04-23T14:41:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.206+0000 7f4962200640 1 -- 172.21.15.59:0/3722041054 wait complete. 2024-04-23T14:41:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.207+0000 7f4962200640 1 Processor -- start 2024-04-23T14:41:40.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.208+0000 7f4962200640 1 -- start start 2024-04-23T14:41:40.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.208+0000 7f4962200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f495c0d0d10 0x7f495c0997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.208+0000 7f4962200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f495c1567e0 0x7f495c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.208+0000 7f4962200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c15d810 0x7f495c158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4962200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1609a0 con 0x7f495c15d810 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4962200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f495c160820 con 0x7f495c1567e0 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4962200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f495c160b20 con 0x7f495c0d0d10 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c15d810 0x7f495c158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c15d810 0x7f495c158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49276/0 (socket says 172.21.15.59:49276) 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 -- 172.21.15.59:0/1639705128 learned_addr learned my addr 172.21.15.59:0/1639705128 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4960e00640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f495c0d0d10 0x7f495c0997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f495be00640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f495c1567e0 0x7f495c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 -- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f495c0d0d10 msgr2=0x7f495c0997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f495c0d0d10 0x7f495c0997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 -- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f495c1567e0 msgr2=0x7f495c14f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f495c1567e0 0x7f495c14f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.209+0000 7f4961800640 1 -- 172.21.15.59:0/1639705128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c14bb20 con 0x7f495c15d810 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.210+0000 7f4961800640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c15d810 0x7f495c158800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f495400ca80 tx=0x7f495400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.210+0000 7f4958c00640 1 -- 172.21.15.59:0/1639705128 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954017070 con 0x7f495c15d810 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.210+0000 7f4958c00640 1 -- 172.21.15.59:0/1639705128 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495400ac30 con 0x7f495c15d810 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.210+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c157d20 con 0x7f495c15d810 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.210+0000 7f4958c00640 1 -- 172.21.15.59:0/1639705128 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954012600 con 0x7f495c15d810 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.210+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c099cf0 con 0x7f495c15d810 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.212+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c0d08b0 con 0x7f495c15d810 2024-04-23T14:41:40.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.213+0000 7f4958c00640 1 -- 172.21.15.59:0/1639705128 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f495401a070 con 0x7f495c15d810 2024-04-23T14:41:40.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.214+0000 7f4958c00640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f493c06c8f0 0x7f493c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.214+0000 7f4958c00640 1 -- 172.21.15.59:0/1639705128 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f495408b7a0 con 0x7f495c15d810 2024-04-23T14:41:40.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.214+0000 7f4960e00640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f493c06c8f0 0x7f493c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.215+0000 7f4960e00640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f493c06c8f0 0x7f493c06edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4950034000 tx=0x7f495002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:40.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.218+0000 7f4958c00640 1 -- 172.21.15.59:0/1639705128 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f495c15cab0 con 0x7f495c15d810 2024-04-23T14:41:40.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.428+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f495c04fd90 con 0x7f495c15d810 2024-04-23T14:41:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.429+0000 7f4958c00640 1 -- 172.21.15.59:0/1639705128 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f495400ae20 con 0x7f495c15d810 2024-04-23T14:41:40.430 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:40.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f493c06c8f0 msgr2=0x7f493c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f493c06c8f0 0x7f493c06edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4950034000 tx=0x7f495002e000 comp rx=0 tx=0).stop 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c15d810 msgr2=0x7f495c158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c15d810 0x7f495c158800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f495400ca80 tx=0x7f495400cf40 comp rx=0 tx=0).stop 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 shutdown_connections 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f493c06c8f0 0x7f493c06edb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f495c15d810 0x7f495c158800 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f495c1567e0 0x7f495c14f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 --2- 172.21.15.59:0/1639705128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f495c0d0d10 0x7f495c0997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 >> 172.21.15.59:0/1639705128 conn(0x7f495c06bdf0 msgr2=0x7f495c15dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 shutdown_connections 2024-04-23T14:41:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.432+0000 7f4962200640 1 -- 172.21.15.59:0/1639705128 wait complete. 2024-04-23T14:41:40.453 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:41:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.662+0000 7f61db200640 1 Processor -- start 2024-04-23T14:41:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.663+0000 7f61db200640 1 -- start start 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.664+0000 7f61db200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61dc155a80 0x7f61dc06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.664+0000 7f61db200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61dc0af2a0 0x7f61dc0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.664+0000 7f61db200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61dc15ef20 con 0x7f61dc0d0600 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.664+0000 7f61db200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f61dc15f1c0 con 0x7f61dc155a80 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.664+0000 7f61db200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f61dc15f4a0 con 0x7f61dc0af2a0 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61dc155a80 0x7f61dc06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d1400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f61dc0d0600 0x7f61dc0b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56448/0 (socket says 172.21.15.59:56448) 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d1400640 1 -- 172.21.15.59:0/1757259029 learned_addr learned my addr 172.21.15.59:0/1757259029 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61dc155a80 0x7f61dc06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49292/0 (socket says 172.21.15.59:49292) 2024-04-23T14:41:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61da800640 1 --2- 172.21.15.59:0/1757259029 >> v2:172.21.15.76:3300/0 conn(0x7f61dc0af2a0 0x7f61dc0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d9e00640 1 -- 172.21.15.59:0/1757259029 >> v2:172.21.15.76:3300/0 conn(0x7f61dc0af2a0 msgr2=0x7f61dc0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d9e00640 1 --2- 172.21.15.59:0/1757259029 >> v2:172.21.15.76:3300/0 conn(0x7f61dc0af2a0 0x7f61dc0d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d9e00640 1 -- 172.21.15.59:0/1757259029 >> v1:172.21.15.59:6789/0 conn(0x7f61dc0d0600 legacy=0x7f61dc0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d9e00640 1 -- 172.21.15.59:0/1757259029 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc15f780 con 0x7f61dc155a80 2024-04-23T14:41:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.665+0000 7f61d9400640 1 -- 172.21.15.59:0/1757259029 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924508699 0 0) 0x7f61dc15ef20 con 0x7f61dc0d0600 2024-04-23T14:41:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.666+0000 7f61d9e00640 1 --2- 172.21.15.59:0/1757259029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc155a80 0x7f61dc06f9c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f61dc06df60 tx=0x7f61cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f873eceb23d2b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.667+0000 7f61d9400640 1 -- 172.21.15.59:0/1757259029 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc037070 con 0x7f61dc155a80 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.667+0000 7f61d9400640 1 -- 172.21.15.59:0/1757259029 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61cc032440 con 0x7f61dc155a80 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.667+0000 7f61d9400640 1 -- 172.21.15.59:0/1757259029 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc032810 con 0x7f61dc155a80 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.667+0000 7f61db200640 1 -- 172.21.15.59:0/1757259029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc155a80 msgr2=0x7f61dc06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.667+0000 7f61db200640 1 --2- 172.21.15.59:0/1757259029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc155a80 0x7f61dc06f9c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f61dc06df60 tx=0x7f61cc02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.667+0000 7f61db200640 1 -- 172.21.15.59:0/1757259029 shutdown_connections 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.667+0000 7f61db200640 1 --2- 172.21.15.59:0/1757259029 >> v2:172.21.15.76:3300/0 conn(0x7f61dc0af2a0 0x7f61dc0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.668+0000 7f61db200640 1 --2- 172.21.15.59:0/1757259029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc155a80 0x7f61dc06f9c0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.668+0000 7f61db200640 1 -- 172.21.15.59:0/1757259029 >> 172.21.15.59:0/1757259029 conn(0x7f61dc06bdf0 msgr2=0x7f61dc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.668+0000 7f61db200640 1 -- 172.21.15.59:0/1757259029 shutdown_connections 2024-04-23T14:41:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.668+0000 7f61db200640 1 -- 172.21.15.59:0/1757259029 wait complete. 2024-04-23T14:41:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.669+0000 7f61db200640 1 Processor -- start 2024-04-23T14:41:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.670+0000 7f61db200640 1 -- start start 2024-04-23T14:41:40.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61db200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc0af2a0 0x7f61dc0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61db200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61dc0d0600 0x7f61dc154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61db200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61dc155a80 0x7f61dc15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61db200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc06ff00 con 0x7f61dc0af2a0 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61db200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc15fd40 con 0x7f61dc155a80 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61db200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f61dc06ef80 con 0x7f61dc0d0600 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc0af2a0 0x7f61dc0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc0af2a0 0x7f61dc0b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49302/0 (socket says 172.21.15.59:49302) 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 -- 172.21.15.59:0/545555998 learned_addr learned my addr 172.21.15.59:0/545555998 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 -- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61dc0d0600 msgr2=0x7f61dc154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d1400640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61dc0d0600 0x7f61dc154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61dc0d0600 0x7f61dc154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61da800640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61dc155a80 0x7f61dc15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 -- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61dc155a80 msgr2=0x7f61dc15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61dc155a80 0x7f61dc15a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.671+0000 7f61d9e00640 1 -- 172.21.15.59:0/545555998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc154140 con 0x7f61dc0af2a0 2024-04-23T14:41:40.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.672+0000 7f61d9e00640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc0af2a0 0x7f61dc0b1790 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f61cc000c00 tx=0x7f61cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:40.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.672+0000 7f61d2a00640 1 -- 172.21.15.59:0/545555998 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc037040 con 0x7f61dc0af2a0 2024-04-23T14:41:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.672+0000 7f61d2a00640 1 -- 172.21.15.59:0/545555998 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61cc042070 con 0x7f61dc0af2a0 2024-04-23T14:41:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.672+0000 7f61d2a00640 1 -- 172.21.15.59:0/545555998 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc030a50 con 0x7f61dc0af2a0 2024-04-23T14:41:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.672+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc09e070 con 0x7f61dc0af2a0 2024-04-23T14:41:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.672+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1546c0 con 0x7f61dc0af2a0 2024-04-23T14:41:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.673+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc04fd90 con 0x7f61dc0af2a0 2024-04-23T14:41:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.674+0000 7f61d2a00640 1 -- 172.21.15.59:0/545555998 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61cc030bf0 con 0x7f61dc0af2a0 2024-04-23T14:41:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.675+0000 7f61d2a00640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f61b806c8f0 0x7f61b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.676+0000 7f61d2a00640 1 -- 172.21.15.59:0/545555998 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f61cc0ac5f0 con 0x7f61dc0af2a0 2024-04-23T14:41:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.676+0000 7f61d1400640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f61b806c8f0 0x7f61b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.676+0000 7f61d1400640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f61b806c8f0 0x7f61b806edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61c4007fd0 tx=0x7f61c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:40.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.680+0000 7f61d2a00640 1 -- 172.21.15.59:0/545555998 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61cc07aa10 con 0x7f61dc0af2a0 2024-04-23T14:41:40.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.889+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f61dc06eba0 con 0x7f61dc0af2a0 2024-04-23T14:41:40.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.890+0000 7f61d2a00640 1 -- 172.21.15.59:0/545555998 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f61cc07a310 con 0x7f61dc0af2a0 2024-04-23T14:41:40.891 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f61b806c8f0 msgr2=0x7f61b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f61b806c8f0 0x7f61b806edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61c4007fd0 tx=0x7f61c4009040 comp rx=0 tx=0).stop 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc0af2a0 msgr2=0x7f61dc0b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc0af2a0 0x7f61dc0b1790 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f61cc000c00 tx=0x7f61cc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 shutdown_connections 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f61b806c8f0 0x7f61b806edb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61dc0d0600 0x7f61dc154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61dc155a80 0x7f61dc15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 --2- 172.21.15.59:0/545555998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61dc0af2a0 0x7f61dc0b1790 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:40.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.892+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 >> 172.21.15.59:0/545555998 conn(0x7f61dc06bdf0 msgr2=0x7f61dc0b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:40.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.893+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 shutdown_connections 2024-04-23T14:41:40.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:40.893+0000 7f61db200640 1 -- 172.21.15.59:0/545555998 wait complete. 2024-04-23T14:41:40.913 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:41:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.123+0000 7fdfe9600640 1 Processor -- start 2024-04-23T14:41:41.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.123+0000 7fdfe9600640 1 -- start start 2024-04-23T14:41:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.124+0000 7fdfe9600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdfe4155a80 0x7fdfe406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.124+0000 7fdfe9600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe40d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.124+0000 7fdfe9600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfe415ef20 con 0x7fdfe40b43a0 2024-04-23T14:41:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.124+0000 7fdfe9600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe40d0fa0 con 0x7fdfe4155a80 2024-04-23T14:41:41.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.124+0000 7fdfe9600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe415f1c0 con 0x7fdfe40d0600 2024-04-23T14:41:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.125+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe40d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.125+0000 7fdfe3e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdfe4155a80 0x7fdfe406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.125+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe40d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:54364/0 (socket says 172.21.15.59:54364) 2024-04-23T14:41:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.125+0000 7fdfe3e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdfe4155a80 0x7fdfe406f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:38352/0 (socket says 172.21.15.59:38352) 2024-04-23T14:41:41.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.125+0000 7fdfdb400640 1 -- 172.21.15.59:0/966966777 learned_addr learned my addr 172.21.15.59:0/966966777 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:41.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.126+0000 7fdfdb400640 1 -- 172.21.15.59:0/966966777 >> v2:172.21.15.76:3300/0 conn(0x7fdfe4155a80 msgr2=0x7fdfe406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:41.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.126+0000 7fdfdb400640 1 --2- 172.21.15.59:0/966966777 >> v2:172.21.15.76:3300/0 conn(0x7fdfe4155a80 0x7fdfe406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.126+0000 7fdfdb400640 1 -- 172.21.15.59:0/966966777 >> v1:172.21.15.76:6789/0 conn(0x7fdfe40b43a0 legacy=0x7fdfe40b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.126+0000 7fdfdb400640 1 -- 172.21.15.59:0/966966777 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe415f4a0 con 0x7fdfe40d0600 2024-04-23T14:41:41.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.127+0000 7fdfdb400640 1 --2- 172.21.15.59:0/966966777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe40d0a60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009870 tx=0x7fdfcc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6acc5ead7387dc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe3400640 1 -- 172.21.15.59:0/966966777 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc037070 con 0x7fdfe40d0600 2024-04-23T14:41:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe3400640 1 -- 172.21.15.59:0/966966777 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfcc032440 con 0x7fdfe40d0600 2024-04-23T14:41:41.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe3400640 1 -- 172.21.15.59:0/966966777 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc032810 con 0x7fdfe40d0600 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe9600640 1 -- 172.21.15.59:0/966966777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 msgr2=0x7fdfe40d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe9600640 1 --2- 172.21.15.59:0/966966777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe40d0a60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009870 tx=0x7fdfcc02a270 comp rx=0 tx=0).stop 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe9600640 1 -- 172.21.15.59:0/966966777 shutdown_connections 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe9600640 1 --2- 172.21.15.59:0/966966777 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe40d0a60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.128+0000 7fdfe9600640 1 --2- 172.21.15.59:0/966966777 >> v2:172.21.15.76:3300/0 conn(0x7fdfe4155a80 0x7fdfe406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.129+0000 7fdfe9600640 1 -- 172.21.15.59:0/966966777 >> 172.21.15.59:0/966966777 conn(0x7fdfe406bdf0 msgr2=0x7fdfe406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.129+0000 7fdfe9600640 1 -- 172.21.15.59:0/966966777 shutdown_connections 2024-04-23T14:41:41.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.129+0000 7fdfe9600640 1 -- 172.21.15.59:0/966966777 wait complete. 2024-04-23T14:41:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.130+0000 7fdfe9600640 1 Processor -- start 2024-04-23T14:41:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.130+0000 7fdfe9600640 1 -- start start 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.131+0000 7fdfe9600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfe40b43a0 0x7fdfe414e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.131+0000 7fdfe9600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.131+0000 7fdfe9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe4155a80 0x7fdfe415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.131+0000 7fdfe9600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe415fd40 con 0x7fdfe4155a80 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.131+0000 7fdfe9600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe415fa60 con 0x7fdfe40b43a0 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.131+0000 7fdfe9600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe406ff00 con 0x7fdfe40d0600 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe4155a80 0x7fdfe415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe4155a80 0x7fdfe415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49304/0 (socket says 172.21.15.59:49304) 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 -- 172.21.15.59:0/3335784947 learned_addr learned my addr 172.21.15.59:0/3335784947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfdb400640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe3e00640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfe40b43a0 0x7fdfe414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 -- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 msgr2=0x7fdfe415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 -- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfe40b43a0 msgr2=0x7fdfe414e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfe40b43a0 0x7fdfe414e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.132+0000 7fdfe8c00640 1 -- 172.21.15.59:0/3335784947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe415db20 con 0x7fdfe4155a80 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.133+0000 7fdfe8c00640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe4155a80 0x7fdfe415a270 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c7a0 tx=0x7fdfdc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.133+0000 7fdfe0c00640 1 -- 172.21.15.59:0/3335784947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc003b70 con 0x7fdfe4155a80 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.133+0000 7fdfe0c00640 1 -- 172.21.15.59:0/3335784947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfdc003d30 con 0x7fdfe4155a80 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.133+0000 7fdfe0c00640 1 -- 172.21.15.59:0/3335784947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00aa90 con 0x7fdfe4155a80 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.133+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe41f5640 con 0x7fdfe4155a80 2024-04-23T14:41:41.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.133+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe415cb00 con 0x7fdfe4155a80 2024-04-23T14:41:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.134+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe406fd20 con 0x7fdfe4155a80 2024-04-23T14:41:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.135+0000 7fdfe0c00640 1 -- 172.21.15.59:0/3335784947 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfdc00ac30 con 0x7fdfe4155a80 2024-04-23T14:41:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.136+0000 7fdfe0c00640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdfc006c8f0 0x7fdfc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.136+0000 7fdfe0c00640 1 -- 172.21.15.59:0/3335784947 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fdfdc08bb00 con 0x7fdfe4155a80 2024-04-23T14:41:41.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.137+0000 7fdfe3e00640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdfc006c8f0 0x7fdfc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.137+0000 7fdfe3e00640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdfc006c8f0 0x7fdfc006edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009a30 tx=0x7fdfd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:41.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.141+0000 7fdfe0c00640 1 -- 172.21.15.59:0/3335784947 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfe406fd20 con 0x7fdfe4155a80 2024-04-23T14:41:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.344+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfe404fd90 con 0x7fdfe4155a80 2024-04-23T14:41:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.345+0000 7fdfe0c00640 1 -- 172.21.15.59:0/3335784947 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fdfdc08be20 con 0x7fdfe4155a80 2024-04-23T14:41:41.346 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:41:41.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdfc006c8f0 msgr2=0x7fdfc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdfc006c8f0 0x7fdfc006edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009a30 tx=0x7fdfd4008040 comp rx=0 tx=0).stop 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe4155a80 msgr2=0x7fdfe415a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe4155a80 0x7fdfe415a270 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c7a0 tx=0x7fdfdc00cc60 comp rx=0 tx=0).stop 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 shutdown_connections 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdfc006c8f0 0x7fdfc006edb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdfe40d0600 0x7fdfe415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe4155a80 0x7fdfe415a270 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 --2- 172.21.15.59:0/3335784947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfe40b43a0 0x7fdfe414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.347+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 >> 172.21.15.59:0/3335784947 conn(0x7fdfe406bdf0 msgr2=0x7fdfe40ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.348+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 shutdown_connections 2024-04-23T14:41:41.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.348+0000 7fdfe9600640 1 -- 172.21.15.59:0/3335784947 wait complete. 2024-04-23T14:41:41.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:41:41.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.581+0000 7f291dc00640 1 Processor -- start 2024-04-23T14:41:41.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.581+0000 7f291dc00640 1 -- start start 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f291dc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29180cd8b0 0x7f29180d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f291dc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f29180d0b00 0x7f291806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f291dc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29180b4cf0 con 0x7f291815c670 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f291dc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f291815ca50 con 0x7f29180cd8b0 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f291dc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f29180ccb80 con 0x7f29180d0b00 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f2917e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f291815c670 0x7f29180b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56452/0 (socket says 172.21.15.59:56452) 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f2917e00640 1 -- 172.21.15.59:0/1310985819 learned_addr learned my addr 172.21.15.59:0/1310985819 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.583+0000 7f2917400640 1 --2- 172.21.15.59:0/1310985819 >> v2:172.21.15.59:3300/0 conn(0x7f29180cd8b0 0x7f29180d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.584+0000 7f291d200640 1 --2- 172.21.15.59:0/1310985819 >> v2:172.21.15.76:3300/0 conn(0x7f29180d0b00 0x7f291806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.584+0000 7f2917400640 1 -- 172.21.15.59:0/1310985819 >> v2:172.21.15.76:3300/0 conn(0x7f29180d0b00 msgr2=0x7f291806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.584+0000 7f2917400640 1 --2- 172.21.15.59:0/1310985819 >> v2:172.21.15.76:3300/0 conn(0x7f29180d0b00 0x7f291806ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.584+0000 7f2917400640 1 -- 172.21.15.59:0/1310985819 >> v1:172.21.15.59:6789/0 conn(0x7f291815c670 legacy=0x7f29180b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.584+0000 7f2917400640 1 -- 172.21.15.59:0/1310985819 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29180ccdd0 con 0x7f29180cd8b0 2024-04-23T14:41:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.584+0000 7f2916a00640 1 -- 172.21.15.59:0/1310985819 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2855983129 0 0) 0x7f29180b4cf0 con 0x7f291815c670 2024-04-23T14:41:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.584+0000 7f291d200640 1 --2- 172.21.15.59:0/1310985819 >> v2:172.21.15.76:3300/0 conn(0x7f29180d0b00 0x7f291806ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:41.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.585+0000 7f2917400640 1 --2- 172.21.15.59:0/1310985819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180cd8b0 0x7f29180d05c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f291806c970 tx=0x7f290002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2346b590e15a14c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:41.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.585+0000 7f2916a00640 1 -- 172.21.15.59:0/1310985819 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900037070 con 0x7f29180cd8b0 2024-04-23T14:41:41.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f2916a00640 1 -- 172.21.15.59:0/1310985819 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2900032440 con 0x7f29180cd8b0 2024-04-23T14:41:41.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f2916a00640 1 -- 172.21.15.59:0/1310985819 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900032810 con 0x7f29180cd8b0 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 -- 172.21.15.59:0/1310985819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180cd8b0 msgr2=0x7f29180d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 --2- 172.21.15.59:0/1310985819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180cd8b0 0x7f29180d05c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f291806c970 tx=0x7f290002a1a0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 -- 172.21.15.59:0/1310985819 shutdown_connections 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 --2- 172.21.15.59:0/1310985819 >> v2:172.21.15.76:3300/0 conn(0x7f29180d0b00 0x7f291806ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 --2- 172.21.15.59:0/1310985819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180cd8b0 0x7f29180d05c0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 -- 172.21.15.59:0/1310985819 >> 172.21.15.59:0/1310985819 conn(0x7f291806bdf0 msgr2=0x7f291806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 -- 172.21.15.59:0/1310985819 shutdown_connections 2024-04-23T14:41:41.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.586+0000 7f291dc00640 1 -- 172.21.15.59:0/1310985819 wait complete. 2024-04-23T14:41:41.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.588+0000 7f291dc00640 1 Processor -- start 2024-04-23T14:41:41.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.588+0000 7f291dc00640 1 -- start start 2024-04-23T14:41:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.589+0000 7f291dc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29180cd8b0 0x7f29180b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.589+0000 7f291dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180d0b00 0x7f29180ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f291dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291815c670 0x7f29181f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f291dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29180b3e60 con 0x7f29180d0b00 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f291dc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29180cd390 con 0x7f29180cd8b0 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f291dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f29180b3fe0 con 0x7f291815c670 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180d0b00 0x7f29180ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180d0b00 0x7f29180ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49328/0 (socket says 172.21.15.59:49328) 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 -- 172.21.15.59:0/3537772884 learned_addr learned my addr 172.21.15.59:0/3537772884 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917e00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29180cd8b0 0x7f29180b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f291d200640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291815c670 0x7f29181f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 -- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291815c670 msgr2=0x7f29181f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291815c670 0x7f29181f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 -- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29180cd8b0 msgr2=0x7f29180b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29180cd8b0 0x7f29180b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917400640 1 -- 172.21.15.59:0/3537772884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29180caec0 con 0x7f29180d0b00 2024-04-23T14:41:41.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.590+0000 7f2917e00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29180cd8b0 0x7f29180b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:41.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.591+0000 7f2917400640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180d0b00 0x7f29180ca980 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2900009890 tx=0x7f29000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:41.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.591+0000 7f290fe00640 1 -- 172.21.15.59:0/3537772884 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900037040 con 0x7f29180d0b00 2024-04-23T14:41:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.591+0000 7f290fe00640 1 -- 172.21.15.59:0/3537772884 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2900042070 con 0x7f29180d0b00 2024-04-23T14:41:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.591+0000 7f290fe00640 1 -- 172.21.15.59:0/3537772884 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900030a50 con 0x7f29180d0b00 2024-04-23T14:41:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.591+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29180b1900 con 0x7f29180d0b00 2024-04-23T14:41:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.591+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29180b1ec0 con 0x7f29180d0b00 2024-04-23T14:41:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.592+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291804fd90 con 0x7f29180d0b00 2024-04-23T14:41:41.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.593+0000 7f290fe00640 1 -- 172.21.15.59:0/3537772884 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2900030bf0 con 0x7f29180d0b00 2024-04-23T14:41:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.595+0000 7f290fe00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.595+0000 7f290fe00640 1 -- 172.21.15.59:0/3537772884 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f29000ac650 con 0x7f29180d0b00 2024-04-23T14:41:41.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.595+0000 7f2917e00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:41.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.596+0000 7f2917e00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28f406c8f0 0x7f28f406edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2908007fd0 tx=0x7f2908009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:41.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.600+0000 7f290fe00640 1 -- 172.21.15.59:0/3537772884 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29180caec0 con 0x7f29180d0b00 2024-04-23T14:41:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.817+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f29180ceab0 con 0x7f29180d0b00 2024-04-23T14:41:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.818+0000 7f290fe00640 1 -- 172.21.15.59:0/3537772884 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7f290007a9e0 con 0x7f29180d0b00 2024-04-23T14:41:41.820 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:41:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28f406c8f0 msgr2=0x7f28f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28f406c8f0 0x7f28f406edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2908007fd0 tx=0x7f2908009040 comp rx=0 tx=0).stop 2024-04-23T14:41:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180d0b00 msgr2=0x7f29180ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180d0b00 0x7f29180ca980 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2900009890 tx=0x7f29000046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 shutdown_connections 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291815c670 0x7f29181f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29180d0b00 0x7f29180ca980 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 --2- 172.21.15.59:0/3537772884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29180cd8b0 0x7f29180b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 >> 172.21.15.59:0/3537772884 conn(0x7f291806bdf0 msgr2=0x7f291806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 shutdown_connections 2024-04-23T14:41:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:41.822+0000 7f291dc00640 1 -- 172.21.15.59:0/3537772884 wait complete. 2024-04-23T14:41:41.843 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:41.843 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:41.897 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:42.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.111+0000 7f2e53000640 1 Processor -- start 2024-04-23T14:41:42.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.111+0000 7f2e53000640 1 -- start start 2024-04-23T14:41:42.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.112+0000 7f2e53000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 0x7f2e4c169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.112+0000 7f2e53000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4c0b87f0 con 0x7f2e4c15eb60 2024-04-23T14:41:42.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.112+0000 7f2e53000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4c0d2f30 con 0x7f2e4c0b3490 2024-04-23T14:41:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.112+0000 7f2e53000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c0ba7a0 con 0x7f2e4c0b99a0 2024-04-23T14:41:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.112+0000 7f2e4be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2e4c15eb60 0x7f2e4c0d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56460/0 (socket says 172.21.15.59:56460) 2024-04-23T14:41:42.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.112+0000 7f2e4be00640 1 -- 172.21.15.59:0/3208104711 learned_addr learned my addr 172.21.15.59:0/3208104711 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e51000640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 0x7f2e4c169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515699264 0 0) 0x7f2e4c0b87f0 con 0x7f2e4c15eb60 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e30003520 con 0x7f2e4c15eb60 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 748121038 0 0) 0x7f2e30003520 con 0x7f2e4c15eb60 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 msgr2=0x7f2e4c169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 0x7f2e4c169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 >> v1:172.21.15.76:6789/0 conn(0x7f2e4c0b3490 legacy=0x7f2e4c0b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c0ba1e0 con 0x7f2e4c15eb60 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2e3c001670 con 0x7f2e4c15eb60 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 >> v1:172.21.15.59:6789/0 conn(0x7f2e4c15eb60 legacy=0x7f2e4c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e300039e0 0x7f2e30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.113+0000 7f2e4aa00640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 0x7f2e30005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.114+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0ba7a0 con 0x7f2e30004800 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.114+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3000bb50 con 0x7f2e300039e0 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.114+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3000bd50 con 0x7f2e4c0b99a0 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.114+0000 7f2e4b400640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.114+0000 7f2e4be00640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e300039e0 0x7f2e30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.114+0000 7f2e51000640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 0x7f2e30005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.115+0000 7f2e4b400640 1 -- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 msgr2=0x7f2e30005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.115+0000 7f2e4b400640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b99a0 0x7f2e30005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.115+0000 7f2e4b400640 1 -- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e300039e0 msgr2=0x7f2e30003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.115+0000 7f2e4b400640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e300039e0 0x7f2e30003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.115+0000 7f2e4b400640 1 -- 172.21.15.59:0/3208104711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e30003520 con 0x7f2e30004800 2024-04-23T14:41:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.116+0000 7f2e4b400640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e30004c80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2e34001750 tx=0x7f2e30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bb2574ee446a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.116+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34001370 con 0x7f2e30004800 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.116+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e340052d0 con 0x7f2e30004800 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.116+0000 7f2e4aa00640 1 -- 172.21.15.59:0/3208104711 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3402abe0 con 0x7f2e30004800 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.117+0000 7f2e53000640 1 -- 172.21.15.59:0/3208104711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 msgr2=0x7f2e30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.117+0000 7f2e53000640 1 --2- 172.21.15.59:0/3208104711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e30004c80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2e34001750 tx=0x7f2e30005830 comp rx=0 tx=0).stop 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.117+0000 7f2e51000640 1 -- 172.21.15.59:0/3208104711 reap_dead start 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.117+0000 7f2e53000640 1 -- 172.21.15.59:0/3208104711 shutdown_connections 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.117+0000 7f2e53000640 1 -- 172.21.15.59:0/3208104711 >> 172.21.15.59:0/3208104711 conn(0x7f2e4c067b50 msgr2=0x7f2e4c067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.117+0000 7f2e53000640 1 -- 172.21.15.59:0/3208104711 shutdown_connections 2024-04-23T14:41:42.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.117+0000 7f2e53000640 1 -- 172.21.15.59:0/3208104711 wait complete. 2024-04-23T14:41:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.119+0000 7f2e53000640 1 Processor -- start 2024-04-23T14:41:42.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.119+0000 7f2e53000640 1 -- start start 2024-04-23T14:41:42.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.120+0000 7f2e53000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e4c14dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.120+0000 7f2e53000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b5860 0x7f2e4c0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.120+0000 7f2e53000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e4c0b5ce0 0x7f2e4c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.120+0000 7f2e53000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0b8d70 con 0x7f2e30004800 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.120+0000 7f2e53000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0b8a90 con 0x7f2e4c0b5ce0 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.120+0000 7f2e53000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c0b3c80 con 0x7f2e4c0b5860 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.120+0000 7f2e4be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e4c14dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e4c14dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49358/0 (socket says 172.21.15.59:49358) 2024-04-23T14:41:42.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 -- 172.21.15.59:0/2747379591 learned_addr learned my addr 172.21.15.59:0/2747379591 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:42.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e51000640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e4c0b5ce0 0x7f2e4c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4b400640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b5860 0x7f2e4c0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 -- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b5860 msgr2=0x7f2e4c0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b5860 0x7f2e4c0ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 -- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e4c0b5ce0 msgr2=0x7f2e4c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e4c0b5ce0 0x7f2e4c0caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 -- 172.21.15.59:0/2747379591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c0b62b0 con 0x7f2e30004800 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4b400640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b5860 0x7f2e4c0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.121+0000 7f2e4be00640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e4c14dec0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007290 tx=0x7f2e3c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.122+0000 7f2e43e00640 1 -- 172.21.15.59:0/2747379591 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c001650 con 0x7f2e30004800 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.122+0000 7f2e43e00640 1 -- 172.21.15.59:0/2747379591 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3c018070 con 0x7f2e30004800 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.122+0000 7f2e43e00640 1 -- 172.21.15.59:0/2747379591 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00fb60 con 0x7f2e30004800 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.122+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c0b6590 con 0x7f2e30004800 2024-04-23T14:41:42.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.122+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c0b6b50 con 0x7f2e30004800 2024-04-23T14:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.123+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c068a80 con 0x7f2e30004800 2024-04-23T14:41:42.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.124+0000 7f2e43e00640 1 -- 172.21.15.59:0/2747379591 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e3c00fd00 con 0x7f2e30004800 2024-04-23T14:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.125+0000 7f2e43e00640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e2806c920 0x7f2e2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.125+0000 7f2e43e00640 1 -- 172.21.15.59:0/2747379591 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f2e3c08bb80 con 0x7f2e30004800 2024-04-23T14:41:42.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.125+0000 7f2e4b400640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e2806c920 0x7f2e2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.126+0000 7f2e4b400640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e2806c920 0x7f2e2806ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2e34001750 tx=0x7f2e3402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.130+0000 7f2e43e00640 1 -- 172.21.15.59:0/2747379591 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e3c059950 con 0x7f2e30004800 2024-04-23T14:41:42.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.341+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e4c0d2310 con 0x7f2e30004800 2024-04-23T14:41:42.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.342+0000 7f2e43e00640 1 -- 172.21.15.59:0/2747379591 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16548 (secure 0 0 0) 0x7f2e3c05dc50 con 0x7f2e30004800 2024-04-23T14:41:42.344 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:42.344 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":99,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:31.158736+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.344+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e2806c920 msgr2=0x7f2e2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.344+0000 7f2e53000640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e2806c920 0x7f2e2806ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2e34001750 tx=0x7f2e3402e000 comp rx=0 tx=0).stop 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.344+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 msgr2=0x7f2e4c14dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.344+0000 7f2e53000640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e4c14dec0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007290 tx=0x7f2e3c007750 comp rx=0 tx=0).stop 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 shutdown_connections 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e2806c920 0x7f2e2806ede0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e4c0b5860 0x7f2e4c0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e4c0b5ce0 0x7f2e4c0caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 --2- 172.21.15.59:0/2747379591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e30004800 0x7f2e4c14dec0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 >> 172.21.15.59:0/2747379591 conn(0x7f2e4c067b50 msgr2=0x7f2e4c155c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 shutdown_connections 2024-04-23T14:41:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.345+0000 7f2e53000640 1 -- 172.21.15.59:0/2747379591 wait complete. 2024-04-23T14:41:42.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:41:27.465708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:41:28.586680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:42.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:42.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.566+0000 7f1a7a400640 1 Processor -- start 2024-04-23T14:41:42.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.567+0000 7f1a7a400640 1 -- start start 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a7a400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a7415ed20 0x7f1a740d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a7a400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3600 0x7f1a740d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a7a400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a740b5da0 con 0x7f1a740b3f10 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a7a400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7415f100 con 0x7f1a7415ed20 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a7a400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a740d0a20 con 0x7f1a740b3600 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a79000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a7415ed20 0x7f1a740d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a79000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a7415ed20 0x7f1a740d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49366/0 (socket says 172.21.15.59:49366) 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.568+0000 7f1a79000640 1 -- 172.21.15.59:0/2359583109 learned_addr learned my addr 172.21.15.59:0/2359583109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:42.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.569+0000 7f1a73e00640 1 --2- 172.21.15.59:0/2359583109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3600 0x7f1a740d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.569+0000 7f1a79000640 1 -- 172.21.15.59:0/2359583109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3600 msgr2=0x7f1a740d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:42.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.569+0000 7f1a79000640 1 --2- 172.21.15.59:0/2359583109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3600 0x7f1a740d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.569+0000 7f1a79000640 1 -- 172.21.15.59:0/2359583109 >> v1:172.21.15.76:6789/0 conn(0x7f1a740b3f10 legacy=0x7f1a740b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:42.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.569+0000 7f1a79000640 1 -- 172.21.15.59:0/2359583109 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a740d2ed0 con 0x7f1a7415ed20 2024-04-23T14:41:42.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.570+0000 7f1a79000640 1 --2- 172.21.15.59:0/2359583109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740d0ea0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009870 tx=0x7f1a6402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d256b20caa2f455b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.570+0000 7f1a73400640 1 -- 172.21.15.59:0/2359583109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64037070 con 0x7f1a7415ed20 2024-04-23T14:41:42.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.570+0000 7f1a73400640 1 -- 172.21.15.59:0/2359583109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6402ae10 con 0x7f1a7415ed20 2024-04-23T14:41:42.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.570+0000 7f1a73400640 1 -- 172.21.15.59:0/2359583109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64032660 con 0x7f1a7415ed20 2024-04-23T14:41:42.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 -- 172.21.15.59:0/2359583109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 msgr2=0x7f1a740d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 --2- 172.21.15.59:0/2359583109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740d0ea0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009870 tx=0x7f1a6402a100 comp rx=0 tx=0).stop 2024-04-23T14:41:42.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 -- 172.21.15.59:0/2359583109 shutdown_connections 2024-04-23T14:41:42.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 --2- 172.21.15.59:0/2359583109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3600 0x7f1a740d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 --2- 172.21.15.59:0/2359583109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740d0ea0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 -- 172.21.15.59:0/2359583109 >> 172.21.15.59:0/2359583109 conn(0x7f1a7408bf90 msgr2=0x7f1a740d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:42.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 -- 172.21.15.59:0/2359583109 shutdown_connections 2024-04-23T14:41:42.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.571+0000 7f1a7a400640 1 -- 172.21.15.59:0/2359583109 wait complete. 2024-04-23T14:41:42.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.573+0000 7f1a7a400640 1 Processor -- start 2024-04-23T14:41:42.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.573+0000 7f1a7a400640 1 -- start start 2024-04-23T14:41:42.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.574+0000 7f1a7a400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a740b3600 0x7f1a740a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.574+0000 7f1a7a400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3f10 0x7f1a741f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.574+0000 7f1a7a400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.574+0000 7f1a7a400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a740d0290 con 0x7f1a7415ed20 2024-04-23T14:41:42.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.574+0000 7f1a7a400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a740d3490 con 0x7f1a740b3600 2024-04-23T14:41:42.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.574+0000 7f1a7a400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a740d0410 con 0x7f1a740b3f10 2024-04-23T14:41:42.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a73e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3f10 0x7f1a741f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a73e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3f10 0x7f1a741f8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:54412/0 (socket says 172.21.15.59:54412) 2024-04-23T14:41:42.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a79000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a740b3600 0x7f1a740a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a73e00640 1 -- 172.21.15.59:0/179142538 learned_addr learned my addr 172.21.15.59:0/179142538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:42.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a79a00640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a79a00640 1 -- 172.21.15.59:0/179142538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3f10 msgr2=0x7f1a741f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a79a00640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3f10 0x7f1a741f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a79a00640 1 -- 172.21.15.59:0/179142538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a740b3600 msgr2=0x7f1a740a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a79a00640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a740b3600 0x7f1a740a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.575+0000 7f1a79a00640 1 -- 172.21.15.59:0/179142538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a740b7770 con 0x7f1a7415ed20 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.576+0000 7f1a79a00640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740cd090 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c780 tx=0x7f1a6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.576+0000 7f1a70c00640 1 -- 172.21.15.59:0/179142538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c016020 con 0x7f1a7415ed20 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.576+0000 7f1a70c00640 1 -- 172.21.15.59:0/179142538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6c00aa50 con 0x7f1a7415ed20 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.576+0000 7f1a70c00640 1 -- 172.21.15.59:0/179142538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00ae40 con 0x7f1a7415ed20 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.576+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a740b7a50 con 0x7f1a7415ed20 2024-04-23T14:41:42.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.576+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a741f8410 con 0x7f1a7415ed20 2024-04-23T14:41:42.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.577+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7404fd90 con 0x7f1a7415ed20 2024-04-23T14:41:42.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.578+0000 7f1a70c00640 1 -- 172.21.15.59:0/179142538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a6c019020 con 0x7f1a7415ed20 2024-04-23T14:41:42.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.579+0000 7f1a70c00640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5006c8f0 0x7f1a5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.579+0000 7f1a70c00640 1 -- 172.21.15.59:0/179142538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f1a6c08b920 con 0x7f1a7415ed20 2024-04-23T14:41:42.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.580+0000 7f1a79000640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5006c8f0 0x7f1a5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.580+0000 7f1a79000640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5006c8f0 0x7f1a5006edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1a64034000 tx=0x7f1a6402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.584+0000 7f1a70c00640 1 -- 172.21.15.59:0/179142538 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a6c020030 con 0x7f1a7415ed20 2024-04-23T14:41:42.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.784+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a740b1610 con 0x7f1a7415ed20 2024-04-23T14:41:42.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.785+0000 7f1a70c00640 1 -- 172.21.15.59:0/179142538 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f1a6c0596f0 con 0x7f1a7415ed20 2024-04-23T14:41:42.787 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:42.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5006c8f0 msgr2=0x7f1a5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5006c8f0 0x7f1a5006edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1a64034000 tx=0x7f1a6402e000 comp rx=0 tx=0).stop 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 msgr2=0x7f1a740cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740cd090 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c780 tx=0x7f1a6c00cc40 comp rx=0 tx=0).stop 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 shutdown_connections 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a5006c8f0 0x7f1a5006edb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a740b3f10 0x7f1a741f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a7415ed20 0x7f1a740cd090 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 --2- 172.21.15.59:0/179142538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a740b3600 0x7f1a740a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 >> 172.21.15.59:0/179142538 conn(0x7f1a7408bf90 msgr2=0x7f1a7406e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 shutdown_connections 2024-04-23T14:41:42.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:42.788+0000 7f1a7a400640 1 -- 172.21.15.59:0/179142538 wait complete. 2024-04-23T14:41:42.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:41:43.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.014+0000 7fd5ccc00640 1 Processor -- start 2024-04-23T14:41:43.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.015+0000 7fd5ccc00640 1 -- start start 2024-04-23T14:41:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.016+0000 7fd5ccc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5c80d15b0 0x7fd5c80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.016+0000 7fd5ccc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80b5da0 con 0x7fd5c815ed20 2024-04-23T14:41:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.016+0000 7fd5ccc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80d0a20 con 0x7fd5c80b4f20 2024-04-23T14:41:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.016+0000 7fd5ccc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c815f100 con 0x7fd5c80d15b0 2024-04-23T14:41:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.017+0000 7fd5c7400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd5c815ed20 0x7fd5c80d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56466/0 (socket says 172.21.15.59:56466) 2024-04-23T14:41:43.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.017+0000 7fd5c7400640 1 -- 172.21.15.59:0/2238834176 learned_addr learned my addr 172.21.15.59:0/2238834176 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:43.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.017+0000 7fd5c6a00640 1 --2- 172.21.15.59:0/2238834176 >> v2:172.21.15.76:3300/0 conn(0x7fd5c80d15b0 0x7fd5c80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.017+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673337532 0 0) 0x7fd5c80b5da0 con 0x7fd5c815ed20 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.017+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5ac003520 con 0x7fd5c815ed20 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1114195666 0 0) 0x7fd5ac003520 con 0x7fd5c815ed20 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 >> v2:172.21.15.76:3300/0 conn(0x7fd5c80d15b0 msgr2=0x7fd5c80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 --2- 172.21.15.59:0/2238834176 >> v2:172.21.15.76:3300/0 conn(0x7fd5c80d15b0 0x7fd5c80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 >> v1:172.21.15.76:6789/0 conn(0x7fd5c80b4f20 legacy=0x7fd5c80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80d2ed0 con 0x7fd5c815ed20 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fd5b8001670 con 0x7fd5c815ed20 2024-04-23T14:41:43.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 >> v1:172.21.15.59:6789/0 conn(0x7fd5c815ed20 legacy=0x7fd5c80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0039b0 0x7fd5ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5ac004800 0x7fd5ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac0052e0 0x7fd5ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac00c180 con 0x7fd5ac004800 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac00c330 con 0x7fd5ac0052e0 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac00c530 con 0x7fd5ac0039b0 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.018+0000 7fd5c6a00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5ac004800 0x7fd5ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c7400640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0039b0 0x7fd5ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c6a00640 1 -- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0039b0 msgr2=0x7fd5ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c6a00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0039b0 0x7fd5ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c6a00640 1 -- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac0052e0 msgr2=0x7fd5ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c7e00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac0052e0 0x7fd5ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c6a00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac0052e0 0x7fd5ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c6a00640 1 -- 172.21.15.59:0/2238834176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80d2ed0 con 0x7fd5ac004800 2024-04-23T14:41:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c7400640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0039b0 0x7fd5ac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c7e00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac0052e0 0x7fd5ac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.019+0000 7fd5c7e00640 1 -- 172.21.15.59:0/2238834176 reap_dead start 2024-04-23T14:41:43.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.020+0000 7fd5c6a00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5ac004800 0x7fd5ac004c80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009a90 tx=0x7fd5ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dd0e62e8454331 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.020+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4036070 con 0x7fd5ac004800 2024-04-23T14:41:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.020+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b4031400 con 0x7fd5ac004800 2024-04-23T14:41:43.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.020+0000 7fd5c6000640 1 -- 172.21.15.59:0/2238834176 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4009690 con 0x7fd5ac004800 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.021+0000 7fd5ccc00640 1 -- 172.21.15.59:0/2238834176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5ac004800 msgr2=0x7fd5ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.021+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5ac004800 0x7fd5ac004c80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009a90 tx=0x7fd5ac005830 comp rx=0 tx=0).stop 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.021+0000 7fd5ccc00640 1 -- 172.21.15.59:0/2238834176 shutdown_connections 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.021+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/2238834176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5ac004800 0x7fd5ac004c80 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.021+0000 7fd5ccc00640 1 -- 172.21.15.59:0/2238834176 >> 172.21.15.59:0/2238834176 conn(0x7fd5c808bf90 msgr2=0x7fd5c80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.021+0000 7fd5ccc00640 1 -- 172.21.15.59:0/2238834176 shutdown_connections 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.021+0000 7fd5ccc00640 1 -- 172.21.15.59:0/2238834176 wait complete. 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.023+0000 7fd5ccc00640 1 Processor -- start 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.023+0000 7fd5ccc00640 1 -- start start 2024-04-23T14:41:43.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5ccc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac004800 0x7fd5c80b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5ccc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c80b16d0 0x7fd5c806db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5ccc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5c815a920 0x7fd5c806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5ccc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80b3600 con 0x7fd5c815a920 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5ccc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80d3490 con 0x7fd5ac004800 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5ccc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80b3780 con 0x7fd5c80b16d0 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5c815a920 0x7fd5c806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5c815a920 0x7fd5c806e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49392/0 (socket says 172.21.15.59:49392) 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 -- 172.21.15.59:0/3406066232 learned_addr learned my addr 172.21.15.59:0/3406066232 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7400640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac004800 0x7fd5c80b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c6a00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c80b16d0 0x7fd5c806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 -- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c80b16d0 msgr2=0x7fd5c806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c80b16d0 0x7fd5c806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 -- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac004800 msgr2=0x7fd5c80b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac004800 0x7fd5c80b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.024+0000 7fd5c7e00640 1 -- 172.21.15.59:0/3406066232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80b1ba0 con 0x7fd5c815a920 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.025+0000 7fd5c7e00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5c815a920 0x7fd5c806e0b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80d1c80 tx=0x7fd5c000d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.025+0000 7fd5bf400640 1 -- 172.21.15.59:0/3406066232 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0018070 con 0x7fd5c815a920 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.025+0000 7fd5bf400640 1 -- 172.21.15.59:0/3406066232 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c00050d0 con 0x7fd5c815a920 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.025+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c80b1e80 con 0x7fd5c815a920 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.025+0000 7fd5bf400640 1 -- 172.21.15.59:0/3406066232 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0003e60 con 0x7fd5c815a920 2024-04-23T14:41:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.025+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c815f100 con 0x7fd5c815a920 2024-04-23T14:41:43.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.026+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c804fd90 con 0x7fd5c815a920 2024-04-23T14:41:43.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.027+0000 7fd5bf400640 1 -- 172.21.15.59:0/3406066232 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5c001b070 con 0x7fd5c815a920 2024-04-23T14:41:43.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.028+0000 7fd5bf400640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd5a806c8f0 0x7fd5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.028+0000 7fd5bf400640 1 -- 172.21.15.59:0/3406066232 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fd5c008b370 con 0x7fd5c815a920 2024-04-23T14:41:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.029+0000 7fd5c7400640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd5a806c8f0 0x7fd5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.029+0000 7fd5c7400640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd5a806c8f0 0x7fd5a806edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8001050 tx=0x7fd5b8000fe0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.033+0000 7fd5bf400640 1 -- 172.21.15.59:0/3406066232 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5c0059700 con 0x7fd5c815a920 2024-04-23T14:41:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.242+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd5c806ba80 con 0x7fd5c815a920 2024-04-23T14:41:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.243+0000 7fd5bf400640 1 -- 172.21.15.59:0/3406066232 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fd5c0059000 con 0x7fd5c815a920 2024-04-23T14:41:43.244 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd5a806c8f0 msgr2=0x7fd5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd5a806c8f0 0x7fd5a806edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8001050 tx=0x7fd5b8000fe0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5c815a920 msgr2=0x7fd5c806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5c815a920 0x7fd5c806e0b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80d1c80 tx=0x7fd5c000d8e0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 shutdown_connections 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd5a806c8f0 0x7fd5a806edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c80b16d0 0x7fd5c806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5c815a920 0x7fd5c806e0b0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 --2- 172.21.15.59:0/3406066232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac004800 0x7fd5c80b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.246+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 >> 172.21.15.59:0/3406066232 conn(0x7fd5c808bf90 msgr2=0x7fd5c806d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:43.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.247+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 shutdown_connections 2024-04-23T14:41:43.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.247+0000 7fd5ccc00640 1 -- 172.21.15.59:0/3406066232 wait complete. 2024-04-23T14:41:43.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:41:43.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.479+0000 7f90eb200640 1 Processor -- start 2024-04-23T14:41:43.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.479+0000 7f90eb200640 1 -- start start 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90eb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec0d0600 0x7f90ec0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90eb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90ec06dcb0 0x7f90ec162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90eb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90ec163240 con 0x7f90ec0d09e0 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90eb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f90ec072a20 con 0x7f90ec06dcb0 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90eb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f90ec071e10 con 0x7f90ec0d0600 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90e9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90ec06dcb0 0x7f90ec162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90e9400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90ec06dcb0 0x7f90ec162d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49394/0 (socket says 172.21.15.59:49394) 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.481+0000 7f90e9400640 1 -- 172.21.15.59:0/3990787263 learned_addr learned my addr 172.21.15.59:0/3990787263 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:43.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.482+0000 7f90e9e00640 1 --2- 172.21.15.59:0/3990787263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec0d0600 0x7f90ec0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.482+0000 7f90e9400640 1 -- 172.21.15.59:0/3990787263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec0d0600 msgr2=0x7f90ec0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:43.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.482+0000 7f90e9400640 1 --2- 172.21.15.59:0/3990787263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec0d0600 0x7f90ec0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.482+0000 7f90e9400640 1 -- 172.21.15.59:0/3990787263 >> v1:172.21.15.59:6789/0 conn(0x7f90ec0d09e0 legacy=0x7f90ec06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.482+0000 7f90e9400640 1 -- 172.21.15.59:0/3990787263 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec0720f0 con 0x7f90ec06dcb0 2024-04-23T14:41:43.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.482+0000 7f90e9e00640 1 --2- 172.21.15.59:0/3990787263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec0d0600 0x7f90ec0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:43.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.483+0000 7f90e9400640 1 --2- 172.21.15.59:0/3990787263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec06dcb0 0x7f90ec162d00 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009870 tx=0x7f90d802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f21a28335e2a619 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.483+0000 7f90e8a00640 1 -- 172.21.15.59:0/3990787263 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8037070 con 0x7f90ec06dcb0 2024-04-23T14:41:43.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.483+0000 7f90e8a00640 1 -- 172.21.15.59:0/3990787263 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d8032440 con 0x7f90ec06dcb0 2024-04-23T14:41:43.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.483+0000 7f90e8a00640 1 -- 172.21.15.59:0/3990787263 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8032810 con 0x7f90ec06dcb0 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 -- 172.21.15.59:0/3990787263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec06dcb0 msgr2=0x7f90ec162d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 --2- 172.21.15.59:0/3990787263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec06dcb0 0x7f90ec162d00 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009870 tx=0x7f90d802a250 comp rx=0 tx=0).stop 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 -- 172.21.15.59:0/3990787263 shutdown_connections 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 --2- 172.21.15.59:0/3990787263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec06dcb0 0x7f90ec162d00 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 --2- 172.21.15.59:0/3990787263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec0d0600 0x7f90ec0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 -- 172.21.15.59:0/3990787263 >> 172.21.15.59:0/3990787263 conn(0x7f90ec06bdf0 msgr2=0x7f90ec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 -- 172.21.15.59:0/3990787263 shutdown_connections 2024-04-23T14:41:43.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.484+0000 7f90eb200640 1 -- 172.21.15.59:0/3990787263 wait complete. 2024-04-23T14:41:43.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.486+0000 7f90eb200640 1 Processor -- start 2024-04-23T14:41:43.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.486+0000 7f90eb200640 1 -- start start 2024-04-23T14:41:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90eb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec06dcb0 0x7f90ec14f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90eb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90ec0d0600 0x7f90ec15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90eb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec0d09e0 0x7f90ec15f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90eb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec15eaf0 con 0x7f90ec0d09e0 2024-04-23T14:41:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90eb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec0726b0 con 0x7f90ec0d0600 2024-04-23T14:41:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90eb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f90ec0d0e70 con 0x7f90ec06dcb0 2024-04-23T14:41:43.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90ea800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec0d09e0 0x7f90ec15f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90ea800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec0d09e0 0x7f90ec15f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49410/0 (socket says 172.21.15.59:49410) 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.487+0000 7f90ea800640 1 -- 172.21.15.59:0/2967205109 learned_addr learned my addr 172.21.15.59:0/2967205109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.488+0000 7f90e9e00640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec06dcb0 0x7f90ec14f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.488+0000 7f90ea800640 1 -- 172.21.15.59:0/2967205109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec06dcb0 msgr2=0x7f90ec14f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.488+0000 7f90ea800640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec06dcb0 0x7f90ec14f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.488+0000 7f90ea800640 1 -- 172.21.15.59:0/2967205109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90ec0d0600 msgr2=0x7f90ec15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.488+0000 7f90ea800640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90ec0d0600 0x7f90ec15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.488+0000 7f90ea800640 1 -- 172.21.15.59:0/2967205109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec160a40 con 0x7f90ec0d09e0 2024-04-23T14:41:43.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.488+0000 7f90ea800640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec0d09e0 0x7f90ec15f1b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c6b0 tx=0x7f90e400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.489+0000 7f90e2000640 1 -- 172.21.15.59:0/2967205109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4003b70 con 0x7f90ec0d09e0 2024-04-23T14:41:43.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.489+0000 7f90e2000640 1 -- 172.21.15.59:0/2967205109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e4003d50 con 0x7f90ec0d09e0 2024-04-23T14:41:43.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.489+0000 7f90e2000640 1 -- 172.21.15.59:0/2967205109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4005310 con 0x7f90ec0d09e0 2024-04-23T14:41:43.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.489+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec160d20 con 0x7f90ec0d09e0 2024-04-23T14:41:43.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.489+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec1612e0 con 0x7f90ec0d09e0 2024-04-23T14:41:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.490+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ec04fd90 con 0x7f90ec0d09e0 2024-04-23T14:41:43.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.491+0000 7f90e2000640 1 -- 172.21.15.59:0/2967205109 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90e40054b0 con 0x7f90ec0d09e0 2024-04-23T14:41:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.492+0000 7f90e2000640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90cc06c920 0x7f90cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.492+0000 7f90e2000640 1 -- 172.21.15.59:0/2967205109 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f90e408ba70 con 0x7f90ec0d09e0 2024-04-23T14:41:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.493+0000 7f90e9e00640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90cc06c920 0x7f90cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.493+0000 7f90e9e00640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90cc06c920 0x7f90cc06ede0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f90dc008850 tx=0x7f90dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.497+0000 7f90e2000640 1 -- 172.21.15.59:0/2967205109 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90ec163240 con 0x7f90ec0d09e0 2024-04-23T14:41:43.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.697+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90ec0cff50 con 0x7f90ec0d09e0 2024-04-23T14:41:43.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.698+0000 7f90e2000640 1 -- 172.21.15.59:0/2967205109 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f90e4059e00 con 0x7f90ec0d09e0 2024-04-23T14:41:43.700 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:41:43.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90cc06c920 msgr2=0x7f90cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90cc06c920 0x7f90cc06ede0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f90dc008850 tx=0x7f90dc008040 comp rx=0 tx=0).stop 2024-04-23T14:41:43.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec0d09e0 msgr2=0x7f90ec15f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec0d09e0 0x7f90ec15f1b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c6b0 tx=0x7f90e400cb70 comp rx=0 tx=0).stop 2024-04-23T14:41:43.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 shutdown_connections 2024-04-23T14:41:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90cc06c920 0x7f90cc06ede0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ec0d09e0 0x7f90ec15f1b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90ec0d0600 0x7f90ec15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 --2- 172.21.15.59:0/2967205109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90ec06dcb0 0x7f90ec14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 >> 172.21.15.59:0/2967205109 conn(0x7f90ec06bdf0 msgr2=0x7f90ec0cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 shutdown_connections 2024-04-23T14:41:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.701+0000 7f90eb200640 1 -- 172.21.15.59:0/2967205109 wait complete. 2024-04-23T14:41:43.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:41:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.932+0000 7fe79de00640 1 Processor -- start 2024-04-23T14:41:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.932+0000 7fe79de00640 1 -- start start 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.933+0000 7fe79de00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe79815c670 0x7fe7980cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.933+0000 7fe79de00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7980d0cd0 0x7fe7980b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.933+0000 7fe79de00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7980b4cf0 con 0x7fe7980b4230 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.933+0000 7fe79de00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe79815ca50 con 0x7fe79815c670 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.933+0000 7fe79de00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe7980ccb80 con 0x7fe7980d0cd0 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe79ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe79815c670 0x7fe7980cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe797e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe7980b4230 0x7fe7980d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56492/0 (socket says 172.21.15.59:56492) 2024-04-23T14:41:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe797e00640 1 -- 172.21.15.59:0/2628109635 learned_addr learned my addr 172.21.15.59:0/2628109635 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe79d400640 1 --2- 172.21.15.59:0/2628109635 >> v2:172.21.15.76:3300/0 conn(0x7fe7980d0cd0 0x7fe7980b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe79ca00640 1 -- 172.21.15.59:0/2628109635 >> v2:172.21.15.76:3300/0 conn(0x7fe7980d0cd0 msgr2=0x7fe7980b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe79ca00640 1 --2- 172.21.15.59:0/2628109635 >> v2:172.21.15.76:3300/0 conn(0x7fe7980d0cd0 0x7fe7980b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe79ca00640 1 -- 172.21.15.59:0/2628109635 >> v1:172.21.15.59:6789/0 conn(0x7fe7980b4230 legacy=0x7fe7980d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.934+0000 7fe79ca00640 1 -- 172.21.15.59:0/2628109635 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7980ccdd0 con 0x7fe79815c670 2024-04-23T14:41:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.936+0000 7fe79ca00640 1 --2- 172.21.15.59:0/2628109635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79815c670 0x7fe7980cd8b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe79806c970 tx=0x7fe78c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850da397f0aa65f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.936+0000 7fe797400640 1 -- 172.21.15.59:0/2628109635 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c037070 con 0x7fe79815c670 2024-04-23T14:41:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.936+0000 7fe797400640 1 -- 172.21.15.59:0/2628109635 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c032440 con 0x7fe79815c670 2024-04-23T14:41:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.936+0000 7fe797400640 1 -- 172.21.15.59:0/2628109635 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c032810 con 0x7fe79815c670 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.936+0000 7fe79de00640 1 -- 172.21.15.59:0/2628109635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79815c670 msgr2=0x7fe7980cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.936+0000 7fe79de00640 1 --2- 172.21.15.59:0/2628109635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79815c670 0x7fe7980cd8b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe79806c970 tx=0x7fe78c02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.937+0000 7fe79de00640 1 -- 172.21.15.59:0/2628109635 shutdown_connections 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.937+0000 7fe79de00640 1 --2- 172.21.15.59:0/2628109635 >> v2:172.21.15.76:3300/0 conn(0x7fe7980d0cd0 0x7fe7980b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.937+0000 7fe79de00640 1 --2- 172.21.15.59:0/2628109635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79815c670 0x7fe7980cd8b0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.937+0000 7fe79de00640 1 -- 172.21.15.59:0/2628109635 >> 172.21.15.59:0/2628109635 conn(0x7fe79806bdf0 msgr2=0x7fe79806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.937+0000 7fe79de00640 1 -- 172.21.15.59:0/2628109635 shutdown_connections 2024-04-23T14:41:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.937+0000 7fe79de00640 1 -- 172.21.15.59:0/2628109635 wait complete. 2024-04-23T14:41:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.939+0000 7fe79de00640 1 Processor -- start 2024-04-23T14:41:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.939+0000 7fe79de00640 1 -- start start 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7980b4230 0x7fe7980b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79de00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7980d0cd0 0x7fe798157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79de00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79815c670 0x7fe7980acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79de00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe7980b3e60 con 0x7fe7980b4230 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79de00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7980cd390 con 0x7fe79815c670 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79de00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe7980b3fe0 con 0x7fe7980d0cd0 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7980b4230 0x7fe7980b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7980b4230 0x7fe7980b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49428/0 (socket says 172.21.15.59:49428) 2024-04-23T14:41:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 -- 172.21.15.59:0/4165198434 learned_addr learned my addr 172.21.15.59:0/4165198434 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79d400640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79815c670 0x7fe7980acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 -- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7980d0cd0 msgr2=0x7fe798157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe797e00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7980d0cd0 0x7fe798157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7980d0cd0 0x7fe798157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 -- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79815c670 msgr2=0x7fe7980acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79815c670 0x7fe7980acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.940+0000 7fe79ca00640 1 -- 172.21.15.59:0/4165198434 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7980adde0 con 0x7fe7980b4230 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.941+0000 7fe79ca00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7980b4230 0x7fe7980b2b90 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009890 tx=0x7fe78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.941+0000 7fe794c00640 1 -- 172.21.15.59:0/4165198434 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c037040 con 0x7fe7980b4230 2024-04-23T14:41:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.941+0000 7fe794c00640 1 -- 172.21.15.59:0/4165198434 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c042070 con 0x7fe7980b4230 2024-04-23T14:41:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.941+0000 7fe794c00640 1 -- 172.21.15.59:0/4165198434 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c030bd0 con 0x7fe7980b4230 2024-04-23T14:41:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.941+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe798158640 con 0x7fe7980b4230 2024-04-23T14:41:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.941+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981445d0 con 0x7fe7980b4230 2024-04-23T14:41:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.943+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79804fd90 con 0x7fe7980b4230 2024-04-23T14:41:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.944+0000 7fe794c00640 1 -- 172.21.15.59:0/4165198434 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe78c047460 con 0x7fe7980b4230 2024-04-23T14:41:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.945+0000 7fe794c00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe77806c920 0x7fe77806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.945+0000 7fe794c00640 1 -- 172.21.15.59:0/4165198434 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fe78c0ac720 con 0x7fe7980b4230 2024-04-23T14:41:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.945+0000 7fe797e00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe77806c920 0x7fe77806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:43.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.946+0000 7fe797e00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe77806c920 0x7fe77806ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe79815f020 tx=0x7fe784009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:43.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:43.949+0000 7fe794c00640 1 -- 172.21.15.59:0/4165198434 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe78c032e20 con 0x7fe7980b4230 2024-04-23T14:41:44.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.157+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe7980ceab0 con 0x7fe7980b4230 2024-04-23T14:41:44.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.158+0000 7fe794c00640 1 -- 172.21.15.59:0/4165198434 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7fe78c07aab0 con 0x7fe7980b4230 2024-04-23T14:41:44.159 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe77806c920 msgr2=0x7fe77806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe77806c920 0x7fe77806ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe79815f020 tx=0x7fe784009040 comp rx=0 tx=0).stop 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7980b4230 msgr2=0x7fe7980b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7980b4230 0x7fe7980b2b90 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009890 tx=0x7fe78c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 shutdown_connections 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe77806c920 0x7fe77806ede0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7980d0cd0 0x7fe798157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79815c670 0x7fe7980acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 --2- 172.21.15.59:0/4165198434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7980b4230 0x7fe7980b2b90 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.160+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 >> 172.21.15.59:0/4165198434 conn(0x7fe79806bdf0 msgr2=0x7fe79809a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:44.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.161+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 shutdown_connections 2024-04-23T14:41:44.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.161+0000 7fe79de00640 1 -- 172.21.15.59:0/4165198434 wait complete. 2024-04-23T14:41:44.180 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:41:44.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:44.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:44.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[70092]: fuse finished with error 0 and tester_r 0 2024-04-23T14:41:44.255 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:41:44.289 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:41:44.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:44.290 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:41:44.383 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:44.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.582+0000 7f0c12400640 1 Processor -- start 2024-04-23T14:41:44.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.582+0000 7f0c12400640 1 -- start start 2024-04-23T14:41:44.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c12400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c0c142070 0x7f0c0c0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:44.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c12400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0c0c0ce390 0x7f0c0c06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:44.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c12400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c0c142650 con 0x7f0c0c15fea0 2024-04-23T14:41:44.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c12400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0c160510 con 0x7f0c0c142070 2024-04-23T14:41:44.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c12400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0c142930 con 0x7f0c0c0ce390 2024-04-23T14:41:44.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c0aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c0c142070 0x7f0c0c0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:44.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c0aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c0c142070 0x7f0c0c0cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49438/0 (socket says 172.21.15.59:49438) 2024-04-23T14:41:44.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c0aa00640 1 -- 172.21.15.59:0/2772467037 learned_addr learned my addr 172.21.15.59:0/2772467037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:44.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.584+0000 7f0c0b400640 1 --1- 172.21.15.59:0/2772467037 >> v1:172.21.15.59:6789/0 conn(0x7f0c0c15fea0 0x7f0c0c0b29d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56502/0 (socket says 172.21.15.59:56502) 2024-04-23T14:41:44.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.585+0000 7f0c0aa00640 1 -- 172.21.15.59:0/2772467037 >> v2:172.21.15.76:3300/0 conn(0x7f0c0c0ce390 msgr2=0x7f0c0c06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:44.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.585+0000 7f0c0be00640 1 --2- 172.21.15.59:0/2772467037 >> v2:172.21.15.76:3300/0 conn(0x7f0c0c0ce390 0x7f0c0c06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:44.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.585+0000 7f0c0aa00640 1 --2- 172.21.15.59:0/2772467037 >> v2:172.21.15.76:3300/0 conn(0x7f0c0c0ce390 0x7f0c0c06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.585+0000 7f0c0aa00640 1 -- 172.21.15.59:0/2772467037 >> v1:172.21.15.59:6789/0 conn(0x7f0c0c15fea0 legacy=0x7f0c0c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.585+0000 7f0c0aa00640 1 -- 172.21.15.59:0/2772467037 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c142c10 con 0x7f0c0c142070 2024-04-23T14:41:44.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.586+0000 7f0c0aa00640 1 --2- 172.21.15.59:0/2772467037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c142070 0x7f0c0c0cde50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c073860 tx=0x7f0bf802a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685525e0bb0087a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:44.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.586+0000 7f0c0a000640 1 -- 172.21.15.59:0/2772467037 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8037070 con 0x7f0c0c142070 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.586+0000 7f0c0a000640 1 -- 172.21.15.59:0/2772467037 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf8032440 con 0x7f0c0c142070 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.586+0000 7f0c0a000640 1 -- 172.21.15.59:0/2772467037 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8032810 con 0x7f0c0c142070 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 -- 172.21.15.59:0/2772467037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c142070 msgr2=0x7f0c0c0cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 --2- 172.21.15.59:0/2772467037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c142070 0x7f0c0c0cde50 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c073860 tx=0x7f0bf802a280 comp rx=0 tx=0).stop 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 -- 172.21.15.59:0/2772467037 shutdown_connections 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 --2- 172.21.15.59:0/2772467037 >> v2:172.21.15.76:3300/0 conn(0x7f0c0c0ce390 0x7f0c0c06b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 --2- 172.21.15.59:0/2772467037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c142070 0x7f0c0c0cde50 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 -- 172.21.15.59:0/2772467037 >> 172.21.15.59:0/2772467037 conn(0x7f0c0c073450 msgr2=0x7f0c0c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 -- 172.21.15.59:0/2772467037 shutdown_connections 2024-04-23T14:41:44.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.587+0000 7f0c12400640 1 -- 172.21.15.59:0/2772467037 wait complete. 2024-04-23T14:41:44.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.589+0000 7f0c12400640 1 Processor -- start 2024-04-23T14:41:44.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.589+0000 7f0c12400640 1 -- start start 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c12400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c0ce390 0x7f0c0c1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c12400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c0c142070 0x7f0c0c145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c12400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c0c15fea0 0x7f0c0c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c12400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c06c090 con 0x7f0c0c0ce390 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c12400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c06bf10 con 0x7f0c0c142070 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c12400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c06c210 con 0x7f0c0c15fea0 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c0ce390 0x7f0c0c1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c0ce390 0x7f0c0c1438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49448/0 (socket says 172.21.15.59:49448) 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0b400640 1 -- 172.21.15.59:0/2983760354 learned_addr learned my addr 172.21.15.59:0/2983760354 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0be00640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c0c15fea0 0x7f0c0c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:44.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0aa00640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c0c142070 0x7f0c0c145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:44.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0b400640 1 -- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c0c15fea0 msgr2=0x7f0c0c0afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0b400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c0c15fea0 0x7f0c0c0afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0b400640 1 -- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c0c142070 msgr2=0x7f0c0c145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.590+0000 7f0c0b400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c0c142070 0x7f0c0c145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c0b400640 1 -- 172.21.15.59:0/2983760354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c145b40 con 0x7f0c0c0ce390 2024-04-23T14:41:44.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c0aa00640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c0c142070 0x7f0c0c145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:44.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c0b400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c0ce390 0x7f0c0c1438f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c3f0 tx=0x7f0bfc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:44.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c03400640 1 -- 172.21.15.59:0/2983760354 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0074e0 con 0x7f0c0c0ce390 2024-04-23T14:41:44.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c03400640 1 -- 172.21.15.59:0/2983760354 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc007680 con 0x7f0c0c0ce390 2024-04-23T14:41:44.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c03400640 1 -- 172.21.15.59:0/2983760354 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004a00 con 0x7f0c0c0ce390 2024-04-23T14:41:44.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c145e20 con 0x7f0c0c0ce390 2024-04-23T14:41:44.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.591+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c0af3b0 con 0x7f0c0c0ce390 2024-04-23T14:41:44.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.592+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c069190 con 0x7f0c0c0ce390 2024-04-23T14:41:44.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.593+0000 7f0c03400640 1 -- 172.21.15.59:0/2983760354 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bfc01a020 con 0x7f0c0c0ce390 2024-04-23T14:41:44.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.594+0000 7f0c03400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bec06c8f0 0x7f0bec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:44.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.595+0000 7f0c03400640 1 -- 172.21.15.59:0/2983760354 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f0bfc08b3d0 con 0x7f0c0c0ce390 2024-04-23T14:41:44.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.595+0000 7f0c0aa00640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bec06c8f0 0x7f0bec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:44.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.595+0000 7f0c0aa00640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bec06c8f0 0x7f0bec06edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8034000 tx=0x7f0bf802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:44.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.599+0000 7f0c03400640 1 -- 172.21.15.59:0/2983760354 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0bfc010300 con 0x7f0c0c0ce390 2024-04-23T14:41:44.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.835+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c0c06c810 con 0x7f0c0c0ce390 2024-04-23T14:41:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.836+0000 7f0c03400640 1 -- 172.21.15.59:0/2983760354 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5180 (secure 0 0 0) 0x7f0bfc059760 con 0x7f0c0c0ce390 2024-04-23T14:41:44.837 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:44.837 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:41:32.686386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8514":{"gid":8514,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/620420969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":620420969},{"type":"v1","addr":"172.21.15.59:6837","nonce":620420969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8514,"qdb_cluster":[8514]},"id":7}]} 2024-04-23T14:41:44.837 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 82 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bec06c8f0 msgr2=0x7f0bec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bec06c8f0 0x7f0bec06edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8034000 tx=0x7f0bf802e000 comp rx=0 tx=0).stop 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c0ce390 msgr2=0x7f0c0c1438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c0ce390 0x7f0c0c1438f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c3f0 tx=0x7f0bfc00c8b0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 shutdown_connections 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0bec06c8f0 0x7f0bec06edb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c0c15fea0 0x7f0c0c0afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c0c142070 0x7f0c0c145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 --2- 172.21.15.59:0/2983760354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c0c0ce390 0x7f0c0c1438f0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:44.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.838+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 >> 172.21.15.59:0/2983760354 conn(0x7f0c0c073450 msgr2=0x7f0c0c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:44.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.839+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 shutdown_connections 2024-04-23T14:41:44.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:44.839+0000 7f0c12400640 1 -- 172.21.15.59:0/2983760354 wait complete. 2024-04-23T14:41:44.857 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-23T14:41:45.095 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:41:45.095 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:41:45.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:41:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.302+0000 7fd2dac00640 1 Processor -- start 2024-04-23T14:41:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.302+0000 7fd2dac00640 1 -- start start 2024-04-23T14:41:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.304+0000 7fd2dac00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2d406f7c0 0x7fd2d40b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.304+0000 7fd2dac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.304+0000 7fd2dac00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d406ec60 con 0x7fd2d415b840 2024-04-23T14:41:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.304+0000 7fd2dac00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d40b46e0 con 0x7fd2d406f7c0 2024-04-23T14:41:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.304+0000 7fd2dac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d40b49c0 con 0x7fd2d406df00 2024-04-23T14:41:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2d406f7c0 0x7fd2d40b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2d406f7c0 0x7fd2d40b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49452/0 (socket says 172.21.15.59:49452) 2024-04-23T14:41:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d3e00640 1 -- 172.21.15.59:0/2139536299 learned_addr learned my addr 172.21.15.59:0/2139536299 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d8c00640 1 --2- 172.21.15.59:0/2139536299 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d3e00640 1 -- 172.21.15.59:0/2139536299 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 msgr2=0x7fd2d406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d3e00640 1 --2- 172.21.15.59:0/2139536299 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d3e00640 1 -- 172.21.15.59:0/2139536299 >> v1:172.21.15.76:6789/0 conn(0x7fd2d415b840 legacy=0x7fd2d406fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.305+0000 7fd2d3e00640 1 -- 172.21.15.59:0/2139536299 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d40b4ca0 con 0x7fd2d406f7c0 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.307+0000 7fd2d3e00640 1 --2- 172.21.15.59:0/2139536299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d40b54a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40098a0 tx=0x7fd2c402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edc34989a91e089 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.307+0000 7fd2d2a00640 1 -- 172.21.15.59:0/2139536299 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4037070 con 0x7fd2d406f7c0 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.307+0000 7fd2d2a00640 1 -- 172.21.15.59:0/2139536299 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4032440 con 0x7fd2d406f7c0 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.307+0000 7fd2d2a00640 1 -- 172.21.15.59:0/2139536299 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4032810 con 0x7fd2d406f7c0 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 -- 172.21.15.59:0/2139536299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 msgr2=0x7fd2d40b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 --2- 172.21.15.59:0/2139536299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d40b54a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40098a0 tx=0x7fd2c402a250 comp rx=0 tx=0).stop 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 -- 172.21.15.59:0/2139536299 shutdown_connections 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 --2- 172.21.15.59:0/2139536299 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 --2- 172.21.15.59:0/2139536299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d40b54a0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 -- 172.21.15.59:0/2139536299 >> 172.21.15.59:0/2139536299 conn(0x7fd2d4067ea0 msgr2=0x7fd2d406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 -- 172.21.15.59:0/2139536299 shutdown_connections 2024-04-23T14:41:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.308+0000 7fd2dac00640 1 -- 172.21.15.59:0/2139536299 wait complete. 2024-04-23T14:41:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.310+0000 7fd2dac00640 1 Processor -- start 2024-04-23T14:41:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.310+0000 7fd2dac00640 1 -- start start 2024-04-23T14:41:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2dac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d4151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2dac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d41458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2dac00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2d4145df0 0x7fd2d415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2dac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40ce1c0 con 0x7fd2d406f7c0 2024-04-23T14:41:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2dac00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d406e8d0 con 0x7fd2d4145df0 2024-04-23T14:41:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2dac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d40ce340 con 0x7fd2d406df00 2024-04-23T14:41:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d41458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d41458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49458/0 (socket says 172.21.15.59:49458) 2024-04-23T14:41:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.311+0000 7fd2d3400640 1 -- 172.21.15.59:0/1273395269 learned_addr learned my addr 172.21.15.59:0/1273395269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3e00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d4151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d8c00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2d4145df0 0x7fd2d415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3400640 1 -- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 msgr2=0x7fd2d4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3400640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d4151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3400640 1 -- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2d4145df0 msgr2=0x7fd2d415cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3400640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2d4145df0 0x7fd2d415cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3400640 1 -- 172.21.15.59:0/1273395269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d414a6a0 con 0x7fd2d406f7c0 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d8c00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2d4145df0 0x7fd2d415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3e00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d4151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2d3400640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d41458b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000b3f0 tx=0x7fd2c000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2cbe00640 1 -- 172.21.15.59:0/1273395269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0015020 con 0x7fd2d406f7c0 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.312+0000 7fd2cbe00640 1 -- 172.21.15.59:0/1273395269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0011830 con 0x7fd2d406f7c0 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.313+0000 7fd2cbe00640 1 -- 172.21.15.59:0/1273395269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0011c20 con 0x7fd2d406f7c0 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.313+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d414a980 con 0x7fd2d406f7c0 2024-04-23T14:41:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.313+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d41522a0 con 0x7fd2d406f7c0 2024-04-23T14:41:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.314+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d415c640 con 0x7fd2d406f7c0 2024-04-23T14:41:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.315+0000 7fd2cbe00640 1 -- 172.21.15.59:0/1273395269 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2c0018020 con 0x7fd2d406f7c0 2024-04-23T14:41:45.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.316+0000 7fd2cbe00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.316+0000 7fd2cbe00640 1 -- 172.21.15.59:0/1273395269 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fd2c008b760 con 0x7fd2d406f7c0 2024-04-23T14:41:45.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.316+0000 7fd2d3e00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.317+0000 7fd2d3e00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406c8f0 0x7fd2b406edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4034000 tx=0x7fd2c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.320+0000 7fd2cbe00640 1 -- 172.21.15.59:0/1273395269 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2c001f030 con 0x7fd2d406f7c0 2024-04-23T14:41:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.573+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2d414ab50 con 0x7fd2d406f7c0 2024-04-23T14:41:45.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.574+0000 7fd2cbe00640 1 -- 172.21.15.59:0/1273395269 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5180 (secure 0 0 0) 0x7fd2d414ab50 con 0x7fd2d406f7c0 2024-04-23T14:41:45.575 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:45.575 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:41:32.686386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8514":{"gid":8514,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/620420969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":620420969},{"type":"v1","addr":"172.21.15.59:6837","nonce":620420969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8514,"qdb_cluster":[8514]},"id":7}]} 2024-04-23T14:41:45.575 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 82 2024-04-23T14:41:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406c8f0 msgr2=0x7fd2b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406c8f0 0x7fd2b406edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4034000 tx=0x7fd2c402e000 comp rx=0 tx=0).stop 2024-04-23T14:41:45.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 msgr2=0x7fd2d41458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d41458b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000b3f0 tx=0x7fd2c000b8b0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 shutdown_connections 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2d4145df0 0x7fd2d415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2d406f7c0 0x7fd2d41458b0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 --2- 172.21.15.59:0/1273395269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2d406df00 0x7fd2d4151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 >> 172.21.15.59:0/1273395269 conn(0x7fd2d4067ea0 msgr2=0x7fd2d406fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 shutdown_connections 2024-04-23T14:41:45.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.577+0000 7fd2dac00640 1 -- 172.21.15.59:0/1273395269 wait complete. 2024-04-23T14:41:45.597 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-23T14:41:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.700+0000 7fc99ee2ef40 1 Processor -- start 2024-04-23T14:41:45.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.701+0000 7fc99ee2ef40 1 -- start start 2024-04-23T14:41:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.701+0000 7fc99ee2ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9800022f0 0x55893ff89ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.701+0000 7fc99ee2ef40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 0x55893ff02ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.701+0000 7fc99ee2ef40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55893ff8ef70 0x55893ff03010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99ee2ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55893fe5dcc0 con 0x7fc9800022f0 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99ee2ef40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55893fe61d00 con 0x55893ff8ab10 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99ee2ef40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55893ff01ec0 con 0x55893ff8ef70 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9800022f0 0x55893ff89ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9800022f0 0x55893ff89ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49472/0 (socket says 172.21.15.59:49472) 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 -- 172.21.15.59:0/2482328558 learned_addr learned my addr 172.21.15.59:0/2482328558 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99b600640 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 0x55893ff02ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99ca00640 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55893ff8ef70 0x55893ff03010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 -- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55893ff8ef70 msgr2=0x55893ff03010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55893ff8ef70 0x55893ff03010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 -- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 msgr2=0x55893ff02ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 0x55893ff02ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.702+0000 7fc99c000640 1 -- 172.21.15.59:0/2482328558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893ff773f0 con 0x7fc9800022f0 2024-04-23T14:41:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.703+0000 7fc99c000640 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9800022f0 0x55893ff89ae0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fc98c046540 tx=0x7fc98c04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fecb5af3b8bdae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.703+0000 7fc99a200640 1 -- 172.21.15.59:0/2482328558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c005050 con 0x7fc9800022f0 2024-04-23T14:41:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.703+0000 7fc99a200640 1 -- 172.21.15.59:0/2482328558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c005210 con 0x7fc9800022f0 2024-04-23T14:41:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.703+0000 7fc99a200640 1 -- 172.21.15.59:0/2482328558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c04d910 con 0x7fc9800022f0 2024-04-23T14:41:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/2482328558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9800022f0 msgr2=0x55893ff89ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9800022f0 0x55893ff89ae0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fc98c046540 tx=0x7fc98c04f000 comp rx=0 tx=0).stop 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/2482328558 shutdown_connections 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55893ff8ef70 0x55893ff03010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 0x55893ff02ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/2482328558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9800022f0 0x55893ff89ae0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/2482328558 >> 172.21.15.59:0/2482328558 conn(0x55893ff00cc0 msgr2=0x55893ff90790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/2482328558 shutdown_connections 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.704+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/2482328558 wait complete. 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.705+0000 7fc99ee2ef40 1 Processor -- start 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.705+0000 7fc99ee2ef40 1 -- start start 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ee2ef40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 0x55893ff740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ee2ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55893ff8ef70 0x55893ff71490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ee2ef40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9800022f0 0x55893ff719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ee2ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55893ff72880 con 0x55893ff8ef70 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ee2ef40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55893ff8b630 con 0x55893ff8ab10 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ee2ef40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55893fee5560 con 0x7fc9800022f0 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55893ff8ef70 0x55893ff71490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55893ff8ef70 0x55893ff71490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49482/0 (socket says 172.21.15.59:49482) 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 -- 172.21.15.59:0/607840348 learned_addr learned my addr 172.21.15.59:0/607840348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99c000640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 0x55893ff740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ca00640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9800022f0 0x55893ff719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 -- 172.21.15.59:0/607840348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9800022f0 msgr2=0x55893ff719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9800022f0 0x55893ff719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 -- 172.21.15.59:0/607840348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 msgr2=0x55893ff740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55893ff8ab10 0x55893ff740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99b600640 1 -- 172.21.15.59:0/607840348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893ff02ef0 con 0x55893ff8ef70 2024-04-23T14:41:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.706+0000 7fc99ca00640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9800022f0 0x55893ff719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.707+0000 7fc99b600640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55893ff8ef70 0x55893ff71490 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b520 tx=0x7fc98800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.707+0000 7fc998e00640 1 -- 172.21.15.59:0/607840348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988014020 con 0x55893ff8ef70 2024-04-23T14:41:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.707+0000 7fc998e00640 1 -- 172.21.15.59:0/607840348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98800f640 con 0x55893ff8ef70 2024-04-23T14:41:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.707+0000 7fc998e00640 1 -- 172.21.15.59:0/607840348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988008030 con 0x55893ff8ef70 2024-04-23T14:41:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.707+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55893ff031d0 con 0x55893ff8ef70 2024-04-23T14:41:45.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.707+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893ff72290 con 0x55893ff8ef70 2024-04-23T14:41:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.709+0000 7fc998e00640 1 -- 172.21.15.59:0/607840348 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc988017020 con 0x55893ff8ef70 2024-04-23T14:41:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.710+0000 7fc998e00640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc97c06b2d0 0x7fc97c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.710+0000 7fc998e00640 1 -- 172.21.15.59:0/607840348 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fc98808ab90 con 0x55893ff8ef70 2024-04-23T14:41:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.710+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55893ff82ec0 0x55893ff832a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.710+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x55893ff837e0 con 0x55893ff82ec0 2024-04-23T14:41:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.711+0000 7fc99c000640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc97c06b2d0 0x7fc97c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.711+0000 7fc99ca00640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55893ff82ec0 0x55893ff832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.711+0000 7fc99c000640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc97c06b2d0 0x7fc97c06d790 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55893fea0fe0 tx=0x7fc98c051000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.711+0000 7fc99ca00640 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55893ff82ec0 0x55893ff832a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ca00640 1 -- 172.21.15.59:0/607840348 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fc994011420 con 0x55893ff82ec0 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55893ff82ec0 msgr2=0x55893ff832a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55893ff82ec0 0x55893ff832a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc97c06b2d0 msgr2=0x7fc97c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc97c06b2d0 0x7fc97c06d790 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55893fea0fe0 tx=0x7fc98c051000 comp rx=0 tx=0).stop 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55893ff8ef70 msgr2=0x55893ff71490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 --2- 172.21.15.59:0/607840348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55893ff8ef70 0x55893ff71490 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b520 tx=0x7fc98800b9e0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ca00640 1 -- 172.21.15.59:0/607840348 reap_dead start 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 shutdown_connections 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 >> 172.21.15.59:0/607840348 conn(0x55893ff00cc0 msgr2=0x55893fee4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 shutdown_connections 2024-04-23T14:41:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.712+0000 7fc99ee2ef40 1 -- 172.21.15.59:0/607840348 wait complete. 2024-04-23T14:41:45.718 INFO:teuthology.orchestra.run.smithi059.stdout:����� 2024-04-23T14:41:45.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:41:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.911+0000 7f36fce00640 1 Processor -- start 2024-04-23T14:41:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.912+0000 7f36fce00640 1 -- start start 2024-04-23T14:41:45.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.913+0000 7f36fce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 0x7f36f806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.913+0000 7f36fce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f815f160 con 0x7f36f80b3080 2024-04-23T14:41:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.913+0000 7f36fce00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f815f430 con 0x7f36f815ece0 2024-04-23T14:41:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.913+0000 7f36fce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f36f80ced20 con 0x7f36f80cd050 2024-04-23T14:41:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.914+0000 7f36f7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f36f80b3080 0x7f36f80cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56528/0 (socket says 172.21.15.59:56528) 2024-04-23T14:41:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.914+0000 7f36f7e00640 1 -- 172.21.15.59:0/3837772991 learned_addr learned my addr 172.21.15.59:0/3837772991 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.914+0000 7f36f7400640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 0x7f36f806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.914+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4102239289 0 0) 0x7f36f815f160 con 0x7f36f80b3080 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.914+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36dc003520 con 0x7f36f80b3080 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3728436249 0 0) 0x7f36dc003520 con 0x7f36f80b3080 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 msgr2=0x7f36f806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 0x7f36f806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 >> v1:172.21.15.76:6789/0 conn(0x7f36f815ece0 legacy=0x7f36f80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f815f710 con 0x7f36f80b3080 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f36f00019f0 con 0x7f36f80b3080 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 >> v1:172.21.15.59:6789/0 conn(0x7f36f80b3080 legacy=0x7f36f80cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36dc0039e0 0x7f36dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 0x7f36dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36dc004d40 0x7f36dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc00ba70 con 0x7f36dc004d40 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc00bbf0 con 0x7f36dc0039e0 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f36dc00bdf0 con 0x7f36f80cd050 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.915+0000 7f36f7e00640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36dc004d40 0x7f36dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.916+0000 7f36f6a00640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 0x7f36dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.916+0000 7f36f7400640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36dc0039e0 0x7f36dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.916+0000 7f36f7e00640 1 -- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 msgr2=0x7f36dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.916+0000 7f36f7e00640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f80cd050 0x7f36dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.916+0000 7f36f7e00640 1 -- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36dc0039e0 msgr2=0x7f36dc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.916+0000 7f36f7e00640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36dc0039e0 0x7f36dc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.916+0000 7f36f7e00640 1 -- 172.21.15.59:0/3837772991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc003520 con 0x7f36dc004d40 2024-04-23T14:41:45.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.917+0000 7f36f7e00640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36dc004d40 0x7f36dc0051c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f36f000a920 tx=0x7f36dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eecc4f8593de3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.917+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f000a4c0 con 0x7f36dc004d40 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.917+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f002f070 con 0x7f36dc004d40 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.917+0000 7f36f6000640 1 -- 172.21.15.59:0/3837772991 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f002bce0 con 0x7f36dc004d40 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.918+0000 7f36fce00640 1 -- 172.21.15.59:0/3837772991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36dc004d40 msgr2=0x7f36dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.918+0000 7f36fce00640 1 --2- 172.21.15.59:0/3837772991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36dc004d40 0x7f36dc0051c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f36f000a920 tx=0x7f36dc005830 comp rx=0 tx=0).stop 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.918+0000 7f36f7e00640 1 -- 172.21.15.59:0/3837772991 reap_dead start 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.918+0000 7f36fce00640 1 -- 172.21.15.59:0/3837772991 shutdown_connections 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.918+0000 7f36fce00640 1 -- 172.21.15.59:0/3837772991 >> 172.21.15.59:0/3837772991 conn(0x7f36f80b21a0 msgr2=0x7f36f80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.918+0000 7f36fce00640 1 -- 172.21.15.59:0/3837772991 shutdown_connections 2024-04-23T14:41:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.918+0000 7f36fce00640 1 -- 172.21.15.59:0/3837772991 wait complete. 2024-04-23T14:41:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.920+0000 7f36fce00640 1 Processor -- start 2024-04-23T14:41:45.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.920+0000 7f36fce00640 1 -- start start 2024-04-23T14:41:45.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36fce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f8150fb0 0x7f36f80c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36fce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f80c8260 0x7f36f81f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36fce00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36f81f4b40 0x7f36f81f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36fce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36f80cef50 con 0x7f36f80c8260 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36fce00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f36f815fcd0 con 0x7f36f81f4b40 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36fce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f36f8159110 con 0x7f36f8150fb0 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36f6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f80c8260 0x7f36f81f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36f6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f80c8260 0x7f36f81f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49504/0 (socket says 172.21.15.59:49504) 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.921+0000 7f36f6a00640 1 -- 172.21.15.59:0/793856331 learned_addr learned my addr 172.21.15.59:0/793856331 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f7400640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f8150fb0 0x7f36f80c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f6a00640 1 -- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f8150fb0 msgr2=0x7f36f80c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f7e00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36f81f4b40 0x7f36f81f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f6a00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f8150fb0 0x7f36f80c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f6a00640 1 -- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36f81f4b40 msgr2=0x7f36f81f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f6a00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36f81f4b40 0x7f36f81f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f6a00640 1 -- 172.21.15.59:0/793856331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f80c7560 con 0x7f36f80c8260 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f7400640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f8150fb0 0x7f36f80c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:45.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36f6a00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f80c8260 0x7f36f81f4600 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f36e000e5d0 tx=0x7f36e000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36ef400640 1 -- 172.21.15.59:0/793856331 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00053a0 con 0x7f36f80c8260 2024-04-23T14:41:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36ef400640 1 -- 172.21.15.59:0/793856331 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e0018070 con 0x7f36f80c8260 2024-04-23T14:41:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36ef400640 1 -- 172.21.15.59:0/793856331 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0003830 con 0x7f36f80c8260 2024-04-23T14:41:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f809c570 con 0x7f36f80c8260 2024-04-23T14:41:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.922+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f809cb30 con 0x7f36f80c8260 2024-04-23T14:41:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.924+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f804fd90 con 0x7f36f80c8260 2024-04-23T14:41:45.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.925+0000 7f36ef400640 1 -- 172.21.15.59:0/793856331 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36e00039d0 con 0x7f36f80c8260 2024-04-23T14:41:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.926+0000 7f36ef400640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f36d406c920 0x7f36d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.926+0000 7f36ef400640 1 -- 172.21.15.59:0/793856331 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f36e008b8f0 con 0x7f36f80c8260 2024-04-23T14:41:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.926+0000 7f36f7400640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f36d406c920 0x7f36d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:45.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.927+0000 7f36f7400640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f36d406c920 0x7f36d406ede0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009a00 tx=0x7f36e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:45.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:45.931+0000 7f36ef400640 1 -- 172.21.15.59:0/793856331 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36e0059d20 con 0x7f36f80c8260 2024-04-23T14:41:46.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.182+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36f809e140 con 0x7f36f80c8260 2024-04-23T14:41:46.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.183+0000 7f36ef400640 1 -- 172.21.15.59:0/793856331 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f36e0059620 con 0x7f36f80c8260 2024-04-23T14:41:46.184 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f36d406c920 msgr2=0x7f36d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f36d406c920 0x7f36d406ede0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009a00 tx=0x7f36e8008040 comp rx=0 tx=0).stop 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f80c8260 msgr2=0x7f36f81f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f80c8260 0x7f36f81f4600 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f36e000e5d0 tx=0x7f36e000ea90 comp rx=0 tx=0).stop 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 shutdown_connections 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f36d406c920 0x7f36d406ede0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36f81f4b40 0x7f36f81f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36f80c8260 0x7f36f81f4600 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 --2- 172.21.15.59:0/793856331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36f8150fb0 0x7f36f80c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 >> 172.21.15.59:0/793856331 conn(0x7f36f80b21a0 msgr2=0x7f36f80ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 shutdown_connections 2024-04-23T14:41:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46.186+0000 7f36fce00640 1 -- 172.21.15.59:0/793856331 wait complete. 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:41:46.206 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:41:46.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:46.206 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:41:46.261 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:41:46.261 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:41:46.261 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:41:46.261 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:46.261 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:41:46.261 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:46.261 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41843sec preferred_lft 41843sec 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:46.262 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:46.263 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:41:46.264 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:41:46.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:46.264 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:41:46.342 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:41:46.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:46.346 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:41:46.400 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:41:46.400 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:41:46.400 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:41:46.400 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:41:46.400 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:41:46.400 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:41:46.400 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:41:46.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:41:46.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:41:46.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:41:46.401 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:41:46.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:41:46.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:46.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:46.621 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:41:46.672 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:41:46.672 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:41:46.673 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:41:46.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:46.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:41:46.764 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:41:46.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:41:46.764 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:46.822 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:41:46.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:41:46.823 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:46.875 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:41:46.964 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:41:47.023 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:41:47.024 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:47.025 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:47.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:47.026 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:41:47.084 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:41:47.084 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:41:47.089 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:41:47.159 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:41:47.204 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:41:47.204 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:41:47.205 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:41:47.206 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:41:47.207 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:41:47.207 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:41:47.207 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:41:47.207 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:47.207 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:41:47.207 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:47.207 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:41:47.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:47.208 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:41:47.260 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:41:47.260+0000 7fdcaa671480 -1 init, newargv = 0x561b1b9cb660 newargc=15 2024-04-23T14:41:47.261 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[71171]: starting ceph client 2024-04-23T14:41:47.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[71171]: starting fuse 2024-04-23T14:41:47.291 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:41:47.291 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:41:47.291 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:41:47.291 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:41:47.291 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:41:47.291 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:41:47.292 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:41:47.293 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:41:47.293 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:41:47.293 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:41:47.293 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:41:47.293 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:47.293 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:41:47.293 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:41:47.489 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.71171.asok 2024-04-23T14:41:47.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:47.506 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.71171.asok 2024-04-23T14:41:47.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:47.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.71171.asok status 2024-04-23T14:41:47.742 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:41:47.742 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "71171", 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8772, 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:41:47.743 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 8772 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2944420328 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2944420328 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.8772 192.168.0.1:0/2944420328", 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2944420328", 2024-04-23T14:41:47.744 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:41:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 82, 2024-04-23T14:41:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 99, 2024-04-23T14:41:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:41:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:41:47.745 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:41:47.745 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:41:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:47.758 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:41:47.758 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:47.815 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:41:47.816 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:47.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:47.816 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:47.904 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:47.904 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:47.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:47.962 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-23T14:41:48.186 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records in 2024-04-23T14:41:48.186 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records out 2024-04-23T14:41:48.187 INFO:teuthology.orchestra.run.smithi059.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.157882 s, 39.8 MB/s 2024-04-23T14:41:48.189 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:48.189 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:48.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:48.245 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-23T14:41:48.457 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records in 2024-04-23T14:41:48.458 INFO:teuthology.orchestra.run.smithi059.stderr:6+0 records out 2024-04-23T14:41:48.458 INFO:teuthology.orchestra.run.smithi059.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.154411 s, 40.7 MB/s 2024-04-23T14:41:48.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:48.460 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-23T14:41:48.527 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-23T14:41:48.527 INFO:teuthology.orchestra.run.smithi059.stdout:./file2_sixmegs 2024-04-23T14:41:48.527 INFO:teuthology.orchestra.run.smithi059.stdout:./file3_sixmegs 2024-04-23T14:41:48.527 INFO:teuthology.orchestra.run.smithi059.stdout:./file1_sixmegs 2024-04-23T14:41:48.528 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:41:48.528 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:41:48.528 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:41:48.528 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:48.528 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-23T14:41:48.529 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:41:48.666 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:41:48.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:48.681 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:41:48.682 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:41:48.682 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:41:48.682 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:48.682 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-23T14:41:48.682 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:41:48.875 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-23T14:41:48.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:48.890 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:41:48.890 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:41:48.890 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:41:48.890 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:48.890 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-23T14:41:48.890 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:41:49.083 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-23T14:41:49.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:49.098 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:41:49.098 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:41:49.098 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:41:49.098 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:41:49.098 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-23T14:41:49.098 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:41:49.294 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-23T14:41:49.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:41:49.308 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:49.308 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:49.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:49.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.617+0000 7f883d800640 1 Processor -- start 2024-04-23T14:41:49.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.617+0000 7f883d800640 1 -- start start 2024-04-23T14:41:49.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.619+0000 7f883d800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 0x7f883806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.619+0000 7f883d800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883815f160 con 0x7f88380b3080 2024-04-23T14:41:49.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.619+0000 7f883d800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883815f430 con 0x7f883815ece0 2024-04-23T14:41:49.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.619+0000 7f883d800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f88380ced20 con 0x7f88380cd050 2024-04-23T14:41:49.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.620+0000 7f883ce00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f88380b3080 0x7f88380cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56534/0 (socket says 172.21.15.59:56534) 2024-04-23T14:41:49.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.620+0000 7f883ce00640 1 -- 172.21.15.59:0/1991932242 learned_addr learned my addr 172.21.15.59:0/1991932242 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:49.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.620+0000 7f8837e00640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 0x7f883806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:49.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.620+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379077855 0 0) 0x7f883815f160 con 0x7f88380b3080 2024-04-23T14:41:49.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.620+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f881c003520 con 0x7f88380b3080 2024-04-23T14:41:49.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1027142303 0 0) 0x7f881c003520 con 0x7f88380b3080 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 msgr2=0x7f883806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 0x7f883806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 >> v1:172.21.15.76:6789/0 conn(0x7f883815ece0 legacy=0x7f88380cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883815f710 con 0x7f88380b3080 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8830001670 con 0x7f88380b3080 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 >> v1:172.21.15.59:6789/0 conn(0x7f88380b3080 legacy=0x7f88380cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 0x7f881c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f881c003f60 0x7f881c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c004d40 0x7f881c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f881c00bb50 con 0x7f881c003f60 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f881c00bcd0 con 0x7f881c004d40 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f881c00bed0 con 0x7f88380cd050 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8837400640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f881c003f60 0x7f881c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:49.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.621+0000 7f8837e00640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 0x7f881c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:49.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.622+0000 7f8837400640 1 -- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 msgr2=0x7f881c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:49.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.622+0000 7f8837400640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 0x7f881c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.622+0000 7f8837400640 1 -- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c004d40 msgr2=0x7f881c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:49.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.622+0000 7f8837400640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c004d40 0x7f881c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.622+0000 7f8837400640 1 -- 172.21.15.59:0/1991932242 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c003520 con 0x7f881c003f60 2024-04-23T14:41:49.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.622+0000 7f8837e00640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88380cd050 0x7f881c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:49.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.623+0000 7f8837400640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f881c003f60 0x7f881c004800 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8824001e50 tx=0x7f881c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bd719a6de79f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.623+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824001a70 con 0x7f881c003f60 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.623+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8824003970 con 0x7f881c003f60 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.623+0000 7f8836a00640 1 -- 172.21.15.59:0/1991932242 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824003d60 con 0x7f881c003f60 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.624+0000 7f883d800640 1 -- 172.21.15.59:0/1991932242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f881c003f60 msgr2=0x7f881c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.624+0000 7f883d800640 1 --2- 172.21.15.59:0/1991932242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f881c003f60 0x7f881c004800 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8824001e50 tx=0x7f881c005830 comp rx=0 tx=0).stop 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.624+0000 7f883ce00640 1 -- 172.21.15.59:0/1991932242 reap_dead start 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.624+0000 7f883d800640 1 -- 172.21.15.59:0/1991932242 shutdown_connections 2024-04-23T14:41:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.624+0000 7f883d800640 1 -- 172.21.15.59:0/1991932242 >> 172.21.15.59:0/1991932242 conn(0x7f88380b21a0 msgr2=0x7f88380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:49.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.624+0000 7f883d800640 1 -- 172.21.15.59:0/1991932242 shutdown_connections 2024-04-23T14:41:49.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.624+0000 7f883d800640 1 -- 172.21.15.59:0/1991932242 wait complete. 2024-04-23T14:41:49.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.626+0000 7f883d800640 1 Processor -- start 2024-04-23T14:41:49.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.626+0000 7f883d800640 1 -- start start 2024-04-23T14:41:49.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f883d800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c003f60 0x7f88381f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f883d800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380c6700 0x7f8838072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f883d800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f883809da10 0x7f88380c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f883d800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88380cef50 con 0x7f88380c6700 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f883d800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f883815fcd0 con 0x7f881c003f60 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f883d800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8838159110 con 0x7f883809da10 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f8837400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380c6700 0x7f8838072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f8837400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380c6700 0x7f8838072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49528/0 (socket says 172.21.15.59:49528) 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f8837400640 1 -- 172.21.15.59:0/3713124407 learned_addr learned my addr 172.21.15.59:0/3713124407 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f8837e00640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c003f60 0x7f88381f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f883ce00640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f883809da10 0x7f88380c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.627+0000 7f8837400640 1 -- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f883809da10 msgr2=0x7f88380c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f8837400640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f883809da10 0x7f88380c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f8837400640 1 -- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c003f60 msgr2=0x7f88381f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f8837400640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c003f60 0x7f88381f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f8837400640 1 -- 172.21.15.59:0/3713124407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8838071ea0 con 0x7f88380c6700 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f8837e00640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c003f60 0x7f88381f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f883ce00640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f883809da10 0x7f88380c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f8837400640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380c6700 0x7f8838072f40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f882402a920 tx=0x7f88240014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f882fe00640 1 -- 172.21.15.59:0/3713124407 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824040020 con 0x7f88380c6700 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f882fe00640 1 -- 172.21.15.59:0/3713124407 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88240052e0 con 0x7f88380c6700 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f882fe00640 1 -- 172.21.15.59:0/3713124407 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88240056d0 con 0x7f88380c6700 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.628+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8838072180 con 0x7f88380c6700 2024-04-23T14:41:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.629+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8838072740 con 0x7f88380c6700 2024-04-23T14:41:49.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.630+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8838072b80 con 0x7f88380c6700 2024-04-23T14:41:49.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.631+0000 7f882fe00640 1 -- 172.21.15.59:0/3713124407 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8824043020 con 0x7f88380c6700 2024-04-23T14:41:49.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.631+0000 7f882fe00640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f881806c940 0x7f881806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:49.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.632+0000 7f882fe00640 1 -- 172.21.15.59:0/3713124407 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f88240ac130 con 0x7f88380c6700 2024-04-23T14:41:49.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.632+0000 7f8837e00640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f881806c940 0x7f881806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:49.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.632+0000 7f8837e00640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f881806c940 0x7f881806ee00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f88380c64f0 tx=0x7f8828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:49.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.636+0000 7f882fe00640 1 -- 172.21.15.59:0/3713124407 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f882404a020 con 0x7f88380c6700 2024-04-23T14:41:49.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.840+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f883804fd90 con 0x7f88380c6700 2024-04-23T14:41:49.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.843+0000 7f882fe00640 1 -- 172.21.15.59:0/3713124407 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16548 (secure 0 0 0) 0x7f882403a360 con 0x7f88380c6700 2024-04-23T14:41:49.845 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:49.845 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":99,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:31.158736+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f881806c940 msgr2=0x7f881806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f881806c940 0x7f881806ee00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f88380c64f0 tx=0x7f8828008040 comp rx=0 tx=0).stop 2024-04-23T14:41:49.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380c6700 msgr2=0x7f8838072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380c6700 0x7f8838072f40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f882402a920 tx=0x7f88240014a0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 shutdown_connections 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f881806c940 0x7f881806ee00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f883809da10 0x7f88380c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380c6700 0x7f8838072f40 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 --2- 172.21.15.59:0/3713124407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f881c003f60 0x7f88381f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 >> 172.21.15.59:0/3713124407 conn(0x7f88380b21a0 msgr2=0x7f88380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.846+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 shutdown_connections 2024-04-23T14:41:49.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:49.847+0000 7f883d800640 1 -- 172.21.15.59:0/3713124407 wait complete. 2024-04-23T14:41:49.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:41:27.465708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:41:28.586680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:49.913 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.132+0000 7fc95fe00640 1 Processor -- start 2024-04-23T14:41:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.133+0000 7fc95fe00640 1 -- start start 2024-04-23T14:41:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.134+0000 7fc95fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc96015ed20 0x7fc9600d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.134+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9600b3600 0x7fc9600d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.134+0000 7fc95fe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9600b5da0 con 0x7fc9600b3f10 2024-04-23T14:41:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.134+0000 7fc95fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc96015f100 con 0x7fc96015ed20 2024-04-23T14:41:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.134+0000 7fc95fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc9600d0a20 con 0x7fc9600b3600 2024-04-23T14:41:50.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.135+0000 7fc956000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9600b3600 0x7fc9600d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.135+0000 7fc95ea00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc96015ed20 0x7fc9600d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.135+0000 7fc95ea00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc96015ed20 0x7fc9600d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:54328/0 (socket says 172.21.15.59:54328) 2024-04-23T14:41:50.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.135+0000 7fc95ea00640 1 -- 172.21.15.59:0/799261698 learned_addr learned my addr 172.21.15.59:0/799261698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:50.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.136+0000 7fc956000640 1 -- 172.21.15.59:0/799261698 >> v2:172.21.15.76:3300/0 conn(0x7fc96015ed20 msgr2=0x7fc9600d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:50.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.136+0000 7fc956000640 1 --2- 172.21.15.59:0/799261698 >> v2:172.21.15.76:3300/0 conn(0x7fc96015ed20 0x7fc9600d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.136+0000 7fc956000640 1 -- 172.21.15.59:0/799261698 >> v1:172.21.15.76:6789/0 conn(0x7fc9600b3f10 legacy=0x7fc9600b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.136+0000 7fc956000640 1 -- 172.21.15.59:0/799261698 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9600d2ed0 con 0x7fc9600b3600 2024-04-23T14:41:50.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.136+0000 7fc95ea00640 1 --2- 172.21.15.59:0/799261698 >> v2:172.21.15.76:3300/0 conn(0x7fc96015ed20 0x7fc9600d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:41:50.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.138+0000 7fc956000640 1 --2- 172.21.15.59:0/799261698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9600b3600 0x7fc9600d13e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9480097a0 tx=0x7fc94802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d541fc7869480f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.138+0000 7fc95e000640 1 -- 172.21.15.59:0/799261698 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948037070 con 0x7fc9600b3600 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.138+0000 7fc95e000640 1 -- 172.21.15.59:0/799261698 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94802adb0 con 0x7fc9600b3600 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.138+0000 7fc95e000640 1 -- 172.21.15.59:0/799261698 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948032660 con 0x7fc9600b3600 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 -- 172.21.15.59:0/799261698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9600b3600 msgr2=0x7fc9600d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 --2- 172.21.15.59:0/799261698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9600b3600 0x7fc9600d13e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9480097a0 tx=0x7fc94802a060 comp rx=0 tx=0).stop 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 -- 172.21.15.59:0/799261698 shutdown_connections 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 --2- 172.21.15.59:0/799261698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc9600b3600 0x7fc9600d13e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 --2- 172.21.15.59:0/799261698 >> v2:172.21.15.76:3300/0 conn(0x7fc96015ed20 0x7fc9600d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 -- 172.21.15.59:0/799261698 >> 172.21.15.59:0/799261698 conn(0x7fc96008bf90 msgr2=0x7fc9600d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 -- 172.21.15.59:0/799261698 shutdown_connections 2024-04-23T14:41:50.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.139+0000 7fc95fe00640 1 -- 172.21.15.59:0/799261698 wait complete. 2024-04-23T14:41:50.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.141+0000 7fc95fe00640 1 Processor -- start 2024-04-23T14:41:50.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.141+0000 7fc95fe00640 1 -- start start 2024-04-23T14:41:50.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.142+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9600b3600 0x7fc9600ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.142+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9600b3f10 0x7fc9600b9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.142+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96015ed20 0x7fc96015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.142+0000 7fc95fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9600d0290 con 0x7fc9600b3600 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.142+0000 7fc95fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9600d3490 con 0x7fc9600b3f10 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.142+0000 7fc95fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc9600d0410 con 0x7fc96015ed20 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.142+0000 7fc95ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9600b3600 0x7fc9600ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9600b3600 0x7fc9600ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40650/0 (socket says 172.21.15.59:40650) 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 -- 172.21.15.59:0/75143568 learned_addr learned my addr 172.21.15.59:0/75143568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc956000640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9600b3f10 0x7fc9600b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95f400640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96015ed20 0x7fc96015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 -- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96015ed20 msgr2=0x7fc96015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96015ed20 0x7fc96015b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 -- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9600b3f10 msgr2=0x7fc9600b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9600b3f10 0x7fc9600b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 -- 172.21.15.59:0/75143568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9600cd290 con 0x7fc9600b3600 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95f400640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96015ed20 0x7fc96015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:50.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.143+0000 7fc95ea00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9600b3600 0x7fc9600ca930 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc950009e80 tx=0x7fc95000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.144+0000 7fc957400640 1 -- 172.21.15.59:0/75143568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95000ce60 con 0x7fc9600b3600 2024-04-23T14:41:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.144+0000 7fc957400640 1 -- 172.21.15.59:0/75143568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc950019070 con 0x7fc9600b3600 2024-04-23T14:41:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.144+0000 7fc957400640 1 -- 172.21.15.59:0/75143568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950014680 con 0x7fc9600b3600 2024-04-23T14:41:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.144+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9600ce010 con 0x7fc9600b3600 2024-04-23T14:41:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.144+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9600cd810 con 0x7fc9600b3600 2024-04-23T14:41:50.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.145+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96004fd90 con 0x7fc9600b3600 2024-04-23T14:41:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.146+0000 7fc957400640 1 -- 172.21.15.59:0/75143568 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc950005d10 con 0x7fc9600b3600 2024-04-23T14:41:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.147+0000 7fc957400640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc93c06c8f0 0x7fc93c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.147+0000 7fc957400640 1 -- 172.21.15.59:0/75143568 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fc95008c0c0 con 0x7fc9600b3600 2024-04-23T14:41:50.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.147+0000 7fc956000640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc93c06c8f0 0x7fc93c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.148+0000 7fc956000640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc93c06c8f0 0x7fc93c06edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc948034000 tx=0x7fc94802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.152+0000 7fc957400640 1 -- 172.21.15.59:0/75143568 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc950010040 con 0x7fc9600b3600 2024-04-23T14:41:50.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.359+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc960068f10 con 0x7fc9600b3600 2024-04-23T14:41:50.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.360+0000 7fc957400640 1 -- 172.21.15.59:0/75143568 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fc95005a450 con 0x7fc9600b3600 2024-04-23T14:41:50.362 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:50.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.362+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc93c06c8f0 msgr2=0x7fc93c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.362+0000 7fc95fe00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc93c06c8f0 0x7fc93c06edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc948034000 tx=0x7fc94802e000 comp rx=0 tx=0).stop 2024-04-23T14:41:50.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.362+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9600b3600 msgr2=0x7fc9600ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.362+0000 7fc95fe00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9600b3600 0x7fc9600ca930 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc950009e80 tx=0x7fc95000c490 comp rx=0 tx=0).stop 2024-04-23T14:41:50.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 shutdown_connections 2024-04-23T14:41:50.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc93c06c8f0 0x7fc93c06edb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96015ed20 0x7fc96015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9600b3f10 0x7fc9600b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 --2- 172.21.15.59:0/75143568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9600b3600 0x7fc9600ca930 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 >> 172.21.15.59:0/75143568 conn(0x7fc96008bf90 msgr2=0x7fc96015d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:50.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 shutdown_connections 2024-04-23T14:41:50.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.363+0000 7fc95fe00640 1 -- 172.21.15.59:0/75143568 wait complete. 2024-04-23T14:41:50.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:41:50.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.601+0000 7f6310400640 1 Processor -- start 2024-04-23T14:41:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.601+0000 7f6310400640 1 -- start start 2024-04-23T14:41:50.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.603+0000 7f6310400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6308145e10 0x7f630806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.603+0000 7f6310400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63080d0cc0 0x7f63080cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.603+0000 7f6310400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63080cfc20 con 0x7f6308071d70 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.603+0000 7f6310400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f63080cff00 con 0x7f6308145e10 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.603+0000 7f6310400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f63080d01e0 con 0x7f63080d0cc0 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.603+0000 7f630da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6308145e10 0x7f630806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.604+0000 7f630da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6308145e10 0x7f630806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40666/0 (socket says 172.21.15.59:40666) 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.604+0000 7f630da00640 1 -- 172.21.15.59:0/3585662292 learned_addr learned my addr 172.21.15.59:0/3585662292 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.604+0000 7f630da00640 1 -- 172.21.15.59:0/3585662292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63080d0cc0 msgr2=0x7f63080cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.604+0000 7f630da00640 1 --2- 172.21.15.59:0/3585662292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63080d0cc0 0x7f63080cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.604+0000 7f630da00640 1 -- 172.21.15.59:0/3585662292 >> v1:172.21.15.76:6789/0 conn(0x7f6308071d70 legacy=0x7f630815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:50.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.604+0000 7f630da00640 1 -- 172.21.15.59:0/3585662292 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63080d04c0 con 0x7f6308145e10 2024-04-23T14:41:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.605+0000 7f630da00640 1 --2- 172.21.15.59:0/3585662292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6308145e10 0x7f630806f130 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0098a0 tx=0x7f62fc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f60a4aae666f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:50.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6307e00640 1 -- 172.21.15.59:0/3585662292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc037070 con 0x7f6308145e10 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6307e00640 1 -- 172.21.15.59:0/3585662292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc032440 con 0x7f6308145e10 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6307e00640 1 -- 172.21.15.59:0/3585662292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc032810 con 0x7f6308145e10 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 -- 172.21.15.59:0/3585662292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6308145e10 msgr2=0x7f630806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 --2- 172.21.15.59:0/3585662292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6308145e10 0x7f630806f130 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0098a0 tx=0x7f62fc02a250 comp rx=0 tx=0).stop 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 -- 172.21.15.59:0/3585662292 shutdown_connections 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 --2- 172.21.15.59:0/3585662292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63080d0cc0 0x7f63080cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 --2- 172.21.15.59:0/3585662292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6308145e10 0x7f630806f130 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 -- 172.21.15.59:0/3585662292 >> 172.21.15.59:0/3585662292 conn(0x7f6308067d90 msgr2=0x7f630806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 -- 172.21.15.59:0/3585662292 shutdown_connections 2024-04-23T14:41:50.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.606+0000 7f6310400640 1 -- 172.21.15.59:0/3585662292 wait complete. 2024-04-23T14:41:50.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.608+0000 7f6310400640 1 Processor -- start 2024-04-23T14:41:50.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.608+0000 7f6310400640 1 -- start start 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.609+0000 7f6310400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308071d70 0x7f63080a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.609+0000 7f6310400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63080d0cc0 0x7f63080adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.609+0000 7f6310400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6308145e10 0x7f63080ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.609+0000 7f6310400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63080cfa00 con 0x7f63080d0cc0 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.609+0000 7f6310400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63080cf880 con 0x7f6308071d70 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.609+0000 7f6310400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f63081461f0 con 0x7f6308145e10 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63080d0cc0 0x7f63080adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63080d0cc0 0x7f63080adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40680/0 (socket says 172.21.15.59:40680) 2024-04-23T14:41:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 -- 172.21.15.59:0/252507074 learned_addr learned my addr 172.21.15.59:0/252507074 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630da00640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308071d70 0x7f63080a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630e400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6308145e10 0x7f63080ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 -- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6308145e10 msgr2=0x7f63080ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6308145e10 0x7f63080ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 -- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308071d70 msgr2=0x7f63080a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308071d70 0x7f63080a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 -- 172.21.15.59:0/252507074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63080a9b60 con 0x7f63080d0cc0 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630e400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6308145e10 0x7f63080ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630da00640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308071d70 0x7f63080a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.610+0000 7f630d000640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63080d0cc0 0x7f63080adf40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b4d0 tx=0x7f62f400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.611+0000 7f6305600640 1 -- 172.21.15.59:0/252507074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4014070 con 0x7f63080d0cc0 2024-04-23T14:41:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.611+0000 7f6305600640 1 -- 172.21.15.59:0/252507074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f4010870 con 0x7f63080d0cc0 2024-04-23T14:41:50.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.611+0000 7f6305600640 1 -- 172.21.15.59:0/252507074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4010c60 con 0x7f63080d0cc0 2024-04-23T14:41:50.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.611+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63080a9e40 con 0x7f63080d0cc0 2024-04-23T14:41:50.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.611+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63080aa400 con 0x7f63080d0cc0 2024-04-23T14:41:50.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.612+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6308068a80 con 0x7f63080d0cc0 2024-04-23T14:41:50.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.613+0000 7f6305600640 1 -- 172.21.15.59:0/252507074 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62f4017070 con 0x7f63080d0cc0 2024-04-23T14:41:50.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.614+0000 7f6305600640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62e806c8f0 0x7f62e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:50.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.614+0000 7f6305600640 1 -- 172.21.15.59:0/252507074 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f62f408b9f0 con 0x7f63080d0cc0 2024-04-23T14:41:50.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.614+0000 7f630da00640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62e806c8f0 0x7f62e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:50.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.615+0000 7f630da00640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62e806c8f0 0x7f62e806edb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f62fc034000 tx=0x7f62fc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:50.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.618+0000 7f6305600640 1 -- 172.21.15.59:0/252507074 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62f4059d80 con 0x7f63080d0cc0 2024-04-23T14:41:50.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.825+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f63080d1190 con 0x7f63080d0cc0 2024-04-23T14:41:50.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.826+0000 7f6305600640 1 -- 172.21.15.59:0/252507074 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f62f4059680 con 0x7f63080d0cc0 2024-04-23T14:41:50.827 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.828+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62e806c8f0 msgr2=0x7f62e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.828+0000 7f6310400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62e806c8f0 0x7f62e806edb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f62fc034000 tx=0x7f62fc02e000 comp rx=0 tx=0).stop 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.828+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63080d0cc0 msgr2=0x7f63080adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.828+0000 7f6310400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63080d0cc0 0x7f63080adf40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b4d0 tx=0x7f62f400b990 comp rx=0 tx=0).stop 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 shutdown_connections 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f62e806c8f0 0x7f62e806edb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6308145e10 0x7f63080ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63080d0cc0 0x7f63080adf40 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 --2- 172.21.15.59:0/252507074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6308071d70 0x7f63080a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:50.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 >> 172.21.15.59:0/252507074 conn(0x7f6308067d90 msgr2=0x7f630806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 shutdown_connections 2024-04-23T14:41:50.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:50.829+0000 7f6310400640 1 -- 172.21.15.59:0/252507074 wait complete. 2024-04-23T14:41:50.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:41:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.056+0000 7fa2b6e00640 1 Processor -- start 2024-04-23T14:41:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.056+0000 7fa2b6e00640 1 -- start start 2024-04-23T14:41:51.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.058+0000 7fa2b6e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0162c10 0x7fa2b00d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.058+0000 7fa2b6e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2b0072480 0x7fa2b006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.058+0000 7fa2b6e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b00d0fd0 con 0x7fa2b00b4670 2024-04-23T14:41:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.058+0000 7fa2b6e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b00b35d0 con 0x7fa2b0072480 2024-04-23T14:41:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.058+0000 7fa2b6e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b00b38b0 con 0x7fa2b0162c10 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.059+0000 7fa2b4e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2b0072480 0x7fa2b006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.059+0000 7fa2afe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0162c10 0x7fa2b00d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.059+0000 7fa2af400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa2b00b4670 0x7fa2b0071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:54026/0 (socket says 172.21.15.59:54026) 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.059+0000 7fa2af400640 1 -- 172.21.15.59:0/3878046395 learned_addr learned my addr 172.21.15.59:0/3878046395 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.060+0000 7fa2b4e00640 1 -- 172.21.15.59:0/3878046395 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0162c10 msgr2=0x7fa2b00d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.060+0000 7fa2b4e00640 1 --2- 172.21.15.59:0/3878046395 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0162c10 0x7fa2b00d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.060+0000 7fa2b4e00640 1 -- 172.21.15.59:0/3878046395 >> v1:172.21.15.76:6789/0 conn(0x7fa2b00b4670 legacy=0x7fa2b0071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.060+0000 7fa2b4e00640 1 -- 172.21.15.59:0/3878046395 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b00b3b90 con 0x7fa2b0072480 2024-04-23T14:41:51.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.061+0000 7fa2b4e00640 1 --2- 172.21.15.59:0/3878046395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b0072480 0x7fa2b006ddd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009d90 tx=0x7fa2a802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf55f6acec2ab78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2aea00640 1 -- 172.21.15.59:0/3878046395 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8007740 con 0x7fa2b0072480 2024-04-23T14:41:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2aea00640 1 -- 172.21.15.59:0/3878046395 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a80078e0 con 0x7fa2b0072480 2024-04-23T14:41:51.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2aea00640 1 -- 172.21.15.59:0/3878046395 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8007cb0 con 0x7fa2b0072480 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2b6e00640 1 -- 172.21.15.59:0/3878046395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b0072480 msgr2=0x7fa2b006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/3878046395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b0072480 0x7fa2b006ddd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009d90 tx=0x7fa2a802a660 comp rx=0 tx=0).stop 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2b6e00640 1 -- 172.21.15.59:0/3878046395 shutdown_connections 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/3878046395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b0072480 0x7fa2b006ddd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/3878046395 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0162c10 0x7fa2b00d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.062+0000 7fa2b6e00640 1 -- 172.21.15.59:0/3878046395 >> 172.21.15.59:0/3878046395 conn(0x7fa2b0067d90 msgr2=0x7fa2b006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.063+0000 7fa2b6e00640 1 -- 172.21.15.59:0/3878046395 shutdown_connections 2024-04-23T14:41:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.063+0000 7fa2b6e00640 1 -- 172.21.15.59:0/3878046395 wait complete. 2024-04-23T14:41:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.064+0000 7fa2b6e00640 1 Processor -- start 2024-04-23T14:41:51.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.064+0000 7fa2b6e00640 1 -- start start 2024-04-23T14:41:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.065+0000 7fa2b6e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0072480 0x7fa2b01f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.065+0000 7fa2b6e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b00b4670 0x7fa2b01fc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.065+0000 7fa2b6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2b0162c10 0x7fa2b01fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.065+0000 7fa2b6e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0162ff0 con 0x7fa2b0162c10 2024-04-23T14:41:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.065+0000 7fa2b6e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b00b4150 con 0x7fa2b00b4670 2024-04-23T14:41:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b6e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b0163170 con 0x7fa2b0072480 2024-04-23T14:41:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2b0162c10 0x7fa2b01fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2afe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0072480 0x7fa2b01f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2b0162c10 0x7fa2b01fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40688/0 (socket says 172.21.15.59:40688) 2024-04-23T14:41:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 -- 172.21.15.59:0/2487288747 learned_addr learned my addr 172.21.15.59:0/2487288747 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 -- 172.21.15.59:0/2487288747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0072480 msgr2=0x7fa2b01f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0072480 0x7fa2b01f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 -- 172.21.15.59:0/2487288747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b00b4670 msgr2=0x7fa2b01fc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b00b4670 0x7fa2b01fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.066+0000 7fa2b4e00640 1 -- 172.21.15.59:0/2487288747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b00a9790 con 0x7fa2b0162c10 2024-04-23T14:41:51.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.067+0000 7fa2b4e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2b0162c10 0x7fa2b01fc8d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009eb0 tx=0x7fa2a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.067+0000 7fa2a7e00640 1 -- 172.21.15.59:0/2487288747 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8040070 con 0x7fa2b0162c10 2024-04-23T14:41:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.067+0000 7fa2a7e00640 1 -- 172.21.15.59:0/2487288747 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a8031b40 con 0x7fa2b0162c10 2024-04-23T14:41:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.067+0000 7fa2a7e00640 1 -- 172.21.15.59:0/2487288747 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8045460 con 0x7fa2b0162c10 2024-04-23T14:41:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.067+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b00b1670 con 0x7fa2b0162c10 2024-04-23T14:41:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.067+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b01ed0e0 con 0x7fa2b0162c10 2024-04-23T14:41:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.068+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b0068300 con 0x7fa2b0162c10 2024-04-23T14:41:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.069+0000 7fa2a7e00640 1 -- 172.21.15.59:0/2487288747 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2a8039040 con 0x7fa2b0162c10 2024-04-23T14:41:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.071+0000 7fa2a7e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa29006c8f0 0x7fa29006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.071+0000 7fa2a7e00640 1 -- 172.21.15.59:0/2487288747 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fa2a80ad340 con 0x7fa2b0162c10 2024-04-23T14:41:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.071+0000 7fa2afe00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa29006c8f0 0x7fa29006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.072+0000 7fa2afe00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa29006c8f0 0x7fa29006edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa2b015ef10 tx=0x7fa2a0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:51.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.076+0000 7fa2a7e00640 1 -- 172.21.15.59:0/2487288747 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2b0068300 con 0x7fa2b0162c10 2024-04-23T14:41:51.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.281+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2b0068a80 con 0x7fa2b0162c10 2024-04-23T14:41:51.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.282+0000 7fa2a7e00640 1 -- 172.21.15.59:0/2487288747 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fa2b00a9790 con 0x7fa2b0162c10 2024-04-23T14:41:51.283 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:41:51.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.284+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa29006c8f0 msgr2=0x7fa29006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.284+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa29006c8f0 0x7fa29006edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa2b015ef10 tx=0x7fa2a0009190 comp rx=0 tx=0).stop 2024-04-23T14:41:51.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2b0162c10 msgr2=0x7fa2b01fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2b0162c10 0x7fa2b01fc8d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009eb0 tx=0x7fa2a8004a40 comp rx=0 tx=0).stop 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 shutdown_connections 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa29006c8f0 0x7fa29006edb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2b0162c10 0x7fa2b01fc8d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b00b4670 0x7fa2b01fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 --2- 172.21.15.59:0/2487288747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa2b0072480 0x7fa2b01f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 >> 172.21.15.59:0/2487288747 conn(0x7fa2b0067d90 msgr2=0x7fa2b006ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 shutdown_connections 2024-04-23T14:41:51.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.285+0000 7fa2b6e00640 1 -- 172.21.15.59:0/2487288747 wait complete. 2024-04-23T14:41:51.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:41:51.345 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:41:51.357 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:41:51.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.498+0000 7fd28bc00640 1 Processor -- start 2024-04-23T14:41:51.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.498+0000 7fd28bc00640 1 -- start start 2024-04-23T14:41:51.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.500+0000 7fd28bc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd28415e9c0 0x7fd2840d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.500+0000 7fd28bc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2840d2d70 0x7fd2840d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.500+0000 7fd28bc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2840cf3c0 con 0x7fd2840d10f0 2024-04-23T14:41:51.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.500+0000 7fd28bc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd2840d3710 con 0x7fd28415e9c0 2024-04-23T14:41:51.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.500+0000 7fd28bc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2840cf640 con 0x7fd2840d2d70 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.501+0000 7fd283e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2840d2d70 0x7fd2840d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.501+0000 7fd289200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd28415e9c0 0x7fd2840d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.501+0000 7fd283e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2840d2d70 0x7fd2840d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:39032/0 (socket says 172.21.15.59:39032) 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.501+0000 7fd283e00640 1 -- 172.21.15.59:0/1504850347 learned_addr learned my addr 172.21.15.59:0/1504850347 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.502+0000 7fd289200640 1 -- 172.21.15.59:0/1504850347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2840d2d70 msgr2=0x7fd2840d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.502+0000 7fd289200640 1 --2- 172.21.15.59:0/1504850347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2840d2d70 0x7fd2840d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.502+0000 7fd289200640 1 -- 172.21.15.59:0/1504850347 >> v1:172.21.15.76:6789/0 conn(0x7fd2840d10f0 legacy=0x7fd2840d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.502+0000 7fd289200640 1 -- 172.21.15.59:0/1504850347 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2840cf900 con 0x7fd28415e9c0 2024-04-23T14:41:51.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.503+0000 7fd289200640 1 --2- 172.21.15.59:0/1504850347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd28415e9c0 0x7fd2840d03e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd274009de0 tx=0x7fd27402a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9fd389ed72f8d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:51.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.504+0000 7fd283400640 1 -- 172.21.15.59:0/1504850347 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274007740 con 0x7fd28415e9c0 2024-04-23T14:41:51.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.504+0000 7fd283400640 1 -- 172.21.15.59:0/1504850347 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2740078e0 con 0x7fd28415e9c0 2024-04-23T14:41:51.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.504+0000 7fd283400640 1 -- 172.21.15.59:0/1504850347 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274007cb0 con 0x7fd28415e9c0 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.504+0000 7fd28bc00640 1 -- 172.21.15.59:0/1504850347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd28415e9c0 msgr2=0x7fd2840d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.504+0000 7fd28bc00640 1 --2- 172.21.15.59:0/1504850347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd28415e9c0 0x7fd2840d03e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd274009de0 tx=0x7fd27402a620 comp rx=0 tx=0).stop 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.505+0000 7fd28bc00640 1 -- 172.21.15.59:0/1504850347 shutdown_connections 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.505+0000 7fd28bc00640 1 --2- 172.21.15.59:0/1504850347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2840d2d70 0x7fd2840d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.505+0000 7fd28bc00640 1 --2- 172.21.15.59:0/1504850347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd28415e9c0 0x7fd2840d03e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.505+0000 7fd28bc00640 1 -- 172.21.15.59:0/1504850347 >> 172.21.15.59:0/1504850347 conn(0x7fd28408bf90 msgr2=0x7fd2840d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.505+0000 7fd28bc00640 1 -- 172.21.15.59:0/1504850347 shutdown_connections 2024-04-23T14:41:51.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.505+0000 7fd28bc00640 1 -- 172.21.15.59:0/1504850347 wait complete. 2024-04-23T14:41:51.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.507+0000 7fd28bc00640 1 Processor -- start 2024-04-23T14:41:51.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.507+0000 7fd28bc00640 1 -- start start 2024-04-23T14:41:51.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd28bc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2840d10f0 0x7fd2840c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd28bc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2840d2d70 0x7fd28415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd28bc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd28415e9c0 0x7fd2840ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd28bc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd28406cb90 con 0x7fd2840d10f0 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd28bc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd28415eda0 con 0x7fd2840d2d70 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd28bc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd28406cd10 con 0x7fd28415e9c0 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd289200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2840d10f0 0x7fd2840c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd289200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2840d10f0 0x7fd2840c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40698/0 (socket says 172.21.15.59:40698) 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd289200640 1 -- 172.21.15.59:0/4222302298 learned_addr learned my addr 172.21.15.59:0/4222302298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.508+0000 7fd283e00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2840d2d70 0x7fd28415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd289c00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd28415e9c0 0x7fd2840ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd289200640 1 -- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd28415e9c0 msgr2=0x7fd2840ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd289200640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd28415e9c0 0x7fd2840ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd289200640 1 -- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2840d2d70 msgr2=0x7fd28415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd289200640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2840d2d70 0x7fd28415a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd289200640 1 -- 172.21.15.59:0/4222302298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2840cd6f0 con 0x7fd2840d10f0 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd283e00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2840d2d70 0x7fd28415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd289200640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2840d10f0 0x7fd2840c9690 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fd274009f00 tx=0x7fd274004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd280c00640 1 -- 172.21.15.59:0/4222302298 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27403f070 con 0x7fd2840d10f0 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd280c00640 1 -- 172.21.15.59:0/4222302298 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274031bc0 con 0x7fd2840d10f0 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2840cd9d0 con 0x7fd2840d10f0 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.509+0000 7fd280c00640 1 -- 172.21.15.59:0/4222302298 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274044460 con 0x7fd2840d10f0 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.510+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2840cdf90 con 0x7fd2840d10f0 2024-04-23T14:41:51.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.511+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd284068a80 con 0x7fd2840d10f0 2024-04-23T14:41:51.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.512+0000 7fd280c00640 1 -- 172.21.15.59:0/4222302298 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd274044600 con 0x7fd2840d10f0 2024-04-23T14:41:51.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.513+0000 7fd280c00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd26006c8f0 0x7fd26006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:51.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.513+0000 7fd280c00640 1 -- 172.21.15.59:0/4222302298 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fd2740ad2c0 con 0x7fd2840d10f0 2024-04-23T14:41:51.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.513+0000 7fd283e00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd26006c8f0 0x7fd26006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:51.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.514+0000 7fd283e00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd26006c8f0 0x7fd26006edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd2840ba6d0 tx=0x7fd26c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:51.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.518+0000 7fd280c00640 1 -- 172.21.15.59:0/4222302298 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd27407b650 con 0x7fd2840d10f0 2024-04-23T14:41:51.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.732+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd2840d2630 con 0x7fd2840d10f0 2024-04-23T14:41:51.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.733+0000 7fd280c00640 1 -- 172.21.15.59:0/4222302298 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7fd27407af50 con 0x7fd2840d10f0 2024-04-23T14:41:51.734 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:41:51.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd26006c8f0 msgr2=0x7fd26006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd26006c8f0 0x7fd26006edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd2840ba6d0 tx=0x7fd26c008040 comp rx=0 tx=0).stop 2024-04-23T14:41:51.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2840d10f0 msgr2=0x7fd2840c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2840d10f0 0x7fd2840c9690 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fd274009f00 tx=0x7fd274004a40 comp rx=0 tx=0).stop 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 shutdown_connections 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd26006c8f0 0x7fd26006edb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd28415e9c0 0x7fd2840ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2840d2d70 0x7fd28415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 --2- 172.21.15.59:0/4222302298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2840d10f0 0x7fd2840c9690 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 >> 172.21.15.59:0/4222302298 conn(0x7fd28408bf90 msgr2=0x7fd2840b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 shutdown_connections 2024-04-23T14:41:51.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:51.736+0000 7fd28bc00640 1 -- 172.21.15.59:0/4222302298 wait complete. 2024-04-23T14:41:51.757 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:41:51.757 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:41:51.810 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:41:52.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.023+0000 7fc90be00640 1 Processor -- start 2024-04-23T14:41:52.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.023+0000 7fc90be00640 1 -- start start 2024-04-23T14:41:52.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.024+0000 7fc90be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc90c162df0 0x7fc90c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.024+0000 7fc90be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c0cfb90 0x7fc90c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.024+0000 7fc90be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c06ed40 con 0x7fc90c0b3f60 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.024+0000 7fc90be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c06eaf0 con 0x7fc90c162df0 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.024+0000 7fc90be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c0b3590 con 0x7fc90c0cfb90 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.024+0000 7fc90aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc90c162df0 0x7fc90c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.025+0000 7fc90aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc90c162df0 0x7fc90c06f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40700/0 (socket says 172.21.15.59:40700) 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.025+0000 7fc90aa00640 1 -- 172.21.15.59:0/2253684452 learned_addr learned my addr 172.21.15.59:0/2253684452 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.025+0000 7fc90aa00640 1 -- 172.21.15.59:0/2253684452 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c0cfb90 msgr2=0x7fc90c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.025+0000 7fc90aa00640 1 --2- 172.21.15.59:0/2253684452 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c0cfb90 0x7fc90c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.025+0000 7fc90aa00640 1 -- 172.21.15.59:0/2253684452 >> v1:172.21.15.76:6789/0 conn(0x7fc90c0b3f60 legacy=0x7fc90c0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.025+0000 7fc90aa00640 1 -- 172.21.15.59:0/2253684452 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0b3220 con 0x7fc90c162df0 2024-04-23T14:41:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.026+0000 7fc90aa00640 1 --2- 172.21.15.59:0/2253684452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c162df0 0x7fc90c06f680 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009870 tx=0x7fc8fc02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2997309063c962c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.026+0000 7fc90a000640 1 -- 172.21.15.59:0/2253684452 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc037070 con 0x7fc90c162df0 2024-04-23T14:41:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.026+0000 7fc90a000640 1 -- 172.21.15.59:0/2253684452 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc032440 con 0x7fc90c162df0 2024-04-23T14:41:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.026+0000 7fc90a000640 1 -- 172.21.15.59:0/2253684452 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc032810 con 0x7fc90c162df0 2024-04-23T14:41:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 -- 172.21.15.59:0/2253684452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c162df0 msgr2=0x7fc90c06f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 --2- 172.21.15.59:0/2253684452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c162df0 0x7fc90c06f680 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009870 tx=0x7fc8fc02a200 comp rx=0 tx=0).stop 2024-04-23T14:41:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 -- 172.21.15.59:0/2253684452 shutdown_connections 2024-04-23T14:41:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 --2- 172.21.15.59:0/2253684452 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c0cfb90 0x7fc90c0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 --2- 172.21.15.59:0/2253684452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c162df0 0x7fc90c06f680 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 -- 172.21.15.59:0/2253684452 >> 172.21.15.59:0/2253684452 conn(0x7fc90c0b21a0 msgr2=0x7fc90c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 -- 172.21.15.59:0/2253684452 shutdown_connections 2024-04-23T14:41:52.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.027+0000 7fc90be00640 1 -- 172.21.15.59:0/2253684452 wait complete. 2024-04-23T14:41:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.029+0000 7fc90be00640 1 Processor -- start 2024-04-23T14:41:52.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.029+0000 7fc90be00640 1 -- start start 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc90be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc90c0b3f60 0x7fc90c1fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc90be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c0cfb90 0x7fc90c0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc90be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c162df0 0x7fc90c0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc90be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0c3a30 con 0x7fc90c0cfb90 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc90be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0c38b0 con 0x7fc90c0b3f60 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc90be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c0ce200 con 0x7fc90c162df0 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc903e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c0cfb90 0x7fc90c0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc903e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c0cfb90 0x7fc90c0c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40710/0 (socket says 172.21.15.59:40710) 2024-04-23T14:41:52.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.030+0000 7fc903e00640 1 -- 172.21.15.59:0/68842247 learned_addr learned my addr 172.21.15.59:0/68842247 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc90aa00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc90c0b3f60 0x7fc90c1fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc90b400640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c162df0 0x7fc90c0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc903e00640 1 -- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c162df0 msgr2=0x7fc90c0c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc903e00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c162df0 0x7fc90c0c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc903e00640 1 -- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc90c0b3f60 msgr2=0x7fc90c1fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc903e00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc90c0b3f60 0x7fc90c1fbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc903e00640 1 -- 172.21.15.59:0/68842247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c1fc040 con 0x7fc90c0cfb90 2024-04-23T14:41:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.031+0000 7fc903e00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c0cfb90 0x7fc90c0c6200 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400b4a0 tx=0x7fc8f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.032+0000 7fc902a00640 1 -- 172.21.15.59:0/68842247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4014070 con 0x7fc90c0cfb90 2024-04-23T14:41:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.032+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c072d70 con 0x7fc90c0cfb90 2024-04-23T14:41:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.032+0000 7fc902a00640 1 -- 172.21.15.59:0/68842247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f4010610 con 0x7fc90c0cfb90 2024-04-23T14:41:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.032+0000 7fc902a00640 1 -- 172.21.15.59:0/68842247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4010a00 con 0x7fc90c0cfb90 2024-04-23T14:41:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.032+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c073330 con 0x7fc90c0cfb90 2024-04-23T14:41:52.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.033+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c04fd90 con 0x7fc90c0cfb90 2024-04-23T14:41:52.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.034+0000 7fc902a00640 1 -- 172.21.15.59:0/68842247 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8f4017070 con 0x7fc90c0cfb90 2024-04-23T14:41:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.035+0000 7fc902a00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8e806c8f0 0x7fc8e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.035+0000 7fc902a00640 1 -- 172.21.15.59:0/68842247 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fc8f408b560 con 0x7fc90c0cfb90 2024-04-23T14:41:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.036+0000 7fc90aa00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8e806c8f0 0x7fc8e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.036+0000 7fc90aa00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8e806c8f0 0x7fc8e806edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.040+0000 7fc902a00640 1 -- 172.21.15.59:0/68842247 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8f4059a20 con 0x7fc90c0cfb90 2024-04-23T14:41:52.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.245+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc90c06f690 con 0x7fc90c0cfb90 2024-04-23T14:41:52.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.247+0000 7fc902a00640 1 -- 172.21.15.59:0/68842247 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16548 (secure 0 0 0) 0x7fc8f4059320 con 0x7fc90c0cfb90 2024-04-23T14:41:52.249 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:41:52.249 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":99,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:41:31.158736+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:41:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.249+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8e806c8f0 msgr2=0x7fc8e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.249+0000 7fc90be00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8e806c8f0 0x7fc8e806edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc02e000 comp rx=0 tx=0).stop 2024-04-23T14:41:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c0cfb90 msgr2=0x7fc90c0c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c0cfb90 0x7fc90c0c6200 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400b4a0 tx=0x7fc8f400b960 comp rx=0 tx=0).stop 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 shutdown_connections 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc8e806c8f0 0x7fc8e806edb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc90c162df0 0x7fc90c0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc90c0cfb90 0x7fc90c0c6200 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 --2- 172.21.15.59:0/68842247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc90c0b3f60 0x7fc90c1fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 >> 172.21.15.59:0/68842247 conn(0x7fc90c0b21a0 msgr2=0x7fc90c0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 shutdown_connections 2024-04-23T14:41:52.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.250+0000 7fc90be00640 1 -- 172.21.15.59:0/68842247 wait complete. 2024-04-23T14:41:52.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:41:27.465708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:41:28.586680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:41:52.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:41:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.479+0000 7f32e7200640 1 Processor -- start 2024-04-23T14:41:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.479+0000 7f32e7200640 1 -- start start 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.480+0000 7f32e7200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32e815ed20 0x7f32e80d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.480+0000 7f32e7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b3600 0x7f32e80d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e7200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e80b5da0 con 0x7f32e80b3f10 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e7200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f32e815f100 con 0x7f32e815ed20 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32e80d0a20 con 0x7f32e80b3600 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32e815ed20 0x7f32e80d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32e815ed20 0x7f32e80d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40714/0 (socket says 172.21.15.59:40714) 2024-04-23T14:41:52.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5e00640 1 -- 172.21.15.59:0/3913446410 learned_addr learned my addr 172.21.15.59:0/3913446410 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e6800640 1 --1- 172.21.15.59:0/3913446410 >> v1:172.21.15.59:6789/0 conn(0x7f32e80b3f10 0x7f32e80b43a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40370/0 (socket says 172.21.15.59:40370) 2024-04-23T14:41:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5400640 1 --2- 172.21.15.59:0/3913446410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b3600 0x7f32e80d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5e00640 1 -- 172.21.15.59:0/3913446410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b3600 msgr2=0x7f32e80d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:52.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5e00640 1 --2- 172.21.15.59:0/3913446410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b3600 0x7f32e80d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5e00640 1 -- 172.21.15.59:0/3913446410 >> v1:172.21.15.59:6789/0 conn(0x7f32e80b3f10 legacy=0x7f32e80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5e00640 1 -- 172.21.15.59:0/3913446410 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e80d2ed0 con 0x7f32e815ed20 2024-04-23T14:41:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.481+0000 7f32e5400640 1 --2- 172.21.15.59:0/3913446410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b3600 0x7f32e80d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:52.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.482+0000 7f32e5e00640 1 --2- 172.21.15.59:0/3913446410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815ed20 0x7f32e80d0ea0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009870 tx=0x7f32d802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551b1c75f967fa01 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.483+0000 7f32e4a00640 1 -- 172.21.15.59:0/3913446410 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8037070 con 0x7f32e815ed20 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.483+0000 7f32e4a00640 1 -- 172.21.15.59:0/3913446410 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d8032440 con 0x7f32e815ed20 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.483+0000 7f32e4a00640 1 -- 172.21.15.59:0/3913446410 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8032810 con 0x7f32e815ed20 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.483+0000 7f32e7200640 1 -- 172.21.15.59:0/3913446410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815ed20 msgr2=0x7f32e80d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.483+0000 7f32e7200640 1 --2- 172.21.15.59:0/3913446410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815ed20 0x7f32e80d0ea0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009870 tx=0x7f32d802a140 comp rx=0 tx=0).stop 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.484+0000 7f32e7200640 1 -- 172.21.15.59:0/3913446410 shutdown_connections 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.484+0000 7f32e7200640 1 --2- 172.21.15.59:0/3913446410 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b3600 0x7f32e80d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.484+0000 7f32e7200640 1 --2- 172.21.15.59:0/3913446410 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815ed20 0x7f32e80d0ea0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.484+0000 7f32e7200640 1 -- 172.21.15.59:0/3913446410 >> 172.21.15.59:0/3913446410 conn(0x7f32e808bf90 msgr2=0x7f32e80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.484+0000 7f32e7200640 1 -- 172.21.15.59:0/3913446410 shutdown_connections 2024-04-23T14:41:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.484+0000 7f32e7200640 1 -- 172.21.15.59:0/3913446410 wait complete. 2024-04-23T14:41:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.486+0000 7f32e7200640 1 Processor -- start 2024-04-23T14:41:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.486+0000 7f32e7200640 1 -- start start 2024-04-23T14:41:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e7200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80b3600 0x7f32e80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e7200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e80b3f10 0x7f32e815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e7200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b6bc0 0x7f32e80b9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e7200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32e80d0290 con 0x7f32e80b3f10 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e7200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32e80d3490 con 0x7f32e80b3600 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e7200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f32e80d0410 con 0x7f32e80b6bc0 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e80b3f10 0x7f32e815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e80b3f10 0x7f32e815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40728/0 (socket says 172.21.15.59:40728) 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.487+0000 7f32e5400640 1 -- 172.21.15.59:0/449656360 learned_addr learned my addr 172.21.15.59:0/449656360 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5e00640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80b3600 0x7f32e80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5400640 1 -- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b6bc0 msgr2=0x7f32e80b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e6800640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b6bc0 0x7f32e80b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5400640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b6bc0 0x7f32e80b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5400640 1 -- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80b3600 msgr2=0x7f32e80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5400640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80b3600 0x7f32e80b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5400640 1 -- 172.21.15.59:0/449656360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e815b070 con 0x7f32e80b3f10 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5e00640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80b3600 0x7f32e80b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e5400640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e80b3f10 0x7f32e815ab30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f32d000cf60 tx=0x7f32d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32de000640 1 -- 172.21.15.59:0/449656360 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d001a070 con 0x7f32e80b3f10 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32de000640 1 -- 172.21.15.59:0/449656360 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d000d840 con 0x7f32e80b3f10 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32de000640 1 -- 172.21.15.59:0/449656360 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d000dc10 con 0x7f32e80b3f10 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.488+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e80b73a0 con 0x7f32e80b3f10 2024-04-23T14:41:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.489+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e80b9040 con 0x7f32e80b3f10 2024-04-23T14:41:52.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.490+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e804fd90 con 0x7f32e80b3f10 2024-04-23T14:41:52.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.491+0000 7f32de000640 1 -- 172.21.15.59:0/449656360 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32d001d070 con 0x7f32e80b3f10 2024-04-23T14:41:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.492+0000 7f32de000640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c406c940 0x7f32c406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.492+0000 7f32de000640 1 -- 172.21.15.59:0/449656360 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f32d008c3a0 con 0x7f32e80b3f10 2024-04-23T14:41:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.492+0000 7f32e5e00640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c406c940 0x7f32c406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.493+0000 7f32e5e00640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c406c940 0x7f32c406ee00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f32d8034000 tx=0x7f32d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.497+0000 7f32de000640 1 -- 172.21.15.59:0/449656360 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32e815b070 con 0x7f32e80b3f10 2024-04-23T14:41:52.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.701+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32e80b3090 con 0x7f32e80b3f10 2024-04-23T14:41:52.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.702+0000 7f32de000640 1 -- 172.21.15.59:0/449656360 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f32d005a160 con 0x7f32e80b3f10 2024-04-23T14:41:52.703 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:41:52.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c406c940 msgr2=0x7f32c406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c406c940 0x7f32c406ee00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f32d8034000 tx=0x7f32d802e000 comp rx=0 tx=0).stop 2024-04-23T14:41:52.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e80b3f10 msgr2=0x7f32e815ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e80b3f10 0x7f32e815ab30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f32d000cf60 tx=0x7f32d000cf90 comp rx=0 tx=0).stop 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 shutdown_connections 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f32c406c940 0x7f32c406ee00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e80b6bc0 0x7f32e80b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e80b3f10 0x7f32e815ab30 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 --2- 172.21.15.59:0/449656360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80b3600 0x7f32e80b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 >> 172.21.15.59:0/449656360 conn(0x7f32e808bf90 msgr2=0x7f32e80d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 shutdown_connections 2024-04-23T14:41:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.705+0000 7f32e7200640 1 -- 172.21.15.59:0/449656360 wait complete. 2024-04-23T14:41:52.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:41:52.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.936+0000 7fd8b3200640 1 Processor -- start 2024-04-23T14:41:52.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.937+0000 7fd8b3200640 1 -- start start 2024-04-23T14:41:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b3200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8b415ed20 0x7fd8b40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8b40d13e0 0x7fd8b40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b3200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b40b5da0 con 0x7fd8b40b3600 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b415f100 con 0x7fd8b40d13e0 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b3200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b40d07d0 con 0x7fd8b415ed20 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8b40d13e0 0x7fd8b40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8b40d13e0 0x7fd8b40d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40730/0 (socket says 172.21.15.59:40730) 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b2800640 1 -- 172.21.15.59:0/4127661091 learned_addr learned my addr 172.21.15.59:0/4127661091 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.938+0000 7fd8b1400640 1 --1- 172.21.15.59:0/4127661091 >> v1:172.21.15.59:6789/0 conn(0x7fd8b40b3600 0x7fd8b40b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40386/0 (socket says 172.21.15.59:40386) 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.939+0000 7fd8b1e00640 1 --2- 172.21.15.59:0/4127661091 >> v2:172.21.15.76:3300/0 conn(0x7fd8b415ed20 0x7fd8b40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.939+0000 7fd8b2800640 1 -- 172.21.15.59:0/4127661091 >> v2:172.21.15.76:3300/0 conn(0x7fd8b415ed20 msgr2=0x7fd8b40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.939+0000 7fd8b2800640 1 --2- 172.21.15.59:0/4127661091 >> v2:172.21.15.76:3300/0 conn(0x7fd8b415ed20 0x7fd8b40d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.939+0000 7fd8b2800640 1 -- 172.21.15.59:0/4127661091 >> v1:172.21.15.59:6789/0 conn(0x7fd8b40b3600 legacy=0x7fd8b40b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.939+0000 7fd8b2800640 1 -- 172.21.15.59:0/4127661091 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40d0a20 con 0x7fd8b40d13e0 2024-04-23T14:41:52.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.940+0000 7fd8b2800640 1 --2- 172.21.15.59:0/4127661091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40d0290 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd8b408e2a0 tx=0x7fd8ac02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16efcdd357a512d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.940+0000 7fd8b0a00640 1 -- 172.21.15.59:0/4127661091 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac037070 con 0x7fd8b40d13e0 2024-04-23T14:41:52.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.940+0000 7fd8b0a00640 1 -- 172.21.15.59:0/4127661091 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ac032440 con 0x7fd8b40d13e0 2024-04-23T14:41:52.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.940+0000 7fd8b0a00640 1 -- 172.21.15.59:0/4127661091 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac032810 con 0x7fd8b40d13e0 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 -- 172.21.15.59:0/4127661091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 msgr2=0x7fd8b40d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 --2- 172.21.15.59:0/4127661091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40d0290 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd8b408e2a0 tx=0x7fd8ac02a160 comp rx=0 tx=0).stop 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 -- 172.21.15.59:0/4127661091 shutdown_connections 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 --2- 172.21.15.59:0/4127661091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40d0290 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 --2- 172.21.15.59:0/4127661091 >> v2:172.21.15.76:3300/0 conn(0x7fd8b415ed20 0x7fd8b40d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 -- 172.21.15.59:0/4127661091 >> 172.21.15.59:0/4127661091 conn(0x7fd8b408bf90 msgr2=0x7fd8b40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 -- 172.21.15.59:0/4127661091 shutdown_connections 2024-04-23T14:41:52.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.941+0000 7fd8b3200640 1 -- 172.21.15.59:0/4127661091 wait complete. 2024-04-23T14:41:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.943+0000 7fd8b3200640 1 Processor -- start 2024-04-23T14:41:52.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.943+0000 7fd8b3200640 1 -- start start 2024-04-23T14:41:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8b40b3600 0x7fd8b415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8b415ed20 0x7fd8b40a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b40d3490 con 0x7fd8b40d13e0 2024-04-23T14:41:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b40d31b0 con 0x7fd8b415ed20 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b40b4620 con 0x7fd8b40b3600 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40738/0 (socket says 172.21.15.59:40738) 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.944+0000 7fd8b1400640 1 -- 172.21.15.59:0/2498881815 learned_addr learned my addr 172.21.15.59:0/2498881815 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b1e00640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8b40b3600 0x7fd8b415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b2800640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8b415ed20 0x7fd8b40a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b1400640 1 -- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8b40b3600 msgr2=0x7fd8b415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b1400640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8b40b3600 0x7fd8b415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b1400640 1 -- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8b415ed20 msgr2=0x7fd8b40a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b1400640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8b415ed20 0x7fd8b40a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b1400640 1 -- 172.21.15.59:0/2498881815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b41f4cd0 con 0x7fd8b40d13e0 2024-04-23T14:41:52.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8b1400640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40cd0e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c400 tx=0x7fd8a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.945+0000 7fd8aa000640 1 -- 172.21.15.59:0/2498881815 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00074e0 con 0x7fd8b40d13e0 2024-04-23T14:41:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.946+0000 7fd8aa000640 1 -- 172.21.15.59:0/2498881815 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a0007680 con 0x7fd8b40d13e0 2024-04-23T14:41:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.946+0000 7fd8aa000640 1 -- 172.21.15.59:0/2498881815 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0004a00 con 0x7fd8b40d13e0 2024-04-23T14:41:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.946+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b41f4fb0 con 0x7fd8b40d13e0 2024-04-23T14:41:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.946+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41f5570 con 0x7fd8b40d13e0 2024-04-23T14:41:52.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.947+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b404fd90 con 0x7fd8b40d13e0 2024-04-23T14:41:52.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.948+0000 7fd8aa000640 1 -- 172.21.15.59:0/2498881815 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8a001a020 con 0x7fd8b40d13e0 2024-04-23T14:41:52.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.949+0000 7fd8aa000640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd89406c8f0 0x7fd89406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:52.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.949+0000 7fd8aa000640 1 -- 172.21.15.59:0/2498881815 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fd8a008b3c0 con 0x7fd8b40d13e0 2024-04-23T14:41:52.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.949+0000 7fd8b1e00640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd89406c8f0 0x7fd89406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:52.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.950+0000 7fd8b1e00640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd89406c8f0 0x7fd89406edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40099c0 tx=0x7fd8a4003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:52.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:52.954+0000 7fd8aa000640 1 -- 172.21.15.59:0/2498881815 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8a00597e0 con 0x7fd8b40d13e0 2024-04-23T14:41:53.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.159+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd8b40b3dd0 con 0x7fd8b40d13e0 2024-04-23T14:41:53.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.160+0000 7fd8aa000640 1 -- 172.21.15.59:0/2498881815 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fd8a00590e0 con 0x7fd8b40d13e0 2024-04-23T14:41:53.162 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd89406c8f0 msgr2=0x7fd89406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd89406c8f0 0x7fd89406edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40099c0 tx=0x7fd8a4003990 comp rx=0 tx=0).stop 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 msgr2=0x7fd8b40cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40cd0e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c400 tx=0x7fd8a000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 shutdown_connections 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd89406c8f0 0x7fd89406edb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8b415ed20 0x7fd8b40a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8b40d13e0 0x7fd8b40cd0e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 --2- 172.21.15.59:0/2498881815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8b40b3600 0x7fd8b415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.163+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 >> 172.21.15.59:0/2498881815 conn(0x7fd8b408bf90 msgr2=0x7fd8b40d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.164+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 shutdown_connections 2024-04-23T14:41:53.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.164+0000 7fd8b3200640 1 -- 172.21.15.59:0/2498881815 wait complete. 2024-04-23T14:41:53.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:41:53.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.397+0000 7f9a6f200640 1 Processor -- start 2024-04-23T14:41:53.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.397+0000 7f9a6f200640 1 -- start start 2024-04-23T14:41:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.399+0000 7f9a6f200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9a701567e0 0x7f9a70156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.399+0000 7f9a6f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a7015d810 0x7f9a700d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.399+0000 7f9a6f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a700d0b40 0x7f9a70160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.399+0000 7f9a6f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9a700d0f60 con 0x7f9a7015d810 2024-04-23T14:41:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.399+0000 7f9a6f200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9a7015cab0 con 0x7f9a701567e0 2024-04-23T14:41:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.399+0000 7f9a6f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9a70160e40 con 0x7f9a700d0b40 2024-04-23T14:41:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.399+0000 7f9a65400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a7015d810 0x7f9a700d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a65400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a7015d810 0x7f9a700d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40740/0 (socket says 172.21.15.59:40740) 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a65400640 1 -- 172.21.15.59:0/1410632554 learned_addr learned my addr 172.21.15.59:0/1410632554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a6de00640 1 --2- 172.21.15.59:0/1410632554 >> v2:172.21.15.76:3300/0 conn(0x7f9a701567e0 0x7f9a70156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a6e800640 1 --2- 172.21.15.59:0/1410632554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a700d0b40 0x7f9a70160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a65400640 1 -- 172.21.15.59:0/1410632554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a700d0b40 msgr2=0x7f9a70160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a65400640 1 --2- 172.21.15.59:0/1410632554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a700d0b40 0x7f9a70160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a65400640 1 -- 172.21.15.59:0/1410632554 >> v2:172.21.15.76:3300/0 conn(0x7f9a701567e0 msgr2=0x7f9a70156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a65400640 1 --2- 172.21.15.59:0/1410632554 >> v2:172.21.15.76:3300/0 conn(0x7f9a701567e0 0x7f9a70156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a65400640 1 -- 172.21.15.59:0/1410632554 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7015cd30 con 0x7f9a7015d810 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a6de00640 1 --2- 172.21.15.59:0/1410632554 >> v2:172.21.15.76:3300/0 conn(0x7f9a701567e0 0x7f9a70156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:41:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.400+0000 7f9a6e800640 1 --2- 172.21.15.59:0/1410632554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a700d0b40 0x7f9a70160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:41:53.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.402+0000 7f9a65400640 1 --2- 172.21.15.59:0/1410632554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a7015d810 0x7f9a700d0600 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9a700b6360 tx=0x7f9a5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247c34b91ae29133 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:53.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.402+0000 7f9a6d400640 1 -- 172.21.15.59:0/1410632554 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58037070 con 0x7f9a7015d810 2024-04-23T14:41:53.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.402+0000 7f9a6d400640 1 -- 172.21.15.59:0/1410632554 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a58032440 con 0x7f9a7015d810 2024-04-23T14:41:53.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.402+0000 7f9a6d400640 1 -- 172.21.15.59:0/1410632554 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58032810 con 0x7f9a7015d810 2024-04-23T14:41:53.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 -- 172.21.15.59:0/1410632554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a7015d810 msgr2=0x7f9a700d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 --2- 172.21.15.59:0/1410632554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a7015d810 0x7f9a700d0600 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9a700b6360 tx=0x7f9a5802a1e0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 -- 172.21.15.59:0/1410632554 shutdown_connections 2024-04-23T14:41:53.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 --2- 172.21.15.59:0/1410632554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a700d0b40 0x7f9a70160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 --2- 172.21.15.59:0/1410632554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a7015d810 0x7f9a700d0600 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 --2- 172.21.15.59:0/1410632554 >> v2:172.21.15.76:3300/0 conn(0x7f9a701567e0 0x7f9a70156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 -- 172.21.15.59:0/1410632554 >> 172.21.15.59:0/1410632554 conn(0x7f9a7006bdf0 msgr2=0x7f9a7006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:53.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 -- 172.21.15.59:0/1410632554 shutdown_connections 2024-04-23T14:41:53.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.403+0000 7f9a6f200640 1 -- 172.21.15.59:0/1410632554 wait complete. 2024-04-23T14:41:53.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.405+0000 7f9a6f200640 1 Processor -- start 2024-04-23T14:41:53.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.405+0000 7f9a6f200640 1 -- start start 2024-04-23T14:41:53.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.406+0000 7f9a6f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a700d0b40 0x7f9a7009a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.406+0000 7f9a6f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a701567e0 0x7f9a701f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.406+0000 7f9a6f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a7015d810 0x7f9a7009afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.406+0000 7f9a6f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70160650 con 0x7f9a701567e0 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.406+0000 7f9a6f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7015d2f0 con 0x7f9a700d0b40 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.406+0000 7f9a6f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9a701607d0 con 0x7f9a7015d810 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a701567e0 0x7f9a701f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a701567e0 0x7f9a701f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40752/0 (socket says 172.21.15.59:40752) 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 -- 172.21.15.59:0/278942610 learned_addr learned my addr 172.21.15.59:0/278942610 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:53.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a6de00640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a700d0b40 0x7f9a7009a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a6e800640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a7015d810 0x7f9a7009afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 -- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a7015d810 msgr2=0x7f9a7009afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a7015d810 0x7f9a7009afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 -- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a700d0b40 msgr2=0x7f9a7009a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a700d0b40 0x7f9a7009a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a65400640 1 -- 172.21.15.59:0/278942610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a701f5080 con 0x7f9a701567e0 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.407+0000 7f9a6de00640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a700d0b40 0x7f9a7009a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:41:53.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.408+0000 7f9a65400640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a701567e0 0x7f9a701f3ef0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802a220 tx=0x7f9a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:53.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.408+0000 7f9a66a00640 1 -- 172.21.15.59:0/278942610 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58041020 con 0x7f9a701567e0 2024-04-23T14:41:53.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.408+0000 7f9a66a00640 1 -- 172.21.15.59:0/278942610 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a580306c0 con 0x7f9a701567e0 2024-04-23T14:41:53.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.408+0000 7f9a66a00640 1 -- 172.21.15.59:0/278942610 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58037040 con 0x7f9a701567e0 2024-04-23T14:41:53.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.408+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a701f5360 con 0x7f9a701567e0 2024-04-23T14:41:53.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.408+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701f5920 con 0x7f9a701567e0 2024-04-23T14:41:53.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.409+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7004fd90 con 0x7f9a701567e0 2024-04-23T14:41:53.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.410+0000 7f9a66a00640 1 -- 172.21.15.59:0/278942610 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a58044020 con 0x7f9a701567e0 2024-04-23T14:41:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.411+0000 7f9a66a00640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a4c06c8f0 0x7f9a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.411+0000 7f9a66a00640 1 -- 172.21.15.59:0/278942610 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f9a580ac2d0 con 0x7f9a701567e0 2024-04-23T14:41:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.412+0000 7f9a6de00640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a4c06c8f0 0x7f9a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.412+0000 7f9a6de00640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a4c06c8f0 0x7f9a4c06edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9a60006fd0 tx=0x7f9a60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:53.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.417+0000 7f9a66a00640 1 -- 172.21.15.59:0/278942610 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a5804b030 con 0x7f9a701567e0 2024-04-23T14:41:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.620+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9a70160b80 con 0x7f9a701567e0 2024-04-23T14:41:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.621+0000 7f9a66a00640 1 -- 172.21.15.59:0/278942610 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f9a5807a790 con 0x7f9a701567e0 2024-04-23T14:41:53.623 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:41:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a4c06c8f0 msgr2=0x7f9a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a4c06c8f0 0x7f9a4c06edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9a60006fd0 tx=0x7f9a60008040 comp rx=0 tx=0).stop 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a701567e0 msgr2=0x7f9a701f3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a701567e0 0x7f9a701f3ef0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802a220 tx=0x7f9a580046c0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 shutdown_connections 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a4c06c8f0 0x7f9a4c06edb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a7015d810 0x7f9a7009afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a701567e0 0x7f9a701f3ef0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 --2- 172.21.15.59:0/278942610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a700d0b40 0x7f9a7009a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 >> 172.21.15.59:0/278942610 conn(0x7f9a7006bdf0 msgr2=0x7f9a7015eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 shutdown_connections 2024-04-23T14:41:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.624+0000 7f9a6f200640 1 -- 172.21.15.59:0/278942610 wait complete. 2024-04-23T14:41:53.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:41:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.856+0000 7f98fa600640 1 Processor -- start 2024-04-23T14:41:53.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.856+0000 7f98fa600640 1 -- start start 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.857+0000 7f98fa600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98f4161e60 0x7f98f40aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98fa600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4162de0 0x7f98f40b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98fa600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98f40afa80 0x7f98f40b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98fa600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f98f40aefd0 con 0x7f98f4161e60 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98fa600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f98f41632e0 con 0x7f98f40afa80 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98fa600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f98f40adcd0 con 0x7f98f4162de0 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98f3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98f4161e60 0x7f98f40aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98f3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98f4161e60 0x7f98f40aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40760/0 (socket says 172.21.15.59:40760) 2024-04-23T14:41:53.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98f3400640 1 -- 172.21.15.59:0/1665936259 learned_addr learned my addr 172.21.15.59:0/1665936259 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:53.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98f3e00640 1 --2- 172.21.15.59:0/1665936259 >> v2:172.21.15.76:3300/0 conn(0x7f98f40afa80 0x7f98f40b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98f2a00640 1 --2- 172.21.15.59:0/1665936259 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4162de0 0x7f98f40b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.858+0000 7f98f3400640 1 -- 172.21.15.59:0/1665936259 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4162de0 msgr2=0x7f98f40b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.859+0000 7f98f3400640 1 --2- 172.21.15.59:0/1665936259 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4162de0 0x7f98f40b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.859+0000 7f98f3400640 1 -- 172.21.15.59:0/1665936259 >> v2:172.21.15.76:3300/0 conn(0x7f98f40afa80 msgr2=0x7f98f40b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.859+0000 7f98f3400640 1 --2- 172.21.15.59:0/1665936259 >> v2:172.21.15.76:3300/0 conn(0x7f98f40afa80 0x7f98f40b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.859+0000 7f98f3400640 1 -- 172.21.15.59:0/1665936259 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f40adfb0 con 0x7f98f4161e60 2024-04-23T14:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.860+0000 7f98f3400640 1 --2- 172.21.15.59:0/1665936259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4161e60 0x7f98f40aea90 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f98f4075670 tx=0x7f98e402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1851d57de7e78b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.860+0000 7f98f2000640 1 -- 172.21.15.59:0/1665936259 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4037070 con 0x7f98f4161e60 2024-04-23T14:41:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.860+0000 7f98f2000640 1 -- 172.21.15.59:0/1665936259 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e4032440 con 0x7f98f4161e60 2024-04-23T14:41:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.860+0000 7f98f2000640 1 -- 172.21.15.59:0/1665936259 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4032810 con 0x7f98f4161e60 2024-04-23T14:41:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 -- 172.21.15.59:0/1665936259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4161e60 msgr2=0x7f98f40aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 --2- 172.21.15.59:0/1665936259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4161e60 0x7f98f40aea90 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f98f4075670 tx=0x7f98e402a140 comp rx=0 tx=0).stop 2024-04-23T14:41:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 -- 172.21.15.59:0/1665936259 shutdown_connections 2024-04-23T14:41:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 --2- 172.21.15.59:0/1665936259 >> v2:172.21.15.76:3300/0 conn(0x7f98f40afa80 0x7f98f40b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 --2- 172.21.15.59:0/1665936259 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4162de0 0x7f98f40b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 --2- 172.21.15.59:0/1665936259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4161e60 0x7f98f40aea90 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 -- 172.21.15.59:0/1665936259 >> 172.21.15.59:0/1665936259 conn(0x7f98f40b2130 msgr2=0x7f98f40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 -- 172.21.15.59:0/1665936259 shutdown_connections 2024-04-23T14:41:53.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.861+0000 7f98fa600640 1 -- 172.21.15.59:0/1665936259 wait complete. 2024-04-23T14:41:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.863+0000 7f98fa600640 1 Processor -- start 2024-04-23T14:41:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.863+0000 7f98fa600640 1 -- start start 2024-04-23T14:41:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98fa600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f40afa80 0x7f98f41f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98fa600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4161e60 0x7f98f41f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98fa600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4162de0 0x7f98f4147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98fa600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98f40af480 con 0x7f98f4162de0 2024-04-23T14:41:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98fa600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f98f40ae570 con 0x7f98f40afa80 2024-04-23T14:41:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98fa600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f98f40af600 con 0x7f98f4161e60 2024-04-23T14:41:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4162de0 0x7f98f4147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4162de0 0x7f98f4147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40766/0 (socket says 172.21.15.59:40766) 2024-04-23T14:41:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.864+0000 7f98f3e00640 1 -- 172.21.15.59:0/439056030 learned_addr learned my addr 172.21.15.59:0/439056030 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:41:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f3400640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f40afa80 0x7f98f41f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f2a00640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4161e60 0x7f98f41f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f3e00640 1 -- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4161e60 msgr2=0x7f98f41f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f3e00640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4161e60 0x7f98f41f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f3e00640 1 -- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f40afa80 msgr2=0x7f98f41f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f3e00640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f40afa80 0x7f98f41f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f3e00640 1 -- 172.21.15.59:0/439056030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f4148c80 con 0x7f98f4162de0 2024-04-23T14:41:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98f3e00640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4162de0 0x7f98f4147d50 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f98ec007e60 tx=0x7f98ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98eb400640 1 -- 172.21.15.59:0/439056030 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec014420 con 0x7f98f4162de0 2024-04-23T14:41:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.865+0000 7f98eb400640 1 -- 172.21.15.59:0/439056030 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec019070 con 0x7f98f4162de0 2024-04-23T14:41:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.866+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f41f4060 con 0x7f98f4162de0 2024-04-23T14:41:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.866+0000 7f98eb400640 1 -- 172.21.15.59:0/439056030 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec014820 con 0x7f98f4162de0 2024-04-23T14:41:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.866+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f41489a0 con 0x7f98f4162de0 2024-04-23T14:41:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.867+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f4068a80 con 0x7f98f4162de0 2024-04-23T14:41:53.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.868+0000 7f98eb400640 1 -- 172.21.15.59:0/439056030 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98ec0149c0 con 0x7f98f4162de0 2024-04-23T14:41:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.869+0000 7f98eb400640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f98d406c920 0x7f98d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.870+0000 7f98eb400640 1 -- 172.21.15.59:0/439056030 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f98ec08c390 con 0x7f98f4162de0 2024-04-23T14:41:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.870+0000 7f98f3400640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f98d406c920 0x7f98d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:53.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.871+0000 7f98f3400640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f98d406c920 0x7f98d406ede0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f98e4034000 tx=0x7f98e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:53.874+0000 7f98eb400640 1 -- 172.21.15.59:0/439056030 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98ec05a7b0 con 0x7f98f4162de0 2024-04-23T14:41:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.094+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f98f406cd70 con 0x7f98f4162de0 2024-04-23T14:41:54.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.095+0000 7f98eb400640 1 -- 172.21.15.59:0/439056030 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7f98ec05a0b0 con 0x7f98f4162de0 2024-04-23T14:41:54.096 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:41:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f98d406c920 msgr2=0x7f98d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f98d406c920 0x7f98d406ede0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f98e4034000 tx=0x7f98e402e000 comp rx=0 tx=0).stop 2024-04-23T14:41:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4162de0 msgr2=0x7f98f4147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4162de0 0x7f98f4147d50 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f98ec007e60 tx=0x7f98ec00e4b0 comp rx=0 tx=0).stop 2024-04-23T14:41:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 shutdown_connections 2024-04-23T14:41:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f98d406c920 0x7f98d406ede0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f4161e60 0x7f98f41f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f4162de0 0x7f98f4147d50 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 --2- 172.21.15.59:0/439056030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f40afa80 0x7f98f41f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 >> 172.21.15.59:0/439056030 conn(0x7f98f40b2130 msgr2=0x7f98f40d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 shutdown_connections 2024-04-23T14:41:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:41:54.098+0000 7f98fa600640 1 -- 172.21.15.59:0/439056030 wait complete. 2024-04-23T14:41:54.119 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:41:54.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:41:54.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:41:54.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[71171]: fuse finished with error 0 and tester_r 0 2024-04-23T14:41:54.205 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:42:00.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:42:00.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:42:00.307 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:42:00.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:42:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.586+0000 7fc972600640 1 Processor -- start 2024-04-23T14:42:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.586+0000 7fc972600640 1 -- start start 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.587+0000 7fc972600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c06f620 0x7fc96c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.587+0000 7fc972600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc96c0b3080 0x7fc96c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.587+0000 7fc972600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c15f430 con 0x7fc96c0cd050 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.587+0000 7fc972600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc96c0ced20 con 0x7fc96c0b3080 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.587+0000 7fc972600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c15fcd0 con 0x7fc96c06f620 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.588+0000 7fc971200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc96c0cd050 0x7fc96c0cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50386/0 (socket says 172.21.15.59:50386) 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.588+0000 7fc971200640 1 -- 172.21.15.59:0/4084908608 learned_addr learned my addr 172.21.15.59:0/4084908608 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.588+0000 7fc971c00640 1 --2- 172.21.15.59:0/4084908608 >> v2:172.21.15.59:3300/0 conn(0x7fc96c0b3080 0x7fc96c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.588+0000 7fc96be00640 1 --2- 172.21.15.59:0/4084908608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c06f620 0x7fc96c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.588+0000 7fc96b400640 1 -- 172.21.15.59:0/4084908608 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790546866 0 0) 0x7fc96c15f430 con 0x7fc96c0cd050 2024-04-23T14:42:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.588+0000 7fc96b400640 1 -- 172.21.15.59:0/4084908608 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc950003520 con 0x7fc96c0cd050 2024-04-23T14:42:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.589+0000 7fc971c00640 1 -- 172.21.15.59:0/4084908608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c06f620 msgr2=0x7fc96c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.589+0000 7fc971c00640 1 --2- 172.21.15.59:0/4084908608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c06f620 0x7fc96c15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.589+0000 7fc971c00640 1 -- 172.21.15.59:0/4084908608 >> v1:172.21.15.59:6789/0 conn(0x7fc96c0cd050 legacy=0x7fc96c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.589+0000 7fc971c00640 1 -- 172.21.15.59:0/4084908608 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c15f710 con 0x7fc96c0b3080 2024-04-23T14:42:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.590+0000 7fc971c00640 1 --2- 172.21.15.59:0/4084908608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c0b3080 0x7fc96c0cf470 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097a0 tx=0x7fc96402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2164632c09118b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.590+0000 7fc96b400640 1 -- 172.21.15.59:0/4084908608 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964037070 con 0x7fc96c0b3080 2024-04-23T14:42:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.590+0000 7fc96b400640 1 -- 172.21.15.59:0/4084908608 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96402ae50 con 0x7fc96c0b3080 2024-04-23T14:42:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.590+0000 7fc96b400640 1 -- 172.21.15.59:0/4084908608 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964032640 con 0x7fc96c0b3080 2024-04-23T14:42:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.590+0000 7fc972600640 1 -- 172.21.15.59:0/4084908608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c0b3080 msgr2=0x7fc96c0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.590+0000 7fc972600640 1 --2- 172.21.15.59:0/4084908608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c0b3080 0x7fc96c0cf470 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097a0 tx=0x7fc96402a100 comp rx=0 tx=0).stop 2024-04-23T14:42:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.591+0000 7fc972600640 1 -- 172.21.15.59:0/4084908608 shutdown_connections 2024-04-23T14:42:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.591+0000 7fc972600640 1 --2- 172.21.15.59:0/4084908608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c0b3080 0x7fc96c0cf470 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.591+0000 7fc972600640 1 --2- 172.21.15.59:0/4084908608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c06f620 0x7fc96c15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.591+0000 7fc972600640 1 -- 172.21.15.59:0/4084908608 >> 172.21.15.59:0/4084908608 conn(0x7fc96c0b21a0 msgr2=0x7fc96c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.591+0000 7fc972600640 1 -- 172.21.15.59:0/4084908608 shutdown_connections 2024-04-23T14:42:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.591+0000 7fc972600640 1 -- 172.21.15.59:0/4084908608 wait complete. 2024-04-23T14:42:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.592+0000 7fc972600640 1 Processor -- start 2024-04-23T14:42:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.593+0000 7fc972600640 1 -- start start 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.593+0000 7fc972600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c06f620 0x7fc96c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.593+0000 7fc972600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0b3080 0x7fc96c157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.593+0000 7fc972600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c0cd050 0x7fc96c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.593+0000 7fc972600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c0cef50 con 0x7fc96c06f620 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.593+0000 7fc972600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c06faa0 con 0x7fc96c0b3080 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.593+0000 7fc972600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c159110 con 0x7fc96c0cd050 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c06f620 0x7fc96c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c06f620 0x7fc96c159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39818/0 (socket says 172.21.15.59:39818) 2024-04-23T14:42:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 -- 172.21.15.59:0/3378949860 learned_addr learned my addr 172.21.15.59:0/3378949860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 -- 172.21.15.59:0/3378949860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c0cd050 msgr2=0x7fc96c0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c0cd050 0x7fc96c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 -- 172.21.15.59:0/3378949860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0b3080 msgr2=0x7fc96c157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0b3080 0x7fc96c157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 -- 172.21.15.59:0/3378949860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0c74d0 con 0x7fc96c06f620 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.594+0000 7fc971200640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c06f620 0x7fc96c159490 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc95400c390 tx=0x7fc95400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.595+0000 7fc968c00640 1 -- 172.21.15.59:0/3378949860 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954017070 con 0x7fc96c06f620 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.595+0000 7fc968c00640 1 -- 172.21.15.59:0/3378949860 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc954005ce0 con 0x7fc96c06f620 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.595+0000 7fc968c00640 1 -- 172.21.15.59:0/3378949860 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9540072f0 con 0x7fc96c06f620 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.595+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c0c77b0 con 0x7fc96c06f620 2024-04-23T14:42:00.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.595+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c1599d0 con 0x7fc96c06f620 2024-04-23T14:42:00.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.596+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96c0c7260 con 0x7fc96c06f620 2024-04-23T14:42:00.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.597+0000 7fc968c00640 1 -- 172.21.15.59:0/3378949860 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc95401a070 con 0x7fc96c06f620 2024-04-23T14:42:00.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.598+0000 7fc968c00640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc94806c8f0 0x7fc94806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:00.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.598+0000 7fc968c00640 1 -- 172.21.15.59:0/3378949860 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fc95408b0a0 con 0x7fc96c06f620 2024-04-23T14:42:00.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.598+0000 7fc96be00640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc94806c8f0 0x7fc94806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:00.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.599+0000 7fc96be00640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc94806c8f0 0x7fc94806edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0070b0 tx=0x7fc95c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.603+0000 7fc968c00640 1 -- 172.21.15.59:0/3378949860 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9540593a0 con 0x7fc96c06f620 2024-04-23T14:42:00.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.846+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc96c0c7ed0 con 0x7fc96c06f620 2024-04-23T14:42:00.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.847+0000 7fc968c00640 1 -- 172.21.15.59:0/3378949860 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5180 (secure 0 0 0) 0x7fc954058ca0 con 0x7fc96c06f620 2024-04-23T14:42:00.848 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:00.848 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:41:32.686386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8514":{"gid":8514,"name":"a","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/620420969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":620420969},{"type":"v1","addr":"172.21.15.59:6837","nonce":620420969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8514,"qdb_cluster":[8514]},"id":7}]} 2024-04-23T14:42:00.849 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 82 2024-04-23T14:42:00.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.849+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc94806c8f0 msgr2=0x7fc94806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:00.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.849+0000 7fc972600640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc94806c8f0 0x7fc94806edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0070b0 tx=0x7fc95c007040 comp rx=0 tx=0).stop 2024-04-23T14:42:00.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c06f620 msgr2=0x7fc96c159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:00.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c06f620 0x7fc96c159490 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc95400c390 tx=0x7fc95400c850 comp rx=0 tx=0).stop 2024-04-23T14:42:00.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 shutdown_connections 2024-04-23T14:42:00.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc94806c8f0 0x7fc94806edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc96c0cd050 0x7fc96c0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0b3080 0x7fc96c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 --2- 172.21.15.59:0/3378949860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc96c06f620 0x7fc96c159490 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:00.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 >> 172.21.15.59:0/3378949860 conn(0x7fc96c0b21a0 msgr2=0x7fc96c0cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:00.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 shutdown_connections 2024-04-23T14:42:00.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:00.850+0000 7fc972600640 1 -- 172.21.15.59:0/3378949860 wait complete. 2024-04-23T14:42:00.870 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-23T14:42:01.114 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2024-04-23T14:42:01.115 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-23T14:42:01.115 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:42:01.115 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:42:01.171 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:42:01.172 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi059.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-23T14:42:01.172 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:42:01.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:42:01.172 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:42:01.223 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-23T14:42:01.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:42:01.225 DEBUG:teuthology.orchestra.run.smithi059:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-23T14:42:01.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:42:01.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-23T14:42:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.416+0000 7f5f07600640 1 Processor -- start 2024-04-23T14:42:01.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.416+0000 7f5f07600640 1 -- start start 2024-04-23T14:42:01.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f07600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f000d0810 0x7f5f0015f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f07600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5f000afb30 0x7f5f000adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f07600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f000b4720 con 0x7f5f0015e9b0 2024-04-23T14:42:01.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f07600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5f000b3430 con 0x7f5f000afb30 2024-04-23T14:42:01.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f07600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5f0015fe40 con 0x7f5f000d0810 2024-04-23T14:42:01.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f04c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5f0015e9b0 0x7f5f000d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50394/0 (socket says 172.21.15.59:50394) 2024-04-23T14:42:01.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f04c00640 1 -- 172.21.15.59:0/4041268934 learned_addr learned my addr 172.21.15.59:0/4041268934 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:01.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.418+0000 7f5f05600640 1 --2- 172.21.15.59:0/4041268934 >> v2:172.21.15.59:3300/0 conn(0x7f5f000afb30 0x7f5f000adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.419+0000 7f5effe00640 1 --2- 172.21.15.59:0/4041268934 >> v2:172.21.15.76:3300/0 conn(0x7f5f000d0810 0x7f5f0015f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.419+0000 7f5eff400640 1 -- 172.21.15.59:0/4041268934 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2481174450 0 0) 0x7f5f000b4720 con 0x7f5f0015e9b0 2024-04-23T14:42:01.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.419+0000 7f5f05600640 1 -- 172.21.15.59:0/4041268934 >> v2:172.21.15.76:3300/0 conn(0x7f5f000d0810 msgr2=0x7f5f0015f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.419+0000 7f5f05600640 1 --2- 172.21.15.59:0/4041268934 >> v2:172.21.15.76:3300/0 conn(0x7f5f000d0810 0x7f5f0015f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.419+0000 7f5f05600640 1 -- 172.21.15.59:0/4041268934 >> v1:172.21.15.59:6789/0 conn(0x7f5f0015e9b0 legacy=0x7f5f000d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.419+0000 7f5f05600640 1 -- 172.21.15.59:0/4041268934 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f000b3680 con 0x7f5f000afb30 2024-04-23T14:42:01.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.420+0000 7f5f05600640 1 --2- 172.21.15.59:0/4041268934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f000adb50 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f5f0006fe80 tx=0x7f5ef802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c0e50f7efc90ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.420+0000 7f5eff400640 1 -- 172.21.15.59:0/4041268934 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8007740 con 0x7f5f000afb30 2024-04-23T14:42:01.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.420+0000 7f5eff400640 1 -- 172.21.15.59:0/4041268934 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef80078e0 con 0x7f5f000afb30 2024-04-23T14:42:01.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.421+0000 7f5eff400640 1 -- 172.21.15.59:0/4041268934 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8007cb0 con 0x7f5f000afb30 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.421+0000 7f5f07600640 1 -- 172.21.15.59:0/4041268934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 msgr2=0x7f5f000adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.421+0000 7f5f07600640 1 --2- 172.21.15.59:0/4041268934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f000adb50 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f5f0006fe80 tx=0x7f5ef802a680 comp rx=0 tx=0).stop 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.421+0000 7f5f07600640 1 -- 172.21.15.59:0/4041268934 shutdown_connections 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.421+0000 7f5f07600640 1 --2- 172.21.15.59:0/4041268934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f000adb50 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.421+0000 7f5f07600640 1 --2- 172.21.15.59:0/4041268934 >> v2:172.21.15.76:3300/0 conn(0x7f5f000d0810 0x7f5f0015f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.421+0000 7f5f07600640 1 -- 172.21.15.59:0/4041268934 >> 172.21.15.59:0/4041268934 conn(0x7f5f0006b9d0 msgr2=0x7f5f0006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.422+0000 7f5f07600640 1 -- 172.21.15.59:0/4041268934 shutdown_connections 2024-04-23T14:42:01.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.422+0000 7f5f07600640 1 -- 172.21.15.59:0/4041268934 wait complete. 2024-04-23T14:42:01.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.423+0000 7f5f07600640 1 Processor -- start 2024-04-23T14:42:01.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.423+0000 7f5f07600640 1 -- start start 2024-04-23T14:42:01.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.424+0000 7f5f07600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f0014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.424+0000 7f5f07600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f000d0810 0x7f5f000b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.424+0000 7f5f07600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f0015e9b0 0x7f5f0015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.424+0000 7f5f07600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000b3f20 con 0x7f5f000afb30 2024-04-23T14:42:01.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.424+0000 7f5f07600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000b3c40 con 0x7f5f000d0810 2024-04-23T14:42:01.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.424+0000 7f5f07600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5f000b4200 con 0x7f5f0015e9b0 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f0014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f0014d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39846/0 (socket says 172.21.15.59:39846) 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 -- 172.21.15.59:0/1422946759 learned_addr learned my addr 172.21.15.59:0/1422946759 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 -- 172.21.15.59:0/1422946759 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f0015e9b0 msgr2=0x7f5f0015ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f0015e9b0 0x7f5f0015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5effe00640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f000d0810 0x7f5f000b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 -- 172.21.15.59:0/1422946759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f000d0810 msgr2=0x7f5f000b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f000d0810 0x7f5f000b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 -- 172.21.15.59:0/1422946759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f000b2060 con 0x7f5f000afb30 2024-04-23T14:42:01.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.425+0000 7f5f04c00640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f0014d030 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400c430 tx=0x7f5ef400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.426+0000 7f5efcc00640 1 -- 172.21.15.59:0/1422946759 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40074e0 con 0x7f5f000afb30 2024-04-23T14:42:01.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.426+0000 7f5efcc00640 1 -- 172.21.15.59:0/1422946759 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef40047e0 con 0x7f5f000afb30 2024-04-23T14:42:01.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.426+0000 7f5efcc00640 1 -- 172.21.15.59:0/1422946759 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4004bd0 con 0x7f5f000afb30 2024-04-23T14:42:01.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.426+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f000b2340 con 0x7f5f000afb30 2024-04-23T14:42:01.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.426+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f000b2900 con 0x7f5f000afb30 2024-04-23T14:42:01.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.428+0000 7f5efcc00640 1 -- 172.21.15.59:0/1422946759 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ef4011070 con 0x7f5f000afb30 2024-04-23T14:42:01.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.430+0000 7f5efcc00640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ee006c920 0x7f5ee006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.430+0000 7f5efcc00640 1 -- 172.21.15.59:0/1422946759 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f5ef408b760 con 0x7f5f000afb30 2024-04-23T14:42:01.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.430+0000 7f5effe00640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ee006c920 0x7f5ee006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.430+0000 7f5f0a42f740 1 Processor -- start 2024-04-23T14:42:01.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.430+0000 7f5f0a42f740 1 -- start start 2024-04-23T14:42:01.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5effe00640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ee006c920 0x7f5ee006ede0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5f00158e10 tx=0x7f5eec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb01c1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb01f1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f1e40 0x55edb01c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55edb012be70 con 0x55edb01f2580 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55edb00d2380 con 0x55edb01f1e40 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55edb0127e30 con 0x55edb020f460 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.431+0000 7f5effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb01f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb01f1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39850/0 (socket says 172.21.15.59:39850) 2024-04-23T14:42:01.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 -- 172.21.15.59:0/2198876160 learned_addr learned my addr 172.21.15.59:0/2198876160 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5f05600640 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f1e40 0x55edb01c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5f04c00640 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb01c1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 -- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 msgr2=0x55edb01c1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb01c1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 -- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f1e40 msgr2=0x55edb01c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f1e40 0x55edb01c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 -- 172.21.15.59:0/2198876160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edaff067e0 con 0x55edb01f2580 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5effe00640 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb01f1900 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e920 tx=0x7f5eec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f5f07fdb2b6e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.432+0000 7f5ef1200640 1 -- 172.21.15.59:0/2198876160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec025070 con 0x55edb01f2580 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5ef1200640 1 -- 172.21.15.59:0/2198876160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec015700 con 0x55edb01f2580 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5ef1200640 1 -- 172.21.15.59:0/2198876160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00f040 con 0x55edb01f2580 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 -- 172.21.15.59:0/2198876160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 msgr2=0x55edb01f1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb01f1900 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e920 tx=0x7f5eec011000 comp rx=0 tx=0).stop 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 -- 172.21.15.59:0/2198876160 shutdown_connections 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f1e40 0x55edb01c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb01f1900 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/2198876160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb01c1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 -- 172.21.15.59:0/2198876160 >> 172.21.15.59:0/2198876160 conn(0x55edb01c4060 msgr2=0x55edb020c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 -- 172.21.15.59:0/2198876160 shutdown_connections 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.433+0000 7f5f0a42f740 1 -- 172.21.15.59:0/2198876160 wait complete. 2024-04-23T14:42:01.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.434+0000 7f5f0a42f740 1 -- start start 2024-04-23T14:42:01.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.434+0000 7f5f0a42f740 1 Processor -- start 2024-04-23T14:42:01.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7fdf92600640 1 Processor -- start 2024-04-23T14:42:01.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7fdf92600640 1 -- start start 2024-04-23T14:42:01.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb01e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb020f460 0x55edb01f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb0203fd0 0x55edb020da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55edb01c6210 con 0x55edb020f460 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55edb00e5660 con 0x55edb01f2580 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.435+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55edb01c5600 con 0x55edb0203fd0 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5f0a42f740 10 client.0 _ll_register_callbacks cb 0x55edb00d3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb020f460 0x55edb01f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb020f460 0x55edb01f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39852/0 (socket says 172.21.15.59:39852) 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 -- 172.21.15.59:0/3935407216 learned_addr learned my addr 172.21.15.59:0/3935407216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5f04c00640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb01e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5f05600640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb0203fd0 0x55edb020da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7fdf92600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf8c15fdb0 0x7fdf8c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 -- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb0203fd0 msgr2=0x55edb020da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb0203fd0 0x55edb020da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 -- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 msgr2=0x55edb01e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb01e2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7f5effe00640 1 -- 172.21.15.59:0/3935407216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edaff66f70 con 0x55edb020f460 2024-04-23T14:42:01.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7fdf92600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf8c146000 0x7fdf8c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7fdf92600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c0cf330 con 0x7fdf8c0cd090 2024-04-23T14:42:01.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7fdf92600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8c0cfc00 con 0x7fdf8c15fdb0 2024-04-23T14:42:01.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.436+0000 7fdf92600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8c06f370 con 0x7fdf8c146000 2024-04-23T14:42:01.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf8c15fdb0 0x7fdf8c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdf8c15fdb0 0x7fdf8c15cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39868/0 (socket says 172.21.15.59:39868) 2024-04-23T14:42:01.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8b400640 1 -- 172.21.15.59:0/3315677301 learned_addr learned my addr 172.21.15.59:0/3315677301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7f5f04c00640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb01e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7f5effe00640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb020f460 0x55edb01f4e10 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e8f0 tx=0x7f5eec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbeed06cc7b7930 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7f5ee5600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8aa00640 1 --2- 172.21.15.59:0/3315677301 >> v2:172.21.15.76:3300/0 conn(0x7fdf8c146000 0x7fdf8c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8b400640 1 -- 172.21.15.59:0/3315677301 >> v2:172.21.15.76:3300/0 conn(0x7fdf8c146000 msgr2=0x7fdf8c06ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8b400640 1 --2- 172.21.15.59:0/3315677301 >> v2:172.21.15.76:3300/0 conn(0x7fdf8c146000 0x7fdf8c06ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8b400640 1 -- 172.21.15.59:0/3315677301 >> v1:172.21.15.59:6789/0 conn(0x7fdf8c0cd090 legacy=0x7fdf8c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7fdf8b400640 1 -- 172.21.15.59:0/3315677301 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c0cf5b0 con 0x7fdf8c15fdb0 2024-04-23T14:42:01.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7f5ee5600640 1 -- 172.21.15.59:0/3935407216 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec025040 con 0x55edb020f460 2024-04-23T14:42:01.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7f5ee5600640 1 -- 172.21.15.59:0/3935407216 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f5eec02c680 con 0x55edb020f460 2024-04-23T14:42:01.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.437+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec015700 con 0x55edb020f460 2024-04-23T14:42:01.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5ee5600640 1 -- 172.21.15.59:0/3935407216 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5eec02c880 con 0x55edb020f460 2024-04-23T14:42:01.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 10 client.8853 fetch_fsmap learned FSMap version 82 2024-04-23T14:42:01.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55edafdfe170 con 0x55edb020f460 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5ee5600640 1 -- 172.21.15.59:0/3935407216 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec015700 con 0x55edb020f460 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5ee5600640 1 -- 172.21.15.59:0/3935407216 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f5eec012030 con 0x55edb020f460 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 10 client.8853 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 10 client.8853 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 20 client.8853 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb021cc10 0x55edb021d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 4 client.8853 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edb0139890 con 0x55edb021cc10 2024-04-23T14:42:01.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7fdf8b400640 1 --2- 172.21.15.59:0/3315677301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c15fdb0 0x7fdf8c15cd00 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0b58b0 tx=0x7fdf7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44c254bb51b829 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.438+0000 7f5f04c00640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb021cc10 0x55edb021d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.439+0000 7fdf8a000640 1 -- 172.21.15.59:0/3315677301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c037070 con 0x7fdf8c15fdb0 2024-04-23T14:42:01.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.439+0000 7fdf8a000640 1 -- 172.21.15.59:0/3315677301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c032440 con 0x7fdf8c15fdb0 2024-04-23T14:42:01.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.439+0000 7fdf8a000640 1 -- 172.21.15.59:0/3315677301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c032810 con 0x7fdf8c15fdb0 2024-04-23T14:42:01.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.439+0000 7f5f04c00640 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb021cc10 0x55edb021d050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4fb1ad0af522ec8 server_cookie=e4cab2515bdabb3b in_seq=0 out_seq=0 2024-04-23T14:42:01.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.439+0000 7f5ee5600640 10 client.8853 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.439+0000 7fdf92600640 1 -- 172.21.15.59:0/3315677301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c15fdb0 msgr2=0x7fdf8c15cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.439+0000 7fdf92600640 1 --2- 172.21.15.59:0/3315677301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c15fdb0 0x7fdf8c15cd00 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0b58b0 tx=0x7fdf7c02a270 comp rx=0 tx=0).stop 2024-04-23T14:42:01.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7fdf92600640 1 -- 172.21.15.59:0/3315677301 shutdown_connections 2024-04-23T14:42:01.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7fdf92600640 1 --2- 172.21.15.59:0/3315677301 >> v2:172.21.15.76:3300/0 conn(0x7fdf8c146000 0x7fdf8c06ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7fdf92600640 1 --2- 172.21.15.59:0/3315677301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c15fdb0 0x7fdf8c15cd00 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7fdf92600640 1 -- 172.21.15.59:0/3315677301 >> 172.21.15.59:0/3315677301 conn(0x7fdf8c067ea0 msgr2=0x7fdf8c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7fdf92600640 1 -- 172.21.15.59:0/3315677301 shutdown_connections 2024-04-23T14:42:01.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7fdf92600640 1 -- 172.21.15.59:0/3315677301 wait complete. 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5ee5600640 1 -- 172.21.15.59:0/3935407216 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edb0139890 con 0x55edb021cc10 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5ee5600640 10 client.8853 handle_command_reply: tid=0 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 client.8853 shutdown 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb020f460 msgr2=0x55edb01f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb020f460 0x55edb01f4e10 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e8f0 tx=0x7f5eec004730 comp rx=0 tx=0).stop 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 shutdown_connections 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb0203fd0 0x55edb020da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb021cc10 0x55edb021d050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb020f460 0x55edb01f4e10 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3935407216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb01e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 >> 172.21.15.59:0/3935407216 conn(0x55edb01f1e40 msgr2=0x55edb020c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5ee5600640 0 client.8853 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.440+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 shutdown_connections 2024-04-23T14:42:01.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.441+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3935407216 wait complete. 2024-04-23T14:42:01.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.441+0000 7f5f0a42f740 20 client.8853 trim_cache size 0 max 16384 2024-04-23T14:42:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.441+0000 7fdf92600640 1 Processor -- start 2024-04-23T14:42:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.442+0000 7fdf92600640 1 -- start start 2024-04-23T14:42:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.442+0000 7fdf92600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c0cd090 0x7fdf8c13c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.442+0000 7fdf92600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf8c146000 0x7fdf8c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.442+0000 7fdf92600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf8c15fdb0 0x7fdf8c13fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.442+0000 7fdf92600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c06efb0 con 0x7fdf8c0cd090 2024-04-23T14:42:01.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.442+0000 7fdf92600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c0cf890 con 0x7fdf8c146000 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.442+0000 7fdf92600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8c06f130 con 0x7fdf8c15fdb0 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c0cd090 0x7fdf8c13c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c0cd090 0x7fdf8c13c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39878/0 (socket says 172.21.15.59:39878) 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 -- 172.21.15.59:0/3829459923 learned_addr learned my addr 172.21.15.59:0/3829459923 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 -- 172.21.15.59:0/3829459923 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf8c15fdb0 msgr2=0x7fdf8c13fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf8c15fdb0 0x7fdf8c13fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 -- 172.21.15.59:0/3829459923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf8c146000 msgr2=0x7fdf8c13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8aa00640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf8c146000 0x7fdf8c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf8c146000 0x7fdf8c13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8b400640 1 -- 172.21.15.59:0/3829459923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c13f5f0 con 0x7fdf8c0cd090 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.443+0000 7fdf8aa00640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf8c146000 0x7fdf8c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.444+0000 7fdf8b400640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf8c0cd090 0x7fdf8c13c930 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c02a2b0 tx=0x7fdf7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.444+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c041020 con 0x7fdf8c0cd090 2024-04-23T14:42:01.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.444+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c0306c0 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.444+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c037040 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.444+0000 7fdf92600640 1 -- 172.21.15.59:0/3829459923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c13f8d0 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.444+0000 7fdf92600640 1 -- 172.21.15.59:0/3829459923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8c0ca290 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.445+0000 7fdf92600640 1 -- 172.21.15.59:0/3829459923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdf8c073800 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.446+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf7c044020 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.447+0000 7fdf83400640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.447+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fdf7c0ac500 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.447+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+515 (secure 0 0 0) 0x7fdf7c04b030 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.447+0000 7fdf8aa00640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.447+0000 7fdf92600640 1 -- 172.21.15.59:0/3829459923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fdf8c15da20 con 0x7fdf8c0cd090 2024-04-23T14:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.448+0000 7fdf8aa00640 1 --2- 172.21.15.59:0/3829459923 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf6c06c8f0 0x7fdf6c06edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c144df0 tx=0x7fdf78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.461+0000 7f5f0a42f740 1 Processor -- start 2024-04-23T14:42:01.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.461+0000 7f5f0a42f740 1 -- start start 2024-04-23T14:42:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb020c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb0203fd0 0x55edb021d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb01c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00200cb0 con 0x55edb0203fd0 2024-04-23T14:42:01.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001ffd70 con 0x55edb01f2580 2024-04-23T14:42:01.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5f000c9020 con 0x55edb020f460 2024-04-23T14:42:01.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb0203fd0 0x55edb021d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5effe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb0203fd0 0x55edb021d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39890/0 (socket says 172.21.15.59:39890) 2024-04-23T14:42:01.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.462+0000 7f5effe00640 1 -- 172.21.15.59:0/921629055 learned_addr learned my addr 172.21.15.59:0/921629055 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5effe00640 1 -- 172.21.15.59:0/921629055 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 msgr2=0x55edb01c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5effe00640 1 --2- 172.21.15.59:0/921629055 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb01c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5effe00640 1 -- 172.21.15.59:0/921629055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 msgr2=0x55edb020c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5effe00640 1 --2- 172.21.15.59:0/921629055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb020c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5effe00640 1 -- 172.21.15.59:0/921629055 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb01c77b0 con 0x55edb0203fd0 2024-04-23T14:42:01.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5effe00640 1 --2- 172.21.15.59:0/921629055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb0203fd0 0x55edb021d090 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5eec016c70 tx=0x7f5eec026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b1ec7eb9741831 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5ee7e00640 1 -- 172.21.15.59:0/921629055 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec03d070 con 0x55edb0203fd0 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5ee7e00640 1 -- 172.21.15.59:0/921629055 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec02c6a0 con 0x55edb0203fd0 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.463+0000 7f5ee7e00640 1 -- 172.21.15.59:0/921629055 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec025040 con 0x55edb0203fd0 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 -- 172.21.15.59:0/921629055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb0203fd0 msgr2=0x55edb021d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/921629055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb0203fd0 0x55edb021d090 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5eec016c70 tx=0x7f5eec026000 comp rx=0 tx=0).stop 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 -- 172.21.15.59:0/921629055 shutdown_connections 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/921629055 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb01c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/921629055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb0203fd0 0x55edb021d090 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/921629055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb01f2580 0x55edb020c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 -- 172.21.15.59:0/921629055 >> 172.21.15.59:0/921629055 conn(0x55edb01f1e40 msgr2=0x55edb01f4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 -- 172.21.15.59:0/921629055 shutdown_connections 2024-04-23T14:42:01.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.464+0000 7f5f0a42f740 1 -- 172.21.15.59:0/921629055 wait complete. 2024-04-23T14:42:01.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.465+0000 7f5f0a42f740 1 -- start start 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.465+0000 7f5f0a42f740 1 Processor -- start 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb02108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb0203fd0 0x55edb020ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f0a42f740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb02256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55edb020e300 con 0x55edb01f2580 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55edb01f46c0 con 0x55edb0203fd0 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f0a42f740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55edb0229370 con 0x55edb020f460 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f0a42f740 10 client.0 _ll_register_callbacks cb 0x7f5eec023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f04c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb02108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f04c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb02108e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39892/0 (socket says 172.21.15.59:39892) 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.466+0000 7f5f04c00640 1 -- 172.21.15.59:0/3727860804 learned_addr learned my addr 172.21.15.59:0/3727860804 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f04c00640 1 -- 172.21.15.59:0/3727860804 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 msgr2=0x55edb02256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f04c00640 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb02256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f04c00640 1 -- 172.21.15.59:0/3727860804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb0203fd0 msgr2=0x55edb020ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f04c00640 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb0203fd0 0x55edb020ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f04c00640 1 -- 172.21.15.59:0/3727860804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb0226290 con 0x55edb01f2580 2024-04-23T14:42:01.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f04c00640 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb02108e0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4059180 tx=0x7f5ef400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f6977868cbc0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5ee5600640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:01.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5ee5600640 1 -- 172.21.15.59:0/3727860804 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4076060 con 0x55edb01f2580 2024-04-23T14:42:01.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5ee5600640 1 -- 172.21.15.59:0/3727860804 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f5ef4004d70 con 0x55edb01f2580 2024-04-23T14:42:01.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec02c6a0 con 0x55edb01f2580 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5ee5600640 1 -- 172.21.15.59:0/3727860804 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5ef4088050 con 0x55edb01f2580 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f0a42f740 10 client.8868 fetch_fsmap learned FSMap version 82 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.467+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55edb020b780 con 0x55edb01f2580 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5ee5600640 1 -- 172.21.15.59:0/3727860804 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec02c6a0 con 0x55edb01f2580 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5ee5600640 1 -- 172.21.15.59:0/3727860804 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f5ef4055af0 con 0x55edb01f2580 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5f0a42f740 10 client.8868 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5f0a42f740 10 client.8868 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5f0a42f740 20 client.8868 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb01e2370 0x55edb0225c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5f0a42f740 4 client.8868 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55edb0133700 con 0x55edb01e2370 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.468+0000 7f5effe00640 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb01e2370 0x55edb0225c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.469+0000 7f5effe00640 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb01e2370 0x55edb0225c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adcfb424073b3de2 server_cookie=f8591dcc6406137e in_seq=0 out_seq=0 2024-04-23T14:42:01.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.469+0000 7f5ee5600640 10 client.8868 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.469+0000 7f5ee5600640 1 -- 172.21.15.59:0/3727860804 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55edb0133700 con 0x55edb01e2370 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.469+0000 7f5ee5600640 10 client.8868 handle_command_reply: tid=0 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.469+0000 7f5f0a42f740 1 client.8868 shutdown 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 msgr2=0x55edb02108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb02108e0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4059180 tx=0x7f5ef400cf20 comp rx=0 tx=0).stop 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 shutdown_connections 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55edb020f460 0x55edb02256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x55edb01e2370 0x55edb0225c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55edb0203fd0 0x55edb020ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 --2- 172.21.15.59:0/3727860804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55edb01f2580 0x55edb02108e0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5ee5600640 0 client.8868 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 >> 172.21.15.59:0/3727860804 conn(0x55edb01f1e40 msgr2=0x55edb020c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 shutdown_connections 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 1 -- 172.21.15.59:0/3727860804 wait complete. 2024-04-23T14:42:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.470+0000 7f5f0a42f740 20 client.8868 trim_cache size 0 max 16384 2024-04-23T14:42:01.474 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:42:01.474 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:42:01.474 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "49b75886-342a-4957-82c1-1a0d0fb346aa", 2024-04-23T14:42:01.474 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:42:01.474 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ee006c920 msgr2=0x7f5ee006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ee006c920 0x7f5ee006ede0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5f00158e10 tx=0x7f5eec008040 comp rx=0 tx=0).stop 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 msgr2=0x7f5f0014d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f0014d030 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400c430 tx=0x7f5ef400c8f0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 shutdown_connections 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5ee006c920 0x7f5ee006ede0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5f0015e9b0 0x7f5f0015ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f000d0810 0x7f5f000b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 --2- 172.21.15.59:0/1422946759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f000afb30 0x7f5f0014d030 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 >> 172.21.15.59:0/1422946759 conn(0x7f5f0006b9d0 msgr2=0x7f5f000ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 shutdown_connections 2024-04-23T14:42:01.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.471+0000 7f5f07600640 1 -- 172.21.15.59:0/1422946759 wait complete. 2024-04-23T14:42:01.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:42:01.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-23T14:42:01.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.668+0000 7f01e9a00640 1 Processor -- start 2024-04-23T14:42:01.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.668+0000 7f01e9a00640 1 -- start start 2024-04-23T14:42:01.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.669+0000 7f01e9a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01e415e960 0x7f01e40d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.669+0000 7f01e9a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e41631c0 con 0x7f01e40d1720 2024-04-23T14:42:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.669+0000 7f01e9a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01e40d29b0 con 0x7f01e4162c80 2024-04-23T14:42:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.669+0000 7f01e9a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f01e40d1bb0 con 0x7f01e415e960 2024-04-23T14:42:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.670+0000 7f01e2000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f01e40d1720 0x7f01e4162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:50406/0 (socket says 172.21.15.59:50406) 2024-04-23T14:42:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.670+0000 7f01e2000640 1 -- 172.21.15.59:0/2232880272 learned_addr learned my addr 172.21.15.59:0/2232880272 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.670+0000 7f01e2a00640 1 --2- 172.21.15.59:0/2232880272 >> v2:172.21.15.76:3300/0 conn(0x7f01e415e960 0x7f01e40d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.670+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722978399 0 0) 0x7f01e41631c0 con 0x7f01e40d1720 2024-04-23T14:42:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.670+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01c4003520 con 0x7f01e40d1720 2024-04-23T14:42:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3278240082 0 0) 0x7f01c4003520 con 0x7f01e40d1720 2024-04-23T14:42:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 >> v2:172.21.15.76:3300/0 conn(0x7f01e415e960 msgr2=0x7f01e40d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 --2- 172.21.15.59:0/2232880272 >> v2:172.21.15.76:3300/0 conn(0x7f01e415e960 0x7f01e40d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 >> v1:172.21.15.76:6789/0 conn(0x7f01e4162c80 legacy=0x7f01e40d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e40d2c90 con 0x7f01e40d1720 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f01cc001670 con 0x7f01e40d1720 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 >> v1:172.21.15.59:6789/0 conn(0x7f01e40d1720 legacy=0x7f01e4162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4003a00 0x7f01c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01c4005210 0x7f01c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01c400c130 con 0x7f01c4004800 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01c400c2e0 con 0x7f01c4005210 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f01c400c4e0 con 0x7f01c4003a00 2024-04-23T14:42:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.671+0000 7f01e2000640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e3400640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01c4005210 0x7f01c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e2a00640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4003a00 0x7f01c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e2000640 1 -- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4003a00 msgr2=0x7f01c4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e2000640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4003a00 0x7f01c4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e2000640 1 -- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01c4005210 msgr2=0x7f01c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e2000640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01c4005210 0x7f01c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e2000640 1 -- 172.21.15.59:0/2232880272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c4003520 con 0x7f01c4004800 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e3400640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01c4005210 0x7f01c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e2a00640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4003a00 0x7f01c4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.672+0000 7f01e3400640 1 -- 172.21.15.59:0/2232880272 reap_dead start 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.673+0000 7f01e2000640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c0e0 tx=0x7f01c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797b26f2a2896019 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.673+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc007c40 con 0x7f01c4004800 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.673+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc005af0 con 0x7f01c4004800 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.673+0000 7f01e1600640 1 -- 172.21.15.59:0/2232880272 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc007070 con 0x7f01c4004800 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.674+0000 7f01e9a00640 1 -- 172.21.15.59:0/2232880272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01c4004800 msgr2=0x7f01c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.674+0000 7f01e9a00640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c0e0 tx=0x7f01c4005830 comp rx=0 tx=0).stop 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.674+0000 7f01e9a00640 1 -- 172.21.15.59:0/2232880272 shutdown_connections 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.674+0000 7f01e9a00640 1 --2- 172.21.15.59:0/2232880272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01c4004800 0x7f01c4004c80 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.674+0000 7f01e9a00640 1 -- 172.21.15.59:0/2232880272 >> 172.21.15.59:0/2232880272 conn(0x7f01e4067760 msgr2=0x7f01e4067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.674+0000 7f01e9a00640 1 -- 172.21.15.59:0/2232880272 shutdown_connections 2024-04-23T14:42:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.674+0000 7f01e9a00640 1 -- 172.21.15.59:0/2232880272 wait complete. 2024-04-23T14:42:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.676+0000 7f01e9a00640 1 Processor -- start 2024-04-23T14:42:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.676+0000 7f01e9a00640 1 -- start start 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e9a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4004800 0x7f01e41f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40b4c70 0x7f01e40ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e9a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01e40ca940 0x7f01e40cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01e415ed40 con 0x7f01e40b4c70 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e9a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40b7930 con 0x7f01e40ca940 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e9a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f01e415eec0 con 0x7f01c4004800 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40b4c70 0x7f01e40ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40b4c70 0x7f01e40ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39922/0 (socket says 172.21.15.59:39922) 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e2000640 1 -- 172.21.15.59:0/28805493 learned_addr learned my addr 172.21.15.59:0/28805493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e2a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4004800 0x7f01e41f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e2000640 1 -- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4004800 msgr2=0x7f01e41f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e2000640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4004800 0x7f01e41f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e3400640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01e40ca940 0x7f01e40cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.677+0000 7f01e2000640 1 -- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01e40ca940 msgr2=0x7f01e40cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01e2000640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01e40ca940 0x7f01e40cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01e2000640 1 -- 172.21.15.59:0/28805493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e40b5100 con 0x7f01e40b4c70 2024-04-23T14:42:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01e2000640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40b4c70 0x7f01e40ca400 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02ad30 tx=0x7f01e4163390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01daa00640 1 -- 172.21.15.59:0/28805493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc042070 con 0x7f01e40b4c70 2024-04-23T14:42:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01daa00640 1 -- 172.21.15.59:0/28805493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc0047e0 con 0x7f01e40b4c70 2024-04-23T14:42:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01daa00640 1 -- 172.21.15.59:0/28805493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc037040 con 0x7f01e40b4c70 2024-04-23T14:42:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e40b53e0 con 0x7f01e40b4c70 2024-04-23T14:42:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.678+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e40b59a0 con 0x7f01e40b4c70 2024-04-23T14:42:01.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.681+0000 7f01daa00640 1 -- 172.21.15.59:0/28805493 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01cc045070 con 0x7f01e40b4c70 2024-04-23T14:42:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.682+0000 7f01daa00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01c006c8f0 0x7f01c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.682+0000 7f01daa00640 1 -- 172.21.15.59:0/28805493 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f01cc0ab3f0 con 0x7f01e40b4c70 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.683+0000 7f01e2a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01c006c8f0 0x7f01c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.683+0000 7f01ec7ec740 1 Processor -- start 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.683+0000 7f01ec7ec740 1 -- start start 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.683+0000 7f01e2a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01c006c8f0 0x7f01c006edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f01e406ad70 tx=0x7f01d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41610 0x563335b16420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563335a7b1f0 con 0x563335b41610 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563335a21350 con 0x563335b41d50 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563335a771b0 con 0x563335b5ec30 2024-04-23T14:42:01.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41610 0x563335b16420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41610 0x563335b16420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39928/0 (socket says 172.21.15.59:39928) 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.684+0000 7f01e3400640 1 -- 172.21.15.59:0/4080700866 learned_addr learned my addr 172.21.15.59:0/4080700866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e2000640 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e3400640 1 -- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 msgr2=0x563335b115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e3400640 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e3400640 1 -- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 msgr2=0x563335b410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e3400640 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e3400640 1 -- 172.21.15.59:0/4080700866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563335a75f50 con 0x563335b41610 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e2000640 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01e3400640 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41610 0x563335b16420 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f01e408da70 tx=0x7f01dc001ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60689800e405dcc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01d3400640 1 -- 172.21.15.59:0/4080700866 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc019070 con 0x563335b41610 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01d3400640 1 -- 172.21.15.59:0/4080700866 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc003d80 con 0x563335b41610 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.685+0000 7f01d3400640 1 -- 172.21.15.59:0/4080700866 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00e9e0 con 0x563335b41610 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 -- 172.21.15.59:0/4080700866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41610 msgr2=0x563335b16420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41610 0x563335b16420 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f01e408da70 tx=0x7f01dc001ed0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 -- 172.21.15.59:0/4080700866 shutdown_connections 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41610 0x563335b16420 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/4080700866 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 -- 172.21.15.59:0/4080700866 >> 172.21.15.59:0/4080700866 conn(0x563335b13830 msgr2=0x563335b5c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 -- 172.21.15.59:0/4080700866 shutdown_connections 2024-04-23T14:42:01.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.686+0000 7f01ec7ec740 1 -- 172.21.15.59:0/4080700866 wait complete. 2024-04-23T14:42:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.687+0000 7f01ec7ec740 1 -- start start 2024-04-23T14:42:01.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.687+0000 7f01ec7ec740 1 Processor -- start 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.688+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b6ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.688+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.688+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b15de0 0x563335b16260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.688+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563335b159e0 con 0x563335b15de0 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.688+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563335a34690 con 0x563335b41d50 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.688+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563335b14dd0 con 0x563335b5ec30 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.688+0000 7f01ec7ec740 10 client.0 _ll_register_callbacks cb 0x563335a70650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:01.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b15de0 0x563335b16260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b15de0 0x563335b16260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39938/0 (socket says 172.21.15.59:39938) 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 -- 172.21.15.59:0/2794737966 learned_addr learned my addr 172.21.15.59:0/2794737966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e2a00640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b6ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e2000640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 -- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 msgr2=0x563335b441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 -- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 msgr2=0x563335b6ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b6ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 -- 172.21.15.59:0/2794737966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563335a33920 con 0x563335b15de0 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e2a00640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b6ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01e3400640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b15de0 0x563335b16260 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ba970 tx=0x7f01dc001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c575b45e4f6998d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.689+0000 7f01cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:01.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01cbe00640 1 -- 172.21.15.59:0/2794737966 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc019070 con 0x563335b15de0 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01cbe00640 1 -- 172.21.15.59:0/2794737966 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f01dc0050d0 con 0x563335b15de0 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01dc003d80 con 0x563335b15de0 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01cbe00640 1 -- 172.21.15.59:0/2794737966 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f01dc02b050 con 0x563335b15de0 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01ec7ec740 10 client.8880 fetch_fsmap learned FSMap version 82 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5633358577d0 con 0x563335b15de0 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01cbe00640 1 -- 172.21.15.59:0/2794737966 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc003d80 con 0x563335b15de0 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01cbe00640 1 -- 172.21.15.59:0/2794737966 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f01dc02e020 con 0x563335b15de0 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01ec7ec740 10 client.8880 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.690+0000 7f01ec7ec740 10 client.8880 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.691+0000 7f01ec7ec740 20 client.8880 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.691+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335b6c500 0x563335b6c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.691+0000 7f01ec7ec740 4 client.8880 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:42:01.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.691+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563335a88c10 con 0x563335b6c500 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.691+0000 7f01e2a00640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335b6c500 0x563335b6c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.691+0000 7f01e2a00640 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335b6c500 0x563335b6c950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621f12f0c3ae37e9 server_cookie=f058941ecd6a76a2 in_seq=0 out_seq=0 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.691+0000 7f01cbe00640 10 client.8880 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.692+0000 7f01cbe00640 1 -- 172.21.15.59:0/2794737966 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563335a88c10 con 0x563335b6c500 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.692+0000 7f01cbe00640 10 client.8880 handle_command_reply: tid=0 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.692+0000 7f01ec7ec740 1 client.8880 shutdown 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.692+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b15de0 msgr2=0x563335b16260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.692+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b15de0 0x563335b16260 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ba970 tx=0x7f01dc001510 comp rx=0 tx=0).stop 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 shutdown_connections 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335b6c500 0x563335b6c950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b15de0 0x563335b16260 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2794737966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b41d50 0x563335b6ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 >> 172.21.15.59:0/2794737966 conn(0x563335b41610 msgr2=0x563335b10c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01cbe00640 0 client.8880 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 shutdown_connections 2024-04-23T14:42:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2794737966 wait complete. 2024-04-23T14:42:01.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.693+0000 7f01ec7ec740 20 client.8880 trim_cache size 0 max 16384 2024-04-23T14:42:01.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.712+0000 7f01ec7ec740 1 Processor -- start 2024-04-23T14:42:01.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.712+0000 7f01ec7ec740 1 -- start start 2024-04-23T14:42:01.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.712+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b15de0 0x563335b161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b7fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b5ec30 0x563335b168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4201510 con 0x563335b41d50 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01e42005b0 con 0x563335b5ec30 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f01e40b6b80 con 0x563335b15de0 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b7fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b7fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39940/0 (socket says 172.21.15.59:39940) 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 -- 172.21.15.59:0/2111633247 learned_addr learned my addr 172.21.15.59:0/2111633247 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 -- 172.21.15.59:0/2111633247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b15de0 msgr2=0x563335b161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 --2- 172.21.15.59:0/2111633247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b15de0 0x563335b161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 -- 172.21.15.59:0/2111633247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b5ec30 msgr2=0x563335b168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 --2- 172.21.15.59:0/2111633247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b5ec30 0x563335b168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.713+0000 7f01e2000640 1 -- 172.21.15.59:0/2111633247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333578d3f0 con 0x563335b41d50 2024-04-23T14:42:01.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.714+0000 7f01e2000640 1 --2- 172.21.15.59:0/2111633247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b7fa30 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0a97f0 tx=0x7f01cc0789e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b38629618c19bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.714+0000 7f01d2a00640 1 -- 172.21.15.59:0/2111633247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc079080 con 0x563335b41d50 2024-04-23T14:42:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.714+0000 7f01d2a00640 1 -- 172.21.15.59:0/2111633247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc07d430 con 0x563335b41d50 2024-04-23T14:42:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.714+0000 7f01d2a00640 1 -- 172.21.15.59:0/2111633247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc081c60 con 0x563335b41d50 2024-04-23T14:42:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2111633247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 msgr2=0x563335b7fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2111633247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b7fa30 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0a97f0 tx=0x7f01cc0789e0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2111633247 shutdown_connections 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2111633247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b5ec30 0x563335b168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2111633247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b7fa30 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/2111633247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b15de0 0x563335b161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2111633247 >> 172.21.15.59:0/2111633247 conn(0x563335b41610 msgr2=0x563335b3e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2111633247 shutdown_connections 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.715+0000 7f01ec7ec740 1 -- 172.21.15.59:0/2111633247 wait complete. 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.716+0000 7f01ec7ec740 1 -- start start 2024-04-23T14:42:01.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.716+0000 7f01ec7ec740 1 Processor -- start 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b15de0 0x563335b5fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b3b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01ec7ec740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b7e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563335b42ce0 con 0x563335b41d50 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4200110 con 0x563335b15de0 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01ec7ec740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x563335b14580 con 0x563335b5ec30 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01ec7ec740 10 client.0 _ll_register_callbacks cb 0x7f01dc017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b3b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b3b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39946/0 (socket says 172.21.15.59:39946) 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 -- 172.21.15.59:0/813262089 learned_addr learned my addr 172.21.15.59:0/813262089 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2a00640 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b15de0 0x563335b5fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 -- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 msgr2=0x563335b7e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b7e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 -- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b15de0 msgr2=0x563335b5fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b15de0 0x563335b5fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2000640 1 -- 172.21.15.59:0/813262089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333574d0f0 con 0x563335b41d50 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.717+0000 7f01e2a00640 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b15de0 0x563335b5fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01e2000640 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b3b7c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f01cc079010 tx=0x7f01cc034040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0188911953ee917 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01cbe00640 1 -- 172.21.15.59:0/813262089 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc079080 con 0x563335b41d50 2024-04-23T14:42:01.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01cbe00640 1 -- 172.21.15.59:0/813262089 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f01cc0ab0c0 con 0x563335b41d50 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01cc07d430 con 0x563335b41d50 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01cbe00640 1 -- 172.21.15.59:0/813262089 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f01cc0756f0 con 0x563335b41d50 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01ec7ec740 10 client.8886 fetch_fsmap learned FSMap version 82 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.718+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x563335a40810 con 0x563335b41d50 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01cbe00640 1 -- 172.21.15.59:0/813262089 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc07d430 con 0x563335b41d50 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01cbe00640 1 -- 172.21.15.59:0/813262089 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f01cc0a9ce0 con 0x563335b41d50 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01ec7ec740 10 client.8886 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01ec7ec740 10 client.8886 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01ec7ec740 20 client.8886 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335bd5360 0x563335b16870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01ec7ec740 4 client.8886 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:42:01.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563335a82a80 con 0x563335bd5360 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.719+0000 7f01e2a00640 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335bd5360 0x563335b16870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01e2a00640 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335bd5360 0x563335b16870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1cf301c6bbc63c server_cookie=4a50eb3ce3c58976 in_seq=0 out_seq=0 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01cbe00640 10 client.8886 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01cbe00640 1 -- 172.21.15.59:0/813262089 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x563335a82a80 con 0x563335bd5360 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01cbe00640 10 client.8886 handle_command_reply: tid=0 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 client.8886 shutdown 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 msgr2=0x563335b3b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b3b7c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f01cc079010 tx=0x7f01cc034040 comp rx=0 tx=0).stop 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 shutdown_connections 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x563335b5ec30 0x563335b7e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x563335bd5360 0x563335b16870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563335b41d50 0x563335b3b7c0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 --2- 172.21.15.59:0/813262089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563335b15de0 0x563335b5fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01cbe00640 0 client.8886 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 >> 172.21.15.59:0/813262089 conn(0x563335b41610 msgr2=0x563335b5bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 shutdown_connections 2024-04-23T14:42:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.720+0000 7f01ec7ec740 1 -- 172.21.15.59:0/813262089 wait complete. 2024-04-23T14:42:01.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01ec7ec740 20 client.8886 trim_cache size 0 max 16384 2024-04-23T14:42:01.726 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:42:01.726 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:42:01.726 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:42:01.727 INFO:teuthology.orchestra.run.smithi059.stdout: "49b75886-342a-4957-82c1-1a0d0fb346aa": { 2024-04-23T14:42:01.727 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-23T14:42:01.727 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "49b75886-342a-4957-82c1-1a0d0fb346aa", 2024-04-23T14:42:01.727 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-23T14:42:01.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:42:01.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:42:01.727 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01c006c8f0 msgr2=0x7f01c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01c006c8f0 0x7f01c006edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f01e406ad70 tx=0x7f01d4008040 comp rx=0 tx=0).stop 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40b4c70 msgr2=0x7f01e40ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40b4c70 0x7f01e40ca400 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02ad30 tx=0x7f01e4163390 comp rx=0 tx=0).stop 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 shutdown_connections 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01c006c8f0 0x7f01c006edb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01e40ca940 0x7f01e40cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40b4c70 0x7f01e40ca400 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 --2- 172.21.15.59:0/28805493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01c4004800 0x7f01e41f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 >> 172.21.15.59:0/28805493 conn(0x7f01e4067760 msgr2=0x7f01e40c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 shutdown_connections 2024-04-23T14:42:01.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:01.721+0000 7f01e9a00640 1 -- 172.21.15.59:0/28805493 wait complete. 2024-04-23T14:42:01.739 INFO:tasks.cephfs.filesystem:scrub status for tag:49b75886-342a-4957-82c1-1a0d0fb346aa - {'path': '/', 'tag': '49b75886-342a-4957-82c1-1a0d0fb346aa', 'options': 'recursive,repair'} 2024-04-23T14:42:02.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:02.315+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-23T14:42:01.470521+0000) ==== 695+0+0 (secure 0 0 0) 0x7fdf7c0781a0 con 0x7fdf8c0cd090 2024-04-23T14:42:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: cluster: 2024-04-23T14:42:02.317 INFO:teuthology.orchestra.run.smithi059.stdout: id: b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: health: HEALTH_OK 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: services: 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: mon: 3 daemons, quorum a,b,c (age 11m) 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: mgr: y(active, since 10m), standbys: x 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: mds: 1/1 daemons up, 3 standby 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: data: 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: volumes: 1/1 healthy 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: pools: 3 pools, 97 pgs 2024-04-23T14:42:02.318 INFO:teuthology.orchestra.run.smithi059.stdout: objects: 30 objects, 19 MiB 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: pgs: 97 active+clean 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: io: 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 3 op/s wr 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:01.470521+0000 mds.a [INF] scrub queued for path: / 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:01.470525+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:01.470529+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-23T14:42:02.319 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:01.472380+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-23T14:42:04.330 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:03.725432+0000 mds.a [INF] scrub summary: idle 2024-04-23T14:42:04.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:04.328+0000 7fdf83400640 1 -- 172.21.15.59:0/3829459923 <== mon.0 v2:172.21.15.59:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-23T14:42:03.725432+0000) ==== 189+0+0 (secure 0 0 0) 0x7fdf7c078400 con 0x7fdf8c0cd090 2024-04-23T14:42:21.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:42:21.518 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:42:31.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:42:31.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-23T14:42:31.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.964+0000 7fea8ac00640 1 Processor -- start 2024-04-23T14:42:31.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.964+0000 7fea8ac00640 1 -- start start 2024-04-23T14:42:31.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.965+0000 7fea8ac00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea84161c50 0x7fea840d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.965+0000 7fea8ac00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea840cfc50 0x7fea840d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.965+0000 7fea8ac00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea840d22e0 con 0x7fea840d0de0 2024-04-23T14:42:31.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.965+0000 7fea8ac00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fea840d2ba0 con 0x7fea84161c50 2024-04-23T14:42:31.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.965+0000 7fea8ac00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fea840d2550 con 0x7fea840cfc50 2024-04-23T14:42:31.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea89800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea84161c50 0x7fea840d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea89800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea84161c50 0x7fea840d5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35116/0 (socket says 172.21.15.59:35116) 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea89800640 1 -- 172.21.15.59:0/2504047470 learned_addr learned my addr 172.21.15.59:0/2504047470 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea88e00640 1 --2- 172.21.15.59:0/2504047470 >> v2:172.21.15.76:3300/0 conn(0x7fea840cfc50 0x7fea840d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea89800640 1 -- 172.21.15.59:0/2504047470 >> v2:172.21.15.76:3300/0 conn(0x7fea840cfc50 msgr2=0x7fea840d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea89800640 1 --2- 172.21.15.59:0/2504047470 >> v2:172.21.15.76:3300/0 conn(0x7fea840cfc50 0x7fea840d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea89800640 1 -- 172.21.15.59:0/2504047470 >> v1:172.21.15.76:6789/0 conn(0x7fea840d0de0 legacy=0x7fea8406cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea89800640 1 -- 172.21.15.59:0/2504047470 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea840d2830 con 0x7fea84161c50 2024-04-23T14:42:31.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.966+0000 7fea88e00640 1 --2- 172.21.15.59:0/2504047470 >> v2:172.21.15.76:3300/0 conn(0x7fea840cfc50 0x7fea840d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:31.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.967+0000 7fea89800640 1 --2- 172.21.15.59:0/2504047470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea84161c50 0x7fea840d5240 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fea840d2d20 tx=0x7fea7802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4577402e1a516526 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:31.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea83e00640 1 -- 172.21.15.59:0/2504047470 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78037070 con 0x7fea84161c50 2024-04-23T14:42:31.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea83e00640 1 -- 172.21.15.59:0/2504047470 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea78032440 con 0x7fea84161c50 2024-04-23T14:42:31.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea83e00640 1 -- 172.21.15.59:0/2504047470 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78032810 con 0x7fea84161c50 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea8ac00640 1 -- 172.21.15.59:0/2504047470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea84161c50 msgr2=0x7fea840d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea8ac00640 1 --2- 172.21.15.59:0/2504047470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea84161c50 0x7fea840d5240 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fea840d2d20 tx=0x7fea7802a210 comp rx=0 tx=0).stop 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea8ac00640 1 -- 172.21.15.59:0/2504047470 shutdown_connections 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea8ac00640 1 --2- 172.21.15.59:0/2504047470 >> v2:172.21.15.76:3300/0 conn(0x7fea840cfc50 0x7fea840d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea8ac00640 1 --2- 172.21.15.59:0/2504047470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea84161c50 0x7fea840d5240 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.968+0000 7fea8ac00640 1 -- 172.21.15.59:0/2504047470 >> 172.21.15.59:0/2504047470 conn(0x7fea8408bf90 msgr2=0x7fea840d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.969+0000 7fea8ac00640 1 -- 172.21.15.59:0/2504047470 shutdown_connections 2024-04-23T14:42:31.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.969+0000 7fea8ac00640 1 -- 172.21.15.59:0/2504047470 wait complete. 2024-04-23T14:42:31.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.970+0000 7fea8ac00640 1 Processor -- start 2024-04-23T14:42:31.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.970+0000 7fea8ac00640 1 -- start start 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.971+0000 7fea8ac00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840cfc50 0x7fea840b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.971+0000 7fea8ac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea840d0de0 0x7fea840b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.971+0000 7fea8ac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea84161c50 0x7fea840b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.971+0000 7fea8ac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea840d3570 con 0x7fea840d0de0 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.971+0000 7fea8ac00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea840d0460 con 0x7fea840cfc50 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.971+0000 7fea8ac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fea840d0090 con 0x7fea84161c50 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea840d0de0 0x7fea840b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea840d0de0 0x7fea840b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35120/0 (socket says 172.21.15.59:35120) 2024-04-23T14:42:31.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 -- 172.21.15.59:0/57827368 learned_addr learned my addr 172.21.15.59:0/57827368 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea89800640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840cfc50 0x7fea840b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 -- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea84161c50 msgr2=0x7fea840b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea8a200640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea84161c50 0x7fea840b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea84161c50 0x7fea840b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 -- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840cfc50 msgr2=0x7fea840b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840cfc50 0x7fea840b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea88e00640 1 -- 172.21.15.59:0/57827368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea840b8150 con 0x7fea840d0de0 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea89800640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840cfc50 0x7fea840b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.972+0000 7fea8a200640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea84161c50 0x7fea840b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.973+0000 7fea88e00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea840d0de0 0x7fea840b73e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c820 tx=0x7fea7400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.973+0000 7fea81600640 1 -- 172.21.15.59:0/57827368 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74015070 con 0x7fea840d0de0 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.973+0000 7fea81600640 1 -- 172.21.15.59:0/57827368 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74007a30 con 0x7fea840d0de0 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.973+0000 7fea81600640 1 -- 172.21.15.59:0/57827368 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74007e00 con 0x7fea840d0de0 2024-04-23T14:42:31.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.973+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea840b8430 con 0x7fea840d0de0 2024-04-23T14:42:31.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.973+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea840b89f0 con 0x7fea840d0de0 2024-04-23T14:42:31.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.975+0000 7fea81600640 1 -- 172.21.15.59:0/57827368 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea74018070 con 0x7fea840d0de0 2024-04-23T14:42:31.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.977+0000 7fea81600640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea6806c8f0 0x7fea6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.977+0000 7fea81600640 1 -- 172.21.15.59:0/57827368 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fea7408bb30 con 0x7fea840d0de0 2024-04-23T14:42:31.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.978+0000 7fea89800640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea6806c8f0 0x7fea6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.978+0000 7fea8effb740 1 Processor -- start 2024-04-23T14:42:31.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.978+0000 7fea8effb740 1 -- start start 2024-04-23T14:42:31.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.978+0000 7fea89800640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea6806c8f0 0x7fea6806edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fea78034000 tx=0x7fea7802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea8effb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d3ee00 0x555f75cf1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d217e0 0x555f75cf65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea8effb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f75c5b1f0 con 0x555f75d3ee00 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555f75c01350 con 0x555f75d217e0 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555f75c571b0 con 0x555f75d21f20 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea89800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d3ee00 0x555f75cf1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea89800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d3ee00 0x555f75cf1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35134/0 (socket says 172.21.15.59:35134) 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea89800640 1 -- 172.21.15.59:0/3353420512 learned_addr learned my addr 172.21.15.59:0/3353420512 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.979+0000 7fea88e00640 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea89800640 1 -- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 msgr2=0x555f75d212a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea89800640 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d212a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea89800640 1 -- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d217e0 msgr2=0x555f75cf65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:31.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea8a200640 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d217e0 0x555f75cf65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea89800640 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d217e0 0x555f75cf65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea89800640 1 -- 172.21.15.59:0/3353420512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f75c55f50 con 0x555f75d3ee00 2024-04-23T14:42:31.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea88e00640 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:31.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea89800640 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d3ee00 0x555f75cf1790 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fea7802a800 tx=0x7fea7802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7abcb7ec0ce3d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:31.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea72000640 1 -- 172.21.15.59:0/3353420512 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7804b070 con 0x555f75d3ee00 2024-04-23T14:42:31.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea72000640 1 -- 172.21.15.59:0/3353420512 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7803c460 con 0x555f75d3ee00 2024-04-23T14:42:31.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.980+0000 7fea72000640 1 -- 172.21.15.59:0/3353420512 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78037040 con 0x555f75d3ee00 2024-04-23T14:42:31.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 -- 172.21.15.59:0/3353420512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d3ee00 msgr2=0x555f75cf1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:31.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d3ee00 0x555f75cf1790 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fea7802a800 tx=0x7fea7802c000 comp rx=0 tx=0).stop 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 -- 172.21.15.59:0/3353420512 shutdown_connections 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d217e0 0x555f75cf65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 --2- 172.21.15.59:0/3353420512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d3ee00 0x555f75cf1790 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 -- 172.21.15.59:0/3353420512 >> 172.21.15.59:0/3353420512 conn(0x555f75cf3a00 msgr2=0x555f75d3c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 -- 172.21.15.59:0/3353420512 shutdown_connections 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.981+0000 7fea8effb740 1 -- 172.21.15.59:0/3353420512 wait complete. 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.982+0000 7fea8effb740 1 -- start start 2024-04-23T14:42:31.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.982+0000 7fea8effb740 1 Processor -- start 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.983+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d3b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.983+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.983+0000 7fea8effb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75d1e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.983+0000 7fea8effb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f75cf5bb0 con 0x555f75d1cbf0 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.983+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555f75c14690 con 0x555f75d3ee00 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.983+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555f75cf4fa0 con 0x555f75d21f20 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.983+0000 7fea8effb740 10 client.0 _ll_register_callbacks cb 0x555f75c50650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75d1e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75d1e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35148/0 (socket says 172.21.15.59:35148) 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 -- 172.21.15.59:0/1870298703 learned_addr learned my addr 172.21.15.59:0/1870298703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea89800640 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d3b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea88e00640 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 -- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 msgr2=0x555f75d3b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d3b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 -- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 msgr2=0x555f75d1de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 -- 172.21.15.59:0/1870298703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f75c13920 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea8a200640 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75d1e360 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00e520 tx=0x7fea7c00e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a8d1a284142203 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.984+0000 7fea66a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea66a00640 1 -- 172.21.15.59:0/1870298703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c018060 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7803c460 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea66a00640 1 -- 172.21.15.59:0/1870298703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fea7c003d80 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea66a00640 1 -- 172.21.15.59:0/1870298703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fea7c01e050 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 10 client.8898 fetch_fsmap learned FSMap version 82 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x555f75a377d0 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea66a00640 1 -- 172.21.15.59:0/1870298703 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7803c460 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea66a00640 1 -- 172.21.15.59:0/1870298703 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7fea7c021020 con 0x555f75d1cbf0 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 10 client.8898 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:42:31.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 10 client.8898 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:31.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 20 client.8898 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:31.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d247e0 0x555f75d24bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:31.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 4 client.8898 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:42:31.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.985+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f75c68c10 con 0x555f75d247e0 2024-04-23T14:42:31.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.986+0000 7fea89800640 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d247e0 0x555f75d24bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:31.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.986+0000 7fea89800640 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d247e0 0x555f75d24bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbae865acf36d922 server_cookie=ded610daf484655e in_seq=0 out_seq=0 2024-04-23T14:42:31.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.986+0000 7fea66a00640 10 client.8898 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:31.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea66a00640 1 -- 172.21.15.59:0/1870298703 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f75c68c10 con 0x555f75d247e0 2024-04-23T14:42:31.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea66a00640 10 client.8898 handle_command_reply: tid=0 2024-04-23T14:42:31.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 client.8898 shutdown 2024-04-23T14:42:31.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 msgr2=0x555f75d1e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:31.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75d1e360 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00e520 tx=0x7fea7c00e9e0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 shutdown_connections 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d247e0 0x555f75d24bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75d1e360 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 --2- 172.21.15.59:0/1870298703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d21f20 0x555f75d3b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea66a00640 0 client.8898 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.987+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 >> 172.21.15.59:0/1870298703 conn(0x555f75d217e0 msgr2=0x555f75d3f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.988+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 shutdown_connections 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.988+0000 7fea8effb740 1 -- 172.21.15.59:0/1870298703 wait complete. 2024-04-23T14:42:31.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:31.988+0000 7fea8effb740 20 client.8898 trim_cache size 0 max 16384 2024-04-23T14:42:32.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.007+0000 7fea8effb740 1 Processor -- start 2024-04-23T14:42:32.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.007+0000 7fea8effb740 1 -- start start 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.008+0000 7fea8effb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75cf7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.008+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d21f20 0x555f75cf7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.008+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d3ee00 0x555f75d4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.008+0000 7fea8effb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea84200c60 con 0x555f75d1cbf0 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.008+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea841ffce0 con 0x555f75d21f20 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.008+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fea84150920 con 0x555f75d3ee00 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.008+0000 7fea89800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75cf7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75cf7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35164/0 (socket says 172.21.15.59:35164) 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 -- 172.21.15.59:0/1224188339 learned_addr learned my addr 172.21.15.59:0/1224188339 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 -- 172.21.15.59:0/1224188339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d3ee00 msgr2=0x555f75d4c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 --2- 172.21.15.59:0/1224188339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d3ee00 0x555f75d4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 -- 172.21.15.59:0/1224188339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d21f20 msgr2=0x555f75cf7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 --2- 172.21.15.59:0/1224188339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d21f20 0x555f75cf7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 -- 172.21.15.59:0/1224188339 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7596d3f0 con 0x555f75d1cbf0 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea89800640 1 --2- 172.21.15.59:0/1224188339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75cf7250 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fea780090b0 tx=0x7fea7803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e50ce94bd2a409 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea71600640 1 -- 172.21.15.59:0/1224188339 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7804b070 con 0x555f75d1cbf0 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea71600640 1 -- 172.21.15.59:0/1224188339 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea78050400 con 0x555f75d1cbf0 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.009+0000 7fea71600640 1 -- 172.21.15.59:0/1224188339 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78050ae0 con 0x555f75d1cbf0 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 -- 172.21.15.59:0/1224188339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 msgr2=0x555f75cf7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 --2- 172.21.15.59:0/1224188339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75cf7250 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fea780090b0 tx=0x7fea7803a000 comp rx=0 tx=0).stop 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 -- 172.21.15.59:0/1224188339 shutdown_connections 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 --2- 172.21.15.59:0/1224188339 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d3ee00 0x555f75d4c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 --2- 172.21.15.59:0/1224188339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d21f20 0x555f75cf7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 --2- 172.21.15.59:0/1224188339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d1cbf0 0x555f75cf7250 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 -- 172.21.15.59:0/1224188339 >> 172.21.15.59:0/1224188339 conn(0x555f75d217e0 msgr2=0x555f75cf1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 -- 172.21.15.59:0/1224188339 shutdown_connections 2024-04-23T14:42:32.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.010+0000 7fea8effb740 1 -- 172.21.15.59:0/1224188339 wait complete. 2024-04-23T14:42:32.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.011+0000 7fea8effb740 1 -- start start 2024-04-23T14:42:32.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.011+0000 7fea8effb740 1 Processor -- start 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.012+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d1cbf0 0x555f75d5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.012+0000 7fea8effb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d21f20 0x555f75d5fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.012+0000 7fea8effb740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.012+0000 7fea8effb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f75d3ea90 con 0x555f75d21f20 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.012+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea84150f70 con 0x555f75d3ee00 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.012+0000 7fea8effb740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555f75db5e10 con 0x555f75d1cbf0 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.012+0000 7fea8effb740 10 client.0 _ll_register_callbacks cb 0x7fea7c0014a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d21f20 0x555f75d5fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d21f20 0x555f75d5fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35176/0 (socket says 172.21.15.59:35176) 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 -- 172.21.15.59:0/3782672924 learned_addr learned my addr 172.21.15.59:0/3782672924 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea89800640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d1cbf0 0x555f75d5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea8a200640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 -- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d1cbf0 msgr2=0x555f75d5f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d1cbf0 0x555f75d5f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 -- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 msgr2=0x555f75d1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea88e00640 1 -- 172.21.15.59:0/3782672924 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7592d0f0 con 0x555f75d21f20 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.013+0000 7fea8a200640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea88e00640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d21f20 0x555f75d5fca0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fea74061f00 tx=0x7fea74059850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f58b04011d4665d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea66a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea66a00640 1 -- 172.21.15.59:0/3782672924 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7401f050 con 0x555f75d21f20 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea66a00640 1 -- 172.21.15.59:0/3782672924 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fea7405ab80 con 0x555f75d21f20 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea78050400 con 0x555f75d21f20 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea66a00640 1 -- 172.21.15.59:0/3782672924 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fea74062430 con 0x555f75d21f20 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea8effb740 10 client.8904 fetch_fsmap learned FSMap version 82 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x555f75c20810 con 0x555f75d21f20 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea66a00640 1 -- 172.21.15.59:0/3782672924 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea78050400 con 0x555f75d21f20 2024-04-23T14:42:32.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.014+0000 7fea66a00640 1 -- 172.21.15.59:0/3782672924 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7fea74058ca0 con 0x555f75d21f20 2024-04-23T14:42:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea8effb740 10 client.8904 fetch_fsmap finished waiting for FSMap version 82 2024-04-23T14:42:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea8effb740 10 client.8904 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea8effb740 20 client.8904 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea8effb740 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d1cfd0 0x555f75d1dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea8effb740 4 client.8904 mds_command: new command op to 8514 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:42:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 --> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f75d4fbb0 con 0x555f75d1cfd0 2024-04-23T14:42:32.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea89800640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d1cfd0 0x555f75d1dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea89800640 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d1cfd0 0x555f75d1dc60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe8605202772644 server_cookie=e05cf81bedd982b9 in_seq=0 out_seq=0 2024-04-23T14:42:32.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.015+0000 7fea66a00640 10 client.8904 ms_handle_connect on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:32.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea66a00640 1 -- 172.21.15.59:0/3782672924 <== mds.0 v2:172.21.15.59:6835/620420969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555f75d4fbb0 con 0x555f75d1cfd0 2024-04-23T14:42:32.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea66a00640 10 client.8904 handle_command_reply: tid=0 2024-04-23T14:42:32.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 client.8904 shutdown 2024-04-23T14:42:32.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d21f20 msgr2=0x555f75d5fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d21f20 0x555f75d5fca0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fea74061f00 tx=0x7fea74059850 comp rx=0 tx=0).stop 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 shutdown_connections 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:6835/620420969,v1:172.21.15.59:6837/620420969] conn(0x555f75d1cfd0 0x555f75d1dc60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555f75d3ee00 0x555f75d1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f75d21f20 0x555f75d5fca0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 --2- 172.21.15.59:0/3782672924 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555f75d1cbf0 0x555f75d5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 >> 172.21.15.59:0/3782672924 conn(0x555f75d217e0 msgr2=0x555f75cf1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea66a00640 0 client.8904 ms_handle_reset on v2:172.21.15.59:6835/620420969 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 shutdown_connections 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 1 -- 172.21.15.59:0/3782672924 wait complete. 2024-04-23T14:42:32.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.016+0000 7fea8effb740 20 client.8904 trim_cache size 0 max 16384 2024-04-23T14:42:32.021 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:42:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:42:32.021 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:42:32.021 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:42:32.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea6806c8f0 msgr2=0x7fea6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea6806c8f0 0x7fea6806edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fea78034000 tx=0x7fea7802e000 comp rx=0 tx=0).stop 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea840d0de0 msgr2=0x7fea840b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea840d0de0 0x7fea840b73e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c820 tx=0x7fea7400cc20 comp rx=0 tx=0).stop 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 shutdown_connections 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fea6806c8f0 0x7fea6806edb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fea84161c50 0x7fea840b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea840d0de0 0x7fea840b73e0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 --2- 172.21.15.59:0/57827368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840cfc50 0x7fea840b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.017+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 >> 172.21.15.59:0/57827368 conn(0x7fea8408bf90 msgr2=0x7fea8406dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.018+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 shutdown_connections 2024-04-23T14:42:32.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.018+0000 7fea8ac00640 1 -- 172.21.15.59:0/57827368 wait complete. 2024-04-23T14:42:32.037 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:42:32.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:42:32.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.353+0000 7f814d400640 1 Processor -- start 2024-04-23T14:42:32.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.353+0000 7f814d400640 1 -- start start 2024-04-23T14:42:32.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f814d400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 0x7f814806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f814d400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81480cd880 con 0x7f81480cf410 2024-04-23T14:42:32.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f814d400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8148071ec0 con 0x7f8148072680 2024-04-23T14:42:32.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f814d400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f81480cdb20 con 0x7f814806ddf0 2024-04-23T14:42:32.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f8146800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f81480cf410 0x7f814815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56378/0 (socket says 172.21.15.59:56378) 2024-04-23T14:42:32.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f8146800640 1 -- 172.21.15.59:0/1179345219 learned_addr learned my addr 172.21.15.59:0/1179345219 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:32.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f813de00640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 0x7f814806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.355+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 500244958 0 0) 0x7f81480cd880 con 0x7f81480cf410 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f812c003520 con 0x7f81480cf410 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3366745128 0 0) 0x7f812c003520 con 0x7f81480cf410 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 msgr2=0x7f814806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 0x7f814806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 >> v1:172.21.15.76:6789/0 conn(0x7f8148072680 legacy=0x7f81480cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8148072160 con 0x7f81480cf410 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8138001670 con 0x7f81480cf410 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 >> v1:172.21.15.59:6789/0 conn(0x7f81480cf410 legacy=0x7f814815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 0x7f812c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c003fb0 0x7f812c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812c004d40 0x7f812c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81480cdb20 con 0x7f812c004d40 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f812c00bbb0 con 0x7f812c003fb0 2024-04-23T14:42:32.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f812c00bdb0 con 0x7f814806ddf0 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.356+0000 7f8147200640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812c004d40 0x7f812c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f813de00640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c003fb0 0x7f812c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f8146800640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 0x7f812c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f8147200640 1 -- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 msgr2=0x7f812c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f8147200640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 0x7f812c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f8147200640 1 -- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c003fb0 msgr2=0x7f812c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f8147200640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c003fb0 0x7f812c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f8147200640 1 -- 172.21.15.59:0/1179345219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c003520 con 0x7f812c004d40 2024-04-23T14:42:32.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.357+0000 7f8146800640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814806ddf0 0x7f812c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:32.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.358+0000 7f8147200640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812c004d40 0x7f812c0051c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f8140001e20 tx=0x7f812c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77a4083f85ceb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.358+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140001a70 con 0x7f812c004d40 2024-04-23T14:42:32.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.358+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f814002ae40 con 0x7f812c004d40 2024-04-23T14:42:32.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.358+0000 7f8145e00640 1 -- 172.21.15.59:0/1179345219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140003bb0 con 0x7f812c004d40 2024-04-23T14:42:32.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.359+0000 7f814d400640 1 -- 172.21.15.59:0/1179345219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812c004d40 msgr2=0x7f812c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.359+0000 7f814d400640 1 --2- 172.21.15.59:0/1179345219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812c004d40 0x7f812c0051c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f8140001e20 tx=0x7f812c005830 comp rx=0 tx=0).stop 2024-04-23T14:42:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.359+0000 7f8147200640 1 -- 172.21.15.59:0/1179345219 reap_dead start 2024-04-23T14:42:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.359+0000 7f814d400640 1 -- 172.21.15.59:0/1179345219 shutdown_connections 2024-04-23T14:42:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.359+0000 7f814d400640 1 -- 172.21.15.59:0/1179345219 >> 172.21.15.59:0/1179345219 conn(0x7f81480b2130 msgr2=0x7f81480b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.359+0000 7f814d400640 1 -- 172.21.15.59:0/1179345219 shutdown_connections 2024-04-23T14:42:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.359+0000 7f814d400640 1 -- 172.21.15.59:0/1179345219 wait complete. 2024-04-23T14:42:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.361+0000 7f814d400640 1 Processor -- start 2024-04-23T14:42:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.361+0000 7f814d400640 1 -- start start 2024-04-23T14:42:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f814d400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c004d40 0x7f81480b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f814d400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81481f52c0 0x7f81480b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f814d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81480b1720 0x7f8148154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f814d400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f814815a820 con 0x7f81480b1720 2024-04-23T14:42:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f814d400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f814815a6a0 con 0x7f812c004d40 2024-04-23T14:42:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f814d400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f81480cff00 con 0x7f81481f52c0 2024-04-23T14:42:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f8147200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81480b1720 0x7f8148154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f8147200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81480b1720 0x7f8148154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35182/0 (socket says 172.21.15.59:35182) 2024-04-23T14:42:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.362+0000 7f8147200640 1 -- 172.21.15.59:0/1327238345 learned_addr learned my addr 172.21.15.59:0/1327238345 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f813de00640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81481f52c0 0x7f81480b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f8147200640 1 -- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81481f52c0 msgr2=0x7f81480b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f8146800640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c004d40 0x7f81480b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f8147200640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81481f52c0 0x7f81480b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f8147200640 1 -- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c004d40 msgr2=0x7f81480b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f8147200640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c004d40 0x7f81480b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f8147200640 1 -- 172.21.15.59:0/1327238345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814815ea60 con 0x7f81480b1720 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f813de00640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81481f52c0 0x7f81480b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f8147200640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81480b1720 0x7f8148154e30 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f814002a8f0 tx=0x7f81400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f813f400640 1 -- 172.21.15.59:0/1327238345 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140007dd0 con 0x7f81480b1720 2024-04-23T14:42:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f813f400640 1 -- 172.21.15.59:0/1327238345 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8140041070 con 0x7f81480b1720 2024-04-23T14:42:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.363+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814815ed40 con 0x7f81480b1720 2024-04-23T14:42:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.364+0000 7f813f400640 1 -- 172.21.15.59:0/1327238345 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81400050d0 con 0x7f81480b1720 2024-04-23T14:42:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.364+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814815f300 con 0x7f81480b1720 2024-04-23T14:42:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.365+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8148068a80 con 0x7f81480b1720 2024-04-23T14:42:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.366+0000 7f813f400640 1 -- 172.21.15.59:0/1327238345 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f814003a070 con 0x7f81480b1720 2024-04-23T14:42:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.367+0000 7f813f400640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f812406c920 0x7f812406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.367+0000 7f813f400640 1 -- 172.21.15.59:0/1327238345 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f81400ab440 con 0x7f81480b1720 2024-04-23T14:42:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.367+0000 7f8146800640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f812406c920 0x7f812406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.368+0000 7f8146800640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f812406c920 0x7f812406ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8138001200 tx=0x7f8138009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.371+0000 7f813f400640 1 -- 172.21.15.59:0/1327238345 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81400798f0 con 0x7f81480b1720 2024-04-23T14:42:32.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.616+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f81480700b0 con 0x7f81480b1720 2024-04-23T14:42:32.619 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:42:32.618+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:42:32.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.621+0000 7f813f400640 1 -- 172.21.15.59:0/1327238345 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f81400791f0 con 0x7f81480b1720 2024-04-23T14:42:32.622 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:42:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.624+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f812406c920 msgr2=0x7f812406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.624+0000 7f814d400640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f812406c920 0x7f812406ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8138001200 tx=0x7f8138009040 comp rx=0 tx=0).stop 2024-04-23T14:42:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.624+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81480b1720 msgr2=0x7f8148154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.624+0000 7f814d400640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81480b1720 0x7f8148154e30 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f814002a8f0 tx=0x7f81400014a0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 shutdown_connections 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f812406c920 0x7f812406ede0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81481f52c0 0x7f81480b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81480b1720 0x7f8148154e30 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 --2- 172.21.15.59:0/1327238345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c004d40 0x7f81480b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 >> 172.21.15.59:0/1327238345 conn(0x7f81480b2130 msgr2=0x7f814815aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 shutdown_connections 2024-04-23T14:42:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.625+0000 7f814d400640 1 -- 172.21.15.59:0/1327238345 wait complete. 2024-04-23T14:42:32.637 INFO:tasks.ceph.mds.a.smithi059.stderr: -6365> 2024-04-23T14:41:34.545+0000 7f6a67000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:42:32.646 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-23T14:42:32.694 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:42:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.738+0000 7f44e3481280 1 -- start start 2024-04-23T14:42:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.738+0000 7f44e3481280 1 Processor -- start 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e3481280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8db90c00 0x555c8e95c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e3481280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8db91800 0x555c8e95c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e3481280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8db91c00 0x555c8e95cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e3481280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c8e96c600 con 0x555c8db91c00 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e3481280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555c8e96c480 con 0x555c8db91800 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e3481280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555c8e96c300 con 0x555c8db90c00 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8db91c00 0x555c8e95cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8db91c00 0x555c8e95cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35204/0 (socket says 172.21.15.59:35204) 2024-04-23T14:42:32.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.739+0000 7f44e2000640 1 -- 172.21.15.59:0/3127259567 learned_addr learned my addr 172.21.15.59:0/3127259567 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e2000640 1 -- 172.21.15.59:0/3127259567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8db90c00 msgr2=0x555c8e95c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e2000640 1 --2- 172.21.15.59:0/3127259567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8db90c00 0x555c8e95c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e2000640 1 -- 172.21.15.59:0/3127259567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8db91800 msgr2=0x555c8e95c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e2000640 1 --2- 172.21.15.59:0/3127259567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8db91800 0x555c8e95c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e2000640 1 -- 172.21.15.59:0/3127259567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8dbc4780 con 0x555c8db91c00 2024-04-23T14:42:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e2000640 1 --2- 172.21.15.59:0/3127259567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8db91c00 0x555c8e95cb00 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x555c8e9aea80 tx=0x555c8e92aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564092bb79c1bbc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44de400640 1 -- 172.21.15.59:0/3127259567 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555c8dbb69c0 con 0x555c8db91c00 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44de400640 1 -- 172.21.15.59:0/3127259567 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555c8dbbce00 con 0x555c8db91c00 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e3481280 1 -- 172.21.15.59:0/3127259567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8dbc54a0 con 0x555c8db91c00 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e3481280 1 -- 172.21.15.59:0/3127259567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555c8dbc4000 con 0x555c8db91c00 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44e3481280 4 waiting for MDS map... 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44de400640 1 -- 172.21.15.59:0/3127259567 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555c8dbb6680 con 0x555c8db91c00 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.740+0000 7f44de400640 1 -- 172.21.15.59:0/3127259567 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x555c8ea64200 con 0x555c8db91c00 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.741+0000 7f44de400640 1 -- 172.21.15.59:0/3127259567 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x555c8dbeb180 con 0x555c8db91c00 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.741+0000 7f44e3481280 4 Got MDS map 83 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.741+0000 7f44e3481280 10 main: JournalTool::main 2024-04-23T14:42:32.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.741+0000 7f44e3481280 4 main: JournalTool: connecting to RADOS... 2024-04-23T14:42:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.741+0000 7f44e3481280 1 Processor -- start 2024-04-23T14:42:32.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.742+0000 7f44e3481280 1 -- start start 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.742+0000 7f44e3481280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4000 0x555c8e95d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e3481280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4400 0x555c8e95d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e3481280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4800 0x555c8e95db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e3481280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c8e96c180 con 0x555c8eac4000 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e3481280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555c8dbf2480 con 0x555c8eac4800 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e3481280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555c8dbf2900 con 0x555c8eac4400 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4000 0x555c8e95d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4000 0x555c8e95d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35218/0 (socket says 172.21.15.59:35218) 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 -- 172.21.15.59:0/283597969 learned_addr learned my addr 172.21.15.59:0/283597969 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 -- 172.21.15.59:0/283597969 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4400 msgr2=0x555c8e95d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 --2- 172.21.15.59:0/283597969 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4400 0x555c8e95d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 -- 172.21.15.59:0/283597969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4800 msgr2=0x555c8e95db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 --2- 172.21.15.59:0/283597969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4800 0x555c8e95db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.743+0000 7f44e0c00640 1 -- 172.21.15.59:0/283597969 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8dbc43c0 con 0x555c8eac4000 2024-04-23T14:42:32.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.744+0000 7f44e0c00640 1 --2- 172.21.15.59:0/283597969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4000 0x555c8e95d080 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x555c8ea7a720 tx=0x555c8ea76d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b120065e985552 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.744+0000 7f44dbc00640 1 -- 172.21.15.59:0/283597969 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555c8dbb69c0 con 0x555c8eac4000 2024-04-23T14:42:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.744+0000 7f44dbc00640 1 -- 172.21.15.59:0/283597969 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555c8dbbca80 con 0x555c8eac4000 2024-04-23T14:42:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.744+0000 7f44dbc00640 1 -- 172.21.15.59:0/283597969 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555c8dbb6680 con 0x555c8eac4000 2024-04-23T14:42:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.744+0000 7f44e3481280 1 -- 172.21.15.59:0/283597969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4000 msgr2=0x555c8e95d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.744+0000 7f44e3481280 1 --2- 172.21.15.59:0/283597969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4000 0x555c8e95d080 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x555c8ea7a720 tx=0x555c8ea76d20 comp rx=0 tx=0).stop 2024-04-23T14:42:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 -- 172.21.15.59:0/283597969 shutdown_connections 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 --2- 172.21.15.59:0/283597969 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4400 0x555c8e95d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 --2- 172.21.15.59:0/283597969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4800 0x555c8e95db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 --2- 172.21.15.59:0/283597969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4000 0x555c8e95d080 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 -- 172.21.15.59:0/283597969 >> 172.21.15.59:0/283597969 conn(0x555c8e96fc00 msgr2=0x555c8dc80800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 -- 172.21.15.59:0/283597969 shutdown_connections 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 -- 172.21.15.59:0/283597969 wait complete. 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.745+0000 7f44e3481280 1 Processor -- start 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.746+0000 7f44e3481280 1 -- start start 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.746+0000 7f44e3481280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4000 0x555c8e95d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.746+0000 7f44e3481280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4400 0x555c8e95d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.746+0000 7f44e3481280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4800 0x555c8e95db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.746+0000 7f44e3481280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c8e96d380 con 0x555c8eac4800 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.746+0000 7f44e3481280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555c8e96c180 con 0x555c8eac4400 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.746+0000 7f44e3481280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555c8e96d500 con 0x555c8eac4000 2024-04-23T14:42:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4800 0x555c8e95db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4800 0x555c8e95db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35228/0 (socket says 172.21.15.59:35228) 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 -- 172.21.15.59:0/2494576141 learned_addr learned my addr 172.21.15.59:0/2494576141 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e1600640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4400 0x555c8e95d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4000 0x555c8e95d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 -- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4000 msgr2=0x555c8e95d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8eac4000 0x555c8e95d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 -- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4400 msgr2=0x555c8e95d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4400 0x555c8e95d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 -- 172.21.15.59:0/2494576141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8dbc4960 con 0x555c8eac4800 2024-04-23T14:42:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e1600640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8eac4400 0x555c8e95d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:42:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.747+0000 7f44e0c00640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4800 0x555c8e95db80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x555c8ea7b3e0 tx=0x555c8ea76d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.748+0000 7f44d9e00640 1 -- 172.21.15.59:0/2494576141 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555c8dbb64e0 con 0x555c8eac4800 2024-04-23T14:42:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.748+0000 7f44d9e00640 1 -- 172.21.15.59:0/2494576141 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555c8dbbca80 con 0x555c8eac4800 2024-04-23T14:42:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.748+0000 7f44d9e00640 1 -- 172.21.15.59:0/2494576141 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555c8dbb69c0 con 0x555c8eac4800 2024-04-23T14:42:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.748+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555c8dbc5680 con 0x555c8eac4800 2024-04-23T14:42:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.748+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8dbc5860 con 0x555c8eac4800 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.748+0000 7f44e3481280 4 main: JournalTool: resolving pool 14 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.749+0000 7f44d9e00640 1 -- 172.21.15.59:0/2494576141 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x555c8ead4c00 con 0x555c8eac4800 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44d9e00640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555c8df0c000 0x555c8e95e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44d9e00640 1 -- 172.21.15.59:0/2494576141 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x555c8ead2a00 con 0x555c8eac4800 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44e3481280 4 main: JournalTool: creating IoCtx.. 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44e3481280 4 main: Executing for rank 0 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44e3481280 4 Filtering by inode '1099511627777' 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x555c8eac4c00 0x555c8e95e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x555c8eac5000 con 0x555c8eac4c00 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.750+0000 7f44e1600640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555c8df0c000 0x555c8e95e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.751+0000 7f44e0c00640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x555c8eac4c00 0x555c8e95e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.751+0000 7f44e1600640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555c8df0c000 0x555c8e95e100 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x555c8e9af320 tx=0x555c8ea77140 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.752+0000 7f44e0c00640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x555c8eac4c00 0x555c8e95e680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.759+0000 7f44e0c00640 1 -- 172.21.15.59:0/2494576141 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x555c8dc8a000 con 0x555c8eac4c00 2024-04-23T14:42:32.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.759+0000 7f44e3481280 4 Pointer 400.00000000 is readable 2024-04-23T14:42:32.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.759+0000 7f44e3481280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-23T14:42:32.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.759+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x555c8df0d000 0x555c8e95ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.759+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x555c8df0d400 con 0x555c8df0d000 2024-04-23T14:42:32.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.760+0000 7f44e2000640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x555c8df0d000 0x555c8e95ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.761+0000 7f44e2000640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x555c8df0d000 0x555c8e95ec00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.762+0000 7f44e2000640 1 -- 172.21.15.59:0/2494576141 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x555c8dc8a500 con 0x555c8df0d000 2024-04-23T14:42:32.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.762+0000 7f44e3481280 10 Header 0x400000 0x405f0e 0x40628a 2024-04-23T14:42:32.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.762+0000 7f44e3481280 10 Starting journal scan from offset 0x405f0e 2024-04-23T14:42:32.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.762+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x555c8df0d800 con 0x555c8df0d000 2024-04-23T14:42:32.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.763+0000 7f44e2000640 1 -- 172.21.15.59:0/2494576141 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x555c8dc8a500 con 0x555c8df0d000 2024-04-23T14:42:32.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e3481280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-23T14:42:32.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e3481280 10 Parsing data, 0x37c bytes available 2024-04-23T14:42:32.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e3481280 10 Attempting decode at 0x405f0e 2024-04-23T14:42:32.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e3481280 10 Consumed 0x37c bytes 2024-04-23T14:42:32.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e3481280 10 Valid entry at 0x405f0e 2024-04-23T14:42:32.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x555c8df0dc00 0x555c8e95f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x555c8df28000 con 0x555c8df0dc00 2024-04-23T14:42:32.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.764+0000 7f44e1600640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x555c8df0dc00 0x555c8e95f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.765+0000 7f44e1600640 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x555c8df0dc00 0x555c8e95f180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e1600640 1 -- 172.21.15.59:0/2494576141 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555c8dc8a780 con 0x555c8df0dc00 2024-04-23T14:42:32.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 4 Reached end of journal objects 2024-04-23T14:42:32.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 4 Scanned objects, 0 missing, 1 valid 2024-04-23T14:42:32.770 INFO:teuthology.orchestra.run.smithi059.stdout:Events by type: 2024-04-23T14:42:32.770 INFO:teuthology.orchestra.run.smithi059.stdout:Errors: 0 2024-04-23T14:42:32.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 4 Events scanned, 0 gaps 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 4 Found 1 valid events 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 4 Selected 0 events events for processing 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 4 0 events 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x555c8df0dc00 msgr2=0x555c8e95f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x555c8df0dc00 0x555c8e95f180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x555c8df0d000 msgr2=0x555c8e95ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x555c8df0d000 0x555c8e95ec00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x555c8eac4c00 msgr2=0x555c8e95e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.768+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x555c8eac4c00 0x555c8e95e680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555c8df0c000 msgr2=0x555c8e95e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555c8df0c000 0x555c8e95e100 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x555c8e9af320 tx=0x555c8ea77140 comp rx=0 tx=0).stop 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e1600640 1 -- 172.21.15.59:0/2494576141 reap_dead start 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4800 msgr2=0x555c8e95db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4800 0x555c8e95db80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x555c8ea7b3e0 tx=0x555c8ea76d80 comp rx=0 tx=0).stop 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 shutdown_connections 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555c8df0c000 0x555c8e95e100 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 --2- 172.21.15.59:0/2494576141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8eac4800 0x555c8e95db80 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 >> 172.21.15.59:0/2494576141 conn(0x555c8e96fc00 msgr2=0x555c8dc80800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 shutdown_connections 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.769+0000 7f44e3481280 1 -- 172.21.15.59:0/2494576141 wait complete. 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 -- 172.21.15.59:0/3127259567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8db91c00 msgr2=0x555c8e95cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 --2- 172.21.15.59:0/3127259567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8db91c00 0x555c8e95cb00 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x555c8e9aea80 tx=0x555c8e92aa20 comp rx=0 tx=0).stop 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 -- 172.21.15.59:0/3127259567 shutdown_connections 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 --2- 172.21.15.59:0/3127259567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c8db91c00 0x555c8e95cb00 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 --2- 172.21.15.59:0/3127259567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c8db91800 0x555c8e95c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 --2- 172.21.15.59:0/3127259567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555c8db90c00 0x555c8e95c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 -- 172.21.15.59:0/3127259567 >> 172.21.15.59:0/3127259567 conn(0x555c8db91000 msgr2=0x555c8dc81000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 -- 172.21.15.59:0/3127259567 shutdown_connections 2024-04-23T14:42:32.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.770+0000 7f44e3481280 1 -- 172.21.15.59:0/3127259567 wait complete. 2024-04-23T14:42:32.776 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.130016, result: Events by type: Errors: 0 2024-04-23T14:42:32.777 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-23T14:42:32.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.924+0000 7fd8b8d7b280 1 -- start start 2024-04-23T14:42:32.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.924+0000 7fd8b8d7b280 1 Processor -- start 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51324800 0x55ea5131cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51325c00 0x55ea5131db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51325400 0x55ea5131d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea513ac480 con 0x55ea51324800 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ea513ac600 con 0x55ea51325c00 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ea513ac780 con 0x55ea51325400 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51324800 0x55ea5131cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51324800 0x55ea5131cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35252/0 (socket says 172.21.15.59:35252) 2024-04-23T14:42:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 -- 172.21.15.59:0/962451957 learned_addr learned my addr 172.21.15.59:0/962451957 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6400640 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51325c00 0x55ea5131db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b7800640 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51325400 0x55ea5131d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 -- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51325400 msgr2=0x55ea5131d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51325400 0x55ea5131d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 -- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51325c00 msgr2=0x55ea5131db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51325c00 0x55ea5131db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.925+0000 7fd8b6e00640 1 -- 172.21.15.59:0/962451957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea505ec780 con 0x55ea51324800 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51324800 0x55ea5131cb00 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55ea513e96e0 tx=0x55ea51354ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecbf44664f876f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b3c00640 1 -- 172.21.15.59:0/962451957 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea505de9c0 con 0x55ea51324800 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b3c00640 1 -- 172.21.15.59:0/962451957 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea505e4700 con 0x55ea51324800 2024-04-23T14:42:32.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b3c00640 1 -- 172.21.15.59:0/962451957 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea505deb60 con 0x55ea51324800 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/962451957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea505ec000 con 0x55ea51324800 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/962451957 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ea505eda40 con 0x55ea51324800 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b8d7b280 4 waiting for MDS map... 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b3c00640 1 -- 172.21.15.59:0/962451957 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x55ea5148e400 con 0x55ea51324800 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b3c00640 1 -- 172.21.15.59:0/962451957 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x55ea50613180 con 0x55ea51324800 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.926+0000 7fd8b8d7b280 4 Got MDS map 83 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.927+0000 7fd8b8d7b280 10 main: JournalTool::main 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.927+0000 7fd8b8d7b280 4 main: JournalTool: connecting to RADOS... 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.927+0000 7fd8b8d7b280 1 Processor -- start 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.927+0000 7fd8b8d7b280 1 -- start start 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.928+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.928+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 0x55ea5131e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.928+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491800 0x55ea5131e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.928+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea5061a180 con 0x55ea51491800 2024-04-23T14:42:32.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.928+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ea513acf00 con 0x55ea51491000 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.928+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ea513ad080 con 0x55ea51491400 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.928+0000 7fd8b6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491800 0x55ea5131e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491800 0x55ea5131e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35264/0 (socket says 172.21.15.59:35264) 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 -- 172.21.15.59:0/3895716930 learned_addr learned my addr 172.21.15.59:0/3895716930 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b7800640 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 -- 172.21.15.59:0/3895716930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 msgr2=0x55ea5131e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 0x55ea5131e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 -- 172.21.15.59:0/3895716930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491000 msgr2=0x55ea5131c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 -- 172.21.15.59:0/3895716930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea505ed860 con 0x55ea51491800 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b6400640 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491800 0x55ea5131e680 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x55ea514a89f0 tx=0x55ea514aac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303cce110f06f8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b1400640 1 -- 172.21.15.59:0/3895716930 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea505de9c0 con 0x55ea51491800 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b1400640 1 -- 172.21.15.59:0/3895716930 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea505e4fc0 con 0x55ea51491800 2024-04-23T14:42:32.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.929+0000 7fd8b1400640 1 -- 172.21.15.59:0/3895716930 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea505deb60 con 0x55ea51491800 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/3895716930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491800 msgr2=0x55ea5131e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491800 0x55ea5131e680 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x55ea514a89f0 tx=0x55ea514aac00 comp rx=0 tx=0).stop 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/3895716930 shutdown_connections 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 0x55ea5131e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491800 0x55ea5131e680 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/3895716930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/3895716930 >> 172.21.15.59:0/3895716930 conn(0x55ea51325800 msgr2=0x55ea5069f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/3895716930 shutdown_connections 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.930+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/3895716930 wait complete. 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.931+0000 7fd8b8d7b280 1 Processor -- start 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.931+0000 7fd8b8d7b280 1 -- start start 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 0x55ea5131e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b8d7b280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491800 0x55ea5131e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea513ad380 con 0x55ea51491000 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ea513ad200 con 0x55ea51491800 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b8d7b280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ea513ad500 con 0x55ea51491400 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b7800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b7800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35274/0 (socket says 172.21.15.59:35274) 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b7800640 1 -- 172.21.15.59:0/1683752704 learned_addr learned my addr 172.21.15.59:0/1683752704 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b6400640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491800 0x55ea5131e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 0x55ea5131e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b7800640 1 -- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 msgr2=0x55ea5131e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b7800640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51491400 0x55ea5131e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.932+0000 7fd8b7800640 1 -- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491800 msgr2=0x55ea5131e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8b7800640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51491800 0x55ea5131e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8b7800640 1 -- 172.21.15.59:0/1683752704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea505ec780 con 0x55ea51491000 2024-04-23T14:42:32.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8b7800640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491000 0x55ea5131c580 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x55ea513e92f0 tx=0x55ea51354f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8af600640 1 -- 172.21.15.59:0/1683752704 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea505de680 con 0x55ea51491000 2024-04-23T14:42:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8af600640 1 -- 172.21.15.59:0/1683752704 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea505e4c40 con 0x55ea51491000 2024-04-23T14:42:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8af600640 1 -- 172.21.15.59:0/1683752704 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea505de9c0 con 0x55ea51491000 2024-04-23T14:42:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea505ec000 con 0x55ea51491000 2024-04-23T14:42:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea505ed680 con 0x55ea51491000 2024-04-23T14:42:32.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.933+0000 7fd8b8d7b280 4 main: JournalTool: resolving pool 14 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.935+0000 7fd8af600640 1 -- 172.21.15.59:0/1683752704 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55ea514f8c00 con 0x55ea51491000 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8af600640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55ea514fec00 0x55ea5131f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8af600640 1 -- 172.21.15.59:0/1683752704 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x55ea514f6a00 con 0x55ea51491000 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8b8d7b280 4 main: JournalTool: creating IoCtx.. 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8b8d7b280 4 main: Executing for rank 0 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8b8d7b280 4 Filtering by inode '1099511627778' 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55ea51491c00 0x55ea5131ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea514ff000 con 0x55ea51491c00 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.936+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55ea514fec00 0x55ea5131f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.937+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55ea514fec00 0x55ea5131f700 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55ea513e99e0 tx=0x55ea51354c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.937+0000 7fd8b6400640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55ea51491c00 0x55ea5131ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.938+0000 7fd8b6400640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55ea51491c00 0x55ea5131ec00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.939+0000 7fd8b6400640 1 -- 172.21.15.59:0/1683752704 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55ea506b2000 con 0x55ea51491c00 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.939+0000 7fd8b8d7b280 4 Pointer 400.00000000 is readable 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.939+0000 7fd8b8d7b280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.939+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55ea514ff400 0x55ea5131f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.939+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea514ff800 con 0x55ea514ff400 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.939+0000 7fd8b7800640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55ea514ff400 0x55ea5131f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.940+0000 7fd8b7800640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55ea514ff400 0x55ea5131f180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.941+0000 7fd8b7800640 1 -- 172.21.15.59:0/1683752704 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55ea506b2500 con 0x55ea514ff400 2024-04-23T14:42:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.941+0000 7fd8b8d7b280 10 Header 0x400000 0x405f0e 0x40628a 2024-04-23T14:42:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.941+0000 7fd8b8d7b280 10 Starting journal scan from offset 0x405f0e 2024-04-23T14:42:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.941+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea514ffc00 con 0x55ea514ff400 2024-04-23T14:42:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.942+0000 7fd8b7800640 1 -- 172.21.15.59:0/1683752704 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55ea506b2500 con 0x55ea514ff400 2024-04-23T14:42:32.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.942+0000 7fd8b8d7b280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.942+0000 7fd8b8d7b280 10 Parsing data, 0x37c bytes available 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.942+0000 7fd8b8d7b280 10 Attempting decode at 0x405f0e 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.942+0000 7fd8b8d7b280 10 Consumed 0x37c bytes 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.943+0000 7fd8b8d7b280 10 Valid entry at 0x405f0e 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.943+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ea508c1000 0x55ea5094e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.943+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea508c1400 con 0x55ea508c1000 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.943+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ea508c1000 0x55ea5094e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:32.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b6e00640 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ea508c1000 0x55ea5094e000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:32.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b6e00640 1 -- 172.21.15.59:0/1683752704 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ea506b2780 con 0x55ea508c1000 2024-04-23T14:42:32.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 4 Reached end of journal objects 2024-04-23T14:42:32.946 INFO:teuthology.orchestra.run.smithi059.stdout:Events by type: 2024-04-23T14:42:32.946 INFO:teuthology.orchestra.run.smithi059.stdout:Errors: 0 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 4 Scanned objects, 0 missing, 1 valid 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 4 Events scanned, 0 gaps 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 4 Found 1 valid events 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 4 Selected 0 events events for processing 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 4 0 events 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ea508c1000 msgr2=0x55ea5094e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55ea508c1000 0x55ea5094e000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55ea514ff400 msgr2=0x55ea5131f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55ea514ff400 0x55ea5131f180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55ea51491c00 msgr2=0x55ea5131ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55ea51491c00 0x55ea5131ec00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b6400640 1 -- 172.21.15.59:0/1683752704 reap_dead start 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55ea514fec00 msgr2=0x55ea5131f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55ea514fec00 0x55ea5131f700 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x55ea513e99e0 tx=0x55ea51354c00 comp rx=0 tx=0).stop 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491000 msgr2=0x55ea5131c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491000 0x55ea5131c580 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x55ea513e92f0 tx=0x55ea51354f00 comp rx=0 tx=0).stop 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 shutdown_connections 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55ea514fec00 0x55ea5131f700 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/1683752704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51491000 0x55ea5131c580 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 >> 172.21.15.59:0/1683752704 conn(0x55ea51325800 msgr2=0x55ea5069f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.944+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 shutdown_connections 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/1683752704 wait complete. 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/962451957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51324800 msgr2=0x55ea5131cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51324800 0x55ea5131cb00 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55ea513e96e0 tx=0x55ea51354ea0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/962451957 shutdown_connections 2024-04-23T14:42:32.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ea51325400 0x55ea5131d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ea51325c00 0x55ea5131db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 --2- 172.21.15.59:0/962451957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea51324800 0x55ea5131cb00 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/962451957 >> 172.21.15.59:0/962451957 conn(0x55ea505b9000 msgr2=0x55ea5069d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/962451957 shutdown_connections 2024-04-23T14:42:32.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:32.945+0000 7fd8b8d7b280 1 -- 172.21.15.59:0/962451957 wait complete. 2024-04-23T14:42:32.952 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.175387, result: Events by type: Errors: 0 2024-04-23T14:42:32.952 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-23T14:42:33.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.045+0000 7fdd2b8b1f40 1 Processor -- start 2024-04-23T14:42:33.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.046+0000 7fdd2b8b1f40 1 -- start start 2024-04-23T14:42:33.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.046+0000 7fdd2b8b1f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd0c002850 0x564bbf344b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.046+0000 7fdd2b8b1f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564bbf3d0630 0x564bbf347d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.046+0000 7fdd2b8b1f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d4640 0x564bbf3d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.046+0000 7fdd2b8b1f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564bbf2a2cc0 con 0x564bbf3d0630 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.046+0000 7fdd2b8b1f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bbf2a6d00 con 0x7fdd0c002850 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.046+0000 7fdd2b8b1f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x564bbf2fb9c0 con 0x564bbf3d4640 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564bbf3d0630 0x564bbf347d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564bbf3d0630 0x564bbf347d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35294/0 (socket says 172.21.15.59:35294) 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 -- 172.21.15.59:0/1654538712 learned_addr learned my addr 172.21.15.59:0/1654538712 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:33.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28a00640 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd0c002850 0x564bbf344b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd29400640 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d4640 0x564bbf3d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 -- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d4640 msgr2=0x564bbf3d4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d4640 0x564bbf3d4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 -- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd0c002850 msgr2=0x564bbf344b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd0c002850 0x564bbf344b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28000640 1 -- 172.21.15.59:0/1654538712 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbf345050 con 0x564bbf3d0630 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.047+0000 7fdd28a00640 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd0c002850 0x564bbf344b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd28000640 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564bbf3d0630 0x564bbf347d50 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fdd1004e010 tx=0x7fdd10051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d615da251293d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd26c00640 1 -- 172.21.15.59:0/1654538712 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10055440 con 0x564bbf3d0630 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd26c00640 1 -- 172.21.15.59:0/1654538712 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd10055810 con 0x564bbf3d0630 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd26c00640 1 -- 172.21.15.59:0/1654538712 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10055c00 con 0x564bbf3d0630 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/1654538712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564bbf3d0630 msgr2=0x564bbf347d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564bbf3d0630 0x564bbf347d50 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fdd1004e010 tx=0x7fdd10051000 comp rx=0 tx=0).stop 2024-04-23T14:42:33.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/1654538712 shutdown_connections 2024-04-23T14:42:33.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d4640 0x564bbf3d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564bbf3d0630 0x564bbf347d50 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/1654538712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd0c002850 0x564bbf344b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/1654538712 >> 172.21.15.59:0/1654538712 conn(0x564bbf345cc0 msgr2=0x564bbf32a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:33.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.048+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/1654538712 shutdown_connections 2024-04-23T14:42:33.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.049+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/1654538712 wait complete. 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.049+0000 7fdd2b8b1f40 1 Processor -- start 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.049+0000 7fdd2b8b1f40 1 -- start start 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.050+0000 7fdd2b8b1f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d0630 0x564bbf33df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.050+0000 7fdd2b8b1f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bbf3d4640 0x564bbf32e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.050+0000 7fdd2b8b1f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c002850 0x564bbf3b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.050+0000 7fdd2b8b1f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564bbf3d0330 con 0x7fdd0c002850 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.050+0000 7fdd2b8b1f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bbf3d00e0 con 0x564bbf3d4640 2024-04-23T14:42:33.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.050+0000 7fdd2b8b1f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x564bbf3c7bd0 con 0x564bbf3d0630 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c002850 0x564bbf3b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c002850 0x564bbf3b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35300/0 (socket says 172.21.15.59:35300) 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 -- 172.21.15.59:0/2166514800 learned_addr learned my addr 172.21.15.59:0/2166514800 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd28a00640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d0630 0x564bbf33df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd28000640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bbf3d4640 0x564bbf32e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 -- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d0630 msgr2=0x564bbf33df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d0630 0x564bbf33df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 -- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bbf3d4640 msgr2=0x564bbf32e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bbf3d4640 0x564bbf32e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 -- 172.21.15.59:0/2166514800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbf32a420 con 0x7fdd0c002850 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd28000640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bbf3d4640 0x564bbf32e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd28a00640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x564bbf3d0630 0x564bbf33df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.051+0000 7fdd29400640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c002850 0x564bbf3b6d60 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c002fb0 tx=0x7fdd20006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.052+0000 7fdd25800640 1 -- 172.21.15.59:0/2166514800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20017070 con 0x7fdd0c002850 2024-04-23T14:42:33.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.052+0000 7fdd25800640 1 -- 172.21.15.59:0/2166514800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd20003450 con 0x7fdd0c002850 2024-04-23T14:42:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.052+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564bbf32dc80 con 0x7fdd0c002850 2024-04-23T14:42:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.052+0000 7fdd25800640 1 -- 172.21.15.59:0/2166514800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20003840 con 0x7fdd0c002850 2024-04-23T14:42:33.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.052+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbf32a700 con 0x7fdd0c002850 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.054+0000 7fdd25800640 1 -- 172.21.15.59:0/2166514800 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd2001a070 con 0x7fdd0c002850 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.055+0000 7fdd25800640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdd0406b2d0 0x7fdd0406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.055+0000 7fdd25800640 1 -- 172.21.15.59:0/2166514800 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fdd20089e70 con 0x7fdd0c002850 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.055+0000 7fdd28a00640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdd0406b2d0 0x7fdd0406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.055+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x564bbf3d32c0 0x564bbf3406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.055+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x564bbf340c00 con 0x564bbf3d32c0 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.056+0000 7fdd28000640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x564bbf3d32c0 0x564bbf3406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.056+0000 7fdd28a00640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdd0406b2d0 0x7fdd0406d790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdd18005fd0 tx=0x7fdd1800c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.056+0000 7fdd28000640 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x564bbf3d32c0 0x564bbf3406c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd28000640 1 -- 172.21.15.59:0/2166514800 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v100'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd100559c0 con 0x564bbf3d32c0 2024-04-23T14:42:33.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x564bbf3d32c0 msgr2=0x564bbf3406c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x564bbf3d32c0 0x564bbf3406c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdd0406b2d0 msgr2=0x7fdd0406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdd0406b2d0 0x7fdd0406d790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdd18005fd0 tx=0x7fdd1800c040 comp rx=0 tx=0).stop 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c002850 msgr2=0x564bbf3b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 --2- 172.21.15.59:0/2166514800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c002850 0x564bbf3b6d60 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c002fb0 tx=0x7fdd20006f90 comp rx=0 tx=0).stop 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd29400640 1 -- 172.21.15.59:0/2166514800 reap_dead start 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 shutdown_connections 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 >> 172.21.15.59:0/2166514800 conn(0x564bbf345cc0 msgr2=0x564bbf3448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 shutdown_connections 2024-04-23T14:42:33.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.060+0000 7fdd2b8b1f40 1 -- 172.21.15.59:0/2166514800 wait complete. 2024-04-23T14:42:33.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-23T14:42:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.266+0000 7fcafe800640 1 Processor -- start 2024-04-23T14:42:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.266+0000 7fcafe800640 1 -- start start 2024-04-23T14:42:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafe800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcaf815c940 0x7fcaf80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafe800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf80b21e0 0x7fcaf80b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafe800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaf806b9d0 0x7fcaf806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafe800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf80b2ca0 con 0x7fcaf806b9d0 2024-04-23T14:42:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafe800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf80cdaa0 con 0x7fcaf815c940 2024-04-23T14:42:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafe800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf80cdd80 con 0x7fcaf80b21e0 2024-04-23T14:42:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaf806b9d0 0x7fcaf806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafde00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcaf806b9d0 0x7fcaf806be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35314/0 (socket says 172.21.15.59:35314) 2024-04-23T14:42:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.268+0000 7fcafde00640 1 -- 172.21.15.59:0/3362000206 learned_addr learned my addr 172.21.15.59:0/3362000206 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.269+0000 7fcafd400640 1 --2- 172.21.15.59:0/3362000206 >> v2:172.21.15.76:3300/0 conn(0x7fcaf815c940 0x7fcaf80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.269+0000 7fcafca00640 1 --2- 172.21.15.59:0/3362000206 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf80b21e0 0x7fcaf80b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.269+0000 7fcafde00640 1 -- 172.21.15.59:0/3362000206 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf80b21e0 msgr2=0x7fcaf80b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.269+0000 7fcafde00640 1 --2- 172.21.15.59:0/3362000206 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf80b21e0 0x7fcaf80b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.269+0000 7fcafde00640 1 -- 172.21.15.59:0/3362000206 >> v2:172.21.15.76:3300/0 conn(0x7fcaf815c940 msgr2=0x7fcaf80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.269+0000 7fcafde00640 1 --2- 172.21.15.59:0/3362000206 >> v2:172.21.15.76:3300/0 conn(0x7fcaf815c940 0x7fcaf80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.269+0000 7fcafde00640 1 -- 172.21.15.59:0/3362000206 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf80ce060 con 0x7fcaf806b9d0 2024-04-23T14:42:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.270+0000 7fcafde00640 1 --2- 172.21.15.59:0/3362000206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf806be60 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8073b80 tx=0x7fcaf002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92783a29dc86afed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.270+0000 7fcaf7e00640 1 -- 172.21.15.59:0/3362000206 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0036070 con 0x7fcaf806b9d0 2024-04-23T14:42:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.270+0000 7fcaf7e00640 1 -- 172.21.15.59:0/3362000206 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf0031600 con 0x7fcaf806b9d0 2024-04-23T14:42:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.270+0000 7fcaf7e00640 1 -- 172.21.15.59:0/3362000206 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00319d0 con 0x7fcaf806b9d0 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 -- 172.21.15.59:0/3362000206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 msgr2=0x7fcaf806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 --2- 172.21.15.59:0/3362000206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf806be60 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8073b80 tx=0x7fcaf002a6a0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 -- 172.21.15.59:0/3362000206 shutdown_connections 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 --2- 172.21.15.59:0/3362000206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf806be60 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 --2- 172.21.15.59:0/3362000206 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf80b21e0 0x7fcaf80b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 --2- 172.21.15.59:0/3362000206 >> v2:172.21.15.76:3300/0 conn(0x7fcaf815c940 0x7fcaf80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 -- 172.21.15.59:0/3362000206 >> 172.21.15.59:0/3362000206 conn(0x7fcaf80b2f10 msgr2=0x7fcaf80b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 -- 172.21.15.59:0/3362000206 shutdown_connections 2024-04-23T14:42:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.271+0000 7fcafe800640 1 -- 172.21.15.59:0/3362000206 wait complete. 2024-04-23T14:42:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.273+0000 7fcafe800640 1 Processor -- start 2024-04-23T14:42:33.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.273+0000 7fcafe800640 1 -- start start 2024-04-23T14:42:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.274+0000 7fcafe800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf81f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.274+0000 7fcafe800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf80b21e0 0x7fcaf81f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.274+0000 7fcafe800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf815c940 0x7fcaf815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.274+0000 7fcafe800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf80b4210 con 0x7fcaf806b9d0 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.274+0000 7fcafe800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf80c4ba0 con 0x7fcaf80b21e0 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.274+0000 7fcafe800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf80b4390 con 0x7fcaf815c940 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf81f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf81f5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35320/0 (socket says 172.21.15.59:35320) 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 -- 172.21.15.59:0/148872619 learned_addr learned my addr 172.21.15.59:0/148872619 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafde00640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf815c940 0x7fcaf815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafca00640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf80b21e0 0x7fcaf81f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 -- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf815c940 msgr2=0x7fcaf815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf815c940 0x7fcaf815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 -- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf80b21e0 msgr2=0x7fcaf81f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf80b21e0 0x7fcaf81f77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafd400640 1 -- 172.21.15.59:0/148872619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf80c4e50 con 0x7fcaf806b9d0 2024-04-23T14:42:33.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.275+0000 7fcafde00640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf815c940 0x7fcaf815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:33.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.276+0000 7fcafd400640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf81f5a40 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00d620 tx=0x7fcaec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.276+0000 7fcaf5600640 1 -- 172.21.15.59:0/148872619 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00b620 con 0x7fcaf806b9d0 2024-04-23T14:42:33.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.276+0000 7fcaf5600640 1 -- 172.21.15.59:0/148872619 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec019070 con 0x7fcaf806b9d0 2024-04-23T14:42:33.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.276+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf80c5130 con 0x7fcaf806b9d0 2024-04-23T14:42:33.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.276+0000 7fcaf5600640 1 -- 172.21.15.59:0/148872619 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00ba00 con 0x7fcaf806b9d0 2024-04-23T14:42:33.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.276+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf80c56f0 con 0x7fcaf806b9d0 2024-04-23T14:42:33.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.278+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf804fd90 con 0x7fcaf806b9d0 2024-04-23T14:42:33.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.278+0000 7fcaf5600640 1 -- 172.21.15.59:0/148872619 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcaec00bba0 con 0x7fcaf806b9d0 2024-04-23T14:42:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.280+0000 7fcaf5600640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.280+0000 7fcaf5600640 1 -- 172.21.15.59:0/148872619 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fcaec08c040 con 0x7fcaf806b9d0 2024-04-23T14:42:33.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.280+0000 7fcafca00640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.281+0000 7fcafca00640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcad806c8f0 0x7fcad806edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcaf815e450 tx=0x7fcae4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.285+0000 7fcaf5600640 1 -- 172.21.15.59:0/148872619 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcaec05a400 con 0x7fcaf806b9d0 2024-04-23T14:42:33.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.530+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fcaf80c8070 con 0x7fcaf806b9d0 2024-04-23T14:42:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.624+0000 7fcaf5600640 1 -- 172.21.15.59:0/148872619 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v84) ==== 162+0+0 (secure 0 0 0) 0x7fcaec059d00 con 0x7fcaf806b9d0 2024-04-23T14:42:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-23T14:42:33.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.627+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcad806c8f0 msgr2=0x7fcad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.627+0000 7fcafe800640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcad806c8f0 0x7fcad806edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcaf815e450 tx=0x7fcae4008040 comp rx=0 tx=0).stop 2024-04-23T14:42:33.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 msgr2=0x7fcaf81f5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf81f5a40 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00d620 tx=0x7fcaec00dae0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 shutdown_connections 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcaf815c940 0x7fcaf815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaf80b21e0 0x7fcaf81f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 --2- 172.21.15.59:0/148872619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcaf806b9d0 0x7fcaf81f5a40 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 >> 172.21.15.59:0/148872619 conn(0x7fcaf80b2f10 msgr2=0x7fcaf80d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 shutdown_connections 2024-04-23T14:42:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.628+0000 7fcafe800640 1 -- 172.21.15.59:0/148872619 wait complete. 2024-04-23T14:42:33.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:42:33.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.826+0000 7ff699c00640 1 Processor -- start 2024-04-23T14:42:33.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.826+0000 7ff699c00640 1 -- start start 2024-04-23T14:42:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.827+0000 7ff699c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff69406f7c0 0x7ff6940b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.827+0000 7ff699c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.827+0000 7ff699c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69406ec60 con 0x7ff69415b840 2024-04-23T14:42:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.827+0000 7ff699c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff6940b46e0 con 0x7ff69406f7c0 2024-04-23T14:42:33.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.827+0000 7ff699c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff6940b49c0 con 0x7ff69406df00 2024-04-23T14:42:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.828+0000 7ff693400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.828+0000 7ff692a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff69406f7c0 0x7ff6940b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.828+0000 7ff693400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69406e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:38118/0 (socket says 172.21.15.59:38118) 2024-04-23T14:42:33.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.828+0000 7ff693400640 1 -- 172.21.15.59:0/1377567947 learned_addr learned my addr 172.21.15.59:0/1377567947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.828+0000 7ff693400640 1 -- 172.21.15.59:0/1377567947 >> v2:172.21.15.76:3300/0 conn(0x7ff69406f7c0 msgr2=0x7ff6940b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.828+0000 7ff693400640 1 --2- 172.21.15.59:0/1377567947 >> v2:172.21.15.76:3300/0 conn(0x7ff69406f7c0 0x7ff6940b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.828+0000 7ff693400640 1 -- 172.21.15.59:0/1377567947 >> v1:172.21.15.76:6789/0 conn(0x7ff69415b840 legacy=0x7ff69406fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:33.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.829+0000 7ff693400640 1 -- 172.21.15.59:0/1377567947 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6940b4ca0 con 0x7ff69406df00 2024-04-23T14:42:33.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.830+0000 7ff693400640 1 --2- 172.21.15.59:0/1377567947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69406e390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009e50 tx=0x7ff68c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7409b898844724da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff691600640 1 -- 172.21.15.59:0/1377567947 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c036070 con 0x7ff69406df00 2024-04-23T14:42:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff691600640 1 -- 172.21.15.59:0/1377567947 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c031600 con 0x7ff69406df00 2024-04-23T14:42:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff691600640 1 -- 172.21.15.59:0/1377567947 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0319d0 con 0x7ff69406df00 2024-04-23T14:42:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 -- 172.21.15.59:0/1377567947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 msgr2=0x7ff69406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 --2- 172.21.15.59:0/1377567947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69406e390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009e50 tx=0x7ff68c02a6c0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 -- 172.21.15.59:0/1377567947 shutdown_connections 2024-04-23T14:42:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 --2- 172.21.15.59:0/1377567947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69406e390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 --2- 172.21.15.59:0/1377567947 >> v2:172.21.15.76:3300/0 conn(0x7ff69406f7c0 0x7ff6940b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 -- 172.21.15.59:0/1377567947 >> 172.21.15.59:0/1377567947 conn(0x7ff694067ea0 msgr2=0x7ff69406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 -- 172.21.15.59:0/1377567947 shutdown_connections 2024-04-23T14:42:33.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.831+0000 7ff699c00640 1 -- 172.21.15.59:0/1377567947 wait complete. 2024-04-23T14:42:33.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.833+0000 7ff699c00640 1 Processor -- start 2024-04-23T14:42:33.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.833+0000 7ff699c00640 1 -- start start 2024-04-23T14:42:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.834+0000 7ff699c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.834+0000 7ff699c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff69406f7c0 0x7ff69415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.834+0000 7ff699c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6941461a0 0x7ff6940ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.834+0000 7ff699c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6940ce1c0 con 0x7ff69406f7c0 2024-04-23T14:42:33.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.834+0000 7ff699c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff69406e8d0 con 0x7ff6941461a0 2024-04-23T14:42:33.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.834+0000 7ff699c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff6940ce340 con 0x7ff69406df00 2024-04-23T14:42:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff69406f7c0 0x7ff69415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69414e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:38128/0 (socket says 172.21.15.59:38128) 2024-04-23T14:42:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692a00640 1 -- 172.21.15.59:0/1370175222 learned_addr learned my addr 172.21.15.59:0/1370175222 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff693400640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6941461a0 0x7ff6940ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692000640 1 -- 172.21.15.59:0/1370175222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 msgr2=0x7ff69414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692000640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692000640 1 -- 172.21.15.59:0/1370175222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6941461a0 msgr2=0x7ff6940ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692000640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6941461a0 0x7ff6940ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.835+0000 7ff692000640 1 -- 172.21.15.59:0/1370175222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69415b060 con 0x7ff69406f7c0 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.836+0000 7ff692000640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff69406f7c0 0x7ff69415ab20 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff68000b3e0 tx=0x7ff68000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.836+0000 7ff68aa00640 1 -- 172.21.15.59:0/1370175222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680015030 con 0x7ff69406f7c0 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.836+0000 7ff68aa00640 1 -- 172.21.15.59:0/1370175222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff680004020 con 0x7ff69406f7c0 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.836+0000 7ff68aa00640 1 -- 172.21.15.59:0/1370175222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680004450 con 0x7ff69406f7c0 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.836+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69414efe0 con 0x7ff69406f7c0 2024-04-23T14:42:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.836+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6940ada10 con 0x7ff69406f7c0 2024-04-23T14:42:33.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.838+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff694068a80 con 0x7ff69406f7c0 2024-04-23T14:42:33.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.839+0000 7ff68aa00640 1 -- 172.21.15.59:0/1370175222 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff680018070 con 0x7ff69406f7c0 2024-04-23T14:42:33.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.840+0000 7ff68aa00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff67406c8f0 0x7ff67406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:33.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.840+0000 7ff68aa00640 1 -- 172.21.15.59:0/1370175222 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7ff68008b580 con 0x7ff69406f7c0 2024-04-23T14:42:33.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.840+0000 7ff692a00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff67406c8f0 0x7ff67406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.841+0000 7ff692a00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff67406c8f0 0x7ff67406edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff684009c00 tx=0x7ff684009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:33.845+0000 7ff68aa00640 1 -- 172.21.15.59:0/1370175222 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff680059260 con 0x7ff69406f7c0 2024-04-23T14:42:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.092+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6940b4270 con 0x7ff69406f7c0 2024-04-23T14:42:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.093+0000 7ff68aa00640 1 -- 172.21.15.59:0/1370175222 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5178 (secure 0 0 0) 0x7ff68005d560 con 0x7ff69406f7c0 2024-04-23T14:42:34.094 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:34.094 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:42:33.630548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/2265807893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2265807893},{"type":"v1","addr":"172.21.15.59:6837","nonce":2265807893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-23T14:42:34.094 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 85 2024-04-23T14:42:34.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.095+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff67406c8f0 msgr2=0x7ff67406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:34.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.095+0000 7ff699c00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff67406c8f0 0x7ff67406edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff684009c00 tx=0x7ff684009190 comp rx=0 tx=0).stop 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff69406f7c0 msgr2=0x7ff69415ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff69406f7c0 0x7ff69415ab20 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff68000b3e0 tx=0x7ff68000b8a0 comp rx=0 tx=0).stop 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 shutdown_connections 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff67406c8f0 0x7ff67406edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6941461a0 0x7ff6940ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff69406f7c0 0x7ff69415ab20 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 --2- 172.21.15.59:0/1370175222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff69406df00 0x7ff69414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 >> 172.21.15.59:0/1370175222 conn(0x7ff694067ea0 msgr2=0x7ff6940cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 shutdown_connections 2024-04-23T14:42:34.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:34.096+0000 7ff699c00640 1 -- 172.21.15.59:0/1370175222 wait complete. 2024-04-23T14:42:34.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:41:29.612586+0000', 'modified': '2024-04-23T14:42:33.630548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8916}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8916': {'gid': 8916, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/2265807893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2265807893}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2265807893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:42:34.520 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8916:up:replay 2024-04-23T14:42:35.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:42:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.744+0000 7f46e7e00640 1 Processor -- start 2024-04-23T14:42:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.744+0000 7f46e7e00640 1 -- start start 2024-04-23T14:42:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.746+0000 7f46e7e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f46e80d14a0 0x7f46e80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.746+0000 7f46e7e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e80cf030 con 0x7f46e815e7c0 2024-04-23T14:42:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.746+0000 7f46e7e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e80b6a40 con 0x7f46e80b5ff0 2024-04-23T14:42:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.746+0000 7f46e7e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f46e80b7000 con 0x7f46e80d14a0 2024-04-23T14:42:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.746+0000 7f46e6a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f46e815e7c0 0x7f46e80d20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56402/0 (socket says 172.21.15.59:56402) 2024-04-23T14:42:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.746+0000 7f46e6a00640 1 -- 172.21.15.59:0/1048473862 learned_addr learned my addr 172.21.15.59:0/1048473862 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e7400640 1 --2- 172.21.15.59:0/1048473862 >> v2:172.21.15.76:3300/0 conn(0x7f46e80d14a0 0x7f46e80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911900186 0 0) 0x7f46d8001490 con 0x7f46e815e7c0 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46c8003520 con 0x7f46e815e7c0 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4115365015 0 0) 0x7f46c8003520 con 0x7f46e815e7c0 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 >> v2:172.21.15.76:3300/0 conn(0x7f46e80d14a0 msgr2=0x7f46e80b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 --2- 172.21.15.59:0/1048473862 >> v2:172.21.15.76:3300/0 conn(0x7f46e80d14a0 0x7f46e80b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 >> v1:172.21.15.76:6789/0 conn(0x7f46e80b5ff0 legacy=0x7f46e80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e80b6d20 con 0x7f46e815e7c0 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f46d8001bc0 con 0x7f46e815e7c0 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 >> v1:172.21.15.59:6789/0 conn(0x7f46e815e7c0 legacy=0x7f46e80d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46c8003a00 0x7f46c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c8004800 0x7f46c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c80051c0 0x7f46c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b7000 con 0x7f46c8004800 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f46c800c0c0 con 0x7f46c8003a00 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.747+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f46c800c2c0 con 0x7f46c80051c0 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e6000640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c8004800 0x7f46c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e6a00640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46c8003a00 0x7f46c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e7400640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c80051c0 0x7f46c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e6000640 1 -- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c80051c0 msgr2=0x7f46c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e6000640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c80051c0 0x7f46c800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e6000640 1 -- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46c8003a00 msgr2=0x7f46c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e6000640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46c8003a00 0x7f46c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e6000640 1 -- 172.21.15.59:0/1048473862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c8003520 con 0x7f46c8004800 2024-04-23T14:42:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.748+0000 7f46e7400640 1 -- 172.21.15.59:0/1048473862 reap_dead start 2024-04-23T14:42:35.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.749+0000 7f46e6000640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c8004800 0x7f46c8004c80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f46d0001750 tx=0x7f46c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac99d2681b24c84c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.749+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0001370 con 0x7f46c8004800 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.749+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d00052d0 con 0x7f46c8004800 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.749+0000 7f46e5600640 1 -- 172.21.15.59:0/1048473862 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d002abc0 con 0x7f46c8004800 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.750+0000 7f46e7e00640 1 -- 172.21.15.59:0/1048473862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c8004800 msgr2=0x7f46c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.750+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c8004800 0x7f46c8004c80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f46d0001750 tx=0x7f46c8005830 comp rx=0 tx=0).stop 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.750+0000 7f46e7e00640 1 -- 172.21.15.59:0/1048473862 shutdown_connections 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.750+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1048473862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c8004800 0x7f46c8004c80 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.750+0000 7f46e7e00640 1 -- 172.21.15.59:0/1048473862 >> 172.21.15.59:0/1048473862 conn(0x7f46e809d920 msgr2=0x7f46e808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.750+0000 7f46e7e00640 1 -- 172.21.15.59:0/1048473862 shutdown_connections 2024-04-23T14:42:35.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.750+0000 7f46e7e00640 1 -- 172.21.15.59:0/1048473862 wait complete. 2024-04-23T14:42:35.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.752+0000 7f46e7e00640 1 Processor -- start 2024-04-23T14:42:35.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.752+0000 7f46e7e00640 1 -- start start 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.753+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c8004800 0x7f46e8152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.753+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80b20b0 0x7f46e80cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.753+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e80cc540 0x7f46e81f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.753+0000 7f46e7e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80d1a80 con 0x7f46e80cc540 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.753+0000 7f46e7e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80cf620 con 0x7f46e80b20b0 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.753+0000 7f46e7e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f46e80d1c00 con 0x7f46c8004800 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e80cc540 0x7f46e81f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e80cc540 0x7f46e81f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35348/0 (socket says 172.21.15.59:35348) 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 -- 172.21.15.59:0/1093354603 learned_addr learned my addr 172.21.15.59:0/1093354603 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e6a00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c8004800 0x7f46e8152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e6000640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80b20b0 0x7f46e80cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 -- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c8004800 msgr2=0x7f46e8152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c8004800 0x7f46e8152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 -- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80b20b0 msgr2=0x7f46e80cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80b20b0 0x7f46e80cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 -- 172.21.15.59:0/1093354603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e80b4210 con 0x7f46e80cc540 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.754+0000 7f46e7400640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e80cc540 0x7f46e81f86d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f46e80a24f0 tx=0x7f46e0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.755+0000 7f46dea00640 1 -- 172.21.15.59:0/1093354603 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000f040 con 0x7f46e80cc540 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.755+0000 7f46dea00640 1 -- 172.21.15.59:0/1093354603 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e001c070 con 0x7f46e80cc540 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.755+0000 7f46dea00640 1 -- 172.21.15.59:0/1093354603 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000eaa0 con 0x7f46e80cc540 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.755+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e80ccb70 con 0x7f46e80cc540 2024-04-23T14:42:35.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.755+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e80b4760 con 0x7f46e80cc540 2024-04-23T14:42:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.756+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e80b6600 con 0x7f46e80cc540 2024-04-23T14:42:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.757+0000 7f46dea00640 1 -- 172.21.15.59:0/1093354603 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46e000ec40 con 0x7f46e80cc540 2024-04-23T14:42:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.758+0000 7f46dea00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46c406c8f0 0x7f46c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.758+0000 7f46dea00640 1 -- 172.21.15.59:0/1093354603 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f46e008bb30 con 0x7f46e80cc540 2024-04-23T14:42:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.759+0000 7f46e6a00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46c406c8f0 0x7f46c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.760+0000 7f46e6a00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46c406c8f0 0x7f46c406edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f46d80013d0 tx=0x7f46d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:35.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:35.763+0000 7f46dea00640 1 -- 172.21.15.59:0/1093354603 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46e80b6600 con 0x7f46e80cc540 2024-04-23T14:42:36.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.010+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46e8069480 con 0x7f46e80cc540 2024-04-23T14:42:36.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.011+0000 7f46dea00640 1 -- 172.21.15.59:0/1093354603 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5178 (secure 0 0 0) 0x7f46e8069480 con 0x7f46e80cc540 2024-04-23T14:42:36.081 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:36.081 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:42:34.644453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.59:6837/2265807893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2265807893},{"type":"v1","addr":"172.21.15.59:6837","nonce":2265807893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 87 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46c406c8f0 msgr2=0x7f46c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46c406c8f0 0x7f46c406edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f46d80013d0 tx=0x7f46d8009040 comp rx=0 tx=0).stop 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e80cc540 msgr2=0x7f46e81f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e80cc540 0x7f46e81f86d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f46e80a24f0 tx=0x7f46e0010c20 comp rx=0 tx=0).stop 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 shutdown_connections 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f46c406c8f0 0x7f46c406edb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46e80cc540 0x7f46e81f86d0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:36.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46e80b20b0 0x7f46e80cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 --2- 172.21.15.59:0/1093354603 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f46c8004800 0x7f46e8152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 >> 172.21.15.59:0/1093354603 conn(0x7f46e809d920 msgr2=0x7f46e8161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 shutdown_connections 2024-04-23T14:42:36.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:36.014+0000 7f46e7e00640 1 -- 172.21.15.59:0/1093354603 wait complete. 2024-04-23T14:42:36.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:41:29.612586+0000', 'modified': '2024-04-23T14:42:34.644453+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8916}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8916': {'gid': 8916, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.59:6837/2265807893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2265807893}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2265807893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:42:36.488 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8916:up:rejoin 2024-04-23T14:42:37.489 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:42:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.712+0000 7fae0ac00640 1 Processor -- start 2024-04-23T14:42:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.712+0000 7fae0ac00640 1 -- start start 2024-04-23T14:42:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.713+0000 7fae0ac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 0x7fae041696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.713+0000 7fae0ac00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae040cea70 con 0x7fae0415ed60 2024-04-23T14:42:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.713+0000 7fae0ac00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0406bc00 con 0x7fae040b24b0 2024-04-23T14:42:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.713+0000 7fae0ac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae0406b9d0 con 0x7fae040cbb90 2024-04-23T14:42:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.714+0000 7fae03400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fae0415ed60 0x7fae040b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56410/0 (socket says 172.21.15.59:56410) 2024-04-23T14:42:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.714+0000 7fae03400640 1 -- 172.21.15.59:0/3403102049 learned_addr learned my addr 172.21.15.59:0/3403102049 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.714+0000 7fae08c00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 0x7fae041696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.714+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 24883519 0 0) 0x7fae040cea70 con 0x7fae0415ed60 2024-04-23T14:42:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.714+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fade4003520 con 0x7fae0415ed60 2024-04-23T14:42:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 420642346 0 0) 0x7fade4003520 con 0x7fae0415ed60 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 msgr2=0x7fae041696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 0x7fae041696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 >> v1:172.21.15.76:6789/0 conn(0x7fae040b24b0 legacy=0x7fae040b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0406bee0 con 0x7fae0415ed60 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fadec001670 con 0x7fae0415ed60 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 >> v1:172.21.15.59:6789/0 conn(0x7fae0415ed60 legacy=0x7fae040b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039e0 0x7fade4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 0x7fade4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 0x7fade40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fade400ba70 con 0x7fade40039e0 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fade400bc20 con 0x7fade4004d40 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fade400be20 con 0x7fae040cbb90 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.715+0000 7fae03e00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039e0 0x7fade4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.716+0000 7fae03400640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 0x7fade4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.716+0000 7fae08c00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 0x7fade40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.716+0000 7fae03e00640 1 -- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 msgr2=0x7fade4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.716+0000 7fae03e00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae040cbb90 0x7fade4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.716+0000 7fae03e00640 1 -- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 msgr2=0x7fade40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.716+0000 7fae03e00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4004d40 0x7fade40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.716+0000 7fae03e00640 1 -- 172.21.15.59:0/3403102049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade4003520 con 0x7fade40039e0 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.717+0000 7fae03e00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039e0 0x7fade4003e00 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fadf4001720 tx=0x7fade4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7688dd19ccb5f959 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.717+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4001370 con 0x7fade40039e0 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.717+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf40052d0 con 0x7fade40039e0 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.717+0000 7fae02a00640 1 -- 172.21.15.59:0/3403102049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf402abc0 con 0x7fade40039e0 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.718+0000 7fae0ac00640 1 -- 172.21.15.59:0/3403102049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039e0 msgr2=0x7fade4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.718+0000 7fae0ac00640 1 --2- 172.21.15.59:0/3403102049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40039e0 0x7fade4003e00 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fadf4001720 tx=0x7fade4005830 comp rx=0 tx=0).stop 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.718+0000 7fae08c00640 1 -- 172.21.15.59:0/3403102049 reap_dead start 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.718+0000 7fae0ac00640 1 -- 172.21.15.59:0/3403102049 shutdown_connections 2024-04-23T14:42:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.718+0000 7fae0ac00640 1 -- 172.21.15.59:0/3403102049 >> 172.21.15.59:0/3403102049 conn(0x7fae04073450 msgr2=0x7fae0409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:37.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.718+0000 7fae0ac00640 1 -- 172.21.15.59:0/3403102049 shutdown_connections 2024-04-23T14:42:37.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.718+0000 7fae0ac00640 1 -- 172.21.15.59:0/3403102049 wait complete. 2024-04-23T14:42:37.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.720+0000 7fae0ac00640 1 Processor -- start 2024-04-23T14:42:37.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.720+0000 7fae0ac00640 1 -- start start 2024-04-23T14:42:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae0ac00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039e0 0x7fae0415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae0ac00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae04155230 0x7fae040b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae0ac00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae04154650 0x7fae04154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae0ac00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae0406c250 con 0x7fae04154650 2024-04-23T14:42:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae0ac00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae040ce720 con 0x7fae04155230 2024-04-23T14:42:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae0ac00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae041676d0 con 0x7fade40039e0 2024-04-23T14:42:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae08c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae04154650 0x7fae04154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae08c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae04154650 0x7fae04154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35374/0 (socket says 172.21.15.59:35374) 2024-04-23T14:42:37.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae08c00640 1 -- 172.21.15.59:0/1901734668 learned_addr learned my addr 172.21.15.59:0/1901734668 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:37.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae03e00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039e0 0x7fae0415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae03400640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae04155230 0x7fae040b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae08c00640 1 -- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039e0 msgr2=0x7fae0415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae08c00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039e0 0x7fae0415d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.721+0000 7fae08c00640 1 -- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae04155230 msgr2=0x7fae040b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fae08c00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae04155230 0x7fae040b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fae08c00640 1 -- 172.21.15.59:0/1901734668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae040c9a40 con 0x7fae04154650 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fae03400640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae04155230 0x7fae040b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fae08c00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae04154650 0x7fae04154ad0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fae040cda50 tx=0x7fadfc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:37.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fadfbe00640 1 -- 172.21.15.59:0/1901734668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc01c070 con 0x7fae04154650 2024-04-23T14:42:37.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fadfbe00640 1 -- 172.21.15.59:0/1901734668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc017400 con 0x7fae04154650 2024-04-23T14:42:37.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fadfbe00640 1 -- 172.21.15.59:0/1901734668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0177f0 con 0x7fae04154650 2024-04-23T14:42:37.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae04155010 con 0x7fae04154650 2024-04-23T14:42:37.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.722+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0415bad0 con 0x7fae04154650 2024-04-23T14:42:37.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.724+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae04069190 con 0x7fae04154650 2024-04-23T14:42:37.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.725+0000 7fadfbe00640 1 -- 172.21.15.59:0/1901734668 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadfc005d10 con 0x7fae04154650 2024-04-23T14:42:37.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.725+0000 7fadfbe00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.726+0000 7fadfbe00640 1 -- 172.21.15.59:0/1901734668 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fadfc08bfe0 con 0x7fae04154650 2024-04-23T14:42:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.726+0000 7fae03e00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.727+0000 7fae03e00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fadf4032000 tx=0x7fadf402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:37.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.731+0000 7fadfbe00640 1 -- 172.21.15.59:0/1901734668 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fadfc012070 con 0x7fae04154650 2024-04-23T14:42:37.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.979+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae0406def0 con 0x7fae04154650 2024-04-23T14:42:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.980+0000 7fadfbe00640 1 -- 172.21.15.59:0/1901734668 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5185 (secure 0 0 0) 0x7fadfc05a330 con 0x7fae04154650 2024-04-23T14:42:37.980 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:37.980 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:42:36.642216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/2265807893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":2265807893},{"type":"v1","addr":"172.21.15.59:6837","nonce":2265807893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8916,"qdb_cluster":[8916]},"id":7}]} 2024-04-23T14:42:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 88 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 msgr2=0x7fade006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fadf4032000 tx=0x7fadf402e000 comp rx=0 tx=0).stop 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae04154650 msgr2=0x7fae04154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae04154650 0x7fae04154ad0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fae040cda50 tx=0x7fadfc00f4a0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 shutdown_connections 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fade006c8f0 0x7fade006edb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae04154650 0x7fae04154ad0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae04155230 0x7fae040b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 --2- 172.21.15.59:0/1901734668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fade40039e0 0x7fae0415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:37.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.982+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 >> 172.21.15.59:0/1901734668 conn(0x7fae04073450 msgr2=0x7fae0406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:37.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.983+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 shutdown_connections 2024-04-23T14:42:37.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:37.983+0000 7fae0ac00640 1 -- 172.21.15.59:0/1901734668 wait complete. 2024-04-23T14:42:38.405 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:41:29.612586+0000', 'modified': '2024-04-23T14:42:36.642216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8916}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8916': {'gid': 8916, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6837/2265807893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6835', 'nonce': 2265807893}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2265807893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8916, 'qdb_cluster': [8916]} 2024-04-23T14:42:38.405 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:42:38.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-23T14:42:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.627+0000 7fcdd6400640 1 Processor -- start 2024-04-23T14:42:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.627+0000 7fcdd6400640 1 -- start start 2024-04-23T14:42:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.629+0000 7fcdd6400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcdd006ddf0 0x7fcdd006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.629+0000 7fcdd6400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd0072680 0x7fcdd0072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.629+0000 7fcdd6400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd00cd880 con 0x7fcdd00cf410 2024-04-23T14:42:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.629+0000 7fcdd6400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd0071ec0 con 0x7fcdd006ddf0 2024-04-23T14:42:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.629+0000 7fcdd6400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd00cdb20 con 0x7fcdd0072680 2024-04-23T14:42:38.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.630+0000 7fcdc6a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcdd006ddf0 0x7fcdd006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.630+0000 7fcdcf400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcdd00cf410 0x7fcdd015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:49320/0 (socket says 172.21.15.59:49320) 2024-04-23T14:42:38.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.630+0000 7fcdcfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd0072680 0x7fcdd0072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.630+0000 7fcdcf400640 1 -- 172.21.15.59:0/3410258533 learned_addr learned my addr 172.21.15.59:0/3410258533 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:38.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.631+0000 7fcdcea00640 1 -- 172.21.15.59:0/3410258533 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975752148 0 0) 0x7fcdd00cd880 con 0x7fcdd00cf410 2024-04-23T14:42:38.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.631+0000 7fcdcfe00640 1 -- 172.21.15.59:0/3410258533 >> v2:172.21.15.76:3300/0 conn(0x7fcdd006ddf0 msgr2=0x7fcdd006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.631+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/3410258533 >> v2:172.21.15.76:3300/0 conn(0x7fcdd006ddf0 0x7fcdd006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.631+0000 7fcdcfe00640 1 -- 172.21.15.59:0/3410258533 >> v1:172.21.15.76:6789/0 conn(0x7fcdd00cf410 legacy=0x7fcdd015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.631+0000 7fcdcfe00640 1 -- 172.21.15.59:0/3410258533 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd0072160 con 0x7fcdd0072680 2024-04-23T14:42:38.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.631+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/3410258533 >> v2:172.21.15.76:3300/0 conn(0x7fcdd006ddf0 0x7fcdd006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:42:38.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.632+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/3410258533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd0072680 0x7fcdd0072b10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80097a0 tx=0x7fcdc802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d24e0b78bb19de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.632+0000 7fcdcea00640 1 -- 172.21.15.59:0/3410258533 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8037070 con 0x7fcdd0072680 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdcea00640 1 -- 172.21.15.59:0/3410258533 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc802ae40 con 0x7fcdd0072680 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdcea00640 1 -- 172.21.15.59:0/3410258533 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8032660 con 0x7fcdd0072680 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 -- 172.21.15.59:0/3410258533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd0072680 msgr2=0x7fcdd0072b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 --2- 172.21.15.59:0/3410258533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd0072680 0x7fcdd0072b10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80097a0 tx=0x7fcdc802a0f0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 -- 172.21.15.59:0/3410258533 shutdown_connections 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 --2- 172.21.15.59:0/3410258533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd0072680 0x7fcdd0072b10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 --2- 172.21.15.59:0/3410258533 >> v2:172.21.15.76:3300/0 conn(0x7fcdd006ddf0 0x7fcdd006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 -- 172.21.15.59:0/3410258533 >> 172.21.15.59:0/3410258533 conn(0x7fcdd00b2130 msgr2=0x7fcdd00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 -- 172.21.15.59:0/3410258533 shutdown_connections 2024-04-23T14:42:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.633+0000 7fcdd6400640 1 -- 172.21.15.59:0/3410258533 wait complete. 2024-04-23T14:42:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.635+0000 7fcdd6400640 1 Processor -- start 2024-04-23T14:42:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.635+0000 7fcdd6400640 1 -- start start 2024-04-23T14:42:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.636+0000 7fcdd6400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd006ddf0 0x7fcdd014c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.636+0000 7fcdd6400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd0072680 0x7fcdd00b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.636+0000 7fcdd6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd00cf410 0x7fcdd00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.636+0000 7fcdd6400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd015a820 con 0x7fcdd00cf410 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.636+0000 7fcdd6400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd015a6a0 con 0x7fcdd0072680 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.636+0000 7fcdd6400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd00cf7f0 con 0x7fcdd006ddf0 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd00cf410 0x7fcdd00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd00cf410 0x7fcdd00b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35386/0 (socket says 172.21.15.59:35386) 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 -- 172.21.15.59:0/610245724 learned_addr learned my addr 172.21.15.59:0/610245724 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcf400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd006ddf0 0x7fcdd014c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd0072680 0x7fcdd00b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 -- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd006ddf0 msgr2=0x7fcdd014c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd006ddf0 0x7fcdd014c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 -- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd0072680 msgr2=0x7fcdd00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd0072680 0x7fcdd00b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.637+0000 7fcdcfe00640 1 -- 172.21.15.59:0/610245724 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd0157d30 con 0x7fcdd00cf410 2024-04-23T14:42:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.638+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd00cf410 0x7fcdd00b48a0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8035040 tx=0x7fcdc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.638+0000 7fcdc7e00640 1 -- 172.21.15.59:0/610245724 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8037040 con 0x7fcdd00cf410 2024-04-23T14:42:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.638+0000 7fcdc7e00640 1 -- 172.21.15.59:0/610245724 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc8004030 con 0x7fcdd00cf410 2024-04-23T14:42:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.638+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd0158010 con 0x7fcdd00cf410 2024-04-23T14:42:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.638+0000 7fcdc7e00640 1 -- 172.21.15.59:0/610245724 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8004420 con 0x7fcdd00cf410 2024-04-23T14:42:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.638+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd01585d0 con 0x7fcdd00cf410 2024-04-23T14:42:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.640+0000 7fcdc7e00640 1 -- 172.21.15.59:0/610245724 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdc8003610 con 0x7fcdd00cf410 2024-04-23T14:42:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.641+0000 7fcdc7e00640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcdac06c8f0 0x7fcdac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.642 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:42:38.642+0000 7ff6eb600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:42:38.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.642+0000 7fcdc7e00640 1 -- 172.21.15.59:0/610245724 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fcdc80ac8a0 con 0x7fcdd00cf410 2024-04-23T14:42:38.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.642+0000 7fcdcf400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcdac06c8f0 0x7fcdac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.642+0000 7fcdd916b740 1 Processor -- start 2024-04-23T14:42:38.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.642+0000 7fcdd916b740 1 -- start start 2024-04-23T14:42:38.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdcf400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcdac06c8f0 0x7fcdac06edb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0007ed0 tx=0x7fcdc0007e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04befe40 0x557f04bf0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bed430 0x557f04bc0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bc73a0 0x557f04bc4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdd916b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f04b2ab10 con 0x557f04bc73a0 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557f04ad12b0 con 0x557f04bed430 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557f04b26ad0 con 0x557f04befe40 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.643+0000 7fcdcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bc73a0 0x557f04bc4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bc73a0 0x557f04bc4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35398/0 (socket says 172.21.15.59:35398) 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 -- 172.21.15.59:0/3205486151 learned_addr learned my addr 172.21.15.59:0/3205486151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bed430 0x557f04bc0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcf400640 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04befe40 0x557f04bf0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 -- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04befe40 msgr2=0x557f04bf0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04befe40 0x557f04bf0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 -- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bed430 msgr2=0x557f04bc0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bed430 0x557f04bc0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 -- 172.21.15.59:0/3205486151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0483d2d0 con 0x557f04bc73a0 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bed430 0x557f04bc0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcf400640 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04befe40 0x557f04bf0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bc73a0 0x557f04bc4360 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8082c10 tx=0x7fcdc807a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9193af0aa7df37 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdbfe00640 1 -- 172.21.15.59:0/3205486151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80a9070 con 0x557f04bc73a0 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdbfe00640 1 -- 172.21.15.59:0/3205486151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc807a560 con 0x557f04bc73a0 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.644+0000 7fcdbfe00640 1 -- 172.21.15.59:0/3205486151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc807e910 con 0x557f04bc73a0 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 -- 172.21.15.59:0/3205486151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bc73a0 msgr2=0x557f04bc4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bc73a0 0x557f04bc4360 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8082c10 tx=0x7fcdc807a0a0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 -- 172.21.15.59:0/3205486151 shutdown_connections 2024-04-23T14:42:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bc73a0 0x557f04bc4360 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bed430 0x557f04bc0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 --2- 172.21.15.59:0/3205486151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04befe40 0x557f04bf0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 -- 172.21.15.59:0/3205486151 >> 172.21.15.59:0/3205486151 conn(0x557f04bc23b0 msgr2=0x557f04c0ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 -- 172.21.15.59:0/3205486151 shutdown_connections 2024-04-23T14:42:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.645+0000 7fcdd916b740 1 -- 172.21.15.59:0/3205486151 wait complete. 2024-04-23T14:42:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.646+0000 7fcdd916b740 1 -- start start 2024-04-23T14:42:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.646+0000 7fcdd916b740 1 Processor -- start 2024-04-23T14:42:38.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04beca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bfe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bbf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdd916b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f04ad12b0 con 0x557f04befe40 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557f04ae4590 con 0x557f04bfe650 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557f04b2ab10 con 0x557f04bed430 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdd916b740 10 client.0 _ll_register_callbacks cb 0x557f04ac73a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bfe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bfe110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35400/0 (socket says 172.21.15.59:35400) 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdc6a00640 1 -- 172.21.15.59:0/825578617 learned_addr learned my addr 172.21.15.59:0/825578617 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdcf400640 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04beca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bbf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.648+0000 7fcdc6a00640 1 -- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 msgr2=0x557f04beca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04beca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdc6a00640 1 -- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 msgr2=0x557f04bbf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bbf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdc6a00640 1 -- 172.21.15.59:0/825578617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f049076b0 con 0x557f04befe40 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bfe110 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009ec0 tx=0x7fcdb8011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173b9e95b962f2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdbcc00640 1 -- 172.21.15.59:0/825578617 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8019020 con 0x557f04befe40 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc807a560 con 0x557f04befe40 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdbcc00640 1 -- 172.21.15.59:0/825578617 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fcdb8004440 con 0x557f04befe40 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdbcc00640 1 -- 172.21.15.59:0/825578617 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fcdb8023050 con 0x557f04befe40 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdd916b740 10 client.8979 fetch_fsmap learned FSMap version 88 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.649+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x557f04c26ce0 con 0x557f04befe40 2024-04-23T14:42:38.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdbcc00640 1 -- 172.21.15.59:0/825578617 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc807a560 con 0x557f04befe40 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdbcc00640 1 -- 172.21.15.59:0/825578617 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fcdb8026020 con 0x557f04befe40 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdd916b740 10 client.8979 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdd916b740 10 client.8979 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdd916b740 20 client.8979 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdd916b740 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04bfd3e0 0x557f04bf1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdd916b740 4 client.8979 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f04b38530 con 0x557f04bfd3e0 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.650+0000 7fcdcf400640 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04bfd3e0 0x557f04bf1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.651+0000 7fcdcf400640 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04bfd3e0 0x557f04bf1080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd892ef37cb20316 server_cookie=d22fbd59afb4f075 in_seq=0 out_seq=0 2024-04-23T14:42:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.651+0000 7fcdbcc00640 10 client.8979 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdbcc00640 1 -- 172.21.15.59:0/825578617 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f04b38530 con 0x557f04bfd3e0 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdbcc00640 10 client.8979 handle_command_reply: tid=0 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 client.8979 shutdown 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 msgr2=0x557f04bfe110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bfe110 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009ec0 tx=0x7fcdb8011630 comp rx=0 tx=0).stop 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 shutdown_connections 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04bfd3e0 0x557f04bf1080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bbf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bfe110 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 --2- 172.21.15.59:0/825578617 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04beca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdbcc00640 0 client.8979 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 >> 172.21.15.59:0/825578617 conn(0x557f04bc73a0 msgr2=0x557f04bc4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.652+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 shutdown_connections 2024-04-23T14:42:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.653+0000 7fcdd916b740 1 -- 172.21.15.59:0/825578617 wait complete. 2024-04-23T14:42:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.653+0000 7fcdd916b740 20 client.8979 trim_cache size 0 max 16384 2024-04-23T14:42:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.673+0000 7fcdd916b740 1 Processor -- start 2024-04-23T14:42:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.673+0000 7fcdd916b740 1 -- start start 2024-04-23T14:42:38.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04c22680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bbfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bc0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdd916b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd00ab9d0 con 0x557f04befe40 2024-04-23T14:42:38.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0159e20 con 0x557f04bfe650 2024-04-23T14:42:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd00ab380 con 0x557f04bed430 2024-04-23T14:42:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bbfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bbfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35414/0 (socket says 172.21.15.59:35414) 2024-04-23T14:42:38.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.674+0000 7fcdc6a00640 1 -- 172.21.15.59:0/2351853303 learned_addr learned my addr 172.21.15.59:0/2351853303 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bc0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdc6a00640 1 -- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 msgr2=0x557f04c22680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdcf400640 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04c22680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04c22680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdc6a00640 1 -- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 msgr2=0x557f04bc0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bc0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdc6a00640 1 -- 172.21.15.59:0/2351853303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f04bec250 con 0x557f04befe40 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdcf400640 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04c22680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdcfe00640 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bc0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.675+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bbfc30 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8003710 tx=0x7fcdb8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3d57c521a5d043 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdbf400640 1 -- 172.21.15.59:0/2351853303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8019040 con 0x557f04befe40 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdbf400640 1 -- 172.21.15.59:0/2351853303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb8004440 con 0x557f04befe40 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdbf400640 1 -- 172.21.15.59:0/2351853303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8000e60 con 0x557f04befe40 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 -- 172.21.15.59:0/2351853303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 msgr2=0x557f04bbfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bbfc30 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8003710 tx=0x7fcdb8016000 comp rx=0 tx=0).stop 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 -- 172.21.15.59:0/2351853303 shutdown_connections 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04bfe650 0x557f04bc0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04befe40 0x557f04bbfc30 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2351853303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bed430 0x557f04c22680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 -- 172.21.15.59:0/2351853303 >> 172.21.15.59:0/2351853303 conn(0x557f04bc73a0 msgr2=0x557f04c08ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 -- 172.21.15.59:0/2351853303 shutdown_connections 2024-04-23T14:42:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.676+0000 7fcdd916b740 1 -- 172.21.15.59:0/2351853303 wait complete. 2024-04-23T14:42:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.677+0000 7fcdd916b740 1 -- start start 2024-04-23T14:42:38.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.677+0000 7fcdd916b740 1 Processor -- start 2024-04-23T14:42:38.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bed430 0x557f04c09d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04befe40 0x557f04c0a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdd916b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bfe650 0x557f04c0d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdd916b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557f04bfd260 con 0x557f04bed430 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557f04c27080 con 0x557f04befe40 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdd916b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557f04bfd8b0 con 0x557f04bfe650 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdd916b740 10 client.0 _ll_register_callbacks cb 0x7fcdb8009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.678+0000 7fcdcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bed430 0x557f04c09d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bed430 0x557f04c09d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35420/0 (socket says 172.21.15.59:35420) 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 -- 172.21.15.59:0/2700584085 learned_addr learned my addr 172.21.15.59:0/2700584085 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04befe40 0x557f04c0a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 -- 172.21.15.59:0/2700584085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bfe650 msgr2=0x557f04c0d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bfe650 0x557f04c0d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 -- 172.21.15.59:0/2700584085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04befe40 msgr2=0x557f04c0a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04befe40 0x557f04c0a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 -- 172.21.15.59:0/2700584085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f04c8e580 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdcf400640 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bed430 0x557f04c09d70 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x557f04bf08e0 tx=0x7fcdc0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779e32d407bef039 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.679+0000 7fcdbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2700584085 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0028070 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2700584085 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fcdc0024420 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb8004440 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2700584085 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fcdc0033050 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 10 client.8985 fetch_fsmap learned FSMap version 88 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x557f04c0e6c0 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2700584085 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb8004440 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2700584085 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fcdc0036020 con 0x557f04bed430 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 10 client.8985 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:42:38.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 10 client.8985 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 20 client.8985 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04c0e270 0x557f04bf2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 4 client.8985 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.680+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "status"}) -- 0x557f04b323a0 con 0x557f04c0e270 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.681+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04c0e270 0x557f04bf2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.681+0000 7fcdc6a00640 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04c0e270 0x557f04bf2ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5bd52e64a52afb server_cookie=2b11ed369f9224bb in_seq=0 out_seq=0 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.681+0000 7fcdbcc00640 10 client.8985 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdbcc00640 1 -- 172.21.15.59:0/2700584085 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x557f04b323a0 con 0x557f04c0e270 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdbcc00640 10 client.8985 handle_command_reply: tid=0 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 client.8985 shutdown 2024-04-23T14:42:38.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bed430 msgr2=0x557f04c09d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bed430 0x557f04c09d70 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x557f04bf08e0 tx=0x7fcdc0001250 comp rx=0 tx=0).stop 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 shutdown_connections 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557f04bfe650 0x557f04c0d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x557f04c0e270 0x557f04bf2ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557f04befe40 0x557f04c0a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 --2- 172.21.15.59:0/2700584085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557f04bed430 0x557f04c09d70 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdbcc00640 0 client.8985 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 >> 172.21.15.59:0/2700584085 conn(0x557f04bc73a0 msgr2=0x557f04bbfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 shutdown_connections 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 1 -- 172.21.15.59:0/2700584085 wait complete. 2024-04-23T14:42:38.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.682+0000 7fcdd916b740 20 client.8985 trim_cache size 0 max 16384 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 8916, 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0428483970000002, 2024-04-23T14:42:38.688 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 88, 2024-04-23T14:42:38.689 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 100, 2024-04-23T14:42:38.689 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 100, 2024-04-23T14:42:38.689 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 5.9831069130000003 2024-04-23T14:42:38.689 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcdac06c8f0 msgr2=0x7fcdac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcdac06c8f0 0x7fcdac06edb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0007ed0 tx=0x7fcdc0007e60 comp rx=0 tx=0).stop 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd00cf410 msgr2=0x7fcdd00b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd00cf410 0x7fcdd00b48a0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8035040 tx=0x7fcdc80046c0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 shutdown_connections 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcdac06c8f0 0x7fcdac06edb0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdd00cf410 0x7fcdd00b48a0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd0072680 0x7fcdd00b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 --2- 172.21.15.59:0/610245724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdd006ddf0 0x7fcdd014c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 >> 172.21.15.59:0/610245724 conn(0x7fcdd00b2130 msgr2=0x7fcdd00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 shutdown_connections 2024-04-23T14:42:38.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.683+0000 7fcdd6400640 1 -- 172.21.15.59:0/610245724 wait complete. 2024-04-23T14:42:38.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-23T14:42:38.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:42:38.709 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-23T14:42:38.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.898+0000 7f1e96400640 1 Processor -- start 2024-04-23T14:42:38.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.899+0000 7f1e96400640 1 -- start start 2024-04-23T14:42:38.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.900+0000 7f1e96400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e9015e990 0x7f1e900d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.900+0000 7f1e96400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e900cec40 0x7f1e900d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.900+0000 7f1e96400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e900cfc70 con 0x7f1e900d4d00 2024-04-23T14:42:38.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.900+0000 7f1e96400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1e900d0230 con 0x7f1e900cec40 2024-04-23T14:42:38.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.900+0000 7f1e96400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1e900d21c0 con 0x7f1e9015e990 2024-04-23T14:42:38.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.901+0000 7f1e8fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e900cec40 0x7f1e900d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.901+0000 7f1e95000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e9015e990 0x7f1e900d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.901+0000 7f1e8fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e900cec40 0x7f1e900d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:34028/0 (socket says 172.21.15.59:34028) 2024-04-23T14:42:38.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.901+0000 7f1e8fe00640 1 -- 172.21.15.59:0/3900040271 learned_addr learned my addr 172.21.15.59:0/3900040271 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.902+0000 7f1e8fe00640 1 -- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e9015e990 msgr2=0x7f1e900d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.902+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e9015e990 0x7f1e900d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.902+0000 7f1e8fe00640 1 -- 172.21.15.59:0/3900040271 >> v1:172.21.15.76:6789/0 conn(0x7f1e900d4d00 legacy=0x7f1e9006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.902+0000 7f1e8fe00640 1 -- 172.21.15.59:0/3900040271 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e900cff50 con 0x7f1e900cec40 2024-04-23T14:42:38.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.902+0000 7f1e95000640 1 --2- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e9015e990 0x7f1e900d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:42:38.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.903+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e900cec40 0x7f1e900d1540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e900d2340 tx=0x7f1e7c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f34a7301ec631045 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.903+0000 7f1e8f400640 1 -- 172.21.15.59:0/3900040271 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c004030 con 0x7f1e900cec40 2024-04-23T14:42:38.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.903+0000 7f1e8f400640 1 -- 172.21.15.59:0/3900040271 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7c0041d0 con 0x7f1e900cec40 2024-04-23T14:42:38.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e8f400640 1 -- 172.21.15.59:0/3900040271 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c003610 con 0x7f1e900cec40 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e96400640 1 -- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e900cec40 msgr2=0x7f1e900d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e96400640 1 --2- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e900cec40 0x7f1e900d1540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e900d2340 tx=0x7f1e7c02a230 comp rx=0 tx=0).stop 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e96400640 1 -- 172.21.15.59:0/3900040271 shutdown_connections 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e96400640 1 --2- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e900cec40 0x7f1e900d1540 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e96400640 1 --2- 172.21.15.59:0/3900040271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e9015e990 0x7f1e900d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e96400640 1 -- 172.21.15.59:0/3900040271 >> 172.21.15.59:0/3900040271 conn(0x7f1e9008bf90 msgr2=0x7f1e900d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.904+0000 7f1e96400640 1 -- 172.21.15.59:0/3900040271 shutdown_connections 2024-04-23T14:42:38.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.905+0000 7f1e96400640 1 -- 172.21.15.59:0/3900040271 wait complete. 2024-04-23T14:42:38.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.907+0000 7f1e96400640 1 Processor -- start 2024-04-23T14:42:38.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.907+0000 7f1e96400640 1 -- start start 2024-04-23T14:42:38.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e96400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e900cec40 0x7f1e900b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e96400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e900d4d00 0x7f1e900d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e96400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e9015e990 0x7f1e900a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e96400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9006e1f0 con 0x7f1e900cec40 2024-04-23T14:42:38.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e96400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e900d26f0 con 0x7f1e9015e990 2024-04-23T14:42:38.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e96400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1e900b76c0 con 0x7f1e900d4d00 2024-04-23T14:42:38.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e95000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e900cec40 0x7f1e900b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e95000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e900cec40 0x7f1e900b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35434/0 (socket says 172.21.15.59:35434) 2024-04-23T14:42:38.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.908+0000 7f1e95000640 1 -- 172.21.15.59:0/972540697 learned_addr learned my addr 172.21.15.59:0/972540697 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e95a00640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e9015e990 0x7f1e900a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e900d4d00 0x7f1e900d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e95000640 1 -- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e900d4d00 msgr2=0x7f1e900d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e95000640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e900d4d00 0x7f1e900d2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e95000640 1 -- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e9015e990 msgr2=0x7f1e900a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e95000640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e9015e990 0x7f1e900a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e95000640 1 -- 172.21.15.59:0/972540697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e901f4810 con 0x7f1e900cec40 2024-04-23T14:42:38.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.909+0000 7f1e95000640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e900cec40 0x7f1e900b7180 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400e960 tx=0x7f1e8400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.910+0000 7f1e8cc00640 1 -- 172.21.15.59:0/972540697 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84017070 con 0x7f1e900cec40 2024-04-23T14:42:38.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.910+0000 7f1e8cc00640 1 -- 172.21.15.59:0/972540697 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8400ca00 con 0x7f1e900cec40 2024-04-23T14:42:38.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.910+0000 7f1e8cc00640 1 -- 172.21.15.59:0/972540697 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400cdd0 con 0x7f1e900cec40 2024-04-23T14:42:38.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.910+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e901f4af0 con 0x7f1e900cec40 2024-04-23T14:42:38.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.910+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e901f4500 con 0x7f1e900cec40 2024-04-23T14:42:38.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.913+0000 7f1e8cc00640 1 -- 172.21.15.59:0/972540697 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e8401a070 con 0x7f1e900cec40 2024-04-23T14:42:38.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.914+0000 7f1e8cc00640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e7006c8f0 0x7f1e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.914+0000 7f1e8cc00640 1 -- 172.21.15.59:0/972540697 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f1e8408bec0 con 0x7f1e900cec40 2024-04-23T14:42:38.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.914+0000 7f1e9a82d740 1 Processor -- start 2024-04-23T14:42:38.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.914+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e7006c8f0 0x7f1e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.915+0000 7f1e9a82d740 1 -- start start 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e7006c8f0 0x7f1e7006edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c033000 tx=0x7f1e7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095e3020 0x55c6095b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c60951ce70 con 0x55c609600640 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c6094c3380 con 0x55c6095e3760 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c609518e30 con 0x55c6095e3020 2024-04-23T14:42:38.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e95000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e95000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35446/0 (socket says 172.21.15.59:35446) 2024-04-23T14:42:38.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e95000640 1 -- 172.21.15.59:0/3715282894 learned_addr learned my addr 172.21.15.59:0/3715282894 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e95000640 1 -- 172.21.15.59:0/3715282894 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095e3020 msgr2=0x55c6095b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.916+0000 7f1e95000640 1 --2- 172.21.15.59:0/3715282894 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095e3020 0x55c6095b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.917+0000 7f1e95000640 1 -- 172.21.15.59:0/3715282894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 msgr2=0x55c6095e2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.917+0000 7f1e95000640 1 --2- 172.21.15.59:0/3715282894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.917+0000 7f1e95000640 1 -- 172.21.15.59:0/3715282894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6092f77e0 con 0x55c609600640 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.917+0000 7f1e95000640 1 --2- 172.21.15.59:0/3715282894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095b2fd0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f1e8408a260 tx=0x7f1e84059af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbb66633391e8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.917+0000 7f1e81600640 1 -- 172.21.15.59:0/3715282894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84021040 con 0x55c609600640 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.917+0000 7f1e81600640 1 -- 172.21.15.59:0/3715282894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e84057b90 con 0x55c609600640 2024-04-23T14:42:38.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.917+0000 7f1e81600640 1 -- 172.21.15.59:0/3715282894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84056160 con 0x55c609600640 2024-04-23T14:42:38.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3715282894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 msgr2=0x55c6095b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3715282894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095b2fd0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f1e8408a260 tx=0x7f1e84059af0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3715282894 shutdown_connections 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3715282894 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095e3020 0x55c6095b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3715282894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095e2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3715282894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095b2fd0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3715282894 >> 172.21.15.59:0/3715282894 conn(0x55c6095b5240 msgr2=0x55c6095fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3715282894 shutdown_connections 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.918+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3715282894 wait complete. 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.919+0000 7f1e9a82d740 1 -- start start 2024-04-23T14:42:38.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.920+0000 7f1e9a82d740 1 Processor -- start 2024-04-23T14:42:38.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.921+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c609611340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.921+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095de540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.921+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c6095e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.921+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c6095b73f0 con 0x55c609600640 2024-04-23T14:42:38.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c6094d6660 con 0x55c6095e3760 2024-04-23T14:42:38.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c6095b67e0 con 0x55c6095fd170 2024-04-23T14:42:38.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e9a82d740 10 client.0 _ll_register_callbacks cb 0x55c6094c4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:38.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095de540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095de540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35454/0 (socket says 172.21.15.59:35454) 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 -- 172.21.15.59:0/2457030948 learned_addr learned my addr 172.21.15.59:0/2457030948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e95000640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c609611340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e95a00640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c6095e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 -- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 msgr2=0x55c6095e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c6095e5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 -- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 msgr2=0x55c609611340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c609611340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e8fe00640 1 -- 172.21.15.59:0/2457030948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c609357f70 con 0x55c609600640 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.922+0000 7f1e95a00640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c6095e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095de540 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009940 tx=0x7f1e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31909158ce2b44de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e76000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:38.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e76000640 1 -- 172.21.15.59:0/2457030948 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c006240 con 0x55c609600640 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e76000640 1 -- 172.21.15.59:0/2457030948 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f1e7c0063e0 con 0x55c609600640 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e84057b90 con 0x55c609600640 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e76000640 1 -- 172.21.15.59:0/2457030948 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c054050 con 0x55c609600640 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e9a82d740 10 client.8994 fetch_fsmap learned FSMap version 88 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.923+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55c6091ef170 con 0x55c609600640 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e76000640 1 -- 172.21.15.59:0/2457030948 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e84057b90 con 0x55c609600640 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e76000640 1 -- 172.21.15.59:0/2457030948 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f1e7c057020 con 0x55c609600640 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e9a82d740 10 client.8994 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e9a82d740 10 client.8994 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e9a82d740 20 client.8994 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c60960e240 0x55c60960e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e9a82d740 4 client.8994 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c60952a890 con 0x55c60960e240 2024-04-23T14:42:38.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.924+0000 7f1e95000640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c60960e240 0x55c60960e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.925+0000 7f1e95000640 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c60960e240 0x55c60960e650 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e31a11dffcc437a server_cookie=90cb1dbacaefa91f in_seq=0 out_seq=0 2024-04-23T14:42:38.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.925+0000 7f1e76000640 10 client.8994 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e76000640 1 -- 172.21.15.59:0/2457030948 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c60952a890 con 0x55c60960e240 2024-04-23T14:42:38.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e76000640 10 client.8994 handle_command_reply: tid=0 2024-04-23T14:42:38.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 client.8994 shutdown 2024-04-23T14:42:38.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f8bbc400640 1 Processor -- start 2024-04-23T14:42:38.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 msgr2=0x55c6095de540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095de540 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009940 tx=0x7f1e7c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 shutdown_connections 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c6095e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c60960e240 0x55c60960e650 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c6095de540 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/2457030948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c609611340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e76000640 0 client.8994 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 >> 172.21.15.59:0/2457030948 conn(0x55c6095e3020 msgr2=0x55c6095fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f8bbc400640 1 -- start start 2024-04-23T14:42:38.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 shutdown_connections 2024-04-23T14:42:38.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 1 -- 172.21.15.59:0/2457030948 wait complete. 2024-04-23T14:42:38.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.926+0000 7f1e9a82d740 20 client.8994 trim_cache size 0 max 16384 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bbc400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8bb40cd7a0 0x7f8bb415cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bbc400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb40cf330 con 0x7f8bb415fdb0 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bbc400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb406f370 con 0x7f8bb4146000 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bbc400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb40cfc00 con 0x7f8bb40cd7a0 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bb9a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8bb415fdb0 0x7f8bb406ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56426/0 (socket says 172.21.15.59:56426) 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bb9a00640 1 -- 172.21.15.59:0/680631436 learned_addr learned my addr 172.21.15.59:0/680631436 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bba400640 1 --2- 172.21.15.59:0/680631436 >> v2:172.21.15.59:3300/0 conn(0x7f8bb40cd7a0 0x7f8bb415cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bb3e00640 1 -- 172.21.15.59:0/680631436 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170704104 0 0) 0x7f8ba8001110 con 0x7f8bb415fdb0 2024-04-23T14:42:38.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bb3e00640 1 -- 172.21.15.59:0/680631436 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b9c003520 con 0x7f8bb415fdb0 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bba400640 1 -- 172.21.15.59:0/680631436 >> v1:172.21.15.76:6789/0 conn(0x7f8bb4146000 legacy=0x7f8bb40cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bba400640 1 -- 172.21.15.59:0/680631436 >> v1:172.21.15.59:6789/0 conn(0x7f8bb415fdb0 legacy=0x7f8bb406ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.928+0000 7f8bba400640 1 -- 172.21.15.59:0/680631436 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb40cf5b0 con 0x7f8bb40cd7a0 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.930+0000 7f8bba400640 1 --2- 172.21.15.59:0/680631436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb40cd7a0 0x7f8bb415cd00 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009d90 tx=0x7f8bac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2b63848c784475 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.930+0000 7f8bb3e00640 1 -- 172.21.15.59:0/680631436 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac007740 con 0x7f8bb40cd7a0 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.930+0000 7f8bb3e00640 1 -- 172.21.15.59:0/680631436 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bac0078e0 con 0x7f8bb40cd7a0 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.930+0000 7f8bb3e00640 1 -- 172.21.15.59:0/680631436 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac007c90 con 0x7f8bb40cd7a0 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.931+0000 7f8bbc400640 1 -- 172.21.15.59:0/680631436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb40cd7a0 msgr2=0x7f8bb415cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.931+0000 7f8bbc400640 1 --2- 172.21.15.59:0/680631436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb40cd7a0 0x7f8bb415cd00 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009d90 tx=0x7f8bac02a6f0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.931+0000 7f8bbc400640 1 -- 172.21.15.59:0/680631436 shutdown_connections 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.931+0000 7f8bbc400640 1 --2- 172.21.15.59:0/680631436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb40cd7a0 0x7f8bb415cd00 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.931+0000 7f8bbc400640 1 -- 172.21.15.59:0/680631436 >> 172.21.15.59:0/680631436 conn(0x7f8bb4067ea0 msgr2=0x7f8bb406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.931+0000 7f8bbc400640 1 -- 172.21.15.59:0/680631436 shutdown_connections 2024-04-23T14:42:38.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.931+0000 7f8bbc400640 1 -- 172.21.15.59:0/680631436 wait complete. 2024-04-23T14:42:38.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.933+0000 7f8bbc400640 1 Processor -- start 2024-04-23T14:42:38.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.933+0000 7f8bbc400640 1 -- start start 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bbc400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb40cd7a0 0x7f8bb41410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bbc400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb4146000 0x7f8bb40c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bbc400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb415fdb0 0x7f8bb40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bbc400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb40d0090 con 0x7f8bb415fdb0 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bbc400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb40cf890 con 0x7f8bb40cd7a0 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bbc400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb40d0210 con 0x7f8bb4146000 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bba400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb415fdb0 0x7f8bb40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bba400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb415fdb0 0x7f8bb40c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35474/0 (socket says 172.21.15.59:35474) 2024-04-23T14:42:38.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bba400640 1 -- 172.21.15.59:0/3686846165 learned_addr learned my addr 172.21.15.59:0/3686846165 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bb9a00640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb40cd7a0 0x7f8bb41410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bb9000640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb4146000 0x7f8bb40c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.934+0000 7f8bba400640 1 -- 172.21.15.59:0/3686846165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb4146000 msgr2=0x7f8bb40c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bba400640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb4146000 0x7f8bb40c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bba400640 1 -- 172.21.15.59:0/3686846165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb40cd7a0 msgr2=0x7f8bb41410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bba400640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb40cd7a0 0x7f8bb41410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bba400640 1 -- 172.21.15.59:0/3686846165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb40c6fe0 con 0x7f8bb415fdb0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bba400640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb415fdb0 0x7f8bb40c8190 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009eb0 tx=0x7f8bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac009450 con 0x7f8bb415fdb0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bac004240 con 0x7f8bb415fdb0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bbc400640 1 -- 172.21.15.59:0/3686846165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb41e7a80 con 0x7f8bb415fdb0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac004630 con 0x7f8bb415fdb0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.935+0000 7f8bbc400640 1 -- 172.21.15.59:0/3686846165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb41e8040 con 0x7f8bb415fdb0 2024-04-23T14:42:38.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.936+0000 7f8bbc400640 1 -- 172.21.15.59:0/3686846165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8bb4073800 con 0x7f8bb415fdb0 2024-04-23T14:42:38.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.937+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bac039040 con 0x7f8bb415fdb0 2024-04-23T14:42:38.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.938+0000 7f8bb1600640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b9406c8f0 0x7f8b9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.939+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f8bac0accb0 con 0x7f8bb415fdb0 2024-04-23T14:42:38.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.939+0000 7f8bb9a00640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b9406c8f0 0x7f8b9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.939+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+516 (secure 0 0 0) 0x7f8bac040030 con 0x7f8bb415fdb0 2024-04-23T14:42:38.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.939+0000 7f8bbc400640 1 -- 172.21.15.59:0/3686846165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f8bb404fe70 con 0x7f8bb415fdb0 2024-04-23T14:42:38.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.940+0000 7f8bb9a00640 1 --2- 172.21.15.59:0/3686846165 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8b9406c8f0 0x7f8b9406edb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8007fd0 tx=0x7f8ba8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.948+0000 7f1e9a82d740 1 Processor -- start 2024-04-23T14:42:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.948+0000 7f1e9a82d740 1 -- start start 2024-04-23T14:42:38.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.949+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095feae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.949+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.949+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.949+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90200cf0 con 0x55c609600640 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.949+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e901ffe60 con 0x55c6095e3760 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.949+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9015f220 con 0x55c6095fd170 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35484/0 (socket says 172.21.15.59:35484) 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 -- 172.21.15.59:0/3378154997 learned_addr learned my addr 172.21.15.59:0/3378154997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95000640 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 -- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 msgr2=0x55c609600f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 -- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 msgr2=0x55c6095feae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095feae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 -- 172.21.15.59:0/3378154997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6095ba690 con 0x55c609600640 2024-04-23T14:42:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95000640 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.950+0000 7f1e95a00640 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c609601440 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1e88001cb0 tx=0x7f1e88002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4b771d3fc7a97a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e80c00640 1 -- 172.21.15.59:0/3378154997 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88003b80 con 0x55c609600640 2024-04-23T14:42:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e80c00640 1 -- 172.21.15.59:0/3378154997 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88018070 con 0x55c609600640 2024-04-23T14:42:38.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e80c00640 1 -- 172.21.15.59:0/3378154997 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8800f910 con 0x55c609600640 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3378154997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 msgr2=0x55c609601440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c609601440 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1e88001cb0 tx=0x7f1e88002170 comp rx=0 tx=0).stop 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3378154997 shutdown_connections 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3378154997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c6095e3760 0x55c6095feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.951+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3378154997 >> 172.21.15.59:0/3378154997 conn(0x55c6095e3020 msgr2=0x55c6095b9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.952+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3378154997 shutdown_connections 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.952+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3378154997 wait complete. 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.952+0000 7f1e9a82d740 1 -- start start 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.952+0000 7f1e9a82d740 1 Processor -- start 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.953+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c6095e3760 0x55c609661b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.953+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.953+0000 7f1e9a82d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.953+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9015f870 con 0x55c6095e3760 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.953+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c609620920 con 0x55c609600640 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.953+0000 7f1e9a82d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c609678660 con 0x55c6095fd170 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.953+0000 7f1e9a82d740 10 client.0 _ll_register_callbacks cb 0x7f1e84009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c6095e3760 0x55c609661b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c6095e3760 0x55c609661b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35494/0 (socket says 172.21.15.59:35494) 2024-04-23T14:42:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 -- 172.21.15.59:0/3958373622 learned_addr learned my addr 172.21.15.59:0/3958373622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 -- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 msgr2=0x55c609600f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95a00640 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 -- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c609600640 msgr2=0x55c609601440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.954+0000 7f1e95000640 1 -- 172.21.15.59:0/3958373622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6095ddd00 con 0x55c6095e3760 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e95000640 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c6095e3760 0x55c609661b30 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f1e84055f00 tx=0x7f1e84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1da2c320db226ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e76000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e76000640 1 -- 172.21.15.59:0/3958373622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84021040 con 0x55c6095e3760 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e76000640 1 -- 172.21.15.59:0/3958373622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f1e84057df0 con 0x55c6095e3760 2024-04-23T14:42:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88018070 con 0x55c6095e3760 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e76000640 1 -- 172.21.15.59:0/3958373622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1e84056160 con 0x55c6095e3760 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e9a82d740 10 client.9009 fetch_fsmap learned FSMap version 88 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55c6095fd6b0 con 0x55c6095e3760 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e76000640 1 -- 172.21.15.59:0/3958373622 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88018070 con 0x55c6095e3760 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.955+0000 7f1e76000640 1 -- 172.21.15.59:0/3958373622 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f1e84058f40 con 0x55c6095e3760 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.956+0000 7f1e9a82d740 10 client.9009 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.956+0000 7f1e9a82d740 10 client.9009 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.956+0000 7f1e9a82d740 20 client.9009 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.956+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c609601ac0 0x55c60967bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.956+0000 7f1e9a82d740 4 client.9009 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.956+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55c6095b4690 con 0x55c609601ac0 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.956+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c609601ac0 0x55c60967bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.957+0000 7f1e8fe00640 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c609601ac0 0x55c60967bb50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fd39e887d673e0 server_cookie=d4bdd0aea0ab08a6 in_seq=0 out_seq=0 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.957+0000 7f1e76000640 10 client.9009 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.957+0000 7f1e76000640 1 -- 172.21.15.59:0/3958373622 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c6095b4690 con 0x55c609601ac0 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.957+0000 7f1e76000640 10 client.9009 handle_command_reply: tid=0 2024-04-23T14:42:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 client.9009 shutdown 2024-04-23T14:42:38.961 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-23T14:42:38.961 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-23T14:42:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c6095e3760 msgr2=0x55c609661b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c6095e3760 0x55c609661b30 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f1e84055f00 tx=0x7f1e84000bc0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.961 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-23T14:42:38.961 INFO:tasks.ceph.mds.a.smithi059.stderr:2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 shutdown_connections 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c6095fd170 0x55c609600f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c609601ac0 0x55c60967bb50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c609600640 0x55c609601440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 --2- 172.21.15.59:0/3958373622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c6095e3760 0x55c609661b30 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 >> 172.21.15.59:0/3958373622 conn(0x55c6095e3020 msgr2=0x55c6095feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e76000640 0 client.9009 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 shutdown_connections 2024-04-23T14:42:38.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.958+0000 7f1e9a82d740 1 -- 172.21.15.59:0/3958373622 wait complete. 2024-04-23T14:42:38.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.959+0000 7f1e9a82d740 20 client.9009 trim_cache size 0 max 16384 2024-04-23T14:42:38.963 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:42:38.963 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0, 2024-04-23T14:42:38.964 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_tag": "43b3ed15-9260-40bd-9efb-ae2d23963b5f", 2024-04-23T14:42:38.964 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": "asynchronous" 2024-04-23T14:42:38.964 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.959+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e7006c8f0 msgr2=0x7f1e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.959+0000 7f1e96400640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e7006c8f0 0x7f1e7006edb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c033000 tx=0x7f1e7c02e000 comp rx=0 tx=0).stop 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.959+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e900cec40 msgr2=0x7f1e900b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.959+0000 7f1e96400640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e900cec40 0x7f1e900b7180 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400e960 tx=0x7f1e8400ee20 comp rx=0 tx=0).stop 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 shutdown_connections 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1e7006c8f0 0x7f1e7006edb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e900d4d00 0x7f1e900d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e9015e990 0x7f1e900a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 --2- 172.21.15.59:0/972540697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e900cec40 0x7f1e900b7180 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 >> 172.21.15.59:0/972540697 conn(0x7f1e9008bf90 msgr2=0x7f1e900d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 shutdown_connections 2024-04-23T14:42:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:38.960+0000 7f1e96400640 1 -- 172.21.15.59:0/972540697 wait complete. 2024-04-23T14:42:38.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:42:38.977 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-23T14:42:39.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.167+0000 7fd20fe00640 1 Processor -- start 2024-04-23T14:42:39.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.167+0000 7fd20fe00640 1 -- start start 2024-04-23T14:42:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.168+0000 7fd20fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd21015ed20 0x7fd2100d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.168+0000 7fd20fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2100d13e0 0x7fd2100d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.168+0000 7fd20fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2100b5da0 con 0x7fd2100b3600 2024-04-23T14:42:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.168+0000 7fd20fe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd21015f100 con 0x7fd21015ed20 2024-04-23T14:42:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.168+0000 7fd20fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd2100d07d0 con 0x7fd2100d13e0 2024-04-23T14:42:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.169+0000 7fd20ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd21015ed20 0x7fd2100d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.169+0000 7fd20ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd21015ed20 0x7fd2100d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35502/0 (socket says 172.21.15.59:35502) 2024-04-23T14:42:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.169+0000 7fd20ea00640 1 -- 172.21.15.59:0/2469005031 learned_addr learned my addr 172.21.15.59:0/2469005031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.169+0000 7fd20e000640 1 --1- 172.21.15.59:0/2469005031 >> v1:172.21.15.59:6789/0 conn(0x7fd2100b3600 0x7fd2100b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56430/0 (socket says 172.21.15.59:56430) 2024-04-23T14:42:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.169+0000 7fd20f400640 1 --2- 172.21.15.59:0/2469005031 >> v2:172.21.15.76:3300/0 conn(0x7fd2100d13e0 0x7fd2100d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.170+0000 7fd20ea00640 1 -- 172.21.15.59:0/2469005031 >> v2:172.21.15.76:3300/0 conn(0x7fd2100d13e0 msgr2=0x7fd2100d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.170+0000 7fd20ea00640 1 --2- 172.21.15.59:0/2469005031 >> v2:172.21.15.76:3300/0 conn(0x7fd2100d13e0 0x7fd2100d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.170+0000 7fd20ea00640 1 -- 172.21.15.59:0/2469005031 >> v1:172.21.15.59:6789/0 conn(0x7fd2100b3600 legacy=0x7fd2100b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.170+0000 7fd20ea00640 1 -- 172.21.15.59:0/2469005031 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2100d0a20 con 0x7fd21015ed20 2024-04-23T14:42:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.170+0000 7fd20d600640 1 -- 172.21.15.59:0/2469005031 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2570270256 0 0) 0x7fd2100b5da0 con 0x7fd2100b3600 2024-04-23T14:42:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.171+0000 7fd20ea00640 1 --2- 172.21.15.59:0/2469005031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100d0ea0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fd21008e630 tx=0x7fd20002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2f3bb527369782 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.171+0000 7fd20d600640 1 -- 172.21.15.59:0/2469005031 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200037070 con 0x7fd21015ed20 2024-04-23T14:42:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.171+0000 7fd20d600640 1 -- 172.21.15.59:0/2469005031 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd200032440 con 0x7fd21015ed20 2024-04-23T14:42:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.171+0000 7fd20d600640 1 -- 172.21.15.59:0/2469005031 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200032810 con 0x7fd21015ed20 2024-04-23T14:42:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 -- 172.21.15.59:0/2469005031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 msgr2=0x7fd2100d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 --2- 172.21.15.59:0/2469005031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100d0ea0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fd21008e630 tx=0x7fd20002a150 comp rx=0 tx=0).stop 2024-04-23T14:42:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 -- 172.21.15.59:0/2469005031 shutdown_connections 2024-04-23T14:42:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 --2- 172.21.15.59:0/2469005031 >> v2:172.21.15.76:3300/0 conn(0x7fd2100d13e0 0x7fd2100d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 --2- 172.21.15.59:0/2469005031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100d0ea0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 -- 172.21.15.59:0/2469005031 >> 172.21.15.59:0/2469005031 conn(0x7fd21008bf90 msgr2=0x7fd2100d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 -- 172.21.15.59:0/2469005031 shutdown_connections 2024-04-23T14:42:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.172+0000 7fd20fe00640 1 -- 172.21.15.59:0/2469005031 wait complete. 2024-04-23T14:42:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.174+0000 7fd20fe00640 1 Processor -- start 2024-04-23T14:42:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.174+0000 7fd20fe00640 1 -- start start 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2100b3600 0x7fd2100a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2100d13e0 0x7fd2100cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100d3490 con 0x7fd21015ed20 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100d31b0 con 0x7fd2100b3600 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2100b4620 con 0x7fd2100d13e0 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35508/0 (socket says 172.21.15.59:35508) 2024-04-23T14:42:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.175+0000 7fd20f400640 1 -- 172.21.15.59:0/1984952457 learned_addr learned my addr 172.21.15.59:0/1984952457 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20ea00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2100b3600 0x7fd2100a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20e000640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2100d13e0 0x7fd2100cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20f400640 1 -- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2100d13e0 msgr2=0x7fd2100cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20f400640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2100d13e0 0x7fd2100cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20f400640 1 -- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2100b3600 msgr2=0x7fd2100a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20f400640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2100b3600 0x7fd2100a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20f400640 1 -- 172.21.15.59:0/1984952457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21015a810 con 0x7fd21015ed20 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20e000640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2100d13e0 0x7fd2100cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20f400640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100ccaf0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd208007e70 tx=0x7fd20800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd206a00640 1 -- 172.21.15.59:0/1984952457 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208014620 con 0x7fd21015ed20 2024-04-23T14:42:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd206a00640 1 -- 172.21.15.59:0/1984952457 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd208019070 con 0x7fd21015ed20 2024-04-23T14:42:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd206a00640 1 -- 172.21.15.59:0/1984952457 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208014a20 con 0x7fd21015ed20 2024-04-23T14:42:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.176+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21015aaf0 con 0x7fd21015ed20 2024-04-23T14:42:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.177+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21015b0b0 con 0x7fd21015ed20 2024-04-23T14:42:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.179+0000 7fd206a00640 1 -- 172.21.15.59:0/1984952457 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd208014bc0 con 0x7fd21015ed20 2024-04-23T14:42:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.180+0000 7fd206a00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f006c8f0 0x7fd1f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.180+0000 7fd206a00640 1 -- 172.21.15.59:0/1984952457 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fd20808c5a0 con 0x7fd21015ed20 2024-04-23T14:42:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.180+0000 7fd20ea00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f006c8f0 0x7fd1f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.181+0000 7fd21863d740 1 Processor -- start 2024-04-23T14:42:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.181+0000 7fd21863d740 1 -- start start 2024-04-23T14:42:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.181+0000 7fd20ea00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f006c8f0 0x7fd1f006edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd200034000 tx=0x7fd20002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd21863d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b65ce00 0x55c90b60f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b63ff20 0x55c90b63f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b63f7e0 0x55c90b6145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd21863d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c90b5791f0 con 0x55c90b65ce00 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c90b51f350 con 0x55c90b63ff20 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c90b5751b0 con 0x55c90b63f7e0 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b65ce00 0x55c90b60f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b65ce00 0x55c90b60f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35516/0 (socket says 172.21.15.59:35516) 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.182+0000 7fd20ea00640 1 -- 172.21.15.59:0/2187931650 learned_addr learned my addr 172.21.15.59:0/2187931650 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20ea00640 1 -- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b63f7e0 msgr2=0x55c90b6145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20f400640 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b63f7e0 0x55c90b6145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20e000640 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b63ff20 0x55c90b63f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20ea00640 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b63f7e0 0x55c90b6145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20ea00640 1 -- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b63ff20 msgr2=0x55c90b63f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20ea00640 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b63ff20 0x55c90b63f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20ea00640 1 -- 172.21.15.59:0/2187931650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c90b573f50 con 0x55c90b65ce00 2024-04-23T14:42:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd20ea00640 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b65ce00 0x55c90b60f790 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd20002a740 tx=0x7fd20002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4585fcd4a91d20d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd1fb400640 1 -- 172.21.15.59:0/2187931650 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200037040 con 0x55c90b65ce00 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd1fb400640 1 -- 172.21.15.59:0/2187931650 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd200032e00 con 0x55c90b65ce00 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.183+0000 7fd1fb400640 1 -- 172.21.15.59:0/2187931650 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200004250 con 0x55c90b65ce00 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 -- 172.21.15.59:0/2187931650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b65ce00 msgr2=0x55c90b60f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b65ce00 0x55c90b60f790 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd20002a740 tx=0x7fd20002c000 comp rx=0 tx=0).stop 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 -- 172.21.15.59:0/2187931650 shutdown_connections 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b63f7e0 0x55c90b6145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b63ff20 0x55c90b63f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 --2- 172.21.15.59:0/2187931650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b65ce00 0x55c90b60f790 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 -- 172.21.15.59:0/2187931650 >> 172.21.15.59:0/2187931650 conn(0x55c90b611a00 msgr2=0x55c90b65a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 -- 172.21.15.59:0/2187931650 shutdown_connections 2024-04-23T14:42:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.184+0000 7fd21863d740 1 -- 172.21.15.59:0/2187931650 wait complete. 2024-04-23T14:42:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.185+0000 7fd21863d740 1 -- start start 2024-04-23T14:42:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.185+0000 7fd21863d740 1 Processor -- start 2024-04-23T14:42:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.186+0000 7fd21863d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b615110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.186+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b66a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.186+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b66aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.186+0000 7fd21863d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c90b613bb0 con 0x55c90b63ff20 2024-04-23T14:42:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.186+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c90b532690 con 0x55c90b65ce00 2024-04-23T14:42:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.186+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c90b612fa0 con 0x55c90b66d960 2024-04-23T14:42:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd21863d740 10 client.0 _ll_register_callbacks cb 0x55c90b56e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b615110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b615110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35522/0 (socket says 172.21.15.59:35522) 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 -- 172.21.15.59:0/1777579648 learned_addr learned my addr 172.21.15.59:0/1777579648 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20e000640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b66a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20f400640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b66aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 -- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 msgr2=0x55c90b66aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b66aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 -- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 msgr2=0x55c90b66a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b66a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20ea00640 1 -- 172.21.15.59:0/1777579648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c90b531920 con 0x55c90b63ff20 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20f400640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b66aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.187+0000 7fd20e000640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b66a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd20ea00640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b615110 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fd200000c00 tx=0x7fd200004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3c38fafa93555b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd1efe00640 1 -- 172.21.15.59:0/1777579648 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20004e070 con 0x55c90b63ff20 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd1efe00640 1 -- 172.21.15.59:0/1777579648 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fd200003b60 con 0x55c90b63ff20 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd200032e00 con 0x55c90b63ff20 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd1efe00640 1 -- 172.21.15.59:0/1777579648 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd200037040 con 0x55c90b63ff20 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd21863d740 10 client.9024 fetch_fsmap learned FSMap version 88 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55c90b3557d0 con 0x55c90b63ff20 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.188+0000 7fd1efe00640 1 -- 172.21.15.59:0/1777579648 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd200032e00 con 0x55c90b63ff20 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd1efe00640 1 -- 172.21.15.59:0/1777579648 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fd200046070 con 0x55c90b63ff20 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd21863d740 10 client.9024 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd21863d740 10 client.9024 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd21863d740 20 client.9024 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd21863d740 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b63a560 0x55c90b63a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd21863d740 4 client.9024 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c90b586c10 con 0x55c90b63a560 2024-04-23T14:42:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.189+0000 7fd20e000640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b63a560 0x55c90b63a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.190+0000 7fd20e000640 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b63a560 0x55c90b63a940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5403b3a22f4354 server_cookie=7e2ea553d8178ff5 in_seq=0 out_seq=0 2024-04-23T14:42:39.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.190+0000 7fd1efe00640 10 client.9024 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.190+0000 7fd1efe00640 1 -- 172.21.15.59:0/1777579648 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c90b586c10 con 0x55c90b63a560 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.190+0000 7fd1efe00640 10 client.9024 handle_command_reply: tid=0 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.190+0000 7fd21863d740 1 client.9024 shutdown 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 msgr2=0x55c90b615110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b615110 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fd200000c00 tx=0x7fd200004730 comp rx=0 tx=0).stop 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 shutdown_connections 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b66aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b63a560 0x55c90b63a940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b66a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 --2- 172.21.15.59:0/1777579648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b615110 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd1efe00640 0 client.9024 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 >> 172.21.15.59:0/1777579648 conn(0x55c90b63f7e0 msgr2=0x55c90b6133f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 shutdown_connections 2024-04-23T14:42:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 1 -- 172.21.15.59:0/1777579648 wait complete. 2024-04-23T14:42:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.191+0000 7fd21863d740 20 client.9024 trim_cache size 0 max 16384 2024-04-23T14:42:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.211+0000 7fd21863d740 1 Processor -- start 2024-04-23T14:42:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.211+0000 7fd21863d740 1 -- start start 2024-04-23T14:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd21863d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b6131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b613720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b613e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd21863d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd210204da0 con 0x55c90b63ff20 2024-04-23T14:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd210203e90 con 0x55c90b65ce00 2024-04-23T14:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd21015c240 con 0x55c90b66d960 2024-04-23T14:42:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b6131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b6131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35534/0 (socket says 172.21.15.59:35534) 2024-04-23T14:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 -- 172.21.15.59:0/832274008 learned_addr learned my addr 172.21.15.59:0/832274008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20f400640 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b613e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 -- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 msgr2=0x55c90b613e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b613e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 -- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 msgr2=0x55c90b613720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20e000640 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b613720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b613720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.212+0000 7fd20ea00640 1 -- 172.21.15.59:0/832274008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c90b28b3f0 con 0x55c90b63ff20 2024-04-23T14:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.213+0000 7fd20ea00640 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b6131e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd200032fd0 tx=0x7fd200049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467a053ccff4c7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.213+0000 7fd1faa00640 1 -- 172.21.15.59:0/832274008 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200032e00 con 0x55c90b63ff20 2024-04-23T14:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.213+0000 7fd1faa00640 1 -- 172.21.15.59:0/832274008 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd200004420 con 0x55c90b63ff20 2024-04-23T14:42:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.213+0000 7fd1faa00640 1 -- 172.21.15.59:0/832274008 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200003b60 con 0x55c90b63ff20 2024-04-23T14:42:39.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 -- 172.21.15.59:0/832274008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 msgr2=0x55c90b6131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b6131e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd200032fd0 tx=0x7fd200049000 comp rx=0 tx=0).stop 2024-04-23T14:42:39.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 -- 172.21.15.59:0/832274008 shutdown_connections 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b66d960 0x55c90b613e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b65ce00 0x55c90b613720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 --2- 172.21.15.59:0/832274008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b6131e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 -- 172.21.15.59:0/832274008 >> 172.21.15.59:0/832274008 conn(0x55c90b63f7e0 msgr2=0x55c90b614f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 -- 172.21.15.59:0/832274008 shutdown_connections 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.214+0000 7fd21863d740 1 -- 172.21.15.59:0/832274008 wait complete. 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.215+0000 7fd21863d740 1 -- start start 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.215+0000 7fd21863d740 1 Processor -- start 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd21863d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b658820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b65ce00 0x55c90b658d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd21863d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b66d960 0x55c90b639180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd21863d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c90b611680 con 0x55c90b63ff20 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd210159ce0 con 0x55c90b66d960 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd21863d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55c90b51e790 con 0x55c90b65ce00 2024-04-23T14:42:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd21863d740 10 client.0 _ll_register_callbacks cb 0x7fd2000096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b658820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b658820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35540/0 (socket says 172.21.15.59:35540) 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.216+0000 7fd20ea00640 1 -- 172.21.15.59:0/3656253470 learned_addr learned my addr 172.21.15.59:0/3656253470 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd20ea00640 1 -- 172.21.15.59:0/3656253470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b65ce00 msgr2=0x55c90b658d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd20ea00640 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b65ce00 0x55c90b658d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd20ea00640 1 -- 172.21.15.59:0/3656253470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b66d960 msgr2=0x55c90b639180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd20ea00640 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b66d960 0x55c90b639180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd20ea00640 1 -- 172.21.15.59:0/3656253470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c90b24b0f0 con 0x55c90b63ff20 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd20ea00640 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b658820 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd20002a740 tx=0x7fd200008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb0c92682099a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd1efe00640 1 -- 172.21.15.59:0/3656253470 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20005ca60 con 0x55c90b63ff20 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd1efe00640 1 -- 172.21.15.59:0/3656253470 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fd20005cc00 con 0x55c90b63ff20 2024-04-23T14:42:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.217+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd200004420 con 0x55c90b63ff20 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd1efe00640 1 -- 172.21.15.59:0/3656253470 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd20005ce00 con 0x55c90b63ff20 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 10 client.9030 fetch_fsmap learned FSMap version 88 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55c90b53e810 con 0x55c90b63ff20 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd1efe00640 1 -- 172.21.15.59:0/3656253470 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd200004420 con 0x55c90b63ff20 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd1efe00640 1 -- 172.21.15.59:0/3656253470 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fd20006b020 con 0x55c90b63ff20 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 10 client.9030 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 10 client.9030 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 20 client.9030 populate_metadata read hostname 'smithi059' 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b613e70 0x55c90b6396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 4 client.9030 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c90b610e50 con 0x55c90b613e70 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.218+0000 7fd20e000640 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b613e70 0x55c90b6396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd20e000640 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b613e70 0x55c90b6396c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a2ff84b1def556 server_cookie=ccdd16885a7507e in_seq=0 out_seq=0 2024-04-23T14:42:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd1efe00640 10 client.9030 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd1efe00640 1 -- 172.21.15.59:0/3656253470 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55c90b610e50 con 0x55c90b613e70 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd1efe00640 10 client.9030 handle_command_reply: tid=0 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 client.9030 shutdown 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 msgr2=0x55c90b658820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b658820 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd20002a740 tx=0x7fd200008030 comp rx=0 tx=0).stop 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 shutdown_connections 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55c90b65ce00 0x55c90b658d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55c90b613e70 0x55c90b6396c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c90b66d960 0x55c90b639180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 --2- 172.21.15.59:0/3656253470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c90b63ff20 0x55c90b658820 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd1efe00640 0 client.9030 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:42:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.219+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 >> 172.21.15.59:0/3656253470 conn(0x55c90b63f7e0 msgr2=0x55c90b66c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.220+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 shutdown_connections 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.220+0000 7fd21863d740 1 -- 172.21.15.59:0/3656253470 wait complete. 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.220+0000 7fd21863d740 20 client.9030 trim_cache size 0 max 16384 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": { 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stdout: "43b3ed15-9260-40bd-9efb-ae2d23963b5f": { 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/", 2024-04-23T14:42:39.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tag": "43b3ed15-9260-40bd-9efb-ae2d23963b5f", 2024-04-23T14:42:39.228 INFO:teuthology.orchestra.run.smithi059.stdout: "options": "recursive,repair" 2024-04-23T14:42:39.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:42:39.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:42:39.228 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.220+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f006c8f0 msgr2=0x7fd1f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.220+0000 7fd20fe00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f006c8f0 0x7fd1f006edb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd200034000 tx=0x7fd20002e000 comp rx=0 tx=0).stop 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.220+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 msgr2=0x7fd2100ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.220+0000 7fd20fe00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100ccaf0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd208007e70 tx=0x7fd20800e4b0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 shutdown_connections 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd1f006c8f0 0x7fd1f006edb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2100d13e0 0x7fd2100cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21015ed20 0x7fd2100ccaf0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 --2- 172.21.15.59:0/1984952457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2100b3600 0x7fd2100a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 >> 172.21.15.59:0/1984952457 conn(0x7fd21008bf90 msgr2=0x7fd2100d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 shutdown_connections 2024-04-23T14:42:39.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.221+0000 7fd20fe00640 1 -- 172.21.15.59:0/1984952457 wait complete. 2024-04-23T14:42:39.237 INFO:tasks.cephfs.filesystem:scrub status for tag:43b3ed15-9260-40bd-9efb-ae2d23963b5f - {'path': '/', 'tag': '43b3ed15-9260-40bd-9efb-ae2d23963b5f', 'options': 'recursive,repair'} 2024-04-23T14:42:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:39.795+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-23T14:42:38.957566+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f8bac0047d0 con 0x7f8bb415fdb0 2024-04-23T14:42:39.796 INFO:teuthology.orchestra.run.smithi059.stdout: cluster: 2024-04-23T14:42:39.796 INFO:teuthology.orchestra.run.smithi059.stdout: id: b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:42:39.796 INFO:teuthology.orchestra.run.smithi059.stdout: health: HEALTH_OK 2024-04-23T14:42:39.796 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:39.796 INFO:teuthology.orchestra.run.smithi059.stdout: services: 2024-04-23T14:42:39.796 INFO:teuthology.orchestra.run.smithi059.stdout: mon: 3 daemons, quorum a,b,c (age 11m) 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: mgr: y(active, since 11m), standbys: x 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: mds: 1/1 daemons up, 3 standby 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: osd: 8 osds: 8 up (since 11m), 8 in (since 11m) 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: data: 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: volumes: 1/1 healthy 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: pools: 3 pools, 97 pgs 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: objects: 30 objects, 19 MiB 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: pgs: 97 active+clean 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: io: 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:39.797 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.957566+0000 mds.a [INF] scrub queued for path: / 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.957568+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.957573+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.958407+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.958414+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.958526+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.958532+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-04-23T14:42:39.798 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:38.959813+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-23T14:42:43.807 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:42:42.702367+0000 mds.a [INF] scrub summary: idle 2024-04-23T14:42:43.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:42:43.807+0000 7f8bb1600640 1 -- 172.21.15.59:0/3686846165 <== mon.0 v2:172.21.15.59:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-23T14:42:42.702367+0000) ==== 189+0+0 (secure 0 0 0) 0x7f8bac078a40 con 0x7f8bb415fdb0 2024-04-23T14:42:51.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:42:51.648 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:43:09.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:09.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-23T14:43:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.451+0000 7f7f4e600640 1 Processor -- start 2024-04-23T14:43:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.452+0000 7f7f4e600640 1 -- start start 2024-04-23T14:43:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.453+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f480b4f20 0x7f7f480cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.453+0000 7f7f4e600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f480b5da0 con 0x7f7f480d15b0 2024-04-23T14:43:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.453+0000 7f7f4e600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f480d0a20 con 0x7f7f4815ed20 2024-04-23T14:43:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.453+0000 7f7f4e600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4815f100 con 0x7f7f480b4f20 2024-04-23T14:43:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.454+0000 7f7f47e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7f480d15b0 0x7f7f480cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40624/0 (socket says 172.21.15.59:40624) 2024-04-23T14:43:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.454+0000 7f7f47e00640 1 -- 172.21.15.59:0/3461561853 learned_addr learned my addr 172.21.15.59:0/3461561853 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.454+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2427555263 0 0) 0x7f7f480b5da0 con 0x7f7f480d15b0 2024-04-23T14:43:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.454+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f2c003520 con 0x7f7f480d15b0 2024-04-23T14:43:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.454+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f480b4f20 0x7f7f480cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.454+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 524082582 0 0) 0x7f7f2c003520 con 0x7f7f480d15b0 2024-04-23T14:43:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f480b4f20 msgr2=0x7f7f480cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f480b4f20 0x7f7f480cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 >> v1:172.21.15.76:6789/0 conn(0x7f7f4815ed20 legacy=0x7f7f480d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f480d2ed0 con 0x7f7f480d15b0 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f7f34001670 con 0x7f7f480d15b0 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 >> v1:172.21.15.59:6789/0 conn(0x7f7f480d15b0 legacy=0x7f7f480cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f2c0039e0 0x7f7f2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f480b4f20 0x7f7f2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c005540 con 0x7f7f2c004800 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c00bf00 con 0x7f7f2c0039e0 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c00c080 con 0x7f7f480b4f20 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47e00640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f4d200640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f2c0039e0 0x7f7f2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47e00640 1 -- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f480b4f20 msgr2=0x7f7f2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47e00640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f480b4f20 0x7f7f2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47e00640 1 -- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f2c0039e0 msgr2=0x7f7f2c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47e00640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f2c0039e0 0x7f7f2c003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.455+0000 7f7f47e00640 1 -- 172.21.15.59:0/3461561853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c003520 con 0x7f7f2c004800 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f47e00640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f2c004c80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400c210 tx=0x7f7f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa0132b5c16931a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34007d30 con 0x7f7f2c004800 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f34005ad0 con 0x7f7f2c004800 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f47400640 1 -- 172.21.15.59:0/3461561853 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34007070 con 0x7f7f2c004800 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f4e600640 1 -- 172.21.15.59:0/3461561853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 msgr2=0x7f7f2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f4e600640 1 --2- 172.21.15.59:0/3461561853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f2c004c80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400c210 tx=0x7f7f2c005830 comp rx=0 tx=0).stop 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f4dc00640 1 -- 172.21.15.59:0/3461561853 reap_dead start 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f4e600640 1 -- 172.21.15.59:0/3461561853 shutdown_connections 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.457+0000 7f7f4e600640 1 -- 172.21.15.59:0/3461561853 >> 172.21.15.59:0/3461561853 conn(0x7f7f4808bf90 msgr2=0x7f7f480d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.458+0000 7f7f4e600640 1 -- 172.21.15.59:0/3461561853 shutdown_connections 2024-04-23T14:43:09.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.458+0000 7f7f4e600640 1 -- 172.21.15.59:0/3461561853 wait complete. 2024-04-23T14:43:09.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.459+0000 7f7f4e600640 1 Processor -- start 2024-04-23T14:43:09.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.460+0000 7f7f4e600640 1 -- start start 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.460+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f481e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.460+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4815d5f0 0x7f7f480cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.460+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f480cbde0 0x7f7f480cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.460+0000 7f7f4e600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480b3600 con 0x7f7f2c004800 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.460+0000 7f7f4e600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480d3490 con 0x7f7f480cbde0 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.460+0000 7f7f4e600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f480b3780 con 0x7f7f4815d5f0 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f481e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f481e6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47370/0 (socket says 172.21.15.59:47370) 2024-04-23T14:43:09.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 -- 172.21.15.59:0/454579254 learned_addr learned my addr 172.21.15.59:0/454579254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f480cbde0 0x7f7f480cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f47e00640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4815d5f0 0x7f7f480cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 -- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4815d5f0 msgr2=0x7f7f480cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4815d5f0 0x7f7f480cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 -- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f480cbde0 msgr2=0x7f7f480cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f480cbde0 0x7f7f480cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4d200640 1 -- 172.21.15.59:0/454579254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f480cc460 con 0x7f7f2c004800 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.461+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f480cbde0 0x7f7f480cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.462+0000 7f7f4d200640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f481e6220 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800d570 tx=0x7f7f3800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.462+0000 7f7f44c00640 1 -- 172.21.15.59:0/454579254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38005480 con 0x7f7f2c004800 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.462+0000 7f7f44c00640 1 -- 172.21.15.59:0/454579254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38017070 con 0x7f7f2c004800 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.462+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f480b8220 con 0x7f7f2c004800 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.462+0000 7f7f44c00640 1 -- 172.21.15.59:0/454579254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38003810 con 0x7f7f2c004800 2024-04-23T14:43:09.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.462+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f480b87e0 con 0x7f7f2c004800 2024-04-23T14:43:09.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.464+0000 7f7f44c00640 1 -- 172.21.15.59:0/454579254 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f380039b0 con 0x7f7f2c004800 2024-04-23T14:43:09.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.466+0000 7f7f44c00640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2806c920 0x7f7f2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.466+0000 7f7f44c00640 1 -- 172.21.15.59:0/454579254 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f7f3808b9a0 con 0x7f7f2c004800 2024-04-23T14:43:09.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.466+0000 7f7f47e00640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2806c920 0x7f7f2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.467+0000 7f7f52a6b740 1 Processor -- start 2024-04-23T14:43:09.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.467+0000 7f7f52a6b740 1 -- start start 2024-04-23T14:43:09.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.467+0000 7f7f47e00640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2806c920 0x7f7f2806ede0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7f34033000 tx=0x7f7f3402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f87fe00 0x55b12f832790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f8622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f8627e0 0x55b12f8375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b12f79c1f0 con 0x55b12f87fe00 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b12f742350 con 0x55b12f862f20 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55b12f7981b0 con 0x55b12f8627e0 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f87fe00 0x55b12f832790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f87fe00 0x55b12f832790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47378/0 (socket says 172.21.15.59:47378) 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 -- 172.21.15.59:0/3882126440 learned_addr learned my addr 172.21.15.59:0/3882126440 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 -- 172.21.15.59:0/3882126440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f8627e0 msgr2=0x55b12f8375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 --2- 172.21.15.59:0/3882126440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f8627e0 0x55b12f8375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 -- 172.21.15.59:0/3882126440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 msgr2=0x55b12f8622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 --2- 172.21.15.59:0/3882126440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f8622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.468+0000 7f7f4d200640 1 -- 172.21.15.59:0/3882126440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12f796f50 con 0x55b12f87fe00 2024-04-23T14:43:09.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.469+0000 7f7f4d200640 1 --2- 172.21.15.59:0/3882126440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f87fe00 0x55b12f832790 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7f38055ac0 tx=0x7f7f38059ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1d65992bb8c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.469+0000 7f7f3d600640 1 -- 172.21.15.59:0/3882126440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38062240 con 0x55b12f87fe00 2024-04-23T14:43:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.469+0000 7f7f3d600640 1 -- 172.21.15.59:0/3882126440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38057960 con 0x55b12f87fe00 2024-04-23T14:43:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.469+0000 7f7f3d600640 1 -- 172.21.15.59:0/3882126440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38003ca0 con 0x55b12f87fe00 2024-04-23T14:43:09.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 -- 172.21.15.59:0/3882126440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f87fe00 msgr2=0x55b12f832790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/3882126440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f87fe00 0x55b12f832790 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7f38055ac0 tx=0x7f7f38059ea0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 -- 172.21.15.59:0/3882126440 shutdown_connections 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/3882126440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f8627e0 0x55b12f8375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/3882126440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f8622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/3882126440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f87fe00 0x55b12f832790 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 -- 172.21.15.59:0/3882126440 >> 172.21.15.59:0/3882126440 conn(0x55b12f834a00 msgr2=0x55b12f87d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 -- 172.21.15.59:0/3882126440 shutdown_connections 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.470+0000 7f7f52a6b740 1 -- 172.21.15.59:0/3882126440 wait complete. 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.471+0000 7f7f52a6b740 1 -- start start 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.471+0000 7f7f52a6b740 1 Processor -- start 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f873f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f865a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b12f836bb0 con 0x55b12f85df40 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b12f755690 con 0x55b12f862f20 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55b12f835fa0 con 0x55b12f87fe00 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f52a6b740 10 client.0 _ll_register_callbacks cb 0x55b12f791650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f4dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f4dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47388/0 (socket says 172.21.15.59:47388) 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f4dc00640 1 -- 172.21.15.59:0/2547771998 learned_addr learned my addr 172.21.15.59:0/2547771998 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.472+0000 7f7f47e00640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f865a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f873f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4dc00640 1 -- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 msgr2=0x55b12f865a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f865a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4dc00640 1 -- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 msgr2=0x55b12f873f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f873f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4dc00640 1 -- 172.21.15.59:0/2547771998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12f754920 con 0x55b12f85df40 2024-04-23T14:43:09.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f873f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865fc0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7f480d1c80 tx=0x7f7f40007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfb299cf0e9350a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f32000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f32000640 1 -- 172.21.15.59:0/2547771998 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40018020 con 0x55b12f85df40 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f32000640 1 -- 172.21.15.59:0/2547771998 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f7f4000fa30 con 0x55b12f85df40 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.473+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f38057960 con 0x55b12f85df40 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f32000640 1 -- 172.21.15.59:0/2547771998 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7f4000fc30 con 0x55b12f85df40 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 10 client.9042 fetch_fsmap learned FSMap version 88 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55b12f5787d0 con 0x55b12f85df40 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f32000640 1 -- 172.21.15.59:0/2547771998 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38057960 con 0x55b12f85df40 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f32000640 1 -- 172.21.15.59:0/2547771998 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7f40026080 con 0x55b12f85df40 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 10 client.9042 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 10 client.9042 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 20 client.9042 populate_metadata read hostname 'smithi059' 2024-04-23T14:43:09.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f863f00 0x55b12f864330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 4 client.9042 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.474+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12f7a9c10 con 0x55b12f863f00 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.475+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f863f00 0x55b12f864330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.475+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f863f00 0x55b12f864330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3debd4e4941d7c03 server_cookie=7ee4ca97c1d02258 in_seq=0 out_seq=0 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.475+0000 7f7f32000640 10 client.9042 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f32000640 1 -- 172.21.15.59:0/2547771998 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b12f7a9c10 con 0x55b12f863f00 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f32000640 10 client.9042 handle_command_reply: tid=0 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 client.9042 shutdown 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 msgr2=0x55b12f865fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865fc0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7f480d1c80 tx=0x7f7f40007d60 comp rx=0 tx=0).stop 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 shutdown_connections 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f865a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f863f00 0x55b12f864330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865fc0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2547771998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f873f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f32000640 0 client.9042 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:43:09.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 >> 172.21.15.59:0/2547771998 conn(0x55b12f8627e0 msgr2=0x55b12f87f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:09.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 shutdown_connections 2024-04-23T14:43:09.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.476+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2547771998 wait complete. 2024-04-23T14:43:09.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.477+0000 7f7f52a6b740 20 client.9042 trim_cache size 0 max 16384 2024-04-23T14:43:09.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.496+0000 7f7f52a6b740 1 Processor -- start 2024-04-23T14:43:09.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.496+0000 7f7f52a6b740 1 -- start start 2024-04-23T14:43:09.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f88cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f862f20 0x55b12f865830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f87fe00 0x55b12f865d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48201390 con 0x55b12f85df40 2024-04-23T14:43:09.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48200450 con 0x55b12f87fe00 2024-04-23T14:43:09.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f480c9a00 con 0x55b12f862f20 2024-04-23T14:43:09.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f88cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f88cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47392/0 (socket says 172.21.15.59:47392) 2024-04-23T14:43:09.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.497+0000 7f7f4d200640 1 -- 172.21.15.59:0/2397912481 learned_addr learned my addr 172.21.15.59:0/2397912481 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:09.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f4d200640 1 -- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f862f20 msgr2=0x55b12f865830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f47e00640 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f862f20 0x55b12f865830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f87fe00 0x55b12f865d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f862f20 0x55b12f865830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f4d200640 1 -- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f87fe00 msgr2=0x55b12f865d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f87fe00 0x55b12f865d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f4d200640 1 -- 172.21.15.59:0/2397912481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12f4ae3f0 con 0x55b12f85df40 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f47e00640 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f862f20 0x55b12f865830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f88cdf0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800df30 tx=0x7f7f3806f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4091fa215fcbae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f3cc00640 1 -- 172.21.15.59:0/2397912481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38057780 con 0x55b12f85df40 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f3cc00640 1 -- 172.21.15.59:0/2397912481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38003ca0 con 0x55b12f85df40 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.498+0000 7f7f3cc00640 1 -- 172.21.15.59:0/2397912481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3805eda0 con 0x55b12f85df40 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2397912481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 msgr2=0x55b12f88cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f88cdf0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800df30 tx=0x7f7f3806f000 comp rx=0 tx=0).stop 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2397912481 shutdown_connections 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f862f20 0x55b12f865830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f87fe00 0x55b12f865d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2397912481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f88cdf0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2397912481 >> 172.21.15.59:0/2397912481 conn(0x55b12f8627e0 msgr2=0x55b12f876440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:09.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2397912481 shutdown_connections 2024-04-23T14:43:09.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.499+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2397912481 wait complete. 2024-04-23T14:43:09.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.500+0000 7f7f52a6b740 1 -- start start 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.500+0000 7f7f52a6b740 1 Processor -- start 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f865d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f52a6b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f85f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481fffc0 con 0x55b12f85df40 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b12f85c180 con 0x55b12f862f20 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f52a6b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55b12f8ffc30 con 0x55b12f87fe00 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f52a6b740 10 client.0 _ll_register_callbacks cb 0x7f7f400096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.501+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47406/0 (socket says 172.21.15.59:47406) 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 -- 172.21.15.59:0/2626419711 learned_addr learned my addr 172.21.15.59:0/2626419711 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f85f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f47e00640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f865d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 -- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 msgr2=0x55b12f85f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f85f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 -- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 msgr2=0x55b12f865d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f865d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 -- 172.21.15.59:0/2626419711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12f46e0f0 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4dc00640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f85f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f4d200640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865800 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005620 tx=0x7f7f3801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26202da9fcabdb8c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f32000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.502+0000 7f7f32000640 1 -- 172.21.15.59:0/2626419711 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3807d060 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f32000640 1 -- 172.21.15.59:0/2626419711 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7f7f3805eda0 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f38003ca0 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f32000640 1 -- 172.21.15.59:0/2626419711 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7f38083050 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 10 client.9048 fetch_fsmap learned FSMap version 88 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55b12f761810 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f32000640 1 -- 172.21.15.59:0/2626419711 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38003ca0 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f32000640 1 -- 172.21.15.59:0/2626419711 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7f38058ab0 con 0x55b12f85df40 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 10 client.9048 fetch_fsmap finished waiting for FSMap version 88 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 10 client.9048 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-23T14:43:09.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 20 client.9048 populate_metadata read hostname 'smithi059' 2024-04-23T14:43:09.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f8e8ca0 0x55b12f85f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 4 client.9048 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:43:09.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.503+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 --> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b12f7a3a80 con 0x55b12f8e8ca0 2024-04-23T14:43:09.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.504+0000 7f7f47e00640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f8e8ca0 0x55b12f85f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.504+0000 7f7f47e00640 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f8e8ca0 0x55b12f85f750 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11d2ecce165d53f server_cookie=ac62b959d9656008 in_seq=0 out_seq=0 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.504+0000 7f7f32000640 10 client.9048 ms_handle_connect on v2:172.21.15.59:6835/2265807893 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.504+0000 7f7f32000640 1 -- 172.21.15.59:0/2626419711 <== mds.0 v2:172.21.15.59:6835/2265807893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b12f7a3a80 con 0x55b12f8e8ca0 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.504+0000 7f7f32000640 10 client.9048 handle_command_reply: tid=0 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.504+0000 7f7f52a6b740 1 client.9048 shutdown 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 msgr2=0x55b12f865800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865800 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005620 tx=0x7f7f3801e040 comp rx=0 tx=0).stop 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 shutdown_connections 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b12f87fe00 0x55b12f85f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:6835/2265807893,v1:172.21.15.59:6837/2265807893] conn(0x55b12f8e8ca0 0x55b12f85f750 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b12f862f20 0x55b12f865d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 --2- 172.21.15.59:0/2626419711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b12f85df40 0x55b12f865800 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f32000640 0 client.9048 ms_handle_reset on v2:172.21.15.59:6835/2265807893 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 >> 172.21.15.59:0/2626419711 conn(0x55b12f8627e0 msgr2=0x55b12f88cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 shutdown_connections 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 1 -- 172.21.15.59:0/2626419711 wait complete. 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.505+0000 7f7f52a6b740 20 client.9048 trim_cache size 0 max 16384 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "status": "no active scrubs running", 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stdout: "scrubs": {} 2024-04-23T14:43:09.509 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:43:09.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2806c920 msgr2=0x7f7f2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2806c920 0x7f7f2806ede0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7f34033000 tx=0x7f7f3402e000 comp rx=0 tx=0).stop 2024-04-23T14:43:09.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 msgr2=0x7f7f481e6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f481e6220 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800d570 tx=0x7f7f3800da30 comp rx=0 tx=0).stop 2024-04-23T14:43:09.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 shutdown_connections 2024-04-23T14:43:09.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7f2806c920 0x7f7f2806ede0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f4815d5f0 0x7f7f480cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f480cbde0 0x7f7f480cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 --2- 172.21.15.59:0/454579254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f2c004800 0x7f7f481e6220 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 >> 172.21.15.59:0/454579254 conn(0x7f7f4808bf90 msgr2=0x7f7f4815dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:09.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 shutdown_connections 2024-04-23T14:43:09.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.506+0000 7f7f4e600640 1 -- 172.21.15.59:0/454579254 wait complete. 2024-04-23T14:43:09.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:43:09.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:43:09.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.843+0000 7fbf3be00640 1 Processor -- start 2024-04-23T14:43:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.843+0000 7fbf3be00640 1 -- start start 2024-04-23T14:43:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf3c15c670 0x7fbf3c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf3c0d05c0 0x7fbf3c0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3be00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3c15ca50 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3c0ccb80 con 0x7fbf3c15c670 2024-04-23T14:43:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c0b4cf0 con 0x7fbf3c0b4230 2024-04-23T14:43:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf3c0d05c0 0x7fbf3c0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf3c0d05c0 0x7fbf3c0d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47410/0 (socket says 172.21.15.59:47410) 2024-04-23T14:43:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.845+0000 7fbf3b400640 1 -- 172.21.15.59:0/565362556 learned_addr learned my addr 172.21.15.59:0/565362556 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.846+0000 7fbf3aa00640 1 --2- 172.21.15.59:0/565362556 >> v2:172.21.15.76:3300/0 conn(0x7fbf3c15c670 0x7fbf3c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.846+0000 7fbf32000640 1 --2- 172.21.15.59:0/565362556 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.846+0000 7fbf3b400640 1 -- 172.21.15.59:0/565362556 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 msgr2=0x7fbf3c0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.846+0000 7fbf3b400640 1 --2- 172.21.15.59:0/565362556 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c0b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.846+0000 7fbf3b400640 1 -- 172.21.15.59:0/565362556 >> v2:172.21.15.76:3300/0 conn(0x7fbf3c15c670 msgr2=0x7fbf3c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.846+0000 7fbf3b400640 1 --2- 172.21.15.59:0/565362556 >> v2:172.21.15.76:3300/0 conn(0x7fbf3c15c670 0x7fbf3c0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.846+0000 7fbf3b400640 1 -- 172.21.15.59:0/565362556 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c0ccdd0 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.847+0000 7fbf3b400640 1 --2- 172.21.15.59:0/565362556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0d0a40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0d1270 tx=0x7fbf3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3209a88d680bd807 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.847+0000 7fbf3a000640 1 -- 172.21.15.59:0/565362556 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34036070 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3a000640 1 -- 172.21.15.59:0/565362556 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf34031600 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3a000640 1 -- 172.21.15.59:0/565362556 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340319d0 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 -- 172.21.15.59:0/565362556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 msgr2=0x7fbf3c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 --2- 172.21.15.59:0/565362556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0d0a40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0d1270 tx=0x7fbf3402a6a0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 -- 172.21.15.59:0/565362556 shutdown_connections 2024-04-23T14:43:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 --2- 172.21.15.59:0/565362556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0d0a40 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 --2- 172.21.15.59:0/565362556 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c0b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 --2- 172.21.15.59:0/565362556 >> v2:172.21.15.76:3300/0 conn(0x7fbf3c15c670 0x7fbf3c0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 -- 172.21.15.59:0/565362556 >> 172.21.15.59:0/565362556 conn(0x7fbf3c06bdf0 msgr2=0x7fbf3c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.848+0000 7fbf3be00640 1 -- 172.21.15.59:0/565362556 shutdown_connections 2024-04-23T14:43:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.849+0000 7fbf3be00640 1 -- 172.21.15.59:0/565362556 wait complete. 2024-04-23T14:43:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.850+0000 7fbf3be00640 1 Processor -- start 2024-04-23T14:43:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.850+0000 7fbf3be00640 1 -- start start 2024-04-23T14:43:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.851+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.851+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.851+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf3c15c670 0x7fbf3c0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.851+0000 7fbf3be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c0b3e60 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.851+0000 7fbf3be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c0cd390 con 0x7fbf3c15c670 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.851+0000 7fbf3be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c0b3fe0 con 0x7fbf3c0b4230 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47414/0 (socket says 172.21.15.59:47414) 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 -- 172.21.15.59:0/1610087863 learned_addr learned my addr 172.21.15.59:0/1610087863 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf3b400640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf3c15c670 0x7fbf3c0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf3aa00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 -- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 msgr2=0x7fbf3c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 -- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf3c15c670 msgr2=0x7fbf3c0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf3c15c670 0x7fbf3c0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf32000640 1 -- 172.21.15.59:0/1610087863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c0b2ac0 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.852+0000 7fbf3b400640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf3c15c670 0x7fbf3c0b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.853+0000 7fbf32000640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0b1f50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400d7d0 tx=0x7fbf2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.853+0000 7fbf33e00640 1 -- 172.21.15.59:0/1610087863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24018030 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.853+0000 7fbf33e00640 1 -- 172.21.15.59:0/1610087863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf24004020 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.853+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c148b50 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.853+0000 7fbf33e00640 1 -- 172.21.15.59:0/1610087863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24004230 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.853+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1f7ff0 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.854+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c04fd90 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.856+0000 7fbf33e00640 1 -- 172.21.15.59:0/1610087863 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf2401b070 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.857+0000 7fbf33e00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf1806c8f0 0x7fbf1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.857+0000 7fbf33e00640 1 -- 172.21.15.59:0/1610087863 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5795+0+0 (secure 0 0 0) 0x7fbf2408bda0 con 0x7fbf3c0d05c0 2024-04-23T14:43:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.857+0000 7fbf3aa00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf1806c8f0 0x7fbf1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.858+0000 7fbf3aa00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf1806c8f0 0x7fbf1806edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006fd0 tx=0x7fbf2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:09.862+0000 7fbf33e00640 1 -- 172.21.15.59:0/1610087863 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf2405a0d0 con 0x7fbf3c0d05c0 2024-04-23T14:43:10.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.117+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbf3c0ceab0 con 0x7fbf3c0d05c0 2024-04-23T14:43:10.119 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:43:10.118+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:43:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.121+0000 7fbf33e00640 1 -- 172.21.15.59:0/1610087863 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7fbf240599d0 con 0x7fbf3c0d05c0 2024-04-23T14:43:10.122 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:43:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.124+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf1806c8f0 msgr2=0x7fbf1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.124+0000 7fbf3be00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf1806c8f0 0x7fbf1806edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006fd0 tx=0x7fbf2c008040 comp rx=0 tx=0).stop 2024-04-23T14:43:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.124+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 msgr2=0x7fbf3c0b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.124+0000 7fbf3be00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0b1f50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400d7d0 tx=0x7fbf2400dc90 comp rx=0 tx=0).stop 2024-04-23T14:43:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 shutdown_connections 2024-04-23T14:43:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbf1806c8f0 0x7fbf1806edb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf3c15c670 0x7fbf3c0b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf3c0d05c0 0x7fbf3c0b1f50 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 --2- 172.21.15.59:0/1610087863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf3c0b4230 0x7fbf3c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 >> 172.21.15.59:0/1610087863 conn(0x7fbf3c06bdf0 msgr2=0x7fbf3c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:10.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 shutdown_connections 2024-04-23T14:43:10.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.125+0000 7fbf3be00640 1 -- 172.21.15.59:0/1610087863 wait complete. 2024-04-23T14:43:10.133 INFO:tasks.ceph.mds.a.smithi059.stderr: -1332> 2024-04-23T14:42:38.642+0000 7ff6eb600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:43:10.134 INFO:tasks.ceph.mds.a.smithi059.stderr: -1145> 2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-23T14:43:10.134 INFO:tasks.ceph.mds.a.smithi059.stderr: -1142> 2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-23T14:43:10.134 INFO:tasks.ceph.mds.a.smithi059.stderr: -1126> 2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-23T14:43:10.134 INFO:tasks.ceph.mds.a.smithi059.stderr: -1123> 2024-04-23T14:42:38.958+0000 7ff6e6600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-23T14:43:10.145 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-23T14:43:10.185 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:43:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.224+0000 7f755c920280 1 -- start start 2024-04-23T14:43:10.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.224+0000 7f755c920280 1 Processor -- start 2024-04-23T14:43:10.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755c920280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bafa4ec00 0x555bb0822000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755c920280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bafa4f800 0x555bb0822580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755c920280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bafa4fc00 0x555bb0822b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755c920280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555bafbc4180 con 0x555bafa4ec00 2024-04-23T14:43:10.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755c920280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555bafbc4300 con 0x555bafa4f800 2024-04-23T14:43:10.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755c920280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555bafbc4480 con 0x555bafa4fc00 2024-04-23T14:43:10.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bafa4ec00 0x555bb0822000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bafa4ec00 0x555bb0822000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45998/0 (socket says 172.21.15.59:45998) 2024-04-23T14:43:10.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f755a000640 1 -- 172.21.15.59:0/3483342798 learned_addr learned my addr 172.21.15.59:0/3483342798 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:10.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.225+0000 7f7559600640 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bafa4f800 0x555bb0822580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755a000640 1 -- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bafa4fc00 msgr2=0x555bb0822b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:10.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755a000640 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bafa4fc00 0x555bb0822b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755a000640 1 -- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bafa4f800 msgr2=0x555bb0822580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:10.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755a000640 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bafa4f800 0x555bb0822580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755a000640 1 -- 172.21.15.59:0/3483342798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bafa82780 con 0x555bafa4ec00 2024-04-23T14:43:10.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f7559600640 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bafa4f800 0x555bb0822580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:10.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755a000640 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bafa4ec00 0x555bb0822000 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x555bb087f1d0 tx=0x555bb07ea900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706f475171ab0da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:10.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f7556e00640 1 -- 172.21.15.59:0/3483342798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555bafa74680 con 0x555bafa4ec00 2024-04-23T14:43:10.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f7556e00640 1 -- 172.21.15.59:0/3483342798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555bafa7a700 con 0x555bafa4ec00 2024-04-23T14:43:10.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f7556e00640 1 -- 172.21.15.59:0/3483342798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555bafa74b60 con 0x555bafa4ec00 2024-04-23T14:43:10.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755c920280 1 -- 172.21.15.59:0/3483342798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bafa834a0 con 0x555bafa4ec00 2024-04-23T14:43:10.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755c920280 1 -- 172.21.15.59:0/3483342798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555bafa82000 con 0x555bafa4ec00 2024-04-23T14:43:10.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f755c920280 4 waiting for MDS map... 2024-04-23T14:43:10.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.226+0000 7f7556e00640 1 -- 172.21.15.59:0/3483342798 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x555bb0914600 con 0x555bafa4ec00 2024-04-23T14:43:10.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.227+0000 7f7556e00640 1 -- 172.21.15.59:0/3483342798 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 89) ==== 2885+0+0 (secure 0 0 0) 0x555bafaa9180 con 0x555bafa4ec00 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.227+0000 7f755c920280 4 Got MDS map 89 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.227+0000 7f755c920280 10 main: main 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.227+0000 7f755c920280 4 main: connecting to RADOS... 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.227+0000 7f755c920280 1 Processor -- start 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.228+0000 7f755c920280 1 -- start start 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.228+0000 7f755c920280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917400 0x555bb0823080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.228+0000 7f755c920280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917800 0x555bb0823600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.228+0000 7f755c920280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.228+0000 7f755c920280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555bafbc4600 con 0x555bb0917c00 2024-04-23T14:43:10.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.228+0000 7f755c920280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555bafbc4780 con 0x555bb0917800 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.228+0000 7f755c920280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555bafbc4900 con 0x555bb0917400 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46010/0 (socket says 172.21.15.59:46010) 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 -- 172.21.15.59:0/1501192743 learned_addr learned my addr 172.21.15.59:0/1501192743 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f755a000640 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917800 0x555bb0823600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f755aa00640 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917400 0x555bb0823080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 -- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917400 msgr2=0x555bb0823080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917400 0x555bb0823080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 -- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917800 msgr2=0x555bb0823600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917800 0x555bb0823600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.229+0000 7f7559600640 1 -- 172.21.15.59:0/1501192743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bafa823c0 con 0x555bb0917c00 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f7559600640 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917c00 0x555bb0823b80 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x555bb087ffb0 tx=0x555bb07e88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e352b17986bc34ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f7554600640 1 -- 172.21.15.59:0/1501192743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555bafa749c0 con 0x555bb0917c00 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f7554600640 1 -- 172.21.15.59:0/1501192743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555bafa7afc0 con 0x555bb0917c00 2024-04-23T14:43:10.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f7554600640 1 -- 172.21.15.59:0/1501192743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555bafa74680 con 0x555bb0917c00 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 -- 172.21.15.59:0/1501192743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917c00 msgr2=0x555bb0823b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917c00 0x555bb0823b80 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x555bb087ffb0 tx=0x555bb07e88a0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 -- 172.21.15.59:0/1501192743 shutdown_connections 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917800 0x555bb0823600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 --2- 172.21.15.59:0/1501192743 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917400 0x555bb0823080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 -- 172.21.15.59:0/1501192743 >> 172.21.15.59:0/1501192743 conn(0x555bb0917000 msgr2=0x555bafb35000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.230+0000 7f755c920280 1 -- 172.21.15.59:0/1501192743 shutdown_connections 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.231+0000 7f755c920280 1 -- 172.21.15.59:0/1501192743 wait complete. 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.231+0000 7f755c920280 1 Processor -- start 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.231+0000 7f755c920280 1 -- start start 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755c920280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917400 0x555bb0823080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755c920280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917800 0x555bb0823600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755c920280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755c920280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555bafbc5b00 con 0x555bb0917400 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755c920280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555bafbc5980 con 0x555bb0917c00 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755c920280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x555bafbc5800 con 0x555bb0917800 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917400 0x555bb0823080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917400 0x555bb0823080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46018/0 (socket says 172.21.15.59:46018) 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.232+0000 7f755aa00640 1 -- 172.21.15.59:0/1397726074 learned_addr learned my addr 172.21.15.59:0/1397726074 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f7559600640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755aa00640 1 -- 172.21.15.59:0/1397726074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917800 msgr2=0x555bb0823600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755a000640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917800 0x555bb0823600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755aa00640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bb0917800 0x555bb0823600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755aa00640 1 -- 172.21.15.59:0/1397726074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917c00 msgr2=0x555bb0823b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755aa00640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755aa00640 1 -- 172.21.15.59:0/1397726074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bafa82780 con 0x555bb0917400 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f7559600640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bb0917c00 0x555bb0823b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755aa00640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917400 0x555bb0823080 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x555bb098b470 tx=0x555bb0930d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f7552800640 1 -- 172.21.15.59:0/1397726074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555bafa749c0 con 0x555bb0917400 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f7552800640 1 -- 172.21.15.59:0/1397726074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555bafa7ae00 con 0x555bb0917400 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f7552800640 1 -- 172.21.15.59:0/1397726074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555bafa74680 con 0x555bb0917400 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.233+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555bafa834a0 con 0x555bb0917400 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.234+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bafa82960 con 0x555bb0917400 2024-04-23T14:43:10.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.234+0000 7f755c920280 4 main: resolving pool 14 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.235+0000 7f7552800640 1 -- 172.21.15.59:0/1397726074 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x555bb0986c00 con 0x555bb0917400 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.236+0000 7f7552800640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555bb0858c00 0x555bb0824100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.236+0000 7f7552800640 1 -- 172.21.15.59:0/1397726074 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x555bb0990800 con 0x555bb0917400 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.236+0000 7f755c920280 4 main: creating IoCtx.. 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.236+0000 7f755c920280 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x555bb0859000 0x555bb0824680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.236+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x555bb0859400 con 0x555bb0859000 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.236+0000 7f755a000640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555bb0858c00 0x555bb0824100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.237+0000 7f7559600640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x555bb0859000 0x555bb0824680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.237+0000 7f755a000640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555bb0858c00 0x555bb0824100 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x555bb087f4d0 tx=0x555bb07e8600 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.237+0000 7f7559600640 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x555bb0859000 0x555bb0824680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f7559600640 1 -- 172.21.15.59:0/1397726074 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x555bb07d8000 con 0x555bb0859000 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stdout: "0": { 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stdout: "data": { 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 5, 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stdout: "inotable": { 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stdout: "projected_free": [ 2024-04-23T14:43:10.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: "start": 1099511627779, 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: "len": 1099511627773 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: "free": [ 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: "start": 1099511627779, 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: "len": 1099511627773 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: "result": 0 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:43:10.251 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x555bb0859000 msgr2=0x555bb0824680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f755c920280 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x555bb0859000 0x555bb0824680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555bb0858c00 msgr2=0x555bb0824100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f755c920280 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x555bb0858c00 0x555bb0824100 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x555bb087f4d0 tx=0x555bb07e8600 comp rx=0 tx=0).stop 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917400 msgr2=0x555bb0823080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f755c920280 1 --2- 172.21.15.59:0/1397726074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bb0917400 0x555bb0823080 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x555bb098b470 tx=0x555bb0930d20 comp rx=0 tx=0).stop 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.240+0000 7f7559600640 1 -- 172.21.15.59:0/1397726074 reap_dead start 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 shutdown_connections 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 >> 172.21.15.59:0/1397726074 conn(0x555bb0917000 msgr2=0x555bafb35000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 shutdown_connections 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/1397726074 wait complete. 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/3483342798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bafa4ec00 msgr2=0x555bb0822000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bafa4ec00 0x555bb0822000 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x555bb087f1d0 tx=0x555bb07ea900 comp rx=0 tx=0).stop 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/3483342798 shutdown_connections 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x555bafa4fc00 0x555bb0822b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bafa4f800 0x555bb0822580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 --2- 172.21.15.59:0/3483342798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555bafa4ec00 0x555bb0822000 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/3483342798 >> 172.21.15.59:0/3483342798 conn(0x555bafa4f000 msgr2=0x555bafb34800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:10.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.241+0000 7f755c920280 1 -- 172.21.15.59:0/3483342798 shutdown_connections 2024-04-23T14:43:10.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:10.242+0000 7f755c920280 1 -- 172.21.15.59:0/3483342798 wait complete. 2024-04-23T14:43:10.255 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.109832, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-23T14:43:10.255 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:43:10.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:43:10.408 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:43:10.510 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:43:10.510 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:43:10.510 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:43:10.510 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:10.571 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:43:10.572 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:43:10.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:10.572 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:43:10.626 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-23T14:43:10.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:43:10.628 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:43:10.628 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:43:10.628 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:10.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:43:10.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:43:10.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:10.684 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:43:10.734 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:43:10.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:43:10.735 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:43:10.735 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:43:10.735 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:10.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:43:10.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:43:10.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:10.797 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:43:10.852 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:43:10.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:43:10.854 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:43:10.854 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:43:10.854 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:10.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:43:10.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:43:10.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:10.909 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:43:10.955 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:43:10.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:43:10.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:43:11.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.178+0000 7f061b200640 1 Processor -- start 2024-04-23T14:43:11.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.178+0000 7f061b200640 1 -- start start 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f061c15e7c0 0x7f061c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f061c0d0d90 0x7f061c0d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f061c0cf030 con 0x7f061c0d2630 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f061c0b7000 con 0x7f061c0d0d90 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f061c0cf620 con 0x7f061c15e7c0 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f061c0d0d90 0x7f061c0d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f061c0d0d90 0x7f061c0d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46024/0 (socket says 172.21.15.59:46024) 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f061a800640 1 -- 172.21.15.59:0/1163892299 learned_addr learned my addr 172.21.15.59:0/1163892299 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.180+0000 7f0619e00640 1 --2- 172.21.15.59:0/1163892299 >> v2:172.21.15.76:3300/0 conn(0x7f061c15e7c0 0x7f061c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.181+0000 7f061a800640 1 -- 172.21.15.59:0/1163892299 >> v2:172.21.15.76:3300/0 conn(0x7f061c15e7c0 msgr2=0x7f061c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.181+0000 7f061a800640 1 --2- 172.21.15.59:0/1163892299 >> v2:172.21.15.76:3300/0 conn(0x7f061c15e7c0 0x7f061c0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.181+0000 7f061a800640 1 -- 172.21.15.59:0/1163892299 >> v1:172.21.15.76:6789/0 conn(0x7f061c0d2630 legacy=0x7f061c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:11.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.181+0000 7f061a800640 1 -- 172.21.15.59:0/1163892299 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061c0b6a40 con 0x7f061c0d0d90 2024-04-23T14:43:11.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.182+0000 7f061a800640 1 --2- 172.21.15.59:0/1163892299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c0d0d90 0x7f061c0d11c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f061c08c6d0 tx=0x7f061402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7371ddb8a5ee25 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:11.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.182+0000 7f0618a00640 1 -- 172.21.15.59:0/1163892299 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614037070 con 0x7f061c0d0d90 2024-04-23T14:43:11.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.182+0000 7f0618a00640 1 -- 172.21.15.59:0/1163892299 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0614032440 con 0x7f061c0d0d90 2024-04-23T14:43:11.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.182+0000 7f0618a00640 1 -- 172.21.15.59:0/1163892299 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614032810 con 0x7f061c0d0d90 2024-04-23T14:43:11.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 -- 172.21.15.59:0/1163892299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c0d0d90 msgr2=0x7f061c0d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 --2- 172.21.15.59:0/1163892299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c0d0d90 0x7f061c0d11c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f061c08c6d0 tx=0x7f061402a230 comp rx=0 tx=0).stop 2024-04-23T14:43:11.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 -- 172.21.15.59:0/1163892299 shutdown_connections 2024-04-23T14:43:11.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 --2- 172.21.15.59:0/1163892299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c0d0d90 0x7f061c0d11c0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 --2- 172.21.15.59:0/1163892299 >> v2:172.21.15.76:3300/0 conn(0x7f061c15e7c0 0x7f061c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 -- 172.21.15.59:0/1163892299 >> 172.21.15.59:0/1163892299 conn(0x7f061c09d920 msgr2=0x7f061c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 -- 172.21.15.59:0/1163892299 shutdown_connections 2024-04-23T14:43:11.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.183+0000 7f061b200640 1 -- 172.21.15.59:0/1163892299 wait complete. 2024-04-23T14:43:11.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.185+0000 7f061b200640 1 Processor -- start 2024-04-23T14:43:11.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.185+0000 7f061b200640 1 -- start start 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f061c0d0d90 0x7f061c1623f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061c0d2630 0x7f061c0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c15e7c0 0x7f061c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f061c0cf2b0 con 0x7f061c15e7c0 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f061c0d1700 con 0x7f061c0d2630 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f061c0b6700 con 0x7f061c0d0d90 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c15e7c0 0x7f061c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c15e7c0 0x7f061c15cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46030/0 (socket says 172.21.15.59:46030) 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f061a800640 1 -- 172.21.15.59:0/2212251183 learned_addr learned my addr 172.21.15.59:0/2212251183 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f0619e00640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f061c0d0d90 0x7f061c1623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.186+0000 7f0619400640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061c0d2630 0x7f061c0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f061a800640 1 -- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f061c0d0d90 msgr2=0x7f061c1623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f061a800640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f061c0d0d90 0x7f061c1623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f061a800640 1 -- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061c0d2630 msgr2=0x7f061c0c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f061a800640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061c0d2630 0x7f061c0c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f061a800640 1 -- 172.21.15.59:0/2212251183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061c0cb690 con 0x7f061c15e7c0 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f0619e00640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f061c0d0d90 0x7f061c1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f0619400640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061c0d2630 0x7f061c0c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f061a800640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c15e7c0 0x7f061c15cd40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f061402a270 tx=0x7f06140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:11.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f0612000640 1 -- 172.21.15.59:0/2212251183 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614041020 con 0x7f061c15e7c0 2024-04-23T14:43:11.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f0612000640 1 -- 172.21.15.59:0/2212251183 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06140306c0 con 0x7f061c15e7c0 2024-04-23T14:43:11.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.187+0000 7f0612000640 1 -- 172.21.15.59:0/2212251183 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614037040 con 0x7f061c15e7c0 2024-04-23T14:43:11.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.188+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061c0cb930 con 0x7f061c15e7c0 2024-04-23T14:43:11.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.188+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f061c0cbef0 con 0x7f061c15e7c0 2024-04-23T14:43:11.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.189+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f061c04fd90 con 0x7f061c15e7c0 2024-04-23T14:43:11.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.190+0000 7f0612000640 1 -- 172.21.15.59:0/2212251183 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0614044020 con 0x7f061c15e7c0 2024-04-23T14:43:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.191+0000 7f0612000640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f05f806c8f0 0x7f05f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.191+0000 7f0612000640 1 -- 172.21.15.59:0/2212251183 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7f06140ac460 con 0x7f061c15e7c0 2024-04-23T14:43:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.191+0000 7f0619e00640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f05f806c8f0 0x7f05f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.192+0000 7f0619e00640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f05f806c8f0 0x7f05f806edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0604006fd0 tx=0x7f0604008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:11.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.196+0000 7f0612000640 1 -- 172.21.15.59:0/2212251183 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f061404b030 con 0x7f061c15e7c0 2024-04-23T14:43:11.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.447+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f061c0d1eb0 con 0x7f061c15e7c0 2024-04-23T14:43:11.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.448+0000 7f0612000640 1 -- 172.21.15.59:0/2212251183 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5170 (secure 0 0 0) 0x7f061c0cf030 con 0x7f061c15e7c0 2024-04-23T14:43:11.450 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:11.450 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9060,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3956898282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3956898282},{"type":"v1","addr":"172.21.15.59:6837","nonce":3956898282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:43:10.119265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-23T14:43:11.450 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 90 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.451+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f05f806c8f0 msgr2=0x7f05f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.451+0000 7f061b200640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f05f806c8f0 0x7f05f806edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0604006fd0 tx=0x7f0604008040 comp rx=0 tx=0).stop 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.451+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c15e7c0 msgr2=0x7f061c15cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.451+0000 7f061b200640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c15e7c0 0x7f061c15cd40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f061402a270 tx=0x7f06140046c0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 shutdown_connections 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f05f806c8f0 0x7f05f806edb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f061c15e7c0 0x7f061c15cd40 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f061c0d2630 0x7f061c0c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 --2- 172.21.15.59:0/2212251183 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f061c0d0d90 0x7f061c1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 >> 172.21.15.59:0/2212251183 conn(0x7f061c09d920 msgr2=0x7f061c15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:11.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 shutdown_connections 2024-04-23T14:43:11.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.452+0000 7f061b200640 1 -- 172.21.15.59:0/2212251183 wait complete. 2024-04-23T14:43:11.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.680+0000 7fb127200640 1 Processor -- start 2024-04-23T14:43:11.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.681+0000 7fb127200640 1 -- start start 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.682+0000 7fb127200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1280b5ff0 0x7fb1280b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.682+0000 7fb127200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1280d0d90 0x7fb1280d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.682+0000 7fb127200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1280cf030 con 0x7fb12815e7c0 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.682+0000 7fb127200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb1280b7000 con 0x7fb1280b5ff0 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.682+0000 7fb127200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb1280cf620 con 0x7fb1280d0d90 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.683+0000 7fb125400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1280b5ff0 0x7fb1280b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.683+0000 7fb125400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1280b5ff0 0x7fb1280b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46036/0 (socket says 172.21.15.59:46036) 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.683+0000 7fb125400640 1 -- 172.21.15.59:0/3767517808 learned_addr learned my addr 172.21.15.59:0/3767517808 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.683+0000 7fb126800640 1 --2- 172.21.15.59:0/3767517808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1280d0d90 0x7fb1280d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.683+0000 7fb125400640 1 -- 172.21.15.59:0/3767517808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1280d0d90 msgr2=0x7fb1280d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.683+0000 7fb125400640 1 --2- 172.21.15.59:0/3767517808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1280d0d90 0x7fb1280d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.683+0000 7fb125400640 1 -- 172.21.15.59:0/3767517808 >> v1:172.21.15.76:6789/0 conn(0x7fb12815e7c0 legacy=0x7fb1280d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.684+0000 7fb125400640 1 -- 172.21.15.59:0/3767517808 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1280b6a40 con 0x7fb1280b5ff0 2024-04-23T14:43:11.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.684+0000 7fb126800640 1 --2- 172.21.15.59:0/3767517808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1280d0d90 0x7fb1280d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:43:11.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.685+0000 7fb125400640 1 --2- 172.21.15.59:0/3767517808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280b6450 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb110009870 tx=0x7fb11002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1a876b3efaa832 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:11.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.685+0000 7fb124a00640 1 -- 172.21.15.59:0/3767517808 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110037070 con 0x7fb1280b5ff0 2024-04-23T14:43:11.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.685+0000 7fb124a00640 1 -- 172.21.15.59:0/3767517808 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb110032440 con 0x7fb1280b5ff0 2024-04-23T14:43:11.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.685+0000 7fb124a00640 1 -- 172.21.15.59:0/3767517808 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110032810 con 0x7fb1280b5ff0 2024-04-23T14:43:11.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 -- 172.21.15.59:0/3767517808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 msgr2=0x7fb1280b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 --2- 172.21.15.59:0/3767517808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280b6450 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb110009870 tx=0x7fb11002a250 comp rx=0 tx=0).stop 2024-04-23T14:43:11.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 -- 172.21.15.59:0/3767517808 shutdown_connections 2024-04-23T14:43:11.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 --2- 172.21.15.59:0/3767517808 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1280d0d90 0x7fb1280d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 --2- 172.21.15.59:0/3767517808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280b6450 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 -- 172.21.15.59:0/3767517808 >> 172.21.15.59:0/3767517808 conn(0x7fb12809d920 msgr2=0x7fb12808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:11.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 -- 172.21.15.59:0/3767517808 shutdown_connections 2024-04-23T14:43:11.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.686+0000 7fb127200640 1 -- 172.21.15.59:0/3767517808 wait complete. 2024-04-23T14:43:11.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.688+0000 7fb127200640 1 Processor -- start 2024-04-23T14:43:11.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.688+0000 7fb127200640 1 -- start start 2024-04-23T14:43:11.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.689+0000 7fb127200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.689+0000 7fb127200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1280d0d90 0x7fb12814c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.689+0000 7fb127200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb12815e7c0 0x7fb12815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.689+0000 7fb127200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1280d2800 con 0x7fb1280b5ff0 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.689+0000 7fb127200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1280cf2b0 con 0x7fb1280d0d90 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.689+0000 7fb127200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb1280d1a80 con 0x7fb12815e7c0 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.689+0000 7fb125e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46050/0 (socket says 172.21.15.59:46050) 2024-04-23T14:43:11.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 -- 172.21.15.59:0/1456326117 learned_addr learned my addr 172.21.15.59:0/1456326117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb126800640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb12815e7c0 0x7fb12815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 -- 172.21.15.59:0/1456326117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb12815e7c0 msgr2=0x7fb12815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb12815e7c0 0x7fb12815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 -- 172.21.15.59:0/1456326117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1280d0d90 msgr2=0x7fb12814c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1280d0d90 0x7fb12814c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 -- 172.21.15.59:0/1456326117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1280ce3b0 con 0x7fb1280b5ff0 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.690+0000 7fb125e00640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280cde70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c730 tx=0x7fb11800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.691+0000 7fb11e000640 1 -- 172.21.15.59:0/1456326117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118016020 con 0x7fb1280b5ff0 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.691+0000 7fb11e000640 1 -- 172.21.15.59:0/1456326117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb118006d50 con 0x7fb1280b5ff0 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.691+0000 7fb11e000640 1 -- 172.21.15.59:0/1456326117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118003bb0 con 0x7fb1280b5ff0 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.691+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1280b2e30 con 0x7fb1280b5ff0 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.691+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12815b000 con 0x7fb1280b5ff0 2024-04-23T14:43:11.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.692+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12815ae30 con 0x7fb1280b5ff0 2024-04-23T14:43:11.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.693+0000 7fb11e000640 1 -- 172.21.15.59:0/1456326117 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb11800a850 con 0x7fb1280b5ff0 2024-04-23T14:43:11.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.694+0000 7fb11e000640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb10406c8f0 0x7fb10406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:11.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.694+0000 7fb11e000640 1 -- 172.21.15.59:0/1456326117 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7fb11808b930 con 0x7fb1280b5ff0 2024-04-23T14:43:11.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.694+0000 7fb125400640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb10406c8f0 0x7fb10406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:11.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.695+0000 7fb125400640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb10406c8f0 0x7fb10406edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb110034000 tx=0x7fb11002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:11.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.699+0000 7fb11e000640 1 -- 172.21.15.59:0/1456326117 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb118020020 con 0x7fb1280b5ff0 2024-04-23T14:43:11.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.906+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb12804fd90 con 0x7fb1280b5ff0 2024-04-23T14:43:11.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.909+0000 7fb11e000640 1 -- 172.21.15.59:0/1456326117 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16806 (secure 0 0 0) 0x7fb1280cf030 con 0x7fb1280b5ff0 2024-04-23T14:43:11.911 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:11.911 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":101,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:10.119057+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6835/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6837/620420969":"2024-04-24T14:42:32.618734+0000","172.21.15.59:6835/620420969":"2024-04-24T14:42:32.618734+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:11.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.911+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb10406c8f0 msgr2=0x7fb10406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.911+0000 7fb127200640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb10406c8f0 0x7fb10406edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb110034000 tx=0x7fb11002c000 comp rx=0 tx=0).stop 2024-04-23T14:43:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.911+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 msgr2=0x7fb1280cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.911+0000 7fb127200640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280cde70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c730 tx=0x7fb11800cbf0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 shutdown_connections 2024-04-23T14:43:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb10406c8f0 0x7fb10406edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb12815e7c0 0x7fb12815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1280d0d90 0x7fb12814c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 --2- 172.21.15.59:0/1456326117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1280b5ff0 0x7fb1280cde70 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 >> 172.21.15.59:0/1456326117 conn(0x7fb12809d920 msgr2=0x7fb1280d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 shutdown_connections 2024-04-23T14:43:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:11.912+0000 7fb127200640 1 -- 172.21.15.59:0/1456326117 wait complete. 2024-04-23T14:43:11.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:41:27.465708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:41:28.586680+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:43:11.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:43:12.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.146+0000 7fe3d9e00640 1 Processor -- start 2024-04-23T14:43:12.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.147+0000 7fe3d9e00640 1 -- start start 2024-04-23T14:43:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.148+0000 7fe3d9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 0x7fe3d40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.148+0000 7fe3d9e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d40d01c0 con 0x7fe3d40d1950 2024-04-23T14:43:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.148+0000 7fe3d9e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d415f6a0 con 0x7fe3d4163770 2024-04-23T14:43:12.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.148+0000 7fe3d9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d415cf00 con 0x7fe3d415cb20 2024-04-23T14:43:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.149+0000 7fe3d3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe3d40d1950 0x7fe3d40d2be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56236/0 (socket says 172.21.15.59:56236) 2024-04-23T14:43:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.149+0000 7fe3d3400640 1 -- 172.21.15.59:0/3911883821 learned_addr learned my addr 172.21.15.59:0/3911883821 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.149+0000 7fe3d2a00640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 0x7fe3d40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.149+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1334755150 0 0) 0x7fe3d40d01c0 con 0x7fe3d40d1950 2024-04-23T14:43:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.149+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3b8003520 con 0x7fe3d40d1950 2024-04-23T14:43:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.149+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4205238862 0 0) 0x7fe3b8003520 con 0x7fe3d40d1950 2024-04-23T14:43:12.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 msgr2=0x7fe3d40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 0x7fe3d40cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 >> v1:172.21.15.76:6789/0 conn(0x7fe3d4163770 legacy=0x7fe3d40d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d415f980 con 0x7fe3d40d1950 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fe3cc001670 con 0x7fe3d40d1950 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 >> v1:172.21.15.59:6789/0 conn(0x7fe3d40d1950 legacy=0x7fe3d40d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b80039e0 0x7fe3b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 0x7fe3b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b8004d40 0x7fe3b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d415cf00 con 0x7fe3b8004d40 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b800bb50 con 0x7fe3b80039e0 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b800bd50 con 0x7fe3d415cb20 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d3400640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b8004d40 0x7fe3b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d2a00640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b80039e0 0x7fe3b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.150+0000 7fe3d2000640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 0x7fe3b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.151+0000 7fe3d3400640 1 -- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 msgr2=0x7fe3b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.151+0000 7fe3d3400640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d415cb20 0x7fe3b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.151+0000 7fe3d3400640 1 -- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b80039e0 msgr2=0x7fe3b8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.151+0000 7fe3d3400640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b80039e0 0x7fe3b8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.151+0000 7fe3d3400640 1 -- 172.21.15.59:0/3911883821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b8003520 con 0x7fe3b8004d40 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.152+0000 7fe3d3400640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b8004d40 0x7fe3b80051c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40cb8b0 tx=0x7fe3b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c140541fc7c022f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.152+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc007c20 con 0x7fe3b8004d40 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.152+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3cc005af0 con 0x7fe3b8004d40 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.152+0000 7fe3d1600640 1 -- 172.21.15.59:0/3911883821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc007070 con 0x7fe3b8004d40 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.153+0000 7fe3d9e00640 1 -- 172.21.15.59:0/3911883821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b8004d40 msgr2=0x7fe3b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.153+0000 7fe3d9e00640 1 --2- 172.21.15.59:0/3911883821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b8004d40 0x7fe3b80051c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40cb8b0 tx=0x7fe3b8005830 comp rx=0 tx=0).stop 2024-04-23T14:43:12.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.153+0000 7fe3d3400640 1 -- 172.21.15.59:0/3911883821 reap_dead start 2024-04-23T14:43:12.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.153+0000 7fe3d9e00640 1 -- 172.21.15.59:0/3911883821 shutdown_connections 2024-04-23T14:43:12.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.153+0000 7fe3d9e00640 1 -- 172.21.15.59:0/3911883821 >> 172.21.15.59:0/3911883821 conn(0x7fe3d406cc70 msgr2=0x7fe3d406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:12.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.153+0000 7fe3d9e00640 1 -- 172.21.15.59:0/3911883821 shutdown_connections 2024-04-23T14:43:12.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.153+0000 7fe3d9e00640 1 -- 172.21.15.59:0/3911883821 wait complete. 2024-04-23T14:43:12.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.154+0000 7fe3d9e00640 1 Processor -- start 2024-04-23T14:43:12.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.155+0000 7fe3d9e00640 1 -- start start 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d9e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b8004d40 0x7fe3d40ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d41e94c0 0x7fe3d4158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4158f50 0x7fe3d4159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40cf840 con 0x7fe3d4158f50 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d9e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41603d0 con 0x7fe3b8004d40 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d40cf9c0 con 0x7fe3d41e94c0 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4158f50 0x7fe3d4159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4158f50 0x7fe3d4159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46068/0 (socket says 172.21.15.59:46068) 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 -- 172.21.15.59:0/2425352540 learned_addr learned my addr 172.21.15.59:0/2425352540 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d2a00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b8004d40 0x7fe3d40ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d2000640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d41e94c0 0x7fe3d4158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 -- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d41e94c0 msgr2=0x7fe3d4158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d41e94c0 0x7fe3d4158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 -- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b8004d40 msgr2=0x7fe3d40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b8004d40 0x7fe3d40ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.156+0000 7fe3d3400640 1 -- 172.21.15.59:0/2425352540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d40cc490 con 0x7fe3d4158f50 2024-04-23T14:43:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.157+0000 7fe3d3400640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4158f50 0x7fe3d4159400 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc007fd0 tx=0x7fe3d415fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.157+0000 7fe3cb400640 1 -- 172.21.15.59:0/2425352540 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc037040 con 0x7fe3d4158f50 2024-04-23T14:43:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.157+0000 7fe3cb400640 1 -- 172.21.15.59:0/2425352540 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3cc041070 con 0x7fe3d4158f50 2024-04-23T14:43:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.157+0000 7fe3cb400640 1 -- 172.21.15.59:0/2425352540 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc007270 con 0x7fe3d4158f50 2024-04-23T14:43:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.157+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d40cc770 con 0x7fe3d4158f50 2024-04-23T14:43:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.157+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d40cca50 con 0x7fe3d4158f50 2024-04-23T14:43:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.158+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d4068a80 con 0x7fe3d4158f50 2024-04-23T14:43:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.159+0000 7fe3cb400640 1 -- 172.21.15.59:0/2425352540 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3cc007410 con 0x7fe3d4158f50 2024-04-23T14:43:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.160+0000 7fe3cb400640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b006c920 0x7fe3b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.160+0000 7fe3cb400640 1 -- 172.21.15.59:0/2425352540 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7fe3cc0abb00 con 0x7fe3d4158f50 2024-04-23T14:43:12.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.161+0000 7fe3d2a00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b006c920 0x7fe3b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.161+0000 7fe3d2a00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b006c920 0x7fe3b006ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4007520 tx=0x7fe3c400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:12.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.165+0000 7fe3cb400640 1 -- 172.21.15.59:0/2425352540 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3cc079e70 con 0x7fe3d4158f50 2024-04-23T14:43:12.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.366+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe3d40cb370 con 0x7fe3d4158f50 2024-04-23T14:43:12.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.367+0000 7fe3cb400640 1 -- 172.21.15.59:0/2425352540 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7fe3cc079770 con 0x7fe3d4158f50 2024-04-23T14:43:12.368 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:43:12.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b006c920 msgr2=0x7fe3b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b006c920 0x7fe3b006ede0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4007520 tx=0x7fe3c400b040 comp rx=0 tx=0).stop 2024-04-23T14:43:12.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4158f50 msgr2=0x7fe3d4159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4158f50 0x7fe3d4159400 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc007fd0 tx=0x7fe3d415fb50 comp rx=0 tx=0).stop 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 shutdown_connections 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe3b006c920 0x7fe3b006ede0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3d41e94c0 0x7fe3d4158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d4158f50 0x7fe3d4159400 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 --2- 172.21.15.59:0/2425352540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3b8004d40 0x7fe3d40ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 >> 172.21.15.59:0/2425352540 conn(0x7fe3d406cc70 msgr2=0x7fe3d4162a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 shutdown_connections 2024-04-23T14:43:12.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.370+0000 7fe3d9e00640 1 -- 172.21.15.59:0/2425352540 wait complete. 2024-04-23T14:43:12.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:43:12.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.591+0000 7fdf62200640 1 Processor -- start 2024-04-23T14:43:12.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.591+0000 7fdf62200640 1 -- start start 2024-04-23T14:43:12.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.593+0000 7fdf62200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf5c0d15b0 0x7fdf5c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.593+0000 7fdf62200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0b4f20 0x7fdf5c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.593+0000 7fdf62200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf5c0b5da0 con 0x7fdf5c15ed20 2024-04-23T14:43:12.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.593+0000 7fdf62200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdf5c15f100 con 0x7fdf5c0d15b0 2024-04-23T14:43:12.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.593+0000 7fdf62200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c0d0a20 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.593+0000 7fdf60e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdf5c15ed20 0x7fdf5c0d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56240/0 (socket says 172.21.15.59:56240) 2024-04-23T14:43:12.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.593+0000 7fdf60e00640 1 -- 172.21.15.59:0/2402583856 learned_addr learned my addr 172.21.15.59:0/2402583856 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:12.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5be00640 1 --2- 172.21.15.59:0/2402583856 >> v2:172.21.15.76:3300/0 conn(0x7fdf5c0d15b0 0x7fdf5c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf61800640 1 --2- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0b4f20 0x7fdf5c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5b400640 1 -- 172.21.15.59:0/2402583856 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3611990629 0 0) 0x7fdf5c0b5da0 con 0x7fdf5c15ed20 2024-04-23T14:43:12.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5b400640 1 -- 172.21.15.59:0/2402583856 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf40003520 con 0x7fdf5c15ed20 2024-04-23T14:43:12.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5be00640 1 -- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0b4f20 msgr2=0x7fdf5c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:12.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5be00640 1 --2- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0b4f20 0x7fdf5c0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5be00640 1 -- 172.21.15.59:0/2402583856 >> v1:172.21.15.59:6789/0 conn(0x7fdf5c15ed20 legacy=0x7fdf5c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5b400640 1 -- 172.21.15.59:0/2402583856 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 971424809 0 0) 0x7fdf40003520 con 0x7fdf5c15ed20 2024-04-23T14:43:12.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.594+0000 7fdf5be00640 1 -- 172.21.15.59:0/2402583856 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0d2ed0 con 0x7fdf5c0d15b0 2024-04-23T14:43:12.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.596+0000 7fdf5be00640 1 --2- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c0d15b0 0x7fdf5c0cf280 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5bb60568a7977d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:12.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.596+0000 7fdf5b400640 1 -- 172.21.15.59:0/2402583856 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44037070 con 0x7fdf5c0d15b0 2024-04-23T14:43:12.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.596+0000 7fdf5b400640 1 -- 172.21.15.59:0/2402583856 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf4402aa90 con 0x7fdf5c0d15b0 2024-04-23T14:43:12.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.596+0000 7fdf5b400640 1 -- 172.21.15.59:0/2402583856 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0019f0 con 0x7fdf5c0d15b0 2024-04-23T14:43:12.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 -- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c0d15b0 msgr2=0x7fdf5c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 --2- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c0d15b0 0x7fdf5c0cf280 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402a000 comp rx=0 tx=0).stop 2024-04-23T14:43:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 -- 172.21.15.59:0/2402583856 shutdown_connections 2024-04-23T14:43:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 --2- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0b4f20 0x7fdf5c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 --2- 172.21.15.59:0/2402583856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c0d15b0 0x7fdf5c0cf280 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 -- 172.21.15.59:0/2402583856 >> 172.21.15.59:0/2402583856 conn(0x7fdf5c08bf90 msgr2=0x7fdf5c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 -- 172.21.15.59:0/2402583856 shutdown_connections 2024-04-23T14:43:12.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.597+0000 7fdf62200640 1 -- 172.21.15.59:0/2402583856 wait complete. 2024-04-23T14:43:12.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.599+0000 7fdf62200640 1 Processor -- start 2024-04-23T14:43:12.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.599+0000 7fdf62200640 1 -- start start 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf62200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf5c0b4f20 0x7fdf5c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf62200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0d15b0 0x7fdf5c0cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf62200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c15ed20 0x7fdf5c150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf62200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c0b3600 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf62200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c0d3490 con 0x7fdf5c15ed20 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf62200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c0b3780 con 0x7fdf5c0d15b0 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf60e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf5c0b4f20 0x7fdf5c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf60e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf5c0b4f20 0x7fdf5c0ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46074/0 (socket says 172.21.15.59:46074) 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.600+0000 7fdf60e00640 1 -- 172.21.15.59:0/2891907461 learned_addr learned my addr 172.21.15.59:0/2891907461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf60e00640 1 -- 172.21.15.59:0/2891907461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0d15b0 msgr2=0x7fdf5c0cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf60e00640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0d15b0 0x7fdf5c0cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf61800640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c15ed20 0x7fdf5c150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf60e00640 1 -- 172.21.15.59:0/2891907461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c15ed20 msgr2=0x7fdf5c150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf60e00640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c15ed20 0x7fdf5c150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf60e00640 1 -- 172.21.15.59:0/2891907461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0ca640 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf60e00640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf5c0b4f20 0x7fdf5c0ca100 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0018d0 tx=0x7fdf4c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.601+0000 7fdf58c00640 1 -- 172.21.15.59:0/2891907461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c017070 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.602+0000 7fdf58c00640 1 -- 172.21.15.59:0/2891907461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf4c012400 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.602+0000 7fdf58c00640 1 -- 172.21.15.59:0/2891907461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c012830 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.602+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c0c8580 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.602+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c0c8dc0 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.603+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c0b7bc0 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.605+0000 7fdf58c00640 1 -- 172.21.15.59:0/2891907461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf4c01a070 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.605+0000 7fdf58c00640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf3806c8f0 0x7fdf3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.606+0000 7fdf58c00640 1 -- 172.21.15.59:0/2891907461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7fdf4c08bc50 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.606+0000 7fdf5be00640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf3806c8f0 0x7fdf3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:12.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.607+0000 7fdf5be00640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf3806c8f0 0x7fdf3806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdf44034000 tx=0x7fdf4402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:12.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.610+0000 7fdf58c00640 1 -- 172.21.15.59:0/2891907461 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf5c0b7bc0 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.812+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdf5c04fd90 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.813+0000 7fdf58c00640 1 -- 172.21.15.59:0/2891907461 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7fdf5c0c8580 con 0x7fdf5c0b4f20 2024-04-23T14:43:12.814 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:43:12.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.815+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf3806c8f0 msgr2=0x7fdf3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf3806c8f0 0x7fdf3806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdf44034000 tx=0x7fdf4402c000 comp rx=0 tx=0).stop 2024-04-23T14:43:12.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf5c0b4f20 msgr2=0x7fdf5c0ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:12.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf5c0b4f20 0x7fdf5c0ca100 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0018d0 tx=0x7fdf4c00a430 comp rx=0 tx=0).stop 2024-04-23T14:43:12.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 shutdown_connections 2024-04-23T14:43:12.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdf3806c8f0 0x7fdf3806edb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdf5c0d15b0 0x7fdf5c0cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf5c15ed20 0x7fdf5c150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 --2- 172.21.15.59:0/2891907461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf5c0b4f20 0x7fdf5c0ca100 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:12.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 >> 172.21.15.59:0/2891907461 conn(0x7fdf5c08bf90 msgr2=0x7fdf5c15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:12.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 shutdown_connections 2024-04-23T14:43:12.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:12.816+0000 7fdf62200640 1 -- 172.21.15.59:0/2891907461 wait complete. 2024-04-23T14:43:12.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:43:13.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.044+0000 7f613a200640 1 Processor -- start 2024-04-23T14:43:13.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.044+0000 7f613a200640 1 -- start start 2024-04-23T14:43:13.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f613a200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61341567e0 0x7f6134156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f613a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f613415d810 0x7f61340d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f613a200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613415cab0 con 0x7f61340d0b40 2024-04-23T14:43:13.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f613a200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f61340d0f70 con 0x7f61341567e0 2024-04-23T14:43:13.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f613a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6134160e40 con 0x7f613415d810 2024-04-23T14:43:13.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f6138e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61341567e0 0x7f6134156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f6133e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f613415d810 0x7f61340d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.046+0000 7f6139800640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f61340d0b40 0x7f6134160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:35646/0 (socket says 172.21.15.59:35646) 2024-04-23T14:43:13.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.047+0000 7f6139800640 1 -- 172.21.15.59:0/1578746699 learned_addr learned my addr 172.21.15.59:0/1578746699 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.047+0000 7f6138e00640 1 -- 172.21.15.59:0/1578746699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f613415d810 msgr2=0x7f61340d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.047+0000 7f6138e00640 1 --2- 172.21.15.59:0/1578746699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f613415d810 0x7f61340d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.047+0000 7f6138e00640 1 -- 172.21.15.59:0/1578746699 >> v1:172.21.15.76:6789/0 conn(0x7f61340d0b40 legacy=0x7f6134160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.047+0000 7f6138e00640 1 -- 172.21.15.59:0/1578746699 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613415cd30 con 0x7f61341567e0 2024-04-23T14:43:13.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.049+0000 7f6138e00640 1 --2- 172.21.15.59:0/1578746699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61341567e0 0x7f6134156bc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f612402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41519c3424870ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.050+0000 7f6133400640 1 -- 172.21.15.59:0/1578746699 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124007740 con 0x7f61341567e0 2024-04-23T14:43:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.050+0000 7f6133400640 1 -- 172.21.15.59:0/1578746699 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61240078e0 con 0x7f61341567e0 2024-04-23T14:43:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.050+0000 7f6133400640 1 -- 172.21.15.59:0/1578746699 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124007cb0 con 0x7f61341567e0 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.050+0000 7f613a200640 1 -- 172.21.15.59:0/1578746699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61341567e0 msgr2=0x7f6134156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.050+0000 7f613a200640 1 --2- 172.21.15.59:0/1578746699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61341567e0 0x7f6134156bc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f612402a660 comp rx=0 tx=0).stop 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.051+0000 7f613a200640 1 -- 172.21.15.59:0/1578746699 shutdown_connections 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.051+0000 7f613a200640 1 --2- 172.21.15.59:0/1578746699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f613415d810 0x7f61340d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.051+0000 7f613a200640 1 --2- 172.21.15.59:0/1578746699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61341567e0 0x7f6134156bc0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.051+0000 7f613a200640 1 -- 172.21.15.59:0/1578746699 >> 172.21.15.59:0/1578746699 conn(0x7f613406bdf0 msgr2=0x7f613406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.051+0000 7f613a200640 1 -- 172.21.15.59:0/1578746699 shutdown_connections 2024-04-23T14:43:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.051+0000 7f613a200640 1 -- 172.21.15.59:0/1578746699 wait complete. 2024-04-23T14:43:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.053+0000 7f613a200640 1 Processor -- start 2024-04-23T14:43:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.053+0000 7f613a200640 1 -- start start 2024-04-23T14:43:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.054+0000 7f613a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61340d0b40 0x7f61341f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.054+0000 7f613a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61341567e0 0x7f613414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.054+0000 7f613a200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f613415d810 0x7f613414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.054+0000 7f613a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6134160820 con 0x7f61340d0b40 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.054+0000 7f613a200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f613415d2f0 con 0x7f613415d810 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.054+0000 7f613a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f61341609a0 con 0x7f61341567e0 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61340d0b40 0x7f61341f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61340d0b40 0x7f61341f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46082/0 (socket says 172.21.15.59:46082) 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 -- 172.21.15.59:0/4070392424 learned_addr learned my addr 172.21.15.59:0/4070392424 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6139800640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f613415d810 0x7f613414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6133e00640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61341567e0 0x7f613414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 -- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61341567e0 msgr2=0x7f613414c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61341567e0 0x7f613414c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 -- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f613415d810 msgr2=0x7f613414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f613415d810 0x7f613414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.055+0000 7f6138e00640 1 -- 172.21.15.59:0/4070392424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61341539e0 con 0x7f61340d0b40 2024-04-23T14:43:13.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.056+0000 7f6138e00640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61340d0b40 0x7f61341f3df0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f6124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.056+0000 7f6130c00640 1 -- 172.21.15.59:0/4070392424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124040070 con 0x7f61340d0b40 2024-04-23T14:43:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.056+0000 7f6130c00640 1 -- 172.21.15.59:0/4070392424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6124031c40 con 0x7f61340d0b40 2024-04-23T14:43:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.056+0000 7f6130c00640 1 -- 172.21.15.59:0/4070392424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124009450 con 0x7f61340d0b40 2024-04-23T14:43:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.056+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6134153c80 con 0x7f61340d0b40 2024-04-23T14:43:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.056+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6134154240 con 0x7f61340d0b40 2024-04-23T14:43:13.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.058+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613404fd90 con 0x7f61340d0b40 2024-04-23T14:43:13.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.059+0000 7f6130c00640 1 -- 172.21.15.59:0/4070392424 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6124039040 con 0x7f61340d0b40 2024-04-23T14:43:13.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.060+0000 7f6130c00640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f611006c920 0x7f611006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.060+0000 7f6130c00640 1 -- 172.21.15.59:0/4070392424 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7f61240ad4c0 con 0x7f61340d0b40 2024-04-23T14:43:13.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.060+0000 7f6133e00640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f611006c920 0x7f611006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.061+0000 7f6133e00640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f611006c920 0x7f611006ede0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f613414d440 tx=0x7f611c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:13.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.065+0000 7f6130c00640 1 -- 172.21.15.59:0/4070392424 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f612407b8c0 con 0x7f61340d0b40 2024-04-23T14:43:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.276+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61340b5150 con 0x7f61340d0b40 2024-04-23T14:43:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.277+0000 7f6130c00640 1 -- 172.21.15.59:0/4070392424 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f612407b1c0 con 0x7f61340d0b40 2024-04-23T14:43:13.278 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:43:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f611006c920 msgr2=0x7f611006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f611006c920 0x7f611006ede0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f613414d440 tx=0x7f611c008040 comp rx=0 tx=0).stop 2024-04-23T14:43:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61340d0b40 msgr2=0x7f61341f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61340d0b40 0x7f61341f3df0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f6124004a40 comp rx=0 tx=0).stop 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 shutdown_connections 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f611006c920 0x7f611006ede0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61341567e0 0x7f613414c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f613415d810 0x7f613414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 --2- 172.21.15.59:0/4070392424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61340d0b40 0x7f61341f3df0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 >> 172.21.15.59:0/4070392424 conn(0x7f613406bdf0 msgr2=0x7f613406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 shutdown_connections 2024-04-23T14:43:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.280+0000 7f613a200640 1 -- 172.21.15.59:0/4070392424 wait complete. 2024-04-23T14:43:13.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:43:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.505+0000 7fc03a600640 1 Processor -- start 2024-04-23T14:43:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.506+0000 7fc03a600640 1 -- start start 2024-04-23T14:43:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.507+0000 7fc03a600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc03415e7c0 0x7fc0340d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.507+0000 7fc03a600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc0340d2630 0x7fc0340b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.507+0000 7fc03a600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0340cf030 con 0x7fc0340b6530 2024-04-23T14:43:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.507+0000 7fc03a600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc0340b7000 con 0x7fc03415e7c0 2024-04-23T14:43:13.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.507+0000 7fc03a600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340cf620 con 0x7fc0340d2630 2024-04-23T14:43:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc039200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc03415e7c0 0x7fc0340d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc039200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc03415e7c0 0x7fc0340d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46092/0 (socket says 172.21.15.59:46092) 2024-04-23T14:43:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc039200640 1 -- 172.21.15.59:0/2629814963 learned_addr learned my addr 172.21.15.59:0/2629814963 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc033e00640 1 --2- 172.21.15.59:0/2629814963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc0340d2630 0x7fc0340b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc039200640 1 -- 172.21.15.59:0/2629814963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc0340d2630 msgr2=0x7fc0340b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc039200640 1 --2- 172.21.15.59:0/2629814963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc0340d2630 0x7fc0340b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc039200640 1 -- 172.21.15.59:0/2629814963 >> v1:172.21.15.76:6789/0 conn(0x7fc0340b6530 legacy=0x7fc0340d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.508+0000 7fc039200640 1 -- 172.21.15.59:0/2629814963 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0340b6a40 con 0x7fc03415e7c0 2024-04-23T14:43:13.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.509+0000 7fc039200640 1 --2- 172.21.15.59:0/2629814963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc03415e7c0 0x7fc0340d20f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc024009870 tx=0x7fc02402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd37420898e9f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:13.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc033400640 1 -- 172.21.15.59:0/2629814963 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024037070 con 0x7fc03415e7c0 2024-04-23T14:43:13.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc033400640 1 -- 172.21.15.59:0/2629814963 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc024032440 con 0x7fc03415e7c0 2024-04-23T14:43:13.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc033400640 1 -- 172.21.15.59:0/2629814963 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024032810 con 0x7fc03415e7c0 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc03a600640 1 -- 172.21.15.59:0/2629814963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc03415e7c0 msgr2=0x7fc0340d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc03a600640 1 --2- 172.21.15.59:0/2629814963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc03415e7c0 0x7fc0340d20f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc024009870 tx=0x7fc02402a250 comp rx=0 tx=0).stop 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc03a600640 1 -- 172.21.15.59:0/2629814963 shutdown_connections 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc03a600640 1 --2- 172.21.15.59:0/2629814963 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc0340d2630 0x7fc0340b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc03a600640 1 --2- 172.21.15.59:0/2629814963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc03415e7c0 0x7fc0340d20f0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.510+0000 7fc03a600640 1 -- 172.21.15.59:0/2629814963 >> 172.21.15.59:0/2629814963 conn(0x7fc03409d920 msgr2=0x7fc03408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.511+0000 7fc03a600640 1 -- 172.21.15.59:0/2629814963 shutdown_connections 2024-04-23T14:43:13.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.511+0000 7fc03a600640 1 -- 172.21.15.59:0/2629814963 wait complete. 2024-04-23T14:43:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.512+0000 7fc03a600640 1 Processor -- start 2024-04-23T14:43:13.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.512+0000 7fc03a600640 1 -- start start 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.513+0000 7fc03a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0340b6530 0x7fc03414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.513+0000 7fc03a600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0340d2630 0x7fc0340cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.513+0000 7fc03a600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc03415e7c0 0x7fc0340cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.513+0000 7fc03a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340d1a80 con 0x7fc0340b6530 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.513+0000 7fc03a600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340cf2b0 con 0x7fc0340d2630 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.513+0000 7fc03a600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340d1c00 con 0x7fc03415e7c0 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0340b6530 0x7fc03414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0340b6530 0x7fc03414fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46106/0 (socket says 172.21.15.59:46106) 2024-04-23T14:43:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 -- 172.21.15.59:0/4026276641 learned_addr learned my addr 172.21.15.59:0/4026276641 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 -- 172.21.15.59:0/4026276641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc03415e7c0 msgr2=0x7fc0340cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc03415e7c0 0x7fc0340cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 -- 172.21.15.59:0/4026276641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0340d2630 msgr2=0x7fc0340cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0340d2630 0x7fc0340cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 -- 172.21.15.59:0/4026276641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0340b1240 con 0x7fc0340b6530 2024-04-23T14:43:13.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.514+0000 7fc039200640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0340b6530 0x7fc03414fa70 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc024000c00 tx=0x7fc0240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.515+0000 7fc030c00640 1 -- 172.21.15.59:0/4026276641 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024041020 con 0x7fc0340b6530 2024-04-23T14:43:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.515+0000 7fc030c00640 1 -- 172.21.15.59:0/4026276641 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0240306c0 con 0x7fc0340b6530 2024-04-23T14:43:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.515+0000 7fc030c00640 1 -- 172.21.15.59:0/4026276641 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024037040 con 0x7fc0340b6530 2024-04-23T14:43:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.515+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03414ffb0 con 0x7fc0340b6530 2024-04-23T14:43:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.515+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0340b1790 con 0x7fc0340b6530 2024-04-23T14:43:13.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.516+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03404fd90 con 0x7fc0340b6530 2024-04-23T14:43:13.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.517+0000 7fc030c00640 1 -- 172.21.15.59:0/4026276641 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc024044020 con 0x7fc0340b6530 2024-04-23T14:43:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.518+0000 7fc030c00640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc01006c8f0 0x7fc01006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.519+0000 7fc030c00640 1 -- 172.21.15.59:0/4026276641 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7fc0240ac4b0 con 0x7fc0340b6530 2024-04-23T14:43:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.519+0000 7fc033e00640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc01006c8f0 0x7fc01006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.519+0000 7fc033e00640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc01006c8f0 0x7fc01006edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc0341502f0 tx=0x7fc01c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:13.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.523+0000 7fc030c00640 1 -- 172.21.15.59:0/4026276641 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc02404b030 con 0x7fc0340b6530 2024-04-23T14:43:13.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.763+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0340b4df0 con 0x7fc0340b6530 2024-04-23T14:43:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.763+0000 7fc030c00640 1 -- 172.21.15.59:0/4026276641 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5170 (secure 0 0 0) 0x7fc02407a790 con 0x7fc0340b6530 2024-04-23T14:43:13.765 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:13.765 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9060,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3956898282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3956898282},{"type":"v1","addr":"172.21.15.59:6837","nonce":3956898282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:43:10.119265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-23T14:43:13.765 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 90 2024-04-23T14:43:13.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc01006c8f0 msgr2=0x7fc01006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc01006c8f0 0x7fc01006edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc0341502f0 tx=0x7fc01c008040 comp rx=0 tx=0).stop 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0340b6530 msgr2=0x7fc03414fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0340b6530 0x7fc03414fa70 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc024000c00 tx=0x7fc0240046c0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 shutdown_connections 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc01006c8f0 0x7fc01006edb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc03415e7c0 0x7fc0340cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0340d2630 0x7fc0340cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 --2- 172.21.15.59:0/4026276641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0340b6530 0x7fc03414fa70 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 >> 172.21.15.59:0/4026276641 conn(0x7fc03409d920 msgr2=0x7fc0340b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 shutdown_connections 2024-04-23T14:43:13.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.766+0000 7fc03a600640 1 -- 172.21.15.59:0/4026276641 wait complete. 2024-04-23T14:43:13.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:13.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.985+0000 7f1fe2800640 1 Processor -- start 2024-04-23T14:43:13.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.986+0000 7f1fe2800640 1 -- start start 2024-04-23T14:43:13.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.987+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc15e7c0 0x7f1fdc0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.987+0000 7f1fe2800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fdc0cf030 con 0x7f1fdc0d0d90 2024-04-23T14:43:13.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.988+0000 7f1fe2800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fdc0b6a40 con 0x7f1fdc0d2630 2024-04-23T14:43:13.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.988+0000 7f1fe2800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1fdc0b7000 con 0x7f1fdc15e7c0 2024-04-23T14:43:13.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.988+0000 7f1fe1e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1fdc0d0d90 0x7f1fdc0cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56254/0 (socket says 172.21.15.59:56254) 2024-04-23T14:43:13.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.988+0000 7f1fe1e00640 1 -- 172.21.15.59:0/1873804280 learned_addr learned my addr 172.21.15.59:0/1873804280 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:13.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.988+0000 7f1fe1400640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc15e7c0 0x7f1fdc0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.988+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154587957 0 0) 0x7f1fd4001490 con 0x7f1fdc0d0d90 2024-04-23T14:43:13.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.988+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fc0003520 con 0x7f1fdc0d0d90 2024-04-23T14:43:13.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1681123987 0 0) 0x7f1fc0003520 con 0x7f1fdc0d0d90 2024-04-23T14:43:13.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc15e7c0 msgr2=0x7f1fdc0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc15e7c0 0x7f1fdc0d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 >> v1:172.21.15.76:6789/0 conn(0x7f1fdc0d2630 legacy=0x7f1fdc0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc0b6d20 con 0x7f1fdc0d0d90 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1fd4001bc0 con 0x7f1fdc0d0d90 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 >> v1:172.21.15.59:6789/0 conn(0x7f1fdc0d0d90 legacy=0x7f1fdc0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc00039e0 0x7f1fc0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc15e7c0 0x7f1fc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0b7000 con 0x7f1fc0004d40 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc000ba70 con 0x7f1fc00039e0 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc000bc70 con 0x7f1fdc15e7c0 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.989+0000 7f1fe1e00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.990+0000 7f1fe1400640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc00039e0 0x7f1fc0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.990+0000 7f1fe1e00640 1 -- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc15e7c0 msgr2=0x7f1fc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:13.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.990+0000 7f1fe1e00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc15e7c0 0x7f1fc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.990+0000 7f1fe1e00640 1 -- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc00039e0 msgr2=0x7f1fc0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:13.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.990+0000 7f1fe1e00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc00039e0 0x7f1fc0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.990+0000 7f1fe1e00640 1 -- 172.21.15.59:0/1873804280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc0003520 con 0x7f1fc0004d40 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.991+0000 7f1fe1e00640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fc00051c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400aa20 tx=0x7f1fc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721a494def1dd7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.991+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd400a670 con 0x7f1fc0004d40 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.991+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd402f070 con 0x7f1fc0004d40 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.991+0000 7f1fdbe00640 1 -- 172.21.15.59:0/1873804280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4034470 con 0x7f1fc0004d40 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.991+0000 7f1fe2800640 1 -- 172.21.15.59:0/1873804280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 msgr2=0x7f1fc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.991+0000 7f1fe2800640 1 --2- 172.21.15.59:0/1873804280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fc00051c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400aa20 tx=0x7f1fc0005830 comp rx=0 tx=0).stop 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.991+0000 7f1fe1e00640 1 -- 172.21.15.59:0/1873804280 reap_dead start 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.992+0000 7f1fe2800640 1 -- 172.21.15.59:0/1873804280 shutdown_connections 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.992+0000 7f1fe2800640 1 -- 172.21.15.59:0/1873804280 >> 172.21.15.59:0/1873804280 conn(0x7f1fdc09d920 msgr2=0x7f1fdc08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.992+0000 7f1fe2800640 1 -- 172.21.15.59:0/1873804280 shutdown_connections 2024-04-23T14:43:13.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.992+0000 7f1fe2800640 1 -- 172.21.15.59:0/1873804280 wait complete. 2024-04-23T14:43:13.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.994+0000 7f1fe2800640 1 Processor -- start 2024-04-23T14:43:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.994+0000 7f1fe2800640 1 -- start start 2024-04-23T14:43:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fdc14dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fdc0cd230 0x7f1fdc160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc0cd6b0 0x7f1fdc1613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe2800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0b6700 con 0x7f1fc0004d40 2024-04-23T14:43:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe2800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0cf620 con 0x7f1fdc0cd230 2024-04-23T14:43:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe2800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1fdc0d1a80 con 0x7f1fdc0cd6b0 2024-04-23T14:43:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fdc14dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fdc14dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46128/0 (socket says 172.21.15.59:46128) 2024-04-23T14:43:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1400640 1 -- 172.21.15.59:0/1095421505 learned_addr learned my addr 172.21.15.59:0/1095421505 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe0a00640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fdc0cd230 0x7f1fdc160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1400640 1 -- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc0cd6b0 msgr2=0x7f1fdc1613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1e00640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc0cd6b0 0x7f1fdc1613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1400640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc0cd6b0 0x7f1fdc1613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1400640 1 -- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fdc0cd230 msgr2=0x7f1fdc160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.996+0000 7f1fe1400640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fdc0cd230 0x7f1fdc160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fe1400640 1 -- 172.21.15.59:0/1095421505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc14e000 con 0x7f1fc0004d40 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fe0a00640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fdc0cd230 0x7f1fdc160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fe1400640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fdc14dac0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0108c0 tx=0x7f1fcc010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fd9600640 1 -- 172.21.15.59:0/1095421505 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00f040 con 0x7f1fc0004d40 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fd9600640 1 -- 172.21.15.59:0/1095421505 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fcc00eb60 con 0x7f1fc0004d40 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fd9600640 1 -- 172.21.15.59:0/1095421505 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc01c400 con 0x7f1fc0004d40 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc0b14b0 con 0x7f1fc0004d40 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.997+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc0b1a70 con 0x7f1fc0004d40 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.999+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fdc04fd90 con 0x7f1fc0004d40 2024-04-23T14:43:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:13.999+0000 7f1fd9600640 1 -- 172.21.15.59:0/1095421505 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fcc01c5a0 con 0x7f1fc0004d40 2024-04-23T14:43:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.001+0000 7f1fd9600640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.001+0000 7f1fd9600640 1 -- 172.21.15.59:0/1095421505 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7f1fcc08c340 con 0x7f1fc0004d40 2024-04-23T14:43:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.001+0000 7f1fe0a00640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.002+0000 7f1fe0a00640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc161b10 tx=0x7f1fc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.005+0000 7f1fd9600640 1 -- 172.21.15.59:0/1095421505 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fcc018030 con 0x7f1fc0004d40 2024-04-23T14:43:14.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.213+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fdc0d1eb0 con 0x7f1fc0004d40 2024-04-23T14:43:14.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.215+0000 7f1fd9600640 1 -- 172.21.15.59:0/1095421505 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16806 (secure 0 0 0) 0x7f1fcc059fd0 con 0x7f1fc0004d40 2024-04-23T14:43:14.217 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:14.217 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":101,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:10.119057+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6835/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6837/620420969":"2024-04-24T14:42:32.618734+0000","172.21.15.59:6835/620420969":"2024-04-24T14:42:32.618734+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fbc06c8f0 msgr2=0x7f1fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc161b10 tx=0x7f1fc8009040 comp rx=0 tx=0).stop 2024-04-23T14:43:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 msgr2=0x7f1fdc14dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fdc14dac0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0108c0 tx=0x7f1fcc010d80 comp rx=0 tx=0).stop 2024-04-23T14:43:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 shutdown_connections 2024-04-23T14:43:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1fdc0cd6b0 0x7f1fdc1613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fdc0cd230 0x7f1fdc160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 --2- 172.21.15.59:0/1095421505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0004d40 0x7f1fdc14dac0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 >> 172.21.15.59:0/1095421505 conn(0x7f1fdc09d920 msgr2=0x7f1fdc0d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 shutdown_connections 2024-04-23T14:43:14.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.217+0000 7f1fe2800640 1 -- 172.21.15.59:0/1095421505 wait complete. 2024-04-23T14:43:14.238 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:43:14.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:43:14.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.446+0000 7fee66400640 1 Processor -- start 2024-04-23T14:43:14.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.446+0000 7fee66400640 1 -- start start 2024-04-23T14:43:14.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee66400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fee6015c3d0 0x7fee600cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee66400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee600b4860 0x7fee600cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee66400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee60160210 con 0x7fee600cd590 2024-04-23T14:43:14.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee66400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fee601604f0 con 0x7fee600b4860 2024-04-23T14:43:14.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee66400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fee601607d0 con 0x7fee6015c3d0 2024-04-23T14:43:14.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee65a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee600b4860 0x7fee600cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee65a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fee600b4860 0x7fee600cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46130/0 (socket says 172.21.15.59:46130) 2024-04-23T14:43:14.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.448+0000 7fee65a00640 1 -- 172.21.15.59:0/1462680493 learned_addr learned my addr 172.21.15.59:0/1462680493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:14.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.449+0000 7fee65000640 1 --2- 172.21.15.59:0/1462680493 >> v2:172.21.15.76:3300/0 conn(0x7fee6015c3d0 0x7fee600cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.449+0000 7fee65a00640 1 -- 172.21.15.59:0/1462680493 >> v2:172.21.15.76:3300/0 conn(0x7fee6015c3d0 msgr2=0x7fee600cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:14.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.449+0000 7fee65a00640 1 --2- 172.21.15.59:0/1462680493 >> v2:172.21.15.76:3300/0 conn(0x7fee6015c3d0 0x7fee600cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.449+0000 7fee65a00640 1 -- 172.21.15.59:0/1462680493 >> v1:172.21.15.76:6789/0 conn(0x7fee600cd590 legacy=0x7fee600b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:14.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.449+0000 7fee65a00640 1 -- 172.21.15.59:0/1462680493 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee60160ab0 con 0x7fee600b4860 2024-04-23T14:43:14.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.449+0000 7fee65000640 1 --2- 172.21.15.59:0/1462680493 >> v2:172.21.15.76:3300/0 conn(0x7fee6015c3d0 0x7fee600cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:14.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.450+0000 7fee65a00640 1 --2- 172.21.15.59:0/1462680493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee600b4860 0x7fee600cb7d0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fee6006d9b0 tx=0x7fee5802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1584ff2147348c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.450+0000 7fee5fe00640 1 -- 172.21.15.59:0/1462680493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58037070 con 0x7fee600b4860 2024-04-23T14:43:14.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.450+0000 7fee5fe00640 1 -- 172.21.15.59:0/1462680493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee58032440 con 0x7fee600b4860 2024-04-23T14:43:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.450+0000 7fee5fe00640 1 -- 172.21.15.59:0/1462680493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58032810 con 0x7fee600b4860 2024-04-23T14:43:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 -- 172.21.15.59:0/1462680493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee600b4860 msgr2=0x7fee600cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 --2- 172.21.15.59:0/1462680493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee600b4860 0x7fee600cb7d0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fee6006d9b0 tx=0x7fee5802a230 comp rx=0 tx=0).stop 2024-04-23T14:43:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 -- 172.21.15.59:0/1462680493 shutdown_connections 2024-04-23T14:43:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 --2- 172.21.15.59:0/1462680493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee600b4860 0x7fee600cb7d0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 --2- 172.21.15.59:0/1462680493 >> v2:172.21.15.76:3300/0 conn(0x7fee6015c3d0 0x7fee600cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 -- 172.21.15.59:0/1462680493 >> 172.21.15.59:0/1462680493 conn(0x7fee600b21a0 msgr2=0x7fee600b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 -- 172.21.15.59:0/1462680493 shutdown_connections 2024-04-23T14:43:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.451+0000 7fee66400640 1 -- 172.21.15.59:0/1462680493 wait complete. 2024-04-23T14:43:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.453+0000 7fee66400640 1 Processor -- start 2024-04-23T14:43:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.453+0000 7fee66400640 1 -- start start 2024-04-23T14:43:14.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee66400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee600b4860 0x7fee6015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee66400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee600cd590 0x7fee600c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee66400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee6015c3d0 0x7fee60158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee66400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee6015c930 con 0x7fee6015c3d0 2024-04-23T14:43:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee66400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee6015c7b0 con 0x7fee600cd590 2024-04-23T14:43:14.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee66400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fee600cbd10 con 0x7fee600b4860 2024-04-23T14:43:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee65a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee6015c3d0 0x7fee60158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee65a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee6015c3d0 0x7fee60158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46146/0 (socket says 172.21.15.59:46146) 2024-04-23T14:43:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.454+0000 7fee65a00640 1 -- 172.21.15.59:0/770486756 learned_addr learned my addr 172.21.15.59:0/770486756 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.455+0000 7fee57e00640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee600cd590 0x7fee600c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.455+0000 7fee65a00640 1 -- 172.21.15.59:0/770486756 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee600b4860 msgr2=0x7fee6015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.455+0000 7fee65a00640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee600b4860 0x7fee6015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.455+0000 7fee65a00640 1 -- 172.21.15.59:0/770486756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee600cd590 msgr2=0x7fee600c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.455+0000 7fee65a00640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee600cd590 0x7fee600c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.455+0000 7fee65a00640 1 -- 172.21.15.59:0/770486756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee60158960 con 0x7fee6015c3d0 2024-04-23T14:43:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.455+0000 7fee65a00640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee6015c3d0 0x7fee60158420 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fee580090b0 tx=0x7fee580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.456+0000 7fee5d600640 1 -- 172.21.15.59:0/770486756 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58041020 con 0x7fee6015c3d0 2024-04-23T14:43:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.456+0000 7fee5d600640 1 -- 172.21.15.59:0/770486756 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee58030990 con 0x7fee6015c3d0 2024-04-23T14:43:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.456+0000 7fee5d600640 1 -- 172.21.15.59:0/770486756 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58037040 con 0x7fee6015c3d0 2024-04-23T14:43:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.456+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee600ad7d0 con 0x7fee6015c3d0 2024-04-23T14:43:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.456+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee600add90 con 0x7fee6015c3d0 2024-04-23T14:43:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.457+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6004fd90 con 0x7fee6015c3d0 2024-04-23T14:43:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.458+0000 7fee5d600640 1 -- 172.21.15.59:0/770486756 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee58044020 con 0x7fee6015c3d0 2024-04-23T14:43:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.460+0000 7fee5d600640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3c06c920 0x7fee3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.460+0000 7fee5d600640 1 -- 172.21.15.59:0/770486756 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7fee580ac520 con 0x7fee6015c3d0 2024-04-23T14:43:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.460+0000 7fee65000640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3c06c920 0x7fee3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.461+0000 7fee65000640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3c06c920 0x7fee3c06ede0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fee50008e00 tx=0x7fee50008330 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.464+0000 7fee5d600640 1 -- 172.21.15.59:0/770486756 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee5804b030 con 0x7fee6015c3d0 2024-04-23T14:43:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.706+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee60074ae0 con 0x7fee6015c3d0 2024-04-23T14:43:14.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.707+0000 7fee5d600640 1 -- 172.21.15.59:0/770486756 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5170 (secure 0 0 0) 0x7fee5807a8a0 con 0x7fee6015c3d0 2024-04-23T14:43:14.709 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:14.709 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9060,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3956898282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3956898282},{"type":"v1","addr":"172.21.15.59:6837","nonce":3956898282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:43:10.119265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-23T14:43:14.709 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 90 2024-04-23T14:43:14.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3c06c920 msgr2=0x7fee3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3c06c920 0x7fee3c06ede0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fee50008e00 tx=0x7fee50008330 comp rx=0 tx=0).stop 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee6015c3d0 msgr2=0x7fee60158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee6015c3d0 0x7fee60158420 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fee580090b0 tx=0x7fee580046c0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 shutdown_connections 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee3c06c920 0x7fee3c06ede0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee6015c3d0 0x7fee60158420 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee600cd590 0x7fee600c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 --2- 172.21.15.59:0/770486756 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fee600b4860 0x7fee6015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 >> 172.21.15.59:0/770486756 conn(0x7fee600b21a0 msgr2=0x7fee600afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 shutdown_connections 2024-04-23T14:43:14.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.710+0000 7fee66400640 1 -- 172.21.15.59:0/770486756 wait complete. 2024-04-23T14:43:14.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:43:14.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.931+0000 7fe873e00640 1 Processor -- start 2024-04-23T14:43:14.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.931+0000 7fe873e00640 1 -- start start 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe873e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe87415c3d0 0x7fe8740cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe873e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8740cd590 0x7fe8740b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe873e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe874160210 con 0x7fe8740b4690 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe873e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe8741604f0 con 0x7fe8740cd590 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe873e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe8741607d0 con 0x7fe87415c3d0 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe872000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8740cd590 0x7fe8740b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe872000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe8740cd590 0x7fe8740b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46148/0 (socket says 172.21.15.59:46148) 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe872000640 1 -- 172.21.15.59:0/3816156348 learned_addr learned my addr 172.21.15.59:0/3816156348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:14.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe872a00640 1 --2- 172.21.15.59:0/3816156348 >> v2:172.21.15.76:3300/0 conn(0x7fe87415c3d0 0x7fe8740cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.933+0000 7fe872000640 1 -- 172.21.15.59:0/3816156348 >> v2:172.21.15.76:3300/0 conn(0x7fe87415c3d0 msgr2=0x7fe8740cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:14.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.934+0000 7fe872000640 1 --2- 172.21.15.59:0/3816156348 >> v2:172.21.15.76:3300/0 conn(0x7fe87415c3d0 0x7fe8740cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.934+0000 7fe872000640 1 -- 172.21.15.59:0/3816156348 >> v1:172.21.15.76:6789/0 conn(0x7fe8740b4690 legacy=0x7fe87406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:14.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.934+0000 7fe872000640 1 -- 172.21.15.59:0/3816156348 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe874160ab0 con 0x7fe8740cd590 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.935+0000 7fe872000640 1 --2- 172.21.15.59:0/3816156348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8740cd590 0x7fe8740b4150 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe87406d9b0 tx=0x7fe86002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ef7827b7b6a1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.935+0000 7fe871600640 1 -- 172.21.15.59:0/3816156348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860037070 con 0x7fe8740cd590 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.935+0000 7fe871600640 1 -- 172.21.15.59:0/3816156348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe860032440 con 0x7fe8740cd590 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.935+0000 7fe871600640 1 -- 172.21.15.59:0/3816156348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860032810 con 0x7fe8740cd590 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 -- 172.21.15.59:0/3816156348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8740cd590 msgr2=0x7fe8740b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 --2- 172.21.15.59:0/3816156348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8740cd590 0x7fe8740b4150 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe87406d9b0 tx=0x7fe86002a230 comp rx=0 tx=0).stop 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 -- 172.21.15.59:0/3816156348 shutdown_connections 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 --2- 172.21.15.59:0/3816156348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8740cd590 0x7fe8740b4150 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 --2- 172.21.15.59:0/3816156348 >> v2:172.21.15.76:3300/0 conn(0x7fe87415c3d0 0x7fe8740cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 -- 172.21.15.59:0/3816156348 >> 172.21.15.59:0/3816156348 conn(0x7fe8740b21a0 msgr2=0x7fe8740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:14.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 -- 172.21.15.59:0/3816156348 shutdown_connections 2024-04-23T14:43:14.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.936+0000 7fe873e00640 1 -- 172.21.15.59:0/3816156348 wait complete. 2024-04-23T14:43:14.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.937+0000 7fe873e00640 1 Processor -- start 2024-04-23T14:43:14.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.938+0000 7fe873e00640 1 -- start start 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.938+0000 7fe873e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8740b4690 0x7fe8740c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.938+0000 7fe873e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8740cd590 0x7fe8740ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.938+0000 7fe873e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe87415c3d0 0x7fe8740c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.938+0000 7fe873e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe87415c7b0 con 0x7fe87415c3d0 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.938+0000 7fe873e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8740b4ac0 con 0x7fe8740cd590 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.938+0000 7fe873e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe87415c930 con 0x7fe8740b4690 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe87415c3d0 0x7fe8740c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe87415c3d0 0x7fe8740c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46152/0 (socket says 172.21.15.59:46152) 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 -- 172.21.15.59:0/3458190379 learned_addr learned my addr 172.21.15.59:0/3458190379 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe872000640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8740cd590 0x7fe8740ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe872a00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8740b4690 0x7fe8740c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 -- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8740b4690 msgr2=0x7fe8740c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8740b4690 0x7fe8740c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 -- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8740cd590 msgr2=0x7fe8740ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8740cd590 0x7fe8740ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.939+0000 7fe873400640 1 -- 172.21.15.59:0/3458190379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8740c8520 con 0x7fe87415c3d0 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.940+0000 7fe873400640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe87415c3d0 0x7fe8740c7fe0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00c780 tx=0x7fe86c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.940+0000 7fe86aa00640 1 -- 172.21.15.59:0/3458190379 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c017070 con 0x7fe87415c3d0 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.940+0000 7fe86aa00640 1 -- 172.21.15.59:0/3458190379 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c003da0 con 0x7fe87415c3d0 2024-04-23T14:43:14.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.940+0000 7fe86aa00640 1 -- 172.21.15.59:0/3458190379 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0052b0 con 0x7fe87415c3d0 2024-04-23T14:43:14.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.940+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8740c87a0 con 0x7fe87415c3d0 2024-04-23T14:43:14.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.940+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8740c8d60 con 0x7fe87415c3d0 2024-04-23T14:43:14.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.941+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8740cfd40 con 0x7fe87415c3d0 2024-04-23T14:43:14.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.942+0000 7fe86aa00640 1 -- 172.21.15.59:0/3458190379 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe86c01a070 con 0x7fe87415c3d0 2024-04-23T14:43:14.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.943+0000 7fe86aa00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe85406c8f0 0x7fe85406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:14.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.944+0000 7fe86aa00640 1 -- 172.21.15.59:0/3458190379 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7fe86c08b310 con 0x7fe87415c3d0 2024-04-23T14:43:14.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.944+0000 7fe872a00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe85406c8f0 0x7fe85406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:14.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.944+0000 7fe872a00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe85406c8f0 0x7fe85406edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe864006fd0 tx=0x7fe864008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:14.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:14.948+0000 7fe86aa00640 1 -- 172.21.15.59:0/3458190379 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8740c8520 con 0x7fe87415c3d0 2024-04-23T14:43:15.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.195+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe87406e400 con 0x7fe87415c3d0 2024-04-23T14:43:15.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.196+0000 7fe86aa00640 1 -- 172.21.15.59:0/3458190379 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5170 (secure 0 0 0) 0x7fe86c0595f0 con 0x7fe87415c3d0 2024-04-23T14:43:15.198 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:15.198 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9060,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3956898282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3956898282},{"type":"v1","addr":"172.21.15.59:6837","nonce":3956898282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:41:29.612586+0000","modified":"2024-04-23T14:43:10.119265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-23T14:43:15.198 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 90 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe85406c8f0 msgr2=0x7fe85406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe85406c8f0 0x7fe85406edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe864006fd0 tx=0x7fe864008040 comp rx=0 tx=0).stop 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe87415c3d0 msgr2=0x7fe8740c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe87415c3d0 0x7fe8740c7fe0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00c780 tx=0x7fe86c00cc40 comp rx=0 tx=0).stop 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 shutdown_connections 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe85406c8f0 0x7fe85406edb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe87415c3d0 0x7fe8740c7fe0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8740cd590 0x7fe8740ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 --2- 172.21.15.59:0/3458190379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8740b4690 0x7fe8740c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 >> 172.21.15.59:0/3458190379 conn(0x7fe8740b21a0 msgr2=0x7fe8740b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.199+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 shutdown_connections 2024-04-23T14:43:15.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.200+0000 7fe873e00640 1 -- 172.21.15.59:0/3458190379 wait complete. 2024-04-23T14:43:15.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:15.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.434+0000 7f50f9a00640 1 Processor -- start 2024-04-23T14:43:15.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.434+0000 7f50f9a00640 1 -- start start 2024-04-23T14:43:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.436+0000 7f50f9a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f50f415c5e0 0x7f50f40af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.436+0000 7f50f9a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f415f0f0 con 0x7f50f40d08b0 2024-04-23T14:43:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.436+0000 7f50f9a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f415f3d0 con 0x7f50f40af920 2024-04-23T14:43:15.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.436+0000 7f50f9a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f50f40afd80 con 0x7f50f415c5e0 2024-04-23T14:43:15.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f3400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f50f40d08b0 0x7f50f41624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56260/0 (socket says 172.21.15.59:56260) 2024-04-23T14:43:15.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f3400640 1 -- 172.21.15.59:0/2807436425 learned_addr learned my addr 172.21.15.59:0/2807436425 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:15.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f2a00640 1 --2- 172.21.15.59:0/2807436425 >> v2:172.21.15.76:3300/0 conn(0x7f50f415c5e0 0x7f50f40af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f1600640 1 -- 172.21.15.59:0/2807436425 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638496854 0 0) 0x7f50f415f0f0 con 0x7f50f40d08b0 2024-04-23T14:43:15.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f1600640 1 -- 172.21.15.59:0/2807436425 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50d8003520 con 0x7f50f40d08b0 2024-04-23T14:43:15.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f2a00640 1 -- 172.21.15.59:0/2807436425 >> v1:172.21.15.76:6789/0 conn(0x7f50f40af920 legacy=0x7f50f40d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f2a00640 1 -- 172.21.15.59:0/2807436425 >> v1:172.21.15.59:6789/0 conn(0x7f50f40d08b0 legacy=0x7f50f41624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.437+0000 7f50f2a00640 1 -- 172.21.15.59:0/2807436425 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f415f6b0 con 0x7f50f415c5e0 2024-04-23T14:43:15.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.439+0000 7f50f2a00640 1 --2- 172.21.15.59:0/2807436425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f40af3e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097a0 tx=0x7f50e402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b2e9135efc66235 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:15.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.439+0000 7f50f1600640 1 -- 172.21.15.59:0/2807436425 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4037070 con 0x7f50f415c5e0 2024-04-23T14:43:15.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.439+0000 7f50f1600640 1 -- 172.21.15.59:0/2807436425 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e4032440 con 0x7f50f415c5e0 2024-04-23T14:43:15.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f1600640 1 -- 172.21.15.59:0/2807436425 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4032810 con 0x7f50f415c5e0 2024-04-23T14:43:15.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f9a00640 1 -- 172.21.15.59:0/2807436425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 msgr2=0x7f50f40af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f9a00640 1 --2- 172.21.15.59:0/2807436425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f40af3e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097a0 tx=0x7f50e402a230 comp rx=0 tx=0).stop 2024-04-23T14:43:15.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f9a00640 1 -- 172.21.15.59:0/2807436425 shutdown_connections 2024-04-23T14:43:15.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f9a00640 1 --2- 172.21.15.59:0/2807436425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f40af3e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f9a00640 1 -- 172.21.15.59:0/2807436425 >> 172.21.15.59:0/2807436425 conn(0x7f50f4067ea0 msgr2=0x7f50f406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:15.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f9a00640 1 -- 172.21.15.59:0/2807436425 shutdown_connections 2024-04-23T14:43:15.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.440+0000 7f50f9a00640 1 -- 172.21.15.59:0/2807436425 wait complete. 2024-04-23T14:43:15.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.442+0000 7f50f9a00640 1 Processor -- start 2024-04-23T14:43:15.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.442+0000 7f50f9a00640 1 -- start start 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.443+0000 7f50f9a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50f40af920 0x7f50f40ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.443+0000 7f50f9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f40d08b0 0x7f50f4143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.443+0000 7f50f9a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f4145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.443+0000 7f50f9a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4162be0 con 0x7f50f40d08b0 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.443+0000 7f50f9a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50f415fa20 con 0x7f50f415c5e0 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.443+0000 7f50f9a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4162d60 con 0x7f50f40af920 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.443+0000 7f50f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f40d08b0 0x7f50f4143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f40d08b0 0x7f50f4143430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46166/0 (socket says 172.21.15.59:46166) 2024-04-23T14:43:15.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 -- 172.21.15.59:0/3026566084 learned_addr learned my addr 172.21.15.59:0/3026566084 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f3400640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f4145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 -- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50f40af920 msgr2=0x7f50f40ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50f40af920 0x7f50f40ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50f40af920 0x7f50f40ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 -- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 msgr2=0x7f50f4145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f4145d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 -- 172.21.15.59:0/3026566084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f4143970 con 0x7f50f40d08b0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f3400640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f4145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.444+0000 7f50f2000640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f40d08b0 0x7f50f4143430 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c400 tx=0x7f50e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.445+0000 7f50eaa00640 1 -- 172.21.15.59:0/3026566084 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e00052d0 con 0x7f50f40d08b0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.445+0000 7f50eaa00640 1 -- 172.21.15.59:0/3026566084 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e0005490 con 0x7f50f40d08b0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.445+0000 7f50eaa00640 1 -- 172.21.15.59:0/3026566084 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0007260 con 0x7f50f40d08b0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.445+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f4146780 con 0x7f50f40d08b0 2024-04-23T14:43:15.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.445+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f415d6e0 con 0x7f50f40d08b0 2024-04-23T14:43:15.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.446+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4068a80 con 0x7f50f40d08b0 2024-04-23T14:43:15.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.447+0000 7f50eaa00640 1 -- 172.21.15.59:0/3026566084 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50e0007400 con 0x7f50f40d08b0 2024-04-23T14:43:15.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.448+0000 7f50eaa00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f50d406c8f0 0x7f50d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.448+0000 7f50eaa00640 1 -- 172.21.15.59:0/3026566084 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7f50e008b140 con 0x7f50f40d08b0 2024-04-23T14:43:15.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.448+0000 7f50f2a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f50d406c8f0 0x7f50d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.449+0000 7f50f2a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f50d406c8f0 0x7f50d406edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f50e4034000 tx=0x7f50e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:15.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.453+0000 7f50eaa00640 1 -- 172.21.15.59:0/3026566084 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50e00594b0 con 0x7f50f40d08b0 2024-04-23T14:43:15.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.658+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50f406e260 con 0x7f50f40d08b0 2024-04-23T14:43:15.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.661+0000 7f50eaa00640 1 -- 172.21.15.59:0/3026566084 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16806 (secure 0 0 0) 0x7f50e0058db0 con 0x7f50f40d08b0 2024-04-23T14:43:15.664 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:15.664 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":101,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:10.119057+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:41:27.465708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-23T14:41:28.586680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6835/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6837/620420969":"2024-04-24T14:42:32.618734+0000","172.21.15.59:6835/620420969":"2024-04-24T14:42:32.618734+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f50d406c8f0 msgr2=0x7f50d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f50d406c8f0 0x7f50d406edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f50e4034000 tx=0x7f50e402e000 comp rx=0 tx=0).stop 2024-04-23T14:43:15.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f40d08b0 msgr2=0x7f50f4143430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f40d08b0 0x7f50f4143430 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c400 tx=0x7f50e000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 shutdown_connections 2024-04-23T14:43:15.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f50d406c8f0 0x7f50d406edb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f415c5e0 0x7f50f4145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50f40d08b0 0x7f50f4143430 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 --2- 172.21.15.59:0/3026566084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50f40af920 0x7f50f40ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 >> 172.21.15.59:0/3026566084 conn(0x7f50f4067ea0 msgr2=0x7f50f4157530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.664+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 shutdown_connections 2024-04-23T14:43:15.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.665+0000 7f50f9a00640 1 -- 172.21.15.59:0/3026566084 wait complete. 2024-04-23T14:43:15.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:43:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.903+0000 7fcd7f200640 1 Processor -- start 2024-04-23T14:43:15.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.903+0000 7fcd7f200640 1 -- start start 2024-04-23T14:43:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.904+0000 7fcd7f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd800cd8b0 0x7fcd800d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.904+0000 7fcd7f200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd800d0b00 0x7fcd8006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.904+0000 7fcd7f200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd800b4cf0 con 0x7fcd8015c670 2024-04-23T14:43:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.904+0000 7fcd7f200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8015ca50 con 0x7fcd800d0b00 2024-04-23T14:43:15.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.904+0000 7fcd7f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcd800ccb80 con 0x7fcd800cd8b0 2024-04-23T14:43:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.905+0000 7fcd7e800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd800d0b00 0x7fcd8006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.905+0000 7fcd7e800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd800d0b00 0x7fcd8006ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:44550/0 (socket says 172.21.15.59:44550) 2024-04-23T14:43:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.905+0000 7fcd7e800640 1 -- 172.21.15.59:0/2882355465 learned_addr learned my addr 172.21.15.59:0/2882355465 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:15.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.905+0000 7fcd7de00640 1 --1- 172.21.15.59:0/2882355465 >> v1:172.21.15.76:6789/0 conn(0x7fcd8015c670 0x7fcd800b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:35698/0 (socket says 172.21.15.59:35698) 2024-04-23T14:43:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.905+0000 7fcd7d400640 1 --2- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd800cd8b0 0x7fcd800d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.906+0000 7fcd7e800640 1 -- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd800cd8b0 msgr2=0x7fcd800d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.906+0000 7fcd7e800640 1 --2- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd800cd8b0 0x7fcd800d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.906+0000 7fcd7e800640 1 -- 172.21.15.59:0/2882355465 >> v1:172.21.15.76:6789/0 conn(0x7fcd8015c670 legacy=0x7fcd800b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.906+0000 7fcd7e800640 1 -- 172.21.15.59:0/2882355465 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd800ccdd0 con 0x7fcd800d0b00 2024-04-23T14:43:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.907+0000 7fcd7e800640 1 --2- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800d0b00 0x7fcd8006ec50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009870 tx=0x7fcd7802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4f8bfd01a3bf8f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.907+0000 7fcd7ca00640 1 -- 172.21.15.59:0/2882355465 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78037070 con 0x7fcd800d0b00 2024-04-23T14:43:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.907+0000 7fcd7ca00640 1 -- 172.21.15.59:0/2882355465 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd78032440 con 0x7fcd800d0b00 2024-04-23T14:43:15.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.907+0000 7fcd7ca00640 1 -- 172.21.15.59:0/2882355465 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78032810 con 0x7fcd800d0b00 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.908+0000 7fcd7f200640 1 -- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800d0b00 msgr2=0x7fcd8006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.908+0000 7fcd7f200640 1 --2- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800d0b00 0x7fcd8006ec50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009870 tx=0x7fcd7802a250 comp rx=0 tx=0).stop 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.908+0000 7fcd7f200640 1 -- 172.21.15.59:0/2882355465 shutdown_connections 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.908+0000 7fcd7f200640 1 --2- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800d0b00 0x7fcd8006ec50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.908+0000 7fcd7f200640 1 --2- 172.21.15.59:0/2882355465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd800cd8b0 0x7fcd800d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.908+0000 7fcd7f200640 1 -- 172.21.15.59:0/2882355465 >> 172.21.15.59:0/2882355465 conn(0x7fcd8006bdf0 msgr2=0x7fcd8006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.909+0000 7fcd7f200640 1 -- 172.21.15.59:0/2882355465 shutdown_connections 2024-04-23T14:43:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.909+0000 7fcd7f200640 1 -- 172.21.15.59:0/2882355465 wait complete. 2024-04-23T14:43:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.910+0000 7fcd7f200640 1 Processor -- start 2024-04-23T14:43:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.910+0000 7fcd7f200640 1 -- start start 2024-04-23T14:43:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.911+0000 7fcd7f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800cd8b0 0x7fcd80155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.911+0000 7fcd7f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd800d0b00 0x7fcd80158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.911+0000 7fcd7f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd8015c670 0x7fcd801e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.911+0000 7fcd7f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd800b3e60 con 0x7fcd800d0b00 2024-04-23T14:43:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.911+0000 7fcd7f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcd800cd390 con 0x7fcd800cd8b0 2024-04-23T14:43:15.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.911+0000 7fcd7f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcd800b3fe0 con 0x7fcd8015c670 2024-04-23T14:43:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd800d0b00 0x7fcd80158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd800d0b00 0x7fcd80158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46178/0 (socket says 172.21.15.59:46178) 2024-04-23T14:43:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 -- 172.21.15.59:0/597661633 learned_addr learned my addr 172.21.15.59:0/597661633 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7de00640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800cd8b0 0x7fcd80155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7e800640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd8015c670 0x7fcd801e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 -- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd8015c670 msgr2=0x7fcd801e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd8015c670 0x7fcd801e5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 -- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800cd8b0 msgr2=0x7fcd80155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800cd8b0 0x7fcd80155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.912+0000 7fcd7d400640 1 -- 172.21.15.59:0/597661633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd801570b0 con 0x7fcd800d0b00 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.913+0000 7fcd7d400640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd800d0b00 0x7fcd80158150 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800e9b0 tx=0x7fcd6800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.913+0000 7fcd76000640 1 -- 172.21.15.59:0/597661633 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68017070 con 0x7fcd800d0b00 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.913+0000 7fcd76000640 1 -- 172.21.15.59:0/597661633 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6800c9a0 con 0x7fcd800d0b00 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.913+0000 7fcd76000640 1 -- 172.21.15.59:0/597661633 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800cd70 con 0x7fcd800d0b00 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.913+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd80157390 con 0x7fcd800d0b00 2024-04-23T14:43:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.913+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd80157950 con 0x7fcd800d0b00 2024-04-23T14:43:15.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.915+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd80157e40 con 0x7fcd800d0b00 2024-04-23T14:43:15.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.916+0000 7fcd76000640 1 -- 172.21.15.59:0/597661633 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd6801a070 con 0x7fcd800d0b00 2024-04-23T14:43:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.917+0000 7fcd76000640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:15.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.917+0000 7fcd76000640 1 -- 172.21.15.59:0/597661633 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7fcd6808bbe0 con 0x7fcd800d0b00 2024-04-23T14:43:15.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.917+0000 7fcd7de00640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:15.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.918+0000 7fcd7de00640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fcd700012e0 tx=0x7fcd70005b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:15.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:15.922+0000 7fcd76000640 1 -- 172.21.15.59:0/597661633 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd68059f60 con 0x7fcd800d0b00 2024-04-23T14:43:16.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:16.165+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd800ceab0 con 0x7fcd800d0b00 2024-04-23T14:43:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.172+0000 7fcd76000640 1 -- 172.21.15.59:0/597661633 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7fcd68059860 con 0x7fcd800d0b00 2024-04-23T14:43:17.173 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:43:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.175+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcd5c06c8f0 msgr2=0x7fcd5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.175+0000 7fcd7f200640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fcd700012e0 tx=0x7fcd70005b70 comp rx=0 tx=0).stop 2024-04-23T14:43:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.175+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd800d0b00 msgr2=0x7fcd80158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.175+0000 7fcd7f200640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd800d0b00 0x7fcd80158150 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800e9b0 tx=0x7fcd6800ee70 comp rx=0 tx=0).stop 2024-04-23T14:43:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 shutdown_connections 2024-04-23T14:43:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fcd5c06c8f0 0x7fcd5c06edb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd8015c670 0x7fcd801e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd800d0b00 0x7fcd80158150 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 --2- 172.21.15.59:0/597661633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd800cd8b0 0x7fcd80155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 >> 172.21.15.59:0/597661633 conn(0x7fcd8006bdf0 msgr2=0x7fcd8015cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 shutdown_connections 2024-04-23T14:43:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.176+0000 7fcd7f200640 1 -- 172.21.15.59:0/597661633 wait complete. 2024-04-23T14:43:17.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:43:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.409+0000 7f7ad9e00640 1 Processor -- start 2024-04-23T14:43:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.409+0000 7f7ad9e00640 1 -- start start 2024-04-23T14:43:17.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.411+0000 7f7ad9e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ad406f6f0 0x7f7ad40aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:17.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.411+0000 7f7ad9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad4071d70 0x7f7ad40721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:17.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.411+0000 7f7ad9e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad40adc80 con 0x7f7ad415e790 2024-04-23T14:43:17.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.411+0000 7f7ad9e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad40aef40 con 0x7f7ad406f6f0 2024-04-23T14:43:17.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.411+0000 7f7ad9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad4072720 con 0x7f7ad4071d70 2024-04-23T14:43:17.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.412+0000 7f7ad2000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ad406f6f0 0x7f7ad40aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:17.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.412+0000 7f7ad3400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad4071d70 0x7f7ad40721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:17.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.412+0000 7f7ad2000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ad406f6f0 0x7f7ad40aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:44574/0 (socket says 172.21.15.59:44574) 2024-04-23T14:43:17.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.412+0000 7f7ad2000640 1 -- 172.21.15.59:0/587762346 learned_addr learned my addr 172.21.15.59:0/587762346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.413+0000 7f7ad3400640 1 -- 172.21.15.59:0/587762346 >> v2:172.21.15.76:3300/0 conn(0x7f7ad406f6f0 msgr2=0x7f7ad40aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.413+0000 7f7ad3400640 1 --2- 172.21.15.59:0/587762346 >> v2:172.21.15.76:3300/0 conn(0x7f7ad406f6f0 0x7f7ad40aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.413+0000 7f7ad3400640 1 -- 172.21.15.59:0/587762346 >> v1:172.21.15.76:6789/0 conn(0x7f7ad415e790 legacy=0x7f7ad4162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.413+0000 7f7ad3400640 1 -- 172.21.15.59:0/587762346 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad40adf20 con 0x7f7ad4071d70 2024-04-23T14:43:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.413+0000 7f7ad1600640 1 -- 172.21.15.59:0/587762346 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247803620 0 0) 0x7f7ad40adc80 con 0x7f7ad415e790 2024-04-23T14:43:17.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.413+0000 7f7ad2000640 1 --2- 172.21.15.59:0/587762346 >> v2:172.21.15.76:3300/0 conn(0x7f7ad406f6f0 0x7f7ad40aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:43:17.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.414+0000 7f7ad3400640 1 --2- 172.21.15.59:0/587762346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad4071d70 0x7f7ad40721e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009d90 tx=0x7f7acc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8d8f607b4a335c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.415+0000 7f7ad1600640 1 -- 172.21.15.59:0/587762346 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc007740 con 0x7f7ad4071d70 2024-04-23T14:43:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.415+0000 7f7ad1600640 1 -- 172.21.15.59:0/587762346 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7acc0078e0 con 0x7f7ad4071d70 2024-04-23T14:43:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.415+0000 7f7ad1600640 1 -- 172.21.15.59:0/587762346 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc007cb0 con 0x7f7ad4071d70 2024-04-23T14:43:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 -- 172.21.15.59:0/587762346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad4071d70 msgr2=0x7f7ad40721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/587762346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad4071d70 0x7f7ad40721e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009d90 tx=0x7f7acc02a660 comp rx=0 tx=0).stop 2024-04-23T14:43:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 -- 172.21.15.59:0/587762346 shutdown_connections 2024-04-23T14:43:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/587762346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad4071d70 0x7f7ad40721e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/587762346 >> v2:172.21.15.76:3300/0 conn(0x7f7ad406f6f0 0x7f7ad40aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 -- 172.21.15.59:0/587762346 >> 172.21.15.59:0/587762346 conn(0x7f7ad4067d90 msgr2=0x7f7ad406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:17.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 -- 172.21.15.59:0/587762346 shutdown_connections 2024-04-23T14:43:17.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.416+0000 7f7ad9e00640 1 -- 172.21.15.59:0/587762346 wait complete. 2024-04-23T14:43:17.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.418+0000 7f7ad9e00640 1 Processor -- start 2024-04-23T14:43:17.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.418+0000 7f7ad9e00640 1 -- start start 2024-04-23T14:43:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad406f6f0 0x7f7ad415d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad9e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad4071d70 0x7f7ad41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad9e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad415e790 0x7f7ad40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad9e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad40ae4e0 con 0x7f7ad406f6f0 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad9e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad406fbb0 con 0x7f7ad4071d70 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad9e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad415eb70 con 0x7f7ad415e790 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad406f6f0 0x7f7ad415d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.419+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad406f6f0 0x7f7ad415d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46188/0 (socket says 172.21.15.59:46188) 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2a00640 1 -- 172.21.15.59:0/2245005156 learned_addr learned my addr 172.21.15.59:0/2245005156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad3400640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad415e790 0x7f7ad40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2000640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad4071d70 0x7f7ad41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2a00640 1 -- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad415e790 msgr2=0x7f7ad40cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2a00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad415e790 0x7f7ad40cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2a00640 1 -- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad4071d70 msgr2=0x7f7ad41545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2a00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad4071d70 0x7f7ad41545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2a00640 1 -- 172.21.15.59:0/2245005156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad415c340 con 0x7f7ad406f6f0 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.420+0000 7f7ad2a00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad406f6f0 0x7f7ad415d3e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c410 tx=0x7f7ac400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.421+0000 7f7acb400640 1 -- 172.21.15.59:0/2245005156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4017070 con 0x7f7ad406f6f0 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.421+0000 7f7acb400640 1 -- 172.21.15.59:0/2245005156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac40035f0 con 0x7f7ad406f6f0 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.421+0000 7f7acb400640 1 -- 172.21.15.59:0/2245005156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40039e0 con 0x7f7ad406f6f0 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.421+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad415c620 con 0x7f7ad406f6f0 2024-04-23T14:43:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.421+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad415cbe0 con 0x7f7ad406f6f0 2024-04-23T14:43:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.422+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad4068a80 con 0x7f7ad406f6f0 2024-04-23T14:43:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.423+0000 7f7acb400640 1 -- 172.21.15.59:0/2245005156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ac401a070 con 0x7f7ad406f6f0 2024-04-23T14:43:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.424+0000 7f7acb400640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ab406c8f0 0x7f7ab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.424+0000 7f7acb400640 1 -- 172.21.15.59:0/2245005156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7f7ac408b140 con 0x7f7ad406f6f0 2024-04-23T14:43:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.424+0000 7f7ad2000640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ab406c8f0 0x7f7ab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:17.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.425+0000 7f7ad2000640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ab406c8f0 0x7f7ab406edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7ad406bc80 tx=0x7f7ac0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.429+0000 7f7acb400640 1 -- 172.21.15.59:0/2245005156 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ac4059390 con 0x7f7ad406f6f0 2024-04-23T14:43:17.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:17.677+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7ad4162190 con 0x7f7ad406f6f0 2024-04-23T14:43:18.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.186+0000 7f7acb400640 1 -- 172.21.15.59:0/2245005156 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) ==== 104+0+0 (secure 0 0 0) 0x7f7ac4058c90 con 0x7f7ad406f6f0 2024-04-23T14:43:18.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ab406c8f0 msgr2=0x7f7ab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:18.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ab406c8f0 0x7f7ab406edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7ad406bc80 tx=0x7f7ac0008040 comp rx=0 tx=0).stop 2024-04-23T14:43:18.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad406f6f0 msgr2=0x7f7ad415d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:18.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad406f6f0 0x7f7ad415d3e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c410 tx=0x7f7ac400c8d0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 shutdown_connections 2024-04-23T14:43:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ab406c8f0 0x7f7ab406edb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7ad415e790 0x7f7ad40cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad4071d70 0x7f7ad41545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 --2- 172.21.15.59:0/2245005156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad406f6f0 0x7f7ad415d3e0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 >> 172.21.15.59:0/2245005156 conn(0x7f7ad4067d90 msgr2=0x7f7ad40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.190+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 shutdown_connections 2024-04-23T14:43:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.191+0000 7f7ad9e00640 1 -- 172.21.15.59:0/2245005156 wait complete. 2024-04-23T14:43:18.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:43:18.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.439+0000 7f8f63e00640 1 Processor -- start 2024-04-23T14:43:18.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.440+0000 7f8f63e00640 1 -- start start 2024-04-23T14:43:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.441+0000 7f8f63e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f6415e8e0 0x7f8f640cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.441+0000 7f8f63e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f640d4210 0x7f8f640d4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.441+0000 7f8f63e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f640cf070 con 0x7f8f6406cbb0 2024-04-23T14:43:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.441+0000 7f8f63e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8f640d4bd0 con 0x7f8f6415e8e0 2024-04-23T14:43:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.441+0000 7f8f63e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f640cf2f0 con 0x7f8f640d4210 2024-04-23T14:43:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.442+0000 7f8f63400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8f6406cbb0 0x7f8f6406d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56270/0 (socket says 172.21.15.59:56270) 2024-04-23T14:43:18.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.442+0000 7f8f63400640 1 -- 172.21.15.59:0/1149342876 learned_addr learned my addr 172.21.15.59:0/1149342876 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.442+0000 7f8f62a00640 1 --2- 172.21.15.59:0/1149342876 >> v2:172.21.15.76:3300/0 conn(0x7f8f6415e8e0 0x7f8f640cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.442+0000 7f8f62000640 1 --2- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f640d4210 0x7f8f640d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.442+0000 7f8f61600640 1 -- 172.21.15.59:0/1149342876 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883475447 0 0) 0x7f8f5c001110 con 0x7f8f6406cbb0 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.443+0000 7f8f61600640 1 -- 172.21.15.59:0/1149342876 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f48003520 con 0x7f8f6406cbb0 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.443+0000 7f8f62a00640 1 -- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f640d4210 msgr2=0x7f8f640d4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.443+0000 7f8f62a00640 1 --2- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f640d4210 0x7f8f640d4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.443+0000 7f8f62a00640 1 -- 172.21.15.59:0/1149342876 >> v1:172.21.15.59:6789/0 conn(0x7f8f6406cbb0 legacy=0x7f8f6406d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:18.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.443+0000 7f8f62a00640 1 -- 172.21.15.59:0/1149342876 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f640d1c60 con 0x7f8f6415e8e0 2024-04-23T14:43:18.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.444+0000 7f8f62a00640 1 --2- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f6415e8e0 0x7f8f640cfdb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009d90 tx=0x7f8f5402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d8f7c7ad8465db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:18.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f61600640 1 -- 172.21.15.59:0/1149342876 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54007740 con 0x7f8f6415e8e0 2024-04-23T14:43:18.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f61600640 1 -- 172.21.15.59:0/1149342876 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f540078e0 con 0x7f8f6415e8e0 2024-04-23T14:43:18.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f61600640 1 -- 172.21.15.59:0/1149342876 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54007cb0 con 0x7f8f6415e8e0 2024-04-23T14:43:18.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 -- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f6415e8e0 msgr2=0x7f8f640cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:18.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f6415e8e0 0x7f8f640cfdb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009d90 tx=0x7f8f5402a660 comp rx=0 tx=0).stop 2024-04-23T14:43:18.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 -- 172.21.15.59:0/1149342876 shutdown_connections 2024-04-23T14:43:18.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f640d4210 0x7f8f640d4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1149342876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f6415e8e0 0x7f8f640cfdb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 -- 172.21.15.59:0/1149342876 >> 172.21.15.59:0/1149342876 conn(0x7f8f6408e060 msgr2=0x7f8f6409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:18.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 -- 172.21.15.59:0/1149342876 shutdown_connections 2024-04-23T14:43:18.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.445+0000 7f8f63e00640 1 -- 172.21.15.59:0/1149342876 wait complete. 2024-04-23T14:43:18.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.447+0000 7f8f63e00640 1 Processor -- start 2024-04-23T14:43:18.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.447+0000 7f8f63e00640 1 -- start start 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.448+0000 7f8f63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406cbb0 0x7f8f640a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.448+0000 7f8f63e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f640d4210 0x7f8f640b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.448+0000 7f8f63e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f6415e8e0 0x7f8f6415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.448+0000 7f8f63e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6415ecc0 con 0x7f8f6406cbb0 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.448+0000 7f8f63e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f640cf890 con 0x7f8f640d4210 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.448+0000 7f8f63e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8f640d02f0 con 0x7f8f6415e8e0 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406cbb0 0x7f8f640a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406cbb0 0x7f8f640a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46194/0 (socket says 172.21.15.59:46194) 2024-04-23T14:43:18.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 -- 172.21.15.59:0/1286158441 learned_addr learned my addr 172.21.15.59:0/1286158441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:18.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f63400640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f6415e8e0 0x7f8f6415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62000640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f640d4210 0x7f8f640b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 -- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f6415e8e0 msgr2=0x7f8f6415bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f6415e8e0 0x7f8f6415bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 -- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f640d4210 msgr2=0x7f8f640b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f640d4210 0x7f8f640b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.449+0000 7f8f62a00640 1 -- 172.21.15.59:0/1286158441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6415c240 con 0x7f8f6406cbb0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.450+0000 7f8f62a00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406cbb0 0x7f8f640a1f20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009d90 tx=0x7f8f54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.450+0000 7f8f5aa00640 1 -- 172.21.15.59:0/1286158441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5403f070 con 0x7f8f6406cbb0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.450+0000 7f8f5aa00640 1 -- 172.21.15.59:0/1286158441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54031c40 con 0x7f8f6406cbb0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.450+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f640b7960 con 0x7f8f6406cbb0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.450+0000 7f8f5aa00640 1 -- 172.21.15.59:0/1286158441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54009450 con 0x7f8f6406cbb0 2024-04-23T14:43:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.450+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f640b7f20 con 0x7f8f6406cbb0 2024-04-23T14:43:18.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.451+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6404fd90 con 0x7f8f6406cbb0 2024-04-23T14:43:18.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.452+0000 7f8f5aa00640 1 -- 172.21.15.59:0/1286158441 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f54044460 con 0x7f8f6406cbb0 2024-04-23T14:43:18.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.454+0000 7f8f5aa00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f4006c8f0 0x7f8f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:18.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.454+0000 7f8f5aa00640 1 -- 172.21.15.59:0/1286158441 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5881+0+0 (secure 0 0 0) 0x7f8f540ad370 con 0x7f8f6406cbb0 2024-04-23T14:43:18.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.454+0000 7f8f62000640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f4006c8f0 0x7f8f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.455+0000 7f8f62000640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f4006c8f0 0x7f8f4006edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8f640b40b0 tx=0x7f8f4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.458+0000 7f8f5aa00640 1 -- 172.21.15.59:0/1286158441 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f5407b6e0 con 0x7f8f6406cbb0 2024-04-23T14:43:18.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:18.664+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8f6406c990 con 0x7f8f6406cbb0 2024-04-23T14:43:19.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.197+0000 7f8f5aa00640 1 -- 172.21.15.59:0/1286158441 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f8f5407afe0 con 0x7f8f6406cbb0 2024-04-23T14:43:19.198 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:43:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.200+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f4006c8f0 msgr2=0x7f8f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.200+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f4006c8f0 0x7f8f4006edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8f640b40b0 tx=0x7f8f4c008040 comp rx=0 tx=0).stop 2024-04-23T14:43:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.200+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406cbb0 msgr2=0x7f8f640a1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.200+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406cbb0 0x7f8f640a1f20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009d90 tx=0x7f8f54004a40 comp rx=0 tx=0).stop 2024-04-23T14:43:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 shutdown_connections 2024-04-23T14:43:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8f4006c8f0 0x7f8f4006edb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:19.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f6415e8e0 0x7f8f6415bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f640d4210 0x7f8f640b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 --2- 172.21.15.59:0/1286158441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6406cbb0 0x7f8f640a1f20 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 >> 172.21.15.59:0/1286158441 conn(0x7f8f6408e060 msgr2=0x7f8f640d35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 shutdown_connections 2024-04-23T14:43:19.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.201+0000 7f8f63e00640 1 -- 172.21.15.59:0/1286158441 wait complete. 2024-04-23T14:43:19.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.428+0000 7fa63a800640 1 Processor -- start 2024-04-23T14:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.429+0000 7fa63a800640 1 -- start start 2024-04-23T14:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.430+0000 7fa63a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa63406cbb0 0x7fa63406d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.430+0000 7fa63a800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340cf070 con 0x7fa6340d4210 2024-04-23T14:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.430+0000 7fa63a800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340cf2f0 con 0x7fa63415e8e0 2024-04-23T14:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.430+0000 7fa63a800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fa6340d04c0 con 0x7fa63406cbb0 2024-04-23T14:43:19.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.431+0000 7fa638a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa6340d4210 0x7fa6340d46c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56284/0 (socket says 172.21.15.59:56284) 2024-04-23T14:43:19.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.431+0000 7fa638a00640 1 -- 172.21.15.59:0/279306003 learned_addr learned my addr 172.21.15.59:0/279306003 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:19.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.431+0000 7fa639e00640 1 --2- 172.21.15.59:0/279306003 >> v2:172.21.15.59:3300/0 conn(0x7fa63406cbb0 0x7fa63406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.431+0000 7fa633e00640 1 -- 172.21.15.59:0/279306003 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975528717 0 0) 0x7fa620001290 con 0x7fa6340d4210 2024-04-23T14:43:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.431+0000 7fa633e00640 1 -- 172.21.15.59:0/279306003 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa61c003520 con 0x7fa6340d4210 2024-04-23T14:43:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.431+0000 7fa639e00640 1 -- 172.21.15.59:0/279306003 >> v1:172.21.15.76:6789/0 conn(0x7fa63415e8e0 legacy=0x7fa6340cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.431+0000 7fa639e00640 1 -- 172.21.15.59:0/279306003 >> v1:172.21.15.59:6789/0 conn(0x7fa6340d4210 legacy=0x7fa6340d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.432+0000 7fa639e00640 1 -- 172.21.15.59:0/279306003 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340d1c60 con 0x7fa63406cbb0 2024-04-23T14:43:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.433+0000 7fa639e00640 1 --2- 172.21.15.59:0/279306003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa63406cbb0 0x7fa63406d040 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009d90 tx=0x7fa62c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59034802e8cdbbc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.433+0000 7fa633e00640 1 -- 172.21.15.59:0/279306003 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c007740 con 0x7fa63406cbb0 2024-04-23T14:43:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.433+0000 7fa633e00640 1 -- 172.21.15.59:0/279306003 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62c0078e0 con 0x7fa63406cbb0 2024-04-23T14:43:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.433+0000 7fa633e00640 1 -- 172.21.15.59:0/279306003 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c007cb0 con 0x7fa63406cbb0 2024-04-23T14:43:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.434+0000 7fa63a800640 1 -- 172.21.15.59:0/279306003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa63406cbb0 msgr2=0x7fa63406d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.434+0000 7fa63a800640 1 --2- 172.21.15.59:0/279306003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa63406cbb0 0x7fa63406d040 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009d90 tx=0x7fa62c02a660 comp rx=0 tx=0).stop 2024-04-23T14:43:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.434+0000 7fa63a800640 1 -- 172.21.15.59:0/279306003 shutdown_connections 2024-04-23T14:43:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.434+0000 7fa63a800640 1 --2- 172.21.15.59:0/279306003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa63406cbb0 0x7fa63406d040 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.434+0000 7fa63a800640 1 -- 172.21.15.59:0/279306003 >> 172.21.15.59:0/279306003 conn(0x7fa63408e060 msgr2=0x7fa63409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.434+0000 7fa63a800640 1 -- 172.21.15.59:0/279306003 shutdown_connections 2024-04-23T14:43:19.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.434+0000 7fa63a800640 1 -- 172.21.15.59:0/279306003 wait complete. 2024-04-23T14:43:19.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.436+0000 7fa63a800640 1 Processor -- start 2024-04-23T14:43:19.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.436+0000 7fa63a800640 1 -- start start 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa63a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa63406cbb0 0x7fa6340b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa63a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6340d4210 0x7fa63409e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa63a800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa63415e8e0 0x7fa6341547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa63a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa63406d580 con 0x7fa6340d4210 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa63a800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6340cf890 con 0x7fa63415e8e0 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa63a800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa63415ecc0 con 0x7fa63406cbb0 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa638a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6340d4210 0x7fa63409e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa638a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6340d4210 0x7fa63409e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46220/0 (socket says 172.21.15.59:46220) 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.437+0000 7fa638a00640 1 -- 172.21.15.59:0/3408859751 learned_addr learned my addr 172.21.15.59:0/3408859751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa639e00640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa63415e8e0 0x7fa6341547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa639400640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa63406cbb0 0x7fa6340b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa638a00640 1 -- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa63406cbb0 msgr2=0x7fa6340b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:19.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa638a00640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa63406cbb0 0x7fa6340b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa638a00640 1 -- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa63415e8e0 msgr2=0x7fa6341547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa638a00640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa63415e8e0 0x7fa6341547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa638a00640 1 -- 172.21.15.59:0/3408859751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340b8270 con 0x7fa6340d4210 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa639e00640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa63415e8e0 0x7fa6341547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa638a00640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6340d4210 0x7fa63409e2f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa62000c500 tx=0x7fa62000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa631600640 1 -- 172.21.15.59:0/3408859751 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620017070 con 0x7fa6340d4210 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa631600640 1 -- 172.21.15.59:0/3408859751 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa620007500 con 0x7fa6340d4210 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa631600640 1 -- 172.21.15.59:0/3408859751 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6200059c0 con 0x7fa6340d4210 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.438+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6340b8550 con 0x7fa6340d4210 2024-04-23T14:43:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.439+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6340b8b10 con 0x7fa6340d4210 2024-04-23T14:43:19.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.440+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63404fd90 con 0x7fa6340d4210 2024-04-23T14:43:19.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.441+0000 7fa631600640 1 -- 172.21.15.59:0/3408859751 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa62001a070 con 0x7fa6340d4210 2024-04-23T14:43:19.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.442+0000 7fa631600640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:19.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.442+0000 7fa631600640 1 -- 172.21.15.59:0/3408859751 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5443+0+0 (secure 0 0 0) 0x7fa62008aea0 con 0x7fa6340d4210 2024-04-23T14:43:19.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.442+0000 7fa639400640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.443+0000 7fa639400640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa61406c8f0 0x7fa61406edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa6280015a0 tx=0x7fa628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.447+0000 7fa631600640 1 -- 172.21.15.59:0/3408859751 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6340cf070 con 0x7fa6340d4210 2024-04-23T14:43:19.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:19.660+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa63406c990 con 0x7fa6340d4210 2024-04-23T14:43:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.210+0000 7fa631600640 1 -- 172.21.15.59:0/3408859751 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7fa6200593c0 con 0x7fa6340d4210 2024-04-23T14:43:20.212 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:43:20.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.218+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa61406c8f0 msgr2=0x7fa61406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.218+0000 7fa63a800640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa61406c8f0 0x7fa61406edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa6280015a0 tx=0x7fa628009040 comp rx=0 tx=0).stop 2024-04-23T14:43:20.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.218+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6340d4210 msgr2=0x7fa63409e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.218+0000 7fa63a800640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6340d4210 0x7fa63409e2f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa62000c500 tx=0x7fa62000c9c0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 shutdown_connections 2024-04-23T14:43:20.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa61406c8f0 0x7fa61406edb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa63415e8e0 0x7fa6341547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6340d4210 0x7fa63409e2f0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 --2- 172.21.15.59:0/3408859751 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa63406cbb0 0x7fa6340b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 >> 172.21.15.59:0/3408859751 conn(0x7fa63408e060 msgr2=0x7fa63406d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 shutdown_connections 2024-04-23T14:43:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.219+0000 7fa63a800640 1 -- 172.21.15.59:0/3408859751 wait complete. 2024-04-23T14:43:20.250 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:43:20.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.446+0000 7f98a6c00640 1 Processor -- start 2024-04-23T14:43:20.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.447+0000 7f98a6c00640 1 -- start start 2024-04-23T14:43:20.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.448+0000 7f98a6c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 0x7f98a00d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.448+0000 7f98a6c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98a00d1960 0x7f98a009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.448+0000 7f98a6c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a00cce30 con 0x7f98a009af40 2024-04-23T14:43:20.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.448+0000 7f98a6c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f98a009b3b0 con 0x7f98a00d1960 2024-04-23T14:43:20.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.448+0000 7f98a6c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f98a00b8590 con 0x7f98a00b81b0 2024-04-23T14:43:20.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.449+0000 7f98a4c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f98a009af40 0x7f98a006ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36476/0 (socket says 172.21.15.59:36476) 2024-04-23T14:43:20.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.449+0000 7f98a4c00640 1 -- 172.21.15.59:0/1344131323 learned_addr learned my addr 172.21.15.59:0/1344131323 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.449+0000 7f989fe00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 0x7f98a00d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.449+0000 7f9897400640 1 --2- 172.21.15.59:0/1344131323 >> v2:172.21.15.76:3300/0 conn(0x7f98a00d1960 0x7f98a009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.449+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093317883 0 0) 0x7f98a00cce30 con 0x7f98a009af40 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.449+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9884003520 con 0x7f98a009af40 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1790033888 0 0) 0x7f9884003520 con 0x7f98a009af40 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 msgr2=0x7f98a00d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 0x7f98a00d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 >> v2:172.21.15.76:3300/0 conn(0x7f98a00d1960 msgr2=0x7f98a009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 --2- 172.21.15.59:0/1344131323 >> v2:172.21.15.76:3300/0 conn(0x7f98a00d1960 0x7f98a009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a00cd110 con 0x7f98a009af40 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f98980019f0 con 0x7f98a009af40 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 >> v1:172.21.15.59:6789/0 conn(0x7f98a009af40 legacy=0x7f98a006ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f9884003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9884004800 0x7f9884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 0x7f98840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98a009b3b0 con 0x7f98840039b0 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f988400b9c0 con 0x7f9884004800 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f988400bb40 con 0x7f98a00b81b0 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.450+0000 7f989fe00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f9884003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f98a4c00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 0x7f98840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f9897400640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9884004800 0x7f9884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f989fe00640 1 -- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 msgr2=0x7f98840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f989fe00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 0x7f98840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f989fe00640 1 -- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9884004800 msgr2=0x7f9884004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:20.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f989fe00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9884004800 0x7f9884004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f989fe00640 1 -- 172.21.15.59:0/1344131323 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9884003520 con 0x7f98840039b0 2024-04-23T14:43:20.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.451+0000 7f98a4c00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b81b0 0x7f98840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.452+0000 7f989fe00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f9884003dd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f9890009ac0 tx=0x7f9884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e3d8ad79c0d169 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.452+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890036070 con 0x7f98840039b0 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.452+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890031400 con 0x7f98840039b0 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.452+0000 7f989f400640 1 -- 172.21.15.59:0/1344131323 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890009690 con 0x7f98840039b0 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.453+0000 7f98a6c00640 1 -- 172.21.15.59:0/1344131323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 msgr2=0x7f9884003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.453+0000 7f98a6c00640 1 --2- 172.21.15.59:0/1344131323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f9884003dd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f9890009ac0 tx=0x7f9884005830 comp rx=0 tx=0).stop 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.453+0000 7f98a4c00640 1 -- 172.21.15.59:0/1344131323 reap_dead start 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.453+0000 7f98a6c00640 1 -- 172.21.15.59:0/1344131323 shutdown_connections 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.453+0000 7f98a6c00640 1 -- 172.21.15.59:0/1344131323 >> 172.21.15.59:0/1344131323 conn(0x7f98a00b4130 msgr2=0x7f98a00b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:20.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.453+0000 7f98a6c00640 1 -- 172.21.15.59:0/1344131323 shutdown_connections 2024-04-23T14:43:20.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.453+0000 7f98a6c00640 1 -- 172.21.15.59:0/1344131323 wait complete. 2024-04-23T14:43:20.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.455+0000 7f98a6c00640 1 Processor -- start 2024-04-23T14:43:20.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.455+0000 7f98a6c00640 1 -- start start 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f98a6c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f98a0157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f98a6c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a00ca5b0 0x7f98a00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f98a6c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b6820 0x7f98a00b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f98a6c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00b4f10 con 0x7f98840039b0 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f98a6c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f98a00cd6d0 con 0x7f98a00ca5b0 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f98a6c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f98a00b5490 con 0x7f98a00b6820 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f989fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f98a0157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f989fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f98a0157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46880/0 (socket says 172.21.15.59:46880) 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.456+0000 7f989fe00640 1 -- 172.21.15.59:0/3457072150 learned_addr learned my addr 172.21.15.59:0/3457072150 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989fe00640 1 -- 172.21.15.59:0/3457072150 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b6820 msgr2=0x7f98a00b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989fe00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b6820 0x7f98a00b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989fe00640 1 -- 172.21.15.59:0/3457072150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a00ca5b0 msgr2=0x7f98a00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989fe00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a00ca5b0 0x7f98a00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989fe00640 1 -- 172.21.15.59:0/3457072150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a0161380 con 0x7f98840039b0 2024-04-23T14:43:20.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989fe00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f98a0157b30 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9890009be0 tx=0x7f9890004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:20.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989cc00640 1 -- 172.21.15.59:0/3457072150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890040020 con 0x7f98840039b0 2024-04-23T14:43:20.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989cc00640 1 -- 172.21.15.59:0/3457072150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890003bc0 con 0x7f98840039b0 2024-04-23T14:43:20.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f989cc00640 1 -- 172.21.15.59:0/3457072150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890036040 con 0x7f98840039b0 2024-04-23T14:43:20.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a0161660 con 0x7f98840039b0 2024-04-23T14:43:20.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.457+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a0161c20 con 0x7f98840039b0 2024-04-23T14:43:20.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.459+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a006be90 con 0x7f98840039b0 2024-04-23T14:43:20.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.460+0000 7f989cc00640 1 -- 172.21.15.59:0/3457072150 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9890043020 con 0x7f98840039b0 2024-04-23T14:43:20.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.460+0000 7f989cc00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f987c06c8f0 0x7f987c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.461+0000 7f989cc00640 1 -- 172.21.15.59:0/3457072150 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5029+0+0 (secure 0 0 0) 0x7f98900ac0e0 con 0x7f98840039b0 2024-04-23T14:43:20.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.461+0000 7f9897400640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f987c06c8f0 0x7f987c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.461+0000 7f9897400640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f987c06c8f0 0x7f987c06edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f98a0070c90 tx=0x7f9888008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.465+0000 7f989cc00640 1 -- 172.21.15.59:0/3457072150 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98a006be90 con 0x7f98840039b0 2024-04-23T14:43:20.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.667+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f98a0068a80 con 0x7f98840039b0 2024-04-23T14:43:20.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.674+0000 7f989cc00640 1 -- 172.21.15.59:0/3457072150 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f989004a020 con 0x7f98840039b0 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f987c06c8f0 msgr2=0x7f987c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f987c06c8f0 0x7f987c06edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f98a0070c90 tx=0x7f9888008040 comp rx=0 tx=0).stop 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 msgr2=0x7f98a0157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f98a0157b30 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9890009be0 tx=0x7f9890004730 comp rx=0 tx=0).stop 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 shutdown_connections 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f987c06c8f0 0x7f987c06edb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98a00b6820 0x7f98a00b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a00ca5b0 0x7f98a00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 --2- 172.21.15.59:0/3457072150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98840039b0 0x7f98a0157b30 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 >> 172.21.15.59:0/3457072150 conn(0x7f98a00b4130 msgr2=0x7f98a00cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:20.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 shutdown_connections 2024-04-23T14:43:20.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.677+0000 7f98a6c00640 1 -- 172.21.15.59:0/3457072150 wait complete. 2024-04-23T14:43:20.698 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-23T14:43:20.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.910+0000 7f57aee00640 1 Processor -- start 2024-04-23T14:43:20.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.910+0000 7f57aee00640 1 -- start start 2024-04-23T14:43:20.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57aee00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57a815eb60 0x7f57a80d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57aee00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57a80d2d60 0x7f57a80b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57aee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 0x7f57a80b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57aee00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f57a80ba7a0 con 0x7f57a815eb60 2024-04-23T14:43:20.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57aee00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f57a80ba1e0 con 0x7f57a80d2d60 2024-04-23T14:43:20.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57aee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f57a80ba4c0 con 0x7f57a80b3490 2024-04-23T14:43:20.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57a7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57a815eb60 0x7f57a80d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57a7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57a815eb60 0x7f57a80d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46886/0 (socket says 172.21.15.59:46886) 2024-04-23T14:43:20.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.912+0000 7f57a7e00640 1 -- 172.21.15.59:0/334509207 learned_addr learned my addr 172.21.15.59:0/334509207 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.913+0000 7f57a7e00640 1 -- 172.21.15.59:0/334509207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 msgr2=0x7f57a80b97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.913+0000 7f57a7e00640 1 --2- 172.21.15.59:0/334509207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 0x7f57a80b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.913+0000 7f57a7e00640 1 -- 172.21.15.59:0/334509207 >> v2:172.21.15.76:3300/0 conn(0x7f57a80d2d60 msgr2=0x7f57a80b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.913+0000 7f57a7400640 1 --2- 172.21.15.59:0/334509207 >> v2:172.21.15.76:3300/0 conn(0x7f57a80d2d60 0x7f57a80b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.913+0000 7f57a7e00640 1 --2- 172.21.15.59:0/334509207 >> v2:172.21.15.76:3300/0 conn(0x7f57a80d2d60 0x7f57a80b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.913+0000 7f57a7e00640 1 -- 172.21.15.59:0/334509207 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a80b8a90 con 0x7f57a815eb60 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.913+0000 7f57a7400640 1 --2- 172.21.15.59:0/334509207 >> v2:172.21.15.76:3300/0 conn(0x7f57a80d2d60 0x7f57a80b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:20.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.914+0000 7f57a7e00640 1 --2- 172.21.15.59:0/334509207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a80d2820 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f57a80d2140 tx=0x7f579802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7af158d10a7218b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:20.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.914+0000 7f57a6a00640 1 -- 172.21.15.59:0/334509207 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798036070 con 0x7f57a815eb60 2024-04-23T14:43:20.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.914+0000 7f57a6a00640 1 -- 172.21.15.59:0/334509207 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798031600 con 0x7f57a815eb60 2024-04-23T14:43:20.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.914+0000 7f57a6a00640 1 -- 172.21.15.59:0/334509207 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980319d0 con 0x7f57a815eb60 2024-04-23T14:43:20.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 -- 172.21.15.59:0/334509207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 msgr2=0x7f57a80d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 --2- 172.21.15.59:0/334509207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a80d2820 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f57a80d2140 tx=0x7f579802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 -- 172.21.15.59:0/334509207 shutdown_connections 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 --2- 172.21.15.59:0/334509207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 0x7f57a80b97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 --2- 172.21.15.59:0/334509207 >> v2:172.21.15.76:3300/0 conn(0x7f57a80d2d60 0x7f57a80b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 --2- 172.21.15.59:0/334509207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a80d2820 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 -- 172.21.15.59:0/334509207 >> 172.21.15.59:0/334509207 conn(0x7f57a8067b50 msgr2=0x7f57a8067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 -- 172.21.15.59:0/334509207 shutdown_connections 2024-04-23T14:43:20.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.915+0000 7f57aee00640 1 -- 172.21.15.59:0/334509207 wait complete. 2024-04-23T14:43:20.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.918+0000 7f57aee00640 1 Processor -- start 2024-04-23T14:43:20.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.918+0000 7f57aee00640 1 -- start start 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.919+0000 7f57aee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 0x7f57a81f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.919+0000 7f57aee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a806ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.919+0000 7f57aee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80ce100 0x7f57a80cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.919+0000 7f57aee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80b6280 con 0x7f57a815eb60 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.919+0000 7f57aee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80b3c80 con 0x7f57a80ce100 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.919+0000 7f57aee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f57a80b6400 con 0x7f57a80b3490 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a806ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a806ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46888/0 (socket says 172.21.15.59:46888) 2024-04-23T14:43:20.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 -- 172.21.15.59:0/2812282056 learned_addr learned my addr 172.21.15.59:0/2812282056 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7e00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 0x7f57a81f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57ace00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80ce100 0x7f57a80cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 -- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 msgr2=0x7f57a81f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 0x7f57a81f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 -- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80ce100 msgr2=0x7f57a80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80ce100 0x7f57a80cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.920+0000 7f57a7400640 1 -- 172.21.15.59:0/2812282056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a80cc920 con 0x7f57a815eb60 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.921+0000 7f57a7400640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a806ce80 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f579000c7f0 tx=0x7f579000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.921+0000 7f579fe00640 1 -- 172.21.15.59:0/2812282056 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790015070 con 0x7f57a815eb60 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.921+0000 7f579fe00640 1 -- 172.21.15.59:0/2812282056 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5790007a30 con 0x7f57a815eb60 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.921+0000 7f579fe00640 1 -- 172.21.15.59:0/2812282056 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790007e00 con 0x7f57a815eb60 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.921+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a81f7e30 con 0x7f57a815eb60 2024-04-23T14:43:20.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.921+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a81f83b0 con 0x7f57a815eb60 2024-04-23T14:43:20.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.922+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a8068a80 con 0x7f57a815eb60 2024-04-23T14:43:20.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.923+0000 7f579fe00640 1 -- 172.21.15.59:0/2812282056 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5790018070 con 0x7f57a815eb60 2024-04-23T14:43:20.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.924+0000 7f579fe00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:20.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.924+0000 7f579fe00640 1 -- 172.21.15.59:0/2812282056 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5029+0+0 (secure 0 0 0) 0x7f579008b830 con 0x7f57a815eb60 2024-04-23T14:43:20.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.924+0000 7f57a7e00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:20.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.925+0000 7f57a7e00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5798033000 tx=0x7f5798009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:20.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:20.929+0000 7f579fe00640 1 -- 172.21.15.59:0/2812282056 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5790059f90 con 0x7f57a815eb60 2024-04-23T14:43:21.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:21.225+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f57a80c7ca0 con 0x7f57a815eb60 2024-04-23T14:43:21.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:43:21.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:43:22.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.226+0000 7f579fe00640 1 -- 172.21.15.59:0/2812282056 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v555) ==== 146+0+0 (secure 0 0 0) 0x7f5790059890 con 0x7f57a815eb60 2024-04-23T14:43:22.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 msgr2=0x7f578406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5798033000 tx=0x7f5798009000 comp rx=0 tx=0).stop 2024-04-23T14:43:22.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 msgr2=0x7f57a806ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a806ce80 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f579000c7f0 tx=0x7f579000cbf0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 shutdown_connections 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f578406c8f0 0x7f578406edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80ce100 0x7f57a80cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a815eb60 0x7f57a806ce80 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 --2- 172.21.15.59:0/2812282056 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a80b3490 0x7f57a81f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 >> 172.21.15.59:0/2812282056 conn(0x7f57a8067b50 msgr2=0x7f57a815daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 shutdown_connections 2024-04-23T14:43:22.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.230+0000 7f57aee00640 1 -- 172.21.15.59:0/2812282056 wait complete. 2024-04-23T14:43:22.253 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-23T14:43:22.253 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-23T14:43:22.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:22.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.456+0000 7f11a3e00640 1 Processor -- start 2024-04-23T14:43:22.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.457+0000 7f11a3e00640 1 -- start start 2024-04-23T14:43:22.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.458+0000 7f11a3e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11a415e7c0 0x7f11a40d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.458+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 0x7f11a40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.458+0000 7f11a3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a40cf030 con 0x7f11a40d2630 2024-04-23T14:43:22.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.458+0000 7f11a3e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f11a40d1740 con 0x7f11a415e7c0 2024-04-23T14:43:22.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.458+0000 7f11a3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f11a40b7000 con 0x7f11a40d0d90 2024-04-23T14:43:22.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f119a000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f11a40d2630 0x7f11a40b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36490/0 (socket says 172.21.15.59:36490) 2024-04-23T14:43:22.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f119a000640 1 -- 172.21.15.59:0/398023938 learned_addr learned my addr 172.21.15.59:0/398023938 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:22.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2a00640 1 --2- 172.21.15.59:0/398023938 >> v2:172.21.15.76:3300/0 conn(0x7f11a415e7c0 0x7f11a40d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a3400640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 0x7f11a40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4005751260 0 0) 0x7f118c001110 con 0x7f11a40d2630 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1188003520 con 0x7f11a40d2630 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3969559469 0 0) 0x7f1188003520 con 0x7f11a40d2630 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 msgr2=0x7f11a40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 0x7f11a40d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 >> v2:172.21.15.76:3300/0 conn(0x7f11a415e7c0 msgr2=0x7f11a40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 --2- 172.21.15.59:0/398023938 >> v2:172.21.15.76:3300/0 conn(0x7f11a415e7c0 0x7f11a40d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.459+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a40b6a40 con 0x7f11a40d2630 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f118c001840 con 0x7f11a40d2630 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 >> v1:172.21.15.59:6789/0 conn(0x7f11a40d2630 legacy=0x7f11a40b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 0x7f11880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1188003f30 0x7f1188004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1188004d40 0x7f11880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40d1740 con 0x7f1188003f30 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40b7000 con 0x7f1188004d40 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f118800ba10 con 0x7f11a40d0d90 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f119a000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1188003f30 0x7f1188004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a2a00640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 0x7f11880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f11a3400640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1188004d40 0x7f11880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f119a000640 1 -- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 msgr2=0x7f11880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f119a000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11a40d0d90 0x7f11880039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f119a000640 1 -- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1188004d40 msgr2=0x7f11880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f119a000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1188004d40 0x7f11880051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.460+0000 7f119a000640 1 -- 172.21.15.59:0/398023938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1188003520 con 0x7f1188003f30 2024-04-23T14:43:22.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.461+0000 7f119a000640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1188003f30 0x7f1188004800 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c3b0 tx=0x7f1188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4726c4e96823010 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:22.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c00c0e0 con 0x7f1188003f30 2024-04-23T14:43:22.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c034070 con 0x7f1188003f30 2024-04-23T14:43:22.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a2000640 1 -- 172.21.15.59:0/398023938 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c005d70 con 0x7f1188003f30 2024-04-23T14:43:22.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a3e00640 1 -- 172.21.15.59:0/398023938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1188003f30 msgr2=0x7f1188004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a3e00640 1 --2- 172.21.15.59:0/398023938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1188003f30 0x7f1188004800 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c3b0 tx=0x7f1188005830 comp rx=0 tx=0).stop 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a3400640 1 -- 172.21.15.59:0/398023938 reap_dead start 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a3e00640 1 -- 172.21.15.59:0/398023938 shutdown_connections 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a3e00640 1 -- 172.21.15.59:0/398023938 >> 172.21.15.59:0/398023938 conn(0x7f11a409d920 msgr2=0x7f11a408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a3e00640 1 -- 172.21.15.59:0/398023938 shutdown_connections 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.462+0000 7f11a3e00640 1 -- 172.21.15.59:0/398023938 wait complete. 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.464+0000 7f11a3e00640 1 Processor -- start 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.464+0000 7f11a3e00640 1 -- start start 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1188003f30 0x7f11a41f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a40ca850 0x7f11a40b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11a40b59b0 0x7f11a40cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40cf2b0 con 0x7f11a40b59b0 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40cf620 con 0x7f11a40ca850 2024-04-23T14:43:22.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f11a40b6700 con 0x7f1188003f30 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11a40b59b0 0x7f11a40cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11a40b59b0 0x7f11a40cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46904/0 (socket says 172.21.15.59:46904) 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.465+0000 7f11a3400640 1 -- 172.21.15.59:0/2439110476 learned_addr learned my addr 172.21.15.59:0/2439110476 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f11a3400640 1 -- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1188003f30 msgr2=0x7f11a41f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f11a3400640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1188003f30 0x7f11a41f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f11a3400640 1 -- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a40ca850 msgr2=0x7f11a40b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f119a000640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a40ca850 0x7f11a40b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f11a3400640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a40ca850 0x7f11a40b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f11a3400640 1 -- 172.21.15.59:0/2439110476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a40b5e30 con 0x7f11a40b59b0 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f119a000640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a40ca850 0x7f11a40b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f11a3400640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11a40b59b0 0x7f11a40cb5f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f11a40d0b30 tx=0x7f119c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.466+0000 7f119b400640 1 -- 172.21.15.59:0/2439110476 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c019070 con 0x7f11a40b59b0 2024-04-23T14:43:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.467+0000 7f119b400640 1 -- 172.21.15.59:0/2439110476 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c007d40 con 0x7f11a40b59b0 2024-04-23T14:43:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.467+0000 7f119b400640 1 -- 172.21.15.59:0/2439110476 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c014640 con 0x7f11a40b59b0 2024-04-23T14:43:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.467+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a40c9a90 con 0x7f11a40b59b0 2024-04-23T14:43:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.467+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a40ca050 con 0x7f11a40b59b0 2024-04-23T14:43:22.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.468+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a40b3bd0 con 0x7f11a40b59b0 2024-04-23T14:43:22.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.469+0000 7f119b400640 1 -- 172.21.15.59:0/2439110476 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f119c01c070 con 0x7f11a40b59b0 2024-04-23T14:43:22.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.470+0000 7f119b400640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f118006c920 0x7f118006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.470+0000 7f119b400640 1 -- 172.21.15.59:0/2439110476 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5029+0+0 (secure 0 0 0) 0x7f119c08b2a0 con 0x7f11a40b59b0 2024-04-23T14:43:22.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.470+0000 7f11a2a00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f118006c920 0x7f118006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.471+0000 7f11a2a00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f118006c920 0x7f118006ede0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f11940099d0 tx=0x7f1194008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:22.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.475+0000 7f119b400640 1 -- 172.21.15.59:0/2439110476 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11a40b3bd0 con 0x7f11a40b59b0 2024-04-23T14:43:22.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.680+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11a404fd90 con 0x7f11a40b59b0 2024-04-23T14:43:22.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.682+0000 7f119b400640 1 -- 172.21.15.59:0/2439110476 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+13090 (secure 0 0 0) 0x7f119c059960 con 0x7f11a40b59b0 2024-04-23T14:43:22.683 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:22.683 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":103,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:20.198566+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6835/2265807893":"2024-04-24T14:43:10.119032+0000","172.21.15.59:6837/620420969":"2024-04-24T14:42:32.618734+0000","172.21.15.59:6835/620420969":"2024-04-24T14:42:32.618734+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:22.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.684+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f118006c920 msgr2=0x7f118006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f118006c920 0x7f118006ede0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f11940099d0 tx=0x7f1194008040 comp rx=0 tx=0).stop 2024-04-23T14:43:22.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11a40b59b0 msgr2=0x7f11a40cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11a40b59b0 0x7f11a40cb5f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f11a40d0b30 tx=0x7f119c009cb0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 shutdown_connections 2024-04-23T14:43:22.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f118006c920 0x7f118006ede0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11a40b59b0 0x7f11a40cb5f0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11a40ca850 0x7f11a40b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 --2- 172.21.15.59:0/2439110476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1188003f30 0x7f11a41f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 >> 172.21.15.59:0/2439110476 conn(0x7f11a409d920 msgr2=0x7f11a408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:22.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 shutdown_connections 2024-04-23T14:43:22.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.685+0000 7f11a3e00640 1 -- 172.21.15.59:0/2439110476 wait complete. 2024-04-23T14:43:22.707 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:43:22.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:43:22.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.922+0000 7f911ce00640 1 Processor -- start 2024-04-23T14:43:22.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.922+0000 7f911ce00640 1 -- start start 2024-04-23T14:43:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.924+0000 7f911ce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f911815d300 0x7f9118071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.924+0000 7f911ce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9118163320 con 0x7f91180d0600 2024-04-23T14:43:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.924+0000 7f911ce00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9118072380 con 0x7f9118167180 2024-04-23T14:43:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.924+0000 7f911ce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9118163650 con 0x7f911815d300 2024-04-23T14:43:22.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.925+0000 7f9117e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f91180d0600 0x7f911806e1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36498/0 (socket says 172.21.15.59:36498) 2024-04-23T14:43:22.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.925+0000 7f9117e00640 1 -- 172.21.15.59:0/3954296286 learned_addr learned my addr 172.21.15.59:0/3954296286 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:22.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.925+0000 7f9117400640 1 --2- 172.21.15.59:0/3954296286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f911815d300 0x7f9118071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.925+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476838993 0 0) 0x7f9118163320 con 0x7f91180d0600 2024-04-23T14:43:22.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.925+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90fc003520 con 0x7f91180d0600 2024-04-23T14:43:22.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.926+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2854133588 0 0) 0x7f9118072380 con 0x7f9118167180 2024-04-23T14:43:22.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.926+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9118163320 con 0x7f9118167180 2024-04-23T14:43:22.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.926+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 813898024 0 0) 0x7f90fc003520 con 0x7f91180d0600 2024-04-23T14:43:22.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.926+0000 7f9117400640 1 -- 172.21.15.59:0/3954296286 >> v1:172.21.15.76:6789/0 conn(0x7f9118167180 legacy=0x7f9118163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.926+0000 7f9117400640 1 -- 172.21.15.59:0/3954296286 >> v1:172.21.15.59:6789/0 conn(0x7f91180d0600 legacy=0x7f911806e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.926+0000 7f9117400640 1 -- 172.21.15.59:0/3954296286 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9118072660 con 0x7f911815d300 2024-04-23T14:43:22.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.927+0000 7f9117400640 1 --2- 172.21.15.59:0/3954296286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f911815d300 0x7f9118071c70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9108009df0 tx=0x7f910802a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=115be0438851b178 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:22.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.928+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110001670 con 0x7f911815d300 2024-04-23T14:43:22.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.928+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9108007540 con 0x7f911815d300 2024-04-23T14:43:22.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.928+0000 7f9116a00640 1 -- 172.21.15.59:0/3954296286 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108007710 con 0x7f911815d300 2024-04-23T14:43:22.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.929+0000 7f911ce00640 1 -- 172.21.15.59:0/3954296286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f911815d300 msgr2=0x7f9118071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.929+0000 7f911ce00640 1 --2- 172.21.15.59:0/3954296286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f911815d300 0x7f9118071c70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9108009df0 tx=0x7f910802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.929+0000 7f911ce00640 1 -- 172.21.15.59:0/3954296286 shutdown_connections 2024-04-23T14:43:22.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.929+0000 7f911ce00640 1 --2- 172.21.15.59:0/3954296286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f911815d300 0x7f9118071c70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.929+0000 7f911ce00640 1 -- 172.21.15.59:0/3954296286 >> 172.21.15.59:0/3954296286 conn(0x7f911806bdf0 msgr2=0x7f911806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:22.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.929+0000 7f911ce00640 1 -- 172.21.15.59:0/3954296286 shutdown_connections 2024-04-23T14:43:22.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.929+0000 7f911ce00640 1 -- 172.21.15.59:0/3954296286 wait complete. 2024-04-23T14:43:22.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.931+0000 7f911ce00640 1 Processor -- start 2024-04-23T14:43:22.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.931+0000 7f911ce00640 1 -- start start 2024-04-23T14:43:22.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f911ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91180d0600 0x7f911814c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f911ce00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f911815d300 0x7f911809a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f911ce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9118167180 0x7f911809a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f911ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f911806e8f0 con 0x7f91180d0600 2024-04-23T14:43:22.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f911ce00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91181675e0 con 0x7f911815d300 2024-04-23T14:43:22.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f911ce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f911806ea70 con 0x7f9118167180 2024-04-23T14:43:22.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f9117400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91180d0600 0x7f911814c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f9117400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91180d0600 0x7f911814c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46910/0 (socket says 172.21.15.59:46910) 2024-04-23T14:43:22.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.932+0000 7f9117400640 1 -- 172.21.15.59:0/2084857210 learned_addr learned my addr 172.21.15.59:0/2084857210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:22.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f910ea00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f911815d300 0x7f911809a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117e00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9118167180 0x7f911809a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117400640 1 -- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9118167180 msgr2=0x7f911809a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117400640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9118167180 0x7f911809a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117400640 1 -- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f911815d300 msgr2=0x7f911809a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117400640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f911815d300 0x7f911809a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117400640 1 -- 172.21.15.59:0/2084857210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911809adf0 con 0x7f91180d0600 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117e00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9118167180 0x7f911809a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.933+0000 7f9117400640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91180d0600 0x7f911814c6b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9108009df0 tx=0x7f9108004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.934+0000 7f910fe00640 1 -- 172.21.15.59:0/2084857210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108040070 con 0x7f91180d0600 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.934+0000 7f910fe00640 1 -- 172.21.15.59:0/2084857210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910803c830 con 0x7f91180d0600 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.934+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911809b0d0 con 0x7f91180d0600 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.934+0000 7f910fe00640 1 -- 172.21.15.59:0/2084857210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910803cc00 con 0x7f91180d0600 2024-04-23T14:43:22.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.934+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91180b2750 con 0x7f91180d0600 2024-04-23T14:43:22.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.935+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911809b2a0 con 0x7f91180d0600 2024-04-23T14:43:22.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.936+0000 7f910fe00640 1 -- 172.21.15.59:0/2084857210 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9108039040 con 0x7f91180d0600 2024-04-23T14:43:22.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.937+0000 7f910fe00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90f406c8f0 0x7f90f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:22.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.938+0000 7f910fe00640 1 -- 172.21.15.59:0/2084857210 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5029+0+0 (secure 0 0 0) 0x7f91080acea0 con 0x7f91180d0600 2024-04-23T14:43:22.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.938+0000 7f910ea00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90f406c8f0 0x7f90f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:22.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.939+0000 7f910ea00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90f406c8f0 0x7f90f406edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f91181638e0 tx=0x7f9100009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:22.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:22.942+0000 7f910fe00640 1 -- 172.21.15.59:0/2084857210 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f910807b560 con 0x7f91180d0600 2024-04-23T14:43:23.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.150+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f911804fd90 con 0x7f91180d0600 2024-04-23T14:43:23.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.151+0000 7f910fe00640 1 -- 172.21.15.59:0/2084857210 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f910807ae60 con 0x7f91180d0600 2024-04-23T14:43:23.152 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.153+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90f406c8f0 msgr2=0x7f90f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.153+0000 7f911ce00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90f406c8f0 0x7f90f406edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f91181638e0 tx=0x7f9100009040 comp rx=0 tx=0).stop 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.153+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91180d0600 msgr2=0x7f911814c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.153+0000 7f911ce00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91180d0600 0x7f911814c6b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9108009df0 tx=0x7f9108004a40 comp rx=0 tx=0).stop 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 shutdown_connections 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f90f406c8f0 0x7f90f406edb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9118167180 0x7f911809a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f911815d300 0x7f911809a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 --2- 172.21.15.59:0/2084857210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91180d0600 0x7f911814c6b0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 >> 172.21.15.59:0/2084857210 conn(0x7f911806bdf0 msgr2=0x7f9118163ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:23.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 shutdown_connections 2024-04-23T14:43:23.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.154+0000 7f911ce00640 1 -- 172.21.15.59:0/2084857210 wait complete. 2024-04-23T14:43:23.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-23T14:43:23.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.387+0000 7f4164a00640 1 Processor -- start 2024-04-23T14:43:23.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.387+0000 7f4164a00640 1 -- start start 2024-04-23T14:43:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f4164a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4160071de0 0x7f41600721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f4164a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41600cd8e0 con 0x7f41600ce380 2024-04-23T14:43:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f4164a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4160072700 con 0x7f4160072e30 2024-04-23T14:43:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f4164a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4160070180 con 0x7f4160071de0 2024-04-23T14:43:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f415e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f41600ce380 0x7f41600ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36506/0 (socket says 172.21.15.59:36506) 2024-04-23T14:43:23.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f415e800640 1 -- 172.21.15.59:0/4251440107 learned_addr learned my addr 172.21.15.59:0/4251440107 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:23.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f415de00640 1 --2- 172.21.15.59:0/4251440107 >> v2:172.21.15.76:3300/0 conn(0x7f4160071de0 0x7f41600721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:23.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.389+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861174002 0 0) 0x7f41600cd8e0 con 0x7f41600ce380 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4144003520 con 0x7f41600ce380 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3590760697 0 0) 0x7f4144003520 con 0x7f41600ce380 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 >> v2:172.21.15.76:3300/0 conn(0x7f4160071de0 msgr2=0x7f41600721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 --2- 172.21.15.59:0/4251440107 >> v2:172.21.15.76:3300/0 conn(0x7f4160071de0 0x7f41600721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 >> v1:172.21.15.76:6789/0 conn(0x7f4160072e30 legacy=0x7f416006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416006fbc0 con 0x7f41600ce380 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f41580019f0 con 0x7f41600ce380 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 >> v1:172.21.15.59:6789/0 conn(0x7f41600ce380 legacy=0x7f41600ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41440039b0 0x7f4144003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41440051c0 0x7f414400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4160070180 con 0x7f4144004800 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f414400c0f0 con 0x7f41440039b0 2024-04-23T14:43:23.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f414400c2f0 con 0x7f41440051c0 2024-04-23T14:43:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.390+0000 7f4155400640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.391+0000 7f4155400640 1 -- 172.21.15.59:0/4251440107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41440051c0 msgr2=0x7f414400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.391+0000 7f4155400640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41440051c0 0x7f414400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.391+0000 7f4155400640 1 -- 172.21.15.59:0/4251440107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41440039b0 msgr2=0x7f4144003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.391+0000 7f4155400640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41440039b0 0x7f4144003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.391+0000 7f4155400640 1 -- 172.21.15.59:0/4251440107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4144003520 con 0x7f4144004800 2024-04-23T14:43:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.391+0000 7f415e800640 1 -- 172.21.15.59:0/4251440107 reap_dead start 2024-04-23T14:43:23.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.392+0000 7f4155400640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4144004c80 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f4148003d80 tx=0x7f4144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e0f51110703aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:23.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.392+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148037070 con 0x7f4144004800 2024-04-23T14:43:23.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.392+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414802a990 con 0x7f4144004800 2024-04-23T14:43:23.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.392+0000 7f415d400640 1 -- 172.21.15.59:0/4251440107 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148003980 con 0x7f4144004800 2024-04-23T14:43:23.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.393+0000 7f4164a00640 1 -- 172.21.15.59:0/4251440107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 msgr2=0x7f4144004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:23.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.393+0000 7f4164a00640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4144004c80 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f4148003d80 tx=0x7f4144005830 comp rx=0 tx=0).stop 2024-04-23T14:43:23.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.393+0000 7f4164a00640 1 -- 172.21.15.59:0/4251440107 shutdown_connections 2024-04-23T14:43:23.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.393+0000 7f4164a00640 1 --2- 172.21.15.59:0/4251440107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4144004c80 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.393+0000 7f4164a00640 1 -- 172.21.15.59:0/4251440107 >> 172.21.15.59:0/4251440107 conn(0x7f4160067ea0 msgr2=0x7f416006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:23.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.393+0000 7f4164a00640 1 -- 172.21.15.59:0/4251440107 shutdown_connections 2024-04-23T14:43:23.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.393+0000 7f4164a00640 1 -- 172.21.15.59:0/4251440107 wait complete. 2024-04-23T14:43:23.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.395+0000 7f4164a00640 1 Processor -- start 2024-04-23T14:43:23.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.395+0000 7f4164a00640 1 -- start start 2024-04-23T14:43:23.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f4164a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4160155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f4164a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41601ed890 0x7f41601f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f4164a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41601f4040 0x7f41601f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f4164a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41600cde60 con 0x7f4144004800 2024-04-23T14:43:23.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f4164a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f41600cdb80 con 0x7f41601f4040 2024-04-23T14:43:23.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f4164a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f416015e4e0 con 0x7f41601ed890 2024-04-23T14:43:23.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f415de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4160155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:23.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f415de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4160155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46938/0 (socket says 172.21.15.59:46938) 2024-04-23T14:43:23.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.396+0000 7f415de00640 1 -- 172.21.15.59:0/3335867348 learned_addr learned my addr 172.21.15.59:0/3335867348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:23.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415de00640 1 -- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41601ed890 msgr2=0x7f41601f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:23.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415e800640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41601f4040 0x7f41601f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415de00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41601ed890 0x7f41601f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415de00640 1 -- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41601f4040 msgr2=0x7f41601f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415de00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41601f4040 0x7f41601f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415de00640 1 -- 172.21.15.59:0/3335867348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4160153800 con 0x7f4144004800 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415e800640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41601f4040 0x7f41601f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f415de00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4160155400 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f415000d800 tx=0x7f415000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f4156a00640 1 -- 172.21.15.59:0/3335867348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415000b6c0 con 0x7f4144004800 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f4156a00640 1 -- 172.21.15.59:0/3335867348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4150019070 con 0x7f4144004800 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.397+0000 7f4156a00640 1 -- 172.21.15.59:0/3335867348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415000baa0 con 0x7f4144004800 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.398+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4160153ae0 con 0x7f4144004800 2024-04-23T14:43:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.398+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41601540a0 con 0x7f4144004800 2024-04-23T14:43:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.399+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416015d4f0 con 0x7f4144004800 2024-04-23T14:43:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.400+0000 7f4156a00640 1 -- 172.21.15.59:0/3335867348 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f415000bc40 con 0x7f4144004800 2024-04-23T14:43:23.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.401+0000 7f4156a00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f413c06c920 0x7f413c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:23.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.401+0000 7f4156a00640 1 -- 172.21.15.59:0/3335867348 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5029+0+0 (secure 0 0 0) 0x7f415008bd60 con 0x7f4144004800 2024-04-23T14:43:23.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.401+0000 7f4155400640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f413c06c920 0x7f413c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:23.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.402+0000 7f4155400640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f413c06c920 0x7f413c06ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4148034000 tx=0x7f414802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:23.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.405+0000 7f4156a00640 1 -- 172.21.15.59:0/3335867348 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f416015d4f0 con 0x7f4144004800 2024-04-23T14:43:23.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:23.682+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f4160068a80 con 0x7f4144004800 2024-04-23T14:43:24.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.240+0000 7f4156a00640 1 -- 172.21.15.59:0/3335867348 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v557) ==== 147+0+0 (secure 0 0 0) 0x7f415005a420 con 0x7f4144004800 2024-04-23T14:43:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.243+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f413c06c920 msgr2=0x7f413c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.243+0000 7f4164a00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f413c06c920 0x7f413c06ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4148034000 tx=0x7f414802c000 comp rx=0 tx=0).stop 2024-04-23T14:43:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.243+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 msgr2=0x7f4160155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.243+0000 7f4164a00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4160155400 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f415000d800 tx=0x7f415000dcc0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 shutdown_connections 2024-04-23T14:43:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f413c06c920 0x7f413c06ede0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41601ed890 0x7f41601f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41601f4040 0x7f41601f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 --2- 172.21.15.59:0/3335867348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4144004800 0x7f4160155400 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 >> 172.21.15.59:0/3335867348 conn(0x7f4160067ea0 msgr2=0x7f416015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 shutdown_connections 2024-04-23T14:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.244+0000 7f4164a00640 1 -- 172.21.15.59:0/3335867348 wait complete. 2024-04-23T14:43:24.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-23T14:43:24.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.487+0000 7f74ffe00640 1 Processor -- start 2024-04-23T14:43:24.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.487+0000 7f74ffe00640 1 -- start start 2024-04-23T14:43:24.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.488+0000 7f74ffe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f75000d15b0 0x7f75000cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.488+0000 7f74ffe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75000b5da0 con 0x7f75000b4f20 2024-04-23T14:43:24.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.488+0000 7f74ffe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75000d0a20 con 0x7f750015ed20 2024-04-23T14:43:24.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.488+0000 7f74ffe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f750015f100 con 0x7f75000d15b0 2024-04-23T14:43:24.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.489+0000 7f74ff400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f75000b4f20 0x7f75000cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36510/0 (socket says 172.21.15.59:36510) 2024-04-23T14:43:24.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.489+0000 7f74ff400640 1 -- 172.21.15.59:0/619891014 learned_addr learned my addr 172.21.15.59:0/619891014 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:24.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.489+0000 7f74fe000640 1 --2- 172.21.15.59:0/619891014 >> v2:172.21.15.76:3300/0 conn(0x7f75000d15b0 0x7f75000cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.489+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883004599 0 0) 0x7f75000b5da0 con 0x7f75000b4f20 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.489+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74e4003520 con 0x7f75000b4f20 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2700441031 0 0) 0x7f74e4003520 con 0x7f75000b4f20 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 >> v2:172.21.15.76:3300/0 conn(0x7f75000d15b0 msgr2=0x7f75000cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 --2- 172.21.15.59:0/619891014 >> v2:172.21.15.76:3300/0 conn(0x7f75000d15b0 0x7f75000cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 >> v1:172.21.15.76:6789/0 conn(0x7f750015ed20 legacy=0x7f75000d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75000d2ed0 con 0x7f75000b4f20 2024-04-23T14:43:24.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f74f8001670 con 0x7f75000b4f20 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 >> v1:172.21.15.59:6789/0 conn(0x7f75000b4f20 legacy=0x7f75000cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74e40039b0 0x7f74e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e4004800 0x7f74e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f74e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74e400c180 con 0x7f74e40052e0 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74e400c330 con 0x7f74e4004800 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f74e400c530 con 0x7f74e40039b0 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74ff400640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f74e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fe000640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e4004800 0x7f74e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.490+0000 7f74fea00640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74e40039b0 0x7f74e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.491+0000 7f74ff400640 1 -- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74e40039b0 msgr2=0x7f74e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.491+0000 7f74ff400640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74e40039b0 0x7f74e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.491+0000 7f74ff400640 1 -- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e4004800 msgr2=0x7f74e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.491+0000 7f74ff400640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e4004800 0x7f74e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.491+0000 7f74ff400640 1 -- 172.21.15.59:0/619891014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e4003520 con 0x7f74e40052e0 2024-04-23T14:43:24.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.491+0000 7f74fea00640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f74e40039b0 0x7f74e4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.491+0000 7f74ff400640 1 -- 172.21.15.59:0/619891014 reap_dead start 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.492+0000 7f74ff400640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f74e400b9c0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f74f8001eb0 tx=0x7f74e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8888b647eccde068 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.492+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8007d30 con 0x7f74e40052e0 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.492+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f80058d0 con 0x7f74e40052e0 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.492+0000 7f74fd600640 1 -- 172.21.15.59:0/619891014 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8005ca0 con 0x7f74e40052e0 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.492+0000 7f74ffe00640 1 -- 172.21.15.59:0/619891014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 msgr2=0x7f74e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.492+0000 7f74ffe00640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f74e400b9c0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f74f8001eb0 tx=0x7f74e4005830 comp rx=0 tx=0).stop 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.493+0000 7f74ffe00640 1 -- 172.21.15.59:0/619891014 shutdown_connections 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.493+0000 7f74ffe00640 1 --2- 172.21.15.59:0/619891014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f74e400b9c0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.493+0000 7f74ffe00640 1 -- 172.21.15.59:0/619891014 >> 172.21.15.59:0/619891014 conn(0x7f750008bf90 msgr2=0x7f75000d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.493+0000 7f74ffe00640 1 -- 172.21.15.59:0/619891014 shutdown_connections 2024-04-23T14:43:24.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.493+0000 7f74ffe00640 1 -- 172.21.15.59:0/619891014 wait complete. 2024-04-23T14:43:24.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.495+0000 7f74ffe00640 1 Processor -- start 2024-04-23T14:43:24.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.495+0000 7f74ffe00640 1 -- start start 2024-04-23T14:43:24.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f75000cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74ffe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75000c8c10 0x7f750014f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74ffe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500150340 0x7f75001507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74ffe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75000b3600 con 0x7f74e40052e0 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74ffe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f75000d3490 con 0x7f7500150340 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74ffe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f75000b3780 con 0x7f75000c8c10 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f75000cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f75000cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46956/0 (socket says 172.21.15.59:46956) 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 -- 172.21.15.59:0/3395706892 learned_addr learned my addr 172.21.15.59:0/3395706892 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fe000640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75000c8c10 0x7f750014f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 -- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75000c8c10 msgr2=0x7f750014f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74ff400640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500150340 0x7f75001507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75000c8c10 0x7f750014f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 -- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500150340 msgr2=0x7f75001507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500150340 0x7f75001507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.496+0000 7f74fea00640 1 -- 172.21.15.59:0/3395706892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75000cbc70 con 0x7f74e40052e0 2024-04-23T14:43:24.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.497+0000 7f74fea00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f75000cb690 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f74f000db10 tx=0x7f74f0002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:24.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.497+0000 7f74f7400640 1 -- 172.21.15.59:0/3395706892 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0003da0 con 0x7f74e40052e0 2024-04-23T14:43:24.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.497+0000 7f74f7400640 1 -- 172.21.15.59:0/3395706892 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f0017070 con 0x7f74e40052e0 2024-04-23T14:43:24.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.497+0000 7f74f7400640 1 -- 172.21.15.59:0/3395706892 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f000b980 con 0x7f74e40052e0 2024-04-23T14:43:24.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.497+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75000cbf50 con 0x7f74e40052e0 2024-04-23T14:43:24.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.497+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75000cc510 con 0x7f74e40052e0 2024-04-23T14:43:24.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.498+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750004fd90 con 0x7f74e40052e0 2024-04-23T14:43:24.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.499+0000 7f74f7400640 1 -- 172.21.15.59:0/3395706892 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74f000bb20 con 0x7f74e40052e0 2024-04-23T14:43:24.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.501+0000 7f74f7400640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74dc06c920 0x7f74dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:24.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.501+0000 7f74f7400640 1 -- 172.21.15.59:0/3395706892 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5029+0+0 (secure 0 0 0) 0x7f74f008ba40 con 0x7f74e40052e0 2024-04-23T14:43:24.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.501+0000 7f74fe000640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74dc06c920 0x7f74dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:24.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.502+0000 7f74fe000640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74dc06c920 0x7f74dc06ede0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f75000b5910 tx=0x7f74e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:24.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.505+0000 7f74f7400640 1 -- 172.21.15.59:0/3395706892 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74f005a100 con 0x7f74e40052e0 2024-04-23T14:43:24.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.710+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f75000c7a40 con 0x7f74e40052e0 2024-04-23T14:43:24.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.715+0000 7f74f7400640 1 -- 172.21.15.59:0/3395706892 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f74f0059a00 con 0x7f74e40052e0 2024-04-23T14:43:24.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74dc06c920 msgr2=0x7f74dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74dc06c920 0x7f74dc06ede0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f75000b5910 tx=0x7f74e8008040 comp rx=0 tx=0).stop 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 msgr2=0x7f75000cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f75000cb690 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f74f000db10 tx=0x7f74f0002070 comp rx=0 tx=0).stop 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 shutdown_connections 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f74dc06c920 0x7f74dc06ede0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75000c8c10 0x7f750014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7500150340 0x7f75001507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 --2- 172.21.15.59:0/3395706892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74e40052e0 0x7f75000cb690 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 >> 172.21.15.59:0/3395706892 conn(0x7f750008bf90 msgr2=0x7f750006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 shutdown_connections 2024-04-23T14:43:24.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:24.718+0000 7f74ffe00640 1 -- 172.21.15.59:0/3395706892 wait complete. 2024-04-23T14:43:24.739 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:43:24.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:43:24.847 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:43:24.976 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:43:24.976 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:43:24.976 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:25.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:43:25.036 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:43:25.037 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:25.091 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:43:25.092 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:43:25.092 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:25.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:43:25.152 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:43:25.152 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:25.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:43:25.209 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-23T14:43:25.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:43:25.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.427+0000 7fccdfe00640 1 Processor -- start 2024-04-23T14:43:25.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.428+0000 7fccdfe00640 1 -- start start 2024-04-23T14:43:25.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.429+0000 7fccdfe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcce00d0ec0 0x7fcce00d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:25.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.429+0000 7fccdfe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcce00d28b0 0x7fcce015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:25.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.429+0000 7fccdfe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce00d17e0 con 0x7fcce00cff10 2024-04-23T14:43:25.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.429+0000 7fccdfe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fcce00b3b60 con 0x7fcce00d0ec0 2024-04-23T14:43:25.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.429+0000 7fccdfe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcce015fd00 con 0x7fcce00d28b0 2024-04-23T14:43:25.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccdea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcce00d0ec0 0x7fcce00d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:25.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccde000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcce00cff10 0x7fcce00d21a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:36514/0 (socket says 172.21.15.59:36514) 2024-04-23T14:43:25.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccdea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcce00d0ec0 0x7fcce00d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46968/0 (socket says 172.21.15.59:46968) 2024-04-23T14:43:25.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccde000640 1 -- 172.21.15.59:0/457328094 learned_addr learned my addr 172.21.15.59:0/457328094 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:25.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccdf400640 1 --2- 172.21.15.59:0/457328094 >> v2:172.21.15.76:3300/0 conn(0x7fcce00d28b0 0x7fcce015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:25.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccdea00640 1 -- 172.21.15.59:0/457328094 >> v2:172.21.15.76:3300/0 conn(0x7fcce00d28b0 msgr2=0x7fcce015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:25.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccdea00640 1 --2- 172.21.15.59:0/457328094 >> v2:172.21.15.76:3300/0 conn(0x7fcce00d28b0 0x7fcce015eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccdea00640 1 -- 172.21.15.59:0/457328094 >> v1:172.21.15.59:6789/0 conn(0x7fcce00cff10 legacy=0x7fcce00d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:25.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.430+0000 7fccdea00640 1 -- 172.21.15.59:0/457328094 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce015fa20 con 0x7fcce00d0ec0 2024-04-23T14:43:25.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.431+0000 7fccdea00640 1 --2- 172.21.15.59:0/457328094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d0ec0 0x7fcce00d12a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcce0162280 tx=0x7fccd002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982f01db8b8c4398 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:25.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdd600640 1 -- 172.21.15.59:0/457328094 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0037070 con 0x7fcce00d0ec0 2024-04-23T14:43:25.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdd600640 1 -- 172.21.15.59:0/457328094 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd002adc0 con 0x7fcce00d0ec0 2024-04-23T14:43:25.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdd600640 1 -- 172.21.15.59:0/457328094 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0032660 con 0x7fcce00d0ec0 2024-04-23T14:43:25.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdfe00640 1 -- 172.21.15.59:0/457328094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d0ec0 msgr2=0x7fcce00d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:25.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdfe00640 1 --2- 172.21.15.59:0/457328094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d0ec0 0x7fcce00d12a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcce0162280 tx=0x7fccd002a130 comp rx=0 tx=0).stop 2024-04-23T14:43:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdfe00640 1 -- 172.21.15.59:0/457328094 shutdown_connections 2024-04-23T14:43:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdfe00640 1 --2- 172.21.15.59:0/457328094 >> v2:172.21.15.76:3300/0 conn(0x7fcce00d28b0 0x7fcce015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdfe00640 1 --2- 172.21.15.59:0/457328094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d0ec0 0x7fcce00d12a0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.432+0000 7fccdfe00640 1 -- 172.21.15.59:0/457328094 >> 172.21.15.59:0/457328094 conn(0x7fcce008bf90 msgr2=0x7fcce00d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.433+0000 7fccdfe00640 1 -- 172.21.15.59:0/457328094 shutdown_connections 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.433+0000 7fccdfe00640 1 -- 172.21.15.59:0/457328094 wait complete. 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.434+0000 7fccdfe00640 1 Processor -- start 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.434+0000 7fccdfe00640 1 -- start start 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.435+0000 7fccdfe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcce00cff10 0x7fcce015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.435+0000 7fccdfe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcce00d0ec0 0x7fcce00c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.435+0000 7fccdfe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d28b0 0x7fcce00cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.435+0000 7fccdfe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00cf9f0 con 0x7fcce00d28b0 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.435+0000 7fccdfe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00cf710 con 0x7fcce00cff10 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.435+0000 7fccdfe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fcce00d0330 con 0x7fcce00d0ec0 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d28b0 0x7fcce00cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:25.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d28b0 0x7fcce00cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46976/0 (socket says 172.21.15.59:46976) 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 -- 172.21.15.59:0/1034293868 learned_addr learned my addr 172.21.15.59:0/1034293868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdea00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcce00cff10 0x7fcce015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccde000640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcce00d0ec0 0x7fcce00c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 -- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcce00d0ec0 msgr2=0x7fcce00c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcce00d0ec0 0x7fcce00c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 -- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcce00cff10 msgr2=0x7fcce015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcce00cff10 0x7fcce015add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.436+0000 7fccdf400640 1 -- 172.21.15.59:0/1034293868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce00c9b30 con 0x7fcce00d28b0 2024-04-23T14:43:25.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.437+0000 7fccdf400640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d28b0 0x7fcce00cd360 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fccd8007e60 tx=0x7fccd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.437+0000 7fccd6a00640 1 -- 172.21.15.59:0/1034293868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8014420 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.437+0000 7fccd6a00640 1 -- 172.21.15.59:0/1034293868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd8019070 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.437+0000 7fccd6a00640 1 -- 172.21.15.59:0/1034293868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8014820 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.437+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce00c9e70 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.437+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce01e5aa0 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.438+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce004fd90 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.439+0000 7fccd6a00640 1 -- 172.21.15.59:0/1034293868 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccd80149c0 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.440+0000 7fccd6a00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fccbc06c920 0x7fccbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.440+0000 7fccd6a00640 1 -- 172.21.15.59:0/1034293868 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5029+0+0 (secure 0 0 0) 0x7fccd808c180 con 0x7fcce00d28b0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.441+0000 7fccdea00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fccbc06c920 0x7fccbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:25.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.441+0000 7fccdea00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fccbc06c920 0x7fccbc06ede0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fccd0034000 tx=0x7fccd002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:25.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.445+0000 7fccd6a00640 1 -- 172.21.15.59:0/1034293868 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcce00c9e70 con 0x7fcce00d28b0 2024-04-23T14:43:25.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.689+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcce00d1f00 con 0x7fcce00d28b0 2024-04-23T14:43:25.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.690+0000 7fccd6a00640 1 -- 172.21.15.59:0/1034293868 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+3869 (secure 0 0 0) 0x7fccd805a7b0 con 0x7fcce00d28b0 2024-04-23T14:43:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:25.692 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/1755322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1755322213},{"type":"v1","addr":"172.21.15.76:6834","nonce":1755322213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1618378997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":1618378997},{"type":"v1","addr":"172.21.15.76:6835","nonce":1618378997}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6836/2360192331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2360192331},{"type":"v1","addr":"172.21.15.59:6836","nonce":2360192331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9060,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/3956898282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6835","nonce":3956898282},{"type":"v1","addr":"172.21.15.59:6837","nonce":3956898282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-04-23T14:43:25.692 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 92 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fccbc06c920 msgr2=0x7fccbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fccbc06c920 0x7fccbc06ede0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fccd0034000 tx=0x7fccd002e000 comp rx=0 tx=0).stop 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d28b0 msgr2=0x7fcce00cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d28b0 0x7fcce00cd360 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fccd8007e60 tx=0x7fccd800e4b0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 shutdown_connections 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fccbc06c920 0x7fccbc06ede0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcce00d0ec0 0x7fcce00c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce00d28b0 0x7fcce00cd360 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 --2- 172.21.15.59:0/1034293868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcce00cff10 0x7fcce015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 >> 172.21.15.59:0/1034293868 conn(0x7fcce008bf90 msgr2=0x7fcce00b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:25.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 shutdown_connections 2024-04-23T14:43:25.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:25.693+0000 7fccdfe00640 1 -- 172.21.15.59:0/1034293868 wait complete. 2024-04-23T14:43:25.714 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-23T14:43:25.715 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-23T14:43:25.715 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:43:25.715 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:43:25.715 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-23T14:43:25.715 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-23T14:43:25.715 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:43:25.715 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:43:25.716 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-23T14:43:25.716 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-23T14:43:25.716 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:43:25.716 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:43:25.716 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-23T14:43:25.716 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-23T14:43:25.716 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:43:25.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:43:25.739 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:43:25.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-23T14:43:25.741 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:43:25.741 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-23T14:43:25.747 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:43:25.747 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-23T14:43:25.787 INFO:tasks.ceph.mds.a:Started 2024-04-23T14:43:25.788 DEBUG:teuthology.parallel:result is None 2024-04-23T14:43:25.793 INFO:tasks.ceph.mds.d:Started 2024-04-23T14:43:25.794 DEBUG:teuthology.parallel:result is None 2024-04-23T14:43:25.800 INFO:tasks.ceph.mds.b:Started 2024-04-23T14:43:25.801 DEBUG:teuthology.parallel:result is None 2024-04-23T14:43:25.946 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2024-04-23T14:43:25.949 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2024-04-23T14:43:25.968 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2024-04-23T14:43:30.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-23T14:43:31.739 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:43:31.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-23T14:43:31.748 INFO:tasks.ceph.mds.c:Started 2024-04-23T14:43:31.748 DEBUG:teuthology.parallel:result is None 2024-04-23T14:43:31.748 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-23T14:43:31.904 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:43:31.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.934+0000 7f650dc00640 1 Processor -- start 2024-04-23T14:43:31.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.934+0000 7f650dc00640 1 -- start start 2024-04-23T14:43:31.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f650dc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65080cf410 0x7f650815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:31.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f650dc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f650806ddf0 0x7f650806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:31.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f650dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6508072680 0x7f6508072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:31.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f650dc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f650815dda0 con 0x7f65080cf410 2024-04-23T14:43:31.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f650dc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f65080cd880 con 0x7f650806ddf0 2024-04-23T14:43:31.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f650dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6508071ec0 con 0x7f6508072680 2024-04-23T14:43:31.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f6506a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65080cf410 0x7f650815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:31.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f6506a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65080cf410 0x7f650815d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33762/0 (socket says 172.21.15.59:33762) 2024-04-23T14:43:31.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.935+0000 7f6506a00640 1 -- 172.21.15.59:0/2382504497 learned_addr learned my addr 172.21.15.59:0/2382504497 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f6506a00640 1 -- 172.21.15.59:0/2382504497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6508072680 msgr2=0x7f6508072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f64ffe00640 1 --2- 172.21.15.59:0/2382504497 >> v2:172.21.15.76:3300/0 conn(0x7f650806ddf0 0x7f650806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f6507400640 1 --2- 172.21.15.59:0/2382504497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6508072680 0x7f6508072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f6506a00640 1 --2- 172.21.15.59:0/2382504497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6508072680 0x7f6508072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f6506a00640 1 -- 172.21.15.59:0/2382504497 >> v2:172.21.15.76:3300/0 conn(0x7f650806ddf0 msgr2=0x7f650806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f6506a00640 1 --2- 172.21.15.59:0/2382504497 >> v2:172.21.15.76:3300/0 conn(0x7f650806ddf0 0x7f650806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f6506a00640 1 -- 172.21.15.59:0/2382504497 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65080cdb20 con 0x7f65080cf410 2024-04-23T14:43:31.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f64ffe00640 1 --2- 172.21.15.59:0/2382504497 >> v2:172.21.15.76:3300/0 conn(0x7f650806ddf0 0x7f650806e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:31.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.936+0000 7f6507400640 1 --2- 172.21.15.59:0/2382504497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6508072680 0x7f6508072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:31.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.937+0000 7f6506a00640 1 --2- 172.21.15.59:0/2382504497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65080cf410 0x7f650815d860 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f65080735c0 tx=0x7f64f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a934ce85eef4ff7d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.937+0000 7f6506000640 1 -- 172.21.15.59:0/2382504497 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8037070 con 0x7f65080cf410 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.937+0000 7f6506000640 1 -- 172.21.15.59:0/2382504497 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f8032440 con 0x7f65080cf410 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.937+0000 7f6506000640 1 -- 172.21.15.59:0/2382504497 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8032810 con 0x7f65080cf410 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 -- 172.21.15.59:0/2382504497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65080cf410 msgr2=0x7f650815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 --2- 172.21.15.59:0/2382504497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65080cf410 0x7f650815d860 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f65080735c0 tx=0x7f64f802a140 comp rx=0 tx=0).stop 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 -- 172.21.15.59:0/2382504497 shutdown_connections 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 --2- 172.21.15.59:0/2382504497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6508072680 0x7f6508072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 --2- 172.21.15.59:0/2382504497 >> v2:172.21.15.76:3300/0 conn(0x7f650806ddf0 0x7f650806e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 --2- 172.21.15.59:0/2382504497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65080cf410 0x7f650815d860 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 -- 172.21.15.59:0/2382504497 >> 172.21.15.59:0/2382504497 conn(0x7f65080b2130 msgr2=0x7f65080b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 -- 172.21.15.59:0/2382504497 shutdown_connections 2024-04-23T14:43:31.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.938+0000 7f650dc00640 1 -- 172.21.15.59:0/2382504497 wait complete. 2024-04-23T14:43:31.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.940+0000 7f650dc00640 1 Processor -- start 2024-04-23T14:43:31.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.940+0000 7f650dc00640 1 -- start start 2024-04-23T14:43:31.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f650dc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f650806ddf0 0x7f6508145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:31.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f650dc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508072680 0x7f65081e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:31.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f650dc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65080cf410 0x7f65081570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f650dc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f650815a6a0 con 0x7f6508072680 2024-04-23T14:43:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f650dc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65080cde00 con 0x7f650806ddf0 2024-04-23T14:43:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f650dc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f650815a820 con 0x7f65080cf410 2024-04-23T14:43:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508072680 0x7f65081e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508072680 0x7f65081e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33778/0 (socket says 172.21.15.59:33778) 2024-04-23T14:43:31.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.941+0000 7f64ffe00640 1 -- 172.21.15.59:0/3860237836 learned_addr learned my addr 172.21.15.59:0/3860237836 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:31.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f6507400640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65080cf410 0x7f65081570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:31.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f6506a00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f650806ddf0 0x7f6508145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:31.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f64ffe00640 1 -- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65080cf410 msgr2=0x7f65081570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:31.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f64ffe00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65080cf410 0x7f65081570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:31.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f64ffe00640 1 -- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f650806ddf0 msgr2=0x7f6508145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:31.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f64ffe00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f650806ddf0 0x7f6508145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f64ffe00640 1 -- 172.21.15.59:0/3860237836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6508146a50 con 0x7f6508072680 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f6506a00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f650806ddf0 0x7f6508145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f64ffe00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508072680 0x7f65081e4f70 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c7f0 tx=0x7f64f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.942+0000 7f64fea00640 1 -- 172.21.15.59:0/3860237836 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0015070 con 0x7f6508072680 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.943+0000 7f64fea00640 1 -- 172.21.15.59:0/3860237836 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0007820 con 0x7f6508072680 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.943+0000 7f64fea00640 1 -- 172.21.15.59:0/3860237836 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007c10 con 0x7f6508072680 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.943+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6508146d30 con 0x7f6508072680 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.943+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65081472f0 con 0x7f6508072680 2024-04-23T14:43:31.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.944+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6508068a80 con 0x7f6508072680 2024-04-23T14:43:31.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.945+0000 7f64fea00640 1 -- 172.21.15.59:0/3860237836 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64f0018070 con 0x7f6508072680 2024-04-23T14:43:31.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.946+0000 7f64fea00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f64e406c940 0x7f64e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:31.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.946+0000 7f64fea00640 1 -- 172.21.15.59:0/3860237836 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5029+0+0 (secure 0 0 0) 0x7f64f008bae0 con 0x7f6508072680 2024-04-23T14:43:31.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.946+0000 7f6506a00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f64e406c940 0x7f64e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:31.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.947+0000 7f6506a00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f64e406c940 0x7f64e406ee00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034000 tx=0x7f64f802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:31.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:31.951+0000 7f64fea00640 1 -- 172.21.15.59:0/3860237836 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64f005a080 con 0x7f6508072680 2024-04-23T14:43:32.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:32.166+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65080700b0 con 0x7f6508072680 2024-04-23T14:43:33.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.292+0000 7f64fea00640 1 -- 172.21.15.59:0/3860237836 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) ==== 97+0+0 (secure 0 0 0) 0x7f64f0059980 con 0x7f6508072680 2024-04-23T14:43:33.294 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2024-04-23T14:43:33.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f64e406c940 msgr2=0x7f64e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f64e406c940 0x7f64e406ee00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034000 tx=0x7f64f802e000 comp rx=0 tx=0).stop 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508072680 msgr2=0x7f65081e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508072680 0x7f65081e4f70 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f64f000c7f0 tx=0x7f64f000ccb0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 shutdown_connections 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f64e406c940 0x7f64e406ee00 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65080cf410 0x7f65081570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508072680 0x7f65081e4f70 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 --2- 172.21.15.59:0/3860237836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f650806ddf0 0x7f6508145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.296+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 >> 172.21.15.59:0/3860237836 conn(0x7f65080b2130 msgr2=0x7f650815e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:33.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.297+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 shutdown_connections 2024-04-23T14:43:33.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.297+0000 7f650dc00640 1 -- 172.21.15.59:0/3860237836 wait complete. 2024-04-23T14:43:33.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-23T14:43:33.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.530+0000 7f3621000640 1 Processor -- start 2024-04-23T14:43:33.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.530+0000 7f3621000640 1 -- start start 2024-04-23T14:43:33.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.532+0000 7f3621000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c160910 0x7f361c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.532+0000 7f3621000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361c0cd520 con 0x7f361c0cd020 2024-04-23T14:43:33.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.532+0000 7f3621000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361c0cd800 con 0x7f361c1454e0 2024-04-23T14:43:33.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.532+0000 7f3621000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f361c0cddc0 con 0x7f361c160910 2024-04-23T14:43:33.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.532+0000 7f3611400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f361c0cd020 0x7f361c0b3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47666/0 (socket says 172.21.15.59:47666) 2024-04-23T14:43:33.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.532+0000 7f3611400640 1 -- 172.21.15.59:0/3775446901 learned_addr learned my addr 172.21.15.59:0/3775446901 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f361a800640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c160910 0x7f361c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2048498139 0 0) 0x7f361c0cd520 con 0x7f361c0cd020 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3600003520 con 0x7f361c0cd020 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 173766042 0 0) 0x7f3600003520 con 0x7f361c0cd020 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c160910 msgr2=0x7f361c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c160910 0x7f361c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 >> v1:172.21.15.76:6789/0 conn(0x7f361c1454e0 legacy=0x7f361c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c0cdae0 con 0x7f361c0cd020 2024-04-23T14:43:33.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f3604001670 con 0x7f361c0cd020 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 >> v1:172.21.15.59:6789/0 conn(0x7f361c0cd020 legacy=0x7f361c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36000039e0 0x7f3600003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3600004800 0x7f3600004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36000052e0 0x7f360000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0cddc0 con 0x7f3600004800 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f360000c160 con 0x7f36000052e0 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f360000c360 con 0x7f36000039e0 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f361a800640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c160910 0x7f361c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.533+0000 7f3611400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3600004800 0x7f3600004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.534+0000 7f3619e00640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36000039e0 0x7f3600003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.534+0000 7f3611400640 1 -- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36000039e0 msgr2=0x7f3600003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.534+0000 7f3611400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36000039e0 0x7f3600003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.534+0000 7f3611400640 1 -- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36000052e0 msgr2=0x7f360000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.534+0000 7f3611400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36000052e0 0x7f360000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.534+0000 7f3611400640 1 -- 172.21.15.59:0/3775446901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3600003520 con 0x7f3600004800 2024-04-23T14:43:33.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.535+0000 7f3611400640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3600004800 0x7f3600004c80 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f360400c190 tx=0x7f3600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429bac95ea51aea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.535+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604007c60 con 0x7f3600004800 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.535+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604005ad0 con 0x7f3600004800 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.535+0000 7f3619400640 1 -- 172.21.15.59:0/3775446901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604007070 con 0x7f3600004800 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.536+0000 7f3621000640 1 -- 172.21.15.59:0/3775446901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3600004800 msgr2=0x7f3600004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.536+0000 7f3621000640 1 --2- 172.21.15.59:0/3775446901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3600004800 0x7f3600004c80 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f360400c190 tx=0x7f3600005830 comp rx=0 tx=0).stop 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.536+0000 7f361a800640 1 -- 172.21.15.59:0/3775446901 reap_dead start 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.536+0000 7f3621000640 1 -- 172.21.15.59:0/3775446901 shutdown_connections 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.536+0000 7f3621000640 1 -- 172.21.15.59:0/3775446901 >> 172.21.15.59:0/3775446901 conn(0x7f361c067ea0 msgr2=0x7f361c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:33.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.536+0000 7f3621000640 1 -- 172.21.15.59:0/3775446901 shutdown_connections 2024-04-23T14:43:33.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.536+0000 7f3621000640 1 -- 172.21.15.59:0/3775446901 wait complete. 2024-04-23T14:43:33.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.538+0000 7f3621000640 1 Processor -- start 2024-04-23T14:43:33.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.538+0000 7f3621000640 1 -- start start 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3621000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3600004800 0x7f361c0ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3621000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f361c0a9770 0x7f361c0aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3621000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c13f940 0x7f361c06f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3621000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0b4010 con 0x7f361c0a9770 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3621000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1462c0 con 0x7f3600004800 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3621000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f361c1439c0 con 0x7f361c13f940 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3611400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f361c0a9770 0x7f361c0aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3611400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f361c0a9770 0x7f361c0aa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33790/0 (socket says 172.21.15.59:33790) 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.539+0000 7f3611400640 1 -- 172.21.15.59:0/3851573666 learned_addr learned my addr 172.21.15.59:0/3851573666 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:33.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.540+0000 7f3611400640 1 -- 172.21.15.59:0/3851573666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c13f940 msgr2=0x7f361c06f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.540+0000 7f3611400640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c13f940 0x7f361c06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.540+0000 7f3611400640 1 -- 172.21.15.59:0/3851573666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3600004800 msgr2=0x7f361c0ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.540+0000 7f3611400640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3600004800 0x7f361c0ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.540+0000 7f3611400640 1 -- 172.21.15.59:0/3851573666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c0aa790 con 0x7f361c0a9770 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.540+0000 7f3611400640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f361c0a9770 0x7f361c0aa250 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f360400c2b0 tx=0x7f3604001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.541+0000 7f3612a00640 1 -- 172.21.15.59:0/3851573666 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604040020 con 0x7f361c0a9770 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.541+0000 7f3612a00640 1 -- 172.21.15.59:0/3851573666 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604001430 con 0x7f361c0a9770 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.541+0000 7f3612a00640 1 -- 172.21.15.59:0/3851573666 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604037040 con 0x7f361c0a9770 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.541+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c1433b0 con 0x7f361c0a9770 2024-04-23T14:43:33.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.541+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c143650 con 0x7f361c0a9770 2024-04-23T14:43:33.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.542+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361c068a80 con 0x7f361c0a9770 2024-04-23T14:43:33.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.543+0000 7f3612a00640 1 -- 172.21.15.59:0/3851573666 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3604043020 con 0x7f361c0a9770 2024-04-23T14:43:33.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.544+0000 7f3612a00640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35f806c8f0 0x7f35f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:33.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.544+0000 7f3612a00640 1 -- 172.21.15.59:0/3851573666 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7f36040aada0 con 0x7f361c0a9770 2024-04-23T14:43:33.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.544+0000 7f3619e00640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35f806c8f0 0x7f35f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:33.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.545+0000 7f3619e00640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35f806c8f0 0x7f35f806edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f360c007fd0 tx=0x7f360c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:33.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.548+0000 7f3612a00640 1 -- 172.21.15.59:0/3851573666 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f361c0aa790 con 0x7f361c0a9770 2024-04-23T14:43:33.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.829+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f361c160030 con 0x7f361c0a9770 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.831+0000 7f3612a00640 1 -- 172.21.15.59:0/3851573666 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f3604079480 con 0x7f361c0a9770 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmYXSWBBAAEeeBn7Ky0ELpRWdobS6W7A==", 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmazm1HhAAI01cFbo3Nnt9qQd1nO91MA==", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmXf/4EBAAl0bs6Po0NQDZJ91MnkBYvw==", 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2024-04-23T14:43:33.834 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmgnBzKxAAdZ0a4nFkjbjWV6NHNInahg==", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQACxidmY0yVJxAACBMZtvdM6SERv90w35P9ow==", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.835 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAFxidmGwg9NBAAzmKfWOmbcJTcVhqxvpZUaQ==", 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAJxidmi+SeExAAQFIqRPO2FwquLvcMGD/6LQ==", 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAMxidmRlxyHxAAn21xDKtvk0BUrYeii69M4g==", 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQASxidmtJbqKxAA+W3JMZPQfPqPmH7nOfBRVA==", 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2024-04-23T14:43:33.838 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWxidmM2xeCRAA1Q0dAMY7igHmGeiHTmgcyA==", 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAZxidm/wIdIxAAq/3crdAE++7NSWy2VMz1EA==", 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAcxidmIbj0LRAAw6JqzNUCogffYvgkgXP4bQ==", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ==", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2024-04-23T14:43:33.840 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmuTJqDhAAuYwzXtD290Gia85sYhEj2Q==", 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdm0tIoBRAA80OaSrKTU0CpFPqqHXb9eg==", 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD/xSdmYnfWFhAAzsn7677w4ds2wi2JpysJmg==", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2024-04-23T14:43:33.842 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmEteHAhAAwfUHmvRmEMcPIphwikq6sw==", 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmUf/LNxAArHypu1wKN+01dd0OzIdIMA==", 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD9xSdmQRX6KxAACFBriKK8dWVVkorYVfyyuQ==", 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-04-23T14:43:33.844 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:43:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.833+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35f806c8f0 msgr2=0x7f35f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.833+0000 7f3621000640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35f806c8f0 0x7f35f806edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f360c007fd0 tx=0x7f360c009040 comp rx=0 tx=0).stop 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.833+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f361c0a9770 msgr2=0x7f361c0aa250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.833+0000 7f3621000640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f361c0a9770 0x7f361c0aa250 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f360400c2b0 tx=0x7f3604001970 comp rx=0 tx=0).stop 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 shutdown_connections 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f35f806c8f0 0x7f35f806edb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f361c13f940 0x7f361c06f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f361c0a9770 0x7f361c0aa250 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 --2- 172.21.15.59:0/3851573666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3600004800 0x7f361c0ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 >> 172.21.15.59:0/3851573666 conn(0x7f361c067ea0 msgr2=0x7f361c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 shutdown_connections 2024-04-23T14:43:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:33.834+0000 7f3621000640 1 -- 172.21.15.59:0/3851573666 wait complete. 2024-04-23T14:43:33.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:34.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.067+0000 7f7a19c00640 1 Processor -- start 2024-04-23T14:43:34.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.067+0000 7f7a19c00640 1 -- start start 2024-04-23T14:43:34.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.068+0000 7f7a19c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a140cd760 0x7f7a140b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.068+0000 7f7a19c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7a140b4690 0x7f7a140cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.068+0000 7f7a19c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a14160210 con 0x7f7a1415c3d0 2024-04-23T14:43:34.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.068+0000 7f7a19c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7a141604f0 con 0x7f7a140b4690 2024-04-23T14:43:34.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.068+0000 7f7a19c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7a141607d0 con 0x7f7a140cd760 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a19200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7a140b4690 0x7f7a140cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a19200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7a140b4690 0x7f7a140cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33806/0 (socket says 172.21.15.59:33806) 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a19200640 1 -- 172.21.15.59:0/1499283070 learned_addr learned my addr 172.21.15.59:0/1499283070 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a13400640 1 --2- 172.21.15.59:0/1499283070 >> v2:172.21.15.76:3300/0 conn(0x7f7a140cd760 0x7f7a140b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a19200640 1 -- 172.21.15.59:0/1499283070 >> v2:172.21.15.76:3300/0 conn(0x7f7a140cd760 msgr2=0x7f7a140b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a19200640 1 --2- 172.21.15.59:0/1499283070 >> v2:172.21.15.76:3300/0 conn(0x7f7a140cd760 0x7f7a140b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a19200640 1 -- 172.21.15.59:0/1499283070 >> v1:172.21.15.76:6789/0 conn(0x7f7a1415c3d0 legacy=0x7f7a140cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.069+0000 7f7a19200640 1 -- 172.21.15.59:0/1499283070 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a14160ab0 con 0x7f7a140b4690 2024-04-23T14:43:34.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.070+0000 7f7a19200640 1 --2- 172.21.15.59:0/1499283070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a140b4690 0x7f7a140cb7d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f7a1406e620 tx=0x7f7a0c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3031254b1473e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.071+0000 7f7a12a00640 1 -- 172.21.15.59:0/1499283070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c037070 con 0x7f7a140b4690 2024-04-23T14:43:34.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.071+0000 7f7a12a00640 1 -- 172.21.15.59:0/1499283070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a0c032440 con 0x7f7a140b4690 2024-04-23T14:43:34.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.071+0000 7f7a12a00640 1 -- 172.21.15.59:0/1499283070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c032810 con 0x7f7a140b4690 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.071+0000 7f7a19c00640 1 -- 172.21.15.59:0/1499283070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a140b4690 msgr2=0x7f7a140cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.072+0000 7f7a19c00640 1 --2- 172.21.15.59:0/1499283070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a140b4690 0x7f7a140cb7d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f7a1406e620 tx=0x7f7a0c02a230 comp rx=0 tx=0).stop 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.072+0000 7f7a19c00640 1 -- 172.21.15.59:0/1499283070 shutdown_connections 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.072+0000 7f7a19c00640 1 --2- 172.21.15.59:0/1499283070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a140b4690 0x7f7a140cb7d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.072+0000 7f7a19c00640 1 --2- 172.21.15.59:0/1499283070 >> v2:172.21.15.76:3300/0 conn(0x7f7a140cd760 0x7f7a140b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.072+0000 7f7a19c00640 1 -- 172.21.15.59:0/1499283070 >> 172.21.15.59:0/1499283070 conn(0x7f7a140b21a0 msgr2=0x7f7a140b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.072+0000 7f7a19c00640 1 -- 172.21.15.59:0/1499283070 shutdown_connections 2024-04-23T14:43:34.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.072+0000 7f7a19c00640 1 -- 172.21.15.59:0/1499283070 wait complete. 2024-04-23T14:43:34.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.074+0000 7f7a19c00640 1 Processor -- start 2024-04-23T14:43:34.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.074+0000 7f7a19c00640 1 -- start start 2024-04-23T14:43:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.075+0000 7f7a19c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a140b4690 0x7f7a14157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a140cd760 0x7f7a141f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a1415c3d0 0x7f7a141550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1415c7b0 con 0x7f7a1415c3d0 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a140b4ae0 con 0x7f7a140cd760 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1415c930 con 0x7f7a140b4690 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a1415c3d0 0x7f7a141550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a13e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a140b4690 0x7f7a14157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a1415c3d0 0x7f7a141550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33810/0 (socket says 172.21.15.59:33810) 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a13e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a140b4690 0x7f7a14157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49056/0 (socket says 172.21.15.59:49056) 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 -- 172.21.15.59:0/743113888 learned_addr learned my addr 172.21.15.59:0/743113888 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a13400640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a140cd760 0x7f7a141f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 -- 172.21.15.59:0/743113888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a140b4690 msgr2=0x7f7a14157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a140b4690 0x7f7a14157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 -- 172.21.15.59:0/743113888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a140cd760 msgr2=0x7f7a141f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a140cd760 0x7f7a141f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.076+0000 7f7a19200640 1 -- 172.21.15.59:0/743113888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a141f0650 con 0x7f7a1415c3d0 2024-04-23T14:43:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.077+0000 7f7a19200640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a1415c3d0 0x7f7a141550d0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009960 tx=0x7f7a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.077+0000 7f7a0be00640 1 -- 172.21.15.59:0/743113888 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c041020 con 0x7f7a1415c3d0 2024-04-23T14:43:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.077+0000 7f7a0be00640 1 -- 172.21.15.59:0/743113888 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a0c030700 con 0x7f7a1415c3d0 2024-04-23T14:43:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.077+0000 7f7a0be00640 1 -- 172.21.15.59:0/743113888 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c037040 con 0x7f7a1415c3d0 2024-04-23T14:43:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.077+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a141f43a0 con 0x7f7a1415c3d0 2024-04-23T14:43:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.077+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a141f4960 con 0x7f7a1415c3d0 2024-04-23T14:43:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.079+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1404fd90 con 0x7f7a1415c3d0 2024-04-23T14:43:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.079+0000 7f7a0be00640 1 -- 172.21.15.59:0/743113888 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a0c044020 con 0x7f7a1415c3d0 2024-04-23T14:43:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.080+0000 7f7a0be00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79f406c920 0x7f79f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.081+0000 7f7a0be00640 1 -- 172.21.15.59:0/743113888 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7f7a0c0abf30 con 0x7f7a1415c3d0 2024-04-23T14:43:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.081+0000 7f7a13e00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79f406c920 0x7f79f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.082+0000 7f7a13e00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79f406c920 0x7f79f406ede0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400b450 tx=0x7f7a0400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.087+0000 7f7a0be00640 1 -- 172.21.15.59:0/743113888 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a0c04b030 con 0x7f7a1415c3d0 2024-04-23T14:43:34.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.291+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a14073b00 con 0x7f7a1415c3d0 2024-04-23T14:43:34.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.293+0000 7f7a0be00640 1 -- 172.21.15.59:0/743113888 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+12833 (secure 0 0 0) 0x7f7a0c07a6a0 con 0x7f7a1415c3d0 2024-04-23T14:43:34.294 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:34.294 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":108,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:33.289935+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.295+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79f406c920 msgr2=0x7f79f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.295+0000 7f7a19c00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79f406c920 0x7f79f406ede0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400b450 tx=0x7f7a0400d040 comp rx=0 tx=0).stop 2024-04-23T14:43:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.295+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a1415c3d0 msgr2=0x7f7a141550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.295+0000 7f7a19c00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a1415c3d0 0x7f7a141550d0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009960 tx=0x7f7a0c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 shutdown_connections 2024-04-23T14:43:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f79f406c920 0x7f79f406ede0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a1415c3d0 0x7f7a141550d0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a140cd760 0x7f7a141f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 --2- 172.21.15.59:0/743113888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a140b4690 0x7f7a14157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 >> 172.21.15.59:0/743113888 conn(0x7f7a140b21a0 msgr2=0x7f7a140cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:34.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 shutdown_connections 2024-04-23T14:43:34.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.296+0000 7f7a19c00640 1 -- 172.21.15.59:0/743113888 wait complete. 2024-04-23T14:43:34.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:43:34.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:43:34.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.519+0000 7fa58f200640 1 Processor -- start 2024-04-23T14:43:34.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.519+0000 7fa58f200640 1 -- start start 2024-04-23T14:43:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.521+0000 7fa58f200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5900ce860 0x7fa59006b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.521+0000 7fa58f200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5900b2ca0 con 0x7fa59006bf10 2024-04-23T14:43:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.521+0000 7fa58f200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5900cdaa0 con 0x7fa59015c940 2024-04-23T14:43:34.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.521+0000 7fa58f200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa59006c380 con 0x7fa5900ce860 2024-04-23T14:43:34.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.521+0000 7fa58e800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa59006bf10 0x7fa5900b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47672/0 (socket says 172.21.15.59:47672) 2024-04-23T14:43:34.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.521+0000 7fa58e800640 1 -- 172.21.15.59:0/2640805194 learned_addr learned my addr 172.21.15.59:0/2640805194 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:34.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.521+0000 7fa58d400640 1 --2- 172.21.15.59:0/2640805194 >> v2:172.21.15.76:3300/0 conn(0x7fa5900ce860 0x7fa59006b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545494557 0 0) 0x7fa5900b2ca0 con 0x7fa59006bf10 2024-04-23T14:43:34.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa574003520 con 0x7fa59006bf10 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2962592311 0 0) 0x7fa574003520 con 0x7fa59006bf10 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 >> v2:172.21.15.76:3300/0 conn(0x7fa5900ce860 msgr2=0x7fa59006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 --2- 172.21.15.59:0/2640805194 >> v2:172.21.15.76:3300/0 conn(0x7fa5900ce860 0x7fa59006b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 >> v1:172.21.15.76:6789/0 conn(0x7fa59015c940 legacy=0x7fa5900b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5900cdd80 con 0x7fa59006bf10 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fa5880019f0 con 0x7fa59006bf10 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 >> v1:172.21.15.59:6789/0 conn(0x7fa59006bf10 legacy=0x7fa5900b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5740039b0 0x7fa574003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa574004800 0x7fa574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5740051c0 0x7fa57400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa59006c380 con 0x7fa574004800 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa57400c010 con 0x7fa5740039b0 2024-04-23T14:43:34.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.522+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa57400c210 con 0x7fa5740051c0 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58d400640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa574004800 0x7fa574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58de00640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5740039b0 0x7fa574003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58e800640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5740051c0 0x7fa57400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58d400640 1 -- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5740051c0 msgr2=0x7fa57400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58d400640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5740051c0 0x7fa57400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58d400640 1 -- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5740039b0 msgr2=0x7fa574003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58d400640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5740039b0 0x7fa574003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58d400640 1 -- 172.21.15.59:0/2640805194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa574003520 con 0x7fa574004800 2024-04-23T14:43:34.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.523+0000 7fa58e800640 1 -- 172.21.15.59:0/2640805194 reap_dead start 2024-04-23T14:43:34.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.524+0000 7fa58d400640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa574004800 0x7fa574004c80 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa580009ac0 tx=0x7fa574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422a1f052e0dd0e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580036070 con 0x7fa574004800 2024-04-23T14:43:34.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa580031400 con 0x7fa574004800 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58ca00640 1 -- 172.21.15.59:0/2640805194 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580009690 con 0x7fa574004800 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58f200640 1 -- 172.21.15.59:0/2640805194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa574004800 msgr2=0x7fa574004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58f200640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa574004800 0x7fa574004c80 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa580009ac0 tx=0x7fa574005830 comp rx=0 tx=0).stop 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58f200640 1 -- 172.21.15.59:0/2640805194 shutdown_connections 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58f200640 1 --2- 172.21.15.59:0/2640805194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa574004800 0x7fa574004c80 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58f200640 1 -- 172.21.15.59:0/2640805194 >> 172.21.15.59:0/2640805194 conn(0x7fa5900b2f10 msgr2=0x7fa5900b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.525+0000 7fa58f200640 1 -- 172.21.15.59:0/2640805194 shutdown_connections 2024-04-23T14:43:34.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.526+0000 7fa58f200640 1 -- 172.21.15.59:0/2640805194 wait complete. 2024-04-23T14:43:34.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.527+0000 7fa58f200640 1 Processor -- start 2024-04-23T14:43:34.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.527+0000 7fa58f200640 1 -- start start 2024-04-23T14:43:34.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.528+0000 7fa58f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa574004800 0x7fa59014a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.528+0000 7fa58f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5900ae1f0 0x7fa59015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.528+0000 7fa58f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59015a330 0x7fa59015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.528+0000 7fa58f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5900c4ba0 con 0x7fa59015a330 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.528+0000 7fa58f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5900ce340 con 0x7fa574004800 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa59006b9d0 con 0x7fa5900ae1f0 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59015a330 0x7fa59015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59015a330 0x7fa59015a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33832/0 (socket says 172.21.15.59:33832) 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 -- 172.21.15.59:0/4151204649 learned_addr learned my addr 172.21.15.59:0/4151204649 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58de00640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa574004800 0x7fa59014a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58d400640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5900ae1f0 0x7fa59015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 -- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5900ae1f0 msgr2=0x7fa59015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5900ae1f0 0x7fa59015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 -- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa574004800 msgr2=0x7fa59014a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa574004800 0x7fa59014a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58e800640 1 -- 172.21.15.59:0/4151204649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa590158ef0 con 0x7fa59015a330 2024-04-23T14:43:34.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.529+0000 7fa58de00640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa574004800 0x7fa59014a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:43:34.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.530+0000 7fa58e800640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59015a330 0x7fa59015a7e0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa5901602a0 tx=0x7fa58800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.530+0000 7fa586000640 1 -- 172.21.15.59:0/4151204649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58800eb90 con 0x7fa59015a330 2024-04-23T14:43:34.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.530+0000 7fa586000640 1 -- 172.21.15.59:0/4151204649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58800ed30 con 0x7fa59015a330 2024-04-23T14:43:34.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.530+0000 7fa586000640 1 -- 172.21.15.59:0/4151204649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588014650 con 0x7fa59015a330 2024-04-23T14:43:34.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.530+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5900c5f40 con 0x7fa59015a330 2024-04-23T14:43:34.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.530+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5900c6500 con 0x7fa59015a330 2024-04-23T14:43:34.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.531+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59004fd90 con 0x7fa59015a330 2024-04-23T14:43:34.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.532+0000 7fa586000640 1 -- 172.21.15.59:0/4151204649 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5880147f0 con 0x7fa59015a330 2024-04-23T14:43:34.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.534+0000 7fa586000640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa56c06c8f0 0x7fa56c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.534+0000 7fa586000640 1 -- 172.21.15.59:0/4151204649 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7fa58808b6b0 con 0x7fa59015a330 2024-04-23T14:43:34.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.534+0000 7fa58de00640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa56c06c8f0 0x7fa56c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.535+0000 7fa58de00640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa56c06c8f0 0x7fa56c06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa5900cb140 tx=0x7fa578007c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.538+0000 7fa586000640 1 -- 172.21.15.59:0/4151204649 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa588059d90 con 0x7fa59015a330 2024-04-23T14:43:34.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.745+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5900cb800 con 0x7fa59015a330 2024-04-23T14:43:34.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.746+0000 7fa586000640 1 -- 172.21.15.59:0/4151204649 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7fa588059690 con 0x7fa59015a330 2024-04-23T14:43:34.747 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:43:34.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa56c06c8f0 msgr2=0x7fa56c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa56c06c8f0 0x7fa56c06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa5900cb140 tx=0x7fa578007c50 comp rx=0 tx=0).stop 2024-04-23T14:43:34.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59015a330 msgr2=0x7fa59015a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59015a330 0x7fa59015a7e0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa5901602a0 tx=0x7fa58800e530 comp rx=0 tx=0).stop 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 shutdown_connections 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa56c06c8f0 0x7fa56c06edb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5900ae1f0 0x7fa59015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59015a330 0x7fa59015a7e0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 --2- 172.21.15.59:0/4151204649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa574004800 0x7fa59014a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 >> 172.21.15.59:0/4151204649 conn(0x7fa5900b2f10 msgr2=0x7fa5900703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 shutdown_connections 2024-04-23T14:43:34.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.749+0000 7fa58f200640 1 -- 172.21.15.59:0/4151204649 wait complete. 2024-04-23T14:43:34.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:43:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.984+0000 7fe593600640 1 Processor -- start 2024-04-23T14:43:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.985+0000 7fe593600640 1 -- start start 2024-04-23T14:43:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.986+0000 7fe593600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c1454e0 0x7fe58c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.986+0000 7fe593600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe58c0cd020 0x7fe58c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.986+0000 7fe593600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c0cd520 con 0x7fe58c0b3e40 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.986+0000 7fe593600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c0cddc0 con 0x7fe58c0cd020 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.986+0000 7fe593600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c0cd800 con 0x7fe58c1454e0 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe58be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe58c0cd020 0x7fe58c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe591600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe58c0b3e40 0x7fe58c160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47680/0 (socket says 172.21.15.59:47680) 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe591600640 1 -- 172.21.15.59:0/2626226939 learned_addr learned my addr 172.21.15.59:0/2626226939 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe590c00640 1 --2- 172.21.15.59:0/2626226939 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c1454e0 0x7fe58c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe58be00640 1 -- 172.21.15.59:0/2626226939 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c1454e0 msgr2=0x7fe58c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe58be00640 1 --2- 172.21.15.59:0/2626226939 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c1454e0 0x7fe58c145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe58be00640 1 -- 172.21.15.59:0/2626226939 >> v1:172.21.15.59:6789/0 conn(0x7fe58c0b3e40 legacy=0x7fe58c160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.987+0000 7fe58be00640 1 -- 172.21.15.59:0/2626226939 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c0cdae0 con 0x7fe58c0cd020 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.988+0000 7fe58be00640 1 --2- 172.21.15.59:0/2626226939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c0cd020 0x7fe58c0b3900 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fe5780097a0 tx=0x7fe57802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7ef792f36521a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe58b400640 1 -- 172.21.15.59:0/2626226939 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578037070 con 0x7fe58c0cd020 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe58b400640 1 -- 172.21.15.59:0/2626226939 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57802ae40 con 0x7fe58c0cd020 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe58b400640 1 -- 172.21.15.59:0/2626226939 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578032660 con 0x7fe58c0cd020 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe593600640 1 -- 172.21.15.59:0/2626226939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c0cd020 msgr2=0x7fe58c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe593600640 1 --2- 172.21.15.59:0/2626226939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c0cd020 0x7fe58c0b3900 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fe5780097a0 tx=0x7fe57802a0f0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe593600640 1 -- 172.21.15.59:0/2626226939 shutdown_connections 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe593600640 1 --2- 172.21.15.59:0/2626226939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c0cd020 0x7fe58c0b3900 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe593600640 1 --2- 172.21.15.59:0/2626226939 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c1454e0 0x7fe58c145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.989+0000 7fe593600640 1 -- 172.21.15.59:0/2626226939 >> 172.21.15.59:0/2626226939 conn(0x7fe58c067ea0 msgr2=0x7fe58c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.990+0000 7fe593600640 1 -- 172.21.15.59:0/2626226939 shutdown_connections 2024-04-23T14:43:34.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.990+0000 7fe593600640 1 -- 172.21.15.59:0/2626226939 wait complete. 2024-04-23T14:43:34.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.991+0000 7fe593600640 1 Processor -- start 2024-04-23T14:43:34.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.992+0000 7fe593600640 1 -- start start 2024-04-23T14:43:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe593600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58c0b3e40 0x7fe58c13e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe593600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c0cd020 0x7fe58c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe593600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c1454e0 0x7fe58c0b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe593600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c146270 con 0x7fe58c1454e0 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe593600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1460f0 con 0x7fe58c0b3e40 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe593600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c161020 con 0x7fe58c0cd020 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c1454e0 0x7fe58c0b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c1454e0 0x7fe58c0b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33850/0 (socket says 172.21.15.59:33850) 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 -- 172.21.15.59:0/238657637 learned_addr learned my addr 172.21.15.59:0/238657637 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe590c00640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58c0b3e40 0x7fe58c13e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe58be00640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c0cd020 0x7fe58c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 -- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c0cd020 msgr2=0x7fe58c0b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c0cd020 0x7fe58c0b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 -- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58c0b3e40 msgr2=0x7fe58c13e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58c0b3e40 0x7fe58c13e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:34.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe591600640 1 -- 172.21.15.59:0/238657637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c0aaef0 con 0x7fe58c1454e0 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe58be00640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c0cd020 0x7fe58c0b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.993+0000 7fe590c00640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58c0b3e40 0x7fe58c13e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.994+0000 7fe591600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c1454e0 0x7fe58c0b5320 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe58400cbc0 tx=0x7fe58400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.994+0000 7fe588c00640 1 -- 172.21.15.59:0/238657637 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584017070 con 0x7fe58c1454e0 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.994+0000 7fe588c00640 1 -- 172.21.15.59:0/238657637 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58400ad40 con 0x7fe58c1454e0 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.994+0000 7fe588c00640 1 -- 172.21.15.59:0/238657637 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584012600 con 0x7fe58c1454e0 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.994+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c0aa930 con 0x7fe58c1454e0 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.994+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c142bf0 con 0x7fe58c1454e0 2024-04-23T14:43:34.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.996+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c068a80 con 0x7fe58c1454e0 2024-04-23T14:43:34.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.997+0000 7fe588c00640 1 -- 172.21.15.59:0/238657637 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe58401a070 con 0x7fe58c1454e0 2024-04-23T14:43:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.997+0000 7fe588c00640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe56c06c940 0x7fe56c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.998+0000 7fe588c00640 1 -- 172.21.15.59:0/238657637 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7fe58408b3e0 con 0x7fe58c1454e0 2024-04-23T14:43:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.998+0000 7fe590c00640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe56c06c940 0x7fe56c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:34.999+0000 7fe590c00640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe56c06c940 0x7fe56c06ee00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe57c006fd0 tx=0x7fe57c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:35.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.002+0000 7fe588c00640 1 -- 172.21.15.59:0/238657637 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5840595f0 con 0x7fe58c1454e0 2024-04-23T14:43:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.216+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe58c0af850 con 0x7fe58c1454e0 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.216+0000 7fe588c00640 1 -- 172.21.15.59:0/238657637 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) ==== 84+0+65 (secure 0 0 0) 0x7fe58405d8f0 con 0x7fe58c1454e0 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:35.218 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe56c06c940 msgr2=0x7fe56c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe56c06c940 0x7fe56c06ee00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe57c006fd0 tx=0x7fe57c008040 comp rx=0 tx=0).stop 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c1454e0 msgr2=0x7fe58c0b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c1454e0 0x7fe58c0b5320 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe58400cbc0 tx=0x7fe58400a430 comp rx=0 tx=0).stop 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 shutdown_connections 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe56c06c940 0x7fe56c06ee00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe58c0cd020 0x7fe58c0b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe58c1454e0 0x7fe58c0b5320 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 --2- 172.21.15.59:0/238657637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58c0b3e40 0x7fe58c13e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.219+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 >> 172.21.15.59:0/238657637 conn(0x7fe58c067ea0 msgr2=0x7fe58c15c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.220+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 shutdown_connections 2024-04-23T14:43:35.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.220+0000 7fe593600640 1 -- 172.21.15.59:0/238657637 wait complete. 2024-04-23T14:43:35.240 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:43:35.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:43:35.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.450+0000 7feea6200640 1 Processor -- start 2024-04-23T14:43:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.450+0000 7feea6200640 1 -- start start 2024-04-23T14:43:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.452+0000 7feea6200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feea015e9b0 0x7feea015f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.452+0000 7feea6200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00d0810 0x7feea00adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.452+0000 7feea6200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea00b4720 con 0x7feea00afb30 2024-04-23T14:43:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.452+0000 7feea6200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7feea00b3430 con 0x7feea015e9b0 2024-04-23T14:43:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.452+0000 7feea6200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7feea015fe40 con 0x7feea00d0810 2024-04-23T14:43:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.453+0000 7fee9f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feea015e9b0 0x7feea015f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.453+0000 7fee9fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00d0810 0x7feea00adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.453+0000 7fee9f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feea015e9b0 0x7feea015f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:60512/0 (socket says 172.21.15.59:60512) 2024-04-23T14:43:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.453+0000 7fee9f400640 1 -- 172.21.15.59:0/1921122803 learned_addr learned my addr 172.21.15.59:0/1921122803 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.454+0000 7fee9f400640 1 -- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00d0810 msgr2=0x7feea00adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.454+0000 7fee9f400640 1 --2- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00d0810 0x7feea00adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.454+0000 7fee9f400640 1 -- 172.21.15.59:0/1921122803 >> v1:172.21.15.76:6789/0 conn(0x7feea00afb30 legacy=0x7feea00d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.454+0000 7fee9f400640 1 -- 172.21.15.59:0/1921122803 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea00b3680 con 0x7feea015e9b0 2024-04-23T14:43:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.454+0000 7fee9ea00640 1 -- 172.21.15.59:0/1921122803 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761851410 0 0) 0x7feea00b4720 con 0x7feea00afb30 2024-04-23T14:43:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.454+0000 7fee9fe00640 1 --2- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00d0810 0x7feea00adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:43:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.455+0000 7fee9f400640 1 --2- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 0x7feea015f900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fee90009870 tx=0x7fee9002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293d98be70f891e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.456+0000 7fee9ea00640 1 -- 172.21.15.59:0/1921122803 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90037070 con 0x7feea015e9b0 2024-04-23T14:43:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.456+0000 7fee9ea00640 1 -- 172.21.15.59:0/1921122803 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee90032440 con 0x7feea015e9b0 2024-04-23T14:43:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.456+0000 7fee9ea00640 1 -- 172.21.15.59:0/1921122803 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90032810 con 0x7feea015e9b0 2024-04-23T14:43:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 -- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 msgr2=0x7feea015f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 --2- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 0x7feea015f900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fee90009870 tx=0x7fee9002a250 comp rx=0 tx=0).stop 2024-04-23T14:43:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 -- 172.21.15.59:0/1921122803 shutdown_connections 2024-04-23T14:43:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 --2- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00d0810 0x7feea00adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 --2- 172.21.15.59:0/1921122803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 0x7feea015f900 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 -- 172.21.15.59:0/1921122803 >> 172.21.15.59:0/1921122803 conn(0x7feea006b9d0 msgr2=0x7feea006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 -- 172.21.15.59:0/1921122803 shutdown_connections 2024-04-23T14:43:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.457+0000 7feea6200640 1 -- 172.21.15.59:0/1921122803 wait complete. 2024-04-23T14:43:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.459+0000 7feea6200640 1 Processor -- start 2024-04-23T14:43:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.459+0000 7feea6200640 1 -- start start 2024-04-23T14:43:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7feea6200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00afb30 0x7feea0156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7feea6200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feea00d0810 0x7feea0152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7feea6200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 0x7feea015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7feea6200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feea00b3f20 con 0x7feea00d0810 2024-04-23T14:43:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7feea6200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feea00b3c40 con 0x7feea015e9b0 2024-04-23T14:43:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7feea6200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7feea00b4200 con 0x7feea00afb30 2024-04-23T14:43:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7fee97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feea00d0810 0x7feea0152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7fee97e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feea00d0810 0x7feea0152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33856/0 (socket says 172.21.15.59:33856) 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7fee97e00640 1 -- 172.21.15.59:0/335352394 learned_addr learned my addr 172.21.15.59:0/335352394 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7fee9fe00640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 0x7feea015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.460+0000 7fee9f400640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00afb30 0x7feea0156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97e00640 1 -- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00afb30 msgr2=0x7feea0156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97e00640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00afb30 0x7feea0156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97e00640 1 -- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 msgr2=0x7feea015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97e00640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 0x7feea015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97e00640 1 -- 172.21.15.59:0/335352394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea01533f0 con 0x7feea00d0810 2024-04-23T14:43:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97e00640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feea00d0810 0x7feea0152cf0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c450 tx=0x7fee8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97400640 1 -- 172.21.15.59:0/335352394 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880052d0 con 0x7feea00d0810 2024-04-23T14:43:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97400640 1 -- 172.21.15.59:0/335352394 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88005470 con 0x7feea00d0810 2024-04-23T14:43:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7fee97400640 1 -- 172.21.15.59:0/335352394 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88012870 con 0x7feea00d0810 2024-04-23T14:43:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.461+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea015ba20 con 0x7feea00d0810 2024-04-23T14:43:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.462+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea015bfe0 con 0x7feea00d0810 2024-04-23T14:43:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.463+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea015c1e0 con 0x7feea00d0810 2024-04-23T14:43:35.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.464+0000 7fee97400640 1 -- 172.21.15.59:0/335352394 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee88012a10 con 0x7feea00d0810 2024-04-23T14:43:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.465+0000 7fee97400640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee7c06c8f0 0x7fee7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.465+0000 7fee97400640 1 -- 172.21.15.59:0/335352394 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7fee8808b4e0 con 0x7feea00d0810 2024-04-23T14:43:35.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.465+0000 7fee9f400640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee7c06c8f0 0x7fee7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.466+0000 7fee9f400640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee7c06c8f0 0x7fee7c06edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fee90034000 tx=0x7fee9002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:35.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.470+0000 7fee97400640 1 -- 172.21.15.59:0/335352394 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feea015c1e0 con 0x7feea00d0810 2024-04-23T14:43:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:35.675+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feea0068a80 con 0x7feea00d0810 2024-04-23T14:43:36.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.352+0000 7fee97400640 1 -- 172.21.15.59:0/335352394 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) ==== 138+0+0 (secure 0 0 0) 0x7fee88059c80 con 0x7feea00d0810 2024-04-23T14:43:36.354 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-04-23T14:43:36.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee7c06c8f0 msgr2=0x7fee7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:36.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee7c06c8f0 0x7fee7c06edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fee90034000 tx=0x7fee9002e000 comp rx=0 tx=0).stop 2024-04-23T14:43:36.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feea00d0810 msgr2=0x7feea0152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:36.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feea00d0810 0x7feea0152cf0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c450 tx=0x7fee8800c910 comp rx=0 tx=0).stop 2024-04-23T14:43:36.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 shutdown_connections 2024-04-23T14:43:36.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fee7c06c8f0 0x7fee7c06edb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feea015e9b0 0x7feea015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.356+0000 7feea6200640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feea00d0810 0x7feea0152cf0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.357+0000 7feea6200640 1 --2- 172.21.15.59:0/335352394 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feea00afb30 0x7feea0156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.357+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 >> 172.21.15.59:0/335352394 conn(0x7feea006b9d0 msgr2=0x7feea00b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:36.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.357+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 shutdown_connections 2024-04-23T14:43:36.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.357+0000 7feea6200640 1 -- 172.21.15.59:0/335352394 wait complete. 2024-04-23T14:43:36.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:43:36.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.584+0000 7fbb0a200640 1 Processor -- start 2024-04-23T14:43:36.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.584+0000 7fbb0a200640 1 -- start start 2024-04-23T14:43:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.586+0000 7fbb0a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04154f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.586+0000 7fbb0a200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0415ed90 con 0x7fbb0415af40 2024-04-23T14:43:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.586+0000 7fbb0a200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0415f070 con 0x7fbb0406fe60 2024-04-23T14:43:36.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.586+0000 7fbb0a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040b34d0 con 0x7fbb0406f080 2024-04-23T14:43:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.586+0000 7fbb02a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbb0415af40 0x7fbb0406e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47696/0 (socket says 172.21.15.59:47696) 2024-04-23T14:43:36.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.586+0000 7fbb02a00640 1 -- 172.21.15.59:0/2748350812 learned_addr learned my addr 172.21.15.59:0/2748350812 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.587+0000 7fbb03e00640 1 --2- 172.21.15.59:0/2748350812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.587+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869137417 0 0) 0x7fbb0415ed90 con 0x7fbb0415af40 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.587+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbae8003520 con 0x7fbb0415af40 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.588+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003326684 0 0) 0x7fbb0415f070 con 0x7fbb0406fe60 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.588+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb0415ed90 con 0x7fbb0406fe60 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.588+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3601145437 0 0) 0x7fbae8003520 con 0x7fbb0415af40 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.588+0000 7fbb03e00640 1 -- 172.21.15.59:0/2748350812 >> v1:172.21.15.76:6789/0 conn(0x7fbb0406fe60 legacy=0x7fbb0415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.588+0000 7fbb03e00640 1 -- 172.21.15.59:0/2748350812 >> v1:172.21.15.59:6789/0 conn(0x7fbb0415af40 legacy=0x7fbb0406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:36.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.588+0000 7fbb03e00640 1 -- 172.21.15.59:0/2748350812 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0415c080 con 0x7fbb0406f080 2024-04-23T14:43:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.589+0000 7fbb03e00640 1 --2- 172.21.15.59:0/2748350812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04154f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009dc0 tx=0x7fbafc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767c20b3c49f51d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.589+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0001670 con 0x7fbb0406f080 2024-04-23T14:43:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.590+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc007540 con 0x7fbb0406f080 2024-04-23T14:43:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.590+0000 7fbb02000640 1 -- 172.21.15.59:0/2748350812 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007910 con 0x7fbb0406f080 2024-04-23T14:43:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.590+0000 7fbb0a200640 1 -- 172.21.15.59:0/2748350812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 msgr2=0x7fbb04154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.590+0000 7fbb0a200640 1 --2- 172.21.15.59:0/2748350812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04154f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009dc0 tx=0x7fbafc02a5d0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.590+0000 7fbb0a200640 1 -- 172.21.15.59:0/2748350812 shutdown_connections 2024-04-23T14:43:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.590+0000 7fbb0a200640 1 --2- 172.21.15.59:0/2748350812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04154f90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.590+0000 7fbb0a200640 1 -- 172.21.15.59:0/2748350812 >> 172.21.15.59:0/2748350812 conn(0x7fbb04067d90 msgr2=0x7fbb0406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.591+0000 7fbb0a200640 1 -- 172.21.15.59:0/2748350812 shutdown_connections 2024-04-23T14:43:36.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.591+0000 7fbb0a200640 1 -- 172.21.15.59:0/2748350812 wait complete. 2024-04-23T14:43:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.592+0000 7fbb0a200640 1 Processor -- start 2024-04-23T14:43:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.592+0000 7fbb0a200640 1 -- start start 2024-04-23T14:43:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.593+0000 7fbb0a200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.593+0000 7fbb0a200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0406fe60 0x7fbb041571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.593+0000 7fbb0a200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415af40 0x7fbb040cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.593+0000 7fbb0a200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040b3e50 con 0x7fbb0415af40 2024-04-23T14:43:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.593+0000 7fbb0a200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040b3cd0 con 0x7fbb0406fe60 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.593+0000 7fbb0a200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbb040b3fd0 con 0x7fbb0406f080 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415af40 0x7fbb040cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415af40 0x7fbb040cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33860/0 (socket says 172.21.15.59:33860) 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 -- 172.21.15.59:0/1062108151 learned_addr learned my addr 172.21.15.59:0/1062108151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03400640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb02a00640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0406fe60 0x7fbb041571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 -- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 msgr2=0x7fbb04153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 -- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0406fe60 msgr2=0x7fbb041571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0406fe60 0x7fbb041571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.594+0000 7fbb03e00640 1 -- 172.21.15.59:0/1062108151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb04156160 con 0x7fbb0415af40 2024-04-23T14:43:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.595+0000 7fbb03e00640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415af40 0x7fbb040cfc50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009ee0 tx=0x7fbafc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.595+0000 7fbafb400640 1 -- 172.21.15.59:0/1062108151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc040070 con 0x7fbb0415af40 2024-04-23T14:43:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.595+0000 7fbafb400640 1 -- 172.21.15.59:0/1062108151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc031bc0 con 0x7fbb0415af40 2024-04-23T14:43:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.595+0000 7fbafb400640 1 -- 172.21.15.59:0/1062108151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc009450 con 0x7fbb0415af40 2024-04-23T14:43:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.595+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb04156420 con 0x7fbb0415af40 2024-04-23T14:43:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.595+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041569e0 con 0x7fbb0415af40 2024-04-23T14:43:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.596+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb04068a80 con 0x7fbb0415af40 2024-04-23T14:43:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.598+0000 7fbafb400640 1 -- 172.21.15.59:0/1062108151 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbafc039040 con 0x7fbb0415af40 2024-04-23T14:43:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.599+0000 7fbafb400640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.599+0000 7fbafb400640 1 -- 172.21.15.59:0/1062108151 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5227+0+0 (secure 0 0 0) 0x7fbafc0ad160 con 0x7fbb0415af40 2024-04-23T14:43:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.599+0000 7fbb03400640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.600+0000 7fbb03400640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbb0406f900 tx=0x7fbaf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.604+0000 7fbafb400640 1 -- 172.21.15.59:0/1062108151 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbafc07b6d0 con 0x7fbb0415af40 2024-04-23T14:43:36.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:36.812+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbb04159cb0 con 0x7fbb0415af40 2024-04-23T14:43:37.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.359+0000 7fbafb400640 1 -- 172.21.15.59:0/1062108151 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) ==== 170+0+0 (secure 0 0 0) 0x7fbafc07afd0 con 0x7fbb0415af40 2024-04-23T14:43:37.361 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-04-23T14:43:37.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 msgr2=0x7fbae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:37.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbb0406f900 tx=0x7fbaf4009040 comp rx=0 tx=0).stop 2024-04-23T14:43:37.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415af40 msgr2=0x7fbb040cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:37.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415af40 0x7fbb040cfc50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009ee0 tx=0x7fbafc004a40 comp rx=0 tx=0).stop 2024-04-23T14:43:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 shutdown_connections 2024-04-23T14:43:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbae406c8f0 0x7fbae406edb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb0415af40 0x7fbb040cfc50 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb0406fe60 0x7fbb041571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 --2- 172.21.15.59:0/1062108151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbb0406f080 0x7fbb04153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.363+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 >> 172.21.15.59:0/1062108151 conn(0x7fbb04067d90 msgr2=0x7fbb04071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.364+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 shutdown_connections 2024-04-23T14:43:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.364+0000 7fbb0a200640 1 -- 172.21.15.59:0/1062108151 wait complete. 2024-04-23T14:43:37.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:43:37.582 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-23T14:43:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.674+0000 7fb386400640 1 Processor -- start 2024-04-23T14:43:37.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.674+0000 7fb386400640 1 -- start start 2024-04-23T14:43:37.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.676+0000 7fb386400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 0x7fb380163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.676+0000 7fb386400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb380163320 con 0x7fb38015d300 2024-04-23T14:43:37.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.676+0000 7fb386400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3800720d0 con 0x7fb3800d0600 2024-04-23T14:43:37.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.676+0000 7fb386400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb380163650 con 0x7fb380071c70 2024-04-23T14:43:37.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.676+0000 7fb385000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb38015d300 0x7fb380167180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47698/0 (socket says 172.21.15.59:47698) 2024-04-23T14:43:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.676+0000 7fb385000640 1 -- 172.21.15.59:0/2364275761 learned_addr learned my addr 172.21.15.59:0/2364275761 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.676+0000 7fb37fe00640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 0x7fb380163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3728402183 0 0) 0x7fb380163320 con 0x7fb38015d300 2024-04-23T14:43:37.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb364003520 con 0x7fb38015d300 2024-04-23T14:43:37.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3939797033 0 0) 0x7fb364003520 con 0x7fb38015d300 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 msgr2=0x7fb380163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 0x7fb380163ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 >> v1:172.21.15.76:6789/0 conn(0x7fb3800d0600 legacy=0x7fb38006e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb380072380 con 0x7fb38015d300 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fb370001670 con 0x7fb38015d300 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 >> v1:172.21.15.59:6789/0 conn(0x7fb38015d300 legacy=0x7fb380167180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3640039e0 0x7fb364003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb364004800 0x7fb364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 0x7fb3640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb380163650 con 0x7fb3640039e0 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb36400ba70 con 0x7fb364004800 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.677+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb36400bc70 con 0x7fb380071c70 2024-04-23T14:43:37.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb385000640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3640039e0 0x7fb364003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb37fe00640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb364004800 0x7fb364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb385a00640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 0x7fb3640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb385000640 1 -- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 msgr2=0x7fb3640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:37.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb385000640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb380071c70 0x7fb3640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb385000640 1 -- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb364004800 msgr2=0x7fb364004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:37.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb385000640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb364004800 0x7fb364004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.678+0000 7fb385000640 1 -- 172.21.15.59:0/2364275761 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb364003520 con 0x7fb3640039e0 2024-04-23T14:43:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.679+0000 7fb385000640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3640039e0 0x7fb364003e00 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c210 tx=0x7fb364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5497fe423b0ce418 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370007d10 con 0x7fb3640039e0 2024-04-23T14:43:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb370005ad0 con 0x7fb3640039e0 2024-04-23T14:43:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb37f400640 1 -- 172.21.15.59:0/2364275761 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370007070 con 0x7fb3640039e0 2024-04-23T14:43:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb386400640 1 -- 172.21.15.59:0/2364275761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3640039e0 msgr2=0x7fb364003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb386400640 1 --2- 172.21.15.59:0/2364275761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3640039e0 0x7fb364003e00 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c210 tx=0x7fb364005830 comp rx=0 tx=0).stop 2024-04-23T14:43:37.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb385a00640 1 -- 172.21.15.59:0/2364275761 reap_dead start 2024-04-23T14:43:37.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb386400640 1 -- 172.21.15.59:0/2364275761 shutdown_connections 2024-04-23T14:43:37.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb386400640 1 -- 172.21.15.59:0/2364275761 >> 172.21.15.59:0/2364275761 conn(0x7fb38006bdf0 msgr2=0x7fb38006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:37.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb386400640 1 -- 172.21.15.59:0/2364275761 shutdown_connections 2024-04-23T14:43:37.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.680+0000 7fb386400640 1 -- 172.21.15.59:0/2364275761 wait complete. 2024-04-23T14:43:37.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.682+0000 7fb386400640 1 Processor -- start 2024-04-23T14:43:37.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.682+0000 7fb386400640 1 -- start start 2024-04-23T14:43:37.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.683+0000 7fb386400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3640039e0 0x7fb38015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.683+0000 7fb386400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3800b1ed0 0x7fb3800b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.683+0000 7fb386400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb38009a9b0 0x7fb38009ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.683+0000 7fb386400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb38006e8f0 con 0x7fb38009a9b0 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.683+0000 7fb386400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb380072940 con 0x7fb3640039e0 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.683+0000 7fb386400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb38006ea70 con 0x7fb3800b1ed0 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb38009a9b0 0x7fb38009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb38009a9b0 0x7fb38009ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33886/0 (socket says 172.21.15.59:33886) 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 -- 172.21.15.59:0/1368165546 learned_addr learned my addr 172.21.15.59:0/1368165546 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385000640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3640039e0 0x7fb38015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb37fe00640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3800b1ed0 0x7fb3800b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 -- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3800b1ed0 msgr2=0x7fb3800b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3800b1ed0 0x7fb3800b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 -- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3640039e0 msgr2=0x7fb38015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3640039e0 0x7fb38015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 -- 172.21.15.59:0/1368165546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38009b370 con 0x7fb38009a9b0 2024-04-23T14:43:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb37fe00640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3800b1ed0 0x7fb3800b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:37.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385000640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3640039e0 0x7fb38015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:37.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.684+0000 7fb385a00640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb38009a9b0 0x7fb38009ae30 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b6360 tx=0x7fb37800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:37.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.685+0000 7fb37cc00640 1 -- 172.21.15.59:0/1368165546 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378003970 con 0x7fb38009a9b0 2024-04-23T14:43:37.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.685+0000 7fb37cc00640 1 -- 172.21.15.59:0/1368165546 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb378018070 con 0x7fb38009a9b0 2024-04-23T14:43:37.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.685+0000 7fb37cc00640 1 -- 172.21.15.59:0/1368165546 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378003d70 con 0x7fb38009a9b0 2024-04-23T14:43:37.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.685+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38015de50 con 0x7fb38009a9b0 2024-04-23T14:43:37.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.685+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38009c730 con 0x7fb38009a9b0 2024-04-23T14:43:37.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.686+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38004fd90 con 0x7fb38009a9b0 2024-04-23T14:43:37.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.688+0000 7fb37cc00640 1 -- 172.21.15.59:0/1368165546 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3780050d0 con 0x7fb38009a9b0 2024-04-23T14:43:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.689+0000 7fb37cc00640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb35c06c8f0 0x7fb35c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.689+0000 7fb37cc00640 1 -- 172.21.15.59:0/1368165546 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5609+0+0 (secure 0 0 0) 0x7fb37808c4c0 con 0x7fb38009a9b0 2024-04-23T14:43:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.689+0000 7fb385000640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb35c06c8f0 0x7fb35c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.690+0000 7fb385000640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb35c06c8f0 0x7fb35c06edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb370032000 tx=0x7fb37002d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:37.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.692+0000 7fb37cc00640 1 -- 172.21.15.59:0/1368165546 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb37805a930 con 0x7fb38009a9b0 2024-04-23T14:43:37.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:37.937+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb38009c120 con 0x7fb38009a9b0 2024-04-23T14:43:38.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.390+0000 7fb37cc00640 1 -- 172.21.15.59:0/1368165546 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v96) ==== 174+0+0 (secure 0 0 0) 0x7fb37805a230 con 0x7fb38009a9b0 2024-04-23T14:43:38.392 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 16 and data pool 17 2024-04-23T14:43:38.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb35c06c8f0 msgr2=0x7fb35c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb35c06c8f0 0x7fb35c06edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb370032000 tx=0x7fb37002d000 comp rx=0 tx=0).stop 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb38009a9b0 msgr2=0x7fb38009ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb38009a9b0 0x7fb38009ae30 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fb3800b6360 tx=0x7fb37800ec10 comp rx=0 tx=0).stop 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 shutdown_connections 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb35c06c8f0 0x7fb35c06edb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3800b1ed0 0x7fb3800b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb38009a9b0 0x7fb38009ae30 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 --2- 172.21.15.59:0/1368165546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3640039e0 0x7fb38015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.394+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 >> 172.21.15.59:0/1368165546 conn(0x7fb38006bdf0 msgr2=0x7fb38006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.395+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 shutdown_connections 2024-04-23T14:43:38.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.395+0000 7fb386400640 1 -- 172.21.15.59:0/1368165546 wait complete. 2024-04-23T14:43:38.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.615+0000 7f5a39a00640 1 Processor -- start 2024-04-23T14:43:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.615+0000 7f5a39a00640 1 -- start start 2024-04-23T14:43:38.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.617+0000 7f5a39a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a341454e0 0x7f5a34145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.617+0000 7f5a39a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a340cd020 0x7f5a340b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.617+0000 7f5a39a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a340cd520 con 0x7f5a340b3e40 2024-04-23T14:43:38.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.617+0000 7f5a39a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5a340cddc0 con 0x7f5a340cd020 2024-04-23T14:43:38.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.617+0000 7f5a39a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5a340cd800 con 0x7f5a341454e0 2024-04-23T14:43:38.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.617+0000 7f5a33400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5a340b3e40 0x7f5a34160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47714/0 (socket says 172.21.15.59:47714) 2024-04-23T14:43:38.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.617+0000 7f5a33400640 1 -- 172.21.15.59:0/2854065581 learned_addr learned my addr 172.21.15.59:0/2854065581 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:38.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a2be00640 1 --2- 172.21.15.59:0/2854065581 >> v2:172.21.15.76:3300/0 conn(0x7f5a340cd020 0x7f5a340b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32a00640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a341454e0 0x7f5a34145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004262396 0 0) 0x7f5a340cd520 con 0x7f5a340b3e40 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a18003520 con 0x7f5a340b3e40 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1684792901 0 0) 0x7f5a18003520 con 0x7f5a340b3e40 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a341454e0 msgr2=0x7f5a34145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a341454e0 0x7f5a34145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 >> v2:172.21.15.76:3300/0 conn(0x7f5a340cd020 msgr2=0x7f5a340b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 --2- 172.21.15.59:0/2854065581 >> v2:172.21.15.76:3300/0 conn(0x7f5a340cd020 0x7f5a340b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a340cdae0 con 0x7f5a340b3e40 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f5a2c0019f0 con 0x7f5a340b3e40 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 >> v1:172.21.15.59:6789/0 conn(0x7f5a340b3e40 legacy=0x7f5a34160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a341454e0 0x7f5a180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18003f30 0x7f5a18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18004d40 0x7f5a180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a340cddc0 con 0x7f5a18003f30 2024-04-23T14:43:38.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5a340cd800 con 0x7f5a18004d40 2024-04-23T14:43:38.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.618+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1800ba10 con 0x7f5a341454e0 2024-04-23T14:43:38.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.619+0000 7f5a2be00640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18003f30 0x7f5a18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.619+0000 7f5a2be00640 1 -- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a341454e0 msgr2=0x7f5a180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:38.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.619+0000 7f5a2be00640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a341454e0 0x7f5a180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.619+0000 7f5a2be00640 1 -- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18004d40 msgr2=0x7f5a180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:38.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.619+0000 7f5a2be00640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18004d40 0x7f5a180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.619+0000 7f5a2be00640 1 -- 172.21.15.59:0/2854065581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a18003520 con 0x7f5a18003f30 2024-04-23T14:43:38.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.620+0000 7f5a2be00640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18003f30 0x7f5a18004800 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009af0 tx=0x7f5a18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a267a227fc2d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:38.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.620+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c036070 con 0x7f5a18003f30 2024-04-23T14:43:38.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.620+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a1c031400 con 0x7f5a18003f30 2024-04-23T14:43:38.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.620+0000 7f5a32000640 1 -- 172.21.15.59:0/2854065581 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c009690 con 0x7f5a18003f30 2024-04-23T14:43:38.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.621+0000 7f5a39a00640 1 -- 172.21.15.59:0/2854065581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18003f30 msgr2=0x7f5a18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.621+0000 7f5a39a00640 1 --2- 172.21.15.59:0/2854065581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18003f30 0x7f5a18004800 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009af0 tx=0x7f5a18005830 comp rx=0 tx=0).stop 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.621+0000 7f5a33400640 1 -- 172.21.15.59:0/2854065581 reap_dead start 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.621+0000 7f5a39a00640 1 -- 172.21.15.59:0/2854065581 shutdown_connections 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.621+0000 7f5a39a00640 1 -- 172.21.15.59:0/2854065581 >> 172.21.15.59:0/2854065581 conn(0x7f5a34067ea0 msgr2=0x7f5a3406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.621+0000 7f5a39a00640 1 -- 172.21.15.59:0/2854065581 shutdown_connections 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.621+0000 7f5a39a00640 1 -- 172.21.15.59:0/2854065581 wait complete. 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.623+0000 7f5a39a00640 1 Processor -- start 2024-04-23T14:43:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.623+0000 7f5a39a00640 1 -- start start 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.624+0000 7f5a39a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18003f30 0x7f5a340b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.624+0000 7f5a39a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a3415f6d0 0x7f5a3415db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.624+0000 7f5a39a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a3415e0b0 0x7f5a3415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.624+0000 7f5a39a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a341611a0 con 0x7f5a3415e0b0 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.624+0000 7f5a39a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34161020 con 0x7f5a18003f30 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.624+0000 7f5a39a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5a34145bb0 con 0x7f5a3415f6d0 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a3415e0b0 0x7f5a3415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a3415e0b0 0x7f5a3415ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33914/0 (socket says 172.21.15.59:33914) 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 -- 172.21.15.59:0/4026184843 learned_addr learned my addr 172.21.15.59:0/4026184843 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a32a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18003f30 0x7f5a340b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a2be00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a3415f6d0 0x7f5a3415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 -- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a3415f6d0 msgr2=0x7f5a3415db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a3415f6d0 0x7f5a3415db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 -- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18003f30 msgr2=0x7f5a340b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18003f30 0x7f5a340b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 -- 172.21.15.59:0/4026184843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3415e630 con 0x7f5a3415e0b0 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a32a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18003f30 0x7f5a340b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.625+0000 7f5a33400640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a3415e0b0 0x7f5a3415ba90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0012e0 tx=0x7f5a2c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.626+0000 7f5a2aa00640 1 -- 172.21.15.59:0/4026184843 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00a610 con 0x7f5a3415e0b0 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.626+0000 7f5a2aa00640 1 -- 172.21.15.59:0/4026184843 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a2c00a7b0 con 0x7f5a3415e0b0 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.626+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3415e910 con 0x7f5a3415e0b0 2024-04-23T14:43:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.626+0000 7f5a2aa00640 1 -- 172.21.15.59:0/4026184843 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00aba0 con 0x7f5a3415e0b0 2024-04-23T14:43:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.626+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3415eed0 con 0x7f5a3415e0b0 2024-04-23T14:43:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.627+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a34068a80 con 0x7f5a3415e0b0 2024-04-23T14:43:38.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.628+0000 7f5a2aa00640 1 -- 172.21.15.59:0/4026184843 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a2c011070 con 0x7f5a3415e0b0 2024-04-23T14:43:38.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.629+0000 7f5a2aa00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a1006c920 0x7f5a1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.629+0000 7f5a2aa00640 1 -- 172.21.15.59:0/4026184843 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f5a2c08b680 con 0x7f5a3415e0b0 2024-04-23T14:43:38.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.630+0000 7f5a32a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a1006c920 0x7f5a1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.630+0000 7f5a32a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a1006c920 0x7f5a1006ede0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5a240047b0 tx=0x7f5a24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.634+0000 7f5a2aa00640 1 -- 172.21.15.59:0/4026184843 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a2c01a060 con 0x7f5a3415e0b0 2024-04-23T14:43:38.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.833+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a34160030 con 0x7f5a3415e0b0 2024-04-23T14:43:38.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.836+0000 7f5a2aa00640 1 -- 172.21.15.59:0/4026184843 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16580 (secure 0 0 0) 0x7f5a2c059a10 con 0x7f5a3415e0b0 2024-04-23T14:43:38.837 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:38.838 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":111,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:37.939581+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a1006c920 msgr2=0x7f5a1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a1006c920 0x7f5a1006ede0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5a240047b0 tx=0x7f5a24008040 comp rx=0 tx=0).stop 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a3415e0b0 msgr2=0x7f5a3415ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a3415e0b0 0x7f5a3415ba90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0012e0 tx=0x7f5a2c00cf00 comp rx=0 tx=0).stop 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 shutdown_connections 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a1006c920 0x7f5a1006ede0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a3415f6d0 0x7f5a3415db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a3415e0b0 0x7f5a3415ba90 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 --2- 172.21.15.59:0/4026184843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a18003f30 0x7f5a340b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 >> 172.21.15.59:0/4026184843 conn(0x7f5a34067ea0 msgr2=0x7f5a3413fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 shutdown_connections 2024-04-23T14:43:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:38.838+0000 7f5a39a00640 1 -- 172.21.15.59:0/4026184843 wait complete. 2024-04-23T14:43:38.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:39.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.066+0000 7f9953400640 1 Processor -- start 2024-04-23T14:43:39.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.066+0000 7f9953400640 1 -- start start 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.067+0000 7f9953400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c15eb60 0x7f994c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.067+0000 7f9953400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f994c0b99a0 0x7f994c0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.067+0000 7f9953400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994c0ba1e0 con 0x7f994c0d2d60 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9953400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f994c0ba7a0 con 0x7f994c0b99a0 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9953400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0ba4c0 con 0x7f994c15eb60 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9951400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f994c0b99a0 0x7f994c0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f994be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f994c0d2d60 0x7f994c0b9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:47716/0 (socket says 172.21.15.59:47716) 2024-04-23T14:43:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f994be00640 1 -- 172.21.15.59:0/1991273979 learned_addr learned my addr 172.21.15.59:0/1991273979 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9950a00640 1 --2- 172.21.15.59:0/1991273979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c15eb60 0x7f994c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9951400640 1 -- 172.21.15.59:0/1991273979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c15eb60 msgr2=0x7f994c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9951400640 1 --2- 172.21.15.59:0/1991273979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c15eb60 0x7f994c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9951400640 1 -- 172.21.15.59:0/1991273979 >> v1:172.21.15.59:6789/0 conn(0x7f994c0d2d60 legacy=0x7f994c0b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.068+0000 7f9951400640 1 -- 172.21.15.59:0/1991273979 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c0b8a90 con 0x7f994c0b99a0 2024-04-23T14:43:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.069+0000 7f9951400640 1 --2- 172.21.15.59:0/1991273979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0b77e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f994402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed62373de0cf997 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f994b400640 1 -- 172.21.15.59:0/1991273979 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944007740 con 0x7f994c0b99a0 2024-04-23T14:43:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f994b400640 1 -- 172.21.15.59:0/1991273979 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99440078e0 con 0x7f994c0b99a0 2024-04-23T14:43:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f994b400640 1 -- 172.21.15.59:0/1991273979 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944007cb0 con 0x7f994c0b99a0 2024-04-23T14:43:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f9953400640 1 -- 172.21.15.59:0/1991273979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 msgr2=0x7f994c0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f9953400640 1 --2- 172.21.15.59:0/1991273979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0b77e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d90 tx=0x7f994402a660 comp rx=0 tx=0).stop 2024-04-23T14:43:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f9953400640 1 -- 172.21.15.59:0/1991273979 shutdown_connections 2024-04-23T14:43:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f9953400640 1 --2- 172.21.15.59:0/1991273979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0b77e0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f9953400640 1 --2- 172.21.15.59:0/1991273979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c15eb60 0x7f994c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.070+0000 7f9953400640 1 -- 172.21.15.59:0/1991273979 >> 172.21.15.59:0/1991273979 conn(0x7f994c067b50 msgr2=0x7f994c067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.071+0000 7f9953400640 1 -- 172.21.15.59:0/1991273979 shutdown_connections 2024-04-23T14:43:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.071+0000 7f9953400640 1 -- 172.21.15.59:0/1991273979 wait complete. 2024-04-23T14:43:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.072+0000 7f9953400640 1 Processor -- start 2024-04-23T14:43:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.072+0000 7f9953400640 1 -- start start 2024-04-23T14:43:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.073+0000 7f9953400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.073+0000 7f9953400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f994c15eb60 0x7f994c0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.073+0000 7f9953400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c0b6a20 0x7f994c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.073+0000 7f9953400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0b3490 con 0x7f994c0b99a0 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.073+0000 7f9953400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f994c0b3c80 con 0x7f994c15eb60 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.073+0000 7f9953400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f994c0b3610 con 0x7f994c0b6a20 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.073+0000 7f9950a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33932/0 (socket says 172.21.15.59:33932) 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 -- 172.21.15.59:0/848031615 learned_addr learned my addr 172.21.15.59:0/848031615 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f994be00640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f994c15eb60 0x7f994c0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 -- 172.21.15.59:0/848031615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c0b6a20 msgr2=0x7f994c157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c0b6a20 0x7f994c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 -- 172.21.15.59:0/848031615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f994c15eb60 msgr2=0x7f994c0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f994c15eb60 0x7f994c0b64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 -- 172.21.15.59:0/848031615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c0b6fa0 con 0x7f994c0b99a0 2024-04-23T14:43:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.074+0000 7f9950a00640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0cac80 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d5f0 tx=0x7f993c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.075+0000 7f9948c00640 1 -- 172.21.15.59:0/848031615 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0125f0 con 0x7f994c0b99a0 2024-04-23T14:43:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.075+0000 7f9948c00640 1 -- 172.21.15.59:0/848031615 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c019070 con 0x7f994c0b99a0 2024-04-23T14:43:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.075+0000 7f9948c00640 1 -- 172.21.15.59:0/848031615 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0129d0 con 0x7f994c0b99a0 2024-04-23T14:43:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.075+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c0c8580 con 0x7f994c0b99a0 2024-04-23T14:43:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.075+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1530e0 con 0x7f994c0b99a0 2024-04-23T14:43:39.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.076+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994c068a80 con 0x7f994c0b99a0 2024-04-23T14:43:39.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.077+0000 7f9948c00640 1 -- 172.21.15.59:0/848031615 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f993c012b70 con 0x7f994c0b99a0 2024-04-23T14:43:39.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.078+0000 7f9948c00640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f992c06c8f0 0x7f992c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.078+0000 7f9948c00640 1 -- 172.21.15.59:0/848031615 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f993c08bfd0 con 0x7f994c0b99a0 2024-04-23T14:43:39.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.078+0000 7f994be00640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f992c06c8f0 0x7f992c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.079+0000 7f994be00640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f992c06c8f0 0x7f992c06edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f994c0c7cb0 tx=0x7f9938009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:39.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.082+0000 7f9948c00640 1 -- 172.21.15.59:0/848031615 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f993c05a360 con 0x7f994c0b99a0 2024-04-23T14:43:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.287+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f994c162200 con 0x7f994c0b99a0 2024-04-23T14:43:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.290+0000 7f9948c00640 1 -- 172.21.15.59:0/848031615 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16580 (secure 0 0 0) 0x7f993c059c60 con 0x7f994c0b99a0 2024-04-23T14:43:39.291 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:39.291 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":111,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:37.939581+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f992c06c8f0 msgr2=0x7f992c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f992c06c8f0 0x7f992c06edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f994c0c7cb0 tx=0x7f9938009040 comp rx=0 tx=0).stop 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 msgr2=0x7f994c0cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0cac80 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d5f0 tx=0x7f993c00dab0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 shutdown_connections 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f992c06c8f0 0x7f992c06edb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f994c0b6a20 0x7f994c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f994c15eb60 0x7f994c0b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 --2- 172.21.15.59:0/848031615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f994c0b99a0 0x7f994c0cac80 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.292+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 >> 172.21.15.59:0/848031615 conn(0x7f994c067b50 msgr2=0x7f994c159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:39.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.293+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 shutdown_connections 2024-04-23T14:43:39.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.293+0000 7f9953400640 1 -- 172.21.15.59:0/848031615 wait complete. 2024-04-23T14:43:39.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:43:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.530+0000 7f1095800640 1 Processor -- start 2024-04-23T14:43:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.530+0000 7f1095800640 1 -- start start 2024-04-23T14:43:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.531+0000 7f1095800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.531+0000 7f1095800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1090160180 0x7f10900d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.532+0000 7f1095800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10900cff50 con 0x7f109015cb20 2024-04-23T14:43:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.532+0000 7f1095800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f109015cf00 con 0x7f1090160180 2024-04-23T14:43:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.532+0000 7f1095800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10900d01c0 con 0x7f1090163770 2024-04-23T14:43:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.532+0000 7f108f400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1090160180 0x7f10900d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.532+0000 7f1087e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.533+0000 7f1087e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49236/0 (socket says 172.21.15.59:49236) 2024-04-23T14:43:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.533+0000 7f1087e00640 1 -- 172.21.15.59:0/3007065065 learned_addr learned my addr 172.21.15.59:0/3007065065 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.533+0000 7f1087e00640 1 -- 172.21.15.59:0/3007065065 >> v2:172.21.15.76:3300/0 conn(0x7f1090160180 msgr2=0x7f10900d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.533+0000 7f1087e00640 1 --2- 172.21.15.59:0/3007065065 >> v2:172.21.15.76:3300/0 conn(0x7f1090160180 0x7f10900d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.533+0000 7f1087e00640 1 -- 172.21.15.59:0/3007065065 >> v1:172.21.15.76:6789/0 conn(0x7f109015cb20 legacy=0x7f10900cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.533+0000 7f1087e00640 1 -- 172.21.15.59:0/3007065065 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109015f6a0 con 0x7f1090163770 2024-04-23T14:43:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.535+0000 7f1087e00640 1 --2- 172.21.15.59:0/3007065065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900d1240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1078009870 tx=0x7f107802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c4bd7183d28c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.535+0000 7f108e000640 1 -- 172.21.15.59:0/3007065065 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078037070 con 0x7f1090163770 2024-04-23T14:43:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.535+0000 7f108e000640 1 -- 172.21.15.59:0/3007065065 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1078032440 con 0x7f1090163770 2024-04-23T14:43:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.535+0000 7f108e000640 1 -- 172.21.15.59:0/3007065065 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078032810 con 0x7f1090163770 2024-04-23T14:43:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 -- 172.21.15.59:0/3007065065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 msgr2=0x7f10900d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 --2- 172.21.15.59:0/3007065065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900d1240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1078009870 tx=0x7f107802a1c0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 -- 172.21.15.59:0/3007065065 shutdown_connections 2024-04-23T14:43:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 --2- 172.21.15.59:0/3007065065 >> v2:172.21.15.76:3300/0 conn(0x7f1090160180 0x7f10900d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 --2- 172.21.15.59:0/3007065065 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900d1240 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 -- 172.21.15.59:0/3007065065 >> 172.21.15.59:0/3007065065 conn(0x7f109006cc70 msgr2=0x7f109006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 -- 172.21.15.59:0/3007065065 shutdown_connections 2024-04-23T14:43:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.536+0000 7f1095800640 1 -- 172.21.15.59:0/3007065065 wait complete. 2024-04-23T14:43:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.538+0000 7f1095800640 1 Processor -- start 2024-04-23T14:43:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.538+0000 7f1095800640 1 -- start start 2024-04-23T14:43:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f1095800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109015cb20 0x7f10900cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f1095800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1090160180 0x7f10900ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f1095800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f1095800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10900d2d60 con 0x7f109015cb20 2024-04-23T14:43:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f1095800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10900d2be0 con 0x7f1090160180 2024-04-23T14:43:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f1095800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f10900d2ee0 con 0x7f1090163770 2024-04-23T14:43:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f108ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109015cb20 0x7f10900cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f108ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109015cb20 0x7f10900cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33948/0 (socket says 172.21.15.59:33948) 2024-04-23T14:43:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.539+0000 7f108ea00640 1 -- 172.21.15.59:0/4271548978 learned_addr learned my addr 172.21.15.59:0/4271548978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f1087e00640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1090160180 0x7f10900ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f108ea00640 1 -- 172.21.15.59:0/4271548978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 msgr2=0x7f10900cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f108ea00640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f108ea00640 1 -- 172.21.15.59:0/4271548978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1090160180 msgr2=0x7f10900ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f108ea00640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1090160180 0x7f10900ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f108ea00640 1 -- 172.21.15.59:0/4271548978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10900cba30 con 0x7f109015cb20 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f108ea00640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109015cb20 0x7f10900cb220 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f108000c400 tx=0x7f108000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f1086a00640 1 -- 172.21.15.59:0/4271548978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800052d0 con 0x7f109015cb20 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f1086a00640 1 -- 172.21.15.59:0/4271548978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1080005490 con 0x7f109015cb20 2024-04-23T14:43:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10900cbd10 con 0x7f109015cb20 2024-04-23T14:43:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f1086a00640 1 -- 172.21.15.59:0/4271548978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080007260 con 0x7f109015cb20 2024-04-23T14:43:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.540+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10900cc2d0 con 0x7f109015cb20 2024-04-23T14:43:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.541+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1090068a80 con 0x7f109015cb20 2024-04-23T14:43:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.542+0000 7f1086a00640 1 -- 172.21.15.59:0/4271548978 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10800074c0 con 0x7f109015cb20 2024-04-23T14:43:39.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.543+0000 7f1086a00640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.544+0000 7f1086a00640 1 -- 172.21.15.59:0/4271548978 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f108008b080 con 0x7f109015cb20 2024-04-23T14:43:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.544+0000 7f1087e00640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.544+0000 7f1087e00640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1078034000 tx=0x7f107802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:39.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.547+0000 7f1086a00640 1 -- 172.21.15.59:0/4271548978 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10800594a0 con 0x7f109015cb20 2024-04-23T14:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:39.793+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f10900d29c0 con 0x7f109015cb20 2024-04-23T14:43:40.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.437+0000 7f1086a00640 1 -- 172.21.15.59:0/4271548978 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) ==== 118+0+0 (secure 0 0 0) 0x7f1080058da0 con 0x7f109015cb20 2024-04-23T14:43:40.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f106c06c8f0 msgr2=0x7f106c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:40.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1078034000 tx=0x7f107802e000 comp rx=0 tx=0).stop 2024-04-23T14:43:40.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109015cb20 msgr2=0x7f10900cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:40.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109015cb20 0x7f10900cb220 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f108000c400 tx=0x7f108000c8c0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 shutdown_connections 2024-04-23T14:43:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1090163770 0x7f10900cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1090160180 0x7f10900ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.440+0000 7f1095800640 1 --2- 172.21.15.59:0/4271548978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109015cb20 0x7f10900cb220 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.441+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 >> 172.21.15.59:0/4271548978 conn(0x7f109006cc70 msgr2=0x7f109015e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.441+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 shutdown_connections 2024-04-23T14:43:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.441+0000 7f1095800640 1 -- 172.21.15.59:0/4271548978 wait complete. 2024-04-23T14:43:40.468 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-23T14:43:40.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-23T14:43:40.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.681+0000 7f4109a00640 1 Processor -- start 2024-04-23T14:43:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.681+0000 7f4109a00640 1 -- start start 2024-04-23T14:43:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.683+0000 7f4109a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41040d15b0 0x7f41040cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.683+0000 7f4109a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040b5da0 con 0x7f41040b4f20 2024-04-23T14:43:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.683+0000 7f4109a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040d0a20 con 0x7f410415ed20 2024-04-23T14:43:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.683+0000 7f4109a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f410415f100 con 0x7f41040d15b0 2024-04-23T14:43:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.683+0000 7f4103400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41040d15b0 0x7f41040cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.683+0000 7f4103400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41040d15b0 0x7f41040cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55830/0 (socket says 172.21.15.59:55830) 2024-04-23T14:43:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.684+0000 7f4103400640 1 -- 172.21.15.59:0/2463339717 learned_addr learned my addr 172.21.15.59:0/2463339717 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.683+0000 7f4109000640 1 --1- 172.21.15.59:0/2463339717 >> v1:172.21.15.59:6789/0 conn(0x7f41040b4f20 0x7f41040cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40908/0 (socket says 172.21.15.59:40908) 2024-04-23T14:43:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.684+0000 7f4103400640 1 -- 172.21.15.59:0/2463339717 >> v1:172.21.15.76:6789/0 conn(0x7f410415ed20 legacy=0x7f41040d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:40.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.684+0000 7f4103400640 1 -- 172.21.15.59:0/2463339717 >> v1:172.21.15.59:6789/0 conn(0x7f41040b4f20 legacy=0x7f41040cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:40.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.684+0000 7f4103400640 1 -- 172.21.15.59:0/2463339717 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040d2ed0 con 0x7f41040d15b0 2024-04-23T14:43:40.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.685+0000 7f4103400640 1 --2- 172.21.15.59:0/2463339717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41040d15b0 0x7f41040cf280 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009870 tx=0x7f40ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb63d0a31ea17bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:40.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.685+0000 7f4102a00640 1 -- 172.21.15.59:0/2463339717 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec037070 con 0x7f41040d15b0 2024-04-23T14:43:40.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.685+0000 7f4102a00640 1 -- 172.21.15.59:0/2463339717 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ec032440 con 0x7f41040d15b0 2024-04-23T14:43:40.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4102a00640 1 -- 172.21.15.59:0/2463339717 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec032810 con 0x7f41040d15b0 2024-04-23T14:43:40.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4109a00640 1 -- 172.21.15.59:0/2463339717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41040d15b0 msgr2=0x7f41040cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:40.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4109a00640 1 --2- 172.21.15.59:0/2463339717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41040d15b0 0x7f41040cf280 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009870 tx=0x7f40ec02a180 comp rx=0 tx=0).stop 2024-04-23T14:43:40.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4109a00640 1 -- 172.21.15.59:0/2463339717 shutdown_connections 2024-04-23T14:43:40.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4109a00640 1 --2- 172.21.15.59:0/2463339717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41040d15b0 0x7f41040cf280 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4109a00640 1 -- 172.21.15.59:0/2463339717 >> 172.21.15.59:0/2463339717 conn(0x7f410408bf90 msgr2=0x7f41040d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:40.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4109a00640 1 -- 172.21.15.59:0/2463339717 shutdown_connections 2024-04-23T14:43:40.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.686+0000 7f4109a00640 1 -- 172.21.15.59:0/2463339717 wait complete. 2024-04-23T14:43:40.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.688+0000 7f4109a00640 1 Processor -- start 2024-04-23T14:43:40.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.688+0000 7f4109a00640 1 -- start start 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.689+0000 7f4109a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41040b4f20 0x7f410415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.689+0000 7f4109a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41040d15b0 0x7f41041f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.689+0000 7f4109a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f410415ed20 0x7f41041f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.689+0000 7f4109a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f41040b3600 con 0x7f410415ed20 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.689+0000 7f4109a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f41040d3490 con 0x7f41040d15b0 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.689+0000 7f4109a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f41040b3780 con 0x7f41040b4f20 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f410415ed20 0x7f41041f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f410415ed20 0x7f41041f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55842/0 (socket says 172.21.15.59:55842) 2024-04-23T14:43:40.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 -- 172.21.15.59:0/2090199811 learned_addr learned my addr 172.21.15.59:0/2090199811 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4103400640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41040d15b0 0x7f41041f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4103e00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41040b4f20 0x7f410415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 -- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41040b4f20 msgr2=0x7f410415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41040b4f20 0x7f410415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 -- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41040d15b0 msgr2=0x7f41041f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41040d15b0 0x7f41041f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4109000640 1 -- 172.21.15.59:0/2090199811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410415baa0 con 0x7f410415ed20 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4103e00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41040b4f20 0x7f410415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.690+0000 7f4103400640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41040d15b0 0x7f41041f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.691+0000 7f4109000640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f410415ed20 0x7f41041f4e70 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00c430 tx=0x7f40fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.691+0000 7f40fbe00640 1 -- 172.21.15.59:0/2090199811 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0074e0 con 0x7f410415ed20 2024-04-23T14:43:40.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.691+0000 7f40fbe00640 1 -- 172.21.15.59:0/2090199811 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc007680 con 0x7f410415ed20 2024-04-23T14:43:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.691+0000 7f40fbe00640 1 -- 172.21.15.59:0/2090199811 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0049e0 con 0x7f410415ed20 2024-04-23T14:43:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.691+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41041f8450 con 0x7f410415ed20 2024-04-23T14:43:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.691+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41041f8a10 con 0x7f410415ed20 2024-04-23T14:43:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.693+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410404fd90 con 0x7f410415ed20 2024-04-23T14:43:40.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.694+0000 7f40fbe00640 1 -- 172.21.15.59:0/2090199811 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40fc01a020 con 0x7f410415ed20 2024-04-23T14:43:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.695+0000 7f40fbe00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f40e006c8f0 0x7f40e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.695+0000 7f40fbe00640 1 -- 172.21.15.59:0/2090199811 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f40fc08b260 con 0x7f410415ed20 2024-04-23T14:43:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.695+0000 7f4103e00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f40e006c8f0 0x7f40e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:40.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.695+0000 7f4103e00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f40e006c8f0 0x7f40e006edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f41040b2e30 tx=0x7f40f40073c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:40.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.699+0000 7f40fbe00640 1 -- 172.21.15.59:0/2090199811 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f40fc010300 con 0x7f410415ed20 2024-04-23T14:43:40.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:40.945+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f410406ba80 con 0x7f410415ed20 2024-04-23T14:43:41.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.447+0000 7f40fbe00640 1 -- 172.21.15.59:0/2090199811 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7f40fc059720 con 0x7f410415ed20 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f40e006c8f0 msgr2=0x7f40e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f40e006c8f0 0x7f40e006edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f41040b2e30 tx=0x7f40f40073c0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f410415ed20 msgr2=0x7f41041f4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f410415ed20 0x7f41041f4e70 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00c430 tx=0x7f40fc00c8f0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 shutdown_connections 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f40e006c8f0 0x7f40e006edb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f410415ed20 0x7f41041f4e70 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41040d15b0 0x7f41041f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 --2- 172.21.15.59:0/2090199811 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f41040b4f20 0x7f410415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 >> 172.21.15.59:0/2090199811 conn(0x7f410408bf90 msgr2=0x7f41040cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.451+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 shutdown_connections 2024-04-23T14:43:41.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.452+0000 7f4109a00640 1 -- 172.21.15.59:0/2090199811 wait complete. 2024-04-23T14:43:41.477 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-23T14:43:41.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:43:41.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.683+0000 7fdbbb200640 1 Processor -- start 2024-04-23T14:43:41.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.683+0000 7fdbbb200640 1 -- start start 2024-04-23T14:43:41.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.685+0000 7fdbbb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdbbc15c3d0 0x7fdbbc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:41.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.685+0000 7fdbbb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbbc160210 con 0x7fdbbc0b4860 2024-04-23T14:43:41.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.685+0000 7fdbbb200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbbc1604f0 con 0x7fdbbc0cd590 2024-04-23T14:43:41.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.685+0000 7fdbbb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdbbc1607d0 con 0x7fdbbc15c3d0 2024-04-23T14:43:41.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.685+0000 7fdbba800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdbbc0b4860 0x7fdbbc0cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40912/0 (socket says 172.21.15.59:40912) 2024-04-23T14:43:41.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.685+0000 7fdbba800640 1 -- 172.21.15.59:0/2010924278 learned_addr learned my addr 172.21.15.59:0/2010924278 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:41.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.685+0000 7fdbb9e00640 1 --2- 172.21.15.59:0/2010924278 >> v2:172.21.15.59:3300/0 conn(0x7fdbbc15c3d0 0x7fdbbc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.686+0000 7fdbb8a00640 1 -- 172.21.15.59:0/2010924278 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559354732 0 0) 0x7fdbbc160210 con 0x7fdbbc0b4860 2024-04-23T14:43:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.686+0000 7fdbb8a00640 1 -- 172.21.15.59:0/2010924278 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba0003520 con 0x7fdbbc0b4860 2024-04-23T14:43:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.686+0000 7fdbb9e00640 1 -- 172.21.15.59:0/2010924278 >> v1:172.21.15.76:6789/0 conn(0x7fdbbc0cd590 legacy=0x7fdbbc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.686+0000 7fdbb9e00640 1 -- 172.21.15.59:0/2010924278 >> v1:172.21.15.59:6789/0 conn(0x7fdbbc0b4860 legacy=0x7fdbbc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.686+0000 7fdbb8a00640 1 -- 172.21.15.59:0/2010924278 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1878094418 0 0) 0x7fdba0003520 con 0x7fdbbc0b4860 2024-04-23T14:43:41.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.686+0000 7fdbb9e00640 1 -- 172.21.15.59:0/2010924278 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc160ab0 con 0x7fdbbc15c3d0 2024-04-23T14:43:41.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.687+0000 7fdbb9e00640 1 --2- 172.21.15.59:0/2010924278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc15c3d0 0x7fdbbc0cd050 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0097a0 tx=0x7fdbac02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd873c6dbd073585 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:41.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.688+0000 7fdbb8a00640 1 -- 172.21.15.59:0/2010924278 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac037070 con 0x7fdbbc15c3d0 2024-04-23T14:43:41.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.688+0000 7fdbb8a00640 1 -- 172.21.15.59:0/2010924278 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac02ac50 con 0x7fdbbc15c3d0 2024-04-23T14:43:41.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.688+0000 7fdbb8a00640 1 -- 172.21.15.59:0/2010924278 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb40019f0 con 0x7fdbbc15c3d0 2024-04-23T14:43:41.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.688+0000 7fdbbb200640 1 -- 172.21.15.59:0/2010924278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc15c3d0 msgr2=0x7fdbbc0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.688+0000 7fdbbb200640 1 --2- 172.21.15.59:0/2010924278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc15c3d0 0x7fdbbc0cd050 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0097a0 tx=0x7fdbac02a1c0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.689+0000 7fdbbb200640 1 -- 172.21.15.59:0/2010924278 shutdown_connections 2024-04-23T14:43:41.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.689+0000 7fdbbb200640 1 --2- 172.21.15.59:0/2010924278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc15c3d0 0x7fdbbc0cd050 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.689+0000 7fdbbb200640 1 -- 172.21.15.59:0/2010924278 >> 172.21.15.59:0/2010924278 conn(0x7fdbbc0b21a0 msgr2=0x7fdbbc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:41.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.689+0000 7fdbbb200640 1 -- 172.21.15.59:0/2010924278 shutdown_connections 2024-04-23T14:43:41.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.689+0000 7fdbbb200640 1 -- 172.21.15.59:0/2010924278 wait complete. 2024-04-23T14:43:41.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.691+0000 7fdbbb200640 1 Processor -- start 2024-04-23T14:43:41.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.691+0000 7fdbbb200640 1 -- start start 2024-04-23T14:43:41.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbbb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0b4860 0x7fdbbc0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:41.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbbb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbbc0cd590 0x7fdbbc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:41.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbbb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbbc15c3d0 0x7fdbbc15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:41.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbbb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc15c930 con 0x7fdbbc0b4860 2024-04-23T14:43:41.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbbb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc15c7b0 con 0x7fdbbc0cd590 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbbb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdbbc0cbee0 con 0x7fdbbc15c3d0 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0b4860 0x7fdbbc0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0b4860 0x7fdbbc0c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55866/0 (socket says 172.21.15.59:55866) 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 -- 172.21.15.59:0/1667262623 learned_addr learned my addr 172.21.15.59:0/1667262623 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 -- 172.21.15.59:0/1667262623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbbc15c3d0 msgr2=0x7fdbbc15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbbc15c3d0 0x7fdbbc15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 -- 172.21.15.59:0/1667262623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbbc0cd590 msgr2=0x7fdbbc0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbbc0cd590 0x7fdbbc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.692+0000 7fdbb9e00640 1 -- 172.21.15.59:0/1667262623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc0b1940 con 0x7fdbbc0b4860 2024-04-23T14:43:41.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.693+0000 7fdbb9e00640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0b4860 0x7fdbbc0c68f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0098c0 tx=0x7fdbac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:41.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.693+0000 7fdbb2000640 1 -- 172.21.15.59:0/1667262623 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac041070 con 0x7fdbbc0b4860 2024-04-23T14:43:41.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.693+0000 7fdbb2000640 1 -- 172.21.15.59:0/1667262623 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac03c460 con 0x7fdbbc0b4860 2024-04-23T14:43:41.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.693+0000 7fdbb2000640 1 -- 172.21.15.59:0/1667262623 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac037040 con 0x7fdbbc0b4860 2024-04-23T14:43:41.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.693+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc15d970 con 0x7fdbbc0b4860 2024-04-23T14:43:41.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.693+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc15df30 con 0x7fdbbc0b4860 2024-04-23T14:43:41.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.694+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc04fd90 con 0x7fdbbc0b4860 2024-04-23T14:43:41.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.695+0000 7fdbb2000640 1 -- 172.21.15.59:0/1667262623 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbac044070 con 0x7fdbbc0b4860 2024-04-23T14:43:41.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.696+0000 7fdbb2000640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb9806c8f0 0x7fdb9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:41.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.696+0000 7fdbb2000640 1 -- 172.21.15.59:0/1667262623 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fdbac0ac0b0 con 0x7fdbbc0b4860 2024-04-23T14:43:41.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.697+0000 7fdbb9400640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb9806c8f0 0x7fdb9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:41.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.698+0000 7fdbb9400640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb9806c8f0 0x7fdb9806edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdba4000f60 tx=0x7fdba40079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:41.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.701+0000 7fdbb2000640 1 -- 172.21.15.59:0/1667262623 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbac07a570 con 0x7fdbbc0b4860 2024-04-23T14:43:41.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.947+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbbc075070 con 0x7fdbbc0b4860 2024-04-23T14:43:41.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.948+0000 7fdbb2000640 1 -- 172.21.15.59:0/1667262623 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5185 (secure 0 0 0) 0x7fdbac079e70 con 0x7fdbbc0b4860 2024-04-23T14:43:41.950 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:41.950 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:43:40.947368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/1061483588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1061483588},{"type":"v1","addr":"172.21.15.59:6837","nonce":1061483588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9210,"qdb_cluster":[9210]},"id":8}]} 2024-04-23T14:43:41.950 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 100 2024-04-23T14:43:41.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb9806c8f0 msgr2=0x7fdb9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb9806c8f0 0x7fdb9806edb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdba4000f60 tx=0x7fdba40079d0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0b4860 msgr2=0x7fdbbc0c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0b4860 0x7fdbbc0c68f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0098c0 tx=0x7fdbac0046c0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 shutdown_connections 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdb9806c8f0 0x7fdb9806edb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdbbc15c3d0 0x7fdbbc15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbbc0cd590 0x7fdbbc0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 --2- 172.21.15.59:0/1667262623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0b4860 0x7fdbbc0c68f0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.951+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 >> 172.21.15.59:0/1667262623 conn(0x7fdbbc0b21a0 msgr2=0x7fdbbc0cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.952+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 shutdown_connections 2024-04-23T14:43:41.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:41.952+0000 7fdbbb200640 1 -- 172.21.15.59:0/1667262623 wait complete. 2024-04-23T14:43:41.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.173+0000 7fe4bb200640 1 Processor -- start 2024-04-23T14:43:42.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.173+0000 7fe4bb200640 1 -- start start 2024-04-23T14:43:42.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.174+0000 7fe4bb200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4b415c3d0 0x7fe4b40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.174+0000 7fe4bb200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe4b40b4860 0x7fe4b40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4bb200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b4160210 con 0x7fe4b40cd590 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4bb200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b41604f0 con 0x7fe4b415c3d0 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4bb200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b41607d0 con 0x7fe4b40b4860 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4b9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4b415c3d0 0x7fe4b40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4b9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4b415c3d0 0x7fe4b40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55876/0 (socket says 172.21.15.59:55876) 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4b9e00640 1 -- 172.21.15.59:0/1975564333 learned_addr learned my addr 172.21.15.59:0/1975564333 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4ba800640 1 --2- 172.21.15.59:0/1975564333 >> v2:172.21.15.76:3300/0 conn(0x7fe4b40b4860 0x7fe4b40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4b9e00640 1 -- 172.21.15.59:0/1975564333 >> v2:172.21.15.76:3300/0 conn(0x7fe4b40b4860 msgr2=0x7fe4b40cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4b9e00640 1 --2- 172.21.15.59:0/1975564333 >> v2:172.21.15.76:3300/0 conn(0x7fe4b40b4860 0x7fe4b40cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4b9e00640 1 -- 172.21.15.59:0/1975564333 >> v1:172.21.15.76:6789/0 conn(0x7fe4b40cd590 legacy=0x7fe4b40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:42.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.175+0000 7fe4b9e00640 1 -- 172.21.15.59:0/1975564333 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b4160ab0 con 0x7fe4b415c3d0 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.176+0000 7fe4b9e00640 1 --2- 172.21.15.59:0/1975564333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b415c3d0 0x7fe4b40cd050 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40b2db0 tx=0x7fe4ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c56cfc27da31a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4b8a00640 1 -- 172.21.15.59:0/1975564333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac036070 con 0x7fe4b415c3d0 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4b8a00640 1 -- 172.21.15.59:0/1975564333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac031600 con 0x7fe4b415c3d0 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4b8a00640 1 -- 172.21.15.59:0/1975564333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0319d0 con 0x7fe4b415c3d0 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4bb200640 1 -- 172.21.15.59:0/1975564333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b415c3d0 msgr2=0x7fe4b40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4bb200640 1 --2- 172.21.15.59:0/1975564333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b415c3d0 0x7fe4b40cd050 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40b2db0 tx=0x7fe4ac02a6f0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4bb200640 1 -- 172.21.15.59:0/1975564333 shutdown_connections 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4bb200640 1 --2- 172.21.15.59:0/1975564333 >> v2:172.21.15.76:3300/0 conn(0x7fe4b40b4860 0x7fe4b40cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4bb200640 1 --2- 172.21.15.59:0/1975564333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b415c3d0 0x7fe4b40cd050 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4bb200640 1 -- 172.21.15.59:0/1975564333 >> 172.21.15.59:0/1975564333 conn(0x7fe4b40b21a0 msgr2=0x7fe4b40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.177+0000 7fe4bb200640 1 -- 172.21.15.59:0/1975564333 shutdown_connections 2024-04-23T14:43:42.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.178+0000 7fe4bb200640 1 -- 172.21.15.59:0/1975564333 wait complete. 2024-04-23T14:43:42.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.179+0000 7fe4bb200640 1 Processor -- start 2024-04-23T14:43:42.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.179+0000 7fe4bb200640 1 -- start start 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.180+0000 7fe4bb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40b4860 0x7fe4b415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.180+0000 7fe4bb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4b40cd590 0x7fe4b40c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.180+0000 7fe4bb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4b415c3d0 0x7fe4b40c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.180+0000 7fe4bb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b415c930 con 0x7fe4b40b4860 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.180+0000 7fe4bb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b415c7b0 con 0x7fe4b40cd590 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.180+0000 7fe4bb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b40cbd10 con 0x7fe4b415c3d0 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.180+0000 7fe4b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40b4860 0x7fe4b415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40b4860 0x7fe4b415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55892/0 (socket says 172.21.15.59:55892) 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 -- 172.21.15.59:0/3811797235 learned_addr learned my addr 172.21.15.59:0/3811797235 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9400640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4b40cd590 0x7fe4b40c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4ba800640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4b415c3d0 0x7fe4b40c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 -- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4b415c3d0 msgr2=0x7fe4b40c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4b415c3d0 0x7fe4b40c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 -- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4b40cd590 msgr2=0x7fe4b40c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4b40cd590 0x7fe4b40c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 -- 172.21.15.59:0/3811797235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b40c7280 con 0x7fe4b40b4860 2024-04-23T14:43:42.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.181+0000 7fe4b9e00640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40b4860 0x7fe4b415db00 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac034040 tx=0x7fe4ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:42.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.182+0000 7fe4aa000640 1 -- 172.21.15.59:0/3811797235 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac036050 con 0x7fe4b40b4860 2024-04-23T14:43:42.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.182+0000 7fe4aa000640 1 -- 172.21.15.59:0/3811797235 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac042070 con 0x7fe4b40b4860 2024-04-23T14:43:42.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.182+0000 7fe4aa000640 1 -- 172.21.15.59:0/3811797235 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac009510 con 0x7fe4b40b4860 2024-04-23T14:43:42.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.182+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b40c7560 con 0x7fe4b40b4860 2024-04-23T14:43:42.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.182+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b415b230 con 0x7fe4b40b4860 2024-04-23T14:43:42.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.183+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b404fd90 con 0x7fe4b40b4860 2024-04-23T14:43:42.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.184+0000 7fe4aa000640 1 -- 172.21.15.59:0/3811797235 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4ac007c50 con 0x7fe4b40b4860 2024-04-23T14:43:42.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.185+0000 7fe4aa000640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe49806c920 0x7fe49806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.185+0000 7fe4aa000640 1 -- 172.21.15.59:0/3811797235 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fe4ac0aca60 con 0x7fe4b40b4860 2024-04-23T14:43:42.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.185+0000 7fe4b9400640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe49806c920 0x7fe49806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.186+0000 7fe4b9400640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe49806c920 0x7fe49806ede0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40c8af0 tx=0x7fe4a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:42.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.189+0000 7fe4aa000640 1 -- 172.21.15.59:0/3811797235 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4ac07ae90 con 0x7fe4b40b4860 2024-04-23T14:43:42.309 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:43:42.308+0000 7f586fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:43:42.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.392+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4b40c9810 con 0x7fe4b40b4860 2024-04-23T14:43:42.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.394+0000 7fe4aa000640 1 -- 172.21.15.59:0/3811797235 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16567 (secure 0 0 0) 0x7fe4ac07a790 con 0x7fe4b40b4860 2024-04-23T14:43:42.395 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:42.396 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":112,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:39.410742+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe49806c920 msgr2=0x7fe49806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe49806c920 0x7fe49806ede0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40c8af0 tx=0x7fe4a4009040 comp rx=0 tx=0).stop 2024-04-23T14:43:42.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40b4860 msgr2=0x7fe4b415db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40b4860 0x7fe4b415db00 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac034040 tx=0x7fe4ac004a40 comp rx=0 tx=0).stop 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 shutdown_connections 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe49806c920 0x7fe49806ede0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4b415c3d0 0x7fe4b40c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4b40cd590 0x7fe4b40c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 --2- 172.21.15.59:0/3811797235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40b4860 0x7fe4b415db00 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 >> 172.21.15.59:0/3811797235 conn(0x7fe4b40b21a0 msgr2=0x7fe4b40cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 shutdown_connections 2024-04-23T14:43:42.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.397+0000 7fe4bb200640 1 -- 172.21.15.59:0/3811797235 wait complete. 2024-04-23T14:43:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:43:42.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:43:42.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.629+0000 7f88f1000640 1 Processor -- start 2024-04-23T14:43:42.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.629+0000 7f88f1000640 1 -- start start 2024-04-23T14:43:42.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.631+0000 7f88f1000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f88ec1454e0 0x7f88ec145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.631+0000 7f88f1000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88ec0cd020 0x7f88ec0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.631+0000 7f88f1000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88ec0cd520 con 0x7f88ec0b3e40 2024-04-23T14:43:42.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.631+0000 7f88f1000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f88ec0cddc0 con 0x7f88ec1454e0 2024-04-23T14:43:42.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.631+0000 7f88f1000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f88ec0cd800 con 0x7f88ec0cd020 2024-04-23T14:43:42.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.631+0000 7f88e9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f88ec1454e0 0x7f88ec145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.631+0000 7f88e9e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f88ec1454e0 0x7f88ec145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55894/0 (socket says 172.21.15.59:55894) 2024-04-23T14:43:42.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.632+0000 7f88e9e00640 1 -- 172.21.15.59:0/1980444939 learned_addr learned my addr 172.21.15.59:0/1980444939 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:42.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.632+0000 7f88e9400640 1 --2- 172.21.15.59:0/1980444939 >> v2:172.21.15.76:3300/0 conn(0x7f88ec0cd020 0x7f88ec0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.632+0000 7f88e9e00640 1 -- 172.21.15.59:0/1980444939 >> v2:172.21.15.76:3300/0 conn(0x7f88ec0cd020 msgr2=0x7f88ec0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:42.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.632+0000 7f88e9e00640 1 --2- 172.21.15.59:0/1980444939 >> v2:172.21.15.76:3300/0 conn(0x7f88ec0cd020 0x7f88ec0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.632+0000 7f88e9e00640 1 -- 172.21.15.59:0/1980444939 >> v1:172.21.15.76:6789/0 conn(0x7f88ec0b3e40 legacy=0x7f88ec160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:42.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.632+0000 7f88e9e00640 1 -- 172.21.15.59:0/1980444939 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec0cdae0 con 0x7f88ec1454e0 2024-04-23T14:43:42.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.632+0000 7f88e9400640 1 --2- 172.21.15.59:0/1980444939 >> v2:172.21.15.76:3300/0 conn(0x7f88ec0cd020 0x7f88ec0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:42.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.633+0000 7f88e9e00640 1 --2- 172.21.15.59:0/1980444939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec1454e0 0x7f88ec145bb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0d1490 tx=0x7f88dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e584e41f5969ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:42.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88e8a00640 1 -- 172.21.15.59:0/1980444939 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc037070 con 0x7f88ec1454e0 2024-04-23T14:43:42.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88e8a00640 1 -- 172.21.15.59:0/1980444939 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc032440 con 0x7f88ec1454e0 2024-04-23T14:43:42.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88e8a00640 1 -- 172.21.15.59:0/1980444939 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc032810 con 0x7f88ec1454e0 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88f1000640 1 -- 172.21.15.59:0/1980444939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec1454e0 msgr2=0x7f88ec145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88f1000640 1 --2- 172.21.15.59:0/1980444939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec1454e0 0x7f88ec145bb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0d1490 tx=0x7f88dc02a1a0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88f1000640 1 -- 172.21.15.59:0/1980444939 shutdown_connections 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88f1000640 1 --2- 172.21.15.59:0/1980444939 >> v2:172.21.15.76:3300/0 conn(0x7f88ec0cd020 0x7f88ec0b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88f1000640 1 --2- 172.21.15.59:0/1980444939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec1454e0 0x7f88ec145bb0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88f1000640 1 -- 172.21.15.59:0/1980444939 >> 172.21.15.59:0/1980444939 conn(0x7f88ec067ea0 msgr2=0x7f88ec06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.634+0000 7f88f1000640 1 -- 172.21.15.59:0/1980444939 shutdown_connections 2024-04-23T14:43:42.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.635+0000 7f88f1000640 1 -- 172.21.15.59:0/1980444939 wait complete. 2024-04-23T14:43:42.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.636+0000 7f88f1000640 1 Processor -- start 2024-04-23T14:43:42.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.636+0000 7f88f1000640 1 -- start start 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88f1000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec0b3e40 0x7f88ec0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88f1000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88ec0cd020 0x7f88ec143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88f1000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88ec1454e0 0x7f88ec1404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88f1000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec146270 con 0x7f88ec0b3e40 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88f1000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1460f0 con 0x7f88ec0cd020 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88f1000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec161020 con 0x7f88ec1454e0 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec0b3e40 0x7f88ec0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.637+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec0b3e40 0x7f88ec0a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55904/0 (socket says 172.21.15.59:55904) 2024-04-23T14:43:42.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9e00640 1 -- 172.21.15.59:0/1908121035 learned_addr learned my addr 172.21.15.59:0/1908121035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:42.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9400640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88ec0cd020 0x7f88ec143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9e00640 1 -- 172.21.15.59:0/1908121035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88ec1454e0 msgr2=0x7f88ec1404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9e00640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88ec1454e0 0x7f88ec1404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9e00640 1 -- 172.21.15.59:0/1908121035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88ec0cd020 msgr2=0x7f88ec143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9e00640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88ec0cd020 0x7f88ec143b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9e00640 1 -- 172.21.15.59:0/1908121035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec0a9cb0 con 0x7f88ec0b3e40 2024-04-23T14:43:42.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e9e00640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec0b3e40 0x7f88ec0a9460 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f88dc000c00 tx=0x7f88dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:42.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.638+0000 7f88e2000640 1 -- 172.21.15.59:0/1908121035 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc041020 con 0x7f88ec0b3e40 2024-04-23T14:43:42.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.639+0000 7f88e2000640 1 -- 172.21.15.59:0/1908121035 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc030970 con 0x7f88ec0b3e40 2024-04-23T14:43:42.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.639+0000 7f88e2000640 1 -- 172.21.15.59:0/1908121035 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc037040 con 0x7f88ec0b3e40 2024-04-23T14:43:42.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.639+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec0a9f90 con 0x7f88ec0b3e40 2024-04-23T14:43:42.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.639+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec0aa550 con 0x7f88ec0b3e40 2024-04-23T14:43:42.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.640+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec0aa980 con 0x7f88ec0b3e40 2024-04-23T14:43:42.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.641+0000 7f88e2000640 1 -- 172.21.15.59:0/1908121035 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88dc044020 con 0x7f88ec0b3e40 2024-04-23T14:43:42.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.642+0000 7f88e2000640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88cc06c920 0x7f88cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:42.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.642+0000 7f88e2000640 1 -- 172.21.15.59:0/1908121035 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f88dc0ac5e0 con 0x7f88ec0b3e40 2024-04-23T14:43:42.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.642+0000 7f88e9400640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88cc06c920 0x7f88cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:42.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.643+0000 7f88e9400640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88cc06c920 0x7f88cc06ede0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f88ec144e60 tx=0x7f88d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:42.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.646+0000 7f88e2000640 1 -- 172.21.15.59:0/1908121035 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88dc04b030 con 0x7f88ec0b3e40 2024-04-23T14:43:42.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.852+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88ec160030 con 0x7f88cc06c920 2024-04-23T14:43:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.866+0000 7f88e2000640 1 -- 172.21.15.59:0/1908121035 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252436 (secure 0 0 0) 0x7f88ec160030 con 0x7f88cc06c920 2024-04-23T14:43:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88cc06c920 msgr2=0x7f88cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88cc06c920 0x7f88cc06ede0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f88ec144e60 tx=0x7f88d8008040 comp rx=0 tx=0).stop 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec0b3e40 msgr2=0x7f88ec0a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec0b3e40 0x7f88ec0a9460 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f88dc000c00 tx=0x7f88dc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 shutdown_connections 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f88cc06c920 0x7f88cc06ede0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f88ec1454e0 0x7f88ec1404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88ec0cd020 0x7f88ec143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 --2- 172.21.15.59:0/1908121035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88ec0b3e40 0x7f88ec0a9460 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.872+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 >> 172.21.15.59:0/1908121035 conn(0x7f88ec067ea0 msgr2=0x7f88ec0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.873+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 shutdown_connections 2024-04-23T14:43:42.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:42.873+0000 7f88f1000640 1 -- 172.21.15.59:0/1908121035 wait complete. 2024-04-23T14:43:42.894 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-04-23T14:43:42.616338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261476,"kb_used_data":12020,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713244,"statfs":{"total":773094113280,"available":772826361856,"internally_reserved":0,"allocated":12308480,"data_stored":5983424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001938"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433378+0000","last_change":"2024-04-23T14:43:38.425583+0000","last_active":"2024-04-23T14:43:38.433378+0000","last_peered":"2024-04-23T14:43:38.433378+0000","last_clean":"2024-04-23T14:43:38.433378+0000","last_became_active":"2024-04-23T14:43:38.425299+0000","last_became_peered":"2024-04-23T14:43:38.425299+0000","last_unstale":"2024-04-23T14:43:38.433378+0000","last_undegraded":"2024-04-23T14:43:38.433378+0000","last_fullsized":"2024-04-23T14:43:38.433378+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:16.911838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431681+0000","last_change":"2024-04-23T14:43:38.424369+0000","last_active":"2024-04-23T14:43:39.431681+0000","last_peered":"2024-04-23T14:43:39.431681+0000","last_clean":"2024-04-23T14:43:39.431681+0000","last_became_active":"2024-04-23T14:43:38.423891+0000","last_became_peered":"2024-04-23T14:43:38.423891+0000","last_unstale":"2024-04-23T14:43:39.431681+0000","last_undegraded":"2024-04-23T14:43:39.431681+0000","last_fullsized":"2024-04-23T14:43:39.431681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:44.483663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.447610+0000","last_change":"2024-04-23T14:43:38.446596+0000","last_active":"2024-04-23T14:43:38.447610+0000","last_peered":"2024-04-23T14:43:38.447610+0000","last_clean":"2024-04-23T14:43:38.447610+0000","last_became_active":"2024-04-23T14:43:38.446103+0000","last_became_peered":"2024-04-23T14:43:38.446103+0000","last_unstale":"2024-04-23T14:43:38.447610+0000","last_undegraded":"2024-04-23T14:43:38.447610+0000","last_fullsized":"2024-04-23T14:43:38.447610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:21.218195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439807+0000","last_change":"2024-04-23T14:43:38.423704+0000","last_active":"2024-04-23T14:43:38.439807+0000","last_peered":"2024-04-23T14:43:38.439807+0000","last_clean":"2024-04-23T14:43:38.439807+0000","last_became_active":"2024-04-23T14:43:38.420553+0000","last_became_peered":"2024-04-23T14:43:38.420553+0000","last_unstale":"2024-04-23T14:43:38.439807+0000","last_undegraded":"2024-04-23T14:43:38.439807+0000","last_fullsized":"2024-04-23T14:43:38.439807+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:58:50.829518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439528+0000","last_change":"2024-04-23T14:43:38.432262+0000","last_active":"2024-04-23T14:43:38.439528+0000","last_peered":"2024-04-23T14:43:38.439528+0000","last_clean":"2024-04-23T14:43:38.439528+0000","last_became_active":"2024-04-23T14:43:38.431889+0000","last_became_peered":"2024-04-23T14:43:38.431889+0000","last_unstale":"2024-04-23T14:43:38.439528+0000","last_undegraded":"2024-04-23T14:43:38.439528+0000","last_fullsized":"2024-04-23T14:43:38.439528+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:48.200293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.431111+0000","last_change":"2024-04-23T14:43:38.425701+0000","last_active":"2024-04-23T14:43:38.431111+0000","last_peered":"2024-04-23T14:43:38.431111+0000","last_clean":"2024-04-23T14:43:38.431111+0000","last_became_active":"2024-04-23T14:43:38.425437+0000","last_became_peered":"2024-04-23T14:43:38.425437+0000","last_unstale":"2024-04-23T14:43:38.431111+0000","last_undegraded":"2024-04-23T14:43:38.431111+0000","last_fullsized":"2024-04-23T14:43:38.431111+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:14:24.232135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432263+0000","last_change":"2024-04-23T14:43:38.425675+0000","last_active":"2024-04-23T14:43:38.432263+0000","last_peered":"2024-04-23T14:43:38.432263+0000","last_clean":"2024-04-23T14:43:38.432263+0000","last_became_active":"2024-04-23T14:43:38.425117+0000","last_became_peered":"2024-04-23T14:43:38.425117+0000","last_unstale":"2024-04-23T14:43:38.432263+0000","last_undegraded":"2024-04-23T14:43:38.432263+0000","last_fullsized":"2024-04-23T14:43:38.432263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:48:35.913730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445899+0000","last_change":"2024-04-23T14:43:38.443325+0000","last_active":"2024-04-23T14:43:38.445899+0000","last_peered":"2024-04-23T14:43:38.445899+0000","last_clean":"2024-04-23T14:43:38.445899+0000","last_became_active":"2024-04-23T14:43:38.442945+0000","last_became_peered":"2024-04-23T14:43:38.442945+0000","last_unstale":"2024-04-23T14:43:38.445899+0000","last_undegraded":"2024-04-23T14:43:38.445899+0000","last_fullsized":"2024-04-23T14:43:38.445899+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:39.671402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433358+0000","last_change":"2024-04-23T14:43:38.425381+0000","last_active":"2024-04-23T14:43:38.433358+0000","last_peered":"2024-04-23T14:43:38.433358+0000","last_clean":"2024-04-23T14:43:38.433358+0000","last_became_active":"2024-04-23T14:43:38.424503+0000","last_became_peered":"2024-04-23T14:43:38.424503+0000","last_unstale":"2024-04-23T14:43:38.433358+0000","last_undegraded":"2024-04-23T14:43:38.433358+0000","last_fullsized":"2024-04-23T14:43:38.433358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:15.924427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.442437+0000","last_change":"2024-04-23T14:43:38.435393+0000","last_active":"2024-04-23T14:43:38.442437+0000","last_peered":"2024-04-23T14:43:38.442437+0000","last_clean":"2024-04-23T14:43:38.442437+0000","last_became_active":"2024-04-23T14:43:38.435059+0000","last_became_peered":"2024-04-23T14:43:38.435059+0000","last_unstale":"2024-04-23T14:43:38.442437+0000","last_undegraded":"2024-04-23T14:43:38.442437+0000","last_fullsized":"2024-04-23T14:43:38.442437+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:24.042337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432089+0000","last_change":"2024-04-23T14:43:38.428967+0000","last_active":"2024-04-23T14:43:39.432089+0000","last_peered":"2024-04-23T14:43:39.432089+0000","last_clean":"2024-04-23T14:43:39.432089+0000","last_became_active":"2024-04-23T14:43:38.428659+0000","last_became_peered":"2024-04-23T14:43:38.428659+0000","last_unstale":"2024-04-23T14:43:39.432089+0000","last_undegraded":"2024-04-23T14:43:39.432089+0000","last_fullsized":"2024-04-23T14:43:39.432089+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:23.492135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443188+0000","last_change":"2024-04-23T14:43:38.438522+0000","last_active":"2024-04-23T14:43:38.443188+0000","last_peered":"2024-04-23T14:43:38.443188+0000","last_clean":"2024-04-23T14:43:38.443188+0000","last_became_active":"2024-04-23T14:43:38.438168+0000","last_became_peered":"2024-04-23T14:43:38.438168+0000","last_unstale":"2024-04-23T14:43:38.443188+0000","last_undegraded":"2024-04-23T14:43:38.443188+0000","last_fullsized":"2024-04-23T14:43:38.443188+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:10.181288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445498+0000","last_change":"2024-04-23T14:43:38.442247+0000","last_active":"2024-04-23T14:43:38.445498+0000","last_peered":"2024-04-23T14:43:38.445498+0000","last_clean":"2024-04-23T14:43:38.445498+0000","last_became_active":"2024-04-23T14:43:38.441811+0000","last_became_peered":"2024-04-23T14:43:38.441811+0000","last_unstale":"2024-04-23T14:43:38.445498+0000","last_undegraded":"2024-04-23T14:43:38.445498+0000","last_fullsized":"2024-04-23T14:43:38.445498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:23.492135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443310+0000","last_change":"2024-04-23T14:43:38.439372+0000","last_active":"2024-04-23T14:43:38.443310+0000","last_peered":"2024-04-23T14:43:38.443310+0000","last_clean":"2024-04-23T14:43:38.443310+0000","last_became_active":"2024-04-23T14:43:38.438897+0000","last_became_peered":"2024-04-23T14:43:38.438897+0000","last_unstale":"2024-04-23T14:43:38.443310+0000","last_undegraded":"2024-04-23T14:43:38.443310+0000","last_fullsized":"2024-04-23T14:43:38.443310+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:23.605727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432939+0000","last_change":"2024-04-23T14:43:38.427723+0000","last_active":"2024-04-23T14:43:38.432939+0000","last_peered":"2024-04-23T14:43:38.432939+0000","last_clean":"2024-04-23T14:43:38.432939+0000","last_became_active":"2024-04-23T14:43:38.426770+0000","last_became_peered":"2024-04-23T14:43:38.426770+0000","last_unstale":"2024-04-23T14:43:38.432939+0000","last_undegraded":"2024-04-23T14:43:38.432939+0000","last_fullsized":"2024-04-23T14:43:38.432939+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:28.624694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443178+0000","last_change":"2024-04-23T14:43:38.438336+0000","last_active":"2024-04-23T14:43:38.443178+0000","last_peered":"2024-04-23T14:43:38.443178+0000","last_clean":"2024-04-23T14:43:38.443178+0000","last_became_active":"2024-04-23T14:43:38.438042+0000","last_became_peered":"2024-04-23T14:43:38.438042+0000","last_unstale":"2024-04-23T14:43:38.443178+0000","last_undegraded":"2024-04-23T14:43:38.443178+0000","last_fullsized":"2024-04-23T14:43:38.443178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:26.911748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432135+0000","last_change":"2024-04-23T14:43:38.427603+0000","last_active":"2024-04-23T14:43:39.432135+0000","last_peered":"2024-04-23T14:43:39.432135+0000","last_clean":"2024-04-23T14:43:39.432135+0000","last_became_active":"2024-04-23T14:43:38.427284+0000","last_became_peered":"2024-04-23T14:43:38.427284+0000","last_unstale":"2024-04-23T14:43:39.432135+0000","last_undegraded":"2024-04-23T14:43:39.432135+0000","last_fullsized":"2024-04-23T14:43:39.432135+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:13.182967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429519+0000","last_change":"2024-04-23T14:43:38.425991+0000","last_active":"2024-04-23T14:43:39.429519+0000","last_peered":"2024-04-23T14:43:39.429519+0000","last_clean":"2024-04-23T14:43:39.429519+0000","last_became_active":"2024-04-23T14:43:38.422815+0000","last_became_peered":"2024-04-23T14:43:38.422815+0000","last_unstale":"2024-04-23T14:43:39.429519+0000","last_undegraded":"2024-04-23T14:43:39.429519+0000","last_fullsized":"2024-04-23T14:43:39.429519+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:34.339244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.446360+0000","last_change":"2024-04-23T14:43:38.442869+0000","last_active":"2024-04-23T14:43:38.446360+0000","last_peered":"2024-04-23T14:43:38.446360+0000","last_clean":"2024-04-23T14:43:38.446360+0000","last_became_active":"2024-04-23T14:43:38.442367+0000","last_became_peered":"2024-04-23T14:43:38.442367+0000","last_unstale":"2024-04-23T14:43:38.446360+0000","last_undegraded":"2024-04-23T14:43:38.446360+0000","last_fullsized":"2024-04-23T14:43:38.446360+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:13.182967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431498+0000","last_change":"2024-04-23T14:43:38.438917+0000","last_active":"2024-04-23T14:43:39.431498+0000","last_peered":"2024-04-23T14:43:39.431498+0000","last_clean":"2024-04-23T14:43:39.431498+0000","last_became_active":"2024-04-23T14:43:38.438619+0000","last_became_peered":"2024-04-23T14:43:38.438619+0000","last_unstale":"2024-04-23T14:43:39.431498+0000","last_undegraded":"2024-04-23T14:43:39.431498+0000","last_fullsized":"2024-04-23T14:43:39.431498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:02.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432609+0000","last_change":"2024-04-23T14:43:38.432925+0000","last_active":"2024-04-23T14:43:39.432609+0000","last_peered":"2024-04-23T14:43:39.432609+0000","last_clean":"2024-04-23T14:43:39.432609+0000","last_became_active":"2024-04-23T14:43:38.432255+0000","last_became_peered":"2024-04-23T14:43:38.432255+0000","last_unstale":"2024-04-23T14:43:39.432609+0000","last_undegraded":"2024-04-23T14:43:39.432609+0000","last_fullsized":"2024-04-23T14:43:39.432609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:34.339244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426856+0000","last_change":"2024-04-23T14:43:38.429890+0000","last_active":"2024-04-23T14:43:39.426856+0000","last_peered":"2024-04-23T14:43:39.426856+0000","last_clean":"2024-04-23T14:43:39.426856+0000","last_became_active":"2024-04-23T14:43:38.429085+0000","last_became_peered":"2024-04-23T14:43:38.429085+0000","last_unstale":"2024-04-23T14:43:39.426856+0000","last_undegraded":"2024-04-23T14:43:39.426856+0000","last_fullsized":"2024-04-23T14:43:39.426856+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:10.181288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430528+0000","last_change":"2024-04-23T14:43:38.436846+0000","last_active":"2024-04-23T14:43:39.430528+0000","last_peered":"2024-04-23T14:43:39.430528+0000","last_clean":"2024-04-23T14:43:39.430528+0000","last_became_active":"2024-04-23T14:43:38.436603+0000","last_became_peered":"2024-04-23T14:43:38.436603+0000","last_unstale":"2024-04-23T14:43:39.430528+0000","last_undegraded":"2024-04-23T14:43:39.430528+0000","last_fullsized":"2024-04-23T14:43:39.430528+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:37.437229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425662+0000","last_change":"2024-04-23T14:43:38.438926+0000","last_active":"2024-04-23T14:43:39.425662+0000","last_peered":"2024-04-23T14:43:39.425662+0000","last_clean":"2024-04-23T14:43:39.425662+0000","last_became_active":"2024-04-23T14:43:38.438638+0000","last_became_peered":"2024-04-23T14:43:38.438638+0000","last_unstale":"2024-04-23T14:43:39.425662+0000","last_undegraded":"2024-04-23T14:43:39.425662+0000","last_fullsized":"2024-04-23T14:43:39.425662+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:15.924427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430609+0000","last_change":"2024-04-23T14:43:38.430684+0000","last_active":"2024-04-23T14:43:39.430609+0000","last_peered":"2024-04-23T14:43:39.430609+0000","last_clean":"2024-04-23T14:43:39.430609+0000","last_became_active":"2024-04-23T14:43:38.430412+0000","last_became_peered":"2024-04-23T14:43:38.430412+0000","last_unstale":"2024-04-23T14:43:39.430609+0000","last_undegraded":"2024-04-23T14:43:39.430609+0000","last_fullsized":"2024-04-23T14:43:39.430609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:03.435484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.442225+0000","last_change":"2024-04-23T14:43:38.436193+0000","last_active":"2024-04-23T14:43:38.442225+0000","last_peered":"2024-04-23T14:43:38.442225+0000","last_clean":"2024-04-23T14:43:38.442225+0000","last_became_active":"2024-04-23T14:43:38.435832+0000","last_became_peered":"2024-04-23T14:43:38.435832+0000","last_unstale":"2024-04-23T14:43:38.442225+0000","last_undegraded":"2024-04-23T14:43:38.442225+0000","last_fullsized":"2024-04-23T14:43:38.442225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:19:22.917967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432221+0000","last_change":"2024-04-23T14:43:38.429551+0000","last_active":"2024-04-23T14:43:38.432221+0000","last_peered":"2024-04-23T14:43:38.432221+0000","last_clean":"2024-04-23T14:43:38.432221+0000","last_became_active":"2024-04-23T14:43:38.429250+0000","last_became_peered":"2024-04-23T14:43:38.429250+0000","last_unstale":"2024-04-23T14:43:38.432221+0000","last_undegraded":"2024-04-23T14:43:38.432221+0000","last_fullsized":"2024-04-23T14:43:38.432221+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:16.866077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425851+0000","last_change":"2024-04-23T14:43:38.420978+0000","last_active":"2024-04-23T14:43:39.425851+0000","last_peered":"2024-04-23T14:43:39.425851+0000","last_clean":"2024-04-23T14:43:39.425851+0000","last_became_active":"2024-04-23T14:43:38.420687+0000","last_became_peered":"2024-04-23T14:43:38.420687+0000","last_unstale":"2024-04-23T14:43:39.425851+0000","last_undegraded":"2024-04-23T14:43:39.425851+0000","last_fullsized":"2024-04-23T14:43:39.425851+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:52:36.665403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431864+0000","last_change":"2024-04-23T14:43:38.440424+0000","last_active":"2024-04-23T14:43:39.431864+0000","last_peered":"2024-04-23T14:43:39.431864+0000","last_clean":"2024-04-23T14:43:39.431864+0000","last_became_active":"2024-04-23T14:43:38.439803+0000","last_became_peered":"2024-04-23T14:43:38.439803+0000","last_unstale":"2024-04-23T14:43:39.431864+0000","last_undegraded":"2024-04-23T14:43:39.431864+0000","last_fullsized":"2024-04-23T14:43:39.431864+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:23.605727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.431207+0000","last_change":"2024-04-23T14:43:38.425793+0000","last_active":"2024-04-23T14:43:38.431207+0000","last_peered":"2024-04-23T14:43:38.431207+0000","last_clean":"2024-04-23T14:43:38.431207+0000","last_became_active":"2024-04-23T14:43:38.425387+0000","last_became_peered":"2024-04-23T14:43:38.425387+0000","last_unstale":"2024-04-23T14:43:38.431207+0000","last_undegraded":"2024-04-23T14:43:38.431207+0000","last_fullsized":"2024-04-23T14:43:38.431207+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:25.560889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.434212+0000","last_change":"2024-04-23T14:43:38.426390+0000","last_active":"2024-04-23T14:43:38.434212+0000","last_peered":"2024-04-23T14:43:38.434212+0000","last_clean":"2024-04-23T14:43:38.434212+0000","last_became_active":"2024-04-23T14:43:38.426088+0000","last_became_peered":"2024-04-23T14:43:38.426088+0000","last_unstale":"2024-04-23T14:43:38.434212+0000","last_undegraded":"2024-04-23T14:43:38.434212+0000","last_fullsized":"2024-04-23T14:43:38.434212+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:52:36.665403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432081+0000","last_change":"2024-04-23T14:43:38.427501+0000","last_active":"2024-04-23T14:43:38.432081+0000","last_peered":"2024-04-23T14:43:38.432081+0000","last_clean":"2024-04-23T14:43:38.432081+0000","last_became_active":"2024-04-23T14:43:38.427115+0000","last_became_peered":"2024-04-23T14:43:38.427115+0000","last_unstale":"2024-04-23T14:43:38.432081+0000","last_undegraded":"2024-04-23T14:43:38.432081+0000","last_fullsized":"2024-04-23T14:43:38.432081+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:42.374475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433581+0000","last_change":"2024-04-23T14:43:38.431007+0000","last_active":"2024-04-23T14:43:38.433581+0000","last_peered":"2024-04-23T14:43:38.433581+0000","last_clean":"2024-04-23T14:43:38.433581+0000","last_became_active":"2024-04-23T14:43:38.430728+0000","last_became_peered":"2024-04-23T14:43:38.430728+0000","last_unstale":"2024-04-23T14:43:38.433581+0000","last_undegraded":"2024-04-23T14:43:38.433581+0000","last_fullsized":"2024-04-23T14:43:38.433581+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:27.583007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.441879+0000","last_change":"2024-04-23T14:43:37.398296+0000","last_active":"2024-04-23T14:43:38.441879+0000","last_peered":"2024-04-23T14:43:38.441879+0000","last_clean":"2024-04-23T14:43:38.441879+0000","last_became_active":"2024-04-23T14:43:37.397990+0000","last_became_peered":"2024-04-23T14:43:37.397990+0000","last_unstale":"2024-04-23T14:43:38.441879+0000","last_undegraded":"2024-04-23T14:43:38.441879+0000","last_fullsized":"2024-04-23T14:43:38.441879+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:47.886102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430441+0000","last_change":"2024-04-23T14:43:38.437869+0000","last_active":"2024-04-23T14:43:39.430441+0000","last_peered":"2024-04-23T14:43:39.430441+0000","last_clean":"2024-04-23T14:43:39.430441+0000","last_became_active":"2024-04-23T14:43:38.437082+0000","last_became_peered":"2024-04-23T14:43:38.437082+0000","last_unstale":"2024-04-23T14:43:39.430441+0000","last_undegraded":"2024-04-23T14:43:39.430441+0000","last_fullsized":"2024-04-23T14:43:39.430441+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:36.121951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439124+0000","last_change":"2024-04-23T14:43:37.399408+0000","last_active":"2024-04-23T14:43:38.439124+0000","last_peered":"2024-04-23T14:43:38.439124+0000","last_clean":"2024-04-23T14:43:38.439124+0000","last_became_active":"2024-04-23T14:43:37.399089+0000","last_became_peered":"2024-04-23T14:43:37.399089+0000","last_unstale":"2024-04-23T14:43:38.439124+0000","last_undegraded":"2024-04-23T14:43:38.439124+0000","last_fullsized":"2024-04-23T14:43:38.439124+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:44.679192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.431412+0000","last_change":"2024-04-23T14:43:38.425098+0000","last_active":"2024-04-23T14:43:38.431412+0000","last_peered":"2024-04-23T14:43:38.431412+0000","last_clean":"2024-04-23T14:43:38.431412+0000","last_became_active":"2024-04-23T14:43:38.424830+0000","last_became_peered":"2024-04-23T14:43:38.424830+0000","last_unstale":"2024-04-23T14:43:38.431412+0000","last_undegraded":"2024-04-23T14:43:38.431412+0000","last_fullsized":"2024-04-23T14:43:38.431412+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:33.803734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.437989+0000","last_change":"2024-04-23T14:43:37.381509+0000","last_active":"2024-04-23T14:43:38.437989+0000","last_peered":"2024-04-23T14:43:38.437989+0000","last_clean":"2024-04-23T14:43:38.437989+0000","last_became_active":"2024-04-23T14:43:37.381054+0000","last_became_peered":"2024-04-23T14:43:37.381054+0000","last_unstale":"2024-04-23T14:43:38.437989+0000","last_undegraded":"2024-04-23T14:43:38.437989+0000","last_fullsized":"2024-04-23T14:43:38.437989+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:14.241544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.446380+0000","last_change":"2024-04-23T14:43:38.443297+0000","last_active":"2024-04-23T14:43:38.446380+0000","last_peered":"2024-04-23T14:43:38.446380+0000","last_clean":"2024-04-23T14:43:38.446380+0000","last_became_active":"2024-04-23T14:43:38.442960+0000","last_became_peered":"2024-04-23T14:43:38.442960+0000","last_unstale":"2024-04-23T14:43:38.446380+0000","last_undegraded":"2024-04-23T14:43:38.446380+0000","last_fullsized":"2024-04-23T14:43:38.446380+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:34.339244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.440105+0000","last_change":"2024-04-23T14:43:37.389936+0000","last_active":"2024-04-23T14:43:38.440105+0000","last_peered":"2024-04-23T14:43:38.440105+0000","last_clean":"2024-04-23T14:43:38.440105+0000","last_became_active":"2024-04-23T14:43:37.389643+0000","last_became_peered":"2024-04-23T14:43:37.389643+0000","last_unstale":"2024-04-23T14:43:38.440105+0000","last_undegraded":"2024-04-23T14:43:38.440105+0000","last_fullsized":"2024-04-23T14:43:38.440105+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:12:58.319727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.434539+0000","last_change":"2024-04-23T14:43:38.426535+0000","last_active":"2024-04-23T14:43:38.434539+0000","last_peered":"2024-04-23T14:43:38.434539+0000","last_clean":"2024-04-23T14:43:38.434539+0000","last_became_active":"2024-04-23T14:43:38.426223+0000","last_became_peered":"2024-04-23T14:43:38.426223+0000","last_unstale":"2024-04-23T14:43:38.434539+0000","last_undegraded":"2024-04-23T14:43:38.434539+0000","last_fullsized":"2024-04-23T14:43:38.434539+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:28:41.176649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.441408+0000","last_change":"2024-04-23T14:43:37.396839+0000","last_active":"2024-04-23T14:43:38.441408+0000","last_peered":"2024-04-23T14:43:38.441408+0000","last_clean":"2024-04-23T14:43:38.441408+0000","last_became_active":"2024-04-23T14:43:37.396428+0000","last_became_peered":"2024-04-23T14:43:37.396428+0000","last_unstale":"2024-04-23T14:43:38.441408+0000","last_undegraded":"2024-04-23T14:43:38.441408+0000","last_fullsized":"2024-04-23T14:43:38.441408+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:12:45.596979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431919+0000","last_change":"2024-04-23T14:43:38.425624+0000","last_active":"2024-04-23T14:43:39.431919+0000","last_peered":"2024-04-23T14:43:39.431919+0000","last_clean":"2024-04-23T14:43:39.431919+0000","last_became_active":"2024-04-23T14:43:38.423946+0000","last_became_peered":"2024-04-23T14:43:38.423946+0000","last_unstale":"2024-04-23T14:43:39.431919+0000","last_undegraded":"2024-04-23T14:43:39.431919+0000","last_fullsized":"2024-04-23T14:43:39.431919+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:00:23.088263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426176+0000","last_change":"2024-04-23T14:43:37.390987+0000","last_active":"2024-04-23T14:43:39.426176+0000","last_peered":"2024-04-23T14:43:39.426176+0000","last_clean":"2024-04-23T14:43:39.426176+0000","last_became_active":"2024-04-23T14:43:37.390665+0000","last_became_peered":"2024-04-23T14:43:37.390665+0000","last_unstale":"2024-04-23T14:43:39.426176+0000","last_undegraded":"2024-04-23T14:43:39.426176+0000","last_fullsized":"2024-04-23T14:43:39.426176+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:15.377239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433099+0000","last_change":"2024-04-23T14:43:38.417842+0000","last_active":"2024-04-23T14:43:38.433099+0000","last_peered":"2024-04-23T14:43:38.433099+0000","last_clean":"2024-04-23T14:43:38.433099+0000","last_became_active":"2024-04-23T14:43:38.416006+0000","last_became_peered":"2024-04-23T14:43:38.416006+0000","last_unstale":"2024-04-23T14:43:38.433099+0000","last_undegraded":"2024-04-23T14:43:38.433099+0000","last_fullsized":"2024-04-23T14:43:38.433099+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:41:41.960832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430761+0000","last_change":"2024-04-23T14:43:37.393128+0000","last_active":"2024-04-23T14:43:39.430761+0000","last_peered":"2024-04-23T14:43:39.430761+0000","last_clean":"2024-04-23T14:43:39.430761+0000","last_became_active":"2024-04-23T14:43:37.392718+0000","last_became_peered":"2024-04-23T14:43:37.392718+0000","last_unstale":"2024-04-23T14:43:39.430761+0000","last_undegraded":"2024-04-23T14:43:39.430761+0000","last_fullsized":"2024-04-23T14:43:39.430761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:05:18.274497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.434199+0000","last_change":"2024-04-23T14:43:38.426856+0000","last_active":"2024-04-23T14:43:38.434199+0000","last_peered":"2024-04-23T14:43:38.434199+0000","last_clean":"2024-04-23T14:43:38.434199+0000","last_became_active":"2024-04-23T14:43:38.426564+0000","last_became_peered":"2024-04-23T14:43:38.426564+0000","last_unstale":"2024-04-23T14:43:38.434199+0000","last_undegraded":"2024-04-23T14:43:38.434199+0000","last_fullsized":"2024-04-23T14:43:38.434199+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:02.655011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.404792+0000","last_change":"2024-04-23T14:43:37.398466+0000","last_active":"2024-04-23T14:43:38.404792+0000","last_peered":"2024-04-23T14:43:38.404792+0000","last_clean":"2024-04-23T14:43:38.404792+0000","last_became_active":"2024-04-23T14:43:37.397653+0000","last_became_peered":"2024-04-23T14:43:37.397653+0000","last_unstale":"2024-04-23T14:43:38.404792+0000","last_undegraded":"2024-04-23T14:43:38.404792+0000","last_fullsized":"2024-04-23T14:43:38.404792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:02.577062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.426475+0000","last_change":"2024-04-23T14:43:38.421478+0000","last_active":"2024-04-23T14:43:38.426475+0000","last_peered":"2024-04-23T14:43:38.426475+0000","last_clean":"2024-04-23T14:43:38.426475+0000","last_became_active":"2024-04-23T14:43:38.421151+0000","last_became_peered":"2024-04-23T14:43:38.421151+0000","last_unstale":"2024-04-23T14:43:38.426475+0000","last_undegraded":"2024-04-23T14:43:38.426475+0000","last_fullsized":"2024-04-23T14:43:38.426475+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:42:15.032645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405904+0000","last_change":"2024-04-23T14:43:37.396127+0000","last_active":"2024-04-23T14:43:38.405904+0000","last_peered":"2024-04-23T14:43:38.405904+0000","last_clean":"2024-04-23T14:43:38.405904+0000","last_became_active":"2024-04-23T14:43:37.395826+0000","last_became_peered":"2024-04-23T14:43:37.395826+0000","last_unstale":"2024-04-23T14:43:38.405904+0000","last_undegraded":"2024-04-23T14:43:38.405904+0000","last_fullsized":"2024-04-23T14:43:38.405904+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:17.382285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430656+0000","last_change":"2024-04-23T14:43:38.428596+0000","last_active":"2024-04-23T14:43:39.430656+0000","last_peered":"2024-04-23T14:43:39.430656+0000","last_clean":"2024-04-23T14:43:39.430656+0000","last_became_active":"2024-04-23T14:43:38.426742+0000","last_became_peered":"2024-04-23T14:43:38.426742+0000","last_unstale":"2024-04-23T14:43:39.430656+0000","last_undegraded":"2024-04-23T14:43:39.430656+0000","last_fullsized":"2024-04-23T14:43:39.430656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:21.218195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425870+0000","last_change":"2024-04-23T14:43:37.391615+0000","last_active":"2024-04-23T14:43:39.425870+0000","last_peered":"2024-04-23T14:43:39.425870+0000","last_clean":"2024-04-23T14:43:39.425870+0000","last_became_active":"2024-04-23T14:43:37.390345+0000","last_became_peered":"2024-04-23T14:43:37.390345+0000","last_unstale":"2024-04-23T14:43:39.425870+0000","last_undegraded":"2024-04-23T14:43:39.425870+0000","last_fullsized":"2024-04-23T14:43:39.425870+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:02.577062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425963+0000","last_change":"2024-04-23T14:43:38.421117+0000","last_active":"2024-04-23T14:43:39.425963+0000","last_peered":"2024-04-23T14:43:39.425963+0000","last_clean":"2024-04-23T14:43:39.425963+0000","last_became_active":"2024-04-23T14:43:38.420822+0000","last_became_peered":"2024-04-23T14:43:38.420822+0000","last_unstale":"2024-04-23T14:43:39.425963+0000","last_undegraded":"2024-04-23T14:43:39.425963+0000","last_fullsized":"2024-04-23T14:43:39.425963+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:28:41.176649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.435442+0000","last_change":"2024-04-23T14:43:37.398066+0000","last_active":"2024-04-23T14:43:38.435442+0000","last_peered":"2024-04-23T14:43:38.435442+0000","last_clean":"2024-04-23T14:43:38.435442+0000","last_became_active":"2024-04-23T14:43:37.397746+0000","last_became_peered":"2024-04-23T14:43:37.397746+0000","last_unstale":"2024-04-23T14:43:38.435442+0000","last_undegraded":"2024-04-23T14:43:38.435442+0000","last_fullsized":"2024-04-23T14:43:38.435442+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:32:11.839954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443371+0000","last_change":"2024-04-23T14:43:38.436801+0000","last_active":"2024-04-23T14:43:38.443371+0000","last_peered":"2024-04-23T14:43:38.443371+0000","last_clean":"2024-04-23T14:43:38.443371+0000","last_became_active":"2024-04-23T14:43:38.436479+0000","last_became_peered":"2024-04-23T14:43:38.436479+0000","last_unstale":"2024-04-23T14:43:38.443371+0000","last_undegraded":"2024-04-23T14:43:38.443371+0000","last_fullsized":"2024-04-23T14:43:38.443371+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:50.162787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445127+0000","last_change":"2024-04-23T14:43:37.399207+0000","last_active":"2024-04-23T14:43:38.445127+0000","last_peered":"2024-04-23T14:43:38.445127+0000","last_clean":"2024-04-23T14:43:38.445127+0000","last_became_active":"2024-04-23T14:43:37.398925+0000","last_became_peered":"2024-04-23T14:43:37.398925+0000","last_unstale":"2024-04-23T14:43:38.445127+0000","last_undegraded":"2024-04-23T14:43:38.445127+0000","last_fullsized":"2024-04-23T14:43:38.445127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:38.440250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426101+0000","last_change":"2024-04-23T14:43:38.414115+0000","last_active":"2024-04-23T14:43:39.426101+0000","last_peered":"2024-04-23T14:43:39.426101+0000","last_clean":"2024-04-23T14:43:39.426101+0000","last_became_active":"2024-04-23T14:43:38.412682+0000","last_became_peered":"2024-04-23T14:43:38.412682+0000","last_unstale":"2024-04-23T14:43:39.426101+0000","last_undegraded":"2024-04-23T14:43:39.426101+0000","last_fullsized":"2024-04-23T14:43:39.426101+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:24.042337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.441447+0000","last_change":"2024-04-23T14:43:37.396663+0000","last_active":"2024-04-23T14:43:38.441447+0000","last_peered":"2024-04-23T14:43:38.441447+0000","last_clean":"2024-04-23T14:43:38.441447+0000","last_became_active":"2024-04-23T14:43:37.396385+0000","last_became_peered":"2024-04-23T14:43:37.396385+0000","last_unstale":"2024-04-23T14:43:38.441447+0000","last_undegraded":"2024-04-23T14:43:38.441447+0000","last_fullsized":"2024-04-23T14:43:38.441447+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:47.181215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426565+0000","last_change":"2024-04-23T14:43:38.439386+0000","last_active":"2024-04-23T14:43:39.426565+0000","last_peered":"2024-04-23T14:43:39.426565+0000","last_clean":"2024-04-23T14:43:39.426565+0000","last_became_active":"2024-04-23T14:43:38.437256+0000","last_became_peered":"2024-04-23T14:43:38.437256+0000","last_unstale":"2024-04-23T14:43:39.426565+0000","last_undegraded":"2024-04-23T14:43:39.426565+0000","last_fullsized":"2024-04-23T14:43:39.426565+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:19:22.917967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.428075+0000","last_change":"2024-04-23T14:43:37.396068+0000","last_active":"2024-04-23T14:43:39.428075+0000","last_peered":"2024-04-23T14:43:39.428075+0000","last_clean":"2024-04-23T14:43:39.428075+0000","last_became_active":"2024-04-23T14:43:37.395051+0000","last_became_peered":"2024-04-23T14:43:37.395051+0000","last_unstale":"2024-04-23T14:43:39.428075+0000","last_undegraded":"2024-04-23T14:43:39.428075+0000","last_fullsized":"2024-04-23T14:43:39.428075+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:01.586328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.427010+0000","last_change":"2024-04-23T14:43:38.423144+0000","last_active":"2024-04-23T14:43:39.427010+0000","last_peered":"2024-04-23T14:43:39.427010+0000","last_clean":"2024-04-23T14:43:39.427010+0000","last_became_active":"2024-04-23T14:43:38.422859+0000","last_became_peered":"2024-04-23T14:43:38.422859+0000","last_unstale":"2024-04-23T14:43:39.427010+0000","last_undegraded":"2024-04-23T14:43:39.427010+0000","last_fullsized":"2024-04-23T14:43:39.427010+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:19:58.747750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":20,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.442052+0000","last_change":"2024-04-23T14:43:37.391943+0000","last_active":"2024-04-23T14:43:38.442052+0000","last_peered":"2024-04-23T14:43:38.442052+0000","last_clean":"2024-04-23T14:43:38.442052+0000","last_became_active":"2024-04-23T14:43:37.391681+0000","last_became_peered":"2024-04-23T14:43:37.391681+0000","last_unstale":"2024-04-23T14:43:38.442052+0000","last_undegraded":"2024-04-23T14:43:38.442052+0000","last_fullsized":"2024-04-23T14:43:38.442052+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:25.911325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433773+0000","last_change":"2024-04-23T14:43:38.421354+0000","last_active":"2024-04-23T14:43:38.433773+0000","last_peered":"2024-04-23T14:43:38.433773+0000","last_clean":"2024-04-23T14:43:38.433773+0000","last_became_active":"2024-04-23T14:43:38.421080+0000","last_became_peered":"2024-04-23T14:43:38.421080+0000","last_unstale":"2024-04-23T14:43:38.433773+0000","last_undegraded":"2024-04-23T14:43:38.433773+0000","last_fullsized":"2024-04-23T14:43:38.433773+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:53.936303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439412+0000","last_change":"2024-04-23T14:43:37.397956+0000","last_active":"2024-04-23T14:43:38.439412+0000","last_peered":"2024-04-23T14:43:38.439412+0000","last_clean":"2024-04-23T14:43:38.439412+0000","last_became_active":"2024-04-23T14:43:37.396553+0000","last_became_peered":"2024-04-23T14:43:37.396553+0000","last_unstale":"2024-04-23T14:43:38.439412+0000","last_undegraded":"2024-04-23T14:43:38.439412+0000","last_fullsized":"2024-04-23T14:43:38.439412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:31:14.821283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.438957+0000","last_change":"2024-04-23T14:43:38.435200+0000","last_active":"2024-04-23T14:43:38.438957+0000","last_peered":"2024-04-23T14:43:38.438957+0000","last_clean":"2024-04-23T14:43:38.438957+0000","last_became_active":"2024-04-23T14:43:38.434921+0000","last_became_peered":"2024-04-23T14:43:38.434921+0000","last_unstale":"2024-04-23T14:43:38.438957+0000","last_undegraded":"2024-04-23T14:43:38.438957+0000","last_fullsized":"2024-04-23T14:43:38.438957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:05:19.293575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445701+0000","last_change":"2024-04-23T14:43:37.389166+0000","last_active":"2024-04-23T14:43:38.445701+0000","last_peered":"2024-04-23T14:43:38.445701+0000","last_clean":"2024-04-23T14:43:38.445701+0000","last_became_active":"2024-04-23T14:43:37.388846+0000","last_became_peered":"2024-04-23T14:43:37.388846+0000","last_unstale":"2024-04-23T14:43:38.445701+0000","last_undegraded":"2024-04-23T14:43:38.445701+0000","last_fullsized":"2024-04-23T14:43:38.445701+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:15.342538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445957+0000","last_change":"2024-04-23T14:43:38.444056+0000","last_active":"2024-04-23T14:43:38.445957+0000","last_peered":"2024-04-23T14:43:38.445957+0000","last_clean":"2024-04-23T14:43:38.445957+0000","last_became_active":"2024-04-23T14:43:38.443817+0000","last_became_peered":"2024-04-23T14:43:38.443817+0000","last_unstale":"2024-04-23T14:43:38.445957+0000","last_undegraded":"2024-04-23T14:43:38.445957+0000","last_fullsized":"2024-04-23T14:43:38.445957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:21.145162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429013+0000","last_change":"2024-04-23T14:43:37.379621+0000","last_active":"2024-04-23T14:43:39.429013+0000","last_peered":"2024-04-23T14:43:39.429013+0000","last_clean":"2024-04-23T14:43:39.429013+0000","last_became_active":"2024-04-23T14:43:37.378871+0000","last_became_peered":"2024-04-23T14:43:37.378871+0000","last_unstale":"2024-04-23T14:43:39.429013+0000","last_undegraded":"2024-04-23T14:43:39.429013+0000","last_fullsized":"2024-04-23T14:43:39.429013+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:02.416406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443173+0000","last_change":"2024-04-23T14:43:38.438949+0000","last_active":"2024-04-23T14:43:38.443173+0000","last_peered":"2024-04-23T14:43:38.443173+0000","last_clean":"2024-04-23T14:43:38.443173+0000","last_became_active":"2024-04-23T14:43:38.438654+0000","last_became_peered":"2024-04-23T14:43:38.438654+0000","last_unstale":"2024-04-23T14:43:38.443173+0000","last_undegraded":"2024-04-23T14:43:38.443173+0000","last_fullsized":"2024-04-23T14:43:38.443173+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:44.483663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405907+0000","last_change":"2024-04-23T14:43:37.401006+0000","last_active":"2024-04-23T14:43:38.405907+0000","last_peered":"2024-04-23T14:43:38.405907+0000","last_clean":"2024-04-23T14:43:38.405907+0000","last_became_active":"2024-04-23T14:43:37.400680+0000","last_became_peered":"2024-04-23T14:43:37.400680+0000","last_unstale":"2024-04-23T14:43:38.405907+0000","last_undegraded":"2024-04-23T14:43:38.405907+0000","last_fullsized":"2024-04-23T14:43:38.405907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:09.201305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.447113+0000","last_change":"2024-04-23T14:43:38.446200+0000","last_active":"2024-04-23T14:43:38.447113+0000","last_peered":"2024-04-23T14:43:38.447113+0000","last_clean":"2024-04-23T14:43:38.447113+0000","last_became_active":"2024-04-23T14:43:38.445886+0000","last_became_peered":"2024-04-23T14:43:38.445886+0000","last_unstale":"2024-04-23T14:43:38.447113+0000","last_undegraded":"2024-04-23T14:43:38.447113+0000","last_fullsized":"2024-04-23T14:43:38.447113+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:36.121951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.438718+0000","last_change":"2024-04-23T14:43:37.382042+0000","last_active":"2024-04-23T14:43:38.438718+0000","last_peered":"2024-04-23T14:43:38.438718+0000","last_clean":"2024-04-23T14:43:38.438718+0000","last_became_active":"2024-04-23T14:43:37.381706+0000","last_became_peered":"2024-04-23T14:43:37.381706+0000","last_unstale":"2024-04-23T14:43:38.438718+0000","last_undegraded":"2024-04-23T14:43:38.438718+0000","last_fullsized":"2024-04-23T14:43:38.438718+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:44.296302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443345+0000","last_change":"2024-04-23T14:43:38.440348+0000","last_active":"2024-04-23T14:43:38.443345+0000","last_peered":"2024-04-23T14:43:38.443345+0000","last_clean":"2024-04-23T14:43:38.443345+0000","last_became_active":"2024-04-23T14:43:38.436703+0000","last_became_peered":"2024-04-23T14:43:38.436703+0000","last_unstale":"2024-04-23T14:43:38.443345+0000","last_undegraded":"2024-04-23T14:43:38.443345+0000","last_fullsized":"2024-04-23T14:43:38.443345+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:19:58.747750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405783+0000","last_change":"2024-04-23T14:43:37.390219+0000","last_active":"2024-04-23T14:43:38.405783+0000","last_peered":"2024-04-23T14:43:38.405783+0000","last_clean":"2024-04-23T14:43:38.405783+0000","last_became_active":"2024-04-23T14:43:37.389935+0000","last_became_peered":"2024-04-23T14:43:37.389935+0000","last_unstale":"2024-04-23T14:43:38.405783+0000","last_undegraded":"2024-04-23T14:43:38.405783+0000","last_fullsized":"2024-04-23T14:43:38.405783+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:01.502283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445351+0000","last_change":"2024-04-23T14:43:38.441697+0000","last_active":"2024-04-23T14:43:38.445351+0000","last_peered":"2024-04-23T14:43:38.445351+0000","last_clean":"2024-04-23T14:43:38.445351+0000","last_became_active":"2024-04-23T14:43:38.441053+0000","last_became_peered":"2024-04-23T14:43:38.441053+0000","last_unstale":"2024-04-23T14:43:38.445351+0000","last_undegraded":"2024-04-23T14:43:38.445351+0000","last_fullsized":"2024-04-23T14:43:38.445351+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:00:23.088263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.406523+0000","last_change":"2024-04-23T14:43:37.382899+0000","last_active":"2024-04-23T14:43:38.406523+0000","last_peered":"2024-04-23T14:43:38.406523+0000","last_clean":"2024-04-23T14:43:38.406523+0000","last_became_active":"2024-04-23T14:43:37.382591+0000","last_became_peered":"2024-04-23T14:43:37.382591+0000","last_unstale":"2024-04-23T14:43:38.406523+0000","last_undegraded":"2024-04-23T14:43:38.406523+0000","last_fullsized":"2024-04-23T14:43:38.406523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:52.917225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426557+0000","last_change":"2024-04-23T14:43:38.416822+0000","last_active":"2024-04-23T14:43:39.426557+0000","last_peered":"2024-04-23T14:43:39.426557+0000","last_clean":"2024-04-23T14:43:39.426557+0000","last_became_active":"2024-04-23T14:43:38.416552+0000","last_became_peered":"2024-04-23T14:43:38.416552+0000","last_unstale":"2024-04-23T14:43:39.426557+0000","last_undegraded":"2024-04-23T14:43:39.426557+0000","last_fullsized":"2024-04-23T14:43:39.426557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:26.911748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.404965+0000","last_change":"2024-04-23T14:43:37.400199+0000","last_active":"2024-04-23T14:43:38.404965+0000","last_peered":"2024-04-23T14:43:38.404965+0000","last_clean":"2024-04-23T14:43:38.404965+0000","last_became_active":"2024-04-23T14:43:37.399833+0000","last_became_peered":"2024-04-23T14:43:37.399833+0000","last_unstale":"2024-04-23T14:43:38.404965+0000","last_undegraded":"2024-04-23T14:43:38.404965+0000","last_fullsized":"2024-04-23T14:43:38.404965+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:02.908466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429746+0000","last_change":"2024-04-23T14:43:38.421412+0000","last_active":"2024-04-23T14:43:39.429746+0000","last_peered":"2024-04-23T14:43:39.429746+0000","last_clean":"2024-04-23T14:43:39.429746+0000","last_became_active":"2024-04-23T14:43:38.420856+0000","last_became_peered":"2024-04-23T14:43:38.420856+0000","last_unstale":"2024-04-23T14:43:39.429746+0000","last_undegraded":"2024-04-23T14:43:39.429746+0000","last_fullsized":"2024-04-23T14:43:39.429746+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:21.145162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426156+0000","last_change":"2024-04-23T14:43:37.393619+0000","last_active":"2024-04-23T14:43:39.426156+0000","last_peered":"2024-04-23T14:43:39.426156+0000","last_clean":"2024-04-23T14:43:39.426156+0000","last_became_active":"2024-04-23T14:43:37.393330+0000","last_became_peered":"2024-04-23T14:43:37.393330+0000","last_unstale":"2024-04-23T14:43:39.426156+0000","last_undegraded":"2024-04-23T14:43:39.426156+0000","last_fullsized":"2024-04-23T14:43:39.426156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:02.908466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433334+0000","last_change":"2024-04-23T14:43:38.424909+0000","last_active":"2024-04-23T14:43:38.433334+0000","last_peered":"2024-04-23T14:43:38.433334+0000","last_clean":"2024-04-23T14:43:38.433334+0000","last_became_active":"2024-04-23T14:43:38.424142+0000","last_became_peered":"2024-04-23T14:43:38.424142+0000","last_unstale":"2024-04-23T14:43:38.433334+0000","last_undegraded":"2024-04-23T14:43:38.433334+0000","last_fullsized":"2024-04-23T14:43:38.433334+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:28.320297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429765+0000","last_change":"2024-04-23T14:43:37.397787+0000","last_active":"2024-04-23T14:43:39.429765+0000","last_peered":"2024-04-23T14:43:39.429765+0000","last_clean":"2024-04-23T14:43:39.429765+0000","last_became_active":"2024-04-23T14:43:37.397440+0000","last_became_peered":"2024-04-23T14:43:37.397440+0000","last_unstale":"2024-04-23T14:43:39.429765+0000","last_undegraded":"2024-04-23T14:43:39.429765+0000","last_fullsized":"2024-04-23T14:43:39.429765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:44.679192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.435995+0000","last_change":"2024-04-23T14:43:38.416055+0000","last_active":"2024-04-23T14:43:38.435995+0000","last_peered":"2024-04-23T14:43:38.435995+0000","last_clean":"2024-04-23T14:43:38.435995+0000","last_became_active":"2024-04-23T14:43:38.415782+0000","last_became_peered":"2024-04-23T14:43:38.415782+0000","last_unstale":"2024-04-23T14:43:38.435995+0000","last_undegraded":"2024-04-23T14:43:38.435995+0000","last_fullsized":"2024-04-23T14:43:38.435995+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:30:50.538534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.409053+0000","last_change":"2024-04-23T14:43:37.398697+0000","last_active":"2024-04-23T14:43:38.409053+0000","last_peered":"2024-04-23T14:43:38.409053+0000","last_clean":"2024-04-23T14:43:38.409053+0000","last_became_active":"2024-04-23T14:43:37.398407+0000","last_became_peered":"2024-04-23T14:43:37.398407+0000","last_unstale":"2024-04-23T14:43:38.409053+0000","last_undegraded":"2024-04-23T14:43:38.409053+0000","last_fullsized":"2024-04-23T14:43:38.409053+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:41:52.734029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.428855+0000","last_change":"2024-04-23T14:43:38.429473+0000","last_active":"2024-04-23T14:43:39.428855+0000","last_peered":"2024-04-23T14:43:39.428855+0000","last_clean":"2024-04-23T14:43:39.428855+0000","last_became_active":"2024-04-23T14:43:38.429182+0000","last_became_peered":"2024-04-23T14:43:38.429182+0000","last_unstale":"2024-04-23T14:43:39.428855+0000","last_undegraded":"2024-04-23T14:43:39.428855+0000","last_fullsized":"2024-04-23T14:43:39.428855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:39.671402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426089+0000","last_change":"2024-04-23T14:43:37.395259+0000","last_active":"2024-04-23T14:43:39.426089+0000","last_peered":"2024-04-23T14:43:39.426089+0000","last_clean":"2024-04-23T14:43:39.426089+0000","last_became_active":"2024-04-23T14:43:37.394927+0000","last_became_peered":"2024-04-23T14:43:37.394927+0000","last_unstale":"2024-04-23T14:43:39.426089+0000","last_undegraded":"2024-04-23T14:43:39.426089+0000","last_fullsized":"2024-04-23T14:43:39.426089+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:08.314795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431831+0000","last_change":"2024-04-23T14:43:38.436770+0000","last_active":"2024-04-23T14:43:39.431831+0000","last_peered":"2024-04-23T14:43:39.431831+0000","last_clean":"2024-04-23T14:43:39.431831+0000","last_became_active":"2024-04-23T14:43:38.436516+0000","last_became_peered":"2024-04-23T14:43:38.436516+0000","last_unstale":"2024-04-23T14:43:39.431831+0000","last_undegraded":"2024-04-23T14:43:39.431831+0000","last_fullsized":"2024-04-23T14:43:39.431831+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:45.698270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.406600+0000","last_change":"2024-04-23T14:43:37.395910+0000","last_active":"2024-04-23T14:43:38.406600+0000","last_peered":"2024-04-23T14:43:38.406600+0000","last_clean":"2024-04-23T14:43:38.406600+0000","last_became_active":"2024-04-23T14:43:37.395587+0000","last_became_peered":"2024-04-23T14:43:37.395587+0000","last_unstale":"2024-04-23T14:43:38.406600+0000","last_undegraded":"2024-04-23T14:43:38.406600+0000","last_fullsized":"2024-04-23T14:43:38.406600+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:11.865890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432310+0000","last_change":"2024-04-23T14:43:38.438404+0000","last_active":"2024-04-23T14:43:39.432310+0000","last_peered":"2024-04-23T14:43:39.432310+0000","last_clean":"2024-04-23T14:43:39.432310+0000","last_became_active":"2024-04-23T14:43:38.435446+0000","last_became_peered":"2024-04-23T14:43:38.435446+0000","last_unstale":"2024-04-23T14:43:39.432310+0000","last_undegraded":"2024-04-23T14:43:39.432310+0000","last_fullsized":"2024-04-23T14:43:39.432310+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:39.671402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431745+0000","last_change":"2024-04-23T14:43:37.396854+0000","last_active":"2024-04-23T14:43:39.431745+0000","last_peered":"2024-04-23T14:43:39.431745+0000","last_clean":"2024-04-23T14:43:39.431745+0000","last_became_active":"2024-04-23T14:43:37.396531+0000","last_became_peered":"2024-04-23T14:43:37.396531+0000","last_unstale":"2024-04-23T14:43:39.431745+0000","last_undegraded":"2024-04-23T14:43:39.431745+0000","last_fullsized":"2024-04-23T14:43:39.431745+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:28:39.199117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443165+0000","last_change":"2024-04-23T14:43:38.438162+0000","last_active":"2024-04-23T14:43:38.443165+0000","last_peered":"2024-04-23T14:43:38.443165+0000","last_clean":"2024-04-23T14:43:38.443165+0000","last_became_active":"2024-04-23T14:43:38.436395+0000","last_became_peered":"2024-04-23T14:43:38.436395+0000","last_unstale":"2024-04-23T14:43:38.443165+0000","last_undegraded":"2024-04-23T14:43:38.443165+0000","last_fullsized":"2024-04-23T14:43:38.443165+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:02.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.408195+0000","last_change":"2024-04-23T14:43:37.398005+0000","last_active":"2024-04-23T14:43:38.408195+0000","last_peered":"2024-04-23T14:43:38.408195+0000","last_clean":"2024-04-23T14:43:38.408195+0000","last_became_active":"2024-04-23T14:43:37.396171+0000","last_became_peered":"2024-04-23T14:43:37.396171+0000","last_unstale":"2024-04-23T14:43:38.408195+0000","last_undegraded":"2024-04-23T14:43:38.408195+0000","last_fullsized":"2024-04-23T14:43:38.408195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:56.816922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432987+0000","last_change":"2024-04-23T14:43:38.417445+0000","last_active":"2024-04-23T14:43:38.432987+0000","last_peered":"2024-04-23T14:43:38.432987+0000","last_clean":"2024-04-23T14:43:38.432987+0000","last_became_active":"2024-04-23T14:43:38.417168+0000","last_became_peered":"2024-04-23T14:43:38.417168+0000","last_unstale":"2024-04-23T14:43:38.432987+0000","last_undegraded":"2024-04-23T14:43:38.432987+0000","last_fullsized":"2024-04-23T14:43:38.432987+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:38.019894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.406480+0000","last_change":"2024-04-23T14:43:37.398203+0000","last_active":"2024-04-23T14:43:38.406480+0000","last_peered":"2024-04-23T14:43:38.406480+0000","last_clean":"2024-04-23T14:43:38.406480+0000","last_became_active":"2024-04-23T14:43:37.397184+0000","last_became_peered":"2024-04-23T14:43:37.397184+0000","last_unstale":"2024-04-23T14:43:38.406480+0000","last_undegraded":"2024-04-23T14:43:38.406480+0000","last_fullsized":"2024-04-23T14:43:38.406480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:41.182034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425848+0000","last_change":"2024-04-23T14:43:38.437357+0000","last_active":"2024-04-23T14:43:39.425848+0000","last_peered":"2024-04-23T14:43:39.425848+0000","last_clean":"2024-04-23T14:43:39.425848+0000","last_became_active":"2024-04-23T14:43:38.437046+0000","last_became_peered":"2024-04-23T14:43:38.437046+0000","last_unstale":"2024-04-23T14:43:39.425848+0000","last_undegraded":"2024-04-23T14:43:39.425848+0000","last_fullsized":"2024-04-23T14:43:39.425848+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:16.911838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405833+0000","last_change":"2024-04-23T14:43:37.397741+0000","last_active":"2024-04-23T14:43:38.405833+0000","last_peered":"2024-04-23T14:43:38.405833+0000","last_clean":"2024-04-23T14:43:38.405833+0000","last_became_active":"2024-04-23T14:43:37.397327+0000","last_became_peered":"2024-04-23T14:43:37.397327+0000","last_unstale":"2024-04-23T14:43:38.405833+0000","last_undegraded":"2024-04-23T14:43:38.405833+0000","last_fullsized":"2024-04-23T14:43:38.405833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:50:23.865313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.427669+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:43:38.427669+0000","last_peered":"2024-04-23T14:43:38.427669+0000","last_clean":"2024-04-23T14:43:38.427669+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:43:38.427669+0000","last_undegraded":"2024-04-23T14:43:38.427669+0000","last_fullsized":"2024-04-23T14:43:38.427669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1982464,"data_stored":1189930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":605316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1388544,"data_stored":599550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1380352,"data_stored":598946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":600628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":598946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1388544,"data_stored":599070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1994752,"data_stored":1191038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:43:42.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:43:43.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.151+0000 7fc89d000640 1 Processor -- start 2024-04-23T14:43:43.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.151+0000 7fc89d000640 1 -- start start 2024-04-23T14:43:43.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.152+0000 7fc89d000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc89815b450 0x7fc8980b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.152+0000 7fc89d000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc89806f3f0 0x7fc8980cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.152+0000 7fc89d000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8980b3870 con 0x7fc8980b3080 2024-04-23T14:43:43.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.152+0000 7fc89d000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc8980b41a0 con 0x7fc89815b450 2024-04-23T14:43:43.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.152+0000 7fc89d000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc8980b3b50 con 0x7fc89806f3f0 2024-04-23T14:43:43.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc897400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc89815b450 0x7fc8980b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc897400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc89815b450 0x7fc8980b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55908/0 (socket says 172.21.15.59:55908) 2024-04-23T14:43:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc897400640 1 -- 172.21.15.59:0/3690160034 learned_addr learned my addr 172.21.15.59:0/3690160034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc88fe00640 1 --2- 172.21.15.59:0/3690160034 >> v2:172.21.15.76:3300/0 conn(0x7fc89806f3f0 0x7fc8980cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc897400640 1 -- 172.21.15.59:0/3690160034 >> v2:172.21.15.76:3300/0 conn(0x7fc89806f3f0 msgr2=0x7fc8980cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc897400640 1 --2- 172.21.15.59:0/3690160034 >> v2:172.21.15.76:3300/0 conn(0x7fc89806f3f0 0x7fc8980cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc897400640 1 -- 172.21.15.59:0/3690160034 >> v1:172.21.15.76:6789/0 conn(0x7fc8980b3080 legacy=0x7fc8980ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc897400640 1 -- 172.21.15.59:0/3690160034 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8980b3e30 con 0x7fc89815b450 2024-04-23T14:43:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.153+0000 7fc88fe00640 1 --2- 172.21.15.59:0/3690160034 >> v2:172.21.15.76:3300/0 conn(0x7fc89806f3f0 0x7fc8980cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:43:43.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.155+0000 7fc897400640 1 --2- 172.21.15.59:0/3690160034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980b4630 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fc8980af4a0 tx=0x7fc88802c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e57815b8cb7090 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:43.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.155+0000 7fc896a00640 1 -- 172.21.15.59:0/3690160034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888036020 con 0x7fc89815b450 2024-04-23T14:43:43.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.155+0000 7fc896a00640 1 -- 172.21.15.59:0/3690160034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc888032830 con 0x7fc89815b450 2024-04-23T14:43:43.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.155+0000 7fc896a00640 1 -- 172.21.15.59:0/3690160034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888032c00 con 0x7fc89815b450 2024-04-23T14:43:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 -- 172.21.15.59:0/3690160034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 msgr2=0x7fc8980b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 --2- 172.21.15.59:0/3690160034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980b4630 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fc8980af4a0 tx=0x7fc88802c970 comp rx=0 tx=0).stop 2024-04-23T14:43:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 -- 172.21.15.59:0/3690160034 shutdown_connections 2024-04-23T14:43:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 --2- 172.21.15.59:0/3690160034 >> v2:172.21.15.76:3300/0 conn(0x7fc89806f3f0 0x7fc8980cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 --2- 172.21.15.59:0/3690160034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980b4630 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 -- 172.21.15.59:0/3690160034 >> 172.21.15.59:0/3690160034 conn(0x7fc8980b21a0 msgr2=0x7fc8980b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:43.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 -- 172.21.15.59:0/3690160034 shutdown_connections 2024-04-23T14:43:43.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.156+0000 7fc89d000640 1 -- 172.21.15.59:0/3690160034 wait complete. 2024-04-23T14:43:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.158+0000 7fc89d000640 1 Processor -- start 2024-04-23T14:43:43.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.158+0000 7fc89d000640 1 -- start start 2024-04-23T14:43:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc89d000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc89806f3f0 0x7fc8981f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc89d000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8980b3080 0x7fc8980c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc89d000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc89d000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc8980b4ff0 con 0x7fc89815b450 2024-04-23T14:43:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc89d000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc89815b830 con 0x7fc8980b3080 2024-04-23T14:43:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc89d000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc8980cd590 con 0x7fc89806f3f0 2024-04-23T14:43:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc897e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc897e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55916/0 (socket says 172.21.15.59:55916) 2024-04-23T14:43:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.159+0000 7fc897e00640 1 -- 172.21.15.59:0/3048102826 learned_addr learned my addr 172.21.15.59:0/3048102826 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc897400640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc89806f3f0 0x7fc8981f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc88fe00640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8980b3080 0x7fc8980c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc897e00640 1 -- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc89806f3f0 msgr2=0x7fc8981f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc897e00640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc89806f3f0 0x7fc8981f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc897e00640 1 -- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8980b3080 msgr2=0x7fc8980c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc897e00640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8980b3080 0x7fc8980c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc897e00640 1 -- 172.21.15.59:0/3048102826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8981f55c0 con 0x7fc89815b450 2024-04-23T14:43:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc897e00640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980c5bb0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c780 tx=0x7fc89000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.160+0000 7fc88f400640 1 -- 172.21.15.59:0/3048102826 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890004b60 con 0x7fc89815b450 2024-04-23T14:43:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.161+0000 7fc88f400640 1 -- 172.21.15.59:0/3048102826 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc890004d00 con 0x7fc89815b450 2024-04-23T14:43:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.161+0000 7fc88f400640 1 -- 172.21.15.59:0/3048102826 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89000aa90 con 0x7fc89815b450 2024-04-23T14:43:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.161+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8980c6470 con 0x7fc89815b450 2024-04-23T14:43:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.161+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8980c6a30 con 0x7fc89815b450 2024-04-23T14:43:43.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.162+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8981f5790 con 0x7fc89815b450 2024-04-23T14:43:43.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.163+0000 7fc88f400640 1 -- 172.21.15.59:0/3048102826 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc89000ac30 con 0x7fc89815b450 2024-04-23T14:43:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.164+0000 7fc88f400640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc87406c8f0 0x7fc87406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.164+0000 7fc88f400640 1 -- 172.21.15.59:0/3048102826 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fc89008be90 con 0x7fc89815b450 2024-04-23T14:43:43.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.164+0000 7fc897400640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc87406c8f0 0x7fc87406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.165+0000 7fc897400640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc87406c8f0 0x7fc87406edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc888037000 tx=0x7fc888008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:43.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.168+0000 7fc88f400640 1 -- 172.21.15.59:0/3048102826 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc890020070 con 0x7fc89815b450 2024-04-23T14:43:43.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.369+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc89804fd90 con 0x7fc87406c8f0 2024-04-23T14:43:43.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.385+0000 7fc88f400640 1 -- 172.21.15.59:0/3048102826 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252436 (secure 0 0 0) 0x7fc89804fd90 con 0x7fc87406c8f0 2024-04-23T14:43:43.390 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:43.391 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.390+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc87406c8f0 msgr2=0x7fc87406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.390+0000 7fc89d000640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc87406c8f0 0x7fc87406edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc888037000 tx=0x7fc888008040 comp rx=0 tx=0).stop 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 msgr2=0x7fc8980c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980c5bb0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c780 tx=0x7fc89000cc40 comp rx=0 tx=0).stop 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 shutdown_connections 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc87406c8f0 0x7fc87406edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc89815b450 0x7fc8980c5bb0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8980b3080 0x7fc8980c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 --2- 172.21.15.59:0/3048102826 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc89806f3f0 0x7fc8981f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 >> 172.21.15.59:0/3048102826 conn(0x7fc8980b21a0 msgr2=0x7fc8980d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 shutdown_connections 2024-04-23T14:43:43.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.391+0000 7fc89d000640 1 -- 172.21.15.59:0/3048102826 wait complete. 2024-04-23T14:43:43.413 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-04-23T14:43:42.616338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261476,"kb_used_data":12020,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713244,"statfs":{"total":773094113280,"available":772826361856,"internally_reserved":0,"allocated":12308480,"data_stored":5983424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001938"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433378+0000","last_change":"2024-04-23T14:43:38.425583+0000","last_active":"2024-04-23T14:43:38.433378+0000","last_peered":"2024-04-23T14:43:38.433378+0000","last_clean":"2024-04-23T14:43:38.433378+0000","last_became_active":"2024-04-23T14:43:38.425299+0000","last_became_peered":"2024-04-23T14:43:38.425299+0000","last_unstale":"2024-04-23T14:43:38.433378+0000","last_undegraded":"2024-04-23T14:43:38.433378+0000","last_fullsized":"2024-04-23T14:43:38.433378+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:16.911838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431681+0000","last_change":"2024-04-23T14:43:38.424369+0000","last_active":"2024-04-23T14:43:39.431681+0000","last_peered":"2024-04-23T14:43:39.431681+0000","last_clean":"2024-04-23T14:43:39.431681+0000","last_became_active":"2024-04-23T14:43:38.423891+0000","last_became_peered":"2024-04-23T14:43:38.423891+0000","last_unstale":"2024-04-23T14:43:39.431681+0000","last_undegraded":"2024-04-23T14:43:39.431681+0000","last_fullsized":"2024-04-23T14:43:39.431681+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:44.483663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.447610+0000","last_change":"2024-04-23T14:43:38.446596+0000","last_active":"2024-04-23T14:43:38.447610+0000","last_peered":"2024-04-23T14:43:38.447610+0000","last_clean":"2024-04-23T14:43:38.447610+0000","last_became_active":"2024-04-23T14:43:38.446103+0000","last_became_peered":"2024-04-23T14:43:38.446103+0000","last_unstale":"2024-04-23T14:43:38.447610+0000","last_undegraded":"2024-04-23T14:43:38.447610+0000","last_fullsized":"2024-04-23T14:43:38.447610+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:21.218195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439807+0000","last_change":"2024-04-23T14:43:38.423704+0000","last_active":"2024-04-23T14:43:38.439807+0000","last_peered":"2024-04-23T14:43:38.439807+0000","last_clean":"2024-04-23T14:43:38.439807+0000","last_became_active":"2024-04-23T14:43:38.420553+0000","last_became_peered":"2024-04-23T14:43:38.420553+0000","last_unstale":"2024-04-23T14:43:38.439807+0000","last_undegraded":"2024-04-23T14:43:38.439807+0000","last_fullsized":"2024-04-23T14:43:38.439807+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:58:50.829518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439528+0000","last_change":"2024-04-23T14:43:38.432262+0000","last_active":"2024-04-23T14:43:38.439528+0000","last_peered":"2024-04-23T14:43:38.439528+0000","last_clean":"2024-04-23T14:43:38.439528+0000","last_became_active":"2024-04-23T14:43:38.431889+0000","last_became_peered":"2024-04-23T14:43:38.431889+0000","last_unstale":"2024-04-23T14:43:38.439528+0000","last_undegraded":"2024-04-23T14:43:38.439528+0000","last_fullsized":"2024-04-23T14:43:38.439528+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:48.200293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.431111+0000","last_change":"2024-04-23T14:43:38.425701+0000","last_active":"2024-04-23T14:43:38.431111+0000","last_peered":"2024-04-23T14:43:38.431111+0000","last_clean":"2024-04-23T14:43:38.431111+0000","last_became_active":"2024-04-23T14:43:38.425437+0000","last_became_peered":"2024-04-23T14:43:38.425437+0000","last_unstale":"2024-04-23T14:43:38.431111+0000","last_undegraded":"2024-04-23T14:43:38.431111+0000","last_fullsized":"2024-04-23T14:43:38.431111+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:14:24.232135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432263+0000","last_change":"2024-04-23T14:43:38.425675+0000","last_active":"2024-04-23T14:43:38.432263+0000","last_peered":"2024-04-23T14:43:38.432263+0000","last_clean":"2024-04-23T14:43:38.432263+0000","last_became_active":"2024-04-23T14:43:38.425117+0000","last_became_peered":"2024-04-23T14:43:38.425117+0000","last_unstale":"2024-04-23T14:43:38.432263+0000","last_undegraded":"2024-04-23T14:43:38.432263+0000","last_fullsized":"2024-04-23T14:43:38.432263+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:48:35.913730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445899+0000","last_change":"2024-04-23T14:43:38.443325+0000","last_active":"2024-04-23T14:43:38.445899+0000","last_peered":"2024-04-23T14:43:38.445899+0000","last_clean":"2024-04-23T14:43:38.445899+0000","last_became_active":"2024-04-23T14:43:38.442945+0000","last_became_peered":"2024-04-23T14:43:38.442945+0000","last_unstale":"2024-04-23T14:43:38.445899+0000","last_undegraded":"2024-04-23T14:43:38.445899+0000","last_fullsized":"2024-04-23T14:43:38.445899+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:39.671402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433358+0000","last_change":"2024-04-23T14:43:38.425381+0000","last_active":"2024-04-23T14:43:38.433358+0000","last_peered":"2024-04-23T14:43:38.433358+0000","last_clean":"2024-04-23T14:43:38.433358+0000","last_became_active":"2024-04-23T14:43:38.424503+0000","last_became_peered":"2024-04-23T14:43:38.424503+0000","last_unstale":"2024-04-23T14:43:38.433358+0000","last_undegraded":"2024-04-23T14:43:38.433358+0000","last_fullsized":"2024-04-23T14:43:38.433358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:15.924427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.442437+0000","last_change":"2024-04-23T14:43:38.435393+0000","last_active":"2024-04-23T14:43:38.442437+0000","last_peered":"2024-04-23T14:43:38.442437+0000","last_clean":"2024-04-23T14:43:38.442437+0000","last_became_active":"2024-04-23T14:43:38.435059+0000","last_became_peered":"2024-04-23T14:43:38.435059+0000","last_unstale":"2024-04-23T14:43:38.442437+0000","last_undegraded":"2024-04-23T14:43:38.442437+0000","last_fullsized":"2024-04-23T14:43:38.442437+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:24.042337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432089+0000","last_change":"2024-04-23T14:43:38.428967+0000","last_active":"2024-04-23T14:43:39.432089+0000","last_peered":"2024-04-23T14:43:39.432089+0000","last_clean":"2024-04-23T14:43:39.432089+0000","last_became_active":"2024-04-23T14:43:38.428659+0000","last_became_peered":"2024-04-23T14:43:38.428659+0000","last_unstale":"2024-04-23T14:43:39.432089+0000","last_undegraded":"2024-04-23T14:43:39.432089+0000","last_fullsized":"2024-04-23T14:43:39.432089+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:23.492135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443188+0000","last_change":"2024-04-23T14:43:38.438522+0000","last_active":"2024-04-23T14:43:38.443188+0000","last_peered":"2024-04-23T14:43:38.443188+0000","last_clean":"2024-04-23T14:43:38.443188+0000","last_became_active":"2024-04-23T14:43:38.438168+0000","last_became_peered":"2024-04-23T14:43:38.438168+0000","last_unstale":"2024-04-23T14:43:38.443188+0000","last_undegraded":"2024-04-23T14:43:38.443188+0000","last_fullsized":"2024-04-23T14:43:38.443188+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:10.181288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445498+0000","last_change":"2024-04-23T14:43:38.442247+0000","last_active":"2024-04-23T14:43:38.445498+0000","last_peered":"2024-04-23T14:43:38.445498+0000","last_clean":"2024-04-23T14:43:38.445498+0000","last_became_active":"2024-04-23T14:43:38.441811+0000","last_became_peered":"2024-04-23T14:43:38.441811+0000","last_unstale":"2024-04-23T14:43:38.445498+0000","last_undegraded":"2024-04-23T14:43:38.445498+0000","last_fullsized":"2024-04-23T14:43:38.445498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:24:23.492135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443310+0000","last_change":"2024-04-23T14:43:38.439372+0000","last_active":"2024-04-23T14:43:38.443310+0000","last_peered":"2024-04-23T14:43:38.443310+0000","last_clean":"2024-04-23T14:43:38.443310+0000","last_became_active":"2024-04-23T14:43:38.438897+0000","last_became_peered":"2024-04-23T14:43:38.438897+0000","last_unstale":"2024-04-23T14:43:38.443310+0000","last_undegraded":"2024-04-23T14:43:38.443310+0000","last_fullsized":"2024-04-23T14:43:38.443310+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:23.605727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432939+0000","last_change":"2024-04-23T14:43:38.427723+0000","last_active":"2024-04-23T14:43:38.432939+0000","last_peered":"2024-04-23T14:43:38.432939+0000","last_clean":"2024-04-23T14:43:38.432939+0000","last_became_active":"2024-04-23T14:43:38.426770+0000","last_became_peered":"2024-04-23T14:43:38.426770+0000","last_unstale":"2024-04-23T14:43:38.432939+0000","last_undegraded":"2024-04-23T14:43:38.432939+0000","last_fullsized":"2024-04-23T14:43:38.432939+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:28.624694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443178+0000","last_change":"2024-04-23T14:43:38.438336+0000","last_active":"2024-04-23T14:43:38.443178+0000","last_peered":"2024-04-23T14:43:38.443178+0000","last_clean":"2024-04-23T14:43:38.443178+0000","last_became_active":"2024-04-23T14:43:38.438042+0000","last_became_peered":"2024-04-23T14:43:38.438042+0000","last_unstale":"2024-04-23T14:43:38.443178+0000","last_undegraded":"2024-04-23T14:43:38.443178+0000","last_fullsized":"2024-04-23T14:43:38.443178+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:26.911748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432135+0000","last_change":"2024-04-23T14:43:38.427603+0000","last_active":"2024-04-23T14:43:39.432135+0000","last_peered":"2024-04-23T14:43:39.432135+0000","last_clean":"2024-04-23T14:43:39.432135+0000","last_became_active":"2024-04-23T14:43:38.427284+0000","last_became_peered":"2024-04-23T14:43:38.427284+0000","last_unstale":"2024-04-23T14:43:39.432135+0000","last_undegraded":"2024-04-23T14:43:39.432135+0000","last_fullsized":"2024-04-23T14:43:39.432135+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:13.182967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429519+0000","last_change":"2024-04-23T14:43:38.425991+0000","last_active":"2024-04-23T14:43:39.429519+0000","last_peered":"2024-04-23T14:43:39.429519+0000","last_clean":"2024-04-23T14:43:39.429519+0000","last_became_active":"2024-04-23T14:43:38.422815+0000","last_became_peered":"2024-04-23T14:43:38.422815+0000","last_unstale":"2024-04-23T14:43:39.429519+0000","last_undegraded":"2024-04-23T14:43:39.429519+0000","last_fullsized":"2024-04-23T14:43:39.429519+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:34.339244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.446360+0000","last_change":"2024-04-23T14:43:38.442869+0000","last_active":"2024-04-23T14:43:38.446360+0000","last_peered":"2024-04-23T14:43:38.446360+0000","last_clean":"2024-04-23T14:43:38.446360+0000","last_became_active":"2024-04-23T14:43:38.442367+0000","last_became_peered":"2024-04-23T14:43:38.442367+0000","last_unstale":"2024-04-23T14:43:38.446360+0000","last_undegraded":"2024-04-23T14:43:38.446360+0000","last_fullsized":"2024-04-23T14:43:38.446360+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:13.182967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431498+0000","last_change":"2024-04-23T14:43:38.438917+0000","last_active":"2024-04-23T14:43:39.431498+0000","last_peered":"2024-04-23T14:43:39.431498+0000","last_clean":"2024-04-23T14:43:39.431498+0000","last_became_active":"2024-04-23T14:43:38.438619+0000","last_became_peered":"2024-04-23T14:43:38.438619+0000","last_unstale":"2024-04-23T14:43:39.431498+0000","last_undegraded":"2024-04-23T14:43:39.431498+0000","last_fullsized":"2024-04-23T14:43:39.431498+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:02.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432609+0000","last_change":"2024-04-23T14:43:38.432925+0000","last_active":"2024-04-23T14:43:39.432609+0000","last_peered":"2024-04-23T14:43:39.432609+0000","last_clean":"2024-04-23T14:43:39.432609+0000","last_became_active":"2024-04-23T14:43:38.432255+0000","last_became_peered":"2024-04-23T14:43:38.432255+0000","last_unstale":"2024-04-23T14:43:39.432609+0000","last_undegraded":"2024-04-23T14:43:39.432609+0000","last_fullsized":"2024-04-23T14:43:39.432609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:34.339244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426856+0000","last_change":"2024-04-23T14:43:38.429890+0000","last_active":"2024-04-23T14:43:39.426856+0000","last_peered":"2024-04-23T14:43:39.426856+0000","last_clean":"2024-04-23T14:43:39.426856+0000","last_became_active":"2024-04-23T14:43:38.429085+0000","last_became_peered":"2024-04-23T14:43:38.429085+0000","last_unstale":"2024-04-23T14:43:39.426856+0000","last_undegraded":"2024-04-23T14:43:39.426856+0000","last_fullsized":"2024-04-23T14:43:39.426856+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:20:10.181288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430528+0000","last_change":"2024-04-23T14:43:38.436846+0000","last_active":"2024-04-23T14:43:39.430528+0000","last_peered":"2024-04-23T14:43:39.430528+0000","last_clean":"2024-04-23T14:43:39.430528+0000","last_became_active":"2024-04-23T14:43:38.436603+0000","last_became_peered":"2024-04-23T14:43:38.436603+0000","last_unstale":"2024-04-23T14:43:39.430528+0000","last_undegraded":"2024-04-23T14:43:39.430528+0000","last_fullsized":"2024-04-23T14:43:39.430528+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:37.437229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425662+0000","last_change":"2024-04-23T14:43:38.438926+0000","last_active":"2024-04-23T14:43:39.425662+0000","last_peered":"2024-04-23T14:43:39.425662+0000","last_clean":"2024-04-23T14:43:39.425662+0000","last_became_active":"2024-04-23T14:43:38.438638+0000","last_became_peered":"2024-04-23T14:43:38.438638+0000","last_unstale":"2024-04-23T14:43:39.425662+0000","last_undegraded":"2024-04-23T14:43:39.425662+0000","last_fullsized":"2024-04-23T14:43:39.425662+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:14:15.924427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430609+0000","last_change":"2024-04-23T14:43:38.430684+0000","last_active":"2024-04-23T14:43:39.430609+0000","last_peered":"2024-04-23T14:43:39.430609+0000","last_clean":"2024-04-23T14:43:39.430609+0000","last_became_active":"2024-04-23T14:43:38.430412+0000","last_became_peered":"2024-04-23T14:43:38.430412+0000","last_unstale":"2024-04-23T14:43:39.430609+0000","last_undegraded":"2024-04-23T14:43:39.430609+0000","last_fullsized":"2024-04-23T14:43:39.430609+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:03.435484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.442225+0000","last_change":"2024-04-23T14:43:38.436193+0000","last_active":"2024-04-23T14:43:38.442225+0000","last_peered":"2024-04-23T14:43:38.442225+0000","last_clean":"2024-04-23T14:43:38.442225+0000","last_became_active":"2024-04-23T14:43:38.435832+0000","last_became_peered":"2024-04-23T14:43:38.435832+0000","last_unstale":"2024-04-23T14:43:38.442225+0000","last_undegraded":"2024-04-23T14:43:38.442225+0000","last_fullsized":"2024-04-23T14:43:38.442225+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:19:22.917967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432221+0000","last_change":"2024-04-23T14:43:38.429551+0000","last_active":"2024-04-23T14:43:38.432221+0000","last_peered":"2024-04-23T14:43:38.432221+0000","last_clean":"2024-04-23T14:43:38.432221+0000","last_became_active":"2024-04-23T14:43:38.429250+0000","last_became_peered":"2024-04-23T14:43:38.429250+0000","last_unstale":"2024-04-23T14:43:38.432221+0000","last_undegraded":"2024-04-23T14:43:38.432221+0000","last_fullsized":"2024-04-23T14:43:38.432221+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:16.866077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425851+0000","last_change":"2024-04-23T14:43:38.420978+0000","last_active":"2024-04-23T14:43:39.425851+0000","last_peered":"2024-04-23T14:43:39.425851+0000","last_clean":"2024-04-23T14:43:39.425851+0000","last_became_active":"2024-04-23T14:43:38.420687+0000","last_became_peered":"2024-04-23T14:43:38.420687+0000","last_unstale":"2024-04-23T14:43:39.425851+0000","last_undegraded":"2024-04-23T14:43:39.425851+0000","last_fullsized":"2024-04-23T14:43:39.425851+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:52:36.665403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431864+0000","last_change":"2024-04-23T14:43:38.440424+0000","last_active":"2024-04-23T14:43:39.431864+0000","last_peered":"2024-04-23T14:43:39.431864+0000","last_clean":"2024-04-23T14:43:39.431864+0000","last_became_active":"2024-04-23T14:43:38.439803+0000","last_became_peered":"2024-04-23T14:43:38.439803+0000","last_unstale":"2024-04-23T14:43:39.431864+0000","last_undegraded":"2024-04-23T14:43:39.431864+0000","last_fullsized":"2024-04-23T14:43:39.431864+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:10:23.605727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.431207+0000","last_change":"2024-04-23T14:43:38.425793+0000","last_active":"2024-04-23T14:43:38.431207+0000","last_peered":"2024-04-23T14:43:38.431207+0000","last_clean":"2024-04-23T14:43:38.431207+0000","last_became_active":"2024-04-23T14:43:38.425387+0000","last_became_peered":"2024-04-23T14:43:38.425387+0000","last_unstale":"2024-04-23T14:43:38.431207+0000","last_undegraded":"2024-04-23T14:43:38.431207+0000","last_fullsized":"2024-04-23T14:43:38.431207+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:25.560889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.434212+0000","last_change":"2024-04-23T14:43:38.426390+0000","last_active":"2024-04-23T14:43:38.434212+0000","last_peered":"2024-04-23T14:43:38.434212+0000","last_clean":"2024-04-23T14:43:38.434212+0000","last_became_active":"2024-04-23T14:43:38.426088+0000","last_became_peered":"2024-04-23T14:43:38.426088+0000","last_unstale":"2024-04-23T14:43:38.434212+0000","last_undegraded":"2024-04-23T14:43:38.434212+0000","last_fullsized":"2024-04-23T14:43:38.434212+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:52:36.665403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432081+0000","last_change":"2024-04-23T14:43:38.427501+0000","last_active":"2024-04-23T14:43:38.432081+0000","last_peered":"2024-04-23T14:43:38.432081+0000","last_clean":"2024-04-23T14:43:38.432081+0000","last_became_active":"2024-04-23T14:43:38.427115+0000","last_became_peered":"2024-04-23T14:43:38.427115+0000","last_unstale":"2024-04-23T14:43:38.432081+0000","last_undegraded":"2024-04-23T14:43:38.432081+0000","last_fullsized":"2024-04-23T14:43:38.432081+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:42.374475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433581+0000","last_change":"2024-04-23T14:43:38.431007+0000","last_active":"2024-04-23T14:43:38.433581+0000","last_peered":"2024-04-23T14:43:38.433581+0000","last_clean":"2024-04-23T14:43:38.433581+0000","last_became_active":"2024-04-23T14:43:38.430728+0000","last_became_peered":"2024-04-23T14:43:38.430728+0000","last_unstale":"2024-04-23T14:43:38.433581+0000","last_undegraded":"2024-04-23T14:43:38.433581+0000","last_fullsized":"2024-04-23T14:43:38.433581+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:27.583007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.441879+0000","last_change":"2024-04-23T14:43:37.398296+0000","last_active":"2024-04-23T14:43:38.441879+0000","last_peered":"2024-04-23T14:43:38.441879+0000","last_clean":"2024-04-23T14:43:38.441879+0000","last_became_active":"2024-04-23T14:43:37.397990+0000","last_became_peered":"2024-04-23T14:43:37.397990+0000","last_unstale":"2024-04-23T14:43:38.441879+0000","last_undegraded":"2024-04-23T14:43:38.441879+0000","last_fullsized":"2024-04-23T14:43:38.441879+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:47.886102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430441+0000","last_change":"2024-04-23T14:43:38.437869+0000","last_active":"2024-04-23T14:43:39.430441+0000","last_peered":"2024-04-23T14:43:39.430441+0000","last_clean":"2024-04-23T14:43:39.430441+0000","last_became_active":"2024-04-23T14:43:38.437082+0000","last_became_peered":"2024-04-23T14:43:38.437082+0000","last_unstale":"2024-04-23T14:43:39.430441+0000","last_undegraded":"2024-04-23T14:43:39.430441+0000","last_fullsized":"2024-04-23T14:43:39.430441+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:36.121951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439124+0000","last_change":"2024-04-23T14:43:37.399408+0000","last_active":"2024-04-23T14:43:38.439124+0000","last_peered":"2024-04-23T14:43:38.439124+0000","last_clean":"2024-04-23T14:43:38.439124+0000","last_became_active":"2024-04-23T14:43:37.399089+0000","last_became_peered":"2024-04-23T14:43:37.399089+0000","last_unstale":"2024-04-23T14:43:38.439124+0000","last_undegraded":"2024-04-23T14:43:38.439124+0000","last_fullsized":"2024-04-23T14:43:38.439124+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:44.679192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.431412+0000","last_change":"2024-04-23T14:43:38.425098+0000","last_active":"2024-04-23T14:43:38.431412+0000","last_peered":"2024-04-23T14:43:38.431412+0000","last_clean":"2024-04-23T14:43:38.431412+0000","last_became_active":"2024-04-23T14:43:38.424830+0000","last_became_peered":"2024-04-23T14:43:38.424830+0000","last_unstale":"2024-04-23T14:43:38.431412+0000","last_undegraded":"2024-04-23T14:43:38.431412+0000","last_fullsized":"2024-04-23T14:43:38.431412+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:33.803734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.437989+0000","last_change":"2024-04-23T14:43:37.381509+0000","last_active":"2024-04-23T14:43:38.437989+0000","last_peered":"2024-04-23T14:43:38.437989+0000","last_clean":"2024-04-23T14:43:38.437989+0000","last_became_active":"2024-04-23T14:43:37.381054+0000","last_became_peered":"2024-04-23T14:43:37.381054+0000","last_unstale":"2024-04-23T14:43:38.437989+0000","last_undegraded":"2024-04-23T14:43:38.437989+0000","last_fullsized":"2024-04-23T14:43:38.437989+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:50:14.241544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.446380+0000","last_change":"2024-04-23T14:43:38.443297+0000","last_active":"2024-04-23T14:43:38.446380+0000","last_peered":"2024-04-23T14:43:38.446380+0000","last_clean":"2024-04-23T14:43:38.446380+0000","last_became_active":"2024-04-23T14:43:38.442960+0000","last_became_peered":"2024-04-23T14:43:38.442960+0000","last_unstale":"2024-04-23T14:43:38.446380+0000","last_undegraded":"2024-04-23T14:43:38.446380+0000","last_fullsized":"2024-04-23T14:43:38.446380+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:34.339244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.440105+0000","last_change":"2024-04-23T14:43:37.389936+0000","last_active":"2024-04-23T14:43:38.440105+0000","last_peered":"2024-04-23T14:43:38.440105+0000","last_clean":"2024-04-23T14:43:38.440105+0000","last_became_active":"2024-04-23T14:43:37.389643+0000","last_became_peered":"2024-04-23T14:43:37.389643+0000","last_unstale":"2024-04-23T14:43:38.440105+0000","last_undegraded":"2024-04-23T14:43:38.440105+0000","last_fullsized":"2024-04-23T14:43:38.440105+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:12:58.319727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.434539+0000","last_change":"2024-04-23T14:43:38.426535+0000","last_active":"2024-04-23T14:43:38.434539+0000","last_peered":"2024-04-23T14:43:38.434539+0000","last_clean":"2024-04-23T14:43:38.434539+0000","last_became_active":"2024-04-23T14:43:38.426223+0000","last_became_peered":"2024-04-23T14:43:38.426223+0000","last_unstale":"2024-04-23T14:43:38.434539+0000","last_undegraded":"2024-04-23T14:43:38.434539+0000","last_fullsized":"2024-04-23T14:43:38.434539+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:28:41.176649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.441408+0000","last_change":"2024-04-23T14:43:37.396839+0000","last_active":"2024-04-23T14:43:38.441408+0000","last_peered":"2024-04-23T14:43:38.441408+0000","last_clean":"2024-04-23T14:43:38.441408+0000","last_became_active":"2024-04-23T14:43:37.396428+0000","last_became_peered":"2024-04-23T14:43:37.396428+0000","last_unstale":"2024-04-23T14:43:38.441408+0000","last_undegraded":"2024-04-23T14:43:38.441408+0000","last_fullsized":"2024-04-23T14:43:38.441408+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:12:45.596979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431919+0000","last_change":"2024-04-23T14:43:38.425624+0000","last_active":"2024-04-23T14:43:39.431919+0000","last_peered":"2024-04-23T14:43:39.431919+0000","last_clean":"2024-04-23T14:43:39.431919+0000","last_became_active":"2024-04-23T14:43:38.423946+0000","last_became_peered":"2024-04-23T14:43:38.423946+0000","last_unstale":"2024-04-23T14:43:39.431919+0000","last_undegraded":"2024-04-23T14:43:39.431919+0000","last_fullsized":"2024-04-23T14:43:39.431919+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:00:23.088263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426176+0000","last_change":"2024-04-23T14:43:37.390987+0000","last_active":"2024-04-23T14:43:39.426176+0000","last_peered":"2024-04-23T14:43:39.426176+0000","last_clean":"2024-04-23T14:43:39.426176+0000","last_became_active":"2024-04-23T14:43:37.390665+0000","last_became_peered":"2024-04-23T14:43:37.390665+0000","last_unstale":"2024-04-23T14:43:39.426176+0000","last_undegraded":"2024-04-23T14:43:39.426176+0000","last_fullsized":"2024-04-23T14:43:39.426176+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:29:15.377239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433099+0000","last_change":"2024-04-23T14:43:38.417842+0000","last_active":"2024-04-23T14:43:38.433099+0000","last_peered":"2024-04-23T14:43:38.433099+0000","last_clean":"2024-04-23T14:43:38.433099+0000","last_became_active":"2024-04-23T14:43:38.416006+0000","last_became_peered":"2024-04-23T14:43:38.416006+0000","last_unstale":"2024-04-23T14:43:38.433099+0000","last_undegraded":"2024-04-23T14:43:38.433099+0000","last_fullsized":"2024-04-23T14:43:38.433099+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:41:41.960832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430761+0000","last_change":"2024-04-23T14:43:37.393128+0000","last_active":"2024-04-23T14:43:39.430761+0000","last_peered":"2024-04-23T14:43:39.430761+0000","last_clean":"2024-04-23T14:43:39.430761+0000","last_became_active":"2024-04-23T14:43:37.392718+0000","last_became_peered":"2024-04-23T14:43:37.392718+0000","last_unstale":"2024-04-23T14:43:39.430761+0000","last_undegraded":"2024-04-23T14:43:39.430761+0000","last_fullsized":"2024-04-23T14:43:39.430761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:05:18.274497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.434199+0000","last_change":"2024-04-23T14:43:38.426856+0000","last_active":"2024-04-23T14:43:38.434199+0000","last_peered":"2024-04-23T14:43:38.434199+0000","last_clean":"2024-04-23T14:43:38.434199+0000","last_became_active":"2024-04-23T14:43:38.426564+0000","last_became_peered":"2024-04-23T14:43:38.426564+0000","last_unstale":"2024-04-23T14:43:38.434199+0000","last_undegraded":"2024-04-23T14:43:38.434199+0000","last_fullsized":"2024-04-23T14:43:38.434199+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:02.655011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.404792+0000","last_change":"2024-04-23T14:43:37.398466+0000","last_active":"2024-04-23T14:43:38.404792+0000","last_peered":"2024-04-23T14:43:38.404792+0000","last_clean":"2024-04-23T14:43:38.404792+0000","last_became_active":"2024-04-23T14:43:37.397653+0000","last_became_peered":"2024-04-23T14:43:37.397653+0000","last_unstale":"2024-04-23T14:43:38.404792+0000","last_undegraded":"2024-04-23T14:43:38.404792+0000","last_fullsized":"2024-04-23T14:43:38.404792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:02.577062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.426475+0000","last_change":"2024-04-23T14:43:38.421478+0000","last_active":"2024-04-23T14:43:38.426475+0000","last_peered":"2024-04-23T14:43:38.426475+0000","last_clean":"2024-04-23T14:43:38.426475+0000","last_became_active":"2024-04-23T14:43:38.421151+0000","last_became_peered":"2024-04-23T14:43:38.421151+0000","last_unstale":"2024-04-23T14:43:38.426475+0000","last_undegraded":"2024-04-23T14:43:38.426475+0000","last_fullsized":"2024-04-23T14:43:38.426475+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:42:15.032645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405904+0000","last_change":"2024-04-23T14:43:37.396127+0000","last_active":"2024-04-23T14:43:38.405904+0000","last_peered":"2024-04-23T14:43:38.405904+0000","last_clean":"2024-04-23T14:43:38.405904+0000","last_became_active":"2024-04-23T14:43:37.395826+0000","last_became_peered":"2024-04-23T14:43:37.395826+0000","last_unstale":"2024-04-23T14:43:38.405904+0000","last_undegraded":"2024-04-23T14:43:38.405904+0000","last_fullsized":"2024-04-23T14:43:38.405904+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:17.382285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.430656+0000","last_change":"2024-04-23T14:43:38.428596+0000","last_active":"2024-04-23T14:43:39.430656+0000","last_peered":"2024-04-23T14:43:39.430656+0000","last_clean":"2024-04-23T14:43:39.430656+0000","last_became_active":"2024-04-23T14:43:38.426742+0000","last_became_peered":"2024-04-23T14:43:38.426742+0000","last_unstale":"2024-04-23T14:43:39.430656+0000","last_undegraded":"2024-04-23T14:43:39.430656+0000","last_fullsized":"2024-04-23T14:43:39.430656+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:43:21.218195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425870+0000","last_change":"2024-04-23T14:43:37.391615+0000","last_active":"2024-04-23T14:43:39.425870+0000","last_peered":"2024-04-23T14:43:39.425870+0000","last_clean":"2024-04-23T14:43:39.425870+0000","last_became_active":"2024-04-23T14:43:37.390345+0000","last_became_peered":"2024-04-23T14:43:37.390345+0000","last_unstale":"2024-04-23T14:43:39.425870+0000","last_undegraded":"2024-04-23T14:43:39.425870+0000","last_fullsized":"2024-04-23T14:43:39.425870+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:02.577062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425963+0000","last_change":"2024-04-23T14:43:38.421117+0000","last_active":"2024-04-23T14:43:39.425963+0000","last_peered":"2024-04-23T14:43:39.425963+0000","last_clean":"2024-04-23T14:43:39.425963+0000","last_became_active":"2024-04-23T14:43:38.420822+0000","last_became_peered":"2024-04-23T14:43:38.420822+0000","last_unstale":"2024-04-23T14:43:39.425963+0000","last_undegraded":"2024-04-23T14:43:39.425963+0000","last_fullsized":"2024-04-23T14:43:39.425963+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:28:41.176649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.435442+0000","last_change":"2024-04-23T14:43:37.398066+0000","last_active":"2024-04-23T14:43:38.435442+0000","last_peered":"2024-04-23T14:43:38.435442+0000","last_clean":"2024-04-23T14:43:38.435442+0000","last_became_active":"2024-04-23T14:43:37.397746+0000","last_became_peered":"2024-04-23T14:43:37.397746+0000","last_unstale":"2024-04-23T14:43:38.435442+0000","last_undegraded":"2024-04-23T14:43:38.435442+0000","last_fullsized":"2024-04-23T14:43:38.435442+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:32:11.839954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443371+0000","last_change":"2024-04-23T14:43:38.436801+0000","last_active":"2024-04-23T14:43:38.443371+0000","last_peered":"2024-04-23T14:43:38.443371+0000","last_clean":"2024-04-23T14:43:38.443371+0000","last_became_active":"2024-04-23T14:43:38.436479+0000","last_became_peered":"2024-04-23T14:43:38.436479+0000","last_unstale":"2024-04-23T14:43:38.443371+0000","last_undegraded":"2024-04-23T14:43:38.443371+0000","last_fullsized":"2024-04-23T14:43:38.443371+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:50.162787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445127+0000","last_change":"2024-04-23T14:43:37.399207+0000","last_active":"2024-04-23T14:43:38.445127+0000","last_peered":"2024-04-23T14:43:38.445127+0000","last_clean":"2024-04-23T14:43:38.445127+0000","last_became_active":"2024-04-23T14:43:37.398925+0000","last_became_peered":"2024-04-23T14:43:37.398925+0000","last_unstale":"2024-04-23T14:43:38.445127+0000","last_undegraded":"2024-04-23T14:43:38.445127+0000","last_fullsized":"2024-04-23T14:43:38.445127+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:38.440250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426101+0000","last_change":"2024-04-23T14:43:38.414115+0000","last_active":"2024-04-23T14:43:39.426101+0000","last_peered":"2024-04-23T14:43:39.426101+0000","last_clean":"2024-04-23T14:43:39.426101+0000","last_became_active":"2024-04-23T14:43:38.412682+0000","last_became_peered":"2024-04-23T14:43:38.412682+0000","last_unstale":"2024-04-23T14:43:39.426101+0000","last_undegraded":"2024-04-23T14:43:39.426101+0000","last_fullsized":"2024-04-23T14:43:39.426101+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:39:24.042337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.441447+0000","last_change":"2024-04-23T14:43:37.396663+0000","last_active":"2024-04-23T14:43:38.441447+0000","last_peered":"2024-04-23T14:43:38.441447+0000","last_clean":"2024-04-23T14:43:38.441447+0000","last_became_active":"2024-04-23T14:43:37.396385+0000","last_became_peered":"2024-04-23T14:43:37.396385+0000","last_unstale":"2024-04-23T14:43:38.441447+0000","last_undegraded":"2024-04-23T14:43:38.441447+0000","last_fullsized":"2024-04-23T14:43:38.441447+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:47.181215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426565+0000","last_change":"2024-04-23T14:43:38.439386+0000","last_active":"2024-04-23T14:43:39.426565+0000","last_peered":"2024-04-23T14:43:39.426565+0000","last_clean":"2024-04-23T14:43:39.426565+0000","last_became_active":"2024-04-23T14:43:38.437256+0000","last_became_peered":"2024-04-23T14:43:38.437256+0000","last_unstale":"2024-04-23T14:43:39.426565+0000","last_undegraded":"2024-04-23T14:43:39.426565+0000","last_fullsized":"2024-04-23T14:43:39.426565+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:19:22.917967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.428075+0000","last_change":"2024-04-23T14:43:37.396068+0000","last_active":"2024-04-23T14:43:39.428075+0000","last_peered":"2024-04-23T14:43:39.428075+0000","last_clean":"2024-04-23T14:43:39.428075+0000","last_became_active":"2024-04-23T14:43:37.395051+0000","last_became_peered":"2024-04-23T14:43:37.395051+0000","last_unstale":"2024-04-23T14:43:39.428075+0000","last_undegraded":"2024-04-23T14:43:39.428075+0000","last_fullsized":"2024-04-23T14:43:39.428075+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:01.586328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.427010+0000","last_change":"2024-04-23T14:43:38.423144+0000","last_active":"2024-04-23T14:43:39.427010+0000","last_peered":"2024-04-23T14:43:39.427010+0000","last_clean":"2024-04-23T14:43:39.427010+0000","last_became_active":"2024-04-23T14:43:38.422859+0000","last_became_peered":"2024-04-23T14:43:38.422859+0000","last_unstale":"2024-04-23T14:43:39.427010+0000","last_undegraded":"2024-04-23T14:43:39.427010+0000","last_fullsized":"2024-04-23T14:43:39.427010+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:19:58.747750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":20,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.442052+0000","last_change":"2024-04-23T14:43:37.391943+0000","last_active":"2024-04-23T14:43:38.442052+0000","last_peered":"2024-04-23T14:43:38.442052+0000","last_clean":"2024-04-23T14:43:38.442052+0000","last_became_active":"2024-04-23T14:43:37.391681+0000","last_became_peered":"2024-04-23T14:43:37.391681+0000","last_unstale":"2024-04-23T14:43:38.442052+0000","last_undegraded":"2024-04-23T14:43:38.442052+0000","last_fullsized":"2024-04-23T14:43:38.442052+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:17:25.911325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433773+0000","last_change":"2024-04-23T14:43:38.421354+0000","last_active":"2024-04-23T14:43:38.433773+0000","last_peered":"2024-04-23T14:43:38.433773+0000","last_clean":"2024-04-23T14:43:38.433773+0000","last_became_active":"2024-04-23T14:43:38.421080+0000","last_became_peered":"2024-04-23T14:43:38.421080+0000","last_unstale":"2024-04-23T14:43:38.433773+0000","last_undegraded":"2024-04-23T14:43:38.433773+0000","last_fullsized":"2024-04-23T14:43:38.433773+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:53.936303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":13,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.439412+0000","last_change":"2024-04-23T14:43:37.397956+0000","last_active":"2024-04-23T14:43:38.439412+0000","last_peered":"2024-04-23T14:43:38.439412+0000","last_clean":"2024-04-23T14:43:38.439412+0000","last_became_active":"2024-04-23T14:43:37.396553+0000","last_became_peered":"2024-04-23T14:43:37.396553+0000","last_unstale":"2024-04-23T14:43:38.439412+0000","last_undegraded":"2024-04-23T14:43:38.439412+0000","last_fullsized":"2024-04-23T14:43:38.439412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:31:14.821283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.438957+0000","last_change":"2024-04-23T14:43:38.435200+0000","last_active":"2024-04-23T14:43:38.438957+0000","last_peered":"2024-04-23T14:43:38.438957+0000","last_clean":"2024-04-23T14:43:38.438957+0000","last_became_active":"2024-04-23T14:43:38.434921+0000","last_became_peered":"2024-04-23T14:43:38.434921+0000","last_unstale":"2024-04-23T14:43:38.438957+0000","last_undegraded":"2024-04-23T14:43:38.438957+0000","last_fullsized":"2024-04-23T14:43:38.438957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:05:19.293575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445701+0000","last_change":"2024-04-23T14:43:37.389166+0000","last_active":"2024-04-23T14:43:38.445701+0000","last_peered":"2024-04-23T14:43:38.445701+0000","last_clean":"2024-04-23T14:43:38.445701+0000","last_became_active":"2024-04-23T14:43:37.388846+0000","last_became_peered":"2024-04-23T14:43:37.388846+0000","last_unstale":"2024-04-23T14:43:38.445701+0000","last_undegraded":"2024-04-23T14:43:38.445701+0000","last_fullsized":"2024-04-23T14:43:38.445701+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:15.342538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445957+0000","last_change":"2024-04-23T14:43:38.444056+0000","last_active":"2024-04-23T14:43:38.445957+0000","last_peered":"2024-04-23T14:43:38.445957+0000","last_clean":"2024-04-23T14:43:38.445957+0000","last_became_active":"2024-04-23T14:43:38.443817+0000","last_became_peered":"2024-04-23T14:43:38.443817+0000","last_unstale":"2024-04-23T14:43:38.445957+0000","last_undegraded":"2024-04-23T14:43:38.445957+0000","last_fullsized":"2024-04-23T14:43:38.445957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:21.145162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429013+0000","last_change":"2024-04-23T14:43:37.379621+0000","last_active":"2024-04-23T14:43:39.429013+0000","last_peered":"2024-04-23T14:43:39.429013+0000","last_clean":"2024-04-23T14:43:39.429013+0000","last_became_active":"2024-04-23T14:43:37.378871+0000","last_became_peered":"2024-04-23T14:43:37.378871+0000","last_unstale":"2024-04-23T14:43:39.429013+0000","last_undegraded":"2024-04-23T14:43:39.429013+0000","last_fullsized":"2024-04-23T14:43:39.429013+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:27:02.416406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443173+0000","last_change":"2024-04-23T14:43:38.438949+0000","last_active":"2024-04-23T14:43:38.443173+0000","last_peered":"2024-04-23T14:43:38.443173+0000","last_clean":"2024-04-23T14:43:38.443173+0000","last_became_active":"2024-04-23T14:43:38.438654+0000","last_became_peered":"2024-04-23T14:43:38.438654+0000","last_unstale":"2024-04-23T14:43:38.443173+0000","last_undegraded":"2024-04-23T14:43:38.443173+0000","last_fullsized":"2024-04-23T14:43:38.443173+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:40:44.483663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405907+0000","last_change":"2024-04-23T14:43:37.401006+0000","last_active":"2024-04-23T14:43:38.405907+0000","last_peered":"2024-04-23T14:43:38.405907+0000","last_clean":"2024-04-23T14:43:38.405907+0000","last_became_active":"2024-04-23T14:43:37.400680+0000","last_became_peered":"2024-04-23T14:43:37.400680+0000","last_unstale":"2024-04-23T14:43:38.405907+0000","last_undegraded":"2024-04-23T14:43:38.405907+0000","last_fullsized":"2024-04-23T14:43:38.405907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:09.201305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.447113+0000","last_change":"2024-04-23T14:43:38.446200+0000","last_active":"2024-04-23T14:43:38.447113+0000","last_peered":"2024-04-23T14:43:38.447113+0000","last_clean":"2024-04-23T14:43:38.447113+0000","last_became_active":"2024-04-23T14:43:38.445886+0000","last_became_peered":"2024-04-23T14:43:38.445886+0000","last_unstale":"2024-04-23T14:43:38.447113+0000","last_undegraded":"2024-04-23T14:43:38.447113+0000","last_fullsized":"2024-04-23T14:43:38.447113+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:58:36.121951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.438718+0000","last_change":"2024-04-23T14:43:37.382042+0000","last_active":"2024-04-23T14:43:38.438718+0000","last_peered":"2024-04-23T14:43:38.438718+0000","last_clean":"2024-04-23T14:43:38.438718+0000","last_became_active":"2024-04-23T14:43:37.381706+0000","last_became_peered":"2024-04-23T14:43:37.381706+0000","last_unstale":"2024-04-23T14:43:38.438718+0000","last_undegraded":"2024-04-23T14:43:38.438718+0000","last_fullsized":"2024-04-23T14:43:38.438718+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:38:44.296302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443345+0000","last_change":"2024-04-23T14:43:38.440348+0000","last_active":"2024-04-23T14:43:38.443345+0000","last_peered":"2024-04-23T14:43:38.443345+0000","last_clean":"2024-04-23T14:43:38.443345+0000","last_became_active":"2024-04-23T14:43:38.436703+0000","last_became_peered":"2024-04-23T14:43:38.436703+0000","last_unstale":"2024-04-23T14:43:38.443345+0000","last_undegraded":"2024-04-23T14:43:38.443345+0000","last_fullsized":"2024-04-23T14:43:38.443345+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:19:58.747750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405783+0000","last_change":"2024-04-23T14:43:37.390219+0000","last_active":"2024-04-23T14:43:38.405783+0000","last_peered":"2024-04-23T14:43:38.405783+0000","last_clean":"2024-04-23T14:43:38.405783+0000","last_became_active":"2024-04-23T14:43:37.389935+0000","last_became_peered":"2024-04-23T14:43:37.389935+0000","last_unstale":"2024-04-23T14:43:38.405783+0000","last_undegraded":"2024-04-23T14:43:38.405783+0000","last_fullsized":"2024-04-23T14:43:38.405783+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:01.502283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.445351+0000","last_change":"2024-04-23T14:43:38.441697+0000","last_active":"2024-04-23T14:43:38.445351+0000","last_peered":"2024-04-23T14:43:38.445351+0000","last_clean":"2024-04-23T14:43:38.445351+0000","last_became_active":"2024-04-23T14:43:38.441053+0000","last_became_peered":"2024-04-23T14:43:38.441053+0000","last_unstale":"2024-04-23T14:43:38.445351+0000","last_undegraded":"2024-04-23T14:43:38.445351+0000","last_fullsized":"2024-04-23T14:43:38.445351+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:00:23.088263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.406523+0000","last_change":"2024-04-23T14:43:37.382899+0000","last_active":"2024-04-23T14:43:38.406523+0000","last_peered":"2024-04-23T14:43:38.406523+0000","last_clean":"2024-04-23T14:43:38.406523+0000","last_became_active":"2024-04-23T14:43:37.382591+0000","last_became_peered":"2024-04-23T14:43:37.382591+0000","last_unstale":"2024-04-23T14:43:38.406523+0000","last_undegraded":"2024-04-23T14:43:38.406523+0000","last_fullsized":"2024-04-23T14:43:38.406523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:52.917225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426557+0000","last_change":"2024-04-23T14:43:38.416822+0000","last_active":"2024-04-23T14:43:39.426557+0000","last_peered":"2024-04-23T14:43:39.426557+0000","last_clean":"2024-04-23T14:43:39.426557+0000","last_became_active":"2024-04-23T14:43:38.416552+0000","last_became_peered":"2024-04-23T14:43:38.416552+0000","last_unstale":"2024-04-23T14:43:39.426557+0000","last_undegraded":"2024-04-23T14:43:39.426557+0000","last_fullsized":"2024-04-23T14:43:39.426557+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:26.911748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.404965+0000","last_change":"2024-04-23T14:43:37.400199+0000","last_active":"2024-04-23T14:43:38.404965+0000","last_peered":"2024-04-23T14:43:38.404965+0000","last_clean":"2024-04-23T14:43:38.404965+0000","last_became_active":"2024-04-23T14:43:37.399833+0000","last_became_peered":"2024-04-23T14:43:37.399833+0000","last_unstale":"2024-04-23T14:43:38.404965+0000","last_undegraded":"2024-04-23T14:43:38.404965+0000","last_fullsized":"2024-04-23T14:43:38.404965+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:02.908466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429746+0000","last_change":"2024-04-23T14:43:38.421412+0000","last_active":"2024-04-23T14:43:39.429746+0000","last_peered":"2024-04-23T14:43:39.429746+0000","last_clean":"2024-04-23T14:43:39.429746+0000","last_became_active":"2024-04-23T14:43:38.420856+0000","last_became_peered":"2024-04-23T14:43:38.420856+0000","last_unstale":"2024-04-23T14:43:39.429746+0000","last_undegraded":"2024-04-23T14:43:39.429746+0000","last_fullsized":"2024-04-23T14:43:39.429746+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:21.145162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426156+0000","last_change":"2024-04-23T14:43:37.393619+0000","last_active":"2024-04-23T14:43:39.426156+0000","last_peered":"2024-04-23T14:43:39.426156+0000","last_clean":"2024-04-23T14:43:39.426156+0000","last_became_active":"2024-04-23T14:43:37.393330+0000","last_became_peered":"2024-04-23T14:43:37.393330+0000","last_unstale":"2024-04-23T14:43:39.426156+0000","last_undegraded":"2024-04-23T14:43:39.426156+0000","last_fullsized":"2024-04-23T14:43:39.426156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:02.908466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.433334+0000","last_change":"2024-04-23T14:43:38.424909+0000","last_active":"2024-04-23T14:43:38.433334+0000","last_peered":"2024-04-23T14:43:38.433334+0000","last_clean":"2024-04-23T14:43:38.433334+0000","last_became_active":"2024-04-23T14:43:38.424142+0000","last_became_peered":"2024-04-23T14:43:38.424142+0000","last_unstale":"2024-04-23T14:43:38.433334+0000","last_undegraded":"2024-04-23T14:43:38.433334+0000","last_fullsized":"2024-04-23T14:43:38.433334+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:32:28.320297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.429765+0000","last_change":"2024-04-23T14:43:37.397787+0000","last_active":"2024-04-23T14:43:39.429765+0000","last_peered":"2024-04-23T14:43:39.429765+0000","last_clean":"2024-04-23T14:43:39.429765+0000","last_became_active":"2024-04-23T14:43:37.397440+0000","last_became_peered":"2024-04-23T14:43:37.397440+0000","last_unstale":"2024-04-23T14:43:39.429765+0000","last_undegraded":"2024-04-23T14:43:39.429765+0000","last_fullsized":"2024-04-23T14:43:39.429765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:44.679192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.435995+0000","last_change":"2024-04-23T14:43:38.416055+0000","last_active":"2024-04-23T14:43:38.435995+0000","last_peered":"2024-04-23T14:43:38.435995+0000","last_clean":"2024-04-23T14:43:38.435995+0000","last_became_active":"2024-04-23T14:43:38.415782+0000","last_became_peered":"2024-04-23T14:43:38.415782+0000","last_unstale":"2024-04-23T14:43:38.435995+0000","last_undegraded":"2024-04-23T14:43:38.435995+0000","last_fullsized":"2024-04-23T14:43:38.435995+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:30:50.538534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.409053+0000","last_change":"2024-04-23T14:43:37.398697+0000","last_active":"2024-04-23T14:43:38.409053+0000","last_peered":"2024-04-23T14:43:38.409053+0000","last_clean":"2024-04-23T14:43:38.409053+0000","last_became_active":"2024-04-23T14:43:37.398407+0000","last_became_peered":"2024-04-23T14:43:37.398407+0000","last_unstale":"2024-04-23T14:43:38.409053+0000","last_undegraded":"2024-04-23T14:43:38.409053+0000","last_fullsized":"2024-04-23T14:43:38.409053+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:41:52.734029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.428855+0000","last_change":"2024-04-23T14:43:38.429473+0000","last_active":"2024-04-23T14:43:39.428855+0000","last_peered":"2024-04-23T14:43:39.428855+0000","last_clean":"2024-04-23T14:43:39.428855+0000","last_became_active":"2024-04-23T14:43:38.429182+0000","last_became_peered":"2024-04-23T14:43:38.429182+0000","last_unstale":"2024-04-23T14:43:39.428855+0000","last_undegraded":"2024-04-23T14:43:39.428855+0000","last_fullsized":"2024-04-23T14:43:39.428855+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:39.671402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.426089+0000","last_change":"2024-04-23T14:43:37.395259+0000","last_active":"2024-04-23T14:43:39.426089+0000","last_peered":"2024-04-23T14:43:39.426089+0000","last_clean":"2024-04-23T14:43:39.426089+0000","last_became_active":"2024-04-23T14:43:37.394927+0000","last_became_peered":"2024-04-23T14:43:37.394927+0000","last_unstale":"2024-04-23T14:43:39.426089+0000","last_undegraded":"2024-04-23T14:43:39.426089+0000","last_fullsized":"2024-04-23T14:43:39.426089+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:08.314795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431831+0000","last_change":"2024-04-23T14:43:38.436770+0000","last_active":"2024-04-23T14:43:39.431831+0000","last_peered":"2024-04-23T14:43:39.431831+0000","last_clean":"2024-04-23T14:43:39.431831+0000","last_became_active":"2024-04-23T14:43:38.436516+0000","last_became_peered":"2024-04-23T14:43:38.436516+0000","last_unstale":"2024-04-23T14:43:39.431831+0000","last_undegraded":"2024-04-23T14:43:39.431831+0000","last_fullsized":"2024-04-23T14:43:39.431831+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:03:45.698270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.406600+0000","last_change":"2024-04-23T14:43:37.395910+0000","last_active":"2024-04-23T14:43:38.406600+0000","last_peered":"2024-04-23T14:43:38.406600+0000","last_clean":"2024-04-23T14:43:38.406600+0000","last_became_active":"2024-04-23T14:43:37.395587+0000","last_became_peered":"2024-04-23T14:43:37.395587+0000","last_unstale":"2024-04-23T14:43:38.406600+0000","last_undegraded":"2024-04-23T14:43:38.406600+0000","last_fullsized":"2024-04-23T14:43:38.406600+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:37:11.865890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.432310+0000","last_change":"2024-04-23T14:43:38.438404+0000","last_active":"2024-04-23T14:43:39.432310+0000","last_peered":"2024-04-23T14:43:39.432310+0000","last_clean":"2024-04-23T14:43:39.432310+0000","last_became_active":"2024-04-23T14:43:38.435446+0000","last_became_peered":"2024-04-23T14:43:38.435446+0000","last_unstale":"2024-04-23T14:43:39.432310+0000","last_undegraded":"2024-04-23T14:43:39.432310+0000","last_fullsized":"2024-04-23T14:43:39.432310+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:54:39.671402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.431745+0000","last_change":"2024-04-23T14:43:37.396854+0000","last_active":"2024-04-23T14:43:39.431745+0000","last_peered":"2024-04-23T14:43:39.431745+0000","last_clean":"2024-04-23T14:43:39.431745+0000","last_became_active":"2024-04-23T14:43:37.396531+0000","last_became_peered":"2024-04-23T14:43:37.396531+0000","last_unstale":"2024-04-23T14:43:39.431745+0000","last_undegraded":"2024-04-23T14:43:39.431745+0000","last_fullsized":"2024-04-23T14:43:39.431745+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:28:39.199117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.443165+0000","last_change":"2024-04-23T14:43:38.438162+0000","last_active":"2024-04-23T14:43:38.443165+0000","last_peered":"2024-04-23T14:43:38.443165+0000","last_clean":"2024-04-23T14:43:38.443165+0000","last_became_active":"2024-04-23T14:43:38.436395+0000","last_became_peered":"2024-04-23T14:43:38.436395+0000","last_unstale":"2024-04-23T14:43:38.443165+0000","last_undegraded":"2024-04-23T14:43:38.443165+0000","last_fullsized":"2024-04-23T14:43:38.443165+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:02.605406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.408195+0000","last_change":"2024-04-23T14:43:37.398005+0000","last_active":"2024-04-23T14:43:38.408195+0000","last_peered":"2024-04-23T14:43:38.408195+0000","last_clean":"2024-04-23T14:43:38.408195+0000","last_became_active":"2024-04-23T14:43:37.396171+0000","last_became_peered":"2024-04-23T14:43:37.396171+0000","last_unstale":"2024-04-23T14:43:38.408195+0000","last_undegraded":"2024-04-23T14:43:38.408195+0000","last_fullsized":"2024-04-23T14:43:38.408195+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:56.816922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.432987+0000","last_change":"2024-04-23T14:43:38.417445+0000","last_active":"2024-04-23T14:43:38.432987+0000","last_peered":"2024-04-23T14:43:38.432987+0000","last_clean":"2024-04-23T14:43:38.432987+0000","last_became_active":"2024-04-23T14:43:38.417168+0000","last_became_peered":"2024-04-23T14:43:38.417168+0000","last_unstale":"2024-04-23T14:43:38.432987+0000","last_undegraded":"2024-04-23T14:43:38.432987+0000","last_fullsized":"2024-04-23T14:43:38.432987+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:38.019894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.406480+0000","last_change":"2024-04-23T14:43:37.398203+0000","last_active":"2024-04-23T14:43:38.406480+0000","last_peered":"2024-04-23T14:43:38.406480+0000","last_clean":"2024-04-23T14:43:38.406480+0000","last_became_active":"2024-04-23T14:43:37.397184+0000","last_became_peered":"2024-04-23T14:43:37.397184+0000","last_unstale":"2024-04-23T14:43:38.406480+0000","last_undegraded":"2024-04-23T14:43:38.406480+0000","last_fullsized":"2024-04-23T14:43:38.406480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:41.182034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-23T14:43:39.425848+0000","last_change":"2024-04-23T14:43:38.437357+0000","last_active":"2024-04-23T14:43:39.425848+0000","last_peered":"2024-04-23T14:43:39.425848+0000","last_clean":"2024-04-23T14:43:39.425848+0000","last_became_active":"2024-04-23T14:43:38.437046+0000","last_became_peered":"2024-04-23T14:43:38.437046+0000","last_unstale":"2024-04-23T14:43:39.425848+0000","last_undegraded":"2024-04-23T14:43:39.425848+0000","last_fullsized":"2024-04-23T14:43:39.425848+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:37.353164+0000","last_clean_scrub_stamp":"2024-04-23T14:43:37.353164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:42:16.911838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.405833+0000","last_change":"2024-04-23T14:43:37.397741+0000","last_active":"2024-04-23T14:43:38.405833+0000","last_peered":"2024-04-23T14:43:38.405833+0000","last_clean":"2024-04-23T14:43:38.405833+0000","last_became_active":"2024-04-23T14:43:37.397327+0000","last_became_peered":"2024-04-23T14:43:37.397327+0000","last_unstale":"2024-04-23T14:43:38.405833+0000","last_undegraded":"2024-04-23T14:43:38.405833+0000","last_fullsized":"2024-04-23T14:43:38.405833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:43:36.334086+0000","last_clean_scrub_stamp":"2024-04-23T14:43:36.334086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:50:23.865313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":341,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-23T14:43:38.427669+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:43:38.427669+0000","last_peered":"2024-04-23T14:43:38.427669+0000","last_clean":"2024-04-23T14:43:38.427669+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:43:38.427669+0000","last_undegraded":"2024-04-23T14:43:38.427669+0000","last_fullsized":"2024-04-23T14:43:38.427669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1982464,"data_stored":1189930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":605316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1388544,"data_stored":599550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1380352,"data_stored":598946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":600628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":598946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1388544,"data_stored":599070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1994752,"data_stored":1191038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:43:43.417 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:43:43.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-23T14:43:43.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.674+0000 7fb6f9c00640 1 Processor -- start 2024-04-23T14:43:43.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.674+0000 7fb6f9c00640 1 -- start start 2024-04-23T14:43:43.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.676+0000 7fb6f9c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6f415ec70 0x7fb6f40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.676+0000 7fb6f9c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f40cf6d0 con 0x7fb6f40d3030 2024-04-23T14:43:43.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.676+0000 7fb6f9c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f40cf9b0 con 0x7fb6f40d24a0 2024-04-23T14:43:43.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.676+0000 7fb6f9c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f40d02e0 con 0x7fb6f415ec70 2024-04-23T14:43:43.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.676+0000 7fb6f3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6f415ec70 0x7fb6f40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.677+0000 7fb6f3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6f415ec70 0x7fb6f40b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55932/0 (socket says 172.21.15.59:55932) 2024-04-23T14:43:43.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.677+0000 7fb6f3e00640 1 -- 172.21.15.59:0/3611602273 learned_addr learned my addr 172.21.15.59:0/3611602273 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:43.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.677+0000 7fb6f3e00640 1 -- 172.21.15.59:0/3611602273 >> v1:172.21.15.76:6789/0 conn(0x7fb6f40d24a0 legacy=0x7fb6f40d2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:43.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.677+0000 7fb6f3e00640 1 -- 172.21.15.59:0/3611602273 >> v1:172.21.15.59:6789/0 conn(0x7fb6f40d3030 legacy=0x7fb6f406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.677+0000 7fb6f2a00640 1 -- 172.21.15.59:0/3611602273 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282698469 0 0) 0x7fb6f40cf6d0 con 0x7fb6f40d3030 2024-04-23T14:43:43.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.677+0000 7fb6f3e00640 1 -- 172.21.15.59:0/3611602273 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f40cfc90 con 0x7fb6f415ec70 2024-04-23T14:43:43.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.678+0000 7fb6f3e00640 1 --2- 172.21.15.59:0/3611602273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b3630 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f30f6f252863db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:43.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.679+0000 7fb6f2a00640 1 -- 172.21.15.59:0/3611602273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4037070 con 0x7fb6f415ec70 2024-04-23T14:43:43.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.679+0000 7fb6f2a00640 1 -- 172.21.15.59:0/3611602273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e4032440 con 0x7fb6f415ec70 2024-04-23T14:43:43.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.679+0000 7fb6f2a00640 1 -- 172.21.15.59:0/3611602273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4032810 con 0x7fb6f415ec70 2024-04-23T14:43:43.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.680+0000 7fb6f9c00640 1 -- 172.21.15.59:0/3611602273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f415ec70 msgr2=0x7fb6f40b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.680+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/3611602273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b3630 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e402a2a0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.680+0000 7fb6f9c00640 1 -- 172.21.15.59:0/3611602273 shutdown_connections 2024-04-23T14:43:43.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.680+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/3611602273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b3630 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.680+0000 7fb6f9c00640 1 -- 172.21.15.59:0/3611602273 >> 172.21.15.59:0/3611602273 conn(0x7fb6f408bf90 msgr2=0x7fb6f40d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:43.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.680+0000 7fb6f9c00640 1 -- 172.21.15.59:0/3611602273 shutdown_connections 2024-04-23T14:43:43.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.680+0000 7fb6f9c00640 1 -- 172.21.15.59:0/3611602273 wait complete. 2024-04-23T14:43:43.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.682+0000 7fb6f9c00640 1 Processor -- start 2024-04-23T14:43:43.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.682+0000 7fb6f9c00640 1 -- start start 2024-04-23T14:43:43.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f9c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6f40d24a0 0x7fb6f40b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f9c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f40d3030 0x7fb6f4157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f9c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f9c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f40cf260 con 0x7fb6f40d3030 2024-04-23T14:43:43.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f9c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f415f050 con 0x7fb6f415ec70 2024-04-23T14:43:43.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f9c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f40cf3e0 con 0x7fb6f40d24a0 2024-04-23T14:43:43.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f40d3030 0x7fb6f4157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f40d3030 0x7fb6f4157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55948/0 (socket says 172.21.15.59:55948) 2024-04-23T14:43:43.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f3400640 1 -- 172.21.15.59:0/1854703000 learned_addr learned my addr 172.21.15.59:0/1854703000 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:43.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.683+0000 7fb6f9200640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f3400640 1 -- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6f40d24a0 msgr2=0x7fb6f40b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f3e00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6f40d24a0 0x7fb6f40b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f3400640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6f40d24a0 0x7fb6f40b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f3400640 1 -- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6f415ec70 msgr2=0x7fb6f40b9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f3400640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f3400640 1 -- 172.21.15.59:0/1854703000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f40b8700 con 0x7fb6f40d3030 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f9200640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f3400640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f40d3030 0x7fb6f4157620 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c430 tx=0x7fb6e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6ebe00640 1 -- 172.21.15.59:0/1854703000 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0011790 con 0x7fb6f40d3030 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6ebe00640 1 -- 172.21.15.59:0/1854703000 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0017070 con 0x7fb6f40d3030 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6ebe00640 1 -- 172.21.15.59:0/1854703000 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0011bb0 con 0x7fb6f40d3030 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.684+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f4157b60 con 0x7fb6f40d3030 2024-04-23T14:43:43.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.685+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f40b7c20 con 0x7fb6f40d3030 2024-04-23T14:43:43.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.686+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f404fd90 con 0x7fb6f40d3030 2024-04-23T14:43:43.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.687+0000 7fb6ebe00640 1 -- 172.21.15.59:0/1854703000 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6e001c460 con 0x7fb6f40d3030 2024-04-23T14:43:43.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.688+0000 7fb6ebe00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d406c8f0 0x7fb6d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:43.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.688+0000 7fb6ebe00640 1 -- 172.21.15.59:0/1854703000 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fb6e008ba90 con 0x7fb6f40d3030 2024-04-23T14:43:43.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.688+0000 7fb6f3e00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d406c8f0 0x7fb6d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:43.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.689+0000 7fb6f3e00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d406c8f0 0x7fb6d406edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4034000 tx=0x7fb6e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:43.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.693+0000 7fb6ebe00640 1 -- 172.21.15.59:0/1854703000 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6e0059e20 con 0x7fb6f40d3030 2024-04-23T14:43:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.970+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb6f415daf0 con 0x7fb6f40d3030 2024-04-23T14:43:43.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.975+0000 7fb6ebe00640 1 -- 172.21.15.59:0/1854703000 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7fb6e0059720 con 0x7fb6f40d3030 2024-04-23T14:43:43.976 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-04-23T14:43:43.976 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+xSdmlFhcNhAAg6cgAosFpj15r4q57cCtpQ== 2024-04-23T14:43:43.976 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2024-04-23T14:43:43.977 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2024-04-23T14:43:43.977 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-23T14:43:43.977 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d406c8f0 msgr2=0x7fb6d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d406c8f0 0x7fb6d406edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4034000 tx=0x7fb6e402e000 comp rx=0 tx=0).stop 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f40d3030 msgr2=0x7fb6f4157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f40d3030 0x7fb6f4157620 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c430 tx=0x7fb6e000c8f0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 shutdown_connections 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb6d406c8f0 0x7fb6d406edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6f415ec70 0x7fb6f40b9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6f40d3030 0x7fb6f4157620 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 --2- 172.21.15.59:0/1854703000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6f40d24a0 0x7fb6f40b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 >> 172.21.15.59:0/1854703000 conn(0x7fb6f408bf90 msgr2=0x7fb6f415dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.979+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 shutdown_connections 2024-04-23T14:43:43.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:43.980+0000 7fb6f9c00640 1 -- 172.21.15.59:0/1854703000 wait complete. 2024-04-23T14:43:44.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:43:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.210+0000 7fa3f0a00640 1 Processor -- start 2024-04-23T14:43:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.210+0000 7fa3f0a00640 1 -- start start 2024-04-23T14:43:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.212+0000 7fa3f0a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa3ec15c3d0 0x7fa3ec0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.212+0000 7fa3f0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.212+0000 7fa3f0a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec160210 con 0x7fa3ec0b4690 2024-04-23T14:43:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.212+0000 7fa3f0a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ec1604f0 con 0x7fa3ec15c3d0 2024-04-23T14:43:44.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.212+0000 7fa3f0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ec1607d0 con 0x7fa3ec0cd590 2024-04-23T14:43:44.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.212+0000 7fa3eb200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa3ec0b4690 0x7fa3ec06b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40922/0 (socket says 172.21.15.59:40922) 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.212+0000 7fa3eb200640 1 -- 172.21.15.59:0/1608095243 learned_addr learned my addr 172.21.15.59:0/1608095243 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3ea800640 1 --2- 172.21.15.59:0/1608095243 >> v2:172.21.15.76:3300/0 conn(0x7fa3ec15c3d0 0x7fa3ec0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3e1e00640 1 --2- 172.21.15.59:0/1608095243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3e9e00640 1 -- 172.21.15.59:0/1608095243 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3505458149 0 0) 0x7fa3ec160210 con 0x7fa3ec0b4690 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3e9e00640 1 -- 172.21.15.59:0/1608095243 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3d0003520 con 0x7fa3ec0b4690 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3e1e00640 1 -- 172.21.15.59:0/1608095243 >> v2:172.21.15.76:3300/0 conn(0x7fa3ec15c3d0 msgr2=0x7fa3ec0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3e1e00640 1 --2- 172.21.15.59:0/1608095243 >> v2:172.21.15.76:3300/0 conn(0x7fa3ec15c3d0 0x7fa3ec0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3e9e00640 1 -- 172.21.15.59:0/1608095243 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3658844306 0 0) 0x7fa3d0003520 con 0x7fa3ec0b4690 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.213+0000 7fa3e1e00640 1 -- 172.21.15.59:0/1608095243 >> v1:172.21.15.59:6789/0 conn(0x7fa3ec0b4690 legacy=0x7fa3ec06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.214+0000 7fa3e1e00640 1 -- 172.21.15.59:0/1608095243 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec160ab0 con 0x7fa3ec0cd590 2024-04-23T14:43:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.214+0000 7fa3ea800640 1 --2- 172.21.15.59:0/1608095243 >> v2:172.21.15.76:3300/0 conn(0x7fa3ec15c3d0 0x7fa3ec0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:43:44.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.215+0000 7fa3e1e00640 1 --2- 172.21.15.59:0/1608095243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec0b4150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40097a0 tx=0x7fa3d402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d544493d975daf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:44.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.216+0000 7fa3e9e00640 1 -- 172.21.15.59:0/1608095243 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4037070 con 0x7fa3ec0cd590 2024-04-23T14:43:44.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.216+0000 7fa3e9e00640 1 -- 172.21.15.59:0/1608095243 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d402aba0 con 0x7fa3ec0cd590 2024-04-23T14:43:44.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.216+0000 7fa3e9e00640 1 -- 172.21.15.59:0/1608095243 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40019f0 con 0x7fa3ec0cd590 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.216+0000 7fa3f0a00640 1 -- 172.21.15.59:0/1608095243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 msgr2=0x7fa3ec0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.216+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/1608095243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec0b4150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40097a0 tx=0x7fa3d402a110 comp rx=0 tx=0).stop 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.217+0000 7fa3f0a00640 1 -- 172.21.15.59:0/1608095243 shutdown_connections 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.217+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/1608095243 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec0b4150 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.217+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/1608095243 >> v2:172.21.15.76:3300/0 conn(0x7fa3ec15c3d0 0x7fa3ec0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.217+0000 7fa3f0a00640 1 -- 172.21.15.59:0/1608095243 >> 172.21.15.59:0/1608095243 conn(0x7fa3ec0b21a0 msgr2=0x7fa3ec0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.217+0000 7fa3f0a00640 1 -- 172.21.15.59:0/1608095243 shutdown_connections 2024-04-23T14:43:44.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.217+0000 7fa3f0a00640 1 -- 172.21.15.59:0/1608095243 wait complete. 2024-04-23T14:43:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.219+0000 7fa3f0a00640 1 Processor -- start 2024-04-23T14:43:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.219+0000 7fa3f0a00640 1 -- start start 2024-04-23T14:43:44.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.220+0000 7fa3f0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3ec0b4690 0x7fa3ec1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3f0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3f0a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3ec15c3d0 0x7fa3ec156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3f0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec15c7b0 con 0x7fa3ec0b4690 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3f0a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec0b4b00 con 0x7fa3ec15c3d0 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3f0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ec15c930 con 0x7fa3ec0cd590 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3ec0b4690 0x7fa3ec1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3ec0b4690 0x7fa3ec1599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55954/0 (socket says 172.21.15.59:55954) 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 -- 172.21.15.59:0/3546058214 learned_addr learned my addr 172.21.15.59:0/3546058214 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 -- 172.21.15.59:0/3546058214 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 msgr2=0x7fa3ec155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 -- 172.21.15.59:0/3546058214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3ec15c3d0 msgr2=0x7fa3ec156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3ec15c3d0 0x7fa3ec156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.221+0000 7fa3ea800640 1 -- 172.21.15.59:0/3546058214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec15a370 con 0x7fa3ec0b4690 2024-04-23T14:43:44.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.222+0000 7fa3ea800640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3ec0b4690 0x7fa3ec1599c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00ebe0 tx=0x7fa3dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:44.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.222+0000 7fa3e3400640 1 -- 172.21.15.59:0/3546058214 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc017070 con 0x7fa3ec0b4690 2024-04-23T14:43:44.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.222+0000 7fa3e3400640 1 -- 172.21.15.59:0/3546058214 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3dc00cc30 con 0x7fa3ec0b4690 2024-04-23T14:43:44.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.222+0000 7fa3e3400640 1 -- 172.21.15.59:0/3546058214 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc013470 con 0x7fa3ec0b4690 2024-04-23T14:43:44.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.222+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec15a650 con 0x7fa3ec0b4690 2024-04-23T14:43:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.222+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ec0ad060 con 0x7fa3ec0b4690 2024-04-23T14:43:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.223+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ec04fd90 con 0x7fa3ec0b4690 2024-04-23T14:43:44.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.225+0000 7fa3e3400640 1 -- 172.21.15.59:0/3546058214 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3dc01a070 con 0x7fa3ec0b4690 2024-04-23T14:43:44.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.225+0000 7fa3e3400640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3c806c920 0x7fa3c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.226+0000 7fa3e3400640 1 -- 172.21.15.59:0/3546058214 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fa3dc08bea0 con 0x7fa3ec0b4690 2024-04-23T14:43:44.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.226+0000 7fa3e1e00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3c806c920 0x7fa3c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:44.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.227+0000 7fa3e1e00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3c806c920 0x7fa3c806ede0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4034000 tx=0x7fa3d402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:44.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.230+0000 7fa3e3400640 1 -- 172.21.15.59:0/3546058214 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3dc05a230 con 0x7fa3ec0b4690 2024-04-23T14:43:44.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.474+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3ec06e400 con 0x7fa3ec0b4690 2024-04-23T14:43:44.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.475+0000 7fa3e3400640 1 -- 172.21.15.59:0/3546058214 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5185 (secure 0 0 0) 0x7fa3dc059b30 con 0x7fa3ec0b4690 2024-04-23T14:43:44.477 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:44.477 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:43:40.947368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9210},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9210":{"gid":9210,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6837/1061483588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1061483588},{"type":"v1","addr":"172.21.15.59:6837","nonce":1061483588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9210,"qdb_cluster":[9210]},"id":8}]} 2024-04-23T14:43:44.477 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 100 2024-04-23T14:43:44.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3c806c920 msgr2=0x7fa3c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3c806c920 0x7fa3c806ede0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4034000 tx=0x7fa3d402c000 comp rx=0 tx=0).stop 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3ec0b4690 msgr2=0x7fa3ec1599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3ec0b4690 0x7fa3ec1599c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00ebe0 tx=0x7fa3dc00c520 comp rx=0 tx=0).stop 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 shutdown_connections 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa3c806c920 0x7fa3c806ede0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa3ec0cd590 0x7fa3ec155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3ec15c3d0 0x7fa3ec156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 --2- 172.21.15.59:0/3546058214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3ec0b4690 0x7fa3ec1599c0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 >> 172.21.15.59:0/3546058214 conn(0x7fa3ec0b21a0 msgr2=0x7fa3ec0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 shutdown_connections 2024-04-23T14:43:44.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:44.478+0000 7fa3f0a00640 1 -- 172.21.15.59:0/3546058214 wait complete. 2024-04-23T14:43:44.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:43:37.939514+0000', 'modified': '2024-04-23T14:43:40.947368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9210}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9210': {'gid': 9210, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6837/1061483588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 1061483588}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 1061483588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9210, 'qdb_cluster': [9210]} 2024-04-23T14:43:44.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:43:44.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:43:45.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.125+0000 7fbd8aa00640 1 Processor -- start 2024-04-23T14:43:45.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.125+0000 7fbd8aa00640 1 -- start start 2024-04-23T14:43:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd8aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd840af4b0 0x7fbd8415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd8aa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840af890 0x7fbd84144a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd8aa00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8415f3f0 con 0x7fbd840b3010 2024-04-23T14:43:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd8aa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbd84144f60 con 0x7fbd840af4b0 2024-04-23T14:43:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd8aa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8415f6d0 con 0x7fbd840af890 2024-04-23T14:43:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd89600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd840af4b0 0x7fbd8415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd89600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd840af4b0 0x7fbd8415ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55960/0 (socket says 172.21.15.59:55960) 2024-04-23T14:43:45.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd89600640 1 -- 172.21.15.59:0/1960653783 learned_addr learned my addr 172.21.15.59:0/1960653783 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd89600640 1 -- 172.21.15.59:0/1960653783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840af890 msgr2=0x7fbd84144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd89600640 1 --2- 172.21.15.59:0/1960653783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840af890 0x7fbd84144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.127+0000 7fbd89600640 1 -- 172.21.15.59:0/1960653783 >> v1:172.21.15.76:6789/0 conn(0x7fbd840b3010 legacy=0x7fbd840d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.128+0000 7fbd89600640 1 -- 172.21.15.59:0/1960653783 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8415f9b0 con 0x7fbd840af4b0 2024-04-23T14:43:45.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.129+0000 7fbd89600640 1 --2- 172.21.15.59:0/1960653783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd840af4b0 0x7fbd8415ed50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009870 tx=0x7fbd7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100d6b8401e5f246 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.129+0000 7fbd83e00640 1 -- 172.21.15.59:0/1960653783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74037070 con 0x7fbd840af4b0 2024-04-23T14:43:45.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.129+0000 7fbd83e00640 1 -- 172.21.15.59:0/1960653783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd74032440 con 0x7fbd840af4b0 2024-04-23T14:43:45.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.129+0000 7fbd83e00640 1 -- 172.21.15.59:0/1960653783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74032810 con 0x7fbd840af4b0 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.129+0000 7fbd8aa00640 1 -- 172.21.15.59:0/1960653783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd840af4b0 msgr2=0x7fbd8415ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.129+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/1960653783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd840af4b0 0x7fbd8415ed50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009870 tx=0x7fbd7402a210 comp rx=0 tx=0).stop 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.130+0000 7fbd8aa00640 1 -- 172.21.15.59:0/1960653783 shutdown_connections 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.130+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/1960653783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840af890 0x7fbd84144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.130+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/1960653783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd840af4b0 0x7fbd8415ed50 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.130+0000 7fbd8aa00640 1 -- 172.21.15.59:0/1960653783 >> 172.21.15.59:0/1960653783 conn(0x7fbd8406bdf0 msgr2=0x7fbd8406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.130+0000 7fbd8aa00640 1 -- 172.21.15.59:0/1960653783 shutdown_connections 2024-04-23T14:43:45.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.130+0000 7fbd8aa00640 1 -- 172.21.15.59:0/1960653783 wait complete. 2024-04-23T14:43:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.132+0000 7fbd8aa00640 1 Processor -- start 2024-04-23T14:43:45.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.132+0000 7fbd8aa00640 1 -- start start 2024-04-23T14:43:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd840af4b0 0x7fbd841f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840b3010 0x7fbd84145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd84150cd0 0x7fbd84151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd840d0e50 con 0x7fbd84150cd0 2024-04-23T14:43:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbd840d0cd0 con 0x7fbd840af4b0 2024-04-23T14:43:45.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbd840b34c0 con 0x7fbd840b3010 2024-04-23T14:43:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd84150cd0 0x7fbd84151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd84150cd0 0x7fbd84151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55972/0 (socket says 172.21.15.59:55972) 2024-04-23T14:43:45.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 -- 172.21.15.59:0/3947399766 learned_addr learned my addr 172.21.15.59:0/3947399766 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 -- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840b3010 msgr2=0x7fbd84145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd89600640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd840af4b0 0x7fbd841f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd88c00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840b3010 0x7fbd84145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840b3010 0x7fbd84145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 -- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd840af4b0 msgr2=0x7fbd841f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd840af4b0 0x7fbd841f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 -- 172.21.15.59:0/3947399766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8414ede0 con 0x7fbd84150cd0 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd89600640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd840af4b0 0x7fbd841f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd88c00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840b3010 0x7fbd84145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8a000640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd84150cd0 0x7fbd84151180 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00b9a0 tx=0x7fbd7c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd81600640 1 -- 172.21.15.59:0/3947399766 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c019070 con 0x7fbd84150cd0 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd81600640 1 -- 172.21.15.59:0/3947399766 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c00cba0 con 0x7fbd84150cd0 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd81600640 1 -- 172.21.15.59:0/3947399766 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c014440 con 0x7fbd84150cd0 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd84147f70 con 0x7fbd84150cd0 2024-04-23T14:43:45.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.134+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd84145640 con 0x7fbd84150cd0 2024-04-23T14:43:45.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.137+0000 7fbd81600640 1 -- 172.21.15.59:0/3947399766 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd7c01c070 con 0x7fbd84150cd0 2024-04-23T14:43:45.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.138+0000 7fbd81600640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd6806c920 0x7fbd6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.138+0000 7fbd81600640 1 -- 172.21.15.59:0/3947399766 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fbd7c08b8b0 con 0x7fbd84150cd0 2024-04-23T14:43:45.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.138+0000 7fbd89600640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd6806c920 0x7fbd6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.138+0000 7fbd8ee2c740 1 Processor -- start 2024-04-23T14:43:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.138+0000 7fbd8ee2c740 1 -- start start 2024-04-23T14:43:45.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.139+0000 7fbd89600640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd6806c920 0x7fbd6806ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbd74034000 tx=0x7fbd7402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.139+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad44388b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.139+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 0x55ad443b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.139+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b8b70 0x55ad4438d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.139+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad442eab10 con 0x55ad443b8b70 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.139+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ad442912b0 con 0x55ad443b92b0 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.139+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ad442e6ad0 con 0x55ad443d6190 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b8b70 0x55ad4438d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b8b70 0x55ad4438d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55976/0 (socket says 172.21.15.59:55976) 2024-04-23T14:43:45.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 -- 172.21.15.59:0/1326660709 learned_addr learned my addr 172.21.15.59:0/1326660709 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 -- 172.21.15.59:0/1326660709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 msgr2=0x55ad44388b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 --2- 172.21.15.59:0/1326660709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad44388b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 -- 172.21.15.59:0/1326660709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 msgr2=0x55ad443b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 --2- 172.21.15.59:0/1326660709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 0x55ad443b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 -- 172.21.15.59:0/1326660709 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad43ffd2d0 con 0x55ad443b8b70 2024-04-23T14:43:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.140+0000 7fbd8a000640 1 --2- 172.21.15.59:0/1326660709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b8b70 0x55ad4438d980 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00bfb0 tx=0x7fbd7c059570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7e56aa02debc14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd71c00640 1 -- 172.21.15.59:0/1326660709 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0577b0 con 0x55ad443b8b70 2024-04-23T14:43:45.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd71c00640 1 -- 172.21.15.59:0/1326660709 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c055b20 con 0x55ad443b8b70 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd71c00640 1 -- 172.21.15.59:0/1326660709 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c061eb0 con 0x55ad443b8b70 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1326660709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b8b70 msgr2=0x55ad4438d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1326660709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b8b70 0x55ad4438d980 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00bfb0 tx=0x7fbd7c059570 comp rx=0 tx=0).stop 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1326660709 shutdown_connections 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1326660709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b8b70 0x55ad4438d980 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1326660709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 0x55ad443b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1326660709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad44388b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.141+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1326660709 >> 172.21.15.59:0/1326660709 conn(0x55ad4438ad90 msgr2=0x55ad443d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.142+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1326660709 shutdown_connections 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.142+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1326660709 wait complete. 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.142+0000 7fbd8ee2c740 1 -- start start 2024-04-23T14:43:45.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.142+0000 7fbd8ee2c740 1 Processor -- start 2024-04-23T14:43:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.143+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 0x55ad443a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.143+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443d6190 0x55ad443e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.143+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443e6f90 0x55ad4438e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.143+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad4438cf40 con 0x55ad443d6190 2024-04-23T14:43:45.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.143+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ad442a4590 con 0x55ad443b92b0 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.143+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ad4438c330 con 0x55ad443e6f90 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.143+0000 7fbd8ee2c740 10 client.0 _ll_register_callbacks cb 0x55ad442873a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443d6190 0x55ad443e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443d6190 0x55ad443e6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55980/0 (socket says 172.21.15.59:55980) 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 -- 172.21.15.59:0/1909030733 learned_addr learned my addr 172.21.15.59:0/1909030733 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd89600640 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 0x55ad443a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 -- 172.21.15.59:0/1909030733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443e6f90 msgr2=0x55ad4438e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443e6f90 0x55ad4438e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 -- 172.21.15.59:0/1909030733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 msgr2=0x55ad443a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 0x55ad443a9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.144+0000 7fbd88c00640 1 -- 172.21.15.59:0/1909030733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad440c76b0 con 0x55ad443d6190 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd88c00640 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443d6190 0x55ad443e6a50 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800af80 tx=0x7fbd7800c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c189ec1985b367e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd66000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:43:45.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd66000640 1 -- 172.21.15.59:0/1909030733 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78015070 con 0x55ad443d6190 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd66000640 1 -- 172.21.15.59:0/1909030733 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fbd78011800 con 0x55ad443d6190 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7c055b20 con 0x55ad443d6190 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd66000640 1 -- 172.21.15.59:0/1909030733 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fbd7800ce10 con 0x55ad443d6190 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd8ee2c740 10 client.9339 fetch_fsmap learned FSMap version 100 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x55ad443ba170 con 0x55ad443d6190 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.145+0000 7fbd66000640 1 -- 172.21.15.59:0/1909030733 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c055b20 con 0x55ad443d6190 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd66000640 1 -- 172.21.15.59:0/1909030733 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fbd78011a00 con 0x55ad443d6190 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd8ee2c740 10 client.9339 fetch_fsmap finished waiting for FSMap version 100 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd8ee2c740 10 client.9339 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd8ee2c740 20 client.9339 populate_metadata read hostname 'smithi059' 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad443bace0 0x55ad443b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd8ee2c740 4 client.9339 mds_command: new command op to 9210 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 --> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad442f8530 con 0x55ad443bace0 2024-04-23T14:43:45.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.146+0000 7fbd89600640 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad443bace0 0x55ad443b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.147+0000 7fbd89600640 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad443bace0 0x55ad443b7000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7775c2cd4abb3aaf server_cookie=23378b8e1a78a878 in_seq=0 out_seq=0 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.147+0000 7fbd66000640 10 client.9339 ms_handle_connect on v2:172.21.15.59:6836/1061483588 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.147+0000 7fbd66000640 1 -- 172.21.15.59:0/1909030733 <== mds.0 v2:172.21.15.59:6836/1061483588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad442f8530 con 0x55ad443bace0 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.147+0000 7fbd66000640 10 client.9339 handle_command_reply: tid=0 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.147+0000 7fbd8ee2c740 1 client.9339 shutdown 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443d6190 msgr2=0x55ad443e6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443d6190 0x55ad443e6a50 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800af80 tx=0x7fbd7800c5e0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 shutdown_connections 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443e6f90 0x55ad4438e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad443bace0 0x55ad443b7000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443d6190 0x55ad443e6a50 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/1909030733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443b92b0 0x55ad443a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd66000640 0 client.9339 ms_handle_reset on v2:172.21.15.59:6836/1061483588 2024-04-23T14:43:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 >> 172.21.15.59:0/1909030733 conn(0x55ad443b8b70 msgr2=0x55ad443d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 shutdown_connections 2024-04-23T14:43:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/1909030733 wait complete. 2024-04-23T14:43:45.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.148+0000 7fbd8ee2c740 20 client.9339 trim_cache size 0 max 16384 2024-04-23T14:43:45.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.168+0000 7fbd8ee2c740 1 Processor -- start 2024-04-23T14:43:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.168+0000 7fbd8ee2c740 1 -- start start 2024-04-23T14:43:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad443f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad44388ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 0x55ad44389010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd841477e0 con 0x55ad443b92b0 2024-04-23T14:43:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbd84146b40 con 0x55ad443e6f90 2024-04-23T14:43:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbd84147190 con 0x55ad443d6190 2024-04-23T14:43:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd89600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad443f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd89600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad443f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55992/0 (socket says 172.21.15.59:55992) 2024-04-23T14:43:45.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.169+0000 7fbd89600640 1 -- 172.21.15.59:0/667134792 learned_addr learned my addr 172.21.15.59:0/667134792 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.170+0000 7fbd89600640 1 -- 172.21.15.59:0/667134792 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 msgr2=0x55ad44388ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.170+0000 7fbd89600640 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad44388ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.170+0000 7fbd89600640 1 -- 172.21.15.59:0/667134792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 msgr2=0x55ad44389010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.170+0000 7fbd8a000640 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 0x55ad44389010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.170+0000 7fbd89600640 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 0x55ad44389010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.170+0000 7fbd89600640 1 -- 172.21.15.59:0/667134792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad443caea0 con 0x55ad443b92b0 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.170+0000 7fbd89600640 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad443f6ec0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009990 tx=0x7fbd7402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ed7acd867efdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd71200640 1 -- 172.21.15.59:0/667134792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74052070 con 0x55ad443b92b0 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd71200640 1 -- 172.21.15.59:0/667134792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7404c6f0 con 0x55ad443b92b0 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd71200640 1 -- 172.21.15.59:0/667134792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74037040 con 0x55ad443b92b0 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/667134792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 msgr2=0x55ad443f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad443f6ec0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009990 tx=0x7fbd7402c000 comp rx=0 tx=0).stop 2024-04-23T14:43:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/667134792 shutdown_connections 2024-04-23T14:43:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad44388ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 0x55ad44389010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/667134792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad443f6ec0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/667134792 >> 172.21.15.59:0/667134792 conn(0x55ad443b8b70 msgr2=0x55ad4437b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/667134792 shutdown_connections 2024-04-23T14:43:45.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.171+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/667134792 wait complete. 2024-04-23T14:43:45.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.172+0000 7fbd8ee2c740 1 -- start start 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.172+0000 7fbd8ee2c740 1 Processor -- start 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.173+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad4438c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.173+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad443a7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.173+0000 7fbd8ee2c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 0x55ad443a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.173+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ad443bab60 con 0x55ad443b92b0 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.173+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ad443cba60 con 0x55ad443e6f90 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.173+0000 7fbd8ee2c740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55ad443bb1b0 con 0x55ad443d6190 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.173+0000 7fbd8ee2c740 10 client.0 _ll_register_callbacks cb 0x7fbd7800ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:43:45.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad4438c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad4438c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56002/0 (socket says 172.21.15.59:56002) 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 -- 172.21.15.59:0/4033408008 learned_addr learned my addr 172.21.15.59:0/4033408008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 -- 172.21.15.59:0/4033408008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 msgr2=0x55ad443a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad443a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 -- 172.21.15.59:0/4033408008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 msgr2=0x55ad443a7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 0x55ad443a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 -- 172.21.15.59:0/4033408008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad443d6ee0 con 0x55ad443b92b0 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd89600640 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad4438c570 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fbd74032a50 tx=0x7fbd74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8f19e34f9eb053 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.174+0000 7fbd66000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd66000640 1 -- 172.21.15.59:0/4033408008 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74052070 con 0x55ad443b92b0 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd66000640 1 -- 172.21.15.59:0/4033408008 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fbd7404c8a0 con 0x55ad443b92b0 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7404c6f0 con 0x55ad443b92b0 2024-04-23T14:43:45.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd66000640 1 -- 172.21.15.59:0/4033408008 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fbd7405e050 con 0x55ad443b92b0 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 10 client.9345 fetch_fsmap learned FSMap version 100 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x55ad4438cb00 con 0x55ad443b92b0 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd66000640 1 -- 172.21.15.59:0/4033408008 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7404c6f0 con 0x55ad443b92b0 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd66000640 1 -- 172.21.15.59:0/4033408008 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fbd74061020 con 0x55ad443b92b0 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 10 client.9345 fetch_fsmap finished waiting for FSMap version 100 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 10 client.9345 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 20 client.9345 populate_metadata read hostname 'smithi059' 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad4437f790 0x55ad443d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 4 client.9345 mds_command: new command op to 9210 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.175+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 --> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] -- command(tid 0: {"prefix": "status"}) -- 0x55ad442f23a0 con 0x55ad4437f790 2024-04-23T14:43:45.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.176+0000 7fbd88c00640 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad4437f790 0x55ad443d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.176+0000 7fbd88c00640 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad4437f790 0x55ad443d46f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515abd0972f7db4b server_cookie=a9ae91d4e118e1d0 in_seq=0 out_seq=0 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.176+0000 7fbd66000640 10 client.9345 ms_handle_connect on v2:172.21.15.59:6836/1061483588 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.176+0000 7fbd66000640 1 -- 172.21.15.59:0/4033408008 <== mds.0 v2:172.21.15.59:6836/1061483588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ad442f23a0 con 0x55ad4437f790 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.176+0000 7fbd66000640 10 client.9345 handle_command_reply: tid=0 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.176+0000 7fbd8ee2c740 1 client.9345 shutdown 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 msgr2=0x55ad4438c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad4438c570 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fbd74032a50 tx=0x7fbd74004730 comp rx=0 tx=0).stop 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 shutdown_connections 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55ad443d6190 0x55ad443a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x55ad4437f790 0x55ad443d46f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad443e6f90 0x55ad443a7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 --2- 172.21.15.59:0/4033408008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ad443b92b0 0x55ad4438c570 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd66000640 0 client.9345 ms_handle_reset on v2:172.21.15.59:6836/1061483588 2024-04-23T14:43:45.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 >> 172.21.15.59:0/4033408008 conn(0x55ad443b8b70 msgr2=0x55ad44388ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 shutdown_connections 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 1 -- 172.21.15.59:0/4033408008 wait complete. 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.177+0000 7fbd8ee2c740 20 client.9345 trim_cache size 0 max 16384 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9210, 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:43:45.182 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:43:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:43:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.7844326519999996, 2024-04-23T14:43:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 100, 2024-04-23T14:43:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 112, 2024-04-23T14:43:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 112, 2024-04-23T14:43:45.183 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 13.269588280000001 2024-04-23T14:43:45.183 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd6806c920 msgr2=0x7fbd6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd6806c920 0x7fbd6806ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbd74034000 tx=0x7fbd7402e000 comp rx=0 tx=0).stop 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd84150cd0 msgr2=0x7fbd84151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd84150cd0 0x7fbd84151180 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00b9a0 tx=0x7fbd7c00be60 comp rx=0 tx=0).stop 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 shutdown_connections 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd6806c920 0x7fbd6806ede0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd840b3010 0x7fbd84145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd84150cd0 0x7fbd84151180 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 --2- 172.21.15.59:0/3947399766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd840af4b0 0x7fbd841f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 >> 172.21.15.59:0/3947399766 conn(0x7fbd8406bdf0 msgr2=0x7fbd840d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 shutdown_connections 2024-04-23T14:43:45.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.178+0000 7fbd8aa00640 1 -- 172.21.15.59:0/3947399766 wait complete. 2024-04-23T14:43:45.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:43:45.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.406+0000 7f6963000640 1 Processor -- start 2024-04-23T14:43:45.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.406+0000 7f6963000640 1 -- start start 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6963000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f695c15e7c0 0x7f695c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6963000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0d2630 0x7f695c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6963000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f695c0cf030 con 0x7f695c0b6530 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6963000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f695c0b7000 con 0x7f695c15e7c0 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6963000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f695c0cf620 con 0x7f695c0d2630 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f695c15e7c0 0x7f695c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f695c15e7c0 0x7f695c0d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56006/0 (socket says 172.21.15.59:56006) 2024-04-23T14:43:45.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961c00640 1 -- 172.21.15.59:0/1509938783 learned_addr learned my addr 172.21.15.59:0/1509938783 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961200640 1 --2- 172.21.15.59:0/1509938783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0d2630 0x7f695c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961c00640 1 -- 172.21.15.59:0/1509938783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0d2630 msgr2=0x7f695c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961c00640 1 --2- 172.21.15.59:0/1509938783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0d2630 0x7f695c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961c00640 1 -- 172.21.15.59:0/1509938783 >> v1:172.21.15.76:6789/0 conn(0x7f695c0b6530 legacy=0x7f695c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:45.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.408+0000 7f6961c00640 1 -- 172.21.15.59:0/1509938783 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c0b6a40 con 0x7f695c15e7c0 2024-04-23T14:43:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6961c00640 1 --2- 172.21.15.59:0/1509938783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0d20f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f6950009870 tx=0x7f695002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8727d75c772889 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f695be00640 1 -- 172.21.15.59:0/1509938783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950037070 con 0x7f695c15e7c0 2024-04-23T14:43:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f695be00640 1 -- 172.21.15.59:0/1509938783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6950032440 con 0x7f695c15e7c0 2024-04-23T14:43:45.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f695be00640 1 -- 172.21.15.59:0/1509938783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950032810 con 0x7f695c15e7c0 2024-04-23T14:43:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 -- 172.21.15.59:0/1509938783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 msgr2=0x7f695c0d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 --2- 172.21.15.59:0/1509938783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0d20f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f6950009870 tx=0x7f695002a270 comp rx=0 tx=0).stop 2024-04-23T14:43:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 -- 172.21.15.59:0/1509938783 shutdown_connections 2024-04-23T14:43:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 --2- 172.21.15.59:0/1509938783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0d2630 0x7f695c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 --2- 172.21.15.59:0/1509938783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0d20f0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 -- 172.21.15.59:0/1509938783 >> 172.21.15.59:0/1509938783 conn(0x7f695c09d920 msgr2=0x7f695c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 -- 172.21.15.59:0/1509938783 shutdown_connections 2024-04-23T14:43:45.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.411+0000 7f6963000640 1 -- 172.21.15.59:0/1509938783 wait complete. 2024-04-23T14:43:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 Processor -- start 2024-04-23T14:43:45.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 -- start start 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0b6530 0x7f695c14bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f695c0d2630 0x7f695c14c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f695c0d1a80 con 0x7f695c15e7c0 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f695c0cf2b0 con 0x7f695c0d2630 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6963000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f695c0d1c00 con 0x7f695c0b6530 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6962600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6962600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56020/0 (socket says 172.21.15.59:56020) 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.414+0000 7f6962600640 1 -- 172.21.15.59:0/1863771076 learned_addr learned my addr 172.21.15.59:0/1863771076 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6961200640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f695c0d2630 0x7f695c14c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6961c00640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0b6530 0x7f695c14bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6962600640 1 -- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0b6530 msgr2=0x7f695c14bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6962600640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0b6530 0x7f695c14bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6962600640 1 -- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f695c0d2630 msgr2=0x7f695c14c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6962600640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f695c0d2630 0x7f695c14c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6962600640 1 -- 172.21.15.59:0/1863771076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c14c770 con 0x7f695c15e7c0 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6961200640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f695c0d2630 0x7f695c14c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:45.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6962600640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0cd930 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f695400c730 tx=0x7f695400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.415+0000 7f6959600640 1 -- 172.21.15.59:0/1863771076 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954003b70 con 0x7f695c15e7c0 2024-04-23T14:43:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.416+0000 7f6959600640 1 -- 172.21.15.59:0/1863771076 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6954003d10 con 0x7f695c15e7c0 2024-04-23T14:43:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.416+0000 7f6959600640 1 -- 172.21.15.59:0/1863771076 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695400aa30 con 0x7f695c15e7c0 2024-04-23T14:43:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.416+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c14ca50 con 0x7f695c15e7c0 2024-04-23T14:43:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.416+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c15a630 con 0x7f695c15e7c0 2024-04-23T14:43:45.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.417+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695c04fd90 con 0x7f695c15e7c0 2024-04-23T14:43:45.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.418+0000 7f6959600640 1 -- 172.21.15.59:0/1863771076 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f695400abd0 con 0x7f695c15e7c0 2024-04-23T14:43:45.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.419+0000 7f6959600640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f693c06c920 0x7f693c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:45.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.419+0000 7f6959600640 1 -- 172.21.15.59:0/1863771076 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f695408b9c0 con 0x7f695c15e7c0 2024-04-23T14:43:45.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.419+0000 7f6961c00640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f693c06c920 0x7f693c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:45.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.420+0000 7f6961c00640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f693c06c920 0x7f693c06ede0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6950034000 tx=0x7f695002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:45.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.423+0000 7f6959600640 1 -- 172.21.15.59:0/1863771076 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f695c0cf030 con 0x7f695c15e7c0 2024-04-23T14:43:45.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.669+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f695c0d1eb0 con 0x7f695c15e7c0 2024-04-23T14:43:45.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.670+0000 7f6959600640 1 -- 172.21.15.59:0/1863771076 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7f6954059de0 con 0x7f695c15e7c0 2024-04-23T14:43:45.671 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f693c06c920 msgr2=0x7f693c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f693c06c920 0x7f693c06ede0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6950034000 tx=0x7f695002e000 comp rx=0 tx=0).stop 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 msgr2=0x7f695c0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0cd930 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f695400c730 tx=0x7f695400cbf0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 shutdown_connections 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f693c06c920 0x7f693c06ede0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f695c15e7c0 0x7f695c0cd930 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f695c0d2630 0x7f695c14c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 --2- 172.21.15.59:0/1863771076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f695c0b6530 0x7f695c14bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 >> 172.21.15.59:0/1863771076 conn(0x7f695c09d920 msgr2=0x7f695c0b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 shutdown_connections 2024-04-23T14:43:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:45.673+0000 7f6963000640 1 -- 172.21.15.59:0/1863771076 wait complete. 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:43:45.694 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:43:45.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:45.695 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:43:45.750 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:43:45.751 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41723sec preferred_lft 41723sec 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:43:45.752 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:43:45.753 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:43:45.753 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:43:45.753 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:43:45.753 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:43:45.753 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:43:45.753 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:43:45.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:45.753 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:43:45.836 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:43:45.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:45.841 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:43:45.897 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:43:45.898 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:43:45.898 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:43:45.898 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:43:45.898 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:43:45.898 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:43:45.898 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:43:45.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:43:45.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:43:45.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:43:45.899 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:43:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:43:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:46.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:46.120 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:43:46.169 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:43:46.169 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:43:46.170 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:43:46.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:46.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:43:46.256 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:43:46.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:43:46.257 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:43:46.313 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:43:46.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:43:46.313 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:43:46.362 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:43:46.455 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:43:46.508 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:43:46.508 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:43:46.508 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:43:46.508 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:43:46.508 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:43:46.508 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:43:46.508 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:43:46.509 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:43:46.510 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:43:46.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:46.511 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:43:46.561 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:43:46.562 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:43:46.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:43:46.635 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:43:46.676 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:43:46.677 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:43:46.678 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:43:46.679 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:43:46.679 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:43:46.679 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:43:46.679 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:43:46.679 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:43:46.679 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:43:46.679 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:43:46.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:46.681 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:43:46.735 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:43:46.735+0000 7f2e47246480 -1 init, newargv = 0x55ca1edb44c0 newargc=15 2024-04-23T14:43:46.735 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[75344]: starting ceph client 2024-04-23T14:43:46.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[75344]: starting fuse 2024-04-23T14:43:46.765 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:43:46.766 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:43:46.766 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:43:46.767 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:46.966 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.75344.asok 2024-04-23T14:43:46.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:46.984 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75344.asok 2024-04-23T14:43:46.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:46.984 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75344.asok status 2024-04-23T14:43:47.224 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:43:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:43:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:43:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:43:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:43:47.224 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "75344", 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9360, 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9360 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:43:47.225 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3286882576 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3286882576 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9360 192.168.0.1:0/3286882576", 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3286882576", 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 100, 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 112, 2024-04-23T14:43:47.226 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:43:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:43:47.227 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:43:47.227 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:43:47.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:47.240 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:43:47.240 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:43:47.299 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:43:47.299 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:43:47.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:47.299 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> parent = os.path.join(mount_point, "parent") 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> os.mkdir(parent) 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> flushed = os.path.join(parent, "flushed") 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> os.mkdir(flushed) 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> for f in ["alpha", "bravo", "charlie"]: 2024-04-23T14:43:47.390 DEBUG:teuthology.orchestra.run.smithi059:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-23T14:43:47.391 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:47.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:47.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:47.596 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-23T14:43:47.726 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-23T14:43:47.726 INFO:teuthology.orchestra.run.smithi059.stdout:./parent 2024-04-23T14:43:47.727 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed 2024-04-23T14:43:47.727 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/charlie 2024-04-23T14:43:47.727 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/alpha 2024-04-23T14:43:47.727 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/bravo 2024-04-23T14:43:47.728 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:43:47.728 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:47.729 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:43:47.729 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:47.729 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-23T14:43:47.729 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:47.878 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:43:47.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:47.892 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:43:47.892 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:47.892 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:43:47.892 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:47.892 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-23T14:43:47.892 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:48.084 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-23T14:43:48.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:48.099 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:43:48.099 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:48.099 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:43:48.099 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:48.099 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-23T14:43:48.099 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:48.295 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-23T14:43:48.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:48.309 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:43:48.309 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:48.310 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:43:48.310 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:48.310 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-23T14:43:48.310 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:48.491 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-23T14:43:48.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:48.506 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:43:48.506 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:48.507 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:43:48.507 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:48.507 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-23T14:43:48.507 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:48.705 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-23T14:43:48.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:48.721 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:43:48.721 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:43:48.721 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:43:48.721 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:43:48.721 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-23T14:43:48.721 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:43:48.920 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628279 2024-04-23T14:43:48.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:43:48.935 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:43:48.935 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:49.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:49.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.252+0000 7fc537800640 1 Processor -- start 2024-04-23T14:43:49.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.252+0000 7fc537800640 1 -- start start 2024-04-23T14:43:49.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc537800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5301454e0 0x7fc530145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc537800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc5300b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc537800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5300cd520 con 0x7fc5300b3e40 2024-04-23T14:43:49.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc537800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc5300cddc0 con 0x7fc5301454e0 2024-04-23T14:43:49.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc537800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc5300cd800 con 0x7fc5300cd020 2024-04-23T14:43:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc534e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5301454e0 0x7fc530145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc534e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc5301454e0 0x7fc530145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56028/0 (socket says 172.21.15.59:56028) 2024-04-23T14:43:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.254+0000 7fc534e00640 1 -- 172.21.15.59:0/240077721 learned_addr learned my addr 172.21.15.59:0/240077721 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:49.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.255+0000 7fc52fe00640 1 --2- 172.21.15.59:0/240077721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc5300b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.255+0000 7fc534e00640 1 -- 172.21.15.59:0/240077721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 msgr2=0x7fc5300b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:49.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.255+0000 7fc534e00640 1 --2- 172.21.15.59:0/240077721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc5300b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.255+0000 7fc534e00640 1 -- 172.21.15.59:0/240077721 >> v1:172.21.15.76:6789/0 conn(0x7fc5300b3e40 legacy=0x7fc530160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.255+0000 7fc534e00640 1 -- 172.21.15.59:0/240077721 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5300cdae0 con 0x7fc5301454e0 2024-04-23T14:43:49.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.257+0000 7fc534e00640 1 --2- 172.21.15.59:0/240077721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc530145bb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fc524009870 tx=0x7fc52402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5479180a6506e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:49.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.257+0000 7fc52f400640 1 -- 172.21.15.59:0/240077721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524037070 con 0x7fc5301454e0 2024-04-23T14:43:49.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.257+0000 7fc52f400640 1 -- 172.21.15.59:0/240077721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc524032440 con 0x7fc5301454e0 2024-04-23T14:43:49.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.257+0000 7fc52f400640 1 -- 172.21.15.59:0/240077721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524032810 con 0x7fc5301454e0 2024-04-23T14:43:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 -- 172.21.15.59:0/240077721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 msgr2=0x7fc530145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 --2- 172.21.15.59:0/240077721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc530145bb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fc524009870 tx=0x7fc52402a1c0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 -- 172.21.15.59:0/240077721 shutdown_connections 2024-04-23T14:43:49.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 --2- 172.21.15.59:0/240077721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc5300b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 --2- 172.21.15.59:0/240077721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc530145bb0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 -- 172.21.15.59:0/240077721 >> 172.21.15.59:0/240077721 conn(0x7fc530067ea0 msgr2=0x7fc53006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:49.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 -- 172.21.15.59:0/240077721 shutdown_connections 2024-04-23T14:43:49.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.258+0000 7fc537800640 1 -- 172.21.15.59:0/240077721 wait complete. 2024-04-23T14:43:49.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.260+0000 7fc537800640 1 Processor -- start 2024-04-23T14:43:49.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.260+0000 7fc537800640 1 -- start start 2024-04-23T14:43:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc537800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5300b3e40 0x7fc5301f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc537800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc53006dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc537800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc53006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc537800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc530146270 con 0x7fc5301454e0 2024-04-23T14:43:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc537800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc5301460f0 con 0x7fc5300b3e40 2024-04-23T14:43:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc537800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc530161020 con 0x7fc5300cd020 2024-04-23T14:43:49.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc535800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc53006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc535800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc53006e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56034/0 (socket says 172.21.15.59:56034) 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc535800640 1 -- 172.21.15.59:0/846053716 learned_addr learned my addr 172.21.15.59:0/846053716 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc534e00640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5300b3e40 0x7fc5301f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.262+0000 7fc52fe00640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc53006dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc535800640 1 -- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 msgr2=0x7fc53006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc535800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc53006dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc535800640 1 -- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5300b3e40 msgr2=0x7fc5301f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc535800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5300b3e40 0x7fc5301f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc535800640 1 -- 172.21.15.59:0/846053716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc530143890 con 0x7fc5301454e0 2024-04-23T14:43:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc535800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc53006e320 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c710 tx=0x7fc52800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc52cc00640 1 -- 172.21.15.59:0/846053716 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528016020 con 0x7fc5301454e0 2024-04-23T14:43:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc52cc00640 1 -- 172.21.15.59:0/846053716 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc528006d50 con 0x7fc5301454e0 2024-04-23T14:43:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc52cc00640 1 -- 172.21.15.59:0/846053716 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528003bb0 con 0x7fc5301454e0 2024-04-23T14:43:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.263+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc530143b70 con 0x7fc5301454e0 2024-04-23T14:43:49.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.264+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5301f18c0 con 0x7fc5301454e0 2024-04-23T14:43:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.265+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5301f1ac0 con 0x7fc5301454e0 2024-04-23T14:43:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.266+0000 7fc52cc00640 1 -- 172.21.15.59:0/846053716 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc52800a850 con 0x7fc5301454e0 2024-04-23T14:43:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.267+0000 7fc52cc00640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc51006c8f0 0x7fc51006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.267+0000 7fc52cc00640 1 -- 172.21.15.59:0/846053716 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fc52808b790 con 0x7fc5301454e0 2024-04-23T14:43:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.267+0000 7fc534e00640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc51006c8f0 0x7fc51006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.268+0000 7fc534e00640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc51006c8f0 0x7fc51006edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc524034000 tx=0x7fc52402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:49.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.272+0000 7fc52cc00640 1 -- 172.21.15.59:0/846053716 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5301f1ac0 con 0x7fc5301454e0 2024-04-23T14:43:49.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.478+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc530068a80 con 0x7fc5301454e0 2024-04-23T14:43:49.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.480+0000 7fc52cc00640 1 -- 172.21.15.59:0/846053716 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16567 (secure 0 0 0) 0x7fc528020020 con 0x7fc5301454e0 2024-04-23T14:43:49.481 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:49.481 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":112,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:39.410742+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc51006c8f0 msgr2=0x7fc51006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc51006c8f0 0x7fc51006edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc524034000 tx=0x7fc52402e000 comp rx=0 tx=0).stop 2024-04-23T14:43:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 msgr2=0x7fc53006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc53006e320 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c710 tx=0x7fc52800cbd0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 shutdown_connections 2024-04-23T14:43:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc51006c8f0 0x7fc51006edb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc5300cd020 0x7fc53006dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5301454e0 0x7fc53006e320 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 --2- 172.21.15.59:0/846053716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5300b3e40 0x7fc5301f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.483+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 >> 172.21.15.59:0/846053716 conn(0x7fc530067ea0 msgr2=0x7fc5300b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.484+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 shutdown_connections 2024-04-23T14:43:49.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.484+0000 7fc537800640 1 -- 172.21.15.59:0/846053716 wait complete. 2024-04-23T14:43:49.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:43:49.503 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:43:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.715+0000 7ff2bb200640 1 Processor -- start 2024-04-23T14:43:49.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.715+0000 7ff2bb200640 1 -- start start 2024-04-23T14:43:49.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.716+0000 7ff2bb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 0x7ff2bc0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.716+0000 7ff2bb200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2bc06dcb0 0x7ff2bc162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.717+0000 7ff2bb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2bc163240 con 0x7ff2bc0d09e0 2024-04-23T14:43:49.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.717+0000 7ff2bb200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff2bc072a20 con 0x7ff2bc06dcb0 2024-04-23T14:43:49.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.717+0000 7ff2bb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2bc071e10 con 0x7ff2bc0d0600 2024-04-23T14:43:49.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.717+0000 7ff2ba800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff2bc0d09e0 0x7ff2bc06e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40928/0 (socket says 172.21.15.59:40928) 2024-04-23T14:43:49.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.717+0000 7ff2ba800640 1 -- 172.21.15.59:0/3969496545 learned_addr learned my addr 172.21.15.59:0/3969496545 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:49.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.717+0000 7ff2b3e00640 1 --2- 172.21.15.59:0/3969496545 >> v2:172.21.15.76:3300/0 conn(0x7ff2bc06dcb0 0x7ff2bc162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.717+0000 7ff2b9e00640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 0x7ff2bc0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3878185948 0 0) 0x7ff2b4001490 con 0x7ff2bc0d09e0 2024-04-23T14:43:49.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2a0003520 con 0x7ff2bc0d09e0 2024-04-23T14:43:49.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 404620772 0 0) 0x7ff2a0003520 con 0x7ff2bc0d09e0 2024-04-23T14:43:49.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 msgr2=0x7ff2bc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 0x7ff2bc0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 >> v2:172.21.15.76:3300/0 conn(0x7ff2bc06dcb0 msgr2=0x7ff2bc162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 --2- 172.21.15.59:0/3969496545 >> v2:172.21.15.76:3300/0 conn(0x7ff2bc06dcb0 0x7ff2bc162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc0720f0 con 0x7ff2bc0d09e0 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7ff2b4001bc0 con 0x7ff2bc0d09e0 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 >> v1:172.21.15.59:6789/0 conn(0x7ff2bc0d09e0 legacy=0x7ff2bc06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2a00039b0 0x7ff2a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 0x7ff2a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc072a20 con 0x7ff2a0004d40 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc071e10 con 0x7ff2a00039b0 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a000b9c0 con 0x7ff2bc0d0600 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.718+0000 7ff2b3e00640 1 --2- 172.21.15.59:0/3969496545 >> v2:172.21.15.76:3300/0 conn(0x7ff2bc06dcb0 0x7ff2bc162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:43:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2ba800640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2b9e00640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2a00039b0 0x7ff2a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2b3e00640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 0x7ff2a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2ba800640 1 -- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 msgr2=0x7ff2a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:49.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2ba800640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc0d0600 0x7ff2a0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2ba800640 1 -- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2a00039b0 msgr2=0x7ff2a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:49.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2ba800640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2a00039b0 0x7ff2a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.719+0000 7ff2ba800640 1 -- 172.21.15.59:0/3969496545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a0003520 con 0x7ff2a0004d40 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.721+0000 7ff2ba800640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2a00051c0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400aa70 tx=0x7ff2a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdf41f0d54ac861 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.721+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400a690 con 0x7ff2a0004d40 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.721+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b402f070 con 0x7ff2a0004d40 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.721+0000 7ff2b9400640 1 -- 172.21.15.59:0/3969496545 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4034470 con 0x7ff2a0004d40 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.722+0000 7ff2bb200640 1 -- 172.21.15.59:0/3969496545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 msgr2=0x7ff2a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.722+0000 7ff2bb200640 1 --2- 172.21.15.59:0/3969496545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2a00051c0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400aa70 tx=0x7ff2a0005830 comp rx=0 tx=0).stop 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.722+0000 7ff2ba800640 1 -- 172.21.15.59:0/3969496545 reap_dead start 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.722+0000 7ff2bb200640 1 -- 172.21.15.59:0/3969496545 shutdown_connections 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.722+0000 7ff2bb200640 1 -- 172.21.15.59:0/3969496545 >> 172.21.15.59:0/3969496545 conn(0x7ff2bc06bdf0 msgr2=0x7ff2bc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:49.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.722+0000 7ff2bb200640 1 -- 172.21.15.59:0/3969496545 shutdown_connections 2024-04-23T14:43:49.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.722+0000 7ff2bb200640 1 -- 172.21.15.59:0/3969496545 wait complete. 2024-04-23T14:43:49.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.724+0000 7ff2bb200640 1 Processor -- start 2024-04-23T14:43:49.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.724+0000 7ff2bb200640 1 -- start start 2024-04-23T14:43:49.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2bb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2bc0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2bb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc14fd90 0x7ff2bc150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2bb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2bc0b2410 0x7ff2bc0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2bb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc15eaf0 con 0x7ff2a0004d40 2024-04-23T14:43:49.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2bb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc0726b0 con 0x7ff2bc0b2410 2024-04-23T14:43:49.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2bb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2bc0d0e70 con 0x7ff2bc14fd90 2024-04-23T14:43:49.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2bc0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2b9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2bc0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56038/0 (socket says 172.21.15.59:56038) 2024-04-23T14:43:49.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.725+0000 7ff2b9e00640 1 -- 172.21.15.59:0/3323357558 learned_addr learned my addr 172.21.15.59:0/3323357558 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:49.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b3e00640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc14fd90 0x7ff2bc150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b9e00640 1 -- 172.21.15.59:0/3323357558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc14fd90 msgr2=0x7ff2bc150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b9e00640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc14fd90 0x7ff2bc150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b9e00640 1 -- 172.21.15.59:0/3323357558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2bc0b2410 msgr2=0x7ff2bc0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b9e00640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2bc0b2410 0x7ff2bc0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b9e00640 1 -- 172.21.15.59:0/3323357558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc0c9510 con 0x7ff2a0004d40 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b9e00640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2bc0ca890 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac011570 tx=0x7ff2ac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b2000640 1 -- 172.21.15.59:0/3323357558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac015930 con 0x7ff2a0004d40 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b2000640 1 -- 172.21.15.59:0/3323357558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ac015ad0 con 0x7ff2a0004d40 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2b2000640 1 -- 172.21.15.59:0/3323357558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac01d400 con 0x7ff2a0004d40 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.726+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc0c97f0 con 0x7ff2a0004d40 2024-04-23T14:43:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.727+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc0c9db0 con 0x7ff2a0004d40 2024-04-23T14:43:49.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.728+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2bc04fd90 con 0x7ff2a0004d40 2024-04-23T14:43:49.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.729+0000 7ff2b2000640 1 -- 172.21.15.59:0/3323357558 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2ac01d5a0 con 0x7ff2a0004d40 2024-04-23T14:43:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.730+0000 7ff2b2000640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff29806c920 0x7ff29806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.730+0000 7ff2b2000640 1 -- 172.21.15.59:0/3323357558 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7ff2ac08c3f0 con 0x7ff2a0004d40 2024-04-23T14:43:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.730+0000 7ff2b3e00640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff29806c920 0x7ff29806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.731+0000 7ff2b3e00640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff29806c920 0x7ff29806ede0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc1ea9f0 tx=0x7ff2a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.734+0000 7ff2b2000640 1 -- 172.21.15.59:0/3323357558 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff2ac019030 con 0x7ff2a0004d40 2024-04-23T14:43:49.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.937+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff2bc0cff50 con 0x7ff2a0004d40 2024-04-23T14:43:49.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.938+0000 7ff2b2000640 1 -- 172.21.15.59:0/3323357558 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7ff2ac05a780 con 0x7ff2a0004d40 2024-04-23T14:43:49.938 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:43:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff29806c920 msgr2=0x7ff29806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff29806c920 0x7ff29806ede0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc1ea9f0 tx=0x7ff2a4008040 comp rx=0 tx=0).stop 2024-04-23T14:43:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 msgr2=0x7ff2bc0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2bc0ca890 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac011570 tx=0x7ff2ac011a30 comp rx=0 tx=0).stop 2024-04-23T14:43:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 shutdown_connections 2024-04-23T14:43:49.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff29806c920 0x7ff29806ede0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2bc14fd90 0x7ff2bc150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2bc0b2410 0x7ff2bc0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 --2- 172.21.15.59:0/3323357558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a0004d40 0x7ff2bc0ca890 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:49.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.941+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 >> 172.21.15.59:0/3323357558 conn(0x7ff2bc06bdf0 msgr2=0x7ff2bc0cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:49.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.942+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 shutdown_connections 2024-04-23T14:43:49.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:49.942+0000 7ff2bb200640 1 -- 172.21.15.59:0/3323357558 wait complete. 2024-04-23T14:43:49.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:43:50.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.174+0000 7f9b91a00640 1 Processor -- start 2024-04-23T14:43:50.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.175+0000 7f9b91a00640 1 -- start start 2024-04-23T14:43:50.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.176+0000 7f9b91a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9b8c161fd0 0x7f9b8c0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.176+0000 7f9b91a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b8c0cdc90 0x7f9b8c0d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.176+0000 7f9b91a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c09b880 con 0x7f9b8c09aa00 2024-04-23T14:43:50.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.176+0000 7f9b91a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c06dce0 con 0x7f9b8c0cdc90 2024-04-23T14:43:50.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.176+0000 7f9b91a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c06d690 con 0x7f9b8c161fd0 2024-04-23T14:43:50.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b91000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b8c0cdc90 0x7f9b8c0d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b91000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b8c0cdc90 0x7f9b8c0d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40542/0 (socket says 172.21.15.59:40542) 2024-04-23T14:43:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b91000640 1 -- 172.21.15.59:0/1145530451 learned_addr learned my addr 172.21.15.59:0/1145530451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b8be00640 1 --2- 172.21.15.59:0/1145530451 >> v2:172.21.15.76:3300/0 conn(0x7f9b8c161fd0 0x7f9b8c0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b91000640 1 -- 172.21.15.59:0/1145530451 >> v2:172.21.15.76:3300/0 conn(0x7f9b8c161fd0 msgr2=0x7f9b8c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b91000640 1 --2- 172.21.15.59:0/1145530451 >> v2:172.21.15.76:3300/0 conn(0x7f9b8c161fd0 0x7f9b8c0b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b91000640 1 -- 172.21.15.59:0/1145530451 >> v1:172.21.15.76:6789/0 conn(0x7f9b8c09aa00 legacy=0x7f9b8c09ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.177+0000 7f9b91000640 1 -- 172.21.15.59:0/1145530451 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c06d970 con 0x7f9b8c0cdc90 2024-04-23T14:43:50.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.178+0000 7f9b91000640 1 --2- 172.21.15.59:0/1145530451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0d0610 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c073520 tx=0x7f9b8402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e254b112c31b613 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:50.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b8b400640 1 -- 172.21.15.59:0/1145530451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84036070 con 0x7f9b8c0cdc90 2024-04-23T14:43:50.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b8b400640 1 -- 172.21.15.59:0/1145530451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b84031600 con 0x7f9b8c0cdc90 2024-04-23T14:43:50.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b8b400640 1 -- 172.21.15.59:0/1145530451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b840319d0 con 0x7f9b8c0cdc90 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b91a00640 1 -- 172.21.15.59:0/1145530451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c0cdc90 msgr2=0x7f9b8c0d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b91a00640 1 --2- 172.21.15.59:0/1145530451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0d0610 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c073520 tx=0x7f9b8402a5d0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b91a00640 1 -- 172.21.15.59:0/1145530451 shutdown_connections 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b91a00640 1 --2- 172.21.15.59:0/1145530451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0d0610 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b91a00640 1 --2- 172.21.15.59:0/1145530451 >> v2:172.21.15.76:3300/0 conn(0x7f9b8c161fd0 0x7f9b8c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.179+0000 7f9b91a00640 1 -- 172.21.15.59:0/1145530451 >> 172.21.15.59:0/1145530451 conn(0x7f9b8c0b3f70 msgr2=0x7f9b8c0b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.180+0000 7f9b91a00640 1 -- 172.21.15.59:0/1145530451 shutdown_connections 2024-04-23T14:43:50.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.180+0000 7f9b91a00640 1 -- 172.21.15.59:0/1145530451 wait complete. 2024-04-23T14:43:50.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.181+0000 7f9b91a00640 1 Processor -- start 2024-04-23T14:43:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.182+0000 7f9b91a00640 1 -- start start 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.182+0000 7f9b91a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9b8c09aa00 0x7f9b8c0b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.182+0000 7f9b91a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.182+0000 7f9b91a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c161fd0 0x7f9b8c14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.182+0000 7f9b91a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0d0cd0 con 0x7f9b8c161fd0 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.182+0000 7f9b91a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0d0b50 con 0x7f9b8c0cdc90 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.182+0000 7f9b91a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c0d0e50 con 0x7f9b8c09aa00 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c161fd0 0x7f9b8c14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c161fd0 0x7f9b8c14e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40550/0 (socket says 172.21.15.59:40550) 2024-04-23T14:43:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 -- 172.21.15.59:0/4152379137 learned_addr learned my addr 172.21.15.59:0/4152379137 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:50.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 -- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9b8c09aa00 msgr2=0x7f9b8c0b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9b8c09aa00 0x7f9b8c0b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 -- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b8c0cdc90 msgr2=0x7f9b8c0c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b83e00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b91000640 1 -- 172.21.15.59:0/4152379137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c0c7a90 con 0x7f9b8c161fd0 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.183+0000 7f9b83e00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.184+0000 7f9b91000640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c161fd0 0x7f9b8c14e750 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f9b84034040 tx=0x7f9b84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.184+0000 7f9b88c00640 1 -- 172.21.15.59:0/4152379137 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84036050 con 0x7f9b8c161fd0 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.184+0000 7f9b88c00640 1 -- 172.21.15.59:0/4152379137 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b84042070 con 0x7f9b8c161fd0 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.184+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c0c9660 con 0x7f9b8c161fd0 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.184+0000 7f9b88c00640 1 -- 172.21.15.59:0/4152379137 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84007bd0 con 0x7f9b8c161fd0 2024-04-23T14:43:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.184+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c0b8010 con 0x7f9b8c161fd0 2024-04-23T14:43:50.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.185+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c04fd90 con 0x7f9b8c161fd0 2024-04-23T14:43:50.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.186+0000 7f9b88c00640 1 -- 172.21.15.59:0/4152379137 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b84047460 con 0x7f9b8c161fd0 2024-04-23T14:43:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.187+0000 7f9b88c00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.187+0000 7f9b88c00640 1 -- 172.21.15.59:0/4152379137 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f9b840acbb0 con 0x7f9b8c161fd0 2024-04-23T14:43:50.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.187+0000 7f9b8be00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.188+0000 7f9b8be00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b6806c8f0 0x7f9b6806edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c006fd0 tx=0x7f9b7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:50.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.191+0000 7f9b88c00640 1 -- 172.21.15.59:0/4152379137 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b8407af40 con 0x7f9b8c161fd0 2024-04-23T14:43:50.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.401+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b8c0d21e0 con 0x7f9b8c161fd0 2024-04-23T14:43:50.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.402+0000 7f9b88c00640 1 -- 172.21.15.59:0/4152379137 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7f9b8407a840 con 0x7f9b8c161fd0 2024-04-23T14:43:50.402 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:43:50.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.404+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b6806c8f0 msgr2=0x7f9b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.404+0000 7f9b91a00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b6806c8f0 0x7f9b6806edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c006fd0 tx=0x7f9b7c008040 comp rx=0 tx=0).stop 2024-04-23T14:43:50.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.404+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c161fd0 msgr2=0x7f9b8c14e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.404+0000 7f9b91a00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c161fd0 0x7f9b8c14e750 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f9b84034040 tx=0x7f9b84004a40 comp rx=0 tx=0).stop 2024-04-23T14:43:50.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 shutdown_connections 2024-04-23T14:43:50.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9b6806c8f0 0x7f9b6806edb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b8c161fd0 0x7f9b8c14e750 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b8c0cdc90 0x7f9b8c0c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 --2- 172.21.15.59:0/4152379137 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9b8c09aa00 0x7f9b8c0b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 >> 172.21.15.59:0/4152379137 conn(0x7f9b8c0b3f70 msgr2=0x7f9b8c160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:50.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 shutdown_connections 2024-04-23T14:43:50.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.405+0000 7f9b91a00640 1 -- 172.21.15.59:0/4152379137 wait complete. 2024-04-23T14:43:50.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:43:50.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.637+0000 7fe464e00640 1 Processor -- start 2024-04-23T14:43:50.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.637+0000 7fe464e00640 1 -- start start 2024-04-23T14:43:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.638+0000 7fe464e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe46015ed20 0x7fe4600d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.638+0000 7fe464e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe4600d13e0 0x7fe4600d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.639+0000 7fe464e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4600b5da0 con 0x7fe4600b3600 2024-04-23T14:43:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.639+0000 7fe464e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe46015f100 con 0x7fe4600d13e0 2024-04-23T14:43:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.639+0000 7fe464e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe4600d07d0 con 0x7fe46015ed20 2024-04-23T14:43:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.639+0000 7fe45ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe4600b3600 0x7fe4600b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40732/0 (socket says 172.21.15.59:40732) 2024-04-23T14:43:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.639+0000 7fe45ea00640 1 -- 172.21.15.59:0/3770782424 learned_addr learned my addr 172.21.15.59:0/3770782424 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.639+0000 7fe45fe00640 1 --2- 172.21.15.59:0/3770782424 >> v2:172.21.15.76:3300/0 conn(0x7fe4600d13e0 0x7fe4600d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.639+0000 7fe45f400640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe46015ed20 0x7fe4600d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3527097949 0 0) 0x7fe4600b5da0 con 0x7fe4600b3600 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe444003520 con 0x7fe4600b3600 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2520952229 0 0) 0x7fe444003520 con 0x7fe4600b3600 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe46015ed20 msgr2=0x7fe4600d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe46015ed20 0x7fe4600d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 >> v2:172.21.15.76:3300/0 conn(0x7fe4600d13e0 msgr2=0x7fe4600d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 --2- 172.21.15.59:0/3770782424 >> v2:172.21.15.76:3300/0 conn(0x7fe4600d13e0 0x7fe4600d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4600d0a20 con 0x7fe4600b3600 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fe448001670 con 0x7fe4600b3600 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 >> v1:172.21.15.59:6789/0 conn(0x7fe4600b3600 legacy=0x7fe4600b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4440039b0 0x7fe444003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe46015ed20 0x7fe444004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600d07d0 con 0x7fe444004d40 2024-04-23T14:43:50.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe44400b9c0 con 0x7fe4440039b0 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe44400bbc0 con 0x7fe46015ed20 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.640+0000 7fe45fe00640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.641+0000 7fe45f400640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4440039b0 0x7fe444003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.641+0000 7fe45fe00640 1 -- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe46015ed20 msgr2=0x7fe444004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.641+0000 7fe45fe00640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe46015ed20 0x7fe444004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.641+0000 7fe45fe00640 1 -- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4440039b0 msgr2=0x7fe444003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.641+0000 7fe45fe00640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4440039b0 0x7fe444003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.641+0000 7fe45fe00640 1 -- 172.21.15.59:0/3770782424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe444003520 con 0x7fe444004d40 2024-04-23T14:43:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.641+0000 7fe45f400640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4440039b0 0x7fe444003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:50.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.642+0000 7fe45fe00640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe458009ac0 tx=0x7fe444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd0c0ce2704da2f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:50.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.642+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458036070 con 0x7fe444004d40 2024-04-23T14:43:50.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.642+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe458031400 con 0x7fe444004d40 2024-04-23T14:43:50.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.642+0000 7fe45e000640 1 -- 172.21.15.59:0/3770782424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458009690 con 0x7fe444004d40 2024-04-23T14:43:50.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.643+0000 7fe464e00640 1 -- 172.21.15.59:0/3770782424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe444004d40 msgr2=0x7fe4440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.643+0000 7fe464e00640 1 --2- 172.21.15.59:0/3770782424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe444004d40 0x7fe4440051c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe458009ac0 tx=0x7fe444005830 comp rx=0 tx=0).stop 2024-04-23T14:43:50.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.643+0000 7fe45fe00640 1 -- 172.21.15.59:0/3770782424 reap_dead start 2024-04-23T14:43:50.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.643+0000 7fe464e00640 1 -- 172.21.15.59:0/3770782424 shutdown_connections 2024-04-23T14:43:50.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.643+0000 7fe464e00640 1 -- 172.21.15.59:0/3770782424 >> 172.21.15.59:0/3770782424 conn(0x7fe46008bf90 msgr2=0x7fe4600d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:50.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.643+0000 7fe464e00640 1 -- 172.21.15.59:0/3770782424 shutdown_connections 2024-04-23T14:43:50.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.643+0000 7fe464e00640 1 -- 172.21.15.59:0/3770782424 wait complete. 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.645+0000 7fe464e00640 1 Processor -- start 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.645+0000 7fe464e00640 1 -- start start 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe464e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe444004d40 0x7fe4601f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe464e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4600cd020 0x7fe4601f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe464e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600ca490 0x7fe4600ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe464e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600d3490 con 0x7fe4600ca490 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe464e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600d31b0 con 0x7fe444004d40 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe464e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe4600d0ea0 con 0x7fe4600cd020 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe45fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600ca490 0x7fe4600ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe45fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600ca490 0x7fe4600ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40580/0 (socket says 172.21.15.59:40580) 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.646+0000 7fe45fe00640 1 -- 172.21.15.59:0/1938485710 learned_addr learned my addr 172.21.15.59:0/1938485710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45f400640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe444004d40 0x7fe4601f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45fe00640 1 -- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4600cd020 msgr2=0x7fe4601f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45fe00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4600cd020 0x7fe4601f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45fe00640 1 -- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe444004d40 msgr2=0x7fe4601f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45fe00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe444004d40 0x7fe4601f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45fe00640 1 -- 172.21.15.59:0/1938485710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4600cd7b0 con 0x7fe4600ca490 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45f400640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe444004d40 0x7fe4601f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:50.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe45fe00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600ca490 0x7fe4600ca910 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe458009be0 tx=0x7fe458004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:50.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe457400640 1 -- 172.21.15.59:0/1938485710 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458036040 con 0x7fe4600ca490 2024-04-23T14:43:50.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe457400640 1 -- 172.21.15.59:0/1938485710 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe458041070 con 0x7fe4600ca490 2024-04-23T14:43:50.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe457400640 1 -- 172.21.15.59:0/1938485710 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458009690 con 0x7fe4600ca490 2024-04-23T14:43:50.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.647+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4600cda90 con 0x7fe4600ca490 2024-04-23T14:43:50.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.648+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4600ce050 con 0x7fe4600ca490 2024-04-23T14:43:50.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.649+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46004fd90 con 0x7fe4600ca490 2024-04-23T14:43:50.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.650+0000 7fe457400640 1 -- 172.21.15.59:0/1938485710 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe458039070 con 0x7fe4600ca490 2024-04-23T14:43:50.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.651+0000 7fe457400640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe43c06c8f0 0x7fe43c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:50.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.651+0000 7fe457400640 1 -- 172.21.15.59:0/1938485710 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fe4580ab280 con 0x7fe4600ca490 2024-04-23T14:43:50.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.651+0000 7fe45f400640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe43c06c8f0 0x7fe43c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:50.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.652+0000 7fe45f400640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe43c06c8f0 0x7fe43c06edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe450007540 tx=0x7fe45000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:50.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.656+0000 7fe457400640 1 -- 172.21.15.59:0/1938485710 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe45804a020 con 0x7fe4600ca490 2024-04-23T14:43:50.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.863+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe4600b5360 con 0x7fe4600ca490 2024-04-23T14:43:50.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.864+0000 7fe457400640 1 -- 172.21.15.59:0/1938485710 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7fe4580796a0 con 0x7fe4600ca490 2024-04-23T14:43:50.864 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe43c06c8f0 msgr2=0x7fe43c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe43c06c8f0 0x7fe43c06edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe450007540 tx=0x7fe45000d040 comp rx=0 tx=0).stop 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600ca490 msgr2=0x7fe4600ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600ca490 0x7fe4600ca910 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe458009be0 tx=0x7fe458004730 comp rx=0 tx=0).stop 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 shutdown_connections 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe43c06c8f0 0x7fe43c06edb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4600cd020 0x7fe4601f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600ca490 0x7fe4600ca910 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 --2- 172.21.15.59:0/1938485710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe444004d40 0x7fe4601f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.866+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 >> 172.21.15.59:0/1938485710 conn(0x7fe46008bf90 msgr2=0x7fe46015ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.867+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 shutdown_connections 2024-04-23T14:43:50.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:50.867+0000 7fe464e00640 1 -- 172.21.15.59:0/1938485710 wait complete. 2024-04-23T14:43:50.887 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:43:51.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.092+0000 7fe701600640 1 Processor -- start 2024-04-23T14:43:51.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.092+0000 7fe701600640 1 -- start start 2024-04-23T14:43:51.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.093+0000 7fe701600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc09b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.093+0000 7fe701600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6fc09aa00 0x7fe6fc09ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.093+0000 7fe701600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6fc06d830 con 0x7fe6fc161ef0 2024-04-23T14:43:51.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.094+0000 7fe701600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe6fc09b390 con 0x7fe6fc09aa00 2024-04-23T14:43:51.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.094+0000 7fe701600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc06db10 con 0x7fe6fc0b1930 2024-04-23T14:43:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.094+0000 7fe6fb400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6fc09aa00 0x7fe6fc09ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.094+0000 7fe6fa000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc09b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.094+0000 7fe6fb400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6fc09aa00 0x7fe6fc09ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:57086/0 (socket says 172.21.15.59:57086) 2024-04-23T14:43:51.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.094+0000 7fe6fb400640 1 -- 172.21.15.59:0/2773327952 learned_addr learned my addr 172.21.15.59:0/2773327952 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.095+0000 7fe6fb400640 1 -- 172.21.15.59:0/2773327952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 msgr2=0x7fe6fc09b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.095+0000 7fe6fb400640 1 --2- 172.21.15.59:0/2773327952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc09b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.095+0000 7fe6fb400640 1 -- 172.21.15.59:0/2773327952 >> v1:172.21.15.76:6789/0 conn(0x7fe6fc161ef0 legacy=0x7fe6fc0b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.095+0000 7fe6fb400640 1 -- 172.21.15.59:0/2773327952 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc06ddf0 con 0x7fe6fc09aa00 2024-04-23T14:43:51.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.096+0000 7fe6fb400640 1 --2- 172.21.15.59:0/2773327952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 0x7fe6fc09ae50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ea31a0a51a2d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.097+0000 7fe6f9600640 1 -- 172.21.15.59:0/2773327952 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4036070 con 0x7fe6fc09aa00 2024-04-23T14:43:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.097+0000 7fe6f9600640 1 -- 172.21.15.59:0/2773327952 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f4031600 con 0x7fe6fc09aa00 2024-04-23T14:43:51.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.097+0000 7fe6f9600640 1 -- 172.21.15.59:0/2773327952 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40319d0 con 0x7fe6fc09aa00 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 -- 172.21.15.59:0/2773327952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 msgr2=0x7fe6fc09ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 --2- 172.21.15.59:0/2773327952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 0x7fe6fc09ae50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f402a680 comp rx=0 tx=0).stop 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 -- 172.21.15.59:0/2773327952 shutdown_connections 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 --2- 172.21.15.59:0/2773327952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 0x7fe6fc09ae50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 --2- 172.21.15.59:0/2773327952 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc09b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 -- 172.21.15.59:0/2773327952 >> 172.21.15.59:0/2773327952 conn(0x7fe6fc0b4030 msgr2=0x7fe6fc0b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 -- 172.21.15.59:0/2773327952 shutdown_connections 2024-04-23T14:43:51.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.098+0000 7fe701600640 1 -- 172.21.15.59:0/2773327952 wait complete. 2024-04-23T14:43:51.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.100+0000 7fe701600640 1 Processor -- start 2024-04-23T14:43:51.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.100+0000 7fe701600640 1 -- start start 2024-04-23T14:43:51.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe701600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 0x7fe6fc150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe701600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc0b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe701600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6fc161ef0 0x7fe6fc15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe701600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc0d08b0 con 0x7fe6fc161ef0 2024-04-23T14:43:51.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe701600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1622d0 con 0x7fe6fc09aa00 2024-04-23T14:43:51.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe701600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc0d1120 con 0x7fe6fc0b1930 2024-04-23T14:43:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe6fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6fc161ef0 0x7fe6fc15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe6fb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6fc161ef0 0x7fe6fc15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40586/0 (socket says 172.21.15.59:40586) 2024-04-23T14:43:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe6fb400640 1 -- 172.21.15.59:0/1212360677 learned_addr learned my addr 172.21.15.59:0/1212360677 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe6fa000640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc0b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.101+0000 7fe6faa00640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 0x7fe6fc150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6fb400640 1 -- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 msgr2=0x7fe6fc0b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6fb400640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc0b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6fb400640 1 -- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 msgr2=0x7fe6fc150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6fb400640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 0x7fe6fc150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6fb400640 1 -- 172.21.15.59:0/1212360677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc0c62a0 con 0x7fe6fc161ef0 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6fb400640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6fc161ef0 0x7fe6fc15e340 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6f2a00640 1 -- 172.21.15.59:0/1212360677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4036050 con 0x7fe6fc161ef0 2024-04-23T14:43:51.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6f2a00640 1 -- 172.21.15.59:0/1212360677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f4042070 con 0x7fe6fc161ef0 2024-04-23T14:43:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc0c6580 con 0x7fe6fc161ef0 2024-04-23T14:43:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe6f2a00640 1 -- 172.21.15.59:0/1212360677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4009510 con 0x7fe6fc161ef0 2024-04-23T14:43:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.102+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc0c6b40 con 0x7fe6fc161ef0 2024-04-23T14:43:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.103+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc068a80 con 0x7fe6fc161ef0 2024-04-23T14:43:51.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.104+0000 7fe6f2a00640 1 -- 172.21.15.59:0/1212360677 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6f4007bf0 con 0x7fe6fc161ef0 2024-04-23T14:43:51.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.105+0000 7fe6f2a00640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6dc06c8f0 0x7fe6dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.105+0000 7fe6f2a00640 1 -- 172.21.15.59:0/1212360677 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fe6f40ac960 con 0x7fe6fc161ef0 2024-04-23T14:43:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.106+0000 7fe6faa00640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6dc06c8f0 0x7fe6dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.106+0000 7fe6faa00640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6dc06c8f0 0x7fe6dc06edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec007fd0 tx=0x7fe6ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:51.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.110+0000 7fe6f2a00640 1 -- 172.21.15.59:0/1212360677 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6f407ad80 con 0x7fe6fc161ef0 2024-04-23T14:43:51.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.328+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe6fc06e200 con 0x7fe6fc161ef0 2024-04-23T14:43:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.329+0000 7fe6f2a00640 1 -- 172.21.15.59:0/1212360677 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7fe6f407a680 con 0x7fe6fc161ef0 2024-04-23T14:43:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:43:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6dc06c8f0 msgr2=0x7fe6dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6dc06c8f0 0x7fe6dc06edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec007fd0 tx=0x7fe6ec009040 comp rx=0 tx=0).stop 2024-04-23T14:43:51.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6fc161ef0 msgr2=0x7fe6fc15e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6fc161ef0 0x7fe6fc15e340 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f4004a40 comp rx=0 tx=0).stop 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 shutdown_connections 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6dc06c8f0 0x7fe6dc06edb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6fc0b1930 0x7fe6fc0b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6fc161ef0 0x7fe6fc15e340 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 --2- 172.21.15.59:0/1212360677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6fc09aa00 0x7fe6fc150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 >> 172.21.15.59:0/1212360677 conn(0x7fe6fc0b4030 msgr2=0x7fe6fc15d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 shutdown_connections 2024-04-23T14:43:51.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.332+0000 7fe701600640 1 -- 172.21.15.59:0/1212360677 wait complete. 2024-04-23T14:43:51.352 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:43:51.352 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:43:51.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:43:51.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.616+0000 7f2e7ca00640 1 Processor -- start 2024-04-23T14:43:51.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.616+0000 7f2e7ca00640 1 -- start start 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.617+0000 7f2e7ca00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e780cd020 0x7f2e780b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.617+0000 7f2e7ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e780b3e40 0x7f2e78160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.617+0000 7f2e7ca00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e780cd520 con 0x7f2e781454e0 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e7ca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2e780cddc0 con 0x7f2e780b3e40 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e7ca00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2e781462c0 con 0x7f2e780cd020 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e75e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2e781454e0 0x7f2e78145bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40744/0 (socket says 172.21.15.59:40744) 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e75e00640 1 -- 172.21.15.59:0/2224661793 learned_addr learned my addr 172.21.15.59:0/2224661793 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:51.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e76800640 1 --2- 172.21.15.59:0/2224661793 >> v2:172.21.15.59:3300/0 conn(0x7f2e780b3e40 0x7f2e78160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e75400640 1 -- 172.21.15.59:0/2224661793 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 281357605 0 0) 0x7f2e780cd520 con 0x7f2e781454e0 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e6fe00640 1 --2- 172.21.15.59:0/2224661793 >> v2:172.21.15.76:3300/0 conn(0x7f2e780cd020 0x7f2e780b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e75400640 1 -- 172.21.15.59:0/2224661793 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e5c003520 con 0x7f2e781454e0 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e76800640 1 -- 172.21.15.59:0/2224661793 >> v2:172.21.15.76:3300/0 conn(0x7f2e780cd020 msgr2=0x7f2e780b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e76800640 1 --2- 172.21.15.59:0/2224661793 >> v2:172.21.15.76:3300/0 conn(0x7f2e780cd020 0x7f2e780b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e76800640 1 -- 172.21.15.59:0/2224661793 >> v1:172.21.15.59:6789/0 conn(0x7f2e781454e0 legacy=0x7f2e78145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.618+0000 7f2e76800640 1 -- 172.21.15.59:0/2224661793 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e780cd800 con 0x7f2e780b3e40 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.619+0000 7f2e76800640 1 --2- 172.21.15.59:0/2224661793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e78160910 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f2e780d1490 tx=0x7f2e7002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634c07109b992a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.620+0000 7f2e75400640 1 -- 172.21.15.59:0/2224661793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70007740 con 0x7f2e780b3e40 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.620+0000 7f2e75400640 1 -- 172.21.15.59:0/2224661793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e700078e0 con 0x7f2e780b3e40 2024-04-23T14:43:51.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.620+0000 7f2e75400640 1 -- 172.21.15.59:0/2224661793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e70007cb0 con 0x7f2e780b3e40 2024-04-23T14:43:51.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.620+0000 7f2e7ca00640 1 -- 172.21.15.59:0/2224661793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 msgr2=0x7f2e78160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.620+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/2224661793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e78160910 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f2e780d1490 tx=0x7f2e7002a5f0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.620+0000 7f2e7ca00640 1 -- 172.21.15.59:0/2224661793 shutdown_connections 2024-04-23T14:43:51.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.621+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/2224661793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e78160910 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.621+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/2224661793 >> v2:172.21.15.76:3300/0 conn(0x7f2e780cd020 0x7f2e780b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.621+0000 7f2e7ca00640 1 -- 172.21.15.59:0/2224661793 >> 172.21.15.59:0/2224661793 conn(0x7f2e78067ea0 msgr2=0x7f2e7806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.621+0000 7f2e7ca00640 1 -- 172.21.15.59:0/2224661793 shutdown_connections 2024-04-23T14:43:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.621+0000 7f2e7ca00640 1 -- 172.21.15.59:0/2224661793 wait complete. 2024-04-23T14:43:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.622+0000 7f2e7ca00640 1 Processor -- start 2024-04-23T14:43:51.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.623+0000 7f2e7ca00640 1 -- start start 2024-04-23T14:43:51.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.623+0000 7f2e7ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e7813efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.623+0000 7f2e7ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e780cd020 0x7f2e7815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e7ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e781454e0 0x7f2e7815d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e7ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78160e50 con 0x7f2e780b3e40 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e7ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781603f0 con 0x7f2e780cd020 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e7ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e78160fd0 con 0x7f2e781454e0 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e7813efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e7813efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40590/0 (socket says 172.21.15.59:40590) 2024-04-23T14:43:51.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 -- 172.21.15.59:0/780684766 learned_addr learned my addr 172.21.15.59:0/780684766 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e76800640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e781454e0 0x7f2e7815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e6fe00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e780cd020 0x7f2e7815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 -- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e781454e0 msgr2=0x7f2e7815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e781454e0 0x7f2e7815d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 -- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e780cd020 msgr2=0x7f2e7815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e780cd020 0x7f2e7815eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.624+0000 7f2e75e00640 1 -- 172.21.15.59:0/780684766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7813f6f0 con 0x7f2e780b3e40 2024-04-23T14:43:51.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.625+0000 7f2e75e00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e7813efd0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c390 tx=0x7f2e6800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.625+0000 7f2e6e000640 1 -- 172.21.15.59:0/780684766 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68017070 con 0x7f2e780b3e40 2024-04-23T14:43:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.625+0000 7f2e6e000640 1 -- 172.21.15.59:0/780684766 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e68005ce0 con 0x7f2e780b3e40 2024-04-23T14:43:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.625+0000 7f2e6e000640 1 -- 172.21.15.59:0/780684766 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e680072f0 con 0x7f2e780b3e40 2024-04-23T14:43:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.625+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7815f1c0 con 0x7f2e780b3e40 2024-04-23T14:43:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.625+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7815f780 con 0x7f2e780b3e40 2024-04-23T14:43:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.626+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e78068a80 con 0x7f2e780b3e40 2024-04-23T14:43:51.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.628+0000 7f2e6e000640 1 -- 172.21.15.59:0/780684766 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e6801a070 con 0x7f2e780b3e40 2024-04-23T14:43:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.629+0000 7f2e6e000640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e5406c8f0 0x7f2e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.629+0000 7f2e6e000640 1 -- 172.21.15.59:0/780684766 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f2e6808af80 con 0x7f2e780b3e40 2024-04-23T14:43:51.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.629+0000 7f2e6fe00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e5406c8f0 0x7f2e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:51.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.630+0000 7f2e6fe00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e5406c8f0 0x7f2e5406edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2e781f7ea0 tx=0x7f2e60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:51.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.634+0000 7f2e6e000640 1 -- 172.21.15.59:0/780684766 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e680593a0 con 0x7f2e780b3e40 2024-04-23T14:43:51.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.839+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e78160030 con 0x7f2e780b3e40 2024-04-23T14:43:51.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.842+0000 7f2e6e000640 1 -- 172.21.15.59:0/780684766 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16567 (secure 0 0 0) 0x7f2e68058ca0 con 0x7f2e780b3e40 2024-04-23T14:43:51.845 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:43:51.845 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":112,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:39.410742+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:43:51.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e5406c8f0 msgr2=0x7f2e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e5406c8f0 0x7f2e5406edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2e781f7ea0 tx=0x7f2e60008040 comp rx=0 tx=0).stop 2024-04-23T14:43:51.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 msgr2=0x7f2e7813efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:51.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e7813efd0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c390 tx=0x7f2e6800c850 comp rx=0 tx=0).stop 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 shutdown_connections 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e5406c8f0 0x7f2e5406edb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e781454e0 0x7f2e7815d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e780cd020 0x7f2e7815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 --2- 172.21.15.59:0/780684766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e780b3e40 0x7f2e7813efd0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 >> 172.21.15.59:0/780684766 conn(0x7f2e78067ea0 msgr2=0x7f2e780727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 shutdown_connections 2024-04-23T14:43:51.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:51.845+0000 7f2e7ca00640 1 -- 172.21.15.59:0/780684766 wait complete. 2024-04-23T14:43:51.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:43:51.866 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:43:51.912 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:43:51.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:43:52.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.076+0000 7f9c1ae00640 1 Processor -- start 2024-04-23T14:43:52.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.076+0000 7f9c1ae00640 1 -- start start 2024-04-23T14:43:52.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.078+0000 7f9c1ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c14162c10 0x7f9c140d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.078+0000 7f9c1ae00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c14072480 0x7f9c1406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.078+0000 7f9c1ae00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c140d0fd0 con 0x7f9c140b4670 2024-04-23T14:43:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.078+0000 7f9c1ae00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9c140b35d0 con 0x7f9c14072480 2024-04-23T14:43:52.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.078+0000 7f9c1ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9c140b38b0 con 0x7f9c14162c10 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.079+0000 7f9c18e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c14072480 0x7f9c1406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.079+0000 7f9c13e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c14162c10 0x7f9c140d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.079+0000 7f9c13400640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9c140b4670 0x7f9c14071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:59452/0 (socket says 172.21.15.59:59452) 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.079+0000 7f9c13400640 1 -- 172.21.15.59:0/3910174895 learned_addr learned my addr 172.21.15.59:0/3910174895 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.080+0000 7f9c13e00640 1 -- 172.21.15.59:0/3910174895 >> v2:172.21.15.76:3300/0 conn(0x7f9c14072480 msgr2=0x7f9c1406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.080+0000 7f9c13e00640 1 --2- 172.21.15.59:0/3910174895 >> v2:172.21.15.76:3300/0 conn(0x7f9c14072480 0x7f9c1406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.080+0000 7f9c13e00640 1 -- 172.21.15.59:0/3910174895 >> v1:172.21.15.76:6789/0 conn(0x7f9c140b4670 legacy=0x7f9c14071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.080+0000 7f9c13e00640 1 -- 172.21.15.59:0/3910174895 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c140b3b90 con 0x7f9c14162c10 2024-04-23T14:43:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.080+0000 7f9c12a00640 1 -- 172.21.15.59:0/3910174895 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796716583 0 0) 0x7f9c00001490 con 0x7f9c140b4670 2024-04-23T14:43:52.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.081+0000 7f9c13e00640 1 --2- 172.21.15.59:0/3910174895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c14162c10 0x7f9c140d0a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c040097a0 tx=0x7f9c0402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b375e47bb402451e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.082+0000 7f9c12a00640 1 -- 172.21.15.59:0/3910174895 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04037070 con 0x7f9c14162c10 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.082+0000 7f9c12a00640 1 -- 172.21.15.59:0/3910174895 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c04032440 con 0x7f9c14162c10 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.082+0000 7f9c12a00640 1 -- 172.21.15.59:0/3910174895 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04032810 con 0x7f9c14162c10 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.082+0000 7f9c1ae00640 1 -- 172.21.15.59:0/3910174895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c14162c10 msgr2=0x7f9c140d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.082+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/3910174895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c14162c10 0x7f9c140d0a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c040097a0 tx=0x7f9c0402a180 comp rx=0 tx=0).stop 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.083+0000 7f9c1ae00640 1 -- 172.21.15.59:0/3910174895 shutdown_connections 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.083+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/3910174895 >> v2:172.21.15.76:3300/0 conn(0x7f9c14072480 0x7f9c1406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.083+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/3910174895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c14162c10 0x7f9c140d0a90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.083+0000 7f9c1ae00640 1 -- 172.21.15.59:0/3910174895 >> 172.21.15.59:0/3910174895 conn(0x7f9c14067d90 msgr2=0x7f9c1406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.083+0000 7f9c1ae00640 1 -- 172.21.15.59:0/3910174895 shutdown_connections 2024-04-23T14:43:52.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.083+0000 7f9c1ae00640 1 -- 172.21.15.59:0/3910174895 wait complete. 2024-04-23T14:43:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.084+0000 7f9c1ae00640 1 Processor -- start 2024-04-23T14:43:52.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.085+0000 7f9c1ae00640 1 -- start start 2024-04-23T14:43:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.085+0000 7f9c1ae00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c14072480 0x7f9c1415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.085+0000 7f9c1ae00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c140b4670 0x7f9c14163f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c1ae00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c14162c10 0x7f9c14164490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c1ae00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c14162ff0 con 0x7f9c14162c10 2024-04-23T14:43:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c1ae00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9c140b4150 con 0x7f9c14072480 2024-04-23T14:43:52.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c1ae00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9c14163170 con 0x7f9c140b4670 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c14162c10 0x7f9c14164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c14162c10 0x7f9c14164490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40592/0 (socket says 172.21.15.59:40592) 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 -- 172.21.15.59:0/709877880 learned_addr learned my addr 172.21.15.59:0/709877880 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c13e00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c14072480 0x7f9c1415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c13400640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c140b4670 0x7f9c14163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 -- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c140b4670 msgr2=0x7f9c14163f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c140b4670 0x7f9c14163f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 -- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c14072480 msgr2=0x7f9c1415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c14072480 0x7f9c1415e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.086+0000 7f9c18e00640 1 -- 172.21.15.59:0/709877880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1415e670 con 0x7f9c14162c10 2024-04-23T14:43:52.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.087+0000 7f9c18e00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c14162c10 0x7f9c14164490 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00e930 tx=0x7f9c0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:52.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.087+0000 7f9c0be00640 1 -- 172.21.15.59:0/709877880 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c017070 con 0x7f9c14162c10 2024-04-23T14:43:52.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.087+0000 7f9c0be00640 1 -- 172.21.15.59:0/709877880 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0c00c980 con 0x7f9c14162c10 2024-04-23T14:43:52.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.087+0000 7f9c0be00640 1 -- 172.21.15.59:0/709877880 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00cd70 con 0x7f9c14162c10 2024-04-23T14:43:52.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.087+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c140aeed0 con 0x7f9c14162c10 2024-04-23T14:43:52.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.087+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c141f8a90 con 0x7f9c14162c10 2024-04-23T14:43:52.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.088+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c14068a80 con 0x7f9c14162c10 2024-04-23T14:43:52.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.090+0000 7f9c0be00640 1 -- 172.21.15.59:0/709877880 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c0c01a070 con 0x7f9c14162c10 2024-04-23T14:43:52.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.091+0000 7f9c0be00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bf406c8f0 0x7f9bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.091+0000 7f9c0be00640 1 -- 172.21.15.59:0/709877880 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f9c0c08bba0 con 0x7f9c14162c10 2024-04-23T14:43:52.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.091+0000 7f9c13e00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bf406c8f0 0x7f9bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.092+0000 7f9c13e00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bf406c8f0 0x7f9bf406edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9c04034000 tx=0x7f9c0402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:52.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.096+0000 7f9c0be00640 1 -- 172.21.15.59:0/709877880 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c140aeed0 con 0x7f9c14162c10 2024-04-23T14:43:52.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.300+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c1406bd50 con 0x7f9c14162c10 2024-04-23T14:43:52.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.301+0000 7f9c0be00640 1 -- 172.21.15.59:0/709877880 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f9c0c05a080 con 0x7f9c14162c10 2024-04-23T14:43:52.301 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:43:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bf406c8f0 msgr2=0x7f9bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bf406c8f0 0x7f9bf406edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9c04034000 tx=0x7f9c0402e000 comp rx=0 tx=0).stop 2024-04-23T14:43:52.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c14162c10 msgr2=0x7f9c14164490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c14162c10 0x7f9c14164490 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00e930 tx=0x7f9c0c00edf0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 shutdown_connections 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9bf406c8f0 0x7f9bf406edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9c140b4670 0x7f9c14163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c14162c10 0x7f9c14164490 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 --2- 172.21.15.59:0/709877880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c14072480 0x7f9c1415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 >> 172.21.15.59:0/709877880 conn(0x7f9c14067d90 msgr2=0x7f9c14165ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 shutdown_connections 2024-04-23T14:43:52.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.304+0000 7f9c1ae00640 1 -- 172.21.15.59:0/709877880 wait complete. 2024-04-23T14:43:52.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:43:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.544+0000 7f1bdcc00640 1 Processor -- start 2024-04-23T14:43:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.544+0000 7f1bdcc00640 1 -- start start 2024-04-23T14:43:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.546+0000 7f1bdcc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1bd80d0a60 0x7f1bd806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.546+0000 7f1bdcc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd815bf60 con 0x7f1bd80b56f0 2024-04-23T14:43:52.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.546+0000 7f1bdcc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd815c240 con 0x7f1bd806fa50 2024-04-23T14:43:52.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.546+0000 7f1bdcc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd815c520 con 0x7f1bd80d0a60 2024-04-23T14:43:52.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd5e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1bd80b56f0 0x7f1bd806f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40754/0 (socket says 172.21.15.59:40754) 2024-04-23T14:43:52.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd5e00640 1 -- 172.21.15.59:0/563687247 learned_addr learned my addr 172.21.15.59:0/563687247 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:52.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd6800640 1 --2- 172.21.15.59:0/563687247 >> v2:172.21.15.59:3300/0 conn(0x7f1bd80d0a60 0x7f1bd806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd4a00640 1 -- 172.21.15.59:0/563687247 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222283652 0 0) 0x7f1bd815bf60 con 0x7f1bd80b56f0 2024-04-23T14:43:52.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd4a00640 1 -- 172.21.15.59:0/563687247 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bbc005730 con 0x7f1bd80b56f0 2024-04-23T14:43:52.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd6800640 1 -- 172.21.15.59:0/563687247 >> v1:172.21.15.76:6789/0 conn(0x7f1bd806fa50 legacy=0x7f1bd80d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd6800640 1 -- 172.21.15.59:0/563687247 >> v1:172.21.15.59:6789/0 conn(0x7f1bd80b56f0 legacy=0x7f1bd806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.547+0000 7f1bd6800640 1 -- 172.21.15.59:0/563687247 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd815c800 con 0x7f1bd80d0a60 2024-04-23T14:43:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.549+0000 7f1bd6800640 1 --2- 172.21.15.59:0/563687247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80d0a60 0x7f1bd806ddd0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009d90 tx=0x7f1bd002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95dd556994ec06d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.549+0000 7f1bd4a00640 1 -- 172.21.15.59:0/563687247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0007740 con 0x7f1bd80d0a60 2024-04-23T14:43:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.549+0000 7f1bd4a00640 1 -- 172.21.15.59:0/563687247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd00078e0 con 0x7f1bd80d0a60 2024-04-23T14:43:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.549+0000 7f1bd4a00640 1 -- 172.21.15.59:0/563687247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0007cb0 con 0x7f1bd80d0a60 2024-04-23T14:43:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.550+0000 7f1bdcc00640 1 -- 172.21.15.59:0/563687247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80d0a60 msgr2=0x7f1bd806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.550+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/563687247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80d0a60 0x7f1bd806ddd0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009d90 tx=0x7f1bd002a6f0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.550+0000 7f1bdcc00640 1 -- 172.21.15.59:0/563687247 shutdown_connections 2024-04-23T14:43:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.550+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/563687247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80d0a60 0x7f1bd806ddd0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.550+0000 7f1bdcc00640 1 -- 172.21.15.59:0/563687247 >> 172.21.15.59:0/563687247 conn(0x7f1bd8067d90 msgr2=0x7f1bd806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:52.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.550+0000 7f1bdcc00640 1 -- 172.21.15.59:0/563687247 shutdown_connections 2024-04-23T14:43:52.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.550+0000 7f1bdcc00640 1 -- 172.21.15.59:0/563687247 wait complete. 2024-04-23T14:43:52.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.551+0000 7f1bdcc00640 1 Processor -- start 2024-04-23T14:43:52.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.552+0000 7f1bdcc00640 1 -- start start 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bdcc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bd806fa50 0x7f1bd81f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bdcc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80b56f0 0x7f1bd81487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bdcc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bd80d0a60 0x7f1bd80cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bdcc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd80b3710 con 0x7f1bd80b56f0 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bdcc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd815bc10 con 0x7f1bd80d0a60 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bdcc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd80b3890 con 0x7f1bd806fa50 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80b56f0 0x7f1bd81487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80b56f0 0x7f1bd81487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40616/0 (socket says 172.21.15.59:40616) 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 -- 172.21.15.59:0/2380785563 learned_addr learned my addr 172.21.15.59:0/2380785563 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5e00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bd806fa50 0x7f1bd81f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd6800640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bd80d0a60 0x7f1bd80cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 -- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bd806fa50 msgr2=0x7f1bd81f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bd806fa50 0x7f1bd81f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 -- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bd80d0a60 msgr2=0x7f1bd80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bd80d0a60 0x7f1bd80cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd5400640 1 -- 172.21.15.59:0/2380785563 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd815d580 con 0x7f1bd80b56f0 2024-04-23T14:43:52.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.553+0000 7f1bd6800640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bd80d0a60 0x7f1bd80cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:43:52.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.554+0000 7f1bd5400640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80b56f0 0x7f1bd81487e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400c400 tx=0x7f1bc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:52.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.554+0000 7f1bce000640 1 -- 172.21.15.59:0/2380785563 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40052d0 con 0x7f1bd80b56f0 2024-04-23T14:43:52.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.554+0000 7f1bce000640 1 -- 172.21.15.59:0/2380785563 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc4005490 con 0x7f1bd80b56f0 2024-04-23T14:43:52.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.554+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd81f4b20 con 0x7f1bd80b56f0 2024-04-23T14:43:52.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.554+0000 7f1bce000640 1 -- 172.21.15.59:0/2380785563 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4007260 con 0x7f1bd80b56f0 2024-04-23T14:43:52.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.554+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd8149310 con 0x7f1bd80b56f0 2024-04-23T14:43:52.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.555+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd8068a80 con 0x7f1bd80b56f0 2024-04-23T14:43:52.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.557+0000 7f1bce000640 1 -- 172.21.15.59:0/2380785563 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bc4007400 con 0x7f1bd80b56f0 2024-04-23T14:43:52.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.559+0000 7f1bce000640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bb806c8f0 0x7f1bb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:52.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.559+0000 7f1bce000640 1 -- 172.21.15.59:0/2380785563 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f1bc408b120 con 0x7f1bd80b56f0 2024-04-23T14:43:52.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.559+0000 7f1bd5e00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bb806c8f0 0x7f1bb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:52.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.560+0000 7f1bd5e00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bb806c8f0 0x7f1bb806edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80011e0 tx=0x7f1bc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:52.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.563+0000 7f1bce000640 1 -- 172.21.15.59:0/2380785563 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bc40594b0 con 0x7f1bd80b56f0 2024-04-23T14:43:52.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.774+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1bd806f3b0 con 0x7f1bd80b56f0 2024-04-23T14:43:52.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.775+0000 7f1bce000640 1 -- 172.21.15.59:0/2380785563 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7f1bc4058db0 con 0x7f1bd80b56f0 2024-04-23T14:43:52.776 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bb806c8f0 msgr2=0x7f1bb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bb806c8f0 0x7f1bb806edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80011e0 tx=0x7f1bc8009040 comp rx=0 tx=0).stop 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80b56f0 msgr2=0x7f1bd81487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80b56f0 0x7f1bd81487e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400c400 tx=0x7f1bc400c8c0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 shutdown_connections 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1bb806c8f0 0x7f1bb806edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bd80d0a60 0x7f1bd80cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1bd80b56f0 0x7f1bd81487e0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 --2- 172.21.15.59:0/2380785563 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1bd806fa50 0x7f1bd81f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.778+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 >> 172.21.15.59:0/2380785563 conn(0x7f1bd8067d90 msgr2=0x7f1bd806ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.779+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 shutdown_connections 2024-04-23T14:43:52.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:52.779+0000 7f1bdcc00640 1 -- 172.21.15.59:0/2380785563 wait complete. 2024-04-23T14:43:52.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:43:53.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.013+0000 7f8e1d600640 1 Processor -- start 2024-04-23T14:43:53.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.013+0000 7f8e1d600640 1 -- start start 2024-04-23T14:43:53.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.015+0000 7f8e1d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 0x7f8e1806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.015+0000 7f8e1d600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e180cfc20 con 0x7f8e18071d70 2024-04-23T14:43:53.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.015+0000 7f8e1d600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e180cff00 con 0x7f8e18145e10 2024-04-23T14:43:53.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.015+0000 7f8e1d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8e180d01e0 con 0x7f8e180d0cc0 2024-04-23T14:43:53.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.015+0000 7f8e17400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8e18071d70 0x7f8e1815cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40770/0 (socket says 172.21.15.59:40770) 2024-04-23T14:43:53.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.015+0000 7f8e17400640 1 -- 172.21.15.59:0/226726079 learned_addr learned my addr 172.21.15.59:0/226726079 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e16000640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 0x7f8e1806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745498362 0 0) 0x7f8e180cfc20 con 0x7f8e18071d70 2024-04-23T14:43:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dfc003520 con 0x7f8e18071d70 2024-04-23T14:43:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 68129595 0 0) 0x7f8dfc003520 con 0x7f8e18071d70 2024-04-23T14:43:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 msgr2=0x7f8e1806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 0x7f8e1806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 >> v1:172.21.15.76:6789/0 conn(0x7f8e18145e10 legacy=0x7f8e180cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e180d04c0 con 0x7f8e18071d70 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f8e10001670 con 0x7f8e18071d70 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 >> v1:172.21.15.59:6789/0 conn(0x7f8e18071d70 legacy=0x7f8e1815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dfc0039e0 0x7f8dfc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dfc004800 0x7f8dfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 0x7f8dfc0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e180d01e0 con 0x7f8dfc004800 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc00bad0 con 0x7f8dfc0039e0 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.016+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8dfc00bcd0 con 0x7f8e180d0cc0 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e16000640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dfc004800 0x7f8dfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e16a00640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dfc0039e0 0x7f8dfc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e17400640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 0x7f8dfc0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e16000640 1 -- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 msgr2=0x7f8dfc0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e16000640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e180d0cc0 0x7f8dfc0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e16000640 1 -- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dfc0039e0 msgr2=0x7f8dfc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e16000640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dfc0039e0 0x7f8dfc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.017+0000 7f8e16000640 1 -- 172.21.15.59:0/226726079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc003520 con 0x7f8dfc004800 2024-04-23T14:43:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.019+0000 7f8e16000640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dfc004800 0x7f8dfc004c80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8dfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd062c830c939bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.019+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00036070 con 0x7f8dfc004800 2024-04-23T14:43:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.019+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e00031400 con 0x7f8dfc004800 2024-04-23T14:43:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.019+0000 7f8e15600640 1 -- 172.21.15.59:0/226726079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00009690 con 0x7f8dfc004800 2024-04-23T14:43:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.020+0000 7f8e1d600640 1 -- 172.21.15.59:0/226726079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dfc004800 msgr2=0x7f8dfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.020+0000 7f8e1d600640 1 --2- 172.21.15.59:0/226726079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dfc004800 0x7f8dfc004c80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8dfc005830 comp rx=0 tx=0).stop 2024-04-23T14:43:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.020+0000 7f8e17400640 1 -- 172.21.15.59:0/226726079 reap_dead start 2024-04-23T14:43:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.020+0000 7f8e1d600640 1 -- 172.21.15.59:0/226726079 shutdown_connections 2024-04-23T14:43:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.020+0000 7f8e1d600640 1 -- 172.21.15.59:0/226726079 >> 172.21.15.59:0/226726079 conn(0x7f8e18067d90 msgr2=0x7f8e1806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.020+0000 7f8e1d600640 1 -- 172.21.15.59:0/226726079 shutdown_connections 2024-04-23T14:43:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.020+0000 7f8e1d600640 1 -- 172.21.15.59:0/226726079 wait complete. 2024-04-23T14:43:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.022+0000 7f8e1d600640 1 Processor -- start 2024-04-23T14:43:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.022+0000 7f8e1d600640 1 -- start start 2024-04-23T14:43:53.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.023+0000 7f8e1d600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1813e1d0 0x7f8e1813e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.023+0000 7f8e1d600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1813ef40 0x7f8e1815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.023+0000 7f8e1d600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e181e70b0 0x7f8e181e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.023+0000 7f8e1d600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e181461f0 con 0x7f8e181e70b0 2024-04-23T14:43:53.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.023+0000 7f8e1d600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1806f130 con 0x7f8e1813e1d0 2024-04-23T14:43:53.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.023+0000 7f8e1d600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1815ab10 con 0x7f8e1813ef40 2024-04-23T14:43:53.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e181e70b0 0x7f8e181e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e181e70b0 0x7f8e181e7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40632/0 (socket says 172.21.15.59:40632) 2024-04-23T14:43:53.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 -- 172.21.15.59:0/3634806685 learned_addr learned my addr 172.21.15.59:0/3634806685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:53.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e16a00640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1813e1d0 0x7f8e1813e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e16000640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1813ef40 0x7f8e1815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 -- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1813ef40 msgr2=0x7f8e1815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1813ef40 0x7f8e1815b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 -- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1813e1d0 msgr2=0x7f8e1813e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1813e1d0 0x7f8e1813e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.024+0000 7f8e17400640 1 -- 172.21.15.59:0/3634806685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e181e60e0 con 0x7f8e181e70b0 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.025+0000 7f8e17400640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e181e70b0 0x7f8e181e7530 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004e40 tx=0x7f8e10007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.025+0000 7f8e0ea00640 1 -- 172.21.15.59:0/3634806685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10018070 con 0x7f8e181e70b0 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.025+0000 7f8e0ea00640 1 -- 172.21.15.59:0/3634806685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1000f730 con 0x7f8e181e70b0 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.025+0000 7f8e0ea00640 1 -- 172.21.15.59:0/3634806685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1000fb00 con 0x7f8e181e70b0 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.025+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e181e63c0 con 0x7f8e181e70b0 2024-04-23T14:43:53.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.025+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1815b0b0 con 0x7f8e181e70b0 2024-04-23T14:43:53.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.026+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e18068a80 con 0x7f8e181e70b0 2024-04-23T14:43:53.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.028+0000 7f8e0ea00640 1 -- 172.21.15.59:0/3634806685 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e1001b070 con 0x7f8e181e70b0 2024-04-23T14:43:53.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.029+0000 7f8e0ea00640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8df406c8f0 0x7f8df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.029+0000 7f8e0ea00640 1 -- 172.21.15.59:0/3634806685 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f8e1008ad80 con 0x7f8e181e70b0 2024-04-23T14:43:53.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.029+0000 7f8e16a00640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8df406c8f0 0x7f8df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.030+0000 7f8e16a00640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8df406c8f0 0x7f8df406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8e08001e70 tx=0x7f8e08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.033+0000 7f8e0ea00640 1 -- 172.21.15.59:0/3634806685 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e10058ac0 con 0x7f8e181e70b0 2024-04-23T14:43:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.239+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e180d1190 con 0x7f8e181e70b0 2024-04-23T14:43:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.240+0000 7f8e0ea00640 1 -- 172.21.15.59:0/3634806685 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7f8e1005cdc0 con 0x7f8e181e70b0 2024-04-23T14:43:53.240 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:43:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8df406c8f0 msgr2=0x7f8df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8df406c8f0 0x7f8df406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8e08001e70 tx=0x7f8e08009040 comp rx=0 tx=0).stop 2024-04-23T14:43:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e181e70b0 msgr2=0x7f8e181e7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e181e70b0 0x7f8e181e7530 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f8e10004e40 tx=0x7f8e10007430 comp rx=0 tx=0).stop 2024-04-23T14:43:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 shutdown_connections 2024-04-23T14:43:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8df406c8f0 0x7f8df406edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8e1813ef40 0x7f8e1815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e181e70b0 0x7f8e181e7530 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 --2- 172.21.15.59:0/3634806685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e1813e1d0 0x7f8e1813e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.242+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 >> 172.21.15.59:0/3634806685 conn(0x7f8e18067d90 msgr2=0x7f8e180b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:53.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.243+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 shutdown_connections 2024-04-23T14:43:53.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.243+0000 7f8e1d600640 1 -- 172.21.15.59:0/3634806685 wait complete. 2024-04-23T14:43:53.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:43:53.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.470+0000 7f1f8b000640 1 Processor -- start 2024-04-23T14:43:53.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.470+0000 7f1f8b000640 1 -- start start 2024-04-23T14:43:53.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.471+0000 7f1f8b000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f841416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.471+0000 7f1f8b000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1f8415c250 0x7f1f841555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.471+0000 7f1f8b000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8415ba70 con 0x7f1f8415f060 2024-04-23T14:43:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.471+0000 7f1f8b000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1f84141c20 con 0x7f1f8415c250 2024-04-23T14:43:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.471+0000 7f1f8b000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8415bdc0 con 0x7f1f840d05c0 2024-04-23T14:43:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.471+0000 7f1f89200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1f8415f060 0x7f1f840d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:40778/0 (socket says 172.21.15.59:40778) 2024-04-23T14:43:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f89200640 1 -- 172.21.15.59:0/1627856500 learned_addr learned my addr 172.21.15.59:0/1627856500 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:43:53.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f8a600640 1 --2- 172.21.15.59:0/1627856500 >> v2:172.21.15.59:3300/0 conn(0x7f1f8415c250 0x7f1f841555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f89c00640 1 --2- 172.21.15.59:0/1627856500 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f841416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f83e00640 1 -- 172.21.15.59:0/1627856500 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131668533 0 0) 0x7f1f8415ba70 con 0x7f1f8415f060 2024-04-23T14:43:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f83e00640 1 -- 172.21.15.59:0/1627856500 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f6c003520 con 0x7f1f8415f060 2024-04-23T14:43:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f8a600640 1 -- 172.21.15.59:0/1627856500 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 msgr2=0x7f1f841416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:43:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f8a600640 1 --2- 172.21.15.59:0/1627856500 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f841416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f8a600640 1 -- 172.21.15.59:0/1627856500 >> v1:172.21.15.59:6789/0 conn(0x7f1f8415f060 legacy=0x7f1f840d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f8a600640 1 -- 172.21.15.59:0/1627856500 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8415f800 con 0x7f1f8415c250 2024-04-23T14:43:53.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.472+0000 7f1f89c00640 1 --2- 172.21.15.59:0/1627856500 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f841416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.473+0000 7f1f8a600640 1 --2- 172.21.15.59:0/1627856500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f841555d0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009d90 tx=0x7f1f7c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f24331fa6a19fda server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f83e00640 1 -- 172.21.15.59:0/1627856500 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c007740 con 0x7f1f8415c250 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f83e00640 1 -- 172.21.15.59:0/1627856500 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0078e0 con 0x7f1f8415c250 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f83e00640 1 -- 172.21.15.59:0/1627856500 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c007cb0 con 0x7f1f8415c250 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 -- 172.21.15.59:0/1627856500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 msgr2=0x7f1f841555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 --2- 172.21.15.59:0/1627856500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f841555d0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009d90 tx=0x7f1f7c02a660 comp rx=0 tx=0).stop 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 -- 172.21.15.59:0/1627856500 shutdown_connections 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 --2- 172.21.15.59:0/1627856500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f841555d0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 --2- 172.21.15.59:0/1627856500 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f841416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 -- 172.21.15.59:0/1627856500 >> 172.21.15.59:0/1627856500 conn(0x7f1f8406bdf0 msgr2=0x7f1f8406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:53.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 -- 172.21.15.59:0/1627856500 shutdown_connections 2024-04-23T14:43:53.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.474+0000 7f1f8b000640 1 -- 172.21.15.59:0/1627856500 wait complete. 2024-04-23T14:43:53.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.476+0000 7f1f8b000640 1 Processor -- start 2024-04-23T14:43:53.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.476+0000 7f1f8b000640 1 -- start start 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.477+0000 7f1f8b000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f84097c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.477+0000 7f1f8b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f8415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.477+0000 7f1f8b000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f8415f060 0x7f1f8415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.477+0000 7f1f8b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8415f4c0 con 0x7f1f8415c250 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.477+0000 7f1f8b000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8415fdc0 con 0x7f1f8415f060 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.477+0000 7f1f8b000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8415c6b0 con 0x7f1f840d05c0 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f8415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f8415adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40642/0 (socket says 172.21.15.59:40642) 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 -- 172.21.15.59:0/3513599303 learned_addr learned my addr 172.21.15.59:0/3513599303 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89c00640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f84097c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 -- 172.21.15.59:0/3513599303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 msgr2=0x7f1f84097c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f84097c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 -- 172.21.15.59:0/3513599303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f8415f060 msgr2=0x7f1f8415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f8415f060 0x7f1f8415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 -- 172.21.15.59:0/3513599303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f84091660 con 0x7f1f8415c250 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.478+0000 7f1f89200640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f8415adb0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c3c0 tx=0x7f1f7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.479+0000 7f1f81600640 1 -- 172.21.15.59:0/3513599303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70017070 con 0x7f1f8415c250 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.479+0000 7f1f81600640 1 -- 172.21.15.59:0/3513599303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f70005ce0 con 0x7f1f8415c250 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.479+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f84091940 con 0x7f1f8415c250 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.479+0000 7f1f81600640 1 -- 172.21.15.59:0/3513599303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f700072f0 con 0x7f1f8415c250 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.479+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f84091f00 con 0x7f1f8415c250 2024-04-23T14:43:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.480+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8404fd90 con 0x7f1f8415c250 2024-04-23T14:43:53.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.481+0000 7f1f81600640 1 -- 172.21.15.59:0/3513599303 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f7001a070 con 0x7f1f8415c250 2024-04-23T14:43:53.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.482+0000 7f1f81600640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f6406c8f0 0x7f1f6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:53.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.482+0000 7f1f81600640 1 -- 172.21.15.59:0/3513599303 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f1f7008aed0 con 0x7f1f8415c250 2024-04-23T14:43:53.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.482+0000 7f1f89c00640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f6406c8f0 0x7f1f6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:53.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.483+0000 7f1f89c00640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f6406c8f0 0x7f1f6406edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1f78006fd0 tx=0x7f1f78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:53.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.487+0000 7f1f81600640 1 -- 172.21.15.59:0/3513599303 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f70059300 con 0x7f1f8415c250 2024-04-23T14:43:53.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.694+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f840cfec0 con 0x7f1f8415c250 2024-04-23T14:43:53.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.695+0000 7f1f81600640 1 -- 172.21.15.59:0/3513599303 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7f1f70058c00 con 0x7f1f8415c250 2024-04-23T14:43:53.695 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f6406c8f0 msgr2=0x7f1f6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f6406c8f0 0x7f1f6406edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1f78006fd0 tx=0x7f1f78008040 comp rx=0 tx=0).stop 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 msgr2=0x7f1f8415adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f8415adb0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c3c0 tx=0x7f1f7000c880 comp rx=0 tx=0).stop 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 shutdown_connections 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1f6406c8f0 0x7f1f6406edb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f8415f060 0x7f1f8415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f8415c250 0x7f1f8415adb0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 --2- 172.21.15.59:0/3513599303 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f840d05c0 0x7f1f84097c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:53.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.697+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 >> 172.21.15.59:0/3513599303 conn(0x7f1f8406bdf0 msgr2=0x7f1f8415e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:53.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.698+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 shutdown_connections 2024-04-23T14:43:53.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:43:53.698+0000 7f1f8b000640 1 -- 172.21.15.59:0/3513599303 wait complete. 2024-04-23T14:43:53.716 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:43:53.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:53.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:43:53.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[75344]: fuse finished with error 0 and tester_r 0 2024-04-23T14:43:53.796 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:43:59.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:43:59.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:43:59.898 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:43:59.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-23T14:44:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.185+0000 7ff21da00640 1 Processor -- start 2024-04-23T14:44:00.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.186+0000 7ff21da00640 1 -- start start 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.187+0000 7ff21da00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff21806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.187+0000 7ff21da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff218071d70 0x7ff218072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.187+0000 7ff21da00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2180cfc20 con 0x7ff218145e10 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.187+0000 7ff21da00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff2180cff00 con 0x7ff218071d70 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.187+0000 7ff21da00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180d01e0 con 0x7ff2180d0cc0 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff216a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff218145e10 0x7ff2180cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39518/0 (socket says 172.21.15.59:39518) 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff216a00640 1 -- 172.21.15.59:0/1838292796 learned_addr learned my addr 172.21.15.59:0/1838292796 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff217400640 1 --2- 172.21.15.59:0/1838292796 >> v2:172.21.15.59:3300/0 conn(0x7ff218071d70 0x7ff218072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff216000640 1 --2- 172.21.15.59:0/1838292796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff21806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff217400640 1 -- 172.21.15.59:0/1838292796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 msgr2=0x7ff21806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff217400640 1 --2- 172.21.15.59:0/1838292796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff21806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff217400640 1 -- 172.21.15.59:0/1838292796 >> v1:172.21.15.59:6789/0 conn(0x7ff218145e10 legacy=0x7ff2180cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.188+0000 7ff217400640 1 -- 172.21.15.59:0/1838292796 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2180d04c0 con 0x7ff218071d70 2024-04-23T14:44:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.189+0000 7ff217400640 1 --2- 172.21.15.59:0/1838292796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218071d70 0x7ff218072200 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff210009dc0 tx=0x7ff21002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4783da7b27577b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff215600640 1 -- 172.21.15.59:0/1838292796 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21002ae60 con 0x7ff218071d70 2024-04-23T14:44:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff215600640 1 -- 172.21.15.59:0/1838292796 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff210007740 con 0x7ff218071d70 2024-04-23T14:44:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff215600640 1 -- 172.21.15.59:0/1838292796 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210007b10 con 0x7ff218071d70 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff21da00640 1 -- 172.21.15.59:0/1838292796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218071d70 msgr2=0x7ff218072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff21da00640 1 --2- 172.21.15.59:0/1838292796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218071d70 0x7ff218072200 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff210009dc0 tx=0x7ff21002a680 comp rx=0 tx=0).stop 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff21da00640 1 -- 172.21.15.59:0/1838292796 shutdown_connections 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff21da00640 1 --2- 172.21.15.59:0/1838292796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218071d70 0x7ff218072200 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff21da00640 1 --2- 172.21.15.59:0/1838292796 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff21806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.190+0000 7ff21da00640 1 -- 172.21.15.59:0/1838292796 >> 172.21.15.59:0/1838292796 conn(0x7ff218067d90 msgr2=0x7ff21806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.191+0000 7ff21da00640 1 -- 172.21.15.59:0/1838292796 shutdown_connections 2024-04-23T14:44:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.191+0000 7ff21da00640 1 -- 172.21.15.59:0/1838292796 wait complete. 2024-04-23T14:44:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.192+0000 7ff21da00640 1 Processor -- start 2024-04-23T14:44:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.192+0000 7ff21da00640 1 -- start start 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.193+0000 7ff21da00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff218071d70 0x7ff21813d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.193+0000 7ff21da00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff2180b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.193+0000 7ff21da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218145e10 0x7ff2180b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.193+0000 7ff21da00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181461f0 con 0x7ff218145e10 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.193+0000 7ff21da00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff21815cdd0 con 0x7ff218071d70 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.193+0000 7ff21da00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180af990 con 0x7ff2180d0cc0 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218145e10 0x7ff2180b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218145e10 0x7ff2180b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48522/0 (socket says 172.21.15.59:48522) 2024-04-23T14:44:00.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 -- 172.21.15.59:0/4030304028 learned_addr learned my addr 172.21.15.59:0/4030304028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:00.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff216000640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff2180b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff216a00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff218071d70 0x7ff21813d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 -- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 msgr2=0x7ff2180b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff2180b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 -- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff218071d70 msgr2=0x7ff21813d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff218071d70 0x7ff21813d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.194+0000 7ff217400640 1 -- 172.21.15.59:0/4030304028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21815c6f0 con 0x7ff218145e10 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.195+0000 7ff217400640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218145e10 0x7ff2180b4480 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff210009ee0 tx=0x7ff210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.195+0000 7ff20ea00640 1 -- 172.21.15.59:0/4030304028 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210040070 con 0x7ff218145e10 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.195+0000 7ff20ea00640 1 -- 172.21.15.59:0/4030304028 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21002ae60 con 0x7ff218145e10 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.195+0000 7ff20ea00640 1 -- 172.21.15.59:0/4030304028 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210009450 con 0x7ff218145e10 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.195+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2180b4ab0 con 0x7ff218145e10 2024-04-23T14:44:00.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.195+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2180af620 con 0x7ff218145e10 2024-04-23T14:44:00.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.197+0000 7ff20ea00640 1 -- 172.21.15.59:0/4030304028 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff210039040 con 0x7ff218145e10 2024-04-23T14:44:00.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.198+0000 7ff20ea00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff1f406c8f0 0x7ff1f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.198+0000 7ff20ea00640 1 -- 172.21.15.59:0/4030304028 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7ff2100ad230 con 0x7ff218145e10 2024-04-23T14:44:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.199+0000 7ff216a00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff1f406c8f0 0x7ff1f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.199+0000 7ff220993740 1 Processor -- start 2024-04-23T14:44:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.200+0000 7ff220993740 1 -- start start 2024-04-23T14:44:00.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.200+0000 7ff216a00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff1f406c8f0 0x7ff1f406edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff2180af2d0 tx=0x7ff208009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.200+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 0x5555c5c465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.200+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.200+0000 7ff220993740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76610 0x5555c5c4b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff220993740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555c5bb01f0 con 0x5555c5c76610 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555c5b56350 con 0x5555c5c76d50 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5555c5bac1b0 con 0x5555c5c93c30 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76610 0x5555c5c4b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76610 0x5555c5c4b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48536/0 (socket says 172.21.15.59:48536) 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 -- 172.21.15.59:0/3245239925 learned_addr learned my addr 172.21.15.59:0/3245239925 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff216000640 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 -- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 msgr2=0x5555c5c465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 0x5555c5c465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 -- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 msgr2=0x5555c5c760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff217400640 1 -- 172.21.15.59:0/3245239925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c5baaf50 con 0x5555c5c76610 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.201+0000 7ff216000640 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.202+0000 7ff217400640 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76610 0x5555c5c4b420 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff210083720 tx=0x7ff21007a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2254cc6751db3c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.202+0000 7ff207400640 1 -- 172.21.15.59:0/3245239925 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210043360 con 0x5555c5c76610 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.202+0000 7ff207400640 1 -- 172.21.15.59:0/3245239925 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21007b770 con 0x5555c5c76610 2024-04-23T14:44:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.202+0000 7ff207400640 1 -- 172.21.15.59:0/3245239925 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21007b070 con 0x5555c5c76610 2024-04-23T14:44:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 -- 172.21.15.59:0/3245239925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76610 msgr2=0x5555c5c4b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76610 0x5555c5c4b420 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff210083720 tx=0x7ff21007a9d0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 -- 172.21.15.59:0/3245239925 shutdown_connections 2024-04-23T14:44:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76610 0x5555c5c4b420 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 --2- 172.21.15.59:0/3245239925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 0x5555c5c465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 -- 172.21.15.59:0/3245239925 >> 172.21.15.59:0/3245239925 conn(0x5555c5c48830 msgr2=0x5555c5c910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 -- 172.21.15.59:0/3245239925 shutdown_connections 2024-04-23T14:44:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.203+0000 7ff220993740 1 -- 172.21.15.59:0/3245239925 wait complete. 2024-04-23T14:44:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.204+0000 7ff220993740 1 -- start start 2024-04-23T14:44:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.204+0000 7ff220993740 1 Processor -- start 2024-04-23T14:44:00.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c63920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff220993740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c93c30 0x5555c5c78ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff220993740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555c5bb01f0 con 0x5555c5c93c30 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555c5b69690 con 0x5555c5c76d50 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5555c5c4a9e0 con 0x5555c5c791e0 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff220993740 10 client.0 _ll_register_callbacks cb 0x5555c5ba5650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff216000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c93c30 0x5555c5c78ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff216000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c93c30 0x5555c5c78ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48548/0 (socket says 172.21.15.59:48548) 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff216000640 1 -- 172.21.15.59:0/1805117234 learned_addr learned my addr 172.21.15.59:0/1805117234 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.205+0000 7ff216a00640 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c63920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff217400640 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff216000640 1 -- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 msgr2=0x5555c5c910d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff216000640 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c910d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff216000640 1 -- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 msgr2=0x5555c5c63920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff216000640 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c63920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff216000640 1 -- 172.21.15.59:0/1805117234 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c5b68920 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff216000640 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c93c30 0x5555c5c78ca0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff200010350 tx=0x7ff200010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5935537e1361839 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff1fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1805117234 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20000f040 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1805117234 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7ff200014800 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.206+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff21007b770 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1805117234 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff200014a00 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 10 client.9438 fetch_fsmap learned FSMap version 100 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5555c598c7d0 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1805117234 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21007b770 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1805117234 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7ff200014ba0 con 0x5555c5c93c30 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 10 client.9438 fetch_fsmap finished waiting for FSMap version 100 2024-04-23T14:44:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 10 client.9438 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-23T14:44:00.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 20 client.9438 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:00.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5ca4ad0 0x5555c5ca4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 4 client.9438 mds_command: new command op to 9210 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:44:00.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 --> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c5bbdc10 con 0x5555c5ca4ad0 2024-04-23T14:44:00.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.207+0000 7ff216a00640 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5ca4ad0 0x5555c5ca4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.208+0000 7ff216a00640 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5ca4ad0 0x5555c5ca4eb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52a5fc12d42cb85 server_cookie=84d201a0db8bedad in_seq=0 out_seq=0 2024-04-23T14:44:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.208+0000 7ff1fbe00640 10 client.9438 ms_handle_connect on v2:172.21.15.59:6836/1061483588 2024-04-23T14:44:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff1fbe00640 1 -- 172.21.15.59:0/1805117234 <== mds.0 v2:172.21.15.59:6836/1061483588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555c5bbdc10 con 0x5555c5ca4ad0 2024-04-23T14:44:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff1fbe00640 10 client.9438 handle_command_reply: tid=0 2024-04-23T14:44:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 client.9438 shutdown 2024-04-23T14:44:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c93c30 msgr2=0x5555c5c78ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c93c30 0x5555c5c78ca0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff200010350 tx=0x7ff200010810 comp rx=0 tx=0).stop 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 shutdown_connections 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c910d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5ca4ad0 0x5555c5ca4eb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c93c30 0x5555c5c78ca0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 --2- 172.21.15.59:0/1805117234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c76d50 0x5555c5c63920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 >> 172.21.15.59:0/1805117234 conn(0x5555c5c76610 msgr2=0x5555c5c92200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff1fbe00640 0 client.9438 ms_handle_reset on v2:172.21.15.59:6836/1061483588 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 shutdown_connections 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 1 -- 172.21.15.59:0/1805117234 wait complete. 2024-04-23T14:44:00.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.209+0000 7ff220993740 20 client.9438 trim_cache size 0 max 16384 2024-04-23T14:44:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.226+0000 7ff220993740 1 Processor -- start 2024-04-23T14:44:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.226+0000 7ff220993740 1 -- start start 2024-04-23T14:44:00.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff220993740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c791e0 0x5555c5c89c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 0x5555c5c78830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff220993740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff218200ed0 con 0x5555c5c76d50 2024-04-23T14:44:00.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181fff40 con 0x5555c5c791e0 2024-04-23T14:44:00.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff2181ff2a0 con 0x5555c5c93c30 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff216a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff216a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c894d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48554/0 (socket says 172.21.15.59:48554) 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.227+0000 7ff216a00640 1 -- 172.21.15.59:0/1899918821 learned_addr learned my addr 172.21.15.59:0/1899918821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216000640 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c791e0 0x5555c5c89c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216a00640 1 -- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 msgr2=0x5555c5c78830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216a00640 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 0x5555c5c78830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216a00640 1 -- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c791e0 msgr2=0x5555c5c89c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216a00640 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c791e0 0x5555c5c89c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216a00640 1 -- 172.21.15.59:0/1899918821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c5ca5220 con 0x5555c5c76d50 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216000640 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c791e0 0x5555c5c89c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:00.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.228+0000 7ff216a00640 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c894d0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x5555c5c89b80 tx=0x7ff208012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36041fadb99ecdd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff206a00640 1 -- 172.21.15.59:0/1899918821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208007690 con 0x5555c5c76d50 2024-04-23T14:44:00.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff206a00640 1 -- 172.21.15.59:0/1899918821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff208029070 con 0x5555c5c76d50 2024-04-23T14:44:00.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff206a00640 1 -- 172.21.15.59:0/1899918821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208024450 con 0x5555c5c76d50 2024-04-23T14:44:00.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 -- 172.21.15.59:0/1899918821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 msgr2=0x5555c5c894d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c894d0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x5555c5c89b80 tx=0x7ff208012000 comp rx=0 tx=0).stop 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 -- 172.21.15.59:0/1899918821 shutdown_connections 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c93c30 0x5555c5c78830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c791e0 0x5555c5c89c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 --2- 172.21.15.59:0/1899918821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c894d0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 -- 172.21.15.59:0/1899918821 >> 172.21.15.59:0/1899918821 conn(0x5555c5c76610 msgr2=0x5555c5c8ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.229+0000 7ff220993740 1 -- 172.21.15.59:0/1899918821 shutdown_connections 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.230+0000 7ff220993740 1 -- 172.21.15.59:0/1899918821 wait complete. 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.230+0000 7ff220993740 1 -- start start 2024-04-23T14:44:00.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.230+0000 7ff220993740 1 Processor -- start 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.231+0000 7ff220993740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c92090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.231+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c8a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.231+0000 7ff220993740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c93c30 0x5555c5d0bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.231+0000 7ff220993740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555c5cb4e80 con 0x5555c5c76d50 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.231+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181ff8f0 con 0x5555c5c93c30 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.231+0000 7ff220993740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5555c5b64df0 con 0x5555c5c791e0 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.231+0000 7ff220993740 10 client.0 _ll_register_callbacks cb 0x7ff20000e170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c92090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c92090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48556/0 (socket says 172.21.15.59:48556) 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 -- 172.21.15.59:0/2952579612 learned_addr learned my addr 172.21.15.59:0/2952579612 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff217400640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c93c30 0x5555c5d0bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216000640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c8a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 -- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 msgr2=0x5555c5c8a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c8a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 -- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c93c30 msgr2=0x5555c5d0bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c93c30 0x5555c5d0bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff216a00640 1 -- 172.21.15.59:0/2952579612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c58c23f0 con 0x5555c5c76d50 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.232+0000 7ff217400640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c93c30 0x5555c5d0bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff216a00640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c92090 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff208026040 tx=0x7ff208000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7552630c1281860e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff1fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff1fbe00640 1 -- 172.21.15.59:0/2952579612 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208032070 con 0x5555c5c76d50 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff1fbe00640 1 -- 172.21.15.59:0/2952579612 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7ff20800e4a0 con 0x5555c5c76d50 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff208029070 con 0x5555c5c76d50 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff1fbe00640 1 -- 172.21.15.59:0/2952579612 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff20803d050 con 0x5555c5c76d50 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff220993740 10 client.9444 fetch_fsmap learned FSMap version 100 2024-04-23T14:44:00.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5555c58820f0 con 0x5555c5c76d50 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.233+0000 7ff1fbe00640 1 -- 172.21.15.59:0/2952579612 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff208029070 con 0x5555c5c76d50 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff1fbe00640 1 -- 172.21.15.59:0/2952579612 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7ff208040020 con 0x5555c5c76d50 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff220993740 10 client.9444 fetch_fsmap finished waiting for FSMap version 100 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff220993740 10 client.9444 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff220993740 20 client.9444 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff220993740 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5cee080 0x5555c5c45c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff220993740 4 client.9444 mds_command: new command op to 9210 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 --> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555c5bb7a80 con 0x5555c5cee080 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.234+0000 7ff216000640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5cee080 0x5555c5c45c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.235+0000 7ff216000640 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5cee080 0x5555c5c45c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fadf7522dd9177f server_cookie=7fb5acd5e1f7431a in_seq=0 out_seq=0 2024-04-23T14:44:00.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.235+0000 7ff1fbe00640 10 client.9444 ms_handle_connect on v2:172.21.15.59:6836/1061483588 2024-04-23T14:44:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.247+0000 7ff1fbe00640 1 -- 172.21.15.59:0/2952579612 <== mds.0 v2:172.21.15.59:6836/1061483588 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555c5bb7a80 con 0x5555c5cee080 2024-04-23T14:44:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.247+0000 7ff1fbe00640 10 client.9444 handle_command_reply: tid=0 2024-04-23T14:44:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.247+0000 7ff220993740 1 client.9444 shutdown 2024-04-23T14:44:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 msgr2=0x5555c5c92090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c92090 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff208026040 tx=0x7ff208000bc0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 shutdown_connections 2024-04-23T14:44:00.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5555c5c791e0 0x5555c5c8a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:6836/1061483588,v1:172.21.15.59:6837/1061483588] conn(0x5555c5cee080 0x5555c5c45c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555c5c93c30 0x5555c5d0bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 --2- 172.21.15.59:0/2952579612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555c5c76d50 0x5555c5c92090 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff1fbe00640 0 client.9444 ms_handle_reset on v2:172.21.15.59:6836/1061483588 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 >> 172.21.15.59:0/2952579612 conn(0x5555c5c76610 msgr2=0x5555c5c4a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 shutdown_connections 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 1 -- 172.21.15.59:0/2952579612 wait complete. 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.248+0000 7ff220993740 20 client.9444 trim_cache size 0 max 16384 2024-04-23T14:44:00.249 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:44:00.250 INFO:teuthology.orchestra.run.smithi059.stdout: "message": "", 2024-04-23T14:44:00.250 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0 2024-04-23T14:44:00.250 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff1f406c8f0 msgr2=0x7ff1f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff1f406c8f0 0x7ff1f406edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff2180af2d0 tx=0x7ff208009040 comp rx=0 tx=0).stop 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218145e10 msgr2=0x7ff2180b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218145e10 0x7ff2180b4480 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff210009ee0 tx=0x7ff210004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 shutdown_connections 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff1f406c8f0 0x7ff1f406edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff2180d0cc0 0x7ff2180b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218145e10 0x7ff2180b4480 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 --2- 172.21.15.59:0/4030304028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff218071d70 0x7ff21813d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.249+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 >> 172.21.15.59:0/4030304028 conn(0x7ff218067d90 msgr2=0x7ff21815e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.250+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 shutdown_connections 2024-04-23T14:44:00.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.250+0000 7ff21da00640 1 -- 172.21.15.59:0/4030304028 wait complete. 2024-04-23T14:44:00.271 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:44:00.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.476+0000 7f776fe00640 1 Processor -- start 2024-04-23T14:44:00.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.476+0000 7f776fe00640 1 -- start start 2024-04-23T14:44:00.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.478+0000 7f776fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700b4130 0x7f777006f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.478+0000 7f776fe00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f77700cfb90 0x7f77700c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.478+0000 7f776fe00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f777006ed40 con 0x7f7770162df0 2024-04-23T14:44:00.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.478+0000 7f776fe00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f777006eaf0 con 0x7f77700cfb90 2024-04-23T14:44:00.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.478+0000 7f776fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f77700b3590 con 0x7f77700b4130 2024-04-23T14:44:00.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.479+0000 7f776ea00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7770162df0 0x7f77700b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39522/0 (socket says 172.21.15.59:39522) 2024-04-23T14:44:00.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.479+0000 7f776ea00640 1 -- 172.21.15.59:0/803060373 learned_addr learned my addr 172.21.15.59:0/803060373 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:00.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.479+0000 7f776f400640 1 --2- 172.21.15.59:0/803060373 >> v2:172.21.15.76:3300/0 conn(0x7f77700cfb90 0x7f77700c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.479+0000 7f7767e00640 1 --2- 172.21.15.59:0/803060373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700b4130 0x7f777006f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.479+0000 7f776e000640 1 -- 172.21.15.59:0/803060373 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1585390207 0 0) 0x7f777006ed40 con 0x7f7770162df0 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.479+0000 7f776e000640 1 -- 172.21.15.59:0/803060373 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7754003520 con 0x7f7770162df0 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.480+0000 7f776e000640 1 -- 172.21.15.59:0/803060373 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3718671415 0 0) 0x7f7754003520 con 0x7f7770162df0 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.480+0000 7f7767e00640 1 -- 172.21.15.59:0/803060373 >> v2:172.21.15.76:3300/0 conn(0x7f77700cfb90 msgr2=0x7f77700c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.480+0000 7f7767e00640 1 --2- 172.21.15.59:0/803060373 >> v2:172.21.15.76:3300/0 conn(0x7f77700cfb90 0x7f77700c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.480+0000 7f7767e00640 1 -- 172.21.15.59:0/803060373 >> v1:172.21.15.59:6789/0 conn(0x7f7770162df0 legacy=0x7f77700b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.480+0000 7f7767e00640 1 -- 172.21.15.59:0/803060373 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77700b3220 con 0x7f77700b4130 2024-04-23T14:44:00.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.480+0000 7f776f400640 1 --2- 172.21.15.59:0/803060373 >> v2:172.21.15.76:3300/0 conn(0x7f77700cfb90 0x7f77700c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:44:00.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.481+0000 7f7767e00640 1 --2- 172.21.15.59:0/803060373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700b4130 0x7f777006f680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f77580097a0 tx=0x7f775802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9938b1784d6fd05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776e000640 1 -- 172.21.15.59:0/803060373 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758037070 con 0x7f77700b4130 2024-04-23T14:44:00.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776e000640 1 -- 172.21.15.59:0/803060373 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775802aba0 con 0x7f77700b4130 2024-04-23T14:44:00.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776e000640 1 -- 172.21.15.59:0/803060373 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760001670 con 0x7f77700b4130 2024-04-23T14:44:00.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776fe00640 1 -- 172.21.15.59:0/803060373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700b4130 msgr2=0x7f777006f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776fe00640 1 --2- 172.21.15.59:0/803060373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700b4130 0x7f777006f680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f77580097a0 tx=0x7f775802a110 comp rx=0 tx=0).stop 2024-04-23T14:44:00.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776fe00640 1 -- 172.21.15.59:0/803060373 shutdown_connections 2024-04-23T14:44:00.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776fe00640 1 --2- 172.21.15.59:0/803060373 >> v2:172.21.15.76:3300/0 conn(0x7f77700cfb90 0x7f77700c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776fe00640 1 --2- 172.21.15.59:0/803060373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700b4130 0x7f777006f680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776fe00640 1 -- 172.21.15.59:0/803060373 >> 172.21.15.59:0/803060373 conn(0x7f77700b21a0 msgr2=0x7f77700b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.482+0000 7f776fe00640 1 -- 172.21.15.59:0/803060373 shutdown_connections 2024-04-23T14:44:00.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.483+0000 7f776fe00640 1 -- 172.21.15.59:0/803060373 wait complete. 2024-04-23T14:44:00.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.484+0000 7f776fe00640 1 Processor -- start 2024-04-23T14:44:00.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.484+0000 7f776fe00640 1 -- start start 2024-04-23T14:44:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.485+0000 7f776fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77700b4130 0x7f77700c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.485+0000 7f776fe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700cfb90 0x7f77700c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.485+0000 7f776fe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7770162df0 0x7f77700c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.485+0000 7f776fe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77700c3860 con 0x7f77700b4130 2024-04-23T14:44:00.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.485+0000 7f776fe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f77700c36e0 con 0x7f7770162df0 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.485+0000 7f776fe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f77700c39e0 con 0x7f77700cfb90 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77700b4130 0x7f77700c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77700b4130 0x7f77700c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48558/0 (socket says 172.21.15.59:48558) 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 -- 172.21.15.59:0/151878203 learned_addr learned my addr 172.21.15.59:0/151878203 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f7767e00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700cfb90 0x7f77700c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776f400640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7770162df0 0x7f77700c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 -- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700cfb90 msgr2=0x7f77700c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700cfb90 0x7f77700c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 -- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7770162df0 msgr2=0x7f77700c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7770162df0 0x7f77700c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.486+0000 7f776ea00640 1 -- 172.21.15.59:0/151878203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77700c9ce0 con 0x7f77700b4130 2024-04-23T14:44:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.487+0000 7f776ea00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77700b4130 0x7f77700c6760 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f7760001eb0 tx=0x7f776000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.487+0000 7f7766a00640 1 -- 172.21.15.59:0/151878203 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760017070 con 0x7f77700b4130 2024-04-23T14:44:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.487+0000 7f7766a00640 1 -- 172.21.15.59:0/151878203 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7760005ce0 con 0x7f77700b4130 2024-04-23T14:44:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.487+0000 7f7766a00640 1 -- 172.21.15.59:0/151878203 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760007290 con 0x7f77700b4130 2024-04-23T14:44:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.487+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77700c9fc0 con 0x7f77700b4130 2024-04-23T14:44:00.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.487+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77700afbc0 con 0x7f77700b4130 2024-04-23T14:44:00.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.488+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777004fd90 con 0x7f77700b4130 2024-04-23T14:44:00.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.489+0000 7f7766a00640 1 -- 172.21.15.59:0/151878203 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f776001a070 con 0x7f77700b4130 2024-04-23T14:44:00.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.491+0000 7f7766a00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:00.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.491+0000 7f7766a00640 1 -- 172.21.15.59:0/151878203 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f776008b100 con 0x7f77700b4130 2024-04-23T14:44:00.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.491+0000 7f7767e00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:00.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.492+0000 7f7767e00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f774c06c8f0 0x7f774c06edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7758034000 tx=0x7f775802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:00.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.497+0000 7f7766a00640 1 -- 172.21.15.59:0/151878203 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7760059400 con 0x7f77700b4130 2024-04-23T14:44:00.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.743+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f777006cbe0 con 0x7f77700b4130 2024-04-23T14:44:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.744+0000 7f7766a00640 1 -- 172.21.15.59:0/151878203 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7f7760058d00 con 0x7f77700b4130 2024-04-23T14:44:00.744 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f774c06c8f0 msgr2=0x7f774c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f774c06c8f0 0x7f774c06edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7758034000 tx=0x7f775802e000 comp rx=0 tx=0).stop 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77700b4130 msgr2=0x7f77700c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77700b4130 0x7f77700c6760 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f7760001eb0 tx=0x7f776000c9b0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 shutdown_connections 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f77700cfb90 0x7f77700c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7770162df0 0x7f77700c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 --2- 172.21.15.59:0/151878203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77700b4130 0x7f77700c6760 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.746+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 >> 172.21.15.59:0/151878203 conn(0x7f77700b21a0 msgr2=0x7f77700ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.747+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 shutdown_connections 2024-04-23T14:44:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:00.747+0000 7f776fe00640 1 -- 172.21.15.59:0/151878203 wait complete. 2024-04-23T14:44:00.766 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:44:00.766 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:44:00.766 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:44:00.766 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:44:00.766 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:00.766 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:44:00.767 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:44:00.767 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:44:00.767 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:00.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:00.767 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:44:00.819 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:44:00.820 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41708sec preferred_lft 41708sec 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:44:00.821 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:44:00.822 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:00.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:00.823 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:44:00.904 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:44:00.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:00.908 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:44:00.962 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:44:00.962 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:44:00.963 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:44:00.963 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:44:00.963 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:44:00.963 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:44:00.963 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:44:00.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:44:00.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:44:00.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:44:00.963 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:44:01.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:44:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:01.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:01.185 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:44:01.235 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:44:01.235 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:44:01.235 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:01.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:01.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:44:01.326 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:44:01.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:44:01.327 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:01.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:44:01.386 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:01.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:44:01.525 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:44:01.583 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:44:01.584 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:44:01.585 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:44:01.585 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:01.585 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:01.585 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:01.585 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:01.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:01.586 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:44:01.639 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:44:01.639 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:44:01.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:44:01.716 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:44:01.758 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:44:01.758 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:44:01.758 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:44:01.758 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:44:01.758 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:44:01.758 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:44:01.759 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:01.760 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:01.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:01.761 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:44:01.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:44:01.818+0000 7fd020320480 -1 init, newargv = 0x5587f463b600 newargc=15 2024-04-23T14:44:01.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[76470]: starting ceph client 2024-04-23T14:44:01.826 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[76470]: starting fuse 2024-04-23T14:44:01.846 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:44:01.846 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:44:01.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:44:01.846 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:44:01.847 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:44:01.848 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:02.043 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.76470.asok 2024-04-23T14:44:02.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:02.061 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76470.asok 2024-04-23T14:44:02.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:02.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76470.asok status 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "76470", 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:44:02.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9456, 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9456 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 577163794 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:02.293 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 577163794 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9456 192.168.0.1:0/577163794", 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/577163794", 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 100, 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 112, 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:44:02.294 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:44:02.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:02.306 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:44:02.307 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:02.364 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:44:02.364 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:02.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:02.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:02.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:02.456 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-23T14:44:02.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:02.521 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-23T14:44:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:8+0 records in 2024-04-23T14:44:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:8+0 records out 2024-04-23T14:44:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.168377 s, 49.8 MB/s 2024-04-23T14:44:02.751 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:02.751 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:02.751 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:02.752 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:02.752 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-23T14:44:02.752 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:02.889 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627779 2024-04-23T14:44:02.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:02.903 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:02.903 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:02.903 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:02.903 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:02.903 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-23T14:44:02.903 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:03.083 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627780 2024-04-23T14:44:03.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:03.096 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:44:03.097 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:03.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:03.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.409+0000 7fb1faa00640 1 Processor -- start 2024-04-23T14:44:03.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.410+0000 7fb1faa00640 1 -- start start 2024-04-23T14:44:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1faa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1f406f7c0 0x7fb1f40b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1faa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406df00 0x7fb1f406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1faa00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f406ec60 con 0x7fb1f415b840 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1faa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb1f40b46e0 con 0x7fb1f406f7c0 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1faa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f40b49c0 con 0x7fb1f406df00 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1f3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1f406f7c0 0x7fb1f40b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1f3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1f406f7c0 0x7fb1f40b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48570/0 (socket says 172.21.15.59:48570) 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.411+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2679460794 learned_addr learned my addr 172.21.15.59:0/2679460794 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.412+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2679460794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406df00 msgr2=0x7fb1f406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.412+0000 7fb1f3e00640 1 --2- 172.21.15.59:0/2679460794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406df00 0x7fb1f406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.412+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2679460794 >> v1:172.21.15.76:6789/0 conn(0x7fb1f415b840 legacy=0x7fb1f406fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:03.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.412+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2679460794 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f40b4ca0 con 0x7fb1f406f7c0 2024-04-23T14:44:03.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.413+0000 7fb1f3e00640 1 --2- 172.21.15.59:0/2679460794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406f7c0 0x7fb1f40b54a0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009e50 tx=0x7fb1e402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7f308f7e9f3de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:03.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.413+0000 7fb1f3400640 1 -- 172.21.15.59:0/2679460794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40333f0 con 0x7fb1f406f7c0 2024-04-23T14:44:03.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.413+0000 7fb1f3400640 1 -- 172.21.15.59:0/2679460794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e4033590 con 0x7fb1f406f7c0 2024-04-23T14:44:03.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.413+0000 7fb1f3400640 1 -- 172.21.15.59:0/2679460794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4033960 con 0x7fb1f406f7c0 2024-04-23T14:44:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 -- 172.21.15.59:0/2679460794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406f7c0 msgr2=0x7fb1f40b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2679460794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406f7c0 0x7fb1f40b54a0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009e50 tx=0x7fb1e402c8e0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 -- 172.21.15.59:0/2679460794 shutdown_connections 2024-04-23T14:44:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2679460794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406df00 0x7fb1f406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2679460794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406f7c0 0x7fb1f40b54a0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 -- 172.21.15.59:0/2679460794 >> 172.21.15.59:0/2679460794 conn(0x7fb1f4067ea0 msgr2=0x7fb1f406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:03.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 -- 172.21.15.59:0/2679460794 shutdown_connections 2024-04-23T14:44:03.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.414+0000 7fb1faa00640 1 -- 172.21.15.59:0/2679460794 wait complete. 2024-04-23T14:44:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.416+0000 7fb1faa00640 1 Processor -- start 2024-04-23T14:44:03.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.416+0000 7fb1faa00640 1 -- start start 2024-04-23T14:44:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1faa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406df00 0x7fb1f4153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1faa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406f7c0 0x7fb1f413fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1faa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f4151330 0x7fb1f415f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1faa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f40ce1c0 con 0x7fb1f406df00 2024-04-23T14:44:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1faa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f406e8d0 con 0x7fb1f4151330 2024-04-23T14:44:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1faa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f40ce340 con 0x7fb1f406f7c0 2024-04-23T14:44:03.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406df00 0x7fb1f4153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406df00 0x7fb1f4153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48582/0 (socket says 172.21.15.59:48582) 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.417+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2671956680 learned_addr learned my addr 172.21.15.59:0/2671956680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f8a00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f4151330 0x7fb1f415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2671956680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406f7c0 msgr2=0x7fb1f413fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f3e00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406f7c0 0x7fb1f413fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2671956680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f4151330 msgr2=0x7fb1f415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f3e00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f4151330 0x7fb1f415f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f3e00640 1 -- 172.21.15.59:0/2671956680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f415c9f0 con 0x7fb1f406df00 2024-04-23T14:44:03.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f3e00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406df00 0x7fb1f4153a70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000f80 tx=0x7fb1e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.418+0000 7fb1f0c00640 1 -- 172.21.15.59:0/2671956680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40333f0 con 0x7fb1f406df00 2024-04-23T14:44:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.419+0000 7fb1f0c00640 1 -- 172.21.15.59:0/2671956680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e4008c40 con 0x7fb1f406df00 2024-04-23T14:44:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.419+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f415ccd0 con 0x7fb1f406df00 2024-04-23T14:44:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.419+0000 7fb1f0c00640 1 -- 172.21.15.59:0/2671956680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4009510 con 0x7fb1f406df00 2024-04-23T14:44:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.419+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f415d290 con 0x7fb1f406df00 2024-04-23T14:44:03.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.420+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f4068a80 con 0x7fb1f406df00 2024-04-23T14:44:03.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.421+0000 7fb1f0c00640 1 -- 172.21.15.59:0/2671956680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1e4045020 con 0x7fb1f406df00 2024-04-23T14:44:03.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.422+0000 7fb1f0c00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb1d006c920 0x7fb1d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.422+0000 7fb1f0c00640 1 -- 172.21.15.59:0/2671956680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e4030070 con 0x7fb1f406df00 2024-04-23T14:44:03.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.422+0000 7fb1eb400640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb1d006c920 0x7fb1d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.423+0000 7fb1eb400640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb1d006c920 0x7fb1d006ede0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40cfce0 tx=0x7fb1dc006ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:03.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.427+0000 7fb1f0c00640 1 -- 172.21.15.59:0/2671956680 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1e407cc10 con 0x7fb1f406df00 2024-04-23T14:44:03.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.636+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1f406f370 con 0x7fb1f406df00 2024-04-23T14:44:03.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.638+0000 7fb1f0c00640 1 -- 172.21.15.59:0/2671956680 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16567 (secure 0 0 0) 0x7fb1e407c510 con 0x7fb1f406df00 2024-04-23T14:44:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:03.640 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":112,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:39.410742+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:03.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb1d006c920 msgr2=0x7fb1d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb1d006c920 0x7fb1d006ede0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40cfce0 tx=0x7fb1dc006ba0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406df00 msgr2=0x7fb1f4153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406df00 0x7fb1f4153a70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000f80 tx=0x7fb1e4004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 shutdown_connections 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb1d006c920 0x7fb1d006ede0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1f406f7c0 0x7fb1f413fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f4151330 0x7fb1f415f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 --2- 172.21.15.59:0/2671956680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1f406df00 0x7fb1f4153a70 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 >> 172.21.15.59:0/2671956680 conn(0x7fb1f4067ea0 msgr2=0x7fb1f406fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 shutdown_connections 2024-04-23T14:44:03.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.641+0000 7fb1faa00640 1 -- 172.21.15.59:0/2671956680 wait complete. 2024-04-23T14:44:03.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:44:03.661 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:44:03.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.882+0000 7f5a2ca00640 1 Processor -- start 2024-04-23T14:44:03.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.882+0000 7f5a2ca00640 1 -- start start 2024-04-23T14:44:03.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.883+0000 7f5a2ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28163770 0x7f5a280d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.883+0000 7f5a2ca00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a28160180 0x7f5a280d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.884+0000 7f5a2ca00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a280cff50 con 0x7f5a2815cb20 2024-04-23T14:44:03.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.884+0000 7f5a2ca00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5a2815cf00 con 0x7f5a28160180 2024-04-23T14:44:03.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.884+0000 7f5a2ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280d01c0 con 0x7f5a28163770 2024-04-23T14:44:03.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.884+0000 7f5a26800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a28160180 0x7f5a280d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.884+0000 7f5a26800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a28160180 0x7f5a280d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:40308/0 (socket says 172.21.15.59:40308) 2024-04-23T14:44:03.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.884+0000 7f5a25400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28163770 0x7f5a280d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.884+0000 7f5a26800640 1 -- 172.21.15.59:0/3009193191 learned_addr learned my addr 172.21.15.59:0/3009193191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:03.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.885+0000 7f5a26800640 1 -- 172.21.15.59:0/3009193191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28163770 msgr2=0x7f5a280d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:03.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.885+0000 7f5a26800640 1 --2- 172.21.15.59:0/3009193191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28163770 0x7f5a280d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.885+0000 7f5a26800640 1 -- 172.21.15.59:0/3009193191 >> v1:172.21.15.76:6789/0 conn(0x7f5a2815cb20 legacy=0x7f5a280cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:03.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.885+0000 7f5a26800640 1 -- 172.21.15.59:0/3009193191 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2815f6a0 con 0x7f5a28160180 2024-04-23T14:44:03.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.886+0000 7f5a26800640 1 --2- 172.21.15.59:0/3009193191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a28160180 0x7f5a280d1780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009870 tx=0x7f5a2002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76611e796aacbd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:03.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.887+0000 7f5a24a00640 1 -- 172.21.15.59:0/3009193191 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20037070 con 0x7f5a28160180 2024-04-23T14:44:03.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.887+0000 7f5a24a00640 1 -- 172.21.15.59:0/3009193191 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a20032440 con 0x7f5a28160180 2024-04-23T14:44:03.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.887+0000 7f5a24a00640 1 -- 172.21.15.59:0/3009193191 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20032810 con 0x7f5a28160180 2024-04-23T14:44:03.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.887+0000 7f5a2ca00640 1 -- 172.21.15.59:0/3009193191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a28160180 msgr2=0x7f5a280d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.887+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/3009193191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a28160180 0x7f5a280d1780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009870 tx=0x7f5a2002a1c0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.888+0000 7f5a2ca00640 1 -- 172.21.15.59:0/3009193191 shutdown_connections 2024-04-23T14:44:03.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.888+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/3009193191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a28160180 0x7f5a280d1780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.888+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/3009193191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28163770 0x7f5a280d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.888+0000 7f5a2ca00640 1 -- 172.21.15.59:0/3009193191 >> 172.21.15.59:0/3009193191 conn(0x7f5a2806cc70 msgr2=0x7f5a2806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:03.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.888+0000 7f5a2ca00640 1 -- 172.21.15.59:0/3009193191 shutdown_connections 2024-04-23T14:44:03.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.888+0000 7f5a2ca00640 1 -- 172.21.15.59:0/3009193191 wait complete. 2024-04-23T14:44:03.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.890+0000 7f5a2ca00640 1 Processor -- start 2024-04-23T14:44:03.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.890+0000 7f5a2ca00640 1 -- start start 2024-04-23T14:44:03.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a2ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a2815cb20 0x7f5a28152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a2ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28160180 0x7f5a28156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a2ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a28163770 0x7f5a28159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a2ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280d2d60 con 0x7f5a28163770 2024-04-23T14:44:03.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a2ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280d2be0 con 0x7f5a2815cb20 2024-04-23T14:44:03.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a2ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280d2ee0 con 0x7f5a28160180 2024-04-23T14:44:03.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a26800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a28163770 0x7f5a28159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a26800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a28163770 0x7f5a28159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48592/0 (socket says 172.21.15.59:48592) 2024-04-23T14:44:03.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a26800640 1 -- 172.21.15.59:0/82737360 learned_addr learned my addr 172.21.15.59:0/82737360 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a25400640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28160180 0x7f5a28156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.891+0000 7f5a25e00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a2815cb20 0x7f5a28152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a26800640 1 -- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28160180 msgr2=0x7f5a28156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a26800640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28160180 0x7f5a28156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a26800640 1 -- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a2815cb20 msgr2=0x7f5a28152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a26800640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a2815cb20 0x7f5a28152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a26800640 1 -- 172.21.15.59:0/82737360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a28157080 con 0x7f5a28163770 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a26800640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a28163770 0x7f5a28159600 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009990 tx=0x7f5a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:03.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a1e000640 1 -- 172.21.15.59:0/82737360 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20037040 con 0x7f5a28163770 2024-04-23T14:44:03.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a1e000640 1 -- 172.21.15.59:0/82737360 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a20042070 con 0x7f5a28163770 2024-04-23T14:44:03.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.892+0000 7f5a1e000640 1 -- 172.21.15.59:0/82737360 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2003b930 con 0x7f5a28163770 2024-04-23T14:44:03.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.893+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a28157360 con 0x7f5a28163770 2024-04-23T14:44:03.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.893+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a28156d40 con 0x7f5a28163770 2024-04-23T14:44:03.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.894+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28068a80 con 0x7f5a28163770 2024-04-23T14:44:03.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.895+0000 7f5a1e000640 1 -- 172.21.15.59:0/82737360 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a2002ad10 con 0x7f5a28163770 2024-04-23T14:44:03.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.896+0000 7f5a1e000640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:03.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.896+0000 7f5a1e000640 1 -- 172.21.15.59:0/82737360 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f5a200ac100 con 0x7f5a28163770 2024-04-23T14:44:03.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.896+0000 7f5a25e00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:03.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.897+0000 7f5a25e00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a0406c920 0x7f5a0406ede0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5a18001d40 tx=0x7f5a18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:03.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:03.901+0000 7f5a1e000640 1 -- 172.21.15.59:0/82737360 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a2007a400 con 0x7f5a28163770 2024-04-23T14:44:04.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.103+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5a280d29c0 con 0x7f5a28163770 2024-04-23T14:44:04.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.103+0000 7f5a1e000640 1 -- 172.21.15.59:0/82737360 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f5a20079d00 con 0x7f5a28163770 2024-04-23T14:44:04.105 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:44:04.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a0406c920 msgr2=0x7f5a0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a0406c920 0x7f5a0406ede0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5a18001d40 tx=0x7f5a18009040 comp rx=0 tx=0).stop 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a28163770 msgr2=0x7f5a28159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a28163770 0x7f5a28159600 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009990 tx=0x7f5a200046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 shutdown_connections 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5a0406c920 0x7f5a0406ede0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a28160180 0x7f5a28156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a28163770 0x7f5a28159600 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 --2- 172.21.15.59:0/82737360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a2815cb20 0x7f5a28152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 >> 172.21.15.59:0/82737360 conn(0x7f5a2806cc70 msgr2=0x7f5a2815aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 shutdown_connections 2024-04-23T14:44:04.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.107+0000 7f5a2ca00640 1 -- 172.21.15.59:0/82737360 wait complete. 2024-04-23T14:44:04.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:44:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.342+0000 7fda1b200640 1 Processor -- start 2024-04-23T14:44:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.342+0000 7fda1b200640 1 -- start start 2024-04-23T14:44:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.343+0000 7fda1b200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda1c15d810 0x7fda1c160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.343+0000 7fda1b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1c15cab0 con 0x7fda1c1567e0 2024-04-23T14:44:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.343+0000 7fda1b200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1c160e40 con 0x7fda1c0d0d10 2024-04-23T14:44:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.343+0000 7fda1b200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fda1c15cd30 con 0x7fda1c15d810 2024-04-23T14:44:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.344+0000 7fda19e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fda1c1567e0 0x7fda1c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39524/0 (socket says 172.21.15.59:39524) 2024-04-23T14:44:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.344+0000 7fda19e00640 1 -- 172.21.15.59:0/3620969281 learned_addr learned my addr 172.21.15.59:0/3620969281 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.344+0000 7fda1a800640 1 --2- 172.21.15.59:0/3620969281 >> v2:172.21.15.76:3300/0 conn(0x7fda1c15d810 0x7fda1c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.344+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3572349490 0 0) 0x7fda0c001110 con 0x7fda1c1567e0 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.344+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda00003520 con 0x7fda1c1567e0 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4196473920 0 0) 0x7fda00003520 con 0x7fda1c1567e0 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 >> v2:172.21.15.76:3300/0 conn(0x7fda1c15d810 msgr2=0x7fda1c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 --2- 172.21.15.59:0/3620969281 >> v2:172.21.15.76:3300/0 conn(0x7fda1c15d810 0x7fda1c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 >> v1:172.21.15.76:6789/0 conn(0x7fda1c0d0d10 legacy=0x7fda1c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c15d010 con 0x7fda1c1567e0 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fda0c001840 con 0x7fda1c1567e0 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 >> v1:172.21.15.59:6789/0 conn(0x7fda1c1567e0 legacy=0x7fda1c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda00003a00 0x7fda00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda00004800 0x7fda00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda0000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c15cd30 con 0x7fda000051c0 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda000104d0 con 0x7fda00003a00 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda000106d0 con 0x7fda00004800 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda1a800640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda0000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.345+0000 7fda19e00640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda00003a00 0x7fda00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.346+0000 7fda1a800640 1 -- 172.21.15.59:0/3620969281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda00004800 msgr2=0x7fda00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.346+0000 7fda1a800640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda00004800 0x7fda00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.346+0000 7fda1a800640 1 -- 172.21.15.59:0/3620969281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda00003a00 msgr2=0x7fda00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.346+0000 7fda1a800640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda00003a00 0x7fda00003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.346+0000 7fda1a800640 1 -- 172.21.15.59:0/3620969281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda00003520 con 0x7fda000051c0 2024-04-23T14:44:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.347+0000 7fda1a800640 1 -- 172.21.15.59:0/3620969281 reap_dead start 2024-04-23T14:44:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.347+0000 7fda1a800640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda0000fdd0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fda14009a90 tx=0x7fda00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38f16275fb7d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.347+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14009690 con 0x7fda000051c0 2024-04-23T14:44:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.347+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda14004020 con 0x7fda000051c0 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.347+0000 7fda18a00640 1 -- 172.21.15.59:0/3620969281 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140043d0 con 0x7fda000051c0 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.348+0000 7fda1b200640 1 -- 172.21.15.59:0/3620969281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 msgr2=0x7fda0000fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.348+0000 7fda1b200640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda0000fdd0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fda14009a90 tx=0x7fda00005830 comp rx=0 tx=0).stop 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.348+0000 7fda1b200640 1 -- 172.21.15.59:0/3620969281 shutdown_connections 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.348+0000 7fda1b200640 1 --2- 172.21.15.59:0/3620969281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda0000fdd0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.348+0000 7fda1b200640 1 -- 172.21.15.59:0/3620969281 >> 172.21.15.59:0/3620969281 conn(0x7fda1c06bdf0 msgr2=0x7fda1c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.348+0000 7fda1b200640 1 -- 172.21.15.59:0/3620969281 shutdown_connections 2024-04-23T14:44:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.348+0000 7fda1b200640 1 -- 172.21.15.59:0/3620969281 wait complete. 2024-04-23T14:44:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.350+0000 7fda1b200640 1 Processor -- start 2024-04-23T14:44:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.350+0000 7fda1b200640 1 -- start start 2024-04-23T14:44:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda1b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda1c1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda1b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c0998e0 0x7fda1c155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda1b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c14de20 0x7fda1c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda1b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c156d40 con 0x7fda000051c0 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda1b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c156bc0 con 0x7fda1c14de20 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda1b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c156ec0 con 0x7fda1c0998e0 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda1c1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda1c1575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48612/0 (socket says 172.21.15.59:48612) 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 -- 172.21.15.59:0/1491837305 learned_addr learned my addr 172.21.15.59:0/1491837305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda1a800640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c14de20 0x7fda1c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 -- 172.21.15.59:0/1491837305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c0998e0 msgr2=0x7fda1c155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c0998e0 0x7fda1c155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 -- 172.21.15.59:0/1491837305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c14de20 msgr2=0x7fda1c14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c14de20 0x7fda1c14e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.351+0000 7fda19e00640 1 -- 172.21.15.59:0/1491837305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c159860 con 0x7fda000051c0 2024-04-23T14:44:04.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.352+0000 7fda19e00640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda1c1575d0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fda0c001050 tx=0x7fda0c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:04.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.352+0000 7fda12000640 1 -- 172.21.15.59:0/1491837305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00ee00 con 0x7fda000051c0 2024-04-23T14:44:04.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.352+0000 7fda12000640 1 -- 172.21.15.59:0/1491837305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c018070 con 0x7fda000051c0 2024-04-23T14:44:04.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.352+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c159ae0 con 0x7fda000051c0 2024-04-23T14:44:04.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.352+0000 7fda12000640 1 -- 172.21.15.59:0/1491837305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00cab0 con 0x7fda000051c0 2024-04-23T14:44:04.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.352+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c15a0a0 con 0x7fda000051c0 2024-04-23T14:44:04.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.354+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c15a2a0 con 0x7fda000051c0 2024-04-23T14:44:04.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.354+0000 7fda12000640 1 -- 172.21.15.59:0/1491837305 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda0c00cc50 con 0x7fda000051c0 2024-04-23T14:44:04.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.355+0000 7fda12000640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.356+0000 7fda12000640 1 -- 172.21.15.59:0/1491837305 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fda0c08b950 con 0x7fda000051c0 2024-04-23T14:44:04.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.356+0000 7fda19400640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.357+0000 7fda19400640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fda1c14eb90 tx=0x7fda08007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:04.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.360+0000 7fda12000640 1 -- 172.21.15.59:0/1491837305 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda1c159860 con 0x7fda000051c0 2024-04-23T14:44:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.568+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fda1c04fd90 con 0x7fda000051c0 2024-04-23T14:44:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.569+0000 7fda12000640 1 -- 172.21.15.59:0/1491837305 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7fda1c15cab0 con 0x7fda000051c0 2024-04-23T14:44:04.570 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:44:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 msgr2=0x7fd9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fda1c14eb90 tx=0x7fda08007d30 comp rx=0 tx=0).stop 2024-04-23T14:44:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 msgr2=0x7fda1c1575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda1c1575d0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fda0c001050 tx=0x7fda0c00ec10 comp rx=0 tx=0).stop 2024-04-23T14:44:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 shutdown_connections 2024-04-23T14:44:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda1c0998e0 0x7fda1c155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c14de20 0x7fda1c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 --2- 172.21.15.59:0/1491837305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda000051c0 0x7fda1c1575d0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 >> 172.21.15.59:0/1491837305 conn(0x7fda1c06bdf0 msgr2=0x7fda1c15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 shutdown_connections 2024-04-23T14:44:04.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.572+0000 7fda1b200640 1 -- 172.21.15.59:0/1491837305 wait complete. 2024-04-23T14:44:04.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:44:04.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.802+0000 7f4eea400640 1 Processor -- start 2024-04-23T14:44:04.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.802+0000 7f4eea400640 1 -- start start 2024-04-23T14:44:04.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.803+0000 7f4eea400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4ee40d15b0 0x7f4ee40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4eea400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ee40b4f20 0x7f4ee40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4eea400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee40b5da0 con 0x7f4ee415ed20 2024-04-23T14:44:04.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4eea400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee415f100 con 0x7f4ee40b4f20 2024-04-23T14:44:04.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4eea400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee40d0a20 con 0x7f4ee40d15b0 2024-04-23T14:44:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4ee9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ee40b4f20 0x7f4ee40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4ee9a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ee40b4f20 0x7f4ee40cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48616/0 (socket says 172.21.15.59:48616) 2024-04-23T14:44:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4ee9a00640 1 -- 172.21.15.59:0/2742802403 learned_addr learned my addr 172.21.15.59:0/2742802403 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.804+0000 7f4ee3e00640 1 --2- 172.21.15.59:0/2742802403 >> v2:172.21.15.76:3300/0 conn(0x7f4ee40d15b0 0x7f4ee40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.805+0000 7f4ee9a00640 1 -- 172.21.15.59:0/2742802403 >> v2:172.21.15.76:3300/0 conn(0x7f4ee40d15b0 msgr2=0x7f4ee40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.805+0000 7f4ee9a00640 1 --2- 172.21.15.59:0/2742802403 >> v2:172.21.15.76:3300/0 conn(0x7f4ee40d15b0 0x7f4ee40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.805+0000 7f4ee9a00640 1 -- 172.21.15.59:0/2742802403 >> v1:172.21.15.76:6789/0 conn(0x7f4ee415ed20 legacy=0x7f4ee40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.805+0000 7f4ee9a00640 1 -- 172.21.15.59:0/2742802403 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee40d2ed0 con 0x7f4ee40b4f20 2024-04-23T14:44:04.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.806+0000 7f4ee9a00640 1 --2- 172.21.15.59:0/2742802403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee40cf7c0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40d4520 tx=0x7f4edc02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfddad63e5d11ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:04.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.807+0000 7f4ee3400640 1 -- 172.21.15.59:0/2742802403 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc037070 con 0x7f4ee40b4f20 2024-04-23T14:44:04.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.807+0000 7f4ee3400640 1 -- 172.21.15.59:0/2742802403 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc032440 con 0x7f4ee40b4f20 2024-04-23T14:44:04.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.807+0000 7f4ee3400640 1 -- 172.21.15.59:0/2742802403 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc032810 con 0x7f4ee40b4f20 2024-04-23T14:44:04.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 -- 172.21.15.59:0/2742802403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 msgr2=0x7f4ee40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 --2- 172.21.15.59:0/2742802403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee40cf7c0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40d4520 tx=0x7f4edc02a160 comp rx=0 tx=0).stop 2024-04-23T14:44:04.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 -- 172.21.15.59:0/2742802403 shutdown_connections 2024-04-23T14:44:04.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 --2- 172.21.15.59:0/2742802403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee40cf7c0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 --2- 172.21.15.59:0/2742802403 >> v2:172.21.15.76:3300/0 conn(0x7f4ee40d15b0 0x7f4ee40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 -- 172.21.15.59:0/2742802403 >> 172.21.15.59:0/2742802403 conn(0x7f4ee408bf90 msgr2=0x7f4ee40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:04.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 -- 172.21.15.59:0/2742802403 shutdown_connections 2024-04-23T14:44:04.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.808+0000 7f4eea400640 1 -- 172.21.15.59:0/2742802403 wait complete. 2024-04-23T14:44:04.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.810+0000 7f4eea400640 1 Processor -- start 2024-04-23T14:44:04.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.810+0000 7f4eea400640 1 -- start start 2024-04-23T14:44:04.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4eea400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee41f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4eea400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee40d15b0 0x7f4ee40a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4eea400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee415ed20 0x7f4ee40a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4eea400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40b3600 con 0x7f4ee40b4f20 2024-04-23T14:44:04.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4eea400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40d3490 con 0x7f4ee40d15b0 2024-04-23T14:44:04.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4eea400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee40b3780 con 0x7f4ee415ed20 2024-04-23T14:44:04.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee41f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee41f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48624/0 (socket says 172.21.15.59:48624) 2024-04-23T14:44:04.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.811+0000 7f4ee9000640 1 -- 172.21.15.59:0/3989276994 learned_addr learned my addr 172.21.15.59:0/3989276994 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee9000640 1 -- 172.21.15.59:0/3989276994 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee415ed20 msgr2=0x7f4ee40a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee9000640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee415ed20 0x7f4ee40a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee9000640 1 -- 172.21.15.59:0/3989276994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee40d15b0 msgr2=0x7f4ee40a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee9000640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee40d15b0 0x7f4ee40a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee9000640 1 -- 172.21.15.59:0/3989276994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41f4450 con 0x7f4ee40b4f20 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee9000640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee41f1740 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c790 tx=0x7f4ed400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee0c00640 1 -- 172.21.15.59:0/3989276994 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4016020 con 0x7f4ee40b4f20 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee0c00640 1 -- 172.21.15.59:0/3989276994 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed4006d50 con 0x7f4ee40b4f20 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41f4710 con 0x7f4ee40b4f20 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4ee0c00640 1 -- 172.21.15.59:0/3989276994 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4003bb0 con 0x7f4ee40b4f20 2024-04-23T14:44:04.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.812+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41f4cd0 con 0x7f4ee40b4f20 2024-04-23T14:44:04.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.814+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee404fd90 con 0x7f4ee40b4f20 2024-04-23T14:44:04.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.814+0000 7f4ee0c00640 1 -- 172.21.15.59:0/3989276994 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ed400a850 con 0x7f4ee40b4f20 2024-04-23T14:44:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.816+0000 7f4ee0c00640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ec406c8f0 0x7f4ec406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.816+0000 7f4ee0c00640 1 -- 172.21.15.59:0/3989276994 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f4ed408b8b0 con 0x7f4ee40b4f20 2024-04-23T14:44:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.816+0000 7f4ee3e00640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ec406c8f0 0x7f4ec406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:04.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.817+0000 7f4ee3e00640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ec406c8f0 0x7f4ec406edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006a90 tx=0x7f4ed0006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:04.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:04.821+0000 7f4ee0c00640 1 -- 172.21.15.59:0/3989276994 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ed4020020 con 0x7f4ee40b4f20 2024-04-23T14:44:05.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.027+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ee415ddd0 con 0x7f4ee40b4f20 2024-04-23T14:44:05.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.027+0000 7f4ee0c00640 1 -- 172.21.15.59:0/3989276994 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7f4ed4019310 con 0x7f4ee40b4f20 2024-04-23T14:44:05.029 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.030+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ec406c8f0 msgr2=0x7f4ec406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.030+0000 7f4eea400640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ec406c8f0 0x7f4ec406edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006a90 tx=0x7f4ed0006a20 comp rx=0 tx=0).stop 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.030+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 msgr2=0x7f4ee41f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.030+0000 7f4eea400640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee41f1740 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c790 tx=0x7f4ed400cc50 comp rx=0 tx=0).stop 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 shutdown_connections 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ec406c8f0 0x7f4ec406edb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ee415ed20 0x7f4ee40a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ee40d15b0 0x7f4ee40a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 --2- 172.21.15.59:0/3989276994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ee40b4f20 0x7f4ee41f1740 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 >> 172.21.15.59:0/3989276994 conn(0x7f4ee408bf90 msgr2=0x7f4ee415d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:05.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 shutdown_connections 2024-04-23T14:44:05.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.031+0000 7f4eea400640 1 -- 172.21.15.59:0/3989276994 wait complete. 2024-04-23T14:44:05.052 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:44:05.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.263+0000 7fabc8800640 1 Processor -- start 2024-04-23T14:44:05.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.264+0000 7fabc8800640 1 -- start start 2024-04-23T14:44:05.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.265+0000 7fabc8800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 0x7fabc00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.265+0000 7fabc8800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc00cd880 con 0x7fabc006ddf0 2024-04-23T14:44:05.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.266+0000 7fabc8800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc0071ec0 con 0x7fabc00cf410 2024-04-23T14:44:05.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.266+0000 7fabc8800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fabc00cdb20 con 0x7fabc006e250 2024-04-23T14:44:05.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.266+0000 7fabc5400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fabc006ddf0 0x7fabc0072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39538/0 (socket says 172.21.15.59:39538) 2024-04-23T14:44:05.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.266+0000 7fabc5400640 1 -- 172.21.15.59:0/71559627 learned_addr learned my addr 172.21.15.59:0/71559627 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:05.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.266+0000 7fabc6800640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 0x7fabc00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.266+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862394194 0 0) 0x7fabc00cd880 con 0x7fabc006ddf0 2024-04-23T14:44:05.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faba8003520 con 0x7fabc006ddf0 2024-04-23T14:44:05.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 624708082 0 0) 0x7faba8003520 con 0x7fabc006ddf0 2024-04-23T14:44:05.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 msgr2=0x7fabc00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:05.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 0x7fabc00cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 >> v1:172.21.15.76:6789/0 conn(0x7fabc00cf410 legacy=0x7fabc015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc0072160 con 0x7fabc006ddf0 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fabb0001670 con 0x7fabc006ddf0 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 >> v1:172.21.15.59:6789/0 conn(0x7fabc006ddf0 legacy=0x7fabc0072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80039e0 0x7faba8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 0x7faba8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7faba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabc00cdb20 con 0x7faba8004d40 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faba800bb20 con 0x7faba80039e0 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faba800bd20 con 0x7fabc006e250 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.267+0000 7fabc6800640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7faba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc5e00640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80039e0 0x7faba8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc5400640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 0x7faba8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc6800640 1 -- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 msgr2=0x7faba8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:05.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc6800640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 0x7faba8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc6800640 1 -- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80039e0 msgr2=0x7faba8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:05.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc6800640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faba80039e0 0x7faba8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc6800640 1 -- 172.21.15.59:0/71559627 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba8003520 con 0x7faba8004d40 2024-04-23T14:44:05.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.268+0000 7fabc5400640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc006e250 0x7faba8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:05.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.269+0000 7fabc6800640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7faba80051c0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009fc0 tx=0x7faba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d878df726d57275 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:05.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.269+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc009c10 con 0x7faba8004d40 2024-04-23T14:44:05.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.269+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc036070 con 0x7faba8004d40 2024-04-23T14:44:05.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.269+0000 7fabc4a00640 1 -- 172.21.15.59:0/71559627 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc031880 con 0x7faba8004d40 2024-04-23T14:44:05.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.270+0000 7fabc8800640 1 -- 172.21.15.59:0/71559627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 msgr2=0x7faba80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.270+0000 7fabc8800640 1 --2- 172.21.15.59:0/71559627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7faba80051c0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009fc0 tx=0x7faba8005830 comp rx=0 tx=0).stop 2024-04-23T14:44:05.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.270+0000 7fabc6800640 1 -- 172.21.15.59:0/71559627 reap_dead start 2024-04-23T14:44:05.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.270+0000 7fabc8800640 1 -- 172.21.15.59:0/71559627 shutdown_connections 2024-04-23T14:44:05.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.270+0000 7fabc8800640 1 -- 172.21.15.59:0/71559627 >> 172.21.15.59:0/71559627 conn(0x7fabc00b2130 msgr2=0x7fabc00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:05.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.270+0000 7fabc8800640 1 -- 172.21.15.59:0/71559627 shutdown_connections 2024-04-23T14:44:05.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.270+0000 7fabc8800640 1 -- 172.21.15.59:0/71559627 wait complete. 2024-04-23T14:44:05.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.272+0000 7fabc8800640 1 Processor -- start 2024-04-23T14:44:05.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.272+0000 7fabc8800640 1 -- start start 2024-04-23T14:44:05.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc8800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7fabc0158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc8800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc015d3a0 0x7fabc0156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc8800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc015d820 0x7fabc0156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc8800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabc015a820 con 0x7faba8004d40 2024-04-23T14:44:05.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc8800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fabc015a6a0 con 0x7fabc015d3a0 2024-04-23T14:44:05.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc8800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fabc0072d90 con 0x7fabc015d820 2024-04-23T14:44:05.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7fabc0158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc5e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7fabc0158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48636/0 (socket says 172.21.15.59:48636) 2024-04-23T14:44:05.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.273+0000 7fabc5e00640 1 -- 172.21.15.59:0/1643394577 learned_addr learned my addr 172.21.15.59:0/1643394577 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc5e00640 1 -- 172.21.15.59:0/1643394577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc015d820 msgr2=0x7fabc0156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc5e00640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc015d820 0x7fabc0156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc5e00640 1 -- 172.21.15.59:0/1643394577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc015d3a0 msgr2=0x7fabc0156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc5e00640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc015d3a0 0x7fabc0156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc5e00640 1 -- 172.21.15.59:0/1643394577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc015dd60 con 0x7faba8004d40 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc5e00640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7fabc0158c80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e550 tx=0x7fabb400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabba000640 1 -- 172.21.15.59:0/1643394577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb40052e0 con 0x7faba8004d40 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabba000640 1 -- 172.21.15.59:0/1643394577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb4005480 con 0x7faba8004d40 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc015e040 con 0x7faba8004d40 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabba000640 1 -- 172.21.15.59:0/1643394577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4007020 con 0x7faba8004d40 2024-04-23T14:44:05.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.274+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc015e600 con 0x7faba8004d40 2024-04-23T14:44:05.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.276+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc0068a80 con 0x7faba8004d40 2024-04-23T14:44:05.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.277+0000 7fabba000640 1 -- 172.21.15.59:0/1643394577 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabb401a020 con 0x7faba8004d40 2024-04-23T14:44:05.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.278+0000 7fabba000640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.278+0000 7fabba000640 1 -- 172.21.15.59:0/1643394577 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fabb408b1f0 con 0x7faba8004d40 2024-04-23T14:44:05.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.278+0000 7fabc5400640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.279+0000 7fabc5400640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fabc0159830 tx=0x7fabb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:05.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.282+0000 7fabba000640 1 -- 172.21.15.59:0/1643394577 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fabb401a310 con 0x7faba8004d40 2024-04-23T14:44:05.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.503+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fabc00700b0 con 0x7faba8004d40 2024-04-23T14:44:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.504+0000 7fabba000640 1 -- 172.21.15.59:0/1643394577 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7fabb4059580 con 0x7faba8004d40 2024-04-23T14:44:05.505 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 msgr2=0x7faba406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fabc0159830 tx=0x7fabb0009040 comp rx=0 tx=0).stop 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 msgr2=0x7fabc0158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7fabc0158c80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e550 tx=0x7fabb400ea10 comp rx=0 tx=0).stop 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 shutdown_connections 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faba406c8f0 0x7faba406edb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fabc015d820 0x7fabc0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabc015d3a0 0x7fabc0156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 --2- 172.21.15.59:0/1643394577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faba8004d40 0x7fabc0158c80 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 >> 172.21.15.59:0/1643394577 conn(0x7fabc00b2130 msgr2=0x7fabc00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.506+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 shutdown_connections 2024-04-23T14:44:05.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.507+0000 7fabc8800640 1 -- 172.21.15.59:0/1643394577 wait complete. 2024-04-23T14:44:05.527 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:44:05.527 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:05.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:05.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.795+0000 7f30f8200640 1 Processor -- start 2024-04-23T14:44:05.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.795+0000 7f30f8200640 1 -- start start 2024-04-23T14:44:05.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.797+0000 7f30f8200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30f00afaf0 0x7f30f00d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.797+0000 7f30f8200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f00d06e0 0x7f30f01624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.797+0000 7f30f8200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f015f0f0 con 0x7f30f015c5e0 2024-04-23T14:44:05.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.797+0000 7f30f8200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f30f015fa20 con 0x7f30f00afaf0 2024-04-23T14:44:05.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.797+0000 7f30f8200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30f015f3d0 con 0x7f30f00d06e0 2024-04-23T14:44:05.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.798+0000 7f30f4e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30f00afaf0 0x7f30f00d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.798+0000 7f30f4e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f30f00afaf0 0x7f30f00d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:40378/0 (socket says 172.21.15.59:40378) 2024-04-23T14:44:05.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.798+0000 7f30f4e00640 1 -- 172.21.15.59:0/4181407908 learned_addr learned my addr 172.21.15.59:0/4181407908 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:05.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.798+0000 7f30f5800640 1 --1- 172.21.15.59:0/4181407908 >> v1:172.21.15.76:6789/0 conn(0x7f30f015c5e0 0x7f30f00af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:34458/0 (socket says 172.21.15.59:34458) 2024-04-23T14:44:05.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.798+0000 7f30f6200640 1 --2- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f00d06e0 0x7f30f01624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.799+0000 7f30f4e00640 1 -- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f00d06e0 msgr2=0x7f30f01624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:05.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.799+0000 7f30f4e00640 1 --2- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f00d06e0 0x7f30f01624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.799+0000 7f30f4e00640 1 -- 172.21.15.59:0/4181407908 >> v1:172.21.15.76:6789/0 conn(0x7f30f015c5e0 legacy=0x7f30f00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.799+0000 7f30efe00640 1 -- 172.21.15.59:0/4181407908 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084532502 0 0) 0x7f30f015f0f0 con 0x7f30f015c5e0 2024-04-23T14:44:05.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.799+0000 7f30f4e00640 1 -- 172.21.15.59:0/4181407908 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f015f6b0 con 0x7f30f00afaf0 2024-04-23T14:44:05.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.800+0000 7f30f4e00640 1 --2- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00afaf0 0x7f30f00d01a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009870 tx=0x7f30dc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6534129ee6cb010 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:05.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.801+0000 7f30efe00640 1 -- 172.21.15.59:0/4181407908 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc037070 con 0x7f30f00afaf0 2024-04-23T14:44:05.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.801+0000 7f30efe00640 1 -- 172.21.15.59:0/4181407908 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30dc032440 con 0x7f30f00afaf0 2024-04-23T14:44:05.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.801+0000 7f30efe00640 1 -- 172.21.15.59:0/4181407908 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc032810 con 0x7f30f00afaf0 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.801+0000 7f30f8200640 1 -- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00afaf0 msgr2=0x7f30f00d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.801+0000 7f30f8200640 1 --2- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00afaf0 0x7f30f00d01a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009870 tx=0x7f30dc02a270 comp rx=0 tx=0).stop 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.802+0000 7f30f8200640 1 -- 172.21.15.59:0/4181407908 shutdown_connections 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.802+0000 7f30f8200640 1 --2- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f00d06e0 0x7f30f01624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.802+0000 7f30f8200640 1 --2- 172.21.15.59:0/4181407908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00afaf0 0x7f30f00d01a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.802+0000 7f30f8200640 1 -- 172.21.15.59:0/4181407908 >> 172.21.15.59:0/4181407908 conn(0x7f30f0067ea0 msgr2=0x7f30f006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.802+0000 7f30f8200640 1 -- 172.21.15.59:0/4181407908 shutdown_connections 2024-04-23T14:44:05.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.802+0000 7f30f8200640 1 -- 172.21.15.59:0/4181407908 wait complete. 2024-04-23T14:44:05.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.804+0000 7f30f8200640 1 Processor -- start 2024-04-23T14:44:05.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.804+0000 7f30f8200640 1 -- start start 2024-04-23T14:44:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.805+0000 7f30f8200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30f00afaf0 0x7f30f01f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.805+0000 7f30f8200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00d06e0 0x7f30f015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.805+0000 7f30f8200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f015c5e0 0x7f30f0157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.805+0000 7f30f8200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f30f006e600 con 0x7f30f00afaf0 2024-04-23T14:44:05.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.805+0000 7f30f8200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30f006e480 con 0x7f30f00d06e0 2024-04-23T14:44:05.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.805+0000 7f30f8200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f30f0162a10 con 0x7f30f015c5e0 2024-04-23T14:44:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30f00afaf0 0x7f30f01f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30f00afaf0 0x7f30f01f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48644/0 (socket says 172.21.15.59:48644) 2024-04-23T14:44:05.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 -- 172.21.15.59:0/615004722 learned_addr learned my addr 172.21.15.59:0/615004722 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f4e00640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00d06e0 0x7f30f015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f6200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f015c5e0 0x7f30f0157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 -- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f015c5e0 msgr2=0x7f30f0157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f015c5e0 0x7f30f0157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 -- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00d06e0 msgr2=0x7f30f015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00d06e0 0x7f30f015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 -- 172.21.15.59:0/615004722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f0158b50 con 0x7f30f00afaf0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f6200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f015c5e0 0x7f30f0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.806+0000 7f30f5800640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30f00afaf0 0x7f30f01f5000 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c420 tx=0x7f30e400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.807+0000 7f30ed600640 1 -- 172.21.15.59:0/615004722 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e40074e0 con 0x7f30f00afaf0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.807+0000 7f30ed600640 1 -- 172.21.15.59:0/615004722 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e40047e0 con 0x7f30f00afaf0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.807+0000 7f30ed600640 1 -- 172.21.15.59:0/615004722 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4004c10 con 0x7f30f00afaf0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.807+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f0158e30 con 0x7f30f00afaf0 2024-04-23T14:44:05.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.807+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f01593f0 con 0x7f30f00afaf0 2024-04-23T14:44:05.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.808+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f0068a80 con 0x7f30f00afaf0 2024-04-23T14:44:05.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.810+0000 7f30ed600640 1 -- 172.21.15.59:0/615004722 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30e4011070 con 0x7f30f00afaf0 2024-04-23T14:44:05.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.810+0000 7f30ed600640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30d006c8f0 0x7f30d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.811+0000 7f30ed600640 1 -- 172.21.15.59:0/615004722 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f30e408b850 con 0x7f30f00afaf0 2024-04-23T14:44:05.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.811+0000 7f30f4e00640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30d006c8f0 0x7f30d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:05.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.812+0000 7f30f4e00640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30d006c8f0 0x7f30d006edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f30dc034000 tx=0x7f30dc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:05.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:05.816+0000 7f30ed600640 1 -- 172.21.15.59:0/615004722 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30e400e280 con 0x7f30f00afaf0 2024-04-23T14:44:06.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.022+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30f006e260 con 0x7f30f00afaf0 2024-04-23T14:44:06.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.024+0000 7f30ed600640 1 -- 172.21.15.59:0/615004722 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16567 (secure 0 0 0) 0x7f30e4059be0 con 0x7f30f00afaf0 2024-04-23T14:44:06.026 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:06.026 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":112,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:43:39.410742+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.026+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30d006c8f0 msgr2=0x7f30d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.026+0000 7f30f8200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30d006c8f0 0x7f30d006edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f30dc034000 tx=0x7f30dc02e000 comp rx=0 tx=0).stop 2024-04-23T14:44:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.026+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30f00afaf0 msgr2=0x7f30f01f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.026+0000 7f30f8200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30f00afaf0 0x7f30f01f5000 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c420 tx=0x7f30e400c8e0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 shutdown_connections 2024-04-23T14:44:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f30d006c8f0 0x7f30d006edb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f30f015c5e0 0x7f30f0157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30f00d06e0 0x7f30f015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 --2- 172.21.15.59:0/615004722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30f00afaf0 0x7f30f01f5000 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 >> 172.21.15.59:0/615004722 conn(0x7f30f0067ea0 msgr2=0x7f30f00adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 shutdown_connections 2024-04-23T14:44:06.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.027+0000 7f30f8200640 1 -- 172.21.15.59:0/615004722 wait complete. 2024-04-23T14:44:06.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:44:06.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:44:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.248+0000 7f9781a00640 1 Processor -- start 2024-04-23T14:44:06.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.249+0000 7f9781a00640 1 -- start start 2024-04-23T14:44:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f9781a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c155a80 0x7f977c06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f9781a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f977c0af2a0 0x7f977c0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f9781a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f977c15ef20 con 0x7f977c0d0600 2024-04-23T14:44:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f9781a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f977c0d0fa0 con 0x7f977c0af2a0 2024-04-23T14:44:06.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f9781a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f977c15f1c0 con 0x7f977c155a80 2024-04-23T14:44:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f977be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c155a80 0x7f977c06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f977be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c155a80 0x7f977c06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:46476/0 (socket says 172.21.15.59:46476) 2024-04-23T14:44:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f977be00640 1 -- 172.21.15.59:0/1774859221 learned_addr learned my addr 172.21.15.59:0/1774859221 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:06.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.251+0000 7f9781000640 1 --2- 172.21.15.59:0/1774859221 >> v2:172.21.15.76:3300/0 conn(0x7f977c0af2a0 0x7f977c0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.252+0000 7f9781000640 1 -- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c155a80 msgr2=0x7f977c06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.252+0000 7f9781000640 1 --2- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c155a80 0x7f977c06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.252+0000 7f9781000640 1 -- 172.21.15.59:0/1774859221 >> v1:172.21.15.76:6789/0 conn(0x7f977c0d0600 legacy=0x7f977c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.252+0000 7f9781000640 1 -- 172.21.15.59:0/1774859221 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c15f4a0 con 0x7f977c0af2a0 2024-04-23T14:44:06.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.252+0000 7f977be00640 1 --2- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c155a80 0x7f977c06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:44:06.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.254+0000 7f9781000640 1 --2- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c0af2a0 0x7f977c0d0a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9774009d90 tx=0x7f977402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e8243c81a57a2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.255+0000 7f977b400640 1 -- 172.21.15.59:0/1774859221 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774007740 con 0x7f977c0af2a0 2024-04-23T14:44:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.255+0000 7f977b400640 1 -- 172.21.15.59:0/1774859221 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97740078e0 con 0x7f977c0af2a0 2024-04-23T14:44:06.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.255+0000 7f977b400640 1 -- 172.21.15.59:0/1774859221 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774007cb0 con 0x7f977c0af2a0 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.255+0000 7f9781a00640 1 -- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c0af2a0 msgr2=0x7f977c0d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.255+0000 7f9781a00640 1 --2- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c0af2a0 0x7f977c0d0a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9774009d90 tx=0x7f977402a660 comp rx=0 tx=0).stop 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.256+0000 7f9781a00640 1 -- 172.21.15.59:0/1774859221 shutdown_connections 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.256+0000 7f9781a00640 1 --2- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c0af2a0 0x7f977c0d0a60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.256+0000 7f9781a00640 1 --2- 172.21.15.59:0/1774859221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c155a80 0x7f977c06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.256+0000 7f9781a00640 1 -- 172.21.15.59:0/1774859221 >> 172.21.15.59:0/1774859221 conn(0x7f977c06bdf0 msgr2=0x7f977c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.256+0000 7f9781a00640 1 -- 172.21.15.59:0/1774859221 shutdown_connections 2024-04-23T14:44:06.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.256+0000 7f9781a00640 1 -- 172.21.15.59:0/1774859221 wait complete. 2024-04-23T14:44:06.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.258+0000 7f9781a00640 1 Processor -- start 2024-04-23T14:44:06.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.258+0000 7f9781a00640 1 -- start start 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9781a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c0af2a0 0x7f977c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9781a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f977c0d0600 0x7f977c168d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9781a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c155a80 0x7f977c14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9781a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f977c15fd40 con 0x7f977c0d0600 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9781a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f977c15fa60 con 0x7f977c155a80 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9781a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f977c06ff00 con 0x7f977c0af2a0 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9773e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f977c0d0600 0x7f977c168d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9773e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f977c0d0600 0x7f977c168d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48658/0 (socket says 172.21.15.59:48658) 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9773e00640 1 -- 172.21.15.59:0/2626097933 learned_addr learned my addr 172.21.15.59:0/2626097933 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f977be00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c0af2a0 0x7f977c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9781000640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c155a80 0x7f977c14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9773e00640 1 -- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c0af2a0 msgr2=0x7f977c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9773e00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c0af2a0 0x7f977c0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9773e00640 1 -- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c155a80 msgr2=0x7f977c14b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.260+0000 7f9773e00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c155a80 0x7f977c14b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.261+0000 7f9773e00640 1 -- 172.21.15.59:0/2626097933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c0b2d80 con 0x7f977c0d0600 2024-04-23T14:44:06.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.261+0000 7f9773e00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f977c0d0600 0x7f977c168d60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f976400c450 tx=0x7f976400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.261+0000 7f9778c00640 1 -- 172.21.15.59:0/2626097933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640052d0 con 0x7f977c0d0600 2024-04-23T14:44:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.261+0000 7f9778c00640 1 -- 172.21.15.59:0/2626097933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9764005470 con 0x7f977c0d0600 2024-04-23T14:44:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.261+0000 7f9778c00640 1 -- 172.21.15.59:0/2626097933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764012870 con 0x7f977c0d0600 2024-04-23T14:44:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.261+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1692a0 con 0x7f977c0d0600 2024-04-23T14:44:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.261+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c0b27c0 con 0x7f977c0d0600 2024-04-23T14:44:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.263+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c0b3140 con 0x7f977c0d0600 2024-04-23T14:44:06.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.264+0000 7f9778c00640 1 -- 172.21.15.59:0/2626097933 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9764012a10 con 0x7f977c0d0600 2024-04-23T14:44:06.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.264+0000 7f9778c00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f975806c8f0 0x7f975806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.265+0000 7f9778c00640 1 -- 172.21.15.59:0/2626097933 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f976408b7a0 con 0x7f977c0d0600 2024-04-23T14:44:06.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.265+0000 7f977be00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f975806c8f0 0x7f975806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.266+0000 7f977be00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f975806c8f0 0x7f975806edb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f977c073070 tx=0x7f976c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:06.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.269+0000 7f9778c00640 1 -- 172.21.15.59:0/2626097933 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f977c0b3140 con 0x7f977c0d0600 2024-04-23T14:44:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.480+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f977c04fd90 con 0x7f977c0d0600 2024-04-23T14:44:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.481+0000 7f9778c00640 1 -- 172.21.15.59:0/2626097933 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f9764059bc0 con 0x7f977c0d0600 2024-04-23T14:44:06.482 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:44:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f975806c8f0 msgr2=0x7f975806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f975806c8f0 0x7f975806edb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f977c073070 tx=0x7f976c008040 comp rx=0 tx=0).stop 2024-04-23T14:44:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f977c0d0600 msgr2=0x7f977c168d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f977c0d0600 0x7f977c168d60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f976400c450 tx=0x7f976400c910 comp rx=0 tx=0).stop 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 shutdown_connections 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f975806c8f0 0x7f975806edb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f977c155a80 0x7f977c14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f977c0d0600 0x7f977c168d60 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 --2- 172.21.15.59:0/2626097933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f977c0af2a0 0x7f977c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 >> 172.21.15.59:0/2626097933 conn(0x7f977c06bdf0 msgr2=0x7f977c1546b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 shutdown_connections 2024-04-23T14:44:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.484+0000 7f9781a00640 1 -- 172.21.15.59:0/2626097933 wait complete. 2024-04-23T14:44:06.503 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:44:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.715+0000 7fa272a00640 1 Processor -- start 2024-04-23T14:44:06.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.715+0000 7fa272a00640 1 -- start start 2024-04-23T14:44:06.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.717+0000 7fa272a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa26c145e10 0x7fa26c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.717+0000 7fa272a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c0d0cc0 0x7fa26c0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.717+0000 7fa272a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa26c0cfc20 con 0x7fa26c071d70 2024-04-23T14:44:06.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.717+0000 7fa272a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa26c0cff00 con 0x7fa26c145e10 2024-04-23T14:44:06.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.717+0000 7fa272a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa26c0d01e0 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa26be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa26c145e10 0x7fa26c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa26be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa26c145e10 0x7fa26c06f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:40408/0 (socket says 172.21.15.59:40408) 2024-04-23T14:44:06.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa26be00640 1 -- 172.21.15.59:0/1043189547 learned_addr learned my addr 172.21.15.59:0/1043189547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:06.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa263e00640 1 --2- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c0d0cc0 0x7fa26c0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa26be00640 1 -- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c0d0cc0 msgr2=0x7fa26c0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:06.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa26be00640 1 --2- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c0d0cc0 0x7fa26c0cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa26be00640 1 -- 172.21.15.59:0/1043189547 >> v1:172.21.15.76:6789/0 conn(0x7fa26c071d70 legacy=0x7fa26c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.718+0000 7fa26be00640 1 -- 172.21.15.59:0/1043189547 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c0d04c0 con 0x7fa26c145e10 2024-04-23T14:44:06.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.720+0000 7fa26be00640 1 --2- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 0x7fa26c06f130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0098a0 tx=0x7fa25c02a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79857c0eef134e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:06.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.721+0000 7fa26b400640 1 -- 172.21.15.59:0/1043189547 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c037070 con 0x7fa26c145e10 2024-04-23T14:44:06.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.721+0000 7fa26b400640 1 -- 172.21.15.59:0/1043189547 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25c032440 con 0x7fa26c145e10 2024-04-23T14:44:06.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.721+0000 7fa26b400640 1 -- 172.21.15.59:0/1043189547 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c032810 con 0x7fa26c145e10 2024-04-23T14:44:06.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 -- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 msgr2=0x7fa26c06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 --2- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 0x7fa26c06f130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0098a0 tx=0x7fa25c02a290 comp rx=0 tx=0).stop 2024-04-23T14:44:06.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 -- 172.21.15.59:0/1043189547 shutdown_connections 2024-04-23T14:44:06.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 --2- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c0d0cc0 0x7fa26c0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 --2- 172.21.15.59:0/1043189547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 0x7fa26c06f130 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 -- 172.21.15.59:0/1043189547 >> 172.21.15.59:0/1043189547 conn(0x7fa26c067d90 msgr2=0x7fa26c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:06.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 -- 172.21.15.59:0/1043189547 shutdown_connections 2024-04-23T14:44:06.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.722+0000 7fa272a00640 1 -- 172.21.15.59:0/1043189547 wait complete. 2024-04-23T14:44:06.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.724+0000 7fa272a00640 1 Processor -- start 2024-04-23T14:44:06.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.725+0000 7fa272a00640 1 -- start start 2024-04-23T14:44:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa272a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c071d70 0x7fa26c143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa272a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa26c0d0cc0 0x7fa26c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa272a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 0x7fa26c15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa272a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c0cfa00 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa272a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c0cf880 con 0x7fa26c145e10 2024-04-23T14:44:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa272a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fa26c1461f0 con 0x7fa26c071d70 2024-04-23T14:44:06.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa263e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa26c0d0cc0 0x7fa26c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa263e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa26c0d0cc0 0x7fa26c15a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48664/0 (socket says 172.21.15.59:48664) 2024-04-23T14:44:06.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa263e00640 1 -- 172.21.15.59:0/558807703 learned_addr learned my addr 172.21.15.59:0/558807703 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa26be00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c071d70 0x7fa26c143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.726+0000 7fa270a00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 0x7fa26c15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa263e00640 1 -- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c071d70 msgr2=0x7fa26c143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa263e00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c071d70 0x7fa26c143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa263e00640 1 -- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 msgr2=0x7fa26c15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa263e00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 0x7fa26c15dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa263e00640 1 -- 172.21.15.59:0/558807703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c15b430 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa26be00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c071d70 0x7fa26c143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa263e00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa26c0d0cc0 0x7fa26c15a920 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fa25400e9b0 tx=0x7fa25400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa268c00640 1 -- 172.21.15.59:0/558807703 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254017070 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa268c00640 1 -- 172.21.15.59:0/558807703 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25400c9a0 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26c15b710 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.727+0000 7fa268c00640 1 -- 172.21.15.59:0/558807703 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25400cd90 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.728+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26c15bcd0 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.729+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26c068a80 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.730+0000 7fa268c00640 1 -- 172.21.15.59:0/558807703 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa25401a070 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.731+0000 7fa268c00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c8f0 0x7fa24806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:06.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.731+0000 7fa268c00640 1 -- 172.21.15.59:0/558807703 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fa25408bf50 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.731+0000 7fa26be00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c8f0 0x7fa24806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:06.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.732+0000 7fa26be00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c8f0 0x7fa24806edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa25c034000 tx=0x7fa25c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.736+0000 7fa268c00640 1 -- 172.21.15.59:0/558807703 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa25405a2e0 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.944+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa26c166bc0 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.945+0000 7fa268c00640 1 -- 172.21.15.59:0/558807703 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7fa254059be0 con 0x7fa26c0d0cc0 2024-04-23T14:44:06.947 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:44:06.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c8f0 msgr2=0x7fa24806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c8f0 0x7fa24806edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa25c034000 tx=0x7fa25c02e000 comp rx=0 tx=0).stop 2024-04-23T14:44:06.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa26c0d0cc0 msgr2=0x7fa26c15a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa26c0d0cc0 0x7fa26c15a920 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fa25400e9b0 tx=0x7fa25400ee70 comp rx=0 tx=0).stop 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 shutdown_connections 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa24806c8f0 0x7fa24806edb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa26c145e10 0x7fa26c15dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa26c0d0cc0 0x7fa26c15a920 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 --2- 172.21.15.59:0/558807703 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa26c071d70 0x7fa26c143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 >> 172.21.15.59:0/558807703 conn(0x7fa26c067d90 msgr2=0x7fa26c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 shutdown_connections 2024-04-23T14:44:06.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:06.948+0000 7fa272a00640 1 -- 172.21.15.59:0/558807703 wait complete. 2024-04-23T14:44:06.967 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:44:07.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.179+0000 7f785c800640 1 Processor -- start 2024-04-23T14:44:07.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.180+0000 7f785c800640 1 -- start start 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.181+0000 7f785c800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f78540b56f0 0x7f785406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.181+0000 7f785c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f785406f340 0x7f785406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.181+0000 7f785c800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78540d0350 0x7f78540d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.181+0000 7f785c800640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f78540d0d20 con 0x7f78540b56f0 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.181+0000 7f785c800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f785415bf60 con 0x7f78540d0350 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.181+0000 7f785c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f785415c240 con 0x7f785406f340 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f78540b56f0 0x7f785406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f78540b56f0 0x7f785406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48674/0 (socket says 172.21.15.59:48674) 2024-04-23T14:44:07.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 -- 172.21.15.59:0/3022639441 learned_addr learned my addr 172.21.15.59:0/3022639441 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:07.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f785a800640 1 --2- 172.21.15.59:0/3022639441 >> v2:172.21.15.76:3300/0 conn(0x7f78540d0350 0x7f78540d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859400640 1 --2- 172.21.15.59:0/3022639441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f785406f340 0x7f785406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 -- 172.21.15.59:0/3022639441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f785406f340 msgr2=0x7f785406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:07.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 --2- 172.21.15.59:0/3022639441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f785406f340 0x7f785406f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 -- 172.21.15.59:0/3022639441 >> v2:172.21.15.76:3300/0 conn(0x7f78540d0350 msgr2=0x7f78540d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:07.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 --2- 172.21.15.59:0/3022639441 >> v2:172.21.15.76:3300/0 conn(0x7f78540d0350 0x7f78540d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f7859e00640 1 -- 172.21.15.59:0/3022639441 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785415c520 con 0x7f78540b56f0 2024-04-23T14:44:07.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.182+0000 7f785a800640 1 --2- 172.21.15.59:0/3022639441 >> v2:172.21.15.76:3300/0 conn(0x7f78540d0350 0x7f78540d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:07.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.184+0000 7f7859e00640 1 --2- 172.21.15.59:0/3022639441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78540b56f0 0x7f785406ddd0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f785406c620 tx=0x7f784802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3176b7a00e93ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:07.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.184+0000 7f7858a00640 1 -- 172.21.15.59:0/3022639441 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848037070 con 0x7f78540b56f0 2024-04-23T14:44:07.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.184+0000 7f7858a00640 1 -- 172.21.15.59:0/3022639441 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7848032440 con 0x7f78540b56f0 2024-04-23T14:44:07.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.184+0000 7f7858a00640 1 -- 172.21.15.59:0/3022639441 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848032810 con 0x7f78540b56f0 2024-04-23T14:44:07.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 -- 172.21.15.59:0/3022639441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78540b56f0 msgr2=0x7f785406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 --2- 172.21.15.59:0/3022639441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78540b56f0 0x7f785406ddd0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f785406c620 tx=0x7f784802a1e0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 -- 172.21.15.59:0/3022639441 shutdown_connections 2024-04-23T14:44:07.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 --2- 172.21.15.59:0/3022639441 >> v2:172.21.15.76:3300/0 conn(0x7f78540d0350 0x7f78540d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 --2- 172.21.15.59:0/3022639441 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f785406f340 0x7f785406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 --2- 172.21.15.59:0/3022639441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78540b56f0 0x7f785406ddd0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 -- 172.21.15.59:0/3022639441 >> 172.21.15.59:0/3022639441 conn(0x7f7854067d90 msgr2=0x7f785406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:07.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 -- 172.21.15.59:0/3022639441 shutdown_connections 2024-04-23T14:44:07.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.185+0000 7f785c800640 1 -- 172.21.15.59:0/3022639441 wait complete. 2024-04-23T14:44:07.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.187+0000 7f785c800640 1 Processor -- start 2024-04-23T14:44:07.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.187+0000 7f785c800640 1 -- start start 2024-04-23T14:44:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.188+0000 7f785c800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f785406f340 0x7f7854146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.188+0000 7f785c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78540b56f0 0x7f78541476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.188+0000 7f785c800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78540d0350 0x7f785415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.188+0000 7f785c800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f785415bc10 con 0x7f785406f340 2024-04-23T14:44:07.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.188+0000 7f785c800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f785415cae0 con 0x7f78540d0350 2024-04-23T14:44:07.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.188+0000 7f785c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f78540b3710 con 0x7f78540b56f0 2024-04-23T14:44:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f785406f340 0x7f7854146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f785406f340 0x7f7854146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48682/0 (socket says 172.21.15.59:48682) 2024-04-23T14:44:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 -- 172.21.15.59:0/2892576916 learned_addr learned my addr 172.21.15.59:0/2892576916 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f785a800640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78540d0350 0x7f785415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859400640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78540b56f0 0x7f78541476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 -- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78540b56f0 msgr2=0x7f78541476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78540b56f0 0x7f78541476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 -- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78540d0350 msgr2=0x7f785415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78540d0350 0x7f785415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 -- 172.21.15.59:0/2892576916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78541473f0 con 0x7f785406f340 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.189+0000 7f7859e00640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f785406f340 0x7f7854146eb0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f784802a220 tx=0x7f78480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.190+0000 7f784e000640 1 -- 172.21.15.59:0/2892576916 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848037040 con 0x7f785406f340 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.190+0000 7f784e000640 1 -- 172.21.15.59:0/2892576916 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7848042070 con 0x7f785406f340 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.190+0000 7f784e000640 1 -- 172.21.15.59:0/2892576916 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848030bb0 con 0x7f785406f340 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.190+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785415d140 con 0x7f785406f340 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.190+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785415d6c0 con 0x7f785406f340 2024-04-23T14:44:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.191+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7854068a80 con 0x7f785406f340 2024-04-23T14:44:07.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.192+0000 7f784e000640 1 -- 172.21.15.59:0/2892576916 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7848047460 con 0x7f785406f340 2024-04-23T14:44:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.193+0000 7f784e000640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f783806c920 0x7f783806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.193+0000 7f784e000640 1 -- 172.21.15.59:0/2892576916 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f78480ac7d0 con 0x7f785406f340 2024-04-23T14:44:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.193+0000 7f7859400640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f783806c920 0x7f783806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.194+0000 7f7859400640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f783806c920 0x7f783806ede0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f78540cf010 tx=0x7f7844008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:07.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.197+0000 7f784e000640 1 -- 172.21.15.59:0/2892576916 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78541473f0 con 0x7f785406f340 2024-04-23T14:44:07.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.400+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78540d0130 con 0x7f785406f340 2024-04-23T14:44:07.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.401+0000 7f784e000640 1 -- 172.21.15.59:0/2892576916 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7f7848045360 con 0x7f785406f340 2024-04-23T14:44:07.403 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f783806c920 msgr2=0x7f783806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f783806c920 0x7f783806ede0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f78540cf010 tx=0x7f7844008040 comp rx=0 tx=0).stop 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f785406f340 msgr2=0x7f7854146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f785406f340 0x7f7854146eb0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f784802a220 tx=0x7f78480046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 shutdown_connections 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f783806c920 0x7f783806ede0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78540b56f0 0x7f78541476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78540d0350 0x7f785415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 --2- 172.21.15.59:0/2892576916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f785406f340 0x7f7854146eb0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 >> 172.21.15.59:0/2892576916 conn(0x7f7854067d90 msgr2=0x7f7854071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:07.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 shutdown_connections 2024-04-23T14:44:07.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.404+0000 7f785c800640 1 -- 172.21.15.59:0/2892576916 wait complete. 2024-04-23T14:44:07.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:44:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.633+0000 7f1465200640 1 Processor -- start 2024-04-23T14:44:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.633+0000 7f1465200640 1 -- start start 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.635+0000 7f1465200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f146006ddf0 0x7f146006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.635+0000 7f1465200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1460072680 0x7f1460072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.635+0000 7f1465200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14600cd880 con 0x7f14600cf410 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.635+0000 7f1465200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1460071ec0 con 0x7f1460072680 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.635+0000 7f1465200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f14600cdb20 con 0x7f146006ddf0 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.635+0000 7f145de00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f14600cf410 0x7f146015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:39544/0 (socket says 172.21.15.59:39544) 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f145de00640 1 -- 172.21.15.59:0/952020812 learned_addr learned my addr 172.21.15.59:0/952020812 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f145e800640 1 --2- 172.21.15.59:0/952020812 >> v2:172.21.15.59:3300/0 conn(0x7f1460072680 0x7f1460072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f1457e00640 1 --2- 172.21.15.59:0/952020812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f146006ddf0 0x7f146006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f145d400640 1 -- 172.21.15.59:0/952020812 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521384216 0 0) 0x7f14600cd880 con 0x7f14600cf410 2024-04-23T14:44:07.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f145e800640 1 -- 172.21.15.59:0/952020812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f146006ddf0 msgr2=0x7f146006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:07.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f145e800640 1 --2- 172.21.15.59:0/952020812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f146006ddf0 0x7f146006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f145e800640 1 -- 172.21.15.59:0/952020812 >> v1:172.21.15.59:6789/0 conn(0x7f14600cf410 legacy=0x7f146015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.636+0000 7f145e800640 1 -- 172.21.15.59:0/952020812 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1460072160 con 0x7f1460072680 2024-04-23T14:44:07.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.638+0000 7f145e800640 1 --2- 172.21.15.59:0/952020812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f1460072b10 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1458009dc0 tx=0x7f145802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718633287cfa3fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:07.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.638+0000 7f145d400640 1 -- 172.21.15.59:0/952020812 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458007740 con 0x7f1460072680 2024-04-23T14:44:07.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.638+0000 7f145d400640 1 -- 172.21.15.59:0/952020812 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14580078e0 con 0x7f1460072680 2024-04-23T14:44:07.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.638+0000 7f145d400640 1 -- 172.21.15.59:0/952020812 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458007cb0 con 0x7f1460072680 2024-04-23T14:44:07.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 -- 172.21.15.59:0/952020812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 msgr2=0x7f1460072b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 --2- 172.21.15.59:0/952020812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f1460072b10 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1458009dc0 tx=0x7f145802a5f0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 -- 172.21.15.59:0/952020812 shutdown_connections 2024-04-23T14:44:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 --2- 172.21.15.59:0/952020812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f1460072b10 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 --2- 172.21.15.59:0/952020812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f146006ddf0 0x7f146006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 -- 172.21.15.59:0/952020812 >> 172.21.15.59:0/952020812 conn(0x7f14600b2130 msgr2=0x7f14600b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 -- 172.21.15.59:0/952020812 shutdown_connections 2024-04-23T14:44:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.639+0000 7f1465200640 1 -- 172.21.15.59:0/952020812 wait complete. 2024-04-23T14:44:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.641+0000 7f1465200640 1 Processor -- start 2024-04-23T14:44:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.642+0000 7f1465200640 1 -- start start 2024-04-23T14:44:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1465200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f146006ddf0 0x7f1460145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1465200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f146015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1465200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14600cf410 0x7f146015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1465200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f146015a820 con 0x7f1460072680 2024-04-23T14:44:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1465200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f146015a6a0 con 0x7f146006ddf0 2024-04-23T14:44:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1465200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f14600cf7f0 con 0x7f14600cf410 2024-04-23T14:44:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1457e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f146015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1457e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f146015f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48698/0 (socket says 172.21.15.59:48698) 2024-04-23T14:44:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.643+0000 7f1457e00640 1 -- 172.21.15.59:0/3804324620 learned_addr learned my addr 172.21.15.59:0/3804324620 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:07.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f145de00640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f146006ddf0 0x7f1460145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f145e800640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14600cf410 0x7f146015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1457e00640 1 -- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14600cf410 msgr2=0x7f146015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1457e00640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14600cf410 0x7f146015ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1457e00640 1 -- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f146006ddf0 msgr2=0x7f1460145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1457e00640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f146006ddf0 0x7f1460145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1457e00640 1 -- 172.21.15.59:0/3804324620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14600af750 con 0x7f1460072680 2024-04-23T14:44:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1457e00640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f146015f730 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f144800e9e0 tx=0x7f144800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1456000640 1 -- 172.21.15.59:0/3804324620 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448017070 con 0x7f1460072680 2024-04-23T14:44:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1456000640 1 -- 172.21.15.59:0/3804324620 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144800c9d0 con 0x7f1460072680 2024-04-23T14:44:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.644+0000 7f1456000640 1 -- 172.21.15.59:0/3804324620 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144800cdc0 con 0x7f1460072680 2024-04-23T14:44:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.645+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146015ec50 con 0x7f1460072680 2024-04-23T14:44:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.645+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1460146290 con 0x7f1460072680 2024-04-23T14:44:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.646+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1460068a80 con 0x7f1460072680 2024-04-23T14:44:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.647+0000 7f1456000640 1 -- 172.21.15.59:0/3804324620 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f144801a070 con 0x7f1460072680 2024-04-23T14:44:07.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.648+0000 7f1456000640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f143c06c920 0x7f143c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:07.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.648+0000 7f1456000640 1 -- 172.21.15.59:0/3804324620 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f144808bf30 con 0x7f1460072680 2024-04-23T14:44:07.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.648+0000 7f145de00640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f143c06c920 0x7f143c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:07.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.649+0000 7f145de00640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f143c06c920 0x7f143c06ede0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1450000f60 tx=0x7f1450007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:07.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.652+0000 7f1456000640 1 -- 172.21.15.59:0/3804324620 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f144805a2d0 con 0x7f1460072680 2024-04-23T14:44:07.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.869+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f14601544d0 con 0x7f1460072680 2024-04-23T14:44:07.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.870+0000 7f1456000640 1 -- 172.21.15.59:0/3804324620 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7f1448059bd0 con 0x7f1460072680 2024-04-23T14:44:07.871 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.872+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f143c06c920 msgr2=0x7f143c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.872+0000 7f1465200640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f143c06c920 0x7f143c06ede0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1450000f60 tx=0x7f1450007c00 comp rx=0 tx=0).stop 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 msgr2=0x7f146015f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f146015f730 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f144800e9e0 tx=0x7f144800eea0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 shutdown_connections 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f143c06c920 0x7f143c06ede0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14600cf410 0x7f146015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1460072680 0x7f146015f730 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 --2- 172.21.15.59:0/3804324620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f146006ddf0 0x7f1460145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:07.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 >> 172.21.15.59:0/3804324620 conn(0x7f14600b2130 msgr2=0x7f146006f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:07.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 shutdown_connections 2024-04-23T14:44:07.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:07.873+0000 7f1465200640 1 -- 172.21.15.59:0/3804324620 wait complete. 2024-04-23T14:44:07.893 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:44:07.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:07.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:07.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[76470]: fuse finished with error 0 and tester_r 0 2024-04-23T14:44:07.976 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:44:14.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:44:14.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:14.077 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:44:14.139 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:44:14.140 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:14.197 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:44:14.197 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi059.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:14.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:44:14.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:14.197 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:44:14.251 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-23T14:44:14.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:44:14.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:44:14.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.472+0000 7faa4a800640 1 Processor -- start 2024-04-23T14:44:14.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.472+0000 7faa4a800640 1 -- start start 2024-04-23T14:44:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.473+0000 7faa4a800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faa440b1930 0x7faa4409b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa4a800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4406d830 con 0x7faa44161ef0 2024-04-23T14:44:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa4a800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa4406db10 con 0x7faa4409aa00 2024-04-23T14:44:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa4a800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faa4406ddf0 con 0x7faa440b1930 2024-04-23T14:44:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa43400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faa44161ef0 0x7faa440b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37330/0 (socket says 172.21.15.59:37330) 2024-04-23T14:44:14.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa43400640 1 -- 172.21.15.59:0/3764508292 learned_addr learned my addr 172.21.15.59:0/3764508292 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa3aa00640 1 --2- 172.21.15.59:0/3764508292 >> v2:172.21.15.76:3300/0 conn(0x7faa440b1930 0x7faa4409b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692400366 0 0) 0x7faa4406d830 con 0x7faa44161ef0 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.474+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa28003520 con 0x7faa44161ef0 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1612430107 0 0) 0x7faa28003520 con 0x7faa44161ef0 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 >> v2:172.21.15.76:3300/0 conn(0x7faa440b1930 msgr2=0x7faa4409b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 --2- 172.21.15.59:0/3764508292 >> v2:172.21.15.76:3300/0 conn(0x7faa440b1930 0x7faa4409b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 >> v1:172.21.15.76:6789/0 conn(0x7faa4409aa00 legacy=0x7faa4409ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa440d0c00 con 0x7faa44161ef0 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7faa34001670 con 0x7faa44161ef0 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 >> v1:172.21.15.59:6789/0 conn(0x7faa44161ef0 legacy=0x7faa440b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:14.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa280039b0 0x7faa28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa28004800 0x7faa28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa4406ddf0 con 0x7faa280051c0 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa2800c160 con 0x7faa280039b0 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faa2800c360 con 0x7faa28004800 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.475+0000 7faa43e00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.476+0000 7faa43400640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa280039b0 0x7faa28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.476+0000 7faa3aa00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa28004800 0x7faa28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.476+0000 7faa43e00640 1 -- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa28004800 msgr2=0x7faa28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.476+0000 7faa43e00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa28004800 0x7faa28004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.476+0000 7faa43e00640 1 -- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa280039b0 msgr2=0x7faa28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.476+0000 7faa43e00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa280039b0 0x7faa28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.476+0000 7faa43e00640 1 -- 172.21.15.59:0/3764508292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa28003520 con 0x7faa280051c0 2024-04-23T14:44:14.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.477+0000 7faa43e00640 1 -- 172.21.15.59:0/3764508292 reap_dead start 2024-04-23T14:44:14.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.477+0000 7faa43e00640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa2800b9c0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7faa3c001750 tx=0x7faa28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3381a454ba91b7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.477+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c001370 con 0x7faa280051c0 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.477+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa3c005310 con 0x7faa280051c0 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.477+0000 7faa42a00640 1 -- 172.21.15.59:0/3764508292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c02abe0 con 0x7faa280051c0 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.478+0000 7faa4a800640 1 -- 172.21.15.59:0/3764508292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 msgr2=0x7faa2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.478+0000 7faa4a800640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa2800b9c0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7faa3c001750 tx=0x7faa28005830 comp rx=0 tx=0).stop 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.478+0000 7faa4a800640 1 -- 172.21.15.59:0/3764508292 shutdown_connections 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.478+0000 7faa4a800640 1 --2- 172.21.15.59:0/3764508292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa2800b9c0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.478+0000 7faa4a800640 1 -- 172.21.15.59:0/3764508292 >> 172.21.15.59:0/3764508292 conn(0x7faa440b4030 msgr2=0x7faa440b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.478+0000 7faa4a800640 1 -- 172.21.15.59:0/3764508292 shutdown_connections 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.478+0000 7faa4a800640 1 -- 172.21.15.59:0/3764508292 wait complete. 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.480+0000 7faa4a800640 1 Processor -- start 2024-04-23T14:44:14.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.480+0000 7faa4a800640 1 -- start start 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa4a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa440c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa4a800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa4415d040 0x7faa440b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa4a800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa440b79d0 0x7faa4409d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa4a800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faa4409b740 con 0x7faa280051c0 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa4a800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa440d08b0 con 0x7faa440b79d0 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa4a800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faa4409b8c0 con 0x7faa4415d040 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa440c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa43400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa440c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41254/0 (socket says 172.21.15.59:41254) 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa43400640 1 -- 172.21.15.59:0/3469728545 learned_addr learned my addr 172.21.15.59:0/3469728545 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.481+0000 7faa3aa00640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa4415d040 0x7faa440b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43e00640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa440b79d0 0x7faa4409d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43400640 1 -- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa4415d040 msgr2=0x7faa440b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43400640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa4415d040 0x7faa440b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43400640 1 -- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa440b79d0 msgr2=0x7faa4409d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43400640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa440b79d0 0x7faa4409d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43400640 1 -- 172.21.15.59:0/3469728545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa440b6870 con 0x7faa280051c0 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43e00640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa440b79d0 0x7faa4409d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa43400640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa440c8b00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e7b0 tx=0x7faa3400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa3be00640 1 -- 172.21.15.59:0/3469728545 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3400ee60 con 0x7faa280051c0 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa3be00640 1 -- 172.21.15.59:0/3469728545 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa34018070 con 0x7faa280051c0 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa440c6a50 con 0x7faa280051c0 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.482+0000 7faa3be00640 1 -- 172.21.15.59:0/3469728545 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3400cab0 con 0x7faa280051c0 2024-04-23T14:44:14.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.483+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa440c7010 con 0x7faa280051c0 2024-04-23T14:44:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.484+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa44068a80 con 0x7faa280051c0 2024-04-23T14:44:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.485+0000 7faa3be00640 1 -- 172.21.15.59:0/3469728545 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa3400cc50 con 0x7faa280051c0 2024-04-23T14:44:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.486+0000 7faa3be00640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.486+0000 7faa3be00640 1 -- 172.21.15.59:0/3469728545 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5767+0+0 (secure 0 0 0) 0x7faa3408bb30 con 0x7faa280051c0 2024-04-23T14:44:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.486+0000 7faa3aa00640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.487+0000 7faa3aa00640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faa2006c920 0x7faa2006ede0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faa440c8650 tx=0x7faa2c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.490+0000 7faa3be00640 1 -- 172.21.15.59:0/3469728545 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa34059df0 con 0x7faa280051c0 2024-04-23T14:44:14.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.734+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7faa440b7db0 con 0x7faa280051c0 2024-04-23T14:44:14.737 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:44:14.736+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:44:14.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.739+0000 7faa3be00640 1 -- 172.21.15.59:0/3469728545 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7faa340596f0 con 0x7faa280051c0 2024-04-23T14:44:14.741 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.742+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faa2006c920 msgr2=0x7faa2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.742+0000 7faa4a800640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faa2006c920 0x7faa2006ede0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faa440c8650 tx=0x7faa2c00d040 comp rx=0 tx=0).stop 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.742+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 msgr2=0x7faa440c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.742+0000 7faa4a800640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa440c8b00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e7b0 tx=0x7faa3400ec70 comp rx=0 tx=0).stop 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.743+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 shutdown_connections 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.743+0000 7faa4a800640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.743+0000 7faa4a800640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faa4415d040 0x7faa440b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.743+0000 7faa4a800640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa440b79d0 0x7faa4409d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.743+0000 7faa4a800640 1 --2- 172.21.15.59:0/3469728545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa280051c0 0x7faa440c8b00 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:14.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.743+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 >> 172.21.15.59:0/3469728545 conn(0x7faa440b4030 msgr2=0x7faa440b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:14.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.743+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 shutdown_connections 2024-04-23T14:44:14.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:14.744+0000 7faa4a800640 1 -- 172.21.15.59:0/3469728545 wait complete. 2024-04-23T14:44:14.765 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:14.772 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:14.773 INFO:tasks.ceph.mds.c.smithi059.stderr: -8472> 2024-04-23T14:43:42.308+0000 7f586fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:44:14.852 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:14.899 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.900 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-23T14:44:14.901 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.902 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.903 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:14.904 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:14.905 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:14.906 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:14.907 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-23T14:44:14.908 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = False 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-04-23T14:44:14.909 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:14.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:14.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-04-23T14:44:14.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.913 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: bdev async discard = True 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: bdev enable discard = True 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore allocator = bitmap 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-04-23T14:44:14.914 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-04-23T14:44:14.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:14.916 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon down mkfs grace = 300 2024-04-23T14:44:14.917 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:14.918 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = False 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:14.919 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:14.942 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:14.973 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:15.100 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-23T14:44:15.100 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-23T14:44:15.100 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:15.100 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:15.101 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.102 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-23T14:44:15.103 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.104 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-23T14:44:15.105 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:15.106 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:15.107 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-23T14:44:15.108 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-23T14:44:15.109 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = False 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = False 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:15.110 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-04-23T14:44:15.113 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:15.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-04-23T14:44:15.115 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-04-23T14:44:15.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: bdev async discard = True 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: bdev enable discard = True 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore allocator = bitmap 2024-04-23T14:44:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-04-23T14:44:15.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:15.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon down mkfs grace = 300 2024-04-23T14:44:15.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-23T14:44:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = False 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = False 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:15.122 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:15.147 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-23T14:44:15.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.264+0000 7fa1682edf40 1 Processor -- start 2024-04-23T14:44:15.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.264+0000 7fa1682edf40 1 -- start start 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.265+0000 7fa1682edf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8ce441d0 0x557c8cf1d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.265+0000 7fa1682edf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf03ac0 0x557c8ce746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.265+0000 7fa1682edf40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8cf1ead0 0x557c8ce74be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.265+0000 7fa1682edf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c8cdedcc0 con 0x557c8cf03ac0 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.265+0000 7fa1682edf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557c8cdf1d00 con 0x557c8ce441d0 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.265+0000 7fa1682edf40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557c8ce469c0 con 0x557c8cf1ead0 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf03ac0 0x557c8ce746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf03ac0 0x557c8ce746a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41288/0 (socket says 172.21.15.59:41288) 2024-04-23T14:44:15.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 -- 172.21.15.59:0/3852765393 learned_addr learned my addr 172.21.15.59:0/3852765393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa165400640 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8ce441d0 0x557c8cf1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa165e00640 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8cf1ead0 0x557c8ce74be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 -- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8cf1ead0 msgr2=0x557c8ce74be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8cf1ead0 0x557c8ce74be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 -- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8ce441d0 msgr2=0x557c8cf1d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8ce441d0 0x557c8cf1d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.266+0000 7fa164a00640 1 -- 172.21.15.59:0/3852765393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8cf1cfd0 con 0x557c8cf03ac0 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.267+0000 7fa164a00640 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf03ac0 0x557c8ce746a0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fa15000a440 tx=0x7fa150051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e38ee065e7aab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.267+0000 7fa15f400640 1 -- 172.21.15.59:0/3852765393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150063070 con 0x557c8cf03ac0 2024-04-23T14:44:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.267+0000 7fa15f400640 1 -- 172.21.15.59:0/3852765393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa150055b20 con 0x557c8cf03ac0 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.267+0000 7fa15f400640 1 -- 172.21.15.59:0/3852765393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150068460 con 0x557c8cf03ac0 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.267+0000 7fa1682edf40 1 -- 172.21.15.59:0/3852765393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf03ac0 msgr2=0x557c8ce746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.267+0000 7fa1682edf40 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf03ac0 0x557c8ce746a0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fa15000a440 tx=0x7fa150051000 comp rx=0 tx=0).stop 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.268+0000 7fa1682edf40 1 -- 172.21.15.59:0/3852765393 shutdown_connections 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.268+0000 7fa1682edf40 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8cf1ead0 0x557c8ce74be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.268+0000 7fa1682edf40 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf03ac0 0x557c8ce746a0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.268+0000 7fa1682edf40 1 --2- 172.21.15.59:0/3852765393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8ce441d0 0x557c8cf1d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.268+0000 7fa1682edf40 1 -- 172.21.15.59:0/3852765393 >> 172.21.15.59:0/3852765393 conn(0x557c8ce90cc0 msgr2=0x557c8ce30be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.268+0000 7fa1682edf40 1 -- 172.21.15.59:0/3852765393 shutdown_connections 2024-04-23T14:44:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.268+0000 7fa1682edf40 1 -- 172.21.15.59:0/3852765393 wait complete. 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.269+0000 7fa1682edf40 1 Processor -- start 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.269+0000 7fa1682edf40 1 -- start start 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa1682edf40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8ce441d0 0x557c8ce77320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa1682edf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8cf03ac0 0x557c8ceff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa1682edf40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf1ead0 0x557c8cf05ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa1682edf40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557c8ce90470 con 0x557c8cf1ead0 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa1682edf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557c8ce73bc0 con 0x557c8cf03ac0 2024-04-23T14:44:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa1682edf40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x557c8ce906c0 con 0x557c8ce441d0 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa165e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf1ead0 0x557c8cf05ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa165e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf1ead0 0x557c8cf05ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41300/0 (socket says 172.21.15.59:41300) 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa165e00640 1 -- 172.21.15.59:0/1783273775 learned_addr learned my addr 172.21.15.59:0/1783273775 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa164a00640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8cf03ac0 0x557c8ceff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.270+0000 7fa165400640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8ce441d0 0x557c8ce77320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa165e00640 1 -- 172.21.15.59:0/1783273775 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8ce441d0 msgr2=0x557c8ce77320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa165e00640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x557c8ce441d0 0x557c8ce77320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa165e00640 1 -- 172.21.15.59:0/1783273775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8cf03ac0 msgr2=0x557c8ceff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa165e00640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8cf03ac0 0x557c8ceff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa165e00640 1 -- 172.21.15.59:0/1783273775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8cf02190 con 0x557c8cf1ead0 2024-04-23T14:44:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa165e00640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf1ead0 0x557c8cf05ba0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fa16000d4c0 tx=0x7fa16000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa15e000640 1 -- 172.21.15.59:0/1783273775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1600125f0 con 0x557c8cf1ead0 2024-04-23T14:44:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa15e000640 1 -- 172.21.15.59:0/1783273775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa160018070 con 0x557c8cf1ead0 2024-04-23T14:44:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa15e000640 1 -- 172.21.15.59:0/1783273775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1600129d0 con 0x557c8cf1ead0 2024-04-23T14:44:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.271+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c8cf02470 con 0x557c8cf1ead0 2024-04-23T14:44:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.272+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c8cf02a30 con 0x557c8cf1ead0 2024-04-23T14:44:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.274+0000 7fa15e000640 1 -- 172.21.15.59:0/1783273775 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa160012b70 con 0x557c8cf1ead0 2024-04-23T14:44:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.275+0000 7fa15e000640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14406b300 0x7fa14406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.275+0000 7fa15e000640 1 -- 172.21.15.59:0/1783273775 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fa16008a630 con 0x557c8cf1ead0 2024-04-23T14:44:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.275+0000 7fa165400640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14406b300 0x7fa14406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.275+0000 7fa1682edf40 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x557c8cf1bf80 0x557c8cfb7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.275+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557c8cfb7e60 con 0x557c8cf1bf80 2024-04-23T14:44:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.275+0000 7fa164a00640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x557c8cf1bf80 0x557c8cfb7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.276+0000 7fa165400640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14406b300 0x7fa14406d7c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa158001ed0 tx=0x7fa158001670 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.276+0000 7fa164a00640 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x557c8cf1bf80 0x557c8cfb7920 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa164a00640 1 -- 172.21.15.59:0/1783273775 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v115'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa150055d50 con 0x557c8cf1bf80 2024-04-23T14:44:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x557c8cf1bf80 msgr2=0x557c8cfb7920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x557c8cf1bf80 0x557c8cfb7920 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14406b300 msgr2=0x7fa14406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fa14406b300 0x7fa14406d7c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa158001ed0 tx=0x7fa158001670 comp rx=0 tx=0).stop 2024-04-23T14:44:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf1ead0 msgr2=0x557c8cf05ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 --2- 172.21.15.59:0/1783273775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557c8cf1ead0 0x557c8cf05ba0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fa16000d4c0 tx=0x7fa16000d980 comp rx=0 tx=0).stop 2024-04-23T14:44:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa165e00640 1 -- 172.21.15.59:0/1783273775 reap_dead start 2024-04-23T14:44:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 shutdown_connections 2024-04-23T14:44:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 >> 172.21.15.59:0/1783273775 conn(0x557c8ce90cc0 msgr2=0x557c8ce30be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 shutdown_connections 2024-04-23T14:44:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.281+0000 7fa1682edf40 1 -- 172.21.15.59:0/1783273775 wait complete. 2024-04-23T14:44:15.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-23T14:44:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.487+0000 7fb408e00640 1 Processor -- start 2024-04-23T14:44:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.487+0000 7fb408e00640 1 -- start start 2024-04-23T14:44:15.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.488+0000 7fb408e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d0600 0x7fb4040ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb408e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb40406dcb0 0x7fb404162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb408e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb404163240 con 0x7fb4040d09e0 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb408e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb404072a20 con 0x7fb40406dcb0 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb408e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb404071e10 con 0x7fb4040d0600 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb402a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb40406dcb0 0x7fb404162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb402a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb40406dcb0 0x7fb404162d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41312/0 (socket says 172.21.15.59:41312) 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb402a00640 1 -- 172.21.15.59:0/2117362305 learned_addr learned my addr 172.21.15.59:0/2117362305 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.489+0000 7fb403400640 1 --2- 172.21.15.59:0/2117362305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d0600 0x7fb4040ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.490+0000 7fb402a00640 1 -- 172.21.15.59:0/2117362305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d0600 msgr2=0x7fb4040ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.490+0000 7fb402a00640 1 --2- 172.21.15.59:0/2117362305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d0600 0x7fb4040ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.490+0000 7fb402a00640 1 -- 172.21.15.59:0/2117362305 >> v1:172.21.15.76:6789/0 conn(0x7fb4040d09e0 legacy=0x7fb40406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.490+0000 7fb402a00640 1 -- 172.21.15.59:0/2117362305 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040720f0 con 0x7fb40406dcb0 2024-04-23T14:44:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.490+0000 7fb403400640 1 --2- 172.21.15.59:0/2117362305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d0600 0x7fb4040ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:44:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.491+0000 7fb402a00640 1 --2- 172.21.15.59:0/2117362305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404162d00 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009870 tx=0x7fb3ec02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9569dc5bdc5f2b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.492+0000 7fb402000640 1 -- 172.21.15.59:0/2117362305 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec037070 con 0x7fb40406dcb0 2024-04-23T14:44:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.492+0000 7fb402000640 1 -- 172.21.15.59:0/2117362305 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec032440 con 0x7fb40406dcb0 2024-04-23T14:44:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.492+0000 7fb402000640 1 -- 172.21.15.59:0/2117362305 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec032810 con 0x7fb40406dcb0 2024-04-23T14:44:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.492+0000 7fb408e00640 1 -- 172.21.15.59:0/2117362305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 msgr2=0x7fb404162d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.492+0000 7fb408e00640 1 --2- 172.21.15.59:0/2117362305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404162d00 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009870 tx=0x7fb3ec02a200 comp rx=0 tx=0).stop 2024-04-23T14:44:15.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.493+0000 7fb408e00640 1 -- 172.21.15.59:0/2117362305 shutdown_connections 2024-04-23T14:44:15.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.493+0000 7fb408e00640 1 --2- 172.21.15.59:0/2117362305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404162d00 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.493+0000 7fb408e00640 1 --2- 172.21.15.59:0/2117362305 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d0600 0x7fb4040ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.493+0000 7fb408e00640 1 -- 172.21.15.59:0/2117362305 >> 172.21.15.59:0/2117362305 conn(0x7fb40406bdf0 msgr2=0x7fb40406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:15.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.493+0000 7fb408e00640 1 -- 172.21.15.59:0/2117362305 shutdown_connections 2024-04-23T14:44:15.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.493+0000 7fb408e00640 1 -- 172.21.15.59:0/2117362305 wait complete. 2024-04-23T14:44:15.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.494+0000 7fb408e00640 1 Processor -- start 2024-04-23T14:44:15.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.495+0000 7fb408e00640 1 -- start start 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb408e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb408e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4040d0600 0x7fb4041f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb408e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d09e0 0x7fb404150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb408e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb40415eaf0 con 0x7fb40406dcb0 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb408e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040726b0 con 0x7fb4040d0600 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb408e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040d0e70 con 0x7fb4040d09e0 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb403400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb403400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404161750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41320/0 (socket says 172.21.15.59:41320) 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb403400640 1 -- 172.21.15.59:0/4148198577 learned_addr learned my addr 172.21.15.59:0/4148198577 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb402a00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4040d0600 0x7fb4041f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.496+0000 7fb403e00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d09e0 0x7fb404150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb403400640 1 -- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d09e0 msgr2=0x7fb404150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb403400640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d09e0 0x7fb404150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb403400640 1 -- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4040d0600 msgr2=0x7fb4041f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb403400640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4040d0600 0x7fb4041f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb403400640 1 -- 172.21.15.59:0/4148198577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb404150d50 con 0x7fb40406dcb0 2024-04-23T14:44:15.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb403400640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404161750 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e810 tx=0x7fb3f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:15.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb3fb400640 1 -- 172.21.15.59:0/4148198577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400f040 con 0x7fb40406dcb0 2024-04-23T14:44:15.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb3fb400640 1 -- 172.21.15.59:0/4148198577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f4004030 con 0x7fb40406dcb0 2024-04-23T14:44:15.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb3fb400640 1 -- 172.21.15.59:0/4148198577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4004400 con 0x7fb40406dcb0 2024-04-23T14:44:15.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.497+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4041f82c0 con 0x7fb40406dcb0 2024-04-23T14:44:15.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.498+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041f8880 con 0x7fb40406dcb0 2024-04-23T14:44:15.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.499+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4041f8ac0 con 0x7fb40406dcb0 2024-04-23T14:44:15.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.500+0000 7fb3fb400640 1 -- 172.21.15.59:0/4148198577 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3f400c6d0 con 0x7fb40406dcb0 2024-04-23T14:44:15.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.501+0000 7fb3fb400640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb3e006c8f0 0x7fb3e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:15.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.501+0000 7fb3fb400640 1 -- 172.21.15.59:0/4148198577 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fb3f408ba40 con 0x7fb40406dcb0 2024-04-23T14:44:15.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.501+0000 7fb402a00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb3e006c8f0 0x7fb3e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:15.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.502+0000 7fb402a00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb3e006c8f0 0x7fb3e006edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec034000 tx=0x7fb3ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:15.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.505+0000 7fb3fb400640 1 -- 172.21.15.59:0/4148198577 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4041f8ac0 con 0x7fb40406dcb0 2024-04-23T14:44:15.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.758+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb40404fd90 con 0x7fb40406dcb0 2024-04-23T14:44:15.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.830+0000 7fb3fb400640 1 -- 172.21.15.59:0/4148198577 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v102) ==== 162+0+0 (secure 0 0 0) 0x7fb3f4017020 con 0x7fb40406dcb0 2024-04-23T14:44:15.831 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb3e006c8f0 msgr2=0x7fb3e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb3e006c8f0 0x7fb3e006edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec034000 tx=0x7fb3ec02e000 comp rx=0 tx=0).stop 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 msgr2=0x7fb404161750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404161750 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e810 tx=0x7fb3f400ecd0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 shutdown_connections 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb3e006c8f0 0x7fb3e006edb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4040d09e0 0x7fb404150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4040d0600 0x7fb4041f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 --2- 172.21.15.59:0/4148198577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb40406dcb0 0x7fb404161750 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 >> 172.21.15.59:0/4148198577 conn(0x7fb40406bdf0 msgr2=0x7fb4040ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 shutdown_connections 2024-04-23T14:44:15.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:15.834+0000 7fb408e00640 1 -- 172.21.15.59:0/4148198577 wait complete. 2024-04-23T14:44:15.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:16.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.048+0000 7faad7000640 1 Processor -- start 2024-04-23T14:44:16.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.049+0000 7faad7000640 1 -- start start 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.050+0000 7faad7000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faad00b2660 0x7faad00b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.050+0000 7faad7000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad00b0a90 0x7faad00b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.050+0000 7faad7000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad01628f0 con 0x7faad006b9d0 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.050+0000 7faad7000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7faad0162bd0 con 0x7faad00b2660 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.050+0000 7faad7000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faad0162eb0 con 0x7faad00b0a90 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faacf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faad00b2660 0x7faad00b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faacf400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faad00b2660 0x7faad00b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41330/0 (socket says 172.21.15.59:41330) 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faacf400640 1 -- 172.21.15.59:0/3591518749 learned_addr learned my addr 172.21.15.59:0/3591518749 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:16.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faad5000640 1 --2- 172.21.15.59:0/3591518749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad00b0a90 0x7faad00b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:16.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faacf400640 1 -- 172.21.15.59:0/3591518749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad00b0a90 msgr2=0x7faad00b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:16.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faacf400640 1 --2- 172.21.15.59:0/3591518749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad00b0a90 0x7faad00b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faacf400640 1 -- 172.21.15.59:0/3591518749 >> v1:172.21.15.76:6789/0 conn(0x7faad006b9d0 legacy=0x7faad006bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:16.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.051+0000 7faacf400640 1 -- 172.21.15.59:0/3591518749 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad0163190 con 0x7faad00b2660 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.052+0000 7faacf400640 1 --2- 172.21.15.59:0/3591518749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b2660 0x7faad00b2aa0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7faab8009e50 tx=0x7faab802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0f394a3aba7f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faacea00640 1 -- 172.21.15.59:0/3591518749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8036070 con 0x7faad00b2660 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faacea00640 1 -- 172.21.15.59:0/3591518749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8031600 con 0x7faad00b2660 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faacea00640 1 -- 172.21.15.59:0/3591518749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80319d0 con 0x7faad00b2660 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faad7000640 1 -- 172.21.15.59:0/3591518749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b2660 msgr2=0x7faad00b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faad7000640 1 --2- 172.21.15.59:0/3591518749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b2660 0x7faad00b2aa0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7faab8009e50 tx=0x7faab802a680 comp rx=0 tx=0).stop 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faad7000640 1 -- 172.21.15.59:0/3591518749 shutdown_connections 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faad7000640 1 --2- 172.21.15.59:0/3591518749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad00b0a90 0x7faad00b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faad7000640 1 --2- 172.21.15.59:0/3591518749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b2660 0x7faad00b2aa0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.053+0000 7faad7000640 1 -- 172.21.15.59:0/3591518749 >> 172.21.15.59:0/3591518749 conn(0x7faad00733c0 msgr2=0x7faad009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:16.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.054+0000 7faad7000640 1 -- 172.21.15.59:0/3591518749 shutdown_connections 2024-04-23T14:44:16.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.054+0000 7faad7000640 1 -- 172.21.15.59:0/3591518749 wait complete. 2024-04-23T14:44:16.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.055+0000 7faad7000640 1 Processor -- start 2024-04-23T14:44:16.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.056+0000 7faad7000640 1 -- start start 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.056+0000 7faad7000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad006b9d0 0x7faad0155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faad7000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b0a90 0x7faad01610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faad7000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00b2660 0x7faad01615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faad7000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faad0162690 con 0x7faad00b0a90 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faad7000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faad0162510 con 0x7faad00b2660 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faad7000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faad00b1460 con 0x7faad006b9d0 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b0a90 0x7faad01610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b0a90 0x7faad01610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41346/0 (socket says 172.21.15.59:41346) 2024-04-23T14:44:16.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 -- 172.21.15.59:0/2616516444 learned_addr learned my addr 172.21.15.59:0/2616516444 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacfe00640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad006b9d0 0x7faad0155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faad5000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00b2660 0x7faad01615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 -- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad006b9d0 msgr2=0x7faad0155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad006b9d0 0x7faad0155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 -- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00b2660 msgr2=0x7faad01615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00b2660 0x7faad01615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faacf400640 1 -- 172.21.15.59:0/2616516444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad015eb10 con 0x7faad00b0a90 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.057+0000 7faad5000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00b2660 0x7faad01615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:16.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.058+0000 7faacf400640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b0a90 0x7faad01610a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7faab8034040 tx=0x7faab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.058+0000 7faac7e00640 1 -- 172.21.15.59:0/2616516444 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8041020 con 0x7faad00b0a90 2024-04-23T14:44:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.058+0000 7faac7e00640 1 -- 172.21.15.59:0/2616516444 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8007800 con 0x7faad00b0a90 2024-04-23T14:44:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.058+0000 7faac7e00640 1 -- 172.21.15.59:0/2616516444 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8036040 con 0x7faad00b0a90 2024-04-23T14:44:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.058+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad01f8410 con 0x7faad00b0a90 2024-04-23T14:44:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.058+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01f89d0 con 0x7faad00b0a90 2024-04-23T14:44:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.059+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad0069190 con 0x7faad00b0a90 2024-04-23T14:44:16.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.060+0000 7faac7e00640 1 -- 172.21.15.59:0/2616516444 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faab8044020 con 0x7faad00b0a90 2024-04-23T14:44:16.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.061+0000 7faac7e00640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaac06c8f0 0x7faaac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:16.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.062+0000 7faac7e00640 1 -- 172.21.15.59:0/2616516444 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7faab80ac9e0 con 0x7faad00b0a90 2024-04-23T14:44:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.062+0000 7faacfe00640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaac06c8f0 0x7faaac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:16.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.062+0000 7faacfe00640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaac06c8f0 0x7faaac06edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7faad01f81f0 tx=0x7faac0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:16.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.066+0000 7faac7e00640 1 -- 172.21.15.59:0/2616516444 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faab804b030 con 0x7faad00b0a90 2024-04-23T14:44:16.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.308+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad00cd050 con 0x7faad00b0a90 2024-04-23T14:44:16.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.309+0000 7faac7e00640 1 -- 172.21.15.59:0/2616516444 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5181 (secure 0 0 0) 0x7faab807ace0 con 0x7faad00b0a90 2024-04-23T14:44:16.311 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:16.311 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:15.834939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"c","rank":0,"incarnation":103,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/3383882209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3383882209},{"type":"v1","addr":"172.21.15.59:6837","nonce":3383882209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-23T14:44:16.311 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 103 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaac06c8f0 msgr2=0x7faaac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaac06c8f0 0x7faaac06edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7faad01f81f0 tx=0x7faac0009040 comp rx=0 tx=0).stop 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b0a90 msgr2=0x7faad01610a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b0a90 0x7faad01610a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7faab8034040 tx=0x7faab8004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 shutdown_connections 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faaac06c8f0 0x7faaac06edb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00b2660 0x7faad01615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faad00b0a90 0x7faad01610a0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 --2- 172.21.15.59:0/2616516444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faad006b9d0 0x7faad0155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:16.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.312+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 >> 172.21.15.59:0/2616516444 conn(0x7faad00733c0 msgr2=0x7faad00cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:16.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.313+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 shutdown_connections 2024-04-23T14:44:16.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:16.313+0000 7faad7000640 1 -- 172.21.15.59:0/2616516444 wait complete. 2024-04-23T14:44:16.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:43:37.939514+0000', 'modified': '2024-04-23T14:44:15.834939+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9522}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9522': {'gid': 9522, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/3383882209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 3383882209}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3383882209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:44:16.736 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9522:up:replay 2024-04-23T14:44:17.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.961+0000 7fdc2cc00640 1 Processor -- start 2024-04-23T14:44:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.961+0000 7fdc2cc00640 1 -- start start 2024-04-23T14:44:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.962+0000 7fdc2cc00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc280d14a0 0x7fdc280b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.962+0000 7fdc2cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc280cf030 con 0x7fdc280b5ff0 2024-04-23T14:44:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.962+0000 7fdc2cc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc280b6a40 con 0x7fdc2815e7c0 2024-04-23T14:44:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.962+0000 7fdc2cc00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdc280b7000 con 0x7fdc280d14a0 2024-04-23T14:44:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.962+0000 7fdc26a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdc280b5ff0 0x7fdc280d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37358/0 (socket says 172.21.15.59:37358) 2024-04-23T14:44:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.962+0000 7fdc26a00640 1 -- 172.21.15.59:0/3647196308 learned_addr learned my addr 172.21.15.59:0/3647196308 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:17.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc27e00640 1 --2- 172.21.15.59:0/3647196308 >> v2:172.21.15.76:3300/0 conn(0x7fdc280d14a0 0x7fdc280b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3604441557 0 0) 0x7fdc10001490 con 0x7fdc280b5ff0 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc0c003520 con 0x7fdc280b5ff0 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 349638744 0 0) 0x7fdc0c003520 con 0x7fdc280b5ff0 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 >> v2:172.21.15.76:3300/0 conn(0x7fdc280d14a0 msgr2=0x7fdc280b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 --2- 172.21.15.59:0/3647196308 >> v2:172.21.15.76:3300/0 conn(0x7fdc280d14a0 0x7fdc280b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 >> v1:172.21.15.76:6789/0 conn(0x7fdc2815e7c0 legacy=0x7fdc280d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc280b6d20 con 0x7fdc280b5ff0 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fdc10001bc0 con 0x7fdc280b5ff0 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 >> v1:172.21.15.59:6789/0 conn(0x7fdc280b5ff0 legacy=0x7fdc280d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc0c0039b0 0x7fdc0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.963+0000 7fdc26000640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c004800 0x7fdc0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc26000640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc0c0051c0 0x7fdc0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280b7000 con 0x7fdc0c0051c0 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c00c060 con 0x7fdc0c0039b0 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0c00c260 con 0x7fdc0c004800 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc27e00640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc0c0051c0 0x7fdc0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc27400640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc0c0039b0 0x7fdc0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc26a00640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c004800 0x7fdc0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc27e00640 1 -- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c004800 msgr2=0x7fdc0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc27e00640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c004800 0x7fdc0c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc27e00640 1 -- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc0c0039b0 msgr2=0x7fdc0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc27e00640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc0c0039b0 0x7fdc0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.964+0000 7fdc27e00640 1 -- 172.21.15.59:0/3647196308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c003520 con 0x7fdc0c0051c0 2024-04-23T14:44:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.965+0000 7fdc27e00640 1 -- 172.21.15.59:0/3647196308 reap_dead start 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.965+0000 7fdc27e00640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc0c0051c0 0x7fdc0c00b9c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009a90 tx=0x7fdc0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a865cc96c5458cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20009690 con 0x7fdc0c0051c0 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc20004020 con 0x7fdc0c0051c0 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc26000640 1 -- 172.21.15.59:0/3647196308 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20004410 con 0x7fdc0c0051c0 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc2cc00640 1 -- 172.21.15.59:0/3647196308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc0c0051c0 msgr2=0x7fdc0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc0c0051c0 0x7fdc0c00b9c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009a90 tx=0x7fdc0c005830 comp rx=0 tx=0).stop 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc2cc00640 1 -- 172.21.15.59:0/3647196308 shutdown_connections 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/3647196308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc0c0051c0 0x7fdc0c00b9c0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.966+0000 7fdc2cc00640 1 -- 172.21.15.59:0/3647196308 >> 172.21.15.59:0/3647196308 conn(0x7fdc2809d920 msgr2=0x7fdc2808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.967+0000 7fdc2cc00640 1 -- 172.21.15.59:0/3647196308 shutdown_connections 2024-04-23T14:44:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.967+0000 7fdc2cc00640 1 -- 172.21.15.59:0/3647196308 wait complete. 2024-04-23T14:44:17.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.968+0000 7fdc2cc00640 1 Processor -- start 2024-04-23T14:44:17.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.969+0000 7fdc2cc00640 1 -- start start 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc2cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c0051c0 0x7fdc280cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc2cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc280b9a40 0x7fdc2814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc2cc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc280b21f0 0x7fdc280b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc2cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280d1a80 con 0x7fdc280b9a40 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc2cc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280cf620 con 0x7fdc280b21f0 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc2cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdc280d1c00 con 0x7fdc0c0051c0 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc280b9a40 0x7fdc2814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc280b9a40 0x7fdc2814fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41366/0 (socket says 172.21.15.59:41366) 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc26a00640 1 -- 172.21.15.59:0/2369140688 learned_addr learned my addr 172.21.15.59:0/2369140688 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc27400640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c0051c0 0x7fdc280cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.970+0000 7fdc27e00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc280b21f0 0x7fdc280b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc26a00640 1 -- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c0051c0 msgr2=0x7fdc280cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc26a00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c0051c0 0x7fdc280cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc26a00640 1 -- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc280b21f0 msgr2=0x7fdc280b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc26a00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc280b21f0 0x7fdc280b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc26a00640 1 -- 172.21.15.59:0/2369140688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc281500d0 con 0x7fdc280b9a40 2024-04-23T14:44:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc26a00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc280b9a40 0x7fdc2814fb90 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000de80 tx=0x7fdc1000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:17.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc1f400640 1 -- 172.21.15.59:0/2369140688 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10019070 con 0x7fdc280b9a40 2024-04-23T14:44:17.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc1f400640 1 -- 172.21.15.59:0/2369140688 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10004d60 con 0x7fdc280b9a40 2024-04-23T14:44:17.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc280b89a0 con 0x7fdc280b9a40 2024-04-23T14:44:17.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc1f400640 1 -- 172.21.15.59:0/2369140688 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100072d0 con 0x7fdc280b9a40 2024-04-23T14:44:17.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.971+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc280b8f60 con 0x7fdc280b9a40 2024-04-23T14:44:17.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.973+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2804fd90 con 0x7fdc280b9a40 2024-04-23T14:44:17.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.974+0000 7fdc1f400640 1 -- 172.21.15.59:0/2369140688 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc1001c070 con 0x7fdc280b9a40 2024-04-23T14:44:17.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.975+0000 7fdc1f400640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc0406c920 0x7fdc0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:17.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.975+0000 7fdc1f400640 1 -- 172.21.15.59:0/2369140688 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fdc1008bcf0 con 0x7fdc280b9a40 2024-04-23T14:44:17.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.975+0000 7fdc27400640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc0406c920 0x7fdc0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:17.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.976+0000 7fdc27400640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc0406c920 0x7fdc0406ede0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdc18007fd0 tx=0x7fdc18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:17.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:17.979+0000 7fdc1f400640 1 -- 172.21.15.59:0/2369140688 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc280cf030 con 0x7fdc280b9a40 2024-04-23T14:44:18.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.222+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc280d05c0 con 0x7fdc280b9a40 2024-04-23T14:44:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.223+0000 7fdc1f400640 1 -- 172.21.15.59:0/2369140688 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5181 (secure 0 0 0) 0x7fdc1005a080 con 0x7fdc280b9a40 2024-04-23T14:44:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:18.225 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:16.856809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"c","rank":0,"incarnation":103,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.59:6837/3383882209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3383882209},{"type":"v1","addr":"172.21.15.59:6837","nonce":3383882209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-23T14:44:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 105 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.225+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc0406c920 msgr2=0x7fdc0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.225+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc0406c920 0x7fdc0406ede0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdc18007fd0 tx=0x7fdc18009040 comp rx=0 tx=0).stop 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.225+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc280b9a40 msgr2=0x7fdc2814fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.225+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc280b9a40 0x7fdc2814fb90 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000de80 tx=0x7fdc1000b4a0 comp rx=0 tx=0).stop 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 shutdown_connections 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fdc0406c920 0x7fdc0406ede0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc280b21f0 0x7fdc280b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc280b9a40 0x7fdc2814fb90 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 --2- 172.21.15.59:0/2369140688 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc0c0051c0 0x7fdc280cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:18.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 >> 172.21.15.59:0/2369140688 conn(0x7fdc2809d920 msgr2=0x7fdc280b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:18.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 shutdown_connections 2024-04-23T14:44:18.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:18.226+0000 7fdc2cc00640 1 -- 172.21.15.59:0/2369140688 wait complete. 2024-04-23T14:44:18.649 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:43:37.939514+0000', 'modified': '2024-04-23T14:44:16.856809+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9522}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9522': {'gid': 9522, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.59:6837/3383882209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 3383882209}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3383882209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:44:18.650 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9522:up:rejoin 2024-04-23T14:44:19.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:19.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.869+0000 7f73aa000640 1 Processor -- start 2024-04-23T14:44:19.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.869+0000 7f73aa000640 1 -- start start 2024-04-23T14:44:19.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.870+0000 7f73aa000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73a415e7c0 0x7f73a40d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:19.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.870+0000 7f73aa000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a40cf030 con 0x7f73a40d2630 2024-04-23T14:44:19.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.870+0000 7f73aa000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a40b6a40 con 0x7f73a40d0d90 2024-04-23T14:44:19.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.870+0000 7f73aa000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f73a40b7000 con 0x7f73a415e7c0 2024-04-23T14:44:19.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.871+0000 7f73a3e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f73a40d2630 0x7f73a40b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:37370/0 (socket says 172.21.15.59:37370) 2024-04-23T14:44:19.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.871+0000 7f73a3e00640 1 -- 172.21.15.59:0/1543426334 learned_addr learned my addr 172.21.15.59:0/1543426334 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:19.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.871+0000 7f73a8c00640 1 --2- 172.21.15.59:0/1543426334 >> v2:172.21.15.76:3300/0 conn(0x7f73a415e7c0 0x7f73a40d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:19.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.872+0000 7f73a8c00640 1 -- 172.21.15.59:0/1543426334 >> v1:172.21.15.76:6789/0 conn(0x7f73a40d0d90 legacy=0x7f73a40cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:19.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.872+0000 7f73a8c00640 1 -- 172.21.15.59:0/1543426334 >> v1:172.21.15.59:6789/0 conn(0x7f73a40d2630 legacy=0x7f73a40b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:19.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.872+0000 7f73a8c00640 1 -- 172.21.15.59:0/1543426334 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a40b6d20 con 0x7f73a415e7c0 2024-04-23T14:44:19.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.874+0000 7f73a8c00640 1 --2- 172.21.15.59:0/1543426334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a415e7c0 0x7f73a40d20f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f738c0097a0 tx=0x7f738c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe21583185c8a99e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:19.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73a3400640 1 -- 172.21.15.59:0/1543426334 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c037070 con 0x7f73a415e7c0 2024-04-23T14:44:19.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73a3400640 1 -- 172.21.15.59:0/1543426334 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738c032440 con 0x7f73a415e7c0 2024-04-23T14:44:19.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73a3400640 1 -- 172.21.15.59:0/1543426334 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c032810 con 0x7f73a415e7c0 2024-04-23T14:44:19.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73aa000640 1 -- 172.21.15.59:0/1543426334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a415e7c0 msgr2=0x7f73a40d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:19.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73aa000640 1 --2- 172.21.15.59:0/1543426334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a415e7c0 0x7f73a40d20f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f738c0097a0 tx=0x7f738c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:44:19.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73aa000640 1 -- 172.21.15.59:0/1543426334 shutdown_connections 2024-04-23T14:44:19.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73aa000640 1 --2- 172.21.15.59:0/1543426334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a415e7c0 0x7f73a40d20f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:19.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.877+0000 7f73aa000640 1 -- 172.21.15.59:0/1543426334 >> 172.21.15.59:0/1543426334 conn(0x7f73a409d920 msgr2=0x7f73a408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:19.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.878+0000 7f73aa000640 1 -- 172.21.15.59:0/1543426334 shutdown_connections 2024-04-23T14:44:19.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.878+0000 7f73aa000640 1 -- 172.21.15.59:0/1543426334 wait complete. 2024-04-23T14:44:19.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.879+0000 7f73aa000640 1 Processor -- start 2024-04-23T14:44:19.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.880+0000 7f73aa000640 1 -- start start 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.880+0000 7f73aa000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73a40d0d90 0x7f73a4157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.880+0000 7f73aa000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a40d2630 0x7f73a4157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.880+0000 7f73aa000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a415e7c0 0x7f73a41f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.880+0000 7f73aa000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73a40b6700 con 0x7f73a415e7c0 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.880+0000 7f73aa000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f73a40cf620 con 0x7f73a40d2630 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.880+0000 7f73aa000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f73a40d1a80 con 0x7f73a40d0d90 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a415e7c0 0x7f73a41f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a415e7c0 0x7f73a41f7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41380/0 (socket says 172.21.15.59:41380) 2024-04-23T14:44:19.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 -- 172.21.15.59:0/3882423788 learned_addr learned my addr 172.21.15.59:0/3882423788 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a8c00640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73a40d0d90 0x7f73a4157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a3e00640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a40d2630 0x7f73a4157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 -- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73a40d0d90 msgr2=0x7f73a4157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73a40d0d90 0x7f73a4157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 -- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a40d2630 msgr2=0x7f73a4157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a40d2630 0x7f73a4157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.881+0000 7f73a9600640 1 -- 172.21.15.59:0/3882423788 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a41f8a00 con 0x7f73a415e7c0 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.882+0000 7f73a9600640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a415e7c0 0x7f73a41f7b60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c780 tx=0x7f739c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:19.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.882+0000 7f73a0c00640 1 -- 172.21.15.59:0/3882423788 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c004b60 con 0x7f73a415e7c0 2024-04-23T14:44:19.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.882+0000 7f73a0c00640 1 -- 172.21.15.59:0/3882423788 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f739c004d00 con 0x7f73a415e7c0 2024-04-23T14:44:19.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.882+0000 7f73a0c00640 1 -- 172.21.15.59:0/3882423788 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c00aa90 con 0x7f73a415e7c0 2024-04-23T14:44:19.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.882+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a40cc0a0 con 0x7f73a415e7c0 2024-04-23T14:44:19.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.882+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a40cc660 con 0x7f73a415e7c0 2024-04-23T14:44:19.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.883+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a40c9660 con 0x7f73a415e7c0 2024-04-23T14:44:19.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.884+0000 7f73a0c00640 1 -- 172.21.15.59:0/3882423788 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f739c00ac30 con 0x7f73a415e7c0 2024-04-23T14:44:19.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.885+0000 7f73a0c00640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f738006c8f0 0x7f738006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:19.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.885+0000 7f73a0c00640 1 -- 172.21.15.59:0/3882423788 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f739c08c210 con 0x7f73a415e7c0 2024-04-23T14:44:19.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.886+0000 7f73a8c00640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f738006c8f0 0x7f738006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:19.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.886+0000 7f73a8c00640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f738006c8f0 0x7f738006edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f738c034000 tx=0x7f738c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:19.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:19.890+0000 7f73a0c00640 1 -- 172.21.15.59:0/3882423788 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f739c05a510 con 0x7f73a415e7c0 2024-04-23T14:44:20.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.130+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73a404fd90 con 0x7f73a415e7c0 2024-04-23T14:44:20.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.131+0000 7f73a0c00640 1 -- 172.21.15.59:0/3882423788 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5188 (secure 0 0 0) 0x7f739c059e10 con 0x7f73a415e7c0 2024-04-23T14:44:20.133 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:20.133 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:18.865935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3383882209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3383882209},{"type":"v1","addr":"172.21.15.59:6837","nonce":3383882209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9522,"qdb_cluster":[9522]},"id":8}]} 2024-04-23T14:44:20.134 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 106 2024-04-23T14:44:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f738006c8f0 msgr2=0x7f738006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f738006c8f0 0x7f738006edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f738c034000 tx=0x7f738c02e000 comp rx=0 tx=0).stop 2024-04-23T14:44:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a415e7c0 msgr2=0x7f73a41f7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a415e7c0 0x7f73a41f7b60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c780 tx=0x7f739c00cc40 comp rx=0 tx=0).stop 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 shutdown_connections 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f738006c8f0 0x7f738006edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a415e7c0 0x7f73a41f7b60 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73a40d2630 0x7f73a4157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 --2- 172.21.15.59:0/3882423788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73a40d0d90 0x7f73a4157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 >> 172.21.15.59:0/3882423788 conn(0x7f73a409d920 msgr2=0x7f73a415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 shutdown_connections 2024-04-23T14:44:20.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.134+0000 7f73aa000640 1 -- 172.21.15.59:0/3882423788 wait complete. 2024-04-23T14:44:20.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:43:37.939514+0000', 'modified': '2024-04-23T14:44:18.865935+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9522}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9522': {'gid': 9522, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6837/3383882209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 3383882209}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3383882209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9522, 'qdb_cluster': [9522]} 2024-04-23T14:44:20.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:44:20.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:44:20.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.774+0000 7f64a8c00640 1 Processor -- start 2024-04-23T14:44:20.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.774+0000 7f64a8c00640 1 -- start start 2024-04-23T14:44:20.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.775+0000 7f64a8c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64a415aca0 0x7f64a4071e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.775+0000 7f64a8c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f64a40b45d0 0x7f64a415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a8c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a415f800 con 0x7f64a40cc920 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a8c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f64a40b4c30 con 0x7f64a415aca0 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a8c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f64a40b4f10 con 0x7f64a40b45d0 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64a415aca0 0x7f64a4071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a3400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64a415aca0 0x7f64a4071e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48096/0 (socket says 172.21.15.59:48096) 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a3400640 1 -- 172.21.15.59:0/2452591190 learned_addr learned my addr 172.21.15.59:0/2452591190 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f649be00640 1 --2- 172.21.15.59:0/2452591190 >> v2:172.21.15.76:3300/0 conn(0x7f64a40b45d0 0x7f64a415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a3400640 1 -- 172.21.15.59:0/2452591190 >> v2:172.21.15.76:3300/0 conn(0x7f64a40b45d0 msgr2=0x7f64a415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a3400640 1 --2- 172.21.15.59:0/2452591190 >> v2:172.21.15.76:3300/0 conn(0x7f64a40b45d0 0x7f64a415f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a3400640 1 -- 172.21.15.59:0/2452591190 >> v1:172.21.15.76:6789/0 conn(0x7f64a40cc920 legacy=0x7f64a40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f64a3400640 1 -- 172.21.15.59:0/2452591190 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a4073b50 con 0x7f64a415aca0 2024-04-23T14:44:20.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.776+0000 7f649be00640 1 --2- 172.21.15.59:0/2452591190 >> v2:172.21.15.76:3300/0 conn(0x7f64a40b45d0 0x7f64a415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.777+0000 7f64a3400640 1 --2- 172.21.15.59:0/2452591190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a415aca0 0x7f64a4071e70 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f64a415f9d0 tx=0x7f649402c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1125694f1b2ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.778+0000 7f64a2a00640 1 -- 172.21.15.59:0/2452591190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649403a070 con 0x7f64a415aca0 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.778+0000 7f64a2a00640 1 -- 172.21.15.59:0/2452591190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494032480 con 0x7f64a415aca0 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.778+0000 7f64a2a00640 1 -- 172.21.15.59:0/2452591190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494032650 con 0x7f64a415aca0 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.778+0000 7f64a8c00640 1 -- 172.21.15.59:0/2452591190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a415aca0 msgr2=0x7f64a4071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.778+0000 7f64a8c00640 1 --2- 172.21.15.59:0/2452591190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a415aca0 0x7f64a4071e70 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f64a415f9d0 tx=0x7f649402c620 comp rx=0 tx=0).stop 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.779+0000 7f64a8c00640 1 -- 172.21.15.59:0/2452591190 shutdown_connections 2024-04-23T14:44:20.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.779+0000 7f64a8c00640 1 --2- 172.21.15.59:0/2452591190 >> v2:172.21.15.76:3300/0 conn(0x7f64a40b45d0 0x7f64a415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.779+0000 7f64a8c00640 1 --2- 172.21.15.59:0/2452591190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a415aca0 0x7f64a4071e70 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.779+0000 7f64a8c00640 1 -- 172.21.15.59:0/2452591190 >> 172.21.15.59:0/2452591190 conn(0x7f64a406bdf0 msgr2=0x7f64a406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.779+0000 7f64a8c00640 1 -- 172.21.15.59:0/2452591190 shutdown_connections 2024-04-23T14:44:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.779+0000 7f64a8c00640 1 -- 172.21.15.59:0/2452591190 wait complete. 2024-04-23T14:44:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.780+0000 7f64a8c00640 1 Processor -- start 2024-04-23T14:44:20.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.781+0000 7f64a8c00640 1 -- start start 2024-04-23T14:44:20.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.781+0000 7f64a8c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64a40b45d0 0x7f64a40b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a8c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a40cc920 0x7f64a41501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a8c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64a415aca0 0x7f64a414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a8c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4073800 con 0x7f64a40cc920 2024-04-23T14:44:20.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a8c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f64a415b080 con 0x7f64a40b45d0 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a8c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f64a40ccd70 con 0x7f64a415aca0 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a40cc920 0x7f64a41501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a40cc920 0x7f64a41501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48104/0 (socket says 172.21.15.59:48104) 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 -- 172.21.15.59:0/1764027492 learned_addr learned my addr 172.21.15.59:0/1764027492 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a3400640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64a40b45d0 0x7f64a40b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 -- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64a415aca0 msgr2=0x7f64a414f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a3e00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64a415aca0 0x7f64a414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64a415aca0 0x7f64a414f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 -- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64a40b45d0 msgr2=0x7f64a40b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64a40b45d0 0x7f64a40b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f649be00640 1 -- 172.21.15.59:0/1764027492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a414fc70 con 0x7f64a40cc920 2024-04-23T14:44:20.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.782+0000 7f64a3400640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64a40b45d0 0x7f64a40b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.783+0000 7f649be00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a40cc920 0x7f64a41501a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c7f0 tx=0x7f648c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.783+0000 7f649b400640 1 -- 172.21.15.59:0/1764027492 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c015070 con 0x7f64a40cc920 2024-04-23T14:44:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.783+0000 7f649b400640 1 -- 172.21.15.59:0/1764027492 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c007a30 con 0x7f64a40cc920 2024-04-23T14:44:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.783+0000 7f649b400640 1 -- 172.21.15.59:0/1764027492 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c007e00 con 0x7f64a40cc920 2024-04-23T14:44:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.783+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a41508f0 con 0x7f64a40cc920 2024-04-23T14:44:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.783+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a4150eb0 con 0x7f64a40cc920 2024-04-23T14:44:20.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.785+0000 7f649b400640 1 -- 172.21.15.59:0/1764027492 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f648c018070 con 0x7f64a40cc920 2024-04-23T14:44:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.786+0000 7f649b400640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f648006c8f0 0x7f648006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.786+0000 7f649b400640 1 -- 172.21.15.59:0/1764027492 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f648c08bb60 con 0x7f64a40cc920 2024-04-23T14:44:20.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.786+0000 7f64a3400640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f648006c8f0 0x7f648006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.787+0000 7f64ad183740 1 Processor -- start 2024-04-23T14:44:20.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.787+0000 7f64a3400640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f648006c8f0 0x7f648006edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6494037000 tx=0x7f6494030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:20.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.787+0000 7f64ad183740 1 -- start start 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.788+0000 7f64ad183740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x556895835b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.788+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895865630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.788+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895865b70 0x55689583a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.788+0000 7f64ad183740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556895797b10 con 0x556895883190 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.788+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55689573e2b0 con 0x5568958662b0 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.788+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556895793ad0 con 0x556895865b70 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.788+0000 7f64a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x556895835b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x556895835b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48116/0 (socket says 172.21.15.59:48116) 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 -- 172.21.15.59:0/3560211517 learned_addr learned my addr 172.21.15.59:0/3560211517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:20.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f649be00640 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895865630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3e00640 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895865b70 0x55689583a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 -- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895865b70 msgr2=0x55689583a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895865b70 0x55689583a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 -- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 msgr2=0x556895865630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895865630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 -- 172.21.15.59:0/3560211517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568954aa2d0 con 0x556895883190 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3e00640 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895865b70 0x55689583a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.789+0000 7f64a3400640 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x556895835b20 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6494032230 tx=0x7f6494032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2170204a0f923 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f6493400640 1 -- 172.21.15.59:0/3560211517 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649403a040 con 0x556895883190 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f6493400640 1 -- 172.21.15.59:0/3560211517 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649404f070 con 0x556895883190 2024-04-23T14:44:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f6493400640 1 -- 172.21.15.59:0/3560211517 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494040870 con 0x556895883190 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 -- 172.21.15.59:0/3560211517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 msgr2=0x556895835b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x556895835b20 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6494032230 tx=0x7f6494032000 comp rx=0 tx=0).stop 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 -- 172.21.15.59:0/3560211517 shutdown_connections 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895865b70 0x55689583a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895865630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 --2- 172.21.15.59:0/3560211517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x556895835b20 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 -- 172.21.15.59:0/3560211517 >> 172.21.15.59:0/3560211517 conn(0x556895837d90 msgr2=0x556895880630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 -- 172.21.15.59:0/3560211517 shutdown_connections 2024-04-23T14:44:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.790+0000 7f64ad183740 1 -- 172.21.15.59:0/3560211517 wait complete. 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.791+0000 7f64ad183740 1 -- start start 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.791+0000 7f64ad183740 1 Processor -- start 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.792+0000 7f64ad183740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568958662b0 0x556895868420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.792+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895883190 0x556895880630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.792+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x556895880b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.792+0000 7f64ad183740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556895839f40 con 0x5568958662b0 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.792+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556895751590 con 0x556895883190 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.792+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556895839330 con 0x556895868e40 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.792+0000 7f64ad183740 10 client.0 _ll_register_callbacks cb 0x5568957343a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568958662b0 0x556895868420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568958662b0 0x556895868420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48120/0 (socket says 172.21.15.59:48120) 2024-04-23T14:44:20.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 -- 172.21.15.59:0/2878509511 learned_addr learned my addr 172.21.15.59:0/2878509511 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f649be00640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895883190 0x556895880630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3e00640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x556895880b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 -- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 msgr2=0x556895880b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x556895880b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 -- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895883190 msgr2=0x556895880630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895883190 0x556895880630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 -- 172.21.15.59:0/2878509511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568955746b0 con 0x5568958662b0 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f649be00640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895883190 0x556895880630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f64a3400640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568958662b0 0x556895868420 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f649400bbc0 tx=0x7f6494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89d21bd25b231c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.793+0000 7f6487e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:20.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f6487e00640 1 -- 172.21.15.59:0/2878509511 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494040920 con 0x5568958662b0 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f6487e00640 1 -- 172.21.15.59:0/2878509511 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f649405eb30 con 0x5568958662b0 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f649404f070 con 0x5568958662b0 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f6487e00640 1 -- 172.21.15.59:0/2878509511 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f6494065050 con 0x5568958662b0 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 10 client.9567 fetch_fsmap learned FSMap version 106 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x556895893d80 con 0x5568958662b0 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f6487e00640 1 -- 172.21.15.59:0/2878509511 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649404f070 con 0x5568958662b0 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f6487e00640 1 -- 172.21.15.59:0/2878509511 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f6494068020 con 0x5568958662b0 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 10 client.9567 fetch_fsmap finished waiting for FSMap version 106 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 10 client.9567 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 20 client.9567 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958779d0 0x556895877db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 4 client.9567 mds_command: new command op to 9522 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.794+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 --> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568957a5530 con 0x5568958779d0 2024-04-23T14:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.795+0000 7f649be00640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958779d0 0x556895877db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.795+0000 7f649be00640 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958779d0 0x556895877db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44f355b0735b309 server_cookie=cebddf026c0422c7 in_seq=0 out_seq=0 2024-04-23T14:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.795+0000 7f6487e00640 10 client.9567 ms_handle_connect on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f6487e00640 1 -- 172.21.15.59:0/2878509511 <== mds.0 v2:172.21.15.59:6836/3383882209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568957a5530 con 0x5568958779d0 2024-04-23T14:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f6487e00640 10 client.9567 handle_command_reply: tid=0 2024-04-23T14:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 client.9567 shutdown 2024-04-23T14:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568958662b0 msgr2=0x556895868420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568958662b0 0x556895868420 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f649400bbc0 tx=0x7f6494004730 comp rx=0 tx=0).stop 2024-04-23T14:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 shutdown_connections 2024-04-23T14:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958779d0 0x556895877db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x556895880b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895883190 0x556895880630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 --2- 172.21.15.59:0/2878509511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5568958662b0 0x556895868420 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f6487e00640 0 client.9567 ms_handle_reset on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:20.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 >> 172.21.15.59:0/2878509511 conn(0x556895865b70 msgr2=0x5568958827d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:20.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 shutdown_connections 2024-04-23T14:44:20.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.796+0000 7f64ad183740 1 -- 172.21.15.59:0/2878509511 wait complete. 2024-04-23T14:44:20.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.797+0000 7f64ad183740 20 client.9567 trim_cache size 0 max 16384 2024-04-23T14:44:20.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.816+0000 7f64ad183740 1 Processor -- start 2024-04-23T14:44:20.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.816+0000 7f64ad183740 1 -- start start 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.817+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5568958662b0 0x556895836860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.817+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895868e40 0x556895839870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.817+0000 7f64ad183740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x55689582c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.817+0000 7f64ad183740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64a40c64e0 con 0x556895883190 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.817+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4152930 con 0x556895868e40 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.817+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f64a4152f80 con 0x5568958662b0 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x55689582c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x55689582c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48130/0 (socket says 172.21.15.59:48130) 2024-04-23T14:44:20.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 -- 172.21.15.59:0/104401142 learned_addr learned my addr 172.21.15.59:0/104401142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3400640 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5568958662b0 0x556895836860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f649be00640 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895868e40 0x556895839870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 -- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5568958662b0 msgr2=0x556895836860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5568958662b0 0x556895836860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 -- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895868e40 msgr2=0x556895839870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895868e40 0x556895839870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 -- 172.21.15.59:0/104401142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568958360e0 con 0x556895883190 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f649be00640 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895868e40 0x556895839870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.818+0000 7f64a3e00640 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x55689582c500 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f649c0042d0 tx=0x7f649c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91f6a25166868c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f6492a00640 1 -- 172.21.15.59:0/104401142 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00fce0 con 0x556895883190 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f6492a00640 1 -- 172.21.15.59:0/104401142 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649c004920 con 0x556895883190 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f6492a00640 1 -- 172.21.15.59:0/104401142 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00fea0 con 0x556895883190 2024-04-23T14:44:20.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 -- 172.21.15.59:0/104401142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 msgr2=0x55689582c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x55689582c500 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f649c0042d0 tx=0x7f649c004790 comp rx=0 tx=0).stop 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 -- 172.21.15.59:0/104401142 shutdown_connections 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x55689582c500 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556895868e40 0x556895839870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 --2- 172.21.15.59:0/104401142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5568958662b0 0x556895836860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 -- 172.21.15.59:0/104401142 >> 172.21.15.59:0/104401142 conn(0x556895865b70 msgr2=0x5568958629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.819+0000 7f64ad183740 1 -- 172.21.15.59:0/104401142 shutdown_connections 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.820+0000 7f64ad183740 1 -- 172.21.15.59:0/104401142 wait complete. 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.820+0000 7f64ad183740 1 -- start start 2024-04-23T14:44:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.820+0000 7f64ad183740 1 Processor -- start 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.821+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895861cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.821+0000 7f64ad183740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x5568958667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.821+0000 7f64ad183740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x5568958671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.821+0000 7f64ad183740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556895839f40 con 0x556895883190 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.821+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556895839330 con 0x5568958662b0 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.821+0000 7f64ad183740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556895751590 con 0x556895868e40 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.821+0000 7f64ad183740 10 client.0 _ll_register_callbacks cb 0x7f6494006da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x5568958671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x5568958671b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48146/0 (socket says 172.21.15.59:48146) 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 -- 172.21.15.59:0/10323992 learned_addr learned my addr 172.21.15.59:0/10323992 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f649be00640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x5568958667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 -- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 msgr2=0x5568958667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3400640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895861cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x5568958667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 -- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 msgr2=0x556895861cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895861cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 -- 172.21.15.59:0/10323992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568958815f0 con 0x556895883190 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f649be00640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x5568958667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f64a3e00640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x5568958671b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f649c00f600 tx=0x7f649c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4270a322d80b1a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.822+0000 7f6487e00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f6487e00640 1 -- 172.21.15.59:0/10323992 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00fdc0 con 0x556895883190 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f6487e00640 1 -- 172.21.15.59:0/10323992 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f649c027b60 con 0x556895883190 2024-04-23T14:44:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f649c004920 con 0x556895883190 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f6487e00640 1 -- 172.21.15.59:0/10323992 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f649c027d80 con 0x556895883190 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 10 client.9573 fetch_fsmap learned FSMap version 106 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x556895860e30 con 0x556895883190 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f6487e00640 1 -- 172.21.15.59:0/10323992 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649c004920 con 0x556895883190 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f6487e00640 1 -- 172.21.15.59:0/10323992 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f649c02e050 con 0x556895883190 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 10 client.9573 fetch_fsmap finished waiting for FSMap version 106 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 10 client.9573 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 20 client.9573 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958f97b0 0x55689584f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 4 client.9573 mds_command: new command op to 9522 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.823+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 --> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] -- command(tid 0: {"prefix": "status"}) -- 0x556895884ad0 con 0x5568958f97b0 2024-04-23T14:44:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.824+0000 7f64a3400640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958f97b0 0x55689584f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.824+0000 7f64a3400640 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958f97b0 0x55689584f470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c03e163bd7563561 server_cookie=eb5ea93d79fb27d8 in_seq=0 out_seq=0 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.824+0000 7f6487e00640 10 client.9573 ms_handle_connect on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.824+0000 7f6487e00640 1 -- 172.21.15.59:0/10323992 <== mds.0 v2:172.21.15.59:6836/3383882209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x556895884ad0 con 0x5568958f97b0 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.824+0000 7f6487e00640 10 client.9573 handle_command_reply: tid=0 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.824+0000 7f64ad183740 1 client.9573 shutdown 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 msgr2=0x5568958671b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x5568958671b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f649c00f600 tx=0x7f649c003a70 comp rx=0 tx=0).stop 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 shutdown_connections 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x5568958f97b0 0x55689584f470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556895868e40 0x5568958667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556895883190 0x5568958671b0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 --2- 172.21.15.59:0/10323992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5568958662b0 0x556895861cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f6487e00640 0 client.9573 ms_handle_reset on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 >> 172.21.15.59:0/10323992 conn(0x556895865b70 msgr2=0x556895892d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 shutdown_connections 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 1 -- 172.21.15.59:0/10323992 wait complete. 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.825+0000 7f64ad183740 20 client.9573 trim_cache size 0 max 16384 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9522, 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:44:20.832 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:44:20.833 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:44:20.833 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 4.9833400110000001, 2024-04-23T14:44:20.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 106, 2024-04-23T14:44:20.833 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 115, 2024-04-23T14:44:20.833 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 115, 2024-04-23T14:44:20.833 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 5.9692936059999999 2024-04-23T14:44:20.833 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:44:20.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f648006c8f0 msgr2=0x7f648006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f648006c8f0 0x7f648006edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6494037000 tx=0x7f6494030000 comp rx=0 tx=0).stop 2024-04-23T14:44:20.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a40cc920 msgr2=0x7f64a41501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:20.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a40cc920 0x7f64a41501a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c7f0 tx=0x7f648c00cbf0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 shutdown_connections 2024-04-23T14:44:20.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f648006c8f0 0x7f648006edb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64a415aca0 0x7f64a414f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64a40cc920 0x7f64a41501a0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 --2- 172.21.15.59:0/1764027492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64a40b45d0 0x7f64a40b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:20.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 >> 172.21.15.59:0/1764027492 conn(0x7f64a406bdf0 msgr2=0x7f64a40b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:20.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 shutdown_connections 2024-04-23T14:44:20.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:20.826+0000 7f64a8c00640 1 -- 172.21.15.59:0/1764027492 wait complete. 2024-04-23T14:44:20.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:44:21.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.030+0000 7f7e4fa00640 1 Processor -- start 2024-04-23T14:44:21.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.030+0000 7f7e4fa00640 1 -- start start 2024-04-23T14:44:21.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.031+0000 7f7e4fa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e4815c5e0 0x7f7e480af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:21.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4fa00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7e480af920 0x7f7e480d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4fa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e480d06e0 0x7f7e48071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4fa00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7e480afd80 con 0x7f7e480d06e0 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4fa00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7e4815fa20 con 0x7f7e480af920 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4fa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4815f0f0 con 0x7f7e4815c5e0 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e480d06e0 0x7f7e48071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e480d06e0 0x7f7e48071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48160/0 (socket says 172.21.15.59:48160) 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 -- 172.21.15.59:0/556928389 learned_addr learned my addr 172.21.15.59:0/556928389 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4d000640 1 --2- 172.21.15.59:0/556928389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e4815c5e0 0x7f7e480af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e47e00640 1 --2- 172.21.15.59:0/556928389 >> v2:172.21.15.76:3300/0 conn(0x7f7e480af920 0x7f7e480d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 -- 172.21.15.59:0/556928389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e4815c5e0 msgr2=0x7f7e480af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:21.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 --2- 172.21.15.59:0/556928389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e4815c5e0 0x7f7e480af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 -- 172.21.15.59:0/556928389 >> v2:172.21.15.76:3300/0 conn(0x7f7e480af920 msgr2=0x7f7e480d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:21.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 --2- 172.21.15.59:0/556928389 >> v2:172.21.15.76:3300/0 conn(0x7f7e480af920 0x7f7e480d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e4da00640 1 -- 172.21.15.59:0/556928389 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4815f3d0 con 0x7f7e480d06e0 2024-04-23T14:44:21.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.032+0000 7f7e47e00640 1 --2- 172.21.15.59:0/556928389 >> v2:172.21.15.76:3300/0 conn(0x7f7e480af920 0x7f7e480d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:21.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.033+0000 7f7e4da00640 1 --2- 172.21.15.59:0/556928389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e480d06e0 0x7f7e48071de0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7e4806c790 tx=0x7f7e4002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdd227e1e958610 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:21.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.034+0000 7f7e47400640 1 -- 172.21.15.59:0/556928389 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40036070 con 0x7f7e480d06e0 2024-04-23T14:44:21.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.034+0000 7f7e47400640 1 -- 172.21.15.59:0/556928389 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e40031600 con 0x7f7e480d06e0 2024-04-23T14:44:21.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.034+0000 7f7e47400640 1 -- 172.21.15.59:0/556928389 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400319d0 con 0x7f7e480d06e0 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.034+0000 7f7e4fa00640 1 -- 172.21.15.59:0/556928389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e480d06e0 msgr2=0x7f7e48071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.034+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/556928389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e480d06e0 0x7f7e48071de0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7e4806c790 tx=0x7f7e4002a6a0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.035+0000 7f7e4fa00640 1 -- 172.21.15.59:0/556928389 shutdown_connections 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.035+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/556928389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e480d06e0 0x7f7e48071de0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.035+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/556928389 >> v2:172.21.15.76:3300/0 conn(0x7f7e480af920 0x7f7e480d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.035+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/556928389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e4815c5e0 0x7f7e480af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.035+0000 7f7e4fa00640 1 -- 172.21.15.59:0/556928389 >> 172.21.15.59:0/556928389 conn(0x7f7e48067ea0 msgr2=0x7f7e4806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.035+0000 7f7e4fa00640 1 -- 172.21.15.59:0/556928389 shutdown_connections 2024-04-23T14:44:21.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.035+0000 7f7e4fa00640 1 -- 172.21.15.59:0/556928389 wait complete. 2024-04-23T14:44:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.036+0000 7f7e4fa00640 1 Processor -- start 2024-04-23T14:44:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.037+0000 7f7e4fa00640 1 -- start start 2024-04-23T14:44:21.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.037+0000 7f7e4fa00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e480af920 0x7f7e48151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:21.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.037+0000 7f7e4fa00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e480d06e0 0x7f7e48155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:21.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4fa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e4815c5e0 0x7f7e48153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4fa00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e480d0cf0 con 0x7f7e4815c5e0 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4fa00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7e480d0b70 con 0x7f7e480d06e0 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4fa00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4806e480 con 0x7f7e480af920 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e4815c5e0 0x7f7e48153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e4815c5e0 0x7f7e48153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48166/0 (socket says 172.21.15.59:48166) 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 -- 172.21.15.59:0/20310698 learned_addr learned my addr 172.21.15.59:0/20310698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4d000640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e480af920 0x7f7e48151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e47e00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e480d06e0 0x7f7e48155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 -- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e480af920 msgr2=0x7f7e48151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e480af920 0x7f7e48151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 -- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e480d06e0 msgr2=0x7f7e48155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e480d06e0 0x7f7e48155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e4da00640 1 -- 172.21.15.59:0/20310698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e48158270 con 0x7f7e4815c5e0 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.038+0000 7f7e47e00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e480d06e0 0x7f7e48155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.039+0000 7f7e4da00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e4815c5e0 0x7f7e48153c30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009f40 tx=0x7f7e40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.039+0000 7f7e44c00640 1 -- 172.21.15.59:0/20310698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40036050 con 0x7f7e4815c5e0 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.039+0000 7f7e44c00640 1 -- 172.21.15.59:0/20310698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e40042070 con 0x7f7e4815c5e0 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.039+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e48158510 con 0x7f7e4815c5e0 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.039+0000 7f7e44c00640 1 -- 172.21.15.59:0/20310698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40009510 con 0x7f7e4815c5e0 2024-04-23T14:44:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.039+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e48158ad0 con 0x7f7e4815c5e0 2024-04-23T14:44:21.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.040+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e480b20a0 con 0x7f7e4815c5e0 2024-04-23T14:44:21.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.041+0000 7f7e44c00640 1 -- 172.21.15.59:0/20310698 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e40007bd0 con 0x7f7e4815c5e0 2024-04-23T14:44:21.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.042+0000 7f7e44c00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e2806c8f0 0x7f7e2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:21.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.042+0000 7f7e44c00640 1 -- 172.21.15.59:0/20310698 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f7e400acae0 con 0x7f7e4815c5e0 2024-04-23T14:44:21.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.043+0000 7f7e4d000640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e2806c8f0 0x7f7e2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:21.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.043+0000 7f7e4d000640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e2806c8f0 0x7f7e2806edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e34006fd0 tx=0x7f7e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:21.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.047+0000 7f7e44c00640 1 -- 172.21.15.59:0/20310698 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e480b20a0 con 0x7f7e4815c5e0 2024-04-23T14:44:21.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.287+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7e48068a80 con 0x7f7e4815c5e0 2024-04-23T14:44:21.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.288+0000 7f7e44c00640 1 -- 172.21.15.59:0/20310698 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v106) ==== 53+0+73 (secure 0 0 0) 0x7f7e40045360 con 0x7f7e4815c5e0 2024-04-23T14:44:21.290 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e2806c8f0 msgr2=0x7f7e2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e2806c8f0 0x7f7e2806edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e34006fd0 tx=0x7f7e34008040 comp rx=0 tx=0).stop 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e4815c5e0 msgr2=0x7f7e48153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e4815c5e0 0x7f7e48153c30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009f40 tx=0x7f7e40004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 shutdown_connections 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7e2806c8f0 0x7f7e2806edb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e4815c5e0 0x7f7e48153c30 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e480d06e0 0x7f7e48155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 --2- 172.21.15.59:0/20310698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7e480af920 0x7f7e48151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 >> 172.21.15.59:0/20310698 conn(0x7f7e48067ea0 msgr2=0x7f7e4815b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 shutdown_connections 2024-04-23T14:44:21.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21.291+0000 7f7e4fa00640 1 -- 172.21.15.59:0/20310698 wait complete. 2024-04-23T14:44:21.311 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:44:21.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:44:21.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:44:21.311 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:44:21.312 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:21.312 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:44:21.312 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:44:21.312 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:44:21.312 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:21.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:21.312 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:44:21.363 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:44:21.363 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:44:21.363 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:44:21.363 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:21.363 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:44:21.363 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41687sec preferred_lft 41687sec 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:21.365 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:44:21.366 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:21.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:21.367 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:44:21.450 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:44:21.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:21.455 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:44:21.510 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:44:21.510 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:44:21.510 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:44:21.510 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:44:21.510 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:44:21.511 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:44:21.511 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:44:21.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:44:21.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:44:21.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:44:21.511 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:44:21.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:44:21.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:21.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:21.736 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:44:21.781 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:44:21.782 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:44:21.782 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:21.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:21.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:44:21.845 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:44:21.844+0000 7f6d2fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:44:21.922 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:44:21.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:44:21.922 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:22.023 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:22.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:44:22.023 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:22.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:44:22.073 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:44:22.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:44:22.187 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:44:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:44:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:44:22.288 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:44:22.289 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:44:22.290 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:44:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:44:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:44:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:22.291 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:22.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:22.292 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:44:22.348 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:44:22.348 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:44:22.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:44:22.424 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:44:22.470 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:44:22.470 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:44:22.470 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:44:22.471 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:22.472 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:22.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:22.473 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:44:22.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:44:22.527+0000 7f75b5fd5480 -1 init, newargv = 0x561606e1c6f0 newargc=15 2024-04-23T14:44:22.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[77904]: starting ceph client 2024-04-23T14:44:22.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[77904]: starting fuse 2024-04-23T14:44:23.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:44:23.712 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:44:23.769 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:44:23.769 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:44:23.769 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:44:23.769 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:44:23.770 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:44:23.771 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:44:23.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:44:23.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:23.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:23.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:23.772 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:23.772 INFO:teuthology.orchestra.run.smithi059.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:23.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:23.773 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:44:23.827 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:44:23.827 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:44:23.827 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:23.828 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:44:23.829 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:24.000 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.77904.asok 2024-04-23T14:44:24.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:24.018 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77904.asok 2024-04-23T14:44:24.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:24.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77904.asok status 2024-04-23T14:44:24.254 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:44:24.254 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:44:24.254 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:44:24.254 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:44:24.254 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "77904", 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9582, 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9582 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:44:24.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3849712084 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3849712084 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9582 192.168.0.1:0/3849712084", 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3849712084", 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 106, 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 115, 2024-04-23T14:44:24.256 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:44:24.257 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:44:24.257 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:44:24.257 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:44:24.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:24.269 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:44:24.269 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:24.320 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:44:24.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:24.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:24.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:24.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:24.403 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-23T14:44:24.483 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-23T14:44:24.483 INFO:teuthology.orchestra.run.smithi059.stdout:./parent 2024-04-23T14:44:24.483 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed 2024-04-23T14:44:24.483 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/charlie 2024-04-23T14:44:24.483 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/alpha 2024-04-23T14:44:24.483 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed 2024-04-23T14:44:24.483 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed/jfile 2024-04-23T14:44:24.485 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:24.485 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:24.485 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:24.485 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:24.485 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-23T14:44:24.485 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:24.627 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:44:24.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:24.642 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:24.642 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:24.642 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:24.642 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:24.642 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-23T14:44:24.642 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:24.837 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-23T14:44:24.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:24.852 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:24.852 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:24.852 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:24.852 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:24.852 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-23T14:44:24.852 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:25.047 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-23T14:44:25.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:25.062 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:25.062 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:25.063 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:25.063 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:25.063 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-23T14:44:25.063 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:25.258 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-23T14:44:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:25.273 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:25.273 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:25.273 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:25.273 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:25.273 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-23T14:44:25.273 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:25.472 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-23T14:44:25.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:25.487 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:25.487 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:25.487 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:25.487 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:25.488 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-23T14:44:25.488 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:25.681 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627779 2024-04-23T14:44:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:25.696 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:25.696 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:25.696 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:25.696 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:25.696 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-23T14:44:25.696 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:25.890 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627780 2024-04-23T14:44:25.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:25.905 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:44:25.906 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:26.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:26.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.217+0000 7f7200a00640 1 Processor -- start 2024-04-23T14:44:26.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.217+0000 7f7200a00640 1 -- start start 2024-04-23T14:44:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.219+0000 7f7200a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71fc0cd760 0x7f71fc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.219+0000 7f7200a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc160210 con 0x7f71fc0b4690 2024-04-23T14:44:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.219+0000 7f7200a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc1604f0 con 0x7f71fc15c3d0 2024-04-23T14:44:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.219+0000 7f7200a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f71fc1607d0 con 0x7f71fc0cd760 2024-04-23T14:44:26.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.219+0000 7f71f3e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71fc0cd760 0x7f71fc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.219+0000 7f71fb200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f71fc0b4690 0x7f71fc0cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49536/0 (socket says 172.21.15.59:49536) 2024-04-23T14:44:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.219+0000 7f71fb200640 1 -- 172.21.15.59:0/1186322893 learned_addr learned my addr 172.21.15.59:0/1186322893 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.220+0000 7f71f3e00640 1 -- 172.21.15.59:0/1186322893 >> v1:172.21.15.76:6789/0 conn(0x7f71fc15c3d0 legacy=0x7f71fc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.220+0000 7f71f3e00640 1 -- 172.21.15.59:0/1186322893 >> v1:172.21.15.59:6789/0 conn(0x7f71fc0b4690 legacy=0x7f71fc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.220+0000 7f71f3e00640 1 -- 172.21.15.59:0/1186322893 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc160ab0 con 0x7f71fc0cd760 2024-04-23T14:44:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.221+0000 7f71f3e00640 1 --2- 172.21.15.59:0/1186322893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0cd760 0x7f71fc0b4150 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f71e40097a0 tx=0x7f71e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8144ab29819b91b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.221+0000 7f71f9e00640 1 -- 172.21.15.59:0/1186322893 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4037070 con 0x7f71fc0cd760 2024-04-23T14:44:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.221+0000 7f71f9e00640 1 -- 172.21.15.59:0/1186322893 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e4032440 con 0x7f71fc0cd760 2024-04-23T14:44:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.221+0000 7f71f9e00640 1 -- 172.21.15.59:0/1186322893 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4032810 con 0x7f71fc0cd760 2024-04-23T14:44:26.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.222+0000 7f7200a00640 1 -- 172.21.15.59:0/1186322893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0cd760 msgr2=0x7f71fc0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.222+0000 7f7200a00640 1 --2- 172.21.15.59:0/1186322893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0cd760 0x7f71fc0b4150 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f71e40097a0 tx=0x7f71e402a1e0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.222+0000 7f7200a00640 1 -- 172.21.15.59:0/1186322893 shutdown_connections 2024-04-23T14:44:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.222+0000 7f7200a00640 1 --2- 172.21.15.59:0/1186322893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0cd760 0x7f71fc0b4150 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.222+0000 7f7200a00640 1 -- 172.21.15.59:0/1186322893 >> 172.21.15.59:0/1186322893 conn(0x7f71fc0b21a0 msgr2=0x7f71fc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.222+0000 7f7200a00640 1 -- 172.21.15.59:0/1186322893 shutdown_connections 2024-04-23T14:44:26.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.222+0000 7f7200a00640 1 -- 172.21.15.59:0/1186322893 wait complete. 2024-04-23T14:44:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.224+0000 7f7200a00640 1 Processor -- start 2024-04-23T14:44:26.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.224+0000 7f7200a00640 1 -- start start 2024-04-23T14:44:26.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f7200a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0b4690 0x7f71fc0998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f7200a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71fc0cd760 0x7f71fc0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f7200a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71fc15c3d0 0x7f71fc1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f7200a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc15c7b0 con 0x7f71fc0b4690 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f7200a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc0b4ae0 con 0x7f71fc0cd760 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f7200a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f71fc15c930 con 0x7f71fc15c3d0 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f71fa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0b4690 0x7f71fc0998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f71fa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0b4690 0x7f71fc0998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48184/0 (socket says 172.21.15.59:48184) 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.225+0000 7f71fa800640 1 -- 172.21.15.59:0/3180316747 learned_addr learned my addr 172.21.15.59:0/3180316747 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71f3e00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71fc0cd760 0x7f71fc0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71fa800640 1 -- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71fc15c3d0 msgr2=0x7f71fc1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71fa800640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71fc15c3d0 0x7f71fc1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71fa800640 1 -- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71fc0cd760 msgr2=0x7f71fc0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71fa800640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71fc0cd760 0x7f71fc0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71fa800640 1 -- 172.21.15.59:0/3180316747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc0aefc0 con 0x7f71fc0b4690 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71f3e00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71fc0cd760 0x7f71fc0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71fa800640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0b4690 0x7f71fc0998d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00b3f0 tx=0x7f71ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71f2a00640 1 -- 172.21.15.59:0/3180316747 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec015020 con 0x7f71fc0b4690 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71f2a00640 1 -- 172.21.15.59:0/3180316747 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ec011830 con 0x7f71fc0b4690 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f71f2a00640 1 -- 172.21.15.59:0/3180316747 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec011c20 con 0x7f71fc0b4690 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.226+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc0af2a0 con 0x7f71fc0b4690 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.227+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc0af860 con 0x7f71fc0b4690 2024-04-23T14:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.228+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71fc04fd90 con 0x7f71fc0b4690 2024-04-23T14:44:26.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.229+0000 7f71f2a00640 1 -- 172.21.15.59:0/3180316747 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71ec018020 con 0x7f71fc0b4690 2024-04-23T14:44:26.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.230+0000 7f71f2a00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f71d806c8f0 0x7f71d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.230+0000 7f71f2a00640 1 -- 172.21.15.59:0/3180316747 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f71ec08b920 con 0x7f71fc0b4690 2024-04-23T14:44:26.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.230+0000 7f71f3e00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f71d806c8f0 0x7f71d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.231+0000 7f71f3e00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f71d806c8f0 0x7f71d806edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f71e4034000 tx=0x7f71e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:26.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.234+0000 7f71f2a00640 1 -- 172.21.15.59:0/3180316747 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71ec01f030 con 0x7f71fc0b4690 2024-04-23T14:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.445+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71fc0c6950 con 0x7f71fc0b4690 2024-04-23T14:44:26.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.448+0000 7f71f2a00640 1 -- 172.21.15.59:0/3180316747 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16789 (secure 0 0 0) 0x7f71ec0172c0 con 0x7f71fc0b4690 2024-04-23T14:44:26.450 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:26.450 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":115,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:44:14.736971+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/1061483588":"2024-04-24T14:44:14.736925+0000","172.21.15.59:6836/1061483588":"2024-04-24T14:44:14.736925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f71d806c8f0 msgr2=0x7f71d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f71d806c8f0 0x7f71d806edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f71e4034000 tx=0x7f71e402e000 comp rx=0 tx=0).stop 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0b4690 msgr2=0x7f71fc0998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0b4690 0x7f71fc0998d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00b3f0 tx=0x7f71ec00b8b0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 shutdown_connections 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f71d806c8f0 0x7f71d806edb0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71fc15c3d0 0x7f71fc1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71fc0cd760 0x7f71fc0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 --2- 172.21.15.59:0/3180316747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71fc0b4690 0x7f71fc0998d0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 >> 172.21.15.59:0/3180316747 conn(0x7f71fc0b21a0 msgr2=0x7f71fc06ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:26.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.451+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 shutdown_connections 2024-04-23T14:44:26.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.452+0000 7f7200a00640 1 -- 172.21.15.59:0/3180316747 wait complete. 2024-04-23T14:44:26.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:44:26.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:44:26.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.679+0000 7f3e42400640 1 Processor -- start 2024-04-23T14:44:26.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.679+0000 7f3e42400640 1 -- start start 2024-04-23T14:44:26.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.680+0000 7f3e42400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3e3c0d0d10 0x7f3e3c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.680+0000 7f3e42400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e3c15d810 0x7f3e3c160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.680+0000 7f3e42400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3c15cab0 con 0x7f3e3c1567e0 2024-04-23T14:44:26.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.680+0000 7f3e42400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3c160e40 con 0x7f3e3c0d0d10 2024-04-23T14:44:26.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.680+0000 7f3e42400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c15cd30 con 0x7f3e3c15d810 2024-04-23T14:44:26.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.681+0000 7f3e41000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3e3c1567e0 0x7f3e3c0d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49540/0 (socket says 172.21.15.59:49540) 2024-04-23T14:44:26.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.681+0000 7f3e41000640 1 -- 172.21.15.59:0/1718133093 learned_addr learned my addr 172.21.15.59:0/1718133093 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:26.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.681+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898384006 0 0) 0x7f3e30001110 con 0x7f3e3c1567e0 2024-04-23T14:44:26.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.681+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e24003520 con 0x7f3e3c1567e0 2024-04-23T14:44:26.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.681+0000 7f3e3be00640 1 --2- 172.21.15.59:0/1718133093 >> v2:172.21.15.76:3300/0 conn(0x7f3e3c0d0d10 0x7f3e3c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.681+0000 7f3e41a00640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e3c15d810 0x7f3e3c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4128065317 0 0) 0x7f3e24003520 con 0x7f3e3c1567e0 2024-04-23T14:44:26.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e3c15d810 msgr2=0x7f3e3c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e3c15d810 0x7f3e3c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 >> v2:172.21.15.76:3300/0 conn(0x7f3e3c0d0d10 msgr2=0x7f3e3c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 --2- 172.21.15.59:0/1718133093 >> v2:172.21.15.76:3300/0 conn(0x7f3e3c0d0d10 0x7f3e3c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c15d010 con 0x7f3e3c1567e0 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f3e30001840 con 0x7f3e3c1567e0 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 >> v1:172.21.15.59:6789/0 conn(0x7f3e3c1567e0 legacy=0x7f3e3c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e240039b0 0x7f3e24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e24004800 0x7f3e24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e240051c0 0x7f3e2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c160e40 con 0x7f3e24004800 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c15cd30 con 0x7f3e240051c0 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2400bf80 con 0x7f3e240039b0 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.682+0000 7f3e3be00640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e24004800 0x7f3e24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.683+0000 7f3e41000640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e240039b0 0x7f3e24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.683+0000 7f3e3be00640 1 -- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e240039b0 msgr2=0x7f3e24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:26.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.683+0000 7f3e3be00640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e240039b0 0x7f3e24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.683+0000 7f3e3be00640 1 -- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e240051c0 msgr2=0x7f3e2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:26.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.683+0000 7f3e3be00640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e240051c0 0x7f3e2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.683+0000 7f3e3be00640 1 -- 172.21.15.59:0/1718133093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e24003520 con 0x7f3e24004800 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.684+0000 7f3e3be00640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e24004800 0x7f3e24004c80 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0afdb0 tx=0x7f3e24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc08f60d90836a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.684+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28036070 con 0x7f3e24004800 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.684+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e28031400 con 0x7f3e24004800 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.684+0000 7f3e3b400640 1 -- 172.21.15.59:0/1718133093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28009690 con 0x7f3e24004800 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.685+0000 7f3e42400640 1 -- 172.21.15.59:0/1718133093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e24004800 msgr2=0x7f3e24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.685+0000 7f3e42400640 1 --2- 172.21.15.59:0/1718133093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e24004800 0x7f3e24004c80 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0afdb0 tx=0x7f3e24005830 comp rx=0 tx=0).stop 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.685+0000 7f3e41a00640 1 -- 172.21.15.59:0/1718133093 reap_dead start 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.685+0000 7f3e42400640 1 -- 172.21.15.59:0/1718133093 shutdown_connections 2024-04-23T14:44:26.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.685+0000 7f3e42400640 1 -- 172.21.15.59:0/1718133093 >> 172.21.15.59:0/1718133093 conn(0x7f3e3c06bdf0 msgr2=0x7f3e3c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:26.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.685+0000 7f3e42400640 1 -- 172.21.15.59:0/1718133093 shutdown_connections 2024-04-23T14:44:26.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.685+0000 7f3e42400640 1 -- 172.21.15.59:0/1718133093 wait complete. 2024-04-23T14:44:26.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.687+0000 7f3e42400640 1 Processor -- start 2024-04-23T14:44:26.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.687+0000 7f3e42400640 1 -- start start 2024-04-23T14:44:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e42400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e24004800 0x7f3e3c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e42400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c151480 0x7f3e3c1f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e42400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e3c1518c0 0x7f3e3c14d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e42400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c160290 con 0x7f3e3c151480 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e42400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c160110 con 0x7f3e3c1518c0 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e42400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c160410 con 0x7f3e24004800 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c151480 0x7f3e3c1f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e3be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c151480 0x7f3e3c1f8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48196/0 (socket says 172.21.15.59:48196) 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.688+0000 7f3e3be00640 1 -- 172.21.15.59:0/3194048644 learned_addr learned my addr 172.21.15.59:0/3194048644 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e3be00640 1 -- 172.21.15.59:0/3194048644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e24004800 msgr2=0x7f3e3c158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e3be00640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e24004800 0x7f3e3c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e3be00640 1 -- 172.21.15.59:0/3194048644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e3c1518c0 msgr2=0x7f3e3c14d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e3be00640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e3c1518c0 0x7f3e3c14d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e3be00640 1 -- 172.21.15.59:0/3194048644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c158020 con 0x7f3e3c151480 2024-04-23T14:44:26.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e3be00640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c151480 0x7f3e3c1f8870 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009b80 tx=0x7f3e28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:26.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e38c00640 1 -- 172.21.15.59:0/3194048644 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2803f020 con 0x7f3e3c151480 2024-04-23T14:44:26.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e38c00640 1 -- 172.21.15.59:0/3194048644 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e28037040 con 0x7f3e3c151480 2024-04-23T14:44:26.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e38c00640 1 -- 172.21.15.59:0/3194048644 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28036070 con 0x7f3e3c151480 2024-04-23T14:44:26.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.689+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c1f9480 con 0x7f3e3c151480 2024-04-23T14:44:26.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.690+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c154410 con 0x7f3e3c151480 2024-04-23T14:44:26.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.691+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c159030 con 0x7f3e3c151480 2024-04-23T14:44:26.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.692+0000 7f3e38c00640 1 -- 172.21.15.59:0/3194048644 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e28042020 con 0x7f3e3c151480 2024-04-23T14:44:26.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.692+0000 7f3e38c00640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e1c06c8f0 0x7f3e1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:26.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.693+0000 7f3e38c00640 1 -- 172.21.15.59:0/3194048644 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f3e280ab990 con 0x7f3e3c151480 2024-04-23T14:44:26.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.693+0000 7f3e41000640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e1c06c8f0 0x7f3e1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:26.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.694+0000 7f3e41000640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e1c06c8f0 0x7f3e1c06edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3e30001050 tx=0x7f3e300012b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.698+0000 7f3e38c00640 1 -- 172.21.15.59:0/3194048644 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e28049070 con 0x7f3e3c151480 2024-04-23T14:44:26.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.909+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e3c04fd90 con 0x7f3e3c151480 2024-04-23T14:44:26.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.910+0000 7f3e38c00640 1 -- 172.21.15.59:0/3194048644 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f3e280ab070 con 0x7f3e3c151480 2024-04-23T14:44:26.911 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:44:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.912+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e1c06c8f0 msgr2=0x7f3e1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.912+0000 7f3e42400640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e1c06c8f0 0x7f3e1c06edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3e30001050 tx=0x7f3e300012b0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.912+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c151480 msgr2=0x7f3e3c1f8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.912+0000 7f3e42400640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c151480 0x7f3e3c1f8870 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009b80 tx=0x7f3e28004730 comp rx=0 tx=0).stop 2024-04-23T14:44:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 shutdown_connections 2024-04-23T14:44:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3e1c06c8f0 0x7f3e1c06edb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e3c1518c0 0x7f3e3c14d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e3c151480 0x7f3e3c1f8870 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 --2- 172.21.15.59:0/3194048644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e24004800 0x7f3e3c158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 >> 172.21.15.59:0/3194048644 conn(0x7f3e3c06bdf0 msgr2=0x7f3e3c073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 shutdown_connections 2024-04-23T14:44:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:26.913+0000 7f3e42400640 1 -- 172.21.15.59:0/3194048644 wait complete. 2024-04-23T14:44:26.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:44:27.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.141+0000 7fd5b0a00640 1 Processor -- start 2024-04-23T14:44:27.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.141+0000 7fd5b0a00640 1 -- start start 2024-04-23T14:44:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.143+0000 7fd5b0a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5ac155a80 0x7fd5ac06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.143+0000 7fd5b0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 0x7fd5ac0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.143+0000 7fd5b0a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ac15ef20 con 0x7fd5ac0b43a0 2024-04-23T14:44:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.143+0000 7fd5b0a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ac0d0fa0 con 0x7fd5ac155a80 2024-04-23T14:44:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.143+0000 7fd5b0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac15f1c0 con 0x7fd5ac0d0600 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.143+0000 7fd5ab200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd5ac0b43a0 0x7fd5ac0b4830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49544/0 (socket says 172.21.15.59:49544) 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.143+0000 7fd5ab200640 1 -- 172.21.15.59:0/108368701 learned_addr learned my addr 172.21.15.59:0/108368701 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5aa800640 1 --2- 172.21.15.59:0/108368701 >> v2:172.21.15.76:3300/0 conn(0x7fd5ac155a80 0x7fd5ac06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 0x7fd5ac0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593801959 0 0) 0x7fd5ac15ef20 con 0x7fd5ac0b43a0 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd590003520 con 0x7fd5ac0b43a0 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3551115493 0 0) 0x7fd590003520 con 0x7fd5ac0b43a0 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 msgr2=0x7fd5ac0d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 0x7fd5ac0d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 >> v2:172.21.15.76:3300/0 conn(0x7fd5ac155a80 msgr2=0x7fd5ac06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 --2- 172.21.15.59:0/108368701 >> v2:172.21.15.76:3300/0 conn(0x7fd5ac155a80 0x7fd5ac06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac15f4a0 con 0x7fd5ac0b43a0 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.144+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fd5a40019f0 con 0x7fd5ac0b43a0 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 >> v1:172.21.15.59:6789/0 conn(0x7fd5ac0b43a0 legacy=0x7fd5ac0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a9e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5900039b0 0x7fd590003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a9e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a9e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 0x7fd5900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac15f1c0 con 0x7fd590004800 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd59000b9c0 con 0x7fd5900039b0 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd59000bb40 con 0x7fd5ac0d0600 2024-04-23T14:44:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5aa800640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5900039b0 0x7fd590003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a3e00640 1 -- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 msgr2=0x7fd5900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5ab200640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 0x7fd5900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac0d0600 0x7fd5900051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a3e00640 1 -- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5900039b0 msgr2=0x7fd590003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5900039b0 0x7fd590003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.145+0000 7fd5a3e00640 1 -- 172.21.15.59:0/108368701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd590003520 con 0x7fd590004800 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.147+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd590004c80 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd594009ac0 tx=0x7fd590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193ae0386bc5ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.147+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594036070 con 0x7fd590004800 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.147+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd594031400 con 0x7fd590004800 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.147+0000 7fd5a9e00640 1 -- 172.21.15.59:0/108368701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594009690 con 0x7fd590004800 2024-04-23T14:44:27.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.147+0000 7fd5b0a00640 1 -- 172.21.15.59:0/108368701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 msgr2=0x7fd590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.147+0000 7fd5b0a00640 1 --2- 172.21.15.59:0/108368701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd590004c80 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd594009ac0 tx=0x7fd590005830 comp rx=0 tx=0).stop 2024-04-23T14:44:27.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.147+0000 7fd5ab200640 1 -- 172.21.15.59:0/108368701 reap_dead start 2024-04-23T14:44:27.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.148+0000 7fd5b0a00640 1 -- 172.21.15.59:0/108368701 shutdown_connections 2024-04-23T14:44:27.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.148+0000 7fd5b0a00640 1 -- 172.21.15.59:0/108368701 >> 172.21.15.59:0/108368701 conn(0x7fd5ac06bdf0 msgr2=0x7fd5ac06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:27.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.148+0000 7fd5b0a00640 1 -- 172.21.15.59:0/108368701 shutdown_connections 2024-04-23T14:44:27.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.148+0000 7fd5b0a00640 1 -- 172.21.15.59:0/108368701 wait complete. 2024-04-23T14:44:27.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.149+0000 7fd5b0a00640 1 Processor -- start 2024-04-23T14:44:27.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.150+0000 7fd5b0a00640 1 -- start start 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.150+0000 7fd5b0a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd5ac0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.150+0000 7fd5b0a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac14c250 0x7fd5ac15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.150+0000 7fd5b0a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac15b930 0x7fd5ac15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.150+0000 7fd5b0a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac15fd40 con 0x7fd590004800 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.150+0000 7fd5b0a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac15fa60 con 0x7fd5ac15b930 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.150+0000 7fd5b0a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac06ff00 con 0x7fd5ac14c250 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd5ac0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd5ac0b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48216/0 (socket says 172.21.15.59:48216) 2024-04-23T14:44:27.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 -- 172.21.15.59:0/1506095328 learned_addr learned my addr 172.21.15.59:0/1506095328 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:27.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5ab200640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac15b930 0x7fd5ac15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac14c250 0x7fd5ac15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 -- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac14c250 msgr2=0x7fd5ac15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac14c250 0x7fd5ac15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 -- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac15b930 msgr2=0x7fd5ac15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac15b930 0x7fd5ac15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5aa800640 1 -- 172.21.15.59:0/1506095328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac15e720 con 0x7fd590004800 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5ab200640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac15b930 0x7fd5ac15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.151+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac14c250 0x7fd5ac15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.152+0000 7fd5aa800640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd5ac0b18c0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fd59c011730 tx=0x7fd59c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.152+0000 7fd5a2a00640 1 -- 172.21.15.59:0/1506095328 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00f9c0 con 0x7fd590004800 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.152+0000 7fd5a2a00640 1 -- 172.21.15.59:0/1506095328 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59c00fb60 con 0x7fd590004800 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.152+0000 7fd5a2a00640 1 -- 172.21.15.59:0/1506095328 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c015460 con 0x7fd590004800 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.152+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac14c7f0 con 0x7fd590004800 2024-04-23T14:44:27.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.152+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac14cdb0 con 0x7fd590004800 2024-04-23T14:44:27.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.153+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac04fd90 con 0x7fd590004800 2024-04-23T14:44:27.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.154+0000 7fd5a2a00640 1 -- 172.21.15.59:0/1506095328 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd59c015600 con 0x7fd590004800 2024-04-23T14:44:27.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.155+0000 7fd5a2a00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd58806c8f0 0x7fd58806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.155+0000 7fd5a2a00640 1 -- 172.21.15.59:0/1506095328 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fd59c08c670 con 0x7fd590004800 2024-04-23T14:44:27.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.155+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd58806c8f0 0x7fd58806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.156+0000 7fd5a3e00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd58806c8f0 0x7fd58806edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd594033000 tx=0x7fd594008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:27.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.160+0000 7fd5a2a00640 1 -- 172.21.15.59:0/1506095328 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd59c019030 con 0x7fd590004800 2024-04-23T14:44:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.363+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd5ac06eba0 con 0x7fd590004800 2024-04-23T14:44:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.364+0000 7fd5a2a00640 1 -- 172.21.15.59:0/1506095328 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7fd59c05a320 con 0x7fd590004800 2024-04-23T14:44:27.365 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:44:27.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd58806c8f0 msgr2=0x7fd58806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd58806c8f0 0x7fd58806edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd594033000 tx=0x7fd594008030 comp rx=0 tx=0).stop 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 msgr2=0x7fd5ac0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd5ac0b18c0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fd59c011730 tx=0x7fd59c011bf0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 shutdown_connections 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd58806c8f0 0x7fd58806edb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5ac14c250 0x7fd5ac15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5ac15b930 0x7fd5ac15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 --2- 172.21.15.59:0/1506095328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd590004800 0x7fd5ac0b18c0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 >> 172.21.15.59:0/1506095328 conn(0x7fd5ac06bdf0 msgr2=0x7fd5ac0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 shutdown_connections 2024-04-23T14:44:27.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.367+0000 7fd5b0a00640 1 -- 172.21.15.59:0/1506095328 wait complete. 2024-04-23T14:44:27.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:44:27.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.600+0000 7fc512c00640 1 Processor -- start 2024-04-23T14:44:27.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.600+0000 7fc512c00640 1 -- start start 2024-04-23T14:44:27.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc512c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 0x7fc50c15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc512c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc50c06e740 0x7fc50c072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc512c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc50c06efe0 con 0x7fc50c072b50 2024-04-23T14:44:27.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc512c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc50c072fc0 con 0x7fc50c06e740 2024-04-23T14:44:27.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc512c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c06f2c0 con 0x7fc50c155280 2024-04-23T14:44:27.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc510e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc50c06e740 0x7fc50c072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc512200640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc50c072b50 0x7fc50c15e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49560/0 (socket says 172.21.15.59:49560) 2024-04-23T14:44:27.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc512200640 1 -- 172.21.15.59:0/1782309050 learned_addr learned my addr 172.21.15.59:0/1782309050 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:27.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.602+0000 7fc511800640 1 --2- 172.21.15.59:0/1782309050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 0x7fc50c15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.603+0000 7fc510e00640 1 -- 172.21.15.59:0/1782309050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 msgr2=0x7fc50c15bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:27.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.603+0000 7fc510e00640 1 --2- 172.21.15.59:0/1782309050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 0x7fc50c15bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.603+0000 7fc510e00640 1 -- 172.21.15.59:0/1782309050 >> v1:172.21.15.59:6789/0 conn(0x7fc50c072b50 legacy=0x7fc50c15e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.603+0000 7fc510e00640 1 -- 172.21.15.59:0/1782309050 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c06f5a0 con 0x7fc50c06e740 2024-04-23T14:44:27.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.603+0000 7fc50be00640 1 -- 172.21.15.59:0/1782309050 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182076522 0 0) 0x7fc50c06efe0 con 0x7fc50c072b50 2024-04-23T14:44:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.604+0000 7fc510e00640 1 --2- 172.21.15.59:0/1782309050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c072610 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80097a0 tx=0x7fc4f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db9bcc7234d1580 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.604+0000 7fc50be00640 1 -- 172.21.15.59:0/1782309050 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8037070 con 0x7fc50c06e740 2024-04-23T14:44:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.604+0000 7fc50be00640 1 -- 172.21.15.59:0/1782309050 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f8032440 con 0x7fc50c06e740 2024-04-23T14:44:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.604+0000 7fc50be00640 1 -- 172.21.15.59:0/1782309050 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8032810 con 0x7fc50c06e740 2024-04-23T14:44:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 -- 172.21.15.59:0/1782309050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 msgr2=0x7fc50c072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 --2- 172.21.15.59:0/1782309050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c072610 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80097a0 tx=0x7fc4f802a180 comp rx=0 tx=0).stop 2024-04-23T14:44:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 -- 172.21.15.59:0/1782309050 shutdown_connections 2024-04-23T14:44:27.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 --2- 172.21.15.59:0/1782309050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c072610 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 --2- 172.21.15.59:0/1782309050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 0x7fc50c15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 -- 172.21.15.59:0/1782309050 >> 172.21.15.59:0/1782309050 conn(0x7fc50c06bdf0 msgr2=0x7fc50c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:27.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 -- 172.21.15.59:0/1782309050 shutdown_connections 2024-04-23T14:44:27.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.605+0000 7fc512c00640 1 -- 172.21.15.59:0/1782309050 wait complete. 2024-04-23T14:44:27.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.607+0000 7fc512c00640 1 Processor -- start 2024-04-23T14:44:27.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.607+0000 7fc512c00640 1 -- start start 2024-04-23T14:44:27.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc512c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c09a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc512c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc50c072b50 0x7fc50c158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc512c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 0x7fc50c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc512c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c0b41a0 con 0x7fc50c06e740 2024-04-23T14:44:27.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc512c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c06fb60 con 0x7fc50c072b50 2024-04-23T14:44:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc512c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c0b4320 con 0x7fc50c155280 2024-04-23T14:44:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc511800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c09a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc511800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c09a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48230/0 (socket says 172.21.15.59:48230) 2024-04-23T14:44:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc511800640 1 -- 172.21.15.59:0/1475614035 learned_addr learned my addr 172.21.15.59:0/1475614035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc511800640 1 -- 172.21.15.59:0/1475614035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 msgr2=0x7fc50c1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc511800640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 0x7fc50c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.608+0000 7fc511800640 1 -- 172.21.15.59:0/1475614035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc50c072b50 msgr2=0x7fc50c158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc511800640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc50c072b50 0x7fc50c158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc511800640 1 -- 172.21.15.59:0/1475614035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c0cdaf0 con 0x7fc50c06e740 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc511800640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c09a280 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fc50000d5f0 tx=0x7fc50000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc509600640 1 -- 172.21.15.59:0/1475614035 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5000125f0 con 0x7fc50c06e740 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc509600640 1 -- 172.21.15.59:0/1475614035 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc500019070 con 0x7fc50c06e740 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc509600640 1 -- 172.21.15.59:0/1475614035 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5000129d0 con 0x7fc50c06e740 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50c0cddd0 con 0x7fc50c06e740 2024-04-23T14:44:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.609+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50c15d510 con 0x7fc50c06e740 2024-04-23T14:44:27.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.610+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50c04fd90 con 0x7fc50c06e740 2024-04-23T14:44:27.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.611+0000 7fc509600640 1 -- 172.21.15.59:0/1475614035 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc500012b70 con 0x7fc50c06e740 2024-04-23T14:44:27.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.612+0000 7fc509600640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc4ec06c8f0 0x7fc4ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:27.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.613+0000 7fc509600640 1 -- 172.21.15.59:0/1475614035 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fc50008c100 con 0x7fc50c06e740 2024-04-23T14:44:27.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.613+0000 7fc510e00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc4ec06c8f0 0x7fc4ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:27.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.614+0000 7fc510e00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc4ec06c8f0 0x7fc4ec06edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8034000 tx=0x7fc4f802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:27.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.618+0000 7fc509600640 1 -- 172.21.15.59:0/1475614035 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc50005a400 con 0x7fc50c06e740 2024-04-23T14:44:27.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.819+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc50c06e2f0 con 0x7fc50c06e740 2024-04-23T14:44:27.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.820+0000 7fc509600640 1 -- 172.21.15.59:0/1475614035 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7fc500059d00 con 0x7fc50c06e740 2024-04-23T14:44:27.822 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:44:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.822+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc4ec06c8f0 msgr2=0x7fc4ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.822+0000 7fc512c00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc4ec06c8f0 0x7fc4ec06edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8034000 tx=0x7fc4f802c000 comp rx=0 tx=0).stop 2024-04-23T14:44:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.822+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 msgr2=0x7fc50c09a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.822+0000 7fc512c00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c09a280 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fc50000d5f0 tx=0x7fc50000dab0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 shutdown_connections 2024-04-23T14:44:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc4ec06c8f0 0x7fc4ec06edb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc50c155280 0x7fc50c1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc50c072b50 0x7fc50c158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 --2- 172.21.15.59:0/1475614035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc50c06e740 0x7fc50c09a280 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:27.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 >> 172.21.15.59:0/1475614035 conn(0x7fc50c06bdf0 msgr2=0x7fc50c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:27.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 shutdown_connections 2024-04-23T14:44:27.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:27.823+0000 7fc512c00640 1 -- 172.21.15.59:0/1475614035 wait complete. 2024-04-23T14:44:27.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:44:28.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.059+0000 7f9abd400640 1 Processor -- start 2024-04-23T14:44:28.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.059+0000 7f9abd400640 1 -- start start 2024-04-23T14:44:28.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.061+0000 7f9abd400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 0x7f9ab815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.061+0000 7f9abd400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab80b3720 con 0x7f9ab80af4b0 2024-04-23T14:44:28.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.061+0000 7f9abd400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab815f3f0 con 0x7f9ab80af890 2024-04-23T14:44:28.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.061+0000 7f9abd400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab815f6d0 con 0x7f9ab80d0cd0 2024-04-23T14:44:28.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.061+0000 7f9ab7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9ab80af4b0 0x7f9ab80b3010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49566/0 (socket says 172.21.15.59:49566) 2024-04-23T14:44:28.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.061+0000 7f9ab7e00640 1 -- 172.21.15.59:0/1555074806 learned_addr learned my addr 172.21.15.59:0/1555074806 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9abca00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 0x7f9ab815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2031593428 0 0) 0x7f9ab80b3720 con 0x7f9ab80af4b0 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a9c003520 con 0x7f9ab80af4b0 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 442742219 0 0) 0x7f9a9c003520 con 0x7f9ab80af4b0 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 msgr2=0x7f9ab815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 0x7f9ab815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 >> v1:172.21.15.76:6789/0 conn(0x7f9ab80af890 legacy=0x7f9ab80d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab815f9b0 con 0x7f9ab80af4b0 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9aa8001670 con 0x7f9ab80af4b0 2024-04-23T14:44:28.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 >> v1:172.21.15.59:6789/0 conn(0x7f9ab80af4b0 legacy=0x7f9ab80b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c0039e0 0x7f9a9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 0x7f9a9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a9c004d40 0x7f9a9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab815f6d0 con 0x7f9a9c004d40 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c00baa0 con 0x7f9a9c0039e0 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.062+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9c00bca0 con 0x7f9ab80d0cd0 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9abca00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a9c004d40 0x7f9a9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9ab7e00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c0039e0 0x7f9a9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9ab7400640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 0x7f9a9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9abca00640 1 -- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 msgr2=0x7f9a9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9abca00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 0x7f9a9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9abca00640 1 -- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c0039e0 msgr2=0x7f9a9c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:28.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9abca00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c0039e0 0x7f9a9c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9abca00640 1 -- 172.21.15.59:0/1555074806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c003520 con 0x7f9a9c004d40 2024-04-23T14:44:28.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.063+0000 7f9ab7400640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab80d0cd0 0x7f9a9c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.064+0000 7f9abca00640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a9c004d40 0x7f9a9c0051c0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009f90 tx=0x7f9a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d9a53710763525 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0009c10 con 0x7f9a9c004d40 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0036070 con 0x7f9a9c004d40 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9ab6a00640 1 -- 172.21.15.59:0/1555074806 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0031850 con 0x7f9a9c004d40 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9abd400640 1 -- 172.21.15.59:0/1555074806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a9c004d40 msgr2=0x7f9a9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9abd400640 1 --2- 172.21.15.59:0/1555074806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a9c004d40 0x7f9a9c0051c0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009f90 tx=0x7f9a9c005830 comp rx=0 tx=0).stop 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9abca00640 1 -- 172.21.15.59:0/1555074806 reap_dead start 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9abd400640 1 -- 172.21.15.59:0/1555074806 shutdown_connections 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.065+0000 7f9abd400640 1 -- 172.21.15.59:0/1555074806 >> 172.21.15.59:0/1555074806 conn(0x7f9ab806bdf0 msgr2=0x7f9ab806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.066+0000 7f9abd400640 1 -- 172.21.15.59:0/1555074806 shutdown_connections 2024-04-23T14:44:28.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.066+0000 7f9abd400640 1 -- 172.21.15.59:0/1555074806 wait complete. 2024-04-23T14:44:28.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.067+0000 7f9abd400640 1 Processor -- start 2024-04-23T14:44:28.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.068+0000 7f9abd400640 1 -- start start 2024-04-23T14:44:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abd400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c004d40 0x7f9ab80afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abd400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab81f7fc0 0x7f9ab8157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abd400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ab8154ae0 0x7f9ab8154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abd400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab815eed0 con 0x7f9ab8154ae0 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abd400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab815ed50 con 0x7f9a9c004d40 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abd400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab815f050 con 0x7f9ab81f7fc0 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ab8154ae0 0x7f9ab8154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ab8154ae0 0x7f9ab8154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48244/0 (socket says 172.21.15.59:48244) 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 -- 172.21.15.59:0/2296526466 learned_addr learned my addr 172.21.15.59:0/2296526466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9ab7e00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c004d40 0x7f9ab80afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9ab7400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab81f7fc0 0x7f9ab8157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 -- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab81f7fc0 msgr2=0x7f9ab8157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab81f7fc0 0x7f9ab8157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 -- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c004d40 msgr2=0x7f9ab80afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c004d40 0x7f9ab80afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9abca00640 1 -- 172.21.15.59:0/2296526466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab80b1780 con 0x7f9ab8154ae0 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9ab7e00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c004d40 0x7f9ab80afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:28.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.069+0000 7f9ab7400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab81f7fc0 0x7f9ab8157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.070+0000 7f9abca00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ab8154ae0 0x7f9ab8154f60 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e230 tx=0x7f9ab0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.070+0000 7f9aafe00640 1 -- 172.21.15.59:0/2296526466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0009c10 con 0x7f9ab8154ae0 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.070+0000 7f9aafe00640 1 -- 172.21.15.59:0/2296526466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0003990 con 0x7f9ab8154ae0 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.070+0000 7f9aafe00640 1 -- 172.21.15.59:0/2296526466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0003d80 con 0x7f9ab8154ae0 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.070+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab81f8440 con 0x7f9ab8154ae0 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.070+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab80b1a20 con 0x7f9ab8154ae0 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.072+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab804fd90 con 0x7f9ab8154ae0 2024-04-23T14:44:28.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.073+0000 7f9aafe00640 1 -- 172.21.15.59:0/2296526466 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ab0005d10 con 0x7f9ab8154ae0 2024-04-23T14:44:28.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.074+0000 7f9aafe00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a9806c8f0 0x7f9a9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.074+0000 7f9aafe00640 1 -- 172.21.15.59:0/2296526466 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f9ab00accb0 con 0x7f9ab8154ae0 2024-04-23T14:44:28.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.074+0000 7f9ab7e00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a9806c8f0 0x7f9a9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.075+0000 7f9ab7e00640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a9806c8f0 0x7f9a9806edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80058b0 tx=0x7f9aa8001430 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:28.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.079+0000 7f9aafe00640 1 -- 172.21.15.59:0/2296526466 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ab002e050 con 0x7f9ab8154ae0 2024-04-23T14:44:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.289+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9ab80b4f70 con 0x7f9ab8154ae0 2024-04-23T14:44:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.290+0000 7f9aafe00640 1 -- 172.21.15.59:0/2296526466 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v115) ==== 80+0+122 (secure 0 0 0) 0x7f9ab007b070 con 0x7f9ab8154ae0 2024-04-23T14:44:28.291 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6837/1061483588 2024-04-24T14:44:14.736925+0000 2024-04-23T14:44:28.291 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6836/1061483588 2024-04-24T14:44:14.736925+0000 2024-04-23T14:44:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-23T14:44:28.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.292+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a9806c8f0 msgr2=0x7f9a9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.292+0000 7f9abd400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a9806c8f0 0x7f9a9806edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80058b0 tx=0x7f9aa8001430 comp rx=0 tx=0).stop 2024-04-23T14:44:28.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.292+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ab8154ae0 msgr2=0x7f9ab8154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.292+0000 7f9abd400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ab8154ae0 0x7f9ab8154f60 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e230 tx=0x7f9ab0009750 comp rx=0 tx=0).stop 2024-04-23T14:44:28.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 shutdown_connections 2024-04-23T14:44:28.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f9a9806c8f0 0x7f9a9806edb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9ab81f7fc0 0x7f9ab8157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ab8154ae0 0x7f9ab8154f60 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 --2- 172.21.15.59:0/2296526466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a9c004d40 0x7f9ab80afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 >> 172.21.15.59:0/2296526466 conn(0x7f9ab806bdf0 msgr2=0x7f9ab80ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 shutdown_connections 2024-04-23T14:44:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.293+0000 7f9abd400640 1 -- 172.21.15.59:0/2296526466 wait complete. 2024-04-23T14:44:28.313 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:44:28.313 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:28.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:28.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.569+0000 7f258ca00640 1 Processor -- start 2024-04-23T14:44:28.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.569+0000 7f258ca00640 1 -- start start 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.570+0000 7f258ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d3260 0x7f25880d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.570+0000 7f258ca00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25880d02e0 0x7f25880d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.570+0000 7f258ca00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25880cf520 con 0x7f258815e9d0 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.570+0000 7f258ca00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f25880cf800 con 0x7f25880d02e0 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.570+0000 7f258ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f25880cfae0 con 0x7f25880d3260 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.570+0000 7f2587e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25880d02e0 0x7f25880d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.571+0000 7f2587e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25880d02e0 0x7f25880d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48260/0 (socket says 172.21.15.59:48260) 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.571+0000 7f2587e00640 1 -- 172.21.15.59:0/2034320656 learned_addr learned my addr 172.21.15.59:0/2034320656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:28.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.571+0000 7f2586a00640 1 --2- 172.21.15.59:0/2034320656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d3260 0x7f25880d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.571+0000 7f2587e00640 1 -- 172.21.15.59:0/2034320656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d3260 msgr2=0x7f25880d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:28.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.571+0000 7f2587e00640 1 --2- 172.21.15.59:0/2034320656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d3260 0x7f25880d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.571+0000 7f2587e00640 1 -- 172.21.15.59:0/2034320656 >> v1:172.21.15.76:6789/0 conn(0x7f258815e9d0 legacy=0x7f25880d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:28.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.571+0000 7f2587e00640 1 -- 172.21.15.59:0/2034320656 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880cfdc0 con 0x7f25880d02e0 2024-04-23T14:44:28.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.572+0000 7f2587e00640 1 --2- 172.21.15.59:0/2034320656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d02e0 0x7f25880d14b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f2580009e50 tx=0x7f258002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6306b89f106c68c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:28.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.572+0000 7f2586000640 1 -- 172.21.15.59:0/2034320656 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580036070 con 0x7f25880d02e0 2024-04-23T14:44:28.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.572+0000 7f2586000640 1 -- 172.21.15.59:0/2034320656 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2580031600 con 0x7f25880d02e0 2024-04-23T14:44:28.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.572+0000 7f2586000640 1 -- 172.21.15.59:0/2034320656 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800319d0 con 0x7f25880d02e0 2024-04-23T14:44:28.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 -- 172.21.15.59:0/2034320656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d02e0 msgr2=0x7f25880d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 --2- 172.21.15.59:0/2034320656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d02e0 0x7f25880d14b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f2580009e50 tx=0x7f258002a6c0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 -- 172.21.15.59:0/2034320656 shutdown_connections 2024-04-23T14:44:28.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 --2- 172.21.15.59:0/2034320656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d02e0 0x7f25880d14b0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 --2- 172.21.15.59:0/2034320656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d3260 0x7f25880d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 -- 172.21.15.59:0/2034320656 >> 172.21.15.59:0/2034320656 conn(0x7f258808bf90 msgr2=0x7f25880d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:28.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 -- 172.21.15.59:0/2034320656 shutdown_connections 2024-04-23T14:44:28.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.573+0000 7f258ca00640 1 -- 172.21.15.59:0/2034320656 wait complete. 2024-04-23T14:44:28.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.575+0000 7f258ca00640 1 Processor -- start 2024-04-23T14:44:28.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.575+0000 7f258ca00640 1 -- start start 2024-04-23T14:44:28.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f258ca00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d02e0 0x7f25880cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f258ca00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d3260 0x7f25880b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f258ca00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2588151c50 0x7f25880b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f258ca00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f258806caf0 con 0x7f25880d3260 2024-04-23T14:44:28.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f258ca00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f25880cf1d0 con 0x7f2588151c50 2024-04-23T14:44:28.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f258ca00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f258806cc70 con 0x7f25880d02e0 2024-04-23T14:44:28.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f2586a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d3260 0x7f25880b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f2586a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d3260 0x7f25880b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48274/0 (socket says 172.21.15.59:48274) 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.576+0000 7f2586a00640 1 -- 172.21.15.59:0/2510060778 learned_addr learned my addr 172.21.15.59:0/2510060778 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2587400640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d02e0 0x7f25880cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2586a00640 1 -- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d02e0 msgr2=0x7f25880cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2587e00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2588151c50 0x7f25880b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2586a00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d02e0 0x7f25880cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2586a00640 1 -- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2588151c50 msgr2=0x7f25880b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2586a00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2588151c50 0x7f25880b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2586a00640 1 -- 172.21.15.59:0/2510060778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880b5180 con 0x7f25880d3260 2024-04-23T14:44:28.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f2586a00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d3260 0x7f25880b49e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f257400d7d0 tx=0x7f257400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:28.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f257f400640 1 -- 172.21.15.59:0/2510060778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257400b6c0 con 0x7f25880d3260 2024-04-23T14:44:28.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f257f400640 1 -- 172.21.15.59:0/2510060778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574019070 con 0x7f25880d3260 2024-04-23T14:44:28.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25880b5460 con 0x7f25880d3260 2024-04-23T14:44:28.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.577+0000 7f257f400640 1 -- 172.21.15.59:0/2510060778 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257400ba80 con 0x7f25880d3260 2024-04-23T14:44:28.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.578+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25880b5a20 con 0x7f25880d3260 2024-04-23T14:44:28.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.579+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258804fd90 con 0x7f25880d3260 2024-04-23T14:44:28.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.580+0000 7f257f400640 1 -- 172.21.15.59:0/2510060778 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f257400bc20 con 0x7f25880d3260 2024-04-23T14:44:28.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.581+0000 7f257f400640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f256806c920 0x7f256806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:28.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.581+0000 7f257f400640 1 -- 172.21.15.59:0/2510060778 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f257408c140 con 0x7f25880d3260 2024-04-23T14:44:28.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.582+0000 7f2587400640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f256806c920 0x7f256806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:28.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.582+0000 7f2587400640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f256806c920 0x7f256806ede0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f257800a750 tx=0x7f2578008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:28.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.586+0000 7f257f400640 1 -- 172.21.15.59:0/2510060778 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2574011340 con 0x7f25880d3260 2024-04-23T14:44:28.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.790+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25880d1d50 con 0x7f25880d3260 2024-04-23T14:44:28.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.792+0000 7f257f400640 1 -- 172.21.15.59:0/2510060778 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16789 (secure 0 0 0) 0x7f257405a3b0 con 0x7f25880d3260 2024-04-23T14:44:28.794 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:28.794 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":115,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:44:14.736971+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/1061483588":"2024-04-24T14:44:14.736925+0000","172.21.15.59:6836/1061483588":"2024-04-24T14:44:14.736925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f256806c920 msgr2=0x7f256806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f256806c920 0x7f256806ede0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f257800a750 tx=0x7f2578008040 comp rx=0 tx=0).stop 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d3260 msgr2=0x7f25880b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d3260 0x7f25880b49e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f257400d7d0 tx=0x7f257400dc90 comp rx=0 tx=0).stop 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 shutdown_connections 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f256806c920 0x7f256806ede0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2588151c50 0x7f25880b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25880d3260 0x7f25880b49e0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 --2- 172.21.15.59:0/2510060778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25880d02e0 0x7f25880cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 >> 172.21.15.59:0/2510060778 conn(0x7f258808bf90 msgr2=0x7f25880b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 shutdown_connections 2024-04-23T14:44:28.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:28.795+0000 7f258ca00640 1 -- 172.21.15.59:0/2510060778 wait complete. 2024-04-23T14:44:28.816 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:44:28.816 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:44:29.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.028+0000 7f0222400640 1 Processor -- start 2024-04-23T14:44:29.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.028+0000 7f0222400640 1 -- start start 2024-04-23T14:44:29.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.030+0000 7f0222400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 0x7f021c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.030+0000 7f0222400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f021c15cab0 con 0x7f021c0d0d10 2024-04-23T14:44:29.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.030+0000 7f0222400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f021c160e40 con 0x7f021c15d810 2024-04-23T14:44:29.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.030+0000 7f0222400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f021c15cd30 con 0x7f021c1567e0 2024-04-23T14:44:29.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.030+0000 7f0221a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f021c0d0d10 0x7f021c160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49572/0 (socket says 172.21.15.59:49572) 2024-04-23T14:44:29.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.030+0000 7f0221a00640 1 -- 172.21.15.59:0/792073314 learned_addr learned my addr 172.21.15.59:0/792073314 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:29.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.030+0000 7f0221000640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 0x7f021c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2060397579 0 0) 0x7f0214001490 con 0x7f021c0d0d10 2024-04-23T14:44:29.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0200003520 con 0x7f021c0d0d10 2024-04-23T14:44:29.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1167044382 0 0) 0x7f0200003520 con 0x7f021c0d0d10 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 msgr2=0x7f021c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 0x7f021c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 >> v1:172.21.15.76:6789/0 conn(0x7f021c15d810 legacy=0x7f021c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c15d010 con 0x7f021c0d0d10 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f0214001bc0 con 0x7f021c0d0d10 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 >> v1:172.21.15.59:6789/0 conn(0x7f021c0d0d10 legacy=0x7f021c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02000039e0 0x7f0200003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 0x7f0200004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0200004d40 0x7f02000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f021c15cd30 con 0x7f02000039e0 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f020000baa0 con 0x7f0200004d40 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.031+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f020000bca0 con 0x7f021c1567e0 2024-04-23T14:44:29.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0221000640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02000039e0 0x7f0200003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0221a00640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0200004d40 0x7f02000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0213e00640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 0x7f0200004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0221000640 1 -- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 msgr2=0x7f0200004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0221000640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 0x7f0200004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0221000640 1 -- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0200004d40 msgr2=0x7f02000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0221000640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0200004d40 0x7f02000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0221000640 1 -- 172.21.15.59:0/792073314 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0200003520 con 0x7f02000039e0 2024-04-23T14:44:29.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.032+0000 7f0213e00640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c1567e0 0x7f0200004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.034+0000 7f0221000640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02000039e0 0x7f0200003e00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f020c009a90 tx=0x7f0200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02c8c4f78b0a381 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.034+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c036070 con 0x7f02000039e0 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.034+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020c031400 con 0x7f02000039e0 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.034+0000 7f021be00640 1 -- 172.21.15.59:0/792073314 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c009690 con 0x7f02000039e0 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.035+0000 7f0222400640 1 -- 172.21.15.59:0/792073314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02000039e0 msgr2=0x7f0200003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.035+0000 7f0222400640 1 --2- 172.21.15.59:0/792073314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02000039e0 0x7f0200003e00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f020c009a90 tx=0x7f0200005830 comp rx=0 tx=0).stop 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.035+0000 7f0221a00640 1 -- 172.21.15.59:0/792073314 reap_dead start 2024-04-23T14:44:29.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.035+0000 7f0222400640 1 -- 172.21.15.59:0/792073314 shutdown_connections 2024-04-23T14:44:29.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.035+0000 7f0222400640 1 -- 172.21.15.59:0/792073314 >> 172.21.15.59:0/792073314 conn(0x7f021c06bdf0 msgr2=0x7f021c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:29.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.035+0000 7f0222400640 1 -- 172.21.15.59:0/792073314 shutdown_connections 2024-04-23T14:44:29.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.035+0000 7f0222400640 1 -- 172.21.15.59:0/792073314 wait complete. 2024-04-23T14:44:29.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.037+0000 7f0222400640 1 Processor -- start 2024-04-23T14:44:29.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.037+0000 7f0222400640 1 -- start start 2024-04-23T14:44:29.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0222400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c14d280 0x7f021c1e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0222400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c1e6400 0x7f021c0afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0222400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f021c0b0320 0x7f021c13f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0222400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f021c156d40 con 0x7f021c0b0320 2024-04-23T14:44:29.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0222400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f021c156bc0 con 0x7f021c1e6400 2024-04-23T14:44:29.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0222400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f021c156ec0 con 0x7f021c14d280 2024-04-23T14:44:29.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f021c0b0320 0x7f021c13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f021c0b0320 0x7f021c13f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48302/0 (socket says 172.21.15.59:48302) 2024-04-23T14:44:29.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 -- 172.21.15.59:0/1707065730 learned_addr learned my addr 172.21.15.59:0/1707065730 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:29.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0213e00640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c1e6400 0x7f021c0afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221000640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c14d280 0x7f021c1e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 -- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c14d280 msgr2=0x7f021c1e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c14d280 0x7f021c1e5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 -- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c1e6400 msgr2=0x7f021c0afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c1e6400 0x7f021c0afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.038+0000 7f0221a00640 1 -- 172.21.15.59:0/1707065730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c14dd70 con 0x7f021c0b0320 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.039+0000 7f0221a00640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f021c0b0320 0x7f021c13f490 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f021c0b4cb0 tx=0x7f0214007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.039+0000 7f0219600640 1 -- 172.21.15.59:0/1707065730 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021400eb00 con 0x7f021c0b0320 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.039+0000 7f0219600640 1 -- 172.21.15.59:0/1707065730 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021400eca0 con 0x7f021c0b0320 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.039+0000 7f0219600640 1 -- 172.21.15.59:0/1707065730 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021400ee90 con 0x7f021c0b0320 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.039+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c1e6a80 con 0x7f021c0b0320 2024-04-23T14:44:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.039+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1e7040 con 0x7f021c0b0320 2024-04-23T14:44:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.040+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c1e67e0 con 0x7f021c0b0320 2024-04-23T14:44:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.041+0000 7f0219600640 1 -- 172.21.15.59:0/1707065730 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f021401a020 con 0x7f021c0b0320 2024-04-23T14:44:29.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.042+0000 7f0219600640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01f806cac0 0x7f01f806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.042+0000 7f0219600640 1 -- 172.21.15.59:0/1707065730 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f021408b680 con 0x7f021c0b0320 2024-04-23T14:44:29.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.043+0000 7f0221000640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01f806cac0 0x7f01f806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.043+0000 7f0221000640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01f806cac0 0x7f01f806ef80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f020c033000 tx=0x7f020c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.046+0000 7f0219600640 1 -- 172.21.15.59:0/1707065730 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0214021020 con 0x7f021c0b0320 2024-04-23T14:44:29.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.249+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f021c04fd90 con 0x7f021c0b0320 2024-04-23T14:44:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.250+0000 7f0219600640 1 -- 172.21.15.59:0/1707065730 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f021c14dd70 con 0x7f021c0b0320 2024-04-23T14:44:29.251 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.252+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01f806cac0 msgr2=0x7f01f806ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.252+0000 7f0222400640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01f806cac0 0x7f01f806ef80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f020c033000 tx=0x7f020c008030 comp rx=0 tx=0).stop 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.252+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f021c0b0320 msgr2=0x7f021c13f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.252+0000 7f0222400640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f021c0b0320 0x7f021c13f490 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f021c0b4cb0 tx=0x7f0214007be0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 shutdown_connections 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f01f806cac0 0x7f01f806ef80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f021c0b0320 0x7f021c13f490 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c1e6400 0x7f021c0afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 --2- 172.21.15.59:0/1707065730 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f021c14d280 0x7f021c1e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 >> 172.21.15.59:0/1707065730 conn(0x7f021c06bdf0 msgr2=0x7f021c15ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:29.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 shutdown_connections 2024-04-23T14:44:29.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.253+0000 7f0222400640 1 -- 172.21.15.59:0/1707065730 wait complete. 2024-04-23T14:44:29.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:44:29.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.479+0000 7ff081c00640 1 Processor -- start 2024-04-23T14:44:29.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.480+0000 7ff081c00640 1 -- start start 2024-04-23T14:44:29.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.481+0000 7ff081c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 0x7ff07c071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.481+0000 7ff081c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff07c0d0600 0x7ff07c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.481+0000 7ff081c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff07c163320 con 0x7ff07c167180 2024-04-23T14:44:29.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.481+0000 7ff081c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff07c163650 con 0x7ff07c0d0600 2024-04-23T14:44:29.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.481+0000 7ff081c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c072380 con 0x7ff07c15d300 2024-04-23T14:44:29.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff081200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff07c0d0600 0x7ff07c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff073e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff07c167180 0x7ff07c163ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49576/0 (socket says 172.21.15.59:49576) 2024-04-23T14:44:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff073e00640 1 -- 172.21.15.59:0/1148285487 learned_addr learned my addr 172.21.15.59:0/1148285487 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff07be00640 1 --2- 172.21.15.59:0/1148285487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 0x7ff07c071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff081200640 1 -- 172.21.15.59:0/1148285487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 msgr2=0x7ff07c071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff081200640 1 --2- 172.21.15.59:0/1148285487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 0x7ff07c071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff081200640 1 -- 172.21.15.59:0/1148285487 >> v1:172.21.15.59:6789/0 conn(0x7ff07c167180 legacy=0x7ff07c163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.482+0000 7ff081200640 1 -- 172.21.15.59:0/1148285487 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c072660 con 0x7ff07c0d0600 2024-04-23T14:44:29.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.483+0000 7ff081200640 1 --2- 172.21.15.59:0/1148285487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c0d0600 0x7ff07c0d0a90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7ff07c1637d0 tx=0x7ff07402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66064402329e5f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff07b400640 1 -- 172.21.15.59:0/1148285487 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07402ae60 con 0x7ff07c0d0600 2024-04-23T14:44:29.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff07b400640 1 -- 172.21.15.59:0/1148285487 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff074007740 con 0x7ff07c0d0600 2024-04-23T14:44:29.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff07b400640 1 -- 172.21.15.59:0/1148285487 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074007b10 con 0x7ff07c0d0600 2024-04-23T14:44:29.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 -- 172.21.15.59:0/1148285487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c0d0600 msgr2=0x7ff07c0d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 --2- 172.21.15.59:0/1148285487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c0d0600 0x7ff07c0d0a90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7ff07c1637d0 tx=0x7ff07402a680 comp rx=0 tx=0).stop 2024-04-23T14:44:29.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 -- 172.21.15.59:0/1148285487 shutdown_connections 2024-04-23T14:44:29.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 --2- 172.21.15.59:0/1148285487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c0d0600 0x7ff07c0d0a90 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 --2- 172.21.15.59:0/1148285487 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 0x7ff07c071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 -- 172.21.15.59:0/1148285487 >> 172.21.15.59:0/1148285487 conn(0x7ff07c06bdf0 msgr2=0x7ff07c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:29.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 -- 172.21.15.59:0/1148285487 shutdown_connections 2024-04-23T14:44:29.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.484+0000 7ff081c00640 1 -- 172.21.15.59:0/1148285487 wait complete. 2024-04-23T14:44:29.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.486+0000 7ff081c00640 1 Processor -- start 2024-04-23T14:44:29.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.486+0000 7ff081c00640 1 -- start start 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07c0d0600 0x7ff07c14c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 0x7ff07c09d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c167180 0x7ff07c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c06e1e0 con 0x7ff07c167180 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1675e0 con 0x7ff07c0d0600 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c06e360 con 0x7ff07c15d300 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c167180 0x7ff07c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c167180 0x7ff07c158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48314/0 (socket says 172.21.15.59:48314) 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.487+0000 7ff081200640 1 -- 172.21.15.59:0/452552191 learned_addr learned my addr 172.21.15.59:0/452552191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:29.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff081200640 1 -- 172.21.15.59:0/452552191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 msgr2=0x7ff07c09d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff081200640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 0x7ff07c09d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff081200640 1 -- 172.21.15.59:0/452552191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07c0d0600 msgr2=0x7ff07c14c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff081200640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07c0d0600 0x7ff07c14c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff081200640 1 -- 172.21.15.59:0/452552191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c14cd10 con 0x7ff07c167180 2024-04-23T14:44:29.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff081200640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c167180 0x7ff07c158a90 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7ff074009eb0 tx=0x7ff074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff078c00640 1 -- 172.21.15.59:0/452552191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074040070 con 0x7ff07c167180 2024-04-23T14:44:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff078c00640 1 -- 172.21.15.59:0/452552191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07402ae60 con 0x7ff07c167180 2024-04-23T14:44:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c0b30b0 con 0x7ff07c167180 2024-04-23T14:44:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.488+0000 7ff078c00640 1 -- 172.21.15.59:0/452552191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074031e60 con 0x7ff07c167180 2024-04-23T14:44:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.489+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c0b3670 con 0x7ff07c167180 2024-04-23T14:44:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.489+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff07c04fd90 con 0x7ff07c167180 2024-04-23T14:44:29.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.491+0000 7ff078c00640 1 -- 172.21.15.59:0/452552191 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff074039040 con 0x7ff07c167180 2024-04-23T14:44:29.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.492+0000 7ff078c00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff05806c8f0 0x7ff05806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.492+0000 7ff078c00640 1 -- 172.21.15.59:0/452552191 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ff0740ad380 con 0x7ff07c167180 2024-04-23T14:44:29.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.492+0000 7ff07be00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff05806c8f0 0x7ff05806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.492+0000 7ff07be00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff05806c8f0 0x7ff05806edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0098f0 tx=0x7ff06c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.496+0000 7ff078c00640 1 -- 172.21.15.59:0/452552191 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff07407b0c0 con 0x7ff07c167180 2024-04-23T14:44:29.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.698+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff07c0cff50 con 0x7ff07c167180 2024-04-23T14:44:29.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.699+0000 7ff078c00640 1 -- 172.21.15.59:0/452552191 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7ff07407f3c0 con 0x7ff07c167180 2024-04-23T14:44:29.701 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff05806c8f0 msgr2=0x7ff05806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff05806c8f0 0x7ff05806edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0098f0 tx=0x7ff06c008040 comp rx=0 tx=0).stop 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c167180 msgr2=0x7ff07c158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c167180 0x7ff07c158a90 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7ff074009eb0 tx=0x7ff074004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 shutdown_connections 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff05806c8f0 0x7ff05806edb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff07c15d300 0x7ff07c09d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff07c167180 0x7ff07c158a90 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 --2- 172.21.15.59:0/452552191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07c0d0600 0x7ff07c14c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.703+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 >> 172.21.15.59:0/452552191 conn(0x7ff07c06bdf0 msgr2=0x7ff07c164bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.704+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 shutdown_connections 2024-04-23T14:44:29.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.704+0000 7ff081c00640 1 -- 172.21.15.59:0/452552191 wait complete. 2024-04-23T14:44:29.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:44:29.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.941+0000 7ffaddc00640 1 Processor -- start 2024-04-23T14:44:29.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.942+0000 7ffaddc00640 1 -- start start 2024-04-23T14:44:29.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffaddc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffad80d15b0 0x7ffad80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 0x7ffad80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffaddc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad80b5da0 con 0x7ffad815ed20 2024-04-23T14:44:29.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffaddc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ffad815f100 con 0x7ffad80d15b0 2024-04-23T14:44:29.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffaddc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffad80d0a20 con 0x7ffad80b4f20 2024-04-23T14:44:29.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffad7e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffad815ed20 0x7ffad80d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:49578/0 (socket says 172.21.15.59:49578) 2024-04-23T14:44:29.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffad7e00640 1 -- 172.21.15.59:0/694235574 learned_addr learned my addr 172.21.15.59:0/694235574 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.943+0000 7ffad7400640 1 --2- 172.21.15.59:0/694235574 >> v2:172.21.15.59:3300/0 conn(0x7ffad80d15b0 0x7ffad80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.944+0000 7ffadd200640 1 --2- 172.21.15.59:0/694235574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 0x7ffad80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.944+0000 7ffad6a00640 1 -- 172.21.15.59:0/694235574 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1315326355 0 0) 0x7ffad80b5da0 con 0x7ffad815ed20 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.944+0000 7ffad6a00640 1 -- 172.21.15.59:0/694235574 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffabc003520 con 0x7ffad815ed20 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.944+0000 7ffad7400640 1 -- 172.21.15.59:0/694235574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 msgr2=0x7ffad80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.944+0000 7ffad7400640 1 --2- 172.21.15.59:0/694235574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 0x7ffad80cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.944+0000 7ffad7400640 1 -- 172.21.15.59:0/694235574 >> v1:172.21.15.59:6789/0 conn(0x7ffad815ed20 legacy=0x7ffad80d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.944+0000 7ffad7400640 1 -- 172.21.15.59:0/694235574 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad80d2ed0 con 0x7ffad80d15b0 2024-04-23T14:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.945+0000 7ffad7400640 1 --2- 172.21.15.59:0/694235574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad80d15b0 0x7ffad80cf280 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097a0 tx=0x7ffac002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d21ced938a07d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.945+0000 7ffad6a00640 1 -- 172.21.15.59:0/694235574 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0037070 con 0x7ffad80d15b0 2024-04-23T14:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.945+0000 7ffad6a00640 1 -- 172.21.15.59:0/694235574 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac002adb0 con 0x7ffad80d15b0 2024-04-23T14:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.945+0000 7ffad6a00640 1 -- 172.21.15.59:0/694235574 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0032660 con 0x7ffad80d15b0 2024-04-23T14:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 -- 172.21.15.59:0/694235574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad80d15b0 msgr2=0x7ffad80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 --2- 172.21.15.59:0/694235574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad80d15b0 0x7ffad80cf280 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097a0 tx=0x7ffac002a060 comp rx=0 tx=0).stop 2024-04-23T14:44:29.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 -- 172.21.15.59:0/694235574 shutdown_connections 2024-04-23T14:44:29.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 --2- 172.21.15.59:0/694235574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 0x7ffad80cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 --2- 172.21.15.59:0/694235574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad80d15b0 0x7ffad80cf280 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 -- 172.21.15.59:0/694235574 >> 172.21.15.59:0/694235574 conn(0x7ffad808bf90 msgr2=0x7ffad80d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:29.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 -- 172.21.15.59:0/694235574 shutdown_connections 2024-04-23T14:44:29.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.946+0000 7ffaddc00640 1 -- 172.21.15.59:0/694235574 wait complete. 2024-04-23T14:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.948+0000 7ffaddc00640 1 Processor -- start 2024-04-23T14:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.948+0000 7ffaddc00640 1 -- start start 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.949+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 0x7ffad81560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.949+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad815ed20 0x7ffad80ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.949+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad815b970 0x7ffad815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.949+0000 7ffaddc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffad80b3600 con 0x7ffad815ed20 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.949+0000 7ffaddc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffad80d3490 con 0x7ffad815b970 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.949+0000 7ffaddc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ffad80b3780 con 0x7ffad80b4f20 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.949+0000 7ffad7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad815ed20 0x7ffad80ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffad7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad815ed20 0x7ffad80ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48334/0 (socket says 172.21.15.59:48334) 2024-04-23T14:44:29.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffad7400640 1 -- 172.21.15.59:0/2228855888 learned_addr learned my addr 172.21.15.59:0/2228855888 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffad7400640 1 -- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 msgr2=0x7ffad81560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffad7400640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 0x7ffad81560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffad7400640 1 -- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad815b970 msgr2=0x7ffad815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffadd200640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad815b970 0x7ffad815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffad7400640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad815b970 0x7ffad815be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffad7400640 1 -- 172.21.15.59:0/2228855888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad81565e0 con 0x7ffad815ed20 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.950+0000 7ffadd200640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad815b970 0x7ffad815be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.951+0000 7ffad7400640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad815ed20 0x7ffad80ca8c0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7ffac002a5d0 tx=0x7ffac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.951+0000 7ffacfe00640 1 -- 172.21.15.59:0/2228855888 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0037040 con 0x7ffad815ed20 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.951+0000 7ffacfe00640 1 -- 172.21.15.59:0/2228855888 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac0004030 con 0x7ffad815ed20 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.951+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad80ccb40 con 0x7ffad815ed20 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.951+0000 7ffacfe00640 1 -- 172.21.15.59:0/2228855888 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004200 con 0x7ffad815ed20 2024-04-23T14:44:29.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.951+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad80cd100 con 0x7ffad815ed20 2024-04-23T14:44:29.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.952+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad804fd90 con 0x7ffad815ed20 2024-04-23T14:44:29.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.954+0000 7ffacfe00640 1 -- 172.21.15.59:0/2228855888 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffac003a070 con 0x7ffad815ed20 2024-04-23T14:44:29.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.955+0000 7ffacfe00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffab406c8f0 0x7ffab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:29.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.955+0000 7ffacfe00640 1 -- 172.21.15.59:0/2228855888 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ffac00ac100 con 0x7ffad815ed20 2024-04-23T14:44:29.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.955+0000 7ffad7e00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffab406c8f0 0x7ffab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:29.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.956+0000 7ffad7e00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffab406c8f0 0x7ffab406edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffac8000f60 tx=0x7ffac80078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:29.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:29.960+0000 7ffacfe00640 1 -- 172.21.15.59:0/2228855888 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffac004a030 con 0x7ffad815ed20 2024-04-23T14:44:30.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.163+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffad80b5600 con 0x7ffad815ed20 2024-04-23T14:44:30.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.164+0000 7ffacfe00640 1 -- 172.21.15.59:0/2228855888 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7ffac0041030 con 0x7ffad815ed20 2024-04-23T14:44:30.165 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffab406c8f0 msgr2=0x7ffab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffab406c8f0 0x7ffab406edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffac8000f60 tx=0x7ffac80078f0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad815ed20 msgr2=0x7ffad80ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad815ed20 0x7ffad80ca8c0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7ffac002a5d0 tx=0x7ffac00046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 shutdown_connections 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ffab406c8f0 0x7ffab406edb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad815b970 0x7ffad815be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffad815ed20 0x7ffad80ca8c0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.166+0000 7ffaddc00640 1 --2- 172.21.15.59:0/2228855888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffad80b4f20 0x7ffad81560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.167+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 >> 172.21.15.59:0/2228855888 conn(0x7ffad808bf90 msgr2=0x7ffad80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.167+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 shutdown_connections 2024-04-23T14:44:30.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.167+0000 7ffaddc00640 1 -- 172.21.15.59:0/2228855888 wait complete. 2024-04-23T14:44:30.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:44:30.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.387+0000 7fda18a00640 1 Processor -- start 2024-04-23T14:44:30.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.387+0000 7fda18a00640 1 -- start start 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda18a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda140d05c0 0x7fda141416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda18a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fda1415f060 0x7fda140d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda18a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda140d0ee0 con 0x7fda1415c250 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda18a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fda14141c20 con 0x7fda1415f060 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda18a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fda1415bdc0 con 0x7fda140d05c0 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda11e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fda1415f060 0x7fda140d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda11e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fda1415f060 0x7fda140d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35062/0 (socket says 172.21.15.59:35062) 2024-04-23T14:44:30.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.389+0000 7fda11e00640 1 -- 172.21.15.59:0/1182511101 learned_addr learned my addr 172.21.15.59:0/1182511101 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.390+0000 7fda12800640 1 --2- 172.21.15.59:0/1182511101 >> v2:172.21.15.76:3300/0 conn(0x7fda140d05c0 0x7fda141416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.390+0000 7fda11e00640 1 -- 172.21.15.59:0/1182511101 >> v2:172.21.15.76:3300/0 conn(0x7fda140d05c0 msgr2=0x7fda141416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.390+0000 7fda11e00640 1 --2- 172.21.15.59:0/1182511101 >> v2:172.21.15.76:3300/0 conn(0x7fda140d05c0 0x7fda141416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.390+0000 7fda11e00640 1 -- 172.21.15.59:0/1182511101 >> v1:172.21.15.76:6789/0 conn(0x7fda1415c250 legacy=0x7fda141555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.390+0000 7fda11e00640 1 -- 172.21.15.59:0/1182511101 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1415ba70 con 0x7fda1415f060 2024-04-23T14:44:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.390+0000 7fda12800640 1 --2- 172.21.15.59:0/1182511101 >> v2:172.21.15.76:3300/0 conn(0x7fda140d05c0 0x7fda141416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:30.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.391+0000 7fda11e00640 1 --2- 172.21.15.59:0/1182511101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda140d09a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fda140d11b0 tx=0x7fda0002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4682f10d6378db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:30.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.391+0000 7fda11400640 1 -- 172.21.15.59:0/1182511101 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000325a0 con 0x7fda1415f060 2024-04-23T14:44:30.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.391+0000 7fda11400640 1 -- 172.21.15.59:0/1182511101 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda00038070 con 0x7fda1415f060 2024-04-23T14:44:30.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.391+0000 7fda11400640 1 -- 172.21.15.59:0/1182511101 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00032960 con 0x7fda1415f060 2024-04-23T14:44:30.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 -- 172.21.15.59:0/1182511101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 msgr2=0x7fda140d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:30.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 --2- 172.21.15.59:0/1182511101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda140d09a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fda140d11b0 tx=0x7fda0002c8e0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 -- 172.21.15.59:0/1182511101 shutdown_connections 2024-04-23T14:44:30.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 --2- 172.21.15.59:0/1182511101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda140d09a0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 --2- 172.21.15.59:0/1182511101 >> v2:172.21.15.76:3300/0 conn(0x7fda140d05c0 0x7fda141416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 -- 172.21.15.59:0/1182511101 >> 172.21.15.59:0/1182511101 conn(0x7fda1406bdf0 msgr2=0x7fda1406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:30.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 -- 172.21.15.59:0/1182511101 shutdown_connections 2024-04-23T14:44:30.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.392+0000 7fda18a00640 1 -- 172.21.15.59:0/1182511101 wait complete. 2024-04-23T14:44:30.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.394+0000 7fda18a00640 1 Processor -- start 2024-04-23T14:44:30.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.395+0000 7fda18a00640 1 -- start start 2024-04-23T14:44:30.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda18a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda140d05c0 0x7fda14150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:30.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda18a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1415c250 0x7fda1409a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:30.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda18a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda1409a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:30.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda18a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda1415fdc0 con 0x7fda1415f060 2024-04-23T14:44:30.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda18a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda1415fae0 con 0x7fda1415c250 2024-04-23T14:44:30.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda18a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fda1415f4f0 con 0x7fda140d05c0 2024-04-23T14:44:30.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda13200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda1409a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:30.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda13200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda1409a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35066/0 (socket says 172.21.15.59:35066) 2024-04-23T14:44:30.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.396+0000 7fda13200640 1 -- 172.21.15.59:0/2775384657 learned_addr learned my addr 172.21.15.59:0/2775384657 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:30.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda13200640 1 -- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda140d05c0 msgr2=0x7fda14150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:30.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda12800640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda140d05c0 0x7fda14150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:30.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda11e00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1415c250 0x7fda1409a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:30.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda13200640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda140d05c0 0x7fda14150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda13200640 1 -- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1415c250 msgr2=0x7fda1409a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda13200640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1415c250 0x7fda1409a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda13200640 1 -- 172.21.15.59:0/2775384657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda14156040 con 0x7fda1415f060 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda11e00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1415c250 0x7fda1409a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda13200640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda1409a6b0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c780 tx=0x7fda0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda0aa00640 1 -- 172.21.15.59:0/2775384657 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c017070 con 0x7fda1415f060 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda0aa00640 1 -- 172.21.15.59:0/2775384657 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c003da0 con 0x7fda1415f060 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda0aa00640 1 -- 172.21.15.59:0/2775384657 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c0052b0 con 0x7fda1415f060 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.397+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda14156320 con 0x7fda1415f060 2024-04-23T14:44:30.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.398+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141568e0 con 0x7fda1415f060 2024-04-23T14:44:30.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.399+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1404fd90 con 0x7fda1415f060 2024-04-23T14:44:30.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.400+0000 7fda0aa00640 1 -- 172.21.15.59:0/2775384657 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda0c01a070 con 0x7fda1415f060 2024-04-23T14:44:30.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.401+0000 7fda0aa00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9f406c940 0x7fd9f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:30.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.401+0000 7fda0aa00640 1 -- 172.21.15.59:0/2775384657 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fda0c08b2a0 con 0x7fda1415f060 2024-04-23T14:44:30.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.401+0000 7fda12800640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9f406c940 0x7fd9f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:30.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.402+0000 7fda12800640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9f406c940 0x7fd9f406ee00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda04008850 tx=0x7fda04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:30.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.405+0000 7fda0aa00640 1 -- 172.21.15.59:0/2775384657 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda0c059620 con 0x7fda1415f060 2024-04-23T14:44:30.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.621+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda140cfec0 con 0x7fda1415f060 2024-04-23T14:44:30.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.622+0000 7fda0aa00640 1 -- 172.21.15.59:0/2775384657 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v115) ==== 80+0+122 (secure 0 0 0) 0x7fda0c058f20 con 0x7fda1415f060 2024-04-23T14:44:30.623 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6837/1061483588 2024-04-24T14:44:14.736925+0000 2024-04-23T14:44:30.624 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6836/1061483588 2024-04-24T14:44:14.736925+0000 2024-04-23T14:44:30.624 INFO:teuthology.orchestra.run.smithi059.stderr:listed 2 entries 2024-04-23T14:44:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9f406c940 msgr2=0x7fd9f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9f406c940 0x7fd9f406ee00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda04008850 tx=0x7fda04008040 comp rx=0 tx=0).stop 2024-04-23T14:44:30.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 msgr2=0x7fda1409a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda1409a6b0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c780 tx=0x7fda0c00cc40 comp rx=0 tx=0).stop 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 shutdown_connections 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd9f406c940 0x7fd9f406ee00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda1415f060 0x7fda1409a6b0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1415c250 0x7fda1409a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 --2- 172.21.15.59:0/2775384657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fda140d05c0 0x7fda14150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 >> 172.21.15.59:0/2775384657 conn(0x7fda1406bdf0 msgr2=0x7fda1415df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 shutdown_connections 2024-04-23T14:44:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:30.625+0000 7fda18a00640 1 -- 172.21.15.59:0/2775384657 wait complete. 2024-04-23T14:44:30.645 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:44:30.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:30.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:30.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[77904]: fuse finished with error 0 and tester_r 0 2024-04-23T14:44:30.729 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:44:36.831 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:44:36.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:36.831 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:44:36.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:37.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.108+0000 7f2c5f200640 1 Processor -- start 2024-04-23T14:44:37.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.108+0000 7f2c5f200640 1 -- start start 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.109+0000 7f2c5f200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c6015e7c0 0x7f2c600d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.109+0000 7f2c5f200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c600d2630 0x7f2c600b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.109+0000 7f2c5f200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c600cf030 con 0x7f2c600b6530 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.109+0000 7f2c5f200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2c600b7000 con 0x7f2c600d2630 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.109+0000 7f2c5f200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2c600cf620 con 0x7f2c6015e7c0 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c600d2630 0x7f2c600b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5d400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c600d2630 0x7f2c600b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35078/0 (socket says 172.21.15.59:35078) 2024-04-23T14:44:37.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5d400640 1 -- 172.21.15.59:0/563971811 learned_addr learned my addr 172.21.15.59:0/563971811 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5de00640 1 --2- 172.21.15.59:0/563971811 >> v2:172.21.15.76:3300/0 conn(0x7f2c6015e7c0 0x7f2c600d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5d400640 1 -- 172.21.15.59:0/563971811 >> v2:172.21.15.76:3300/0 conn(0x7f2c6015e7c0 msgr2=0x7f2c600d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5d400640 1 --2- 172.21.15.59:0/563971811 >> v2:172.21.15.76:3300/0 conn(0x7f2c6015e7c0 0x7f2c600d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5d400640 1 -- 172.21.15.59:0/563971811 >> v1:172.21.15.76:6789/0 conn(0x7f2c600b6530 legacy=0x7f2c600d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.110+0000 7f2c5d400640 1 -- 172.21.15.59:0/563971811 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c600b6a40 con 0x7f2c600d2630 2024-04-23T14:44:37.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.111+0000 7f2c5d400640 1 --2- 172.21.15.59:0/563971811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600d2630 0x7f2c600b5ff0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f2c6008c6d0 tx=0x7f2c4c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecc43162d61b250 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.112+0000 7f2c5ca00640 1 -- 172.21.15.59:0/563971811 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c037070 con 0x7f2c600d2630 2024-04-23T14:44:37.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.112+0000 7f2c5ca00640 1 -- 172.21.15.59:0/563971811 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4c032440 con 0x7f2c600d2630 2024-04-23T14:44:37.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.112+0000 7f2c5ca00640 1 -- 172.21.15.59:0/563971811 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c032810 con 0x7f2c600d2630 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.112+0000 7f2c5f200640 1 -- 172.21.15.59:0/563971811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600d2630 msgr2=0x7f2c600b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.112+0000 7f2c5f200640 1 --2- 172.21.15.59:0/563971811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600d2630 0x7f2c600b5ff0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f2c6008c6d0 tx=0x7f2c4c02a230 comp rx=0 tx=0).stop 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.113+0000 7f2c5f200640 1 -- 172.21.15.59:0/563971811 shutdown_connections 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.113+0000 7f2c5f200640 1 --2- 172.21.15.59:0/563971811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600d2630 0x7f2c600b5ff0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.113+0000 7f2c5f200640 1 --2- 172.21.15.59:0/563971811 >> v2:172.21.15.76:3300/0 conn(0x7f2c6015e7c0 0x7f2c600d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.113+0000 7f2c5f200640 1 -- 172.21.15.59:0/563971811 >> 172.21.15.59:0/563971811 conn(0x7f2c6009d920 msgr2=0x7f2c6008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.113+0000 7f2c5f200640 1 -- 172.21.15.59:0/563971811 shutdown_connections 2024-04-23T14:44:37.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.113+0000 7f2c5f200640 1 -- 172.21.15.59:0/563971811 wait complete. 2024-04-23T14:44:37.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.114+0000 7f2c5f200640 1 Processor -- start 2024-04-23T14:44:37.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.114+0000 7f2c5f200640 1 -- start start 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.115+0000 7f2c5f200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600b6530 0x7f2c6014fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.115+0000 7f2c5f200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c600d2630 0x7f2c600c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.115+0000 7f2c5f200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015e7c0 0x7f2c600cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.115+0000 7f2c5f200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c600d1a80 con 0x7f2c600b6530 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.115+0000 7f2c5f200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c600cf2b0 con 0x7f2c6015e7c0 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.115+0000 7f2c5f200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2c600d1c00 con 0x7f2c600d2630 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600b6530 0x7f2c6014fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600b6530 0x7f2c6014fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35092/0 (socket says 172.21.15.59:35092) 2024-04-23T14:44:37.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 -- 172.21.15.59:0/2489198738 learned_addr learned my addr 172.21.15.59:0/2489198738 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5d400640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c600d2630 0x7f2c600c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 -- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c600d2630 msgr2=0x7f2c600c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c600d2630 0x7f2c600c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 -- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015e7c0 msgr2=0x7f2c600cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015e7c0 0x7f2c600cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5de00640 1 -- 172.21.15.59:0/2489198738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c600b3200 con 0x7f2c600b6530 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.116+0000 7f2c5d400640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c600d2630 0x7f2c600c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.117+0000 7f2c5de00640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600b6530 0x7f2c6014fa00 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000d5f0 tx=0x7f2c5000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.117+0000 7f2c56000640 1 -- 172.21.15.59:0/2489198738 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500125f0 con 0x7f2c600b6530 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.117+0000 7f2c56000640 1 -- 172.21.15.59:0/2489198738 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c50019070 con 0x7f2c600b6530 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.117+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6015a6a0 con 0x7f2c600b6530 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.117+0000 7f2c56000640 1 -- 172.21.15.59:0/2489198738 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500129d0 con 0x7f2c600b6530 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.117+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6015ac60 con 0x7f2c600b6530 2024-04-23T14:44:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.118+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c6004fd90 con 0x7f2c600b6530 2024-04-23T14:44:37.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.119+0000 7f2c56000640 1 -- 172.21.15.59:0/2489198738 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c50012b70 con 0x7f2c600b6530 2024-04-23T14:44:37.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.120+0000 7f2c56000640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c4006c920 0x7f2c4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.120+0000 7f2c56000640 1 -- 172.21.15.59:0/2489198738 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f2c5008c0d0 con 0x7f2c600b6530 2024-04-23T14:44:37.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.120+0000 7f2c5d400640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c4006c920 0x7f2c4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.121+0000 7f2c5d400640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c4006c920 0x7f2c4006ede0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c034000 tx=0x7f2c4c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.125+0000 7f2c56000640 1 -- 172.21.15.59:0/2489198738 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c5005a340 con 0x7f2c600b6530 2024-04-23T14:44:37.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.379+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c600d3350 con 0x7f2c600b6530 2024-04-23T14:44:37.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.380+0000 7f2c56000640 1 -- 172.21.15.59:0/2489198738 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5188 (secure 0 0 0) 0x7f2c50059c40 con 0x7f2c600b6530 2024-04-23T14:44:37.381 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:37.381 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:18.865935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3383882209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3383882209},{"type":"v1","addr":"172.21.15.59:6837","nonce":3383882209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9522,"qdb_cluster":[9522]},"id":8}]} 2024-04-23T14:44:37.382 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 106 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c4006c920 msgr2=0x7f2c4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c4006c920 0x7f2c4006ede0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c034000 tx=0x7f2c4c02e000 comp rx=0 tx=0).stop 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600b6530 msgr2=0x7f2c6014fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600b6530 0x7f2c6014fa00 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000d5f0 tx=0x7f2c5000dab0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 shutdown_connections 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2c4006c920 0x7f2c4006ede0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c600d2630 0x7f2c600c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015e7c0 0x7f2c600cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 --2- 172.21.15.59:0/2489198738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c600b6530 0x7f2c6014fa00 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 >> 172.21.15.59:0/2489198738 conn(0x7f2c6009d920 msgr2=0x7f2c6015c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.382+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 shutdown_connections 2024-04-23T14:44:37.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.383+0000 7f2c5f200640 1 -- 172.21.15.59:0/2489198738 wait complete. 2024-04-23T14:44:37.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-23T14:44:37.644 INFO:teuthology.orchestra.run.smithi059.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-23T14:44:37.644 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-23T14:44:37.644 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-23T14:44:37.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.759+0000 7f4a67e72f40 1 Processor -- start 2024-04-23T14:44:37.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.759+0000 7f4a67e72f40 1 -- start start 2024-04-23T14:44:37.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a67e72f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349d00940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a67e72f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349c63de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a67e72f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 0x565349c64320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a67e72f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565349c66230 con 0x565349c7a160 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a67e72f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565349bdccc0 con 0x565349d06cd0 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a67e72f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x565349c7c590 con 0x565349d070b0 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a65000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349d00940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a65000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349d00940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35110/0 (socket says 172.21.15.59:35110) 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.760+0000 7f4a65000640 1 -- 172.21.15.59:0/2381798359 learned_addr learned my addr 172.21.15.59:0/2381798359 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a64600640 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349c63de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a65000640 1 -- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 msgr2=0x565349c64320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a65a00640 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 0x565349c64320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a65000640 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 0x565349c64320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a65000640 1 -- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 msgr2=0x565349c63de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a65000640 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349c63de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a65000640 1 -- 172.21.15.59:0/2381798359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565349d04810 con 0x565349c7a160 2024-04-23T14:44:37.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a64600640 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349c63de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:37.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.761+0000 7f4a65000640 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349d00940 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400a500 tx=0x7f4a5400a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5015af603f70e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a63200640 1 -- 172.21.15.59:0/2381798359 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540115f0 con 0x565349c7a160 2024-04-23T14:44:37.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a63200640 1 -- 172.21.15.59:0/2381798359 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a54018070 con 0x565349c7a160 2024-04-23T14:44:37.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a63200640 1 -- 172.21.15.59:0/2381798359 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540119b0 con 0x565349c7a160 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2381798359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 msgr2=0x565349d00940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349d00940 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400a500 tx=0x7f4a5400a9c0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2381798359 shutdown_connections 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 0x565349c64320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349c63de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2381798359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349d00940 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.762+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2381798359 >> 172.21.15.59:0/2381798359 conn(0x565349c81290 msgr2=0x565349c7e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.763+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2381798359 shutdown_connections 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.763+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2381798359 wait complete. 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.763+0000 7f4a67e72f40 1 Processor -- start 2024-04-23T14:44:37.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.764+0000 7f4a67e72f40 1 -- start start 2024-04-23T14:44:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.764+0000 7f4a67e72f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349c7e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.764+0000 7f4a67e72f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349d02b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.764+0000 7f4a67e72f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 0x565349d037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.764+0000 7f4a67e72f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x565349d05360 con 0x565349c7a160 2024-04-23T14:44:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.764+0000 7f4a67e72f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565349d04a80 con 0x565349d06cd0 2024-04-23T14:44:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.764+0000 7f4a67e72f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x565349c62130 con 0x565349d070b0 2024-04-23T14:44:37.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349c7e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349c7e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35120/0 (socket says 172.21.15.59:35120) 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 -- 172.21.15.59:0/2505166740 learned_addr learned my addr 172.21.15.59:0/2505166740 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65a00640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 0x565349d037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a64600640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349d02b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 -- 172.21.15.59:0/2505166740 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 msgr2=0x565349d037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x565349d070b0 0x565349d037c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 -- 172.21.15.59:0/2505166740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 msgr2=0x565349d02b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349d02b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a65000640 1 -- 172.21.15.59:0/2505166740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565349d0d590 con 0x565349c7a160 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.765+0000 7f4a64600640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565349d06cd0 0x565349d02b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.766+0000 7f4a65000640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349c7e5a0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4a540136e0 tx=0x7f4a540033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.766+0000 7f4a61e00640 1 -- 172.21.15.59:0/2505166740 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011e60 con 0x565349c7a160 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.766+0000 7f4a61e00640 1 -- 172.21.15.59:0/2505166740 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a54005c40 con 0x565349c7a160 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.766+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565349c389d0 con 0x565349c7a160 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.766+0000 7f4a61e00640 1 -- 172.21.15.59:0/2505166740 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54013cf0 con 0x565349c7a160 2024-04-23T14:44:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.766+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565349d03090 con 0x565349c7a160 2024-04-23T14:44:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.768+0000 7f4a61e00640 1 -- 172.21.15.59:0/2505166740 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a54024020 con 0x565349c7a160 2024-04-23T14:44:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.770+0000 7f4a61e00640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a4406b320 0x7f4a4406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.770+0000 7f4a61e00640 1 -- 172.21.15.59:0/2505166740 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f4a5408c580 con 0x565349c7a160 2024-04-23T14:44:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.770+0000 7f4a64600640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a4406b320 0x7f4a4406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.770+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x565349d03d00 0x565349c38100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.770+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x565349c61cf0 con 0x565349d03d00 2024-04-23T14:44:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.771+0000 7f4a65a00640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x565349d03d00 0x565349c38100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.771+0000 7f4a64600640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a4406b320 0x7f4a4406d7e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4a50056000 tx=0x7f4a50050000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.771+0000 7f4a65a00640 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x565349d03d00 0x565349c38100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a65a00640 1 -- 172.21.15.59:0/2505166740 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f4a5c00e440 con 0x565349d03d00 2024-04-23T14:44:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x565349d03d00 msgr2=0x565349c38100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x565349d03d00 0x565349c38100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a4406b320 msgr2=0x7f4a4406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4a4406b320 0x7f4a4406d7e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4a50056000 tx=0x7f4a50050000 comp rx=0 tx=0).stop 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 msgr2=0x565349c7e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 --2- 172.21.15.59:0/2505166740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x565349c7a160 0x565349c7e5a0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4a540136e0 tx=0x7f4a540033a0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a65a00640 1 -- 172.21.15.59:0/2505166740 reap_dead start 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 shutdown_connections 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 >> 172.21.15.59:0/2505166740 conn(0x565349c81290 msgr2=0x565349c63de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 shutdown_connections 2024-04-23T14:44:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.772+0000 7f4a67e72f40 1 -- 172.21.15.59:0/2505166740 wait complete. 2024-04-23T14:44:37.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:44:37.778 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-23T14:44:37.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.867+0000 7f0a28d04f40 1 Processor -- start 2024-04-23T14:44:37.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.867+0000 7f0a28d04f40 1 -- start start 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.868+0000 7f0a28d04f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaac0b1d0 0x55deaace75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.868+0000 7f0a28d04f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 0x55deaac55660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.868+0000 7f0a28d04f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaace5ad0 0x55deaac3b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.868+0000 7f0a28d04f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55deaabb4cc0 con 0x55deaac0b1d0 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.868+0000 7f0a28d04f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55deaabb8d00 con 0x55deaace5ad0 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.868+0000 7f0a28d04f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55deaac0d9c0 con 0x55deaaccaac0 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaac0b1d0 0x55deaace75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaac0b1d0 0x55deaace75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35136/0 (socket says 172.21.15.59:35136) 2024-04-23T14:44:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 -- 172.21.15.59:0/3820603172 learned_addr learned my addr 172.21.15.59:0/3820603172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a25600640 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 0x55deaac55660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26a00640 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaace5ad0 0x55deaac3b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 -- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 msgr2=0x55deaac55660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 0x55deaac55660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 -- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaace5ad0 msgr2=0x55deaac3b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaace5ad0 0x55deaac3b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26000640 1 -- 172.21.15.59:0/3820603172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deaace63c0 con 0x55deaac0b1d0 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.869+0000 7f0a26a00640 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaace5ad0 0x55deaac3b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.870+0000 7f0a26000640 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaac0b1d0 0x55deaace75e0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800a440 tx=0x7f0a18051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e1cbe3cc9db099 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.870+0000 7f0a1fe00640 1 -- 172.21.15.59:0/3820603172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18063070 con 0x55deaac0b1d0 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.870+0000 7f0a1fe00640 1 -- 172.21.15.59:0/3820603172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a18055b90 con 0x55deaac0b1d0 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.870+0000 7f0a1fe00640 1 -- 172.21.15.59:0/3820603172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18068460 con 0x55deaac0b1d0 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.870+0000 7f0a28d04f40 1 -- 172.21.15.59:0/3820603172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaac0b1d0 msgr2=0x55deaace75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.870+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaac0b1d0 0x55deaace75e0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800a440 tx=0x7f0a18051000 comp rx=0 tx=0).stop 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 -- 172.21.15.59:0/3820603172 shutdown_connections 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 0x55deaac55660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaace5ad0 0x55deaac3b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/3820603172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaac0b1d0 0x55deaace75e0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 -- 172.21.15.59:0/3820603172 >> 172.21.15.59:0/3820603172 conn(0x55deaac57cc0 msgr2=0x55deaac59ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 -- 172.21.15.59:0/3820603172 shutdown_connections 2024-04-23T14:44:37.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 -- 172.21.15.59:0/3820603172 wait complete. 2024-04-23T14:44:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.871+0000 7f0a28d04f40 1 Processor -- start 2024-04-23T14:44:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.872+0000 7f0a28d04f40 1 -- start start 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a28d04f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaac0b1d0 0x55deaac56cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a28d04f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 0x55deaacdfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a28d04f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaace5ad0 0x55deaac3f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a28d04f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55deaace4ec0 con 0x55deaace5ad0 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a28d04f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55deaac5a1e0 con 0x55deaac0b1d0 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a28d04f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55deaace7b20 con 0x55deaaccaac0 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a26a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaace5ad0 0x55deaac3f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a26a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaace5ad0 0x55deaac3f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35142/0 (socket says 172.21.15.59:35142) 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a26a00640 1 -- 172.21.15.59:0/136455089 learned_addr learned my addr 172.21.15.59:0/136455089 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a26000640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaac0b1d0 0x55deaac56cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a26a00640 1 -- 172.21.15.59:0/136455089 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 msgr2=0x55deaacdfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a25600640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 0x55deaacdfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.873+0000 7f0a26a00640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55deaaccaac0 0x55deaacdfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a26a00640 1 -- 172.21.15.59:0/136455089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaac0b1d0 msgr2=0x55deaac56cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a26a00640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55deaac0b1d0 0x55deaac56cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a26a00640 1 -- 172.21.15.59:0/136455089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deaac3f110 con 0x55deaace5ad0 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a26a00640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaace5ad0 0x55deaac3f3f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c150 tx=0x7f0a2000c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a1ea00640 1 -- 172.21.15.59:0/136455089 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20003650 con 0x55deaace5ad0 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a1ea00640 1 -- 172.21.15.59:0/136455089 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a200037f0 con 0x55deaace5ad0 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a1ea00640 1 -- 172.21.15.59:0/136455089 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20003be0 con 0x55deaace5ad0 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55deaacc5f90 con 0x55deaace5ad0 2024-04-23T14:44:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.874+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deaacc6550 con 0x55deaace5ad0 2024-04-23T14:44:37.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.877+0000 7f0a1ea00640 1 -- 172.21.15.59:0/136455089 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a2001a020 con 0x55deaace5ad0 2024-04-23T14:44:37.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.878+0000 7f0a1ea00640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a0806b2d0 0x7f0a0806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.878+0000 7f0a1ea00640 1 -- 172.21.15.59:0/136455089 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f0a2008a650 con 0x55deaace5ad0 2024-04-23T14:44:37.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.878+0000 7f0a26000640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a0806b2d0 0x7f0a0806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.878+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55deaaccbc00 0x55deaaccc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.878+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55deaaccc550 con 0x55deaaccbc00 2024-04-23T14:44:37.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.878+0000 7f0a25600640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55deaaccbc00 0x55deaaccc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.879+0000 7f0a26000640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a0806b2d0 0x7f0a0806d790 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0a18064000 tx=0x7f0a18053000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.879+0000 7f0a25600640 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55deaaccbc00 0x55deaaccc010 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.879+0000 7f0a25600640 1 -- 172.21.15.59:0/136455089 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f0a1400c660 con 0x55deaaccbc00 2024-04-23T14:44:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.879+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55deaaccbc00 msgr2=0x55deaaccc010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55deaaccbc00 0x55deaaccc010 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a0806b2d0 msgr2=0x7f0a0806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f0a0806b2d0 0x7f0a0806d790 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0a18064000 tx=0x7f0a18053000 comp rx=0 tx=0).stop 2024-04-23T14:44:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaace5ad0 msgr2=0x55deaac3f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 --2- 172.21.15.59:0/136455089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55deaace5ad0 0x55deaac3f3f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c150 tx=0x7f0a2000c610 comp rx=0 tx=0).stop 2024-04-23T14:44:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a26a00640 1 -- 172.21.15.59:0/136455089 reap_dead start 2024-04-23T14:44:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 shutdown_connections 2024-04-23T14:44:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 >> 172.21.15.59:0/136455089 conn(0x55deaac57cc0 msgr2=0x55deaac562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 shutdown_connections 2024-04-23T14:44:37.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.880+0000 7f0a28d04f40 1 -- 172.21.15.59:0/136455089 wait complete. 2024-04-23T14:44:37.886 INFO:teuthology.orchestra.run.smithi059.stdout:mytag123 2024-04-23T14:44:37.886 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-23T14:44:37.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.972+0000 7ff6cdd1ef40 1 Processor -- start 2024-04-23T14:44:37.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.973+0000 7ff6cdd1ef40 1 -- start start 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.973+0000 7ff6cdd1ef40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990bca1d0 0x556990ca5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.973+0000 7ff6cdd1ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990c89ac0 0x556990c9ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.973+0000 7ff6cdd1ef40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.973+0000 7ff6cdd1ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556990bcc9c0 con 0x556990c89ac0 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.973+0000 7ff6cdd1ef40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556990b73cc0 con 0x556990ca4ad0 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.973+0000 7ff6cdd1ef40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556990bcab20 con 0x556990bca1d0 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990c89ac0 0x556990c9ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990c89ac0 0x556990c9ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35164/0 (socket says 172.21.15.59:35164) 2024-04-23T14:44:37.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 -- 172.21.15.59:0/2297701754 learned_addr learned my addr 172.21.15.59:0/2297701754 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6ca400640 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990bca1d0 0x556990ca5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 -- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990bca1d0 msgr2=0x556990ca5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990bca1d0 0x556990ca5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6cae00640 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 -- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 msgr2=0x556990bf9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 -- 172.21.15.59:0/2297701754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556990c8ce40 con 0x556990c89ac0 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.974+0000 7ff6c9a00640 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990c89ac0 0x556990c9ffb0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400a3f0 tx=0x7ff6b400a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f6c1b620777915 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6c3e00640 1 -- 172.21.15.59:0/2297701754 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4010070 con 0x556990c89ac0 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6c3e00640 1 -- 172.21.15.59:0/2297701754 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b400c8f0 con 0x556990c89ac0 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6c3e00640 1 -- 172.21.15.59:0/2297701754 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400cce0 con 0x556990c89ac0 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/2297701754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990c89ac0 msgr2=0x556990c9ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990c89ac0 0x556990c9ffb0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400a3f0 tx=0x7ff6b400a8b0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/2297701754 shutdown_connections 2024-04-23T14:44:37.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990c89ac0 0x556990c9ffb0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/2297701754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990bca1d0 0x556990ca5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/2297701754 >> 172.21.15.59:0/2297701754 conn(0x556990c16cc0 msgr2=0x556990c18ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/2297701754 shutdown_connections 2024-04-23T14:44:37.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.975+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/2297701754 wait complete. 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.976+0000 7ff6cdd1ef40 1 Processor -- start 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.976+0000 7ff6cdd1ef40 1 -- start start 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.977+0000 7ff6cdd1ef40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990bca1d0 0x556990bfd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.977+0000 7ff6cdd1ef40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990c89ac0 0x556990c889b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.977+0000 7ff6cdd1ef40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.977+0000 7ff6cdd1ef40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556990c19310 con 0x556990bca1d0 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.977+0000 7ff6cdd1ef40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556990c8d120 con 0x556990ca4ad0 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.977+0000 7ff6cdd1ef40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x556990bfb760 con 0x556990c89ac0 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.977+0000 7ff6ca400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990bca1d0 0x556990bfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990bca1d0 0x556990bfd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35178/0 (socket says 172.21.15.59:35178) 2024-04-23T14:44:37.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 -- 172.21.15.59:0/3351767713 learned_addr learned my addr 172.21.15.59:0/3351767713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6c9a00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990c89ac0 0x556990c889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6cae00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 -- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990c89ac0 msgr2=0x556990c889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x556990c89ac0 0x556990c889b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 -- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 msgr2=0x556990bf9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 -- 172.21.15.59:0/3351767713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556990bfdb70 con 0x556990bca1d0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6cae00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556990ca4ad0 0x556990bf9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.978+0000 7ff6ca400640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990bca1d0 0x556990bfd590 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc000f80 tx=0x7ff6bc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.979+0000 7ff6c2a00640 1 -- 172.21.15.59:0/3351767713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc055050 con 0x556990bca1d0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.979+0000 7ff6c2a00640 1 -- 172.21.15.59:0/3351767713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc064640 con 0x556990bca1d0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.979+0000 7ff6c2a00640 1 -- 172.21.15.59:0/3351767713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc064a10 con 0x556990bca1d0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.979+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556990bfde50 con 0x556990bca1d0 2024-04-23T14:44:37.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.979+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556990bfe410 con 0x556990bca1d0 2024-04-23T14:44:37.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.981+0000 7ff6c2a00640 1 -- 172.21.15.59:0/3351767713 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6bc058020 con 0x556990bca1d0 2024-04-23T14:44:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.982+0000 7ff6c2a00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff6a806b2d0 0x7ff6a806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.982+0000 7ff6c2a00640 1 -- 172.21.15.59:0/3351767713 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ff6bc0ce5b0 con 0x556990bca1d0 2024-04-23T14:44:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.982+0000 7ff6c9a00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff6a806b2d0 0x7ff6a806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.982+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x556990d39f50 0x556990d3a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:37.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.982+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x556990c88ef0 con 0x556990d39f50 2024-04-23T14:44:37.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.983+0000 7ff6c9a00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff6a806b2d0 0x7ff6a806d790 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x556990ca5060 tx=0x7ff6b4017040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.983+0000 7ff6cae00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x556990d39f50 0x556990d3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:37.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.984+0000 7ff6cae00640 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x556990d39f50 0x556990d3a350 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:37.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cae00640 1 -- 172.21.15.59:0/3351767713 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7ff6c4009690 con 0x556990d39f50 2024-04-23T14:44:37.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x556990d39f50 msgr2=0x556990d3a350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x556990d39f50 0x556990d3a350 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff6a806b2d0 msgr2=0x7ff6a806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff6a806b2d0 0x7ff6a806d790 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x556990ca5060 tx=0x7ff6b4017040 comp rx=0 tx=0).stop 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990bca1d0 msgr2=0x556990bfd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 --2- 172.21.15.59:0/3351767713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556990bca1d0 0x556990bfd590 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc000f80 tx=0x7ff6bc003330 comp rx=0 tx=0).stop 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cae00640 1 -- 172.21.15.59:0/3351767713 reap_dead start 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 shutdown_connections 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 >> 172.21.15.59:0/3351767713 conn(0x556990c16cc0 msgr2=0x556990c18ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 shutdown_connections 2024-04-23T14:44:37.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:37.985+0000 7ff6cdd1ef40 1 -- 172.21.15.59:0/3351767713 wait complete. 2024-04-23T14:44:37.991 INFO:teuthology.orchestra.run.smithi059.stdout:mytag123 2024-04-23T14:44:37.991 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-23T14:44:38.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 Processor -- start 2024-04-23T14:44:38.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 -- start start 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 0x55e209adc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209abeac0 0x55e209a2fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209ad9ad0 0x55e209a30280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e209a019c0 con 0x55e209abeac0 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e2099a8cc0 con 0x55e2099ff1d0 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.093+0000 7fd9bcb28f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e2099ffb20 con 0x55e209ad9ad0 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209abeac0 0x55e209a2fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209abeac0 0x55e209a2fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35204/0 (socket says 172.21.15.59:35204) 2024-04-23T14:44:38.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 -- 172.21.15.59:0/386672576 learned_addr learned my addr 172.21.15.59:0/386672576 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b9e00640 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 0x55e209adc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9ba800640 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209ad9ad0 0x55e209a30280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 -- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209ad9ad0 msgr2=0x55e209a30280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209ad9ad0 0x55e209a30280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 -- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 msgr2=0x55e209adc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 0x55e209adc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.094+0000 7fd9b1400640 1 -- 172.21.15.59:0/386672576 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e209adb8d0 con 0x55e209abeac0 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.095+0000 7fd9b1400640 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209abeac0 0x55e209a2fd40 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400a440 tx=0x7fd9a400a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195f7ee9b38cc952 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.095+0000 7fd9b8a00640 1 -- 172.21.15.59:0/386672576 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400b930 con 0x55e209abeac0 2024-04-23T14:44:38.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.095+0000 7fd9b8a00640 1 -- 172.21.15.59:0/386672576 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a400f070 con 0x55e209abeac0 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.095+0000 7fd9b8a00640 1 -- 172.21.15.59:0/386672576 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400bcf0 con 0x55e209abeac0 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/386672576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209abeac0 msgr2=0x55e209a2fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209abeac0 0x55e209a2fd40 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400a440 tx=0x7fd9a400a900 comp rx=0 tx=0).stop 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/386672576 shutdown_connections 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209ad9ad0 0x55e209a30280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209abeac0 0x55e209a2fd40 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/386672576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 0x55e209adc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/386672576 >> 172.21.15.59:0/386672576 conn(0x55e209a4bcc0 msgr2=0x55e209ad9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/386672576 shutdown_connections 2024-04-23T14:44:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/386672576 wait complete. 2024-04-23T14:44:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.096+0000 7fd9bcb28f40 1 Processor -- start 2024-04-23T14:44:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.097+0000 7fd9bcb28f40 1 -- start start 2024-04-23T14:44:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9bcb28f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 0x55e209aba960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9bcb28f40 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209abeac0 0x55e209a2fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9bcb28f40 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209ad9ad0 0x55e209a300a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9bcb28f40 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e209ad86c0 con 0x55e209ad9ad0 2024-04-23T14:44:38.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9bcb28f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e209ad8470 con 0x55e2099ff1d0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9bcb28f40 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55e209ad89a0 con 0x55e209abeac0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209ad9ad0 0x55e209a300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209ad9ad0 0x55e209a300a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35208/0 (socket says 172.21.15.59:35208) 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 -- 172.21.15.59:0/3327728391 learned_addr learned my addr 172.21.15.59:0/3327728391 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9b9e00640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 0x55e209aba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9b1400640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209abeac0 0x55e209a2fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 -- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209abeac0 msgr2=0x55e209a2fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55e209abeac0 0x55e209a2fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 -- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 msgr2=0x55e209aba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e2099ff1d0 0x55e209aba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.098+0000 7fd9ba800640 1 -- 172.21.15.59:0/3327728391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e209ad7b90 con 0x55e209ad9ad0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.099+0000 7fd9ba800640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209ad9ad0 0x55e209a300a0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400a4f0 tx=0x7fd9b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.099+0000 7fd9b3400640 1 -- 172.21.15.59:0/3327728391 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4069070 con 0x55e209ad9ad0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.099+0000 7fd9b3400640 1 -- 172.21.15.59:0/3327728391 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b4051af0 con 0x55e209ad9ad0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.099+0000 7fd9b3400640 1 -- 172.21.15.59:0/3327728391 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4055050 con 0x55e209ad9ad0 2024-04-23T14:44:38.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.099+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e209ad7e70 con 0x55e209ad9ad0 2024-04-23T14:44:38.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.099+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e209abdc90 con 0x55e209ad9ad0 2024-04-23T14:44:38.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.101+0000 7fd9b3400640 1 -- 172.21.15.59:0/3327728391 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9b406c070 con 0x55e209ad9ad0 2024-04-23T14:44:38.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.102+0000 7fd9b3400640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd99806b2d0 0x7fd99806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.102+0000 7fd9b3400640 1 -- 172.21.15.59:0/3327728391 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fd9b40ce4a0 con 0x55e209ad9ad0 2024-04-23T14:44:38.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.103+0000 7fd9b9e00640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd99806b2d0 0x7fd99806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.103+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55e2099ebbe0 0x55e209a32090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.103+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55e209ac0810 con 0x55e2099ebbe0 2024-04-23T14:44:38.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.103+0000 7fd9b9e00640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd99806b2d0 0x7fd99806d790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0012e0 tx=0x7fd9ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.103+0000 7fd9b1400640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55e2099ebbe0 0x55e209a32090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.104+0000 7fd9b1400640 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55e2099ebbe0 0x55e209a32090 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.105+0000 7fd9b1400640 1 -- 172.21.15.59:0/3327728391 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fd9a400c900 con 0x55e2099ebbe0 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.105+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55e2099ebbe0 msgr2=0x55e209a32090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.105+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55e2099ebbe0 0x55e209a32090 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.105+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd99806b2d0 msgr2=0x7fd99806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.105+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd99806b2d0 0x7fd99806d790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0012e0 tx=0x7fd9ac008040 comp rx=0 tx=0).stop 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.105+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209ad9ad0 msgr2=0x55e209a300a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.105+0000 7fd9bcb28f40 1 --2- 172.21.15.59:0/3327728391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e209ad9ad0 0x55e209a300a0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400a4f0 tx=0x7fd9b4002fb0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.106+0000 7fd9ba800640 1 -- 172.21.15.59:0/3327728391 reap_dead start 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.106+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 shutdown_connections 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.106+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 >> 172.21.15.59:0/3327728391 conn(0x55e209a4bcc0 msgr2=0x55e209a4dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.106+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 shutdown_connections 2024-04-23T14:44:38.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.106+0000 7fd9bcb28f40 1 -- 172.21.15.59:0/3327728391 wait complete. 2024-04-23T14:44:38.112 INFO:teuthology.orchestra.run.smithi059.stdout:mytag123 2024-04-23T14:44:38.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-23T14:44:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.312+0000 7f39f6200640 1 Processor -- start 2024-04-23T14:44:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.312+0000 7f39f6200640 1 -- start start 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.313+0000 7f39f6200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39f015ed20 0x7f39f00d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.313+0000 7f39f6200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39f00b3600 0x7f39f00d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.313+0000 7f39f6200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3f10 0x7f39f00b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.313+0000 7f39f6200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f39f015f100 con 0x7f39f015ed20 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.313+0000 7f39f6200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f39f00b5da0 con 0x7f39f00b3600 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.313+0000 7f39f6200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00d0a20 con 0x7f39f00b3f10 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.314+0000 7f39f4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39f015ed20 0x7f39f00d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.314+0000 7f39f4e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39f015ed20 0x7f39f00d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35220/0 (socket says 172.21.15.59:35220) 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.314+0000 7f39f4e00640 1 -- 172.21.15.59:0/3891866213 learned_addr learned my addr 172.21.15.59:0/3891866213 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.314+0000 7f39efe00640 1 --2- 172.21.15.59:0/3891866213 >> v2:172.21.15.76:3300/0 conn(0x7f39f00b3600 0x7f39f00d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.314+0000 7f39f5800640 1 --2- 172.21.15.59:0/3891866213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3f10 0x7f39f00b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.314+0000 7f39f4e00640 1 -- 172.21.15.59:0/3891866213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3f10 msgr2=0x7f39f00b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.314+0000 7f39f4e00640 1 --2- 172.21.15.59:0/3891866213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3f10 0x7f39f00b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.315+0000 7f39f4e00640 1 -- 172.21.15.59:0/3891866213 >> v2:172.21.15.76:3300/0 conn(0x7f39f00b3600 msgr2=0x7f39f00d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.315+0000 7f39f4e00640 1 --2- 172.21.15.59:0/3891866213 >> v2:172.21.15.76:3300/0 conn(0x7f39f00b3600 0x7f39f00d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.315+0000 7f39f4e00640 1 -- 172.21.15.59:0/3891866213 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f00d2ed0 con 0x7f39f015ed20 2024-04-23T14:44:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.316+0000 7f39f4e00640 1 --2- 172.21.15.59:0/3891866213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f015ed20 0x7f39f00d0ea0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f39f00d4520 tx=0x7f39d802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3269c58e32ab7c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.316+0000 7f39ef400640 1 -- 172.21.15.59:0/3891866213 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8037070 con 0x7f39f015ed20 2024-04-23T14:44:38.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.316+0000 7f39ef400640 1 -- 172.21.15.59:0/3891866213 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d802ae00 con 0x7f39f015ed20 2024-04-23T14:44:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.316+0000 7f39ef400640 1 -- 172.21.15.59:0/3891866213 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8032660 con 0x7f39f015ed20 2024-04-23T14:44:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 -- 172.21.15.59:0/3891866213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f015ed20 msgr2=0x7f39f00d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 --2- 172.21.15.59:0/3891866213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f015ed20 0x7f39f00d0ea0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f39f00d4520 tx=0x7f39d802a0b0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 -- 172.21.15.59:0/3891866213 shutdown_connections 2024-04-23T14:44:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 --2- 172.21.15.59:0/3891866213 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3f10 0x7f39f00b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 --2- 172.21.15.59:0/3891866213 >> v2:172.21.15.76:3300/0 conn(0x7f39f00b3600 0x7f39f00d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 --2- 172.21.15.59:0/3891866213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f015ed20 0x7f39f00d0ea0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 -- 172.21.15.59:0/3891866213 >> 172.21.15.59:0/3891866213 conn(0x7f39f008bf90 msgr2=0x7f39f00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 -- 172.21.15.59:0/3891866213 shutdown_connections 2024-04-23T14:44:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.317+0000 7f39f6200640 1 -- 172.21.15.59:0/3891866213 wait complete. 2024-04-23T14:44:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.319+0000 7f39f6200640 1 Processor -- start 2024-04-23T14:44:38.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.319+0000 7f39f6200640 1 -- start start 2024-04-23T14:44:38.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.320+0000 7f39f6200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3600 0x7f39f01f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.320+0000 7f39f6200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f00b3f10 0x7f39f01f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.320+0000 7f39f6200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f015ed20 0x7f39f015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.320+0000 7f39f6200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00b48e0 con 0x7f39f00b3f10 2024-04-23T14:44:38.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.320+0000 7f39f6200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00d3490 con 0x7f39f015ed20 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.320+0000 7f39f6200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00b4a60 con 0x7f39f00b3600 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f00b3f10 0x7f39f01f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f00b3f10 0x7f39f01f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35226/0 (socket says 172.21.15.59:35226) 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 -- 172.21.15.59:0/553568868 learned_addr learned my addr 172.21.15.59:0/553568868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39f4e00640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3600 0x7f39f01f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39f5800640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f015ed20 0x7f39f015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 -- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3600 msgr2=0x7f39f01f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3600 0x7f39f01f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 -- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f015ed20 msgr2=0x7f39f015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f015ed20 0x7f39f015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.321+0000 7f39efe00640 1 -- 172.21.15.59:0/553568868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f00c8270 con 0x7f39f00b3f10 2024-04-23T14:44:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.322+0000 7f39efe00640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f00b3f10 0x7f39f01f5070 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f39e0007e90 tx=0x7f39e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.322+0000 7f39ecc00640 1 -- 172.21.15.59:0/553568868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0014420 con 0x7f39f00b3f10 2024-04-23T14:44:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.322+0000 7f39ecc00640 1 -- 172.21.15.59:0/553568868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e0019070 con 0x7f39f00b3f10 2024-04-23T14:44:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.322+0000 7f39ecc00640 1 -- 172.21.15.59:0/553568868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0014800 con 0x7f39f00b3f10 2024-04-23T14:44:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.322+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f00c8550 con 0x7f39f00b3f10 2024-04-23T14:44:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.322+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f00c8b10 con 0x7f39f00b3f10 2024-04-23T14:44:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.324+0000 7f39ecc00640 1 -- 172.21.15.59:0/553568868 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39e00149a0 con 0x7f39f00b3f10 2024-04-23T14:44:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.325+0000 7f39ecc00640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.326+0000 7f39ecc00640 1 -- 172.21.15.59:0/553568868 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f39e008c4c0 con 0x7f39f00b3f10 2024-04-23T14:44:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.326+0000 7f39f4e00640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.326+0000 7f39fa67b740 1 Processor -- start 2024-04-23T14:44:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.326+0000 7f39fa67b740 1 -- start start 2024-04-23T14:44:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39f4e00640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f39cc06c8f0 0x7f39cc06edb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034000 tx=0x7f39d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa50920790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa509502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa509507e0 0x55fa509255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39fa67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa5088a1f0 con 0x55fa509507e0 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa50830350 con 0x55fa5096de00 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fa508861b0 con 0x55fa50950f20 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.327+0000 7f39f5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa509507e0 0x55fa509255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa509507e0 0x55fa509255f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35234/0 (socket says 172.21.15.59:35234) 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 -- 172.21.15.59:0/2939921119 learned_addr learned my addr 172.21.15.59:0/2939921119 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39efe00640 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa509502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f4e00640 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa50920790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 -- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 msgr2=0x55fa509502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa509502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 -- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 msgr2=0x55fa50920790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa50920790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 -- 172.21.15.59:0/2939921119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa50884f50 con 0x55fa509507e0 2024-04-23T14:44:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.328+0000 7f39f5800640 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa509507e0 0x55fa509255f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b8b0 tx=0x7f39e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b00fb590ef16f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.329+0000 7f39e5600640 1 -- 172.21.15.59:0/2939921119 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8018020 con 0x55fa509507e0 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.329+0000 7f39e5600640 1 -- 172.21.15.59:0/2939921119 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e800f9c0 con 0x55fa509507e0 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.329+0000 7f39e5600640 1 -- 172.21.15.59:0/2939921119 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e800fd90 con 0x55fa509507e0 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.329+0000 7f39fa67b740 1 -- 172.21.15.59:0/2939921119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa509507e0 msgr2=0x55fa509255f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.329+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa509507e0 0x55fa509255f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b8b0 tx=0x7f39e800bd70 comp rx=0 tx=0).stop 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 -- 172.21.15.59:0/2939921119 shutdown_connections 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa509502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa509507e0 0x55fa509255f0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2939921119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa50920790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 -- 172.21.15.59:0/2939921119 >> 172.21.15.59:0/2939921119 conn(0x55fa50922a00 msgr2=0x55fa5096b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 -- 172.21.15.59:0/2939921119 shutdown_connections 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 -- 172.21.15.59:0/2939921119 wait complete. 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.330+0000 7f39fa67b740 1 -- start start 2024-04-23T14:44:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.331+0000 7f39fa67b740 1 Processor -- start 2024-04-23T14:44:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa50950f20 0x55fa50951410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa5096de00 0x55fa5096d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5094bfd0 0x55fa5097bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39fa67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa50924bb0 con 0x55fa50950f20 2024-04-23T14:44:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa50843690 con 0x55fa5094bfd0 2024-04-23T14:44:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fa50923fa0 con 0x55fa5096de00 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39fa67b740 10 client.0 _ll_register_callbacks cb 0x55fa5087f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa50950f20 0x55fa50951410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa50950f20 0x55fa50951410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35244/0 (socket says 172.21.15.59:35244) 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 -- 172.21.15.59:0/2326071948 learned_addr learned my addr 172.21.15.59:0/2326071948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39efe00640 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa5096de00 0x55fa5096d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f5800640 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5094bfd0 0x55fa5097bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 -- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa5096de00 msgr2=0x55fa5096d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa5096de00 0x55fa5096d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 -- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5094bfd0 msgr2=0x55fa5097bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5094bfd0 0x55fa5097bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.332+0000 7f39f4e00640 1 -- 172.21.15.59:0/2326071948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa50842920 con 0x55fa50950f20 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39f4e00640 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa50950f20 0x55fa50951410 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009960 tx=0x7f39d802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8541f40b44f48069 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39de000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:38.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39de000640 1 -- 172.21.15.59:0/2326071948 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d804d060 con 0x55fa50950f20 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39de000640 1 -- 172.21.15.59:0/2326071948 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f39d803eb70 con 0x55fa50950f20 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e800f9c0 con 0x55fa50950f20 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39de000640 1 -- 172.21.15.59:0/2326071948 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f39d8053050 con 0x55fa50950f20 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39fa67b740 10 client.9717 fetch_fsmap learned FSMap version 106 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.333+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55fa506667d0 con 0x55fa50950f20 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39de000640 1 -- 172.21.15.59:0/2326071948 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e800f9c0 con 0x55fa50950f20 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39de000640 1 -- 172.21.15.59:0/2326071948 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f39d8056020 con 0x55fa50950f20 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39fa67b740 10 client.9717 fetch_fsmap finished waiting for FSMap version 106 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39fa67b740 10 client.9717 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39fa67b740 20 client.9717 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa5094fad0 0x55fa5094ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39fa67b740 4 client.9717 mds_command: new command op to 9522 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 --> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa50897c10 con 0x55fa5094fad0 2024-04-23T14:44:38.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.334+0000 7f39efe00640 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa5094fad0 0x55fa5094ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.335+0000 7f39efe00640 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa5094fad0 0x55fa5094ff10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f14e06fcb38cd server_cookie=bd329a16157cdd44 in_seq=0 out_seq=0 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.335+0000 7f39de000640 10 client.9717 ms_handle_connect on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.335+0000 7f39de000640 1 -- 172.21.15.59:0/2326071948 <== mds.0 v2:172.21.15.59:6836/3383882209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa50897c10 con 0x55fa5094fad0 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.335+0000 7f39de000640 10 client.9717 handle_command_reply: tid=0 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.335+0000 7f39fa67b740 1 client.9717 shutdown 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa50950f20 msgr2=0x55fa50951410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa50950f20 0x55fa50951410 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009960 tx=0x7f39d802c070 comp rx=0 tx=0).stop 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 shutdown_connections 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa5094fad0 0x55fa5094ff10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa5096de00 0x55fa5096d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5094bfd0 0x55fa5097bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 --2- 172.21.15.59:0/2326071948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa50950f20 0x55fa50951410 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39de000640 0 client.9717 ms_handle_reset on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 >> 172.21.15.59:0/2326071948 conn(0x55fa509507e0 msgr2=0x55fa50925720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 shutdown_connections 2024-04-23T14:44:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 1 -- 172.21.15.59:0/2326071948 wait complete. 2024-04-23T14:44:38.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.336+0000 7f39fa67b740 20 client.9717 trim_cache size 0 max 16384 2024-04-23T14:44:38.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.353+0000 7f39fa67b740 1 Processor -- start 2024-04-23T14:44:38.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.353+0000 7f39fa67b740 1 -- start start 2024-04-23T14:44:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa50962180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa50920170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa509206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39fa67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0200f00 con 0x55fa5094bfd0 2024-04-23T14:44:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f39f01fffe0 con 0x55fa5096de00 2024-04-23T14:44:38.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00b1810 con 0x55fa50950f20 2024-04-23T14:44:38.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39f4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa50962180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39f4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa50962180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35248/0 (socket says 172.21.15.59:35248) 2024-04-23T14:44:38.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.354+0000 7f39f4e00640 1 -- 172.21.15.59:0/919405008 learned_addr learned my addr 172.21.15.59:0/919405008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39f4e00640 1 -- 172.21.15.59:0/919405008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 msgr2=0x55fa50920170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39f4e00640 1 --2- 172.21.15.59:0/919405008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa50920170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39f4e00640 1 -- 172.21.15.59:0/919405008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 msgr2=0x55fa509206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39f4e00640 1 --2- 172.21.15.59:0/919405008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa509206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39f4e00640 1 -- 172.21.15.59:0/919405008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa50951780 con 0x55fa5094bfd0 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39f4e00640 1 --2- 172.21.15.59:0/919405008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa50962180 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f39d802a0f0 tx=0x7f39d803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0df948c09a85dae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39e4c00640 1 -- 172.21.15.59:0/919405008 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d805c070 con 0x55fa5094bfd0 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39e4c00640 1 -- 172.21.15.59:0/919405008 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d803eb70 con 0x55fa5094bfd0 2024-04-23T14:44:38.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.355+0000 7f39e4c00640 1 -- 172.21.15.59:0/919405008 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8037040 con 0x55fa5094bfd0 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 -- 172.21.15.59:0/919405008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 msgr2=0x55fa50962180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 --2- 172.21.15.59:0/919405008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa50962180 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f39d802a0f0 tx=0x7f39d803b000 comp rx=0 tx=0).stop 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 -- 172.21.15.59:0/919405008 shutdown_connections 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 --2- 172.21.15.59:0/919405008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa50920170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 --2- 172.21.15.59:0/919405008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa509206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 --2- 172.21.15.59:0/919405008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa50962180 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 -- 172.21.15.59:0/919405008 >> 172.21.15.59:0/919405008 conn(0x55fa509507e0 msgr2=0x55fa5097adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 -- 172.21.15.59:0/919405008 shutdown_connections 2024-04-23T14:44:38.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.356+0000 7f39fa67b740 1 -- 172.21.15.59:0/919405008 wait complete. 2024-04-23T14:44:38.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.357+0000 7f39fa67b740 1 -- start start 2024-04-23T14:44:38.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.357+0000 7f39fa67b740 1 Processor -- start 2024-04-23T14:44:38.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa509ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa5097bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39fa67b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa5096a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39fa67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00b1e60 con 0x55fa5094bfd0 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa50952070 con 0x55fa5096de00 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39fa67b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fa5097c050 con 0x55fa50950f20 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39fa67b740 10 client.0 _ll_register_callbacks cb 0x7f39d8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39f4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa509ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39f4e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa509ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35256/0 (socket says 172.21.15.59:35256) 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.358+0000 7f39f4e00640 1 -- 172.21.15.59:0/3311337718 learned_addr learned my addr 172.21.15.59:0/3311337718 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f5800640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa5096a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39efe00640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa5097bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f4e00640 1 -- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 msgr2=0x55fa5097bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f4e00640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa5097bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f4e00640 1 -- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 msgr2=0x55fa5096a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f4e00640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa5096a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f4e00640 1 -- 172.21.15.59:0/3311337718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5059c3f0 con 0x55fa5094bfd0 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39efe00640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa5097bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:38.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f5800640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa5096a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:38.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39f4e00640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa509ce380 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f39d80090b0 tx=0x7f39d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bb40793020e1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39de000640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:38.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39de000640 1 -- 172.21.15.59:0/3311337718 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d805c070 con 0x55fa5094bfd0 2024-04-23T14:44:38.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39de000640 1 -- 172.21.15.59:0/3311337718 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f39d803cd60 con 0x55fa5094bfd0 2024-04-23T14:44:38.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.359+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d803eb70 con 0x55fa5094bfd0 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39de000640 1 -- 172.21.15.59:0/3311337718 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f39d80485b0 con 0x55fa5094bfd0 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 10 client.9723 fetch_fsmap learned FSMap version 106 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55fa5055c0f0 con 0x55fa5094bfd0 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39de000640 1 -- 172.21.15.59:0/3311337718 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d803eb70 con 0x55fa5094bfd0 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39de000640 1 -- 172.21.15.59:0/3311337718 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f39d8056040 con 0x55fa5094bfd0 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 10 client.9723 fetch_fsmap finished waiting for FSMap version 106 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 10 client.9723 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 20 client.9723 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:38.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa50920170 0x55fa5096a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 4 client.9723 mds_command: new command op to 9522 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T14:44:38.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.360+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 --> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa50891a80 con 0x55fa50920170 2024-04-23T14:44:38.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.361+0000 7f39efe00640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa50920170 0x55fa5096a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.361+0000 7f39efe00640 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa50920170 0x55fa5096a9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826c05f77a49b15e server_cookie=c7555fda4ebd9ff8 in_seq=0 out_seq=0 2024-04-23T14:44:38.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.361+0000 7f39de000640 10 client.9723 ms_handle_connect on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39de000640 1 -- 172.21.15.59:0/3311337718 <== mds.0 v2:172.21.15.59:6836/3383882209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa50891a80 con 0x55fa50920170 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39de000640 10 client.9723 handle_command_reply: tid=0 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 client.9723 shutdown 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 msgr2=0x55fa509ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa509ce380 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f39d80090b0 tx=0x7f39d8004730 comp rx=0 tx=0).stop 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 shutdown_connections 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:6836/3383882209,v1:172.21.15.59:6837/3383882209] conn(0x55fa50920170 0x55fa5096a9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa50950f20 0x55fa5097bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa5096de00 0x55fa5096a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39de000640 0 client.9723 ms_handle_reset on v2:172.21.15.59:6836/3383882209 2024-04-23T14:44:38.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 --2- 172.21.15.59:0/3311337718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa5094bfd0 0x55fa509ce380 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 >> 172.21.15.59:0/3311337718 conn(0x55fa509507e0 msgr2=0x55fa50962180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.376+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 shutdown_connections 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.377+0000 7f39fa67b740 1 -- 172.21.15.59:0/3311337718 wait complete. 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.377+0000 7f39fa67b740 20 client.9723 trim_cache size 0 max 16384 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stdout: "message": "", 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stdout: "return_code": 0 2024-04-23T14:44:38.379 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.377+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f39cc06c8f0 msgr2=0x7f39cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.377+0000 7f39f6200640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f39cc06c8f0 0x7f39cc06edb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034000 tx=0x7f39d802e000 comp rx=0 tx=0).stop 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.377+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f00b3f10 msgr2=0x7f39f01f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.377+0000 7f39f6200640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f00b3f10 0x7f39f01f5070 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f39e0007e90 tx=0x7f39e000e4b0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 shutdown_connections 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f39cc06c8f0 0x7f39cc06edb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f015ed20 0x7f39f015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f00b3f10 0x7f39f01f5070 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 --2- 172.21.15.59:0/553568868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00b3600 0x7f39f01f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 >> 172.21.15.59:0/553568868 conn(0x7f39f008bf90 msgr2=0x7f39f006dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 shutdown_connections 2024-04-23T14:44:38.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.378+0000 7f39f6200640 1 -- 172.21.15.59:0/553568868 wait complete. 2024-04-23T14:44:38.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:44:38.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.599+0000 7ff8cb200640 1 Processor -- start 2024-04-23T14:44:38.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.599+0000 7ff8cb200640 1 -- start start 2024-04-23T14:44:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.601+0000 7ff8cb200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff8cc15c670 0x7ff8cc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.601+0000 7ff8cb200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8cc0b4cf0 con 0x7ff8cc0d0cd0 2024-04-23T14:44:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.601+0000 7ff8cb200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8cc0ccdd0 con 0x7ff8cc0b4230 2024-04-23T14:44:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.601+0000 7ff8cb200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc15ca50 con 0x7ff8cc15c670 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.601+0000 7ff8ca800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff8cc0d0cd0 0x7ff8cc06ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:33360/0 (socket says 172.21.15.59:33360) 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.601+0000 7ff8ca800640 1 -- 172.21.15.59:0/688925615 learned_addr learned my addr 172.21.15.59:0/688925615 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9e00640 1 --2- 172.21.15.59:0/688925615 >> v2:172.21.15.76:3300/0 conn(0x7ff8cc15c670 0x7ff8cc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316190474 0 0) 0x7ff8cc0b4cf0 con 0x7ff8cc0d0cd0 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8b0003520 con 0x7ff8cc0d0cd0 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3675682161 0 0) 0x7ff8b0003520 con 0x7ff8cc0d0cd0 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 >> v2:172.21.15.76:3300/0 conn(0x7ff8cc15c670 msgr2=0x7ff8cc0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 --2- 172.21.15.59:0/688925615 >> v2:172.21.15.76:3300/0 conn(0x7ff8cc15c670 0x7ff8cc0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 >> v1:172.21.15.76:6789/0 conn(0x7ff8cc0b4230 legacy=0x7ff8cc0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc0cd0b0 con 0x7ff8cc0d0cd0 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7ff8c40019f0 con 0x7ff8cc0d0cd0 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 >> v1:172.21.15.59:6789/0 conn(0x7ff8cc0d0cd0 legacy=0x7ff8cc06ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.602+0000 7ff8c9400640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8b00039b0 0x7ff8b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8c9400640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8b0004800 0x7ff8b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8c9400640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b000c030 con 0x7ff8b00051c0 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b000c1e0 con 0x7ff8b00039b0 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b000c3e0 con 0x7ff8b0004800 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8ca800640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8c9e00640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8b00039b0 0x7ff8b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8ca800640 1 -- 172.21.15.59:0/688925615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8b0004800 msgr2=0x7ff8b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8ca800640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8b0004800 0x7ff8b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8ca800640 1 -- 172.21.15.59:0/688925615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8b00039b0 msgr2=0x7ff8b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8ca800640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8b00039b0 0x7ff8b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.603+0000 7ff8ca800640 1 -- 172.21.15.59:0/688925615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b0003520 con 0x7ff8b00051c0 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.604+0000 7ff8ca800640 1 -- 172.21.15.59:0/688925615 reap_dead start 2024-04-23T14:44:38.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.604+0000 7ff8ca800640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8b000b9c0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400a840 tx=0x7ff8b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b8c0d9a01ad46d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c400a460 con 0x7ff8b00051c0 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c402f070 con 0x7ff8b00051c0 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8c9400640 1 -- 172.21.15.59:0/688925615 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c402bcb0 con 0x7ff8b00051c0 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8cb200640 1 -- 172.21.15.59:0/688925615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 msgr2=0x7ff8b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8cb200640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8b000b9c0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400a840 tx=0x7ff8b0005830 comp rx=0 tx=0).stop 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8cb200640 1 -- 172.21.15.59:0/688925615 shutdown_connections 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8cb200640 1 --2- 172.21.15.59:0/688925615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8b000b9c0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.605+0000 7ff8cb200640 1 -- 172.21.15.59:0/688925615 >> 172.21.15.59:0/688925615 conn(0x7ff8cc06bdf0 msgr2=0x7ff8cc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.606+0000 7ff8cb200640 1 -- 172.21.15.59:0/688925615 shutdown_connections 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.606+0000 7ff8cb200640 1 -- 172.21.15.59:0/688925615 wait complete. 2024-04-23T14:44:38.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.607+0000 7ff8cb200640 1 Processor -- start 2024-04-23T14:44:38.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.607+0000 7ff8cb200640 1 -- start start 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.608+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8cc0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.608+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8cc155fb0 0x7ff8cc151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8cc1523a0 0x7ff8cc09c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8cb200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0b3e60 con 0x7ff8b00051c0 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8cb200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0ccb80 con 0x7ff8cc155fb0 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8cb200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc0b3fe0 con 0x7ff8cc1523a0 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8cc0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8cc0b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35280/0 (socket says 172.21.15.59:35280) 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 -- 172.21.15.59:0/3820965634 learned_addr learned my addr 172.21.15.59:0/3820965634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8ca800640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8cc1523a0 0x7ff8cc09c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c3e00640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8cc155fb0 0x7ff8cc151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 -- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8cc1523a0 msgr2=0x7ff8cc09c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8cc1523a0 0x7ff8cc09c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 -- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8cc155fb0 msgr2=0x7ff8cc151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8cc155fb0 0x7ff8cc151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c9e00640 1 -- 172.21.15.59:0/3820965634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc0b26f0 con 0x7ff8b00051c0 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.609+0000 7ff8c3e00640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8cc155fb0 0x7ff8cc151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.610+0000 7ff8c9e00640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8cc0b3270 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc011540 tx=0x7ff8bc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.610+0000 7ff8c2000640 1 -- 172.21.15.59:0/3820965634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc015930 con 0x7ff8b00051c0 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.610+0000 7ff8c2000640 1 -- 172.21.15.59:0/3820965634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc015ad0 con 0x7ff8b00051c0 2024-04-23T14:44:38.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.610+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc071da0 con 0x7ff8b00051c0 2024-04-23T14:44:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.610+0000 7ff8c2000640 1 -- 172.21.15.59:0/3820965634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00f460 con 0x7ff8b00051c0 2024-04-23T14:44:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.610+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc072360 con 0x7ff8b00051c0 2024-04-23T14:44:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.611+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc04fd90 con 0x7ff8b00051c0 2024-04-23T14:44:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.613+0000 7ff8c2000640 1 -- 172.21.15.59:0/3820965634 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8bc00f600 con 0x7ff8b00051c0 2024-04-23T14:44:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.614+0000 7ff8c2000640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff8a806c920 0x7ff8a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.614+0000 7ff8c2000640 1 -- 172.21.15.59:0/3820965634 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ff8bc08c4f0 con 0x7ff8b00051c0 2024-04-23T14:44:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.614+0000 7ff8c3e00640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff8a806c920 0x7ff8a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.615+0000 7ff8c3e00640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff8a806c920 0x7ff8a806ede0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc1e8b40 tx=0x7ff8b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.618+0000 7ff8c2000640 1 -- 172.21.15.59:0/3820965634 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8bc019030 con 0x7ff8b00051c0 2024-04-23T14:44:38.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.867+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff8cc0ceab0 con 0x7ff8b00051c0 2024-04-23T14:44:38.869 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:44:38.868+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:44:38.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.871+0000 7ff8c2000640 1 -- 172.21.15.59:0/3820965634 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v107) ==== 161+0+0 (secure 0 0 0) 0x7ff8bc05a880 con 0x7ff8b00051c0 2024-04-23T14:44:38.872 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:44:38.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.875+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff8a806c920 msgr2=0x7ff8a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.875+0000 7ff8cb200640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff8a806c920 0x7ff8a806ede0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc1e8b40 tx=0x7ff8b4009040 comp rx=0 tx=0).stop 2024-04-23T14:44:38.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.875+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 msgr2=0x7ff8cc0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.875+0000 7ff8cb200640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8cc0b3270 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc011540 tx=0x7ff8bc011a00 comp rx=0 tx=0).stop 2024-04-23T14:44:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 shutdown_connections 2024-04-23T14:44:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff8a806c920 0x7ff8a806ede0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff8cc1523a0 0x7ff8cc09c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8cc155fb0 0x7ff8cc151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 --2- 172.21.15.59:0/3820965634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8b00051c0 0x7ff8cc0b3270 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 >> 172.21.15.59:0/3820965634 conn(0x7ff8cc06bdf0 msgr2=0x7ff8cc15e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 shutdown_connections 2024-04-23T14:44:38.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.877+0000 7ff8cb200640 1 -- 172.21.15.59:0/3820965634 wait complete. 2024-04-23T14:44:38.888 INFO:tasks.ceph.mds.c.smithi059.stderr: -2481> 2024-04-23T14:44:21.844+0000 7f6d2fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:44:38.899 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-23T14:44:38.942 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:44:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.981+0000 7f2436610280 1 -- start start 2024-04-23T14:44:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.982+0000 7f2436610280 1 Processor -- start 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.982+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137053cc00 0x5613706ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.982+0000 7f2436610280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137053d800 0x5613706ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.982+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137053dc00 0x5613706acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.982+0000 7f2436610280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5613712b2180 con 0x56137053d800 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.982+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5613712b2300 con 0x56137053cc00 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.982+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5613712b2480 con 0x56137053dc00 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137053d800 0x5613706ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137053d800 0x5613706ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35304/0 (socket says 172.21.15.59:35304) 2024-04-23T14:44:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 -- 172.21.15.59:0/2002069453 learned_addr learned my addr 172.21.15.59:0/2002069453 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2434400640 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137053dc00 0x5613706acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433a00640 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137053cc00 0x5613706ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 -- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137053dc00 msgr2=0x5613706acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137053dc00 0x5613706acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 -- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137053cc00 msgr2=0x5613706ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137053cc00 0x5613706ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2433000640 1 -- 172.21.15.59:0/2002069453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561370570780 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.983+0000 7f2434400640 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137053dc00 0x5613706acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2433000640 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137053d800 0x5613706ac580 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x56137135f3b0 tx=0x5613712d6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792d0b3693c3e5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2430800640 1 -- 172.21.15.59:0/2002069453 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561370562680 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2430800640 1 -- 172.21.15.59:0/2002069453 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561370568700 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2430800640 1 -- 172.21.15.59:0/2002069453 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561370562b60 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2436610280 1 -- 172.21.15.59:0/2002069453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613705714a0 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2436610280 1 -- 172.21.15.59:0/2002069453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561370570000 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2436610280 4 waiting for MDS map... 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2430800640 1 -- 172.21.15.59:0/2002069453 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x561371414600 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2430800640 1 -- 172.21.15.59:0/2002069453 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x561370597180 con 0x56137053d800 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2436610280 4 Got MDS map 107 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.984+0000 7f2436610280 10 main: JournalTool::main 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.985+0000 7f2436610280 4 main: JournalTool: connecting to RADOS... 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.985+0000 7f2436610280 1 Processor -- start 2024-04-23T14:44:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.985+0000 7f2436610280 1 -- start start 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.986+0000 7f2436610280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.986+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 0x5613706ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.986+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.986+0000 7f2436610280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5613712b2600 con 0x56137141b400 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.986+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5613712b2780 con 0x56137141bc00 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.986+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5613712b2900 con 0x56137141b800 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.986+0000 7f2434400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35314/0 (socket says 172.21.15.59:35314) 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 -- 172.21.15.59:0/1625256643 learned_addr learned my addr 172.21.15.59:0/1625256643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2433000640 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 -- 172.21.15.59:0/1625256643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 msgr2=0x5613706ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 0x5613706ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 -- 172.21.15.59:0/1625256643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141bc00 msgr2=0x5613706adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 -- 172.21.15.59:0/1625256643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613705703c0 con 0x56137141b400 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f2434400640 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141b400 0x5613706ad080 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x56137142c5d0 tx=0x5613712d6a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415431e377dede39 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f242e000640 1 -- 172.21.15.59:0/1625256643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613705629c0 con 0x56137141b400 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f242e000640 1 -- 172.21.15.59:0/1625256643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561370568fc0 con 0x56137141b400 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.987+0000 7f242e000640 1 -- 172.21.15.59:0/1625256643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561370562680 con 0x56137141b400 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 -- 172.21.15.59:0/1625256643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141b400 msgr2=0x5613706ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141b400 0x5613706ad080 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x56137142c5d0 tx=0x5613712d6a80 comp rx=0 tx=0).stop 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 -- 172.21.15.59:0/1625256643 shutdown_connections 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 0x5613706ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 --2- 172.21.15.59:0/1625256643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 -- 172.21.15.59:0/1625256643 >> 172.21.15.59:0/1625256643 conn(0x56137141b000 msgr2=0x561370623000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 -- 172.21.15.59:0/1625256643 shutdown_connections 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.988+0000 7f2436610280 1 -- 172.21.15.59:0/1625256643 wait complete. 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.989+0000 7f2436610280 1 Processor -- start 2024-04-23T14:44:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.989+0000 7f2436610280 1 -- start start 2024-04-23T14:44:38.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 0x5613706ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2436610280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2436610280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5613712b3b00 con 0x56137141bc00 2024-04-23T14:44:38.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5613712b3980 con 0x56137141b400 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5613712b3800 con 0x56137141b800 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35330/0 (socket says 172.21.15.59:35330) 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 -- 172.21.15.59:0/2530990832 learned_addr learned my addr 172.21.15.59:0/2530990832 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2434400640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 -- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 msgr2=0x5613706ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433a00640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 0x5613706ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 0x5613706ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 -- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141b400 msgr2=0x5613706ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2433000640 1 -- 172.21.15.59:0/2530990832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561370570780 con 0x56137141bc00 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.990+0000 7f2434400640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.991+0000 7f2433000640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141bc00 0x5613706adb80 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x56137142d3e0 tx=0x5613712d6780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.991+0000 7f242c200640 1 -- 172.21.15.59:0/2530990832 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561370562820 con 0x56137141bc00 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.991+0000 7f242c200640 1 -- 172.21.15.59:0/2530990832 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561370568e00 con 0x56137141bc00 2024-04-23T14:44:38.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.991+0000 7f242c200640 1 -- 172.21.15.59:0/2530990832 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613705629c0 con 0x56137141bc00 2024-04-23T14:44:38.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.991+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613705714a0 con 0x56137141bc00 2024-04-23T14:44:38.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.991+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561370570960 con 0x56137141bc00 2024-04-23T14:44:38.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.991+0000 7f2436610280 4 main: JournalTool: resolving pool 16 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.993+0000 7f242c200640 1 -- 172.21.15.59:0/2530990832 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x561371330c00 con 0x56137141bc00 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.993+0000 7f242c200640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x561371337000 0x5613706ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.993+0000 7f242c200640 1 -- 172.21.15.59:0/2530990832 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x561371334600 con 0x56137141bc00 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.994+0000 7f2436610280 4 main: JournalTool: creating IoCtx.. 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.994+0000 7f2436610280 4 main: Executing for rank 0 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.994+0000 7f2434400640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x561371337000 0x5613706ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.994+0000 7f2434400640 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x561371337000 0x5613706ae100 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x56137142c5d0 tx=0x5613712d6720 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.995+0000 7f2436610280 1 -- start start 2024-04-23T14:44:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.995+0000 7f2436610280 1 Processor -- start 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561371337800 0x5613706ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2436610280 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561371337c00 0x5613706aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2436610280 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561370912000 0x5613706af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2436610280 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5613712b3500 con 0x561371337c00 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5613712b2780 con 0x561371337800 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2436610280 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5613712b2a80 con 0x561370912000 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2433a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561371337c00 0x5613706aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2433a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561371337c00 0x5613706aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35342/0 (socket says 172.21.15.59:35342) 2024-04-23T14:44:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2433a00640 1 -- 172.21.15.59:0/1614811895 learned_addr learned my addr 172.21.15.59:0/1614811895 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2434400640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561371337800 0x5613706ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2433000640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561370912000 0x5613706af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.996+0000 7f2433a00640 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561370912000 msgr2=0x5613706af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2433a00640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561370912000 0x5613706af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2433a00640 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561371337800 msgr2=0x5613706ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2433a00640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561371337800 0x5613706ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2433a00640 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561370571680 con 0x561371337c00 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2434400640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561371337800 0x5613706ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2433a00640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561371337c00 0x5613706aec00 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x56137086d590 tx=0x5613708bb8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614c4bcfaadac487 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2428600640 1 -- 172.21.15.59:0/1614811895 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56137083ba00 con 0x561371337c00 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2428600640 1 -- 172.21.15.59:0/1614811895 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5613705681c0 con 0x561371337c00 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561370570000 con 0x561371337c00 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561370570780 con 0x561371337c00 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2436610280 4 waiting for MDS map... 2024-04-23T14:44:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2428600640 1 -- 172.21.15.59:0/1614811895 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613714a61a0 con 0x561371337c00 2024-04-23T14:44:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.997+0000 7f2428600640 1 -- 172.21.15.59:0/1614811895 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x561370918600 con 0x561371337c00 2024-04-23T14:44:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.998+0000 7f2428600640 1 -- 172.21.15.59:0/1614811895 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x561370597500 con 0x561371337c00 2024-04-23T14:44:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.998+0000 7f2436610280 4 Got MDS map 107 2024-04-23T14:44:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.998+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x561370912400 0x5613706af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.998+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561370912800 con 0x561370912400 2024-04-23T14:44:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.998+0000 7f2434400640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x561370912400 0x5613706af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:38.999+0000 7f2434400640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x561370912400 0x5613706af700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=adf40b1b2a3cc66d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.005+0000 7f2434400640 1 -- 172.21.15.59:0/1614811895 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v116'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5613712c6000 con 0x561370912400 2024-04-23T14:44:39.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.005+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561370912c00 con 0x561370912400 2024-04-23T14:44:39.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2434400640 1 -- 172.21.15.59:0/1614811895 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v116'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5613712c6000 con 0x561370912400 2024-04-23T14:44:39.008 INFO:teuthology.orchestra.run.smithi059.stdout:writing EResetJournal entry 2024-04-23T14:44:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561370913000 con 0x561370912400 2024-04-23T14:44:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x561370913400 0x56137093c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x56137091b400 con 0x561370913400 2024-04-23T14:44:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x56137091b800 0x56137093c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561370912800 con 0x56137091b800 2024-04-23T14:44:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561370912c00 con 0x561370913400 2024-04-23T14:44:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x561370913000 0x56137093cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5613712d5400 con 0x561370913000 2024-04-23T14:44:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5613712d5000 0x56137093d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.007+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x56137091bc00 con 0x5613712d5000 2024-04-23T14:44:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.008+0000 7f2434400640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x56137091b800 0x56137093c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.008+0000 7f2433000640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x561370913400 0x56137093c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.008+0000 7f2433a00640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x561370913000 0x56137093cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.008+0000 7f2433000640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5613712d5000 0x56137093d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.008+0000 7f2434400640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x56137091b800 0x56137093c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=81efe769d2c631d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.009+0000 7f2434400640 1 -- 172.21.15.59:0/1614811895 <== osd.0 v2:172.21.15.59:6817/2024703144 3 ==== osd_op_reply(3 200.00000001 [delete] v116'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5613712c6000 con 0x561370912400 2024-04-23T14:44:39.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.009+0000 7f2433a00640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x561370913000 0x56137093cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=f940345067582293 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.009+0000 7f2433000640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x561370913400 0x56137093c000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=209da63619a0525d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.009+0000 7f2433000640 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5613712d5000 0x56137093d080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=5b44e2b421ed49e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.012+0000 7f2433a00640 1 -- 172.21.15.59:0/1614811895 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(7 200.00000005 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613712c6500 con 0x561370913000 2024-04-23T14:44:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.012+0000 7f2433000640 1 -- 172.21.15.59:0/1614811895 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(4 200.00000002 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613712c6780 con 0x561370913400 2024-04-23T14:44:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.012+0000 7f2426800640 1 -- 172.21.15.59:0/1614811895 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561370956000 con 0x561370912400 2024-04-23T14:44:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.013+0000 7f2434400640 1 -- 172.21.15.59:0/1614811895 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(5 200.00000003 [delete] v116'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613712c6000 con 0x56137091b800 2024-04-23T14:44:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.013+0000 7f2433000640 1 -- 172.21.15.59:0/1614811895 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(6 200.00000004 [delete] v116'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613712c6780 con 0x561370913400 2024-04-23T14:44:39.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.014+0000 7f2434400640 1 -- 172.21.15.59:0/1614811895 <== osd.0 v2:172.21.15.59:6817/2024703144 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v116'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5613712c6000 con 0x561370912400 2024-04-23T14:44:39.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2433000640 1 -- 172.21.15.59:0/1614811895 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(8 200.00000006 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613712c6780 con 0x5613712d5000 2024-04-23T14:44:39.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-23T14:44:39.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x561370912400 msgr2=0x5613706af700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x561370912400 0x5613706af700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x56137091b800 msgr2=0x56137093c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x56137091b800 0x56137093c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x561370913000 msgr2=0x56137093cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x561370913000 0x56137093cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x561370913400 msgr2=0x56137093c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x561370913400 0x56137093c000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5613712d5000 msgr2=0x56137093d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5613712d5000 0x56137093d080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561371337c00 msgr2=0x5613706aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2433a00640 1 -- 172.21.15.59:0/1614811895 reap_dead start 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 --2- 172.21.15.59:0/1614811895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561371337c00 0x5613706aec00 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x56137086d590 tx=0x5613708bb8c0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2433a00640 1 -- 172.21.15.59:0/1614811895 reap_dead start 2024-04-23T14:44:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 shutdown_connections 2024-04-23T14:44:39.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 >> 172.21.15.59:0/1614811895 conn(0x561371337400 msgr2=0x561370623800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 shutdown_connections 2024-04-23T14:44:39.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.019+0000 7f2436610280 1 -- 172.21.15.59:0/1614811895 wait complete. 2024-04-23T14:44:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x561371337000 msgr2=0x5613706ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x561371337000 0x5613706ae100 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x56137142c5d0 tx=0x5613712d6720 comp rx=0 tx=0).stop 2024-04-23T14:44:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141bc00 msgr2=0x5613706adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141bc00 0x5613706adb80 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x56137142d3e0 tx=0x5613712d6780 comp rx=0 tx=0).stop 2024-04-23T14:44:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 shutdown_connections 2024-04-23T14:44:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x561371337000 0x5613706ae100 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137141b800 0x5613706ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137141bc00 0x5613706adb80 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 --2- 172.21.15.59:0/2530990832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137141b400 0x5613706ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 >> 172.21.15.59:0/2530990832 conn(0x56137141b000 msgr2=0x561370623000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 shutdown_connections 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.020+0000 7f2436610280 1 -- 172.21.15.59:0/2530990832 wait complete. 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 -- 172.21.15.59:0/2002069453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137053d800 msgr2=0x5613706ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137053d800 0x5613706ac580 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x56137135f3b0 tx=0x5613712d6600 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 -- 172.21.15.59:0/2002069453 shutdown_connections 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x56137053dc00 0x5613706acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56137053d800 0x5613706ac580 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 --2- 172.21.15.59:0/2002069453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56137053cc00 0x5613706ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 -- 172.21.15.59:0/2002069453 >> 172.21.15.59:0/2002069453 conn(0x56137053d000 msgr2=0x561370622800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 -- 172.21.15.59:0/2002069453 shutdown_connections 2024-04-23T14:44:39.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.021+0000 7f2436610280 1 -- 172.21.15.59:0/2002069453 wait complete. 2024-04-23T14:44:39.028 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.129075, result: writing EResetJournal entry 2024-04-23T14:44:39.028 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-23T14:44:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.171+0000 7f0144c1a740 1 -- start start 2024-04-23T14:44:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.171+0000 7f0144c1a740 1 Processor -- start 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.171+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a1d38c00 0x5654a2b14000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a1d39800 0x5654a2b14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a1d39c00 0x5654a2b14b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0144c1a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654a2ad6180 con 0x5654a1d39c00 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0144c1a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654a2ad6300 con 0x5654a1d38c00 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0144c1a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5654a2ad6480 con 0x5654a1d39800 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a1d39c00 0x5654a2b14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a1d39c00 0x5654a2b14b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35356/0 (socket says 172.21.15.59:35356) 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 -- 172.21.15.59:0/2217660281 learned_addr learned my addr 172.21.15.59:0/2217660281 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142200640 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a1d38c00 0x5654a2b14000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0141800640 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a1d39800 0x5654a2b14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 -- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a1d39800 msgr2=0x5654a2b14580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a1d39800 0x5654a2b14580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 -- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a1d38c00 msgr2=0x5654a2b14000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a1d38c00 0x5654a2b14000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0142c00640 1 -- 172.21.15.59:0/2217660281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a1d65680 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.172+0000 7f0141800640 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a1d39800 0x5654a2b14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f0142c00640 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a1d39c00 0x5654a2b14b00 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x5654a2b5f200 tx=0x5654a2ad8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ad74237e957663 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f013f000640 1 -- 172.21.15.59:0/2217660281 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5654a1d5e9c0 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f013f000640 1 -- 172.21.15.59:0/2217660281 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5654a1d82e00 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f013f000640 1 -- 172.21.15.59:0/2217660281 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5654a1d5e680 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f0144c1a740 1 -- 172.21.15.59:0/2217660281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a1d65860 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f0144c1a740 1 -- 172.21.15.59:0/2217660281 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5654a1d652c0 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f0144c1a740 4 waiting for MDS map... 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.173+0000 7f013f000640 1 -- 172.21.15.59:0/2217660281 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x5654a2c14200 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.174+0000 7f013f000640 1 -- 172.21.15.59:0/2217660281 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x5654a1d91180 con 0x5654a1d39c00 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.174+0000 7f0144c1a740 4 Got MDS map 107 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.174+0000 7f0144c1a740 4 datascan.main: Using metadata pool output 2024-04-23T14:44:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.174+0000 7f0144c1a740 4 datascan.main: connecting to RADOS... 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.174+0000 7f0144c1a740 1 Processor -- start 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.175+0000 7f0144c1a740 1 -- start start 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.175+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c000 0x5654a2b15080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.175+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c400 0x5654a2b15600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.175+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.175+0000 7f0144c1a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654a2ad6600 con 0x5654a2c5c800 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.175+0000 7f0144c1a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654a2ad6780 con 0x5654a2c5c000 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.175+0000 7f0144c1a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5654a2ad6900 con 0x5654a2c5c400 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35362/0 (socket says 172.21.15.59:35362) 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 -- 172.21.15.59:0/261373750 learned_addr learned my addr 172.21.15.59:0/261373750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0142200640 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c400 0x5654a2b15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c000 0x5654a2b15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 -- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c400 msgr2=0x5654a2b15600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c400 0x5654a2b15600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 -- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c000 msgr2=0x5654a2b15080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c000 0x5654a2b15080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.176+0000 7f0141800640 1 -- 172.21.15.59:0/261373750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a1d654a0 con 0x5654a2c5c800 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.177+0000 7f0141800640 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x5654a2b44810 tx=0x5654a2aa4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2604bd9f6a6818e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.177+0000 7f013c800640 1 -- 172.21.15.59:0/261373750 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5654a1d5e9c0 con 0x5654a2c5c800 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.177+0000 7f013c800640 1 -- 172.21.15.59:0/261373750 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5654a1d82a80 con 0x5654a2c5c800 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.177+0000 7f013c800640 1 -- 172.21.15.59:0/261373750 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5654a1d5e680 con 0x5654a2c5c800 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.177+0000 7f0144c1a740 1 -- 172.21.15.59:0/261373750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c800 msgr2=0x5654a2b15b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.177+0000 7f0144c1a740 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x5654a2b44810 tx=0x5654a2aa4c60 comp rx=0 tx=0).stop 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 -- 172.21.15.59:0/261373750 shutdown_connections 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c400 0x5654a2b15600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 --2- 172.21.15.59:0/261373750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c000 0x5654a2b15080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 -- 172.21.15.59:0/261373750 >> 172.21.15.59:0/261373750 conn(0x5654a2b2dc00 msgr2=0x5654a1e25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 -- 172.21.15.59:0/261373750 shutdown_connections 2024-04-23T14:44:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 -- 172.21.15.59:0/261373750 wait complete. 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.178+0000 7f0144c1a740 1 Processor -- start 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.179+0000 7f0144c1a740 1 -- start start 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.179+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c000 0x5654a2b15080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.179+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c400 0x5654a2b15600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.179+0000 7f0144c1a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.179+0000 7f0144c1a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654a2ad7c80 con 0x5654a2c5c400 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.179+0000 7f0144c1a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654a2ad7b00 con 0x5654a2c5c800 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.179+0000 7f0144c1a740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5654a2ad7980 con 0x5654a2c5c000 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c400 0x5654a2b15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c400 0x5654a2b15600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35364/0 (socket says 172.21.15.59:35364) 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 learned_addr learned my addr 172.21.15.59:0/1726678771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c000 0x5654a2b15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c000 msgr2=0x5654a2b15080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a2c5c000 0x5654a2b15080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c800 msgr2=0x5654a2b15b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a1d650e0 con 0x5654a2c5c400 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.180+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a2c5c800 0x5654a2b15b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.181+0000 7f0142200640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c400 0x5654a2b15600 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x5654a2b45b00 tx=0x5654a2aa4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.181+0000 7f013aa00640 1 -- 172.21.15.59:0/1726678771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5654a1d5e9c0 con 0x5654a2c5c400 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.181+0000 7f013aa00640 1 -- 172.21.15.59:0/1726678771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5654a1d82540 con 0x5654a2c5c400 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.181+0000 7f013aa00640 1 -- 172.21.15.59:0/1726678771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5654a1d5e680 con 0x5654a2c5c400 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.181+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654a1d65a40 con 0x5654a2c5c400 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.181+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a1d643c0 con 0x5654a2c5c400 2024-04-23T14:44:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.181+0000 7f0144c1a740 4 datascan.init: resolving metadata pool 16 2024-04-23T14:44:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.183+0000 7f013aa00640 1 -- 172.21.15.59:0/1726678771 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5654a2c0ac00 con 0x5654a2c5c400 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.183+0000 7f013aa00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5654a2c7ac00 0x5654a2b16100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.183+0000 7f013aa00640 1 -- 172.21.15.59:0/1726678771 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x5654a2c09000 con 0x5654a2c5c400 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.183+0000 7f0144c1a740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.183+0000 7f0144c1a740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.184+0000 7f0144c1a740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.184+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5654a2c5cc00 0x5654a2b16680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.184+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c5d000 con 0x5654a2c5cc00 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.184+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5654a2c7ac00 0x5654a2b16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.184+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5654a2c5cc00 0x5654a2b16680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.184+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5654a2c7ac00 0x5654a2b16100 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x5654a2b44960 tx=0x5654a2ad8ae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.185+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5654a2c5cc00 0x5654a2b16680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.186+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(1 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.186+0000 7f013be00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5654a2c7b000 0x5654a20c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.186+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c7b400 con 0x5654a2c7b000 2024-04-23T14:44:39.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.187+0000 7f0142c00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5654a2c7b000 0x5654a20c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.187+0000 7f0142c00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5654a2c7b000 0x5654a20c2000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.187+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(2 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.188+0000 7f013c800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5654a2c7bc00 0x5654a20c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.188+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca000 con 0x5654a2c7bc00 2024-04-23T14:44:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.188+0000 7f0142200640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5654a2c7bc00 0x5654a20c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.188+0000 7f0142200640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5654a2c7bc00 0x5654a20c2580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.189+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(3 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.189+0000 7f013be00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5654a2c7b800 0x5654a20c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.189+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca800 con 0x5654a2c7b800 2024-04-23T14:44:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.189+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5654a2c7b800 0x5654a20c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.190+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5654a2c7b800 0x5654a20c2b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.190+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(4 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.190+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca400 con 0x5654a2c5cc00 2024-04-23T14:44:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.191+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== osd_op_reply(5 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.191+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cac00 con 0x5654a2c7bc00 2024-04-23T14:44:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.191+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== osd_op_reply(6 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.191+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca800 con 0x5654a2c7b800 2024-04-23T14:44:39.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.192+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== osd_op_reply(7 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.192+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb000 con 0x5654a2c5cc00 2024-04-23T14:44:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.193+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 3 ==== osd_op_reply(8 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.193+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c5dc00 con 0x5654a2c7b800 2024-04-23T14:44:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.193+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 3 ==== osd_op_reply(9 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.193+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca400 con 0x5654a2c7bc00 2024-04-23T14:44:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.194+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 3 ==== osd_op_reply(10 [pgnls start_epoch 116 out=49b] v114'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.194+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c5d000 con 0x5654a2c7bc00 2024-04-23T14:44:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.194+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 4 ==== osd_op_reply(11 [pgnls start_epoch 116 out=49b] v114'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.194+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb400 con 0x5654a2c7b800 2024-04-23T14:44:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.194+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 4 ==== osd_op_reply(12 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.195+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c5d000 con 0x5654a2c5cc00 2024-04-23T14:44:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.195+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 4 ==== osd_op_reply(13 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.195+0000 7f013be00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5654a20cb800 0x5654a20c3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.195+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cbc00 con 0x5654a20cb800 2024-04-23T14:44:39.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.196+0000 7f0142c00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5654a20cb800 0x5654a20c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.197+0000 7f0142c00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5654a20cb800 0x5654a20c3080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.203+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(14 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.203+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca000 con 0x5654a2c7b000 2024-04-23T14:44:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.204+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== osd_op_reply(15 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.204+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c5dc00 con 0x5654a2c7b800 2024-04-23T14:44:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.204+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 5 ==== osd_op_reply(16 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.204+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cac00 con 0x5654a20cb800 2024-04-23T14:44:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.205+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(17 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.206+0000 7f013be00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5654a2c5d000 0x5654a20c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.206+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca800 con 0x5654a2c5d000 2024-04-23T14:44:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.206+0000 7f0142200640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5654a2c5d000 0x5654a20c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.208+0000 7f0142200640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5654a2c5d000 0x5654a20c3600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.209+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(18 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.209+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca400 con 0x5654a2c7bc00 2024-04-23T14:44:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.209+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 5 ==== osd_op_reply(19 [pgnls start_epoch 116 out=81b] v115'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.210+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2000 con 0x5654a2c7bc00 2024-04-23T14:44:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.210+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.210+0000 7f0144c1a740 10 datascan.operator(): handling object 1099511627778.0 2024-04-23T14:44:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.210+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2400 con 0x5654a2c7bc00 2024-04-23T14:44:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.214+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 7 ==== osd_op_reply(21 10000000002.00000000 [call] v116'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.214+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2800 con 0x5654a2c5cc00 2024-04-23T14:44:39.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.215+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 5 ==== osd_op_reply(22 [pgnls start_epoch 116 out=49b] v114'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.215+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3800 con 0x5654a2c5cc00 2024-04-23T14:44:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.216+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 6 ==== osd_op_reply(23 [pgnls start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.216+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb000 con 0x5654a2c7bc00 2024-04-23T14:44:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.216+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 8 ==== osd_op_reply(24 [pgnls start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.217+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3c00 con 0x5654a2c7b800 2024-04-23T14:44:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.217+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 6 ==== osd_op_reply(25 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.217+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb400 con 0x5654a2c7b000 2024-04-23T14:44:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.218+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 3 ==== osd_op_reply(26 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.218+0000 7f013c800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5654a20cbc00 0x5654a20c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.218+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca000 con 0x5654a20cbc00 2024-04-23T14:44:39.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.219+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5654a20cbc00 0x5654a20c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.220+0000 7f0141800640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5654a20cbc00 0x5654a20c4680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.221+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(27 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.221+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb400 con 0x5654a2c7b800 2024-04-23T14:44:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.222+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 7 ==== osd_op_reply(28 [pgnls start_epoch 116 out=49b] v114'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.222+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cac00 con 0x5654a2c5d000 2024-04-23T14:44:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.223+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(29 [pgnls start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.223+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca800 con 0x5654a2c7b800 2024-04-23T14:44:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.224+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 8 ==== osd_op_reply(30 [pgnls start_epoch 116 out=81b] v114'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.224+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2c00 con 0x5654a2c7b800 2024-04-23T14:44:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.225+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 9 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.225+0000 7f0144c1a740 10 datascan.operator(): handling object 1099511628279.0 2024-04-23T14:44:39.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.225+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3000 con 0x5654a2c7b800 2024-04-23T14:44:39.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.229+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 10 ==== osd_op_reply(32 100000001f7.00000000 [call] v116'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.229+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5654a20e3400 0x5654a2b16c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.229+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca000 con 0x5654a20e3400 2024-04-23T14:44:39.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.230+0000 7f0142c00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5654a20e3400 0x5654a2b16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.231+0000 7f0142c00640 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5654a20e3400 0x5654a2b16c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.231+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(33 [pgnls start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20e3400 2024-04-23T14:44:39.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.231+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca400 con 0x5654a20cbc00 2024-04-23T14:44:39.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.232+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== osd_op_reply(34 [pgnls start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.232+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2000 con 0x5654a2c7b000 2024-04-23T14:44:39.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.233+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 4 ==== osd_op_reply(35 [pgnls start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.233+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb000 con 0x5654a20cb800 2024-04-23T14:44:39.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.234+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 3 ==== osd_op_reply(36 [pgnls start_epoch 116 out=49b] v114'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.234+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2400 con 0x5654a20e3400 2024-04-23T14:44:39.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.235+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== osd_op_reply(37 [pgnls start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20e3400 2024-04-23T14:44:39.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.235+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fa000 con 0x5654a2c7b800 2024-04-23T14:44:39.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.236+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 11 ==== osd_op_reply(38 [pgnls start_epoch 116 out=49b] v114'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.236+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c7b400 con 0x5654a2c7bc00 2024-04-23T14:44:39.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.236+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 9 ==== osd_op_reply(39 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.236+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fa400 con 0x5654a2c7b000 2024-04-23T14:44:39.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.237+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 5 ==== osd_op_reply(40 [pgnls start_epoch 116 out=81b] v114'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.237+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb400 con 0x5654a2c7b000 2024-04-23T14:44:39.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.238+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.238+0000 7f0144c1a740 10 datascan.operator(): handling object 1099511627780.1 2024-04-23T14:44:39.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.238+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2800 con 0x5654a20cbc00 2024-04-23T14:44:39.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.242+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 3 ==== osd_op_reply(42 10000000004.00000000 [call] v116'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.242+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3800 con 0x5654a2c7bc00 2024-04-23T14:44:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.242+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 10 ==== osd_op_reply(43 [pgnls start_epoch 116 out=49b] v114'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.243+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fa800 con 0x5654a20cb800 2024-04-23T14:44:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.244+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 4 ==== osd_op_reply(44 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.244+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fa400 con 0x5654a2c5d000 2024-04-23T14:44:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.245+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 3 ==== osd_op_reply(45 [pgnls start_epoch 116 out=81b] v115'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.245+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3c00 con 0x5654a2c5d000 2024-04-23T14:44:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.246+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.246+0000 7f0144c1a740 10 datascan.operator(): handling object 1099511628280.0 2024-04-23T14:44:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.246+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c5dc00 con 0x5654a2c5d000 2024-04-23T14:44:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.251+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v116'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.251+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2c5dc00 con 0x5654a2c5d000 2024-04-23T14:44:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.252+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 6 ==== osd_op_reply(48 [pgnls start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.252+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fbc00 con 0x5654a20cbc00 2024-04-23T14:44:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.253+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 4 ==== osd_op_reply(49 [pgnls start_epoch 116 out=81b] v116'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.253+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3800 con 0x5654a20cbc00 2024-04-23T14:44:39.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.254+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.254+0000 7f0144c1a740 10 datascan.operator(): handling object 1099511627780.0 2024-04-23T14:44:39.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.254+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3c00 con 0x5654a20cbc00 2024-04-23T14:44:39.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.258+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 6 ==== osd_op_reply(51 10000000004.00000000 [call] v116'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.258+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fa400 con 0x5654a2c7b000 2024-04-23T14:44:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.259+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 7 ==== osd_op_reply(52 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.259+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca000 con 0x5654a2c5d000 2024-04-23T14:44:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.260+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 7 ==== osd_op_reply(53 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.260+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fac00 con 0x5654a20cb800 2024-04-23T14:44:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.261+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 5 ==== osd_op_reply(54 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.261+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb000 con 0x5654a2c5cc00 2024-04-23T14:44:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.262+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 7 ==== osd_op_reply(55 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.262+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fb000 con 0x5654a2c5cc00 2024-04-23T14:44:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.264+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 8 ==== osd_op_reply(56 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.264+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20cb400 con 0x5654a2c5cc00 2024-04-23T14:44:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.265+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 9 ==== osd_op_reply(57 [pgnls start_epoch 116 out=49b] v114'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.265+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fb400 con 0x5654a2c5cc00 2024-04-23T14:44:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.266+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 10 ==== osd_op_reply(58 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.266+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2000 con 0x5654a20cb800 2024-04-23T14:44:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.267+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 6 ==== osd_op_reply(59 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.267+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fb800 con 0x5654a20cb800 2024-04-23T14:44:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.269+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 7 ==== osd_op_reply(60 [pgnls start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.269+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2400 con 0x5654a2c7bc00 2024-04-23T14:44:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.270+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.3 v2:172.21.15.59:6800/3205049728 11 ==== osd_op_reply(61 [pgnls start_epoch 116 out=49b] v114'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c7bc00 2024-04-23T14:44:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.270+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fa000 con 0x5654a2c5cc00 2024-04-23T14:44:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.272+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.7 v2:172.21.15.76:6822/3281262326 11 ==== osd_op_reply(62 [pgnls start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c5cc00 2024-04-23T14:44:39.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.272+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2100000 con 0x5654a2c7b800 2024-04-23T14:44:39.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.272+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 12 ==== osd_op_reply(63 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.272+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20fbc00 con 0x5654a20cb800 2024-04-23T14:44:39.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.273+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 8 ==== osd_op_reply(64 [pgnls start_epoch 116 out=49b] v114'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.274+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2100400 con 0x5654a20cb800 2024-04-23T14:44:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.275+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 9 ==== osd_op_reply(65 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.275+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2c00 con 0x5654a20e3400 2024-04-23T14:44:39.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.276+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.2 v2:172.21.15.59:6816/77995666 3 ==== osd_op_reply(66 [pgnls start_epoch 116 out=49b] v114'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20e3400 2024-04-23T14:44:39.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.276+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2100800 con 0x5654a2c7b800 2024-04-23T14:44:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.277+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 13 ==== osd_op_reply(67 [pgnls start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.277+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e3000 con 0x5654a20cb800 2024-04-23T14:44:39.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.278+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 10 ==== osd_op_reply(68 [pgnls start_epoch 116 out=49b] v114'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.279+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2100c00 con 0x5654a20cbc00 2024-04-23T14:44:39.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.280+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 7 ==== osd_op_reply(69 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.280+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20e2800 con 0x5654a2c5d000 2024-04-23T14:44:39.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.281+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 <== osd.5 v2:172.21.15.76:6808/2782996536 8 ==== osd_op_reply(70 [pgnls start_epoch 116 out=49b] v114'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6780 con 0x5654a2c5d000 2024-04-23T14:44:39.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.281+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2101000 con 0x5654a2c7b000 2024-04-23T14:44:39.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.282+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.1 v2:172.21.15.59:6808/4218092099 8 ==== osd_op_reply(71 [pgnls start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a2c7b000 2024-04-23T14:44:39.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.283+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca800 con 0x5654a20cb800 2024-04-23T14:44:39.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.284+0000 7f0142c00640 1 -- 172.21.15.59:0/1726678771 <== osd.4 v2:172.21.15.76:6816/3407674963 11 ==== osd_op_reply(72 [pgnls start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6500 con 0x5654a20cb800 2024-04-23T14:44:39.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.284+0000 7f013c800640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a2101400 con 0x5654a20cbc00 2024-04-23T14:44:39.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.285+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.6 v2:172.21.15.76:6800/1890860772 8 ==== osd_op_reply(73 [pgnls start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a20cbc00 2024-04-23T14:44:39.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.285+0000 7f013be00640 1 -- 172.21.15.59:0/1726678771 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5654a20ca400 con 0x5654a2c7b800 2024-04-23T14:44:39.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0141800640 1 -- 172.21.15.59:0/1726678771 <== osd.0 v2:172.21.15.59:6817/2024703144 14 ==== osd_op_reply(74 [pgnls start_epoch 116 out=49b] v114'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5654a2ac6000 con 0x5654a2c7b800 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5654a2c7b800 msgr2=0x5654a20c2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x5654a2c7b800 0x5654a20c2b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5654a2c7b000 msgr2=0x5654a20c2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x5654a2c7b000 0x5654a20c2000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5654a20e3400 msgr2=0x5654a2b16c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x5654a20e3400 0x5654a2b16c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5654a2c7bc00 msgr2=0x5654a20c2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.286+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x5654a2c7bc00 0x5654a20c2580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5654a20cb800 msgr2=0x5654a20c3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x5654a20cb800 0x5654a20c3080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 reap_dead start 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5654a2c5d000 msgr2=0x5654a20c3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x5654a2c5d000 0x5654a20c3600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 reap_dead start 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5654a20cbc00 msgr2=0x5654a20c4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x5654a20cbc00 0x5654a20c4680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5654a2c5cc00 msgr2=0x5654a2b16680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x5654a2c5cc00 0x5654a2b16680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0142200640 1 -- 172.21.15.59:0/1726678771 reap_dead start 2024-04-23T14:44:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5654a2c7ac00 msgr2=0x5654a2b16100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5654a2c7ac00 0x5654a2b16100 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x5654a2b44960 tx=0x5654a2ad8ae0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c400 msgr2=0x5654a2b15600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c400 0x5654a2b15600 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x5654a2b45b00 tx=0x5654a2aa4d20 comp rx=0 tx=0).stop 2024-04-23T14:44:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 shutdown_connections 2024-04-23T14:44:39.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x5654a2c7ac00 0x5654a2b16100 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.287+0000 7f0144c1a740 1 --2- 172.21.15.59:0/1726678771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a2c5c400 0x5654a2b15600 secure :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x5654a2b45b00 tx=0x5654a2aa4d20 comp rx=0 tx=0).stop 2024-04-23T14:44:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.288+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 >> 172.21.15.59:0/1726678771 conn(0x5654a2b2dc00 msgr2=0x5654a1e25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.288+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 shutdown_connections 2024-04-23T14:44:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.288+0000 7f0144c1a740 1 -- 172.21.15.59:0/1726678771 wait complete. 2024-04-23T14:44:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.289+0000 7f0144c1a740 1 -- 172.21.15.59:0/2217660281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a1d39c00 msgr2=0x5654a2b14b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.289+0000 7f0144c1a740 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a1d39c00 0x5654a2b14b00 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x5654a2b5f200 tx=0x5654a2ad8a80 comp rx=0 tx=0).stop 2024-04-23T14:44:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.289+0000 7f0144c1a740 1 -- 172.21.15.59:0/2217660281 shutdown_connections 2024-04-23T14:44:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.289+0000 7f0144c1a740 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5654a1d39800 0x5654a2b14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.289+0000 7f0144c1a740 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654a1d39c00 0x5654a2b14b00 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.289+0000 7f0144c1a740 1 --2- 172.21.15.59:0/2217660281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a1d38c00 0x5654a2b14000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.289+0000 7f0144c1a740 1 -- 172.21.15.59:0/2217660281 >> 172.21.15.59:0/2217660281 conn(0x5654a1d39000 msgr2=0x5654a1e24800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.290+0000 7f0144c1a740 1 -- 172.21.15.59:0/2217660281 shutdown_connections 2024-04-23T14:44:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.290+0000 7f0144c1a740 1 -- 172.21.15.59:0/2217660281 wait complete. 2024-04-23T14:44:39.303 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.274917, result: 2024-04-23T14:44:39.303 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-23T14:44:39.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.418+0000 7fadc0c1d740 1 -- start start 2024-04-23T14:44:39.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.418+0000 7fadc0c1d740 1 Processor -- start 2024-04-23T14:44:39.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142a4400 0x55b814272b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142a4800 0x55b814273080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b8142a4000 0x55b814272580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8142f6600 con 0x55b8142a4800 2024-04-23T14:44:39.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b8142f6480 con 0x55b8142a4400 2024-04-23T14:44:39.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55b813566180 con 0x55b8142a4000 2024-04-23T14:44:39.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadbd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142a4800 0x55b814273080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadbd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142a4800 0x55b814273080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35378/0 (socket says 172.21.15.59:35378) 2024-04-23T14:44:39.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadbd800640 1 -- 172.21.15.59:0/1254999301 learned_addr learned my addr 172.21.15.59:0/1254999301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.420+0000 7fadbe200640 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142a4400 0x55b814272b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbec00640 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b8142a4000 0x55b814272580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbd800640 1 -- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b8142a4000 msgr2=0x55b814272580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbd800640 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b8142a4000 0x55b814272580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbd800640 1 -- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142a4400 msgr2=0x55b814272b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbd800640 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142a4400 0x55b814272b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbd800640 1 -- 172.21.15.59:0/1254999301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b813533680 con 0x55b8142a4800 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbec00640 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b8142a4000 0x55b814272580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbe200640 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142a4400 0x55b814272b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbd800640 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142a4800 0x55b814273080 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x55b814337440 tx=0x55b8142a7f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de89a220c1c43bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbb000640 1 -- 172.21.15.59:0/1254999301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b81352cd00 con 0x55b8142a4800 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbb000640 1 -- 172.21.15.59:0/1254999301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b813550c40 con 0x55b8142a4800 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadbb000640 1 -- 172.21.15.59:0/1254999301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b81352c680 con 0x55b8142a4800 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1254999301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b813533860 con 0x55b8142a4800 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1254999301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b8135330e0 con 0x55b8142a4800 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.421+0000 7fadc0c1d740 4 waiting for MDS map... 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.422+0000 7fadbb000640 1 -- 172.21.15.59:0/1254999301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x55b8143e8400 con 0x55b8142a4800 2024-04-23T14:44:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.422+0000 7fadbb000640 1 -- 172.21.15.59:0/1254999301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x55b81355f180 con 0x55b8142a4800 2024-04-23T14:44:39.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.422+0000 7fadc0c1d740 4 Got MDS map 107 2024-04-23T14:44:39.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.422+0000 7fadc0c1d740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-23T14:44:39.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.422+0000 7fadc0c1d740 4 datascan.main: Using metadata pool output 2024-04-23T14:44:39.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.422+0000 7fadc0c1d740 4 datascan.main: connecting to RADOS... 2024-04-23T14:44:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.423+0000 7fadc0c1d740 1 Processor -- start 2024-04-23T14:44:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.423+0000 7fadc0c1d740 1 -- start start 2024-04-23T14:44:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142e9800 0x55b814273600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142e9c00 0x55b814273b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b814448000 0x55b814274100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b813566480 con 0x55b8142e9c00 2024-04-23T14:44:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b813566900 con 0x55b8142e9800 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55b8142f7080 con 0x55b814448000 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadbe200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142e9c00 0x55b814273b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadbe200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142e9c00 0x55b814273b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35380/0 (socket says 172.21.15.59:35380) 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadbe200640 1 -- 172.21.15.59:0/1830872947 learned_addr learned my addr 172.21.15.59:0/1830872947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadbec00640 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142e9800 0x55b814273600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadbe200640 1 -- 172.21.15.59:0/1830872947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b814448000 msgr2=0x55b814274100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.424+0000 7fadbe200640 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b814448000 0x55b814274100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.425+0000 7fadbe200640 1 -- 172.21.15.59:0/1830872947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142e9800 msgr2=0x55b814273600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.425+0000 7fadbe200640 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142e9800 0x55b814273600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.425+0000 7fadbe200640 1 -- 172.21.15.59:0/1830872947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8135334a0 con 0x55b8142e9c00 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.425+0000 7fadbe200640 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142e9c00 0x55b814273b80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x55b814337da0 tx=0x55b8143fec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9621dd16ba8181 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.425+0000 7fadb8800640 1 -- 172.21.15.59:0/1830872947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b81352cd00 con 0x55b8142e9c00 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.425+0000 7fadb8800640 1 -- 172.21.15.59:0/1830872947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b813550a80 con 0x55b8142e9c00 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.425+0000 7fadb8800640 1 -- 172.21.15.59:0/1830872947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b81352c680 con 0x55b8142e9c00 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1830872947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142e9c00 msgr2=0x55b814273b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142e9c00 0x55b814273b80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x55b814337da0 tx=0x55b8143fec00 comp rx=0 tx=0).stop 2024-04-23T14:44:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1830872947 shutdown_connections 2024-04-23T14:44:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b814448000 0x55b814274100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142e9c00 0x55b814273b80 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1830872947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142e9800 0x55b814273600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1830872947 >> 172.21.15.59:0/1830872947 conn(0x55b8142e9400 msgr2=0x55b8135d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1830872947 shutdown_connections 2024-04-23T14:44:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.426+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1830872947 wait complete. 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.427+0000 7fadc0c1d740 1 Processor -- start 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.427+0000 7fadc0c1d740 1 -- start start 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.427+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b814449400 0x55b814273600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b814449000 0x55b814273b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadc0c1d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b814448c00 0x55b814274100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b8142f7380 con 0x55b814448c00 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b8142f7200 con 0x55b814449000 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadc0c1d740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55b8142f7500 con 0x55b814449400 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b814448c00 0x55b814274100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b814448c00 0x55b814274100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35396/0 (socket says 172.21.15.59:35396) 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 learned_addr learned my addr 172.21.15.59:0/4209781869 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbe200640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b814449000 0x55b814273b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b814449400 msgr2=0x55b814273600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b814449400 0x55b814273600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b814449000 msgr2=0x55b814273b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b814449000 0x55b814273b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.428+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b813533680 con 0x55b814448c00 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.429+0000 7fadbd800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b814448c00 0x55b814274100 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x55b8143fc180 tx=0x55b8142a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.429+0000 7fadb6a00640 1 -- 172.21.15.59:0/4209781869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b81352c820 con 0x55b814448c00 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.429+0000 7fadb6a00640 1 -- 172.21.15.59:0/4209781869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b813551340 con 0x55b814448c00 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.429+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b8135332c0 con 0x55b814448c00 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.429+0000 7fadb6a00640 1 -- 172.21.15.59:0/4209781869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b81352cd00 con 0x55b814448c00 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.429+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b813533a40 con 0x55b814448c00 2024-04-23T14:44:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.429+0000 7fadc0c1d740 4 datascan.init: resolving metadata pool 16 2024-04-23T14:44:39.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.431+0000 7fadb6a00640 1 -- 172.21.15.59:0/4209781869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55b8143dec00 con 0x55b814448c00 2024-04-23T14:44:39.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.431+0000 7fadb6a00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55b814449c00 0x55b814274680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.431+0000 7fadb6a00640 1 -- 172.21.15.59:0/4209781869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x55b8143dca00 con 0x55b814448c00 2024-04-23T14:44:39.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.431+0000 7fadc0c1d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-23T14:44:39.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.431+0000 7fadc0c1d740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-23T14:44:39.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.431+0000 7fadc0c1d740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-23T14:44:39.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.432+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55b814448800 0x55b814274c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.432+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55b814448400 con 0x55b814448800 2024-04-23T14:44:39.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.432+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55b814449c00 0x55b814274680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.432+0000 7fadbe200640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55b814448800 0x55b814274c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.432+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55b814449c00 0x55b814274680 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55b8142a81b0 tx=0x55b8143fef00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.432+0000 7fadbe200640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55b814448800 0x55b814274c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.433+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv39 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.433+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55b814448000 0x55b814275180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.433+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55b8142e9c00 con 0x55b814448000 2024-04-23T14:44:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.434+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55b814448000 0x55b814275180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.435+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55b814448000 0x55b814275180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.435+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.435+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55b8142e9800 0x55b814275700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.435+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b814449800 con 0x55b8142e9800 2024-04-23T14:44:39.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.436+0000 7fadbe200640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55b8142e9800 0x55b814275700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.437+0000 7fadbe200640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55b8142e9800 0x55b814275700 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.438+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.438+0000 7fadb7e00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55b8138a2000 0x55b8138a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.438+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2400 con 0x55b8138a2000 2024-04-23T14:44:39.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.438+0000 7fadbd800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55b8138a2000 0x55b8138a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.439+0000 7fadbd800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55b8138a2000 0x55b8138a8000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.439+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.440+0000 7fadb8800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55b8138a2c00 0x55b8138a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.440+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3000 con 0x55b8138a2c00 2024-04-23T14:44:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.440+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55b8138a2c00 0x55b8138a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.440+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55b8138a2c00 0x55b8138a8580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.441+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.441+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2800 con 0x55b814448800 2024-04-23T14:44:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.441+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.441+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3400 con 0x55b8142e9800 2024-04-23T14:44:39.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.442+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.442+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3800 con 0x55b8138a2c00 2024-04-23T14:44:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.443+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.443+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3000 con 0x55b814448800 2024-04-23T14:44:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.443+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.443+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3c00 con 0x55b8142e9800 2024-04-23T14:44:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.444+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.444+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3800 con 0x55b814448800 2024-04-23T14:44:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.445+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.445+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2800 con 0x55b8138a2c00 2024-04-23T14:44:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.445+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 116 out=49b] v114'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.445+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8142e9c00 con 0x55b8138a2c00 2024-04-23T14:44:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.445+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 116 out=49b] v114'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.445+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3400 con 0x55b814448800 2024-04-23T14:44:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.446+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.446+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0000 con 0x55b8142e9800 2024-04-23T14:44:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.447+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.447+0000 7fadb7e00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55b8138a3000 0x55b8138a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.447+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b814449800 con 0x55b8138a3000 2024-04-23T14:44:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.448+0000 7fadbe200640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55b8138a3000 0x55b8138a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.449+0000 7fadbe200640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55b8138a3000 0x55b8138a8b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.450+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.450+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0400 con 0x55b8138a2000 2024-04-23T14:44:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.450+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.450+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0000 con 0x55b814448800 2024-04-23T14:44:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.451+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.451+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0800 con 0x55b8138a3000 2024-04-23T14:44:39.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.452+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.452+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b814449800 con 0x55b814448000 2024-04-23T14:44:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.453+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.453+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0c00 con 0x55b8138a2c00 2024-04-23T14:44:39.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.454+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 116 out=81b] v116'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.454+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b814448400 con 0x55b8142e9800 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.455+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.455+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0c00 con 0x55b8138a2000 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.456+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.456+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3c00 con 0x55b8138a2c00 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.456+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.456+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1000 con 0x55b814448800 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.457+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.457+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3800 con 0x55b8142e9800 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.458+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.458+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b814449800 con 0x55b8138a2c00 2024-04-23T14:44:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.459+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.459+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3400 con 0x55b814448800 2024-04-23T14:44:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.460+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.460+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8142e9c00 con 0x55b8142e9800 2024-04-23T14:44:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.461+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.461+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0000 con 0x55b814448800 2024-04-23T14:44:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.462+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.462+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2800 con 0x55b8138a2c00 2024-04-23T14:44:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.462+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 116 out=49b] v114'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.462+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0800 con 0x55b8138a2c00 2024-04-23T14:44:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.463+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 116 out=49b] v114'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.463+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1400 con 0x55b814448800 2024-04-23T14:44:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.464+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.464+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1000 con 0x55b8142e9800 2024-04-23T14:44:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.465+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.465+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1800 con 0x55b8138a3000 2024-04-23T14:44:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.466+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.466+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3800 con 0x55b8138a2000 2024-04-23T14:44:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.467+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.467+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1c00 con 0x55b814448800 2024-04-23T14:44:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.467+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.467+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3400 con 0x55b8138a3000 2024-04-23T14:44:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.468+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.468+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3400 con 0x55b814448000 2024-04-23T14:44:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.469+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.469+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b814448400 con 0x55b8138a2c00 2024-04-23T14:44:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.470+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 116 out=49b] v116'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.470+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0000 con 0x55b8142e9800 2024-04-23T14:44:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.471+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 116 out=49b] v114'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.471+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3c00 con 0x55b8142e9800 2024-04-23T14:44:39.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.472+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.473+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1400 con 0x55b8138a2c00 2024-04-23T14:44:39.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.473+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.473+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2800 con 0x55b814448800 2024-04-23T14:44:39.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.474+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 12 ==== osd_op_reply(44 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.474+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1000 con 0x55b8138a2000 2024-04-23T14:44:39.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.475+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.475+0000 7fadb7e00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55b8138a3400 0x55b8138a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.475+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0800 con 0x55b8138a3400 2024-04-23T14:44:39.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.476+0000 7fadbd800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55b8138a3400 0x55b8138a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.477+0000 7fadbd800640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55b8138a3400 0x55b8138a9080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.478+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a3400 2024-04-23T14:44:39.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.478+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1800 con 0x55b814448800 2024-04-23T14:44:39.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.479+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 13 ==== osd_op_reply(47 [pgnls-filter start_epoch 116 out=49b] v114'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.479+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b814449800 con 0x55b814448000 2024-04-23T14:44:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.481+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.481+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1c00 con 0x55b814448800 2024-04-23T14:44:39.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.482+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 14 ==== osd_op_reply(49 [pgnls-filter start_epoch 116 out=81b] v116'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.482+0000 7fadc0c1d740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-23T14:44:39.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.482+0000 7fadc0c1d740 10 datascan.operator(): handling object 100000001f7.0 2024-04-23T14:44:39.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.482+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0400 con 0x55b814448800 2024-04-23T14:44:39.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.483+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 15 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.483+0000 7fadc0c1d740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-23T14:44:39.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.483+0000 7fadc0c1d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-23T14:44:39.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.483+0000 7fadc0c1d740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-23T14:44:39.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.483+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0c00 con 0x55b8138a2c00 2024-04-23T14:44:39.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.484+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.484+0000 7fadc0c1d740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-23T14:44:39.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.484+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55b8138b1000 0x55b8138a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.484+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0800 con 0x55b8138b1000 2024-04-23T14:44:39.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.484+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55b8138b1000 0x55b8138a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.485+0000 7fadbec00640 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55b8138b1000 0x55b8138a9600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.486+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x55b814294500 con 0x55b8138b1000 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.486+0000 7fadc0c1d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.486+0000 7fadc0c1d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.486+0000 7fadc0c1d740 20 datascan.get_frag_of: frag is * 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.486+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2400 con 0x55b8138a2c00 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.487+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.487+0000 7fadc0c1d740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.487+0000 7fadc0c1d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-23T14:44:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.487+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3800 con 0x55b8138a2c00 2024-04-23T14:44:39.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.488+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.488+0000 7fadc0c1d740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-23T14:44:39.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.488+0000 7fadc0c1d740 20 datascan.read_dentry: bravo_head not found in result 2024-04-23T14:44:39.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.488+0000 7fadc0c1d740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-23T14:44:39.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.488+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2400 con 0x55b8138a2c00 2024-04-23T14:44:39.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.491+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v116'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.491+0000 7fadc0c1d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-23T14:44:39.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.491+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1400 con 0x55b8138b1000 2024-04-23T14:44:39.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.492+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138b1000 2024-04-23T14:44:39.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.492+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8142e9c00 con 0x55b8138a3400 2024-04-23T14:44:39.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.493+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a3400 2024-04-23T14:44:39.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.493+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c8000 con 0x55b8138a2000 2024-04-23T14:44:39.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.494+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.495+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138ca000 con 0x55b8138a3000 2024-04-23T14:44:39.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.496+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 116 out=49b] v114'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.496+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c8400 con 0x55b8138b1000 2024-04-23T14:44:39.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.497+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.2 v2:172.21.15.59:6816/77995666 3 ==== osd_op_reply(60 [pgnls-filter start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138b1000 2024-04-23T14:44:39.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.497+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138ca400 con 0x55b814448800 2024-04-23T14:44:39.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.498+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 16 ==== osd_op_reply(61 [pgnls-filter start_epoch 116 out=49b] v114'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.498+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c8800 con 0x55b8138a2c00 2024-04-23T14:44:39.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.499+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.499+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138ca800 con 0x55b8138a2000 2024-04-23T14:44:39.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.500+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 116 out=49b] v114'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.500+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c8c00 con 0x55b8138a2c00 2024-04-23T14:44:39.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.501+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 116 out=49b] v114'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.501+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138cac00 con 0x55b8138a3000 2024-04-23T14:44:39.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.502+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.502+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c9000 con 0x55b814448000 2024-04-23T14:44:39.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.504+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 116 out=49b] v116'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.504+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138cb000 con 0x55b814448000 2024-04-23T14:44:39.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.505+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.505+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c9400 con 0x55b8138a3400 2024-04-23T14:44:39.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.507+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.6 v2:172.21.15.76:6800/1890860772 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 116 out=49b] v116'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a3400 2024-04-23T14:44:39.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.507+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138cb400 con 0x55b8138a2000 2024-04-23T14:44:39.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.508+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.508+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c9800 con 0x55b814448000 2024-04-23T14:44:39.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.509+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.509+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138cb800 con 0x55b8138a3000 2024-04-23T14:44:39.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.510+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.511+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c9c00 con 0x55b8142e9800 2024-04-23T14:44:39.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.512+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.512+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138cbc00 con 0x55b8142e9800 2024-04-23T14:44:39.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.513+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.513+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1400 con 0x55b8142e9800 2024-04-23T14:44:39.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.514+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 116 out=49b] v114'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.514+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138ca000 con 0x55b8142e9800 2024-04-23T14:44:39.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.516+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.516+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3800 con 0x55b8138a3000 2024-04-23T14:44:39.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.517+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.517+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b813506c00 con 0x55b8138a3000 2024-04-23T14:44:39.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.518+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 116 out=49b] v114'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.518+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2400 con 0x55b8138a2c00 2024-04-23T14:44:39.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.519+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.3 v2:172.21.15.59:6800/3205049728 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 116 out=49b] v114'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138a2c00 2024-04-23T14:44:39.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.520+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0000 con 0x55b8142e9800 2024-04-23T14:44:39.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.521+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.7 v2:172.21.15.76:6822/3281262326 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 116 out=49b] v114'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8142e9800 2024-04-23T14:44:39.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.521+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3c00 con 0x55b814448800 2024-04-23T14:44:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.522+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 17 ==== osd_op_reply(80 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.522+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1800 con 0x55b8138a3000 2024-04-23T14:44:39.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.523+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 116 out=49b] v114'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.523+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a2800 con 0x55b8138a3000 2024-04-23T14:44:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.524+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.525+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b1c00 con 0x55b8138b1000 2024-04-23T14:44:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.526+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.2 v2:172.21.15.59:6816/77995666 4 ==== osd_op_reply(83 [pgnls-filter start_epoch 116 out=49b] v114'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b8138b1000 2024-04-23T14:44:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.526+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c8400 con 0x55b814448800 2024-04-23T14:44:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.527+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 18 ==== osd_op_reply(84 [pgnls-filter start_epoch 116 out=49b] v114'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.527+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138b0400 con 0x55b8138a3000 2024-04-23T14:44:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.528+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 116 out=49b] v114'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.528+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c8800 con 0x55b8138a3400 2024-04-23T14:44:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.530+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.6 v2:172.21.15.76:6800/1890860772 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a3400 2024-04-23T14:44:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.530+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b813506c00 con 0x55b814448000 2024-04-23T14:44:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.531+0000 7fadbec00640 1 -- 172.21.15.59:0/4209781869 <== osd.5 v2:172.21.15.76:6808/2782996536 8 ==== osd_op_reply(87 [pgnls-filter start_epoch 116 out=49b] v114'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294500 con 0x55b814448000 2024-04-23T14:44:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.531+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c8c00 con 0x55b8138a2000 2024-04-23T14:44:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.532+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.1 v2:172.21.15.59:6808/4218092099 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 116 out=49b] v114'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a2000 2024-04-23T14:44:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.532+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138a3800 con 0x55b8138a3000 2024-04-23T14:44:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.533+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.4 v2:172.21.15.76:6816/3407674963 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 116 out=49b] v114'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b8138a3000 2024-04-23T14:44:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.533+0000 7fadb7e00640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138c9000 con 0x55b8138a3400 2024-04-23T14:44:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.534+0000 7fadbd800640 1 -- 172.21.15.59:0/4209781869 <== osd.6 v2:172.21.15.76:6800/1890860772 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 116 out=49b] v114'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294780 con 0x55b8138a3400 2024-04-23T14:44:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.534+0000 7fadb8800640 1 -- 172.21.15.59:0/4209781869 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55b8138ca400 con 0x55b814448800 2024-04-23T14:44:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 <== osd.0 v2:172.21.15.59:6817/2024703144 19 ==== osd_op_reply(91 [pgnls-filter start_epoch 116 out=49b] v114'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b814294000 con 0x55b814448800 2024-04-23T14:44:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55b814448800 msgr2=0x55b814274c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55b814448800 0x55b814274c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55b8138a2000 msgr2=0x55b8138a8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55b8138a2000 0x55b8138a8000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55b8138b1000 msgr2=0x55b8138a9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55b8138b1000 0x55b8138a9600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55b8138a2c00 msgr2=0x55b8138a8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55b8138a2c00 0x55b8138a8580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55b8138a3000 msgr2=0x55b8138a8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55b8138a3000 0x55b8138a8b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55b814448000 msgr2=0x55b814275180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55b814448000 0x55b814275180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55b8138a3400 msgr2=0x55b8138a9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55b8138a3400 0x55b8138a9080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55b8142e9800 msgr2=0x55b814275700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 reap_dead start 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55b8142e9800 0x55b814275700 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 reap_dead start 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadbe200640 1 -- 172.21.15.59:0/4209781869 reap_dead start 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.536+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55b814449c00 msgr2=0x55b814274680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.537+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55b814449c00 0x55b814274680 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55b8142a81b0 tx=0x55b8143fef00 comp rx=0 tx=0).stop 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.537+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b814448c00 msgr2=0x55b814274100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.537+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b814448c00 0x55b814274100 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x55b8143fc180 tx=0x55b8142a6660 comp rx=0 tx=0).stop 2024-04-23T14:44:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.537+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 shutdown_connections 2024-04-23T14:44:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.537+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55b814449c00 0x55b814274680 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.537+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/4209781869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b814448c00 0x55b814274100 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.537+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 >> 172.21.15.59:0/4209781869 conn(0x55b8142e9400 msgr2=0x55b8135d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.538+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 shutdown_connections 2024-04-23T14:44:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.538+0000 7fadc0c1d740 1 -- 172.21.15.59:0/4209781869 wait complete. 2024-04-23T14:44:39.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.539+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1254999301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142a4800 msgr2=0x55b814273080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.539+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142a4800 0x55b814273080 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x55b814337440 tx=0x55b8142a7f20 comp rx=0 tx=0).stop 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.539+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1254999301 shutdown_connections 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.539+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55b8142a4000 0x55b814272580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.539+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b8142a4800 0x55b814273080 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.539+0000 7fadc0c1d740 1 --2- 172.21.15.59:0/1254999301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8142a4400 0x55b814272b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.539+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1254999301 >> 172.21.15.59:0/1254999301 conn(0x55b813507000 msgr2=0x55b8135d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.540+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1254999301 shutdown_connections 2024-04-23T14:44:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.540+0000 7fadc0c1d740 1 -- 172.21.15.59:0/1254999301 wait complete. 2024-04-23T14:44:39.551 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.247980, result: 2024-04-23T14:44:39.551 DEBUG:teuthology.orchestra.run.smithi059:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-23T14:44:39.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.649+0000 7f3237618740 1 -- start start 2024-04-23T14:44:39.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.649+0000 7f3237618740 1 Processor -- start 2024-04-23T14:44:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3237618740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa486f4c00 0x55fa49460000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3237618740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa486f5800 0x55fa49460580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3237618740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa486f5c00 0x55fa49460b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3237618740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa49464180 con 0x55fa486f5800 2024-04-23T14:44:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3237618740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa49464300 con 0x55fa486f4c00 2024-04-23T14:44:39.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3237618740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fa49464480 con 0x55fa486f5c00 2024-04-23T14:44:39.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3234200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa486f5800 0x55fa49460580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3234200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa486f5800 0x55fa49460580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35424/0 (socket says 172.21.15.59:35424) 2024-04-23T14:44:39.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.650+0000 7f3234200640 1 -- 172.21.15.59:0/1576672581 learned_addr learned my addr 172.21.15.59:0/1576672581 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234c00640 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa486f4c00 0x55fa49460000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3235600640 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa486f5c00 0x55fa49460b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234200640 1 -- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa486f5c00 msgr2=0x55fa49460b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234200640 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa486f5c00 0x55fa49460b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234200640 1 -- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa486f4c00 msgr2=0x55fa49460000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234200640 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa486f4c00 0x55fa49460000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234200640 1 -- 172.21.15.59:0/1576672581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa48721680 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234c00640 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa486f4c00 0x55fa49460000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3234200640 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa486f5800 0x55fa49460580 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x55fa4951b230 tx=0x55fa494949c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f783a7896acd9a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.651+0000 7f3231a00640 1 -- 172.21.15.59:0/1576672581 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa4871a9c0 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3231a00640 1 -- 172.21.15.59:0/1576672581 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55fa4873e1c0 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3231a00640 1 -- 172.21.15.59:0/1576672581 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa4871a680 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3237618740 1 -- 172.21.15.59:0/1576672581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa487214a0 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3237618740 1 -- 172.21.15.59:0/1576672581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55fa48721a40 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3237618740 4 waiting for MDS map... 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3231a00640 1 -- 172.21.15.59:0/1576672581 <== mon.0 v2:172.21.15.59:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x55fa495c4200 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3231a00640 1 -- 172.21.15.59:0/1576672581 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x55fa4874d180 con 0x55fa486f5800 2024-04-23T14:44:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3237618740 4 Got MDS map 107 2024-04-23T14:44:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3237618740 4 datascan.main: Using metadata pool output 2024-04-23T14:44:39.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.652+0000 7f3237618740 4 datascan.main: connecting to RADOS... 2024-04-23T14:44:39.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.653+0000 7f3237618740 1 Processor -- start 2024-04-23T14:44:39.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.653+0000 7f3237618740 1 -- start start 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3237618740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 0x55fa49461080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3237618740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c400 0x55fa49461600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3237618740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3237618740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa49464600 con 0x55fa4961c400 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3237618740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa49464780 con 0x55fa4961c800 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3237618740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fa49464900 con 0x55fa4961c000 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3234c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c400 0x55fa49461600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3234c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c400 0x55fa49461600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35438/0 (socket says 172.21.15.59:35438) 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.654+0000 7f3234c00640 1 -- 172.21.15.59:0/2161498991 learned_addr learned my addr 172.21.15.59:0/2161498991 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3234200640 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3235600640 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 0x55fa49461080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3234c00640 1 -- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 msgr2=0x55fa49461080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3234c00640 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 0x55fa49461080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3234c00640 1 -- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c800 msgr2=0x55fa49461b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3234c00640 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3234c00640 1 -- 172.21.15.59:0/2161498991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa48721860 con 0x55fa4961c400 2024-04-23T14:44:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f3234c00640 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c400 0x55fa49461600 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x55fa495ce7e0 tx=0x55fa495d2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb500305f6d3d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f322f200640 1 -- 172.21.15.59:0/2161498991 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa4871a9c0 con 0x55fa4961c400 2024-04-23T14:44:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f322f200640 1 -- 172.21.15.59:0/2161498991 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55fa4873ec40 con 0x55fa4961c400 2024-04-23T14:44:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.655+0000 7f322f200640 1 -- 172.21.15.59:0/2161498991 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa4871a680 con 0x55fa4961c400 2024-04-23T14:44:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 -- 172.21.15.59:0/2161498991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c400 msgr2=0x55fa49461600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c400 0x55fa49461600 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x55fa495ce7e0 tx=0x55fa495d2c00 comp rx=0 tx=0).stop 2024-04-23T14:44:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 -- 172.21.15.59:0/2161498991 shutdown_connections 2024-04-23T14:44:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c400 0x55fa49461600 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 --2- 172.21.15.59:0/2161498991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 0x55fa49461080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 -- 172.21.15.59:0/2161498991 >> 172.21.15.59:0/2161498991 conn(0x55fa49493c00 msgr2=0x55fa487c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 -- 172.21.15.59:0/2161498991 shutdown_connections 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.656+0000 7f3237618740 1 -- 172.21.15.59:0/2161498991 wait complete. 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.657+0000 7f3237618740 1 Processor -- start 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.657+0000 7f3237618740 1 -- start start 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3237618740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 0x55fa49461080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3237618740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c400 0x55fa49461600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3237618740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3237618740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fa49465c80 con 0x55fa4961c800 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3237618740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fa49465b00 con 0x55fa4961c400 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3237618740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x55fa49465980 con 0x55fa4961c000 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35452/0 (socket says 172.21.15.59:35452) 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 learned_addr learned my addr 172.21.15.59:0/541119204 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c400 0x55fa49461600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 0x55fa49461080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 msgr2=0x55fa49461080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa4961c000 0x55fa49461080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c400 msgr2=0x55fa49461600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa4961c400 0x55fa49461600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.658+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa487212c0 con 0x55fa4961c800 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.659+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c800 0x55fa49461b80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x55fa4951be30 tx=0x55fa487a8b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.659+0000 7f322d400640 1 -- 172.21.15.59:0/541119204 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa4871ab60 con 0x55fa4961c800 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.659+0000 7f322d400640 1 -- 172.21.15.59:0/541119204 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55fa4873ee00 con 0x55fa4961c800 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.659+0000 7f322d400640 1 -- 172.21.15.59:0/541119204 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa4871a820 con 0x55fa4961c800 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.659+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fa487210e0 con 0x55fa4961c800 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.659+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa487203c0 con 0x55fa4961c800 2024-04-23T14:44:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.659+0000 7f3237618740 4 datascan.init: resolving metadata pool 16 2024-04-23T14:44:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.661+0000 7f322d400640 1 -- 172.21.15.59:0/541119204 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55fa49628c00 con 0x55fa4961c800 2024-04-23T14:44:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f322d400640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55fa49633000 0x55fa49462100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f322d400640 1 -- 172.21.15.59:0/541119204 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x55fa49623200 con 0x55fa4961c800 2024-04-23T14:44:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f3237618740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-23T14:44:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f3237618740 4 datascan.main: resolving metadata pool 16 2024-04-23T14:44:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55fa4961cc00 0x55fa49462680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55fa49633000 0x55fa49462100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d000 con 0x55fa4961cc00 2024-04-23T14:44:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.662+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55fa4961cc00 0x55fa49462680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.663+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55fa49633000 0x55fa49462100 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x55fa49497020 tx=0x55fa495d3020 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.663+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55fa4961cc00 0x55fa49462680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.663+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== osd_op_reply(1 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.664+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55fa4961dc00 0x55fa48a7a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.664+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d400 con 0x55fa4961dc00 2024-04-23T14:44:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.664+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55fa4961dc00 0x55fa48a7a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.664+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55fa4961dc00 0x55fa48a7a000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.665+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== osd_op_reply(2 [pgnls start_epoch 116 out=177b] v115'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.665+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d400 con 0x55fa4961cc00 2024-04-23T14:44:39.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.665+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== osd_op_reply(3 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.665+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d400 con 0x55fa4961cc00 2024-04-23T14:44:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.666+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 3 ==== osd_op_reply(4 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.666+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55fa4961d400 0x55fa48a7a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.666+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d800 con 0x55fa4961d400 2024-04-23T14:44:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.666+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55fa4961d400 0x55fa48a7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.667+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55fa4961d400 0x55fa48a7a580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.667+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== osd_op_reply(5 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.667+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d800 con 0x55fa4961cc00 2024-04-23T14:44:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.667+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 4 ==== osd_op_reply(6 [pgnls start_epoch 116 out=149b] v116'45 uv45 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.667+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55fa4961d800 0x55fa48a7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.667+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d000 con 0x55fa4961d800 2024-04-23T14:44:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.668+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55fa4961d800 0x55fa48a7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.669+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55fa4961d800 0x55fa48a7ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.670+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== osd_op_reply(7 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.670+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa4961d000 con 0x55fa4961dc00 2024-04-23T14:44:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.670+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== osd_op_reply(8 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.670+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55fa4961d000 0x55fa48a7b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.670+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49633c00 con 0x55fa4961d000 2024-04-23T14:44:39.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.671+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55fa4961d000 0x55fa48a7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.672+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55fa4961d000 0x55fa48a7b080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.673+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== osd_op_reply(9 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961d000 2024-04-23T14:44:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.673+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fa49633c00 0x55fa48a7b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.673+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94000 con 0x55fa49633c00 2024-04-23T14:44:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.673+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fa49633c00 0x55fa48a7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.674+0000 7f3234c00640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fa49633c00 0x55fa48a7b600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.675+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== osd_op_reply(10 [pgnls start_epoch 116 out=81b] v115'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.675+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94000 con 0x55fa4961dc00 2024-04-23T14:44:39.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.676+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 3 ==== osd_op_reply(11 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.676+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94000 con 0x55fa4961d400 2024-04-23T14:44:39.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.676+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== osd_op_reply(12 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.676+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94000 con 0x55fa49633c00 2024-04-23T14:44:39.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.677+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== osd_op_reply(13 [pgnls start_epoch 116 out=77b] v115'13 uv13 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.677+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94000 con 0x55fa4961d400 2024-04-23T14:44:39.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.678+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 3 ==== osd_op_reply(14 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.678+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94000 con 0x55fa4961dc00 2024-04-23T14:44:39.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.678+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 4 ==== osd_op_reply(15 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.678+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55fa48a94000 0x55fa48a7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.678+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94400 con 0x55fa48a94000 2024-04-23T14:44:39.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.679+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55fa48a94000 0x55fa48a7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.680+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55fa48a94000 0x55fa48a7bb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.681+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== osd_op_reply(16 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa48a94000 2024-04-23T14:44:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.681+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94400 con 0x55fa4961dc00 2024-04-23T14:44:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.681+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 5 ==== osd_op_reply(17 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.681+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94400 con 0x55fa4961d800 2024-04-23T14:44:39.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.682+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== osd_op_reply(18 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.682+0000 7f3234200640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55fa48a94400 0x55fa48a7c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:39.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.682+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa48a94400 2024-04-23T14:44:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.683+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55fa48a94400 0x55fa48a7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.683+0000 7f3235600640 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55fa48a94400 0x55fa48a7c100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:39.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.684+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== osd_op_reply(19 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.684+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961d000 2024-04-23T14:44:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.685+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== osd_op_reply(20 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961d000 2024-04-23T14:44:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.685+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961cc00 2024-04-23T14:44:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.685+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 5 ==== osd_op_reply(21 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.685+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961cc00 2024-04-23T14:44:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.686+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 6 ==== osd_op_reply(22 [pgnls start_epoch 116 out=97b] v115'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.686+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa49633c00 2024-04-23T14:44:39.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.687+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 3 ==== osd_op_reply(23 [pgnls start_epoch 116 out=74b] v111'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.687+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961dc00 2024-04-23T14:44:39.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.688+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 6 ==== osd_op_reply(24 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.688+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961d800 2024-04-23T14:44:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.689+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 3 ==== osd_op_reply(25 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.689+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa48a94400 2024-04-23T14:44:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.689+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== osd_op_reply(26 [pgnls start_epoch 116 out=98b] v115'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.689+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa48a94400 2024-04-23T14:44:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.690+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 3 ==== osd_op_reply(27 [pgnls start_epoch 116 out=76b] v116'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.690+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961d800 2024-04-23T14:44:39.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.691+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 4 ==== osd_op_reply(28 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.691+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961d400 2024-04-23T14:44:39.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.692+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 4 ==== osd_op_reply(29 [pgnls start_epoch 116 out=105b] v116'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.692+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961dc00 2024-04-23T14:44:39.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.693+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 7 ==== osd_op_reply(30 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.693+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961cc00 2024-04-23T14:44:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.693+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 7 ==== osd_op_reply(31 [pgnls start_epoch 116 out=73b] v116'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.693+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa49632c00 con 0x55fa4961d800 2024-04-23T14:44:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.694+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 5 ==== osd_op_reply(32 [pgnls start_epoch 116 out=101b] v116'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.694+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-23T14:44:39.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.694+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94c00 con 0x55fa4961dc00 2024-04-23T14:44:39.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.695+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.695+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-23T14:44:39.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.695+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95000 con 0x55fa4961dc00 2024-04-23T14:44:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.696+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.696+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-23T14:44:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.696+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95400 con 0x55fa4961dc00 2024-04-23T14:44:39.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.697+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.697+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-23T14:44:39.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.697+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95800 con 0x55fa4961dc00 2024-04-23T14:44:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.697+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.697+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-23T14:44:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95c00 con 0x55fa4961dc00 2024-04-23T14:44:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-23T14:44:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-23T14:44:39.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-23T14:44:39.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-23T14:44:39.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-23T14:44:39.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-23T14:44:39.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-23T14:44:39.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.698+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94800 con 0x55fa4961cc00 2024-04-23T14:44:39.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.699+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv40 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.699+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-23T14:44:39.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.699+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94c00 con 0x55fa4961dc00 2024-04-23T14:44:39.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.700+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.700+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-23T14:44:39.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.700+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95000 con 0x55fa49633c00 2024-04-23T14:44:39.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.701+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.701+0000 7f3237618740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-23T14:44:39.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.701+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-23T14:44:39.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.701+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-23T14:44:39.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.701+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95400 con 0x55fa4961dc00 2024-04-23T14:44:39.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.701+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.701+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-23T14:44:39.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95800 con 0x55fa4961cc00 2024-04-23T14:44:39.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3237618740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.702+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961dc00 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.703+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.703+0000 7f3237618740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.703+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.703+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-23T14:44:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.703+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8400 con 0x55fa48a94400 2024-04-23T14:44:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.704+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.704+0000 7f3237618740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-23T14:44:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.704+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-23T14:44:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.704+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-23T14:44:39.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.704+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8800 con 0x55fa4961d400 2024-04-23T14:44:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.705+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.705+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-23T14:44:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.705+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8c00 con 0x55fa4961d400 2024-04-23T14:44:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3237618740 20 datascan.scan_links: injected ino detected 2024-04-23T14:44:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-23T14:44:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-23T14:44:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-23T14:44:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-23T14:44:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3237618740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-23T14:44:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.706+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa9000 con 0x55fa4961d800 2024-04-23T14:44:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.707+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.708+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa9400 con 0x55fa4961cc00 2024-04-23T14:44:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.708+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 10 ==== osd_op_reply(48 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.708+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8800 con 0x55fa4961dc00 2024-04-23T14:44:39.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.709+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 16 ==== osd_op_reply(49 [pgnls start_epoch 116 out=177b] v115'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.709+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961cc00 2024-04-23T14:44:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.710+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 11 ==== osd_op_reply(50 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.710+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961cc00 2024-04-23T14:44:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.711+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 12 ==== osd_op_reply(51 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.711+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d400 2024-04-23T14:44:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.711+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 7 ==== osd_op_reply(52 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.711+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961cc00 2024-04-23T14:44:39.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.712+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 13 ==== osd_op_reply(53 [pgnls start_epoch 116 out=149b] v116'45 uv45 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.712+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d800 2024-04-23T14:44:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.713+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 7 ==== osd_op_reply(54 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.713+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961dc00 2024-04-23T14:44:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.714+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 17 ==== osd_op_reply(55 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.714+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d000 2024-04-23T14:44:39.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.715+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.4 v2:172.21.15.76:6816/3407674963 3 ==== osd_op_reply(56 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961d000 2024-04-23T14:44:39.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.715+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa49633c00 2024-04-23T14:44:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.716+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 5 ==== osd_op_reply(57 [pgnls start_epoch 116 out=81b] v115'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.716+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961dc00 2024-04-23T14:44:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.717+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 18 ==== osd_op_reply(58 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.717+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d400 2024-04-23T14:44:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.718+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 8 ==== osd_op_reply(59 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.718+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa49633c00 2024-04-23T14:44:39.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.719+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 6 ==== osd_op_reply(60 [pgnls start_epoch 116 out=77b] v115'13 uv13 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.719+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d400 2024-04-23T14:44:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.720+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 9 ==== osd_op_reply(61 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.720+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961dc00 2024-04-23T14:44:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.721+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 19 ==== osd_op_reply(62 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.721+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa48a94000 2024-04-23T14:44:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.722+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== osd_op_reply(63 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa48a94000 2024-04-23T14:44:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.722+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961dc00 2024-04-23T14:44:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.723+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 20 ==== osd_op_reply(64 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.723+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d800 2024-04-23T14:44:39.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.724+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 8 ==== osd_op_reply(65 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.724+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa48a94400 2024-04-23T14:44:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.725+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 5 ==== osd_op_reply(66 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.725+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d000 2024-04-23T14:44:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.726+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.4 v2:172.21.15.76:6816/3407674963 4 ==== osd_op_reply(67 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961d000 2024-04-23T14:44:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.726+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961cc00 2024-04-23T14:44:39.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.727+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 14 ==== osd_op_reply(68 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.727+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961cc00 2024-04-23T14:44:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.728+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 15 ==== osd_op_reply(69 [pgnls start_epoch 116 out=97b] v115'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.728+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa49633c00 2024-04-23T14:44:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.729+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 7 ==== osd_op_reply(70 [pgnls start_epoch 116 out=74b] v111'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.729+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961dc00 2024-04-23T14:44:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.730+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 21 ==== osd_op_reply(71 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.730+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d800 2024-04-23T14:44:39.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.731+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 9 ==== osd_op_reply(72 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.731+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa48a94400 2024-04-23T14:44:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.732+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 6 ==== osd_op_reply(73 [pgnls start_epoch 116 out=98b] v115'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.732+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa48a94400 2024-04-23T14:44:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.732+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 7 ==== osd_op_reply(74 [pgnls start_epoch 116 out=76b] v116'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.732+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d800 2024-04-23T14:44:39.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.734+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 10 ==== osd_op_reply(75 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.734+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d400 2024-04-23T14:44:39.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.735+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 10 ==== osd_op_reply(76 [pgnls start_epoch 116 out=105b] v116'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.735+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961dc00 2024-04-23T14:44:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.736+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 22 ==== osd_op_reply(77 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.736+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961cc00 2024-04-23T14:44:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.737+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 16 ==== osd_op_reply(78 [pgnls start_epoch 116 out=73b] v116'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.737+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8000 con 0x55fa4961d800 2024-04-23T14:44:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.738+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 11 ==== osd_op_reply(79 [pgnls start_epoch 116 out=101b] v116'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.738+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-23T14:44:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.738+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa9800 con 0x55fa4961dc00 2024-04-23T14:44:39.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.739+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.740+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-23T14:44:39.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.740+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa9c00 con 0x55fa4961dc00 2024-04-23T14:44:39.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.741+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.741+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-23T14:44:39.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.741+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aac000 con 0x55fa4961dc00 2024-04-23T14:44:39.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.742+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.742+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-23T14:44:39.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.742+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aac400 con 0x55fa4961dc00 2024-04-23T14:44:39.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.743+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.744+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-23T14:44:39.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.744+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aac800 con 0x55fa4961dc00 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-23T14:44:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.745+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aacc00 con 0x55fa4961cc00 2024-04-23T14:44:39.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.746+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv40 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.746+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-23T14:44:39.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.746+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aad000 con 0x55fa4961dc00 2024-04-23T14:44:39.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.747+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.747+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-23T14:44:39.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.747+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aad400 con 0x55fa49633c00 2024-04-23T14:44:39.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.748+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.748+0000 7f3237618740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-23T14:44:39.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.748+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-23T14:44:39.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.748+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-23T14:44:39.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.749+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aad800 con 0x55fa4961dc00 2024-04-23T14:44:39.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.750+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.750+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-23T14:44:39.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.750+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aadc00 con 0x55fa4961cc00 2024-04-23T14:44:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.751+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.0 v2:172.21.15.59:6817/2024703144 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961cc00 2024-04-23T14:44:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.751+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-23T14:44:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.751+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-23T14:44:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.751+0000 7f3237618740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-23T14:44:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.751+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-23T14:44:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.751+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-23T14:44:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.751+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa9400 con 0x55fa4961dc00 2024-04-23T14:44:39.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.752+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.2 v2:172.21.15.59:6816/77995666 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482500 con 0x55fa4961dc00 2024-04-23T14:44:39.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.752+0000 7f3237618740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-23T14:44:39.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.752+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-23T14:44:39.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.752+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-23T14:44:39.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.752+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8c00 con 0x55fa48a94400 2024-04-23T14:44:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.754+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.754+0000 7f3237618740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-23T14:44:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.754+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-23T14:44:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.754+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-23T14:44:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.754+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8800 con 0x55fa4961d400 2024-04-23T14:44:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.755+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.755+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-23T14:44:39.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.755+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48aa8400 con 0x55fa4961d400 2024-04-23T14:44:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 20 datascan.scan_links: injected ino detected 2024-04-23T14:44:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-23T14:44:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-23T14:44:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-23T14:44:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-23T14:44:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-23T14:44:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-23T14:44:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.756+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95800 con 0x55fa4961d800 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.757+0000 7f3234200640 1 -- 172.21.15.59:0/541119204 <== osd.5 v2:172.21.15.76:6808/2782996536 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55fa49482780 con 0x55fa4961d800 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.758+0000 7f3237618740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.758+0000 7f3237618740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.758+0000 7f3237618740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.758+0000 7f3237618740 10 datascan.scan_links: processing 1 injected_inos 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.758+0000 7f3237618740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.758+0000 7f3237618740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-23T14:44:39.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.758+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95000 con 0x55fa4961d400 2024-04-23T14:44:39.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.759+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.759+0000 7f3237618740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-23T14:44:39.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.759+0000 7f3237618740 20 datascan.scan_links: first is now 2 2024-04-23T14:44:39.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.759+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94800 con 0x55fa4961d400 2024-04-23T14:44:39.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.763+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.3 v2:172.21.15.59:6800/3205049728 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v116'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55fa49482000 con 0x55fa4961d400 2024-04-23T14:44:39.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.763+0000 7f3237618740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-23T14:44:39.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.763+0000 7f3237618740 10 datascan.scan_links: updating inotable 2024-04-23T14:44:39.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.763+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95400 con 0x55fa48a94400 2024-04-23T14:44:39.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.764+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.764+0000 7f3237618740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-23T14:44:39.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.764+0000 7f3237618740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-23T14:44:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.764+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a94c00 con 0x55fa48a94400 2024-04-23T14:44:39.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.768+0000 7f3235600640 1 -- 172.21.15.59:0/541119204 <== osd.1 v2:172.21.15.59:6808/4218092099 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v116'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55fa49482500 con 0x55fa48a94400 2024-04-23T14:44:39.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.768+0000 7f3237618740 10 datascan.scan_links: updating snaptable 2024-04-23T14:44:39.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.768+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55fa48a95c00 con 0x55fa49633c00 2024-04-23T14:44:39.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 <== osd.6 v2:172.21.15.76:6800/1890860772 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55fa49482000 con 0x55fa49633c00 2024-04-23T14:44:39.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55fa4961cc00 msgr2=0x55fa49462680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x55fa4961cc00 0x55fa49462680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55fa48a94400 msgr2=0x55fa48a7c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x55fa48a94400 0x55fa48a7c100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55fa4961dc00 msgr2=0x55fa48a7a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x55fa4961dc00 0x55fa48a7a000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55fa4961d400 msgr2=0x55fa48a7a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x55fa4961d400 0x55fa48a7a580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55fa4961d000 msgr2=0x55fa48a7b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x55fa4961d000 0x55fa48a7b080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55fa4961d800 msgr2=0x55fa48a7ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x55fa4961d800 0x55fa48a7ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fa49633c00 msgr2=0x55fa48a7b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x55fa49633c00 0x55fa48a7b600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 reap_dead start 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 reap_dead start 2024-04-23T14:44:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55fa48a94000 msgr2=0x55fa48a7bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x55fa48a94000 0x55fa48a7bb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.770+0000 7f3234c00640 1 -- 172.21.15.59:0/541119204 reap_dead start 2024-04-23T14:44:39.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55fa49633000 msgr2=0x55fa49462100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55fa49633000 0x55fa49462100 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x55fa49497020 tx=0x55fa495d3020 comp rx=0 tx=0).stop 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c800 msgr2=0x55fa49461b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c800 0x55fa49461b80 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x55fa4951be30 tx=0x55fa487a8b40 comp rx=0 tx=0).stop 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 shutdown_connections 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x55fa49633000 0x55fa49462100 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 --2- 172.21.15.59:0/541119204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa4961c800 0x55fa49461b80 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.771+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 >> 172.21.15.59:0/541119204 conn(0x55fa49493c00 msgr2=0x55fa487c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.772+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 shutdown_connections 2024-04-23T14:44:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.772+0000 7f3237618740 1 -- 172.21.15.59:0/541119204 wait complete. 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.773+0000 7f3237618740 1 -- 172.21.15.59:0/1576672581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa486f5800 msgr2=0x55fa49460580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.773+0000 7f3237618740 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa486f5800 0x55fa49460580 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x55fa4951b230 tx=0x55fa494949c0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.773+0000 7f3237618740 1 -- 172.21.15.59:0/1576672581 shutdown_connections 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.773+0000 7f3237618740 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fa486f5c00 0x55fa49460b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.773+0000 7f3237618740 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fa486f5800 0x55fa49460580 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.773+0000 7f3237618740 1 --2- 172.21.15.59:0/1576672581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fa486f4c00 0x55fa49460000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.773+0000 7f3237618740 1 -- 172.21.15.59:0/1576672581 >> 172.21.15.59:0/1576672581 conn(0x55fa486f5000 msgr2=0x55fa487c2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.774+0000 7f3237618740 1 -- 172.21.15.59:0/1576672581 shutdown_connections 2024-04-23T14:44:39.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:39.774+0000 7f3237618740 1 -- 172.21.15.59:0/1576672581 wait complete. 2024-04-23T14:44:39.787 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.235844, result: 2024-04-23T14:44:39.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:39.795 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:39.938 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-04-23T14:44:39.938 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-04-23T14:44:39.938 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:39.938 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:39.939 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:39.940 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.941 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-04-23T14:44:39.942 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: bdev async discard = True 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: bdev enable discard = True 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore allocator = bitmap 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-04-23T14:44:39.943 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-04-23T14:44:39.944 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:39.945 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:39.946 INFO:teuthology.orchestra.run.smithi076.stdout: mon down mkfs grace = 300 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-04-23T14:44:39.947 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = False 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:39.948 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:39.949 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:39.950 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.951 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.952 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:39.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:39.954 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.955 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.956 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-23T14:44:39.957 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-23T14:44:39.958 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = False 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:39.959 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:39.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:39.985 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-23T14:44:40.113 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-04-23T14:44:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-04-23T14:44:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-04-23T14:44:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-04-23T14:44:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: bdev async discard = True 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: bdev enable discard = True 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore allocator = bitmap 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: mon down mkfs grace = 300 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-04-23T14:44:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-04-23T14:44:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-23T14:44:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-04-23T14:44:40.124 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-04-23T14:44:40.125 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2024-04-23T14:44:40.126 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = b8f41672-e126-44ae-8657-b688ecdd9da9 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-04-23T14:44:40.127 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-04-23T14:44:40.128 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-04-23T14:44:40.129 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-04-23T14:44:40.130 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-04-23T14:44:40.131 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: mon down mkfs grace = 300 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:40.132 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-04-23T14:44:40.133 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-04-23T14:44:40.134 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-04-23T14:44:40.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-23T14:44:40.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.374+0000 7fd2c3200640 1 Processor -- start 2024-04-23T14:44:40.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.374+0000 7fd2c3200640 1 -- start start 2024-04-23T14:44:40.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.375+0000 7fd2c3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c40d15b0 0x7fd2c40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.375+0000 7fd2c3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2c40b4f20 0x7fd2c40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.375+0000 7fd2c3200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c40b5da0 con 0x7fd2c415ed20 2024-04-23T14:44:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.375+0000 7fd2c3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c415f100 con 0x7fd2c40b4f20 2024-04-23T14:44:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.375+0000 7fd2c3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c40d0a20 con 0x7fd2c40d15b0 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.376+0000 7fd2c2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2c40b4f20 0x7fd2c40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.376+0000 7fd2c2800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2c40b4f20 0x7fd2c40cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55718/0 (socket says 172.21.15.59:55718) 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.376+0000 7fd2c2800640 1 -- 172.21.15.59:0/3955358708 learned_addr learned my addr 172.21.15.59:0/3955358708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.376+0000 7fd2b9400640 1 --2- 172.21.15.59:0/3955358708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c40d15b0 0x7fd2c40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.377+0000 7fd2c2800640 1 -- 172.21.15.59:0/3955358708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c40d15b0 msgr2=0x7fd2c40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.377+0000 7fd2c2800640 1 --2- 172.21.15.59:0/3955358708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c40d15b0 0x7fd2c40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.377+0000 7fd2c2800640 1 -- 172.21.15.59:0/3955358708 >> v1:172.21.15.76:6789/0 conn(0x7fd2c415ed20 legacy=0x7fd2c40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:40.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.377+0000 7fd2c2800640 1 -- 172.21.15.59:0/3955358708 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c40d2ed0 con 0x7fd2c40b4f20 2024-04-23T14:44:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.378+0000 7fd2c2800640 1 --2- 172.21.15.59:0/3955358708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cf7c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f65c8d14953dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.378+0000 7fd2c1400640 1 -- 172.21.15.59:0/3955358708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc036070 con 0x7fd2c40b4f20 2024-04-23T14:44:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.378+0000 7fd2c1400640 1 -- 172.21.15.59:0/3955358708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc031600 con 0x7fd2c40b4f20 2024-04-23T14:44:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.378+0000 7fd2c1400640 1 -- 172.21.15.59:0/3955358708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0319d0 con 0x7fd2c40b4f20 2024-04-23T14:44:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 -- 172.21.15.59:0/3955358708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40b4f20 msgr2=0x7fd2c40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 --2- 172.21.15.59:0/3955358708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cf7c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc02a5f0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 -- 172.21.15.59:0/3955358708 shutdown_connections 2024-04-23T14:44:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 --2- 172.21.15.59:0/3955358708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cf7c0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 --2- 172.21.15.59:0/3955358708 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c40d15b0 0x7fd2c40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 -- 172.21.15.59:0/3955358708 >> 172.21.15.59:0/3955358708 conn(0x7fd2c408bf90 msgr2=0x7fd2c40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:40.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 -- 172.21.15.59:0/3955358708 shutdown_connections 2024-04-23T14:44:40.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.379+0000 7fd2c3200640 1 -- 172.21.15.59:0/3955358708 wait complete. 2024-04-23T14:44:40.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.381+0000 7fd2c3200640 1 Processor -- start 2024-04-23T14:44:40.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.381+0000 7fd2c3200640 1 -- start start 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2c3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2c3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40d15b0 0x7fd2c415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2c3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c415ed20 0x7fd2c40b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2c3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c40b3600 con 0x7fd2c40d15b0 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2c3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c40d3490 con 0x7fd2c40b4f20 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2c3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c40b3780 con 0x7fd2c415ed20 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40d15b0 0x7fd2c415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2b9400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40d15b0 0x7fd2c415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55728/0 (socket says 172.21.15.59:55728) 2024-04-23T14:44:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2b9400640 1 -- 172.21.15.59:0/1906979403 learned_addr learned my addr 172.21.15.59:0/1906979403 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2c1e00640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2b9400640 1 -- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c415ed20 msgr2=0x7fd2c40b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2b9400640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c415ed20 0x7fd2c40b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.382+0000 7fd2b9400640 1 -- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40b4f20 msgr2=0x7fd2c40cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2b9400640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2b9400640 1 -- 172.21.15.59:0/1906979403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c415c0f0 con 0x7fd2c40d15b0 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2c1e00640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2b9400640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40d15b0 0x7fd2c415bbb0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00e9b0 tx=0x7fd2ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2baa00640 1 -- 172.21.15.59:0/1906979403 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac017070 con 0x7fd2c40d15b0 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2baa00640 1 -- 172.21.15.59:0/1906979403 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac00c9a0 con 0x7fd2c40d15b0 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2baa00640 1 -- 172.21.15.59:0/1906979403 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00cd90 con 0x7fd2c40d15b0 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c40b6230 con 0x7fd2c40d15b0 2024-04-23T14:44:40.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.383+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c40b9ce0 con 0x7fd2c40d15b0 2024-04-23T14:44:40.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.384+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c40b9eb0 con 0x7fd2c40d15b0 2024-04-23T14:44:40.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.386+0000 7fd2baa00640 1 -- 172.21.15.59:0/1906979403 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2ac01a070 con 0x7fd2c40d15b0 2024-04-23T14:44:40.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.387+0000 7fd2baa00640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2a006c920 0x7fd2a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:40.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.387+0000 7fd2baa00640 1 -- 172.21.15.59:0/1906979403 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fd2ac08bd70 con 0x7fd2c40d15b0 2024-04-23T14:44:40.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.387+0000 7fd2c1e00640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2a006c920 0x7fd2a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:40.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.388+0000 7fd2c1e00640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2a006c920 0x7fd2a006ede0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40015e0 tx=0x7fd2b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:40.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.391+0000 7fd2baa00640 1 -- 172.21.15.59:0/1906979403 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2ac059f80 con 0x7fd2c40d15b0 2024-04-23T14:44:40.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.640+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd2c404fd90 con 0x7fd2c40d15b0 2024-04-23T14:44:40.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.985+0000 7fd2baa00640 1 -- 172.21.15.59:0/1906979403 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v109) ==== 162+0+0 (secure 0 0 0) 0x7fd2ac059880 con 0x7fd2c40d15b0 2024-04-23T14:44:40.986 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-23T14:44:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2a006c920 msgr2=0x7fd2a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2a006c920 0x7fd2a006ede0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40015e0 tx=0x7fd2b4009040 comp rx=0 tx=0).stop 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40d15b0 msgr2=0x7fd2c415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40d15b0 0x7fd2c415bbb0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00e9b0 tx=0x7fd2ac00ee70 comp rx=0 tx=0).stop 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 shutdown_connections 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fd2a006c920 0x7fd2a006ede0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2c415ed20 0x7fd2c40b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2c40d15b0 0x7fd2c415bbb0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 --2- 172.21.15.59:0/1906979403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40b4f20 0x7fd2c40cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.989+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 >> 172.21.15.59:0/1906979403 conn(0x7fd2c408bf90 msgr2=0x7fd2c406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.990+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 shutdown_connections 2024-04-23T14:44:40.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:40.990+0000 7fd2c3200640 1 -- 172.21.15.59:0/1906979403 wait complete. 2024-04-23T14:44:41.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:41.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.198+0000 7f431be00640 1 Processor -- start 2024-04-23T14:44:41.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.199+0000 7f431be00640 1 -- start start 2024-04-23T14:44:41.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.200+0000 7f431be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f431c0cd760 0x7f431c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.200+0000 7f431be00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431c160210 con 0x7f431c0b4690 2024-04-23T14:44:41.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.200+0000 7f431be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431c1604f0 con 0x7f431c15c3d0 2024-04-23T14:44:41.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.200+0000 7f431be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f431c1607d0 con 0x7f431c0cd760 2024-04-23T14:44:41.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.200+0000 7f431b400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f431c0b4690 0x7f431c0cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45924/0 (socket says 172.21.15.59:45924) 2024-04-23T14:44:41.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.200+0000 7f431b400640 1 -- 172.21.15.59:0/3372087848 learned_addr learned my addr 172.21.15.59:0/3372087848 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:41.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.200+0000 7f4313e00640 1 --2- 172.21.15.59:0/3372087848 >> v2:172.21.15.76:3300/0 conn(0x7f431c0cd760 0x7f431c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:41.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258252424 0 0) 0x7f431c160210 con 0x7f431c0b4690 2024-04-23T14:44:41.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4300003520 con 0x7f431c0b4690 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 992015041 0 0) 0x7f4300003520 con 0x7f431c0b4690 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 >> v2:172.21.15.76:3300/0 conn(0x7f431c0cd760 msgr2=0x7f431c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 --2- 172.21.15.59:0/3372087848 >> v2:172.21.15.76:3300/0 conn(0x7f431c0cd760 0x7f431c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 >> v1:172.21.15.76:6789/0 conn(0x7f431c15c3d0 legacy=0x7f431c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c160ab0 con 0x7f431c0b4690 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f43140019f0 con 0x7f431c0b4690 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 >> v1:172.21.15.59:6789/0 conn(0x7f431c0b4690 legacy=0x7f431c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f4300003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43000051c0 0x7f430000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1607d0 con 0x7f4300003a00 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f430000c0c0 con 0x7f4300004800 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.201+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f430000c2c0 con 0x7f43000051c0 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f431aa00640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f4300003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f4313e00640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f431aa00640 1 -- 172.21.15.59:0/3372087848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43000051c0 msgr2=0x7f430000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f431aa00640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f43000051c0 0x7f430000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f431aa00640 1 -- 172.21.15.59:0/3372087848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4300004800 msgr2=0x7f4300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f431aa00640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f431aa00640 1 -- 172.21.15.59:0/3372087848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4300003520 con 0x7f4300003a00 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.202+0000 7f431b400640 1 -- 172.21.15.59:0/3372087848 reap_dead start 2024-04-23T14:44:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.203+0000 7f431aa00640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f4300003e20 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f430c001750 tx=0x7f4300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef83d2c030f6f351 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.203+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c001370 con 0x7f4300003a00 2024-04-23T14:44:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.203+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c0052d0 con 0x7f4300003a00 2024-04-23T14:44:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.203+0000 7f431a000640 1 -- 172.21.15.59:0/3372087848 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c02abe0 con 0x7f4300003a00 2024-04-23T14:44:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.204+0000 7f431be00640 1 -- 172.21.15.59:0/3372087848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 msgr2=0x7f4300003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.204+0000 7f431be00640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f4300003e20 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f430c001750 tx=0x7f4300005830 comp rx=0 tx=0).stop 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.204+0000 7f431be00640 1 -- 172.21.15.59:0/3372087848 shutdown_connections 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.204+0000 7f431be00640 1 --2- 172.21.15.59:0/3372087848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f4300003e20 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.204+0000 7f431be00640 1 -- 172.21.15.59:0/3372087848 >> 172.21.15.59:0/3372087848 conn(0x7f431c0b21a0 msgr2=0x7f431c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.204+0000 7f431be00640 1 -- 172.21.15.59:0/3372087848 shutdown_connections 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.204+0000 7f431be00640 1 -- 172.21.15.59:0/3372087848 wait complete. 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.206+0000 7f431be00640 1 Processor -- start 2024-04-23T14:44:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.206+0000 7f431be00640 1 -- start start 2024-04-23T14:44:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f431c0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c15b5f0 0x7f431c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f431c15db20 0x7f431c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f431c15c7b0 con 0x7f4300003a00 2024-04-23T14:44:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0b4ae0 con 0x7f431c15b5f0 2024-04-23T14:44:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f431c15c930 con 0x7f431c15db20 2024-04-23T14:44:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f431c0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f431c0c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55744/0 (socket says 172.21.15.59:55744) 2024-04-23T14:44:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.207+0000 7f431aa00640 1 -- 172.21.15.59:0/696567028 learned_addr learned my addr 172.21.15.59:0/696567028 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431aa00640 1 -- 172.21.15.59:0/696567028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f431c15db20 msgr2=0x7f431c0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431aa00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f431c15db20 0x7f431c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431aa00640 1 -- 172.21.15.59:0/696567028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c15b5f0 msgr2=0x7f431c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431aa00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c15b5f0 0x7f431c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431aa00640 1 -- 172.21.15.59:0/696567028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c15dfc0 con 0x7f4300003a00 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431aa00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f431c0c7fb0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f430c000c00 tx=0x7f430c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f4312a00640 1 -- 172.21.15.59:0/696567028 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c03f030 con 0x7f4300003a00 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f4312a00640 1 -- 172.21.15.59:0/696567028 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c005480 con 0x7f4300003a00 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f4312a00640 1 -- 172.21.15.59:0/696567028 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c037040 con 0x7f4300003a00 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c0c74d0 con 0x7f4300003a00 2024-04-23T14:44:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.208+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c0c7a90 con 0x7f4300003a00 2024-04-23T14:44:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.210+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c04fd90 con 0x7f4300003a00 2024-04-23T14:44:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.211+0000 7f4312a00640 1 -- 172.21.15.59:0/696567028 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f430c042070 con 0x7f4300003a00 2024-04-23T14:44:41.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.211+0000 7f4312a00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f42f806c920 0x7f42f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:41.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.212+0000 7f4312a00640 1 -- 172.21.15.59:0/696567028 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f430c0ab2c0 con 0x7f4300003a00 2024-04-23T14:44:41.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.212+0000 7f4313e00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f42f806c920 0x7f42f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:41.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.212+0000 7f4313e00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f42f806c920 0x7f42f806ede0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f431c0cfba0 tx=0x7f4304008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:41.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.216+0000 7f4312a00640 1 -- 172.21.15.59:0/696567028 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f430c0794d0 con 0x7f4300003a00 2024-04-23T14:44:41.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.470+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f431c06e400 con 0x7f4300003a00 2024-04-23T14:44:41.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.471+0000 7f4312a00640 1 -- 172.21.15.59:0/696567028 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5181 (secure 0 0 0) 0x7f430c078dd0 con 0x7f4300003a00 2024-04-23T14:44:41.473 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:41.473 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:40.991946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9735":{"gid":9735,"name":"c","rank":0,"incarnation":110,"state":"up:replay","state_seq":1,"addr":"172.21.15.59:6837/3735308728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3735308728},{"type":"v1","addr":"172.21.15.59:6837","nonce":3735308728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-23T14:44:41.473 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 110 2024-04-23T14:44:41.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f42f806c920 msgr2=0x7f42f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f42f806c920 0x7f42f806ede0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f431c0cfba0 tx=0x7f4304008040 comp rx=0 tx=0).stop 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 msgr2=0x7f431c0c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f431c0c7fb0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f430c000c00 tx=0x7f430c001120 comp rx=0 tx=0).stop 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 shutdown_connections 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f42f806c920 0x7f42f806ede0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f431c15db20 0x7f431c0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f431c15b5f0 0x7f431c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 --2- 172.21.15.59:0/696567028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4300003a00 0x7f431c0c7fb0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 >> 172.21.15.59:0/696567028 conn(0x7f431c0b21a0 msgr2=0x7f431c0b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:41.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 shutdown_connections 2024-04-23T14:44:41.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:41.474+0000 7f431be00640 1 -- 172.21.15.59:0/696567028 wait complete. 2024-04-23T14:44:41.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:43:37.939514+0000', 'modified': '2024-04-23T14:44:40.991946+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9735}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9735': {'gid': 9735, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.59:6837/3735308728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 3735308728}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3735308728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:44:41.897 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9735:up:replay 2024-04-23T14:44:42.897 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:43.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.120+0000 7f70d3e00640 1 Processor -- start 2024-04-23T14:44:43.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.120+0000 7f70d3e00640 1 -- start start 2024-04-23T14:44:43.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.121+0000 7f70d3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d40cd760 0x7f70d40b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.121+0000 7f70d3e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d4160210 con 0x7f70d415c3d0 2024-04-23T14:44:43.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.121+0000 7f70d3e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d41604f0 con 0x7f70d40b4690 2024-04-23T14:44:43.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.121+0000 7f70d3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f70d41607d0 con 0x7f70d40cd760 2024-04-23T14:44:43.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d2a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f70d415c3d0 0x7f70d40cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45936/0 (socket says 172.21.15.59:45936) 2024-04-23T14:44:43.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d2a00640 1 -- 172.21.15.59:0/806644210 learned_addr learned my addr 172.21.15.59:0/806644210 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:43.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d2000640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d40cd760 0x7f70d40b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:43.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 427423061 0 0) 0x7f70d4160210 con 0x7f70d415c3d0 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70b8003520 con 0x7f70d415c3d0 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 198567869 0 0) 0x7f70b8003520 con 0x7f70d415c3d0 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d40cd760 msgr2=0x7f70d40b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d1600640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d40cd760 0x7f70d40b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.122+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 >> v1:172.21.15.76:6789/0 conn(0x7f70d40b4690 legacy=0x7f70d40cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d4160ab0 con 0x7f70d415c3d0 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f70c4001670 con 0x7f70d415c3d0 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 >> v1:172.21.15.59:6789/0 conn(0x7f70d415c3d0 legacy=0x7f70d40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d40cd760 0x7f70b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8004d40 0x7f70b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41607d0 con 0x7f70b80039e0 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70b800baa0 con 0x7f70b8004d40 2024-04-23T14:44:43.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f70b800bca0 con 0x7f70d40cd760 2024-04-23T14:44:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d2a00640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d2a00640 1 -- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d40cd760 msgr2=0x7f70b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d2a00640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d40cd760 0x7f70b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d2a00640 1 -- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8004d40 msgr2=0x7f70b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d2a00640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70b8004d40 0x7f70b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.123+0000 7f70d2a00640 1 -- 172.21.15.59:0/806644210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b8003520 con 0x7f70b80039e0 2024-04-23T14:44:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.124+0000 7f70d2a00640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70b8003e00 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f70c400c240 tx=0x7f70b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4f294a7b047558 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.125+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4007d10 con 0x7f70b80039e0 2024-04-23T14:44:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.125+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c4005ad0 con 0x7f70b80039e0 2024-04-23T14:44:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.125+0000 7f70d1600640 1 -- 172.21.15.59:0/806644210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4007070 con 0x7f70b80039e0 2024-04-23T14:44:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.125+0000 7f70d3e00640 1 -- 172.21.15.59:0/806644210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 msgr2=0x7f70b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.125+0000 7f70d3e00640 1 --2- 172.21.15.59:0/806644210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70b8003e00 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f70c400c240 tx=0x7f70b8005830 comp rx=0 tx=0).stop 2024-04-23T14:44:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.125+0000 7f70d3400640 1 -- 172.21.15.59:0/806644210 reap_dead start 2024-04-23T14:44:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.126+0000 7f70d3e00640 1 -- 172.21.15.59:0/806644210 shutdown_connections 2024-04-23T14:44:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.126+0000 7f70d3e00640 1 -- 172.21.15.59:0/806644210 >> 172.21.15.59:0/806644210 conn(0x7f70d40b21a0 msgr2=0x7f70d40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.126+0000 7f70d3e00640 1 -- 172.21.15.59:0/806644210 shutdown_connections 2024-04-23T14:44:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.126+0000 7f70d3e00640 1 -- 172.21.15.59:0/806644210 wait complete. 2024-04-23T14:44:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.127+0000 7f70d3e00640 1 Processor -- start 2024-04-23T14:44:43.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.128+0000 7f70d3e00640 1 -- start start 2024-04-23T14:44:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70d415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d415aea0 0x7f70d41590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d415d450 0x7f70d40ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f70d415c7b0 con 0x7f70b80039e0 2024-04-23T14:44:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70d40b4ac0 con 0x7f70d415d450 2024-04-23T14:44:43.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f70d415c930 con 0x7f70d415aea0 2024-04-23T14:44:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70d415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:43.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70d415eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55764/0 (socket says 172.21.15.59:55764) 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 -- 172.21.15.59:0/2728780016 learned_addr learned my addr 172.21.15.59:0/2728780016 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3400640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d415d450 0x7f70d40ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 -- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d415aea0 msgr2=0x7f70d41590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d415aea0 0x7f70d41590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 -- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d415d450 msgr2=0x7f70d40ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d415d450 0x7f70d40ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d2a00640 1 -- 172.21.15.59:0/2728780016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d415b5f0 con 0x7f70b80039e0 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.129+0000 7f70d3400640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d415d450 0x7f70d40ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.130+0000 7f70d2a00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70d415eb70 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f70c4001970 tx=0x7f70d41606c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.130+0000 7f70cb400640 1 -- 172.21.15.59:0/2728780016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4042020 con 0x7f70b80039e0 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.130+0000 7f70cb400640 1 -- 172.21.15.59:0/2728780016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c4004bf0 con 0x7f70b80039e0 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.130+0000 7f70cb400640 1 -- 172.21.15.59:0/2728780016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4007d10 con 0x7f70b80039e0 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.130+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d415b8d0 con 0x7f70b80039e0 2024-04-23T14:44:43.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.130+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d415cf30 con 0x7f70b80039e0 2024-04-23T14:44:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.131+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d404fd90 con 0x7f70b80039e0 2024-04-23T14:44:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.132+0000 7f70cb400640 1 -- 172.21.15.59:0/2728780016 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70c4045020 con 0x7f70b80039e0 2024-04-23T14:44:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.133+0000 7f70cb400640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70b406c8f0 0x7f70b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:43.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.133+0000 7f70cb400640 1 -- 172.21.15.59:0/2728780016 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f70c40ac0f0 con 0x7f70b80039e0 2024-04-23T14:44:43.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.134+0000 7f70d2000640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70b406c8f0 0x7f70b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:43.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.134+0000 7f70d2000640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70b406c8f0 0x7f70b406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f70c0006e80 tx=0x7f70c0006dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:43.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.138+0000 7f70cb400640 1 -- 172.21.15.59:0/2728780016 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70c407a4b0 con 0x7f70b80039e0 2024-04-23T14:44:43.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.386+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d406e400 con 0x7f70b80039e0 2024-04-23T14:44:43.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.387+0000 7f70cb400640 1 -- 172.21.15.59:0/2728780016 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5181 (secure 0 0 0) 0x7f70c4079db0 con 0x7f70b80039e0 2024-04-23T14:44:43.389 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:43.389 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:42.004001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9735":{"gid":9735,"name":"c","rank":0,"incarnation":110,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.59:6837/3735308728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3735308728},{"type":"v1","addr":"172.21.15.59:6837","nonce":3735308728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-23T14:44:43.389 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 112 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70b406c8f0 msgr2=0x7f70b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70b406c8f0 0x7f70b406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f70c0006e80 tx=0x7f70c0006dd0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 msgr2=0x7f70d415eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70d415eb70 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f70c4001970 tx=0x7f70d41606c0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 shutdown_connections 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f70b406c8f0 0x7f70b406edb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70d415aea0 0x7f70d41590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70d415d450 0x7f70d40ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 --2- 172.21.15.59:0/2728780016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70b80039e0 0x7f70d415eb70 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 >> 172.21.15.59:0/2728780016 conn(0x7f70d40b21a0 msgr2=0x7f70d40d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 shutdown_connections 2024-04-23T14:44:43.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:43.390+0000 7f70d3e00640 1 -- 172.21.15.59:0/2728780016 wait complete. 2024-04-23T14:44:43.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:43:37.939514+0000', 'modified': '2024-04-23T14:44:42.004001+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9735}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9735': {'gid': 9735, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.59:6837/3735308728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 3735308728}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3735308728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-23T14:44:43.813 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9735:up:rejoin 2024-04-23T14:44:44.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:44.886 INFO:tasks.ceph.mds.c.smithi059.stderr:2024-04-23T14:44:44.885+0000 7f9e5ca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:44:45.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.038+0000 7f41ffe00640 1 Processor -- start 2024-04-23T14:44:45.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.038+0000 7f41ffe00640 1 -- start start 2024-04-23T14:44:45.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.039+0000 7f41ffe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f420015c9d0 0x7f420009b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42000d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41ffe00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42000b35f0 con 0x7f42000b2e20 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41ffe00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f42000cd2d0 con 0x7f420015c9d0 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41ffe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f42000d13b0 con 0x7f42000d09e0 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41fea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f420015c9d0 0x7f420009b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41fea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f420015c9d0 0x7f420009b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55768/0 (socket says 172.21.15.59:55768) 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41fea00640 1 -- 172.21.15.59:0/2859705698 learned_addr learned my addr 172.21.15.59:0/2859705698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41ff400640 1 --2- 172.21.15.59:0/2859705698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42000d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41fea00640 1 -- 172.21.15.59:0/2859705698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 msgr2=0x7f42000d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41fea00640 1 --2- 172.21.15.59:0/2859705698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42000d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41fea00640 1 -- 172.21.15.59:0/2859705698 >> v1:172.21.15.76:6789/0 conn(0x7f42000b2e20 legacy=0x7f42000af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.040+0000 7f41fea00640 1 -- 172.21.15.59:0/2859705698 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42000b3890 con 0x7f420015c9d0 2024-04-23T14:44:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.042+0000 7f41fea00640 1 --2- 172.21.15.59:0/2859705698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420009b410 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009870 tx=0x7f41f002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3135c08fbb13cdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.042+0000 7f41fd600640 1 -- 172.21.15.59:0/2859705698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0037070 con 0x7f420015c9d0 2024-04-23T14:44:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.042+0000 7f41fd600640 1 -- 172.21.15.59:0/2859705698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f0032440 con 0x7f420015c9d0 2024-04-23T14:44:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.042+0000 7f41fd600640 1 -- 172.21.15.59:0/2859705698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0032810 con 0x7f420015c9d0 2024-04-23T14:44:45.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.042+0000 7f41ffe00640 1 -- 172.21.15.59:0/2859705698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 msgr2=0x7f420009b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.043+0000 7f41ffe00640 1 --2- 172.21.15.59:0/2859705698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420009b410 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009870 tx=0x7f41f002a200 comp rx=0 tx=0).stop 2024-04-23T14:44:45.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.043+0000 7f41ffe00640 1 -- 172.21.15.59:0/2859705698 shutdown_connections 2024-04-23T14:44:45.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.043+0000 7f41ffe00640 1 --2- 172.21.15.59:0/2859705698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42000d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.043+0000 7f41ffe00640 1 --2- 172.21.15.59:0/2859705698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420009b410 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.043+0000 7f41ffe00640 1 -- 172.21.15.59:0/2859705698 >> 172.21.15.59:0/2859705698 conn(0x7f42000b3ef0 msgr2=0x7f42000b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:45.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.043+0000 7f41ffe00640 1 -- 172.21.15.59:0/2859705698 shutdown_connections 2024-04-23T14:44:45.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.043+0000 7f41ffe00640 1 -- 172.21.15.59:0/2859705698 wait complete. 2024-04-23T14:44:45.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.044+0000 7f41ffe00640 1 Processor -- start 2024-04-23T14:44:45.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.045+0000 7f41ffe00640 1 -- start start 2024-04-23T14:44:45.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42000b2e20 0x7f420006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42001f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ffe00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f42000b3280 con 0x7f420015c9d0 2024-04-23T14:44:45.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ffe00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42000ccef0 con 0x7f42000b2e20 2024-04-23T14:44:45.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ffe00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f420015cdb0 con 0x7f42000d09e0 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420015e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55782/0 (socket says 172.21.15.59:55782) 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 -- 172.21.15.59:0/1641126696 learned_addr learned my addr 172.21.15.59:0/1641126696 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41fea00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42000b2e20 0x7f420006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41fe000640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42001f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 -- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 msgr2=0x7f42001f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42001f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 -- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42000b2e20 msgr2=0x7f420006e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42000b2e20 0x7f420006e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41ff400640 1 -- 172.21.15.59:0/1641126696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001f4db0 con 0x7f420015c9d0 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41fea00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42000b2e20 0x7f420006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.046+0000 7f41fe000640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42001f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.047+0000 7f41ff400640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420015e310 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f41f800cda0 tx=0x7f41f800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.047+0000 7f41f6a00640 1 -- 172.21.15.59:0/1641126696 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8003b90 con 0x7f420015c9d0 2024-04-23T14:44:45.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.047+0000 7f41f6a00640 1 -- 172.21.15.59:0/1641126696 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f8003d30 con 0x7f420015c9d0 2024-04-23T14:44:45.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.047+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001f5dd0 con 0x7f420015c9d0 2024-04-23T14:44:45.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.047+0000 7f41f6a00640 1 -- 172.21.15.59:0/1641126696 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8004240 con 0x7f420015c9d0 2024-04-23T14:44:45.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.047+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420006cdc0 con 0x7f420015c9d0 2024-04-23T14:44:45.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.048+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420004fd90 con 0x7f420015c9d0 2024-04-23T14:44:45.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.049+0000 7f41f6a00640 1 -- 172.21.15.59:0/1641126696 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41f80043e0 con 0x7f420015c9d0 2024-04-23T14:44:45.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.050+0000 7f41f6a00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f41e006c8f0 0x7f41e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.050+0000 7f41f6a00640 1 -- 172.21.15.59:0/1641126696 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f41f808c770 con 0x7f420015c9d0 2024-04-23T14:44:45.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.050+0000 7f41fea00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f41e006c8f0 0x7f41e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.051+0000 7f41fea00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f41e006c8f0 0x7f41e006edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f41f0034000 tx=0x7f41f002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.054+0000 7f41f6a00640 1 -- 172.21.15.59:0/1641126696 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41f801d340 con 0x7f420015c9d0 2024-04-23T14:44:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.304+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42000d07c0 con 0x7f420015c9d0 2024-04-23T14:44:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.305+0000 7f41f6a00640 1 -- 172.21.15.59:0/1641126696 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7f41f805aaa0 con 0x7f420015c9d0 2024-04-23T14:44:45.306 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:45.307 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:44.015119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9735":{"gid":9735,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3735308728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3735308728},{"type":"v1","addr":"172.21.15.59:6837","nonce":3735308728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9735,"qdb_cluster":[9735]},"id":8}]} 2024-04-23T14:44:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 113 2024-04-23T14:44:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f41e006c8f0 msgr2=0x7f41e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f41e006c8f0 0x7f41e006edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f41f0034000 tx=0x7f41f002e000 comp rx=0 tx=0).stop 2024-04-23T14:44:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 msgr2=0x7f420015e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420015e310 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f41f800cda0 tx=0x7f41f800a660 comp rx=0 tx=0).stop 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 shutdown_connections 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f41e006c8f0 0x7f41e006edb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42000d09e0 0x7f42001f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420015c9d0 0x7f420015e310 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 --2- 172.21.15.59:0/1641126696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42000b2e20 0x7f420006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 >> 172.21.15.59:0/1641126696 conn(0x7f42000b3ef0 msgr2=0x7f42000b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 shutdown_connections 2024-04-23T14:44:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.308+0000 7f41ffe00640 1 -- 172.21.15.59:0/1641126696 wait complete. 2024-04-23T14:44:45.731 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-23T14:43:37.939514+0000', 'modified': '2024-04-23T14:44:44.015119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9735}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9735': {'gid': 9735, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6837/3735308728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 3735308728}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 3735308728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9735, 'qdb_cluster': [9735]} 2024-04-23T14:44:45.731 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-23T14:44:45.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T14:44:45.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.952+0000 7f7138600640 1 Processor -- start 2024-04-23T14:44:45.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.952+0000 7f7138600640 1 -- start start 2024-04-23T14:44:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.954+0000 7f7138600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7130072880 0x7f713016d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.954+0000 7f7138600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713006e040 con 0x7f7130071d90 2024-04-23T14:44:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.954+0000 7f7138600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713006e320 con 0x7f71300cd1b0 2024-04-23T14:44:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.954+0000 7f7138600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f713006ddf0 con 0x7f7130072880 2024-04-23T14:44:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.954+0000 7f7135c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7130071d90 0x7f71300b3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45950/0 (socket says 172.21.15.59:45950) 2024-04-23T14:44:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.954+0000 7f7135c00640 1 -- 172.21.15.59:0/4145952339 learned_addr learned my addr 172.21.15.59:0/4145952339 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:45.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.954+0000 7f7136600640 1 --2- 172.21.15.59:0/4145952339 >> v2:172.21.15.59:3300/0 conn(0x7f7130072880 0x7f713016d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.955+0000 7f712fe00640 1 -- 172.21.15.59:0/4145952339 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487040330 0 0) 0x7f713006e040 con 0x7f7130071d90 2024-04-23T14:44:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.955+0000 7f712fe00640 1 -- 172.21.15.59:0/4145952339 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7118003520 con 0x7f7130071d90 2024-04-23T14:44:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.955+0000 7f7136600640 1 -- 172.21.15.59:0/4145952339 >> v1:172.21.15.76:6789/0 conn(0x7f71300cd1b0 legacy=0x7f7130072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.955+0000 7f7136600640 1 -- 172.21.15.59:0/4145952339 >> v1:172.21.15.59:6789/0 conn(0x7f7130071d90 legacy=0x7f71300b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.955+0000 7f7136600640 1 -- 172.21.15.59:0/4145952339 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713006e600 con 0x7f7130072880 2024-04-23T14:44:45.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.956+0000 7f7136600640 1 --2- 172.21.15.59:0/4145952339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7130072880 0x7f713016d960 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f7128009d90 tx=0x7f712802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b17c11c3cb12e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.956+0000 7f712fe00640 1 -- 172.21.15.59:0/4145952339 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128007740 con 0x7f7130072880 2024-04-23T14:44:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.956+0000 7f712fe00640 1 -- 172.21.15.59:0/4145952339 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71280078e0 con 0x7f7130072880 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.956+0000 7f712fe00640 1 -- 172.21.15.59:0/4145952339 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128007cb0 con 0x7f7130072880 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.957+0000 7f7138600640 1 -- 172.21.15.59:0/4145952339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7130072880 msgr2=0x7f713016d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.957+0000 7f7138600640 1 --2- 172.21.15.59:0/4145952339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7130072880 0x7f713016d960 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f7128009d90 tx=0x7f712802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.957+0000 7f7138600640 1 -- 172.21.15.59:0/4145952339 shutdown_connections 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.957+0000 7f7138600640 1 --2- 172.21.15.59:0/4145952339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7130072880 0x7f713016d960 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.957+0000 7f7138600640 1 -- 172.21.15.59:0/4145952339 >> 172.21.15.59:0/4145952339 conn(0x7f71300b2130 msgr2=0x7f71300b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.957+0000 7f7138600640 1 -- 172.21.15.59:0/4145952339 shutdown_connections 2024-04-23T14:44:45.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.957+0000 7f7138600640 1 -- 172.21.15.59:0/4145952339 wait complete. 2024-04-23T14:44:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.959+0000 7f7138600640 1 Processor -- start 2024-04-23T14:44:45.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.959+0000 7f7138600640 1 -- start start 2024-04-23T14:44:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7138600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7130071d90 0x7f7130153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7138600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7130072880 0x7f71300cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7138600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300cd1b0 0x7f71300ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7138600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f71301665b0 con 0x7f71300cd1b0 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7138600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7130166430 con 0x7f7130071d90 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7138600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f713016dea0 con 0x7f7130072880 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7136600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300cd1b0 0x7f71300ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7136600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300cd1b0 0x7f71300ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55800/0 (socket says 172.21.15.59:55800) 2024-04-23T14:44:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.960+0000 7f7136600640 1 -- 172.21.15.59:0/3540192640 learned_addr learned my addr 172.21.15.59:0/3540192640 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7135200640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7130072880 0x7f71300cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7135c00640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7130071d90 0x7f7130153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7136600640 1 -- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7130072880 msgr2=0x7f71300cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7136600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7130072880 0x7f71300cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7136600640 1 -- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7130071d90 msgr2=0x7f7130153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7136600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7130071d90 0x7f7130153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7136600640 1 -- 172.21.15.59:0/3540192640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7130164c90 con 0x7f71300cd1b0 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7135200640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7130072880 0x7f71300cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7135c00640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7130071d90 0x7f7130153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.961+0000 7f7136600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300cd1b0 0x7f71300ce350 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f7128009d90 tx=0x7f7128004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.962+0000 7f712d600640 1 -- 172.21.15.59:0/3540192640 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128040070 con 0x7f71300cd1b0 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.962+0000 7f712d600640 1 -- 172.21.15.59:0/3540192640 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7128031bc0 con 0x7f71300cd1b0 2024-04-23T14:44:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.962+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7130164f70 con 0x7f71300cd1b0 2024-04-23T14:44:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.962+0000 7f712d600640 1 -- 172.21.15.59:0/3540192640 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128009450 con 0x7f71300cd1b0 2024-04-23T14:44:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.962+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7130165530 con 0x7f71300cd1b0 2024-04-23T14:44:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.964+0000 7f712d600640 1 -- 172.21.15.59:0/3540192640 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7128039040 con 0x7f71300cd1b0 2024-04-23T14:44:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.965+0000 7f712d600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f711006c8f0 0x7f711006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.965+0000 7f712d600640 1 -- 172.21.15.59:0/3540192640 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f71280ad160 con 0x7f71300cd1b0 2024-04-23T14:44:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.966+0000 7f7135c00640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f711006c8f0 0x7f711006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.966+0000 7f713b59b740 1 Processor -- start 2024-04-23T14:44:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.966+0000 7f713b59b740 1 -- start start 2024-04-23T14:44:45.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.966+0000 7f7135c00640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f711006c8f0 0x7f711006edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7124005880 tx=0x7f7124009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8ec7b0 0x5644db89f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 0x5644db8cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f713b59b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf190 0x5644db8a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f713b59b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5644db809b10 con 0x5644db8cf190 2024-04-23T14:44:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644db7b02b0 con 0x5644db8ec7b0 2024-04-23T14:44:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5644db805ad0 con 0x5644db8cf8d0 2024-04-23T14:44:45.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f7136600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf190 0x5644db8a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f7136600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf190 0x5644db8a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55804/0 (socket says 172.21.15.59:55804) 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.967+0000 7f7136600640 1 -- 172.21.15.59:0/2239751026 learned_addr learned my addr 172.21.15.59:0/2239751026 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7136600640 1 -- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 msgr2=0x5644db8cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7135c00640 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8ec7b0 0x5644db89f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7136600640 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 0x5644db8cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7136600640 1 -- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8ec7b0 msgr2=0x5644db89f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7136600640 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8ec7b0 0x5644db89f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7136600640 1 -- 172.21.15.59:0/2239751026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644db51c2d0 con 0x5644db8cf190 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7135c00640 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8ec7b0 0x5644db89f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7136600640 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf190 0x5644db8a3fa0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f7128083440 tx=0x7f7128007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad758fb34c92adb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7122000640 1 -- 172.21.15.59:0/2239751026 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71280a9070 con 0x5644db8cf190 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7122000640 1 -- 172.21.15.59:0/2239751026 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712807f140 con 0x5644db8cf190 2024-04-23T14:44:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.968+0000 7f7122000640 1 -- 172.21.15.59:0/2239751026 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128083970 con 0x5644db8cf190 2024-04-23T14:44:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 -- 172.21.15.59:0/2239751026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf190 msgr2=0x5644db8a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf190 0x5644db8a3fa0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f7128083440 tx=0x7f7128007fc0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 -- 172.21.15.59:0/2239751026 shutdown_connections 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 0x5644db8cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf190 0x5644db8a3fa0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 --2- 172.21.15.59:0/2239751026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8ec7b0 0x5644db89f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 -- 172.21.15.59:0/2239751026 >> 172.21.15.59:0/2239751026 conn(0x5644db8a13b0 msgr2=0x5644db8e9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 -- 172.21.15.59:0/2239751026 shutdown_connections 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.969+0000 7f713b59b740 1 -- 172.21.15.59:0/2239751026 wait complete. 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.970+0000 7f713b59b740 1 -- start start 2024-04-23T14:44:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.970+0000 7f713b59b740 1 Processor -- start 2024-04-23T14:44:45.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.971+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 0x5644db8bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.971+0000 7f713b59b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8ec7b0 0x5644db8d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.971+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.971+0000 7f713b59b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5644db809b10 con 0x5644db8ec7b0 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.971+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644db7c3590 con 0x5644db8bc660 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.971+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5644db8a3560 con 0x5644db8cf8d0 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.971+0000 7f713b59b740 10 client.0 _ll_register_callbacks cb 0x5644db7a63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8ec7b0 0x5644db8d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8ec7b0 0x5644db8d2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55814/0 (socket says 172.21.15.59:55814) 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 -- 172.21.15.59:0/2895127906 learned_addr learned my addr 172.21.15.59:0/2895127906 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135c00640 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 0x5644db8bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7136600640 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 -- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 msgr2=0x5644db8bceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 0x5644db8bceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 -- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 msgr2=0x5644db8cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:45.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.972+0000 7f7135200640 1 -- 172.21.15.59:0/2895127906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644db5e66b0 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f7135200640 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8ec7b0 0x5644db8d2bb0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f711c004560 tx=0x7f711c00d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8233173a47a9b5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f7116a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f7116a00640 1 -- 172.21.15.59:0/2895127906 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c005230 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f7116a00640 1 -- 172.21.15.59:0/2895127906 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f711c0053d0 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f712807f140 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f7116a00640 1 -- 172.21.15.59:0/2895127906 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f711c007020 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f713b59b740 10 client.9831 fetch_fsmap learned FSMap version 113 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5644db8d07d0 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f7116a00640 1 -- 172.21.15.59:0/2895127906 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712807f140 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.973+0000 7f7116a00640 1 -- 172.21.15.59:0/2895127906 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 113) ==== 3494+0+0 (secure 0 0 0) 0x7f711c022080 con 0x5644db8ec7b0 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.974+0000 7f713b59b740 10 client.9831 fetch_fsmap finished waiting for FSMap version 113 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.974+0000 7f713b59b740 10 client.9831 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.974+0000 7f713b59b740 20 client.9831 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:45.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.974+0000 7f713b59b740 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db8cd510 0x5644db908710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.974+0000 7f713b59b740 4 client.9831 mds_command: new command op to 9735 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:44:45.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.974+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 --> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644db817530 con 0x5644db8cd510 2024-04-23T14:44:45.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.974+0000 7f7135c00640 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db8cd510 0x5644db908710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.975+0000 7f7135c00640 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db8cd510 0x5644db908710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=565c9c925ae764a server_cookie=45c6a4a351ac4e49 in_seq=0 out_seq=0 2024-04-23T14:44:45.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.975+0000 7f7116a00640 10 client.9831 ms_handle_connect on v2:172.21.15.59:6836/3735308728 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.975+0000 7f7116a00640 1 -- 172.21.15.59:0/2895127906 <== mds.0 v2:172.21.15.59:6836/3735308728 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644db817530 con 0x5644db8cd510 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.975+0000 7f7116a00640 10 client.9831 handle_command_reply: tid=0 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 client.9831 shutdown 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8ec7b0 msgr2=0x5644db8d2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8ec7b0 0x5644db8d2bb0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f711c004560 tx=0x7f711c00d5e0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 shutdown_connections 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db8cd510 0x5644db908710 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8ec7b0 0x5644db8d2bb0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f7116a00640 0 client.9831 ms_handle_reset on v2:172.21.15.59:6836/3735308728 2024-04-23T14:44:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 --2- 172.21.15.59:0/2895127906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8cf8d0 0x5644db8bceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 >> 172.21.15.59:0/2895127906 conn(0x5644db8cf190 msgr2=0x5644db89fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 shutdown_connections 2024-04-23T14:44:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 1 -- 172.21.15.59:0/2895127906 wait complete. 2024-04-23T14:44:45.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.976+0000 7f713b59b740 20 client.9831 trim_cache size 0 max 16384 2024-04-23T14:44:45.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.997+0000 7f713b59b740 1 Processor -- start 2024-04-23T14:44:45.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.997+0000 7f713b59b740 1 -- start start 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.997+0000 7f713b59b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8bc660 0x5644db8e0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.997+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8cf8d0 0x5644db95dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.997+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 0x5644db95e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f713b59b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f713015e570 con 0x5644db8bc660 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71301569e0 con 0x5644db8cf8d0 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f713015df20 con 0x5644db8ec7b0 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8bc660 0x5644db8e0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8bc660 0x5644db8e0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55818/0 (socket says 172.21.15.59:55818) 2024-04-23T14:44:45.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 -- 172.21.15.59:0/1498409359 learned_addr learned my addr 172.21.15.59:0/1498409359 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 -- 172.21.15.59:0/1498409359 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 msgr2=0x5644db95e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 --2- 172.21.15.59:0/1498409359 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 0x5644db95e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 -- 172.21.15.59:0/1498409359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8cf8d0 msgr2=0x5644db95dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 --2- 172.21.15.59:0/1498409359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8cf8d0 0x5644db95dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.998+0000 7f7135c00640 1 -- 172.21.15.59:0/1498409359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644db8d0130 con 0x5644db8bc660 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f7135c00640 1 --2- 172.21.15.59:0/1498409359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8bc660 0x5644db8e0ee0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f71240014c0 tx=0x7f7124011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb42ca882ef5e6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f7121600640 1 -- 172.21.15.59:0/1498409359 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124007720 con 0x5644db8bc660 2024-04-23T14:44:46.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f7121600640 1 -- 172.21.15.59:0/1498409359 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124029070 con 0x5644db8bc660 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f7121600640 1 -- 172.21.15.59:0/1498409359 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400d9a0 con 0x5644db8bc660 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f713b59b740 1 -- 172.21.15.59:0/1498409359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8bc660 msgr2=0x5644db8e0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f713b59b740 1 --2- 172.21.15.59:0/1498409359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8bc660 0x5644db8e0ee0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f71240014c0 tx=0x7f7124011000 comp rx=0 tx=0).stop 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f713b59b740 1 -- 172.21.15.59:0/1498409359 shutdown_connections 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f713b59b740 1 --2- 172.21.15.59:0/1498409359 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 0x5644db95e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f713b59b740 1 --2- 172.21.15.59:0/1498409359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8cf8d0 0x5644db95dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f713b59b740 1 --2- 172.21.15.59:0/1498409359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8bc660 0x5644db8e0ee0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:45.999+0000 7f713b59b740 1 -- 172.21.15.59:0/1498409359 >> 172.21.15.59:0/1498409359 conn(0x5644db8cf190 msgr2=0x5644db89fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.000+0000 7f713b59b740 1 -- 172.21.15.59:0/1498409359 shutdown_connections 2024-04-23T14:44:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.000+0000 7f713b59b740 1 -- 172.21.15.59:0/1498409359 wait complete. 2024-04-23T14:44:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.000+0000 7f713b59b740 1 -- start start 2024-04-23T14:44:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.001+0000 7f713b59b740 1 Processor -- start 2024-04-23T14:44:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f713b59b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf8d0 0x5644db9631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f713b59b740 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 0x5644db8cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f713b59b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5644db8bd4b0 con 0x5644db8cf8d0 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644db8d0b70 con 0x5644db8bc660 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f713b59b740 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x5644db8ebee0 con 0x5644db8ec7b0 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f713b59b740 10 client.0 _ll_register_callbacks cb 0x7f711c004090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f7135200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf8d0 0x5644db9631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f7135200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf8d0 0x5644db9631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55830/0 (socket says 172.21.15.59:55830) 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f7135200640 1 -- 172.21.15.59:0/2524559591 learned_addr learned my addr 172.21.15.59:0/2524559591 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.002+0000 7f7135c00640 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7135200640 1 -- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 msgr2=0x5644db8cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7135200640 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 0x5644db8cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7135200640 1 -- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 msgr2=0x5644db8e0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7135200640 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8e0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7135200640 1 -- 172.21.15.59:0/2524559591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644db8baeb0 con 0x5644db8cf8d0 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7135c00640 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8e0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7135200640 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf8d0 0x5644db9631d0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e905aff34709c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:46.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7116a00640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-04-23T14:44:46.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7116a00640 1 -- 172.21.15.59:0/2524559591 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c02e060 con 0x5644db8cf8d0 2024-04-23T14:44:46.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f7116a00640 1 -- 172.21.15.59:0/2524559591 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f711c0069d0 con 0x5644db8cf8d0 2024-04-23T14:44:46.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.003+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7124029070 con 0x5644db8cf8d0 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f7116a00640 1 -- 172.21.15.59:0/2524559591 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f711c034050 con 0x5644db8cf8d0 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 10 client.9837 fetch_fsmap learned FSMap version 113 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5644db8ce280 con 0x5644db8cf8d0 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f7116a00640 1 -- 172.21.15.59:0/2524559591 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124029070 con 0x5644db8cf8d0 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f7116a00640 1 -- 172.21.15.59:0/2524559591 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 113) ==== 3494+0+0 (secure 0 0 0) 0x7f711c037020 con 0x5644db8cf8d0 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 10 client.9837 fetch_fsmap finished waiting for FSMap version 113 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 10 client.9837 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 20 client.9837 populate_metadata read hostname 'smithi059' 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db90ce50 0x5644db8ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 4 client.9837 mds_command: new command op to 9735 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 --> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] -- command(tid 0: {"prefix": "status"}) -- 0x5644db8e18f0 con 0x5644db90ce50 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.004+0000 7f7135c00640 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db90ce50 0x5644db8ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.005+0000 7f7135c00640 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db90ce50 0x5644db8ed070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989ef3067450fb79 server_cookie=e2ac878acf3ec82e in_seq=0 out_seq=0 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.005+0000 7f7116a00640 10 client.9837 ms_handle_connect on v2:172.21.15.59:6836/3735308728 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.005+0000 7f7116a00640 1 -- 172.21.15.59:0/2524559591 <== mds.0 v2:172.21.15.59:6836/3735308728 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5644db8e18f0 con 0x5644db90ce50 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.005+0000 7f7116a00640 10 client.9837 handle_command_reply: tid=0 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.005+0000 7f713b59b740 1 client.9837 shutdown 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.005+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf8d0 msgr2=0x5644db9631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.005+0000 7f713b59b740 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf8d0 0x5644db9631d0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c0036f0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 shutdown_connections 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:6836/3735308728,v1:172.21.15.59:6837/3735308728] conn(0x5644db90ce50 0x5644db8ed070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x5644db8ec7b0 0x5644db8cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5644db8cf8d0 0x5644db9631d0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 --2- 172.21.15.59:0/2524559591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644db8bc660 0x5644db8e0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f7116a00640 0 client.9837 ms_handle_reset on v2:172.21.15.59:6836/3735308728 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 >> 172.21.15.59:0/2524559591 conn(0x5644db8cf190 msgr2=0x5644db8e97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 shutdown_connections 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 1 -- 172.21.15.59:0/2524559591 wait complete. 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f713b59b740 20 client.9837 trim_cache size 0 max 16384 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "b8f41672-e126-44ae-8657-b688ecdd9da9", 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9735, 2024-04-23T14:44:46.010 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 5.0107925370000004, 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 113, 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 116, 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 116, 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 7.0589943530000001 2024-04-23T14:44:46.011 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f711006c8f0 msgr2=0x7f711006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.006+0000 7f7138600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f711006c8f0 0x7f711006edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7124005880 tx=0x7f7124009040 comp rx=0 tx=0).stop 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300cd1b0 msgr2=0x7f71300ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300cd1b0 0x7f71300ce350 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f7128009d90 tx=0x7f7128004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 shutdown_connections 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f711006c8f0 0x7f711006edb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7130072880 0x7f71300cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300cd1b0 0x7f71300ce350 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 --2- 172.21.15.59:0/3540192640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7130071d90 0x7f7130153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 >> 172.21.15.59:0/3540192640 conn(0x7f71300b2130 msgr2=0x7f7130072170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:46.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 shutdown_connections 2024-04-23T14:44:46.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.007+0000 7f7138600640 1 -- 172.21.15.59:0/3540192640 wait complete. 2024-04-23T14:44:46.028 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-23T14:44:46.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.228+0000 7f2bd1c00640 1 Processor -- start 2024-04-23T14:44:46.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.229+0000 7f2bd1c00640 1 -- start start 2024-04-23T14:44:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.230+0000 7f2bd1c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bcc15b840 0x7f2bcc0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.230+0000 7f2bd1c00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2bcc06df00 0x7f2bcc06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.230+0000 7f2bd1c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc06ec60 con 0x7f2bcc06f7c0 2024-04-23T14:44:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.230+0000 7f2bd1c00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2bcc0b46e0 con 0x7f2bcc06df00 2024-04-23T14:44:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.230+0000 7f2bd1c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2bcc0b49c0 con 0x7f2bcc15b840 2024-04-23T14:44:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.231+0000 7f2bcaa00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2bcc06f7c0 0x7f2bcc06fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:45960/0 (socket says 172.21.15.59:45960) 2024-04-23T14:44:46.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.231+0000 7f2bcaa00640 1 -- 172.21.15.59:0/2665134926 learned_addr learned my addr 172.21.15.59:0/2665134926 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.231+0000 7f2bcb400640 1 --2- 172.21.15.59:0/2665134926 >> v2:172.21.15.76:3300/0 conn(0x7f2bcc06df00 0x7f2bcc06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.231+0000 7f2bca000640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bcc15b840 0x7f2bcc0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.231+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4286426157 0 0) 0x7f2bcc06ec60 con 0x7f2bcc06f7c0 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.231+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bb0003520 con 0x7f2bcc06f7c0 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3276156757 0 0) 0x7f2bb0003520 con 0x7f2bcc06f7c0 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bcc15b840 msgr2=0x7f2bcc0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bcc15b840 0x7f2bcc0b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 >> v2:172.21.15.76:3300/0 conn(0x7f2bcc06df00 msgr2=0x7f2bcc06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 --2- 172.21.15.59:0/2665134926 >> v2:172.21.15.76:3300/0 conn(0x7f2bcc06df00 0x7f2bcc06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc0b4ca0 con 0x7f2bcc06f7c0 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f2bbc001670 con 0x7f2bcc06f7c0 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 >> v1:172.21.15.59:6789/0 conn(0x7f2bcc06f7c0 legacy=0x7f2bcc06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb00039b0 0x7f2bb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bb0004360 0x7f2bb0004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb0004eb0 0x7f2bb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc0b46e0 con 0x7f2bb0004360 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc0b49c0 con 0x7f2bb0004eb0 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb000bf00 con 0x7f2bb00039b0 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bca000640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bb0004360 0x7f2bb0004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bca000640 1 -- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb00039b0 msgr2=0x7f2bb0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bca000640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb00039b0 0x7f2bb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bca000640 1 -- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb0004eb0 msgr2=0x7f2bb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.232+0000 7f2bca000640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb0004eb0 0x7f2bb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.233+0000 7f2bca000640 1 -- 172.21.15.59:0/2665134926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb0003520 con 0x7f2bb0004360 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.234+0000 7f2bca000640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bb0004360 0x7f2bb0004840 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009650 tx=0x7f2bb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ddf6441d31667e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.234+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8038070 con 0x7f2bb0004360 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.234+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb8033400 con 0x7f2bb0004360 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.234+0000 7f2bc9600640 1 -- 172.21.15.59:0/2665134926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8009690 con 0x7f2bb0004360 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.234+0000 7f2bd1c00640 1 -- 172.21.15.59:0/2665134926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bb0004360 msgr2=0x7f2bb0004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.234+0000 7f2bd1c00640 1 --2- 172.21.15.59:0/2665134926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bb0004360 0x7f2bb0004840 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009650 tx=0x7f2bb0005830 comp rx=0 tx=0).stop 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.234+0000 7f2bcb400640 1 -- 172.21.15.59:0/2665134926 reap_dead start 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.235+0000 7f2bd1c00640 1 -- 172.21.15.59:0/2665134926 shutdown_connections 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.235+0000 7f2bd1c00640 1 -- 172.21.15.59:0/2665134926 >> 172.21.15.59:0/2665134926 conn(0x7f2bcc067ea0 msgr2=0x7f2bcc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:46.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.235+0000 7f2bd1c00640 1 -- 172.21.15.59:0/2665134926 shutdown_connections 2024-04-23T14:44:46.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.235+0000 7f2bd1c00640 1 -- 172.21.15.59:0/2665134926 wait complete. 2024-04-23T14:44:46.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.236+0000 7f2bd1c00640 1 Processor -- start 2024-04-23T14:44:46.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.237+0000 7f2bd1c00640 1 -- start start 2024-04-23T14:44:46.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bd1c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb0004360 0x7f2bcc145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bd1c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bcc14ce90 0x7f2bcc14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bd1c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bcc0b1220 0x7f2bcc0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bd1c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc0b54a0 con 0x7f2bcc0b1220 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bd1c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc06e8d0 con 0x7f2bcc14ce90 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bd1c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2bcc0b5620 con 0x7f2bb0004360 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bcc0b1220 0x7f2bcc0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bcc0b1220 0x7f2bcc0b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55852/0 (socket says 172.21.15.59:55852) 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 -- 172.21.15.59:0/3024995860 learned_addr learned my addr 172.21.15.59:0/3024995860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bca000640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bcc14ce90 0x7f2bcc14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcaa00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb0004360 0x7f2bcc145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 -- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb0004360 msgr2=0x7f2bcc145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb0004360 0x7f2bcc145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 -- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bcc14ce90 msgr2=0x7f2bcc14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bcc14ce90 0x7f2bcc14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.238+0000 7f2bcb400640 1 -- 172.21.15.59:0/3024995860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc146220 con 0x7f2bcc0b1220 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bcaa00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb0004360 0x7f2bcc145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bca000640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bcc14ce90 0x7f2bcc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bcb400640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bcc0b1220 0x7f2bcc0b16a0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400ede0 tx=0x7f2bc400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bc2a00640 1 -- 172.21.15.59:0/3024995860 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4018020 con 0x7f2bcc0b1220 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bc2a00640 1 -- 172.21.15.59:0/3024995860 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc400bcb0 con 0x7f2bcc0b1220 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bc2a00640 1 -- 172.21.15.59:0/3024995860 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400be60 con 0x7f2bcc0b1220 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc1506c0 con 0x7f2bcc0b1220 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.239+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc150c80 con 0x7f2bcc0b1220 2024-04-23T14:44:46.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.240+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bcc068a80 con 0x7f2bcc0b1220 2024-04-23T14:44:46.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.241+0000 7f2bc2a00640 1 -- 172.21.15.59:0/3024995860 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bc4005d10 con 0x7f2bcc0b1220 2024-04-23T14:44:46.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.242+0000 7f2bc2a00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2bac06c8f0 0x7f2bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:46.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.242+0000 7f2bc2a00640 1 -- 172.21.15.59:0/3024995860 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f2bc408c360 con 0x7f2bcc0b1220 2024-04-23T14:44:46.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.243+0000 7f2bcaa00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2bac06c8f0 0x7f2bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:46.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.243+0000 7f2bcaa00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2bac06c8f0 0x7f2bac06edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0017e0 tx=0x7f2bbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:46.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.247+0000 7f2bc2a00640 1 -- 172.21.15.59:0/3024995860 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bc4022030 con 0x7f2bcc0b1220 2024-04-23T14:44:46.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.492+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2bcc06f370 con 0x7f2bcc0b1220 2024-04-23T14:44:46.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.493+0000 7f2bc2a00640 1 -- 172.21.15.59:0/3024995860 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) ==== 53+0+73 (secure 0 0 0) 0x7f2bc405a690 con 0x7f2bcc0b1220 2024-04-23T14:44:46.494 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:44:46.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.495+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2bac06c8f0 msgr2=0x7f2bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.495+0000 7f2bd1c00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2bac06c8f0 0x7f2bac06edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0017e0 tx=0x7f2bbc009040 comp rx=0 tx=0).stop 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bcc0b1220 msgr2=0x7f2bcc0b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bcc0b1220 0x7f2bcc0b16a0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400ede0 tx=0x7f2bc400b480 comp rx=0 tx=0).stop 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 shutdown_connections 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2bac06c8f0 0x7f2bac06edb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bcc0b1220 0x7f2bcc0b16a0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bcc14ce90 0x7f2bcc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 --2- 172.21.15.59:0/3024995860 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2bb0004360 0x7f2bcc145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 >> 172.21.15.59:0/3024995860 conn(0x7f2bcc067ea0 msgr2=0x7f2bcc0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 shutdown_connections 2024-04-23T14:44:46.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46.496+0000 7f2bd1c00640 1 -- 172.21.15.59:0/3024995860 wait complete. 2024-04-23T14:44:46.516 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:44:46.516 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:44:46.516 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-04-23T14:44:46.516 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:44:46.517 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:46.517 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:44:46.517 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:44:46.517 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:44:46.517 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:46.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:46.517 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-04-23T14:44:46.569 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:44:46.570 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41662sec preferred_lft 41662sec 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b84e:baff:feaf:cfd/64 scope link 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:46.571 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:44:46.572 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether a2:15:34:dd:56:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:46.572 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::a015:34ff:fedd:562d/64 scope link 2024-04-23T14:44:46.572 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:46.572 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-23T14:44:46.572 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 02:34:67:a4:86:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:46.572 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::34:67ff:fea4:8622/64 scope link 2024-04-23T14:44:46.572 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:44:46.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:46.573 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:44:46.656 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:44:46.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:46.661 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-04-23T14:44:46.721 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-04-23T14:44:46.722 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:44:46.722 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T14:44:46.722 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T14:44:46.722 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:44:46.723 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:44:46.723 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-04-23T14:44:46.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T14:44:46.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T14:44:46.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T14:44:46.723 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-04-23T14:44:46.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:44:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:46.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:46.948 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:44:47.001 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:44:47.001 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:44:47.002 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:47.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:47.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-04-23T14:44:47.098 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T14:44:47.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:44:47.098 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:47.158 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:44:47.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:44:47.158 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:47.214 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:44:47.304 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:44:47.362 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:44:47.363 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:44:47.364 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:44:47.365 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:44:47.365 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:44:47.365 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:47.365 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:47.365 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:47.365 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:47.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:47.366 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:44:47.421 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T14:44:47.422 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T14:44:47.427 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-04-23T14:44:47.494 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-04-23T14:44:47.537 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-23T14:44:47.538 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-0 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-1 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-2 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph/osd/ceph-3 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:47.539 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:44:47.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:47.540 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-04-23T14:44:47.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:2024-04-23T14:44:47.601+0000 7faae0520480 -1 init, newargv = 0x561f8987d3e0 newargc=15 2024-04-23T14:44:47.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[79809]: starting ceph client 2024-04-23T14:44:47.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[79809]: starting fuse 2024-04-23T14:44:47.627 INFO:teuthology.orchestra.run.smithi059.stdout:42 2024-04-23T14:44:47.627 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-23T14:44:47.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-04-23T14:44:47.627 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-04-23T14:44:47.627 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-04-23T14:44:47.627 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:47.627 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-04-23T14:44:47.627 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:47.627 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-04-23T14:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T14:44:47.629 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:47.629 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.0")) 2024-04-23T14:44:47.629 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:47.829 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.0.79809.asok 2024-04-23T14:44:47.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:47.847 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.79809.asok 2024-04-23T14:44:47.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:47.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.79809.asok status 2024-04-23T14:44:48.083 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:44:48.083 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "d44fddd910b5e94da0f2357af4be37d7bd495ae0", 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "0", 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "79809", 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 9852, 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-04-23T14:44:48.084 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 9852 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1940866355 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1940866355 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.9852 192.168.0.1:0/1940866355", 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/1940866355", 2024-04-23T14:44:48.085 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-04-23T14:44:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 113, 2024-04-23T14:44:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 116, 2024-04-23T14:44:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-04-23T14:44:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-04-23T14:44:48.086 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-04-23T14:44:48.086 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:44:48.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:48.100 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-04-23T14:44:48.100 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:48.155 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-04-23T14:44:48.155 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:48.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:48.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:48.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:48.244 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-23T14:44:48.328 INFO:teuthology.orchestra.run.smithi059.stdout:./ 2024-04-23T14:44:48.328 INFO:teuthology.orchestra.run.smithi059.stdout:./parent 2024-04-23T14:44:48.329 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed 2024-04-23T14:44:48.329 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/charlie 2024-04-23T14:44:48.329 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/alpha 2024-04-23T14:44:48.329 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/flushed/bravo 2024-04-23T14:44:48.329 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed 2024-04-23T14:44:48.329 INFO:teuthology.orchestra.run.smithi059.stdout:./parent/unflushed/jfile 2024-04-23T14:44:48.330 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:48.330 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:48.330 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:48.330 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:48.330 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-23T14:44:48.330 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:48.473 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-04-23T14:44:48.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:48.487 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:48.487 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:48.487 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:48.487 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:48.488 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-23T14:44:48.488 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:48.681 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627776 2024-04-23T14:44:48.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:48.696 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:48.696 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:48.696 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:48.696 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:48.696 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-23T14:44:48.696 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:48.890 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2024-04-23T14:44:48.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:48.905 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:48.905 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:48.905 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:48.905 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:48.905 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-23T14:44:48.905 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:49.100 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628280 2024-04-23T14:44:49.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:49.116 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:49.116 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:49.116 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:49.116 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:49.116 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-23T14:44:49.116 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:49.313 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627778 2024-04-23T14:44:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:49.328 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:49.328 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:49.328 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:49.328 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:49.328 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-23T14:44:49.329 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:49.524 INFO:teuthology.orchestra.run.smithi059.stdout:1099511628279 2024-04-23T14:44:49.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:49.540 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:49.540 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:49.540 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:49.541 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:49.541 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-23T14:44:49.541 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:49.724 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627779 2024-04-23T14:44:49.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:49.738 DEBUG:teuthology.orchestra.run.smithi059:> stdin-killer -- python3 -c ' 2024-04-23T14:44:49.738 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-04-23T14:44:49.738 DEBUG:teuthology.orchestra.run.smithi059:> import stat 2024-04-23T14:44:49.738 DEBUG:teuthology.orchestra.run.smithi059:> 2024-04-23T14:44:49.738 DEBUG:teuthology.orchestra.run.smithi059:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-23T14:44:49.738 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-04-23T14:44:49.931 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627780 2024-04-23T14:44:49.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:44:49.945 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-23T14:44:49.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:44:50.107 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-23T14:44:50.217 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-23T14:44:50.217 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:44:50.217 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:44:50.217 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:50.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:50.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.492+0000 7fe625a00640 1 Processor -- start 2024-04-23T14:44:50.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.493+0000 7fe625a00640 1 -- start start 2024-04-23T14:44:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6200b5ff0 0x7fe6200b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6200d0d90 0x7fe6200d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6200cf030 con 0x7fe62015e7c0 2024-04-23T14:44:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fe6200b7000 con 0x7fe6200d0d90 2024-04-23T14:44:50.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe6200d1720 con 0x7fe6200b5ff0 2024-04-23T14:44:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6200d0d90 0x7fe6200d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe6200d0d90 0x7fe6200d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34574/0 (socket says 172.21.15.59:34574) 2024-04-23T14:44:50.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.494+0000 7fe625000640 1 -- 172.21.15.59:0/3444866090 learned_addr learned my addr 172.21.15.59:0/3444866090 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.495+0000 7fe61f400640 1 --2- 172.21.15.59:0/3444866090 >> v2:172.21.15.76:3300/0 conn(0x7fe6200b5ff0 0x7fe6200b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.495+0000 7fe625000640 1 -- 172.21.15.59:0/3444866090 >> v2:172.21.15.76:3300/0 conn(0x7fe6200b5ff0 msgr2=0x7fe6200b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.495+0000 7fe625000640 1 --2- 172.21.15.59:0/3444866090 >> v2:172.21.15.76:3300/0 conn(0x7fe6200b5ff0 0x7fe6200b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.495+0000 7fe625000640 1 -- 172.21.15.59:0/3444866090 >> v1:172.21.15.76:6789/0 conn(0x7fe62015e7c0 legacy=0x7fe6200d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.495+0000 7fe625000640 1 -- 172.21.15.59:0/3444866090 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200b6a40 con 0x7fe6200d0d90 2024-04-23T14:44:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.496+0000 7fe625000640 1 --2- 172.21.15.59:0/3444866090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6200d0d90 0x7fe6200d11e0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fe6200d55c0 tx=0x7fe61802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17de5aaa2eabceec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:50.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.496+0000 7fe61ea00640 1 -- 172.21.15.59:0/3444866090 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618036070 con 0x7fe6200d0d90 2024-04-23T14:44:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.496+0000 7fe61ea00640 1 -- 172.21.15.59:0/3444866090 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618031600 con 0x7fe6200d0d90 2024-04-23T14:44:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.496+0000 7fe61ea00640 1 -- 172.21.15.59:0/3444866090 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6180319d0 con 0x7fe6200d0d90 2024-04-23T14:44:50.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 -- 172.21.15.59:0/3444866090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6200d0d90 msgr2=0x7fe6200d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 --2- 172.21.15.59:0/3444866090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6200d0d90 0x7fe6200d11e0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fe6200d55c0 tx=0x7fe61802a6f0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 -- 172.21.15.59:0/3444866090 shutdown_connections 2024-04-23T14:44:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 --2- 172.21.15.59:0/3444866090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6200d0d90 0x7fe6200d11e0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 --2- 172.21.15.59:0/3444866090 >> v2:172.21.15.76:3300/0 conn(0x7fe6200b5ff0 0x7fe6200b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 -- 172.21.15.59:0/3444866090 >> 172.21.15.59:0/3444866090 conn(0x7fe62009d920 msgr2=0x7fe62008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 -- 172.21.15.59:0/3444866090 shutdown_connections 2024-04-23T14:44:50.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.497+0000 7fe625a00640 1 -- 172.21.15.59:0/3444866090 wait complete. 2024-04-23T14:44:50.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.499+0000 7fe625a00640 1 Processor -- start 2024-04-23T14:44:50.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.499+0000 7fe625a00640 1 -- start start 2024-04-23T14:44:50.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6200b5ff0 0x7fe62015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6200d0d90 0x7fe6200cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e7c0 0x7fe6200cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200cf2b0 con 0x7fe62015e7c0 2024-04-23T14:44:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200cf620 con 0x7fe6200b5ff0 2024-04-23T14:44:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200d2800 con 0x7fe6200d0d90 2024-04-23T14:44:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e7c0 0x7fe6200cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e7c0 0x7fe6200cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34590/0 (socket says 172.21.15.59:34590) 2024-04-23T14:44:50.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.500+0000 7fe625000640 1 -- 172.21.15.59:0/1211088599 learned_addr learned my addr 172.21.15.59:0/1211088599 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe625000640 1 -- 172.21.15.59:0/1211088599 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6200d0d90 msgr2=0x7fe6200cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe625000640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6200d0d90 0x7fe6200cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe625000640 1 -- 172.21.15.59:0/1211088599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6200b5ff0 msgr2=0x7fe62015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe625000640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6200b5ff0 0x7fe62015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe625000640 1 -- 172.21.15.59:0/1211088599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200cbf20 con 0x7fe62015e7c0 2024-04-23T14:44:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe625000640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e7c0 0x7fe6200cd950 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fe61802ace0 tx=0x7fe618004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:50.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe617e00640 1 -- 172.21.15.59:0/1211088599 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618041020 con 0x7fe62015e7c0 2024-04-23T14:44:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe617e00640 1 -- 172.21.15.59:0/1211088599 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618008690 con 0x7fe62015e7c0 2024-04-23T14:44:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe617e00640 1 -- 172.21.15.59:0/1211088599 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618036040 con 0x7fe62015e7c0 2024-04-23T14:44:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.501+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6200cc200 con 0x7fe62015e7c0 2024-04-23T14:44:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.502+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6200cc7c0 con 0x7fe62015e7c0 2024-04-23T14:44:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.503+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62004fd90 con 0x7fe62015e7c0 2024-04-23T14:44:50.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.504+0000 7fe617e00640 1 -- 172.21.15.59:0/1211088599 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe618044020 con 0x7fe62015e7c0 2024-04-23T14:44:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.505+0000 7fe617e00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.505+0000 7fe617e00640 1 -- 172.21.15.59:0/1211088599 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fe6180acd80 con 0x7fe62015e7c0 2024-04-23T14:44:50.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.505+0000 7fe61fe00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.506+0000 7fe61fe00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe610009650 tx=0x7fe610006ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:50.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.510+0000 7fe617e00640 1 -- 172.21.15.59:0/1211088599 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe61804b030 con 0x7fe62015e7c0 2024-04-23T14:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.717+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe620162c70 con 0x7fe62015e7c0 2024-04-23T14:44:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.720+0000 7fe617e00640 1 -- 172.21.15.59:0/1211088599 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16919 (secure 0 0 0) 0x7fe6200cf030 con 0x7fe62015e7c0 2024-04-23T14:44:50.722 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:50.722 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":116,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:44:38.869299+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6836/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6837/1061483588":"2024-04-24T14:44:14.736925+0000","172.21.15.59:6836/1061483588":"2024-04-24T14:44:14.736925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.722+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe5fc06c8f0 msgr2=0x7fe5fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.722+0000 7fe625a00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe610009650 tx=0x7fe610006ba0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.722+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e7c0 msgr2=0x7fe6200cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e7c0 0x7fe6200cd950 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fe61802ace0 tx=0x7fe618004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 shutdown_connections 2024-04-23T14:44:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6200d0d90 0x7fe6200cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe62015e7c0 0x7fe6200cd950 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 --2- 172.21.15.59:0/1211088599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6200b5ff0 0x7fe62015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 >> 172.21.15.59:0/1211088599 conn(0x7fe62009d920 msgr2=0x7fe62015f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 shutdown_connections 2024-04-23T14:44:50.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.723+0000 7fe625a00640 1 -- 172.21.15.59:0/1211088599 wait complete. 2024-04-23T14:44:50.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:44:50.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:44:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.959+0000 7f3a61400640 1 Processor -- start 2024-04-23T14:44:50.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.959+0000 7f3a61400640 1 -- start start 2024-04-23T14:44:50.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.961+0000 7f3a61400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.961+0000 7f3a61400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a5c0b4f20 0x7f3a5c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.961+0000 7f3a61400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a5c0b5da0 con 0x7f3a5c15ed20 2024-04-23T14:44:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.961+0000 7f3a61400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c15f100 con 0x7f3a5c0b4f20 2024-04-23T14:44:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.961+0000 7f3a61400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d0a20 con 0x7f3a5c0d15b0 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.962+0000 7f3a60a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a5c0b4f20 0x7f3a5c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.962+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.962+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:40580/0 (socket says 172.21.15.59:40580) 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.962+0000 7f3a53e00640 1 -- 172.21.15.59:0/4242554518 learned_addr learned my addr 172.21.15.59:0/4242554518 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.962+0000 7f3a60a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a5c0b4f20 0x7f3a5c0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:55234/0 (socket says 172.21.15.59:55234) 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.963+0000 7f3a53e00640 1 -- 172.21.15.59:0/4242554518 >> v2:172.21.15.76:3300/0 conn(0x7f3a5c0b4f20 msgr2=0x7f3a5c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.963+0000 7f3a53e00640 1 --2- 172.21.15.59:0/4242554518 >> v2:172.21.15.76:3300/0 conn(0x7f3a5c0b4f20 0x7f3a5c0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.963+0000 7f3a53e00640 1 -- 172.21.15.59:0/4242554518 >> v1:172.21.15.76:6789/0 conn(0x7f3a5c15ed20 legacy=0x7f3a5c0d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.963+0000 7f3a53e00640 1 -- 172.21.15.59:0/4242554518 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c0d2ed0 con 0x7f3a5c0d15b0 2024-04-23T14:44:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.964+0000 7f3a53e00640 1 --2- 172.21.15.59:0/4242554518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0cf280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009870 tx=0x7f3a4402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96b1348a3d6644 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.965+0000 7f3a5b400640 1 -- 172.21.15.59:0/4242554518 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44037070 con 0x7f3a5c0d15b0 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.965+0000 7f3a5b400640 1 -- 172.21.15.59:0/4242554518 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a44032440 con 0x7f3a5c0d15b0 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.965+0000 7f3a5b400640 1 -- 172.21.15.59:0/4242554518 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44032810 con 0x7f3a5c0d15b0 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.965+0000 7f3a61400640 1 -- 172.21.15.59:0/4242554518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 msgr2=0x7f3a5c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.965+0000 7f3a61400640 1 --2- 172.21.15.59:0/4242554518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0cf280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009870 tx=0x7f3a4402a130 comp rx=0 tx=0).stop 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.966+0000 7f3a61400640 1 -- 172.21.15.59:0/4242554518 shutdown_connections 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.966+0000 7f3a61400640 1 --2- 172.21.15.59:0/4242554518 >> v2:172.21.15.76:3300/0 conn(0x7f3a5c0b4f20 0x7f3a5c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.966+0000 7f3a61400640 1 --2- 172.21.15.59:0/4242554518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0cf280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.966+0000 7f3a61400640 1 -- 172.21.15.59:0/4242554518 >> 172.21.15.59:0/4242554518 conn(0x7f3a5c08bf90 msgr2=0x7f3a5c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.966+0000 7f3a61400640 1 -- 172.21.15.59:0/4242554518 shutdown_connections 2024-04-23T14:44:50.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.966+0000 7f3a61400640 1 -- 172.21.15.59:0/4242554518 wait complete. 2024-04-23T14:44:50.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.967+0000 7f3a61400640 1 Processor -- start 2024-04-23T14:44:50.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.968+0000 7f3a61400640 1 -- start start 2024-04-23T14:44:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a61400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a5c0b4f20 0x7f3a5c151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a61400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a61400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5c15ed20 0x7f3a5c0b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a61400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0b3600 con 0x7f3a5c15ed20 2024-04-23T14:44:50.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a61400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d3490 con 0x7f3a5c0b4f20 2024-04-23T14:44:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a61400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c0b3780 con 0x7f3a5c0d15b0 2024-04-23T14:44:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a60a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5c15ed20 0x7f3a5c0b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a60a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5c15ed20 0x7f3a5c0b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34594/0 (socket says 172.21.15.59:34594) 2024-04-23T14:44:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a60a00640 1 -- 172.21.15.59:0/2559210714 learned_addr learned my addr 172.21.15.59:0/2559210714 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a5be00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a5c0b4f20 0x7f3a5c151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.969+0000 7f3a53e00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a60a00640 1 -- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 msgr2=0x7f3a5c0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a60a00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a60a00640 1 -- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a5c0b4f20 msgr2=0x7f3a5c151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a60a00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a5c0b4f20 0x7f3a5c151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:50.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a60a00640 1 -- 172.21.15.59:0/2559210714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c0b6330 con 0x7f3a5c15ed20 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a60a00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5c15ed20 0x7f3a5c0b1d90 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400efc0 tx=0x7f3a5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a58c00640 1 -- 172.21.15.59:0/2559210714 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54018020 con 0x7f3a5c15ed20 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a58c00640 1 -- 172.21.15.59:0/2559210714 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54014480 con 0x7f3a5c15ed20 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a58c00640 1 -- 172.21.15.59:0/2559210714 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54014870 con 0x7f3a5c15ed20 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.970+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c152860 con 0x7f3a5c15ed20 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.971+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c152e20 con 0x7f3a5c15ed20 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.972+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c04fd90 con 0x7f3a5c15ed20 2024-04-23T14:44:50.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.973+0000 7f3a58c00640 1 -- 172.21.15.59:0/2559210714 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a54005d10 con 0x7f3a5c15ed20 2024-04-23T14:44:50.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.974+0000 7f3a58c00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c920 0x7f3a3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:50.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.974+0000 7f3a58c00640 1 -- 172.21.15.59:0/2559210714 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f3a5408cbd0 con 0x7f3a5c15ed20 2024-04-23T14:44:50.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.974+0000 7f3a5be00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c920 0x7f3a3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:50.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.975+0000 7f3a5be00640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c920 0x7f3a3806ede0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000f60 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:50.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:50.979+0000 7f3a58c00640 1 -- 172.21.15.59:0/2559210714 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a54022030 con 0x7f3a5c15ed20 2024-04-23T14:44:51.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.188+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a5c0d0f40 con 0x7f3a5c15ed20 2024-04-23T14:44:51.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.189+0000 7f3a58c00640 1 -- 172.21.15.59:0/2559210714 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f3a54010040 con 0x7f3a5c15ed20 2024-04-23T14:44:51.191 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c920 msgr2=0x7f3a3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c920 0x7f3a3806ede0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000f60 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5c15ed20 msgr2=0x7f3a5c0b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5c15ed20 0x7f3a5c0b1d90 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400efc0 tx=0x7f3a5400c490 comp rx=0 tx=0).stop 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 shutdown_connections 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a3806c920 0x7f3a3806ede0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a5c0d15b0 0x7f3a5c0b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a5c15ed20 0x7f3a5c0b1d90 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 --2- 172.21.15.59:0/2559210714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a5c0b4f20 0x7f3a5c151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 >> 172.21.15.59:0/2559210714 conn(0x7f3a5c08bf90 msgr2=0x7f3a5c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:51.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.192+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 shutdown_connections 2024-04-23T14:44:51.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.193+0000 7f3a61400640 1 -- 172.21.15.59:0/2559210714 wait complete. 2024-04-23T14:44:51.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:44:51.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.429+0000 7f5587200640 1 Processor -- start 2024-04-23T14:44:51.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.429+0000 7f5587200640 1 -- start start 2024-04-23T14:44:51.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5587200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f55800d0600 0x7f55800ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5587200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f558006dcb0 0x7f5580162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5587200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55800d09e0 0x7f558006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5587200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5580072a20 con 0x7f55800d0600 2024-04-23T14:44:51.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5587200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5580163240 con 0x7f558006dcb0 2024-04-23T14:44:51.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5587200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5580071e10 con 0x7f55800d09e0 2024-04-23T14:44:51.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5585e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f55800d0600 0x7f55800ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5585e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f55800d0600 0x7f55800ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34606/0 (socket says 172.21.15.59:34606) 2024-04-23T14:44:51.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5585e00640 1 -- 172.21.15.59:0/2284610479 learned_addr learned my addr 172.21.15.59:0/2284610479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.431+0000 7f5586800640 1 --2- 172.21.15.59:0/2284610479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55800d09e0 0x7f558006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.432+0000 7f5585400640 1 --2- 172.21.15.59:0/2284610479 >> v2:172.21.15.76:3300/0 conn(0x7f558006dcb0 0x7f5580162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.432+0000 7f5585e00640 1 -- 172.21.15.59:0/2284610479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55800d09e0 msgr2=0x7f558006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.432+0000 7f5585e00640 1 --2- 172.21.15.59:0/2284610479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55800d09e0 0x7f558006e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.432+0000 7f5585e00640 1 -- 172.21.15.59:0/2284610479 >> v2:172.21.15.76:3300/0 conn(0x7f558006dcb0 msgr2=0x7f5580162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.432+0000 7f5585e00640 1 --2- 172.21.15.59:0/2284610479 >> v2:172.21.15.76:3300/0 conn(0x7f558006dcb0 0x7f5580162d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.432+0000 7f5585e00640 1 -- 172.21.15.59:0/2284610479 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55800720f0 con 0x7f55800d0600 2024-04-23T14:44:51.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.433+0000 7f5585e00640 1 --2- 172.21.15.59:0/2284610479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800ad170 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f55800d11c0 tx=0x7f557802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8fc1460f760871 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:51.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.433+0000 7f5584a00640 1 -- 172.21.15.59:0/2284610479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578036070 con 0x7f55800d0600 2024-04-23T14:44:51.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.433+0000 7f5584a00640 1 -- 172.21.15.59:0/2284610479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5578031600 con 0x7f55800d0600 2024-04-23T14:44:51.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.433+0000 7f5584a00640 1 -- 172.21.15.59:0/2284610479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780319d0 con 0x7f55800d0600 2024-04-23T14:44:51.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 -- 172.21.15.59:0/2284610479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 msgr2=0x7f55800ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 --2- 172.21.15.59:0/2284610479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800ad170 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f55800d11c0 tx=0x7f557802a6a0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 -- 172.21.15.59:0/2284610479 shutdown_connections 2024-04-23T14:44:51.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 --2- 172.21.15.59:0/2284610479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f55800d09e0 0x7f558006e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 --2- 172.21.15.59:0/2284610479 >> v2:172.21.15.76:3300/0 conn(0x7f558006dcb0 0x7f5580162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 --2- 172.21.15.59:0/2284610479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800ad170 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 -- 172.21.15.59:0/2284610479 >> 172.21.15.59:0/2284610479 conn(0x7f558006bdf0 msgr2=0x7f558006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:51.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 -- 172.21.15.59:0/2284610479 shutdown_connections 2024-04-23T14:44:51.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.434+0000 7f5587200640 1 -- 172.21.15.59:0/2284610479 wait complete. 2024-04-23T14:44:51.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.436+0000 7f5587200640 1 Processor -- start 2024-04-23T14:44:51.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.437+0000 7f5587200640 1 -- start start 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5587200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f558006dcb0 0x7f558014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5587200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5587200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55800d09e0 0x7f55800cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5587200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f558015eaf0 con 0x7f55800d0600 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5587200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55800726b0 con 0x7f55800d09e0 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5587200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f55800d0e90 con 0x7f558006dcb0 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5585400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5585400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34622/0 (socket says 172.21.15.59:34622) 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.438+0000 7f5585400640 1 -- 172.21.15.59:0/3786627110 learned_addr learned my addr 172.21.15.59:0/3786627110 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5585e00640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f558006dcb0 0x7f558014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5586800640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55800d09e0 0x7f55800cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5585400640 1 -- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f558006dcb0 msgr2=0x7f558014a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5585400640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f558006dcb0 0x7f558014a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5585400640 1 -- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55800d09e0 msgr2=0x7f55800cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5585400640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55800d09e0 0x7f55800cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5585400640 1 -- 172.21.15.59:0/3786627110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55800b0d60 con 0x7f55800d0600 2024-04-23T14:44:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5585400640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800cb820 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f557000c7f0 tx=0x7f557000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:51.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5576000640 1 -- 172.21.15.59:0/3786627110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570015070 con 0x7f55800d0600 2024-04-23T14:44:51.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.439+0000 7f5576000640 1 -- 172.21.15.59:0/3786627110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5570004220 con 0x7f55800d0600 2024-04-23T14:44:51.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.440+0000 7f5576000640 1 -- 172.21.15.59:0/3786627110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570007620 con 0x7f55800d0600 2024-04-23T14:44:51.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.440+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55800b1040 con 0x7f55800d0600 2024-04-23T14:44:51.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.440+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55800c9a40 con 0x7f55800d0600 2024-04-23T14:44:51.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.441+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55800c9c40 con 0x7f55800d0600 2024-04-23T14:44:51.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.442+0000 7f5576000640 1 -- 172.21.15.59:0/3786627110 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55700079d0 con 0x7f55800d0600 2024-04-23T14:44:51.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.443+0000 7f5576000640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f556406c8f0 0x7f556406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.443+0000 7f5576000640 1 -- 172.21.15.59:0/3786627110 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f557008bcb0 con 0x7f55800d0600 2024-04-23T14:44:51.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.443+0000 7f5585e00640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f556406c8f0 0x7f556406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.444+0000 7f5585e00640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f556406c8f0 0x7f556406edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5578033000 tx=0x7f5578009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:51.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.448+0000 7f5576000640 1 -- 172.21.15.59:0/3786627110 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55800c9c40 con 0x7f55800d0600 2024-04-23T14:44:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.652+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f558004fd90 con 0x7f55800d0600 2024-04-23T14:44:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.653+0000 7f5576000640 1 -- 172.21.15.59:0/3786627110 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7f557005ee20 con 0x7f55800d0600 2024-04-23T14:44:51.654 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f556406c8f0 msgr2=0x7f556406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f556406c8f0 0x7f556406edb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5578033000 tx=0x7f5578009000 comp rx=0 tx=0).stop 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 msgr2=0x7f55800cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800cb820 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f557000c7f0 tx=0x7f557000ccb0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 shutdown_connections 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f556406c8f0 0x7f556406edb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55800d09e0 0x7f55800cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55800d0600 0x7f55800cb820 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 --2- 172.21.15.59:0/3786627110 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f558006dcb0 0x7f558014a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.655+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 >> 172.21.15.59:0/3786627110 conn(0x7f558006bdf0 msgr2=0x7f55800ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.656+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 shutdown_connections 2024-04-23T14:44:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.656+0000 7f5587200640 1 -- 172.21.15.59:0/3786627110 wait complete. 2024-04-23T14:44:51.675 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:44:51.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.890+0000 7f1d32400640 1 Processor -- start 2024-04-23T14:44:51.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.890+0000 7f1d32400640 1 -- start start 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d32400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1d2c06fa50 0x7f1d2c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d32400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d2c0d0350 0x7f1d2c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d32400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d2c15bf60 con 0x7f1d2c0b56f0 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d32400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2c0d0d20 con 0x7f1d2c06fa50 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d32400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2c15c240 con 0x7f1d2c0d0350 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d23e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1d2c06fa50 0x7f1d2c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d23e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1d2c06fa50 0x7f1d2c06ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34630/0 (socket says 172.21.15.59:34630) 2024-04-23T14:44:51.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.891+0000 7f1d23e00640 1 -- 172.21.15.59:0/1269962064 learned_addr learned my addr 172.21.15.59:0/1269962064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:51.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.892+0000 7f1d2be00640 1 --2- 172.21.15.59:0/1269962064 >> v2:172.21.15.76:3300/0 conn(0x7f1d2c0d0350 0x7f1d2c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.892+0000 7f1d23e00640 1 -- 172.21.15.59:0/1269962064 >> v2:172.21.15.76:3300/0 conn(0x7f1d2c0d0350 msgr2=0x7f1d2c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:51.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.892+0000 7f1d23e00640 1 --2- 172.21.15.59:0/1269962064 >> v2:172.21.15.76:3300/0 conn(0x7f1d2c0d0350 0x7f1d2c0d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.892+0000 7f1d23e00640 1 -- 172.21.15.59:0/1269962064 >> v1:172.21.15.59:6789/0 conn(0x7f1d2c0b56f0 legacy=0x7f1d2c06f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.892+0000 7f1d23e00640 1 -- 172.21.15.59:0/1269962064 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c15c520 con 0x7f1d2c06fa50 2024-04-23T14:44:51.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.892+0000 7f1d2aa00640 1 -- 172.21.15.59:0/1269962064 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697349612 0 0) 0x7f1d2c15bf60 con 0x7f1d2c0b56f0 2024-04-23T14:44:51.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.893+0000 7f1d23e00640 1 --2- 172.21.15.59:0/1269962064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c06fa50 0x7f1d2c06ddd0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c06c620 tx=0x7f1d1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d3f37ab8df6e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:51.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.893+0000 7f1d2aa00640 1 -- 172.21.15.59:0/1269962064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14037070 con 0x7f1d2c06fa50 2024-04-23T14:44:51.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.893+0000 7f1d2aa00640 1 -- 172.21.15.59:0/1269962064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d14032440 con 0x7f1d2c06fa50 2024-04-23T14:44:51.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.893+0000 7f1d2aa00640 1 -- 172.21.15.59:0/1269962064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14032810 con 0x7f1d2c06fa50 2024-04-23T14:44:51.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 -- 172.21.15.59:0/1269962064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c06fa50 msgr2=0x7f1d2c06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 --2- 172.21.15.59:0/1269962064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c06fa50 0x7f1d2c06ddd0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c06c620 tx=0x7f1d1402a270 comp rx=0 tx=0).stop 2024-04-23T14:44:51.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 -- 172.21.15.59:0/1269962064 shutdown_connections 2024-04-23T14:44:51.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 --2- 172.21.15.59:0/1269962064 >> v2:172.21.15.76:3300/0 conn(0x7f1d2c0d0350 0x7f1d2c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 --2- 172.21.15.59:0/1269962064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c06fa50 0x7f1d2c06ddd0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 -- 172.21.15.59:0/1269962064 >> 172.21.15.59:0/1269962064 conn(0x7f1d2c067d90 msgr2=0x7f1d2c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:51.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 -- 172.21.15.59:0/1269962064 shutdown_connections 2024-04-23T14:44:51.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.894+0000 7f1d32400640 1 -- 172.21.15.59:0/1269962064 wait complete. 2024-04-23T14:44:51.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.896+0000 7f1d32400640 1 Processor -- start 2024-04-23T14:44:51.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.896+0000 7f1d32400640 1 -- start start 2024-04-23T14:44:51.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d32400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d2c0b56f0 0x7f1d2c1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d32400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c0d0350 0x7f1d2c147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d32400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d2c1f4970 0x7f1d2c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d32400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c15bc10 con 0x7f1d2c0d0350 2024-04-23T14:44:51.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d32400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c15cae0 con 0x7f1d2c0b56f0 2024-04-23T14:44:51.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d32400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c0b3710 con 0x7f1d2c1f4970 2024-04-23T14:44:51.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c0d0350 0x7f1d2c147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c0d0350 0x7f1d2c147f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34646/0 (socket says 172.21.15.59:34646) 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.897+0000 7f1d23e00640 1 -- 172.21.15.59:0/547505592 learned_addr learned my addr 172.21.15.59:0/547505592 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d2b400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d2c0b56f0 0x7f1d2c1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d23e00640 1 -- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d2c1f4970 msgr2=0x7f1d2c156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d23e00640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d2c1f4970 0x7f1d2c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d23e00640 1 -- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d2c0b56f0 msgr2=0x7f1d2c1521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d23e00640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d2c0b56f0 0x7f1d2c1521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d23e00640 1 -- 172.21.15.59:0/547505592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c1f5140 con 0x7f1d2c0d0350 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d2b400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d2c0b56f0 0x7f1d2c1521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:51.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.898+0000 7f1d23e00640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c0d0350 0x7f1d2c147f30 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f1d14035040 tx=0x7f1d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:51.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.899+0000 7f1d23400640 1 -- 172.21.15.59:0/547505592 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14041070 con 0x7f1d2c0d0350 2024-04-23T14:44:51.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.899+0000 7f1d23400640 1 -- 172.21.15.59:0/547505592 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d14004220 con 0x7f1d2c0d0350 2024-04-23T14:44:51.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.899+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c1f5420 con 0x7f1d2c0d0350 2024-04-23T14:44:51.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.899+0000 7f1d23400640 1 -- 172.21.15.59:0/547505592 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14037040 con 0x7f1d2c0d0350 2024-04-23T14:44:51.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.899+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c1f4e20 con 0x7f1d2c0d0350 2024-04-23T14:44:51.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.900+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c068300 con 0x7f1d2c0d0350 2024-04-23T14:44:51.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.901+0000 7f1d23400640 1 -- 172.21.15.59:0/547505592 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d14003610 con 0x7f1d2c0d0350 2024-04-23T14:44:51.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.902+0000 7f1d23400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1d0806c8f0 0x7f1d0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:51.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.902+0000 7f1d23400640 1 -- 172.21.15.59:0/547505592 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f1d140ac500 con 0x7f1d2c0d0350 2024-04-23T14:44:51.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.902+0000 7f1d2b400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1d0806c8f0 0x7f1d0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:51.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.903+0000 7f1d2b400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1d0806c8f0 0x7f1d0806edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c000f60 tx=0x7f1d1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:51.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:51.906+0000 7f1d23400640 1 -- 172.21.15.59:0/547505592 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d2c068300 con 0x7f1d2c0d0350 2024-04-23T14:44:52.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.117+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d2c068a80 con 0x7f1d2c0d0350 2024-04-23T14:44:52.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.118+0000 7f1d23400640 1 -- 172.21.15.59:0/547505592 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7f1d1407a830 con 0x7f1d2c0d0350 2024-04-23T14:44:52.119 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1d0806c8f0 msgr2=0x7f1d0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1d0806c8f0 0x7f1d0806edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c000f60 tx=0x7f1d1c009040 comp rx=0 tx=0).stop 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c0d0350 msgr2=0x7f1d2c147f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c0d0350 0x7f1d2c147f30 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f1d14035040 tx=0x7f1d140046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 shutdown_connections 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1d0806c8f0 0x7f1d0806edb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1d2c1f4970 0x7f1d2c156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d2c0d0350 0x7f1d2c147f30 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 --2- 172.21.15.59:0/547505592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d2c0b56f0 0x7f1d2c1521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 >> 172.21.15.59:0/547505592 conn(0x7f1d2c067d90 msgr2=0x7f1d2c0b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:52.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 shutdown_connections 2024-04-23T14:44:52.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.121+0000 7f1d32400640 1 -- 172.21.15.59:0/547505592 wait complete. 2024-04-23T14:44:52.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:44:52.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:44:52.221 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:44:52.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.344+0000 7f4e4c000640 1 Processor -- start 2024-04-23T14:44:52.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.344+0000 7f4e4c000640 1 -- start start 2024-04-23T14:44:52.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.346+0000 7f4e4c000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4e4415ea90 0x7f4e4415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:52.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.346+0000 7f4e4c000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e4406c0e0 0x7f4e440b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:52.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.346+0000 7f4e4c000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4415f490 con 0x7f4e440b26d0 2024-04-23T14:44:52.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.346+0000 7f4e4c000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4e441600a0 con 0x7f4e4415ea90 2024-04-23T14:44:52.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.346+0000 7f4e4c000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4415f770 con 0x7f4e4406c0e0 2024-04-23T14:44:52.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.346+0000 7f4e48c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4e440b26d0 0x7f4e4406b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44446/0 (socket says 172.21.15.59:44446) 2024-04-23T14:44:52.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.346+0000 7f4e48c00640 1 -- 172.21.15.59:0/105418801 learned_addr learned my addr 172.21.15.59:0/105418801 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:52.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e49600640 1 --2- 172.21.15.59:0/105418801 >> v2:172.21.15.76:3300/0 conn(0x7f4e4415ea90 0x7f4e4415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e4a000640 1 --2- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e4406c0e0 0x7f4e440b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e43e00640 1 -- 172.21.15.59:0/105418801 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68895059 0 0) 0x7f4e4415f490 con 0x7f4e440b26d0 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e43e00640 1 -- 172.21.15.59:0/105418801 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e2c003520 con 0x7f4e440b26d0 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e43e00640 1 -- 172.21.15.59:0/105418801 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4144580325 0 0) 0x7f4e2c003520 con 0x7f4e440b26d0 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e49600640 1 -- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e4406c0e0 msgr2=0x7f4e440b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e49600640 1 --2- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e4406c0e0 0x7f4e440b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e49600640 1 -- 172.21.15.59:0/105418801 >> v1:172.21.15.59:6789/0 conn(0x7f4e440b26d0 legacy=0x7f4e4406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.347+0000 7f4e49600640 1 -- 172.21.15.59:0/105418801 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4415fa50 con 0x7f4e4415ea90 2024-04-23T14:44:52.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.349+0000 7f4e49600640 1 --2- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4415ea90 0x7f4e4415ee70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e380097a0 tx=0x7f4e3802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3670be556e34a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:52.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.349+0000 7f4e43e00640 1 -- 172.21.15.59:0/105418801 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38037070 con 0x7f4e4415ea90 2024-04-23T14:44:52.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.349+0000 7f4e43e00640 1 -- 172.21.15.59:0/105418801 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3802aba0 con 0x7f4e4415ea90 2024-04-23T14:44:52.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.349+0000 7f4e43e00640 1 -- 172.21.15.59:0/105418801 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30001670 con 0x7f4e4415ea90 2024-04-23T14:44:52.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 -- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4415ea90 msgr2=0x7f4e4415ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 --2- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4415ea90 0x7f4e4415ee70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4e380097a0 tx=0x7f4e3802a110 comp rx=0 tx=0).stop 2024-04-23T14:44:52.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 -- 172.21.15.59:0/105418801 shutdown_connections 2024-04-23T14:44:52.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 --2- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e4406c0e0 0x7f4e440b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 --2- 172.21.15.59:0/105418801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4415ea90 0x7f4e4415ee70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 -- 172.21.15.59:0/105418801 >> 172.21.15.59:0/105418801 conn(0x7f4e4409aa00 msgr2=0x7f4e440b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:52.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 -- 172.21.15.59:0/105418801 shutdown_connections 2024-04-23T14:44:52.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.350+0000 7f4e4c000640 1 -- 172.21.15.59:0/105418801 wait complete. 2024-04-23T14:44:52.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.352+0000 7f4e4c000640 1 Processor -- start 2024-04-23T14:44:52.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.352+0000 7f4e4c000640 1 -- start start 2024-04-23T14:44:52.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4c000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4406c0e0 0x7f4e44159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:52.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4c000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e440b26d0 0x7f4e440b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:52.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e4415ea90 0x7f4e44156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:52.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4c000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4e440cbdf0 con 0x7f4e4415ea90 2024-04-23T14:44:52.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4c000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4e440cbc70 con 0x7f4e4406c0e0 2024-04-23T14:44:52.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4c000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4e440cbf70 con 0x7f4e440b26d0 2024-04-23T14:44:52.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e4415ea90 0x7f4e44156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e4415ea90 0x7f4e44156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34658/0 (socket says 172.21.15.59:34658) 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.353+0000 7f4e4a000640 1 -- 172.21.15.59:0/632586827 learned_addr learned my addr 172.21.15.59:0/632586827 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e48c00640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e440b26d0 0x7f4e440b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e49600640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4406c0e0 0x7f4e44159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e4a000640 1 -- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e440b26d0 msgr2=0x7f4e440b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e4a000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e440b26d0 0x7f4e440b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e4a000640 1 -- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4406c0e0 msgr2=0x7f4e44159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e4a000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4406c0e0 0x7f4e44159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e4a000640 1 -- 172.21.15.59:0/632586827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e440b5a70 con 0x7f4e4415ea90 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e49600640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4406c0e0 0x7f4e44159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:44:52.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.354+0000 7f4e4a000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e4415ea90 0x7f4e44156e80 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00efc0 tx=0x7f4e3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:52.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.355+0000 7f4e41600640 1 -- 172.21.15.59:0/632586827 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c018020 con 0x7f4e4415ea90 2024-04-23T14:44:52.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.355+0000 7f4e41600640 1 -- 172.21.15.59:0/632586827 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c014480 con 0x7f4e4415ea90 2024-04-23T14:44:52.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.355+0000 7f4e41600640 1 -- 172.21.15.59:0/632586827 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c014870 con 0x7f4e4415ea90 2024-04-23T14:44:52.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.355+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e440b5d50 con 0x7f4e4415ea90 2024-04-23T14:44:52.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.355+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e440b6310 con 0x7f4e4415ea90 2024-04-23T14:44:52.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.356+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e440d22f0 con 0x7f4e4415ea90 2024-04-23T14:44:52.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.357+0000 7f4e41600640 1 -- 172.21.15.59:0/632586827 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e3c005d10 con 0x7f4e4415ea90 2024-04-23T14:44:52.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.358+0000 7f4e41600640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4e2406c8f0 0x7f4e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:52.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.358+0000 7f4e41600640 1 -- 172.21.15.59:0/632586827 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f4e3c08c7b0 con 0x7f4e4415ea90 2024-04-23T14:44:52.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.358+0000 7f4e49600640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4e2406c8f0 0x7f4e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:52.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.359+0000 7f4e49600640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4e2406c8f0 0x7f4e2406edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4e38034000 tx=0x7f4e3802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:52.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.363+0000 7f4e41600640 1 -- 172.21.15.59:0/632586827 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e440d22f0 con 0x7f4e4415ea90 2024-04-23T14:44:52.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.581+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4e44068a80 con 0x7f4e4415ea90 2024-04-23T14:44:52.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.582+0000 7f4e41600640 1 -- 172.21.15.59:0/632586827 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v116) ==== 80+0+244 (secure 0 0 0) 0x7f4e3c022070 con 0x7f4e4415ea90 2024-04-23T14:44:52.583 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6837/3383882209 2024-04-24T14:44:38.869275+0000 2024-04-23T14:44:52.583 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6836/3383882209 2024-04-24T14:44:38.869275+0000 2024-04-23T14:44:52.584 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6837/1061483588 2024-04-24T14:44:14.736925+0000 2024-04-23T14:44:52.584 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6836/1061483588 2024-04-24T14:44:14.736925+0000 2024-04-23T14:44:52.584 INFO:teuthology.orchestra.run.smithi059.stderr:listed 4 entries 2024-04-23T14:44:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.584+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4e2406c8f0 msgr2=0x7f4e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.584+0000 7f4e4c000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4e2406c8f0 0x7f4e2406edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4e38034000 tx=0x7f4e3802e000 comp rx=0 tx=0).stop 2024-04-23T14:44:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.584+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e4415ea90 msgr2=0x7f4e44156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.584+0000 7f4e4c000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e4415ea90 0x7f4e44156e80 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00efc0 tx=0x7f4e3c00c490 comp rx=0 tx=0).stop 2024-04-23T14:44:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 shutdown_connections 2024-04-23T14:44:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4e2406c8f0 0x7f4e2406edb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e440b26d0 0x7f4e440b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e4415ea90 0x7f4e44156e80 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 --2- 172.21.15.59:0/632586827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e4406c0e0 0x7f4e44159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 >> 172.21.15.59:0/632586827 conn(0x7f4e4409aa00 msgr2=0x7f4e440b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 shutdown_connections 2024-04-23T14:44:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:52.585+0000 7f4e4c000640 1 -- 172.21.15.59:0/632586827 wait complete. 2024-04-23T14:44:52.605 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-04-23T14:44:52.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:52.605 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T14:44:52.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi059.stderr:ceph-fuse[79809]: fuse finished with error 0 and tester_r 0 2024-04-23T14:44:52.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:44:52.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:52.737 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:44:52.792 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:44:52.792 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:44:52.792 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:52.847 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:44:52.847 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:44:52.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:52.847 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-23T14:44:52.895 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-23T14:44:52.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:44:52.896 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:44:52.897 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:44:52.897 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:52.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:44:52.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:44:52.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:52.956 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-23T14:44:53.015 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-23T14:44:53.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:44:53.016 INFO:tasks.cephfs.mount:Terminating background process 2024-04-23T14:44:53.016 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:44:53.016 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:44:53.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:44:53.074 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:44:53.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:44:53.074 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-23T14:44:53.130 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-23T14:44:53.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:44:53.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:53.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.358+0000 7f1a38a00640 1 Processor -- start 2024-04-23T14:44:53.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.359+0000 7f1a38a00640 1 -- start start 2024-04-23T14:44:53.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.360+0000 7f1a38a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 0x7f1a340b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.360+0000 7f1a38a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1a3406df00 0x7f1a3406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.360+0000 7f1a38a00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3406ec60 con 0x7f1a3415b840 2024-04-23T14:44:53.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.360+0000 7f1a38a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1a340b46e0 con 0x7f1a3406df00 2024-04-23T14:44:53.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.360+0000 7f1a38a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a340b49c0 con 0x7f1a3406f7c0 2024-04-23T14:44:53.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a31400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1a3415b840 0x7f1a3406fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44458/0 (socket says 172.21.15.59:44458) 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a31400640 1 -- 172.21.15.59:0/70778850 learned_addr learned my addr 172.21.15.59:0/70778850 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781193415 0 0) 0x7f1a3406ec60 con 0x7f1a3415b840 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a18003520 con 0x7f1a3415b840 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a32800640 1 --2- 172.21.15.59:0/70778850 >> v2:172.21.15.76:3300/0 conn(0x7f1a3406df00 0x7f1a3406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a31e00640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 0x7f1a340b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1356777806 0 0) 0x7f1a18003520 con 0x7f1a3415b840 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 msgr2=0x7f1a340b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a30a00640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 0x7f1a340b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 >> v2:172.21.15.76:3300/0 conn(0x7f1a3406df00 msgr2=0x7f1a3406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.361+0000 7f1a30a00640 1 --2- 172.21.15.59:0/70778850 >> v2:172.21.15.76:3300/0 conn(0x7f1a3406df00 0x7f1a3406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340b4ca0 con 0x7f1a3415b840 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f1a1c001670 con 0x7f1a3415b840 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 >> v1:172.21.15.59:6789/0 conn(0x7f1a3415b840 legacy=0x7f1a3406fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a180039b0 0x7f1a18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a18004800 0x7f1a18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 0x7f1a180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340b46e0 con 0x7f1a18004800 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340b49c0 con 0x7f1a180039b0 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1800dc80 con 0x7f1a3406f7c0 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a31400640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a18004800 0x7f1a18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a31e00640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a180039b0 0x7f1a18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a32800640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 0x7f1a180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a31400640 1 -- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 msgr2=0x7f1a180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a31400640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 0x7f1a180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a31400640 1 -- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a180039b0 msgr2=0x7f1a18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.362+0000 7f1a31400640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a180039b0 0x7f1a18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.363+0000 7f1a31400640 1 -- 172.21.15.59:0/70778850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a18003520 con 0x7f1a18004800 2024-04-23T14:44:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.363+0000 7f1a32800640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a3406f7c0 0x7f1a180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.364+0000 7f1a31400640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a18004800 0x7f1a18004c80 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c001eb0 tx=0x7f1a18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51749959b62e7a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.364+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c007d70 con 0x7f1a18004800 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.364+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c0058d0 con 0x7f1a18004800 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.364+0000 7f1a30a00640 1 -- 172.21.15.59:0/70778850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c005ca0 con 0x7f1a18004800 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.365+0000 7f1a38a00640 1 -- 172.21.15.59:0/70778850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a18004800 msgr2=0x7f1a18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.365+0000 7f1a38a00640 1 --2- 172.21.15.59:0/70778850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a18004800 0x7f1a18004c80 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c001eb0 tx=0x7f1a18005830 comp rx=0 tx=0).stop 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.365+0000 7f1a32800640 1 -- 172.21.15.59:0/70778850 reap_dead start 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.365+0000 7f1a38a00640 1 -- 172.21.15.59:0/70778850 shutdown_connections 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.365+0000 7f1a38a00640 1 -- 172.21.15.59:0/70778850 >> 172.21.15.59:0/70778850 conn(0x7f1a34067ea0 msgr2=0x7f1a3406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.365+0000 7f1a38a00640 1 -- 172.21.15.59:0/70778850 shutdown_connections 2024-04-23T14:44:53.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.365+0000 7f1a38a00640 1 -- 172.21.15.59:0/70778850 wait complete. 2024-04-23T14:44:53.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.367+0000 7f1a38a00640 1 Processor -- start 2024-04-23T14:44:53.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.367+0000 7f1a38a00640 1 -- start start 2024-04-23T14:44:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a38a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a18004800 0x7f1a34146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a38a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415d580 0x7f1a3415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a38a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a34155d40 0x7f1a341561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a38a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340b54a0 con 0x7f1a34155d40 2024-04-23T14:44:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a38a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3406e8d0 con 0x7f1a3415d580 2024-04-23T14:44:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a38a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1a340b5620 con 0x7f1a18004800 2024-04-23T14:44:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a32800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a34155d40 0x7f1a341561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a32800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a34155d40 0x7f1a341561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34682/0 (socket says 172.21.15.59:34682) 2024-04-23T14:44:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.368+0000 7f1a32800640 1 -- 172.21.15.59:0/2009990460 learned_addr learned my addr 172.21.15.59:0/2009990460 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a31e00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a18004800 0x7f1a34146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a31400640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415d580 0x7f1a3415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a32800640 1 -- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a18004800 msgr2=0x7f1a34146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a32800640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a18004800 0x7f1a34146e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a32800640 1 -- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415d580 msgr2=0x7f1a3415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a32800640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415d580 0x7f1a3415da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a32800640 1 -- 172.21.15.59:0/2009990460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a34147340 con 0x7f1a34155d40 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a31400640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415d580 0x7f1a3415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a32800640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a34155d40 0x7f1a341561c0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098b0 tx=0x7f1a2c009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a2a000640 1 -- 172.21.15.59:0/2009990460 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c019070 con 0x7f1a34155d40 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a2a000640 1 -- 172.21.15.59:0/2009990460 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c007ce0 con 0x7f1a34155d40 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3415e040 con 0x7f1a34155d40 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.369+0000 7f1a2a000640 1 -- 172.21.15.59:0/2009990460 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c014640 con 0x7f1a34155d40 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.370+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3415e600 con 0x7f1a34155d40 2024-04-23T14:44:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.371+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34068a80 con 0x7f1a34155d40 2024-04-23T14:44:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.372+0000 7f1a2a000640 1 -- 172.21.15.59:0/2009990460 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a2c01c070 con 0x7f1a34155d40 2024-04-23T14:44:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.373+0000 7f1a2a000640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a1006c8f0 0x7f1a1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.374+0000 7f1a2a000640 1 -- 172.21.15.59:0/2009990460 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f1a2c08b780 con 0x7f1a34155d40 2024-04-23T14:44:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.374+0000 7f1a31e00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a1006c8f0 0x7f1a1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.375+0000 7f1a31e00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a1006c8f0 0x7f1a1006edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1a24006fd0 tx=0x7f1a24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:53.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.378+0000 7f1a2a000640 1 -- 172.21.15.59:0/2009990460 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a2c059ab0 con 0x7f1a34155d40 2024-04-23T14:44:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.620+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a3406f370 con 0x7f1a34155d40 2024-04-23T14:44:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.621+0000 7f1a2a000640 1 -- 172.21.15.59:0/2009990460 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7f1a2c0593b0 con 0x7f1a34155d40 2024-04-23T14:44:53.622 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:53.622 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:44.015119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9735":{"gid":9735,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3735308728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3735308728},{"type":"v1","addr":"172.21.15.59:6837","nonce":3735308728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9735,"qdb_cluster":[9735]},"id":8}]} 2024-04-23T14:44:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 113 2024-04-23T14:44:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.623+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a1006c8f0 msgr2=0x7f1a1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.623+0000 7f1a38a00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a1006c8f0 0x7f1a1006edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1a24006fd0 tx=0x7f1a24008040 comp rx=0 tx=0).stop 2024-04-23T14:44:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.623+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a34155d40 msgr2=0x7f1a341561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.623+0000 7f1a38a00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a34155d40 0x7f1a341561c0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098b0 tx=0x7f1a2c009d70 comp rx=0 tx=0).stop 2024-04-23T14:44:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 shutdown_connections 2024-04-23T14:44:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f1a1006c8f0 0x7f1a1006edb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a34155d40 0x7f1a341561c0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415d580 0x7f1a3415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 --2- 172.21.15.59:0/2009990460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a18004800 0x7f1a34146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 >> 172.21.15.59:0/2009990460 conn(0x7f1a34067ea0 msgr2=0x7f1a3415f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 shutdown_connections 2024-04-23T14:44:53.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.624+0000 7f1a38a00640 1 -- 172.21.15.59:0/2009990460 wait complete. 2024-04-23T14:44:53.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.861+0000 7f2331000640 1 Processor -- start 2024-04-23T14:44:53.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.861+0000 7f2331000640 1 -- start start 2024-04-23T14:44:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.862+0000 7f2331000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.862+0000 7f2331000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f232c160910 0x7f232c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.862+0000 7f2331000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f232c0cd520 con 0x7f232c0cd020 2024-04-23T14:44:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.862+0000 7f2331000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f232c0cddc0 con 0x7f232c160910 2024-04-23T14:44:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.862+0000 7f2331000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f232c0cd800 con 0x7f232c1454e0 2024-04-23T14:44:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f232a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f232c160910 0x7f232c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f232a800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f232c160910 0x7f232c160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34692/0 (socket says 172.21.15.59:34692) 2024-04-23T14:44:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f232a800640 1 -- 172.21.15.59:0/126021724 learned_addr learned my addr 172.21.15.59:0/126021724 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f2329e00640 1 --2- 172.21.15.59:0/126021724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f232a800640 1 -- 172.21.15.59:0/126021724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 msgr2=0x7f232c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:53.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f232a800640 1 --2- 172.21.15.59:0/126021724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f232a800640 1 -- 172.21.15.59:0/126021724 >> v1:172.21.15.76:6789/0 conn(0x7f232c0cd020 legacy=0x7f232c0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.863+0000 7f232a800640 1 -- 172.21.15.59:0/126021724 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c0cdae0 con 0x7f232c160910 2024-04-23T14:44:53.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.865+0000 7f232a800640 1 --2- 172.21.15.59:0/126021724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c160da0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f2324009e50 tx=0x7f232402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90def96a7a17f86e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.865+0000 7f2329400640 1 -- 172.21.15.59:0/126021724 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324036070 con 0x7f232c160910 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.865+0000 7f2329400640 1 -- 172.21.15.59:0/126021724 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2324031600 con 0x7f232c160910 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.865+0000 7f2329400640 1 -- 172.21.15.59:0/126021724 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23240319d0 con 0x7f232c160910 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.865+0000 7f2331000640 1 -- 172.21.15.59:0/126021724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 msgr2=0x7f232c160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.865+0000 7f2331000640 1 --2- 172.21.15.59:0/126021724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c160da0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f2324009e50 tx=0x7f232402a640 comp rx=0 tx=0).stop 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.866+0000 7f2331000640 1 -- 172.21.15.59:0/126021724 shutdown_connections 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.866+0000 7f2331000640 1 --2- 172.21.15.59:0/126021724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c160da0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.866+0000 7f2331000640 1 --2- 172.21.15.59:0/126021724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.866+0000 7f2331000640 1 -- 172.21.15.59:0/126021724 >> 172.21.15.59:0/126021724 conn(0x7f232c067ea0 msgr2=0x7f232c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.866+0000 7f2331000640 1 -- 172.21.15.59:0/126021724 shutdown_connections 2024-04-23T14:44:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.866+0000 7f2331000640 1 -- 172.21.15.59:0/126021724 wait complete. 2024-04-23T14:44:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.867+0000 7f2331000640 1 Processor -- start 2024-04-23T14:44:53.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.867+0000 7f2331000640 1 -- start start 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.868+0000 7f2331000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f232c0cd020 0x7f232c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.868+0000 7f2331000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c1f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.868+0000 7f2331000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.868+0000 7f2331000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f232c0b4010 con 0x7f232c160910 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.868+0000 7f2331000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1460f0 con 0x7f232c0cd020 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.868+0000 7f2331000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f232c15a550 con 0x7f232c1454e0 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34696/0 (socket says 172.21.15.59:34696) 2024-04-23T14:44:53.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 -- 172.21.15.59:0/905650750 learned_addr learned my addr 172.21.15.59:0/905650750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 -- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 msgr2=0x7f232c1f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f2323e00640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c1f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f2329e00640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f232c0cd020 0x7f232c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c1f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 -- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f232c0cd020 msgr2=0x7f232c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f232c0cd020 0x7f232c15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f232a800640 1 -- 172.21.15.59:0/905650750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1f83e0 con 0x7f232c160910 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f2323e00640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c1f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.869+0000 7f2329e00640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f232c0cd020 0x7f232c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.870+0000 7f232a800640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c0b4320 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f2324009e50 tx=0x7f2324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.870+0000 7f2322000640 1 -- 172.21.15.59:0/905650750 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324041020 con 0x7f232c160910 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.870+0000 7f2322000640 1 -- 172.21.15.59:0/905650750 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2324008710 con 0x7f232c160910 2024-04-23T14:44:53.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.870+0000 7f2322000640 1 -- 172.21.15.59:0/905650750 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324036040 con 0x7f232c160910 2024-04-23T14:44:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.870+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1f86c0 con 0x7f232c160910 2024-04-23T14:44:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.870+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1f8c80 con 0x7f232c160910 2024-04-23T14:44:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.871+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c068a80 con 0x7f232c160910 2024-04-23T14:44:53.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.872+0000 7f2322000640 1 -- 172.21.15.59:0/905650750 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2324044020 con 0x7f232c160910 2024-04-23T14:44:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.873+0000 7f2322000640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f230806c920 0x7f230806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.874+0000 7f2322000640 1 -- 172.21.15.59:0/905650750 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f23240ac950 con 0x7f232c160910 2024-04-23T14:44:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.874+0000 7f2329e00640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f230806c920 0x7f230806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:53.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.874+0000 7f2329e00640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f230806c920 0x7f230806ede0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f231c006fd0 tx=0x7f231c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:53.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:53.878+0000 7f2322000640 1 -- 172.21.15.59:0/905650750 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f232404b030 con 0x7f232c160910 2024-04-23T14:44:54.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.086+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f232c160030 con 0x7f232c160910 2024-04-23T14:44:54.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.094+0000 7f2322000640 1 -- 172.21.15.59:0/905650750 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16919 (secure 0 0 0) 0x7f232402ae20 con 0x7f232c160910 2024-04-23T14:44:54.096 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:54.096 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":116,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:44:38.869299+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6836/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6837/1061483588":"2024-04-24T14:44:14.736925+0000","172.21.15.59:6836/1061483588":"2024-04-24T14:44:14.736925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f230806c920 msgr2=0x7f230806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f230806c920 0x7f230806ede0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f231c006fd0 tx=0x7f231c008040 comp rx=0 tx=0).stop 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 msgr2=0x7f232c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c0b4320 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f2324009e50 tx=0x7f2324004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 shutdown_connections 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f230806c920 0x7f230806ede0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f232c1454e0 0x7f232c1f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f232c160910 0x7f232c0b4320 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.097+0000 7f2331000640 1 --2- 172.21.15.59:0/905650750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f232c0cd020 0x7f232c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.098+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 >> 172.21.15.59:0/905650750 conn(0x7f232c067ea0 msgr2=0x7f232c0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.098+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 shutdown_connections 2024-04-23T14:44:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.098+0000 7f2331000640 1 -- 172.21.15.59:0/905650750 wait complete. 2024-04-23T14:44:54.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:31:14.380574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:43:35.694682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:43:36.828690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:44:54.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:44:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.328+0000 7f7f0cc00640 1 Processor -- start 2024-04-23T14:44:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.328+0000 7f7f0cc00640 1 -- start start 2024-04-23T14:44:54.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.329+0000 7f7f0cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 0x7f7f08156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.329+0000 7f7f0cc00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0815cab0 con 0x7f7f0815d810 2024-04-23T14:44:54.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.329+0000 7f7f0cc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f08160e40 con 0x7f7f081567e0 2024-04-23T14:44:54.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.329+0000 7f7f0cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0815cd30 con 0x7f7f080d0d10 2024-04-23T14:44:54.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.330+0000 7f7f07e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7f0815d810 0x7f7f08160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44472/0 (socket says 172.21.15.59:44472) 2024-04-23T14:44:54.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.330+0000 7f7f07e00640 1 -- 172.21.15.59:0/1719226998 learned_addr learned my addr 172.21.15.59:0/1719226998 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.330+0000 7f7f06a00640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 0x7f7f08156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265145888 0 0) 0x7f7f00001490 con 0x7f7f0815d810 2024-04-23T14:44:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eec003520 con 0x7f7f0815d810 2024-04-23T14:44:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2978809308 0 0) 0x7f7f08160e40 con 0x7f7f081567e0 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f00001490 con 0x7f7f081567e0 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1952946785 0 0) 0x7f7eec003520 con 0x7f7f0815d810 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 msgr2=0x7f7f08156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 0x7f7f08156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 >> v1:172.21.15.76:6789/0 conn(0x7f7f081567e0 legacy=0x7f7f080d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0815d010 con 0x7f7f0815d810 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f7f00001bc0 con 0x7f7f0815d810 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 >> v1:172.21.15.59:6789/0 conn(0x7f7f0815d810 legacy=0x7f7f08160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eec003b70 0x7f7eec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 0x7f7eec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7eec004d40 0x7f7eec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0815cd30 con 0x7f7eec004d40 2024-04-23T14:44:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec00ba70 con 0x7f7eec003b70 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.331+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7eec00bc70 con 0x7f7f080d0d10 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f07e00640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7eec004d40 0x7f7eec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f07400640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eec003b70 0x7f7eec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f06a00640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 0x7f7eec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f07e00640 1 -- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 msgr2=0x7f7eec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f07e00640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f080d0d10 0x7f7eec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f07e00640 1 -- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eec003b70 msgr2=0x7f7eec003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f07e00640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eec003b70 0x7f7eec003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.332+0000 7f7f07e00640 1 -- 172.21.15.59:0/1719226998 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec003520 con 0x7f7eec004d40 2024-04-23T14:44:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.333+0000 7f7f07e00640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7eec004d40 0x7f7eec0051c0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f7f0815a6a0 tx=0x7f7eec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67ab8aa2bfc3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.333+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000a520 con 0x7f7eec004d40 2024-04-23T14:44:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.333+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0002f070 con 0x7f7eec004d40 2024-04-23T14:44:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.333+0000 7f7f06000640 1 -- 172.21.15.59:0/1719226998 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0002bce0 con 0x7f7eec004d40 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.334+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1719226998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7eec004d40 msgr2=0x7f7eec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.334+0000 7f7f0cc00640 1 --2- 172.21.15.59:0/1719226998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7eec004d40 0x7f7eec0051c0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f7f0815a6a0 tx=0x7f7eec005830 comp rx=0 tx=0).stop 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.334+0000 7f7f07e00640 1 -- 172.21.15.59:0/1719226998 reap_dead start 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.334+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1719226998 shutdown_connections 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.334+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1719226998 >> 172.21.15.59:0/1719226998 conn(0x7f7f0806bdf0 msgr2=0x7f7f0806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.334+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1719226998 shutdown_connections 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.334+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1719226998 wait complete. 2024-04-23T14:44:54.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.336+0000 7f7f0cc00640 1 Processor -- start 2024-04-23T14:44:54.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.336+0000 7f7f0cc00640 1 -- start start 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.337+0000 7f7f0cc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7eec004d40 0x7f7f081573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.337+0000 7f7f0cc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f08157900 0x7f7f081f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.337+0000 7f7f0cc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f08159d30 0x7f7f081f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.337+0000 7f7f0cc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08156d40 con 0x7f7f08159d30 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.337+0000 7f7f0cc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08156bc0 con 0x7f7f08157900 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.337+0000 7f7f0cc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7f08156ec0 con 0x7f7eec004d40 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f08159d30 0x7f7f081f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f08159d30 0x7f7f081f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34716/0 (socket says 172.21.15.59:34716) 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 -- 172.21.15.59:0/1311200404 learned_addr learned my addr 172.21.15.59:0/1311200404 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:54.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f06a00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f08157900 0x7f7f081f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07400640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7eec004d40 0x7f7f081573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 -- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7eec004d40 msgr2=0x7f7f081573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7eec004d40 0x7f7f081573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 -- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f08157900 msgr2=0x7f7f081f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f08157900 0x7f7f081f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 -- 172.21.15.59:0/1311200404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f08160680 con 0x7f7f08159d30 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.338+0000 7f7f07e00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f08159d30 0x7f7f081f7740 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f7f0002b530 tx=0x7f7f08161010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.339+0000 7f7eff400640 1 -- 172.21.15.59:0/1311200404 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00002070 con 0x7f7f08159d30 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.339+0000 7f7eff400640 1 -- 172.21.15.59:0/1311200404 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f00034470 con 0x7f7f08159d30 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.339+0000 7f7eff400640 1 -- 172.21.15.59:0/1311200404 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00034840 con 0x7f7f08159d30 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.339+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f080b27e0 con 0x7f7f08159d30 2024-04-23T14:44:54.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.339+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f080cef20 con 0x7f7f08159d30 2024-04-23T14:44:54.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.340+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0804fd90 con 0x7f7f08159d30 2024-04-23T14:44:54.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.341+0000 7f7eff400640 1 -- 172.21.15.59:0/1311200404 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f000349e0 con 0x7f7f08159d30 2024-04-23T14:44:54.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.342+0000 7f7eff400640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ee806c8f0 0x7f7ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.342+0000 7f7eff400640 1 -- 172.21.15.59:0/1311200404 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f7f000abe70 con 0x7f7f08159d30 2024-04-23T14:44:54.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.342+0000 7f7f07400640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ee806c8f0 0x7f7ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.343+0000 7f7f07400640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ee806c8f0 0x7f7ee806edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8007ae0 tx=0x7f7ef8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:54.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.346+0000 7f7eff400640 1 -- 172.21.15.59:0/1311200404 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f0815cab0 con 0x7f7f08159d30 2024-04-23T14:44:54.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.559+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f080b5150 con 0x7f7f08159d30 2024-04-23T14:44:54.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.560+0000 7f7eff400640 1 -- 172.21.15.59:0/1311200404 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f7f00079ac0 con 0x7f7f08159d30 2024-04-23T14:44:54.561 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-04-23T14:44:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ee806c8f0 msgr2=0x7f7ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ee806c8f0 0x7f7ee806edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8007ae0 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f08159d30 msgr2=0x7f7f081f7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f08159d30 0x7f7f081f7740 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f7f0002b530 tx=0x7f7f08161010 comp rx=0 tx=0).stop 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 shutdown_connections 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7ee806c8f0 0x7f7ee806edb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f08159d30 0x7f7f081f7740 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f08157900 0x7f7f081f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 --2- 172.21.15.59:0/1311200404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7eec004d40 0x7f7f081573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 >> 172.21.15.59:0/1311200404 conn(0x7f7f0806bdf0 msgr2=0x7f7f0815f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 shutdown_connections 2024-04-23T14:44:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.564+0000 7f7f0cc00640 1 -- 172.21.15.59:0/1311200404 wait complete. 2024-04-23T14:44:54.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:44:54.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.798+0000 7f0723e00640 1 Processor -- start 2024-04-23T14:44:54.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.798+0000 7f0723e00640 1 -- start start 2024-04-23T14:44:54.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0723e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f07241567e0 0x7f0724156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0723e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f072415d810 0x7f07240d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0723e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f072415cab0 con 0x7f07240d0b40 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0723e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f07240d0f90 con 0x7f072415d810 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0723e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0724160e40 con 0x7f07241567e0 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0722000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f072415d810 0x7f07240d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0722000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f072415d810 0x7f07240d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34724/0 (socket says 172.21.15.59:34724) 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.800+0000 7f0722000640 1 -- 172.21.15.59:0/2478120964 learned_addr learned my addr 172.21.15.59:0/2478120964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.801+0000 7f0722a00640 1 --2- 172.21.15.59:0/2478120964 >> v2:172.21.15.76:3300/0 conn(0x7f07241567e0 0x7f0724156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.801+0000 7f0722000640 1 -- 172.21.15.59:0/2478120964 >> v2:172.21.15.76:3300/0 conn(0x7f07241567e0 msgr2=0x7f0724156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.801+0000 7f0722000640 1 --2- 172.21.15.59:0/2478120964 >> v2:172.21.15.76:3300/0 conn(0x7f07241567e0 0x7f0724156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.801+0000 7f0722000640 1 -- 172.21.15.59:0/2478120964 >> v1:172.21.15.76:6789/0 conn(0x7f07240d0b40 legacy=0x7f0724160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.801+0000 7f0722000640 1 -- 172.21.15.59:0/2478120964 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072415cd30 con 0x7f072415d810 2024-04-23T14:44:54.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.803+0000 7f0722000640 1 --2- 172.21.15.59:0/2478120964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072415d810 0x7f07240d0600 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f07240b6360 tx=0x7f071002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ebdcc73c29595d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:54.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.803+0000 7f0721600640 1 -- 172.21.15.59:0/2478120964 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710037070 con 0x7f072415d810 2024-04-23T14:44:54.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.803+0000 7f0721600640 1 -- 172.21.15.59:0/2478120964 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0710032440 con 0x7f072415d810 2024-04-23T14:44:54.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.803+0000 7f0721600640 1 -- 172.21.15.59:0/2478120964 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710032810 con 0x7f072415d810 2024-04-23T14:44:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 -- 172.21.15.59:0/2478120964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072415d810 msgr2=0x7f07240d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 --2- 172.21.15.59:0/2478120964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072415d810 0x7f07240d0600 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f07240b6360 tx=0x7f071002a230 comp rx=0 tx=0).stop 2024-04-23T14:44:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 -- 172.21.15.59:0/2478120964 shutdown_connections 2024-04-23T14:44:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 --2- 172.21.15.59:0/2478120964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072415d810 0x7f07240d0600 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 --2- 172.21.15.59:0/2478120964 >> v2:172.21.15.76:3300/0 conn(0x7f07241567e0 0x7f0724156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 -- 172.21.15.59:0/2478120964 >> 172.21.15.59:0/2478120964 conn(0x7f072406bdf0 msgr2=0x7f072406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:54.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 -- 172.21.15.59:0/2478120964 shutdown_connections 2024-04-23T14:44:54.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.804+0000 7f0723e00640 1 -- 172.21.15.59:0/2478120964 wait complete. 2024-04-23T14:44:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.806+0000 7f0723e00640 1 Processor -- start 2024-04-23T14:44:54.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.806+0000 7f0723e00640 1 -- start start 2024-04-23T14:44:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0723e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f07240d0b40 0x7f07241f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0723e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07241567e0 0x7f072414f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0723e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f072415d810 0x7f072414f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0723e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0724160820 con 0x7f07241567e0 2024-04-23T14:44:54.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0723e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f072415d2f0 con 0x7f072415d810 2024-04-23T14:44:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0723e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f07241609a0 con 0x7f07240d0b40 2024-04-23T14:44:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0722000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07241567e0 0x7f072414f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0722000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07241567e0 0x7f072414f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34738/0 (socket says 172.21.15.59:34738) 2024-04-23T14:44:54.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0722000640 1 -- 172.21.15.59:0/4097086145 learned_addr learned my addr 172.21.15.59:0/4097086145 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:54.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.807+0000 7f0722000640 1 -- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f07240d0b40 msgr2=0x7f07241f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0722a00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f07240d0b40 0x7f07241f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0723400640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f072415d810 0x7f072414f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0722000640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f07240d0b40 0x7f07241f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0722000640 1 -- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f072415d810 msgr2=0x7f072414f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0722000640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f072415d810 0x7f072414f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0722000640 1 -- 172.21.15.59:0/4097086145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07241f48c0 con 0x7f07241567e0 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0723400640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f072415d810 0x7f072414f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0722000640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07241567e0 0x7f072414f220 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f071002a270 tx=0x7f07100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f071b400640 1 -- 172.21.15.59:0/4097086145 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710041020 con 0x7f07241567e0 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f071b400640 1 -- 172.21.15.59:0/4097086145 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07100306c0 con 0x7f07241567e0 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f071b400640 1 -- 172.21.15.59:0/4097086145 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710037040 con 0x7f07241567e0 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.808+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0724072700 con 0x7f07241567e0 2024-04-23T14:44:54.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.809+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072409ddd0 con 0x7f07241567e0 2024-04-23T14:44:54.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.810+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07240720e0 con 0x7f07241567e0 2024-04-23T14:44:54.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.811+0000 7f071b400640 1 -- 172.21.15.59:0/4097086145 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0710044020 con 0x7f07241567e0 2024-04-23T14:44:54.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.812+0000 7f071b400640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:54.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.812+0000 7f071b400640 1 -- 172.21.15.59:0/4097086145 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f07100ac480 con 0x7f07241567e0 2024-04-23T14:44:54.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.812+0000 7f0722a00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:54.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.813+0000 7f0722a00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f070406c8f0 0x7f070406edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0714006fd0 tx=0x7f0714008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:54.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:54.816+0000 7f071b400640 1 -- 172.21.15.59:0/4097086145 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f071004b030 con 0x7f07241567e0 2024-04-23T14:44:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.024+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f072404fd90 con 0x7f07241567e0 2024-04-23T14:44:55.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.025+0000 7f071b400640 1 -- 172.21.15.59:0/4097086145 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7f072415cab0 con 0x7f07241567e0 2024-04-23T14:44:55.026 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-04-23T14:44:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.027+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f070406c8f0 msgr2=0x7f070406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.027+0000 7f0723e00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f070406c8f0 0x7f070406edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0714006fd0 tx=0x7f0714008040 comp rx=0 tx=0).stop 2024-04-23T14:44:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.027+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07241567e0 msgr2=0x7f072414f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.027+0000 7f0723e00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07241567e0 0x7f072414f220 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f071002a270 tx=0x7f07100046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 shutdown_connections 2024-04-23T14:44:55.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f072415d810 0x7f072414f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07241567e0 0x7f072414f220 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 --2- 172.21.15.59:0/4097086145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f07240d0b40 0x7f07241f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 >> 172.21.15.59:0/4097086145 conn(0x7f072406bdf0 msgr2=0x7f0724099400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 shutdown_connections 2024-04-23T14:44:55.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.028+0000 7f0723e00640 1 -- 172.21.15.59:0/4097086145 wait complete. 2024-04-23T14:44:55.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:44:55.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.264+0000 7faf06800640 1 Processor -- start 2024-04-23T14:44:55.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.264+0000 7faf06800640 1 -- start start 2024-04-23T14:44:55.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.266+0000 7faf06800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf001567e0 0x7faf00156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.266+0000 7faf06800640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf0015d810 0x7faf000d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.266+0000 7faf06800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf0015cab0 con 0x7faf000d0b40 2024-04-23T14:44:55.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.266+0000 7faf06800640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faf000d0f70 con 0x7faf0015d810 2024-04-23T14:44:55.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.266+0000 7faf06800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faf00160e40 con 0x7faf001567e0 2024-04-23T14:44:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.267+0000 7faf04a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf0015d810 0x7faf000d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.267+0000 7faf05400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf001567e0 0x7faf00156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.267+0000 7faf04a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf0015d810 0x7faf000d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:55398/0 (socket says 172.21.15.59:55398) 2024-04-23T14:44:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.267+0000 7faf05400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf001567e0 0x7faf00156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:40708/0 (socket says 172.21.15.59:40708) 2024-04-23T14:44:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.267+0000 7faf04a00640 1 -- 172.21.15.59:0/4046483208 learned_addr learned my addr 172.21.15.59:0/4046483208 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:55.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.268+0000 7faf05400640 1 -- 172.21.15.59:0/4046483208 >> v2:172.21.15.76:3300/0 conn(0x7faf0015d810 msgr2=0x7faf000d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:55.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.268+0000 7faf05400640 1 --2- 172.21.15.59:0/4046483208 >> v2:172.21.15.76:3300/0 conn(0x7faf0015d810 0x7faf000d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.268+0000 7faf05400640 1 -- 172.21.15.59:0/4046483208 >> v1:172.21.15.76:6789/0 conn(0x7faf000d0b40 legacy=0x7faf00160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.268+0000 7faf05400640 1 -- 172.21.15.59:0/4046483208 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0015cd30 con 0x7faf001567e0 2024-04-23T14:44:55.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.268+0000 7faf04a00640 1 --2- 172.21.15.59:0/4046483208 >> v2:172.21.15.76:3300/0 conn(0x7faf0015d810 0x7faf000d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:44:55.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.270+0000 7faf05400640 1 --2- 172.21.15.59:0/4046483208 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf001567e0 0x7faf00156bc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faef40097a0 tx=0x7faef402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2352cc6d148b860a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:55.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.270+0000 7faeffe00640 1 -- 172.21.15.59:0/4046483208 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4037070 con 0x7faf001567e0 2024-04-23T14:44:55.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faeffe00640 1 -- 172.21.15.59:0/4046483208 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef4032440 con 0x7faf001567e0 2024-04-23T14:44:55.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faeffe00640 1 -- 172.21.15.59:0/4046483208 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4032810 con 0x7faf001567e0 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 -- 172.21.15.59:0/4046483208 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf001567e0 msgr2=0x7faf00156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 --2- 172.21.15.59:0/4046483208 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf001567e0 0x7faf00156bc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faef40097a0 tx=0x7faef402a180 comp rx=0 tx=0).stop 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 -- 172.21.15.59:0/4046483208 shutdown_connections 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 --2- 172.21.15.59:0/4046483208 >> v2:172.21.15.76:3300/0 conn(0x7faf0015d810 0x7faf000d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 --2- 172.21.15.59:0/4046483208 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf001567e0 0x7faf00156bc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 -- 172.21.15.59:0/4046483208 >> 172.21.15.59:0/4046483208 conn(0x7faf0006bdf0 msgr2=0x7faf0006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 -- 172.21.15.59:0/4046483208 shutdown_connections 2024-04-23T14:44:55.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.271+0000 7faf06800640 1 -- 172.21.15.59:0/4046483208 wait complete. 2024-04-23T14:44:55.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.273+0000 7faf06800640 1 Processor -- start 2024-04-23T14:44:55.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.273+0000 7faf06800640 1 -- start start 2024-04-23T14:44:55.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.274+0000 7faf06800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf000d0b40 0x7faf00154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.274+0000 7faf06800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf001567e0 0x7faf0009c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.274+0000 7faf06800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf0015d810 0x7faf001e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.274+0000 7faf06800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf00160820 con 0x7faf001567e0 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.274+0000 7faf06800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faf0015d2f0 con 0x7faf000d0b40 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.274+0000 7faf06800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7faf001609a0 con 0x7faf0015d810 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf001567e0 0x7faf0009c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf001567e0 0x7faf0009c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34750/0 (socket says 172.21.15.59:34750) 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 -- 172.21.15.59:0/1725614403 learned_addr learned my addr 172.21.15.59:0/1725614403 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf05400640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf000d0b40 0x7faf00154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf05e00640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf0015d810 0x7faf001e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 -- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf0015d810 msgr2=0x7faf001e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf0015d810 0x7faf001e80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 -- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf000d0b40 msgr2=0x7faf00154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf000d0b40 0x7faf00154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 -- 172.21.15.59:0/1725614403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf001550c0 con 0x7faf001567e0 2024-04-23T14:44:55.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf05400640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf000d0b40 0x7faf00154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:55.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf05e00640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf0015d810 0x7faf001e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:55.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.275+0000 7faf04a00640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf001567e0 0x7faf0009c470 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7faeec00e9d0 tx=0x7faeec00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:55.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.276+0000 7faefd600640 1 -- 172.21.15.59:0/1725614403 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec017070 con 0x7faf001567e0 2024-04-23T14:44:55.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.276+0000 7faefd600640 1 -- 172.21.15.59:0/1725614403 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec00ca60 con 0x7faf001567e0 2024-04-23T14:44:55.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.276+0000 7faefd600640 1 -- 172.21.15.59:0/1725614403 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec00ce10 con 0x7faf001567e0 2024-04-23T14:44:55.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.276+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf00156f80 con 0x7faf001567e0 2024-04-23T14:44:55.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.276+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf00157540 con 0x7faf001567e0 2024-04-23T14:44:55.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.277+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0004fd90 con 0x7faf001567e0 2024-04-23T14:44:55.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.278+0000 7faefd600640 1 -- 172.21.15.59:0/1725614403 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faeec01a070 con 0x7faf001567e0 2024-04-23T14:44:55.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.279+0000 7faefd600640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.280+0000 7faefd600640 1 -- 172.21.15.59:0/1725614403 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7faeec08bc00 con 0x7faf001567e0 2024-04-23T14:44:55.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.280+0000 7faf05400640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.281+0000 7faf05400640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faee006c8f0 0x7faee006edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faef4034000 tx=0x7faef402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:55.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.284+0000 7faefd600640 1 -- 172.21.15.59:0/1725614403 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faeec059ea0 con 0x7faf001567e0 2024-04-23T14:44:55.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.489+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faf000b5150 con 0x7faf001567e0 2024-04-23T14:44:55.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.489+0000 7faefd600640 1 -- 172.21.15.59:0/1725614403 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7faeec0597a0 con 0x7faf001567e0 2024-04-23T14:44:55.491 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-04-23T14:44:55.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faee006c8f0 msgr2=0x7faee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faee006c8f0 0x7faee006edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faef4034000 tx=0x7faef402e000 comp rx=0 tx=0).stop 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf001567e0 msgr2=0x7faf0009c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf001567e0 0x7faf0009c470 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7faeec00e9d0 tx=0x7faeec00ee90 comp rx=0 tx=0).stop 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 shutdown_connections 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf0015d810 0x7faf001e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf001567e0 0x7faf0009c470 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 --2- 172.21.15.59:0/1725614403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf000d0b40 0x7faf00154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 >> 172.21.15.59:0/1725614403 conn(0x7faf0006bdf0 msgr2=0x7faf0006e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 shutdown_connections 2024-04-23T14:44:55.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.492+0000 7faf06800640 1 -- 172.21.15.59:0/1725614403 wait complete. 2024-04-23T14:44:55.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.723+0000 7fde4b000640 1 Processor -- start 2024-04-23T14:44:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.723+0000 7fde4b000640 1 -- start start 2024-04-23T14:44:55.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.724+0000 7fde4b000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde440cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.724+0000 7fde4b000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde44160210 con 0x7fde440b4860 2024-04-23T14:44:55.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.724+0000 7fde4b000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde441604f0 con 0x7fde440cd590 2024-04-23T14:44:55.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.724+0000 7fde4b000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fde440cd9f0 con 0x7fde4415c3d0 2024-04-23T14:44:55.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.725+0000 7fde4a600640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fde440b4860 0x7fde440cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44484/0 (socket says 172.21.15.59:44484) 2024-04-23T14:44:55.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.725+0000 7fde4a600640 1 -- 172.21.15.59:0/3127289882 learned_addr learned my addr 172.21.15.59:0/3127289882 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:55.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.725+0000 7fde49c00640 1 --2- 172.21.15.59:0/3127289882 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde440cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.725+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635556459 0 0) 0x7fde44160210 con 0x7fde440b4860 2024-04-23T14:44:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.725+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde2c003520 con 0x7fde440b4860 2024-04-23T14:44:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.726+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073925335 0 0) 0x7fde441604f0 con 0x7fde440cd590 2024-04-23T14:44:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.726+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde44160210 con 0x7fde440cd590 2024-04-23T14:44:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.726+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3631788531 0 0) 0x7fde2c003520 con 0x7fde440b4860 2024-04-23T14:44:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.726+0000 7fde49c00640 1 -- 172.21.15.59:0/3127289882 >> v1:172.21.15.76:6789/0 conn(0x7fde440cd590 legacy=0x7fde440b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.726+0000 7fde49c00640 1 -- 172.21.15.59:0/3127289882 >> v1:172.21.15.59:6789/0 conn(0x7fde440b4860 legacy=0x7fde440cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.726+0000 7fde49c00640 1 -- 172.21.15.59:0/3127289882 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde441607d0 con 0x7fde4415c3d0 2024-04-23T14:44:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.728+0000 7fde49c00640 1 --2- 172.21.15.59:0/3127289882 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde440cd050 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fde380097d0 tx=0x7fde3802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bdae6c7315f3ace server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.729+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38037070 con 0x7fde4415c3d0 2024-04-23T14:44:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.729+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3802ac30 con 0x7fde4415c3d0 2024-04-23T14:44:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.729+0000 7fde43e00640 1 -- 172.21.15.59:0/3127289882 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0019f0 con 0x7fde4415c3d0 2024-04-23T14:44:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.729+0000 7fde4b000640 1 -- 172.21.15.59:0/3127289882 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 msgr2=0x7fde440cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.729+0000 7fde4b000640 1 --2- 172.21.15.59:0/3127289882 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde440cd050 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fde380097d0 tx=0x7fde3802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.730+0000 7fde4b000640 1 -- 172.21.15.59:0/3127289882 shutdown_connections 2024-04-23T14:44:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.730+0000 7fde4b000640 1 --2- 172.21.15.59:0/3127289882 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde440cd050 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.730+0000 7fde4b000640 1 -- 172.21.15.59:0/3127289882 >> 172.21.15.59:0/3127289882 conn(0x7fde440b21a0 msgr2=0x7fde440b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.730+0000 7fde4b000640 1 -- 172.21.15.59:0/3127289882 shutdown_connections 2024-04-23T14:44:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.730+0000 7fde4b000640 1 -- 172.21.15.59:0/3127289882 wait complete. 2024-04-23T14:44:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.732+0000 7fde4b000640 1 Processor -- start 2024-04-23T14:44:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.732+0000 7fde4b000640 1 -- start start 2024-04-23T14:44:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.733+0000 7fde4b000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde440b4860 0x7fde440c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.733+0000 7fde4b000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde440cd590 0x7fde4415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.733+0000 7fde4b000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde441f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.733+0000 7fde4b000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde4415c7b0 con 0x7fde440b4860 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.733+0000 7fde4b000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde44160d90 con 0x7fde440cd590 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde4b000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fde4415c930 con 0x7fde4415c3d0 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde440b4860 0x7fde440c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde440b4860 0x7fde440c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34756/0 (socket says 172.21.15.59:34756) 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 -- 172.21.15.59:0/2855775773 learned_addr learned my addr 172.21.15.59:0/2855775773 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49200640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde440cd590 0x7fde4415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde4a600640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde441f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 -- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 msgr2=0x7fde441f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde441f01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 -- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde440cd590 msgr2=0x7fde4415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde440cd590 0x7fde4415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.734+0000 7fde49c00640 1 -- 172.21.15.59:0/2855775773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde440ae300 con 0x7fde440b4860 2024-04-23T14:44:55.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.735+0000 7fde49c00640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde440b4860 0x7fde440c7d50 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fde38000c00 tx=0x7fde380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:55.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.735+0000 7fde41600640 1 -- 172.21.15.59:0/2855775773 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38037040 con 0x7fde440b4860 2024-04-23T14:44:55.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.735+0000 7fde41600640 1 -- 172.21.15.59:0/2855775773 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde38004270 con 0x7fde440b4860 2024-04-23T14:44:55.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.735+0000 7fde41600640 1 -- 172.21.15.59:0/2855775773 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38003610 con 0x7fde440b4860 2024-04-23T14:44:55.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.735+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde440ae5e0 con 0x7fde440b4860 2024-04-23T14:44:55.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.735+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde440aeba0 con 0x7fde440b4860 2024-04-23T14:44:55.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.737+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde440c82f0 con 0x7fde440b4860 2024-04-23T14:44:55.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.737+0000 7fde41600640 1 -- 172.21.15.59:0/2855775773 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde380037b0 con 0x7fde440b4860 2024-04-23T14:44:55.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.738+0000 7fde41600640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde2406c8f0 0x7fde2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:55.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.738+0000 7fde41600640 1 -- 172.21.15.59:0/2855775773 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fde380acae0 con 0x7fde440b4860 2024-04-23T14:44:55.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.739+0000 7fde49200640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde2406c8f0 0x7fde2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:55.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.739+0000 7fde49200640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde2406c8f0 0x7fde2406edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fde441f36c0 tx=0x7fde30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:55.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.744+0000 7fde41600640 1 -- 172.21.15.59:0/2855775773 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde3807ae20 con 0x7fde440b4860 2024-04-23T14:44:55.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.992+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde4404fd90 con 0x7fde440b4860 2024-04-23T14:44:55.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.993+0000 7fde41600640 1 -- 172.21.15.59:0/2855775773 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7fde3807a720 con 0x7fde440b4860 2024-04-23T14:44:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:55.994 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:44.015119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9735":{"gid":9735,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3735308728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3735308728},{"type":"v1","addr":"172.21.15.59:6837","nonce":3735308728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9735,"qdb_cluster":[9735]},"id":8}]} 2024-04-23T14:44:55.995 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 113 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.995+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde2406c8f0 msgr2=0x7fde2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.995+0000 7fde4b000640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde2406c8f0 0x7fde2406edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fde441f36c0 tx=0x7fde30009040 comp rx=0 tx=0).stop 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.995+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde440b4860 msgr2=0x7fde440c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.995+0000 7fde4b000640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde440b4860 0x7fde440c7d50 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fde38000c00 tx=0x7fde380046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 shutdown_connections 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde2406c8f0 0x7fde2406edb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde4415c3d0 0x7fde441f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde440cd590 0x7fde4415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 --2- 172.21.15.59:0/2855775773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde440b4860 0x7fde440c7d50 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 >> 172.21.15.59:0/2855775773 conn(0x7fde440b21a0 msgr2=0x7fde440b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:55.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 shutdown_connections 2024-04-23T14:44:55.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:55.996+0000 7fde4b000640 1 -- 172.21.15.59:0/2855775773 wait complete. 2024-04-23T14:44:56.017 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:56.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.224+0000 7f8029600640 1 Processor -- start 2024-04-23T14:44:56.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.225+0000 7f8029600640 1 -- start start 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.226+0000 7f8029600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.226+0000 7f8029600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f80240d2630 0x7f80240b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.226+0000 7f8029600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80240cf030 con 0x7f80240b6530 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.226+0000 7f8029600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f80240b7000 con 0x7f80240d2630 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.226+0000 7f8029600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f80240cf620 con 0x7f802415e7c0 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8023400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f80240d2630 0x7f80240b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8023400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f80240d2630 0x7f80240b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34770/0 (socket says 172.21.15.59:34770) 2024-04-23T14:44:56.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8023400640 1 -- 172.21.15.59:0/3415494234 learned_addr learned my addr 172.21.15.59:0/3415494234 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:56.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8023e00640 1 --2- 172.21.15.59:0/3415494234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8023400640 1 -- 172.21.15.59:0/3415494234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 msgr2=0x7f80240d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:56.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8023400640 1 --2- 172.21.15.59:0/3415494234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8023400640 1 -- 172.21.15.59:0/3415494234 >> v1:172.21.15.59:6789/0 conn(0x7f80240b6530 legacy=0x7f80240d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.228+0000 7f8023400640 1 -- 172.21.15.59:0/3415494234 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80240b6a40 con 0x7f80240d2630 2024-04-23T14:44:56.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.227+0000 7f8022a00640 1 -- 172.21.15.59:0/3415494234 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644426677 0 0) 0x7f801c001490 con 0x7f80240b6530 2024-04-23T14:44:56.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.229+0000 7f8023400640 1 --2- 172.21.15.59:0/3415494234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f80240b5ff0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f800c009870 tx=0x7f800c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb707dfcbf2626e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:56.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.229+0000 7f8022a00640 1 -- 172.21.15.59:0/3415494234 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c037070 con 0x7f80240d2630 2024-04-23T14:44:56.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.229+0000 7f8022a00640 1 -- 172.21.15.59:0/3415494234 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c032440 con 0x7f80240d2630 2024-04-23T14:44:56.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.229+0000 7f8022a00640 1 -- 172.21.15.59:0/3415494234 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c032810 con 0x7f80240d2630 2024-04-23T14:44:56.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 -- 172.21.15.59:0/3415494234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 msgr2=0x7f80240b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 --2- 172.21.15.59:0/3415494234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f80240b5ff0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f800c009870 tx=0x7f800c02a270 comp rx=0 tx=0).stop 2024-04-23T14:44:56.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 -- 172.21.15.59:0/3415494234 shutdown_connections 2024-04-23T14:44:56.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 --2- 172.21.15.59:0/3415494234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f80240b5ff0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 --2- 172.21.15.59:0/3415494234 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 -- 172.21.15.59:0/3415494234 >> 172.21.15.59:0/3415494234 conn(0x7f802409d920 msgr2=0x7f802408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:56.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 -- 172.21.15.59:0/3415494234 shutdown_connections 2024-04-23T14:44:56.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.230+0000 7f8029600640 1 -- 172.21.15.59:0/3415494234 wait complete. 2024-04-23T14:44:56.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 Processor -- start 2024-04-23T14:44:56.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 -- start start 2024-04-23T14:44:56.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80240b6530 0x7f802414f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f802415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f80240d1a80 con 0x7f80240d2630 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80240cf2b0 con 0x7f80240b6530 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.233+0000 7f8029600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f80240d1c00 con 0x7f802415e7c0 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f802415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f802415b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34780/0 (socket says 172.21.15.59:34780) 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 -- 172.21.15.59:0/3442880747 learned_addr learned my addr 172.21.15.59:0/3442880747 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023e00640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80240b6530 0x7f802414f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8028c00640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 -- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 msgr2=0x7f80240c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 -- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80240b6530 msgr2=0x7f802414f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80240b6530 0x7f802414f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023400640 1 -- 172.21.15.59:0/3442880747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80240b1820 con 0x7f80240d2630 2024-04-23T14:44:56.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8023e00640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80240b6530 0x7f802414f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:56.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.234+0000 7f8028c00640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:56.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.235+0000 7f8023400640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f802415b520 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f800c000c00 tx=0x7f800c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:56.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.235+0000 7f801be00640 1 -- 172.21.15.59:0/3442880747 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c041020 con 0x7f80240d2630 2024-04-23T14:44:56.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.235+0000 7f801be00640 1 -- 172.21.15.59:0/3442880747 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c0306c0 con 0x7f80240d2630 2024-04-23T14:44:56.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.235+0000 7f801be00640 1 -- 172.21.15.59:0/3442880747 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c037040 con 0x7f80240d2630 2024-04-23T14:44:56.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.235+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80240b1b00 con 0x7f80240d2630 2024-04-23T14:44:56.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.235+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80240b20c0 con 0x7f80240d2630 2024-04-23T14:44:56.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.236+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802404fd90 con 0x7f80240d2630 2024-04-23T14:44:56.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.237+0000 7f801be00640 1 -- 172.21.15.59:0/3442880747 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f800c044020 con 0x7f80240d2630 2024-04-23T14:44:56.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.238+0000 7f801be00640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f800006c940 0x7f800006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.238+0000 7f801be00640 1 -- 172.21.15.59:0/3442880747 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f800c0ac460 con 0x7f80240d2630 2024-04-23T14:44:56.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.238+0000 7f8023e00640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f800006c940 0x7f800006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.239+0000 7f8023e00640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f800006c940 0x7f800006ee00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8014006fd0 tx=0x7f8014008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:56.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.243+0000 7f801be00640 1 -- 172.21.15.59:0/3442880747 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f800c04b030 con 0x7f80240d2630 2024-04-23T14:44:56.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.454+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f802415d4f0 con 0x7f80240d2630 2024-04-23T14:44:56.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.456+0000 7f801be00640 1 -- 172.21.15.59:0/3442880747 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16919 (secure 0 0 0) 0x7f800c07a780 con 0x7f80240d2630 2024-04-23T14:44:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:56.458 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":116,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:44:38.869299+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6836/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6837/1061483588":"2024-04-24T14:44:14.736925+0000","172.21.15.59:6836/1061483588":"2024-04-24T14:44:14.736925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:56.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f800006c940 msgr2=0x7f800006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f800006c940 0x7f800006ee00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8014006fd0 tx=0x7f8014008040 comp rx=0 tx=0).stop 2024-04-23T14:44:56.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 msgr2=0x7f802415b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f802415b520 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f800c000c00 tx=0x7f800c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 shutdown_connections 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f800006c940 0x7f800006ee00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f802415e7c0 0x7f80240c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80240d2630 0x7f802415b520 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 --2- 172.21.15.59:0/3442880747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80240b6530 0x7f802414f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 >> 172.21.15.59:0/3442880747 conn(0x7f802409d920 msgr2=0x7f802415c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 shutdown_connections 2024-04-23T14:44:56.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.459+0000 7f8029600640 1 -- 172.21.15.59:0/3442880747 wait complete. 2024-04-23T14:44:56.481 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:44:56.481 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.693+0000 7fc46a600640 1 Processor -- start 2024-04-23T14:44:56.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.693+0000 7fc46a600640 1 -- start start 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.694+0000 7fc46a600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc46406f7c0 0x7fc4640b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.694+0000 7fc46a600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406df00 0x7fc46406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.694+0000 7fc46a600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46406ec60 con 0x7fc46415b840 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.694+0000 7fc46a600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc4640b46e0 con 0x7fc46406f7c0 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.694+0000 7fc46a600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc4640b49c0 con 0x7fc46406df00 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc463400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc46406f7c0 0x7fc4640b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc462a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc46415b840 0x7fc46406fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44504/0 (socket says 172.21.15.59:44504) 2024-04-23T14:44:56.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc462a00640 1 -- 172.21.15.59:0/1947356575 learned_addr learned my addr 172.21.15.59:0/1947356575 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc463e00640 1 --2- 172.21.15.59:0/1947356575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406df00 0x7fc46406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc462000640 1 -- 172.21.15.59:0/1947356575 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3720739439 0 0) 0x7fc46406ec60 con 0x7fc46415b840 2024-04-23T14:44:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc462000640 1 -- 172.21.15.59:0/1947356575 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc448003520 con 0x7fc46415b840 2024-04-23T14:44:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc463400640 1 -- 172.21.15.59:0/1947356575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406df00 msgr2=0x7fc46406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc463400640 1 --2- 172.21.15.59:0/1947356575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406df00 0x7fc46406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc463400640 1 -- 172.21.15.59:0/1947356575 >> v1:172.21.15.59:6789/0 conn(0x7fc46415b840 legacy=0x7fc46406fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.695+0000 7fc463400640 1 -- 172.21.15.59:0/1947356575 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4640b4ca0 con 0x7fc46406f7c0 2024-04-23T14:44:56.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.696+0000 7fc463400640 1 --2- 172.21.15.59:0/1947356575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406f7c0 0x7fc4640b54a0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fc454009dc0 tx=0x7fc45402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb04fdd5fd59a511 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc462000640 1 -- 172.21.15.59:0/1947356575 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45402ae60 con 0x7fc46406f7c0 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc462000640 1 -- 172.21.15.59:0/1947356575 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc454007740 con 0x7fc46406f7c0 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc462000640 1 -- 172.21.15.59:0/1947356575 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454007b10 con 0x7fc46406f7c0 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc46a600640 1 -- 172.21.15.59:0/1947356575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406f7c0 msgr2=0x7fc4640b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc46a600640 1 --2- 172.21.15.59:0/1947356575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406f7c0 0x7fc4640b54a0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fc454009dc0 tx=0x7fc45402a680 comp rx=0 tx=0).stop 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc46a600640 1 -- 172.21.15.59:0/1947356575 shutdown_connections 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc46a600640 1 --2- 172.21.15.59:0/1947356575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406df00 0x7fc46406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc46a600640 1 --2- 172.21.15.59:0/1947356575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406f7c0 0x7fc4640b54a0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.697+0000 7fc46a600640 1 -- 172.21.15.59:0/1947356575 >> 172.21.15.59:0/1947356575 conn(0x7fc464067ea0 msgr2=0x7fc46406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.698+0000 7fc46a600640 1 -- 172.21.15.59:0/1947356575 shutdown_connections 2024-04-23T14:44:56.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.698+0000 7fc46a600640 1 -- 172.21.15.59:0/1947356575 wait complete. 2024-04-23T14:44:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.699+0000 7fc46a600640 1 Processor -- start 2024-04-23T14:44:56.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.699+0000 7fc46a600640 1 -- start start 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.700+0000 7fc46a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406df00 0x7fc46414b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.700+0000 7fc46a600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406f7c0 0x7fc464071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.700+0000 7fc46a600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc464147160 0x7fc464072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.700+0000 7fc46a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4640ce1c0 con 0x7fc46406df00 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.700+0000 7fc46a600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc46406e8d0 con 0x7fc464147160 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.700+0000 7fc46a600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc4640ce340 con 0x7fc46406f7c0 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406df00 0x7fc46414b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406df00 0x7fc46414b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34800/0 (socket says 172.21.15.59:34800) 2024-04-23T14:44:56.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 -- 172.21.15.59:0/31515235 learned_addr learned my addr 172.21.15.59:0/31515235 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 -- 172.21.15.59:0/31515235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406f7c0 msgr2=0x7fc464071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406f7c0 0x7fc464071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 -- 172.21.15.59:0/31515235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc464147160 msgr2=0x7fc464072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc464147160 0x7fc464072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.701+0000 7fc463400640 1 -- 172.21.15.59:0/31515235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46414bc30 con 0x7fc46406df00 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.702+0000 7fc463400640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406df00 0x7fc46414b330 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc454000f80 tx=0x7fc454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.702+0000 7fc45b400640 1 -- 172.21.15.59:0/31515235 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454040070 con 0x7fc46406df00 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.702+0000 7fc45b400640 1 -- 172.21.15.59:0/31515235 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc454007e40 con 0x7fc46406df00 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.702+0000 7fc45b400640 1 -- 172.21.15.59:0/31515235 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454009450 con 0x7fc46406df00 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.702+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46414bf10 con 0x7fc46406df00 2024-04-23T14:44:56.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.702+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46414c4d0 con 0x7fc46406df00 2024-04-23T14:44:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.703+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc464068a80 con 0x7fc46406df00 2024-04-23T14:44:56.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.704+0000 7fc45b400640 1 -- 172.21.15.59:0/31515235 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc454039040 con 0x7fc46406df00 2024-04-23T14:44:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.706+0000 7fc45b400640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc44406c8f0 0x7fc44406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.706+0000 7fc45b400640 1 -- 172.21.15.59:0/31515235 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fc4540ad420 con 0x7fc46406df00 2024-04-23T14:44:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.706+0000 7fc462a00640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc44406c8f0 0x7fc44406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:56.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.707+0000 7fc462a00640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc44406c8f0 0x7fc44406edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc4500011e0 tx=0x7fc450007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.712+0000 7fc45b400640 1 -- 172.21.15.59:0/31515235 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc45407b6c0 con 0x7fc46406df00 2024-04-23T14:44:56.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.958+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc46414b110 con 0x7fc46406df00 2024-04-23T14:44:56.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.959+0000 7fc45b400640 1 -- 172.21.15.59:0/31515235 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7fc45407afc0 con 0x7fc46406df00 2024-04-23T14:44:56.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:56.961 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:44.015119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9735":{"gid":9735,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3735308728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3735308728},{"type":"v1","addr":"172.21.15.59:6837","nonce":3735308728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9735,"qdb_cluster":[9735]},"id":8}]} 2024-04-23T14:44:56.961 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 113 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc44406c8f0 msgr2=0x7fc44406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc44406c8f0 0x7fc44406edb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc4500011e0 tx=0x7fc450007910 comp rx=0 tx=0).stop 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406df00 msgr2=0x7fc46414b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406df00 0x7fc46414b330 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc454000f80 tx=0x7fc454004a40 comp rx=0 tx=0).stop 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 shutdown_connections 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc44406c8f0 0x7fc44406edb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc46406f7c0 0x7fc464071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc464147160 0x7fc464072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 --2- 172.21.15.59:0/31515235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46406df00 0x7fc46414b330 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 >> 172.21.15.59:0/31515235 conn(0x7fc464067ea0 msgr2=0x7fc46406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.962+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 shutdown_connections 2024-04-23T14:44:56.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:56.963+0000 7fc46a600640 1 -- 172.21.15.59:0/31515235 wait complete. 2024-04-23T14:44:56.983 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:44:57.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.186+0000 7f0631400640 1 Processor -- start 2024-04-23T14:44:57.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.186+0000 7f0631400640 1 -- start start 2024-04-23T14:44:57.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f0631400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f062c15e7c0 0x7f062c0d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f0631400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f062c0d2630 0x7f062c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f0631400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062c0cf030 con 0x7f062c0b6530 2024-04-23T14:44:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f0631400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f062c0b7000 con 0x7f062c0d2630 2024-04-23T14:44:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f0631400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f062c0cf620 con 0x7f062c15e7c0 2024-04-23T14:44:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f062c0d2630 0x7f062c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062b400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f062c0d2630 0x7f062c0b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34814/0 (socket says 172.21.15.59:34814) 2024-04-23T14:44:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062b400640 1 -- 172.21.15.59:0/3959224278 learned_addr learned my addr 172.21.15.59:0/3959224278 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:57.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062be00640 1 --2- 172.21.15.59:0/3959224278 >> v2:172.21.15.76:3300/0 conn(0x7f062c15e7c0 0x7f062c0d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062b400640 1 -- 172.21.15.59:0/3959224278 >> v2:172.21.15.76:3300/0 conn(0x7f062c15e7c0 msgr2=0x7f062c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:57.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062b400640 1 --2- 172.21.15.59:0/3959224278 >> v2:172.21.15.76:3300/0 conn(0x7f062c15e7c0 0x7f062c0d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062b400640 1 -- 172.21.15.59:0/3959224278 >> v1:172.21.15.76:6789/0 conn(0x7f062c0b6530 legacy=0x7f062c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:57.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.188+0000 7f062b400640 1 -- 172.21.15.59:0/3959224278 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c0b6a40 con 0x7f062c0d2630 2024-04-23T14:44:57.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.189+0000 7f062b400640 1 --2- 172.21.15.59:0/3959224278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0b5ff0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f062c08c6d0 tx=0x7f061802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c54e852036e7299 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.189+0000 7f062aa00640 1 -- 172.21.15.59:0/3959224278 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618037070 con 0x7f062c0d2630 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.189+0000 7f062aa00640 1 -- 172.21.15.59:0/3959224278 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0618032440 con 0x7f062c0d2630 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f062aa00640 1 -- 172.21.15.59:0/3959224278 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618032810 con 0x7f062c0d2630 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 -- 172.21.15.59:0/3959224278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 msgr2=0x7f062c0b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 --2- 172.21.15.59:0/3959224278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0b5ff0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f062c08c6d0 tx=0x7f061802a230 comp rx=0 tx=0).stop 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 -- 172.21.15.59:0/3959224278 shutdown_connections 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 --2- 172.21.15.59:0/3959224278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0b5ff0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 --2- 172.21.15.59:0/3959224278 >> v2:172.21.15.76:3300/0 conn(0x7f062c15e7c0 0x7f062c0d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 -- 172.21.15.59:0/3959224278 >> 172.21.15.59:0/3959224278 conn(0x7f062c09d920 msgr2=0x7f062c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 -- 172.21.15.59:0/3959224278 shutdown_connections 2024-04-23T14:44:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.190+0000 7f0631400640 1 -- 172.21.15.59:0/3959224278 wait complete. 2024-04-23T14:44:57.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.192+0000 7f0631400640 1 Processor -- start 2024-04-23T14:44:57.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.192+0000 7f0631400640 1 -- start start 2024-04-23T14:44:57.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f0631400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f062c0b6530 0x7f062c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f0631400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f0631400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f062c15e7c0 0x7f062c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f0631400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f062c0d1a80 con 0x7f062c0d2630 2024-04-23T14:44:57.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f0631400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f062c0cf2b0 con 0x7f062c15e7c0 2024-04-23T14:44:57.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f0631400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f062c0d1c00 con 0x7f062c0b6530 2024-04-23T14:44:57.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f062b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f062b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34816/0 (socket says 172.21.15.59:34816) 2024-04-23T14:44:57.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.193+0000 7f062b400640 1 -- 172.21.15.59:0/377410014 learned_addr learned my addr 172.21.15.59:0/377410014 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:57.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062be00640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f062c0b6530 0x7f062c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062b400640 1 -- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f062c0b6530 msgr2=0x7f062c14fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f0630a00640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f062c15e7c0 0x7f062c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062b400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f062c0b6530 0x7f062c14fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062b400640 1 -- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f062c15e7c0 msgr2=0x7f062c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062b400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f062c15e7c0 0x7f062c0cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062b400640 1 -- 172.21.15.59:0/377410014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c0b3200 con 0x7f062c0d2630 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062be00640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f062c0b6530 0x7f062c14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.194+0000 7f062b400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0c8050 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f06180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.195+0000 7f0623e00640 1 -- 172.21.15.59:0/377410014 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618041020 con 0x7f062c0d2630 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.195+0000 7f0623e00640 1 -- 172.21.15.59:0/377410014 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0618030970 con 0x7f062c0d2630 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.195+0000 7f0623e00640 1 -- 172.21.15.59:0/377410014 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618037040 con 0x7f062c0d2630 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.195+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062c15a6a0 con 0x7f062c0d2630 2024-04-23T14:44:57.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.195+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062c15ac60 con 0x7f062c0d2630 2024-04-23T14:44:57.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.196+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062c04fd90 con 0x7f062c0d2630 2024-04-23T14:44:57.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.197+0000 7f0623e00640 1 -- 172.21.15.59:0/377410014 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0618044020 con 0x7f062c0d2630 2024-04-23T14:44:57.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.199+0000 7f0623e00640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f060c06c920 0x7f060c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.199+0000 7f0623e00640 1 -- 172.21.15.59:0/377410014 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f06180ac7b0 con 0x7f062c0d2630 2024-04-23T14:44:57.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.199+0000 7f062be00640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f060c06c920 0x7f060c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.199+0000 7f062be00640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f060c06c920 0x7f060c06ede0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b820 tx=0x7f061c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:57.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.203+0000 7f0623e00640 1 -- 172.21.15.59:0/377410014 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f061804b030 con 0x7f062c0d2630 2024-04-23T14:44:57.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.456+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f062c0d3350 con 0x7f062c0d2630 2024-04-23T14:44:57.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.457+0000 7f0623e00640 1 -- 172.21.15.59:0/377410014 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7f062c0cf030 con 0x7f062c0d2630 2024-04-23T14:44:57.458 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:57.459 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-23T14:43:37.939514+0000","modified":"2024-04-23T14:44:44.015119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9735":{"gid":9735,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6837/3735308728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":3735308728},{"type":"v1","addr":"172.21.15.59:6837","nonce":3735308728}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9735,"qdb_cluster":[9735]},"id":8}]} 2024-04-23T14:44:57.459 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 113 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f060c06c920 msgr2=0x7f060c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f060c06c920 0x7f060c06ede0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b820 tx=0x7f061c008040 comp rx=0 tx=0).stop 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 msgr2=0x7f062c0c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0c8050 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f06180046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 shutdown_connections 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f060c06c920 0x7f060c06ede0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f062c15e7c0 0x7f062c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c0d2630 0x7f062c0c8050 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 --2- 172.21.15.59:0/377410014 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f062c0b6530 0x7f062c14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 >> 172.21.15.59:0/377410014 conn(0x7f062c09d920 msgr2=0x7f062c15c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:57.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 shutdown_connections 2024-04-23T14:44:57.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.460+0000 7f0631400640 1 -- 172.21.15.59:0/377410014 wait complete. 2024-04-23T14:44:57.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:44:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.690+0000 7f6b83600640 1 Processor -- start 2024-04-23T14:44:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.690+0000 7f6b83600640 1 -- start start 2024-04-23T14:44:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b83600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6b7c0d08b0 0x7f6b7c169860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b83600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7c15f0f0 con 0x7f6b7c0afaf0 2024-04-23T14:44:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b83600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7c15f3d0 con 0x7f6b7c15c5e0 2024-04-23T14:44:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b83600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7c15fa20 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b7be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6b7c0afaf0 0x7f6b7c0d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44518/0 (socket says 172.21.15.59:44518) 2024-04-23T14:44:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b7be00640 1 -- 172.21.15.59:0/2246047101 learned_addr learned my addr 172.21.15.59:0/2246047101 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b81600640 1 --2- 172.21.15.59:0/2246047101 >> v2:172.21.15.59:3300/0 conn(0x7f6b7c0d08b0 0x7f6b7c169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b81600640 1 -- 172.21.15.59:0/2246047101 >> v1:172.21.15.76:6789/0 conn(0x7f6b7c15c5e0 legacy=0x7f6b7c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b81600640 1 -- 172.21.15.59:0/2246047101 >> v1:172.21.15.59:6789/0 conn(0x7f6b7c0afaf0 legacy=0x7f6b7c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.692+0000 7f6b81600640 1 -- 172.21.15.59:0/2246047101 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c15f6b0 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.694+0000 7f6b81600640 1 --2- 172.21.15.59:0/2246047101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c169860 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009d90 tx=0x7f6b7402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c0f31c902f9dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.694+0000 7f6b7b400640 1 -- 172.21.15.59:0/2246047101 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74007740 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.694+0000 7f6b7b400640 1 -- 172.21.15.59:0/2246047101 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b740078e0 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.694+0000 7f6b7b400640 1 -- 172.21.15.59:0/2246047101 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74007cb0 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.694+0000 7f6b83600640 1 -- 172.21.15.59:0/2246047101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 msgr2=0x7f6b7c169860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.694+0000 7f6b83600640 1 --2- 172.21.15.59:0/2246047101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c169860 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009d90 tx=0x7f6b7402a6a0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.695+0000 7f6b83600640 1 -- 172.21.15.59:0/2246047101 shutdown_connections 2024-04-23T14:44:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.695+0000 7f6b83600640 1 --2- 172.21.15.59:0/2246047101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c169860 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.695+0000 7f6b83600640 1 -- 172.21.15.59:0/2246047101 >> 172.21.15.59:0/2246047101 conn(0x7f6b7c067ea0 msgr2=0x7f6b7c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.695+0000 7f6b83600640 1 -- 172.21.15.59:0/2246047101 shutdown_connections 2024-04-23T14:44:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.695+0000 7f6b83600640 1 -- 172.21.15.59:0/2246047101 wait complete. 2024-04-23T14:44:57.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.696+0000 7f6b83600640 1 Processor -- start 2024-04-23T14:44:57.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.697+0000 7f6b83600640 1 -- start start 2024-04-23T14:44:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.697+0000 7f6b83600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6b7c0afaf0 0x7f6b7c14be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b83600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b83600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c15c5e0 0x7f6b7c14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b83600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c169f20 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b83600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c169da0 con 0x7f6b7c15c5e0 2024-04-23T14:44:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b83600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c16a0a0 con 0x7f6b7c0afaf0 2024-04-23T14:44:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c0b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34836/0 (socket says 172.21.15.59:34836) 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 -- 172.21.15.59:0/3219249170 learned_addr learned my addr 172.21.15.59:0/3219249170 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b80c00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6b7c0afaf0 0x7f6b7c14be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b81600640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c15c5e0 0x7f6b7c14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 -- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6b7c0afaf0 msgr2=0x7f6b7c14be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6b7c0afaf0 0x7f6b7c14be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 -- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c15c5e0 msgr2=0x7f6b7c14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c15c5e0 0x7f6b7c14c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b7be00640 1 -- 172.21.15.59:0/3219249170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c0b1930 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.698+0000 7f6b80c00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6b7c0afaf0 0x7f6b7c14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.699+0000 7f6b7be00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c0b08f0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00c400 tx=0x7f6b6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.699+0000 7f6b78c00640 1 -- 172.21.15.59:0/3219249170 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c017070 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.699+0000 7f6b78c00640 1 -- 172.21.15.59:0/3219249170 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6c005ce0 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.699+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c14dab0 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.699+0000 7f6b78c00640 1 -- 172.21.15.59:0/3219249170 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0072f0 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.699+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c0b1370 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.700+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c068a80 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.701+0000 7f6b78c00640 1 -- 172.21.15.59:0/3219249170 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b6c01a070 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.703+0000 7f6b78c00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6b6006c8f0 0x7f6b6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.703+0000 7f6b78c00640 1 -- 172.21.15.59:0/3219249170 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f6b6c08aff0 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.703+0000 7f6b80c00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6b6006c8f0 0x7f6b6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.703+0000 7f6b80c00640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6b6006c8f0 0x7f6b6006edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000a830 tx=0x7f6b70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.708+0000 7f6b78c00640 1 -- 172.21.15.59:0/3219249170 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b6c059360 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.911+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b7c06e260 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.914+0000 7f6b78c00640 1 -- 172.21.15.59:0/3219249170 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16919 (secure 0 0 0) 0x7f6b7c06e260 con 0x7f6b7c0d08b0 2024-04-23T14:44:57.915 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:44:57.915 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":116,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:44:38.869299+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:43:35.694682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-23T14:43:36.828690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6836/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6837/1061483588":"2024-04-24T14:44:14.736925+0000","172.21.15.59:6836/1061483588":"2024-04-24T14:44:14.736925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:44:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.915+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6b6006c8f0 msgr2=0x7f6b6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.915+0000 7f6b83600640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6b6006c8f0 0x7f6b6006edb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000a830 tx=0x7f6b70008040 comp rx=0 tx=0).stop 2024-04-23T14:44:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 msgr2=0x7f6b7c0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c0b08f0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00c400 tx=0x7f6b6c00c8c0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 shutdown_connections 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6b6006c8f0 0x7f6b6006edb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c15c5e0 0x7f6b7c14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0d08b0 0x7f6b7c0b08f0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 --2- 172.21.15.59:0/3219249170 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6b7c0afaf0 0x7f6b7c14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 >> 172.21.15.59:0/3219249170 conn(0x7f6b7c067ea0 msgr2=0x7f6b7c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 shutdown_connections 2024-04-23T14:44:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:57.917+0000 7f6b83600640 1 -- 172.21.15.59:0/3219249170 wait complete. 2024-04-23T14:44:57.935 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:44:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.130+0000 7f6d40e00640 1 Processor -- start 2024-04-23T14:44:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.130+0000 7f6d40e00640 1 -- start start 2024-04-23T14:44:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.131+0000 7f6d40e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d3c0cf410 0x7f6d3c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.131+0000 7f6d40e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c06ddf0 0x7f6d3c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d40e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3c15dda0 con 0x7f6d3c072680 2024-04-23T14:44:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d40e00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3c0cd880 con 0x7f6d3c0cf410 2024-04-23T14:44:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d40e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c071ec0 con 0x7f6d3c06ddf0 2024-04-23T14:44:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d39e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d3c0cf410 0x7f6d3c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d39400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c06ddf0 0x7f6d3c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d39400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c06ddf0 0x7f6d3c06e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:40820/0 (socket says 172.21.15.59:40820) 2024-04-23T14:44:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d39400640 1 -- 172.21.15.59:0/2272906832 learned_addr learned my addr 172.21.15.59:0/2272906832 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.132+0000 7f6d39e00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d3c0cf410 0x7f6d3c15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:55470/0 (socket says 172.21.15.59:55470) 2024-04-23T14:44:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.133+0000 7f6d39400640 1 -- 172.21.15.59:0/2272906832 >> v2:172.21.15.76:3300/0 conn(0x7f6d3c0cf410 msgr2=0x7f6d3c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.133+0000 7f6d39400640 1 --2- 172.21.15.59:0/2272906832 >> v2:172.21.15.76:3300/0 conn(0x7f6d3c0cf410 0x7f6d3c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.133+0000 7f6d39400640 1 -- 172.21.15.59:0/2272906832 >> v1:172.21.15.76:6789/0 conn(0x7f6d3c072680 legacy=0x7f6d3c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.133+0000 7f6d39400640 1 -- 172.21.15.59:0/2272906832 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c0cdb20 con 0x7f6d3c06ddf0 2024-04-23T14:44:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.134+0000 7f6d39400640 1 --2- 172.21.15.59:0/2272906832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c06ddf0 0x7f6d3c06e250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d280098a0 tx=0x7f6d2802a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc380b31d5f144d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.135+0000 7f6d38a00640 1 -- 172.21.15.59:0/2272906832 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28037070 con 0x7f6d3c06ddf0 2024-04-23T14:44:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.135+0000 7f6d38a00640 1 -- 172.21.15.59:0/2272906832 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d28032440 con 0x7f6d3c06ddf0 2024-04-23T14:44:58.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.135+0000 7f6d38a00640 1 -- 172.21.15.59:0/2272906832 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28032810 con 0x7f6d3c06ddf0 2024-04-23T14:44:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 -- 172.21.15.59:0/2272906832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c06ddf0 msgr2=0x7f6d3c06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 --2- 172.21.15.59:0/2272906832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c06ddf0 0x7f6d3c06e250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d280098a0 tx=0x7f6d2802a1f0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 -- 172.21.15.59:0/2272906832 shutdown_connections 2024-04-23T14:44:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 --2- 172.21.15.59:0/2272906832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c06ddf0 0x7f6d3c06e250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 --2- 172.21.15.59:0/2272906832 >> v2:172.21.15.76:3300/0 conn(0x7f6d3c0cf410 0x7f6d3c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 -- 172.21.15.59:0/2272906832 >> 172.21.15.59:0/2272906832 conn(0x7f6d3c0b2130 msgr2=0x7f6d3c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 -- 172.21.15.59:0/2272906832 shutdown_connections 2024-04-23T14:44:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.136+0000 7f6d40e00640 1 -- 172.21.15.59:0/2272906832 wait complete. 2024-04-23T14:44:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.138+0000 7f6d40e00640 1 Processor -- start 2024-04-23T14:44:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.138+0000 7f6d40e00640 1 -- start start 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d40e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d3c06ddf0 0x7f6d3c1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d40e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3c072680 0x7f6d3c1e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d40e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c0cf410 0x7f6d3c1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d40e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c15a6a0 con 0x7f6d3c072680 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d40e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c0cde00 con 0x7f6d3c06ddf0 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d40e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c15a820 con 0x7f6d3c0cf410 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3c072680 0x7f6d3c1e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3c072680 0x7f6d3c1e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34844/0 (socket says 172.21.15.59:34844) 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 -- 172.21.15.59:0/1534056979 learned_addr learned my addr 172.21.15.59:0/1534056979 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:58.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 -- 172.21.15.59:0/1534056979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c0cf410 msgr2=0x7f6d3c1538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c0cf410 0x7f6d3c1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 -- 172.21.15.59:0/1534056979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d3c06ddf0 msgr2=0x7f6d3c1564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d3c06ddf0 0x7f6d3c1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.139+0000 7f6d39400640 1 -- 172.21.15.59:0/1534056979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c1e56c0 con 0x7f6d3c072680 2024-04-23T14:44:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.140+0000 7f6d39400640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3c072680 0x7f6d3c1e4f70 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097c0 tx=0x7f6d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.140+0000 7f6d32000640 1 -- 172.21.15.59:0/1534056979 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28037040 con 0x7f6d3c072680 2024-04-23T14:44:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.140+0000 7f6d32000640 1 -- 172.21.15.59:0/1534056979 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d28030500 con 0x7f6d3c072680 2024-04-23T14:44:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.140+0000 7f6d32000640 1 -- 172.21.15.59:0/1534056979 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d280308f0 con 0x7f6d3c072680 2024-04-23T14:44:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.140+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c1e59a0 con 0x7f6d3c072680 2024-04-23T14:44:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.140+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1e5f60 con 0x7f6d3c072680 2024-04-23T14:44:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.141+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c068a80 con 0x7f6d3c072680 2024-04-23T14:44:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.142+0000 7f6d32000640 1 -- 172.21.15.59:0/1534056979 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d28044030 con 0x7f6d3c072680 2024-04-23T14:44:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.144+0000 7f6d32000640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d1c06c8f0 0x7f6d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.144+0000 7f6d32000640 1 -- 172.21.15.59:0/1534056979 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f6d280abbc0 con 0x7f6d3c072680 2024-04-23T14:44:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.144+0000 7f6d39e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d1c06c8f0 0x7f6d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.145+0000 7f6d39e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d1c06c8f0 0x7f6d1c06edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006a90 tx=0x7f6d2c006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:58.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.148+0000 7f6d32000640 1 -- 172.21.15.59:0/1534056979 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d2804b060 con 0x7f6d3c072680 2024-04-23T14:44:58.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.390+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6d3c0700b0 con 0x7f6d3c072680 2024-04-23T14:44:58.391 INFO:tasks.ceph.mon.a.smithi059.stderr:2024-04-23T14:44:58.391+0000 7f321f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:44:58.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.394+0000 7f6d32000640 1 -- 172.21.15.59:0/1534056979 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) ==== 161+0+0 (secure 0 0 0) 0x7f6d28079ef0 con 0x7f6d3c072680 2024-04-23T14:44:58.395 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:44:58.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d1c06c8f0 msgr2=0x7f6d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d1c06c8f0 0x7f6d1c06edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006a90 tx=0x7f6d2c006a20 comp rx=0 tx=0).stop 2024-04-23T14:44:58.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3c072680 msgr2=0x7f6d3c1e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3c072680 0x7f6d3c1e4f70 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097c0 tx=0x7f6d280046c0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 shutdown_connections 2024-04-23T14:44:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f6d1c06c8f0 0x7f6d1c06edb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d3c0cf410 0x7f6d3c1538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d3c072680 0x7f6d3c1e4f70 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 --2- 172.21.15.59:0/1534056979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d3c06ddf0 0x7f6d3c1564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.397+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 >> 172.21.15.59:0/1534056979 conn(0x7f6d3c0b2130 msgr2=0x7f6d3c0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:58.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.399+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 shutdown_connections 2024-04-23T14:44:58.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.399+0000 7f6d40e00640 1 -- 172.21.15.59:0/1534056979 wait complete. 2024-04-23T14:44:58.407 INFO:tasks.ceph.mds.c.smithi059.stderr: -1539> 2024-04-23T14:44:44.885+0000 7f9e5ca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:44:58.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:44:58.464 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2024-04-23T14:44:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.602+0000 7f4b0c800640 1 Processor -- start 2024-04-23T14:44:58.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.603+0000 7f4b0c800640 1 -- start start 2024-04-23T14:44:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.604+0000 7f4b0c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b040b4670 0x7f4b04071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.604+0000 7f4b0c800640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b040b35d0 con 0x7f4b04162c10 2024-04-23T14:44:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.604+0000 7f4b0c800640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b040b38b0 con 0x7f4b040cfba0 2024-04-23T14:44:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.604+0000 7f4b0c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4b040b3b90 con 0x7f4b040b4670 2024-04-23T14:44:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.604+0000 7f4b09e00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4b04162c10 0x7f4b040d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:44532/0 (socket says 172.21.15.59:44532) 2024-04-23T14:44:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.604+0000 7f4b09e00640 1 -- 172.21.15.59:0/1052809790 learned_addr learned my addr 172.21.15.59:0/1052809790 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b09400640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b040b4670 0x7f4b04071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542282637 0 0) 0x7f4b040b35d0 con 0x7f4b04162c10 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aec003520 con 0x7f4b04162c10 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 985352836 0 0) 0x7f4aec003520 con 0x7f4b04162c10 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b040b4670 msgr2=0x7f4b04071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b040b4670 0x7f4b04071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 >> v1:172.21.15.76:6789/0 conn(0x7f4b040cfba0 legacy=0x7f4b0406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b040b3e70 con 0x7f4b04162c10 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f4af8001670 con 0x7f4b04162c10 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 >> v1:172.21.15.59:6789/0 conn(0x7f4b04162c10 legacy=0x7f4b040d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.605+0000 7f4b08a00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aec0039e0 0x7f4aec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b08a00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec004800 0x7f4aec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b08a00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b040b4670 0x7f4aec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b040b3b90 con 0x7f4aec0039e0 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec00ba70 con 0x7f4aec004800 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec00bc70 con 0x7f4b040b4670 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b09e00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aec0039e0 0x7f4aec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b09400640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec004800 0x7f4aec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b09e00640 1 -- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b040b4670 msgr2=0x7f4aec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b09e00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b040b4670 0x7f4aec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b09e00640 1 -- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec004800 msgr2=0x7f4aec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b09e00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec004800 0x7f4aec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.606+0000 7f4b09e00640 1 -- 172.21.15.59:0/1052809790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec003520 con 0x7f4aec0039e0 2024-04-23T14:44:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.607+0000 7f4b09e00640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aec0039e0 0x7f4aec003e00 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c210 tx=0x7f4aec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926ed1ff8980d41a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8007cf0 con 0x7f4aec0039e0 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af8005ad0 con 0x7f4aec0039e0 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b08a00640 1 -- 172.21.15.59:0/1052809790 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8007070 con 0x7f4aec0039e0 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b0c800640 1 -- 172.21.15.59:0/1052809790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aec0039e0 msgr2=0x7f4aec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b0c800640 1 --2- 172.21.15.59:0/1052809790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aec0039e0 0x7f4aec003e00 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c210 tx=0x7f4aec005830 comp rx=0 tx=0).stop 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b0a800640 1 -- 172.21.15.59:0/1052809790 reap_dead start 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b0c800640 1 -- 172.21.15.59:0/1052809790 shutdown_connections 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b0c800640 1 -- 172.21.15.59:0/1052809790 >> 172.21.15.59:0/1052809790 conn(0x7f4b04067d90 msgr2=0x7f4b0406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b0c800640 1 -- 172.21.15.59:0/1052809790 shutdown_connections 2024-04-23T14:44:58.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.608+0000 7f4b0c800640 1 -- 172.21.15.59:0/1052809790 wait complete. 2024-04-23T14:44:58.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.610+0000 7f4b0c800640 1 Processor -- start 2024-04-23T14:44:58.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.610+0000 7f4b0c800640 1 -- start start 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.611+0000 7f4b0c800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0039e0 0x7f4b041532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.611+0000 7f4b0c800640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b041647a0 0x7f4b040b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.611+0000 7f4b0c800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b040b2570 0x7f4b04165730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.611+0000 7f4b0c800640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04163170 con 0x7f4b040b2570 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0c800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04162ff0 con 0x7f4aec0039e0 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0c800640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0406e4e0 con 0x7f4b041647a0 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b040b2570 0x7f4b04165730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0a800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b040b2570 0x7f4b04165730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34888/0 (socket says 172.21.15.59:34888) 2024-04-23T14:44:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0a800640 1 -- 172.21.15.59:0/3924375942 learned_addr learned my addr 172.21.15.59:0/3924375942 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b09e00640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0039e0 0x7f4b041532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b09400640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b041647a0 0x7f4b040b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0a800640 1 -- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b041647a0 msgr2=0x7f4b040b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0a800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b041647a0 0x7f4b040b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0a800640 1 -- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0039e0 msgr2=0x7f4b041532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.612+0000 7f4b0a800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0039e0 0x7f4b041532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4b0a800640 1 -- 172.21.15.59:0/3924375942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b04163700 con 0x7f4b040b2570 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4b09e00640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0039e0 0x7f4b041532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4b0a800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b040b2570 0x7f4b04165730 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f4b040aff20 tx=0x7f4b0000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4afe000640 1 -- 172.21.15.59:0/3924375942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00017070 con 0x7f4b040b2570 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4afe000640 1 -- 172.21.15.59:0/3924375942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00005d60 con 0x7f4b040b2570 2024-04-23T14:44:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b041639e0 con 0x7f4b040b2570 2024-04-23T14:44:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4afe000640 1 -- 172.21.15.59:0/3924375942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000a7f0 con 0x7f4b040b2570 2024-04-23T14:44:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.613+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b04163fa0 con 0x7f4b040b2570 2024-04-23T14:44:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.615+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b04068a80 con 0x7f4b040b2570 2024-04-23T14:44:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.616+0000 7f4afe000640 1 -- 172.21.15.59:0/3924375942 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b0001a070 con 0x7f4b040b2570 2024-04-23T14:44:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.616+0000 7f4afe000640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae806c920 0x7f4ae806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.617+0000 7f4afe000640 1 -- 172.21.15.59:0/3924375942 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6025+0+0 (secure 0 0 0) 0x7f4b0008ad60 con 0x7f4b040b2570 2024-04-23T14:44:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.617+0000 7f4b09e00640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae806c920 0x7f4ae806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.617+0000 7f4b09e00640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae806c920 0x7f4ae806ede0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4af8032000 tx=0x7f4af8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:58.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.621+0000 7f4afe000640 1 -- 172.21.15.59:0/3924375942 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b00058fb0 con 0x7f4b040b2570 2024-04-23T14:44:58.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:58.870+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4b0406bb00 con 0x7f4b040b2570 2024-04-23T14:44:59.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.398+0000 7f4afe000640 1 -- 172.21.15.59:0/3924375942 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7f4b000588b0 con 0x7f4b040b2570 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae806c920 msgr2=0x7f4ae806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae806c920 0x7f4ae806ede0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4af8032000 tx=0x7f4af8009030 comp rx=0 tx=0).stop 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b040b2570 msgr2=0x7f4b04165730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b040b2570 0x7f4b04165730 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f4b040aff20 tx=0x7f4b0000cb90 comp rx=0 tx=0).stop 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 shutdown_connections 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4ae806c920 0x7f4ae806ede0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4b041647a0 0x7f4b040b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b040b2570 0x7f4b04165730 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:59.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 --2- 172.21.15.59:0/3924375942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0039e0 0x7f4b041532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.402+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 >> 172.21.15.59:0/3924375942 conn(0x7f4b04067d90 msgr2=0x7f4b0415d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.403+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 shutdown_connections 2024-04-23T14:44:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.403+0000 7f4b0c800640 1 -- 172.21.15.59:0/3924375942 wait complete. 2024-04-23T14:44:59.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T14:44:59.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.638+0000 7f428c000640 1 Processor -- start 2024-04-23T14:44:59.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.638+0000 7f428c000640 1 -- start start 2024-04-23T14:44:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.640+0000 7f428c000640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42840724a0 0x7f42840b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.640+0000 7f428c000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42841641e0 con 0x7f4284163ce0 2024-04-23T14:44:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.640+0000 7f428c000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f428406e130 con 0x7f4284162e60 2024-04-23T14:44:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.640+0000 7f428c000640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f428406e410 con 0x7f42840724a0 2024-04-23T14:44:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.641+0000 7f4288c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42840724a0 0x7f42840b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.641+0000 7f4288c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42840724a0 0x7f42840b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34900/0 (socket says 172.21.15.59:34900) 2024-04-23T14:44:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.641+0000 7f4288c00640 1 -- 172.21.15.59:0/817287296 learned_addr learned my addr 172.21.15.59:0/817287296 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:59.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.641+0000 7f4288c00640 1 -- 172.21.15.59:0/817287296 >> v1:172.21.15.76:6789/0 conn(0x7f4284162e60 legacy=0x7f4284071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:44:59.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.641+0000 7f4288c00640 1 -- 172.21.15.59:0/817287296 >> v1:172.21.15.59:6789/0 conn(0x7f4284163ce0 legacy=0x7f42840b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:59.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.641+0000 7f4288c00640 1 -- 172.21.15.59:0/817287296 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428406e6f0 con 0x7f42840724a0 2024-04-23T14:44:59.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.642+0000 7f4288c00640 1 --2- 172.21.15.59:0/817287296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f42840b3430 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4270009870 tx=0x7f427002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3019e749a8d82c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f4283e00640 1 -- 172.21.15.59:0/817287296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270037070 con 0x7f42840724a0 2024-04-23T14:44:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f4283e00640 1 -- 172.21.15.59:0/817287296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4270032440 con 0x7f42840724a0 2024-04-23T14:44:59.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f4283e00640 1 -- 172.21.15.59:0/817287296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270032810 con 0x7f42840724a0 2024-04-23T14:44:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f428c000640 1 -- 172.21.15.59:0/817287296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 msgr2=0x7f42840b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f428c000640 1 --2- 172.21.15.59:0/817287296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f42840b3430 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4270009870 tx=0x7f427002a180 comp rx=0 tx=0).stop 2024-04-23T14:44:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f428c000640 1 -- 172.21.15.59:0/817287296 shutdown_connections 2024-04-23T14:44:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f428c000640 1 --2- 172.21.15.59:0/817287296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f42840b3430 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.643+0000 7f428c000640 1 -- 172.21.15.59:0/817287296 >> 172.21.15.59:0/817287296 conn(0x7f428406b9d0 msgr2=0x7f428406bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.644+0000 7f428c000640 1 -- 172.21.15.59:0/817287296 shutdown_connections 2024-04-23T14:44:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.644+0000 7f428c000640 1 -- 172.21.15.59:0/817287296 wait complete. 2024-04-23T14:44:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.645+0000 7f428c000640 1 Processor -- start 2024-04-23T14:44:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.645+0000 7f428c000640 1 -- start start 2024-04-23T14:44:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.646+0000 7f428c000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f428415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.646+0000 7f428c000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4284162e60 0x7f42840cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.646+0000 7f428c000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4284163ce0 0x7f428415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.646+0000 7f428c000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f42840afe80 con 0x7f42840724a0 2024-04-23T14:44:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.646+0000 7f428c000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42840afd00 con 0x7f4284163ce0 2024-04-23T14:44:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.646+0000 7f428c000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f42840b4080 con 0x7f4284162e60 2024-04-23T14:44:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f428415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f428415a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34906/0 (socket says 172.21.15.59:34906) 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 -- 172.21.15.59:0/706982942 learned_addr learned my addr 172.21.15.59:0/706982942 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 -- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4284162e60 msgr2=0x7f42840cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4288c00640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4284162e60 0x7f42840cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f428a000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4284163ce0 0x7f428415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4284162e60 0x7f42840cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 -- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4284163ce0 msgr2=0x7f428415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4284163ce0 0x7f428415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f4289600640 1 -- 172.21.15.59:0/706982942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42840d02c0 con 0x7f42840724a0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.647+0000 7f428a000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4284163ce0 0x7f428415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.648+0000 7f4289600640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f428415a050 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f427800b3f0 tx=0x7f427800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.648+0000 7f4281600640 1 -- 172.21.15.59:0/706982942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278015020 con 0x7f42840724a0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.648+0000 7f4281600640 1 -- 172.21.15.59:0/706982942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4278011830 con 0x7f42840724a0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.648+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428415a630 con 0x7f42840724a0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.648+0000 7f4281600640 1 -- 172.21.15.59:0/706982942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278011c20 con 0x7f42840724a0 2024-04-23T14:44:59.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.648+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428415abf0 con 0x7f42840724a0 2024-04-23T14:44:59.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.649+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4284068a80 con 0x7f42840724a0 2024-04-23T14:44:59.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.651+0000 7f4281600640 1 -- 172.21.15.59:0/706982942 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4278018020 con 0x7f42840724a0 2024-04-23T14:44:59.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.652+0000 7f4281600640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f426406c8f0 0x7f426406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:59.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.652+0000 7f4281600640 1 -- 172.21.15.59:0/706982942 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6025+0+0 (secure 0 0 0) 0x7f427808b940 con 0x7f42840724a0 2024-04-23T14:44:59.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.652+0000 7f4288c00640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f426406c8f0 0x7f426406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:59.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.653+0000 7f4288c00640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f426406c8f0 0x7f426406edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4270034000 tx=0x7f427002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:59.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.656+0000 7f4281600640 1 -- 172.21.15.59:0/706982942 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f427801f030 con 0x7f42840724a0 2024-04-23T14:44:59.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:44:59.858+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42840b36a0 con 0x7f42840724a0 2024-04-23T14:45:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.412+0000 7f4281600640 1 -- 172.21.15.59:0/706982942 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) ==== 181+0+0 (secure 0 0 0) 0x7f4278059b90 con 0x7f42840724a0 2024-04-23T14:45:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-04-23T14:45:00.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.416+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f426406c8f0 msgr2=0x7f426406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:00.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.416+0000 7f428c000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f426406c8f0 0x7f426406edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4270034000 tx=0x7f427002e000 comp rx=0 tx=0).stop 2024-04-23T14:45:00.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.416+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 msgr2=0x7f428415a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:00.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.416+0000 7f428c000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f428415a050 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f427800b3f0 tx=0x7f427800b8b0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 shutdown_connections 2024-04-23T14:45:00.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f426406c8f0 0x7f426406edb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4284162e60 0x7f42840cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4284163ce0 0x7f428415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 --2- 172.21.15.59:0/706982942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42840724a0 0x7f428415a050 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 >> 172.21.15.59:0/706982942 conn(0x7f428406b9d0 msgr2=0x7f428406bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:00.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 shutdown_connections 2024-04-23T14:45:00.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.417+0000 7f428c000640 1 -- 172.21.15.59:0/706982942 wait complete. 2024-04-23T14:45:00.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T14:45:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.644+0000 7f3aa8600640 1 Processor -- start 2024-04-23T14:45:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.644+0000 7f3aa8600640 1 -- start start 2024-04-23T14:45:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.645+0000 7f3aa8600640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3aa00d0820 0x7f3aa00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.645+0000 7f3aa8600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3aa00ce230 0x7f3aa006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.645+0000 7f3aa8600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa00ce9c0 con 0x7f3aa0144f40 2024-04-23T14:45:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.645+0000 7f3aa8600640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa00ceca0 con 0x7f3aa00d0820 2024-04-23T14:45:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.645+0000 7f3aa8600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa00cef80 con 0x7f3aa00ce230 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.646+0000 7f3aa5c00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3aa0144f40 0x7f3aa00d0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42268/0 (socket says 172.21.15.59:42268) 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.646+0000 7f3aa5c00640 1 -- 172.21.15.59:0/3470863955 learned_addr learned my addr 172.21.15.59:0/3470863955 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.646+0000 7f3aa5200640 1 --2- 172.21.15.59:0/3470863955 >> v2:172.21.15.76:3300/0 conn(0x7f3aa00d0820 0x7f3aa00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.646+0000 7f3aa6600640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3aa00ce230 0x7f3aa006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.646+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2473421232 0 0) 0x7f3aa00ce9c0 con 0x7f3aa0144f40 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a88005730 con 0x7f3aa0144f40 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2210136070 0 0) 0x7f3a88005730 con 0x7f3aa0144f40 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3aa00ce230 msgr2=0x7f3aa006ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3aa00ce230 0x7f3aa006ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 >> v2:172.21.15.76:3300/0 conn(0x7f3aa00d0820 msgr2=0x7f3aa00b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 --2- 172.21.15.59:0/3470863955 >> v2:172.21.15.76:3300/0 conn(0x7f3aa00d0820 0x7f3aa00b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa00cf260 con 0x7f3aa0144f40 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f3a94001670 con 0x7f3aa0144f40 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 >> v1:172.21.15.59:6789/0 conn(0x7f3aa0144f40 legacy=0x7f3aa00d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a88005bc0 0x7f3a88005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88006a10 0x7f3a88006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3aa00ce230 0x7f3a880073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00ceca0 con 0x7f3a88005bc0 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00cef80 con 0x7f3a88006a10 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8800dbd0 con 0x7f3aa00ce230 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.647+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a88005bc0 0x7f3a88005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.648+0000 7f3aa5200640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88006a10 0x7f3a88006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.648+0000 7f3aa5c00640 1 -- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3aa00ce230 msgr2=0x7f3a880073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.648+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3aa00ce230 0x7f3a880073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.648+0000 7f3aa5c00640 1 -- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88006a10 msgr2=0x7f3a88006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.648+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a88006a10 0x7f3a88006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.648+0000 7f3aa5c00640 1 -- 172.21.15.59:0/3470863955 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a88005730 con 0x7f3a88005bc0 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.649+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a88005bc0 0x7f3a88005fe0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c190 tx=0x7f3a88007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24c18d4fab7f73e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.649+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94007c60 con 0x7f3a88005bc0 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.649+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94005af0 con 0x7f3a88005bc0 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.649+0000 7f3a9fe00640 1 -- 172.21.15.59:0/3470863955 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94007070 con 0x7f3a88005bc0 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.650+0000 7f3aa8600640 1 -- 172.21.15.59:0/3470863955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a88005bc0 msgr2=0x7f3a88005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.650+0000 7f3aa8600640 1 --2- 172.21.15.59:0/3470863955 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a88005bc0 0x7f3a88005fe0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c190 tx=0x7f3a88007a40 comp rx=0 tx=0).stop 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.650+0000 7f3aa6600640 1 -- 172.21.15.59:0/3470863955 reap_dead start 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.650+0000 7f3aa8600640 1 -- 172.21.15.59:0/3470863955 shutdown_connections 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.650+0000 7f3aa8600640 1 -- 172.21.15.59:0/3470863955 >> 172.21.15.59:0/3470863955 conn(0x7f3aa006b9d0 msgr2=0x7f3aa006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.650+0000 7f3aa8600640 1 -- 172.21.15.59:0/3470863955 shutdown_connections 2024-04-23T14:45:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.650+0000 7f3aa8600640 1 -- 172.21.15.59:0/3470863955 wait complete. 2024-04-23T14:45:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.652+0000 7f3aa8600640 1 Processor -- start 2024-04-23T14:45:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.652+0000 7f3aa8600640 1 -- start start 2024-04-23T14:45:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.653+0000 7f3aa8600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a88005bc0 0x7f3aa0134c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.653+0000 7f3aa8600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa00c8040 0x7f3aa00c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.653+0000 7f3aa8600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa00ca4e0 0x7f3aa00ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.653+0000 7f3aa8600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0145320 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.653+0000 7f3aa8600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00ce6a0 con 0x7f3aa00c8040 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.653+0000 7f3aa8600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa0071d90 con 0x7f3a88005bc0 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.653+0000 7f3aa6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa00ca4e0 0x7f3aa00ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa00ca4e0 0x7f3aa00ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43430/0 (socket says 172.21.15.59:43430) 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 -- 172.21.15.59:0/4033803261 learned_addr learned my addr 172.21.15.59:0/4033803261 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa5200640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa00c8040 0x7f3aa00c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a88005bc0 0x7f3aa0134c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 -- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a88005bc0 msgr2=0x7f3aa0134c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a88005bc0 0x7f3aa0134c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 -- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa00c8040 msgr2=0x7f3aa00c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa00c8040 0x7f3aa00c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 -- 172.21.15.59:0/4033803261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa00cb280 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa5200640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa00c8040 0x7f3aa00c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:00.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a88005bc0 0x7f3aa0134c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.654+0000 7f3aa6600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa00ca4e0 0x7f3aa00ca990 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0073010 tx=0x7f3a9800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:00.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.655+0000 7f3a9d600640 1 -- 172.21.15.59:0/4033803261 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800bce0 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.655+0000 7f3a9d600640 1 -- 172.21.15.59:0/4033803261 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9801a070 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.655+0000 7f3a9d600640 1 -- 172.21.15.59:0/4033803261 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98014640 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.655+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa00cb560 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.655+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa00cbb20 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.656+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa0068a80 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.657+0000 7f3a9d600640 1 -- 172.21.15.59:0/4033803261 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a98005d10 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.658+0000 7f3a9d600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a8006c920 0x7f3a8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.659+0000 7f3a9d600640 1 -- 172.21.15.59:0/4033803261 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5587+0+0 (secure 0 0 0) 0x7f3a9808bca0 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.659+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a8006c920 0x7f3a8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:00.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.660+0000 7f3aa5c00640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a8006c920 0x7f3a8006ede0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3a94032000 tx=0x7f3a94009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:00.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.663+0000 7f3a9d600640 1 -- 172.21.15.59:0/4033803261 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a9805a0b0 con 0x7f3aa00ca4e0 2024-04-23T14:45:00.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:00.879+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3aa00d12f0 con 0x7f3aa00ca4e0 2024-04-23T14:45:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.438+0000 7f3a9d600640 1 -- 172.21.15.59:0/4033803261 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7f3a980599b0 con 0x7f3aa00ca4e0 2024-04-23T14:45:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-04-23T14:45:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a8006c920 msgr2=0x7f3a8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a8006c920 0x7f3a8006ede0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3a94032000 tx=0x7f3a94009030 comp rx=0 tx=0).stop 2024-04-23T14:45:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa00ca4e0 msgr2=0x7f3aa00ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa00ca4e0 0x7f3aa00ca990 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0073010 tx=0x7f3a9800b480 comp rx=0 tx=0).stop 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 shutdown_connections 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f3a8006c920 0x7f3a8006ede0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa00ca4e0 0x7f3aa00ca990 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa00c8040 0x7f3aa00c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 --2- 172.21.15.59:0/4033803261 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a88005bc0 0x7f3aa0134c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.442+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 >> 172.21.15.59:0/4033803261 conn(0x7f3aa006b9d0 msgr2=0x7f3aa00b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.443+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 shutdown_connections 2024-04-23T14:45:01.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.443+0000 7f3aa8600640 1 -- 172.21.15.59:0/4033803261 wait complete. 2024-04-23T14:45:01.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-23T14:45:01.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.669+0000 7fe6db200640 1 Processor -- start 2024-04-23T14:45:01.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.670+0000 7fe6db200640 1 -- start start 2024-04-23T14:45:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.671+0000 7fe6db200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6d415c410 0x7fe6d40d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.671+0000 7fe6db200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 0x7fe6d40af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.671+0000 7fe6db200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d4160120 con 0x7fe6d40af5c0 2024-04-23T14:45:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.671+0000 7fe6db200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d40d0f70 con 0x7fe6d415c410 2024-04-23T14:45:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.671+0000 7fe6db200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d4160400 con 0x7fe6d40d0b10 2024-04-23T14:45:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.671+0000 7fe6da800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe6d40af5c0 0x7fe6d40b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42278/0 (socket says 172.21.15.59:42278) 2024-04-23T14:45:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.671+0000 7fe6da800640 1 -- 172.21.15.59:0/1440120830 learned_addr learned my addr 172.21.15.59:0/1440120830 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d9e00640 1 --2- 172.21.15.59:0/1440120830 >> v2:172.21.15.76:3300/0 conn(0x7fe6d415c410 0x7fe6d40d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d9400640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 0x7fe6d40af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2146315377 0 0) 0x7fe6d4160120 con 0x7fe6d40af5c0 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6bc003520 con 0x7fe6d40af5c0 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 223594367 0 0) 0x7fe6bc003520 con 0x7fe6d40af5c0 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 msgr2=0x7fe6d40af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 0x7fe6d40af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 >> v2:172.21.15.76:3300/0 conn(0x7fe6d415c410 msgr2=0x7fe6d40d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 --2- 172.21.15.59:0/1440120830 >> v2:172.21.15.76:3300/0 conn(0x7fe6d415c410 0x7fe6d40d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d41606e0 con 0x7fe6d40af5c0 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fe6d00019f0 con 0x7fe6d40af5c0 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.672+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 >> v1:172.21.15.59:6789/0 conn(0x7fe6d40af5c0 legacy=0x7fe6d40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d8a00640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 0x7fe6bc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d8a00640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6bc003f80 0x7fe6bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d8a00640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4160400 con 0x7fe6bc004d40 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc00b9c0 con 0x7fe6bc003f80 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc00bb40 con 0x7fe6d40d0b10 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6da800640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d9400640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6bc003f80 0x7fe6bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d9e00640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 0x7fe6bc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6da800640 1 -- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 msgr2=0x7fe6bc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6da800640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d40d0b10 0x7fe6bc0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6da800640 1 -- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6bc003f80 msgr2=0x7fe6bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6da800640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6bc003f80 0x7fe6bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6da800640 1 -- 172.21.15.59:0/1440120830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc003520 con 0x7fe6bc004d40 2024-04-23T14:45:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.673+0000 7fe6d9400640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6bc003f80 0x7fe6bc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.674+0000 7fe6da800640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000a7e0 tx=0x7fe6bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d644112cc47e84b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000a3d0 con 0x7fe6bc004d40 2024-04-23T14:45:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d002f070 con 0x7fe6bc004d40 2024-04-23T14:45:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6d8a00640 1 -- 172.21.15.59:0/1440120830 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d002bc50 con 0x7fe6bc004d40 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6db200640 1 -- 172.21.15.59:0/1440120830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 msgr2=0x7fe6bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6db200640 1 --2- 172.21.15.59:0/1440120830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000a7e0 tx=0x7fe6bc005830 comp rx=0 tx=0).stop 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6da800640 1 -- 172.21.15.59:0/1440120830 reap_dead start 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6db200640 1 -- 172.21.15.59:0/1440120830 shutdown_connections 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6db200640 1 -- 172.21.15.59:0/1440120830 >> 172.21.15.59:0/1440120830 conn(0x7fe6d406bdf0 msgr2=0x7fe6d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.675+0000 7fe6db200640 1 -- 172.21.15.59:0/1440120830 shutdown_connections 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.676+0000 7fe6db200640 1 -- 172.21.15.59:0/1440120830 wait complete. 2024-04-23T14:45:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.677+0000 7fe6db200640 1 Processor -- start 2024-04-23T14:45:01.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.678+0000 7fe6db200640 1 -- start start 2024-04-23T14:45:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6db200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6d40cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6db200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d4145660 0x7fe6d415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6db200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6d41539b0 0x7fe6d4158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6db200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d415c7f0 con 0x7fe6bc004d40 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6db200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d40afa50 con 0x7fe6d41539b0 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6db200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d41f43f0 con 0x7fe6d4145660 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6d40cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6d40cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43446/0 (socket says 172.21.15.59:43446) 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 -- 172.21.15.59:0/468788682 learned_addr learned my addr 172.21.15.59:0/468788682 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 -- 172.21.15.59:0/468788682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d4145660 msgr2=0x7fe6d415b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9400640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d4145660 0x7fe6d415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d4145660 0x7fe6d415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 -- 172.21.15.59:0/468788682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6d41539b0 msgr2=0x7fe6d4158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6d41539b0 0x7fe6d4158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.679+0000 7fe6d9e00640 1 -- 172.21.15.59:0/468788682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d40ca200 con 0x7fe6bc004d40 2024-04-23T14:45:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.680+0000 7fe6d9e00640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6d40cb2a0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8010940 tx=0x7fe6c8010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.680+0000 7fe6ce000640 1 -- 172.21.15.59:0/468788682 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800f040 con 0x7fe6bc004d40 2024-04-23T14:45:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.680+0000 7fe6ce000640 1 -- 172.21.15.59:0/468788682 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c800ebc0 con 0x7fe6bc004d40 2024-04-23T14:45:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.680+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d40ca4e0 con 0x7fe6bc004d40 2024-04-23T14:45:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.680+0000 7fe6ce000640 1 -- 172.21.15.59:0/468788682 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c801c400 con 0x7fe6bc004d40 2024-04-23T14:45:01.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.680+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d40caaa0 con 0x7fe6bc004d40 2024-04-23T14:45:01.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.681+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d404fd90 con 0x7fe6bc004d40 2024-04-23T14:45:01.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.682+0000 7fe6ce000640 1 -- 172.21.15.59:0/468788682 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6c801c5a0 con 0x7fe6bc004d40 2024-04-23T14:45:01.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.683+0000 7fe6ce000640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6b806c8f0 0x7fe6b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.683+0000 7fe6ce000640 1 -- 172.21.15.59:0/468788682 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fe6c808bfc0 con 0x7fe6bc004d40 2024-04-23T14:45:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.684+0000 7fe6d9400640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6b806c8f0 0x7fe6b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:01.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.684+0000 7fe6d9400640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6b806c8f0 0x7fe6b806edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe6d406fc40 tx=0x7fe6c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:01.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.688+0000 7fe6ce000640 1 -- 172.21.15.59:0/468788682 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6c8018030 con 0x7fe6bc004d40 2024-04-23T14:45:01.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.893+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe6d40b5170 con 0x7fe6bc004d40 2024-04-23T14:45:01.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.899+0000 7fe6ce000640 1 -- 172.21.15.59:0/468788682 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fe6c805a5a0 con 0x7fe6bc004d40 2024-04-23T14:45:01.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.901+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6b806c8f0 msgr2=0x7fe6b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.901+0000 7fe6db200640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6b806c8f0 0x7fe6b806edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe6d406fc40 tx=0x7fe6c4008040 comp rx=0 tx=0).stop 2024-04-23T14:45:01.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.901+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 msgr2=0x7fe6d40cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:01.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.901+0000 7fe6db200640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6d40cb2a0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8010940 tx=0x7fe6c8010e00 comp rx=0 tx=0).stop 2024-04-23T14:45:01.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 shutdown_connections 2024-04-23T14:45:01.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fe6b806c8f0 0x7fe6b806edb0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe6d4145660 0x7fe6d415b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6d41539b0 0x7fe6d4158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 --2- 172.21.15.59:0/468788682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6bc004d40 0x7fe6d40cb2a0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:01.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 >> 172.21.15.59:0/468788682 conn(0x7fe6d406bdf0 msgr2=0x7fe6d415e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:01.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 shutdown_connections 2024-04-23T14:45:01.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:01.902+0000 7fe6db200640 1 -- 172.21.15.59:0/468788682 wait complete. 2024-04-23T14:45:01.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-23T14:45:02.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.123+0000 7f5c91200640 1 Processor -- start 2024-04-23T14:45:02.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.123+0000 7f5c91200640 1 -- start start 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.124+0000 7f5c91200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5c8c0b5ff0 0x7f5c8c0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.124+0000 7f5c91200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c8c0cf030 con 0x7f5c8c15e7c0 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.124+0000 7f5c91200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c8c0b6a40 con 0x7f5c8c0d0d90 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c91200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f5c8c0b7000 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c8aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5c8c0b5ff0 0x7f5c8c0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c8aa00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5c8c0b5ff0 0x7f5c8c0b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43458/0 (socket says 172.21.15.59:43458) 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c8aa00640 1 -- 172.21.15.59:0/1544768818 learned_addr learned my addr 172.21.15.59:0/1544768818 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:02.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c8aa00640 1 -- 172.21.15.59:0/1544768818 >> v1:172.21.15.76:6789/0 conn(0x7f5c8c0d0d90 legacy=0x7f5c8c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:02.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c8aa00640 1 -- 172.21.15.59:0/1544768818 >> v1:172.21.15.59:6789/0 conn(0x7f5c8c15e7c0 legacy=0x7f5c8c0d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c8a000640 1 -- 172.21.15.59:0/1544768818 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1777047078 0 0) 0x7f5c7c001110 con 0x7f5c8c15e7c0 2024-04-23T14:45:02.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.125+0000 7f5c8aa00640 1 -- 172.21.15.59:0/1544768818 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8c0b6d20 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.126+0000 7f5c8aa00640 1 --2- 172.21.15.59:0/1544768818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b6450 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009870 tx=0x7f5c7402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7740e11cabf4abf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:02.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c8a000640 1 -- 172.21.15.59:0/1544768818 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74037070 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c8a000640 1 -- 172.21.15.59:0/1544768818 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c74032440 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c8a000640 1 -- 172.21.15.59:0/1544768818 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74032810 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c91200640 1 -- 172.21.15.59:0/1544768818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 msgr2=0x7f5c8c0b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c91200640 1 --2- 172.21.15.59:0/1544768818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b6450 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009870 tx=0x7f5c7402a2a0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c91200640 1 -- 172.21.15.59:0/1544768818 shutdown_connections 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c91200640 1 --2- 172.21.15.59:0/1544768818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b6450 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.127+0000 7f5c91200640 1 -- 172.21.15.59:0/1544768818 >> 172.21.15.59:0/1544768818 conn(0x7f5c8c09d920 msgr2=0x7f5c8c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.128+0000 7f5c91200640 1 -- 172.21.15.59:0/1544768818 shutdown_connections 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.128+0000 7f5c91200640 1 -- 172.21.15.59:0/1544768818 wait complete. 2024-04-23T14:45:02.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.129+0000 7f5c91200640 1 Processor -- start 2024-04-23T14:45:02.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.130+0000 7f5c91200640 1 -- start start 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.130+0000 7f5c91200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.130+0000 7f5c91200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c8c0d0d90 0x7f5c8c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c91200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c8c15e7c0 0x7f5c8c145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c91200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c0d2800 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c91200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c0cf620 con 0x7f5c8c15e7c0 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c91200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8c0d1a80 con 0x7f5c8c0d0d90 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43468/0 (socket says 172.21.15.59:43468) 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 -- 172.21.15.59:0/1998276492 learned_addr learned my addr 172.21.15.59:0/1998276492 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8be00640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c8c15e7c0 0x7f5c8c145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8aa00640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c8c0d0d90 0x7f5c8c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 -- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c8c0d0d90 msgr2=0x7f5c8c0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c8c0d0d90 0x7f5c8c0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 -- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c8c15e7c0 msgr2=0x7f5c8c145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c8c15e7c0 0x7f5c8c145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8b400640 1 -- 172.21.15.59:0/1998276492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8c0b1ce0 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.131+0000 7f5c8be00640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c8c15e7c0 0x7f5c8c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:02.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.132+0000 7f5c8b400640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b2780 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c750 tx=0x7f5c7c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.132+0000 7f5c83400640 1 -- 172.21.15.59:0/1998276492 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c017070 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.132+0000 7f5c83400640 1 -- 172.21.15.59:0/1998276492 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0052b0 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.132+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8c0b7540 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.132+0000 7f5c83400640 1 -- 172.21.15.59:0/1998276492 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00a880 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.132+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8c0b7820 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.133+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c8c0b23b0 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.134+0000 7f5c83400640 1 -- 172.21.15.59:0/1998276492 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c7c01a070 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.135+0000 7f5c83400640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c6806c8f0 0x7f5c6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.135+0000 7f5c83400640 1 -- 172.21.15.59:0/1998276492 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f5c7c08abc0 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.136+0000 7f5c8aa00640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c6806c8f0 0x7f5c6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.136+0000 7f5c8aa00640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c6806c8f0 0x7f5c6806edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5c74034000 tx=0x7f5c7402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:02.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.140+0000 7f5c83400640 1 -- 172.21.15.59:0/1998276492 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c8c0cf030 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.419+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f5c8c04fd90 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.441+0000 7f5c83400640 1 -- 172.21.15.59:0/1998276492 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v639) ==== 144+0+0 (secure 0 0 0) 0x7f5c7c007d30 con 0x7f5c8c0b5ff0 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c6806c8f0 msgr2=0x7f5c6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c6806c8f0 0x7f5c6806edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5c74034000 tx=0x7f5c7402e000 comp rx=0 tx=0).stop 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 msgr2=0x7f5c8c0b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b2780 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c750 tx=0x7f5c7c00cc10 comp rx=0 tx=0).stop 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 shutdown_connections 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f5c6806c8f0 0x7f5c6806edb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c8c0d0d90 0x7f5c8c0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c8c15e7c0 0x7f5c8c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 --2- 172.21.15.59:0/1998276492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c8c0b5ff0 0x7f5c8c0b2780 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.444+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 >> 172.21.15.59:0/1998276492 conn(0x7f5c8c09d920 msgr2=0x7f5c8c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.445+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 shutdown_connections 2024-04-23T14:45:02.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:02.445+0000 7f5c91200640 1 -- 172.21.15.59:0/1998276492 wait complete. 2024-04-23T14:45:02.464 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-23T14:45:02.465 INFO:tasks.cephfs_test_runner: 2024-04-23T14:45:02.465 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-23T14:45:02.465 INFO:tasks.cephfs_test_runner:Ran 7 tests in 773.381s 2024-04-23T14:45:02.465 INFO:tasks.cephfs_test_runner: 2024-04-23T14:45:02.465 INFO:tasks.cephfs_test_runner:OK 2024-04-23T14:45:02.465 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-23T14:45:02.475 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-23T14:45:02.483 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-23T14:45:02.483 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:45:02.483 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:45:02.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:45:02.537 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-04-23T14:45:02.537 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:45:02.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-23T14:45:02.594 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:45:02.594 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:45:02.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:45:02.651 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-23T14:45:02.651 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:45:02.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-23T14:45:02.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:02.705 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-04-23T14:45:02.763 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:45:02.763 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-23T14:45:02.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:02.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:45:02.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:02.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-23T14:45:02.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:02.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-04-23T14:45:03.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:03.058 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-04-23T14:45:03.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-23T14:45:03.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-23T14:45:03.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:03.108 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T14:45:03.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:03.236 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-23T14:45:03.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:45:03.307 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-04-23T14:45:03.425 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-23T14:45:03.435 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-23T14:45:03.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:45:03.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.656+0000 7ff778a00640 1 Processor -- start 2024-04-23T14:45:03.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.657+0000 7ff778a00640 1 -- start start 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff778a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7740cd760 0x7ff7740b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff778a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7740b4690 0x7ff7740cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff778a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff774160210 con 0x7ff77415c3d0 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff778a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7ff7741604f0 con 0x7ff7740b4690 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff778a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff7741607d0 con 0x7ff7740cd760 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff773e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7740b4690 0x7ff7740cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff773e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7740b4690 0x7ff7740cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43482/0 (socket says 172.21.15.59:43482) 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.658+0000 7ff773e00640 1 -- 172.21.15.59:0/3423559190 learned_addr learned my addr 172.21.15.59:0/3423559190 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:03.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.659+0000 7ff76aa00640 1 --2- 172.21.15.59:0/3423559190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7740cd760 0x7ff7740b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.659+0000 7ff773e00640 1 -- 172.21.15.59:0/3423559190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7740cd760 msgr2=0x7ff7740b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.659+0000 7ff773e00640 1 --2- 172.21.15.59:0/3423559190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7740cd760 0x7ff7740b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.659+0000 7ff773e00640 1 -- 172.21.15.59:0/3423559190 >> v1:172.21.15.76:6789/0 conn(0x7ff77415c3d0 legacy=0x7ff7740cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.659+0000 7ff773e00640 1 -- 172.21.15.59:0/3423559190 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff774160ab0 con 0x7ff7740b4690 2024-04-23T14:45:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.659+0000 7ff76aa00640 1 --2- 172.21.15.59:0/3423559190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7740cd760 0x7ff7740b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.660+0000 7ff773e00640 1 --2- 172.21.15.59:0/3423559190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740b4690 0x7ff7740cb7d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009e50 tx=0x7ff76c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86e6504ad9d3ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:03.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.660+0000 7ff772a00640 1 -- 172.21.15.59:0/3423559190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c036070 con 0x7ff7740b4690 2024-04-23T14:45:03.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.660+0000 7ff772a00640 1 -- 172.21.15.59:0/3423559190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c031600 con 0x7ff7740b4690 2024-04-23T14:45:03.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.660+0000 7ff772a00640 1 -- 172.21.15.59:0/3423559190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0319d0 con 0x7ff7740b4690 2024-04-23T14:45:03.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.661+0000 7ff778a00640 1 -- 172.21.15.59:0/3423559190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740b4690 msgr2=0x7ff7740cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:03.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.661+0000 7ff778a00640 1 --2- 172.21.15.59:0/3423559190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740b4690 0x7ff7740cb7d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009e50 tx=0x7ff76c02a6d0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.661+0000 7ff778a00640 1 -- 172.21.15.59:0/3423559190 shutdown_connections 2024-04-23T14:45:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.661+0000 7ff778a00640 1 --2- 172.21.15.59:0/3423559190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740b4690 0x7ff7740cb7d0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.661+0000 7ff778a00640 1 --2- 172.21.15.59:0/3423559190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7740cd760 0x7ff7740b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.661+0000 7ff778a00640 1 -- 172.21.15.59:0/3423559190 >> 172.21.15.59:0/3423559190 conn(0x7ff7740b21a0 msgr2=0x7ff7740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.661+0000 7ff778a00640 1 -- 172.21.15.59:0/3423559190 shutdown_connections 2024-04-23T14:45:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.662+0000 7ff778a00640 1 -- 172.21.15.59:0/3423559190 wait complete. 2024-04-23T14:45:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.663+0000 7ff778a00640 1 Processor -- start 2024-04-23T14:45:03.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.663+0000 7ff778a00640 1 -- start start 2024-04-23T14:45:03.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.664+0000 7ff778a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7740b4690 0x7ff7740c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.664+0000 7ff778a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740cd760 0x7ff774159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.664+0000 7ff778a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff77415c3d0 0x7ff77415f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.664+0000 7ff778a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff77415c7b0 con 0x7ff7740cd760 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.664+0000 7ff778a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff7740b4ac0 con 0x7ff7740b4690 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.664+0000 7ff778a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7ff77415c930 con 0x7ff77415c3d0 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740cd760 0x7ff774159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740cd760 0x7ff774159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43484/0 (socket says 172.21.15.59:43484) 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 -- 172.21.15.59:0/3321944626 learned_addr learned my addr 172.21.15.59:0/3321944626 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff773400640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7740b4690 0x7ff7740c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff773e00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff77415c3d0 0x7ff77415f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 -- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff77415c3d0 msgr2=0x7ff77415f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff77415c3d0 0x7ff77415f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 -- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7740b4690 msgr2=0x7ff7740c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:03.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7740b4690 0x7ff7740c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.665+0000 7ff76aa00640 1 -- 172.21.15.59:0/3321944626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77415a6b0 con 0x7ff7740cd760 2024-04-23T14:45:03.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.666+0000 7ff76aa00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740cd760 0x7ff774159fb0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c7f0 tx=0x7ff75c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:03.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.666+0000 7ff76be00640 1 -- 172.21.15.59:0/3321944626 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c015070 con 0x7ff7740cd760 2024-04-23T14:45:03.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.666+0000 7ff76be00640 1 -- 172.21.15.59:0/3321944626 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75c007a30 con 0x7ff7740cd760 2024-04-23T14:45:03.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.666+0000 7ff76be00640 1 -- 172.21.15.59:0/3321944626 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c007e00 con 0x7ff7740cd760 2024-04-23T14:45:03.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.666+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77415a990 con 0x7ff7740cd760 2024-04-23T14:45:03.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.666+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77415af50 con 0x7ff7740cd760 2024-04-23T14:45:03.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.667+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77404fd90 con 0x7ff7740cd760 2024-04-23T14:45:03.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.668+0000 7ff76be00640 1 -- 172.21.15.59:0/3321944626 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff75c018070 con 0x7ff7740cd760 2024-04-23T14:45:03.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.669+0000 7ff76be00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff75006c8f0 0x7ff75006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:03.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.669+0000 7ff76be00640 1 -- 172.21.15.59:0/3321944626 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7ff75c08b9b0 con 0x7ff7740cd760 2024-04-23T14:45:03.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.670+0000 7ff773400640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff75006c8f0 0x7ff75006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:03.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.670+0000 7ff773400640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff75006c8f0 0x7ff75006edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff764006fd0 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:03.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.674+0000 7ff76be00640 1 -- 172.21.15.59:0/3321944626 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff75c059f90 con 0x7ff7740cd760 2024-04-23T14:45:03.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.875+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff77406fa10 con 0x7ff75006c8f0 2024-04-23T14:45:03.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.877+0000 7ff76be00640 1 -- 172.21.15.59:0/3321944626 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13523 (secure 0 0 0) 0x7ff77406fa10 con 0x7ff75006c8f0 2024-04-23T14:45:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:45:03.877 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:45:03.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff75006c8f0 msgr2=0x7ff75006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff75006c8f0 0x7ff75006edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff764006fd0 tx=0x7ff764008040 comp rx=0 tx=0).stop 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740cd760 msgr2=0x7ff774159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740cd760 0x7ff774159fb0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c7f0 tx=0x7ff75c00cbf0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 shutdown_connections 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7ff75006c8f0 0x7ff75006edb0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff77415c3d0 0x7ff77415f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7740cd760 0x7ff774159fb0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 --2- 172.21.15.59:0/3321944626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7740b4690 0x7ff7740c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 >> 172.21.15.59:0/3321944626 conn(0x7ff7740b21a0 msgr2=0x7ff77406f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.880+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 shutdown_connections 2024-04-23T14:45:03.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:03.881+0000 7ff778a00640 1 -- 172.21.15.59:0/3321944626 wait complete. 2024-04-23T14:45:03.901 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":526,"stamp":"2024-04-23T14:45:02.644503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278432,"kb_used_data":28992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696288,"statfs":{"total":773094113280,"available":772808998912,"internally_reserved":0,"allocated":29687808,"data_stored":23049618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-186,"num_write_kb":-8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001526"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-23T14:44:58.729855+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:44:58.729855+0000","last_peered":"2024-04-23T14:44:58.729855+0000","last_clean":"2024-04-23T14:44:58.729855+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:44:58.729855+0000","last_undegraded":"2024-04-23T14:44:58.729855+0000","last_fullsized":"2024-04-23T14:44:58.729855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1226913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":5656576,"data_stored":4830310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1470464,"data_stored":636545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":635929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575016,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":5536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":5668864,"data_stored":4831610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1458176,"data_stored":635934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":5656576,"data_stored":4830357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":6120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":6266880,"data_stored":5422020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:45:03.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:45:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.116+0000 7f8c0be00640 1 Processor -- start 2024-04-23T14:45:04.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.116+0000 7f8c0be00640 1 -- start start 2024-04-23T14:45:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.117+0000 7f8c0be00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8c0c15b450 0x7f8c0c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.117+0000 7f8c0be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.117+0000 7f8c0be00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0c0b3870 con 0x7f8c0c0b3080 2024-04-23T14:45:04.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.117+0000 7f8c0be00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0c0b41a0 con 0x7f8c0c15b450 2024-04-23T14:45:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.117+0000 7f8c0be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c0b3b50 con 0x7f8c0c06f3f0 2024-04-23T14:45:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.118+0000 7f8c0aa00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8c0c15b450 0x7f8c0c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.118+0000 7f8c02000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.118+0000 7f8c02000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:57622/0 (socket says 172.21.15.59:57622) 2024-04-23T14:45:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.118+0000 7f8c02000640 1 -- 172.21.15.59:0/3933795841 learned_addr learned my addr 172.21.15.59:0/3933795841 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.118+0000 7f8c0b400640 1 --1- 172.21.15.59:0/3933795841 >> v1:172.21.15.76:6789/0 conn(0x7f8c0c0b3080 0x7f8c0c0ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40000/0 (socket says 172.21.15.59:40000) 2024-04-23T14:45:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.119+0000 7f8c02000640 1 -- 172.21.15.59:0/3933795841 >> v2:172.21.15.76:3300/0 conn(0x7f8c0c15b450 msgr2=0x7f8c0c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.119+0000 7f8c02000640 1 --2- 172.21.15.59:0/3933795841 >> v2:172.21.15.76:3300/0 conn(0x7f8c0c15b450 0x7f8c0c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.119+0000 7f8c02000640 1 -- 172.21.15.59:0/3933795841 >> v1:172.21.15.76:6789/0 conn(0x7f8c0c0b3080 legacy=0x7f8c0c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.119+0000 7f8c02000640 1 -- 172.21.15.59:0/3933795841 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c0b3e30 con 0x7f8c0c06f3f0 2024-04-23T14:45:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.119+0000 7f8c0a000640 1 -- 172.21.15.59:0/3933795841 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385136174 0 0) 0x7f8c0c0b3870 con 0x7f8c0c0b3080 2024-04-23T14:45:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.120+0000 7f8c02000640 1 --2- 172.21.15.59:0/3933795841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0cd050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400bb10 tx=0x7f8bf402c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dacdbebf08185123 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.121+0000 7f8c0a000640 1 -- 172.21.15.59:0/3933795841 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4036020 con 0x7f8c0c06f3f0 2024-04-23T14:45:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.121+0000 7f8c0a000640 1 -- 172.21.15.59:0/3933795841 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf4032820 con 0x7f8c0c06f3f0 2024-04-23T14:45:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.121+0000 7f8c0a000640 1 -- 172.21.15.59:0/3933795841 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4032bf0 con 0x7f8c0c06f3f0 2024-04-23T14:45:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 -- 172.21.15.59:0/3933795841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 msgr2=0x7f8c0c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 --2- 172.21.15.59:0/3933795841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0cd050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400bb10 tx=0x7f8bf402c920 comp rx=0 tx=0).stop 2024-04-23T14:45:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 -- 172.21.15.59:0/3933795841 shutdown_connections 2024-04-23T14:45:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 --2- 172.21.15.59:0/3933795841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0cd050 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 --2- 172.21.15.59:0/3933795841 >> v2:172.21.15.76:3300/0 conn(0x7f8c0c15b450 0x7f8c0c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 -- 172.21.15.59:0/3933795841 >> 172.21.15.59:0/3933795841 conn(0x7f8c0c0b21a0 msgr2=0x7f8c0c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 -- 172.21.15.59:0/3933795841 shutdown_connections 2024-04-23T14:45:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.122+0000 7f8c0be00640 1 -- 172.21.15.59:0/3933795841 wait complete. 2024-04-23T14:45:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.124+0000 7f8c0be00640 1 Processor -- start 2024-04-23T14:45:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.124+0000 7f8c0be00640 1 -- start start 2024-04-23T14:45:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0be00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0be00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c0c0b3080 0x7f8c0c1e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0be00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c0c15b450 0x7f8c0c1e4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0be00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c0b4ff0 con 0x7f8c0c15b450 2024-04-23T14:45:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0be00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c15b830 con 0x7f8c0c0b3080 2024-04-23T14:45:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0be00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0c0cd590 con 0x7f8c0c06f3f0 2024-04-23T14:45:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c0c15b450 0x7f8c0c1e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0b400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c0c15b450 0x7f8c0c1e4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43486/0 (socket says 172.21.15.59:43486) 2024-04-23T14:45:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.125+0000 7f8c0b400640 1 -- 172.21.15.59:0/1054227610 learned_addr learned my addr 172.21.15.59:0/1054227610 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c02000640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c0c0b3080 0x7f8c0c1e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0aa00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0b400640 1 -- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 msgr2=0x7f8c0c0c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0b400640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0b400640 1 -- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c0c0b3080 msgr2=0x7f8c0c1e60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0b400640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c0c0b3080 0x7f8c0c1e60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0b400640 1 -- 172.21.15.59:0/1054227610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c1e5330 con 0x7f8c0c15b450 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c02000640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c0c0b3080 0x7f8c0c1e60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0aa00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c0b400640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c0c15b450 0x7f8c0c1e4470 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c7a0 tx=0x7f8c0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.126+0000 7f8c03400640 1 -- 172.21.15.59:0/1054227610 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04018070 con 0x7f8c0c15b450 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.127+0000 7f8c03400640 1 -- 172.21.15.59:0/1054227610 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0400a640 con 0x7f8c0c15b450 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.127+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c1e5610 con 0x7f8c0c15b450 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.127+0000 7f8c03400640 1 -- 172.21.15.59:0/1054227610 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400aa70 con 0x7f8c0c15b450 2024-04-23T14:45:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.127+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1e6630 con 0x7f8c0c15b450 2024-04-23T14:45:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.128+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c04fd90 con 0x7f8c0c15b450 2024-04-23T14:45:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.129+0000 7f8c03400640 1 -- 172.21.15.59:0/1054227610 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c0401b070 con 0x7f8c0c15b450 2024-04-23T14:45:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.130+0000 7f8c03400640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8be806c8f0 0x7f8be806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.130+0000 7f8c03400640 1 -- 172.21.15.59:0/1054227610 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f8c0408afd0 con 0x7f8c0c15b450 2024-04-23T14:45:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.131+0000 7f8c0aa00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8be806c8f0 0x7f8be806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.131+0000 7f8c0aa00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8be806c8f0 0x7f8be806edb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009a20 tx=0x7f8bfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:04.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.135+0000 7f8c03400640 1 -- 172.21.15.59:0/1054227610 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c040595b0 con 0x7f8c0c15b450 2024-04-23T14:45:04.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.336+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c0c0cf1f0 con 0x7f8be806c8f0 2024-04-23T14:45:04.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.338+0000 7f8c03400640 1 -- 172.21.15.59:0/1054227610 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13523 (secure 0 0 0) 0x7f8c0c0cf1f0 con 0x7f8be806c8f0 2024-04-23T14:45:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:45:04.339 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:45:04.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.340+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8be806c8f0 msgr2=0x7f8be806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.340+0000 7f8c0be00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8be806c8f0 0x7f8be806edb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009a20 tx=0x7f8bfc008040 comp rx=0 tx=0).stop 2024-04-23T14:45:04.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c0c15b450 msgr2=0x7f8c0c1e4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c0c15b450 0x7f8c0c1e4470 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c7a0 tx=0x7f8c0400cc60 comp rx=0 tx=0).stop 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 shutdown_connections 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f8be806c8f0 0x7f8be806edb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c0c15b450 0x7f8c0c1e4470 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c0c0b3080 0x7f8c0c1e60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 --2- 172.21.15.59:0/1054227610 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c0c06f3f0 0x7f8c0c0c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 >> 172.21.15.59:0/1054227610 conn(0x7f8c0c0b21a0 msgr2=0x7f8c0c0af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 shutdown_connections 2024-04-23T14:45:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.341+0000 7f8c0be00640 1 -- 172.21.15.59:0/1054227610 wait complete. 2024-04-23T14:45:04.360 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":526,"stamp":"2024-04-23T14:45:02.644503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278432,"kb_used_data":28992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696288,"statfs":{"total":773094113280,"available":772808998912,"internally_reserved":0,"allocated":29687808,"data_stored":23049618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-186,"num_write_kb":-8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001526"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-23T14:44:58.729855+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:44:58.729855+0000","last_peered":"2024-04-23T14:44:58.729855+0000","last_clean":"2024-04-23T14:44:58.729855+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:44:58.729855+0000","last_undegraded":"2024-04-23T14:44:58.729855+0000","last_fullsized":"2024-04-23T14:44:58.729855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1226913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":5656576,"data_stored":4830310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1470464,"data_stored":636545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":635929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575016,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":5536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":5668864,"data_stored":4831610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1458176,"data_stored":635934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":5656576,"data_stored":4830357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":6120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":6266880,"data_stored":5422020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:45:04.360 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-23T14:45:04.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:45:04.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.562+0000 7f50b3200640 1 Processor -- start 2024-04-23T14:45:04.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.562+0000 7f50b3200640 1 -- start start 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f50b415b450 0x7f50b40b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b406f3f0 0x7f50b40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b3200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50b40b3870 con 0x7f50b40b3080 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f50b40b41a0 con 0x7f50b415b450 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f50b40b3b50 con 0x7f50b406f3f0 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b1e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f50b415b450 0x7f50b40b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b1e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f50b415b450 0x7f50b40b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43500/0 (socket says 172.21.15.59:43500) 2024-04-23T14:45:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b1e00640 1 -- 172.21.15.59:0/2703213645 learned_addr learned my addr 172.21.15.59:0/2703213645 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b1e00640 1 -- 172.21.15.59:0/2703213645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b406f3f0 msgr2=0x7f50b40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.564+0000 7f50b1400640 1 --2- 172.21.15.59:0/2703213645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b406f3f0 0x7f50b40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.565+0000 7f50b1e00640 1 --2- 172.21.15.59:0/2703213645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b406f3f0 0x7f50b40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.565+0000 7f50b1e00640 1 -- 172.21.15.59:0/2703213645 >> v1:172.21.15.76:6789/0 conn(0x7f50b40b3080 legacy=0x7f50b40ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.565+0000 7f50b1e00640 1 -- 172.21.15.59:0/2703213645 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b40b3e30 con 0x7f50b415b450 2024-04-23T14:45:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.566+0000 7f50b1e00640 1 --2- 172.21.15.59:0/2703213645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40b4630 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009e50 tx=0x7f50a402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12d78033b91e291 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.566+0000 7f50b0a00640 1 -- 172.21.15.59:0/2703213645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4036070 con 0x7f50b415b450 2024-04-23T14:45:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.566+0000 7f50b0a00640 1 -- 172.21.15.59:0/2703213645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a4031600 con 0x7f50b415b450 2024-04-23T14:45:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.566+0000 7f50b0a00640 1 -- 172.21.15.59:0/2703213645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a40319d0 con 0x7f50b415b450 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 -- 172.21.15.59:0/2703213645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 msgr2=0x7f50b40b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 --2- 172.21.15.59:0/2703213645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40b4630 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009e50 tx=0x7f50a402a6d0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 -- 172.21.15.59:0/2703213645 shutdown_connections 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 --2- 172.21.15.59:0/2703213645 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b406f3f0 0x7f50b40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 --2- 172.21.15.59:0/2703213645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40b4630 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 -- 172.21.15.59:0/2703213645 >> 172.21.15.59:0/2703213645 conn(0x7f50b40b21a0 msgr2=0x7f50b40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 -- 172.21.15.59:0/2703213645 shutdown_connections 2024-04-23T14:45:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.567+0000 7f50b3200640 1 -- 172.21.15.59:0/2703213645 wait complete. 2024-04-23T14:45:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.569+0000 7f50b3200640 1 Processor -- start 2024-04-23T14:45:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.569+0000 7f50b3200640 1 -- start start 2024-04-23T14:45:04.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50b406f3f0 0x7f50b408e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b40b3080 0x7f50b415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50b40b4ff0 con 0x7f50b415b450 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50b415b830 con 0x7f50b406f3f0 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f50b40cd590 con 0x7f50b40b3080 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b2800640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43502/0 (socket says 172.21.15.59:43502) 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b2800640 1 -- 172.21.15.59:0/2367708143 learned_addr learned my addr 172.21.15.59:0/2367708143 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b1e00640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50b406f3f0 0x7f50b408e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.570+0000 7f50b1400640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b40b3080 0x7f50b415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b2800640 1 -- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b40b3080 msgr2=0x7f50b415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b2800640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b40b3080 0x7f50b415cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b2800640 1 -- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50b406f3f0 msgr2=0x7f50b408e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b2800640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50b406f3f0 0x7f50b408e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b2800640 1 -- 172.21.15.59:0/2367708143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b415d690 con 0x7f50b415b450 2024-04-23T14:45:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b2800640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40c5cf0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c440 tx=0x7f50ac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50aa000640 1 -- 172.21.15.59:0/2367708143 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac011790 con 0x7f50b415b450 2024-04-23T14:45:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50aa000640 1 -- 172.21.15.59:0/2367708143 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac017070 con 0x7f50b415b450 2024-04-23T14:45:04.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b415d970 con 0x7f50b415b450 2024-04-23T14:45:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50aa000640 1 -- 172.21.15.59:0/2367708143 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac011b70 con 0x7f50b415b450 2024-04-23T14:45:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.571+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b415df30 con 0x7f50b415b450 2024-04-23T14:45:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.573+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b404fd90 con 0x7f50b415b450 2024-04-23T14:45:04.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.574+0000 7f50aa000640 1 -- 172.21.15.59:0/2367708143 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50ac011d10 con 0x7f50b415b450 2024-04-23T14:45:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.574+0000 7f50aa000640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f509406c8f0 0x7f509406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.575+0000 7f50aa000640 1 -- 172.21.15.59:0/2367708143 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f50ac08b750 con 0x7f50b415b450 2024-04-23T14:45:04.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.575+0000 7f50b1e00640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f509406c8f0 0x7f509406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.576+0000 7f50b1e00640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f509406c8f0 0x7f509406edb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f50a4033000 tx=0x7f50a4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:04.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.579+0000 7f50aa000640 1 -- 172.21.15.59:0/2367708143 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50ac059d30 con 0x7f50b415b450 2024-04-23T14:45:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.779+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50b4159710 con 0x7f509406c8f0 2024-04-23T14:45:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.780+0000 7f50aa000640 1 -- 172.21.15.59:0/2367708143 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13514 (secure 0 0 0) 0x7f50b4159710 con 0x7f509406c8f0 2024-04-23T14:45:04.781 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:45:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f509406c8f0 msgr2=0x7f509406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f509406c8f0 0x7f509406edb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f50a4033000 tx=0x7f50a4009000 comp rx=0 tx=0).stop 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 msgr2=0x7f50b40c5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40c5cf0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c440 tx=0x7f50ac00c900 comp rx=0 tx=0).stop 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 shutdown_connections 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f509406c8f0 0x7f509406edb0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50b40b3080 0x7f50b415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50b415b450 0x7f50b40c5cf0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 --2- 172.21.15.59:0/2367708143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50b406f3f0 0x7f50b408e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 >> 172.21.15.59:0/2367708143 conn(0x7f50b40b21a0 msgr2=0x7f50b40d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:04.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.784+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 shutdown_connections 2024-04-23T14:45:04.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.785+0000 7f50b3200640 1 -- 172.21.15.59:0/2367708143 wait complete. 2024-04-23T14:45:04.804 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2024-04-23T14:45:04.645062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262236,"kb_used_data":12776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712484,"statfs":{"total":773094113280,"available":772825583616,"internally_reserved":0,"allocated":13082624,"data_stored":6364200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-186,"num_write_kb":-8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.246881"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":357,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-23T14:45:03.043637+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:45:03.043637+0000","last_peered":"2024-04-23T14:45:03.043637+0000","last_clean":"2024-04-23T14:45:03.043637+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:45:03.043637+0000","last_undegraded":"2024-04-23T14:45:03.043637+0000","last_fullsized":"2024-04-23T14:45:03.043637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1238301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":1238301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:45:04.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:45:04.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.993+0000 7f2e3a600640 1 Processor -- start 2024-04-23T14:45:04.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.993+0000 7f2e3a600640 1 -- start start 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e3a600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e3415cb20 0x7f2e340cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e3a600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e34163770 0x7f2e340d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e3a600640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340d01c0 con 0x7f2e34160180 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e3a600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3415cf00 con 0x7f2e3415cb20 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e3a600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3415f6a0 con 0x7f2e34163770 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e33400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e3415cb20 0x7f2e340cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e33400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e3415cb20 0x7f2e340cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43514/0 (socket says 172.21.15.59:43514) 2024-04-23T14:45:04.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e33400640 1 -- 172.21.15.59:0/2186558489 learned_addr learned my addr 172.21.15.59:0/2186558489 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:04.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.995+0000 7f2e32a00640 1 --2- 172.21.15.59:0/2186558489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e34163770 0x7f2e340d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:04.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.996+0000 7f2e33400640 1 -- 172.21.15.59:0/2186558489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e34163770 msgr2=0x7f2e340d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:04.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.996+0000 7f2e33400640 1 --2- 172.21.15.59:0/2186558489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e34163770 0x7f2e340d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.996+0000 7f2e33400640 1 -- 172.21.15.59:0/2186558489 >> v1:172.21.15.59:6789/0 conn(0x7f2e34160180 legacy=0x7f2e340b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.996+0000 7f2e33400640 1 -- 172.21.15.59:0/2186558489 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3415f980 con 0x7f2e3415cb20 2024-04-23T14:45:04.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.997+0000 7f2e33400640 1 --2- 172.21.15.59:0/2186558489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e3415cb20 0x7f2e340cf840 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009870 tx=0x7f2e2402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcc09a3b1ea760b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:04.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.997+0000 7f2e32000640 1 -- 172.21.15.59:0/2186558489 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24037070 con 0x7f2e3415cb20 2024-04-23T14:45:04.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.997+0000 7f2e32000640 1 -- 172.21.15.59:0/2186558489 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e24032440 con 0x7f2e3415cb20 2024-04-23T14:45:04.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.997+0000 7f2e32000640 1 -- 172.21.15.59:0/2186558489 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24032810 con 0x7f2e3415cb20 2024-04-23T14:45:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 -- 172.21.15.59:0/2186558489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e3415cb20 msgr2=0x7f2e340cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 --2- 172.21.15.59:0/2186558489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e3415cb20 0x7f2e340cf840 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009870 tx=0x7f2e2402a1c0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 -- 172.21.15.59:0/2186558489 shutdown_connections 2024-04-23T14:45:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 --2- 172.21.15.59:0/2186558489 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e34163770 0x7f2e340d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 --2- 172.21.15.59:0/2186558489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e3415cb20 0x7f2e340cf840 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 -- 172.21.15.59:0/2186558489 >> 172.21.15.59:0/2186558489 conn(0x7f2e3406cc70 msgr2=0x7f2e3406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 -- 172.21.15.59:0/2186558489 shutdown_connections 2024-04-23T14:45:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:04.998+0000 7f2e3a600640 1 -- 172.21.15.59:0/2186558489 wait complete. 2024-04-23T14:45:05.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.000+0000 7f2e3a600640 1 Processor -- start 2024-04-23T14:45:05.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.000+0000 7f2e3a600640 1 -- start start 2024-04-23T14:45:05.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.001+0000 7f2e3a600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3415cb20 0x7f2e340c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.001+0000 7f2e3a600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e34160180 0x7f2e340ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.001+0000 7f2e3a600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e34163770 0x7f2e341f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.001+0000 7f2e3a600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340d1780 con 0x7f2e34160180 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.001+0000 7f2e3a600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340d1900 con 0x7f2e34163770 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.001+0000 7f2e3a600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f2e340d1a80 con 0x7f2e3415cb20 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e34160180 0x7f2e340ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e32a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e34160180 0x7f2e340ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43520/0 (socket says 172.21.15.59:43520) 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e32a00640 1 -- 172.21.15.59:0/3066721714 learned_addr learned my addr 172.21.15.59:0/3066721714 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e33400640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3415cb20 0x7f2e340c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e33e00640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e34163770 0x7f2e341f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e32a00640 1 -- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3415cb20 msgr2=0x7f2e340c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e32a00640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3415cb20 0x7f2e340c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e32a00640 1 -- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e34163770 msgr2=0x7f2e341f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.002+0000 7f2e32a00640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e34163770 0x7f2e341f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e32a00640 1 -- 172.21.15.59:0/3066721714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e340cac50 con 0x7f2e34160180 2024-04-23T14:45:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e33e00640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e34163770 0x7f2e341f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:45:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e32a00640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e34160180 0x7f2e340ca710 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000d5f0 tx=0x7f2e2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e2b400640 1 -- 172.21.15.59:0/3066721714 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e200125f0 con 0x7f2e34160180 2024-04-23T14:45:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e2b400640 1 -- 172.21.15.59:0/3066721714 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e20019070 con 0x7f2e34160180 2024-04-23T14:45:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e340c93e0 con 0x7f2e34160180 2024-04-23T14:45:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e2b400640 1 -- 172.21.15.59:0/3066721714 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e200129d0 con 0x7f2e34160180 2024-04-23T14:45:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.003+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e340c99a0 con 0x7f2e34160180 2024-04-23T14:45:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.005+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e34068a80 con 0x7f2e34160180 2024-04-23T14:45:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.006+0000 7f2e2b400640 1 -- 172.21.15.59:0/3066721714 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e20012b70 con 0x7f2e34160180 2024-04-23T14:45:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.007+0000 7f2e2b400640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e1406c8f0 0x7f2e1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.008+0000 7f2e2b400640 1 -- 172.21.15.59:0/3066721714 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f2e2008bb90 con 0x7f2e34160180 2024-04-23T14:45:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.008+0000 7f2e33400640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e1406c8f0 0x7f2e1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.009+0000 7f2e33400640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e1406c8f0 0x7f2e1406edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2e24034000 tx=0x7f2e2402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:05.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.012+0000 7f2e2b400640 1 -- 172.21.15.59:0/3066721714 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e2005a2a0 con 0x7f2e34160180 2024-04-23T14:45:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.219+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e340d29c0 con 0x7f2e34160180 2024-04-23T14:45:05.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.221+0000 7f2e2b400640 1 -- 172.21.15.59:0/3066721714 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+13230 (secure 0 0 0) 0x7f2e20059ba0 con 0x7f2e34160180 2024-04-23T14:45:05.221 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:45:05.221 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":119,"fsid":"b8f41672-e126-44ae-8657-b688ecdd9da9","created":"2024-04-23T14:31:01.137449+0000","modified":"2024-04-23T14:45:01.428548+0000","last_up_change":"2024-04-23T14:31:13.638299+0000","last_in_change":"2024-04-23T14:31:04.373113+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:31:14.380574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c32b555b-b29b-44b0-b863-8a04bc4adf5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6817","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6819","nonce":2024703144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6821","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6823","nonce":2024703144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6829","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6831","nonce":2024703144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6825","nonce":2024703144},{"type":"v1","addr":"172.21.15.59:6827","nonce":2024703144}]},"public_addr":"172.21.15.59:6819/2024703144","cluster_addr":"172.21.15.59:6823/2024703144","heartbeat_back_addr":"172.21.15.59:6831/2024703144","heartbeat_front_addr":"172.21.15.59:6827/2024703144","state":["exists","up"]},{"osd":1,"uuid":"722b82c6-69c6-4a02-ae94-6aad5483b240","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6809","nonce":4218092099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6811","nonce":4218092099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6815","nonce":4218092099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":4218092099},{"type":"v1","addr":"172.21.15.59:6813","nonce":4218092099}]},"public_addr":"172.21.15.59:6809/4218092099","cluster_addr":"172.21.15.59:6811/4218092099","heartbeat_back_addr":"172.21.15.59:6815/4218092099","heartbeat_front_addr":"172.21.15.59:6813/4218092099","state":["exists","up"]},{"osd":2,"uuid":"d8c1c4f4-5d5c-4691-967e-9ba1629ec2f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6818","nonce":77995666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6822","nonce":77995666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6830","nonce":77995666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":77995666},{"type":"v1","addr":"172.21.15.59:6826","nonce":77995666}]},"public_addr":"172.21.15.59:6818/77995666","cluster_addr":"172.21.15.59:6822/77995666","heartbeat_back_addr":"172.21.15.59:6830/77995666","heartbeat_front_addr":"172.21.15.59:6826/77995666","state":["exists","up"]},{"osd":3,"uuid":"ee2b3514-932e-467d-9511-732911d272d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6801","nonce":3205049728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6803","nonce":3205049728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6807","nonce":3205049728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3205049728},{"type":"v1","addr":"172.21.15.59:6805","nonce":3205049728}]},"public_addr":"172.21.15.59:6801/3205049728","cluster_addr":"172.21.15.59:6803/3205049728","heartbeat_back_addr":"172.21.15.59:6807/3205049728","heartbeat_front_addr":"172.21.15.59:6805/3205049728","state":["exists","up"]},{"osd":4,"uuid":"5796b331-54b0-4a70-a36b-80cb3c7ad4bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6817","nonce":3407674963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6819","nonce":3407674963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6825","nonce":3407674963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":3407674963},{"type":"v1","addr":"172.21.15.76:6821","nonce":3407674963}]},"public_addr":"172.21.15.76:6817/3407674963","cluster_addr":"172.21.15.76:6819/3407674963","heartbeat_back_addr":"172.21.15.76:6825/3407674963","heartbeat_front_addr":"172.21.15.76:6821/3407674963","state":["exists","up"]},{"osd":5,"uuid":"4b57258f-8745-4ffc-9413-86fb13026f2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6809","nonce":2782996536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6811","nonce":2782996536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6815","nonce":2782996536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2782996536},{"type":"v1","addr":"172.21.15.76:6813","nonce":2782996536}]},"public_addr":"172.21.15.76:6809/2782996536","cluster_addr":"172.21.15.76:6811/2782996536","heartbeat_back_addr":"172.21.15.76:6815/2782996536","heartbeat_front_addr":"172.21.15.76:6813/2782996536","state":["exists","up"]},{"osd":6,"uuid":"9869c195-a732-4e0e-b45d-9d5784308783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6801","nonce":1890860772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6803","nonce":1890860772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6807","nonce":1890860772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1890860772},{"type":"v1","addr":"172.21.15.76:6805","nonce":1890860772}]},"public_addr":"172.21.15.76:6801/1890860772","cluster_addr":"172.21.15.76:6803/1890860772","heartbeat_back_addr":"172.21.15.76:6807/1890860772","heartbeat_front_addr":"172.21.15.76:6805/1890860772","state":["exists","up"]},{"osd":7,"uuid":"82e9249f-85be-4a8a-890e-4f6c83e70085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6824","nonce":3281262326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6827","nonce":3281262326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6831","nonce":3281262326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3281262326},{"type":"v1","addr":"172.21.15.76:6829","nonce":3281262326}]},"public_addr":"172.21.15.76:6824/3281262326","cluster_addr":"172.21.15.76:6827/3281262326","heartbeat_back_addr":"172.21.15.76:6831/3281262326","heartbeat_front_addr":"172.21.15.76:6829/3281262326","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.993794+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.694227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.981689+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.736937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.860159+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.794339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.615622+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:31:09.568779+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6837/3735308728":"2024-04-24T14:44:58.390972+0000","172.21.15.59:6836/3735308728":"2024-04-24T14:44:58.390972+0000","172.21.15.59:6837/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6836/3383882209":"2024-04-24T14:44:38.869275+0000","172.21.15.59:6837/1061483588":"2024-04-24T14:44:14.736925+0000","172.21.15.59:6836/1061483588":"2024-04-24T14:44:14.736925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:45:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e1406c8f0 msgr2=0x7f2e1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e1406c8f0 0x7f2e1406edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2e24034000 tx=0x7f2e2402e000 comp rx=0 tx=0).stop 2024-04-23T14:45:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e34160180 msgr2=0x7f2e340ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e34160180 0x7f2e340ca710 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000d5f0 tx=0x7f2e2000dab0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 shutdown_connections 2024-04-23T14:45:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f2e1406c8f0 0x7f2e1406edb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e34163770 0x7f2e341f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e34160180 0x7f2e340ca710 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 --2- 172.21.15.59:0/3066721714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3415cb20 0x7f2e340c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 >> 172.21.15.59:0/3066721714 conn(0x7f2e3406cc70 msgr2=0x7f2e3415e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 shutdown_connections 2024-04-23T14:45:05.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:05.224+0000 7f2e3a600640 1 -- 172.21.15.59:0/3066721714 wait complete. 2024-04-23T14:45:06.245 INFO:tasks.ceph:Scrubbing osd.0 2024-04-23T14:45:06.245 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:06.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.474+0000 7f96a6a00640 1 Processor -- start 2024-04-23T14:45:06.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.474+0000 7f96a6a00640 1 -- start start 2024-04-23T14:45:06.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.476+0000 7f96a6a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f96a00cd7e0 0x7f96a006e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.476+0000 7f96a6a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f96a006e840 0x7f96a00cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.476+0000 7f96a6a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a00cf340 con 0x7f96a01431b0 2024-04-23T14:45:06.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.476+0000 7f96a6a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f96a006ecd0 con 0x7f96a006e840 2024-04-23T14:45:06.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.476+0000 7f96a6a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f96a00cf670 con 0x7f96a00cd7e0 2024-04-23T14:45:06.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f96a4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f96a006e840 0x7f96a00cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f96a4a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f96a006e840 0x7f96a00cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43536/0 (socket says 172.21.15.59:43536) 2024-04-23T14:45:06.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f96a4a00640 1 -- 172.21.15.59:0/3762504538 learned_addr learned my addr 172.21.15.59:0/3762504538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:06.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f969f400640 1 --2- 172.21.15.59:0/3762504538 >> v2:172.21.15.76:3300/0 conn(0x7f96a00cd7e0 0x7f96a006e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f96a4a00640 1 -- 172.21.15.59:0/3762504538 >> v2:172.21.15.76:3300/0 conn(0x7f96a00cd7e0 msgr2=0x7f96a006e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f96a4a00640 1 --2- 172.21.15.59:0/3762504538 >> v2:172.21.15.76:3300/0 conn(0x7f96a00cd7e0 0x7f96a006e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f96a4a00640 1 -- 172.21.15.59:0/3762504538 >> v1:172.21.15.76:6789/0 conn(0x7f96a01431b0 legacy=0x7f96a00cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f96a4a00640 1 -- 172.21.15.59:0/3762504538 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a006ef10 con 0x7f96a006e840 2024-04-23T14:45:06.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.477+0000 7f969f400640 1 --2- 172.21.15.59:0/3762504538 >> v2:172.21.15.76:3300/0 conn(0x7f96a00cd7e0 0x7f96a006e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.478+0000 7f96a4a00640 1 --2- 172.21.15.59:0/3762504538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a006e840 0x7f96a00cfb00 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f96a0146800 tx=0x7f969802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557f8fb97d6cd66b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.479+0000 7f969ea00640 1 -- 172.21.15.59:0/3762504538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698036070 con 0x7f96a006e840 2024-04-23T14:45:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.479+0000 7f969ea00640 1 -- 172.21.15.59:0/3762504538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9698031600 con 0x7f96a006e840 2024-04-23T14:45:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.479+0000 7f969ea00640 1 -- 172.21.15.59:0/3762504538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980319d0 con 0x7f96a006e840 2024-04-23T14:45:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.479+0000 7f96a6a00640 1 -- 172.21.15.59:0/3762504538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a006e840 msgr2=0x7f96a00cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.479+0000 7f96a6a00640 1 --2- 172.21.15.59:0/3762504538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a006e840 0x7f96a00cfb00 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f96a0146800 tx=0x7f969802a780 comp rx=0 tx=0).stop 2024-04-23T14:45:06.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.480+0000 7f96a6a00640 1 -- 172.21.15.59:0/3762504538 shutdown_connections 2024-04-23T14:45:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.480+0000 7f96a6a00640 1 --2- 172.21.15.59:0/3762504538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a006e840 0x7f96a00cfb00 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.480+0000 7f96a6a00640 1 --2- 172.21.15.59:0/3762504538 >> v2:172.21.15.76:3300/0 conn(0x7f96a00cd7e0 0x7f96a006e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.480+0000 7f96a6a00640 1 -- 172.21.15.59:0/3762504538 >> 172.21.15.59:0/3762504538 conn(0x7f96a006b9d0 msgr2=0x7f96a006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.480+0000 7f96a6a00640 1 -- 172.21.15.59:0/3762504538 shutdown_connections 2024-04-23T14:45:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.480+0000 7f96a6a00640 1 -- 172.21.15.59:0/3762504538 wait complete. 2024-04-23T14:45:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.482+0000 7f96a6a00640 1 Processor -- start 2024-04-23T14:45:06.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.482+0000 7f96a6a00640 1 -- start start 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f96a6a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a006e840 0x7f96a01e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f96a6a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00cd7e0 0x7f96a01e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f96a6a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f96a01431b0 0x7f96a013ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f96a6a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96a00d0040 con 0x7f96a00cd7e0 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f96a6a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f96a006f4d0 con 0x7f96a006e840 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f96a6a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f96a00d01c0 con 0x7f96a01431b0 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f969f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00cd7e0 0x7f96a01e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f969f400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00cd7e0 0x7f96a01e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43548/0 (socket says 172.21.15.59:43548) 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f969f400640 1 -- 172.21.15.59:0/921160265 learned_addr learned my addr 172.21.15.59:0/921160265 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.483+0000 7f969fe00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a006e840 0x7f96a01e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f969f400640 1 -- 172.21.15.59:0/921160265 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f96a01431b0 msgr2=0x7f96a013ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f969f400640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f96a01431b0 0x7f96a013ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f969f400640 1 -- 172.21.15.59:0/921160265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a006e840 msgr2=0x7f96a01e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f969f400640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a006e840 0x7f96a01e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f969f400640 1 -- 172.21.15.59:0/921160265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a01e7380 con 0x7f96a00cd7e0 2024-04-23T14:45:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f969fe00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a006e840 0x7f96a01e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f969f400640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00cd7e0 0x7f96a01e6e40 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c820 tx=0x7f968c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:06.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c015070 con 0x7f96a00cd7e0 2024-04-23T14:45:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968c007a30 con 0x7f96a00cd7e0 2024-04-23T14:45:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c007e00 con 0x7f96a00cd7e0 2024-04-23T14:45:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a01e9a40 con 0x7f96a00cd7e0 2024-04-23T14:45:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.484+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a01ea000 con 0x7f96a00cd7e0 2024-04-23T14:45:06.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.485+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96a0158090 con 0x7f96a00cd7e0 2024-04-23T14:45:06.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.487+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f968c018070 con 0x7f96a00cd7e0 2024-04-23T14:45:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.488+0000 7f9697e00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.488+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f968c08b970 con 0x7f96a00cd7e0 2024-04-23T14:45:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.488+0000 7f969fe00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.488+0000 7f9697e00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f96800728f0 0x7f9680074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.488+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96800753c0 con 0x7f96800728f0 2024-04-23T14:45:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.488+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f968c0902a0 con 0x7f96a00cd7e0 2024-04-23T14:45:06.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.488+0000 7f96a4a00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f96800728f0 0x7f9680074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.489+0000 7f969fe00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f968006c8f0 0x7f968006edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f969000a810 tx=0x7f9690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:06.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.489+0000 7f96a4a00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f96800728f0 0x7f9680074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:06.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.490+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== osd.0 v2:172.21.15.59:6817/2024703144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96800753c0 con 0x7f96800728f0 2024-04-23T14:45:06.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.504+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 --> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f96a0068a80 con 0x7f96800728f0 2024-04-23T14:45:06.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.505+0000 7f9697e00640 1 -- 172.21.15.59:0/921160265 <== osd.0 v2:172.21.15.59:6817/2024703144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f96a0068a80 con 0x7f96800728f0 2024-04-23T14:45:06.505 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:06.505 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:06.505 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:06.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.505+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f96800728f0 msgr2=0x7f9680074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.505+0000 7f96a6a00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6817/2024703144,v1:172.21.15.59:6819/2024703144] conn(0x7f96800728f0 0x7f9680074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f968006c8f0 msgr2=0x7f968006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f968006c8f0 0x7f968006edb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f969000a810 tx=0x7f9690008040 comp rx=0 tx=0).stop 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00cd7e0 msgr2=0x7f96a01e6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 --2- 172.21.15.59:0/921160265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96a00cd7e0 0x7f96a01e6e40 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c820 tx=0x7f968c00cc20 comp rx=0 tx=0).stop 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a4a00640 1 -- 172.21.15.59:0/921160265 reap_dead start 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 shutdown_connections 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 >> 172.21.15.59:0/921160265 conn(0x7f96a006b9d0 msgr2=0x7f96a0072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 shutdown_connections 2024-04-23T14:45:06.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.506+0000 7f96a6a00640 1 -- 172.21.15.59:0/921160265 wait complete. 2024-04-23T14:45:06.523 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-23T14:45:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.726+0000 7f0929400640 1 Processor -- start 2024-04-23T14:45:06.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.727+0000 7f0929400640 1 -- start start 2024-04-23T14:45:06.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.728+0000 7f0929400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240d0cd0 0x7f09241674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.728+0000 7f0929400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09240b4cf0 con 0x7f09240cd8b0 2024-04-23T14:45:06.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.728+0000 7f0929400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09240ccdd0 con 0x7f092415c670 2024-04-23T14:45:06.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.728+0000 7f0929400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f092415ca50 con 0x7f09240d0cd0 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.729+0000 7f091be00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f09240cd8b0 0x7f09240d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42302/0 (socket says 172.21.15.59:42302) 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.729+0000 7f091be00640 1 -- 172.21.15.59:0/1253798916 learned_addr learned my addr 172.21.15.59:0/1253798916 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.729+0000 7f0923e00640 1 --1- 172.21.15.59:0/1253798916 >> v1:172.21.15.76:6789/0 conn(0x7f092415c670 0x7f09240b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40034/0 (socket says 172.21.15.59:40034) 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.729+0000 7f0928a00640 1 --2- 172.21.15.59:0/1253798916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240d0cd0 0x7f09241674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.729+0000 7f0923400640 1 -- 172.21.15.59:0/1253798916 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998845427 0 0) 0x7f09240b4cf0 con 0x7f09240cd8b0 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.729+0000 7f0923400640 1 -- 172.21.15.59:0/1253798916 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0908003520 con 0x7f09240cd8b0 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.730+0000 7f0923400640 1 -- 172.21.15.59:0/1253798916 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2772121689 0 0) 0x7f09240ccdd0 con 0x7f092415c670 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.730+0000 7f0928a00640 1 -- 172.21.15.59:0/1253798916 >> v1:172.21.15.76:6789/0 conn(0x7f092415c670 legacy=0x7f09240b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.730+0000 7f0928a00640 1 -- 172.21.15.59:0/1253798916 >> v1:172.21.15.59:6789/0 conn(0x7f09240cd8b0 legacy=0x7f09240d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.730+0000 7f0928a00640 1 -- 172.21.15.59:0/1253798916 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09240cd0b0 con 0x7f09240d0cd0 2024-04-23T14:45:06.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.731+0000 7f0928a00640 1 --2- 172.21.15.59:0/1253798916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240d0cd0 0x7f09241674f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f091c009dc0 tx=0x7f091c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3a1f16dcb681b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:06.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.731+0000 7f0923400640 1 -- 172.21.15.59:0/1253798916 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c007740 con 0x7f09240d0cd0 2024-04-23T14:45:06.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0923400640 1 -- 172.21.15.59:0/1253798916 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c0078e0 con 0x7f09240d0cd0 2024-04-23T14:45:06.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0923400640 1 -- 172.21.15.59:0/1253798916 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c007c90 con 0x7f09240d0cd0 2024-04-23T14:45:06.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0929400640 1 -- 172.21.15.59:0/1253798916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240d0cd0 msgr2=0x7f09241674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0929400640 1 --2- 172.21.15.59:0/1253798916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240d0cd0 0x7f09241674f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f091c009dc0 tx=0x7f091c02a6d0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0929400640 1 -- 172.21.15.59:0/1253798916 shutdown_connections 2024-04-23T14:45:06.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0929400640 1 --2- 172.21.15.59:0/1253798916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240d0cd0 0x7f09241674f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0929400640 1 -- 172.21.15.59:0/1253798916 >> 172.21.15.59:0/1253798916 conn(0x7f092406bdf0 msgr2=0x7f092406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:06.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0929400640 1 -- 172.21.15.59:0/1253798916 shutdown_connections 2024-04-23T14:45:06.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.732+0000 7f0929400640 1 -- 172.21.15.59:0/1253798916 wait complete. 2024-04-23T14:45:06.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.734+0000 7f0929400640 1 Processor -- start 2024-04-23T14:45:06.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.734+0000 7f0929400640 1 -- start start 2024-04-23T14:45:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.735+0000 7f0929400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240cd8b0 0x7f09240cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.735+0000 7f0929400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09240d0cd0 0x7f09240c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.735+0000 7f0929400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f092415c670 0x7f092414e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.735+0000 7f0929400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09240b3e60 con 0x7f092415c670 2024-04-23T14:45:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.735+0000 7f0929400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f09240ccb80 con 0x7f09240d0cd0 2024-04-23T14:45:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.735+0000 7f0929400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f09240b3fe0 con 0x7f09240cd8b0 2024-04-23T14:45:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f092415c670 0x7f092414e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f092415c670 0x7f092414e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43564/0 (socket says 172.21.15.59:43564) 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 -- 172.21.15.59:0/292826745 learned_addr learned my addr 172.21.15.59:0/292826745 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f091be00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09240d0cd0 0x7f09240c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 -- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240cd8b0 msgr2=0x7f09240cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0923e00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240cd8b0 0x7f09240cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240cd8b0 0x7f09240cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 -- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09240d0cd0 msgr2=0x7f09240c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09240d0cd0 0x7f09240c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 -- 172.21.15.59:0/292826745 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09240ca8a0 con 0x7f092415c670 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f091be00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09240d0cd0 0x7f09240c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.736+0000 7f0928a00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f092415c670 0x7f092414e110 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f091c02acc0 tx=0x7f091c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.737+0000 7f0920c00640 1 -- 172.21.15.59:0/292826745 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c009450 con 0x7f092415c670 2024-04-23T14:45:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.737+0000 7f0920c00640 1 -- 172.21.15.59:0/292826745 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c004240 con 0x7f092415c670 2024-04-23T14:45:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.737+0000 7f0920c00640 1 -- 172.21.15.59:0/292826745 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c004610 con 0x7f092415c670 2024-04-23T14:45:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.737+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09240cab80 con 0x7f092415c670 2024-04-23T14:45:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.737+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092415b760 con 0x7f092415c670 2024-04-23T14:45:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.738+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09240af980 con 0x7f092415c670 2024-04-23T14:45:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.739+0000 7f0920c00640 1 -- 172.21.15.59:0/292826745 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f091c039040 con 0x7f092415c670 2024-04-23T14:45:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.740+0000 7f0920c00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f090006c8f0 0x7f090006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.740+0000 7f0920c00640 1 -- 172.21.15.59:0/292826745 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f091c0acff0 con 0x7f092415c670 2024-04-23T14:45:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.741+0000 7f0923e00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f090006c8f0 0x7f090006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.741+0000 7f0923e00640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f090006c8f0 0x7f090006edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0914000f60 tx=0x7f0914009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.745+0000 7f0920c00640 1 -- 172.21.15.59:0/292826745 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f091c040030 con 0x7f092415c670 2024-04-23T14:45:06.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.951+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f09240ceab0 con 0x7f090006c8f0 2024-04-23T14:45:06.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.952+0000 7f0920c00640 1 -- 172.21.15.59:0/292826745 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f09240ceab0 con 0x7f090006c8f0 2024-04-23T14:45:06.952 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 0 to deep-scrub 2024-04-23T14:45:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f090006c8f0 msgr2=0x7f090006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f090006c8f0 0x7f090006edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0914000f60 tx=0x7f0914009040 comp rx=0 tx=0).stop 2024-04-23T14:45:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f092415c670 msgr2=0x7f092414e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f092415c670 0x7f092414e110 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f091c02acc0 tx=0x7f091c004a40 comp rx=0 tx=0).stop 2024-04-23T14:45:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 shutdown_connections 2024-04-23T14:45:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f090006c8f0 0x7f090006edb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f092415c670 0x7f092414e110 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09240d0cd0 0x7f09240c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 --2- 172.21.15.59:0/292826745 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f09240cd8b0 0x7f09240cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 >> 172.21.15.59:0/292826745 conn(0x7f092406bdf0 msgr2=0x7f0924073690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:06.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 shutdown_connections 2024-04-23T14:45:06.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:06.955+0000 7f0929400640 1 -- 172.21.15.59:0/292826745 wait complete. 2024-04-23T14:45:06.974 INFO:tasks.ceph:Scrubbing osd.1 2024-04-23T14:45:06.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.191+0000 7f1519e00640 1 Processor -- start 2024-04-23T14:45:07.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.191+0000 7f1519e00640 1 -- start start 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1519e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f151406bdf0 0x7f151406c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1519e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1514157000 con 0x7f1514160100 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1519e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15140afb00 con 0x7f1514156c20 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1519e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f151415cc20 con 0x7f151406bdf0 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1513e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f151406bdf0 0x7f151406c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1513e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f151406bdf0 0x7f151406c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43570/0 (socket says 172.21.15.59:43570) 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1513e00640 1 -- 172.21.15.59:0/3552399352 learned_addr learned my addr 172.21.15.59:0/3552399352 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.193+0000 7f1519400640 1 --1- 172.21.15.59:0/3552399352 >> v1:172.21.15.59:6789/0 conn(0x7f1514160100 0x7f1514160520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42308/0 (socket says 172.21.15.59:42308) 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.194+0000 7f1513e00640 1 -- 172.21.15.59:0/3552399352 >> v1:172.21.15.76:6789/0 conn(0x7f1514156c20 legacy=0x7f15140aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:07.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.194+0000 7f1513e00640 1 -- 172.21.15.59:0/3552399352 >> v1:172.21.15.59:6789/0 conn(0x7f1514160100 legacy=0x7f1514160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.194+0000 7f1513e00640 1 -- 172.21.15.59:0/3552399352 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151415cf00 con 0x7f151406bdf0 2024-04-23T14:45:07.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.195+0000 7f1513e00640 1 --2- 172.21.15.59:0/3552399352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f151406bdf0 0x7f151406c270 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f1500009870 tx=0x7f150002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5068af79500f3eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.195+0000 7f1513400640 1 -- 172.21.15.59:0/3552399352 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500037070 con 0x7f151406bdf0 2024-04-23T14:45:07.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.195+0000 7f1513400640 1 -- 172.21.15.59:0/3552399352 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500032440 con 0x7f151406bdf0 2024-04-23T14:45:07.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.195+0000 7f1513400640 1 -- 172.21.15.59:0/3552399352 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500032810 con 0x7f151406bdf0 2024-04-23T14:45:07.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.196+0000 7f1519e00640 1 -- 172.21.15.59:0/3552399352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f151406bdf0 msgr2=0x7f151406c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.196+0000 7f1519e00640 1 --2- 172.21.15.59:0/3552399352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f151406bdf0 0x7f151406c270 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f1500009870 tx=0x7f150002a2a0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.196+0000 7f1519e00640 1 -- 172.21.15.59:0/3552399352 shutdown_connections 2024-04-23T14:45:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.196+0000 7f1519e00640 1 --2- 172.21.15.59:0/3552399352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f151406bdf0 0x7f151406c270 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.196+0000 7f1519e00640 1 -- 172.21.15.59:0/3552399352 >> 172.21.15.59:0/3552399352 conn(0x7f15140b2200 msgr2=0x7f15140b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.196+0000 7f1519e00640 1 -- 172.21.15.59:0/3552399352 shutdown_connections 2024-04-23T14:45:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.196+0000 7f1519e00640 1 -- 172.21.15.59:0/3552399352 wait complete. 2024-04-23T14:45:07.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.198+0000 7f1519e00640 1 Processor -- start 2024-04-23T14:45:07.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.198+0000 7f1519e00640 1 -- start start 2024-04-23T14:45:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.199+0000 7f1519e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f151406bdf0 0x7f15140caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.199+0000 7f1519e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1514156c20 0x7f151414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.199+0000 7f1519e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1514160100 0x7f15140c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.199+0000 7f1519e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f151415d9e0 con 0x7f1514160100 2024-04-23T14:45:07.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.199+0000 7f1519e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f151415d4c0 con 0x7f1514156c20 2024-04-23T14:45:07.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.199+0000 7f1519e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f1514073ba0 con 0x7f151406bdf0 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1514160100 0x7f15140c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1514160100 0x7f15140c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43586/0 (socket says 172.21.15.59:43586) 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 -- 172.21.15.59:0/1811836467 learned_addr learned my addr 172.21.15.59:0/1811836467 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1513e00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1514156c20 0x7f151414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1518a00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f151406bdf0 0x7f15140caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 -- 172.21.15.59:0/1811836467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f151406bdf0 msgr2=0x7f15140caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f151406bdf0 0x7f15140caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 -- 172.21.15.59:0/1811836467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1514156c20 msgr2=0x7f151414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1514156c20 0x7f151414a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 -- 172.21.15.59:0/1811836467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140cb560 con 0x7f1514160100 2024-04-23T14:45:07.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.200+0000 7f1519400640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1514160100 0x7f15140c9470 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c730 tx=0x7f150c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.201+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c017070 con 0x7f1514160100 2024-04-23T14:45:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.201+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c00ce40 con 0x7f1514160100 2024-04-23T14:45:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.201+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c00a7e0 con 0x7f1514160100 2024-04-23T14:45:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.201+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1514154460 con 0x7f1514160100 2024-04-23T14:45:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.201+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1514154a20 con 0x7f1514160100 2024-04-23T14:45:07.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.202+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f151404fd90 con 0x7f1514160100 2024-04-23T14:45:07.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.203+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f150c01a070 con 0x7f1514160100 2024-04-23T14:45:07.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.204+0000 7f1510c00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f14f406c920 0x7f14f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.204+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f150c08ae30 con 0x7f1514160100 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.204+0000 7f1518a00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f14f406c920 0x7f14f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.204+0000 7f1510c00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f14f4072920 0x7f14f4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.204+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14f40753f0 con 0x7f14f4072920 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.204+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f150c08d450 con 0x7f1514160100 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.205+0000 7f1513e00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f14f4072920 0x7f14f4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.205+0000 7f1518a00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f14f406c920 0x7f14f406ede0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1508004ea0 tx=0x7f1508009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.205+0000 7f1513e00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f14f4072920 0x7f14f4074d40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.206+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== osd.1 v2:172.21.15.59:6808/4218092099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14f40753f0 con 0x7f14f4072920 2024-04-23T14:45:07.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.221+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 --> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f15140b5090 con 0x7f14f4072920 2024-04-23T14:45:07.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.221+0000 7f1510c00640 1 -- 172.21.15.59:0/1811836467 <== osd.1 v2:172.21.15.59:6808/4218092099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f15140b5090 con 0x7f14f4072920 2024-04-23T14:45:07.222 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:07.222 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:07.222 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f14f4072920 msgr2=0x7f14f4074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6808/4218092099,v1:172.21.15.59:6809/4218092099] conn(0x7f14f4072920 0x7f14f4074d40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f14f406c920 msgr2=0x7f14f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f14f406c920 0x7f14f406ede0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1508004ea0 tx=0x7f1508009040 comp rx=0 tx=0).stop 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1514160100 msgr2=0x7f15140c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 --2- 172.21.15.59:0/1811836467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1514160100 0x7f15140c9470 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c730 tx=0x7f150c00cbf0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519400640 1 -- 172.21.15.59:0/1811836467 reap_dead start 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 shutdown_connections 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.222+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 >> 172.21.15.59:0/1811836467 conn(0x7f15140b2200 msgr2=0x7f15140729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.223+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 shutdown_connections 2024-04-23T14:45:07.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.223+0000 7f1519e00640 1 -- 172.21.15.59:0/1811836467 wait complete. 2024-04-23T14:45:07.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-23T14:45:07.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.450+0000 7f7d28c00640 1 Processor -- start 2024-04-23T14:45:07.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.450+0000 7f7d28c00640 1 -- start start 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d28c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d2415c670 0x7f7d240cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d28c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d240b4230 0x7f7d240b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d28c00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d240b4cf0 con 0x7f7d240d05c0 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d28c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7d2415ca50 con 0x7f7d240b4230 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d28c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7d240ccb80 con 0x7f7d2415c670 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d22a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d240b4230 0x7f7d240b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d22a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d240b4230 0x7f7d240b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43602/0 (socket says 172.21.15.59:43602) 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.452+0000 7f7d22a00640 1 -- 172.21.15.59:0/4218452914 learned_addr learned my addr 172.21.15.59:0/4218452914 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.453+0000 7f7d22a00640 1 -- 172.21.15.59:0/4218452914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d2415c670 msgr2=0x7f7d240cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.453+0000 7f7d22a00640 1 --2- 172.21.15.59:0/4218452914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d2415c670 0x7f7d240cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.453+0000 7f7d22a00640 1 -- 172.21.15.59:0/4218452914 >> v1:172.21.15.76:6789/0 conn(0x7f7d240d05c0 legacy=0x7f7d24160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:07.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.453+0000 7f7d22a00640 1 -- 172.21.15.59:0/4218452914 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d240ccdd0 con 0x7f7d240b4230 2024-04-23T14:45:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.454+0000 7f7d22a00640 1 --2- 172.21.15.59:0/4218452914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d240b4690 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009870 tx=0x7f7d1002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63b59c24c19e5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.454+0000 7f7d22000640 1 -- 172.21.15.59:0/4218452914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10037070 con 0x7f7d240b4230 2024-04-23T14:45:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.454+0000 7f7d22000640 1 -- 172.21.15.59:0/4218452914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d10032440 con 0x7f7d240b4230 2024-04-23T14:45:07.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.454+0000 7f7d22000640 1 -- 172.21.15.59:0/4218452914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10032810 con 0x7f7d240b4230 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 -- 172.21.15.59:0/4218452914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 msgr2=0x7f7d240b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4218452914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d240b4690 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009870 tx=0x7f7d1002a200 comp rx=0 tx=0).stop 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 -- 172.21.15.59:0/4218452914 shutdown_connections 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4218452914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d240b4690 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4218452914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d2415c670 0x7f7d240cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 -- 172.21.15.59:0/4218452914 >> 172.21.15.59:0/4218452914 conn(0x7f7d2406bdf0 msgr2=0x7f7d2406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 -- 172.21.15.59:0/4218452914 shutdown_connections 2024-04-23T14:45:07.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.455+0000 7f7d28c00640 1 -- 172.21.15.59:0/4218452914 wait complete. 2024-04-23T14:45:07.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.457+0000 7f7d28c00640 1 Processor -- start 2024-04-23T14:45:07.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.457+0000 7f7d28c00640 1 -- start start 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d28c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d2415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d28c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d240d05c0 0x7f7d24147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d28c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d2415c670 0x7f7d24147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d28c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d240b3e60 con 0x7f7d240b4230 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d28c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7d240cd390 con 0x7f7d2415c670 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d28c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7d240b3fe0 con 0x7f7d240d05c0 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d2415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d23400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d2415b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43618/0 (socket says 172.21.15.59:43618) 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d23400640 1 -- 172.21.15.59:0/4255568960 learned_addr learned my addr 172.21.15.59:0/4255568960 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d23e00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d2415c670 0x7f7d24147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d23400640 1 -- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d240d05c0 msgr2=0x7f7d24147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d22a00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d240d05c0 0x7f7d24147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d23400640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d240d05c0 0x7f7d24147e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.458+0000 7f7d23400640 1 -- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d2415c670 msgr2=0x7f7d24147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d23400640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d2415c670 0x7f7d24147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d23400640 1 -- 172.21.15.59:0/4255568960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d24145f90 con 0x7f7d240b4230 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d23e00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d2415c670 0x7f7d24147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d23400640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d2415b110 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400b410 tx=0x7f7d1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d1b400640 1 -- 172.21.15.59:0/4255568960 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14015020 con 0x7f7d240b4230 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d1b400640 1 -- 172.21.15.59:0/4255568960 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d14011850 con 0x7f7d240b4230 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d1b400640 1 -- 172.21.15.59:0/4255568960 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011c40 con 0x7f7d240b4230 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d241462d0 con 0x7f7d240b4230 2024-04-23T14:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.459+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d24146b10 con 0x7f7d240b4230 2024-04-23T14:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.461+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2404fd90 con 0x7f7d240b4230 2024-04-23T14:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.461+0000 7f7d1b400640 1 -- 172.21.15.59:0/4255568960 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d14018020 con 0x7f7d240b4230 2024-04-23T14:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.462+0000 7f7d1b400640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d0406c8f0 0x7f7d0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.462+0000 7f7d1b400640 1 -- 172.21.15.59:0/4255568960 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f7d1408b910 con 0x7f7d240b4230 2024-04-23T14:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.463+0000 7f7d22a00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d0406c8f0 0x7f7d0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.463+0000 7f7d22a00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d0406c8f0 0x7f7d0406edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7d10034000 tx=0x7f7d1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.467+0000 7f7d1b400640 1 -- 172.21.15.59:0/4255568960 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d1401f020 con 0x7f7d240b4230 2024-04-23T14:45:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.668+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f7d24146d50 con 0x7f7d0406c8f0 2024-04-23T14:45:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.669+0000 7f7d1b400640 1 -- 172.21.15.59:0/4255568960 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7d100326f0 con 0x7f7d0406c8f0 2024-04-23T14:45:07.669 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 1 to deep-scrub 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d0406c8f0 msgr2=0x7f7d0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d0406c8f0 0x7f7d0406edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7d10034000 tx=0x7f7d1002e000 comp rx=0 tx=0).stop 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 msgr2=0x7f7d2415b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d2415b110 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400b410 tx=0x7f7d1400b8d0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 shutdown_connections 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7d0406c8f0 0x7f7d0406edb0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d240d05c0 0x7f7d24147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d2415c670 0x7f7d24147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 --2- 172.21.15.59:0/4255568960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d240b4230 0x7f7d2415b110 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 >> 172.21.15.59:0/4255568960 conn(0x7f7d2406bdf0 msgr2=0x7f7d2415cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.671+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 shutdown_connections 2024-04-23T14:45:07.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.672+0000 7f7d28c00640 1 -- 172.21.15.59:0/4255568960 wait complete. 2024-04-23T14:45:07.691 INFO:tasks.ceph:Scrubbing osd.2 2024-04-23T14:45:07.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.905+0000 7fc4b6400640 1 Processor -- start 2024-04-23T14:45:07.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.905+0000 7fc4b6400640 1 -- start start 2024-04-23T14:45:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4b6400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b015f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4b6400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4b0073200 0x7fc4b006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4b6400640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b0071df0 con 0x7fc4b0073680 2024-04-23T14:45:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4b6400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b00720d0 con 0x7fc4b0073200 2024-04-23T14:45:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4b6400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b00723b0 con 0x7fc4b0159020 2024-04-23T14:45:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4b0073200 0x7fc4b006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4afe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4b0073200 0x7fc4b006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43632/0 (socket says 172.21.15.59:43632) 2024-04-23T14:45:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.907+0000 7fc4afe00640 1 -- 172.21.15.59:0/3171408210 learned_addr learned my addr 172.21.15.59:0/3171408210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.908+0000 7fc4b5000640 1 --2- 172.21.15.59:0/3171408210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.908+0000 7fc4afe00640 1 -- 172.21.15.59:0/3171408210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 msgr2=0x7fc4b015f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.908+0000 7fc4afe00640 1 --2- 172.21.15.59:0/3171408210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b015f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.908+0000 7fc4afe00640 1 -- 172.21.15.59:0/3171408210 >> v1:172.21.15.59:6789/0 conn(0x7fc4b0073680 legacy=0x7fc4b015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.908+0000 7fc4afe00640 1 -- 172.21.15.59:0/3171408210 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b0072690 con 0x7fc4b0073200 2024-04-23T14:45:07.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.908+0000 7fc4b5000640 1 --2- 172.21.15.59:0/3171408210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:07.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.909+0000 7fc4afe00640 1 --2- 172.21.15.59:0/3171408210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b006bdf0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c02c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae087a12a4ed808d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.909+0000 7fc4af400640 1 -- 172.21.15.59:0/3171408210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c038070 con 0x7fc4b0073200 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.909+0000 7fc4af400640 1 -- 172.21.15.59:0/3171408210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49c031710 con 0x7fc4b0073200 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.909+0000 7fc4af400640 1 -- 172.21.15.59:0/3171408210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c031ae0 con 0x7fc4b0073200 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 -- 172.21.15.59:0/3171408210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 msgr2=0x7fc4b006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 --2- 172.21.15.59:0/3171408210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b006bdf0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c02c460 comp rx=0 tx=0).stop 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 -- 172.21.15.59:0/3171408210 shutdown_connections 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 --2- 172.21.15.59:0/3171408210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b006bdf0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 --2- 172.21.15.59:0/3171408210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 -- 172.21.15.59:0/3171408210 >> 172.21.15.59:0/3171408210 conn(0x7fc4b00b2200 msgr2=0x7fc4b00b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 -- 172.21.15.59:0/3171408210 shutdown_connections 2024-04-23T14:45:07.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.910+0000 7fc4b6400640 1 -- 172.21.15.59:0/3171408210 wait complete. 2024-04-23T14:45:07.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.912+0000 7fc4b6400640 1 Processor -- start 2024-04-23T14:45:07.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.912+0000 7fc4b6400640 1 -- start start 2024-04-23T14:45:07.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b6400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b014d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b6400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4b0073680 0x7fc4b014b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b6400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b014b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b6400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0072970 con 0x7fc4b0073200 2024-04-23T14:45:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b6400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0159400 con 0x7fc4b0073680 2024-04-23T14:45:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b6400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b006c330 con 0x7fc4b0159020 2024-04-23T14:45:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b014d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b5000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b014d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43646/0 (socket says 172.21.15.59:43646) 2024-04-23T14:45:07.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.913+0000 7fc4b5000640 1 -- 172.21.15.59:0/3190805776 learned_addr learned my addr 172.21.15.59:0/3190805776 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:07.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5a00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b014b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4afe00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4b0073680 0x7fc4b014b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5000640 1 -- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 msgr2=0x7fc4b014b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5000640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b014b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5000640 1 -- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4b0073680 msgr2=0x7fc4b014b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5000640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4b0073680 0x7fc4b014b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5000640 1 -- 172.21.15.59:0/3190805776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b014df30 con 0x7fc4b0073200 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5a00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4b0159020 0x7fc4b014b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4afe00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4b0073680 0x7fc4b014b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4b5000640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b014d460 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400c820 tx=0x7fc4a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.914+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4015070 con 0x7fc4b0073200 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.915+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a4007a30 con 0x7fc4b0073200 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.915+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4007de0 con 0x7fc4b0073200 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.915+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b014e210 con 0x7fc4b0073200 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.915+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b014e7d0 con 0x7fc4b0073200 2024-04-23T14:45:07.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.916+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4b004fd90 con 0x7fc4b0073200 2024-04-23T14:45:07.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.917+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4a4018070 con 0x7fc4b0073200 2024-04-23T14:45:07.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.918+0000 7fc4acc00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc49006c8f0 0x7fc49006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.918+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fc4a408b800 con 0x7fc4b0073200 2024-04-23T14:45:07.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.918+0000 7fc4acc00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7fc4900728f0 0x7fc490074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:07.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.918+0000 7fc4afe00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc49006c8f0 0x7fc49006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.918+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4900753c0 con 0x7fc4900728f0 2024-04-23T14:45:07.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.918+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7fc4a4092050 con 0x7fc4b0073200 2024-04-23T14:45:07.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.918+0000 7fc4b5a00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7fc4900728f0 0x7fc490074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:07.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.919+0000 7fc4afe00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc49006c8f0 0x7fc49006edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc49c035000 tx=0x7fc49c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.919+0000 7fc4b5a00640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7fc4900728f0 0x7fc490074d10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:07.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.920+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== osd.2 v2:172.21.15.59:6816/77995666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4900753c0 con 0x7fc4900728f0 2024-04-23T14:45:07.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.935+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 --> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc4b006ea90 con 0x7fc4900728f0 2024-04-23T14:45:07.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.936+0000 7fc4acc00640 1 -- 172.21.15.59:0/3190805776 <== osd.2 v2:172.21.15.59:6816/77995666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc4b006ea90 con 0x7fc4900728f0 2024-04-23T14:45:07.936 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:07.936 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:07.936 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:07.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.936+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7fc4900728f0 msgr2=0x7fc490074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.936+0000 7fc4b6400640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6816/77995666,v1:172.21.15.59:6818/77995666] conn(0x7fc4900728f0 0x7fc490074d10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc49006c8f0 msgr2=0x7fc49006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fc49006c8f0 0x7fc49006edb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc49c035000 tx=0x7fc49c008030 comp rx=0 tx=0).stop 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 msgr2=0x7fc4b014d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 --2- 172.21.15.59:0/3190805776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b0073200 0x7fc4b014d460 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400c820 tx=0x7fc4a400cce0 comp rx=0 tx=0).stop 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b5a00640 1 -- 172.21.15.59:0/3190805776 reap_dead start 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 shutdown_connections 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 >> 172.21.15.59:0/3190805776 conn(0x7fc4b00b2200 msgr2=0x7fc4b00b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 shutdown_connections 2024-04-23T14:45:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:07.937+0000 7fc4b6400640 1 -- 172.21.15.59:0/3190805776 wait complete. 2024-04-23T14:45:07.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-23T14:45:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.166+0000 7f4c1ee00640 1 Processor -- start 2024-04-23T14:45:08.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.167+0000 7f4c1ee00640 1 -- start start 2024-04-23T14:45:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.168+0000 7f4c1ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c1815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.168+0000 7f4c1ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c1815dda0 con 0x7f4c1806ddf0 2024-04-23T14:45:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.168+0000 7f4c1ee00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c180cd880 con 0x7f4c1806e250 2024-04-23T14:45:08.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.168+0000 7f4c1ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4c18071ec0 con 0x7f4c180cf410 2024-04-23T14:45:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.168+0000 7f4c0f400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4c1806ddf0 0x7f4c18072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42334/0 (socket says 172.21.15.59:42334) 2024-04-23T14:45:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.168+0000 7f4c0f400640 1 -- 172.21.15.59:0/2360658774 learned_addr learned my addr 172.21.15.59:0/2360658774 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.169+0000 7f4c17e00640 1 --2- 172.21.15.59:0/2360658774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c1815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.169+0000 7f4c17400640 1 -- 172.21.15.59:0/2360658774 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120555081 0 0) 0x7f4c00001110 con 0x7f4c1806ddf0 2024-04-23T14:45:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.169+0000 7f4c17400640 1 -- 172.21.15.59:0/2360658774 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bfc003520 con 0x7f4c1806ddf0 2024-04-23T14:45:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.169+0000 7f4c17e00640 1 -- 172.21.15.59:0/2360658774 >> v1:172.21.15.76:6789/0 conn(0x7f4c1806e250 legacy=0x7f4c180cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.169+0000 7f4c17e00640 1 -- 172.21.15.59:0/2360658774 >> v1:172.21.15.59:6789/0 conn(0x7f4c1806ddf0 legacy=0x7f4c18072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.169+0000 7f4c17e00640 1 -- 172.21.15.59:0/2360658774 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c180cdb20 con 0x7f4c180cf410 2024-04-23T14:45:08.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.171+0000 7f4c17e00640 1 --2- 172.21.15.59:0/2360658774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c1815d860 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009df0 tx=0x7f4c0802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960d881ecc939b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.171+0000 7f4c17400640 1 -- 172.21.15.59:0/2360658774 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0802ae60 con 0x7f4c180cf410 2024-04-23T14:45:08.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.171+0000 7f4c17400640 1 -- 172.21.15.59:0/2360658774 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08007740 con 0x7f4c180cf410 2024-04-23T14:45:08.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.171+0000 7f4c17400640 1 -- 172.21.15.59:0/2360658774 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08007b10 con 0x7f4c180cf410 2024-04-23T14:45:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.172+0000 7f4c1ee00640 1 -- 172.21.15.59:0/2360658774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 msgr2=0x7f4c1815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.172+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/2360658774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c1815d860 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009df0 tx=0x7f4c0802a680 comp rx=0 tx=0).stop 2024-04-23T14:45:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.172+0000 7f4c1ee00640 1 -- 172.21.15.59:0/2360658774 shutdown_connections 2024-04-23T14:45:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.172+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/2360658774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c1815d860 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.172+0000 7f4c1ee00640 1 -- 172.21.15.59:0/2360658774 >> 172.21.15.59:0/2360658774 conn(0x7f4c180b2130 msgr2=0x7f4c180b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.172+0000 7f4c1ee00640 1 -- 172.21.15.59:0/2360658774 shutdown_connections 2024-04-23T14:45:08.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.172+0000 7f4c1ee00640 1 -- 172.21.15.59:0/2360658774 wait complete. 2024-04-23T14:45:08.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.174+0000 7f4c1ee00640 1 Processor -- start 2024-04-23T14:45:08.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.174+0000 7f4c1ee00640 1 -- start start 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c1ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1806ddf0 0x7f4c18159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c1ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1806e250 0x7f4c180adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c1ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c181f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c1ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1815a6a0 con 0x7f4c1806ddf0 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c1ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c180cde00 con 0x7f4c1806e250 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c1ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1815a820 con 0x7f4c180cf410 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1806ddf0 0x7f4c18159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c17e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1806ddf0 0x7f4c18159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43654/0 (socket says 172.21.15.59:43654) 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.175+0000 7f4c17e00640 1 -- 172.21.15.59:0/644620697 learned_addr learned my addr 172.21.15.59:0/644620697 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c17e00640 1 -- 172.21.15.59:0/644620697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 msgr2=0x7f4c181f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c17e00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c181f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c17e00640 1 -- 172.21.15.59:0/644620697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1806e250 msgr2=0x7f4c180adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c17e00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1806e250 0x7f4c180adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c17e00640 1 -- 172.21.15.59:0/644620697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c181f5d10 con 0x7f4c1806ddf0 2024-04-23T14:45:08.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c17e00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1806ddf0 0x7f4c18159450 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009f10 tx=0x7f4c08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c14c00640 1 -- 172.21.15.59:0/644620697 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08040070 con 0x7f4c1806ddf0 2024-04-23T14:45:08.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c14c00640 1 -- 172.21.15.59:0/644620697 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08007e40 con 0x7f4c1806ddf0 2024-04-23T14:45:08.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c18159ad0 con 0x7f4c1806ddf0 2024-04-23T14:45:08.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c14c00640 1 -- 172.21.15.59:0/644620697 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0802ae60 con 0x7f4c1806ddf0 2024-04-23T14:45:08.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.176+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c180b04c0 con 0x7f4c1806ddf0 2024-04-23T14:45:08.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.177+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c18068a80 con 0x7f4c1806ddf0 2024-04-23T14:45:08.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.179+0000 7f4c14c00640 1 -- 172.21.15.59:0/644620697 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c08039040 con 0x7f4c1806ddf0 2024-04-23T14:45:08.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.179+0000 7f4c14c00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.180+0000 7f4c14c00640 1 -- 172.21.15.59:0/644620697 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f4c080acfa0 con 0x7f4c1806ddf0 2024-04-23T14:45:08.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.180+0000 7f4c0f400640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.181+0000 7f4c0f400640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4c00007fd0 tx=0x7f4c00001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.184+0000 7f4c14c00640 1 -- 172.21.15.59:0/644620697 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c181f5d10 con 0x7f4c1806ddf0 2024-04-23T14:45:08.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.391+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f4c180700b0 con 0x7f4bf406c8f0 2024-04-23T14:45:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.391+0000 7f4c14c00640 1 -- 172.21.15.59:0/644620697 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4c180700b0 con 0x7f4bf406c8f0 2024-04-23T14:45:08.392 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 2 to deep-scrub 2024-04-23T14:45:08.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4bf406c8f0 msgr2=0x7f4bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4c00007fd0 tx=0x7f4c00001150 comp rx=0 tx=0).stop 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1806ddf0 msgr2=0x7f4c18159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1806ddf0 0x7f4c18159450 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009f10 tx=0x7f4c08004a40 comp rx=0 tx=0).stop 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 shutdown_connections 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c180cf410 0x7f4c181f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c1806e250 0x7f4c180adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 --2- 172.21.15.59:0/644620697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c1806ddf0 0x7f4c18159450 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 >> 172.21.15.59:0/644620697 conn(0x7f4c180b2130 msgr2=0x7f4c1815d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 shutdown_connections 2024-04-23T14:45:08.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.394+0000 7f4c1ee00640 1 -- 172.21.15.59:0/644620697 wait complete. 2024-04-23T14:45:08.412 INFO:tasks.ceph:Scrubbing osd.3 2024-04-23T14:45:08.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.627+0000 7f7b94a00640 1 Processor -- start 2024-04-23T14:45:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.628+0000 7f7b94a00640 1 -- start start 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b94a00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7b9015e990 0x7f7b900d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b94a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7b900d3c20 0x7f7b900d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b94a00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b900cfc70 con 0x7f7b900cec40 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b94a00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f7b900d0230 con 0x7f7b900d3c20 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b94a00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7b900d21c0 con 0x7f7b9015e990 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b8fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7b900d3c20 0x7f7b900d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b8fe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7b900d3c20 0x7f7b900d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43666/0 (socket says 172.21.15.59:43666) 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.629+0000 7f7b8fe00640 1 -- 172.21.15.59:0/3169078445 learned_addr learned my addr 172.21.15.59:0/3169078445 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.630+0000 7f7b8f400640 1 --2- 172.21.15.59:0/3169078445 >> v2:172.21.15.76:3300/0 conn(0x7f7b9015e990 0x7f7b900d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.630+0000 7f7b8fe00640 1 -- 172.21.15.59:0/3169078445 >> v2:172.21.15.76:3300/0 conn(0x7f7b9015e990 msgr2=0x7f7b900d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.630+0000 7f7b8fe00640 1 --2- 172.21.15.59:0/3169078445 >> v2:172.21.15.76:3300/0 conn(0x7f7b9015e990 0x7f7b900d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.630+0000 7f7b8fe00640 1 -- 172.21.15.59:0/3169078445 >> v1:172.21.15.76:6789/0 conn(0x7f7b900cec40 legacy=0x7f7b9006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.630+0000 7f7b8fe00640 1 -- 172.21.15.59:0/3169078445 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b900cff50 con 0x7f7b900d3c20 2024-04-23T14:45:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.630+0000 7f7b8f400640 1 --2- 172.21.15.59:0/3169078445 >> v2:172.21.15.76:3300/0 conn(0x7f7b9015e990 0x7f7b900d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.631+0000 7f7b8fe00640 1 --2- 172.21.15.59:0/3169078445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900d1540 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f7b900d2150 tx=0x7f7b8802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945cee4cf0d0acfc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.631+0000 7f7b8e000640 1 -- 172.21.15.59:0/3169078445 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88036070 con 0x7f7b900d3c20 2024-04-23T14:45:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.631+0000 7f7b8e000640 1 -- 172.21.15.59:0/3169078445 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88031600 con 0x7f7b900d3c20 2024-04-23T14:45:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.631+0000 7f7b8e000640 1 -- 172.21.15.59:0/3169078445 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b880319d0 con 0x7f7b900d3c20 2024-04-23T14:45:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 -- 172.21.15.59:0/3169078445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 msgr2=0x7f7b900d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 --2- 172.21.15.59:0/3169078445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900d1540 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f7b900d2150 tx=0x7f7b8802a6d0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 -- 172.21.15.59:0/3169078445 shutdown_connections 2024-04-23T14:45:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 --2- 172.21.15.59:0/3169078445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900d1540 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 --2- 172.21.15.59:0/3169078445 >> v2:172.21.15.76:3300/0 conn(0x7f7b9015e990 0x7f7b900d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 -- 172.21.15.59:0/3169078445 >> 172.21.15.59:0/3169078445 conn(0x7f7b9008bf90 msgr2=0x7f7b900d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 -- 172.21.15.59:0/3169078445 shutdown_connections 2024-04-23T14:45:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.632+0000 7f7b94a00640 1 -- 172.21.15.59:0/3169078445 wait complete. 2024-04-23T14:45:08.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.634+0000 7f7b94a00640 1 Processor -- start 2024-04-23T14:45:08.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.634+0000 7f7b94a00640 1 -- start start 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.635+0000 7f7b94a00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b900cec40 0x7f7b90157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.635+0000 7f7b94a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900b9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.635+0000 7f7b94a00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b9015e990 0x7f7b900b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.635+0000 7f7b94a00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9006e1f0 con 0x7f7b900d3c20 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.635+0000 7f7b94a00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b900d26f0 con 0x7f7b9015e990 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.635+0000 7f7b94a00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f7b900a1b20 con 0x7f7b900cec40 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.635+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900b9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900b9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43668/0 (socket says 172.21.15.59:43668) 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 -- 172.21.15.59:0/1016057411 learned_addr learned my addr 172.21.15.59:0/1016057411 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8fe00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b9015e990 0x7f7b900b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8f400640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b900cec40 0x7f7b90157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 -- 172.21.15.59:0/1016057411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b900cec40 msgr2=0x7f7b90157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b900cec40 0x7f7b90157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 -- 172.21.15.59:0/1016057411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b9015e990 msgr2=0x7f7b900b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b9015e990 0x7f7b900b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 -- 172.21.15.59:0/1016057411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b900a22e0 con 0x7f7b900d3c20 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8f400640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b900cec40 0x7f7b90157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.636+0000 7f7b8ea00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900b9980 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800c430 tx=0x7f7b7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.637+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78017070 con 0x7f7b900d3c20 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.637+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b78003640 con 0x7f7b900d3c20 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.637+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78003a30 con 0x7f7b900d3c20 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.637+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9009e400 con 0x7f7b900d3c20 2024-04-23T14:45:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.637+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9009e9c0 con 0x7f7b900d3c20 2024-04-23T14:45:08.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.638+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b9004fd90 con 0x7f7b900d3c20 2024-04-23T14:45:08.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.639+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b78003bd0 con 0x7f7b900d3c20 2024-04-23T14:45:08.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.640+0000 7f7b87400640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7b6c06c8f0 0x7f7b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.640+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f7b7808aba0 con 0x7f7b900d3c20 2024-04-23T14:45:08.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.640+0000 7f7b87400640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f7b6c0728f0 0x7f7b6c074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.640+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b7801ce50 con 0x7f7b6c0728f0 2024-04-23T14:45:08.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.640+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f7b7808f2a0 con 0x7f7b900d3c20 2024-04-23T14:45:08.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.640+0000 7f7b8f400640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7b6c06c8f0 0x7f7b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.641+0000 7f7b8fe00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f7b6c0728f0 0x7f7b6c074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.641+0000 7f7b8f400640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7b6c06c8f0 0x7f7b6c06edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.641+0000 7f7b8fe00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f7b6c0728f0 0x7f7b6c074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.642+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== osd.3 v2:172.21.15.59:6800/3205049728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b7801ce50 con 0x7f7b6c0728f0 2024-04-23T14:45:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.657+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 --> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7b9009e7a0 con 0x7f7b6c0728f0 2024-04-23T14:45:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.658+0000 7f7b87400640 1 -- 172.21.15.59:0/1016057411 <== osd.3 v2:172.21.15.59:6800/3205049728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7b9009e7a0 con 0x7f7b6c0728f0 2024-04-23T14:45:08.658 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:08.658 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:08.658 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.658+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f7b6c0728f0 msgr2=0x7f7b6c074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6800/3205049728,v1:172.21.15.59:6801/3205049728] conn(0x7f7b6c0728f0 0x7f7b6c074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7b6c06c8f0 msgr2=0x7f7b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f7b6c06c8f0 0x7f7b6c06edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).stop 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 msgr2=0x7f7b900b9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 --2- 172.21.15.59:0/1016057411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b900d3c20 0x7f7b900b9980 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800c430 tx=0x7f7b7800c8f0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b8fe00640 1 -- 172.21.15.59:0/1016057411 reap_dead start 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 shutdown_connections 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 >> 172.21.15.59:0/1016057411 conn(0x7f7b9008bf90 msgr2=0x7f7b900cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 shutdown_connections 2024-04-23T14:45:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.659+0000 7f7b94a00640 1 -- 172.21.15.59:0/1016057411 wait complete. 2024-04-23T14:45:08.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-23T14:45:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.884+0000 7f8353200640 1 Processor -- start 2024-04-23T14:45:08.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.884+0000 7f8353200640 1 -- start start 2024-04-23T14:45:08.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8353200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83540d05c0 0x7f83541416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8353200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f835415c250 0x7f83541555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8353200640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835415ba70 con 0x7f835415f060 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8353200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f8354141c20 con 0x7f835415c250 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8353200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f835415bdc0 con 0x7f83540d05c0 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8352800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f835415c250 0x7f83541555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8352800640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f835415c250 0x7f83541555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43672/0 (socket says 172.21.15.59:43672) 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.886+0000 7f8352800640 1 -- 172.21.15.59:0/1885993940 learned_addr learned my addr 172.21.15.59:0/1885993940 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.887+0000 7f8351e00640 1 --2- 172.21.15.59:0/1885993940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83540d05c0 0x7f83541416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.887+0000 7f8352800640 1 -- 172.21.15.59:0/1885993940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83540d05c0 msgr2=0x7f83541416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:08.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.887+0000 7f8352800640 1 --2- 172.21.15.59:0/1885993940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83540d05c0 0x7f83541416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.887+0000 7f8352800640 1 -- 172.21.15.59:0/1885993940 >> v1:172.21.15.76:6789/0 conn(0x7f835415f060 legacy=0x7f83540d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.887+0000 7f8352800640 1 -- 172.21.15.59:0/1885993940 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835415f800 con 0x7f835415c250 2024-04-23T14:45:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.888+0000 7f8352800640 1 --2- 172.21.15.59:0/1885993940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541555d0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f834c009e50 tx=0x7f834c02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756594d16a85f9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.888+0000 7f8351400640 1 -- 172.21.15.59:0/1885993940 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c039070 con 0x7f835415c250 2024-04-23T14:45:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.888+0000 7f8351400640 1 -- 172.21.15.59:0/1885993940 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834c034680 con 0x7f835415c250 2024-04-23T14:45:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.888+0000 7f8351400640 1 -- 172.21.15.59:0/1885993940 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c034a50 con 0x7f835415c250 2024-04-23T14:45:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 -- 172.21.15.59:0/1885993940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 msgr2=0x7f83541555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 --2- 172.21.15.59:0/1885993940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541555d0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f834c009e50 tx=0x7f834c02c8d0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 -- 172.21.15.59:0/1885993940 shutdown_connections 2024-04-23T14:45:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 --2- 172.21.15.59:0/1885993940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541555d0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 --2- 172.21.15.59:0/1885993940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83540d05c0 0x7f83541416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 -- 172.21.15.59:0/1885993940 >> 172.21.15.59:0/1885993940 conn(0x7f835406bdf0 msgr2=0x7f835406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 -- 172.21.15.59:0/1885993940 shutdown_connections 2024-04-23T14:45:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.889+0000 7f8353200640 1 -- 172.21.15.59:0/1885993940 wait complete. 2024-04-23T14:45:08.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.891+0000 7f8353200640 1 Processor -- start 2024-04-23T14:45:08.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.891+0000 7f8353200640 1 -- start start 2024-04-23T14:45:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8353200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83540d05c0 0x7f8354155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8353200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8353200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f835415f060 0x7f8354152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8353200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f835415f4c0 con 0x7f835415c250 2024-04-23T14:45:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8353200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f835415fdc0 con 0x7f83540d05c0 2024-04-23T14:45:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8353200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f835415c6b0 con 0x7f835415f060 2024-04-23T14:45:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8349400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8349400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43674/0 (socket says 172.21.15.59:43674) 2024-04-23T14:45:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8349400640 1 -- 172.21.15.59:0/4013026269 learned_addr learned my addr 172.21.15.59:0/4013026269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8351e00640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83540d05c0 0x7f8354155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.892+0000 7f8352800640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f835415f060 0x7f8354152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f8349400640 1 -- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f835415f060 msgr2=0x7f8354152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f8349400640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f835415f060 0x7f8354152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f8349400640 1 -- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83540d05c0 msgr2=0x7f8354155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f8349400640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83540d05c0 0x7f8354155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f8349400640 1 -- 172.21.15.59:0/4013026269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541543a0 con 0x7f835415c250 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f8349400640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541520e0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f833c00b3f0 tx=0x7f833c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f834aa00640 1 -- 172.21.15.59:0/4013026269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c015020 con 0x7f835415c250 2024-04-23T14:45:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f834aa00640 1 -- 172.21.15.59:0/4013026269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c011830 con 0x7f835415c250 2024-04-23T14:45:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f834aa00640 1 -- 172.21.15.59:0/4013026269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c011c20 con 0x7f835415c250 2024-04-23T14:45:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.893+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8354154680 con 0x7f835415c250 2024-04-23T14:45:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.894+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8354154c40 con 0x7f835415c250 2024-04-23T14:45:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.895+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835415d1e0 con 0x7f835415c250 2024-04-23T14:45:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.896+0000 7f834aa00640 1 -- 172.21.15.59:0/4013026269 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f833c018020 con 0x7f835415c250 2024-04-23T14:45:08.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.897+0000 7f834aa00640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f833006c8f0 0x7f833006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:08.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.897+0000 7f834aa00640 1 -- 172.21.15.59:0/4013026269 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f833c08b5b0 con 0x7f835415c250 2024-04-23T14:45:08.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.897+0000 7f8351e00640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f833006c8f0 0x7f833006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:08.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.898+0000 7f8351e00640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f833006c8f0 0x7f833006edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8344006fd0 tx=0x7f8344008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:08.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:08.901+0000 7f834aa00640 1 -- 172.21.15.59:0/4013026269 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f835415d1e0 con 0x7f835415c250 2024-04-23T14:45:09.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.110+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f8354155140 con 0x7f833006c8f0 2024-04-23T14:45:09.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.110+0000 7f834aa00640 1 -- 172.21.15.59:0/4013026269 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f834400b200 con 0x7f833006c8f0 2024-04-23T14:45:09.111 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 3 to deep-scrub 2024-04-23T14:45:09.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f833006c8f0 msgr2=0x7f833006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f833006c8f0 0x7f833006edb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8344006fd0 tx=0x7f8344008040 comp rx=0 tx=0).stop 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 msgr2=0x7f83541520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541520e0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f833c00b3f0 tx=0x7f833c00b8b0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 shutdown_connections 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f833006c8f0 0x7f833006edb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f835415f060 0x7f8354152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f835415c250 0x7f83541520e0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 --2- 172.21.15.59:0/4013026269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83540d05c0 0x7f8354155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.113+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 >> 172.21.15.59:0/4013026269 conn(0x7f835406bdf0 msgr2=0x7f835406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.114+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 shutdown_connections 2024-04-23T14:45:09.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.114+0000 7f8353200640 1 -- 172.21.15.59:0/4013026269 wait complete. 2024-04-23T14:45:09.133 INFO:tasks.ceph:Scrubbing osd.4 2024-04-23T14:45:09.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.343+0000 7f4928000640 1 Processor -- start 2024-04-23T14:45:09.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.343+0000 7f4928000640 1 -- start start 2024-04-23T14:45:09.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.345+0000 7f4928000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49201431b0 0x7f49200cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.345+0000 7f4928000640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49200cda30 con 0x7f492006ea10 2024-04-23T14:45:09.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.345+0000 7f4928000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49200cf340 con 0x7f49200cd610 2024-04-23T14:45:09.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.345+0000 7f4928000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f49200cf670 con 0x7f49201431b0 2024-04-23T14:45:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.345+0000 7f4926000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f492006ea10 0x7f49200cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42336/0 (socket says 172.21.15.59:42336) 2024-04-23T14:45:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.345+0000 7f4926000640 1 -- 172.21.15.59:0/2255566768 learned_addr learned my addr 172.21.15.59:0/2255566768 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.346+0000 7f4925600640 1 --2- 172.21.15.59:0/2255566768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49201431b0 0x7f49200cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.346+0000 7f491fe00640 1 -- 172.21.15.59:0/2255566768 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701644832 0 0) 0x7f49200cda30 con 0x7f492006ea10 2024-04-23T14:45:09.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.346+0000 7f491fe00640 1 -- 172.21.15.59:0/2255566768 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4908003520 con 0x7f492006ea10 2024-04-23T14:45:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.346+0000 7f491fe00640 1 -- 172.21.15.59:0/2255566768 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3897944494 0 0) 0x7f4908003520 con 0x7f492006ea10 2024-04-23T14:45:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.347+0000 7f4925600640 1 -- 172.21.15.59:0/2255566768 >> v1:172.21.15.76:6789/0 conn(0x7f49200cd610 legacy=0x7f492006e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.347+0000 7f4925600640 1 -- 172.21.15.59:0/2255566768 >> v1:172.21.15.59:6789/0 conn(0x7f492006ea10 legacy=0x7f49200cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.347+0000 7f4925600640 1 -- 172.21.15.59:0/2255566768 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492006ef10 con 0x7f49201431b0 2024-04-23T14:45:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.348+0000 7f4925600640 1 --2- 172.21.15.59:0/2255566768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49201431b0 0x7f49200cd0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49200cf510 tx=0x7f491402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9e9fca9da2200 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.348+0000 7f491fe00640 1 -- 172.21.15.59:0/2255566768 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914037070 con 0x7f49201431b0 2024-04-23T14:45:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.348+0000 7f491fe00640 1 -- 172.21.15.59:0/2255566768 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491402abc0 con 0x7f49201431b0 2024-04-23T14:45:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.348+0000 7f491fe00640 1 -- 172.21.15.59:0/2255566768 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180019f0 con 0x7f49201431b0 2024-04-23T14:45:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.349+0000 7f4928000640 1 -- 172.21.15.59:0/2255566768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49201431b0 msgr2=0x7f49200cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.349+0000 7f4928000640 1 --2- 172.21.15.59:0/2255566768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49201431b0 0x7f49200cd0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49200cf510 tx=0x7f491402a130 comp rx=0 tx=0).stop 2024-04-23T14:45:09.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.349+0000 7f4928000640 1 -- 172.21.15.59:0/2255566768 shutdown_connections 2024-04-23T14:45:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.349+0000 7f4928000640 1 --2- 172.21.15.59:0/2255566768 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49201431b0 0x7f49200cd0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.349+0000 7f4928000640 1 -- 172.21.15.59:0/2255566768 >> 172.21.15.59:0/2255566768 conn(0x7f492006b9d0 msgr2=0x7f492006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.349+0000 7f4928000640 1 -- 172.21.15.59:0/2255566768 shutdown_connections 2024-04-23T14:45:09.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.349+0000 7f4928000640 1 -- 172.21.15.59:0/2255566768 wait complete. 2024-04-23T14:45:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.351+0000 7f4928000640 1 Processor -- start 2024-04-23T14:45:09.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.351+0000 7f4928000640 1 -- start start 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.352+0000 7f4928000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492006ea10 0x7f49201598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.352+0000 7f4928000640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49200cd610 0x7f492015e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.352+0000 7f4928000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49201431b0 0x7f49201f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.352+0000 7f4928000640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49200d0210 con 0x7f492006ea10 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.352+0000 7f4928000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f492006f4d0 con 0x7f49201431b0 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.352+0000 7f4928000640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f49200d0390 con 0x7f49200cd610 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492006ea10 0x7f49201598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492006ea10 0x7f49201598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43684/0 (socket says 172.21.15.59:43684) 2024-04-23T14:45:09.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 -- 172.21.15.59:0/2004499533 learned_addr learned my addr 172.21.15.59:0/2004499533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:09.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4924c00640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49200cd610 0x7f492015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 -- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49200cd610 msgr2=0x7f492015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f49200cd610 0x7f492015e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4926000640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49201431b0 0x7f49201f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 -- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49201431b0 msgr2=0x7f49201f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49201431b0 0x7f49201f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4925600640 1 -- 172.21.15.59:0/2004499533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4920156f80 con 0x7f492006ea10 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.353+0000 7f4926000640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49201431b0 0x7f49201f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.354+0000 7f4925600640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492006ea10 0x7f49201598f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f4914009720 tx=0x7f49140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.354+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914041030 con 0x7f492006ea10 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.354+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491402ae30 con 0x7f492006ea10 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.354+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914037040 con 0x7f492006ea10 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.354+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49201f29a0 con 0x7f492006ea10 2024-04-23T14:45:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.354+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4920159e30 con 0x7f492006ea10 2024-04-23T14:45:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.355+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49201f3060 con 0x7f492006ea10 2024-04-23T14:45:09.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.356+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4914044030 con 0x7f492006ea10 2024-04-23T14:45:09.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.357+0000 7f491d600640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.357+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f49140ab9b0 con 0x7f492006ea10 2024-04-23T14:45:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.357+0000 7f491d600640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f49000728f0 0x7f4900074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.357+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49000753c0 con 0x7f49000728f0 2024-04-23T14:45:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.357+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f49140b02a0 con 0x7f492006ea10 2024-04-23T14:45:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.358+0000 7f4924c00640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.358+0000 7f4926000640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f49000728f0 0x7f4900074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.358+0000 7f4924c00640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f490006c8f0 0x7f490006edb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f49201574e0 tx=0x7f490c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.359+0000 7f4926000640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f49000728f0 0x7f4900074d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.361+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== osd.4 v2:172.21.15.76:6816/3407674963 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49000753c0 con 0x7f49000728f0 2024-04-23T14:45:09.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.376+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 --> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4920068a80 con 0x7f49000728f0 2024-04-23T14:45:09.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.377+0000 7f491d600640 1 -- 172.21.15.59:0/2004499533 <== osd.4 v2:172.21.15.76:6816/3407674963 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4920068a80 con 0x7f49000728f0 2024-04-23T14:45:09.377 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:09.377 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:09.377 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.377+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f49000728f0 msgr2=0x7f4900074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.377+0000 7f4928000640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.76:6816/3407674963,v1:172.21.15.76:6817/3407674963] conn(0x7f49000728f0 0x7f4900074d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.377+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f490006c8f0 msgr2=0x7f490006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.377+0000 7f4928000640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f490006c8f0 0x7f490006edb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f49201574e0 tx=0x7f490c009040 comp rx=0 tx=0).stop 2024-04-23T14:45:09.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.377+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492006ea10 msgr2=0x7f49201598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.377+0000 7f4928000640 1 --2- 172.21.15.59:0/2004499533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492006ea10 0x7f49201598f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f4914009720 tx=0x7f49140046c0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.378+0000 7f4926000640 1 -- 172.21.15.59:0/2004499533 reap_dead start 2024-04-23T14:45:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.378+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 shutdown_connections 2024-04-23T14:45:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.378+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 >> 172.21.15.59:0/2004499533 conn(0x7f492006b9d0 msgr2=0x7f49201606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.378+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 shutdown_connections 2024-04-23T14:45:09.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.378+0000 7f4928000640 1 -- 172.21.15.59:0/2004499533 wait complete. 2024-04-23T14:45:09.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-23T14:45:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.610+0000 7f1393e00640 1 Processor -- start 2024-04-23T14:45:09.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.610+0000 7f1393e00640 1 -- start start 2024-04-23T14:45:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.611+0000 7f1393e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f138c161e60 0x7f138c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1393e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138c1632e0 con 0x7f138c162de0 2024-04-23T14:45:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1393e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138c0adcd0 con 0x7f138c0b44c0 2024-04-23T14:45:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1393e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f138c0aefd0 con 0x7f138c161e60 2024-04-23T14:45:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1391400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f138c161e60 0x7f138c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1390a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f138c162de0 0x7f138c0b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:42344/0 (socket says 172.21.15.59:42344) 2024-04-23T14:45:09.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1390a00640 1 -- 172.21.15.59:0/853625507 learned_addr learned my addr 172.21.15.59:0/853625507 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1391400640 1 -- 172.21.15.59:0/853625507 >> v1:172.21.15.76:6789/0 conn(0x7f138c0b44c0 legacy=0x7f138c06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1391400640 1 -- 172.21.15.59:0/853625507 >> v1:172.21.15.59:6789/0 conn(0x7f138c162de0 legacy=0x7f138c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f1391400640 1 -- 172.21.15.59:0/853625507 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c0adfb0 con 0x7f138c161e60 2024-04-23T14:45:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.612+0000 7f138be00640 1 -- 172.21.15.59:0/853625507 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50725774 0 0) 0x7f138c1632e0 con 0x7f138c162de0 2024-04-23T14:45:09.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.614+0000 7f1391400640 1 --2- 172.21.15.59:0/853625507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c161e60 0x7f138c0aea90 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f13800097a0 tx=0x7f138002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ed8dcf3041236 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.614+0000 7f138be00640 1 -- 172.21.15.59:0/853625507 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380037070 con 0x7f138c161e60 2024-04-23T14:45:09.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.614+0000 7f138be00640 1 -- 172.21.15.59:0/853625507 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138002ae50 con 0x7f138c161e60 2024-04-23T14:45:09.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.614+0000 7f138be00640 1 -- 172.21.15.59:0/853625507 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380032660 con 0x7f138c161e60 2024-04-23T14:45:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.614+0000 7f1393e00640 1 -- 172.21.15.59:0/853625507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c161e60 msgr2=0x7f138c0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.614+0000 7f1393e00640 1 --2- 172.21.15.59:0/853625507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c161e60 0x7f138c0aea90 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f13800097a0 tx=0x7f138002a140 comp rx=0 tx=0).stop 2024-04-23T14:45:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.615+0000 7f1393e00640 1 -- 172.21.15.59:0/853625507 shutdown_connections 2024-04-23T14:45:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.615+0000 7f1393e00640 1 --2- 172.21.15.59:0/853625507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c161e60 0x7f138c0aea90 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.615+0000 7f1393e00640 1 -- 172.21.15.59:0/853625507 >> 172.21.15.59:0/853625507 conn(0x7f138c0b2130 msgr2=0x7f138c0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.615+0000 7f1393e00640 1 -- 172.21.15.59:0/853625507 shutdown_connections 2024-04-23T14:45:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.615+0000 7f1393e00640 1 -- 172.21.15.59:0/853625507 wait complete. 2024-04-23T14:45:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.616+0000 7f1393e00640 1 Processor -- start 2024-04-23T14:45:09.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.617+0000 7f1393e00640 1 -- start start 2024-04-23T14:45:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1393e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c0b44c0 0x7f138c14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1393e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138c161e60 0x7f138c152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1393e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c162de0 0x7f138c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1393e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0af480 con 0x7f138c0b44c0 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1393e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0ae570 con 0x7f138c162de0 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1393e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f138c0af600 con 0x7f138c161e60 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1391400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c0b44c0 0x7f138c14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1391400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c0b44c0 0x7f138c14c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43700/0 (socket says 172.21.15.59:43700) 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1391400640 1 -- 172.21.15.59:0/3735382349 learned_addr learned my addr 172.21.15.59:0/3735382349 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1391e00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c162de0 0x7f138c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1390a00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138c161e60 0x7f138c152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1391400640 1 -- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138c161e60 msgr2=0x7f138c152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.618+0000 7f1391400640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138c161e60 0x7f138c152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1391400640 1 -- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c162de0 msgr2=0x7f138c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1391400640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c162de0 0x7f138c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1391400640 1 -- 172.21.15.59:0/3735382349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c14cea0 con 0x7f138c0b44c0 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1390a00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138c161e60 0x7f138c152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1391400640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c0b44c0 0x7f138c14c960 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f13800097a0 tx=0x7f13800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1389600640 1 -- 172.21.15.59:0/3735382349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380037040 con 0x7f138c0b44c0 2024-04-23T14:45:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1389600640 1 -- 172.21.15.59:0/3735382349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1380004030 con 0x7f138c0b44c0 2024-04-23T14:45:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1389600640 1 -- 172.21.15.59:0/3735382349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380006e10 con 0x7f138c0b44c0 2024-04-23T14:45:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c153610 con 0x7f138c0b44c0 2024-04-23T14:45:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.619+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c153bd0 con 0x7f138c0b44c0 2024-04-23T14:45:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.621+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c068a80 con 0x7f138c0b44c0 2024-04-23T14:45:09.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.622+0000 7f1389600640 1 -- 172.21.15.59:0/3735382349 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f138003d650 con 0x7f138c0b44c0 2024-04-23T14:45:09.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.623+0000 7f1389600640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f137006c940 0x7f137006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.623+0000 7f1389600640 1 -- 172.21.15.59:0/3735382349 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f13800ac1b0 con 0x7f138c0b44c0 2024-04-23T14:45:09.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.623+0000 7f1390a00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f137006c940 0x7f137006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.624+0000 7f1390a00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f137006c940 0x7f137006ee00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f138c150bf0 tx=0x7f137c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.627+0000 7f1389600640 1 -- 172.21.15.59:0/3735382349 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f138c14cea0 con 0x7f138c0b44c0 2024-04-23T14:45:09.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.832+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f138c0b4190 con 0x7f137006c940 2024-04-23T14:45:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.833+0000 7f1389600640 1 -- 172.21.15.59:0/3735382349 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f138c0b4190 con 0x7f137006c940 2024-04-23T14:45:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 4 to deep-scrub 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.835+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f137006c940 msgr2=0x7f137006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.835+0000 7f1393e00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f137006c940 0x7f137006ee00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f138c150bf0 tx=0x7f137c009040 comp rx=0 tx=0).stop 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.835+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c0b44c0 msgr2=0x7f138c14c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.835+0000 7f1393e00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c0b44c0 0x7f138c14c960 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f13800097a0 tx=0x7f13800046c0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.835+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 shutdown_connections 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.836+0000 7f1393e00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f137006c940 0x7f137006ee00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.836+0000 7f1393e00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138c161e60 0x7f138c152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.836+0000 7f1393e00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138c162de0 0x7f138c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.836+0000 7f1393e00640 1 --2- 172.21.15.59:0/3735382349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138c0b44c0 0x7f138c14c960 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.836+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 >> 172.21.15.59:0/3735382349 conn(0x7f138c0b2130 msgr2=0x7f138c06e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.836+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 shutdown_connections 2024-04-23T14:45:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:09.836+0000 7f1393e00640 1 -- 172.21.15.59:0/3735382349 wait complete. 2024-04-23T14:45:09.855 INFO:tasks.ceph:Scrubbing osd.5 2024-04-23T14:45:09.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:10.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.070+0000 7f983ce00640 1 Processor -- start 2024-04-23T14:45:10.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.071+0000 7f983ce00640 1 -- start start 2024-04-23T14:45:10.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.072+0000 7f983ce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 0x7f983806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.072+0000 7f983ce00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f983806b9d0 0x7f983806be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.072+0000 7f983ce00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98380b61d0 con 0x7f983809aa00 2024-04-23T14:45:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.072+0000 7f983ce00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f98380b6790 con 0x7f983806b9d0 2024-04-23T14:45:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.072+0000 7f983ce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f98380b64b0 con 0x7f9838161d80 2024-04-23T14:45:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.073+0000 7f982fe00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f983809aa00 0x7f983806f2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52340/0 (socket says 172.21.15.59:52340) 2024-04-23T14:45:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.073+0000 7f982fe00640 1 -- 172.21.15.59:0/3263908361 learned_addr learned my addr 172.21.15.59:0/3263908361 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.073+0000 7f9837400640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 0x7f983806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.073+0000 7f9837e00640 1 --2- 172.21.15.59:0/3263908361 >> v2:172.21.15.76:3300/0 conn(0x7f983806b9d0 0x7f983806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.073+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261428938 0 0) 0x7f98380b61d0 con 0x7f983809aa00 2024-04-23T14:45:10.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.073+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f981c003520 con 0x7f983809aa00 2024-04-23T14:45:10.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1789097903 0 0) 0x7f981c003520 con 0x7f983809aa00 2024-04-23T14:45:10.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 msgr2=0x7f983806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 0x7f983806edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 >> v2:172.21.15.76:3300/0 conn(0x7f983806b9d0 msgr2=0x7f983806be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 --2- 172.21.15.59:0/3263908361 >> v2:172.21.15.76:3300/0 conn(0x7f983806b9d0 0x7f983806be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9838070750 con 0x7f983809aa00 2024-04-23T14:45:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7f9820001670 con 0x7f983809aa00 2024-04-23T14:45:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 >> v1:172.21.15.59:6789/0 conn(0x7f983809aa00 legacy=0x7f983806f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 0x7f981c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981c003f80 0x7f981c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f981c004d40 0x7f981c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98380b6790 con 0x7f981c004d40 2024-04-23T14:45:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f98380b64b0 con 0x7f981c003f80 2024-04-23T14:45:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f981c00b9c0 con 0x7f9838161d80 2024-04-23T14:45:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.074+0000 7f9837e00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f981c004d40 0x7f981c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.075+0000 7f982fe00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981c003f80 0x7f981c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.075+0000 7f9837400640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 0x7f981c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.075+0000 7f9837e00640 1 -- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 msgr2=0x7f981c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.075+0000 7f9837e00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838161d80 0x7f981c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.075+0000 7f9837e00640 1 -- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981c003f80 msgr2=0x7f981c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.075+0000 7f9837e00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f981c003f80 0x7f981c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.075+0000 7f9837e00640 1 -- 172.21.15.59:0/3263908361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c003520 con 0x7f981c004d40 2024-04-23T14:45:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.076+0000 7f9837e00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f981c004d40 0x7f981c0051c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f983000e170 tx=0x7f981c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497455c0c3839d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.076+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830034020 con 0x7f981c004d40 2024-04-23T14:45:10.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.076+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9830030430 con 0x7f981c004d40 2024-04-23T14:45:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.076+0000 7f9836a00640 1 -- 172.21.15.59:0/3263908361 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830009c10 con 0x7f981c004d40 2024-04-23T14:45:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.077+0000 7f983ce00640 1 -- 172.21.15.59:0/3263908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f981c004d40 msgr2=0x7f981c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.077+0000 7f983ce00640 1 --2- 172.21.15.59:0/3263908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f981c004d40 0x7f981c0051c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f983000e170 tx=0x7f981c005830 comp rx=0 tx=0).stop 2024-04-23T14:45:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.077+0000 7f9837e00640 1 -- 172.21.15.59:0/3263908361 reap_dead start 2024-04-23T14:45:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.077+0000 7f983ce00640 1 -- 172.21.15.59:0/3263908361 shutdown_connections 2024-04-23T14:45:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.077+0000 7f983ce00640 1 -- 172.21.15.59:0/3263908361 >> 172.21.15.59:0/3263908361 conn(0x7f9838073250 msgr2=0x7f98380b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.077+0000 7f983ce00640 1 -- 172.21.15.59:0/3263908361 shutdown_connections 2024-04-23T14:45:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.077+0000 7f983ce00640 1 -- 172.21.15.59:0/3263908361 wait complete. 2024-04-23T14:45:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.080+0000 7f983ce00640 1 Processor -- start 2024-04-23T14:45:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.080+0000 7f983ce00640 1 -- start start 2024-04-23T14:45:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f983ce00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98381520f0 0x7f9838155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f983ce00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838152e40 0x7f9838155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f983ce00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9838152880 0x7f98381572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f983ce00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9838162160 con 0x7f98381520f0 2024-04-23T14:45:10.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f983ce00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9838070060 con 0x7f9838152880 2024-04-23T14:45:10.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f983ce00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f98380ae210 con 0x7f9838152e40 2024-04-23T14:45:10.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98381520f0 0x7f9838155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98381520f0 0x7f9838155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35238/0 (socket says 172.21.15.59:35238) 2024-04-23T14:45:10.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 -- 172.21.15.59:0/1371775343 learned_addr learned my addr 172.21.15.59:0/1371775343 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:10.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 -- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838152e40 msgr2=0x7f9838155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9838152e40 0x7f9838155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 -- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9838152880 msgr2=0x7f98381572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9838152880 0x7f98381572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.081+0000 7f9837400640 1 -- 172.21.15.59:0/1371775343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9838154d00 con 0x7f98381520f0 2024-04-23T14:45:10.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.082+0000 7f9837400640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98381520f0 0x7f9838155150 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f9828010860 tx=0x7f9828010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.082+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982800f040 con 0x7f98381520f0 2024-04-23T14:45:10.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.082+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982800ea80 con 0x7f98381520f0 2024-04-23T14:45:10.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.082+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982800ee50 con 0x7f98381520f0 2024-04-23T14:45:10.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.082+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381e8ca0 con 0x7f98381520f0 2024-04-23T14:45:10.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.082+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381e9260 con 0x7f98381520f0 2024-04-23T14:45:10.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.083+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f983804fd90 con 0x7f98381520f0 2024-04-23T14:45:10.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.085+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f982801c400 con 0x7f98381520f0 2024-04-23T14:45:10.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.085+0000 7f982f400640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f981406c920 0x7f981406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.086+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f982808bf60 con 0x7f98381520f0 2024-04-23T14:45:10.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.086+0000 7f982f400640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7f9814072920 0x7f9814074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.086+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9828008eb0 con 0x7f9814072920 2024-04-23T14:45:10.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.086+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f982808e580 con 0x7f98381520f0 2024-04-23T14:45:10.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.086+0000 7f982fe00640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f981406c920 0x7f981406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.086+0000 7f9837e00640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7f9814072920 0x7f9814074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.086+0000 7f982fe00640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f981406c920 0x7f981406ede0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f98380cccd0 tx=0x7f9820000f70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.087+0000 7f9837e00640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7f9814072920 0x7f9814074d40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.089+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== osd.5 v2:172.21.15.76:6808/2782996536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9828008eb0 con 0x7f9814072920 2024-04-23T14:45:10.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.104+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 --> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f983806eb20 con 0x7f9814072920 2024-04-23T14:45:10.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.105+0000 7f982f400640 1 -- 172.21.15.59:0/1371775343 <== osd.5 v2:172.21.15.76:6808/2782996536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f983806eb20 con 0x7f9814072920 2024-04-23T14:45:10.105 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:10.106 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:10.106 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:10.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.105+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7f9814072920 msgr2=0x7f9814074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.105+0000 7f983ce00640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.76:6808/2782996536,v1:172.21.15.76:6809/2782996536] conn(0x7f9814072920 0x7f9814074d40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f981406c920 msgr2=0x7f981406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f981406c920 0x7f981406ede0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f98380cccd0 tx=0x7f9820000f70 comp rx=0 tx=0).stop 2024-04-23T14:45:10.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98381520f0 msgr2=0x7f9838155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 --2- 172.21.15.59:0/1371775343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98381520f0 0x7f9838155150 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f9828010860 tx=0x7f9828010d20 comp rx=0 tx=0).stop 2024-04-23T14:45:10.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f9837e00640 1 -- 172.21.15.59:0/1371775343 reap_dead start 2024-04-23T14:45:10.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 shutdown_connections 2024-04-23T14:45:10.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 >> 172.21.15.59:0/1371775343 conn(0x7f9838073250 msgr2=0x7f98380cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:10.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 shutdown_connections 2024-04-23T14:45:10.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.106+0000 7f983ce00640 1 -- 172.21.15.59:0/1371775343 wait complete. 2024-04-23T14:45:10.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-23T14:45:10.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.341+0000 7fed8ee00640 1 Processor -- start 2024-04-23T14:45:10.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.341+0000 7fed8ee00640 1 -- start start 2024-04-23T14:45:10.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.342+0000 7fed8ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed881416e0 0x7fed8815f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.343+0000 7fed8ee00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8815ba70 con 0x7fed880d05c0 2024-04-23T14:45:10.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.343+0000 7fed8ee00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8815f800 con 0x7fed8815c250 2024-04-23T14:45:10.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.343+0000 7fed8ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fed8815bdc0 con 0x7fed881416e0 2024-04-23T14:45:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.343+0000 7fed8da00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fed880d05c0 0x7fed880d09a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52350/0 (socket says 172.21.15.59:52350) 2024-04-23T14:45:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.343+0000 7fed8da00640 1 -- 172.21.15.59:0/240503417 learned_addr learned my addr 172.21.15.59:0/240503417 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.343+0000 7fed8d000640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed881416e0 0x7fed8815f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136805410 0 0) 0x7fed8815ba70 con 0x7fed880d05c0 2024-04-23T14:45:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed70005730 con 0x7fed880d05c0 2024-04-23T14:45:10.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3931873328 0 0) 0x7fed70005730 con 0x7fed880d05c0 2024-04-23T14:45:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed881416e0 msgr2=0x7fed8815f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed881416e0 0x7fed8815f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 >> v1:172.21.15.76:6789/0 conn(0x7fed8815c250 legacy=0x7fed881555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8815fae0 con 0x7fed880d05c0 2024-04-23T14:45:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fed7c001670 con 0x7fed880d05c0 2024-04-23T14:45:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 >> v1:172.21.15.59:6789/0 conn(0x7fed880d05c0 legacy=0x7fed880d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed70005bf0 0x7fed70006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed70006a10 0x7fed70006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed881416e0 0x7fed700073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed8815bdc0 con 0x7fed70006a10 2024-04-23T14:45:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fed7000dd00 con 0x7fed70005bf0 2024-04-23T14:45:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.344+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fed70003520 con 0x7fed881416e0 2024-04-23T14:45:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.345+0000 7fed8d000640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed70006a10 0x7fed70006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.345+0000 7fed8da00640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed70005bf0 0x7fed70006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.345+0000 7fed8d000640 1 -- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed881416e0 msgr2=0x7fed700073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.345+0000 7fed8d000640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed881416e0 0x7fed700073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.345+0000 7fed8d000640 1 -- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed70005bf0 msgr2=0x7fed70006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.345+0000 7fed8d000640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed70005bf0 0x7fed70006030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.345+0000 7fed8d000640 1 -- 172.21.15.59:0/240503417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed70005730 con 0x7fed70006a10 2024-04-23T14:45:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.346+0000 7fed8d000640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed70006a10 0x7fed70006e90 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fed74009ac0 tx=0x7fed70007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1762abd3c07380ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.346+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74036070 con 0x7fed70006a10 2024-04-23T14:45:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.346+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed74031400 con 0x7fed70006a10 2024-04-23T14:45:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.346+0000 7fed87e00640 1 -- 172.21.15.59:0/240503417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74009690 con 0x7fed70006a10 2024-04-23T14:45:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.347+0000 7fed8ee00640 1 -- 172.21.15.59:0/240503417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed70006a10 msgr2=0x7fed70006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.347+0000 7fed8ee00640 1 --2- 172.21.15.59:0/240503417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed70006a10 0x7fed70006e90 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fed74009ac0 tx=0x7fed70007a40 comp rx=0 tx=0).stop 2024-04-23T14:45:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.347+0000 7fed8e400640 1 -- 172.21.15.59:0/240503417 reap_dead start 2024-04-23T14:45:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.347+0000 7fed8ee00640 1 -- 172.21.15.59:0/240503417 shutdown_connections 2024-04-23T14:45:10.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.347+0000 7fed8ee00640 1 -- 172.21.15.59:0/240503417 >> 172.21.15.59:0/240503417 conn(0x7fed8806bdf0 msgr2=0x7fed8806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:10.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.347+0000 7fed8ee00640 1 -- 172.21.15.59:0/240503417 shutdown_connections 2024-04-23T14:45:10.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.347+0000 7fed8ee00640 1 -- 172.21.15.59:0/240503417 wait complete. 2024-04-23T14:45:10.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.349+0000 7fed8ee00640 1 Processor -- start 2024-04-23T14:45:10.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.349+0000 7fed8ee00640 1 -- start start 2024-04-23T14:45:10.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.350+0000 7fed8ee00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed70006a10 0x7fed8809d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.350+0000 7fed8ee00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed881425f0 0x7fed88143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.350+0000 7fed8ee00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed88142a70 0x7fed88143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.350+0000 7fed8ee00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed88155ce0 con 0x7fed88142a70 2024-04-23T14:45:10.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.350+0000 7fed8ee00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fed88141b40 con 0x7fed881425f0 2024-04-23T14:45:10.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.350+0000 7fed8ee00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fed88155e60 con 0x7fed70006a10 2024-04-23T14:45:10.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed88142a70 0x7fed88143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed88142a70 0x7fed88143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35252/0 (socket says 172.21.15.59:35252) 2024-04-23T14:45:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 -- 172.21.15.59:0/146935658 learned_addr learned my addr 172.21.15.59:0/146935658 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8d000640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed881425f0 0x7fed88143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8da00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed70006a10 0x7fed8809d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 -- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed70006a10 msgr2=0x7fed8809d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed70006a10 0x7fed8809d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 -- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed881425f0 msgr2=0x7fed88143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed881425f0 0x7fed88143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 -- 172.21.15.59:0/146935658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8809c5f0 con 0x7fed88142a70 2024-04-23T14:45:10.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8da00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed70006a10 0x7fed8809d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:10.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.351+0000 7fed8e400640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed88142a70 0x7fed88143540 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fed8815bd30 tx=0x7fed8000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.352+0000 7fed85600640 1 -- 172.21.15.59:0/146935658 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80017070 con 0x7fed88142a70 2024-04-23T14:45:10.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.352+0000 7fed85600640 1 -- 172.21.15.59:0/146935658 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed80005d60 con 0x7fed88142a70 2024-04-23T14:45:10.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.352+0000 7fed85600640 1 -- 172.21.15.59:0/146935658 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8000a850 con 0x7fed88142a70 2024-04-23T14:45:10.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.352+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8809c8d0 con 0x7fed88142a70 2024-04-23T14:45:10.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.352+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8809ce90 con 0x7fed88142a70 2024-04-23T14:45:10.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.353+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8804fd90 con 0x7fed88142a70 2024-04-23T14:45:10.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.354+0000 7fed85600640 1 -- 172.21.15.59:0/146935658 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed8001a070 con 0x7fed88142a70 2024-04-23T14:45:10.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.355+0000 7fed85600640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed6806c8f0 0x7fed6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.355+0000 7fed85600640 1 -- 172.21.15.59:0/146935658 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fed8008a9f0 con 0x7fed88142a70 2024-04-23T14:45:10.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.355+0000 7fed8da00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed6806c8f0 0x7fed6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.356+0000 7fed8da00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed6806c8f0 0x7fed6806edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007970 tx=0x7fed7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.360+0000 7fed85600640 1 -- 172.21.15.59:0/146935658 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed80058fd0 con 0x7fed88142a70 2024-04-23T14:45:10.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.563+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fed880cdb10 con 0x7fed6806c8f0 2024-04-23T14:45:10.564 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 5 to deep-scrub 2024-04-23T14:45:10.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.564+0000 7fed85600640 1 -- 172.21.15.59:0/146935658 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fed880cdb10 con 0x7fed6806c8f0 2024-04-23T14:45:10.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.566+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed6806c8f0 msgr2=0x7fed6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.566+0000 7fed8ee00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed6806c8f0 0x7fed6806edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007970 tx=0x7fed7c009040 comp rx=0 tx=0).stop 2024-04-23T14:45:10.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.566+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed88142a70 msgr2=0x7fed88143540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.566+0000 7fed8ee00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed88142a70 0x7fed88143540 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fed8815bd30 tx=0x7fed8000cb90 comp rx=0 tx=0).stop 2024-04-23T14:45:10.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 shutdown_connections 2024-04-23T14:45:10.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fed6806c8f0 0x7fed6806edb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed88142a70 0x7fed88143540 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed881425f0 0x7fed88143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 --2- 172.21.15.59:0/146935658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed70006a10 0x7fed8809d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 >> 172.21.15.59:0/146935658 conn(0x7fed8806bdf0 msgr2=0x7fed8815cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:10.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 shutdown_connections 2024-04-23T14:45:10.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.567+0000 7fed8ee00640 1 -- 172.21.15.59:0/146935658 wait complete. 2024-04-23T14:45:10.586 INFO:tasks.ceph:Scrubbing osd.6 2024-04-23T14:45:10.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.806+0000 7febadc00640 1 Processor -- start 2024-04-23T14:45:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.806+0000 7febadc00640 1 -- start start 2024-04-23T14:45:10.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.807+0000 7febadc00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feba8158c10 0x7feba8141e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.807+0000 7febadc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba80b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.807+0000 7febadc00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba815f1d0 con 0x7feba806bdf0 2024-04-23T14:45:10.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.807+0000 7febadc00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7feba815fd50 con 0x7feba8158c10 2024-04-23T14:45:10.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.807+0000 7febadc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7feba8142350 con 0x7feba8158ff0 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feba8158c10 0x7feba8141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feba8158c10 0x7feba8141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35266/0 (socket says 172.21.15.59:35266) 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7e00640 1 -- 172.21.15.59:0/2591899060 learned_addr learned my addr 172.21.15.59:0/2591899060 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7400640 1 --2- 172.21.15.59:0/2591899060 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba80b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7e00640 1 -- 172.21.15.59:0/2591899060 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 msgr2=0x7feba80b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7e00640 1 --2- 172.21.15.59:0/2591899060 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba80b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7e00640 1 -- 172.21.15.59:0/2591899060 >> v1:172.21.15.76:6789/0 conn(0x7feba806bdf0 legacy=0x7feba80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7e00640 1 -- 172.21.15.59:0/2591899060 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba815f4b0 con 0x7feba8158c10 2024-04-23T14:45:10.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.808+0000 7feba7400640 1 --2- 172.21.15.59:0/2591899060 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba80b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:10.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.810+0000 7feba7e00640 1 --2- 172.21.15.59:0/2591899060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba8141e10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7feb98009870 tx=0x7feb9802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b909f65f95fa369 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.810+0000 7feba6a00640 1 -- 172.21.15.59:0/2591899060 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98037070 con 0x7feba8158c10 2024-04-23T14:45:10.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.810+0000 7feba6a00640 1 -- 172.21.15.59:0/2591899060 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb98032440 con 0x7feba8158c10 2024-04-23T14:45:10.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.810+0000 7feba6a00640 1 -- 172.21.15.59:0/2591899060 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98032810 con 0x7feba8158c10 2024-04-23T14:45:10.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.810+0000 7febadc00640 1 -- 172.21.15.59:0/2591899060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 msgr2=0x7feba8141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.810+0000 7febadc00640 1 --2- 172.21.15.59:0/2591899060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba8141e10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7feb98009870 tx=0x7feb9802a180 comp rx=0 tx=0).stop 2024-04-23T14:45:10.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.811+0000 7febadc00640 1 -- 172.21.15.59:0/2591899060 shutdown_connections 2024-04-23T14:45:10.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.811+0000 7febadc00640 1 --2- 172.21.15.59:0/2591899060 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba80b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.811+0000 7febadc00640 1 --2- 172.21.15.59:0/2591899060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba8141e10 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.811+0000 7febadc00640 1 -- 172.21.15.59:0/2591899060 >> 172.21.15.59:0/2591899060 conn(0x7feba80b2200 msgr2=0x7feba80b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:10.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.811+0000 7febadc00640 1 -- 172.21.15.59:0/2591899060 shutdown_connections 2024-04-23T14:45:10.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.811+0000 7febadc00640 1 -- 172.21.15.59:0/2591899060 wait complete. 2024-04-23T14:45:10.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.812+0000 7febadc00640 1 Processor -- start 2024-04-23T14:45:10.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.813+0000 7febadc00640 1 -- start start 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febadc00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feba806bdf0 0x7feba81567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febadc00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba80999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febadc00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba809c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febadc00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feba815fa70 con 0x7feba8158c10 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febadc00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feba80b4000 con 0x7feba806bdf0 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febadc00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7feba8071e10 con 0x7feba8158ff0 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba80999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba80999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35274/0 (socket says 172.21.15.59:35274) 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 -- 172.21.15.59:0/132044040 learned_addr learned my addr 172.21.15.59:0/132044040 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7e00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feba806bdf0 0x7feba81567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 -- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 msgr2=0x7feba809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febad200640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 -- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feba806bdf0 msgr2=0x7feba81567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feba806bdf0 0x7feba81567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7400640 1 -- 172.21.15.59:0/132044040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8155700 con 0x7feba8158c10 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7feba7e00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feba806bdf0 0x7feba81567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.814+0000 7febad200640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7feba8158ff0 0x7feba809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.815+0000 7feba7400640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba80999c0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c7f0 tx=0x7feb9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.815+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94015070 con 0x7feba8158c10 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.815+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94007a30 con 0x7feba8158c10 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.815+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94007e00 con 0x7feba8158c10 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.815+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba81559e0 con 0x7feba8158c10 2024-04-23T14:45:10.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.815+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba8155fa0 con 0x7feba8158c10 2024-04-23T14:45:10.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.816+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feba8156380 con 0x7feba8158c10 2024-04-23T14:45:10.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.817+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb94018070 con 0x7feba8158c10 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.818+0000 7feb9fe00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7feb8806c8f0 0x7feb8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.818+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7feb9408b970 con 0x7feba8158c10 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.819+0000 7feb9fe00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7feb880728f0 0x7feb88074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.819+0000 7feba7e00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7feb8806c8f0 0x7feb8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.819+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb94009f80 con 0x7feb880728f0 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.819+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7feb940902a0 con 0x7feba8158c10 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.819+0000 7feba7e00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7feb8806c8f0 0x7feb8806edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7feb98034000 tx=0x7feb9802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.819+0000 7febad200640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7feb880728f0 0x7feb88074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:10.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.820+0000 7febad200640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7feb880728f0 0x7feb88074d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:10.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.822+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== osd.6 v2:172.21.15.76:6800/1890860772 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb94009f80 con 0x7feb880728f0 2024-04-23T14:45:10.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.837+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 --> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feba815ed60 con 0x7feb880728f0 2024-04-23T14:45:10.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.838+0000 7feb9fe00640 1 -- 172.21.15.59:0/132044040 <== osd.6 v2:172.21.15.76:6800/1890860772 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feba815ed60 con 0x7feb880728f0 2024-04-23T14:45:10.839 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:10.839 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:10.839 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:10.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.838+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7feb880728f0 msgr2=0x7feb88074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.838+0000 7febadc00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.76:6800/1890860772,v1:172.21.15.76:6801/1890860772] conn(0x7feb880728f0 0x7feb88074d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7feb8806c8f0 msgr2=0x7feb8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7feb8806c8f0 0x7feb8806edb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7feb98034000 tx=0x7feb9802e000 comp rx=0 tx=0).stop 2024-04-23T14:45:10.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 msgr2=0x7feba80999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:10.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 --2- 172.21.15.59:0/132044040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feba8158c10 0x7feba80999c0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c7f0 tx=0x7feb9400ccb0 comp rx=0 tx=0).stop 2024-04-23T14:45:10.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febad200640 1 -- 172.21.15.59:0/132044040 reap_dead start 2024-04-23T14:45:10.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 shutdown_connections 2024-04-23T14:45:10.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 >> 172.21.15.59:0/132044040 conn(0x7feba80b2200 msgr2=0x7feba80af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:10.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 shutdown_connections 2024-04-23T14:45:10.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:10.839+0000 7febadc00640 1 -- 172.21.15.59:0/132044040 wait complete. 2024-04-23T14:45:10.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-23T14:45:11.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.067+0000 7fb699c00640 1 Processor -- start 2024-04-23T14:45:11.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.067+0000 7fb699c00640 1 -- start start 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb699c00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6940cd1b0 0x7fb694072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb699c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940726b0 0x7fb69406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb699c00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb69406e040 con 0x7fb694071d90 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb699c00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fb69406ddf0 con 0x7fb6940cd1b0 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb699c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb69406e320 con 0x7fb6940726b0 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb68be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6940cd1b0 0x7fb694072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb68be00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb6940cd1b0 0x7fb694072170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35276/0 (socket says 172.21.15.59:35276) 2024-04-23T14:45:11.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb692a00640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb694071d90 0x7fb6940b3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52360/0 (socket says 172.21.15.59:52360) 2024-04-23T14:45:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb68be00640 1 -- 172.21.15.59:0/66745307 learned_addr learned my addr 172.21.15.59:0/66745307 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.069+0000 7fb693400640 1 --2- 172.21.15.59:0/66745307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940726b0 0x7fb69406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.070+0000 7fb68be00640 1 -- 172.21.15.59:0/66745307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940726b0 msgr2=0x7fb69406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.070+0000 7fb68be00640 1 --2- 172.21.15.59:0/66745307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940726b0 0x7fb69406ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.070+0000 7fb68be00640 1 -- 172.21.15.59:0/66745307 >> v1:172.21.15.59:6789/0 conn(0x7fb694071d90 legacy=0x7fb6940b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.070+0000 7fb68be00640 1 -- 172.21.15.59:0/66745307 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69406e600 con 0x7fb6940cd1b0 2024-04-23T14:45:11.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.070+0000 7fb693400640 1 --2- 172.21.15.59:0/66745307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940726b0 0x7fb69406ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:11.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.071+0000 7fb68be00640 1 --2- 172.21.15.59:0/66745307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6940cd1b0 0x7fb694072170 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009870 tx=0x7fb67c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc271c95a72d72d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.071+0000 7fb692000640 1 -- 172.21.15.59:0/66745307 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c037070 con 0x7fb6940cd1b0 2024-04-23T14:45:11.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.071+0000 7fb692000640 1 -- 172.21.15.59:0/66745307 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c032440 con 0x7fb6940cd1b0 2024-04-23T14:45:11.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.071+0000 7fb692000640 1 -- 172.21.15.59:0/66745307 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c032810 con 0x7fb6940cd1b0 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 -- 172.21.15.59:0/66745307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6940cd1b0 msgr2=0x7fb694072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 --2- 172.21.15.59:0/66745307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6940cd1b0 0x7fb694072170 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009870 tx=0x7fb67c02a250 comp rx=0 tx=0).stop 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 -- 172.21.15.59:0/66745307 shutdown_connections 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 --2- 172.21.15.59:0/66745307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940726b0 0x7fb69406ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 --2- 172.21.15.59:0/66745307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6940cd1b0 0x7fb694072170 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 -- 172.21.15.59:0/66745307 >> 172.21.15.59:0/66745307 conn(0x7fb6940b2130 msgr2=0x7fb6940b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 -- 172.21.15.59:0/66745307 shutdown_connections 2024-04-23T14:45:11.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.072+0000 7fb699c00640 1 -- 172.21.15.59:0/66745307 wait complete. 2024-04-23T14:45:11.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.074+0000 7fb699c00640 1 Processor -- start 2024-04-23T14:45:11.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.074+0000 7fb699c00640 1 -- start start 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb699c00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071d90 0x7fb6940b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb699c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6940726b0 0x7fb6941f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb699c00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940cd1b0 0x7fb6941f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb699c00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb694072ac0 con 0x7fb694071d90 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb699c00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb694166430 con 0x7fb6940726b0 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb699c00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fb694161c50 con 0x7fb6940cd1b0 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb692a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071d90 0x7fb6940b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb692a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071d90 0x7fb6940b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35286/0 (socket says 172.21.15.59:35286) 2024-04-23T14:45:11.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.075+0000 7fb692a00640 1 -- 172.21.15.59:0/1384457340 learned_addr learned my addr 172.21.15.59:0/1384457340 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb693400640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940cd1b0 0x7fb6941f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb692a00640 1 -- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940cd1b0 msgr2=0x7fb6941f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb68be00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6940726b0 0x7fb6941f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb692a00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940cd1b0 0x7fb6941f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb692a00640 1 -- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6940726b0 msgr2=0x7fb6941f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb692a00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6940726b0 0x7fb6941f8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb692a00640 1 -- 172.21.15.59:0/1384457340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb694161f30 con 0x7fb694071d90 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb692a00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071d90 0x7fb6940b14e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c3c0 tx=0x7fb68400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb68aa00640 1 -- 172.21.15.59:0/1384457340 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6840074e0 con 0x7fb694071d90 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb68aa00640 1 -- 172.21.15.59:0/1384457340 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb684007680 con 0x7fb694071d90 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb68aa00640 1 -- 172.21.15.59:0/1384457340 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684004a00 con 0x7fb694071d90 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69415d320 con 0x7fb694071d90 2024-04-23T14:45:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.076+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6940b1a20 con 0x7fb694071d90 2024-04-23T14:45:11.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.078+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb694068a80 con 0x7fb694071d90 2024-04-23T14:45:11.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.079+0000 7fb68aa00640 1 -- 172.21.15.59:0/1384457340 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb68401a020 con 0x7fb694071d90 2024-04-23T14:45:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.079+0000 7fb68aa00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.080+0000 7fb68aa00640 1 -- 172.21.15.59:0/1384457340 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fb68408b140 con 0x7fb694071d90 2024-04-23T14:45:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.080+0000 7fb68be00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.081+0000 7fb68be00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67006c8f0 0x7fb67006edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb67c034000 tx=0x7fb67c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.084+0000 7fb68aa00640 1 -- 172.21.15.59:0/1384457340 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb684059720 con 0x7fb694071d90 2024-04-23T14:45:11.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.286+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fb694162100 con 0x7fb67006c8f0 2024-04-23T14:45:11.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.287+0000 7fb68aa00640 1 -- 172.21.15.59:0/1384457340 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb694162100 con 0x7fb67006c8f0 2024-04-23T14:45:11.287 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 6 to deep-scrub 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.289+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67006c8f0 msgr2=0x7fb67006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.289+0000 7fb699c00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67006c8f0 0x7fb67006edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb67c034000 tx=0x7fb67c02e000 comp rx=0 tx=0).stop 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.289+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071d90 msgr2=0x7fb6940b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.289+0000 7fb699c00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071d90 0x7fb6940b14e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c3c0 tx=0x7fb68400c880 comp rx=0 tx=0).stop 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.289+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 shutdown_connections 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.289+0000 7fb699c00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fb67006c8f0 0x7fb67006edb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.290+0000 7fb699c00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6940cd1b0 0x7fb6941f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.290+0000 7fb699c00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6940726b0 0x7fb6941f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.290+0000 7fb699c00640 1 --2- 172.21.15.59:0/1384457340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071d90 0x7fb6940b14e0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.290+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 >> 172.21.15.59:0/1384457340 conn(0x7fb6940b2130 msgr2=0x7fb69406fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:11.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.290+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 shutdown_connections 2024-04-23T14:45:11.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.290+0000 7fb699c00640 1 -- 172.21.15.59:0/1384457340 wait complete. 2024-04-23T14:45:11.310 INFO:tasks.ceph:Scrubbing osd.7 2024-04-23T14:45:11.311 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-23T14:45:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.523+0000 7fbd36600640 1 Processor -- start 2024-04-23T14:45:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.523+0000 7fbd36600640 1 -- start start 2024-04-23T14:45:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd36600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd30162e60 0x7fbd30071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd36600640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd30163ce0 0x7fbd300722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd36600640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd301641e0 con 0x7fbd3006eef0 2024-04-23T14:45:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd36600640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3006e130 con 0x7fbd30163ce0 2024-04-23T14:45:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd36600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3006e410 con 0x7fbd30162e60 2024-04-23T14:45:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd2ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd30163ce0 0x7fbd300722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd2ea00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd30163ce0 0x7fbd300722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35292/0 (socket says 172.21.15.59:35292) 2024-04-23T14:45:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.525+0000 7fbd2ea00640 1 -- 172.21.15.59:0/734854430 learned_addr learned my addr 172.21.15.59:0/734854430 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.526+0000 7fbd2f400640 1 --2- 172.21.15.59:0/734854430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd30162e60 0x7fbd30071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.526+0000 7fbd2ea00640 1 -- 172.21.15.59:0/734854430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd30162e60 msgr2=0x7fbd30071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.526+0000 7fbd2ea00640 1 --2- 172.21.15.59:0/734854430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd30162e60 0x7fbd30071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.526+0000 7fbd2ea00640 1 -- 172.21.15.59:0/734854430 >> v1:172.21.15.76:6789/0 conn(0x7fbd3006eef0 legacy=0x7fbd300b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.526+0000 7fbd2ea00640 1 -- 172.21.15.59:0/734854430 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3006e6f0 con 0x7fbd30163ce0 2024-04-23T14:45:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.528+0000 7fbd2ea00640 1 --2- 172.21.15.59:0/734854430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300722d0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009870 tx=0x7fbd1c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9fd4fcdc3361df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.528+0000 7fbd2e000640 1 -- 172.21.15.59:0/734854430 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c037070 con 0x7fbd30163ce0 2024-04-23T14:45:11.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.528+0000 7fbd2e000640 1 -- 172.21.15.59:0/734854430 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c02ae10 con 0x7fbd30163ce0 2024-04-23T14:45:11.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.528+0000 7fbd2e000640 1 -- 172.21.15.59:0/734854430 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c032660 con 0x7fbd30163ce0 2024-04-23T14:45:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 -- 172.21.15.59:0/734854430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 msgr2=0x7fbd300722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 --2- 172.21.15.59:0/734854430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300722d0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009870 tx=0x7fbd1c02a100 comp rx=0 tx=0).stop 2024-04-23T14:45:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 -- 172.21.15.59:0/734854430 shutdown_connections 2024-04-23T14:45:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 --2- 172.21.15.59:0/734854430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300722d0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 --2- 172.21.15.59:0/734854430 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd30162e60 0x7fbd30071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 -- 172.21.15.59:0/734854430 >> 172.21.15.59:0/734854430 conn(0x7fbd3006b9d0 msgr2=0x7fbd3006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:11.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 -- 172.21.15.59:0/734854430 shutdown_connections 2024-04-23T14:45:11.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.529+0000 7fbd36600640 1 -- 172.21.15.59:0/734854430 wait complete. 2024-04-23T14:45:11.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.531+0000 7fbd36600640 1 Processor -- start 2024-04-23T14:45:11.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.531+0000 7fbd36600640 1 -- start start 2024-04-23T14:45:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd36600640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd3006eef0 0x7fbd3015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd36600640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd30162e60 0x7fbd300cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd36600640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd36600640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd300afe80 con 0x7fbd30163ce0 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd36600640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbd300afd00 con 0x7fbd30162e60 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd36600640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3006f5d0 con 0x7fbd3006eef0 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd2fe00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35298/0 (socket says 172.21.15.59:35298) 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd2fe00640 1 -- 172.21.15.59:0/3416629668 learned_addr learned my addr 172.21.15.59:0/3416629668 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd2ea00640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd30162e60 0x7fbd300cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd2f400640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd3006eef0 0x7fbd3015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.532+0000 7fbd2fe00640 1 -- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd3006eef0 msgr2=0x7fbd3015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd2fe00640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd3006eef0 0x7fbd3015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd2fe00640 1 -- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd30162e60 msgr2=0x7fbd300cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd2fe00640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd30162e60 0x7fbd300cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd2fe00640 1 -- 172.21.15.59:0/3416629668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd30159fc0 con 0x7fbd30163ce0 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd2fe00640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300ce0b0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c780 tx=0x7fbd2800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28004b60 con 0x7fbd30163ce0 2024-04-23T14:45:11.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28004d00 con 0x7fbd30163ce0 2024-04-23T14:45:11.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800aa30 con 0x7fbd30163ce0 2024-04-23T14:45:11.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd300ce6e0 con 0x7fbd30163ce0 2024-04-23T14:45:11.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.533+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3015a280 con 0x7fbd30163ce0 2024-04-23T14:45:11.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.534+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd3006ddf0 con 0x7fbd30163ce0 2024-04-23T14:45:11.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.536+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd2800abd0 con 0x7fbd30163ce0 2024-04-23T14:45:11.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.537+0000 7fbd27400640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd1006c8f0 0x7fbd1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.537+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fbd2808bc20 con 0x7fbd30163ce0 2024-04-23T14:45:11.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.537+0000 7fbd2f400640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd1006c8f0 0x7fbd1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.537+0000 7fbd27400640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7fbd100728f0 0x7fbd10074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.537+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd100753c0 con 0x7fbd100728f0 2024-04-23T14:45:11.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.537+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7fbd280902a0 con 0x7fbd30163ce0 2024-04-23T14:45:11.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.538+0000 7fbd2ea00640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7fbd100728f0 0x7fbd10074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.538+0000 7fbd2f400640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd1006c8f0 0x7fbd1006edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbd20006fd0 tx=0x7fbd20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.539+0000 7fbd2ea00640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7fbd100728f0 0x7fbd10074d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.541+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== osd.7 v2:172.21.15.76:6822/3281262326 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd100753c0 con 0x7fbd100728f0 2024-04-23T14:45:11.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.556+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 --> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbd30068a80 con 0x7fbd100728f0 2024-04-23T14:45:11.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.557+0000 7fbd27400640 1 -- 172.21.15.59:0/3416629668 <== osd.7 v2:172.21.15.76:6822/3281262326 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbd30068a80 con 0x7fbd100728f0 2024-04-23T14:45:11.557 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-04-23T14:45:11.557 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T14:45:11.558 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.557+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7fbd100728f0 msgr2=0x7fbd10074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.557+0000 7fbd36600640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.76:6822/3281262326,v1:172.21.15.76:6824/3281262326] conn(0x7fbd100728f0 0x7fbd10074d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd1006c8f0 msgr2=0x7fbd1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fbd1006c8f0 0x7fbd1006edb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbd20006fd0 tx=0x7fbd20008040 comp rx=0 tx=0).stop 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 msgr2=0x7fbd300ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 --2- 172.21.15.59:0/3416629668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd30163ce0 0x7fbd300ce0b0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c780 tx=0x7fbd2800cc40 comp rx=0 tx=0).stop 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd2fe00640 1 -- 172.21.15.59:0/3416629668 reap_dead start 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 shutdown_connections 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 >> 172.21.15.59:0/3416629668 conn(0x7fbd3006b9d0 msgr2=0x7fbd300b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 shutdown_connections 2024-04-23T14:45:11.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.558+0000 7fbd36600640 1 -- 172.21.15.59:0/3416629668 wait complete. 2024-04-23T14:45:11.575 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-23T14:45:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.777+0000 7fdeb3200640 1 Processor -- start 2024-04-23T14:45:11.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.777+0000 7fdeb3200640 1 -- start start 2024-04-23T14:45:11.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeb3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb415c670 0x7fdeb40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeb3200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdeb40b4230 0x7fdeb40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeb3200640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdeb40d05c0 0x7fdeb40d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeb3200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb415ca50 con 0x7fdeb40b4230 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeb3200640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb40ccb80 con 0x7fdeb40d05c0 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeb3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb40b4cf0 con 0x7fdeb415c670 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeabe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdeb40b4230 0x7fdeb40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeabe00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdeb40b4230 0x7fdeb40b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35300/0 (socket says 172.21.15.59:35300) 2024-04-23T14:45:11.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.778+0000 7fdeabe00640 1 -- 172.21.15.59:0/3883581029 learned_addr learned my addr 172.21.15.59:0/3883581029 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeb2800640 1 --2- 172.21.15.59:0/3883581029 >> v2:172.21.15.76:3300/0 conn(0x7fdeb40d05c0 0x7fdeb40d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeabe00640 1 -- 172.21.15.59:0/3883581029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb415c670 msgr2=0x7fdeb40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeb1e00640 1 --2- 172.21.15.59:0/3883581029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb415c670 0x7fdeb40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeabe00640 1 --2- 172.21.15.59:0/3883581029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb415c670 0x7fdeb40cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeabe00640 1 -- 172.21.15.59:0/3883581029 >> v2:172.21.15.76:3300/0 conn(0x7fdeb40d05c0 msgr2=0x7fdeb40d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeabe00640 1 --2- 172.21.15.59:0/3883581029 >> v2:172.21.15.76:3300/0 conn(0x7fdeb40d05c0 0x7fdeb40d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeabe00640 1 -- 172.21.15.59:0/3883581029 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb40ccdd0 con 0x7fdeb40b4230 2024-04-23T14:45:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeb2800640 1 --2- 172.21.15.59:0/3883581029 >> v2:172.21.15.76:3300/0 conn(0x7fdeb40d05c0 0x7fdeb40d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.779+0000 7fdeb1e00640 1 --2- 172.21.15.59:0/3883581029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb415c670 0x7fdeb40cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:11.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.780+0000 7fdeabe00640 1 --2- 172.21.15.59:0/3883581029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb40b4690 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40cf340 tx=0x7fde9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5569f33be84683b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.781+0000 7fdeb1400640 1 -- 172.21.15.59:0/3883581029 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c037070 con 0x7fdeb40b4230 2024-04-23T14:45:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.781+0000 7fdeb1400640 1 -- 172.21.15.59:0/3883581029 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9c032440 con 0x7fdeb40b4230 2024-04-23T14:45:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.781+0000 7fdeb1400640 1 -- 172.21.15.59:0/3883581029 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c032810 con 0x7fdeb40b4230 2024-04-23T14:45:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.781+0000 7fdeb3200640 1 -- 172.21.15.59:0/3883581029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 msgr2=0x7fdeb40b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.781+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3883581029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb40b4690 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40cf340 tx=0x7fde9c02a1e0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.782+0000 7fdeb3200640 1 -- 172.21.15.59:0/3883581029 shutdown_connections 2024-04-23T14:45:11.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.782+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3883581029 >> v2:172.21.15.76:3300/0 conn(0x7fdeb40d05c0 0x7fdeb40d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.782+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3883581029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb40b4690 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.782+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3883581029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb415c670 0x7fdeb40cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.782+0000 7fdeb3200640 1 -- 172.21.15.59:0/3883581029 >> 172.21.15.59:0/3883581029 conn(0x7fdeb406bdf0 msgr2=0x7fdeb406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:11.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.782+0000 7fdeb3200640 1 -- 172.21.15.59:0/3883581029 shutdown_connections 2024-04-23T14:45:11.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.782+0000 7fdeb3200640 1 -- 172.21.15.59:0/3883581029 wait complete. 2024-04-23T14:45:11.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.783+0000 7fdeb3200640 1 Processor -- start 2024-04-23T14:45:11.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.784+0000 7fdeb3200640 1 -- start start 2024-04-23T14:45:11.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb3200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb4159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb3200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb40d05c0 0x7fdeb41484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb3200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb415c670 0x7fdeb415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb3200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb40b3e60 con 0x7fdeb40b4230 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb3200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb40cd390 con 0x7fdeb415c670 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb3200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb40b3fe0 con 0x7fdeb40d05c0 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb4159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb1e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb4159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35316/0 (socket says 172.21.15.59:35316) 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb1e00640 1 -- 172.21.15.59:0/3636618806 learned_addr learned my addr 172.21.15.59:0/3636618806 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb2800640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb415c670 0x7fdeb415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeabe00640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb40d05c0 0x7fdeb41484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb1e00640 1 -- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb40d05c0 msgr2=0x7fdeb41484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb1e00640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb40d05c0 0x7fdeb41484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb1e00640 1 -- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb415c670 msgr2=0x7fdeb415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.785+0000 7fdeb1e00640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb415c670 0x7fdeb415a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:11.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeb1e00640 1 -- 172.21.15.59:0/3636618806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb4158f30 con 0x7fdeb40b4230 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeabe00640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb40d05c0 0x7fdeb41484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeb1e00640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb4159fd0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c7f0 tx=0x7fdea400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeaa000640 1 -- 172.21.15.59:0/3636618806 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4015070 con 0x7fdeb40b4230 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeaa000640 1 -- 172.21.15.59:0/3636618806 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea4007820 con 0x7fdeb40b4230 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeaa000640 1 -- 172.21.15.59:0/3636618806 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4007bd0 con 0x7fdeb40b4230 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb4159210 con 0x7fdeb40b4230 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.786+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb41597d0 con 0x7fdeb40b4230 2024-04-23T14:45:11.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.788+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb404fd90 con 0x7fdeb40b4230 2024-04-23T14:45:11.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.789+0000 7fdeaa000640 1 -- 172.21.15.59:0/3636618806 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdea4018070 con 0x7fdeb40b4230 2024-04-23T14:45:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.789+0000 7fdeaa000640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde9006c8f0 0x7fde9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.790+0000 7fdeaa000640 1 -- 172.21.15.59:0/3636618806 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fdea408ba80 con 0x7fdeb40b4230 2024-04-23T14:45:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.790+0000 7fdeabe00640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde9006c8f0 0x7fde9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.790+0000 7fdeabe00640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde9006c8f0 0x7fde9006edb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fde9c034000 tx=0x7fde9c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:11.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.794+0000 7fdeaa000640 1 -- 172.21.15.59:0/3636618806 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdea408de20 con 0x7fdeb40b4230 2024-04-23T14:45:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.997+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fdeb4159bb0 con 0x7fde9006c8f0 2024-04-23T14:45:11.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:11.998+0000 7fdeaa000640 1 -- 172.21.15.59:0/3636618806 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdeb4159bb0 con 0x7fde9006c8f0 2024-04-23T14:45:11.998 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 7 to deep-scrub 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde9006c8f0 msgr2=0x7fde9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde9006c8f0 0x7fde9006edb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fde9c034000 tx=0x7fde9c02e000 comp rx=0 tx=0).stop 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 msgr2=0x7fdeb4159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb4159fd0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c7f0 tx=0x7fdea400ccb0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 shutdown_connections 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fde9006c8f0 0x7fde9006edb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdeb40d05c0 0x7fdeb41484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdeb415c670 0x7fdeb415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 --2- 172.21.15.59:0/3636618806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb40b4230 0x7fdeb4159fd0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 >> 172.21.15.59:0/3636618806 conn(0x7fdeb406bdf0 msgr2=0x7fdeb4157b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.000+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 shutdown_connections 2024-04-23T14:45:12.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.001+0000 7fdeb3200640 1 -- 172.21.15.59:0/3636618806 wait complete. 2024-04-23T14:45:12.020 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:45:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.233+0000 7fae63e00640 1 Processor -- start 2024-04-23T14:45:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.233+0000 7fae63e00640 1 -- start start 2024-04-23T14:45:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.235+0000 7fae63e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 0x7fae640cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.235+0000 7fae63e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae6415b270 con 0x7fae6406b9d0 2024-04-23T14:45:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.235+0000 7fae63e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae640b4930 con 0x7fae6406f350 2024-04-23T14:45:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.235+0000 7fae63e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae640b3080 con 0x7fae6415ae90 2024-04-23T14:45:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.235+0000 7fae63400640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fae6406b9d0 0x7fae640b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52366/0 (socket says 172.21.15.59:52366) 2024-04-23T14:45:12.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.235+0000 7fae63400640 1 -- 172.21.15.59:0/3339082058 learned_addr learned my addr 172.21.15.59:0/3339082058 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:12.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62a00640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 0x7fae640cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 812605529 0 0) 0x7fae6415b270 con 0x7fae6406b9d0 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae48003520 con 0x7fae6406b9d0 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3172817137 0 0) 0x7fae48003520 con 0x7fae6406b9d0 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 msgr2=0x7fae640cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 0x7fae640cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 >> v1:172.21.15.76:6789/0 conn(0x7fae6406f350 legacy=0x7fae64071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae640b4c10 con 0x7fae6406b9d0 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 59422580 0 0) 0x7fae5c001670 con 0x7fae6406b9d0 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 >> v1:172.21.15.59:6789/0 conn(0x7fae6406b9d0 legacy=0x7fae640b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae480039e0 0x7fae48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae48004800 0x7fae48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.236+0000 7fae62000640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 0x7fae480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae640b3080 con 0x7fae480039e0 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae4800baa0 con 0x7fae48004800 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae4800deb0 con 0x7fae6415ae90 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62a00640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae480039e0 0x7fae48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae63400640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 0x7fae480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae5a000640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae48004800 0x7fae48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62a00640 1 -- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 msgr2=0x7fae480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62a00640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 0x7fae480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62a00640 1 -- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae48004800 msgr2=0x7fae48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62a00640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae48004800 0x7fae48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae62a00640 1 -- 172.21.15.59:0/3339082058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae48003520 con 0x7fae480039e0 2024-04-23T14:45:12.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.237+0000 7fae63400640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae6415ae90 0x7fae480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:12.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.239+0000 7fae62a00640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae480039e0 0x7fae48003e00 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fae54009f90 tx=0x7fae48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaf633b887ce319 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:12.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.239+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54009c10 con 0x7fae480039e0 2024-04-23T14:45:12.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.239+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae54036070 con 0x7fae480039e0 2024-04-23T14:45:12.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.239+0000 7fae62000640 1 -- 172.21.15.59:0/3339082058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54031830 con 0x7fae480039e0 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.239+0000 7fae63e00640 1 -- 172.21.15.59:0/3339082058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae480039e0 msgr2=0x7fae48003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.239+0000 7fae63e00640 1 --2- 172.21.15.59:0/3339082058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae480039e0 0x7fae48003e00 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fae54009f90 tx=0x7fae48005830 comp rx=0 tx=0).stop 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.239+0000 7fae63400640 1 -- 172.21.15.59:0/3339082058 reap_dead start 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.240+0000 7fae63e00640 1 -- 172.21.15.59:0/3339082058 shutdown_connections 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.240+0000 7fae63e00640 1 -- 172.21.15.59:0/3339082058 >> 172.21.15.59:0/3339082058 conn(0x7fae640b21a0 msgr2=0x7fae640b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.240+0000 7fae63e00640 1 -- 172.21.15.59:0/3339082058 shutdown_connections 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.240+0000 7fae63e00640 1 -- 172.21.15.59:0/3339082058 wait complete. 2024-04-23T14:45:12.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.242+0000 7fae63e00640 1 Processor -- start 2024-04-23T14:45:12.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.242+0000 7fae63e00640 1 -- start start 2024-04-23T14:45:12.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae63e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae480039e0 0x7fae640c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae63e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64159530 0x7fae640c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae63e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae640add90 0x7fae640ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae63e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fae640722f0 con 0x7fae64159530 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae63e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae640b32d0 con 0x7fae480039e0 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae63e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7fae640b45c0 con 0x7fae640add90 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64159530 0x7fae640c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64159530 0x7fae640c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35332/0 (socket says 172.21.15.59:35332) 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 -- 172.21.15.59:0/1534788103 learned_addr learned my addr 172.21.15.59:0/1534788103 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 -- 172.21.15.59:0/1534788103 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae640add90 msgr2=0x7fae640ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae640add90 0x7fae640ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 -- 172.21.15.59:0/1534788103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae480039e0 msgr2=0x7fae640c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae480039e0 0x7fae640c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.243+0000 7fae5a000640 1 -- 172.21.15.59:0/1534788103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6415c550 con 0x7fae64159530 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.244+0000 7fae5a000640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64159530 0x7fae640c6620 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e500 tx=0x7fae4c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.244+0000 7fae5be00640 1 -- 172.21.15.59:0/1534788103 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0053a0 con 0x7fae64159530 2024-04-23T14:45:12.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.244+0000 7fae5be00640 1 -- 172.21.15.59:0/1534788103 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4c018070 con 0x7fae64159530 2024-04-23T14:45:12.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.244+0000 7fae5be00640 1 -- 172.21.15.59:0/1534788103 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c003810 con 0x7fae64159530 2024-04-23T14:45:12.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.244+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6415c830 con 0x7fae64159530 2024-04-23T14:45:12.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.244+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6415cdf0 con 0x7fae64159530 2024-04-23T14:45:12.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.246+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae640ae750 con 0x7fae64159530 2024-04-23T14:45:12.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.247+0000 7fae5be00640 1 -- 172.21.15.59:0/1534788103 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae4c0039b0 con 0x7fae64159530 2024-04-23T14:45:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.248+0000 7fae5be00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae4006c8f0 0x7fae4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.248+0000 7fae5be00640 1 -- 172.21.15.59:0/1534788103 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fae4c08b570 con 0x7fae64159530 2024-04-23T14:45:12.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.248+0000 7fae62a00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae4006c8f0 0x7fae4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.249+0000 7fae62a00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae4006c8f0 0x7fae4006edb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fae54035000 tx=0x7fae5402d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.253+0000 7fae5be00640 1 -- 172.21.15.59:0/1534788103 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae4c059c00 con 0x7fae64159530 2024-04-23T14:45:12.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.454+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae6404fd90 con 0x7fae4006c8f0 2024-04-23T14:45:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.456+0000 7fae5be00640 1 -- 172.21.15.59:0/1534788103 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7fae6404fd90 con 0x7fae4006c8f0 2024-04-23T14:45:12.457 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:45:12.457 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:45:12.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae4006c8f0 msgr2=0x7fae4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae4006c8f0 0x7fae4006edb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fae54035000 tx=0x7fae5402d000 comp rx=0 tx=0).stop 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64159530 msgr2=0x7fae640c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64159530 0x7fae640c6620 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e500 tx=0x7fae4c00e9c0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 shutdown_connections 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7fae4006c8f0 0x7fae4006edb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae640add90 0x7fae640ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae64159530 0x7fae640c6620 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 --2- 172.21.15.59:0/1534788103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae480039e0 0x7fae640c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 >> 172.21.15.59:0/1534788103 conn(0x7fae640b21a0 msgr2=0x7fae640cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 shutdown_connections 2024-04-23T14:45:12.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:12.459+0000 7fae63e00640 1 -- 172.21.15.59:0/1534788103 wait complete. 2024-04-23T14:45:12.479 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2024-04-23T14:45:10.646803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262236,"kb_used_data":12776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712484,"statfs":{"total":773094113280,"available":772825583616,"internally_reserved":0,"allocated":13082624,"data_stored":6364200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002819"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":357,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-23T14:45:03.043637+0000","last_change":"2024-04-23T14:31:15.724355+0000","last_active":"2024-04-23T14:45:03.043637+0000","last_peered":"2024-04-23T14:45:03.043637+0000","last_clean":"2024-04-23T14:45:03.043637+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:45:03.043637+0000","last_undegraded":"2024-04-23T14:45:03.043637+0000","last_fullsized":"2024-04-23T14:45:03.043637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:31:14.670560+0000","last_clean_scrub_stamp":"2024-04-23T14:31:14.670560+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:53:51.135982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1238301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2080768,"data_stored":1238301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:45:12.483 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T14:31:14.670560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=14, tm_min=31, tm_sec=14, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=14, tm_min=45, tm_sec=5, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T14:45:12.483 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T14:45:22.336 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:45:22.344 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:45:32.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.703+0000 7f697c400640 1 Processor -- start 2024-04-23T14:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.703+0000 7f697c400640 1 -- start start 2024-04-23T14:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f697c400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69740d0b70 0x7f69740d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f697c400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69740cd250 0x7f69740d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f697c400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69740d1490 con 0x7f697415e200 2024-04-23T14:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f697c400640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f697409b6f0 con 0x7f69740d0b70 2024-04-23T14:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f697c400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f69740cd9f0 con 0x7f69740cd250 2024-04-23T14:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f6979a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69740d0b70 0x7f69740d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f6979a00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69740d0b70 0x7f69740d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44844/0 (socket says 172.21.15.59:44844) 2024-04-23T14:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.704+0000 7f6979a00640 1 -- 172.21.15.59:0/1614336524 learned_addr learned my addr 172.21.15.59:0/1614336524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.705+0000 7f6979000640 1 --2- 172.21.15.59:0/1614336524 >> v2:172.21.15.76:3300/0 conn(0x7f69740cd250 0x7f69740d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.705+0000 7f6979a00640 1 -- 172.21.15.59:0/1614336524 >> v2:172.21.15.76:3300/0 conn(0x7f69740cd250 msgr2=0x7f69740d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.705+0000 7f6979a00640 1 --2- 172.21.15.59:0/1614336524 >> v2:172.21.15.76:3300/0 conn(0x7f69740cd250 0x7f69740d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.705+0000 7f6979a00640 1 -- 172.21.15.59:0/1614336524 >> v1:172.21.15.76:6789/0 conn(0x7f697415e200 legacy=0x7f697415e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.705+0000 7f6979a00640 1 -- 172.21.15.59:0/1614336524 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69740cdcb0 con 0x7f69740d0b70 2024-04-23T14:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.706+0000 7f6979a00640 1 --2- 172.21.15.59:0/1614336524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740d0b70 0x7f69740d0370 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f6974073970 tx=0x7f696802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c30d9c88591e235 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:32.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.706+0000 7f6973e00640 1 -- 172.21.15.59:0/1614336524 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968037070 con 0x7f69740d0b70 2024-04-23T14:45:32.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.706+0000 7f6973e00640 1 -- 172.21.15.59:0/1614336524 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6968032440 con 0x7f69740d0b70 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.706+0000 7f6973e00640 1 -- 172.21.15.59:0/1614336524 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968032810 con 0x7f69740d0b70 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 -- 172.21.15.59:0/1614336524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740d0b70 msgr2=0x7f69740d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 --2- 172.21.15.59:0/1614336524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740d0b70 0x7f69740d0370 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f6974073970 tx=0x7f696802a1a0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 -- 172.21.15.59:0/1614336524 shutdown_connections 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 --2- 172.21.15.59:0/1614336524 >> v2:172.21.15.76:3300/0 conn(0x7f69740cd250 0x7f69740d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 --2- 172.21.15.59:0/1614336524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740d0b70 0x7f69740d0370 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 -- 172.21.15.59:0/1614336524 >> 172.21.15.59:0/1614336524 conn(0x7f69740b4080 msgr2=0x7f69740b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 -- 172.21.15.59:0/1614336524 shutdown_connections 2024-04-23T14:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.707+0000 7f697c400640 1 -- 172.21.15.59:0/1614336524 wait complete. 2024-04-23T14:45:32.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.709+0000 7f697c400640 1 Processor -- start 2024-04-23T14:45:32.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.709+0000 7f697c400640 1 -- start start 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f697c400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740cd250 0x7f69741f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f697c400640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69740d0b70 0x7f6974070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f697c400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f697415e200 0x7f69740b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f697c400640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f69740cd6c0 con 0x7f69740cd250 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f697c400640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f69740ce270 con 0x7f697415e200 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f697c400640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f697415ed80 con 0x7f69740d0b70 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f6979a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740cd250 0x7f69741f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f6979a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740cd250 0x7f69741f46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44852/0 (socket says 172.21.15.59:44852) 2024-04-23T14:45:32.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.710+0000 7f6979a00640 1 -- 172.21.15.59:0/3896819656 learned_addr learned my addr 172.21.15.59:0/3896819656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:32.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.711+0000 7f6979a00640 1 -- 172.21.15.59:0/3896819656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69740d0b70 msgr2=0x7f6974070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.711+0000 7f6979a00640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69740d0b70 0x7f6974070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.711+0000 7f6979a00640 1 -- 172.21.15.59:0/3896819656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f697415e200 msgr2=0x7f69740b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.711+0000 7f697a400640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f697415e200 0x7f69740b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.711+0000 7f6979a00640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f697415e200 0x7f69740b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.711+0000 7f6979a00640 1 -- 172.21.15.59:0/3896819656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6974071410 con 0x7f69740cd250 2024-04-23T14:45:32.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.711+0000 7f6979a00640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740cd250 0x7f69741f46c0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f69680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:32.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.712+0000 7f6971600640 1 -- 172.21.15.59:0/3896819656 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968041020 con 0x7f69740cd250 2024-04-23T14:45:32.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.712+0000 7f6971600640 1 -- 172.21.15.59:0/3896819656 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6968030970 con 0x7f69740cd250 2024-04-23T14:45:32.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.712+0000 7f6971600640 1 -- 172.21.15.59:0/3896819656 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968037040 con 0x7f69740cd250 2024-04-23T14:45:32.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.712+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69740716f0 con 0x7f69740cd250 2024-04-23T14:45:32.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.712+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69740c7010 con 0x7f69740cd250 2024-04-23T14:45:32.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.713+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6974068a80 con 0x7f69740cd250 2024-04-23T14:45:32.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.714+0000 7f6971600640 1 -- 172.21.15.59:0/3896819656 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6968044020 con 0x7f69740cd250 2024-04-23T14:45:32.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.715+0000 7f6971600640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f695806c940 0x7f695806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.715+0000 7f6971600640 1 -- 172.21.15.59:0/3896819656 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f69680ac2a0 con 0x7f69740cd250 2024-04-23T14:45:32.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.716+0000 7f6979000640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f695806c940 0x7f695806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.716+0000 7f6979000640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f695806c940 0x7f695806ee00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f697409b010 tx=0x7f6964008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.720+0000 7f6971600640 1 -- 172.21.15.59:0/3896819656 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f696804b030 con 0x7f69740cd250 2024-04-23T14:45:32.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.920+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 --> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69740b3e60 con 0x7f695806c940 2024-04-23T14:45:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.922+0000 7f6971600640 1 -- 172.21.15.59:0/3896819656 <== mgr.4101 v2:172.21.15.59:6832/3176066673 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f69740b3e60 con 0x7f695806c940 2024-04-23T14:45:32.924 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:45:32.924 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-04-23T14:45:32.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f695806c940 msgr2=0x7f695806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f695806c940 0x7f695806ee00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f697409b010 tx=0x7f6964008040 comp rx=0 tx=0).stop 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740cd250 msgr2=0x7f69741f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740cd250 0x7f69741f46c0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f69680046c0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 shutdown_connections 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f695806c940 0x7f695806ee00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69740d0b70 0x7f6974070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f697415e200 0x7f69740b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 --2- 172.21.15.59:0/3896819656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69740cd250 0x7f69741f46c0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 >> 172.21.15.59:0/3896819656 conn(0x7f69740b4080 msgr2=0x7f697415f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 shutdown_connections 2024-04-23T14:45:32.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:32.925+0000 7f697c400640 1 -- 172.21.15.59:0/3896819656 wait complete. 2024-04-23T14:45:32.946 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":541,"stamp":"2024-04-23T14:45:32.653174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262224,"kb_used_data":12776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712496,"statfs":{"total":773094113280,"available":772825595904,"internally_reserved":0,"allocated":13082624,"data_stored":6364200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003559"},"pg_stats":[{"pgid":"1.0","version":"119'119","reported_seq":369,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-23T14:45:12.227183+0000","last_change":"2024-04-23T14:45:12.227183+0000","last_active":"2024-04-23T14:45:12.227183+0000","last_peered":"2024-04-23T14:45:12.227183+0000","last_clean":"2024-04-23T14:45:12.227183+0000","last_became_active":"2024-04-23T14:31:15.723549+0000","last_became_peered":"2024-04-23T14:31:15.723549+0000","last_unstale":"2024-04-23T14:45:12.227183+0000","last_undegraded":"2024-04-23T14:45:12.227183+0000","last_fullsized":"2024-04-23T14:45:12.227183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'119","last_scrub_stamp":"2024-04-23T14:45:12.226922+0000","last_deep_scrub":"119'119","last_deep_scrub_stamp":"2024-04-23T14:45:12.226922+0000","last_clean_scrub_stamp":"2024-04-23T14:45:12.226922+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:46:29.741466+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2080768,"data_stored":1238301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2080768,"data_stored":1238301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:45:32.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-23T14:45:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.148+0000 7f957b200640 1 Processor -- start 2024-04-23T14:45:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.148+0000 7f957b200640 1 -- start start 2024-04-23T14:45:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f957b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c1567e0 0x7f957c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f957b200640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f957c15d810 0x7f957c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f957b200640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f957c15cab0 con 0x7f957c0d0b40 2024-04-23T14:45:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f957b200640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f957c0d0f70 con 0x7f957c15d810 2024-04-23T14:45:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f957b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f957c160e40 con 0x7f957c1567e0 2024-04-23T14:45:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f9571400640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f957c15d810 0x7f957c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f957a800640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f957c0d0b40 0x7f957c160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52266/0 (socket says 172.21.15.59:52266) 2024-04-23T14:45:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f957a800640 1 -- 172.21.15.59:0/3698046601 learned_addr learned my addr 172.21.15.59:0/3698046601 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.150+0000 7f9579e00640 1 --2- 172.21.15.59:0/3698046601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c1567e0 0x7f957c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.151+0000 7f9571400640 1 -- 172.21.15.59:0/3698046601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c1567e0 msgr2=0x7f957c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.151+0000 7f9571400640 1 --2- 172.21.15.59:0/3698046601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c1567e0 0x7f957c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.151+0000 7f9571400640 1 -- 172.21.15.59:0/3698046601 >> v1:172.21.15.59:6789/0 conn(0x7f957c0d0b40 legacy=0x7f957c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.151+0000 7f9571400640 1 -- 172.21.15.59:0/3698046601 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c15cd30 con 0x7f957c15d810 2024-04-23T14:45:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.151+0000 7f9579400640 1 -- 172.21.15.59:0/3698046601 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983224713 0 0) 0x7f9574001110 con 0x7f957c0d0b40 2024-04-23T14:45:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.152+0000 7f9571400640 1 --2- 172.21.15.59:0/3698046601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c15d810 0x7f957c0d0600 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f9564009d90 tx=0x7f956402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1e0a6db408d289 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.153+0000 7f9579400640 1 -- 172.21.15.59:0/3698046601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564007740 con 0x7f957c15d810 2024-04-23T14:45:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.153+0000 7f9579400640 1 -- 172.21.15.59:0/3698046601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95640078e0 con 0x7f957c15d810 2024-04-23T14:45:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.153+0000 7f9579400640 1 -- 172.21.15.59:0/3698046601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564007cb0 con 0x7f957c15d810 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.153+0000 7f957b200640 1 -- 172.21.15.59:0/3698046601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c15d810 msgr2=0x7f957c0d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.153+0000 7f957b200640 1 --2- 172.21.15.59:0/3698046601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c15d810 0x7f957c0d0600 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f9564009d90 tx=0x7f956402a660 comp rx=0 tx=0).stop 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.154+0000 7f957b200640 1 -- 172.21.15.59:0/3698046601 shutdown_connections 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.154+0000 7f957b200640 1 --2- 172.21.15.59:0/3698046601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c15d810 0x7f957c0d0600 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.154+0000 7f957b200640 1 --2- 172.21.15.59:0/3698046601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c1567e0 0x7f957c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.154+0000 7f957b200640 1 -- 172.21.15.59:0/3698046601 >> 172.21.15.59:0/3698046601 conn(0x7f957c06bdf0 msgr2=0x7f957c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.154+0000 7f957b200640 1 -- 172.21.15.59:0/3698046601 shutdown_connections 2024-04-23T14:45:33.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.154+0000 7f957b200640 1 -- 172.21.15.59:0/3698046601 wait complete. 2024-04-23T14:45:33.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.155+0000 7f957b200640 1 Processor -- start 2024-04-23T14:45:33.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.156+0000 7f957b200640 1 -- start start 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.156+0000 7f957b200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c0d0b40 0x7f957c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f957b200640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c1567e0 0x7f957c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f957b200640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c15d810 0x7f957c1f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f957b200640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f957c160820 con 0x7f957c1567e0 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f957b200640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f957c15d2f0 con 0x7f957c0d0b40 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f957b200640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f957c1609a0 con 0x7f957c15d810 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c1567e0 0x7f957c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c1567e0 0x7f957c1592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44858/0 (socket says 172.21.15.59:44858) 2024-04-23T14:45:33.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 -- 172.21.15.59:0/373740583 learned_addr learned my addr 172.21.15.59:0/373740583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9579e00640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c0d0b40 0x7f957c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 -- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c15d810 msgr2=0x7f957c1f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f957a800640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c15d810 0x7f957c1f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c15d810 0x7f957c1f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 -- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c0d0b40 msgr2=0x7f957c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c0d0b40 0x7f957c158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9571400640 1 -- 172.21.15.59:0/373740583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c08c230 con 0x7f957c1567e0 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f9579e00640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c0d0b40 0x7f957c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.157+0000 7f957a800640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c15d810 0x7f957c1f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.158+0000 7f9571400640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c1567e0 0x7f957c1592d0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f9564000f80 tx=0x7f9564004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.158+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564040070 con 0x7f957c1567e0 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.158+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9564031c40 con 0x7f957c1567e0 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.158+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c08c510 con 0x7f957c1567e0 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.158+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564009450 con 0x7f957c1567e0 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.158+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c08cad0 con 0x7f957c1567e0 2024-04-23T14:45:33.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.160+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c068290 con 0x7f957c1567e0 2024-04-23T14:45:33.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.161+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9564039040 con 0x7f957c1567e0 2024-04-23T14:45:33.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.161+0000 7f9572a00640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f955806c8f0 0x7f955806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.162+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f95640acc80 con 0x7f957c1567e0 2024-04-23T14:45:33.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.162+0000 7f9579e00640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f955806c8f0 0x7f955806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.162+0000 7f9579e00640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f955806c8f0 0x7f955806edb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f956c006fd0 tx=0x7f956c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:33.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.166+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f957c068290 con 0x7f957c1567e0 2024-04-23T14:45:33.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.375+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f957c04fd90 con 0x7f957c1567e0 2024-04-23T14:45:33.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.382+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f956407b260 con 0x7f957c1567e0 2024-04-23T14:45:33.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.383+0000 7f9572a00640 1 -- 172.21.15.59:0/373740583 <== mon.0 v2:172.21.15.59:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f956407ab60 con 0x7f957c1567e0 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f955806c8f0 msgr2=0x7f955806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f955806c8f0 0x7f955806edb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f956c006fd0 tx=0x7f956c008040 comp rx=0 tx=0).stop 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c1567e0 msgr2=0x7f957c1592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c1567e0 0x7f957c1592d0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f9564000f80 tx=0x7f9564004a40 comp rx=0 tx=0).stop 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 shutdown_connections 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f955806c8f0 0x7f955806edb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f957c15d810 0x7f957c1f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f957c1567e0 0x7f957c1592d0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 --2- 172.21.15.59:0/373740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c0d0b40 0x7f957c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.385+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 >> 172.21.15.59:0/373740583 conn(0x7f957c06bdf0 msgr2=0x7f957c15ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.386+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 shutdown_connections 2024-04-23T14:45:33.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.386+0000 7f957b200640 1 -- 172.21.15.59:0/373740583 wait complete. 2024-04-23T14:45:33.407 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:45:33.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:33.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.614+0000 7f7259e00640 1 Processor -- start 2024-04-23T14:45:33.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.615+0000 7f7259e00640 1 -- start start 2024-04-23T14:45:33.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.616+0000 7f7259e00640 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f725406da30 0x7f72540cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.616+0000 7f7259e00640 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72540cd9f0 con 0x7f72540d0370 2024-04-23T14:45:33.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.616+0000 7f7259e00640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72540cdcb0 con 0x7f72540d0b70 2024-04-23T14:45:33.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.616+0000 7f7259e00640 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 -- 0x7f725409b6f0 con 0x7f725406da30 2024-04-23T14:45:33.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.617+0000 7f7252000640 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f72540d0370 0x7f725415e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52280/0 (socket says 172.21.15.59:52280) 2024-04-23T14:45:33.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.617+0000 7f7252000640 1 -- 172.21.15.59:0/3350969677 learned_addr learned my addr 172.21.15.59:0/3350969677 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-04-23T14:45:33.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.617+0000 7f7253400640 1 --2- 172.21.15.59:0/3350969677 >> v2:172.21.15.59:3300/0 conn(0x7f725406da30 0x7f72540cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.617+0000 7f7253400640 1 -- 172.21.15.59:0/3350969677 >> v1:172.21.15.76:6789/0 conn(0x7f72540d0b70 legacy=0x7f72540d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:45:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.617+0000 7f7253400640 1 -- 172.21.15.59:0/3350969677 >> v1:172.21.15.59:6789/0 conn(0x7f72540d0370 legacy=0x7f725415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.617+0000 7f7253400640 1 -- 172.21.15.59:0/3350969677 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72540cdf90 con 0x7f725406da30 2024-04-23T14:45:33.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.618+0000 7f7253400640 1 --2- 172.21.15.59:0/3350969677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540cd250 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f724c0097d0 tx=0x7f724c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fe6dec01ccdaaa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:33.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.619+0000 7f7251600640 1 -- 172.21.15.59:0/3350969677 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c037070 con 0x7f725406da30 2024-04-23T14:45:33.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.619+0000 7f7251600640 1 -- 172.21.15.59:0/3350969677 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f724c032440 con 0x7f725406da30 2024-04-23T14:45:33.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.619+0000 7f7251600640 1 -- 172.21.15.59:0/3350969677 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c032830 con 0x7f725406da30 2024-04-23T14:45:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.619+0000 7f7259e00640 1 -- 172.21.15.59:0/3350969677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 msgr2=0x7f72540cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.619+0000 7f7259e00640 1 --2- 172.21.15.59:0/3350969677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540cd250 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f724c0097d0 tx=0x7f724c02a180 comp rx=0 tx=0).stop 2024-04-23T14:45:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.620+0000 7f7259e00640 1 -- 172.21.15.59:0/3350969677 shutdown_connections 2024-04-23T14:45:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.620+0000 7f7259e00640 1 --2- 172.21.15.59:0/3350969677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540cd250 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.620+0000 7f7259e00640 1 -- 172.21.15.59:0/3350969677 >> 172.21.15.59:0/3350969677 conn(0x7f72540b4080 msgr2=0x7f72540b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.620+0000 7f7259e00640 1 -- 172.21.15.59:0/3350969677 shutdown_connections 2024-04-23T14:45:33.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.620+0000 7f7259e00640 1 -- 172.21.15.59:0/3350969677 wait complete. 2024-04-23T14:45:33.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.621+0000 7f7259e00640 1 Processor -- start 2024-04-23T14:45:33.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.622+0000 7f7259e00640 1 -- start start 2024-04-23T14:45:33.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7259e00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7259e00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72540d0370 0x7f7254159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7259e00640 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72540d0b70 0x7f72540cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7259e00640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f725415e910 con 0x7f725406da30 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7259e00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72540cd790 con 0x7f72540d0370 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7259e00640 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 -- 0x7f725415ea90 con 0x7f72540d0b70 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44882/0 (socket says 172.21.15.59:44882) 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 -- 172.21.15.59:0/785305298 learned_addr learned my addr 172.21.15.59:0/785305298 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252000640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72540d0370 0x7f7254159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7253400640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72540d0b70 0x7f72540cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 -- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72540d0b70 msgr2=0x7f72540cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72540d0b70 0x7f72540cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 -- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72540d0370 msgr2=0x7f7254159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72540d0370 0x7f7254159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252a00640 1 -- 172.21.15.59:0/785305298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72540c8100 con 0x7f725406da30 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.623+0000 7f7252000640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72540d0370 0x7f7254159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.624+0000 7f7253400640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72540d0b70 0x7f72540cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.624+0000 7f7252a00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540c7b20 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f723c00b3f0 tx=0x7f723c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.624+0000 7f724b400640 1 -- 172.21.15.59:0/785305298 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c015020 con 0x7f725406da30 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.624+0000 7f724b400640 1 -- 172.21.15.59:0/785305298 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f723c011830 con 0x7f725406da30 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.624+0000 7f724b400640 1 -- 172.21.15.59:0/785305298 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c011c40 con 0x7f725406da30 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.624+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72540c83e0 con 0x7f725406da30 2024-04-23T14:45:33.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.624+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72540c89a0 con 0x7f725406da30 2024-04-23T14:45:33.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.626+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7254068a80 con 0x7f725406da30 2024-04-23T14:45:33.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.627+0000 7f724b400640 1 -- 172.21.15.59:0/785305298 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f723c018020 con 0x7f725406da30 2024-04-23T14:45:33.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.628+0000 7f724b400640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f723006c8f0 0x7f723006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:33.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.628+0000 7f724b400640 1 -- 172.21.15.59:0/785305298 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f723c08b670 con 0x7f725406da30 2024-04-23T14:45:33.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.628+0000 7f7252000640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f723006c8f0 0x7f723006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:33.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.629+0000 7f7252000640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f723006c8f0 0x7f723006edb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f72541604d0 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:33.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.632+0000 7f724b400640 1 -- 172.21.15.59:0/785305298 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f723c01f030 con 0x7f725406da30 2024-04-23T14:45:33.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.876+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72540b3e60 con 0x7f725406da30 2024-04-23T14:45:33.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.877+0000 7f724b400640 1 -- 172.21.15.59:0/785305298 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+3871 (secure 0 0 0) 0x7f723c0172c0 con 0x7f725406da30 2024-04-23T14:45:33.879 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-04-23T14:45:33.879 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6904,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/4283096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4283096092},{"type":"v1","addr":"172.21.15.76:6833","nonce":4283096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":6907,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/4131892314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4131892314},{"type":"v1","addr":"172.21.15.76:6835","nonce":4131892314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1022907094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":1022907094},{"type":"v1","addr":"172.21.15.59:6835","nonce":1022907094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9957,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/1550929285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":1550929285},{"type":"v1","addr":"172.21.15.59:6837","nonce":1550929285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115}],"filesystems":[]} 2024-04-23T14:45:33.879 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 115 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f723006c8f0 msgr2=0x7f723006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f723006c8f0 0x7f723006edb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f72541604d0 tx=0x7f7244009040 comp rx=0 tx=0).stop 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 msgr2=0x7f72540c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540c7b20 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f723c00b3f0 tx=0x7f723c00b8b0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 shutdown_connections 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:6832/3176066673,v1:172.21.15.59:6833/3176066673] conn(0x7f723006c8f0 0x7f723006edb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f72540d0b70 0x7f72540cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72540d0370 0x7f7254159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 --2- 172.21.15.59:0/785305298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f725406da30 0x7f72540c7b20 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 >> 172.21.15.59:0/785305298 conn(0x7f72540b4080 msgr2=0x7f725406ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 shutdown_connections 2024-04-23T14:45:33.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-04-23T14:45:33.880+0000 7f7259e00640 1 -- 172.21.15.59:0/785305298 wait complete. 2024-04-23T14:45:33.902 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-23T14:45:33.902 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-23T14:45:33.903 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-23T14:45:33.903 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:33.922 INFO:tasks.ceph.mds.a:Stopped 2024-04-23T14:45:33.922 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-23T14:45:33.922 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:33.997 INFO:tasks.ceph.mds.c:Stopped 2024-04-23T14:45:33.997 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-23T14:45:33.997 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.023 INFO:tasks.ceph.mds.b:Stopped 2024-04-23T14:45:34.023 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-23T14:45:34.023 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.088 INFO:tasks.ceph.mds.d:Stopped 2024-04-23T14:45:34.089 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-23T14:45:34.089 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-23T14:45:34.089 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.171 INFO:tasks.ceph.osd.0:Stopped 2024-04-23T14:45:34.171 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-23T14:45:34.172 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.225 INFO:tasks.ceph.osd.1:Stopped 2024-04-23T14:45:34.226 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-23T14:45:34.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.281 INFO:tasks.ceph.osd.2:Stopped 2024-04-23T14:45:34.282 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-23T14:45:34.282 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.345 INFO:tasks.ceph.osd.3:Stopped 2024-04-23T14:45:34.345 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-23T14:45:34.345 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.427 INFO:tasks.ceph.osd.4:Stopped 2024-04-23T14:45:34.428 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-23T14:45:34.428 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.487 INFO:tasks.ceph.osd.5:Stopped 2024-04-23T14:45:34.488 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-23T14:45:34.488 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.560 INFO:tasks.ceph.osd.6:Stopped 2024-04-23T14:45:34.560 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-23T14:45:34.560 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.630 INFO:tasks.ceph.osd.7:Stopped 2024-04-23T14:45:34.630 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-23T14:45:34.630 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-23T14:45:34.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.722 INFO:tasks.ceph.mgr.y:Stopped 2024-04-23T14:45:34.722 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-23T14:45:34.722 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.800 INFO:tasks.ceph.mgr.x:Stopped 2024-04-23T14:45:34.800 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-23T14:45:34.800 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-23T14:45:34.800 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.838 INFO:tasks.ceph.mon.a:Stopped 2024-04-23T14:45:34.838 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-23T14:45:34.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.877 INFO:tasks.ceph.mon.b:Stopped 2024-04-23T14:45:34.878 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-23T14:45:34.878 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T14:45:34.957 INFO:tasks.ceph.mon.c:Stopped 2024-04-23T14:45:34.958 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-23T14:45:34.958 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-23T14:45:35.071 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:37:00.387871+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-04-23T14:45:35.072 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-23T14:45:35.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-23T14:45:35.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-23T14:45:35.337 INFO:teuthology.orchestra.run.smithi059.stdout:2024-04-23T14:39:09.755546+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-23T14:45:35.338 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:45:35.338 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-23T14:45:35.569 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:45:35.569 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-23T14:45:35.718 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:45:35.718 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-23T14:45:35.912 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:45:35.912 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-23T14:45:36.057 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:45:36.057 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-23T14:45:36.258 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:45:36.258 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-23T14:45:36.391 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:45:36.392 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-23T14:45:36.515 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:45:36.515 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-23T14:45:36.651 INFO:tasks.ceph:Archiving mon data... 2024-04-23T14:45:36.652 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/data/mon.a.tgz 2024-04-23T14:45:36.652 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-04-23T14:45:36.711 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.xlwhHGfD3N 2024-04-23T14:45:36.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xlwhHGfD3N 2024-04-23T14:45:36.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.xlwhHGfD3N 2024-04-23T14:45:37.040 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.xlwhHGfD3N is 395KB 2024-04-23T14:45:37.115 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.xlwhHGfD3N 2024-04-23T14:45:37.175 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/data/mon.b.tgz 2024-04-23T14:45:37.175 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-04-23T14:45:37.233 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.f6sFs1271G 2024-04-23T14:45:37.233 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.f6sFs1271G 2024-04-23T14:45:37.413 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.f6sFs1271G 2024-04-23T14:45:37.549 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.f6sFs1271G is 397KB 2024-04-23T14:45:37.656 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.f6sFs1271G 2024-04-23T14:45:37.711 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/data/mon.c.tgz 2024-04-23T14:45:37.711 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-04-23T14:45:37.761 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.51GfXlClUi 2024-04-23T14:45:37.761 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.51GfXlClUi 2024-04-23T14:45:37.931 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.51GfXlClUi 2024-04-23T14:45:38.067 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.51GfXlClUi is 397KB 2024-04-23T14:45:38.135 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.51GfXlClUi 2024-04-23T14:45:38.191 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-23T14:45:38.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T14:45:38.199 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T14:45:38.343 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T14:45:38.400 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T14:45:38.400 INFO:tasks.ceph:Archiving crash dumps... 2024-04-23T14:45:38.402 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/remote/smithi059/crash 2024-04-23T14:45:38.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T14:45:38.497 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/remote/smithi076/crash 2024-04-23T14:45:38.498 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T14:45:38.584 INFO:tasks.ceph:Shutting down logrotate 2024-04-23T14:45:38.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:45:38.592 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-23T14:45:38.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-23T14:45:38.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-23T14:45:38.885 INFO:tasks.ceph:Compressing logs... 2024-04-23T14:45:38.885 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T14:45:38.893 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T14:45:38.969 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T14:45:38.970 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39353.log 2024-04-23T14:45:38.971 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T14:45:38.973 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.3.39353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38830.log 2024-04-23T14:45:38.973 INFO:teuthology.orchestra.run.smithi076.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.3.39353.log.gz 2024-04-23T14:45:38.974 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-23T14:45:38.975 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.38830.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T14:45:38.975 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-23T14:45:38.975 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68532.log 2024-04-23T14:45:38.976 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.7.log: 90.7% -- replaced with /var/log/ceph/ceph-client.1.38830.log.gz 2024-04-23T14:45:38.976 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-23T14:45:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54202.log 2024-04-23T14:45:38.977 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33324.log 2024-04-23T14:45:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73194.log 2024-04-23T14:45:38.978 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68532.log.gz 2024-04-23T14:45:38.979 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.5.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T14:45:38.979 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-23T14:45:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75920.log 2024-04-23T14:45:38.980 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54202.log.gz 2024-04-23T14:45:38.981 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33404.log 2024-04-23T14:45:38.981 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33324.log.gz 2024-04-23T14:45:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71847.log 2024-04-23T14:45:38.981 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73194.log.gz 2024-04-23T14:45:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61658.log 2024-04-23T14:45:38.982 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75920.log.gz 2024-04-23T14:45:38.982 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33193.log 2024-04-23T14:45:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70752.log 2024-04-23T14:45:38.983 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71847.log.gz 2024-04-23T14:45:38.983 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33444.log 2024-04-23T14:45:38.983 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33404.log.gz 2024-04-23T14:45:38.984 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33364.log 2024-04-23T14:45:38.984 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75964.log 2024-04-23T14:45:38.984 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61658.log.gz 2024-04-23T14:45:38.985 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33193.log.gz 2024-04-23T14:45:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69229.log 2024-04-23T14:45:38.986 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70752.log.gz 2024-04-23T14:45:38.986 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33794.log 2024-04-23T14:45:38.986 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75964.log.gz 2024-04-23T14:45:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33444.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-23T14:45:38.987 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33444.log.gz 2024-04-23T14:45:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65879.log 2024-04-23T14:45:38.987 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69229.log.gz 2024-04-23T14:45:38.988 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33364.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-23T14:45:38.988 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33364.log.gz 2024-04-23T14:45:38.988 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61971.log 2024-04-23T14:45:38.988 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33794.log.gz 2024-04-23T14:45:38.989 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-23T14:45:38.989 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65879.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65879.log.gz 2024-04-23T14:45:38.989 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-04-23T14:45:38.990 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65452.log 2024-04-23T14:45:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33277.log 2024-04-23T14:45:38.991 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61971.log.gz 2024-04-23T14:45:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81635.log 2024-04-23T14:45:38.991 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-04-23T14:45:38.992 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42650.log 2024-04-23T14:45:38.993 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65452.log.gz 2024-04-23T14:45:38.993 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46919.log 2024-04-23T14:45:38.994 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.33277.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33277.log.gz 2024-04-23T14:45:38.994 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33484.log 2024-04-23T14:45:38.995 INFO:teuthology.orchestra.run.smithi059.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.81635.log.gz 2024-04-23T14:45:38.995 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46751.log 2024-04-23T14:45:38.995 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42650.log.gz 2024-04-23T14:45:38.995 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73237.log 2024-04-23T14:45:38.996 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46919.log.gz 2024-04-23T14:45:38.997 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33233.log 2024-04-23T14:45:38.997 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38910.log 2024-04-23T14:45:38.997 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46751.log.gz 2024-04-23T14:45:38.998 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40096.log 2024-04-23T14:45:38.998 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73237.log.gz 2024-04-23T14:45:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81887.log 2024-04-23T14:45:38.999 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33233.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33484.log.gz 2024-04-23T14:45:39.000 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38910.log.gz 2024-04-23T14:45:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54012.log 2024-04-23T14:45:39.000 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40096.log.gz 2024-04-23T14:45:39.001 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-23T14:45:39.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52494.log 88.5% 2024-04-23T14:45:39.001 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81887.log.gz 2024-04-23T14:45:39.002 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-23T14:45:39.003 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.54012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49547.log 2024-04-23T14:45:39.003 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-23T14:45:39.004 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52494.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.52494.log.gz 2024-04-23T14:45:39.004 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45042.log 2024-04-23T14:45:39.005 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50460.log 2024-04-23T14:45:39.005 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49547.log.gz 2024-04-23T14:45:39.005 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.d.log: -- replaced with /var/log/ceph/ceph-client.admin.33233.log.gz 2024-04-23T14:45:39.005 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50250.log 2024-04-23T14:45:39.006 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45042.log.gz 2024-04-23T14:45:39.006 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.54012.log.gz 2024-04-23T14:45:39.006 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58129.log 2024-04-23T14:45:39.007 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50460.log.gz 2024-04-23T14:45:39.007 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50250.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.50250.log.gz -- 2024-04-23T14:45:39.008 INFO:teuthology.orchestra.run.smithi059.stderr: /var/log/ceph/ceph-client.admin.62097.log 2024-04-23T14:45:39.008 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49877.log 2024-04-23T14:45:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62097.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58129.log.gz 2024-04-23T14:45:39.009 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58433.log 2024-04-23T14:45:39.010 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62097.log.gz 2024-04-23T14:45:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62479.log 2024-04-23T14:45:39.011 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49877.log.gz 2024-04-23T14:45:39.011 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64444.log 2024-04-23T14:45:39.012 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58433.log.gz 2024-04-23T14:45:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62479.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62479.log.gz 2024-04-23T14:45:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70837.log 2024-04-23T14:45:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73658.log 2024-04-23T14:45:39.014 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64444.log.gz 2024-04-23T14:45:39.015 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53416.log 2024-04-23T14:45:39.015 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73658.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.70837.log.gz 2024-04-23T14:45:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56974.log 2024-04-23T14:45:39.016 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73658.log.gz 2024-04-23T14:45:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50334.log 2024-04-23T14:45:39.017 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53416.log.gz 2024-04-23T14:45:39.018 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82013.log 2024-04-23T14:45:39.018 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56974.log.gz 2024-04-23T14:45:39.019 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59522.log 2024-04-23T14:45:39.019 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50334.log.gz 2024-04-23T14:45:39.020 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.82013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48017.log 2024-04-23T14:45:39.020 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82013.log.gz 2024-04-23T14:45:39.021 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60879.log 2024-04-23T14:45:39.021 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59522.log.gz 2024-04-23T14:45:39.022 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68261.log 2024-04-23T14:45:39.022 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48017.log.gz 2024-04-23T14:45:39.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.60879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80266.log 2024-04-23T14:45:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68261.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68261.log.gz 2024-04-23T14:45:39.024 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48719.log 2024-04-23T14:45:39.025 INFO:teuthology.orchestra.run.smithi059.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.0.60879.log.gz 2024-04-23T14:45:39.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78327.log 2024-04-23T14:45:39.026 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-23T14:45:39.026 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80266.log.gz 2024-04-23T14:45:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73279.log 2024-04-23T14:45:39.027 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48719.log.gz 2024-04-23T14:45:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82159.log 2024-04-23T14:45:39.028 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78327.log.gz 2024-04-23T14:45:39.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44502.log 2024-04-23T14:45:39.029 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73279.log.gz 2024-04-23T14:45:39.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.82159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45794.log 2024-04-23T14:45:39.030 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82159.log.gz 2024-04-23T14:45:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77536.log 2024-04-23T14:45:39.031 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44502.log.gz 2024-04-23T14:45:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68574.log 2024-04-23T14:45:39.031 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45794.log.gz 2024-04-23T14:45:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52620.log 2024-04-23T14:45:39.033 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.77536.log.gz 2024-04-23T14:45:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71763.log 2024-04-23T14:45:39.034 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68574.log.gz 2024-04-23T14:45:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59438.log 2024-04-23T14:45:39.035 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52620.log.gz 2024-04-23T14:45:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71763.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71763.log.gz 2024-04-23T14:45:39.036 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53542.log 2024-04-23T14:45:39.037 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44202.log 2024-04-23T14:45:39.037 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59438.log.gz 2024-04-23T14:45:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39413.log 2024-04-23T14:45:39.038 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53542.log.gz 2024-04-23T14:45:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44202.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44202.log.gz 2024-04-23T14:45:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79164.log 2024-04-23T14:45:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42735.log 2024-04-23T14:45:39.040 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39413.log.gz 2024-04-23T14:45:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52223.log 2024-04-23T14:45:39.042 INFO:teuthology.orchestra.run.smithi076.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-23T14:45:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42735.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42735.log.gz 2024-04-23T14:45:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55823.log 2024-04-23T14:45:39.043 INFO:teuthology.orchestra.run.smithi059.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.79164.log.gz 2024-04-23T14:45:39.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52325.log 2024-04-23T14:45:39.043 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52223.log.gz 2024-04-23T14:45:39.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46877.log 2024-04-23T14:45:39.044 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55823.log.gz 2024-04-23T14:45:39.045 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79423.log 2024-04-23T14:45:39.045 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52325.log.gz 2024-04-23T14:45:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44628.log 2024-04-23T14:45:39.046 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46877.log.gz 2024-04-23T14:45:39.047 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56413.log 2024-04-23T14:45:39.047 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79423.log.gz 2024-04-23T14:45:39.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52265.log 2024-04-23T14:45:39.048 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44628.log.gz 2024-04-23T14:45:39.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45168.log 2024-04-23T14:45:39.050 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56413.log.gz 2024-04-23T14:45:39.050 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48381.log 2024-04-23T14:45:39.051 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52265.log.gz 2024-04-23T14:45:39.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71805.log 2024-04-23T14:45:39.052 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45168.log.gz 2024-04-23T14:45:39.052 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68345.log 2024-04-23T14:45:39.053 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48381.log.gz 2024-04-23T14:45:39.053 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40098.log 2024-04-23T14:45:39.054 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71805.log.gz 2024-04-23T14:45:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70448.log 2024-04-23T14:45:39.055 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68345.log.gz 2024-04-23T14:45:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74916.log 2024-04-23T14:45:39.056 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40098.log.gz 2024-04-23T14:45:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53286.log 2024-04-23T14:45:39.057 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70448.log.gz 2024-04-23T14:45:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62398.log 2024-04-23T14:45:39.058 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74916.log.gz 2024-04-23T14:45:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68760.log 2024-04-23T14:45:39.059 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53286.log.gz 2024-04-23T14:45:39.059 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48455.log 2024-04-23T14:45:39.060 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62398.log.gz 2024-04-23T14:45:39.061 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65963.log 2024-04-23T14:45:39.061 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68760.log.gz 2024-04-23T14:45:39.062 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-04-23T14:45:39.062 INFO:teuthology.orchestra.run.smithi059.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.48455.log.gz 2024-04-23T14:45:39.063 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51165.log 2024-04-23T14:45:39.063 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65963.log.gz 2024-04-23T14:45:39.064 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76693.log 2024-04-23T14:45:39.064 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-04-23T14:45:39.065 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70893.log 2024-04-23T14:45:39.065 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51165.log.gz 2024-04-23T14:45:39.066 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-04-23T14:45:39.066 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76693.log.gz 2024-04-23T14:45:39.067 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69273.log 2024-04-23T14:45:39.067 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70893.log.gz 2024-04-23T14:45:39.068 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75804.log 2024-04-23T14:45:39.068 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-04-23T14:45:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68616.log 2024-04-23T14:45:39.069 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69273.log.gz 2024-04-23T14:45:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54870.log 2024-04-23T14:45:39.070 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75804.log.gz 2024-04-23T14:45:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64111.log 2024-04-23T14:45:39.071 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68616.log.gz 2024-04-23T14:45:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40091.log 2024-04-23T14:45:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64111.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54870.log.gz 2024-04-23T14:45:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38198.log 2024-04-23T14:45:39.073 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64111.log.gz 2024-04-23T14:45:39.074 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58706.log 2024-04-23T14:45:39.074 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40091.log.gz 2024-04-23T14:45:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43113.log 2024-04-23T14:45:39.075 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38198.log.gz 2024-04-23T14:45:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51929.log 2024-04-23T14:45:39.076 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58706.log.gz 2024-04-23T14:45:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69814.log 2024-04-23T14:45:39.077 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43113.log.gz 2024-04-23T14:45:39.078 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48101.log 2024-04-23T14:45:39.078 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51929.log.gz 2024-04-23T14:45:39.079 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69814.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.69814.log.gz --verbose 2024-04-23T14:45:39.079 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.46961.log 2024-04-23T14:45:39.080 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78869.log 2024-04-23T14:45:39.080 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48101.log.gz 2024-04-23T14:45:39.081 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46961.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46961.log.gz 2024-04-23T14:45:39.082 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53332.log 2024-04-23T14:45:39.082 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78832.log 2024-04-23T14:45:39.082 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78869.log.gz 2024-04-23T14:45:39.083 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49013.log 2024-04-23T14:45:39.084 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53332.log.gz 2024-04-23T14:45:39.084 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74660.log 2024-04-23T14:45:39.084 INFO:teuthology.orchestra.run.smithi059.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.78832.log.gz 2024-04-23T14:45:39.086 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65128.log 2024-04-23T14:45:39.086 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49013.log.gz 2024-04-23T14:45:39.086 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79014.log 2024-04-23T14:45:39.087 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74660.log.gz 2024-04-23T14:45:39.087 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69059.log 2024-04-23T14:45:39.088 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7%/var/log/ceph/ceph-client.admin.79014.log: -- replaced with /var/log/ceph/ceph-client.admin.65128.log.gz 2024-04-23T14:45:39.088 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59144.log 2024-04-23T14:45:39.089 INFO:teuthology.orchestra.run.smithi059.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.79014.log.gz 2024-04-23T14:45:39.089 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69059.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-23T14:45:39.090 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69059.log.gz 2024-04-23T14:45:39.090 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.59144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39246.log 2024-04-23T14:45:39.091 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73363.log 2024-04-23T14:45:39.092 INFO:teuthology.orchestra.run.smithi059.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.59144.log.gz 2024-04-23T14:45:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53063.log 2024-04-23T14:45:39.093 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39246.log.gz 2024-04-23T14:45:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61574.log 2024-04-23T14:45:39.094 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-23T14:45:39.094 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73363.log.gz 2024-04-23T14:45:39.095 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61151.log 2024-04-23T14:45:39.095 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53063.log.gz 2024-04-23T14:45:39.096 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61574.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61574.log.gz 2024-04-23T14:45:39.096 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40380.log 2024-04-23T14:45:39.097 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40674.log 2024-04-23T14:45:39.097 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61151.log.gz 2024-04-23T14:45:39.098 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-04-23T14:45:39.098 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40380.log.gz 2024-04-23T14:45:39.100 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56536.log 2024-04-23T14:45:39.100 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40674.log.gz 2024-04-23T14:45:39.100 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78795.log 2024-04-23T14:45:39.100 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-04-23T14:45:39.101 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39420.log 2024-04-23T14:45:39.101 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56536.log.gz 2024-04-23T14:45:39.102 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81509.log 2024-04-23T14:45:39.102 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78795.log.gz 2024-04-23T14:45:39.104 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39420.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39420.log.gz 2024-04-23T14:45:39.104 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-04-23T14:45:39.105 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53668.log 2024-04-23T14:45:39.105 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81509.log.gz 2024-04-23T14:45:39.106 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62181.log 2024-04-23T14:45:39.106 INFO:teuthology.orchestra.run.smithi059.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-04-23T14:45:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49326.log 2024-04-23T14:45:39.107 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53668.log.gz 2024-04-23T14:45:39.108 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65837.log 2024-04-23T14:45:39.108 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62181.log.gz 2024-04-23T14:45:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61406.log 2024-04-23T14:45:39.109 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49326.log.gz 2024-04-23T14:45:39.110 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2024-04-23T14:45:39.110 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65837.log.gz 2024-04-23T14:45:39.111 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55883.log 2024-04-23T14:45:39.111 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61406.log.gz 2024-04-23T14:45:39.112 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70632.log 2024-04-23T14:45:39.112 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2024-04-23T14:45:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55883.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55883.log.gz 2024-04-23T14:45:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62904.log 2024-04-23T14:45:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73406.log 2024-04-23T14:45:39.115 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70632.log.gz 2024-04-23T14:45:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62904.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62904.log.gz 2024-04-23T14:45:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70364.log 2024-04-23T14:45:39.117 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39421.log 2024-04-23T14:45:39.117 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73406.log.gz 2024-04-23T14:45:39.118 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78595.log 2024-04-23T14:45:39.119 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70364.log.gz 2024-04-23T14:45:39.119 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39421.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39421.log.gz 2024-04-23T14:45:39.119 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46495.log 2024-04-23T14:45:39.120 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40590.log 2024-04-23T14:45:39.120 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78595.log.gz 2024-04-23T14:45:39.122 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78411.log 2024-04-23T14:45:39.122 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46495.log.gz 2024-04-23T14:45:39.122 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39078.log 2024-04-23T14:45:39.123 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40590.log.gz 2024-04-23T14:45:39.124 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50994.log 2024-04-23T14:45:39.124 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78411.log.gz 2024-04-23T14:45:39.124 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39078.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-23T14:45:39.125 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39078.log.gz 2024-04-23T14:45:39.126 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76862.log 2024-04-23T14:45:39.126 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50994.log.gz 2024-04-23T14:45:39.127 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 2024-04-23T14:45:39.128 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72553.log 2024-04-23T14:45:39.128 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76862.log.gz 2024-04-23T14:45:39.129 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66672.log 2024-04-23T14:45:39.129 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2024-04-23T14:45:39.130 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48971.log 2024-04-23T14:45:39.130 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72553.log.gz 2024-04-23T14:45:39.131 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72939.log 2024-04-23T14:45:39.132 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66672.log.gz 2024-04-23T14:45:39.132 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64497.log 2024-04-23T14:45:39.132 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48971.log.gz 2024-04-23T14:45:39.133 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71171.log 2024-04-23T14:45:39.134 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72939.log.gz 2024-04-23T14:45:39.134 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71658.log 2024-04-23T14:45:39.135 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64497.log.gz 2024-04-23T14:45:39.135 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.71171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67688.log 2024-04-23T14:45:39.137 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68658.log 2024-04-23T14:45:39.137 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71658.log.gz 2024-04-23T14:45:39.138 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56932.log 2024-04-23T14:45:39.138 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67688.log.gz 2024-04-23T14:45:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68658.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68658.log.gz 2024-04-23T14:45:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67967.log 2024-04-23T14:45:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60018.log 2024-04-23T14:45:39.140 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56932.log.gz 2024-04-23T14:45:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67967.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67967.log.gz 2024-04-23T14:45:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63783.log 2024-04-23T14:45:39.143 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60018.log: 94.2% -- replaced with /var/log/ceph/ceph-client.0.71171.log.gz 2024-04-23T14:45:39.143 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68855.log 2024-04-23T14:45:39.144 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63783.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.60018.log.gz 2024-04-23T14:45:39.144 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56333.log 2024-04-23T14:45:39.144 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63783.log.gz 2024-04-23T14:45:39.145 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41461.log 2024-04-23T14:45:39.145 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68855.log.gz 2024-04-23T14:45:39.146 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52961.log 2024-04-23T14:45:39.146 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56333.log.gz 2024-04-23T14:45:39.147 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.41461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53584.log 2024-04-23T14:45:39.148 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52961.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.41461.log.gz 2024-04-23T14:45:39.148 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64628.log 2024-04-23T14:45:39.148 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52961.log.gz 2024-04-23T14:45:39.149 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38740.log 2024-04-23T14:45:39.150 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53584.log.gz 2024-04-23T14:45:39.150 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-04-23T14:45:39.151 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38740.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64628.log.gz 2024-04-23T14:45:39.152 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70322.log 2024-04-23T14:45:39.152 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38740.log.gz 2024-04-23T14:45:39.153 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69580.log 2024-04-23T14:45:39.153 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-04-23T14:45:39.154 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51037.log 2024-04-23T14:45:39.154 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70322.log.gz 2024-04-23T14:45:39.155 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70814.log 2024-04-23T14:45:39.155 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69580.log.gz 2024-04-23T14:45:39.156 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72768.log 2024-04-23T14:45:39.156 INFO:teuthology.orchestra.run.smithi059.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51037.log.gz 2024-04-23T14:45:39.157 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40054.log 2024-04-23T14:45:39.157 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70814.log.gz 2024-04-23T14:45:39.158 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33529.log 2024-04-23T14:45:39.158 INFO:teuthology.orchestra.run.smithi059.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.72768.log.gz 2024-04-23T14:45:39.159 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62862.log 2024-04-23T14:45:39.159 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40054.log.gz 2024-04-23T14:45:39.160 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80606.log 2024-04-23T14:45:39.161 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33529.log.gz 2024-04-23T14:45:39.162 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50526.log 2024-04-23T14:45:39.162 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62862.log.gz 2024-04-23T14:45:39.163 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80606.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80606.log.gz 2024-04-23T14:45:39.163 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49284.log 2024-04-23T14:45:39.164 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80140.log 2024-04-23T14:45:39.164 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50526.log.gz 2024-04-23T14:45:39.165 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39418.log 2024-04-23T14:45:39.165 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49284.log.gz 2024-04-23T14:45:39.166 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46170.log 2024-04-23T14:45:39.166 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80140.log.gz 2024-04-23T14:45:39.167 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77406.log 2024-04-23T14:45:39.167 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39418.log.gz 2024-04-23T14:45:39.168 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75000.log 2024-04-23T14:45:39.168 INFO:teuthology.orchestra.run.smithi059.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46170.log.gz 2024-04-23T14:45:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80858.log 2024-04-23T14:45:39.170 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77406.log.gz 2024-04-23T14:45:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46254.log 2024-04-23T14:45:39.171 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.75000.log.gz 2024-04-23T14:45:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61109.log 2024-04-23T14:45:39.172 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80858.log.gz 2024-04-23T14:45:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42987.log 2024-04-23T14:45:39.173 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46254.log.gz 2024-04-23T14:45:39.174 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46212.log 2024-04-23T14:45:39.174 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61109.log.gz 2024-04-23T14:45:39.175 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-04-23T14:45:39.175 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42987.log.gz 2024-04-23T14:45:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69748.log 2024-04-23T14:45:39.176 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46212.log.gz 2024-04-23T14:45:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38868.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-23T14:45:39.177 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-04-23T14:45:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62564.log 2024-04-23T14:45:39.179 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69748.log.gz 2024-04-23T14:45:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55781.log 2024-04-23T14:45:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56112.log 2024-04-23T14:45:39.181 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8%/var/log/ceph/ceph-client.admin.55781.log: -- replaced with /var/log/ceph/ceph-client.admin.62564.log.gz 2024-04-23T14:45:39.182 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55781.log.gz 2024-04-23T14:45:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71889.log 2024-04-23T14:45:39.183 INFO:teuthology.orchestra.run.smithi059.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-23T14:45:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66760.log 2024-04-23T14:45:39.184 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56112.log.gz 2024-04-23T14:45:39.184 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71889.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71889.log.gz 2024-04-23T14:45:39.185 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42471.log 2024-04-23T14:45:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49368.log 2024-04-23T14:45:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42471.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66760.log.gz 2024-04-23T14:45:39.187 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42471.log.gz 2024-04-23T14:45:39.187 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75678.log 2024-04-23T14:45:39.188 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79377.log 2024-04-23T14:45:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75678.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49368.log.gz 2024-04-23T14:45:39.189 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75678.log.gz 2024-04-23T14:45:39.189 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82241.log 2024-04-23T14:45:39.190 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62820.log 2024-04-23T14:45:39.191 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79377.log.gz 2024-04-23T14:45:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.82241.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82241.log.gz 2024-04-23T14:45:39.192 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65194.log 2024-04-23T14:45:39.193 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45504.log 2024-04-23T14:45:39.193 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62820.log.gz 2024-04-23T14:45:39.194 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64153.log 2024-04-23T14:45:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45504.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65194.log.gz 2024-04-23T14:45:39.195 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45504.log.gz 2024-04-23T14:45:39.195 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61448.log 2024-04-23T14:45:39.196 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49966.log 2024-04-23T14:45:39.197 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64153.log.gz 2024-04-23T14:45:39.198 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43239.log 2024-04-23T14:45:39.198 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61448.log.gz 2024-04-23T14:45:39.199 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49966.log.gz 2024-04-23T14:45:39.199 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51650.log 2024-04-23T14:45:39.200 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66147.log 2024-04-23T14:45:39.200 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43239.log.gz 2024-04-23T14:45:39.201 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40089.log 2024-04-23T14:45:39.201 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51650.log.gz 2024-04-23T14:45:39.202 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78553.log 2024-04-23T14:45:39.202 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66147.log.gz 2024-04-23T14:45:39.203 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63114.log 2024-04-23T14:45:39.203 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40089.log.gz 2024-04-23T14:45:39.205 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38612.log 2024-04-23T14:45:39.205 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78553.log.gz 2024-04-23T14:45:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54612.log 2024-04-23T14:45:39.206 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63114.log.gz 2024-04-23T14:45:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52013.log 2024-04-23T14:45:39.207 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38612.log.gz 2024-04-23T14:45:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76920.log 2024-04-23T14:45:39.209 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52013.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54612.log.gz 2024-04-23T14:45:39.209 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60192.log 2024-04-23T14:45:39.209 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52013.log.gz 2024-04-23T14:45:39.210 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39762.log 2024-04-23T14:45:39.210 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76920.log.gz 2024-04-23T14:45:39.211 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69101.log 2024-04-23T14:45:39.211 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60192.log.gz 2024-04-23T14:45:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-04-23T14:45:39.212 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39762.log.gz 2024-04-23T14:45:39.213 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2024-04-23T14:45:39.213 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69101.log.gz 2024-04-23T14:45:39.214 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65755.log 2024-04-23T14:45:39.215 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2024-04-23T14:45:39.215 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66063.log 2024-04-23T14:45:39.216 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2024-04-23T14:45:39.216 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67501.log 2024-04-23T14:45:39.217 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65755.log.gz 2024-04-23T14:45:39.218 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80774.log 2024-04-23T14:45:39.218 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66063.log.gz 2024-04-23T14:45:39.219 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2024-04-23T14:45:39.219 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.67501.log.gz 2024-04-23T14:45:39.220 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48887.log 2024-04-23T14:45:39.220 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80774.log.gz 2024-04-23T14:45:39.221 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80961.log 2024-04-23T14:45:39.221 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2024-04-23T14:45:39.222 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71561.log 88.6% 2024-04-23T14:45:39.222 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48887.log.gz 2024-04-23T14:45:39.223 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2024-04-23T14:45:39.223 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80961.log.gz 2024-04-23T14:45:39.224 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82055.log 2024-04-23T14:45:39.224 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71561.log.gz 2024-04-23T14:45:39.225 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68093.log 2024-04-23T14:45:39.225 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2024-04-23T14:45:39.227 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.82055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80437.log 2024-04-23T14:45:39.227 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82055.log.gz 2024-04-23T14:45:39.228 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52055.log 2024-04-23T14:45:39.228 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68093.log.gz 2024-04-23T14:45:39.229 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46086.log 2024-04-23T14:45:39.229 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80437.log.gz 2024-04-23T14:45:39.230 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46128.log 2024-04-23T14:45:39.230 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52055.log.gz 2024-04-23T14:45:39.231 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65386.log 2024-04-23T14:45:39.231 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46086.log.gz 2024-04-23T14:45:39.232 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54720.log 2024-04-23T14:45:39.232 INFO:teuthology.orchestra.run.smithi059.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46128.log.gz 2024-04-23T14:45:39.233 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42777.log 2024-04-23T14:45:39.234 INFO:teuthology.orchestra.run.smithi059.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.65386.log.gz 2024-04-23T14:45:39.234 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54720.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-23T14:45:39.235 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54720.log.gz 2024-04-23T14:45:39.235 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74958.log 2024-04-23T14:45:39.236 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42777.log.gz 2024-04-23T14:45:39.237 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2024-04-23T14:45:39.238 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38696.log 2024-04-23T14:45:39.238 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74958.log.gz 2024-04-23T14:45:39.239 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52410.log 2024-04-23T14:45:39.239 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2024-04-23T14:45:39.240 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39428.log 2024-04-23T14:45:39.240 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38696.log.gz 2024-04-23T14:45:39.242 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76778.log 2024-04-23T14:45:39.242 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2024-04-23T14:45:39.243 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62692.log 2024-04-23T14:45:39.243 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39428.log.gz 2024-04-23T14:45:39.244 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2024-04-23T14:45:39.244 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76778.log.gz 2024-04-23T14:45:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-04-23T14:45:39.246 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62692.log.gz 2024-04-23T14:45:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-04-23T14:45:39.247 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2024-04-23T14:45:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67459.log 2024-04-23T14:45:39.248 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz 2024-04-23T14:45:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52578.log 2024-04-23T14:45:39.250 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-04-23T14:45:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68700.log 2024-04-23T14:45:39.251 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67459.log.gz 2024-04-23T14:45:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51414.log 2024-04-23T14:45:39.252 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52578.log.gz 2024-04-23T14:45:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73784.log 2024-04-23T14:45:39.254 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68700.log.gz 2024-04-23T14:45:39.255 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54298.log 2024-04-23T14:45:39.255 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51414.log.gz 2024-04-23T14:45:39.257 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75636.log 2024-04-23T14:45:39.257 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73784.log.gz 2024-04-23T14:45:39.258 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66328.log 2024-04-23T14:45:39.258 INFO:teuthology.orchestra.run.smithi059.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-04-23T14:45:39.259 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80732.log 2024-04-23T14:45:39.259 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75636.log.gz 2024-04-23T14:45:39.261 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71721.log 2024-04-23T14:45:39.261 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66328.log.gz 2024-04-23T14:45:39.262 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64540.log 2024-04-23T14:45:39.262 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80732.log.gz 2024-04-23T14:45:39.263 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67567.log 2024-04-23T14:45:39.263 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71721.log.gz 2024-04-23T14:45:39.265 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48929.log 2024-04-23T14:45:39.265 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64540.log.gz 2024-04-23T14:45:39.266 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69017.log 2024-04-23T14:45:39.266 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67567.log.gz 2024-04-23T14:45:39.267 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53374.log 2024-04-23T14:45:39.268 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48929.log.gz 2024-04-23T14:45:39.269 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51608.log 2024-04-23T14:45:39.269 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69017.log.gz 2024-04-23T14:45:39.270 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50208.log 2024-04-23T14:45:39.270 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53374.log.gz 2024-04-23T14:45:39.271 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73700.log 2024-04-23T14:45:39.272 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51608.log.gz 2024-04-23T14:45:39.273 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81245.log 2024-04-23T14:45:39.273 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50208.log.gz 2024-04-23T14:45:39.274 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81929.log 2024-04-23T14:45:39.274 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73700.log.gz 2024-04-23T14:45:39.275 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55613.log 2024-04-23T14:45:39.275 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81245.log.gz 2024-04-23T14:45:39.276 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74618.log 2024-04-23T14:45:39.276 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81929.log.gz 2024-04-23T14:45:39.278 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80918.log 2024-04-23T14:45:39.278 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55613.log.gz 2024-04-23T14:45:39.279 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44958.log 2024-04-23T14:45:39.279 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74618.log.gz 2024-04-23T14:45:39.280 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40716.log 2024-04-23T14:45:39.280 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80918.log.gz 2024-04-23T14:45:39.281 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56494.log 2024-04-23T14:45:39.281 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44958.log.gz 2024-04-23T14:45:39.283 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40716.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40716.log.gz 2024-04-23T14:45:39.283 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57961.log 2024-04-23T14:45:39.284 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69366.log 2024-04-23T14:45:39.284 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56494.log.gz 2024-04-23T14:45:39.286 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39427.log 2024-04-23T14:45:39.286 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57961.log.gz 2024-04-23T14:45:39.287 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51090.log 2024-04-23T14:45:39.287 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69366.log.gz 2024-04-23T14:45:39.288 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65278.log 2024-04-23T14:45:39.288 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39427.log.gz 2024-04-23T14:45:39.289 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61490.log 2024-04-23T14:45:39.289 INFO:teuthology.orchestra.run.smithi059.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51090.log.gz 2024-04-23T14:45:39.291 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45672.log 2024-04-23T14:45:39.291 INFO:teuthology.orchestra.run.smithi059.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.65278.log.gz 2024-04-23T14:45:39.292 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56844.log 2024-04-23T14:45:39.292 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61490.log.gz 2024-04-23T14:45:39.293 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42601.log 2024-04-23T14:45:39.294 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45672.log.gz 2024-04-23T14:45:39.295 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79465.log 2024-04-23T14:45:39.295 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56844.log.gz 2024-04-23T14:45:39.296 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42601.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-23T14:45:39.296 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42601.log.gz 2024-04-23T14:45:39.297 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38994.log 2024-04-23T14:45:39.298 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.79465.log.gz 2024-04-23T14:45:39.299 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43560.log 2024-04-23T14:45:39.300 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80312.log 2024-04-23T14:45:39.301 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38994.log: /var/log/ceph/ceph-client.admin.43560.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38994.log.gz 2024-04-23T14:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48143.log 2024-04-23T14:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43560.log.gz 2024-04-23T14:45:39.303 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2024-04-23T14:45:39.304 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80312.log.gz 2024-04-23T14:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55739.log 2024-04-23T14:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48143.log.gz 2024-04-23T14:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49200.log 2024-04-23T14:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2024-04-23T14:45:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55739.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55739.log.gz 2024-04-23T14:45:39.308 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65713.log 2024-04-23T14:45:39.309 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54678.log 2024-04-23T14:45:39.309 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65713.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49200.log.gz 2024-04-23T14:45:39.310 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65713.log.gzgzip 2024-04-23T14:45:39.310 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71519.log 2024-04-23T14:45:39.312 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54678.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54678.log.gz 2024-04-23T14:45:39.312 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39204.log 2024-04-23T14:45:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74437.log 2024-04-23T14:45:39.313 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39204.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71519.log.gz 2024-04-23T14:45:39.314 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39204.log.gzgzip 2024-04-23T14:45:39.314 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59664.log 2024-04-23T14:45:39.316 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74437.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74437.log.gz 2024-04-23T14:45:39.316 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57394.log 2024-04-23T14:45:39.317 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69664.log 2024-04-23T14:45:39.317 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59664.log.gz 2024-04-23T14:45:39.318 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46541.log 2024-04-23T14:45:39.319 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57394.log.gz 2024-04-23T14:45:39.320 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39415.log 2024-04-23T14:45:39.320 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69664.log.gz 2024-04-23T14:45:39.321 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69185.log 2024-04-23T14:45:39.321 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46541.log.gz 2024-04-23T14:45:39.322 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2024-04-23T14:45:39.322 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39415.log.gz 2024-04-23T14:45:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69185.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69185.log.gz 2024-04-23T14:45:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44318.log 2024-04-23T14:45:39.325 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51164.log 2024-04-23T14:45:39.326 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2024-04-23T14:45:39.326 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62521.log 2024-04-23T14:45:39.327 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44318.log.gz 2024-04-23T14:45:39.328 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51164.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-23T14:45:39.328 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51164.log.gz 2024-04-23T14:45:39.329 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58664.log 2024-04-23T14:45:39.329 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62521.log.gz 2024-04-23T14:45:39.330 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62223.log 2024-04-23T14:45:39.332 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60493.log 2024-04-23T14:45:39.332 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58664.log.gz 2024-04-23T14:45:39.333 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76192.log 2024-04-23T14:45:39.334 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62223.log.gz 2024-04-23T14:45:39.334 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60493.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-23T14:45:39.335 INFO:teuthology.orchestra.run.smithi059.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.60493.log.gz 2024-04-23T14:45:39.336 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69706.log 2024-04-23T14:45:39.336 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76192.log.gz 2024-04-23T14:45:39.337 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74790.log 2024-04-23T14:45:39.339 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-04-23T14:45:39.339 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69706.log.gz 2024-04-23T14:45:39.340 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74790.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74790.log.gz 2024-04-23T14:45:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49452.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-04-23T14:45:39.342 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40338.log 2024-04-23T14:45:39.343 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55926.log 2024-04-23T14:45:39.345 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2024-04-23T14:45:39.345 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40338.log.gz 2024-04-23T14:45:39.346 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65921.log 2024-04-23T14:45:39.346 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55926.log.gz 2024-04-23T14:45:39.347 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43992.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-04-23T14:45:39.348 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.65921.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73970.log 2024-04-23T14:45:39.349 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65921.log.gz 2024-04-23T14:45:39.349 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-23T14:45:39.351 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66189.log 2024-04-23T14:45:39.351 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73970.log.gz 2024-04-23T14:45:39.354 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.66189.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66189.log.gz 2024-04-23T14:45:39.354 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53105.log 2024-04-23T14:45:39.356 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51348.log 2024-04-23T14:45:39.358 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53105.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53105.log.gz 2024-04-23T14:45:39.358 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73532.log 2024-04-23T14:45:39.360 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51348.log: /var/log/ceph/ceph-client.admin.73532.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51348.log.gz 2024-04-23T14:45:39.360 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73532.log.gz 2024-04-23T14:45:39.361 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-04-23T14:45:39.362 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66576.log 2024-04-23T14:45:39.364 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42692.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-04-23T14:45:39.364 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40100.log 2024-04-23T14:45:39.365 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66576.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66576.log.gz 2024-04-23T14:45:39.365 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-23T14:45:39.366 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78637.log 2024-04-23T14:45:39.367 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40100.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40100.log.gz 2024-04-23T14:45:39.368 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72168.log 2024-04-23T14:45:39.369 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78637.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78637.log.gz 2024-04-23T14:45:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46340.log 2024-04-23T14:45:39.372 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72168.log.gz 2024-04-23T14:45:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70280.log 2024-04-23T14:45:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46340.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46340.log.gz 2024-04-23T14:45:39.375 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40422.log 2024-04-23T14:45:39.376 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70280.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70280.log.gz 2024-04-23T14:45:39.377 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55486.log 2024-04-23T14:45:39.379 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40422.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40422.log.gz 2024-04-23T14:45:39.380 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55486.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55486.log.gz 2024-04-23T14:45:39.380 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40632.log 2024-04-23T14:45:39.382 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78469.log 2024-04-23T14:45:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40632.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40632.log.gz 2024-04-23T14:45:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26372.log 2024-04-23T14:45:39.385 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78469.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78469.log.gz 2024-04-23T14:45:39.386 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46296.log 2024-04-23T14:45:39.386 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.tmp-client.admin.26372.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26372.log.gz 2024-04-23T14:45:39.387 INFO:teuthology.orchestra.run.smithi076.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-23T14:45:39.387 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50166.log 2024-04-23T14:45:39.389 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46296.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46296.log.gz 2024-04-23T14:45:39.390 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50060.log 2024-04-23T14:45:39.390 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50166.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50166.log.gz 2024-04-23T14:45:39.391 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38485.log 2024-04-23T14:45:39.392 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50060.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50060.log.gz 2024-04-23T14:45:39.393 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67243.log 2024-04-23T14:45:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38485.log.gz 2024-04-23T14:45:39.395 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80816.log 2024-04-23T14:45:39.397 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67243.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.67243.log.gz 2024-04-23T14:45:39.398 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80816.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80816.log.gz 2024-04-23T14:45:39.398 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44748.log 2024-04-23T14:45:39.400 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70406.log 2024-04-23T14:45:39.402 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44748.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44748.log.gz 2024-04-23T14:45:39.402 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40884.log 2024-04-23T14:45:39.403 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70406.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70406.log.gz 2024-04-23T14:45:39.404 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79122.log 2024-04-23T14:45:39.406 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40884.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40884.log.gz 2024-04-23T14:45:39.406 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39753.log 2024-04-23T14:45:39.409 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79122.log: /var/log/ceph/ceph-client.admin.39753.log: 91.8% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39753.log.gz 2024-04-23T14:45:39.409 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79122.log.gz 2024-04-23T14:45:39.409 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67309.log 2024-04-23T14:45:39.411 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45084.log 2024-04-23T14:45:39.413 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67309.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.67309.log.gz 2024-04-23T14:45:39.413 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59622.log 2024-04-23T14:45:39.414 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45084.log.gz 2024-04-23T14:45:39.415 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67814.log 2024-04-23T14:45:39.416 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59622.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59622.log.gz 2024-04-23T14:45:39.417 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42513.log 2024-04-23T14:45:39.418 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67814.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67814.log.gz 2024-04-23T14:45:39.419 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64195.log 2024-04-23T14:45:39.420 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42513.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42513.log.gz 2024-04-23T14:45:39.420 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54354.log 2024-04-23T14:45:39.422 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64195.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64195.log.gz 2024-04-23T14:45:39.423 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73490.log 2024-04-23T14:45:39.424 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54354.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54354.log.gz 2024-04-23T14:45:39.425 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.73490.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38570.log 2024-04-23T14:45:39.425 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73490.log.gz 2024-04-23T14:45:39.427 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73448.log 2024-04-23T14:45:39.428 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38570.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38570.log.gz 2024-04-23T14:45:39.429 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.73448.log: /var/log/ceph/ceph-client.admin.58355.log 2024-04-23T14:45:39.430 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73448.log.gz 2024-04-23T14:45:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58355.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58355.log.gz 2024-04-23T14:45:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78285.log 2024-04-23T14:45:39.434 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49115.log 2024-04-23T14:45:39.435 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78285.log.gz 2024-04-23T14:45:39.436 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51971.log 2024-04-23T14:45:39.437 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49115.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49115.log.gz 2024-04-23T14:45:39.438 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61193.log 2024-04-23T14:45:39.439 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51971.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51971.log.gz 2024-04-23T14:45:39.440 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72401.log 2024-04-23T14:45:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61193.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61193.log.gz 2024-04-23T14:45:39.441 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40296.log 2024-04-23T14:45:39.443 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72401.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.72401.log.gz 2024-04-23T14:45:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51252.log/var/log/ceph/ceph-client.admin.40296.log: 2024-04-23T14:45:39.444 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40296.log.gz 2024-04-23T14:45:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57268.log 2024-04-23T14:45:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51252.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51252.log.gz 2024-04-23T14:45:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2024-04-23T14:45:39.449 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57268.log.gz 2024-04-23T14:45:39.450 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45630.log 2024-04-23T14:45:39.451 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40758.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2024-04-23T14:45:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45630.log: /var/log/ceph/ceph-client.admin.49242.log 2024-04-23T14:45:39.453 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45630.log.gz 2024-04-23T14:45:39.454 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80182.log 2024-04-23T14:45:39.455 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49242.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49242.log.gz 2024-04-23T14:45:39.456 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54420.log 2024-04-23T14:45:39.457 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80182.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80182.log.gz 2024-04-23T14:45:39.459 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54420.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54420.log.gz 2024-04-23T14:45:39.460 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59480.log 2024-04-23T14:45:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72080.log 2024-04-23T14:45:39.463 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59480.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59480.log.gz 2024-04-23T14:45:39.464 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78991.log 2024-04-23T14:45:39.465 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72080.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.72080.log.gz 2024-04-23T14:45:39.466 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.78991.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38527.log 2024-04-23T14:45:39.466 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78991.log.gz 2024-04-23T14:45:39.467 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73005.log 2024-04-23T14:45:39.469 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38527.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38527.log.gz 2024-04-23T14:45:39.470 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.73005.log: /var/log/ceph/ceph-client.admin.44544.log 2024-04-23T14:45:39.470 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73005.log.gz 2024-04-23T14:45:39.471 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76126.log 2024-04-23T14:45:39.473 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44544.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44544.log.gz 2024-04-23T14:45:39.474 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66473.log 2024-04-23T14:45:39.476 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76126.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.76126.log.gz 2024-04-23T14:45:39.476 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.66473.log: /var/log/ceph/ceph-client.admin.43376.log 2024-04-23T14:45:39.479 INFO:teuthology.orchestra.run.smithi059.stderr: 92.3%/var/log/ceph/ceph-client.admin.43376.log: -- replaced with /var/log/ceph/ceph-client.admin.66473.log.gz 2024-04-23T14:45:39.479 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43376.log.gz 2024-04-23T14:45:39.479 INFO:teuthology.orchestra.run.smithi059.stderr: /var/log/ceph/ceph-client.admin.50124.log 2024-04-23T14:45:39.481 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78243.log 2024-04-23T14:45:39.482 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50124.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50124.log.gz 2024-04-23T14:45:39.483 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49835.log 2024-04-23T14:45:39.484 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78243.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78243.log.gz 2024-04-23T14:45:39.485 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68935.log 2024-04-23T14:45:39.486 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49835.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49835.log.gz 2024-04-23T14:45:39.486 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81551.log 2024-04-23T14:45:39.488 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68935.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68935.log.gz 2024-04-23T14:45:39.489 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39036.log 2024-04-23T14:45:39.490 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81551.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81551.log.gz 2024-04-23T14:45:39.491 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66515.log/var/log/ceph/ceph-client.admin.39036.log: 2024-04-23T14:45:39.491 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39036.log.gz 2024-04-23T14:45:39.493 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61742.log 2024-04-23T14:45:39.496 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66515.log: /var/log/ceph/ceph-client.admin.61742.log: 92.1% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61742.log.gz 2024-04-23T14:45:39.496 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66515.log.gz 2024-04-23T14:45:39.496 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52139.log 2024-04-23T14:45:39.498 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81971.log 2024-04-23T14:45:39.500 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52139.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52139.log.gz 2024-04-23T14:45:39.500 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64317.log/var/log/ceph/ceph-client.admin.81971.log: 2024-04-23T14:45:39.501 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81971.log.gz 2024-04-23T14:45:39.502 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76962.log 2024-04-23T14:45:39.503 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64317.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64317.log.gz 2024-04-23T14:45:39.504 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.76962.log: -- /var/log/ceph/ceph-client.admin.70590.log 2024-04-23T14:45:39.505 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76962.log.gz 2024-04-23T14:45:39.506 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70590.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70590.log.gz 2024-04-23T14:45:39.507 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-04-23T14:45:39.509 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64972.log 2024-04-23T14:45:39.510 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44118.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-04-23T14:45:39.511 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64377.log 2024-04-23T14:45:39.513 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.64972.log: /var/log/ceph/ceph-client.admin.64377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40842.log 2024-04-23T14:45:39.513 INFO:teuthology.orchestra.run.smithi059.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.64972.log.gz 2024-04-23T14:45:39.515 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64377.log.gz 2024-04-23T14:45:39.515 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68009.log 2024-04-23T14:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40842.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40842.log.gz 2024-04-23T14:45:39.518 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74311.log/var/log/ceph/ceph-client.admin.68009.log: 2024-04-23T14:45:39.519 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68009.log.gz 2024-04-23T14:45:39.519 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77046.log 2024-04-23T14:45:39.521 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74311.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74311.log.gz 2024-04-23T14:45:39.522 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77046.log: /var/log/ceph/ceph-client.admin.74706.log 2024-04-23T14:45:39.522 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77046.log.gz 2024-04-23T14:45:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52368.log 2024-04-23T14:45:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74706.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74706.log.gz 2024-04-23T14:45:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52368.log: /var/log/ceph/ceph-client.admin.54786.log 2024-04-23T14:45:39.526 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52368.log.gz 2024-04-23T14:45:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77494.log 2024-04-23T14:45:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54786.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54786.log.gz 2024-04-23T14:45:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75720.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77494.log.gz 2024-04-23T14:45:39.530 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48635.log 2024-04-23T14:45:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75720.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75720.log.gz 2024-04-23T14:45:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-04-23T14:45:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48635.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48635.log.gz 2024-04-23T14:45:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67393.log 2024-04-23T14:45:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43029.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-04-23T14:45:39.538 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76820.log 2024-04-23T14:45:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67393.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67393.log.gz 2024-04-23T14:45:39.539 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63825.log 2024-04-23T14:45:39.541 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76820.log.gz 2024-04-23T14:45:39.541 INFO:teuthology.orchestra.run.smithi059.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-23T14:45:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63825.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63825.log.gz 2024-04-23T14:45:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75066.log 2024-04-23T14:45:39.544 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38308.log 2024-04-23T14:45:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60300.log 2024-04-23T14:45:39.546 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75066.log.gz 2024-04-23T14:45:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38308.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38308.log.gz 2024-04-23T14:45:39.547 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41627.log 2024-04-23T14:45:39.549 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77296.log 2024-04-23T14:45:39.549 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.60300.log.gz 2024-04-23T14:45:39.550 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62988.log 2024-04-23T14:45:39.550 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2024-04-23T14:45:39.551 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61886.log 2024-04-23T14:45:39.551 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77296.log.gz 2024-04-23T14:45:39.553 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-04-23T14:45:39.553 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62988.log.gz 2024-04-23T14:45:39.554 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73826.log 2024-04-23T14:45:39.554 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61886.log.gz 2024-04-23T14:45:39.555 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44076.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-mon.a.log 2024-04-23T14:45:39.556 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-04-23T14:45:39.557 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73826.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72641.log 2024-04-23T14:45:39.557 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73826.log.gz 2024-04-23T14:45:39.558 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78758.log 2024-04-23T14:45:39.560 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72641.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72641.log.gz 2024-04-23T14:45:39.560 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-04-23T14:45:39.561 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78758.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78758.log.gz 2024-04-23T14:45:39.562 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81375.log 2024-04-23T14:45:39.563 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42193.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-04-23T14:45:39.564 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78511.log/var/log/ceph/ceph-client.admin.81375.log: 2024-04-23T14:45:39.565 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81375.log.gz 2024-04-23T14:45:39.566 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2024-04-23T14:45:39.568 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78511.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78511.log.gz 2024-04-23T14:45:39.568 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47749.log: /var/log/ceph/ceph-client.admin.56890.log 2024-04-23T14:45:39.569 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2024-04-23T14:45:39.570 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53734.log 2024-04-23T14:45:39.572 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56890.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56890.log.gz 2024-04-23T14:45:39.572 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53734.log: /var/log/ceph/ceph-client.admin.74481.log 2024-04-23T14:45:39.573 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53734.log.gz 2024-04-23T14:45:39.574 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39764.log 2024-04-23T14:45:39.575 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74481.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74481.log.gz 2024-04-23T14:45:39.576 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55444.log 2024-04-23T14:45:39.577 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39764.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39764.log.gz 2024-04-23T14:45:39.578 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64694.log 2024-04-23T14:45:39.579 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55444.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55444.log.gz 2024-04-23T14:45:39.580 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71603.log 2024-04-23T14:45:39.581 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64694.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64694.log.gz 2024-04-23T14:45:39.582 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57142.log 2024-04-23T14:45:39.583 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71603.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71603.log.gz 2024-04-23T14:45:39.584 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72499.log 2024-04-23T14:45:39.584 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57142.log.gz 2024-04-23T14:45:39.586 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76048.log 2024-04-23T14:45:39.588 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72499.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72499.log.gz 2024-04-23T14:45:39.588 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77004.log 2024-04-23T14:45:39.589 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44276.log 2024-04-23T14:45:39.589 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76048.log.gz 2024-04-23T14:45:39.592 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77004.log: /var/log/ceph/ceph-client.admin.44276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61700.log 91.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-23T14:45:39.592 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:39.592 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44276.log.gz 2024-04-23T14:45:39.592 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77004.log.gz 2024-04-23T14:45:39.593 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53021.log 2024-04-23T14:45:39.595 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61700.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58045.log 2024-04-23T14:45:39.595 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61700.log.gz 2024-04-23T14:45:39.596 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46002.log 2024-04-23T14:45:39.596 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53021.log.gz 2024-04-23T14:45:39.597 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58045.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58045.log.gz 2024-04-23T14:45:39.598 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2024-04-23T14:45:39.598 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46002.log.gz 2024-04-23T14:45:39.599 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52097.log 2024-04-23T14:45:39.600 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74525.log 2024-04-23T14:45:39.601 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2024-04-23T14:45:39.601 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52097.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52097.log.gz 2024-04-23T14:45:39.602 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49627.log 2024-04-23T14:45:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74525.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74525.log.gz 2024-04-23T14:45:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55291.log 2024-04-23T14:45:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46044.log 2024-04-23T14:45:39.605 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49627.log.gz 2024-04-23T14:45:39.606 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73028.log 2024-04-23T14:45:39.607 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55291.log.gz 2024-04-23T14:45:39.607 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69324.log 2024-04-23T14:45:39.607 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46044.log.gz 2024-04-23T14:45:39.609 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80224.log 2024-04-23T14:45:39.609 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.73028.log.gz 2024-04-23T14:45:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69324.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69324.log.gz 2024-04-23T14:45:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74874.log 2024-04-23T14:45:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80224.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80224.log.gz 2024-04-23T14:45:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47389.log 2024-04-23T14:45:39.613 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74874.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74874.log.gz 2024-04-23T14:45:39.613 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42903.log 2024-04-23T14:45:39.614 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.47389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80564.log 2024-04-23T14:45:39.616 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42903.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42903.log.gz 2024-04-23T14:45:39.617 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5/var/log/ceph/ceph-client.admin.80564.log: --verbose -- /var/log/ceph/ceph-client.admin.68051.log 2024-04-23T14:45:39.617 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80564.log.gz 2024-04-23T14:45:39.618 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63030.log 2024-04-23T14:45:39.620 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68051.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68051.log.gz 2024-04-23T14:45:39.621 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5/var/log/ceph/ceph-client.admin.63030.log: --verbose -- /var/log/ceph/ceph-client.admin.45462.log 2024-04-23T14:45:39.621 INFO:teuthology.orchestra.run.smithi059.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.47389.log.gz 2024-04-23T14:45:39.621 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63030.log.gz 2024-04-23T14:45:39.622 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45960.log 2024-04-23T14:45:39.623 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45462.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45462.log.gz 2024-04-23T14:45:39.623 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76735.log 2024-04-23T14:45:39.624 INFO:teuthology.orchestra.run.smithi076.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-23T14:45:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43644.log 2024-04-23T14:45:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45960.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45960.log.gz 2024-04-23T14:45:39.626 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76735.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76735.log.gz 2024-04-23T14:45:39.627 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63072.log 2024-04-23T14:45:39.627 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43644.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43644.log.gz 2024-04-23T14:45:39.627 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81845.log 2024-04-23T14:45:39.629 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55207.log 2024-04-23T14:45:39.629 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63072.log.gz 2024-04-23T14:45:39.630 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61532.log 2024-04-23T14:45:39.631 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81845.log.gz 2024-04-23T14:45:39.632 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43197.log 2024-04-23T14:45:39.632 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55207.log.gz 2024-04-23T14:45:39.633 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38654.log 2024-04-23T14:45:39.633 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61532.log.gz 2024-04-23T14:45:39.634 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66630.log 2024-04-23T14:45:39.634 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43197.log.gz 2024-04-23T14:45:39.635 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38654.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38654.log.gz 2024-04-23T14:45:39.636 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43071.log 2024-04-23T14:45:39.637 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62778.log 2024-04-23T14:45:39.637 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66630.log.gz 2024-04-23T14:45:39.638 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54936.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43071.log.gz 2024-04-23T14:45:39.638 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:39.639 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62778.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62778.log.gz 2024-04-23T14:45:39.640 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56802.log 2024-04-23T14:45:39.641 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62139.log 2024-04-23T14:45:39.642 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54936.log.gz 2024-04-23T14:45:39.643 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44444.log 2024-04-23T14:45:39.643 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56802.log.gz 2024-04-23T14:45:39.644 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62139.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81045.log 2024-04-23T14:45:39.644 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62139.log.gz 2024-04-23T14:45:39.645 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2024-04-23T14:45:39.645 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44444.log.gz 2024-04-23T14:45:39.647 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81045.log: gzip -5 --verbose 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81045.log.gz 2024-04-23T14:45:39.647 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.67772.log 2024-04-23T14:45:39.648 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48059.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2024-04-23T14:45:39.649 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75762.log 2024-04-23T14:45:39.650 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56707.log 2024-04-23T14:45:39.650 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67772.log.gz 2024-04-23T14:45:39.651 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39758.log 2024-04-23T14:45:39.651 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75762.log.gz 2024-04-23T14:45:39.652 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59706.log 2024-04-23T14:45:39.653 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56707.log.gz 2024-04-23T14:45:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39758.log.gz 2024-04-23T14:45:39.654 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44034.log 2024-04-23T14:45:39.655 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59706.log.gz 2024-04-23T14:45:39.656 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73616.log 2024-04-23T14:45:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.44034.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.77602.log 2024-04-23T14:45:39.657 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44034.log.gz 2024-04-23T14:45:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73616.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73616.log.gz 2024-04-23T14:45:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55697.log 2024-04-23T14:45:39.659 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77602.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.77602.log.gz 2024-04-23T14:45:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57100.log 2024-04-23T14:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78925.log 2024-04-23T14:45:39.662 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55697.log.gz 2024-04-23T14:45:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57226.log 2024-04-23T14:45:39.663 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57100.log.gz 2024-04-23T14:45:39.665 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71477.log 2024-04-23T14:45:39.665 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.78925.log.gz 2024-04-23T14:45:39.666 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69496.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57226.log.gz 2024-04-23T14:45:39.666 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71477.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71477.log.gz 2024-04-23T14:45:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66826.log 2024-04-23T14:45:39.668 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62606.log 2024-04-23T14:45:39.669 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69496.log.gz/var/log/ceph/ceph-client.admin.66826.log: 2024-04-23T14:45:39.669 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.56751.log 2024-04-23T14:45:39.670 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66826.log.gz 2024-04-23T14:45:39.670 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62606.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62606.log.gz 2024-04-23T14:45:39.671 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66231.log 2024-04-23T14:45:39.673 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.56751.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39120.log 2024-04-23T14:45:39.673 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56751.log.gz 2024-04-23T14:45:39.673 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66231.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66231.log.gz 2024-04-23T14:45:39.674 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75344.log 2024-04-23T14:45:39.675 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60559.log 2024-04-23T14:45:39.675 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39120.log.gz 2024-04-23T14:45:39.676 INFO:teuthology.orchestra.run.smithi076.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-23T14:45:39.677 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.75344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54978.log 2024-04-23T14:45:39.678 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60559.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60559.log.gz 2024-04-23T14:45:39.679 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2024-04-23T14:45:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54978.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54978.log.gz 2024-04-23T14:45:39.681 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45420.log: /var/log/ceph/ceph-client.admin.47003.log 2024-04-23T14:45:39.681 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2024-04-23T14:45:39.682 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74353.log 2024-04-23T14:45:39.683 INFO:teuthology.orchestra.run.smithi059.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.75344.log.gz 2024-04-23T14:45:39.684 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47003.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53458.log 2024-04-23T14:45:39.684 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47003.log.gz 2024-04-23T14:45:39.685 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74353.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74353.log.gz 2024-04-23T14:45:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60084.log 2024-04-23T14:45:39.686 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53458.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53458.log.gz 2024-04-23T14:45:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33897.log 2024-04-23T14:45:39.688 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60084.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60234.log 2024-04-23T14:45:39.688 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60084.log.gz 2024-04-23T14:45:39.689 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.33897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43155.log 2024-04-23T14:45:39.690 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33897.log.gz 2024-04-23T14:45:39.691 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65560.log 2024-04-23T14:45:39.691 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60234.log.gz 2024-04-23T14:45:39.692 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81330.log 2024-04-23T14:45:39.692 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43155.log.gz 2024-04-23T14:45:39.694 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59564.log 2024-04-23T14:45:39.694 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65560.log.gz/var/log/ceph/ceph-client.admin.81330.log: 2024-04-23T14:45:39.695 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81330.log.gz 2024-04-23T14:45:39.695 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63156.log 2024-04-23T14:45:39.697 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65344.log 2024-04-23T14:45:39.697 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59564.log.gz 2024-04-23T14:45:39.698 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72318.log 2024-04-23T14:45:39.698 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63156.log.gz 2024-04-23T14:45:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65344.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65344.log.gz 2024-04-23T14:45:39.699 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52181.log 2024-04-23T14:45:39.700 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39760.log 2024-04-23T14:45:39.701 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72318.log.gz/var/log/ceph/ceph-client.admin.52181.log: 2024-04-23T14:45:39.702 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60451.log 2024-04-23T14:45:39.702 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52181.log.gz 2024-04-23T14:45:39.703 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39760.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39760.log.gz 2024-04-23T14:45:39.703 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48344.log 2024-04-23T14:45:39.704 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55123.log 2024-04-23T14:45:39.704 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60451.log.gz 2024-04-23T14:45:39.706 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2024-04-23T14:45:39.706 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48344.log.gz 2024-04-23T14:45:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55123.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55123.log.gz 2024-04-23T14:45:39.707 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63264.log 2024-04-23T14:45:39.708 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52837.log 2024-04-23T14:45:39.709 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2024-04-23T14:45:39.709 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63264.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63264.log.gz 2024-04-23T14:45:39.710 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82097.log 2024-04-23T14:45:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55655.log 2024-04-23T14:45:39.712 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52837.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52837.log.gz 2024-04-23T14:45:39.713 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.82097.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82097.log.gz 2024-04-23T14:45:39.713 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.51457.log 2024-04-23T14:45:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55655.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55655.log.gz 2024-04-23T14:45:39.714 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72767.log 2024-04-23T14:45:39.716 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79809.log 2024-04-23T14:45:39.716 INFO:teuthology.orchestra.run.smithi059.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51457.log.gz 2024-04-23T14:45:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72767.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72767.log.gz 2024-04-23T14:45:39.718 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.79809.log: /var/log/ceph/ceph-client.admin.42556.log 2024-04-23T14:45:39.719 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64340.log 2024-04-23T14:45:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42556.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42556.log.gz 2024-04-23T14:45:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81425.log 2024-04-23T14:45:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64340.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64340.log.gz 2024-04-23T14:45:39.722 INFO:teuthology.orchestra.run.smithi059.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.79809.log.gz 2024-04-23T14:45:39.723 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67925.log 2024-04-23T14:45:39.724 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81425.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81425.log.gz 2024-04-23T14:45:39.725 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64069.log 2024-04-23T14:45:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67925.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67925.log.gz 2024-04-23T14:45:39.726 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56621.log 2024-04-23T14:45:39.727 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50804.log 2024-04-23T14:45:39.728 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64069.log.gz 2024-04-23T14:45:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56621.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56621.log.gz 2024-04-23T14:45:39.729 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69143.log 2024-04-23T14:45:39.731 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.50804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2024-04-23T14:45:39.732 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69143.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69143.log.gz 2024-04-23T14:45:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64586.log 2024-04-23T14:45:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43334.log: 93.0% -- replaced with /var/log/ceph/ceph-client.0.50804.log.gz 2024-04-23T14:45:39.733 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2024-04-23T14:45:39.734 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60385.log 2024-04-23T14:45:39.736 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45875.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64586.log.gz 2024-04-23T14:45:39.737 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:39.737 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48418.log 2024-04-23T14:45:39.738 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60385.log.gz 2024-04-23T14:45:39.738 INFO:teuthology.orchestra.run.smithi076.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-23T14:45:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45875.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45875.log.gz 2024-04-23T14:45:39.738 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60126.log 2024-04-23T14:45:39.740 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63867.log 2024-04-23T14:45:39.740 INFO:teuthology.orchestra.run.smithi059.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48418.log.gz 2024-04-23T14:45:39.741 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49410.log 2024-04-23T14:45:39.742 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60126.log.gz 2024-04-23T14:45:39.742 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63867.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44586.log 2024-04-23T14:45:39.743 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63867.log.gz 2024-04-23T14:45:39.744 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77448.log 2024-04-23T14:45:39.744 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49410.log.gz 2024-04-23T14:45:39.745 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44586.log.gz 2024-04-23T14:45:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76470.log 2024-04-23T14:45:39.746 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77448.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77448.log.gz 2024-04-23T14:45:39.747 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62946.log 2024-04-23T14:45:39.748 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.76470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61784.log 2024-04-23T14:45:39.750 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62946.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62946.log.gz 2024-04-23T14:45:39.751 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49055.log 2024-04-23T14:45:39.751 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61784.log.gz 2024-04-23T14:45:39.752 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78715.log 2024-04-23T14:45:39.753 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49055.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49055.log.gz 2024-04-23T14:45:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.78715.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51845.log 2024-04-23T14:45:39.755 INFO:teuthology.orchestra.run.smithi059.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.76470.log.gz 2024-04-23T14:45:39.755 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78715.log.gz 2024-04-23T14:45:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40548.log 2024-04-23T14:45:39.757 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51845.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42345.log 2024-04-23T14:45:39.757 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51845.log.gz 2024-04-23T14:45:39.758 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57460.log 2024-04-23T14:45:39.758 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40548.log.gz 2024-04-23T14:45:39.760 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42345.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42345.log.gz 2024-04-23T14:45:39.760 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46667.log 2024-04-23T14:45:39.761 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57460.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57460.log.gz 2024-04-23T14:45:39.761 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61295.log 2024-04-23T14:45:39.763 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.46667.log: -5 --verbose -- /var/log/ceph/ceph-client.2.42043.log 2024-04-23T14:45:39.763 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46667.log.gz 2024-04-23T14:45:39.764 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61295.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61295.log.gz 2024-04-23T14:45:39.765 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72595.log 2024-04-23T14:45:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.2.42043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55165.log 2024-04-23T14:45:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72595.log: 92.0% -- replaced with /var/log/ceph/ceph-client.2.42043.log.gz 2024-04-23T14:45:39.767 INFO:teuthology.orchestra.run.smithi076.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-23T14:45:39.768 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81677.log 2024-04-23T14:45:39.768 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72595.log.gz 2024-04-23T14:45:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55165.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55165.log.gz 2024-04-23T14:45:39.769 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61016.log 2024-04-23T14:45:39.770 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81677.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81677.log.gz 2024-04-23T14:45:39.770 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56238.log 2024-04-23T14:45:39.771 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-23T14:45:39.771 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m0.830s 2024-04-23T14:45:39.771 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m3.480s 2024-04-23T14:45:39.771 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.389s 2024-04-23T14:45:39.772 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58860.log 2024-04-23T14:45:39.773 INFO:teuthology.orchestra.run.smithi059.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.61016.log.gz/var/log/ceph/ceph-client.admin.56238.log: 2024-04-23T14:45:39.773 INFO:teuthology.orchestra.run.smithi059.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56238.log.gz 2024-04-23T14:45:39.773 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.67104.log 2024-04-23T14:45:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58860.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58860.log.gz 2024-04-23T14:45:39.774 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42429.log 2024-04-23T14:45:39.776 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.67104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56070.log 2024-04-23T14:45:39.777 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42429.log: 92.3% -- replaced with /var/log/ceph/ceph-client.0.67104.log.gz 2024-04-23T14:45:39.777 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42429.log.gz 2024-04-23T14:45:39.778 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56579.log 2024-04-23T14:45:39.778 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56070.log.gz 2024-04-23T14:45:39.779 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39763.log 2024-04-23T14:45:39.780 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56579.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39426.log 2024-04-23T14:45:39.780 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56579.log.gz 2024-04-23T14:45:39.781 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39763.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39763.log.gz 2024-04-23T14:45:39.782 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39426.log: -- /var/log/ceph/ceph-client.admin.48221.log 2024-04-23T14:45:39.782 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39426.log.gz 2024-04-23T14:45:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73321.log 2024-04-23T14:45:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49750.log 2024-04-23T14:45:39.785 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48221.log.gz 2024-04-23T14:45:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73321.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73321.log.gz 2024-04-23T14:45:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49750.log: /var/log/ceph/ceph-client.admin.59934.log 2024-04-23T14:45:39.787 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49750.log.gz 2024-04-23T14:45:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62648.log 2024-04-23T14:45:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80648.log 2024-04-23T14:45:39.789 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59934.log.gz 2024-04-23T14:45:39.790 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62648.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62648.log.gz 2024-04-23T14:45:39.790 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81087.log 2024-04-23T14:45:39.791 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59868.log 2024-04-23T14:45:39.791 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80648.log.gz 2024-04-23T14:45:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81087.log: gzip -5 --verbose 87.9% -- /var/log/ceph/ceph-client.admin.44670.log -- replaced with /var/log/ceph/ceph-client.admin.81087.log.gz 2024-04-23T14:45:39.793 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59868.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59868.log.gz 2024-04-23T14:45:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57184.log 2024-04-23T14:45:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72683.log 2024-04-23T14:45:39.796 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44670.log.gz 2024-04-23T14:45:39.797 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63927.log 2024-04-23T14:45:39.797 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57184.log.gz 2024-04-23T14:45:39.798 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72683.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72683.log.gz 2024-04-23T14:45:39.799 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53193.log 2024-04-23T14:45:39.800 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63927.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63927.log.gz 2024-04-23T14:45:39.800 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43950.log 2024-04-23T14:45:39.801 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53193.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53193.log.gz 2024-04-23T14:45:39.801 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60601.log 2024-04-23T14:45:39.803 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.43950.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58313.log 2024-04-23T14:45:39.803 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43950.log.gz 2024-04-23T14:45:39.804 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.60601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77199.log 2024-04-23T14:45:39.804 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60601.log.gz 2024-04-23T14:45:39.805 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58313.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58313.log.gz 2024-04-23T14:45:39.805 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.77904.log 2024-04-23T14:45:39.806 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56196.log 2024-04-23T14:45:39.806 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77199.log.gz 2024-04-23T14:45:39.808 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.77904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2024-04-23T14:45:39.809 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56196.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56196.log.gz 2024-04-23T14:45:39.810 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73742.log 2024-04-23T14:45:39.810 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2024-04-23T14:45:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69622.log 2024-04-23T14:45:39.812 INFO:teuthology.orchestra.run.smithi059.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.77904.log.gz 2024-04-23T14:45:39.813 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73742.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.73742.log.gz 2024-04-23T14:45:39.813 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63198.log 2024-04-23T14:45:39.814 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64275.log 2024-04-23T14:45:39.814 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69622.log.gz 2024-04-23T14:45:39.815 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73928.log 2024-04-23T14:45:39.816 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.63198.log.gz 2024-04-23T14:45:39.817 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47975.log 2024-04-23T14:45:39.817 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64275.log.gz 2024-04-23T14:45:39.818 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73928.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73928.log.gz 2024-04-23T14:45:39.819 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.47975.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58187.log 2024-04-23T14:45:39.820 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47975.log.gz 2024-04-23T14:45:39.821 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41028.log 2024-04-23T14:45:39.822 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58187.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58187.log.gz 2024-04-23T14:45:39.822 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69408.log 2024-04-23T14:45:39.823 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.41028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54570.log 2024-04-23T14:45:39.823 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41028.log.gz 2024-04-23T14:45:39.824 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69408.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69408.log.gz 2024-04-23T14:45:39.825 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45126.log 2024-04-23T14:45:39.825 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54570.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54570.log.gz 2024-04-23T14:45:39.827 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40926.log 2024-04-23T14:45:39.828 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45126.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45126.log.gz 2024-04-23T14:45:39.828 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79531.log 2024-04-23T14:45:39.829 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68219.log 2024-04-23T14:45:39.830 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0%/var/log/ceph/ceph-client.admin.79531.log: -- replaced with /var/log/ceph/ceph-client.admin.40926.log.gz 2024-04-23T14:45:39.831 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59790.log 2024-04-23T14:45:39.831 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79531.log.gz 2024-04-23T14:45:39.832 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70092.log 2024-04-23T14:45:39.832 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68219.log.gz 2024-04-23T14:45:39.833 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72855.log 2024-04-23T14:45:39.833 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59790.log.gz 2024-04-23T14:45:39.835 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.70092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67646.log 2024-04-23T14:45:39.837 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72855.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72855.log.gz 2024-04-23T14:45:39.837 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67646.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.76006.log 2024-04-23T14:45:39.838 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67646.log.gz 2024-04-23T14:45:39.839 INFO:teuthology.orchestra.run.smithi059.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.70092.log.gz 2024-04-23T14:45:39.839 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68303.log 2024-04-23T14:45:39.841 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.76006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56663.log 2024-04-23T14:45:39.841 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76006.log.gz 2024-04-23T14:45:39.842 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68303.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68303.log.gz 2024-04-23T14:45:39.842 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40800.log 2024-04-23T14:45:39.843 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51692.log 2024-04-23T14:45:39.844 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56663.log.gz 2024-04-23T14:45:39.845 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57738.log 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40800.log.gz 2024-04-23T14:45:39.845 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:39.846 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51692.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51692.log.gz 2024-04-23T14:45:39.846 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52919.log 2024-04-23T14:45:39.847 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.57738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52757.log 2024-04-23T14:45:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52919.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52919.log.gz 2024-04-23T14:45:39.849 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66413.log 2024-04-23T14:45:39.851 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52757.log: 92.7% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52757.log.gz 2024-04-23T14:45:39.851 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.0.57738.log.gz 2024-04-23T14:45:39.851 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51734.log 2024-04-23T14:45:39.852 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50376.log 2024-04-23T14:45:39.853 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51734.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.66413.log.gz 2024-04-23T14:45:39.854 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51734.log.gz 2024-04-23T14:45:39.854 INFO:teuthology.orchestra.run.smithi059.stderr: -- /var/log/ceph/ceph-client.admin.48761.log 2024-04-23T14:45:39.855 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50376.log.gz 2024-04-23T14:45:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66390.log 2024-04-23T14:45:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48761.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48761.log.gz 2024-04-23T14:45:39.856 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58794.log 2024-04-23T14:45:39.858 INFO:teuthology.orchestra.run.smithi059.stderr:gzip/var/log/ceph/ceph-client.admin.66390.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.77352.log 2024-04-23T14:45:39.858 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66390.log.gz 2024-04-23T14:45:39.860 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58794.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58794.log.gz 2024-04-23T14:45:39.860 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2024-04-23T14:45:39.860 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77352.log.gz 2024-04-23T14:45:39.862 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44790.log 2024-04-23T14:45:39.863 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43728.log: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-23T14:45:39.863 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 2024-04-23T14:45:39.863 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44832.log 2024-04-23T14:45:39.864 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45210.log 2024-04-23T14:45:39.864 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44790.log.gz 2024-04-23T14:45:39.865 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58610.log 2024-04-23T14:45:39.865 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44832.log.gz 2024-04-23T14:45:39.866 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72252.log 2024-04-23T14:45:39.866 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45210.log.gz 2024-04-23T14:45:39.867 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73574.log 2024-04-23T14:45:39.867 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58610.log.gz 2024-04-23T14:45:39.868 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74107.log 2024-04-23T14:45:39.869 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72252.log.gz 2024-04-23T14:45:39.869 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40097.log 2024-04-23T14:45:39.869 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73574.log.gz 2024-04-23T14:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68490.log 2024-04-23T14:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74107.log.gz 2024-04-23T14:45:39.872 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44160.log 2024-04-23T14:45:39.872 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40097.log.gz 2024-04-23T14:45:39.872 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68490.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68490.log.gz 2024-04-23T14:45:39.873 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69538.log 2024-04-23T14:45:39.874 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38404.log 2024-04-23T14:45:39.874 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44160.log.gz 2024-04-23T14:45:39.875 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69538.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69538.log.gz 2024-04-23T14:45:39.875 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58513.log 2024-04-23T14:45:39.876 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44916.log 2024-04-23T14:45:39.876 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38404.log.gz 2024-04-23T14:45:39.877 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71967.log 2024-04-23T14:45:39.877 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58513.log.gz 2024-04-23T14:45:39.878 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-23T14:45:39.878 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44916.log.gz 2024-04-23T14:45:39.879 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.71967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78369.log 2024-04-23T14:45:39.880 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71967.log.gz 2024-04-23T14:45:39.881 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58087.log 2024-04-23T14:45:39.881 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-23T14:45:39.881 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.78369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52536.log 2024-04-23T14:45:39.881 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78369.log.gz 2024-04-23T14:45:39.882 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61235.log 88.5% 2024-04-23T14:45:39.882 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58087.log.gz 2024-04-23T14:45:39.884 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40464.log 2024-04-23T14:45:39.884 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52536.log.gz 2024-04-23T14:45:39.885 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46625.log 2024-04-23T14:45:39.885 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61235.log.gz 2024-04-23T14:45:39.886 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68135.log 2024-04-23T14:45:39.886 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40464.log.gz 2024-04-23T14:45:39.887 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2024-04-23T14:45:39.887 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46625.log.gz 2024-04-23T14:45:39.888 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59396.log 2024-04-23T14:45:39.888 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68135.log.gz 2024-04-23T14:45:39.889 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45588.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2024-04-23T14:45:39.889 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62013.log 2024-04-23T14:45:39.890 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-04-23T14:45:39.890 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59396.log.gz 2024-04-23T14:45:39.891 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-04-23T14:45:39.891 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62013.log.gz 2024-04-23T14:45:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56154.log 2024-04-23T14:45:39.892 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-04-23T14:45:39.894 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81003.log 2024-04-23T14:45:39.894 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-04-23T14:45:39.895 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81593.log 2024-04-23T14:45:39.895 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56154.log.gz 2024-04-23T14:45:39.896 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53500.log 2024-04-23T14:45:39.896 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81003.log.gz 2024-04-23T14:45:39.897 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-04-23T14:45:39.897 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81593.log.gz 2024-04-23T14:45:39.898 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53500.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53500.log.gzgzip 2024-04-23T14:45:39.898 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57016.log 2024-04-23T14:45:39.899 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74269.log 2024-04-23T14:45:39.900 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-04-23T14:45:39.901 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66718.log 2024-04-23T14:45:39.901 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57016.log.gz 2024-04-23T14:45:39.901 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82199.log 2024-04-23T14:45:39.902 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74269.log.gz 2024-04-23T14:45:39.903 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61616.log 2024-04-23T14:45:39.903 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66718.log.gz 2024-04-23T14:45:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.82199.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82199.log.gz 2024-04-23T14:45:39.904 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81467.log 2024-04-23T14:45:39.905 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61929.log 2024-04-23T14:45:39.905 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61616.log.gz 2024-04-23T14:45:39.906 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39744.log 2024-04-23T14:45:39.906 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81467.log.gz 2024-04-23T14:45:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47917.log 2024-04-23T14:45:39.907 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61929.log.gz 2024-04-23T14:45:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-04-23T14:45:39.908 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39744.log.gz 2024-04-23T14:45:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74012.log 2024-04-23T14:45:39.909 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47917.log.gz 2024-04-23T14:45:39.910 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72341.log 2024-04-23T14:45:39.910 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-04-23T14:45:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44360.log 2024-04-23T14:45:39.912 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74012.log.gz 2024-04-23T14:45:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70548.log 2024-04-23T14:45:39.913 INFO:teuthology.orchestra.run.smithi059.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.72341.log.gz/var/log/ceph/ceph-client.admin.44360.log: 2024-04-23T14:45:39.914 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44360.log.gz 2024-04-23T14:45:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55044.log 2024-04-23T14:45:39.915 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46453.log 2024-04-23T14:45:39.915 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70548.log.gz 2024-04-23T14:45:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55044.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-23T14:45:39.916 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55044.log.gz 2024-04-23T14:45:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51887.log 2024-04-23T14:45:39.917 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46453.log.gz 2024-04-23T14:45:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81719.log 2024-04-23T14:45:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51887.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51887.log.gz 2024-04-23T14:45:39.920 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2024-04-23T14:45:39.921 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59748.log 2024-04-23T14:45:39.921 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81719.log.gz 2024-04-23T14:45:39.922 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47111.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2024-04-23T14:45:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46835.log 2024-04-23T14:45:39.923 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.59748.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59748.log.gz 2024-04-23T14:45:39.924 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47045.log 2024-04-23T14:45:39.926 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81803.log 2024-04-23T14:45:39.926 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46835.log.gz 2024-04-23T14:45:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74207.log 2024-04-23T14:45:39.927 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47045.log.gz 2024-04-23T14:45:39.928 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79282.log 2024-04-23T14:45:39.929 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3%/var/log/ceph/ceph-client.admin.74207.log: -- replaced with /var/log/ceph/ceph-client.admin.81803.log.gz 2024-04-23T14:45:39.930 INFO:teuthology.orchestra.run.smithi059.stderr:gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74207.log.gz 2024-04-23T14:45:39.930 INFO:teuthology.orchestra.run.smithi059.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77088.log 2024-04-23T14:45:39.931 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79282.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79282.log.gz 2024-04-23T14:45:39.931 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57310.log 2024-04-23T14:45:39.932 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.77088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68405.log 2024-04-23T14:45:39.933 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77088.log.gz 2024-04-23T14:45:39.934 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63969.log 2024-04-23T14:45:39.934 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57310.log.gz 2024-04-23T14:45:39.935 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62736.log 2024-04-23T14:45:39.935 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68405.log.gz 2024-04-23T14:45:39.937 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.63969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48803.log 2024-04-23T14:45:39.937 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63969.log.gz 2024-04-23T14:45:39.938 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62736.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62736.log.gz 2024-04-23T14:45:39.939 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48803.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48803.log.gz 2024-04-23T14:45:39.939 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81761.log 2024-04-23T14:45:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45336.log 2024-04-23T14:45:39.942 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70674.log 2024-04-23T14:45:39.942 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81761.log.gz 2024-04-23T14:45:39.943 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70506.log 2024-04-23T14:45:39.943 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45336.log.gz 2024-04-23T14:45:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70674.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70674.log.gz 2024-04-23T14:45:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.70506.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70506.log.gz 2024-04-23T14:45:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74748.log 2024-04-23T14:45:39.947 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62055.log 2024-04-23T14:45:39.949 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80690.log 2024-04-23T14:45:39.949 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74748.log.gz 2024-04-23T14:45:39.950 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.62055.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62055.log.gz 2024-04-23T14:45:39.952 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80690.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80690.log.gz 2024-04-23T14:45:39.952 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65646.log 2024-04-23T14:45:39.953 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-04-23T14:45:39.955 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68448.log 2024-04-23T14:45:39.955 INFO:teuthology.orchestra.run.smithi059.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65646.log.gz 2024-04-23T14:45:39.955 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43806.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-04-23T14:45:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68448.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68448.log.gz 2024-04-23T14:45:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52662.log 2024-04-23T14:45:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49921.log 2024-04-23T14:45:39.960 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.52662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55249.log 2024-04-23T14:45:39.960 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52662.log.gz 2024-04-23T14:45:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66005.log 2024-04-23T14:45:39.962 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49921.log.gz 2024-04-23T14:45:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55249.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55249.log.gz 2024-04-23T14:45:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66005.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66005.log.gz 2024-04-23T14:45:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39749.log 2024-04-23T14:45:39.965 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48845.log 2024-04-23T14:45:39.967 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.39749.log: gzip -5 --verbose -- 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39749.log.gz 2024-04-23T14:45:39.967 INFO:teuthology.orchestra.run.smithi059.stderr: /var/log/ceph/ceph-client.admin.38826.log 2024-04-23T14:45:39.968 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48845.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48845.log.gz 2024-04-23T14:45:39.968 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73886.log 2024-04-23T14:45:39.970 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58229.log 2024-04-23T14:45:39.971 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38826.log.gz 2024-04-23T14:45:39.972 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.73886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79080.log 2024-04-23T14:45:39.972 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73886.log.gz 2024-04-23T14:45:39.973 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74576.log 2024-04-23T14:45:39.973 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58229.log.gz 2024-04-23T14:45:39.974 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.79080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53626.log 2024-04-23T14:45:39.976 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74576.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.79080.log.gz 2024-04-23T14:45:39.976 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74576.log.gz 2024-04-23T14:45:39.976 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55402.log/var/log/ceph/ceph-client.admin.53626.log: 2024-04-23T14:45:39.976 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53626.log.gz 2024-04-23T14:45:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63542.log 2024-04-23T14:45:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55402.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50292.log 2024-04-23T14:45:39.978 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55402.log.gz 2024-04-23T14:45:39.981 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.63542.log: /var/log/ceph/ceph-client.admin.50292.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50292.log.gz 2024-04-23T14:45:39.981 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58536.log 2024-04-23T14:45:39.982 INFO:teuthology.orchestra.run.smithi059.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.63542.log.gz 2024-04-23T14:45:39.982 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40506.log 2024-04-23T14:45:39.984 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74395.log 2024-04-23T14:45:39.984 INFO:teuthology.orchestra.run.smithi059.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.58536.log.gz 2024-04-23T14:45:39.985 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.40506.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40506.log.gz 2024-04-23T14:45:39.986 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74395.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74395.log.gz 2024-04-23T14:45:39.987 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53149.log 2024-04-23T14:45:39.988 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68177.log 2024-04-23T14:45:39.990 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64027.log 2024-04-23T14:45:39.990 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53149.log.gz 2024-04-23T14:45:39.990 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.68177.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68177.log.gz 2024-04-23T14:45:39.991 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56028.log 2024-04-23T14:45:39.992 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.64027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45294.log 2024-04-23T14:45:39.992 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64027.log.gz 2024-04-23T14:45:39.993 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.56028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54245.log 88.6% 2024-04-23T14:45:39.994 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56028.log.gz 2024-04-23T14:45:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42235.log 2024-04-23T14:45:39.995 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45294.log.gz 2024-04-23T14:45:39.996 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69454.log 2024-04-23T14:45:39.997 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54245.log.gz 2024-04-23T14:45:39.997 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42235.log.gz 2024-04-23T14:45:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74832.log 2024-04-23T14:45:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.69454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58003.log 2024-04-23T14:45:39.999 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69454.log.gz 2024-04-23T14:45:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.74832.log: gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74832.log.gz 2024-04-23T14:45:40.001 INFO:teuthology.orchestra.run.smithi059.stderr: /var/log/ceph/ceph-client.admin.38784.log 2024-04-23T14:45:40.002 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58003.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58003.log.gz 2024-04-23T14:45:40.002 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47833.log 2024-04-23T14:45:40.003 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45546.log 2024-04-23T14:45:40.003 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38784.log.gz 2024-04-23T14:45:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47833.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47833.log.gz 2024-04-23T14:45:40.005 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61826.log 2024-04-23T14:45:40.006 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57058.log 2024-04-23T14:45:40.006 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45546.log.gz 2024-04-23T14:45:40.007 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.61826.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61826.log.gz 2024-04-23T14:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45000.log 2024-04-23T14:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57058.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57058.log.gz 2024-04-23T14:45:40.009 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58271.log 2024-04-23T14:45:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.45000.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45000.log.gz 2024-04-23T14:45:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72443.log 2024-04-23T14:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75862.log 2024-04-23T14:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58271.log.gz 2024-04-23T14:45:40.013 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72443.log: gzip -5 89.9% --verbose -- /var/log/ceph/ceph-client.admin.43686.log 2024-04-23T14:45:40.013 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72443.log.gz 2024-04-23T14:45:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.75862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80479.log 2024-04-23T14:45:40.015 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75862.log.gz 2024-04-23T14:45:40.016 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.43686.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43686.log.gz 2024-04-23T14:45:40.016 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50418.log 2024-04-23T14:45:40.017 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80479.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80479.log.gz 2024-04-23T14:45:40.018 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42387.log 2024-04-23T14:45:40.019 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.50418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40099.log 2024-04-23T14:45:40.019 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50418.log.gz 2024-04-23T14:45:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.42387.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42387.log.gz 2024-04-23T14:45:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45378.log 2024-04-23T14:45:40.021 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40099.log: /var/log/ceph/ceph-client.admin.55571.log 2024-04-23T14:45:40.022 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0%/var/log/ceph/ceph-client.admin.45378.log: -- replaced with /var/log/ceph/ceph-client.admin.40099.log.gz 2024-04-23T14:45:40.023 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45378.log.gz 2024-04-23T14:45:40.023 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72081.log 2024-04-23T14:45:40.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67730.log 2024-04-23T14:45:40.025 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55571.log.gz 2024-04-23T14:45:40.025 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.72081.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72081.log.gz 2024-04-23T14:45:40.026 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53244.log 2024-04-23T14:45:40.027 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.67730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80522.log 2024-04-23T14:45:40.027 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67730.log.gz 2024-04-23T14:45:40.029 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.53244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46709.log 2024-04-23T14:45:40.029 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53244.log.gz 2024-04-23T14:45:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.80522.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80522.log.gz 2024-04-23T14:45:40.030 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66105.log 2024-04-23T14:45:40.031 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.46709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48677.log 2024-04-23T14:45:40.032 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46709.log.gz 2024-04-23T14:45:40.032 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.66105.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66105.log.gz 2024-04-23T14:45:40.033 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49158.log 2024-04-23T14:45:40.034 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.48677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44874.log 2024-04-23T14:45:40.034 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48677.log.gz 2024-04-23T14:45:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.49158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38952.log 2024-04-23T14:45:40.036 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49158.log.gz 2024-04-23T14:45:40.036 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.44874.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44874.log.gz 2024-04-23T14:45:40.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.38952.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38952.log.gz 2024-04-23T14:45:40.038 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47791.log 2024-04-23T14:45:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57352.log 2024-04-23T14:45:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.47791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58752.log 2024-04-23T14:45:40.041 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-04-23T14:45:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.57352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81287.log 2024-04-23T14:45:40.042 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57352.log.gz 2024-04-23T14:45:40.043 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.58752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79335.log 2024-04-23T14:45:40.043 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58752.log.gz 2024-04-23T14:45:40.044 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.81287.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81287.log.gz 2024-04-23T14:45:40.045 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5/var/log/ceph/ceph-client.admin.79335.log: --verbose -- /var/log/ceph/ceph-client.admin.51566.log 2024-04-23T14:45:40.046 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79335.log.gz 2024-04-23T14:45:40.046 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55968.log 2024-04-23T14:45:40.048 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.51566.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54504.log 2024-04-23T14:45:40.048 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51566.log.gz 2024-04-23T14:45:40.049 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.55968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65494.log 2024-04-23T14:45:40.049 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55968.log.gz 2024-04-23T14:45:40.051 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.54504.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54504.log.gz 2024-04-23T14:45:40.052 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.65494.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65494.log.gz 2024-04-23T14:45:40.070 INFO:teuthology.orchestra.run.smithi059.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-23T14:45:40.156 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-23T14:45:40.227 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-23T14:45:40.329 INFO:teuthology.orchestra.run.smithi059.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-23T14:45:40.515 INFO:teuthology.orchestra.run.smithi059.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-23T14:45:40.519 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-04-23T14:45:40.519 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m1.576s 2024-04-23T14:45:40.519 INFO:teuthology.orchestra.run.smithi059.stderr:user 0m5.156s 2024-04-23T14:45:40.519 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m2.427s 2024-04-23T14:45:40.520 INFO:tasks.ceph:Archiving logs... 2024-04-23T14:45:40.520 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/remote/smithi059/log 2024-04-23T14:45:40.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T14:45:41.801 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/remote/smithi076/log 2024-04-23T14:45:41.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T14:45:42.353 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T14:45:42.364 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T14:45:42.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T14:45:42.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T14:45:42.501 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T14:45:42.512 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T14:45:42.512 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:45:42.519 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:45:42.568 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-04-23T14:45:42.571 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-04-23T14:45:42.577 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T14:45:42.577 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-04-23T14:45:42.577 INFO:teuthology.orchestra.run.smithi059.stdout:^- hv01.front.sepia.ceph.com 2 6 377 32 +653us[ +643us] +/- 573ms 2024-04-23T14:45:42.577 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -649us[ -659us] +/- 54ms 2024-04-23T14:45:42.577 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +794us[ +784us] +/- 68ms 2024-04-23T14:45:42.577 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:45:42.580 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T14:45:42.581 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-04-23T14:45:42.581 INFO:teuthology.orchestra.run.smithi076.stdout:^- hv01.front.sepia.ceph.com 2 6 377 33 +665us[ +665us] +/- 573ms 2024-04-23T14:45:42.581 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -656us[ -663us] +/- 54ms 2024-04-23T14:45:42.581 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +802us[ +795us] +/- 68ms 2024-04-23T14:45:42.581 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:45:42.582 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T14:45:42.592 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T14:45:42.592 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T14:45:42.603 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T14:45:42.844 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T14:45:43.041 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T14:45:43.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:45:43.143 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T14:45:43.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:45:43.230 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T14:45:43.239 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T14:45:43.271 INFO:teuthology.task.internal:Duration was 1398.785022 seconds 2024-04-23T14:45:43.272 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T14:45:43.281 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T14:45:43.281 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T14:45:43.289 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T14:45:43.421 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T14:45:43.438 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T14:45:43.893 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T14:45:43.893 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-04-23T14:45:43.893 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T14:45:43.953 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-04-23T14:45:43.953 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T14:45:44.031 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T14:45:44.031 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T14:45:44.039 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T14:45:44.129 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T14:45:44.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T14:45:44.135 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T14:45:44.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T14:45:44.353 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T14:45:44.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T14:45:44.446 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T14:45:44.534 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T14:45:44.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:45:44.553 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:45:44.646 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-04-23T14:45:44.647 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-04-23T14:45:44.689 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:45:44.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:45:44.741 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T14:45:44.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:45:44.796 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T14:45:44.805 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T14:45:44.806 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/remote/smithi059 2024-04-23T14:45:44.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T14:45:44.910 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576/remote/smithi076 2024-04-23T14:45:44.910 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T14:45:45.006 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T14:45:45.007 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T14:45:45.015 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T14:45:45.072 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T14:45:45.082 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T14:45:45.082 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T14:45:45.093 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T14:45:45.093 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T14:45:45.098 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T14:45:45.150 INFO:teuthology.orchestra.run.smithi059.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 14:45 /home/ubuntu/cephtest 2024-04-23T14:45:45.152 INFO:teuthology.orchestra.run.smithi076.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 14:45 /home/ubuntu/cephtest 2024-04-23T14:45:45.155 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T14:45:45.165 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T14:45:45.187 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T14:45:45.216 DEBUG:teuthology.nuke:shortname: smithi059 2024-04-23T14:45:45.216 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T14:45:45.239 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:38.763255', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpnB0LGa4cS+W36Aj/as5sETZJ8Aj1oZaMwoiqsqf7kljO7nt46i/ouz7jwY3Pr3lLkNOX2bweR2pmXsxAiJoE='} 2024-04-23T14:45:45.243 DEBUG:teuthology.nuke:shortname: smithi076 2024-04-23T14:45:45.243 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T14:45:45.265 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:38.761876', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBByGZNsW/Qlm58n8wjoKYEesFXpNRdCXRv8uJXOaipMKNEMW9KHWFUXl4pdf8XasVMXR7jLR//U/s+yzg9lWBNI='} 2024-04-23T14:45:45.292 INFO:teuthology.orchestra.console.smithi059:Power off 2024-04-23T14:45:45.293 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T14:45:45.315 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-04-23T14:45:45.316 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:45:45.330 INFO:teuthology.orchestra.console.smithi076:Power off 2024-04-23T14:45:45.330 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T14:45:45.340 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-23T14:45:45.352 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2024-04-23T14:45:45.352 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:45:45.373 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-04-23T14:45:49.341 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:45:49.374 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:45:49.452 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-23T14:45:49.483 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-04-23T14:45:53.454 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:45:53.483 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:45:53.565 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-04-23T14:45:53.591 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2024-04-23T14:45:53.692 INFO:teuthology.orchestra.console.smithi076:Power off completed 2024-04-23T14:45:57.567 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T14:45:57.678 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-04-23T14:45:57.778 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-04-23T14:45:57.883 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1398.78502202034 failure_reason: '"2024-04-23T14:39:09.755546+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: false 2024-04-23T14:45:57.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T14:45:57.982 INFO:teuthology.run:FAIL